A new reinforcement learning approach addresses the challenge of reward model reliability in open-ended tasks. Existing solutions rely on proprietary LLM-as-a-Judge systems or opaque scalar reward models. Recent generative reward models are limited by static evaluation criteria and lack multilingual support. The work introduces MixReward, a large-scale multilingual dataset spanning six domains and 103 languages, containing both pairwise and listwise data. UniRRM, a unified reasoning reward model, supports multiple languages and evaluation paradigms. UniRRM uses a staged reasoning chain to dynamically generate task-generic and instruction-specific criteria, enabling input-adaptive judgments. Experiments show UniRRM-8B and UniRRM-14B achieve performance close to the state-of-the-art across multiple benchmarks, and are effective for unseen evaluation paradigms. Ablation studies validate the reliability and effectiveness of UniRRM.
Source: https://arxiv.org/abs/2609.05910