Skill-RM: Unifying Heterogeneous Evaluation Criteria via Agent Skill

Hugging Face Daily Papers Papers
reward-modeling llm reinforcement-learning agentic-task evidence-aggregation evaluation-framework

Summary

Skill-RM proposes a unified reward modeling framework that treats reward computation as a structured agentic task, enabling dynamic evidence aggregation and consistent evaluation across diverse applications, outperforming traditional judge baselines.

Reward models (RMs) provide critical feedback signals for LLM post-training, notably in reinforced fine-tuning (RFT) and reinforcement learning (RL) pipelines. However, current reward evaluation relies on heterogeneous criteria such as rule-based verifiers, ground-truth references, procedural checklists, and complex rubrics, where a unified mechanism to integrate all types of evidence remains unexplored. To this end, we propose Skill Reward Model (Skill-RM), a unified framework that reformulates reward modeling as the execution of a reusable Reward-Evaluation Skill. By treating reward computation as a structured agentic task, Skill-RM provides a consistent interface to orchestrate heterogeneous resources, dynamically selecting and aggregating evidence tailored to the specific requirements of each input. This approach enables the reward model to move beyond static evaluation, ensuring consistency and transparency across diverse tasks. Extensive experiments on reward benchmarks and downstream applications, including best-of-N selection and reinforcement learning, demonstrate that Skill-RM consistently outperforms traditional judge baselines. Our findings suggest that Skill-RM not only provides a unified solution for reward modeling but also achieves superior performance through the strategic and dynamic orchestration of evidence. The code is at https://github.com/Qwen-Applications/Skill-RM.
Original Article
View Cached Full Text

Cached at: 06/09/26, 08:41 AM

Paper page - Skill-RM: Unifying Heterogeneous Evaluation Criteria via Agent Skill

Source: https://huggingface.co/papers/2606.03980 Authors:

,

,

,

,

,

,

,

,

,

,

,

Abstract

Skill-RM presents a unified reward modeling framework that treats reward computation as a structured agentic task, enabling dynamic evidence aggregation and consistent evaluation across diverse applications.

Reward models(RMs) provide critical feedback signals for LLM post-training, notably inreinforced fine-tuning(RFT) andreinforcement learning(RL) pipelines. However, current reward evaluation relies onheterogeneous criteriasuch as rule-based verifiers, ground-truth references, procedural checklists, and complex rubrics, where a unified mechanism to integrate all types of evidence remains unexplored. To this end, we propose Skill Reward Model (Skill-RM), a unified framework that reformulatesreward modelingas the execution of a reusable Reward-Evaluation Skill. By treating reward computation as astructured agentic task, Skill-RM provides a consistent interface to orchestrate heterogeneous resources, dynamically selecting and aggregating evidence tailored to the specific requirements of each input. This approach enables the reward model to move beyond static evaluation, ensuring consistency and transparency across diverse tasks. Extensive experiments on reward benchmarks and downstream applications, including best-of-N selection andreinforcement learning, demonstrate that Skill-RM consistently outperforms traditional judge baselines. Our findings suggest that Skill-RM not only provides a unified solution forreward modelingbut also achieves superior performance through the strategic and dynamic orchestration of evidence. The code is at https://github.com/Qwen-Applications/Skill-RM.

View arXiv pageView PDFGitHub4Add to collection

Get this paper in your agent:

hf papers read 2606\.03980

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2606.03980 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2606.03980 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2606.03980 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

SkillJuror: Measuring How Agent Skill Organization Changes Runtime Behavior

arXiv cs.AI

This paper presents SkillJuror, a framework for evaluating how different organizational paradigms for Agent Skills affect LLM agent runtime behavior. A study on 82 tasks shows that Progressive Disclosure increases resource diversity and uptake events compared to a flat baseline, with task-dependent outcome improvements.