TRIAGE: Role-Typed Credit Assignment for Agentic Reinforcement Learning
Summary
Introduces TRIAGE, a role-typed credit assignment framework that improves agentic reinforcement learning by providing more nuanced credit assignment than standard GRPO methods, using a structured judge to classify action segments and assign process rewards based on semantic roles.
View Cached Full Text
Cached at: 07/01/26, 07:44 PM
Paper page - TRIAGE: Role-Typed Credit Assignment for Agentic Reinforcement Learning
Source: https://huggingface.co/papers/2606.32017
Abstract
TRIAGE introduces a role-typed credit assignment framework that enhances agentic reinforcement learning by providing more nuanced credit assignment than standard GRPO methods.
Agentic reinforcement learningrequires assigning credit to environment-facing actions such as searches, clicks, edits, navigation commands, and object interactions. StandardGRPOuses the finalverifier outcomeas a uniform advantage over all action tokens. This outcome signal is useful but structurally incomplete: it punishes useful exploration in failed rollouts and reinforces redundant or regressive actions in successful rollouts. We propose TRIAGE, a role-typedcredit assignmentframework that adds asemantic role axisto outcome credit. Astructured judgeclassifies each segment as decisive progress, useful exploration, no-progress infrastructure, or regression, and a fixedrole-conditioned rulemaps these labels to bounded segment-level process rewards. This keepsverifier outcomes as the source of optimization direction while correcting the two main blind spots of outcome-only credit. We further show that role-conditioned credit is the optimal segment-level correction expressible from role labels alone -- a projection of the per-segment advantage residual onto the role variable -- so that the fixed role constants reduceadvantage estimationerror whenever the judge is reliable, and we connect this to lower-variancepolicy gradients. AcrossALFWorld,Search-QA, andWebShop, TRIAGE improves success rates overGRPOfor two policy models and outperforms both a scalar judge-derived process reward and an outcome-supervised shared-backbone value baseline. Ablations show that the gain comes from role typing rather than merely adding dense rewards: reliable detection of regression inside successful trajectories is the dominant contributor, while exploration credit provides a consistent secondary gain; on completedALFWorldandWebShoprollouts, TRIAGE also reduces environment-facing turns by an additional 10.4% and 14.8% relative toGRPO.
View arXiv pageView PDFAdd to collection
Get this paper in your agent:
hf papers read 2606\.32017
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2606.32017 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2606.32017 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2606.32017 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
@SharonYixuanLi: Scaling outcome-based RL won't solve long-horizon agentic tasks. Credit assignment is the bottleneck, and turn-level re…
TRACE introduces a turn-level reward assignment method using frozen reference model log-probabilities and temporal-difference learning to address credit assignment in long-horizon agentic tasks, achieving significant improvements in search benchmarks without critic or process labels.
TRACE: A Unified Rollout Budget Allocation Framework for Efficient Agentic Reinforcement Learning
TRACE is a unified rollout budget allocation framework that enhances reward contrast in multi-turn agentic reinforcement learning by dynamically distributing resources across tree-structured rollouts based on prefix-level informativeness. It improves efficiency and accuracy on agentic benchmarks like Multi-Hop QA.
Self-Credit: Credit Assignment for Long-Horizon Agentic RL via Intra-group Rescaling
The paper proposes Self-Credit, an agentic RL method that densifies sparse RLVR rewards via self-generated credit assignments and intra-group rescaling, achieving significant gains on BrowseComp-Plus, GAIA, and xBench-DeepSearch benchmarks with Qwen3 models.
@omarsar0: Good paper on credit assignment for agent RL. The main finding is that you want an LLM judge to choose where to check a…
The ProVer paper targets pivotal decisions for credit assignment in agentic RL: an LLM judge compares successful and failed rollouts to propose a key trajectory segment, then rollouts sampled before and after that segment estimate its advantage. It improves over GRPO by 9.91% and 7.12% on Qwen3.5-2B and Qwen3.5-4B across ALFWorld, WebShop, and SearchQA.
StraTA: Incentivizing Agentic Reinforcement Learning with Strategic Trajectory Abstraction
StraTA proposes strategic trajectory abstraction for long-horizon LLM agents, using hierarchical GRPO-style rollout with diverse strategy sampling and critical self-judgment to improve sample efficiency and final performance over frontier models and prior RL baselines.