Tag
This thread discusses modifications to GRPO for agentic RL, focusing on different levels of advantage normalization (prompt-level, task-level, environment-level) to handle higher reward variance in multi-task, multi-turn environments.
LiteResearcher is a scalable reinforcement learning training framework designed for deep research agents.
This article details the latest progress of GLM-5.2 in Agentic RL, including the introduction of slime infrastructure, shifting from GRPO to PPO for handling long trajectories, and an online anti-cheat mechanism; it also explores Qwen's research on verifier quality, proposing three dimensions of scalability, faithfulness, and robustness, and designs multiple verification strategies for different tasks to improve the reliability of reward signals.
Introduces TRIAGE, a role-typed credit assignment framework that improves agentic reinforcement learning by providing more nuanced credit assignment than standard GRPO methods, using a structured judge to classify action segments and assign process rewards based on semantic roles.
该文章由ROLL团队分享了在终端环境中进行Agentic RL训练时的实践经验，包括环境管理器设计、异步训练管线以及多种模式切换，并对比了RLVR与Agentic RL的本质区别。
A thread discussing one of the hardest aspects of agentic reinforcement learning: managing and scaling environments.
OPID is a framework that extracts dense token-level supervision from completed on-policy trajectories for reinforcement learning of language agents, using hierarchical skills (episode-level and step-level) to improve sample efficiency and robustness.
This paper investigates why multi-step tool-use reinforcement learning (RL) often collapses or yields limited gains, identifying probability spikes in control tokens as a key cause. It shows that interleaving supervised fine-tuning with RL improves stability and explores various supervisory signals to guide robust training.
A blog post summarizing ten recent agentic RL frameworks and best practices, covering modular interfaces, trajectory structure, action masks, process rewards, advantage normalization, scalable rollouts, stability/exploration, and task curriculum.
Discussion of recent agentic RL papers, highlighting action masking as a common technique and its evolution with world modeling papers like ECHO and PaW.
APPO improves multi-turn tool-use in LLM agents by refining branching decisions and credit assignment using fine-grained decision points and procedure-level advantage scaling, outperforming baselines by 4 points on 13 benchmarks.
TRACE is a unified rollout budget allocation framework that enhances reward contrast in multi-turn agentic reinforcement learning by dynamically distributing resources across tree-structured rollouts based on prefix-level informativeness. It improves efficiency and accuracy on agentic benchmarks like Multi-Hop QA.
OpenEnv, a library for creating agentic execution environments to train open source agents with reinforcement learning, is becoming more open with a new governance committee including Meta-PyTorch, Hugging Face, Nvidia, and others, aiming to provide a protocol layer that works across models and harnesses.
Sergio Paniego highlights that frontier agents' performance is due to models being trained inside their deployment harness. The new work 'Polar: Agentic RL on Any Harness at Scale' by NVIDIA AI enables turning harnesses like Codex, Claude Code, Qwen Code, or Pi into RL training environments without modifying their internals.
StepPO introduces a step-centric paradigm for agentic reinforcement learning that aligns policy optimization with agent decision granularity, outperforming token-centric methods in multi-turn interaction tasks.
This article explains the 'Token-In, Token-Out' (TITO) invariant in reinforcement learning for LLMs, highlighting a common error when training multi-turn agents with tool calls. It presents two solutions: using per-model renderers or designing training to avoid re-encoding decoded tokens, emphasizing prefix-preserving chat templates.
Sanbu 散步 released a modern RL tutorial Hands-On Modern RL, covering from CartPole+PPO basics to LLM post-training (RLHF, DPO, GRPO) and Agentic RL, code-first, English version coming soon.
Skill0.5 is a novel agentic reinforcement learning framework that combines general skill internalization with task-specific skill utilization via a dynamic difficulty-aware router, improving out-of-distribution generalization in complex task environments as demonstrated on ALFWorld and WebShop.
NVIDIA releases Polar, an open-source infrastructure for black-box agentic reinforcement learning, enabling training of coding agents like Claude Code or Codex with any agent harness or framework.
This paper proposes AKBE, an on-policy method for LLM agent reinforcement learning that dynamically identifies when tool use is needed versus when internal knowledge suffices, improving accuracy by +1.85 on average and reducing tool calls by 18% over standard agentic RL.