agentic-rl

AgentOPSD proposes a critic-free recursive method for turn-level credit assignment in agentic RL, reweighting outcomes via Bayesian belief updates in log-odds space. It achieves 89.1% success on ALFWorld with Qwen2.5-7B, outperforming GRPO and self-distillation baselines.

EnvACE introduces world rehearsal, an agentic reinforcement learning method that replaces external environment interaction by having the policy rehearse environment responses internally, achieving strong performance across multiple benchmarks.

This paper introduces SciDisco, a scalable framework for training scientific discovery agents via process-verifiable environments, DAG-grounded trajectory synthesis, and turn-level reinforcement learning. The proposed SciDisco-14B model achieves state-of-the-art on hypothesis-driven scientific data analysis benchmarks.

SkillRise is a unified reinforcement learning framework that enables LLM agents to learn and reuse skills across related, progressively challenging tasks, outperforming baselines by up to 8.5 percentage points on several benchmarks.

Molt is an open-source, PyTorch-native agentic RL framework from NVIDIA NeMo, designed for scalable, fully-async training of large multimodal agent models using Ray, vLLM, and NVIDIA AutoModel.

NVIDIA released Molt, a PyTorch-native agentic RL framework designed for compactness and readability, with performance comparable to Megatron-based stacks. The framework is open-source and includes a paper.

Molt is a PyTorch-native training framework for agentic reinforcement learning designed to be compact and clean for easy modification, while achieving performance comparable to Megatron-based stacks.

Google announces a major update to Tunix, its post-training library, with an asynchronous decoupled rollout engine to scale agentic reinforcement learning on JAX/TPU, eliminating idle time and improving throughput.

ToolVerse is a framework that automatically builds massive executable agent training environments from 422 real-world MCP environments containing 4438 tools, and proposes a task design strategy using Dynamic Unlocking Sampling Algorithm to generate long-horizon tasks, along with a Turn-Aware Relative Advantage algorithm for credit assignment in agentic reinforcement learning.

This paper investigates the use of the Muon optimizer in reinforcement learning post-training, finding that applying Muon to hidden weight matrices significantly improves success rates on ALFRED tasks compared to AdamW, with results dependent on the advantage estimator and learning rate.

This paper from Tsinghua University introduces Single-rollout Asynchronous Optimization (SAO) for reinforcement learning post-training of LLMs. SAO replaces batch-based GRPO with single-rollout asynchronous training to reduce idle GPU time and improve stability, and it was used to train the GLM-5.2 model (750B-A40B), achieving state-of-the-art results on agentic coding and reasoning benchmarks.

The paper introduces Single-rollout Asynchronous Optimization (SAO) to address stability and off-policy challenges in asynchronous RL for LLM post-training, and demonstrates that SAO consistently outperforms GRPO on agentic coding and reasoning benchmarks.

Tsinghua's SAO algorithm addresses stability and off-policy drift in asynchronous reinforcement learning for LLMs, achieving consistent improvements over GRPO on agentic coding and reasoning benchmarks, and is used to train the GLM-5.2 model.

This paper presents Single-rollout Asynchronous Optimization (SAO) to address stability and off-policy challenges in asynchronous RL for agentic tasks, outperforming GRPO and its variants on coding and reasoning benchmarks. SAO is deployed in the GLM-5.2 model's agentic RL pipeline.

This thread discusses modifications to GRPO for agentic RL, focusing on different levels of advantage normalization (prompt-level, task-level, environment-level) to handle higher reward variance in multi-task, multi-turn environments.

LiteResearcher is a scalable reinforcement learning training framework designed for deep research agents.

This article details the latest progress of GLM-5.2 in Agentic RL, including the introduction of slime infrastructure, shifting from GRPO to PPO for handling long trajectories, and an online anti-cheat mechanism; it also explores Qwen's research on verifier quality, proposing three dimensions of scalability, faithfulness, and robustness, and designs multiple verification strategies for different tasks to improve the reliability of reward signals.

Introduces TRIAGE, a role-typed credit assignment framework that improves agentic reinforcement learning by providing more nuanced credit assignment than standard GRPO methods, using a structured judge to classify action segments and assign process rewards based on semantic roles.

该文章由ROLL团队分享了在终端环境中进行Agentic RL训练时的实践经验，包括环境管理器设计、异步训练管线以及多种模式切换，并对比了RLVR与Agentic RL的本质区别。

A thread discussing one of the hardest aspects of agentic reinforcement learning: managing and scaling environments.