Tag
ESCRAG-R1 is a unified framework that integrates retrieval-based psychological guidance into Group Relative Policy Optimization to enhance emotional support conversations, mitigating superficial splicing and achieving natural integration of professional guidance and empathy.
Proposes HiDiffTIR, a hierarchical difficulty-aware policy optimization framework for multi-turn tool-integrated reasoning in LLM agents, improving performance and tool invocation accuracy through fine-grained credit assignment.
The paper introduces ERPO, a method that moves regularization from the action-side to the input-side by controlling query distribution, addressing the stability-exploration dilemma in LLM policy optimization, and showing improvements on mathematical reasoning benchmarks.
MileGPO proposes a method for credit assignment in long-horizon LLM agents using milestone inference with local evidence, achieving state-of-the-art performance on ALFWorld and WebShop benchmarks.
SAPO is a low-memory and compute-efficient framework for agentic reinforcement learning that shares policy and value functions in a single autoregressive backbone, outperforming PPO and GRPO in experiments on ALFWorld and WebShop.
The paper proposes Reverse-Turn Policy Optimization (RTPO) to stabilize multi-turn agentic reinforcement learning training by organizing rollouts as reverse trees and performing turn-level updates in reverse order, showing improvements over baselines.
This paper proposes GUPO, a Gradient Uncertainty-aware Policy Optimization method that improves post-training of large language models by modeling gradient uncertainties using a Bayesian approach to handle conflicts among group gradients.
PlanPO is a reinforcement learning method that introduces coarse-to-fine advantage signals for multi-turn agentic LLMs, improving performance over GRPO by 27.2% on benchmarks like ALFWorld, WebShop, and SciWorld.
SA-MRPO introduces a saturation-aware advantage reweighting technique for multi-reward policy optimization in reinforcement learning, improving performance on mathematical reasoning, adaptive reasoning, and coding benchmarks by focusing optimization on under-optimized objectives.
Introduces I-SDPO, an instance-level adaptive self-distillation policy optimization method that switches between GRPO and privileged self-distillation based on whether a rollout group contains any successful responses, improving reasoning performance on SciKnowEval.
This paper introduces GCPO, a method to diagnose and constrain subspace geometry in rollout-based RL for LLMs, improving stability and performance over GRPO and variants on reasoning, coding, and tool-use tasks.
This paper introduces Perturbed Parameter Policy Optimization (3PO), a family of parameter-space exploration methods for LLM reinforcement learning, showing consistent improvements over GRPO on math and code tasks.
A new research paper reframes LLM agent training as a cross-episode Meta-RL problem, using critic-free policy gradients to enable in-context adaptation without gradient updates. The LAMER framework improves test-time performance by 11-19% over standard RL baselines on long-horizon tasks and generalizes better to unseen environments.
This paper formalises counterfactual policy optimisation for Markov Decision Processes under probabilistic nondeterministic causal models, which separate latent confounding from inherent stochasticity, and proposes a practical optimisation procedure for deriving robust counterfactual policies. The approach is validated on a sepsis treatment simulator with diabetes as an unobserved global confounder.
The paper proposes OM-GRPO, a label-free RLVR framework that decouples reward estimation from policy optimization by masking gradients on answer spans, and introduces Contrast-Augmented Reward to refine reward estimation without extra rollouts. It consistently outperforms existing label-free methods and matches supervised ground-truth reward training across reasoning benchmarks.
Introduces Q-Steer, a rollout-time action-value guidance for molecular policy optimization that uses a frozen prefix-action value scorer to improve sampling without changing the online oracle budget.
This paper introduces β-OPSD, a generalization of on-policy self-distillation that frames it as a policy optimization family with a controllable KL penalty. The method uses distillation to approximate expensive policy optimization, improving stability and reasoning performance on mathematical benchmarks.
Proposes Progress-conditioned Group Policy Optimization (ProGPO) to overcome credit assignment issues in long-horizon agentic tasks by using first-visit observation coverage when all group samples fail, improving performance on ALFWorld and WebShop.
This paper identifies a reward-variance collapse failure mode in GRPO for multi-turn evidence-reading agents and proposes CIGPO, which uses per-turn contextual information-gain rewards to maintain gradient signal, achieving +105% F1 improvement on HotpotQA.
This paper proposes Group Entropy-Controlled Policy Optimization (GEPO), a lightweight extension to GRPO that uses group entropy to perform entropy-conditioned asymmetric advantage shaping, addressing heterogeneous entropy regimes across tasks during RL-based alignment of LLMs. Experiments show consistent improvements over GRPO and recent entropy-controlled methods across multiple benchmarks.