grpo

PlanPO is a reinforcement learning method that introduces coarse-to-fine advantage signals for multi-turn agentic LLMs, improving performance over GRPO by 27.2% on benchmarks like ALFWorld, WebShop, and SciWorld.

TAMP-Nav is a unified framework that enhances embodied navigation by aligning vision-language models with 2D visual prompting, selective reasoning with compressed memory, and policy optimization, achieving state-of-the-art performance with high runtime and sample efficiency.

This large-scale study investigates GRPO in non-English and multilingual contexts, finding that training to reason in native languages has a small gap to English training and reveals strong crosslingual transfer, though effects are model- and language-dependent.

This paper introduces a unified black-box reinforcement learning framework for stable and scalable optimization of agents through complex harnesses, using sandbox execution and trajectory reconstruction with improvements on benchmarks like ClawGym-Bench.

Introduces I-SDPO, an instance-level adaptive self-distillation policy optimization method that switches between GRPO and privileged self-distillation based on whether a rollout group contains any successful responses, improving reasoning performance on SciKnowEval.

Introduces SSPO, a step-level self-distilled policy optimization method for training deep search agents, which uses evidence anchors and advantage weights to improve credit assignment beyond sparse outcome rewards. SSPO outperforms GRPO on benchmarks like BrowseComp and GAIA with only ~5% overhead per step.

This paper introduces GCPO, a method to diagnose and constrain subspace geometry in rollout-based RL for LLMs, improving stability and performance over GRPO and variants on reasoning, coding, and tool-use tasks.

A Scale AI research paper proposes Rubric Dropout, a dropout-style regularization for rubric criteria in rubric-as-reward RL, showing it mitigates reward hacking and improves out-of-distribution gold judge scores on medical and science benchmarks.

This paper presents a method using Group Relative Policy Optimization (GRPO) to fine-tune an open-weight language model for generating actionable financial advice, outperforming commercial LLMs under a judge-independent CATE evaluation while also matching safety criteria.

This paper introduces ConRub-Med, a reinforcement learning approach that uses consensus rubrics from multiple language models to reward open-ended medical question answering, achieving state-of-the-art results on several benchmarks including HealthBench-Hard.

This paper from Xiaomi introduces reference-free post-training for multilingual machine translation, applying GRPO with quality-estimation rewards to the MiLMMT-46-v0.1 SFT models, producing MiLMMT-46-v1.0 that improves translation across 46 languages and outperforms open and proprietary baselines.

This paper introduces SinkFlex-RL, a modular training system for memory-feasible reinforcement learning in long-horizon tool-use agentic tasks. It combines a Gymnasium-compatible environment wrapper, GRPO-based policy optimization, and a sink-aware FlexAttention path, reducing peak VRAM by 19.7% at 4096 tokens and enabling 8192-token runs where eager attention runs out of memory.

GraphThink is a framework that integrates task graphs and scene graphs to enhance LLM-based planning for long-horizon embodied tasks, achieving state-of-the-art results on the ALFRED benchmark and improving generalization and closed-loop replanning.

This paper introduces MiLMMT-46-v1.0, a multilingual machine translation model improved via reference-free post-training with GRPO and checkpoint interpolation, surpassing strong open and proprietary baselines across 46 languages.

This paper introduces Perturbed Parameter Policy Optimization (3PO), a family of parameter-space exploration methods for LLM reinforcement learning, showing consistent improvements over GRPO on math and code tasks.

Introduces GRASP, a method that uses Group Relative Policy Optimization to train a small on-device language model for adversarial anonymization, improving the privacy-utility trade-off over DPO-distilled baselines while running at a fraction of the cost of frontier teacher models.

This paper introduces LC-GRPO, a flow-based GRPO framework with Langevin correction that bridges the train-inference gap by aligning stochastic training rollouts with deterministic ODE sampling, improving reward optimization on models like SD3.5, FLUX.1-Dev, and HunyuanVideo.

Harness-R1 introduces a method that learns to edit executable runtime harnesses for LLM agents from failure trajectories, using a separate 9B engineer optimized with online GRPO while keeping the target model frozen. It improves success rates across WebShop, ALFWorld, and DBBench.

This paper introduces RSTG, a method that selectively applies on-policy distillation to recover learning signals from zero-variance GRPO groups during LLM post-training, achieving substantial gains on math and code reasoning benchmarks.

SERL-SQL proposes a selective execution-grounded reinforcement learning framework for multi-turn Text-to-SQL agents, using teacher-student likelihood gaps to reweight GRPO advantages on SQL action tokens. It achieves strong results on BIRD and Spider benchmarks.