grpo

Introduces SVR-R1, a multi-turn reinforcement learning framework that uses the model's own verification as a learning signal for multi-modal reasoning, achieving significant accuracy improvements over standard GRPO baselines on vision-language reasoning benchmarks.

Andrej Karpathy frames LLM training as text, conversations, and environments; Prime Intellect's Verifiers is an open-source framework for building and sharing RL environments for LLMs, released under MIT license, with a hub of 2500+ environments.

The tweet lists 15 LLM fine-tuning techniques and introduces ART (Agent Reinforcement Trainer), an open-source framework from OpenPipe for training multi-step agents using GRPO, with serverless RL support via W&B Training.

This paper introduces a learnable control module trained via Group Relative Policy Optimization (GRPO) to optimize the generation order in multimodal masked diffusion models, achieving improvements in text-to-image alignment and multimodal understanding.

This paper proposes using Group Relative Policy Optimization (GRPO) for adapting LLM-based ASR models to regulated domains using only synthetic speech, achieving 40-45% relative WER reduction over supervised fine-tuning.

Proposes Tail-Aware Credit Calibration (TACO) to address positive-credit contamination in LLM reinforcement learning by calibrating uniform credit assignment to suppress updates on implausible tail tokens, improving training stability and performance.

This paper introduces AdaPrefix-GRPO, a method that adaptively controls the length of correct solution prefixes provided to a model during GRPO training, maintaining a 50% success rate to maximize gradient signal. It significantly improves accuracy on hard math reasoning problems while reducing computational cost.

DrugGen-2 fine-tunes GPT-2 using supervised learning and reinforcement learning (GRPO) to generate small molecules conditioned on both disease ontology and target protein sequences, achieving superior diversity and binding affinity for drug discovery.

Introduces Self-Review Reinforcement Learning (SRRL), a training framework that embeds a self-review step into RL episodes to transform sparse environmental feedback into behavioral improvements, using cross-episode memory and selective policy distillation. Evaluated on GSM8K, SRRL outperforms standard RLVR with GRPO across Qwen 3-4B and OLMo-3-7B models.

This paper presents Single-rollout Asynchronous Optimization (SAO) to address stability and off-policy challenges in asynchronous RL for agentic tasks, outperforming GRPO and its variants on coding and reasoning benchmarks. SAO is deployed in the GLM-5.2 model's agentic RL pipeline.

This thread discusses modifications to GRPO for agentic RL, focusing on different levels of advantage normalization (prompt-level, task-level, environment-level) to handle higher reward variance in multi-task, multi-turn environments.

NormWorlds-CF is a solver-verified benchmark for counterfactual normative reasoning. The paper proposes MR-GRPO, a reward mechanism that improves structured reasoning beyond final answers, showing that answer-only accuracy can be misleading in normative tasks.

A technical tutorial on building a reinforcement learning environment for LLMs using the open-source Verifiers library, with Othello as a working example.

TREK is a staged procedure that uses distillation to expand exploration support for policy optimization, improving performance on mathematical reasoning and agentic tasks beyond standard GRPO.

This paper introduces Mastermind, a dual-loop framework that learns reusable vulnerability-reproduction strategies for repository-scale tasks, achieving an 84.5% pass rate with a frozen executor by separating strategy learning from execution.

This paper presents a proof-of-concept using Reinforcement Learning with Verifiable Rewards (RLVR) to train small language models for tool-use in enterprise SaaS workflows like Jira and Confluence. The approach uses synthetic environments and GRPO training to improve tool-call accuracy, achieving significant reward gains over baselines.

This paper proves that GRPO, Dr. GRPO, and DAPO are three formulations of the same underlying mechanism: adjusting the standard deviation of rewards within a group of sampled answers. The group-standard-deviation identity shows that unanimous groups teach nothing while split groups drive learning, revealing a unified dial for training language models to reason.

This paper introduces Autodata, a method that uses an agentic 'data scientist' AI to automate the creation of high-quality synthetic datasets through iterative generation, verification, and refinement, specifically optimized for reinforcement learning (GRPO) to improve reasoning in language models.

Presents a closed-form reduced-order model of GRPO training dynamics, reducing it to a damped oscillator and deriving predictions for stability, group-size invariance, and loss curvature. Validated across multiple models and benchmarks.

PASS is a middleware that fixes three pathologies in process-supervised RL for LLM reasoners, improving GRPO by independently standardizing streams, chunking by value, and using average value density. It shows consistent gains in math reasoning and multi-hop QA.