grpo

The paper proposes Privileged Self-Practice (PSP), a method that injects privileged information into prompts for multi-turn LLM agents, improving task completion rates on AppWorld and SWE-bench benchmarks compared to existing distillation techniques.

This paper tests Reinforcement Learning with Verifiable Rewards (RLVR) on small language models using retrieval-augmented generation, achieving a 3.8-fold gain without distillation and emphasizing the need for reward design tailored to small models.

WanPE is a 397B-parameter prompt enhancement model that improves cinematic planning in text-to-video generation, showing significant human preference boosts over raw prompts and competitive performance with commercial offerings.

The paper introduces CounterCredit, a training method for vision-language agents that ensures visual calls are both needed and used, leading to higher performance and fewer spurious calls on benchmarks.

This controlled study compares supervised fine-tuning and reinforcement learning methods for training tool-calling agents across different datasets and model scales, finding that SFT with LoRA is strongest in-distribution while RL shows slight advantages in cross-dataset transfer.

The paper introduces ActObs, a method that supervises both action and observation tokens in agent trajectories to improve reinforcement learning exploration, showing enhanced performance on benchmarks like Terminal-Bench2.0 and aider-polyglot.

This paper proposes Feedback-Enriched Environments (FEEs) to bootstrap self-evolving agents in long-horizon tasks by enriching feedback for reinforcement learning, showing improved stability and performance on benchmarks using Qwen3 models and RL algorithms.

Jerry Tworek discusses the technical insights and challenges in applying reinforcement learning to scale AI models like o1, highlighting the importance of simplicity and techniques such as multiple rollouts.

DataFlex-RL is an evaluation platform for RLVR data policies that finds uniform sampling performs comparably to adaptive methods across mathematics, logic, and science benchmarks.

This paper introduces Teacher-Gated On-Policy Distillation (TGOPD), a method that verifies teacher reliability at the prompt level to improve on-policy distillation, leading to better performance and increased GPU utilization in asynchronous setups.

DRACO is a reinforcement learning method that dynamically generates rubrics and redistributes trajectory scores to improve long-horizon agent performance without verifiers.

This guide details a fine-tuning recipe using Group Relative Policy Optimization (GRPO) with the TRL library to enhance the LFM2.5-350M model's structured output compliance, improving IFStruct benchmark performance from 22.6% to 29.7%.

The paper proposes Group Adaptive Clustering Policy Optimization (GAPO), a plug-in modification to GRPO methods that adapts the clipping boundary to rollout advantage, improving Pass@1 and Pass@k on math reasoning and coding benchmarks.

A research paper presents a method for training a smaller self-hosted LLM using separate GRPO experts merged via SLERP, which outperforms a larger baseline on enterprise tasks and serves half of platform traffic at lower cost.

A writeup that explains the mathematics of reinforcement learning, deriving algorithms such as Policy Gradient, PPO, and GRPO, and discusses their relevance in aligning large language models through techniques like RLHF and RLVR.

This paper proposes using Bayesian Truth Serum as a reward in reinforcement learning fine-tuning to mitigate sycophancy in large language models, showing improved accuracy and reduced answer-flip rates without labeled data.

The paper introduces Answerable Working Memory (AWM) and AWM-GRPO to enhance the quality of terminal working memory in long-document VQA agents, improving accuracy and reducing memory issues.

This paper investigates Evolution Strategies (ES) as a post-training paradigm for LLM reasoning, showing that ES provides broader reasoning coverage and better Pass@K performance than GRPO through sparse functional updates and population diversity.

The tweet recommends two resources for learning practical RL post-training techniques, focusing on GRPO++ tricks to enhance reinforcement learning in large language models.

An experiment training three from-scratch LLMs with the same GRPO recipe yielded inconsistent results, with GRPO degrading performance in some models, particularly the middle-sized one, and no clear relationship to scale.