rlvr

This paper introduces PAIR, a pairwise-aware inclusion reweighting method for adaptive rollout allocation in RLVR, improving sample efficiency and accuracy over pointwise allocators by correcting biases in pairwise gradient estimation.

Simon Willison analyzes the timeline of OpenAI's accidental attack on Hugging Face, suggesting that RLVR training of a new model explains the lack of safety behaviors and lax monitoring.

Explains why frontier AI models often behave rudely or disobediently, citing former Meta engineer Kun Chen on RLHF and RLVR training that optimizes for task success over human-friendly communication.

This paper introduces RSTG, a method that selectively applies on-policy distillation to recover learning signals from zero-variance GRPO groups during LLM post-training, achieving substantial gains on math and code reasoning benchmarks.

The paper introduces verifier-induced support reshaping, showing that on-policy RL with verifiable rewards can improve the current objective while making successful behaviors for later objectives too rare to sample. Experiments across math reasoning and instruction following demonstrate that endpoint improvements do not guarantee future trainability.

Introduces RLSVR, a task-transformation paradigm that extends RLVR to open-ended tasks via self-verifiable rewards from self-play games, instantiated in SpyRL and Vision-Zero. It improves LLM performance on summarization, creative writing, and math reasoning.

This paper proposes LSPO (LoRA Scaffolded Policy Optimization) to solve the vanishing-gradient problem of GRPO on zero-reward "cliff problems": a temporary LoRA adapter is used for a short SFT, successful trajectories are sampled, corrected via importance sampling, and then fed back into the RL batch; ultimately, only the base model is updated. Experiments on DeepMath-103K show it outperforms the DAPO baseline by an average of 3.8 points.

This paper introduces LEEPS, a latent-guided explore-exploit prompt sampler for efficient reinforcement learning with verifiable rewards (RLVR) in LLMs. It adaptively balances reuse of informative prompts and exploration of uncertain ones, improving reasoning benchmark scores by 2.6-3.7% over baselines while adding only ~2 seconds of overhead per training step.

SAF-OPD introduces a Stable Advantage Fusion framework that combines RLVR with on-policy distillation, addressing magnitude and temporal mismatches to improve training stability and performance on math and code benchmarks.

Investigates the importance of reasoning traces in reinforcement learning for neural machine translation, showing that including reasoning during inference improves quality at the cost of increased computational demands.

This paper studies the optimization layer for reinforcement learning with verifiable rewards (RLVR), proposing Isospectral Optimization (ISO) — a fixed-spectrum framework that reuses base model weight spectra while optimizing input/output singular frames. ISO-Merger and ISO-AdamW achieve strong performance with fewer training steps on reasoning and coding tasks.

Introduces H^2SD, a hybrid hindsight self-distillation framework that improves RLVR by using the teacher model differently for successful and failed trajectories, achieving better reasoning performance.

Researchers from Bridgewater AIA Labs, UIUC, and MIT prove the first non-vacuous generalization bounds for reasoning LLMs trained with RLVR, providing provable accuracy lower bounds on unseen data to guide safe deployment.

This paper introduces Contrastive Policy Optimization (CPO), which uses token-level contrastive disagreement between reference-guided and vanilla generation distributions for correctness-aware advantage shaping in reinforcement learning with verifiable rewards. CPO outperforms entropy-based RLVR methods on both in-domain and out-of-domain benchmarks.

This paper introduces RLVP (Reward the Outcome, Penalize the Path), a reinforcement learning method that uses a verifiable penalty for path violations and outcome reward to achieve near-zero constraint violations with high task success, improving sample efficiency in real-world agentic environments.

This paper proposes a reinforcement learning framework for evidence-seeking diagnostic reasoning using LLMs. The RL-trained 7B model outperforms larger models in multilingual clinical consultation tasks, showing that specialized RL can distill high-level clinical reasoning.

This paper from MIT proposes an adversarial generator-discriminator framework that combines verifiable rewards with a learned signal from human demonstrations to address issues like diversity collapse, unnatural responses, and reward hacking in RLVR training of language models.

This article explores the challenge of applying reinforcement learning to tasks that lack clear verifiability, citing Dario Amodei's prediction about achieving a 'country of geniuses in a data center' and discussing techniques such as RLVR, RLHF, Constitutional AI, and rubric-based rewards from Scale AI.

An analysis discussing the limitations of reinforcement learning with verifiable rewards (RLVR) in math and coding, and the challenge of extending RL to subjective or unverifiable tasks like planning or scientific discovery. It explores techniques such as RLHF and Constitutional AI as alternatives for alignment.

This paper proposes Transfer-Aware Curriculum (TAC), a bandit-style online curriculum for multi-domain RLVR that prioritizes domains whose updates benefit other domains using gradient-geometry alignment. TAC improves macro-averaged accuracy on Qwen3-1.7B and Llama3.2-3B over fixed and learnability-only curricula.