Tag
Investigates the importance of reasoning traces in reinforcement learning for neural machine translation, showing that including reasoning during inference improves quality at the cost of increased computational demands.
This paper studies the optimization layer for reinforcement learning with verifiable rewards (RLVR), proposing Isospectral Optimization (ISO) — a fixed-spectrum framework that reuses base model weight spectra while optimizing input/output singular frames. ISO-Merger and ISO-AdamW achieve strong performance with fewer training steps on reasoning and coding tasks.
Introduces H^2SD, a hybrid hindsight self-distillation framework that improves RLVR by using the teacher model differently for successful and failed trajectories, achieving better reasoning performance.
Researchers from Bridgewater AIA Labs, UIUC, and MIT prove the first non-vacuous generalization bounds for reasoning LLMs trained with RLVR, providing provable accuracy lower bounds on unseen data to guide safe deployment.
This paper introduces Contrastive Policy Optimization (CPO), which uses token-level contrastive disagreement between reference-guided and vanilla generation distributions for correctness-aware advantage shaping in reinforcement learning with verifiable rewards. CPO outperforms entropy-based RLVR methods on both in-domain and out-of-domain benchmarks.
This paper introduces RLVP (Reward the Outcome, Penalize the Path), a reinforcement learning method that uses a verifiable penalty for path violations and outcome reward to achieve near-zero constraint violations with high task success, improving sample efficiency in real-world agentic environments.
This paper proposes a reinforcement learning framework for evidence-seeking diagnostic reasoning using LLMs. The RL-trained 7B model outperforms larger models in multilingual clinical consultation tasks, showing that specialized RL can distill high-level clinical reasoning.
This paper from MIT proposes an adversarial generator-discriminator framework that combines verifiable rewards with a learned signal from human demonstrations to address issues like diversity collapse, unnatural responses, and reward hacking in RLVR training of language models.
This article explores the challenge of applying reinforcement learning to tasks that lack clear verifiability, citing Dario Amodei's prediction about achieving a 'country of geniuses in a data center' and discussing techniques such as RLVR, RLHF, Constitutional AI, and rubric-based rewards from Scale AI.
An analysis discussing the limitations of reinforcement learning with verifiable rewards (RLVR) in math and coding, and the challenge of extending RL to subjective or unverifiable tasks like planning or scientific discovery. It explores techniques such as RLHF and Constitutional AI as alternatives for alignment.
This paper proposes Transfer-Aware Curriculum (TAC), a bandit-style online curriculum for multi-domain RLVR that prioritizes domains whose updates benefit other domains using gradient-geometry alignment. TAC improves macro-averaged accuracy on Qwen3-1.7B and Llama3.2-3B over fixed and learnability-only curricula.
A discussion on the next training paradigm for AI, covering research bets, grindability, RLVR, and a vision for 2027.
This paper introduces 'pigeonholing,' a phenomenon where bad prompts cause LLMs to collapse and repeat errors, leading to a 38-40% performance drop. Experiments across 10 tasks and 10 models show worsening with more conversation turns, and propose RLVR with synthetic errors as a mitigation.
GraphPO is a novel graph-based reinforcement learning framework that represents rollouts as a directed acyclic graph, merging semantically equivalent reasoning paths to reduce redundant exploration and improve credit assignment for large reasoning models.
This paper investigates the 'sparsity curse' in merging RLVR (Reinforcement Learning with Verifiable Reward) models, finding that sparse updates cause near-orthogonal parameter directions that hinder aggregation, and proposes SAR-Merging, which uses Fisher information and sparsification to resolve conflicts and improve merging performance on math and coding tasks.
This paper reveals that the clipping mechanism in PPO and GRPO biases entropy in RLVR for LLMs: clip-low increases entropy, clip-high decreases it. The authors prove that standard clipping reduces entropy even with random rewards, and show that adjusting clip-low can prevent entropy collapse and promote exploration.
An open-source LLM called OpenMythos was trained for cybersecurity tasks using SFT and RLVR, with datasets available on HuggingFace. The model aims to reduce hallucinations and improve precision in security-related queries.
This paper analyzes the thinking-answer inconsistency in multimodal reinforcement learning with verifiable rewards (RLVR) for large vision-language models and proposes CORA, a method that introduces a consistency reward model and hybrid reward advantage splitting to improve faithfulness and task performance.
ProcessThinker introduces a practical post-training pipeline that provides step-level process rewards without training an explicit process reward model. It uses rollout-based rewards to give dense credit assignment for multi-step reasoning in multimodal LLMs, consistently improving performance on video benchmarks.
A recommendation of a post-training article on GRPO/RLVR that was overlooked earlier this year, aimed at those interested in reinforcement learning from verifiable rewards.