REVES: REvision and VErification--Augmented Training for Test-Time Scaling
Summary
Proposes REVES, a two-stage iterative framework that alternates between data augmentation and policy optimization to improve LLM reasoning by leveraging intermediate correction steps, achieving superior performance on coding benchmarks and constraint satisfaction problems.
View Cached Full Text
Cached at: 06/18/26, 11:59 PM
Paper page - REVES: REvision and VErification–Augmented Training for Test-Time Scaling
Source: https://huggingface.co/papers/2606.18910
Abstract
A two-stage iterative framework alternates between data augmentation and policy optimization to improve LLM reasoning by leveraging intermediate correction steps, achieving superior performance on coding benchmarks and constraint satisfaction problems.
Test-time scalingviasequential revisionhas emerged as a powerful paradigm for enhancingLarge Language Model(LLM) reasoning. However, standard post-training methods primarily optimize single-shot objectives, creating a fundamental misalignment withmulti-step inference dynamics. While recent work treats this as multi-turnreinforcement learning(RL), conventional approaches optimize over the multi-step trajectories directly, failing to further exploit the high-quality mistakes inintermediate stepsthat model can learn from correcting them. We propose a two-stage iterative framework that alternates between online data/prompt augmentationandpolicy optimization. By converting theintermediate steps(``near-miss’’ answers) in the successful recovery trajectories into decoupled revision and verification prompts, our approach concentrates training on both effectiveanswer transformationanderror identification. This approach enables efficientoff-policy data generationand reduces the computational overhead of long-horizon sampling compared to standard multi-turn RL. On LiveCodeBench, using publicly available test cases as feedback, we observe gains of +6.5 points over the RL baseline and +4.0 points over standard multi-turn training. Beyond coding, our approach matches the previously reported SOTA result on circle packing while using the smallest base model (4B) and far fewer rollouts than the much larger evolutionary search systems. Math results under ground-truth verification further confirm improved correction ability. It also generalizes to out-of-distribution constraint-satisfaction puzzles such as n\_queens and mini\_sudoku, where correctness is defined entirely by problem constraints. Code is available at https://github.com/yxliu02/REVES.git.
View arXiv pageView PDFGitHubAdd to collection
Get this paper in your agent:
hf papers read 2606\.18910
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2606.18910 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2606.18910 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2606.18910 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
Refining Over Resampling: Test-Time Self-Correction for LLM Reasoning
A new verifier-free breadth-depth refinement framework improves LLM reasoning at test time by sampling multiple rollouts, iteratively refining each via self-critique, and aggregating with majority voting. It consistently outperforms greedy decoding, majority voting, and verifier-based selection across several math benchmarks and open-weight models.
SAGE: Scalable Automated Robustness Augmentation for LLM Knowledge Evaluation
This paper introduces SAGE, a framework for scalable automated robustness augmentation of LLM knowledge evaluation benchmarks. It uses fine-tuned smaller models with reinforcement learning to generate and verify question variants at a lower cost than existing methods.
TEMPO: Scaling Test-time Training for Large Reasoning Models
TEMPO introduces a test-time training framework that alternates policy refinement with critic recalibration to prevent diversity collapse and sustain performance gains in large reasoning models, boosting AIME 2024 scores for Qwen3-14B from 42.3% to 65.8%.
EvoTrainer: Co-Evolving LLM Policies and Training Harnesses for Autonomous Agentic Reinforcement Learning
EvoTrainer introduces an autonomous training framework that co-evolves LLM policies and training harnesses through empirical feedback, outperforming human-engineered RL baselines on mathematical reasoning, code generation, and long-horizon software engineering tasks.
VeriEvol: Scaling Multimodal Mathematical Reasoning via Verifiable Evol-Instruct
VeriEvol is a novel framework for scaling reinforcement learning in visual mathematical reasoning by ensuring reliable reward labels through a two-axis approach separating prompt difficulty from answer reliability, using evolutionary operators and hypothesis-testing verification. It achieves significant accuracy gains on a five-benchmark visual-math suite.