SVR-R1: Bootstrapping Multi-modal Reasoning with Self-verification in Reinforcement Learning
Summary
Introduces SVR-R1, a multi-turn reinforcement learning framework that uses the model's own verification as a learning signal for multi-modal reasoning, achieving significant accuracy improvements over standard GRPO baselines on vision-language reasoning benchmarks.
View Cached Full Text
Cached at: 07/20/26, 09:26 PM
Paper page - SVR-R1: Bootstrapping Multi-modal Reasoning with Self-verification in Reinforcement Learning
Source: https://huggingface.co/papers/2607.10966 Authors:
,
,
,
,
,
,
,
,
,
,
,
,
Abstract
WeintroduceSelf-VerifiedReasoner(SVR-R1),amulti-turnRLframeworkthatturnsamodel’sownverificationintoalearningsignalformultimodalreasoning.Foreachquery,themodelproposesananswerusingthesameweights,andissuesabinaryself-verdict(Yes/No).A’No’triggersasecond-chancerethink;a’Yes,’oraturncap,finalizestheoutputforcomputingtheoutcome-basedreward.SVR-R1isimplementedwithGRPOandanasynchronousmulti-turnrolloutframeworkandneedsnoexternalsupervisionorauxiliarycritics.WeevaluateSVR-R1onvision-languagereasoningbenchmarksandshowthatitimprovesaccuracybyalargemarginoverstrongstandardGRPObaselines.Trainingdynamicsshowdecreasingrelianceonverification-fewerverificationturns,yethighertestaccuracy-indicatingthatthegapbetweenverificationandgenerationnarrowsasthepolicyinternalizesself-correctionandchoosesthemostconfidentanswerviaourframework.SVR-R1bridgesthelessexploredintersectionofinference-timeself-refinementandRLtrainingforVLMs,offeringasimpleyeteffectiverecipeforbootstrappingmultimodalreasoning.Wewillopen-sourceSVR-R1tofacilitatefutureresearchinVLMs.
View arXiv pageView PDFAdd to collection
Get this paper in your agent:
hf papers read 2607\.10966
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2607.10966 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2607.10966 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2607.10966 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
Reinforcing Dual-Path Reasoning in Spatial Vision Language Models
This paper introduces SR-REAL, a unified framework for spatial vision-language models that combines linguistic deduction and 3D geometric reasoning via reinforcement learning, enabling robust multi-step spatial reasoning across diverse tasks.
Tandem Reinforcement Learning with Verifiable Rewards
Proposes Tandem Reinforcement Learning (TRL), extending the tandem training paradigm to RLVR to improve reasoning compatibility and legibility for weaker models and humans, showing that TRL matches solo performance while enhancing handoff robustness and reducing distributional drift.
Video Models Can Reason with Verifiable Rewards
VideoRLVR optimizes video diffusion models for verifiable reasoning tasks using reinforcement learning with rule-based rewards, achieving better performance than supervised methods in constraint-satisfying video generation.
iVGR: Internalizing Visually Grounded Reasoning for MLLMs with Reinforcement Learning
Introduces iVGR, a reinforcement learning framework that internalizes visual localization into textual reasoning for multimodal language models, eliminating the need for explicit visual grounding during inference while improving fine-grained perception performance.
CORA: Analyzing and bridging thinking-answer gap in Multimodal RLVR via Consistency-Oriented Reasoning Alignment
This paper analyzes the thinking-answer inconsistency in multimodal reinforcement learning with verifiable rewards (RLVR) for large vision-language models and proposes CORA, a method that introduces a consistency reward model and hybrid reward advantage splitting to improve faithfulness and task performance.