Video Models Can Reason with Verifiable Rewards
Summary
VideoRLVR optimizes video diffusion models for verifiable reasoning tasks using reinforcement learning with rule-based rewards, achieving better performance than supervised methods in constraint-satisfying video generation.
View Cached Full Text
Cached at: 05/20/26, 06:36 AM
Paper page - Video Models Can Reason with Verifiable Rewards
Source: https://huggingface.co/papers/2605.15458
Abstract
VideoRLVR optimizes video diffusion models for verifiable reasoning tasks using reinforcement learning with rule-based rewards, achieving better performance than supervised methods in constraint-satisfying video generation.
Video diffusion modelshave made rapid progress in perceptual realism and temporal coherence, but they remain primarily optimized for plausible generation rather than verifiable reasoning. This limitation is especially pronounced in tasks where generated videos must satisfy explicit spatial, temporal, or logical constraints. Inspired by the role ofreinforcement learningwithverifiable rewards(RLVR) in reasoning-oriented language models, we introduce VideoRLVR, a practical recipe for optimizingvideo diffusion modelswith rule-based feedback. VideoRLVR formulatesvideo reasoningas the generation of verifiablevisual trajectoriesand consists of anSDE-GRPOoptimization backbone,dense decomposed rewards, and anEarly-Step Focusstrategy for efficient training. TheEarly-Step Focusstrategy restrictspolicy optimizationto the early denoising phase, reducing training latency by about 40% while preserving performance. We evaluate VideoRLVR on Maze, FlowFree, and Sokoban, three procedurally generated domains with objective success criteria. Across these tasks, VideoRLVR consistently improves over supervised fine-tuning baselines, withdense decomposed rewardsproving especially important in low-success-rate settings. Our RL-optimized model also outperforms the evaluated proprietary and open-source video generation models on these verifiable reasoning benchmarks and out-of-domain benchmarks. These results suggest that verifiable RL can move video models beyond perceptual imitation toward more reliable rule-consistent visual reasoning.
View arXiv pageView PDFProject pageGitHub1Add to collection
Get this paper in your agent:
hf papers read 2605\.15458
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2605.15458 in a model README.md to link it from this page.
Datasets citing this paper1
#### DarthZhu/VideoRLVR-Data Preview• Updated26 minutes ago
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2605.15458 in a Space README.md to link it from this page.
Collections including this paper1
Similar Articles
CollabVR: Collaborative Video Reasoning with Vision-Language and Video Generation Models
CollabVR is a research paper proposing a closed-loop framework that collaboratively integrates vision-language models with video generation models to improve visual reasoning and correct failures in real-time.
VLMs are Good Teachers for Video Reasoning via Adaptive Test-Time Optimization
This paper introduces a paradigm where Vision-Language Models (VLMs) act as test-time teachers to guide Video Generation Models (VGMs) via differentiable rewards and LoRA optimization, achieving a 16.7-point average improvement on video reasoning benchmarks.
@dair_ai: Highly-recommended read from MIT on the part of RL with verifiable rewards that everyone keeps hitting. RLVR only optim…
This paper from MIT proposes an adversarial generator-discriminator framework that combines verifiable rewards with a learned signal from human demonstrations to address issues like diversity collapse, unnatural responses, and reward hacking in RLVR training of language models.
VLX-VR: An Agentic-Aware Video Reasoning Model
VLX-VR is an agentic-aware video reasoning model that uses a Think–Memory–Observation loop and reinforcement learning to adaptively gather evidence, achieving state-of-the-art performance on the MINERVA benchmark.
Tandem Reinforcement Learning with Verifiable Rewards
Proposes Tandem Reinforcement Learning (TRL), extending the tandem training paradigm to RLVR to improve reasoning compatibility and legibility for weaker models and humans, showing that TRL matches solo performance while enhancing handoff robustness and reducing distributional drift.