VLMs are Good Teachers for Video Reasoning via Adaptive Test-Time Optimization
Summary
This paper introduces a paradigm where Vision-Language Models (VLMs) act as test-time teachers to guide Video Generation Models (VGMs) via differentiable rewards and LoRA optimization, achieving a 16.7-point average improvement on video reasoning benchmarks.
View Cached Full Text
Cached at: 06/02/26, 03:38 PM
Paper page - VLMs are Good Teachers for Video Reasoning via Adaptive Test-Time Optimization
Source: https://huggingface.co/papers/2606.02564 Published on Jun 1
·
Submitted byhttps://huggingface.co/Howe666
CJHon Jun 2
Abstract
Video generation models combined with vision-language models acting as test-time teachers through differentiable rewards achieve superior video reasoning performance.
The recent “Reasoning with Video” paradigm utilizesVideo Generation Models(VGMs) to generate temporally coherent visual trajectories to complete reasoning tasks. Although state-of-the-art VGMs excel at visual quality, they often struggle to understand and follow task-specific rules, leading to logical failures across diverse reasoning scenarios. Existing efforts try to utilizeVision-Language Models(VLMs) as problem pre-solvers to produce or refine textual guidance for the VGM. However, textual descriptions fail to capture intricate spatiotemporal details, and VGMs often struggle to faithfully execute fine-grained or long-tail instructions even with a valid plan. While VLMs struggle as solvers, they possess strong perception capabilities to evaluate process-constraint satisfaction and final-goal achievement. Leveraging this strength, we introduce a paradigm shift that transitions the role of VLMs to “teachers”. Specifically, a VLM teacher extracts task-specific rules to formulatedifferentiable rewards, guiding a VGM Reasoner via test-time online optimization of a lightweightLoRA module. This strategy enables adaptivetest-time optimizationand extends the reasoning capabilities beyond the VGM’s intrinsic boundaries. Evaluations on symbolic (VBVR-Bench) and general-purpose (RULER-Bench)video reasoning benchmarksshow that the proposed method yields a 16.7-point average performance gain, outperforming the VLM-as-Solver paradigm (+0.4 points) and Best-of-N scaling (+2.2 points) by a large margin at comparable test-time cost. These findings reveal that integrating VLMs as test-time teachers offers a promising paradigm for achieving generalizable video reasoning. Project Page: https://VLM-as-Teacher.github.io/
View arXiv pageView PDFProject pageAdd to collection
Get this paper in your agent:
hf papers read 2606\.02564
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2606.02564 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2606.02564 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2606.02564 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
VisualThink-VLA: Visual Intermediate Reasoning for Effective and Low-Latency Vision-Language-Action Policies
VisualThink-VLA introduces a visual intermediate reasoning framework for vision-language-action policies that preserves spatial precision and dramatically reduces latency compared to text-based reasoning, achieving sub-second inference and state-of-the-art success rates on robot manipulation benchmarks.
Do VLMs Reason Like Engineers? A Benchmark and a Stage-wise Evaluation
This paper introduces EngVQA, a multimodal benchmark for evaluating engineering reasoning in vision-language models, along with an 8-stage automatic evaluation framework that enables fine-grained analysis of reasoning failures. It reveals substantial limitations in current VLMs' engineering reasoning capabilities.
VLX-VR: An Agentic-Aware Video Reasoning Model
VLX-VR is an agentic-aware video reasoning model that uses a Think–Memory–Observation loop and reinforcement learning to adaptively gather evidence, achieving state-of-the-art performance on the MINERVA benchmark.
Video Models Can Reason with Verifiable Rewards
VideoRLVR optimizes video diffusion models for verifiable reasoning tasks using reinforcement learning with rule-based rewards, achieving better performance than supervised methods in constraint-satisfying video generation.
TestHallVQA: Exploring LVLMs' Document-Level Reasoning under Redundant Contexts from Scientific Exams
This paper introduces TestHallVQA, a multi-image VQA benchmark for evaluating Large Vision-Language Models' document-level reasoning under redundant contexts, and proposes a new metric F1-R2 to quantify computational reasoning and robustness.