VLMs are Good Teachers for Video Reasoning via Adaptive Test-Time Optimization
Summary
This paper introduces a paradigm where Vision-Language Models (VLMs) act as test-time teachers to guide Video Generation Models (VGMs) via differentiable rewards and LoRA optimization, achieving a 16.7-point average improvement on video reasoning benchmarks.
View Cached Full Text
Cached at: 06/02/26, 03:38 PM
Paper page - VLMs are Good Teachers for Video Reasoning via Adaptive Test-Time Optimization
Source: https://huggingface.co/papers/2606.02564 Published on Jun 1
·
Submitted byhttps://huggingface.co/Howe666
CJHon Jun 2
Abstract
Video generation models combined with vision-language models acting as test-time teachers through differentiable rewards achieve superior video reasoning performance.
The recent “Reasoning with Video” paradigm utilizesVideo Generation Models(VGMs) to generate temporally coherent visual trajectories to complete reasoning tasks. Although state-of-the-art VGMs excel at visual quality, they often struggle to understand and follow task-specific rules, leading to logical failures across diverse reasoning scenarios. Existing efforts try to utilizeVision-Language Models(VLMs) as problem pre-solvers to produce or refine textual guidance for the VGM. However, textual descriptions fail to capture intricate spatiotemporal details, and VGMs often struggle to faithfully execute fine-grained or long-tail instructions even with a valid plan. While VLMs struggle as solvers, they possess strong perception capabilities to evaluate process-constraint satisfaction and final-goal achievement. Leveraging this strength, we introduce a paradigm shift that transitions the role of VLMs to “teachers”. Specifically, a VLM teacher extracts task-specific rules to formulatedifferentiable rewards, guiding a VGM Reasoner via test-time online optimization of a lightweightLoRA module. This strategy enables adaptivetest-time optimizationand extends the reasoning capabilities beyond the VGM’s intrinsic boundaries. Evaluations on symbolic (VBVR-Bench) and general-purpose (RULER-Bench)video reasoning benchmarksshow that the proposed method yields a 16.7-point average performance gain, outperforming the VLM-as-Solver paradigm (+0.4 points) and Best-of-N scaling (+2.2 points) by a large margin at comparable test-time cost. These findings reveal that integrating VLMs as test-time teachers offers a promising paradigm for achieving generalizable video reasoning. Project Page: https://VLM-as-Teacher.github.io/
View arXiv pageView PDFProject pageAdd to collection
Get this paper in your agent:
hf papers read 2606\.02564
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2606.02564 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2606.02564 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2606.02564 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
VisualThink-VLA: Visual Intermediate Reasoning for Effective and Low-Latency Vision-Language-Action Policies
VisualThink-VLA introduces a visual intermediate reasoning framework for vision-language-action policies that preserves spatial precision and dramatically reduces latency compared to text-based reasoning, achieving sub-second inference and state-of-the-art success rates on robot manipulation benchmarks.
ThinkV2V: Unleashing the Reasoning Capability of MLLMs for Instruction-Guided Video Editing
ThinkV2V introduces a reasoning-driven framework that activates MLLM thinking before visual generation for instruction-guided video editing, using an MLLM-to-DiT architecture with progressive curriculum training and inference-time thinking scaling. The authors also release the ThinkV2V-150K dataset and ThinkV2V-Bench, showing their 5B DiT model outperforms larger 10B baselines.
Do VLMs Reason Like Engineers? A Benchmark and a Stage-wise Evaluation
This paper introduces EngVQA, a multimodal benchmark for evaluating engineering reasoning in vision-language models, along with an 8-stage automatic evaluation framework that enables fine-grained analysis of reasoning failures. It reveals substantial limitations in current VLMs' engineering reasoning capabilities.
WM-VLM: Probing Internal World Models for Interleaved Visual-Textual Reasoning
WM-VLM 论文为预训练 VLM 增加轻量级世界模型分支，通过两阶段训练生成中间视觉状态以支持语言与视觉双通道推理，在 2D/3D 心智旋转任务上较监督微调基线最高提升 39.25 个百分点。
VLX-VR: An Agentic-Aware Video Reasoning Model
VLX-VR is an agentic-aware video reasoning model that uses a Think–Memory–Observation loop and reinforcement learning to adaptively gather evidence, achieving state-of-the-art performance on the MINERVA benchmark.