Imagine Before You Predict: Interleaved Latent Visual Reasoning for Video Event Prediction
Summary
Introduces Future-L1, an interleaved latent visual reasoning framework that improves video event prediction by maintaining visual semantics in latent space. Achieves state-of-the-art results on FutureBench and TwiFF-Bench benchmarks.
View Cached Full Text
Cached at: 06/05/26, 06:07 AM
Paper page - Imagine Before You Predict: Interleaved Latent Visual Reasoning for Video Event Prediction
Source: https://huggingface.co/papers/2606.05769
Abstract
Future-L1, an interleaved latent visual reasoning framework, improves video event prediction by maintaining visual semantics in latent space during autoregressive decoding, achieving state-of-the-art results on FutureBench and TwiFF-Bench benchmarks.
Video event prediction(VEP) requires models to infer unobserved future states from partial video evidence. Existingvideo MLLMsusually verbalize intermediate future reasoning in text space: once visual evidence is verbalized, fine-grained motion, geometry, and interaction cues can be lost, leading to plausible but visually ungrounded hallucinations. We introduce Future-L1, an interleavedlatent visual reasoningframework that lets an MLLM alternate betweenlanguage tokensand continuous latent visual spans duringautoregressive decoding. To train this capability, we construct Future-L1-50K by selecting examples where future visual hints help prediction and align latent states to future-frame embeddings, then further optimize sampled latent trajectories withLA-DAPO, alatent-aware RL objectivewith outcome-contrastive andtemporal-diversity rewards. Future-L1 achieves new state-of-the-art results on both benchmarks: onFutureBench, it improves Qwen3-VL-8B from 61.0 to 85.4 and exceeds the previous best Video-CoE by 10.4 points; onTwiFF-Bench, it improves the average score from 2.44 to 3.04. These results suggest that future-oriented video reasoning benefits from preserving intermediate visual semantics in latent space rather than translating every reasoning step into text.
View arXiv pageView PDFGitHub2Add to collection
Get this paper in your agent:
hf papers read 2606\.05769
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2606.05769 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2606.05769 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2606.05769 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
Rethinking Latent Visual Reasoning: Grounding Latent Reasoning in Visual Evidence
This paper identifies a 'latent evidence-credit gap' in latent visual reasoning for multimodal LLMs and proposes ReaLVR, which supplies visual-evidence supervision to free-running latent trajectories via contrastive correct/wrong answers and relevant/mismatched images. ReaLVR outperforms LVR baselines across three model families (63.7% five-task average on Qwen2.5-VL-7B) and scales robustly to 235B-parameter frontier models.
VisualThink-VLA: Visual Intermediate Reasoning for Effective and Low-Latency Vision-Language-Action Policies
VisualThink-VLA introduces a visual intermediate reasoning framework for vision-language-action policies that preserves spatial precision and dramatically reduces latency compared to text-based reasoning, achieving sub-second inference and state-of-the-art success rates on robot manipulation benchmarks.
Beyond Visual CoT: Internalized Visual Thinking for Proactive Video Reasoning
The paper introduces Internalized Visual Thinking (IVT), a post-training framework that trains multimodal models to predict future frame embeddings, enabling direct answer generation without synthesizing intermediate images, thereby reducing latency over 5x for proactive video reasoning.
Retrieve, Integrate, and Synthesize: Spatial-Semantic Grounded Latent Visual Reasoning
This paper introduces RIS, a framework for spatial-semantic grounded latent visual reasoning in Multimodal Large Language Models to overcome information bottlenecks. It proposes anchoring latent tokens to spatial and semantic evidence, showing improvements on benchmarks like V* and HRBench.
LongE2V: Long-Horizon Event-based Video Reconstruction, Prediction, and Frame Interpolation with Video Diffusion Models
LongE2V leverages pre-trained video diffusion priors and introduces Autoregressive Unrolling, Adaptive Context Switching, Reencoding Alignment, and Event Voxel Density Augmentation to achieve high-quality video reconstruction, prediction, and frame interpolation from sparse event streams, outperforming state-of-the-art methods.