Context-Matched Distillation: Teacher Causality for Autoregressive Video Distillation
Summary
Context-Matched Distillation introduces a causal framework for distilling autoregressive video models, improving control adherence and long-video quality by aligning teacher supervision with the student's causal information context.
View Cached Full Text
Cached at: 08/14/26, 07:29 PM
Paper page - Context-Matched Distillation: Teacher Causality for Autoregressive Video Distillation
Source: https://huggingface.co/papers/2608.13391
Abstract
Context-Matched Distillation aligns teacher supervision with causal generation context for few-step autoregressive video models, improving control adherence and long-video quality.
Interactiveautoregressive video generationdemands both low-latency rollouts and precise online control.Few-step distillationaccelerates generation by reducing denoising steps, while online control imposes a causal constraint: frames and blocks should depend on history and controls available during generation. Existingvideo distribution matching distillation(DMD) pipelines, however, often supervise causal few-step students using bidirectional teachers that score complete clips. The score for a target can therefore depend on future frames and controls that were unavailable when the student generated it, misaligning teacher supervision with the student’s causal information set. We introduceContext-Matched Distillation(CMD), a causal DMD framework that aligns teacher supervision with the information available when each target is generated. CMD replaces bidirectional full-clip scoring with acausal teacherthat evaluates each target without access to future frames or controls. The samecausal teacherinitializes the few-step student, establishing a consistent causal formulation across teacher training, student distillation, and inference. Beyond aligning the temporal information boundary,Prefix Scoringmatches supervision to the student’s realized rollout context by evaluating each target under the cached student-generated prefix that produced it.Prefix Corruptionfurther stabilizes training by perturbing unreliable prefixes produced early in training while preserving this target-context alignment. With a simple causal formulation, CMD naturally extends to frame-wise and chunk-wise generation, long video distillation, andcamera-conditioned distillation. Experiments demonstrate state-of-the-art aggregate performance among autoregressive methods on both short- and long-video benchmarks, together with substantially improved adherence to time-varying camera controls.
View arXiv pageView PDFProject pageAdd to collection
Get this paper in your agent:
hf papers read 2608\.13391
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2608.13391 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2608.13391 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2608.13391 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
Causal-rCM: A Unified Teacher-Forcing and Self-Forcing Open Recipe for Autoregressive Diffusion Distillation in Streaming Video Generation and Interactive World Models
This paper introduces Causal-rCM, a unified teacher-forcing and self-forcing framework for autoregressive diffusion distillation in streaming video generation and interactive world models, achieving state-of-the-art performance with fast convergence.
Causal Forcing++: Scalable Few-Step Autoregressive Diffusion Distillation for Real-Time Interactive Video Generation
Causal Forcing++ presents a novel causal consistency distillation method for frame-wise autoregressive video generation, achieving state-of-the-art quality with reduced latency and training cost.
Rollout-Marginal Distillation for Long-Horizon Autoregressive Video Generation
This paper introduces Rollout-Marginal Distillation (RMD) for autoregressive video diffusion, where each generated chunk is scored independently against a chunk teacher to avoid error-compounding from imperfect temporal context, followed by video-level DMD to restore coherence. RMD reportedly maintains high visual quality far beyond its training horizon and outperforms video-level DMD baselines.
When Context Returns: Toward Robust Internalization in On-Policy Distillation
The paper identifies that reintroducing privileged context to a distilled student model degrades performance (context-induced degradation), and proposes a lightweight consistency regularizer that anchors no-context outputs to mitigate this issue, improving robustness across 12 configurations.
ContextMaster: Interactive Multi-Shot Video Creation via Fixed-Budget Sparse Context Routing
ContextMaster is a unified model for interactive multi-shot video creation that maintains shared history across generation, reference conditioning, and editing while using fixed-budget sparse context routing and two-stage privileged context distillation for efficiency.