Mask Forcing: Improving Autoregressive Video Diffusion Distillation via Dual-Noise Masking Rollout
Summary
Mask Forcing mitigates mode collapse in autoregressive video diffusion distillation by injecting masked cleaner signals during self-rollout, improving visual quality without extra training data.
View Cached Full Text
Cached at: 09/09/26, 08:33 AM
Paper page - Mask Forcing: Improving Autoregressive Video Diffusion Distillation via Dual-Noise Masking Rollout
Source: https://huggingface.co/papers/2609.09123
Abstract
Mask Forcing mitigates mode collapse in distilled autoregressive video diffusion by injecting masked cleaner signals during self-rollout, improving visual quality without extra training data.
Autoregressive (AR) video diffusion models have shown great potential in real-time video generation. Recent methods distill pretrained bidirectional video diffusion models into causal AR students throughDistribution Matching Distillation(DMD), but the generated videos often suffer from over-saturation and over-smoothing issues, resulting in limited visual quality and realism. The key contributing factor is the mode-seeking behavior of thereverse KLobjective in DMD, which can cause the student distribution to collapse onto only a few modes of the teacher distribution. To address this, we proposeMask Forcing, aDual-Noise Masking Rolloutstrategy that perturbs the AR studentself-rolloutto mitigatemode collapseinduced by reverse-KL mode seeking. The core idea is to inject cleaner signals into noisy rollout inputs via random masks along spatial and temporal axes during theself-rolloutprocess of AR diffusion distillation. Such perturbations encourage the student rollouts to explore more regions of the teacher distribution, allowing DMD to provide learning signals beyond the modes already covered by the student. Moreover, the cleaner tokens act asdenoising guidancefor other noisier tokens, improving the intermediate rollout predictions and reducing error accumulation. Extensive experiments demonstrate that our method improves multiple AR video diffusion distillation methods with higher visual quality efficiently, without incorporating real video data or additional post-training stages.
View arXiv pageView PDFProject pageGitHub11Add to collection
Get this paper in your agent:
hf papers read 2609\.09123
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2609.09123 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2609.09123 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2609.09123 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
Flex-Forcing: Towards a Unified Autoregressive and Bidirectional Video Diffusion Model
Introduces Flex-Forcing, a unified training and inference framework that allows video diffusion models to operate under both bidirectional and autoregressive regimes via a flexible chunking mechanism over temporal and denoising steps, achieving better video quality, long-video stability, and faster inference.
Causal Forcing++: Scalable Few-Step Autoregressive Diffusion Distillation for Real-Time Interactive Video Generation
Causal Forcing++ presents a novel causal consistency distillation method for frame-wise autoregressive video generation, achieving state-of-the-art quality with reduced latency and training cost.
Reinforcing the Generation Order of Multimodal Masked Diffusion Models
This paper introduces a learnable control module trained via Group Relative Policy Optimization (GRPO) to optimize the generation order in multimodal masked diffusion models, achieving improvements in text-to-image alignment and multimodal understanding.
MV-Forcing: Long Multi-View Video Generation via 4D-Grounded Spatio-Temporal Self-Forcing
MV-Forcing introduces a diffusion framework that combines temporal and view-wise autoregression to generate long, multi-view consistent videos of dynamic scenes, using a 4D geometric bridge and spatio-temporal distillation to enable arbitrary-length generation from a few-step student model.
Ring Forcing: Towards Precise Long-Term Memory for Autoregressive Video Diffusion
Ring Forcing is an autoregressive video diffusion framework that enhances long-term memory for precise minute-long video coherence through ring-structured training, history compression, and sparse rotary embeddings.