Consistent Distribution Matching for Data-Free Diffusion Distillation
Summary
论文提出 Consistent Distribution Matching (CDMD),通过在单一学生网络内统一样本生成与速度估计,实现无需辅助 fake-score 网络的数据自由扩散蒸馏,在 ImageNet 256×256 上以 1 NFE 达到 2.04 FID,优于 DMD2 等基线,并给出基于 Wasserstein 收敛的理论分析。
View Cached Full Text
Cached at: 10/19/26, 07:27 AM
Paper page - Consistent Distribution Matching for Data-Free Diffusion Distillation
Source: https://huggingface.co/papers/2610.09221 Excited to share our work on Consistent Distribution Matching! 🚀
Inspired by Distribution Matching Distillation (DMD) and Score Identity Distillation (SiD), we ask a simple question: Can we achieve distribution matching without training an auxiliary fake-score network?
Our answer is yes!
CDMD takes a step further by unifying sample generation and velocity estimation within a single student network. This gives us a much simpler distillation framework:
One frozen teacher + one trainable student + one objective.
No external training data, teacher trajectory simulation, or auxiliary score networks in our data-free setting.
Stronger generative performance: 2.04 FID at 1 NFE on ImageNet 256×256, outperforming evaluated data-free baselines (DMD2: 3.43 FID).
Near-teacher quality: 1.37 FID at 4 NFEs, approaching the teacher’s 1.35 FID.
We also provide theoretical insights connecting our consistency objective to distribution matching through Wasserstein convergence.
Our takeaway: Effective distribution matching doesn’t have to come with complicated multi-network optimization.
We’d love to hear your thoughts and feedback!
Similar Articles
Improved Distributional Diffusion Models
The paper presents Improved Distributional Diffusion Models (iDDM), which overcome scaling limits of DDMs via late particle expansion and time-dependent scoring rule schedules, achieving 4.48 FID at 4 steps and 2.38 FID at 50 steps on ImageNet-256 with DiT-XL/2 from a single-stage from-scratch training without teachers or self-distillation.
Multi-Bandwidth Distribution Matching Distillation: On the Equivalence of Distribution Matching Distillation and Drifting Models
本文证明了 Drifting Models 与 Distribution Matching Distillation (DMD/DMD2) 在优化目标上的等价性,并基于此视角提出了改进的 Multi-Bandwidth Distribution Matching Distillation (MBDMD) 方法,用于提升一步生成模型的性能。
DMAD: Distribution Matching as Adversarial Distillation for Fast Visual Generation
DMAD (Distribution Matching as Adversarial Distillation) recasts distribution matching as a classification problem, learning log-density ratios via discriminator heads to train few-step student models for fast visual generation without auxiliary score fitting. It achieves state-of-the-art results, including FID 1.04 on ImageNet-64x64 with one-step generation and VBench score 85.15 for four-step Wan2.1-T2V-14B.
Simplifying and Improving On-Policy Distillation for Few-Step Diffusion
提出简化匹配蒸馏(SMD)框架,通过纯回归的on-policy速度回归与Pivot残差代理,实现无需辅助评分模型或对抗训练的4步少步扩散生成,在图像、3D、视频及音视频联合生成任务上超越强基线并可扩展至33B参数。
Continuous-Time Distribution Matching for Few-Step Diffusion Distillation
This paper introduces Continuous-Time Distribution Matching (CDM), a method for few-step diffusion distillation that migrates from discrete to continuous optimization to improve visual fidelity and preserve fine details.