Consistent Distribution Matching for Data-Free Diffusion Distillation

Hugging Face Daily Papers Papers

Summary

论文提出 Consistent Distribution Matching (CDMD),通过在单一学生网络内统一样本生成与速度估计,实现无需辅助 fake-score 网络的数据自由扩散蒸馏,在 ImageNet 256×256 上以 1 NFE 达到 2.04 FID,优于 DMD2 等基线,并给出基于 Wasserstein 收敛的理论分析。

Flow and diffusion models suffer from slow inference due to computationally expensive numerical integration. Distillation provides a promising way for a student model to learn from a teacher's dynamics, enabling one-step or few-step generation. However, existing methods often depend on curated distillation datasets, costly teacher rollouts, or auxiliary proxy networks, which complicate model training and scaling. In this work, we propose Consistent Distribution Matching, a simulation-free and data-free distillation method for accelerating diffusion and flow models while preserving strong generative capacity. Our key insight is to unify sample generation and score estimation with one student network. Thus, our framework uses only two models, a frozen teacher and a trainable student, and optimizes one objective. We prove that minimizing our objective indicates Wasserstein convergence of the student flow-map pushforwards to the teacher marginals. On ImageNet 256times256, our method attains an FID of 2.04 with a single function evaluation (1-NFE) and a 4-NFE FID of 1.37 within 40 epochs of training, surpassing the state-of-the-art distillation baselines without data. Our code code and model are available at https://consistentdmd.github.io/.
Original Article
View Cached Full Text

Cached at: 10/19/26, 07:27 AM

Paper page - Consistent Distribution Matching for Data-Free Diffusion Distillation

Source: https://huggingface.co/papers/2610.09221 Excited to share our work on Consistent Distribution Matching! 🚀

Inspired by Distribution Matching Distillation (DMD) and Score Identity Distillation (SiD), we ask a simple question: Can we achieve distribution matching without training an auxiliary fake-score network?

Our answer is yes!

CDMD takes a step further by unifying sample generation and velocity estimation within a single student network. This gives us a much simpler distillation framework:

One frozen teacher + one trainable student + one objective.

No external training data, teacher trajectory simulation, or auxiliary score networks in our data-free setting.

Stronger generative performance: 2.04 FID at 1 NFE on ImageNet 256×256, outperforming evaluated data-free baselines (DMD2: 3.43 FID).

Near-teacher quality: 1.37 FID at 4 NFEs, approaching the teacher’s 1.35 FID.

We also provide theoretical insights connecting our consistency objective to distribution matching through Wasserstein convergence.

Our takeaway: Effective distribution matching doesn’t have to come with complicated multi-network optimization.

We’d love to hear your thoughts and feedback!

Similar Articles

Improved Distributional Diffusion Models

Hugging Face Daily Papers

The paper presents Improved Distributional Diffusion Models (iDDM), which overcome scaling limits of DDMs via late particle expansion and time-dependent scoring rule schedules, achieving 4.48 FID at 4 steps and 2.38 FID at 50 steps on ImageNet-256 with DiT-XL/2 from a single-stage from-scratch training without teachers or self-distillation.

DMAD: Distribution Matching as Adversarial Distillation for Fast Visual Generation

Papers with Code Trending

DMAD (Distribution Matching as Adversarial Distillation) recasts distribution matching as a classification problem, learning log-density ratios via discriminator heads to train few-step student models for fast visual generation without auxiliary score fitting. It achieves state-of-the-art results, including FID 1.04 on ImageNet-64x64 with one-step generation and VBench score 85.15 for four-step Wan2.1-T2V-14B.

Simplifying and Improving On-Policy Distillation for Few-Step Diffusion

Hugging Face Daily Papers

提出简化匹配蒸馏(SMD)框架,通过纯回归的on-policy速度回归与Pivot残差代理,实现无需辅助评分模型或对抗训练的4步少步扩散生成,在图像、3D、视频及音视频联合生成任务上超越强基线并可扩展至33B参数。