DMAD: Distribution Matching as Adversarial Distillation for Fast Visual Generation

Papers with Code Trending Papers
distillation generative-models diffusion-models video-generation adversarial-training image-generation few-step-sampling

Summary

DMAD (Distribution Matching as Adversarial Distillation) recasts distribution matching as a classification problem, learning log-density ratios via discriminator heads to train few-step student models for fast visual generation without auxiliary score fitting. It achieves state-of-the-art results, including FID 1.04 on ImageNet-64x64 with one-step generation and VBench score 85.15 for four-step Wan2.1-T2V-14B.

Distribution Matching Distillation (DMD) trains a few-step student from the difference between separately estimated target and student scores, so it must keep an auxiliary diffusion model fitted to the student's evolving distribution at extra memory and computation cost. We introduce DMAD, Distribution Matching as Adversarial Distillation, which recasts distribution matching as classification and learns the required log-density ratios directly. Two discriminator heads on a shared backbone distinguish real data and teacher samples from the student's, and linear losses on their logits train the student without auxiliary score fitting. We prove that at the discriminator optimum these losses recover the distribution-matching gradient underlying DMD, through the classical identity linking discriminator logits to log-density ratios. We further introduce gap-based reweighting, which adapts teacher supervision across noise levels from the real-data head's empirical logit gap between real and teacher samples. DMAD reaches a Fréchet Inception Distance (FID) of 1.04 with one-step generation on ImageNet-64x64, 14.47 with four-step SDXL on COCO-10K, and a VBench total score of 85.15 with four-step Wan2.1-T2V-14B, the best values among the compared few-step methods and the multi-step teachers. On MiniMax-H3-33B, our four-step student achieves overall human preference rates of 79.1% over DMD2 and 84.6% over rCM for joint audio-video generation, excluding ties. Our code, models and demos are available at https://yzmblog.github.io/projects/DMAD.
Original Article
View Cached Full Text

Cached at: 10/08/26, 12:52 PM

Paper page - DMAD: Distribution Matching as Adversarial Distillation for Fast Visual Generation

Source: https://huggingface.co/papers/2610.02188 Authors:

,

,

,

,

,

,

,

,

,

Abstract

DistributionMatchingDistillation(DMD)trainsafew-stepstudentfromthedifferencebetweenseparatelyestimatedtargetandstudentscores,soitmustkeepanauxiliarydiffusionmodelfittedtothestudent’sevolvingdistributionatextramemoryandcomputationcost.WeintroduceDMAD,DistributionMatchingasAdversarialDistillation,whichrecastsdistributionmatchingasclassificationandlearnstherequiredlog-densityratiosdirectly.Twodiscriminatorheadsonasharedbackbonedistinguishrealdataandteachersamplesfromthestudent’s,andlinearlossesontheirlogitstrainthestudentwithoutauxiliaryscorefitting.Weprovethatatthediscriminatoroptimumtheselossesrecoverthedistribution-matchinggradientunderlyingDMD,throughtheclassicalidentitylinkingdiscriminatorlogitstolog-densityratios.Wefurtherintroducegap-basedreweighting,whichadaptsteachersupervisionacrossnoiselevelsfromthereal-datahead’sempiricallogitgapbetweenrealandteachersamples.DMADreachesaFréchetInceptionDistance(FID)of1.04withone-stepgenerationonImageNet-64x64,14.47withfour-stepSDXLonCOCO-10K,andaVBenchtotalscoreof85.15withfour-stepWan2.1-T2V-14B,thebestvaluesamongthecomparedfew-stepmethodsandthemulti-stepteachers.OnMiniMax-H3-33B,ourfour-stepstudentachievesoverallhumanpreferenceratesof79.1%overDMD2and84.6%overrCMforjointaudio-videogeneration,excludingties.Ourcode,modelsanddemosareavailableathttps://yzmblog.github.io/projects/DMAD.

View arXiv pageView PDFProject pageGitHub114Add to collection

Models citing this paper2

#### ZhengmingYu/DMAD Text-to-Video• Updatedabout 14 hours ago • 84 #### nixsn/DMAD Text-to-Video• Updated3 days ago

Datasets citing this paper1

#### ZhengmingYu/DMAD-H3-data Preview• Updated1 day ago • 163 • 2

Spaces citing this paper3

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Representation Distribution Matching for One-Step Visual Generation

Hugging Face Daily Papers

This paper introduces Representation Distribution Matching (RDM), a method for one-step image generation by matching feature distributions under pretrained encoders, achieving state-of-the-art results on ImageNet and enabling post-training of FLUX.2 into a one-step generator with improved performance.

Reinforcing Few-step Generators via Reward-Tilted Distribution Matching

Hugging Face Daily Papers

RTDMD is a two-stage framework combining distribution matching distillation with reward-guided reinforcement learning to improve few-step image generation alignment with human preferences. It achieves state-of-the-art results on multiple models with only 4 inference steps.

Consistent Distribution Matching for Data-Free Diffusion Distillation

Hugging Face Daily Papers

论文提出 Consistent Distribution Matching (CDMD)，通过在单一学生网络内统一样本生成与速度估计，实现无需辅助 fake-score 网络的数据自由扩散蒸馏，在 ImageNet 256×256 上以 1 NFE 达到 2.04 FID，优于 DMD2 等基线，并给出基于 Wasserstein 收敛的理论分析。

Improved Distributional Diffusion Models

Hugging Face Daily Papers

The paper presents Improved Distributional Diffusion Models (iDDM), which overcome scaling limits of DDMs via late particle expansion and time-dependent scoring rule schedules, achieving 4.48 FID at 4 steps and 2.38 FID at 50 steps on ImageNet-256 with DiT-XL/2 from a single-stage from-scratch training without teachers or self-distillation.