data-free-distillation

论文提出 Consistent Distribution Matching (CDMD)，通过在单一学生网络内统一样本生成与速度估计，实现无需辅助 fake-score 网络的数据自由扩散蒸馏，在 ImageNet 256×256 上以 1 NFE 达到 2.04 FID，优于 DMD2 等基线，并给出基于 Wasserstein 收敛的理论分析。

This paper investigates the failure of boundary-seeking knowledge distillation (CAKE) when applied to bottlenecked generative autoencoders, showing that the shared latent manifold creates gradient conflicts that prevent effective synthesis of contrastive samples. A simple noise forward pass baseline is proposed instead.