Posterior Matching Distillation

Hugging Face Daily Papers Papers

Summary

A research paper titled 'Posterior Matching Distillation' (arXiv:2611.26629) introducing a distillation approach based on matching posterior distributions. No linked models, datasets, or Spaces have cited it yet on Hugging Face.

Diffusion distillation accelerates image and video generation by compressing a pretrained teacher's iterative sampling process into a few network evaluations. Distribution-based methods achieve this by matching the student's output distribution to the teacher's. This matching is often driven by reverse-KL objectives, whose mode-seeking bias favors dominant teacher modes at the expense of diversity across samples. We introduce Posterior Matching Distillation (PMD), a new member of the distribution-based distillation family that combines mode-covering posterior sampling with any-step generation. At zero population loss, PMD recovers the teacher's full posterior of clean images at each trained noise level. Its stochastic student supports any-step sampling budgets within the trained timestep grid without further training. To learn these posteriors, PMD matches exponentially reweighted averages of student samples conditioned on a fixed noisy state to corresponding teacher posterior means. Under Gaussian corruption, exponentially tilted posterior mean is a single teacher evaluation at a shifted input. We prove that, matching tilted means over an open neighborhood uniquely identifies the full posterior. PMD thus supervises the entire conditional distribution through a family of mean constraints, without any auxiliary student-score network. Experiments on SDXL and FLUX.2 [klein] 4B demonstrate that PMD achieves the best FID among 4-step methods on both models, while maintaining competitive or leading performance across prompt-alignment and perceptual-quality metrics.
Original Article
View Cached Full Text

Cached at: 11/20/26, 07:35 AM

Paper page - Posterior Matching Distillation

Source: https://huggingface.co/papers/2611.26629 Get this paper in your agent:

hf papers read 2611\.26629

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2611.26629 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2611.26629 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2611.26629 in a Space README.md to link it from this page.

Collections including this paper1

Similar Articles

Dataset Distillation by Influence Matching

Hugging Face Daily Papers

This paper introduces Influence Matching (Inf-Match), a dataset distillation method that aligns the final training outcome by learning a compact synthetic set whose effect on converged parameters matches that of the full dataset. It achieves state-of-the-art accuracy on classification benchmarks and outperforms strong baselines on vision-language distillation tasks.

eDMD: Endpoint Distribution Matching for Few-Step Diffusion Distillation

Hugging Face Daily Papers

eDMD proposes Endpoint Distribution Matching Distillation, which trains a shared fake model only on the student's final outputs and adds precise teacher supervision, achieving improved 4-step diffusion distillation on SD3-Medium, FLUX.2-klein, and Qwen-Image-2.1.

On-policy distillation: one of the hottest terms on PapersWithCode [R]

Reddit r/MachineLearning

Hugging Face's Niels introduces On-policy Distillation (OPD), a key post-training technique used in models like Qwen 3.6/3.7, GLM-5.1, and DeepSeek-V4, now featured on PapersWithCode with a linked whiteboard explanation by Sasha Rush and Dwarkesh Patel.