Dataset Distillation by Influence Matching
Summary
This paper introduces Influence Matching (Inf-Match), a dataset distillation method that aligns the final training outcome by learning a compact synthetic set whose effect on converged parameters matches that of the full dataset. It achieves state-of-the-art accuracy on classification benchmarks and outperforms strong baselines on vision-language distillation tasks.
View Cached Full Text
Cached at: 07/24/26, 05:02 PM
Paper page - Dataset Distillation by Influence Matching
Source: https://huggingface.co/papers/2607.16859
Abstract
Werevisitdatasetdistillationfromanoutcome-centricperspective.Ratherthanaligningprocesssurrogates(per-stepgradientsortrainingtrajectories),InfluenceMatching(Inf-Match)alignsthefinaloutcomeoftraining:itlearnsacompactsyntheticsetwhoseeffectontheconvergedparametersmatchesthatofthefulldataset.Concretely,weintroduceafullydifferentiable,sample-levelinfluenceestimatorthatquantifiesparametershiftsfromaddingorremovingdata,withouttime-consuminginverse-Hessianproductsorconvexityassumptions.Theestimatorrunsinlineartimebyunrollingtheoptimizationdynamicsandapplyingafirst-orderTaylorapproximation.Wethenlearnthesyntheticsetbyminimizingthemismatchbetweenitsinfluenceandthatoftherealdataset,yieldingoutcomealignmentratherthanheuristicprocessimitation.Inf-Matchdeliversthebestaccuracyacrossstandardclassificationbenchmarks.Forinstance,onTiny-ImageNet(IPC=10),Inf-Matchattains31.5\%,a+4.7\%improvementoverNCFM.Beyondclassification,Inf-Matchscalestovision-languagedistillationonFlickr30K,outperformingstrongprocess-matchingbaselines.Forinstance,with200to1000syntheticsamples,ourmethodachievedaleadingimpressiveaverageonimage/textretrievaltasks,higherthanNCFMby2.5\%.Thecodewillbereleasedviahttps://github.com/hrtan/infmatch.
View arXiv pageView PDFProject pageGitHub0Add to collection
Get this paper in your agent:
hf papers read 2607\.16859
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2607.16859 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2607.16859 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2607.16859 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
Posterior Matching Distillation
A research paper titled 'Posterior Matching Distillation' (arXiv:2611.26629) introducing a distillation approach based on matching posterior distributions. No linked models, datasets, or Spaces have cited it yet on Hugging Face.
Data Unlearning via Inverse Distillation
The paper introduces Inverse Distillation Unlearning (IDU), a unified framework that simultaneously distills a multi-step flow-matching or diffusion teacher into an efficient one-step student while suppressing generation of forgotten training data, requiring only the teacher and forget-set samples without access to retained data or auxiliary classifiers.
eDMD: Endpoint Distribution Matching for Few-Step Diffusion Distillation
eDMD proposes Endpoint Distribution Matching Distillation, which trains a shared fake model only on the student's final outputs and adds precise teacher supervision, achieving improved 4-step diffusion distillation on SD3-Medium, FLUX.2-klein, and Qwen-Image-2.1.
TAKE: Trajectory-Aware Knowledge Estimation for Text Dataset Distillation
This paper introduces TAKE (Trajectory-Aware Knowledge Estimation), a text dataset distillation framework that uses influence functions and optimal transport to reduce datasets to as little as 0.1% of their original size while preserving downstream task fidelity.
DMAD: Distribution Matching as Adversarial Distillation for Fast Visual Generation
DMAD (Distribution Matching as Adversarial Distillation) recasts distribution matching as a classification problem, learning log-density ratios via discriminator heads to train few-step student models for fast visual generation without auxiliary score fitting. It achieves state-of-the-art results, including FID 1.04 on ImageNet-64x64 with one-step generation and VBench score 85.15 for four-step Wan2.1-T2V-14B.