The Distillation Game: Adaptive Attacks & Efficient Defenses
Summary
This paper studies distillation attacks where model outputs can enable imitation, proposing a minimax game framework and a forward-pass-only defense called Product-of-Experts, showing that adaptive students recover more capability than passive evaluation suggests.
View Cached Full Text
Cached at: 06/09/26, 08:44 AM
Paper page - The Distillation Game: Adaptive Attacks & Efficient Defenses
Source: https://huggingface.co/papers/2605.22737 Published on May 29
·
Submitted byhttps://huggingface.co/ysfalh
Aon Jun 8
Abstract
Distillation attacks create a trade-off for model providers, where useful outputs also enable imitation, addressed through a minimax game framework with adaptive evaluation and defensive strategies.
Distillation attackscreate a deployment trade-off for model providers: the same outputs that make a model more useful can also make it easier to imitate. We study this trade-off through aminimax gamebetween autility-constrained teacherand anadaptive student. Our framework yields tractable one-sided response rules: anadaptive evaluationrule in which the student reweights high-value examples, and a teacher-sidedefense templatethat suppresses outputs most useful for distillation. From a cheap proxy for example value, we deriveProduct-of-Experts(PoE), a simpleforward-pass-only defensethat combines the teacher with aproxy studentduring generation. Empirically,adaptive evaluationreveals a large passive--adaptive gap: on state-of-the-art defenses,adaptive students recover substantially more capability than passive evaluation suggests on GSM8K and MATH. Under this stronger evaluation, the apparent robustness gap between expensive defenses and PoE narrows considerably, while PoE remains substantially cheaper and preserves higher-quality reasoning traces. Overall, our results suggest that strong distillation remains difficult to stop, and that progress on antidistillation should be judged againstadaptive students rather than passive ones. Our code is available at: https://github.com/ysfalh/distillation-game.
View arXiv pageView PDFAdd to collection
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2605.22737 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2605.22737 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2605.22737 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
Unmasking On-Policy Distillation: Where It Helps, Where It Hurts, and Why
This paper introduces a training-free diagnostic framework to analyze per-token distillation signals for reasoning models, revealing that guidance is more beneficial on incorrect rollouts and depends on student capacity and task context.
Adaptive Teacher Exposure for Self-Distillation in LLM Reasoning
Adaptive Teacher Exposure for Self-Distillation (ATESD) improves LLM reasoning by dynamically adjusting how much of the reference reasoning the teacher shows the student during training, using a learnable policy controller and a discounted learning-progress reward. Experiments on math benchmarks show consistent improvements over existing self-distillation and RL baselines.
@zhaisf: These were some magical results from distillation by @geoffreyhinton that really shocked me when I first saw them, and …
The article discusses surprising robustness of model distillation with respect to training distribution, even with little overlap with target distribution, and its implications for on/off-policy distillation.
Behavior Cloning is Not All You Need: The Optimality of On-Policy Distillation for Noisy Expert Feedback
This paper proposes a noisy expert model to explain the gap between offline and online imitation learning, showing that offline learning from noisy trajectories requires exponential sample complexity while online on-policy distillation achieves polynomial dependence. The analysis leads to an alternative loss function and experiments confirm the theoretical findings.
The Many Faces of On-Policy Distillation: Pitfalls, Mechanisms, and Fixes
This paper presents a comprehensive empirical study on on-policy distillation for large language models, identifying failure mechanisms like distribution mismatch and optimization instability, and proposing fixes such as stop-gradient objectives and RLVR-adapted teachers.