ADS-C: Antidistillation Sampling for Classification

# ADS-C: Antidistillation Sampling for Classification
Source: [https://arxiv.org/html/2607.15467](https://arxiv.org/html/2607.15467)
\\receiveddate

XX Month, XXXX\\reviseddateXX Month, XXXX\\accepteddateXX Month, XXXX\\publisheddateXX Month, XXXX\\currentdateXX Month, XXXX\\doiinfoXXXX\.2022\.1234567

\\corresp

Corresponding author: Khawaja Abaid Ullah \(email: ka4150@rit\.edu\)\.

Mohammad Javad Khojasteh1\{\}^\{\\textbf\{1\}\}Department of Electrical and Microelectronic Engineering, Rochester Institute of Technology, Rochester, NY 14623 USA

###### Abstract

Knowledge distillation enables an adversary to replicate a proprietary classifier by querying its prediction interface and training a surrogate on the returned probability vectors\. Antidistillation sampling, proposed for large language models, counters this threat with an input\-dependent, gradient\-directed perturbation of the served distribution; its transfer to classification has not been studied\. Adapting the defense to classification, we show its behavior is governed by the distribution of the teacher’s per\-input confidence margins\. Because well\-trained classifiers are severely overconfident, the direct transfer exhibits an inert window: below a closed\-form\-predictable threshold, it affects neither attacker nor defender; beyond it, the defense undergoes a phase transition and degrades the teacher faster than the attacker’s student\. Temperature softening rescales the transition in closed form, and every temperature configuration lies on the same unfavorable trade\-off curve\. Our method, ADS\-C, composes the perturbation under a closed\-form, per\-input margin budget that provably preserves every served top\-1 prediction, so the defended teacher’s accuracy equals the undefended teacher’s identically\. Under this guarantee the distilled student still loses 17\.4 percentage points on CIFAR\-100, 29\.6 on CIFAR\-10, and 13\.3 on Tiny\-ImageNet; matching this degradation with the unmodified defense costs 27\.5, 32\.9, and 22\.2 points of teacher accuracy\. Because served labels are unchanged, a hard\-label attacker gains nothing, while the defended soft output trains a student up to 29\.7 points below that floor: the incentive to distill served probabilities is not merely removed but reversed\. To our knowledge, ADS\-C is the first antidistillation defense for classification whose utility cost is exactly zero\.

\{IEEEkeywords\}

Antidistillation, knowledge distillation, model extraction, inference\-time defenses, machine learning security\.

## 1INTRODUCTION

\\IEEEPARstart

Knowledge distillation \(KD\)\[[1](https://arxiv.org/html/2607.15467#bib.bib1),[2](https://arxiv.org/html/2607.15467#bib.bib2)\]transfers the predictive function of a large*teacher*model into a smaller*student*model by training the student on the teacher’s returned probability vector outputs\. This very same mechanism that makes possible the compression of large models into smaller, efficient networks\[[3](https://arxiv.org/html/2607.15467#bib.bib3),[4](https://arxiv.org/html/2607.15467#bib.bib4),[5](https://arxiv.org/html/2607.15467#bib.bib5),[6](https://arxiv.org/html/2607.15467#bib.bib6)\], also enables*model extraction*wherein an adversary with black\-boxApplication Programming Interface\(API\) access systematically queries a deployed classifier, collects the returned probability vectors, and trains a student model that replicates the teacher’s behavior at a fraction of the original cost\[[7](https://arxiv.org/html/2607.15467#bib.bib7),[8](https://arxiv.org/html/2607.15467#bib.bib8)\]\. As recent events have highlighted, this threat is not just hypothetical\. DeepSeek’s 2025 release demonstrated that near\-frontier capability may be obtained largely through distillation from proprietary models\[[9](https://arxiv.org/html/2607.15467#bib.bib9)\]\. Additionally, Anthropic recently disclosed an industrial\-scale extraction campaign against Claude that produced over 16 million exchanges via approximately 24,000 fraudulent accounts\[[10](https://arxiv.org/html/2607.15467#bib.bib10)\]\. Beyond direct intellectual\-property loss\[[11](https://arxiv.org/html/2607.15467#bib.bib11),[12](https://arxiv.org/html/2607.15467#bib.bib12)\], extracted models facilitate downstream membership\-inference\[[13](https://arxiv.org/html/2607.15467#bib.bib13),[14](https://arxiv.org/html/2607.15467#bib.bib14)\], evasion\[[15](https://arxiv.org/html/2607.15467#bib.bib15),[16](https://arxiv.org/html/2607.15467#bib.bib16),[17](https://arxiv.org/html/2607.15467#bib.bib17)\], and privacy attacks against the underlying training distribution\[[18](https://arxiv.org/html/2607.15467#bib.bib18),[19](https://arxiv.org/html/2607.15467#bib.bib19),[20](https://arxiv.org/html/2607.15467#bib.bib20)\], raising the stakes especially in regulated domains such as medicine, and finance where machine learning applications are gaining traction\[[21](https://arxiv.org/html/2607.15467#bib.bib21),[22](https://arxiv.org/html/2607.15467#bib.bib22)\]\. Beyond these settings, the implications of model extraction also extend toCyber\-Physical Systems\(CPS\) as KD is increasingly used in resource\-constrained robotic platforms\[[23](https://arxiv.org/html/2607.15467#bib.bib23)\]\. Distillation\-based extraction may also expose safety\-critical control behavior in the physical world, raising security concerns forCPSdeployments\.

The attacker consumes the returned probability vector, also known as thesoft labels, in contrast tohard labels, where the output contains only the index of the predicted class\. The inter\-class structure of these soft labels, i\.e\., the relative probabilities of the classes other than the top class, constitutes thedark knowledgeof the given model\[[2](https://arxiv.org/html/2607.15467#bib.bib2)\]\. This dark knowledge makes soft\-label distillation markedly more effective than training on hard labels alone\. A naive defense against such distillation attempts would be to always return uninformative output, but that would make the deployed model useless for regular users\. The natural question is therefore how to corrupt the dark knowledge a model deployed through an API serves without destroying the utility of the given model for legitimate users\. Every inference\-time defense implicitly negotiates this trade\-off, and we argue the trade\-off rate should be measured against an explicit break\-even: a defense that removes onepercentage point\(pp\) of attacker accuracy perppof teacher accuracy, a 1:1 trade\-off, is no better than deploying a worse model\. A useful defense must beat the 1:1 trade\-off, ideally by a wide margin in favor of the model being defended\.

Antidistillation Sampling\(ADS\)\[[24](https://arxiv.org/html/2607.15467#bib.bib24)\]is a compelling starting point\. Proposed for autoregressive language models,ADSperturbs each next\-token distribution with an*input\-dependent, gradient\-directed*penalty, computed from a hidden proxy student, that estimates how much serving each token would help a distilling student\. Unlike static perturbation defenses, which apply a fixed transformation that an attacker can invert or meta\-learn\[[25](https://arxiv.org/html/2607.15467#bib.bib25),[26](https://arxiv.org/html/2607.15467#bib.bib26)\], the ADS perturbation is recomputed per query from hidden state\. Whether this idea transfers to classification, which is among the most broadly exposed extraction surfaces in practice, has remained open\. The question is not merely one of engineering: in a classification API the distilling student ingests the*entire*served vector through its KL objective, whereas LLM sampling collapses the perturbation onto a single emitted token, so per unit of perturbationADSshould be*more*potent in classification, not less\.

This paper adapts ADS to classification, analyzes the mechanism that governs its behavior there, and repairs its ineffectiveness with a composition rule whose utility cost is provably zero\. Our contributions are as follows\.

1. 1\.Adaptation\.We instantiate the ADS penalty in the classification setting, preserving its derivation, and identify the structural differences from the LLM setting \(Sec\.[4](https://arxiv.org/html/2607.15467#S4)\)\.
2. 2\.Diagnosis\.We show that the behavior of the transferred defense is determined by the distribution of per\-input*flip thresholds*, whose cumulative distribution, before any attack or defense is run, accurately predicts the defended teacher’s accuracy cost \(Sec\.[5](https://arxiv.org/html/2607.15467#S5)\)\. Classifier overconfidence\[[27](https://arxiv.org/html/2607.15467#bib.bib27)\]renders the defense inert at low strengths and unfavorable at high strengths; the perturbation itself is faithful and correctly aimed at dark knowledge, but*margin\-blind*\.
3. 3\.Analysis of temperature softening\.We show that softening the served distribution rescales the phase transition in closed form, and confirm the prediction in trained students: softening amplifies the perturbation at low strengths while collapsing the teacher at moderate ones, and no temperature configuration improves the trade\-off \(Sec\.[6](https://arxiv.org/html/2607.15467#S6)\)\.
4. 4\.ADS\-C\.We compose the ADS perturbation under a closed\-form, per\-input, margin\-aware budget: each input receives the largest penalty strength that provably keeps its served top\-1 prediction unchanged \(Proposition[1](https://arxiv.org/html/2607.15467#Thmproposition1)\), so the defended teacher’s accuracy equals the undefended teacher’s identically\. The surviving perturbation still degrades the distilled student substantially, by17\.4​pp17\.4\\,\\mathrm\{pp\}on CIFAR\-100,29\.6​pp29\.6\\,\\mathrm\{pp\}on CIFAR\-10, and13\.3​pp13\.3\\,\\mathrm\{pp\}on Tiny\-ImageNet at zero utility cost \(Secs\.[7](https://arxiv.org/html/2607.15467#S7)–[8](https://arxiv.org/html/2607.15467#S8)\)\. Against the complementary hard\-label attacker the defended soft output proves strictly worse as a distillation signal than the argmax it accompanies\. The soft\-label student falls17\.1​pp17\.1\\,\\mathrm\{pp\}and29\.7​pp29\.7\\,\\mathrm\{pp\}below what the hard\-label attacker achieves through distillation \(Sec\.[8\-E](https://arxiv.org/html/2607.15467#S8.SS5)\)\.
5. 5\.Relaxations of the guarantee\.We introduce two mechanisms with closed\-form\-predictable utility cost: first, stochastic enforcement of the budget; second, a minimum served strength\. Both trade a controlled amount of the guarantee for attacker uncertainty\. We price each relaxation against the attacker’s best response between soft\- and hard\-label extraction, and report a negative result on margin\-ranked sacrifice that further clarifies the damage mechanism \(Sec\.[9](https://arxiv.org/html/2607.15467#S9)\)\.
6. 6\.Evaluation methodology and reproducibility\.We argue that once utility cost is zero, defenses should be compared at matched served fidelity rather than matched penalty strength, and show that the two comparisons can disagree \(Sec\.[8\-D](https://arxiv.org/html/2607.15467#S8.SS4)\)\. We release a library and every experiment driver:[https://github\.com/the\-kas\-lab/ADS\-C](https://github.com/the-kas-lab/ADS-C)\.

## 2RELATED WORK

Distillation and dark knowledge\.Distillation trains a student on a teacher’s output distributions\[[1](https://arxiv.org/html/2607.15467#bib.bib1),[28](https://arxiv.org/html/2607.15467#bib.bib28),[29](https://arxiv.org/html/2607.15467#bib.bib29),[2](https://arxiv.org/html/2607.15467#bib.bib2)\]; its advantage over hard\-label training is attributed to the inter\-class structure of those distributions\[[2](https://arxiv.org/html/2607.15467#bib.bib2),[30](https://arxiv.org/html/2607.15467#bib.bib30),[31](https://arxiv.org/html/2607.15467#bib.bib31)\]\. Furlanello*et al\.*\[[30](https://arxiv.org/html/2607.15467#bib.bib30)\]showed diagnostically that destroying non\-target structure destroys most of distillation’s benefit, providing evidence that dark knowledge is the asset a defense should target\.

Model extraction\.Extraction attacks replicate a deployed model from query access\[[7](https://arxiv.org/html/2607.15467#bib.bib7),[32](https://arxiv.org/html/2607.15467#bib.bib32),[33](https://arxiv.org/html/2607.15467#bib.bib33),[34](https://arxiv.org/html/2607.15467#bib.bib34)\]; surveys\[[35](https://arxiv.org/html/2607.15467#bib.bib35),[8](https://arxiv.org/html/2607.15467#bib.bib8),[36](https://arxiv.org/html/2607.15467#bib.bib36),[37](https://arxiv.org/html/2607.15467#bib.bib37)\]document their practicality\. We consider the distillation\-based extraction attacker, who trains on returned soft labels which is the strongest and most common variant when probability vectors are exposed\.

Extraction defenses\.Training\-time defenses harden the teacher’s weights\[[38](https://arxiv.org/html/2607.15467#bib.bib38),[39](https://arxiv.org/html/2607.15467#bib.bib39),[40](https://arxiv.org/html/2607.15467#bib.bib40),[41](https://arxiv.org/html/2607.15467#bib.bib41)\]but require retraining and access to the original corpus, which is often undesirable or even infeasible at deployment scale\. Detection and monitoring\[[42](https://arxiv.org/html/2607.15467#bib.bib42),[43](https://arxiv.org/html/2607.15467#bib.bib43),[44](https://arxiv.org/html/2607.15467#bib.bib44)\]flag anomalous query streams, but rely on out\-of\-distribution assumptions that prove futile against attackers with in\-distribution data\[[41](https://arxiv.org/html/2607.15467#bib.bib41)\]and are circumvented by Sybil accounts\[[45](https://arxiv.org/html/2607.15467#bib.bib45),[46](https://arxiv.org/html/2607.15467#bib.bib46)\]\. Watermarking\[[47](https://arxiv.org/html/2607.15467#bib.bib47),[48](https://arxiv.org/html/2607.15467#bib.bib48),[49](https://arxiv.org/html/2607.15467#bib.bib49),[50](https://arxiv.org/html/2607.15467#bib.bib50)\]supports post\-hoc attribution but erects no barrier against the extraction itself\.*Perturbation*defenses are closest to our approach\. These defenses modify the served probabilities: Reverse Sigmoid\[[25](https://arxiv.org/html/2607.15467#bib.bib25)\], prediction poisoning\[[51](https://arxiv.org/html/2607.15467#bib.bib51)\], ModelGuard\[[52](https://arxiv.org/html/2607.15467#bib.bib52)\], noise\-transition learning\[[53](https://arxiv.org/html/2607.15467#bib.bib53)\], and information\-theoretic output compression\[[54](https://arxiv.org/html/2607.15467#bib.bib54)\]\. All of these pay for protection with utility, and the static members of the family being fixed functions of the input are invertible by meta\-learned recovery attacks such as D\-DAE\[[26](https://arxiv.org/html/2607.15467#bib.bib26)\]\. None offers a per\-query*guarantee*on the served top\-1 prediction\.

ADS\.Antidistillation Sampling\[[24](https://arxiv.org/html/2607.15467#bib.bib24)\]is input\-dependent and gradient\-directed wherein a hidden proxy student defines, per query, a direction in output space that maximally harms a distilling student\. It was derived and evaluated for LLM reasoning traces\. Two questions are open: whether it transfers to classification, and what it costs the defender there\. We answer both: the transfer’s behavior is gated entirely by classifier overconfidence, and the repair \(a margin budget at composition\) makes the cost exactly zero\.

## 3THREAT MODEL AND PROBLEM SETUP

Figure[1](https://arxiv.org/html/2607.15467#S3.F1)summarizes the setting\. A defender trains a classifier𝒯:𝒳→ΔK−1\\mathcal\{T\}\\colon\\mathcal\{X\}\\to\\Delta^\{K\-1\}mapping an input space𝒳\\mathcal\{X\}to the probability simplex overKKclasses and deploys it behind a query API that returns, for each query𝐱∈𝒳\\mathbf\{x\}\\in\\mathcal\{X\}, a served probability vector𝐩^​\(𝐱\)∈ΔK−1\\hat\{\\mathbf\{p\}\}\(\\mathbf\{x\}\)\\in\\Delta^\{K\-1\}\.

Adversary\.The adversary holds \(i\) black\-box access to the API with an arbitrary but bounded query budget; \(ii\) an unlabeled query corpus drawn from the same distribution𝒟\\mathcal\{D\}over𝒳\\mathcal\{X\}as the teacher’s training data—we make no out\-of\-distribution assumption, which disables detection\-style defenses by construction; and \(iii\) a student architecture𝒮\\mathcal\{S\}of its choosing\. The adversary lacks the teacher’s parameters, and ground\-truth labels\. With no labels, the attacker’s objective reduces from the standard hard\-plus\-soft KD loss to pure soft\-label matching,

ℒUKD=DK​L​\(𝐩^​\(𝐱\)∥𝐩𝒮​\(𝐱\)\),\\mathcal\{L\}\_\{\\mathrm\{UKD\}\}\\;=\\;D\_\{KL\}\\bigl\(\\hat\{\\mathbf\{p\}\}\(\\mathbf\{x\}\)\\,\\big\\\|\\,\\mathbf\{p\}\_\{\\mathcal\{S\}\}\(\\mathbf\{x\}\)\\bigr\),\(2\)evaluated at temperature11: the served vector is the attacker’s sole supervisory signal, and hence the natural locus of defense\.

Why the soft\-label attacker\.We scope the threat to the*dark\-knowledge distiller*, and this scoping is principled rather than convenient\. The attacker’s advantage over training on public hard labels is the served soft\-label structure; an attacker who reads only the served argmax gains nothing that a labeling function of equal top\-1 accuracy would not provide\. Our defense will, by construction, never alter the served argmax \(Sec\.[7](https://arxiv.org/html/2607.15467#S7)\), so hard\-label distillation against the defended API yields exactly what it would yield against the undefended API \(verified empirically in Sec\.[8\-E](https://arxiv.org/html/2607.15467#S8.SS5)\): the defense removes the soft\-label advantage\. The exploitation of the guarantee’s determinism itself is treated in Sec\.[9](https://arxiv.org/html/2607.15467#S9)\.

Defender\.The defender owns the teacher, a modest labeled holdout set, and a compact*proxy*student𝒫\\mathcal\{P\}that stands in for the unknown attacker student; it cannot retrain the teacher and must act at inference time only\. The defender’s goals are: \(a\)*utility*: legitimate users, who consume the top\-1 prediction, must be unharmed; \(b\)*distillation resistance*: a student trained via \([2](https://arxiv.org/html/2607.15467#S3.E2)\) on the served outputs should be substantially degraded\.

Metrics\.\(A summary of notation is provided in Appendix[11\-A](https://arxiv.org/html/2607.15467#Sx2.SS1)\.\) We report three quantities\.*Teacher top\-1*: accuracy of the served argmax\.*Student top\-1*: accuracy of the distilled student\.*Served fidelity*:DK​L​\(𝐩𝒯∥𝐩^\)D\_\{KL\}\(\\mathbf\{p\}\_\{\\mathcal\{T\}\}\\,\\\|\\,\\hat\{\\mathbf\{p\}\}\)averaged over the evaluation set, the total distributional perturbation the defense spends; this axis quantifies the defense’s “corruption budget” and becomes essential once utility cost is zero \(Sec\.[8\-D](https://arxiv.org/html/2607.15467#S8.SS4)\)\.

![Refer to caption](https://arxiv.org/html/2607.15467v1/x1.png)\(a\)The adversary constructs a labeled dataset by querying the victim API with unlabeled samples\.
![Refer to caption](https://arxiv.org/html/2607.15467v1/x2.png)\(b\)The adversary trains a surrogate student on the returned soft labels\.

Figure 1:The unauthorized knowledge\-distillation threat model\.
## 4ANTIDISTILLATION SAMPLING FOR CLASSIFICATION

### 4\-APreliminaries

A pretrained teacher with parametersθ𝒯\\theta\_\{\\mathcal\{T\}\}produces logits𝐳𝒯=𝒯​\(𝐱;θ𝒯\)∈ℝK\\mathbf\{z\}\_\{\\mathcal\{T\}\}=\\mathcal\{T\}\(\\mathbf\{x\};\\theta\_\{\\mathcal\{T\}\}\)\\in\\mathbb\{R\}^\{K\}; the softmaxσ​\(𝐳\)i=exp⁡\(zi\)/∑jexp⁡\(zj\)\\sigma\(\\mathbf\{z\}\)\_\{i\}=\\exp\(z\_\{i\}\)/\\sum\_\{j\}\\exp\(z\_\{j\}\)yields probabilities𝐩𝒯\\mathbf\{p\}\_\{\\mathcal\{T\}\}\. Only logit differences matter: adding a constant to𝐳\\mathbf\{z\}leavesσ​\(𝐳\)\\sigma\(\\mathbf\{z\}\)unchanged, so the marginzt−zjz\_\{t\}\-z\_\{j\}between the top classttand a rivaljjis the invariant quantity that decides the argmax\. Distillation trains a student to minimize \([2](https://arxiv.org/html/2607.15467#S3.E2)\) against the served distribution\.

### 4\-BThe ADS penalty

ADS asks, per query and per class:*if the student came to believe classyyis slightly more likely on this input, would that make the student worse?*Formally, letℓ​\(θ𝒫\)\\ell\(\\theta\_\{\\mathcal\{P\}\}\)be a downstream loss of the proxy on the defender’s labeled holdout\. We takeℓ​\(θ𝒫\)=1N​∑i=1NDK​L​\(𝐩𝒯​\(𝐱i\)∥𝐩𝒫​\(𝐱i\)\)\\ell\(\\theta\_\{\\mathcal\{P\}\}\)=\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}D\_\{KL\}\\bigl\(\\mathbf\{p\}\_\{\\mathcal\{T\}\}\(\\mathbf\{x\}\_\{i\}\)\\,\\\|\\,\\mathbf\{p\}\_\{\\mathcal\{P\}\}\(\\mathbf\{x\}\_\{i\}\)\\bigr\), the proxy’s mean disagreement with the teacher over the holdout, following\[[24](https://arxiv.org/html/2607.15467#bib.bib24)\], and letg=∇ℓ​\(θ𝒫\)g=\\nabla\\ell\(\\theta\_\{\\mathcal\{P\}\}\)be its gradient\. One SGD step of distillation on classyymovesθ𝒫\\theta\_\{\\mathcal\{P\}\}along∇θlog⁡p​\(y∣𝐱;θ𝒫\)\\nabla\_\{\\theta\}\\log p\(y\\mid\\mathbf\{x\};\\theta\_\{\\mathcal\{P\}\}\); the resulting change inℓ\\ellis, to first order, the inner product of that update with∇ℓ\\nabla\\ell\. Exactly as in the LLM derivation, symmetry of the directional derivative converts this into a finite difference over two*frozen clones*of the proxy,θ𝒫±=θ𝒫±ε​g\\theta\_\{\\mathcal\{P\}\\pm\}=\\theta\_\{\\mathcal\{P\}\}\\pm\\varepsilon g:

Δ^​\(y∣𝐱\)=log⁡p​\(y∣𝐱;θ𝒫\+\)−log⁡p​\(y∣𝐱;θ𝒫−\)2​ε\.\\widehat\{\\Delta\}\(y\\mid\\mathbf\{x\}\)\\;=\\;\\frac\{\\log p\(y\\mid\\mathbf\{x\};\\theta\_\{\\mathcal\{P\}\+\}\)\-\\log p\(y\\mid\\mathbf\{x\};\\theta\_\{\\mathcal\{P\}\-\}\)\}\{2\\varepsilon\}\.\(3\)A positiveΔ^​\(y∣𝐱\)\\widehat\{\\Delta\}\(y\\mid\\mathbf\{x\}\)means that serving mass on classyyraises the proxy’s holdout loss, which is good for the defender\. The defended distribution adds the penalty to the teacher’s log\-probabilities:

𝐩^\(⋅∣𝐱\)=σ\(log𝐩𝒯\(⋅∣𝐱\)\+λΔ^\(⋅∣𝐱\)\),\\hat\{\\mathbf\{p\}\}\(\\cdot\\mid\\mathbf\{x\}\)\\;=\\;\\sigma\\bigl\(\\log\\mathbf\{p\}\_\{\\mathcal\{T\}\}\(\\cdot\\mid\\mathbf\{x\}\)\+\\lambda\\,\\widehat\{\\Delta\}\(\\cdot\\mid\\mathbf\{x\}\)\\bigr\),\(4\)withλ≥0\\lambda\\geq 0the defense strength;λ=0\\lambda=0recovers the undefended API\.

Parameter choices\.\(i\) Definingℓ\\ellas a per\-sample mean fixes the scale ofggindependently of the holdout size; empirically∥g∥\\lVert g\\rVertis of order one \(1\.001\.00on CIFAR\-100,0\.770\.77on CIFAR\-10,1\.371\.37on Tiny\-ImageNet\), soλ\\lambdaoperates on a comparable scale across datasets\. We sweepλ∈\[0,3\]\\lambda\\in\[0,3\], extended toλ=10\\lambda=10for unmodified ADS to map the saturation regime \(Sec\.[5\-D](https://arxiv.org/html/2607.15467#S5.SS4)\)\. \(ii\) The stepε\\varepsilonis calibrated, not tuned: we selectε\\varepsilonto minimize the discrepancy between \([3](https://arxiv.org/html/2607.15467#S4.E3)\) and the exact Jacobian–vector product it approximates, obtainingε=3×10−3\\varepsilon=3\\times 10^\{\-3\}on both CIFAR datasets andε=10−2\\varepsilon=10^\{\-2\}on Tiny\-ImageNet, each with a magnitude ratio of≈1\.00\\approx\\\!1\.00\(Appendix[11\-B](https://arxiv.org/html/2607.15467#Sx2.SS2)\); the finite difference is a faithful estimate, which will matter for the diagnosis of Sec\.[5](https://arxiv.org/html/2607.15467#S5)\. \(iii\) The proxy \(ResNet\-20 on the CIFAR datasets, ResNet\-18 on Tiny\-ImageNet\) is substantially smaller than the teacher and is trained on the teacher’s training split, with a disjoint10%10\\%held out for computinggg; the two clones are built once, offline, and frozen, so a defended query costs two small forward passes and no gradient computation\. \(iv\) The margin floormmintroduced in Sec\.[7](https://arxiv.org/html/2607.15467#S7)is analyzed in Appendix[11\-C](https://arxiv.org/html/2607.15467#Sx2.SS3)\.

### 4\-CADS Case for Classification

The penalty \([3](https://arxiv.org/html/2607.15467#S4.E3)\) is structurally identical to the next\-token formulation of\[[24](https://arxiv.org/html/2607.15467#bib.bib24)\]; what differs is downstream consumption\. In autoregressive generation the perturbation tilts the next\-token distribution, but the attacker observes only a single sampled token per step: a one\-hot realization whose expectation carries the tilted distribution, so the signal on the non\-sampled coordinates ofΔ^\\widehat\{\\Delta\}reaches the student only in expectation, across many draws\. In classification distillation, the attacker observes the served vector itself: the KL objective \([2](https://arxiv.org/html/2607.15467#S3.E2)\) ingests every coordinate ofΔ^\\widehat\{\\Delta\}exactly, on every query\. The same perturbation is thus delivered through a noiseless channel rather than estimated through a stochastic one, and per unit of perturbation ADS should be*more*potent in classification, not less\. The next section shows that, despite this, the direct transfer is inert over a wide regime, then trades at an unfavorable rate, and finally saturates with both parties near their accuracy floors\. In addition, it locates the cause of this behavior not in the penalty but in its composition with the teacher\.

## 5DIAGNOSIS OF ADS INEFFECTIVENESS

Swept overλ\\lambda, the unmodified ADS exhibits three behaviors in sequence that make it undesirable for deployment in the classification domain\. Forλ≲0\.2\\lambda\\lesssim 0\.2it is*inert*: on both CIFAR datasets, student and teacher accuracy each change by at most11–2​pp2\\,\\mathrm\{pp\}, within seed noise\. The defense measurably affects neither party\. Beyond that, it becomes active at an unfavorable rate: on CIFAR\-100 atλ=0\.5\\lambda=0\.5, the distilled student drops5\.9​pp5\.9\\,\\mathrm\{pp\}while the teacher drops12\.2​pp12\.2\\,\\mathrm\{pp\}, a trade\-off rate of0\.480\.48, half of break\-even; on CIFAR\-10 the rate is0\.850\.85\(9\.6​pp9\.6\\,\\mathrm\{pp\}student for11\.3​pp11\.3\\,\\mathrm\{pp\}teacher\)\. Finally, forλ≳2\\lambda\\gtrsim 2the sweep*saturates*: the trade\-off remains sub\-linear while both models degrade toward their floors, and byλ=3\\lambda=3the defended teacher’s accuracy has fallen to the level of the very student it defends against \(Fig\.[9](https://arxiv.org/html/2607.15467#S8.F9), diagonal curves\)\. This section explains all three behaviors through a single mechanism and derives the design requirement for the remedy\.

### 5\-AOverconfidence of Classifiers

Modern, well\-trained networks are systematically overconfident\[[27](https://arxiv.org/html/2607.15467#bib.bib27),[55](https://arxiv.org/html/2607.15467#bib.bib55)\]: their top\-class probabilitypmaxp\_\{\\max\}concentrates near 1\. Table[1](https://arxiv.org/html/2607.15467#S5.T1)quantifies this for our teachers\. On CIFAR\-10 the median servedpmaxp\_\{\\max\}exceeds0\.999990\.99999\. The median non\-target mass, the entire surface on which dark knowledge lives and on which an additive penalty can visibly act, is below10−610^\{\-6\}\. CIFAR\-100 is less extreme \(median non\-target mass≈6×10−3\\approx 6\\times 10^\{\-3\}\), and Tiny\-ImageNet sits between the two \(4×10−44\\times 10^\{\-4\}\) despite having twice CIFAR\-100’s classes\. The comparison is deliberate: the Tiny\-ImageNet teacher is a strong ImageNet\-pretrained ResNet\-50 fine\-tune, whereas the CIFAR teachers are trained from scratch, and the pretrained model’s strength overrides the intuition that more classes produce more diffuse outputs\. We find that overconfidence is determined by model quality, not class count, and strong pretrained models are precisely what production APIs deploy, so the overconfident regime studied here is the realistic one rather than a small\-benchmark artifact\. \(At the extreme of output dimension, LLMs over\>\>50k\-token vocabularies operate in a naturally diffuse regime which is one reason the original ADS formulation could presuppose sufficient non\-target mass\.\)

The overconfidence is also*heterogeneous*, and this is the central observation\. The top\-logit marginzt−z\(2\)z\_\{t\}\-z\_\{\(2\)\}\(top minus runner\-up\) spans two orders of magnitude within a single dataset: on CIFAR\-100 its 10th/50th/90th percentiles are0\.70\.7/5\.55\.5/15\.015\.0log\-units; on Tiny\-ImageNet,1\.11\.1/8\.38\.3/17\.017\.0; on CIFAR\-10,3\.93\.9/14\.414\.4/21\.021\.0\. Most inputs have margins far too large for a moderate output perturbation to overcome, while a persistent minority —13\.4%13\.4\\%,9\.4%9\.4\\%, and2\.4%2\.4\\%of inputs, respectively, have margins below one log\-unit—lie close to a prediction flip\. Any defense that treats all inputs identically will therefore be simultaneously too weak on the head of this distribution and too strong on its tail\. To our knowledge, no prior antidistillation work has examined how this confidence structure interacts with an output\-perturbing defense; the remainder of this section shows the interaction is decisive\.

Table 1:Teacher confidence structure \(median over the evaluation set\)\.![Refer to caption](https://arxiv.org/html/2607.15467v1/x3.png)Figure 2:Overconfidence and its heterogeneity\. Per\-dataset distribution of the top\-logit marginz\(1\)−z\(2\)z\_\{\(1\)\}\-z\_\{\(2\)\}\(top class minus runner\-up\), drawn as a density \(above\) and a box \(below\) that captures the median, interquartile range, and p5–p95 whisker on a logarithmic axis as within every dataset the margin varies over orders of magnitude\.
### 5\-BSoundness of ADS Penalty

Before locating the futility at composition we rule out the penalty\. Two verifications \(Fig\.[3](https://arxiv.org/html/2607.15467#S5.F3)\):

*Faithfulness\.*At the calibratedε\\varepsilon, the finite\-differenceΔ^\\widehat\{\\Delta\}matches the exact directional derivative with magnitude ratio≈1\.00\\approx\\\!1\.00on both datasets \(Appendix[11\-B](https://arxiv.org/html/2607.15467#Sx2.SS2)\)\. The poison is not a numerical artifact\.

*Aim\.*Δ^\\widehat\{\\Delta\}concentrates exactly where dark knowledge lives\. Averaged over inputs, its value at the teacher’s top class is near zero \(mean−1\.1\-1\.1on CIFAR\-100,−0\.4\-0\.4on CIFAR\-10\) while deep non\-target ranks receive large positive pushes \(up to\+5\.6\+5\.6and\+12\.3\+12\.3respectively\); the class each input’s poison boosts hardest sits at teacher rank≥5\\geq 5for98%98\\%of CIFAR\-100 and63%63\\%of CIFAR\-10 inputs\. ADS corrupts the inter\-class structure beneath the prediction\.

![Refer to caption](https://arxiv.org/html/2607.15467v1/x4.png)Figure 3:Anatomy of the ADS penalty\. MeanΔ^\\widehat\{\\Delta\}as a function of the teacher’s class rank; near zero at rank 0, positive across the non\-target tail\. The penalty is aimed at dark knowledge, not at the argmax\.
### 5\-CMargin Blindness

Whether the served argmax survives \([4](https://arxiv.org/html/2607.15467#S4.E4)\) is a per\-input race between two numbers\. For each rivalj≠tj\\neq t, letcj=Δ^j−Δ^tc\_\{j\}=\\widehat\{\\Delta\}\_\{j\}\-\\widehat\{\\Delta\}\_\{t\}be the rate \(per unitλ\\lambda\) at which the poison closes the margin tojj; we define the poison’s*contrast*on input𝐱\\mathbf\{x\}as the largest such rate,

c​\(𝐱\)=maxj≠t⁡cj,c\(\\mathbf\{x\}\)\\;=\\;\\max\_\{j\\neq t\}\\;c\_\{j\},\(5\)the rate at which the strongest rival gains on the top classtt\. The served top\-1 flips exactly whenλ\\lambdaexceeds the*flip threshold*

λ∗​\(𝐱\)=minj≠t:cj\>0⁡zt−zjcj,\\lambda^\{\\ast\}\(\\mathbf\{x\}\)\\;=\\;\\min\_\{j\\neq t:\\,c\_\{j\}\>0\}\\;\\frac\{z\_\{t\}\-z\_\{j\}\}\{c\_\{j\}\},\(6\)withλ∗​\(𝐱\)=\+∞\\lambda^\{\\ast\}\(\\mathbf\{x\}\)=\+\\inftywhen no rival satisfiescj\>0c\_\{j\}\>0\. Two measurements now explain the unfavorable exchange\. First, the contrast is large: its median is17\.417\.4\(CIFAR\-100\) and16\.316\.3\(CIFAR\-10\) log\-units per unitλ\\lambda—comparable to the 90th\-percentile margin and25×25\\timesthe 10th\-percentile margin\. Second, and critically, the contrast is*margin\-blind*: its Spearman correlation with the margin isρ=\+0\.12\\rho=\+0\.12\(CIFAR\-100\) and\+0\.18\+0\.18\(CIFAR\-10\)\. The poison’s per\-input strength is unrelated to the margin it must respect\. Nothing in the objectiveℓ\\ellever sees the teacher’s margins, so nothing inΔ^\\widehat\{\\Delta\}respects them\.

The consequence follows directly from \([6](https://arxiv.org/html/2607.15467#S5.E6)\) and the thin tail of Fig\.[2](https://arxiv.org/html/2607.15467#S5.F2): asλ\\lambdagrows, served predictions flip in order of their margins, beginning with the thin\-margin inputs\. Atλ=0\.5\\lambda=0\.5, a global penalty has already flipped26\.1%26\.1\\%of CIFAR\-100 and14\.8%14\.8\\%of CIFAR\-10 served predictions \(Fig\.[4](https://arxiv.org/html/2607.15467#S5.F4)\)\. These flips constitute the entirety of the teacher’s cost, and they are an inefficient attack, for a reason the two parties’ metrics make precise\. The teacher’s cost is discontinuous in the perturbation: the moment the top two entries cross, a flip registers as a full served error\. The student’s supervision is the served vector, which is continuous: a flip just past its threshold crosses at near\-parity, so the training signal on that row is almost unchanged\. Moreover, low\-λ\\lambdaflips land exclusively on thin\-margin inputs, whose near\-uniform outputs carry little of the inter\-class structure a distilling student exploits \(Sec\.[9\-B](https://arxiv.org/html/2607.15467#S9.SS2)confirms this independently: corruption concentrated on thin\-margin rows barely moves the student\)\. The defender thus pays full price for perturbations the student scarcely notices—atλ=0\.5\\lambda=0\.5on CIFAR\-100 the flip component costs the teacher12\.2​pp12\.2\\,\\mathrm\{pp\}while accounting for only3\.1​pp3\.1\\,\\mathrm\{pp\}of student damage—whereas the argmax\-safe corruption underneath is what a distilling student cannot discount; we quantify this decomposition in Sec\.[8\-C](https://arxiv.org/html/2607.15467#S8.SS3)\.

![Refer to caption](https://arxiv.org/html/2607.15467v1/x5.png)\(a\)Margin\-blindness
![Refer to caption](https://arxiv.org/html/2607.15467v1/x6.png)\(b\)Flip cascade

Figure 4:Raw ADS penalty on both teachers\. \(a\) Median poison contrastc​\(𝐱\)c\(\\mathbf\{x\}\)against served margin, binned \(log–log; band: middle50%50\\%\), with thec​\(𝐱\)=c\(\\mathbf\{x\}\)=margin boundary at which a served prediction flips atλ=1\\lambda=1\. Each dataset’s contrast is*flat*in the margin \(Spearmanρ=\+0\.12\\rho=\+0\.12and\+0\.18\+0\.18\) and sits far*above*the boundary across almost the whole margin range\. The push is large \(median17\.417\.4/16\.316\.3log\-units\) and unrelated to the margin it must respect\. It meets the boundary only at the largest margins\. \(b\) The consequence of \([6](https://arxiv.org/html/2607.15467#S5.E6)\): asλ\\lambdagrows, served predictions flip in order of their margins, beginning with the thin\-margin tail; byλ=0\.5\\lambda=0\.5a global penalty has already flipped26\.1%26\.1\\%\(CIFAR\-100\) and14\.8%14\.8\\%\(CIFAR\-10\) of served predictions\.
### 5\-DThe inert window and the phase transition

In addition to explaining the flips, Equations \([5](https://arxiv.org/html/2607.15467#S5.E5)\)–\([6](https://arxiv.org/html/2607.15467#S5.E6)\) make the entire teacher\-cost curve predictable in closed form, before any student is trained\. Let

F​\(λ\)=Pr⁡\[λ∗​\(𝐱\)≤λ\]F\(\\lambda\)\\;=\\;\\Pr\\bigl\[\\lambda^\{\\ast\}\(\\mathbf\{x\}\)\\leq\\lambda\\bigr\]\(7\)be the cumulative distribution of the flip threshold over the evaluation set, computable from the teacher’s and the proxy’s outputs in a single forward pass\. The defended teacher’s accuracy drop under unmodified ADS is then the fraction of*correctly classified*inputs whose threshold has been crossed\. This prediction tracks the measured, trained teacher drop to within≈1\{\\approx\}1–2​pp2\\,\\mathrm\{pp\}at everyλ\\lambdaon both datasets \(CIFAR\-100 atλ=1\\lambda=1: predicted34\.934\.9vs\. measured33\.3​pp33\.3\\,\\mathrm\{pp\}; CIFAR\-10 atλ=2\\lambda=2:69\.869\.8vs\.69\.0​pp69\.0\\,\\mathrm\{pp\}\)\. The teacher cost of unmodified ADS is thus fully determined by the flip\-threshold distribution\.

FFinherits its shape from the margin heterogeneity of Sec\.[5\-A](https://arxiv.org/html/2607.15467#S5.SS1), and that shape organizes the sweep into three regimes \(Fig\.[5](https://arxiv.org/html/2607.15467#S5.F5)\):

- •Inert window\(λ\\lambdabelow the 5th\-percentile flip threshold:λ<0\.12\\lambda<0\.12on CIFAR\-100,λ<0\.30\\lambda<0\.30on CIFAR\-10\)\.F≈0F\\approx 0, and—because the same overconfidence leaves almost no non\-target mass for the poison to reweight—the*student*is unaffected as well: measured student and teacher effects are both within11–2​pp2\\,\\mathrm\{pp\}of the undefended anchors, indistinguishable from seed noise\. In this regime the defense has no measurable effect on either party\.
- •Phase transition\(0\.35≲λ≲10\.35\\lesssim\\lambda\\lesssim 1\)\.FFrises through its steep section; teacher cost and student damage both grow from noise level to tens of pp within about1\.51\.5octaves ofλ\\lambda—a narrow band of a control parameter that we sweep over four orders of magnitude\. We define the*transition threshold*λc\\lambda\_\{c\}as the largestλ\\lambdawithF​\(λ\)≤1%F\(\\lambda\)\\leq 1\\%\(λc≈0\.02\\lambda\_\{c\}\\approx 0\.02on CIFAR\-100,0\.090\.09on CIFAR\-10 by this strict definition; the teacher\-cost onset—drop exceeding1​pp1\\,\\mathrm\{pp\}—lies at0\.0660\.066and0\.1350\.135, respectively\)\.λ∗\\lambda^\{\\ast\}is approximately log\-normal, soFFis a smooth sigmoid inlog⁡λ\\log\\lambdawhose inflection lies at the median flip threshold \(0\.910\.91/1\.091\.09\)\.
- •Saturation\(λ≳2\\lambda\\gtrsim 2\)\. Both parties approach their floors and the sweep exhausts itself\. Per\-unit\-λ\\lambdastudent damage falls by two orders of magnitude \(20\.3​pp20\.3\\,\\mathrm\{pp\}nearλ=1\\lambda=1to0\.2​pp0\.2\\,\\mathrm\{pp\}atλ=10\\lambda=10on CIFAR\-100;33\.633\.6to0\.3​pp0\.3\\,\\mathrm\{pp\}on CIFAR\-10\) as the student completes its fall to—and slightly through—the chance floor \(0\.6%0\.6\\%vs\.1%1\\%chance;6\.3%6\.3\\%vs\.10%10\\%\): deeply poisoned vectors are systematically wrong, not merely uninformative\. The teacher’s tail is equally exhausted \(64\.0→70\.2​pp64\.0\\to 70\.2\\,\\mathrm\{pp\}and77\.6→86\.7​pp77\.6\\to 86\.7\\,\\mathrm\{pp\}of drop overλ=3→10\\lambda=3\\to 10\), and \([7](https://arxiv.org/html/2607.15467#S5.E7)\) predicts every measured tail point to within0\.6​pp0\.6\\,\\mathrm\{pp\}\. Byλ=3\\lambda=3the defended teacher’s accuracy has fallen to the level of the very student it defends against \(7\.2%7\.2\\%vs\.4\.2%4\.2\\%;15\.1%15\.1\\%vs\.12\.9%12\.9\\%\), and byλ=10\\lambda=10on CIFAR\-10*below*it \(6\.0%6\.0\\%served vs\.6\.3%6\.3\\%distilled\): the direct transfer does not merely trade unfavorably in this regime—it eliminates, and finally inverts, the accuracy advantage it exists to protect\.

Note the ordering: CIFAR\-10, the*more*overconfident teacher, has the*wider*inert window showing that inertness tracks overconfidence, as the mechanism predicts\. The same computation places Tiny\-ImageNet’s transition threshold atλc≈0\.06\\lambda\_\{c\}\\approx 0\.06, between the two CIFAR values, consistent with its intermediate overconfidence \(Table[1](https://arxiv.org/html/2607.15467#S5.T1)\); and its closed\-formF​\(λ\)F\(\\lambda\)predicts the measured teacher cost of the unmodified sweep to within2​pp2\\,\\mathrm\{pp\}at everyλ\\lambda\(e\.g\.,33\.2​pp33\.2\\,\\mathrm\{pp\}predicted versus31\.4​pp31\.4\\,\\mathrm\{pp\}measured atλ=2\\lambda=2\), validating the diagnosis on a third teacher whose confidence arises from ImageNet pretraining rather than from a narrow benchmark\.

One further property of \([7](https://arxiv.org/html/2607.15467#S5.E7)\) is used in the next section\. Serving the teacher at temperatureτ\\taudivides every logit gap byτ\\tauwhile leaving the poison contrast unchanged, so every flip threshold is divided by exactlyτ\\tau\. A threshold ofλ∗/τ\\lambda^\{\\ast\}/\\tauis crossed by a givenλ\\lambdaprecisely whenλ∗\\lambda^\{\\ast\}is crossed byτ​λ\\tau\\lambda, so the flipped fraction at strengthλ\\lambdaunder temperatureτ\\tauequals the unsoftened fraction atτ​λ\\tau\\lambda:

Fτ​\(λ\)=F​\(τ​λ\),F\_\{\\tau\}\(\\lambda\)\\;=\\;F\(\\tau\\lambda\),\(8\)a parameter\-free prediction \(Fig\.[6](https://arxiv.org/html/2607.15467#S6.F6)\): temperature shifts the entire transition curve toward smallerλ\\lambdaby the factor1/τ1/\\tau\. Softening does not alter the mechanism; it rescales theλ\\lambdaaxis on which the mechanism operates\.

![Refer to caption](https://arxiv.org/html/2607.15467v1/x7.png)\(a\)CIFAR\-100
![Refer to caption](https://arxiv.org/html/2607.15467v1/x8.png)\(b\)CIFAR\-10

Figure 5:The inert window and the phase transition \(log\-λ\\lambda\)\. Measured accuracy change of*both*parties under unmodified ADS across the three regimes shaded by the flip\-threshold CDF \([7](https://arxiv.org/html/2607.15467#S5.E7)\) \(inert:F≤5%F\\leq 5\\%; saturation:F≥80%F\\geq 80\\%, nearλ≈2\\lambda\\approx 2\)\. In the inert window \(λ<0\.12\\lambda<0\.12/0\.300\.30\) the served teacher and the distilled student are both flat within noise—the defense measurably affects neither party—after which both move together through the transition, the teacher falling at least as fast as the student, into saturation\.The design requirement is now explicit\. The penalty direction is sound; what is missing is margin\-awareness, and margin\-awareness cannot be injected through the poison pipeline\. The one free choice inside that pipeline is where the proxy sits, and we tested it: even a proxy distilled directly from the teacher’s served probabilities, corresponding to the state a distilling attacker’s student would be in during distillation rather than a proxy trained conventionally on labeled data, produces a poison whose contrast–margin correlation remainsρ≈0\\rho\\approx 0\(CIFAR\-100\)\. No placement of the proxy makesΔ^\\widehat\{\\Delta\}margin\-aware, because the objectiveℓ\\ellit derives from never sees a margin\. Margin\-awareness must therefore be enforced at composition, the only point where the margin and the poison are simultaneously visible\. That is ADS\-C \(Sec\.[7](https://arxiv.org/html/2607.15467#S7)\)\. First, however, we examine the natural alternative, whose failure the theory above already predicts and whose failure mode is itself informative\.

## 6TEMPERATURE SOFTENING

Table[1](https://arxiv.org/html/2607.15467#S5.T1)suggests an obvious remedy, that is, if overconfidence starves ADS of non\-target mass, soften the served distribution, with a constant distillation temperature\[[2](https://arxiv.org/html/2607.15467#bib.bib2),[27](https://arxiv.org/html/2607.15467#bib.bib27)\]or a confidence\-conditioned variant\[[56](https://arxiv.org/html/2607.15467#bib.bib56),[57](https://arxiv.org/html/2607.15467#bib.bib57),[58](https://arxiv.org/html/2607.15467#bib.bib58)\], to create dark knowledge for the poison to corrupt\. This intuition is partially correct, but what it yields is a demonstration of the mechanism rather than a viable defense\. Softening relaxes both of the constraints that teacher confidence imposes at once\. First, it inflates the non\-target mass visible to the attacker’s KL loss \(activating the poison\), and secondly, it shrinks every served margin by the same factor \(activating the flips, per \([8](https://arxiv.org/html/2607.15467#S5.E8)\)\)\. We serveσ​\(𝐳/T\)\\sigma\(\\mathbf\{z\}/T\)forT∈\{2,4\}T\\in\\\{2,4\\\}, alone and composed with the raw ADS penalty, overλ∈\[0,0\.5\]\\lambda\\in\[0,0\.5\], on both datasets with three seeds, extending the poison\-free sweep on CIFAR\-10 toT∈\{6,8,10,12\}T\\in\\\{6,8,10,12\\\}\. \(Throughout,τ\\tauin \([8](https://arxiv.org/html/2607.15467#S5.E8)\) andTTdenote the same served temperature; we writeTTwhen quoting experimental settings\.\) We report four findings\.

![Refer to caption](https://arxiv.org/html/2607.15467v1/x9.png)\(a\)CIFAR\-100
![Refer to caption](https://arxiv.org/html/2607.15467v1/x10.png)\(b\)CIFAR\-10

Figure 6:Temperature slides the transition curve \(log\-λ\\lambda\)\. Served flip fraction under temperatureτ\\tauisFτ​\(λ\)=F​\(τ​λ\)F\_\{\\tau\}\(\\lambda\)=F\(\\tau\\lambda\)\([8](https://arxiv.org/html/2607.15467#S5.E8)\): serving atτ\\taudivides every flip threshold byτ\\tau, shifting the entire transition curve toward smallerλ\\lambdaby the factor1/τ1/\\tau\(τ=2,4\\tau=2,4slideFFleft by2×2\\times,4×4\\times\)\. This is a parameter\-free prediction—no student is trained—and Secs\.[6\-B](https://arxiv.org/html/2607.15467#S6.SS2)–[6\-C](https://arxiv.org/html/2607.15467#S6.SS3)confirm it against the trained runs\.### 6\-ASoftening alone can improve the attacker’s student

Softened teacher outputs are the classic recipe for knowledge distillation\[[2](https://arxiv.org/html/2607.15467#bib.bib2)\]\. However, whether softening assists the adversary is governed by how much dark knowledge the teacher has to reveal\.

On CIFAR\-100, with no poison at all \(λ=0\\lambda=0\), serving atT=4T=4\(median served confidence0\.420\.42\)*improves*the distilled student by\+4\.7​pp\+4\.7\\,\\mathrm\{pp\}over the undefended baseline \(T=2T=2:\+0\.9​pp\+0\.9\\,\\mathrm\{pp\}\)\. On CIFAR\-10 the benefit is absent: pushing to higher temperatures only degrades the student, monotonically, from−1\.0​pp\-1\.0\\,\\mathrm\{pp\}atT=4T=4to−6\.5​pp\-6\.5\\,\\mathrm\{pp\}atT=12T=12\. AtT=10T=10, CIFAR\-10 reproduces CIFAR\-100’sT=4T=4operating point \(served median confidence0\.420\.42on both\), yet CIFAR\-100 gains\+4\.7​pp\+4\.7\\,\\mathrm\{pp\}where CIFAR\-10 loses−5\.2​pp\-5\.2\\,\\mathrm\{pp\}\. Tiny\-ImageNet interpolates between the two exactly as this account predicts: its non\-target mass is intermediate \(Table[1](https://arxiv.org/html/2607.15467#S5.T1)\), and so is the benefit gained by temperature scaling, that is,\+1\.6​pp\+1\.6\\,\\mathrm\{pp\}atT=4T=4, which lies between CIFAR\-10’s absent benefit and CIFAR\-100’s\+4\.7​pp\+4\.7\\,\\mathrm\{pp\}\. At matched served confidence the sign of the effect is therefore set by how much usable non\-target structure the teacher has to reveal, not by the magnitude of the temperature parameter or by the class count\. Wherever dark knowledge exists, revealing it assists the adversary; where it does not, softening only dilutes the supervision the attacker already had\. Note that constant\-TTscaling preserves the argmax, so this assistance is not even purchased with teacher accuracy: its only cost is served fidelity, which is the axis on which Sec\.[6\-D](https://arxiv.org/html/2607.15467#S6.SS4)prices every softening variant\.

### 6\-BSoftening activates ADS inside the inert window

Composed with the poison, softening behaves exactly as \([8](https://arxiv.org/html/2607.15467#S5.E8)\) predicts: it shifts the entire transition curve into the formerly inert window\. Atλ=0\.2\\lambda=0\.2—inert under plain serving—the poison’s marginal effect on the student \(measured relative to the same serving atλ=0\\lambda=0, which nets out the shift of Sec\.[6\-A](https://arxiv.org/html/2607.15467#S6.SS1)\) grows from−1\.1​pp\-1\.1\\,\\mathrm\{pp\}\(unsoftened\) to−3\.0​pp\-3\.0\\,\\mathrm\{pp\}\(T=2T=2\) to−17\.8​pp\-17\.8\\,\\mathrm\{pp\}\(T=4T=4\) on CIFAR\-100, and from−0\.4\-0\.4to−5\.8\-5\.8to−27\.7​pp\-27\.7\\,\\mathrm\{pp\}on CIFAR\-10 \(Fig\.[7](https://arxiv.org/html/2607.15467#S6.F7)a\): the same smallλ\\lambdabecomes17×17\\times–69×69\\timesmore potent\. This provides direct causal evidence that the availability of non\-target mass, not the penalty strength, is what limits the poison at lowλ\\lambda\. CIFAR\-10 sharpens the point: the sameT=4T=4serving that conferred no benefit on a clean student \(Sec\.[6\-A](https://arxiv.org/html/2607.15467#S6.SS1)\) multiplies the poison’s potency69×69\\times—what softening manufactures is probability mass for the poison to reweight, whether or not that mass encodes anything a student could exploit\. The teacher\-side cost agrees with the theory quantitatively: the pre\-registered predictionF​\(τ​λ\)F\(\\tau\\lambda\)placed theT=4T=4,λ=0\.2\\lambda=0\.2teacher cost at26\.9​pp26\.9\\,\\mathrm\{pp\}\(CIFAR\-100\) and30\.1​pp30\.1\\,\\mathrm\{pp\}\(CIFAR\-10\); the trained runs measured25\.125\.1and29\.1​pp29\.1\\,\\mathrm\{pp\}\.

### 6\-CThe same rescaling collapses the teacher

The teacher\-side cost just quoted is the other half of the same mechanism: softening activates the poison by shrinking the very margins the poison then crosses, so enablement and collapse are one phenomenon, not two\. On the student\-versus\-teacher\-cost plane \(Fig\.[7](https://arxiv.org/html/2607.15467#S6.F7)b\), every constant\-temperature variant \(T=1T=1,22,44, on both datasets\) collapses onto a single trade\-off curve\. Temperature never improves the exchange rate rather it only moves the operating point further along it\. A worked example makes the severity concrete\. A typical softened CIFAR\-100 row has served margin0\.380\.38against poison contrast16\.816\.8, so its prediction flips atλ∗≈0\.022\\lambda^\{\\ast\}\\approx 0\.022; measured end\-to\-end,T=4T=4serving loses25\.125\.1/29\.1​pp29\.1\\,\\mathrm\{pp\}of teacher accuracy \(CIFAR\-100/CIFAR\-10\) atλ=0\.2\\lambda=0\.2, a strength that is inert under plain serving, and45\.045\.0/56\.3​pp56\.3\\,\\mathrm\{pp\}atλ=0\.35\\lambda=0\.35\(Fig\.[7](https://arxiv.org/html/2607.15467#S6.F7)\)\. Softening manufactures precisely the thin margins on which a margin\-blind poison is most destructive because the two mechanisms compete for the same resource, since the softener consumes the margin the poison needs in order to act safely\.

### 6\-DAt matched served fidelity, softening is dominated

Constant temperature is the bluntest possible softener, so its failure does not by itself close the question\. We therefore construct the strongest softening variant we could: a*gated*operator, inspired by\[[59](https://arxiv.org/html/2607.15467#bib.bib59)\], that lowers the confidence of over\-confident inputs only, and does so by lowering the top logit alone, leaving all non\-target structure intact for the dark\-knowledge channel \(Appendix[11\-D](https://arxiv.org/html/2607.15467#Sx2.SS4)\)\. It is composed with the margin budget of Sec\.[7](https://arxiv.org/html/2607.15467#S7), so that its teacher cost is identically zero\. Section[8\-D](https://arxiv.org/html/2607.15467#S8.SS4)evaluates this combination across five gates and the fullλ\\lambdagrid against poison\-only ADS\-C on the axis a defender actually spends, served fidelity: the softener pays a large fidelity cost*up front*, before any poison flows, and at every matched fidelity level in the operating regime poison\-only ADS\-C damages the student more \(Fig\.[11](https://arxiv.org/html/2607.15467#S8.F11)\)\. We therefore include no softening in the proposed defense: softening serves in this paper as the probe that establishes dark knowledge as the resource the poison consumes, and as a defense component it is dominated\.

![Refer to caption](https://arxiv.org/html/2607.15467v1/x11.png)\(a\)CIFAR\-100
![Refer to caption](https://arxiv.org/html/2607.15467v1/x12.png)\(b\)CIFAR\-10

Figure 7:Constant\-temperature serving±\\pmADS on the student\-versus\-teacher\-cost trade\-off plane \(E\-CT; 3 seeds\)\. Every constant\-temperature arm \(T=1,2,4T\{=\}1,2,4\) collapses onto a*single*unfavorable curve: softening activates the poison—atλ=0\.2\\lambda=0\.2theT=4T\{=\}4poison is17×17\\times/69×69\\timesmore potent than unsoftened, in quantitative agreement with the pre\-registeredF​\(τ​λ\)F\(\\tau\\lambda\)prediction—but only by shrinking the margins the poison then crosses, so the extra student damage is bought at proportional teacher cost\. ADS\-C \(vertical line, teacher cost≡0\\equiv 0by construction\) reaches student damage no temperature configuration attains without spending the teacher accuracy the softener pays up front\.

## 7ADS\-C

The diagnosis of Sec\.[5](https://arxiv.org/html/2607.15467#S5)dictates the design requirement: retain the poison, but enforce margin\-awareness at composition, per input\. ADS\-C implements this requirement in closed form\.

### 7\-AThe margin budget

Fix an input𝐱\\mathbf\{x\}with teacher logits𝐳\\mathbf\{z\}, top classt=arg⁡maxk⁡zkt=\\arg\\max\_\{k\}z\_\{k\}, and poisonΔ^\\widehat\{\\Delta\}\. For every rivalj≠tj\\neq t, letgapj=zt−zj\\mathrm\{gap\}\_\{j\}=z\_\{t\}\-z\_\{j\}denote the served margin to that rival and recall from \([5](https://arxiv.org/html/2607.15467#S5.E5)\) the ratecj=Δ^j−Δ^tc\_\{j\}=\\widehat\{\\Delta\}\_\{j\}\-\\widehat\{\\Delta\}\_\{t\}at which the poison closes it\. Given the requested strengthλ\\lambdaand a margin floorm\>0m\>0, the per\-input budget is

λ​\(𝐱\)=clip⁡\(minj:cj\>0⁡gapj−mcj,0,λ\),\\lambda\(\\mathbf\{x\}\)\\;=\\;\\operatorname\{clip\}\\Bigl\(\\;\\min\_\{j:\\,c\_\{j\}\>0\}\\;\\frac\{\\mathrm\{gap\}\_\{j\}\-m\}\{c\_\{j\}\}\\;,\\;0,\\;\\lambda\\Bigr\),\(9\)with the convention that a minimum over an empty set is\+∞\+\\infty, and the API serves𝐩^=σ​\(log⁡𝐩𝒯\+λ​\(𝐱\)​Δ^\)\\hat\{\\mathbf\{p\}\}=\\sigma\\bigl\(\\log\\mathbf\{p\}\_\{\\mathcal\{T\}\}\+\\lambda\(\\mathbf\{x\}\)\\,\\widehat\{\\Delta\}\\bigr\)\(Algorithm[1](https://arxiv.org/html/2607.15467#alg1)\)\. The budget is the flip threshold \([6](https://arxiv.org/html/2607.15467#S5.E6)\) computed withmmlog\-units of every margin held in reserve, clipped to the feasible range, and its behavior at the extremes is the intended one: rivals the poison pushes down \(cj≤0c\_\{j\}\\leq 0\) never bind, and an input none of whose rivals gain passes the full requested strength\. Conversely, an input whose margin is already at or below the floor receives no poison rather than a flip\. Figure[8](https://arxiv.org/html/2607.15467#S7.F8)traces one query through the full pipeline\.

Algorithm 1ADS\-C1:query

𝐱\\mathbf\{x\}; teacher

𝒯\\mathcal\{T\}; frozen proxy clones

𝒫\+,𝒫−\\mathcal\{P\}\_\{\+\},\\mathcal\{P\}\_\{\-\}
2:step

ε\\varepsilon; requested strength

λ\\lambda; margin floor

mm
3:served probability vector

𝐩^\\hat\{\\mathbf\{p\}\}
4:

𝐳←𝒯​\(𝐱\)\\mathbf\{z\}\\leftarrow\\mathcal\{T\}\(\\mathbf\{x\}\);

t←arg⁡maxk⁡zkt\\leftarrow\\arg\\max\_\{k\}z\_\{k\}
5:

Δ^←\[log⁡σ​\(𝒫\+​\(𝐱\)\)−log⁡σ​\(𝒫−​\(𝐱\)\)\]/\(2​ε\)\\widehat\{\\Delta\}\\leftarrow\\bigl\[\\log\\sigma\(\\mathcal\{P\}\_\{\+\}\(\\mathbf\{x\}\)\)\-\\log\\sigma\(\\mathcal\{P\}\_\{\-\}\(\\mathbf\{x\}\)\)\\bigr\]/\(2\\varepsilon\)
6:for

j≠tj\\neq tdo

7:

gapj←zt−zj\\mathrm\{gap\}\_\{j\}\\leftarrow z\_\{t\}\-z\_\{j\};

cj←Δ^j−Δ^tc\_\{j\}\\leftarrow\\widehat\{\\Delta\}\_\{j\}\-\\widehat\{\\Delta\}\_\{t\}
8:endfor

9:

λ​\(𝐱\)←clip⁡\(minj:cj\>0⁡\(gapj−m\)/cj,0,λ\)\\lambda\(\\mathbf\{x\}\)\\leftarrow\\operatorname\{clip\}\\bigl\(\\min\_\{j:c\_\{j\}\>0\}\(\\mathrm\{gap\}\_\{j\}\-m\)/c\_\{j\},\\;0,\\;\\lambda\\bigr\)
10:return

𝐩^←σ​\(log⁡σ​\(𝐳\)\+λ​\(𝐱\)​Δ^\)\\hat\{\\mathbf\{p\}\}\\leftarrow\\sigma\\bigl\(\\log\\sigma\(\\mathbf\{z\}\)\+\\lambda\(\\mathbf\{x\}\)\\,\\widehat\{\\Delta\}\\bigr\)

### 7\-BThe guarantee

###### Proposition 1\(Zero\-cost argmax preservation\)

For every input𝐱\\mathbf\{x\}with unique teacher argmaxtt, the served vector of Algorithm[1](https://arxiv.org/html/2607.15467#alg1)satisfies, for allj≠tj\\neq t,

log⁡p^t−log⁡p^j≥min⁡\(m,gapj\)\>0\.\\log\\hat\{p\}\_\{t\}\-\\log\\hat\{p\}\_\{j\}\\;\\geq\\;\\min\\bigl\(m,\\;\\mathrm\{gap\}\_\{j\}\\bigr\)\\;\>\\;0\.\(10\)Consequentlyarg⁡maxk⁡p^k=arg⁡maxk⁡p𝒯,k\\arg\\max\_\{k\}\\hat\{p\}\_\{k\}=\\arg\\max\_\{k\}p\_\{\\mathcal\{T\},k\}for every query, and the defended teacher’s top\-1 accuracy equals the undefended teacher’s top\-1 accuracy identically—as a deterministic, per\-query property, not in expectation\. Moreover,λ​\(𝐱\)\\lambda\(\\mathbf\{x\}\)is maximal: it is the largest value in\[0,λ\]\[0,\\lambda\]for which the stated margin bound holds\.

###### Proof 7\.1\.

Softmax composition preserves log\-score differences, solog⁡p^t−log⁡p^j=gapj−λ​\(𝐱\)​cj\\log\\hat\{p\}\_\{t\}\-\\log\\hat\{p\}\_\{j\}=\\mathrm\{gap\}\_\{j\}\-\\lambda\(\\mathbf\{x\}\)\\,c\_\{j\}\. Ifcj≤0c\_\{j\}\\leq 0this is≥gapj\\geq\\mathrm\{gap\}\_\{j\}\. Ifcj\>0c\_\{j\}\>0, then by \([9](https://arxiv.org/html/2607.15467#S7.E9)\)λ​\(𝐱\)≤\(gapj−m\)/cj\\lambda\(\\mathbf\{x\}\)\\leq\(\\mathrm\{gap\}\_\{j\}\-m\)/c\_\{j\}whenever that bound is nonnegative, givinggapj−λ​\(𝐱\)​cj≥m\\mathrm\{gap\}\_\{j\}\-\\lambda\(\\mathbf\{x\}\)c\_\{j\}\\geq m; when the bound is negative \(i\.e\.,gapj<m\\mathrm\{gap\}\_\{j\}<m\), the clip yieldsλ​\(𝐱\)=0\\lambda\(\\mathbf\{x\}\)=0and the difference equalsgapj\\mathrm\{gap\}\_\{j\}\. In all cases the difference is≥min⁡\(m,gapj\)\>0\\geq\\min\(m,\\mathrm\{gap\}\_\{j\}\)\>0, sottremains the served argmax\. Maximality: constraints withcj≤0c\_\{j\}\\leq 0hold for everyλ′≥0\\lambda^\{\\prime\}\\geq 0; forcj\>0c\_\{j\}\>0, the constraintgapj−λ′​cj≥min⁡\(m,gapj\)\\mathrm\{gap\}\_\{j\}\-\\lambda^\{\\prime\}c\_\{j\}\\geq\\min\(m,\\mathrm\{gap\}\_\{j\}\)readsλ′≤\(gapj−m\)/cj\\lambda^\{\\prime\}\\leq\(\\mathrm\{gap\}\_\{j\}\-m\)/c\_\{j\}whengapj≥m\\mathrm\{gap\}\_\{j\}\\geq mandλ′≤0\\lambda^\{\\prime\}\\leq 0whengapj<m\\mathrm\{gap\}\_\{j\}<m\. The feasible subset of\[0,λ\]\[0,\\lambda\]is therefore the interval\[0,clip⁡\(minj:cj\>0⁡\(gapj−m\)/cj,0,λ\)\]\\bigl\[0,\\;\\operatorname\{clip\}\\bigl\(\\min\_\{j:c\_\{j\}\>0\}\(\\mathrm\{gap\}\_\{j\}\-m\)/c\_\{j\},\\,0,\\,\\lambda\\bigr\)\\bigr\], whose right endpoint is exactly \([9](https://arxiv.org/html/2607.15467#S7.E9)\)\.

### 7\-CDesign discussion

The floormm\.The floor is the service level the defender guarantees on served margins: every served prediction retains at leastmin⁡\(m,original margin\)\\min\(m,\\text\{original margin\}\)log\-units of separation\. We usem=0\.05m=0\.05throughout; the guarantee itself ismm\-independent \(teacher drop is zero for everymm\), and Appendix[11\-C](https://arxiv.org/html/2607.15467#Sx2.SS3)shows the poison passed through the budget is insensitive overm∈\[0\.01,0\.2\]m\\in\[0\.01,0\.2\]\(served poison\-KL varies by≈10%\{\\approx\}10\\%across the range\)\.

Overconfidence cuts both ways\.The same heterogeneity that made the global penalty fatal makes the budget cheap: most inputs sit far from the floor, so most of the requested strength survives\. Atλ=0\.5\\lambda=0\.5the mean effective strength is0\.440\.44\(CIFAR\-100\) and0\.470\.47\(CIFAR\-10\), corresponding to8888–95%95\\%of the request, while the thin\-margin tail, which a globalλ\\lambdawould flip first, is individually protected\. The budget spends the poison precisely where the confidence structure of Fig\.[2](https://arxiv.org/html/2607.15467#S5.F2)has room for it\.

Deployment properties\.ADS\-C is inference\-time only\. Per query, beyond the teacher’s forward pass, it costs two forward passes through the frozen proxy clones, each substantially smaller than the teacher, andO​\(K\)O\(K\)arithmetic for \([9](https://arxiv.org/html/2607.15467#S7.E9)\)\. The served output remains a valid probability vector with unchanged argmax\. Like ADS, and unlike static perturbation defenses\[[25](https://arxiv.org/html/2607.15467#bib.bib25),[26](https://arxiv.org/html/2607.15467#bib.bib26)\], the perturbation is input\-dependent and computed from state the attacker cannot observe \(the proxy and its gradientgg\), so there is no fixed transformation to invert\. Additionally, unlike ADS, its utility cost is identically zero\.

![Refer to caption](https://arxiv.org/html/2607.15467v1/x13.png)Figure 8:One CIFAR\-10 query through the pipeline \(panels 1–4\)\. The teacher serves a confident vector \(panel 1; truth*dog*,pmax=0\.92p\_\{\\max\}\{=\}0\.92\); the poisonΔ^\\widehat\{\\Delta\}leaves the top class essentially untouched \(Δ^t=−0\.3\\widehat\{\\Delta\}\_\{t\}\{=\}\{\-0\.3\}\) while pushing every rival up, hardest on deep\-ranked classes \(panel 2\); a globalλ=0\.5\\lambda=0\.5flips the served argmax to the hard\-pushed*ship*\(panel 3, teacher hurt\), while the margin budget \([9](https://arxiv.org/html/2607.15467#S7.E9)\) spendsλ​\(𝐱\)=0\.49\\lambda\(\\mathbf\{x\}\)=0\.49—just short of the flip—leaving the top\-1 intact and the dark knowledge corrupted \(panel 4\)\. Pushes surface in the served vector only where the teacher leaves probability mass \(*ship*,*cat*\); equally large pushes on near\-massless classes remain invisible at this strength—the starvation of Sec\.[5\-A](https://arxiv.org/html/2607.15467#S5.SS1)in miniature\.

## 8EXPERIMENTS

### 8\-ASetup

Benchmarks\.CIFAR\-100 \(teacher ResNet\-110→\\tostudent ResNet\-20\), CIFAR\-10 \(ResNet\-56→\\toResNet\-20\), and Tiny\-ImageNet \(K=200K=200,64×6464\\times 64; teacher ResNet\-50, ImageNet\-pretrained and fine\-tuned with a small\-input stem→\\tostudent ResNet\-18\)\. The proxy always matches the student architecture; Appendix[11\-E](https://arxiv.org/html/2607.15467#Sx2.SS5)reports training recipes\. Undefended anchors: CIFAR\-100 teacher71\.23%71\.23\\%, distilled student47\.34±0\.37%47\.34\\pm 0\.37\\%; CIFAR\-10 teacher92\.70%92\.70\\%, student81\.44±0\.43%81\.44\\pm 0\.43\\%; Tiny\-ImageNet teacher80\.07%80\.07\\%, student47\.84±0\.59%47\.84\\pm 0\.59\\%\.

Attacker\.The fixed distillation attacker of Sec\.[3](https://arxiv.org/html/2607.15467#S3): it queries the API once per image with the full unlabeled training split \(50 00050\\,000queries on the CIFAR datasets,100 000100\\,000on Tiny\-ImageNet\), unaugmented and at native resolution, then trains the student for 50 epochs on the returned \(image, served\-vector\) pairs with the pure soft\-label loss \([2](https://arxiv.org/html/2607.15467#S3.E2)\) at temperature 1 \(full optimizer settings in Appendix[11\-E](https://arxiv.org/html/2607.15467#Sx2.SS5)\)\. Every configuration is run over three seeds \(\{92,786,1337\}\\\{92,786,1337\\\}\); we report mean±\\pmstd\.

Defense grid\.λ∈\{0,0\.1,0\.2,0\.35,0\.5,0\.75,1,1\.5,2,3\}\\lambda\\in\\\{0,0\.1,0\.2,0\.35,0\.5,0\.75,1,1\.5,2,3\\\}; margin floorm=0\.05m=0\.05;ε=3×10−3\\varepsilon=3\\times 10^\{\-3\}\(CIFAR\) and10−210^\{\-2\}\(Tiny\-ImageNet; Appendix[11\-B](https://arxiv.org/html/2607.15467#Sx2.SS2)\)\.*ADS*denotes the unmodified transfer \([4](https://arxiv.org/html/2607.15467#S4.E4)\);*ADS\-C*denotes Algorithm[1](https://arxiv.org/html/2607.15467#alg1);*ADS\-C \+ softener*denotes the ablation of Sec\.[8\-D](https://arxiv.org/html/2607.15467#S8.SS4); the guarantee relaxationsqqandλmin\\lambda\_\{\\min\}\(Sec\.[9](https://arxiv.org/html/2607.15467#S9)\) default toq=1q=1,λmin=0\\lambda\_\{\\min\}=0\.

### 8\-BMain result: the trade\-off rate before and after budgeting

Figure[9](https://arxiv.org/html/2607.15467#S8.F9)plots student accuracy against teacher accuracy for both methods on all three benchmarks; Table[2](https://arxiv.org/html/2607.15467#S8.T2)reports representative operating points\.

Unmodified ADS\.The direct transfer degrades the student only by degrading the teacher comparably or faster\. On CIFAR\-100 the sweep passes through\(−12\.2​pp​teacher,−5\.9​pp​student\)\(\-12\.2\\,\\mathrm\{pp\}\\text\{ teacher\},\-5\.9\\,\\mathrm\{pp\}\\text\{ student\}\)atλ=0\.5\\lambda=0\.5and ends at\(−64\.0​pp,−43\.1​pp\)\(\-64\.0\\,\\mathrm\{pp\},\-43\.1\\,\\mathrm\{pp\}\)atλ=3\\lambda=3\. The teacher suffers more than the student\. Tiny\-ImageNet exhibits the same shape shifted by its wider inert window \(Sec\.[5\-D](https://arxiv.org/html/2607.15467#S5.SS4)\)\. Its exchange is still near\-inert atλ=0\.5\\lambda=0\.5\(−1\.7​pp\-1\.7\\,\\mathrm\{pp\}teacher,0\.0​pp0\.0\\,\\mathrm\{pp\}student\), first becomes active atλ=1\\lambda=1\(−9\.0​pp\-9\.0\\,\\mathrm\{pp\}teacher for−4\.6​pp\-4\.6\\,\\mathrm\{pp\}student, rate0\.510\.51\), and reaches\(−46\.7​pp,−28\.8​pp\)\(\-46\.7\\,\\mathrm\{pp\},\-28\.8\\,\\mathrm\{pp\}\)atλ=3\\lambda=3\.

ADS\-C\.With the budget active, teacher top\-1 drop is0\.00​pp0\.00\\,\\mathrm\{pp\}at every cell, while the student falls monotonically withλ\\lambda\. It falls to29\.95±0\.23%29\.95\\pm 0\.23\\%on CIFAR\-100 \(−17\.4​pp\-17\.4\\,\\mathrm\{pp\},37%37\\%relative\),51\.80±0\.38%51\.80\\pm 0\.38\\%on CIFAR\-10 \(−29\.6​pp\-29\.6\\,\\mathrm\{pp\},36%36\\%relative\), and34\.51±0\.28%34\.51\\pm 0\.28\\%on Tiny\-ImageNet \(−13\.3​pp\-13\.3\\,\\mathrm\{pp\},28%28\\%relative\) atλ=3\\lambda=3\.

Matched\-damage comparison\.For unmodified ADS to inflict the student damage that ADS\-C delivers at zero cost, it must spend27\.5​pp27\.5\\,\\mathrm\{pp\}\(CIFAR\-100\),32\.9​pp32\.9\\,\\mathrm\{pp\}\(CIFAR\-10\), and22\.2​pp22\.2\\,\\mathrm\{pp\}\(Tiny\-ImageNet\) of teacher accuracy\. The margin budget converts an unfavorable trade into a free one\.

![Refer to caption](https://arxiv.org/html/2607.15467v1/x14.png)\(a\)CIFAR\-100
![Refer to caption](https://arxiv.org/html/2607.15467v1/x15.png)\(b\)CIFAR\-10
![Refer to caption](https://arxiv.org/html/2607.15467v1/x16.png)\(c\)Tiny\-ImageNet

Figure 9:The main result: distilled\-student top\-1 versus served\-teacher top\-1 \(horizontal axis inverted—the teacher degrades left to right; 3 seeds; the sweptλ\\lambdais annotated along each curve, with ADS extended toλ=10\\lambda=10on the CIFAR panels; dashed gray: the 1:1 break\-even through the undefended anchor⋆\\star\)\. Unmodified ADS traces a diagonal: it degrades the student only by degrading the teacher comparably or faster\. ADS\-C is the vertical segment at zero teacher cost \(Proposition[1](https://arxiv.org/html/2607.15467#Thmproposition1)\)\. The horizontal arrow marks the teacher accuracy unmodified ADS must spend to match ADS\-C’s free student damage atλ=3\\lambda=3\.Table 2:Representative operating points \(mean±\\pmstd over 3 seeds\)\. Teacher drop for ADS\-C is exactly0by Proposition[1](https://arxiv.org/html/2607.15467#Thmproposition1)\(observed≤1\.4×10−6​pp\\leq 1\.4\\times 10^\{\-6\}\\,\\mathrm\{pp\}\)\.
### 8\-CDecomposition of the unbudgeted poison’s damage

The budget also functions as an analytical instrument\. It separates the unbudgeted poison’s student damage into an argmax\-safe component, which survives the budget, and flip collateral, which the budget removes \(Fig\.[10](https://arxiv.org/html/2607.15467#S8.F10)\)\. At matchedλ=0\.5\\lambda=0\.5on CIFAR\-100, the unbudgeted poison’s5\.9​pp5\.9\\,\\mathrm\{pp\}student drop decomposes into2\.8​pp2\.8\\,\\mathrm\{pp\}of argmax\-safe corruption \(retained at zero teacher cost\) and3\.1​pp3\.1\\,\\mathrm\{pp\}of flip collateral \(purchased at12\.2​pp12\.2\\,\\mathrm\{pp\}of teacher accuracy\); on CIFAR\-10,5\.0​pp5\.0\\,\\mathrm\{pp\}of9\.6​pp9\.6\\,\\mathrm\{pp\}survives \(52%52\\%\)\. At smaller strengths the argmax\-safe share is larger still \(87%87\\%atλ=0\.2\\lambda=0\.2on CIFAR\-100\)\. Two conclusions follow\. First, roughly half of the student damage that unmodified ADS purchased with teacher accuracy was genuine dark\-knowledge corruption that required no payment at all\. Second, because the cost ceiling is removed, ADS\-C can increaseλ\\lambdafar past the point where unmodified ADS becomes unusable, more than recovering the remainder: its zero\-cost−17\.4​pp\-17\.4\\,\\mathrm\{pp\}\(CIFAR\-100\) /−29\.6​pp\-29\.6\\,\\mathrm\{pp\}\(CIFAR\-10\) atλ=3\\lambda=3exceeds the unbudgeted poison’s total damage at any teacher cost below2727–33​pp33\\,\\mathrm\{pp\}\. This decomposition is, to our knowledge, the first quantitative account of*how*ADS corrupts dark knowledge in classification: the durable damage is corruption of non\-target structure, not label noise\. This conclusion is further corroborated by Sec\.[9\-B](https://arxiv.org/html/2607.15467#S9.SS2)from an independent direction\.

![Refer to caption](https://arxiv.org/html/2607.15467v1/x17.png)
Figure 10:Decomposition of unmodified ADS damage at matchedλ\\lambda\. The unbudgeted student drop splits into an argmax\-safe part \(survives the budget, at zero cost\) and flip collateral \(purchased at1111–12​pp12\\,\\mathrm\{pp\}of teacher accuracy\)\.
### 8\-DAblation: softening front\-ends at matched served fidelity

Finally we place the strongest softening variant, namely the gated top\-logit reduction of Sec\.[6\-D](https://arxiv.org/html/2607.15467#S6.SS4), composed with the ADS\-C against ADS\-C with no temperature scaling\. We sweep the softener’s gate over five settings andλ\\lambdaover the full grid \(360360runs total across both datasets; teacher drop≡0\\equiv 0everywhere, as guaranteed\)\.

At matchedλ\\lambdathe comparison is misleading as the softener is stronger at lowλ\\lambda\(by up to6\.3​pp6\.3\\,\\mathrm\{pp\}on CIFAR\-100 and22\.2​pp22\.2\\,\\mathrm\{pp\}on CIFAR\-10\) and weaker at highλ\\lambda\. Butλ\\lambdais free\. The axes on which a defender actually pays are teacher drop \(identically zero for both variants\) and served fidelity\. Recomputed on the fidelity axis \(Fig\.[11](https://arxiv.org/html/2607.15467#S8.F11)\), the comparison inverts as the softener spends a large fidelity budget up front \(DK​L=0\.49D\_\{KL\}=0\.49atλ=0\\lambda=0for its most aggressive gate\) before any poison flows, while ADS\-C spends fidelity only asλ\\lambdagrows\. At matched served\-KL, ADS\-C without any softening damages the student more everywhere in the operating regime\. Its advantage widens with fidelity spend, from\+1\.0​pp\+1\.0\\,\\mathrm\{pp\}at KL0\.50\.5to\+3\.8​pp\+3\.8\\,\\mathrm\{pp\}at KL0\.90\.9on CIFAR\-100, and\+1\.0\+1\.0to\+2\.2​pp\+2\.2\\,\\mathrm\{pp\}across KL0\.40\.4–0\.70\.7on CIFAR\-10\. ADS\-C’s deepest zero\-cost damage \(29\.9529\.95/51\.8051\.80\) is unmatched by any softener configuration on either dataset\. The softener retains a≤1​pp\\leq 1\\,\\mathrm\{pp\}advantage only in the low\-damage corner \(KL≤0\.3\\leq 0\.3\) that is undesirable for defense\. The matched\-λ\\lambdacrossover is thus an artifact of comparing on an axis that costs nothing; at matched served fidelity, softening is dominated\. This ablation also answers the natural question of why ADS\-C includes no confidence\-scaling component: the strongest such component we constructed is dominated by not including one\.

![Refer to caption](https://arxiv.org/html/2607.15467v1/x18.png)\(a\)CIFAR\-100
![Refer to caption](https://arxiv.org/html/2607.15467v1/x19.png)\(b\)CIFAR\-10

Figure 11:The served\-fidelity trade\-off: distilled\-student accuracy versus servedDK​L​\(𝐩𝒯∥𝐩^\)D\_\{KL\}\(\\mathbf\{p\}\_\{\\mathcal\{T\}\}\\\|\\hat\{\\mathbf\{p\}\}\); teacher drop≡0\\equiv 0for every point \(mean±\\pmstd whiskers over 3 seeds;⋆\\starmarks the undefendedλ=0\\lambda\{=\}0anchor\)\. Each softener gate starts already displaced right \(open markers\): fidelity spent up front, atλ=0\\lambda\{=\}0, before any poison flows—up toDK​L=0\.49D\_\{KL\}=0\.49\(CIFAR\-100\) and0\.650\.65\(CIFAR\-10\) for the most aggressive gatept=0\.5p\_\{t\}\{=\}0\.5\. Poison\-only ADS\-C is the lower envelope throughout the operating regime, and at the deepest fidelity it spends \(λ=3\\lambda\{=\}3\) the best softener configuration at matched served\-KL leaves the student4\.0​pp4\.0\\,\\mathrm\{pp\}/2\.6​pp2\.6\\,\\mathrm\{pp\}higher \(arrows\): softening never recovers its up\-front spend\.
### 8\-EControls

The hard\-label attacker\.We train the attacker of Sec\.[3](https://arxiv.org/html/2607.15467#S3)on the served argmax alone \(cross\-entropy; implemented as KL against the one\-hot served label, which is identical\)\. Under ADS\-C the served argmax equals the clean argmax on every query\. We verify zero deviations over both50 00050\\,000\-query corpora atλ∈\{2,3\}\\lambda\\in\\\{2,3\\\}\. Therefore this attacker receives identical supervision from the defended and undefended APIs\. Two observations follow\. First, these overconfident teachers serve vectors so nearly one\-hot that, without softening, dark knowledge confers almost no advantage to begin with, consistent with Table[1](https://arxiv.org/html/2607.15467#S5.T1), and complementary to Sec\.[6\-A](https://arxiv.org/html/2607.15467#S6.SS1), where the advantage appears \(\+4\.7​pp\+4\.7\\,\\mathrm\{pp\}\) as soon as softening reveals the non\-target structure\. Second, and decisive for the defense: ADS\-C drives the soft\-label student accuracy down by17\.1​pp17\.1\\,\\mathrm\{pp\}\(CIFAR\-100,λ=3\\lambda=3\) and29\.7​pp29\.7\\,\\mathrm\{pp\}\(CIFAR\-10\)\. The defended soft output is strictly worse distillation signal than the argmax it accompanies\. The defense does not merely remove the incentive to distill from served probabilities, it reverses it, while the argmax channel remains exactly as informative as any accurate API must be\. Sec\.[9](https://arxiv.org/html/2607.15467#S9)addresses the attacker who exploits knowledge of the guarantee itself\.

### 8\-FFidelity of the stolen copy

Top\-1 accuracy understates the damage to an attacker whose goal is a faithful replica rather than a merely accurate one \(e\.g\., to mount downstream evasion or membership\-inference attacks\[[15](https://arxiv.org/html/2607.15467#bib.bib15),[13](https://arxiv.org/html/2607.15467#bib.bib13)\]\)\. Agreement between student and teacher predictions falls in step with student accuracy \(CIFAR\-100:0\.490\.49undefended→\\to0\.310\.31atλ=3\\lambda=3; CIFAR\-10:0\.83→0\.520\.83\\to 0\.52\): the extracted model is not only worse, it is a worse copy\.

## 9TRADING THE GUARANTEE FOR ATTACKER UNCERTAINTY

Proposition[1](https://arxiv.org/html/2607.15467#Thmproposition1)is deliberately absolute, and absoluteness is itself information: an attacker who knows the defense knows that every served top\-1 is honest, and that inputs whose margin lies at or below the threshold are served identically to the clean teacher\. Those provably clean rows are, moreover, exactly the thin\-margin inputs where near\-boundary class structure, corresponding to the richest dark knowledge, resides\. A defender may therefore wish to spend a small, controlled amount of the guarantee to remove these certainties\. We provide two such relaxations, each with closed\-form\-predictable utility cost and each disabled by default; ADS\-C withq=1q=1andλmin=0\\lambda\_\{\\min\}=0remains the proposed method\. All results below are trained\-student measurements \(3 seeds, both CIFAR datasets\)\.

### 9\-AStochastic enforcement of the budget

The first relaxation enforces the budget \([9](https://arxiv.org/html/2607.15467#S7.E9)\) on each input with probabilityqq; with probability1−q1\-qthe input receives the full requestedλ\\lambda\(raw poison\)\. The endpoints recover the two extremes—q=1q=1is ADS\-C,q=0q=0is unmodified ADS—and the expected teacher cost follows in closed form from Sec\.[5\-D](https://arxiv.org/html/2607.15467#S5.SS4):\(1−q\)\(1\-q\)times the unmodified\-ADS teacher cost atλ\\lambdapredicted by \([7](https://arxiv.org/html/2607.15467#S5.E7)\), so the defender can price the relaxation from cached outputs before deploying it\. The enforcement mask is drawn once per input from a fixed seed, not per query \(Sec\.[9\-C](https://arxiv.org/html/2607.15467#S9.SS3)\)\.

Figure[12](https://arxiv.org/html/2607.15467#S9.F12)shows the resulting family of trade\-off curves, which interpolates between the ADS\-C vertical and the ADS diagonal\. The trade\-off is ordered*exactly*byqqon both datasets: at every level of teacher cost, the best achievable student damage is obtained by the largestqqwhose curve reaches that cost, so no configuration with smallerqqis ever preferable at equal spend\. The exchange below the zero\-cost floor is far better than unmodified ADS offers: relaxing toq=0\.9q=0\.9atλ=3\\lambda=3yields a further−4\.1​pp\-4\.1\\,\\mathrm\{pp\}of student damage \(CIFAR\-100, to25\.9%25\.9\\%\) for6\.4​pp6\.4\\,\\mathrm\{pp\}of teacher cost, where unmodified ADS would require32\.8​pp32\.8\\,\\mathrm\{pp\}to match—approximately a5×5\\timesbetter exchange; on CIFAR\-10,−6\.0​pp\-6\.0\\,\\mathrm\{pp\}for7\.9​pp7\.9\\,\\mathrm\{pp\}against39\.3​pp39\.3\\,\\mathrm\{pp\}\(5\.0×5\.0\\times\)\. Extending the sweep toλ=10\\lambda=10on CIFAR\-100 strengthens this to a uniform statement: the relaxed configurations saturate \(q=0\.9q=0\.9reaches student accuracy22\.7%22\.7\\%at6\.9​pp6\.9\\,\\mathrm\{pp\}, more than one student\-pp per teacher\-pp below the floor\), and unmodified ADS becomes dominated at*every*point of the trade\-off—its deepest operating point \(student4\.2%4\.2\\%at64\.0​pp64\.0\\,\\mathrm\{pp\}\) is surpassed byq=0\.3q=0\.3atλ=10\\lambda=10\(student3\.5%3\.5\\%at49\.1​pp49\.1\\,\\mathrm\{pp\}\)\.

The hard\-label measurements of Sec\.[8\-E](https://arxiv.org/html/2607.15467#S8.SS5)add an important qualification \(Fig\.[13](https://arxiv.org/html/2607.15467#S9.F13)\)\. Onceq<1q<1, the two attacker channels separate, and at everyqqthe*hard\-label*student is the stronger one \(38\.5%38\.5\\%versus25\.9%25\.9\\%atq=0\.9q=0\.9on CIFAR\-100;73\.173\.1versus45\.8%45\.8\\%on CIFAR\-10\): a defense\-aware attacker responds to the relaxation by distilling from the served argmax, whose label noise—the flipped rows—arrives at the closed\-form rate\(1−q\)​F​\(λ\)\(1\-q\)F\(\\lambda\)\. Priced against this best response, the exchange below the floor is1\.351\.35,1\.021\.02,0\.890\.89, and0\.810\.81student\-pp per teacher\-pp atq=0\.9q=0\.9,0\.70\.7,0\.50\.5, and0\.30\.3on CIFAR\-100 \(1\.071\.07,1\.021\.02,0\.990\.99, and0\.970\.97on CIFAR\-10\): approximately break\-even near the guarantee, drifting below it at deep relaxation, and in every case far from the5×5\\timesavailable against an attacker committed to soft labels\. Three observations delimit the relaxation’s value\. First, the best\-responding attacker still degrades below the floor at everyqq—the deliberate flips are systematic, input\-consistent noise that the student learns rather than averages away—so no channel recovers the undefended leakage\. Second, the exchange rates place the defensible operating points at shallow relaxation \(q≳0\.7q\\gtrsim 0\.7\), where the trade stays at or above break\-even against either channel; deeper relaxation is justified only against an attacker known to consume soft labels\. Third, the guarantee pointq=1q=1remains uniquely favorable: it is the only setting at which the defender pays nothing while the attacker’s best response yields exactly the floor\.

![Refer to caption](https://arxiv.org/html/2607.15467v1/x20.png)\(a\)CIFAR\-100
![Refer to caption](https://arxiv.org/html/2607.15467v1/x21.png)\(b\)CIFAR\-10

Figure 12:Stochastic enforcement of the budget \(margin cap enforced with probabilityqq; 3 seeds; dashed gray: 1:1 break\-even\)\. The family of trade\-off curves interpolates between ADS\-C \(q=1q=1, vertical\) and ADS \(q=0q=0, diagonal\) and is ordered exactly byqqon both datasets\. Below ADS\-C’s zero\-cost floor,q=0\.9q=0\.9obtains additional student damage at approximately5×5\\timesbetter exchange rate than unmodified ADS against the soft\-label attacker; Fig\.[13](https://arxiv.org/html/2607.15467#S9.F13)prices the same relaxation against the attacker’s best response\.![Refer to caption](https://arxiv.org/html/2607.15467v1/x22.png)\(a\)CIFAR\-100
![Refer to caption](https://arxiv.org/html/2607.15467v1/x23.png)\(b\)CIFAR\-10

Figure 13:The two attacker channels under stochastic enforcement \(λ=3\\lambda=3, 3 seeds\): distilled\-student accuracy of the soft\-label attacker \(served probabilities\) and the hard\-label attacker \(served argmax\) as a function of the enforcement probabilityqq\. Atq=1q=1the argmax channel is identical to the undefended API floor \(dashed\) while the soft channel lies17\.1​pp17\.1\\,\\mathrm\{pp\}\(CIFAR\-100\) and29\.7​pp29\.7\\,\\mathrm\{pp\}\(CIFAR\-10\) below it\. At everyq<1q<1the attacker’s best response flips to hard labels, whose deliberate label noise arrives at the closed\-form rate\(1−q\)​F​\(λ\)\(1\-q\)F\(\\lambda\); both channels nevertheless remain below the floor throughout\.
### 9\-BDefensive flipping with a minimumλ\\lambda

The second relaxation addresses the clean\-row exposure directly: serve every input with strength at leastλmin\\lambda\_\{\\min\}, i\.e\.,λeff​\(𝐱\)=max⁡\(λmin,λ​\(𝐱\)\)\\lambda\_\{\\mathrm\{eff\}\}\(\\mathbf\{x\}\)=\\max\(\\lambda\_\{\\min\},\\lambda\(\\mathbf\{x\}\)\)\. Inputs whose flip threshold lies belowλmin\\lambda\_\{\\min\}are deliberately flipped, which we refer to as*defensive flipping*, concentrated by construction on the thinnest\-margin rows, where flips are least costly\. The gross cost is again available in closed form from the transition curve,F​\(λmin\)F\(\\lambda\_\{\\min\}\), and the*net*cost is smaller, for a geometric reason: thin\-margin rows are disproportionately already misclassified, and on such rows the true class is often the runner\-up \(68%68\\%of flipped misclassified rows on CIFAR\-10,33%33\\%on CIFAR\-100\), so a fraction of defensive flips land on the true label and partially offset the gross cost \(net cost≈12\\approx\{\\tfrac\{1\}\{2\}\}gross on CIFAR\-100; the offset weakens withKK\)\. This is a consequence of margin geometry rather than of any corrective aim in the poison, and we report it as such\.

Measured end\-to\-end, the minimum\-strength floor serves a single purpose, at near\-zero cost: atλmin=0\.05\\lambda\_\{\\min\}=0\.05,λ=3\\lambda=3,*no served row is clean*\(the clean\-row fraction falls from0\.4%0\.4\\%to≈0\{\\approx\}0\) at a teacher cost of0\.30​pp0\.30\\,\\mathrm\{pp\}\(CIFAR\-100\) /0\.16​pp0\.16\\,\\mathrm\{pp\}\(CIFAR\-10\), with student accuracy statistically indistinguishable from the ADS\-C floor \(30\.130\.1vs\.29\.9529\.95;51\.451\.4vs\.51\.8051\.80\)\. It is a hardening mechanism—it removes the attacker’s certainty about which rows are unperturbed—not a means of increasing damage\. Nor does it open a side channel: the hard\-label attacker underλmin∈\{0\.05,0\.2\}\\lambda\_\{\\min\}\\in\\\{0\.05,0\.2\\\}remains within seed noise of its leakage floor \(46\.746\.7/47\.347\.3vs\.47\.0%47\.0\\%on CIFAR\-100;81\.381\.3/81\.581\.5vs\.81\.6%81\.6\\%on CIFAR\-10\), as the≤0\.2%\\leq 0\.2\\%label\-noise rate predicts\.

A stronger variant is conceivable: sacrificing the thin\-margin rows*entirely*\(serving them the full unbudgetedλ\\lambda\) is the margin\-ranked analogue of stochastic enforcement, and on served\-distribution corruption \(poison\-KL\) it appeared to dominate stochastic sacrifice by a factor of∼2\{\\sim\}2in our training\-free pre\-analysis\. The trained students overturn this prediction \(Fig\.[14](https://arxiv.org/html/2607.15467#S9.F14)\): margin\-ranked sacrifice cannot push the student meaningfully below the ADS\-C floor \(best case29\.15%29\.15\\%vs\. floor29\.95%29\.95\\%on CIFAR\-100, at6\.0​pp6\.0\\,\\mathrm\{pp\}teacher cost\), whereas stochastic sacrifice at the same∼6​pp\{\\sim\}6\\,\\mathrm\{pp\}reaches25\.9%25\.9\\%and continues to8\.3%8\.3\\%at higher spend\. Concentrating heavy poison on near\-boundary rows inflates the served KL divergence while teaching the attacker’s student little it would not have learned regardless: those rows carry little*informative*dark knowledge relative to their contribution to the KL divergence\. We report this negative result because it provides independent, converging evidence for the mechanism identified in Sec\.[8\-C](https://arxiv.org/html/2607.15467#S8.SS3): what damages a distilling student is corruption of the informative inter\-class structure on representative inputs, not maximization of distributional distortion where distortion is inexpensive\.

![Refer to caption](https://arxiv.org/html/2607.15467v1/x24.png)\(a\)CIFAR\-100
![Refer to caption](https://arxiv.org/html/2607.15467v1/x25.png)\(b\)CIFAR\-10

Figure 14:Defensive flipping versus stochastic sacrifice, in the low\-cost region \(⋆\\star: ADS\-C at zero teacher cost; dotted rule: the ADS\-C free floor; error bars±1\\pm 1std over 3 seeds\)\. Stochastic sacrifice \(triangles; theqq\-relaxation of Sec\.[9\-A](https://arxiv.org/html/2607.15467#S9.SS1), drawn as its lower envelope over theq,λq,\\lambdasweep\) descends below the floor; margin\-ranked sacrifice \(squares\) is dominated—heavy poison on thin\-margin rows inflates served KL divergence but not student damage; and the minimum\-strength floorλmin\\lambda\_\{\\min\}\(circles\), like margin\-ranked shown at the deployment strengthλ=3\\lambda\{=\}3, sits at the ADS\-C floor at≤0\.3​pp\\leq 0\.3\\,\\mathrm\{pp\}teacher cost while guaranteeing that no served row is clean\.
### 9\-CDeterminism and repeated queries

Any defense that randomizes*per query*is invertible by repetition: querying the same input many times and aggregating the responses recovers the clean signal\. The mechanisms above are immune to this attack by construction\. The base ADS\-C perturbation is a deterministic function of the input \(teacher logits and frozen proxy clones\), so repeated queries return the identical perturbed vector; the enforcement mask of Sec\.[9\-A](https://arxiv.org/html/2607.15467#S9.SS1)is drawn per*input*from a fixed seed, not per query, so a sacrificed input is consistently sacrificed; andλmin\\lambda\_\{\\min\}is inherently per\-input deterministic\. Repeated queries therefore provide the attacker no additional information\. The residual cost of determinism is a persistent error set underλmin\\lambda\_\{\\min\}\(the same flipped inputs are always flipped\)—for a deployed API arguably preferable to nondeterministic answers—and identifying that set would require access to the clean teacher, which is the asset under protection\. The utility statement weakens in form but remains strong: from “accuracy identical” to “accuracy drop at mostF​\(λmin\)F\(\\lambda\_\{\\min\}\), known in closed form before deployment, and concentrated on inputs the teacher was already uncertain about\.”

## 10DISCUSSION AND LIMITATIONS

What the guarantee does and does not cover\.Proposition[1](https://arxiv.org/html/2607.15467#Thmproposition1)protects the served argmax—the quantity consumed by classification API users\. Applications that consume the full served distribution \(e\.g\., downstream risk calibration\) pay the served\-fidelity cost, which we quantify explicitly: it is the horizontal axis of Fig\.[11](https://arxiv.org/html/2607.15467#S8.F11), and at the recommended operating point amounts to a served KL of0\.80\.8–0\.90\.9\(CIFAR\-100\)\. In the dark\-knowledge threat model this cost falls almost entirely on the attacker: the honest consumer of a classification API reads the label\.

Attacker adaptivity\.Our main results use the standard fixed distillation attacker\. The hard\-label attacker is reduced to the distillation floor every classification API shares\. Repetition attacks are addressed by per\-input determinism \(Sec\.[9\-C](https://arxiv.org/html/2607.15467#S9.SS3)\)\. Attacks that estimate and subtract the poison would need to reconstruct a hidden gradient direction through a black\-box interface; we view a formal analysis of this inversion problem as valuable future work\.

Scale and generality\.Our evidence is CIFAR\-10, CIFAR\-100, and Tiny\-ImageNet\. Nothing in ADS\-C is architecture\- or scale\-specific, and the Tiny\-ImageNet teacher already covers the deployment\-realistic case of a strong pretrained model, whose overconfidence \(Table[1](https://arxiv.org/html/2607.15467#S5.T1)\) shows the phenomenon we exploit is a property of model quality rather than of small benchmarks\. Validation on large\-scale production teachers and modern architectures \(e\.g\., vision transformers\) remains future work; the transition\-curve analysis of Sec\.[5\-D](https://arxiv.org/html/2607.15467#S5.SS4)provides a tool, computable from teacher and proxy outputs alone, to predict, before any training, how a new teacher will behave\.

The trade\-off\-rate lens\.We encourage future inference\-time defenses to report \(i\) utility cost with the axes made explicit, and \(ii\) frontiers at matched served fidelity rather than matched internal knob settings; Sec\.[8\-D](https://arxiv.org/html/2607.15467#S8.SS4)shows the latter comparison can invert the former\.

## 11CONCLUSION

We adapted Antidistillation Sampling to classification and showed that its behavior there is governed by teacher overconfidence: an inert window in which the defense measurably affects neither party, a transition curve that predicts the defended teacher’s cost across the entire sweep, and a saturating regime in which the direct transfer trades below break\-even\. The perturbation itself is faithful and correctly aimed at dark knowledge; the failure is margin\-blindness at composition, on teachers whose confidence is extremely heterogeneous\. Temperature softening rescales the transition curve exactly as the theory predicts, which makes it a valuable probe and demonstrates that the availability of dark knowledge is what limits the perturbation at low strengths; but it is not a remedy: the same rescaling collapses the teacher, and every temperature configuration lies on the same unfavorable trade\-off curve\. ADS\-C enforces the missing margin\-awareness where both the margin and the perturbation are visible—at composition—with a closed\-form per\-input budget that provably preserves every served prediction\. The result is, to our knowledge, the first antidistillation defense for classification with exactly zero utility cost:17\.4​pp17\.4\\,\\mathrm\{pp\}\(CIFAR\-100\),29\.6​pp29\.6\\,\\mathrm\{pp\}\(CIFAR\-10\), and13\.3​pp13\.3\\,\\mathrm\{pp\}\(Tiny\-ImageNet\) of distilled\-student degradation at a teacher cost of0\.00​pp0\.00\\,\\mathrm\{pp\}\. Measured against the complementary channel, the guarantee reverses the economics of extraction: an attacker restricted to the served argmax obtains exactly the distillation floor that every accurate classification API shares, while the defended soft output trains a student well below that floor\. The served probabilities become strictly worse supervision than the labels they accompany\. Around the guarantee we provide two relaxations with closed\-form\-predictable cost, stochastic enforcement and defensive flipping, that allow a defender to exchange a known amount of utility for attacker uncertainty, priced throughout against the attacker’s best response across both supervision channels, together with a negative result showing that stochastic sacrifice outperforms margin\-ranked sacrifice, further evidence that the durable damage resides in informative dark knowledge\. A defense that costs nothing changes the deployment calculus as there is no longer a utility argument for serving undefended soft labels\.

## ACKNOWLEDGMENTS

The authors thank the MS Artificial Intelligence Program at the Rochester Institute of Technology for computational resources, and the Fulbright Program for scholarship support to the first author\. M\. J\. Khojasteh acknowledges support from the Gleason Endowment and Provost’s Learning Innovation Grant at RIT\. The authors acknowledge Research Computing at the Rochester Institute of Technology for providing computational resources and support that have contributed to the research results reported in this publication\[[60](https://arxiv.org/html/2607.15467#bib.bib60)\]\.

## References

- \[1\]J\. Schmidhuber, “Learning complex, extended sequences using the principle of history compression,”*Neural computation*, vol\. 4, no\. 2, pp\. 234–242, 1992\.
- \[2\]G\. Hinton, O\. Vinyals, and J\. Dean, “Distilling the knowledge in a neural network,”*arXiv preprint arXiv:1503\.02531*, 2015\.
- \[3\]A\. G\. Howard, M\. Zhu, B\. Chen, D\. Kalenichenko, W\. Wang, T\. Weyand, M\. Andreetto, and H\. Adam, “Mobilenets: Efficient convolutional neural networks for mobile vision applications,”*arXiv preprint arXiv:1704\.04861*, 2017\.
- \[4\]V\. Sanh, L\. Debut, J\. Chaumond, and T\. Wolf, “Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter,”*arXiv preprint arXiv:1910\.01108*, 2019\.
- \[5\]H\. Touvron, M\. Cord, M\. Douze, F\. Massa, A\. Sablayrolles, and H\. Jégou, “Training data\-efficient image transformers & distillation through attention,” in*International conference on machine learning*\. PMLR, 2021, pp\. 10 347–10 357\.
- \[6\]J\. Gou, B\. Yu, S\. J\. Maybank, and D\. Tao, “Knowledge distillation: A survey,”*International journal of computer vision*, vol\. 129, no\. 6, pp\. 1789–1819, 2021\.
- \[7\]F\. Tramèr, F\. Zhang, A\. Juels, M\. K\. Reiter, and T\. Ristenpart, “Stealing machine learning models via prediction\{\\\{APIs\}\\\},” in*25th USENIX security symposium \(USENIX Security 16\)*, 2016, pp\. 601–618\.
- \[8\]K\. Zhao, L\. Li, K\. Ding, N\. Z\. Gong, Y\. Zhao, and Y\. Dong, “A survey on model extraction attacks and defenses for large language models,” in*Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V\. 2*, 2025, pp\. 6227–6236\.
- \[9\]CNBC, “DeepSeek trained AI model using distillation, now a disruptive force,” CNBC, Feb\. 2025,[https://www\.cnbc\.com/2025/02/21/deepseek\-trained\-ai\-model\-using\-distillation\-now\-a\-disruptive\-force\.html](https://www.cnbc.com/2025/02/21/deepseek-trained-ai-model-using-distillation-now-a-disruptive-force.html)\.
- \[10\]Anthropic, “Detecting and preventing distillation attacks,” Anthropic News, February 2026, accessed: March 23, 2026\. \[Online\]\. Available:[https://www\.anthropic\.com/news/detecting\-and\-preventing\-distillation\-attacks](https://www.anthropic.com/news/detecting-and-preventing-distillation-attacks)
- \[11\]Y\. Sun, T\. Liu, P\. Hu, Q\. Liao, S\. Fu, N\. Yu, D\. Guo, Y\. Liu, and L\. Liu, “Deep intellectual property protection: A survey,”*arXiv preprint arXiv:2304\.14613*, 2023\.
- \[12\]Y\. Jiang, Y\. Gao, C\. Zhou, H\. Hu, S\. Chen, A\. Fu, and W\. Susilo, “Intellectual property protection for deep learning model and dataset intelligence,”*Engineering Applications of Artificial Intelligence*, vol\. 163, p\. 113024, 2026\.
- \[13\]R\. Shokri, M\. Stronati, C\. Song, and V\. Shmatikov, “Membership inference attacks against machine learning models,” in*2017 IEEE symposium on security and privacy \(SP\)*\. IEEE, 2017, pp\. 3–18\.
- \[14\]A\. Salem, Y\. Zhang, M\. Humbert, P\. Berrang, M\. Fritz, and M\. Backes, “Ml\-leaks: Model and data independent membership inference attacks and defenses on machine learning models,”*arXiv preprint arXiv:1806\.01246*, 2018\.
- \[15\]N\. Papernot, P\. McDaniel, I\. Goodfellow, S\. Jha, Z\. B\. Celik, and A\. Swami, “Practical black\-box attacks against machine learning,” in*Proceedings of the 2017 ACM on Asia conference on computer and communications security*, 2017, pp\. 506–519\.
- \[16\]B\. Biggio, I\. Corona, D\. Maiorca, B\. Nelson, N\. Šrndić, P\. Laskov, G\. Giacinto, and F\. Roli, “Evasion attacks against machine learning at test time,” in*Joint European conference on machine learning and knowledge discovery in databases*\. Springer, 2013, pp\. 387–402\.
- \[17\]D\. Lowd and C\. Meek, “Adversarial learning,” in*Proceedings of the eleventh ACM SIGKDD international conference on Knowledge discovery in data mining*, 2005, pp\. 641–647\.
- \[18\]M\. Fredrikson, E\. Lantz, S\. Jha, S\. Lin, D\. Page, and T\. Ristenpart, “Privacy in pharmacogenetics: An\{\\\{End\-to\-End\}\\\}case study of personalized warfarin dosing,” in*23rd USENIX security symposium \(USENIX Security 14\)*, 2014, pp\. 17–32\.
- \[19\]M\. Fredrikson, S\. Jha, and T\. Ristenpart, “Model inversion attacks that exploit confidence information and basic countermeasures,” in*Proceedings of the 22nd ACM SIGSAC conference on computer and communications security*, 2015, pp\. 1322–1333\.
- \[20\]G\. Ateniese, L\. V\. Mancini, A\. Spognardi, A\. Villani, D\. Vitali, and G\. Felici, “Hacking smart machines with smarter ones: How to extract meaningful data from machine learning classifiers,”*International Journal of Security and Networks*, vol\. 10, no\. 3, pp\. 137–150, 2015\.
- \[21\]C\. J\. Haug and J\. M\. Drazen, “Artificial intelligence and machine learning in clinical medicine, 2023,”*New England Journal of Medicine*, vol\. 388, no\. 13, pp\. 1201–1208, 2023\.
- \[22\]L\. Hernandez Aros, L\. X\. Bustamante Molano, F\. Gutierrez\-Portela, J\. J\. Moreno Hernandez, and M\. S\. Rodríguez Barrero, “Financial fraud detection through the application of machine learning techniques: a literature review,”*Humanities and Social Sciences Communications*, vol\. 11, no\. 1, pp\. 1–22, 2024\.
- \[23\]S\. Cao, M\. Li, J\. Hays, D\. Ramanan, Y\.\-X\. Wang, and L\. Gui, “Learning lightweight object detectors via multi\-teacher progressive distillation,” in*International Conference on Machine Learning*\. PMLR, 2023, pp\. 3577–3598\.
- \[24\]Y\. Savani, A\. Trockman, Z\. Feng, A\. Schwarzschild, A\. Robey, M\. Finzi, and J\. Z\. Kolter, “Antidistillation sampling,”*arXiv preprint arXiv:2504\.13146*, 2025\.
- \[25\]T\. Lee, B\. Edwards, I\. Molloy, and D\. Su, “Defending against neural network model stealing attacks using deceptive perturbations,” in*2019 IEEE Security and Privacy Workshops \(SPW\)*\. IEEE, 2019, pp\. 43–49\.
- \[26\]Y\. Chen, R\. Guan, X\. Gong, J\. Dong, and M\. Xue, “D\-dae: Defense\-penetrating model extraction attacks,” in*2023 IEEE Symposium on Security and Privacy \(SP\)*\. IEEE, 2023, pp\. 382–399\.
- \[27\]C\. Guo, G\. Pleiss, Y\. Sun, and K\. Q\. Weinberger, “On calibration of modern neural networks,” in*International conference on machine learning*\. PMLR, 2017, pp\. 1321–1330\.
- \[28\]C\. Buciluǎ, R\. Caruana, and A\. Niculescu\-Mizil, “Model compression,” in*Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining*, 2006, pp\. 535–541\.
- \[29\]J\. Ba and R\. Caruana, “Do deep nets really need to be deep?”*Advances in neural information processing systems*, vol\. 27, 2014\.
- \[30\]T\. Furlanello, Z\. Lipton, M\. Tschannen, L\. Itti, and A\. Anandkumar, “Born again neural networks,” in*International conference on machine learning*\. PMLR, 2018, pp\. 1607–1616\.
- \[31\]J\. Tang, R\. Shivanna, Z\. Zhao, D\. Lin, A\. Singh, E\. H\. Chi, and S\. Jain, “Understanding and improving knowledge distillation,”*arXiv preprint arXiv:2002\.03532*, 2020\.
- \[32\]T\. Orekondy, B\. Schiele, and M\. Fritz, “Knockoff nets: Stealing functionality of black\-box models,” in*Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, 2019, pp\. 4954–4963\.
- \[33\]J\.\-B\. Truong, P\. Maini, R\. J\. Walls, and N\. Papernot, “Data\-free model extraction,” in*Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, 2021, pp\. 4771–4780\.
- \[34\]S\. Jahan and R\. Sun, “Black\-box behavioral distillation breaks safety alignment in medical llms,”*arXiv preprint arXiv:2512\.09403*, 2025\.
- \[35\]D\. Oliynyk, R\. Mayer, and A\. Rauber, “I know what you trained last summer: A survey on stealing machine learning models and defences,”*ACM Computing Surveys*, vol\. 55, no\. 14s, pp\. 1–41, 2023\.
- \[36\]W\. Jiang, H\. Li, G\. Xu, T\. Zhang, and R\. Lu, “A comprehensive defense framework against model extraction attacks,”*IEEE Transactions on Dependable and Secure Computing*, vol\. 21, no\. 2, pp\. 685–700, 2023\.
- \[37\]X\. Xu, M\. Li, C\. Tao, T\. Shen, R\. Cheng, J\. Li, C\. Xu, D\. Tao, and T\. Zhou, “A survey on knowledge distillation of large language models,”*arXiv preprint arXiv:2402\.13116*, 2024\.
- \[38\]H\. Ma, T\. Chen, T\.\-K\. Hu, C\. You, X\. Xie, and Z\. Wang, “Undistillable: Making a nasty teacher that cannot teach students,”*arXiv preprint arXiv:2105\.07381*, 2021\.
- \[39\]E\. Yilmaz and H\. Y\. Keles, “Adversarial sparse teacher: Defense against distillation\-based model stealing attacks using adversarial examples,”*IEEE Access*, 2025\.
- \[40\]C\. Liang, J\. Huang, Z\. Zhang, and S\. Zhang, “Defending against model extraction attacks with ood feature learning and decision boundary confusion,”*Computers & Security*, vol\. 136, p\. 103563, 2024\.
- \[41\]X\. Cheng, M\. Zheng, S\. Zhu, and Y\. Dong, “Misleader: Defending against model extraction with ensembles of distilled models,”*arXiv preprint arXiv:2506\.02362*, 2025\.
- \[42\]M\. Juuti, S\. Szyller, S\. Marchal, and N\. Asokan, “Prada: protecting against dnn model stealing attacks,” in*2019 IEEE European Symposium on Security and Privacy \(EuroS&P\)*\. IEEE, 2019, pp\. 512–527\.
- \[43\]M\. Gurve, S\. Behera, S\. Ahlawat, and Y\. Prasad, “Misguide: Defense against data\-free deep learning model extraction,”*arXiv preprint arXiv:2403\.18580*, 2024\.
- \[44\]A\. Chakraborty, S\. F\. Ahamed, S\. Roy, S\. Banerjee, K\. Choi, A\. Rahman, A\. Hu, E\. Bowen, and S\. Shetty, “Radep: A resilient adaptive defense framework against model extraction attacks,” in*2025 International Wireless Communications and Mobile Computing \(IWCMC\)*\. IEEE, 2025, pp\. 1241–1246\.
- \[45\]J\. R\. Douceur, “The Sybil attack,” in*International Workshop on Peer\-to\-Peer Systems*\. Springer, 2002, pp\. 251–260\.
- \[46\]H\. Yu, K\. Yang, T\. Zhang, Y\.\-Y\. Tsai, T\.\-Y\. Ho, and Y\. Jin, “Cloudleak: Large\-scale deep learning models stealing through adversarial examples\.” in*NDSS*, vol\. 38, 2020, p\. 102\.
- \[47\]Y\. Uchida, Y\. Nagai, S\. Sakazawa, and S\. Satoh, “Embedding watermarks into deep neural networks,” in*Proceedings of the 2017 ACM on international conference on multimedia retrieval*, 2017, pp\. 269–277\.
- \[48\]H\. Jia, C\. A\. Choquette\-Choo, V\. Chandrasekaran, and N\. Papernot, “Entangled watermarks as a defense against model extraction,” in*30th USENIX security symposium \(USENIX Security 21\)*, 2021, pp\. 1937–1954\.
- \[49\]S\. Szyller, B\. G\. Atli, S\. Marchal, and N\. Asokan, “Dawn: Dynamic adversarial watermarking of neural networks,” in*Proceedings of the 29th ACM international conference on multimedia*, 2021, pp\. 4417–4425\.
- \[50\]Z\. Wang, Y\. Wu, and H\. Huang, “Defense against model extraction attack by bayesian active watermarking,” in*Forty\-first International Conference on Machine Learning*, 2024\.
- \[51\]T\. Orekondy, B\. Schiele, and M\. Fritz, “Prediction poisoning: Towards defenses against dnn model stealing attacks,”*arXiv preprint arXiv:1906\.10908*, 2019\.
- \[52\]M\. Tang, A\. Dai, L\. DiValentin, A\. Ding, A\. Hass, N\. Z\. Gong, Y\. Chen*et al\.*, “\{\\\{ModelGuard\}\\\}:\{\\\{Information\-Theoretic\}\\\}defense against model extraction attacks,” in*33rd USENIX Security Symposium \(USENIX Security 24\)*, 2024, pp\. 5305–5322\.
- \[53\]D\.\-D\. Wu, C\. Fu, W\. Wu, W\. Xia, X\. Zhang, J\. Zhou, and M\.\-L\. Zhang, “Efficient model stealing defense with noise transition matrix,” in*Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, 2024, pp\. 24 305–24 315\.
- \[54\]H\. Fang, T\. Zhang, T\. Zhuang, J\. Kong, K\. Gao, B\. Chen, L\. Liang, S\.\-T\. Xia, and K\. Xu, “Towards distillation\-resistant large language models: An information\-theoretic perspective,”*arXiv preprint arXiv:2602\.03396*, 2026\.
- \[55\]G\. Pereyra, G\. Tucker, J\. Chorowski, Ł\. Kaiser, and G\. Hinton, “Regularizing neural networks by penalizing confident output distributions,”*arXiv preprint arXiv:1701\.06548*, 2017\.
- \[56\]S\. A\. Balanya, J\. Maronas, and D\. Ramos, “Adaptive temperature scaling for robust calibration of deep neural networks,”*Neural Computing and Applications*, vol\. 36, no\. 14, pp\. 8073–8095, 2024\.
- \[57\]T\. Joy, F\. Pinto, S\.\-N\. Lim, P\. H\. Torr, and P\. K\. Dokania, “Sample\-dependent adaptive temperature scaling for improved calibration,” in*Proceedings of the AAAI Conference on Artificial Intelligence*, vol\. 37, no\. 12, 2023, pp\. 14 919–14 926\.
- \[58\]A\. S\. Mozafari, H\. S\. Gomes, W\. Leão, S\. Janny, and C\. Gagné, “Attended temperature scaling: a practical approach for calibrating deep neural networks,”*arXiv preprint arXiv:1810\.11586*, 2018\.
- \[59\]X\.\-C\. Li, W\.\-S\. Fan, S\. Song, Y\. Li, S\. Yunfeng, D\.\-C\. Zhan*et al\.*, “Asymmetric temperature scaling makes larger networks teach well again,”*Advances in neural information processing systems*, vol\. 35, pp\. 3830–3842, 2022\.
- \[60\]R\. I\. of Technology, “Research computing services,” 2026\. \[Online\]\. Available:[https://www\.rit\.edu/researchcomputing/](https://www.rit.edu/researchcomputing/)

\{IEEEbiography\}

\[![[Uncaptioned image]](https://arxiv.org/html/2607.15467v1/photo/Khawaja.jpg)\] Khawaja Abaid Ullah received the B\.S\. degree in Computer Science from the University of Narowal, Pakistan in 2021, and the M\.S\. degree in Artificial Intelligence from the Rochester Institute of Technology \(RIT\), Rochester, NY, USA, in 2026, as a Fulbright Scholar\. He is currently pursuing the Ph\.D\. degree in Electrical and Computer Engineering at RIT, serving as a Graduate Research Assistant in the Khojasteh Autonomous Systems \(KAS\) Lab\. His research interests include deep learning, robotics, and autonomous systems\.

\{IEEEbiography\}

\[![[Uncaptioned image]](https://arxiv.org/html/2607.15467v1/photo/Mohammad.jpg)\]Mohammad Javad Khojasteh \(Member, IEEE\) received the Ph\.D\. and M\.Sc\. degrees in electrical and computer engineering from the University of California, San Diego in 2019 and 2017, respectively\. He is a Gleason Endowed Assistant Professor with the Rochester Institute of Technology \(RIT\)\. Before joining RIT, he held postdoctoral positions at Marine Physical Laboratory \(MPL\) at Scripps Institution of Oceanography \(SIO\), Department of Mechanical Engineering and Laboratory for Information and Decision Systems \(LIDS\) at Massachusetts Institute of Technology \(MIT\), and Center for Autonomous Systems and Technologies \(CAST\) at California Institute of Technology \(Caltech\), where he worked with Team CoSTAR as visitor at NASA Jet Propulsion Laboratory\. He received the Gleason Chair in 2024 and the 2025 Provost’s Learning Innovation Grant at RIT\. Dr\. Khojasteh’s publications, co\-authored with colleagues and students, have received awards, including Tammy L\. Blair Student Paper Award \(second place\) from the International Society of Information Fusion\. He is an Associate Editor for IEEE Signal Processing Letters\.

## APPENDIX

### 11\-ANotation

Table[3](https://arxiv.org/html/2607.15467#Sx2.T3)summarizes the symbols used throughout the paper, grouped by role\.

Table 3:Summary of notation\.
### 11\-BCalibration of the finite\-difference stepε\\varepsilon

Following the ADS paper\[[24](https://arxiv.org/html/2607.15467#bib.bib24)\], the penalty \([3](https://arxiv.org/html/2607.15467#S4.E3)\) approximates the exact directional derivative⟨∇θlog⁡p​\(y∣𝐱;θ𝒫\),g⟩\\langle\\nabla\_\{\\theta\}\\log p\(y\\mid\\mathbf\{x\};\\theta\_\{\\mathcal\{P\}\}\),\\,g\\rangle, which we compute directly as a Jacobian–vector product \(JVP\) on a reference batch\. We selectε\\varepsilonby sweeping over a log\-grid and scoring the per\-coordinate magnitude ratio and angular agreement between \([3](https://arxiv.org/html/2607.15467#S4.E3)\) and the JVP\. Too\-smallε\\varepsilonloses the signal to floating\-point cancellation; too\-largeε\\varepsilonleaves the linear regime \(Fig\.[15](https://arxiv.org/html/2607.15467#Sx2.F15)\)\. The selected steps,ε=3×10−3\\varepsilon=3\\times 10^\{\-3\}on both CIFAR datasets andε=10−2\\varepsilon=10^\{\-2\}on Tiny\-ImageNet, each achieve a magnitude ratio of≈1\.00\\approx 1\.00under our GPU \(TF32\) inference stack; the calibration is hardware\-arithmetic\-specific and should be re\-run if the serving stack changes numerics\.

![Refer to caption](https://arxiv.org/html/2607.15467v1/x26.png)Figure 15:Calibration of the finite\-difference step\. Magnitude ratio between \([3](https://arxiv.org/html/2607.15467#S4.E3)\) and the exact JVP as a function ofε\\varepsilon\(log axis\); the enlarged marker on each curve is the selected step\. Below the plateau, floating\-point cancellation destroys the signal; above it, the finite difference leaves the linear regime\.
### 11\-CSensitivity to the margin floormm

The guarantee ismm\-independent: recomputing the full serving pipeline overm∈\{0\.01,0\.05,0\.1,0\.2\}m\\in\\\{0\.01,0\.05,0\.1,0\.2\\\}atλ∈\{1,2\}\\lambda\\in\\\{1,2\\\}, the defended teacher’s top\-1 drop is exactly0\.00​pp0\.00\\,\\mathrm\{pp\}at every setting on both datasets, as Proposition[1](https://arxiv.org/html/2607.15467#Thmproposition1)requires\. Whatmmmodulates is only how much poison the budget passes, and mildly: atλ=2\\lambda=2on CIFAR\-100 the mean effective strength moves from1\.0281\.028\(m=0\.01m=0\.01\) to1\.0091\.009\(m=0\.2m=0\.2\) and the served poison\-KL from0\.8420\.842to0\.7570\.757\(≈10%\{\\approx\}10\\%across a20×20\\timesrange ofmm\); CIFAR\-10 behaves identically \(0\.702→0\.6270\.702\\to 0\.627\)\. Atλ=1\\lambda=1the spread is smaller still\. Conclusions are therefore insensitive tomm; we fixm=0\.05m=0\.05as a mid\-range served\-margin service level\. \(Rows whose original margin is belowmmreceive zero poison and keep their original margin, per themin⁡\(m,original\)\\min\(m,\\text\{original\}\)guarantee—hence the minimum achieved served margin can lie belowmm\.\) A student\-side confirmation \(distilled students atm∈\{0\.01,0\.05,0\.1,0\.2\}m\\in\\\{0\.01,0\.05,0\.1,0\.2\\\},λ∈\{1,2\}\\lambda\\in\\\{1,2\\\}, three seeds per cell\) shows the same insensitivity where it matters \(Fig\.[16](https://arxiv.org/html/2607.15467#Sx2.F16)\): student top\-1 varies by at most1\.7​pp1\.7\\,\\mathrm\{pp\}across the20×20\\timesrange ofmmon CIFAR\-100 \(λ=2\\lambda=2:32\.3%32\.3\\%atm=0\.01m=0\.01versus33\.9%33\.9\\%atm=0\.2m=0\.2\) and by at most2\.9​pp2\.9\\,\\mathrm\{pp\}on CIFAR\-10 \(53\.7%53\.7\\%versus56\.6%56\.6\\%\), against a poison\-induced degradation of1010–27​pp27\\,\\mathrm\{pp\}at theseλ\\lambda\. Larger floors admit marginally less poison, as expected, and the defended teacher’s top\-1 drop remains exactly zero at every cell\.

![Refer to caption](https://arxiv.org/html/2607.15467v1/x27.png)\(a\)CIFAR\-100
![Refer to caption](https://arxiv.org/html/2607.15467v1/x28.png)\(b\)CIFAR\-10

Figure 16:Student\-side sensitivity to the margin floormm\(λ∈\{1,2\}\\lambda\\in\\\{1,2\\\}, 3 seeds\)\. Distilled\-student top\-1 varies by at most1\.7​pp1\.7\\,\\mathrm\{pp\}\(CIFAR\-100\) and2\.9​pp2\.9\\,\\mathrm\{pp\}\(CIFAR\-10\) over the20×20\\timesrangem∈\[0\.01,0\.2\]m\\in\[0\.01,0\.2\], against a poison\-induced degradation of1010–27​pp27\\,\\mathrm\{pp\}at these strengths; the defended teacher’s top\-1 drop is exactly zero at every cell\.
### 11\-DSoftening operators used in the ablation

The ablation of Sec\.[8\-D](https://arxiv.org/html/2607.15467#S8.SS4)uses a gated, argmax\-preserving softener defined in log\-odds space\. LetG=zt−logsumexpj≠t⁡zjG=z\_\{t\}\-\\operatorname\{logsumexp\}\_\{j\\neq t\}z\_\{j\}, which equalslogit⁡\(pmax\)\\operatorname\{logit\}\(p\_\{\\max\}\)exactly\. Given a gatepgp\_\{g\}withGg=logit⁡\(pg\)G\_\{g\}=\\operatorname\{logit\}\(p\_\{g\}\), inputs withG≤GgG\\leq G\_\{g\}are untouched; over\-confident inputs are mapped to the served confidenceG′=GgG^\{\\prime\}=G\_\{g\}by lowering the top logit only \(zt←G′\+logsumexpj≠t⁡zjz\_\{t\}\\leftarrow G^\{\\prime\}\+\\operatorname\{logsumexp\}\_\{j\\neq t\}z\_\{j\}\), leaving all non\-target structure intact for the dark\-knowledge channel\. We sweeppg∈\{0\.5,0\.6,0\.7,0\.8,0\.9\}p\_\{g\}\\in\\\{0\.5,0\.6,0\.7,0\.8,0\.9\\\}; the constant\-temperature comparison is Sec\.[6](https://arxiv.org/html/2607.15467#S6)\(E\-CT\)\. The operator is inspired in spirit by asymmetric temperature scaling\[[59](https://arxiv.org/html/2607.15467#bib.bib59)\], which applies separate temperatures to the correct and wrong classes so that wrong\-class discriminability survives softening; ours is the gated, serving\-side limiting case in which the wrong\-class logits are left exactly unchanged\. Additionally, where asymmetric temperature scaling selects a fixed pair of temperatures by grid search over a candidate space, our operator adapts the effective softening per query\.

### 11\-EReproducibility

Table[4](https://arxiv.org/html/2607.15467#Sx2.T4)reports the full training recipes\. All models are trained with SGD \(momentum0\.90\.9, Nesterov, weight decay10−410^\{\-4\}\) under cosine annealing; teachers and proxies use a 5\-epoch linear warmup, plain cross\-entropy, and*no*label smoothing\. Teachers and proxies train on the augmented training split \(random crop with 4\-pixel padding at32×3232\\times 32, 8\-pixel at64×6464\\times 64, plus horizontal flip\); the Tiny\-ImageNet teacher is an ImageNet\-pretrained ResNet\-50 fine\-tuned end to end with the stem adapted to64×6464\\times 64\(first\-conv stride2→12\\to 1, max\-pool removed, fresh 200\-way head\)\. The clone gradientggis computed on a held\-out10%10\\%of the training split\. The attacker’s student trains on the cached \(image, served\-vector\) pairs with no augmentation\. Every figure is generated by a script that emits a sibling metrics JSON with the underlying numbers, and aggregates for every \(variant,λ\\lambda, seed\) cell are archived\. Code, configuration files, and exact command lines:[https://github\.com/the\-kas\-lab/ADS\-C](https://github.com/the-kas-lab/ADS-C)\.

Table 4:Training recipes \(SGD, momentum0\.90\.9, Nesterov, weight decay10−410^\{\-4\}, cosine annealing throughout\)\.ModelEpochsLRBatchC\-100Teacher \(ResNet\-110\)2000\.1256Proxy \(ResNet\-20\)500\.1256Student \(ResNet\-20\)500\.1512C\-10Teacher \(ResNet\-56\)2000\.1256Proxy \(ResNet\-20\)500\.1256Student \(ResNet\-20\)500\.1512Tiny\-INTeacher \(ResNet\-50†\)400\.02128Proxy \(ResNet\-18\)600\.1128Student \(ResNet\-18\)500\.1256†ImageNet\-pretrained, fine\-tuned end to end at64×6464\\times 64\.