Reducing information dependency does not cause training data privacy. Adversarially non-robust features do

# Reducing information dependency does not cause training data privacy. Adversarially non-robust features do.
Source: [https://arxiv.org/html/2607.12354](https://arxiv.org/html/2607.12354)
\\declaretheorem

\[name=Theorem, sibling=theorem\]rThm\\declaretheorem\[name=Lemma, sibling=theorem\]rLem\\declaretheorem\[name=Corollary, sibling=theorem\]rCor

Rasmus Torp Department of Computer Science Dartmouth College rasmus\.torp\.gr@dartmouth\.edu &Shailen K\. Smith11footnotemark:1 Department of Computer Science Dartmouth College shailen\.k\.smith\.gr@dartmouth\.edu &Adam Breuer222Breuer Lab gratefully acknowledges the support of the OpenAI Cybersecurity Grant\.Replication code is available at[https://github\.com/BreuerLabs/Anti\-Adversarial\-Training](https://github.com/BreuerLabs/Anti-Adversarial-Training) Department of Computer Science & Department of Government Dartmouth College adam\.breuer@dartmouth\.edu

###### Abstract

In this paper, we challenge the prevailing view that information dependency \(including rote memorization\) drives training data exposure to image reconstruction attacks\. We show that extensive exposure can persist without rote memorization and is instead caused by a tunable connection to adversarial robustness\. We begin by presenting three surprising results: \(1\) recent defenses that inhibit reconstruction by Model Inversion Attacks \(MIAs\), which evaluate leakage under an idealized attacker, do*not*reduce standard measures of information dependency \(HSIC\); \(2\) models that maximally memorize their training datasets remainrobustto MIA reconstruction; and \(3\) models trained without seeing 97% of the training pixels, where recent information\-theoretic bounds give arbitrarily strong privacy guarantees under standardassumptions, can still be devastatingly reconstructed by MIA\.

To explain these findings, we provide causal evidence thatprivacyunder MIA arises from what the adversarial examples literature calls “non\-robust” features \(generalizable but imperceptible and unstable features\)\. We further show that recent MIA defenses obtain their privacy improvements by unintentionally shifting models toward such features\. To establish this causal relationship, we introduceAntiAdversarialTraining\(AT\-AT![[Uncaptioned image]](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ATATiconV4.png)\), a training regime thatintentionallylearns non\-robust features to obtain both superior reconstruction defense and higher accuracy than state\-of\-the\-art defenses\. Our results revise the prevailing understanding of training data exposure and reveal a new privacy\-robustness tradeoff\.

## 1Introduction

A foundational goal of contemporary machine learning is to learn*generalizable*model parameters without encoding the private training dataset in a manner that*exposes*it to leakage\. However, a rapidly expanding class of practical privacy attacks has demonstrated reconstruction of private training data in a variety of learning settings and model architectures\. Recent variants of training data privacy attacks apply not only to CNNs\(Struppeket al\.,[2022](https://arxiv.org/html/2607.12354#bib.bib36); Carliniet al\.,[2022](https://arxiv.org/html/2607.12354#bib.bib147); Mehnazet al\.,[2022](https://arxiv.org/html/2607.12354#bib.bib13); Yeomet al\.,[2018](https://arxiv.org/html/2607.12354#bib.bib338); Shokriet al\.,[2017](https://arxiv.org/html/2607.12354#bib.bib136)\), but also to LLMs\(Shiet al\.,[2024](https://arxiv.org/html/2607.12354#bib.bib15); Nasret al\.,[2023](https://arxiv.org/html/2607.12354#bib.bib350); Carliniet al\.,[2021](https://arxiv.org/html/2607.12354#bib.bib14)\)and diffusion models\(Carliniet al\.,[2023](https://arxiv.org/html/2607.12354#bib.bib76)\)\.

Across this attack landscape, model inversion attacks \(MIAs\) have emerged as a mainstream tool for studying and evaluating training data exposure, particularly in the high\-resolution image classification domain\. Unlike more conservative attacks that merely probe for the presence of some exposed training examples, MIAs test the degree to which a powerful attacker armed with white\-box model access and significant computational and data resources can reconstruct information from the training examples\. While early MIAs applied SGD to directly optimize reconstructions of individual training examples\(Fredriksonet al\.,[2015](https://arxiv.org/html/2607.12354#bib.bib218)\), contemporary MIAs use sophisticated gradient techniques and external data to attempt to infer the full set of class\-level characteristics for each class in the target model’s training data\(Qiuet al\.,[2024](https://arxiv.org/html/2607.12354#bib.bib300); Struppeket al\.,[2022](https://arxiv.org/html/2607.12354#bib.bib36); Haimet al\.,[2022](https://arxiv.org/html/2607.12354#bib.bib30)\)\.

However, while MIAs lower\-bound the extent to which contemporary vision models expose their training data to reconstruction, they do not explain what aspect of a model encodes this vulnerability, or how to prevent it\. This raises a fundamental question:

> What properties of learned representations encode vulnerability to training data leakage and reconstruction, and how can they be controlled? ⋄⁣⋄⁣⋄\\diamond\\ \\diamond\\ \\diamond

Understanding these properties has far\-reaching implications for learning that extend beyond the privacy domain\. For example, influential recent work conjectures that obtaining stronger learning performance may require*more*extensive model\-to\-training\-data dependencies, including rote memorization and exposure of the training set, not only for vanilla CNNs\(Brownet al\.,[2021](https://arxiv.org/html/2607.12354#bib.bib366); Feldman,[2020](https://arxiv.org/html/2607.12354#bib.bib141)\), but also for recent paradigms including LLMs\(Duet al\.,[2025](https://arxiv.org/html/2607.12354#bib.bib33); Tirumalaet al\.,[2022](https://arxiv.org/html/2607.12354#bib.bib48)\), diffusion\(e\.g\., Shahet al\.,[2025](https://arxiv.org/html/2607.12354#bib.bib80); Carliniet al\.,[2023](https://arxiv.org/html/2607.12354#bib.bib76)\), and self\-supervised learning\(Wanget al\.,[2024](https://arxiv.org/html/2607.12354#bib.bib49)\)\.

The prevailing ‘training data information dependency→\\rightarrowtraining data reconstruction’ theory\.An influential recent line of algorithms has obtained practically performant defenses against high\-resolution MIA image reconstruction\. Their practical performance is widely attributed to their reasonable theoretic hypothesis that training data privacy leakage arises from direct and excessive information dependency between training inputs and the model’s internal representations or outputs\(Dibboet al\.,[2024](https://arxiv.org/html/2607.12354#bib.bib62); Penget al\.,[2022](https://arxiv.org/html/2607.12354#bib.bib303); Wanget al\.,[2020](https://arxiv.org/html/2607.12354#bib.bib63)\)\. To mitigate this hypothesized dependency, Mutual Information Defense \(MID\)\(Wanget al\.,[2020](https://arxiv.org/html/2607.12354#bib.bib63)\)introduces a regularizer that reduces mutual information between the training examples and intermediate feature representations, arguing that this “limit\[s\] the redundant information about the model input contained in the prediction” so that MIAs cannot “exploit this correlation\.” Building on this idea, the popular BiDO algorithm\(Penget al\.,[2022](https://arxiv.org/html/2607.12354#bib.bib303)\)penalizes a kernel\-based information dependency measure \(HSIC\) between training examples and intermediate embeddings, arguing that this “limits redundant information propagated from the inputs to the latent representations that can be exploited by the \[MIA\] adversary\.”

The same hypothesis appears again inSCA\(Dibboet al\.,[2024](https://arxiv.org/html/2607.12354#bib.bib62)\), which argues that the relevant MIA training data reconstruction vulnerability is reduced by directly “jettison\[ing\] unnecessary private information in the input image,” e\.g\. by sparse\-coding training images before they are seen by the model, and by also “jettison\[ing\]…unnecessary private information from downstream layers” to attempt to preclude high\-fidelity memorization of the inputs, rather than disincentivizing it via the loss function as inPenget al\.\([2022](https://arxiv.org/html/2607.12354#bib.bib303)\)andWanget al\.\([2020](https://arxiv.org/html/2607.12354#bib.bib63)\)\.

Similarly, the highly performant Transfer Learning \(TL\-DMI\) approach\(Hoet al\.,[2024](https://arxiv.org/html/2607.12354#bib.bib45)\)advances a main hypothesis that “by reducing the number of parameters fine\-tuned with private dataset, \[TL\-DMI\] could reduce the amount of private information encoded in the model, making it more difficult for adversaries to reconstruct private training data\.” This hypothesis is widely credited with the strong practical performance of TL\-DMI, an approach that uses training data only for fine\-tuning\.

In sum, state\-of\-the\-art privacy defenses advance a shared and intuitively reasonable theoretic argument: they argue that they increase the generic privacy of their training data by reducing information dependency between training examples and the target model’s internal representations or outputs\.

Our contribution I: Challenging the theoretic bases of MIA defenses\.

In this paper, we challenge this foundational hypothesis\. Specifically, we motivate our techniques by first presenting three surprising experiments that indicate that the information dependency view is inadequate to explain MIA reconstruction vulnerability:

1\.We show that recent defenses that successfully inhibit reconstruction by MIAs do*not*reduce the approximation of information dependency, HSIC, used in MIA research\. In fact, when we modify defenses such that they do reduce HSIC, they cease to provide effective reconstruction defense\.

2\.We train models to perfectly rote\-memorize the training data such that their data\-to\-model information dependencies are intuitively maximal \(at least in one intuitive view of dependency\), yet we show that such ‘max\-information\-dependency’ models are*robust*to MIA reconstructions\.

3\.We train models without seeing\>\>97% of each training image’s pixels, such that information dependencies are minimal \(at least in one intuitive view of dependency\)\. Recent information\-theoretic bounds give these\>\>97% pixels arbitrarily strong privacy guarantees under standard assumptions\. Yet, we show that this training data can still be devastatingly reconstructed in practice: deleting\>\>97% of the training data yields*no significant privacy increase*against MIA reconstruction\.

In sum, our motivating experiments formally show that the main theoretic approach in privacy/MIA defense is incorrect: reducing information dependency does not cause privacy as widely believed\. SOTA defenses thus obtain their impressive performance by a different theoretic mechanism\.

Our contribution II: The privacy\-robustness feature tradeoff\.

To explain these results, we provide evidence that privacy under MIA arises instead from what the adversarial examples literature calls “non\-robust” features \(generalizable but imperceptible and unstable features\), and recent MIA defenses obtain their privacy improvements by unintentionally shifting models toward such features\. To accomplish this, we conduct the first systematic evaluation of the robustness of privacy/MIA defenses to adversarial examples\(Szegedyet al\.,[2014](https://arxiv.org/html/2607.12354#bib.bib60); Ilyaset al\.,[2019](https://arxiv.org/html/2607.12354#bib.bib101)\)\. We find that all recent defenses that attempt to reduce information dependency \(MID, BiDO, TL\-DMI\) obtain a reduction in privacy leakage under MIA that is directly proportional to a corresponding increase in their vulnerability to adversarial examples\.

We then explicitly test our hypothesis that these defenses’ privacy leakage reduction \(measured by AttAcc@1 under PPA or IF\-GMI attacks\) is a linear function of their robust accuracy \(or, equivalently, a linear function of their reliance on non\-robust features\)\. We find that we can almost perfectly predict each of these defenses’ MIA robustness via a simple linear function of their vulnerability to adversarial examples \(R2R^\{2\}==0\.95\)\. As such, we can compute the*adversarial robustness cost of MIA privacy gain\.*For example, reducing MIA leakage by 1 percentage point corresponds to a statistically significant 0\.31\-5\.4 percentage point decrease in robust accuracy, depending on the magnitude of the adversarial example chosen, where MIA leakage is measured via the standardAttAcc@1metric using a standard PPA attack\.

As such, MIA defenses unintentionally make a tradeoff: by shifting towards non\-robust features, they increase privacy by stripping semantically meaningful information from the model, but this opens significant new vulnerabilities to adversarial examples\.

Our contribution III:Anti\-AdversarialTraining \(AT\-AT![[Uncaptioned image]](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ATATiconV4.png)\)\.

If recent privacy/MIA defenses appear to work by unintentionally ‘nonrobustifying’ the model’s feature space, then it stands to reason that we can*causally induce*superior defense performance by deliberately exploiting this mechanism\. To establish this causal relationship, we introduceAntiAdversarialTraining \(AT\-AT![[Uncaptioned image]](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ATATiconV4.png)\), a novel generic privacy/MIA defense training regime that balances model accuracy with gradient steps that reverse standard adversarial training\(Madryet al\.,[2017](https://arxiv.org/html/2607.12354#bib.bib246)\)\. Rather than promoting robustness to adversarial perturbations,AT\-ATapplies targeted gradient updates that suppress reliance on robust features\. This compels the model to instead learn non\-robust \(but still generalizable\) features that are semantically meaningless to humans and thereby hinder visual reconstruction under MIA\.

In a causal framework, we show that ‘treating’ a standard ResNet\-152 withAT\-AT’s non\-robust feature reward causes a reduction in reconstruction rate \(AttAcc@1\) from 84% to 6\.5% \(pp<<10−1610^\{\-16\}\)\.

We then show that our anti\-adversarial approach obtains superior MIA defense at higher levels of accuracy than baselines across the standard architectures, datasets, and reconstruction metrics in the high\-resolution MIA domain\. As with other generic defenses,AT\-AT’s performance comes at the cost of increased vulnerability to adversarial examples\. However, in our case, this vulnerability cost is a tunable parameter, offering a new design axis for private models\.

Implications for private learning\.Our results have four implications\. First, among the most celebrated results in ML is the finding that adversarial examples do not reveal bugs, but rather non\-robust features that are generalizable and discriminatory of the classes we want to learn, yet visually imperceptible to humans\(Ilyaset al\.,[2019](https://arxiv.org/html/2607.12354#bib.bib101)\)\. Our argument in this paper is that these properties make them ideal instruments for privacy\-preserving learning, where our goal is to classify accurately without encoding visually meaningful information that is vulnerable to visual reconstructions\.

Second, eliminating training data information dependency does not guarantee privacy, as leakage can persist in its absence\. Indeed, if recent work on learning performance is correct in suggesting that dependencies up to and including rote memorization can confer accuracy benefits, then such benefits may be attainable without incurring all of the privacy risks it is widely assumed to entail\.

On the other hand, our results corroborate the recent idea that there may be an inherent privacy vulnerability associated with robust learned representations\. As such, robust models may be inherently privacy compromising, beyond any specific aspects of how they encode data\. This intuition is consistent with recent works analyzing connections between privacy and robustness \(see App\.[F\.2](https://arxiv.org/html/2607.12354#A6.SS2)\)\.

Finally, our results suggest that the assumptions of recent theoretical privacy guarantees may be optimistic, rendering their protections practically inapplicable against recent MIAs\.

## 2Experimental setup, scope, and replication

The experiments throughout this paper focus on the white\-box high\-resolution vision setting that is central to contemporary empirical privacy and MIA research \(see App\.[C](https://arxiv.org/html/2607.12354#A3)for further scope details\)\. Specifically, our experiments evaluate the facial recognition setting on the standard hi\-res image MIA benchmark datasets \(*FaceScrub*\(Ng and Winkler,[2014](https://arxiv.org/html/2607.12354#bib.bib104)\),*CelebA*\(Liuet al\.,[2015](https://arxiv.org/html/2607.12354#bib.bib87)\)\) under the established benchmarks for hi\-res white\-box attacks \(*PPA*\(Struppeket al\.,[2022](https://arxiv.org/html/2607.12354#bib.bib36)\), plus recent, high\-performance*IF\-GMI*\(Qiuet al\.,[2024](https://arxiv.org/html/2607.12354#bib.bib300)\)and*PPDG*\(Penget al\.,[2024](https://arxiv.org/html/2607.12354#bib.bib9)\)\), on the standard high\-resolution image architectures \(*ResNet\-152*,*ResNet\-18*,*DenseNet\-169*\) that are the focus of recent experiments in the literature\(Struppeket al\.,[2022](https://arxiv.org/html/2607.12354#bib.bib36); Qiuet al\.,[2024](https://arxiv.org/html/2607.12354#bib.bib300); Struppeket al\.,[2024](https://arxiv.org/html/2607.12354#bib.bib95); Hoet al\.,[2024](https://arxiv.org/html/2607.12354#bib.bib45); Kohet al\.,[2024](https://arxiv.org/html/2607.12354#bib.bib268); Liu and Chen,[2024](https://arxiv.org/html/2607.12354#bib.bib10); Penget al\.,[2022](https://arxiv.org/html/2607.12354#bib.bib303)\)\. Section[4](https://arxiv.org/html/2607.12354#S4)adds robustness metrics from four SOTA adversarial example attacks\(Croce and Hein,[2020a](https://arxiv.org/html/2607.12354#bib.bib46);[b](https://arxiv.org/html/2607.12354#bib.bib27); Andriushchenkoet al\.,[2020](https://arxiv.org/html/2607.12354#bib.bib20)\), each under various attackstrengthsϵ\\mathbf\{\\epsilon\}\. App\.[F\.3](https://arxiv.org/html/2607.12354#A6.SS3)gives experimental details for adversarial attacks\. In App\.[I](https://arxiv.org/html/2607.12354#A9)\(Table[26\(b\)](https://arxiv.org/html/2607.12354#A9.T26.st2)\) we also consider an extra*Stanford Dogs*dataset\(Khoslaet al\.,[2011](https://arxiv.org/html/2607.12354#bib.bib41)\)for variety\.

Defense Baselines: generic information defenses and gradient\-suppressing defenses\.As discussed above, our main focus is on class of state\-of\-the\-art*generic defenses*that seek to hinder reconstruction via private information reduction\. This includes*MID*\(Wanget al\.,[2020](https://arxiv.org/html/2607.12354#bib.bib63)\),*BiDO*\(Penget al\.,[2022](https://arxiv.org/html/2607.12354#bib.bib303)\), and the recent*TL\-DMI*\(Hoet al\.,[2024](https://arxiv.org/html/2607.12354#bib.bib45)\)\. For the sake of comparison, we also add four main recent*attack gradient suppressing*MIA defenses,*NegLS*\(Struppeket al\.,[2024](https://arxiv.org/html/2607.12354#bib.bib95)\),*RoLSS*,*RoLSS\-SSF*\(Kohet al\.,[2024](https://arxiv.org/html/2607.12354#bib.bib268)\),*Trap\-MID*\(Liu and Chen,[2024](https://arxiv.org/html/2607.12354#bib.bib10)\)\. These defenses do not seek generic privacy, but rather seek to inhibit the gradient steps taken by modern MIAs during their optimization stages\. See App\.[A](https://arxiv.org/html/2607.12354#A1)for experimental details of target models, datasets, and defense baselines\.

Privacy Metrics\.As in recent work, we measure privacy leakage via the main*Attack Acc@1*metric, which reports the accuracy of an external Inception\-v3 model at guessing the identity of the attacker’s reconstructed image as its \#1 guess\. Where appropriate, we also report the similar*Attack Acc@5*metric;*L2\-FaceNet Distance*\(the minimumℓ2\\ell\_\{2\}\-norm FaceNet embedding distance between the attacker’s reconstruction and any training image in the attacked class\); and the*Average Evaluation Confidence*of the external Inception\-v3 model that the reconstructed image is from the respective original image’s class\. We also show*qualitative reconstruction*comparisons\.

This yields77defenses×\\times33attacks×\\times33architectures×\\times33datasets×\\times\(44robustness\+44privacy metrics\)\.

## 3Three experiments overturn the dependency→\\rightarrowleakage view

If training data exposure to MIA were explained by information dependency/memorization, then three intuitive hypotheses would hold: \(1\) Effective defenses would*reduce*information dependency metrics; \(2\) Models that fully rote\-memorize the training data would be*vulnerable*to MIA; \(3\) Models trained without seeing 97\.5% of each training image’s pixels \(i\.e\. where memorization is upper bounded at 2\.5%\) would be*robust*to MIA reconstruction\. Our goal in this section is to show experimentally that each of these hypotheses is false\. Thus, the prevailing view that information dependency drives training data exposure to reconstruction is incomplete\.

1\. Effective defenses do*not*reduce information dependency metrics\.First, we show that contemporary defenses that are widely believed to work by reducing training\-data\-to\-model dependency do*not*in fact reduce this dependency, and may in fact increase it, in some cases even according to their own metrics\. In the defense literature, the Hilbert–Schmidt Independence Criterion \(HSIC\)\(Grettonet al\.,[2005](https://arxiv.org/html/2607.12354#bib.bib50)\)is the standard dependency metric that can be computed for an arbitrary model\. Defenses such as BiDO and BiDO\+\(Penget al\.,[2022](https://arxiv.org/html/2607.12354#bib.bib303);[2025](https://arxiv.org/html/2607.12354#bib.bib210)\)explicitly penalize \(approximated\) HSIC between inputsXXand intermediate embeddingsZjZ\_\{j\}:

HSIC​\(X,Zj\)=‖𝔼​\[ϕ​\(X\)​ψ​\(Zj\)⊤\]−𝔼​\[ϕ​\(X\)\]​𝔼​\[ψ​\(Zj\)\]⊤‖H​S2\\text\{HSIC\}\(X,Z\_\{j\}\)=\\left\\lVert\\mathbb\{E\}\[\\phi\(X\)\\psi\(Z\_\{j\}\)^\{\\top\}\]\-\\mathbb\{E\}\[\\phi\(X\)\]\\mathbb\{E\}\[\\psi\(Z\_\{j\}\)\]^\{\\top\}\\right\\rVert\_\{HS\}^\{2\}Where nonlinear feature transformationsϕ,ψ\\phi,\\psiare created such thatϕ​\(X\)\\phi\(X\),ψ​\(Zj\)\\psi\(Z\_\{j\}\)are members of reproducing kernel Hilbert spaces, and∥⋅∥H​S2\\left\\lVert\\cdot\\right\\rVert\_\{HS\}^\{2\}denotes the Hilbert\-Schmidt norm\.

We train all defenses described in Section[2](https://arxiv.org/html/2607.12354#S2), then measure their average HSIC across the train set at all standard intermediate embeddingsZiZ\_\{i\}using the exact HSIC implementation fromPenget al\.\([2022](https://arxiv.org/html/2607.12354#bib.bib303)\)\. We then run standard PPA attacks to test their leakage \(AttAcc@1\) under MIA\.

Table 1:HSIC\(X,Zj\)\(X,Z\_\{j\}\)for defenses on FaceScrub \(ResNet\-152\)\. PPAAttAcc@1measures reconstruction\.Arch\.DefenseTestAcc↑\\uparrowAttAcc@1↓\\downarrowHSIC\(X,Z0\)\(X,Z\_\{0\}\)HSIC\(X,Z1\)\(X,Z\_\{1\}\)HSIC\(X,Z2\)\(X,Z\_\{2\}\)HSIC\(X,Z3\)\(X,Z\_\{3\}\)ResNet\-152NoDef0\.9580\.9580\.8820\.88271\.6671\.6672\.3272\.3272\.7272\.7272\.7272\.72MID0\.9500\.9500\.3910\.39171\.5271\.5272\.5772\.5772\.8472\.8461\.9861\.98BiDO0\.9280\.9280\.7800\.78073\.6873\.6873\.6873\.6873\.6873\.6873\.6373\.63TL\-DMI0\.9110\.9110\.1900\.19071\.7371\.7372\.5272\.5272\.9372\.9372\.5872\.58NegLS0\.9060\.9060\.1600\.16071\.6471\.6472\.3372\.3372\.2972\.2973\.6373\.63RoLSS0\.8860\.8860\.4740\.47472\.0972\.0972\.3072\.3029\.6929\.6961\.9261\.92RoLSS\-SSF0\.8690\.8690\.3430\.34371\.8871\.8872\.0272\.0232\.7532\.7564\.9464\.94Trap\-MID0\.9260\.9260\.4010\.40169\.8069\.8071\.8871\.8872\.6272\.6269\.2869\.28BiDO\*\*0\.9400\.9400\.8150\.8152\.1592\.1591\.2561\.2569\.3109\.31071\.8171\.81

HSIC Results: reducing HSIC does not cause privacy\.Table[1](https://arxiv.org/html/2607.12354#S3.T1)reports results \(also see the additional results in App\.[D](https://arxiv.org/html/2607.12354#A4)\)\. Neither BiDO nor the defenses TL\-DMI and NegLS that obtain the largest reductions in reconstruction \(AttAcc@1\) significantly decrease HSIC\(X,Zi\)\(X,Z\_\{i\}\)according to the standard HSIC approximation used in MIA\.

BiDO also allows users to directly manipulate the weightλx\\lambda\_\{x\}on HSIC in its loss\. We use this to experimentally lower HSIC by training a BiDO\*\* model with a differentλx\\lambda\_\{x\}\(see App\.[D\.1](https://arxiv.org/html/2607.12354#A4.SS1)\)\. This experimental BiDO\*\* ‘HSIC treatment’ model succeeds in lowering HSIC\(X,Zi\)\(X,Z\_\{i\}\)compared to standard BiDO parameters, indicating significantly reduced information dependency, yet it obtains*worse*defense\. In sum, reducing HSIC as measured in MIA does not cause privacy under MIA\.

Table 2:PPA results with and without label permutation\.Arch\.LabelsTrainAcc↑\\uparrowTestAcc↑\\uparrowL2\-Face↑\\uparrowRN\-152Not permuted1\.0001\.0000\.9580\.9580\.7680\.768Permuted0\.9960\.9960\.0010\.0011\.2491\.249RN\-18Not permuted1\.0001\.0000\.9500\.9500\.7810\.781Permuted0\.9940\.9940\.0010\.0011\.2511\.251

2\. A model that fully rote\-memorizes the dataset is*not*vulnerable to reconstruction\.Second, if training\-data\-to\-model information dependency drives leakage, then a model that*maximally*memorizes should be maximally vulnerable\. The seminal permuted\-label setting ofZhanget al\.\([2017](https://arxiv.org/html/2607.12354#bib.bib121)\)provides a clean test of this hypothesis: when training labels are randomized, networks are known to achieve perfect training accuracy by rote\-memorizing the entire training dataset without learning any meaningful features\. Table[2](https://arxiv.org/html/2607.12354#S3.T2)shows that MIAs completely fail in this setting: reconstructions collapse, yielding high L2 reconstruction distance \(note that other standard metrics such as AttAcc@1 are not well\-defined in the permuted setting\)\. Despite full rote memorization, leakage is minimal\. This suggests that leakage is not driven by rote training data information dependency, but rather by some other property\. See App\.[A\.3\.1](https://arxiv.org/html/2607.12354#A1.SS3.SSS1)for further details\.

3\. Unseen training data can still be reconstructed under MIA, contra recent theoretic bounds\.Third, if information dependency/memorization drives privacy leakage, then deleting the training data would prevent leakage \(notwithstanding the effect on accuracy\)\. However, we now show that models trained without seeing\>\>97% of the training data pixels, for which recent information\-theoretic privacy results give arbitrarily large bounds against reconstruction under standard assumptions, can still be devastatingly reconstructed by MIAs\. We construct a novel lasso\-based technique that deletes over 97% of the pixels in each training image*prior*to training \(so they are completely unseen by the model\), yet allows us to train models with test accuracy that is within a few points of the accuracy of a vanilla model trained on uncensored images\. Fig\.[1](https://arxiv.org/html/2607.12354#S3.F1)shows examples of pixel\-deleted images used for training\. App\.[E\.2](https://arxiv.org/html/2607.12354#A5.SS2)describes our deletion technique\.

Unseen pixel leakage is theoretically impossible under standard unbiasedness assumptions\.A recent stream of theory research seeks information\-theoretic privacy guarantees against image training data reconstruction by lower\-bounding the variance of any unbiased estimator of the train set via, e\.g\., the Cramér\-Rao bound and Fisher Information Loss\(Hannunet al\.,[2021](https://arxiv.org/html/2607.12354#bib.bib51); Guoet al\.,[2022](https://arxiv.org/html/2607.12354#bib.bib344)\), or the HCR bound\(Guoet al\.,[2022](https://arxiv.org/html/2607.12354#bib.bib344); Chaudhuriet al\.,[2024](https://arxiv.org/html/2607.12354#bib.bib113)\)\. Interestingly, in the same unbiased estimator setting, our model trained on\>\>97% pixel\-deleted data enjoys a perfect privacy guarantee \(unbounded reconstruction variance\) on these\>\>97% deleted pixels \(see App\.[E\.3](https://arxiv.org/html/2607.12354#A5.SS3)\)\.

\>\\mathbf\{\>\}97% Unseen\-Pixels experiment\.Table[3](https://arxiv.org/html/2607.12354#S3.T3)compares PPA reconstructions on models trained on the\>\>97% pixel\-deleted training dataset versus various models trained on the uncensored data, including a vanilla undefended model \(NoDef\) and a SOTA MIA defense \(TL\-DMI\)\. We also compare against an undefended model trained with early\-stopping to provide an extra baseline with accuracy comparable to our Unseen\-Pixel model’s accuracy, which is motivated by the common observation that less accurate models are naturally more robust to MIAs \(seeStruppeket al\.\([2024](https://arxiv.org/html/2607.12354#bib.bib95)\)\)\.

![Refer to caption](https://arxiv.org/html/2607.12354v1/x1.png)Figure 1:Orig\. & pixel\-deleted images \(*zoom in for2\.22\.2% training pixels*\)\.Unseen\-Pixels results\.Surprisingly, deleting\>\>97% of the pixels before training fails to prevent MIA reconstructions\. Table[3](https://arxiv.org/html/2607.12354#S3.T3)shows that PPA attacks on the Unseen\-Pixels model still obtain reconstructions that are classified as their original class\>\>50% of the time \(perAttAcc@1\)\. For comparison, that means the Unseen\-Pixels model trained on\>\>97% censored data is*not*significantly more private than an undefended model with about the same accuracy that is trained on uncensored data\. More specifically, it is only marginally more private in terms of AttAcc@1, and marginally*less*private in terms of the L2 metric on RN\-152\. See addl\. results in App\.[E](https://arxiv.org/html/2607.12354#A5)\. This also suggests that the assumptions of recent privacy bounds are overly optimistic, rendering their protections practically inapplicable against recent attacks \(see App\.[E\.3](https://arxiv.org/html/2607.12354#A5.SS3)\)\.

Table 3:Unseen\-Pixels trained on FaceScrub obtains very limited defense against reconstruction by PPA\.Arch\.Defense%Pix\-deletedTestAcc↑\\uparrowAttAcc@1↓\\downarrowAttAcc@5↓\\downarrowL2\-Face↑\\uparrowEvalConf↓\\downarrowRN\-152NoDef00\.9580\.9580\.8810\.8810\.9820\.9820\.7810\.7810\.8590\.859NoDef\-EarlyStop00\.9170\.9170\.6340\.6340\.8750\.8750\.8520\.8520\.6070\.607TL\-DMI00\.9110\.9110\.1630\.1630\.3980\.3981\.0721\.0720\.1520\.152Unseen\-Pixels97\.80\.9100\.9100\.5920\.5920\.8510\.8510\.8190\.8190\.5660\.566RN\-18NoDef00\.9500\.9500\.9030\.9030\.9850\.9850\.7780\.7780\.8840\.884NoDef\-EarlyStop00\.8900\.8900\.6200\.6200\.8770\.8770\.8400\.8400\.5960\.596TL\-DMI00\.9060\.9060\.2190\.2190\.4870\.4871\.0311\.0310\.2060\.206Unseen\-Pixels97\.20\.8920\.8920\.5460\.5460\.8130\.8130\.8440\.8440\.5220\.522

## 4The privacy\-adversarial example robustness tradeoff

Our goal in this section is to describe a previously unobserved privacy\-adversarial example robustness tradeoff that characterizes the performance of generic information dependency\-based MIA defenses on the main architectures and datasets used to benchmark hi\-res privacy attacks/defenses in recent work\. We show that the defense performance obtained by the class of defenses widely believed to work via information dependency reduction is highly correlated with vulnerability to adversarial examples and non\-robust feature bases\. We accomplish this by presenting the first systematic tests of the robustness of MIA defenses to a very different attack vector: adversarial examples\.

Table 4:Leakage \(AttAcc@1\) vs\. clean/robust accuracies under AutoAttack for variousϵ\\epsilon\.DataArch\.DefensePPA\-AttAcc@1↓\\downarrowIF\-AttAcc@1↓\\downarrowTestAcc↑\\uparrowϵ=0\.031↑\\mathbf\{\\epsilon\{=\}0\.031\{\\uparrow\}\}ϵ=0\.0025↑\\mathbf\{\\epsilon\{=\}0\.0025\{\\uparrow\}\}ϵ=0\.0005↑\\mathbf\{\\epsilon\{=\}0\.0005\{\\uparrow\}\}ϵ=𝟏𝟎−𝟓↑\\mathbf\{\\epsilon\{=\}10^\{\-5\}\{\\uparrow\}\}Defenses that attempt to increase privacy generically by reducing information dependency/memorizationFaceScrubRN\-152NoDef0\.8820\.8820\.9870\.9870\.9580\.9580\.0000\.0000\.1220\.1220\.8990\.8990\.9600\.960MID0\.3910\.3910\.3910\.3910\.9500\.9500\.0000\.0000\.0060\.0060\.7700\.7700\.9280\.928BiDO0\.7800\.7800\.9370\.9370\.9280\.9280\.0000\.0000\.1210\.1210\.8470\.8470\.9230\.923TL\-DMI0\.1900\.1900\.4200\.4200\.9110\.9110\.0000\.0000\.0000\.0000\.0000\.0000\.8670\.867RN\-18NoDef0\.9010\.9010\.9830\.9830\.9500\.9500\.0000\.0000\.1940\.1940\.8960\.8960\.9590\.959MID0\.7140\.7140\.7760\.7760\.9240\.9240\.0000\.0000\.0930\.0930\.8250\.8250\.8560\.856BiDO0\.5300\.5300\.7820\.7820\.8840\.8840\.0000\.0000\.0750\.0750\.7840\.7840\.8790\.879TL\-DMI0\.2400\.2400\.5150\.5150\.9060\.9060\.0000\.0000\.0000\.0000\.0030\.0030\.8860\.886CelebARN\-152NoDef0\.4970\.4970\.7930\.7930\.8380\.8380\.0000\.0000\.0500\.0500\.6750\.6750\.8180\.818MID0\.4740\.4740\.7280\.7280\.8020\.8020\.0040\.0040\.1390\.1390\.5420\.5420\.5740\.574BiDO0\.3620\.3620\.5970\.5970\.7470\.7470\.0000\.0000\.0620\.0620\.5900\.5900\.7480\.748TL\-DMI0\.0800\.0800\.2580\.2580\.6880\.6880\.0000\.0000\.0000\.0000\.0020\.0020\.6410\.641TL\-DMI\-50\.2810\.2810\.6610\.6610\.8140\.8140\.0000\.0000\.0000\.0000\.1190\.1190\.7990\.799RN\-18NoDef0\.6340\.6340\.9130\.9130\.8590\.8590\.0000\.0000\.0840\.0840\.7460\.7460\.8480\.848MID0\.3860\.3860\.5500\.5500\.7370\.7370\.0030\.0030\.1680\.1680\.4520\.4520\.4640\.464BiDO0\.1460\.1460\.3680\.3680\.6250\.6250\.0000\.0000\.0070\.0070\.3940\.3940\.6090\.609TL\-DMI0\.1530\.1530\.4390\.4390\.7520\.7520\.0000\.0000\.0000\.0000\.0280\.0280\.7180\.718TL\-DMI\-50\.3240\.3240\.7500\.7500\.8310\.8310\.0000\.0000\.0000\.0000\.1460\.1460\.8020\.802Defenses that suppress attack gradientsFaceScrubRN\-152NegLS0\.1600\.1600\.1470\.1470\.9060\.9060\.0000\.0000\.0480\.0480\.7690\.7690\.9050\.905RoLSS0\.4740\.4740\.5220\.5220\.8860\.8860\.0000\.0000\.0090\.0090\.7440\.7440\.8820\.882RoLSS\-SSF0\.3430\.3430\.3730\.3730\.8690\.8690\.0000\.0000\.0130\.0130\.7210\.7210\.8570\.857Trap\-MID0\.4010\.4010\.6180\.6180\.9260\.9260\.0000\.0000\.0000\.0000\.0000\.0000\.8560\.856RN\-18NegLS0\.6490\.6490\.9320\.9320\.9290\.9290\.0000\.0000\.2150\.2150\.8540\.8540\.9330\.933RoLSS0\.8580\.8580\.9790\.9790\.9500\.9500\.0000\.0000\.0020\.0020\.8190\.8190\.9480\.948RoLSS\-SSF0\.8490\.8490\.9730\.9730\.9500\.9500\.0000\.0000\.0020\.0020\.8130\.8130\.9510\.951Trap\-MID0\.7350\.7350\.9180\.9180\.9280\.9280\.0000\.0000\.0000\.0000\.5300\.5300\.9140\.914CelebARN\-152NegLS0\.3400\.3400\.5540\.5540\.8050\.8050\.0000\.0000\.0130\.0130\.5590\.5590\.7710\.771RoLSS0\.0460\.0460\.0450\.0450\.5370\.5370\.0000\.0000\.0000\.0000\.0860\.0860\.5300\.530RoLSS\-SSF0\.0390\.0390\.0440\.0440\.4280\.4280\.0000\.0000\.0000\.0000\.0420\.0420\.4060\.406Trap\-MID0\.3300\.3300\.4170\.4170\.8480\.8480\.0000\.0000\.0000\.0000\.3330\.3330\.8280\.828RN\-18NegLS0\.6670\.6670\.9530\.9530\.8380\.8380\.0000\.0000\.1000\.1000\.7110\.7110\.8250\.825RoLSS0\.5060\.5060\.7240\.7240\.8300\.8300\.0000\.0000\.0690\.0690\.7010\.7010\.8130\.813RoLSS\-SSF0\.5190\.5190\.7650\.7650\.8390\.8390\.0000\.0000\.0800\.0800\.7230\.7230\.8340\.834Trap\-MID0\.4420\.4420\.6700\.6700\.8300\.8300\.0000\.0000\.0440\.0440\.6850\.6850\.8270\.827

Tradeoff experiment setup\.Following seminal work byCarliniet al\.\([2019](https://arxiv.org/html/2607.12354#bib.bib40)\); Croce and Hein \([2020b](https://arxiv.org/html/2607.12354#bib.bib27)\)and others, a scholarly consensus has emerged regarding the design principlesnecessaryto ensure the rigor of experiments that measure robustness to adversarial examples\. Wecarefullyfollow these design principles here \(see App\.[F\.1](https://arxiv.org/html/2607.12354#A6.SS1)and[F\.3](https://arxiv.org/html/2607.12354#A6.SS3)\)\. We report*robust accuracy*, i\.e\., worst\-case accuracy on adversarial examples across all four attacks in the standard AutoAttack ensemble\(Croce and Hein,[2020b](https://arxiv.org/html/2607.12354#bib.bib27)\): untargeted Auto\-PGD; targeted DLR Auto\-PGD\(Croce and Hein,[2020b](https://arxiv.org/html/2607.12354#bib.bib27)\); targeted FAB\(Croce and Hein,[2020a](https://arxiv.org/html/2607.12354#bib.bib46)\); and Square\(Andriushchenkoet al\.,[2020](https://arxiv.org/html/2607.12354#bib.bib20)\)\. We find that all defenses have0%0\\%robust accuracy under AutoAttack with defaultϵ\\epsilon==0\.0310\.031\(≈\\approx8/255\)8/255\), rendering comparisons uninformative\. Thus we recompute attacks with a range of smaller perturbations:ϵ∈\{0\.031,0\.0025,0\.0005,𝟏𝟎−𝟓\}\\mathbf\{\\epsilon\\in\\\{0\.031,0\.0025,0\.0005,10^\{\-5\}\\\}\}\. This evaluates defenses’ unintentional reliance on imperceptible ‘non\-robust features’ of different magnitudes\. App\.[F\.3](https://arxiv.org/html/2607.12354#A6.SS3)gives details\.

![Refer to caption](https://arxiv.org/html/2607.12354v1/x2.png)Figure 2:Obs\. vs\. Pred\. leakage \(PPA\), pred\.*only*via robust acc\.β\\betaweights\.Tradeoff raw results & estimating the privacy\-adversarial robustness tradeoff\.Table[4](https://arxiv.org/html/2607.12354#S4.T4)presents results, and Tables[11](https://arxiv.org/html/2607.12354#A6.T11),[12](https://arxiv.org/html/2607.12354#A6.T12),[13](https://arxiv.org/html/2607.12354#A6.T13),[14](https://arxiv.org/html/2607.12354#A6.T14)in App\.[F\.4](https://arxiv.org/html/2607.12354#A6.SS4)also report robust accuracies under each of AutoAttack’s constituent attacks\. Note that for all generic privacy defenses \(MID, BiDO, TL\-DMI\), less privacy leakage \(lower AttAcc@1\) corresponds to*worse*robust accuracy under AutoAttack \(Table[4](https://arxiv.org/html/2607.12354#S4.T4)\), particularly with smallerϵ\\epsilon\. To obtain a principled test of this tradeoff, we estimate OLS linear regressions of the following form\. This allows us to explicitly test our hypothesis that these defenses’ privacy leakage reduction \(AttAcc@1 under PPA or IF\-GMI\) is a linear function of their robust accuracy \(or, equivalently, a linear function of their reliance on non\-robust features\)\. We include clean test accuracy in the regression, as it has been widely observed that more accurate models leak more\(Struppeket al\.,[2024](https://arxiv.org/html/2607.12354#bib.bib95); Hoet al\.,[2024](https://arxiv.org/html/2607.12354#bib.bib45); Liu and Chen,[2024](https://arxiv.org/html/2607.12354#bib.bib10)\):

LeakageAttAcc@1,j=β0\+β1​TestAccj\+β2​Accϵ​=​0\.0025,j\+β3​Accϵ​=​0\.0005,j\+β4​Accϵ​=​10−5,j\+ej\\displaystyle\\text\{Leakage\}\_\{\\text\{AttAcc@1\},j\}\\text\{$=$\}\\beta\_\{0\}\\text\{$\+$\}\\beta\_\{1\}\\text\{TestAcc\}\_\{j\}\\text\{$\+$\}\\beta\_\{2\}\\text\{Acc\}\_\{\\epsilon\\text\{=\}0\.0025,j\}\\text\{$\+$\}\\beta\_\{3\}\\text\{Acc\}\_\{\\epsilon\\text\{=\}0\.0005,j\}\\text\{$\+$\}\\beta\_\{4\}\\text\{Acc\}\_\{\\epsilon\\text\{=\}10^\{\-5\},j\}\\text\{$\+$\}e\_\{j\}\(1\)Specification Invariance\.To explore the stability of results, we are careful to also estimate a total of5656various specifications of this OLS model that explore alternate hypotheses, including alternative Beta regression variants with appropriate distributional assumptions and logit link \(g​\(x\)g\(x\)\) to capture the fact that the response \(AttAcc@1 under PPA or IF\-GMI\) is bounded in\[0,1\]\[0,1\]:

LeakageAttAcc@1,j\\displaystyle\\mathrm\{Leakage\}\_\{\\text\{AttAcc@1\},j\}∼Beta​\(μj​ϕ,\(1−μj\)​ϕ\)\\displaystyle\\sim\\mathrm\{Beta\}\(\\mu\_\{j\}\\phi,\\,\(1\-\\mu\_\{j\}\)\\phi\)g​\(μj\)=β0\\displaystyle g\(\\mu\_\{j\}\)=\\;\\beta\_\{0\}\+β1​TestAccj\+β2​Accϵ=0\.0025,j\+β3​Accϵ=0\.0005,j\+β4​Accϵ=10−5,j\\displaystyle\\text\{$\+$\}\\beta\_\{1\}\\,\\mathrm\{TestAcc\}\_\{j\}\\text\{$\+$\}\\beta\_\{2\}\\,\\mathrm\{Acc\}\_\{\\epsilon=0\.0025,\\,j\}\\text\{$\+$\}\\beta\_\{3\}\\,\\mathrm\{Acc\}\_\{\\epsilon=0\.0005,\\,j\}\\text\{$\+$\}\\beta\_\{4\}\\,\\mathrm\{Acc\}\_\{\\epsilon=10^\{\-5\},\\,j\}\(2\)
Table 5:OLS & Beta leakage \(PPA AttAcc@1\) regressions±\\pm95%95\\%CI\. OLS CIs use small\-samplet0\.975,13t\_\{0\.975,13\}==2\.1602\.160with HC2 SEs; Beta uses Wald \(±1\.96\\pm 1\.96SE\) on the logit scale\.OLS \(±\\pm95% CI\)Beta \(±\\pm95% CI\)Test Acc\.\-0\.011±\\pm1\.033\-0\.192±\\pm3\.101ϵ=0\.0025\\epsilon\{=\}0\.00252\.299±\\pm1\.59612\.827±\\pm3\.939ϵ=0\.0005\\epsilon\{=\}0\.00050\.211±\\pm0\.2340\.737±\\pm0\.637ϵ=10−5\\epsilon\{=\}10^\{\-5\}0\.795±\\pm0\.7784\.390±\\pm2\.312R2R^\{2\}0\.934 \(adj\.\)0\.952 \(pseudo\)MAE0\.0470\.042

Results\.Fig\.[2](https://arxiv.org/html/2607.12354#S4.F2)plots fit and Table[5](https://arxiv.org/html/2607.12354#S4.T5)presents a small selection of results \(see App\.[G](https://arxiv.org/html/2607.12354#A7)\)\. We find that \(1\) leakage \(AttAcc@1under both PPA vs\. IF\-GMI\) is strongly and significantly correlated with robust accuracy at all levels ofϵ\\epsilon\(note 95% CI\), withR2R^\{2\}≈\\approx0\.930\.93−\-0\.950\.95, and mean abs\. error of just0\.0420\.042−\-0\.050\.05\. Contrary to widespread belief, \(2\) clean test accuracy is*only weakly or insignificantly correlated*with leakage after controlling for robust accuracy\. Thus, at a high level, we can almost perfectly predict a defense’s leakage just from knowing its robust accuracy \(Fig\.[2](https://arxiv.org/html/2607.12354#S4.F2)\), and knowing its clean accuracy adds little to no additional information that improves this prediction\. Finally, as we hypothesized, this privacy\-robustness tradeoff does*not*hold for gradient\-suppressing defenses, which \(as expected\) instead achieve their defense by various other means\. Our results hold across all 56 OLS and Beta models, which show they are invariant to specification, PPA and IF\-GMI attacks, different distributional assumptions, controls for dataset/architecture, various robust standard errors estimators, etc\. App\.[G](https://arxiv.org/html/2607.12354#A7)gives details\.

The robustness cost of privacy gain\.We can use these linear models to compute the cost of privacy\. Per the Beta model, reducing PPA leakage \(AttAcc@1\) by 1 percentage point \(pp\) corresponds to a significant*0\.31 pp decrease*in AutoAttack robust accuracy atϵ\\epsilon==0\.0025, \(95% CI: 0\.24–0\.45\), a significant*5\.4 pp decrease*atϵ\\epsilon==0\.0005 \(95% CI: 2\.9–58\.3\), and a significant*0\.91 pp decrease*atϵ\\epsilon==10−510^\{\-5\}\(95% CI: 0\.60–1\.93\)\. These results highlight that privacy\-robustness tradeoffs are nonuniform: the largest and smallest\-magnitude non\-robust features experience small but significant costs, while the ‘medium’ϵ\\epsilon==0\.0005 magnitude experiences disproportionately large and unstable costs\.

> These results provide evidence that non\-robust features were unknowingly*correlated*with privacy\. Can non\-robust features*cause*privacy through deliberate algorithmic design?

## 5AntiAdversarialTraining \(AT\-AT\)![[Uncaptioned image]](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ATATiconV4.png)

We now show how to exploit this mechanism to*causally induce*superior defense at higher accuracy\. To establish this causal relationship, we describeAntiAdversarialTraining \(AT\-AT![[Uncaptioned image]](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ATATiconV4.png)\), a novel privacy/MIA defense training regime that balances model accuracy with gradient steps that reverse standard adversarial training\. These steps allow us to causally ‘treat’ the learned feature basis by intentionally shifting it towards non\-robust \(but still generalizable\) features that are visually imperceptible to humans and thus unamenable to visual reconstruction under MIA\.

For intuition, suppose that on a given iteration of SGD, we draw an imagexYodax\_\{\\text\{Yoda\}\}of classyy==Yoda\.

Vanilla Training\.Vanilla loss:minθ⁡𝔼\(x,y\)∼𝒟​\[ℒ​\(θ,x,y\)\]\\min\_\{\\theta\}\\ \\mathbb\{E\}\_\{\(x,y\)\\sim\\mathcal\{D\}\}\[\\mathcal\{L\}\(\\theta,x,y\)\]just rewards accurate mappings of clean images:xYodax\_\{\\text\{Yoda\}\}→\\rightarrowYoda\. It is well\-known that this approach learns both robust and non\-robust features\.

Classic Adversarial Training \(AT\)\.ClassicAT\(Madryet al\.,[2017](https://arxiv.org/html/2607.12354#bib.bib246)\)replaces each clean training examplexxin the SGD\-loop with the untargeted adversarial perturbationx\+δx\+\\deltathat is most likely to flip thexx’s classified label\. The AT loss function,maxδ∈𝒮⁡ℒ​\(θ,x\+δ,y\)\\max\_\{\\delta\\in\\mathcal\{S\}\}\\mathcal\{L\}\(\\theta,x\+\\delta,y\), then treats features that are robust to this perturbation as the ‘signal’ we seek to learn\. For example, a perturbationδ\\deltamight flip Yoda into Luke, i\.e\.,\(xYoda\+δLuke\)→y′\(x\_\{\\text\{Yoda\}\}\+\\delta\_\{\\text\{Luke\}\}\)\\rightarrow y^\{\\prime\}=Luke≠\\neqYoda\. AT rewards learningxYodax\_\{\\text\{Yoda\}\}features that are*robust*toδ\\delta, i\.e\., rewards\(xYodax\_\{\\text\{Yoda\}\}\+\+δLuke\\delta\_\{\\text\{Luke\}\}\)→\\rightarrowYoda,and penalizes features inδ\\deltathat are*non\-robust*\.

AntiAdversarialTraining \(AT\-AT![[Uncaptioned image]](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ATATiconV4.png)\)\.At a high level, our goal is the opposite:AT\-ATuses a loss term that treats the visually meaningful imagexxas the ‘random noise’ we seek to ignore, and the imperceptible\-but\-generalizable ‘non\-robust’ features exposed in perturbationδ\\deltaas the ‘signal’ we seek to learn\. Thus, compared to classic AT, we optimize for the opposite mapping:\(xYodax\_\{\\text\{Yoda\}\}\+\+δLuke\\delta\_\{\\text\{Luke\}\}\)→\\rightarrowLuke\. More formally, we seekminδ∈𝒮⁡ℒ​\(θ,x\+δ,𝐲′\),where​𝐲′≠y\\min\_\{\\delta\\in\\mathcal\{S\}\}\\mathcal\{L\}\(\\theta,x\+\\delta,\\mathbf\{y^\{\\prime\}\}\),\\ \\text\{where \}\\mathbf\{y^\{\\prime\}\}\\neq y\. However, as a practical matter, it is infeasible to obtain competitive accuracy for challenging learning tasks like hi\-res facial recognition using*only*non\-robust features\. Thus, we adopt a dual loss function that balances vanilla accuracy \(incl\. robust features\) with non\-robust features via user\-chosenλ\\lambda:

AT\-AT Objective:​minθ⁡𝔼\(x,y\)∼𝒟​\[ℒ​\(θ,x,y\)\+λ⋅minδ∈𝒮⁡ℒ​\(θ,x\+δ,𝐲′\)\],where​𝐲′≠y\\displaystyle\\vskip\-13\.99995pt\\text\{\{AT\-AT Objective: \\ \\ \}\}\\min\_\{\\theta\}\\ \\mathbb\{E\}\_\{\(x,y\)\\sim\\mathcal\{D\}\}\[\\mathcal\{L\}\(\\theta,x,y\)\\ \+\\ \\lambda\\cdot\\min\_\{\\delta\\in\\mathcal\{S\}\}\\mathcal\{L\}\(\\theta,x\+\\delta,\\mathbf\{y^\{\\prime\}\}\)\],\\ \\text\{where \}\\mathbf\{y^\{\\prime\}\}\\neq y\\ \\ \\ \\\(3\)
![[Uncaptioned image]](https://arxiv.org/html/2607.12354v1/x3.png)ThisAT\-ATloss can be optimized similarly to a targeted variant of vanilla AT\. Each SGD iterationiidraws a training imagexxand a target classy′y^\{\\prime\}each𝒰​𝒜​ℛ\\mathcal\{UAR\}\. It then computes a targeted adversarial perturbationδ\\delta\(via, e\.g\., PGD\) that causes the current training modelℳi\\mathcal\{M\}\_\{i\}to classifyxxnot asyy, but asy′y^\{\\prime\}, e\.g\., “\(xYodax\_\{\\text\{Yoda\}\}\+\+δLuke\\delta\_\{\\text\{Luke\}\}\)→\\rightarrowLuke\.” Finally, it sums vanilla loss onyyand anti\-adversarial loss ony′y^\{\\prime\}\. The latter penalizes robust features inxxand rewards non\-robust ones exposed by perturbationδ\\delta\.

Causal effect of AT\-AT loss term on privacy\.We train 10 RN\-152’s on FaceScrub and randomly assign half as ‘control’λ\\lambda=0\(equal to Vanilla/NoDef\) and half as ‘treatment’λ\\lambda\>\>0\. Beta regression ofAttAcc@1on treatment shows that AT\-AT’s non\-robust term causes a reduction in reconstruction rate \(PPA AttAcc@1\) from 84% to 6\.5% \(a7777×\\timesreduction in leakage odds,pp<<10−1610^\{\-16\},zz==38\.038\.0\)\. See App\.[H](https://arxiv.org/html/2607.12354#A8)\.

## 6Experiments: AT\-AT vs\. generic & gradient defenses

Our goal in this section is to show thatAT\-ATobtains superior reconstruction defense at higher accuracy versus both generic and gradient\-suppressing defenses\. Fig\.[4](https://arxiv.org/html/2607.12354#S8.F4)on the following page comparesAT\-AT\(*red points*\) to baselines in terms of accuracy and privacy across all datasets under PPA, IF\-GMI, and PPDG inversion attacks via all privacy metrics on ResNet\-152\.

Across all privacy metrics, data, and attacks,AT\-ATobtains superior privacy at higher levels of accuracy than the77state\-of\-the\-art baselines\. In addition to Fig\.[4](https://arxiv.org/html/2607.12354#S8.F4)below, we include full additional results on DenseNet\-169 and ResNet\-18 in App\.[I\.2](https://arxiv.org/html/2607.12354#A9.SS2)\. We also provide full tables with confidence intervals in App\.[I\.3](https://arxiv.org/html/2607.12354#A9.SS3)\. To demonstrate robustness across domains, we also report results on the Stanford Dogs dataset in Table[26](https://arxiv.org/html/2607.12354#A9.T26)\.

## 7Conclusion

We present converging correlational and causal evidence that reducing information dependency \(including rote memorization\) does not cause training data privacy under MIAs, whereas adversarially non\-robust features do\. This result overturns a dominant assumption in MIA and privacy literature\. We describeAT\-AT, a novel training regime that deliberately learns non\-robust features to obtain superior MIA defense\. While our scope is limited to hi\-res image classification, our results raise timely questions of whether privacy\-robustness tradeoffs generalize to paradigms such as LLMs and diffusion, where memorization is similarly thought to drive privacy vulnerabilities\.

## 8Reproducibility statement

Our experiments were designed from the outset to be replicated\. Full pushbutton cluster\-ready replication codes are available from our lab’s GitHub at[https://github\.com/BreuerLabs/Anti\-Adversarial\-Training](https://github.com/BreuerLabs/Anti-Adversarial-Training)\. All MIAs, MIA defense baselines, and adversarial example attacks were computed using their authors’ code and identical parameters, except where explicitly noted in our paper \(e\.g\., our additional BiDO parameter\-tuned variants, which we report with the exact parameter we used below, alongside where we report the BiDO version that we ran with the authors’ original parameters\)\. Further experiment and replication details are provided in App\.[A](https://arxiv.org/html/2607.12354#A1)\.

Our code also provides a suite of tools designed so that researchers can easily apply our experimental setups, evaluations, attacks, and defenses to their own research\.

![Refer to caption](https://arxiv.org/html/2607.12354v1/x4.png)Figure 3:PPA reconstructions on ResNet\-152\. Images selected via max EvalConf\. Full results in Fig\.[6](https://arxiv.org/html/2607.12354#A9.F6)\.![Refer to caption](https://arxiv.org/html/2607.12354v1/x5.png)Figure 4:AT\-AT vs Baselines on RN\-152:*Top:*AttAcc@1 & L2 Distance;*Bottom:*AttAcc@5 & EvalConf\.
## References

- M\. Abadi, A\. Chu, I\. Goodfellow, H\. B\. McMahan, I\. Mironov, K\. Talwar, and L\. Zhang \(2016\)Deep Learning with Differential Privacy\.InProceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security,pp\. 308–318\(en\)\.Note:arXiv:1607\.00133 \[cs, stat\]External Links:[Link](http://arxiv.org/abs/1607.00133),[Document](https://dx.doi.org/10.1145/2976749.2978318)Cited by:[§F\.2](https://arxiv.org/html/2607.12354#A6.SS2.p1.1)\.
- S\. An, G\. Tao, Q\. Xu, Y\. Liu, G\. Shen, Y\. Yao, J\. Xu, and X\. Zhang \(2022\)MIRROR: Model Inversion for Deep Learning Network with High Fidelity\.InProceedings 2022 Network and Distributed System Security Symposium,San Diego, CA, USA\(en\)\.External Links:ISBN 978\-1\-891562\-74\-7,[Link](https://www.ndss-symposium.org/wp-content/uploads/2022-335-paper.pdf),[Document](https://dx.doi.org/10.14722/ndss.2022.24335)Cited by:[§C\.2](https://arxiv.org/html/2607.12354#A3.SS2.p2.1),[§C\.3](https://arxiv.org/html/2607.12354#A3.SS3.p1.1)\.
- M\. Andriushchenko, F\. Croce, N\. Flammarion, and M\. Hein \(2020\)Square Attack: a query\-efficient black\-box adversarial attack via random search\.arXiv\.Note:arXiv:1912\.00049 \[cs\]External Links:[Link](http://arxiv.org/abs/1912.00049),[Document](https://dx.doi.org/10.48550/arXiv.1912.00049)Cited by:[§F\.3](https://arxiv.org/html/2607.12354#A6.SS3.p1.2),[§F\.4](https://arxiv.org/html/2607.12354#A6.SS4.p1.2),[§2](https://arxiv.org/html/2607.12354#S2.p1.1),[§4](https://arxiv.org/html/2607.12354#S4.p2.7)\.
- F\. Boenisch, P\. Sperl, and K\. Böttinger \(2021\)Gradient Masking and the Underestimated Robustness Threats of Differential Privacy in Deep Learning\.arXiv\.Note:arXiv:2105\.07985 \[cs\]External Links:[Link](http://arxiv.org/abs/2105.07985),[Document](https://dx.doi.org/10.48550/arXiv.2105.07985)Cited by:[§F\.2](https://arxiv.org/html/2607.12354#A6.SS2.p1.1)\.
- V\. Borisov, J\. Haug, and G\. Kasneci \(2019\)CancelOut: A Layer for Feature Selection in Deep Neural Networks\.InArtificial Neural Networks and Machine Learning – ICANN 2019: Deep Learning,I\. V\. Tetko, V\. Kůrková, P\. Karpov, and F\. Theis \(Eds\.\),Vol\.11728,pp\. 72–83\(en\)\.Note:Series Title: Lecture Notes in Computer ScienceExternal Links:ISBN 978\-3\-030\-30483\-6 978\-3\-030\-30484\-3,[Link](http://link.springer.com/10.1007/978-3-030-30484-3_6),[Document](https://dx.doi.org/10.1007/978-3-030-30484-3%5F6)Cited by:[§E\.1](https://arxiv.org/html/2607.12354#A5.SS1.p2.3)\.
- G\. Brown, M\. Bun, V\. Feldman, A\. Smith, and K\. Talwar \(2021\)When is memorization of irrelevant training data necessary for high\-accuracy learning?\.InProceedings of the 53rd Annual ACM SIGACT Symposium on Theory of Computing,Virtual Italy,pp\. 123–132\(en\)\.External Links:ISBN 978\-1\-4503\-8053\-9,[Link](https://dl.acm.org/doi/10.1145/3406325.3451131),[Document](https://dx.doi.org/10.1145/3406325.3451131)Cited by:[§1](https://arxiv.org/html/2607.12354#S1.p5.1)\.
- N\. Carlini, A\. Athalye, N\. Papernot, W\. Brendel, J\. Rauber, D\. Tsipras, I\. Goodfellow, A\. Madry, and A\. Kurakin \(2019\)On Evaluating Adversarial Robustness\.arXiv\.Note:arXiv:1902\.06705 \[cs\]External Links:[Link](http://arxiv.org/abs/1902.06705),[Document](https://dx.doi.org/10.48550/arXiv.1902.06705)Cited by:[§F\.3](https://arxiv.org/html/2607.12354#A6.SS3.p1.2),[§4](https://arxiv.org/html/2607.12354#S4.p2.7)\.
- N\. Carlini, S\. Chien, M\. Nasr, S\. Song, A\. Terzis, and F\. Tramèr \(2022\)Membership Inference Attacks From First Principles\.In2022 IEEE Symposium on Security and Privacy \(SP\),pp\. 1897–1914\.Note:ISSN: 2375\-1207External Links:[Link](https://ieeexplore.ieee.org/abstract/document/9833649),[Document](https://dx.doi.org/10.1109/SP46214.2022.9833649)Cited by:[§1](https://arxiv.org/html/2607.12354#S1.p1.1)\.
- N\. Carlini, J\. Hayes, M\. Nasr, M\. Jagielski, V\. Sehwag, F\. Tramèr, B\. Balle, D\. Ippolito, and E\. Wallace \(2023\)Extracting Training Data from Diffusion Models\.arXiv\.Note:arXiv:2301\.13188 \[cs\]External Links:[Link](http://arxiv.org/abs/2301.13188),[Document](https://dx.doi.org/10.48550/arXiv.2301.13188)Cited by:[§E\.3](https://arxiv.org/html/2607.12354#A5.SS3.p5.1),[§1](https://arxiv.org/html/2607.12354#S1.p1.1),[§1](https://arxiv.org/html/2607.12354#S1.p5.1)\.
- N\. Carlini, F\. Tramèr, E\. Wallace, M\. Jagielski, A\. Herbert\-Voss, K\. Lee, A\. Roberts, T\. Brown, D\. Song, Ú\. Erlingsson, A\. Oprea, and C\. Raffel \(2021\)Extracting training data from large language models\.In30th USENIX Security Symposium \(USENIX Security 21\),pp\. 2633–2650\.External Links:ISBN 978\-1\-939133\-24\-3,[Link](https://www.usenix.org/conference/usenixsecurity21/presentation/carlini-extracting)Cited by:[§1](https://arxiv.org/html/2607.12354#S1.p1.1)\.
- K\. Chaudhuri, C\. Guo, L\. v\. d\. Maaten, S\. Mahloujifar, and M\. Tygert \(2024\)Guarantees of confidentiality via Hammersley\-Chapman\-Robbins bounds\.arXiv\.Note:arXiv:2404\.02866 \[cs\]External Links:[Link](http://arxiv.org/abs/2404.02866),[Document](https://dx.doi.org/10.48550/arXiv.2404.02866)Cited by:[§E\.3](https://arxiv.org/html/2607.12354#A5.SS3.p2.13),[§E\.3](https://arxiv.org/html/2607.12354#A5.SS3.p4.2),[§E\.3](https://arxiv.org/html/2607.12354#A5.SS3.p6.1),[§3](https://arxiv.org/html/2607.12354#S3.p8.2)\.
- S\. Chen, M\. Kahla, R\. Jia, and G\. Qi \(2021\)Knowledge\-Enriched Distributional Model Inversion Attacks\.In2021 IEEE/CVF International Conference on Computer Vision \(ICCV\),Montreal, QC, Canada\(en\)\.External Links:[Link](https://ieeexplore.ieee.org/document/9711082/),[Document](https://dx.doi.org/10.1109/iccv48922.2021.01587)Cited by:[§C\.2](https://arxiv.org/html/2607.12354#A3.SS2.p1.1),[§C\.3](https://arxiv.org/html/2607.12354#A3.SS3.p1.1)\.
- Y\. Choi, Y\. Uh, J\. Yoo, and J\. Ha \(2020\)StarGAN v2: diverse image synthesis for multiple domains\.In2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),Vol\.,pp\. 8185–8194\.External Links:[Document](https://dx.doi.org/10.1109/CVPR42600.2020.00821)Cited by:[1st item](https://arxiv.org/html/2607.12354#A1.I5.i1.p1.1)\.
- F\. Croce and M\. Hein \(2020a\)Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attack\.InProceedings of the 37th International Conference on Machine Learning,pp\. 2196–2205\(en\)\.Note:ISSN: 2640\-3498External Links:[Link](https://proceedings.mlr.press/v119/croce20a.html)Cited by:[§F\.3](https://arxiv.org/html/2607.12354#A6.SS3.p1.2),[§F\.4](https://arxiv.org/html/2607.12354#A6.SS4.p1.2),[§2](https://arxiv.org/html/2607.12354#S2.p1.1),[§4](https://arxiv.org/html/2607.12354#S4.p2.7)\.
- F\. Croce and M\. Hein \(2020b\)Reliable evaluation of adversarial robustness with an ensemble of diverse parameter\-free attacks\.InProceedings of the 37th International Conference on Machine Learning,pp\. 2206–2216\(en\)\.Note:ISSN: 2640\-3498External Links:[Link](https://proceedings.mlr.press/v119/croce20b.html)Cited by:[§F\.1](https://arxiv.org/html/2607.12354#A6.SS1.p2.1),[§F\.3](https://arxiv.org/html/2607.12354#A6.SS3.p1.2),[§F\.4](https://arxiv.org/html/2607.12354#A6.SS4.p1.2),[§2](https://arxiv.org/html/2607.12354#S2.p1.1),[§4](https://arxiv.org/html/2607.12354#S4.p2.7)\.
- S\. V\. Dibbo, A\. Breuer, J\. Moore, and M\. Teti \(2024\)Improving robustness to model inversion attacks via sparse coding architectures\.InEuropean Conference on Computer Vision,pp\. 117–136\.Cited by:[§A\.1](https://arxiv.org/html/2607.12354#A1.SS1.p2.1),[§1](https://arxiv.org/html/2607.12354#S1.p6.1),[§1](https://arxiv.org/html/2607.12354#S1.p7.1)\.
- V\. C\. Dinh and L\. S\. Ho \(2020\)Consistent feature selection for analytic deep neural networks\.InAdvances in Neural Information Processing Systems,Vol\.33,pp\. 2420–2431\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2020/hash/1959eb9d5a0f7ebc58ebde81d5df400d-Abstract.html)Cited by:[§E\.1](https://arxiv.org/html/2607.12354#A5.SS1.p1.7),[§E\.2](https://arxiv.org/html/2607.12354#A5.SS2.p3.2)\.
- V\. Dinh and L\. S\. T\. Ho \(2021\)Consistent feature selection for neural networks via Adaptive Group Lasso\.arXiv\.Note:arXiv:2006\.00334 \[stat\]External Links:[Link](http://arxiv.org/abs/2006.00334),[Document](https://dx.doi.org/10.48550/arXiv.2006.00334)Cited by:[§E\.1](https://arxiv.org/html/2607.12354#A5.SS1.p1.7),[§E\.2](https://arxiv.org/html/2607.12354#A5.SS2.p1.6)\.
- Y\. Du, P\. Mondorf, S\. Casola, Y\. Yao, R\. Litschko, and B\. Plank \(2025\)Reason to Rote: Rethinking Memorization in Reasoning\.arXiv\.Note:arXiv:2507\.04782 \[cs\]External Links:[Link](http://arxiv.org/abs/2507.04782),[Document](https://dx.doi.org/10.48550/arXiv.2507.04782)Cited by:[§1](https://arxiv.org/html/2607.12354#S1.p5.1)\.
- L\. Engstrom, A\. Ilyas, S\. Santurkar, D\. Tsipras, B\. Tran, and A\. Madry \(2019\)Adversarial Robustness as a Prior for Learned Representations\.arXiv\.Note:arXiv:1906\.00945 \[stat\]External Links:[Link](http://arxiv.org/abs/1906.00945),[Document](https://dx.doi.org/10.48550/arXiv.1906.00945)Cited by:[§F\.2](https://arxiv.org/html/2607.12354#A6.SS2.p2.1)\.
- H\. Fang, B\. Chen, X\. Wang, Z\. Wang, and S\. Xia \(2023\)Gifd: a generative gradient inversion method with feature domain optimization\.InProceedings of the IEEE/CVF International Conference on Computer Vision,pp\. 4967–4976\.Cited by:[§C\.3](https://arxiv.org/html/2607.12354#A3.SS3.p2.1)\.
- V\. Feldman \(2020\)Does learning require memorization? a short tale about a long tail\.InProceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing,Chicago IL USA,pp\. 954–959\(en\)\.External Links:ISBN 978\-1\-4503\-6979\-4,[Link](https://dl.acm.org/doi/10.1145/3357713.3384290),[Document](https://dx.doi.org/10.1145/3357713.3384290)Cited by:[§1](https://arxiv.org/html/2607.12354#S1.p5.1)\.
- M\. Fredrikson, S\. Jha, and T\. Ristenpart \(2015\)Model Inversion Attacks that Exploit Confidence Information and Basic Countermeasures\.InProceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security,Denver Colorado USA,pp\. 1322–1333\(en\)\.External Links:ISBN 978\-1\-4503\-3832\-5,[Link](https://dl.acm.org/doi/10.1145/2810103.2813677),[Document](https://dx.doi.org/10.1145/2810103.2813677)Cited by:[§C\.1](https://arxiv.org/html/2607.12354#A3.SS1.p1.8),[§1](https://arxiv.org/html/2607.12354#S1.p2.1)\.
- J\. Geiping, H\. Bauermeister, H\. Dröge, and M\. Moeller \(2020\)Inverting gradients\-how easy is it to break privacy in federated learning?\.Advances in neural information processing systems33,pp\. 16937–16947\.Cited by:[§C\.3](https://arxiv.org/html/2607.12354#A3.SS3.p2.1)\.
- A\. Gretton, O\. Bousquet, A\. Smola, and B\. Schölkopf \(2005\)Measuring Statistical Dependence with Hilbert\-Schmidt Norms\.InAlgorithmic Learning Theory,D\. Hutchison, T\. Kanade, J\. Kittler, J\. M\. Kleinberg, F\. Mattern, J\. C\. Mitchell, M\. Naor, O\. Nierstrasz, C\. Pandu Rangan, B\. Steffen, M\. Sudan, D\. Terzopoulos, D\. Tygar, M\. Y\. Vardi, G\. Weikum, S\. Jain, H\. U\. Simon, and E\. Tomita \(Eds\.\),Vol\.3734,pp\. 63–77\(en\)\.Note:Series Title: Lecture Notes in Computer ScienceExternal Links:ISBN 978\-3\-540\-29242\-5 978\-3\-540\-31696\-1,[Link](http://link.springer.com/10.1007/11564089_7),[Document](https://dx.doi.org/10.1007/11564089%5F7)Cited by:[§D\.1](https://arxiv.org/html/2607.12354#A4.SS1.p1.5),[§3](https://arxiv.org/html/2607.12354#S3.p2.2)\.
- C\. Guo, B\. Karrer, K\. Chaudhuri, and L\. van der Maaten \(2022\)Bounding Training Data Reconstruction in Private \(Deep\) Learning\.arXiv\(en\)\.Note:arXiv:2201\.12383 \[cs\]External Links:[Link](http://arxiv.org/abs/2201.12383)Cited by:[§E\.3](https://arxiv.org/html/2607.12354#A5.SS3.p6.1),[§3](https://arxiv.org/html/2607.12354#S3.p8.2)\.
- N\. Haim, G\. Vardi, G\. Yehudai, O\. Shamir, and M\. Irani \(2022\)Reconstructing Training Data from Trained Neural Networks\.InAdvances in Neural Information Processing Systems,\(en\)\.Cited by:[§1](https://arxiv.org/html/2607.12354#S1.p2.1)\.
- G\. Han, J\. Choi, H\. Lee, and J\. Kim \(2023\)Reinforcement Learning\-Based Black\-Box Model Inversion Attacks\.In2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),Vancouver, BC, Canada,pp\. 20504–20513\(en\)\.External Links:[Link](https://ieeexplore.ieee.org/document/10203595/),[Document](https://dx.doi.org/10.1109/cvpr52729.2023.01964)Cited by:[§C\.3](https://arxiv.org/html/2607.12354#A3.SS3.p1.1)\.
- A\. Hannun, C\. Guo, and L\. v\. d\. Maaten \(2021\)Measuring data leakage in machine\-learning models with Fisher information\.InProceedings of the Thirty\-Seventh Conference on Uncertainty in Artificial Intelligence,pp\. 760–770\(en\)\.Note:ISSN: 2640\-3498External Links:[Link](https://proceedings.mlr.press/v161/hannun21a.html)Cited by:[§E\.3](https://arxiv.org/html/2607.12354#A5.SS3.p2.13),[§E\.3](https://arxiv.org/html/2607.12354#A5.SS3.p6.1),[§3](https://arxiv.org/html/2607.12354#S3.p8.2)\.
- J\. Hayes \(2020\)Trade\-offs between membership privacy&adversarially robust learning\.External Links:[Link](https://www.semanticscholar.org/paper/122fb9401248e694b32ad55181c4c15fd9aea257)Cited by:[§F\.2](https://arxiv.org/html/2607.12354#A6.SS2.p1.1)\.
- K\. He, X\. Zhang, S\. Ren, and J\. Sun \(2015\)Deep Residual Learning for Image Recognition\.arXiv\.Note:arXiv:1512\.03385External Links:[Link](http://arxiv.org/abs/1512.03385)Cited by:[1st item](https://arxiv.org/html/2607.12354#A1.I4.i1.p1.1.1),[2nd item](https://arxiv.org/html/2607.12354#A1.I4.i2.p1.1.1)\.
- S\. Ho, K\. J\. Hao, K\. Chandrasegaran, N\. Nguyen, and N\. Cheung \(2024\)Model Inversion Robustness: Can Transfer Learning Help?\.arXiv\.Note:arXiv:2405\.05588 \[cs\]External Links:[Link](http://arxiv.org/abs/2405.05588),[Document](https://dx.doi.org/10.48550/arXiv.2405.05588)Cited by:[4th item](https://arxiv.org/html/2607.12354#A1.I1.i4.p1.1.1),[1st item](https://arxiv.org/html/2607.12354#A1.I3.i1.p1.1),[3rd item](https://arxiv.org/html/2607.12354#A1.I3.i3.p1.1),[2nd item](https://arxiv.org/html/2607.12354#A1.I4.i2.p1.1),[§1](https://arxiv.org/html/2607.12354#S1.p8.1),[§2](https://arxiv.org/html/2607.12354#S2.p1.1),[§2](https://arxiv.org/html/2607.12354#S2.p2.1),[§4](https://arxiv.org/html/2607.12354#S4.p3.1)\.
- G\. Huang, Z\. Liu, L\. Van Der Maaten, and K\. Q\. Weinberger \(2017\)Densely Connected Convolutional Networks\.In2017 IEEE Conference on Computer Vision and Pattern Recognition \(CVPR\),Honolulu, HI,pp\. 2261–2269\(en\)\.External Links:ISBN 978\-1\-5386\-0457\-1,[Link](https://ieeexplore.ieee.org/document/8099726/),[Document](https://dx.doi.org/10.1109/CVPR.2017.243)Cited by:[3rd item](https://arxiv.org/html/2607.12354#A1.I4.i3.p1.1.1)\.
- Y\. Huang, S\. Gupta, Z\. Song, K\. Li, and S\. Arora \(2021\)Evaluating gradient inversion attacks and defenses in federated learning\.Advances in neural information processing systems34,pp\. 7232–7241\.Cited by:[§C\.3](https://arxiv.org/html/2607.12354#A3.SS3.p2.1)\.
- A\. Ilyas, S\. Santurkar, D\. Tsipras, L\. Engstrom, B\. Tran, and A\. Madry \(2019\)Adversarial Examples Are Not Bugs, They Are Features\.arXiv\.Note:arXiv:1905\.02175 \[stat\]External Links:[Link](http://arxiv.org/abs/1905.02175),[Document](https://dx.doi.org/10.48550/arXiv.1905.02175)Cited by:[1st item](https://arxiv.org/html/2607.12354#A1.I4.i1.p1.1),[§1](https://arxiv.org/html/2607.12354#S1.p17.1),[§1](https://arxiv.org/html/2607.12354#S1.p24.1)\.
- M\. J\. Jiménez\-Navarro, M\. Martinez\-Ballesteros, I\. S\. Brito, F\. Martinez\-Álvarez, and G\. Asencio\-Cortes \(2024\)Embedded feature selection for neural networks via learnable drop layer\.Logic Journal of the IGPL,pp\. jzae062\(en\)\.External Links:ISSN 1367\-0751, 1368\-9894,[Link](https://academic.oup.com/jigpal/advance-article/doi/10.1093/jigpal/jzae062/7689640),[Document](https://dx.doi.org/10.1093/jigpal/jzae062)Cited by:[§E\.1](https://arxiv.org/html/2607.12354#A5.SS1.p2.3)\.
- M\. Kahla, S\. Chen, H\. A\. Just, and R\. Jia \(2022\)Label\-Only Model Inversion Attacks via Boundary Repulsion\.In2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),New Orleans, LA, USA,pp\. 15025–15033\(en\)\.External Links:[Link](https://ieeexplore.ieee.org/document/9878400/),[Document](https://dx.doi.org/10.1109/cvpr52688.2022.01462)Cited by:[§C\.3](https://arxiv.org/html/2607.12354#A3.SS3.p1.1)\.
- T\. Karras, S\. Laine, and T\. Aila \(2019\)A style\-based generator architecture for generative adversarial networks\.External Links:1812\.04948,[Link](https://arxiv.org/abs/1812.04948)Cited by:[1st item](https://arxiv.org/html/2607.12354#A1.I5.i1.p1.1)\.
- T\. Karras, S\. Laine, M\. Aittala, J\. Hellsten, J\. Lehtinen, and T\. Aila \(2020\)Analyzing and Improving the Image Quality of StyleGAN\.In2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),Seattle, WA, USA,pp\. 8107–8116\(en\)\.External Links:ISBN 978\-1\-7281\-7168\-5,[Link](https://ieeexplore.ieee.org/document/9156570/),[Document](https://dx.doi.org/10.1109/CVPR42600.2020.00813)Cited by:[1st item](https://arxiv.org/html/2607.12354#A1.I5.i1.p1.1),[2nd item](https://arxiv.org/html/2607.12354#A1.I5.i2.p1.1),[3rd item](https://arxiv.org/html/2607.12354#A1.I5.i3.p1.1),[§C\.2](https://arxiv.org/html/2607.12354#A3.SS2.p2.1)\.
- K\. Khaled, G\. Nicolescu, and F\. G\. De Magalhães \(2022\)Careful What You Wish For: on the Extraction of Adversarially Trained Models\.In2022 19th Annual International Conference on Privacy, Security & Trust \(PST\),pp\. 1–10\.External Links:[Link](https://ieeexplore.ieee.org/document/9851981/),[Document](https://dx.doi.org/10.1109/PST55820.2022.9851981)Cited by:[§F\.2](https://arxiv.org/html/2607.12354#A6.SS2.p1.1)\.
- A\. Khosla, N\. Jayadevaprakash, B\. Yao, and F\. Li \(2011\)Novel dataset for fine\-grained image categorization: stanford dogs\.InProc\. CVPR workshop on fine\-grained visual categorization \(FGVC\),Vol\.2\.Cited by:[3rd item](https://arxiv.org/html/2607.12354#A1.I3.i3.p1.1.1),[§2](https://arxiv.org/html/2607.12354#S2.p1.1)\.
- J\. H\. Koh, S\. Ho, N\. Nguyen, and N\. Cheung \(2024\)On the Vulnerability of Skip Connections to Model Inversion Attacks\.arXiv\.Note:arXiv:2409\.01696 \[cs\]External Links:[Link](http://arxiv.org/abs/2409.01696),[Document](https://dx.doi.org/10.48550/arXiv.2409.01696)Cited by:[2nd item](https://arxiv.org/html/2607.12354#A1.I2.i2.p1.1.1),[3rd item](https://arxiv.org/html/2607.12354#A1.I2.i3.p1.3.1),[1st item](https://arxiv.org/html/2607.12354#A1.I3.i1.p1.1),[3rd item](https://arxiv.org/html/2607.12354#A1.I3.i3.p1.1),[1st item](https://arxiv.org/html/2607.12354#A1.I4.i1.p1.1),[3rd item](https://arxiv.org/html/2607.12354#A1.I4.i3.p1.1),[§2](https://arxiv.org/html/2607.12354#S2.p1.1),[§2](https://arxiv.org/html/2607.12354#S2.p2.1)\.
- A\. Lacoste, A\. Luccioni, V\. Schmidt, and T\. Dandres \(2019\)Quantifying the Carbon Emissions of Machine Learning\.arXiv\.Note:arXiv:1910\.09700 \[cs\]External Links:[Link](http://arxiv.org/abs/1910.09700),[Document](https://dx.doi.org/10.48550/arXiv.1910.09700)Cited by:[§A\.6](https://arxiv.org/html/2607.12354#A1.SS6.p1.1)\.
- I\. Lemhadri, F\. Ruan, L\. Abraham, and R\. Tibshirani \(2021\)LassoNet: A Neural Network with Feature Sparsity\.arXiv\.Note:arXiv:1907\.12207External Links:[Link](http://arxiv.org/abs/1907.12207)Cited by:[§E\.1](https://arxiv.org/html/2607.12354#A5.SS1.p1.7)\.
- Y\. Li, C\. Chen, and W\. W\. Wasserman \(2016\)Deep Feature Selection: Theory and Application to Identify Enhancers and Promoters\.Journal of Computational Biology23\(5\),pp\. 322–336\(en\)\.External Links:ISSN 1066\-5277, 1557\-8666,[Link](http://www.liebertpub.com/doi/10.1089/cmb.2015.0189),[Document](https://dx.doi.org/10.1089/cmb.2015.0189)Cited by:[§E\.1](https://arxiv.org/html/2607.12354#A5.SS1.p2.3)\.
- R\. Liu, D\. Wang, Y\. Ren, Z\. Wang, K\. Guo, Q\. Qin, and X\. Liu \(2024\)Unstoppable Attack: Label\-Only Model Inversion Via Conditional Diffusion Model\.IEEE Transactions on Information Forensics and Security19,pp\. 3958–3973\.External Links:ISSN 1556\-6021,[Link](https://ieeexplore.ieee.org/document/10458692/),[Document](https://dx.doi.org/10.1109/TIFS.2024.3372815)Cited by:[§C\.3](https://arxiv.org/html/2607.12354#A3.SS3.p1.1)\.
- Z\. Liu and S\. Chen \(2024\)Trap\-MID: trapdoor\-based defense against model inversion attacks\.InThe Thirty\-eighth Annual Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=GNhrGRCerd)Cited by:[4th item](https://arxiv.org/html/2607.12354#A1.I2.i4.p1.2.1),[2nd item](https://arxiv.org/html/2607.12354#A1.I3.i2.p1.1),[3rd item](https://arxiv.org/html/2607.12354#A1.I4.i3.p1.1),[§2](https://arxiv.org/html/2607.12354#S2.p1.1),[§2](https://arxiv.org/html/2607.12354#S2.p2.1),[§4](https://arxiv.org/html/2607.12354#S4.p3.1)\.
- Z\. Liu, P\. Luo, X\. Wang, and X\. Tang \(2015\)Deep Learning Face Attributes in the Wild\.In2015 IEEE International Conference on Computer Vision \(ICCV\),Santiago, Chile,pp\. 3730–3738\(en\)\.External Links:ISBN 978\-1\-4673\-8391\-2,[Link](http://ieeexplore.ieee.org/document/7410782/),[Document](https://dx.doi.org/10.1109/ICCV.2015.425)Cited by:[2nd item](https://arxiv.org/html/2607.12354#A1.I3.i2.p1.1.1),[§2](https://arxiv.org/html/2607.12354#S2.p1.1)\.
- A\. Madry, A\. Makelov, L\. Schmidt, D\. Tsipras, and A\. Vladu \(2017\)Towards Deep Learning Models Resistant to Adversarial Attacks\.arXiv\.Note:arXiv:1706\.06083 \[stat\]External Links:[Link](http://arxiv.org/abs/1706.06083),[Document](https://dx.doi.org/10.48550/arXiv.1706.06083)Cited by:[1st item](https://arxiv.org/html/2607.12354#A1.I4.i1.p1.1),[Appendix B](https://arxiv.org/html/2607.12354#A2.p1.7),[§F\.1](https://arxiv.org/html/2607.12354#A6.SS1.p1.6),[§F\.1](https://arxiv.org/html/2607.12354#A6.SS1.p2.1),[§F\.2](https://arxiv.org/html/2607.12354#A6.SS2.p2.1),[§1](https://arxiv.org/html/2607.12354#S1.p21.1),[§5](https://arxiv.org/html/2607.12354#S5.p4.14)\.
- S\. Mehnaz, S\. V\. Dibbo, E\. Kabir, N\. Li, and E\. Bertino \(2022\)Are your sensitive attributes private? novel model inversion attribute inference attacks on classification models\.In31st USENIX Security Symposium \(USENIX Security 22\),Boston, MA,pp\. 4579–4596\.External Links:ISBN 978\-1\-939133\-31\-1,[Link](https://www.usenix.org/conference/usenixsecurity22/presentation/mehnaz)Cited by:[§1](https://arxiv.org/html/2607.12354#S1.p1.1)\.
- F\. A\. Mejia, P\. Gamble, Z\. Hampel\-Arias, M\. Lomnitz, N\. Lopatina, L\. Tindall, and M\. A\. Barrios \(2019\)Robust or Private? Adversarial Training Makes Models More Vulnerable to Privacy Attacks\.arXiv\.Note:arXiv:1906\.06449 \[cs\]External Links:[Link](http://arxiv.org/abs/1906.06449),[Document](https://dx.doi.org/10.48550/arXiv.1906.06449)Cited by:[§F\.2](https://arxiv.org/html/2607.12354#A6.SS2.p2.1),[§F\.2](https://arxiv.org/html/2607.12354#A6.SS2.p4.1)\.
- F\. Naretto, A\. Monreale, and F\. Giannotti \(2022\)Evaluating the privacy exposure of interpretable global explainers\.In2022 IEEE 4th International Conference on Cognitive Machine Intelligence \(CogMI\),Vol\.,pp\. 13–19\.External Links:[Document](https://dx.doi.org/10.1109/CogMI56440.2022.00012)Cited by:[§F\.2](https://arxiv.org/html/2607.12354#A6.SS2.p1.1)\.
- M\. Nasr, N\. Carlini, J\. Hayase, M\. Jagielski, A\. F\. Cooper, D\. Ippolito, C\. A\. Choquette\-Choo, E\. Wallace, F\. Tramèr, and K\. Lee \(2023\)Scalable Extraction of Training Data from \(Production\) Language Models\.arXiv\(en\)\.Note:arXiv:2311\.17035 \[cs\]External Links:[Link](http://arxiv.org/abs/2311.17035)Cited by:[§1](https://arxiv.org/html/2607.12354#S1.p1.1)\.
- H\. Ng and S\. Winkler \(2014\)A data\-driven approach to cleaning large face datasets\.In2014 IEEE International Conference on Image Processing \(ICIP\),pp\. 343–347\.Note:ISSN: 2381\-8549External Links:[Link](https://ieeexplore.ieee.org/document/7025068/),[Document](https://dx.doi.org/10.1109/ICIP.2014.7025068)Cited by:[1st item](https://arxiv.org/html/2607.12354#A1.I3.i1.p1.1.1),[§2](https://arxiv.org/html/2607.12354#S2.p1.1)\.
- N\. Nguyen, K\. Chandrasegaran, M\. Abdollahzadeh, and N\. Cheung \(2023a\)Label\-Only Model Inversion Attacks via Knowledge Transfer\.InAdvances in Neural Information Processing Systems,\(en\)\.Cited by:[§C\.3](https://arxiv.org/html/2607.12354#A3.SS3.p1.1)\.
- N\. Nguyen, K\. Chandrasegaran, M\. Abdollahzadeh, and N\. Cheung \(2023b\)Re\-Thinking Model Inversion Attacks Against Deep Neural Networks\.In2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),Vancouver, BC, Canada,pp\. 16384–16393\(en\)\.External Links:ISBN 979\-8\-3503\-0129\-8,[Link](https://ieeexplore.ieee.org/document/10203343/),[Document](https://dx.doi.org/10.1109/CVPR52729.2023.01572)Cited by:[§C\.2](https://arxiv.org/html/2607.12354#A3.SS2.p1.1)\.
- X\. Peng, B\. Han, F\. Liu, T\. Liu, and M\. Zhou \(2024\)Pseudo\-private data guided model inversion attacks\.InThe Thirty\-eighth Annual Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=pyqPUf36D2)Cited by:[3rd item](https://arxiv.org/html/2607.12354#A1.I5.i3.p1.1.1),[§2](https://arxiv.org/html/2607.12354#S2.p1.1)\.
- X\. Peng, F\. Liu, N\. Wang, L\. Lan, T\. Liu, Y\. Cheung, and B\. Han \(2025\)Unknown\-Aware Bilateral Dependency Optimization for Defending Against Model Inversion Attacks\.IEEE Transactions on Pattern Analysis and Machine Intelligence,pp\. 1–13\.External Links:ISSN 1939\-3539,[Link](https://ieeexplore.ieee.org/document/10949821/),[Document](https://dx.doi.org/10.1109/TPAMI.2025.3558267)Cited by:[§3](https://arxiv.org/html/2607.12354#S3.p2.2)\.
- X\. Peng, F\. Liu, J\. Zhang, L\. Lan, J\. Ye, T\. Liu, and B\. Han \(2022\)Bilateral Dependency Optimization: Defending Against Model\-inversion Attacks\.InProceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining,Washington DC USA,pp\. 1358–1367\(en\)\.External Links:ISBN 978\-1\-4503\-9385\-0,[Link](https://dl.acm.org/doi/10.1145/3534678.3539376),[Document](https://dx.doi.org/10.1145/3534678.3539376)Cited by:[3rd item](https://arxiv.org/html/2607.12354#A1.I1.i3.p1.1),[3rd item](https://arxiv.org/html/2607.12354#A1.I1.i3.p1.1.1),[§D\.1](https://arxiv.org/html/2607.12354#A4.SS1.p1.13),[§D\.1](https://arxiv.org/html/2607.12354#A4.SS1.p1.5),[§D\.1](https://arxiv.org/html/2607.12354#A4.SS1.p2.12),[§1](https://arxiv.org/html/2607.12354#S1.p6.1),[§1](https://arxiv.org/html/2607.12354#S1.p7.1),[§2](https://arxiv.org/html/2607.12354#S2.p1.1),[§2](https://arxiv.org/html/2607.12354#S2.p2.1),[§3](https://arxiv.org/html/2607.12354#S3.p2.2),[§3](https://arxiv.org/html/2607.12354#S3.p3.1)\.
- Y\. Qiu, H\. Fang, H\. Yu, B\. Chen, M\. Qiu, and S\. Xia \(2024\)A Closer Look at GAN Priors: Exploiting Intermediate Features for Enhanced Model Inversion Attacks\.arXiv\.Note:arXiv:2407\.13863External Links:[Link](http://arxiv.org/abs/2407.13863)Cited by:[1st item](https://arxiv.org/html/2607.12354#A1.I3.i1.p1.1),[2nd item](https://arxiv.org/html/2607.12354#A1.I3.i2.p1.1),[3rd item](https://arxiv.org/html/2607.12354#A1.I3.i3.p1.1),[1st item](https://arxiv.org/html/2607.12354#A1.I4.i1.p1.1),[2nd item](https://arxiv.org/html/2607.12354#A1.I4.i2.p1.1),[3rd item](https://arxiv.org/html/2607.12354#A1.I4.i3.p1.1),[2nd item](https://arxiv.org/html/2607.12354#A1.I5.i2.p1.1.1),[§C\.2](https://arxiv.org/html/2607.12354#A3.SS2.p2.1),[§C\.3](https://arxiv.org/html/2607.12354#A3.SS3.p1.1),[§1](https://arxiv.org/html/2607.12354#S1.p2.1),[§2](https://arxiv.org/html/2607.12354#S2.p1.1)\.
- Y\. Qiu, H\. Yu, H\. Fang, T\. Zhuang, W\. Yu, B\. Chen, X\. Wang, S\. Xia, and K\. Xu \(2025\)MIBench: a comprehensive framework for benchmarking model inversion attack and defense\.External Links:2410\.05159,[Link](https://arxiv.org/abs/2410.05159)Cited by:[§C\.3](https://arxiv.org/html/2607.12354#A3.SS3.p1.1)\.
- S\. Scardapane, D\. Comminiello, A\. Hussain, and A\. Uncini \(2017\)Group sparse regularization for deep neural networks\.Neurocomputing241,pp\. 81–89\(en\)\.External Links:ISSN 09252312,[Link](https://linkinghub.elsevier.com/retrieve/pii/S0925231217302990),[Document](https://dx.doi.org/10.1016/j.neucom.2017.02.029)Cited by:[§E\.1](https://arxiv.org/html/2607.12354#A5.SS1.p1.7)\.
- A\. Shafahi, M\. Najibi, A\. Ghiasi, Z\. Xu, J\. Dickerson, C\. Studer, L\. S\. Davis, G\. Taylor, and T\. Goldstein \(2019\)Adversarial Training for Free\!\.arXiv\.Note:arXiv:1904\.12843 \[cs\]External Links:[Link](http://arxiv.org/abs/1904.12843),[Document](https://dx.doi.org/10.48550/arXiv.1904.12843)Cited by:[§B\.1](https://arxiv.org/html/2607.12354#A2.SS1.p1.8),[§F\.1](https://arxiv.org/html/2607.12354#A6.SS1.p2.1)\.
- K\. Shah, A\. Kalavasis, A\. R\. Klivans, and G\. Daras \(2025\)Does Generation Require Memorization? Creative Diffusion Models using Ambient Diffusion\.arXiv\.Note:arXiv:2502\.21278 \[cs\]External Links:[Link](http://arxiv.org/abs/2502.21278),[Document](https://dx.doi.org/10.48550/arXiv.2502.21278)Cited by:[§1](https://arxiv.org/html/2607.12354#S1.p5.1)\.
- W\. Shi, A\. Ajith, M\. Xia, Y\. Huang, D\. Liu, T\. Blevins, D\. Chen, and L\. Zettlemoyer \(2024\)Detecting pretraining data from large language models\.InThe Twelfth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=zWqr3MQuNs)Cited by:[§1](https://arxiv.org/html/2607.12354#S1.p1.1)\.
- R\. Shokri, M\. Strobel, and Y\. Zick \(2021\)On the Privacy Risks of Model Explanations\.InProceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society,Virtual Event USA,pp\. 231–241\(en\)\.External Links:ISBN 978\-1\-4503\-8473\-5,[Link](https://dl.acm.org/doi/10.1145/3461702.3462533),[Document](https://dx.doi.org/10.1145/3461702.3462533)Cited by:[§F\.2](https://arxiv.org/html/2607.12354#A6.SS2.p1.1)\.
- R\. Shokri, M\. Stronati, C\. Song, and V\. Shmatikov \(2017\)Membership Inference Attacks Against Machine Learning Models\.In2017 IEEE Symposium on Security and Privacy \(SP\),pp\. 3–18\.Note:ISSN: 2375\-1207External Links:[Link](https://ieeexplore.ieee.org/document/7958568/),[Document](https://dx.doi.org/10.1109/SP.2017.41)Cited by:[§1](https://arxiv.org/html/2607.12354#S1.p1.1)\.
- L\. Song, R\. Shokri, and P\. Mittal \(2019\)Privacy Risks of Securing Machine Learning Models against Adversarial Examples\.InProceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security,London United Kingdom,pp\. 241–257\(en\)\.External Links:ISBN 978\-1\-4503\-6747\-9,[Link](https://dl.acm.org/doi/10.1145/3319535.3354211),[Document](https://dx.doi.org/10.1145/3319535.3354211)Cited by:[§F\.2](https://arxiv.org/html/2607.12354#A6.SS2.p1.1)\.
- L\. Struppek, D\. Hintersdorf, A\. D\. A\. Correia, A\. Adler, and K\. Kersting \(2022\)Plug & Play Attacks: Towards Robust and Flexible Model Inversion Attacks\.arXiv\(en\)\.Note:arXiv:2201\.12179 \[cs\]External Links:[Link](http://arxiv.org/abs/2201.12179)Cited by:[1st item](https://arxiv.org/html/2607.12354#A1.I3.i1.p1.1),[2nd item](https://arxiv.org/html/2607.12354#A1.I3.i2.p1.1),[3rd item](https://arxiv.org/html/2607.12354#A1.I3.i3.p1.1),[1st item](https://arxiv.org/html/2607.12354#A1.I4.i1.p1.1),[2nd item](https://arxiv.org/html/2607.12354#A1.I4.i2.p1.1),[3rd item](https://arxiv.org/html/2607.12354#A1.I4.i3.p1.1),[1st item](https://arxiv.org/html/2607.12354#A1.I5.i1.p1.1),[1st item](https://arxiv.org/html/2607.12354#A1.I5.i1.p1.1.1),[§A\.2](https://arxiv.org/html/2607.12354#A1.SS2.p1.2),[§A\.3](https://arxiv.org/html/2607.12354#A1.SS3.p1.3),[§C\.2](https://arxiv.org/html/2607.12354#A3.SS2.p2.1),[§C\.3](https://arxiv.org/html/2607.12354#A3.SS3.p1.1),[§1](https://arxiv.org/html/2607.12354#S1.p1.1),[§1](https://arxiv.org/html/2607.12354#S1.p2.1),[§2](https://arxiv.org/html/2607.12354#S2.p1.1)\.
- L\. Struppek, D\. Hintersdorf, and K\. Kersting \(2024\)Be Careful What You Smooth For: Label Smoothing Can Be a Privacy Shield but Also a Catalyst for Model Inversion Attacks\.arXiv\.Note:arXiv:2310\.06549 \[cs\]External Links:[Link](http://arxiv.org/abs/2310.06549),[Document](https://dx.doi.org/10.48550/arXiv.2310.06549)Cited by:[2nd item](https://arxiv.org/html/2607.12354#A1.I1.i2.p1.2),[3rd item](https://arxiv.org/html/2607.12354#A1.I1.i3.p1.1),[1st item](https://arxiv.org/html/2607.12354#A1.I2.i1.p1.1.1),[1st item](https://arxiv.org/html/2607.12354#A1.I3.i1.p1.1),[2nd item](https://arxiv.org/html/2607.12354#A1.I3.i2.p1.1),[1st item](https://arxiv.org/html/2607.12354#A1.I4.i1.p1.1),[1st item](https://arxiv.org/html/2607.12354#A1.I5.i1.p1.1),[§A\.2](https://arxiv.org/html/2607.12354#A1.SS2.p1.2),[§A\.3](https://arxiv.org/html/2607.12354#A1.SS3.p1.3),[§2](https://arxiv.org/html/2607.12354#S2.p1.1),[§2](https://arxiv.org/html/2607.12354#S2.p2.1),[§3](https://arxiv.org/html/2607.12354#S3.p9.2),[§4](https://arxiv.org/html/2607.12354#S4.p3.1)\.
- C\. Szegedy, W\. Zaremba, I\. Sutskever, J\. Bruna, D\. Erhan, I\. Goodfellow, and R\. Fergus \(2014\)Intriguing properties of neural networks\.arXiv\.Note:arXiv:1312\.6199 \[cs\]External Links:[Link](http://arxiv.org/abs/1312.6199),[Document](https://dx.doi.org/10.48550/arXiv.1312.6199)Cited by:[§F\.1](https://arxiv.org/html/2607.12354#A6.SS1.p1.6),[§F\.1](https://arxiv.org/html/2607.12354#A6.SS1.p2.1),[§1](https://arxiv.org/html/2607.12354#S1.p17.1)\.
- R\. Tibshirani \(1996\)Regression Shrinkage and Selection Via the Lasso\.Journal of the Royal Statistical Society Series B: Statistical Methodology58\(1\),pp\. 267–288\(en\)\.External Links:ISSN 1369\-7412, 1467\-9868,[Link](https://academic.oup.com/jrsssb/article/58/1/267/7027929),[Document](https://dx.doi.org/10.1111/j.2517-6161.1996.tb02080.x)Cited by:[§E\.1](https://arxiv.org/html/2607.12354#A5.SS1.p1.7)\.
- K\. Tirumala, A\. H\. Markosyan, L\. Zettlemoyer, and A\. Aghajanyan \(2022\)Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models\.arXiv\.Note:arXiv:2205\.10770 \[cs\]External Links:[Link](http://arxiv.org/abs/2205.10770),[Document](https://dx.doi.org/10.48550/arXiv.2205.10770)Cited by:[§1](https://arxiv.org/html/2607.12354#S1.p5.1)\.
- N\. Tursynbek, A\. Petiushko, and I\. Oseledets \(2021\)Robustness Threats of Differential Privacy\.arXiv\.Note:arXiv:2012\.07828 \[cs\]External Links:[Link](http://arxiv.org/abs/2012.07828),[Document](https://dx.doi.org/10.48550/arXiv.2012.07828)Cited by:[§F\.2](https://arxiv.org/html/2607.12354#A6.SS2.p1.1)\.
- K\. Wang, Y\. Fu, K\. Li, A\. Khisti, R\. Zemel, and A\. Makhzani \(2022\)Variational Model Inversion Attacks\.arXiv\.Note:arXiv:2201\.10787 \[cs\]External Links:[Link](http://arxiv.org/abs/2201.10787),[Document](https://dx.doi.org/10.48550/arXiv.2201.10787)Cited by:[§C\.2](https://arxiv.org/html/2607.12354#A3.SS2.p1.1)\.
- T\. Wang, Y\. Zhang, and R\. Jia \(2020\)Improving Robustness to Model Inversion Attacks via Mutual Information Regularization\.arXiv\.Note:arXiv:2009\.05241 \[cs\]External Links:[Link](http://arxiv.org/abs/2009.05241),[Document](https://dx.doi.org/10.48550/arXiv.2009.05241)Cited by:[2nd item](https://arxiv.org/html/2607.12354#A1.I1.i2.p1.2.1),[§1](https://arxiv.org/html/2607.12354#S1.p6.1),[§1](https://arxiv.org/html/2607.12354#S1.p7.1),[§2](https://arxiv.org/html/2607.12354#S2.p2.1)\.
- W\. Wang, M\. A\. Kaleem, A\. Dziedzic, M\. Backes, N\. Papernot, and F\. Boenisch \(2024\)Memorization in Self\-Supervised Learning Improves Downstream Generalization\.arXiv\.Note:arXiv:2401\.12233 \[cs\]External Links:[Link](http://arxiv.org/abs/2401.12233),[Document](https://dx.doi.org/10.48550/arXiv.2401.12233)Cited by:[§1](https://arxiv.org/html/2607.12354#S1.p5.1)\.
- J\. Wen, S\. Yiu, and L\. C\.K\. Hui \(2021\)Defending Against Model Inversion Attack by Adversarial Examples\.In2021 IEEE International Conference on Cyber Security and Resilience \(CSR\),pp\. 551–556\.External Links:[Link](https://ieeexplore.ieee.org/document/9527945/),[Document](https://dx.doi.org/10.1109/CSR51186.2021.9527945)Cited by:[§F\.2](https://arxiv.org/html/2607.12354#A6.SS2.p3.1)\.
- E\. Wong, L\. Rice, and J\. Z\. Kolter \(2020\)Fast is better than free: Revisiting adversarial training\.arXiv\.Note:arXiv:2001\.03994 \[cs\]External Links:[Link](http://arxiv.org/abs/2001.03994),[Document](https://dx.doi.org/10.48550/arXiv.2001.03994)Cited by:[§B\.1](https://arxiv.org/html/2607.12354#A2.SS1.p1.8),[§F\.1](https://arxiv.org/html/2607.12354#A6.SS1.p2.1)\.
- N\. Xu, F\. Liu, and D\. J\. Sutherland \(2025\)Learning representations for independence testing\.External Links:2409\.06890,[Link](https://arxiv.org/abs/2409.06890)Cited by:[Appendix D](https://arxiv.org/html/2607.12354#A4.p1.1)\.
- Z\. Yang, J\. Zhang, E\. Chang, and Z\. Liang \(2019\)Neural Network Inversion in Adversarial Setting via Background Knowledge Alignment\.InProceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security,London United Kingdom,pp\. 225–240\.External Links:[Link](https://dl.acm.org/doi/10.1145/3319535.3354261),[Document](https://dx.doi.org/10.1145/3319535.3354261)Cited by:[§C\.3](https://arxiv.org/html/2607.12354#A3.SS3.p1.1)\.
- S\. Yeom, I\. Giacomelli, M\. Fredrikson, and S\. Jha \(2018\)Privacy Risk in Machine Learning: Analyzing the Connection to Overfitting\.arXiv\.Note:arXiv:1709\.01604External Links:[Link](http://arxiv.org/abs/1709.01604)Cited by:[§1](https://arxiv.org/html/2607.12354#S1.p1.1)\.
- X\. Yuan, K\. Chen, J\. Zhang, W\. Zhang, N\. Yu, and Y\. Zhang \(2023\)Pseudo Label\-Guided Model Inversion Attack via Conditional Generative Adversarial Network\.arXiv\.Note:arXiv:2302\.09814 \[cs\]External Links:[Link](http://arxiv.org/abs/2302.09814),[Document](https://dx.doi.org/10.48550/arXiv.2302.09814)Cited by:[§C\.2](https://arxiv.org/html/2607.12354#A3.SS2.p1.1)\.
- C\. Zhang, S\. Bengio, M\. Hardt, B\. Recht, and O\. Vinyals \(2017\)Understanding deep learning requires rethinking generalization\.arXiv\.Note:arXiv:1611\.03530 \[cs\]External Links:[Link](http://arxiv.org/abs/1611.03530),[Document](https://dx.doi.org/10.48550/arXiv.1611.03530)Cited by:[§3](https://arxiv.org/html/2607.12354#S3.p6.1)\.
- H\. Zhang, J\. Wang, Z\. Sun, J\. M\. Zurada, and N\. R\. Pal \(2020a\)Feature Selection for Neural Networks Using Group Lasso Regularization\.IEEE Transactions on Knowledge and Data Engineering32\(4\),pp\. 659–673\.Note:Conference Name: IEEE Transactions on Knowledge and Data EngineeringExternal Links:ISSN 1558\-2191,[Link](https://ieeexplore.ieee.org/document/8613832/?arnumber=8613832),[Document](https://dx.doi.org/10.1109/TKDE.2019.2893266)Cited by:[§E\.1](https://arxiv.org/html/2607.12354#A5.SS1.p1.7)\.
- Y\. Zhang and Z\. Bu \(2023\)Differentially Private Optimizers Can Learn Adversarially Robust Models\.arXiv\.Note:arXiv:2211\.08942 \[cs\]External Links:[Link](http://arxiv.org/abs/2211.08942),[Document](https://dx.doi.org/10.48550/arXiv.2211.08942)Cited by:[§F\.2](https://arxiv.org/html/2607.12354#A6.SS2.p1.1)\.
- Y\. Zhang, R\. Jia, H\. Pei, W\. Wang, B\. Li, and D\. Song \(2020b\)The Secret Revealer: Generative Model\-Inversion Attacks Against Deep Neural Networks\.arXiv\.Note:arXiv:1911\.07135External Links:[Link](http://arxiv.org/abs/1911.07135)Cited by:[§C\.1](https://arxiv.org/html/2607.12354#A3.SS1.p1.8),[§C\.2](https://arxiv.org/html/2607.12354#A3.SS2.p1.1),[§C\.3](https://arxiv.org/html/2607.12354#A3.SS3.p1.1)\.
- L\. Zhao, Q\. Hu, and W\. Wang \(2015\)Heterogeneous Feature Selection With Multi\-Modal Deep Neural Networks and Sparse Group LASSO\.IEEE Transactions on Multimedia17\(11\),pp\. 1936–1948\.External Links:ISSN 1941\-0077,[Link](https://ieeexplore.ieee.org/document/7244241/),[Document](https://dx.doi.org/10.1109/TMM.2015.2477058)Cited by:[§E\.1](https://arxiv.org/html/2607.12354#A5.SS1.p1.7)\.
- S\. Zhou, T\. Zhu, D\. Ye, X\. Yu, and W\. Zhou \(2024\)Boosting Model Inversion Attacks With Adversarial Examples\.IEEE Transactions on Dependable and Secure Computing21\(3\),pp\. 1451–1468\.External Links:ISSN 1941\-0018,[Link](https://ieeexplore.ieee.org/document/10148576/),[Document](https://dx.doi.org/10.1109/TDSC.2023.3285015)Cited by:[§F\.2](https://arxiv.org/html/2607.12354#A6.SS2.p2.1)\.
- T\. Zhu, D\. Ye, S\. Zhou, B\. Liu, and W\. Zhou \(2023\)Label\-only model inversion attacks: attack with the least information\.IEEE Transactions on Information Forensics and Security18\(\),pp\. 991–1005\.External Links:[Document](https://dx.doi.org/10.1109/TIFS.2022.3233190)Cited by:[§C\.3](https://arxiv.org/html/2607.12354#A3.SS3.p1.1)\.
- T\. Zhuang, H\. Yu, Y\. Qiu, H\. Fang, B\. Chen, and S\. Xia \(2025\)Stealthy shield defense: a conditional mutual information\-based approach against black\-box model inversion attacks\.InThe Thirteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=p0DjhjPXl3)Cited by:[§C\.3](https://arxiv.org/html/2607.12354#A3.SS3.p1.1)\.
- H\. Zou and T\. Hastie \(2005\)Regularization and Variable Selection Via the Elastic Net\.Journal of the Royal Statistical Society Series B: Statistical Methodology67\(2\),pp\. 301–320\(en\)\.External Links:ISSN 1369\-7412, 1467\-9868,[Link](https://academic.oup.com/jrsssb/article/67/2/301/7109482),[Document](https://dx.doi.org/10.1111/j.1467-9868.2005.00503.x)Cited by:[§E\.1](https://arxiv.org/html/2607.12354#A5.SS1.p2.3)\.

## Appendix ADeferred Details of Experiments

### A\.1Target Model Baseline Defenses

We consider two types of defense baselines\. Our primary baselines are defenses that attempt to generically increase privacy by reducing information dependency/memorization\. In each case, unless otherwise mentioned, we set all parameters identically to those used by their authors, and run defenses using the authors’ replication codes:

- •No Defense \(NoDef\)\.We consider vanilla ResNet\-152, ResNet\-18, and DenseNet\-169 baselines, trained without additional privacy defenses\.
- •Mutual Information Defense \(MID\)\(Wanget al\.,[2020](https://arxiv.org/html/2607.12354#bib.bib63)\)\.Inspired by previous information bottleneck research, MID introduces a regularizer that reduces training\-data\-to\-model information dependency by penalizing \(an approximation of\) mutual information between training examples and the model’s intermediate feature representations\. We adapt the authors’ original implementation to work with ResNet architectures; following previous work \(see e\.g\.Struppeket al\.\([2024](https://arxiv.org/html/2607.12354#bib.bib95)\)\), we add the information bottleneck layer between the average pooling and final linear layer, and set the bottleneck size tok=1024k=1024\. Also following previous work, we set the regularization penaltyβ=0\.01\\beta=0\.01\. On FaceScrub MID models, we use a lower learning rate of 0\.0001, as we found it to provide a stronger defense at similar accuracy to the typical learning rate of 0\.001\. We drop the last batch of the validation set to ensure compatibility with the authors’ code\.
- •Bilateral Dependency Optimization \(BiDO\)\(Penget al\.,[2022](https://arxiv.org/html/2607.12354#bib.bib303)\)\.BiDO introduces two regularizers, the first of which minimizes a kernel\-based information dependency measure between inputs and intermediate embeddings \(i\.e\. to penalize data\-to\-model information dependency\), while the second maximizes the same dependency measure between intermediate embeddings and outputs \(i\.e\. to reward accuracy\)\. Following recent work \(see e\.g\.Struppeket al\.\([2024](https://arxiv.org/html/2607.12354#bib.bib95)\); Penget al\.\([2022](https://arxiv.org/html/2607.12354#bib.bib303)\)\), we use the outputs of the four main ResNet blocks as the intermediate embeddings for the regularized loss\. We drop the last batch of the validation set to ensure compatibility with the authors’ code\. We use the standard BiDO hyperparameters\(λx,λy\)=\(0\.05,0\.5\)\(\\lambda\_\{x\},\\lambda\_\{y\}\)=\(0\.05,0\.5\)both for our main results tables comparing baselines to AT\-AT \(Tables[23](https://arxiv.org/html/2607.12354#A9.T23)and[25](https://arxiv.org/html/2607.12354#A9.T25)\), as well as in our adversarial robustness experiments \(Tables[11](https://arxiv.org/html/2607.12354#A6.T11),[12](https://arxiv.org/html/2607.12354#A6.T12),[13](https://arxiv.org/html/2607.12354#A6.T13),[14](https://arxiv.org/html/2607.12354#A6.T14)\)\. See App\.[D](https://arxiv.org/html/2607.12354#A4)for our additional analysis of other alternative BiDO hyperparameter values\.
- •Transfer Learning Defense against Model Inversion \(TL\-DMI\)\(Hoet al\.,[2024](https://arxiv.org/html/2607.12354#bib.bib45)\)\.When training target models, defenses typically perform full fine\-tuning on𝒟p​r​i​v\\mathcal\{D\}\_\{priv\}of a model pre\-trained on some public dataset; TL\-DMI instead fine\-tunes only later layers of the target model, which is argued to reduce the number of parameters encoding sensitive information within the target model\. We use the authors’ original implementation with the minimal refactoring necessary to ensure compatibility with our codebase\. We additionally remark that the authors of TL\-DMI do not claim that the training data encoding that is relevant is necessarily rote\-memorized training data, though practitioners widely assume that this is so\. However, we note that it is conceivable that this encoded information the authors measure and discuss was unknowingly the very same non\-robust features that we find to be operative here\.
- •TL\-DMI\-5\.TL\-DMI is typically implemented for ResNets by freezing the first 6 layers of the network\. While we find that this parameter choice works well in many cases, we nonetheless find that it fails to obtain competitive test accuracy for on the CelebA dataset in certain cases \(particularly when run on lower\-capacity architectures like ResNet\-18\)\. Therefore, to give TL\-DMI the best opportunity to perform well, we also consider an additional variant, TL\-DMI\-5, which freezes just 5 layers\. We report this additional baseline wherever it offers a performance improvement over the recommended 6\-layer version\.

We note that we exclude the Sparse\-Coding Architecture \(SCA\) defense\(Dibboet al\.,[2024](https://arxiv.org/html/2607.12354#bib.bib62)\)in our experiments, as it is not practical to run in the high\-resolution scenario that is the focus of this work\.

Second, for the sake of comparison, we also consider defenses that attempt to suppress gradients from MIAs, rather than increase generic privacy\. Again, as above, we set all parameters identically to those used by their authors, and run defenses using the authors’ replication codes:

- •Negative Label Smoothing \(NegLS\)\(Struppeket al\.,[2024](https://arxiv.org/html/2607.12354#bib.bib95)\)\.The NegLS defense employs negative label smoothing to make the target model consistently overconfident in its predictions\. This ‘deliberate overconfidence’ does not remove information about the training data from the model\. Instead, this overconfidence inhibits PPA attacks \(and similar gradient\-based GAN attacks, including IF\-GMI\) that rely on a smooth landscape of target model confidence scores to compute the gradient updates they use in their reconstruction steps\. We use the authors’ original implementation with the minimal refactoring necessary to ensure compatibility with our codebase\.
- •Removal of Last Stage Skip\-Connection \(RoLSS\)\(Kohet al\.,[2024](https://arxiv.org/html/2607.12354#bib.bib268)\)\.To impede gradient flow during optimization of PPA and other similar attacks \(including IF\-GMI\), RoLSS proposes to remove the skip connection at the last stage of the target model\. We use the skip\-customized ResNet architectures provided in the RoLSS replication codes for RoLSS and RoLSS\-SSF models\. For the ResNet\-18 RoLSS model on FaceScrub, we use a lower initial learning rate of 0\.0001, as we found it to provide a stronger defense at similar accuracy, compared to the typical learning rate of 0\.001\.
- •Skip\-Connection Scaling Factor \(RoLSS\-SSF\)\(Kohet al\.,[2024](https://arxiv.org/html/2607.12354#bib.bib268)\)\.This variant of RoLSS weakens the weight of the last\-layer skip connection instead of removing it completely\. We use the standard scaling factork=0\.2k=0\.2\(wherek=1k=1corresponds to an unchanged model, andk=0k=0is equivalent to non\-SSF RoLSS\)\. For the ResNet\-18 RoLSS\-SSF model on FaceScrub, we use a lower initial learning rate of 0\.0001, as we found it to provide a stronger defense at similar accuracy, compared to the typical learning rate of 0\.001\.
- •Trapdoor\-based Defense \(Trap\-MID\)\(Liu and Chen,[2024](https://arxiv.org/html/2607.12354#bib.bib10)\)\.The Trap\-MID defense implements a trapdoor integrated into the model to predict a specific label when the input is injected with the corresponding trigger\. The trapdoor information serves as ashortcutfor MI attacks, leading them to extract trapdoor triggers rather than private data\. We run two sets of hyperparameters suggested by the authors: Trap\-MID denotes their tuned hyperparameters for high\-resolution experiments\(α=0\.007,β=0\.5\)\(\\alpha=0\.007,\\beta=0\.5\), and Trap\-MID\* denotes the hyperparameters for their lower\-resolution experiments\(α=0\.02,β=0\.2\)\(\\alpha=0\.02,\\beta=0\.2\), which we add for additional comparison\. We use the original authors’ code with minimal refactoring for compatibility with our codebase\.

### A\.2Datasets

We consider the three datasets that are the focus of recent experiments on high\-definition MIAs:

- •FaceScrub\(Ng and Winkler,[2014](https://arxiv.org/html/2607.12354#bib.bib104)\)\.FaceScrub is a commonly used facial dataset of 530 celebrity classes \(see e\.g\.Struppeket al\.\([2024](https://arxiv.org/html/2607.12354#bib.bib95)\); Kohet al\.\([2024](https://arxiv.org/html/2607.12354#bib.bib268)\); Hoet al\.\([2024](https://arxiv.org/html/2607.12354#bib.bib45)\); Qiuet al\.\([2024](https://arxiv.org/html/2607.12354#bib.bib300)\); Struppeket al\.\([2022](https://arxiv.org/html/2607.12354#bib.bib36)\)\)\. For ease of reproducibility, we use a popular Kaggle\-hosted version111Available at https://www\.kaggle\.com/datasets/rajnishe/facescrub\-fullof the dataset and supply automatic download and processing code\.
- •CelebA\(Liuet al\.,[2015](https://arxiv.org/html/2607.12354#bib.bib87)\)\.CelebA is a facial celebrity dataset containing 202,599 images of 10,177 celebrities\. Following previous work \(see e\.g\.Struppeket al\.\([2024](https://arxiv.org/html/2607.12354#bib.bib95)\); Liu and Chen \([2024](https://arxiv.org/html/2607.12354#bib.bib10)\); Qiuet al\.\([2024](https://arxiv.org/html/2607.12354#bib.bib300)\); Struppeket al\.\([2022](https://arxiv.org/html/2607.12354#bib.bib36)\)\), we choose the most common 1000 class identities out of the 10,177 available, for a total of 30,038 images\. As is standard, we apply the CelebA HD Cropper222Available at https://github\.com/LynnHo/HD\-CelebA\-Cropperto improve the overall image quality\.
- •Stanford Dogs\(Khoslaet al\.,[2011](https://arxiv.org/html/2607.12354#bib.bib41)\)\.Stanford Dogs is a dog image dataset containing 20,580 images of 120 different species of dogs, commonly used in model inversion \(see e\.g\.Hoet al\.\([2024](https://arxiv.org/html/2607.12354#bib.bib45)\); Kohet al\.\([2024](https://arxiv.org/html/2607.12354#bib.bib268)\); Struppeket al\.\([2022](https://arxiv.org/html/2607.12354#bib.bib36)\); Qiuet al\.\([2024](https://arxiv.org/html/2607.12354#bib.bib300)\)\)\. We use the full dataset333Available at http://vision\.stanford\.edu/aditya86/ImageNetDogs/for our experiments and include automatic download and processing code\.

Each dataset was partitioned into 90% training and 10% test\. From the training set, 20% was used as a validation set for tracking performance and early stopping\. When training target models, we strictly follow the standard MIA data pre\-processing steps \(see e\.g\.Struppeket al\.\([2022](https://arxiv.org/html/2607.12354#bib.bib36);[2024](https://arxiv.org/html/2607.12354#bib.bib95)\), as follows: we normalize the images withμ=σ=\(0\.5,0\.5,0\.5\)\\mu=\\sigma=\(0\.5,0\.5,0\.5\), then resize so that the smaller side of the image is 224 pixels\. We apply random cropping with patch sizes between 85% and 100%, keeping the aspect ratio fixed at 1\.0; we then resize the patches to224×224224\\times 224pixels \(RandomResizedCropping\)\. We then applied random color jitter with saturation and hue factors of 0\.1 and contrast and brightness factors of 0\.2\. Half of all samples are also horizontally flipped\.

### A\.3Target model training details

Unless otherwise mentioned, we train target models with the following parameters, following previous work \(see e\.g\.Struppeket al\.\([2024](https://arxiv.org/html/2607.12354#bib.bib95);[2022](https://arxiv.org/html/2607.12354#bib.bib36)\)\): Before training on𝒟p​r​i​v\\mathcal\{D\}\_\{priv\}, we initialize the models with pre\-trained ImageNet weights available through Torchvision\. We use the Adam optimizer withβ=\(0\.9,0\.999\)\\beta=\(0\.9,0\.999\), and batch size of 128\. Models are trained for a total of 100 epochs, saving the model weights of the epoch with the lowest validation loss\. We measure prediction accuracy on the validation split throughout model training, and after training, we measure prediction accuracy on the test split\. We use an initial learning rate of 0\.001, and we employ a MultiStepLR learning rate scheduler withγ=0\.1\\gamma=0\.1\. Full implementation details for target models are contained in our replication codes\.

#### A\.3\.1Permutation model training details

For the models trained with permuted labels in Sec\.[3](https://arxiv.org/html/2607.12354#S3), we use a lower learning rate of 0\.0001, which we found provided more stable training\. All other training hyperparameters are identical to those described above\. The target models with permuted labels achieve over99\.4%99\.4\\%accuracy on their respective altered training sets\.

### A\.4Architectures

We follow recent MIA work by considering three architectures:

- •ResNet\-152\(Heet al\.,[2015](https://arxiv.org/html/2607.12354#bib.bib86)\)\.As in numerous recent works on high\-resolution MIAs \(e\.g\.,Struppeket al\.\([2022](https://arxiv.org/html/2607.12354#bib.bib36)\); Qiuet al\.\([2024](https://arxiv.org/html/2607.12354#bib.bib300)\); Kohet al\.\([2024](https://arxiv.org/html/2607.12354#bib.bib268)\); Struppeket al\.\([2024](https://arxiv.org/html/2607.12354#bib.bib95)\)\), our main focus is ResNet\-152 models, which exhibit sufficient depth to act as a good proxy for mainstream production vision models\. Also, the adversarial examples literature has shown that models of higher capacity can better articulate robust and non\-robust features well\(Ilyaset al\.,[2019](https://arxiv.org/html/2607.12354#bib.bib101); Madryet al\.,[2017](https://arxiv.org/html/2607.12354#bib.bib246)\), so the ResNet\-152 architecture is a natural choice for testing our hypotheses regarding non\-robust features\.
- •ResNet\-18\(Heet al\.,[2015](https://arxiv.org/html/2607.12354#bib.bib86)\)\.As in recent MIA work \(e\.g\.,Struppeket al\.\([2022](https://arxiv.org/html/2607.12354#bib.bib36)\); Hoet al\.\([2024](https://arxiv.org/html/2607.12354#bib.bib45)\); Qiuet al\.\([2024](https://arxiv.org/html/2607.12354#bib.bib300)\)\), we consider a ResNet\-18 architecture, which permits us to probe how well our results generalize even to lower\-capacity networks with more limited ability to capture complex feature bases\.
- •DenseNet\-169\(Huanget al\.,[2017](https://arxiv.org/html/2607.12354#bib.bib81)\)\.As in recent high\-resolution MIA work \(e\.g\.,Struppeket al\.\([2022](https://arxiv.org/html/2607.12354#bib.bib36)\); Qiuet al\.\([2024](https://arxiv.org/html/2607.12354#bib.bib300)\); Kohet al\.\([2024](https://arxiv.org/html/2607.12354#bib.bib268)\); Liu and Chen \([2024](https://arxiv.org/html/2607.12354#bib.bib10)\)\), we also consider a higher\-capacity DenseNet\-169 architecture, which allows us to test how well our results generalize to other deep convolutional architecture families\.

### A\.5Reconstruction attacks

We consider three recent attacks that obtain state\-of\-the\-art performance for our high\-definition vision setting:

- •Plug\-and\-Play Attack \(PPA\)\(Struppeket al\.,[2022](https://arxiv.org/html/2607.12354#bib.bib36)\)\.Through a novel three\-stage inversion attack framework \(latent vector sampling, optimization, and robust final selection\), PPA leverages a pre\-trained StyleGAN\-2\(Karraset al\.,[2020](https://arxiv.org/html/2607.12354#bib.bib93)\)to allow for impressive and efficient attack performance in the high resolution setting\. StyleGAN\-2 was trained on the FFHQ dataset\(Karraset al\.,[2019](https://arxiv.org/html/2607.12354#bib.bib285)\), which has no identity overlap with CelebA or FaceScrub\. For the Stanford Dogs experiments, the StyleGAN\-2 model used was trained on the AFHQ dogs dataset\(Choiet al\.,[2020](https://arxiv.org/html/2607.12354#bib.bib287)\)\. We adapt the original authors’ code to ensure compatibility with our codebase\. During optimization, samples were optimized for 50 steps for FaceScrub and CelebA target models, and 100 steps for Stanford Dogs\. All other hyperparameters are identical to those used in previous work\(Struppeket al\.,[2022](https://arxiv.org/html/2607.12354#bib.bib36);[2024](https://arxiv.org/html/2607.12354#bib.bib95)\)\.
- •Intermediate Features enhanced Generative Model Inversion \(IF\-GMI\)\(Qiuet al\.,[2024](https://arxiv.org/html/2607.12354#bib.bib300)\)IF\-GMI leverages intermediate features of StyleGAN\-2\(Karraset al\.,[2020](https://arxiv.org/html/2607.12354#bib.bib93)\)to improve the expressiveness and transferability of the generated images, becoming the new SOTA for high\-resolution MIAs\. We adapt the original authors’ code to ensure compatibility with our codebase\. All attack hyperparameters are identical to those used by the original authors\.
- •Pseudo\-Private Data Guided Model Inversion Attack \(PPDG\)\(Penget al\.,[2024](https://arxiv.org/html/2607.12354#bib.bib9)\)\.The PPDG authors find that a fixed generative prior, StyleGAN\-2\(Karraset al\.,[2020](https://arxiv.org/html/2607.12354#bib.bib93)\), leads to a low probability of sampling actual private data during the inversion process due to the distributional gap to the private data distribution\. To address this limitation, they increase the density around high\-quality pseudo\-private data that exhibit characteristics of the private training data, and slightly tune the generator\. All attack hyperparameters are identical to those used by the original authors\.

PPA and IF\-GMI attacks on ResNet\-152 and ResNet\-18 models trained on FaceScrub and Stanford Dogs target all classes unless otherwise specified; for all models trained on CelebA, all PPDG attacks, and all DenseNet\-169 results, we instead target a seeded random subset of 75 of the classes, holding the randomly selected classes constant for each dataset across all baselines\. As attack time and compute scale roughly linearly with the number of classes targeted, this approach reduces time and compute by over 90% for our CelebA experiments and over 80% for the relevant FaceScrub experiments, at the cost of a small increase in uncertainty \(see the confidence interval margins of error in Tables[27](https://arxiv.org/html/2607.12354#A10.T27)and[28](https://arxiv.org/html/2607.12354#A10.T28)\)\.

As described in Section[2](https://arxiv.org/html/2607.12354#S2), we evaluate these attacks using the following standard metrics: Attack Acc@1 \(AttAcc@1\), Attack Acc@5 \(AttAcc@5\), L2\-FaceNet Distance \(L2\), and Average Evaluation Confidence \(EvalConf\)\.

### A\.6GPUs and carbon emissions

We run the majority of our target model training on 1x or 2x NVIDIA RTX A6000 GPUs with 48 GiB of vRAM each\. The main attack experiment results were generated primarily through renting virtual machines from Lambda Labs, typically using 1x or 2x NVIDIA A100 GPUs with 80 GiB vRAM, or 1x or 2x NVIDIA H100 GPUs with 80 GiB vRAM\. We estimate our total carbon emissions from GPU usage to be roughly 1200 kg ofCO2\\text\{CO\}\_\{2\}eq\., estimated using the Machine Learning Emissions Calculator\(Lacosteet al\.,[2019](https://arxiv.org/html/2607.12354#bib.bib32)\)\.

## Appendix BAT\-AT hyperparameters and training

Table 6:AT\-AT hyperparameters for the models reported in the main tables\. Columnsε\\varepsilon,Step Size, andLRare×\\times10−410^\{\-4\}\. All hyperparameters not mentioned are identical to their corresponding baseline models\. ’RN’ is ResNet, ’DN’ is DenseNet\.DatasetArch\.𝝀\\bm\{\\lambda\}\#Steps𝜸\\bm\{\\gamma\}𝜺\\bm\{\\varepsilon\}Step SizeLRFaceScrubRN\-1520\.0400\.040330\.40\.41\.51\.50\.50\.544RN\-180\.1250\.125550\.10\.1551111DN\-1690\.1000\.100330\.40\.4220\.70\.744CelebARN\-1520\.0200\.020330\.40\.4110\.340\.3455RN\-18†b0\.0400\.040440\.40\.4330\.750\.751010RN\-18†a0\.0100\.010550\.10\.1505010101010DN\-1690\.0200\.020330\.40\.4331155Stanford DogsRN\-1520\.0800\.080330\.10\.1552211

*Notes:*†aand†bare two distinct sets of hyperparameters for our AT\-AT ResNet\-18 model trained on CelebA, corresponding to the same symbols used in other tables\.

Table 7:Comparison of AT\-AT FaceScrub ResNet\-152 models at differentλ\\lambdavalues\. All other hyperparameters are the same as reported for ResNet\-152 FaceScrub in Table[6](https://arxiv.org/html/2607.12354#A2.T6)\. PPA is performed on a seeded random subset of 75 FaceScrub classes\. The 4 rightmost columns measure AutoAttack robust accuracy under variousϵ\\epsilonlevels\. Theλ=0\.04\\lambda=0\.04model is the same AT\-AT model reported in our main results; for this model, the PPA\-AttAcc@1 value \(marked by \*\) is on all 530 FaceScrub classes, as opposed to 75\.Defense𝝀\\bm\{\\lambda\}TestAcc↑\\uparrowPPA\-AttAcc@1↓\\downarrowϵ=0\.031↑\\epsilon\{=\}0\.031\{\\uparrow\}ϵ=0\.0025↑\\epsilon\{=\}0\.0025\{\\uparrow\}ϵ=0\.0005↑\\epsilon\{=\}0\.0005\{\\uparrow\}ϵ=10−5↑\\epsilon\{=\}10^\{\-5\}\{\\uparrow\}AT\-AT0\.010\.010\.9360\.9360\.0990\.0990000\.3070\.3070\.040\.040\.9390\.9390\.058\*0000\.1190\.1190\.060\.060\.9310\.9310\.0490\.0490000\.0770\.0770\.100\.100\.9210\.9210\.0640\.0640000\.1460\.1460\.150\.150\.8990\.8990\.0710\.0710000\.0360\.036

Our AT\-AT scheme, described in Section[5](https://arxiv.org/html/2607.12354#S5), has multiple tunable hyperparameters; see Table[6](https://arxiv.org/html/2607.12354#A2.T6)for details\.𝝀\\bm\{\\lambda\}governs the relative strength of the anti\-AT loss term\. For the in\-loop generation of adversarial examples, a Projected Gradient Descent \(PGD\) adversarial attack \(see e\.g\.Madryet al\.\([2017](https://arxiv.org/html/2607.12354#bib.bib246)\)\) is used;𝜺\\bm\{\\varepsilon\}is the size of theL∞L\_\{\\infty\}ball around the target image permissible for adversarial example generation,\#Stepsis the number of iterations used by the PGD attack, andStep Sizeis theL∞L\_\{\\infty\}distance the PGD attack moves in one iteration before clipping, which we set to be≳𝜺/\(\#Steps\)\\gtrsim\{\}\\bm\{\\varepsilon\}/\(\\textbf\{\\\#Steps\)\}\. We also tune the learning rate schedule; we report the initial learning rateLR\(same asα\\alphain the Section[5](https://arxiv.org/html/2607.12354#S5)algorithm\) and the decay factor𝜸\\bm\{\\gamma\}of the MultiStepLR scheduler\. Full implementation and hyperparameter details are available in our replication codes\.

In our preliminary experiments, we found that as AT\-AT models begin to rely on non\-robust features, training often becomes unstable, resulting in occasional downward spikes in model accuracy in the later epochs of training\. We also note that AT\-AT occasionally fails to converge when data augmentations are not applied\. To account for this instability, we perform a variant of early stopping: We save AT\-AT models during training at each epoch where they hit a certain pre\-set validation accuracy range \(e\.g\. over 90% for FaceScrub models\)\. Notably, all other defenses are also using the corresponding best model checkpoint from the 100 epochs of training, i\.e the model with the lowest validation loss\. We also note that during AT\-AT training, when model accuracy is unstable in later epochs, the reliance of the training model on non\-robust features remains reliably high\. See Table[7](https://arxiv.org/html/2607.12354#A2.T7)for an assessment of AT\-AT performance at five different values ofλ\\lambda, holding other hyperparameters constant\.

### B\.1Deferred discussion of AT\-AT practical runtimes

AT\-AT’s runtimes are roughly 30% faster than vanilla AT trained via PGD, because AT requires resistance to strong \(high\-ε\\varepsilon\) attacks, whereas AT\-AT seeks to learn from imperceptibly subtle \(low\-ε\\varepsilon\) features, requiring many fewer iterations of PGD\. Specifically, a typical research implementation of AT uses a default of1010steps of PGD, whereas we used just33to55steps for AT\-AT throughout our experiments\. These PGD step counts are the practical runtime bottleneck\. To be concrete, this translates to runtimes of3\.103\.10hours \(AT\) vs\.2\.142\.14hours \(AT\-AT\) for CelebA on ResNet\-18 architectures \(note that all algorithms train for exactly100100epochs throughout our experiments\)\. Importantly, like AT, our AT\-AT could potentially also benefit from the same well\-studied algorithmic and implementation speedups that are widely applied in industry\-scale AT implementations\(e\.g\. Wonget al\.,[2020](https://arxiv.org/html/2607.12354#bib.bib75); Shafahiet al\.,[2019](https://arxiv.org/html/2607.12354#bib.bib100)\)\. As such, AT\-AT can be implemented at scale at significantly lower cost than the AT implementations already in widespread industry use\.

## Appendix CMIAs: notation and related work

The objective of model inversion attacks in image classification models is to generate images semantically similar to the class identities in the training data, causing a leakage of privacy\. App\.[C\.1](https://arxiv.org/html/2607.12354#A3.SS1)sets notation used throughout the paper and describes some early MIAs\. App\.[C\.2](https://arxiv.org/html/2607.12354#A3.SS2)describes more recent generative MIAs\. App\.[C\.3](https://arxiv.org/html/2607.12354#A3.SS3)clarifies the scope of our paper and describes some related inversion attack settings\.

### C\.1Notation and early MIAs

Formally, MIAs consider a private training dataset𝒟p​r​i​v=\{xi,yi\}i=1N\\mathcal\{D\}\_\{priv\}=\\\{x\_\{i\},y\_\{i\}\\\}\_\{i=1\}^\{N\}, wherexi∈ℝdXx\_\{i\}\\in\\mathbb\{R\}^\{d\_\{X\}\},yi∈\{0,1\}Cy\_\{i\}\\in\\\{0,1\\\}^\{C\}, and each\(xi,yi\)\(x\_\{i\},y\_\{i\}\)are sampled i\.i\.d\. from a joint distributionℙX​Y\\mathbb\{P\}\_\{XY\}\. Early MIAs used SGD to directly optimize reconstructions of individual training examples\(Fredriksonet al\.,[2015](https://arxiv.org/html/2607.12354#bib.bib218)\)\. However, this approach suffers from various optimization complexities when applied to contemporary vision architectures like deep CNNs\. Therefore, recent state\-of\-the\-art attacks simplify optimization by adopting an ‘image prior’—that is, by restricting the search space of training example reconstructions to the image outputs of a domain\-relevant GAN, and conducting SGD\-based optimization on the GAN’s input noise vectors instead of directly on the high\-dimensional images’ feature space, as first introduced byZhanget al\.\([2020b](https://arxiv.org/html/2607.12354#bib.bib301)\)\. More specifically, GAN\-based MIAs optimize a latent noise vectorz^∈Z\\hat\{z\}\\in Zsuch that the resultant GAN\-generated image is classified as classccmaximized confidence by the target modelTT:

minz^⁡ℒ​\(T,G,D,z^,c\)\\min\_\{\\hat\{z\}\}\\mathcal\{L\}\(T,G,D,\\hat\{z\},c\)whereℒ\\mathcal\{L\}is a standard loss function \(e\.g\. cross\-entropy\) to maximize the confidence of the GAN\-generated image on classccgiven pretrained GAN generatorG:Z→Xp​r​i​o​rG:Z\\to X\_\{prior\}and GAN discriminatorDD\.

### C\.2Recent generative MIAs

In the following years after the introduction of generative MIAs byZhanget al\.\([2020b](https://arxiv.org/html/2607.12354#bib.bib301)\), low\-resolution generative MIAs saw numerous improvements \(see e\.g\.Chenet al\.\([2021](https://arxiv.org/html/2607.12354#bib.bib58)\); Wanget al\.\([2022](https://arxiv.org/html/2607.12354#bib.bib152)\); Yuanet al\.\([2023](https://arxiv.org/html/2607.12354#bib.bib34)\); Nguyenet al\.\([2023b](https://arxiv.org/html/2607.12354#bib.bib130)\)\)\. While these attacks show impressive performance in the low\-resolution scenario, they have yet to prove effective on higher\-resolution data\.

MIRROR\(Anet al\.,[2022](https://arxiv.org/html/2607.12354#bib.bib205)\)became the first attack to have success in the high\-resolution scenario\. PPA\(Struppeket al\.,[2022](https://arxiv.org/html/2607.12354#bib.bib36)\)made further improvements, proposing a framework that supports the use of a pre\-trained StyleGAN model\(Karraset al\.,[2020](https://arxiv.org/html/2607.12354#bib.bib93)\)unspecific to the target modelTT, which increases robustness to distributional shifts and flexibility in choice of target model while offering impressive performance on high\-resolution image data\. Most recently, IF\-GMI\(Qiuet al\.,[2024](https://arxiv.org/html/2607.12354#bib.bib300)\)leverages intermediate features of StyleGAN to improve the expressiveness and transferability of the generated images, becoming the new SOTA for high\-resolution MIAs\.

### C\.3Inversion attacks in other settings

While the mentioned MIAs are all white\-box attacks, there has been some exploration of black\-box and label\-only model inversion attacks\(Liuet al\.,[2024](https://arxiv.org/html/2607.12354#bib.bib151); Hanet al\.,[2023](https://arxiv.org/html/2607.12354#bib.bib1); Nguyenet al\.,[2023a](https://arxiv.org/html/2607.12354#bib.bib57); Anet al\.,[2022](https://arxiv.org/html/2607.12354#bib.bib205); Yanget al\.,[2019](https://arxiv.org/html/2607.12354#bib.bib42); Kahlaet al\.,[2022](https://arxiv.org/html/2607.12354#bib.bib56); Zhuet al\.,[2023](https://arxiv.org/html/2607.12354#bib.bib4); Qiuet al\.,[2025](https://arxiv.org/html/2607.12354#bib.bib286)\)and defenses\(Zhuanget al\.,[2025](https://arxiv.org/html/2607.12354#bib.bib5); Zhuet al\.,[2023](https://arxiv.org/html/2607.12354#bib.bib4)\)\. It should be noted that while it has previously been shown that some recent black\-box and label\-only attacks have outperformed previous white\-box attacks\(Zhuanget al\.,[2025](https://arxiv.org/html/2607.12354#bib.bib5); Hanet al\.,[2023](https://arxiv.org/html/2607.12354#bib.bib1); Nguyenet al\.,[2023a](https://arxiv.org/html/2607.12354#bib.bib57)\), these performance comparisons had been limited to earlier, low\-resolution MIAs\(Chenet al\.,[2021](https://arxiv.org/html/2607.12354#bib.bib58); Zhanget al\.,[2020b](https://arxiv.org/html/2607.12354#bib.bib301)\); more recent, high\-resolution white\-box MIAs\(Qiuet al\.,[2024](https://arxiv.org/html/2607.12354#bib.bib300); Struppeket al\.,[2022](https://arxiv.org/html/2607.12354#bib.bib36)\)are likely to outperform black\-box and label\-only MIAs, which have primarily been evaluated on low\-resolution data regimes\(Hanet al\.,[2023](https://arxiv.org/html/2607.12354#bib.bib1); Nguyenet al\.,[2023a](https://arxiv.org/html/2607.12354#bib.bib57)\)\. As such, for the purposes of this paper, we focus on the more difficult white\-box scenario\.

Separately, a related stream of research studies gradient inversion attacks and defenses in the federated learning scenario, where neural networks are collaboratively trained on a server\(Fanget al\.,[2023](https://arxiv.org/html/2607.12354#bib.bib7); Huanget al\.,[2021](https://arxiv.org/html/2607.12354#bib.bib6); Geipinget al\.,[2020](https://arxiv.org/html/2607.12354#bib.bib8)\)\. While both gradient inversion attacks and modern MIAs share the goal of leaking private training data, the threat models are fundamentally different; gradient inversion attacks often seek per\-image or per\-batch reconstructions based on gradient information provided by a local user during collaborative training, while modern MIAs instead seek class\-representative reconstructions based on queries to an already\-trained neural network\. As such, we leave the impact of information dependency and adversarial robustness on gradient inversion attacks as an important direction for future research\.

## Appendix DHSIC extended results

Table 8:HSIC\(X,Zj\)\(X,Z\_\{j\}\)for different defenses onFaceScrub\. Models:ResNet\-152andResNet\-18\. AttAcc@1 is from a PPA attack on all classes\.Arch\.DefenseTestAcc↑\\uparrowAttAcc@1↓\\downarrowHSIC\(X,Z0\)\(X,Z\_\{0\}\)HSIC\(X,Z1\)\(X,Z\_\{1\}\)HSIC\(X,Z2\)\(X,Z\_\{2\}\)HSIC\(X,Z3\)\(X,Z\_\{3\}\)ResNet\-152NoDef0\.9580\.9580\.8820\.88271\.6671\.6672\.3272\.3272\.7272\.7272\.7272\.72MID0\.9500\.9500\.3910\.39171\.5271\.5272\.5772\.5772\.8472\.8461\.9861\.98BiDO0\.9280\.9280\.7800\.78073\.6873\.6873\.6873\.6873\.6873\.6873\.6373\.63TL\-DMI0\.9110\.9110\.1900\.19071\.7371\.7372\.5272\.5272\.9372\.9372\.5872\.58NegLS0\.9060\.9060\.1600\.16071\.6471\.6472\.3372\.3372\.2972\.2973\.6373\.63RoLSS0\.8860\.8860\.4740\.47472\.0972\.0972\.3072\.3029\.6929\.6961\.9261\.92RoLSS\-SSF0\.8690\.8690\.3430\.34371\.8871\.8872\.0272\.0232\.7532\.7564\.9464\.94Trap\-MID\*0\.9490\.9490\.5870\.58769\.9369\.9371\.9671\.9672\.5672\.5672\.6572\.65Trap\-MID0\.9260\.9260\.4010\.40169\.8069\.8071\.8871\.8872\.6272\.6269\.2869\.28ResNet\-18NoDef0\.9500\.9500\.9010\.90163\.4863\.4849\.7449\.7437\.8037\.8072\.3872\.38MID0\.9240\.9240\.7140\.71461\.0361\.0348\.3948\.3930\.0830\.0865\.7365\.73BiDO0\.8840\.8840\.5300\.53073\.8073\.8073\.8173\.8173\.8073\.8073\.7573\.75TL\-DMI0\.9060\.9060\.2400\.24061\.1161\.1150\.2950\.2936\.3136\.3172\.6472\.64NegLS0\.9290\.9290\.6490\.64963\.2763\.2750\.7650\.7634\.9734\.9773\.6873\.68RoLSS0\.9500\.9500\.8580\.85861\.2361\.2349\.3849\.3813\.3313\.3372\.2572\.25RoLSS\-SSF0\.9500\.9500\.8490\.84961\.4461\.4449\.3849\.3813\.3013\.3072\.2772\.27Trap\-MID\*0\.9270\.9270\.5020\.50257\.8457\.8447\.8047\.8034\.9334\.9372\.0172\.01Trap\-MID0\.9280\.9280\.7350\.73562\.4462\.4447\.3347\.3334\.9134\.9167\.4467\.44

Table[8](https://arxiv.org/html/2607.12354#A4.T8)is an extended version of Table[1](https://arxiv.org/html/2607.12354#S3.T1)which includes ResNet\-18 models, as well as the extra Trap\-MID\* baseline\. The results are similarly uncorrelated; some defenses, like the ResNet\-18 RoLSS\-SSF model, have some low HSIC values but perform poorly against a PPA attack, while other defenses, like ResNet\-152 NegLS, have similarly high HSIC values to an undefended model, but perform well against a PPA attack\. It should be noted that BiDO utilizes an HSIC*estimator*and that in some problems optimized HSIC tests outperform optimizing the statistic\(Xuet al\.,[2025](https://arxiv.org/html/2607.12354#bib.bib3)\)\.

App\.[D\.1](https://arxiv.org/html/2607.12354#A4.SS1)revisits BiDO hyperparameter tuning, exploring the possibility of creating a model with low HSIC values with the BiDO scheme\.

### D\.1BiDO experiments

LetXXandYYbe random variables representing the inputs and outputs, respectively, and letZ1,…,ZmZ\_\{1\},\\ldots,Z\_\{m\}be intermediate embeddings chosen atmmdifferent points from passingXXas input to the model \(e\.g\. the outputs of the four major blocks of a ResNet\)\. Motivated by theoretical concepts in statistical dependency\(Grettonet al\.,[2005](https://arxiv.org/html/2607.12354#bib.bib50)\), the popular BiDO defensePenget al\.\([2022](https://arxiv.org/html/2607.12354#bib.bib303)\)uses the following training loss function:

ℒ~​\(θ\)=ℒ​\(θ\)\+λx​∑j=1mHSIC​\(X,Zj\)−λy​∑j=1mHSIC​\(Zj,Y\)\\tilde\{\\mathcal\{L\}\}\(\\theta\)=\\mathcal\{L\}\(\\theta\)\+\\lambda\_\{x\}\\sum\_\{j=1\}^\{m\}\\text\{HSIC\}\(X,Z\_\{j\}\)\-\\lambda\_\{y\}\\sum\_\{j=1\}^\{m\}\\text\{HSIC\}\(Z\_\{j\},Y\)whereℒ\\mathcal\{L\}is a standard cross\-entropy loss andλx,λy\>0\\lambda\_\{x\},\\lambda\_\{y\}\>0are hyper\-parameters\. Theλx\\lambda\_\{x\}term is intended to minimize dependency between inputs and intermediate embeddings\. As for theλy\\lambda\_\{y\}term,Penget al\.\([2022](https://arxiv.org/html/2607.12354#bib.bib303)\)argue that solely minimizingHSIC​\(X,Zj\)\\text\{HSIC\}\(X,Z\_\{j\}\)would “…lead to the loss of useful information, so it is necessary to keep the \[λy\\lambda\_\{y\}\] term to make sureZjZ\_\{j\}is informative enough ofYYas well as to maintain the discriminative nature of the classifier\.”

After the surprising discovery that BiDO with\(λx,λy\)=\(0\.05,0\.5\)\(\\lambda\_\{x\},\\lambda\_\{y\}\)=\(0\.05,0\.5\)fails to decrease HSIC\(X,Zj\)\(X,Z\_\{j\}\)\(see Table[8](https://arxiv.org/html/2607.12354#A4.T8)\), we experimented with other pairs\(λx,λy\)\(\\lambda\_\{x\},\\lambda\_\{y\}\)to see if HSIC\(X,Zj\)\(X,Z\_\{j\}\)\(referred to hereafter as ‘HSIC’\) could indeed be lowered through the BiDO training scheme\. To check whether theλy\\lambda\_\{y\}term of the objective function was impeding the model’s ability to reduce HSIC, we first experimented with settingλy=0\\lambda\_\{y\}=0; however, according to the ablation study ofPenget al\.\([2022](https://arxiv.org/html/2607.12354#bib.bib303)\), setting\(λx,λy\)=\(0\.1,0\)\(\\lambda\_\{x\},\\lambda\_\{y\}\)=\(0\.1,0\)completely impedes the model’s ability to train\. We instead surprisingly find success with smaller values ofλx\\lambda\_\{x\}; more specifically, we train ResNet\-18 and ResNet\-152 BiDO models on FaceScrub with\(λx,λy\)=\(0\.001,0\)\(\\lambda\_\{x\},\\lambda\_\{y\}\)=\(0\.001,0\)and\(λx,λy\)=\(0\.0001,0\)\(\\lambda\_\{x\},\\lambda\_\{y\}\)=\(0\.0001,0\)\. We use initial learning rates of0\.0010\.001and0\.00010\.0001for the ResNet\-152 and ResNet\-18 models, respectively\. For the ResNet\-18 models, we train for just 75 epochs, as this was sufficient for them to converge; for the ResNet\-152 models, we train for 100 epochs\. All other training hyperparameters are identical to those described in App\.[A\.3](https://arxiv.org/html/2607.12354#A1.SS3)\. As a natural next step, we run a PPA attack on our new ‘low\-HSIC’ BiDO models\. Due to compute considerations, we attack a seeded random subset of 75 out of the 530 FaceScrub identities; all other attack hyperparameters are identical to those described in App\.[A\.5](https://arxiv.org/html/2607.12354#A1.SS5)\.

Results\.As Table[9](https://arxiv.org/html/2607.12354#A4.T9)shows, across both architectures, BiDO\(0\.001,0\)\(0\.001,0\)and BiDO\(0\.0001,0\)\(0\.0001,0\)generally succeed in lowering HSIC\(X,Zj\)\(X,Z\_\{j\}\)for most values ofjjwhile losing only 0\.1 to 1\.8 percentage points in test accuracy\. However, these ‘low\-HSIC’ BiDO models fail to provide a meaningful defense against model inversion, with top\-1 attack accuracies consistently exceeding 80%\. In sum, Table[9](https://arxiv.org/html/2607.12354#A4.T9)reinforces the notion that the HSIC dependency measure used in BiDO, while theoretically well\-motivated, is orthogonal to model inversion robustness\.

Table 9:HSIC\(X,Zj\)\(X,Z\_\{j\}\)for different BiDO hyperparameter variations, with comparisons to NoDef, MID, and TL\-DMI\. ResNet\-18 and ResNet\-152 models trained on FaceScrub\. AttAcc@1 is from PPA on all 530 classes, except for the four BiDO models withλy=0\\lambda\_\{y\}=0, for which PPA is run on a seeded random subset of 75 classes out of 530\. The ResNet\-152 BiDO\(\.001,0\) model is also reported in Table[1](https://arxiv.org/html/2607.12354#S3.T1)as BiDO\*\*\.Arch\.DefenseTestAcc↑\\uparrowAttAcc@1↓\\downarrow𝐇𝐒𝐈𝐂​\(𝐗,𝐙𝟎\)\\mathbf\{HSIC\(X,Z\_\{0\}\)\}𝐇𝐒𝐈𝐂​\(𝐗,𝐙𝟏\)\\mathbf\{HSIC\(X,Z\_\{1\}\)\}𝐇𝐒𝐈𝐂​\(𝐗,𝐙𝟐\)\\mathbf\{HSIC\(X,Z\_\{2\}\)\}𝐇𝐒𝐈𝐂​\(𝐗,𝐙𝟑\)\\mathbf\{HSIC\(X,Z\_\{3\}\)\}RN\-152NoDef0\.9580\.9580\.8820\.88271\.6671\.6672\.3272\.3272\.7272\.7272\.7272\.72MID0\.9500\.9500\.3910\.39171\.5271\.5272\.5772\.5772\.8472\.8461\.9861\.98BiDO\(\.05,\.5\)0\.9280\.9280\.7800\.78073\.6873\.6873\.6873\.6873\.6873\.6873\.6373\.63BiDO\(\.001,0\)0\.9400\.9400\.8150\.8152\.1592\.1591\.2561\.2569\.3109\.31071\.8171\.81BiDO\(\.0001,0\)0\.9480\.9480\.8490\.84970\.6270\.6272\.1572\.1572\.7172\.7172\.6872\.68TL\-DMI0\.9110\.9110\.1900\.19071\.7371\.7372\.5272\.5272\.9372\.9372\.5872\.58RN\-18NoDef0\.9500\.9500\.9010\.90163\.4863\.4849\.7449\.7437\.8037\.8072\.3872\.38MID0\.9240\.9240\.7140\.71461\.0361\.0348\.3948\.3930\.0830\.0865\.7365\.73BiDO\(\.05,\.5\)0\.8840\.8840\.5300\.53073\.8073\.8073\.8173\.8173\.8073\.8073\.7573\.75BiDO\(\.001,0\)0\.9430\.9430\.8170\.8174\.4034\.4031\.4021\.4021\.5111\.51172\.3472\.34BiDO\(\.0001,0\)0\.9490\.9490\.8340\.83454\.5154\.5141\.0841\.0824\.8324\.8372\.3772\.37TL\-DMI0\.9060\.9060\.2400\.24061\.1161\.1150\.2950\.2936\.3136\.3172\.6472\.64

## Appendix E97% Unseen\-Pixel Model Experiment Details

Table 10:Unseen\-Pixels trained on FaceScrub obtains very limited defense against reconstruction by IF\-GMI\.Arch\.Defense%Pix\-deletedTestAcc↑\\uparrowAttAcc@1↓\\downarrowAttAcc@5↓\\downarrowL2\-Face↑\\uparrowEvalConf↓\\downarrowRN\-152NoDef00\.9580\.9580\.9870\.9870\.9980\.9980\.7000\.7000\.9840\.984NoDef\-EarlyStop00\.9170\.9170\.8860\.8860\.9810\.9810\.7720\.7720\.8750\.875TL\-DMI00\.9110\.9110\.4200\.4200\.6940\.6940\.9460\.9460\.4010\.401Unseen\-Pixels97\.80\.9100\.9100\.8600\.8600\.9660\.9660\.7700\.7700\.8480\.848RN\-18NoDef00\.9500\.9500\.9830\.9830\.9960\.9960\.7400\.7400\.9790\.979NoDef\-EarlyStop00\.8900\.8900\.8970\.8970\.9800\.9800\.7570\.7570\.8810\.881TL\-DMI00\.9060\.9060\.5150\.5150\.7600\.7600\.9350\.9350\.4920\.492Unseen\-Pixels97\.20\.8920\.8920\.7970\.7970\.9350\.9350\.8210\.8210\.7800\.780

This section provides further details for our Unseen\-Pixels experiment in section[3](https://arxiv.org/html/2607.12354#S3)\. Appendix[E\.1](https://arxiv.org/html/2607.12354#A5.SS1)describes related work in feature selection for neural networks; Appendix[E\.2](https://arxiv.org/html/2607.12354#A5.SS2)then describes our method for creating a mask that deletes over 97% of pixels from the training data before training the model, while losing only a few percentage points in accuracy\. Appendix[E\.3](https://arxiv.org/html/2607.12354#A5.SS3)describes the relationship between recent theoretical guarantess for data reconstruction attacks and our Unseen\-Pixels method; Table[10](https://arxiv.org/html/2607.12354#A5.T10)presents additional Unseen\-Pixels results against an IF\-GMI attack\.

### E\.1Background and related work on feature selection in shallow neural networks

Feature selection within the training of shallow fully\-connected neural networks is well\-studied, particularly with techniques derived from the popular Lasso regularization\(Tibshirani,[1996](https://arxiv.org/html/2607.12354#bib.bib29)\)on linear models\. In particular, LassoNet\(Lemhadriet al\.,[2021](https://arxiv.org/html/2607.12354#bib.bib55)\)proposes to insert a skip connection directly from the inputs to the output in a feedforward neural network, then imposes a Lasso penalty on the skip connection weights to the loss function during training, such that some of these weights would be zeroed out\. The features would then be weighted in the main network corresponding to their weight in the skip connection, which created the desired feature selection\. A parallel approach has been to apply a Group Lasso \(orL2,1L\_\{2,1\}norm\) penalty directly to the weights𝑾\\bm\{W\}of the initial fully connected layer\(Zhanget al\.,[2020a](https://arxiv.org/html/2607.12354#bib.bib74); Scardapaneet al\.,[2017](https://arxiv.org/html/2607.12354#bib.bib67); Zhaoet al\.,[2015](https://arxiv.org/html/2607.12354#bib.bib66)\)\. Let𝒘i\\bm\{w\}\_\{i\}be theii\-th column vector of𝑾\\bm\{W\}; the Group Lasso penalty is thenϕ​\(𝑾\)=∑‖𝒘i‖2\\phi\(\\bm\{W\}\)=\\sum\|\|\\bm\{w\}\_\{i\}\|\|\_\{2\}\. Adding this penalty to the loss function then allows for the zeroing out of unhelpful groups of weights in this initial matrix; each of these groups of weights correspond with an input feature\. Building on this idea, the “GroupLasso\+AdaptiveGroupLasso” \(GL\+AGL\) technique\(Dinh and Ho,[2021](https://arxiv.org/html/2607.12354#bib.bib107)\)first calculates a Group Lasso\-regularized version of the weights𝑾\\bm\{W\}, then uses this as a “rough data\-dependent estimate to shrink groups of parameters with different regularization strengths”, creating a more aggressive and consistent feature selector\(Dinh and Ho,[2020](https://arxiv.org/html/2607.12354#bib.bib297)\)\. However, all of the above approaches rely on the existence of a set of weights in the neural network that correspond to each feature; in the deep CNNs that are the focus of our paper, such weights do not exist, and so these approaches cannot be readily applied\.

As an alternative approach,Liet al\.\([2016](https://arxiv.org/html/2607.12354#bib.bib88)\)propose prepending sparse one\-to\-one layer𝑫\\bm\{D\}, of the same dimensions as a member of the training data, to the model; the forward pass is done through elementwise multiplication\. During training, an Elastic\-Net\(Zou and Hastie,[2005](https://arxiv.org/html/2607.12354#bib.bib28)\)penalty is applied to the weights𝑫\\bm\{D\}; features are removed when their corresponding weight in𝑫\\bm\{D\}vanishes\. Other similar approaches include*CancelOut*\(Borisovet al\.,[2019](https://arxiv.org/html/2607.12354#bib.bib90)\)and*Feature\-Aware Drop Layer*\(Jiménez\-Navarroet al\.,[2024](https://arxiv.org/html/2607.12354#bib.bib77)\)\. While the above approaches are all compatible with convolutional networks, none of their empirical analyses explores the hi\-res image classification scenario with deep convolutional networks that is the subject of our paper\.

### E\.2A general method for feature selection in deep CNNs

To achieve our goal of creating a mask that selects only a small subset of pixels from high\-resolution training data images without compromising on model utility, we use a combination of many of the above approaches\. We prepend a one\-to\-onemask layerto the front of the CNN and perform a variant of GL\+AGL regularization\(Dinh and Ho,[2021](https://arxiv.org/html/2607.12354#bib.bib107)\)on this layer\. More specifically, let the mask layer weights be a matrixDDof same shape as the input images \(in our experimental setting, shape3×224×2243\\times 224\\times 224\)\. Then, our new modelTm​a​s​kT\_\{mask\}is simply the normal modelTTafter an elementwise multiplication withDD:Tm​a​s​k​\(X\)=T​\(X⊙D\)T\_\{mask\}\(X\)=T\(X\\odot\{\}D\)\.

All of our datasets are RGB images, and as such, each weight ofDDcorresponds to*one RGB channel*of a pixel, as opposed to a whole pixel\. LetDi​jcD^\{c\}\_\{ij\}represent the channel for colorccfor pixel at position\(i,j\)\(i,j\)in the image\. For visual ease, we seek to select*pixels*, not channels of pixels; to do so, we will apply a variant of GL\+AGL on the group of three channels for each pixel\. In our experimental settings, this creates groups⟨Di​jR,Di​jG,Di​jB⟩\\langle\{\}D^\{R\}\_\{ij\},D^\{G\}\_\{ij\},D^\{B\}\_\{ij\}\\ranglefor each pair\(i,j\)\(i,j\)where0≤i,j≤2230\\leq\{\}i,j\\leq 223, for a total of224×224=50176224\\times 224=50176groups\. LetDN​o​r​mD^\{Norm\}be a224×224224\\times 224matrix whereDi​jN​o​r​m=‖⟨Di​jR,Di​jG,Di​jB⟩‖2D^\{Norm\}\_\{ij\}=\|\|\\langle\{\}D^\{R\}\_\{ij\},D^\{G\}\_\{ij\},D^\{B\}\_\{ij\}\\rangle\|\|\_\{2\}\.

To create the mask we use to delete training data pixels prior to training our model, we proceed in two training stages: \(1\) Ridge base estimator creation and \(2\) GL\+AGL feature selection\. In Stage 1, to generate the base estimator needed for GL\+AGL, we apply a Ridge regression to the grouped weights ofDD:

ℒ~​\(θ\)=ℒ​\(θ\)\+λ1⋅‖DN​o​r​m‖2\.\\tilde\{\\mathcal\{L\}\}\(\\theta\)=\\mathcal\{L\}\(\\theta\)\+\\lambda\_\{1\}\\cdot\|\|D^\{Norm\}\|\|\_\{2\}\.We train in Stage 1 for 100 epochs and save the resulting mask layerD^\\hat\{D\}as a base estimate for Stage 2, discarding the rest of the trained model\. For Stage 2, we apply the penalty formula given inDinh and Ho \([2020](https://arxiv.org/html/2607.12354#bib.bib297)\)with our ridge base estimator:

ℒ~​\(θ\)=ℒ​\(θ\)\+λ2⋅ℳ​\(D\),\\tilde\{\\mathcal\{L\}\}\(\\theta\)=\\mathcal\{L\}\(\\theta\)\+\\lambda\_\{2\}\\cdot\{\}\\mathcal\{M\}\(D\),where

ℳ​\(D\)=∑i224∑j224Di​jN​o​r​m\(D^i​jN​o​r​m\)ρ\.\\mathcal\{M\}\(D\)=\\sum\_\{i\}^\{224\}\\sum\_\{j\}^\{224\}\\frac\{D^\{Norm\}\_\{ij\}\}\{\(\\hat\{D\}^\{Norm\}\_\{ij\}\)^\{\\rho\}\}\.During Stage 2, after each gradient update, we apply a thresholdtttoDDsuch that ifDi​jN​o​r​m≤tD^\{Norm\}\_\{ij\}\\leq\{\}t, then⟨Di​jR,Di​jG,Di​jB⟩\\langle\{\}D^\{R\}\_\{ij\},D^\{G\}\_\{ij\},D^\{B\}\_\{ij\}\\rangleis set to𝟎\\bm\{0\}withinDD, dropping the pixel at position\(i,j\)\(i,j\)from model training\. We begin witht=1​e\-​6t=1\\text\{e\-\}6, and after epoch 10, we sett=0\.0025t=0\.0025\.

We train in Stage 2 for 100 epochs and save the resulting trained maskD∗D^\{\*\}, discarding the rest of the trained model\.D∗D^\{\*\}is then altered such that any nonzero element is set to11, so that elementwise multiplication byD∗D^\{\*\}has no additional effect besides dropping the pixels it has learned to mask\. This completes the creation of the maskD∗D^\{\*\}\.

Finally, the true target model is trained with the maskD∗D^\{\*\}applied\. The result of this ‘true’ training run is the ‘Unseen\-Pixels’ model reported in Table[3](https://arxiv.org/html/2607.12354#S3.T3)\.D∗D^\{\*\}is applied as a pre\-augmentation data transform, ensuring that no data augmentations allow otherwise\-deleted pixels to influence model training\. This training run is done with identical hyperparameters and data augmentations to those described for target models in App\.[A](https://arxiv.org/html/2607.12354#A1)\.

In Figure[1](https://arxiv.org/html/2607.12354#S3.F1), the mask applied isD∗D^\{\*\}for our ResNet\-152 Unseen\-Pixels model\. The reconstructions in the third column are the result of the same PPA attack on this ResNet\-152 Unseen\-Pixels model reported in Table[3](https://arxiv.org/html/2607.12354#S3.T3)\.

λ1\\lambda\_\{1\}controls the relative strength of the ridge penalty in Stage 1, andλ2\\lambda\_\{2\}controls the relative strength of the AGL penalty in Stage 2\.ρ\\rhoregulates the influence of the base estimator during Stage 2\. For both ResNet\-18 and ResNet\-152, we setλ1=0\.01\\lambda\_\{1\}=0\.01,λ2=0\.1\\lambda\_\{2\}=0\.1, andρ=1\\rho=1\. During Stages 1 and 2, we apply only brightness, saturation, and hue augmentations; random resized cropping, contrast augmentations, and random horizontal flipping are removed since their inclusion would allow otherwise\-deleted pixels to influence mask training\. All unmentioned training hyperparameters during Stage 1 and Stage 2 are identical to those described in App\.[A](https://arxiv.org/html/2607.12354#A1); full materials for reproduction of results are given in our replication codes\.

This method of feature selection is primarily intended as a proof\-of\-concept to test the effectiveness of MIAs on a model trained on data with large amounts of hidden pixels\. The creation of the maskD∗D^\{\*\}involved many design choices with how to incorporate ridge vs\. lasso penalties, whether to apply Group Lasso on its own or GL\+AGL, and so on; we do not claim that our approach is the optimal one within this family of design choices, or that our hyperparameter choices forλ1\\lambda\_\{1\},λ2\\lambda\_\{2\},tt, andρ\\rhoare optimal\. We leave this analysis as an interesting direction for future work in feature selection for deep CNNs\.

Nevertheless, our approach succeeds in the creation of masks that allows us to delete over 97% of the training pixels in each image, such that they are never seen during training, while losing only a few percentage points in accuracy\. The fact that this unseen data is still devastatingly reconstructed by MIAs calls into question the intuitive role of training\-data\-to\-model dependencies in MIAs\.

### E\.3Unseen\-Pixels and recent theoretical privacy results

In this section, we describe recent theoretical guarantees for data reconstruction attacks that lower\-bound the variance of unbiased estimators of training data, and we describe why no such estimator exists for the hidden pixels of our Unseen\-Pixels models, suggesting that guarantees based on unbiased reconstruction estimators are overly optimistic for the MIA setting\.

The recent works ofHannunet al\.\([2021](https://arxiv.org/html/2607.12354#bib.bib51)\)andChaudhuriet al\.\([2024](https://arxiv.org/html/2607.12354#bib.bib113)\)seek to provide theoretical guarantees on training data privacy by framing the privacy leakage issue as a statistical parameter estimation problem, as follows: Letz=\[x1⊤,y1,…,xN⊤,yN\]∈ℝN⋅\(dX\+1\)\\textbf\{z\}=\[x\_\{1\}^\{\\top\},y\_\{1\},\\ldots,x\_\{N\}^\{\\top\},y\_\{N\}\]\\in\\mathbb\{R\}^\{N\\cdot\(d\_\{X\}\+1\)\}be formed by concatenating all of the examples in the training dataset𝒟p​r​i​v\\mathcal\{D\}\_\{priv\}\. Given a randomized learning algorithm𝒜​\(z\)\\mathcal\{A\}\(\\textbf\{z\}\)that outputs a model in hypothesis spaceℋ\\mathcal\{H\}after training on data𝒟p​r​i​v\\mathcal\{D\}\_\{priv\}, we consider the probability distribution of possible modelshh\(or set of model outputsh​\(x\)h\(x\)\) to be parameterized by the dataset vectorz\. The adversary then seeks to create an estimatorz^\\hat\{\\textbf\{z\}\}based on the observationhh\(or observationsh​\(x\)h\(x\)\); a successful estimator would leak information about the dataset vectorz, creating a breach of privacy\. To create theoretical privacy guarantees, both papers provide methods to lower\-bound the variance of any unbiased estimatorz^\\hat\{\\textbf\{z\}\};Hannunet al\.\([2021](https://arxiv.org/html/2607.12354#bib.bib51)\)introduce the concept using Fisher Information Loss and the Cramér\-Rao bound, whileChaudhuriet al\.\([2024](https://arxiv.org/html/2607.12354#bib.bib113)\)use Hammersley\-Chapman\-Robbins bounds, which are tractable to compute for deeper convolutional networks\.

In this statistical framing, the hidden pixels of our Unseen\-Pixels models have no effect on the model or model outputs, and so cannot be inferred by an unbiased estimator\. This is the strongest version of this privacy guarantee from the above papers; instead of a lower bound on variance, it is simply the case that there is*no*unbiased estimator of the hidden pixels \(or, equivalently, unbiased reconstruction estimators of the hidden pixels have unbounded variance\)\. However, as Fig\.[1](https://arxiv.org/html/2607.12354#S3.F1)shows, in spite of this strong privacy guarantee on over 97% of pixels in the training data images, MIAs are still able to cause significant privacy leakage\.

This result suggests that privacy guarantees related to unbiased estimatorsz^\\hat\{\\textbf\{z\}\}may not provide adequate privacy protection in the context of MIAs, perhaps unless they can be applied to*all*pixels of the input images\. We hypothesize this is because of the*strong prior*that contemporary MIAs use in the form of a pre\-trained facial GAN\. As mentioned inChaudhuriet al\.\([2024](https://arxiv.org/html/2607.12354#bib.bib113)\), the requirement thatz^\\hat\{\\textbf\{z\}\}is unbiased is “tantamount to forbidding the use of extra, outside information such as a Bayesian prior”, and that unbiasedness is a “reasonable yet significant restriction”\. Our experiment underscores the significance of this restriction, since the GAN prior of PPA is powerful enough to create faithful representations of faces of which less than 3% of the pixels have been seen by the target model\.

We emphasize that our discussion is limited to the usefulness of privacy guarantees given by unbiased reconstruction estimators in the MIA scenario\. The output of an MIA is not intended to be a pixel\-by\-pixel reconstruction of each individual image, but instead a class\-representative image that captures the likeness of a certain private identity\. It is likely that guarantees on unbiased reconstruction estimators have more success against other forms of data privacy attacks that do indeed seek exact reconstructions of individual data points, such as training data extraction from generative models\(Carliniet al\.,[2023](https://arxiv.org/html/2607.12354#bib.bib76)\)\.

On a separate note, biased estimators are also discussed in similar work\(Hannunet al\.,[2021](https://arxiv.org/html/2607.12354#bib.bib51); Guoet al\.,[2022](https://arxiv.org/html/2607.12354#bib.bib344)\); strong guarantees on biased estimators may be more applicable in the MIA experimental setting, as they can potentially account for the presence of strong prior information\(Chaudhuriet al\.,[2024](https://arxiv.org/html/2607.12354#bib.bib113)\)\. We consider further exploration of the practical usefulness of privacy guarantees on biased reconstruction estimators to be an interesting area for future research\.

Regardless, our results caution against the idea that a strong guarantee against unbiased reconstruction estimators yields universal confidentiality protections\.

## Appendix FAdversarial robustness: evaluation details and connections to privacy

Original Imageϵ\\epsilon=11ϵ\\epsilon=0\.50\.5ϵ\\epsilon=0\.0310\.031ϵ\\epsilon=0\.0250\.025ϵ\\epsilon=0\.00250\.0025ϵ\\epsilon=0\.00050\.0005ϵ\\epsilon=0\.000010\.00001![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/adversarial_examples/vayqyjba_original_image_0.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/adversarial_examples/rjl90za1_adversarial_image_epsilon_1_0.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/adversarial_examples/rjl90za1_adversarial_image_epsilon_0.5_0.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/adversarial_examples/rjl90za1_adversarial_image_epsilon_0.031_0.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/adversarial_examples/rjl90za1_adversarial_image_epsilon_0.025_0.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/adversarial_examples/rjl90za1_adversarial_image_epsilon_0.0025_0.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/adversarial_examples/rjl90za1_adversarial_image_epsilon_0.0005_0.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/adversarial_examples/rjl90za1_adversarial_image_epsilon_1e-05_0.png)

Figure 5:Adversarial examples \(PGD attack\) for the different perturbation magnitudesϵ\\epsilon\.### F\.1Adversarial examples: additional background

Adversarial examples\(Szegedyet al\.,[2014](https://arxiv.org/html/2607.12354#bib.bib60)\)are small noise perturbations that when added to an image in the dataset, changes the prediction of the model\. The noise is restricted to be in a small range\|\|δ\|\|p<ϵ\\lvert\\lvert\\delta\\rvert\\rvert\_\{p\}<\\epsilon, whereδ\\deltais the perturbation,ppis the norm, andϵ\\epsilonis the limit of the size of perturbations\. Since the perturbations are limited to very small changes, they are mostly imperceptible for humans and don’t carry semantic meaning\. Figure[5](https://arxiv.org/html/2607.12354#A6.F5)shows adversarial examples generated from a Project Gradient Descent \(PGD\) attack\(Madryet al\.,[2017](https://arxiv.org/html/2607.12354#bib.bib246)\)at variousL∞L\_\{\\infty\}perturbation magnitudesϵ\\epsilon\.

The existence of adversarial examples was first discovered inSzegedyet al\.\([2014](https://arxiv.org/html/2607.12354#bib.bib60)\)\. Algorithms to generate them\(Croce and Hein,[2020b](https://arxiv.org/html/2607.12354#bib.bib27)\)and defend against them\(Madryet al\.,[2017](https://arxiv.org/html/2607.12354#bib.bib246); Shafahiet al\.,[2019](https://arxiv.org/html/2607.12354#bib.bib100); Wonget al\.,[2020](https://arxiv.org/html/2607.12354#bib.bib75)\)have since been heavily studied\.

### F\.2Adversarial examples and privacy: related work

Our work is inspired by many previous studies on adversarial robustness \(or non\-robustness\) and its connection to privacy\.Tursynbeket al\.\([2021](https://arxiv.org/html/2607.12354#bib.bib126)\)andBoenischet al\.\([2021](https://arxiv.org/html/2607.12354#bib.bib125)\)show how different aspects of differentially private training\(Abadiet al\.,[2016](https://arxiv.org/html/2607.12354#bib.bib351)\)can make models vulnerable to adversarial attacks, whileZhang and Bu \([2023](https://arxiv.org/html/2607.12354#bib.bib124)\)provide further analysis showing that differentially private models can indeed also be adversarially robust\.Songet al\.\([2019](https://arxiv.org/html/2607.12354#bib.bib238)\)show empirically that adversarially robust deep learning models are often vulnerable to membership inference attacks, exploring related factors such as model capacity and robustness generalization\.Hayes \([2020](https://arxiv.org/html/2607.12354#bib.bib127)\)explores this relationship further, describing the effect of the size of the adversarial perturbation used, as well as the size of the training set\.Khaledet al\.\([2022](https://arxiv.org/html/2607.12354#bib.bib128)\)show that model extraction attacks are often more effective on adversarially trained models\. We also consider our work related to the recent stream of evidence that more interpretable models are inherently less privacy\-preserving\(Narettoet al\.,[2022](https://arxiv.org/html/2607.12354#bib.bib2); Shokriet al\.,[2021](https://arxiv.org/html/2607.12354#bib.bib139)\)\.

Recent works also focus on the connection of adversarial examples to model inversion specifically\.Mejiaet al\.\([2019](https://arxiv.org/html/2607.12354#bib.bib25)\)show that adversarially trained models allow for the success of previously\-intractable low\-resolution model inversion attacks, arguing that the more semantically meaningful feature space that AT creates\(Engstromet al\.,[2019](https://arxiv.org/html/2607.12354#bib.bib223); Madryet al\.,[2017](https://arxiv.org/html/2607.12354#bib.bib246)\)enhances the performance of MIAs\. Separately, a recent study on black\-box MIAs\(Zhouet al\.,[2024](https://arxiv.org/html/2607.12354#bib.bib243)\)incorporates adversarial examples into training data in order to increase the diversity of semantically meaningful information available to the adversary\.

Wenet al\.\([2021](https://arxiv.org/html/2607.12354#bib.bib84)\)proposes to defend against black\-box MIAs by adding carefully\-chosen adversarial noise to the output of the target model, offering impressive performance\. While this defense does not apply in the white\-box scenario, as adversaries would have access to the output before the adversarial noise is applied, we nonetheless consider this work complementary to our results involving the effectiveness ofAT\-AT\.

Our work is particularly inspired by the ideas presented inMejiaet al\.\([2019](https://arxiv.org/html/2607.12354#bib.bib25)\); we expand upon this work by 1\) considering newer, high\-resolution MIAs; 2\) showing that recent generic MIA defenses obtain their performance through an unintended reliance on non\-robust features; and 3\) introducing a novel algorithm,AT\-AT, that allows for a direct tradeoff between model inversion defense and adversarial robustness\.

### F\.3Adversarial robustness evaluation: deferred experimental details

As explained byCarliniet al\.\([2019](https://arxiv.org/html/2607.12354#bib.bib40)\), adversarial robustness \(and, therefore, adversarial vulnerability\) is often improperly quantified\. To mitigate this issue,Carliniet al\.\([2019](https://arxiv.org/html/2607.12354#bib.bib40)\)provide a checklist of actions to take to ensure proper evaluation of adversarial robustness, including performing adaptive attacks, applying a diverse set of attacks, and verifying that attacks have converged\. In response to this advice, the AutoAttack ensemble\(Croce and Hein,[2020b](https://arxiv.org/html/2607.12354#bib.bib27)\)has emerged as a popular benchmark to measure adversarial robustness due to its fixed hyperparameters, automatic convergence checks, and diversity of attacks\. The four primary adversarial attacks in the ensemble are 1\) untargeted Auto\-PGD with cross\-entropy loss; 2\) targeted Auto\-PGD with Difference of Logits Ratio \(DLR\) loss \(APGD\-T\); 3\) targeted Fast Adaptive Boundary \(FAB\-T\)\(Croce and Hein,[2020a](https://arxiv.org/html/2607.12354#bib.bib46)\); and 4\) Square Attack\(Andriushchenkoet al\.,[2020](https://arxiv.org/html/2607.12354#bib.bib20)\)\. The two Auto\-PGD variants are original to the AutoAttack paper\(Croce and Hein,[2020b](https://arxiv.org/html/2607.12354#bib.bib27)\)\. We run AutoAttack with varyingL∞L\_\{\\infty\}ϵ\\epsilon\-ball constraints on 1024 examples of the validation set for each of the reported target models\.

### F\.4Deferred full results from AutoAttack and adversarial robustness experiments

Tables[11](https://arxiv.org/html/2607.12354#A6.T11),[12](https://arxiv.org/html/2607.12354#A6.T12),[13](https://arxiv.org/html/2607.12354#A6.T13),[14](https://arxiv.org/html/2607.12354#A6.T14)present the full results of the AutoAttack adversarial robustness experiments forϵ∈\{0\.031,0\.0025,0\.0005,10−5\}\\epsilon\\in\\\{0\.031,0\.0025,0\.0005,10^\{\-5\}\\\}, respectively\. Specifically, for each of these values ofϵ\\epsilon, the respective table reports robust accuracy of each defense on each dataset and architecture both \(1\) overall \(i\.e\. worst\-case\), and \(2\) for each of standard AutoAttack’s four constituent attacks: untargeted Auto\-PGD; targeted DLR Auto\-PGD\(Croce and Hein,[2020b](https://arxiv.org/html/2607.12354#bib.bib27)\); targeted FAB\(Croce and Hein,[2020a](https://arxiv.org/html/2607.12354#bib.bib46)\); and Square\(Andriushchenkoet al\.,[2020](https://arxiv.org/html/2607.12354#bib.bib20)\)\.

Table 11:AutoAttack robust accuracies withϵ\\epsilon==0\.0310\.031\. Attack is on 8 batches \(1024 validation set examples\)\.DataArch\.DefenseTestAcc↑\\uparrowOverall↑\\uparrowAPGD\-CE↑\\uparrowAPGD\-T↑\\uparrowFAB\-T↑\\uparrowSquare↑\\uparrowDefenses that attempt to increase privacy generically by reducing information dependency/memorizationFaceScrubRN\-152NoDef0\.9580\.95800000\.0020\.002MID0\.9500\.95000\.0090\.0090\.0090\.0090\.1690\.1690\.8880\.888BiDO0\.9280\.92800000\.0030\.003TL\-DMI0\.9110\.91100000\.0060\.006AT\-AT0\.9390\.93900000\.0380\.038RN\-18NoDef0\.9500\.95000000MID0\.9240\.92400\.0220\.0220\.0300\.0300\.3490\.3490\.8800\.880BiDO0\.8840\.88400000\.0040\.004TL\-DMI0\.9060\.90600000\.0020\.002AT\-AT0\.9210\.92100000\.0080\.008CelebARN\-152NoDef0\.8380\.83800000\.0030\.003MID0\.8020\.8020\.0040\.0040\.0690\.0690\.0860\.0860\.3060\.3060\.7050\.705BiDO0\.7470\.74700000\.0010\.001TL\-DMI0\.6880\.68800000\.0010\.001TL\-DMI\-50\.8140\.81400000\.0050\.005AT\-AT0\.8190\.81900000\.0210\.021RN\-18NoDef0\.8590\.85900000\.0010\.001MID0\.7370\.7370\.0030\.0030\.0990\.0990\.1200\.1200\.3910\.3910\.6240\.624BiDO0\.6250\.62500000\.0030\.003TL\-DMI0\.7520\.75200000\.0010\.001TL\-DMI\-50\.8310\.83100000\.0030\.003AT\-AT†a0\.8250\.82500000\.0040\.004AT\-AT†b0\.7870\.78700000\.0020\.002Defenses that suppress attack gradientsFaceScrubRN\-152RoLSS\-SSF0\.8690\.86900000\.0050\.005RoLSS0\.8860\.88600000\.0050\.005NegLS0\.9060\.90600\.7820\.782000\.0040\.004Trap\-MID0\.9270\.9270000\.0230\.0230\.0010\.001Trap\-MID\*0\.9480\.9480000\.0230\.0230\.0230\.023RN\-18RoLSS\-SSF0\.9500\.95000000\.0030\.003RoLSS0\.9500\.95000000\.0030\.003NegLS0\.9290\.92900\.8150\.815000\.0020\.002Trap\-MID0\.9280\.9280000\.0090\.0090\.0040\.004Trap\-MID\*0\.9270\.9270000\.0040\.0040\.0030\.003CelebARN\-152RoLSS\-SSF0\.4280\.42800000RoLSS0\.5370\.5370000\.0010\.0010NegLS0\.8050\.80500\.1530\.153000\.0030\.003Trap\-MID0\.8480\.8480000\.0680\.0680\.0050\.005Trap\-MID\*0\.8060\.8060000\.0230\.0230\.0020\.002RN\-18RoLSS\-SSF0\.8390\.83900000\.0020\.002RoLSS0\.8300\.83000000\.0010\.001NegLS0\.8380\.83800\.0060\.006000Trap\-MID0\.8300\.8300000\.0200\.0200\.0040\.004Trap\-MID\*0\.8110\.8110000\.0140\.0140\.0040\.004

*Notes:*†aand†bdenote two sets of hyperparameters for AT\-AT \(see Table[6](https://arxiv.org/html/2607.12354#A2.T6)\)\.

Table 12:AutoAttack robust accuracies withϵ\\epsilon==0\.00250\.0025\. Attack is on 8 batches \(1024 validation set examples\)\.DataArch\.DefenseTestAcc↑\\uparrowOverall↑\\uparrowAPGD\-CE↑\\uparrowAPGD\-T↑\\uparrowFAB\-T↑\\uparrowSquare↑\\uparrowDefenses that attempt to increase privacy generically by reducing information dependency/memorizationFaceScrubRN\-152NoDef0\.9580\.9580\.1220\.1220\.150\.150\.1230\.1230\.1430\.1430\.8940\.894MID0\.9500\.9500\.0060\.0060\.0470\.0470\.0790\.0790\.2230\.2230\.9420\.942BiDO0\.9280\.9280\.1210\.1210\.1470\.1470\.1210\.1210\.1330\.1330\.8440\.844TL\-DMI0\.9110\.91100000\.3340\.334AT\-AT0\.9390\.93900000\.1010\.101RN\-18NoDef0\.9500\.9500\.1940\.1940\.2310\.2310\.1940\.1940\.2150\.2150\.8830\.883MID0\.9240\.9240\.0930\.0930\.1270\.1270\.440\.440\.6120\.6120\.9050\.905BiDO0\.8840\.8840\.0750\.0750\.0890\.0890\.0750\.0750\.0920\.0920\.7790\.779TL\-DMI0\.9060\.90600000\.4780\.478AT\-AT0\.9210\.92100000\.0740\.074CelebARN\-152NoDef0\.8380\.8380\.0500\.0500\.0660\.0660\.0500\.0500\.0670\.0670\.6990\.699MID0\.8020\.8020\.1390\.1390\.2220\.2220\.4850\.4850\.5710\.5710\.7260\.726BiDO0\.7470\.7470\.0620\.0620\.0700\.0700\.0620\.0620\.0720\.0720\.6000\.600TL\-DMI0\.6880\.68800000\.2240\.224TL\-DMI\-50\.8140\.81400000\.5410\.541AT\-AT0\.8190\.81900000\.0450\.045RN\-18NoDef0\.8590\.8590\.0840\.0840\.1020\.1020\.0840\.0840\.0990\.0990\.7540\.754MID0\.7370\.7370\.1680\.1680\.2660\.2660\.4230\.4230\.5510\.5510\.6500\.650BiDO0\.6250\.6250\.0070\.0070\.0100\.0100\.0070\.0070\.0110\.0110\.4450\.445TL\-DMI0\.7520\.75200000\.3170\.317TL\-DMI\-50\.8310\.83100000\.5170\.517AT\-AT†a0\.8250\.82500000\.3680\.368AT\-AT†b0\.7870\.78700000\.1910\.191Defenses that suppress attack gradientsFaceScrubRN\-152RoLSS\-SSF0\.8690\.8690\.0130\.0130\.0210\.0210\.0140\.0140\.0290\.0290\.7270\.727RoLSS0\.8860\.8860\.0090\.0090\.0190\.0190\.0090\.0090\.0210\.0210\.7670\.767NegLS0\.9060\.9060\.0480\.0480\.7840\.7840\.0480\.0480\.0650\.0650\.7740\.774Trap\-MID0\.9270\.9270000\.0230\.0230\.0960\.096Trap\-MID\*0\.9480\.9480000\.0230\.0230\.0370\.037RN\-18RoLSS\-SSF0\.9500\.95000\.0020\.00200\.0130\.0130\.8820\.882RoLSS0\.9500\.9500\.0020\.0020\.0040\.0040\.0020\.0020\.0090\.0090\.8800\.880NegLS0\.9290\.9290\.2150\.2150\.8140\.8140\.2150\.2150\.2380\.2380\.8380\.838Trap\-MID0\.9280\.9280000\.0100\.0100\.2950\.295Trap\-MID\*0\.9270\.9270000\.0050\.0050\.0180\.018CelebARN\-152RoLSS\-SSF0\.4280\.42800000\.1850\.185RoLSS0\.5370\.5370000\.0010\.0010\.2340\.234NegLS0\.8050\.8050\.0130\.0130\.1840\.1840\.0130\.0130\.0190\.0190\.6240\.624Trap\-MID0\.8480\.8480000\.0680\.0680\.4980\.498Trap\-MID\*0\.8060\.8060000\.0230\.0230\.0390\.039RN\-18RoLSS\-SSF0\.8390\.8390\.080\.080\.0940\.0940\.080\.080\.0980\.0980\.7290\.729RoLSS0\.8300\.8300\.0690\.0690\.0780\.0780\.0700\.0700\.0830\.0830\.7090\.709NegLS0\.8380\.8380\.1000\.1000\.1470\.1470\.1000\.1000\.1140\.1140\.7240\.724Trap\-MID0\.8300\.8300\.0440\.0440\.1440\.1440\.0440\.0440\.0790\.0790\.6900\.690Trap\-MID\*0\.8110\.8110000\.0150\.0150\.0690\.069

*Notes:*†aand†bdenote two sets of hyperparameters for AT\-AT \(see Table[6](https://arxiv.org/html/2607.12354#A2.T6)\)\.

Table 13:AutoAttack robust accuracies withϵ\\epsilon==0\.00050\.0005\. Attack is on 8 batches \(1024 validation set examples\)\.DataArch\.DefenseTestAcc↑\\uparrowOverall↑\\uparrowAPGD\-CE↑\\uparrowAPGD\-T↑\\uparrowFAB\-T↑\\uparrowSquare↑\\uparrowDefenses that attempt to increase privacy generically by reducing information dependency/memorizationFaceScrubRN\-152NoDef0\.9580\.9580\.8990\.8990\.9000\.9000\.8990\.8990\.9000\.9000\.9550\.955MID0\.9500\.9500\.7700\.7700\.7930\.7930\.8490\.8490\.9040\.9040\.9410\.941BiDO0\.9280\.9280\.8470\.8470\.8480\.8480\.8470\.8470\.8490\.8490\.910\.91TL\-DMI0\.9110\.91100000\.8010\.801AT\-AT0\.9390\.93900000\.5160\.516RN\-18NoDef0\.9500\.9500\.8960\.8960\.8960\.8960\.8960\.8960\.8960\.8960\.9460\.946MID0\.9240\.9240\.8250\.8250\.8710\.8710\.9000\.9000\.8980\.8980\.9040\.904BiDO0\.8840\.8840\.7840\.7840\.7850\.7850\.7840\.7840\.7860\.7860\.8590\.859TL\-DMI0\.9060\.9060\.0030\.0030\.0040\.0040\.0030\.0030\.0050\.0050\.8430\.843AT\-AT0\.9210\.92100000\.6960\.696CelebARN\-152NoDef0\.8380\.8380\.6750\.6750\.6770\.6770\.6750\.6750\.6750\.6750\.8000\.800MID0\.8020\.8020\.5420\.5420\.7040\.7040\.7260\.7260\.7170\.7170\.7280\.728BiDO0\.7470\.7470\.5900\.5900\.5900\.5900\.5900\.5900\.5900\.5900\.7170\.717TL\-DMI0\.6880\.6880\.0020\.0020\.0020\.0020\.0020\.0020\.0040\.0040\.5590\.559TL\-DMI\-50\.8140\.8140\.1190\.1190\.1300\.1300\.1190\.1190\.1320\.1320\.7650\.765AT\-AT0\.8190\.81900000\.1040\.104RN\-18NoDef0\.8590\.8590\.7460\.7460\.7480\.7480\.7460\.7460\.7470\.7470\.8350\.835MID0\.7370\.7370\.4520\.4520\.5990\.5990\.6360\.6360\.6390\.6390\.6460\.646BiDO0\.6250\.6250\.3940\.3940\.3980\.3980\.3940\.3940\.3960\.3960\.5810\.581TL\-DMI0\.7520\.7520\.0280\.0280\.0370\.0370\.0280\.0280\.0410\.0410\.6560\.656TL\-DMI\-50\.8310\.8310\.1460\.1460\.1540\.1540\.1460\.1460\.1650\.1650\.7630\.763AT\-AT†a0\.8250\.82500000\.7210\.721AT\-AT†b0\.7870\.78700000\.6170\.617Defenses that suppress attack gradientsFaceScrubRN\-152RoLSS\-SSF0\.8690\.8690\.7210\.7210\.7210\.7210\.7210\.7210\.7230\.7230\.8430\.843RoLSS0\.8860\.8860\.7440\.7440\.7470\.7470\.7440\.7440\.7500\.7500\.8680\.868NegLS0\.9060\.9060\.7690\.7690\.8280\.8280\.7690\.7690\.7690\.7690\.8820\.882Trap\-MID0\.9270\.9270000\.0230\.0230\.7900\.790Trap\-MID\*0\.9480\.9480000\.0230\.0230\.8900\.890RN\-18RoLSS\-SSF0\.9500\.9500\.8130\.8130\.8160\.8160\.8130\.8130\.8200\.8200\.9410\.941RoLSS0\.9500\.9500\.8190\.8190\.8220\.8220\.8190\.8190\.8260\.8260\.9380\.938NegLS0\.9290\.9290\.8540\.8540\.8690\.8690\.8540\.8540\.8540\.8540\.9160\.916Trap\-MID0\.9280\.9280\.5300\.5300\.5650\.5650\.5480\.5480\.5850\.5850\.8520\.852Trap\-MID\*0\.9270\.9270000\.0040\.0040\.8330\.833CelebARN\-152RoLSS\-SSF0\.4280\.4280\.0420\.0420\.0480\.0480\.0420\.0420\.0540\.0540\.3940\.394RoLSS0\.5370\.5370\.0860\.0860\.0940\.0940\.0870\.0870\.1020\.1020\.4940\.494NegLS0\.8050\.8050\.5590\.5590\.5780\.5780\.5590\.5590\.5620\.5620\.7490\.749Trap\-MID0\.8480\.8480\.3330\.3330\.4120\.4120\.3380\.3380\.4090\.4090\.7870\.787Trap\-MID\*0\.8060\.8060000\.0230\.0230\.6010\.601RN\-18RoLSS\-SSF0\.8390\.8390\.7230\.7230\.7280\.7280\.7230\.7230\.7250\.7250\.8190\.819RoLSS0\.8300\.8300\.7010\.7010\.7040\.7040\.7010\.7010\.7020\.7020\.7960\.796NegLS0\.8380\.8380\.7110\.7110\.7110\.7110\.7110\.7110\.7120\.7120\.8030\.803Trap\-MID0\.8300\.8300\.6850\.6850\.7320\.7320\.6850\.6850\.6900\.6900\.8120\.812Trap\-MID\*0\.8110\.8110\.0010\.0010\.0010\.0010\.0010\.0010\.0160\.0160\.7240\.724

*Notes:*†aand†bdenote two sets of hyperparameters for AT\-AT \(see Table[6](https://arxiv.org/html/2607.12354#A2.T6)\)\.

Table 14:AutoAttack robust accuracies withϵ\\epsilon==10−510^\{\-5\}\. Attack is on 8 batches \(1024 validation set examples\)\.DataArch\.DefenseTestAcc↑\\uparrowOverall↑\\uparrowAPGD\-CE↑\\uparrowAPGD\-T↑\\uparrowFAB\-T↑\\uparrowSquare↑\\uparrowDefenses that attempt to increase privacy generically by reducing information dependency/memorizationFaceScrubRN\-152NoDef0\.9580\.9580\.9600\.9600\.9600\.9600\.9600\.9600\.9600\.9600\.9600\.960MID0\.9500\.9500\.9280\.9280\.9440\.9440\.9400\.9400\.9450\.9450\.9410\.941BiDO0\.9280\.9280\.9230\.9230\.9230\.9230\.9230\.9230\.9230\.9230\.9230\.923TL\-DMI0\.9110\.9110\.8670\.8670\.8670\.8670\.8670\.8670\.8670\.8670\.9020\.902AT\-AT0\.9390\.9390\.1190\.1190\.1270\.1270\.1200\.1200\.1570\.1570\.8970\.897RN\-18NoDef0\.9500\.9500\.9590\.9590\.9590\.9590\.9590\.9590\.9590\.9590\.9590\.959MID0\.9240\.9240\.8560\.8560\.9000\.9000\.9050\.9050\.8980\.8980\.9140\.914BiDO0\.8840\.8840\.8790\.8790\.8800\.8800\.8790\.8790\.8790\.8790\.8810\.881TL\-DMI0\.9060\.9060\.8860\.8860\.8870\.8870\.8860\.8860\.8860\.8860\.8990\.899AT\-AT0\.9210\.9210\.6920\.6920\.7020\.7020\.6920\.6920\.7110\.7110\.9110\.911CelebARN\-152NoDef0\.8380\.8380\.8180\.8180\.8180\.8180\.8180\.8180\.8190\.8190\.8200\.820MID0\.8020\.8020\.5740\.5740\.7290\.7290\.7230\.7230\.7200\.7200\.7330\.733BiDO0\.7470\.7470\.7480\.7480\.7490\.7490\.7480\.7480\.7480\.7480\.7510\.751TL\-DMI0\.6880\.6880\.6410\.6410\.6440\.6440\.6410\.6410\.6430\.6430\.6680\.668TL\-DMI\-50\.8140\.8140\.7990\.7990\.8010\.8010\.7990\.7990\.8010\.8010\.8110\.811AT\-AT0\.8190\.8190\.1550\.1550\.1780\.1780\.1570\.1570\.1800\.1800\.7220\.722RN\-18NoDef0\.8590\.8590\.8480\.8480\.8480\.8480\.8480\.8480\.8480\.8480\.8490\.849MID0\.7370\.7370\.4640\.4640\.6440\.6440\.6480\.6480\.6510\.6510\.6460\.646BiDO0\.6250\.6250\.6090\.6090\.6090\.6090\.6090\.6090\.6090\.6090\.6110\.611TL\-DMI0\.7520\.7520\.7180\.7180\.7180\.7180\.7180\.7180\.7190\.7190\.7340\.734TL\-DMI\-50\.8310\.8310\.8020\.8020\.8020\.8020\.8020\.8020\.8030\.8030\.8160\.816AT\-AT†a0\.8250\.8250\.7560\.7560\.7580\.7580\.7560\.7560\.7570\.7570\.8020\.802AT\-AT†b0\.7870\.7870\.6450\.6450\.6610\.6610\.6450\.6450\.6520\.6520\.7860\.786Defenses that suppress attack gradientsFaceScrubRN\-152RoLSS\-SSF0\.8690\.8690\.8570\.8570\.8570\.8570\.8570\.8570\.8570\.8570\.8600\.860RoLSS0\.8860\.8860\.8820\.8820\.8820\.8820\.8820\.8820\.8820\.8820\.8830\.883NegLS0\.9060\.9060\.9050\.9050\.9050\.9050\.9050\.9050\.9050\.9050\.9110\.911Trap\-MID0\.9270\.9270\.8560\.8560\.8590\.8590\.8560\.8560\.8610\.8610\.9110\.911Trap\-MID0\.9480\.9480\.9430\.9430\.9430\.9430\.9430\.9430\.9430\.9430\.9480\.948RN\-18RoLSS\-SSF0\.9500\.9500\.9510\.9510\.9520\.9520\.9510\.9510\.9520\.9520\.9540\.954RoLSS0\.9500\.9500\.9480\.9480\.9480\.9480\.9480\.9480\.9480\.9480\.9510\.951NegLS0\.9290\.9290\.9330\.9330\.9330\.9330\.9330\.9330\.9330\.9330\.9340\.934Trap\-MID0\.9280\.9280\.9140\.9140\.9140\.9140\.9140\.9140\.9140\.9140\.9160\.916Trap\-MID\*0\.9270\.9270\.9210\.9210\.9210\.9210\.9210\.9210\.9210\.9210\.9340\.934CelebARN\-152RoLSS\-SSF0\.4280\.4280\.4060\.4060\.4080\.4080\.4060\.4060\.4080\.4080\.4140\.414RoLSS0\.5370\.5370\.5300\.5300\.5300\.5300\.5300\.5300\.5300\.5300\.5380\.538NegLS0\.8050\.8050\.7710\.7710\.7710\.7710\.7710\.7710\.7710\.7710\.7760\.776Trap\-MID0\.8480\.8480\.8280\.8280\.8300\.8300\.8280\.8280\.8280\.8280\.8350\.835Trap\-MID\*0\.8060\.8060\.7600\.7600\.7600\.7600\.7600\.7600\.7600\.7600\.8000\.800RN\-18RoLSS\-SSF0\.8390\.8390\.8340\.8340\.8340\.8340\.8340\.8340\.8340\.8340\.8340\.834RoLSS0\.8300\.8300\.8130\.8130\.8130\.8130\.8130\.8130\.8140\.8140\.8150\.815NegLS0\.8380\.8380\.8250\.8250\.8250\.8250\.8250\.8250\.8250\.8250\.8250\.825Trap\-MID0\.8300\.8300\.8270\.8270\.8280\.8280\.8270\.8270\.8280\.8280\.8290\.829Trap\-MID\*0\.8110\.8110\.7900\.7900\.7900\.7900\.7900\.7900\.7910\.7910\.8090\.809

*Notes:*†aand†bdenote two sets of hyperparameters for AT\-AT \(see Table[6](https://arxiv.org/html/2607.12354#A2.T6)\)\.

## Appendix GDeferred privacy\-robustness regression analysis details and results

Section[4](https://arxiv.org/html/2607.12354#S4)investigates whether the privacy associated with mainstream generic information dependency\-based defenses \(MID, BiDO, TL\-DMI\) is actually correlated with, or explained by, a tradeoff with adversarial robustness\. Concretely, we ask:

1. 1\.Can we perfectly predict MIA leakage \(AttAcc@1\) just from a defense’s robust accuracy under AutoAttack?
2. 2\.For each percentage point reduction in leakage \(AttAcc@1 under PPA or IF\-GMI\), what is the associated cost in terms of robust accuracy under AutoAttack at differentε\\varepsilon, which capture non\-robust features of different magnitudes \(See Fig\.[5](https://arxiv.org/html/2607.12354#A6.F5)\)\.

We accomplish this by presenting the first systematic tests of the robustness of MIA defenses to a very different attack vector: adversarial examples\. Tables[11](https://arxiv.org/html/2607.12354#A6.T11),[12](https://arxiv.org/html/2607.12354#A6.T12),[13](https://arxiv.org/html/2607.12354#A6.T13),[14](https://arxiv.org/html/2607.12354#A6.T14)in App\.[F\.4](https://arxiv.org/html/2607.12354#A6.SS4)report robust accuracies under each of AutoAttack’s constituent attacks\. Specifically, we are interested in the standard AutoAttack robust accuracy, which is the worst\-case robust accuracy across the various attacks in the AutoAttack ensemble, at various magnitudesε\\varepsilon\. This is reported in Table[4](https://arxiv.org/html/2607.12354#S4.T4)\.

To obtain a principled test of whether these raw AutoAttack and privacy leakage results reflect a relationship between leakage and robustness under AutoAttack, we consider the following simple*main regression specifications*\. These standard regression models test the degree to which privacy leakage \(our response variable, measured by AttAcc@1∈\[0,1\]\\in\[0,1\]under PPA or under IF\-GMI\) is a linear function of a defense’s*robust accuracy*under AutoAttack at several perturbation magnitudes, above and beyond its*clean*test accuracy \(control variable\)\. Intuitively, these models ask:*holding clean accuracy fixed, do defenses that are less robust to small adversarial perturbations \(i\.e\., that rely more on non\-robust features\) exhibit less leakage under MIA?*

Specifically, the following OLS model estimates the magnitude and significance of this privacy\-robustness relationship:

LeakageAttAcc@1,j=β0\+β1​TestAccj\+β2​Accϵ​=​0\.0025,j\+β3​Accϵ​=​0\.0005,j\+β4​Accϵ​=​10−5,j\+ej\\displaystyle\\text\{Leakage\}\_\{\\text\{AttAcc@1\},j\}=\\beta\_\{0\}\\text\{$\+$\}\\beta\_\{1\}\\text\{TestAcc\}\_\{j\}\\text\{$\+$\}\\beta\_\{2\}\\text\{Acc\}\_\{\\epsilon\\text\{=\}0\.0025,j\}\\text\{$\+$\}\\beta\_\{3\}\\text\{Acc\}\_\{\\epsilon\\text\{=\}0\.0005,j\}\\text\{$\+$\}\\beta\_\{4\}\\text\{Acc\}\_\{\\epsilon\\text\{=\}10^\{\-5\},j\}\+e\_\{j\}\(4\)
We are careful to also estimate Beta regression variants with appropriate distributional assumptions and logit link \(g​\(x\)g\(x\)\) to capture the fact that the response \(AttAcc@1 under PPA or IF\-GMI\) is bounded in\[0,1\]\[0,1\]:

LeakageAttAcc@1,j\\displaystyle\\mathrm\{Leakage\}\_\{\\text\{AttAcc@1\},j\}∼Beta​\(μj​ϕ,\(1−μj\)​ϕ\)\\displaystyle\\sim\\mathrm\{Beta\}\(\\mu\_\{j\}\\phi,\\,\(1\-\\mu\_\{j\}\)\\phi\)g​\(μj\)=β0\\displaystyle g\(\\mu\_\{j\}\)=\\;\\beta\_\{0\}\+β1​TestAccj\+β2​Accϵ=0\.0025,j\+β3​Accϵ=0\.0005,j\+β4​Accϵ=10−5,j\\displaystyle\\text\{$\+$\}\\beta\_\{1\}\\,\\mathrm\{TestAcc\}\_\{j\}\\text\{$\+$\}\\beta\_\{2\}\\,\\mathrm\{Acc\}\_\{\\epsilon=0\.0025,\\,j\}\\text\{$\+$\}\\beta\_\{3\}\\,\\mathrm\{Acc\}\_\{\\epsilon=0\.0005,\\,j\}\\text\{$\+$\}\\beta\_\{4\}\\,\\mathrm\{Acc\}\_\{\\epsilon=10^\{\-5\},\\,j\}\(5\)
Because we estimate these models using a relatively small number of observations, we are also careful to report, for OLS, the appropriate small sample\-robust HC2 standard errors\. We are also careful to compute confidence intervals with the conservative small samplet0\.975,13t\_\{0\.975,13\}==2\.1602\.160\(with the HC2 SEs\) and report*adjusted*R2R^\{2\}\. Beta regressions already accommodate appropriate assumptions, so we report CI with the standard Wald \(±1\.96\\pm 1\.96SE\) on the logit scale\.

To evaluate goodness\-of\-fit, we report*degree\-of\-freedom adjusted*R2R^\{2\}\(for OLS\) or the appropriately estimated pseudo\-R2R^\{2\}\(for Beta regressions\)\. For all specifications, we also report Mean Absolute Error \(MAE\) on the response scale\.

Figure[2](https://arxiv.org/html/2607.12354#S4.F2)visualizes the fitted values\.

##### Interpretation \(units and signs\)\.

All accuracies and leakages are in\[0,1\]\[0,1\]during estimation; for readability we discuss effects in*percentage points*\(pp\)\. A positive regressionβ2,β3,\\beta\_\{2\},\\beta\_\{3\},and/orβ4\\beta\_\{4\}means that,*holding clean accuracy fixed*, higher robust accuracy at that respectiveϵ\\epsilonis associated with*higher*leakage \(worse privacy\)\. Equivalently, privacy gains from generic defenses coincide with losses in robustness to small perturbations \(i\.e\., increased reliance on non\-robust features\)\. The coefficientβ1\\beta\_\{1\}captures any residual association between clean accuracy and leakage after conditioning on robustness\.

*Defenses, scope, and limitations\.*In keeping with our hypothesis that*generic*defenses work, at least in part, by shifting their models’ feature bases towards non\-robust features, we estimate regressions this set of defenses \(MID, BiDO, TL\-DMI, and TL\-DMI\-5 where applicable\)\. Note that TL\-DMI\-5 is a variant of TL\-DMI with 5 layers frozen instead of the standard 6 layers, which we include alongside the standard TL\-DMI to give this defense the best opportunity to perform well on lower\-capacity ResNet\-18 networks that have fewer layers\.

We then reestimate all regressions on the dataset of gradient\-suppressing defenses \(NegLS, RoLSS, RoLSS\-SSF\), which directly target PPA/IF\-GMI gradients, as we do not hypothesize that these defenses work via non\-robust features\.

We emphasize that these regressions provide correlational evidence, and*do not by themselves prove causality*\. Our separate randomized AT\-AT experiment \(Section[5](https://arxiv.org/html/2607.12354#S5)\) provides causal evidence by treating the model’s reliance on non\-robust features in a randomized controlled experiment and observing the induced change in leakage\.

### G\.1Deferred OLS and Beta regression results\.

We find that leakage \(AttAcc@1under both PPA and IF\-GMI\) is strongly and significantly correlated with robust accuracy at all levels ofϵ\\epsilon\(noteppvalues\), withR2R^\{2\}≈\\approx0\.930\.93−\-0\.950\.95, and mean abs\. error of just0\.0420\.042−\-0\.050\.05\. Contrary to widespread belief, \(2\) clean test accuracy is*only weakly or insignificantly correlated*with leakage after controlling for robust accuracy\. To examine the robustness of these results to different regression specifications, PPA versus IF\-GMI attack leakage, different distributional assumptions, controls for dataset/architecture, various robust standard errors estimators, etc\., we show that these results hold across 56 alternate regression specifications\.

Specifically, Table[15](https://arxiv.org/html/2607.12354#A7.T15)presents full deferred OLS and Beta regressions for NoDef and generic defenses \(MID, BiDO, TL\-DMI\), which we hypothesize unintentionally obtain their privacy from non\-robust features\.

Table[18](https://arxiv.org/html/2607.12354#A7.T18)recomputes these regressions on generic privacy defenses using IF\-GMI AttAcc@1 to measure privacy leakage \(response variable\) instead of PPA\. A similar strong relationship exists between robust accuracy and leakage using the IF\-GMI AttAcc@1 as a leakage measure\.

Table[16](https://arxiv.org/html/2607.12354#A7.T16)recomputes these regressions using PPA AttAcc@1, but on gradient\-suppressing defenses \(NegLS, RoLSS, RoLSS\-SSF\), which we do not hypothesize to work via non\-robust features\. As expected, the regression results do not show the same strong, significant relationship between leakage and non\-robust features\. Also note that the model fit metrics \(e\.g\.R2R^\{2\}\) are far lower for these gradient\-suppressing defenses\.

Table[19](https://arxiv.org/html/2607.12354#A7.T19)shows that the lack of clear, significant relationship between leakage and non\-robust features*for gradient\-suppressing defenses*\(NegLS, RoLSS, RoLSS\-SSF\) persists when we measure leakage via IF\-GMI AttAcc@1 instead of PPA AttAcc@1\.

Finally, Tables[17](https://arxiv.org/html/2607.12354#A7.T17)and[20](https://arxiv.org/html/2607.12354#A7.T20)present multiple additional ‘robustness test’ regressions, showing that the strong and highly statistically significant relationship between leakage and non\-robust features is not merely an artifact of the regression specification\. Specifically, Tables[17](https://arxiv.org/html/2607.12354#A7.T17)and[20](https://arxiv.org/html/2607.12354#A7.T20),*left side*show that controlling for dataset interactions still yields a significantly worse fitting regression model compared to the non\-robust features regressions described above, regardless of whether leakage is measured via PPA attacks \(Table[17](https://arxiv.org/html/2607.12354#A7.T17)\) or IF\-GMI attacks \(Table[20](https://arxiv.org/html/2607.12354#A7.T20)\)\. Tables[17](https://arxiv.org/html/2607.12354#A7.T17)and[20](https://arxiv.org/html/2607.12354#A7.T20),*right side*show that the same applies when we instead control for architecture interactions\. Note that the goodness\-of\-fit metrics in both cases are inferior to the goodness\-of\-fit metrics for non\-robust regressions\.

##### Using coefficients to estimate the*cost of privacy*\.

Per the Beta model, we can compute that reducing PPA leakage \(AttAcc@1\) by 1 percentage point \(pp\) corresponds to a significant*0\.31 pp decrease*in AutoAttack robust accuracy atϵ\\epsilon==0\.0025, \(95% CI: 0\.24–0\.45\), a significant*5\.4 pp decrease*atϵ\\epsilon==0\.0005 \(95% CI: 2\.9–58\.3\), and a significant*0\.91 pp decrease*atϵ\\epsilon==10−510^\{\-5\}\(95% CI: 0\.60–1\.93\)\. To derive these ’costs’ of privacy, note that in Beta regression with logit link \(Eq\.[2](https://arxiv.org/html/2607.12354#S4.E2)\),logit​\(μj\)=Xj​β\\mathrm\{logit\}\(\\mu\_\{j\}\)=X\_\{j\}\\beta, coefficientsβϵ\\beta\_\{\\epsilon\}live on the link scale\. The response\-scale marginal effect of robust accuracy atϵ\\epsilonon expected leakage is∂𝔼​\[Leakage\]∂Accϵ=βϵ​μ​\(1−μ\)\\displaystyle\\frac\{\\partial\\,\\mathbb\{E\}\[\\mathrm\{Leakage\}\]\}\{\\partial\\,\\mathrm\{Acc\}\_\{\\epsilon\}\}=\\beta\_\{\\epsilon\}\\,\\mu\(1\-\\mu\)\. Evaluated at the sample mean leakageμ¯\\bar\{\\mu\}from the regression sample, the*robustness cost of a 1 pp privacy gain*is the inverse slope

γϵ≡∂Accϵ∂Leakage\|μ=μ¯=1βϵ​μ¯​\(1−μ¯\)\(pp of robust accuracy per 1 pp change in leakage\)\.\\gamma\_\{\\epsilon\}\\;\\equiv\\;\\left\.\\frac\{\\partial\\,\\mathrm\{Acc\}\_\{\\epsilon\}\}\{\\partial\\,\\mathrm\{Leakage\}\}\\right\|\_\{\\mu=\\bar\{\\mu\}\}=\\frac\{1\}\{\\beta\_\{\\epsilon\}\\,\\bar\{\\mu\}\(1\-\\bar\{\\mu\}\)\}\\quad\\text\{\(pp of robust accuracy per 1 pp change in leakage\)\}\.*Confidence intervals\.*Using the delta method, if we treatμ¯\\bar\{\\mu\}as fixed,SE​\(γ^ϵ\)≈\|∂γ∂βϵ\|​SE​\(β^ϵ\)=1\|β^ϵ\|2​μ¯​\(1−μ¯\)​SE​\(β^ϵ\)\\mathrm\{SE\}\(\\hat\{\\gamma\}\_\{\\epsilon\}\)\\approx\\left\|\\frac\{\\partial\\gamma\}\{\\partial\\beta\_\{\\epsilon\}\}\\right\|\\,\\mathrm\{SE\}\(\\hat\{\\beta\}\_\{\\epsilon\}\)=\\frac\{1\}\{\|\\hat\{\\beta\}\_\{\\epsilon\}\|^\{2\}\\,\\bar\{\\mu\}\(1\-\\bar\{\\mu\}\)\}\\,\\mathrm\{SE\}\(\\hat\{\\beta\}\_\{\\epsilon\}\), so a95%95\\%CI isγ^ϵ±1\.96​SE​\(γ^ϵ\)\\hat\{\\gamma\}\_\{\\epsilon\}\\pm 1\.96\\,\\mathrm\{SE\}\(\\hat\{\\gamma\}\_\{\\epsilon\}\)\. More generally, allowing for uncertainty in the parameterμ\\mu:

Var​\(γ^ϵ\)≈\(∂γ∂βϵ\)2​Var​\(β^ϵ\)⏟link coef\.\+\(∂γ∂μ\)2​Var​\(μ^\)⏟mean leakage\+2​∂γ∂βϵ​∂γ∂μ​Cov​\(β^ϵ,μ^\),\\mathrm\{Var\}\(\\hat\{\\gamma\}\_\{\\epsilon\}\)\\;\\approx\\;\\underbrace\{\\Big\(\\tfrac\{\\partial\\gamma\}\{\\partial\\beta\_\{\\epsilon\}\}\\Big\)^\{2\}\\mathrm\{Var\}\(\\hat\{\\beta\}\_\{\\epsilon\}\)\}\_\{\\text\{link coef\.\}\}\\;\+\\;\\underbrace\{\\Big\(\\tfrac\{\\partial\\gamma\}\{\\partial\\mu\}\\Big\)^\{2\}\\mathrm\{Var\}\(\\hat\{\\mu\}\)\}\_\{\\text\{mean leakage\}\}\\;\+\\;2\\,\\tfrac\{\\partial\\gamma\}\{\\partial\\beta\_\{\\epsilon\}\}\\tfrac\{\\partial\\gamma\}\{\\partial\\mu\}\\,\\mathrm\{Cov\}\(\\hat\{\\beta\}\_\{\\epsilon\},\\hat\{\\mu\}\),with∂γ∂βϵ=−1βϵ2​μ​\(1−μ\)\\displaystyle\\tfrac\{\\partial\\gamma\}\{\\partial\\beta\_\{\\epsilon\}\}=\-\\frac\{1\}\{\\beta\_\{\\epsilon\}^\{2\}\\,\\mu\(1\-\\mu\)\}and∂γ∂μ=−1−2​μβϵ​μ2​\(1−μ\)2\\displaystyle\\tfrac\{\\partial\\gamma\}\{\\partial\\mu\}=\-\\frac\{1\-2\\mu\}\{\\beta\_\{\\epsilon\}\\,\\mu^\{2\}\(1\-\\mu\)^\{2\}\}\. In practice we report CIs using the fixed\-μ¯\\bar\{\\mu\}version \(slightly narrower\); includingVar​\(μ^\)\\mathrm\{Var\}\(\\hat\{\\mu\}\)yields virtually identical conclusions\.

Table 15:Linear \(OLS\) & Beta regressions quantifying the privacy–robustness tradeoff forgeneric information dependency\-based privacy defensesunder various specifications\. Leakage \(response\) isPPA AttAcc@1\. SE’s in parentheses; OLS MacKinnon & White \(HC2\) small\-sample robust SE’s in brackets\. Test accuracy is clean;ϵ\\epsilon’s are robust accuracies under the AutoAttack ensemble at the indicated noise levels\. Note that Beta coefficients are on the logit scale\.All PredictorsAllϵ\\epsilonClean Acc\.ϵ=0\.0025\\epsilon\{=\}0\.0025ϵ=0\.0005\\epsilon\{=\}0\.0005ϵ=10−5\\epsilon\{=\}10^\{\-5\}*OLS**Beta**OLS**Beta**OLS**Beta**OLS**Beta**OLS**Beta**OLS**Beta*PredictorsTest Acc\.\-0\.011\-0\.1921\.759∗∗7\.495∗∗∗\(0\.390\)\(1\.582\)\(0\.477\)\(2\.041\)\[0\.478\]\[0\.458\]ϵ=0\.0025\\epsilon\{=\}0\.00252\.299∗∗∗12\.827∗∗∗2\.291∗∗∗12\.685∗∗∗3\.049∗∗∗13\.689∗∗∗\(0\.452\)\(2\.010\)\(0\.363\)\(1\.672\)\(0\.604\)\(2\.694\)\[0\.739\]\[0\.428\]\[0\.768\]ϵ=0\.0005\\epsilon\{=\}0\.00050\.211∗0\.737∗0\.212∗0\.748∗0\.639∗∗∗2\.915∗∗∗\(0\.080\)\(0\.325\)\(0\.074\)\(0\.313\)\(0\.090\)\(0\.442\)\[0\.108\]\[0\.083\]\[0\.084\]ϵ=10−5\\epsilon\{=\}10^\{\-5\}0\.795∗4\.390∗∗∗0\.788∗∗∗4\.263∗∗∗0\.969∗∗4\.561∗∗\(0\.285\)\(1\.180\)\(0\.130\)\(0\.597\)\(0\.378\)\(1\.524\)\[0\.360\]\[0\.113\]\[0\.474\]Fit statisticsR2R^\{2\}0\.9340\.9520\.9390\.9520\.4250\.4850\.5900\.6130\.7430\.7330\.2470\.304Mean abs\. err\.0\.0470\.0420\.0470\.0420\.1560\.1510\.1260\.1240\.0990\.0950\.1860\.181Note:R2R^\{2\}are Adjusted \(OLS\); Pseudo \(Beta\)\.\*p<<0\.1;\*\*p<<0\.05;\*\*\*p<<0\.01

Table 16:Linear \(OLS\) & Beta regressions quantifying privacy–robustness tradeoff forattack gradient suppressor defenses \(NegLS, RoLSS, RoLSS\-SSF\)under various specifications\. Leakage \(response\) isPPA AttAcc@1\. SE’s in parentheses; OLS MacKinnon & White \(HC2\) small\-sample robust SE’s in brackets\. Test accuracy is clean;ϵ\\epsilon’s are robust accuracies under AutoAttack at the indicated noise levels\. Note that Beta coefficients are on the logit scale\.All PredictorsAllϵ\\epsilonClean Acc\.ϵ=0\.0025\\epsilon\{=\}0\.0025ϵ=0\.0005\\epsilon\{=\}0\.0005ϵ=10−5\\epsilon\{=\}10^\{\-5\}*OLS**Beta**OLS**Beta**OLS**Beta**OLS**Beta**OLS**Beta**OLS**Beta*PredictorsTest Acc\.\-2\.016\-0\.0531\.296∗∗6\.434∗∗∗\(8\.500\)\(27\.413\)\(0\.353\)\(1\.721\)\[3\.780\]\[0\.282\]ϵ=0\.0025\\epsilon\{=\}0\.0025\-0\.167\-0\.912\-0\.026\-0\.9081\.2644\.883\(1\.419\)\(4\.615\)\(1\.211\)\(4\.189\)\(1\.327\)\(4\.833\)\[0\.785\]\[0\.646\]\[0\.906\]ϵ=0\.0005\\epsilon\{=\}0\.00050\.7572\.8680\.5472\.8620\.780∗∗3\.805∗∗∗\(1\.808\)\(6\.437\)\(1\.481\)\(5\.743\)\(0\.208\)\(1\.025\)\[0\.537\]\[0\.691\]\[0\.125\]ϵ=10−5\\epsilon\{=\}10^\{\-5\}2\.0181\.7590\.3871\.7171\.258∗∗6\.171∗∗∗\(7\.307\)\(23\.662\)\(2\.326\)\(8\.951\)\(0\.340\)\(1\.649\)\[4\.067\]\[1\.337\]\[0\.282\]Fit statisticsR2R^\{2\}0\.3550\.7270\.4310\.7270\.5320\.718\-0\.0080\.0800\.5430\.7220\.5350\.713Mean abs\. err\.0\.1200\.1240\.1230\.1240\.1370\.1310\.1940\.1950\.1220\.1180\.1350\.129Note:R2R^\{2\}are Adj\. \(OLS\); Pseudo \(Beta\)\.\*p<<0\.05;\*\*p<<0\.01;\*\*\*p<<0\.001

Table 17:Robustness checks: Leakage regressions with Clean Accuracy×\\timesDataset or Clean Accuracy×\\timesArchitecture interaction as an alternate explanation for the leakage of generic information dependency\-based defenses\. Leakage \(response\) isPPA AttAcc@1\. Left: Dataset interaction; Right: architecture interaction \(ResNet\-152 vs\. ResNet\-18\)\. SE’s in parentheses; OLS MacKinnon & White \(HC2\) small\-sample robust SE’s in brackets\.*OLS**Beta*\(Intercept\)−1\.055\-1\.055−6\.997∗⁣∗∗\-6\.997^\{\*\*\*\}\(0\.664\)\(0\.664\)\(2\.714\)\(2\.714\)\[0\.412\]Test Acc\.1\.805∗1\.805^\{\*\}8\.188∗∗8\.188^\{\*\*\}\(0\.860\)\(0\.860\)\(3\.469\)\(3\.469\)\[0\.538\]DatasetFaceScrub−3\.983\-3\.983−18\.816∗\-18\.816^\{\*\}\(2\.688\)\(2\.688\)\(10\.302\)\(10\.302\)\[3\.746\]Test Acc\.:DatasetFaceScrub4\.2574\.25720\.047∗20\.047^\{\*\}\(2\.940\)\(2\.940\)\(11\.282\)\(11\.282\)\[4\.042\]R2R^\{2\}0\.4420\.574Mean abs\. err\.0\.1500\.146*OLS**Beta*\(Intercept\)−1\.021\-1\.021−6\.447∗∗\-6\.447^\{\*\*\}\(0\.650\)\(0\.650\)\(2\.527\)\(2\.527\)\[0\.617\]Test Acc\.1\.719∗∗1\.719^\{\*\*\}7\.320∗∗7\.320^\{\*\*\}\(0\.762\)\(0\.762\)\(2\.944\)\(2\.944\)\[0\.785\]ArchRN\-18−0\.051\-0\.051−0\.203\-0\.203\(0\.866\)\(0\.866\)\(3\.395\)\(3\.395\)\[0\.775\]Test Acc\.:ArchRN\-180\.1120\.1120\.4430\.443\(1\.024\)\(1\.024\)\(3\.989\)\(3\.989\)\[0\.988\]R2R^\{2\}0\.3530\.493Mean abs\. err\.0\.1550\.152

Note:R2R^\{2\}are Adj\. \(OLS\); Pseudo \(Beta\)\.\*p<<0\.1;\*\*p<<0\.05;\*\*\*p<<0\.01

Table 18:Linear \(OLS\) & Beta regressions quantifying the privacy–robustness tradeoff forgeneric information dependency\-based privacy defensesunder various specifications\. Leakage \(response\) isIF\-GMI AttAcc@1\. SE’s in parentheses; OLS MacKinnon & White \(HC2\) small\-sample robust SE’s in brackets\. Test accuracy is clean;ϵ\\epsilon’s are robust accuracies under the AutoAttack ensemble at the indicated noise levels\. Note that Beta coefficients are on the logit scale\.All PredictorsAllϵ\\epsilonClean Acc\.ϵ=0\.0025\\epsilon\{=\}0\.0025ϵ=0\.0005\\epsilon\{=\}0\.0005ϵ=10−5\\epsilon\{=\}10^\{\-5\}*OLS**Beta**OLS**Beta**OLS**Beta**OLS**Beta**OLS**Beta**OLS**Beta*PredictorsTest Acc\.\-0\.496\-7\.272∗∗1\.343∗∗7\.146∗∗∗\(0\.827\)\(3\.149\)\(0\.471\)\(2\.102\)\[1\.116\]\[0\.466\]ϵ=0\.0025\\epsilon\{=\}0\.00252\.471∗∗20\.719∗∗∗2\.153∗∗13\.175∗∗∗2\.330∗∗∗13\.590∗∗∗\(0\.960\)\(4\.438\)\(0\.781\)\(3\.688\)\(0\.642\)\(3\.129\)\[1\.766\]\[1\.049\]\[0\.748\]ϵ=0\.0005\\epsilon\{=\}0\.00050\.030\-0\.6860\.056\-0\.0430\.458∗∗∗2\.249∗∗∗\(0\.169\)\(0\.622\)\(0\.160\)\(0\.663\)\(0\.117\)\(0\.555\)\[0\.257\]\[0\.204\]\[0\.108\]ϵ=10−5\\epsilon\{=\}10^\{\-5\}1\.072∗10\.162∗∗∗0\.752∗∗4\.688∗∗∗0\.787∗∗4\.146∗∗∗\(0\.605\)\(2\.598\)\(0\.279\)\(1\.268\)\(0\.349\)\(1\.507\)\[0\.840\]\[0\.221\]\[0\.389\]Fit statisticsR2R^\{2\}0\.6280\.8370\.6450\.8010\.2950\.3610\.4180\.4860\.4600\.4880\.1940\.301Mean abs\. err\.0\.0960\.0820\.0940\.0940\.1430\.1380\.1380\.1290\.1230\.1240\.1670\.166Note:R2R^\{2\}are Adjusted \(OLS\); Pseudo \(Beta\)\.\*p<<0\.1;\*\*p<<0\.05;\*\*\*p<<0\.01

Table 19:Linear \(OLS\) & Beta regressions quantifying privacy–robustness tradeoff forattack gradient suppressing privacy defenses \(NegLS, RoLSS, RoLSS\-SSF\)under various specifications\. Leakage \(response\) isIF\-GMI AttAcc@1\. SE’s in parentheses; OLS MacKinnon & White \(HC2\) small\-sample robust SE’s in brackets\. Test accuracy is clean;ϵ\\epsilon’s are robust accuracies under AutoAttack at the indicated noise levels\. Note that Beta coefficients are on the logit scale\.All PredictorsAllϵ\\epsilonClean Acc\.ϵ=0\.0025\\epsilon\{=\}0\.0025ϵ=0\.0005\\epsilon\{=\}0\.0005ϵ=10−5\\epsilon\{=\}10^\{\-5\}*OLS**Beta**OLS**Beta**OLS**Beta**OLS**Beta**OLS**Beta**OLS**Beta*PredictorsTest Acc\.3\.3266\.8281\.577∗∗6\.567∗∗\(11\.153\)\(37\.227\)\(0\.490\)\(2\.107\)\[4\.541\]\[0\.314\]ϵ=0\.0025\\epsilon\{=\}0\.00251\.1022\.4840\.8702\.0152\.5136\.428\(1\.861\)\(6\.437\)\(1\.591\)\(5\.876\)\(1\.606\)\(5\.551\)\[0\.945\]\[0\.742\]\[1\.186\]ϵ=0\.0005\\epsilon\{=\}0\.00050\.8262\.4831\.1723\.2330\.975∗∗4\.027∗∗\(2\.373\)\(8\.159\)\(1\.948\)\(7\.188\)\(0\.281\)\(1\.254\)\[0\.617\]\[0\.850\]\[0\.146\]ϵ=10−5\\epsilon\{=\}10^\{\-5\}\-3\.149\-4\.382\-0\.4601\.0491\.518∗∗6\.327∗∗\(9\.587\)\(32\.113\)\(3\.059\)\(11\.264\)\(0\.477\)\(2\.040\)\[5\.097\]\[1\.579\]\[0\.314\]Fit statisticsR2R^\{2\}0\.3360\.5950\.4120\.5930\.4590\.5720\.1160\.1370\.5010\.5890\.4530\.569Mean abs\. err\.0\.1570\.1800\.1600\.1830\.1960\.2010\.2490\.2640\.1770\.1880\.1970\.201Note:R2R^\{2\}are Adj\. \(OLS\); Pseudo \(Beta\)\.\*p<<0\.05;\*\*p<<0\.01;\*\*\*p<<0\.001

Table 20:Robustness checks: Leakage regressions with Clean Accuracy×\\timesDataset or Clean Accuracy×\\timesArchitecture interaction as an alternate explanation for the leakage of generic information dependency\-based defenses\. Leakage \(response\) isIF\-GMI AttAcc@1\. Left: Dataset interaction; Right: architecture interaction \(ResNet\-152 vs\. ResNet\-18\)\. SE’s in parentheses; OLS MacKinnon & White \(HC2\) small\-sample robust SE’s in brackets\.*OLS**Beta*\(Intercept\)−1\.333∗\-1\.333^\{\*\}−7\.277∗∗\-7\.277^\{\*\*\}\(0\.635\)\(0\.635\)\(2\.722\)\(2\.722\)\[0\.509\]Test Acc\.2\.520∗∗2\.520^\{\*\*\}9\.987∗⁣∗∗9\.987^\{\*\*\*\}\(0\.822\)\(0\.822\)\(3\.541\)\(3\.541\)\[0\.634\]DatasetFaceScrub−0\.631\-0\.631−13\.955\-13\.955\(2\.570\)\(2\.570\)\(11\.315\)\(11\.315\)\[3\.777\]Test Acc\.:DatasetFaceScrub0\.3820\.38214\.26814\.268\(2\.810\)\(2\.810\)\(12\.406\)\(12\.406\)\[4\.109\]R2R^\{2\}0\.4710\.474Mean abs\. err\.0\.1270\.126*OLS**Beta*\(Intercept\)−0\.322\-0\.322−5\.336∗∗\-5\.336^\{\*\*\}\(0\.634\)\(0\.634\)\(2\.604\)\(2\.604\)\[0\.842\]Test Acc\.1\.1361\.1367\.187∗∗7\.187^\{\*\*\}\(0\.743\)\(0\.743\)\(3\.093\)\(3\.093\)\[1\.028\]ArchRN\-18−0\.300\-0\.3000\.1580\.158\(0\.844\)\(0\.844\)\(3\.436\)\(3\.436\)\[0\.883\]Test Acc\.:ArchRN\-180\.4280\.4280\.0260\.026\(0\.998\)\(0\.998\)\(4\.107\)\(4\.107\)\[1\.088\]R2R^\{2\}0\.3630\.369Mean abs\. err\.0\.1440\.138

Note:R2R^\{2\}are Adj\. \(OLS\); Pseudo \(Beta\)\.\*p<<0\.1;\*\*p<<0\.05;\*\*\*p<<0\.01

## Appendix HDeferred details of randomized controlled experiment: AT\-AT causes privacy

Causal effect of AT\-AT non\-robust feature reward term on privacy\.By design, note thatAT\-ATwithλ=0\\lambda=0simplifies to vanilla training i\.e\. NoDef\. Therefore, to estimate the causal effect of includingAT\-AT’s non\-robust feature reward term \(i\.e\.λ\\lambda\>\>0\) on privacy, we train 10 RN\-152’s on FaceScrub and randomly assign half as ‘control’λ\\lambda=0\(equal to Vanilla/NoDef\) and half as ‘treatment’λ\\lambda\>\>0\. For treatment units, we use the defaultλ=0\.04\\lambda=0\.04used throughout our experiments \(See App[6](https://arxiv.org/html/2607.12354#A2.T6)\)\.

##### Causal OLS \(ATE on the probability scale\)\.

LetYj≡LeakagejY\_\{j\}\\equiv\\mathrm\{Leakage\}\_\{j\}\(AttAcc@1 for modeljj\) andDj≡𝟙​\{λj\>0\}D\_\{j\}\\equiv\\mathbb\{1\}\\\{\\lambda\_\{j\}\>0\\\}indicate assignment to nonzeroλ\\lambda, i\.e\., treatment with theAT\-ATnon\-robust reward in the loss function\. With complete randomization,Dj⟂\(Yj​\(0\),Yj​\(1\)\)D\_\{j\}\\perp\(Y\_\{j\}\(0\),Y\_\{j\}\(1\)\)and the ATE isτ=𝔼​\[Yj​\(1\)−Yj​\(0\)\]\\tau=\\mathbb\{E\}\[Y\_\{j\}\(1\)\-Y\_\{j\}\(0\)\]\. We estimate

Yj=α\+τ​Dj\+uj,𝔼​\[uj∣Dj\]=0,\\displaystyle Y\_\{j\}\\;=\\;\\alpha\\;\+\\;\\tau\\,D\_\{j\}\\;\+\\;u\_\{j\},\\qquad\\mathbb\{E\}\[u\_\{j\}\\mid D\_\{j\}\]=0,\(6\)and report HC2\-robust SEs with small\-samplettcritical values\.

##### Causal Beta regression \(randomized AT\-AT vs\. control\)\.

As before, we note that the response \(AttAcc@1\) is bounded in \[0,1\], thus we also estimate the causal effect via the appropriate beta regression\. LetYj≡LeakagejY\_\{j\}\\equiv\\mathrm\{Leakage\}\_\{j\}\(AttAcc@1 for modeljj\) andDj≡𝟙​\{λj\>0\}D\_\{j\}\\equiv\\mathbb\{1\}\\\{\\lambda\_\{j\}\>0\\\}indicate assignment to AT​\-AT \(treatment\)\. Under complete randomization,Dj⟂\(Yj​\(0\),Yj​\(1\)\)D\_\{j\}\\perp\(Y\_\{j\}\(0\),Y\_\{j\}\(1\)\)\.

Yj\\displaystyle Y\_\{j\}∼Beta​\(μj​ϕ,\(1−μj\)​ϕ\),\\displaystyle\\sim\\mathrm\{Beta\}\(\\mu\_\{j\}\\phi,\\,\(1\-\\mu\_\{j\}\)\\phi\),logit⁡\(μj\)\\displaystyle\\operatorname\{logit\}\(\\mu\_\{j\}\)=α\+τ​Dj\.\\displaystyle=\\alpha\\;\+\\;\\tau\\,D\_\{j\}\.\(7\)
*Odds\-ratio interpretation\.*The treatment effect on the log\-odds scale isτ\\tau; the odds ratio isexp⁡\(τ\)\\exp\(\\tau\)\. A “77×77\\timesreduction in odds” corresponds toexp⁡\(τ\)=1/77\\exp\(\\tau\)=1/77, i\.e\.τ=−ln⁡77\\tau=\-\\ln 77\.

*Percentage\-point effect \(ATE on response scale\)\.*Letμ0≡logit−1⁡\(α\)\\mu\_\{0\}\\equiv\\operatorname\{logit\}^\{\-1\}\(\\alpha\)andμ1≡logit−1⁡\(α\+τ\)\\mu\_\{1\}\\equiv\\operatorname\{logit\}^\{\-1\}\(\\alpha\+\\tau\)\. The average treatment effect on the response scale is

Δ≡𝔼​\[Yj∣Dj=1\]−𝔼​\[Yj∣Dj=0\]=μ1−μ0\(in proportion units; multiply by 100 for pp\)\.\\Delta\\;\\equiv\\;\\mathbb\{E\}\[Y\_\{j\}\\mid D\_\{j\}=1\]\-\\mathbb\{E\}\[Y\_\{j\}\\mid D\_\{j\}=0\]\\;=\\;\\mu\_\{1\}\-\\mu\_\{0\}\\quad\\text\{\(in proportion units; multiply by 100 for pp\)\.\}
*Inference\.*We estimate \([7](https://arxiv.org/html/2607.12354#A8.E7)\) by MLE and report Waldzz\-tests and95%95\\%CIs\. Forτ\\tau, a95%95\\%CI for the odds ratio isexp⁡\(τ^±1\.96​SE​\(τ^\)\)\\exp\\\!\\big\(\\hat\{\\tau\}\\pm 1\.96\\,\\mathrm\{SE\}\(\\hat\{\\tau\}\)\\big\)\. ForΔ\\Delta, use the delta method with gradient\(∂Δ/∂α,∂Δ/∂τ\)=\(μ1​\(1−μ1\)−μ0​\(1−μ0\),μ1​\(1−μ1\)\)\\big\(\\partial\\Delta/\\partial\\alpha,\\partial\\Delta/\\partial\\tau\\big\)=\\big\(\\mu\_\{1\}\(1\-\\mu\_\{1\}\)\-\\mu\_\{0\}\(1\-\\mu\_\{0\}\),\\ \\mu\_\{1\}\(1\-\\mu\_\{1\}\)\\big\):

Var​\(Δ^\)≈g⊤​Var​\(α^τ^\)​g,g=\(μ1​\(1−μ1\)−μ0​\(1−μ0\)μ1​\(1−μ1\)\),\\mathrm\{Var\}\(\\hat\{\\Delta\}\)\\approx g^\{\\top\}\\,\\mathrm\{Var\}\\\!\\begin\{pmatrix\}\\hat\{\\alpha\}\\\\ \\hat\{\\tau\}\\end\{pmatrix\}g,\\quad g=\\begin\{pmatrix\}\\mu\_\{1\}\(1\-\\mu\_\{1\}\)\-\\mu\_\{0\}\(1\-\\mu\_\{0\}\)\\\\\[2\.0pt\] \\mu\_\{1\}\(1\-\\mu\_\{1\}\)\\end\{pmatrix\},and a95%95\\%CI isΔ^±1\.96​SE​\(Δ^\)\\hat\{\\Delta\}\\pm 1\.96\\,\\mathrm\{SE\}\(\\hat\{\\Delta\}\)\.

Causal inference results\.Table[21](https://arxiv.org/html/2607.12354#A8.T21)reports causal inference results\. In particular, beta regression of PPA AttAcc@1 on treatment shows that AT\-AT’s non\-robust term causes a7777×\\timesreduction in leakage odds \(zz==38\.038\.0,pp<<10−1610^\{\-16\}, 6\.5% reconstruction rate underAT\-ATvs\. 84% for control\)\.

Table 21:Causal regressions of leakage \(AttAcc@1\) on AT\-AT assignment\. OLS 95% CIs use HC2 SEs witht0\.975,8=2\.306t\_\{0\.975,8\}=2\.306; Beta uses Wald \(±1\.96\\pm 1\.96SE\) on the logit scale\. Baseline isNoDef; the coefficient onAT\-AT \(Treatment\)is the effect relative to NoDef\. Odds ratio \(AT​\-AT vs NoDef\) from Beta:0\.01300\.0130\(95% CI:0\.01040\.0104–0\.01620\.0162\)\.OLS \(±\\pm95% CI\)Beta \(±\\pm95% CI\)Intercept \(NoDef\)0\.842±\\pm0\.0291\.677±\\pm0\.126AT\-AT \(Treatment\)\-0\.778±\\pm0\.029\-4\.344±\\pm0\.224R2R^\{2\}0\.9976 \(adj\.\)0\.995 \(pseudo\)

We can similarly identify the causal effect ofAT\-ATon L2 reconstruction distance\. Specifically, we recompute the OLS regression replacing the response with our L2 reconstruction distance evaluation metric \(note that there is no need for beta regression for L2 response\)\. We find that treating with non\-robust features viaAT\-AT’s reward increases the L2 reconstruction distance by 0\.430±\\pm0\.017:

Table 22:OLS regression of reconstruction distance \(L2L\_\{2\}\) on AT\-AT assignment\. 95% CIs use HC2 SEs witht0\.975,8=2\.306t\_\{0\.975,8\}=2\.306\. Baseline isNoDef; the coefficient onAT\-AT \(Treatment\)is the effect relative to NoDef\.OLS \(±\\pm95% CI\)Intercept \(NoDef\)0\.786±\\pm0\.012AT\-AT \(Treatment\)0\.430±\\pm0\.017R2R^\{2\}0\.9973 \(adj\.\)

## Appendix IMIA Defense Extended Results

Table[23](https://arxiv.org/html/2607.12354#A9.T23)presents full results and privacy metrics for AT\-AT and all baselines run on FaceScrub and CelebA using the standard ResNet\-152 architecture\. These results are an extended tabular version of Fig\.[4](https://arxiv.org/html/2607.12354#S8.F4)\.

Table 23:Full privacy experiments results: AT\-AT vs Baselines on standardResNet\-152architectures on CelebA and FaceScrub data under PPA, IF\-GMI, and PPDG attacks\.DataAttackDefenseTestAcc↑\\uparrowAttAcc@1↓\\downarrowAttAcc@5↓\\downarrowL2↑\\uparrowEvalConf↓\\downarrowDefenses that attempt to increase privacy genericallyFaceScrubPPANoDef0\.9580\.9580\.8820\.8820\.9800\.9800\.7680\.7680\.8620\.862MID0\.9500\.9500\.3910\.3910\.6990\.6990\.9500\.9500\.3640\.364BiDO0\.9280\.9280\.7800\.7800\.9490\.9490\.8200\.8200\.7560\.756TL\-DMI0\.9110\.9110\.1900\.1900\.4350\.4351\.0571\.0570\.1780\.178AT\-AT0\.9390\.0580\.1901\.2240\.055IF\-GMINoDef0\.9580\.9580\.9870\.9870\.9980\.9980\.7000\.7000\.9840\.984MID0\.9500\.9500\.3910\.3910\.6680\.6681\.0021\.0020\.3650\.365BiDO0\.9280\.9280\.9370\.9370\.9880\.9880\.7920\.7920\.9270\.927TL\-DMI0\.9110\.9110\.4200\.4200\.6940\.6940\.9460\.9460\.4010\.401AT\-AT0\.9390\.0840\.2381\.2160\.080PPDGNoDef0\.9580\.9580\.9020\.9020\.9790\.9790\.8050\.8050\.8900\.890MID0\.9500\.9500\.3430\.3430\.5830\.5831\.0561\.0560\.3150\.315BiDO0\.9280\.9280\.7990\.7990\.9370\.9370\.8850\.8850\.7770\.777TL\-DMI0\.9110\.9110\.1420\.1420\.3380\.3381\.1251\.1250\.1320\.132AT\-AT0\.9390\.0220\.0841\.3530\.021CelebAPPANoDef0\.8380\.8380\.4970\.4970\.7530\.7530\.8430\.8430\.4730\.473MID0\.8020\.8020\.4740\.4740\.7250\.7250\.8210\.8210\.4500\.450BiDO0\.7470\.7470\.3620\.3620\.6290\.6290\.9230\.9230\.3400\.340TL\-DMI0\.6880\.6880\.0800\.0800\.2210\.2211\.1031\.1030\.0750\.075TL\-DMI\-50\.8140\.8140\.2810\.2810\.5500\.5500\.9290\.9290\.2650\.265AT\-AT0\.8190\.0490\.1381\.2450\.044IF\-GMINoDef0\.8380\.8380\.7930\.7930\.9400\.9400\.7700\.7700\.7730\.773MID0\.8020\.8020\.7280\.7280\.8640\.8640\.8360\.8360\.7090\.709BiDO0\.7470\.7470\.5970\.5970\.8340\.8340\.8670\.8670\.5750\.575TL\-DMI0\.6880\.6880\.2580\.2580\.5130\.5130\.9640\.9640\.2430\.243TL\-DMI\-50\.8140\.8140\.6610\.6610\.8830\.8830\.8050\.8050\.6410\.641AT\-AT0\.8190\.1880\.3771\.1400\.181PPDGNoDef0\.8380\.8380\.5590\.5590\.7960\.7960\.8630\.8630\.5360\.536MID0\.8020\.8020\.4910\.4910\.6880\.6880\.9280\.9280\.4610\.461BiDO0\.7470\.7470\.3890\.3890\.6470\.6470\.9560\.9560\.3680\.368TL\-DMI0\.6880\.6880\.0790\.0790\.2060\.2061\.1571\.1570\.0760\.076TL\-DMI\-50\.8140\.8140\.3190\.3190\.5850\.5850\.9590\.9590\.3030\.303AT\-AT0\.8190\.0230\.0711\.4300\.021Defenses that suppress attack gradientsFaceScrubPPANegLS0\.9060\.9060\.1600\.1600\.3650\.3651\.1511\.1510\.1520\.152RoLSS0\.8860\.8860\.4740\.4740\.7600\.7600\.8580\.8580\.4500\.450RoLSS\-SSF0\.8690\.8690\.3430\.3430\.6350\.6350\.9180\.9180\.3220\.322Trap\-MID\*0\.9490\.9490\.5870\.5870\.8300\.8300\.9200\.9200\.5560\.556Trap\-MID0\.9260\.9260\.4010\.4010\.6760\.6760\.9800\.9800\.3760\.376IF\-GMINegLS0\.9060\.9060\.1470\.1470\.3520\.3521\.1991\.1990\.1420\.142RoLSS0\.8860\.8860\.5220\.5220\.7850\.7850\.8840\.8840\.4990\.499RoLSS\-SSF0\.8690\.8690\.3730\.3730\.6570\.6570\.9560\.9560\.3540\.354Trap\-MID\*0\.9490\.9490\.7140\.7140\.7920\.7920\.9390\.9390\.7050\.705Trap\-MID0\.9260\.9260\.6180\.6180\.7380\.7381\.0041\.0040\.6030\.603PPDGNegLS0\.9060\.9060\.1050\.1050\.2450\.2451\.2471\.2470\.1010\.101RoLSS0\.8860\.8860\.4560\.4560\.7420\.7420\.9100\.9100\.4320\.432RoLSS\-SSF0\.8690\.8690\.2940\.2940\.5580\.5580\.9990\.9990\.2720\.272Trap\-MID\*0\.9490\.9490\.6340\.6340\.8130\.8130\.9580\.9580\.6160\.616Trap\-MID0\.9260\.9260\.4250\.4250\.6650\.6651\.0391\.0390\.4080\.408CelebAPPANegLS0\.8050\.8050\.3400\.3400\.6230\.6230\.9080\.9080\.3240\.324RoLSS0\.5370\.5370\.0460\.0460\.1450\.1451\.1471\.1470\.0430\.043RoLSS\-SSF0\.4280\.4280\.0390\.0390\.1180\.1181\.1831\.1830\.0350\.035Trap\-MID\*0\.8060\.8060\.0920\.0920\.1930\.1931\.3581\.3580\.0880\.088Trap\-MID0\.8470\.8470\.3300\.3300\.5690\.5690\.9440\.9440\.3090\.309IF\-GMINegLS0\.8050\.8050\.5540\.5540\.8110\.8110\.8850\.8850\.5320\.532RoLSS0\.5370\.5370\.0450\.0450\.1450\.1451\.1721\.1720\.0420\.042RoLSS\-SSF0\.4280\.4280\.0440\.0440\.1310\.1311\.2181\.2180\.0400\.040Trap\-MID\*0\.8060\.8060\.1680\.1680\.2230\.2231\.4371\.4370\.1630\.163Trap\-MID0\.8470\.8470\.4170\.4170\.6010\.6011\.0041\.0040\.4030\.403PPDGNegLS0\.8050\.8050\.3310\.3310\.5800\.5800\.9550\.9550\.3140\.314RoLSS0\.5370\.5370\.0380\.0380\.1100\.1101\.2661\.2660\.0350\.035RoLSS\-SSF0\.4280\.4280\.0290\.0290\.0930\.0931\.2951\.2950\.0270\.027Trap\-MID\*0\.8060\.8060\.0740\.0740\.1480\.1481\.4551\.4550\.0720\.072Trap\-MID0\.8470\.8470\.3310\.3310\.5560\.5561\.0351\.0350\.3150\.315

Table 24:Full privacy experiments results: AT\-AT vs Baselines on standardDenseNet\-169architectures on CelebA and FaceScrub data under PPA, IF\-GMI, and PPDG attacks\.DataAttackDefenseTestAcc↑\\uparrowAttAcc@1↓\\downarrowAttAcc@5↓\\downarrowL2↑\\uparrowEvalConf↓\\downarrowDefenses that attempt to increase privacy genericallyFaceScrubPPANoDef0\.9470\.9470\.8900\.8900\.9800\.9800\.7630\.7630\.8680\.868MID0\.9490\.9490\.5950\.5950\.8350\.8350\.8690\.8690\.5610\.561BiDO0\.9210\.9210\.6930\.6930\.8910\.8910\.8310\.8310\.6670\.667TL\-DMI0\.9210\.9210\.2940\.2940\.5750\.5750\.9960\.9960\.2780\.278AT\-AT0\.9330\.0580\.1901\.1940\.057IF\-GMINoDef0\.9470\.9470\.9830\.9830\.9980\.9980\.7220\.7220\.9790\.979MID0\.9490\.9490\.6210\.6210\.8080\.8080\.9510\.9510\.5870\.587BiDO0\.9210\.9210\.8720\.8720\.9670\.9670\.8260\.8260\.8550\.855TL\-DMI0\.9210\.9210\.6270\.6270\.8450\.8450\.8850\.8850\.6060\.606AT\-AT0\.9330\.1240\.3031\.1570\.116PPDGNoDef0\.9470\.9470\.9150\.9150\.9840\.9840\.7870\.7870\.9030\.903MID0\.9490\.9490\.5370\.5370\.7490\.7490\.9700\.9700\.5100\.510BiDO0\.9210\.9210\.7470\.7470\.9150\.9150\.8560\.8560\.7240\.724TL\-DMI0\.9210\.9210\.3290\.3290\.5860\.5861\.0171\.0170\.3110\.311AT\-AT0\.9330\.0370\.1181\.3050\.035CelebAPPANoDef0\.8560\.8560\.5030\.5030\.7880\.7880\.8000\.8000\.4800\.480MID0\.7300\.7300\.4190\.4190\.6750\.6750\.8360\.8360\.3940\.394BiDO0\.6110\.6110\.1320\.1320\.3130\.3131\.0211\.0210\.1230\.123TL\-DMI0\.7880\.7880\.1510\.1510\.3610\.3611\.0001\.0000\.1450\.145AT\-AT0\.8360\.0860\.2051\.1770\.078IF\-GMINoDef0\.8560\.8560\.8200\.8200\.9430\.9430\.7550\.7550\.8040\.804MID0\.7300\.7300\.5770\.5770\.7310\.7310\.8820\.8820\.5570\.557BiDO0\.6110\.6110\.1990\.1990\.4090\.4091\.0051\.0050\.1920\.192TL\-DMI0\.7880\.7880\.4740\.4740\.7350\.7350\.8610\.8610\.4530\.453AT\-AT0\.8360\.3890\.5990\.9900\.369PPDGNoDef0\.8560\.8560\.5940\.5940\.8260\.8260\.8190\.8190\.5700\.570MID0\.7300\.7300\.4020\.4020\.6160\.6160\.9520\.9520\.3800\.380BiDO0\.6110\.6110\.1420\.1420\.3110\.3111\.0621\.0620\.1350\.135TL\-DMI0\.7880\.7880\.1450\.1450\.3330\.3331\.0681\.0680\.1370\.137AT\-AT0\.8360\.0410\.1011\.3590\.039Defenses that suppress attack gradientsFaceScrubPPANegLS0\.9240\.9240\.2080\.2080\.4500\.4501\.1071\.1070\.2020\.202RoLSS0\.7320\.7320\.0980\.0980\.2670\.2671\.1161\.1160\.0930\.093RoLSS\-SSF0\.8860\.8860\.4620\.4620\.7610\.7610\.8820\.8820\.4380\.438Trap\-MID0\.9200\.9200\.5290\.5290\.7870\.7870\.8930\.8930\.5000\.500Trap\-MID\*0\.9460\.9460\.7080\.7080\.9120\.9120\.8330\.8330\.6800\.680IF\-GMINegLS0\.9240\.9240\.4480\.4480\.7140\.7141\.0471\.0470\.4330\.433RoLSS0\.7320\.7320\.0960\.0960\.2530\.2531\.1051\.1050\.0880\.088RoLSS\-SSF0\.8860\.8860\.5510\.5510\.8230\.8230\.8770\.8770\.5310\.531Trap\-MID0\.9200\.9200\.7070\.7070\.8370\.8370\.9020\.9020\.6890\.689Trap\-MID\*0\.9460\.9460\.9030\.9030\.9390\.9390\.7780\.7780\.8940\.894PPDGNegLS0\.9240\.9240\.2390\.2390\.4780\.4781\.1451\.1450\.2300\.230RoLSS0\.7320\.7320\.0690\.0690\.1970\.1971\.1601\.1600\.0650\.065RoLSS\-SSF0\.8860\.8860\.4860\.4860\.7460\.7460\.9080\.9080\.4650\.465Trap\-MID0\.9200\.9200\.5750\.5750\.7840\.7840\.9280\.9280\.5520\.552Trap\-MID\*0\.9460\.9460\.7830\.7830\.9170\.9170\.8360\.8360\.7640\.764CelebAPPANegLS0\.8200\.8200\.5280\.5280\.8000\.8000\.7810\.7810\.5070\.507RoLSS0\.2750\.2750\.0340\.0340\.1260\.1261\.1491\.1490\.0340\.034RoLSS\-SSF0\.5570\.5570\.1030\.1030\.2850\.2851\.0071\.0070\.1000\.100Trap\-MID0\.6810\.6810\.1950\.1950\.4040\.4040\.9760\.9760\.1850\.185Trap\-MID\*0\.7110\.7110\.1980\.1980\.4430\.4430\.9570\.9570\.1880\.188IF\-GMINegLS0\.8200\.8200\.8190\.8190\.9380\.9380\.7190\.7190\.8000\.800RoLSS0\.2750\.2750\.0380\.0380\.1210\.1211\.1611\.1610\.0350\.035RoLSS\-SSF0\.5570\.5570\.1110\.1110\.3230\.3231\.0051\.0050\.1080\.108Trap\-MID0\.6810\.6810\.2640\.2640\.4610\.4611\.0441\.0440\.2500\.250Trap\-MID\*0\.7110\.7110\.4120\.4120\.6500\.6500\.9420\.9420\.3930\.393PPDGNegLS0\.8200\.8200\.5870\.5870\.8210\.8210\.7930\.7930\.5630\.563RoLSS0\.2750\.2750\.0270\.0270\.1020\.1021\.1931\.1930\.0250\.025RoLSS\-SSF0\.5570\.5570\.0910\.0910\.2590\.2591\.0401\.0400\.0870\.087Trap\-MID0\.6810\.6810\.1640\.1640\.3580\.3581\.0791\.0790\.1570\.157Trap\-MID\*0\.7110\.7110\.2240\.2240\.4530\.4531\.0321\.0320\.2110\.211

Table 25:Additional privacy experiments results: Lower\-capacityResNet\-18architectures on CelebA and FaceScrub data under PPA and IF\-GMI attacks\.DataAttackDefenseTestAcc↑\\uparrowAttAcc@1↓\\downarrowAttAcc@5↓\\downarrowL2↑\\uparrowEvalConf↓\\downarrowDefenses that attempt to increase privacy genericallyFaceScrubPPANoDef0\.9500\.9500\.9010\.9010\.9860\.9860\.7610\.7610\.8810\.881MID0\.9240\.9240\.7140\.7140\.9030\.9030\.8150\.8150\.6880\.688BiDO0\.8840\.8840\.5300\.5300\.8110\.8110\.9150\.9150\.5040\.504TL\-DMI0\.9060\.9060\.2400\.2400\.5050\.5051\.0201\.0200\.2250\.225AT\-AT0\.9210\.1100\.2971\.1430\.104IF\-GMINoDef0\.9500\.9500\.9830\.9830\.9960\.9960\.7400\.7400\.9790\.979MID0\.9240\.9240\.7760\.7760\.8400\.8400\.8990\.8990\.7640\.764BiDO0\.8840\.8840\.7820\.7820\.9250\.9250\.8870\.8870\.7640\.764TL\-DMI0\.9060\.9060\.5150\.5150\.7600\.7600\.9350\.9350\.4920\.492AT\-AT0\.9210\.2830\.5201\.0820\.268CelebAPPANoDef0\.8590\.8590\.6340\.6340\.8590\.8590\.7740\.7740\.6080\.608MID0\.7370\.7370\.3860\.3860\.6710\.6710\.8380\.8380\.3620\.362BiDO0\.6250\.6250\.1460\.1460\.3590\.3591\.0091\.0090\.1360\.136TL\-DMI0\.7520\.7520\.1530\.1530\.3390\.3391\.0131\.0130\.1420\.142TL\-DMI\-50\.8310\.8310\.3240\.3240\.6020\.6020\.8860\.8860\.3020\.302AT\-AT†a0\.8250\.2960\.5610\.9450\.279AT\-AT†b0\.7870\.1550\.3501\.0290\.144IF\-GMINoDef0\.8590\.8590\.9130\.9130\.9700\.9700\.7080\.7080\.9030\.903MID0\.7370\.7370\.5500\.5500\.7400\.7400\.8890\.8890\.5290\.529BiDO0\.6250\.6250\.3680\.3680\.6290\.6290\.9080\.9080\.3470\.347TL\-DMI0\.7520\.7520\.4390\.4390\.7100\.7100\.8720\.8720\.4210\.421TL\-DMI\-50\.8310\.8310\.7500\.7500\.9220\.9220\.7510\.7510\.7280\.728AT\-AT†a0\.8250\.7830\.9120\.7990\.764AT\-AT†b0\.7870\.6170\.8060\.8560\.599Defenses that suppress attack gradientsFaceScrubPPANegLS0\.9290\.9290\.6490\.6490\.8880\.8880\.8760\.8760\.6240\.624RoLSS0\.9500\.9500\.8580\.8580\.9730\.9730\.7460\.7460\.8370\.837RoLSS\-SSF0\.9500\.9500\.8490\.8490\.9730\.9730\.7510\.7510\.8280\.828Trap\-MID\*0\.9270\.9270\.5020\.5020\.7520\.7520\.9490\.9490\.4750\.475Trap\-MID0\.9280\.9280\.7350\.7350\.9300\.9300\.8310\.8310\.7040\.704IF\-GMINegLS0\.9290\.9290\.9320\.9320\.9890\.9890\.8240\.8240\.9230\.923RoLSS0\.9500\.9500\.9790\.9790\.9960\.9960\.6830\.6830\.9750\.975RoLSS\-SSF0\.9500\.9500\.9730\.9730\.9920\.9920\.6990\.6990\.9690\.969Trap\-MID\*0\.9270\.9270\.6870\.6870\.7700\.7700\.9600\.9600\.6730\.673Trap\-MID0\.9280\.9280\.9180\.9180\.9700\.9700\.7900\.7900\.9100\.910CelebAPPANegLS0\.8380\.8380\.6670\.6670\.8870\.8870\.7300\.7300\.6400\.640RoLSS0\.8300\.8300\.5060\.5060\.7690\.7690\.8090\.8090\.4740\.474RoLSS\-SSF0\.8390\.8390\.5190\.5190\.7640\.7640\.8100\.8100\.4900\.490Trap\-MID\*0\.8110\.8110\.0190\.0190\.0310\.0311\.5711\.5710\.0180\.018Trap\-MID0\.8300\.8300\.4430\.4430\.7000\.7000\.8540\.8540\.4230\.423IF\-GMINegLS0\.8380\.8380\.9530\.9530\.9930\.9930\.6490\.6490\.9440\.944RoLSS0\.8300\.8300\.7240\.7240\.9040\.9040\.7780\.7780\.7050\.705RoLSS\-SSF0\.8390\.8390\.7650\.7650\.9090\.9090\.7750\.7750\.7420\.742Trap\-MID\*0\.8110\.8110\.0880\.0880\.1340\.1341\.4821\.4820\.0840\.084Trap\-MID0\.8300\.8300\.6700\.6700\.7970\.7970\.9110\.9110\.6470\.647

*Notes:*†aand†bdenote two sets of hyperparameters for AT\-AT \(see Table[6](https://arxiv.org/html/2607.12354#A2.T6)\)\.

Table 26:Additional privacy experiments results: PPA attack on Stanford Dogs data using the pre\-traineddogStyleGAN\-2\.DataAttackDefenseTestAcc↑\\uparrowAttAcc@1↓\\downarrowAttAcc@5↓\\downarrowL2↑\\uparrowEvalConf↓\\downarrowDefenses that attempt to increase privacy genericallyStanford DogsPPANoDef0\.8670\.8670\.8200\.8200\.9570\.957106\.6106\.60\.7080\.708MID0\.8200\.8200\.4840\.4840\.7590\.759241\.3241\.30\.3810\.381BiDO0\.8170\.8170\.7170\.7170\.9160\.916139\.8139\.80\.7270\.727TL\-DMI0\.8520\.8520\.7540\.7540\.9300\.930127\.3127\.30\.6050\.605AT\-AT0\.8140\.8140\.3860\.632275\.80\.235Defenses that suppress attack gradientsPPANegLS0\.8160\.8160\.6250\.6250\.8580\.858181\.7181\.70\.5320\.532RoLSS0\.6840\.6840\.5510\.5510\.8200\.820198\.7198\.70\.7120\.712RoLSS\-SSF0\.6730\.6730\.4810\.4810\.7890\.789212\.5212\.50\.4150\.415

\(a\)ResNet\-152DataAttackDefenseTestAcc↑\\uparrowAttAcc@1↓\\downarrowAttAcc@5↓\\downarrowL2↑\\uparrowEvalConf↓\\downarrowDefenses that attempt to increase privacy genericallyStanford DogsPPANoDef0\.7760\.7760\.8230\.8230\.9490\.949105\.7105\.70\.7590\.759MID0\.6750\.6750\.6800\.6800\.8500\.850167\.0167\.00\.6150\.615BiDO0\.6600\.6600\.6260\.6260\.8450\.845176\.1176\.10\.5630\.563TL\-DMI0\.7770\.7770\.7740\.7740\.9330\.933118\.1118\.10\.7090\.709AT\-AT0\.7110\.7110\.4680\.724238\.70\.415Defenses that suppress attack gradientsPPANegLS0\.7780\.7780\.8280\.8280\.9530\.953113\.2113\.20\.7450\.745RoLSS0\.7450\.7450\.7930\.7930\.9390\.939113\.8113\.80\.7300\.730RoLSS\-SSF0\.7530\.7530\.7860\.7860\.9350\.935119\.2119\.20\.7210\.721

\(b\)ResNet\-18
### I\.1Qualitative comparisons

Fig\.[6](https://arxiv.org/html/2607.12354#A9.F6)shows examples of original images side\-by\-side with their respective PPA attack reconstructions under each defense\. To avoid cherry\-picking images to include in this comparison, we display the reconstructions for which the evaluation model is maximally confident that the reconstruction class matches the original class\.

ReferenceNo DefenseMIDBiDOTL\-DMIRoLSSRoLSS\-SSFNegLSTrap\-MIDAT\-AT![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/AaronEckhart_reference.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/AaronEckhart_R152_NoDef.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/AaronEckhart_R152_MID.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/AaronEckhart_R152_BiDO.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/AaronEckhart_R152_TL.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/AaronEckhart_R152_RoLSS.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/AaronEckhart_R152_RoLSS-SSF.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/AaronEckhart_R152_NegLS.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/AaronEckhart_R152_trap.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/AaronEckhart_R152_AT-AT.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/AdamBrody_reference.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/AdamBrody_R152_NoDef.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/AdamBrody_R152_MID.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/AdamBrody_R152_BiDO.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/AdamBrody_R152_TL.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/AdamBrody_R152_RoLSS.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/AdamBrody_R152_RoLSS-SSF.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/AdamBrody_R152_NegLS.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/AdamBrody_R152_Trap.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/AdamBrody_R152_AT-AT.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/AdamMcKay_reference.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/AdamMcKay_R152_NoDef.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/AdamMcKay_R152_MID.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/AdamMcKay_R152_BiDO.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/AdamMcKay_R152_TL.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/AdamMcKay_R152_RoLSS.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/AdamMcKay_R152_RoLSS-SSF.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/AdamMcKay_R152_NegLS.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/AdamMcKay_R152_trap.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/AdamMcKay_R152_AT-AT.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/AdamSandler_reference.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/AdamSandler_R152_NoDef.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/AdamSandler_R152_MID.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/AdamSandler_R152_BiDO.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/AdamSandler_R152_TL.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/AdamSandler_R152_RoLSS.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/AdamSandler_R152_RoLSS-SSF.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/AdamSandler_R152_NegLS.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/AdamSandler_R152_trap.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/AdamSandler_R152_AT-AT.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/AdrienBrody_reference.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/AdrienBrody_R152_NoDef.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/AdrienBrody_R152_MID.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/AdrienBrody_R152_BiDO.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/AdrienBrody_R152_TL.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/AdrienBrody_R152_RoLSS.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/AdrienBrody_R152_RoLSS-SSF.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/AdrienBrody_R152_NegLS.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/AdrienBrody_R152_trap.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/AdrienBrody_R152_AT-AT.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/VictoriaJustice_reference.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/VictoriaJustice_R152_NoDef.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/VictoriaJustice_R152_MID.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/VictoriaJustice_R152_BiDO.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/VictoriaJustice_R152_TL.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/VictoriaJustice_R152_RoLSS.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/VictoriaJustice_R152_RoLSS-SSF.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/VictoriaJustice_R152_NegLS.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/VictoriaJustice_R152_trap.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/VictoriaJustice_R152_AT-AT.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/WandaDeJesus_reference.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/WandaDeJesus_R152_NoDef.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/WandaDeJesus_R152_MID.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/WandaDeJesus_R152_BiDO.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/WandaDeJesus_R152_TL.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/WandaDeJesus_R152_RoLSS.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/WandaDeJesus_R152_RoLSS-SSF.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/WandaDeJesus_R152_NegLS.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/WandaDeJesus_R152_trap.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/WandaDeJesus_R152_AT-AT.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/WendieMalick_reference.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/WendieMalick_R152_NoDef.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/WendieMalick_R152_MID.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/WendieMalick_R152_BiDO.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/WendieMalick_R152_TL.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/WendieMalick_R152_RoLSS.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/WendieMalick_R152_RoLSS-SSF.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/WendieMalick_R152_NegLS.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/WendieMalick_R152_trap.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/WendieMalick_R152_AT-AT.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/YasmineBleeth_reference.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/YasmineBleeth_R152_NoDef.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/YasmineBleeth_R152_MID.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/YasmineBleeth_R152_BiDO.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/YasmineBleeth_R152_TL.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/YasmineBleeth_R152_RoLSS.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/YasmineBleeth_R152_RoLSS-SSF.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/YasmineBleeth_R152_NegLS.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/YasmineBleeth_R152_trap.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/YasmineBleeth_R152_AT-AT.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/ZooeyDeschanel_reference.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/ZooeyDeschanel_R152_NoDef.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/ZooeyDeschanel_R152_MID.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/ZooeyDeschanel_R152_BiDO.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/ZooeyDeschanel_R152_TL.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/ZooeyDeschanel_R152_RoLSS.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/ZooeyDeschanel_R152_RoLSS-SSF.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/ZooeyDeschanel_R152_NegLS.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/ZooeyDeschanel_R152_trap.png)![Refer to caption](https://arxiv.org/html/2607.12354v1/dareDEVALII/Figures/ModelInversionGrid-protocol-R152/ZooeyDeschanel_R152_AT-AT.png)

Figure 6:Image reconstructions generated from PPA attacks for each defense applied on a ResNet\-152 trained on FaceScrub\. To avoid cherry\-picking, each image is selected by maximum confidence on the evaluation model \(worst case\)\.
### I\.2Other Architectures and Datasets

Table[24](https://arxiv.org/html/2607.12354#A9.T24)presents full results and privacy metrics for AT\-AT and all baselines run on FaceScrub and CelebA using the DenseNet\-169 architecture\. Table[25](https://arxiv.org/html/2607.12354#A9.T25)presents the results of the same experiments rerun using the lower\-capacity ResNet\-18 architecture\.

Table[26\(a\)](https://arxiv.org/html/2607.12354#A9.T26.st1)presents PPA attack results and privacy metrics for AT\-AT and all baselines using the standard ResNet\-152 architecture on the Stanford Dogs dataset\. Table[26\(b\)](https://arxiv.org/html/2607.12354#A9.T26.st2)presents the results of the same experiments for the lower\-capacity ResNet\-18 architecture\.

### I\.3Tables Including Confidence Intervals

We note that the main results tables, Tables[23](https://arxiv.org/html/2607.12354#A9.T23)and[25](https://arxiv.org/html/2607.12354#A9.T25), omitted confidence intervals for readability\. We therefore include these confidence intervals separately below\. Specifically, Tables[27](https://arxiv.org/html/2607.12354#A10.T27)and[28](https://arxiv.org/html/2607.12354#A10.T28)are expanded versions of the main MIA results \(Tables[23](https://arxiv.org/html/2607.12354#A9.T23)and[25](https://arxiv.org/html/2607.12354#A9.T25)\) that add the deferred per\-class or per\-image confidence intervals\. More specifically, for the AttAcc@1/AttAcc@5 metrics, we run a per\-image 95% Wilson confidence interval with the proportionp=p=AttAcc@1/AttAcc@5, and the sample sizennequal to the total \# of images on which attack accuracy is calculated \(for FaceScrub,n=26500n=26500, and for CelebA,n=3750n=3750\)\. For the L2 distance metric, as it is a mean over attack target classes, we run a 95% per\-classtt\-interval on the list of L2 metric values for each attack target class \(of length 530 for FaceScrub, and length 75 for CelebA\)\.

## Appendix JRequired LLM disclosure

LLMs were used in the creation of this paper to \(1\) read for typos, and \(2\) to suggest LaTeX formatting fixes \(table spacing, algorithm watermark, etc\.\)\. LLMs were not used to write the paper, code the experiments, or for research ideation\.

Table 27:Full privacy experiments results: AT\-AT vs Baselines onResNet\-152on CelebA and FaceScrub under PPA and IF\-GMI\. Entries show mean±\\pm95% CI half\-width for non\-confidence metrics\.DataAttackDefenseTestAcc↑\\uparrowAttAcc@1↓\\downarrowAttAcc@5↓\\downarrowL2↑\\uparrowEvalConf↓\\downarrowDefenses that attempt to increase privacy genericallyFaceScrubPPANoDef0\.9580\.9580\.882±±0\.0040\.882\\,\\pm\\,\\pm 0\.0040\.980±±0\.0020\.980\\,\\pm\\,\\pm 0\.0020\.768±±0\.0100\.768\\,\\pm\\,\\pm 0\.0100\.8620\.862MID0\.9500\.9500\.391±±0\.0060\.391\\,\\pm\\,\\pm 0\.0060\.699±±0\.0060\.699\\,\\pm\\,\\pm 0\.0060\.950±±0\.0100\.950\\,\\pm\\,\\pm 0\.0100\.3640\.364BiDO0\.9280\.9280\.780±±0\.0050\.780\\,\\pm\\,\\pm 0\.0050\.949±±0\.0030\.949\\,\\pm\\,\\pm 0\.0030\.820±±0\.0100\.820\\,\\pm\\,\\pm 0\.0100\.7560\.756TL\-DMI0\.9110\.9110\.190±±0\.0050\.190\\,\\pm\\,\\pm 0\.0050\.435±±0\.0060\.435\\,\\pm\\,\\pm 0\.0061\.057±±0\.0111\.057\\,\\pm\\,\\pm 0\.0110\.1780\.178AT\-AT0\.9390\.9390\.058±±0\.0030\.058\\,\\pm\\,\\pm 0\.0030\.190±±0\.0050\.190\\,\\pm\\,\\pm 0\.0051\.225±±0\.0131\.225\\,\\pm\\,\\pm 0\.0130\.0550\.055IF\-GMINoDef0\.9580\.9580\.987±±0\.0010\.987\\,\\pm\\,\\pm 0\.0010\.998±±0\.0010\.998\\,\\pm\\,\\pm 0\.0010\.700±±0\.0090\.700\\,\\pm\\,\\pm 0\.0090\.9840\.984MID0\.9500\.9500\.391±±0\.0060\.391\\,\\pm\\,\\pm 0\.0060\.668±±0\.0060\.668\\,\\pm\\,\\pm 0\.0061\.002±±0\.0101\.002\\,\\pm\\,\\pm 0\.0100\.3650\.365BiDO0\.9280\.9280\.937±±0\.0030\.937\\,\\pm\\,\\pm 0\.0030\.988±±0\.0010\.988\\,\\pm\\,\\pm 0\.0010\.792±±0\.0090\.792\\,\\pm\\,\\pm 0\.0090\.9270\.927TL\-DMI0\.9110\.9110\.420±±0\.0060\.420\\,\\pm\\,\\pm 0\.0060\.694±±0\.0060\.694\\,\\pm\\,\\pm 0\.0060\.946±±0\.0100\.946\\,\\pm\\,\\pm 0\.0100\.4010\.401AT\-AT0\.9390\.9390\.084±±0\.0030\.084\\,\\pm\\,\\pm 0\.0030\.238±±0\.0050\.238\\,\\pm\\,\\pm 0\.0051\.216±±0\.0131\.216\\,\\pm\\,\\pm 0\.0130\.0800\.080CelebAPPANoDef0\.8380\.8380\.497±±0\.0160\.497\\,\\pm\\,\\pm 0\.0160\.753±±0\.0140\.753\\,\\pm\\,\\pm 0\.0140\.843±±0\.0350\.843\\,\\pm\\,\\pm 0\.0350\.4730\.473MID0\.8020\.8020\.474±±0\.0160\.474\\,\\pm\\,\\pm 0\.0160\.725±±0\.0140\.725\\,\\pm\\,\\pm 0\.0140\.821±±0\.0350\.821\\,\\pm\\,\\pm 0\.0350\.4500\.450BiDO0\.7470\.7470\.362±±0\.0150\.362\\,\\pm\\,\\pm 0\.0150\.629±±0\.0150\.629\\,\\pm\\,\\pm 0\.0150\.923±±0\.0370\.923\\,\\pm\\,\\pm 0\.0370\.3400\.340TL\-DMI0\.6880\.6880\.080±±0\.0090\.080\\,\\pm\\,\\pm 0\.0090\.221±±0\.0130\.221\\,\\pm\\,\\pm 0\.0131\.103±±0\.0371\.103\\,\\pm\\,\\pm 0\.0370\.0750\.075TL\-DMI\-50\.8140\.8140\.281±±0\.0140\.281\\,\\pm\\,\\pm 0\.0140\.550±±0\.0160\.550\\,\\pm\\,\\pm 0\.0160\.929±±0\.0350\.929\\,\\pm\\,\\pm 0\.0350\.2650\.265AT\-AT0\.8190\.8190\.049±±0\.0070\.049\\,\\pm\\,\\pm 0\.0070\.138±±0\.0110\.138\\,\\pm\\,\\pm 0\.0111\.245±±0\.0471\.245\\,\\pm\\,\\pm 0\.0470\.0440\.044IF\-GMINoDef0\.8380\.8380\.793±±0\.0130\.793\\,\\pm\\,\\pm 0\.0130\.940±±0\.0080\.940\\,\\pm\\,\\pm 0\.0080\.770±±0\.0340\.770\\,\\pm\\,\\pm 0\.0340\.7730\.773MID0\.8020\.8020\.728±±0\.0140\.728\\,\\pm\\,\\pm 0\.0140\.864±±0\.0110\.864\\,\\pm\\,\\pm 0\.0110\.836±±0\.0370\.836\\,\\pm\\,\\pm 0\.0370\.7090\.709BiDO0\.7470\.7470\.597±±0\.0160\.597\\,\\pm\\,\\pm 0\.0160\.834±±0\.0120\.834\\,\\pm\\,\\pm 0\.0120\.867±±0\.0360\.867\\,\\pm\\,\\pm 0\.0360\.5750\.575TL\-DMI0\.6880\.6880\.258±±0\.0140\.258\\,\\pm\\,\\pm 0\.0140\.513±±0\.0160\.513\\,\\pm\\,\\pm 0\.0160\.964±±0\.0370\.964\\,\\pm\\,\\pm 0\.0370\.2430\.243TL\-DMI\-50\.8140\.8140\.661±±0\.0150\.661\\,\\pm\\,\\pm 0\.0150\.883±±0\.0100\.883\\,\\pm\\,\\pm 0\.0100\.805±±0\.0330\.805\\,\\pm\\,\\pm 0\.0330\.6410\.641AT\-AT0\.8190\.8190\.188±±0\.0130\.188\\,\\pm\\,\\pm 0\.0130\.377±±0\.0160\.377\\,\\pm\\,\\pm 0\.0161\.140±±0\.0511\.140\\,\\pm\\,\\pm 0\.0510\.1810\.181Defenses that suppress attack gradientsFaceScrubPPANegLS0\.9060\.9060\.160±±0\.0040\.160\\,\\pm\\,\\pm 0\.0040\.365±±0\.0060\.365\\,\\pm\\,\\pm 0\.0061\.151±±0\.0161\.151\\,\\pm\\,\\pm 0\.0160\.1520\.152RoLSS0\.8860\.8860\.474±±0\.0060\.474\\,\\pm\\,\\pm 0\.0060\.760±±0\.0050\.760\\,\\pm\\,\\pm 0\.0050\.858±±0\.0110\.858\\,\\pm\\,\\pm 0\.0110\.4500\.450RoLSS\-SSF0\.8690\.8690\.343±±0\.0060\.343\\,\\pm\\,\\pm 0\.0060\.635±±0\.0060\.635\\,\\pm\\,\\pm 0\.0060\.918±±0\.0120\.918\\,\\pm\\,\\pm 0\.0120\.3220\.322Trap\-MID0\.9260\.9260\.401±±0\.0160\.401\\,\\pm\\,\\pm 0\.0160\.676±±0\.0150\.676\\,\\pm\\,\\pm 0\.0150\.980±±0\.0390\.980\\,\\pm\\,\\pm 0\.0390\.3760\.376Trap\-MID\*0\.9490\.9490\.587±±0\.0160\.587\\,\\pm\\,\\pm 0\.0160\.830±±0\.0120\.830\\,\\pm\\,\\pm 0\.0120\.920±±0\.0300\.920\\,\\pm\\,\\pm 0\.0300\.5560\.556IF\-GMINegLS0\.9060\.9060\.147±±0\.0040\.147\\,\\pm\\,\\pm 0\.0040\.352±±0\.0060\.352\\,\\pm\\,\\pm 0\.0061\.199±±0\.0141\.199\\,\\pm\\,\\pm 0\.0140\.1420\.142RoLSS0\.8860\.8860\.522±±0\.0060\.522\\,\\pm\\,\\pm 0\.0060\.785±±0\.0050\.785\\,\\pm\\,\\pm 0\.0050\.884±±0\.0110\.884\\,\\pm\\,\\pm 0\.0110\.4990\.499RoLSS\-SSF0\.8690\.8690\.373±±0\.0060\.373\\,\\pm\\,\\pm 0\.0060\.657±±0\.0060\.657\\,\\pm\\,\\pm 0\.0060\.956±±0\.0110\.956\\,\\pm\\,\\pm 0\.0110\.3540\.354Trap\-MID0\.9260\.9260\.618±±0\.0160\.618\\,\\pm\\,\\pm 0\.0160\.738±±0\.0140\.738\\,\\pm\\,\\pm 0\.0141\.004±±0\.0461\.004\\,\\pm\\,\\pm 0\.0460\.6030\.603Trap\-MID\*0\.9490\.9490\.713±±0\.0140\.713\\,\\pm\\,\\pm 0\.0140\.792±±0\.0130\.792\\,\\pm\\,\\pm 0\.0130\.939±±0\.0390\.939\\,\\pm\\,\\pm 0\.0390\.7050\.705CelebAPPANegLS0\.8050\.8050\.340±±0\.0150\.340\\,\\pm\\,\\pm 0\.0150\.623±±0\.0160\.623\\,\\pm\\,\\pm 0\.0160\.908±±0\.0370\.908\\,\\pm\\,\\pm 0\.0370\.3240\.324RoLSS0\.5370\.5370\.046±±0\.0070\.046\\,\\pm\\,\\pm 0\.0070\.145±±0\.0110\.145\\,\\pm\\,\\pm 0\.0111\.147±±0\.0441\.147\\,\\pm\\,\\pm 0\.0440\.0430\.043RoLSS\-SSF0\.4280\.4280\.039±±0\.0060\.039\\,\\pm\\,\\pm 0\.0060\.118±±0\.0100\.118\\,\\pm\\,\\pm 0\.0101\.183±±0\.0451\.183\\,\\pm\\,\\pm 0\.0450\.0350\.035Trap\-MID0\.8470\.8470\.330±±0\.0150\.330\\,\\pm\\,\\pm 0\.0150\.569±±0\.0160\.569\\,\\pm\\,\\pm 0\.0160\.944±±0\.0510\.944\\,\\pm\\,\\pm 0\.0510\.3090\.309Trap\-MID\*0\.8060\.8060\.092±±0\.0090\.092\\,\\pm\\,\\pm 0\.0090\.193±±0\.0130\.193\\,\\pm\\,\\pm 0\.0131\.358±±0\.0661\.358\\,\\pm\\,\\pm 0\.0660\.0880\.088IF\-GMINegLS0\.8050\.8050\.554±±0\.0160\.554\\,\\pm\\,\\pm 0\.0160\.811±±0\.0130\.811\\,\\pm\\,\\pm 0\.0130\.885±±0\.0350\.885\\,\\pm\\,\\pm 0\.0350\.5320\.532RoLSS0\.5370\.5370\.045±±0\.0070\.045\\,\\pm\\,\\pm 0\.0070\.145±±0\.0110\.145\\,\\pm\\,\\pm 0\.0111\.172±±0\.0531\.172\\,\\pm\\,\\pm 0\.0530\.0420\.042RoLSS\-SSF0\.4280\.4280\.044±±0\.0070\.044\\,\\pm\\,\\pm 0\.0070\.131±±0\.0110\.131\\,\\pm\\,\\pm 0\.0111\.218±±0\.0561\.218\\,\\pm\\,\\pm 0\.0560\.0400\.040Trap\-MID0\.8470\.8470\.417±±0\.0160\.417\\,\\pm\\,\\pm 0\.0160\.600±±0\.0160\.600\\,\\pm\\,\\pm 0\.0161\.004±±0\.0521\.004\\,\\pm\\,\\pm 0\.0520\.4030\.403Trap\-MID\*0\.8060\.8060\.168±±0\.0120\.168\\,\\pm\\,\\pm 0\.0120\.223±±0\.0130\.223\\,\\pm\\,\\pm 0\.0131\.437±±0\.0581\.437\\,\\pm\\,\\pm 0\.0580\.1630\.163

Table 28:Full privacy experiments results: AT\-AT vs Baselines onResNet\-18on CelebA and FaceScrub under PPA and IF\-GMI\. Entries show mean±\\pm95% CI half\-width for non\-confidence metrics\.DataAttackDefenseTestAcc↑\\uparrowAttAcc@1↓\\downarrowAttAcc@5↓\\downarrowL2↑\\uparrowEvalConf↓\\downarrowDefenses that attempt to increase privacy genericallyFaceScrubPPANoDef0\.9500\.9500\.901±±0\.0040\.901\\,\\pm\\,\\pm 0\.0040\.986±±0\.0020\.986\\,\\pm\\,\\pm 0\.0020\.761±±0\.0100\.761\\,\\pm\\,\\pm 0\.0100\.8810\.881MID0\.9240\.9240\.714±±0\.0060\.714\\,\\pm\\,\\pm 0\.0060\.903±±0\.0040\.903\\,\\pm\\,\\pm 0\.0040\.815±±0\.0120\.815\\,\\pm\\,\\pm 0\.0120\.6880\.688BiDO0\.8840\.8840\.530±±0\.0060\.530\\,\\pm\\,\\pm 0\.0060\.811±±0\.0050\.811\\,\\pm\\,\\pm 0\.0050\.915±±0\.0110\.915\\,\\pm\\,\\pm 0\.0110\.5040\.504TL\-DMI0\.9060\.9060\.240±±0\.0050\.240\\,\\pm\\,\\pm 0\.0050\.505±±0\.0060\.505\\,\\pm\\,\\pm 0\.0061\.020±±0\.0111\.020\\,\\pm\\,\\pm 0\.0110\.2250\.225AT\-AT0\.9210\.9210\.110±±0\.0040\.110\\,\\pm\\,\\pm 0\.0040\.297±±0\.0060\.297\\,\\pm\\,\\pm 0\.0061\.143±±0\.0131\.143\\,\\pm\\,\\pm 0\.0130\.1040\.104IF\-GMINoDef0\.9500\.9500\.983±±0\.0020\.983\\,\\pm\\,\\pm 0\.0020\.996±±0\.0010\.996\\,\\pm\\,\\pm 0\.0010\.740±±0\.0100\.740\\,\\pm\\,\\pm 0\.0100\.9790\.979MID0\.9240\.9240\.776±±0\.0050\.776\\,\\pm\\,\\pm 0\.0050\.840±±0\.0040\.840\\,\\pm\\,\\pm 0\.0040\.899±±0\.0130\.899\\,\\pm\\,\\pm 0\.0130\.7640\.764BiDO0\.8840\.8840\.782±±0\.0050\.782\\,\\pm\\,\\pm 0\.0050\.925±±0\.0030\.925\\,\\pm\\,\\pm 0\.0030\.887±±0\.0110\.887\\,\\pm\\,\\pm 0\.0110\.7640\.764TL\-DMI0\.9060\.9060\.515±±0\.0060\.515\\,\\pm\\,\\pm 0\.0060\.760±±0\.0060\.760\\,\\pm\\,\\pm 0\.0060\.935±±0\.0110\.935\\,\\pm\\,\\pm 0\.0110\.4920\.492AT\-AT0\.9210\.9210\.283±±0\.0060\.283\\,\\pm\\,\\pm 0\.0060\.520±±0\.0060\.520\\,\\pm\\,\\pm 0\.0061\.082±±0\.0141\.082\\,\\pm\\,\\pm 0\.0140\.2680\.268CelebAPPANoDef0\.8590\.8590\.634±±0\.0160\.634\\,\\pm\\,\\pm 0\.0160\.859±±0\.0110\.859\\,\\pm\\,\\pm 0\.0110\.774±±0\.0330\.774\\,\\pm\\,\\pm 0\.0330\.6080\.608MID0\.7370\.7370\.386±±0\.0160\.386\\,\\pm\\,\\pm 0\.0160\.671±±0\.0150\.671\\,\\pm\\,\\pm 0\.0150\.838±±0\.0360\.838\\,\\pm\\,\\pm 0\.0360\.3620\.362BiDO0\.6250\.6250\.146±±0\.0110\.146\\,\\pm\\,\\pm 0\.0110\.359±±0\.0160\.359\\,\\pm\\,\\pm 0\.0161\.009±±0\.0371\.009\\,\\pm\\,\\pm 0\.0370\.1360\.136TL\-DMI0\.7520\.7520\.153±±0\.0120\.153\\,\\pm\\,\\pm 0\.0120\.339±±0\.0160\.339\\,\\pm\\,\\pm 0\.0161\.013±±0\.0371\.013\\,\\pm\\,\\pm 0\.0370\.1420\.142TL\-DMI\-50\.8310\.8310\.324±±0\.0150\.324\\,\\pm\\,\\pm 0\.0150\.602±±0\.0160\.602\\,\\pm\\,\\pm 0\.0160\.886±±0\.0350\.886\\,\\pm\\,\\pm 0\.0350\.3020\.302AT\-AT†a0\.8250\.8250\.296±±0\.0150\.296\\,\\pm\\,\\pm 0\.0150\.561±±0\.0160\.561\\,\\pm\\,\\pm 0\.0160\.945±±0\.0380\.945\\,\\pm\\,\\pm 0\.0380\.2790\.279AT\-AT†b0\.7870\.7870\.156±±0\.0120\.156\\,\\pm\\,\\pm 0\.0120\.350±±0\.0160\.350\\,\\pm\\,\\pm 0\.0161\.029±±0\.0391\.029\\,\\pm\\,\\pm 0\.0390\.1450\.145IF\-GMINoDef0\.8590\.8590\.913±±0\.0090\.913\\,\\pm\\,\\pm 0\.0090\.970±±0\.0060\.970\\,\\pm\\,\\pm 0\.0060\.708±±0\.0310\.708\\,\\pm\\,\\pm 0\.0310\.9030\.903MID0\.7370\.7370\.550±±0\.0160\.550\\,\\pm\\,\\pm 0\.0160\.740±±0\.0140\.740\\,\\pm\\,\\pm 0\.0140\.889±±0\.0450\.889\\,\\pm\\,\\pm 0\.0450\.5290\.529BiDO0\.6250\.6250\.368±±0\.0160\.368\\,\\pm\\,\\pm 0\.0160\.629±±0\.0160\.629\\,\\pm\\,\\pm 0\.0160\.908±±0\.0330\.908\\,\\pm\\,\\pm 0\.0330\.3470\.347TL\-DMI0\.7520\.7520\.439±±0\.0160\.439\\,\\pm\\,\\pm 0\.0160\.710±±0\.0150\.710\\,\\pm\\,\\pm 0\.0150\.872±±0\.0360\.872\\,\\pm\\,\\pm 0\.0360\.4210\.421TL\-DMI\-50\.8310\.8310\.750±±0\.0140\.750\\,\\pm\\,\\pm 0\.0140\.922±±0\.0090\.922\\,\\pm\\,\\pm 0\.0090\.751±±0\.0290\.751\\,\\pm\\,\\pm 0\.0290\.7280\.728AT\-AT†a0\.8250\.8250\.783±±0\.0140\.783\\,\\pm\\,\\pm 0\.0140\.912±±0\.0090\.912\\,\\pm\\,\\pm 0\.0090\.799±±0\.0370\.799\\,\\pm\\,\\pm 0\.0370\.7640\.764AT\-AT†b0\.7870\.7870\.617±±0\.0160\.617\\,\\pm\\,\\pm 0\.0160\.806±±0\.0130\.806\\,\\pm\\,\\pm 0\.0130\.856±±0\.0400\.856\\,\\pm\\,\\pm 0\.0400\.5990\.599Defenses that suppress attack gradientsFaceScrubPPANegLS0\.9290\.9290\.649±±0\.0060\.649\\,\\pm\\,\\pm 0\.0060\.888±±0\.0040\.888\\,\\pm\\,\\pm 0\.0040\.876±±0\.0130\.876\\,\\pm\\,\\pm 0\.0130\.6240\.624RoLSS0\.9500\.9500\.858±±0\.0050\.858\\,\\pm\\,\\pm 0\.0050\.973±±0\.0020\.973\\,\\pm\\,\\pm 0\.0020\.746±±0\.0100\.746\\,\\pm\\,\\pm 0\.0100\.8370\.837RoLSS\-SSF0\.9500\.9500\.849±±0\.0050\.849\\,\\pm\\,\\pm 0\.0050\.973±±0\.0020\.973\\,\\pm\\,\\pm 0\.0020\.751±±0\.0100\.751\\,\\pm\\,\\pm 0\.0100\.8280\.828Trap\-MID0\.9280\.9280\.735±±0\.0140\.735\\,\\pm\\,\\pm 0\.0140\.930±±0\.0080\.930\\,\\pm\\,\\pm 0\.0080\.831±±0\.0270\.831\\,\\pm\\,\\pm 0\.0270\.7040\.704Trap\-MID\*0\.9270\.9270\.502±±0\.0160\.502\\,\\pm\\,\\pm 0\.0160\.752±±0\.0140\.752\\,\\pm\\,\\pm 0\.0140\.949±±0\.0350\.949\\,\\pm\\,\\pm 0\.0350\.4750\.475IF\-GMINegLS0\.9290\.9290\.932±±0\.0040\.932\\,\\pm\\,\\pm 0\.0040\.989±±0\.0010\.989\\,\\pm\\,\\pm 0\.0010\.824±±0\.0140\.824\\,\\pm\\,\\pm 0\.0140\.9230\.923RoLSS0\.9500\.9500\.979±±0\.0020\.979\\,\\pm\\,\\pm 0\.0020\.996±±0\.0010\.996\\,\\pm\\,\\pm 0\.0010\.683±±0\.0090\.683\\,\\pm\\,\\pm 0\.0090\.9750\.975RoLSS\-SSF0\.9500\.9500\.973±±0\.0020\.973\\,\\pm\\,\\pm 0\.0020\.992±±0\.0010\.992\\,\\pm\\,\\pm 0\.0010\.699±±0\.0100\.699\\,\\pm\\,\\pm 0\.0100\.9690\.969Trap\-MID0\.9280\.9280\.918±±0\.0090\.918\\,\\pm\\,\\pm 0\.0090\.970±±0\.0050\.970\\,\\pm\\,\\pm 0\.0050\.790±±0\.0250\.790\\,\\pm\\,\\pm 0\.0250\.9100\.910Trap\-MID\*0\.9270\.9270\.687±±0\.0150\.687\\,\\pm\\,\\pm 0\.0150\.770±±0\.0130\.770\\,\\pm\\,\\pm 0\.0130\.960±±0\.0430\.960\\,\\pm\\,\\pm 0\.0430\.6730\.673CelebAPPANegLS0\.8380\.8380\.667±±0\.0150\.667\\,\\pm\\,\\pm 0\.0150\.887±±0\.0110\.887\\,\\pm\\,\\pm 0\.0110\.730±±0\.0300\.730\\,\\pm\\,\\pm 0\.0300\.6400\.640RoLSS0\.8300\.8300\.506±±0\.0160\.506\\,\\pm\\,\\pm 0\.0160\.769±±0\.0140\.769\\,\\pm\\,\\pm 0\.0140\.809±±0\.0340\.809\\,\\pm\\,\\pm 0\.0340\.4740\.474RoLSS\-SSF0\.8390\.8390\.519±±0\.0160\.519\\,\\pm\\,\\pm 0\.0160\.764±±0\.0140\.764\\,\\pm\\,\\pm 0\.0140\.810±±0\.0330\.810\\,\\pm\\,\\pm 0\.0330\.4900\.490Trap\-MID0\.8300\.8300\.443±±0\.0160\.443\\,\\pm\\,\\pm 0\.0160\.700±±0\.0150\.700\\,\\pm\\,\\pm 0\.0150\.854±±0\.0500\.854\\,\\pm\\,\\pm 0\.0500\.4230\.423Trap\-MID\*0\.8110\.8110\.019±±0\.0040\.019\\,\\pm\\,\\pm 0\.0040\.032±±0\.0060\.032\\,\\pm\\,\\pm 0\.0061\.571±±0\.0401\.571\\,\\pm\\,\\pm 0\.0400\.0180\.018IF\-GMINegLS0\.8380\.8380\.953±±0\.0070\.953\\,\\pm\\,\\pm 0\.0070\.993±±0\.0030\.993\\,\\pm\\,\\pm 0\.0030\.649±±0\.0310\.649\\,\\pm\\,\\pm 0\.0310\.9440\.944RoLSS0\.8300\.8300\.724±±0\.0140\.724\\,\\pm\\,\\pm 0\.0140\.904±±0\.0100\.904\\,\\pm\\,\\pm 0\.0100\.778±±0\.0370\.778\\,\\pm\\,\\pm 0\.0370\.7050\.705RoLSS\-SSF0\.8390\.8390\.765±±0\.0140\.765\\,\\pm\\,\\pm 0\.0140\.909±±0\.0090\.909\\,\\pm\\,\\pm 0\.0090\.775±±0\.0340\.775\\,\\pm\\,\\pm 0\.0340\.7420\.742Trap\-MID0\.8300\.8300\.669±±0\.0150\.669\\,\\pm\\,\\pm 0\.0150\.797±±0\.0130\.797\\,\\pm\\,\\pm 0\.0130\.911±±0\.0620\.911\\,\\pm\\,\\pm 0\.0620\.6470\.647Trap\-MID\*0\.8100\.8100\.088±±0\.0090\.088\\,\\pm\\,\\pm 0\.0090\.134±±0\.0110\.134\\,\\pm\\,\\pm 0\.0111\.482±±0\.0471\.482\\,\\pm\\,\\pm 0\.0470\.0840\.084

*Notes:*†aand†bdenote two AT\-AT hyperparameter settings \(see Table[6](https://arxiv.org/html/2607.12354#A2.T6)\)\.