Revisiting the Provable-Auditable Privacy Gap of DP-SGD

# Revisiting the Provable-Auditable Privacy Gap of DP-SGD
Source: [https://arxiv.org/html/2608.28934](https://arxiv.org/html/2608.28934)
Srivi Balaji††thanks:University of Texas at Austin,srivibalaji@utexas\.edu\. Srivi and Yusong contributed equally\.Yusong Zhu††thanks:University of Texas at Austin,zhuys@utexas\.edu\. Srivi and Yusong contributed equally\.Gautam Kamath††thanks:University of Waterloo and Vector Institute,g@csail\.mit\.edu\. Supported by a Canada CIFAR AI Chair, an NSERC Discovery Grant, and an Ontario Early Researcher Award\.Kevin Tian††thanks:University of Texas at Austin,kjtian@cs\.utexas\.edu

###### Abstract

Differential privacy \(DP\) has traditionally been used to provide theoretical upper bounds on an algorithm’s stability to changing its training data\. In modern private machine learning applications, achieving strong tradeoffs between utility and theoretical privacy is challenging, and thus one may optimistically hope that existing theoretical privacy analyses are loose\. Recent work on*privacy auditing*has adopted a dual viewpoint, instead lower bounding the true privacy of an algorithm by constructing empirical distinguishing events\. The auditing literature has thus far yielded a pessimistic outlook on the looseness of theoretical privacy bounds for DP\-SGD, the de facto private training method in modern ML, as nearly\-matching empirical lower bounds have been achieved under various threat models\[[48](https://arxiv.org/html/2608.28934#bib.bib68),[5](https://arxiv.org/html/2608.28934#bib.bib62),[14](https://arxiv.org/html/2608.28934#bib.bib63)\]\.

In this work, we propose the empirical privacy lower bound of an algorithm as a concrete metric to optimize for, complementary to the theoretical upper bound\. We give a lightweight defense framework that generically augments optimization methods in the ML pipeline to have significantly\-improved empirical privacy on standard benchmarks\. Moreover, we show that our framework comes at*no theoretical privacy cost*when augmenting DP\-SGD, unlike previously\-proposed defenses against membership inference attacks\. We evaluate our defense against a broad range of audit constructions, models, and datasets to demonstrate its flexibility\.

###### Contents

1. [1Introduction](https://arxiv.org/html/2608.28934#S1)1. [1\.1Motivation](https://arxiv.org/html/2608.28934#S1.SS1) 2. [1\.2Our contributions](https://arxiv.org/html/2608.28934#S1.SS2) 3. [1\.3Discussion](https://arxiv.org/html/2608.28934#S1.SS3) 4. [1\.4Prior work](https://arxiv.org/html/2608.28934#S1.SS4)
2. [2Overview](https://arxiv.org/html/2608.28934#S2)1. [2\.1Preliminaries](https://arxiv.org/html/2608.28934#S2.SS1) 2. [2\.2Threat models](https://arxiv.org/html/2608.28934#S2.SS2) 3. [2\.3Framework](https://arxiv.org/html/2608.28934#S2.SS3) 4. [2\.4Measuring empirical privacy](https://arxiv.org/html/2608.28934#S2.SS4)
3. [3Privacy of Filtering via Sample Signatures](https://arxiv.org/html/2608.28934#S3)1. [3\.1Analysis](https://arxiv.org/html/2608.28934#S3.SS1) 2. [3\.2Variants of DP\-SGD](https://arxiv.org/html/2608.28934#S3.SS2)
4. [4Privacy\-Preserving Empirically Private Learning](https://arxiv.org/html/2608.28934#S4)1. [4\.1Experimental and audit setup](https://arxiv.org/html/2608.28934#S4.SS1) 2. [4\.2Input space attacks](https://arxiv.org/html/2608.28934#S4.SS2) 3. [4\.3Label\-only attack](https://arxiv.org/html/2608.28934#S4.SS3) 4. [4\.4Defense\-aware input space attack](https://arxiv.org/html/2608.28934#S4.SS4) 5. [4\.5Gradient space attacks](https://arxiv.org/html/2608.28934#S4.SS5)
5. [References](https://arxiv.org/html/2608.28934#bib)
6. [AEmpirical Privacy Reporting Details](https://arxiv.org/html/2608.28934#A1)1. [A\.1Heuristic conventions and their \(potential\) pitfalls](https://arxiv.org/html/2608.28934#A1.SS1) 2. [A\.2Numerical precision](https://arxiv.org/html/2608.28934#A1.SS2)
7. [BDisparate Impact](https://arxiv.org/html/2608.28934#A2)
8. [CAlternative Input Space Audits](https://arxiv.org/html/2608.28934#A3)1. [C\.1Single run audit](https://arxiv.org/html/2608.28934#A3.SS1) 2. [C\.2Group privacy audit](https://arxiv.org/html/2608.28934#A3.SS2)
9. [DSample Signatures](https://arxiv.org/html/2608.28934#A4)
10. [EPoisson vs\. Shuffled Subsampling](https://arxiv.org/html/2608.28934#A5)
11. [FInterpolating Canaries](https://arxiv.org/html/2608.28934#A6)
12. [GThe Privacy Onion Effect: Do We Expose Anyone Else?](https://arxiv.org/html/2608.28934#A7)
13. [HIncreasing the Trial Count](https://arxiv.org/html/2608.28934#A8)
14. [IAdditional Hyperparameter Tuning](https://arxiv.org/html/2608.28934#A9)1. [I\.1Global Filtering Ablation](https://arxiv.org/html/2608.28934#A9.SS1) 2. [I\.2Varying Local Filtering Defense Bandwidth Ablation](https://arxiv.org/html/2608.28934#A9.SS2) 3. [I\.3Varying Filter Frequency Ablation](https://arxiv.org/html/2608.28934#A9.SS3)
15. [JFull Empirical Privacy Results Across Audit Methods](https://arxiv.org/html/2608.28934#A10)

## 1Introduction

It is well\-documented that machine learning \(ML\) models are vulnerable to a variety of privacy threats\[[34](https://arxiv.org/html/2608.28934#bib.bib39),[58](https://arxiv.org/html/2608.28934#bib.bib51),[13](https://arxiv.org/html/2608.28934#bib.bib29)\]\. To combat these threats, differential privacy \(DP\)\[[22](https://arxiv.org/html/2608.28934#bib.bib33)\]has seen wide adoption as a rigorous notion of data privacy\. This notion is typically phrased as an upper bound \(Definition[1](https://arxiv.org/html/2608.28934#Thmdefinition1)\): for an algorithm𝒜:𝒮∗→Ω\\mathcal\{A\}:\\mathcal\{S\}^\{\*\}\\to\\Omega, DP bounds the value of

ε⋆​\(δ\):=log⁡\(sup𝒟,𝒟′∈𝒮∗neighboringsupℰ⊆ΩPr\[𝒜\(𝒟\)∈ℰ\]−δPr\[𝒜\(𝒟′\)∈ℰ\]\)\.\\varepsilon^\{\\star\}\(\\delta\):=\\log\\left\(\\sup\_\{\\begin\{subarray\}\{c\}\\mathcal\{D\},\\mathcal\{D\}^\{\\prime\}\\in\\mathcal\{S\}^\{\*\}\\\\ \\text\{neighboring\}\\end\{subarray\}\}\\sup\_\{\\mathcal\{E\}\\subseteq\\Omega\}\\frac\{\\Pr\[\\mathcal\{A\}\(\\mathcal\{D\}\)\\in\\mathcal\{E\}\]\-\\delta\}\{\\Pr\[\\mathcal\{A\}\(\\mathcal\{D\}^\{\\prime\}\)\\in\\mathcal\{E\}\]\}\\right\)\.\(1\)The quantityε⋆​\(δ\)\\varepsilon^\{\\star\}\(\\delta\)can be viewed as the “true privacy” parameter at failure probabilityδ\\deltaof𝒜\\mathcal\{A\}, as it is realized by \(or is the limit of\) concrete neighboring𝒟,𝒟′\\mathcal\{D\},\\mathcal\{D\}^\{\\prime\},111Different works in the privacy auditing literature use slightly different definitions of neighboring\. In this work, we use the “add/remove” definition as in\[[45](https://arxiv.org/html/2608.28934#bib.bib47),[48](https://arxiv.org/html/2608.28934#bib.bib68),[14](https://arxiv.org/html/2608.28934#bib.bib63)\]; see Section[2\.1](https://arxiv.org/html/2608.28934#S2.SS1)for more discussion\.and an “audit”ℰ∈Ω\\mathcal\{E\}\\in\\Omega\. Various techniques have been developed to upper boundε⋆\\varepsilon^\{\\star\}\. For example, modern DP\-SGD\[[60](https://arxiv.org/html/2608.28934#bib.bib52),[8](https://arxiv.org/html/2608.28934#bib.bib24),[1](https://arxiv.org/html/2608.28934#bib.bib18)\]privacy analyses proceed using \(advanced\) composition\[[24](https://arxiv.org/html/2608.28934#bib.bib35),[23](https://arxiv.org/html/2608.28934#bib.bib34)\], sometimes on an alternative metric such as Rényi differential privacy \(RDP\)\[[1](https://arxiv.org/html/2608.28934#bib.bib18),[46](https://arxiv.org/html/2608.28934#bib.bib46)\]\. We informally useεub​\(δ\)\\varepsilon\_\{\\textup\{ub\}\}\(\\delta\)to denote the tightest upper bound on \([1](https://arxiv.org/html/2608.28934#S1.E1)\) yielded by existing accounting techniques\.

There are various ways that privacy accounting can be loose\. A simple example is lossiness in converting between privacy definitions\. A more subtle example, concerning the*threat model*, asks: what should we view as the outputΩ\\Omegaof an algorithm, such as DP\-SGD, that \(adaptively\) generates a sequence of private modelsM1,M2,…,MTM\_\{1\},M\_\{2\},\\ldots,M\_\{T\}? One way to defineΩ\\Omegais as a product overTTmodel realizations\. This is implicit in standard \(composition\-based\) DP analyses, for which upper bounds onε⋆​\(δ\)\\varepsilon^\{\\star\}\(\\delta\)hold even ifℰ\\mathcal\{E\}depends on all intermediate models\. However, it is often more realistic to assume that only the finalMTM\_\{T\}is published \(a\.k\.a\. the “hidden state model”\[[68](https://arxiv.org/html/2608.28934#bib.bib71),[14](https://arxiv.org/html/2608.28934#bib.bib63)\]\), or even that our access toMTM\_\{T\}is limited \(e\.g\., query access rather than full model weights\), in which case the feasible auditsℰ\\mathcal\{E\}in \([1](https://arxiv.org/html/2608.28934#S1.E1)\) may be severely restricted\. Indeed, recent theory has illustrated scenarios where changing the threat model can \(significantly\) amplify privacy\[[68](https://arxiv.org/html/2608.28934#bib.bib71),[3](https://arxiv.org/html/2608.28934#bib.bib61)\]\.

### 1\.1Motivation

There is good reason to hope that current DP accounting significantly overestimates the true privacyε⋆\\varepsilon^\{\\star\}\. It is well\-documented that, to retain acceptable utility, current DP optimizers must compromise by yielding fairly large privacy parameters\. For example, Figure 1\(a\) of\[[19](https://arxiv.org/html/2608.28934#bib.bib32)\]summarizes recent state\-of\-the\-art accuracy of private training on CIFAR\-10\. With a strong provable privacyεub≈1\\varepsilon\_\{\\textup\{ub\}\}\\approx 1, the best accuracy was only60%60\\%\[[64](https://arxiv.org/html/2608.28934#bib.bib55)\], compared with a non\-private state of the art of99%99\\%\+\. On the other hand achieving a modest accuracy of\>80%\>80\\%required a much largerεub≈8\\varepsilon\_\{\\textup\{ub\}\}\\approx 8\. The worst\-case implications of such a largeεub\\varepsilon\_\{\\textup\{ub\}\}are dubious, asexp⁡\(8\)≈3000\\exp\(8\)\\approx 3000\. Optimistically, could it be that the true privacy parameter is much smaller than current theory predicts?

A closely\-related question is: how can a practitioner better reason about the true privacyε⋆\\varepsilon^\{\\star\}, taking the threat model into account? This is the key conceptual question that our work addresses\. Motivated by work on privacy auditing\[[35](https://arxiv.org/html/2608.28934#bib.bib40),[50](https://arxiv.org/html/2608.28934#bib.bib48)\], we propose using the following, more operational, “empirical privacy” quantity as an explicit metric for algorithm design:

εlb​\(δ\):=log⁡\(sup𝒟∈𝒮nsupcanary​csupℰ∈𝒯max⁡\(Pr\[𝒜\(𝒟c\)∈ℰ\]−δPr\[𝒜\(𝒟\)∈ℰ\],Pr\[𝒜\(𝒟\)∈ℰ\]−δPr\[𝒜\(𝒟c\)∈ℰ\]\)\)\.\\varepsilon\_\{\\textup\{lb\}\}\(\\delta\):=\\log\\left\(\\sup\_\{\\mathcal\{D\}\\in\\mathcal\{S\}^\{n\}\}\\sup\_\{\\textup\{canary \}c\}\\sup\_\{\\mathcal\{E\}\\in\\mathcal\{T\}\}\\max\\left\(\\frac\{\\Pr\[\\mathcal\{A\}\(\\mathcal\{D\}^\{c\}\)\\in\\mathcal\{E\}\]\-\\delta\}\{\\Pr\[\\mathcal\{A\}\(\\mathcal\{D\}\)\\in\\mathcal\{E\}\]\},\\frac\{\\Pr\[\\mathcal\{A\}\(\\mathcal\{D\}\)\\in\\mathcal\{E\}\]\-\\delta\}\{\\Pr\[\\mathcal\{A\}\(\\mathcal\{D\}^\{c\}\)\\in\\mathcal\{E\}\]\}\\right\)\\right\)\.\(2\)In \([2](https://arxiv.org/html/2608.28934#S1.E2)\),c∈𝒮c\\in\\mathcal\{S\}is a “canary” sample,𝒯\\mathcal\{T\}denotes a family of audits, and the goal ofℰ\\mathcal\{E\}is to detect the presence ofccin a dataset𝒟\\mathcal\{D\}, where𝒟c:=𝒟∪\{c\}\\mathcal\{D\}^\{c\}:=\\mathcal\{D\}\\cup\\\{c\\\}\. Under the strongest possible𝒟\\mathcal\{D\},cc, andℰ\\mathcal\{E\}, the quantities \([1](https://arxiv.org/html/2608.28934#S1.E1)\) and \([2](https://arxiv.org/html/2608.28934#S1.E2)\) are the same\. In practice, we cannot always find this optimal triple \(e\.g\., the rangeΩ\\Omegais typically infinite\), and hence our estimatedεlb\\varepsilon\_\{\\textup\{lb\}\}is only a lower bound forε⋆\\varepsilon^\{\\star\}\. Nonetheless, by optimizing over a comprehensive family of\(𝒟,c,ℰ\)\(\\mathcal\{D\},c,\\mathcal\{E\}\), one can hope to obtain nearly\-tight*certifiable lower bounds*onε⋆​\(δ\)\\varepsilon^\{\\star\}\(\\delta\)\. We review the standard auditing methodology in Section[2\.4](https://arxiv.org/html/2608.28934#S2.SS4)\. A typical setup involves a mislabeled or out\-of\-distribution canarycc, and an audit that thresholds the loss value of the final model when labelingcc\. Intuitively, ifccwas memorized, then it will be correctly classified and hence will incur much lower loss than had the model never seencc\.

Existing work on auditing DP\-SGD, arguably the most well\-studied private training algorithm in ML, is surprisingly pessimistic: nearly\-tight audits \(e\.g\., certifiedεlb\\varepsilon\_\{\\textup\{lb\}\}within a 30% factor ofεub\\varepsilon\_\{\\textup\{ub\}\}\) are achievable in a broad range of threat models\.222In Section[2\.2](https://arxiv.org/html/2608.28934#S2.SS2), we formally define and review different threat models that we consider from the literature\.This was first demonstrated by\[[50](https://arxiv.org/html/2608.28934#bib.bib48),[48](https://arxiv.org/html/2608.28934#bib.bib68)\]in the strong “gradient space canary” threat model, where the attacker can arbitrarily control*gradients*associated with a sample\. Existingεub\\varepsilon\_\{\\textup\{ub\}\}bounds hold in this threat model, but potentially overestimate the true \([1](https://arxiv.org/html/2608.28934#S1.E1)\), in real\-world settings where gradient space attacks are infeasible\[[14](https://arxiv.org/html/2608.28934#bib.bib63),[9](https://arxiv.org/html/2608.28934#bib.bib7)\]\. Later,\[[5](https://arxiv.org/html/2608.28934#bib.bib62),[14](https://arxiv.org/html/2608.28934#bib.bib63)\]obtained similar nearly\-tight audits under “hidden state” threat models where only the final model is released; particularly,\[[5](https://arxiv.org/html/2608.28934#bib.bib62)\]focuses on a more realistic “input space canary” threat model, where the canary is a fixed planted sample image\.

### 1\.2Our contributions

Our work makes two main contributions towards more accurately measuring the true privacy \([1](https://arxiv.org/html/2608.28934#S1.E1)\), and designing algorithms with improved empirical privacy under realistic threat models\.

- •We initiate the study of the*auditable privacy lower bound*εlb\\varepsilon\_\{\\textup\{lb\}\}as a formal metric for algorithms to target\. This line of research should be viewed as complementary to existing algorithmic work in DP, which focuses on designing methods that enjoy a smallerεub\\varepsilon\_\{\\textup\{ub\}\}\.
- •We develop an algorithmic framework for significantly improvingεlb\\varepsilon\_\{\\textup\{lb\}\}, as estimated by state\-of\-the\-art audits in the hidden state, input space threat model \(cf\. Section[2\.2](https://arxiv.org/html/2608.28934#S2.SS2)\)\. Our framework is a lightweight filtering\-based wrapper \(cf\. Section[2\.3](https://arxiv.org/html/2608.28934#S2.SS3)\), carefully designed so that, when applied to a wide range of popular DP\-SGD variants, it retains the same provableεub\\varepsilon\_\{\\textup\{ub\}\}\.

Empirical privacy measures are brittle\[[59](https://arxiv.org/html/2608.28934#bib.bib53),[11](https://arxiv.org/html/2608.28934#bib.bib27),[12](https://arxiv.org/html/2608.28934#bib.bib28),[2](https://arxiv.org/html/2608.28934#bib.bib60)\], so reasoning about their guarantees takes care\. In Section[1\.3](https://arxiv.org/html/2608.28934#S1.SS3), we discuss the broader implications ofεlb\\varepsilon\_\{\\textup\{lb\}\}\([2](https://arxiv.org/html/2608.28934#S1.E2)\) as a definition\. We observe here that targeting \([2](https://arxiv.org/html/2608.28934#S1.E2)\) opens an algorithmic design space that must go beyond DP\-SGD\. Indeed, despite a decade of research, no algorithm has dethroned theεub\\varepsilon\_\{\\textup\{ub\}\}\-utility tradeoff of \(Poisson subsampled\) DP\-SGD\. This algorithm is also tightly auditable\[[50](https://arxiv.org/html/2608.28934#bib.bib48),[48](https://arxiv.org/html/2608.28934#bib.bib68),[5](https://arxiv.org/html/2608.28934#bib.bib62),[14](https://arxiv.org/html/2608.28934#bib.bib63)\], with the notable caveat that existing audits have almost entirely targeted DP\-SGD\. In Section[4](https://arxiv.org/html/2608.28934#S4), we demonstrate that auditing our filtered variant with a comprehensive suite of tests from the literature yieldsεlb≈0\\varepsilon\_\{\\textup\{lb\}\}\\approx 0, at a negligible utility drop from its unfiltered counterpart\.

Although other strategies have been proposed for decreasing the auditability of ML algorithms \(discussed in Section[1\.4](https://arxiv.org/html/2608.28934#S1.SS4)\), our framework has a key qualitative advantage: it comes at*no loss*to the provableεub\\varepsilon\_\{\\textup\{ub\}\}of standard private optimizers\. This is the focus of Section[3](https://arxiv.org/html/2608.28934#S3), and making our proof flexible to a wide range of filters and optimizers is a main technical contribution of our work\. For example, while our framework is related to robust statistics\-inspired filters from the data poisoning literature\[[65](https://arxiv.org/html/2608.28934#bib.bib56),[32](https://arxiv.org/html/2608.28934#bib.bib66)\], these defenses break down when the fraction of poisoned points is too small \(e\.g\.,<1%<1\\%\)\. This is at odds with the auditing regime, where even a single canary can yield nearly\-tight audits\. Moreover, the filtering rules of\[[65](https://arxiv.org/html/2608.28934#bib.bib56),[32](https://arxiv.org/html/2608.28934#bib.bib66)\]were based on aggregate statistics \(e\.g\., PCA\)\. This makes it challenging to reason about privacy in datasets undergoing adaptive filtering, which intuitively could break the neighboring property over time\.

Our privacy proof bypasses both issues, by restricting our filtering rules to be based on what we call*sample signatures*, defined in Section[2\.3](https://arxiv.org/html/2608.28934#S2.SS3)\. Intuitively, these rules score individual samples in a way that only depends on other samples through a joint privately\-trained model \(so aggregate statistics such as the sample covariance of gradients cannot be used\)\. We give a simple yet flexible proof that neighboring datasets that are filtered using sample signatures remain neighboring \(Lemma[5](https://arxiv.org/html/2608.28934#Thmlemma5)\), and that adaptive composition proofs are robust to this interleaved filtering \(Lemma[4](https://arxiv.org/html/2608.28934#Thmlemma4)\)\. Putting together these observations, we prove a privacy guarantee for our filtered DP\-SGD method in Corollary[1](https://arxiv.org/html/2608.28934#Thmcorollary1)that matches Poisson\-subsampled DP\-SGD \(Theorem[1](https://arxiv.org/html/2608.28934#Thmtheorem1)\)\. Section[4\.2](https://arxiv.org/html/2608.28934#S4.SS2)then demonstrates that sample signatures are powerful enough to reliably filter input space canaries, substantially improving the Pareto frontier of utility\-εlb\\varepsilon\_\{\\textup\{lb\}\}tradeoffs against this test suite\.

Of course, the metric \([2](https://arxiv.org/html/2608.28934#S1.E2)\) is only meaningful if the audit suite is sufficiently representative of real attacks\. Section[4](https://arxiv.org/html/2608.28934#S4)and Appendix[C](https://arxiv.org/html/2608.28934#A3)provide systematic evaluations against all proposed input space canaries we are aware of, and Section[4\.4](https://arxiv.org/html/2608.28934#S4.SS4)constructs canaries deliberately targeting our framework\. Across all constructions, our framework yields improved, and sometimes negligible,εlb\\varepsilon\_\{\\textup\{lb\}\}\.

In Section[4\.5](https://arxiv.org/html/2608.28934#S4.SS5), we conclude by critically examining potential vulnerabilities of our framework under a stronger \(and often, unrealistic\)*gradient space*threat model\. Notably, the trainer can often enforce a particular threat model: for example, by computing their own gradients, and not publishing checkpoints, one can restrict training to the hidden state, input space setting\.

### 1\.3Discussion

A wide body of work\[[59](https://arxiv.org/html/2608.28934#bib.bib53),[11](https://arxiv.org/html/2608.28934#bib.bib27),[12](https://arxiv.org/html/2608.28934#bib.bib28),[2](https://arxiv.org/html/2608.28934#bib.bib60)\]cautions against the potential of empirical privacy evaluations to be misleading\. With this context in mind, here we provide some perspective on the value ofεlb\\varepsilon\_\{\\textup\{lb\}\}as an evaluation metric, and other considerations for the community\.

The implications ofεlb\\varepsilon\_\{\\textup\{lb\}\}as a metric\.Ourεlb\\varepsilon\_\{\\textup\{lb\}\}is not a formal security guarantee against*all possible privacy attacks*, and should be interpreted by users of our framework as measuring the vulnerability of an ML model against a*current suite of representative membership inference attacks*\. These attacks’ efficacy is measured by the same statistical distinguishing task as in \([2](https://arxiv.org/html/2608.28934#S1.E2)\), and thus we believe it is reasonable to base ourεlb\\varepsilon\_\{\\textup\{lb\}\}test suite𝒯\\mathcal\{T\}off of state\-of\-the\-art audits\. Our empirical privacy definition has the added benefit of being threat model\-aware, i\.e\., the test suite in \([2](https://arxiv.org/html/2608.28934#S1.E2)\) can be restricted depending on the attacks that are information\-theoretically feasible\.

As the test suite𝒯\\mathcal\{T\}in our definition \([2](https://arxiv.org/html/2608.28934#S1.E2)\) expands with future work,εlb\\varepsilon\_\{\\textup\{lb\}\}becomes a “moving target” by nature\. Notably, any newly\-developed auditing strategies can only strengthen the value of this target\. Our hope is that formalizingεlb\\varepsilon\_\{\\textup\{lb\}\}as an evaluation metric initiates a back\-and\-forth between auditing researchers aiming to expose privacy vulnerabilities \(by expanding𝒯\\mathcal\{T\}\), and algorithm designers aiming to withstand existing audits\. Indeed, our evaluations in Section[4](https://arxiv.org/html/2608.28934#S4)reveal a blind spot of current input space audits: either our filtered algorithms truly have smallerε⋆\\varepsilon^\{\\star\}than the theory predicts, or new tests must be developed to more accurately measureε⋆\\varepsilon^\{\\star\}\. Without proposingεlb\\varepsilon\_\{\\textup\{lb\}\}as an evaluation metric, or assessing audits beyond applying them to a single algorithm \(DP\-SGD\), there would be no impetus to develop these stronger tests\.

Finally, we believe our work has the potential to motivate advances in DP theory\. If a phenomenon is robustly identified \(e\.g\., the community cannot develop strong audits against an algorithm in a particular threat model\), this creates an opportunity for theorists to formalize this guarantee\.

Impact on non\-canary samples\.Prior works also caution against other potential pitfalls of empirically\-private methods, such as their exposure of other samples beyond the planted canary \(the “privacy onion” effect\[[12](https://arxiv.org/html/2608.28934#bib.bib28)\]\), and their potential for disparate impact on minority “benign outlier” groups\. Regarding the former issue, Appendix[G](https://arxiv.org/html/2608.28934#A7)demonstrates that even if we ignore the effects of multiple discovery, our filtered defense does not expose any non\-canary samples at a rate comparable to the standard canary audit on non\-filtered DP\-SGD\. Regarding the latter, the general tension between privacy and fairness is a known phenomenon\[[7](https://arxiv.org/html/2608.28934#bib.bib23),[66](https://arxiv.org/html/2608.28934#bib.bib57)\]that is out of our scope to address in full; nevertheless, Appendix[B](https://arxiv.org/html/2608.28934#A2)shows that the disparate impact of our framework is mild when minority groups are sufficiently large, or relatively few points are filtered\.

### 1\.4Prior work

To our knowledge, no work has previously proposed the goal of lowering the auditableεlb\\varepsilon\_\{\\textup\{lb\}\}\([2](https://arxiv.org/html/2608.28934#S1.E2)\) without compromising either the provableεub\\varepsilon\_\{\\textup\{ub\}\}or utility of a private learning algorithm\. Here, we survey two lines of works that have aimed to balance two of these three criteria\.

Empirically\-private algorithms\.Empirical privacy is often framed through membership inference, and indeed, a line of recent work from the security literature designs heuristics meant to protect against membership inference attacks \(MIAs\)\[[49](https://arxiv.org/html/2608.28934#bib.bib12),[36](https://arxiv.org/html/2608.28934#bib.bib9),[57](https://arxiv.org/html/2608.28934#bib.bib14),[15](https://arxiv.org/html/2608.28934#bib.bib17),[63](https://arxiv.org/html/2608.28934#bib.bib13),[16](https://arxiv.org/html/2608.28934#bib.bib8)\]\. These defenses aim at directly improving tradeoffs between the auditableεlb\\varepsilon\_\{\\textup\{lb\}\}and the model utility, but do not consider provable privacy\. Several of these works modify the training procedure or loss directly in a highly dataset\-dependent way\[[49](https://arxiv.org/html/2608.28934#bib.bib12),[36](https://arxiv.org/html/2608.28934#bib.bib9),[15](https://arxiv.org/html/2608.28934#bib.bib17),[63](https://arxiv.org/html/2608.28934#bib.bib13),[16](https://arxiv.org/html/2608.28934#bib.bib8)\], which makes it difficult to perform theoretical privacy accounting\. Other strategies are based on distillation from a teacher model\[[57](https://arxiv.org/html/2608.28934#bib.bib14)\]and are therefore less flexible to setups where such a model is not available\. We complement this literature by designing a heuristic defense with a privacy proof \(Section[3](https://arxiv.org/html/2608.28934#S3)\), and show that it maintains a \(lossless\)εub\\varepsilon\_\{\\textup\{ub\}\}\-utility tradeoff, competitive with the state\-of\-the\-art\. This modification makes measuring gaps betweenεlb\\varepsilon\_\{\\textup\{lb\}\}andεub\\varepsilon\_\{\\textup\{ub\}\}meaningful\.

Provable alternatives to DP\-SGD\.The literature has long considered tradeoffs between provableεub\\varepsilon\_\{\\textup\{ub\}\}and utility to be the gold standard for evaluating private learning algorithms\. Alternatives to DP\-SGD have been proposed, including PATE\[[51](https://arxiv.org/html/2608.28934#bib.bib49)\]\(which uses public data\), and augmentations of DP\-SGD\[[4](https://arxiv.org/html/2608.28934#bib.bib19),[21](https://arxiv.org/html/2608.28934#bib.bib16),[62](https://arxiv.org/html/2608.28934#bib.bib15)\]that adaptively estimate hyperparameters\.

For simplicity, we focus on \(subsampled variants of\) DP\-SGD as our baseline private optimizer of choice, as it remains an industry gold standard in privately training modern ML models\[[19](https://arxiv.org/html/2608.28934#bib.bib32)\]\. While other private optimizers have gained traction recently as competitive alternatives, such as DP\-FTRL\[[37](https://arxiv.org/html/2608.28934#bib.bib41),[53](https://arxiv.org/html/2608.28934#bib.bib3)\], work on the best way to audit models trained via alternative optimizers, a prerequisite to producing meaningfulεlb\\varepsilon\_\{\\textup\{lb\}\}estimates, is comparatively sparse\. We do note that our framework does not currently apply to DP\-FTRL, because that algorithm is not permutation\-invariant \(see discussion in Section[3](https://arxiv.org/html/2608.28934#S3)\)\. We leave constructing a broader, provably private, filtering framework beyond permutation\-invariant algorithms as an interesting open problem\.

We also mention that\[[27](https://arxiv.org/html/2608.28934#bib.bib38)\]proposed a conceptually similar use of filtering to design improved provably\-private algorithms\. Their approach \(and in particular, their privacy proof\) is quite different from ours, as they require filtering decisions to be based solely on individual statistics, whereas our framework uses rank\-based filtering depending on outlier scores\. Moreover, their goal is to improve utility\-privacy tradeoffs, whereas our work is largely motivated by the third axis of empirical privacy\. Finally, while we are not aware of prior works whose goal is explicitly to improveεub\\varepsilon\_\{\\textup\{ub\}\}\-εlb\\varepsilon\_\{\\textup\{lb\}\}\-utility tradeoffs,\[[35](https://arxiv.org/html/2608.28934#bib.bib40)\]observed that certain modifications to training \(e\.g\., randomized initialization\) result in betterεlb\\varepsilon\_\{\\textup\{lb\}\}, in a similar spirit to our motivation\.

## 2Overview

### 2\.1Preliminaries

Privacy\.For a sample space𝒮\\mathcal\{S\}, we use𝒮∗:=⋃n∈ℕ𝒮n\\mathcal\{S\}^\{\*\}:=\\bigcup\_\{n\\in\\mathbb\{N\}\}\\mathcal\{S\}^\{n\}to denote an arbitrary\-sized dataset from𝒮\\mathcal\{S\}\. We provide the following standard definition of differential privacy\.

###### Definition 1\(\(ε,δ\\varepsilon,\\delta\)\-DP\)\.

Let\(ε,δ\)∈ℝ≥0×\[0,1\]\(\\varepsilon,\\delta\)\\in\\mathbb\{R\}\_\{\\geq 0\}\\times\[0,1\]\. We say algorithm𝒜:𝒮∗→Ω\\mathcal\{A\}:\\mathcal\{S\}^\{\*\}\\to\\Omegasatisfies\(ε,δ\)\(\\varepsilon,\\delta\)\-differential privacy \(or, is\(ε,δ\)\(\\varepsilon,\\delta\)\-DP\) if for all eventsℰ⊆Ω\\mathcal\{E\}\\subseteq\\Omega, and all neighboring𝒟,𝒟′∈𝒮∗\\mathcal\{D\},\\mathcal\{D\}^\{\\prime\}\\in\\mathcal\{S\}^\{\*\},

Pr\[𝒜\(𝒟\)∈ℰ\]≤exp\(ε\)Pr\[𝒜\(𝒟′\)∈ℰ\]\+δ\.\\Pr\\left\[\\mathcal\{A\}\(\\mathcal\{D\}\)\\in\\mathcal\{E\}\\right\]\\leq\\exp\(\\varepsilon\)\\Pr\\left\[\\mathcal\{A\}\(\\mathcal\{D\}^\{\\prime\}\)\\in\\mathcal\{E\}\\right\]\+\\delta\.

One subtlety in Definition[1](https://arxiv.org/html/2608.28934#Thmdefinition1)is that the literature uses slightly different definitions of*neighboring*; these notions of DP often imply each other, but only up to a constant factor loss\. As our paper’s methods are based on Poisson\-subsampled DP\-SGD, the de facto method for training provably\-private ML models,333For example, privacy accounting via Poisson subsampling is the standard implementation in popular privacy packages such as Pytorch Opacus\[[69](https://arxiv.org/html/2608.28934#bib.bib59)\]and Tensorflow Privacy\[[31](https://arxiv.org/html/2608.28934#bib.bib70)\]\.we use the definition admitting the tightest analyses of this algorithm, for fair comparison\. Concretely, we use the add/remove definition of DP, where datasets𝒟∈𝒮n\\mathcal\{D\}\\in\\mathcal\{S\}^\{n\},𝒟′∈𝒮m\\mathcal\{D\}^\{\\prime\}\\in\\mathcal\{S\}^\{m\}are*neighboring*if\|m−n\|=1\|m\-n\|=1, and\|𝒟∖𝒟′\|\+\|𝒟′∖𝒟\|=1\|\\mathcal\{D\}\\setminus\\mathcal\{D\}^\{\\prime\}\|\+\|\\mathcal\{D\}^\{\\prime\}\\setminus\\mathcal\{D\}\|=1, i\.e\., they differ in one element\.

The privacy analysis of DP\-SGD is often performed using the following alternative definition\.

###### Definition 2\(RDP\)\.

Letn∈ℕn\\in\{\\mathbb\{N\}\},α\>1\\alpha\>1, andρ≥0\\rho\\geq 0\. We say that𝒜:𝒮∗→Ω\\mathcal\{A\}:\\mathcal\{S\}^\{\*\}\\to\\Omegasatisfies\(α,ρ\)\(\\alpha,\\rho\)\-RDP if for all neighboring datasets𝒟,𝒟′∈𝒮∗\\mathcal\{D\},\\mathcal\{D\}^\{\\prime\}\\in\\mathcal\{S\}^\{\*\},

Dα\(𝒜\(𝒟\)∥𝒜\(𝒟′\)\)≤ρ,D\_\{\\alpha\}\(\\mathcal\{A\}\(\\mathcal\{D\}\)\\\|\\mathcal\{A\}\(\\mathcal\{D\}^\{\\prime\}\)\)\\leq\\rho,\(3\)whereDαD\_\{\\alpha\}is the Rényi divergence and defined asDα\(P∥Q\)=1α−1log𝔼ω∼P\[\(P⁡\(ω\)Q⁡\(ω\)\)α−1\]D\_\{\\alpha\}\(P\\\|Q\)=\\frac\{1\}\{\\alpha\-1\}\\log\\mathbb\{E\}\_\{\\omega\\sim P\}\[\(\\frac\{P\(\\omega\)\}\{Q\(\\omega\)\}\)^\{\\alpha\-1\}\]\.

Lemma[1](https://arxiv.org/html/2608.28934#Thmlemma1)gives a conversion from RDP to DP\. For more properties of RDP, we refer to\[[46](https://arxiv.org/html/2608.28934#bib.bib46)\]\.

###### Lemma 1\(\[[46](https://arxiv.org/html/2608.28934#bib.bib46)\]\)\.

Ifℳ\\mathcal\{M\}satisfies\(α,ε\)\(\\alpha,\\varepsilon\)\-RDP, it satisfies\(ε\+1α−1​log⁡1δ,δ\)\(\\varepsilon\+\\frac\{1\}\{\\alpha\-1\}\\log\\frac\{1\}\{\\delta\},\\delta\)\-DP for allδ∈\(0,1\)\\delta\\in\(0,1\)\.

DP\-SGD\.Algorithm[1](https://arxiv.org/html/2608.28934#algorithm1)outlines a typical Poisson\-subsampled DP\-SGD framework for training a model with parametersθ\\thetaby minimizing the empirical loss1n​∑\(x,y\)∈Dℓ⁡\(θ,x,y\)\\frac\{1\}\{n\}\\sum\_\{\(x,y\)\\in D\}\\ell\(\\theta;x,y\), where𝒟\\mathcal\{D\}is the dataset ofnnexamples\. At each step of DP\-SGD, we first \(randomly\) subsample a minibatch; then we compute the per\-sample gradient∇θℓ​\(θ,x,y\)\\nabla\_\{\\theta\}\\ell\(\\theta;x,y\)for each\(x,y\)\(x,y\)within the batchℬ\\mathcal\{B\}and clip theirℓ2\\ell\_\{2\}norms; lastly, we average the clipped batch and add Gaussian noise to ensure privacy\.

Algorithm 1𝖣𝖯𝖲𝖦𝖣𝖯𝗈𝗂𝗌𝗌𝗈𝗇⁡\(𝒟,θ0,q,η,C,σ,T\)\\mathsf\{DPSGDPoisson\}\(\\mathcal\{D\},\\theta\_\{0\},q,\\eta,C,\\sigma,T\)Input:Dataset

𝒟\\mathcal\{D\}, model initialization

θ0\\theta\_\{0\}, sample rate

qq, step size

η\\eta, clip bound

CC, noise multiplier

σ\\sigma, step count

TT
Output:Final model parameters

θ\\theta
1

θ←θ0\\theta\\leftarrow\\theta\_\{0\}
2for

t∈\[T\]t\\in\[T\]do

3

ℬ←\{\}\\mathcal\{B\}\\leftarrow\\left\\\{\\right\\\}
4for

\(x,y\)∈𝒟\(x,y\)\\in\\mathcal\{D\}do

5

r←Bern​\(q\)r\\leftarrow\\textup\{Bern\}\(q\)
6if

r=1r=1then

7

ℬ←ℬ∪\(x,y\)\\mathcal\{B\}\\leftarrow\\mathcal\{B\}\\cup\(x,y\)
8end if

9end for

10

ξ∼𝒩⁡\(0,\(σ​Cq​n\)2​I\)\\xi\\sim\\mathcal\{N\}\\left\(0,\\left\(\\frac\{\\sigma C\}\{qn\}\\right\)^\{2\}I\\right\)//Spherical Gaussian with standard deviationσ​Cq​n\\frac\{\\sigma C\}\{qn\}

11

θ←θ−η⁡\(1q​n​∑\(x,y\)∈ℬclip​\(∇θℓ​\(θ,x,y\),C\)\+ξ\)\\theta\\leftarrow\\theta\-\\eta\\left\(\\frac\{1\}\{qn\}\\sum\_\{\(x,y\)\\in\\mathcal\{B\}\}\\text\{clip\}\(\\nabla\_\{\\theta\}\\ell\(\\theta;x,y\),C\)\+\\xi\\right\)//

clip​\(v,C\)=v⋅min⁡\(1,C‖v‖2\)\\text\{clip\}\(v,C\)=v\\cdot\\min\(1,\\frac\{C\}\{\\left\\lVert v\\right\\rVert\_\{2\}\}\)
12end for

13return

θ\\theta

Computing the tightest upper bound of overall privacy cost of training a model is extensively studied in DP\. Various accounting methods have been proposed to tackle this problem, e\.g\., moments accountant in\[[1](https://arxiv.org/html/2608.28934#bib.bib18)\], RDP in\[[45](https://arxiv.org/html/2608.28934#bib.bib47),[73](https://arxiv.org/html/2608.28934#bib.bib72)\], and PLD in\[[39](https://arxiv.org/html/2608.28934#bib.bib67)\]\. We recall the accounting method based on RDP in Theorem[1](https://arxiv.org/html/2608.28934#Thmtheorem1), which gives a clean theoretical bound, implemented by popular privacy packages\[[69](https://arxiv.org/html/2608.28934#bib.bib59),[31](https://arxiv.org/html/2608.28934#bib.bib70)\]\. We mention that PLD\[[39](https://arxiv.org/html/2608.28934#bib.bib67)\]can achieve tighterεub\\varepsilon\_\{\\textup\{ub\}\}estimates in some cases than Theorem[1](https://arxiv.org/html/2608.28934#Thmtheorem1), but uses a numerical method that does not yield an explicit bound; our defense would enjoy an equally\-tighter upper bound, as discussed in Section[3](https://arxiv.org/html/2608.28934#S3)\.

###### Theorem 1\(\[[45](https://arxiv.org/html/2608.28934#bib.bib47)\]\)\.

Ifq​<15,σ\>​4q<\\frac\{1\}\{5\},\\sigma\>4, andα≤12​σ2​L2−ln⁡5−2​ln⁡σL\+ln⁡\(q​α\)\+\(2​σ2\)−1\\alpha\\leq\\frac\{\\frac\{1\}\{2\}\\sigma^\{2\}L^\{2\}\-\\ln 5\-2\\ln\\sigma\}\{L\+\\ln\(q\\alpha\)\+\(2\\sigma^\{2\}\)^\{\-1\}\}whereL:=log⁡\(1\+1q⁡\(α−1\)\)L:=\\log\(1\+\\frac\{1\}\{q\(\\alpha\-1\)\}\), Algorithm[1](https://arxiv.org/html/2608.28934#algorithm1)is\(α,T⋅2​q2​ασ\)\(\\alpha,T\\cdot\\frac\{2q^\{2\}\\alpha\}\{\\sigma\}\)\-RDP\. It is also\(T⋅2​q2​ασ\+1α−1​log⁡1δ,δ\)\(T\\cdot\\frac\{2q^\{2\}\\alpha\}\{\\sigma\}\+\\frac\{1\}\{\\alpha\-1\}\\log\\frac\{1\}\{\\delta\},\\delta\)\-DP for allδ∈\(0,1\)\\delta\\in\(0,1\)\.

Due to practical considerations, alternatives to Algorithm[1](https://arxiv.org/html/2608.28934#algorithm1)are sometimes used, despite their weaker provable privacy\-utility tradeoffs\. For example, a common heuristic is to divide the dataset into randomly\-shuffled minibatches of sizemm\(see Algorithm[4](https://arxiv.org/html/2608.28934#algorithm4)\) forNNepochs; due to implementation details, this strategy often results in improved practical performance\. Despite Theorem[1](https://arxiv.org/html/2608.28934#Thmtheorem1)not formally applying to this alternative algorithm, it is a common practice to apply its bound with the “in\-expectation” equivalent parametersq=mnq=\\frac\{m\}\{n\},T=N⋅⌈nm⌉T=N\\cdot\\lceil\\frac\{n\}\{m\}\\rceilto conclude a heuristic bound\[[19](https://arxiv.org/html/2608.28934#bib.bib32),[54](https://arxiv.org/html/2608.28934#bib.bib69),[18](https://arxiv.org/html/2608.28934#bib.bib30),[41](https://arxiv.org/html/2608.28934#bib.bib43)\]\. We sidestep this issue by directly using Algorithm[1](https://arxiv.org/html/2608.28934#algorithm1)in our experiments, with its provable guarantee in Theorem[1](https://arxiv.org/html/2608.28934#Thmtheorem1)\. In Section[3](https://arxiv.org/html/2608.28934#S3), we discuss how our privacy accounting for our filtered optimizers applies to Algorithm[4](https://arxiv.org/html/2608.28934#algorithm4)as well\. We also explain how our analysis framework extends to other recently\-proposed alternatives, such as\[[17](https://arxiv.org/html/2608.28934#bib.bib31)\]\.

### 2\.2Threat models

The threat model specifies what information is known to the attacker/auditor and what strategies are allowed when constructing attacks\. The literature on privacy auditing often uses “white\-box” and “black\-box” inconsistently as descriptors of threat models\.

To sidestep this ambiguity, in this paper, we describe threat models along two main axes\. The first axis is auditor access: in a*full state*threat model, the auditor has access to internal training information such as model parameters throughout training \(e\.g\., intermediate checkpoints\), while in a*hidden state*setting, intermediate training states are not revealed and the auditor only observes the final trained model \(i\.e\., its weights\)\. The second axis is what the auditor is allowed to perturb when designing an attack:*input space*attacks construct canaries by modifying the raw input \(e\.g\., pixel values\), while*gradient space*attacks operate by targeting the gradient signal a canary would produce during training \(rather than directly optimizing the input itself\)\.

Recent works\[[14](https://arxiv.org/html/2608.28934#bib.bib63),[9](https://arxiv.org/html/2608.28934#bib.bib7)\]suggest that the hidden state, input space threat model most accurately reflects the practice of private ML\. While this setting is our focus, in Section[4](https://arxiv.org/html/2608.28934#S4), we critically examine our framework’s vulnerability to gradient space attacks accessing model history \(Section[4\.5](https://arxiv.org/html/2608.28934#S4.SS5)\), and the reproducibility of these attacks in input space settings \(Section[4\.4](https://arxiv.org/html/2608.28934#S4.SS4)\)\.

### 2\.3Framework

As discussed in Section[1\.2](https://arxiv.org/html/2608.28934#S1.SS2), our defense is a wrapper that periodically filters training samples based on certain scoring rules\. We provide pseudocode for the Poisson subsampled DP\-SGD variant of our defense in Algorithm[2](https://arxiv.org/html/2608.28934#algorithm2); we show how to bound its privacy in Corollary[1](https://arxiv.org/html/2608.28934#Thmcorollary1)\.

Algorithm 2𝖥𝗂𝗅𝗍𝖾𝗋𝖾𝖽𝖣𝖯𝖲𝖦𝖣𝖯𝗈𝗂𝗌𝗌𝗈𝗇⁡\(𝒟,θ0,q,η,C,σ,T,m,f,k,scope\)\\mathsf\{FilteredDPSGDPoisson\}\(\\mathcal\{D\},\\theta\_\{0\},q,\\eta,C,\\sigma,T,m,f,k,\\texttt\{scope\}\)Input:Dataset

𝒟\\mathcal\{D\}, model initialization

θ0\\theta\_\{0\}, sample rate

qq, step size

η\\eta, clip bound

CC, noise multiplier

σ\\sigma, step count

TT, epoch parameter

mm, sample signature

ff, selection parameter

kk, flag

scope∈\{global,class\}\\texttt\{scope\}\\in\\\{\\texttt\{global\},\\texttt\{class\}\\\}
Output:Final model parameters

θ\\theta
1

\(ℱ,S,θ\)←\(∅,\{0\}\|𝒟\|,θ0\)\(\\mathcal\{F\},S,\\theta\)\\leftarrow\(\\emptyset,\\\{0\\\}^\{\|\\mathcal\{D\}\|\},\\theta\_\{0\}\)
2for

t∈\[T\]t\\in\[T\]do

3

ℬ←\{\}\\mathcal\{B\}\\leftarrow\\\{\\\}
4for

\(x,y\)∈𝒟\(x,y\)\\in\\mathcal\{D\}do

5

r←Bern​\(q\)r\\leftarrow\\textup\{Bern\}\(q\)
6if

r=1r=1then

7

ℬ←ℬ∪\(x,y\)\\mathcal\{B\}\\leftarrow\\mathcal\{B\}\\cup\(x,y\)
8end if

9end for

10foreach

\(xi,yi\)∈ℬ\(x\_\{i\},y\_\{i\}\)\\in\\mathcal\{B\}do

11

gi←\{clip​\(∇θℓ​\(θ,xi,yi\),C\)i∉ℱ0i∈ℱg\_\{i\}\\leftarrow\\begin\{cases\}\\textup\{clip\}\(\\nabla\_\{\\theta\}\\ell\(\\theta;x\_\{i\},y\_\{i\}\),C\)&i\\not\\in\\mathcal\{F\}\\\\ 0&i\\in\\mathcal\{F\}\\end\{cases\}
12

S⁡\[i\]←fθ​\(xi,yi\)S\[i\]\\leftarrow f\_\{\\theta\}\(x\_\{i\},y\_\{i\}\)
13end foreach

14

ξ∼𝒩⁡\(0,\(σ​Cq​n\)2​I\)\\xi\\sim\\mathcal\{N\}\(0,\(\\frac\{\\sigma C\}\{qn\}\)^\{2\}I\)
15

θ←θ−η⁡\(1q​n​∑\(xi,yi\)∈ℬgi\+ξ\)\\theta\\leftarrow\\theta\-\\eta\(\\frac\{1\}\{qn\}\\sum\_\{\(x\_\{i\},y\_\{i\}\)\\in\\mathcal\{B\}\}g\_\{i\}\+\\xi\)
16if

m\|tm\\mid tthen

//Waitmmminibatch iterations before filtering\.

17

ℱ←ℱ∪𝖲𝖾𝗅𝖾𝖼𝗍𝖳𝗈𝗉⁡\(S,𝒟,ℱ,k,scope\)\\mathcal\{F\}\\leftarrow\\mathcal\{F\}\\cup\\mathsf\{SelectTop\}\(S,\\mathcal\{D\},\\mathcal\{F\},k,\\texttt\{scope\}\)
18end if

19end for

20return

θ\\theta

Sample signatures\.Asample signature\(orscoring function\) maps samples to real numbers, based solely on information derived from that sample and privatized model parameters\. Crucially, a sample signature does not depend on any other samples, except through the learned private model, in contrast to previous “batch” filtering rules proposed in the data poisoning literature\[[65](https://arxiv.org/html/2608.28934#bib.bib56),[32](https://arxiv.org/html/2608.28934#bib.bib66)\], which depended on e\.g\., the mean or covariance of the dataset gradients \(a batch statistic\)\. Use of sample signatures is essential for preserving privacy, an argument we formalize in Section[3](https://arxiv.org/html/2608.28934#S3)via a generalized composition theorem robust to sample signature filtering\.

Algorithm 3𝖲𝖾𝗅𝖾𝖼𝗍𝖳𝗈𝗉⁡\(S,𝒟,ℱ,k,scope\)\\mathsf\{SelectTop\}\(S,\\mathcal\{D\},\\mathcal\{F\},k,\\texttt\{scope\}\)Input:Computed sample signatures

SS, dataset

𝒟\\mathcal\{D\}, filtered set

ℱ⊆𝒟\\mathcal\{F\}\\subseteq\\mathcal\{D\}, count

k∈ℕk\\in\\mathbb\{N\}, mode

scope∈\{global,class\}\\texttt\{scope\}\\in\\\{\\texttt\{global\},\\texttt\{class\}\\\}
Output:Selected samples

𝒯\\mathcal\{T\}
1

𝒰←𝒟∖ℱ\\mathcal\{U\}\\leftarrow\\mathcal\{D\}\\setminus\\mathcal\{F\}
2if

scope=global\\texttt\{scope\}=\\texttt\{global\}then

3return

kkindices

i∈𝒰i\\in\\mathcal\{U\}with largest

S⁡\[i\]S\[i\]
4else

5foreachclass

ccrepresented in

𝒰\\mathcal\{U\}do

6

𝒯c←\\mathcal\{T\}\_\{c\}\\leftarrowkkindices

i∈𝒰c:=\{i∈𝒰∣yi=c\}i\\in\\mathcal\{U\}\_\{c\}:=\\\{i\\in\\mathcal\{U\}\\mid y\_\{i\}=c\\\}with largest

S⁡\[i\]S\[i\]
7end foreach

8return

⋃class​c​represented in​𝒰𝒯c\\bigcup\_\{\\text\{class \}c\\text\{ represented in \}\\mathcal\{U\}\}\\mathcal\{T\}\_\{c\}
9end if

We denote a sample signature byfθ​\(x,y\)f\_\{\\theta\}\(x,y\)when scoring a sample\(x,y\)\(x,y\), where the subscript indicates implicit dependence on the model\. We consider several choices offθf\_\{\\theta\}; a few examples follow\.

- •Gradient norm:fθ​\(x,y\):=‖∇θℓ​\(θ,x,y\)‖pf\_\{\\theta\}\(x,y\):=\\left\\lVert\\nabla\_\{\\theta\}\\ell\(\\theta;x,y\)\\right\\rVert\_\{p\}where, e\.g\.,p∈\{1,2,∞\}p\\in\\\{1,2,\\infty\\\}\. High scores suggest an influential sample in the trajectory of the model updates\.
- •Prediction margin: Let the \(softmaxed\) logits of example\(x,y\)\(x,y\)with respect to modelθ\\thetabe denotedpθ​\(x\)​\[c\]p\_\{\\theta\}\(x\)\[c\]for a classcc\. Then we definefθ​\(x,y\):=−\(pθ​\(x\)​\[y\]−maxc≠y⁡pθ​\(x\)​\[c\]\)f\_\{\\theta\}\(x,y\):=\-\(p\_\{\\theta\}\(x\)\[y\]\-\\max\_\{c\\neq y\}p\_\{\\theta\}\(x\)\[c\]\)\. High scores suggest a sample with an unusual or misclassified label\.

In Appendix[D](https://arxiv.org/html/2608.28934#A4), we introduce several additional sample signatures, and provide an ablation study on the ability of these various scores to detect samples with high risk of memorization\.

One major benefit of our outlier detection strategy is that it can detect outliers with significantly less signal than the “batch” update rules used by\[[65](https://arxiv.org/html/2608.28934#bib.bib56),[32](https://arxiv.org/html/2608.28934#bib.bib66)\], which state they require a constant fraction \(e\.g\.,1%1\\%\) of poisoned data to be effective\. In DP auditing settings, this distinction is crucial, as memorization of even a single canary can nullify privacy bounds\.

Design rationale\.We make two key design choices in our implementation of Algorithm[2](https://arxiv.org/html/2608.28934#algorithm2)\. The first is to only allow for sample signature\-based filtering \(as opposed to “batch” scoring rules\)\. This enables using our generalized composition theorems from Section[3](https://arxiv.org/html/2608.28934#S3)to argue that our modifications do not affect the privacy of Algorithm[2](https://arxiv.org/html/2608.28934#algorithm2)compared to Algorithm[1](https://arxiv.org/html/2608.28934#algorithm1)\. Regarding the “scope” parameter in Algorithm[2](https://arxiv.org/html/2608.28934#algorithm2), gradient statistics vary between classes\. Global filtering risks flagging representative samples from classes with naturally higher gradient norms as outliers\. Per\-class filtering \(Algorithm[3](https://arxiv.org/html/2608.28934#algorithm3), Lines[3](https://arxiv.org/html/2608.28934#algorithm3)to[3](https://arxiv.org/html/2608.28934#algorithm3)\) standardizes the detection threshold by comparing each sample only against others in its class, ensuring that outlier detection is relative to class\-specific distributions\.

### 2\.4Measuring empirical privacy

We next review LiRA\-style membership inference attacks for DP auditing\[[11](https://arxiv.org/html/2608.28934#bib.bib27),[48](https://arxiv.org/html/2608.28934#bib.bib68)\]\.

Trade\-off functions\.Consider two arbitrary distributionsP,QP,Qsupported on the same spaceΩ\\Omega, and a*decision rule*ϕ:Ω→\[0,1\]\\phi:\\Omega\\to\[0,1\]\. Intuitively,ϕ\\phiacts as a distinguisher betweenPPandQQ, whereϕ⁡\(ω\)=0\\phi\(\\omega\)=0can be taken as guessing thatω\\omegawas sampled fromPP, andϕ⁡\(ω\)=1\\phi\(\\omega\)=1indicates a sample fromQQ\(fractional values thus represent uncertainty\)\. We accordingly define

αP,Q​\(ϕ\):=𝔼ω∼P​\[ϕ⁡\(ω\)\],βP,Q​\(ϕ\):=𝔼ω∼Q​\[1−ϕ⁡\(ω\)\]\\displaystyle\\alpha\_\{P,Q\}\(\\phi\):=\\mathbb\{E\}\_\{\\omega\\sim P\}\[\\phi\(\\omega\)\],\\quad\\beta\_\{P,Q\}\(\\phi\):=\\mathbb\{E\}\_\{\\omega\\sim Q\}\[1\-\\phi\(\\omega\)\]to be the type I \(FPR\) and type II \(FNR\) error rates ofϕ\\phi\. For each FPRα∈\[0,1\]\\alpha\\in\[0,1\], we can consider the optimal test which minimizes the FNRβ∈\[0,1\]\\beta\\in\[0,1\]at the specifiedα\\alphalevel, formalized as follows\.

###### Definition 3\(Trade\-off function\)\.

For two distributionsP,QP,Qsupported onΩ\\Omega, we define the*trade\-off function*TP,Q:\[0,1\]→\[0,1\]T\_\{P,Q\}:\[0,1\]\\to\[0,1\]byTP,Q\(α\):=infϕ:Ω→\[0,1\],αP,Q​\(ϕ\)≤αβP,Q\(ϕ\)T\_\{P,Q\}\(\\alpha\):=\\inf\_\{\\begin\{subarray\}\{c\}\\phi:\\Omega\\to\[0,1\],\\alpha\_\{P,Q\}\(\\phi\)\\leq\\alpha\\end\{subarray\}\}\\beta\_\{P,Q\}\(\\phi\)\.

More generally, we say thatf:\[0,1\]→\[0,1\]f:\[0,1\]\\to\[0,1\]is a trade\-off function iff=TP,Qf=T\_\{P,Q\}for someP,QP,Q\.

The Neyman\-Pearson lemma characterizesϕ\\phiachieving the infimum inTP,QT\_\{P,Q\}as thresholded likelihood ratio tests \(Theorem A\.1,\[[20](https://arxiv.org/html/2608.28934#bib.bib64)\]\)\. Next, we state some basic properties of trade\-off functions\.

###### Lemma 2\(Proposition 2\.2,\[[20](https://arxiv.org/html/2608.28934#bib.bib64)\]\)\.

f:\[0,1\]→\[0,1\]f:\[0,1\]\\to\[0,1\]is a trade\-off function iff it is convex, non\-increasing, and satisfiesf⁡\(α\)≤1−αf\(\\alpha\)\\leq 1\-\\alphafor allα∈\[0,1\]\\alpha\\in\[0,1\]\.

The first property in Lemma[2](https://arxiv.org/html/2608.28934#Thmlemma2)follows because given testsϕ,ϕ′\\phi,\\phi^\{\\prime\}with FPRsα,α′\\alpha,\\alpha^\{\\prime\}, and mixing parameterλ∈\[0,1\]\\lambda\\in\[0,1\], we can always define a test that outputsϕ\\phiwith probabilityλ\\lambdaandϕ′\\phi^\{\\prime\}with probability1−λ1\-\\lambda\. The second property is because the constraint set onϕ\\phiis only larger asα\\alphagrows\.

Finally, the last property is because we can always simultaneously obtain FPRα\\alphaand FNR1−α1\-\\alpha, for anyα∈\[0,1\]\\alpha\\in\[0,1\], by ignoring the sample and uniformly settingϕ=α\\phi=\\alpha\. Iff⁡\(α\)=1−αf\(\\alpha\)=1\-\\alphauniformly, thenPPandQQare undistinguishable by any test, i\.e\.,P=QP=Qalmost everywhere\.

ff\-DP and\(ε,δ\)\(\\varepsilon,\\delta\)\-DP\.The definition offf\-DP is naturally motivated by Definitions[1](https://arxiv.org/html/2608.28934#Thmdefinition1)and[3](https://arxiv.org/html/2608.28934#Thmdefinition3)\.

###### Definition 4\(ff\-DP\)\.

Letf:\[0,1\]→\[0,1\]f:\[0,1\]\\to\[0,1\]be a trade\-off function\. We say that a mechanismℳ:𝒮n→Ω\\mathcal\{M\}:\\mathcal\{S\}^\{n\}\\to\\Omegasatisfiesff\-differential privacy \(or, isff\-DP\) if for all neighboring𝒟,𝒟′∈𝒮n\\mathcal\{D\},\\mathcal\{D\}^\{\\prime\}\\in\\mathcal\{S\}^\{n\},

T𝒜⁡\(𝒟\),𝒜⁡\(𝒟′\)​\(α\)≥f⁡\(α\)​for all​α∈\[0,1\]\.T\_\{\\mathcal\{A\}\(\\mathcal\{D\}\),\\mathcal\{A\}\(\\mathcal\{D\}^\{\\prime\}\)\}\(\\alpha\)\\geq f\(\\alpha\)\\text\{ for all \}\\alpha\\in\[0,1\]\.\(4\)

A higher trade\-off function corresponds to distributions being difficult to distinguish\. Definition[4](https://arxiv.org/html/2608.28934#Thmdefinition4)captures this intuition by positing that every pair𝒜⁡\(𝒟\)\\mathcal\{A\}\(\\mathcal\{D\}\),𝒜⁡\(𝒟′\)\\mathcal\{A\}\(\\mathcal\{D\}^\{\\prime\}\)induced by neighboring datasets is at least hard to distinguish as specified byff\. For example, iff⁡\(α\)=1−αf\(\\alpha\)=1\-\\alphauniformly, then anyff\-DP𝒜\\mathcal\{A\}has𝒜⁡\(𝒟\)=𝒜⁡\(𝒟′\)\\mathcal\{A\}\(\\mathcal\{D\}\)=\\mathcal\{A\}\(\\mathcal\{D\}^\{\\prime\}\)almost everywhere, so its outcome is independent from the dataset it is trained on with probability11\. We call𝒜\\mathcal\{A\}that satisfies such anff\-DP*perfectly private*\.

Importantly,\[[67](https://arxiv.org/html/2608.28934#bib.bib58),[20](https://arxiv.org/html/2608.28934#bib.bib64)\]observed the following equivalence between Definitions[1](https://arxiv.org/html/2608.28934#Thmdefinition1)and[4](https://arxiv.org/html/2608.28934#Thmdefinition4)\.

###### Lemma 3\(Proposition 2\.12,\[[20](https://arxiv.org/html/2608.28934#bib.bib64)\]\)\.

Letf:\[0,1\]→\[0,1\]f:\[0,1\]\\to\[0,1\]be a symmetric trade\-off function\. Then a randomized algorithm𝒜\\mathcal\{A\}isff\-DP iff for allδ∈\(0,1\)\\delta\\in\(0,1\),𝒜\\mathcal\{A\}is\(ε⁡\(δ\),δ\)\(\\varepsilon\(\\delta\),\\delta\)\-DP, where

ε⁡\(δ\):=max⁡\{0,maxα∈\[0,1\]⁡log⁡\(1−f⁡\(α\)−δα\)\},\\varepsilon\(\\delta\):=\\max\\left\\\{0,\\max\_\{\\alpha\\in\[0,1\]\}\\log\\left\(\\frac\{1\-f\(\\alpha\)\-\\delta\}\{\\alpha\}\\right\)\\right\\\},\(5\)

Note that if we increaseffpointwise, applying \([5](https://arxiv.org/html/2608.28934#S2.E5)\) leads to smaller privacy parameters\.

Pipeline\.Following\[[50](https://arxiv.org/html/2608.28934#bib.bib48),[48](https://arxiv.org/html/2608.28934#bib.bib68)\], we can certify privacy lower bounds via Lemma[3](https://arxiv.org/html/2608.28934#Thmlemma3)\. Recall that the true DP functionε⋆\\varepsilon^\{\\star\}\([1](https://arxiv.org/html/2608.28934#S1.E1)\) of an algorithm𝒜\\mathcal\{A\}are the largest parameters attainable via a set of tests \(corresponding to a dataset\-canary\-audit triple \([2](https://arxiv.org/html/2608.28934#S1.E2)\) at eachδ∈\(0,1\)\\delta\\in\(0,1\)\)\. Analogously, one can define the trueff\-DP curve to be the pointwise largestffso that𝒜\\mathcal\{A\}isff\-DP\. Recalling \([4](https://arxiv.org/html/2608.28934#S2.E4)\), each point on this true curve is again realized by a dataset, canary, and auditϕ:Ω→\[0,1\]\\phi:\\Omega\\to\[0,1\]\.

In practice we can only realize a subset of all possible dataset\-canary\-audit triples, and thus the best we can do is to upper bound the trueff\-DP curve as tightly as possible\. Each test is parameterized by neighboring datasets\(𝒟,𝒟′\)\(\\mathcal\{D\},\\mathcal\{D\}^\{\\prime\}\)and a candidate testϕ:Ω→\[0,1\]\\phi:\\Omega\\to\[0,1\]whereΩ\\Omegais the output space of𝒜\\mathcal\{A\}\. Suppose we ascertain that for this particular triple,

𝔼ω∼𝒜⁡\(𝒟\)​\[ϕ⁡\(ω\)\]≤α^,𝔼ω∼𝒜⁡\(𝒟′\)​\[1−ϕ⁡\(ω\)\]≤β^\.\\mathbb\{E\}\_\{\\omega\\sim\\mathcal\{A\}\(\\mathcal\{D\}\)\}\[\\phi\(\\omega\)\]\\leq\\hat\{\\alpha\},\\quad\\mathbb\{E\}\_\{\\omega\\sim\\mathcal\{A\}\(\\mathcal\{D\}^\{\\prime\}\)\}\[1\-\\phi\(\\omega\)\]\\leq\\hat\{\\beta\}\.A standard way to obtain\(α^,β^\)\(\\hat\{\\alpha\},\\hat\{\\beta\}\)is to generate holdout copies of𝒜⁡\(𝒟\)\\mathcal\{A\}\(\\mathcal\{D\}\)and𝒜⁡\(𝒟′\)\\mathcal\{A\}\(\\mathcal\{D\}^\{\\prime\}\), estimate the two expectations above from empirical averages, and apply a Clopper\-Pearson correction\. This certifiesf⋆​\(α\)≤β^f^\{\\star\}\(\\alpha\)\\leq\\hat\{\\beta\}for allα≥α^\\alpha\\geq\\hat\{\\alpha\}, wheref⋆f^\{\\star\}denotes the trueff\-DP curve for𝒜\\mathcal\{A\}\. Any such triple\(ϕ,𝒟,𝒟′\)\(\\phi,\\mathcal\{D\},\\mathcal\{D\}^\{\\prime\}\)yields an empirical upper bound onf⋆f^\{\\star\}, which translates to a lower bound onε⋆\\varepsilon^\{\\star\}via \([5](https://arxiv.org/html/2608.28934#S2.E5)\)\.

A well\-documented difficulty in carrying out this methodology is that unless the number of holdout copies used is \(very\) large, the Clopper\-Pearson correction can dramatically loosen the audit\. As a result, prior works have adopted several heuristic approximations \(multiple hypothesis testing, and use of Gaussian DP curves\) that can inflate the estimatedεlb\\varepsilon\_\{\\textup\{lb\}\}, while sometimes sacrificing formal correctness\. We discuss these heuristics at length in Appendix[A](https://arxiv.org/html/2608.28934#A1), and provide counterexamples flagging situations where their guarantees do not formally apply\.

We make explicit that in the rest of the paper \(up until Appendix[J](https://arxiv.org/html/2608.28934#A10)\), all of our reportedεlb\\varepsilon\_\{\\textup\{lb\}\}use*both*the multiple hypothesis testing and GDP extrapolation heuristics described in Appendix[A](https://arxiv.org/html/2608.28934#A1)\. This reporting is consistent with prior privacy auditing works advertising tight or nearly\-tight audits, which also used these heuristics\[[48](https://arxiv.org/html/2608.28934#bib.bib68),[5](https://arxiv.org/html/2608.28934#bib.bib62),[14](https://arxiv.org/html/2608.28934#bib.bib63)\]\. Notably, by using both heuristics, the reportedεlb\\varepsilon\_\{\\textup\{lb\}\}metrics of our algorithm \(as well as all other defenses\) is as unfavorable as possible\.

For completeness, in Appendix[J](https://arxiv.org/html/2608.28934#A10)we report*all*of the estimatedεlb\\varepsilon\_\{\\textup\{lb\}\}for DP\-SGD and our filtered variants, both with and without these heuristics, and using different sample splits\. Our filtered DP\-SGD variant consistently achieves a lowerεlb\\varepsilon\_\{\\textup\{lb\}\}than the unfiltered implementation against input space, hidden state audits across all reporting methods \(formal or not\)\.

## 3Privacy of Filtering via Sample Signatures

Here we provide a framework to analyze the privacy of augmenting a DP algorithm with filtering\. The unfiltered variants of our algorithms, Algorithm[1](https://arxiv.org/html/2608.28934#algorithm1)and[4](https://arxiv.org/html/2608.28934#algorithm4), have two useful properties\. First, their privacy is based on composition of individual steps’ \(Rényi\) privacy\. Second, the algorithms are permutation\-invariant, in that the same sequence of random operations \(in distribution\) is performed even if we arbitrarily shuffle the data\. These properties hold for many DP algorithms; the latter specifically is natural if assumptions placed on dataset samples are uniform\. Our framework for arguing about privacy with interleaved filtering steps carefully uses these two properties\.

### 3\.1Analysis

The first piece of our framework is a generalized privacy composition theorem\. Intuitively, it states that if before we apply privacy composition, we can arbitrarily replace a pair of datasets with new datasets with the same number of neighbors, this cannot change the final privacy bound\.

###### Lemma 4\(Interleaved Rényi composition\)\.

Let𝒜1:𝒮∗→Ω1\\mathcal\{A\}\_\{1\}:\\mathcal\{S\}^\{\*\}\\to\\Omega\_\{1\}be\(α,ρ1\)\(\\alpha,\\rho\_\{1\}\)\-RDP, and let𝒜2:𝒮∗×Ω1→Ω2\\mathcal\{A\}\_\{2\}:\\mathcal\{S\}^\{\*\}\\times\\Omega\_\{1\}\\to\\Omega\_\{2\}be such that𝒜2​\(⋅,ω1\)\\mathcal\{A\}\_\{2\}\(\\cdot,\\omega\_\{1\}\)is\(α,ρ2\)\(\\alpha,\\rho\_\{2\}\)\-RDP for anyω1∈Ω1\\omega\_\{1\}\\in\\Omega\_\{1\}\. Also, letℱ:𝒮∗×Ω1→𝒮∗\\mathcal\{F\}:\\mathcal\{S\}^\{\*\}\\times\\Omega\_\{1\}\\to\\mathcal\{S\}^\{\*\}be a*filter*such that for any neighboring𝒟,𝒟′∈𝒮∗\\mathcal\{D\},\\mathcal\{D\}^\{\\prime\}\\in\\mathcal\{S\}^\{\*\}, we always have thatℱ⁡\(𝒟,ω1\)\\mathcal\{F\}\(\\mathcal\{D\},\\omega\_\{1\}\)andℱ⁡\(𝒟′,ω1\)\\mathcal\{F\}\(\\mathcal\{D\}^\{\\prime\},\\omega\_\{1\}\)are neighboring for anyω1∈Ω1\\omega\_\{1\}\\in\\Omega\_\{1\}\. Then, the mechanism𝒜\\mathcal\{A\}that first samplesω1←𝒜1​\(𝒟\)\\omega\_\{1\}\\leftarrow\\mathcal\{A\}\_\{1\}\(\\mathcal\{D\}\)and then samplesω2←𝒜2​\(ℱ⁡\(𝒟,ω1\),ω1\)\\omega\_\{2\}\\leftarrow\\mathcal\{A\}\_\{2\}\(\\mathcal\{F\}\(\\mathcal\{D\},\\omega\_\{1\}\),\\omega\_\{1\}\)\(i\.e\., appliesℱ\\mathcal\{F\}to𝒟\\mathcal\{D\}before𝒜2\\mathcal\{A\}\_\{2\}\) is\(α,ρ1\+ρ2\)\(\\alpha,\\rho\_\{1\}\+\\rho\_\{2\}\)\-RDP\.

###### Proof\.

We closely follow the proof of Proposition 1,\[[46](https://arxiv.org/html/2608.28934#bib.bib46)\]\. Fix a pair of outcomesω=\(ω1,ω2\)∈Ω1×Ω2\\omega=\(\\omega\_\{1\},\\omega\_\{2\}\)\\in\\Omega\_\{1\}\\times\\Omega\_\{2\}\. For simplicity, we overload notation so that𝒜1​\(𝒟\)​\(ω1\)\\mathcal\{A\}\_\{1\}\(\\mathcal\{D\}\)\(\\omega\_\{1\}\)refers to the density of outcomes according to𝒜1​\(𝒟\)\\mathcal\{A\}\_\{1\}\(\\mathcal\{D\}\), and define𝒜2​\(𝒟,ω1\)​\(ω2\)\\mathcal\{A\}\_\{2\}\(\\mathcal\{D\},\\omega\_\{1\}\)\(\\omega\_\{2\}\),𝒜​\(𝒟\)​\(ω\)\\mathcal\{A\}\(\\mathcal\{D\}\)\(\\omega\)similarly\. Finally, we use the shorthand𝒟ω1\\mathcal\{D\}\_\{\\omega\_\{1\}\}to meanℱ⁡\(𝒟,ω1\)\\mathcal\{F\}\(\\mathcal\{D\},\\omega\_\{1\}\)for allω1∈Ω1\\omega\_\{1\}\\in\\Omega\_\{1\}\. We have the desired

exp\(\(α−1\)Dα\(𝒜\(𝒟\)∥𝒜\(𝒟′\)\)\)=∫ω∈Ω1×Ω2\(𝒜​\(𝒟\)​\(ω\)𝒜​\(𝒟′\)​\(ω\)\)α𝒜\(𝒟′\)\(ω\)dω\\displaystyle\\exp\\left\(\(\\alpha\-1\)D\_\{\\alpha\}\\left\(\\mathcal\{A\}\(\\mathcal\{D\}\)\\\|\\mathcal\{A\}\(\\mathcal\{D\}^\{\\prime\}\)\\right\)\\right\)=\\int\_\{\\omega\\in\\Omega\_\{1\}\\times\\Omega\_\{2\}\}\\left\(\\frac\{\\mathcal\{A\}\(\\mathcal\{D\}\)\(\\omega\)\}\{\\mathcal\{A\}\(\\mathcal\{D\}^\{\\prime\}\)\(\\omega\)\}\\right\)^\{\\alpha\}\\mathcal\{A\}\(\\mathcal\{D\}^\{\\prime\}\)\(\\omega\)\\textup\{d\}\\omega=∫ω1∈Ω1\(𝒜1​\(𝒟\)​\(ω1\)𝒜1​\(𝒟′\)​\(ω1\)\)α​\(∫ω2∈Ω2\(𝒜2​\(𝒟ω1,ω1\)​\(ω2\)𝒜2​\(𝒟ω1′,ω1\)​\(ω2\)\)α​𝒜2​\(𝒟ω1′,ω1\)​\(ω2\)​d​ω2\)​𝒜1​\(𝒟′\)​\(ω1\)​d​ω1\\displaystyle=\\int\_\{\\omega\_\{1\}\\in\\Omega\_\{1\}\}\\left\(\\frac\{\\mathcal\{A\}\_\{1\}\(\\mathcal\{D\}\)\(\\omega\_\{1\}\)\}\{\\mathcal\{A\}\_\{1\}\(\\mathcal\{D\}^\{\\prime\}\)\(\\omega\_\{1\}\)\}\\right\)^\{\\alpha\}\\left\(\\int\_\{\\omega\_\{2\}\\in\\Omega\_\{2\}\}\\left\(\\frac\{\\mathcal\{A\}\_\{2\}\(\\mathcal\{D\}\_\{\\omega\_\{1\}\},\\omega\_\{1\}\)\(\\omega\_\{2\}\)\}\{\\mathcal\{A\}\_\{2\}\(\\mathcal\{D\}^\{\\prime\}\_\{\\omega\_\{1\}\},\\omega\_\{1\}\)\(\\omega\_\{2\}\)\}\\right\)^\{\\alpha\}\\mathcal\{A\}\_\{2\}\(\\mathcal\{D\}^\{\\prime\}\_\{\\omega\_\{1\}\},\\omega\_\{1\}\)\(\\omega\_\{2\}\)\\textup\{d\}\\omega\_\{2\}\\right\)\\mathcal\{A\}\_\{1\}\(\\mathcal\{D\}^\{\\prime\}\)\(\\omega\_\{1\}\)\\textup\{d\}\\omega\_\{1\}≤exp⁡\(\(α−1\)​ρ2\)​∫ω1∈Ω1\(𝒜1​\(𝒟\)​\(ω1\)𝒜1​\(𝒟′\)​\(ω1\)\)α​𝒜1​\(𝒟′\)​\(ω1\)​d​ω1≤exp⁡\(\(α−1\)​\(ρ1\+ρ2\)\)\.\\displaystyle\\leq\\exp\(\(\\alpha\-1\)\\rho\_\{2\}\)\\int\_\{\\omega\_\{1\}\\in\\Omega\_\{1\}\}\\left\(\\frac\{\\mathcal\{A\}\_\{1\}\(\\mathcal\{D\}\)\(\\omega\_\{1\}\)\}\{\\mathcal\{A\}\_\{1\}\(\\mathcal\{D\}^\{\\prime\}\)\(\\omega\_\{1\}\)\}\\right\)^\{\\alpha\}\\mathcal\{A\}\_\{1\}\(\\mathcal\{D\}^\{\\prime\}\)\(\\omega\_\{1\}\)\\textup\{d\}\\omega\_\{1\}\\leq\\exp\(\(\\alpha\-1\)\(\\rho\_\{1\}\+\\rho\_\{2\}\)\)\.The first inequality used the privacy assumption on𝒜2​\(⋅,ω1\)\\mathcal\{A\}\_\{2\}\(\\cdot,\\omega\_\{1\}\), and that𝒟ω1\\mathcal\{D\}\_\{\\omega\_\{1\}\}and𝒟ω1′\\mathcal\{D\}\_\{\\omega\_\{1\}\}^\{\\prime\}are neighboring, for any choice ofω1\\omega\_\{1\}\. The second inequality used the privacy assumption on𝒜1\\mathcal\{A\}\_\{1\}\. ∎

The strategy in Lemma[4](https://arxiv.org/html/2608.28934#Thmlemma4)is quite general, and we expect it to apply to many if not all settings where privacy composition holds\. For example, it is straightforward to apply the strategy to prove generalized variants of basic and advanced composition under filtering \(Theorems 3\.16, 3\.20,\[[23](https://arxiv.org/html/2608.28934#bib.bib34)\]\)\. This strategy also works for composition of PLD curves, see e\.g\., Lemma C\.3 of\[[20](https://arxiv.org/html/2608.28934#bib.bib64)\], which also first conditions on the shared outcome of𝒜1\\mathcal\{A\}\_\{1\}before applying composition with𝒜2\\mathcal\{A\}\_\{2\}\.

Importantly in our applications to our filtered defense, if we are applying Lemma[4](https://arxiv.org/html/2608.28934#Thmlemma4)sequentially \(e\.g\.,TTtimes to composeTTDP\-SGD steps\),ω1\\omega\_\{1\}in the lemma statement encompasses the entire history ofT−1T\-1prior operations, so the operationsℱ⁡\(⋅,ω1\)\\mathcal\{F\}\(\\cdot,\\omega\_\{1\}\)can depend on all previous iterations\.

The second piece of our framework is an observation that taking two neighboring datasets, scoring samples based on a coupled rule, and dropping thekklargest\-scored samples each, cannot increase the Hamming distance between the datasets\. This observation treats dropped samples as equivalent \(e\.g\., replaced with a standardized dummy sample\), regardless of their indices\. This is where we require permutation\-invariance: otherwise \(if indices are accounted for\) the distance can grow\.444An illustrative example is neighboring size\-22datasets with scores\{1,2\}\\\{1,2\\\}and\{3,2\}\\\{3,2\\\}\. Replacing the largest score with a common dummy causes22\-neighboring datasets, but they remain11\-neighboring if indices are ignored\.Our proof in fact handles both main definitions of neighboring used in the DP literature, namely, the add/remove notion used in our work, as well as the replacement notion\.

###### Lemma 5\.

LetD1,D2D\_\{1\},D\_\{2\}be multisets of real numbers, and letk≤min⁡\{\|D1\|,\|D2\|\}k\\leq\\min\\\{\|D\_\{1\}\|,\|D\_\{2\}\|\\\}\. LetF1F\_\{1\}beD1D\_\{1\}with itskklargest elements removed, and defineF2F\_\{2\}similarly with respect toD2D\_\{2\}\. Then,

\|D1∖D2\|\+\|D2∖D1\|≥\|F1∖F2\|\+\|F2∖F1\|\.\|D\_\{1\}\\setminus D\_\{2\}\|\+\|D\_\{2\}\\setminus D\_\{1\}\|\\geq\|F\_\{1\}\\setminus F\_\{2\}\|\+\|F\_\{2\}\\setminus F\_\{1\}\|\.\(6\)

###### Proof\.

Denotes:=\|D1∩D2\|s:=\|D\_\{1\}\\cap D\_\{2\}\|andt:=\|F1∩F2\|t:=\|F\_\{1\}\\cap F\_\{2\}\|\. Because

\|D1∖D2\|\+\|D2∖D1\|=\(\|D1\|−s\)\+\(\|D2\|−s\)=\|D1\|\+\|D2\|−2​s,\|D\_\{1\}\\setminus D\_\{2\}\|\+\|D\_\{2\}\\setminus D\_\{1\}\|=\\left\(\|D\_\{1\}\|\-s\\right\)\+\\left\(\|D\_\{2\}\|\-s\\right\)=\|D\_\{1\}\|\+\|D\_\{2\}\|\-2s,and similarly the right\-hand side of \([6](https://arxiv.org/html/2608.28934#S3.E6)\) is\|D1\|\+\|D2\|−2​t−2​k\|D\_\{1\}\|\+\|D\_\{2\}\|\-2t\-2k, our goal in \([6](https://arxiv.org/html/2608.28934#S3.E6)\) is equivalent to showing thatt≥s−kt\\geq s\-k\. Ifs≤ks\\leq kthen this is clear\. Assumes\>ks\>khenceforth\.

SortD1∩D2D\_\{1\}\\cap D\_\{2\}in order:x1≥x2≥…≥xsx\_\{1\}\\geq x\_\{2\}\\geq\\ldots\\geq x\_\{s\}\. Sinces\>ks\>k, then for anyj\>kj\>k, we must havexj∈F1∩F2x\_\{j\}\\in F\_\{1\}\\cap F\_\{2\}because we remove thekkhighest scores of each set only \(coupling any copies of duplicated items removed\)\. Thus at leasts−ks\-kcommon elements survive andt≥s−kt\\geq s\-kas claimed\. ∎

Notice that the condition \([6](https://arxiv.org/html/2608.28934#S3.E6)\) captures neighboring\-preservation under filtering, up to assigning indices to the elements \(i\.e\., the locations of the removed elements do not matter\)\. This last caveat explains why our framework requires permutation\-invariance of the algorithm\. Indeed, under the add/remove definition of neighboring, the two sides of \([6](https://arxiv.org/html/2608.28934#S3.E6)\) respectively equal the number of add/remove operations needed to change one set into the other, before and after filtering\. Similarly, under the replacement notion of DP \(Definition 1,\[[22](https://arxiv.org/html/2608.28934#bib.bib33)\]\), the two sides of \([6](https://arxiv.org/html/2608.28934#S3.E6)\) are both twice the number of replacement operations needed for this transformation\.

By combining Lemma[4](https://arxiv.org/html/2608.28934#Thmlemma4)and[5](https://arxiv.org/html/2608.28934#Thmlemma5), we can conclude that any permutation\-invariant algorithm whose privacy analysis is based on RDP composition retains the same privacy bound under arbitrary sample signature\-based filtering\. The fact that filtering is performed based on sample signatures \(so that the score of each sample is the same as long as the model is coupled\) is critical in applying Lemma[5](https://arxiv.org/html/2608.28934#Thmlemma5)\. As an example application, we analyze the privacy of Algorithm[2](https://arxiv.org/html/2608.28934#algorithm2)\.

###### Corollary 1\.

Algorithm[2](https://arxiv.org/html/2608.28934#algorithm2)satisfies the same privacy bound as stated in Theorem[1](https://arxiv.org/html/2608.28934#Thmtheorem1)for Algorithm[1](https://arxiv.org/html/2608.28934#algorithm1)\.

###### Proof\.

Theorem[1](https://arxiv.org/html/2608.28934#Thmtheorem1)proceeds by applying the RDP composition theorem to individual DP\-SGD steps, whose RDP is analyzed directly\. We apply Lemma[4](https://arxiv.org/html/2608.28934#Thmlemma4)in place of standard RDP composition\. Lemma[5](https://arxiv.org/html/2608.28934#Thmlemma5)shows that every time we wish to apply Lemma[4](https://arxiv.org/html/2608.28934#Thmlemma4), the datasets indeed satisfy the precondition of being neighboring, with one subtlety: the indices of deleted samples \(which are effectively replaced by a dummy sample with zero gradient\) are not guaranteed to be coupled\. However, Algorithm[2](https://arxiv.org/html/2608.28934#algorithm2)remains unchanged in distribution if after every filtering step, we randomly permute the dataset \(dummy samples included\)\. We can thus couple the random permutations between runs of Algorithm[2](https://arxiv.org/html/2608.28934#algorithm2)on neighboring datasets, so that the dummy indices are always aligned\. This coupling does not affect the privacy proof, as it preserves marginal distributions\. ∎

### 3\.2Variants of DP\-SGD

In this section, we briefly comment on the applicability of the framework in Section[3\.1](https://arxiv.org/html/2608.28934#S3.SS1)to variants of DP\-SGD that see use in practice, beyond the Poisson\-subsampled variant in Algorithm[1](https://arxiv.org/html/2608.28934#algorithm1)\.

First, consider the variant using randomly\-shuffled minibatches \(as discussed in Section[2\.1](https://arxiv.org/html/2608.28934#S2.SS1), with pseudocode in Algorithm[4](https://arxiv.org/html/2608.28934#algorithm4)\)\. It is straightforward to check that, due to the random permutation at the start of every epoch \(Line[4](https://arxiv.org/html/2608.28934#algorithm4)\), the privacy framework of Section[3\.1](https://arxiv.org/html/2608.28934#S3.SS1)applies if filtering is performed after any number of gradient steps are taken\. This is true whether the privacy accounting is done using the standard \(formal\) accounting based on RDP of the Gaussian mechanism, or the \(heuristic\) “in\-expectation equivalent” variant of Theorem[1](https://arxiv.org/html/2608.28934#Thmtheorem1)discussed in Section[3\.1](https://arxiv.org/html/2608.28934#S3.SS1)\. In Algorithm[5](https://arxiv.org/html/2608.28934#algorithm5), we give pseudocode of one potential implementation of a filtered variant of Algorithm[4](https://arxiv.org/html/2608.28934#algorithm4)\.

Algorithm 4𝖣𝖯𝖲𝖦𝖣𝖲𝗁𝗎𝖿𝖿𝗅𝖾⁡\(𝒟,θ0,m,η,C,σ,N\)\\mathsf\{DPSGDShuffle\}\(\\mathcal\{D\},\\theta\_\{0\},m,\\eta,C,\\sigma,N\)Input:Dataset

𝒟\\mathcal\{D\}, model initialization

θ0\\theta\_\{0\}, batch size

mm, step size

η\\eta, clip bound

CC, noise multiplier

σ\\sigma, epoch count

NN
Output:Final model parameters

θ\\theta
1

θ←θ0\\theta\\leftarrow\\theta\_\{0\}
2for

t∈\[N\]t\\in\[N\]do

3

\{\(xi,yi\)\}i∈\[n\]←\\\{\(x\_\{i\},y\_\{i\}\)\\\}\_\{i\\in\[n\]\}\\leftarrowrandomly permuted ordering of elements of

𝒟\\mathcal\{D\}
4

ℬj←\{\(xi,yi\)\}i∈\[n\]\|j=⌈im⌉\\mathcal\{B\}\_\{j\}\\leftarrow\\left\\\{\(x\_\{i\},y\_\{i\}\)\\right\\\}\_\{i\\in\[n\]\\mid j=\\lceil\\frac\{i\}\{m\}\\rceil\}for all

j∈\[⌈nm⌉\]j\\in\[\\lceil\\frac\{n\}\{m\}\\rceil\]
5for

j∈\[⌈nm⌉\]j\\in\[\\lceil\\frac\{n\}\{m\}\\rceil\]do

6

ξ∼𝒩⁡\(0,\(σ​C\|ℬj\|\)2​I\)\\xi\\sim\\mathcal\{N\}\\Big\(0,\\Big\(\\frac\{\\sigma C\}\{\|\\mathcal\{B\}\_\{j\}\|\}\\Big\)^\{2\}I\\Big\)
7

θ←θ−η⁡\(1\|ℬj\|​∑\(xi,yi\)∈ℬjclip​\(∇θℓ​\(θ,xi,yi\),C\)\+ξ\)\\theta\\leftarrow\\theta\-\\eta\(\\frac\{1\}\{\|\\mathcal\{B\}\_\{j\}\|\}\\sum\_\{\(x\_\{i\},y\_\{i\}\)\\in\\mathcal\{B\}\_\{j\}\}\\text\{clip\}\(\\nabla\_\{\\theta\}\\ell\(\\theta;x\_\{i\},y\_\{i\}\),C\)\+\\xi\)
8end for

9end for

10return

θ\\theta

Algorithm 5𝖥𝗂𝗅𝗍𝖾𝗋𝖾𝖽𝖣𝖯𝖲𝖦𝖣𝖲𝗁𝗎𝖿𝖿𝗅𝖾⁡\(𝒟,θ0,m,η,C,σ,N,f,k,scope\)\\mathsf\{FilteredDPSGDShuffle\}\(\\mathcal\{D\},\\theta\_\{0\},m,\\eta,C,\\sigma,N,f,k,\\texttt\{scope\}\)Input:Dataset

𝒟\\mathcal\{D\}, model initialization

θ0\\theta\_\{0\}, batch size

mm, step size

η\\eta, clip bound

CC, noise multiplier

σ\\sigma, epoch count

NN, sample signature

ff, selection parameter

kk, flag

scope∈\{global,class\}\\texttt\{scope\}\\in\\\{\\texttt\{global\},\\texttt\{class\}\\\}
Output:Final model parameters

θ\\theta
1

\(ℱ,S,θ\)←\(∅,\{0\}\|𝒟\|,θ0\)\(\\mathcal\{F\},S,\\theta\)\\leftarrow\(\\emptyset,\\\{0\\\}^\{\|\\mathcal\{D\}\|\},\\theta\_\{0\}\)
2for

t∈\[N\]t\\in\[N\]do

3

\{\(xi,yi\)\}i∈\[n\]←\\\{\(x\_\{i\},y\_\{i\}\)\\\}\_\{i\\in\[n\]\}\\leftarrowrandomly permuted ordering of elements of

𝒟\\mathcal\{D\}
4

ℬj←\{\(xi,yi\)\}i∈\[n\]\|j=⌈im⌉\\mathcal\{B\}\_\{j\}\\leftarrow\\left\\\{\(x\_\{i\},y\_\{i\}\)\\right\\\}\_\{i\\in\[n\]\\mid j=\\lceil\\frac\{i\}\{m\}\\rceil\}for all

j∈\[⌈nm⌉\]j\\in\[\\lceil\\frac\{n\}\{m\}\\rceil\]
5for

j∈\[⌈nm⌉\]j\\in\[\\lceil\\frac\{n\}\{m\}\\rceil\]do

6foreach

\(xi,yi\)∈ℬj\(x\_\{i\},y\_\{i\}\)\\in\\mathcal\{B\}\_\{j\}do

7

gi←\{clip​\(∇θℓ​\(θ,xi,yi\),C\)i∉ℱ0i∈ℱg\_\{i\}\\leftarrow\\begin\{cases\}\\textup\{clip\}\(\\nabla\_\{\\theta\}\\ell\(\\theta;x\_\{i\},y\_\{i\}\),C\)&i\\not\\in\\mathcal\{F\}\\\\ 0&i\\in\\mathcal\{F\}\\end\{cases\}
8

S⁡\[i\]←fθ​\(xi,yi\)S\[i\]\\leftarrow f\_\{\\theta\}\(x\_\{i\},y\_\{i\}\)
9end foreach

10

ξ∼𝒩⁡\(0,\(σ​C\|ℬj\|\)2​I\)\\xi\\sim\\mathcal\{N\}\(0,\(\\frac\{\\sigma C\}\{\|\\mathcal\{B\}\_\{j\}\|\}\)^\{2\}I\)
11

θ←θ−η⁡\(1\|ℬj\|​∑\(xi,yi\)∈ℬjgi\+ξ\)\\theta\\leftarrow\\theta\-\\eta\(\\frac\{1\}\{\|\\mathcal\{B\}\_\{j\}\|\}\\sum\_\{\(x\_\{i\},y\_\{i\}\)\\in\\mathcal\{B\}\_\{j\}\}g\_\{i\}\+\\xi\)
12end for

13

ℱ←ℱ∪𝖲𝖾𝗅𝖾𝖼𝗍𝖳𝗈𝗉⁡\(S,𝒟,ℱ,k,scope\)\\mathcal\{F\}\\leftarrow\\mathcal\{F\}\\cup\\mathsf\{SelectTop\}\(S,\\mathcal\{D\},\\mathcal\{F\},k,\\texttt\{scope\}\)
14end for

15return

θ\\theta

Second, we observe that unless provable privacy accounting methods change substantially, our framework is likely to extend to future \(permutation\-invariant\) variants of DP\-SGD that are developed\. Indeed, any algorithm that is based on an iterative optimization method, whose individual steps are history\-dependent and data\-dependent, presumably requires an adaptive composition property\. In essentially all existing privacy frameworks, adaptive composition is proven through an argument compatible with Lemma[4](https://arxiv.org/html/2608.28934#Thmlemma4): after conditioning on the outcome of the first mechanism, the proof no longer requires that the datasets be the same in the second mechanism\.

To give an example of this phenomenon, consider the recent algorithm of\[[17](https://arxiv.org/html/2608.28934#bib.bib31)\]\. This scheme is also permutation\-invariant, and its privacy \(as a*random allocation scheme*\) is established using RDP and PLD\[[25](https://arxiv.org/html/2608.28934#bib.bib36),[26](https://arxiv.org/html/2608.28934#bib.bib37)\]\. These accounting frameworks \(particularly, their proofs of adaptive composition\) are compatible with Lemma[4](https://arxiv.org/html/2608.28934#Thmlemma4), and thus a variant of\[[17](https://arxiv.org/html/2608.28934#bib.bib31)\]using sample signature\-based filtering would inherit the privacy analyses of\[[25](https://arxiv.org/html/2608.28934#bib.bib36),[26](https://arxiv.org/html/2608.28934#bib.bib37)\]\.

## 4Privacy\-Preserving Empirically Private Learning

In this section, we overview our evaluations of our framework in the hidden state, input space threat model\. We also provide an evaluation in the gradient space setting in Section[4\.5](https://arxiv.org/html/2608.28934#S4.SS5), and additional experiments in the appendix\. Code for all experiments can be found[here](https://github.com/pineappleEnthusiast/empirical-privacy-defense)\.

### 4\.1Experimental and audit setup

In this section, we describe the experimental setup used to evaluate our defense\. Then, we evaluate our defense framework against a suite of privacy audits\.

Datasets\.We evaluate audits and defenses on three standard supervised learning benchmarks spanning both vision and tabular domains\. For image classification, we use MNIST\[[42](https://arxiv.org/html/2608.28934#bib.bib44)\], a grayscale handwritten digit dataset with 10 classes and 28×28 inputs, and CIFAR\-10\[[40](https://arxiv.org/html/2608.28934#bib.bib42)\], a natural image dataset with 10 classes and 32×32 RGB inputs\. We also use Purchase100\[[58](https://arxiv.org/html/2608.28934#bib.bib51)\], a tabular dataset of customer purchase histories\. Each example is represented as a binary feature vector indicating the presence/absence of purchased items, with 100 class labels\.

Initializations\.We reuse the same model initialization across all training runs for each experiment\. Fixed initialization reduces run\-to\-run variability, which makes it easier to attribute differences in the final model to the data \(e\.g\., whether a canary was included\) rather than to randomness in initialization\. We use Xavier initialization\[[28](https://arxiv.org/html/2608.28934#bib.bib11)\]for CNN\-based models and Kaiming initialization\[[33](https://arxiv.org/html/2608.28934#bib.bib10)\]for WideResNet models\[[70](https://arxiv.org/html/2608.28934#bib.bib6)\]\. All models are trained from scratch\.

Model architectures\.For MNIST, we use a convolutional neural network \(CNN\)\. For CIFAR\-10, we test both a CNN and a WideResNet architecture to demonstrate robustness across different model families\.555Our WideResNet configuration is inspired by the WRN\-16\-4 model of\[[19](https://arxiv.org/html/2608.28934#bib.bib32)\]\. Their results are near SOTA, in particular, achieving 79\.5% test accuracy for CIFAR\-10 atε=8\\varepsilon=8\. Our pipeline is slightly modified from theirs \(e\.g\., smaller batch size, no augmentation multiplicity\), largely due to computational constraints\. Consequently, our model achieves a slightly lower accuracy for the same setting, but we anticipate qualitatively similar findings would hold if our audit were run in their exact setup\.For Purchase100, we use a 3\-layer multilayer perceptron \(MLP\)\.

Training configuration\.For all experiments, unless otherwise specified, we use privacy parametersε\\varepsilon= 10 andδ\\delta=10−510^\{\-5\}for DP\-SGD and our composed defense\. We train forN=100N=100epochs using Poisson subsampling\. We perform hyperparameter grid search to identify learning rates that maximize test accuracy:η\\eta= 3 for MNIST and CIFAR\-10, andη\\eta= 10 for Purchase100\. To disentangle improvements due to the proposed defense from improvements due to tuning, we use the same hyperparameter configurations when auditing with and without our defense\.

Defense configuration\.For all experiments, we run our defense using theL∞L\_\{\\infty\}norm as the scoring function \(Algorithm[2](https://arxiv.org/html/2608.28934#algorithm2), Line[2](https://arxiv.org/html/2608.28934#algorithm2)\) unless otherwise specified\. Ablation studies \(Appendix[D](https://arxiv.org/html/2608.28934#A4)\) show that alternative scoring functions perform comparably\. We use per\-class \(local\) filtering rather than global filtering to align with prior work\[[65](https://arxiv.org/html/2608.28934#bib.bib56)\]\. We find that for blank canaries, global filtering also works if we change the scoring function \(Appendix[I\.1](https://arxiv.org/html/2608.28934#A9.SS1)\)\. For MNIST and CIFAR10, we filter 5 samples from each class per epoch, amounting to dropping8\.3%8\.3\\%and10%10\\%of each training set, respectively\. For Purchase100, we filter 1 sample from each class per epoch, amounting to dropping6\.5%6\.5\\%of the training set\. The choice of\#\\\#samples to drop per class per epoch was arbitrary, as long as the total\#\\\#samples dropped was reasonable \(≤10%\\leq 10\\%\)\.

We find that for blank canaries, our defense is robust to filtering frequency \(Appendix[I\.3](https://arxiv.org/html/2608.28934#A9.SS3)\) and “bandwidth” \(the number of samples discarded, Appendix[I\.2](https://arxiv.org/html/2608.28934#A9.SS2)\)\.

Augmentation multiplicity\.Practitioners sometimes generate multiple augmented versions of the same sample within a single batch and average their gradients before clipping, a technique known as augmentation multiplicity\. This approach is used to improve the privacy\-utility tradeoff by reducing the variance of the gradient estimate for each sample\. While we do not employ augmentation multiplicity in all of our experiments, our evaluation of our defense includes this setting \(cf\. Table[1](https://arxiv.org/html/2608.28934#S4.T1)\) to ensure it remains effective against this commonly\-used privacy\-enhancing modification\.

Audit models and evaluation\.We trainkk= 400 models for each audit: 200 with the canary and 200 without\. While some prior works train more models, we find that with the heuristics in Appendix[A](https://arxiv.org/html/2608.28934#A1)applied, 400 shadow models offers a sufficient scale to observe a substantial gap in attack success between filtered and unfiltered settings\. Our alternativeεlb\\varepsilon\_\{\\textup\{lb\}\}reporting in Appendix[J](https://arxiv.org/html/2608.28934#A10)shows that even without these heuristics, a smaller but noticeable gap persists\. As validation, we also provide an MNIST audit with 1000 models \(Appendix[H](https://arxiv.org/html/2608.28934#A8)\), which shows limited improvement in audit tightness\. Our primary focus is the relative gap in empirical privacy with versus without the filter, so we adopt the same methodology for auditing all optimizers\.

Compute resources\.All experiments were run on NVIDIA H200 GPUs with 96GB HBM3 memory\. Individual audit jobs used between 5 and 20 GPUs and ran for 5–30 hours, with resource requirements scaling according to model architecture and dataset size\. Specific job configurations \(number of GPUs, hours per job, batch scripts\) are available in the scripts directory of our code repository\.

Multi\-canary audits\.For attacks involving multiple canaries, we compute each model’s score as the maximum loss over all canary samples\. Note that theεlb\\varepsilon\_\{\\text\{lb\}\}value reported for these attacks is a measure ofkk\-group privacy, wherekkdenotes the number of canaries; while it is reasonable to expect thekk\-group privacy parameter to scale as≈k×\\approx k\\timesthe11\-group privacy parameter, one cannot formally convertkk\-group privacy into11\-group privacy in this way\.

### 4\.2Input space attacks

Input space attacks operate at the data level by injecting carefully crafted training samples into the dataset\. We first evaluate our defense on a singular canary across varying privacy budgets \(ε∈\{2,4,6,8,10\}\\varepsilon\\in\\\{2,4,6,8,10\\\}\), datasets, model architectures, and training configurations \(“AM4” refers to training with augmentation multiplicity using 4 augmentations\)\. For MNIST and CIFAR\-10, we use a blank \(zero\) canary; for Purchase100, we use a random dense vector canary, as a blank input to an MLP propagates no signal through the network \(the hidden states are driven entirely by the biases\), and is therefore not auditable\.

We find that our defense generally reducesεlb\\varepsilon\_\{\\textup\{lb\}\}with minimal impact on model utility\. Notably, our defense reduced theεlb\\varepsilon\_\{\\textup\{lb\}\}for MNIST on a CNN and CIFAR\-10 on a WRN\-16 \(the model with the test utility on a more “complex” dataset\) to≈\\approx0 for almost every setting ofεub∈\{2,4,6,8,10\}\\varepsilon\_\{\\textup\{ub\}\}\\in\\\{2,4,6,8,10\\\}even after applying both heuristics in Appendix[A](https://arxiv.org/html/2608.28934#A1)\. Moreover, we found that alternative scoring rules detailed in Appendix[D](https://arxiv.org/html/2608.28934#A4), including prediction entropy and gradient kurtosis, substantially reduce the auditableεlb\\varepsilon\_\{\\textup\{lb\}\}for several of the other dataset / model combinations\.

Our defense may not universally lowerεlb\\varepsilon\_\{\\textup\{lb\}\}; MNIST and CIFAR\-10 on WRN\-16 atεub=2\\varepsilon\_\{\\textup\{ub\}\}=2and Purchase atεub=6\\varepsilon\_\{\\textup\{ub\}\}=6observe an increase inεlb\\varepsilon\_\{\\textup\{lb\}\}post\-defense\. We believe that this apparent discrepancy may potentially arise from finite sample error\. In particular, the90%90\\%two\-sided confidence intervals overlap for MNIST atεub=2\\varepsilon\_\{\\textup\{ub\}\}=2and Purchase atεub=6\\varepsilon\_\{\\textup\{ub\}\}=6, so this could just be an artifact of the number of models used\. The90%90\\%confidence interval for CIFAR\-10 atεub=2\\varepsilon\_\{\\textup\{ub\}\}=2without the defense was not very informative, as it was\(0\.0,0\.0\)\(0\.0,0\.0\), but theεlb\\varepsilon\_\{\\textup\{lb\}\}with the defense was also negligible in this case\. Notably, the reversal cases identified either had a smallεlb\\varepsilon\_\{\\textup\{lb\}\}to begin with, or were on the Purchase dataset, previously remarked on as potentially ill\-suited for privacy audits in\[[11](https://arxiv.org/html/2608.28934#bib.bib27)\]\.

While the accuracy gaps are generally contained to±1\\pm 1, notice the anomalous and relatively large accuracy gap for the Purchase dataset post\-defense atεub=10\\varepsilon\_\{\\textup\{ub\}\}=10\. In general, we believe that our finding that the accuracy was minimally affected is fairly robust: on all 24 other tasks, both training and test accuracy dropped by1\.5%1\.5\\%or less\. For the specific Purchase atεub=10\\varepsilon\_\{\\textup\{ub\}\}=10task, we believe the steeper accuracy drop may be due to the simplicity of the dataset \(as highlighted in\[[11](https://arxiv.org/html/2608.28934#bib.bib27)\]\), or due to use of a highεub\\varepsilon\_\{\\textup\{ub\}\}level causing more sensitivity to dropping a small amount of data\.

![Refer to caption](https://arxiv.org/html/2608.28934v1/new_private_tradeoff_curves.png)Figure 1:εlb\\varepsilon\_\{\\textup\{lb\}\}across varyingεub\\varepsilon\_\{\\textup\{ub\}\}budgets\.Table 1:Defense performance on private models across privacy budgetsε∈\{2,4,6,8,10\}\\varepsilon\\in\\\{2,4,6,8,10\\\}\. Training and testing accuracies are reported in the table asNo Defense/Defense\.We also evaluate our defense against harder input space constructions that better reflect membership inference threats\. We test three attack variants on CIFAR\-10 and MNIST \(Table[2](https://arxiv.org/html/2608.28934#S4.T2)\)\. First, we used adversarial FGSM examples, which iteratively apply small bounded perturbations to a sample until the model predicts a target class\[[30](https://arxiv.org/html/2608.28934#bib.bib65)\]\. Second, we tested mislabeled in\-distribution images, where random training samples are assigned incorrect labels uniformly at random from the remaining classes\. Third, we implemented ClipBKD, which identifies the direction of least variance in the training data distribution via singular value decomposition and constructs a canary along the smallest singular vector\[[35](https://arxiv.org/html/2608.28934#bib.bib40)\]\. Across all attacks, our defense reduced empirical privacy loss\.

We note that in this section, we only presented results on single\-canary input space audits; however, see Appendix[C](https://arxiv.org/html/2608.28934#A3)for experiments on input space canaries under multi\-canary audits\.

Table 2:Empirical privacy loss under non\-blank input space attacks forεub=10\\varepsilon\_\{\\textup\{ub\}\}=10\.
### 4\.3Label\-only attack

Prior work has proposed defenses without provable privacy guarantees against membership inference attacks, and follow\-up work has proposed targeted attacks to break them\. In this section, we evaluate one such attack proposed by\[[2](https://arxiv.org/html/2608.28934#bib.bib60)\], which works in the hidden state, input space setting, and is thus consistent with our threat model\. This adaptive, label\-only attack is known to break several previously\-proposed empirical defenses, including HAMP\[[16](https://arxiv.org/html/2608.28934#bib.bib8)\]\.

HAMP combines a training\-time modification with a test\-time output defense\. During training, the model is trained on soft labels rather than hard labels: the target class is assigned probability determined by an entropy threshold, and the remaining probability mass is spread uniformly over the other classes\. The training loss combines a KL\-divergence term \(between the model’s predicted class distribution and the soft label distribution\) with a term that rewards higher entropy of the model’s predicted class distribution, weighted by a tunable regularization strength\. Training uses a batch size of 64, weight decay of 1e\-5, and a learning rate of 0\.5 with a step schedule \(decayed by 10x at epochs 60, 90, and 150\), for 200 epochs\. At test time, HAMP replaces the model’s output logit vector with a logit vector obtained by querying the model on random noise images, sorting those logits, and reordering them to match the rank order of the real prediction’s logits\. This preserves the predicted label while destroying all information about the model’s confidence\.

Follow\-up work\[[2](https://arxiv.org/html/2608.28934#bib.bib60)\]proposes a label\-only attack to break defenses, like HAMP, that rely on confidence masking\. For a target sample, the attack constructs an 18\-dimensional binary feature vector by querying the model on 18 fixed augmentations of the sample \(all combinations of horizontal flip none, flip and pixel shifts\{0,−4,\+4\}\\\{0,\-4,\+4\\\}along each axis\), recording whether the model’s predicted label matches the true label under each augmentation\. A logistic regression classifier \(l2 penalty, C = 1\.0\) is trained in a leave\-one\-out fashion across 64 shadow models, with 500 in\-distribution mislabeled canary samples: for each target model, the classifier is fit on the binary feature vectors and membership labels of all other shadow models, and evaluated on the held\-out target model’s vector\. The classifier’s predicted probability of membership serves as the attack’s membership score\. Attack success is measured by\[[2](https://arxiv.org/html/2608.28934#bib.bib60)\]as TPR at 0\.1% FPR\.

We implemented both HAMP and the label\-only attack in our codebase and reproduced the results of\[[2](https://arxiv.org/html/2608.28934#bib.bib60)\]: the label\-only attack achieves a TPR of 28\.5% at 0\.1% FPR against HAMP \(evaluated on mislabeled canaries\), showing that HAMP’s confidence\-masking can be easily circumvented by a label\-only signal, since the defense only erases confidence information while leaving the predicted label \(and hence the model’s tendency to correctly classify memorized canary samples\) fully intact\.

We then ran our filtering defense, in the non\-private setting, against the same label\-only attack, using the same number of canaries \(500\), shadow models \(64\), and learning rate \(0\.5\) as the HAMP configuration\. We omit HAMP’s momentum and weight decay: weight decay causes numerical instability in our custom WideResNet implementation \(which replaces batch norm with group norm to support vmap\), and we found that omitting momentum yields higher model utility for our defense\.

Under this setup, the attack achieves a TPR of only 0\.95% at 0\.1% FPR against our defense\. This suggests that, unlike HAMP, our defense’s protection is not reliant on obscuring confidence information that a label\-only attack could otherwise exploit; instead, filtering removes the underlying vulnerable canary samples themselves before they are memorized\.

### 4\.4Defense\-aware input space attack

Beyond evaluating previously\-proposed input space canaries in the literature \(in Section[4\.2](https://arxiv.org/html/2608.28934#S4.SS2)\), here we detail an attempt to learn a defense\-aware canary that dodges our filter while remaining auditable\. We formulate this as a bi\-level optimization problem, where the inner loop simulates the training procedure and the outer loop updates the canary\.

Letxxdenote a candidate canary; we want to mapxxtoywy\_\{w\}, a target incorrect label\. For a givenxx, the inner loop trains the model using DP\-SGD and our gradient\-norm filtering defense\. LetK⁡\(x\)K\(x\)denote the epoch at which the defense detectsxx, withK⁡\(x\)=−1K\(x\)=\-1if the canary is never detected\. When the canary is detected on epochK⁡\(x\)K\(x\), letθK⁡\(x\)\\theta\_\{K\(x\)\}denote the model parameters used to compute the defense score on that epoch, and letτK⁡\(x\)\\tau\_\{K\(x\)\}denote the corresponding detection threshold i\.e\. thekk\-th largest gradient score in that epoch\.

ℒ⁡\(x\)=ℓ⁡\(fθ0​\(x\),yw\)\+λ⋅\{max⁡\(0,‖∇θℓ​\(fθK⁡\(x\)​\(x\),yw\)‖∞−τK⁡\(x\)\),if​K​\(x\)≠−1,0,otherwise\\mathcal\{L\}\(x\)=\\ell\\big\(f\_\{\\theta\_\{0\}\}\(x\),\\,y\_\{w\}\\big\)\+\\lambda\\cdot\\begin\{cases\}\\max\\left\(0,\\;\\left\\\|\\nabla\_\{\\theta\}\\ell\\big\(f\_\{\\theta\_\{K\(x\)\}\}\(x\),\\,y\_\{w\}\\big\)\\right\\\|\_\{\\infty\}\-\\tau\_\{K\(x\)\}\\right\),&\\text\{if \}K\(x\)\\neq\-1,\\\\ 0,&\\text\{otherwise\}\\end\{cases\}\(7\)
The first term of \([7](https://arxiv.org/html/2608.28934#S4.E7)\) pushes the canary to be compatible with its assigned label under the initial model\. We use this initial\-model loss as a tractable proxy for auditability because directly optimizing the canary’s loss after training would require differentiating through the entire DP\-SGD training trajectory\. Using the loss at the detection epoch suffers from the same problem and would also create a dependency onK⁡\(x\)K\(x\)\. The model initialization,θ0\\theta\_\{0\}, is set independent of the canary, so this offers a simple differentiable objective with respect toxx\.

The second term of \([7](https://arxiv.org/html/2608.28934#S4.E7)\) penalizes the canary when it has a gradient above the detection threshold, with a penalty proportional to its distance above the detection threshold\.

Theλ\\lambdaparameter controls the trade\-off between auditability and defense evasion\.

The canary is updated using an FGSM\-style step\. For our experiments, we optimize a canary chosen from class 1 and assigned a target label 0\. We use 50 outer optimization steps with a fixed step size of 1e\-2 and 3 independent inner training runs per outer step\.

Table[3](https://arxiv.org/html/2608.28934#S4.T3)shows that across variousλ\\lambda, while we increase the average epoch on which the canary is dropped, we cannot push the drop epoch past the end of training \(100 epochs\)\. We run a full audit on the learned canary with the largest drop epoch, but we find that the initial canary \(pre\-optimization\) yields a higherεlb\\varepsilon\_\{\\textup\{lb\}\}than our learned canary \(2\.453 vs 1\.962\)\. Post defense, the initial canary’sεlb\\varepsilon\_\{\\textup\{lb\}\}drops to 0\.65, while our learned canary’sεlb\\varepsilon\_\{\\textup\{lb\}\}drops to 0\.

Table 3:Defense\-aware canary optimization attempts across different values ofλ\\lambda\.
### 4\.5Gradient space attacks

In this section, we consider defense\-aware canaries that operate under a gradient space threat model\. Our goal is to qualitatively examine the effectiveness of our filtered defense when exposed to a stronger, not always realizable, threat model for canary construction\.

Gradient space attacks\.For gradient space attacks, we follow the construction of\[[14](https://arxiv.org/html/2608.28934#bib.bib63)\], where the attacker uses a holdout model to construct canary gradients as one\-hot vectors in the direction of the largest parameter update:arg⁡maxi⁡\(Δ​θT−Δ​θ0\)\\arg\\max\_\{i\}\(\\Delta\\theta\_\{T\}\-\\Delta\\theta\_\{0\}\), whereθT\\theta\_\{T\}is the final model andθ0\\theta\_\{0\}is the initialization\. The auditor then measures the parameter differenceΔ​θ=θT−θ0\\Delta\\theta=\\theta\_\{T\}\-\\theta\_\{0\}at indexii, corresponding to the one\-hot index of the canary gradient vector\.

Gradient bandwidth attack\.In this attack, we allow the attacker to observe intermediate gradient distributions and defense hyperparameters over the course of training for some holdout model\. The attacker first trackssts\_\{t\}, the 6th largest gradient norm within class 0 at each epochtt\. Then, the attacker constructs 500 copies of a one\-hot gradient space canary, each withL∞L\_\{\\infty\}norm equal tomint⁡st\\min\_\{t\}s\_\{t\}\(the minimum6th6^\{\\text\{th\}\}largest gradient norm observed across all training epochs\)\. This construction exemplifies an attacker with the specific goal of dodging our filtering bandwidth in the gradient space, full state threat model\. If we choose to filter 5 samples from each class on every epoch, the gradient space canaries will remain just below the filtering threshold throughout training\. Under this attack, we obtained \(group privacy\)εlb=27\.26\\varepsilon\_\{\\textup\{lb\}\}=27\.26both with and without the defense\. We emphasize that thisεlb\\varepsilon\_\{\\textup\{lb\}\}is a*group privacy*statistic, so it does not contradict theεub\\varepsilon\_\{\\textup\{ub\}\}setting\.

Gradient canceling attack\.We were able to break our defense more dramatically by using a more complex*gradient canceling*attack\[[38](https://arxiv.org/html/2608.28934#bib.bib2),[43](https://arxiv.org/html/2608.28934#bib.bib1)\], designed with knowledge of our defense’s hyperparameter choices\. This attack exploits our filter by constructing two groups of gradient space canaries with norms that sum to zero\. Group A contains 500 canaries withL∞L\_\{\\infty\}norm 1, while Group B contains 600 canaries withL∞L\_\{\\infty\}norm56\\frac\{5\}\{6\}in the opposite direction\. Without our defense, the two groups cancel each other out during training, yielding anεlb\\varepsilon\_\{\\textup\{lb\}\}near zero\. However, when the defense is active and the defense hyperparameters are chosen so that we only filter Group A, it inadvertently exposes Group B, resulting in non\-zero privacy loss\. We evaluate privacy using the groupεlb\\varepsilon\_\{\\textup\{lb\}\}\(Table[4](https://arxiv.org/html/2608.28934#S4.T4)\)\.

Table 4:Group\-privacyεlb\\varepsilon\_\{\\mathrm\{lb\}\}under the gradient canceling attack on MNIST\.This attack assumes a stronger adversary with knowledge of the defense hyperparameters to choose the canary group sizes\. For example, if we instead construct Group A with 10 canaries and Group B with 5 canaries, even if we only allow filtering 1 canary across the whole dataset per epoch, we can still discard both canary groups from the training data, yieldingεlb=0\\varepsilon\_\{\\textup\{lb\}\}=0\.

We also attempt to reproduce this gradient canceling attack within our intended hidden state, input space threat model by leveraging the structure of a single\-layer MLP, where sample gradients are aligned with sample features by the chain rule\. This suggests that, in principle, one could construct input space canaries whose gradients mimic the desired cancellation behavior; the key challenge is to simulate the desired magnitude, not just the direction, of a gradient space attack in input space\.

To this end, we explore several input space constructions for Group A: \(i\) one\-hot vectors withL∞L\_\{\\infty\}norm exceeding the maximum per\-sample gradient norm observed across all epochs on a holdout set, \(ii\) dense random vectors withL2L\_\{2\}norm similarly exceeding this maximum, and \(iii\) adversarially optimized inputs obtained via FGSM to explicitly induce large gradients\. However, these input space constructions fail to replicate the gradient space attack\. The core issue is that we cannot reliably control the per\-sample gradient magnitudes across training epochs\. As a result, only a small fraction of canaries \(1%1\\%\) are consistently filtered, which is insufficient to disrupt the cancellation effect\.

## Acknowledgments

KT thanks Jonathan Ullman and Florian Tramèr for helpful conversations during this project’s conception\. We thank Milad Nasr for clarifying conversations regarding the prior work\[[48](https://arxiv.org/html/2608.28934#bib.bib68)\], and both Milad Nasr and Thomas Steinke for providing feedback on drafts of this paper\. We are grateful to the Texas Advanced Computing Center \(TACC\) and the UT Austin Center for Generative AI for providing the computing resources used in this project\.

## References

- \[ACG\+16\]M\. Abadi, A\. Chu, I\. Goodfellow, H\. B\. McMahan, I\. Mironov, K\. Talwar, and L\. Zhang\(2016\)Deep learning with differential privacy\.InProceedings of the 2016 ACM Conference on Computer and Communications Security,CCS ’16,pp\. 308–318\.Cited by:[§1](https://arxiv.org/html/2608.28934#S1.p1.2),[§2\.1](https://arxiv.org/html/2608.28934#S2.SS1.p6.1)\.
- \[AZT24\]M\. Aerni, J\. Zhang, and F\. Tramèr\(2024\)Evaluations of machine learning privacy defenses are misleading\.InProceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security,pp\. 1271–1284\.Cited by:[§1\.2](https://arxiv.org/html/2608.28934#S1.SS2.p3.1),[§1\.3](https://arxiv.org/html/2608.28934#S1.SS3.p1.1),[§4\.3](https://arxiv.org/html/2608.28934#S4.SS3.p1.1),[§4\.3](https://arxiv.org/html/2608.28934#S4.SS3.p3.1),[§4\.3](https://arxiv.org/html/2608.28934#S4.SS3.p4.1)\.
- \[AT22\]J\. M\. Altschuler and K\. Talwar\(2022\)Privacy of noisy stochastic gradient descent: more iterations without more privacy loss\.InAdvances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022,Cited by:[§1](https://arxiv.org/html/2608.28934#S1.p2.1)\.
- \[ATM\+21\]G\. Andrew, O\. Thakkar, B\. McMahan, and S\. Ramaswamy\(2021\)Differentially private learning with adaptive clipping\.InAdvances in Neural Information Processing Systems 34,NeurIPS ’21\.Cited by:[§1\.4](https://arxiv.org/html/2608.28934#S1.SS4.p3.1)\.
- \[AC24\]M\. S\. M\. S\. Annamalai and E\. D\. Cristofaro\(2024\)Nearly tight black\-box auditing of differentially private machine learning\.InAdvances in Neural Information Processing Systems 37,NeurIPS ’24,pp\. 131482–131502\.Cited by:[§A\.1](https://arxiv.org/html/2608.28934#A1.SS1.p10.1),[§A\.1](https://arxiv.org/html/2608.28934#A1.SS1.p4.1),[§1\.1](https://arxiv.org/html/2608.28934#S1.SS1.p3.1),[§1\.2](https://arxiv.org/html/2608.28934#S1.SS2.p3.1),[§2\.4](https://arxiv.org/html/2608.28934#S2.SS4.p13.1),[Abstract](https://arxiv.org/html/2608.28934#abstract1.1)\.
- \[ABG\+19\]M\. Arjovsky, L\. Bottou, I\. Gulrajani, and D\. Lopez\-Paz\(2019\)Invariant risk minimization\.CoRRabs/1907\.02893\.Cited by:[Appendix B](https://arxiv.org/html/2608.28934#A2.p1.1)\.
- \[BPS19\]E\. Bagdasaryan, O\. Poursaeed, and V\. Shmatikov\(2019\)Differential privacy has disparate impact on model accuracy\.InAdvances in Neural Information Processing Systems 32,NeurIPS ’19,pp\. 15479–15488\.Cited by:[Appendix B](https://arxiv.org/html/2608.28934#A2.p4.1),[§1\.3](https://arxiv.org/html/2608.28934#S1.SS3.p5.1)\.
- \[BST14\]R\. Bassily, A\. Smith, and A\. Thakurta\(2014\)Private empirical risk minimization: efficient algorithms and tight error bounds\.InProceedings of the 55th Annual IEEE Symposium on Foundations of Computer Science,FOCS ’14,pp\. 464–473\.Cited by:[§1](https://arxiv.org/html/2608.28934#S1.p1.2)\.
- \[BLI\+25\]M\. Boglioni, T\. Liu, A\. Ilyas, and Z\. S\. Wu\(2025\)Optimizing canaries for privacy auditing with metagradient descent\.CoRRabs/2507\.15836\.Cited by:[§1\.1](https://arxiv.org/html/2608.28934#S1.SS1.p3.1),[§2\.2](https://arxiv.org/html/2608.28934#S2.SS2.p3.1)\.
- \[BV04\]S\. Boyd and L\. Vandenberghe\(2004\)Convex optimization\.Cambridge university press\.Cited by:[footnote 6](https://arxiv.org/html/2608.28934#footnote6)\.
- \[CCN\+22a\]N\. Carlini, S\. Chien, M\. Nasr, S\. Song, A\. Terzis, and F\. Tramèr\(2022\)Membership inference attacks from first principles\.In2022 IEEE Symposium on Security and Privacy \(SP\),pp\. 1897–1914\.External Links:[Document](https://dx.doi.org/10.1109/SP46214.2022.9833649)Cited by:[§1\.2](https://arxiv.org/html/2608.28934#S1.SS2.p3.1),[§1\.3](https://arxiv.org/html/2608.28934#S1.SS3.p1.1),[§2\.4](https://arxiv.org/html/2608.28934#S2.SS4.p1.1),[§4\.2](https://arxiv.org/html/2608.28934#S4.SS2.p3.1),[§4\.2](https://arxiv.org/html/2608.28934#S4.SS2.p4.1)\.
- \[CJZ\+22b\]N\. Carlini, M\. Jagielski, C\. Zhang, N\. Papernot, A\. Terzis, and F\. Tramer\(2022\)The privacy onion effect: memorization is relative\.InAdvances in Neural Information Processing Systems 35,NeurIPS ’22,pp\. 13263–13276\.Cited by:[6th item](https://arxiv.org/html/2608.28934#A1.I2.i6.p1.1),[Appendix G](https://arxiv.org/html/2608.28934#A7.p1.1),[§1\.2](https://arxiv.org/html/2608.28934#S1.SS2.p3.1),[§1\.3](https://arxiv.org/html/2608.28934#S1.SS3.p1.1),[§1\.3](https://arxiv.org/html/2608.28934#S1.SS3.p5.1)\.
- \[CTW\+21\]N\. Carlini, F\. Tramèr, E\. Wallace, M\. Jagielski, A\. Herbert\-Voss, K\. Lee, A\. Roberts, T\. Brown, D\. Song, U\. Erlingsson, A\. Oprea, and C\. Raffel\(2021\)Extracting training data from large language models\.In30th USENIX Security Symposium,USENIX Security ’21,pp\. 2633–2650\.Cited by:[§1](https://arxiv.org/html/2608.28934#S1.p1.1)\.
- \[CBP25\]T\. I\. Cebere, A\. Bellet, and N\. Papernot\(2025\)Tighter privacy auditing of DP\-SGD in the hidden state threat model\.InThe Thirteenth International Conference on Learning Representations, ICLR 2025,Cited by:[§A\.1](https://arxiv.org/html/2608.28934#A1.SS1.p10.1),[§A\.1](https://arxiv.org/html/2608.28934#A1.SS1.p12.1),[§A\.1](https://arxiv.org/html/2608.28934#A1.SS1.p4.1),[§1\.1](https://arxiv.org/html/2608.28934#S1.SS1.p3.1),[§1\.2](https://arxiv.org/html/2608.28934#S1.SS2.p3.1),[§1](https://arxiv.org/html/2608.28934#S1.p2.1),[§2\.2](https://arxiv.org/html/2608.28934#S2.SS2.p3.1),[§2\.4](https://arxiv.org/html/2608.28934#S2.SS4.p13.1),[§4\.5](https://arxiv.org/html/2608.28934#S4.SS5.p2.1),[Abstract](https://arxiv.org/html/2608.28934#abstract1.1),[footnote 1](https://arxiv.org/html/2608.28934#footnote1)\.
- \[CYF22\]D\. Chen, N\. Yu, and M\. Fritz\(2022\)RelaxLoss: defending membership inference attacks without losing utility\.InThe Tenth International Conference on Learning Representations, ICLR 2022,Cited by:[§1\.4](https://arxiv.org/html/2608.28934#S1.SS4.p2.1)\.
- \[CP24\]Z\. Chen and K\. Pattabiraman\(2024\)Overconfidence is a dangerous thing: mitigating membership inference attacks by enforcing less confident prediction\.In31st Annual Network and Distributed System Security Symposium, NDSS 2024,Cited by:[§1\.4](https://arxiv.org/html/2608.28934#S1.SS4.p2.1),[§4\.3](https://arxiv.org/html/2608.28934#S4.SS3.p1.1)\.
- \[CGH\+25\]L\. Chua, B\. Ghazi, C\. Harrison, P\. Kamath, R\. Kumar, E\. Leeman, P\. Manurangsi, A\. Sinha, and C\. Zhang\(2025\)Balls\-and\-bins sampling for DP\-SGD\.InInternational Conference on Artificial Intelligence and Statistics, AISTATS 2025,Proceedings of Machine Learning Research,pp\. 946–954\.Cited by:[§2\.1](https://arxiv.org/html/2608.28934#S2.SS1.p7.1),[§3\.2](https://arxiv.org/html/2608.28934#S3.SS2.p4.1)\.
- \[CGK\+24\]L\. Chua, B\. Ghazi, P\. Kamath, R\. Kumar, P\. Manurangsi, A\. Sinha, and C\. Zhang\(2024\)How private are DP\-SGD implementations?\.InProceedings of the 41st International Conference on Machine Learning,ICML ’24\.Cited by:[§2\.1](https://arxiv.org/html/2608.28934#S2.SS1.p7.1)\.
- \[DBH\+22\]S\. De, L\. Berrada, J\. Hayes, S\. L\. Smith, and B\. Balle\(2022\)Unlocking high\-accuracy differentially private image classification through scale\.arXiv preprint arXiv:2204\.13650\.Cited by:[§1\.1](https://arxiv.org/html/2608.28934#S1.SS1.p1.1),[§1\.4](https://arxiv.org/html/2608.28934#S1.SS4.p4.1),[§2\.1](https://arxiv.org/html/2608.28934#S2.SS1.p7.1),[footnote 5](https://arxiv.org/html/2608.28934#footnote5)\.
- \[DRS22\]J\. Dong, A\. Roth, and W\. J\. Su\(2022\)Gaussian differential privacy\.Journal of the Royal Statistical Society Series B: Statistical Methodology84\(1\),pp\. 3–37\.Cited by:[§A\.1](https://arxiv.org/html/2608.28934#A1.SS1.p10.1),[§A\.1](https://arxiv.org/html/2608.28934#A1.SS1.p12.1),[§2\.4](https://arxiv.org/html/2608.28934#S2.SS4.p3.1),[§2\.4](https://arxiv.org/html/2608.28934#S2.SS4.p8.1),[§3\.1](https://arxiv.org/html/2608.28934#S3.SS1.p3.1),[Lemma 2](https://arxiv.org/html/2608.28934#Thmlemma2),[Lemma 3](https://arxiv.org/html/2608.28934#Thmlemma3)\.
- \[DLC\+21\]J\. Du, S\. Li, X\. Chen, S\. Chen, and M\. Hong\(2021\)Dynamic differential\-privacy preserving sgd\.arXiv preprint arXiv:2111\.00173\.Cited by:[§1\.4](https://arxiv.org/html/2608.28934#S1.SS4.p3.1)\.
- \[DMN\+06\]C\. Dwork, F\. McSherry, K\. Nissim, and A\. Smith\(2006\)Calibrating noise to sensitivity in private data analysis\.InProceedings of the 3rd Conference on Theory of Cryptography,TCC ’06,Berlin, Heidelberg,pp\. 265–284\.Cited by:[§1](https://arxiv.org/html/2608.28934#S1.p1.1),[§3\.1](https://arxiv.org/html/2608.28934#S3.SS1.p8.1)\.
- \[DR14\]C\. Dwork and A\. Roth\(2014\)The algorithmic foundations of differential privacy\.Foundations and Trends® in Theoretical Computer Science9\(3–4\),pp\. 211–407\.Cited by:[§1](https://arxiv.org/html/2608.28934#S1.p1.2),[§3\.1](https://arxiv.org/html/2608.28934#S3.SS1.p3.1)\.
- \[DRV10\]C\. Dwork, G\. N\. Rothblum, and S\. Vadhan\(2010\)Boosting and differential privacy\.InProceedings of the 51st Annual IEEE Symposium on Foundations of Computer Science,FOCS ’10,pp\. 51–60\.Cited by:[§1](https://arxiv.org/html/2608.28934#S1.p1.2)\.
- \[FS25\]V\. Feldman and M\. Shenfeld\(2025\)Privacy amplification by random allocation\.CoRRabs/2502\.08202\.Cited by:[§3\.2](https://arxiv.org/html/2608.28934#S3.SS2.p4.1)\.
- \[FS26\]V\. Feldman and M\. Shenfeld\(2026\)Efficient privacy loss accounting for subsampling and random allocation\.CoRRabs/2602\.17284\.Cited by:[§3\.2](https://arxiv.org/html/2608.28934#S3.SS2.p4.1)\.
- \[FZ21\]V\. Feldman and T\. Zrnic\(2021\)Individual privacy accounting via a Renyi filter\.InAdvances in Neural Information Processing Systems 34,NeurIPS ’21\.Cited by:[§1\.4](https://arxiv.org/html/2608.28934#S1.SS4.p5.1)\.
- \[GB10\]X\. Glorot and Y\. Bengio\(2010\)Understanding the difficulty of training deep feedforward neural networks\.InProceedings of the 13th International Conference on Artificial Intelligence and Statistics,AISTATS ’10,pp\. 249–256\.Cited by:[§4\.1](https://arxiv.org/html/2608.28934#S4.SS1.p3.1)\.
- \[GKK\+26\]J\. F\. Gomez, B\. Kulynych, G\. Kaissis, F\. du Pin Calmon, J\. Hayes, B\. Balle, and A\. Honkela\(2026\)Gaussian dp for reporting differential privacy guarantees in machine learning\.In2026 IEEE Conference on Secure and Trustworthy Machine Learning \(SaTML\),SaTML ’26\.Cited by:[§A\.1](https://arxiv.org/html/2608.28934#A1.SS1.p11.1)\.
- \[GSS15\]I\. J\. Goodfellow, J\. Shlens, and C\. Szegedy\(2015\)Explaining and harnessing adversarial examples\.InInternational Conference on Learning Representations,ICLR ’15\.Cited by:[§4\.2](https://arxiv.org/html/2608.28934#S4.SS2.p5.1)\.
- \[GOO19\]Google\(2019\)TensorFlow privacy\.Google\.Note:[https://www\.tensorflow\.org/responsible\_ai/privacy/api\_docs/python/tf\_privacy](https://www.tensorflow.org/responsible_ai/privacy/api_docs/python/tf_privacy)Cited by:[§2\.1](https://arxiv.org/html/2608.28934#S2.SS1.p6.1),[footnote 3](https://arxiv.org/html/2608.28934#footnote3)\.
- \[HKS\+21\]J\. Hayase, W\. Kong, R\. Somani, and S\. Oh\(2021\)Defense against backdoor attacks via robust covariance estimation\.InProceedings of the 38th International Conference on Machine Learning, ICML 2021,Proceedings of Machine Learning Research, Vol\.139,pp\. 4129–4139\.Cited by:[§1\.2](https://arxiv.org/html/2608.28934#S1.SS2.p4.1),[§2\.3](https://arxiv.org/html/2608.28934#S2.SS3.p2.1),[§2\.3](https://arxiv.org/html/2608.28934#S2.SS3.p6.1)\.
- \[HZR\+15\]K\. He, X\. Zhang, S\. Ren, and J\. Sun\(2015\)Delving deep into rectifiers: surpassing human\-level performance on imagenet classification\.InProceedings of the 2015 IEEE International Conference on Computer Vision,ICCV ’15,pp\. 1026–1034\.Cited by:[§4\.1](https://arxiv.org/html/2608.28934#S4.SS1.p3.1)\.
- \[HSR\+08\]N\. Homer, S\. Szelinger, M\. Redman, D\. Duggan, W\. Tembe, J\. Muehling, J\. V\. Pearson, D\. A\. Stephan, S\. F\. Nelson, and D\. W\. Craig\(2008\)Resolving individuals contributing trace amounts of DNA to highly complex mixtures using high\-density SNP genotyping microarrays\.PLoS Genetics4\(8\),pp\. 1–9\.Cited by:[§1](https://arxiv.org/html/2608.28934#S1.p1.1)\.
- \[JUO20\]M\. Jagielski, J\. Ullman, and A\. Oprea\(2020\)Auditing differentially private machine learning: how private is private sgd?\.InAdvances in Neural Information Processing Systems 33,NeurIPS ’20\.Cited by:[§A\.1](https://arxiv.org/html/2608.28934#A1.SS1.p3.1),[§1\.1](https://arxiv.org/html/2608.28934#S1.SS1.p2.1),[§1\.4](https://arxiv.org/html/2608.28934#S1.SS4.p5.1),[§4\.2](https://arxiv.org/html/2608.28934#S4.SS2.p5.1)\.
- \[JSB\+19\]J\. Jia, A\. Salem, M\. Backes, Y\. Zhang, and N\. Z\. Gong\(2019\)MemGuard: defending against black\-box membership inference attacks via adversarial examples\.InProceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, CCS 2019,pp\. 259–274\.Cited by:[§1\.4](https://arxiv.org/html/2608.28934#S1.SS4.p2.1)\.
- \[KMS\+21\]P\. Kairouz, B\. McMahan, S\. Song, O\. Thakkar, A\. Thakurta, and Z\. Xu\(2021\)Practical and private \(deep\) learning without sampling or shuffling\.InProceedings of the 38th International Conference on Machine Learning,ICML ’21,pp\. 5213–5225\.Cited by:[§1\.4](https://arxiv.org/html/2608.28934#S1.SS4.p4.1)\.
- \[KSL22\]P\. W\. Koh, J\. Steinhardt, and P\. Liang\(2022\)Stronger data poisoning attacks break data sanitization defenses\.Machine Learning111\(1\),pp\. 1–47\.Cited by:[§4\.5](https://arxiv.org/html/2608.28934#S4.SS5.p4.1)\.
- \[KJH20\]A\. Koskela, J\. Jälkö, and A\. Honkela\(2020\)Computing tight differential privacy guarantees using fft\.InProceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics,Proceedings of Machine Learning Research,pp\. 2560–2569\.Cited by:[§2\.1](https://arxiv.org/html/2608.28934#S2.SS1.p6.1)\.
- \[KRI09\]A\. Krizhevsky\(2009\)Learning multiple layers of features from tiny images\.Technical reportUniversity of Toronto\.Cited by:[§4\.1](https://arxiv.org/html/2608.28934#S4.SS1.p2.1)\.
- \[LRK\+25\]C\. J\. Lebeda, M\. Regehr, G\. Kamath, and T\. Steinke\(2025\)Avoiding pitfalls for privacy accounting of subsampled mechanisms under composition\.InThird IEEE Conference on Secure and Trustworthy Machine Learning,SaTML ’25\.Cited by:[§2\.1](https://arxiv.org/html/2608.28934#S2.SS1.p7.1)\.
- \[LBB\+98\]Y\. LeCun, L\. Bottou, Y\. Bengio, and P\. Haffner\(1998\)Gradient\-based learning applied to document recognition\.Proceedings of the IEEE86\(11\),pp\. 2278–2324\.Cited by:[§4\.1](https://arxiv.org/html/2608.28934#S4.SS1.p2.1)\.
- \[LKY23\]Y\. Lu, G\. Kamath, and Y\. Yu\(2023\)Exploring the limits of model\-targeted indiscriminate data poisoning attacks\.InProceedings of the 40th International Conference on Machine Learning,ICML ’23,pp\. 22856–22879\.Cited by:[§4\.5](https://arxiv.org/html/2608.28934#S4.SS5.p4.1)\.
- \[MSS23\]S\. Maddock, A\. Sablayrolles, and P\. Stock\(2023\)CANIFE: crafting canaries for empirical privacy measurement in federated learning\.InProceedings of the 11th International Conference on Learning Representations,ICLR ’23\.Cited by:[§A\.1](https://arxiv.org/html/2608.28934#A1.SS1.p4.1)\.
- \[MTZ19\]I\. Mironov, K\. Talwar, and L\. Zhang\(2019\)Rényi differential privacy of the sampled gaussian mechanism\.arXiv preprint arXiv:1908\.10530\.Cited by:[§2\.1](https://arxiv.org/html/2608.28934#S2.SS1.p6.1),[Theorem 1](https://arxiv.org/html/2608.28934#Thmtheorem1),[footnote 1](https://arxiv.org/html/2608.28934#footnote1)\.
- \[MIR17\]I\. Mironov\(2017\)Rényi differential privacy\.InProceedings of the 30th IEEE Computer Security Foundations Symposium,CSF ’17,pp\. 263–275\.Cited by:[§1](https://arxiv.org/html/2608.28934#S1.p1.2),[§2\.1](https://arxiv.org/html/2608.28934#S2.SS1.p4.1),[§3\.1](https://arxiv.org/html/2608.28934#S3.SS1.p2.1.1),[Lemma 1](https://arxiv.org/html/2608.28934#Thmlemma1)\.
- \[MSH\+22\]S\. Mohapatra, S\. Sasy, X\. He, G\. Kamath, and O\. Thakkar\(2022\)The role of adaptive optimizers for honest private hyperparameter selection\.InProceedings of the Thirty\-Sixth AAAI Conference on Artificial Intelligence,AAAI ’22, Vol\.36,pp\. 7806–7813\.Cited by:[§A\.1](https://arxiv.org/html/2608.28934#A1.SS1.p6.1)\.
- \[NHS\+23\]M\. Nasr, J\. Hayes, T\. Steinke, B\. Balle, F\. Tramèr, M\. Jagielski, N\. Carlini, and A\. Terzis\(2023\)Tight auditing of differentially private machine learning\.In32nd USENIX Security Symposium, USENIX Security 2023,pp\. 1631–1648\.Cited by:[§A\.1](https://arxiv.org/html/2608.28934#A1.SS1.p10.1),[§A\.1](https://arxiv.org/html/2608.28934#A1.SS1.p11.1),[§A\.1](https://arxiv.org/html/2608.28934#A1.SS1.p4.1),[§1\.1](https://arxiv.org/html/2608.28934#S1.SS1.p3.1),[§1\.2](https://arxiv.org/html/2608.28934#S1.SS2.p3.1),[§2\.4](https://arxiv.org/html/2608.28934#S2.SS4.p1.1),[§2\.4](https://arxiv.org/html/2608.28934#S2.SS4.p10.1),[§2\.4](https://arxiv.org/html/2608.28934#S2.SS4.p13.1),[Acknowledgments](https://arxiv.org/html/2608.28934#Sx1.p1.1),[Abstract](https://arxiv.org/html/2608.28934#abstract1.1),[footnote 1](https://arxiv.org/html/2608.28934#footnote1)\.
- \[NSH18\]M\. Nasr, R\. Shokri, and A\. Houmansadr\(2018\)Machine learning with membership privacy using adversarial regularization\.InProceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, CCS 2018,pp\. 634–646\.Cited by:[§1\.4](https://arxiv.org/html/2608.28934#S1.SS4.p2.1)\.
- \[NST\+21\]M\. Nasr, S\. Song, A\. Thakurta, N\. Papernot, and N\. Carlini\(2021\)Adversary instantiation: lower bounds for differentially private machine learning\.In2021 IEEE Symposium on Security and Privacy \(SP\),SP ’21,pp\. 866–882\.Cited by:[§1\.1](https://arxiv.org/html/2608.28934#S1.SS1.p2.1),[§1\.1](https://arxiv.org/html/2608.28934#S1.SS1.p3.1),[§1\.2](https://arxiv.org/html/2608.28934#S1.SS2.p3.1),[§2\.4](https://arxiv.org/html/2608.28934#S2.SS4.p10.1)\.
- \[PAE\+17\]N\. Papernot, M\. Abadi, U\. Erlingsson, I\. Goodfellow, and K\. Talwar\(2017\)Semi\-supervised knowledge transfer for deep learning from private training data\.InProceedings of the 5th International Conference on Learning Representations,ICLR ’17\.Cited by:[§1\.4](https://arxiv.org/html/2608.28934#S1.SS4.p3.1)\.
- \[PS22\]N\. Papernot and T\. Steinke\(2022\)Hyperparameter tuning with renyi differential privacy\.InProceedings of the 10th International Conference on Learning Representations,ICLR ’22\.Cited by:[§A\.1](https://arxiv.org/html/2608.28934#A1.SS1.p6.1)\.
- \[PUC\+25\]K\. Pillutla, J\. Upadhyay, C\. A\. Choquette\-Choo, K\. Dvijotham, A\. Ganesh, M\. Henzinger, J\. Katz, R\. McKenna, H\. B\. McMahan, K\. Rush, T\. Steinke, and A\. Thakurta\(2025\)Correlated noise mechanisms for differentially private learning\.arXiv preprint arXiv:2506\.08201\.Cited by:[§1\.4](https://arxiv.org/html/2608.28934#S1.SS4.p4.1)\.
- \[PHK\+23\]N\. Ponomareva, H\. Hazimeh, A\. Kurakin, Z\. Xu, C\. Denison, H\. B\. McMahan, S\. Vassilvitskii, S\. Chien, and A\. G\. Thakurta\(2023\)How to dp\-fy ml: a practical guide to machine learning with differential privacy\.Journal of Artificial Intelligence Research77,pp\. 1113–1201\.External Links:ISSN 1076\-9757,[Link](http://dx.doi.org/10.1613/jair.1.14649),[Document](https://dx.doi.org/10.1613/jair.1.14649)Cited by:[§A\.1](https://arxiv.org/html/2608.28934#A1.SS1.p6.1),[§2\.1](https://arxiv.org/html/2608.28934#S2.SS1.p7.1)\.
- \[ROC97\]R\. T\. Rockafellar\(1997\)Convex analysis\.Vol\.28,Princeton university press\.Cited by:[§A\.1](https://arxiv.org/html/2608.28934#A1.SS1.p8.2.1)\.
- \[SKH\+20\]S\. Sagawa, P\. W\. Koh, T\. B\. Hashimoto, and P\. Liang\(2020\)Distributionally robust neural networks\.In8th International Conference on Learning Representations, ICLR 2020,Cited by:[Appendix B](https://arxiv.org/html/2608.28934#A2.p1.1)\.
- \[SH21\]V\. Shejwalkar and A\. Houmansadr\(2021\)Membership privacy for machine learning models through knowledge transfer\.InThirty\-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty\-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021,pp\. 9549–9557\.Cited by:[§1\.4](https://arxiv.org/html/2608.28934#S1.SS4.p2.1)\.
- \[SSS\+17\]R\. Shokri, M\. Stronati, C\. Song, and V\. Shmatikov\(2017\)Membership inference attacks against machine learning models\.InProceedings of the 38th IEEE Symposium on Security and Privacy,SP ’17,pp\. 3–18\.Cited by:[§1](https://arxiv.org/html/2608.28934#S1.p1.1),[§4\.1](https://arxiv.org/html/2608.28934#S4.SS1.p2.1)\.
- \[SM21\]L\. Song and P\. Mittal\(2021\)Systematic evaluation of privacy risks of machine learning models\.In30th USENIX Security Symposium, USENIX Security 2021,pp\. 2615–2632\.Cited by:[§1\.2](https://arxiv.org/html/2608.28934#S1.SS2.p3.1),[§1\.3](https://arxiv.org/html/2608.28934#S1.SS3.p1.1)\.
- \[SCS13\]S\. Song, K\. Chaudhuri, and A\. D\. Sarwate\(2013\)Stochastic gradient descent with differentially private updates\.InProceedings of the 2013 IEEE Global Conference on Signal and Information Processing,GlobalSIP ’13,pp\. 245–248\.Cited by:[§1](https://arxiv.org/html/2608.28934#S1.p1.2)\.
- \[SNJ23\]T\. Steinke, M\. Nasr, and M\. Jagielski\(2023\)Privacy auditing with one \(1\) training run\.InAdvances in Neural Information Processing Systems 36,NeurIPS ’23,pp\. 49268–49280\.Cited by:[2nd item](https://arxiv.org/html/2608.28934#A1.I2.i2.p1.1),[§C\.1](https://arxiv.org/html/2608.28934#A3.SS1.p1.1)\.
- \[TSL24\]Q\. Tang, F\. Shpilevskiy, and M\. Lécuyer\(2024\)Dp\-adambc: your dp\-adam is actually dp\-sgd \(unless you apply bias correction\)\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.38,pp\. 15276–15283\.Cited by:[§1\.4](https://arxiv.org/html/2608.28934#S1.SS4.p3.1)\.
- \[TMS\+22\]X\. Tang, S\. Mahloujifar, L\. Song, V\. Shejwalkar, M\. Nasr, A\. Houmansadr, and P\. Mittal\(2022\)Mitigating membership inference attacks by self\-distillation through a novel ensemble architecture\.In31st USENIX Security Symposium, USENIX Security 2022,pp\. 1433–1450\.Cited by:[§1\.4](https://arxiv.org/html/2608.28934#S1.SS4.p2.1)\.
- \[TB21\]F\. Tramèr and D\. Boneh\(2021\)Differentially private learning needs better features \(or much more data\)\.InProceedings of the 9th International Conference on Learning Representations,ICLR ’21\.Cited by:[§1\.1](https://arxiv.org/html/2608.28934#S1.SS1.p1.1)\.
- \[TLM18\]B\. Tran, J\. Li, and A\. Madry\(2018\)Spectral signatures in backdoor attacks\.InAdvances in Neural Information Processing Systems 31,NeurIPS ’18,pp\. 8011–8021\.Cited by:[Appendix D](https://arxiv.org/html/2608.28934#A4.p1.1),[§I\.1](https://arxiv.org/html/2608.28934#A9.SS1.p1.1),[§1\.2](https://arxiv.org/html/2608.28934#S1.SS2.p4.1),[§2\.3](https://arxiv.org/html/2608.28934#S2.SS3.p2.1),[§2\.3](https://arxiv.org/html/2608.28934#S2.SS3.p6.1),[§4\.1](https://arxiv.org/html/2608.28934#S4.SS1.p6.1)\.
- \[UNK\+21\]A\. Uniyal, R\. Naidu, S\. Kotti, S\. Singh, P\. J\. Kenfack, F\. Mireshghallah, and A\. Trask\(2021\)DP\-SGD vs PATE: which has less disparate impact on model accuracy?\.CoRRabs/2106\.12576\.Cited by:[Appendix B](https://arxiv.org/html/2608.28934#A2.p4.1),[§1\.3](https://arxiv.org/html/2608.28934#S1.SS3.p5.1)\.
- \[WZ10\]L\. Wasserman and S\. Zhou\(2010\)A statistical framework for differential privacy\.Journal of the American Statistical Association105\(489\),pp\. 375–389\.Cited by:[§2\.4](https://arxiv.org/html/2608.28934#S2.SS4.p8.1)\.
- \[YS22\]J\. Ye and R\. Shokri\(2022\)Differentially private learning needs hidden state \(or much faster convergence\)\.InAdvances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022,Cited by:[§1](https://arxiv.org/html/2608.28934#S1.p2.1)\.
- \[YSS\+21\]A\. Yousefpour, I\. Shilov, A\. Sablayrolles, D\. Testuggine, K\. Prasad, M\. Malek, J\. Nguyen, S\. Ghosh, A\. Bharadwaj, J\. Zhao, G\. Cormode, and I\. Mironov\(2021\)Opacus: user\-friendly differential privacy library in PyTorch\.InNeurIPS 2021 Workshop Privacy in Machine Learning,Cited by:[§2\.1](https://arxiv.org/html/2608.28934#S2.SS1.p6.1),[footnote 3](https://arxiv.org/html/2608.28934#footnote3)\.
- \[ZK16\]S\. Zagoruyko and N\. Komodakis\(2016\)Wide residual networks\.InProceedings of the 2016 British Machine Vision Conference,BMVC ’16,pp\. 87:1–87:12\.Cited by:[§4\.1](https://arxiv.org/html/2608.28934#S4.SS1.p3.1)\.
- \[ZWT\+23\]S\. Zanella\-Beguelin, L\. Wutschitz, S\. Tople, A\. Salem, V\. Rühle, A\. Paverd, M\. Naseri, B\. Köpf, and D\. Jones\(2023\)Bayesian estimation of differential privacy\.InProceedings of the 40th International Conference on Machine Learning,ICML ’23,pp\. 40624–40636\.Cited by:[§A\.1](https://arxiv.org/html/2608.28934#A1.SS1.p4.1)\.
- \[ZSZ\+22\]M\. Zhang, N\. S\. Sohoni, H\. R\. Zhang, C\. Finn, and C\. Ré\(2022\)Correct\-n\-contrast: a contrastive approach for improving robustness to spurious correlations\.InInternational Conference on Machine Learning, ICML 2022,Proceedings of Machine Learning Research,pp\. 26484–26516\.Cited by:[Appendix B](https://arxiv.org/html/2608.28934#A2.p1.1)\.
- \[ZW19\]Y\. Zhu and Y\. Wang\(2019\)Poisson subsampled rényi differential privacy\.InInternational Conference on Machine Learning,pp\. 7634–7642\.Cited by:[§2\.1](https://arxiv.org/html/2608.28934#S2.SS1.p6.1)\.

## Appendix AEmpirical Privacy Reporting Details

### A\.1Heuristic conventions and their \(potential\) pitfalls

In this section, we detail two practices often used in the privacy auditing literature, and point out when their guarantees formally apply \(contrasted with when they should be treated as a heuristic\)\. While this discussion is sometimes present in prior works, we believe making it explicit, with specific counterexamples to the validity of heuristics, will prove valuable to future auditing research, as well as practitioners reasoning about the guarantees of these reporting metrics\.

Following Section[2\.4](https://arxiv.org/html/2608.28934#S2.SS4), the formal way to certify a privacy lower bound on an algorithm𝒜\\mathcal\{A\}is as follows\. First, a testϕ\\phihowever the auditor desires \(e\.g\., the most effective loss threshold is selected using a set of models, which are discarded\)\. Next, fresh holdout models are trained using𝒜\\mathcal\{A\}and evaluated according to the testϕ\\phi, producing an empirical FPR\-FNR tradeoff\(α,β\)\(\\alpha,\\beta\)\. Then, a Clopper\-Pearson correction with failure probabilityγ\\gammais applied to obtain\(α^,β^\)\(\\hat\{\\alpha\},\\hat\{\\beta\}\)lying above theff\-DP curve with probability1−γ1\-\\gamma\. Finally, \([5](https://arxiv.org/html/2608.28934#S2.E5)\) converts this point into anεlb\\varepsilon\_\{\\textup\{lb\}\}estimate\. The two main heuristics we discuss in this section are*multiple hypothesis testing*and*GDP extrapolation*\. We describe these two heuristics in turn, and then examine their potential pitfalls\.

Multiple hypothesis testing\.Algorithm[6](https://arxiv.org/html/2608.28934#algorithm6)provides pseudocode for the*multiple hypothesis testing*heuristic, where the same models are used for selecting the testϕ\\phi, and evaluating it to produce a point\(α^,β^\)\(\\hat\{\\alpha\},\\hat\{\\beta\}\)above theff\-DP curve\. Observe that the samekki\.i\.d\. copies are used in all of the tests\{ϕi\}i∈\[m\]\\\{\\phi\_\{i\}\\\}\_\{i\\in\[m\]\}, and then the bestεlb\\varepsilon\_\{\\textup\{lb\}\}estimate amongst these tests is reported using \([5](https://arxiv.org/html/2608.28934#S2.E5)\)\. It is formally incorrect to say that the selected pair\(α^i,β^i\)\(\\hat\{\\alpha\}\_\{i\},\\hat\{\\beta\}\_\{i\}\)lies above the trueff\-DP curve with probability1−γ1\-\\gamma, due to the potential for false discovery\. The formal fix is to either use a Bonferroni correction \(union bound\) and set the coverage of each Clopper\-Pearson correction in Algorithm[6](https://arxiv.org/html/2608.28934#algorithm6)to1−γm1\-\\frac\{\\gamma\}\{m\}, or to apply a selected testϕi\\phi\_\{i\}to holdout samples \(with Clopper\-Pearson coverage1−γ1\-\\gamma\)\. Indeed, the original work of\[[35](https://arxiv.org/html/2608.28934#bib.bib40)\]prescribes using the holdout strategy to ensure statistical validity\.

Algorithm 6𝖤𝗆𝗉𝗂𝗋𝗂𝖼𝖺𝗅𝖣𝖯𝖫𝖡⁡\(𝒜,\{𝒟i,𝒟i′,ϕi\}i∈\[m\],δ,γ,k\)\\mathsf\{EmpiricalDPLB\}\(\\mathcal\{A\},\\\{\\mathcal\{D\}\_\{i\},\\mathcal\{D\}^\{\\prime\}\_\{i\},\\phi\_\{i\}\\\}\_\{i\\in\[m\]\},\\delta,\\gamma,k\)Input:Randomized algorithm

𝒜:𝒳n→Ω\\mathcal\{A\}:\\mathcal\{X\}^\{n\}\\to\\Omega, neighboring datasets

𝒟,𝒟′∈𝒮∗\\mathcal\{D\},\\mathcal\{D\}^\{\\prime\}\\in\\mathcal\{S\}^\{\*\}and

mmtests

ϕi:Ω→\[0,1\]\\phi\_\{i\}:\\Omega\\to\[0,1\],

δ∈\(0,1\)\\delta\\in\(0,1\),

γ∈\(0,1\)\\gamma\\in\(0,1\),

k∈ℕk\\in\\mathbb\{N\}
1Output:Empirical privacy lower bound

εlb​\(δ\)\\varepsilon\_\{\\textup\{lb\}\}\(\\delta\)
2

\{ωj\}j∈\[k\]∼i\.i\.d\.𝒜\(𝒟\)\\\{\\omega\_\{j\}\\\}\_\{j\\in\[k\]\}\\sim\_\{\\textup\{i\.i\.d\.\}\}\\mathcal\{A\}\(\\mathcal\{D\}\)
3

\{ωj′\}j∈\[k\]∼i\.i\.d\.𝒜\(𝒟′\)\\\{\\omega^\{\\prime\}\_\{j\}\\\}\_\{j\\in\[k\]\}\\sim\_\{\\textup\{i\.i\.d\.\}\}\\mathcal\{A\}\(\\mathcal\{D\}^\{\\prime\}\)
4for

i∈\[m\]i\\in\[m\]do

5

αiinit←1k​∑j∈\[k\]ϕi​\(ωj\)\\alpha\_\{i\}^\{\\text\{init\}\}\\leftarrow\\frac\{1\}\{k\}\\sum\_\{j\\in\[k\]\}\\phi\_\{i\}\(\\omega\_\{j\}\)
6

βiinit←1−1k​∑j∈\[k\]ϕi​\(ωj′\)\\beta\_\{i\}^\{\\text\{init\}\}\\leftarrow 1\-\\frac\{1\}\{k\}\\sum\_\{j\\in\[k\]\}\\phi\_\{i\}\(\\omega^\{\\prime\}\_\{j\}\)
7

\(α^i,β^i\)←\(Betak​αiinit\+1,k⁡\(1−αiinit\)−1​\(1−γ\),Betak​βiinit\+1,k⁡\(1−βiinit\)−1​\(1−γ\)\)\(\\hat\{\\alpha\}\_\{i\},\\hat\{\\beta\}\_\{i\}\)\\leftarrow\(\\textup\{Beta\}^\{\-1\}\_\{k\\alpha\_\{i\}^\{\\textup\{init\}\}\+1,k\(1\-\\alpha\_\{i\}^\{\\textup\{init\}\}\)\}\(1\-\\gamma\),\\textup\{Beta\}^\{\-1\}\_\{k\\beta\_\{i\}^\{\\textup\{init\}\}\+1,k\(1\-\\beta\_\{i\}^\{\\textup\{init\}\}\)\}\(1\-\\gamma\)\)//Clopper\-Pearson,Betaa,b\\textup\{Beta\}\_\{a,b\}is CDF of beta distribution with parametersa,ba,b

8end for

9Return:

εlb\(δ\)←maxi:α^i\>0log\(1−β^i−δα^i\)\\varepsilon\_\{\\textup\{lb\}\}\(\\delta\)\\leftarrow\\max\_\{i:\\hat\{\\alpha\}\_\{i\}\>0\}\\log\(\\frac\{1\-\\hat\{\\beta\}\_\{i\}\-\\delta\}\{\\hat\{\\alpha\}\_\{i\}\}\)

Subsequent works in privacy auditing are less careful on this matter, with many \(e\.g\.,\[[44](https://arxiv.org/html/2608.28934#bib.bib4),[71](https://arxiv.org/html/2608.28934#bib.bib5),[48](https://arxiv.org/html/2608.28934#bib.bib68),[5](https://arxiv.org/html/2608.28934#bib.bib62),[14](https://arxiv.org/html/2608.28934#bib.bib63)\]\) utilizing a variant of the multiple hypothesis testing method in Algorithm[6](https://arxiv.org/html/2608.28934#algorithm6),*without*explicitly using a Bonferroni correction or holdouts\. Indeed, Section 5\.2 of\[[48](https://arxiv.org/html/2608.28934#bib.bib68)\]explicitly flags this issue, where they state that selecting an audit in this way is “technically not valid,” but that “it has become common” to report lower bounds in this way\.

Our experiments \(cf\. Appendix[J](https://arxiv.org/html/2608.28934#A10)\) suggest that the Bonferroni correction yields substantially smaller \(often negligible\)εlb\\varepsilon\_\{\\textup\{lb\}\}estimates\. On the other hand, the holdout method still results in nontrivial formalεlb\\varepsilon\_\{\\textup\{lb\}\}estimates when auditing models trained with DP\-SGD, although the tightness of these estimates often worsens by a factor of33\-10×10\\times\. We thus caution against taking reportedεlb\\varepsilon\_\{\\textup\{lb\}\}wholly at face value if multiple discovery is not accounted for, at least without ablations justifying the heuristic\. For completeness, we report estimatedεlb\\varepsilon\_\{\\textup\{lb\}\}both with and without multiple discovery, as trends between our algorithms and DP\-SGD remain consistent across reporting methods\.

Another subtlety regarding multiple discovery that arises in the practice of privacy auditing \(and private ML more broadly\) is*hyperparameter selection*, see e\.g\.,\[[47](https://arxiv.org/html/2608.28934#bib.bib45),[52](https://arxiv.org/html/2608.28934#bib.bib50),[54](https://arxiv.org/html/2608.28934#bib.bib69)\]\. To circumvent this issue, all of our hyperparameter selection is explicitly performed using holdout models that are discarded before selecting our audits and reporting ourεlb\\varepsilon\_\{\\textup\{lb\}\}estimates\.

Finally, we make one observation that is implicit in prior works using Algorithm[6](https://arxiv.org/html/2608.28934#algorithm6), but which has not received explicit discussion to our knowledge\. A reasonable modification to Line[6](https://arxiv.org/html/2608.28934#algorithm6)of Algorithm[6](https://arxiv.org/html/2608.28934#algorithm6)uses Lemma[2](https://arxiv.org/html/2608.28934#Thmlemma2), i\.e\., it first computes the largest convex functionffbelow the points\{\(αi^,βi\)\}i∈\[m\]\\\{\(\\hat\{\\alpha\_\{i\}\},\\beta\_\{i\}\)\\\}\_\{i\\in\[m\]\}, before applying the formula \([5](https://arxiv.org/html/2608.28934#S2.E5)\)\. This modification could in principle yield tighter audits, as it leverages the ability to use points on theff\-DP curve outside the\{\(α^i,β^i\)\}i∈\[m\]\\\{\(\\hat\{\\alpha\}\_\{i\},\\hat\{\\beta\}\_\{i\}\)\\\}\_\{i\\in\[m\]\}\. As shown in Lemma[6](https://arxiv.org/html/2608.28934#Thmlemma6), this convexification step in fact does not change the computedεlb\\varepsilon\_\{\\textup\{lb\}\}\.

###### Lemma 6\.

Let𝒮=\{\(αi,βi\)\}i∈\[n\]⊂\[0,1\]2\\mathcal\{S\}=\\\{\(\\alpha\_\{i\},\\beta\_\{i\}\)\\\}\_\{i\\in\[n\]\}\\subset\[0,1\]^\{2\}have distinct first coordinates, and letffbe the lower convex envelope of𝒮\\mathcal\{S\}\(the pointwise largest convex function withf⁡\(αi\)≤βif\(\\alpha\_\{i\}\)\\leq\\beta\_\{i\}for alli∈\[n\]i\\in\[n\]\)\.666The lower convex envelope is uniquely defined, see Exercise 3\.30 of\[[10](https://arxiv.org/html/2608.28934#bib.bib25)\]\.Also, assume that\{\(0,1\),\(1,0\)\}⊂𝒮\\\{\(0,1\),\(1,0\)\\\}\\subset\\mathcal\{S\}\. Then for allδ∈\(0,1\)\\delta\\in\(0,1\),

supx∈\(0,1\]log\(1−f⁡\(x\)−δx\)=maxi:αi\>0log\(1−βi−δαi\),\\sup\_\{x\\in\(0,1\]\}\\log\\left\(\\frac\{1\-f\(x\)\-\\delta\}\{x\}\\right\)=\\max\_\{i:\\alpha\_\{i\}\>0\}\\log\\left\(\\frac\{1\-\\beta\_\{i\}\-\\delta\}\{\\alpha\_\{i\}\}\\right\),with the convention that the logarithm of a nonpositive number is−∞\-\\infty\.

###### Proof\.

Throughout the proof, fix someδ∈\(0,1\)\\delta\\in\(0,1\)and define

ψ\(x\):=log\(1−f⁡\(x\)−δx\),M1:=supx∈\(0,1\]ψ\(x\),M2:=maxi:αi\>0log\(1−βi−δαi\)\.\\psi\(x\):=\\log\\left\(\\frac\{1\-f\(x\)\-\\delta\}\{x\}\\right\),\\quad M\_\{1\}:=\\sup\_\{x\\in\(0,1\]\}\\psi\(x\),\\quad M\_\{2\}:=\\max\_\{i:\\alpha\_\{i\}\>0\}\\log\\left\(\\frac\{1\-\\beta\_\{i\}\-\\delta\}\{\\alpha\_\{i\}\}\\right\)\.It is immediate thatM2≤M1M\_\{2\}\\leq M\_\{1\}, becauseβi≥f⁡\(αi\)\\beta\_\{i\}\\geq f\(\\alpha\_\{i\}\)andαi\\alpha\_\{i\}is a candidate argument in the definition ofM1M\_\{1\}\. For the reverse inequality, the lower convex envelope of finitely many points is piecewise affine, and its breakpoints occur at pointsαi\\alpha\_\{i\}where the constraint is tight, i\.e\.,f⁡\(αi\)=βif\(\\alpha\_\{i\}\)=\\beta\_\{i\}\(see Theorem 19\.1,\[[55](https://arxiv.org/html/2608.28934#bib.bib26)\]\)\. Therefore, it is enough to show that on an intervalx∈\[αi,αj\]x\\in\[\\alpha\_\{i\},\\alpha\_\{j\}\]such thatf⁡\(x\)=a​x\+bf\(x\)=ax\+b,f⁡\(αi\)=βif\(\\alpha\_\{i\}\)=\\beta\_\{i\}, andf⁡\(αj\)=βjf\(\\alpha\_\{j\}\)=\\beta\_\{j\},ψ\\psiachieves its maximum at an endpointαi\\alpha\_\{i\}orαj\\alpha\_\{j\}\.

To see this last fact, observe that on this interval,

ψ⁡\(x\)=log⁡\(1−b−δ−a​xx\),ψ′​\(x\)=−1−b−δx⁡\(1−b−δ−a​x\),for​x<1−b−δa\.\\psi\(x\)=\\log\\left\(\\frac\{1\-b\-\\delta\-ax\}\{x\}\\right\),\\quad\\psi^\{\\prime\}\(x\)=\-\\frac\{1\-b\-\\delta\}\{x\(1\-b\-\\delta\-ax\)\},\\quad\\text\{ for \}x<\\frac\{1\-b\-\\delta\}\{a\}\.Henceψ′\\psi^\{\\prime\}has a fixed sign on this interval \(wheneverψ\>−∞\\psi\>\-\\infty\), soψ\\psiis supremized at an endpoint\. ∎

Gaussian DP \(GDP\) extrapolation\.GDP curves\[[20](https://arxiv.org/html/2608.28934#bib.bib64)\]are a family offf\-DP curves that are particularly analytically convenient\. Informally, these are a one\-parameter family offf\-DP curves that exactly capture the trade\-off functionsTP,QT\_\{P,Q\}\(Definition[3](https://arxiv.org/html/2608.28934#Thmdefinition3)\) whenP=𝒩⁡\(0,1\)P=\\mathcal\{N\}\(0,1\)andQ=𝒩⁡\(μ,1\)Q=\\mathcal\{N\}\(\\mu,1\)are equal\-variance Gaussians; here, the one parameter isμ\\mu\. This family is relevant to our setting because DP\-SGD utilizes the Gaussian mechanism, and the composition of GDP curves remains a GDP curve \(Corollary 3\.3,\[[20](https://arxiv.org/html/2608.28934#bib.bib64)\]\)\. GDP curves are notably compatible with sample\-efficient auditing: if the true privacy loss curve is GDP, then the auditor can extrapolate from a single tradeoff point\(α,β\)\(\\alpha,\\beta\)to rule out part of the GDP parametric family, improving statistical efficiency\. Consequently, prior works starting from\[[48](https://arxiv.org/html/2608.28934#bib.bib68)\]\(see also\[[5](https://arxiv.org/html/2608.28934#bib.bib62),[14](https://arxiv.org/html/2608.28934#bib.bib63)\]\) propose using GDP methodology to achieve tighter audits\. We call this strategy*GDP extrapolation*, and note that it should be treated as a heuristic whenever the trueff\-DP curve is not a GDP curve\.

In this section, we point out two potential pitfalls to using this methodology to formally audit DP\-SGD variants\. These pitfalls stem from the facts that GDP curves are*not*closed under the following operations: \(1\) Poisson subsampling, and \(2\) marginalizing intermediate outputs \(i\.e\., in the hidden state threat model\)\. We formalize these points with explicit counterexamples in Corollaries[2](https://arxiv.org/html/2608.28934#Thmcorollary2)and[3](https://arxiv.org/html/2608.28934#Thmcorollary3)\. Prior works have also discussed these issues to some extent, e\.g\., Section 5\.5 of\[[48](https://arxiv.org/html/2608.28934#bib.bib68)\], and an extended discussion of the subsampling issue in\[[29](https://arxiv.org/html/2608.28934#bib.bib73)\]\.

One common justification for use of GDP\-based auditing, when theff\-DP curve is not formally a GDP curve, is theff\-DP central limit theorem \(Theorem 3\.4,\[[20](https://arxiv.org/html/2608.28934#bib.bib64)\]\), as essentially all private optimizers in ML are iterative methods that successively apply the same mechanism\. We caution that again, the proof of theff\-DP central limit theorem relies on accessing intermediate outputs \(which becomes problematic under hidden state threats, e\.g\., as in\[[14](https://arxiv.org/html/2608.28934#bib.bib63)\]\), and hence this logic is susceptible to the counterexample in Corollary[3](https://arxiv.org/html/2608.28934#Thmcorollary3)\. Altogether, GDP\-based auditing requires careful justification depending on the setup; for this reason, our experiments report both the GDP\-basedεlb\\varepsilon\_\{\\textup\{lb\}\}and a weaker, but more formal, estimate based on Clopper\-Pearson correction\.

LetΦ\\Phidenote the CDF of a standard Gaussian, and letφ=Φ′\\varphi=\\Phi^\{\\prime\}denote its PDF\. The following lemma is the basis of our counterexamples, and states that the trade\-off function between a particular Gaussian and mixture of two Gaussians is not a GDP curve\.

###### Lemma 7\.

Consider the trade\-off functionTP,Q​\(α\)T\_\{P,Q\}\(\\alpha\)forP=𝒩⁡\(0,1\)P=\\mathcal\{N\}\(0,1\)andQ=12​𝒩​\(0,1\)\+12​𝒩​\(1,1\)Q=\\frac\{1\}\{2\}\\mathcal\{N\}\(0,1\)\+\\frac\{1\}\{2\}\\mathcal\{N\}\(1,1\)\. There does not exist aμ\>0\\mu\>0such thatTP,Q=GμT\_\{P,Q\}=G\_\{\\mu\}, whereGμ​\(α\):=Φ⁡\(Φ−1​\(1−α\)−μ\)G\_\{\\mu\}\(\\alpha\):=\\Phi\(\\Phi^\{\-1\}\(1\-\\alpha\)\-\\mu\)\.

###### Proof\.

By the Neyman–Pearson lemma, optimal tests threshold the likelihood ratio\. Here

d​Qd​P​\(x\)=12\+12​exp⁡\(x−12\),\\frac\{dQ\}\{dP\}\(x\)=\\frac\{1\}\{2\}\+\\frac\{1\}\{2\}\\exp\\\!\\left\(x\-\\frac\{1\}\{2\}\\right\),which is strictly increasing inxx\. Therefore, for false\-positive rateα=Pr\[X≥t\]=1−Φ\(t\)\\alpha=\\Pr\[X\\geq t\]=1\-\\Phi\(t\), the optimal test is𝕀x≥t\\mathbb\{I\}\_\{x\\geq t\}, , where𝕀ℰ\\mathbb\{I\}\_\{\\mathcal\{E\}\}is the indicator of an eventℰ\\mathcal\{E\}, and its false\-negative rate is

TP,Q\(α\)=PrX∼Q\[X<t\]=12Φ\(t\)\+12Φ\(t−1\),t=Φ−1\(1−α\)\.T\_\{P,Q\}\(\\alpha\)=\\Pr\_\{X\\sim Q\}\[X<t\]=\\frac\{1\}\{2\}\\Phi\(t\)\+\\frac\{1\}\{2\}\\Phi\(t\-1\),\\qquad t=\\Phi^\{\-1\}\(1\-\\alpha\)\.IfTP,Q=GμT\_\{P,Q\}=G\_\{\\mu\}, then for allt∈ℝt\\in\\mathbb\{R\},12​Φ​\(t\)\+12​Φ​\(t−1\)=Φ⁡\(t−μ\)\\frac\{1\}\{2\}\\Phi\(t\)\+\\frac\{1\}\{2\}\\Phi\(t\-1\)=\\Phi\(t\-\\mu\)\. Differentiating gives

12​φ​\(t\)\+12​φ​\(t−1\)=φ⁡\(t−μ\)\.\\frac\{1\}\{2\}\\varphi\(t\)\+\\frac\{1\}\{2\}\\varphi\(t\-1\)=\\varphi\(t\-\\mu\)\.Dividing byφ⁡\(t\)\\varphi\(t\),

12\+12​et−12=et​μ−μ22\.\\frac\{1\}\{2\}\+\\frac\{1\}\{2\}e^\{t\-\\frac\{1\}\{2\}\}=e^\{t\\mu\-\\frac\{\\mu^\{2\}\}\{2\}\}\.Ast→−∞t\\to\-\\infty, the two sides tend to12\\frac\{1\}\{2\}and00respectively for everyμ\>0\\mu\>0, a contradiction\. ∎

Our claimed counterexamples now follow simply from Lemma[7](https://arxiv.org/html/2608.28934#Thmlemma7)\.

###### Corollary 2\.

The trade\-off function for the Gaussian mechanism with Poisson subsampling does not always correspond to aμ\\mu\-GDP trade\-off function for someμ\>0\\mu\>0\.

###### Proof\.

Consider a single\-element dataset, either00or11, and Poisson subsampling with parameterq=12q=\\frac\{1\}\{2\}composed with the Gaussian mechanism \(adding𝒩⁡\(0,1\)\\mathcal\{N\}\(0,1\)to the empirical mean\)\. If the dataset is00, then the resulting distribution isP=𝒩⁡\(0,1\)P=\\mathcal\{N\}\(0,1\), while if the dataset is11, then the resulting distribution isQ=12​𝒩​\(0,1\)\+12​𝒩​\(1,1\)Q=\\frac\{1\}\{2\}\\mathcal\{N\}\(0,1\)\+\\frac\{1\}\{2\}\\mathcal\{N\}\(1,1\), which is exactly the example described in Lemma[7](https://arxiv.org/html/2608.28934#Thmlemma7)\. ∎

###### Corollary 3\.

Consider composing several Gaussian mechanisms \(adaptively\) and outputting only the result of the final mechanism\. The trade\-off function of this mechanism does not always correspond to aμ\\mu\-GDP trade\-off function for someμ\>0\\mu\>0\.

###### Proof\.

Consider a two\-stage Gaussian mechanism, with a single\-element datasetddthat is either00or11\. The first stage is to drawX∼𝒩⁡\(0,1\)X\\sim\\mathcal\{N\}\(0,1\), independent of the dataset\. We adaptively choose the second stage based on the result of the first: we output a sample from𝒩\(d⋅𝕀\{X≥0\},1\)\\mathcal\{N\}\(d\\cdot\\mathbb\{I\}\_\{\\\{X\\geq 0\\\}\},1\)\. Considering the various choices ofdd, this again results in exactly the aforementioned distributionsP=𝒩⁡\(0,1\)P=\\mathcal\{N\}\(0,1\)orQ=12​𝒩​\(0,1\)\+12​𝒩​\(1,1\)Q=\\frac\{1\}\{2\}\\mathcal\{N\}\(0,1\)\+\\frac\{1\}\{2\}\\mathcal\{N\}\(1,1\)\. ∎

### A\.2Numerical precision

Applying a fully formal empirical privacy estimation methodology, e\.g\., Clopper–Pearson corrections after holdout, can sometimes report highly\-conservativeεlb\\varepsilon\_\{\\textup\{lb\}\}figures that sometimes even show up as vanishing\. We numerically examine how the Clopper–Pearson \(CP\) correction changes empirical privacy lower bounds in the finite\-sample regime\. For this discussion, we fix the significant level of confidence internal toγ=0\.05\\gamma=0\.05and ignore the union bound over multiple tests \(i\.e\., the false discovery direction in Appendix[A](https://arxiv.org/html/2608.28934#A1), so that each of the two error rates,α\\alphaandβ\\beta, is corrected using a one\-sided CP upper confidence bound with failure probabilityγ2=0\.025\\frac\{\\gamma\}\{2\}=0\.025\. For each sample sizek∈\{200,500,1000,5000,10000\}k\\in\\\{200,500,1000,5000,10000\\\}, we enumerate all possible empirical count pairs\(xα,xβ\)∈\[k\]×\[k\]\(x\_\{\\alpha\},x\_\{\\beta\}\)\\in\[k\]\\times\[k\], whereαemp=xαk,βemp=xβk\.\\alpha\_\{\\rm emp\}=\\frac\{x\_\{\\alpha\}\}\{k\},\\beta\_\{\\rm emp\}=\\frac\{x\_\{\\beta\}\}\{k\}\.For each pair, we compute the raw empirical lower bound and the CP corrected lower bound:

εraw=max⁡\{0,log⁡1−βemp−δαemp\},εCP=max⁡\{0,log⁡1−β^−δα^\},\\varepsilon\_\{\\rm raw\}=\\max\\left\\\{0,\\log\\frac\{1\-\\beta\_\{\\rm emp\}\-\\delta\}\{\\alpha\_\{\\rm emp\}\}\\right\\\},\\quad\\varepsilon\_\{\\rm CP\}=\\max\\left\\\{0,\\log\\frac\{1\-\\hat\{\\beta\}\-\\delta\}\{\\hat\{\\alpha\}\}\\right\\\},whereα^\\hat\{\\alpha\}andβ^\\hat\{\\beta\}are the CP upper confidence bounds, and restrict attention to count pairs withεraw\>0\\varepsilon\_\{\\rm raw\}\>0\. This isolates empirical observations that initially provide a nonzero privacy signal and shows how many of them are weakened or collapsed to zero after CP correction\. We show our result in Figure[2](https://arxiv.org/html/2608.28934#A1.F2)\. We make two observations from the numerical results\.

1. 1\.In the experiment, we filter out all\(α,β\)\(\\alpha,\\beta\)pairs for whichεraw=0\\varepsilon\_\{\\rm raw\}=0orεraw=\+∞\\varepsilon\_\{\\rm raw\}=\+\\infty\. After this filtering, the range of finite empirical lower bounds that can appear is still limited by the sample size\. In particular, the largest possible auditableεlb\\varepsilon\_\{\\textup\{lb\}\}grows only logarithmically withkk: it is around44fork=200k=200in our plotted range and around99fork=10000k=10000\.
2. 2\.Askkincreases, fewer nonzero raw observations collapse to zero after CP correction, and the average gap betweenεraw\\varepsilon\_\{\\rm raw\}andεcp\\varepsilon\_\{\\rm cp\}also decreases\. However, the worst\-case raw value among the collapsed observations does not vanish; in our experiments, it remains around2\.42\.4\.

Both observations have simple theoretical explanations\. For the first observation, over all finite count pairs with sample sizekk, the largest possible raw lower bound is attained whenxα=1x\_\{\\alpha\}=1andxβ=0x\_\{\\beta\}=0, giving

εraw⋆=log⁡1−δ1/k=log⁡\(\(1−δ\)​k\)\.\\varepsilon\_\{\\rm raw\}^\{\\star\}=\\log\\frac\{1\-\\delta\}\{1/k\}=\\log\\bigl\(\(1\-\\delta\)k\\bigr\)\.Thus, the auditable range of finite empirical lower bounds grows only logarithmically with the number of samples, suggesting that accurately auditing largeε\\varepsilonvalues requires very largekk\.

For the second observation, the non\-vanishing worst\-case collapse is caused by extreme low\-count configurations near the boundary of the binomial experiment\. Letxα=1,xβ=k−cx\_\{\\alpha\}=1,x\_\{\\beta\}=k\-c, whereccis some constant that will be specified later\. Equivalentlyαemp=1k,1−βemp=ck\.\\alpha\_\{\\rm emp\}=\\frac\{1\}\{k\},1\-\\beta\_\{\\rm emp\}=\\frac\{c\}\{k\}\.Therefore the raw lower bound isεraw=log⁡c/k−δ1/k\.\\varepsilon\_\{\\rm raw\}=\\log\\frac\{c/k\-\\delta\}\{1/k\}\.Whenδ\\deltais negligible compared withc/kc/k, this is approximatelyεraw≈log⁡c\\varepsilon\_\{\\rm raw\}\\approx\\log c, which is independent ofkk\. We then consider the CP correction:

α^≈q1−γ/2​\(Γ​\(2,1\)\)k,β^≈1−qγ/2​\(Γ​\(c,1\)\)k\.\\hat\{\\alpha\}\\approx\\frac\{q\_\{1\-\\gamma/2\}\(\\Gamma\(2,1\)\)\}\{k\},\\quad\\hat\{\\beta\}\\approx 1\-\\frac\{q\_\{\\gamma/2\}\(\\Gamma\(c,1\)\)\}\{k\}\.whereqp​\(Γ​\(a,1\)\)q\_\{p\}\(\\Gamma\(a,1\)\)denotes thepp\-quantile of a Gamma distribution\. The CP\-corrected lower bound collapses to zero whenα^\+β^\+δ≥1\\hat\{\\alpha\}\+\\hat\{\\beta\}\+\\delta\\geq 1\. Substituting the approximations above, the collapse condition becomes

qγ/2​\(Γ⁡\(c,1\)\)−k​δ≲q1−γ/2​\(Γ⁡\(2,1\)\)\.q\_\{\\gamma/2\}\(\\Gamma\(c,1\)\)\-k\\delta\\lesssim q\_\{1\-\\gamma/2\}\(\\Gamma\(2,1\)\)\.Forγ=0\.05\\gamma=0\.05, the right\-hand side is approximately5\.575\.57\. Whenk​δk\\deltais small, the largest integerccsatisfying this condition is aboutc=11c=11, giving

εraw=log⁡\(c−k​δ\)≈log⁡11≈2\.4\.\\varepsilon\_\{\\rm raw\}=\\log\(c\-k\\delta\)\\approx\\log 11\\approx 2\.4\.Thus, the non\-vanishing maximum collapsed value is a finite\-count boundary effect: it persists for largekkas long ask​δk\\deltaremains small relative to the relevant true\-positive countcc\. In the pure\-DP case\(δ=0\)\(\\delta=0\), this gap persists for any choice of finitekk\. In the common auditing setting withδ=10−5\\delta=10^\{\-5\}, the gap begins to disappear only whenk​δk\\deltabecomes comparable tocc; forc≈11c\\approx 11, this requiresk≳106k\\gtrsim 10^\{6\}, which is a rather demanding sample size\.

![Refer to caption](https://arxiv.org/html/2608.28934v1/plots/hist_raw_vs_cp_raw_positive_with_max_k200.png)εraw\\varepsilon\_\{\\rm raw\}v\.s\.εcp\\varepsilon\_\{\\rm cp\}
![Refer to caption](https://arxiv.org/html/2608.28934v1/plots/hist_eps_drop_raw_positive_with_max_k200.png)εraw−εCP\\varepsilon\_\{\\mathrm\{raw\}\}\-\\varepsilon\_\{\\mathrm\{CP\}\}
![Refer to caption](https://arxiv.org/html/2608.28934v1/plots/hist_raw_eps_collapsed_to_zero_with_max_k200.png)εraw\\varepsilon\_\{\\rm raw\}among collapse points
\(a\)k=200k=200

![Refer to caption](https://arxiv.org/html/2608.28934v1/plots/hist_raw_vs_cp_raw_positive_with_max_k1000.png)εraw\\varepsilon\_\{\\rm raw\}v\.s\.εcp\\varepsilon\_\{\\rm cp\}
![Refer to caption](https://arxiv.org/html/2608.28934v1/plots/hist_eps_drop_raw_positive_with_max_k1000.png)εraw−εCP\\varepsilon\_\{\\mathrm\{raw\}\}\-\\varepsilon\_\{\\mathrm\{CP\}\}
![Refer to caption](https://arxiv.org/html/2608.28934v1/plots/hist_raw_eps_collapsed_to_zero_with_max_k1000.png)εraw\\varepsilon\_\{\\rm raw\}among collapse points
\(b\)k=1000k=1000

![Refer to caption](https://arxiv.org/html/2608.28934v1/plots/hist_raw_vs_cp_raw_positive_with_max_k5000.png)εraw\\varepsilon\_\{\\rm raw\}v\.s\.εcp\\varepsilon\_\{\\rm cp\}
![Refer to caption](https://arxiv.org/html/2608.28934v1/plots/hist_eps_drop_raw_positive_with_max_k5000.png)εraw−εCP\\varepsilon\_\{\\mathrm\{raw\}\}\-\\varepsilon\_\{\\mathrm\{CP\}\}
![Refer to caption](https://arxiv.org/html/2608.28934v1/plots/hist_raw_eps_collapsed_to_zero_with_max_k5000.png)εraw\\varepsilon\_\{\\rm raw\}among collapse points
\(c\)k=5000k=5000

![Refer to caption](https://arxiv.org/html/2608.28934v1/plots/hist_raw_vs_cp_raw_positive_with_max_k10000.png)εraw\\varepsilon\_\{\\rm raw\}v\.s\.εcp\\varepsilon\_\{\\rm cp\}
![Refer to caption](https://arxiv.org/html/2608.28934v1/plots/hist_eps_drop_raw_positive_with_max_k10000.png)εraw−εCP\\varepsilon\_\{\\mathrm\{raw\}\}\-\\varepsilon\_\{\\mathrm\{CP\}\}
![Refer to caption](https://arxiv.org/html/2608.28934v1/plots/hist_raw_eps_collapsed_to_zero_with_max_k10000.png)εraw\\varepsilon\_\{\\rm raw\}among collapse points
\(d\)k=10000k=10000

Figure 2:Finite\-sample effect of Clopper–Pearson \(CP\) correction under different sample sizeskk, withγ=0\.05\\gamma=0\.05andδ=10−5\\delta=10^\{\-5\}\. Each row corresponds to a fixedkk\. The left column compares the distributions of the raw lower boundεraw\\varepsilon\_\{\\rm raw\}and the CP\-corrected lower boundεcp\\varepsilon\_\{\\rm cp\}; the middle column shows the distribution of the correction gapεraw−εcp\\varepsilon\_\{\\rm raw\}\-\\varepsilon\_\{\\rm cp\}; and the right column shows the distribution of raw lower bounds that collapse to00after CP correction\.Our remaining appendices provide additional experimental results and ablation studies that complement our main findings\. We summarize the different appendices here\.

- •[B](https://arxiv.org/html/2608.28934#A2): Disparate Impact\.Exploring how defense targets minority vs majority subgroups for sample removal, for varying minority/majority ratios\.
- •[C](https://arxiv.org/html/2608.28934#A3): Alternative Input Space Audits\.Evaluations of input space audit constructions in the multiple canary setting, and the randomized canary setting of\[[61](https://arxiv.org/html/2608.28934#bib.bib54)\]\.
- •[D](https://arxiv.org/html/2608.28934#A4): Sample Signatures\.Ablation study comparing different scoring functions for filtering\.
- •[E](https://arxiv.org/html/2608.28934#A5): Poisson vs\. Shuffled Subsampling\.Comparison of DP\-SGD subsampling variants, and the effect of DP\-SGD implementation changes on our estimatedεlb\\varepsilon\_\{\\textup\{lb\}\}\.
- •[F](https://arxiv.org/html/2608.28934#A6): Interpolating Canaries\.Testing defense effectiveness across different scalings of canary strengths on a mixture of a blank canary and a natural image\.
- •[G](https://arxiv.org/html/2608.28934#A7): Auditing via Non\-Canary Samples\.Analyzing privacy loss on non\-canary samples, motivated by the “privacy onion effect” observed by\[[12](https://arxiv.org/html/2608.28934#bib.bib28)\]\.
- •[H](https://arxiv.org/html/2608.28934#A8): Increasing Trial Count\.Validating audit methodology with 1000 shadow models\.
- •[I](https://arxiv.org/html/2608.28934#A9): Additional Hyperparameter Tuning\.Ablations on global filtering, local filtering bandwidth, and filter frequency\.
- •[J](https://arxiv.org/html/2608.28934#A10): Empirical Privacy Results Across Audit Methods\.Fully specifyingεlb\\varepsilon\_\{\\textup\{lb\}\}computed under GDP and CP methodology with different holdout ratios, as described in Appendix[A](https://arxiv.org/html/2608.28934#A1)\.

## Appendix BDisparate Impact

A common concern for any filtering\-based algorithm is disparate impact\. We evaluate our filtering defense on the ColoredMNIST dataset\[[6](https://arxiv.org/html/2608.28934#bib.bib20)\], a variant of MNIST with 2 classes \(class 0 has even digits and class 1 has odd digits\) wherep%p\\%of evens and\(1−p\)%\(1\-p\)\\%of odds are colored red;\(1−p\)%\(1\-p\)\\%of evens andp%p\\%of odds are colored blue\. This creates symmetric majority and minority subgroups within both classes\. Due to its distribution of relevant \(shape\) and irrelevant \(color\) features, balanced by group ratios, ColoredMNIST is a popular synthetic benchmark in the algorithmic fairness and spurious correlations literature\[[56](https://arxiv.org/html/2608.28934#bib.bib21),[72](https://arxiv.org/html/2608.28934#bib.bib22)\]\.

We consider ColoredMNIST under the following values ofpp:99\.5%,95%,90%,85%,80%,75%99\.5\\%,95\\%,90\\%,85\\%,80\\%,75\\%\. For each instantiation of ColoredMNIST, we evaluate how many samples are filtered by our defense from each of the 4 subgroups, and the resulting impact on groupwise model utility\.

Table[5](https://arxiv.org/html/2608.28934#A2.T5)indicates that our defense does target minority groups over majority groups when marking samples for removal, yielding a disparate reduction in minority group utility\. This targeting effect becomes more benign as the fractionppbecomes more mild, as expected\.

Moreover, we find that the observed bias towards minority group removal is in some sense definitional from the perspective of sample memorization, which any method with a low auditableεlb\\varepsilon\_\{\\textup\{lb\}\}must combat\. This finding is thematically consistent with other works investigating tradeoffs between privacy and fairness\[[7](https://arxiv.org/html/2608.28934#bib.bib23),[66](https://arxiv.org/html/2608.28934#bib.bib57)\]\. To formalize this correlation, we audit ColoredMNIST \(atp=75%p=75\\%\) without our defense twice, once with the canary as a member of a minority subgroup and once again with the canary as a member of a majority subgroup\. Under the minority\-group canary, we observeεlb=1\.145\\varepsilon\_\{\\textup\{lb\}\}=1\.145whereas under the majority\-group canary, we observeεlb=0\.043\\varepsilon\_\{\\textup\{lb\}\}=0\.043, indicating that minority\-group canaries are indeed more prone to memorization\.

Table 5:Defense Removal Rate \(%\): mean±\\pmstd over repetitions
## Appendix CAlternative Input Space Audits

### C\.1Single run audit

A recent work\[[61](https://arxiv.org/html/2608.28934#bib.bib54)\]provides a computationally cheaper alternative to LiRA\-style audits by requiring only a single model training run instead of2​T2Tshadow models \(TTbeing the number of repetitions\)\. This audit measures individual privacy by generatingNNcanaries and independently selectingMMof them at random for inclusion in the training dataset\. After training, we compute a score \(cross entropy loss\) for each of the originalNNcanaries\. We then makek\+k\_\{\+\}guesses about which canaries were included in the training andk−k\_\{\-\}guesses about which were excluded, abstaining from guessing on the remaining canaries\. The TPR and FPR from these guesses are used to compute anεlb\\varepsilon\_\{\\textup\{lb\}\}via theff\-DP framework\. We evaluate our defense under the multi\-canary individual privacy audit using 500 randomly mislabeled canaries in CIFAR\-10\. We perform a grid search over all pairs \(k\+k\_\{\+\},k−k\_\{\-\}\) and report the maximumεlb\\varepsilon\_\{\\textup\{lb\}\}achieved \(Table[6](https://arxiv.org/html/2608.28934#A3.T6)\)\. With our defense, we filter out approximately75%75\\%of the canaries, substantially reducing theεlb\\varepsilon\_\{\\textup\{lb\}\}\.

### C\.2Group privacy audit

We extended LiRA to group privacy by using the maximum loss across all canaries as the audit score for distinguishing between models trained versus without the canary group\. We evaluate our defense under this threat model usingk=500k=500mislabeled canaries on CIFAR\-10 \(Table[6](https://arxiv.org/html/2608.28934#A3.T6)\)\. We evaluate privacy usingkk\-group privacyεlb\\varepsilon\_\{\\textup\{lb\}\}; as discussed in Section[4\.1](https://arxiv.org/html/2608.28934#S4.SS1), this value does not formally imply a11\-group privacyεlb\\varepsilon\_\{\\textup\{lb\}\}that isk×k\\timessmaller, but this is a reasonable scaling to expect\. We find that even in the group privacy setting, our defense substantially reducesεlb\\varepsilon\_\{\\textup\{lb\}\}\.

Table 6:Empirical privacy loss under individual and group privacy audits\.

## Appendix DSample Signatures

This experiment tested the sensitivity of the local filtering defense to the choice of scoring function used to identify high\-risk samples\. Using these scoring functions, we can detect backdoor canaries that spectral methods\[[65](https://arxiv.org/html/2608.28934#bib.bib56)\]may fail to identify\. For each attack, the scoring function should identify and filter out canaries as early as possible in training\. Filtering canaries too late is ineffective since they have already been memorized\. Future work could address late\-stage canary detection through privatized gradient ascent\.

On CIFAR\-10, we considered the following scoring functions\.

1. 1\.Gradient cosine similarity: how aligned sample gradient direction is with the overall parameter update direction from initialization\.
2. 2\.L2L\_\{2\}gradient norm: magnitude of their clipped per\-sample gradient under theL2L\_\{2\}norm\. This is computed using both clipped and unclipped gradients\.
3. 3\.L∞L\_\{\\infty\}gradient norm: magnitude of the clipped per\-sample gradient under theL∞L\_\{\\infty\}norm\. This is computed using both clipped and unclipped gradients\.
4. 4\.Gradient kurtosis unclipped: kurtosis of the gradient distribution, emphasizing heavy\-tailed or outlier\-like gradients\.
5. 5\.Directional uniqueness: combines gradient magnitude with a proxy for directional “uniqueness” based on how atypical a sample’s gradient direction is relative to a running history\.
6. 6\.Prediction entropy: entropy of the predicted probability distribution, emphasizing high\-uncertainty predictions\.
7. 7\.Prediction margin: inverted gap between the predicted probability of the true class and the strongest competing class\.
8. 8\.Random projection variability: projects gradients onto random directions and measures variability in those projections, capturing atypical gradient structure\.

All other defense parameters were kept fixed \(filtering bandwidth, gradient ascent step, etc\.\)\. Across the scoring function choices, the defense consistently detected and removed the canary, with low empirical privacy loss and utility metrics similarly high across configurations \(Table[7](https://arxiv.org/html/2608.28934#A4.T7)\)\. This allows practitioners to select or tune the scoring function that best matches their constraints and threat model without changing the defense behavior\.

Table 7:Scoring function ablation on CIFAR\-10 \(CNN\)\.
## Appendix EPoisson vs\. Shuffled Subsampling

As discussed in Section[3\.2](https://arxiv.org/html/2608.28934#S3.SS2), in practice, many large\-scale implementations of DP\-SGD favor shuffled mini\-batching over Poisson sampling for efficiency reasons; we therefore verify that our defense remains effective under this more practical sampling scheme\. Concretely, we compared the results of estimatingεlb\\varepsilon\_\{\\textup\{lb\}\}after running our defense versus no defense on a blank canary, for the MNIST dataset, with both Poisson sampling and shuffled subsampling \(Figure[3](https://arxiv.org/html/2608.28934#A5.F3)\)\. The results show that the gap in the estimatedεlb\\varepsilon\_\{\\textup\{lb\}\}persists regardless of sampling strategy\.

![Refer to caption](https://arxiv.org/html/2608.28934v1/new_poisson_vs_shuffled_epsilon.png)Figure 3:εlb\\varepsilon\_\{\\textup\{lb\}\}with/without defense on MNIST \(CNN\) across Poisson and shuffled sampling methods\.
## Appendix FInterpolating Canaries

We linearly interpolated a blank canary with a mislabeled canary with varying degrees of interpolation to test how the effectiveness of the defense scales with canary strength\. We constructed canaries as \(α⋅mislabeled sample\\alpha\\cdot\\text\{mislabeled sample\}\)\. Whenα\\alphais 0 the canary is a pure blank image; whenα\\alphais 1, it is a fully mislabeled image\. We evaluated the setup on both MNIST and CIFAR\-10 \(Table[8](https://arxiv.org/html/2608.28934#A6.T8)\)\.

We found that varyingα\\alphadid not produce a clear change in privacy loss\. This suggests that privacy loss is not necessarily proportional to the “strength” of the signal, and the defense remained effective across the different canary constructions\.

Table 8:Empiricalεlb\\varepsilon\_\{\\mathrm\{lb\}\}across interpolated canary strengths on CIFAR\-10 and MNIST \(CNN\)\.
## Appendix GThe Privacy Onion Effect: Do We Expose Anyone Else?

Prior work\[[12](https://arxiv.org/html/2608.28934#bib.bib28)\]finds that removing the most privacy vulnerable samples from a dataset may increase the vulnerability of retained samples rather than yielding an aggregate privacy improvement\. This “privacy onion” effect suggests that empirical privacy can offer a false sense of security and practitioners ought to employ algorithms with formal privacy guarantees rather than optimizing on empirical privacy\. This phenomenon is algorithm\-specific, so we analyze its presence for our specific filtering\-based defense in this section, by auditing*non\-canary samples*\.

To empirically test our defense, we trained 1000 shadow models \(trained with DP\-SGDε=10\\varepsilon=10; 500 with the blank canary, 500 without\) and recorded per\-sample loss under each final model for every sample in the MNIST training dataset\. For each non\-canary sample, we compute theεlb\\varepsilon\_\{\\textup\{lb\}\}post\-defense and compare this to the maximumεlb\\varepsilon\_\{\\textup\{lb\}\}pre\-defense \(corresponding to the canary’sεlb\\varepsilon\_\{\\textup\{lb\}\}\)\. Our results are striking: the \(adaptively\-chosen\) strongest non\-canary audit produces a much weakerεlb\\varepsilon\_\{\\textup\{lb\}\}than a fixed canary audit\. This is true whether or not the formal holdout \(false discovery correction\) method in Appendix[A](https://arxiv.org/html/2608.28934#A1)is used, on both the canary audit and the non\-canary audit; in the holdout case, the auditableεlb\\varepsilon\_\{\\textup\{lb\}\}of the best non\-canary audit drops to00\.

Our auditedεlb\\varepsilon\_\{\\textup\{lb\}\}comparing the canary with the strongest non\-canary sample are displayed in Figures[4](https://arxiv.org/html/2608.28934#A7.F4)and[5](https://arxiv.org/html/2608.28934#A7.F5)\. The experiments in these figures respectively apply the multiple hypothesis heuristic from Appendix[A](https://arxiv.org/html/2608.28934#A1), and use a holdout set to estimateεlb\\varepsilon\_\{\\textup\{lb\}\}, for all audits\.

![Refer to caption](https://arxiv.org/html/2608.28934v1/ultra_new_do_we_expose_1.png)Figure 4:Distribution of empiricalεlb\\varepsilon\_\{\\textup\{lb\}\}for non\-canary training samples \(post\-defense\) vs\. canaryεlb\\varepsilon\_\{\\textup\{lb\}\}\(pre\-defense\) across two auditing methods\. In both cases, 0\.0% of non\-canary samples exceed the canary reference\.![Refer to caption](https://arxiv.org/html/2608.28934v1/ultra_new_do_we_expose_2.png)Figure 5:Empiricalεlb\\varepsilon\_\{\\textup\{lb\}\}of the worst\-case non\-canary sample post\-defense vs\. canaryεlb\\varepsilon\_\{\\textup\{lb\}\}pre\-defense, under GDP 50% holdout and CP 50% holdout\. The worst\-case non\-canary sample is selected as the sample achieving the highest empiricalεlb\\varepsilon\_\{\\textup\{lb\}\}under no defense; its empiricalεlb\\varepsilon\_\{\\textup\{lb\}\}drops to 0\.000 post\-defense under both auditing methods\.
## Appendix HIncreasing the Trial Count

Our primary experiments used 400 shadow models \(200 with the canary included, 200 without\) to balance computational constraints\. However, the Clopper\-Pearson confidence intervals used to compute empirical privacy bounds are looser with fewer trials\. We conducted a scaled\-up experiment on MNIST with a blank canary using 1000 shadow models \(500 per condition\) instead of the standard 400\. We compared theεlb\\varepsilon\_\{\\textup\{lb\}\}and model utility between defense and non\-defense settings under the higher sample regime \(Table[9](https://arxiv.org/html/2608.28934#A8.T9)\)\. Theεlb\\varepsilon\_\{\\textup\{lb\}\}gap with and without the defense remained consistent with the 400 trial experiments\.

Table 9:Empirical privacy loss on MNIST \(CNN\) with 1000 shadow models\.
## Appendix IAdditional Hyperparameter Tuning

### I\.1Global Filtering Ablation

We conducted experiments on MNIST using a blank input space canary to evaluate how global filtering \(selecting the top\-kksamples across the entire dataset\) compares with local filtering \(selecting the top\-kksamples within each class\)\. Our experiments bias towards local filtering to align with prior work\[[65](https://arxiv.org/html/2608.28934#bib.bib56)\], which found that backdoored data poisons exhibit a stronger signature when compared relative to other samples in the same class rather than the entire dataset\. We tested three global filtering bandwidths:k∈\{10,25,50\}k\\in\\\{10,25,50\\\}samples filtered per epoch \(Table[10](https://arxiv.org/html/2608.28934#A9.T10)\)\. For comparison, our default per\-class filtering withk=5k=5filters, for 50 samples per epoch in total \(5 samples per class across 10 classes\)\.

Global filtering achieved similar empirical privacy and test accuracy compared to per\-class filtering across all bandwidth configurations\. These results suggest that practitioners can choose between filtering strategies based on implementation convenience or specific dataset characteristics rather than privacy\-utility tradeoffs alone\. For instance, a model trainer may prefer global filtering to limit the total number of discarded samples, or local filtering when class imbalance is a concern\. Future work could explore adaptive scheduling approaches that monitor training dynamics and switch between global and local filtering on a per\-epoch basis\.

Table 10:Global filtering ablation on MNIST \(CNN\)\.
### I\.2Varying Local Filtering Defense Bandwidth Ablation

To empirically evaluate how the bandwidth parameterkk\(the number of samples filtered per class per epoch\) affects the defense efficacy, we conducted experiments on CIFAR\-10 using a blank canary with gradient\-norm based local filtering\. We tested bandwidth valuesk∈\{1,2,3,4,5\}k\\in\\\{1,2,3,4,5\\\}\(Table[11](https://arxiv.org/html/2608.28934#A9.T11)\)\. The defense reduced empirical privacy loss across all bandwidth configurations, though the degree of reduction varied withkk\. These results suggest that the choice of k does have some effect on empirical privacy, and practitioners should consider tuning it alongside computational and utility constraints\.

Table 11:Local filtering bandwidth ablation on CIFAR\-10 \(CNN\)\.
### I\.3Varying Filter Frequency Ablation

In this experiment, we varied the filtering frequency during training, applying the defense everykkepochs fork∈\{1,5,10,20\}k\\in\\\{1,5,10,20\\\}\. Experiments used the CIFAR\-10/CNN architecture with a blank input space canary\. We observed that filtering frequency has a meaningful impact on empirical privacy outcomes, with more frequent filtering achieving the lowestε\\varepsilonlb of 0\.43, while less frequent filtering led to substantially higher loss, peaking at 1\.39 when filtering every 10 epochs \(Table[12](https://arxiv.org/html/2608.28934#A9.T12)\)\. These results suggest that practitioners should prefer more frequent filtering when empirical privacy is a priority, though utility metrics remained stable across all configurations\. Filtering frequency may become a more critical hyperparameter for complex canaries or auditing threat models, where model trainers seek to limit the number of discarded samples during training\. Future work could explore adaptive scheduling approaches that monitor training dynamics and apply heuristics to determine when filtering is necessary on a per\-epoch basis\.

Table 12:Effect of filtering frequency on CIFAR\-10 \(CNN\)\.

## Appendix JFull Empirical Privacy Results Across Audit Methods

In all experiments throughout the paper, we reportεlb\\varepsilon\_\{\\textup\{lb\}\}using the GDP no holdout auditing method as our primary metric\. For completeness, we report full results across all auditing configurations\. For GDP, we vary the fraction of shadow models held out for threshold selection at25%25\\%,50%50\\%, and75%75\\%\. For CP, we include results with no holdout as well as the same three holdout levels\. These additional metrics provide a more comprehensive picture of the empirical privacy landscape across all experiments, and allow the reader to assess the robustness of our conclusions to the choice of auditing method\.

Table 13:Full empirical privacy results across privacy budgetsεub∈\{2,4,6,8,10\}\\varepsilon\_\{\\mathrm\{ub\}\}\\in\\\{2,4,6,8,10\\\}, datasets, and model architectures\. See Section[4\.2](https://arxiv.org/html/2608.28934#S4.SS2)\.Table 14:Scoring function ablation on CIFAR\-10 \(CNN\): full empirical privacy results across auditing methods\. See Section[D](https://arxiv.org/html/2608.28934#A4)\.Table 15:Empirical privacy loss on MNIST \(CNN\) with 1000 shadow models, full results across auditing methods\. See Section[H](https://arxiv.org/html/2608.28934#A8)\.Table 16:εlb\\varepsilon\_\{\\mathrm\{lb\}\}with/without defense on MNIST \(CNN\) across Poisson and shuffled sampling methods, full results across auditing methods\. See Section[E](https://arxiv.org/html/2608.28934#A5)\.Table 17:Full empirical privacy results under group privacy \(multi\-canary\) mislabeled audits\. See Section[C\.2](https://arxiv.org/html/2608.28934#A3.SS2)\.Table 18:Full empirical privacy results for the fairness audit on Colored MNIST, under the no\-defense setting\. See Section[B](https://arxiv.org/html/2608.28934#A2)\.Table 19:Full empirical privacy results under ClipBKD attacks on MNIST and CIFAR\-10 \(CNN\)\. See Section[4\.2](https://arxiv.org/html/2608.28934#S4.SS2)\.Table 20:Full empirical privacy results under FGSM attacks on MNIST and CIFAR\-10 \(CNN\)\. See Section[4\.2](https://arxiv.org/html/2608.28934#S4.SS2)\.Table 21:Full empirical privacy results under mislabeled input space attacks on MNIST and CIFAR\-10 \(CNN\)\. See Section[4\.2](https://arxiv.org/html/2608.28934#S4.SS2)\.Table 22:Full empirical privacy results under the gradient canceling attack on MNIST \(CNN\)\. See Section[4\.5](https://arxiv.org/html/2608.28934#S4.SS5)\.Table 23:Full empirical privacy results across interpolated canary strengths \(α\\alpha\) on CIFAR\-10 and MNIST \(CNN\)\. See Section[F](https://arxiv.org/html/2608.28934#A6)\.Table 24:Full empirical privacy results across filter frequencies on CIFAR\-10 \(CNN\)\. See Section[I\.3](https://arxiv.org/html/2608.28934#A9.SS3)\.Table 25:Full empirical privacy results across local filtering bandwidth values on CIFAR\-10 \(CNN\)\. See Section[I\.2](https://arxiv.org/html/2608.28934#A9.SS2)\.Table 26:Full empirical privacy results for the defense\-aware audit on MNIST \(CNN\), comparing standard mislabeled canaries against the fixed\-point attack\. See Section[4\.4](https://arxiv.org/html/2608.28934#S4.SS4)\.Table 27:Full empirical privacy results for global filtering ablation on MNIST \(CNN\)\. See Section[I\.1](https://arxiv.org/html/2608.28934#A9.SS1)\.Table 28:Full empirical privacy results under the gradient bandwidth attack on MNIST \(CNN\)\. See Section[4\.5](https://arxiv.org/html/2608.28934#S4.SS5)\.