Loss Curvature as a Sampling-Free Proxy for Memorization in Diffusion Models

# Loss Curvature as a Sampling-Free Proxy for Memorization in Diffusion Models
Source: [https://arxiv.org/html/2610.19042](https://arxiv.org/html/2610.19042)
Xinyue YuBoonam ShinYuxuan LiuAbolfazl HashemiHayden Schaeffer

###### Abstract

Understanding the mechanism behind why diffusion models memorize their training data remains an open question in generative modeling\. In this paper, we examine this question from two perspectives: \(a\) the model\-level perspective concerning the onset of memorization in a model during training, and \(b\) the sample\-level perspective concerning the proclivity of an individual sample to be memorized by a given model\. We show that the so\-called input loss curvature, or the Frobenius norm of the Hessian of the diffusion model’s training loss, is an effective metric for quantifying both perspectives on memorization\. In particular, we empirically demonstrate that input curvatures averaged across training and validation samples can identify when a model is likely to start regurgitating its training data when tracked over training time\. However, this metric is not sufficient for measuring an individual sample’s likelihood of being memorized\. To address this perspective, we introduce a counterfactual definition of memorization in diffusion models, extending the analogous notion from supervised learning proposed by \[Feldman, 2019\]\. We prove that this counterfactual memorization is controlled by the input curvature of the leave\-one\-out model, thus establishing a direct theoretical link between a sample’s local loss landscape and its generation likelihood\. We validate our derived bounds through several empirical experiments and show that input\-dependent curvature provides a proxy for memorization\.

$\\ast$$\\ast$footnotetext:Equal contribution\.11footnotetext:Department of Mathematics, University of California, Los Angeles, CA 90095, USA\.22footnotetext:Elmore Family School of Electrical and Computer Engineering, Purdue University, West Lafayette, IN 47907, USA\.## 1Introduction

Diffusion and score\-based models are state\-of\-the\-art generative models across domains from vision to drug discovery\[[40](https://arxiv.org/html/2610.19042#bib.bib11),[21](https://arxiv.org/html/2610.19042#bib.bib12),[44](https://arxiv.org/html/2610.19042#bib.bib44),[45](https://arxiv.org/html/2610.19042#bib.bib13),[35](https://arxiv.org/html/2610.19042#bib.bib45),[31](https://arxiv.org/html/2610.19042#bib.bib15),[1](https://arxiv.org/html/2610.19042#bib.bib48),[32](https://arxiv.org/html/2610.19042#bib.bib49)\]\. Their appeal lies in generating novel yet realistic data, suggesting they estimate the data distribution from finitely many samples\. Yet diffusion models sometimes regurgitate training samples, a failure mode called*memorization*, which poses privacy and copyright risks\[[12](https://arxiv.org/html/2610.19042#bib.bib27),[41](https://arxiv.org/html/2610.19042#bib.bib25),[42](https://arxiv.org/html/2610.19042#bib.bib26),[20](https://arxiv.org/html/2610.19042#bib.bib24)\]\. Existing work studies memorization at two levels:

1. 1\.*Model\-level memorization*asks how closely the trained score matches the minimizer of the empirical denoising score matching loss\[[48](https://arxiv.org/html/2610.19042#bib.bib18),[5](https://arxiv.org/html/2610.19042#bib.bib51)\], namely the score of the Gaussian KDE of the training set, whose sampling reproduces the training data exactly\. Tracking training timeτ\\tau, models first generalize \(atτg​e​n\\tau\_\{gen\}\) and later memorize \(atτm​e​m\\tau\_\{mem\}\)\[[8](https://arxiv.org/html/2610.19042#bib.bib19),[7](https://arxiv.org/html/2610.19042#bib.bib20),[50](https://arxiv.org/html/2610.19042#bib.bib22),[2](https://arxiv.org/html/2610.19042#bib.bib21)\], but these time\-scales are known only asymptotically, and measuring the memorized fractionfm​e​m​\(τ\)f\_\{mem\}\(\\tau\)requires costly sampling\.
2. 2\.*Sample\-level memorization*asks which training points are copied\. Previous work attributes sample\-level memorization to the presence of duplicates\[[12](https://arxiv.org/html/2610.19042#bib.bib27),[42](https://arxiv.org/html/2610.19042#bib.bib26),[52](https://arxiv.org/html/2610.19042#bib.bib28),[13](https://arxiv.org/html/2610.19042#bib.bib46)\]\. More generally, sample\-level memorization refers to the influence of an individual sample on a model\. This connects to to the long\-tail theory of\[[18](https://arxiv.org/html/2610.19042#bib.bib4)\], which motivates the counterfactual memorization measure of\[[17](https://arxiv.org/html/2610.19042#bib.bib3)\]\(see also\[[51](https://arxiv.org/html/2610.19042#bib.bib47)\]\)\. Extending this measure to diffusion models is nontrivial because the training objective \(denoising loss\) differs from the test objective \(likelihood\), and generalization in diffusion models has its own geometric structure\[[23](https://arxiv.org/html/2610.19042#bib.bib23)\]\.

These perspectives motivate*sampling\-free*proxies of memorization which can ideally be measured during training\. We propose using*input loss curvature*from\[[34](https://arxiv.org/html/2610.19042#bib.bib30)\]given by

Curvϕ​\(x\)=𝔼ϕ∼𝒜⁡\(S\)​\[‖∇x2ℓ​\(ϕ,x\)‖F2\],\\displaystyle\\text\{Curv\}\_\{\\phi\}\(x\)=\\mathbb\{E\}\_\{\\phi\\sim\\mathcal\{A\}\(S\)\}\\left\[\\\|\\nabla^\{2\}\_\{x\}\\ell\(\\phi,x\)\\\|^\{2\}\_\{F\}\\right\],\(1\)as such a proxy\. We use it for two reasons: \(i\) it is the leading order term that bounds Feldman’s memorization score, is among the strongest membership\-inference signals, and estimates differential privacy\[[33](https://arxiv.org/html/2610.19042#bib.bib2),[34](https://arxiv.org/html/2610.19042#bib.bib30),[19](https://arxiv.org/html/2610.19042#bib.bib31),[39](https://arxiv.org/html/2610.19042#bib.bib32),[11](https://arxiv.org/html/2610.19042#bib.bib33),[16](https://arxiv.org/html/2610.19042#bib.bib29)\]; \(ii\) the Hessian of the potential−log⁡p⁡\(x\)\-\\log p\(x\)governs the local geometry of sampling, with higher curvature confining Langevin trajectories near a minimizer for longer\[[10](https://arxiv.org/html/2610.19042#bib.bib34),[6](https://arxiv.org/html/2610.19042#bib.bib35)\]\. However, during training it is prohibitively expensive to compute these negative log likelihoods–instead, we investigate whether the second order information on the loss can help predict the properties implied by the second order information of the negative log likelihood\.

#### Contributions\.

1. \(a\)*Curvature over training image tracks memorization\.*We demonstrate that on ImageNet64×6464\\times 64and CelebA32×3232\\times 32,Curvϕ​\(x\)\\text\{Curv\}\_\{\\phi\}\(x\)averaged over the training set corresponds to thefm​e​mf\_\{mem\}score and rises as the model overfits\. Moreover, at the onset of memorization, samples that are copied \(i\.e appear to be in the training set\) have higher curvature than samples that are not copied\.
2. \(b\)*Held\-out curvature separates from training curvature during memorization\.*While curvature over training images rises, the curvature over the validation set starts separating from the training curvature at the onset of memorization\. Surprisingly, validation curvature hits a minimum during training and starts rising as memorization proceeds, while validation loss continues to decrease\. The local minimum of validation curvature provides an early\-stopping measure for diffusion models that indicates the prevalence of model\-level memorization\.
3. \(c\)*A counterfactual notion of memorization\.*We illustrate that while curvature indicates model\-level memorization, it cannot signal the copy frequency of individual samples\. To quantify such individual sample influence, we introduce a new notion of memorization, termed counterfactual memorization given by the log\-odds of samplingxix\_\{i\}with and without its inclusion in the training data\.
4. \(d\)*Counterfactual memorization is bounded by curvature\.*By lifting the differences in log\-likelihoods to model losses via a decomposition given in\[[22](https://arxiv.org/html/2610.19042#bib.bib10)\], we quantify the influence of training dynamics on counterfactual memorization\. In particular, we demonstrate that when the model starts overfitting, the log\-likelihood difference is captured at the leading order by the curvature of the leave\-one\-out model and the nearest neighbour distance\.

## 2Background

### 2\.1Diffusion Models

A diffusion model is a generative algorithm for sampling from an unknown densityppby transforming noise into clean data\. To learn such a model we first establish the following forward process:

d​Yt=f⁡\(t,Yt\),d​t\+g⁡\(t\),d​Wt,Y0=x∼p,\\displaystyle dY\_\{t\}=f\(t,Y\_\{t\}\),dt\+g\(t\),dW\_\{t\},\\qquad Y\_\{0\}=x\\sim p,\(2\)Whenf⁡\(t,y\)=f⁡\(t\)​yf\(t,y\)=f\(t\)y, the transition kernel is Gaussian:Yt\|Y0=x∼𝒩⁡\(m⁡\(t\)​x,σ​\(t\)2​I\)Y\_\{t\}\\mid Y\_\{0\}=x\\sim\\mathcal\{N\}\(m\(t\)x,\\sigma\(t\)^\{2\}I\)\. The schedules\(m,σ\)\(m,\\sigma\)and the coefficients\(f,g\)\(f,g\)thus determine each other:

f\(t\)=m′​\(t\)m⁡\(t\),g\(t\)2=dd​tσ\(t\)2−2m′​\(t\)m⁡\(t\),σ\(t\)2\.\\displaystyle f\(t\)=\\frac\{m^\{\\prime\}\(t\)\}\{m\(t\)\},\\qquad g\(t\)^\{2\}=\\frac\{d\}\{dt\}\\sigma\(t\)^\{2\}\-2\\frac\{m^\{\\prime\}\(t\)\}\{m\(t\)\},\\sigma\(t\)^\{2\}\.\(3\)Anderson’s theorem\[[3](https://arxiv.org/html/2610.19042#bib.bib1)\]gives the backward SDE, whose law at timettis that ofYT−tY\_\{T\-t\}:

d​X¯t\\displaystyle d\\overline\{X\}\_\{t\}=\(g¯\(t\)2∇logpT−t\(X¯t\)−f¯\(t,X¯t\)\),dt\\displaystyle=\\big\(\\bar\{g\}\(t\)^\{2\}\\nabla\\log p\_\{T\-t\}\(\\overline\{X\}\_\{t\}\)\-\\bar\{f\}\(t,\\overline\{X\}\_\{t\}\)\\big\),dt\(4\)\+g¯​\(t\),d​Bt\\displaystyle\+\\bar\{g\}\(t\),dB\_\{t\}\(5\)wheref¯​\(t,x\)=f​\(T−t,x\)\\bar\{f\}\(t,x\)=f\(T\-t,x\)andg¯​\(t\)=g​\(T−t\)\\bar\{g\}\(t\)=g\(T\-t\)\. Thus, givenffandgg, sampling fromppreduces to knowing the score∇log⁡pt\\nabla\\log p\_\{t\}and running the backward SDE fromt=0t=0tot=Tt=T\. Via the Tweedie\-Miyasawa formula

𝔼⁡\[X∣Yt=y\]=y\+σ2\(t\)∇logp\(t,y\)m⁡\(t\),\\displaystyle\\mathbb\{E\}\[X\\mid Y\_\{t\}=y\]=\\frac\{y\+\\sigma^\{2\}\(t\)\\nabla\\log p\(t,y\)\}\{m\(t\)\},\(6\)we can instead parameterize the denoiser𝔼⁡\[X∣Yt=y\]\\mathbb\{E\}\[X\\mid Y\_\{t\}=y\]and recover the score by rearranging \([6](https://arxiv.org/html/2610.19042#S2.E6)\)\. We therefore train a modelDϕD\_\{\\phi\}by MMSE denoising:

ℓ\(ϕ,x\):=𝔼t∈\[0,T\],yt∼Yt\[λ\(t\)∥x−Dϕ\(yt,t\)\]∥22∣Y0=x\]\.\\displaystyle\\ell\(\\phi,x\):=\\mathbb\{E\}\_\{t\\in\[0,T\],y\_\{t\}\\sim Y\_\{t\}\}\\left\[\\lambda\(t\)\\\|x\-D\_\{\\phi\}\(y\_\{t\},t\)\]\\\|\_\{2\}^\{2\}\\mid Y\_\{0\}=x\\right\]\.\(7\)Hereλ⁡\(t\)\\lambda\(t\)is a choice of weighting function\. Substituting the learned scoresϕ=σ​\(t\)−2​\(m⁡\(t\)​Dϕ−y\)s\_\{\\phi\}=\\sigma\(t\)^\{\-2\}\(m\(t\)D\_\{\\phi\}\-y\)gives the*plug\-in SDE*:

d​Xt=\(g¯​\(t\)2​s​\(t,Xt\)−f⁡\(t,Xt\)¯\)​d​t\+g¯​\(t\)​d​Bt\.\\displaystyle dX\_\{t\}=\(\\overline\{g\}\(t\)^\{2\}s\(t,X\_\{t\}\)\-\\overline\{f\(t,X\_\{t\}\)\}\)\\,dt\+\\overline\{g\}\(t\)\\,dB\_\{t\}\.\(8\)
The plug\-in SDE has densitypϕ​\(y,t\)p\_\{\\phi\}\(y,t\)\. At generation time we integrate \([8](https://arxiv.org/html/2610.19042#S2.E8)\) forward, obtaining the terminal distributionpϕ​\(⋅,T\)p\_\{\\phi\}\(\\cdot,T\)as an approximation to the true distributionpp\.

### 2\.2DDPMs

In this paper, we use Denoising Diffusion Probabilistic Models \(DDPMs\)\[[21](https://arxiv.org/html/2610.19042#bib.bib12)\]corresponding to takingf⁡\(t\)=−12​β​\(t\)f\(t\)=\-\\tfrac\{1\}\{2\}\\beta\(t\)andg⁡\(t\)=β⁡\(t\)g\(t\)=\\sqrt\{\\beta\(t\)\}for some non\-negative rate functionβ\\beta\. Writingα¯\(t\):=exp\(−∫0tβ\(u\)du\)\\bar\{\\alpha\}\(t\):=\\exp\\left\(\-\\int\_\{0\}^\{t\}\\beta\(u\)\\,du\\right\), one findsΦ⁡\(t\)=α¯​\(t\)\\Phi\(t\)=\\sqrt\{\\bar\{\\alpha\}\(t\)\}and, usingdd​s​α¯​\(s\)−1=β⁡\(s\)​α¯​\(s\)−1\\frac\{d\}\{ds\}\\bar\{\\alpha\}\(s\)^\{\-1\}=\\beta\(s\)\\bar\{\\alpha\}\(s\)^\{\-1\},

m⁡\(t\)=α¯​\(t\),σ​\(t\)2=1−α¯​\(t\)\.\\displaystyle m\(t\)=\\sqrt\{\\bar\{\\alpha\}\(t\)\},\\qquad\\sigma\(t\)^\{2\}=1\-\\bar\{\\alpha\}\(t\)\.\(9\)To sample from DDPMs, we use an accelerated non\-Markovian sampler given in\[[43](https://arxiv.org/html/2610.19042#bib.bib14)\]\. More generally, note that the sampling process via \([8](https://arxiv.org/html/2610.19042#S2.E8)\) is implemented by discretizing the SDE via Euler\-Maruyama, leading to a stochastic sampling trajectory\. A deterministic sampling, using the probability flow ODE or flow\-matching may also be used\[[45](https://arxiv.org/html/2610.19042#bib.bib13),[25](https://arxiv.org/html/2610.19042#bib.bib16)\]\. In Appendix[B](https://arxiv.org/html/2610.19042#A2), we provide experimental details and additional details on flow matching\.

### 2\.3Log\-Likelihoods via the Denoising Loss

Diffusion models are trained by using the loss \([7](https://arxiv.org/html/2610.19042#S2.E7)\)\. However, at test time, their performance is measured by the fidelity ofp⁡\(⋅,T\)p\(\\cdot,T\)to the true distributionpp\. Therefore, their training and test tasks are rather different\. However, an evidence lower bound \(ELBO\) shows that the denoising loss bounds the negative log\-likelihood of the distributionpϕ​\(⋅,T\)p\_\{\\phi\}\(\\cdot,T\)\. In fact,\[[22](https://arxiv.org/html/2610.19042#bib.bib10)\]described the gap between the ELBO and−log⁡pϕ​\(⋅,T\)\-\\log p\_\{\\phi\}\(\\cdot,T\)exactly under the likelihood weightingλ​\(t\)=g2​\(t\)\\lambda\(t\)=g^\{2\}\(t\):

###### Proposition 2\.1\(Theorem 4 in\[[22](https://arxiv.org/html/2610.19042#bib.bib10)\]\)\.

Letℓ⁡\(ϕ,x\)\\ell\(\\phi,x\)be the denoising loss \([7](https://arxiv.org/html/2610.19042#S2.E7)\),pϕ​\(x,T\)p\_\{\\phi\}\(x,T\)the distribution of the plug\-in SDE \([8](https://arxiv.org/html/2610.19042#S2.E8)\) at timeTTandsϕs\_\{\\phi\}the learned score function in \([8](https://arxiv.org/html/2610.19042#S2.E8)\)\. Then

−log⁡pϕ​\(x,T\)−ℓ⁡\(ϕ,x\)=C\\displaystyle\-\\log p\_\{\\phi\}\(x,T\)\-\\ell\(\\phi,x\)=C\+∫0T𝔼⁡\[g2​\(t\)​‖sϕ​\(Yt,t\)−∇logϕ​\(Yt,T−t\)‖22∣Y0=x\]​dt⏟Gapϕ​\(x\),\\displaystyle\+\\underbrace\{\\int\_\{0\}^\{T\}\\mathbb\{E\}\\left\[g^\{2\}\(t\)\\\|s\_\{\\phi\}\(Y\_\{t\},t\)\-\\nabla\\log\_\{\\phi\}\(Y\_\{t\},T\-t\)\\\|\_\{2\}^\{2\}\\mid Y\_\{0\}=x\\right\]\\,dt\}\_\{\\text\{Gap\}\_\{\\phi\}\(x\)\},\(10\)whereCCis a constant independent ofxxorϕ\\phibut dependent on the data dimensiondd, and the noise schedulesf,gf,g\.

The RHS in \([10](https://arxiv.org/html/2610.19042#S2.E10)\) is purely a*self\-consistency*gap, i\.e it measures how close the score of the reversed plug\-in SDE is to the original forward SDE\.

### 2\.4Memorization

Here we summarize the framework of\[[18](https://arxiv.org/html/2610.19042#bib.bib4),[17](https://arxiv.org/html/2610.19042#bib.bib3)\]\. Let𝒜\\mathcal\{A\}be a training algorithm, possibly randomized, and writehSϕh^\{\\phi\}\_\{S\}for the model with parametersϕ∼𝒜⁡\(S\)\\phi\\sim\\mathcal\{A\}\(S\)\. Letℓ⁡\(hSϕ,z\)\\ell\(h^\{\\phi\}\_\{S\},z\)be its loss on an examplezz, and letS∖i=S∖ziS^\{\\setminus i\}=S\\setminus\{z\_\{i\}\}\. The memorization ofziz\_\{i\}is the change in expected loss atziz\_\{i\}caused by its inclusion in the training set,

M⁡\(zi,S\)\\displaystyle M\(z\_\{i\},S\)=\|,𝔼ϕ∼𝒜⁡\(S\)\[ℓ\(hSϕ,zi\)\]−𝔼ϕ′∼𝒜⁡\(S∖i\)\[ℓ\(hS∖iϕ′,zi\)\]\|\.\\displaystyle=\\Big\|,\\mathbb\{E\}\_\{\\phi\\sim\\mathcal\{A\}\(S\)\}\[\\ell\(h^\{\\phi\}\_\{S\},z\_\{i\}\)\]\\ \-\\mathbb\{E\}\_\{\\phi^\{\\prime\}\\sim\\mathcal\{A\}\(S^\{\\setminus i\}\)\}\[\\ell\(h^\{\\phi^\{\\prime\}\}\_\{S^\{\\setminus i\}\},z\_\{i\}\)\]\\Big\|\.A large value means the model only fitsziz\_\{i\}because it was trained onziz\_\{i\}\.

## 3Empirical Results

We begin with an empirical investigation of curvature as a proxy for memorization in diffusion models\. To do so, we train DDPMs to sample two datasets: \(1\)64×6464\\times 64RGB ImageNet\-1K\[[14](https://arxiv.org/html/2610.19042#bib.bib36),[37](https://arxiv.org/html/2610.19042#bib.bib37)\], and \(2\)32×3232\\times 32greyscale CelebA images, also used in\[[8](https://arxiv.org/html/2610.19042#bib.bib19)\]\)\. To further isolate the memorization phenomenon, we train six distinct DDPMs on datasets of sizen=2kn=2^\{k\}for7≤k≤127\\leq k\\leq 12for ImageNet\-1k and two distinct DDPMs on datasets of sizen=128,10240n=128,10240for CelebA, noting that for CelebA,n=128n=128corresponds to a memorizing model whilen=10240n=10240corresponds to a generalizing model\. Across the training timeτ\\tau, we track the mean curvature over the training samples and validation samples:

Avg\. train curvature=𝔼x∼train​\[‖∇x2ℓ​\(ϕ⁡\(τ\),x\)‖F2\],\\displaystyle=\\mathbb\{E\}\_\{x\\sim\\text\{train\}\}\[\\\|\\nabla^\{2\}\_\{x\}\\ell\(\\phi\(\\tau\),x\)\\\|^\{2\}\_\{F\}\],\(11\)Avg\. val curvature=𝔼x∼val\.​\[‖∇x2ℓ​\(ϕ⁡\(τ\),x\)‖F2\]\.\\displaystyle=\\mathbb\{E\}\_\{x\\sim\\text\{val\.\}\}\[\\\|\\nabla^\{2\}\_\{x\}\\ell\(\\phi\(\\tau\),x\)\\\|^\{2\}\_\{F\}\]\.\(12\)We also generate10,00010,000samples to compute thefm​e​mf\_\{mem\}score\. Following\[[20](https://arxiv.org/html/2610.19042#bib.bib24),[50](https://arxiv.org/html/2610.19042#bib.bib22),[9](https://arxiv.org/html/2610.19042#bib.bib43)\], a generated samplexxis considered memorized if

\[‖x−aμ1‖2‖x−aμ2‖2<k\],\\left\[\\frac\{\\\|x\-a^\{\\mu\_\{1\}\}\\\|\_\{2\}\}\{\\\|x\-a^\{\\mu\_\{2\}\}\\\|\_\{2\}\}<k\\right\],\(13\)whereaμ1a^\{\\mu\_\{1\}\}andaμ2a^\{\\mu\_\{2\}\}are the nearest and second nearest neighbors in the training set toxx\. We say that a training samplexix\_\{i\}is*copied*if it appears in the generated set\. Its*copy count*F⁡\(i\)F\(i\)refers to the frequency of occurrences ofxix\_\{i\}in the generations\. In Appendix[C](https://arxiv.org/html/2610.19042#A3)we visualize the samples generated from our various models\. Here we assess the input loss curvature of our models during training\.

Evolution of input curvature during training\.In Figure[1](https://arxiv.org/html/2610.19042#S3.F1), we report our experiments on CelebA\. We find that:

1. 1\.Mean training curvature rises over training together withfmem​\(τ\)f\_\{\\mathrm\{mem\}\}\(\\tau\)\(panel \(C\)\)\. The rise alone does not mark the onset: training curvature is already increasing whilefmemf\_\{\\mathrm\{mem\}\}is still flat\.
2. 2\.The onset shows up in the validation curvature\. It stays with the training curvature and then diverges, in both panels \(A\) and \(B\), at the step wherefmem​\(τ\)f\_\{\\mathrm\{mem\}\}\(\\tau\)starts to accelerate\. The same split appears for the generalizing model \(panel \(D\)\), where memorization is delayed untilτ≈105\\tau\\approx 10^\{5\}\. This motivates defining the relative change in avg\. train and validation curvatures as a stopping criterion: Δ​Curv​\(τ\)\\displaystyle\\Delta\\text\{Curv\}\(\\tau\)\(14\):=\|Avg\. train curvature−Avg\. val\. curvature\|\|Avg\. train curvature\|\\displaystyle:=\\frac\{\|\\text\{Avg\. train curvature\}\-\\text\{Avg\. val\. curvature\}\|\}\{\|\\text\{Avg\. train curvature\}\|\} we may consider stopping training whenΔ​Curv​\(τ\)≥ε\\Delta\\text\{Curv\}\(\\tau\)\\geq\\varepsilon, whereε\\varepsilonmodulates the threshold required to limit memorization\. Notably this estimator*does not need*to generate any samples during training, saving significant computational cost\.
3. 3\.Once memorization is heavy, every training image is reproduced but at different frequencies\. However, the curvature of an individual image does not predict its copy count \(Panel \(F\)\)\.

In Figure[2](https://arxiv.org/html/2610.19042#S3.F2), we repeat these experiments for ImageNet, observing patterns consistent with CelebA\. In Panel \(F\), we confirm thatfm​e​m​\(τ\)f\_\{mem\}\(\\tau\)rises and reaches 100% byτ≈106\\tau\\approx 10^\{6\}for every model exceptn=4096n=4096\. The validation loss, a zeroth\-order quantity, falls throughout training for everynn\(Panel \(E\)\), so the held\-out loss alone does not detect memorization\. However, in panel \(A\), we find that input curvature for training samples rises withτ\\tau, but the average validation curvature*reaches a local minimum*and then rises once the model begins to memorize heavily\. In Panel \(D\), we show thatΔ​Curv​\(τ\)\\Delta\\text\{Curv\}\(\\tau\)reaches a local maximum at aroundτ=105\\tau=10^\{5\}, where most models are memorizing\.

Appendices[D](https://arxiv.org/html/2610.19042#A4)and[E](https://arxiv.org/html/2610.19042#A5)provide additional numerical experiments\.

![Refer to caption](https://arxiv.org/html/2610.19042v1/celebA.png)Figure 1:\(A\) Train vs\. validation input curvature for CelebA, n=128, \(B\) Val\. curvature starts to diverge from train curvature whenfm​e​m​\(τ\)f\_\{mem\}\(\\tau\)starts accelerating, \(C\)fm​e​m​\(τ\)f\_\{mem\}\(\\tau\)values forn=128,10240n=128,10240on CelebA, \(D\) and \(E\) Same experiment for CelebA, n=10240\. Note that this model undergoes generalization, and memorization sets in much later atτ≈105\\tau\\approx 10^\{5\}, \(F\) Curvature of an individual sample is not a good predictor of its copy count\.![Refer to caption](https://arxiv.org/html/2610.19042v1/imagenet.png)Figure 2:\(A\) Input \([11](https://arxiv.org/html/2610.19042#S3.E11)\) curvatures consistently rise but validation \([12](https://arxiv.org/html/2610.19042#S3.E12)\) curvatures reach a local minimum at approximately peak memorization for DDPMs on different data sizes, \(B\) Sizes of gradients of validation loss also display local minima, \(C\) Input curvature on training data tracks the largest frequency of memorized samples, \(D\) Relative error between train and validation curvatures exhibits a local maximum, leading to an early stopping criterion, \(E\) Memorization cannot be detected just based on validation loss, \(F\) Memorization rises as models train further\.
## 4Sample\-Level Memorization

Our results in Section[3](https://arxiv.org/html/2610.19042#S3)show that while curvature, aggregated over either training or held\-out sets, is a good indicator of model\-level memorization, it is not a good signal for whether a given sample will be memorized more than another one\. To address this gap, we introduce a counterfactual memorization, analogous to what was defined in\[[52](https://arxiv.org/html/2610.19042#bib.bib28)\]\. Before we begin, we establish the relevant notation\.

### 4\.1Notation

The notation will be borrowed from\[[33](https://arxiv.org/html/2610.19042#bib.bib2)\]\. Let𝒳\\mathcal\{X\}be the space of inputs,𝒴\\mathcal\{Y\}the space of outputs, andhha modelh:𝒳→𝒴h:\\mathcal\{X\}\\to\\mathcal\{Y\}\. The joint distribution in the input\-output pairz=\(x,y\)z=\(x,y\)will be denotedpZp\_\{Z\}and itsXXandYYmarginals will be the input and output distributions\. The model will be drawn from some parametric function class𝒞\\mathcal\{C\}parameterized byΦ⊆ℝP\\Phi\\subseteq\\mathbb\{R\}^\{P\}wherePPrepresents the total number of parameters\. To mark its dependence on parameters, the model will therefore be denoted ashϕh^\{\\phi\}\. In practice, we draw a finite number of samplesS=\{zi\}i=1n=\{\(xi,yi\)\}i=1nS=\\\{z\_\{i\}\\\}\_\{i=1\}^\{n\}=\\\{\(x\_\{i\},y\_\{i\}\)\\\}\_\{i=1\}^\{n\}from𝒮n=\(𝒳×𝒴\)n\\mathcal\{S\}^\{n\}=\(\\mathcal\{X\}\\times\\mathcal\{Y\}\)^\{n\}and*train*to fit a lossc:𝒴×𝒴′c:\\mathcal\{Y\}\\times\\mathcal\{Y\}^\{\\prime\}on the data\. We assume thenn\-data points are sampled i\.i\.d; thereforepZp\_\{Z\}induces a distribution \(via independence\) on𝒮n\\mathcal\{S\}^\{n\}, termedpZnp^\{n\}\_\{Z\}\. GivenS∼pZnS\\sim p^\{n\}\_\{Z\}, the training process is quantified by some algorithm𝒜:𝒮n→ℳ\\mathcal\{A\}:\\mathcal\{S\}^\{n\}\\to\\mathcal\{M\}, whereℳ⊆𝒫2​\(Φ\)\\mathcal\{M\}\\subseteq\\mathcal\{P\}\_\{2\}\(\\Phi\)and𝒫2​\(Φ\)\\mathcal\{P\}\_\{2\}\(\\Phi\)is the class of square integrable random m onΦ\\Phi\. A model trained with dataSSwith parametersϕ\\phiis thus denotedhSϕh^\{\\phi\}\_\{S\}\. We may define the loss over the modelhSϕh^\{\\phi\}\_\{S\}andzzas

ℓ⁡\(hSϕ,z\)=c⁡\(hSϕ​\(x\),y\)\.\\displaystyle\\ell\(h^\{\\phi\}\_\{S\},z\)=c\(h^\{\\phi\}\_\{S\}\(x\),y\)\.\(15\)Wherever necessary we will abuse notation and identify the functionϕ,z→ℓ⁡\(hSϕ,z\)\\phi,z\\to\\ell\(h^\{\\phi\}\_\{S\},z\)withℓ\\ell\. Given a datasetSSwe will also define the losses onSSas follows:

ℓ⁡\(ϕ,S\):=1\|S\|​∑i=1\|S\|ℓ⁡\(hSϕ,zi\)\.\\displaystyle\\ell\(\\phi,S\):=\\frac\{1\}\{\|S\|\}\\sum\_\{i=1\}^\{\|S\|\}\\ell\(h^\{\\phi\}\_\{S\},z\_\{i\}\)\.\(16\)We also setℓ∗​\(S\):=infϕ∈Φℓ⁡\(ϕ,S\)\\ell^\{\*\}\(S\):=\\inf\_\{\\phi\\in\\Phi\}\\ell\(\\phi,S\)\.

### 4\.2The Counterfactual Log\-Odds of Sampling

We may simplify the notation above in the specific case of diffusion models\. In particular, the training objective \([7](https://arxiv.org/html/2610.19042#S2.E7)\) is*unsupervised*, and thuszi=xiz\_\{i\}=x\_\{i\}\. Moreover, we assumehϕh^\{\\phi\}is produced via score matching/denoising and that it is then used to drive the SDE \([8](https://arxiv.org/html/2610.19042#S2.E8)\)\. We now define the counterfactual memorization:

###### Definition 4\.1\.

Letzi∈Sz\_\{i\}\\in Sbe a training example andϕ∼𝒜⁡\(S\),ϕ′∼𝒜⁡\(S∖i\)\\phi\\sim\\mathcal\{A\}\(S\),\\phi^\{\\prime\}\\sim\\mathcal\{A\}\(S^\{\\setminus i\}\)be parameter distributions after optimization on the datasetsS,S∖iS,S^\{\\setminus i\}\. Letpϕ​\(⋅,T\),pϕ′​\(⋅,T\)p\_\{\\phi\}\(\\cdot,T\),p\_\{\\phi^\{\\prime\}\}\(\\cdot,T\)be the time marginals of the distributions produced by driving the plug\-in SDE \([8](https://arxiv.org/html/2610.19042#S2.E8)\)\. Then the*sampling time memorization*ofziz\_\{i\}is given by:

m⁡\(zi,S\):=𝔼ϕ,ϕ′​\[log⁡pϕ​\(zi,T\)−log⁡pϕ′​\(zi,T\)\]\.\\displaystyle m\(z\_\{i\},S\):=\\mathbb\{E\}\_\{\\phi,\\phi^\{\\prime\}\}\[\\log p\_\{\\phi\}\(z\_\{i\},T\)\-\\log p\_\{\\phi^\{\\prime\}\}\(z\_\{i\},T\)\]\.\(17\)

Note thatm⁡\(zi,S\)m\(z\_\{i\},S\)represents the log\-odds of samplingziz\_\{i\}with and without its presence in the training data\. Largemmimplies that the model would be much less likely to sampleziz\_\{i\}if it wasn’t included in the dataset\. We also introduce a difference in the counterfactual*losses*, borrowed from\[[34](https://arxiv.org/html/2610.19042#bib.bib30),[33](https://arxiv.org/html/2610.19042#bib.bib2),[18](https://arxiv.org/html/2610.19042#bib.bib4),[17](https://arxiv.org/html/2610.19042#bib.bib3)\]:

M⁡\(zi,S\)\\displaystyle M\(z\_\{i\},S\)=𝔼ϕ∼𝒜⁡\(S\)​\[ℓ⁡\(hSϕ,zi\)\]\\displaystyle=\\mathbb\{E\}\_\{\\phi\\sim\\mathcal\{A\}\(S\)\}\[\\ell\(h^\{\\phi\}\_\{S\},z\_\{i\}\)\]\(18\)−𝔼ϕ′∼A⁡\(S∖i\)​\[ℓ⁡\(hS∖iϕ′,zi\)\]\.\\displaystyle\-\\mathbb\{E\}\_\{\\phi^\{\\prime\}\\sim A\(S^\{\\setminus i\}\)\}\[\\ell\(h^\{\\phi^\{\\prime\}\}\_\{S^\{\\setminus i\}\},z\_\{i\}\)\]\.\(19\)We now use the decomposition \([10](https://arxiv.org/html/2610.19042#S2.E10)\) to writemmin terms ofMM:

###### Proposition 4\.2\.

Letϕ,ϕ′\\phi,\\phi^\{\\prime\}be parameter distributions as defined in[4\.1](https://arxiv.org/html/2610.19042#S4.Thmtheorem1)andmm,MMbe defined as in \([17](https://arxiv.org/html/2610.19042#S4.E17)\) and \([19](https://arxiv.org/html/2610.19042#S4.E19)\)\. Fixxi∈Sx\_\{i\}\\in S\. Then:

−𝔼ϕ′​\[Gapϕ′​\(xi\)\]−M⁡\(xi,S\)≤m⁡\(xi,S\)≤\\displaystyle\-\\mathbb\{E\}\_\{\\phi^\{\\prime\}\}\[\\text\{Gap\}\_\{\\phi^\{\\prime\}\}\(x\_\{i\}\)\]\-M\(x\_\{i\},S\)\\leq m\(x\_\{i\},S\)\\leq\(20\)M⁡\(xi,S\)\+𝔼ϕ​\[Gapϕ​\(xi\)\]\.\\displaystyle M\(x\_\{i\},S\)\+\\mathbb\{E\}\_\{\\phi\}\[\\text\{Gap\}\_\{\\phi\}\(x\_\{i\}\)\]\.\(21\)

We now boundMMpurely in terms of the properties of the leave\-one\-out modelhϕ′h\_\{\\phi^\{\\prime\}\}\. We establish a number of mathematical preliminaries\.

### 4\.3Mathematical Preliminaries

For a pairzi∈Sz\_\{i\}\\in S,zj∈S∖iz\_\{j\}\\in S^\{\\setminus i\}we use four scalar quantities throughout:

di​j\\displaystyle d\_\{ij\}:=‖zi−zj‖2,γj:=‖∇xℓ​\(ϕ,zj\)‖2,\\displaystyle:=\\\|z\_\{i\}\-z\_\{j\}\\\|\_\{2\},\\gamma\_\{j\}:=\\\|\\nabla\_\{x\}\\ell\(\\phi,z\_\{j\}\)\\\|\_\{2\},\(22\)κi\\displaystyle\\kappa\_\{i\}:=𝔼ϕ′∼𝒜⁡\(S∖i\)​\[‖∇z2ℓ​\(ϕ′,zi\)‖F2\]\\displaystyle:=\\mathbb\{E\}\_\{\\phi^\{\\prime\}\\sim\\mathcal\{A\}\(S^\{\\setminus i\}\)\}\[\\\|\\nabla^\{2\}\_\{z\}\\ell\(\\phi^\{\\prime\},z\_\{i\}\)\\\|\_\{F\}^\{2\}\]\(23\)Ci​j\\displaystyle C\_\{ij\}:=\|𝔼ϕ∼𝒜⁡\(S\)​ℓ​\(ϕ,zi\)−𝔼ϕ′∼𝒜⁡\(S∖i\)​ℓ​\(ϕ′,zj\)\|\.\\displaystyle:=\\big\|\\mathbb\{E\}\_\{\\phi\\sim\\mathcal\{A\}\(S\)\}\\ell\(\\phi,z\_\{i\}\)\-\\mathbb\{E\}\_\{\\phi^\{\\prime\}\\sim\\mathcal\{A\}\(S^\{\\setminus i\}\)\}\\ell\(\\phi^\{\\prime\},z\_\{j\}\)\\big\|\.\(24\)
###### Assumption 1\.

\(ℓ\\ellhasρ\\rho\-Lipschitz hessians\) The Hessian of the lossℓ\\ellis Lipschitz in thezz\-variable, i\.e for everyϕ∈Φ\\phi\\in\\Phi, there existsρ\\rho\(possibly dependent onϕ\\phi\) such that:

‖∇z2​ℓ​\(ϕ,z′\)−∇z2​ℓ​\(ϕ,z\)‖2≤ρ​‖z′−z‖2,\\displaystyle\\\|\\nabla^\{2\}\_\{z\}\\ell\(\\phi,z^\{\\prime\}\)\-\\nabla^\{2\}\_\{z\}\\ell\(\\phi,z\)\\\|\_\{2\}\\leq\\rho\\\|z^\{\\prime\}\-z\\\|\_\{2\},\(25\)for everyz,z′∈𝒮z,z^\{\\prime\}\\in\\mathcal\{S\}\.

###### Assumption 2\.

\(Loss behaves like MSE\) We assume that the cost functionc⁡\(x,y\)c\(x,y\)satisfies

‖∇yc​\(x,y\)‖22≤2​β​c​\(x,y\)\.\\displaystyle\\\|\\nabla\_\{y\}c\(x,y\)\\\|^\{2\}\_\{2\}\\leq 2\\beta c\(x,y\)\.\(26\)Note that anyccthat isβ\\beta\-Lipschitz in theyy\-variable satisfies \([26](https://arxiv.org/html/2610.19042#S4.E26)\)\[[46](https://arxiv.org/html/2610.19042#bib.bib42)\]\. For instance, the MSEc⁡\(x,y\)=‖x−y‖22c\(x,y\)=\\\|x\-y\\\|\_\{2\}^\{2\}, orc⁡\(x,y\)=‖h⁡\(x\)−y‖22c\(x,y\)=\\\|h\(x\)\-y\\\|\_\{2\}^\{2\}whereh⁡\(x\)h\(x\)is itself Lipschitz satisfy \([26](https://arxiv.org/html/2610.19042#S4.E26)\)\.

###### Assumption 3\.

\(The underlying function is neural network\-like\) We assume that allhϕ=g⁡\(ϕ1​x,ϕ2\)h^\{\\phi\}=g\(\\phi\_\{1\}x,\\phi\_\{2\}\)whereϕ=\(ϕ1,ϕ2\)\\phi=\(\\phi\_\{1\},\\phi\_\{2\}\)\. Most neural network architectures satisfy this assumption\.

Note that the term in \([28](https://arxiv.org/html/2610.19042#S4.E28)\) is*not quite*M⁡\(zi,S\)M\(z\_\{i\},S\)because the second term in the absolute value involveszjz\_\{j\}\(which is part ofS∖iS^\{\\setminus i\}\) and notziz\_\{i\}\.

###### Assumption 4\.

We assume that𝒜\\mathcal\{A\}is a first order optimizer in the parameters, i\.e there existεgrad\\varepsilon\_\{\\mathrm\{grad\}\}andεloss\\varepsilon\_\{\\mathrm\{loss\}\}such that

𝔼ϕ∼𝒜⁡\(S\)​\[‖∇ϕℓ​\(ϕ,zj\)‖2\]≤εgrad,𝔼ϕ∼𝒜⁡\(S\)​\[l⁡\(ϕ,zj\)\]≤εloss,\\displaystyle\\mathbb\{E\}\_\{\\phi\\sim\\mathcal\{A\}\(S\)\}\[\\\|\\nabla\_\{\\phi\}\\ell\(\\phi,z\_\{j\}\)\\\|\_\{2\}\]\\leq\\varepsilon\_\{\\mathrm\{grad\}\},\\mathbb\{E\}\_\{\\phi\\sim\\mathcal\{A\}\(S\)\}\[l\(\\phi,z\_\{j\}\)\]\\leq\\varepsilon\_\{\\mathrm\{loss\}\},\(27\)for everyzj∈Sz\_\{j\}\\in S\. Note that condition \([27](https://arxiv.org/html/2610.19042#S4.E27)\) is typically met by most deterministic and stochastic optimizers like SGD, Adam, or even full batch gradient descent\.

###### Assumption 5\.

We assume that under𝒜⁡\(S\)\\mathcal\{A\}\(S\), the weightsϕ\\phiare almost surely bounded byBB\.

###### Assumption 6\.

There exists a constantCCindependent ofxi,xjx\_\{i\},x\_\{j\}in the leading order \(i\.eC=O⁡\(1\)C=O\(1\)inxi,xjx\_\{i\},x\_\{j\}\) such that with high probability inpZnp^\{n\}\_\{Z\}, for anyzi∈S,zj∈S∖iz\_\{i\}\\in S,z\_\{j\}\\in S^\{\\setminus i\}the following difference is uniformly bounded byCC:

\|𝔼ϕ∼𝒜⁡\(S\)\[ℓ\(hSϕ,zi\)\]−𝔼ϕ′∼𝒜⁡\(S∖i\)\[ℓ\(hS∖iϕ′,zj\)\]\|≤C\.\\displaystyle\\lvert\\mathbb\{E\}\_\{\\phi\\sim\\mathcal\{A\}\(S\)\}\[\\ell\(h^\{\\phi\}\_\{S\},z\_\{i\}\)\]\-\\mathbb\{E\}\_\{\\phi^\{\\prime\}\\sim\\mathcal\{A\}\(S^\{\\setminus i\}\)\}\[\\ell\(h^\{\\phi^\{\\prime\}\}\_\{S^\{\\setminus i\}\},z\_\{j\}\)\]\|\\leq C\.\(28\)

Assumption[4](https://arxiv.org/html/2610.19042#Thmassumption4)models𝒜⁡\(S\)\\mathcal\{A\}\(S\)as a first order optimizer that leads to bounded weights upon optimization\. In the context of neural networks the weight\-boundedness under standard optimization algorithms has been observed in several contexts, from deep linear models\[[38](https://arxiv.org/html/2610.19042#bib.bib7)\], feedforward ReLU \(or leaky ReLU\) networks\[[15](https://arxiv.org/html/2610.19042#bib.bib9)\]to transformers\[[4](https://arxiv.org/html/2610.19042#bib.bib8)\]\.

### 4\.4Training\-Time Memorization is Controlled by Leave\-One\-Out Curvature

###### Theorem 4\.3\.

Under Assumption[1](https://arxiv.org/html/2610.19042#Thmassumption1), for everyzi∈Sz\_\{i\}\\in Sand everyzj∈S∖iz\_\{j\}\\in S^\{\\setminus i\},

\|M⁡\(zi,S\)\|≤Ci​j\+γj​di​j\+32​κi​di​j2\+2​ρ3​di​j3\.\|M\(z\_\{i\};S\)\|\\;\\leq\\;C\_\{ij\}\+\\gamma\_\{j\}\\,d\_\{ij\}\+\\tfrac\{3\}\{2\}\\,\\kappa\_\{i\}\\,d\_\{ij\}^\{2\}\+\\tfrac\{2\\rho\}\{3\}\\,d\_\{ij\}^\{3\}\.\(29\)If Assumptions[2](https://arxiv.org/html/2610.19042#Thmassumption2)–[5](https://arxiv.org/html/2610.19042#Thmassumption5)hold, then

γj≤B​εgrad‖zj‖2\+εloss,\\gamma\_\{j\}\\;\\leq\\;\\frac\{B\\,\\varepsilon\_\{\\mathrm\{grad\}\}\}\{\\\|z\_\{j\}\\\|\_\{2\}\}\+\\varepsilon\_\{\\mathrm\{loss\}\},\(30\)and if Assumption[6](https://arxiv.org/html/2610.19042#Thmassumption6)holds as well, then withdNN​\(i\):=minj≠i⁡di​jd\_\{\\mathrm\{NN\}\}\(i\):=\\min\_\{j\\neq i\}d\_\{ij\},

\|M⁡\(zi,S\)\|≤\\displaystyle\|M\(z\_\{i\};S\)\|\\;\\leq\\;C\+dNN​\(i\)2​\(32​κi\+2​ρ3​dNN​\(i\)\)\\displaystyle C\+d\_\{\\mathrm\{NN\}\}\(i\)^\{2\}\\left\(\\tfrac\{3\}\{2\}\\,\\kappa\_\{i\}\\,\+\\tfrac\{2\\rho\}\{3\}\\,d\_\{\\mathrm\{NN\}\}\(i\)\\right\)\+\(B​εgradminj⁡‖zj‖\+εloss\)​dNN​\(i\)\.\\displaystyle\+\\Big\(\\frac\{B\\varepsilon\_\{\\mathrm\{grad\}\}\}\{\\min\_\{j\}\\\|z\_\{j\}\\\|\}\+\\varepsilon\_\{\\mathrm\{loss\}\}\\Big\)d\_\{\\mathrm\{NN\}\}\(i\)\.\(31\)

###### Corollary 4\.4\.

Let Assumptions[1](https://arxiv.org/html/2610.19042#Thmassumption1)–[6](https://arxiv.org/html/2610.19042#Thmassumption6)hold and additionally supposeεgrad,εgrad≪dNN​\(i\)\\varepsilon\_\{\\mathrm\{grad\}\},\\varepsilon\_\{\\mathrm\{grad\}\}\\ll d\_\{\\mathrm\{NN\}\}\(i\)\. SetC⁡\(i\):=\(C\+𝔼ϕ′​\[Gapϕ′​\(xi\)\]\)C\(i\):=\(C\+\\mathbb\{E\}\_\{\\phi^\{\\prime\}\}\[\\text\{Gap\}\_\{\\phi^\{\\prime\}\}\(x\_\{i\}\)\]\)\. Then:

−\(C⁡\(i\)\+o⁡\(1\)\)−dNN​\(i\)2​\(32​κi\+2​ρ3​dNN​\(i\)\)\\displaystyle\-\(C\(i\)\+o\(1\)\)\-d\_\{\\mathrm\{NN\}\}\(i\)^\{2\}\\left\(\\tfrac\{3\}\{2\}\\,\\kappa\_\{i\}\\,\+\\tfrac\{2\\rho\}\{3\}\\,d\_\{\\mathrm\{NN\}\}\(i\)\\right\)\(32\)≤m⁡\(xi,S\)≤dNN​\(i\)2​\(32​κi\+2​ρ3​dNN​\(i\)\)\+C⁡\(i\)\+o⁡\(1\)\.\\displaystyle\\leq m\(x\_\{i\},S\)\\leq d\_\{\\mathrm\{NN\}\}\(i\)^\{2\}\\left\(\\tfrac\{3\}\{2\}\\,\\kappa\_\{i\}\\,\+\\tfrac\{2\\rho\}\{3\}\\,d\_\{\\mathrm\{NN\}\}\(i\)\\right\)\+C\(i\)\+o\(1\)\.\(33\)

The proofs are provided in Appendix[A](https://arxiv.org/html/2610.19042#A1)\. We make the following comments regarding Theorem[4\.3](https://arxiv.org/html/2610.19042#S4.Thmtheorem3):

1. 1\.Under Assumption[1](https://arxiv.org/html/2610.19042#Thmassumption1)alone, \([29](https://arxiv.org/html/2610.19042#S4.E29)\) is a cubic expansion about an arbitrary second training pointzjz\_\{j\}\. Each coefficient can be large and can depend on the query pointziz\_\{i\}: the training\-loss gapCi​jC\_\{ij\}, the leave\-one\-out input gradientγj\\gamma\_\{j\}, the leave\-one\-out curvatureκi\\kappa\_\{i\}, and the Hessian Lipschitz constantρ\\rho\.
2. 2\.Assumptions[2](https://arxiv.org/html/2610.19042#Thmassumption2)–[5](https://arxiv.org/html/2610.19042#Thmassumption5)put a neural network and a first\-order method into the bound\. When the algorithm overfits,εgrad\\varepsilon\_\{\\mathrm\{grad\}\}andεloss\\varepsilon\_\{\\mathrm\{loss\}\}can be considered to be comparably small againstκi​di​j\\kappa\_\{i\}\\,d\_\{ij\}, the curvature of the*leave\-one\-out*lossℓ⁡\(ϕ′,zi\)\\ell\(\\phi^\{\\prime\},z\_\{i\}\)\. Curvature thus bounds counterfactual memorization at the leading order, during overfitting\.
3. 3\.Assumption[6](https://arxiv.org/html/2610.19042#Thmassumption6)asks for a uniformCCthat isO⁡\(1\)O\(1\)inxx\. We can undertake a splitting argument to show that thisC=O⁡\(n\)C=O\(n\)–this is comparable to an analogous bound obtained in\[[34](https://arxiv.org/html/2610.19042#bib.bib30)\]\(Theorem 5\.1\) which is alsoO⁡\(n\)O\(n\)\.
4. 4\.\[[33](https://arxiv.org/html/2610.19042#bib.bib2)\]obtainC=O⁡\(n\)C=O\(n\)usingβ\\beta\-uniform stability and remove the first order termsγi\\gamma\_\{i\}by comparing𝒜⁡\(S\)\\mathcal\{A\}\(S\)with𝒜⁡\(S′\)\\mathcal\{A\}\(S^\{\\prime\}\)onυ\\upsilon\-adjacent datasets, where any two points lie at most a distance ofυ\\upsilonapart, for smallυ\\upsilon\.\[[34](https://arxiv.org/html/2610.19042#bib.bib30)\]remark that a training set must be augmented*post\-hoc*to make itυ\\upsilon\-adjacent\. Here Assumption[6](https://arxiv.org/html/2610.19042#Thmassumption6)replaces the uniform stability assumption\. Moreover, we do not require such a condition on the data–instead we restrict the model class to be structured like a neural network in order to use its inductive biases to make the first order terms negligible\. Overall, our bound does not require uniform stability,υ\\upsilon\-adjacency, or bounded loss as is required by Theorem 5\.1 in\[[34](https://arxiv.org/html/2610.19042#bib.bib30)\]–however, we make more structural but nonetheless realistic assumptions on the model class and the algorithms at hand\.

### 4\.5Application to Diffusion Models

We can make further specifications with additional assumptions for the denoising loss \([7](https://arxiv.org/html/2610.19042#S2.E7)\) and the fact that typically, the hypothesis classhϕh^\{\\phi\}contains linear functions\.

###### Assumption 7\.

We assume that for anyM∈ℝd×mM\\in\\mathbb\{R\}^\{d\\times m\}andb∈ℝmb\\in\\mathbb\{R\}^\{m\}there exists someϕ\\phisuch thathϕ=M​x\+bh^\{\\phi\}=Mx\+b\.

In objective \([7](https://arxiv.org/html/2610.19042#S2.E7)\) we train a denoiserDϕ:ℝd×ℝ→ℝdD\_\{\\phi\}:\\mathbb\{R\}^\{d\}\\times\\mathbb\{R\}\\to\\mathbb\{R\}^\{d\}\. Thus we require that the underlying function class contain at least all linear denoisers\. This assumption is satisfied by ReLU networks or ReLU U\-Nets\. Next we state the PL\-inequality assumption used for simplifying the analysis of first\-order methods in non\-convex optimization\. Note that neural network loss landscapes have been known to satisfy the local\-PL inequality below for the parameter regions of most commonly used neural networks\[[26](https://arxiv.org/html/2610.19042#bib.bib52)\]\.

###### Assumption 8\.

Letϕ\\phibe the parameters and∇ϕℓ​\(ϕ,S\)\\nabla\_\{\\phi\}\\ell\(\\phi,S\)the parameter gradient of the empirical loss\. We assume that there existsμ\>0\\mu\>0such that almost surely under𝒜⁡\(S\)\\mathcal\{A\}\(S\),

\(ℓ⁡\(ϕ,S\)−ℓ⁡\(S∗\)\)≤12​μ​‖∇ϕℓ​\(ϕ,S\)‖22\\displaystyle\(\\ell\(\\phi,S\)\-\\ell\(S^\{\*\}\)\)\\leq\\frac\{1\}\{2\\mu\}\\\|\\nabla\_\{\\phi\}\\ell\(\\phi,S\)\\\|^\{2\}\_\{2\}\(34\)

###### Proposition 4\.5\.

Under assumptions[1](https://arxiv.org/html/2610.19042#Thmassumption1)–[8](https://arxiv.org/html/2610.19042#Thmassumption8), ifℓ\\ellis given by \([7](https://arxiv.org/html/2610.19042#S2.E7)\), there existsA⁡\(T\)A\(T\)depending onf,g,Tf,g,Tsuch that the constantCCin assumption[6](https://arxiv.org/html/2610.19042#Thmassumption6)is of the form:

C≤nμ​εgrad\+n​A​\(T\)\.\\displaystyle C\\leq\\frac\{n\}\{\\mu\}\\varepsilon\_\{\\mathrm\{grad\}\}\+nA\(T\)\.\(35\)

We provide a proof in Appendix[A](https://arxiv.org/html/2610.19042#A1)\. Note that in this setting the constantCCis comparable to the constantc1c\_\{1\}given in Eq\. 8 in\[[34](https://arxiv.org/html/2610.19042#bib.bib30)\]\.

### 4\.6Held\-Out Curvature During Training

Finally, we validate our bound on the DDPMs on Imagenet, to check whether held\-out curvature can stand as a proxy for counterfactual memorization\. In order to track it during training, we assume bounded ELBO gaps in \([10](https://arxiv.org/html/2610.19042#S2.E10)\) and track the counterfactual loss difference \([19](https://arxiv.org/html/2610.19042#S4.E19)\) instead of the log odds \([17](https://arxiv.org/html/2610.19042#S4.E17)\)\. Since training a diffusion model onS∖iS^\{\\setminus i\}for everyiiis prohibitively expensive, we remove the top 10 most memorized examples for the DDPMs corresponding ton=128n=128andn=1024n=1024\. We then estimateM⁡\(xi,S\)M\(x\_\{i\},S\)for these ten points, with the loss on theS′:=S∖\{top 10 memorized points\}S^\{\\prime\}:=S\\setminus\\\{\\text\{top 10 memorized points\}\\\}with parametersϕ′\\phi^\{\\prime\}evaluated onxix\_\{i\}as an estimator forℓ⁡\(hS∖iϕ′,xi\)\\ell\(h^\{\\phi^\{\\prime\}\}\_\{S^\{\\setminus i\}\},x\_\{i\}\)\. We present our results in Figures[3](https://arxiv.org/html/2610.19042#S4.F3)and[4](https://arxiv.org/html/2610.19042#S4.F4)where we track:

Δ​ℓ​\(τ\):=ℓ⁡\(hSϕ,zi\)−ℓ⁡\(hS′ϕ′,zi\)\.\\displaystyle\\Delta\\ell\(\\tau\):=\\ell\(h^\{\\phi\}\_\{S\},z\_\{i\}\)\-\\ell\(h^\{\\phi^\{\\prime\}\}\_\{S^\{\\prime\}\},z\_\{i\}\)\.\(36\)Note thatΔ​ℓ\\Delta\\ellis the signed version of \([19](https://arxiv.org/html/2610.19042#S4.E19)\)\. We observe the following phenomenon withΔ​ℓ\\Delta\\ell: as training progresses,Δ​ℓ\\Delta\\elldrops sharply at the initial phase of training, showing that the full model attempts to fit at the seen examplexix\_\{i\}\. Then the loss stabilizes as both models enter their respective memorization phases\. As thefm​e​mf\_\{mem\}approaches100%100\\%Δ⁡\(τ\)\\Delta\(\\tau\)starts dropping quickly again\. Throughout this whole process, however, the curvature of the held out models at the held out samples roughly corresponds to the drops and stability ofΔ⁡\(τ\)\\Delta\(\\tau\), thus suggesting that our bound \([29](https://arxiv.org/html/2610.19042#S4.E29)\) is a meaningful proxy for the counterfactual loss\. At the very tail of training, the curvature for the held\-out models*rises*, reflecting the local minimum phenomenon seen in Figures[1](https://arxiv.org/html/2610.19042#S3.F1)and[2](https://arxiv.org/html/2610.19042#S3.F2)\. However,Δ⁡\(τ\)\\Delta\(\\tau\)keeps falling, suggested that the held out point’s influence onΔ⁡\(τ\)\\Delta\(\\tau\)via curvature breaks down in the highly overfitted regime\.

![Refer to caption](https://arxiv.org/html/2610.19042v1/figures/loss_delta_vs_curvature_over_training_ddpm_128.png)Figure 3:Loss difference and curvature for Imagenet \(n=128\) DDPM noise prediction \- leave\-10\-out training: \(left\) difference between per sample loss of full model and leave\-10\-out model vs\. training step for each of the 10 left\-out training samples, \(right\) per sample curvature of the leave\-10\-out model vs\. training step for each of the 10 left\-out training samples\.![Refer to caption](https://arxiv.org/html/2610.19042v1/figures/loss_delta_vs_curvature_over_training_ddpm_1024.png)Figure 4:Loss difference and curvature for Imagenet \(n=1024\) DDPM noise prediction \- leave\-10\-out\-training: \(left\) difference between per sample loss of full model and leave\-10\-out model vs\. training step for each of the 10 left\-out training samples, \(right\) per sample curvature of the leave\-10\-out model vs\. training step for each of the 10 left\-out training samples\.

## 5Conclusion

In this paper, we introduced the input\-loss curvature as a sampling\-free proxy forfmemf\_\{\\mathrm\{mem\}\}\. Averaged over the training set, it tracksfmemf\_\{\\mathrm\{mem\}\}on ImageNet and on CelebA as training proceeds, albeit it is not sufficient to detect the onset or prevalence of memorization\. Instead, we find that the step at which the validation curvature separates from the training curvature marks the transition into a memorizing model\. However, over individual points, the input loss curvature cannot reflect the copy count of individual samples\. For that question, we use a counterfactual definition, the change in loss and in log\-likelihood atxxwhenxxis removed from the training set, and we prove that this quantity is influenced, to leading order in the displacement ofxx, by the input\-loss curvature of the leave\-one\-out model\.

We enumerate the limitations of our findings below:

1. 1\.We provide narrow evidence ranging over only two datasets, ImageNet and CelebA\. Moreover, a comparison between the various training losses used to train diffusion models merits further investigation that is not performed at length here\.
2. 2\.We propose using the local maximum of the relative train–validation curvature error as a stopping criterion\. However, our experiments show that this local maximum indicates a period of already high memorization\. An improved statistic would be needed to locate the onset of memorization\. For small datasets, Figure[2](https://arxiv.org/html/2610.19042#S3.F2)shows that the norm of the loss gradient averaged over the validation data might prove useful, but its role and behaviour for larger datasets is not well\-understood\.
3. 3\.The bound is a plausibility argument and depends on a list of assumptions on the loss, the model, and the optimizer\. It is also unsettled whether the curvature term dominates the ELBO gap atxx, which is the comparison that decides whether curvature is the leading contribution to \([17](https://arxiv.org/html/2610.19042#S4.E17)\)\.

## Acknowledgment

This work was supported in part by NSF DMS 2502561\.

## References

- \[1\]J\. Abramson, J\. Adler, J\. Dunger, R\. Evans, T\. Green, A\. Pritzel, O\. Ronneberger, L\. Willmore, A\. J\. Ballard, J\. Bambrick,et al\.\(2024\)Accurate structure prediction of biomolecular interactions with alphafold 3\.Nature630\(8016\),pp\. 493–500\.Cited by:[§1](https://arxiv.org/html/2610.19042#S1.p1.1)\.
- \[2\]B\. Achilli, E\. Ventura, G\. Silvestri, B\. Pham, G\. Raya, D\. Krotov, C\. Lucibello, and L\. Ambrogioni\(2024\)Losing dimensions: geometric memorization in generative diffusion\.arXiv preprint arXiv:2410\.08727\.Cited by:[item 1](https://arxiv.org/html/2610.19042#S1.I1.i1.p1.1)\.
- \[3\]B\. D\. O\. Anderson\(1982\)Reverse\-time diffusion equation models\.Stochastic Processes and their Applications12\(3\),pp\. 313–326\.Cited by:[§2\.1](https://arxiv.org/html/2610.19042#S2.SS1.p1.3)\.
- \[4\]M\. Aubry, H\. Meng, A\. Sugolov, and V\. Papyan\(2024\)Transformer block coupling and its correlation with generalization in LLMs\.arXiv preprint arXiv:2407\.07810\.Cited by:[§4\.3](https://arxiv.org/html/2610.19042#S4.SS3.p3.1)\.
- \[5\]R\. Baptista, A\. Dasgupta, N\. B\. Kovachki, A\. Oberai, and A\. M\. Stuart\(2025\)Memorization and regularization in generative diffusion models\.arXiv preprint arXiv:2501\.15785\.Cited by:[item 1](https://arxiv.org/html/2610.19042#S1.I1.i1.p1.1)\.
- \[6\]N\. Berglund\(2013\)Kramers’ law: validity, derivations and generalisations\.Markov Processes and Related Fields19\(3\),pp\. 459–490\.Cited by:[§1](https://arxiv.org/html/2610.19042#S1.p2.2)\.
- \[7\]G\. Biroli, T\. Bonnaire, V\. de Bortoli, and M\. Mézard\(2024\)Dynamical regimes of diffusion models\.Nature Communications15,pp\. 9957\.Cited by:[item 1](https://arxiv.org/html/2610.19042#S1.I1.i1.p1.1)\.
- \[8\]G\. Biroli and M\. Mézard\(2023\)Generative diffusion in very large dimensions\.Journal of Statistical Mechanics: Theory and Experiment2023\(9\),pp\. 093402\.Cited by:[item 1](https://arxiv.org/html/2610.19042#S1.I1.i1.p1.1),[§3](https://arxiv.org/html/2610.19042#S3.p1.1)\.
- \[9\]T\. Bonnaire, R\. Urfin, G\. Biroli, and M\. Mézard\(2025\)Why diffusion models don’t memorize: the role of implicit dynamical regularization in training\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§3](https://arxiv.org/html/2610.19042#S3.p1.2)\.
- \[10\]A\. Bovier and F\. den Hollander\(2015\)Metastability: a potential\-theoretic approach\.Grundlehren der mathematischen Wissenschaften, Vol\.351,Springer\.Cited by:[§1](https://arxiv.org/html/2610.19042#S1.p2.2)\.
- \[11\]N\. Carlini, S\. Chien, M\. Nasr, S\. Song, A\. Terzis, and F\. Tramèr\(2022\)Membership inference attacks from first principles\.In2022 IEEE Symposium on Security and Privacy,pp\. 1897–1914\.Cited by:[§1](https://arxiv.org/html/2610.19042#S1.p2.2)\.
- \[12\]N\. Carlini, J\. Hayes, M\. Nasr, M\. Jagielski, V\. Sehwag, F\. Tramèr, B\. Balle, D\. Ippolito, and E\. Wallace\(2023\)Extracting training data from diffusion models\.In32nd USENIX Security Symposium,pp\. 5253–5270\.Cited by:[item 2](https://arxiv.org/html/2610.19042#S1.I1.i2.p1.1),[§1](https://arxiv.org/html/2610.19042#S1.p1.1)\.
- \[13\]N\. Carlini, F\. Tramèr, E\. Wallace, M\. Jagielski, A\. Herbert\-Voss, K\. Lee, A\. Roberts, T\. Brown, D\. Song, Ú\. Erlingsson,et al\.\(2021\)Extracting training data from large language models\.In30th USENIX Security Symposium,pp\. 2633–2650\.Cited by:[item 2](https://arxiv.org/html/2610.19042#S1.I1.i2.p1.1)\.
- \[14\]J\. Deng, W\. Dong, R\. Socher, L\. Li, K\. Li, and L\. Fei\-Fei\(2009\)ImageNet: a large\-scale hierarchical image database\.In2009 IEEE Conference on Computer Vision and Pattern Recognition,pp\. 248–255\.Cited by:[Appendix B](https://arxiv.org/html/2610.19042#A2.p3.1),[§3](https://arxiv.org/html/2610.19042#S3.p1.1)\.
- \[15\]S\. S\. Du, W\. Hu, and J\. D\. Lee\(2018\)Algorithmic regularization in learning deep homogeneous models: layers are automatically balanced\.Advances in Neural Information Processing Systems31\.Cited by:[§4\.3](https://arxiv.org/html/2610.19042#S4.SS3.p3.1)\.
- \[16\]C\. Dwork, F\. McSherry, K\. Nissim, and A\. Smith\(2006\)Calibrating noise to sensitivity in private data analysis\.InTheory of Cryptography Conference,pp\. 265–284\.Cited by:[§1](https://arxiv.org/html/2610.19042#S1.p2.2)\.
- \[17\]V\. Feldman and C\. Zhang\(2020\)What neural networks memorize and why: discovering the long tail via influence estimation\.Advances in Neural Information Processing Systems33,pp\. 2881–2891\.Cited by:[item 2](https://arxiv.org/html/2610.19042#S1.I1.i2.p1.1),[§2\.4](https://arxiv.org/html/2610.19042#S2.SS4.p1.1),[§4\.2](https://arxiv.org/html/2610.19042#S4.SS2.p2.1)\.
- \[18\]V\. Feldman\(2020\)Does learning require memorization? A short tale about a long tail\.InProceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing,pp\. 954–959\.Cited by:[item 2](https://arxiv.org/html/2610.19042#S1.I1.i2.p1.1),[§2\.4](https://arxiv.org/html/2610.19042#S2.SS4.p1.1),[§4\.2](https://arxiv.org/html/2610.19042#S4.SS2.p2.1)\.
- \[19\]I\. Garg, D\. Ravikumar, and K\. Roy\(2024\)Memorization through the lens of curvature of loss function around samples\.InProceedings of the 41st International Conference on Machine Learning,Cited by:[§1](https://arxiv.org/html/2610.19042#S1.p2.2)\.
- \[20\]X\. Gu, C\. Du, T\. Pang, C\. Li, M\. Lin, and Y\. Wang\(2023\)On memorization in diffusion models\.arXiv preprint arXiv:2310\.02664\.Cited by:[§1](https://arxiv.org/html/2610.19042#S1.p1.1),[§3](https://arxiv.org/html/2610.19042#S3.p1.2)\.
- \[21\]J\. Ho, A\. Jain, and P\. Abbeel\(2020\)Denoising diffusion probabilistic models\.Advances in Neural Information Processing Systems33,pp\. 6840–6851\.Cited by:[Appendix B](https://arxiv.org/html/2610.19042#A2.p3.1),[§1](https://arxiv.org/html/2610.19042#S1.p1.1),[§2\.2](https://arxiv.org/html/2610.19042#S2.SS2.p1.1)\.
- \[22\]C\. Huang, J\. H\. Lim, and A\. C\. Courville\(2021\)A variational perspective on diffusion\-based generative models and score matching\.Advances in Neural Information Processing Systems34,pp\. 22863–22876\.Cited by:[item \(d\)](https://arxiv.org/html/2610.19042#S1.I2.i4.p1.1),[§2\.3](https://arxiv.org/html/2610.19042#S2.SS3.p1.1),[Proposition 2\.1](https://arxiv.org/html/2610.19042#S2.Thmtheorem1)\.
- \[23\]Z\. Kadkhodaie, F\. Guth, E\. P\. Simoncelli, and S\. Mallat\(2024\)Generalization in diffusion models arises from geometry\-adaptive harmonic representations\.InInternational Conference on Learning Representations,Cited by:[item 2](https://arxiv.org/html/2610.19042#S1.I1.i2.p1.1)\.
- \[24\]D\. P\. Kingma\(2014\)Adam: a method for stochastic optimization\.arXiv preprint arXiv:1412\.6980\.Cited by:[Appendix B](https://arxiv.org/html/2610.19042#A2.p3.1)\.
- \[25\]Y\. Lipman, R\. T\. Q\. Chen, H\. Ben\-Hamu, M\. Nickel, and M\. Le\(2023\)Flow matching for generative modeling\.InInternational Conference on Learning Representations,Cited by:[Appendix B](https://arxiv.org/html/2610.19042#A2.p2.1),[Appendix B](https://arxiv.org/html/2610.19042#A2.p3.1),[§2\.2](https://arxiv.org/html/2610.19042#S2.SS2.p1.2)\.
- \[26\]C\. Liu, L\. Zhu, and M\. Belkin\(2022\)Loss landscapes and optimization in over\-parameterized non\-linear systems and neural networks\.Applied and Computational Harmonic Analysis59,pp\. 85–116\.Cited by:[§4\.5](https://arxiv.org/html/2610.19042#S4.SS5.p2.1)\.
- \[27\]X\. Liu, C\. Gong, and Q\. Liu\(2023\)Flow straight and fast: learning to generate and transfer data with rectified flow\.InInternational Conference on Learning Representations,Cited by:[Appendix B](https://arxiv.org/html/2610.19042#A2.p2.1)\.
- \[28\]Z\. Liu, P\. Luo, X\. Wang, and X\. Tang\(2015\)Deep learning face attributes in the wild\.InProceedings of the IEEE International Conference on Computer Vision,pp\. 3730–3738\.Cited by:[Appendix B](https://arxiv.org/html/2610.19042#A2.p3.1)\.
- \[29\]C\. Ma and L\. Ying\(2021\)On linear stability of SGD and input\-smoothness of neural networks\.Advances in Neural Information Processing Systems34,pp\. 16805–16817\.Cited by:[Appendix A](https://arxiv.org/html/2610.19042#A1.p2.1)\.
- \[30\]Y\. Nesterov and B\. T\. Polyak\(2006\)Cubic regularization of Newton method and its global performance\.Mathematical Programming108\(1\),pp\. 177–205\.Cited by:[Lemma A\.1](https://arxiv.org/html/2610.19042#A1.Thmtheorem1)\.
- \[31\]A\. Q\. Nichol and P\. Dhariwal\(2021\)Improved denoising diffusion probabilistic models\.InProceedings of the 38th International Conference on Machine Learning,pp\. 8162–8171\.Cited by:[§1](https://arxiv.org/html/2610.19042#S1.p1.1)\.
- \[32\]A\. Ramesh, M\. Pavlov, G\. Goh, S\. Gray, C\. Voss, A\. Radford, M\. Chen, and I\. Sutskever\(2021\)Zero\-shot text\-to\-image generation\.InInternational conference on machine learning,pp\. 8821–8831\.Cited by:[§1](https://arxiv.org/html/2610.19042#S1.p1.1)\.
- \[33\]D\. Ravikumar, E\. Soufleri, A\. Hashemi, and K\. Roy\(2024\)Unveiling privacy, memorization, and input curvature links\.InProceedings of the 41st International Conference on Machine Learning,pp\. 42192–42212\.Cited by:[§1](https://arxiv.org/html/2610.19042#S1.p2.2),[item 4](https://arxiv.org/html/2610.19042#S4.I1.i4.p1.1),[§4\.1](https://arxiv.org/html/2610.19042#S4.SS1.p1.1),[§4\.2](https://arxiv.org/html/2610.19042#S4.SS2.p2.1)\.
- \[34\]D\. Ravikumar, E\. Soufleri, and K\. Roy\(2024\)Curvature clues: decoding deep learning privacy with input loss curvature\.Advances in Neural Information Processing Systems37\.Cited by:[§1](https://arxiv.org/html/2610.19042#S1.p2.1),[§1](https://arxiv.org/html/2610.19042#S1.p2.2),[item 3](https://arxiv.org/html/2610.19042#S4.I1.i3.p1.1),[item 4](https://arxiv.org/html/2610.19042#S4.I1.i4.p1.1),[§4\.2](https://arxiv.org/html/2610.19042#S4.SS2.p2.1),[§4\.5](https://arxiv.org/html/2610.19042#S4.SS5.p3.1)\.
- \[35\]R\. Rombach, A\. Blattmann, D\. Lorenz, P\. Esser, and B\. Ommer\(2022\)High\-resolution image synthesis with latent diffusion models\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 10684–10695\.Cited by:[§1](https://arxiv.org/html/2610.19042#S1.p1.1)\.
- \[36\]O\. Ronneberger, P\. Fischer, and T\. Brox\(2015\)U\-Net: convolutional networks for biomedical image segmentation\.InMedical Image Computing and Computer\-Assisted Intervention – MICCAI 2015,N\. Navab, J\. Hornegger, W\. M\. Wells, and A\. F\. Frangi \(Eds\.\),Lecture Notes in Computer Science, Vol\.9351,pp\. 234–241\.External Links:[Document](https://dx.doi.org/10.1007/978-3-319-24574-4%5F28)Cited by:[Appendix B](https://arxiv.org/html/2610.19042#A2.p3.1)\.
- \[37\]O\. Russakovsky, J\. Deng, H\. Su, J\. Krause, S\. Satheesh, S\. Ma, Z\. Huang, A\. Karpathy, A\. Khosla, M\. Bernstein, A\. C\. Berg, and L\. Fei\-Fei\(2015\)ImageNet large scale visual recognition challenge\.International Journal of Computer Vision115\(3\),pp\. 211–252\.Cited by:[Appendix B](https://arxiv.org/html/2610.19042#A2.p3.1),[§3](https://arxiv.org/html/2610.19042#S3.p1.1)\.
- \[38\]A\. M\. Saxe, J\. L\. McClelland, and S\. Ganguli\(2019\)A mathematical theory of semantic development in deep neural networks\.Proceedings of the National Academy of Sciences116\(23\),pp\. 11537–11546\.Cited by:[§4\.3](https://arxiv.org/html/2610.19042#S4.SS3.p3.1)\.
- \[39\]R\. Shokri, M\. Stronati, C\. Song, and V\. Shmatikov\(2017\)Membership inference attacks against machine learning models\.In2017 IEEE Symposium on Security and Privacy,pp\. 3–18\.Cited by:[§1](https://arxiv.org/html/2610.19042#S1.p2.2)\.
- \[40\]J\. Sohl\-Dickstein, E\. Weiss, N\. Maheswaranathan, and S\. Ganguli\(2015\)Deep unsupervised learning using nonequilibrium thermodynamics\.InProceedings of the 32nd International Conference on Machine Learning,pp\. 2256–2265\.Cited by:[§1](https://arxiv.org/html/2610.19042#S1.p1.1)\.
- \[41\]G\. Somepalli, V\. Singla, M\. Goldblum, J\. Geiping, and T\. Goldstein\(2023\)Diffusion art or digital forgery? Investigating data replication in diffusion models\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 6048–6058\.Cited by:[§1](https://arxiv.org/html/2610.19042#S1.p1.1)\.
- \[42\]G\. Somepalli, V\. Singla, M\. Goldblum, J\. Geiping, and T\. Goldstein\(2023\)Understanding and mitigating copying in diffusion models\.Advances in Neural Information Processing Systems36,pp\. 47783–47803\.Cited by:[item 2](https://arxiv.org/html/2610.19042#S1.I1.i2.p1.1),[§1](https://arxiv.org/html/2610.19042#S1.p1.1)\.
- \[43\]J\. Song, C\. Meng, and S\. Ermon\(2021\)Denoising diffusion implicit models\.InInternational Conference on Learning Representations,Cited by:[§2\.2](https://arxiv.org/html/2610.19042#S2.SS2.p1.2)\.
- \[44\]Y\. Song and S\. Ermon\(2019\)Generative modeling by estimating gradients of the data distribution\.InAdvances in Neural Information Processing Systems,Vol\.32\.Cited by:[§1](https://arxiv.org/html/2610.19042#S1.p1.1)\.
- \[45\]Y\. Song, J\. Sohl\-Dickstein, D\. P\. Kingma, A\. Kumar, S\. Ermon, and B\. Poole\(2021\)Score\-based generative modeling through stochastic differential equations\.InInternational Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2610.19042#S1.p1.1),[§2\.2](https://arxiv.org/html/2610.19042#S2.SS2.p1.2)\.
- \[46\]N\. Srebro, K\. Sridharan, and A\. Tewari\(2010\)Optimistic rates for learning with a smooth loss\.arXiv preprint arXiv:1009\.3896\.Cited by:[Assumption 2](https://arxiv.org/html/2610.19042#Thmassumption2.p1.2.1)\.
- \[47\]A\. Vaswani, N\. Shazeer, N\. Parmar, J\. Uszkoreit, L\. Jones, A\. N\. Gomez, Ł\. Kaiser, and I\. Polosukhin\(2017\)Attention is all you need\.InAdvances in Neural Information Processing Systems,I\. Guyon, U\. V\. Luxburg, S\. Bengio, H\. Wallach, R\. Fergus, S\. Vishwanathan, and R\. Garnett \(Eds\.\),Vol\.30,pp\.\.Cited by:[Appendix B](https://arxiv.org/html/2610.19042#A2.p3.1)\.
- \[48\]P\. Vincent\(2011\)A connection between score matching and denoising autoencoders\.Neural Computation23\(7\),pp\. 1661–1674\.Cited by:[item 1](https://arxiv.org/html/2610.19042#S1.I1.i1.p1.1)\.
- \[49\]D\. Weitzner, M\. Delbracio, P\. Milanfar, and R\. Giryes\(2025\)The diffusion process as a correlation machine: linear denoising insights\.Transactions on Machine Learning Research\.Cited by:[§A\.1](https://arxiv.org/html/2610.19042#A1.SS1.p1.1)\.
- \[50\]T\. Yoon, J\. Y\. Choi, S\. Kwon, and E\. K\. Ryu\(2023\)Diffusion probabilistic models generalize when they fail to memorize\.InICML 2023 Workshop on Structured Probabilistic Inference and Generative Modeling,Cited by:[item 1](https://arxiv.org/html/2610.19042#S1.I1.i1.p1.1),[§3](https://arxiv.org/html/2610.19042#S3.p1.2)\.
- \[51\]C\. Zhang, S\. Bengio, M\. Hardt, B\. Recht, and O\. Vinyals\(2017\)Understanding deep learning requires rethinking generalization\.InInternational Conference on Learning Representations,Cited by:[item 2](https://arxiv.org/html/2610.19042#S1.I1.i2.p1.1)\.
- \[52\]C\. Zhang, D\. Ippolito, K\. Lee, M\. Jagielski, F\. Tramèr, and N\. Carlini\(2023\)Counterfactual memorization in neural language models\.Advances in Neural Information Processing Systems36,pp\. 39321–39362\.Cited by:[item 2](https://arxiv.org/html/2610.19042#S1.I1.i2.p1.1),[§4](https://arxiv.org/html/2610.19042#S4.p1.1)\.

## Appendix AProofs for Section[4](https://arxiv.org/html/2610.19042#S4)

###### Lemma A\.1\(Lemma 1 from\[[30](https://arxiv.org/html/2610.19042#bib.bib5)\]\)\.

Supposeℓ⁡\(ϕ,z\)\\ell\(\\phi,z\)satsifies Assumption[1](https://arxiv.org/html/2610.19042#Thmassumption1)ofρ\\rho\-Lipschitz Hessians\. Then for anyz,z′∈ℝdz,z^\{\\prime\}\\in\\mathbb\{R\}^\{d\}

−ρ6​‖z′−z‖23≤ℓ⁡\(ϕ,z′\)−ℓ⁡\(ϕ,z\)−⟨∇ℓ​\(ϕ,z\),z−z′⟩−\(z′−z\)⊤​∇z2ℓ​\(ϕ,z\)​\(z′−z\)≤ρ6​‖z′−z‖23\.\\displaystyle\-\\frac\{\\rho\}\{6\}\\\|z^\{\\prime\}\-z\\\|\_\{2\}^\{3\}\\leq\\ell\(\\phi,z^\{\\prime\}\)\-\\ell\(\\phi,z\)\-\\langle\\nabla\\ell\(\\phi,z\),z\-z^\{\\prime\}\\rangle\-\(z^\{\\prime\}\-z\)^\{\\top\}\\nabla^\{2\}\_\{z\}\\ell\(\\phi,z\)\(z^\{\\prime\}\-z\)\\leq\\frac\{\\rho\}\{6\}\\\|z^\{\\prime\}\-z\\\|\_\{2\}^\{3\}\.\(37\)

Using Assumption \([1](https://arxiv.org/html/2610.19042#Thmassumption1)\) we can also obtain the following bound on the gradients ofℓ\\ell:

###### Lemma A\.2\.

Supposeℓ⁡\(ϕ,z\)\\ell\(\\phi,z\)satisfies Assumption[1](https://arxiv.org/html/2610.19042#Thmassumption1)ofρ\\rho\-Lipschitz Hessians\. Then for anyz,z′∈ℝdz,z^\{\\prime\}\\in\\mathbb\{R\}^\{d\}

‖∇zℓ​\(ϕ,z′\)−∇zℓ​\(ϕ,z\)‖2≤‖∇z2ℓ​\(ϕ,z\)‖2→2​‖z′−z‖2\+ρ2​‖z′−z‖22\.\\displaystyle\\\|\\nabla\_\{z\}\\ell\(\\phi,z^\{\\prime\}\)\-\\nabla\_\{z\}\\ell\(\\phi,z\)\\\|\_\{2\}\\leq\\\|\\nabla^\{2\}\_\{z\}\\ell\(\\phi,z\)\\\|\_\{2\\to 2\}\\\|z^\{\\prime\}\-z\\\|\_\{2\}\+\\frac\{\\rho\}\{2\}\\\|z^\{\\prime\}\-z\\\|\_\{2\}^\{2\}\.\(38\)Here‖∇z2​ℓ​\(ϕ,z\)‖2→2\\\|\\nabla^\{2\}\_\{z\}\\ell\(\\phi,z\)\\\|\_\{2\\to 2\}is the operator norm of the Hessian\.

Following an argument of Ma & Ying\[[29](https://arxiv.org/html/2610.19042#bib.bib6)\]we use the multiplicative nature of neural networks to prove a bound on the input gradients in terms of the parameter gradients\.

###### Lemma A\.3\.

Supposeℓ⁡\(ϕ,z\)=g⁡\(ϕ1​z,ϕ2,z\)\\ell\(\\phi,z\)=g\(\\phi\_\{1\}z,\\phi\_\{2\},z\)and there exists a constantC2C\_\{2\}such that‖∇cg‖≤C2​g\\\|\\nabla\_\{c\}g\\\|\\leq C\_\{2\}g\. Then we have

‖∇zℓ​\(ϕ,z\)‖2≤‖ϕ1‖‖z‖​‖∇ϕ1ℓ​\(ϕ,z\)‖\+C2​ℓ​\(ϕ,z\)\.\\displaystyle\\\|\\nabla\_\{z\}\\ell\(\\phi,z\)\\\|\_\{2\}\\leq\\frac\{\\\|\\phi\_\{1\}\\\|\}\{\\\|z\\\|\}\\\|\\nabla\_\{\\phi\_\{1\}\}\\ell\(\\phi,z\)\\\|\+C\_\{2\}\\ell\(\\phi,z\)\.\(39\)

###### Proof\.

We compute the gradient with respect to the inputs:

∇zℓ​\(ϕ,z\)=∇zg​\(ϕ1​z,ϕ2,z\)=ϕ1⊤​∇ag​\(ϕ1​z,ϕ2,z\)\+∇cg​\(ϕ1​z,ϕ2,z\),\\displaystyle\\nabla\_\{z\}\\ell\(\\phi,z\)=\\nabla\_\{z\}g\(\\phi\_\{1\}z,\\phi\_\{2\},z\)=\\phi\_\{1\}^\{\\top\}\\nabla\_\{a\}g\(\\phi\_\{1\}z,\\phi\_\{2\},z\)\+\\nabla\_\{c\}g\(\\phi\_\{1\}z,\\phi\_\{2\},z\),\(40\)∇ϕ1ℓ​\(ϕ,z\)=∇ag​\(ϕ1​z,ϕ2,z\)​z⊤\.\\displaystyle\\nabla\_\{\\phi\_\{1\}\}\\ell\(\\phi,z\)=\\nabla\_\{a\}g\(\\phi\_\{1\}z,\\phi\_\{2\},z\)z^\{\\top\}\.\(41\)Multiplying the first equation on the right byz⊤z^\{\\top\}we get

∇zℓ​\(ϕ,z\)​z⊤=ϕ1⊤​∇ag​\(ϕ1​z,ϕ2,z\)​z⊤\+∇cg​\(ϕ1​z,ϕ2,z\)​z⊤\\displaystyle\\nabla\_\{z\}\\ell\(\\phi,z\)z^\{\\top\}=\\phi\_\{1\}^\{\\top\}\\nabla\_\{a\}g\(\\phi\_\{1\}z,\\phi\_\{2\},z\)z^\{\\top\}\+\\nabla\_\{c\}g\(\\phi\_\{1\}z,\\phi\_\{2\},z\)z^\{\\top\}\(42\)⇔∇zℓ​\(ϕ,z\)​z⊤=ϕ1⊤​∇ϕ1ℓ​\(ϕ,z\)\+∇cg​\(ϕ1​z,ϕ2,z\)​z⊤\\displaystyle\\iff\\nabla\_\{z\}\\ell\(\\phi,z\)z^\{\\top\}=\\phi\_\{1\}^\{\\top\}\\nabla\_\{\\phi\_\{1\}\}\\ell\(\\phi,z\)\+\\nabla\_\{c\}g\(\\phi\_\{1\}z,\\phi\_\{2\},z\)z^\{\\top\}\(43\)⇔∇zℓ​\(ϕ,z\)=ϕ1⊤​∇ϕ1ℓ​\(ϕ,z\)​z‖z‖22\+∇cg​\(ϕ1​z,ϕ2,z\)\.\\displaystyle\\iff\\nabla\_\{z\}\\ell\(\\phi,z\)=\\frac\{\\phi\_\{1\}^\{\\top\}\\nabla\_\{\\phi\_\{1\}\}\\ell\(\\phi,z\)z\}\{\\\|z\\\|\_\{2\}^\{2\}\}\+\\nabla\_\{c\}g\(\\phi\_\{1\}z,\\phi\_\{2\},z\)\.\(44\)Now, taking norms and using our assumption we have:

‖∇zℓ​\(ϕ,z\)‖2≤‖ϕ1‖‖z‖​‖∇ϕ1ℓ​\(ϕ,z\)‖\+C2​ℓ​\(ϕ,z\)\.\\displaystyle\\\|\\nabla\_\{z\}\\ell\(\\phi,z\)\\\|\_\{2\}\\leq\\frac\{\\\|\\phi\_\{1\}\\\|\}\{\\\|z\\\|\}\\\|\\nabla\_\{\\phi\_\{1\}\}\\ell\(\\phi,z\)\\\|\+C\_\{2\}\\ell\(\\phi,z\)\.\(45\)∎

Before going further, we remark on the significance of Lemma[A\.3](https://arxiv.org/html/2610.19042#A1.Thmtheorem3)\. In machine learning, we optimize the loss with respect to the parameters in the modelϕ\\phi\. However, since we are interested in the effect of the inclusion of a single training sample, we are more interested in the behaviour of the loss with respect to its*inputs*zz\. Lemma[A\.3](https://arxiv.org/html/2610.19042#A1.Thmtheorem3)gives us a way to relate these two, specifically relating the sizes of the input and parameter gradients of the loss\. In particular, it says that the input gradients can be \(roughly\) controlled by the parameter gradients, which we explicitly control via optimization\.

Using the cubic envelope from Lemma \([A\.1](https://arxiv.org/html/2610.19042#A1.Thmtheorem1)\) we may bound the memorization termM⁡\(zi,S\)M\(z\_\{i\},S\)from \([46](https://arxiv.org/html/2610.19042#A1.E46)\)\.

###### Lemma A\.4\.

LetSSbe a finite dataset and letS′⊆SS^\{\\prime\}\\subseteq Sand letzi∈S∖S′z\_\{i\}\\in S\\setminus S^\{\\prime\}\. Then under assumptions \([1](https://arxiv.org/html/2610.19042#Thmassumption1)\) and \([6](https://arxiv.org/html/2610.19042#Thmassumption6)\), for anyzj≠zi∈S′z\_\{j\}\\neq z\_\{i\}\\in S^\{\\prime\}we have:

𝔼ϕ​\[ℓ⁡\(hSϕ,zi\)\]−𝔼ϕ′​\[ℓ⁡\(hS′ϕ′,zj\)\]\\displaystyle\\mathbb\{E\}\_\{\\phi\}\[\\ell\(h^\{\\phi\}\_\{S\},z\_\{i\}\)\]\-\\mathbb\{E\}\_\{\\phi^\{\\prime\}\}\[\\ell\(h^\{\\phi^\{\\prime\}\}\_\{S^\{\\prime\}\},z\_\{j\}\)\]≤C\+\|⟨𝔼ϕ∼𝒜⁡\(S′\)​\[∇zℓ​\(hS′ϕ′,zi\)\],zj−zi⟩\|\\displaystyle\\leq C\+\\lvert\\langle\\mathbb\{E\}\_\{\\phi\\sim\\mathcal\{A\}\(S^\{\\prime\}\)\}\[\\nabla\_\{z\}\\ell\(h^\{\\phi^\{\\prime\}\}\_\{S^\{\\prime\}\},z\_\{i\}\)\],z\_\{j\}\-z\_\{i\}\\rangle\\rvert\+\|\(zj−zi\)⊤​𝔼ϕ∼𝒜⁡\(S′\)​\[∇z2​ℓ​\(hS′ϕ′,zi\)\]​\(zj−zi\)\|\+ρ6​‖zj−zi‖23\.\\displaystyle\+\\lvert\(z\_\{j\}\-z\_\{i\}\)^\{\\top\}\\mathbb\{E\}\_\{\\phi\\sim\\mathcal\{A\}\(S^\{\\prime\}\)\}\[\\nabla^\{2\}\_\{z\}\\ell\(h^\{\\phi^\{\\prime\}\}\_\{S^\{\\prime\}\},z\_\{i\}\)\]\(z\_\{j\}\-z\_\{i\}\)\\rvert\+\\frac\{\\rho\}\{6\}\\\|z\_\{j\}\-z\_\{i\}\\\|^\{3\}\_\{2\}\.\(46\)

###### Proof\.

Letϕ′∼𝒜⁡\(S′\)\\phi^\{\\prime\}\\sim\\mathcal\{A\}\(S^\{\\prime\}\)\. Forzj,ziz\_\{j\},z\_\{i\}, we have, using Lemma[A\.1](https://arxiv.org/html/2610.19042#A1.Thmtheorem1)that:

−ℓ⁡\(hS′ϕ′,zj\)≥−ℓ⁡\(hS′ϕ′,zi\)−⟨∇zℓ​\(hS′ϕ′,zi\),zj−zi⟩−\(zj−zi\)⊤​∇z2ℓ​\(hS′ϕ′,zi\)​\(zj−zi\)−ρ6​‖zj−zi‖23,\\displaystyle\-\\ell\(h^\{\\phi^\{\\prime\}\}\_\{S^\{\\prime\}\},z\_\{j\}\)\\geq\-\\ell\(h^\{\\phi^\{\\prime\}\}\_\{S^\{\\prime\}\},z\_\{i\}\)\-\\langle\\nabla\_\{z\}\\ell\(h^\{\\phi^\{\\prime\}\}\_\{S^\{\\prime\}\},z\_\{i\}\),z\_\{j\}\-z\_\{i\}\\rangle\-\(z\_\{j\}\-z\_\{i\}\)^\{\\top\}\\nabla^\{2\}\_\{z\}\\ell\(h^\{\\phi^\{\\prime\}\}\_\{S^\{\\prime\}\},z\_\{i\}\)\(z\_\{j\}\-z\_\{i\}\)\-\\frac\{\\rho\}\{6\}\\\|z\_\{j\}\-z\_\{i\}\\\|^\{3\}\_\{2\},\(47\)ℓ⁡\(hS′ϕ′,zj\)≤ℓ⁡\(hS′ϕ′,zi\)\+⟨∇zℓ​\(hS′ϕ′,zi\),zj−zi⟩\+\(zj−zi\)⊤​∇z2ℓ​\(hS′ϕ′,zi\)​\(zj−zi\)\+ρ6​‖zj−zi‖23\.\\displaystyle\\ell\(h^\{\\phi^\{\\prime\}\}\_\{S^\{\\prime\}\},z\_\{j\}\)\\leq\\ell\(h^\{\\phi^\{\\prime\}\}\_\{S^\{\\prime\}\},z\_\{i\}\)\+\\langle\\nabla\_\{z\}\\ell\(h^\{\\phi^\{\\prime\}\}\_\{S^\{\\prime\}\},z\_\{i\}\),z\_\{j\}\-z\_\{i\}\\rangle\+\(z\_\{j\}\-z\_\{i\}\)^\{\\top\}\\nabla^\{2\}\_\{z\}\\ell\(h^\{\\phi^\{\\prime\}\}\_\{S^\{\\prime\}\},z\_\{i\}\)\(z\_\{j\}\-z\_\{i\}\)\+\\frac\{\\rho\}\{6\}\\\|z\_\{j\}\-z\_\{i\}\\\|^\{3\}\_\{2\}\.\(48\)Using the monotonicity of expectation we therefore get:

−𝔼ϕ∼𝒜⁡\(S′\)​\[ℓ⁡\(hS′ϕ′,zj\)\]\\displaystyle\-\\mathbb\{E\}\_\{\\phi\\sim\\mathcal\{A\}\(S^\{\\prime\}\)\}\[\\ell\(h^\{\\phi^\{\\prime\}\}\_\{S^\{\\prime\}\},z\_\{j\}\)\]≥−𝔼ϕ∼𝒜⁡\(S′\)​\[ℓ⁡\(hS′ϕ′,zi\)\]−⟨𝔼ϕ∼𝒜⁡\(S′\)​\[∇zℓ​\(hS′ϕ′,zi\)\],zj−zi⟩\\displaystyle\\geq\-\\mathbb\{E\}\_\{\\phi\\sim\\mathcal\{A\}\(S^\{\\prime\}\)\}\[\\ell\(h^\{\\phi^\{\\prime\}\}\_\{S^\{\\prime\}\},z\_\{i\}\)\]\-\\langle\\mathbb\{E\}\_\{\\phi\\sim\\mathcal\{A\}\(S^\{\\prime\}\)\}\[\\nabla\_\{z\}\\ell\(h^\{\\phi^\{\\prime\}\}\_\{S^\{\\prime\}\},z\_\{i\}\)\],z\_\{j\}\-z\_\{i\}\\rangle\(49\)−\(zj−zi\)⊤​𝔼ϕ∼𝒜⁡\(S′\)​\[∇z2​ℓ​\(hS′ϕ′,zi\)\]​\(zj−zi\)−ρ6​‖zj−zi‖23,\\displaystyle\-\(z\_\{j\}\-z\_\{i\}\)^\{\\top\}\\mathbb\{E\}\_\{\\phi\\sim\\mathcal\{A\}\(S^\{\\prime\}\)\}\[\\nabla^\{2\}\_\{z\}\\ell\(h^\{\\phi^\{\\prime\}\}\_\{S^\{\\prime\}\},z\_\{i\}\)\]\(z\_\{j\}\-z\_\{i\}\)\-\\frac\{\\rho\}\{6\}\\\|z\_\{j\}\-z\_\{i\}\\\|^\{3\}\_\{2\},\(50\)𝔼ϕ∼𝒜⁡\(S′\)​\[ℓ⁡\(hS′ϕ′,zj\)\]\\displaystyle\\mathbb\{E\}\_\{\\phi\\sim\\mathcal\{A\}\(S^\{\\prime\}\)\}\[\\ell\(h^\{\\phi^\{\\prime\}\}\_\{S^\{\\prime\}\},z\_\{j\}\)\]≤𝔼ϕ∼𝒜⁡\(S′\)​\[ℓ⁡\(hS′ϕ′,zi\)\]\+⟨𝔼ϕ∼𝒜⁡\(S′\)​\[∇zℓ​\(hS′ϕ′,zi\)\],zj−zi⟩\\displaystyle\\leq\\mathbb\{E\}\_\{\\phi\\sim\\mathcal\{A\}\(S^\{\\prime\}\)\}\[\\ell\(h^\{\\phi^\{\\prime\}\}\_\{S^\{\\prime\}\},z\_\{i\}\)\]\+\\langle\\mathbb\{E\}\_\{\\phi\\sim\\mathcal\{A\}\(S^\{\\prime\}\)\}\[\\nabla\_\{z\}\\ell\(h^\{\\phi^\{\\prime\}\}\_\{S^\{\\prime\}\},z\_\{i\}\)\],z\_\{j\}\-z\_\{i\}\\rangle\(51\)\+\(zj−zi\)⊤​𝔼ϕ∼𝒜⁡\(S′\)​\[∇z2​ℓ​\(hS′ϕ′,zi\)\]​\(zj−zi\)\+ρ6​‖zj−zi‖23\.\\displaystyle\+\(z\_\{j\}\-z\_\{i\}\)^\{\\top\}\\mathbb\{E\}\_\{\\phi\\sim\\mathcal\{A\}\(S^\{\\prime\}\)\}\[\\nabla^\{2\}\_\{z\}\\ell\(h^\{\\phi^\{\\prime\}\}\_\{S^\{\\prime\}\},z\_\{i\}\)\]\(z\_\{j\}\-z\_\{i\}\)\+\\frac\{\\rho\}\{6\}\\\|z\_\{j\}\-z\_\{i\}\\\|^\{3\}\_\{2\}\.\(52\)Plugging these into the two\-sided bound implied by Assumption[6](https://arxiv.org/html/2610.19042#Thmassumption6)we get:

C\\displaystyle C≥𝔼ϕ​\[ℓ⁡\(hSϕ,zi\)\]−𝔼ϕ′​\[ℓ⁡\(hS′ϕ′,zj\)\]\\displaystyle\\geq\\mathbb\{E\}\_\{\\phi\}\[\\ell\(h^\{\\phi\}\_\{S\},z\_\{i\}\)\]\-\\mathbb\{E\}\_\{\\phi^\{\\prime\}\}\[\\ell\(h^\{\\phi^\{\\prime\}\}\_\{S^\{\\prime\}\},z\_\{j\}\)\]\(53\)≥𝔼ϕ​\[ℓ⁡\(hSϕ,zi\)\]−𝔼ϕ∼𝒜⁡\(S′\)​\[ℓ⁡\(hS′ϕ′,zi\)\]−⟨𝔼ϕ∼𝒜⁡\(S′\)​\[∇zℓ​\(hS′ϕ′,zi\)\],zj−zi⟩\\displaystyle\\geq\\mathbb\{E\}\_\{\\phi\}\[\\ell\(h^\{\\phi\}\_\{S\},z\_\{i\}\)\]\-\\mathbb\{E\}\_\{\\phi\\sim\\mathcal\{A\}\(S^\{\\prime\}\)\}\[\\ell\(h^\{\\phi^\{\\prime\}\}\_\{S^\{\\prime\}\},z\_\{i\}\)\]\-\\langle\\mathbb\{E\}\_\{\\phi\\sim\\mathcal\{A\}\(S^\{\\prime\}\)\}\[\\nabla\_\{z\}\\ell\(h^\{\\phi^\{\\prime\}\}\_\{S^\{\\prime\}\},z\_\{i\}\)\],z\_\{j\}\-z\_\{i\}\\rangle\(54\)−\(zj−zi\)⊤​𝔼ϕ∼𝒜⁡\(S′\)​\[∇z2​ℓ​\(hS′ϕ′,zi\)\]​\(zj−zi\)−ρ6​‖zj−zi‖23\.\\displaystyle\-\(z\_\{j\}\-z\_\{i\}\)^\{\\top\}\\mathbb\{E\}\_\{\\phi\\sim\\mathcal\{A\}\(S^\{\\prime\}\)\}\[\\nabla^\{2\}\_\{z\}\\ell\(h^\{\\phi^\{\\prime\}\}\_\{S^\{\\prime\}\},z\_\{i\}\)\]\(z\_\{j\}\-z\_\{i\}\)\-\\frac\{\\rho\}\{6\}\\\|z\_\{j\}\-z\_\{i\}\\\|^\{3\}\_\{2\}\.\(55\)By rearranging, we have

𝔼ϕ​\[ℓ⁡\(hSϕ,zi\)\]−𝔼ϕ′​\[ℓ⁡\(hS′ϕ′,zj\)\]\\displaystyle\\mathbb\{E\}\_\{\\phi\}\[\\ell\(h^\{\\phi\}\_\{S\},z\_\{i\}\)\]\-\\mathbb\{E\}\_\{\\phi^\{\\prime\}\}\[\\ell\(h^\{\\phi^\{\\prime\}\}\_\{S^\{\\prime\}\},z\_\{j\}\)\]≤C\+⟨𝔼ϕ∼𝒜⁡\(S′\)​\[∇zℓ​\(hS′ϕ′,zi\)\],zj−zi⟩\\displaystyle\\leq C\+\\langle\\mathbb\{E\}\_\{\\phi\\sim\\mathcal\{A\}\(S^\{\\prime\}\)\}\[\\nabla\_\{z\}\\ell\(h^\{\\phi^\{\\prime\}\}\_\{S^\{\\prime\}\},z\_\{i\}\)\],z\_\{j\}\-z\_\{i\}\\rangle\(56\)\+\(zj−zi\)⊤​𝔼ϕ∼𝒜⁡\(S′\)​\[∇z2​ℓ​\(hS′ϕ′,zi\)\]​\(zj−zi\)\+ρ6​‖zj−zi‖23\.\\displaystyle\+\(z\_\{j\}\-z\_\{i\}\)^\{\\top\}\\mathbb\{E\}\_\{\\phi\\sim\\mathcal\{A\}\(S^\{\\prime\}\)\}\[\\nabla^\{2\}\_\{z\}\\ell\(h^\{\\phi^\{\\prime\}\}\_\{S^\{\\prime\}\},z\_\{i\}\)\]\(z\_\{j\}\-z\_\{i\}\)\+\\frac\{\\rho\}\{6\}\\\|z\_\{j\}\-z\_\{i\}\\\|^\{3\}\_\{2\}\.\(57\)Using the same tactic, we may prove a lower bound for𝔼ϕ​\[ℓ⁡\(hSϕ,zi\)\]−𝔼ϕ′​\[ℓ⁡\(hS′ϕ′,zj\)\]\\mathbb\{E\}\_\{\\phi\}\[\\ell\(h^\{\\phi\}\_\{S\},z\_\{i\}\)\]\-\\mathbb\{E\}\_\{\\phi^\{\\prime\}\}\[\\ell\(h^\{\\phi^\{\\prime\}\}\_\{S^\{\\prime\}\},z\_\{j\}\)\]using the negative of the term above\. This yields a bound for the absolute value, and recalling the definition ofMMyields the bound \([46](https://arxiv.org/html/2610.19042#A1.E46)\)\. ∎

We can now and prove our main result[4\.3](https://arxiv.org/html/2610.19042#S4.Thmtheorem3)\.

###### Proof\.

First, we construct the high probability event under which the bound should hold\. Set the threshold in assumption[4](https://arxiv.org/html/2610.19042#Thmassumption4)to beo\(n−1/2\)o\(n^\{\-1/2\}\)wherennis the size of the dataset\. Then fix a threshold1−δ⁡\(n\)1\-\\delta\(n\)such that both events under which assumptions[6](https://arxiv.org/html/2610.19042#Thmassumption6)and[5](https://arxiv.org/html/2610.19042#Thmassumption5)hold\. In this high probability event we apply Lemma[A\.4](https://arxiv.org/html/2610.19042#A1.Thmtheorem4)to the setS′=S∖i:=S∖\{zi\}S^\{\\prime\}=S^\{\\setminus i\}:=S\\setminus\\\{z\_\{i\}\\\}to obtain \([46](https://arxiv.org/html/2610.19042#A1.E46)\):

M⁡\(zi,S\)\\displaystyle M\(z\_\{i\},S\)≤C\+\|⟨𝔼ϕ∼𝒜⁡\(S∖i\)​\[∇zℓ​\(hS∖iϕ′,zi\)\],zj−zi⟩\|⏟A\\displaystyle\\leq C\+\\underbrace\{\{\\lvert\\langle\\mathbb\{E\}\_\{\\phi\\sim\\mathcal\{A\}\(S^\{\\setminus i\}\)\}\[\\nabla\_\{z\}\\ell\(h^\{\\phi^\{\\prime\}\}\_\{S^\{\\setminus i\}\},z\_\{i\}\)\],z\_\{j\}\-z\_\{i\}\\rangle\\rvert\}\}\_\{A\}\+\|\(zj−zi\)⊤​𝔼ϕ∼𝒜⁡\(S∖i\)​\[∇z2​ℓ​\(hS∖iϕ′,zi\)\]​\(zj−zi\)\|\+ρ6​‖zj−zi‖23⏟B\.\\displaystyle\+\\underbrace\{\{\\lvert\(z\_\{j\}\-z\_\{i\}\)^\{\\top\}\\mathbb\{E\}\_\{\\phi\\sim\\mathcal\{A\}\(S^\{\\setminus i\}\)\}\[\\nabla^\{2\}\_\{z\}\\ell\(h^\{\\phi^\{\\prime\}\}\_\{S^\{\\setminus i\}\},z\_\{i\}\)\]\(z\_\{j\}\-z\_\{i\}\)\\rvert\+\\frac\{\\rho\}\{6\}\\\|z\_\{j\}\-z\_\{i\}\\\|^\{3\}\_\{2\}\}\}\_\{B\}\.\([46](https://arxiv.org/html/2610.19042#A1.E46)\)
To bound\(A\), we use Cauchy\-Schwarz and Jensen’s inequality:

\|⟨𝔼ϕ∼𝒜⁡\(S∖i\)​\[∇zℓ​\(hS∖iϕ′,zi\)\],zj−zi⟩\|≤𝔼ϕ∼𝒜⁡\(S∖i\)​\[‖∇zℓ​\(hS∖iϕ′,zi\)‖2\]​‖zj−zi‖2\.\\displaystyle\\lvert\\langle\\mathbb\{E\}\_\{\\phi\\sim\\mathcal\{A\}\(S^\{\\setminus i\}\)\}\[\\nabla\_\{z\}\\ell\(h^\{\\phi^\{\\prime\}\}\_\{S^\{\\setminus i\}\},z\_\{i\}\)\],z\_\{j\}\-z\_\{i\}\\rangle\\rvert\\leq\\mathbb\{E\}\_\{\\phi\\sim\\mathcal\{A\}\(S^\{\\setminus i\}\)\}\[\\\|\\nabla\_\{z\}\\ell\(h^\{\\phi^\{\\prime\}\}\_\{S^\{\\setminus i\}\},z\_\{i\}\)\\\|\_\{2\}\]\\\|z\_\{j\}\-z\_\{i\}\\\|\_\{2\}\.\(58\)We now apply Lemma[A\.2](https://arxiv.org/html/2610.19042#A1.Thmtheorem2)on∇zℓ\(hS∖iϕ′,zi\)∥2\\nabla\_\{z\}\\ell\(h^\{\\phi^\{\\prime\}\}\_\{S^\{\\setminus i\}\},z\_\{i\}\)\\\|\_\{2\}and take expectations to obtain:

𝔼ϕ∼𝒜⁡\(S∖i\)​\[‖∇zℓ​\(hS∖iϕ′,zi\)‖2\]​‖zj−zi‖2\\displaystyle\\mathbb\{E\}\_\{\\phi\\sim\\mathcal\{A\}\(S^\{\\setminus i\}\)\}\[\\\|\\nabla\_\{z\}\\ell\(h^\{\\phi^\{\\prime\}\}\_\{S^\{\\setminus i\}\},z\_\{i\}\)\\\|\_\{2\}\]\\\|z\_\{j\}\-z\_\{i\}\\\|\_\{2\}≤𝔼ϕ∼𝒜⁡\(S∖i\)​\[‖∇zℓ​\(hS∖iϕ′,zj\)‖2\]​‖zj−zi‖2\\displaystyle\\leq\\mathbb\{E\}\_\{\\phi\\sim\\mathcal\{A\}\(S^\{\\setminus i\}\)\}\[\\\|\\nabla\_\{z\}\\ell\(h^\{\\phi^\{\\prime\}\}\_\{S^\{\\setminus i\}\},z\_\{j\}\)\\\|\_\{2\}\]\\\|z\_\{j\}\-z\_\{i\}\\\|\_\{2\}\(59\)\+𝔼ϕ′∼𝒜⁡\(S∖i\)​\[‖∇z2ℓ​\(hS∖iϕ′,zi\)‖2→2\]​‖zj−zi‖22\+ρ2​‖zj−zi‖23\.\\displaystyle\+\\mathbb\{E\}\_\{\\phi^\{\\prime\}\\sim\\mathcal\{A\}\(S^\{\\setminus i\}\)\}\[\\\|\\nabla^\{2\}\_\{z\}\\ell\(h^\{\\phi^\{\\prime\}\}\_\{S^\{\\setminus i\}\},z\_\{i\}\)\\\|\_\{2\\to 2\}\]\\\|z\_\{j\}\-z\_\{i\}\\\|\_\{2\}^\{2\}\+\\frac\{\\rho\}\{2\}\\\|z\_\{j\}\-z\_\{i\}\\\|\_\{2\}^\{3\}\.\(60\)We now apply Lemma[A\.3](https://arxiv.org/html/2610.19042#A1.Thmtheorem3)to \([59](https://arxiv.org/html/2610.19042#A1.E59)\) and thus get:

𝔼ϕ∼𝒜⁡\(S∖i\)​\[‖∇zℓ​\(hS∖iϕ′,zj\)‖2\]​‖zj−zi‖2\\displaystyle\\mathbb\{E\}\_\{\\phi\\sim\\mathcal\{A\}\(S^\{\\setminus i\}\)\}\[\\\|\\nabla\_\{z\}\\ell\(h^\{\\phi^\{\\prime\}\}\_\{S^\{\\setminus i\}\},z\_\{j\}\)\\\|\_\{2\}\]\\\|z\_\{j\}\-z\_\{i\}\\\|\_\{2\}≤\(𝔼ϕ′∼𝒜⁡\(S∖i\)​\[‖ϕ1′‖‖z‖​‖∇ϕ1′ℓ​\(hS∖iϕ′,zj\)‖\+2​β​ℓ​\(hϕ′,zj\)\]\)​‖zj−zi‖2\\displaystyle\\leq\\Bigg\(\\mathbb\{E\}\_\{\\phi^\{\\prime\}\\sim\\mathcal\{A\}\(S^\{\\setminus i\}\)\}\\Bigg\[\\frac\{\\\|\\phi^\{\\prime\}\_\{1\}\\\|\}\{\\\|z\\\|\}\\\|\\nabla\_\{\\phi^\{\\prime\}\_\{1\}\}\\ell\(h^\{\\phi^\{\\prime\}\}\_\{S^\{\\setminus i\}\},z\_\{j\}\)\\\|\+2\\beta\\ell\(h^\{\\phi^\{\\prime\}\},z\_\{j\}\)\\Bigg\]\\Bigg\)\\\|z\_\{j\}\-z\_\{i\}\\\|\_\{2\}\(61\)OPEN=𝔼ϕ′∼𝒜⁡\(S∖iCLOSE\)​\(Bmink⁡‖zk‖​εgrad\+2​β​εloss\)​‖zj−zi‖2\\displaystyle=\\mathbb\{E\}\_\{\\phi^\{\\prime\}\\sim\\mathcal\{A\}\(S^\{\\setminus i\}\}\)\\left\(\\frac\{B\}\{\\min\_\{k\}\\\|z\_\{k\}\\\|\}\\varepsilon\_\{\\mathrm\{grad\}\}\+2\\beta\\varepsilon\_\{\\mathrm\{loss\}\}\\right\)\\\|z\_\{j\}\-z\_\{i\}\\\|\_\{2\}\(62\)The last equality follows due to assumption[4](https://arxiv.org/html/2610.19042#Thmassumption4)and[5](https://arxiv.org/html/2610.19042#Thmassumption5)which results in both the loss and the gradients of the loss of a trained model at a training sample to be small \(compared to the parameters and the nearest neighbor distance‖zj−zi‖\\\|z\_\{j\}\-z\_\{i\}\\\|\)\. Putting \([62](https://arxiv.org/html/2610.19042#A1.E62)\) with \([59](https://arxiv.org/html/2610.19042#A1.E59)\) we get:

\|⟨𝔼ϕ∼𝒜⁡\(S∖i\)​\[∇zℓ​\(hS∖iϕ′,zi\)\],zj−zi⟩\|\\displaystyle\\lvert\\langle\\mathbb\{E\}\_\{\\phi\\sim\\mathcal\{A\}\(S^\{\\setminus i\}\)\}\[\\nabla\_\{z\}\\ell\(h^\{\\phi^\{\\prime\}\}\_\{S^\{\\setminus i\}\},z\_\{i\}\)\],z\_\{j\}\-z\_\{i\}\\rangle\\rvert≤𝔼ϕ′∼𝒜⁡\(S∖i\)​\[‖∇z2ℓ​\(hS∖iϕ′,zi\)‖2→2\]​‖zj−zi‖22\\displaystyle\\leq\\mathbb\{E\}\_\{\\phi^\{\\prime\}\\sim\\mathcal\{A\}\(S^\{\\setminus i\}\)\}\[\\\|\\nabla^\{2\}\_\{z\}\\ell\(h^\{\\phi^\{\\prime\}\}\_\{S^\{\\setminus i\}\},z\_\{i\}\)\\\|\_\{2\\to 2\}\]\\\|z\_\{j\}\-z\_\{i\}\\\|\_\{2\}^\{2\}\(63\)OPEN\+𝔼ϕ′∼𝒜⁡\(S∖iCLOSE\)​\(Bmink⁡‖zk‖​εgrad\+2​β​εloss\)​‖zj−zi‖2\+O⁡\(‖zj−zi‖23\)\.\\displaystyle\+\\mathbb\{E\}\_\{\\phi^\{\\prime\}\\sim\\mathcal\{A\}\(S^\{\\setminus i\}\}\)\\left\(\\frac\{B\}\{\\min\_\{k\}\\\|z\_\{k\}\\\|\}\\varepsilon\_\{\\mathrm\{grad\}\}\+2\\beta\\varepsilon\_\{\\mathrm\{loss\}\}\\right\)\\\|z\_\{j\}\-z\_\{i\}\\\|\_\{2\}\+O\(\\\|z\_\{j\}\-z\_\{i\}\\\|\_\{2\}^\{3\}\)\.\(64\)To bound\(B\)we use an operator norm upper bound along with Jensen’s inequality to bound the quadratic term in \([46](https://arxiv.org/html/2610.19042#A1.E46)\) also by the operator norm of the hessian appearing in \([64](https://arxiv.org/html/2610.19042#A1.E64)\):

\|\(zj−zi\)⊤​𝔼ϕ′∼𝒜⁡\(S∖i\)​\[∇z2​ℓ​\(hS∖iϕ′,zi\)\]​\(zj−zi\)\|\\displaystyle\\lvert\(z\_\{j\}\-z\_\{i\}\)^\{\\top\}\\mathbb\{E\}\_\{\\phi^\{\\prime\}\\sim\\mathcal\{A\}\(S^\{\\setminus i\}\)\}\[\\nabla^\{2\}\_\{z\}\\ell\(h^\{\\phi^\{\\prime\}\}\_\{S^\{\\setminus i\}\},z\_\{i\}\)\]\(z\_\{j\}\-z\_\{i\}\)\\rvert=\|\(zj−zi\)⊤​𝔼ϕ′∼𝒜⁡\(S∖i\)​\[∇z2​ℓ​\(hS∖iϕ′,zi\)​\(zj−zi\)\]\|\\displaystyle=\\lvert\(z\_\{j\}\-z\_\{i\}\)^\{\\top\}\\mathbb\{E\}\_\{\\phi^\{\\prime\}\\sim\\mathcal\{A\}\(S^\{\\setminus i\}\)\}\[\\nabla^\{2\}\_\{z\}\\ell\(h^\{\\phi^\{\\prime\}\}\_\{S^\{\\setminus i\}\},z\_\{i\}\)\(z\_\{j\}\-z\_\{i\}\)\]\\rvert\(65\)≤‖𝔼ϕ∼𝒜⁡\(S∖i\)​\[∇z2ℓ​\(hS∖iϕ′,zi\)​\(zj−zi\)\]‖​‖zj−zi‖\\displaystyle\\leq\\\|\\mathbb\{E\}\_\{\\phi\\sim\\mathcal\{A\}\(S^\{\\setminus i\}\)\}\[\\nabla^\{2\}\_\{z\}\\ell\(h^\{\\phi^\{\\prime\}\}\_\{S^\{\\setminus i\}\},z\_\{i\}\)\(z\_\{j\}\-z\_\{i\}\)\]\\\|\\\|z\_\{j\}\-z\_\{i\}\\\|\(66\)≤𝔼ϕ′∼𝒜⁡\(S∖i\)​\[‖∇z2ℓ​\(hS∖iϕ′,zi\)‖2→2\]​‖zj−zi‖22\\displaystyle\\leq\\mathbb\{E\}\_\{\\phi^\{\\prime\}\\sim\\mathcal\{A\}\(S^\{\\setminus i\}\)\}\[\\\|\\nabla^\{2\}\_\{z\}\\ell\(h^\{\\phi^\{\\prime\}\}\_\{S^\{\\setminus i\}\},z\_\{i\}\)\\\|\_\{2\\to 2\}\]\\\|z\_\{j\}\-z\_\{i\}\\\|\_\{2\}^\{2\}\(67\)≤𝔼ϕ′∼𝒜⁡\(S∖i\)​\[‖∇z2ℓ​\(hS∖iϕ′,zi\)‖F2\]​‖zj−zi‖22\\displaystyle\\leq\\mathbb\{E\}\_\{\\phi^\{\\prime\}\\sim\\mathcal\{A\}\(S^\{\\setminus i\}\)\}\[\\\|\\nabla^\{2\}\_\{z\}\\ell\(h^\{\\phi^\{\\prime\}\}\_\{S^\{\\setminus i\}\},z\_\{i\}\)\\\|^\{2\}\_\{F\}\]\\\|z\_\{j\}\-z\_\{i\}\\\|\_\{2\}^\{2\}\(68\)Plugging \([64](https://arxiv.org/html/2610.19042#A1.E64)\) and \([68](https://arxiv.org/html/2610.19042#A1.E68)\) yields \([29](https://arxiv.org/html/2610.19042#S4.E29)\)\. ∎

### A\.1Proof of Proposition[4\.5](https://arxiv.org/html/2610.19042#S4.Thmtheorem5)

For brevity we setz=zi,z′:=zjz=z\_\{i\},z^\{\\prime\}:=z\_\{j\}, andS′=S∖iS^\{\\prime\}=S^\{\\setminus i\}\. First, since the function classhϕh^\{\\phi\}contains linear denoisers, the performance ofhϕh^\{\\phi\}under𝒜⁡\(S\)\\mathcal\{A\}\(S\)is*better*than any linear denoiser\. However, the best linear denoiser attains an MSE dependent onT,f,gT,f,g\[[49](https://arxiv.org/html/2610.19042#bib.bib50)\]that we termA⁡\(T\)A\(T\)\. Thusℓ∗​\(S\)≤A⁡\(T\)\\ell^\{\*\}\(S\)\\leq A\(T\)\. We may now decompose the difference in assumption[6](https://arxiv.org/html/2610.19042#Thmassumption6)as follows by introducing the rest of the datasetS,S′S,S^\{\\prime\}:

ℓ⁡\(hϕ,z\)−ℓ⁡\(hϕ′,z′\)\\displaystyle\\ell\(h^\{\\phi\},z\)\-\\ell\(h^\{\\phi^\{\\prime\}\},z^\{\\prime\}\)=\|S​\|\(ℓ⁡\(hϕ,S\)−ℓ∗​\(S\)\)−\|​S′​\|\(ℓ⁡\(hϕ′,S′\)−ℓ∗​\(S′\)\)\+\|​S′\|​\(ℓ∗​\(S\)−ℓ∗​\(S′\)\)\\displaystyle=\|S\|\\left\(\\ell\(h^\{\\phi\},S\)\-\\ell^\{\*\}\(S\)\\right\)\-\|S^\{\\prime\}\|\\left\(\\ell\(h^\{\\phi^\{\\prime\}\},S^\{\\prime\}\)\-\\ell^\{\*\}\(S^\{\\prime\}\)\\right\)\+\|S^\{\\prime\}\|\\left\(\\ell^\{\*\}\(S\)\-\\ell^\{\*\}\(S^\{\\prime\}\)\\right\)\+\|S∖S′\|​ℓ∗​\(S\)\+∑zk∈S′,zk≠z′ℓ⁡\(hϕ′,zk\)−∑zk∈S,zk≠zℓ⁡\(hϕ,zk\)\\displaystyle\+\|S\\setminus S^\{\\prime\}\|\\ell^\{\*\}\(S\)\+\\sum\_\{z\_\{k\}\\in S^\{\\prime\},z\_\{k\}\\neq z^\{\\prime\}\}\\ell\(h^\{\\phi^\{\\prime\}\},z\_\{k\}\)\-\\sum\_\{z\_\{k\}\\in S,z\_\{k\}\\neq z\}\\ell\(h^\{\\phi\},z\_\{k\}\)≤\|S​\|\(ℓ⁡\(hϕ,S\)−ℓ∗​\(S\)\)−\|​S′​\|\(ℓ⁡\(hϕ′,S′\)−ℓ∗​\(S′\)\)\+\|​S′\|​\(ℓ∗​\(S\)−ℓ∗​\(S′\)\)\\displaystyle\\leq\{\|S\|\\left\(\\ell\(h^\{\\phi\},S\)\-\\ell^\{\*\}\(S\)\\right\)\}\-\{\|S^\{\\prime\}\|\\left\(\\ell\(h^\{\\phi^\{\\prime\}\},S^\{\\prime\}\)\-\\ell^\{\*\}\(S^\{\\prime\}\)\\right\)\}\+\|S^\{\\prime\}\|\{\\left\(\\ell^\{\*\}\(S\)\-\\ell^\{\*\}\(S^\{\\prime\}\)\\right\)\}\+\|S−S′\|​ℓ∗​\(S\)\.\\displaystyle\+\|S\-S^\{\\prime\}\|\\ell^\{\*\}\(S\)\.
Here the inequality on the third line follows because of the following calculation:

∑zk∈S′,zk≠z′ℓ⁡\(hϕ′,zk\)−∑zk∈S,zk≠zℓ⁡\(hϕ,zk\)\\displaystyle\\sum\_\{z\_\{k\}\\in S^\{\\prime\},z\_\{k\}\\neq z^\{\\prime\}\}\\ell\(h^\{\\phi^\{\\prime\}\},z\_\{k\}\)\-\\sum\_\{z\_\{k\}\\in S,z\_\{k\}\\neq z\}\\ell\(h^\{\\phi\},z\_\{k\}\)≤∑zk∈S′ℓ⁡\(hϕ′,zk\)−∑zk∈S′ℓ⁡\(hϕ,zk\)=\|S′\|\(ℓ⁡\(hϕ′,S′\)−ℓ⁡\(hϕ,S′\)\)≤0\.\\displaystyle\\leq\\sum\_\{z\_\{k\}\\in S^\{\\prime\}\}\\ell\(h^\{\\phi^\{\\prime\}\},z\_\{k\}\)\-\\sum\_\{z\_\{k\}\\in S^\{\\prime\}\}\\ell\(h^\{\\phi\},z\_\{k\}\)=\|S^\{\\prime\}\|\(\\ell\(h^\{\\phi^\{\\prime\}\},S^\{\\prime\}\)\-\\ell\(h^\{\\phi\},S^\{\\prime\}\)\)\\leq 0\.Using the PL\-inequality in assumption[8](https://arxiv.org/html/2610.19042#Thmassumption8), the boundℓ∗​\(S\)≤A⁡\(T\)\\ell^\{\*\}\(S\)\\leq A\(T\)and the fact thatl∗​\(S′\)≥0l^\{\*\}\(S^\{\\prime\}\)\\geq 0we get:

ℓ⁡\(hϕ,z\)−ℓ⁡\(hϕ′,z′\)≤nμ​εgrad\+n​A​\(T\)\.\\displaystyle\\ell\(h^\{\\phi\},z\)\-\\ell\(h^\{\\phi^\{\\prime\}\},z^\{\\prime\}\)\\leq\\frac\{n\}\{\\mu\}\\varepsilon\_\{\\mathrm\{grad\}\}\+nA\(T\)\.\(69\)

## Appendix BExperimental Details

Here we describe our experimental details\. Note that we also conducted analogous versions of our experiments for*flow\-matching*which we describe below\.

Flow matching\[[25](https://arxiv.org/html/2610.19042#bib.bib16),[27](https://arxiv.org/html/2610.19042#bib.bib17)\]learns a time\-dependent velocity fieldvθ​\(x,t\)v\_\{\\theta\}\(x,t\)whose flow transports a noise distributionp0p\_\{0\}to the data distributionp1p\_\{1\}\. GivenX0∼p0X\_\{0\}\\sim p\_\{0\}andX1∼p1X\_\{1\}\\sim p\_\{1\}, one forms the linear interpolantXt=\(1−t\)​X0\+t​X1X\_\{t\}=\(1\-t\)X\_\{0\}\+tX\_\{1\}fort∈\[0,1\]t\\in\[0,1\], whose conditional velocity isdd​t​Xt=X1−X0\\tfrac\{d\}\{dt\}X\_\{t\}=X\_\{1\}\-X\_\{0\}\. The field is trained by regressing onto this target,

ℓ⁡\(ϕ,x\)=𝔼t,X0,X1​\[‖vθ​\(Xt,t\)−\(X1−X0\)‖22∣X0=x\],\\displaystyle\\ell\(\\phi,x\)=\\mathbb\{E\}\_\{t,\\,X\_\{0\},\\,X\_\{1\}\}\\left\[\\,\\big\\\|v\_\{\\theta\}\(X\_\{t\},t\)\-\(X\_\{1\}\-X\_\{0\}\)\\big\\\|\_\{2\}^\{2\}\\,\\mid X\_\{0\}=x\\right\],\(70\)and the minimizer is the marginal velocityv⋆​\(x,t\)=𝔼⁡\[X1−X0∣Xt=x\]v^\{\\star\}\(x,t\)=\\mathbb\{E\}\[X\_\{1\}\-X\_\{0\}\\mid X\_\{t\}=x\]\. Samples are then generated by integrating the ODEx˙t=vθ​\(xt,t\)\\dot\{x\}\_\{t\}=v\_\{\\theta\}\(x\_\{t\},t\)fromt=0t=0tot=1t=1\.

We conduct numerical experiments on the Imagenet and CelebA datasets\. We use ImageNet\-1k\[[14](https://arxiv.org/html/2610.19042#bib.bib36),[37](https://arxiv.org/html/2610.19042#bib.bib37)\]and convert to RGB downscaled images of sized=64×64d=64\\times 64\. For Imagenet, we train onn∈\{128,256,512,1024,2048,4096,10240\}n\\in\\\{128,256,512,1024,2048,4096,10240\\\}\. We convert the CelebA\[[28](https://arxiv.org/html/2610.19042#bib.bib38)\]face images to grayscale and downsample them to sized=32×32d=32\\times 32\. We use a training set size ofn∈\{128,10240\}n\\in\\\{128,10240\\\}for CelebA, yielding a memorizing vs\. generalizing model\. All our score models have a U\-Net architecture\[[36](https://arxiv.org/html/2610.19042#bib.bib39)\]with four resolution levels, base channel widthW=64W=64, and channel multipliers 1, 2, 4, and 4 respectively\. Self\-attention is applied at the two lowest\-resolution levels, and each level has two residual blocks in the encoder and three in the decoder\. The models are conditioned on diffusion timettthrough the sinusoidal embedding\[[47](https://arxiv.org/html/2610.19042#bib.bib40)\]in every residual block, so a single network serves all noise levels\. We train the network in one of three ways: DDPM\[[21](https://arxiv.org/html/2610.19042#bib.bib12)\]with either noise prediction \(ϵ\\epsilon\-prediction\) or clean\-sample prediction \(x0x\_\{0\}\-prediction\), or flow matching\[[25](https://arxiv.org/html/2610.19042#bib.bib16)\]\. The DDPM noise prediction predicts the injected Gaussian noise at diffusion timettusing the Adam optimizer\[[24](https://arxiv.org/html/2610.19042#bib.bib41)\]with a fixed batch size of 128\. DDPM sample prediction takes the noisy imagesxtx\_\{t\}and timettand predicts the clean imagex0x\_\{0\}directly using lossMSE⁡\(x0^​\(xt,t\),x0\)\\mathrm\{MSE\}\(\\hat\{x\_\{0\}\}\(x\_\{t\},t\),x\_\{0\}\)\. DDPM models are sampled using DDIM with 100 steps and stochasticity parameterη=0\\eta=0, starting fromxT∼𝒩⁡\(0,I\)x\_\{T\}\\sim\\mathcal\{N\}\(0,I\)\. For the noise prediction models, the predicted noiseϵ^\\hat\{\\epsilon\}is converted to a clean image estimatex0^=\(xt−\(1−α¯t\)​ϵ^\)/α¯t\\hat\{x\_\{0\}\}=\(x\_\{t\}\-\\sqrt\{\(1\-\\bar\{\\alpha\}\_\{t\}\)\}\\hat\{\\epsilon\}\)/\\sqrt\{\\bar\{\\alpha\}\_\{t\}\}before stepping to the next time\. The sample prediction models use the same DDIM sampler with the network output taken as the clean image estimatex0^\\hat\{x\_\{0\}\}directly\. The flow matching networks predict the velocityu=x0−ϵu=x\_\{0\}\-\\epsilonalong the linear pathxt=\(1−τ\)​ϵ\+τ​x0x\_\{t\}=\(1\-\\tau\)\\epsilon\+\\tau x\_\{0\}, whereϵ∼𝒩⁡\(0,I\)\\epsilon\\sim\\mathcal\{N\}\(0,I\)is the noise,x0x\_\{0\}is the training image, andτ\\tauis the normalized time\. Training minimizesMSE​\(v^​\(xt,t\),u\)\\mathrm\{MSE\}\(\\hat\{v\}\(x\_\{t\},t\),u\), and sampling integratesd​xd​τ=v^\\tfrac\{\\mathrm\{d\}x\}\{\\mathrm\{d\}\\tau\}=\\hat\{v\}fromτ=0\\tau=0\(noise\) toτ=1\\tau=1\(data\) with 100 Euler steps\.

## Appendix CGenerated Samples

Figures[5](https://arxiv.org/html/2610.19042#A3.F5)\-[12](https://arxiv.org/html/2610.19042#A3.F12)shows generated samples at the pre\-memorization, beginning of memorization, and full memorization stages with their nearest training neighbor for DDPM noise prediction and flow matching on CelebA withn∈\{128,10240\}n\\in\\\{128,10240\\\}as well as the leave\-10\-out models for Imagenet withn∈\{128,1024\}n\\in\\\{128,1024\\\}\.

![Refer to caption](https://arxiv.org/html/2610.19042v1/figures/nn_samples_celeba_ddpm_128.png)Figure 5:Generated samples vs\. nearest training image for CelebA \(n=128\) DDPM: generated samples at the pre\-memorization, start of memorization, and full memorization stages alongside their nearest neighbor in the training set\.![Refer to caption](https://arxiv.org/html/2610.19042v1/figures/nn_samples_celeba_fm_128.png)Figure 6:Generated samples vs\. nearest training image for CelebA \(n=128\) flow matching: generated samples at the pre\-memorization, start of memorization, and full memorization stages alongside their nearest neighbor in the training set\.![Refer to caption](https://arxiv.org/html/2610.19042v1/figures/nn_samples_celeba_ddpm_10240.png)Figure 7:Generated samples vs\. nearest training image for CelebA \(n=10240\) DDPM: generated samples at the pre\-memorization, start of memorization, and full memorization stages alongside their nearest neighbor in the training set\.![Refer to caption](https://arxiv.org/html/2610.19042v1/figures/nn_samples_celeba_fm_10240.png)Figure 8:Generated samples vs\. nearest training image for CelebA \(n=10240\) flow matching: generated samples at the pre\-memorization, start of memorization, and full memorization stages alongside their nearest neighbor in the training set\.![Refer to caption](https://arxiv.org/html/2610.19042v1/figures/nn_samples_imagenet_ddpm_128_leave10out.png)Figure 9:Generated samples vs\. nearest training image for Imagenet \(n=128\) DDPM leave\-10\-out model: generated samples at the pre\-memorization, start of memorization, and full memorization stages alongside their nearest neighbor in the training set\.![Refer to caption](https://arxiv.org/html/2610.19042v1/figures/nn_samples_imagenet_ddpm_1024_leave10out.png)Figure 10:Generated samples vs\. nearest training image for Imagenet \(n=1024\) DDPM leave\-10\-out model: generated samples at the pre\-memorization, start of memorization, and full memorization stages alongside their nearest neighbor in the training set\.![Refer to caption](https://arxiv.org/html/2610.19042v1/figures/nn_samples_imagenet_fm_128_leave10out.png)Figure 11:Generated samples vs\. nearest training image for Imagenet \(n=128\) flow matching leave\-10\-out model: generated samples at the pre\-memorization, start of memorization, and full memorization stages alongside their nearest neighbor in the training set\.![Refer to caption](https://arxiv.org/html/2610.19042v1/figures/nn_samples_imagenet_fm_1024_leave10out.png)Figure 12:Generated samples vs\. nearest training image for Imagenet \(n=1024\) flow matching leave\-10\-out model: generated samples at the pre\-memorization, start of memorization, and full memorization stages alongside their nearest neighbor in the training set\.Figures[13](https://arxiv.org/html/2610.19042#A3.F13)–[20](https://arxiv.org/html/2610.19042#A3.F20)show generated samples and their nearest training images for models trained on allnnimages: DDPM noise prediction withn∈\{128,256,512,1024,2048,4096\}n\\in\\\{128,256,512,1024,2048,4096\\\}and flow matching withn∈\{128,1024\}n\\in\\\{128,1024\\\}\. Each row contains five generated–training image pairs\. All memorization fractions are measured over10,00010\{,\}000generations\.

For models with sustained memorization, pre\_mem denotes the last evaluated checkpoint withfmem=0f\_\{\\mathrm\{mem\}\}=0before onset, and start\_mem denotes the first checkpoint after whichfmemf\_\{\\mathrm\{mem\}\}stays above zero\. Full memorization \(full\_mem\) denotes the first evaluated checkpoint withfmem≥95%f\_\{\\mathrm\{mem\}\}\\geq 95\\%\. For DDPM withn=512n=512andn=1024n=1024, which never reach this threshold, peak\_mem denotes the highest observed memorization fraction\. For DDPM withn=4096n=4096, the two rows instead show a zero\-copy checkpoint followed by an isolated copy event\.

![Refer to caption](https://arxiv.org/html/2610.19042v1/figures/boonam/images/stages_n128.png)Figure 13:Generated samples vs\. nearest training image for ImageNet \(n=128n=128\) DDPM\.pre\_mem: step2,0002\{,\}000,fmem=0f\_\{\\mathrm\{mem\}\}=0, five random samples\. start\_mem: step3,0003\{,\}000,fmem=0\.10%f\_\{\\mathrm\{mem\}\}=0\.10\\%\(10 copies of10410^\{4\}\), five randomly selected copies\. full\_mem: step150,000150\{,\}000,fmem=99\.8%f\_\{\\mathrm\{mem\}\}=99\.8\\%, five randomly selected copies\.![Refer to caption](https://arxiv.org/html/2610.19042v1/figures/boonam/images/stages_n256.png)Figure 14:Generated samples vs\. nearest training image for ImageNet \(n=256n=256\) DDPM\.pre\_mem: step3,0003\{,\}000,fmem=0f\_\{\\mathrm\{mem\}\}=0, five random samples\. start\_mem: step5,0005\{,\}000,fmem=0\.04%f\_\{\\mathrm\{mem\}\}=0\.04\\%\(4 copies of10410^\{4\}\); the four copies, followed in the rightmost pair by the non\-copy with the smallest gap ratio\. full\_mem: step200,000200\{,\}000,fmem=95\.5%f\_\{\\mathrm\{mem\}\}=95\.5\\%, five randomly selected copies\.![Refer to caption](https://arxiv.org/html/2610.19042v1/figures/boonam/images/stages_n512.png)Figure 15:Generated samples vs\. nearest training image for ImageNet \(n=512n=512\) DDPM\.pre\_mem: step5,0005\{,\}000,fmem=0f\_\{\\mathrm\{mem\}\}=0, five random samples\. start\_mem: step7,0007\{,\}000,fmem=0\.03%f\_\{\\mathrm\{mem\}\}=0\.03\\%\(3 copies of10410^\{4\}\); the three copies, followed in the two rightmost pairs by the non\-copies with the smallest gap ratios\. peak\_mem: step2,000,0002\{,\}000\{,\}000,fmem=85\.0%f\_\{\\mathrm\{mem\}\}=85\.0\\%, five randomly selected copies\. This model never reaches the full\-memorization threshold\.![Refer to caption](https://arxiv.org/html/2610.19042v1/figures/boonam/images/stages_n1024.png)Figure 16:Generated samples vs\. nearest training image for ImageNet \(n=1024n=1024\) DDPM\.pre\_mem: step15,00015\{,\}000,fmem=0f\_\{\\mathrm\{mem\}\}=0, five random samples\. start\_mem: step20,00020\{,\}000,fmem=0\.10%f\_\{\\mathrm\{mem\}\}=0\.10\\%\(10 copies of10410^\{4\}\), five randomly selected copies\. peak\_mem: step2,000,0002\{,\}000\{,\}000,fmem=19\.9%f\_\{\\mathrm\{mem\}\}=19\.9\\%, five randomly selected copies\. This model never reaches the full\-memorization threshold\.![Refer to caption](https://arxiv.org/html/2610.19042v1/figures/boonam/images/stages_n2048.png)Figure 17:Generated samples vs\. nearest training image for ImageNet \(n=2048n=2048\) DDPM\.pre\_mem: step70,00070\{,\}000,fmem=0f\_\{\\mathrm\{mem\}\}=0, five random samples\. start\_mem: step100,000100\{,\}000,fmem=0\.02%f\_\{\\mathrm\{mem\}\}=0\.02\\%\(2 copies of10410^\{4\}\); the two copies, followed in the three rightmost pairs by the non\-copies with the smallest gap ratios\. There is no full\_mem row:fmemf\_\{\\mathrm\{mem\}\}peaks at0\.57%0\.57\\%\.![Refer to caption](https://arxiv.org/html/2610.19042v1/figures/boonam/images/stages_n4096.png)Figure 18:Generated samples vs\. nearest training image for ImageNet \(n=4096n=4096\) DDPM\.pre\_mem: step300,000300\{,\}000,fmem=0f\_\{\\mathrm\{mem\}\}=0, five random samples\. start\_mem: step500,000500\{,\}000, an isolated checkpoint withfmem=0\.03%f\_\{\\mathrm\{mem\}\}=0\.03\\%\(3 copies of10410^\{4\}\); the three copies, followed in the two rightmost pairs by the non\-copies with the smallest gap ratios\. Here start\_mem labels the isolated copy event\. No sustained memorization is observed, and there is no full\_mem row\.![Refer to caption](https://arxiv.org/html/2610.19042v1/figures/boonam/images/stages_fm_n128.png)Figure 19:Generated samples vs\. nearest training image for ImageNet \(n=128n=128\) flow matching\.pre\_mem: step1,0001\{,\}000,fmem=0f\_\{\\mathrm\{mem\}\}=0, five random samples\. start\_mem: step2,0002\{,\}000,fmem=0\.10%f\_\{\\mathrm\{mem\}\}=0\.10\\%\(10 copies of10410^\{4\}\), five randomly selected copies\. full\_mem: step10,00010\{,\}000,fmem=95\.7%f\_\{\\mathrm\{mem\}\}=95\.7\\%, five randomly selected copies\.![Refer to caption](https://arxiv.org/html/2610.19042v1/figures/boonam/images/stages_fm_n1024.png)Figure 20:Generated samples vs\. nearest training image for ImageNet \(n=1024n=1024\) flow matching\.pre\_mem: step10,00010\{,\}000,fmem=0f\_\{\\mathrm\{mem\}\}=0, five random samples\. start\_mem: step15,00015\{,\}000,fmem=0\.46%f\_\{\\mathrm\{mem\}\}=0\.46\\%\(46 copies of10410^\{4\}\), five randomly selected copies\. full\_mem: step500,000500\{,\}000,fmem=96\.2%f\_\{\\mathrm\{mem\}\}=96\.2\\%, five randomly selected copies\.
## Appendix DAdditional Experiments

We provide additional experiments on Imagenet and CelebA\. Figures[21](https://arxiv.org/html/2610.19042#A4.F21)\-[25](https://arxiv.org/html/2610.19042#A4.F25)plot the average training input loss curvature over diffusion time and fraction of generated samples that are considered memorized with memorization thresholdk=1/3k=1/3vs\. training step for DDPM noise prediction and flow matching for the datasets CelebA with training set sizen∈\{128,1024\}n\\in\\\{128,1024\\\}and Imagenet with training set sizen∈\{128,1024,10240\}n\\in\\\{128,1024,10240\\\}, respectively\. DDPM and flow matching on CelebA withn=128n=128is a memorizing model, since fraction memorization starts increasing rapidly to100%100\\%around training stepτ=103\\tau=10^\{3\}, whereas DDPM and flow matching on CelebA withn=10240n=10240is a generalizing model with fraction memorization increasing much later around training stepτ=105\\tau=10^\{5\}and never reaching full memorization of100%100\\%before training ends\. We observe that the average input loss curvature increases more drastically for the memorizing model as compared to the generalizing model\. A similar trend can be seen for the DDPM and flow matching models for Imagenet as training sizennincreases\.

For the flow matching model on CelebA with training sizen=128n=128, Figure[26](https://arxiv.org/html/2610.19042#A4.F26)shows the average training and validation input loss curvatures over diffusion time vs\. training step, as well as the relative training and validation curvature gaps, which is calculated as\|𝔼x∼train​\[‖∇x2ℓ​\(ϕ⁡\(τ\),x\)‖F2\]−𝔼x∼val\.​\[‖∇x2ℓ​\(ϕ⁡\(τ\),x\)‖F2\]\|/𝔼x∼train​\[‖∇x2ℓ​\(ϕ⁡\(τ\),x\)‖F2\]\|\\mathbb\{E\}\_\{x\\sim\\text\{train\}\}\[\\\|\\nabla^\{2\}\_\{x\}\\ell\(\\phi\(\\tau\),x\)\\\|^\{2\}\_\{F\}\]\-\\mathbb\{E\}\_\{x\\sim\\text\{val\.\}\}\[\\\|\\nabla^\{2\}\_\{x\}\\ell\(\\phi\(\\tau\),x\)\\\|^\{2\}\_\{F\}\]\|/\\mathbb\{E\}\_\{x\\sim\\text\{train\}\}\[\\\|\\nabla^\{2\}\_\{x\}\\ell\(\\phi\(\\tau\),x\)\\\|^\{2\}\_\{F\}\]and\|𝔼x∼train​\[‖∇x2ℓ​\(ϕ⁡\(τ\),x\)‖F2\]−𝔼x∼val\.​\[‖∇x2ℓ​\(ϕ⁡\(τ\),x\)‖F2\]\|/𝔼x∼val​\[‖∇x2ℓ​\(ϕ⁡\(τ\),x\)‖F2\]\|\\mathbb\{E\}\_\{x\\sim\\text\{train\}\}\[\\\|\\nabla^\{2\}\_\{x\}\\ell\(\\phi\(\\tau\),x\)\\\|^\{2\}\_\{F\}\]\-\\mathbb\{E\}\_\{x\\sim\\text\{val\.\}\}\[\\\|\\nabla^\{2\}\_\{x\}\\ell\(\\phi\(\\tau\),x\)\\\|^\{2\}\_\{F\}\]\|/\\mathbb\{E\}\_\{x\\sim\\text\{val\}\}\[\\\|\\nabla^\{2\}\_\{x\}\\ell\(\\phi\(\\tau\),x\)\\\|^\{2\}\_\{F\}\], vs\. training step\. We observe that the training curvature separates from the validation curvature right before training stepτ=103\\tau=10^\{3\}, which is around where memorization begins, as shown in Figure[21](https://arxiv.org/html/2610.19042#A4.F21)\.

Figure[27](https://arxiv.org/html/2610.19042#A4.F27)shows the fraction memorization score for DDPM noise prediction and flow matching on Imagenet\(n=10240\)\(n=10240\)with different values for the memorization thresholdk∈\{1/3,0\.8,0\.9\}k\\in\\\{1/3,0\.8,0\.9\\\}\. There is no trend in the nearest\-neighbor gap ratios for DDPM even whenkkis increased\. However, for flow matching, whenkkis large, the nearest\-neighbor gap ratios increase as training continues\. Nonetheless, this pattern in the gap ratios does not indicate that this model is a memorizing model, since fork=1/3k=1/3, the fraction memorized is approximately0%0\\%for all training steps\.

Figure[28](https://arxiv.org/html/2610.19042#A4.F28)shows the average training curvature over diffusion time and fraction memorized withk=1/3k=1/3vs\. training step for DDPM sample prediction on Imagenet\(n=10240\)\(n=10240\)as well as the fraction memorization withk∈\{1/3,0\.8,0\.9\}k\\in\\\{1/3,0\.8,0\.9\\\}\. This model is a generalizing model, since fork=1/3k=1/3, approximately0%0\\%of the generated samples are considered memorized for all training steps, even though for large values ofkk, the nearest\-neighbor gap ratios increase as training steps increase\. We also observe that the average input loss curvature barely changes across training steps, validating the connection between memorization and curvature\.

In Figure[29](https://arxiv.org/html/2610.19042#A4.F29), the average training input loss curvature over diffusion time vs\. training step is plotted for DDPM noise prediction and flow matching on the CelebA and Imagenet datasets with training set sizen=10240n=10240\. CelebA has larger curvature values than Imagenet for both models, which correspond to higher memorization scores as seen in Figures[22](https://arxiv.org/html/2610.19042#A4.F22)and[25](https://arxiv.org/html/2610.19042#A4.F25)\. In particular, DDPM and flow matching for CelebA\(n=10240\)\(n=10240\)have fraction memorization scores of around23%23\\%and40%40\\%, respectively, at the end of training, whereas DDPM and flow matching for Imagenet\(n=10240\)\(n=10240\)have fraction memorization scores of around0%0\\%and0\.1%0\.1\\%, respectively, at the end of training\.

Figure[30](https://arxiv.org/html/2610.19042#A4.F30)compares two DDPM noise prediction models on Imagenet, one that is trained on a training set of sizen=1024n=1024and another that is trained on the same training set with the top1010memorized samples from the fulln=1024n=1024model removed\. The figure plots fraction memorization and average training curvature over diffusion time vs\. training step for both the full and leave\-10\-out models\. The leave\-10\-out model has a lower memorization score at the end of training and lower input loss curvature for all training steps, indicating that it is worthwhile to remove the top memorized samples from a model to improve generalization\. Figure[31](https://arxiv.org/html/2610.19042#A4.F31)shows similar results for flow matching\. We repeat these experiments for Imagenetn=128n=128in Figures[32](https://arxiv.org/html/2610.19042#A4.F32)and[33](https://arxiv.org/html/2610.19042#A4.F33)\. Figures[34](https://arxiv.org/html/2610.19042#A4.F34)and[35](https://arxiv.org/html/2610.19042#A4.F35)plot the difference between the per sample losses of the full flow matching models and the leave\-10\-out models vs\. training step, as well as the per sample curvatures of the leave\-10\-out models, for each of the top1010left\-out samples\.

![Refer to caption](https://arxiv.org/html/2610.19042v1/figures/celeba_ddpm_128_mem_curv.png)\(a\)
![Refer to caption](https://arxiv.org/html/2610.19042v1/figures/celeba_fm_128_mem_curv.png)\(b\)

Figure 21:Curvature and memorization for CelebA \(n=128\): average input loss curvature over diffusion time and training set and fraction memorized \(%\) with memorization thresholdk=1/3k=1/3vs\. training step for \(a\) DDPM noise prediction and \(b\) flow matching\.![Refer to caption](https://arxiv.org/html/2610.19042v1/figures/celeba_ddpm_mem_curv.png)\(a\)
![Refer to caption](https://arxiv.org/html/2610.19042v1/figures/celeba_fm_mem_curv.png)\(b\)

Figure 22:Curvature and memorization for CelebA \(n=10240\): average input loss curvature over diffusion time and training set and fraction memorized \(%\) with memorization thresholdk=1/3k=1/3vs\. training step for \(a\) DDPM noise prediction and \(b\) flow matching\.![Refer to caption](https://arxiv.org/html/2610.19042v1/figures/ddpm_128_mem_curv.png)\(a\)
![Refer to caption](https://arxiv.org/html/2610.19042v1/figures/fm_128_mem_curv.png)\(b\)

Figure 23:Curvature and memorization for Imagenet \(n=128\): average input loss curvature over diffusion time and training set and fraction memorized \(%\) with memorization thresholdk=1/3k=1/3vs\. training step for \(a\) DDPM noise prediction and \(b\) flow matching\.![Refer to caption](https://arxiv.org/html/2610.19042v1/figures/ddpm_1024_mem_curv.png)\(a\)
![Refer to caption](https://arxiv.org/html/2610.19042v1/figures/fm_1024_mem_curv.png)\(b\)

Figure 24:Curvature and memorization for Imagenet \(n=1024\): average input loss curvature over diffusion time and training set and fraction memorized \(%\) with memorization thresholdk=1/3k=1/3vs\. training step for \(a\) DDPM noise prediction and \(b\) flow matching\.![Refer to caption](https://arxiv.org/html/2610.19042v1/figures/ddpm_noise_pred_mem_curv.png)\(a\)
![Refer to caption](https://arxiv.org/html/2610.19042v1/figures/fm_mem_curv.png)\(b\)

Figure 25:Curvature and memorization for Imagenet \(n=10240\): average input loss curvature over diffusion time and training set and fraction memorized \(%\) with memorization thresholdk=1/3k=1/3vs\. training step for \(a\) DDPM noise prediction and \(b\) flow matching\.![Refer to caption](https://arxiv.org/html/2610.19042v1/figures/celeba_fm_128_train_val_curv.png)Figure 26:Training vs\. validation curvature for CelebA \(n=128\) flow matching: \(left\) average input loss curvature over diffusion time and training \(blue\) / validation \(orange\) set, \(right\) relative training/validation curvature gap calculated as\|𝔼x∼train​\[‖∇x2ℓ​\(ϕ⁡\(τ\),x\)‖F2\]−𝔼x∼val\.​\[‖∇x2ℓ​\(ϕ⁡\(τ\),x\)‖F2\]\|/𝔼x∼train​\[‖∇x2ℓ​\(ϕ⁡\(τ\),x\)‖F2\]\|\\mathbb\{E\}\_\{x\\sim\\text\{train\}\}\[\\\|\\nabla^\{2\}\_\{x\}\\ell\(\\phi\(\\tau\),x\)\\\|^\{2\}\_\{F\}\]\-\\mathbb\{E\}\_\{x\\sim\\text\{val\.\}\}\[\\\|\\nabla^\{2\}\_\{x\}\\ell\(\\phi\(\\tau\),x\)\\\|^\{2\}\_\{F\}\]\|/\\mathbb\{E\}\_\{x\\sim\\text\{train\}\}\[\\\|\\nabla^\{2\}\_\{x\}\\ell\(\\phi\(\\tau\),x\)\\\|^\{2\}\_\{F\}\]\(blue\) and\|𝔼x∼train​\[‖∇x2ℓ​\(ϕ⁡\(τ\),x\)‖F2\]−𝔼x∼val\.​\[‖∇x2ℓ​\(ϕ⁡\(τ\),x\)‖F2\]\|/𝔼x∼val​\[‖∇x2ℓ​\(ϕ⁡\(τ\),x\)‖F2\]\|\\mathbb\{E\}\_\{x\\sim\\text\{train\}\}\[\\\|\\nabla^\{2\}\_\{x\}\\ell\(\\phi\(\\tau\),x\)\\\|^\{2\}\_\{F\}\]\-\\mathbb\{E\}\_\{x\\sim\\text\{val\.\}\}\[\\\|\\nabla^\{2\}\_\{x\}\\ell\(\\phi\(\\tau\),x\)\\\|^\{2\}\_\{F\}\]\|/\\mathbb\{E\}\_\{x\\sim\\text\{val\}\}\[\\\|\\nabla^\{2\}\_\{x\}\\ell\(\\phi\(\\tau\),x\)\\\|^\{2\}\_\{F\}\]\(orange\)\.![Refer to caption](https://arxiv.org/html/2610.19042v1/figures/ddpm_noise_pred_mem.png)\(a\)
![Refer to caption](https://arxiv.org/html/2610.19042v1/figures/fm_mem.png)\(b\)

Figure 27:Memorization for Imagenet \(n=10240\): fraction memorized \(%\) with memorization thresholdk∈\{1/3,0\.8,0\.9\}k\\in\\\{1/3,0\.8,0\.9\\\}vs\. training step for \(a\) DDPM noise prediction and \(b\) flow matching\.![Refer to caption](https://arxiv.org/html/2610.19042v1/figures/ddpm_sample_pred_mem_curv.png)\(a\)
![Refer to caption](https://arxiv.org/html/2610.19042v1/figures/ddpm_sample_pred_mem.png)\(b\)

Figure 28:Curvature and memorization for Imagenet \(n=10240\) DDPM Sample Prediction: \(a\) average input loss curvature over diffusion time and training set and fraction memorized \(%\) with memorization thresholdk=1/3k=1/3vs\. training step, and \(b\) fraction memorized \(%\) with memorization thresholdk∈\{1/3,0\.8,0\.9\}k\\in\\\{1/3,0\.8,0\.9\\\}vs\. training step\.![Refer to caption](https://arxiv.org/html/2610.19042v1/figures/ddpm_celeba_imagenet_curv.png)\(a\)
![Refer to caption](https://arxiv.org/html/2610.19042v1/figures/fm_celeba_imagenet_curv.png)\(b\)

Figure 29:CelebA vs\. Imagenet \(n=10240\) Curvature: average input loss curvature over diffusion time and training set vs\. training step for \(a\) DDPM noise prediction and \(b\) flow matching\.![Refer to caption](https://arxiv.org/html/2610.19042v1/figures/ddpm_1024_mem.png)\(a\)
![Refer to caption](https://arxiv.org/html/2610.19042v1/figures/ddpm_1024_avg_curv.png)\(b\)

Figure 30:Memorization and curvature for Imagenet \(n=1024\) DDPM noise prediction \- full vs\. leave\-10\-out training: \(a\) fraction memorized \(%\) with memorization thresholdk=1/3k=1/3vs\. training step and \(b\) average input loss curvature over diffusion time and training set vs\. training step for fulln=1024n=1024training set and leave\-10\-out training set with the top1010memorized training samples removed\.![Refer to caption](https://arxiv.org/html/2610.19042v1/figures/fm_1024_mem.png)\(a\)
![Refer to caption](https://arxiv.org/html/2610.19042v1/figures/fm_1024_avg_curv.png)\(b\)

Figure 31:Memorization and curvature for Imagenet \(n=1024\) flow matching \- full vs\. leave\-10\-out training: \(a\) fraction memorized \(%\) with memorization thresholdk=1/3k=1/3vs\. training step and \(b\) average input loss curvature over diffusion time and training set vs\. training step for fulln=1024n=1024training set and leave\-10\-out training set with the top1010memorized training samples removed\.![Refer to caption](https://arxiv.org/html/2610.19042v1/figures/ddpm_128_mem.png)\(a\)
![Refer to caption](https://arxiv.org/html/2610.19042v1/figures/ddpm_128_avg_curv.png)\(b\)

Figure 32:Memorization and curvature for Imagenet \(n=128\) DDPM noise prediction \- full vs\. leave\-10\-out training: \(a\) fraction memorized \(%\) with memorization thresholdk=1/3k=1/3vs\. training step and \(b\) average input loss curvature over diffusion time and training set vs\. training step for fulln=128n=128training set and leave\-10\-out training set with the top1010memorized training samples removed\.![Refer to caption](https://arxiv.org/html/2610.19042v1/figures/fm_128_mem.png)\(a\)
![Refer to caption](https://arxiv.org/html/2610.19042v1/figures/fm_128_avg_curv.png)\(b\)

Figure 33:Memorization and curvature for Imagenet \(n=128\) flow matching \- full vs\. leave\-10\-out training: \(a\) fraction memorized \(%\) with memorization thresholdk=1/3k=1/3vs\. training step and \(b\) average input loss curvature over diffusion time and training set vs\. training step for fulln=128n=128training set and leave\-10\-out training set with the top1010memorized training samples removed\.![Refer to caption](https://arxiv.org/html/2610.19042v1/figures/loss_delta_vs_curvature_over_training_fm_128.png)Figure 34:Loss difference and curvature for Imagenet \(n=128\) flow matching \- leave\-10\-out\-training: \(left\) difference between per sample loss of full model and leave\-10\-out model vs\. training step for each of the 10 left\-out training samples, \(right\) per sample curvature of the leave\-10\-out model vs\. training step for each of the 10 left\-out training samples\.![Refer to caption](https://arxiv.org/html/2610.19042v1/figures/loss_delta_vs_curvature_over_training_fm_1024.png)Figure 35:Loss difference and curvature for Imagenet \(n=1024\) flow matching \- leave\-10\-out\-training: \(left\) difference between per sample loss of full model and leave\-10\-out model vs\. training step for each of the 10 left\-out training samples, \(right\) per sample curvature of the leave\-10\-out model vs\. training step for each of the 10 left\-out training samples\.![Refer to caption](https://arxiv.org/html/2610.19042v1/figures/fm_128_mem.png)\(a\)
![Refer to caption](https://arxiv.org/html/2610.19042v1/figures/fm_128_avg_curv.png)\(b\)

Figure 36:Memorization and curvature for Imagenet \(n=128\) flow matching \- full vs\. leave\-10\-out training: \(a\) fraction memorized \(%\) with memorization thresholdk=1/3k=1/3vs\. training step and \(b\) average input loss curvature over diffusion time and training set vs\. training step for fulln=128n=128training set and leave\-10\-out training set with the top1010memorized training samples removed\.
## Appendix EAdditional ImageNet DDPM experiments

### E\.1Experimental Details

We use64×6464\\times 64RGB ImageNet images and nested training sets of sizen∈\{128,256,512,1024,2048,4096\}n\\in\\\{128,256,512,1024,2048,4096\\\}, taking the firstnnimages in a fixed dataset ordering\. We evaluate all models on the same disjoint set of1,0241\{,\}024held\-out images\. Each training size corresponds to one DDPM trained with the noise\-prediction objective\. The U\-Net has base width6464and channel multipliers\(1,2,4,4\)\(1,2,4,4\)\. We use Adam with learning rate10−410^\{\-4\}and a linear noise schedule with1,0001\{,\}000diffusion steps\. We train all models for10610^\{6\}updates and continue training those withn∈\{512,1024,2048,4096\}n\\in\\\{512,1024,2048,4096\\\}for another10610^\{6\}updates to examine their behavior over longer training\.

For the curvature, gradient, and loss measurements, we convert the predicted noise to a clean\-image estimatex^0\\hat\{x\}\_\{0\}and evaluate the mean squared reconstruction error

ℓτ​\(x,t,ϵ\)=1d​‖x^0​\(xt,t,ϕ⁡\(τ\)\)−x‖22,d=3⋅642,\\ell\_\{\\tau\}\(x;t,\\epsilon\)=\\frac\{1\}\{d\}\\left\\\|\\hat\{x\}\_\{0\}\(x\_\{t\},t;\\phi\(\\tau\)\)\-x\\right\\\|\_\{2\}^\{2\},\\qquad d=3\\cdot 64^\{2\},whereddis the number of scalar image values across the three RGB channels, so1/d1/daverages the squared error over all channels and pixels\. Herexxis the clean image after the input normalization used for training, andxtx\_\{t\}is its noisy version at diffusion steptt, obtained using Gaussian noiseϵ\\epsilon\. The training checkpoint is indexed byτ\\tau, andϕ⁡\(τ\)\\phi\(\\tau\)denotes the model parameters at that checkpoint\. Derivatives are taken with respect toxx, including its contribution throughxtx\_\{t\}\. We estimate the input\-Hessian trace using3232Monte Carlo draws per image\. Each draw samplesttuniformly from\{1,…,999\}\\\{1,\\ldots,999\\\}, Gaussian forward noiseϵ\\epsilon, and a Rademacher vectorvv, and evaluatesv⊤​∇x2ℓτ​\(x,t,ϵ\)​vv^\{\\top\}\\nabla\_\{x\}^\{2\}\\ell\_\{\\tau\}\(x;t,\\epsilon\)v\. The gradient curves average‖∇xℓτ​\(x,t,ϵ\)‖2\\\|\\nabla\_\{x\}\\ell\_\{\\tau\}\(x;t,\\epsilon\)\\\|\_\{2\}over3232draws, and the loss curves averageℓτ\\ell\_\{\\tau\}over6464draws\. Training means use allnnimages; validation means use the fixed held\-out set\. These measurements do not require generating samples from the model\.

At each checkpoint used to evaluate memorization, we generate10,00010\{,\}000images using100100deterministic DDIM steps \(η=0\\eta=0\)\. A generated image is classified as a copy when the ratio of its nearest to second\-nearest training\-image distances is below1/31/3\. We use Euclidean distances in the normalized image space used for training\. The copy countF⁡\(i,τ\)F\(i,\\tau\)is the number of generated copies assigned to training imageiiat checkpointτ\\tau, andfmem​\(τ\)f\_\{\\mathrm\{mem\}\}\(\\tau\)is the fraction of generations classified as copies\.

### E\.2Supporting Figures

Figure[37](https://arxiv.org/html/2610.19042#A5.F37)compares the mean curvature of training images copied at a checkpoint with that of the remaining training images\. The copied group generally has higher curvature, although the separation varies with training size and checkpoint\. Figure[38](https://arxiv.org/html/2610.19042#A5.F38)compares the maximum copy count with the relative train–validation curvature gap\. Smaller training sets tend to exhibit both larger maximum copy counts and larger curvature gaps\.

![Refer to caption](https://arxiv.org/html/2610.19042v1/figures/boonam/copied_vs_not_ddpm.png)Figure 37:Curvature of copied and non\-copied training images\.Each panel shows one ImageNet DDPM training size\. Orange and blue curves average the input\-Hessian trace over images withF⁡\(i,τ\)\>0F\(i,\\tau\)\>0andF⁡\(i,τ\)=0F\(i,\\tau\)=0, respectively\. Group membership is determined separately at each checkpoint; “never copied” in the legend refers only to that checkpoint’s generations\. Shading shows one standard error across images in the group, and orange numbers give the copied\-group size\. A group is plotted only when it contains at least ten images\. Dashed vertical lines mark the first evaluated checkpoint withfmem\>1%f\_\{\\mathrm\{mem\}\}\>1\\%, used as a reference threshold\.![Refer to caption](https://arxiv.org/html/2610.19042v1/figures/boonam/maxfreq_vs_relcurverror.png)Figure 38:Maximum copy count and relative curvature gap\.Each point represents one evaluated checkpoint of an ImageNet DDPM; colors indicate training\-set size\. The horizontal coordinate ismaxi⁡F⁡\(i,τ\)\\max\_\{i\}F\(i,\\tau\)among10,00010\{,\}000generations\. The vertical coordinate is\|Ctrain​\(τ\)−Cval​\(τ\)\|/\|Ctrain​\(τ\)\|\|C\_\{\\mathrm\{train\}\}\(\\tau\)\-C\_\{\\mathrm\{val\}\}\(\\tau\)\|/\|C\_\{\\mathrm\{train\}\}\(\\tau\)\|, where the two means use all training images and the same1,0241\{,\}024held\-out images, respectively\. Curvature is the input\-Hessian trace estimated as described above\. The horizontal scale is logarithmic above one and linear near zero to retain checkpoints with no observed copies\.