Uniform Stability and Generalization Error of GD and SGD on Fixed-Point Parameters

# Uniform Stability and Generalization Error of GD and SGD on Fixed-Point Parameters
Source: [https://arxiv.org/html/2606.06934](https://arxiv.org/html/2606.06934)
Jonghyun Shin Department of Artificial Intelligence Korea University Seoul, Korea uenjgieonj5448@korea\.ac\.kr &Sejun Park Department of Artificial Intelligence Korea University Seoul, Korea sejun\.park000@gmail\.com

###### Abstract

We analyze generalization error, uniform stability, and uniform argument stability of gradient descent \(GD\) and stochastic gradient descent \(SGD\) over discrete parameter spaces, where each update involves deterministic or stochastic rounding\. We show that deterministic rounding degrades the generalization error of GD on convex, Lipschitz, and smooth loss functions, increasing the rate fromO​\(T/n\)O\(T/n\)toO​\(T/n\)O\(T/\\sqrt\{n\}\), and establish matching lower bounds\. We further prove that uniform stability of GD becomesΩ​\(T\)\\Omega\(T\), showing that stability\-based generalization bounds are vacuous in this setting\. In contrast, for the same losses, stochastic gradient descent with deterministic rounding admits nontrivial uniform stability guarantees, which differ qualitatively from the real\-valued case and exhibit distinct dependencies on the number of iterations and the dimension: we prove tight boundsO​\(T/n\)O\(T/n\)for one dimension andO​\(T2/n\)O\(T^\{2\}/n\)for higher dimensions\. We also show that stochastic rounding can introduce generalization error that increases with the dimension; such a phenomenon is absent in standard real\-valued optimization and in the deterministic rounding case\. Finally, we provide upper bounds on uniform argument stability for stochastic rounding schemes and show that these bounds are tight when the loss can be represented as a sum of coordinate\-wise functions\.

## 1Introduction

Gradient\-based optimization algorithms are the backbone of modern machine learning and are widely used to train models ranging from linear predictors to deep neural networks\. Starting from classical methods such as gradient descent \(GD\) and stochastic gradient descent \(SGD\), numerous variants have been developed to improve convergence speed and robustness in large\-scale and non\-convex settings\(Duchiet al\.,[2011](https://arxiv.org/html/2606.06934#bib.bib27); Tieleman and Hinton,[2012](https://arxiv.org/html/2606.06934#bib.bib30); Kingma and Ba,[2015](https://arxiv.org/html/2606.06934#bib.bib28); Liuet al\.,[2025](https://arxiv.org/html/2606.06934#bib.bib29)\)\. These algorithms have been extensively studied, with a rich theory characterizing their convergence behavior under various assumptions\(Bottouet al\.,[2018](https://arxiv.org/html/2606.06934#bib.bib34); Reddiet al\.,[2018](https://arxiv.org/html/2606.06934#bib.bib35); Zhouet al\.,[2024](https://arxiv.org/html/2606.06934#bib.bib36)\), as well as their generalization performance in statistical learning frameworks\(Hardtet al\.,[2016](https://arxiv.org/html/2606.06934#bib.bib4); Bassilyet al\.,[2020](https://arxiv.org/html/2606.06934#bib.bib2); Zhouet al\.,[2020](https://arxiv.org/html/2606.06934#bib.bib38); Nguyenet al\.,[2022](https://arxiv.org/html/2606.06934#bib.bib37)\)\. However, most existing analyses rely on an idealized setting in which the optimization variables lie in a continuous space and all arithmetic operations are performed exactly\.

In practice, optimization algorithms are implemented on digital hardware using discrete parameters, such as fixed\-point or floating\-point representations\(Goldberg,[1991](https://arxiv.org/html/2606.06934#bib.bib33); Guptaet al\.,[2015](https://arxiv.org/html/2606.06934#bib.bib1)\)\. In this setting, the parameter space is inherently discrete, and every arithmetic operation incurs round\-off errors\. As a result, the optimization dynamics can differ fundamentally from their real\-valued counterparts: updates are quantized and numerical errors accumulate over time\. Recent works have begun to study gradient\-based methods in discrete or quantized settings by establishing convergence guarantees\(Markovet al\.,[2023](https://arxiv.org/html/2606.06934#bib.bib10); Xinet al\.,[2025](https://arxiv.org/html/2606.06934#bib.bib11); Xiaet al\.,[2025](https://arxiv.org/html/2606.06934#bib.bib22)\)\. Nevertheless, a theoretical understanding of how discrete parameters affect generalization remains limited\.

Beyond convergence, understanding the generalization properties of optimization algorithms is a central question in statistical learning theory\. Classical approaches based on uniform convergence provide bounds on the gap between empirical and population risks uniformly over a hypothesis class\. However, such bounds typically depend on the ambient dimension\(Shalev\-Shwartz and Ben\-David,[2014](https://arxiv.org/html/2606.06934#bib.bib31); Feldman,[2016](https://arxiv.org/html/2606.06934#bib.bib32)\)and are agnostic to the specific optimization algorithm used, often leading to overly pessimistic bounds in modern high\-dimensional settings\. This limitation has motivated*algorithm\-dependent*analyses, which take into account the dynamics of the optimization process\.

A canonical framework in this direction is*algorithmic stability*\(Bousquet and Elisseeff,[2002](https://arxiv.org/html/2606.06934#bib.bib7); Shalev\-Shwartzet al\.,[2010](https://arxiv.org/html/2606.06934#bib.bib9)\)\. In this framework, a learning algorithmAAmaps a datasetS=\(z1,…,zn\)S=\(z\_\{1\},\\dots,z\_\{n\}\)to a parameter vectorA​\(S\)A\(S\), and the generalization error is bounded by the sensitivity ofA​\(S\)A\(S\)to small perturbations of the dataset\. In particular,*uniform stability*measures how much the loss changes when a single training example is replaced, while*uniform argument stability*measures how much the learned parameters themselves change under such perturbations\. When the loss function is Lipschitz, these two notions are directly related, allowing one to bound the generalization error by tracking how perturbations propagate through the optimization dynamics\. This framework has been particularly successful for gradient\-based methods, yielding dimension\-independent generalization guarantees for GD and SGD\(Hardtet al\.,[2016](https://arxiv.org/html/2606.06934#bib.bib4); Chenet al\.,[2018](https://arxiv.org/html/2606.06934#bib.bib6); Bassilyet al\.,[2020](https://arxiv.org/html/2606.06934#bib.bib2)\)\.

### 1\.1Our contributions

In this paper, we revisit this framework with fixed\-point parameters\. Specifically, we consider gradient\-based optimization over a discrete parameter space\{Δ​x:x∈ℤ\}d\\\{\\Delta x:x\\in\\mathbb\{Z\}\\\}^\{d\}, where each update involves deterministic or stochastic rounding\. Within this setting, we analyze how finite\-precision effects influence generalization error, uniform stability, and uniform argument stability111generalization error≤uniform stability≤Lipschitz constant×uniform argument stability\\text\{generalization error\}\\leq\\text\{uniform stability\}\\leq\\text\{Lipschitz constant\}\\times\\text\{uniform argument stability\}\(see[Section2\.4](https://arxiv.org/html/2606.06934#S2.SS4)\)\.of gradient\-based optimization algorithms when the loss functions are convex, Lipschitz, and smooth\. We show that the discrete and inexact nature of machine arithmetic fundamentally alters the behavior of these stability measures, leading to qualitatively different generalization guarantees compared to the classical real\-valued setting\. In particular, our contributions can be summarized as below \(see also[Table˜1](https://arxiv.org/html/2606.06934#S1.T1)\)\. Here,𝖣𝖱​\-GD\{\\mathsf\{DR\}\\text\{\\rm\{\-GD\}\}\},𝖲𝖱​\-GD\{\\mathsf\{SR\}\\text\{\\rm\{\-GD\}\}\},𝖣𝖱​\-SGD\{\\mathsf\{DR\}\\text\{\\rm\{\-SGD\}\}\}, and𝖲𝖱​\-SGD\{\\mathsf\{SR\}\\text\{\\rm\{\-SGD\}\}\}denote \(stochastic\) gradient descent on\{Δ​x:x∈ℤ\}d\\\{\\Delta x:x\\in\\mathbb\{Z\}\\\}^\{d\}with deterministic/stochastic rounding, andGDandSGDdenote standard \(stochastic\) gradient descent onℝd\\mathbb\{R\}^\{d\}without rounding operations\. In addition, we hide the Lipschitz constant andΔ\\Deltafor concise representation; we present the precise bounds including them in[Sections˜3](https://arxiv.org/html/2606.06934#S3)and[4](https://arxiv.org/html/2606.06934#S4)\.

- •We first prove that the generalization error ofTT\-step𝖣𝖱​\-GD\{\\mathsf\{DR\}\\text\{\\rm\{\-GD\}\}\}is upper bounded byO​\(η​T/n\)O\(\\eta T/\\sqrt\{n\}\)222η\\etadenotes the learning rate andnndenotes the number of samples\.when the underlying samples are binary \([Theorem˜2](https://arxiv.org/html/2606.06934#Thmtheorem2)\); we also show the tightness of this bound, even for an arbitrary sample space \([Theorem˜3](https://arxiv.org/html/2606.06934#Thmtheorem3)\)\. Since the generalization error ofGDisO​\(η​T/n\)O\(\\eta T/n\)for convex, Lipschitz, and smooth losses, these results imply that deterministic rounding to the fixed\-point parameter space degrades the sample complexity ofGD\.
- •However, a stability\-based approach cannot achieve this bound: the uniform stability of𝖣𝖱​\-GD\{\\mathsf\{DR\}\\text\{\\rm\{\-GD\}\}\}can be lower bounded byΩ​\(T\)\\Omega\(T\)\([Theorem˜4](https://arxiv.org/html/2606.06934#Thmtheorem4)\)\. Hence, the uniform stability of𝖣𝖱​\-GD\{\\mathsf\{DR\}\\text\{\\rm\{\-GD\}\}\}can only produce vacuous generalization bounds in this case\.
- •On the other hand, this is not the case for𝖣𝖱​\-SGD\{\\mathsf\{DR\}\\text\{\\rm\{\-SGD\}\}\}\. We prove that the uniform stability of𝖣𝖱​\-SGD\{\\mathsf\{DR\}\\text\{\\rm\{\-SGD\}\}\}is bounded byO​\(η​T/n\)O\(\\eta T/n\)ford=1d=1\([Theorem˜5](https://arxiv.org/html/2606.06934#Thmtheorem5)\), and it is bounded byO​\(η​T2/n\)O\(\\eta T^\{2\}/n\)ford\>1d\>1\([Theorem˜8](https://arxiv.org/html/2606.06934#Thmtheorem8)\)\. We further show the tightness of these bounds \([Theorems˜6](https://arxiv.org/html/2606.06934#Thmtheorem6)and[7](https://arxiv.org/html/2606.06934#Thmtheorem7)\), which implies that depending on the ambient dimensiondd, the uniform stability can have a different dependence onTT, while such a dependency was not observed inSGD\.
- •We next consider stochastic rounding and show that the generalization errors of𝖲𝖱​\-GD\{\\mathsf\{SR\}\\text\{\\rm\{\-GD\}\}\}and𝖲𝖱​\-SGD\{\\mathsf\{SR\}\\text\{\\rm\{\-SGD\}\}\}can increase withdd\. Specifically,[Theorem˜9](https://arxiv.org/html/2606.06934#Thmtheorem9)states that there exists a convex, Lipschitz, and smooth loss and a data distribution for which the generalization errors of𝖲𝖱​\-GD\{\\mathsf\{SR\}\\text\{\\rm\{\-GD\}\}\}and𝖲𝖱​\-SGD\{\\mathsf\{SR\}\\text\{\\rm\{\-SGD\}\}\}areΩ​\(η​d1/2/n\)\\Omega\(\\sqrt\{\\eta d^\{1/2\}\}/n\)\.333We assumeη​d1/2=Ω​\(1\)\\eta d^\{1/2\}=\\Omega\(1\)here\. See[Section4](https://arxiv.org/html/2606.06934#S4)for bounds without this assumption\.We note that such a dimension dependency is from the stochasticity in rounding, and does not arise inGD,SGD,𝖣𝖱​\-GD\{\\text\{\\rm\{GD\}\}\},\{\\text\{\\rm\{SGD\}\}\},\{\\mathsf\{DR\}\\text\{\\rm\{\-GD\}\}\}, and𝖣𝖱​\-SGD\{\\mathsf\{DR\}\\text\{\\rm\{\-SGD\}\}\}\.
- •We also prove upper bounds on the uniform argument stability:O​\(η​d1/2​T3/n\)O\(\\sqrt\{\\eta d^\{1/2\}T^\{3\}\}/n\)for𝖲𝖱​\-SGD\{\\mathsf\{SR\}\\text\{\\rm\{\-SGD\}\}\}andO​\(η​T/n\+min⁡\{1,η​d1/2​T/n\}​η​d1/2​T\)O\(\\eta T/n\+\\min\\\{1,\\eta d^\{1/2\}T/n\\\}\\sqrt\{\\eta d^\{1/2\}T\}\)for𝖲𝖱​\-GD\{\\mathsf\{SR\}\\text\{\\rm\{\-GD\}\}\}\([Theorems˜11](https://arxiv.org/html/2606.06934#Thmtheorem11)and[10](https://arxiv.org/html/2606.06934#Thmtheorem10)\)\.[Footnote˜3](https://arxiv.org/html/2606.06934#footnote3)We further reduce theT3/2T^\{3/2\}dependency here toTTfor losses that can be represented as a sum of coordinate\-wise functions \([Theorem˜12](https://arxiv.org/html/2606.06934#Thmtheorem12)\), and show the tightness of these reduced bounds \([Theorem˜13](https://arxiv.org/html/2606.06934#Thmtheorem13)\)\.

Table 1:A summary of generalization and stability bounds for iterative algorithms\. All results are for convex,LL\-Lipschitz, and smooth losses except for\(Bassilyet al\.,[2020](https://arxiv.org/html/2606.06934#bib.bib2)\), which considers convex, Lipschitz but non\-smooth losses\.εg,εu​s,εu​a​s\\varepsilon\_\{g\},\\varepsilon\_\{us\},\\varepsilon\_\{uas\}denote the generalization error, uniform stability, and uniform argument stability, respectively\. The lower bounds hold for worst\-case examples, and the upper bounds hold for all examples\. For simplicity, we considerL,Δ=Θ​\(1\)L,\\Delta=\\Theta\(1\),n\>Tn\>T,η=O​\(1\)\\eta=O\(1\), andη​d1/2=Ω​\(1\)\\eta d^\{1/2\}=\\Omega\(1\)here; the precise bounds without these assumptions are in[Sections˜3](https://arxiv.org/html/2606.06934#S3)and[4](https://arxiv.org/html/2606.06934#S4)\.nn,η\\eta,TT, anddddenote the number of samples, learning rate, total number of iterations, and parameter dimension, respectively\. \(1\)requiresd≥min⁡\{T,1/η2\}d\\geq\\min\\\{T,1/\\eta^\{2\}\\\}for the lower bounds,\(2\)requires the sample space to be binary,\(3\)requiresd=1d=1,\(4\)requiresd≥2d\\geq 2,\(5\)requiresT=1T=1, and\(6\)requires that the loss function is a sum of coordinate\-wise functions\.

## 2Preliminaries

### 2\.1Notations and problem setup

Forn∈ℕn\\in\\mathbb\{N\}, we use\[n\]\[n\]to denote\{1,⋯,n\}\\\{1,\\cdots,n\\\}\. Ford∈ℕd\\in\\mathbb\{N\}, we use𝟎d\\boldsymbol\{0\}\_\{d\}to denote\(0,⋯,0\)∈ℝd\(0,\\cdots,0\)\\in\\mathbb\{R\}^\{d\}and𝟏d\\boldsymbol\{1\}\_\{d\}to denote\(1,⋯,1\)∈ℝd\(1,\\cdots,1\)\\in\\mathbb\{R\}^\{d\}\. Forn∈ℕn\\in\\mathbb\{N\}andS,S′∈ℝnS,S^\{\\prime\}\\in\\mathbb\{R\}^\{n\}, we saySSandS′S^\{\\prime\}are neighboring, denoted byS≃S′S\\simeq S^\{\\prime\}, if they differ at a single entry\. ForΔ\>0\\Delta\>0, we useℤΔ\\mathbb\{Z\}\_\{\\Delta\}to denote the set\{Δ​n:n∈ℤ\}\\\{\\Delta n:n\\in\\mathbb\{Z\}\\\}\. For anyx∈ℝx\\in\\mathbb\{R\}, we use⌊x⌋Δ\\lfloor x\\rfloor\_\{\\Delta\}to denotemax⁡\{y∈ℤΔ:y≤x\}\\max\\\{y\\in\\mathbb\{Z\}\_\{\\Delta\}:y\\leq x\\\}\. We say a functionf:ℝd→ℝf:\\mathbb\{R\}^\{d\}\\to\\mathbb\{R\}is separable if there exist functionsf1,⋯,fd:ℝ→ℝf\_\{1\},\\cdots,f\_\{d\}:\\mathbb\{R\}\\to\\mathbb\{R\}such thatf​\(x1,⋯,xd\)=∑i=1dfi​\(xi\)f\(x\_\{1\},\\cdots,x\_\{d\}\)=\\sum\_\{i=1\}^\{d\}f\_\{i\}\(x\_\{i\}\)\. For a logical propositionPP, we define𝟙​\[P\]=1\\mathbbm\{1\}\[P\]=1ifPPis true and𝟙​\[P\]=0\\mathbbm\{1\}\[P\]=0ifPPis false\.

### 2\.2Supervised learning and generalization error

We consider a supervised learning problem\. Let𝒟\\mathcal\{D\}be an underlying distribution over a sample space𝒵\\mathcal\{Z\}and letS=\(z1,⋯,zn\)S=\(z\_\{1\},\\cdots,z\_\{n\}\)be a dataset ofnnsamples drawn i\.i\.d\. from𝒟\\mathcal\{D\}\. For a parameter spaceℝd\\mathbb\{R\}^\{d\}, a loss functionf:ℝd×𝒵→ℝf:\\mathbb\{R\}^\{d\}\\times\\mathcal\{Z\}\\to\\mathbb\{R\}, and a learning algorithmAf:𝒵n→ℝdA\_\{f\}:\\mathcal\{Z\}^\{n\}\\to\\mathbb\{R\}^\{d\}, we define the*generalization error*ofAfA\_\{f\}as follows:

εg​\(Af;𝒟\)≜\|𝔼Af,S=\(z1,…,zn\)​\[f​\(Af​\(S\);S\)−f​\(Af​\(S\);𝒟\)\]\|\\displaystyle\\varepsilon\_\{g\}\(A\_\{f\};\\mathcal\{D\}\)\\triangleq\|\\mathbb\{E\}\_\{A\_\{f\},S=\(z\_\{1\},\\dots,z\_\{n\}\)\}\[f\(A\_\{f\}\(S\);S\)\-f\(A\_\{f\}\(S\);\\mathcal\{D\}\)\]\|wheref​\(w;𝒟\)≜𝔼z∼𝒟​\[f​\(w;z\)\]f\(w;\\mathcal\{D\}\)\\triangleq\\mathbb\{E\}\_\{z\\sim\\mathcal\{D\}\}\[f\(w;z\)\]andf​\(w;S\)≜1n​∑i=1nf​\(w;zi\)f\(w;S\)\\triangleq\\frac\{1\}\{n\}\\sum\_\{i=1\}^\{n\}f\(w;z\_\{i\}\)denote the population loss and the empirical loss, respectively\. Throughout this paper, we primarily focus on loss functions such that for anyz∈𝒵z\\in\\mathcal\{Z\},f​\(⋅,z\)f\(\\cdot,z\)is differentiable, convex, Lipschitz, and smooth\.

###### Definition 1\.

f:ℝd→ℝf:\\mathbb\{R\}^\{d\}\\to\\mathbb\{R\}is convex if for anyu,v∈ℝdu,v\\in\\mathbb\{R\}^\{d\},f​\(u\)≥f​\(v\)\+⟨∇f​\(v\),u−v⟩\.f\(u\)\\geq f\(v\)\+\\langle\\nabla f\(v\),u\-v\\rangle\.

###### Definition 2\.

f:ℝd→ℝf:\\mathbb\{R\}^\{d\}\\to\\mathbb\{R\}isLL\-Lipschitz if for anyu,v∈ℝdu,v\\in\\mathbb\{R\}^\{d\},\|f​\(u\)−f​\(v\)\|≤L​‖u−v‖2\.\|f\(u\)\-f\(v\)\|\\leq L\\\|u\-v\\\|\_\{2\}\.

###### Definition 3\.

f:ℝd→ℝf:\\mathbb\{R\}^\{d\}\\to\\mathbb\{R\}isMM\-smooth if for anyu,v∈ℝdu,v\\in\\mathbb\{R\}^\{d\},‖∇f​\(u\)−∇f​\(v\)‖2≤M​‖u−v‖2\.\\\|\\nabla f\(u\)\-\\nabla f\(v\)\\\|\_\{2\}\\leq M\\\|u\-v\\\|\_\{2\}\.

### 2\.3Gradient descent and stochastic gradient descent with rounding operations

Gradient descent \(GD\) and stochastic gradient descent \(SGD\) are iterative learning algorithms defined as follows: given a learning rateη\>0\\eta\>0, initial parameterw0∈ℝdw\_\{0\}\\in\\mathbb\{R\}^\{d\}, total number of iterationsTT,

Af,θGD​\(S\)=Gf,η,S,TGD∘⋯∘Gf,η,S,1GD​\(w0\)​and​Af,θSGD​\(S\)=Gf,η,S,TSGD∘⋯∘Gf,η,S,1SGD​\(w0\)\\displaystyle A^\{\{\\text\{\\rm\{GD\}\}\}\}\_\{f,\\theta\}\(S\)=G^\{\\text\{\\rm\{GD\}\}\}\_\{f,\\eta,S,T\}\\circ\\cdots\\circ G^\{\\text\{\\rm\{GD\}\}\}\_\{f,\\eta,S,1\}\(w\_\{0\}\)~~\\text\{and\}~~A^\{\{\\text\{\\rm\{SGD\}\}\}\}\_\{f,\\theta\}\(S\)=G^\{\\text\{\\rm\{SGD\}\}\}\_\{f,\\eta,S,T\}\\circ\\cdots\\circ G^\{\\text\{\\rm\{SGD\}\}\}\_\{f,\\eta,S,1\}\(w\_\{0\}\)\(1\)whereθ=\(η,T,w0\)\\theta=\(\\eta,T,w\_\{0\}\),

Gf,η,S,tGD​\(w\)=w−η⋅∇f​\(w;S\),and​Gf,η,S,tSGD​\(w\)=w−η⋅∇f​\(w;zit\)\\displaystyle G^\{\\text\{\\rm\{GD\}\}\}\_\{f,\\eta,S,t\}\(w\)=w\-\\eta\\cdot\\nabla f\(w;S\),~~\\text\{and\}~~G^\{\\text\{\\rm\{SGD\}\}\}\_\{f,\\eta,S,t\}\(w\)=w\-\\eta\\cdot\\nabla f\(w;z\_\{i\_\{t\}\}\)\(2\)for allt∈\[T\]t\\in\[T\]\. Here,iti\_\{t\}is a random variable following the uniform distribution over\[n\]\[n\]\.

Given a discrete gridℤΔd\\mathbb\{Z\}\_\{\\Delta\}^\{d\}, we define deterministic rounding𝖣𝖱\\mathsf\{DR\}and stochastic rounding𝖲𝖱\\mathsf\{SR\}that mapsℝd\\mathbb\{R\}^\{d\}toℤΔd\\mathbb\{Z\}\_\{\\Delta\}^\{d\}as follows:

𝖣𝖱​\(x\)j∈arg​miny∈ℤΔ⁡\|y−xj\|,𝖲𝖱​\(x\)j=⌊xj⌋Δ\+Δ⋅𝟙​\[uj≤xj−⌊xj⌋ΔΔ\]\\displaystyle\\mathsf\{DR\}\(x\)\_\{j\}\\in\\operatorname\*\{arg\\,min\}\_\{y\\in\\mathbb\{Z\}\_\{\\Delta\}\}\|y\-x\_\{j\}\|,\\quad\\mathsf\{SR\}\(x\)\_\{j\}=\\lfloor x\_\{j\}\\rfloor\_\{\\Delta\}\+\\Delta\\cdot\\mathbbm\{1\}\\left\[u\_\{j\}\\leq\\frac\{x\_\{j\}\-\\lfloor x\_\{j\}\\rfloor\_\{\\Delta\}\}\{\\Delta\}\\right\]\(3\)where𝖣𝖱​\(x\)j\\mathsf\{DR\}\(x\)\_\{j\}is the even multiple ofΔ\\Deltaifminy∈ℤΔ⁡\|y−xj\|=0\.5​Δ\\min\_\{y\\in\\mathbb\{Z\}\_\{\\Delta\}\}\|y\-x\_\{j\}\|=0\.5\\Deltaanduju\_\{j\}is a random variable following the uniform distribution over\[0,1\]\[0,1\]\. We note that choosing an arbitrary𝖣𝖱​\(x\)j\\mathsf\{DR\}\(x\)\_\{j\}inarg​miny∈ℤΔ⁡\|y−xj\|\\operatorname\*\{arg\\,min\}\_\{y\\in\\mathbb\{Z\}\_\{\\Delta\}\}\|y\-x\_\{j\}\|does not affect our main results as long as it is deterministic\.

Givenθ=\(η,T,w0∈ℤΔd\)\\theta=\(\\eta,T,w\_\{0\}\\in\\mathbb\{Z\}\_\{\\Delta\}^\{d\}\), we define GD with deterministic rounding ontoℤΔd\\mathbb\{Z\}\_\{\\Delta\}^\{d\}\(𝖣𝖱​\-GD\{\\mathsf\{DR\}\\text\{\\rm\{\-GD\}\}\}\) as

Af,θ𝖣𝖱​\-GD​\(S\)=Gf,η,S,T𝖣𝖱​\-GD∘⋯∘Gf,η,S,1𝖣𝖱​\-GD​\(w0\)​where​Gf,η,S,t𝖣𝖱​\-GD​\(w\)=w−𝖣𝖱​\(η⋅∇f​\(w;S\)\)\.\\displaystyle A\_\{f,\\theta\}^\{\\mathsf\{DR\}\\text\{\\rm\{\-GD\}\}\}\(S\)=G^\{\\mathsf\{DR\}\\text\{\\rm\{\-GD\}\}\}\_\{f,\\eta,S,T\}\\circ\\cdots\\circ G^\{\\mathsf\{DR\}\\text\{\\rm\{\-GD\}\}\}\_\{f,\\eta,S,1\}\(w\_\{0\}\)~~\\text\{where\}~~G^\{\\mathsf\{DR\}\\text\{\\rm\{\-GD\}\}\}\_\{f,\\eta,S,t\}\(w\)=w\-\\mathsf\{DR\}\(\\eta\\cdot\\nabla f\(w;S\)\)\.We similarly define GD with stochastic rounding \(𝖲𝖱​\-GD\{\\mathsf\{SR\}\\text\{\\rm\{\-GD\}\}\}\) and SGD with deterministic and stochastic rounding \(𝖣𝖱​\-SGD\{\\mathsf\{DR\}\\text\{\\rm\{\-SGD\}\}\}and𝖲𝖱​\-SGD\{\\mathsf\{SR\}\\text\{\\rm\{\-SGD\}\}\}, respectively\)\. For𝖲𝖱​\-GD\{\\mathsf\{SR\}\\text\{\\rm\{\-GD\}\}\}and𝖲𝖱​\-SGD\{\\mathsf\{SR\}\\text\{\\rm\{\-SGD\}\}\},𝖲𝖱\\mathsf\{SR\}at thett\-th iteration uses fresh independent random variablesut,1,⋯,ut,du\_\{t,1\},\\cdots,u\_\{t,d\}in[Eq\.˜3](https://arxiv.org/html/2606.06934#S2.E3)\.

### 2\.4Algorithmic stability

Algorithmic stability is a framework for bounding the generalization error of a learning algorithm based on its sensitivity to dataset perturbations\. Specifically, if a learning algorithmAAis*symmetric*\(i\.e\., the distribution ofA​\(S\)A\(S\)does not depend on the order of the samples inSS\), then we can represent the generalization error as stated in the following lemma\.

###### Lemma 1\(Lemma 7 in\(Bousquet and Elisseeff,[2002](https://arxiv.org/html/2606.06934#bib.bib7)\)\)\.

Let𝒟\\mathcal\{D\}be a distribution over𝒵\\mathcal\{Z\},f:ℝd×𝒵→ℝf:\\mathbb\{R\}^\{d\}\\times\\mathcal\{Z\}\\to\\mathbb\{R\}be a loss function,S=\(z1,⋯,zn\)S=\(z\_\{1\},\\cdots,z\_\{n\}\)andS′=\(z1′,z2,⋯,zn\)S^\{\\prime\}=\(z\_\{1\}^\{\\prime\},z\_\{2\},\\cdots,z\_\{n\}\)be neighboring datasets wherez1′,z1,…,znz\_\{1\}^\{\\prime\},z\_\{1\},\\dots,z\_\{n\}are drawn from𝒟\\mathcal\{D\}independently\. IfAfA\_\{f\}is symmetric, then

εg​\(Af;𝒟\)=\|𝔼Af,S′,z1​\[f​\(Af​\(S\);z1\)−f​\(Af​\(S′\);z1\)\]\|\.\\displaystyle\\varepsilon\_\{g\}\(A\_\{f\};\\mathcal\{D\}\)=\\left\|\\mathbb\{E\}\_\{A\_\{f\},S^\{\\prime\},z\_\{1\}\}\\left\[f\(A\_\{f\}\(S\);z\_\{1\}\)\-f\(A\_\{f\}\(S^\{\\prime\}\);z\_\{1\}\)\\right\]\\right\|\.

From[Section˜2\.4](https://arxiv.org/html/2606.06934#S2.SS4), one can observe that the*uniform stability*ofAfA\_\{f\}defined as

εu​s​\(Af\)=supS≃S′,z𝔼Af​\[f​\(Af​\(S\);z\)−f​\(Af​\(S′\);z\)\]\\displaystyle\\varepsilon\_\{us\}\(A\_\{f\}\)=\\sup\_\{S\\simeq S^\{\\prime\},z\}\\mathbb\{E\}\_\{A\_\{f\}\}\[f\(A\_\{f\}\(S\);z\)\-f\(A\_\{f\}\(S^\{\\prime\}\);z\)\]upper bounds the generalization error\. In particular, if the loss function isLL\-Lipschitz, a typical process to bound the uniform stability is to bound the*uniform argument stability*defined as follows:

εu​a​s​\(Af\)=supS≃S′𝔼Af​\[‖Af​\(S\)−Af​\(S′\)‖2\]\.\\displaystyle\\varepsilon\_\{uas\}\(A\_\{f\}\)=\\sup\_\{S\\simeq S^\{\\prime\}\}\\mathbb\{E\}\_\{A\_\{f\}\}\[\\\|A\_\{f\}\(S\)\-A\_\{f\}\(S^\{\\prime\}\)\\\|\_\{2\}\]\.For randomized algorithms, the expectation here is taken under*coupled*randomness: foralg∈\{SGD,𝖣𝖱​\-SGD,𝖲𝖱​\-SGD\}\{\\text\{\\rm\{alg\}\}\}\\in\\\{\{\\text\{\\rm\{SGD\}\}\},\{\\mathsf\{DR\}\\text\{\\rm\{\-SGD\}\}\},\{\\mathsf\{SR\}\\text\{\\rm\{\-SGD\}\}\}\\\},Af,θalg​\(S\)A\_\{f,\\theta\}^\{\\text\{\\rm\{alg\}\}\}\(S\)andAf,θalg​\(S′\)A\_\{f,\\theta\}^\{\\text\{\\rm\{alg\}\}\}\(S^\{\\prime\}\)share the same random indicesi1,⋯,iTi\_\{1\},\\cdots,i\_\{T\}in[Eq\.˜2](https://arxiv.org/html/2606.06934#S2.E2), and foralg∈\{𝖲𝖱​\-GD,𝖲𝖱​\-SGD\}\{\\text\{\\rm\{alg\}\}\}\\in\\\{\{\\mathsf\{SR\}\\text\{\\rm\{\-GD\}\}\},\{\\mathsf\{SR\}\\text\{\\rm\{\-SGD\}\}\}\\\}, stochastic rounding operations use the same random variableut,ju\_\{t,j\}for eacht∈\[T\]t\\in\[T\]andj∈\[d\]j\\in\[d\]in[Eq\.˜3](https://arxiv.org/html/2606.06934#S2.E3)\. We note that this coupling is used only for our upper bounds forεu​a​s​\(Af\)\\varepsilon\_\{uas\}\(A\_\{f\}\); our lower bounds hold for any joint distribution ofAf​\(S\)A\_\{f\}\(S\)andAf​\(S′\)A\_\{f\}\(S^\{\\prime\}\)\.

Then, the following relation between the generalization error, uniform stability, and uniform argument stability holds:

εg​\(Af;𝒟\)≤εu​s​\(Af\)≤L⋅εu​a​s​\(Af\)\.\\displaystyle\\varepsilon\_\{g\}\(A\_\{f\};\\mathcal\{D\}\)\\leq\\varepsilon\_\{us\}\(A\_\{f\}\)\\leq L\\cdot\\varepsilon\_\{uas\}\(A\_\{f\}\)\.
We sayAf,θalgA^\{\\text\{\\rm\{alg\}\}\}\_\{f,\\theta\}is*non\-expansive*if for anyu,v∈ℤΔdu,v\\in\\mathbb\{Z\}\_\{\\Delta\}^\{d\}andt∈\[T\]t\\in\[T\], there exists a joint distributionppofGf,η,S,talg​\(u\)≜u′G\_\{f,\\eta,S,t\}^\{\\text\{\\rm\{alg\}\}\}\(u\)\\triangleq u^\{\\prime\}andGf,η,S,talg​\(v\)≜v′G\_\{f,\\eta,S,t\}^\{\\text\{\\rm\{alg\}\}\}\(v\)\\triangleq v^\{\\prime\}such that𝔼p​\[‖u′−v′‖2\]≤‖u−v‖2\\mathbb\{E\}\_\{p\}\[\\\|u^\{\\prime\}\-v^\{\\prime\}\\\|\_\{2\}\]\\leq\\\|u\-v\\\|\_\{2\}\.

## 3Generalization error and uniform stability of𝖣𝖱​\-GD\{\\mathsf\{DR\}\\text\{\\rm\{\-GD\}\}\}and𝖣𝖱​\-SGD\{\\mathsf\{DR\}\\text\{\\rm\{\-SGD\}\}\}

In this section, we formally present our bounds on generalization error, uniform stability, and uniform argument stability of gradient methods overℤΔd\\mathbb\{Z\}\_\{\\Delta\}^\{d\}with deterministic rounding schemes\. In our theoretical results, we use asymptotic notation to hide universal constants\. Specifically, for nonnegative functionsffandggofn,T,d,η,Δ,M,n,T,d,\\eta,\\Delta,M,andLL, we writef≲gf\\lesssim gif there exists a universal constantC\>0C\>0such thatf≤C⋅gf\\leq C\\cdot g\. We writef≳gf\\gtrsim gifg≲fg\\lesssim f, andf≍gf\\asymp gif bothf≲gf\\lesssim gandf≳gf\\gtrsim ghold\. We usef=O​\(g\)f=O\(g\),f=Ω​\(g\)f=\\Omega\(g\), andf=Θ​\(g\)f=\\Theta\(g\)to denotef≲gf\\lesssim g,f≳gf\\gtrsim g, andf≍gf\\asymp g, respectively\.

We first provide lower and upper bounds on the generalization error of𝖣𝖱​\-GD\{\\mathsf\{DR\}\\text\{\\rm\{\-GD\}\}\}\.

###### Theorem 2\.

Letn,d,T∈ℕn,d,T\\in\\mathbb\{N\},η,Δ,L\>0\\eta,\\Delta,L\>0,w0∈ℤΔdw\_\{0\}\\in\\mathbb\{Z\}\_\{\\Delta\}^\{d\},𝒟\\mathcal\{D\}be a distribution over\{0,1\}\\\{0,1\\\}, andf:ℝd×\{0,1\}→ℝf:\\mathbb\{R\}^\{d\}\\times\\\{0,1\\\}\\to\\mathbb\{R\}be anLL\-Lipschitz loss function\. Then, forθ=\(η,T,w0\)\\theta=\(\\eta,T,w\_\{0\}\),

εg​\(Af,θ𝖣𝖱​\-GD;𝒟\)≤2​η​L2​Tn\.\\displaystyle\\varepsilon\_\{g\}\(A\_\{f,\\theta\}^\{\{\\mathsf\{DR\}\\text\{\\rm\{\-GD\}\}\}\};\\mathcal\{D\}\)\\leq\\frac\{2\\eta L^\{2\}T\}\{\\sqrt\{n\}\}\.

###### Theorem 3\.

Letn,T∈ℕn,T\\in\\mathbb\{N\}andη,Δ,L\>0\\eta,\\Delta,L\>0such thatη​L\>c​Δ\\eta L\>c\\Deltafor some universal constantc\>0\.5c\>0\.5, andn≥max⁡\{8​c/\(2​c−1\),6\}n\\geq\\max\\\{8c/\(2c\-1\),6\\\}\. Then, there exist a distribution𝒟\\mathcal\{D\}over\{0,1\}\\\{0,1\\\}and anLL\-Lipschitz, affine loss functionf:ℝ×\{0,1\}→ℝf:\\mathbb\{R\}\\times\\\{0,1\\\}\\to\\mathbb\{R\}such that forθ=\(η,T,w0=0\)\\theta=\(\\eta,T,w\_\{0\}=0\),

εg​\(Af,θ𝖣𝖱​\-GD;𝒟\)≳max⁡\{Δ​L​Tn,L​T​\(η​Ln−Δ\)\}\.\\displaystyle\\varepsilon\_\{g\}\(A^\{\{\\mathsf\{DR\}\\text\{\\rm\{\-GD\}\}\}\}\_\{f,\\theta\};\\mathcal\{D\}\)\\gtrsim\\max\\left\\\{\\frac\{\\Delta LT\}\{\\sqrt\{n\}\},LT\\left\(\\frac\{\\eta L\}\{n\}\-\\Delta\\right\)\\right\\\}\.

[Theorem˜2](https://arxiv.org/html/2606.06934#Thmtheorem2)shows that the generalization error of𝖣𝖱​\-GD\{\\mathsf\{DR\}\\text\{\\rm\{\-GD\}\}\}isO​\(η​L2​T/n\)O\(\\eta L^\{2\}T/\\sqrt\{n\}\)when the sample space is binary \(i\.e\.,𝒵=\{0,1\}\\mathcal\{Z\}=\\\{0,1\\\}\), and[Theorem˜3](https://arxiv.org/html/2606.06934#Thmtheorem3)shows the tightness of this bound: there exists an affine functionffwith\|f′​\(x;z\)\|≤L\|f^\{\\prime\}\(x;z\)\|\\leq Land a data distribution𝒟\\mathcal\{D\}such that the generalization error of𝖣𝖱\\mathsf\{DR\}\-GD is lower bounded byΩ​\(Δ​L​T/n\)\\Omega\(\\Delta LT/\\sqrt\{n\}\)\. We note that our counterexample functionffin[Theorem˜3](https://arxiv.org/html/2606.06934#Thmtheorem3)can easily be extended to generald\>1d\>1and a non\-binary sample space, e\.g\., considerg​\(x1,…,xd\)=f​\(x1\)g\(x\_\{1\},\\dots,x\_\{d\}\)=f\(x\_\{1\}\)and data distribution supported on two samples\.

These results show that𝖣𝖱​\-GD\{\\mathsf\{DR\}\\text\{\\rm\{\-GD\}\}\}and GD have fundamentally different generalization properties since a stability\-based approach can achieveεg​\(Af,θGD;𝒟\)=O​\(η​L2​T/n\)\\varepsilon\_\{g\}\(A\_\{f,\\theta\}^\{\\text\{\\rm\{GD\}\}\};\\mathcal\{D\}\)=O\(\\eta L^\{2\}T/n\)\(Chenet al\.,[2018](https://arxiv.org/html/2606.06934#bib.bib6)\), i\.e\., the deterministic rounding ontoℤΔ\\mathbb\{Z\}\_\{\\Delta\}induces worse sample complexity compared to the exact real counterpart\. We further note that, asΔ→0\\Delta\\to 0, our bound in[Theorem˜3](https://arxiv.org/html/2606.06934#Thmtheorem3)converges to the existing uniform stability boundO​\(η​L2​T/n\)O\(\\eta L^\{2\}T/n\)\(Chenet al\.,[2018](https://arxiv.org/html/2606.06934#bib.bib6)\)\.

To prove[Theorem˜3](https://arxiv.org/html/2606.06934#Thmtheorem3), we explicitly choose the underlying data space𝒵=\{0,1\}\\mathcal\{Z\}=\\\{0,1\\\}and a loss functionf​\(w,z\)=−z​L​wf\(w,z\)=\-zLwforz∈𝒵z\\in\\mathcal\{Z\}\. We then show that for some neighboring datasetsS≃S′S\\simeq S^\{\\prime\}whereS′S^\{\\prime\}contains more ones compared toSS, the𝖣𝖱​\-GD\{\\mathsf\{DR\}\\text\{\\rm\{\-GD\}\}\}iterate forS′S^\{\\prime\}moves strictly faster than that forSS\. That is, we have𝖣𝖱​\(η​∇f​\(wt;S\)\)\>𝖣𝖱​\(η​∇f​\(wt′;S′\)\)\\mathsf\{DR\}\(\\eta\\nabla f\(w\_\{t\};S\)\)\>\\mathsf\{DR\}\(\\eta\\nabla f\(w\_\{t\}^\{\\prime\};S^\{\\prime\}\)\), which implies\|wT′−wT\|≥Δ​T\|w\_\{T\}^\{\\prime\}\-w\_\{T\}\|\\geq\\Delta T, and hence,\|f​\(Af,θ𝖣𝖱​\-GD​\(S\);1\)−f​\(Af,θ𝖣𝖱​\-GD​\(S′\);1\)\|≥L​Δ​T\|f\(A^\{\{\\mathsf\{DR\}\\text\{\\rm\{\-GD\}\}\}\}\_\{f,\\theta\}\(S\);1\)\-f\(A^\{\{\\mathsf\{DR\}\\text\{\\rm\{\-GD\}\}\}\}\_\{f,\\theta\}\(S^\{\\prime\}\);1\)\|\\geq L\\Delta T\. Here, we choose the underlying data distribution over𝒵\\mathcal\{Z\}so that such an event occurs with probabilityΩ​\(1/n\)\\Omega\(1/\\sqrt\{n\}\), which results in theΩ​\(Δ​L​T/n\)\\Omega\(\\Delta LT/\\sqrt\{n\}\)lower bound\. We note that such an observation only holds forGDwith deterministic rounding: forGD,SGD,𝖣𝖱​\-SGD,𝖲𝖱​\-GD,\{\\text\{\\rm\{GD\}\}\},\{\\text\{\\rm\{SGD\}\}\},\{\\mathsf\{DR\}\\text\{\\rm\{\-SGD\}\}\},\{\\mathsf\{SR\}\\text\{\\rm\{\-GD\}\}\},and𝖲𝖱​\-SGD\{\\mathsf\{SR\}\\text\{\\rm\{\-SGD\}\}\},\|wT′−wT\|→0\|w\_\{T\}^\{\\prime\}\-w\_\{T\}\|\\to 0asn→∞n\\to\\infty\(see[Section˜4](https://arxiv.org/html/2606.06934#S4)for the𝖲𝖱​\-GD\{\\mathsf\{SR\}\\text\{\\rm\{\-GD\}\}\}and𝖲𝖱​\-SGD\{\\mathsf\{SR\}\\text\{\\rm\{\-SGD\}\}\}cases\)\. We present the full proofs of[Theorems˜3](https://arxiv.org/html/2606.06934#Thmtheorem3)and[2](https://arxiv.org/html/2606.06934#Thmtheorem2)in[Sections˜A\.2](https://arxiv.org/html/2606.06934#A1.SS2)and[A\.1](https://arxiv.org/html/2606.06934#A1.SS1), respectively\.

We next show that the bound in[Theorem˜2](https://arxiv.org/html/2606.06934#Thmtheorem2)cannot be achieved by using the uniform stability\. While the bound in[Theorem˜2](https://arxiv.org/html/2606.06934#Thmtheorem2)converges to zero asn→∞n\\to\\infty, the following theorem shows that it is not the case for the uniform stability of𝖣𝖱​\-GD\{\\mathsf\{DR\}\\text\{\\rm\{\-GD\}\}\}\. In particular, it shows that the uniform stability can be independent ofnnbut can increase withTTat the same time\.

###### Theorem 4\.

Letn,T∈ℕn,T\\in\\mathbb\{N\},η,L,Δ\>0\\eta,L,\\Delta\>0withη​L\>0\.5​Δ\\eta L\>0\.5\\Delta\. Then, there exists anLL\-Lipschitz, affine loss functionf:ℝ×\{0,1\}→ℝf:\\mathbb\{R\}\\times\\\{0,1\\\}\\to\\mathbb\{R\}such that forθ=\(η,T,w0=0\)\\theta=\(\\eta,T,w\_\{0\}=0\),

εu​s​\(Af,θ𝖣𝖱​\-GD\)≳max⁡\{Δ​L​T,L​T​\(η​Ln−Δ\)\}\.\\displaystyle\\varepsilon\_\{us\}\(A^\{\\mathsf\{DR\}\\text\{\\rm\{\-GD\}\}\}\_\{f,\\theta\}\)\\gtrsim\\max\\left\\\{\\Delta LT,LT\\left\(\\frac\{\\eta L\}\{n\}\-\\Delta\\right\)\\right\\\}\.

[Theorem˜4](https://arxiv.org/html/2606.06934#Thmtheorem4)shows the existence of an affine functionffwith\|f′​\(x;z\)\|≤L\|f^\{\\prime\}\(x;z\)\|\\leq Lsuch that the uniform stability\-based approach can only produce a vacuous boundΩ​\(Δ​L​T\)\\Omega\(\\Delta LT\)\. As in[Theorem˜3](https://arxiv.org/html/2606.06934#Thmtheorem3), the bound in[Theorem˜4](https://arxiv.org/html/2606.06934#Thmtheorem4)naturally extends to a general input dimensiond\>1d\>1\.

The main proof idea behind[Theorem˜4](https://arxiv.org/html/2606.06934#Thmtheorem4)is almost identical to that of[Theorem˜3](https://arxiv.org/html/2606.06934#Thmtheorem3): we also use the counterexample loss functionf=−z​L​wf=\-zLwand two datasetsS≃S′S\\simeq S^\{\\prime\}such that𝖣𝖱​\(η​∇f​\(wt;S\)\)\>𝖣𝖱​\(η​∇f​\(wt′;S′\)\)\\mathsf\{DR\}\(\\eta\\nabla f\(w\_\{t\};S\)\)\>\\mathsf\{DR\}\(\\eta\\nabla f\(w\_\{t\}^\{\\prime\};S^\{\\prime\}\)\)\. However, unlike theΩ​\(Δ​L​T/n\)\\Omega\(\\Delta LT/\\sqrt\{n\}\)bound in[Theorem˜3](https://arxiv.org/html/2606.06934#Thmtheorem3), the boundΩ​\(Δ​L​T\)\\Omega\(\\Delta LT\)in[Theorem˜4](https://arxiv.org/html/2606.06934#Thmtheorem4)does not contain the1/n1/\\sqrt\{n\}factor\. This is because the definition of the uniform stabilityεu​s\\varepsilon\_\{us\}does not depend on the underlying data distribution, which incurs the1/n1/\\sqrt\{n\}factor in[Theorem˜3](https://arxiv.org/html/2606.06934#Thmtheorem3)\(see our discussions after[Theorem˜3](https://arxiv.org/html/2606.06934#Thmtheorem3)and[Section˜A\.2](https://arxiv.org/html/2606.06934#A1.SS2)\)\. See[Section˜A\.3](https://arxiv.org/html/2606.06934#A1.SS3)for the full proof\.

Bassilyet al\.\([2020](https://arxiv.org/html/2606.06934#bib.bib2)\)also derived the lower boundΩ​\(η​L​T/n\+η​L​T\)\\Omega\(\\eta LT/n\+\\eta L\\sqrt\{T\}\)on the uniform argument stability ofGDfor high\-dimensional non\-smooth and convex losses, by exploiting the expansiveness ofGDfor such losses\. Specifically, they designed loss functions so that at thett\-th iteration, oneGDiterate usingSSmovesΘ​\(η\)\\Theta\(\\eta\)along thett\-th coordinate, while the otherGDiterate usingS′≃SS^\{\\prime\}\\simeq Sstays at the origin\. Hence, the distance between two iterates scales withT\\sqrt\{T\}\. We note that𝖣𝖱​\-GD\{\\mathsf\{DR\}\\text\{\\rm\{\-GD\}\}\}is non\-expansive for convex and smooth losses ifd=1d=1\(see[Section˜A\.4](https://arxiv.org/html/2606.06934#A1.SS4)in[Section˜A\.4](https://arxiv.org/html/2606.06934#A1.SS4)\), but the rounding operation incurs a similar effect so that the bound in[Theorem˜4](https://arxiv.org/html/2606.06934#Thmtheorem4)follows\. In addition, while there is a clear gap between the generalization error and the uniform stability of𝖣𝖱​\-GD\{\\mathsf\{DR\}\\text\{\\rm\{\-GD\}\}\}\([Theorems˜2](https://arxiv.org/html/2606.06934#Thmtheorem2)and[4](https://arxiv.org/html/2606.06934#Thmtheorem4)\), to our knowledge, such an observation has not been made forGDwith convex, Lipschitz, but non\-smooth losses due to the lack of tight bounds on the generalization error\.

On the other hand, the uniform stability of𝖣𝖱​\-SGD\{\\mathsf\{DR\}\\text\{\\rm\{\-SGD\}\}\}converges to zero asn→∞n\\to\\infty\. We present the proofs of[Theorems˜5](https://arxiv.org/html/2606.06934#Thmtheorem5)and[6](https://arxiv.org/html/2606.06934#Thmtheorem6)in[Sections˜A\.4](https://arxiv.org/html/2606.06934#A1.SS4)and[A\.5](https://arxiv.org/html/2606.06934#A1.SS5), respectively\.

###### Theorem 5\.

Let𝒵\\mathcal\{Z\}be a sample space,n,T∈ℕn,T\\in\\mathbb\{N\},Δ,L,η,M\>0\\Delta,L,\\eta,M\>0withη<2/M\\eta<2/M,w0∈ℤΔw\_\{0\}\\in\\mathbb\{Z\}\_\{\\Delta\}, andf:ℝ×𝒵→ℝf:\\mathbb\{R\}\\times\\mathcal\{Z\}\\to\\mathbb\{R\}be a convex,LL\-Lipschitz, andMM\-smooth loss function\. Then, forθ=\(η,T,w0\)\\theta=\(\\eta,T,w\_\{0\}\),

εu​s​\(Af,θ𝖣𝖱​\-SGD\)≤\(2​η​L\+Δ\)​L​Tn\.\\displaystyle\\varepsilon\_\{us\}\(A^\{\{\\mathsf\{DR\}\\text\{\\rm\{\-SGD\}\}\}\}\_\{f,\\theta\}\)\\leq\\frac\{\(2\\eta L\+\\Delta\)LT\}\{n\}\.

[Theorem˜5](https://arxiv.org/html/2606.06934#Thmtheorem5)states that whend=1d=1, the uniform stability of𝖣𝖱\\mathsf\{DR\}\-SGD for convex, Lipschitz, and smooth losses is upper bounded by\(2​η​L\+Δ\)​L​T/n\{\(2\\eta L\+\\Delta\)LT\}/\{n\}\. AsΔ→0\\Delta\\to 0, this bound converges to the existing boundO​\(η​L2​T/n\)O\(\\eta L^\{2\}T/n\)forSGD\(Hardtet al\.,[2016](https://arxiv.org/html/2606.06934#bib.bib4)\)\.

Unlike generalization boundsΘ​\(η​L2​T/n\)\\Theta\(\\eta L^\{2\}T/n\)for GD and SGD with a convex, Lipschitz, and smooth loss\(Hardtet al\.,[2016](https://arxiv.org/html/2606.06934#bib.bib4); Chenet al\.,[2018](https://arxiv.org/html/2606.06934#bib.bib6)\), these results show that there is a fundamental difference between𝖣𝖱\\mathsf\{DR\}\-GD and𝖣𝖱\\mathsf\{DR\}\-SGD in terms of the sample complexity: for convex, Lipschitz, and smooth loss function, we haveεg​\(Af,θ𝖣𝖱​\-GD\)≥Ω​\(Δ​L​T/n\)\\varepsilon\_\{g\}\(A^\{\\mathsf\{DR\}\\text\{\\rm\-GD\}\}\_\{f,\\theta\}\)\\geq\\Omega\(\\Delta LT/\\sqrt\{n\}\)in general butεg​\(Af,θ𝖣𝖱​\-SGD\)≤O​\(\(η​L\+Δ\)​L​T/n\)\\varepsilon\_\{g\}\(A^\{\\mathsf\{DR\}\\text\{\\rm\-SGD\}\}\_\{f,\\theta\}\)\\leq O\(\(\\eta L\+\\Delta\)LT/n\)whend=1d=1\. Such a difference is due to the stochasticity in𝖣𝖱​\-SGD\{\\mathsf\{DR\}\\text\{\\rm\{\-SGD\}\}\}\. Specifically, for coupled𝖣𝖱​\-SGD\{\\mathsf\{DR\}\\text\{\\rm\{\-SGD\}\}\}iterates for neighboring datasetsS≃S′S\\simeq S^\{\\prime\}, at each iteration, they choose the same loss function with probability\(n−1\)/n\(n\-1\)/n\. Hence, the coupledSGDiterates are non\-expansive with probability\(n−1\)/n\(n\-1\)/nbefore rounding, which also implies that they are non\-expansive after rounding sinced=1d=1\. Namely, the distance between them can increase with probability1/n1/n\.

We also show the tightness of the bound in[Theorem˜5](https://arxiv.org/html/2606.06934#Thmtheorem5)\. The proof of[Theorem˜6](https://arxiv.org/html/2606.06934#Thmtheorem6)is in[Section˜A\.5](https://arxiv.org/html/2606.06934#A1.SS5)\.

###### Theorem 6\.

Letn,T∈ℕn,T\\in\\mathbb\{N\}andΔ,L,η\>0\\Delta,L,\\eta\>0such thatη​L\>0\.5​Δ\\eta L\>0\.5\\Delta\. Then there exists anLL\-Lipschitz affine loss functionf:ℝ×\{0,1\}→ℝf:\\mathbb\{R\}\\times\\\{0,1\\\}\\to\\mathbb\{R\}such that forθ=\(η,T,w0=0\)\\theta=\(\\eta,T,w\_\{0\}=0\),

εu​s​\(Af,θ𝖣𝖱​\-SGD\)≳\(2​η​L−Δ\)​L​Tn\.\\displaystyle\\varepsilon\_\{us\}\(A\_\{f,\\theta\}^\{\\mathsf\{DR\}\\text\{\\rm\{\-SGD\}\}\}\)\\gtrsim\\frac\{\(2\\eta L\-\\Delta\)LT\}\{n\}\.

However, the bound linear toTTin[Theorem˜5](https://arxiv.org/html/2606.06934#Thmtheorem5)does not extend to generald\>1d\>1\. Our next result shows that the uniform stability of𝖣𝖱​\-SGD\{\\mathsf\{DR\}\\text\{\\rm\{\-SGD\}\}\}can increase withT2T^\{2\}whenT/n≤1T/n\\leq 1\.

###### Theorem 7\.

Letn,d,T∈ℕn,d,T\\in\\mathbb\{N\}withn,d,T≥2n,d,T\\geq 2, andΔ\>0\\Delta\>0\. Then, there exist a convex,Δ\\Delta\-Lipschitz and\(910​T2\+85\)\\left\(\\frac\{9\}\{10T^\{2\}\}\+\\frac\{8\}\{5\}\\right\)\-smooth loss functionf:ℝd×\{0,1\}→ℝf:\\mathbb\{R\}^\{d\}\\times\\\{0,1\\\}\\to\\mathbb\{R\}such that forθ=\(η=1,T,w0=𝟎d\)\\theta=\(\\eta=1,T,w\_\{0\}=\\boldsymbol\{0\}\_\{d\}\),

εu​s​\(Af,θ𝖣𝖱​\-SGD\)≳min⁡\{1,Tn\}⋅Δ2​T\.\\displaystyle\\varepsilon\_\{us\}\(A\_\{f,\\theta\}^\{\\mathsf\{DR\}\\text\{\\rm\{\-SGD\}\}\}\)\\gtrsim\\min\\left\\\{1,\\frac\{T\}\{n\}\\right\\\}\\cdot\\Delta^\{2\}T\.

[Theorem˜7](https://arxiv.org/html/2606.06934#Thmtheorem7)also implies the difference between the generalization properties ofSGDand𝖣𝖱​\-SGD\{\\mathsf\{DR\}\\text\{\\rm\{\-SGD\}\}\}whend\>1d\>1sinceεu​s​\(Af,θSGD\)=O​\(η​L2​T/n\)\\varepsilon\_\{us\}\(A^\{\\text\{\\rm\{SGD\}\}\}\_\{f,\\theta\}\)=O\(\\eta L^\{2\}T/n\)\(Hardtet al\.,[2016](https://arxiv.org/html/2606.06934#bib.bib4)\)\. Here, the main difference betweenSGDand𝖣𝖱​\-SGD\{\\mathsf\{DR\}\\text\{\\rm\{\-SGD\}\}\}is that whend\>1d\>1,𝖣𝖱​\-SGD\{\\mathsf\{DR\}\\text\{\\rm\{\-SGD\}\}\}can be expansive due to the rounding scheme, even if a loss function is convex and smooth\. Using this property, we design a counterexample loss functionf:ℝd×\{0,1\}→ℝf:\\mathbb\{R\}^\{d\}\\times\\\{0,1\\\}\\to\\mathbb\{R\}that satisfies𝖣𝖱​\(η​∇f​\(𝟎d;0\)\)=𝟎d\\mathsf\{DR\}\(\\eta\\nabla f\(\\boldsymbol\{0\}\_\{d\};0\)\)=\\boldsymbol\{0\}\_\{d\}and𝖣𝖱​\(η​∇f​\(𝟎d;1\)\)=u≠0\\mathsf\{DR\}\(\\eta\\nabla f\(\\boldsymbol\{0\}\_\{d\};1\)\)=u\\neq 0, i\.e\.,𝖣𝖱​\-SGD\{\\mathsf\{DR\}\\text\{\\rm\{\-SGD\}\}\}iterate never moves with datasetS=\(0,…,0\)S=\(0,\\dots,0\)but it can move withS′=\(1,0,…,0\)≃SS^\{\\prime\}=\(1,0,\\dots,0\)\\simeq S, with probability1/n1/n\. Here, we designffso that once the𝖣𝖱​\-SGD\{\\mathsf\{DR\}\\text\{\\rm\{\-SGD\}\}\}iterate touchesuu, then for each iteration, the distance between the iterate and the origin increases byΘ​\(Δ\)\\Theta\(\\Delta\)\. Here, theΘ​\(T/n\)\\Theta\(T/n\)term of the bound in[Theorem˜7](https://arxiv.org/html/2606.06934#Thmtheorem7)can be considered as the probability that one is sampled fromS′S^\{\\prime\}in the firstT/2T/2𝖣𝖱​\-SGD\{\\mathsf\{DR\}\\text\{\\rm\{\-SGD\}\}\}iterations\. Once one is sampled, the distance between theTT\-th𝖣𝖱​\-SGD\{\\mathsf\{DR\}\\text\{\\rm\{\-SGD\}\}\}iterate forS′S^\{\\prime\}and the origin will beΘ​\(Δ​T\)\\Theta\(\\Delta T\), which results in aΘ​\(Δ2​T\)\\Theta\(\\Delta^\{2\}T\)difference in the loss function valuesf​\(Af,θ𝖣𝖱​\-SGD​\(S\);0\)f\(A^\{\\mathsf\{DR\}\\text\{\\rm\{\-SGD\}\}\}\_\{f,\\theta\}\(S\);0\)andf​\(Af,θ𝖣𝖱​\-SGD​\(S′\);0\)f\(A^\{\\mathsf\{DR\}\\text\{\\rm\{\-SGD\}\}\}\_\{f,\\theta\}\(S^\{\\prime\}\);0\)\. For the full proof, see[Section˜A\.6](https://arxiv.org/html/2606.06934#A1.SS6)\.

We next show that the bound in[Theorem˜7](https://arxiv.org/html/2606.06934#Thmtheorem7)is tight in general, by providing a matching upper bound in the following theorem\. The proof of[Theorem˜8](https://arxiv.org/html/2606.06934#Thmtheorem8)is presented in[Section˜A\.7](https://arxiv.org/html/2606.06934#A1.SS7)\.

###### Theorem 8\.

Let𝒵\\mathcal\{Z\}be a sample space,n,d,T∈ℕn,d,T\\in\\mathbb\{N\},Δ,L,η\>0\\Delta,L,\\eta\>0,w0∈ℤΔdw\_\{0\}\\in\\mathbb\{Z\}\_\{\\Delta\}^\{d\}, andf:ℝd×𝒵→ℝf:\\mathbb\{R\}^\{d\}\\times\\mathcal\{Z\}\\to\\mathbb\{R\}be anLL\-Lipschitz loss function\. Then, forθ=\(η,T,w0\)\\theta=\(\\eta,T,w\_\{0\}\),

εu​s​\(Af,θ𝖣𝖱​\-SGD\)≤min⁡\{1,Tn\}⋅8​η​L2​T\.\\displaystyle\\varepsilon\_\{us\}\(A\_\{f,\\theta\}^\{\\mathsf\{DR\}\\text\{\\rm\{\-SGD\}\}\}\)\\leq\\min\\left\\\{1,\\frac\{T\}\{n\}\\right\\\}\\cdot 8\\eta L^\{2\}T\.

We remark that forSGDwith non\-smooth and convex losses, the uniform argument stability isΘ​\(η​L​T/n\+min⁡\{1,T/n\}​η​L​T\)\\Theta\(\\eta LT/n\+\\min\\\{1,T/n\\\}\\eta L\\sqrt\{T\}\)\(Bassilyet al\.,[2020](https://arxiv.org/html/2606.06934#bib.bib2)\)\. This bound is also derived using the expansiveness of counterexample losses\. While the distance between the coupled𝖣𝖱​\-SGD\{\\mathsf\{DR\}\\text\{\\rm\{\-SGD\}\}\}iterates can increase byΔ\\Deltaper iteration, the squared distance between the coupledSGDiterates can increase byΘ​\(η2\)\\Theta\(\\eta^\{2\}\), yielding theT3/2T^\{3/2\}factor in the bound\.

## 4Generalization error and uniform stability of𝖲𝖱​\-GD\{\\mathsf\{SR\}\\text\{\\rm\{\-GD\}\}\}and𝖲𝖱​\-SGD\{\\mathsf\{SR\}\\text\{\\rm\{\-SGD\}\}\}

We next present our bounds for𝖲𝖱​\-GD\{\\mathsf\{SR\}\\text\{\\rm\{\-GD\}\}\}and𝖲𝖱​\-SGD\{\\mathsf\{SR\}\\text\{\\rm\{\-SGD\}\}\}\.

###### Theorem 9\.

Letη,L,Δ\>0\\eta,L,\\Delta\>0andn,d∈ℕn,d\\in\\mathbb\{N\}withn≥14n\\geq 14andη​L<Δ​d\\eta L<\\Delta\\sqrt\{d\}\. There exist a data distribution𝒟\\mathcal\{D\}over\{0,1\}\\\{0,1\\\}and a convex,LL\-Lipschitz, and\(4​L/Δ\)\(4L/\\Delta\)\-smooth loss functionf:ℝd×\{0,1\}→ℝf:\\mathbb\{R\}^\{d\}\\times\\\{0,1\\\}\\to\\mathbb\{R\}such that forθ=\(η,T=1,w0=𝟎d\)\\theta=\(\\eta,T=1,w\_\{0\}=\\boldsymbol\{0\}\_\{d\}\),

εg​\(Af,θ𝖲𝖱​\-GD,𝒟\),εg​\(Af,θ𝖲𝖱​\-SGD,𝒟\)≳min⁡\{Δ,η​L​d1/2\}​η​L3​d1/2n\.\\displaystyle\\varepsilon\_\{g\}\(A\_\{f,\\theta\}^\{\\mathsf\{SR\}\\text\{\\rm\{\-GD\}\}\},\\mathcal\{D\}\),\\varepsilon\_\{g\}\(A\_\{f,\\theta\}^\{\\mathsf\{SR\}\\text\{\\rm\{\-SGD\}\}\},\\mathcal\{D\}\)\\gtrsim\\frac\{\\sqrt\{\\min\\\{\\Delta,\\eta Ld^\{1/2\}\\\}\\eta L^\{3\}d^\{1/2\}\}\}\{n\}\.

[Theorem˜9](https://arxiv.org/html/2606.06934#Thmtheorem9)shows that the generalization error of𝖲𝖱​\-GD\{\\mathsf\{SR\}\\text\{\\rm\{\-GD\}\}\}and𝖲𝖱​\-SGD\{\\mathsf\{SR\}\\text\{\\rm\{\-SGD\}\}\}is inherently dependent on the ambient dimension: there exist a data distribution𝒟\\mathcal\{D\}and a convex,LL\-Lipschitz, and\(4​L/Δ\)\(4L/\\Delta\)\-smooth loss functionffsuch that the generalization error of𝖲𝖱​\-GD\{\\mathsf\{SR\}\\text\{\\rm\{\-GD\}\}\}and𝖲𝖱​\-SGD\{\\mathsf\{SR\}\\text\{\\rm\{\-SGD\}\}\}withT=1T=1can scale with \(at least\)d1/4d^\{1/4\}\. This result shows that the generalization properties of𝖲𝖱​\-GD\{\\mathsf\{SR\}\\text\{\\rm\{\-GD\}\}\}and𝖲𝖱​\-SGD\{\\mathsf\{SR\}\\text\{\\rm\{\-SGD\}\}\}are fundamentally different from those ofGDandSGD, which are independent ofdd\(Hardtet al\.,[2016](https://arxiv.org/html/2606.06934#bib.bib4)\)\.

Here, the dimension dependency in[Theorem˜9](https://arxiv.org/html/2606.06934#Thmtheorem9)is from the stochastic rounding\. ForGDandSGD, the distance between coupled iterates for neighboring datasets increases at mostO​\(η​L/n\)O\(\\eta L/n\)\. Then, afterTTiterations, the distance can be bounded byO​\(η​L​T/n\)O\(\\eta LT/n\), and hence, the difference in the function value can be bounded byO​\(η​L2​T/n\)O\(\\eta L^\{2\}T/n\)\. However, if we apply the stochastic rounding to two points with distanceℓ\\ell, the resulting distance can beΩ​\(ℓ​d\)\\Omega\(\\ell\\sqrt\{d\}\)in expectation \(e\.g\., consider one point is𝟎d\\boldsymbol\{0\}\_\{d\}and the other point is\(ℓ/d,…,ℓ/d\)\(\\ell/\\sqrt\{d\},\\dots,\\ell/\\sqrt\{d\}\)\)\. Motivated by this observation, we design the loss function and a data distribution so that the generalization error scales withdd\. See[Section˜B\.1](https://arxiv.org/html/2606.06934#A2.SS1)for the full proof\.

We now shift our focus to the stability bounds of𝖲𝖱​\-GD\{\\mathsf\{SR\}\\text\{\\rm\{\-GD\}\}\}and𝖲𝖱​\-SGD\{\\mathsf\{SR\}\\text\{\\rm\{\-SGD\}\}\}for generalT\>1T\>1\. To this end, we present an upper bound of the uniform argument stability of𝖲𝖱​\-SGD\{\\mathsf\{SR\}\\text\{\\rm\{\-SGD\}\}\}in the following theorem\.

###### Theorem 10\.

Let𝒵\\mathcal\{Z\}be a sample space,n,d,T∈ℕn,d,T\\in\\mathbb\{N\},Δ,η,L,M\>0\\Delta,\\eta,L,M\>0withη<2/M\\eta<2/M,w0∈ℤΔdw\_\{0\}\\in\\mathbb\{Z\}\_\{\\Delta\}^\{d\}, andf:ℝd×𝒵→ℝf:\\mathbb\{R\}^\{d\}\\times\\mathcal\{Z\}\\to\\mathbb\{R\}be a convex,LL\-Lipschitz, andMM\-smooth loss function\. Then, forθ=\(η,T,w0\)\\theta=\(\\eta,T,w\_\{0\}\),

εu​a​s​\(Af,θ𝖲𝖱​\-SGD\)≤8​η​L​Tn\+min⁡\{1,Tn\}⋅2​Δ​η​L​d1/2​T\.\\displaystyle\\varepsilon\_\{uas\}\(A\_\{f,\\theta\}^\{\\mathsf\{SR\}\\text\{\\rm\{\-SGD\}\}\}\)\\leq\\frac\{8\\eta LT\}\{n\}\+\\min\\left\\\{1,\\frac\{T\}\{n\}\\right\\\}\\cdot\\sqrt\{2\\Delta\\eta Ld^\{1/2\}T\}\.

[Theorem˜10](https://arxiv.org/html/2606.06934#Thmtheorem10)provides an upper bound onεu​a​s​\(Af,θ𝖲𝖱​\-SGD\)\\varepsilon\_\{uas\}\(A\_\{f,\\theta\}^\{\\mathsf\{SR\}\\text\{\\rm\{\-SGD\}\}\}\)that scales withd1/4d^\{1/4\}; such dimension dependence is unavoidable in general as shown by[Theorem˜9](https://arxiv.org/html/2606.06934#Thmtheorem9)\. Unlike bounds forSGD, which scales withTT, our bound scales withT3/2T^\{3/2\}as in the uniform argument stability bound forSGDon a non\-smooth convex loss:εu​a​s​\(Af,θSGD\)=O​\(min⁡\{1,T/n\}​η​L​T\+η​L​T/n\)\\varepsilon\_\{uas\}\(A\_\{f,\\theta\}^\{\\text\{\\rm\{SGD\}\}\}\)=O\(\\min\\\{1,T/n\\\}\\eta L\\sqrt\{T\}\+\\eta LT/n\)\(Bassilyet al\.,[2020](https://arxiv.org/html/2606.06934#bib.bib2)\)\. This follows from the potential expansiveness of𝖲𝖱​\-SGD\{\\mathsf\{SR\}\\text\{\\rm\{\-SGD\}\}\}even for convex and smooth losses\. Here, themin⁡\{1,T/n\}\\min\\\{1,T/n\\\}term corresponds to the probability that different examples \(i\.e\.z1z\_\{1\}andz1′z\_\{1\}^\{\\prime\}\) are selected from neighboring datasets during the coupled SGD iterates, and theΘ​\(Δ​η​L​d1/2​T\)\\Theta\(\\sqrt\{\\Delta\\eta Ld^\{1/2\}T\}\)term is the accumulated distance between two coupled𝖲𝖱​\-SGD\{\\mathsf\{SR\}\\text\{\\rm\{\-SGD\}\}\}iterates after the differing samples are selected\. We also note that this bound recovers the uniform argument stability bound forSGDasΔ→0\\Delta\\to 0\. For the full proof of[Theorem˜10](https://arxiv.org/html/2606.06934#Thmtheorem10), see[Section˜B\.2](https://arxiv.org/html/2606.06934#A2.SS2)\.

We also provide an upper bound on the uniform argument stability of𝖲𝖱​\-GD\{\\mathsf\{SR\}\\text\{\\rm\{\-GD\}\}\}in the following theorem, whose proof is presented in[Section˜B\.3](https://arxiv.org/html/2606.06934#A2.SS3)\.

###### Theorem 11\.

Let𝒵\\mathcal\{Z\}be a sample space,n,d,T∈ℕn,d,T\\in\\mathbb\{N\},Δ,η,L,M\>0\\Delta,\\eta,L,M\>0withη<2/M\\eta<2/M,w0∈ℤΔdw\_\{0\}\\in\\mathbb\{Z\}\_\{\\Delta\}^\{d\}, andf:ℝd×𝒵→ℝf:\\mathbb\{R\}^\{d\}\\times\\mathcal\{Z\}\\to\\mathbb\{R\}be a convex,LL\-Lipschitz andMM\-smooth loss function\. Then, forθ=\(η,T,w0\)\\theta=\(\\eta,T,w\_\{0\}\),

εu​a​s​\(Af,θ𝖲𝖱​\-GD\)≤6​η​L​Tn\+min⁡\{1,2​η​L​d​Tn​Δ\}⋅Δ2\+2​Δ​η​L​d1/2​\(1n\+T−1\)\.\\displaystyle\\varepsilon\_\{uas\}\(A\_\{f,\\theta\}^\{\\mathsf\{SR\}\\text\{\\rm\{\-GD\}\}\}\)\\leq\\frac\{6\\eta LT\}\{n\}\+\\min\\left\\\{1,\\frac\{2\\eta L\\sqrt\{d\}T\}\{n\\Delta\}\\right\\\}\\cdot\\sqrt\{\\Delta^\{2\}\+2\\Delta\\eta Ld^\{1/2\}\\left\(\\frac\{1\}\{n\}\+T\-1\\right\)\}\.

Unlike the lower bound on the uniform stability of𝖣𝖱​\-GD\{\\mathsf\{DR\}\\text\{\\rm\{\-GD\}\}\}\([Theorem˜4](https://arxiv.org/html/2606.06934#Thmtheorem4)\), the bound in[Theorem˜11](https://arxiv.org/html/2606.06934#Thmtheorem11)converges to zero asn→∞n\\to\\inftywhenΔ\\Deltais independent ofnn\. In𝖣𝖱​\-GD\{\\mathsf\{DR\}\\text\{\\rm\{\-GD\}\}\}, even anO​\(1/n\)O\(1/n\)change in the gradient of empirical loss can deterministically move the update across a rounding threshold, producing a separation of orderΔ\\Deltabetween two iterates on neighboring datasets even after the first iterate \(see our discussion after[Theorem˜4](https://arxiv.org/html/2606.06934#Thmtheorem4)\)\. In contrast, under stochastic rounding, a small change in the gradient of empirical loss causes different rounded values only with probability proportional to the magnitude of that change relative toΔ\\Delta\. Therefore, the probability that two𝖲𝖱​\-GD\{\\mathsf\{SR\}\\text\{\\rm\{\-GD\}\}\}iterates first separate overTTiterations is of ordermin⁡\{1,η​L​d​T/\(n​Δ\)\}\\min\\\{1,\\eta L\\sqrt\{d\}T/\(n\\Delta\)\\\}, yielding the sample\-dependent bound in[Theorem˜11](https://arxiv.org/html/2606.06934#Thmtheorem11)\. Here, the following termΘ​\(Δ2\+Δ​η​L​d1/2​T\)\\Theta\(\\sqrt\{\\Delta^\{2\}\+\\Delta\\eta Ld^\{1/2\}T\}\)corresponds to the accumulated distance between two𝖲𝖱​\-GD\{\\mathsf\{SR\}\\text\{\\rm\{\-GD\}\}\}iterates after the first separation\. Specifically, the squared distance between the two𝖲𝖱​\-GD\{\\mathsf\{SR\}\\text\{\\rm\{\-GD\}\}\}iterates is at mostO​\(Δ2\+Δ​η​L​d/n\)O\(\\Delta^\{2\}\+\\Delta\\eta L\\sqrt\{d\}/n\)at the first separation, and then increases at mostO​\(Δ​η​L​d\)O\(\\Delta\\eta L\\sqrt\{d\}\)per iteration\. See[Section˜B\.3](https://arxiv.org/html/2606.06934#A2.SS3)for more details\.

The bound in[Theorem˜11](https://arxiv.org/html/2606.06934#Thmtheorem11)can be interpreted differently depending on the value ofΔ\\Delta\. ForΔ≲Δ1=Θ​\(η​L​d​T/n\)\\Delta\\lesssim\\Delta\_\{1\}=\\Theta\(\\eta L\\sqrt\{d\}T/n\), the bound becomesεu​a​s​\(Af,θ𝖲𝖱​\-GD\)=O​\(η​L​T/n\+Δ​η​L​d1/2​T\)\\varepsilon\_\{uas\}\(A\_\{f,\\theta\}^\{\\mathsf\{SR\}\\text\{\\rm\{\-GD\}\}\}\)=O\(\\eta LT/n\+\\sqrt\{\\Delta\\eta Ld^\{1/2\}T\}\)\. ForΔ1≲Δ≲Δ2=Θ​\(η​L​d​T\)\\Delta\_\{1\}\\lesssim\\Delta\\lesssim\\Delta\_\{2\}=\\Theta\(\\eta L\\sqrt\{d\}T\), the bound becomesO​\(η3/2​L3/2​d3/4​T3/2/\(n​Δ1/2\)\)O\(\\eta^\{3/2\}L^\{3/2\}d^\{3/4\}T^\{3/2\}/\(n\\Delta^\{1/2\}\)\), which can scale withT3/2T^\{3/2\}, e\.g\., for constantΔ\\DeltawhenΔ1=O​\(1\)\\Delta\_\{1\}=O\(1\)\. IfΔ≳Δ2\\Delta\\gtrsim\\Delta\_\{2\}, the bound becomesO​\(η​L​d​T/n\)O\(\\eta L\\sqrt\{d\}T/n\)\. As in the𝖲𝖱​\-SGD\{\\mathsf\{SR\}\\text\{\\rm\{\-SGD\}\}\}case, we recover the uniform argument stability bound forGDasΔ→0\\Delta\\to 0\.

While the bounds in[Theorems˜10](https://arxiv.org/html/2606.06934#Thmtheorem10)and[11](https://arxiv.org/html/2606.06934#Thmtheorem11)can scale withT3/2/nT^\{3/2\}/n, our next results show that the bounds can be relaxed toT/nT/n\(orT/n\\sqrt\{T/n\}\) if losses are separable \(i\.e\., sum of coordinate\-wise functions\) as in the following theorem\. The proof of[Theorem˜12](https://arxiv.org/html/2606.06934#Thmtheorem12)is presented in[Section˜B\.4](https://arxiv.org/html/2606.06934#A2.SS4)\.

###### Theorem 12\.

Let𝒵\\mathcal\{Z\}be a sample space,d,n,T∈ℕd,n,T\\in\\mathbb\{N\},Δ,η,M,L\>0\\Delta,\\eta,M,L\>0such thatη<1/M\\eta<1/M,w0∈ℤΔdw\_\{0\}\\in\\mathbb\{Z\}\_\{\\Delta\}^\{d\}, andf:ℝd×𝒵→ℝf:\\mathbb\{R\}^\{d\}\\times\\mathcal\{Z\}\\to\\mathbb\{R\}be a convex,LL\-Lipschitz,MM\-smooth, and separable loss function\. Then, forθ=\(η,T,w0\)\\theta=\(\\eta,T,w\_\{0\}\),

εu​a​s​\(Af,θ𝖲𝖱​\-GD\)\\displaystyle\\varepsilon\_\{uas\}\(A\_\{f,\\theta\}^\{\\mathsf\{SR\}\\text\{\\rm\{\-GD\}\}\}\)≤6​η​L​Tn\+2​Δ​η​L​d1/2​Tn,εu​a​s​\(Af,θ𝖲𝖱​\-SGD\)≤2​η​L​Tn\+T​2​Δ​η​L​d1/2n\.\\displaystyle\\leq\\frac\{6\\eta LT\}\{n\}\+\\sqrt\{\\frac\{2\\Delta\\eta Ld^\{1/2\}T\}\{n\}\},\\quad\\varepsilon\_\{uas\}\(A\_\{f,\\theta\}^\{\\mathsf\{SR\}\\text\{\\rm\{\-SGD\}\}\}\)\\leq\\frac\{2\\eta LT\}\{n\}\+\\frac\{T\\sqrt\{2\\Delta\\eta Ld^\{1/2\}\}\}\{n\}\.

Unlike the potential expansiveness of𝖲𝖱​\-GD\{\\mathsf\{SR\}\\text\{\\rm\{\-GD\}\}\}and𝖲𝖱​\-SGD\{\\mathsf\{SR\}\\text\{\\rm\{\-SGD\}\}\}for convex and smooth losses, they are non\-expansive if the losses are additionally separable\. By exploiting this property, we can better bound the accumulated distance between coupled iterates, and derive the bounds in[Theorem˜12](https://arxiv.org/html/2606.06934#Thmtheorem12)\.

We lastly show that the bounds in[Theorem˜12](https://arxiv.org/html/2606.06934#Thmtheorem12)are tight in general, as stated in the following theorem\. The proof of[Theorem˜13](https://arxiv.org/html/2606.06934#Thmtheorem13)is in[Section˜B\.5](https://arxiv.org/html/2606.06934#A2.SS5)\.

###### Theorem 13\.

Letη,Δ,L\>0\\eta,\\Delta,L\>0andn,T,d∈ℕn,T,d\\in\\mathbb\{N\}\. Then, there exists anLL\-Lipschitz affine loss functionf:ℝd×\{0,1\}→ℝf:\\mathbb\{R\}^\{d\}\\times\\\{0,1\\\}\\to\\mathbb\{R\}such that forθ=\(η,T,w0=𝟎d\)\\theta=\(\\eta,T,w\_\{0\}=\\boldsymbol\{0\}\_\{d\}\), ifη<n​Δ​d/L\\eta<n\\Delta\\sqrt\{d\}/L, then

εu​a​s​\(Af,θ𝖲𝖱​\-GD\)\\displaystyle\\varepsilon\_\{uas\}\(A\_\{f,\\theta\}^\{\\mathsf\{SR\}\\text\{\\rm\{\-GD\}\}\}\)≳η​L​Tn\+min⁡\{n​Δ,η​L​d1/2​T\}​η​L​d1/2​Tn\.\\displaystyle\\gtrsim\\frac\{\\eta LT\}\{n\}\+\\frac\{\\sqrt\{\\min\\\{n\\Delta,\\eta Ld^\{1/2\}T\\\}\\eta Ld^\{1/2\}T\}\}\{n\}\.In addition, ifη<Δ​d/L\\eta<\\Delta\\sqrt\{d\}/L, then

εu​a​s​\(Af,θ𝖲𝖱​\-SGD\)\\displaystyle\\varepsilon\_\{uas\}\(A\_\{f,\\theta\}^\{\\mathsf\{SR\}\\text\{\\rm\{\-SGD\}\}\}\)≳min⁡\{1,Tn\}​\(η​L\+min⁡\{Δ,η​L​d1/2\}​η​L​d1/2\)\.\\displaystyle\\gtrsim\\min\\left\\\{1,\\frac\{T\}\{n\}\\right\\\}\\left\(\\eta L\+\\sqrt\{\\min\\\{\\Delta,\\eta Ld^\{1/2\}\\\}\\eta Ld^\{1/2\}\}\\right\)\.

Together with[Theorem˜12](https://arxiv.org/html/2606.06934#Thmtheorem12),[Theorem˜13](https://arxiv.org/html/2606.06934#Thmtheorem13)can tightly characterize the uniform argument stability of𝖲𝖱​\-GD\{\\mathsf\{SR\}\\text\{\\rm\{\-GD\}\}\}and𝖲𝖱​\-SGD\{\\mathsf\{SR\}\\text\{\\rm\{\-SGD\}\}\}when the loss function is separable\. Specifically, forη​L/\(n​d\)<Δ≲η​L​d​T/n\\eta L/\(n\\sqrt\{d\}\)<\\Delta\\lesssim\\eta L\\sqrt\{d\}T/n, we haveεu​a​s​\(Af,θ𝖲𝖱​\-GD\)=Θ​\(Δ​η​L​d1/2​T/n\)\\varepsilon\_\{uas\}\(A\_\{f,\\theta\}^\{\\mathsf\{SR\}\\text\{\\rm\{\-GD\}\}\}\)=\\Theta\(\\sqrt\{\\Delta\\eta Ld^\{1/2\}T/n\}\)\. Likewise, forT<nT<nandη​L/d<Δ≲η​L​d\\eta L/\\sqrt\{d\}<\\Delta\\lesssim\\eta L\\sqrt\{d\}, it holds thatεu​a​s​\(Af,θ𝖲𝖱​\-SGD\)=Θ​\(Δ​η​L​d1/2​T/n\)\\varepsilon\_\{uas\}\(A\_\{f,\\theta\}^\{\\mathsf\{SR\}\\text\{\\rm\{\-SGD\}\}\}\)=\\Theta\(\\sqrt\{\\Delta\\eta Ld^\{1/2\}\}T/n\)\. This result shows a qualitative difference between𝖲𝖱​\-GD\{\\mathsf\{SR\}\\text\{\\rm\{\-GD\}\}\}and𝖲𝖱​\-SGD\{\\mathsf\{SR\}\\text\{\\rm\{\-SGD\}\}\}: since𝖲𝖱​\-SGD\{\\mathsf\{SR\}\\text\{\\rm\{\-SGD\}\}\}contains the extraT/n\\sqrt\{T/n\}compared to𝖲𝖱​\-GD\{\\mathsf\{SR\}\\text\{\\rm\{\-GD\}\}\},𝖲𝖱​\-SGD\{\\mathsf\{SR\}\\text\{\\rm\{\-SGD\}\}\}is more favorable in sample complexity, but less favorable in its dependence onTT\.

## 5Discussions

Limitations\.Our theoretical analysis isolates parameter rounding in idealizedGD/SGDupdates \(see[Section˜2\.3](https://arxiv.org/html/2606.06934#S2.SS3)\)\. Accordingly, it may not cover several ingredients in real implementations\. First, our model assumes that the gradient of the real\-valued loss is computed exactly\. However, automatic differentiation over machine\-representable numbers can exhibit incorrect behavior even when the loss is differentiable due to the round\-off error in the intermediate computations\. Second, modern low\-precision training often relies on quantization\-aware training\(Jacobet al\.,[2017](https://arxiv.org/html/2606.06934#bib.bib40)\), which maintains high\-precision parameters while training a model with fake\-quantization operators in the forward pass\. Thus, unlike our setup, it does not directly optimize over a discrete parameter space\.

Extension to non\-convex loss functions\.Although our main results are stated for convex, Lipschitz, and smooth losses, our stability upper bound results can be extended beyond this class\. In particular, the proofs of[Theorems˜5](https://arxiv.org/html/2606.06934#Thmtheorem5),[8](https://arxiv.org/html/2606.06934#Thmtheorem8),[10](https://arxiv.org/html/2606.06934#Thmtheorem10),[11](https://arxiv.org/html/2606.06934#Thmtheorem11)and[12](https://arxiv.org/html/2606.06934#Thmtheorem12)only use the loss values and gradients at the points inℤΔd\\mathbb\{Z\}\_\{\\Delta\}^\{d\}that can be visited by the rounded iteration\. Hence, these results may also apply to non\-convex losses whose gradients onℤΔd\\mathbb\{Z\}\_\{\\Delta\}^\{d\}can be interpolated by those of a convex, Lipschitz, and smooth function\. This interpolation condition can be checked using known characterizations of smooth convex interpolation\. For example,Tayloret al\.\([2015](https://arxiv.org/html/2606.06934#bib.bib42)\)show that for any finite collection\{\(xi,gi,fi\)\}i=1n⊂ℝd×ℝd×ℝ\\\{\(x\_\{i\},g\_\{i\},f\_\{i\}\)\\\}\_\{i=1\}^\{n\}\\subset\\mathbb\{R\}^\{d\}\\times\\mathbb\{R\}^\{d\}\\times\\mathbb\{R\}, where eachxix\_\{i\}is an interpolation point with gradientgig\_\{i\}and valuefif\_\{i\}, there exists a convex andMM\-smooth functionf:ℝd→ℝf:\\mathbb\{R\}^\{d\}\\to\\mathbb\{R\}satisfyingf​\(xi\)=fif\(x\_\{i\}\)=f\_\{i\}and∇f​\(xi\)=gi\\nabla f\(x\_\{i\}\)=g\_\{i\}for alli∈\[n\]i\\in\[n\]if and only if

fi≥fj\+gj⊤​\(xi−xj\)\+12​M​‖gi−gj‖22​for all​i​and​j\.\\displaystyle f\_\{i\}\\geq f\_\{j\}\+g\_\{j\}^\{\\top\}\(x\_\{i\}\-x\_\{j\}\)\+\\frac\{1\}\{2M\}\\\|g\_\{i\}\-g\_\{j\}\\\|\_\{2\}^\{2\}~~\\text\{for all \}i~\\text\{and \}j\.
Extension to floating\-point numbers\.Unlike fixed\-point numbers, floating\-point numbers form a non\-uniform grid: the spacing between representable numbers depends on the exponent\. Thus, the magnitude of rounding errors depends on the scale of the iterates themselves\. Nevertheless, our analysis may extend to floating\-point arithmetic in regimes where the iterates remain within a fixed exponent range\. In such a local regime, the set of representable numbers behaves similarly to fixed\-point numbers\. However, extending our results to the full floating\-point setting requires controlling changes in the exponent along the optimization trajectory, as well as the resulting scale\-dependent rounding errors\. Developing a stability and generalization theory that captures these effects is an important direction for future work\.

## 6Conclusion

In this work, we analyze the generalization error and uniform stability ofGDandSGDwith deterministic and stochastic rounding onto fixed\-point spaces, on convex, Lipschitz, and smooth losses\. We first show that deterministic rounding can degrade the generalization error toΘ​\(T/n\)\\Theta\(T/\\sqrt\{n\}\), with a vacuousΩ​\(T\)\\Omega\(T\)uniform stability bound\. In contrast,SGDwith deterministic rounding yields tight, dimension\-dependent bounds:O​\(T/n\)O\(T/n\)for one dimension andO​\(T2/n\)O\(T^\{2\}/n\)for higher dimensions, which is qualitatively different fromSGDwithout rounding\. Furthermore, we show that both forGDandSGDwith stochastic rounding, the generalization error can scale with the ambient dimension\. We also provide the upper and lower bounds on uniform argument stability, which are tight when the loss function can be represented as a sum of coordinate\-wise functions\. We believe our results contribute to bridging the gap between theoretical optimization and the real implementation\.

## References

- R\. Bassily, V\. Feldman, C\. Guzmán, and K\. Talwar \(2020\)Stability of stochastic gradient descent on nonsmooth convex losses\.InAnnual Conference on Neural Information Processing Systems \(NeurIPS\),Cited by:[Table 1](https://arxiv.org/html/2606.06934#S1.T1),[Table 1](https://arxiv.org/html/2606.06934#S1.T1.12.12.6),[Table 1](https://arxiv.org/html/2606.06934#S1.T1.15.15.3.2),[Table 1](https://arxiv.org/html/2606.06934#S1.T1.18.18.6.2),[§1](https://arxiv.org/html/2606.06934#S1.p1.1),[§1](https://arxiv.org/html/2606.06934#S1.p4.4),[§3](https://arxiv.org/html/2606.06934#S3.p17.7),[§3](https://arxiv.org/html/2606.06934#S3.p9.15),[§4](https://arxiv.org/html/2606.06934#S4.p5.15)\.
- L\. Bottou, F\. E\. Curtis, and J\. Nocedal \(2018\)Optimization methods for large\-scale machine learning\.SIAM Review60\(2\),pp\. 223–311\.External Links:https://doi\.org/10\.1137/16M1080173Cited by:[§1](https://arxiv.org/html/2606.06934#S1.p1.1)\.
- O\. Bousquet and A\. Elisseeff \(2002\)Stability and generalization\.\.Journal of Machine Learning Research2,pp\. 499–526\.Cited by:[§1](https://arxiv.org/html/2606.06934#S1.p4.4),[Lemma 1](https://arxiv.org/html/2606.06934#S2.SS4.1)\.
- Y\. Chen, C\. Jin, and B\. Yu \(2018\)Stability and convergence trade\-off of iterative optimization algorithms\.arXiv preprint arXiv:1804\.01619\.Cited by:[Table 1](https://arxiv.org/html/2606.06934#S1.T1.14.14.2.2),[§1](https://arxiv.org/html/2606.06934#S1.p4.4),[§3](https://arxiv.org/html/2606.06934#S3.p12.14),[§3](https://arxiv.org/html/2606.06934#S3.p4.5)\.
- J\. Duchi, E\. Hazan, and Y\. Singer \(2011\)Adaptive subgradient methods for online learning and stochastic optimization\.Journal of Machine Learning Research12\(61\),pp\. 2121–2159\.Cited by:[§1](https://arxiv.org/html/2606.06934#S1.p1.1)\.
- V\. Feldman \(2016\)Generalization of ERM in stochastic convex optimization: the dimension strikes back\.InAnnual Conference on Neural Information Processing Systems \(NeurIPS\),Cited by:[§1](https://arxiv.org/html/2606.06934#S1.p3.1)\.
- D\. Goldberg \(1991\)What every computer scientist should know about floating\-point arithmetic\.ACM Computing Surveys23\(1\),pp\. 5–48\.Cited by:[§1](https://arxiv.org/html/2606.06934#S1.p2.1)\.
- S\. Gupta, A\. Agrawal, K\. Gopalakrishnan, and P\. Narayanan \(2015\)Deep learning with limited numerical precision\.InInternational Conference on Machine Learning \(ICML\),Cited by:[§1](https://arxiv.org/html/2606.06934#S1.p2.1)\.
- M\. Hardt, B\. Recht, and Y\. Singer \(2016\)Train faster, generalize better: stability of stochastic gradient descent\.InInternational Conference on Machine Learning \(ICML\),Cited by:[Lemma 27](https://arxiv.org/html/2606.06934#A2.SS2.2),[Table 1](https://arxiv.org/html/2606.06934#S1.T1.16.16.4.3),[§1](https://arxiv.org/html/2606.06934#S1.p1.1),[§1](https://arxiv.org/html/2606.06934#S1.p4.4),[§3](https://arxiv.org/html/2606.06934#S3.p11.6),[§3](https://arxiv.org/html/2606.06934#S3.p12.14),[§3](https://arxiv.org/html/2606.06934#S3.p15.30),[§4](https://arxiv.org/html/2606.06934#S4.p2.15)\.
- P\. J\. Huber \(1964\)Robust Estimation of a Location Parameter\.The Annals of Mathematical Statistics35\(1\),pp\. 73 – 101\.Cited by:[§B\.1\.1](https://arxiv.org/html/2606.06934#A2.SS1.SSS1.p1.1)\.
- B\. Jacob, S\. Kligys, B\. Chen, M\. Zhu, M\. Tang, A\. G\. Howard, H\. Adam, and D\. Kalenichenko \(2017\)Quantization and training of neural networks for efficient integer\-arithmetic\-only inference\.IEEE Conference on Computer Vision and Pattern Recognition \(CVPR\)\.Cited by:[§5](https://arxiv.org/html/2606.06934#S5.p1.2)\.
- D\. P\. Kingma and J\. Ba \(2015\)Adam: a method for stochastic optimization\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§1](https://arxiv.org/html/2606.06934#S1.p1.1)\.
- J\. Liu, J\. Su, X\. Yao, Z\. Jiang, G\. Lai, Y\. Du, Y\. Qin, W\. Xu, E\. Lu, J\. Yan, Y\. Chen, H\. Zheng, Y\. Liu, S\. Liu, B\. Yin, W\. He, H\. Zhu, Y\. Wang, J\. Wang, M\. Dong, Z\. Zhang, Y\. Kang, H\. Zhang, X\. Xu, Y\. Zhang, Y\. Wu, X\. Zhou, and Z\. Yang \(2025\)Muon is scalable for LLM training\.arXiv preprint arXiv:2502\.16982\.Cited by:[§1](https://arxiv.org/html/2606.06934#S1.p1.1)\.
- I\. Markov, A\. Vladu, Q\. Guo, and D\. Alistarh \(2023\)Quantized distributed training of large models with convergence guarantees\.InInternational Conference on Machine Learning \(ICML\),Cited by:[§1](https://arxiv.org/html/2606.06934#S1.p2.1)\.
- H\. V\. Nguyen, H\. Pham, S\. J\. Reddi, and B\. Póczos \(2022\)On the algorithmic stability and generalization of adaptive optimization methods\.arXiv preprint arXiv:2211\.03970\.Cited by:[§1](https://arxiv.org/html/2606.06934#S1.p1.1)\.
- K\. E\. Nikolakakis, A\. Karbasi, and D\. Kalogerias \(2025\)Select without fear: Almost all minibatch schedules generalize optimally\.SIAM Journal on Mathematics of Data Science7\(3\),pp\. 965–992\.External Links:https://doi\.org/10\.1137/23M1617096Cited by:[Table 1](https://arxiv.org/html/2606.06934#S1.T1.13.13.1.3)\.
- S\. J\. Reddi, S\. Kale, and S\. Kumar \(2018\)On the convergence of adam and beyond\.\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§1](https://arxiv.org/html/2606.06934#S1.p1.1)\.
- H\. Robbins \(1955\)A remark on Stirling’s formula\.The American Mathematical Monthly62\(1\),pp\. 26–29\.Cited by:[§A\.2\.3](https://arxiv.org/html/2606.06934#A1.SS2.SSS3.p2.1)\.
- S\. Shalev\-Shwartz and S\. Ben\-David \(2014\)Understanding machine learning: from theory to algorithms\.Cambridge University Press\.Cited by:[§1](https://arxiv.org/html/2606.06934#S1.p3.1)\.
- S\. Shalev\-Shwartz, O\. Shamir, N\. Srebro, and K\. Sridharan \(2010\)Learnability, stability and uniform convergence\.Journal of Machine Learning Research11,pp\. 2635–2670\.External Links:ISSN 1532\-4435Cited by:[§1](https://arxiv.org/html/2606.06934#S1.p4.4)\.
- A\. B\. Taylor, J\. M\. Hendrickx, and F\. Glineur \(2015\)Smooth strongly convex interpolation and exact worst\-case performance of first\-order methods\.Mathematical Programming161,pp\. 307 – 345\.Cited by:[§5](https://arxiv.org/html/2606.06934#S5.p2.11)\.
- T\. Tieleman and G\. Hinton \(2012\)Lecture 6\.5\-RMSProp: divide the gradient by a running average of its recent magnitude\.Note:COURSERA: Neural networks for machine learning, Lecture notesCited by:[§1](https://arxiv.org/html/2606.06934#S1.p1.1)\.
- L\. Xia, S\. Massei, and M\. E\. Hochstenbach \(2025\)On the convergence of the gradient descent method with stochastic fixed\-point rounding errors under the polyak\-Łojasiewicz inequality\.Computational Optimization and Applications90\(3\),pp\. 753–799\.Cited by:[§1](https://arxiv.org/html/2606.06934#S1.p2.1)\.
- J\. Xin, M\. Canini, P\. Richtárik, and S\. Horváth \(2025\)Global\-qsgd: allreduce\-compatible quantization for distributed learning with theoretical guarantees\.InProceedings of the 5th Workshop on Machine Learning and Systems,Cited by:[§1](https://arxiv.org/html/2606.06934#S1.p2.1)\.
- Y\. Zhang, W\. Zhang, S\. Bald, V\. Pingali, C\. Chen, and M\. Goswami \(2022\)Stability of SGD: tightness analysis and improved bounds\.InConference on Uncertainty in Artificial Intelligence \(UAI\),J\. Cussens and K\. Zhang \(Eds\.\),Cited by:[Table 1](https://arxiv.org/html/2606.06934#S1.T1.17.17.5.2)\.
- D\. Zhou, J\. Chen, Y\. Cao, Z\. Yang, and Q\. Gu \(2024\)On the convergence of adaptive gradient methods for nonconvex optimization\.Transactions on Machine Learning Research\.Cited by:[§1](https://arxiv.org/html/2606.06934#S1.p1.1)\.
- Y\. Zhou, B\. Karimi, J\. Yu, Z\. Xu, and P\. Li \(2020\)Towards better generalization of adaptive gradient methods\.InAnnual Conference on Neural Information Processing Systems \(NeurIPS\),Cited by:[§1](https://arxiv.org/html/2606.06934#S1.p1.1)\.

## Appendix AProofs for𝖣𝖱​\-GD\{\\mathsf\{DR\}\\text\{\\rm\{\-GD\}\}\}and𝖣𝖱​\-SGD\{\\mathsf\{DR\}\\text\{\\rm\{\-SGD\}\}\}

In this section, we prove the generalization and stability bounds for𝖣𝖱​\-GD\{\\mathsf\{DR\}\\text\{\\rm\{\-GD\}\}\}and𝖣𝖱​\-SGD\{\\mathsf\{DR\}\\text\{\\rm\{\-SGD\}\}\}\. We first present the proofs of[Theorems˜2](https://arxiv.org/html/2606.06934#Thmtheorem2)and[3](https://arxiv.org/html/2606.06934#Thmtheorem3), and[Theorem˜4](https://arxiv.org/html/2606.06934#Thmtheorem4)for𝖣𝖱​\-GD\{\\mathsf\{DR\}\\text\{\\rm\{\-GD\}\}\}in[Sections˜A\.1](https://arxiv.org/html/2606.06934#A1.SS1)and[A\.2](https://arxiv.org/html/2606.06934#A1.SS2), and[Section˜A\.3](https://arxiv.org/html/2606.06934#A1.SS3), respectively\. We then present the proofs of[Theorem˜5](https://arxiv.org/html/2606.06934#Thmtheorem5),[Theorem˜6](https://arxiv.org/html/2606.06934#Thmtheorem6),[Theorem˜7](https://arxiv.org/html/2606.06934#Thmtheorem7), and[Theorem˜8](https://arxiv.org/html/2606.06934#Thmtheorem8)for𝖣𝖱​\-SGD\{\\mathsf\{DR\}\\text\{\\rm\{\-SGD\}\}\}in[Section˜A\.4](https://arxiv.org/html/2606.06934#A1.SS4),[Section˜A\.5](https://arxiv.org/html/2606.06934#A1.SS5),[Section˜A\.6](https://arxiv.org/html/2606.06934#A1.SS6),[Section˜A\.7](https://arxiv.org/html/2606.06934#A1.SS7), respectively\.

### A\.1Proof of[Theorem˜2](https://arxiv.org/html/2606.06934#Thmtheorem2)

In this section, we prove[Theorem˜2](https://arxiv.org/html/2606.06934#Thmtheorem2)\. Let𝒟=B​e​r​\(p\)\\mathcal\{D\}=Ber\(p\)be a data distribution,S∼𝒟nS\\sim\\mathcal\{D\}^\{n\}be a sample drawn from𝒟\\mathcal\{D\}, and\(wt\)t=1T\(w\_\{t\}\)\_\{t=1\}^\{T\}be a𝖣𝖱​\-GD\{\\mathsf\{DR\}\\text\{\\rm\{\-GD\}\}\}trajectory fromAf,θ𝖣𝖱​\-GD​\(S\)A\_\{f,\\theta\}^\{\\mathsf\{DR\}\\text\{\\rm\{\-GD\}\}\}\(S\)\. Then, the generalization error ofAf,θ𝖣𝖱​\-GDA\_\{f,\\theta\}^\{\\mathsf\{DR\}\\text\{\\rm\{\-GD\}\}\}is given by

εg​\(Af,θ𝖣𝖱​\-GD;𝒟\)=\|𝔼S∼𝒟n​\[f​\(wT;S\)−f​\(wT;𝒟\)\]\|\.\\displaystyle\\varepsilon\_\{g\}\(A\_\{f,\\theta\}^\{\\mathsf\{DR\}\\text\{\\rm\{\-GD\}\}\};\\mathcal\{D\}\)=\|\\mathbb\{E\}\_\{S\\sim\\mathcal\{D\}^\{n\}\}\[f\(w\_\{T\};S\)\-f\(w\_\{T\};\\mathcal\{D\}\)\]\|\.\(4\)LetkSk\_\{S\}be the number of ones inSS, i\.e\.,kS=‖S‖1k\_\{S\}=\\\|S\\\|\_\{1\}\. Then, we havekS∼B​i​n​\(n,p\)k\_\{S\}\\sim Bin\(n,p\), and hence we can explicitly expand[Eq\.˜4](https://arxiv.org/html/2606.06934#A1.E4):

εg​\(Af,θ𝖣𝖱​\-GD;𝒟\)\\displaystyle\\varepsilon\_\{g\}\(A\_\{f,\\theta\}^\{\\mathsf\{DR\}\\text\{\\rm\{\-GD\}\}\};\\mathcal\{D\}\)=\|𝔼​\[kSn​f​\(wT;1\)\+\(1−kSn\)​f​\(wT;0\)−\(p​f​\(wT;1\)\+\(1−p\)​f​\(wT;0\)\)\]\|\\displaystyle=\\left\|\\mathbb\{E\}\\left\[\\frac\{k\_\{S\}\}\{n\}f\(w\_\{T\};1\)\+\\left\(1\-\\frac\{k\_\{S\}\}\{n\}\\right\)f\(w\_\{T\};0\)\-\\left\(pf\(w\_\{T\};1\)\+\(1\-p\)f\(w\_\{T\};0\)\\right\)\\right\]\\right\|=\|𝔼​\[\(kSn−p\)​\(f​\(wT;1\)−f​\(wT;0\)\)\]\|\.\\displaystyle=\\left\|\\mathbb\{E\}\\left\[\\left\(\\frac\{k\_\{S\}\}\{n\}\-p\\right\)\(f\(w\_\{T\};1\)\-f\(w\_\{T\};0\)\)\\right\]\\right\|\.Without loss of generality, we assume thatf​\(w0;1\)=f​\(w0;0\)=0f\(w\_\{0\};1\)=f\(w\_\{0\};0\)=0\. Otherwise, we replacef​\(wT;1\)f\(w\_\{T\};1\)andf​\(wT;0\)f\(w\_\{T\};0\)byg​\(wT;1\)=f​\(wT;1\)−f​\(w0;1\)g\(w\_\{T\};1\)=f\(w\_\{T\};1\)\-f\(w\_\{0\};1\)andg​\(wT;0\)=f​\(wT;0\)−f​\(w0;0\)g\(w\_\{T\};0\)=f\(w\_\{T\};0\)\-f\(w\_\{0\};0\), respectively; note that

𝔼​\[\(kSn−p\)​\(g​\(wT;1\)−g​\(wT;0\)\)\]=\\displaystyle\\mathbb\{E\}\\left\[\\left\(\\frac\{k\_\{S\}\}\{n\}\-p\\right\)\(g\(w\_\{T\};1\)\-g\(w\_\{T\};0\)\)\\right\]=𝔼​\[\(kSn−p\)​\(f​\(wT;1\)−f​\(wT;0\)\)\]\\displaystyle\\mathbb\{E\}\\left\[\\left\(\\frac\{k\_\{S\}\}\{n\}\-p\\right\)\(f\(w\_\{T\};1\)\-f\(w\_\{T\};0\)\)\\right\]\+𝔼​\[kSn−p\]​\(f​\(w0;1\)−f​\(w0;0\)\)\\displaystyle\+\\mathbb\{E\}\\left\[\\frac\{k\_\{S\}\}\{n\}\-p\\right\]\(f\(w\_\{0\};1\)\-f\(w\_\{0\};0\)\)=\\displaystyle=𝔼​\[\(kSn−p\)​\(f​\(wT;1\)−f​\(wT;0\)\)\]\\displaystyle\\mathbb\{E\}\\left\[\\left\(\\frac\{k\_\{S\}\}\{n\}\-p\\right\)\(f\(w\_\{T\};1\)\-f\(w\_\{T\};0\)\)\\right\]where the last equality follows since𝔼​\[kS\]=n​p\\mathbb\{E\}\[k\_\{S\}\]=np\. Then, by the triangle inequality, we have

εg​\(Af,θ𝖣𝖱​\-GD;𝒟\)\\displaystyle\\varepsilon\_\{g\}\(A\_\{f,\\theta\}^\{\\mathsf\{DR\}\\text\{\\rm\{\-GD\}\}\};\\mathcal\{D\}\)≤\|𝔼​\[\(kSn−p\)​\(f​\(wT;1\)−f​\(wT;0\)\)\]\|\\displaystyle\\leq\\left\|\\mathbb\{E\}\\left\[\\left\(\\frac\{k\_\{S\}\}\{n\}\-p\\right\)\(f\(w\_\{T\};1\)\-f\(w\_\{T\};0\)\)\\right\]\\right\|≤𝔼​\[\|kSn−p\|⋅\(\|f​\(wT;1\)\|\+\|f​\(wT;0\)\|\)\]\.\\displaystyle\\leq\\mathbb\{E\}\\left\[\\left\|\\frac\{k\_\{S\}\}\{n\}\-p\\right\|\\cdot\(\|f\(w\_\{T\};1\)\|\+\|f\(w\_\{T\};0\)\|\)\\right\]\.
In this proof, we first prove the upper bound of\|f​\(wT;1\)\|\+\|f​\(wT;0\)\|\|f\(w\_\{T\};1\)\|\+\|f\(w\_\{T\};0\)\|, and then show the upper bound of𝔼​\[\|kS/n−p\|\]\\mathbb\{E\}\[\|k\_\{S\}/n\-p\|\]\. For the first part, we consider the upper bound of‖wT−w0‖2\\\|w\_\{T\}\-w\_\{0\}\\\|\_\{2\}\. Then, sinceffisLL\-Lipschitz andf​\(w0;0\)=f​\(w0;1\)=0f\(w\_\{0\};0\)=f\(w\_\{0\};1\)=0, we have\|f​\(wT;0\)\|\+\|f​\(wT;1\)\|≤2​L​‖wT−w0‖2\|f\(w\_\{T\};0\)\|\+\|f\(w\_\{T\};1\)\|\\leq 2L\\\|w\_\{T\}\-w\_\{0\}\\\|\_\{2\}\.

To this end, we introduce the following lemma\. The proof of[Section˜A\.1](https://arxiv.org/html/2606.06934#A1.SS1)is in[Section˜A\.1\.1](https://arxiv.org/html/2606.06934#A1.SS1.SSS1)

###### Lemma 14\.

Letd∈ℕd\\in\\mathbb\{N\}andΔ,R\>0\\Delta,R\>0anda∈ℝda\\in\\mathbb\{R\}^\{d\}such that‖a‖2≤R\\\|a\\\|\_\{2\}\\leq R\. Then, it holds that

‖𝖣𝖱​\(a\)‖2≤2​R\.\\displaystyle\\\|\\mathsf\{DR\}\(a\)\\\|\_\{2\}\\leq 2R\.

SinceffisLL\-Lipschitz,‖η​∇f​\(w;S\)‖2≤η​L\\\|\\eta\\nabla f\(w;S\)\\\|\_\{2\}\\leq\\eta Lfor allw∈ℤΔdw\\in\\mathbb\{Z\}\_\{\\Delta\}^\{d\}\. Then, by[Section˜A\.1](https://arxiv.org/html/2606.06934#A1.SS1), we have‖wt−wt−1‖2=‖𝖣𝖱​\(η​∇f​\(wt−1;S\)\)‖2≤2​η​L\\\|w\_\{t\}\-w\_\{t\-1\}\\\|\_\{2\}=\\\|\\mathsf\{DR\}\(\\eta\\nabla f\(w\_\{t\-1\};S\)\)\\\|\_\{2\}\\leq 2\\eta Lfor allt∈\[T\]t\\in\[T\]\. Then, we have

‖wT−w0‖2≤‖wT−wT−1‖2\+⋯\+‖w1−w0‖2≤2​η​L​T\.\\displaystyle\\\|w\_\{T\}\-w\_\{0\}\\\|\_\{2\}\\leq\\\|w\_\{T\}\-w\_\{T\-1\}\\\|\_\{2\}\+\\cdots\+\\\|w\_\{1\}\-w\_\{0\}\\\|\_\{2\}\\leq 2\\eta LT\.Then, we have\|f​\(wT;0\)\|\+\|f​\(wT;1\)\|≤2​L​‖wT−w0‖2≤4​η​L2​T\|f\(w\_\{T\};0\)\|\+\|f\(w\_\{T\};1\)\|\\leq 2L\\\|w\_\{T\}\-w\_\{0\}\\\|\_\{2\}\\leq 4\\eta L^\{2\}T\.

We now consider𝔼​\[\|kS/n−p\|\]\\mathbb\{E\}\[\|k\_\{S\}/n\-p\|\]\. By Jensen’s inequality, we have

𝔼​\[\|kSn−p\|\]≤𝔼​\[\(kSn−p\)2\]=V​a​r​\(kSn\)=p​\(1−p\)n≤12​n\\displaystyle\\mathbb\{E\}\\left\[\\left\|\\frac\{k\_\{S\}\}\{n\}\-p\\right\|\\right\]\\leq\\sqrt\{\\mathbb\{E\}\\left\[\\left\(\\frac\{k\_\{S\}\}\{n\}\-p\\right\)^\{2\}\\right\]\}=\\sqrt\{Var\\left\(\\frac\{k\_\{S\}\}\{n\}\\right\)\}=\\sqrt\{\\frac\{p\(1\-p\)\}\{n\}\}\\leq\\frac\{1\}\{2\\sqrt\{n\}\}where the last inequality follows from the fact thatp​\(1−p\)p\(1\-p\)is maximized whenp=1/2p=1/2\.

Conclusively, we have

εg​\(Af,θ𝖣𝖱​\-GD,𝒟\)\\displaystyle\\varepsilon\_\{g\}\(A\_\{f,\\theta\}^\{\\mathsf\{DR\}\\text\{\\rm\{\-GD\}\}\},\\mathcal\{D\}\)≤𝔼​\[\|kSn−p\|⋅\(\|f​\(wT;1\)\|\+\|f​\(wT;0\)\|\)\]\\displaystyle\\leq\\mathbb\{E\}\\left\[\\left\|\\frac\{k\_\{S\}\}\{n\}\-p\\right\|\\cdot\(\|f\(w\_\{T\};1\)\|\+\|f\(w\_\{T\};0\)\|\)\\right\]≤4​η​L2​T⋅𝔼​\[\|kSn−p\|\]≤2​η​L2​Tn\\displaystyle\\leq 4\\eta L^\{2\}T\\cdot\\mathbb\{E\}\\left\[\\left\|\\frac\{k\_\{S\}\}\{n\}\-p\\right\|\\right\]\\leq\\frac\{2\\eta L^\{2\}T\}\{\\sqrt\{n\}\}which completes the proof\.

#### A\.1\.1Proof of[Section˜A\.1](https://arxiv.org/html/2606.06934#A1.SS1)

In this section, we prove[Section˜A\.1](https://arxiv.org/html/2606.06934#A1.SS1)\. Leta=\(a1,⋯,ad\)a=\(a\_\{1\},\\cdots,a\_\{d\}\)\. It suffices to show that anyx∈ℝx\\in\\mathbb\{R\},\|𝖣𝖱​\(x\)\|≤2​\|x\|\|\\mathsf\{DR\}\(x\)\|\\leq 2\|x\|\. Then, we have

‖𝖣𝖱​\(a\)‖22=∑i=1d\|𝖣𝖱​\(ai\)\|2≤∑i=1d4​\|ai\|2=\(2​‖a‖2\)2≤\(2​R\)2\\displaystyle\\\|\\mathsf\{DR\}\(a\)\\\|\_\{2\}^\{2\}=\\sum\_\{i=1\}^\{d\}\|\\mathsf\{DR\}\(a\_\{i\}\)\|^\{2\}\\leq\\sum\_\{i=1\}^\{d\}4\|a\_\{i\}\|^\{2\}=\(2\\\|a\\\|\_\{2\}\)^\{2\}\\leq\(2R\)^\{2\}which implies‖𝖣𝖱​\(a\)‖2≤2​R\\\|\\mathsf\{DR\}\(a\)\\\|\_\{2\}\\leq 2Rand this completes the proof\.

Since\|x\|<Δ/2\|x\|<\\Delta/2is trivial \(𝖣𝖱​\(x\)=0\\mathsf\{DR\}\(x\)=0\), we assume that\|x\|≥Δ/2\|x\|\\geq\\Delta/2\. Since\|x−𝖣𝖱​\(x\)\|≤Δ/2\|x\-\\mathsf\{DR\}\(x\)\|\\leq\\Delta/2, by the triangle inequality, we have

\|𝖣𝖱​\(x\)\|≤\|x\|\+Δ2≤\|x\|\+\|x\|=2​\|x\|\\displaystyle\|\\mathsf\{DR\}\(x\)\|\\leq\|x\|\+\\frac\{\\Delta\}\{2\}\\leq\|x\|\+\|x\|=2\|x\|which completes the proof\.

### A\.2Proof of[Theorem˜3](https://arxiv.org/html/2606.06934#Thmtheorem3)

In this section, we prove[Theorem˜3](https://arxiv.org/html/2606.06934#Thmtheorem3)\. We define a loss functionf:ℝ×\{0,1\}→ℝf:\\mathbb\{R\}\\times\\\{0,1\\\}\\to\\mathbb\{R\}and an underlying data distribution𝒟\\mathcal\{D\}as follows:

𝒟=B​e​r​\(p\),f​\(w;z\)=−z​L​w\\displaystyle\\mathcal\{D\}=Ber\(p\),\\quad f\(w;z\)=\-zLwfor somep∈\(0,1\)p\\in\(0,1\); we will assign an explicit value topplater\. We note that∇f​\(w;z\)=−z​L∈\{0,−L\}\\nabla f\(w;z\)=\-zL\\in\\\{0,\-L\\\}\. LetS=\(z1,⋯,zn\)S=\(z\_\{1\},\\cdots,z\_\{n\}\)andS′=\(z1′,z2,⋯,zn\)S^\{\\prime\}=\(z\_\{1\}^\{\\prime\},z\_\{2\},\\cdots,z\_\{n\}\)wherez1′,z1,⋯,znz\_\{1\}^\{\\prime\},z\_\{1\},\\cdots,z\_\{n\}are independently sampled from𝒟\\mathcal\{D\},z¯S=‖S‖1/n\\bar\{z\}\_\{S\}=\\\|S\\\|\_\{1\}/n,z¯S′=‖S′‖1/n\\bar\{z\}\_\{S^\{\\prime\}\}=\\\|S^\{\\prime\}\\\|\_\{1\}/n,\(wt\)t=1T\(w\_\{t\}\)\_\{t=1\}^\{T\}and\(wt′\)t=1T\(w\_\{t\}^\{\\prime\}\)\_\{t=1\}^\{T\}be the𝖣𝖱​\-GD\{\\mathsf\{DR\}\\text\{\\rm\{\-GD\}\}\}trajectories fromAf,θ𝖣𝖱​\-GD​\(S\)A\_\{f,\\theta\}^\{\{\\mathsf\{DR\}\\text\{\\rm\{\-GD\}\}\}\}\(S\)andAf,θ𝖣𝖱​\-GD​\(S′\)A\_\{f,\\theta\}^\{\{\\mathsf\{DR\}\\text\{\\rm\{\-GD\}\}\}\}\(S^\{\\prime\}\), respectively\. Since the gradients of the empirical loss functions onSSandS′S^\{\\prime\}are∇f​\(w;S\)=−L​z¯S\\nabla f\(w;S\)=\-L\\bar\{z\}\_\{S\}and∇f​\(w;S′\)=−L​z¯S′\\nabla f\(w;S^\{\\prime\}\)=\-L\\bar\{z\}\_\{S^\{\\prime\}\}for allww, we have

wT=−T​𝖣𝖱​\(−η​L​z¯S\),wT′=−T​𝖣𝖱​\(−η​L​z¯S′\)\.\\displaystyle w\_\{T\}=\-T\\mathsf\{DR\}\(\-\\eta L\\bar\{z\}\_\{S\}\),\\quad w\_\{T\}^\{\\prime\}=\-T\\mathsf\{DR\}\(\-\\eta L\\bar\{z\}\_\{S^\{\\prime\}\}\)\.
Here, we do not use𝖣𝖱​\(−x\)=−𝖣𝖱​\(x\)\\mathsf\{DR\}\(\-x\)=\-\\mathsf\{DR\}\(x\)since a general tie\-breaking rule need not make deterministic rounding an odd function\.

Then, we havewT′−wT=T​\(𝖣𝖱​\(−η​L​z¯S\)−𝖣𝖱​\(−η​L​z¯S′\)\)w\_\{T\}^\{\\prime\}\-w\_\{T\}=T\(\\mathsf\{DR\}\(\-\\eta L\\bar\{z\}\_\{S\}\)\-\\mathsf\{DR\}\(\-\\eta L\\bar\{z\}\_\{S^\{\\prime\}\}\)\)\. We now consider the generalization error ofAf,θ𝖣𝖱​\-GDA\_\{f,\\theta\}^\{\\mathsf\{DR\}\\text\{\\rm\{\-GD\}\}\}\. By[Section˜2\.4](https://arxiv.org/html/2606.06934#S2.SS4),εg​\(Af,θ𝖣𝖱​\-GD;𝒟\)\\varepsilon\_\{g\}\(A\_\{f,\\theta\}^\{\\mathsf\{DR\}\\text\{\\rm\{\-GD\}\}\};\\mathcal\{D\}\)can be expressed by

εg​\(Af,θ𝖣𝖱​\-GD;𝒟\)\\displaystyle\\varepsilon\_\{g\}\(A\_\{f,\\theta\}^\{\\mathsf\{DR\}\\text\{\\rm\{\-GD\}\}\};\\mathcal\{D\}\)=\|𝔼​\[f​\(wT′;z1′\)−f​\(wT;z1′\)\]\|\\displaystyle=\\left\\rvert\\mathbb\{E\}\[f\(w\_\{T\}^\{\\prime\};z\_\{1\}^\{\\prime\}\)\-f\(w\_\{T\};z\_\{1\}^\{\\prime\}\)\]\\right\\rvert=L​\|𝔼​\[z1′​\(wT′−wT\)\]\|\.\\displaystyle=L\\left\\rvert\\mathbb\{E\}\\left\[z\_\{1\}^\{\\prime\}\\left\(w\_\{T\}^\{\\prime\}\-w\_\{T\}\\right\)\\right\]\\right\\rvert\.Ifz1=z1′z\_\{1\}=z\_\{1\}^\{\\prime\}, thenS=S′S=S^\{\\prime\}and hencewT=wT′w\_\{T\}=w\_\{T\}^\{\\prime\}\. Furthermore, ifz1′=0z\_\{1\}^\{\\prime\}=0, thenz1′​\(wT′−wT\)=0z\_\{1\}^\{\\prime\}\(w\_\{T\}^\{\\prime\}\-w\_\{T\}\)=0\. Thus, only the event\(z1′,z1\)=\(1,0\)\(z\_\{1\}^\{\\prime\},z\_\{1\}\)=\(1,0\)contributes to this expectation, hence we obtain

εg​\(Af,θ𝖣𝖱​\-GD;𝒟\)\\displaystyle\\varepsilon\_\{g\}\(A\_\{f,\\theta\}^\{\\mathsf\{DR\}\\text\{\\rm\{\-GD\}\}\};\\mathcal\{D\}\)=p\(1−p\)L\|𝔼\[wT′−wT\|\(z1′,z1\)=\(1,0\)\]\|\\displaystyle=p\(1\-p\)L\\left\\rvert\\mathbb\{E\}\[w\_\{T\}^\{\\prime\}\-w\_\{T\}\|\(z\_\{1\}^\{\\prime\},z\_\{1\}\)=\(1,0\)\]\\right\\rvert=p\(1−p\)LT\|𝔼\[𝖣𝖱\(−ηLz¯S\)−𝖣𝖱\(−ηLz¯S′\)\|\(z1′,z1\)=\(1,0\)\]\|\\displaystyle=p\(1\-p\)LT\|\\mathbb\{E\}\[\\mathsf\{DR\}\(\-\\eta L\\bar\{z\}\_\{S\}\)\-\\mathsf\{DR\}\(\-\\eta L\\bar\{z\}\_\{S^\{\\prime\}\}\)\|\(z\_\{1\}^\{\\prime\},z\_\{1\}\)=\(1,0\)\]\|=p​\(1−p\)​L​T​\|𝔼​\[𝖣𝖱​\(−η​L​Kn\)−𝖣𝖱​\(−η​L​\(K\+1\)n\)\]\|\.\\displaystyle=p\(1\-p\)LT\\left\|\\mathbb\{E\}\\left\[\\mathsf\{DR\}\\left\(\-\\frac\{\\eta LK\}\{n\}\\right\)\-\\mathsf\{DR\}\\left\(\-\\frac\{\\eta L\(K\+1\)\}\{n\}\\right\)\\right\]\\right\|\.whereK=∑i=2nziK=\\sum\_\{i=2\}^\{n\}z\_\{i\}\. Here, we note thatK∼B​i​n​\(n−1,p\)K\\sim Bin\(n\-1,p\)and

z¯S=Kn,z¯S′=K\+1n\.\\displaystyle\\bar\{z\}\_\{S\}=\\frac\{K\}\{n\},\\quad\\bar\{z\}\_\{S^\{\\prime\}\}=\\frac\{K\+1\}\{n\}\.
To show the lower bound, we use the following two lemmas\. See[Sections˜A\.2\.1](https://arxiv.org/html/2606.06934#A1.SS2.SSS1)and[A\.2\.2](https://arxiv.org/html/2606.06934#A1.SS2.SSS2)for the proofs of[Sections˜A\.2](https://arxiv.org/html/2606.06934#A1.SS2)and[A\.2](https://arxiv.org/html/2606.06934#A1.SS2)\.

###### Lemma 15\.

For anya,b∈ℝa,b\\in\\mathbb\{R\}, witha\>ba\>b, we have

𝖣𝖱​\(a\)−𝖣𝖱​\(b\)≥a−b−Δ\.\\displaystyle\\mathsf\{DR\}\(a\)\-\\mathsf\{DR\}\(b\)\\geq a\-b\-\\Delta\.

###### Lemma 16\.

Letn∈ℕn\\in\\mathbb\{N\},Δ,R\>0\\Delta,R\>0such thatR\>c​ΔR\>c\\Deltafor some fixed constantc\>0\.5c\>0\.5, andn≥max⁡\{8​c/\(2​c−1\),6\}n\\geq\\max\\\{8c/\(2c\-1\),6\\\}\. Then, there existsp∈\(0,1\)p\\in\(0,1\)such thatp​\(1−p\)=Ω​\(1\)p\(1\-p\)=\\Omega\(1\)and forK∼B​i​n​\(n−1,p\)K\\sim Bin\(n\-1,p\),

𝔼​\[𝖣𝖱​\(−R​Kn\)−𝖣𝖱​\(−R​\(K\+1\)n\)\]=Ω​\(Δn\)\.\\displaystyle\\mathbb\{E\}\\left\[\\mathsf\{DR\}\\left\(\-\\frac\{RK\}\{n\}\\right\)\-\\mathsf\{DR\}\\left\(\-\\frac\{R\(K\+1\)\}\{n\}\\right\)\\right\]=\\Omega\\left\(\\frac\{\\Delta\}\{\\sqrt\{n\}\}\\right\)\.

Under the event\(z1′,z1\)=\(1,0\)\(z\_\{1\}^\{\\prime\},z\_\{1\}\)=\(1,0\),−η​L​z¯S−\(−η​L​z¯S′\)=η​L/n\-\\eta L\\bar\{z\}\_\{S\}\-\(\-\\eta L\\bar\{z\}\_\{S^\{\\prime\}\}\)=\\eta L/n\. Therefore, by[Section˜A\.2](https://arxiv.org/html/2606.06934#A1.SS2), we have

𝖣𝖱​\(−η​L​z¯S\)−𝖣𝖱​\(−η​L​z¯S′\)≥η​Ln−Δ\.\\displaystyle\\mathsf\{DR\}\(\-\\eta L\\bar\{z\}\_\{S\}\)\-\\mathsf\{DR\}\(\-\\eta L\\bar\{z\}\_\{S^\{\\prime\}\}\)\\geq\\frac\{\\eta L\}\{n\}\-\\Delta\.Taking expectations gives

𝔼​\[𝖣𝖱​\(−η​L​z¯S\)−𝖣𝖱​\(−η​L​z¯S′\)\|\(z1′,z1\)=\(1,0\)\]≥η​Ln−Δ\.\\displaystyle\\mathbb\{E\}\[\\mathsf\{DR\}\(\-\\eta L\\bar\{z\}\_\{S\}\)\-\\mathsf\{DR\}\(\-\\eta L\\bar\{z\}\_\{S^\{\\prime\}\}\)\|\(z\_\{1\}^\{\\prime\},z\_\{1\}\)=\(1,0\)\]\\geq\\frac\{\\eta L\}\{n\}\-\\Delta\.Furthermore, by applying[Section˜A\.2](https://arxiv.org/html/2606.06934#A1.SS2)withR=η​LR=\\eta L, we obtainp∈\(0,1\)p\\in\(0,1\)such thatp​\(1−p\)=Ω​\(1\)p\(1\-p\)=\\Omega\(1\)andK∼B​i​n​\(n−1,p\)K\\sim Bin\(n\-1,p\)and

\|𝔼\[𝖣𝖱\(−ηLz¯S\)−𝖣𝖱\(−ηLz¯S′\)\|\(z1′,z1\)=\(1,0\)\]\|≳Δn\.\\displaystyle\|\\mathbb\{E\}\[\\mathsf\{DR\}\(\-\\eta L\\bar\{z\}\_\{S\}\)\-\\mathsf\{DR\}\(\-\\eta L\\bar\{z\}\_\{S^\{\\prime\}\}\)\|\(z\_\{1\}^\{\\prime\},z\_\{1\}\)=\(1,0\)\]\|\\gtrsim\\frac\{\\Delta\}\{\\sqrt\{n\}\}\.
Conclusively, we have

εg​\(Af,θ𝖣𝖱​\-GD;𝒟\)≳L​T​max⁡\{Δn,\(η​Ln−Δ\)\}\\displaystyle\\varepsilon\_\{g\}\(A^\{\{\\mathsf\{DR\}\\text\{\\rm\{\-GD\}\}\}\}\_\{f,\\theta\};\\mathcal\{D\}\)\\gtrsim LT\\max\\left\\\{\\frac\{\\Delta\}\{\\sqrt\{n\}\},\\left\(\\frac\{\\eta L\}\{n\}\-\\Delta\\right\)\\right\\\}which completes the proof\.

#### A\.2\.1Proof of[Section˜A\.2](https://arxiv.org/html/2606.06934#A1.SS2)

In this section, we prove[Section˜A\.2](https://arxiv.org/html/2606.06934#A1.SS2)by considering the lower bound of𝖣𝖱​\(a\)\\mathsf\{DR\}\(a\)and the upper bound of𝖣𝖱​\(b\)\\mathsf\{DR\}\(b\)\. Since𝖣𝖱​\(x\)∈\[x−Δ/2,x\+Δ/2\]\\mathsf\{DR\}\(x\)\\in\[x\-\\Delta/2,x\+\\Delta/2\]for anyx∈ℝx\\in\\mathbb\{R\}, we have

𝖣𝖱​\(a\)−𝖣𝖱​\(b\)≥\(a−Δ2\)−\(b\+Δ2\)=a−b−Δ\\displaystyle\\mathsf\{DR\}\(a\)\-\\mathsf\{DR\}\(b\)\\geq\\left\(a\-\\frac\{\\Delta\}\{2\}\\right\)\-\\left\(b\+\\frac\{\\Delta\}\{2\}\\right\)=a\-b\-\\Deltawhich completes the proof\.

#### A\.2\.2Proof of[Section˜A\.2](https://arxiv.org/html/2606.06934#A1.SS2)

In this section, we prove[Section˜A\.2](https://arxiv.org/html/2606.06934#A1.SS2)\. To this end, we findk∗∈ℕk^\{\\ast\}\\in\\mathbb\{N\}such that there exist constantsc1,c2∈\(0,1\)c\_\{1\},c\_\{2\}\\in\(0,1\)such thatc1​n≤k∗≤c2​nc\_\{1\}n\\leq k^\{\\ast\}\\leq c\_\{2\}nand

𝖣𝖱​\(−R​k∗n\)−𝖣𝖱​\(−R​\(k∗\+1\)n\)≥Δ\.\\displaystyle\\mathsf\{DR\}\\left\(\-\\frac\{Rk^\{\\ast\}\}\{n\}\\right\)\-\\mathsf\{DR\}\\left\(\-\\frac\{R\(k^\{\\ast\}\+1\)\}\{n\}\\right\)\\geq\\Delta\.\(5\)Once suchk∗k^\{\\ast\}is found, we choosep=k∗/\(n−1\)p=k^\{\\ast\}/\(n\-1\)\. Then,0<p<10<p<1and𝔼​\[K\]=k∗\\mathbb\{E\}\[K\]=k^\{\\ast\}\. Here, the following lemma givesℙ​\(K=k∗\)=Ω​\(1/n\)\\mathbb\{P\}\(K=k^\{\\ast\}\)=\\Omega\(1/\\sqrt\{n\}\)\. See[Section˜A\.2\.3](https://arxiv.org/html/2606.06934#A1.SS2.SSS3)for the proof of[Section˜A\.2\.2](https://arxiv.org/html/2606.06934#A1.SS2.SSS2)\.

###### Lemma 17\.

Letn∈ℕn\\in\\mathbb\{N\}andp∈\(0,1\)p\\in\(0,1\)such thatn≥2n\\geq 2andn​p∈\{1,⋯,n−1\}np\\in\\\{1,\\cdots,n\-1\\\}\. ForX∼B​i​n​\(n,p\)X\\sim Bin\(n,p\), it holds that

ℙ​\(X=n​p\)=Ω​\(1n\)\.\\displaystyle\\mathbb\{P\}\(X=np\)=\\Omega\\left\(\\frac\{1\}\{\\sqrt\{n\}\}\\right\)\.

Moreover, since the deterministic rounding operation is monotone,𝖣𝖱​\(−R​k/n\)−𝖣𝖱​\(−R​\(k\+1\)/n\)\\mathsf\{DR\}\(\-Rk/n\)\-\\mathsf\{DR\}\(\-R\(k\+1\)/n\)is nonnegative for allk∈\{0,⋯,n−1\}k\\in\\\{0,\\cdots,n\-1\\\}\. Hence, it holds that

𝔼​\[𝖣𝖱​\(−R​Kn\)−𝖣𝖱​\(−R​\(K\+1\)n\)\]\\displaystyle\\mathbb\{E\}\\left\[\\mathsf\{DR\}\\left\(\-\\frac\{RK\}\{n\}\\right\)\-\\mathsf\{DR\}\\left\(\-\\frac\{R\(K\+1\)\}\{n\}\\right\)\\right\]≥\(𝖣𝖱​\(−R​k∗n\)−𝖣𝖱​\(−R​\(k∗\+1\)n\)\)​ℙ​\(K=k∗\)\\displaystyle\\geq\\left\(\\mathsf\{DR\}\\left\(\-\\frac\{Rk^\{\\ast\}\}\{n\}\\right\)\-\\mathsf\{DR\}\\left\(\-\\frac\{R\(k^\{\\ast\}\+1\)\}\{n\}\\right\)\\right\)\\mathbb\{P\}\(K=k^\{\\ast\}\)=Ω​\(Δn\)\.\\displaystyle=\\Omega\\left\(\\frac\{\\Delta\}\{\\sqrt\{n\}\}\\right\)\.
In the rest of the proof, we findk∗∈\[n−1\],c1k^\{\\ast\}\\in\[n\-1\],c\_\{1\}, andc2c\_\{2\}such thatk∗∈\[c1​n,c2​n\]k^\{\\ast\}\\in\[c\_\{1\}n,c\_\{2\}n\]andk∗k^\{\\ast\}satisfies[Eq\.˜5](https://arxiv.org/html/2606.06934#A1.E5)\.

Letm=⌊R\+0\.5​Δ⌋Δ/Δm=\\lfloor R\+0\.5\\Delta\\rfloor\_\{\\Delta\}/\\Delta\. Then,mmis the natural number satisfyingR∈\[\(m−0\.5\)​Δ,\(m\+0\.5\)​Δ\)R\\in\[\(m\-0\.5\)\\Delta,\(m\+0\.5\)\\Delta\)\. We split the construction into two cases:m=1m=1andm\>1m\>1\.

Suppose thatm=1m=1, which implies0\.5​Δ≤R<1\.5​Δ0\.5\\Delta\\leq R<1\.5\\Delta\. Then, we haveR∈\(c​Δ,1\.5​Δ\)R\\in\(c\\Delta,1\.5\\Delta\)\. Since the sequence\{−R​k/n\}k=0n\\\{\-Rk/n\\\}\_\{k=0\}^\{n\}crosses−0\.5​Δ\-0\.5\\Delta, there existsk∗k^\{\\ast\}such that

𝖣𝖱​\(−R​k∗n\)−𝖣𝖱​\(−R​\(k∗\+1\)n\)≥Δ\.\\displaystyle\\mathsf\{DR\}\\left\(\-\\frac\{Rk^\{\\ast\}\}\{n\}\\right\)\-\\mathsf\{DR\}\\left\(\-\\frac\{R\(k^\{\\ast\}\+1\)\}\{n\}\\right\)\\geq\\Delta\.Here, suchk∗k^\{\\ast\}satisfies

\|−R​k∗n−\(−Δ2\)\|≤Rn\\displaystyle\\left\|\-\\frac\{Rk^\{\\ast\}\}\{n\}\-\\left\(\-\\frac\{\\Delta\}\{2\}\\right\)\\right\|\\leq\\frac\{R\}\{n\}which is equivalent to\|k∗−Δ​n/\(2​R\)\|≤1\\left\|k^\{\\ast\}\-\\Delta n/\(2R\)\\right\|\\leq 1\.

SinceΔ/2​R∈\(1/3,1/2​c\)\\Delta/2R\\in\(1/3,1/2c\), it holds that

k∗∈\[Δ​n2​R−1,Δ​n2​R\+1\]⊂\(n3−1,n​\(12​c\+1n\)\)\\displaystyle k^\{\\ast\}\\in\\left\[\\frac\{\\Delta n\}\{2R\}\-1,\\frac\{\\Delta n\}\{2R\}\+1\\right\]\\subset\\left\(\\frac\{n\}\{3\}\-1,n\\left\(\\frac\{1\}\{2c\}\+\\frac\{1\}\{n\}\\right\)\\right\)Then, ifn≥max⁡\{8​c/\(2​c−1\),4\}n\\geq\\max\\\{8c/\(2c\-1\),4\\\}, we have

n12<k∗<n​\(14\+38​c\)\\displaystyle\\frac\{n\}\{12\}<k^\{\\ast\}<n\\left\(\\frac\{1\}\{4\}\+\\frac\{3\}\{8c\}\\right\)where the first inequality follows fromn≥4n\\geq 4, and the second inequality follows fromn≥8​c/\(2​c−1\)n\\geq 8c/\(2c\-1\), which is equivalent to1/n≤1/4−1/\(8​c\)1/n\\leq 1/4\-1/\(8c\)\.

We now consider the case thatm\>1m\>1\. As in the casem=1m=1, we choose a rounding threshold nearR/2R/2and findk∗k^\{\\ast\}that induces separation between𝖣𝖱​\(−R​\(k∗\+1\)/n\)\\mathsf\{DR\}\(\-R\(k^\{\\ast\}\+1\)/n\)and𝖣𝖱​\(−R​k∗/n\)\\mathsf\{DR\}\(\-Rk^\{\\ast\}/n\)\.

Ifm=2​jm=2j, then we setq=\(j−0\.5\)​Δq=\(j\-0\.5\)\\Delta, and ifm=2​j\+1m=2j\+1, then we setq=\(j\+0\.5\)​Δq=\(j\+0\.5\)\\Delta\. Since the sequence\{R​k/n\}k=0n\\\{Rk/n\\\}\_\{k=0\}^\{n\}crossesqq, there existsk∗k^\{\\ast\}such that

𝖣𝖱​\(−R​k∗n\)−𝖣𝖱​\(−R​\(k∗\+1\)n\)≥Δ\.\\displaystyle\\mathsf\{DR\}\\left\(\-\\frac\{Rk^\{\\ast\}\}\{n\}\\right\)\-\\mathsf\{DR\}\\left\(\-\\frac\{R\(k^\{\\ast\}\+1\)\}\{n\}\\right\)\\geq\\Delta\.Here, suchk∗k^\{\\ast\}satisfies

\|R​k∗n−q\|≤Rn\\displaystyle\\left\|\\frac\{Rk^\{\\ast\}\}\{n\}\-q\\right\|\\leq\\frac\{R\}\{n\}which is equivalent to\|k∗−q​n/R\|≤1\|k^\{\\ast\}\-qn/R\|\\leq 1\.

Here, sinceR∈\[\(m−0\.5\)​Δ,\(m\+0\.5\)​Δ\)R\\in\[\(m\-0\.5\)\\Delta,\(m\+0\.5\)\\Delta\), ifm=2​jm=2j, we have

qR=\(m−1\)​Δ2​R∈\(m−12​m\+1,m−12​m−1\]⊂\(15,12\]\\displaystyle\\frac\{q\}\{R\}=\\frac\{\(m\-1\)\\Delta\}\{2R\}\\in\\left\(\\frac\{m\-1\}\{2m\+1\},\\frac\{m\-1\}\{2m\-1\}\\right\]\\subset\\left\(\\frac\{1\}\{5\},\\frac\{1\}\{2\}\\right\]form≥2m\\geq 2\. Ifm=2​j\+1m=2j\+1, then we have

qR=m​Δ2​R∈\(m2​m\+1,m2​m−1\]⊂\(37,35\]\\displaystyle\\frac\{q\}\{R\}=\\frac\{m\\Delta\}\{2R\}\\in\\left\(\\frac\{m\}\{2m\+1\},\\frac\{m\}\{2m\-1\}\\right\]\\subset\\left\(\\frac\{3\}\{7\},\\frac\{3\}\{5\}\\right\]form≥3m\\geq 3\. Therefore, in all cases withm\>1m\>1, we have

15<qR≤35\.\\displaystyle\\frac\{1\}\{5\}<\\frac\{q\}\{R\}\\leq\\frac\{3\}\{5\}\.Here, since\|k∗−q​n/R\|≤1\|k^\{\\ast\}\-qn/R\|\\leq 1, we have

k∗∈\[q​nR−1,q​nR\+1\]⊂\(n5−1,3​n5\+1\]\.\\displaystyle k^\{\\ast\}\\in\\left\[\\frac\{qn\}\{R\}\-1,\\frac\{qn\}\{R\}\+1\\right\]\\subset\\left\(\\frac\{n\}\{5\}\-1,\\frac\{3n\}\{5\}\+1\\right\]\.In particular, forn≥6n\\geq 6, it holds thatn/30<k∗≤23​n/30n/30<k^\{\\ast\}\\leq 23n/30\.

Hence, if we choosec1=1/30c\_\{1\}=1/30,c2=max⁡\{1/4\+3/\(8​c\),23/30\}c\_\{2\}=\\max\\\{1/4\+3/\(8c\),23/30\\\}, then there existsk∗k^\{\\ast\}such thatc1​n<k∗≤c2​nc\_\{1\}n<k^\{\\ast\}\\leq c\_\{2\}nand[Eq\.˜5](https://arxiv.org/html/2606.06934#A1.E5)holds\. Lastly, we show thatp=k∗/\(n−1\)p=k^\{\\ast\}/\(n\-1\)satisfiesp​\(1−p\)=Ω​\(1\)p\(1\-p\)=\\Omega\(1\)\. From

c1​nn−1<p≤c2​nn−1,\\displaystyle\\frac\{c\_\{1\}n\}\{n\-1\}<p\\leq\\frac\{c\_\{2\}n\}\{n\-1\},we havec1≤pc\_\{1\}\\leq psince1≤n/\(n−1\)1\\leq n/\(n\-1\)\. Sincen≥max⁡\{6,8​c/\(2​c−1\)\}n\\geq\\max\\\{6,8c/\(2c\-1\)\\\}, it holds thatn/\(n−1\)≤min⁡\{6/5,8​c/\(6​c\+1\)\}n/\(n\-1\)\\leq\\min\\\{6/5,8c/\(6c\+1\)\\\}, yielding

p≤c2​nn−1≤max⁡\{14\+38​c,2330\}×min⁡\{65,8​c6​c\+1\}<1\\displaystyle p\\leq\\frac\{c\_\{2\}n\}\{n\-1\}\\leq\\max\\left\\\{\\frac\{1\}\{4\}\+\\frac\{3\}\{8c\},\\frac\{23\}\{30\}\\right\\\}\\times\\min\\left\\\{\\frac\{6\}\{5\},\\frac\{8c\}\{6c\+1\}\\right\\\}<1where the last inequality follows from

2330×65=2325<1,\(14\+38​c\)×8​c6​c\+1=2​c\+36​c\+1<1\\displaystyle\\frac\{23\}\{30\}\\times\\frac\{6\}\{5\}=\\frac\{23\}\{25\}<1,\\quad\\left\(\\frac\{1\}\{4\}\+\\frac\{3\}\{8c\}\\right\)\\times\\frac\{8c\}\{6c\+1\}=\\frac\{2c\+3\}\{6c\+1\}<1forc\>1/2c\>1/2\.

Sinceccis a universal constant,p​\(1−p\)=Ω​\(1\)p\(1\-p\)=\\Omega\(1\)and this completes the proof\.

#### A\.2\.3Proof of[Section˜A\.2\.2](https://arxiv.org/html/2606.06934#A1.SS2.SSS2)

In this section, we prove[Section˜A\.2\.2](https://arxiv.org/html/2606.06934#A1.SS2.SSS2)\. We first directly compute the probability\. Letq=1−pq=1\-p\.

ℙ​\(X=n​p\)\\displaystyle\\mathbb\{P\}\(X=np\)=\(nn​p\)​pn​p​qn​q\\displaystyle=\\binom\{n\}\{np\}p^\{np\}q^\{nq\}=n\!\(n​p\)\!​\(n​q\)\!​pn​p​qn​q\\displaystyle=\\frac\{n\!\}\{\(np\)\!\(nq\)\!\}p^\{np\}q^\{nq\}
Here, we note that from Stirling’s approximation \(Robbins \[[1955](https://arxiv.org/html/2606.06934#bib.bib25)\]\), for anym∈ℕm\\in\\mathbb\{N\}, we have

2​π​m​\(me\)m<m\!<2​π​m​\(me\)m​e112​m\.\\displaystyle\\sqrt\{2\\pi m\}\\left\(\\frac\{m\}\{e\}\\right\)^\{m\}<m\!<\\sqrt\{2\\pi m\}\\left\(\\frac\{m\}\{e\}\\right\)^\{m\}e^\{\\frac\{1\}\{12m\}\}\.Applying this inequality, we have the lower bound ofℙ​\(X=n​p\)\\mathbb\{P\}\(X=np\):

ℙ​\(X=n​p\)\\displaystyle\\mathbb\{P\}\(X=np\)=n\!\(n​p\)\!​\(n​q\)\!​pn​p​qn​q\\displaystyle=\\frac\{n\!\}\{\(np\)\!\(nq\)\!\}p^\{np\}q^\{nq\}≥2​π​n​\(n/e\)n2​π​n​p​q​\(n​p/e\)n​p​\(n​q/e\)n​q​e112​n​p\+112​n​q​pn​p​qn​q\\displaystyle\\geq\\frac\{\\sqrt\{2\\pi n\}\(n/e\)^\{n\}\}\{2\\pi n\\sqrt\{pq\}\(np/e\)^\{np\}\(nq/e\)^\{nq\}e^\{\\frac\{1\}\{12np\}\+\\frac\{1\}\{12nq\}\}\}p^\{np\}q^\{nq\}=e−112​n​p​q2​π​p​q​1n\.\\displaystyle=\\frac\{e^\{\-\\frac\{1\}\{12npq\}\}\}\{\\sqrt\{2\\pi pq\}\}\\frac\{1\}\{\\sqrt\{n\}\}\.Letk=n​p∈\{1,⋯,n−1\}k=np\\in\\\{1,\\cdots,n\-1\\\}\. Then, we have

n​p​q=k​\(1−kn\)≥1−1n≥12\\displaystyle npq=k\\left\(1\-\\frac\{k\}\{n\}\\right\)\\geq 1\-\\frac\{1\}\{n\}\\geq\\frac\{1\}\{2\}forn≥2n\\geq 2\. Then,exp⁡\(−1/\(12​n​p​q\)\)≥exp⁡\(−1/6\)=Ω​\(1\)\\exp\\left\(\-1/\(12npq\)\\right\)\\geq\\exp\\left\(\-1/6\\right\)=\\Omega\(1\)\. Furthermore, sincep​q=p​\(1−p\)≤1/4pq=p\(1\-p\)\\leq 1/4, we have1/2​π​p​q=Ω​\(1\)1/\\sqrt\{2\\pi pq\}=\\Omega\(1\)\. Hence, we haveℙ​\(X=n​p\)=Ω​\(1/n\)\\mathbb\{P\}\(X=np\)=\\Omega\\left\(1/\\sqrt\{n\}\\right\)\. This completes the proof\.

### A\.3Proof of[Theorem˜4](https://arxiv.org/html/2606.06934#Thmtheorem4)

In this section, we prove[Theorem˜4](https://arxiv.org/html/2606.06934#Thmtheorem4)\. Here, we use the same loss functionf:ℝ×\{0,1\}→ℝf:\\mathbb\{R\}\\times\\\{0,1\\\}\\to\\mathbb\{R\}as in the proof of[Theorem˜3](https://arxiv.org/html/2606.06934#Thmtheorem3):

f​\(w;z\)=−z​L​w\.\\displaystyle f\(w;z\)=\-zLw\.Sinceη​L\>0\.5​Δ\\eta L\>0\.5\\Delta, the sequence\{η​L​k/n\}k=0n\\\{\\eta Lk/n\\\}\_\{k=0\}^\{n\}crosses the threshold0\.5​Δ0\.5\\Delta\. Thus, there existsk∗∈\{0,1,⋯,n−1\}k^\{\\ast\}\\in\\\{0,1,\\cdots,n\-1\\\}such that

𝖣𝖱​\(−η​L​k∗n\)−𝖣𝖱​\(−η​L​\(k∗\+1\)n\)≥Δ\.\\displaystyle\\mathsf\{DR\}\\left\(\-\\frac\{\\eta Lk^\{\\ast\}\}\{n\}\\right\)\-\\mathsf\{DR\}\\left\(\-\\frac\{\\eta L\(k^\{\\ast\}\+1\)\}\{n\}\\right\)\\geq\\Delta\.We set the neighboring two datasetsSSandS′S^\{\\prime\}such that‖S‖1=k∗\\\|S\\\|\_\{1\}=k^\{\\ast\}and‖S′‖1=k∗\+1\\\|S^\{\\prime\}\\\|\_\{1\}=k^\{\\ast\}\+1\.

Let\(wt\)t=1T\(w\_\{t\}\)\_\{t=1\}^\{T\}and\(wt′\)t=1T\(w\_\{t\}^\{\\prime\}\)\_\{t=1\}^\{T\}be the𝖣𝖱\\mathsf\{DR\}\-GD iterates onffoverSSandS′S^\{\\prime\}, respectively\. Then, we have

wT′−wT=T​\(𝖣𝖱​\(−η​L​k∗n\)−𝖣𝖱​\(−η​L​\(k∗\+1\)n\)\)≥T​Δ\.\\displaystyle w\_\{T\}^\{\\prime\}\-w\_\{T\}=T\\left\(\\mathsf\{DR\}\\left\(\-\\frac\{\\eta Lk^\{\\ast\}\}\{n\}\\right\)\-\\mathsf\{DR\}\\left\(\-\\frac\{\\eta L\(k^\{\\ast\}\+1\)\}\{n\}\\right\)\\right\)\\geq T\\Delta\.
Furthermore, by[Section˜A\.2](https://arxiv.org/html/2606.06934#A1.SS2), we have

𝖣𝖱​\(−η​L​k∗n\)−𝖣𝖱​\(−η​L​\(k∗\+1\)n\)≥η​Ln−Δ\\displaystyle\\mathsf\{DR\}\\left\(\-\\frac\{\\eta Lk^\{\\ast\}\}\{n\}\\right\)\-\\mathsf\{DR\}\\left\(\-\\frac\{\\eta L\(k^\{\\ast\}\+1\)\}\{n\}\\right\)\\geq\\frac\{\\eta L\}\{n\}\-\\Deltaand hence

wT′−wT≥T​\(η​Ln−Δ\)\.\\displaystyle w\_\{T\}^\{\\prime\}\-w\_\{T\}\\geq T\\left\(\\frac\{\\eta L\}\{n\}\-\\Delta\\right\)\.Then, we have

f​\(wT;1\)−f​\(wT′;1\)=L​\(wT′−wT\)≥L​max⁡\{Δ​T,T​\(η​Ln−Δ\)\}\\displaystyle f\(w\_\{T\};1\)\-f\(w\_\{T\}^\{\\prime\};1\)=L\(w\_\{T\}^\{\\prime\}\-w\_\{T\}\)\\geq L\\max\\left\\\{\\Delta T,T\\left\(\\frac\{\\eta L\}\{n\}\-\\Delta\\right\)\\right\\\}which completes the proof\.

### A\.4Proof of[Theorem˜5](https://arxiv.org/html/2606.06934#Thmtheorem5)

LetS=\(z1,⋯,zn\)S=\(z\_\{1\},\\cdots,z\_\{n\}\)andS′=\(z1′,⋯,zn′\)∈𝒵nS^\{\\prime\}=\(z\_\{1\}^\{\\prime\},\\cdots,z\_\{n\}^\{\\prime\}\)\\in\\mathcal\{Z\}^\{n\}be the neighboring datasets,δt=\|wt−wt′\|\\delta\_\{t\}=\|w\_\{t\}\-w\_\{t\}^\{\\prime\}\|where\(wt\)t=1T\(w\_\{t\}\)\_\{t=1\}^\{T\}and\(wt′\)t=1T\(w\_\{t\}^\{\\prime\}\)\_\{t=1\}^\{T\}be the coupled iterates fromAf,θ𝖣𝖱​\-SGD​\(S\)A\_\{f,\\theta\}^\{\{\\mathsf\{DR\}\\text\{\\rm\{\-SGD\}\}\}\}\(S\)andAf,θ𝖣𝖱​\-SGD​\(S′\)A\_\{f,\\theta\}^\{\{\\mathsf\{DR\}\\text\{\\rm\{\-SGD\}\}\}\}\(S^\{\\prime\}\), respectively\. Sinceεu​s​\(Af,θ𝖣𝖱​\-SGD\)≤L⋅εu​a​s​\(Af,θ𝖣𝖱​\-SGD\)\\varepsilon\_\{us\}\(A\_\{f,\\theta\}^\{\\mathsf\{DR\}\\text\{\\rm\{\-SGD\}\}\}\)\\leq L\\cdot\\varepsilon\_\{uas\}\(A\_\{f,\\theta\}^\{\\mathsf\{DR\}\\text\{\\rm\{\-SGD\}\}\}\), it suffices to show that𝔼​\[δT\]≤\(2​η​L\+Δ\)​T/n\\mathbb\{E\}\[\\delta\_\{T\}\]\\leq\(2\\eta L\+\\Delta\)T/n\. By the law of total expectation, we have

𝔼​\[δt\]=n−1n​𝔼​\[δt\|zit=zit′\]\+1n​𝔼​\[δt\|zit≠zit′\]\.\\displaystyle\\mathbb\{E\}\[\\delta\_\{t\}\]=\\frac\{n\-1\}\{n\}\\mathbb\{E\}\[\\delta\_\{t\}\|z\_\{i\_\{t\}\}=z\_\{i\_\{t\}\}^\{\\prime\}\]\+\\frac\{1\}\{n\}\\mathbb\{E\}\[\\delta\_\{t\}\|z\_\{i\_\{t\}\}\\neq z\_\{i\_\{t\}\}^\{\\prime\}\]\.
We first consider the upper bound of𝔼​\[δt\|zit=zit′\]\\mathbb\{E\}\[\\delta\_\{t\}\|z\_\{i\_\{t\}\}=z\_\{i\_\{t\}\}^\{\\prime\}\]\. To this end, we use the following lemma:

###### Lemma 18\.

Letf:ℝ→ℝf:\\mathbb\{R\}\\to\\mathbb\{R\}be a convex,MM\-smooth function\. Letx,y∈ℤΔx,y\\in\\mathbb\{Z\}\_\{\\Delta\}andη\>0\\eta\>0withη<2/M\\eta<2/M\. Then, it holds that

\|x−y−\(𝖣𝖱​\(η​∇f​\(x\)\)−𝖣𝖱​\(η​∇f​\(y\)\)\)\|≤\|x−y\|\.\\displaystyle\|x\-y\-\(\\mathsf\{DR\}\(\\eta\\nabla f\(x\)\)\-\\mathsf\{DR\}\(\\eta\\nabla f\(y\)\)\)\|\\leq\|x\-y\|\.

[Section˜A\.4](https://arxiv.org/html/2606.06934#A1.SS4)implies that non\-expansiveness holds for11\-dimensional𝖣𝖱​\-SGD\{\\mathsf\{DR\}\\text\{\\rm\{\-SGD\}\}\}\. See[Section˜A\.4\.1](https://arxiv.org/html/2606.06934#A1.SS4.SSS1)for the proof of[Section˜A\.4](https://arxiv.org/html/2606.06934#A1.SS4)\. By applying[Section˜A\.4](https://arxiv.org/html/2606.06934#A1.SS4), we have𝔼​\[δt\|zit=zit′\]≤𝔼​\[δt−1\]\\mathbb\{E\}\[\\delta\_\{t\}\|z\_\{i\_\{t\}\}=z\_\{i\_\{t\}\}^\{\\prime\}\]\\leq\\mathbb\{E\}\[\\delta\_\{t\-1\}\]\.

We now show the upper bound of𝔼​\[δt\|zit≠zit′\]\\mathbb\{E\}\[\\delta\_\{t\}\|z\_\{i\_\{t\}\}\\neq z\_\{i\_\{t\}\}^\{\\prime\}\]\. Conditioned onzit≠zit′z\_\{i\_\{t\}\}\\neq z\_\{i\_\{t\}\}^\{\\prime\}, we have

δt\\displaystyle\\delta\_\{t\}≤\|wt−1−wt−1′\|\+\|𝖣𝖱\(η∇f\(wt−1;zit\)\)−𝖣𝖱\(η∇f\(wt−1′;zit′\)\|\\displaystyle\\leq\|w\_\{t\-1\}\-w\_\{t\-1\}^\{\\prime\}\|\+\|\\mathsf\{DR\}\(\\eta\\nabla f\(w\_\{t\-1\};z\_\{i\_\{t\}\}\)\)\-\\mathsf\{DR\}\(\\eta\\nabla f\(w\_\{t\-1\}^\{\\prime\};z\_\{i\_\{t\}\}^\{\\prime\}\)\|≤δt−1\+2​η​L\+Δ\\displaystyle\\leq\\delta\_\{t\-1\}\+2\\eta L\+\\Deltawhere the first inequality follows from the triangle inequality, and the second inequality holds since𝖣𝖱​\(x\)∈\[x−Δ/2,x\+Δ/2\]\\mathsf\{DR\}\(x\)\\in\[x\-\\Delta/2,x\+\\Delta/2\]for allx∈ℝx\\in\\mathbb\{R\}, and\|η​∇f​\(x;zit\)−η​∇f​\(y;zit′\)\|≤2​η​L\|\\eta\\nabla f\(x;z\_\{i\_\{t\}\}\)\-\\eta\\nabla f\(y;z\_\{i\_\{t\}\}^\{\\prime\}\)\|\\leq 2\\eta Lfor allx,y∈ℝx,y\\in\\mathbb\{R\}\.

Conclusively, we have

𝔼​\[δT\]≤𝔼​\[δT−1\]\+2​η​L\+Δn≤\(2​η​L\+Δ\)​Tn\\displaystyle\\mathbb\{E\}\[\\delta\_\{T\}\]\\leq\\mathbb\{E\}\[\\delta\_\{T\-1\}\]\+\\frac\{2\\eta L\+\\Delta\}\{n\}\\leq\\frac\{\(2\\eta L\+\\Delta\)T\}\{n\}which completes the proof of the upper bound\.

#### A\.4\.1Proof of[Section˜A\.4](https://arxiv.org/html/2606.06934#A1.SS4)

For notational simplicity, we setu=η​∇f​\(x\)u=\\eta\\nabla f\(x\)andv=η​∇f​\(y\)v=\\eta\\nabla f\(y\)\. Without loss of generality, we assume thatx\>yx\>y;x=yx=yis trivial\. Sincex,y∈ℤΔx,y\\in\\mathbb\{Z\}\_\{\\Delta\}, there existsk∈ℕk\\in\\mathbb\{N\}such thatx−y=k​Δx\-y=k\\Delta\. Then,\|x−y−\(𝖣𝖱​\(u\)−𝖣𝖱​\(v\)\)\|≤\|x−y\|\|x\-y\-\(\\mathsf\{DR\}\(u\)\-\\mathsf\{DR\}\(v\)\)\|\\leq\|x\-y\|is equivalent to𝖣𝖱​\(u\)−𝖣𝖱​\(v\)∈\[0,2​k​Δ\]\\mathsf\{DR\}\(u\)\-\\mathsf\{DR\}\(v\)\\in\[0,2k\\Delta\]\.

Sinceu≥vu\\geq vand𝖣𝖱\\mathsf\{DR\}is monotonically increasing,𝖣𝖱​\(u\)−𝖣𝖱​\(v\)≥0\\mathsf\{DR\}\(u\)\-\\mathsf\{DR\}\(v\)\\geq 0\. Furthermore, since𝖣𝖱​\(x\)∈\[x−0\.5​Δ,x\+0\.5​Δ\]\\mathsf\{DR\}\(x\)\\in\[x\-0\.5\\Delta,x\+0\.5\\Delta\]for anyx∈ℝx\\in\\mathbb\{R\}, we have

𝖣𝖱​\(u\)−𝖣𝖱​\(v\)≤u\+0\.5​Δ−\(v−0\.5​Δ\)=u−v\+Δ\.\\displaystyle\\mathsf\{DR\}\(u\)\-\\mathsf\{DR\}\(v\)\\leq u\+0\.5\\Delta\-\(v\-0\.5\\Delta\)=u\-v\+\\Delta\.Here, sinceffis convex,f′f^\{\\prime\}is monotonically increasing\. Furthermore, sinceffisMM\-smooth, andη<2/M\\eta<2/M, we have

0≤u−v≤η​M​\(x−y\)<2​k​Δ\\displaystyle 0\\leq u\-v\\leq\\eta M\(x\-y\)<2k\\Deltawhich implies0≤𝖣𝖱​\(u\)−𝖣𝖱​\(v\)<\(2​k\+1\)​Δ0\\leq\\mathsf\{DR\}\(u\)\-\\mathsf\{DR\}\(v\)<\(2k\+1\)\\Delta\. Since𝖣𝖱​\(u\),𝖣𝖱​\(v\)∈ℤΔ\\mathsf\{DR\}\(u\),\\mathsf\{DR\}\(v\)\\in\\mathbb\{Z\}\_\{\\Delta\}, we have𝖣𝖱​\(u\)−𝖣𝖱​\(v\)∈\[0,2​k​Δ\]\\mathsf\{DR\}\(u\)\-\\mathsf\{DR\}\(v\)\\in\[0,2k\\Delta\]\. This completes the proof\.

### A\.5Proof of[Theorem˜6](https://arxiv.org/html/2606.06934#Thmtheorem6)

In this section, we prove[Theorem˜6](https://arxiv.org/html/2606.06934#Thmtheorem6)by considering the following loss functionf:ℝ×\{0,1\}→ℝf:\\mathbb\{R\}\\times\\\{0,1\\\}\\to\\mathbb\{R\}and neighboring datasetsSSandS′S^\{\\prime\}:

f​\(x;z\)=−z​L​x,S=\(0,0,⋯,0\),S′=\(1,0,⋯,0\)\.\\displaystyle f\(x;z\)=\-zLx,\\quad S=\(0,0,\\cdots,0\),\\quad S^\{\\prime\}=\(1,0,\\cdots,0\)\.We consider two coupled𝖣𝖱​\-SGD\{\\mathsf\{DR\}\\text\{\\rm\{\-SGD\}\}\}iterates\(wt\)t=1T\(w\_\{t\}\)\_\{t=1\}^\{T\}and\(wt′\)t=1T\(w\_\{t\}^\{\\prime\}\)\_\{t=1\}^\{T\}fromAf,θ𝖣𝖱​\-SGD​\(S\)A\_\{f,\\theta\}^\{\{\\mathsf\{DR\}\\text\{\\rm\{\-SGD\}\}\}\}\(S\)andAf,θ𝖣𝖱​\-SGD​\(S′\)A\_\{f,\\theta\}^\{\\mathsf\{DR\}\\text\{\\rm\{\-SGD\}\}\}\(S^\{\\prime\}\), respectively\. Then, it is easy to observe thatwt=0w\_\{t\}=0for allt∈\[T\]t\\in\[T\]\. Thus, we have

𝔼​\[f​\(wT;1\)−f​\(wT′;1\)\]=−𝔼​\[f​\(wT′;1\)\]=L​𝔼​\[wT′\]\.\\displaystyle\\mathbb\{E\}\[f\(w\_\{T\};1\)\-f\(w\_\{T\}^\{\\prime\};1\)\]=\-\\mathbb\{E\}\[f\(w\_\{T\}^\{\\prime\};1\)\]=L\\mathbb\{E\}\[w\_\{T\}^\{\\prime\}\]\.Thus, it suffices to show that𝔼​\[wT′\]=Ω​\(\(2​η​L−Δ\)​T/n\)\\mathbb\{E\}\[w\_\{T\}^\{\\prime\}\]=\\Omega\(\(2\\eta L\-\\Delta\)T/n\)\.

We note that conditioned onw1′,⋯,wT−1′w\_\{1\}^\{\\prime\},\\cdots,w\_\{T\-1\}^\{\\prime\}, it holds that𝔼​\[wT′\|iT≠1\]=wT−1′\\mathbb\{E\}\[w\_\{T\}^\{\\prime\}\|i\_\{T\}\\neq 1\]=w\_\{T\-1\}^\{\\prime\}, and

𝔼​\[wT′\|iT=1\]\\displaystyle\\mathbb\{E\}\[w\_\{T\}^\{\\prime\}\|i\_\{T\}=1\]=wT−1′−𝖣𝖱​\(−η​L\)\\displaystyle=w\_\{T\-1\}^\{\\prime\}\-\\mathsf\{DR\}\(\-\\eta L\)≥wT−1′\+η​L−Δ2\\displaystyle\\geq w\_\{T\-1\}^\{\\prime\}\+\\eta L\-\\frac\{\\Delta\}\{2\}where the inequality in the second line follows from the fact that𝖣𝖱​\(x\)∈\[x−Δ/2,x\+Δ/2\]\\mathsf\{DR\}\(x\)\\in\[x\-\\Delta/2,x\+\\Delta/2\]for allx∈ℝx\\in\\mathbb\{R\}\. Then, we have

𝔼​\[wT′\]\\displaystyle\\mathbb\{E\}\[w\_\{T\}^\{\\prime\}\]=1n​𝔼​\[wT′\|iT=1\]\+\(1−1n\)​𝔼​\[wT′\|iT≠1\]\\displaystyle=\\frac\{1\}\{n\}\\mathbb\{E\}\[w\_\{T\}^\{\\prime\}\|i\_\{T\}=1\]\+\\left\(1\-\\frac\{1\}\{n\}\\right\)\\mathbb\{E\}\[w\_\{T\}^\{\\prime\}\|i\_\{T\}\\neq 1\]≥1n​\(𝔼​\[wT−1′\]\+η​L−Δ2\)\+\(1−1n\)​𝔼​\[wT−1′\]\\displaystyle\\geq\\frac\{1\}\{n\}\\left\(\\mathbb\{E\}\\left\[w\_\{T\-1\}^\{\\prime\}\\right\]\+\\eta L\-\\frac\{\\Delta\}\{2\}\\right\)\+\\left\(1\-\\frac\{1\}\{n\}\\right\)\\mathbb\{E\}\[w\_\{T\-1\}^\{\\prime\}\]=𝔼​\[wT−1′\]\+2​η​L−Δ2​n≥⋯≥\(2​η​L−Δ\)​T2​n\\displaystyle=\\mathbb\{E\}\[w\_\{T\-1\}^\{\\prime\}\]\+\\frac\{2\\eta L\-\\Delta\}\{2n\}\\geq\\cdots\\geq\\frac\{\(2\\eta L\-\\Delta\)T\}\{2n\}which completes the proof\.

### A\.6Proof of[Theorem˜7](https://arxiv.org/html/2606.06934#Thmtheorem7)

In this section, we prove[Theorem˜7](https://arxiv.org/html/2606.06934#Thmtheorem7)\. Here, we construct a counterexample functionf:ℝ2×\{0,1\}→ℝf:\\mathbb\{R\}^\{2\}\\times\\\{0,1\\\}\\to\\mathbb\{R\}that achieves a desired uniform stability bound in[Theorem˜7](https://arxiv.org/html/2606.06934#Thmtheorem7)\. We note that it can easily be extended to generald\>2d\>2by considering a functiong​\(x1,⋯,xd;z\)=f​\(x1,x2;z\)g\(x\_\{1\},\\cdots,x\_\{d\};z\)=f\(x\_\{1\},x\_\{2\};z\)\.

To this end, we construct a loss functionf:ℝ2×\{0,1\}→ℝf:\\mathbb\{R\}^\{2\}\\times\\\{0,1\\\}\\to\\mathbb\{R\}by

f​\(x,y;0\)=g​\(x,y\),f​\(x,y;1\)=Δ​y\\displaystyle f\(x,y;0\)=g\(x,y\),\\quad f\(x,y;1\)=\\Delta ywhereggis a function obtained from the following lemma:

###### Lemma 19\.

LetΔ\>0\\Delta\>0andT∈ℕT\\in\\mathbb\{N\}\. Then, there existsα∈\(0,Δ/2\)\\alpha\\in\(0,\\Delta/2\)and a convex,Δ\\Delta\-Lipschitz, and\(910​T2\+85\)\\left\(\\frac\{9\}\{10T^\{2\}\}\+\\frac\{8\}\{5\}\\right\)\-smooth functiong:ℝ2→ℝg:\\mathbb\{R\}^\{2\}\\to\\mathbb\{R\}such that

g​\(0,0\)=0,𝖣𝖱​\(∇g​\(0,0\)\)=\(0,0\),g​\(x,y\)=−\(Δ2\+α\)​x−Δ220,𝖣𝖱​\(∇g​\(x,y\)\)=\(−Δ,0\)\\displaystyle g\(0,0\)=0,~\\mathsf\{DR\}\(\\nabla g\(0,0\)\)=\(0,0\),~g\(x,y\)=\-\\left\(\\frac\{\\Delta\}\{2\}\+\\alpha\\right\)x\-\\frac\{\\Delta^\{2\}\}\{20\},~\\mathsf\{DR\}\(\\nabla g\(x,y\)\)=\(\-\\Delta,0\)for allx,yx,ysuch thatx∈\{0,Δ,2​Δ,⋯,T​Δ\}x\\in\\\{0,\\Delta,2\\Delta,\\cdots,T\\Delta\\\}andy≤−Δy\\leq\-\\Delta\.

[Section˜A\.6](https://arxiv.org/html/2606.06934#A1.SS6)implies that there exists a convex and smooth functiong:ℝ2→ℝg:\\mathbb\{R\}^\{2\}\\to\\mathbb\{R\}whose rounded gradient is\(0,0\)\(0,0\)at the origin but consistently induces\(−Δ,0\)\(\-\\Delta,0\)gradient on the region\{\(x,y\):x∈\{0,⋯,T​Δ\},y≤−Δ\}\\\{\(x,y\):x\\in\\\{0,\\cdots,T\\Delta\\\},y\\leq\-\\Delta\\\}\. The proof of[Section˜A\.6](https://arxiv.org/html/2606.06934#A1.SS6)is in[Section˜A\.6\.1](https://arxiv.org/html/2606.06934#A1.SS6.SSS1)\.

LetS=\(0,⋯,0\)S=\(0,\\cdots,0\)andS′=\(1,0,⋯,0\)S^\{\\prime\}=\(1,0,\\cdots,0\)be the neighboring datasets,\{wt\}t=1T\\\{w\_\{t\}\\\}\_\{t=1\}^\{T\}and\{wt′\}t=1T\\\{w\_\{t\}^\{\\prime\}\\\}\_\{t=1\}^\{T\}be trajectories fromAf,θ𝖣𝖱​\-SGD​\(S\)A\_\{f,\\theta\}^\{\{\\mathsf\{DR\}\\text\{\\rm\{\-SGD\}\}\}\}\(S\)andAf,θ𝖣𝖱​\-SGD​\(S′\)A\_\{f,\\theta\}^\{\\mathsf\{DR\}\\text\{\\rm\{\-SGD\}\}\}\(S^\{\\prime\}\), respectively, wherewt=\(wt,1,wt,2\)w\_\{t\}=\(w\_\{t,1\},w\_\{t,2\}\)andwt′=\(wt,1′,wt,2′\)w\_\{t\}^\{\\prime\}=\(w\_\{t,1\}^\{\\prime\},w\_\{t,2\}^\{\\prime\}\)for eacht∈\[T\]t\\in\[T\]\. Then sinceS=\(0,⋯,0\)S=\(0,\\cdots,0\)and𝖣𝖱​\(∇f​\(w0;0\)\)=\(0,0\)\\mathsf\{DR\}\(\\nabla f\(w\_\{0\};0\)\)=\(0,0\), we havewt=\(0,0\)w\_\{t\}=\(0,0\)for allt∈\[T\]t\\in\[T\]\.

Then, we have

𝔼​\[f​\(wT;0\)−f​\(wT′;0\)\]\\displaystyle\\mathbb\{E\}\[f\(w\_\{T\};0\)\-f\(w\_\{T\}^\{\\prime\};0\)\]=𝔼​\[g​\(wT\)−g​\(wT′\)\]=−𝔼​\[g​\(wT′\)\]\.\\displaystyle=\\mathbb\{E\}\[g\(w\_\{T\}\)\-g\(w\_\{T\}^\{\\prime\}\)\]=\-\\mathbb\{E\}\[g\(w\_\{T\}^\{\\prime\}\)\]\.Here, since𝖣𝖱​\(∇f​\(w0;0\)\)=\(0,0\)\\mathsf\{DR\}\(\\nabla f\(w\_\{0\};0\)\)=\(0,0\), we havewt′=\(0,0\)w\_\{t\}^\{\\prime\}=\(0,0\)for allt<τt<\\tauwhereτ:=min⁡\{t:it=1\}\\tau:=\\min\\\{t:i\_\{t\}=1\\\}\. Whent=τt=\\tau, i\.e\.,it=1i\_\{t\}=1is observed, we havewτ′=\(0,−Δ\)w\_\{\\tau\}^\{\\prime\}=\(0,\-\\Delta\)since𝖣𝖱​\(η​∇f​\(x,y;1\)\)=\(0,Δ\)\\mathsf\{DR\}\(\\eta\\nabla f\(x,y;1\)\)=\(0,\\Delta\)\. Afterτ\\tau, wheneverit≠1i\_\{t\}\\neq 1, the update usesgg, and hencewt′=wt−1′\+\(Δ,0\)w\_\{t\}^\{\\prime\}=w\_\{t\-1\}^\{\\prime\}\+\(\\Delta,0\)\. Wheneverit=1i\_\{t\}=1, the update only decreases the second coordinate byΔ\\Delta, so the iterate remains in the regiony≤−Δy\\leq\-\\Delta\. Thus, we have

𝔼​\[f​\(wT;0\)−f​\(wT′;0\)\]=−𝔼​\[g​\(wT′\)\]≥\(Δ2\+α\)​𝔼​\[wT,1′\]=Ω​\(Δ​𝔼​\[wT,1′\]\)\.\\displaystyle\\mathbb\{E\}\[f\(w\_\{T\};0\)\-f\(w\_\{T\}^\{\\prime\};0\)\]=\-\\mathbb\{E\}\[g\(w\_\{T\}^\{\\prime\}\)\]\\geq\\left\(\\frac\{\\Delta\}\{2\}\+\\alpha\\right\)\\mathbb\{E\}\[w\_\{T,1\}^\{\\prime\}\]=\\Omega\(\\Delta\\mathbb\{E\}\[w\_\{T,1\}^\{\\prime\}\]\)\.
LetItI\_\{t\}be an indicator variable such thatIt=1I\_\{t\}=1ifit≠1i\_\{t\}\\neq 1andt\>τt\>\\tau, i\.e\., the event thatwt′=wt−1′\+\(Δ,0\)w\_\{t\}^\{\\prime\}=w\_\{t\-1\}^\{\\prime\}\+\(\\Delta,0\)occurs\. Then, we have

ℙ​\(It=1\)=\(1−\(1−1n\)t−1\)​\(1−1n\)\.\\displaystyle\\mathbb\{P\}\(I\_\{t\}=1\)=\\left\(1\-\\left\(1\-\\frac\{1\}\{n\}\\right\)^\{t\-1\}\\right\)\\left\(1\-\\frac\{1\}\{n\}\\right\)\.Letq=1−1/nq=1\-1/n\. Then, we have

𝔼​\[wT,1′\]\\displaystyle\\mathbb\{E\}\[w\_\{T,1\}^\{\\prime\}\]=Δ​∑t=1Tq​\(1−qt−1\)\\displaystyle=\\Delta\\sum\_\{t=1\}^\{T\}q\(1\-q^\{t\-1\}\)=Δ​q​\(T−1−qT1−q\)\\displaystyle=\\Delta q\\left\(T\-\\frac\{1\-q^\{T\}\}\{1\-q\}\\right\)=Δ​\(1−1n\)​\(T−n​\(1−\(1−1n\)T\)\)\.\\displaystyle=\\Delta\\left\(1\-\\frac\{1\}\{n\}\\right\)\\left\(T\-n\\left\(1\-\\left\(1\-\\frac\{1\}\{n\}\\right\)^\{T\}\\right\)\\right\)\.We complete the proof using the following technical lemma\. The proof of[Section˜A\.6](https://arxiv.org/html/2606.06934#A1.SS6)is in[Section˜A\.6\.2](https://arxiv.org/html/2606.06934#A1.SS6.SSS2)\.

###### Lemma 20\.

Letn∈ℕn\\in\\mathbb\{N\}andT∈ℕT\\in\\mathbb\{N\}withn≥2,T≥2n\\geq 2,T\\geq 2\. Then, it holds that

\(1−1n\)​\(T−n​\(1−\(1−1n\)T\)\)=Ω​\(min⁡\{1,Tn\}​T\)\.\\displaystyle\\left\(1\-\\frac\{1\}\{n\}\\right\)\\left\(T\-n\\left\(1\-\\left\(1\-\\frac\{1\}\{n\}\\right\)^\{T\}\\right\)\\right\)=\\Omega\\left\(\\min\\left\\\{1,\\frac\{T\}\{n\}\\right\\\}T\\right\)\.

By applying[Section˜A\.6](https://arxiv.org/html/2606.06934#A1.SS6), we have

𝔼​\[wT,1′\]=Ω​\(min⁡\{1,Tn\}​Δ​T\)\.\\displaystyle\\mathbb\{E\}\[w\_\{T,1\}^\{\\prime\}\]=\\Omega\\left\(\\min\\left\\\{1,\\frac\{T\}\{n\}\\right\\\}\\Delta T\\right\)\.Conclusively, we have

𝔼​\[f​\(wT;0\)−f​\(wT′;0\)\]=−𝔼​\[g​\(wT′\)\]=Ω​\(Δ​𝔼​\[wT,1′\]\)=Ω​\(min⁡\{1,Tn\}​Δ2​T\)\\displaystyle\\mathbb\{E\}\[f\(w\_\{T\};0\)\-f\(w\_\{T\}^\{\\prime\};0\)\]=\-\\mathbb\{E\}\[g\(w\_\{T\}^\{\\prime\}\)\]=\\Omega\(\\Delta\\mathbb\{E\}\[w\_\{T,1\}^\{\\prime\}\]\)=\\Omega\\left\(\\min\\left\\\{1,\\frac\{T\}\{n\}\\right\\\}\\Delta^\{2\}T\\right\)which completes the proof\.

#### A\.6\.1Proof of[Section˜A\.6](https://arxiv.org/html/2606.06934#A1.SS6)

In this section, we prove[Section˜A\.6](https://arxiv.org/html/2606.06934#A1.SS6)\. To this end, we first consider a smoothed version of a piecewise\-linear functionmax⁡\{L1,L2\}\\max\\\{L\_\{1\},L\_\{2\}\\\}that satisfies the gradient conditions in[Section˜A\.6](https://arxiv.org/html/2606.06934#A1.SS6)\.

We first consider a smoothing function: forε\>0\\varepsilon\>0,Hε:ℝ→ℝH\_\{\\varepsilon\}:\\mathbb\{R\}\\to\\mathbb\{R\}is defined by

Hε​\(x\)=\{0if​x≤−ε,\(x\+ε\)24​εif−ε<x<ε,xif​x≥ε\.\\displaystyle H\_\{\\varepsilon\}\(x\)=\\begin\{cases\}0&\\text\{if \}x\\leq\-\\varepsilon,\\\\ \\frac\{\(x\+\\varepsilon\)^\{2\}\}\{4\\varepsilon\}&\\text\{if \}\-\\varepsilon<x<\\varepsilon,\\\\ x&\\text\{if \}x\\geq\\varepsilon\.\\end\{cases\}One can observe thatHε​\(x\)H\_\{\\varepsilon\}\(x\)can be regarded as a smoothed version ofmax⁡\{0,x\}\\max\\\{0,x\\\}andHεH\_\{\\varepsilon\}is convex\.

We define a loss functiong:ℝ2→ℝg:\\mathbb\{R\}^\{2\}\\to\\mathbb\{R\}by

g​\(x,y\)=L1​\(x,y\)\+Hε​\(L2​\(x,y\)−L1​\(x,y\)\)\\displaystyle g\(x,y\)=L\_\{1\}\(x,y\)\+H\_\{\\varepsilon\}\\left\(L\_\{2\}\(x,y\)\-L\_\{1\}\(x,y\)\\right\)where

L1​\(x,y\)=−\(Δ2−Δ10​T\)​x\+2​Δ5​y,L2​\(x,y\)=−\(Δ2\+Δ5​T\)​x−Δ220\\displaystyle L\_\{1\}\(x,y\)=\-\\left\(\\frac\{\\Delta\}\{2\}\-\\frac\{\\Delta\}\{10T\}\\right\)x\+\\frac\{2\\Delta\}\{5\}y,\\quad L\_\{2\}\(x,y\)=\-\\left\(\\frac\{\\Delta\}\{2\}\+\\frac\{\\Delta\}\{5T\}\\right\)x\-\\frac\{\\Delta^\{2\}\}\{20\}andε\>0\\varepsilon\>0\. We note that sinceL2−L1L\_\{2\}\-L\_\{1\}is affine andHεH\_\{\\varepsilon\}is convex,Hε∘\(L2−L1\)H\_\{\\varepsilon\}\\circ\(L\_\{2\}\-L\_\{1\}\)is convex; adding the affine functionL1L\_\{1\}preserves convexity\. We now assign an explicit value toε\\varepsilonto satisfy the gradient conditions of[Section˜A\.6](https://arxiv.org/html/2606.06934#A1.SS6)\. To this end, we determineε\>0\\varepsilon\>0so that∇g​\(0,0\)=∇L1​\(0,0\)\\nabla g\(0,0\)=\\nabla L\_\{1\}\(0,0\), and∇g​\(x,y\)=∇L2​\(x,y\)\\nabla g\(x,y\)=\\nabla L\_\{2\}\(x,y\)for allx∈\{0,Δ,2​Δ,⋯,T​Δ\}x\\in\\\{0,\\Delta,2\\Delta,\\cdots,T\\Delta\\\}andy≤−Δy\\leq\-\\Delta\. Then, we have

∇g​\(0,0\)=∇L1​\(0,0\)=\(−Δ2\+Δ10​T,2​Δ5\),∇g​\(x,y\)=∇L2​\(x,y\)=\(−Δ2−Δ5​T,0\)\\displaystyle\\nabla g\(0,0\)=\\nabla L\_\{1\}\(0,0\)=\\left\(\-\\frac\{\\Delta\}\{2\}\+\\frac\{\\Delta\}\{10T\},\\frac\{2\\Delta\}\{5\}\\right\),\\quad\\nabla g\(x,y\)=\\nabla L\_\{2\}\(x,y\)=\\left\(\-\\frac\{\\Delta\}\{2\}\-\\frac\{\\Delta\}\{5T\},0\\right\)for eachx∈\{0,Δ,⋯,T​Δ\}x\\in\\\{0,\\Delta,\\cdots,T\\Delta\\\}andy≤−Δy\\leq\-\\Delta\.

We first derive the condition forε\>0\\varepsilon\>0such that∇g​\(0,0\)=∇L1​\(0,0\)\\nabla g\(0,0\)=\\nabla L\_\{1\}\(0,0\)\. Sinceg​\(x,y\)=L1​\(x,y\)g\(x,y\)=L\_\{1\}\(x,y\)ifHε​\(L2​\(x,y\)−L1​\(x,y\)\)=0H\_\{\\varepsilon\}\(L\_\{2\}\(x,y\)\-L\_\{1\}\(x,y\)\)=0, it suffices to considerε\>0\\varepsilon\>0such thatL2​\(0,0\)−L1​\(0,0\)≤−εL\_\{2\}\(0,0\)\-L\_\{1\}\(0,0\)\\leq\-\\varepsilon\. Thus, we obtain the first requirement ofε\\varepsilon:

L2​\(0,0\)−L1​\(0,0\)=−Δ220≤−ε⇔ε≤Δ220\.\\displaystyle L\_\{2\}\(0,0\)\-L\_\{1\}\(0,0\)=\-\\frac\{\\Delta^\{2\}\}\{20\}\\leq\-\\varepsilon\\iff\\varepsilon\\leq\\frac\{\\Delta^\{2\}\}\{20\}\.
We now consider∇g​\(x,y\)=∇L2​\(x,y\)\\nabla g\(x,y\)=\\nabla L\_\{2\}\(x,y\)such thatx∈\{0,Δ,⋯,T​Δ\}x\\in\\\{0,\\Delta,\\cdots,T\\Delta\\\}andy≤−Δy\\leq\-\\Delta\. Sinceg​\(x,y\)=L2​\(x,y\)g\(x,y\)=L\_\{2\}\(x,y\)ifHε​\(L2​\(x,y\)−L1​\(x,y\)\)=L2​\(x,y\)−L1​\(x,y\)H\_\{\\varepsilon\}\(L\_\{2\}\(x,y\)\-L\_\{1\}\(x,y\)\)=L\_\{2\}\(x,y\)\-L\_\{1\}\(x,y\), we considerε\>0\\varepsilon\>0satisfyingL2​\(x,y\)−L1​\(x,y\)≥εL\_\{2\}\(x,y\)\-L\_\{1\}\(x,y\)\\geq\\varepsilon\. Here, since

L2​\(x,y\)−L1​\(x,y\)=−3​Δ10​T​x−2​Δ5​y−Δ220\\displaystyle L\_\{2\}\(x,y\)\-L\_\{1\}\(x,y\)=\-\\frac\{3\\Delta\}\{10T\}x\-\\frac\{2\\Delta\}\{5\}y\-\\frac\{\\Delta^\{2\}\}\{20\}is minimized at\(T​Δ,−Δ\)\(T\\Delta,\-\\Delta\), it suffices to derive the condition forε\\varepsilonsuch thatL2​\(T​Δ,−Δ\)−L1​\(T​Δ,−Δ\)≥εL\_\{2\}\(T\\Delta,\-\\Delta\)\-L\_\{1\}\(T\\Delta,\-\\Delta\)\\geq\\varepsilonforx∈\{0,⋯,T​Δ\}x\\in\\\{0,\\cdots,T\\Delta\\\}\. We have

L2​\(x,−Δ\)−L1​\(x,−Δ\)\\displaystyle L\_\{2\}\(x,\-\\Delta\)\-L\_\{1\}\(x,\-\\Delta\)=\(−\(Δ2\+Δ5​T\)​x−Δ220\)−\(−\(Δ2−Δ10​T\)​x−2​Δ25\)\\displaystyle=\\left\(\-\\left\(\\frac\{\\Delta\}\{2\}\+\\frac\{\\Delta\}\{5T\}\\right\)x\-\\frac\{\\Delta^\{2\}\}\{20\}\\right\)\-\\left\(\-\\left\(\\frac\{\\Delta\}\{2\}\-\\frac\{\\Delta\}\{10T\}\\right\)x\-\\frac\{2\\Delta^\{2\}\}\{5\}\\right\)=2​Δ25−3​Δ10​T​x−Δ220\\displaystyle=\\frac\{2\\Delta^\{2\}\}\{5\}\-\\frac\{3\\Delta\}\{10T\}x\-\\frac\{\\Delta^\{2\}\}\{20\}=7​Δ220−3​Δ​x10​T\.\\displaystyle=\\frac\{7\\Delta^\{2\}\}\{20\}\-\\frac\{3\\Delta x\}\{10T\}\.The above expression has a minimum value atx=T​Δx=T\\Delta:

L2​\(x,−Δ\)−L1​\(x,−Δ\)≥7​Δ220−3​Δ210=Δ220\\displaystyle L\_\{2\}\(x,\-\\Delta\)\-L\_\{1\}\(x,\-\\Delta\)\\geq\\frac\{7\\Delta^\{2\}\}\{20\}\-\\frac\{3\\Delta^\{2\}\}\{10\}=\\frac\{\\Delta^\{2\}\}\{20\}Thus, we needε≤Δ2/20\\varepsilon\\leq\\Delta^\{2\}/20\. Thus, if we chooseε=Δ2/20\\varepsilon=\\Delta^\{2\}/20then ourggsatisfies the gradient conditions at the origin, and at\(w,−Δ\)\(w,\-\\Delta\)for eachw∈\{0,⋯,T​Δ\}w\\in\\\{0,\\cdots,T\\Delta\\\}\.

We now evaluate the Lipschitzness and smoothness constant ofgg\. Letε0=Δ2/20\\varepsilon\_\{0\}=\\Delta^\{2\}/20andL0​\(x,y\)=L2​\(x,y\)−L1​\(x,y\)L\_\{0\}\(x,y\)=L\_\{2\}\(x,y\)\-L\_\{1\}\(x,y\)\. Then, we have

∇g​\(x,y\)=∇L1​\(x,y\)\+Hε′​\(L0​\(x,y\)\)​∇L0​\(x,y\)\.\\displaystyle\\nabla g\(x,y\)=\\nabla L\_\{1\}\(x,y\)\+H\_\{\\varepsilon\}^\{\\prime\}\(L\_\{0\}\(x,y\)\)\\nabla L\_\{0\}\(x,y\)\.Since0≤Hε′​\(x\)≤10\\leq H^\{\\prime\}\_\{\\varepsilon\}\(x\)\\leq 1, one can observe that∇g\\nabla gis a convex combination of∇L1\\nabla L\_\{1\}and∇L2\\nabla L\_\{2\}\. Here, since‖∇L1‖2=‖\(Δ/2−Δ/\(10​T\),2​Δ/5\)‖2≤Δ\\\|\\nabla L\_\{1\}\\\|\_\{2\}=\\\|\(\\Delta/2\-\\Delta/\(10T\),2\\Delta/5\)\\\|\_\{2\}\\leq\\Deltaand‖∇L2‖2=‖\(Δ/2\+Δ/\(5​T\),0\)‖2≤Δ\\\|\\nabla L\_\{2\}\\\|\_\{2\}=\\\|\(\\Delta/2\+\\Delta/\(5T\),0\)\\\|\_\{2\}\\leq\\Deltafor allT≥1T\\geq 1,ggisΔ\\Delta\-Lipschitz\.

Furthermore, sinceHε0′H\_\{\\varepsilon\_\{0\}\}^\{\\prime\}is1/\(2​ε0\)1/\(2\\varepsilon\_\{0\}\)\-Lipschitz,∇g​\(x,y\)\\nabla g\(x,y\)is Lipschitz with constant at most

12​ε0​‖∇L0​\(x,y\)‖22=12​ε0​\(9​Δ2100​T2\+4​Δ225\)=910​T2\+85\\displaystyle\\frac\{1\}\{2\\varepsilon\_\{0\}\}\\\|\\nabla L\_\{0\}\(x,y\)\\\|^\{2\}\_\{2\}=\\frac\{1\}\{2\\varepsilon\_\{0\}\}\\left\(\\frac\{9\\Delta^\{2\}\}\{100T^\{2\}\}\+\\frac\{4\\Delta^\{2\}\}\{25\}\\right\)=\\frac\{9\}\{10T^\{2\}\}\+\\frac\{8\}\{5\}which implies that ourggis\(9/\(10​T2\)\+8/5\)\(9/\(10T^\{2\}\)\+8/5\)\-smooth\. This completes the proof with the choice ofα=Δ/\(5​T\)\\alpha=\\Delta/\(5T\)\.

#### A\.6\.2Proof of[Section˜A\.6](https://arxiv.org/html/2606.06934#A1.SS6)

In this section, we prove[Section˜A\.6](https://arxiv.org/html/2606.06934#A1.SS6)\. To this end, we show the lower bound for the following two cases: \(1\)T≥nT\\geq n, and \(2\)T<nT<n\. For notational simplicity, we denoteE​\(n,T\)E\(n,T\)by the given expression in[Section˜A\.6](https://arxiv.org/html/2606.06934#A1.SS6):E​\(n,T\)=\(1−1n\)​\(T−n​\(1−\(1−1n\)T\)\)E\(n,T\)=\\left\(1\-\\frac\{1\}\{n\}\\right\)\\left\(T\-n\\left\(1\-\\left\(1\-\\frac\{1\}\{n\}\\right\)^\{T\}\\right\)\\right\)\. It suffices to show that

E​\(n,T\)=Ω​\(min⁡\{1,Tn\}​T\)\.\\displaystyle E\(n,T\)=\\Omega\\left\(\\min\\left\\\{1,\\frac\{T\}\{n\}\\right\\\}T\\right\)\.Forn≥2n\\geq 2,1−1/n≥1/21\-1/n\\geq 1/2holds\. Thus, we have

E​\(n,T\)\\displaystyle E\(n,T\)≥12​\(T−n​\(1−\(1−1n\)T\)\)\\displaystyle\\geq\\frac\{1\}\{2\}\\left\(T\-n\\left\(1\-\\left\(1\-\\frac\{1\}\{n\}\\right\)^\{T\}\\right\)\\right\)\(6\)
We first show that ifT≥nT\\geq n, thenE​\(n,T\)=Ω​\(T\)E\(n,T\)=\\Omega\(T\)\. Here, we consider the following two sub\-cases:T≥2​nT\\geq 2nandn≤T≤2​nn\\leq T\\leq 2n\.

The case whereT≥2​nT\\geq 2nis easily obtained since

T−n​\(1−\(1−1n\)T\)≥T−n≥T2\.\\displaystyle T\-n\\left\(1\-\\left\(1\-\\frac\{1\}\{n\}\\right\)^\{T\}\\right\)\\geq T\-n\\geq\\frac\{T\}\{2\}\.
We now suppose thatn≤T≤2​nn\\leq T\\leq 2n\. Then, we have

\(1−1n\)T≥\(1−1n\)2​n≥116\\displaystyle\\left\(1\-\\frac\{1\}\{n\}\\right\)^\{T\}\\geq\\left\(1\-\\frac\{1\}\{n\}\\right\)^\{2n\}\\geq\\frac\{1\}\{16\}since\(1−1/n\)n\\left\(1\-1/n\\right\)^\{n\}is strictly increasing forn≥2n\\geq 2\. Thus, we have

T−n​\(1−\(1−1n\)T\)≥T−15​n16≥T16\.\\displaystyle T\-n\\left\(1\-\\left\(1\-\\frac\{1\}\{n\}\\right\)^\{T\}\\right\)\\geq T\-\\frac\{15n\}\{16\}\\geq\\frac\{T\}\{16\}\.By combining these results with[Eq\.˜6](https://arxiv.org/html/2606.06934#A1.E6), we haveE​\(n,T\)=Ω​\(T\)E\(n,T\)=\\Omega\(T\)ifT≥nT\\geq n\.

We now show that ifT<nT<n, thenE​\(n,T\)=Ω​\(T2/n\)E\(n,T\)=\\Omega\(T^\{2\}/n\)\. To this end, we consider the binomial expansion of\(1−1/n\)T\(1\-1/n\)^\{T\}:

\(1−1n\)T=∑i=0T\(Ti\)​\(−1n\)i\\displaystyle\\left\(1\-\\frac\{1\}\{n\}\\right\)^\{T\}=\\sum\_\{i=0\}^\{T\}\{T\\choose i\}\\left\(\-\\frac\{1\}\{n\}\\right\)^\{i\}Here, sinceT<nT<n, the magnitude of the terms\(Ti\)​\(−1n\)i\{T\\choose i\}\\left\(\-\\frac\{1\}\{n\}\\right\)^\{i\}is monotonically decreasing:

\(Ti\+1\)​n−\(i\+1\)\(Ti\)​n−i=T−i\(i\+1\)​n≤Tn<1\\displaystyle\\frac\{\{T\\choose i\+1\}n^\{\-\(i\+1\)\}\}\{\{T\\choose i\}n^\{\-i\}\}=\\frac\{T\-i\}\{\(i\+1\)n\}\\leq\\frac\{T\}\{n\}<1
Then, truncating the series at the negative term gives a lower bound of the binomial expansion:

\(1−1n\)T≥1−Tn\+T​\(T−1\)2​n2−T​\(T−1\)​\(T−2\)6​n3\.\\displaystyle\\left\(1\-\\frac\{1\}\{n\}\\right\)^\{T\}\\geq 1\-\\frac\{T\}\{n\}\+\\frac\{T\(T\-1\)\}\{2n^\{2\}\}\-\\frac\{T\(T\-1\)\(T\-2\)\}\{6n^\{3\}\}\.Then, we have

T−n​\(1−\(1−1n\)T\)\\displaystyle T\-n\\left\(1\-\\left\(1\-\\frac\{1\}\{n\}\\right\)^\{T\}\\right\)≥T−n\(1−\(1−Tn\+T​\(T−1\)2​n2−T​\(T−1\)​\(T−2\)6​n3\.\)\)\\displaystyle\\geq T\-n\\left\(1\-\\left\(1\-\\frac\{T\}\{n\}\+\\frac\{T\(T\-1\)\}\{2n^\{2\}\}\-\\frac\{T\(T\-1\)\(T\-2\)\}\{6n^\{3\}\}\.\\right\)\\right\)=T​\(T−1\)2​n−T​\(T−1\)​\(T−2\)6​n2\\displaystyle=\\frac\{T\(T\-1\)\}\{2n\}\-\\frac\{T\(T\-1\)\(T\-2\)\}\{6n^\{2\}\}=T​\(T−1\)2​n​\(1−T−23​n\)\\displaystyle=\\frac\{T\(T\-1\)\}\{2n\}\\left\(1\-\\frac\{T\-2\}\{3n\}\\right\)SinceT<nT<n, we have1−T−23​n≥1−T3​n≥231\-\\frac\{T\-2\}\{3n\}\\geq 1\-\\frac\{T\}\{3n\}\\geq\\frac\{2\}\{3\}\. Furthermore, ifT≥2T\\geq 2,T​\(T−1\)≥T2/2T\(T\-1\)\\geq T^\{2\}/2\. Then, by combining this result with[Eq\.˜6](https://arxiv.org/html/2606.06934#A1.E6), we obtain that ifT<nT<n,

E​\(n,T\)≥12​\(T26​n\)=Ω​\(T2n\)\.\\displaystyle E\(n,T\)\\geq\\frac\{1\}\{2\}\\left\(\\frac\{T^\{2\}\}\{6n\}\\right\)=\\Omega\\left\(\\frac\{T^\{2\}\}\{n\}\\right\)\.Conclusively, we have

E​\(n,T\)=\{Ω​\(T\)T≥nΩ​\(T2/n\)T<n=Ω​\(min⁡\{1,Tn\}​T\),\\displaystyle E\(n,T\)=\\begin\{cases\}\\Omega\(T\)&T\\geq n\\\\ \\Omega\(T^\{2\}/n\)&T<n\\end\{cases\}=\\Omega\\left\(\\min\\left\\\{1,\\frac\{T\}\{n\}\\right\\\}T\\right\),which completes the proof\.

### A\.7Proof of[Theorem˜8](https://arxiv.org/html/2606.06934#Thmtheorem8)

In this section, we prove[Theorem˜8](https://arxiv.org/html/2606.06934#Thmtheorem8)\. LetS,S′∈𝒵nS,S^\{\\prime\}\\in\\mathcal\{Z\}^\{n\}be the neighboring datasets, and\(wt\)t=1T\(w\_\{t\}\)\_\{t=1\}^\{T\}and\(wt′\)t=1T\(w\_\{t\}^\{\\prime\}\)\_\{t=1\}^\{T\}be the coupled𝖣𝖱​\-SGD\{\\mathsf\{DR\}\\text\{\\rm\{\-SGD\}\}\}iterates fromAf,θ𝖣𝖱​\-SGD​\(S\)A^\{\{\\mathsf\{DR\}\\text\{\\rm\{\-SGD\}\}\}\}\_\{f,\\theta\}\(S\)andAf,θ𝖣𝖱​\-SGD​\(S′\)A\_\{f,\\theta\}^\{\{\\mathsf\{DR\}\\text\{\\rm\{\-SGD\}\}\}\}\(S^\{\\prime\}\), respectively\. Letδt=wt−wt′\\delta\_\{t\}=w\_\{t\}\-w\_\{t\}^\{\\prime\}for eacht∈\[T\]t\\in\[T\]\.

For eacht∈\[T\]t\\in\[T\], letBtB\_\{t\}be the event that there existsτ≤t\\tau\\leq tsuch thatziτ≠ziτ′z\_\{i\_\{\\tau\}\}\\neq z\_\{i\_\{\\tau\}\}^\{\\prime\}andAtA\_\{t\}be the event thatzit≠zit′z\_\{i\_\{t\}\}\\neq z\_\{i\_\{t\}\}^\{\\prime\}\. SincewT=wT′w\_\{T\}=w\_\{T\}^\{\\prime\}conditioned onBTcB\_\{T\}^\{c\}, we have𝔼​\[‖δT‖2\]=P​\(BT\)​𝔼​\[‖δT‖2\|BT\]\\mathbb\{E\}\[\\\|\\delta\_\{T\}\\\|\_\{2\}\]=P\(B\_\{T\}\)\\mathbb\{E\}\[\\\|\\delta\_\{T\}\\\|\_\{2\}\|B\_\{T\}\]\.

Here, we have

P​\(BT\)=1−\(1−1n\)T≤min⁡\{1,Tn\}\.\\displaystyle P\(B\_\{T\}\)=1\-\\left\(1\-\\frac\{1\}\{n\}\\right\)^\{T\}\\leq\\min\\left\\\{1,\\frac\{T\}\{n\}\\right\\\}\.\(7\)
In the rest of the proof, we show the upper bound of𝔼​\[‖δT‖2\|BT\]\\mathbb\{E\}\[\\\|\\delta\_\{T\}\\\|\_\{2\}\|B\_\{T\}\]by constructing a recurrence of𝔼​\[‖δt‖2\|Bt\]\\mathbb\{E\}\[\\\|\\delta\_\{t\}\\\|\_\{2\}\|B\_\{t\}\]\. Fort=1t=1, we have

𝔼​\[‖δ1‖2\|B1\]=𝔼​\[‖δ1‖2\|A1\]\\displaystyle\\mathbb\{E\}\[\\\|\\delta\_\{1\}\\\|\_\{2\}\|B\_\{1\}\]=\\mathbb\{E\}\[\\\|\\delta\_\{1\}\\\|\_\{2\}\|A\_\{1\}\]=𝔼​\[‖𝖣𝖱​\(η​∇f​\(w0;zi1\)\)−𝖣𝖱​\(η​∇f​\(w0;zi1′\)\)‖2\|zi1≠zi1′\]\\displaystyle=\\mathbb\{E\}\[\\\|\\mathsf\{DR\}\(\\eta\\nabla f\(w\_\{0\};z\_\{i\_\{1\}\}\)\)\-\\mathsf\{DR\}\(\\eta\\nabla f\(w\_\{0\};z\_\{i\_\{1\}\}^\{\\prime\}\)\)\\\|\_\{2\}\|z\_\{i\_\{1\}\}\\neq z\_\{i\_\{1\}\}^\{\\prime\}\]≤‖𝖣𝖱​\(η​∇f​\(w0;zi1\)\)‖2\+‖𝖣𝖱​\(η​∇f​\(w0;zi1′\)\)‖2≤4​η​L\\displaystyle\\leq\\\|\\mathsf\{DR\}\(\\eta\\nabla f\(w\_\{0\};z\_\{i\_\{1\}\}\)\)\\\|\_\{2\}\+\\\|\\mathsf\{DR\}\(\\eta\\nabla f\(w\_\{0\};z\_\{i\_\{1\}\}^\{\\prime\}\)\)\\\|\_\{2\}\\leq 4\\eta Lwhere the second inequality follows from the triangle inequality, and the last inequality follows from[Section˜A\.1](https://arxiv.org/html/2606.06934#A1.SS1)\.

We now consider generalt≥2t\\geq 2\. Here,BtB\_\{t\}can be regarded as the union of two mutually exclusive events,Bt−1c∩AtB\_\{t\-1\}^\{c\}\\cap A\_\{t\}andBt−1B\_\{t\-1\}\. Thus, we can decompose𝔼​\[‖δt‖2\|Bt\]\\mathbb\{E\}\[\\\|\\delta\_\{t\}\\\|\_\{2\}\|B\_\{t\}\]as follows:

𝔼​\[∥δt∥2\|Bt\]\\displaystyle\\mathbb\{E\}\[\\lVert\\delta\_\{t\}\\rVert\_\{2\}\|B\_\{t\}\]=P​\(Bt−1c∩At\)P​\(Bt\)​𝔼​\[∥δt∥2\|Bt−1c∩At\]\+P​\(Bt−1\)P​\(Bt\)​𝔼​\[∥δt∥2\|Bt−1\]\\displaystyle=\\frac\{P\(B\_\{t\-1\}^\{c\}\\cap A\_\{t\}\)\}\{P\(B\_\{t\}\)\}\\mathbb\{E\}\[\\lVert\\delta\_\{t\}\\rVert\_\{2\}\|B\_\{t\-1\}^\{c\}\\cap A\_\{t\}\]\+\\frac\{P\(B\_\{t\-1\}\)\}\{P\(B\_\{t\}\)\}\\mathbb\{E\}\[\\lVert\\delta\_\{t\}\\rVert\_\{2\}\|B\_\{t\-1\}\]≤𝔼​\[‖δt‖2\|Bt−1c∩At\]\+𝔼​\[‖δt‖2\|Bt−1\]\.\\displaystyle\\leq\\mathbb\{E\}\[\\\|\\delta\_\{t\}\\\|\_\{2\}\|B\_\{t\-1\}^\{c\}\\cap A\_\{t\}\]\+\\mathbb\{E\}\[\\\|\\delta\_\{t\}\\\|\_\{2\}\|B\_\{t\-1\}\]\.We now show the upper bounds for each term\. Conditioned onw1,⋯,wt−1w\_\{1\},\\cdots,w\_\{t\-1\}andw1′,⋯,wt−1′w\_\{1\}^\{\\prime\},\\cdots,w\_\{t\-1\}^\{\\prime\}, we have

‖δt‖2\\displaystyle\\\|\\delta\_\{t\}\\\|\_\{2\}=‖δt−1−\(𝖣𝖱​\(η​∇f​\(wt−1;zit\)\)−𝖣𝖱​\(η​∇f​\(wt−1′;zit′\)\)\)‖2\\displaystyle=\\\|\\delta\_\{t\-1\}\-\(\\mathsf\{DR\}\(\\eta\\nabla f\(w\_\{t\-1\};z\_\{i\_\{t\}\}\)\)\-\\mathsf\{DR\}\(\\eta\\nabla f\(w\_\{t\-1\}^\{\\prime\};z\_\{i\_\{t\}\}^\{\\prime\}\)\)\)\\\|\_\{2\}≤‖δt−1‖2\+‖𝖣𝖱​\(η​∇f​\(wt−1;zit\)\)‖2\+‖𝖣𝖱​\(η​∇f​\(wt−1′;zit′\)\)‖2\\displaystyle\\leq\\\|\\delta\_\{t\-1\}\\\|\_\{2\}\+\\\|\\mathsf\{DR\}\(\\eta\\nabla f\(w\_\{t\-1\};z\_\{i\_\{t\}\}\)\)\\\|\_\{2\}\+\\\|\\mathsf\{DR\}\(\\eta\\nabla f\(w\_\{t\-1\}^\{\\prime\};z\_\{i\_\{t\}\}^\{\\prime\}\)\)\\\|\_\{2\}≤‖δt−1‖2\+4​η​L\\displaystyle\\leq\\\|\\delta\_\{t\-1\}\\\|\_\{2\}\+4\\eta Lwhere the last inequality follows from[Section˜A\.1](https://arxiv.org/html/2606.06934#A1.SS1)andLL\-Lipschitzness offf\. Then, we have

𝔼​\[‖δt‖2\|Bt−1c∩At\]≤4​η​L,\\displaystyle\\mathbb\{E\}\[\\\|\\delta\_\{t\}\\\|\_\{2\}\|B\_\{t\-1\}^\{c\}\\cap A\_\{t\}\]\\leq 4\\eta L,𝔼​\[‖δt‖2\|Bt−1\]≤𝔼​\[‖δt−1‖2\|Bt−1\]\+4​η​L\.\\displaystyle\\mathbb\{E\}\[\\\|\\delta\_\{t\}\\\|\_\{2\}\|B\_\{t\-1\}\]\\leq\\mathbb\{E\}\[\\\|\\delta\_\{t\-1\}\\\|\_\{2\}\|B\_\{t\-1\}\]\+4\\eta L\.Then, we have

𝔼​\[‖δT‖2\|BT\]≤𝔼​\[‖δT−1‖2\|BT−1\]\+8​η​L≤⋯≤8​η​L​T\.\\displaystyle\\mathbb\{E\}\[\\\|\\delta\_\{T\}\\\|\_\{2\}\|B\_\{T\}\]\\leq\\mathbb\{E\}\[\\\|\\delta\_\{T\-1\}\\\|\_\{2\}\|B\_\{T\-1\}\]\+8\\eta L\\leq\\cdots\\leq 8\\eta LT\.Conclusively, we have

εu​s​\(Af,θ𝖣𝖱​\-SGD\)≤L​𝔼​\[‖δT‖2\]≤L​𝔼​\[‖δT‖2\|BT\]​P​\(BT\)≤min⁡\{1,Tn\}​8​η​L2​T\\displaystyle\\varepsilon\_\{us\}\(A^\{\\mathsf\{DR\}\\text\{\\rm\{\-SGD\}\}\}\_\{f,\\theta\}\)\\leq L\\mathbb\{E\}\[\\\|\\delta\_\{T\}\\\|\_\{2\}\]\\leq L\\mathbb\{E\}\[\\\|\\delta\_\{T\}\\\|\_\{2\}\|B\_\{T\}\]P\(B\_\{T\}\)\\leq\\min\\left\\\{1,\\frac\{T\}\{n\}\\right\\\}8\\eta L^\{2\}Twhich completes the proof\.

## Appendix BProofs for𝖲𝖱​\-GD\{\\mathsf\{SR\}\\text\{\\rm\{\-GD\}\}\}and𝖲𝖱​\-SGD\{\\mathsf\{SR\}\\text\{\\rm\{\-SGD\}\}\}

In this section, we prove generalization and stability bounds of𝖲𝖱​\-GD\{\\mathsf\{SR\}\\text\{\\rm\{\-GD\}\}\}and𝖲𝖱​\-SGD\{\\mathsf\{SR\}\\text\{\\rm\{\-SGD\}\}\}\. We first prove[Theorem˜9](https://arxiv.org/html/2606.06934#Thmtheorem9)in[Section˜B\.1](https://arxiv.org/html/2606.06934#A2.SS1)\. We then show the upper bounds of uniform argument stability for𝖲𝖱​\-SGD\{\\mathsf\{SR\}\\text\{\\rm\{\-SGD\}\}\}and𝖲𝖱​\-GD\{\\mathsf\{SR\}\\text\{\\rm\{\-GD\}\}\}\([Theorems˜10](https://arxiv.org/html/2606.06934#Thmtheorem10)and[11](https://arxiv.org/html/2606.06934#Thmtheorem11)\) in[Sections˜B\.2](https://arxiv.org/html/2606.06934#A2.SS2)and[B\.3](https://arxiv.org/html/2606.06934#A2.SS3), respectively\. We then present the proofs of uniform argument stability bounds for separable loss functions \([Theorems˜12](https://arxiv.org/html/2606.06934#Thmtheorem12)and[13](https://arxiv.org/html/2606.06934#Thmtheorem13)\) in[Sections˜B\.4](https://arxiv.org/html/2606.06934#A2.SS4)and[B\.5](https://arxiv.org/html/2606.06934#A2.SS5), respectively\.

### B\.1Proof of[Theorem˜9](https://arxiv.org/html/2606.06934#Thmtheorem9)

In this section, we prove[Theorem˜9](https://arxiv.org/html/2606.06934#Thmtheorem9)\. In this proof, we use𝒟=B​e​r​\(1/2\)\\mathcal\{D\}=Ber\(1/2\)as a data distribution andf:ℝd×\{0,1\}→ℝf:\\mathbb\{R\}^\{d\}\\times\\\{0,1\\\}\\to\\mathbb\{R\}as a loss function defined as follows:f​\(w;0\)=0f\(w;0\)=0andf​\(w;1\)=g​\(w\)f\(w;1\)=g\(w\)whereg:ℝd→ℝg:\\mathbb\{R\}^\{d\}\\to\\mathbb\{R\}is obtained from the following lemma\.

###### Lemma 21\.

Letd∈ℕd\\in\\mathbb\{N\},L,Δ\>0L,\\Delta\>0\. Then, there existsα\>0\\alpha\>0andγ\>0\\gamma\>0such thatα\>\(e2−1\)​γ\\alpha\>\(e^\{2\}\-1\)\\gamma, and a functiong:ℝd→ℝg:\\mathbb\{R\}^\{d\}\\to\\mathbb\{R\}such thatggis convex,LL\-Lipschitz,4​L/Δ4L/\\Delta\-smooth, and satisfies the following:

∇g​\(𝟎d\)=−Ld​𝟏d,g​\(c\)=L​Δ​\(α​kc\+γ−γ2\),∀c∈\{0,Δ\}d\\displaystyle\\nabla g\(\\boldsymbol\{0\}\_\{d\}\)=\-\\frac\{L\}\{\\sqrt\{d\}\}\\mathbf\{1\}\_\{d\},\\quad g\(c\)=L\\Delta\\left\(\\sqrt\{\\alpha k\_\{c\}\+\\gamma\}\-\\frac\{\\sqrt\{\\gamma\}\}\{2\}\\right\),\\quad\\forall c\\in\\\{0,\\Delta\\\}^\{d\}wherekck\_\{c\}denotes the number of non\-zero entries incc, i\.e\.,kc=‖c‖1/Δk\_\{c\}=\\\|c\\\|\_\{1\}/\\Delta\.

[Section˜B\.1](https://arxiv.org/html/2606.06934#A2.SS1)implies the existence of a functionggsuch that∇g​\(𝟎d\)\\nabla g\(\\boldsymbol\{0\}\_\{d\}\)is proportional to the all\-ones vector, andg​\(c\)g\(c\)scales linearly with‖c‖2\\\|c\\\|\_\{2\}for eachc∈\{0,Δ\}dc\\in\\\{0,\\Delta\\\}^\{d\}; we note that‖c‖2=Δ​kc\\\|c\\\|\_\{2\}=\\Delta\\sqrt\{k\_\{c\}\}\. The proof of[Section˜B\.1](https://arxiv.org/html/2606.06934#A2.SS1)is in[Section˜B\.1\.1](https://arxiv.org/html/2606.06934#A2.SS1.SSS1)\.

We now prove the lower bound of the generalization error of𝖲𝖱\\mathsf\{SR\}\-GD and𝖲𝖱\\mathsf\{SR\}\-SGD using our construction of𝒟\\mathcal\{D\}andff\. From[Section˜2\.4](https://arxiv.org/html/2606.06934#S2.SS4), the generalization ofA∈\{Af,θ𝖲𝖱​\-GD,Af,θ𝖲𝖱​\-SGD\}A\\in\\\{A\_\{f,\\theta\}^\{\\mathsf\{SR\}\\text\{\\rm\{\-GD\}\}\},A\_\{f,\\theta\}^\{\\mathsf\{SR\}\\text\{\\rm\{\-SGD\}\}\}\\\}can be written as

εg​\(A;𝒟\)=\|𝔼A,S′,z1​\[f​\(A​\(S\);z1\)−f​\(A​\(S′\);z1\)\]\|\\displaystyle\\varepsilon\_\{g\}\(A;\\mathcal\{D\}\)=\|\\mathbb\{E\}\_\{A,S^\{\\prime\},z\_\{1\}\}\[f\(A\(S\);z\_\{1\}\)\-f\(A\(S^\{\\prime\}\);z\_\{1\}\)\]\|whereS=\(z1,⋯,zn\)∈𝒟nS=\(z\_\{1\},\\cdots,z\_\{n\}\)\\in\\mathcal\{D\}^\{n\}andS′=\(z1′,z2,⋯,zn\)S^\{\\prime\}=\(z\_\{1\}^\{\\prime\},z\_\{2\},\\cdots,z\_\{n\}\)wherez1′,z1,⋯,zn∼𝒟z\_\{1\}^\{\\prime\},z\_\{1\},\\cdots,z\_\{n\}\\sim\\mathcal\{D\}are independent\. We note that ifz1=z1′z\_\{1\}=z\_\{1\}^\{\\prime\}orz1=0z\_\{1\}=0, thenf​\(A​\(S\);z1\)−f​\(A​\(S′\);z1\)=0f\(A\(S\);z\_\{1\}\)\-f\(A\(S^\{\\prime\}\);z\_\{1\}\)=0\. Thus, only the event\(z1,z1′\)=\(1,0\)\(z\_\{1\},z\_\{1\}^\{\\prime\}\)=\(1,0\)contributes to this expectation\. Hence, we can rewrite the generalization error by

εg​\(A;𝒟\)=14​\|𝔼A,S​\[f​\(A​\(S1\);1\)−f​\(A​\(S0\);1\)\]\|=14​\|𝔼A,S​\[g​\(A​\(S1\)\)−g​\(A​\(S0\)\)\]\|\\displaystyle\\varepsilon\_\{g\}\(A;\\mathcal\{D\}\)=\\frac\{1\}\{4\}\|\\mathbb\{E\}\_\{A,S\}\[f\(A\(S\_\{1\}\);1\)\-f\(A\(S\_\{0\}\);1\)\]\|=\\frac\{1\}\{4\}\|\\mathbb\{E\}\_\{A,S\}\[g\(A\(S\_\{1\}\)\)\-g\(A\(S\_\{0\}\)\)\]\|\(8\)whereS1=\(1,z2,⋯,zn\)S\_\{1\}=\(1,z\_\{2\},\\cdots,z\_\{n\}\)andS0=\(0,z2,⋯,zn\)S\_\{0\}=\(0,z\_\{2\},\\cdots,z\_\{n\}\)\.

We show the lower bound of[Eq\.˜8](https://arxiv.org/html/2606.06934#A2.E8)for𝖲𝖱\\mathsf\{SR\}\-GD and𝖲𝖱\\mathsf\{SR\}\-SGD by applying the following lemmas\.

###### Lemma 22\.

Letη,L,Δ\>0\\eta,L,\\Delta\>0,n,d∈ℕn,d\\in\\mathbb\{N\}withn≥14n\\geq 14,η​L<Δ​d\\eta L<\\Delta\\sqrt\{d\}\. LetS1=\(1,z2,⋯,zn\)S\_\{1\}=\(1,z\_\{2\},\\cdots,z\_\{n\}\)andS0=\(0,z2,⋯,zn\)S\_\{0\}=\(0,z\_\{2\},\\cdots,z\_\{n\}\)wherez2,⋯,zn∼B​e​r​\(1/2\)z\_\{2\},\\cdots,z\_\{n\}\\sim Ber\(1/2\),f:ℝd×\{0,1\}→ℝf:\\mathbb\{R\}^\{d\}\\times\\\{0,1\\\}\\to\\mathbb\{R\}be a loss function defined byf​\(x;0\)=0f\(x;0\)=0andf​\(x;1\)=g​\(x\)f\(x;1\)=g\(x\)whereggis obtained from[Section˜B\.1](https://arxiv.org/html/2606.06934#A2.SS1)\. Then, forθ=\(η,T=1,w0=𝟎d\)\\theta=\(\\eta,T=1,w\_\{0\}=\\boldsymbol\{0\}\_\{d\}\), it holds that

\|𝔼​\[g​\(Af,θ𝖲𝖱​\-GD​\(S1\)\)−g​\(Af,θ𝖲𝖱​\-GD​\(S0\)\)\]\|=Ω​\(min⁡\{Δ,η​L​d1/2\}​η​L3​d1/2n\)\.\\displaystyle\|\\mathbb\{E\}\[g\(A\_\{f,\\theta\}^\{\{\\mathsf\{SR\}\\text\{\\rm\{\-GD\}\}\}\}\(S\_\{1\}\)\)\-g\(A\_\{f,\\theta\}^\{\{\\mathsf\{SR\}\\text\{\\rm\{\-GD\}\}\}\}\(S\_\{0\}\)\)\]\|=\\Omega\\left\(\\frac\{\\sqrt\{\\min\\\{\\Delta,\\eta Ld^\{1/2\}\\\}\\eta L^\{3\}d^\{1/2\}\}\}\{n\}\\right\)\.

###### Lemma 23\.

Letη,L,Δ\>0\\eta,L,\\Delta\>0,n,d∈ℕn,d\\in\\mathbb\{N\}withη​L<Δ​d\\eta L<\\Delta\\sqrt\{d\}\. LetS1=\(1,z2,⋯,zn\)S\_\{1\}=\(1,z\_\{2\},\\cdots,z\_\{n\}\)andS0=\(0,z2,⋯,zn\)S\_\{0\}=\(0,z\_\{2\},\\cdots,z\_\{n\}\)wherez2,⋯,zn∼B​e​r​\(1/2\)z\_\{2\},\\cdots,z\_\{n\}\\sim Ber\(1/2\)are independent,f:ℝd×\{0,1\}→ℝf:\\mathbb\{R\}^\{d\}\\times\\\{0,1\\\}\\to\\mathbb\{R\}be a loss function defined byf​\(x;0\)=0f\(x;0\)=0andf​\(x;1\)=g​\(x\)f\(x;1\)=g\(x\)whereggis obtained from[Section˜B\.1](https://arxiv.org/html/2606.06934#A2.SS1)\. Forθ=\(η,T=1,w0=𝟎d\)\\theta=\(\\eta,T=1,w\_\{0\}=\\boldsymbol\{0\}\_\{d\}\), it holds that

\|𝔼​\[g​\(Af,θ𝖲𝖱​\-SGD​\(S1\)\)−g​\(Af,θ𝖲𝖱​\-SGD​\(S0\)\)\]\|=Ω​\(min⁡\{Δ,η​L​d1/2\}​η​L3​d1/2n\)\.\\displaystyle\|\\mathbb\{E\}\[g\(A\_\{f,\\theta\}^\{\{\\mathsf\{SR\}\\text\{\\rm\{\-SGD\}\}\}\}\(S\_\{1\}\)\)\-g\(A\_\{f,\\theta\}^\{\{\\mathsf\{SR\}\\text\{\\rm\{\-SGD\}\}\}\}\(S\_\{0\}\)\)\]\|=\\Omega\\left\(\\frac\{\\sqrt\{\\min\\\{\\Delta,\\eta Ld^\{1/2\}\\\}\\eta L^\{3\}d^\{1/2\}\}\}\{n\}\\right\)\.

The proof of[Section˜B\.1](https://arxiv.org/html/2606.06934#A2.SS1)is in[Section˜B\.1\.2](https://arxiv.org/html/2606.06934#A2.SS1.SSS2), and the proof of[Section˜B\.1](https://arxiv.org/html/2606.06934#A2.SS1)is in[Section˜B\.1\.3](https://arxiv.org/html/2606.06934#A2.SS1.SSS3)\.[Sections˜B\.1](https://arxiv.org/html/2606.06934#A2.SS1)and[B\.1](https://arxiv.org/html/2606.06934#A2.SS1)give us explicit lower bounds on the generalization errors of𝖲𝖱\\mathsf\{SR\}\-GD and𝖲𝖱\\mathsf\{SR\}\-SGD, respectively\. This completes the proof\.

#### B\.1\.1Proof of[Section˜B\.1](https://arxiv.org/html/2606.06934#A2.SS1)

We construct a Huber function \(Huber \[[1964](https://arxiv.org/html/2606.06934#bib.bib24)\]\)g:ℝd→ℝg:\\mathbb\{R\}^\{d\}\\to\\mathbb\{R\}defined by

g​\(x\)=\{2​LΔ​‖x−Δ4​d​𝟏d‖22x∈ℬΔ/4​\(Δ4​d​𝟏d\),L​‖x−Δ4​d​𝟏d‖2−L​Δ8o\.w\.\\displaystyle g\(x\)=\\begin\{cases\}\\frac\{2L\}\{\\Delta\}\\left\\lVert x\-\\frac\{\\Delta\}\{4\\sqrt\{d\}\}\\mathbf\{1\}\_\{d\}\\right\\rVert\_\{2\}^\{2\}&x\\in\\mathcal\{B\}\_\{\\Delta/4\}\\left\(\\frac\{\\Delta\}\{4\\sqrt\{d\}\}\\mathbf\{1\}\_\{d\}\\right\),\\\\ L\\left\\lVert x\-\\frac\{\\Delta\}\{4\\sqrt\{d\}\}\\mathbf\{1\}\_\{d\}\\right\\rVert\_\{2\}\-\\frac\{L\\Delta\}\{8\}&o\.w\.\\end\{cases\}Intuitively,ggconsists of a quadratic part within a ball of radiusΔ/4\\Delta/4centered at\(Δ/\(4​d\)\)​𝟏d\(\\Delta/\(4\\sqrt\{d\}\)\)\\boldsymbol\{1\}\_\{d\}, and a linear part outside this ball\. It is easy to observe thatggis convex,LL\-Lipschitz, and4​L/Δ4L/\\Delta\-smooth\. Furthermore, we have

∇g​\(𝟎d\)=−Ld​𝟏d,g​\(c\)=L​Δ​\(\(1−12​d\)​kc\+116−18\),∀c∈\{0,Δ\}d\\displaystyle\\nabla g\(\\boldsymbol\{0\}\_\{d\}\)=\-\\frac\{L\}\{\\sqrt\{d\}\}\\mathbf\{1\}\_\{d\},\\quad g\(c\)=L\\Delta\\left\(\\sqrt\{\\left\(1\-\\frac\{1\}\{2\\sqrt\{d\}\}\\right\)k\_\{c\}\+\\frac\{1\}\{16\}\}\-\\frac\{1\}\{8\}\\right\),\\quad\\forall c\\in\\\{0,\\Delta\\\}^\{d\}wherekck\_\{c\}denotes the number of non\-zero entries incc\. We note that if we chooseα=1−1/\(2​d\)\\alpha=1\-1/\(2\\sqrt\{d\}\)andγ=1/16\\gamma=1/16, then ourggsatisfies the conditions for the gradient at the origin and the values atc∈\{0,Δ\}dc\\in\\\{0,\\Delta\\\}^\{d\}given in[Section˜B\.1](https://arxiv.org/html/2606.06934#A2.SS1)\. Furthermore,α\>\(e2−1\)​γ\\alpha\>\(e^\{2\}\-1\)\\gammaholds since

1−12​d≥12\>e2−116\\displaystyle 1\-\\frac\{1\}\{2\\sqrt\{d\}\}\\geq\\frac\{1\}\{2\}\>\\frac\{e^\{2\}\-1\}\{16\}for alld∈ℕd\\in\\mathbb\{N\}\. This completes the proof\.

#### B\.1\.2Proof of[Section˜B\.1](https://arxiv.org/html/2606.06934#A2.SS1)

In this section, we prove[Section˜B\.1](https://arxiv.org/html/2606.06934#A2.SS1)\. Letw=Af,θ𝖲𝖱​\-GD​\(S1\)w=A\_\{f,\\theta\}^\{\{\\mathsf\{SR\}\\text\{\\rm\{\-GD\}\}\}\}\(S\_\{1\}\),w′=Af,θ𝖲𝖱​\-GD​\(S0\)w^\{\\prime\}=A\_\{f,\\theta\}^\{\{\\mathsf\{SR\}\\text\{\\rm\{\-GD\}\}\}\}\(S\_\{0\}\),kkbe the number of ones inS0S\_\{0\}, andKKbe the event thatk∈\[\(n−1\)/3,2​\(n−1\)/3\]k\\in\[\(n\-1\)/3,2\(n\-1\)/3\]\. Then, we have

𝔼​\[g​\(w\)−g​\(w′\)\]≥P​\(K\)​𝔼​\[g​\(w\)−g​\(w′\)\|K\]\.\\displaystyle\\mathbb\{E\}\[g\(w\)\-g\(w^\{\\prime\}\)\]\\geq P\(K\)\\mathbb\{E\}\[g\(w\)\-g\(w^\{\\prime\}\)\|K\]\.
Here, we note that under the coordinate\-wise monotone coupling, each coordinate ofwwis greater than or equal to the corresponding coordinate ofw′w^\{\\prime\}\. Since the value ofg​\(c\)g\(c\)in[Section˜B\.1](https://arxiv.org/html/2606.06934#A2.SS1)is increasing in the number of non\-zero entries incc, we haveg​\(w\)−g​\(w′\)≥0g\(w\)\-g\(w^\{\\prime\}\)\\geq 0almost surely\. Hence,𝔼​\[g​\(w\)−g​\(w′\)\|Kc\]≥0\\mathbb\{E\}\[g\(w\)\-g\(w^\{\\prime\}\)\|K^\{c\}\]\\geq 0and the inequality above holds\.

We first show thatP​\(K\)=Ω​\(1\)P\(K\)=\\Omega\(1\)\. By Hoeffding’s inequality, it holds that

P​\(K\)=ℙ​\(\|k−n−12\|≤n−16\)≥1−2​exp⁡\(−n−118\)\\displaystyle P\(K\)=\\mathbb\{P\}\\left\(\\left\|k\-\\frac\{n\-1\}\{2\}\\right\|\\leq\\frac\{n\-1\}\{6\}\\right\)\\geq 1\-2\\exp\\left\(\-\\frac\{n\-1\}\{18\}\\right\)Here, since1−2​exp⁡\(−\(n−1\)/18\)\>0⇔n≥1\+18​ln⁡2≈13\.481\-2\\exp\(\-\(n\-1\)/18\)\>0\\iff n\\geq 1\+18\\ln 2\\approx 13\.48, and1−2​exp⁡\(−\(n−1\)/18\)1\-2\\exp\(\-\(n\-1\)/18\)is an increasing function ofnn, we haveP​\(K\)=Ω​\(1\)P\(K\)=\\Omega\(1\)for alln≥14n\\geq 14\.

Since two parameters are initialized at the origin,∇f​\(𝟎d;0\)=0\\nabla f\(\\boldsymbol\{0\}\_\{d\};0\)=0, and∇f​\(𝟎d;1\)=−\(L/d\)​𝟏d\\nabla f\(\\boldsymbol\{0\}\_\{d\};1\)=\-\(L/\\sqrt\{d\}\)\\mathbf\{1\}\_\{d\}, we have

w=−𝖲𝖱​\(η​∇f​\(𝟎d;S1\)\)=𝖲𝖱​\(η​L​\(k\+1\)n​d​𝟏d\),w′=−𝖲𝖱​\(η​∇f​\(𝟎d;S0\)\)=𝖲𝖱​\(η​L​kn​d​𝟏d\)\.\\displaystyle w=\-\\mathsf\{SR\}\\left\(\\eta\\nabla f\(\\boldsymbol\{0\}\_\{d\};S\_\{1\}\)\\right\)=\\mathsf\{SR\}\\left\(\\frac\{\\eta L\(k\+1\)\}\{n\\sqrt\{d\}\}\\mathbf\{1\}\_\{d\}\\right\),~~~w^\{\\prime\}=\-\\mathsf\{SR\}\(\\eta\\nabla f\(\\boldsymbol\{0\}\_\{d\};S\_\{0\}\)\)=\\mathsf\{SR\}\\left\(\\frac\{\\eta Lk\}\{n\\sqrt\{d\}\}\\mathbf\{1\}\_\{d\}\\right\)\.Sinceη​L<Δ​d\\eta L<\\Delta\\sqrt\{d\}, it holds thatw,w′∈\{0,Δ\}dw,w^\{\\prime\}\\in\\\{0,\\Delta\\\}^\{d\}\. LetXXandYYbe random variables that take the number of non\-zero entries ofwwandw′w^\{\\prime\}, respectively\. Then, it is easy to observe thatX∼B​i​n​\(d,η​L​\(k\+1\)n​Δ​d\),Y∼B​i​n​\(d,η​L​kn​Δ​d\)X\\sim Bin\\left\(d,\\frac\{\\eta L\(k\+1\)\}\{n\\Delta\\sqrt\{d\}\}\\right\),Y\\sim Bin\\left\(d,\\frac\{\\eta Lk\}\{n\\Delta\\sqrt\{d\}\}\\right\)\. Sinceggis obtained from[Section˜B\.1](https://arxiv.org/html/2606.06934#A2.SS1), we have

\|𝔼​\[g​\(w\)−g​\(w′\)\]\|≳Δ​L​𝔼​\[α​X\+γ−α​Y\+γ\|K\]\\displaystyle\|\\mathbb\{E\}\[g\(w\)\-g\(w^\{\\prime\}\)\]\|\\gtrsim\\Delta L\\mathbb\{E\}\\left\[\\sqrt\{\\alpha X\+\\gamma\}\-\\sqrt\{\\alpha Y\+\\gamma\}\|K\\right\]\(9\)for someα\>0\\alpha\>0andγ\>0\\gamma\>0\. To show the lower bound of[Eq\.˜9](https://arxiv.org/html/2606.06934#A2.E9), we introduce the following technical lemma\. The proof of[Section˜B\.1\.2](https://arxiv.org/html/2606.06934#A2.SS1.SSS2)is in[Section˜B\.1\.4](https://arxiv.org/html/2606.06934#A2.SS1.SSS4)\.

###### Lemma 24\.

Leta,b\>0a,b\>0be universal constants such thata\>b​\(e2−1\)a\>b\(e^\{2\}\-1\),n∈ℕn\\in\\mathbb\{N\},p,p′∈\(0,1\)p,p^\{\\prime\}\\in\(0,1\)such thatp\>p′p\>p^\{\\prime\},X∼B​i​n​\(n,p\)X\\sim Bin\(n,p\)andY∼B​i​n​\(n,p′\)Y\\sim Bin\(n,p^\{\\prime\}\)\. Then

𝔼​\[a​X\+b−a​Y\+b\]=Ω​\(\(1−p′p\)​min⁡\{𝔼​\[X\],𝔼​\[X\]\}\)\.\\displaystyle\\mathbb\{E\}\[\\sqrt\{aX\+b\}\-\\sqrt\{aY\+b\}\]=\\Omega\\left\(\\left\(1\-\\frac\{p^\{\\prime\}\}\{p\}\\right\)\\min\\left\\\{\\sqrt\{\\mathbb\{E\}\[X\]\},\\mathbb\{E\}\[X\]\\right\\\}\\right\)\.

SinceX∼B​i​n​\(d,η​L​\(k\+1\)n​Δ​d\),Y∼B​i​n​\(d,η​L​kn​Δ​d\)X\\sim Bin\\left\(d,\\frac\{\\eta L\(k\+1\)\}\{n\\Delta\\sqrt\{d\}\}\\right\),Y\\sim Bin\\left\(d,\\frac\{\\eta Lk\}\{n\\Delta\\sqrt\{d\}\}\\right\), by[Section˜B\.1\.2](https://arxiv.org/html/2606.06934#A2.SS1.SSS2), we have

𝔼​\[α​X\+γ−α​Y\+γ\|K\]\\displaystyle\\mathbb\{E\}\\left\[\\sqrt\{\\alpha X\+\\gamma\}\-\\sqrt\{\\alpha Y\+\\gamma\}\|K\\right\]≳\(1−η​L​kn​Δ​dη​L​\(k\+1\)n​Δ​d\)​min⁡\{η​L​\(k\+1\)​d1/2n​Δ,η​L​\(k\+1\)​dn​Δ\}\\displaystyle\\gtrsim\\left\(1\-\\frac\{\\frac\{\\eta Lk\}\{n\\Delta\\sqrt\{d\}\}\}\{\\frac\{\\eta L\(k\+1\)\}\{n\\Delta\\sqrt\{d\}\}\}\\right\)\\min\\left\\\{\\sqrt\{\\frac\{\\eta L\(k\+1\)d^\{1/2\}\}\{n\\Delta\}\},\\frac\{\\eta L\(k\+1\)\\sqrt\{d\}\}\{n\\Delta\}\\right\\\}≳\(1−kk\+1\)​min⁡\{η​L​d1/2Δ,η​L​dΔ\}\\displaystyle\\gtrsim\\left\(1\-\\frac\{k\}\{k\+1\}\\right\)\\min\\left\\\{\\sqrt\{\\frac\{\\eta Ld^\{1/2\}\}\{\\Delta\}\},\\frac\{\\eta L\\sqrt\{d\}\}\{\\Delta\}\\right\\\}≳1n​min⁡\{η​L​d1/2Δ,η​L​dΔ\}\\displaystyle\\gtrsim\\frac\{1\}\{n\}\\min\\left\\\{\\sqrt\{\\frac\{\\eta Ld^\{1/2\}\}\{\\Delta\}\},\\frac\{\\eta L\\sqrt\{d\}\}\{\\Delta\}\\right\\\}where the second and third inequalities follow underKK:k=Θ​\(n\)k=\\Theta\(n\)\.

By combining this result with[Eq\.˜9](https://arxiv.org/html/2606.06934#A2.E9), we have

\|𝔼​\[g​\(w\)−g​\(w′\)\]\|≳Δ​Ln​min⁡\{η​L​d1/2Δ,η​L​dΔ\}=min⁡\{Δ,η​L​d1/2\}​η​L3​d1/2n\\displaystyle\|\\mathbb\{E\}\[g\(w\)\-g\(w^\{\\prime\}\)\]\|\\gtrsim\\frac\{\\Delta L\}\{n\}\\min\\left\\\{\\sqrt\{\\frac\{\\eta Ld^\{1/2\}\}\{\\Delta\}\},\\frac\{\\eta L\\sqrt\{d\}\}\{\\Delta\}\\right\\\}=\\frac\{\\sqrt\{\\min\\\{\\Delta,\\eta Ld^\{1/2\}\\\}\\eta L^\{3\}d^\{1/2\}\}\}\{n\}which completes the proof\.

#### B\.1\.3Proof of[Section˜B\.1](https://arxiv.org/html/2606.06934#A2.SS1)

In this section, we prove[Section˜B\.1](https://arxiv.org/html/2606.06934#A2.SS1)\. Letv=Af,θ𝖲𝖱​\-SGD​\(S1\)v=A\_\{f,\\theta\}^\{\{\\mathsf\{SR\}\\text\{\\rm\{\-SGD\}\}\}\}\(S\_\{1\}\),v′=Af,θ𝖲𝖱​\-SGD​\(S0\)v^\{\\prime\}=A\_\{f,\\theta\}^\{\{\\mathsf\{SR\}\\text\{\\rm\{\-SGD\}\}\}\}\(S\_\{0\}\), andkkbe the number of11inS0S\_\{0\}\. Since two parameters are initialized at𝟎d\\boldsymbol\{0\}\_\{d\}and∇g​\(𝟎d\)=−\(L/d\)​𝟏d\\nabla g\(\\boldsymbol\{0\}\_\{d\}\)=\-\(L/\\sqrt\{d\}\)\\mathbf\{1\}\_\{d\}, we have

𝔼​\[g​\(v\)\]\\displaystyle\\mathbb\{E\}\[g\(v\)\]=𝔼​\[\(1−k\+1n\)​g​\(𝟎d\)\+k\+1n​g​\(𝖲𝖱​\(η​Ld​𝟏d\)\)\],and\\displaystyle=\\mathbb\{E\}\\left\[\\left\(1\-\\frac\{k\+1\}\{n\}\\right\)g\(\\boldsymbol\{0\}\_\{d\}\)\+\\frac\{k\+1\}\{n\}g\\left\(\\mathsf\{SR\}\\left\(\\frac\{\\eta L\}\{\\sqrt\{d\}\}\\mathbf\{1\}\_\{d\}\\right\)\\right\)\\right\],\\text\{ and\}𝔼​\[g​\(v′\)\]\\displaystyle\\mathbb\{E\}\[g\(v^\{\\prime\}\)\]=𝔼​\[\(1−kn\)​g​\(𝟎d\)\+kn​g​\(𝖲𝖱​\(η​Ld​𝟏d\)\)\]\.\\displaystyle=\\mathbb\{E\}\\left\[\\left\(1\-\\frac\{k\}\{n\}\\right\)g\(\\boldsymbol\{0\}\_\{d\}\)\+\\frac\{k\}\{n\}g\\left\(\\mathsf\{SR\}\\left\(\\frac\{\\eta L\}\{\\sqrt\{d\}\}\\mathbf\{1\}\_\{d\}\\right\)\\right\)\\right\]\.LetXXbe the number of non\-zero entries in𝖲𝖱​\(\(η​L/d\)​𝟏d\)\\mathsf\{SR\}\(\(\\eta L/\\sqrt\{d\}\)\\boldsymbol\{1\}\_\{d\}\)\. Then, sinceη​L/d<Δ\\eta L/\\sqrt\{d\}<\\Delta, we haveX∼B​i​n​\(d,η​L/\(d​Δ\)\)X\\sim Bin\(d,\\eta L/\(\\sqrt\{d\}\\Delta\)\)\. Sinceggis obtained from[Section˜B\.1](https://arxiv.org/html/2606.06934#A2.SS1), we have

𝔼​\[g​\(v\)−g​\(v′\)\]\\displaystyle\\mathbb\{E\}\[g\(v\)\-g\(v^\{\\prime\}\)\]=1n​𝔼​\[g​\(𝖲𝖱​\(η​Ld​𝟏d\)\)−g​\(𝟎d\)\]\\displaystyle=\\frac\{1\}\{n\}\\mathbb\{E\}\\left\[g\\left\(\\mathsf\{SR\}\\left\(\\frac\{\\eta L\}\{\\sqrt\{d\}\}\\mathbf\{1\}\_\{d\}\\right\)\\right\)\-g\(\\boldsymbol\{0\}\_\{d\}\)\\right\]=L​Δn​𝔼​\[α​X\+γ−γ\]\\displaystyle=\\frac\{L\\Delta\}\{n\}\\mathbb\{E\}\[\\sqrt\{\\alpha X\+\\gamma\}\-\\sqrt\{\\gamma\}\]for someα\>0\\alpha\>0andγ\>0\\gamma\>0\. We now introduce the following lemma to compute the expectation\. The proof of[Section˜B\.1\.3](https://arxiv.org/html/2606.06934#A2.SS1.SSS3)is in[Section˜B\.1\.5](https://arxiv.org/html/2606.06934#A2.SS1.SSS5)\.

###### Lemma 25\.

Leta\>0a\>0andb≥0b\\geq 0be universal constants such thata\>b​\(e2−1\)a\>b\(e^\{2\}\-1\),n∈ℕn\\in\\mathbb\{N\},p∈\(0,1\)p\\in\(0,1\), andX∼B​i​n​\(n,p\)X\\sim Bin\(n,p\)\. Then it holds that

𝔼​\[a​X\+b\]−b=Ω​\(min⁡\{𝔼​\[X\],𝔼​\[X\]\}\)\.\\displaystyle\\mathbb\{E\}\[\\sqrt\{aX\+b\}\]\-\\sqrt\{b\}=\\Omega\\left\(\\min\\left\\\{\\sqrt\{\\mathbb\{E\}\[X\]\},\\mathbb\{E\}\[X\]\\right\\\}\\right\)\.

Then, by[Section˜B\.1\.3](https://arxiv.org/html/2606.06934#A2.SS1.SSS3), we have

𝔼​\[α​X\+γ−γ\]=Ω​\(min⁡\{η​L​d1/2Δ,η​L​dΔ\}\)\.\\displaystyle\\mathbb\{E\}\[\\sqrt\{\\alpha X\+\\gamma\}\-\\sqrt\{\\gamma\}\]=\\Omega\\left\(\\min\\left\\\{\\sqrt\{\\frac\{\\eta Ld^\{1/2\}\}\{\\Delta\}\},\\frac\{\\eta L\\sqrt\{d\}\}\{\\Delta\}\\right\\\}\\right\)\.Conclusively, we have

𝔼​\[g​\(v\)−g​\(v′\)\]\\displaystyle\\mathbb\{E\}\[g\(v\)\-g\(v^\{\\prime\}\)\]=L​Δn​𝔼​\[α​X\+γ−γ\]=Ω​\(min⁡\{Δ,η​L​d1/2\}​η​L3​d1/2n\)\\displaystyle=\\frac\{L\\Delta\}\{n\}\\mathbb\{E\}\[\\sqrt\{\\alpha X\+\\gamma\}\-\\sqrt\{\\gamma\}\]=\\Omega\\left\(\\frac\{\\sqrt\{\\min\\\{\\Delta,\\eta Ld^\{1/2\}\\\}\\eta L^\{3\}d^\{1/2\}\}\}\{n\}\\right\)which completes the proof\.

#### B\.1\.4Proof of[Section˜B\.1\.2](https://arxiv.org/html/2606.06934#A2.SS1.SSS2)

In this section, we prove[Section˜B\.1\.2](https://arxiv.org/html/2606.06934#A2.SS1.SSS2)\. Sincep\>p′p\>p^\{\\prime\}, we can apply a monotone coupling such thatX≥YX\\geq Yalmost surely\.

We first compute𝔼​\[Y\|X\]\\mathbb\{E\}\[Y\|X\]\. LetX1,⋯,Xn∼B​e​r​\(p\)X\_\{1\},\\cdots,X\_\{n\}\\sim Ber\(p\)andY1,⋯,Yn∼B​e​r​\(p′\)Y\_\{1\},\\cdots,Y\_\{n\}\\sim Ber\(p^\{\\prime\}\)be two sets of i\.i\.d\. random variables such thatX=∑i=1nXiX=\\sum\_\{i=1\}^\{n\}X\_\{i\}andY=∑i=1nYiY=\\sum\_\{i=1\}^\{n\}Y\_\{i\}\. SinceXXandYYare monotonically coupled, we have

ℙ​\(Yi=1\|Xi=0\)=0,ℙ​\(Yi=1\|Xi=1\)=p′p\.\\displaystyle\\mathbb\{P\}\(Y\_\{i\}=1\|X\_\{i\}=0\)=0,\\quad\\mathbb\{P\}\(Y\_\{i\}=1\|X\_\{i\}=1\)=\\frac\{p^\{\\prime\}\}\{p\}\.Thus,Y\|X=∑i=1nYi\|Xi∼B​i​n​\(X,p′/p\)Y\|X=\\sum\_\{i=1\}^\{n\}Y\_\{i\}\|X\_\{i\}\\sim Bin\(X,p^\{\\prime\}/p\)and𝔼​\[Y\|X\]=\(p′/p\)​X\\mathbb\{E\}\[Y\|X\]=\(p^\{\\prime\}/p\)X\.

We now show the lower bound of𝔼​\[a​X\+b−a​Y\+b\]\\mathbb\{E\}\[\\sqrt\{aX\+b\}\-\\sqrt\{aY\+b\}\]\.

𝔼​\[a​X\+b−a​Y\+b\]\\displaystyle\\mathbb\{E\}\[\\sqrt\{aX\+b\}\-\\sqrt\{aY\+b\}\]=𝔼​\[a​\(X−Y\)a​X\+b\+a​Y\+b\]\\displaystyle=\\mathbb\{E\}\\left\[\\frac\{a\(X\-Y\)\}\{\\sqrt\{aX\+b\}\+\\sqrt\{aY\+b\}\}\\right\]≥𝔼​\[a​\(X−Y\)2​a​X\+b\]\\displaystyle\\geq\\mathbb\{E\}\\left\[\\frac\{a\(X\-Y\)\}\{2\\sqrt\{aX\+b\}\}\\right\]=𝔼​\[𝔼​\[a​\(X−Y\)2​a​X\+b\|X\]\]\\displaystyle=\\mathbb\{E\}\\left\[\\mathbb\{E\}\\left\[\\frac\{a\(X\-Y\)\}\{2\\sqrt\{aX\+b\}\}\\bigg\|X\\right\]\\right\]=𝔼​\[a​\(X−𝔼​\[Y\|X\]\)2​a​X\+b\]\\displaystyle=\\mathbb\{E\}\\left\[\\frac\{a\(X\-\\mathbb\{E\}\[Y\|X\]\)\}\{2\\sqrt\{aX\+b\}\}\\right\]=a2​\(1−p′p\)​𝔼​\[Xa​X\+b\]\.\\displaystyle=\\frac\{a\}\{2\}\\left\(1\-\\frac\{p^\{\\prime\}\}\{p\}\\right\)\\mathbb\{E\}\\left\[\\frac\{X\}\{\\sqrt\{aX\+b\}\}\\right\]\.It suffices to show that𝔼​\[X/a​X\+b\]=Ω​\(min⁡\{𝔼​\[X\],𝔼​\[X\]\}\)\\mathbb\{E\}\\left\[X/\\sqrt\{aX\+b\}\\right\]=\\Omega\(\\min\\\{\\sqrt\{\\mathbb\{E\}\[X\]\},\\mathbb\{E\}\[X\]\\\}\)\.

𝔼​\[Xa​X\+b\]\\displaystyle\\mathbb\{E\}\\left\[\\frac\{X\}\{\\sqrt\{aX\+b\}\}\\right\]=ℙ​\(X≥1\)​𝔼​\[Xa​X\+b\|X≥1\]\\displaystyle=\\mathbb\{P\}\(X\\geq 1\)\\mathbb\{E\}\\left\[\\frac\{X\}\{\\sqrt\{aX\+b\}\}\\bigg\|X\\geq 1\\right\]≥ℙ​\(X≥1\)​𝔼​\[X\(a\+b\)​X\|X≥1\]\\displaystyle\\geq\\mathbb\{P\}\(X\\geq 1\)\\mathbb\{E\}\\left\[\\frac\{X\}\{\\sqrt\{\(a\+b\)X\}\}\\bigg\|X\\geq 1\\right\]≥1a\+b​ℙ​\(X≥1\)​𝔼​\[X\|X≥1\]\\displaystyle\\geq\\frac\{1\}\{\\sqrt\{a\+b\}\}\\mathbb\{P\}\(X\\geq 1\)\\mathbb\{E\}\[\\sqrt\{X\}\|X\\geq 1\]=1a\+b​𝔼​\[X\]=Ω​\(min⁡\{𝔼​\[X\],𝔼​\[X\]\}\)\\displaystyle=\\frac\{1\}\{\\sqrt\{a\+b\}\}\\mathbb\{E\}\[\\sqrt\{X\}\]=\\Omega\\left\(\\min\\left\\\{\\sqrt\{\\mathbb\{E\}\[X\]\},\\mathbb\{E\}\[X\]\\right\\\}\\right\)where the last equality follows from[Section˜B\.1\.3](https://arxiv.org/html/2606.06934#A2.SS1.SSS3)\. Conclusively, we have

𝔼​\[a​X\+b−a​Y\+b\]\\displaystyle\\mathbb\{E\}\[\\sqrt\{aX\+b\}\-\\sqrt\{aY\+b\}\]=Ω​\(\(1−p′p\)​𝔼​\[Xa​X\+b\]\)\\displaystyle=\\Omega\\left\(\\left\(1\-\\frac\{p^\{\\prime\}\}\{p\}\\right\)\\mathbb\{E\}\\left\[\\frac\{X\}\{\\sqrt\{aX\+b\}\}\\right\]\\right\)=Ω​\(\(1−p′p\)​min⁡\{𝔼​\[X\],𝔼​\[X\]\}\)\\displaystyle=\\Omega\\left\(\\left\(1\-\\frac\{p^\{\\prime\}\}\{p\}\\right\)\\min\\left\\\{\\sqrt\{\\mathbb\{E\}\[X\]\},\\mathbb\{E\}\[X\]\\right\\\}\\right\)which completes the proof\.

#### B\.1\.5Proof of[Section˜B\.1\.3](https://arxiv.org/html/2606.06934#A2.SS1.SSS3)

In this section, we prove[Section˜B\.1\.3](https://arxiv.org/html/2606.06934#A2.SS1.SSS3)\. Here, we assumen≥2n\\geq 2; the case wheren=1n=1follows directly sinceX∼B​e​r​\(p\)X\\sim Ber\(p\)and𝔼​\[a​X\+b\]−b=p​\(a\+b−b\)=Ω​\(p\)\\mathbb\{E\}\[\\sqrt\{aX\+b\}\]\-\\sqrt\{b\}=p\(\\sqrt\{a\+b\}\-\\sqrt\{b\}\)=\\Omega\(p\)\. To this end, we prove the following two statements: \(1\) if𝔼​\[X\]≥1\\mathbb\{E\}\[X\]\\geq 1, then𝔼​\[a​X\+b−b\]=Ω​\(𝔼​\[X\]\)\\mathbb\{E\}\[\\sqrt\{aX\+b\}\-\\sqrt\{b\}\]=\\Omega\(\\sqrt\{\\mathbb\{E\}\[X\]\}\), and \(2\) if𝔼​\[X\]<1\\mathbb\{E\}\[X\]<1, then𝔼​\[a​X\+b−b\]=Ω​\(𝔼​\[X\]\)\\mathbb\{E\}\[\\sqrt\{aX\+b\}\-\\sqrt\{b\}\]=\\Omega\(\\mathbb\{E\}\[X\]\)\. Then, we have𝔼​\[a​X\+b−b\]=Ω​\(min⁡\{𝔼​\[X\],𝔼​\[X\]\}\)\\mathbb\{E\}\[\\sqrt\{aX\+b\}\-\\sqrt\{b\}\]=\\Omega\(\\min\\\{\\sqrt\{\\mathbb\{E\}\[X\]\},\\mathbb\{E\}\[X\]\\\}\)\.

We first show that the statement \(1\): if𝔼​\[X\]≥1\\mathbb\{E\}\[X\]\\geq 1, then𝔼​\[a​X\+b−b\]=Ω​\(𝔼​\[X\]\)\\mathbb\{E\}\[\\sqrt\{aX\+b\}\-\\sqrt\{b\}\]=\\Omega\(\\sqrt\{\\mathbb\{E\}\[X\]\}\)\. Sincea​X\+b−b=0\\sqrt\{aX\+b\}\-\\sqrt\{b\}=0ifX=0X=0, we have

𝔼​\[a​X\+b\]−b\\displaystyle\\mathbb\{E\}\[\\sqrt\{aX\+b\}\]\-\\sqrt\{b\}=ℙ​\(X≥1\)​𝔼​\[a​X\+b−b\|X≥1\]\.\\displaystyle=\\mathbb\{P\}\(X\\geq 1\)\\mathbb\{E\}\[\\sqrt\{aX\+b\}\-\\sqrt\{b\}\|X\\geq 1\]\.Since𝔼​\[X\]≥1\\mathbb\{E\}\[X\]\\geq 1, we haveℙ​\(X≥1\)=Ω​\(1\)\\mathbb\{P\}\(X\\geq 1\)=\\Omega\(1\);ℙ​\(X≥1\)=1−\(1−p\)n≥1−e−n​p≥1−e−1\\mathbb\{P\}\(X\\geq 1\)=1\-\(1\-p\)^\{n\}\\geq 1\-e^\{\-np\}\\geq 1\-e^\{\-1\}\. We now lower bound the expectation term\. GivenX≥1X\\geq 1, we have

a​X\+b−b\\displaystyle\\sqrt\{aX\+b\}\-\\sqrt\{b\}=a​Xa​X\+b\+b\\displaystyle=\\frac\{aX\}\{\\sqrt\{aX\+b\}\+\\sqrt\{b\}\}≥a​XX​\(a\+b\+b\)\\displaystyle\\geq\\frac\{aX\}\{\\sqrt\{X\}\(\\sqrt\{a\+b\}\+\\sqrt\{b\}\)\}=aa\+b\+b​X\.\\displaystyle=\\frac\{a\}\{\\sqrt\{a\+b\}\+\\sqrt\{b\}\}\\sqrt\{X\}\.
Thus, we have

ℙ​\(X≥1\)​𝔼​\[a​X\+b−b\|X≥1\]\\displaystyle\\mathbb\{P\}\(X\\geq 1\)\\mathbb\{E\}\[\\sqrt\{aX\+b\}\-\\sqrt\{b\}\|X\\geq 1\]≥ℙ​\(X≥1\)​aa\+b\+b​𝔼​\[X\|X≥1\]=Ω​\(𝔼​\[X\]\)\.\\displaystyle\\geq\\mathbb\{P\}\(X\\geq 1\)\\frac\{a\}\{\\sqrt\{a\+b\}\+\\sqrt\{b\}\}\\mathbb\{E\}\\left\[\\sqrt\{X\}\|X\\geq 1\\right\]=\\Omega\(\\mathbb\{E\}\[\\sqrt\{X\}\]\)\.
We now show that𝔼​\[X\]=Ω​\(n​p\)=Ω​\(𝔼​\[X\]\)\\mathbb\{E\}\[\\sqrt\{X\}\]=\\Omega\(\\sqrt\{np\}\)=\\Omega\(\\sqrt\{\\mathbb\{E\}\[X\]\}\)\. We note that for anyδ\>0\\delta\>0, it holds that

𝔼​\[X\]\\displaystyle\\mathbb\{E\}\\left\[\\sqrt\{X\}\\right\]≥δ​ℙ​\(X≥δ\)=δ​ℙ​\(X≥δ2\)\.\\displaystyle\\geq\\delta\\mathbb\{P\}\\left\(\\sqrt\{X\}\\geq\\delta\\right\)=\\delta\\mathbb\{P\}\\left\(X\\geq\\delta^\{2\}\\right\)\.\(∗\\ast\)By choosingδ=n​p/2\\delta=\\sqrt\{np/2\}, we have

\(∗\)\\displaystyle\(\\ast\)=n​p2​ℙ​\(X≥n​p2\)≥14​n​p2​𝔼​\[X\]2𝔼​\[X2\]=14​n​p2​n​pn​p\+1−p≥18​2​n​p\\displaystyle=\\sqrt\{\\frac\{np\}\{2\}\}\\mathbb\{P\}\\left\(X\\geq\\frac\{np\}\{2\}\\right\)\\geq\\frac\{1\}\{4\}\\sqrt\{\\frac\{np\}\{2\}\}\\frac\{\\mathbb\{E\}\[X\]^\{2\}\}\{\\mathbb\{E\}\[X^\{2\}\]\}=\\frac\{1\}\{4\}\\sqrt\{\\frac\{np\}\{2\}\}\\frac\{np\}\{np\+1\-p\}\\geq\\frac\{1\}\{8\\sqrt\{2\}\}\\sqrt\{np\}where the second inequality follows from the Paley\-Zygmund inequality, and the last inequality follows from the fact thatn​p≥1np\\geq 1andp<1p<1\.

We now prove the second statement: if𝔼​\[X\]<1\\mathbb\{E\}\[X\]<1, then𝔼​\[a​X\+b−b\]=Ω​\(𝔼​\[X\]\)\\mathbb\{E\}\[\\sqrt\{aX\+b\}\-\\sqrt\{b\}\]=\\Omega\(\\mathbb\{E\}\[X\]\)\. To this end, we consider the lower bound of𝔼​\[a​X\+b\]\\mathbb\{E\}\[\\sqrt\{aX\+b\}\]:

𝔼​\[a​X\+b\]\\displaystyle\\mathbb\{E\}\[\\sqrt\{aX\+b\}\]≥b​ℙ​\(X=0\)\+a\+b​ℙ​\(X=1\)\\displaystyle\\geq\\sqrt\{b\}\\mathbb\{P\}\(X=0\)\+\\sqrt\{a\+b\}\\mathbb\{P\}\(X=1\)=b​\(1−p\)n\+a\+b​n​p​\(1−p\)n−1\\displaystyle=\\sqrt\{b\}\\left\(1\-p\\right\)^\{n\}\+\\sqrt\{a\+b\}np\(1\-p\)^\{n\-1\}≥b​\(1−n​p\)\+a\+b​n​p​\(1−p\)n−1\\displaystyle\\geq\\sqrt\{b\}\(1\-np\)\+\\sqrt\{a\+b\}np\(1\-p\)^\{n\-1\}=b\+n​p​\(a\+b​\(1−p\)n−1−b\)\\displaystyle=\\sqrt\{b\}\+np\(\\sqrt\{a\+b\}\(1\-p\)^\{n\-1\}\-\\sqrt\{b\}\)≥b\+n​p​\(a\+be−b\)\\displaystyle\\geq\\sqrt\{b\}\+np\\left\(\\frac\{\\sqrt\{a\+b\}\}\{e\}\-\\sqrt\{b\}\\right\)where the inequality in the third line follows from Bernoulli’s inequality, and the last inequality follows from the fact that𝔼​\[X\]=n​p<1\\mathbb\{E\}\[X\]=np<1, which implies that

\(1−p\)n−1\>\(1−1n\)n−1≥1e\\displaystyle\(1\-p\)^\{n\-1\}\>\\left\(1\-\\frac\{1\}\{n\}\\right\)^\{n\-1\}\\geq\\frac\{1\}\{e\}forn≥2n\\geq 2\. Here, ifb=0b=0, then we have

𝔼​\[a​X\]≥n​p​ae=Ω​\(𝔼​\[X\]\)\.\\displaystyle\\mathbb\{E\}\[\\sqrt\{aX\}\]\\geq\\frac\{np\\sqrt\{a\}\}\{e\}=\\Omega\(\\mathbb\{E\}\[X\]\)\.
Suppose thatb\>0b\>0\. Sincea\>b​\(e2−1\)a\>b\(e^\{2\}\-1\), we have

a\+be−b=be​\(1\+ab−e\)\>be​\(1\+e2−1−e\)=0\\displaystyle\\frac\{\\sqrt\{a\+b\}\}\{e\}\-\\sqrt\{b\}=\\frac\{\\sqrt\{b\}\}\{e\}\\left\(\\sqrt\{1\+\\frac\{a\}\{b\}\}\-e\\right\)\>\\frac\{\\sqrt\{b\}\}\{e\}\\left\(\\sqrt\{1\+e^\{2\}\-1\}\-e\\right\)=0which implies

𝔼​\[a​X\+b\]−b=n​p​\(a\+be−b\)=Ω​\(𝔼​\[X\]\)\.\\displaystyle\\mathbb\{E\}\[\\sqrt\{aX\+b\}\]\-\\sqrt\{b\}=np\\left\(\\frac\{\\sqrt\{a\+b\}\}\{e\}\-\\sqrt\{b\}\\right\)=\\Omega\(\\mathbb\{E\}\[X\]\)\.It completes the proof\.

### B\.2Proof of[Theorem˜10](https://arxiv.org/html/2606.06934#Thmtheorem10)

In this section, we prove[Theorem˜10](https://arxiv.org/html/2606.06934#Thmtheorem10)\. LetS=\(z1,⋯,zn\)S=\(z\_\{1\},\\cdots,z\_\{n\}\)andS′=\(z1′,⋯,zn′\)S^\{\\prime\}=\(z\_\{1\}^\{\\prime\},\\cdots,z\_\{n\}^\{\\prime\}\)be neighboring datasets\. Let\(wt\)t=1T\(w\_\{t\}\)\_\{t=1\}^\{T\}and\(wt′\)t=1T\(w\_\{t\}^\{\\prime\}\)\_\{t=1\}^\{T\}be the trajectories fromAf,θ𝖲𝖱​\-SGD​\(S\)A\_\{f,\\theta\}^\{\\mathsf\{SR\}\\text\{\\rm\{\-SGD\}\}\}\(S\)andAf,θ𝖲𝖱​\-SGD​\(S′\)A\_\{f,\\theta\}^\{\\mathsf\{SR\}\\text\{\\rm\{\-SGD\}\}\}\(S^\{\\prime\}\), respectively\. We note that the stochastic rounding operations in two𝖲𝖱​\-SGD\{\\mathsf\{SR\}\\text\{\\rm\{\-SGD\}\}\}parameters are monotonically coupled: for each iterationt∈\[T\]t\\in\[T\]and coordinatej∈\[d\]j\\in\[d\], they use the same uniform random variableut,ju\_\{t,j\}in[Eq\.˜3](https://arxiv.org/html/2606.06934#S2.E3)\. For eacht∈\[T\]t\\in\[T\], letδt=wt−wt′\\delta\_\{t\}=w\_\{t\}\-w\_\{t\}^\{\\prime\}andBtB\_\{t\}be the event that there existsτ≤t\\tau\\leq tsuch thatziτ≠ziτ′z\_\{i\_\{\\tau\}\}\\neq z\_\{i\_\{\\tau\}\}^\{\\prime\}andAtA\_\{t\}be the event thatzit≠zit′z\_\{i\_\{t\}\}\\neq z\_\{i\_\{t\}\}^\{\\prime\}\. SincewT=wT′w\_\{T\}=w\_\{T\}^\{\\prime\}conditioned onBTcB\_\{T\}^\{c\}, we have𝔼​\[‖δT‖\]=P​\(BT\)​𝔼​\[‖δT‖\|BT\]\\mathbb\{E\}\[\\\|\\delta\_\{T\}\\\|\]=P\(B\_\{T\}\)\\mathbb\{E\}\[\\\|\\delta\_\{T\}\\\|\|B\_\{T\}\]\.

Here, we have

P​\(BT\)=1−\(1−1n\)T≤min⁡\{1,Tn\}\.\\displaystyle P\(B\_\{T\}\)=1\-\\left\(1\-\\frac\{1\}\{n\}\\right\)^\{T\}\\leq\\min\\left\\\{1,\\frac\{T\}\{n\}\\right\\\}\.\(10\)In the rest of the proof, we show the upper bound on𝔼​\[‖δT‖2\|BT\]\\mathbb\{E\}\[\\\|\\delta\_\{T\}\\\|\_\{2\}\|B\_\{T\}\]\. To this end, we construct a recurrence with respect to𝔼​\[‖δt‖22\|Bt\]\\mathbb\{E\}\[\\\|\\delta\_\{t\}\\\|\_\{2\}^\{2\}\|B\_\{t\}\]and then reduce it to the upper bound of𝔼​\[‖δT‖2\|BT\]\\mathbb\{E\}\[\\\|\\delta\_\{T\}\\\|\_\{2\}\|B\_\{T\}\]\.

We first consider the upper bound of𝔼​\[‖δ1‖22\|B1\]\\mathbb\{E\}\[\\\|\\delta\_\{1\}\\\|\_\{2\}^\{2\}\|B\_\{1\}\]:

𝔼​\[‖δ1‖22\|B1\]=𝔼​\[‖δ1‖22\|A1\]=𝔼​\[‖𝖲𝖱​\(η​∇f​\(w0;zi1\)\)−𝖲𝖱​\(η​∇f​\(w0′;zi1′\)\)‖2\|zi1≠zi1′\]\.\\displaystyle\\mathbb\{E\}\[\\\|\\delta\_\{1\}\\\|\_\{2\}^\{2\}\|B\_\{1\}\]=\\mathbb\{E\}\[\\\|\\delta\_\{1\}\\\|\_\{2\}^\{2\}\|A\_\{1\}\]=\\mathbb\{E\}\[\\\|\\mathsf\{SR\}\(\\eta\\nabla f\(w\_\{0\};z\_\{i\_\{1\}\}\)\)\-\\mathsf\{SR\}\(\\eta\\nabla f\(w\_\{0\}^\{\\prime\};z\_\{i\_\{1\}\}^\{\\prime\}\)\)\\\|\_\{2\}\|z\_\{i\_\{1\}\}\\neq z\_\{i\_\{1\}\}^\{\\prime\}\]\.To upper bound this expectation, we introduce the following lemma\. The proof of[Section˜B\.2](https://arxiv.org/html/2606.06934#A2.SS2)is in[Section˜B\.2\.1](https://arxiv.org/html/2606.06934#A2.SS2.SSS1)\.

###### Lemma 26\.

LetΔ\>0\\Delta\>0andx,y,p,q∈ℝdx,y,p,q\\in\\mathbb\{R\}^\{d\}\. Then, under the coordinate\-wise monotone coupling, it holds that

𝔼​\[‖x−𝖲𝖱​\(p\)−\(y−𝖲𝖱​\(q\)\)‖22\]≤‖x−p−\(y−q\)‖22\+Δ​‖p−q‖1\.\\displaystyle\\mathbb\{E\}\[\\\|x\-\\mathsf\{SR\}\(p\)\-\(y\-\\mathsf\{SR\}\(q\)\)\\\|\_\{2\}^\{2\}\]\\leq\\\|x\-p\-\(y\-q\)\\\|\_\{2\}^\{2\}\+\\Delta\\\|p\-q\\\|\_\{1\}\.

SinceffisLL\-Lipschitz, we have‖η​∇f​\(wt;zit\)−η​∇f​\(wt′;zit′\)‖2≤2​η​L\\\|\\eta\\nabla f\(w\_\{t\};z\_\{i\_\{t\}\}\)\-\\eta\\nabla f\(w\_\{t\}^\{\\prime\};z\_\{i\_\{t\}\}^\{\\prime\}\)\\\|\_\{2\}\\leq 2\\eta Land‖η​∇f​\(wt;zit\)−η​∇f​\(wt′;zit′\)‖1≤2​η​L​d\\\|\\eta\\nabla f\(w\_\{t\};z\_\{i\_\{t\}\}\)\-\\eta\\nabla f\(w\_\{t\}^\{\\prime\};z\_\{i\_\{t\}\}^\{\\prime\}\)\\\|\_\{1\}\\leq 2\\eta L\\sqrt\{d\}\. Therefore, by[Section˜B\.2](https://arxiv.org/html/2606.06934#A2.SS2), we have

𝔼​\[‖δ1‖22\|B1\]≤\(2​η​L\)2\+2​Δ​η​L​d\.\\displaystyle\\mathbb\{E\}\[\\\|\\delta\_\{1\}\\\|\_\{2\}^\{2\}\|B\_\{1\}\]\\leq\\left\(2\\eta L\\right\)^\{2\}\+2\\Delta\\eta L\\sqrt\{d\}\.\(11\)
We now consider the generalt=2,⋯,Tt=2,\\cdots,T\. Here,BtB\_\{t\}can be considered as the union of two mutually exclusive events,Bt−1c∩AtB\_\{t\-1\}^\{c\}\\cap A\_\{t\}andBt−1B\_\{t\-1\}\. Then, we can decompose𝔼​\[‖δt‖22\|Bt\]\\mathbb\{E\}\[\\\|\\delta\_\{t\}\\\|\_\{2\}^\{2\}\|B\_\{t\}\]as follows:

𝔼​\[∥δt∥22\|Bt\]\\displaystyle\\mathbb\{E\}\[\\lVert\\delta\_\{t\}\\rVert\_\{2\}^\{2\}\|B\_\{t\}\]=P​\(Bt−1c∩At\)P​\(Bt\)​𝔼​\[∥δt∥22\|Bt−1c∩At\]\+P​\(Bt−1\)P​\(Bt\)​𝔼​\[∥δt∥22\|Bt−1\]\\displaystyle=\\frac\{P\(B\_\{t\-1\}^\{c\}\\cap A\_\{t\}\)\}\{P\(B\_\{t\}\)\}\\mathbb\{E\}\[\\lVert\\delta\_\{t\}\\rVert\_\{2\}^\{2\}\|B\_\{t\-1\}^\{c\}\\cap A\_\{t\}\]\+\\frac\{P\(B\_\{t\-1\}\)\}\{P\(B\_\{t\}\)\}\\mathbb\{E\}\[\\lVert\\delta\_\{t\}\\rVert\_\{2\}^\{2\}\|B\_\{t\-1\}\]≤max⁡\{𝔼​\[∥δt∥22\|Bt−1c∩At\],𝔼​\[∥δt∥22\|Bt−1\]\}\.\\displaystyle\\leq\\max\\\{\\mathbb\{E\}\[\\lVert\\delta\_\{t\}\\rVert\_\{2\}^\{2\}\|B\_\{t\-1\}^\{c\}\\cap A\_\{t\}\],\\mathbb\{E\}\[\\lVert\\delta\_\{t\}\\rVert\_\{2\}^\{2\}\|B\_\{t\-1\}\]\\\}\.We now show the upper bounds of𝔼​\[‖δt‖22\|Bt−1c∩At\]\\mathbb\{E\}\[\\\|\\delta\_\{t\}\\\|\_\{2\}^\{2\}\|B\_\{t\-1\}^\{c\}\\cap A\_\{t\}\]and𝔼​\[‖δt‖22\|Bt−1\]\\mathbb\{E\}\[\\\|\\delta\_\{t\}\\\|\_\{2\}^\{2\}\|B\_\{t\-1\}\], respectively\.

Conditioned onw1,⋯,wt−1w\_\{1\},\\cdots,w\_\{t\-1\}andw1′,⋯,wt−1′w\_\{1\}^\{\\prime\},\\cdots,w\_\{t\-1\}^\{\\prime\}, by[Section˜B\.2](https://arxiv.org/html/2606.06934#A2.SS2), we have

𝔼​\[‖δt‖22\]≤‖δt−1−η​\(∇f​\(wt−1;zit\)−∇f​\(wt−1′;zit′\)\)‖22\+2​Δ​η​L​d\.\\displaystyle\\mathbb\{E\}\[\\\|\\delta\_\{t\}\\\|\_\{2\}^\{2\}\]\\leq\\\|\\delta\_\{t\-1\}\-\\eta\(\\nabla f\(w\_\{t\-1\};z\_\{i\_\{t\}\}\)\-\\nabla f\(w\_\{t\-1\}^\{\\prime\};z\_\{i\_\{t\}\}^\{\\prime\}\)\)\\\|\_\{2\}^\{2\}\+2\\Delta\\eta L\\sqrt\{d\}\.We note that the first term of the right\-hand side is equal to the squared norm of twoSGDupdates\. Thus, one can apply the non\-expansiveness ofSGD\. See the following lemma\.

###### Lemma 27\(Lemma 3\.7 in\[Hardtet al\.,[2016](https://arxiv.org/html/2606.06934#bib.bib4)\]\)\.

Letd∈ℕd\\in\\mathbb\{N\},η,M\>0\\eta,M\>0withη<2/M\\eta<2/M,f:ℝd→ℝf:\\mathbb\{R\}^\{d\}\\to\\mathbb\{R\}be a convex,MM\-smooth function\. Then, for allu,v∈ℝdu,v\\in\\mathbb\{R\}^\{d\}, it holds that

‖u−η​∇f​\(u\)−\(v−η​∇f​\(v\)\)‖2≤‖u−v‖2\.\\displaystyle\\\|u\-\\eta\\nabla f\(u\)\-\(v\-\\eta\\nabla f\(v\)\)\\\|\_\{2\}\\leq\\\|u\-v\\\|\_\{2\}\.

Ifzit=zit′z\_\{i\_\{t\}\}=z\_\{i\_\{t\}\}^\{\\prime\}, sinceη<2/M\\eta<2/M, we have‖δt−1−η​\(∇f​\(wt−1;zit\)−∇f​\(wt−1′;zit′\)\)‖22≤‖δt−1‖22\\\|\\delta\_\{t\-1\}\-\\eta\(\\nabla f\(w\_\{t\-1\};z\_\{i\_\{t\}\}\)\-\\nabla f\(w\_\{t\-1\}^\{\\prime\};z\_\{i\_\{t\}\}^\{\\prime\}\)\)\\\|\_\{2\}^\{2\}\\leq\\\|\\delta\_\{t\-1\}\\\|\_\{2\}^\{2\}by[Section˜B\.2](https://arxiv.org/html/2606.06934#A2.SS2)\. Ifzit≠zit′z\_\{i\_\{t\}\}\\neq z\_\{i\_\{t\}\}^\{\\prime\}, by the triangle inequality, we have

‖δt−1−η​\(∇f​\(wt−1;zit\)−∇f​\(wt−1′;zit′\)\)‖22\\displaystyle\\\|\\delta\_\{t\-1\}\-\\eta\(\\nabla f\(w\_\{t\-1\};z\_\{i\_\{t\}\}\)\-\\nabla f\(w\_\{t\-1\}^\{\\prime\};z\_\{i\_\{t\}\}^\{\\prime\}\)\)\\\|\_\{2\}^\{2\}≤\(‖δt−1‖2\+η​‖∇f​\(wt−1;zit\)−∇f​\(wt−1′;zit′\)‖2\)2\\displaystyle\\leq\(\\\|\\delta\_\{t\-1\}\\\|\_\{2\}\+\\eta\\\|\\nabla f\(w\_\{t\-1\};z\_\{i\_\{t\}\}\)\-\\nabla f\(w\_\{t\-1\}^\{\\prime\};z\_\{i\_\{t\}\}^\{\\prime\}\)\\\|\_\{2\}\)^\{2\}≤\(‖δt−1‖2\+2​η​L\)2\\displaystyle\\leq\(\\\|\\delta\_\{t\-1\}\\\|\_\{2\}\+2\\eta L\)^\{2\}≤‖δt−1‖22\+4​η​L​‖δt−1‖2\+4​η2​L2\.\\displaystyle\\leq\\\|\\delta\_\{t\-1\}\\\|\_\{2\}^\{2\}\+4\\eta L\\\|\\delta\_\{t\-1\}\\\|\_\{2\}\+4\\eta^\{2\}L^\{2\}\.
Conclusively, we have

𝔼​\[‖δt‖22\]≤\{‖δt−1‖22\+2​Δ​η​L​dzit=zit′,‖δt−1‖22\+4​η​L​‖δt−1‖2\+4​η2​L2\+2​Δ​η​L​dzit≠zit′\.\\displaystyle\\mathbb\{E\}\[\\\|\\delta\_\{t\}\\\|\_\{2\}^\{2\}\]\\leq\\begin\{cases\}\\\|\\delta\_\{t\-1\}\\\|\_\{2\}^\{2\}\+2\\Delta\\eta L\\sqrt\{d\}&z\_\{i\_\{t\}\}=z\_\{i\_\{t\}\}^\{\\prime\},\\\\ \\\|\\delta\_\{t\-1\}\\\|\_\{2\}^\{2\}\+4\\eta L\\\|\\delta\_\{t\-1\}\\\|\_\{2\}\+4\\eta^\{2\}L^\{2\}\+2\\Delta\\eta L\\sqrt\{d\}&z\_\{i\_\{t\}\}\\neq z\_\{i\_\{t\}\}^\{\\prime\}\.\\end\{cases\}By conditioning on the eventsBt−1c∩AtB\_\{t\-1\}^\{c\}\\cap A\_\{t\}andBt−1B\_\{t\-1\}, we have

𝔼​\[‖δt‖22\|Bt−1c∩At\]≤4​η2​L2\+2​Δ​η​L​d,\\displaystyle\\mathbb\{E\}\[\\\|\\delta\_\{t\}\\\|\_\{2\}^\{2\}\|B\_\{t\-1\}^\{c\}\\cap A\_\{t\}\]\\leq 4\\eta^\{2\}L^\{2\}\+2\\Delta\\eta L\\sqrt\{d\},𝔼​\[‖δt‖22\|Bt−1\]≤𝔼​\[‖δt−1‖22\|Bt−1\]\+4​η​Ln​𝔼​\[‖δt−1‖2\|Bt−1\]\+4​η2​L2n\+2​Δ​η​L​d\.\\displaystyle\\mathbb\{E\}\[\\\|\\delta\_\{t\}\\\|\_\{2\}^\{2\}\|B\_\{t\-1\}\]\\leq\\mathbb\{E\}\[\\\|\\delta\_\{t\-1\}\\\|\_\{2\}^\{2\}\|B\_\{t\-1\}\]\+\\frac\{4\\eta L\}\{n\}\\mathbb\{E\}\[\\\|\\delta\_\{t\-1\}\\\|\_\{2\}\|B\_\{t\-1\}\]\+\\frac\{4\\eta^\{2\}L^\{2\}\}\{n\}\+2\\Delta\\eta L\\sqrt\{d\}\.Thus, we have

𝔼​\[‖δt‖22\|Bt\]≤max⁡\{𝔼​\[‖δt−1‖22\|Bt−1\]\+4​η​Ln​𝔼​\[‖δt−1‖2\|Bt−1\]\+γ,ω\}\\displaystyle\\mathbb\{E\}\[\\\|\\delta\_\{t\}\\\|\_\{2\}^\{2\}\|B\_\{t\}\]\\leq\\max\\left\\\{\\mathbb\{E\}\[\\\|\\delta\_\{t\-1\}\\\|\_\{2\}^\{2\}\|B\_\{t\-1\}\]\+\\frac\{4\\eta L\}\{n\}\\mathbb\{E\}\[\\\|\\delta\_\{t\-1\}\\\|\_\{2\}\|B\_\{t\-1\}\]\+\\gamma,\\omega\\right\\\}whereγ=4​η2​L2/n\+2​Δ​η​L​d\\gamma=4\\eta^\{2\}L^\{2\}/n\+2\\Delta\\eta L\\sqrt\{d\}andω=4​η2​L2\+2​Δ​η​L​d\\omega=4\\eta^\{2\}L^\{2\}\+2\\Delta\\eta L\\sqrt\{d\}\.

We now solve this recurrence using the following lemma\. The proof of[Section˜B\.2](https://arxiv.org/html/2606.06934#A2.SS2)is in[Section˜B\.2\.2](https://arxiv.org/html/2606.06934#A2.SS2.SSS2)\.

###### Lemma 28\.

LetT∈ℕT\\in\\mathbb\{N\},ω,ξ≥0\\omega,\\xi\\geq 0such thatξ≥ω\\xi\\geq\\omega, andα,γ\>0\\alpha,\\gamma\>0\. Suppose that the non\-negative random variablesX1,⋯,XTX\_\{1\},\\cdots,X\_\{T\}satisfy that𝔼​\[X12\]≤ξ\\mathbb\{E\}\[X\_\{1\}^\{2\}\]\\leq\\xi, and𝔼​\[Xt2\]≤max⁡\{𝔼​\[Xt−12\]\+α​𝔼​\[Xt−1\]\+γ,ω\}\\mathbb\{E\}\[X\_\{t\}^\{2\}\]\\leq\\max\\\{\\mathbb\{E\}\[X\_\{t\-1\}^\{2\}\]\+\\alpha\\mathbb\{E\}\[X\_\{t\-1\}\]\+\\gamma,\\omega\\\}for allt∈\{2,⋯,T\}t\\in\\\{2,\\cdots,T\\\}\. Then it holds that

𝔼​\[Xt\]≤α​\(t−1\)\+γ​\(t−1\)\+ξ\\displaystyle\\mathbb\{E\}\[X\_\{t\}\]\\leq\\alpha\(t\-1\)\+\\sqrt\{\\gamma\(t\-1\)\+\\xi\}for allt∈\[T\]t\\in\[T\]\.

Recall[Eq\.˜11](https://arxiv.org/html/2606.06934#A2.E11)for the base case:𝔼​\[‖δ1‖22\|B1\]≤4​η2​L2\+2​Δ​η​L​d\\mathbb\{E\}\[\\\|\\delta\_\{1\}\\\|\_\{2\}^\{2\}\|B\_\{1\}\]\\leq 4\\eta^\{2\}L^\{2\}\+2\\Delta\\eta L\\sqrt\{d\}\. Then, by applying[Section˜B\.2](https://arxiv.org/html/2606.06934#A2.SS2)with the choices ofα=4​η​L/n\\alpha=4\\eta L/n,γ=4​η2​L2/n\+2​Δ​η​L​d\\gamma=4\\eta^\{2\}L^\{2\}/n\+2\\Delta\\eta L\\sqrt\{d\}, andξ=4​η2​L2\+2​Δ​η​L​d\\xi=4\\eta^\{2\}L^\{2\}\+2\\Delta\\eta L\\sqrt\{d\}, we have

𝔼​\[‖δT‖2\|BT\]\\displaystyle\\mathbb\{E\}\[\\\|\\delta\_\{T\}\\\|\_\{2\}\|B\_\{T\}\]≤4​η​L​\(T−1\)n\+γ​\(T−1\)\+ω\\displaystyle\\leq\\frac\{4\\eta L\(T\-1\)\}\{n\}\+\\sqrt\{\\gamma\(T\-1\)\+\\omega\}=4​η​L​\(T−1\)n\+4​η2​L2\+4​η2​L2​\(T−1\)n\+2​Δ​η​L​d1/2​T\.\\displaystyle=\\frac\{4\\eta L\(T\-1\)\}\{n\}\+\\sqrt\{4\\eta^\{2\}L^\{2\}\+\\frac\{4\\eta^\{2\}L^\{2\}\(T\-1\)\}\{n\}\+2\\Delta\\eta Ld^\{1/2\}T\}\.By combining this result with[Eq\.˜10](https://arxiv.org/html/2606.06934#A2.E10), we have

𝔼​\[‖δT‖2\]\\displaystyle\\mathbb\{E\}\[\\\|\\delta\_\{T\}\\\|\_\{2\}\]≤min⁡\{1,Tn\}​\(4​η​L​\(T−1\)n\+4​η2​L2\+4​η2​L2​\(T−1\)n\+2​Δ​η​L​d1/2​T\)\\displaystyle\\leq\\min\\left\\\{1,\\frac\{T\}\{n\}\\right\\\}\\left\(\\frac\{4\\eta L\(T\-1\)\}\{n\}\+\\sqrt\{4\\eta^\{2\}L^\{2\}\+\\frac\{4\\eta^\{2\}L^\{2\}\(T\-1\)\}\{n\}\+2\\Delta\\eta Ld^\{1/2\}T\}\\right\)≤4​η​L​\(T−1\)n\+min⁡\{1,Tn\}​\(2​η​L​\(1\+T−1n\)\+2​Δ​η​L​d1/2​T\)\.\\displaystyle\\leq\\frac\{4\\eta L\(T\-1\)\}\{n\}\+\\min\\left\\\{1,\\frac\{T\}\{n\}\\right\\\}\\left\(2\\eta L\\left\(1\+\\sqrt\{\\frac\{T\-1\}\{n\}\}\\right\)\+\\sqrt\{2\\Delta\\eta Ld^\{1/2\}T\}\\right\)\.
We note that

min⁡\{1,Tn\}​\(1\+T−1n\)≤2​Tn\.\\displaystyle\\min\\left\\\{1,\\frac\{T\}\{n\}\\right\\\}\\left\(1\+\\sqrt\{\\frac\{T\-1\}\{n\}\}\\right\)\\leq\\frac\{2T\}\{n\}\.Indeed, ifT≤nT\\leq n, then\(T−1\)/n<1\\sqrt\{\(T\-1\)/n\}<1, yielding the above inequality, and ifT\>nT\>n, then1\+\(T−1\)/n<T/n\+T/n=2​T/n1\+\\sqrt\{\(T\-1\)/n\}<T/n\+T/n=2T/n\. Then, we obtain

εu​a​s​\(Af,θ𝖲𝖱​\-SGD\)≤8​η​L​Tn\+min⁡\{1,Tn\}​2​Δ​η​L​d1/2​T\.\\displaystyle\\varepsilon\_\{uas\}\(A\_\{f,\\theta\}^\{\\mathsf\{SR\}\\text\{\\rm\{\-SGD\}\}\}\)\\leq\\frac\{8\\eta LT\}\{n\}\+\\min\\left\\\{1,\\frac\{T\}\{n\}\\right\\\}\\sqrt\{2\\Delta\\eta Ld^\{1/2\}T\}\.which completes the proof\.

#### B\.2\.1Proof of[Section˜B\.2](https://arxiv.org/html/2606.06934#A2.SS2)

In this section, we prove[Section˜B\.2](https://arxiv.org/html/2606.06934#A2.SS2)\. We now expand the left side of the given expression:

‖x−y‖22−2​𝔼​\[⟨x−y,𝖲𝖱​\(p\)−𝖲𝖱​\(q\)⟩\]\+𝔼​\[‖𝖲𝖱​\(p\)−𝖲𝖱​\(q\)‖22\]\.\\displaystyle\\\|x\-y\\\|\_\{2\}^\{2\}\-2\\mathbb\{E\}\[\\left\\langle\{x\-y\},\{\\mathsf\{SR\}\(p\)\-\\mathsf\{SR\}\(q\)\}\\right\\rangle\]\+\\mathbb\{E\}\[\\\|\\mathsf\{SR\}\(p\)\-\\mathsf\{SR\}\(q\)\\\|\_\{2\}^\{2\}\]\.Here, since𝔼​\[𝖲𝖱​\(p\)\]=p\\mathbb\{E\}\[\\mathsf\{SR\}\(p\)\]=pand𝔼​\[𝖲𝖱​\(q\)\]=q\\mathbb\{E\}\[\\mathsf\{SR\}\(q\)\]=qand𝔼​\[⋅\]\\mathbb\{E\}\[\\cdot\]is a linear operator, we have

𝔼​\[⟨x−y,𝖲𝖱​\(p\)−𝖲𝖱​\(q\)⟩\]=⟨x−y,p−q⟩\.\\displaystyle\\mathbb\{E\}\[\\left\\langle\{x\-y\},\{\\mathsf\{SR\}\(p\)\-\\mathsf\{SR\}\(q\)\}\\right\\rangle\]=\\left\\langle\{x\-y\},\{p\-q\}\\right\\rangle\.Furthermore, we note that

𝔼​\[‖𝖲𝖱​\(p\)−𝖲𝖱​\(q\)‖22\]\\displaystyle\\mathbb\{E\}\[\\\|\\mathsf\{SR\}\(p\)\-\\mathsf\{SR\}\(q\)\\\|\_\{2\}^\{2\}\]=∑i=1d𝔼​\[\(𝖲𝖱​\(pi\)−𝖲𝖱​\(qi\)\)2\]\\displaystyle=\\sum\_\{i=1\}^\{d\}\\mathbb\{E\}\[\(\\mathsf\{SR\}\(p\_\{i\}\)\-\\mathsf\{SR\}\(q\_\{i\}\)\)^\{2\}\]wherepip\_\{i\}andqiq\_\{i\}are theii\-th coordinate ofppandqq, respectively\. For eachi∈\[d\]i\\in\[d\], let\|pi−qi\|/Δ=mi\+ri\|p\_\{i\}\-q\_\{i\}\|/\\Delta=m\_\{i\}\+r\_\{i\}wheremi∈ℤ≥0m\_\{i\}\\in\\mathbb\{Z\}\_\{\\geq 0\}andri∈\[0,1\)r\_\{i\}\\in\[0,1\)\. Then, under the monotone coupling, we have

\|𝖲𝖱​\(pi\)−𝖲𝖱​\(qi\)\|=\{\(mi\+1\)​Δwith probability​ri,mi​Δwith probability​1−ri\.\\displaystyle\|\\mathsf\{SR\}\(p\_\{i\}\)\-\\mathsf\{SR\}\(q\_\{i\}\)\|=\\begin\{cases\}\(m\_\{i\}\+1\)\\Delta&\\text\{with probability \}r\_\{i\},\\\\ m\_\{i\}\\Delta&\\text\{with probability \}1\-r\_\{i\}\.\\end\{cases\}Then, we have

𝔼​\[\|𝖲𝖱​\(pi\)−𝖲𝖱​\(qi\)\|2\]\\displaystyle\\mathbb\{E\}\[\|\\mathsf\{SR\}\(p\_\{i\}\)\-\\mathsf\{SR\}\(q\_\{i\}\)\|^\{2\}\]=Δ2​\(ri​\(mi\+1\)2\+\(1−ri\)​mi2\)\\displaystyle=\\Delta^\{2\}\(r\_\{i\}\(m\_\{i\}\+1\)^\{2\}\+\(1\-r\_\{i\}\)m\_\{i\}^\{2\}\)=Δ2​\(mi2\+2​mi​ri\+ri\)\\displaystyle=\\Delta^\{2\}\(m\_\{i\}^\{2\}\+2m\_\{i\}r\_\{i\}\+r\_\{i\}\)=Δ2​\(mi2\+2​mi​ri\+ri2−ri2\+ri\)\\displaystyle=\\Delta^\{2\}\(m\_\{i\}^\{2\}\+2m\_\{i\}r\_\{i\}\+r\_\{i\}^\{2\}\-r\_\{i\}^\{2\}\+r\_\{i\}\)=\|pi−qi\|2\+Δ2​ri​\(1−ri\)\.\\displaystyle=\|p\_\{i\}\-q\_\{i\}\|^\{2\}\+\\Delta^\{2\}r\_\{i\}\(1\-r\_\{i\}\)\.Here, sinceri​\(1−ri\)≤ri≤\|pi−qi\|/Δr\_\{i\}\(1\-r\_\{i\}\)\\leq r\_\{i\}\\leq\|p\_\{i\}\-q\_\{i\}\|/\\Delta, we have𝔼​\[\|𝖲𝖱​\(pi\)−𝖲𝖱​\(qi\)\|2\]≤\|pi−qi\|2\+Δ​\|pi−qi\|\\mathbb\{E\}\[\|\\mathsf\{SR\}\(p\_\{i\}\)\-\\mathsf\{SR\}\(q\_\{i\}\)\|^\{2\}\]\\leq\|p\_\{i\}\-q\_\{i\}\|^\{2\}\+\\Delta\|p\_\{i\}\-q\_\{i\}\|\. Hence, we obtain

𝔼​\[‖𝖲𝖱​\(p\)−𝖲𝖱​\(q\)‖22\]≤‖p−q‖22\+Δ​‖p−q‖1\.\\displaystyle\\mathbb\{E\}\[\\\|\\mathsf\{SR\}\(p\)\-\\mathsf\{SR\}\(q\)\\\|\_\{2\}^\{2\}\]\\leq\\\|p\-q\\\|\_\{2\}^\{2\}\+\\Delta\\\|p\-q\\\|\_\{1\}\.
Conclusively, we have

𝔼​\[‖x−𝖲𝖱​\(p\)−y\+𝖲𝖱​\(q\)‖22\]\\displaystyle\\mathbb\{E\}\[\\\|x\-\\mathsf\{SR\}\(p\)\-y\+\\mathsf\{SR\}\(q\)\\\|\_\{2\}^\{2\}\]≤‖x−y‖22−2​⟨x−y,p−q⟩\+‖p−q‖22\+Δ​‖p−q‖1\\displaystyle\\leq\\\|x\-y\\\|\_\{2\}^\{2\}\-2\\left\\langle\{x\-y\},\{p\-q\}\\right\\rangle\+\\\|p\-q\\\|\_\{2\}^\{2\}\+\\Delta\\\|p\-q\\\|\_\{1\}=‖x−y−\(p−q\)‖22\+Δ​‖p−q‖1\\displaystyle=\\\|x\-y\-\(p\-q\)\\\|\_\{2\}^\{2\}\+\\Delta\\\|p\-q\\\|\_\{1\}which completes the proof\.

#### B\.2\.2Proof of[Section˜B\.2](https://arxiv.org/html/2606.06934#A2.SS2)

In this section, we prove[Section˜B\.2](https://arxiv.org/html/2606.06934#A2.SS2)\. Here, we prove a stronger statement on the second moment𝔼​\[Xt\+12\]\\mathbb\{E\}\[X\_\{t\+1\}^\{2\}\], and then use Jensen’s inequality: if we prove

𝔼​\[Xt\+12\]≤\(α​t\+γ​t\+ξ\)2,\\displaystyle\\mathbb\{E\}\[X\_\{t\+1\}^\{2\}\]\\leq\(\\alpha t\+\\sqrt\{\\gamma t\+\\xi\}\)^\{2\},\(12\)then it holds that𝔼​\[Xt\+1\]≤𝔼​\[Xt\+12\]≤α​t\+γ​t\+ξ\\mathbb\{E\}\[X\_\{t\+1\}\]\\leq\\sqrt\{\\mathbb\{E\}\[X\_\{t\+1\}^\{2\}\]\}\\leq\\alpha t\+\\sqrt\{\\gamma t\+\\xi\}, which completes the proof\.

The base case \(t=0t=0\) can be easily obtained from the assumption:𝔼​\[X12\]≤ξ\\mathbb\{E\}\[X\_\{1\}^\{2\}\]\\leq\\xi\.

We now fixt≥1t\\geq 1, and assume the claim holds fors=t−1s=t\-1, i\.e\.,

𝔼​\[Xt2\]≤\(α​\(t−1\)\+γ​\(t−1\)\+ξ\)2\.\\displaystyle\\mathbb\{E\}\[X\_\{t\}^\{2\}\]\\leq\\left\(\\alpha\(t\-1\)\+\\sqrt\{\\gamma\(t\-1\)\+\\xi\}\\right\)^\{2\}\.We note that

𝔼​\[Xt\+12\]≤max⁡\{𝔼​\[Xt2\]\+α​𝔼​\[Xt\]\+γ,ω\}\.\\displaystyle\\mathbb\{E\}\[X\_\{t\+1\}^\{2\}\]\\leq\\max\\\{\\mathbb\{E\}\[X\_\{t\}^\{2\}\]\+\\alpha\\mathbb\{E\}\[X\_\{t\}\]\+\\gamma,\\omega\\\}\.Here, if𝔼​\[Xt\+12\]≤ω\\mathbb\{E\}\[X\_\{t\+1\}^\{2\}\]\\leq\\omega, then, sinceξ≥ω\\xi\\geq\\omega, it holds that

𝔼​\[Xt\+12\]≤ω≤ξ≤\(α​t\+γ​t\+ξ\)2\\displaystyle\\mathbb\{E\}\[X\_\{t\+1\}^\{2\}\]\\leq\\omega\\leq\\xi\\leq\(\\alpha t\+\\sqrt\{\\gamma t\+\\xi\}\)^\{2\}sinceα,γ\>0\\alpha,\\gamma\>0\.

For the rest of the proof, we consider the case where𝔼​\[Xt\+12\]≤𝔼​\[Xt2\]\+α​𝔼​\[Xt\]\+γ\\mathbb\{E\}\[X\_\{t\+1\}^\{2\}\]\\leq\\mathbb\{E\}\[X\_\{t\}^\{2\}\]\+\\alpha\\mathbb\{E\}\[X\_\{t\}\]\+\\gamma\. LetA=γ​\(t−1\)\+ξA=\\sqrt\{\\gamma\(t\-1\)\+\\xi\},B=α​\(t−1\)\+AB=\\alpha\(t\-1\)\+A, andC=γ​t\+ξ−AC=\\sqrt\{\\gamma t\+\\xi\}\-A\. Then, by the induction hypothesis and Jensen’s inequality, it holds that

𝔼​\[Xt\+12\]≤𝔼​\[Xt2\]\+α​𝔼​\[Xt\]\+γ\\displaystyle\\mathbb\{E\}\[X\_\{t\+1\}^\{2\}\]\\leq\\mathbb\{E\}\[X\_\{t\}^\{2\}\]\+\\alpha\\mathbb\{E\}\[X\_\{t\}\]\+\\gamma≤𝔼​\[Xt2\]\+α​𝔼​\[Xt2\]\+γ≤B2\+α​B\+γ\.\\displaystyle\\leq\\mathbb\{E\}\[X\_\{t\}^\{2\}\]\+\\alpha\\sqrt\{\\mathbb\{E\}\[X\_\{t\}^\{2\}\]\}\+\\gamma\\leq B^\{2\}\+\\alpha B\+\\gamma\.We now show that

B2\+α​B\+γ≤\(B\+C\+α\)2=\(α​t\+γ​t\+ξ\)2\.\\displaystyle B^\{2\}\+\\alpha B\+\\gamma\\leq\(B\+C\+\\alpha\)^\{2\}=\(\\alpha t\+\\sqrt\{\\gamma t\+\\xi\}\)^\{2\}\.\(13\)
We note that sinceA\+C=γ​t\+ξA\+C=\\sqrt\{\\gamma t\+\\xi\}, we have

\(A\+C\)2=γ​t\+ξ=γ\+γ​\(t−1\)\+ξ=γ\+A2\\displaystyle\(A\+C\)^\{2\}=\\gamma t\+\\xi=\\gamma\+\\gamma\(t\-1\)\+\\xi=\\gamma\+A^\{2\}which impliesγ=2​C​A\+C2\\gamma=2CA\+C^\{2\}\. By applying this relation to the left side of[Eq\.˜13](https://arxiv.org/html/2606.06934#A2.E13), and by expanding the right side of[Eq\.˜13](https://arxiv.org/html/2606.06934#A2.E13), it can be written as

B2\+α​B\+2​C​A\+C2≤B2\+C2\+α2\+2​B​C\+2​C​α\+2​α​B\\displaystyle B^\{2\}\+\\alpha B\+2CA\+C^\{2\}\\leq B^\{2\}\+C^\{2\}\+\\alpha^\{2\}\+2BC\+2C\\alpha\+2\\alpha Bwhich implies

2​C​A≤α2\+2​B​C\+2​C​α\+α​B\.\\displaystyle 2CA\\leq\\alpha^\{2\}\+2BC\+2C\\alpha\+\\alpha B\.We now evaluate this inequality\. SinceA≤BA\\leq B, we have2​C​A≤2​B​C≤2​B​C\+α2\+2​C​α\+α​B2CA\\leq 2BC\\leq 2BC\+\\alpha^\{2\}\+2C\\alpha\+\\alpha B\. Therefore, the inductive step holds, and this completes the proof\.

### B\.3Proof of[Theorem˜11](https://arxiv.org/html/2606.06934#Thmtheorem11)

In this section, we prove[Theorem˜11](https://arxiv.org/html/2606.06934#Thmtheorem11)\. For eacht∈\[T\]t\\in\[T\], letδt=wt−wt′\\delta\_\{t\}=w\_\{t\}\-w\_\{t\}^\{\\prime\}where\(wt\)t=1T\(w\_\{t\}\)\_\{t=1\}^\{T\}and\(wt′\)t=1T\(w\_\{t\}^\{\\prime\}\)\_\{t=1\}^\{T\}are the iterates ofAf,θ𝖲𝖱​\-GD​\(S\)A\_\{f,\\theta\}^\{\\mathsf\{SR\}\\text\{\\rm\{\-GD\}\}\}\(S\)andAf,θ𝖲𝖱​\-GD​\(S′\)A\_\{f,\\theta\}^\{\\mathsf\{SR\}\\text\{\\rm\{\-GD\}\}\}\(S^\{\\prime\}\)whereS≃S′S\\simeq S^\{\\prime\}, respectively\. Note that the stochastic rounding operations of two𝖲𝖱​\-GD\{\\mathsf\{SR\}\\text\{\\rm\{\-GD\}\}\}parameters are coupled coordinate\-wise and monotonically\. That is, for each iterationt∈\[T\]t\\in\[T\]and coordinatej∈\[d\]j\\in\[d\], two rounding operations inwt,jw\_\{t,j\}andwt,j′w\_\{t,j\}^\{\\prime\}use the same uniform random variable, independently acrossttandjj\. LetBtB\_\{t\}be an event that there existsτ≤t\\tau\\leq tsuch thatwτ≠wτ′w\_\{\\tau\}\\neq w\_\{\\tau\}^\{\\prime\}, andAtA\_\{t\}be an event thatwt≠wt′w\_\{t\}\\neq w\_\{t\}^\{\\prime\}; we defineB0=∅B\_\{0\}=\\varnothing\. Since𝔼​\[‖δT‖2\|BTc\]=0\\mathbb\{E\}\[\\\|\\delta\_\{T\}\\\|\_\{2\}\|B\_\{T\}^\{c\}\]=0, we have𝔼​\[‖δT‖2\]=ℙ​\(BT\)​𝔼​\[‖δT‖2\|BT\]\\mathbb\{E\}\[\\\|\\delta\_\{T\}\\\|\_\{2\}\]=\\mathbb\{P\}\(B\_\{T\}\)\\mathbb\{E\}\[\\\|\\delta\_\{T\}\\\|\_\{2\}\|B\_\{T\}\]\.

Here, the upper bound ofℙ​\(BT\)\\mathbb\{P\}\(B\_\{T\}\)can be obtained by the following lemma\. We present the proof of[Section˜B\.3](https://arxiv.org/html/2606.06934#A2.SS3)in[Section˜B\.3\.1](https://arxiv.org/html/2606.06934#A2.SS3.SSS1)\.

###### Lemma 29\.

LetR,Δ\>0R,\\Delta\>0andd,T∈ℕd,T\\in\\mathbb\{N\}\. Consider two sequences\(xt\)t=0T\(x\_\{t\}\)\_\{t=0\}^\{T\}and\(yt\)t=0T\(y\_\{t\}\)\_\{t=0\}^\{T\}such that for eacht∈\[T\]t\\in\[T\],xt=xt−1−𝖲𝖱​\(pt\)x\_\{t\}=x\_\{t\-1\}\-\\mathsf\{SR\}\(p\_\{t\}\)andyt=yt−1−𝖲𝖱​\(qt\)y\_\{t\}=y\_\{t\-1\}\-\\mathsf\{SR\}\(q\_\{t\}\)wherex0=y0x\_\{0\}=y\_\{0\}andpt,qtp\_\{t\},q\_\{t\}aredd\-dimensional random vectors depending onxt−1x\_\{t\-1\}andyt−1y\_\{t\-1\}, respectively\. For eacht∈\[T\]t\\in\[T\], letBtB\_\{t\}be the event that there existsτ∈\[t\]\\tau\\in\[t\]such thatxτ≠yτx\_\{\\tau\}\\neq y\_\{\\tau\}\. Suppose that‖pt−qt‖2≤R\\\|p\_\{t\}\-q\_\{t\}\\\|\_\{2\}\\leq Ron the eventBt−1cB\_\{t\-1\}^\{c\}\. Then, under the coordinate\-wise monotone coupling, it holds that

ℙ​\(BT\)≤min⁡\{1,R​T​dΔ\}\.\\displaystyle\\mathbb\{P\}\(B\_\{T\}\)\\leq\\min\\left\\\{1,\\frac\{RT\\sqrt\{d\}\}\{\\Delta\}\\right\\\}\.

On the eventBt−1cB\_\{t\-1\}^\{c\}, we havewt−1=wt−1′w\_\{t\-1\}=w\_\{t\-1\}^\{\\prime\}, so‖η​∇f​\(wt−1;S\)−η​∇f​\(wt−1′;S′\)‖2≤2​η​L/n\\\|\\eta\\nabla f\(w\_\{t\-1\};S\)\-\\eta\\nabla f\(w\_\{t\-1\}^\{\\prime\};S^\{\\prime\}\)\\\|\_\{2\}\\leq 2\\eta L/nby theLL\-Lipschitzness offf\. By applying[Section˜B\.3](https://arxiv.org/html/2606.06934#A2.SS3)with the choices ofxt=wtx\_\{t\}=w\_\{t\},pt=η​∇f​\(wt−1;S\)p\_\{t\}=\\eta\\nabla f\(w\_\{t\-1\};S\),yt=wt′y\_\{t\}=w\_\{t\}^\{\\prime\}, andqt=η​∇f​\(wt−1′;S′\)q\_\{t\}=\\eta\\nabla f\(w\_\{t\-1\}^\{\\prime\};S^\{\\prime\}\)for allt∈\[T\]t\\in\[T\], we have

ℙ​\(BT\)≤min⁡\{1,2​η​L​T​dn​Δ\}\.\\displaystyle\\mathbb\{P\}\(B\_\{T\}\)\\leq\\min\\left\\\{1,\\frac\{2\\eta LT\\sqrt\{d\}\}\{n\\Delta\}\\right\\\}\.\(14\)In the rest of the proof, we show the upper bound on𝔼​\[‖δT‖2\|BT\]\\mathbb\{E\}\[\\\|\\delta\_\{T\}\\\|\_\{2\}\|B\_\{T\}\]\. To this end, we firstly construct a recurrence with respect to𝔼​\[‖δt‖22\|Bt\]\\mathbb\{E\}\[\\\|\\delta\_\{t\}\\\|\_\{2\}^\{2\}\|B\_\{t\}\]; we then reduce it to the upper bound on𝔼​\[‖δT‖2\|BT\]\\mathbb\{E\}\[\\\|\\delta\_\{T\}\\\|\_\{2\}\|B\_\{T\}\]\.

We note that for eacht∈\[T\]t\\in\[T\],BtB\_\{t\}is the union of two mutually exclusive events,Bt−1c∩AtB\_\{t\-1\}^\{c\}\\cap A\_\{t\}andBt−1B\_\{t\-1\}\. Then, we can decompose𝔼​\[‖δt‖22\|Bt\]\\mathbb\{E\}\[\\\|\\delta\_\{t\}\\\|\_\{2\}^\{2\}\|B\_\{t\}\]as follows:

𝔼​\[∥δt∥22\|Bt\]\\displaystyle\\mathbb\{E\}\[\\lVert\\delta\_\{t\}\\rVert\_\{2\}^\{2\}\|B\_\{t\}\]=P​\(Bt−1c∩At\)P​\(Bt\)​𝔼​\[∥δt∥22\|Bt−1c∩At\]\+P​\(Bt−1\)P​\(Bt\)​𝔼​\[∥δt∥22\|Bt−1\]\\displaystyle=\\frac\{P\(B\_\{t\-1\}^\{c\}\\cap A\_\{t\}\)\}\{P\(B\_\{t\}\)\}\\mathbb\{E\}\[\\lVert\\delta\_\{t\}\\rVert\_\{2\}^\{2\}\|B\_\{t\-1\}^\{c\}\\cap A\_\{t\}\]\+\\frac\{P\(B\_\{t\-1\}\)\}\{P\(B\_\{t\}\)\}\\mathbb\{E\}\[\\lVert\\delta\_\{t\}\\rVert\_\{2\}^\{2\}\|B\_\{t\-1\}\]≤max⁡\{𝔼​\[∥δt∥22\|Bt−1c∩At\],𝔼​\[∥δt∥22\|Bt−1\]\}\.\\displaystyle\\leq\\max\\\{\\mathbb\{E\}\[\\lVert\\delta\_\{t\}\\rVert\_\{2\}^\{2\}\|B\_\{t\-1\}^\{c\}\\cap A\_\{t\}\],\\mathbb\{E\}\[\\lVert\\delta\_\{t\}\\rVert\_\{2\}^\{2\}\|B\_\{t\-1\}\]\\\}\.
To show the upper bound of𝔼​\[‖δt‖22\|Bt−1c∩At\]\\mathbb\{E\}\[\\\|\\delta\_\{t\}\\\|\_\{2\}^\{2\}\|B\_\{t\-1\}^\{c\}\\cap A\_\{t\}\], we introduce the following lemma\. The proof of[Section˜B\.3](https://arxiv.org/html/2606.06934#A2.SS3)is in[Section˜B\.3\.3](https://arxiv.org/html/2606.06934#A2.SS3.SSS3)\.

###### Lemma 30\.

Letη,L,Δ\>0\\eta,L,\\Delta\>0,T,d∈ℕT,d\\in\\mathbb\{N\},w0∈ℤΔdw\_\{0\}\\in\\mathbb\{Z\}\_\{\\Delta\}^\{d\},ffbe a convex,LL\-Lipschitz loss function\. Forθ=\(η,T,w0\)\\theta=\(\\eta,T,w\_\{0\}\)andS≃S′∈𝒵nS\\simeq S^\{\\prime\}\\in\\mathcal\{Z\}^\{n\}, let\(wt\)t=1T\(w\_\{t\}\)\_\{t=1\}^\{T\}and\(wt′\)t=1T\(w\_\{t\}^\{\\prime\}\)\_\{t=1\}^\{T\}be the trajectories fromAf,θ𝖲𝖱​\-GD​\(S\)A\_\{f,\\theta\}^\{\\mathsf\{SR\}\\text\{\\rm\{\-GD\}\}\}\(S\)andAf,θ𝖲𝖱​\-GD​\(S′\)A\_\{f,\\theta\}^\{\\mathsf\{SR\}\\text\{\\rm\{\-GD\}\}\}\(S^\{\\prime\}\), respectively\. For eacht∈\[T\]t\\in\[T\], letδt=wt−wt′\\delta\_\{t\}=w\_\{t\}\-w\_\{t\}^\{\\prime\}andBtB\_\{t\}be an event that there existsτ≤t\\tau\\leq tsuch thatwτ≠wτ′w\_\{\\tau\}\\neq w\_\{\\tau\}^\{\\prime\}, andAtA\_\{t\}be an event thatwt≠wt′w\_\{t\}\\neq w\_\{t\}^\{\\prime\}\. Then, under the coordinate\-wise monotone coupling, it holds that

𝔼\[∥δt∥22\|Bt−1c∩At\]≤max\{Δ,2​η​Ln\}2\+2​Δ​η​L​dn\.\\displaystyle\\mathbb\{E\}\[\\lVert\\delta\_\{t\}\\rVert\_\{2\}^\{2\}\|B\_\{t\-1\}^\{c\}\\cap A\_\{t\}\]\\leq\\max\\left\\\{\\Delta,\\frac\{2\\eta L\}\{n\}\\right\\\}^\{2\}\+\\frac\{2\\Delta\\eta L\\sqrt\{d\}\}\{n\}\.

By[Section˜B\.3](https://arxiv.org/html/2606.06934#A2.SS3), we have𝔼\[∥δt∥22\|Bt−1c∩At\]≤max\{Δ,2ηL/n\}2\+2ΔηLd/n\\mathbb\{E\}\[\\\|\\delta\_\{t\}\\\|\_\{2\}^\{2\}\|B\_\{t\-1\}^\{c\}\\cap A\_\{t\}\]\\leq\\max\\\{\\Delta,2\\eta L/n\\\}^\{2\}\+2\\Delta\\eta L\\sqrt\{d\}/n\.

We now show the upper bound of𝔼​\[‖δt‖22\|Bt−1\]\\mathbb\{E\}\[\\\|\\delta\_\{t\}\\\|\_\{2\}^\{2\}\|B\_\{t\-1\}\]\. To this end, we introduce the following lemma\. The proof of[Section˜B\.3](https://arxiv.org/html/2606.06934#A2.SS3)is in[Section˜B\.3\.5](https://arxiv.org/html/2606.06934#A2.SS3.SSS5)\.

###### Lemma 31\.

Letη\>0\\eta\>0,T,d∈ℕT,d\\in\\mathbb\{N\},f:ℝd×𝒵→ℝf:\\mathbb\{R\}^\{d\}\\times\\mathcal\{Z\}\\to\\mathbb\{R\}be a convex,LL\-Lipschitz, andMM\-smooth function\. LetS,S′∈𝒵nS,S^\{\\prime\}\\in\\mathcal\{Z\}^\{n\}be the neighboring datasets\. Then, ifη<2/M\\eta<2/M, it holds that

‖x−η​∇f​\(x;S\)−\(y−η​∇f​\(y;S′\)\)‖22≤‖x−y‖22\+4​η​Ln​‖x−y‖2\+4​η2​L2n2\.\\displaystyle\\\|x\-\\eta\\nabla f\(x;S\)\-\(y\-\\eta\\nabla f\(y;S^\{\\prime\}\)\)\\\|\_\{2\}^\{2\}\\leq\\\|x\-y\\\|\_\{2\}^\{2\}\+\\frac\{4\\eta L\}\{n\}\\\|x\-y\\\|\_\{2\}\+\\frac\{4\\eta^\{2\}L^\{2\}\}\{n^\{2\}\}\.

Then, conditioned onBt−1B\_\{t\-1\}, we have

𝔼​\[‖δt‖22\]\\displaystyle\\mathbb\{E\}\[\\\|\\delta\_\{t\}\\\|\_\{2\}^\{2\}\]≤𝔼​\[‖δt−1−η​\(∇f​\(wt−1;S\)−∇f​\(wt−1′;S′\)\)‖22\]\+η​Δ​‖∇f​\(wt−1;S\)−∇f​\(wt−1′;S′\)‖1\\displaystyle\\leq\\mathbb\{E\}\[\\\|\\delta\_\{t\-1\}\-\\eta\(\\nabla f\(w\_\{t\-1\};S\)\-\\nabla f\(w\_\{t\-1\}^\{\\prime\};S^\{\\prime\}\)\)\\\|\_\{2\}^\{2\}\]\+\\eta\\Delta\\\|\\nabla f\(w\_\{t\-1\};S\)\-\\nabla f\(w\_\{t\-1\}^\{\\prime\};S^\{\\prime\}\)\\\|\_\{1\}≤𝔼​\[‖δt−1‖22\]\+4​η​Ln​𝔼​\[‖δt−1‖2\]\+4​η2​L2n2\+2​Δ​η​L​d\\displaystyle\\leq\\mathbb\{E\}\[\\\|\\delta\_\{t\-1\}\\\|\_\{2\}^\{2\}\]\+\\frac\{4\\eta L\}\{n\}\\mathbb\{E\}\[\\\|\\delta\_\{t\-1\}\\\|\_\{2\}\]\+\\frac\{4\\eta^\{2\}L^\{2\}\}\{n^\{2\}\}\+2\\Delta\\eta L\\sqrt\{d\}where the first inequality follows from[Section˜B\.2](https://arxiv.org/html/2606.06934#A2.SS2)and the last inequality follows from[Section˜B\.3](https://arxiv.org/html/2606.06934#A2.SS3)\.

Then, we have

𝔼​\[‖δt‖22\|Bt\]≤max⁡\{𝔼​\[‖δt−1‖22\|Bt−1\]\+4​η​Ln​𝔼​\[‖δt−1‖2\|Bt−1\]\+γ,ω\}\\displaystyle\\mathbb\{E\}\[\\\|\\delta\_\{t\}\\\|\_\{2\}^\{2\}\|B\_\{t\}\]\\leq\\max\\left\\\{\\mathbb\{E\}\[\\\|\\delta\_\{t\-1\}\\\|\_\{2\}^\{2\}\|B\_\{t\-1\}\]\+\\frac\{4\\eta L\}\{n\}\\mathbb\{E\}\[\\\|\\delta\_\{t\-1\}\\\|\_\{2\}\|B\_\{t\-1\}\]\+\\gamma,\\omega\\right\\\}whereγ=4​η2​L2/n2\+2​Δ​η​L​d\\gamma=4\\eta^\{2\}L^\{2\}/n^\{2\}\+2\\Delta\\eta L\\sqrt\{d\}andω=max\{Δ,2ηL/n\}2\+2ΔηLd/n\\omega=\\max\\\{\\Delta,2\\eta L/n\\\}^\{2\}\+2\\Delta\\eta L\\sqrt\{d\}/n\.

We now solve the recurrence of𝔼​\[‖δt‖22\|Bt\]\\mathbb\{E\}\[\\\|\\delta\_\{t\}\\\|\_\{2\}^\{2\}\|B\_\{t\}\]using[Section˜B\.2](https://arxiv.org/html/2606.06934#A2.SS2)\. By[Section˜B\.3](https://arxiv.org/html/2606.06934#A2.SS3), we have𝔼\[∥δ1∥22\|B1\]=𝔼\[∥δ1∥22\|B0c∩A1\]≤ω=max\{Δ,2ηL/n\}2\+2ΔηLd/n\\mathbb\{E\}\[\\\|\\delta\_\{1\}\\\|\_\{2\}^\{2\}\|B\_\{1\}\]=\\mathbb\{E\}\[\\\|\\delta\_\{1\}\\\|\_\{2\}^\{2\}\|B\_\{0\}^\{c\}\\cap A\_\{1\}\]\\leq\\omega=\\max\\\{\\Delta,2\\eta L/n\\\}^\{2\}\+2\\Delta\\eta L\\sqrt\{d\}/n\. Then, by applying[Section˜B\.2](https://arxiv.org/html/2606.06934#A2.SS2)with the choices ofα=4​η​L/n\\alpha=4\\eta L/n,γ=4​η2​L2/n2\+2​Δ​η​L​d\\gamma=4\\eta^\{2\}L^\{2\}/n^\{2\}\+2\\Delta\\eta L\\sqrt\{d\}, andξ=max\{Δ,2ηL/n\}2\+2ΔηLd/n\\xi=\\max\\\{\\Delta,2\\eta L/n\\\}^\{2\}\+2\\Delta\\eta L\\sqrt\{d\}/n, we have

𝔼​\[‖δT‖2\|BT\]\\displaystyle\\mathbb\{E\}\[\\\|\\delta\_\{T\}\\\|\_\{2\}\|B\_\{T\}\]≤4​η​L​\(T−1\)n\+γ​\(T−1\)\+ω\\displaystyle\\leq\\frac\{4\\eta L\(T\-1\)\}\{n\}\+\\sqrt\{\\gamma\(T\-1\)\+\\omega\}=4​η​L​\(T−1\)n\+\(4​η2​L2n2\+2ΔηLd1/2\)\(T−1\)\+max\{Δ,2​η​Ln\}2\+2​Δ​η​L​d1/2n\\displaystyle=\\frac\{4\\eta L\(T\-1\)\}\{n\}\+\\sqrt\{\\left\(\\frac\{4\\eta^\{2\}L^\{2\}\}\{n^\{2\}\}\+2\\Delta\\eta Ld^\{1/2\}\\right\)\(T\-1\)\+\\max\\left\\\{\\Delta,\\frac\{2\\eta L\}\{n\}\\right\\\}^\{2\}\+\\frac\{2\\Delta\\eta Ld^\{1/2\}\}\{n\}\}≤6​η​L​Tn\+Δ2\+2​Δ​η​L​d1/2n\+2​Δ​η​L​d1/2​\(T−1\)\.\\displaystyle\\leq\\frac\{6\\eta LT\}\{n\}\+\\sqrt\{\\Delta^\{2\}\+\\frac\{2\\Delta\\eta Ld^\{1/2\}\}\{n\}\+2\\Delta\\eta Ld^\{1/2\}\(T\-1\)\}\.where the last inequality follows from the sub\-additivity of square root,4​η2​L2​\(T−1\)/n2≤2​η​L​\(T−1\)/n\\sqrt\{4\\eta^\{2\}L^\{2\}\(T\-1\)/n^\{2\}\}\\leq 2\\eta L\(T\-1\)/nandmax\{Δ,2ηL/n\}2≤Δ2\+\(2ηL/n\)2\\max\\\{\\Delta,2\\eta L/n\\\}^\{2\}\\leq\\Delta^\{2\}\+\(2\\eta L/n\)^\{2\}\. By multiplying this result byℙ​\(BT\)\\mathbb\{P\}\(B\_\{T\}\)in[Eq\.˜14](https://arxiv.org/html/2606.06934#A2.E14), we obtained the desired upper bound, and this completes the proof\.

#### B\.3\.1Proof of[Section˜B\.3](https://arxiv.org/html/2606.06934#A2.SS3)

In this section, we prove[Section˜B\.3](https://arxiv.org/html/2606.06934#A2.SS3)\. For eacht∈\[T\]t\\in\[T\], letAtA\_\{t\}be the event thatxt≠ytx\_\{t\}\\neq y\_\{t\}\. Then, sinceBT=⋃t=1T\(At∩Bt−1c\)B\_\{T\}=\\bigcup\_\{t=1\}^\{T\}\(A\_\{t\}\\cap B\_\{t\-1\}^\{c\}\), we haveℙ​\(BT\)≤∑t=1Tℙ​\(At∩Bt−1c\)≤∑t=1Tℙ​\(At\|Bt−1c\)\\mathbb\{P\}\(B\_\{T\}\)\\leq\\sum\_\{t=1\}^\{T\}\\mathbb\{P\}\(A\_\{t\}\\cap B\_\{t\-1\}^\{c\}\)\\leq\\sum\_\{t=1\}^\{T\}\\mathbb\{P\}\(A\_\{t\}\|B\_\{t\-1\}^\{c\}\)\. Thus, it suffices to show that

ℙ​\(At\|Bt−1c\)≤R​dΔ\\displaystyle\\mathbb\{P\}\(A\_\{t\}\|B\_\{t\-1\}^\{c\}\)\\leq\\frac\{R\\sqrt\{d\}\}\{\\Delta\}\(15\)for allt∈\[T\]t\\in\[T\]\. Then, we haveℙ​\(BT\)≤∑t=1TR​d/Δ=R​T​d/Δ\\mathbb\{P\}\(B\_\{T\}\)\\leq\\sum\_\{t=1\}^\{T\}R\\sqrt\{d\}/\\Delta=RT\\sqrt\{d\}/\\Delta, which completes the proof\.

We now show that[Eq\.˜15](https://arxiv.org/html/2606.06934#A2.E15)follows\. Specifically, we show that for fixedt∈\[T\]t\\in\[T\],

ℙ​\(At\|x1,⋯,xt−1,y1,⋯,yt−1,Bt−1c\)≤R​dΔ\\displaystyle\\mathbb\{P\}\(A\_\{t\}\|x\_\{1\},\\cdots,x\_\{t\-1\},y\_\{1\},\\cdots,y\_\{t\-1\},B\_\{t\-1\}^\{c\}\)\\leq\\frac\{R\\sqrt\{d\}\}\{\\Delta\}which directly implies that[Eq\.˜15](https://arxiv.org/html/2606.06934#A2.E15)follows\. Conditioned onx1,⋯,xt−1,y1,⋯,yt−1x\_\{1\},\\cdots,x\_\{t\-1\},y\_\{1\},\\cdots,y\_\{t\-1\}and the eventBt−1cB\_\{t\-1\}^\{c\}, two vectorsptp\_\{t\}andqtq\_\{t\}are fixed andxt−1=yt−1x\_\{t\-1\}=y\_\{t\-1\}\. Thus, the left\-hand side of the above inequality is equal toℙ​\(𝖲𝖱​\(pt\)≠𝖲𝖱​\(qt\)\)\\mathbb\{P\}\(\\mathsf\{SR\}\(p\_\{t\}\)\\neq\\mathsf\{SR\}\(q\_\{t\}\)\)\. Here, if‖pt−qt‖∞≥Δ\\\|p\_\{t\}\-q\_\{t\}\\\|\_\{\\infty\}\\geq\\Delta, then the desired inequalityℙ​\(𝖲𝖱​\(pt\)≠𝖲𝖱​\(qt\)\)≤R​d/Δ\\mathbb\{P\}\(\\mathsf\{SR\}\(p\_\{t\}\)\\neq\\mathsf\{SR\}\(q\_\{t\}\)\)\\leq R\\sqrt\{d\}/\\Deltaholds sinceR≥ΔR\\geq\\Deltaandd∈ℕd\\in\\mathbb\{N\}\. Thus, we assume that‖pt−qt‖∞<Δ\\\|p\_\{t\}\-q\_\{t\}\\\|\_\{\\infty\}<\\Delta\.

To this end, we introduce the following lemma\. The proof of[Section˜B\.3\.1](https://arxiv.org/html/2606.06934#A2.SS3.SSS1)is in[Section˜B\.3\.2](https://arxiv.org/html/2606.06934#A2.SS3.SSS2)\.

###### Lemma 32\.

LetΔ\>0\\Delta\>0andx,y∈ℝx,y\\in\\mathbb\{R\}such that\|x−y\|<Δ\|x\-y\|<\\Delta\. Under the monotone coupling, it holds that

\|𝖲𝖱​\(x\)−𝖲𝖱​\(y\)\|/Δ∼B​e​r​\(\|x−y\|/Δ\)\.\\displaystyle\|\\mathsf\{SR\}\(x\)\-\\mathsf\{SR\}\(y\)\|/\\Delta\\sim Ber\(\|x\-y\|/\\Delta\)\.

Letpt=\(pt,1,⋯,pt,d\)p\_\{t\}=\(p\_\{t,1\},\\cdots,p\_\{t,d\}\)andqt=\(qt,1,⋯,qt,d\)q\_\{t\}=\(q\_\{t,1\},\\cdots,q\_\{t,d\}\)\. Since‖pt−qt‖∞<Δ\\\|p\_\{t\}\-q\_\{t\}\\\|\_\{\\infty\}<\\Delta, we have\|pt,i−qt,i\|<Δ\|p\_\{t,i\}\-q\_\{t,i\}\|<\\Deltafor alli∈\[d\]i\\in\[d\]\. Then, by[Section˜B\.3\.1](https://arxiv.org/html/2606.06934#A2.SS3.SSS1), we have

\|𝖲𝖱​\(pt,i\)−𝖲𝖱​\(qt,i\)\|=\{Δw\.p\.\|pt,i−qt,i\|/Δ,0w\.p​\. 1−\|pt,i−qt,i\|/Δ​for all​i∈\[d\]\.\\displaystyle\|\\mathsf\{SR\}\(p\_\{t,i\}\)\-\\mathsf\{SR\}\(q\_\{t,i\}\)\|=\\begin\{cases\}\\Delta&w\.p\.\\ \\ \|p\_\{t,i\}\-q\_\{t,i\}\|/\\Delta,\\\\ 0&w\.p\.\\ \\ 1\-\|p\_\{t,i\}\-q\_\{t,i\}\|/\\Delta\\end\{cases\}~~\\text\{for all \}i\\in\[d\]\.Then, we have

ℙ​\(𝖲𝖱​\(pt\)≠𝖲𝖱​\(qt\)\)\\displaystyle\\mathbb\{P\}\(\\mathsf\{SR\}\(p\_\{t\}\)\\neq\\mathsf\{SR\}\(q\_\{t\}\)\)=ℙ​\(𝖲𝖱​\(pt,i\)≠𝖲𝖱​\(qt,i\)​for some​i∈\[d\]\)\\displaystyle=\\mathbb\{P\}\\left\(\\mathsf\{SR\}\(p\_\{t,i\}\)\\neq\\mathsf\{SR\}\(q\_\{t,i\}\)\\text\{ for some \}i\\in\[d\]\\right\)≤∑i=1dℙ​\(𝖲𝖱​\(pt,i\)≠𝖲𝖱​\(qt,i\)\)\\displaystyle\\leq\\sum\_\{i=1\}^\{d\}\\mathbb\{P\}\(\\mathsf\{SR\}\(p\_\{t,i\}\)\\neq\\mathsf\{SR\}\(q\_\{t,i\}\)\)=∑i=1d\|pt,i−qt,i\|Δ\\displaystyle=\\sum\_\{i=1\}^\{d\}\\frac\{\|p\_\{t,i\}\-q\_\{t,i\}\|\}\{\\Delta\}=‖pt−qt‖1Δ≤d​‖pt−qt‖2Δ≤R​dΔ\.\\displaystyle=\\frac\{\\\|p\_\{t\}\-q\_\{t\}\\\|\_\{1\}\}\{\\Delta\}\\leq\\frac\{\\sqrt\{d\}\\\|p\_\{t\}\-q\_\{t\}\\\|\_\{2\}\}\{\\Delta\}\\leq\\frac\{R\\sqrt\{d\}\}\{\\Delta\}\.This completes the proof\.

#### B\.3\.2Proof of[Section˜B\.3\.1](https://arxiv.org/html/2606.06934#A2.SS3.SSS1)

In this section, we prove[Section˜B\.3\.1](https://arxiv.org/html/2606.06934#A2.SS3.SSS1)\. Without loss of generality, we assume thatx\>yx\>y\. From the definition of𝖲𝖱\\mathsf\{SR\}\(See[Section˜2\.3](https://arxiv.org/html/2606.06934#S2.SS3)\), we have

𝖲𝖱​\(x\)=⌊x⌋Δ\+Δ​𝟙​\[U≤rxΔ\],𝖲𝖱​\(y\)=⌊y⌋Δ\+Δ​𝟙​\[U≤ryΔ\]\\displaystyle\\mathsf\{SR\}\(x\)=\\lfloor x\\rfloor\_\{\\Delta\}\+\\Delta\\mathbbm\{1\}\\left\[U\\leq\\frac\{r\_\{x\}\}\{\\Delta\}\\right\],\\quad\\mathsf\{SR\}\(y\)=\\lfloor y\\rfloor\_\{\\Delta\}\+\\Delta\\mathbbm\{1\}\\left\[U\\leq\\frac\{r\_\{y\}\}\{\\Delta\}\\right\]whererx=x−⌊x⌋Δr\_\{x\}=x\-\\lfloor x\\rfloor\_\{\\Delta\},ry=y−⌊y⌋Δr\_\{y\}=y\-\\lfloor y\\rfloor\_\{\\Delta\},U∼U​n​i​f​\(\[0,1\]\)U\\sim Unif\(\[0,1\]\)\. We note that under the monotone coupling,𝖲𝖱​\(x\)\\mathsf\{SR\}\(x\)and𝖲𝖱​\(y\)\\mathsf\{SR\}\(y\)use the sameUU\. Then, we have

𝟙​\[U≤rxΔ\]−𝟙​\[U≤ryΔ\]=\{1ryΔ<U≤rxΔ−1rxΔ<U≤ryΔ0o\.w\.\\displaystyle\\mathbbm\{1\}\\left\[U\\leq\\frac\{r\_\{x\}\}\{\\Delta\}\\right\]\-\\mathbbm\{1\}\\left\[U\\leq\\frac\{r\_\{y\}\}\{\\Delta\}\\right\]=\\begin\{cases\}1&\\frac\{r\_\{y\}\}\{\\Delta\}<U\\leq\\frac\{r\_\{x\}\}\{\\Delta\}\\\\ \-1&\\frac\{r\_\{x\}\}\{\\Delta\}<U\\leq\\frac\{r\_\{y\}\}\{\\Delta\}\\\\ 0&o\.w\.\\end\{cases\}Then, we have

\|𝖲𝖱​\(x\)−𝖲𝖱​\(y\)\|\\displaystyle\|\\mathsf\{SR\}\(x\)\-\\mathsf\{SR\}\(y\)\|=\|⌊x⌋Δ−⌊y⌋Δ\+Δ​\(𝟙​\[U≤rxΔ\]−𝟙​\[U≤ryΔ\]\)\|\.\\displaystyle=\\left\|\\lfloor x\\rfloor\_\{\\Delta\}\-\\lfloor y\\rfloor\_\{\\Delta\}\+\\Delta\\left\(\\mathbbm\{1\}\\left\[U\\leq\\frac\{r\_\{x\}\}\{\\Delta\}\\right\]\-\\mathbbm\{1\}\\left\[U\\leq\\frac\{r\_\{y\}\}\{\\Delta\}\\right\]\\right\)\\right\|\.Since\|x−y\|<Δ\|x\-y\|<\\Delta,⌊x⌋Δ−⌊y⌋Δ∈\{0,Δ\}\\lfloor x\\rfloor\_\{\\Delta\}\-\\lfloor y\\rfloor\_\{\\Delta\}\\in\\\{0,\\Delta\\\}\. If⌊x⌋Δ=⌊y⌋Δ\\lfloor x\\rfloor\_\{\\Delta\}=\\lfloor y\\rfloor\_\{\\Delta\}, thenrx\>ryr\_\{x\}\>r\_\{y\}\. In this case,𝖲𝖱​\(x\)−𝖲𝖱​\(y\)=Δ\\mathsf\{SR\}\(x\)\-\\mathsf\{SR\}\(y\)=\\Deltaif and only ifry/Δ<U<rx/Δr\_\{y\}/\\Delta<U<r\_\{x\}/\\Deltawhich occurs with probability\(rx−ry\)/Δ=\(x−y\)/Δ\(r\_\{x\}\-r\_\{y\}\)/\\Delta=\(x\-y\)/\\Delta\.

On the other hand, if⌊x⌋Δ−⌊y⌋Δ=Δ\\lfloor x\\rfloor\_\{\\Delta\}\-\\lfloor y\\rfloor\_\{\\Delta\}=\\Delta, thenrx<ryr\_\{x\}<r\_\{y\}\. In this case,\|𝖲𝖱​\(x\)−𝖲𝖱​\(y\)\|=Δ\|\\mathsf\{SR\}\(x\)\-\\mathsf\{SR\}\(y\)\|=\\Deltaif and only ifU≤rx/ΔU\\leq r\_\{x\}/\\DeltaorU\>ry/ΔU\>r\_\{y\}/\\Delta, which occurs with probabilityrx/Δ\+\(1−ry/Δ\)=1−\(ry−rx\)/Δ=\(x−y\)/Δr\_\{x\}/\\Delta\+\(1\-r\_\{y\}/\\Delta\)=1\-\(r\_\{y\}\-r\_\{x\}\)/\\Delta=\(x\-y\)/\\Delta\.

Conclusively, in both cases, we have

\|𝖲𝖱​\(x\)−𝖲𝖱​\(y\)\|=\{Δw\.p\.\(x−y\)/Δ,0w\.p\.1−\(x−y\)/Δ\.\\displaystyle\|\\mathsf\{SR\}\(x\)\-\\mathsf\{SR\}\(y\)\|=\\begin\{cases\}\\Delta&w\.p\.~~\(x\-y\)/\\Delta,\\\\ 0&w\.p\.~~1\-\(x\-y\)/\\Delta\.\\end\{cases\}which completes the proof\.

#### B\.3\.3Proof of[Section˜B\.3](https://arxiv.org/html/2606.06934#A2.SS3)

In this section, we prove[Section˜B\.3](https://arxiv.org/html/2606.06934#A2.SS3)\. Lett∈\[T\]t\\in\[T\]\. Conditioned onBt−1c∩AtB\_\{t\-1\}^\{c\}\\cap A\_\{t\}, we havew1=w1′,⋯,wt−1=wt−1′w\_\{1\}=w\_\{1\}^\{\\prime\},\\cdots,w\_\{t\-1\}=w\_\{t\-1\}^\{\\prime\}andwt≠wt′w\_\{t\}\\neq w\_\{t\}^\{\\prime\}\. Thus, it holds that

δt=𝖲𝖱​\(η​∇f​\(wt−1′;S′\)\)−𝖲𝖱​\(η​∇f​\(wt−1;S\)\)\.\\displaystyle\\delta\_\{t\}=\\mathsf\{SR\}\(\\eta\\nabla f\(w\_\{t\-1\}^\{\\prime\};S^\{\\prime\}\)\)\-\\mathsf\{SR\}\(\\eta\\nabla f\(w\_\{t\-1\};S\)\)\.We show the upper bound on𝔼​\[‖δt‖22\|Bt−1c∩At\]\\mathbb\{E\}\[\\\|\\delta\_\{t\}\\\|\_\{2\}^\{2\}\|B\_\{t\-1\}^\{c\}\\cap A\_\{t\}\]by applying the following lemma, whose proof is presented in[Section˜B\.3\.4](https://arxiv.org/html/2606.06934#A2.SS3.SSS4)\.

###### Lemma 33\.

LetR\>0R\>0andx,y∈ℝdx,y\\in\\mathbb\{R\}^\{d\}such that0<∥x−y∥2≤R0<\\lVert x\-y\\rVert\_\{2\}\\leq R\. Under the coordinate\-wise monotone coupling, we have

𝔼\[∥𝖲𝖱\(x\)−𝖲𝖱\(y\)∥22\|𝖲𝖱\(x\)≠𝖲𝖱\(y\)\]≤max\{Δ,R\}2\+ΔRd\.\\displaystyle\\mathbb\{E\}\[\\\|\\mathsf\{SR\}\(x\)\-\\mathsf\{SR\}\(y\)\\\|\_\{2\}^\{2\}\|\\mathsf\{SR\}\(x\)\\neq\\mathsf\{SR\}\(y\)\]\\leq\\max\\\{\\Delta,R\\\}^\{2\}\+\\Delta R\\sqrt\{d\}\.

Sincewt−1=wt−1′w\_\{t\-1\}=w\_\{t\-1\}^\{\\prime\}andffisLL\-Lipschitz, it holds that‖η​∇f​\(wt−1′;S′\)−η​∇f​\(wt−1;S\)‖2≤2​η​L/n\\\|\\eta\\nabla f\(w\_\{t\-1\}^\{\\prime\};S^\{\\prime\}\)\-\\eta\\nabla f\(w\_\{t\-1\};S\)\\\|\_\{2\}\\leq 2\\eta L/n\. Then, by applying[Section˜B\.3\.3](https://arxiv.org/html/2606.06934#A2.SS3.SSS3)with the choice ofR=2​η​L/nR=2\\eta L/n, we have

𝔼\[∥δt∥22\|Bt−1c∩At\]≤max\{Δ,2​η​Ln\}2\+2​Δ​η​L​dn\.\\displaystyle\\mathbb\{E\}\[\\lVert\\delta\_\{t\}\\rVert\_\{2\}^\{2\}\|B\_\{t\-1\}^\{c\}\\cap A\_\{t\}\]\\leq\\max\\left\\\{\\Delta,\\frac\{2\\eta L\}\{n\}\\right\\\}^\{2\}\+\\frac\{2\\Delta\\eta L\\sqrt\{d\}\}\{n\}\.This completes the proof\.

#### B\.3\.4Proof of[Section˜B\.3\.3](https://arxiv.org/html/2606.06934#A2.SS3.SSS3)

In this section, we prove[Section˜B\.3\.3](https://arxiv.org/html/2606.06934#A2.SS3.SSS3)\. Sincex≠yx\\neq yand‖𝖲𝖱​\(x\)−𝖲𝖱​\(y\)‖22=0\\\|\\mathsf\{SR\}\(x\)\-\\mathsf\{SR\}\(y\)\\\|\_\{2\}^\{2\}=0if𝖲𝖱​\(x\)=𝖲𝖱​\(y\)\\mathsf\{SR\}\(x\)=\\mathsf\{SR\}\(y\), we have

𝔼​\[‖𝖲𝖱​\(x\)−𝖲𝖱​\(y\)‖22\|𝖲𝖱​\(x\)≠𝖲𝖱​\(y\)\]\\displaystyle\\mathbb\{E\}\[\\\|\\mathsf\{SR\}\(x\)\-\\mathsf\{SR\}\(y\)\\\|\_\{2\}^\{2\}\|\\mathsf\{SR\}\(x\)\\neq\\mathsf\{SR\}\(y\)\]=𝔼​\[‖𝖲𝖱​\(x\)−𝖲𝖱​\(y\)‖22​𝟙​\{𝖲𝖱​\(x\)≠𝖲𝖱​\(y\)\}\]ℙ​\(𝖲𝖱​\(x\)≠𝖲𝖱​\(y\)\)\\displaystyle=\\frac\{\\mathbb\{E\}\[\\\|\\mathsf\{SR\}\(x\)\-\\mathsf\{SR\}\(y\)\\\|\_\{2\}^\{2\}\\mathbbm\{1\}\\\{\\mathsf\{SR\}\(x\)\\neq\\mathsf\{SR\}\(y\)\\\}\]\}\{\\mathbb\{P\}\(\\mathsf\{SR\}\(x\)\\neq\\mathsf\{SR\}\(y\)\)\}=𝔼​\[‖𝖲𝖱​\(x\)−𝖲𝖱​\(y\)‖22\]ℙ​\(𝖲𝖱​\(x\)≠𝖲𝖱​\(y\)\)\.\\displaystyle=\\frac\{\\mathbb\{E\}\[\\\|\\mathsf\{SR\}\(x\)\-\\mathsf\{SR\}\(y\)\\\|\_\{2\}^\{2\}\]\}\{\\mathbb\{P\}\(\\mathsf\{SR\}\(x\)\\neq\\mathsf\{SR\}\(y\)\)\}\.We prove[Section˜B\.3\.3](https://arxiv.org/html/2606.06934#A2.SS3.SSS3)by showing the lower bound ofℙ​\(𝖲𝖱​\(x\)≠𝖲𝖱​\(y\)\)\\mathbb\{P\}\(\\mathsf\{SR\}\(x\)\\neq\\mathsf\{SR\}\(y\)\)and the upper bound of𝔼​\[‖𝖲𝖱​\(x\)−𝖲𝖱​\(y\)‖22\]\\mathbb\{E\}\[\\\|\\mathsf\{SR\}\(x\)\-\\mathsf\{SR\}\(y\)\\\|\_\{2\}^\{2\}\]\. We first prove the lower bound ofℙ​\(𝖲𝖱​\(x\)≠𝖲𝖱​\(y\)\)\\mathbb\{P\}\(\\mathsf\{SR\}\(x\)\\neq\\mathsf\{SR\}\(y\)\)\. Letx=\(x1,⋯,xd\)x=\(x\_\{1\},\\cdots,x\_\{d\}\)andy=\(y1,⋯,yd\)y=\(y\_\{1\},\\cdots,y\_\{d\}\)\. For eachi∈\[d\]i\\in\[d\], let\|xi−yi\|=mi​Δ\+ri\|x\_\{i\}\-y\_\{i\}\|=m\_\{i\}\\Delta\+r\_\{i\}formi∈ℤ≥0m\_\{i\}\\in\\mathbb\{Z\}\_\{\\geq 0\}andri∈\[0,Δ\)r\_\{i\}\\in\[0,\\Delta\)\. Ifmi≠0m\_\{i\}\\neq 0, then𝖲𝖱​\(xi\)=𝖲𝖱​\(yi\)\\mathsf\{SR\}\(x\_\{i\}\)=\\mathsf\{SR\}\(y\_\{i\}\)never occurs under monotone coupling\. Then, we have

ℙ​\(𝖲𝖱​\(xi\)=𝖲𝖱​\(yi\)\)=\{1−ri/Δmi=0,0o\.w\.\\displaystyle\\mathbb\{P\}\(\\mathsf\{SR\}\(x\_\{i\}\)=\\mathsf\{SR\}\(y\_\{i\}\)\)=\\begin\{cases\}1\-r\_\{i\}/\\Delta&m\_\{i\}=0,\\\\ 0&o\.w\.\\end\{cases\}We note that ifmi=0m\_\{i\}=0, thenri=\|xi−yi\|r\_\{i\}=\|x\_\{i\}\-y\_\{i\}\|\. Ifmi=0m\_\{i\}=0for alli∈\[d\]i\\in\[d\], i\.e\.,‖x−y‖∞<Δ\\\|x\-y\\\|\_\{\\infty\}<\\Delta, we have

ℙ​\(𝖲𝖱​\(x\)≠𝖲𝖱​\(y\)\)\\displaystyle\\mathbb\{P\}\(\\mathsf\{SR\}\(x\)\\neq\\mathsf\{SR\}\(y\)\)=1−∏i=1dℙ​\(𝖲𝖱​\(xi\)=𝖲𝖱​\(yi\)\)\\displaystyle=1\-\\prod\_\{i=1\}^\{d\}\\mathbb\{P\}\(\\mathsf\{SR\}\(x\_\{i\}\)=\\mathsf\{SR\}\(y\_\{i\}\)\)=1−∏i=1d\(1−\|xi−yi\|Δ\)\\displaystyle=1\-\\prod\_\{i=1\}^\{d\}\\left\(1\-\\frac\{\|x\_\{i\}\-y\_\{i\}\|\}\{\\Delta\}\\right\)≥1−exp⁡\(−∑i=1d\|xi−yi\|Δ\)=1−exp⁡\(−‖x−y‖1Δ\)\\displaystyle\\geq 1\-\\exp\\left\(\-\\sum\_\{i=1\}^\{d\}\\frac\{\|x\_\{i\}\-y\_\{i\}\|\}\{\\Delta\}\\right\)=1\-\\exp\\left\(\-\\frac\{\\\|x\-y\\\|\_\{1\}\}\{\\Delta\}\\right\)where the inequality follows from the standard inequality1−x≤exp⁡\(−x\)1\-x\\leq\\exp\(\-x\)\. If there existsi∈\[d\]i\\in\[d\]such thatmi\>0m\_\{i\}\>0, i\.e\.,\|xi−yi\|≥Δ\|x\_\{i\}\-y\_\{i\}\|\\geq\\Delta, thenℙ​\(𝖲𝖱​\(x\)≠𝖲𝖱​\(y\)\)=1\\mathbb\{P\}\(\\mathsf\{SR\}\(x\)\\neq\\mathsf\{SR\}\(y\)\)=1\.

We now show the upper bound of𝔼​\[‖𝖲𝖱​\(x\)−𝖲𝖱​\(y\)‖22\]\\mathbb\{E\}\[\\\|\\mathsf\{SR\}\(x\)\-\\mathsf\{SR\}\(y\)\\\|\_\{2\}^\{2\}\]\. If‖x−y‖∞<Δ\\\|x\-y\\\|\_\{\\infty\}<\\Delta, then by[Section˜B\.3\.1](https://arxiv.org/html/2606.06934#A2.SS3.SSS1),\|𝖲𝖱​\(xi\)−𝖲𝖱​\(yi\)\|/Δ∼B​e​r​\(\|xi−yi\|/Δ\)\|\\mathsf\{SR\}\(x\_\{i\}\)\-\\mathsf\{SR\}\(y\_\{i\}\)\|/\\Delta\\sim Ber\(\|x\_\{i\}\-y\_\{i\}\|/\\Delta\)for alli∈\[d\]i\\in\[d\]\. Then, we have

𝔼​\[‖𝖲𝖱​\(x\)−𝖲𝖱​\(y\)‖22\]\\displaystyle\\mathbb\{E\}\[\\\|\\mathsf\{SR\}\(x\)\-\\mathsf\{SR\}\(y\)\\\|\_\{2\}^\{2\}\]=Δ2​∑i=1d𝔼​\[\(\|𝖲𝖱​\(xi\)−𝖲𝖱​\(yi\)\|/Δ\)2\]\\displaystyle=\\Delta^\{2\}\\sum\_\{i=1\}^\{d\}\\mathbb\{E\}\[\(\|\\mathsf\{SR\}\(x\_\{i\}\)\-\\mathsf\{SR\}\(y\_\{i\}\)\|/\\Delta\)^\{2\}\]=Δ2​∑i=1d\(\(\|xi−yi\|Δ\)2\+\|xi−yi\|Δ​\(1−\|xi−yi\|Δ\)\)\\displaystyle=\\Delta^\{2\}\\sum\_\{i=1\}^\{d\}\\left\(\\left\(\\frac\{\|x\_\{i\}\-y\_\{i\}\|\}\{\\Delta\}\\right\)^\{2\}\+\\frac\{\|x\_\{i\}\-y\_\{i\}\|\}\{\\Delta\}\\left\(1\-\\frac\{\|x\_\{i\}\-y\_\{i\}\|\}\{\\Delta\}\\right\)\\right\)=Δ2×∑i=1d\|xi−yi\|Δ=Δ​‖x−y‖1\.\\displaystyle=\\Delta^\{2\}\\times\\frac\{\\sum\_\{i=1\}^\{d\}\|x\_\{i\}\-y\_\{i\}\|\}\{\\Delta\}=\\Delta\\\|x\-y\\\|\_\{1\}\.
Using these results, we now show the upper bound of𝔼​\[‖𝖲𝖱​\(x\)−𝖲𝖱​\(y\)‖22\|𝖲𝖱​\(x\)≠𝖲𝖱​\(y\)\]\\mathbb\{E\}\[\\\|\\mathsf\{SR\}\(x\)\-\\mathsf\{SR\}\(y\)\\\|\_\{2\}^\{2\}\|\\mathsf\{SR\}\(x\)\\neq\\mathsf\{SR\}\(y\)\]\. If‖x−y‖∞≥Δ\\\|x\-y\\\|\_\{\\infty\}\\geq\\Delta, the upper bound can be easily obtained by

𝔼​\[‖𝖲𝖱​\(x\)−𝖲𝖱​\(y\)‖22\|𝖲𝖱​\(x\)≠𝖲𝖱​\(y\)\]\\displaystyle\\mathbb\{E\}\[\\\|\\mathsf\{SR\}\(x\)\-\\mathsf\{SR\}\(y\)\\\|\_\{2\}^\{2\}\|\\mathsf\{SR\}\(x\)\\neq\\mathsf\{SR\}\(y\)\]=𝔼​\[‖𝖲𝖱​\(x\)−𝖲𝖱​\(y\)‖22\]ℙ​\(𝖲𝖱​\(x\)≠𝖲𝖱​\(y\)\)\\displaystyle=\\frac\{\\mathbb\{E\}\[\\\|\\mathsf\{SR\}\(x\)\-\\mathsf\{SR\}\(y\)\\\|\_\{2\}^\{2\}\]\}\{\\mathbb\{P\}\(\\mathsf\{SR\}\(x\)\\neq\\mathsf\{SR\}\(y\)\)\}≤‖x−y‖22\+Δ​‖x−y‖1\\displaystyle\\leq\\\|x\-y\\\|\_\{2\}^\{2\}\+\\Delta\\\|x\-y\\\|\_\{1\}≤R2\+Δ​R​d\.\\displaystyle\\leq R^\{2\}\+\\Delta R\\sqrt\{d\}\.Thus, we assume that‖x−y‖∞<Δ\\\|x\-y\\\|\_\{\\infty\}<\\Delta\. Letγ=‖x−y‖1/Δ\\gamma=\\\|x\-y\\\|\_\{1\}/\\Delta\. Then, by applying algebraic fact thatγ/\(1−exp⁡\(−γ\)\)≤1\+γ\\gamma/\(1\-\\exp\(\-\\gamma\)\)\\leq 1\+\\gamma, we have

𝔼​\[‖𝖲𝖱​\(x\)−𝖲𝖱​\(y\)‖22\|𝖲𝖱​\(x\)≠𝖲𝖱​\(y\)\]\\displaystyle\\mathbb\{E\}\[\\\|\\mathsf\{SR\}\(x\)\-\\mathsf\{SR\}\(y\)\\\|\_\{2\}^\{2\}\|\\mathsf\{SR\}\(x\)\\neq\\mathsf\{SR\}\(y\)\]=𝔼​\[‖𝖲𝖱​\(x\)−𝖲𝖱​\(y\)‖22\]ℙ​\(𝖲𝖱​\(x\)≠𝖲𝖱​\(y\)\)\\displaystyle=\\frac\{\\mathbb\{E\}\[\\\|\\mathsf\{SR\}\(x\)\-\\mathsf\{SR\}\(y\)\\\|\_\{2\}^\{2\}\]\}\{\\mathbb\{P\}\(\\mathsf\{SR\}\(x\)\\neq\\mathsf\{SR\}\(y\)\)\}≤Δ​‖x−y‖1ℙ​\(𝖲𝖱​\(x\)≠𝖲𝖱​\(y\)\)\\displaystyle\\leq\\frac\{\\Delta\\\|x\-y\\\|\_\{1\}\}\{\\mathbb\{P\}\(\\mathsf\{SR\}\(x\)\\neq\\mathsf\{SR\}\(y\)\)\}≤Δ2​γ1−exp⁡\(−γ\)\\displaystyle\\leq\\Delta^\{2\}\\frac\{\\gamma\}\{1\-\\exp\(\-\\gamma\)\}≤Δ2​\(1\+γ\)\\displaystyle\\leq\\Delta^\{2\}\(1\+\\gamma\)=Δ2\+Δ​‖x−y‖1≤Δ2\+Δ​d​R\.\\displaystyle=\\Delta^\{2\}\+\\Delta\\\|x\-y\\\|\_\{1\}\\leq\\Delta^\{2\}\+\\Delta\\sqrt\{d\}R\.Conclusively, we have

𝔼​\[‖𝖲𝖱​\(x\)−𝖲𝖱​\(y\)‖22\|𝖲𝖱​\(x\)≠𝖲𝖱​\(y\)\]≤max⁡\{Δ2,R2\}\+Δ​R​d\\displaystyle\\mathbb\{E\}\[\\\|\\mathsf\{SR\}\(x\)\-\\mathsf\{SR\}\(y\)\\\|\_\{2\}^\{2\}\|\\mathsf\{SR\}\(x\)\\neq\\mathsf\{SR\}\(y\)\]\\leq\\max\\\{\\Delta^\{2\},R^\{2\}\\\}\+\\Delta R\\sqrt\{d\}which completes the proof\.

#### B\.3\.5Proof of[Section˜B\.3](https://arxiv.org/html/2606.06934#A2.SS3)

In this section, we prove[Section˜B\.3](https://arxiv.org/html/2606.06934#A2.SS3)\. To this end, we use the fact that GD on the convex andMMsmooth function is non\-expansive when the learning rateη<2/M\\eta<2/M\([Section˜B\.2](https://arxiv.org/html/2606.06934#A2.SS2)\)\. Without loss of generality, suppose thatS=\(z1,⋯,zn\)S=\(z\_\{1\},\\cdots,z\_\{n\}\)andS′=\(z1′,z2,⋯,zn\)S^\{\\prime\}=\(z\_\{1\}^\{\\prime\},z\_\{2\},\\cdots,z\_\{n\}\)wherez1≠z1′z\_\{1\}\\neq z\_\{1\}^\{\\prime\}\. Then, since∇f​\(y;S′\)=∇f​\(y;S\)\+\(∇f​\(y;z1′\)−∇f​\(y;z1\)\)/n\\nabla f\(y;S^\{\\prime\}\)=\\nabla f\(y;S\)\+\(\\nabla f\(y;z\_\{1\}^\{\\prime\}\)\-\\nabla f\(y;z\_\{1\}\)\)/nand‖∇f​\(y;​z1\)−∇f​\(y;z1′\)‖2≤2​L\\\|\\nabla f\(y\_\{;\}z\_\{1\}\)\-\\nabla f\(y;z\_\{1\}^\{\\prime\}\)\\\|\_\{2\}\\leq 2L, by the triangle inequality, we have

‖x−η​∇f​\(x;S\)−\(y−η​∇f​\(y;S′\)\)‖2\\displaystyle\\\|x\-\\eta\\nabla f\(x;S\)\-\(y\-\\eta\\nabla f\(y;S^\{\\prime\}\)\)\\\|\_\{2\}≤‖x−η​∇f​\(x;S\)−\(y−η​∇f​\(y;S\)\)‖2\+2​η​Ln\\displaystyle\\leq\\\|x\-\\eta\\nabla f\(x;S\)\-\(y\-\\eta\\nabla f\(y;S\)\)\\\|\_\{2\}\+\\frac\{2\\eta L\}\{n\}≤‖x−y‖2\+2​η​Ln\\displaystyle\\leq\\\|x\-y\\\|\_\{2\}\+\\frac\{2\\eta L\}\{n\}By squaring both sides, we have

‖x−η​∇f​\(x;S\)−\(y−η​∇f​\(y;S′\)\)‖22≤‖x−y‖22\+4​η​Ln​‖x−y‖2\+4​η2​L2n2\\displaystyle\\\|x\-\\eta\\nabla f\(x;S\)\-\(y\-\\eta\\nabla f\(y;S^\{\\prime\}\)\)\\\|\_\{2\}^\{2\}\\leq\\\|x\-y\\\|\_\{2\}^\{2\}\+\\frac\{4\\eta L\}\{n\}\\\|x\-y\\\|\_\{2\}\+\\frac\{4\\eta^\{2\}L^\{2\}\}\{n^\{2\}\}which completes the proof\.

### B\.4Proof of[Theorem˜12](https://arxiv.org/html/2606.06934#Thmtheorem12)

In this section, we prove[Theorem˜12](https://arxiv.org/html/2606.06934#Thmtheorem12)\. We first show the upper bound for𝖲𝖱​\-GD\{\\mathsf\{SR\}\\text\{\\rm\{\-GD\}\}\}\. LetS=\(z1,⋯,zn\)S=\(z\_\{1\},\\cdots,z\_\{n\}\)andS′=\(z1′,⋯,zn′\)S^\{\\prime\}=\(z\_\{1\}^\{\\prime\},\\cdots,z\_\{n\}^\{\\prime\}\)be the neighboring datasets\. Let\(wt\)t=1T\(w\_\{t\}\)\_\{t=1\}^\{T\}and\(wt′\)t=1T\(w\_\{t\}^\{\\prime\}\)\_\{t=1\}^\{T\}be the trajectories fromAf,θ𝖲𝖱​\-GD​\(S\)A\_\{f,\\theta\}^\{\\mathsf\{SR\}\\text\{\\rm\{\-GD\}\}\}\(S\)andAf,θ𝖲𝖱​\-GD​\(S′\)A\_\{f,\\theta\}^\{\\mathsf\{SR\}\\text\{\\rm\{\-GD\}\}\}\(S^\{\\prime\}\), respectively, andδt=wt−wt′\\delta\_\{t\}=w\_\{t\}\-w\_\{t\}^\{\\prime\}\. In this proof, we couple the stochastic rounding operations coordinate\-wise and monotonically; the two rounding operations use the same random variableut,ju\_\{t,j\}in[Eq\.˜3](https://arxiv.org/html/2606.06934#S2.E3)for eacht∈\[T\]t\\in\[T\]andj∈\[d\]j\\in\[d\]\.

Conditioned onwtw\_\{t\}andwt′w\_\{t\}^\{\\prime\}, we have

𝔼​\[‖δt\+1‖22\]\\displaystyle\\mathbb\{E\}\[\\\|\\delta\_\{t\+1\}\\\|\_\{2\}^\{2\}\]=𝔼​\[‖δt−𝖲𝖱​\(η​∇f​\(wt;S\)\)\+𝖲𝖱​\(η​∇f​\(wt′;S′\)\)‖22\]\\displaystyle=\\mathbb\{E\}\[\\\|\\delta\_\{t\}\-\\mathsf\{SR\}\(\\eta\\nabla f\(w\_\{t\};S\)\)\+\\mathsf\{SR\}\(\\eta\\nabla f\(w\_\{t\}^\{\\prime\};S^\{\\prime\}\)\)\\\|\_\{2\}^\{2\}\]≤‖δt−η​\(∇f​\(wt;S\)−∇f​\(wt′;S′\)\)‖22\+η​Δ​‖∇f​\(wt;S\)−∇f​\(wt′;S′\)‖1\\displaystyle\\leq\\\|\\delta\_\{t\}\-\\eta\(\\nabla f\(w\_\{t\};S\)\-\\nabla f\(w\_\{t\}^\{\\prime\};S^\{\\prime\}\)\)\\\|\_\{2\}^\{2\}\+\\eta\\Delta\\\|\\nabla f\(w\_\{t\};S\)\-\\nabla f\(w\_\{t\}^\{\\prime\};S^\{\\prime\}\)\\\|\_\{1\}where the inequality in the second line follows from[Section˜B\.2](https://arxiv.org/html/2606.06934#A2.SS2)\. Letat=∇f​\(wt;S\)−∇f​\(wt′;S\)a\_\{t\}=\\nabla f\(w\_\{t\};S\)\-\\nabla f\(w\_\{t\}^\{\\prime\};S\)andbt=∇f​\(wt′;S\)−∇f​\(wt′;S′\)b\_\{t\}=\\nabla f\(w\_\{t\}^\{\\prime\};S\)\-\\nabla f\(w\_\{t\}^\{\\prime\};S^\{\\prime\}\)\. Then, we have∇f​\(wt;S\)−∇f​\(wt′;S′\)=at\+bt\\nabla f\(w\_\{t\};S\)\-\\nabla f\(w\_\{t\}^\{\\prime\};S^\{\\prime\}\)=a\_\{t\}\+b\_\{t\}\. Thus, we have

𝔼​\[‖δt\+1‖22\]\\displaystyle\\mathbb\{E\}\[\\\|\\delta\_\{t\+1\}\\\|\_\{2\}^\{2\}\]≤‖δt−η​\(at\+bt\)‖22\+η​Δ​‖at\+bt‖1\\displaystyle\\leq\\\|\\delta\_\{t\}\-\\eta\(a\_\{t\}\+b\_\{t\}\)\\\|\_\{2\}^\{2\}\+\\eta\\Delta\\\|a\_\{t\}\+b\_\{t\}\\\|\_\{1\}≤\(‖δt−η​at‖2\+η​‖bt‖2\)2\+η​Δ​\(‖at‖1\+‖bt‖1\)\\displaystyle\\leq\(\\\|\\delta\_\{t\}\-\\eta a\_\{t\}\\\|\_\{2\}\+\\eta\\\|b\_\{t\}\\\|\_\{2\}\)^\{2\}\+\\eta\\Delta\(\\\|a\_\{t\}\\\|\_\{1\}\+\\\|b\_\{t\}\\\|\_\{1\}\)≤‖δt−η​at‖22\+η​Δ​‖at‖1\+2​η​‖δt−η​at‖2​‖bt‖2\+η2​‖bt‖22\+η​Δ​‖bt‖1\.\\displaystyle\\leq\\\|\\delta\_\{t\}\-\\eta a\_\{t\}\\\|\_\{2\}^\{2\}\+\\eta\\Delta\\\|a\_\{t\}\\\|\_\{1\}\+2\\eta\\\|\\delta\_\{t\}\-\\eta a\_\{t\}\\\|\_\{2\}\\\|b\_\{t\}\\\|\_\{2\}\+\\eta^\{2\}\\\|b\_\{t\}\\\|\_\{2\}^\{2\}\+\\eta\\Delta\\\|b\_\{t\}\\\|\_\{1\}\.We now introduce the following lemma, where the proof is deferred to[Section˜B\.4\.1](https://arxiv.org/html/2606.06934#A2.SS4.SSS1)\.

###### Lemma 34\.

LetΔ,L,M,η\>0\\Delta,L,M,\\eta\>0such thatη<1/M\\eta<1/M\. Suppose that a functionf:ℝd→ℝf:\\mathbb\{R\}^\{d\}\\to\\mathbb\{R\}is convex,LL\-Lipschitz,MM\-smooth, and separable\. Then, for anyx,y∈ℤΔdx,y\\in\\mathbb\{Z\}\_\{\\Delta\}^\{d\}, it holds that

‖x−y−η​\(∇f​\(x\)−∇f​\(y\)\)‖22\+η​Δ​‖∇f​\(x\)−∇f​\(y\)‖1≤‖x−y‖22\.\\displaystyle\\\|x\-y\-\\eta\(\\nabla f\(x\)\-\\nabla f\(y\)\)\\\|\_\{2\}^\{2\}\+\\eta\\Delta\\\|\\nabla f\(x\)\-\\nabla f\(y\)\\\|\_\{1\}\\leq\\\|x\-y\\\|\_\{2\}^\{2\}\.

By[Section˜B\.4](https://arxiv.org/html/2606.06934#A2.SS4), we have‖δt−η​at‖22\+η​Δ​‖at‖1≤‖δt‖22\\\|\\delta\_\{t\}\-\\eta a\_\{t\}\\\|\_\{2\}^\{2\}\+\\eta\\Delta\\\|a\_\{t\}\\\|\_\{1\}\\leq\\\|\\delta\_\{t\}\\\|\_\{2\}^\{2\}\. Furthermore, sinceffis convex andMMsmooth withη<1/M\\eta<1/M,‖δt−η​at‖2≤‖δt‖2\\\|\\delta\_\{t\}\-\\eta a\_\{t\}\\\|\_\{2\}\\leq\\\|\\delta\_\{t\}\\\|\_\{2\}by[Section˜B\.2](https://arxiv.org/html/2606.06934#A2.SS2)\. Moreover, sinceffisLL\-Lipschitz andS≃S′S\\simeq S^\{\\prime\}, it holds that‖bt‖2≤2​L/n\\\|b\_\{t\}\\\|\_\{2\}\\leq 2L/nand hence‖bt‖1≤2​L​d/n\\\|b\_\{t\}\\\|\_\{1\}\\leq 2L\\sqrt\{d\}/n\. Then, we obtain

𝔼​\[‖δt\+1‖22\]\\displaystyle\\mathbb\{E\}\[\\\|\\delta\_\{t\+1\}\\\|\_\{2\}^\{2\}\]≤𝔼​\[‖δt‖22\]\+4​η​Ln​𝔼​\[‖δt‖2\]\+4​η2​L2n2\+2​Δ​η​L​dn\\displaystyle\\leq\\mathbb\{E\}\[\\\|\\delta\_\{t\}\\\|\_\{2\}^\{2\}\]\+\\frac\{4\\eta L\}\{n\}\\mathbb\{E\}\[\\\|\\delta\_\{t\}\\\|\_\{2\}\]\+\\frac\{4\\eta^\{2\}L^\{2\}\}\{n^\{2\}\}\+\\frac\{2\\Delta\\eta L\\sqrt\{d\}\}\{n\}=max⁡\{𝔼​\[‖δt‖22\]\+4​η​Ln​𝔼​\[‖δt‖2\]\+4​η2​L2n2\+2​Δ​η​L​dn,0\}\.\\displaystyle=\\max\\left\\\{\\mathbb\{E\}\[\\\|\\delta\_\{t\}\\\|\_\{2\}^\{2\}\]\+\\frac\{4\\eta L\}\{n\}\\mathbb\{E\}\[\\\|\\delta\_\{t\}\\\|\_\{2\}\]\+\\frac\{4\\eta^\{2\}L^\{2\}\}\{n^\{2\}\}\+\\frac\{2\\Delta\\eta L\\sqrt\{d\}\}\{n\},0\\right\\\}\.We now solve this recursion using[Section˜B\.2](https://arxiv.org/html/2606.06934#A2.SS2)\. To this end, we evaluate the upper bound of𝔼​\[‖δ1‖22\]\\mathbb\{E\}\[\\\|\\delta\_\{1\}\\\|\_\{2\}^\{2\}\]: by[Section˜B\.2](https://arxiv.org/html/2606.06934#A2.SS2), we have

𝔼​\[‖δ1‖22\]=𝔼​\[‖𝖲𝖱​\(η​∇f​\(w0;S\)\)−𝖲𝖱​\(η​∇f​\(w0;S′\)\)‖22\]≤4​η2​L2n2\+2​Δ​η​L​dn\.\\displaystyle\\mathbb\{E\}\[\\\|\\delta\_\{1\}\\\|\_\{2\}^\{2\}\]=\\mathbb\{E\}\[\\\|\\mathsf\{SR\}\(\\eta\\nabla f\(w\_\{0\};S\)\)\-\\mathsf\{SR\}\(\\eta\\nabla f\(w\_\{0\};S^\{\\prime\}\)\)\\\|\_\{2\}^\{2\}\]\\leq\\frac\{4\\eta^\{2\}L^\{2\}\}\{n^\{2\}\}\+\\frac\{2\\Delta\\eta L\\sqrt\{d\}\}\{n\}\.Then, by applying[Section˜B\.2](https://arxiv.org/html/2606.06934#A2.SS2)withα=4​η​L/n\\alpha=4\\eta L/nandγ=ξ=4​η2​L2/n2\+2​Δ​η​L​d/n\\gamma=\\xi=4\\eta^\{2\}L^\{2\}/n^\{2\}\+2\\Delta\\eta L\\sqrt\{d\}/n, we have

𝔼​\[‖δT‖2\]\\displaystyle\\mathbb\{E\}\[\\\|\\delta\_\{T\}\\\|\_\{2\}\]≤4​η​L​\(T−1\)n\+4​η2​L2​Tn2\+2​Δ​η​L​d1/2​Tn\\displaystyle\\leq\\frac\{4\\eta L\(T\-1\)\}\{n\}\+\\sqrt\{\\frac\{4\\eta^\{2\}L^\{2\}T\}\{n^\{2\}\}\+\\frac\{2\\Delta\\eta Ld^\{1/2\}T\}\{n\}\}≤6​η​L​Tn\+2​Δ​η​L​d1/2​Tn\\displaystyle\\leq\\frac\{6\\eta LT\}\{n\}\+\\sqrt\{\\frac\{2\\Delta\\eta Ld^\{1/2\}T\}\{n\}\}which completes the proof for𝖲𝖱​\-GD\{\\mathsf\{SR\}\\text\{\\rm\{\-GD\}\}\}\.

We now show the upper bound for𝖲𝖱​\-SGD\{\\mathsf\{SR\}\\text\{\\rm\{\-SGD\}\}\}\. Letδt′=vt−vt′\\delta^\{\\prime\}\_\{t\}=v\_\{t\}\-v\_\{t\}^\{\\prime\}where\(vt\)t=1T\(v\_\{t\}\)\_\{t=1\}^\{T\}and\(vt′\)t=1T\(v\_\{t\}^\{\\prime\}\)\_\{t=1\}^\{T\}are the coupled𝖲𝖱​\-SGD\{\\mathsf\{SR\}\\text\{\\rm\{\-SGD\}\}\}iterates fromAf,θ𝖲𝖱​\-SGD​\(S\)A\_\{f,\\theta\}^\{\\mathsf\{SR\}\\text\{\\rm\{\-SGD\}\}\}\(S\)andAf,θ𝖲𝖱​\-SGD​\(S′\)A\_\{f,\\theta\}^\{\\mathsf\{SR\}\\text\{\\rm\{\-SGD\}\}\}\(S^\{\\prime\}\), respectively\. Then, we have

𝔼​\[‖δt′‖2\]=\(1−1n\)​𝔼​\[‖δt′‖2\|zit=zit′\]\+1n​𝔼​\[‖δt′‖2\|zit≠zit′\]\.\\displaystyle\\mathbb\{E\}\[\\\|\\delta\_\{t\}^\{\\prime\}\\\|\_\{2\}\]=\\left\(1\-\\frac\{1\}\{n\}\\right\)\\mathbb\{E\}\[\\\|\\delta^\{\\prime\}\_\{t\}\\\|\_\{2\}\|z\_\{i\_\{t\}\}=z\_\{i\_\{t\}\}^\{\\prime\}\]\+\\frac\{1\}\{n\}\\mathbb\{E\}\[\\\|\\delta^\{\\prime\}\_\{t\}\\\|\_\{2\}\|z\_\{i\_\{t\}\}\\neq z\_\{i\_\{t\}\}^\{\\prime\}\]\.\(16\)We now bound each term in[Eq\.˜16](https://arxiv.org/html/2606.06934#A2.E16)using the following lemmas\. The proofs of[Sections˜B\.4](https://arxiv.org/html/2606.06934#A2.SS4)and[B\.4](https://arxiv.org/html/2606.06934#A2.SS4)are in[Sections˜B\.4\.2](https://arxiv.org/html/2606.06934#A2.SS4.SSS2)and[B\.4\.3](https://arxiv.org/html/2606.06934#A2.SS4.SSS3), respectively\.

###### Lemma 35\.

Letd∈ℕd\\in\\mathbb\{N\},η,Δ\>0\\eta,\\Delta\>0andx,y∈ℤΔdx,y\\in\\mathbb\{Z\}\_\{\\Delta\}^\{d\}andf:ℝd→ℝf:\\mathbb\{R\}^\{d\}\\to\\mathbb\{R\}be a convex,MM\-smooth, and separable function andη<1/M\\eta<1/M\. Then, under the coordinate\-wise monotone coupling, it holds that

𝔼​\[‖x−𝖲𝖱​\(η​∇f​\(x\)\)−\(y−𝖲𝖱​\(η​∇f​\(y\)\)\)‖2\]≤‖x−y‖2\\displaystyle\\mathbb\{E\}\[\\\|x\-\\mathsf\{SR\}\(\\eta\\nabla f\(x\)\)\-\(y\-\\mathsf\{SR\}\(\\eta\\nabla f\(y\)\)\)\\\|\_\{2\}\]\\leq\\\|x\-y\\\|\_\{2\}

###### Lemma 36\.

Letd∈ℕd\\in\\mathbb\{N\},Δ,η,M\>0\\Delta,\\eta,M\>0such thatη<1/M\\eta<1/M,f,g:ℝd→ℝf,g:\\mathbb\{R\}^\{d\}\\to\\mathbb\{R\}be convex,LL\-Lipschitz,MM\-smooth, and separable functions\. Then, under the coordinate\-wise monotone coupling, for anyx,y∈ℤΔdx,y\\in\\mathbb\{Z\}\_\{\\Delta\}^\{d\}, it holds that

𝔼​\[‖x−𝖲𝖱​\(η​∇f​\(x\)\)−\(y−𝖲𝖱​\(η​∇g​\(y\)\)\)‖2\]≤‖x−y‖2\+2​η​L\+2​Δ​η​L​d1/2\.\\displaystyle\\mathbb\{E\}\[\\\|x\-\\mathsf\{SR\}\(\\eta\\nabla f\(x\)\)\-\(y\-\\mathsf\{SR\}\(\\eta\\nabla g\(y\)\)\)\\\|\_\{2\}\]\\leq\\\|x\-y\\\|\_\{2\}\+2\\eta L\+\\sqrt\{2\\Delta\\eta Ld^\{1/2\}\}\.

[Section˜B\.4](https://arxiv.org/html/2606.06934#A2.SS4)implies that if the loss function is convex and separable, and the learning rateη\\etais sufficiently small, then𝖲𝖱\\mathsf\{SR\}\-SGD is non\-expansive\. Furthermore,[Section˜B\.4](https://arxiv.org/html/2606.06934#A2.SS4)implies that if the loss function is convex and separable, and the learning rateη\\etais sufficiently small, then the expected distance between𝖲𝖱\\mathsf\{SR\}\-SGD updates is upper bounded by their original distance plus2​η​L\+2​Δ​η​L​d1/22\\eta L\+\\sqrt\{2\\Delta\\eta Ld^\{1/2\}\}\. By applying[Sections˜B\.4](https://arxiv.org/html/2606.06934#A2.SS4)and[B\.4](https://arxiv.org/html/2606.06934#A2.SS4), we have

𝔼​\[‖δt′‖2\]\\displaystyle\\mathbb\{E\}\[\\\|\\delta\_\{t\}^\{\\prime\}\\\|\_\{2\}\]=\(1−1n\)​𝔼​\[‖δt′‖2\|zit=zit′\]\+1n​𝔼​\[‖δt′‖2\|zit≠zit′\]\\displaystyle=\\left\(1\-\\frac\{1\}\{n\}\\right\)\\mathbb\{E\}\[\\\|\\delta\_\{t\}^\{\\prime\}\\\|\_\{2\}\|z\_\{i\_\{t\}\}=z\_\{i\_\{t\}\}^\{\\prime\}\]\+\\frac\{1\}\{n\}\\mathbb\{E\}\[\\\|\\delta\_\{t\}^\{\\prime\}\\\|\_\{2\}\|z\_\{i\_\{t\}\}\\neq z\_\{i\_\{t\}\}^\{\\prime\}\]≤𝔼​\[‖δt−1′‖2\]\+2​η​Ln\+2​Δ​η​L​d1/2n≤2​η​L​Tn\+T​2​Δ​η​L​d1/2n\\displaystyle\\leq\\mathbb\{E\}\[\\\|\\delta\_\{t\-1\}^\{\\prime\}\\\|\_\{2\}\]\+\\frac\{2\\eta L\}\{n\}\+\\frac\{\\sqrt\{2\\Delta\\eta Ld^\{1/2\}\}\}\{n\}\\leq\\frac\{2\\eta LT\}\{n\}\+\\frac\{T\\sqrt\{2\\Delta\\eta Ld^\{1/2\}\}\}\{n\}which completes the proof\.

#### B\.4\.1Proof of[Section˜B\.4](https://arxiv.org/html/2606.06934#A2.SS4)

In this section, we prove[Section˜B\.4](https://arxiv.org/html/2606.06934#A2.SS4)\. For notational convenience, letF​\(x\):=∇f​\(x\)F\(x\):=\\nabla f\(x\)for allx∈ℝdx\\in\\mathbb\{R\}^\{d\}\. By expanding the given expression in[Section˜B\.4](https://arxiv.org/html/2606.06934#A2.SS4), we have

‖x−y‖22\\displaystyle\\\|x\-y\\\|\_\{2\}^\{2\}−η​⟨x−y,F​\(x\)−F​\(y\)⟩\+η2​‖F​\(x\)−F​\(y\)‖22\\displaystyle\-\\eta\\left\\langle\{x\-y\},\{F\(x\)\-F\(y\)\}\\right\\rangle\+\\eta^\{2\}\\\|F\(x\)\-F\(y\)\\\|\_\{2\}^\{2\}−η​⟨x−y,F​\(x\)−F​\(y\)⟩\+η​Δ​‖F​\(x\)−F​\(y\)‖1\.\\displaystyle\-\\eta\\left\\langle\{x\-y\},\{F\(x\)\-F\(y\)\}\\right\\rangle\+\\eta\\Delta\\\|F\(x\)\-F\(y\)\\\|\_\{1\}\.Here, it is easy to verify that−η​⟨x−y,F​\(x\)−F​\(y\)⟩\+η2​‖F​\(x\)−F​\(y\)‖22≤0\-\\eta\\left\\langle\{x\-y\},\{F\(x\)\-F\(y\)\}\\right\\rangle\+\\eta^\{2\}\\\|F\(x\)\-F\(y\)\\\|\_\{2\}^\{2\}\\leq 0since gradient for convex and smooth function is co\-coercive:⟨x−y,F​\(x\)−F​\(y\)⟩≥1/M​‖F​\(x\)−F​\(y\)‖22\\left\\langle\{x\-y\},\{F\(x\)\-F\(y\)\}\\right\\rangle\\geq 1/M\\\|F\(x\)\-F\(y\)\\\|\_\{2\}^\{2\}\. Sinceη<1/M\\eta<1/M, we have

−η​⟨x−y,F​\(x\)−F​\(y\)⟩\+η2​‖F​\(x\)−F​\(y\)‖22≤η​\(η−1M\)​‖F​\(x\)−F​\(y\)‖22≤0\.\\displaystyle\-\\eta\\left\\langle\{x\-y\},\{F\(x\)\-F\(y\)\}\\right\\rangle\+\\eta^\{2\}\\\|F\(x\)\-F\(y\)\\\|\_\{2\}^\{2\}\\leq\\eta\\left\(\\eta\-\\frac\{1\}\{M\}\\right\)\\\|F\(x\)\-F\(y\)\\\|\_\{2\}^\{2\}\\leq 0\.
In the rest of the proof, we show that

−η​⟨x−y,F​\(x\)−F​\(y\)⟩\+η​Δ​‖F​\(x\)−F​\(y\)‖1\\displaystyle\-\\eta\\left\\langle\{x\-y\},\{F\(x\)\-F\(y\)\}\\right\\rangle\+\\eta\\Delta\\\|F\(x\)\-F\(y\)\\\|\_\{1\}\(17\)is less than or equal to0\. Sinceffis separable, there existsf1,⋯,fd:ℝ→ℝf\_\{1\},\\cdots,f\_\{d\}:\\mathbb\{R\}\\to\\mathbb\{R\}such thatf​\(x\)=∑i=1dfi​\(xi\)f\(x\)=\\sum\_\{i=1\}^\{d\}f\_\{i\}\(x\_\{i\}\)for all\(x1,⋯,xd\)∈ℝd\(x\_\{1\},\\cdots,x\_\{d\}\)\\in\\mathbb\{R\}^\{d\}\. Then,[Eq\.˜17](https://arxiv.org/html/2606.06934#A2.E17)can be expressed by

η​∑i=1d\(Δ​\|fi′​\(xi\)−fi′​\(yi\)\|−\(xi−yi\)​\(fi′​\(xi\)−fi′​\(yi\)\)\)\.\\displaystyle\\eta\\sum\_\{i=1\}^\{d\}\(\\Delta\|f\_\{i\}^\{\\prime\}\(x\_\{i\}\)\-f\_\{i\}^\{\\prime\}\(y\_\{i\}\)\|\-\(x\_\{i\}\-y\_\{i\}\)\(f\_\{i\}^\{\\prime\}\(x\_\{i\}\)\-f^\{\\prime\}\_\{i\}\(y\_\{i\}\)\)\)\.Here, we note that sinceffis convex, each univariate functionfif\_\{i\}is convex\. Thus,fi′f\_\{i\}^\{\\prime\}is monotone and, therefore\(xi−yi\)​\(fi′​\(xi\)−fi′​\(yi\)\)≥0\(x\_\{i\}\-y\_\{i\}\)\(f\_\{i\}^\{\\prime\}\(x\_\{i\}\)\-f^\{\\prime\}\_\{i\}\(y\_\{i\}\)\)\\geq 0\. Furthermore, sincexi,yi∈ℤΔx\_\{i\},y\_\{i\}\\in\\mathbb\{Z\}\_\{\\Delta\},\|xi−yi\|≥Δ\|x\_\{i\}\-y\_\{i\}\|\\geq\\Deltaifxi≠yix\_\{i\}\\neq y\_\{i\}\. Hence, we have

\(xi−yi\)​\(fi′​\(xi\)−fi′​\(yi\)\)≥Δ​\|fi′​\(xi\)−fi′​\(yi\)\|\.\\displaystyle\(x\_\{i\}\-y\_\{i\}\)\(f\_\{i\}^\{\\prime\}\(x\_\{i\}\)\-f^\{\\prime\}\_\{i\}\(y\_\{i\}\)\)\\geq\\Delta\|f\_\{i\}^\{\\prime\}\(x\_\{i\}\)\-f\_\{i\}^\{\\prime\}\(y\_\{i\}\)\|\.Conclusively, we have

‖x−y−η​\(∇f​\(x\)−∇f​\(y\)\)‖22\+η​Δ​‖∇f​\(x\)−∇f​\(y\)‖1≤‖x−y‖22\\displaystyle\\\|x\-y\-\\eta\(\\nabla f\(x\)\-\\nabla f\(y\)\)\\\|\_\{2\}^\{2\}\+\\eta\\Delta\\\|\\nabla f\(x\)\-\\nabla f\(y\)\\\|\_\{1\}\\leq\\\|x\-y\\\|\_\{2\}^\{2\}which completes the proof\.

#### B\.4\.2Proof of[Section˜B\.4](https://arxiv.org/html/2606.06934#A2.SS4)

In this section, we prove[Section˜B\.4](https://arxiv.org/html/2606.06934#A2.SS4)\. For notational convenience, letF​\(x\):=∇f​\(x\)F\(x\):=\\nabla f\(x\)for allx∈ℝdx\\in\\mathbb\{R\}^\{d\}\. By[Section˜B\.2](https://arxiv.org/html/2606.06934#A2.SS2), we have

𝔼​\[‖x−y−𝖲𝖱​\(η​F​\(x\)\)\+𝖲𝖱​\(η​F​\(y\)\)‖22\]\\displaystyle\\mathbb\{E\}\[\\\|x\-y\-\\mathsf\{SR\}\(\\eta F\(x\)\)\+\\mathsf\{SR\}\(\\eta F\(y\)\)\\\|\_\{2\}^\{2\}\]≤‖x−y−η​\(F​\(x\)−F​\(y\)\)‖22\+η​Δ​‖F​\(x\)−F​\(y\)‖1\.\\displaystyle\\leq\\\|x\-y\-\\eta\(F\(x\)\-F\(y\)\)\\\|\_\{2\}^\{2\}\+\\eta\\Delta\\\|F\(x\)\-F\(y\)\\\|\_\{1\}\.Then, by[Section˜B\.4](https://arxiv.org/html/2606.06934#A2.SS4), we have

𝔼​\[‖x−y−𝖲𝖱​\(η​F​\(x\)\)\+𝖲𝖱​\(η​F​\(y\)\)‖22\]≤‖x−y‖22\.\\displaystyle\\mathbb\{E\}\[\\\|x\-y\-\\mathsf\{SR\}\(\\eta F\(x\)\)\+\\mathsf\{SR\}\(\\eta F\(y\)\)\\\|\_\{2\}^\{2\}\]\\leq\\\|x\-y\\\|\_\{2\}^\{2\}\.Then, by Jensen’s inequality, we have the desired inequality:

𝔼​\[‖x−𝖲𝖱​\(η​∇f​\(x\)\)−\(y−𝖲𝖱​\(η​∇f​\(y\)\)\)‖2\]\\displaystyle\\mathbb\{E\}\[\\\|x\-\\mathsf\{SR\}\(\\eta\\nabla f\(x\)\)\-\(y\-\\mathsf\{SR\}\(\\eta\\nabla f\(y\)\)\)\\\|\_\{2\}\]≤𝔼​\[‖x−𝖲𝖱​\(η​∇f​\(x\)\)−\(y−𝖲𝖱​\(η​∇f​\(y\)\)\)‖22\]\\displaystyle\\leq\\sqrt\{\\mathbb\{E\}\[\\\|x\-\\mathsf\{SR\}\(\\eta\\nabla f\(x\)\)\-\(y\-\\mathsf\{SR\}\(\\eta\\nabla f\(y\)\)\)\\\|\_\{2\}^\{2\}\]\}≤‖x−y‖2\\displaystyle\\leq\\\|x\-y\\\|\_\{2\}which completes the proof\.

#### B\.4\.3Proof of[Section˜B\.4](https://arxiv.org/html/2606.06934#A2.SS4)

In this section, we prove[Section˜B\.4](https://arxiv.org/html/2606.06934#A2.SS4)\. By the triangle inequality,𝔼​\[‖x−y−\(𝖲𝖱​\(η​∇f​\(x\)\)−𝖲𝖱​\(η​∇g​\(y\)\)\)‖2\]\\mathbb\{E\}\[\\\|x\-y\-\(\\mathsf\{SR\}\(\\eta\\nabla f\(x\)\)\-\\mathsf\{SR\}\(\\eta\\nabla g\(y\)\)\)\\\|\_\{2\}\]is upper bounded by

𝔼​\[‖x−y−\(𝖲𝖱​\(η​∇f​\(x\)\)−𝖲𝖱​\(η​∇f​\(y\)\)\)‖2\]\+𝔼​\[‖𝖲𝖱​\(η​∇f​\(y\)\)−𝖲𝖱​\(η​∇g​\(y\)\)‖2\]\.\\displaystyle\\mathbb\{E\}\[\\\|x\-y\-\(\\mathsf\{SR\}\(\\eta\\nabla f\(x\)\)\-\\mathsf\{SR\}\(\\eta\\nabla f\(y\)\)\)\\\|\_\{2\}\]\+\\mathbb\{E\}\[\\\|\\mathsf\{SR\}\(\\eta\\nabla f\(y\)\)\-\\mathsf\{SR\}\(\\eta\\nabla g\(y\)\)\\\|\_\{2\}\]\.Here, by[Section˜B\.4](https://arxiv.org/html/2606.06934#A2.SS4), we have

𝔼​\[‖x−y−\(𝖲𝖱​\(η​∇f​\(x\)\)−𝖲𝖱​\(η​∇f​\(y\)\)\)‖2\]≤‖x−y‖2\.\\displaystyle\\mathbb\{E\}\[\\\|x\-y\-\(\\mathsf\{SR\}\(\\eta\\nabla f\(x\)\)\-\\mathsf\{SR\}\(\\eta\\nabla f\(y\)\)\)\\\|\_\{2\}\]\\leq\\\|x\-y\\\|\_\{2\}\.Furthermore, it holds that

𝔼​\[‖𝖲𝖱​\(η​∇f​\(y\)\)−𝖲𝖱​\(η​∇g​\(y\)\)‖2\]\\displaystyle\\mathbb\{E\}\[\\\|\\mathsf\{SR\}\(\\eta\\nabla f\(y\)\)\-\\mathsf\{SR\}\(\\eta\\nabla g\(y\)\)\\\|\_\{2\}\]≤𝔼​\[‖𝖲𝖱​\(η​∇f​\(y\)\)−𝖲𝖱​\(η​∇g​\(y\)\)‖22\]\\displaystyle\\leq\\sqrt\{\\mathbb\{E\}\[\\\|\\mathsf\{SR\}\(\\eta\\nabla f\(y\)\)\-\\mathsf\{SR\}\(\\eta\\nabla g\(y\)\)\\\|\_\{2\}^\{2\}\]\}≤‖η​∇f​\(y\)−η​∇g​\(y\)‖22\+Δ​‖η​∇f​\(y\)−η​∇g​\(y\)‖1\\displaystyle\\leq\\sqrt\{\\\|\\eta\\nabla f\(y\)\-\\eta\\nabla g\(y\)\\\|\_\{2\}^\{2\}\+\\Delta\\\|\\eta\\nabla f\(y\)\-\\eta\\nabla g\(y\)\\\|\_\{1\}\}≤‖η​∇f​\(y\)−η​∇g​\(y\)‖2\+Δ​d1/2​‖η​∇f​\(y\)−η​∇g​\(y\)‖2\\displaystyle\\leq\\\|\\eta\\nabla f\(y\)\-\\eta\\nabla g\(y\)\\\|\_\{2\}\+\\sqrt\{\\Delta d^\{1/2\}\\\|\\eta\\nabla f\(y\)\-\\eta\\nabla g\(y\)\\\|\_\{2\}\}≤2​η​L\+2​Δ​η​L​d1/2\\displaystyle\\leq 2\\eta L\+\\sqrt\{2\\Delta\\eta Ld^\{1/2\}\}where the first inequality follows from Jensen’s inequality, the inequality in the second line follows from[Section˜B\.2](https://arxiv.org/html/2606.06934#A2.SS2), the inequality in the third line follows from sub\-additivity of the square root and the fact that‖x‖1≤d​‖x‖2\\\|x\\\|\_\{1\}\\leq\\sqrt\{d\}\\\|x\\\|\_\{2\}for anyx∈ℝdx\\in\\mathbb\{R\}^\{d\}, and the last inequality follows from theLL\-Lipschitzness offfandgg\.

Combining these results, we have

𝔼​\[‖x−𝖲𝖱​\(η​∇f​\(x\)\)−\(y−𝖲𝖱​\(η​∇g​\(y\)\)\)‖2\]≤‖x−y‖2\+2​η​L\+2​Δ​η​L​d1/2\\displaystyle\\mathbb\{E\}\[\\\|x\-\\mathsf\{SR\}\(\\eta\\nabla f\(x\)\)\-\(y\-\\mathsf\{SR\}\(\\eta\\nabla g\(y\)\)\)\\\|\_\{2\}\]\\leq\\\|x\-y\\\|\_\{2\}\+2\\eta L\+\\sqrt\{2\\Delta\\eta Ld^\{1/2\}\}which completes the proof\.

### B\.5Proof of[Theorem˜13](https://arxiv.org/html/2606.06934#Thmtheorem13)

In this section, we prove[Theorem˜13](https://arxiv.org/html/2606.06934#Thmtheorem13)\. To this end, we prove a stronger theorem as stated in the following theorem\.

###### Theorem 37\.

Letη,Δ,L\>0\\eta,\\Delta,L\>0andn,T,d∈ℕn,T,d\\in\\mathbb\{N\}\. There exists anLL\-Lipschitz affine loss functionf:ℝd×\{0,1\}→ℝf:\\mathbb\{R\}^\{d\}\\times\\\{0,1\\\}\\to\\mathbb\{R\}such that forθ=\(η,T,w0=𝟎d\)\\theta=\(\\eta,T,w\_\{0\}=\\boldsymbol\{0\}\_\{d\}\), the following holds: ifη​L/\(n​d\)=m0​Δ\+r0\\eta L/\(n\\sqrt\{d\}\)=m\_\{0\}\\Delta\+r\_\{0\}wherem0∈ℤ≥0m\_\{0\}\\in\\mathbb\{Z\}\_\{\\geq 0\}andr0∈\[0,Δ\)r\_\{0\}\\in\[0,\\Delta\), then,

εu​a​s​\(Af,θ𝖲𝖱​\-GD\)\\displaystyle\\varepsilon\_\{uas\}\(A\_\{f,\\theta\}^\{\\mathsf\{SR\}\\text\{\\rm\{\-GD\}\}\}\)≳η​L​Tn\+min⁡\{r0​T​d,Δ\}​r0​T​d\.\\displaystyle\\gtrsim\\frac\{\\eta LT\}\{n\}\+\\sqrt\{\\min\\\{r\_\{0\}Td,\\Delta\\\}r\_\{0\}Td\}\.Furthermore, ifη​L/d=m1​Δ\+r1\\eta L/\\sqrt\{d\}=m\_\{1\}\\Delta\+r\_\{1\}wherem1∈ℤ≥0m\_\{1\}\\in\\mathbb\{Z\}\_\{\\geq 0\}andr1∈\[0,Δ\)r\_\{1\}\\in\[0,\\Delta\), then

εu​a​s​\(Af,θ𝖲𝖱​\-SGD\)\\displaystyle\\varepsilon\_\{uas\}\(A\_\{f,\\theta\}^\{\\mathsf\{SR\}\\text\{\\rm\{\-SGD\}\}\}\)≳min⁡\{1,Tn\}​\(η​L\+min⁡\{r1​d,Δ\}​r1​d\)\.\\displaystyle\\gtrsim\\min\\left\\\{1,\\frac\{T\}\{n\}\\right\\\}\\left\(\\eta L\+\\sqrt\{\\min\\\{r\_\{1\}d,\\Delta\\\}r\_\{1\}d\}\\right\)\.

[Theorem˜37](https://arxiv.org/html/2606.06934#Thmtheorem37)provides lower bounds of uniform argument stability of𝖲𝖱​\-GD\{\\mathsf\{SR\}\\text\{\\rm\{\-GD\}\}\}and𝖲𝖱​\-SGD\{\\mathsf\{SR\}\\text\{\\rm\{\-SGD\}\}\}for generalη\>0\\eta\>0\. We note that whenη<n​Δ​d/L\\eta<n\\Delta\\sqrt\{d\}/L, thenm0=0m\_\{0\}=0andr0=η​L/\(n​d\)r\_\{0\}=\\eta L/\(n\\sqrt\{d\}\), yielding the bound for𝖲𝖱​\-GD\{\\mathsf\{SR\}\\text\{\\rm\{\-GD\}\}\}in[Theorem˜13](https://arxiv.org/html/2606.06934#Thmtheorem13)\. Likewise, ifη<Δ​d/L\\eta<\\Delta\\sqrt\{d\}/L, thenm1=0m\_\{1\}=0andr1=η​L/dr\_\{1\}=\\eta L/\\sqrt\{d\}, resulting in the bound for𝖲𝖱​\-SGD\{\\mathsf\{SR\}\\text\{\\rm\{\-SGD\}\}\}in[Theorem˜13](https://arxiv.org/html/2606.06934#Thmtheorem13)\.

We now prove[Theorem˜37](https://arxiv.org/html/2606.06934#Thmtheorem37)\. To this end, we consider neighboring datasetsS=\(1,0,⋯,0\)S=\(1,0,\\cdots,0\),S′=\(0,0,⋯,0\)S^\{\\prime\}=\(0,0,\\cdots,0\)and a loss functionf:ℝd×\{0,1\}→ℝf:\\mathbb\{R\}^\{d\}\\times\\\{0,1\\\}\\to\\mathbb\{R\}defined by

f​\(w,z\)=−⟨L​zd​𝟏d,w⟩\.\\displaystyle f\(w,z\)=\-\\left\\langle\{\\frac\{Lz\}\{\\sqrt\{d\}\}\\mathbf\{1\}\_\{d\}\},\{w\}\\right\\rangle\.We note that∇f​\(w;z\)=−\(L​z/d\)​𝟏d\\nabla f\(w;z\)=\-\(Lz/\\sqrt\{d\}\)\\mathbf\{1\}\_\{d\}for allw∈ℝdw\\in\\mathbb\{R\}^\{d\}\. We now show the lower bound of uniform argument stability of𝖲𝖱\\mathsf\{SR\}\-GD and𝖲𝖱\\mathsf\{SR\}\-SGD, respectively\.

We first consider𝖲𝖱\\mathsf\{SR\}\-GD\. Forθ=\(η,T,w0=𝟎d\)\\theta=\(\\eta,T,w\_\{0\}=\\boldsymbol\{0\}\_\{d\}\), let\(wt\)t=1T\(w\_\{t\}\)\_\{t=1\}^\{T\}and\{wt′\}t=1T\\\{w^\{\\prime\}\_\{t\}\\\}\_\{t=1\}^\{T\}be the𝖲𝖱​\-GD\{\\mathsf\{SR\}\\text\{\\rm\{\-GD\}\}\}iterates fromAf,θ𝖲𝖱​\-GD​\(S\)A\_\{f,\\theta\}^\{\\mathsf\{SR\}\\text\{\\rm\{\-GD\}\}\}\(S\)andAf,θ𝖲𝖱​\-GD​\(S′\)A\_\{f,\\theta\}^\{\\mathsf\{SR\}\\text\{\\rm\{\-GD\}\}\}\(S^\{\\prime\}\), respectively\. Obviously,wt′=𝟎dw\_\{t\}^\{\\prime\}=\\boldsymbol\{0\}\_\{d\}almost surely for allt∈\[T\]t\\in\[T\]\. Thus, we have𝔼​\[‖wT−wT′‖2\]=𝔼​\[‖wT‖2\]\\mathbb\{E\}\[\\\|w\_\{T\}\-w\_\{T\}^\{\\prime\}\\\|\_\{2\}\]=\\mathbb\{E\}\[\\\|w\_\{T\}\\\|\_\{2\}\]\. To lower bound this expectation, we introduce the following lemma\. The proof of[Section˜B\.5](https://arxiv.org/html/2606.06934#A2.SS5)is in[Section˜B\.5\.1](https://arxiv.org/html/2606.06934#A2.SS5.SSS1)\.

###### Lemma 38\.

Letd,T∈ℕd,T\\in\\mathbb\{N\},c,Δ\>0c,\\Delta\>0such thatc=m​Δ\+rc=m\\Delta\+rfor somem∈ℤ≥0m\\in\\mathbb\{Z\}\_\{\\geq 0\}andr∈\[0,Δ\)r\\in\[0,\\Delta\)\. Suppose that the random vectorsx0,⋯,xT∈ℤΔdx\_\{0\},\\cdots,x\_\{T\}\\in\\mathbb\{Z\}\_\{\\Delta\}^\{d\}are recursively defined as follows:

x0=𝟎d,xt=xt−1\+𝖲𝖱​\(c​𝟏d\)\\displaystyle x\_\{0\}=\\boldsymbol\{0\}\_\{d\},\\quad x\_\{t\}=x\_\{t\-1\}\+\\mathsf\{SR\}\(c\\mathbf\{1\}\_\{d\}\)for eacht∈\[T\]t\\in\[T\]\. Then, it holds that for anyt∈\[T\]t\\in\[T\],

𝔼​\[‖xt‖2\]=Ω​\(c​t​d\+min⁡\{r​t​d​Δ,r​t​d\}\)\.\\displaystyle\\mathbb\{E\}\[\\\|x\_\{t\}\\\|\_\{2\}\]=\\Omega\\left\(ct\\sqrt\{d\}\+\\min\\\{\\sqrt\{rtd\\Delta\},rtd\\\}\\right\)\.

[Section˜B\.5](https://arxiv.org/html/2606.06934#A2.SS5)provides the lower bound of expected22\-norm of𝖲𝖱\\mathsf\{SR\}\-GD iterates initialized at the origin, where the loss function is linear and has a gradient in a diagonal direction\. Since∇f​\(w;S\)=−L/\(n​d\)​𝟏d\\nabla f\(w;S\)=\-L/\(n\\sqrt\{d\}\)\\boldsymbol\{1\}\_\{d\}for allw∈ℝdw\\in\\mathbb\{R\}^\{d\}, we have the update rule forwtw\_\{t\}: for allt∈\[T\]t\\in\[T\],

wt=wt−1−𝖲𝖱​\(−η​Ln​d​𝟏d\)\.\\displaystyle w\_\{t\}=w\_\{t\-1\}\-\\mathsf\{SR\}\\left\(\-\\frac\{\\eta L\}\{n\\sqrt\{d\}\}\\boldsymbol\{1\}\_\{d\}\\right\)\.Here, we note that for anyx∈ℝdx\\in\\mathbb\{R\}^\{d\},𝖲𝖱​\(x\)\\mathsf\{SR\}\(x\)and−𝖲𝖱​\(−x\)\-\\mathsf\{SR\}\(\-x\)have the same distribution\. Therefore, if we define\(w~t\)t=1T\(\\tilde\{w\}\_\{t\}\)\_\{t=1\}^\{T\}such thatw~0=w0\\tilde\{w\}\_\{0\}=w\_\{0\}and

w~t=w~t−1\+𝖲𝖱​\(η​Ln​d​𝟏d\),\\displaystyle\\tilde\{w\}\_\{t\}=\\tilde\{w\}\_\{t\-1\}\+\\mathsf\{SR\}\\left\(\\frac\{\\eta L\}\{n\\sqrt\{d\}\}\\boldsymbol\{1\}\_\{d\}\\right\),then𝔼​\[‖wT‖2\]=𝔼​\[‖w~T‖2\]\\mathbb\{E\}\[\\\|w\_\{T\}\\\|\_\{2\}\]=\\mathbb\{E\}\[\\\|\\tilde\{w\}\_\{T\}\\\|\_\{2\}\]\. Then, by applying[Section˜B\.5](https://arxiv.org/html/2606.06934#A2.SS5), we have

𝔼​\[‖wT‖2\]=𝔼​\[‖w~T‖2\]=Ω​\(η​L​Tn\+min⁡\{Δ,r0​T​d\}​r0​T​d\)\\displaystyle\\mathbb\{E\}\[\\\|w\_\{T\}\\\|\_\{2\}\]=\\mathbb\{E\}\[\\\|\\tilde\{w\}\_\{T\}\\\|\_\{2\}\]=\\Omega\\left\(\\frac\{\\eta LT\}\{n\}\+\\sqrt\{\\min\\left\\\{\\Delta,r\_\{0\}Td\\right\\\}r\_\{0\}Td\}\\right\)which completes the proof for𝖲𝖱\\mathsf\{SR\}\-GD\.

We now consider𝖲𝖱\\mathsf\{SR\}\-SGD\. Forθ=\(η,T,w0=𝟎d\)\\theta=\(\\eta,T,w\_\{0\}=\\boldsymbol\{0\}\_\{d\}\), let\(vt\)t=1T\(v\_\{t\}\)\_\{t=1\}^\{T\}and\(vt′\)t=1T\(v\_\{t\}^\{\\prime\}\)\_\{t=1\}^\{T\}be coupled𝖲𝖱​\-SGD\{\\mathsf\{SR\}\\text\{\\rm\{\-SGD\}\}\}iterates fromAf,θ𝖲𝖱​\-SGD​\(S\)A\_\{f,\\theta\}^\{\\mathsf\{SR\}\\text\{\\rm\{\-SGD\}\}\}\(S\)andAf,θ𝖲𝖱​\-SGD​\(S′\)A\_\{f,\\theta\}^\{\\mathsf\{SR\}\\text\{\\rm\{\-SGD\}\}\}\(S^\{\\prime\}\)\. Obviously,vt′=𝟎dv\_\{t\}^\{\\prime\}=\\boldsymbol\{0\}\_\{d\}almost surely for allt∈\[T\]t\\in\[T\]\. Then, we have𝔼​\[‖vT−vT′‖2\]=𝔼​\[‖vT‖2\]\\mathbb\{E\}\[\\\|v\_\{T\}\-v\_\{T\}^\{\\prime\}\\\|\_\{2\}\]=\\mathbb\{E\}\[\\\|v\_\{T\}\\\|\_\{2\}\]\.

LetK=\|\{t∈\[T\]:zit=1\}\|K=\|\\\{t\\in\[T\]:z\_\{i\_\{t\}\}=1\\\}\|\. Then, we haveK∼B​i​n​\(T,1/n\)K\\sim Bin\(T,1/n\)\. We note that ifzit=0z\_\{i\_\{t\}\}=0is observed, thenvt=vt−1v\_\{t\}=v\_\{t\-1\}almost surely\. Furthermore, ifzit=1z\_\{i\_\{t\}\}=1is observed, then we havevt=vt−1−𝖲𝖱​\(−\(η​L/d\)​𝟏d\)v\_\{t\}=v\_\{t\-1\}\-\\mathsf\{SR\}\(\-\(\\eta L/\\sqrt\{d\}\)\\mathbf\{1\}\_\{d\}\)\. Then, it holds that𝔼​\[‖vT‖2\]=ℙ​\(K≥1\)​𝔼​\[‖vT‖2\|K≥1\]\\mathbb\{E\}\[\\\|v\_\{T\}\\\|\_\{2\}\]=\\mathbb\{P\}\(K\\geq 1\)\\mathbb\{E\}\[\\\|v\_\{T\}\\\|\_\{2\}\|K\\geq 1\]; ifk=0k=0, i\.e\.,zi1=⋯=ziT=0z\_\{i\_\{1\}\}=\\cdots=z\_\{i\_\{T\}\}=0, then we obtainvT=0v\_\{T\}=0\. We now show the lower bound onℙ​\(K≥1\)\\mathbb\{P\}\(K\\geq 1\)and𝔼​\[‖vT‖2\|K≥1\]\\mathbb\{E\}\[\\\|v\_\{T\}\\\|\_\{2\}\|K\\geq 1\]\.

The lower bound ofℙ​\(K≥1\)\\mathbb\{P\}\(K\\geq 1\)is obtained by

ℙ​\(K≥1\)=1−\(1−1n\)T=Ω​\(min⁡\{1,Tn\}\)\.\\displaystyle\\mathbb\{P\}\(K\\geq 1\)=1\-\\left\(1\-\\frac\{1\}\{n\}\\right\)^\{T\}=\\Omega\\left\(\\min\\left\\\{1,\\frac\{T\}\{n\}\\right\\\}\\right\)\.\(18\)
We now consider𝔼​\[‖vT‖2\|K≥1\]\\mathbb\{E\}\[\\\|v\_\{T\}\\\|\_\{2\}\|K\\geq 1\]\. GivenK=k≥1K=k\\geq 1, let1≤τ1<⋯<τk≤T1\\leq\\tau\_\{1\}<\\cdots<\\tau\_\{k\}\\leq Tbe the iteration times whereiτj=1i\_\{\\tau\_\{j\}\}=1\. We consider a sequence\(v~t\)t=1k\(\\tilde\{v\}\_\{t\}\)\_\{t=1\}^\{k\}such thatv~0=v0\\tilde\{v\}\_\{0\}=v\_\{0\}andv~j=v~j−1\+𝖲𝖱​\(\(η​L/d\)​𝟏d\)\\tilde\{v\}\_\{j\}=\\tilde\{v\}\_\{j\-1\}\+\\mathsf\{SR\}\(\(\\eta L/\\sqrt\{d\}\)\\boldsymbol\{1\}\_\{d\}\)for allj∈\[k\]j\\in\[k\]\. Then,v~j\\tilde\{v\}\_\{j\}andvτjv\_\{\\tau\_\{j\}\}have the same distribution for allj∈\[k\]j\\in\[k\]\. Then, by[Section˜B\.5](https://arxiv.org/html/2606.06934#A2.SS5), we have

𝔼​\[‖vT‖2\|K=k\]=𝔼​\[‖v~T‖2\|K=k\]\\displaystyle\\mathbb\{E\}\[\\\|v\_\{T\}\\\|\_\{2\}\|K=k\]=\\mathbb\{E\}\[\\\|\\tilde\{v\}\_\{T\}\\\|\_\{2\}\|K=k\]=Ω​\(η​L​k\+min⁡\{r1​k​d​Δ,r1​k​d\}\)\\displaystyle=\\Omega\\left\(\\eta Lk\+\\min\\left\\\{\\sqrt\{r\_\{1\}kd\\Delta\},r\_\{1\}kd\\right\\\}\\right\)Since the right\-hand side does not decrease inkk, on the eventK≥1K\\geq 1, it has the minimum value atk=1k=1\. By combining this result with[Eq\.˜18](https://arxiv.org/html/2606.06934#A2.E18), we have

𝔼​\[‖vT‖2\]=ℙ​\(K≥1\)​𝔼​\[‖vT‖2\|K≥1\]≳min⁡\{1,Tn\}​\(η​L\+min⁡\{Δ,r1​d\}​r1​d\)\.\\displaystyle\\mathbb\{E\}\[\\\|v\_\{T\}\\\|\_\{2\}\]=\\mathbb\{P\}\(K\\geq 1\)\\mathbb\{E\}\[\\\|v\_\{T\}\\\|\_\{2\}\|K\\geq 1\]\\gtrsim\\min\\left\\\{1,\\frac\{T\}\{n\}\\right\\\}\\left\(\\eta L\+\\sqrt\{\\min\\\{\\Delta,r\_\{1\}d\\\}r\_\{1\}d\}\\right\)\.which completes the proof\.

#### B\.5\.1Proof of[Section˜B\.5](https://arxiv.org/html/2606.06934#A2.SS5)

In this section, we prove[Section˜B\.5](https://arxiv.org/html/2606.06934#A2.SS5)\. Since stochastic rounding is applied to any vector coordinate\-wise independently,xtx\_\{t\}is updated symmetrically with respect to coordinates\. Letyty\_\{t\}be a first coordinate ofxtx\_\{t\}for allt∈\[T\]t\\in\[T\]\.

We now show that for allt∈\[T\]t\\in\[T\],yt=m​Δ​t\+Δ​Xty\_\{t\}=m\\Delta t\+\\Delta X\_\{t\}whereXt∼B​i​n​\(t,r/Δ\)X\_\{t\}\\sim Bin\(t,r/\\Delta\)via mathematical induction ont∈\[T\]t\\in\[T\]\.

We first consider the base case\. Sincey0=0y\_\{0\}=0, we havey1=y0\+𝖲𝖱​\(c\)=𝖲𝖱​\(m​Δ\+r\)=m​Δ\+𝖲𝖱​\(r\)y\_\{1\}=y\_\{0\}\+\\mathsf\{SR\}\(c\)=\\mathsf\{SR\}\(m\\Delta\+r\)=m\\Delta\+\\mathsf\{SR\}\(r\)\. Sincer∈\[0,Δ\)r\\in\[0,\\Delta\), we have

𝖲𝖱​\(r\)=\{0w\.p​1−r/Δ,Δw\.pr/Δ\.\\displaystyle\\mathsf\{SR\}\(r\)=\\begin\{cases\}0&w\.p\\ \\ 1\-r/\\Delta,\\\\ \\Delta&w\.p\\ \\ r/\\Delta\.\\end\{cases\}Thus,y1=m​Δ\+Δ​X1y\_\{1\}=m\\Delta\+\\Delta X\_\{1\}whereX1∼B​e​r​\(r/Δ\)=B​i​n​\(1,r/Δ\)X\_\{1\}\\sim Ber\(r/\\Delta\)=Bin\(1,r/\\Delta\)\.

We now consider the general case\. To this end, assume thatyt=m​Δ​t\+Δ​Xty\_\{t\}=m\\Delta t\+\\Delta X\_\{t\}\. Since𝖲𝖱​\(c\)=m​Δ\+Δ​Bt\+1\\mathsf\{SR\}\(c\)=m\\Delta\+\\Delta B\_\{t\+1\}whereBt\+1∼B​e​r​\(r/Δ\)B\_\{t\+1\}\\sim Ber\(r/\\Delta\), we have the following from the update rule ofyty\_\{t\}:

yt\+1=yt\+𝖲𝖱​\(c\)\\displaystyle y\_\{t\+1\}=y\_\{t\}\+\\mathsf\{SR\}\(c\)=m​Δ​t\+Δ​Xt\+m​Δ\+Δ​Bt\+1\\displaystyle=m\\Delta t\+\\Delta X\_\{t\}\+m\\Delta\+\\Delta B\_\{t\+1\}=m​Δ​\(t\+1\)\+Δ​\(Xt\+Bt\+1\)\.\\displaystyle=m\\Delta\(t\+1\)\+\\Delta\(X\_\{t\}\+B\_\{t\+1\}\)\.Here, sinceXt∼B​i​n​\(t,r/Δ\)X\_\{t\}\\sim Bin\(t,r/\\Delta\)andBt\+1∼B​e​r​\(r/Δ\)B\_\{t\+1\}\\sim Ber\(r/\\Delta\)are independent, we haveXt\+Bt\+1≜Xt\+1∼B​i​n​\(t\+1,r/Δ\)X\_\{t\}\+B\_\{t\+1\}\\triangleq X\_\{t\+1\}\\sim Bin\(t\+1,r/\\Delta\)\. This completes the mathematical induction\.

We now show the lower bound in[Section˜B\.5](https://arxiv.org/html/2606.06934#A2.SS5)\. Here, ifr=0r=0, then the stochastic term is zero, yielding𝔼​\[‖xt‖2\]=m​Δ​t​d=c​t​d\\mathbb\{E\}\[\\\|x\_\{t\}\\\|\_\{2\}\]=m\\Delta t\\sqrt\{d\}=ct\\sqrt\{d\}\. We now assume thatr∈\(0,Δ\)r\\in\(0,\\Delta\)\. To this end, we introduce the following technical lemma\. The proof of[Section˜B\.5\.1](https://arxiv.org/html/2606.06934#A2.SS5.SSS1)is in[Section˜B\.5\.2](https://arxiv.org/html/2606.06934#A2.SS5.SSS2)

###### Lemma 39\.

Letn,d∈ℕn,d\\in\\mathbb\{N\},p∈\(0,1\)p\\in\(0,1\)andX1,⋯,Xd∼B​i​n​\(n,p\)X\_\{1\},\\cdots,X\_\{d\}\\sim Bin\(n,p\)be independent\. Then, we have

𝔼​\[∑i=1dXi2\]=Ω​\(n​p​d\+min⁡\{d​n​p,d​n​p\}\)\.\\displaystyle\\mathbb\{E\}\\left\[\\sqrt\{\\sum\_\{i=1\}^\{d\}X\_\{i\}^\{2\}\}\\right\]=\\Omega\(np\\sqrt\{d\}\+\\min\\\{\\sqrt\{dnp\},dnp\\\}\)\.

By directly computing the expectation of the22\-norm ofxtx\_\{t\}, we have

𝔼​\[‖xt‖2\]=𝔼​\[∑i=1dxt,i2\]\\displaystyle\\mathbb\{E\}\[\\\|x\_\{t\}\\\|\_\{2\}\]=\\mathbb\{E\}\\left\[\\sqrt\{\\sum\_\{i=1\}^\{d\}x\_\{t,i\}^\{2\}\}\\right\]=𝔼​\[∑i=1d\(m​Δ​t\+Δ​Xt,i\)2\]\\displaystyle=\\mathbb\{E\}\\left\[\\sqrt\{\\sum\_\{i=1\}^\{d\}\(m\\Delta t\+\\Delta X\_\{t,i\}\)^\{2\}\}\\right\]≥𝔼​\[∑i=1d\(m​Δ​t\)2\+\(Δ​Xt,i\)2\]\\displaystyle\\geq\\mathbb\{E\}\\left\[\\sqrt\{\\sum\_\{i=1\}^\{d\}\(m\\Delta t\)^\{2\}\+\(\\Delta X\_\{t,i\}\)^\{2\}\}\\right\]≳m​d​Δ​t\+Δ​𝔼​\[∑i=1dXt,i2\]\\displaystyle\\gtrsim m\\sqrt\{d\}\\Delta t\+\\Delta\\mathbb\{E\}\\left\[\\sqrt\{\\sum\_\{i=1\}^\{d\}X\_\{t,i\}^\{2\}\}\\right\]≳m​d​Δ​t\+Δ​\(r​t​dΔ\+min⁡\{r​t​dΔ,r​t​dΔ\}\)\\displaystyle\\gtrsim m\\sqrt\{d\}\\Delta t\+\\Delta\\left\(\\frac\{rt\\sqrt\{d\}\}\{\\Delta\}\+\\min\\left\\\{\\sqrt\{\\frac\{rtd\}\{\\Delta\}\},\\frac\{rtd\}\{\\Delta\}\\right\\\}\\right\)=\(m​Δ\+r\)​t​d\+min⁡\{r​t​d​Δ,r​t​d\}\\displaystyle=\(m\\Delta\+r\)t\\sqrt\{d\}\+\\min\\\{\\sqrt\{rtd\\Delta\},rtd\\\}=c​t​d\+min⁡\{r​t​d​Δ,r​t​d\}\\displaystyle=ct\\sqrt\{d\}\+\\min\\\{\\sqrt\{rtd\\Delta\},rtd\\\}where the inequality in the second line follows from\(a\+b\)2≥a2\+b2\(a\+b\)^\{2\}\\geq a^\{2\}\+b^\{2\}ifa,b≥0a,b\\geq 0, the asymptotic relation in the third line follows froma\+b≥\(a\+b\)/2\\sqrt\{a\+b\}\\geq\(\\sqrt\{a\}\+\\sqrt\{b\}\)/\\sqrt\{2\}, the asymptotic relation in the fourth line follows from[Section˜B\.5\.1](https://arxiv.org/html/2606.06934#A2.SS5.SSS1)\. This completes the proof\.

#### B\.5\.2Proof of[Section˜B\.5\.1](https://arxiv.org/html/2606.06934#A2.SS5.SSS1)

In this section, we prove[Section˜B\.5\.1](https://arxiv.org/html/2606.06934#A2.SS5.SSS1)\. LetY1=∑i=1dXiY\_\{1\}=\\sum\_\{i=1\}^\{d\}X\_\{i\}andY2=∑i=1dXi2Y\_\{2\}=\\sum\_\{i=1\}^\{d\}X\_\{i\}^\{2\}\. SinceX1,⋯,Xd∼B​i​n​\(n,p\)X\_\{1\},\\cdots,X\_\{d\}\\sim Bin\(n,p\)are independent, it holds thatY1∼B​i​n​\(d​n,p\)Y\_\{1\}\\sim Bin\(dn,p\)\. We now derive the lower bound of

𝔼​\[∑i=1dXi2\]=𝔼​\[Y2\]\\displaystyle\\mathbb\{E\}\\left\[\\sqrt\{\\sum\_\{i=1\}^\{d\}X\_\{i\}^\{2\}\}\\right\]=\\mathbb\{E\}\[\\sqrt\{Y\_\{2\}\}\]by showing that𝔼​\[Y2\]=Ω​\(n​p​d\)\\mathbb\{E\}\[\\sqrt\{Y\_\{2\}\}\]=\\Omega\(np\\sqrt\{d\}\)and𝔼​\[Y2\]=Ω​\(min⁡\{d​n​p,d​n​p\}\)\\mathbb\{E\}\[\\sqrt\{Y\_\{2\}\}\]=\\Omega\\left\(\\min\\\{\\sqrt\{dnp\},dnp\\\}\\right\), respectively\.

Firstly, by the Cauchy\-Schwarz inequality, we have

𝔼​\[Y2\]≥1d​𝔼​\[Y1\]=d​n​pd=n​p​d\.\\displaystyle\\mathbb\{E\}\[\\sqrt\{Y\_\{2\}\}\]\\geq\\frac\{1\}\{\\sqrt\{d\}\}\\mathbb\{E\}\[Y\_\{1\}\]=\\frac\{dnp\}\{\\sqrt\{d\}\}=np\\sqrt\{d\}\.
We now show that𝔼​\[Y2\]=Ω​\(min⁡\{d​n​p,d​n​p\}\)\\mathbb\{E\}\[\\sqrt\{Y\_\{2\}\}\]=\\Omega\\left\(\\min\\\{\\sqrt\{dnp\},dnp\\\}\\right\)\. Since eachXiX\_\{i\}takes nonnegative integer values,𝔼​\[Y2\]≥𝔼​\[Y1\]\\mathbb\{E\}\[\\sqrt\{Y\_\{2\}\}\]\\geq\\mathbb\{E\}\[\\sqrt\{Y\_\{1\}\}\]follows\.

SinceY1∼B​i​n​\(d​n,p\)Y\_\{1\}\\sim Bin\(dn,p\), by applying[Section˜B\.1\.3](https://arxiv.org/html/2606.06934#A2.SS1.SSS3), we have

𝔼​\[Y1\]=Ω​\(min⁡\{𝔼​\[Y1\],𝔼​\[Y1\]\}\)=Ω​\(min⁡\{d​n​p,d​n​p\}\),\\displaystyle\\mathbb\{E\}\[\\sqrt\{Y\_\{1\}\}\]=\\Omega\\left\(\\min\\left\\\{\\mathbb\{E\}\[Y\_\{1\}\],\\sqrt\{\\mathbb\{E\}\[Y\_\{1\}\]\}\\right\\\}\\right\)=\\Omega\(\\min\\\{dnp,\\sqrt\{dnp\}\\\}\),which completes the proof\.