何处、何时以及多少比特？采用随机舍入的 SGD 的缩放定律

# Where, When, and How Many Bits? Scaling Laws for SGD with Stochastic Rounding
Source: [https://arxiv.org/html/2610.41074](https://arxiv.org/html/2610.41074)
Zhichao WangICSI and Department of StatisticsUniversity of California, Berkeley, USAzhichao\.wang@berkeley\.eduMichael W\. MahoneyICSI, LBNL and Department of StatisticsUniversity of California, Berkeley, USAmmahoney@stat\.berkeley\.edu

###### Abstract

Low\-precision training reduces computation but introduces additional rounding error\. We study how stochastic rounding \(SR\) affects prediction risk as model size, training time and precision change\. We consider a two\-layer linear network with a fixed random first layer, trained by mini\-batch SGD on fresh Gaussian data with power\-law spectra\. At each SGD step, the features used in forward and backward computations are rounded independently\. We prove matched upper and lower bounds on prediction error for the final model and for uniform parameter averaging, with general learning\-rate and precision schedules chosen before training\. For fixed precisions and constant learning rates with parameter averaging, we derive how the smallest prediction error decreases with compute and identify model sizes and training times that achieve this optimal\-compute scaling\. Rounding only the forward features preserves the compute scaling of exact arithmetic, while rounding backward features changes it\. Averaging parameters can change which computation needs higher precision and when higher precision is useful\. The laws also give bit requirements for a given accuracy tolerance and show how backward rounding changes learning\-rate schedule design\. At a fixed compute budget, lower precision can improve prediction when the benefit of a larger model or longer training outweighs the additional SR\-induced error\. Model simulations illustrate these tradeoffs, and experiments with an MLP and a small transformer examine rounding choices beyond the theoretical model\.

## 1Introduction

Training large machine learning models in low precision can reduce memory usage and computational costs\. For example, mixed\-precision training computes matrix multiplications of the forward and backward passes in 16\-bit arithmetic while keeping a high\-precision copy of the weights\([Micikevicius et al\., 2018](https://arxiv.org/html/2610.41074#bib.bib36)\)\. Recent LLMs are even pretrained with 8\-bit and 4\-bit floating\-point matrix products\([Micikevicius et al\., 2022](https://arxiv.org/html/2610.41074#bib.bib37);[DeepSeek\-AI, 2024](https://arxiv.org/html/2610.41074#bib.bib38);[Tseng et al\., 2025](https://arxiv.org/html/2610.41074#bib.bib22);[Castro et al\., 2025](https://arxiv.org/html/2610.41074#bib.bib3);[Chmiel et al\., 2025](https://arxiv.org/html/2610.41074#bib.bib6);[Panferov et al\., 2026](https://arxiv.org/html/2610.41074#bib.bib19)\)\. Meanwhile, quantization\-aware training \(QAT\) simulates low\-precision weights and activations during training, making the trained model run in low precision at inference\([Jacob et al\., 2018](https://arxiv.org/html/2610.41074#bib.bib35);[Esser et al\., 2020](https://arxiv.org/html/2610.41074#bib.bib39);[Liu et al\., 2024](https://arxiv.org/html/2610.41074#bib.bib40)\)\. A third setting saves training memory by quantizing the activations saved for backpropagation, while leaving the forward pass unchanged\([Chen et al\., 2021](https://arxiv.org/html/2610.41074#bib.bib63);[Liu et al\., 2022](https://arxiv.org/html/2610.41074#bib.bib16)\)\. Separately,[Chen et al\. \(2020\)](https://arxiv.org/html/2610.41074#bib.bib62)show extra gradient variance caused by rounding in fully quantized training, where weights, activations and gradients are quantized\.

In this paper, we focus on the first setting, in which the arithmetic of training itself is rounded\. Stochastic rounding \(SR\) rounds a number to one of its neighboring representable values with probabilities that preserve the input value in expectation\. This unbiasedness can avoid update stagnation at precisions where round\-to\-nearest stalls\([Gupta et al\., 2015](https://arxiv.org/html/2610.41074#bib.bib41);[Zhang et al\., 2017](https://arxiv.org/html/2610.41074#bib.bib28);[De Sa et al\., 2018](https://arxiv.org/html/2610.41074#bib.bib9);[Zhao et al\., 2025](https://arxiv.org/html/2610.41074#bib.bib51);[Ozkara et al\., 2025](https://arxiv.org/html/2610.41074#bib.bib18)\)\. Existing SR theory provides probabilistic bounds on rounding errors in numerical computations\([Higham and Mary, 2019](https://arxiv.org/html/2610.41074#bib.bib52);[Higham and Mary, 2020](https://arxiv.org/html/2610.41074#bib.bib53);[Connolly et al\., 2021](https://arxiv.org/html/2610.41074#bib.bib7);[Croci et al\., 2022](https://arxiv.org/html/2610.41074#bib.bib8);[El Arar et al\., 2023](https://arxiv.org/html/2610.41074#bib.bib11);[Drineas and Ipsen, 2024](https://arxiv.org/html/2610.41074#bib.bib54);[El Arar et al\., 2026](https://arxiv.org/html/2610.41074#bib.bib29)\)and convergence guarantees for low\-precision SGD\([Li et al\., 2017](https://arxiv.org/html/2610.41074#bib.bib42);[De Sa et al\., 2018](https://arxiv.org/html/2610.41074#bib.bib9);[Ozkara et al\., 2025](https://arxiv.org/html/2610.41074#bib.bib18)\)\. Its benefit on small singular values has also been studied\([Boutsikas et al\., 2024](https://arxiv.org/html/2610.41074#bib.bib55);[Dexter et al\., 2025](https://arxiv.org/html/2610.41074#bib.bib56);[Ma et al\., 2026](https://arxiv.org/html/2610.41074#bib.bib57)\)\. These results do not further describe how prediction risk depends jointly on model width, training time and precision, or how to allocate precision bits across operations and training steps\. So we aim to answer the following question in this paper:

*Where, when, and with how many bits should we use stochastic rounding during training?*

To answer this question, we prove a neural scaling law, which describes how loss asymptotically decreases with model size, training time and compute\([Kaplan et al\., 2020](https://arxiv.org/html/2610.41074#bib.bib43);[Hoffmann et al\., 2022](https://arxiv.org/html/2610.41074#bib.bib44);[Subramanian et al\., 2026](https://arxiv.org/html/2610.41074#bib.bib64)\), This law practically guides the allocation of compute resources during training\. Recent empirical work extends these laws to low\-precision training and model quantization\([Kumar et al\., 2025](https://arxiv.org/html/2610.41074#bib.bib12);[Sun et al\., 2025](https://arxiv.org/html/2610.41074#bib.bib33);[Frantar et al\., 2025](https://arxiv.org/html/2610.41074#bib.bib30);[Chen et al\., 2026a](https://arxiv.org/html/2610.41074#bib.bib58)\)\. Typically,[Kumar et al\. \(2025\)](https://arxiv.org/html/2610.41074#bib.bib12);[Sun et al\. \(2025\)](https://arxiv.org/html/2610.41074#bib.bib33)empirically use fitted scaling laws to choose model size, training data and precision for a given compute budget\. These findings motivate a theoretical study of precision\-dependent loss scaling and quantitative analyses of training dynamics with quantization\([Zhang et al\., 2026a](https://arxiv.org/html/2610.41074#bib.bib26);[Zhang et al\., 2026b](https://arxiv.org/html/2610.41074#bib.bib27)\)\.

In the exact\-arithmetic setting, theory of neural scaling law has been extensively studied\([Bahri et al\., 2024](https://arxiv.org/html/2610.41074#bib.bib45);[Maloney et al\., 2022](https://arxiv.org/html/2610.41074#bib.bib46);[Bordelon et al\., 2024](https://arxiv.org/html/2610.41074#bib.bib47);[Paquette et al\., 2024](https://arxiv.org/html/2610.41074#bib.bib21);[Paquette et al\., 2025](https://arxiv.org/html/2610.41074#bib.bib20);[Lin et al\., 2024](https://arxiv.org/html/2610.41074#bib.bib14);[Lin et al\., 2025](https://arxiv.org/html/2610.41074#bib.bib15);[Bordelon and Mori, 2026](https://arxiv.org/html/2610.41074#bib.bib2);[Chen et al\., 2026b](https://arxiv.org/html/2610.41074#bib.bib4)\)\. Specifically,[Paquette et al\. \(2024\)](https://arxiv.org/html/2610.41074#bib.bib21)gives the baseline compute optimal scaling law\. We follow these results and study a two\-layer linear network with a fixed random first layer and train its output layer using online SGD with SR\. We consider both the final model and a model formed by averaging parameters during training\. And the features used in the forward and backward computations are rounded independently, possibly at different precisions over the training\.[Kim et al\. \(2026\)](https://arxiv.org/html/2610.41074#bib.bib13)concurrently fit separate forward/backward precision effects and hardware\-aware allocations, supported by a convex\-optimization upper bound\. We theoretically derive binary\-SR risk scaling laws and optimal compute allocations in low precision\.

Figure 1:\(a\) We round the features either at forward pass or backward pass in SGD dynamics \([5](https://arxiv.org/html/2610.41074#S2.E5)\) with one\-bit precision and compare the SR\-induced error over the corresponding risk without rounding\. Solid lines evaluate the final model \(last iterate\); dashed lines evaluate the model obtained by averaging parameters over training \(uniform average\)\. Forward rounding causes more error for last iterate, while backward causes more error for the average\. \(b\) Using three bits in the first half of training and one bit in the second, or the reverse, with nearly equal average precision\. Each bar divides the SR\-induced risk with higher precision early by that with higher precision late\. Values above one indicate higher precision at late phase of training outperforms the case with higher precision at early phase of training\. \(c\) Each point selects the smallest tested precision that keeps the final prediction risk no more than1%1\\%higher than its value without rounding, at the same width and duration\. Both locations use the same precision from 1 to 4 bits\. Appendix[F](https://arxiv.org/html/2610.41074#A6)gives experimental details\.##### Summary of main contributions\.

- •Neural scaling laws and compute scaling laws\(Section[3](https://arxiv.org/html/2610.41074#S3)\)\. We prove general neural scaling laws, showing how prediction error changes with model size, training time and SR precision\. For fixed precisions and constant learning rates with parameter averaging, we also derive how the smallest prediction error decreases with compute and which model sizes and training times achieve this scaling under our compute model\.
- •Where, when and how much to round\(Sections[4\.1](https://arxiv.org/html/2610.41074#S4.SS1)–[4\.3](https://arxiv.org/html/2610.41074#S4.SS3)\)\. Using our scaling laws, we explain which computations need higher precision, when SR is useful during training, and how many needed bits meet an accuracy target\. Figure[1](https://arxiv.org/html/2610.41074#S1.F1)illustrates how the answers change with different predictors, model size and training time\.
- •Learning\-rate design and compute allocation\(Sections[4\.4](https://arxiv.org/html/2610.41074#S4.SS4)–[4\.5](https://arxiv.org/html/2610.41074#S4.SS5)\)\. Our scaling law covers general learning\-rate and precision schedules\. We show how backward rounding changes learning\-rate schedule design and compare precision choices at a fixed compute budget\. With a fixed compute budget, lower precision can improve prediction when the benefit of a larger model or longer training outweighs the added rounding error; Figure[3](https://arxiv.org/html/2610.41074#S4.F3)illustrates this trade\-off\.

For more related work, see Appendix[A](https://arxiv.org/html/2610.41074#A1)\. Appendices[B](https://arxiv.org/html/2610.41074#A2)–[E](https://arxiv.org/html/2610.41074#A5)give the mathematical foundations and proofs of our theorems\. Appendix[F](https://arxiv.org/html/2610.41074#A6)shows the experimental details\.

## 2Model setup

##### Dataset and random features\.

We consider online SGD for Gaussian squared\-loss regression with input dimensionddand feature widthMM\. For each samplex∼N⁡\(0,Σd\)x\\sim N\(0,\\Sigma\_\{d\}\)and label noiseϵ∼N⁡\(0,σ2\)\\epsilon\\sim N\(0,\\sigma^\{2\}\)with fixedσ2\>0\\sigma^\{2\}\>0, set the label to bey=x⊤​θ∗\+ϵy=x^\{\\top\}\\theta\_\{\*\}\+\\epsilon\. HereΣd=diag⁡\(λj\)∈ℝd×d\\Sigma\_\{d\}=\\diag\(\\lambda\_\{j\}\)\\in\\mathbb\{R\}^\{d\\times d\}is the input covariance, andθ∗∼N⁡\(0,Cθ\)\\theta\_\{\*\}\\sim N\(0,C\_\{\\theta\}\)is a target parameter with covarianceCθ=diag⁡\(ϑj\)∈ℝd×dC\_\{\\theta\}=\\diag\(\\vartheta\_\{j\}\)\\in\\mathbb\{R\}^\{d\\times d\}\. We assume, for fixed constantscλ,Cλ,cλ​ϑ,Cλ​ϑ\>0c\_\{\\lambda\},C\_\{\\lambda\},c\_\{\\lambda\\vartheta\},C\_\{\\lambda\\vartheta\}\>0,

d=⌈cdM⌉,cd\>1,1<b<a,cλj−a≤λj≤Cλj−a,cλ​ϑj−b≤λjϑj≤Cλ​ϑj−b\(1≤j≤d\)\.\\begin\{gathered\}d=\\lceil c\_\{d\}M\\rceil,\\quad c\_\{d\}\>1,\\quad 1<b<a,\\\\ c\_\{\\lambda\}j^\{\-a\}\\leq\\lambda\_\{j\}\\leq C\_\{\\lambda\}j^\{\-a\},\\qquad c\_\{\\lambda\\vartheta\}j^\{\-b\}\\leq\\lambda\_\{j\}\\vartheta\_\{j\}\\leq C\_\{\\lambda\\vartheta\}j^\{\-b\}\\quad\(1\\leq j\\leq d\)\.\\end\{gathered\}\(1\)The exponentsa,ba,bdescribe spectral and target\-energy decay, relate to the source and capacity conditions\([Paquette et al\., 2024](https://arxiv.org/html/2610.41074#bib.bib21)\)\. The constants in \([1](https://arxiv.org/html/2610.41074#S2.E1)\) do not depend onMM\. LetS∈ℝM×dS\\in\\mathbb\{R\}^\{M\\times d\}be a random weight matrix with i\.i\.d\.N⁡\(0,1/M\)N\(0,1/M\)entries, drawn once and held fixed during training\. The sketchSS,θ∗\\theta\_\{\*\}, inputsxxand label noises are jointly independent\. The feature vector isz=S​xz=Sxand the predictor is the two\-layer linear modelv⊤​zv^\{\\top\}z\. The population covariance of the feature vectorzzand its largest eigenvalue are defined by

G=S​Σd​S⊤,g1=λmax​\(G\)\.G=S\\Sigma\_\{d\}S^\{\\top\},\\qquad g\_\{1\}=\\lambda\_\{\\max\}\(G\)\.\(2\)

##### Stochastic rounding \(SR\)\.

For integerp≥0p\\geq 0we use the binary grid𝔽p=\{0\}∪\{±2e\(1\+k2−p\):e,k∈ℤ,0≤k<2p\}\\mathbb\{F\}\_\{p\}=\\\{0\\\}\\cup\\\{\\pm 2^\{e\}\(1\+k2^\{\-p\}\):e,k\\in\\mathbb\{Z\},\\ 0\\leq k<2^\{p\}\\\}\. Here exponenteeis unbounded\. For anyx∈ℝx\\in\\mathbb\{R\}, letℓp​\(x\)\\ell\_\{p\}\(x\)andup​\(x\)u\_\{p\}\(x\)be the neighboring grid points such thatℓp​\(x\)<x<up​\(x\)\\ell\_\{p\}\(x\)<x<u\_\{p\}\(x\)\. The stochastic rounding operatorQp​\(x\)Q\_\{p\}\(x\)quantizexxto the upper pointup​\(x\)u\_\{p\}\(x\)with probability\(x−ℓp​\(x\)\)/\(up​\(x\)−ℓp​\(x\)\)\(x\-\\ell\_\{p\}\(x\)\)/\(u\_\{p\}\(x\)\-\\ell\_\{p\}\(x\)\)and the lower pointℓp​\(x\)\\ell\_\{p\}\(x\)otherwise\. Grid points are fixed and rounding is unbiased:𝔼​\[Qp​\(x\)∣x\]=x\\mathbb\{E\}\[Q\_\{p\}\(x\)\\mid x\]=x\. Besides, the variance of the rounding error is bounded by

νp​\(x\)=𝔼⁡\[\(Qp​\(x\)−x\)2∣x\]=\(x−ℓp​\(x\)\)​\(up​\(x\)−x\)≤14​4−p​x2\.\\nu\_\{p\}\(x\)=\\mathbb\{E\}\[\(Q\_\{p\}\(x\)\-x\)^\{2\}\\mid x\]=\(x\-\\ell\_\{p\}\(x\)\)\(u\_\{p\}\(x\)\-x\)\\leq\\tfrac\{1\}\{4\}\\,4^\{\-p\}x^\{2\}\.\(3\)Denote the exact arithmetic byQ∞​\(x\)=xQ\_\{\\infty\}\(x\)=x\. On vectors,QpQ\_\{p\}acts coordinatewise independently\.

##### The SGD training algorithm\.

Start fromv0=0v\_\{0\}=0and takeT≥1T\\geq 1online SGD steps with fixed batch sizeB≥1B\\geq 1for mean squared loss of power\-law random features\. At updateuu, we drawBBfresh samples\{xu​ℓ,yu​ℓ\}ℓ=1B\\\{x\_\{u\\ell\},y\_\{u\\ell\}\\\}\_\{\\ell=1\}^\{B\}and*independently*round features for prediction and gradient computation:

au​ℓ=Qpfw,u​\(S​xu​ℓ\),bu​ℓ=Qpbw,u​\(S​xu​ℓ\)\.a\_\{u\\ell\}=Q\_\{p\_\{\\mathrm\{fw\},u\}\}\(Sx\_\{u\\ell\}\),\\qquad b\_\{u\\ell\}=Q\_\{p\_\{\\mathrm\{bw\},u\}\}\(Sx\_\{u\\ell\}\)\.\(4\)Herepfw,up\_\{\\mathrm\{fw\},u\}andpbw,up\_\{\\mathrm\{bw\},u\}are the mantissa precisions of the two independently rounded copies ofS​xu​ℓSx\_\{u\\ell\}\. The forward copyau​ℓa\_\{u\\ell\}is used to compute the predictionau​ℓ⊤​vua\_\{u\\ell\}^\{\\top\}v\_\{u\}; the backward copybu​ℓb\_\{u\\ell\}multiplies the prediction residual in the gradient\. We round only these two feature copies\. The SGD update is

vu\+1=vu−ηuB​∑ℓ=1Bbu​ℓ​\(au​ℓ⊤​vu−yu​ℓ\),0≤u<T,v\_\{u\+1\}=v\_\{u\}\-\\frac\{\\eta\_\{u\}\}\{B\}\\sum\_\{\\ell=1\}^\{B\}b\_\{u\\ell\}\(a\_\{u\\ell\}^\{\\top\}v\_\{u\}\-y\_\{u\\ell\}\),\\qquad 0\\leq u<T,\(5\)whereηu≥0\\eta\_\{u\}\\geq 0is the learning rate at stepuu\. All other arithmetic is exact\. Write

qp=4−p,qr,u=qpr,u,r∈\{fw,bw\},q∞=0,q\_\{p\}=4^\{\-p\},\\qquad q\_\{r,u\}=q\_\{p\_\{r,u\}\},\\quad r\\in\\\{\\mathrm\{fw\},\\mathrm\{bw\}\\\},\\qquad q\_\{\\infty\}=0,\(6\)τt=∑u<tηu,ρu=ηu/B,hu=ηu2/B\.\\tau\_\{t\}=\\sum\_\{u<t\}\\eta\_\{u\},\\qquad\\rho\_\{u\}=\\eta\_\{u\}/B,\\qquad h\_\{u\}=\\eta\_\{u\}^\{2\}/B\.\(7\)Here cumulative learning rateτt\\tau\_\{t\}measures optimization time\. We denote constant learning rate ifηu=η\\eta\_\{u\}=\\etafor alluu\. In this case, defineρ=η/B\\rho=\\eta/Bandτt=η​t\\tau\_\{t\}=\\eta t\.

##### Outputs and prediction risks\.

For SGD training in \([5](https://arxiv.org/html/2610.41074#S2.E5)\), we consider the last iterate and the uniform average outputs at final stepTTas our predictors

vTlast=vT,vTavg=1T​∑t=1Tvt\.v\_\{T\}^\{\\rm last\}=v\_\{T\},\\qquad v\_\{T\}^\{\\rm avg\}=\\frac\{1\}\{T\}\\sum\_\{t=1\}^\{T\}v\_\{t\}\.\(8\)Averaging can reduce SGD variance\([Polyak and Juditsky, 1992](https://arxiv.org/html/2610.41074#bib.bib61);[Yang et al\., 2019](https://arxiv.org/html/2610.41074#bib.bib24)\)in SGD dynamics\. Write‖u‖A2=u⊤​A​u\\\|u\\\|\_\{A\}^\{2\}=u^\{\\top\}Au\. The population prediction excess risk can be written and simplified as

ℰS\(v,θ∗\)=12𝔼x,ϵ\[\(y−v⊤Sx\)2\|S,θ∗\]−σ22=12∥S⊤v−θ∗∥Σd2\.\\mathcal\{E\}\_\{S\}\(v,\\theta\_\{\*\}\)=\\frac\{1\}\{2\}\\mathbb\{E\}\_\{x,\\epsilon\}\[\(y\-v^\{\\top\}Sx\)^\{2\}\|S,\\theta\_\{\*\}\]\-\\frac\{\\sigma^\{2\}\}\{2\}=\\tfrac\{1\}\{2\}\\\|S^\{\\top\}v\-\\theta\_\{\*\}\\\|\_\{\\Sigma\_\{d\}\}^\{2\}\.\(9\)Evaluation uses exact arithmetic\. Define its prediction risk and rounding\-induced risk increase by

ℛout​\(T\)\\displaystyle\\mathscr\{R\}^\{\{\\rm out\}\}\(T\)=𝔼⁡\[ℰS​\(vTout,θ∗\)∣S\],\\displaystyle=\\mathbb\{E\}\[\\mathcal\{E\}\_\{S\}\(v\_\{T\}^\{\{\\rm out\}\},\\theta\_\{\*\}\)\\mid S\],\(10\)Δout​\(T\)\\displaystyle\\Delta^\{\{\\rm out\}\}\(T\)=ℛout​\(T\)−ℛ0out​\(T\),\\displaystyle=\\mathscr\{R\}^\{\{\\rm out\}\}\(T\)\-\\mathscr\{R\}\_\{0\}^\{\{\\rm out\}\}\(T\),where indexout∈\{last,avg\}\{\\rm out\}\\in\\\{\\mathrm\{last\},\\mathrm\{avg\}\\\}, expectation averages over the teacher, training data and SR, conditional onSS\. The exact\-arithmetic riskℛ0out​\(T\)\\mathscr\{R\}\_\{0\}^\{\{\\rm out\}\}\(T\)is the prediction risk withpfw,u,pbw,u=∞p\_\{\\mathrm\{fw\},u\},p\_\{\\mathrm\{bw\},u\}=\\inftyfor alluu, under the same initialization, learning rates, batch size, time horizon and output\.

## 3Scaling laws for stochastic\-rounding SGD

In this section, we give a neural scaling law of the prediction risks in \([10](https://arxiv.org/html/2610.41074#S2.E10)\) for general learning\-rate and precision schedules, then evaluate it at constant learning rate\. The law is decomposed into the error of exact\-arithmetic training and the additional error due to rounding\. It applies to both outputs in \([8](https://arxiv.org/html/2610.41074#S2.E8)\)\. The difference of their scaling laws explain why they need different precision policies\.

### 3\.1Main result for general learning\-rate and precision schedules

A rounding error introduced at updateuufirst affectsvu\+1v\_\{u\+1\}\. Its contribution to the final predictor depends on the subsequent learning rates and the output rule\. We introduce the following scalars to describe how rounding error at stepuucontributes to prediction risk for last iterate and uniform average predictors\. Forr=1,2r=1,2and0≤u<T0\\leq u<T, define

Krlast​\(u\)\\displaystyle K\_\{r\}^\{\\rm last\}\(u\)=\[1\+2​\(τT−τu\+1\)\]1a−r,\\displaystyle=\[1\+2\(\\tau\_\{T\}\-\\tau\_\{u\+1\}\)\]^\{\\frac\{1\}\{a\}\-r\},\(11\)Kravg​\(u\)\\displaystyle K\_\{r\}^\{\\rm avg\}\(u\)=1T2​∑v,w=u\+1T\[1\+τv\+τw−2​τu\+1\]1a−r\.\\displaystyle=\\frac\{1\}\{T^\{2\}\}\\sum\_\{v,w=u\+1\}^\{T\}\[1\+\\tau\_\{v\}\+\\tau\_\{w\}\-2\\tau\_\{u\+1\}\]^\{\\frac\{1\}\{a\}\-r\}\.\(12\)For the last iterate, an error introduced at updateuuis propagated through updatesu\+1,…,T−1u\+1,\\ldots,T\-1before the final prediction\. The cumulative learning rateτT−τu\+1=∑v=u\+1T−1ηv\\tau\_\{T\}\-\\tau\_\{u\+1\}=\\sum\_\{v=u\+1\}^\{T\-1\}\\eta\_\{v\}measures the amount of learning after that error occurs\. For uniform averaging, the same error contributes to every later iteratevu\+1,…,vTv\_\{u\+1\},\\ldots,v\_\{T\}\. These contributions are correlated because they originate from the same error\. Squaring their average therefore produces a term for every pair of later iterates, indexed byv,wv,w, and the factorT−2T^\{\-2\}in \([12](https://arxiv.org/html/2610.41074#S3.E12)\)\. For exact\-arithmetic training andout∈\{last,avg\}\{\\rm out\}\\in\\\{\\mathrm\{last\},\\mathrm\{avg\}\\\}, we define

B^last\\displaystyle\\widehat\{B\}^\{\\rm last\}=12\(1\+2τT\)−\(b−1\)/a,\\displaystyle=\\tfrac\{1\}\{2\}\(1\+2\\tau\_\{T\}\)^\{\-\(b\-1\)/a\},B^avg\\displaystyle\\widehat\{B\}^\{\\rm avg\}=12​T2∑v,w=1T\(1\+τv\+τw\)−\(b−1\)/a,\\displaystyle=\\frac\{1\}\{2T^\{2\}\}\\sum\_\{v,w=1\}^\{T\}\(1\+\\tau\_\{v\}\+\\tau\_\{w\}\)^\{\-\(b\-1\)/a\},\(13\)V^out\\displaystyle\\widehat\{V\}^\{\{\\rm out\}\}=σ22​∑u<Thu​K2out​\(u\),\\displaystyle=\\frac\{\\sigma^\{2\}\}\{2\}\\sum\_\{u<T\}h\_\{u\}K\_\{2\}^\{\{\\rm out\}\}\(u\),hu\\displaystyle h\_\{u\}=ηu2/B,\\displaystyle=\\eta\_\{u\}^\{2\}/B,to characterize the bias and the variance from minibatch sampling at stepTT\. For0≤t≤T0\\leq t\\leq T, define

U^t=τt2\(1\+τt\)\(a\+b−1\)/a\+σ2​∑u<thu​\[1\+2​\(τt−τu\+1\)\]1/a−1\.\\widehat\{U\}\_\{t\}=\\frac\{\\tau\_\{t\}^\{2\}\}\{\(1\+\\tau\_\{t\}\)^\{\(a\+b\-1\)/a\}\}\+\\sigma^\{2\}\\sum\_\{u<t\}h\_\{u\}\[1\+2\(\\tau\_\{t\}\-\\tau\_\{u\+1\}\)\]^\{1/a\-1\}\.\(14\)Forout∈\{last,avg\}\{\\rm out\}\\in\\\{\\rm last,\\rm avg\\\}, defineN^out\\widehat\{N\}^\{\{\\rm out\}\}to control the rounding\-induced risk increase:

N^out=\\displaystyle\\widehat\{N\}^\{\{\\rm out\}\}=12​∑u<Thu​\[qfw,u​U^uM​K2out​\(u\)\+\(σ2​qbw,uM\+qfw,u​qbw,u​U^uM2\)​K1out​\(u\)\]\\displaystyle\\frac\{1\}\{2\}\\sum\_\{u<T\}h\_\{u\}\\left\[\\frac\{q\_\{\\mathrm\{fw\},u\}\\widehat\{U\}\_\{u\}\}\{M\}K\_\{2\}^\{\{\\rm out\}\}\(u\)\+\\left\(\\frac\{\\sigma^\{2\}q\_\{\\mathrm\{bw\},u\}\}\{M\}\+\\frac\{q\_\{\\mathrm\{fw\},u\}q\_\{\\mathrm\{bw\},u\}\\widehat\{U\}\_\{u\}\}\{M^\{2\}\}\\right\)K\_\{1\}^\{\{\\rm out\}\}\(u\)\\right\]\(15\)\+12∑u<v<Thuhv\[σ2​qbw,u​qfw,vMK2out\(v\)\+qfw,u​qbw,v​U^uM2\[1\+2\(τv−τu\+1\)\]1a−2K1out\(v\)\]\.\\displaystyle\+\\frac\{1\}\{2\}\\sum\_\{u<v<T\}h\_\{u\}h\_\{v\}\\left\[\\frac\{\\sigma^\{2\}q\_\{\\mathrm\{bw\},u\}q\_\{\\mathrm\{fw\},v\}\}\{M\}K\_\{2\}^\{\{\\rm out\}\}\(v\)\+\\frac\{q\_\{\\mathrm\{fw\},u\}q\_\{\\mathrm\{bw\},v\}\\widehat\{U\}\_\{u\}\}\{M^\{2\}\}\[1\+2\(\\tau\_\{v\}\-\\tau\_\{u\+1\}\)\]^\{\\frac\{1\}\{a\}\-2\}K\_\{1\}^\{\{\\rm out\}\}\(v\)\\right\]\.For more explanation of these quantities, see Appendix[D\.1](https://arxiv.org/html/2610.41074#A4.SS1)\.

Fixc0∈\(0,1\)c\_\{0\}\\in\(0,1\)\. Appendix[C\.6](https://arxiv.org/html/2610.41074#A3.SS6)constructs positive thresholdsτ∗sch\\tau\_\{\*\}^\{\\rm sch\}andρsch\\rho\_\{\\rm sch\}, depending only on the fixed model quantities andc0c\_\{0\}\. For sufficiently largeMM, we require the learning rates\{ηu\}u=0T\\\{\\eta\_\{u\}\\\}\_\{u=0\}^\{T\}to satisfy

τ∗sch≤τT≤Ma,0≤ηu​g1≤c0<1,ηu/B≤ρsch\.\\tau\_\{\*\}^\{\\rm sch\}\\leq\\tau\_\{T\}\\leq M^\{a\},\\qquad 0\\leq\\eta\_\{u\}g\_\{1\}\\leq c\_\{0\}<1,\\qquad\\eta\_\{u\}/B\\leq\\rho\_\{\\rm sch\}\.\(16\)The thresholds are independent of width, time horizon, batch size, schedules and the realized sketch\. The learning\-rate and precision schedules are deterministically chosen before the SR\.

The following theorem gives matching upper and lower bounds for the error already present in exact\-arithmetic SGD and the additional error caused by SR\. It covers both the final iterate and uniform parameter averaging, allowing learning rates and precisions to change during training\.

###### Theorem 1\(Scaling law for general SR and learning\-rate schedules\)\.

Recall \([10](https://arxiv.org/html/2610.41074#S2.E10)\)\. Consider any deterministic learning\-rate schedule\{ηu\}u=0T\\\{\\eta\_\{u\}\\\}\_\{u=0\}^\{T\}satisfying \([16](https://arxiv.org/html/2610.41074#S3.E16)\) and precision schedule\{pfw,u,pbw,u\}u=0T\\\{p\_\{\\mathrm\{fw\},u\},p\_\{\\mathrm\{bw\},u\}\\\}\_\{u=0\}^\{T\}\. Under Assumption[C\.1](https://arxiv.org/html/2610.41074#Thmtheorem1b), for eachD\>0D\>0, there exist some constantsc,KD\>0c,K\_\{D\}\>0such that, for sufficiently large widthMM, with probability at least1−6​e−c​M−KD​M−D1\-6e^\{\-cM\}\-K\_\{D\}M^\{\-D\}, the following holds\. Forout∈\{last,avg\}\{\\rm out\}\\in\\\{\\mathrm\{last\},\\mathrm\{avg\}\\\}, the prediction risk with SR at timeTT,ℛout​\(T\)=ℛ0out​\(T\)\+Δout​\(T\)\\mathscr\{R\}^\{\{\\rm out\}\}\(T\)=\\mathscr\{R\}\_\{0\}^\{\{\\rm out\}\}\(T\)\+\\Delta^\{\{\\rm out\}\}\(T\), satisfies

ℛ0out​\(T\)\\displaystyle\\mathscr\{R\}\_\{0\}^\{\{\\rm out\}\}\(T\)≍M1−b\+B^out\+V^out,Δout\(T\)≍N^out,\\displaystyle\\asymp M^\{1\-b\}\+\\widehat\{B\}^\{\{\\rm out\}\}\+\\widehat\{V\}^\{\{\\rm out\}\},\\qquad\\Delta^\{\{\\rm out\}\}\(T\)\\asymp\\widehat\{N\}^\{\{\\rm out\}\},\(17\)where the constants are independent of widthMM, time horizonTT, batchBB, learning\-rate and precision schedules, andSS\.

Assumption[C\.1](https://arxiv.org/html/2610.41074#Thmtheorem1b)restricts the learning\-rate and precision schedules so that earlier rounding errors do not excessively amplify the variance of later rounding errors\. The proof of Theorem[1](https://arxiv.org/html/2610.41074#Thmtheorem1)mainly uses second\-moment recursions in \([40](https://arxiv.org/html/2610.41074#A2.E40)\) along the training trajectory of \([5](https://arxiv.org/html/2610.41074#S2.E5)\) and is given in Appendix[D\.1](https://arxiv.org/html/2610.41074#A4.SS1)\.

### 3\.2Special case: constant learning rate and precisions

We now specialize Theorem[1](https://arxiv.org/html/2610.41074#Thmtheorem1)to constant learning rates and precisions\. Takeηu=η\\eta\_\{u\}=\\eta,pfw,u=pfwp\_\{\\mathrm\{fw\},u\}=p\_\{\\mathrm\{fw\}\}, andpbw,u=pbwp\_\{\\mathrm\{bw\},u\}=p\_\{\\mathrm\{bw\}\}throughout the training in \([5](https://arxiv.org/html/2610.41074#S2.E5)\) for allu=0,…,Tu=0,\\dots,T\. At stepTT,ρT=η/B=ρ\\rho\_\{T\}=\\eta/B=\\rhoandτT=η​T=τ\\tau\_\{T\}=\\eta T=\\tau\. Then Theorem[1](https://arxiv.org/html/2610.41074#Thmtheorem1)implies that

ℛ0last​\(T\)\\displaystyle\\mathscr\{R\}\_\{0\}^\{\\rm last\}\(T\)≍M1−b\+τ−\(b−1\)/a\+ρsM,\\displaystyle\\asymp M^\{1\-b\}\+\\tau^\{\-\(b\-1\)/a\}\+\\rho s\_\{M\},\(18\)ℛ0avg​\(T\)\\displaystyle\\mathscr\{R\}\_\{0\}^\{\\rm avg\}\(T\)≍M1−b\+τ−\(b−1\)/a\+ρsMτ−\(a−1\)/a,sM=σ2\+M1−b,\\displaystyle\\asymp M^\{1\-b\}\+\\tau^\{\-\(b\-1\)/a\}\+\\rho s\_\{M\}\\tau^\{\-\(a\-1\)/a\},\\qquad s\_\{M\}=\\sigma^\{2\}\+M^\{1\-b\},\(19\)and their additional error induced by SR training are given by

Δlast​\(T\)≍\\displaystyle\\Delta^\{\\rm last\}\(T\)\\asymp\{\}ρ​qfwM​τa−b\+1a\+ρ​qbwM​\(sM\+τ−b−1a\)​τ1a\+ρ2​qfw​qbwM​\(sM\+τ−b−1a\)​τ,\\displaystyle\\frac\{\\rho q\_\{\\mathrm\{fw\}\}\}\{M\}\\tau^\{\\frac\{a\-b\+1\}\{a\}\}\+\\frac\{\\rho q\_\{\\mathrm\{bw\}\}\}\{M\}\(s\_\{M\}\+\\tau^\{\-\\frac\{b\-1\}\{a\}\}\)\\tau^\{\\frac\{1\}\{a\}\}\+\\frac\{\\rho^\{2\}q\_\{\\mathrm\{fw\}\}q\_\{\\mathrm\{bw\}\}\}\{M\}\(s\_\{M\}\+\\tau^\{\-\\frac\{b\-1\}\{a\}\}\)\\tau,\(20\)Δavg​\(T\)≍\\displaystyle\\Delta^\{\\rm avg\}\(T\)\\asymp\{\}ρ​qfwM​τ2−ba\+ρ​qbwM​\(sM\+τ−b−1a\)​τ1a\+ρ​qfw​qbwM2​τa−b\+2a\.\\displaystyle\\frac\{\\rho q\_\{\\mathrm\{fw\}\}\}\{M\}\\tau^\{\\frac\{2\-b\}\{a\}\}\+\\frac\{\\rho q\_\{\\mathrm\{bw\}\}\}\{M\}\(s\_\{M\}\+\\tau^\{\-\\frac\{b\-1\}\{a\}\}\)\\tau^\{\\frac\{1\}\{a\}\}\+\\frac\{\\rho q\_\{\\mathrm\{fw\}\}q\_\{\\mathrm\{bw\}\}\}\{M^\{2\}\}\\tau^\{\\frac\{a\-b\+2\}\{a\}\}\.\(21\)For full statements and proofs, see Corollaries[D\.3](https://arxiv.org/html/2610.41074#Thmtheorem3b)and[D\.4](https://arxiv.org/html/2610.41074#Thmtheorem4b)\. In \([20](https://arxiv.org/html/2610.41074#S3.E20)\) and \([21](https://arxiv.org/html/2610.41074#S3.E21)\) the terms in each SR\-induced error are forward, backward and joint rounding\. Exact forward or backward arithmetic setsqfw=0q\_\{\\mathrm\{fw\}\}=0orqbw=0q\_\{\\mathrm\{bw\}\}=0, respectively\. Full riskℛlast​\(T\)\\mathscr\{R\}^\{\\rm last\}\(T\)andℛavg​\(T\)\\mathscr\{R\}^\{\\rm avg\}\(T\)are the sums of the corresponding terms in \([18](https://arxiv.org/html/2610.41074#S3.E18)\), \([19](https://arxiv.org/html/2610.41074#S3.E19)\) and \([20](https://arxiv.org/html/2610.41074#S3.E20)\), \([21](https://arxiv.org/html/2610.41074#S3.E21)\)\. As a remark, in this special case, a sufficient condition for Assumption[C\.1](https://arxiv.org/html/2610.41074#Thmtheorem1b)in Theorem[1](https://arxiv.org/html/2610.41074#Thmtheorem1)is, for some constantκc\>0\\kappa\_\{\\rm c\}\>0,

ρ​qfw​qbw​τ/M≤κc\.\\rho q\_\{\\mathrm\{fw\}\}q\_\{\\mathrm\{bw\}\}\\tau/M\\leq\\kappa\_\{\\rm c\}\.\(22\)
##### How rounding changes the learning curve\.

For growing widthMMand optimization timeτ\\tau, prediction riskℛavg​\(T\)\\mathscr\{R\}^\{\\rm avg\}\(T\)can approach zero even when SR makes it decrease more slowly than its exact\-arithmetic counterpart\. At fixed positiveρ,σ2\\rho,\\sigma^\{2\}, withτ→∞\\tau\\to\\inftyandτ≤Ma\\tau\\leq M^\{a\}from \([16](https://arxiv.org/html/2610.41074#S3.E16)\), the exact\-arithmetic risk decays asℛ0avg\(T\)≍τ−\(b−1\)/a\\mathscr\{R\}\_\{0\}^\{\\rm avg\}\(T\)\\asymp\\tau^\{\-\(b\-1\)/a\}\. The last iterate instead retains an SGD\-noise floor of orderρ​σ2\\rho\\sigma^\{2\}, so increasing width and training time alone cannot make its risk vanish at fixedρ\\rho\.

For the average iterate, preserving the exact\-arithmetic risk order requiresΔavg​\(T\)=O⁡\(ℛ0avg​\(T\)\)\\Delta^\{\\rm avg\}\(T\)=O\(\\mathscr\{R\}\_\{0\}^\{\\rm avg\}\(T\)\); rounding error becomes negligible whenΔavg​\(T\)=o⁡\(ℛ0avg​\(T\)\)\\Delta^\{\\rm avg\}\(T\)=o\(\\mathscr\{R\}\_\{0\}^\{\\rm avg\}\(T\)\)\. With forward arithmetic exact and fixed finite backward precision, one can easily checkτ=o⁡\(Ma\)\\tau=o\(M^\{a\}\)implies \([21](https://arxiv.org/html/2610.41074#S3.E21)\) vanishing, whereas preserving the order ofℛ0avg​\(T\)\\mathscr\{R\}\_\{0\}^\{\\rm avg\}\(T\)requiresτ=O⁡\(Ma/b\)\\tau=O\(M^\{a/b\}\)\. Thus SR\-induced excess risk in \([21](https://arxiv.org/html/2610.41074#S3.E21)\) can determine the decay rate even whileℛavg​\(T\)=o​\(1\)\\mathscr\{R\}^\{\\rm avg\}\(T\)=o\(1\)\. On the other hand, with SR for both forward and backward passes,τ≍M\\tau\\asymp Mwith a sufficiently small proportionality constant satisfies \([22](https://arxiv.org/html/2610.41074#S3.E22)\) which directly implies that the riskℛavg​\(T\)\\mathscr\{R\}^\{\\rm avg\}\(T\)vanishes and rounding error is negligible\. Along this family, risk of the uniformly averaged predictor tends to zero as width and training time increase, while both mantissa precisions remain fixed\.

### 3\.3Compute\-optimal scaling law with precision

We now use the constant\-learning\-rate laws to determine how model width and training duration should grow with a compute budget\. We focus on the uniformly averaged predictor\. We charge the forward and gradient feature computations against a compute budgetFF:

f:=B​M​∑u<T\[c⁡\(pfw,u\)\+c⁡\(pbw,u\)\]≤F,c⁡\(p\)=\{\(p\+p0\)/\(pref\+p0\),p<∞,1,p=∞\.f:=BM\\sum\_\{u<T\}\[c\(p\_\{\\mathrm\{fw\},u\}\)\+c\(p\_\{\\mathrm\{bw\},u\}\)\]\\leq F,\\qquad c\(p\)=\\begin\{cases\}\(p\+p\_\{0\}\)/\(p\_\{\\rm ref\}\+p\_\{0\}\),&p<\\infty,\\\\ 1,&p=\\infty\.\\end\{cases\}\(23\)Here sign/exponent usesp0\>0p\_\{0\}\>0bits andprefp\_\{\\rm ref\}is a fixed large mantissa bit count for normalization\. Each forward or gradient feature computation at precisionppcostsB​M​c​\(p\)BM\\,c\(p\), so fewer bits lower the charge for the same updates\. Empirical scaling studies also include precision\-dependent training costs\([Kumar et al\., 2025](https://arxiv.org/html/2610.41074#bib.bib12);[Sun et al\., 2025](https://arxiv.org/html/2610.41074#bib.bib33);[Kim et al\., 2026](https://arxiv.org/html/2610.41074#bib.bib13)\)\. We use the above \([23](https://arxiv.org/html/2610.41074#S3.E23)\) to model the compute cost of SGD training in \([5](https://arxiv.org/html/2610.41074#S2.E5)\) with different precisions for simplicity\.

Consider the setup of Section[3\.2](https://arxiv.org/html/2610.41074#S3.SS2)with constantη,B\\eta,Band fixed precision pair\(pfw,pbw\)\(p\_\{\\mathrm\{fw\}\},p\_\{\\mathrm\{bw\}\}\), independently of the budgetFF\. At each budget, we choose widthMMand training durationTTto minimize the prediction riskℛavg​\(T\)\\mathscr\{R\}^\{\\rm avg\}\(T\), subject tof≤Ff\\leq Fand the assumptions of Theorem[1](https://arxiv.org/html/2610.41074#Thmtheorem1)\. Table[1](https://arxiv.org/html/2610.41074#S3.T1)presents compute\-optimal scaling laws \(the order of the smallest achievable risk\), and the orders ofMMandτ=η​T\\tau=\\eta Tattaining it\. The formal statements and proofs are given in Proposition[D\.5](https://arxiv.org/html/2610.41074#Thmtheorem5b)\. Besides, Proposition[D\.6](https://arxiv.org/html/2610.41074#Thmtheorem6b)justifies the same scaling laws but allowingη,B\\eta,Bto vary across budgets\. Proposition[D\.7](https://arxiv.org/html/2610.41074#Thmtheorem7b)addresses similar laws for the last iterate predictor\.

Table 1:Compute\-optimal prediction riskℛavg​\(T\)\\mathscr\{R\}^\{\\rm avg\}\(T\)with fixed precisions,η,B\\eta,B, and compute budgetFF\. Precisionspfw,pbwp\_\{\\mathrm\{fw\}\},p\_\{\\mathrm\{bw\}\}may differ when they are finite\. Columns ofMMandτ\\taushow the orders attaining the compute\-optimal risk\. Here we only focus on the scaling order asF→∞F\\to\\infty\.1<b<a1<b<a\.Table[1](https://arxiv.org/html/2610.41074#S3.T1)shows that rounding location changes both the risk exponent and the optimal allocations of width and training time\. ForvTavgv\_\{T\}^\{\\rm avg\}, fixed finite forward precision preserves the exact\-arithmetic exponent when backward features remain exact withτ≍Ma\\tau\\asymp M^\{a\}\. However, the attaining allocation for backward rounding only usesτ≍Ma/b\\tau\\asymp M^\{a/b\}; both finite precisions case haveτ≍M\\tau\\asymp M\. These allocations use progressively more width for a given optimization time, but their risk decays successively more slowly with compute\. Low precision therefore need not create an accuracy floor whenMMand training time increase together\. The exponents alone in Table[1](https://arxiv.org/html/2610.41074#S3.T1)do not determine which configuration has lower risk at a fixed budget: precision affects both exponents and risk coefficients\. Section[4\.5](https://arxiv.org/html/2610.41074#S4.SS5)develops a more detailed analysis of compute allocation mechanisms and finite\-budget comparisons\.

## 4Implications for low\-precision training

In this section, we apply scaling laws in Section[3](https://arxiv.org/html/2610.41074#S3)to answer three precision questions: where rounding is least harmful, when training needs higher precision, and how many bits meet an accuracy target\. We then specifically consider learning\-rate design and compute allocation\. Appendix[E](https://arxiv.org/html/2610.41074#A5)gives the full statements and proofs of the claims in this section\.

### 4\.1Where: rounding location for different predictors

Which location in training updates \([5](https://arxiv.org/html/2610.41074#S2.E5)\) needs more precision? We compare two different SGD dynamics with different rounding locations: forward\-only and backward\-only SR in \([4](https://arxiv.org/html/2610.41074#S2.E4)\) using the same mantissa bitpp\. Letq=4−p\>0q=4^\{\-p\}\>0\. Forout∈\{last,avg\}\{\\rm out\}\\in\\\{\{\\rm last\},\{\\rm avg\}\\\}, letΔout​\(q,0\)\\Delta^\{\{\\rm out\}\}\(q,0\)andΔout​\(0,q\)\\Delta^\{\{\\rm out\}\}\(0,q\)denoteΔout​\(T\)\\Delta^\{\{\\rm out\}\}\(T\)in \([20](https://arxiv.org/html/2610.41074#S3.E20)\) and \([21](https://arxiv.org/html/2610.41074#S3.E21)\) with\(qfw,qbw\)=\(q,0\)\(q\_\{\\mathrm\{fw\}\},q\_\{\\mathrm\{bw\}\}\)=\(q,0\)and\(qfw,qbw\)=\(0,q\)\(q\_\{\\mathrm\{fw\}\},q\_\{\\mathrm\{bw\}\}\)=\(0,q\), respectively; the other location is exact at each step\. Considerτ≍Ma\\tau\\asymp M^\{a\}and setup of Section[3\.2](https://arxiv.org/html/2610.41074#S3.SS2)\. Proposition[E\.1](https://arxiv.org/html/2610.41074#Thmtheorem1d)derives the following ratios for SR\-induced risk increase:

Δlast​\(q,0\)Δlast​\(0,q\)≍Ma−bσ2\+M1−b,Δavg​\(q,0\)Δavg​\(0,q\)≍M1−bσ2\+M1−b\.\\frac\{\\Delta^\{\\rm last\}\(q,0\)\}\{\\Delta^\{\\rm last\}\(0,q\)\}\\asymp\\frac\{M^\{a\-b\}\}\{\\sigma^\{2\}\+M^\{1\-b\}\},\\qquad\\frac\{\\Delta^\{\\rm avg\}\(q,0\)\}\{\\Delta^\{\\rm avg\}\(0,q\)\}\\asymp\\frac\{M^\{1\-b\}\}\{\\sigma^\{2\}\+M^\{1\-b\}\}\.\(24\)With fixed label noise, the first ratio grows asMa−bM^\{a\-b\}and the second decreases asM1−bM^\{1\-b\}\. Thus, forward\-only rounding causes a larger SR\-induced error for the last iterate, whereas backward\-only rounding leads to a larger error for the uniform average output, for sufficiently largeMM\.

From \([20](https://arxiv.org/html/2610.41074#S3.E20)\) to \([21](https://arxiv.org/html/2610.41074#S3.E21)\) withτ≍Ma\\tau\\asymp M^\{a\}, averaging makes the forward\-only SR\-induced error smaller by a factor of orderMa−1M^\{a\-1\}, whereas backward\-only errors have the same order for both outputs\. Averaging also reduces exact\-arithmetic risk in \([19](https://arxiv.org/html/2610.41074#S3.E19)\), so backward rounding error becomes relatively dominant\. Figure[1](https://arxiv.org/html/2610.41074#S1.F1)\(a\) illustrates this reversal with one\-bit rounding at one location\. The practical lesson is to choose between the last iterate and the uniform average outputs before allocating precision across computations; under the conditions above, we need higher forward precision for the last iterate and higher backward precision for the uniform average\. As a remark, \([24](https://arxiv.org/html/2610.41074#S4.E24)\) is only for single precision location comparison, hence when both locations are rounded, the interaction terms related toqfw​qbwq\_\{\\mathrm\{fw\}\}q\_\{\\mathrm\{bw\}\}change the overall error behavior\. Figure[2](https://arxiv.org/html/2610.41074#S4.F2)\(a\) illustrates joint allocation at a fixed total bit count: the best tested split is\(pfw,pbw\)=\(4,1\)\(p\_\{\\mathrm\{fw\}\},p\_\{\\mathrm\{bw\}\}\)=\(4,1\)for the last iterate and\(2,3\)\(2,3\)for the uniform average\.

### 4\.2When: precision allocation during training

Which updates deserve higher precision under compute budget? Fix widthMM, batchBB, horizonTT, and constant learning rateη\\eta\. We allocate the compute budget in \([23](https://arxiv.org/html/2610.41074#S3.E23)\) across training updates\. First keep one feature computation exact throughout training and use finite precision at the other\. For the rounded locationr∈\{fw,bw\}r\\in\\\{\\mathrm\{fw\},\\mathrm\{bw\}\\\}, Theorem[1](https://arxiv.org/html/2610.41074#Thmtheorem1)shows

Δout​\(T\)≍N^out=∑u<TAr,u​4−pr,u,Afw,u=η2​U^u​K2out​\(u\)2​B​M,Abw,u=η2​σ2​K1out​\(u\)2​B​M\\begin\{gathered\}\\Delta^\{\{\\rm out\}\}\(T\)\\asymp\\widehat\{N\}^\{\{\\rm out\}\}=\\sum\_\{u<T\}A\_\{r,u\}4^\{\-p\_\{r,u\}\},\\quad A\_\{\\mathrm\{fw\},u\}=\\frac\{\\eta^\{2\}\\widehat\{U\}\_\{u\}K\_\{2\}^\{\{\\rm out\}\}\(u\)\}\{2BM\},\\quad A\_\{\\mathrm\{bw\},u\}=\\frac\{\\eta^\{2\}\\sigma^\{2\}K\_\{1\}^\{\{\\rm out\}\}\(u\)\}\{2BM\}\\end\{gathered\}\(25\)whereK1out​\(u\),K2out​\(u\),U^uK\_\{1\}^\{\{\\rm out\}\}\(u\),K\_\{2\}^\{\{\\rm out\}\}\(u\),\\widehat\{U\}\_\{u\}are defined in \([11](https://arxiv.org/html/2610.41074#S3.E11)\), \([12](https://arxiv.org/html/2610.41074#S3.E12)\), and \([14](https://arxiv.org/html/2610.41074#S3.E14)\)\. These coefficientsAr,uA\_\{r,u\}are fixed before precision is allocated\. Given fixed compute budget \([23](https://arxiv.org/html/2610.41074#S3.E23)\) and fixedM,TM,T, we have a total precision constraint:∑u<Tpr,u≤P\\sum\_\{u<T\}p\_\{r,u\}\\leq Pfor a fixedP\>0P\>0\. Proposition[E\.3](https://arxiv.org/html/2610.41074#Thmtheorem3c)finds the minimizer ofN^out\\widehat\{N\}^\{\{\\rm out\}\}as

pr,u=\{max⁡\{0,log4⁡\(Ar,u/λ\)\},Ar,u\>0,0,Ar,u=0\.p\_\{r,u\}=\\begin\{cases\}\\max\\\{0,\\log\_\{4\}\(A\_\{r,u\}/\\lambda\)\\\},&A\_\{r,u\}\>0,\\\\ 0,&A\_\{r,u\}=0\.\\end\{cases\}\(26\)Hereλ\>0\\lambda\>0is a constant so that∑u<Tpr,u=P\\sum\_\{u<T\}p\_\{r,u\}=P\. We assign more bits whenN^out\\widehat\{N\}^\{\{\\rm out\}\}reduces most\.

The allocation in \([26](https://arxiv.org/html/2610.41074#S4.E26)\) assigns more bits to larger coefficients\. For backward rounding at constant learning rate,K1last​\(u\)K\_\{1\}^\{\\rm last\}\(u\)increases withuu, whereasK1avg​\(u\)K\_\{1\}^\{\\rm avg\}\(u\)decreases\. This allocation gives nondecreasing backward precision for the last iterate and nonincreasing backward precision for the uniform average\. The practical lesson is that, at the same precision budget, switching from the last iterate to a uniform average can reverse when higher backward precision is useful\. Figure[1](https://arxiv.org/html/2610.41074#S1.F1)\(b\) compares one\- and three\-bit precision in opposite time orders at nearly equal average precision: using higher precision late during training gives smaller last\-iterate error at both locations, whereas using higher backward precision early during training gives smaller uniform\-average error\. When both locations are rounded, interaction term changes their precision schedules\. Appendix[E\.2\.2](https://arxiv.org/html/2610.41074#A5.SS2.SSS2)and Proposition[E\.4](https://arxiv.org/html/2610.41074#Thmtheorem4c)study the joint allocation of precision over training\.

### 4\.3How many bits: limiting the risk increase from rounding

Table 2:Following Section[3\.2](https://arxiv.org/html/2610.41074#S3.SS2)withτ≍Ma\\tau\\asymp M^\{a\}\. Scaling definitions in \([28](https://arxiv.org/html/2610.41074#S4.E28)\) withsMs\_\{M\}defined in \([19](https://arxiv.org/html/2610.41074#S3.E19)\)\.How many forward and backward mantissa bits are needed to keep the risk close to that of exact\-arithmetic SGD? Consider the setting and results in Section[3\.2](https://arxiv.org/html/2610.41074#S3.SS2)\. We require the risk increase from rounding to be at most a fractionϑ\\varthetaof the exact\-arithmetic risk:

Δout​\(T\)≤ϑ​ℛ0out​\(T\),ϑ\>0\.\\Delta^\{\{\\rm out\}\}\(T\)\\leq\\vartheta\\mathscr\{R\}\_\{0\}^\{\{\\rm out\}\}\(T\),\\qquad\\vartheta\>0\.\(27\)Atτ≍Ma\\tau\\asymp M^\{a\}, forout∈\{last,avg\}\{\\rm out\}\\in\\\{\{\\rm last\},\{\\rm avg\}\\\}, results in \([18](https://arxiv.org/html/2610.41074#S3.E18)\)\-\([21](https://arxiv.org/html/2610.41074#S3.E21)\) can be simplified as

ℛ0out​\(T\)≍Dout,Δout​\(T\)≍Afw​qfw\+Abw​qbw\+A×​qfw​qbw,\\mathscr\{R\}\_\{0\}^\{\{\\rm out\}\}\(T\)\\asymp D^\{\{\\rm out\}\},\\quad\\Delta^\{\{\\rm out\}\}\(T\)\\asymp A\_\{\\mathrm\{fw\}\}q\_\{\\mathrm\{fw\}\}\+A\_\{\\mathrm\{bw\}\}q\_\{\\mathrm\{bw\}\}\+A\_\{\\times\}q\_\{\\mathrm\{fw\}\}q\_\{\\mathrm\{bw\}\},\(28\)where exact\-arithmetic risk scaleDoutD^\{\{\\rm out\}\}and three rounding coefficientsAfw,Abw,A×A\_\{\\mathrm\{fw\}\},A\_\{\\mathrm\{bw\}\},A\_\{\\times\}are defined in Table[2](https://arxiv.org/html/2610.41074#S4.T2), which only depend onM,σ,η,BM,\\sigma,\\eta,B\. Corollary[E\.5](https://arxiv.org/html/2610.41074#Thmtheorem5c)then further proves that to ensure \([27](https://arxiv.org/html/2610.41074#S4.E27)\), it suffices to satisfy the following bit requirements, for some constantC\>0C\>0\.

pfw\\displaystyle p\_\{\\mathrm\{fw\}\}≥\[log4⁡C​Afwϑ​Dout\]\+,\\displaystyle\\geq\\left\[\\log\_\{4\}\\frac\{CA\_\{\\mathrm\{fw\}\}\}\{\\vartheta D^\{\{\\rm out\}\}\}\\right\]\_\{\+\},pbw\\displaystyle p\_\{\\mathrm\{bw\}\}≥\[log4⁡C​Abwϑ​Dout\]\+,\\displaystyle\\geq\\left\[\\log\_\{4\}\\frac\{CA\_\{\\mathrm\{bw\}\}\}\{\\vartheta D^\{\{\\rm out\}\}\}\\right\]\_\{\+\},\(29\)pfw\+pbw\\displaystyle p\_\{\\mathrm\{fw\}\}\+p\_\{\\mathrm\{bw\}\}≥\[log4⁡C​A×ϑ​Dout\]\+,\\displaystyle\\geq\\left\[\\log\_\{4\}\\frac\{CA\_\{\\times\}\}\{\\vartheta D^\{\{\\rm out\}\}\}\\right\]\_\{\+\},\[x\]\+\\displaystyle\[x\]\_\{\+\}=max⁡\{x,0\}\.\\displaystyle=\\max\\\{x,0\\\}\.The first two bounds in \([29](https://arxiv.org/html/2610.41074#S4.E29)\) control precisions for forward\-only or backward\-only rounding, respectively, like Section[4\.1](https://arxiv.org/html/2610.41074#S4.SS1)\. For joint rounding at two locations, the precisions must jointly satisfy all the bounds in \([29](https://arxiv.org/html/2610.41074#S4.E29)\), though unequal forward and backward precisions are allowed\.

For example, consider uniform averaging with exact forward computation and fixed positiveρ,σ2,ϑ\\rho,\\sigma^\{2\},\\varthetaatτ≍Ma\\tau\\asymp M^\{a\}\. The lower bound on SR\-induced error and upper bound on exact\-arithmetic risk in Corollary[E\.5](https://arxiv.org/html/2610.41074#Thmtheorem5c), together with Table[2](https://arxiv.org/html/2610.41074#S4.T2), imply the*necessary*condition for \([27](https://arxiv.org/html/2610.41074#S4.E27)\),pbw≥\[\(b−1\)​log4​M\+log4⁡c​ρ​σ2ϑ\]\+,p\_\{\\mathrm\{bw\}\}\\geq\\left\[\(b\-1\)\\log\_\{4\}M\+\\log\_\{4\}\\frac\{c\\rho\\sigma^\{2\}\}\{\\vartheta\}\\right\]\_\{\+\},wherec\>0c\>0is a fixed constant\. Thus backward precision must grow at least logarithmically with width to retain the prescribed relative accuracy\. Figure[1](https://arxiv.org/html/2610.41074#S1.F1)\(c\) uses equal forward and backward precisions for the last iterate\. Among one to four tested bit counts, the smallest that keeps additional SR\-induced risk within1%1\\%of exact arithmetic risk is two bits atM=128M=128and three atM=256M=256\. In practice, we should choose precision alongside width and the output: fixed backward bits can prevent the model from keeping its improving exact\-arithmetic accuracy as width grows\. When both computations are rounded, we should carefully consider their interaction in addition to their separate contributions before comparing compute costs in Section[4\.5](https://arxiv.org/html/2610.41074#S4.SS5)\.

Figure 2:\(a\) With five mantissa bits in total, more forward bits reduce last\-iterate error; averaged\-output error is smallest at\(pfw,pbw\)=\(2,3\)\(p\_\{\\mathrm\{fw\}\},p\_\{\\mathrm\{bw\}\}\)=\(2,3\)\. Errors are relative to the corresponding exact\-arithmetic risks\. \(b\) The orange schedule accounts for backward\-rounding error using \([31](https://arxiv.org/html/2610.41074#S4.E31)\); the blue schedule uses the exact\-arithmetic design \([30](https://arxiv.org/html/2610.41074#S4.E30)\)\. Both use one\-bit backward SR; the dotted line is a constant learning rate\. \(c\) Full last\-iterate prediction risk under the precision\-dependent design minus that under the exact\-SGD design, at the same backward precision\. Positive values favor the exact\-SGD design\. In \(c\), points average eight independent regression problems\. Forward features are exact in \(b,c\)\. Appendix[F\.2](https://arxiv.org/html/2610.41074#A6.SS2)gives details\.
### 4\.4How precision changes learning\-rate design

How should learning\-rate decay change in low precision training? Fix widthMM, batch sizeBB, horizonTT\. We keep forward features exact and consider a constant backward precisionpbwp\_\{\\mathrm\{bw\}\}and last\-iterate output\. Consider learning\-rate upper boundηmax\\eta\_\{\\max\}defined in Appendix[E\.4](https://arxiv.org/html/2610.41074#A5.SS4), chosen before the sketch is drawn\. For either schedule below, we use continuous timet∈\[0,T\]t\\in\[0,T\]and writeτt=∫0tη⁡\(s\)​𝑑s\\tau\_\{t\}=\\int\_\{0\}^\{t\}\\eta\(s\)\\,ds, withτ=τT\\tau=\\tau\_\{T\}\. The constantsλ0\\lambda\_\{0\}andλbw\>0\\lambda\_\{\\mathrm\{bw\}\}\>0are chosen separately so that the total learning rate accumulates toτ\\tau\.

Following the variational approach of[Bordelon and Mori \(2026\)](https://arxiv.org/html/2610.41074#bib.bib2), minimizing our continuous scalar risk objective at fixedτ\\taugives the exact\-arithmetic schedule

ηexact​\(t\)=min⁡\{ηmax,λ0​\[1\+2​\(τ−τt\)\]1−1/\(2​a\)\}\.\\eta\_\{\\rm exact\}\(t\)=\\min\\left\\\{\\eta\_\{\\max\},\\,\\sqrt\{\\lambda\_\{0\}\}\\,\[1\+2\(\\tau\-\\tau\_\{t\}\)\]^\{1\-1/\(2a\)\}\\right\\\}\.\(30\)With backward rounding, minimizing the corresponding continuous scalar objective leads to

ηSR​\(t\)=min⁡\{ηmax,λbw​\[1\+2​\(τ−τt\)\]1−1/\(2​a\)1\+4−pbw​\[1\+2​\(τ−τt\)\]/M\}\.\\eta\_\{\\rm SR\}\(t\)=\\min\\left\\\{\\eta\_\{\\max\},\\,\\frac\{\\sqrt\{\\lambda\_\{\\mathrm\{bw\}\}\}\\,\[1\+2\(\\tau\-\\tau\_\{t\}\)\]^\{1\-1/\(2a\)\}\}\{\\sqrt\{1\+4^\{\-p\_\{\\mathrm\{bw\}\}\}\[1\+2\(\\tau\-\\tau\_\{t\}\)\]/M\}\}\\right\\\}\.\(31\)Here ifpbw=∞p\_\{\\mathrm\{bw\}\}=\\infty, the denominator equals one and the normalizationλbw=λ0\\lambda\_\{\\mathrm\{bw\}\}=\\lambda\_\{0\}recovers \([30](https://arxiv.org/html/2610.41074#S4.E30)\)\. After optimizing the total learning rate and averaging over update intervals, these formulas give discrete rates whose last\-iterate risk is within a fixed factor of the best admissible schedule at the sameM,B,T,pbwM,B,T,p\_\{\\mathrm\{bw\}\}\(Theorem[E\.7](https://arxiv.org/html/2610.41074#Thmtheorem7c)\)\.

The precision\-dependent term4−pbw​\[1\+2​\(τ−τt\)\]/M4^\{\-p\_\{\\mathrm\{bw\}\}\}\[1\+2\(\\tau\-\\tau\_\{t\}\)\]/Min \([31](https://arxiv.org/html/2610.41074#S4.E31)\) is largest at the start of training and decreases as training progresses\. If4−pbw​\(1\+2​τ\)/M≪14^\{\-p\_\{\\mathrm\{bw\}\}\}\(1\+2\\tau\)/M\\ll 1, this correction is small and the exact\-SGD design in \([30](https://arxiv.org/html/2610.41074#S4.E30)\) remains appropriate\. When the rate is belowηmax\\eta\_\{\\max\},τ−τt\\tau\-\\tau\_\{t\}is large, and this term is much larger than one, the time\-domain power\-law approximation has exponent\(a−1\)/\(a\+1\)\(a\-1\)/\(a\+1\), compared with2​a−12a\-1for exact arithmetic\. The practical lesson is to tune precision and learning\-rate schedulers together\. Lower precision does not universally imply a lower peak or an earlier start of decay for learning rate\. Figure[2](https://arxiv.org/html/2610.41074#S4.F2)\(b\) illustrates how the learning rate schedule changes with the one\-bit backward rounding, compared to the exact\-SGD learning rate schedule\. Figure[2](https://arxiv.org/html/2610.41074#S4.F2)\(c\) shows that the precision\-dependent learning rate design has slightly higher prediction risk; the difference is smaller at higher precisions rounding\.

### 4\.5Model width, training duration, and precision at fixed compute

Given compute budgetFFin \([23](https://arxiv.org/html/2610.41074#S3.E23)\), how should model width, training duration, and precision be chosen, and when can lower precision even improve prediction risk? We now use the uniformly averaged iterate with fixed learning rateη\\eta, batchBBand label noiseσ2\\sigma^\{2\}\. We consider the setting of Section[3\.2](https://arxiv.org/html/2610.41074#S3.SS2), accuracy toleranceϑ\\varthetain \([27](https://arxiv.org/html/2610.41074#S4.E27)\) and compute budgetFFin \([23](https://arxiv.org/html/2610.41074#S3.E23)\)\.

##### Fixed precisions: balancing width and training duration\.

For fixed precisionspfw,pbwp\_\{\\mathrm\{fw\}\},p\_\{\\mathrm\{bw\}\}over training, the compute budget in \([23](https://arxiv.org/html/2610.41074#S3.E23)\) boundsM​τM\\tau, so increasing width shortens the affordable training duration\. In \([19](https://arxiv.org/html/2610.41074#S3.E19)\), larger width reduces approximation error, while longer training reduces optimization error\. Lower precision reduces compute cost, allowing a wider model and longer training\. These additional training steps and larger model reduce approximation and optimization error in \([19](https://arxiv.org/html/2610.41074#S3.E19)\)\. But they improve full prediction risk only when those gains exceed the additional SR\-induced errorΔavg​\(T\)\\Delta^\{\\rm avg\}\(T\)\. Specifically, \([233](https://arxiv.org/html/2610.41074#A5.E233)\) in Appendix[E](https://arxiv.org/html/2610.41074#A5)gives sufficient conditions onM,τM,\\tauto ensure the accuracy tolerance in \([27](https://arxiv.org/html/2610.41074#S4.E27)\)\. In practice, this means we should choose widthMMand training durationTTjointly like the result in Table[1](https://arxiv.org/html/2610.41074#S3.T1)\. Lower precisions can provide more choices of model sizes and training time given a fixed compute budget\.

##### Growing precisions: maintaining accuracy as width increases\.

How should precision change when model width and training duration increase? Forτ≍Ma\\tau\\asymp M^\{a\}, Proposition[E\.8](https://arxiv.org/html/2610.41074#Thmtheorem8a)simplifies \([29](https://arxiv.org/html/2610.41074#S4.E29)\) and provespbw≥b−12​log2​M−O⁡\(1\)p\_\{\\mathrm\{bw\}\}\\geq\\frac\{b\-1\}\{2\}\\log\_\{2\}M\-O\(1\)andpfw\+pbw≥a−12​log2​M−O⁡\(1\)p\_\{\\mathrm\{fw\}\}\+p\_\{\\mathrm\{bw\}\}\\geq\\frac\{a\-1\}\{2\}\\log\_\{2\}M\-O\(1\)to ensure risk tolerance in \([27](https://arxiv.org/html/2610.41074#S4.E27)\)\. TheO⁡\(1\)O\(1\)terms only depend on theσ\\sigma,η\\eta,ϑ\\varthetaand constants\. Backward precision must grow at least logarithmically\. Forward precision can remain fixed if backward precision grows enough to meet the total\-bit condition\. Total mantissa length of orderlog⁡M\\log Mimplies computef≍Ma\+1​log⁡Mf\\asymp M^\{a\+1\}\\log Mwhich improves the decay exponent in the last row of Table[1](https://arxiv.org/html/2610.41074#S3.T1)by

ℛavg\(T\)≍\(F/logF\)−\(b−1\)/\(a\+1\)\.\\mathscr\{R\}^\{\\rm avg\}\(T\)\\asymp\(F/\\log F\)^\{\-\(b\-1\)/\(a\+1\)\}\.\(32\)This scaling only has an additionalpolylog⁡\(F\)\\rm\{polylog\}\(F\)factor comparing with optimal orderF−\(b−1\)/\(a\+1\)F^\{\-\(b\-1\)/\(a\+1\)\}from exact SGD training\. Proposition[E\.8](https://arxiv.org/html/2610.41074#Thmtheorem8a)proves \([32](https://arxiv.org/html/2610.41074#S4.E32)\)\. Hence, increasing precisions logarithmically as model size increases can prevent SR from becoming the accuracy bottleneck as training scales\.

Figure 3:Prediction risk of uniformly averaged outputs under a compute budget\. \(a,b\) Exact\-arithmetic and joint one\-bit SR training \(pfw=pbw=1p\_\{\\mathrm\{fw\}\}=p\_\{\\mathrm\{bw\}\}=1\) at several widths\. Black curves show the smallest prediction risks among tested widths and training durations\. Red dashed curves use the optimal\-compute scaling in Table[1](https://arxiv.org/html/2610.41074#S3.T1)for exact\-arithmetic training\. \(c\) At a fixed budgetFF, joint SR permits a wider model and longer training with lower prediction risk\. We average over three sketches\. Appendix[F\.3](https://arxiv.org/html/2610.41074#A6.SS3.SSS0.Px1)gives details\.
##### When does lower precision improve risk at a fixed budget?

For fixed width, time, and learning rate, SR cannot reduce the expected risk of the uniformly averaged iterate\. Its benefit requires cheaper updates to permit a better training configuration\. Compare lower\-cost \(LL\) and higher\-cost \(HH\) precision choices and fix other training setup the same\. LetTL,THT\_\{L\},T\_\{H\}be time horizons satisfying the same budget in \([23](https://arxiv.org/html/2610.41074#S3.E23)\), andΔL,ΔH\\Delta\_\{L\},\\Delta\_\{H\}their averaged SR\-induced risk increasesΔavg​\(T\)\\Delta^\{\\rm avg\}\(T\)\. Then the lower precision choice has smaller risk if and only if

ΔL−ΔH<ℛ0avg​\(TH\)−ℛ0avg​\(TL\)\.\\Delta\_\{L\}\-\\Delta\_\{H\}<\\mathscr\{R\}\_\{0\}^\{\\rm avg\}\(T\_\{H\}\)\-\\mathscr\{R\}\_\{0\}^\{\\rm avg\}\(T\_\{L\}\)\.\(33\)Appendix[E\.5\.2](https://arxiv.org/html/2610.41074#A5.SS5.SSS2)presents an example in which \([33](https://arxiv.org/html/2610.41074#S4.E33)\) holds\. This indicates that lower precision training can outperform exact\-arithmetic training at a fixed compute budget by training longer or with a wider model\. At fixed width and training duration, lower precision adds rounding error without using the computation it saves\. At a fixed compute budget, those savings can instead pay for more training updates or a wider model\. Lower precision can therefore reduce total prediction risk when the benefit of the additional training or model capacity exceeds the rounding error\.

## 5Experiments

In this section, we examine the compute tradeoffs in Section[4\.5](https://arxiv.org/html/2610.41074#S4.SS5), then study data reuse and neural\-network computations beyond our theoretical analysis\. Considera=2a=2,b=3/2b=3/2,σ2=0\.1\\sigma^\{2\}=0\.1,B=32B=32andd=4​Md=4M\. In simulations we may train beyondτT=Ma\\tau\_\{T\}=M^\{a\}, so these comparisons do not follow all assumptions of Theorem[1](https://arxiv.org/html/2610.41074#Thmtheorem1)\. Compute follows \([23](https://arxiv.org/html/2610.41074#S3.E23)\)\. Appendix[F](https://arxiv.org/html/2610.41074#A6)shows the experimental details\.

##### Width and training duration at fixed compute\.

Figure[3](https://arxiv.org/html/2610.41074#S4.F3)illustrates the joint choice of width, duration and precision in Section[4\.5](https://arxiv.org/html/2610.41074#S4.SS5)for uniformly averaged outputs\. Figure[3](https://arxiv.org/html/2610.41074#S4.F3)\(a,b\) compare prediction risks across widths\. Both red dashed curves use the exact\-arithmetic scalingF−1/6F^\{\-1/6\}in Table[1](https://arxiv.org/html/2610.41074#S3.T1)\. Figure[3](https://arxiv.org/html/2610.41074#S4.F3)\(b\) has better optimal scaling than the joint\-SR scalingF−1/8F^\{\-1/8\}in Table[1](https://arxiv.org/html/2610.41074#S3.T1), since simulation here does not impose all conditions of Theorem[1](https://arxiv.org/html/2610.41074#Thmtheorem1)\. In Figure[3](https://arxiv.org/html/2610.41074#S4.F3)\(c\), joint one\-bit SR permits a wider model and longer training, reducing mean prediction risk by9\.7%9\.7\\%at the same compute budget among the tested configurations\. The gain from a wider model and longer training outweighs the SR\-induced error: precision should be selected together with width and time by comparing full risks at a fixed compute budget\.

##### Data reuse at fixed compute\.

Figure[4](https://arxiv.org/html/2610.41074#S5.F4)asks whether SR updates remain useful when training revisits existing examples during training\. At fixed width and compute capF=227F=2^\{27\}, with fresh data or either reused dataset, lower precision allows longer training and wider model\. The uniform average output then achieves about 5% lower prediction risk than exact\-arithmetic training\. Figure[4](https://arxiv.org/html/2610.41074#S5.F4)\(b\) shows lower mean prediction risk for low\-precision training at every tested compute budget\. Additional training offsets the SR\-induced error, as in \([33](https://arxiv.org/html/2610.41074#S4.E33)\)\. The more data reused, the lower the prediction risk in Figure[4](https://arxiv.org/html/2610.41074#S5.F4)\(c\)\. Thus the compute benefit can persist with data reuse\. We leave a data\-reuse scaling law with SR for future work\.

Figure 4:Prediction risk under a fixedcompute budget with fresh and reused data\. Uniformly averaged outputs atM=256M=256, using exact arithmetic or joint two\-bit feature SR\. \(a\) Full prediction risks atF=227F=2^\{27\}\. \(b\) The difference between SR and exact prediction risks across budgets; negative values indicate SR is beneficial\. \(c\) Reuse\-minus\-fresh risk differences atF=227F=2^\{27\}for different data sizes\. For more details, see Appendix[F\.3](https://arxiv.org/html/2610.41074#A6.SS3.SSS0.Px2)\.
##### Precision choices in neural networks\.

Figure[5](https://arxiv.org/html/2610.41074#S5.F5)extends the location question in Section[4\.1](https://arxiv.org/html/2610.41074#S4.SS1)to a six\-layer transformer trained on 200M FineWeb\-Edu tokens and a CIFAR\-10 MLP\. The combined rounding configuration has higher transformer validation loss than activation\-only rounding in Figure[5](https://arxiv.org/html/2610.41074#S5.F5)\(a\)\. Our theory only considered independent rounding of features in \([4](https://arxiv.org/html/2610.41074#S2.E4)\)\. Sharing introduces dependence between rounding errors\. Figure[5](https://arxiv.org/html/2610.41074#S5.F5)\(b\) compares this independent rounding with shared rounding, which reuses the same rounded activation for the forward computation and the weight\-gradient computation\. Remarkably, it shows that sharing the rounded activation between forward and weight\-gradient computations also changes performance\. At one bit, sharing lowers the validation losses relative to independent rounding for both models\. Compared with training without SR, shared rounding gives lower loss for MLP but higher loss for transformer\. Figure[5](https://arxiv.org/html/2610.41074#S5.F5)\(c\) separates rounding during training from rounding at test time\. The one\-bit forward\-trained MLP has test loss1\.6241\.624without SR during test\-time predictions and1\.6721\.672with forward\-activation SR at test time\. Precision comparisons should therefore specify all rounded computations and whether activations are shared, and compare models with SR disabled at test time before assessing low\-precision inference\.

Figure 5:Rounding locations, shared activations and test\-time prediction in neural networks\. \(a\) Final transformer validation loss; the combined case rounds both activation computations, incoming gradients and forward weights to two bits, with seven\-bit parameter storage\. \(b\) Shared SR reuses the same forward\-rounded activation in the weight\-gradient computation\. Points show final loss differences from same\-seed training with SR and without SR\. Negative values indicate lower loss than this baseline\. \(c\) Each trained MLP is tested twice with its parameters fixed: with forward\-activation SR disabled, and with SR enabled at its training precision\. Both tests use the same examples\. Appendix[F\.5](https://arxiv.org/html/2610.41074#A6.SS5)gives details and final performance values\.

## 6Conclusion

We derived scaling laws for stochastic rounding in random\-feature regression\. Precision, model size and training time should be chosen together\. Under our compute model, lower precision can improve prediction at a fixed budget when the benefits of a larger model or longer training outweigh the added rounding error\. The learning rates and the choice to average model parameters also affect where and when higher precision is needed\. Extending the theory to data reuse, more quantization locations, feature learning regimes\([Ba et al\., 2022](https://arxiv.org/html/2610.41074#bib.bib60)\), and measuring savings on hardware, remain future work\.

##### Use of AI assistance\.

The authors used ChatGPT to develop proof candidates, especially for tracking constants in the proofs, and Claude and Codex to set up initial experiment codes and figures\. The authors are responsible for the results, their stated scope and the final manuscript\.

##### Acknowledgements\.

ZW and MWM would like to acknowledge the NSF and the DARPA DIAL and DARPA AIQ programs for partial support of this work\.

## References

- J\. Ba, M\. A\. Erdogdu, T\. Suzuki, Z\. Wang, D\. Wu, and G\. YangHigh\-dimensional asymptotics of feature learning: how one gradient step improves the representation\.Advances in Neural Information Processing Systems35,pp\. 37932–37946\.Cited by:[§6](https://arxiv.org/html/2610.41074#S6.p1.1)\.
- Bach \(2024\)F\. BachHigh\-dimensional analysis of double descent for linear regression with random projections\.SIAM Journal on Mathematics of Data Science6\(1\),pp\. 26–50\.External Links:[Document](https://dx.doi.org/10.1137/23M1558781),2303\.01372,[Link](https://epubs.siam.org/doi/10.1137/23M1558781)Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px2.p1.1)\.
- Bahriet al\.\(2024\)Y\. Bahri, E\. Dyer, J\. Kaplan, J\. Lee, and U\. SharmaExplaining neural scaling laws\.Proceedings of the National Academy of Sciences121\(27\),pp\. e2311878121\.External Links:[Document](https://dx.doi.org/10.1073/pnas.2311878121),2102\.06701Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2610.41074#S1.p4.1)\.
- Bordelonet al\.\(2024\)B\. Bordelon, A\. Atanasov, and C\. PehlevanA dynamical model of neural scaling laws\.InProceedings of the 41st International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.235,pp\. 4345–4382\.External Links:2402\.01092,[Link](https://proceedings.mlr.press/v235/bordelon24a.html)Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2610.41074#S1.p4.1)\.
- Bordelon and Mori \(2026\)B\. Bordelon and F\. MoriTheory of optimal learning rate schedules and scaling laws for a random feature model\.Note:Version 2External Links:2602\.04774,[Link](https://arxiv.org/abs/2602.04774v2)Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px2.p1.1),[§C\.3](https://arxiv.org/html/2610.41074#A3.SS3.p1.1),[§E\.4\.2](https://arxiv.org/html/2610.41074#A5.SS4.SSS2.Px1.p1.3),[§E\.4](https://arxiv.org/html/2610.41074#A5.SS4.p1.1),[§1](https://arxiv.org/html/2610.41074#S1.p4.1),[§4\.4](https://arxiv.org/html/2610.41074#S4.SS4.p2.1)\.
- Bordelon and Pehlevan \(2022\)B\. Bordelon and C\. PehlevanLearning curves for SGD on structured features\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=WPI2vbkAl3Q),2106\.02713Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px2.p1.1)\.
- Boutsikaset al\.\(2024\)C\. Boutsikas, P\. Drineas, and I\. C\. F\. IpsenSmall singular values can increase in lower precision\.SIAM Journal on Matrix Analysis and Applications45\(3\),pp\. 1518–1540\.External Links:[Document](https://dx.doi.org/10.1137/23M1557209),2303\.03547Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px5.p1.1),[§1](https://arxiv.org/html/2610.41074#S1.p2.1)\.
- Castroet al\.\(2025\)R\. L\. Castro, A\. Panferov, S\. Tabesh, O\. Sieberling, J\. Chen, M\. Nikdan, S\. Ashkboos, and D\. AlistarhQuartet: native FP4 training can be optimal for large language models\.InAdvances in Neural Information Processing Systems,Vol\.38\.Note:Version 4, revised January 2026External Links:2505\.14669,[Link](https://arxiv.org/abs/2505.14669v4)Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px4.p1.1),[§1](https://arxiv.org/html/2610.41074#S1.p1.1)\.
- Chenet al\.\(2020\)J\. Chen, Y\. Gai, Z\. Yao, M\. W\. Mahoney, and J\. E\. GonzalezA statistical framework for low\-bitwidth training of deep neural networks\.InAdvances in Neural Information Processing Systems,Vol\.33\.External Links:2010\.14298Cited by:[§1](https://arxiv.org/html/2610.41074#S1.p1.1)\.
- Chenet al\.\(2021\)J\. Chen, L\. Zheng, Z\. Yao, D\. Wang, I\. Stoica, M\. W\. Mahoney, and J\. E\. GonzalezActNN: reducing training memory footprint via 2\-bit activation compressed training\.InProceedings of the 38th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.139,pp\. 1803–1813\.External Links:2104\.14129Cited by:[§1](https://arxiv.org/html/2610.41074#S1.p1.1)\.
- Chenet al\.\(2026a\)M\. Chen, C\. Zhang, J\. Liu, Y\. Zeng, Z\. Xue, Z\. Liu, Y\. Li, J\. Ma, J\. Huang, X\. Zhou, and P\. LuoScaling law for quantization\-aware training\.InProceedings of the 43rd International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.306,pp\. 18317–18336\.External Links:2505\.14302,[Link](https://proceedings.mlr.press/v306/chen26gl.html)Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2610.41074#S1.p3.1)\.
- Chenet al\.\(2026b\)Z\. Chen, Z\. Zhou, and D\. ZhouScaling laws for dynamic mini\-batch SGD in sketched linear regression\.Note:Version 3\. Earlier versions titled From One\-Pass SGD to Data Reuse: Mini\-Batch Scaling Laws in Sketched Linear RegressionExternal Links:2605\.24316,[Link](https://arxiv.org/abs/2605.24316v3)Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2610.41074#S1.p4.1)\.
- Cheng and Montanari \(2024\)C\. Cheng and A\. MontanariDimension free ridge regression\.The Annals of Statistics52\(6\),pp\. 2879–2912\.External Links:2210\.08571,[Link](https://arxiv.org/abs/2210.08571)Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px2.p1.1)\.
- Chmielet al\.\(2025\)B\. Chmiel, M\. Fishman, R\. Banner, and D\. SoudryFP4 all the way: fully quantized training of LLMs\.External Links:2505\.19115,[Link](https://arxiv.org/abs/2505.19115)Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px4.p1.1),[§1](https://arxiv.org/html/2610.41074#S1.p1.1)\.
- Connollyet al\.\(2021\)M\. P\. Connolly, N\. J\. Higham, and T\. MaryStochastic rounding and its probabilistic backward error analysis\.SIAM Journal on Scientific Computing43\(1\),pp\. A566–A585\.External Links:[Document](https://dx.doi.org/10.1137/20M1334796),[Link](https://epubs.siam.org/doi/10.1137/20M1334796)Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px5.p1.1),[§1](https://arxiv.org/html/2610.41074#S1.p2.1)\.
- Courbariauxet al\.\(2015\)M\. Courbariaux, Y\. Bengio, and J\. DavidBinaryConnect: training deep neural networks with binary weights during propagations\.InAdvances in Neural Information Processing Systems,Vol\.28,pp\. 3123–3131\.External Links:1511\.00363Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px4.p1.1)\.
- Crociet al\.\(2022\)M\. Croci, M\. Fasi, N\. J\. Higham, T\. Mary, and M\. MikaitisStochastic rounding: implementation, error analysis and applications\.Royal Society Open Science9\(3\),pp\. 211631\.External Links:[Document](https://dx.doi.org/10.1098/rsos.211631),[Link](https://doi.org/10.1098/rsos.211631)Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px5.p1.1),[§1](https://arxiv.org/html/2610.41074#S1.p2.1)\.
- De Saet al\.\(2018\)C\. De Sa, M\. Leszczynski, J\. Zhang, A\. Marzoev, C\. R\. Aberger, K\. Olukotun, and C\. RéHigh\-accuracy low\-precision training\.External Links:1803\.03383,[Link](https://arxiv.org/abs/1803.03383)Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px4.p1.1),[§1](https://arxiv.org/html/2610.41074#S1.p2.1)\.
- DeepSeek\-AI \(2024\)DeepSeek\-AIDeepSeek\-V3 technical report\.External Links:2412\.19437,[Link](https://arxiv.org/abs/2412.19437)Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px4.p1.1),[§1](https://arxiv.org/html/2610.41074#S1.p1.1)\.
- Defilippiset al\.\(2024\)L\. Defilippis, B\. Loureiro, and T\. MisiakiewiczDimension\-free deterministic equivalents and scaling laws for random feature regression\.InAdvances in Neural Information Processing Systems,Vol\.37\.External Links:2405\.15699,[Link](https://proceedings.neurips.cc/paper_files/paper/2024/hash/bd18189308a4c45c7d71ca83acf3deaa-Abstract-Conference.html)Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px2.p1.1)\.
- Dexteret al\.\(2025\)G\. Dexter, C\. Boutsikas, L\. Ma, I\. C\. F\. Ipsen, and P\. DrineasStochastic rounding implicitly regularizes tall\-and\-thin matrices\.SIAM Journal on Matrix Analysis and Applications46\(1\),pp\. 341–369\.External Links:[Document](https://dx.doi.org/10.1137/24M1647679),2403\.12278Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px5.p1.1),[§1](https://arxiv.org/html/2610.41074#S1.p2.1)\.
- Drineas and Ipsen \(2024\)P\. Drineas and I\. C\. F\. IpsenStochastic rounding 2\.0, with a view towards complexity analysis\.External Links:2410\.10517,[Link](https://arxiv.org/abs/2410.10517)Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px5.p1.1),[§1](https://arxiv.org/html/2610.41074#S1.p2.1)\.
- El Araret al\.\(2026\)E\. El Arar, M\. Fasi, S\. Filip, and M\. MikaitisWhat is new in stochastic rounding: a survey on theory, hardware, and applications\.Note:Version 2External Links:2603\.06060,[Link](https://arxiv.org/abs/2603.06060v2)Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px5.p1.1),[§1](https://arxiv.org/html/2610.41074#S1.p2.1)\.
- El Araret al\.\(2023\)E\. El Arar, D\. Sohier, P\. de Oliveira Castro, and E\. PetitStochastic rounding variance and probabilistic bounds: a new approach\.SIAM Journal on Scientific Computing45\(5\),pp\. C255–C275\.External Links:[Document](https://dx.doi.org/10.1137/22M1510819),2207\.10321,[Link](https://epubs.siam.org/doi/10.1137/22M1510819)Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px5.p1.1),[§1](https://arxiv.org/html/2610.41074#S1.p2.1)\.
- Esseret al\.\(2020\)S\. K\. Esser, J\. L\. McKinstry, D\. Bablani, R\. Appuswamy, and D\. S\. ModhaLearned step size quantization\.InInternational Conference on Learning Representations,External Links:1902\.08153Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px4.p1.1),[§1](https://arxiv.org/html/2610.41074#S1.p1.1)\.
- Fitzgibbon and Felix \(2025\)A\. Fitzgibbon and S\. FelixOn stochastic rounding with few random bits\.In2025 IEEE 32nd Symposium on Computer Arithmetic \(ARITH\),pp\. 133–140\.External Links:[Document](https://dx.doi.org/10.1109/ARITH64983.2025.00029),2504\.20634,[Link](https://arxiv.org/abs/2504.20634)Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px5.p1.1)\.
- Frantaret al\.\(2025\)E\. Frantar, U\. Evci, W\. Park, N\. Houlsby, and D\. AlistarhCompression scaling laws: unifying sparsity and quantization\.External Links:2502\.16440,[Link](https://arxiv.org/abs/2502.16440)Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2610.41074#S1.p3.1)\.
- Guptaet al\.\(2015\)S\. Gupta, A\. Agrawal, K\. Gopalakrishnan, and P\. NarayananDeep learning with limited numerical precision\.InProceedings of the 32nd International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.37,pp\. 1737–1746\.External Links:1502\.02551,[Link](https://proceedings.mlr.press/v37/gupta15.html)Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px4.p1.1),[§1](https://arxiv.org/html/2610.41074#S1.p2.1)\.
- Higham and Mary \(2019\)N\. J\. Higham and T\. MaryA new approach to probabilistic rounding error analysis\.SIAM Journal on Scientific Computing41\(5\),pp\. A2815–A2835\.External Links:[Document](https://dx.doi.org/10.1137/18M1226312)Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px5.p1.1),[§1](https://arxiv.org/html/2610.41074#S1.p2.1)\.
- Higham and Mary \(2020\)N\. J\. Higham and T\. MarySharper probabilistic backward error analysis for basic linear algebra kernels with random data\.SIAM Journal on Scientific Computing42\(5\),pp\. A3427–A3446\.External Links:[Document](https://dx.doi.org/10.1137/20M1314355)Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px5.p1.1),[§1](https://arxiv.org/html/2610.41074#S1.p2.1)\.
- Hodgkinsonet al\.\(2025\)L\. Hodgkinson, Z\. Wang, and M\. W\. MahoneyModels of heavy\-tailed mechanistic universality\.InProceedings of the 42nd International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.267,pp\. 23290–23329\.External Links:2506\.03470,[Link](https://proceedings.mlr.press/v267/hodgkinson25a.html)Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px2.p1.1)\.
- Hoffmannet al\.\(2022\)J\. Hoffmann, S\. Borgeaud, A\. Mensch, E\. Buchatskaya, T\. Cai, E\. Rutherford,et al\.Training compute\-optimal large language models\.InAdvances in Neural Information Processing Systems,Vol\.35\.External Links:2203\.15556Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2610.41074#S1.p3.1)\.
- Jacobet al\.\(2018\)B\. Jacob, S\. Kligys, B\. Chen, M\. Zhu, M\. Tang, A\. Howard, H\. Adam, and D\. KalenichenkoQuantization and training of neural networks for efficient integer\-arithmetic\-only inference\.InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition \(CVPR\),pp\. 2704–2713\.External Links:1712\.05877Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px4.p1.1),[§1](https://arxiv.org/html/2610.41074#S1.p1.1)\.
- Kaplanet al\.\(2020\)J\. Kaplan, S\. McCandlish, T\. Henighan, T\. B\. Brown, B\. Chess, R\. Child, S\. Gray, A\. Radford, J\. Wu, and D\. AmodeiScaling laws for neural language models\.External Links:2001\.08361,[Link](https://arxiv.org/abs/2001.08361)Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2610.41074#S1.p3.1)\.
- Kimet al\.\(2026\)J\. Kim, R\. Saha, M\. F\. Balin, and Y\. ParkMixed precision scaling laws for quantized pretraining\.Note:Version 1External Links:2610\.17637,[Link](https://arxiv.org/abs/2610.17637v1)Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2610.41074#S1.p4.1),[§3\.3](https://arxiv.org/html/2610.41074#S3.SS3.p1.2)\.
- Kumaret al\.\(2025\)T\. Kumar, Z\. Ankner, B\. F\. Spector, B\. Bordelon, N\. Muennighoff, M\. Paul, C\. Pehlevan, C\. Ré, and A\. RaghunathanScaling laws for precision\.InInternational Conference on Learning Representations,External Links:2411\.04330,[Link](https://proceedings.iclr.cc/paper_files/paper/2025/hash/b2cac94f82928a85055987d9fd44753f-Abstract-Conference.html)Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2610.41074#S1.p3.1),[§3\.3](https://arxiv.org/html/2610.41074#S3.SS3.p1.2)\.
- LeJeuneet al\.\(2024\)D\. LeJeune, P\. Patil, H\. Javadi, R\. G\. Baraniuk, and R\. J\. TibshiraniAsymptotics of the sketched pseudoinverse\.SIAM Journal on Mathematics of Data Science6\(1\),pp\. 199–225\.Note:Version 4, revised October 2023External Links:[Document](https://dx.doi.org/10.1137/22M1530264),2211\.03751,[Link](https://arxiv.org/abs/2211.03751v4)Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px2.p1.1)\.
- Liet al\.\(2017\)H\. Li, S\. De, Z\. Xu, C\. Studer, H\. Samet, and T\. GoldsteinTraining quantized nets: a deeper understanding\.InAdvances in Neural Information Processing Systems,Vol\.30,pp\. 5811–5821\.External Links:1706\.02379Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px4.p1.1),[§1](https://arxiv.org/html/2610.41074#S1.p2.1)\.
- Linet al\.\(2025\)L\. Lin, J\. Wu, and P\. L\. BartlettImproved scaling laws in linear regression via data reuse\.InAdvances in Neural Information Processing Systems,Vol\.38\.External Links:2506\.08415,[Link](https://arxiv.org/abs/2506.08415)Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2610.41074#S1.p4.1)\.
- Linet al\.\(2024\)L\. Lin, J\. Wu, S\. M\. Kakade, P\. L\. Bartlett, and J\. D\. LeeScaling laws in linear regression: compute, parameters, and data\.InAdvances in Neural Information Processing Systems,Vol\.37\.External Links:2406\.08466,[Link](https://proceedings.neurips.cc/paper_files/paper/2024/hash/6fcb1afcc1e9c2c82c8ddddf03bcf0f6-Abstract-Conference.html)Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px2.p1.1),[§C\.3](https://arxiv.org/html/2610.41074#A3.SS3.p1.1),[§1](https://arxiv.org/html/2610.41074#S1.p4.1)\.
- Liuet al\.\(2022\)X\. Liu, L\. Zheng, D\. Wang, Y\. Cen, W\. Chen, X\. Han, J\. Chen, Z\. Liu, J\. Tang, J\. Gonzalez, M\. Mahoney, and A\. CheungGACT: activation compressed training for generic network architectures\.InProceedings of the 39th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.162,pp\. 14139–14152\.External Links:[Link](https://proceedings.mlr.press/v162/liu22v.html)Cited by:[§1](https://arxiv.org/html/2610.41074#S1.p1.1)\.
- Liuet al\.\(2024\)Z\. Liu, B\. Oguz, C\. Zhao, E\. Chang, P\. Stock, Y\. Mehdad, Y\. Shi, R\. Krishnamoorthi, and V\. ChandraLLM\-QAT: data\-free quantization aware training for large language models\.InFindings of the Association for Computational Linguistics: ACL 2024,pp\. 467–484\.External Links:2305\.17888,[Link](https://aclanthology.org/2024.findings-acl.26/)Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px4.p1.1),[§1](https://arxiv.org/html/2610.41074#S1.p1.1)\.
- Maet al\.\(2026\)L\. Ma, T\. Yu, and P\. DrineasStochastic rounding increases small singular values\.External Links:2606\.00312,[Link](https://arxiv.org/abs/2606.00312)Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px5.p1.1),[§1](https://arxiv.org/html/2610.41074#S1.p2.1)\.
- Maloneyet al\.\(2022\)A\. Maloney, D\. A\. Roberts, and J\. SullyA solvable model of neural scaling laws\.External Links:2210\.16859,[Link](https://arxiv.org/abs/2210.16859)Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2610.41074#S1.p4.1)\.
- Martin and Mahoney \(2019\)C\. H\. Martin and M\. W\. MahoneyTraditional and heavy\-tailed self regularization in neural network models\.InProceedings of the 36th International Conference on Machine Learning,pp\. 4284–4293\.Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px2.p1.1)\.
- Martin and Mahoney \(2020\)C\. H\. Martin and M\. W\. MahoneyHeavy\-tailed Universality predicts trends in test accuracies for very large pre\-trained deep neural networks\.InProceedings of the 20th SIAM International Conference on Data Mining,pp\. 505–513\.External Links:[Document](https://dx.doi.org/10.1137/1.9781611976236.57)Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px2.p1.1)\.
- Martinet al\.\(2021\)C\. H\. Martin, T\. S\. Peng, and M\. W\. MahoneyPredicting trends in the quality of state\-of\-the\-art neural networks without access to training or testing data\.Nature Communications12,pp\. 4122\.External Links:[Document](https://dx.doi.org/10.1038/s41467-021-24025-8)Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px2.p1.1)\.
- Meckes \(2019\)E\. S\. MeckesThe random matrix theory of the classical compact groups\.Cambridge Tracts in Mathematics, Vol\.218,Cambridge University Press\.External Links:[Document](https://dx.doi.org/10.1017/9781108303453),[Link](https://case.edu/artsci/math/mwmeckes/elizabeth/Haar_book.pdf)Cited by:[§B\.3](https://arxiv.org/html/2610.41074#A2.SS3.p8.1.1),[§B\.3](https://arxiv.org/html/2610.41074#A2.SS3.p9.1.1)\.
- Mei and Montanari \(2022\)S\. Mei and A\. MontanariThe generalization error of random features regression: precise asymptotics and the double descent curve\.Communications on Pure and Applied Mathematics75\(4\),pp\. 667–766\.External Links:[Document](https://dx.doi.org/10.1002/cpa.22008),[Link](https://onlinelibrary.wiley.com/doi/10.1002/cpa.22008)Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px2.p1.1)\.
- Micikeviciuset al\.\(2018\)P\. Micikevicius, S\. Narang, J\. Alben, G\. Diamos, E\. Elsen, D\. Garcia, B\. Ginsburg, M\. Houston, O\. Kuchaiev, G\. Venkatesh, and H\. WuMixed precision training\.InInternational Conference on Learning Representations,External Links:1710\.03740Cited by:[§1](https://arxiv.org/html/2610.41074#S1.p1.1)\.
- Micikeviciuset al\.\(2022\)P\. Micikevicius, D\. Stosic, N\. Burgess, M\. Cornea, P\. Dubey, R\. Grisenthwaite, S\. Ha, A\. Heinecke, P\. Judd, J\. Kamalu, N\. Mellempudi, S\. Oberman, M\. Shoeybi, M\. Siu, and H\. WuFP8 formats for deep learning\.External Links:2209\.05433,[Link](https://arxiv.org/abs/2209.05433)Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px4.p1.1),[§1](https://arxiv.org/html/2610.41074#S1.p1.1)\.
- Ozkaraet al\.\(2025\)K\. Ozkara, T\. Yu, and Y\. ParkStochastic rounding for LLM training: theory and practice\.InProceedings of the 28th International Conference on Artificial Intelligence and Statistics,Proceedings of Machine Learning Research, Vol\.258,pp\. 4402–4410\.External Links:2502\.20566,[Link](https://proceedings.mlr.press/v258/ozkara25b.html)Cited by:[§1](https://arxiv.org/html/2610.41074#S1.p2.1)\.
- Panferovet al\.\(2026\)A\. Panferov, E\. Schultheis, S\. Tabesh, and D\. AlistarhQuartet II: accurate LLM pre\-training in NVFP4 by improved unbiased gradient estimation\.Note:Version 2External Links:2601\.22813,[Link](https://arxiv.org/abs/2601.22813v2)Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px4.p1.1),[§1](https://arxiv.org/html/2610.41074#S1.p1.1)\.
- Paquetteet al\.\(2025\)C\. Paquette, E\. Paquette, B\. Adlam, and J\. PenningtonHomogenization of SGD in high\-dimensions: exact dynamics and generalization properties\.Mathematical Programming214,pp\. 1–90\.External Links:[Document](https://dx.doi.org/10.1007/s10107-024-02171-3),[Link](https://link.springer.com/article/10.1007/s10107-024-02171-3)Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2610.41074#S1.p4.1)\.
- Paquetteet al\.\(2024\)E\. Paquette, C\. Paquette, L\. Xiao, and J\. Pennington4\+3 phases of compute\-optimal neural scaling laws\.InAdvances in Neural Information Processing Systems,Vol\.37\.External Links:2405\.15074,[Link](https://proceedings.neurips.cc/paper_files/paper/2024/hash/1dccfc3ee01871d05e33457c61037d59-Abstract-Conference.html)Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2610.41074#S1.p4.1),[§2](https://arxiv.org/html/2610.41074#S2.SS0.SSS0.Px1.p1.2)\.
- Polyak and Juditsky \(1992\)B\. T\. Polyak and A\. B\. JuditskyAcceleration of stochastic approximation by averaging\.SIAM journal on control and optimization30\(4\),pp\. 838–855\.Cited by:[§2](https://arxiv.org/html/2610.41074#S2.SS0.SSS0.Px4.p1.3)\.
- Subramanianet al\.\(2026\)S\. Subramanian, A\. Kiefer, A\. Nigmetov, A\. Gholami, D\. Morozov, and M\. W\. MahoneyOn neural scaling laws for weather emulation through continual training\.arXiv preprint arXiv:2603\.25687\.Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2610.41074#S1.p3.1)\.
- Sunet al\.\(2025\)X\. Sun, S\. Li, R\. Xie, W\. Han, K\. Wu, Z\. Yang, Y\. Li, A\. Wang, S\. Li, J\. Xue, Y\. Cheng, Y\. Tao, Z\. Kang, C\. Xu, D\. Wang, and J\. JiangScaling laws for floating point quantization training\.InProceedings of the 42nd International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.267,pp\. 57544–57570\.External Links:2501\.02423,[Link](https://proceedings.mlr.press/v267/sun25j.html)Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2610.41074#S1.p3.1),[§3\.3](https://arxiv.org/html/2610.41074#S3.SS3.p1.2)\.
- Tropp \(2022\)J\. A\. TroppACM 204: matrix analysis\.Caltech CMS Lecture NotesTechnical Report2022\-01,California Institute of Technology\.External Links:[Document](https://dx.doi.org/10.7907/nwsv-df59),[Link](https://tropp.caltech.edu/notes/Tro22-Matrix-Analysis-LN.pdf)Cited by:[§B\.3](https://arxiv.org/html/2610.41074#A2.SS3.p7.1.1)\.
- Tsenget al\.\(2025\)A\. Tseng, T\. Yu, and Y\. ParkTraining LLMs with MXFP4\.InProceedings of the 28th International Conference on Artificial Intelligence and Statistics,Proceedings of Machine Learning Research, Vol\.258,pp\. 1630–1638\.External Links:2502\.20586,[Link](https://proceedings.mlr.press/v258/tseng25a.html)Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px4.p1.1),[§1](https://arxiv.org/html/2610.41074#S1.p1.1)\.
- Vershynin \(2011\)R\. VershyninIntroduction to the non\-asymptotic analysis of random matrices\.Note:Version 7External Links:1011\.3027,[Link](https://arxiv.org/abs/1011.3027v7)Cited by:[§B\.3](https://arxiv.org/html/2610.41074#A2.SS3.p3.1.1)\.
- Wolfe and Kyrillidis \(2024\)C\. R\. Wolfe and A\. KyrillidisBetter schedules for low precision training of deep neural networks\.Machine Learning113,pp\. 3569–3587\.External Links:[Document](https://dx.doi.org/10.1007/s10994-023-06480-0),2403\.02243,[Link](https://arxiv.org/abs/2403.02243)Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px4.p1.1)\.
- Wuet al\.\(2022\)J\. Wu, D\. Zou, V\. Braverman, Q\. Gu, and S\. KakadeLast iterate risk bounds of SGD with decaying stepsize for overparameterized linear regression\.InProceedings of the 39th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.162,pp\. 24280–24314\.External Links:[Link](https://proceedings.mlr.press/v162/wu22p.html)Cited by:[§C\.3](https://arxiv.org/html/2610.41074#A3.SS3.p1.1)\.
- Yanget al\.\(2019\)G\. Yang, T\. Zhang, P\. Kirichenko, J\. Bai, A\. G\. Wilson, and C\. De SaSWALP: stochastic weight averaging in low precision training\.InProceedings of the 36th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.97,pp\. 7015–7024\.External Links:[Link](https://proceedings.mlr.press/v97/yang19d.html)Cited by:[§2](https://arxiv.org/html/2610.41074#S2.SS0.SSS0.Px4.p1.3)\.
- Yuet al\.\(2022\)Z\. Yu, Y\. Fu, S\. Wu, M\. Li, H\. You, and Y\. LinLDP: learnable dynamic precision for efficient deep neural network training and inference\.External Links:2203\.07713,[Link](https://arxiv.org/abs/2203.07713)Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px4.p1.1)\.
- Zhanget al\.\(2026a\)D\. Zhang, J\. Su, and D\. ZouLearning under quantization for high\-dimensional linear regression\.InInternational Conference on Learning Representations,Note:Version 3, revised February 2026External Links:2510\.18259,[Link](https://arxiv.org/abs/2510.18259v3)Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px3.p1.1),[§1](https://arxiv.org/html/2610.41074#S1.p3.1)\.
- Zhanget al\.\(2026b\)D\. Zhang, X\. Tang, Y\. Liang, and D\. ZouScaling laws for precision in high\-dimensional linear regression\.InProceedings of the 43rd International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.306,pp\. 158654–158750\.External Links:[Link](https://proceedings.mlr.press/v306/zhang26fq.html),2602\.19241Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px3.p1.1),[§1](https://arxiv.org/html/2610.41074#S1.p3.1)\.
- Zhanget al\.\(2017\)H\. Zhang, J\. Li, K\. Kara, D\. Alistarh, J\. Liu, and C\. ZhangZipML: training linear models with end\-to\-end low precision, and a little bit of deep learning\.InProceedings of the 34th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.70,pp\. 4035–4043\.External Links:[Link](https://proceedings.mlr.press/v70/zhang17e.html)Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px4.p1.1),[§1](https://arxiv.org/html/2610.41074#S1.p2.1)\.
- Zhaoet al\.\(2025\)K\. Zhao, T\. Tabaru, K\. Kobayashi, T\. Honda, M\. Yamazaki, and Y\. TsuruokaDirect quantized training of language models with stochastic rounding\.InProceedings of the 17th Asian Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.304,pp\. 1150–1165\.External Links:2412\.04787Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px4.p1.1),[§1](https://arxiv.org/html/2610.41074#S1.p2.1)\.
- Zhouet al\.\(2016\)S\. Zhou, Y\. Wu, Z\. Ni, X\. Zhou, H\. Wen, and Y\. ZouDoReFa\-Net: training low bitwidth convolutional neural networks with low bitwidth gradients\.External Links:1606\.06160,[Link](https://arxiv.org/abs/1606.06160)Cited by:[Appendix A](https://arxiv.org/html/2610.41074#A1.SS0.SSS0.Px4.p1.1)\.

## Appendix ARelated work

This appendix reviews work on scaling laws, quantization and stochastic rounding, and explains how our model and results relate to these studies\.

##### Empirical scaling laws\.

Scaling laws try to analyze how loss changes as model size, sample size and compute increasing\([Kaplan et al\., 2020](https://arxiv.org/html/2610.41074#bib.bib43);[Hoffmann et al\., 2022](https://arxiv.org/html/2610.41074#bib.bib44)\), with extensions to continual weather training\([Subramanian et al\., 2026](https://arxiv.org/html/2610.41074#bib.bib64)\)\.[Kumar et al\. \(2025\)](https://arxiv.org/html/2610.41074#bib.bib12);[Sun et al\. \(2025\)](https://arxiv.org/html/2610.41074#bib.bib33)use fitted laws to choose model size, the number of data points and precision for a given compute budget\. Related work studies scaling in quantization\-aware training\([Chen et al\., 2026a](https://arxiv.org/html/2610.41074#bib.bib58)\)and joint quantization and sparsity\([Frantar et al\., 2025](https://arxiv.org/html/2610.41074#bib.bib30)\)\.[Kim et al\. \(2026\)](https://arxiv.org/html/2610.41074#bib.bib13)extend fitted laws to different forward and backward precisions and hardware\-dependent compute costs\.

##### Theory of scaling law\.

One of the solvable models for scaling law is the random feature model\([Mei and Montanari, 2022](https://arxiv.org/html/2610.41074#bib.bib17);[Bach, 2024](https://arxiv.org/html/2610.41074#bib.bib32);[LeJeune et al\., 2024](https://arxiv.org/html/2610.41074#bib.bib31);[Cheng and Montanari, 2024](https://arxiv.org/html/2610.41074#bib.bib5);[Defilippis et al\., 2024](https://arxiv.org/html/2610.41074#bib.bib10)\)\. Linear models with power\-law random features \(PLRF\) have been applied to theoretically understand the scaling law of prediction risk\([Bahri et al\., 2024](https://arxiv.org/html/2610.41074#bib.bib45);[Maloney et al\., 2022](https://arxiv.org/html/2610.41074#bib.bib46);[Bordelon et al\., 2024](https://arxiv.org/html/2610.41074#bib.bib47);[Bordelon and Pehlevan, 2022](https://arxiv.org/html/2610.41074#bib.bib1);[Paquette et al\., 2024](https://arxiv.org/html/2610.41074#bib.bib21);[Paquette et al\., 2025](https://arxiv.org/html/2610.41074#bib.bib20);[Lin et al\., 2024](https://arxiv.org/html/2610.41074#bib.bib14)\)\. These results are also extended to the setting of data\-reuse SGD\([Lin et al\., 2025](https://arxiv.org/html/2610.41074#bib.bib15)\), mini\-batch control\([Chen et al\., 2026b](https://arxiv.org/html/2610.41074#bib.bib4)\)and general learning\-rate schedules\([Bordelon and Mori, 2026](https://arxiv.org/html/2610.41074#bib.bib2)\)\. Power\-law spectra have also been observed and analyzed in trained weights of networks\([Hodgkinson et al\., 2025](https://arxiv.org/html/2610.41074#bib.bib59);[Martin and Mahoney, 2019](https://arxiv.org/html/2610.41074#bib.bib65);[Martin and Mahoney, 2020](https://arxiv.org/html/2610.41074#bib.bib66);[Martin et al\., 2021](https://arxiv.org/html/2610.41074#bib.bib67)\)\.

##### Quantization in linear regression\.

[Zhang et al\. \(2026a\)](https://arxiv.org/html/2610.41074#bib.bib26)derive prediction\-risk upper bounds for linear regression with quantized data, labels, parameters, activations and gradients\.[Zhang et al\. \(2026b\)](https://arxiv.org/html/2610.41074#bib.bib27)study scaling laws with model size and training data in sketched linear regression, giving upper and lower bounds for their quantization models\. Both use constant learning rates and parameter averaging\. Their general upper bounds allow unbiased quantization, while their scaling results make additional assumptions on rounding\-error covariance\. Our quantization from SR does not generally follow their multiplicative or additive quantization assumptions\. Our laws also cover last\-iterate predictor, and general learning\-rate and precision schedules\.

##### Low\-precision training\.

QAT uses low\-precision weights and activations during training for efficient inference\([Jacob et al\., 2018](https://arxiv.org/html/2610.41074#bib.bib35);[Esser et al\., 2020](https://arxiv.org/html/2610.41074#bib.bib39);[Liu et al\., 2024](https://arxiv.org/html/2610.41074#bib.bib40)\)\. Low\-precision training methods include BinaryConnect\([Courbariaux et al\., 2015](https://arxiv.org/html/2610.41074#bib.bib48)\), quantized gradients\([Zhou et al\., 2016](https://arxiv.org/html/2610.41074#bib.bib49)\), SR training\([Gupta et al\., 2015](https://arxiv.org/html/2610.41074#bib.bib41)\), and direct low\-precision weight updates\([Zhao et al\., 2025](https://arxiv.org/html/2610.41074#bib.bib51)\)\. Theoretical works can be found by[Li et al\. \(2017\)](https://arxiv.org/html/2610.41074#bib.bib42);[De Sa et al\. \(2018\)](https://arxiv.org/html/2610.41074#bib.bib9)\. Practical LLMs use FP8, MXFP4, and FP4 in training\([Micikevicius et al\., 2022](https://arxiv.org/html/2610.41074#bib.bib37);[DeepSeek\-AI, 2024](https://arxiv.org/html/2610.41074#bib.bib38);[Tseng et al\., 2025](https://arxiv.org/html/2610.41074#bib.bib22);[Castro et al\., 2025](https://arxiv.org/html/2610.41074#bib.bib3);[Chmiel et al\., 2025](https://arxiv.org/html/2610.41074#bib.bib6);[Panferov et al\., 2026](https://arxiv.org/html/2610.41074#bib.bib19)\)\.[Zhang et al\. \(2017\)](https://arxiv.org/html/2610.41074#bib.bib28)use independent rounding in \([4](https://arxiv.org/html/2610.41074#S2.E4)\) for low\-precision training\. Learned precision\([Yu et al\., 2022](https://arxiv.org/html/2610.41074#bib.bib25)\)and learning\-rate schedules\([Wolfe and Kyrillidis, 2024](https://arxiv.org/html/2610.41074#bib.bib34)\)for low\-precision training studied how precision and optimization choices work together\.

##### Numerical analysis for SR\.

[Higham and Mary \(2019\)](https://arxiv.org/html/2610.41074#bib.bib52);[Higham and Mary \(2020\)](https://arxiv.org/html/2610.41074#bib.bib53);[Connolly et al\. \(2021\)](https://arxiv.org/html/2610.41074#bib.bib7);[El Arar et al\. \(2023\)](https://arxiv.org/html/2610.41074#bib.bib11);[Croci et al\. \(2022\)](https://arxiv.org/html/2610.41074#bib.bib8)analyze the bound of the rounding error for SR in numerical computations\. Other work also studies SR with few random bits\([Fitzgibbon and Felix, 2025](https://arxiv.org/html/2610.41074#bib.bib50)\), computational complexity\([Drineas and Ipsen, 2024](https://arxiv.org/html/2610.41074#bib.bib54)\), and its effects on small singular values of random matrices\([Boutsikas et al\., 2024](https://arxiv.org/html/2610.41074#bib.bib55);[Dexter et al\., 2025](https://arxiv.org/html/2610.41074#bib.bib56);[Ma et al\., 2026](https://arxiv.org/html/2610.41074#bib.bib57)\)\. We refer to[El Arar et al\. \(2026\)](https://arxiv.org/html/2610.41074#bib.bib29)as a comprehensive review of SR theory, hardware and applications\.

## Appendix BStochastic rounding and sketch geometry

This appendix develops the rounding identities and random\-feature geometry used to prove Theorem[1](https://arxiv.org/html/2610.41074#Thmtheorem1)in Appendix[D\.1](https://arxiv.org/html/2610.41074#A4.SS1)\. Lemmas[B\.1](https://arxiv.org/html/2610.41074#Thmtheorem1a)–[B\.2](https://arxiv.org/html/2610.41074#Thmtheorem2)bound Gaussian rounding moments, and Proposition[B\.3](https://arxiv.org/html/2610.41074#Thmtheorem3)derives the training second\-moment recursions\. Proposition[B\.9](https://arxiv.org/html/2610.41074#Thmtheorem9)combines the spectral, teacher and rounding\-coordinate bounds into one high\-probability sketch event used in the subsequent analysis\.

### B\.1Shared notation and second\-moment comparison

We define the oracle predictor and second moments used in the training recursions of Proposition[B\.3](https://arxiv.org/html/2610.41074#Thmtheorem3)\. Throughout this appendix, unless a sketch probability is stated, expectations condition onSSand average over the teacher, training data and rounding variables\.

##### Oracle predictor and rounding geometry\.

For random sketchSS, useGGfrom \([2](https://arxiv.org/html/2610.41074#S2.E2)\) and define

G=∑j=1Mgjϕjϕj⊤,v∗S=LSθ∗,LS=G−1SΣd,V=𝔼\[v∗S\(v∗S\)⊤∣S\]=LSCθLS⊤\.\\begin\{gathered\}G=\\sum\_\{j=1\}^\{M\}g\_\{j\}\\phi\_\{j\}\\phi\_\{j\}^\{\\top\},\\qquad v\_\{\*\}^\{S\}=L\_\{S\}\\theta\_\{\*\},\\quad L\_\{S\}=G^\{\-1\}S\\Sigma\_\{d\},\\quad V=\\mathbb\{E\}\[v\_\{\*\}^\{S\}\(v\_\{\*\}^\{S\}\)^\{\\top\}\\mid S\]=L\_\{S\}C\_\{\\theta\}L\_\{S\}^\{\\top\}\.\\end\{gathered\}\(34\)The eigenvectorsϕj\\phi\_\{j\}are orthonormal andg1≥⋯≥gM\>0g\_\{1\}\\geq\\cdots\\geq g\_\{M\}\>0\. The matrixVVneed not commute withGG\. Write the oracle predictor asv∗=v∗Sv\_\{\*\}=v\_\{\*\}^\{S\}andΣ=Σd\\Sigma=\\Sigma\_\{d\}for simplicity\. Define

θG=tr⁡G,J=MθG​diag⁡\(G11,…,GM​M\)\.\\theta\_\{G\}=\\operatorname\{tr\}G,\\qquad J=\\frac\{M\}\{\\theta\_\{G\}\}\\operatorname\{diag\}\(G\_\{11\},\\ldots,G\_\{MM\}\)\.\(35\)HereθG\\theta\_\{G\}is total feature variance andJJrecords normalized coordinate variances in the rounding basis\. It need not commute withGG\.

##### Approximation error and residual noise\.

Define

Σ⟂=Σd−Σd​S⊤​G−1​S​Σd,AS=12θ∗⊤Σ⟂θ∗,A¯\(S\)=12tr\(CθΣ⟂\),s\(S\)=σ2\+2A¯\(S\)\.\\begin\{gathered\}\\Sigma\_\{\\perp\}=\\Sigma\_\{d\}\-\\Sigma\_\{d\}S^\{\\top\}G^\{\-1\}S\\Sigma\_\{d\},\\\\ A\_\{S\}=\\tfrac\{1\}\{2\}\\theta\_\{\*\}^\{\\top\}\\Sigma\_\{\\perp\}\\theta\_\{\*\},\\qquad\\bar\{A\}\(S\)=\\tfrac\{1\}\{2\}\\operatorname\{tr\}\(C\_\{\\theta\}\\Sigma\_\{\\perp\}\),\\qquad s\(S\)=\\sigma^\{2\}\+2\\bar\{A\}\(S\)\.\\end\{gathered\}\(36\)Theny=z⊤​v∗S\+ϵ~y=z^\{\\top\}v\_\{\*\}^\{S\}\+\\widetilde\{\\epsilon\}, whereϵ~=x⊤​\(θ∗−S⊤​v∗S\)\+ϵ\\widetilde\{\\epsilon\}=x^\{\\top\}\(\\theta\_\{\*\}\-S^\{\\top\}v\_\{\*\}^\{S\}\)\+\\epsilon\. Conditional onS,θ∗S,\\theta\_\{\*\}, this residual is centered Gaussian and independent ofzz, because their cross\-covariance isS​Σd​θ∗−G​v∗S=0S\\Sigma\_\{d\}\\theta\_\{\*\}\-Gv\_\{\*\}^\{S\}=0\. Its variance isσ2\+2​AS\\sigma^\{2\}\+2A\_\{S\}; averaging over the teacher givess⁡\(S\)s\(S\)\. WriteA¯=A¯​\(S\)\\bar\{A\}=\\bar\{A\}\(S\)ands=s⁡\(S\)s=s\(S\)\. The normal equationG​v∗S=S​Σd​θ∗Gv\_\{\*\}^\{S\}=S\\Sigma\_\{d\}\\theta\_\{\*\}splits the risk in \([9](https://arxiv.org/html/2610.41074#S2.E9)\) as

ℰS​\(v,θ∗\)=AS\+12​‖v−v∗S‖G2\\mathcal\{E\}\_\{S\}\(v,\\theta\_\{\*\}\)=A\_\{S\}\+\\tfrac\{1\}\{2\}\\\|v\-v\_\{\*\}^\{S\}\\\|\_\{G\}^\{2\}\(37\)whereASA\_\{S\}is the approximation error and the second term is the estimation error\. Define

E0out=ℛ0out​\(T\)−A¯​\(S\),ℛout​\(T\)=A¯​\(S\)\+E0out\+Δout​\(T\)\.E\_\{0\}^\{\{\\rm out\}\}=\\mathscr\{R\}\_\{0\}^\{\{\\rm out\}\}\(T\)\-\\bar\{A\}\(S\),\\qquad\\mathscr\{R\}^\{\{\\rm out\}\}\(T\)=\\bar\{A\}\(S\)\+E\_\{0\}^\{\{\\rm out\}\}\+\\Delta^\{\{\\rm out\}\}\(T\)\.\(38\)

##### Output weights and second moments\.

For outputs in \([8](https://arxiv.org/html/2610.41074#S2.E8)\), writevTout=∑t=1Twtout​vtv\_\{T\}^\{\{\\rm out\}\}=\\sum\_\{t=1\}^\{T\}w\_\{t\}^\{\{\\rm out\}\}v\_\{t\}, with

wtlast=𝟏\{t=T\},wtavg=1/T\(1≤t≤T\)\.w\_\{t\}^\{\\rm last\}=\\mathbf\{1\}\\\{t=T\\\},\\qquad w\_\{t\}^\{\\rm avg\}=1/T\\quad\(1\\leq t\\leq T\)\.\(39\)Both choices have nonnegative weights summing to one\. Define

Ct=𝔼\[\(vt−v∗\)\(vt−v∗\)⊤∣S\],Wt=𝔼\[vtvt⊤∣S\],Rt=s\+tr\(GCt0\),Ut=tr\(JWt0\),\\begin\{gathered\}C\_\{t\}=\\mathbb\{E\}\[\(v\_\{t\}\-v\_\{\*\}\)\(v\_\{t\}\-v\_\{\*\}\)^\{\\top\}\\mid S\],\\qquad W\_\{t\}=\\mathbb\{E\}\[v\_\{t\}v\_\{t\}^\{\\top\}\\mid S\],\\\\ R\_\{t\}=s\+\\operatorname\{tr\}\(GC\_\{t\}^\{0\}\),\\qquad U\_\{t\}=\\operatorname\{tr\}\(JW\_\{t\}^\{0\}\),\\end\{gathered\}\(40\)whereCt0,Wt0C\_\{t\}^\{0\},W\_\{t\}^\{0\}are the corresponding second moments for exact\-arithmetic SGD\. These are full second moments, not centered covariances\. The sourcesRtR\_\{t\}andUtU\_\{t\}measure its residual variance and parameter magnitude in rounding coordinates, respectively\.

In addition to the second moments in \([40](https://arxiv.org/html/2610.41074#A2.E40)\), define

Xt=Ct−Ct0,xt=tr⁡\(G​Xt\),yt=tr⁡\(J​Xt\)\.X\_\{t\}=C\_\{t\}\-C\_\{t\}^\{0\},\\qquad x\_\{t\}=\\operatorname\{tr\}\(GX\_\{t\}\),\\qquad y\_\{t\}=\\operatorname\{tr\}\(JX\_\{t\}\)\.\(41\)Proposition[B\.3](https://arxiv.org/html/2610.41074#Thmtheorem3)will show thatXt=Wt−Wt0⪰0X\_\{t\}=W\_\{t\}\-W\_\{t\}^\{0\}\\succeq 0\. ThusXtX\_\{t\}is a difference of second moments; it is not the second moment ofvt−vt0v\_\{t\}\-v\_\{t\}^\{0\}\.

### B\.2Gaussian SR moments and training recursions

We prove the rounding\-moment bounds in Lemmas[B\.1](https://arxiv.org/html/2610.41074#Thmtheorem1a)–[B\.2](https://arxiv.org/html/2610.41074#Thmtheorem2)and use them to derive the recursions in Proposition[B\.3](https://arxiv.org/html/2610.41074#Thmtheorem3)\. Use the binary grid, stochastic rounding rule and variance identity in Section[2](https://arxiv.org/html/2610.41074#S2)and \([3](https://arxiv.org/html/2610.41074#S2.E3)\)\. At representable values putνp​\(x\)=0\\nu\_\{p\}\(x\)=0, and setν∞=0\\nu\_\{\\infty\}=0\.

With the precision scale \([6](https://arxiv.org/html/2610.41074#S2.E6)\), binary rounding also obeys

νp​\(−x\)=νp​\(x\),νp​\(x/2\)=νp​\(x\)/4,\|Qp​\(x\)\|≤2​\|x\|\.\\nu\_\{p\}\(\-x\)=\\nu\_\{p\}\(x\),\\qquad\\nu\_\{p\}\(x/2\)=\\nu\_\{p\}\(x\)/4,\\qquad\|Q\_\{p\}\(x\)\|\\leq 2\|x\|\.\(42\)
###### Lemma B\.1\(Gaussian scalar and matrix moments\)\.

Withqpq\_\{p\}defined in \([6](https://arxiv.org/html/2610.41074#S2.E6)\), forX∼N⁡\(0,g\)X\\sim N\(0,g\),g\>0g\>0, andp∈ℕ0p\\in\\mathbb\{N\}\_\{0\}, putmp​\(g\)=𝔼​νp​\(X\)m\_\{p\}\(g\)=\\mathbb\{E\}\\nu\_\{p\}\(X\)andnp​\(g\)=𝔼⁡\[νp​\(g​Z\)​Z2\]n\_\{p\}\(g\)=\\mathbb\{E\}\[\\nu\_\{p\}\(\\sqrt\{g\}Z\)Z^\{2\}\], whereZ∼N⁡\(0,1\)Z\\sim N\(0,1\)\. Then

qp​g/192≤mp​\(g\)≤qp​g/4,np​\(g\)≥2​log⁡2​mp​\(g\)≥mp​\(g\),np​\(g\)≤3​qp​g/4\.q\_\{p\}g/192\\leq m\_\{p\}\(g\)\\leq q\_\{p\}g/4,\\qquad n\_\{p\}\(g\)\\geq 2\\log 2\\,m\_\{p\}\(g\)\\geq m\_\{p\}\(g\),\\qquad n\_\{p\}\(g\)\\leq 3q\_\{p\}g/4\.IfG≻0G\\succ 0,z∼N⁡\(0,G\)z\\sim N\(0,G\)andγi=Gi​i\\gamma\_\{i\}=G\_\{ii\}, then

qp​γi192​G⪯Kp,i:=𝔼⁡\[νp​\(zi\)​z​z⊤\]⪯34​qp​γi​G\.\\frac\{q\_\{p\}\\gamma\_\{i\}\}\{192\}G\\preceq K\_\{p,i\}:=\\mathbb\{E\}\[\\nu\_\{p\}\(z\_\{i\}\)zz^\{\\top\}\]\\preceq\\frac\{3\}\{4\}q\_\{p\}\\gamma\_\{i\}G\.\(43\)

###### Proof\.

WriteX=g​ZX=\\sqrt\{g\}Z, letϵ=2−p≤1\\epsilon=2^\{\-p\}\\leq 1, and denote theN⁡\(0,g\)N\(0,g\)density byfgf\_\{g\}\. Letℓ<u\\ell<ube consecutive positive grid points, and choosee∈ℤe\\in\\mathbb\{Z\}so that2e≤ℓ<2e\+12^\{e\}\\leq\\ell<2^\{e\+1\}\. Their rounding cell\[ℓ,u\]\[\\ell,u\]has widthΔ=u−ℓ=ϵ​2e\\Delta=u\-\\ell=\\epsilon 2^\{e\}\. By \([3](https://arxiv.org/html/2610.41074#S2.E3)\),

∫ℓuνp​\(x\)​fg​\(x\)​𝑑x≥fg​\(u\)​Δ3/6≥ϵ26​∫ℓu22​e​fg​\(\(1\+ϵ\)​x\)​𝑑x\.\\int\_\{\\ell\}^\{u\}\\nu\_\{p\}\(x\)f\_\{g\}\(x\)\\,dx\\geq f\_\{g\}\(u\)\\Delta^\{3\}/6\\geq\\frac\{\\epsilon^\{2\}\}\{6\}\\int\_\{\\ell\}^\{u\}2^\{2e\}f\_\{g\}\(\(1\+\\epsilon\)x\)\\,dx\.Hereu≤\(1\+ϵ\)​ℓu\\leq\(1\+\\epsilon\)\\ell, andfgf\_\{g\}decreases on the positive half\-line\. Summing over cells, using reflection and22​e≥x2/42^\{2e\}\\geq x^\{2\}/4on each cell, gives

mp​\(g\)≥qp24​∫ℝx2​fg​\(\(1\+ϵ\)​x\)​𝑑x=qp​g24​\(1\+ϵ\)3≥qp​g/192\.m\_\{p\}\(g\)\\geq\\frac\{q\_\{p\}\}\{24\}\\int\_\{\\mathbb\{R\}\}x^\{2\}f\_\{g\}\(\(1\+\\epsilon\)x\)\\,dx=\\frac\{q\_\{p\}g\}\{24\(1\+\\epsilon\)^\{3\}\}\\geq q\_\{p\}g/192\.Tonelli applies because the integrands are nonnegative\. The upper bounds onmp,npm\_\{p\},n\_\{p\}follow from \([3](https://arxiv.org/html/2610.41074#S2.E3)\) and𝔼​Z4=3\\mathbb\{E\}Z^\{4\}=3\. For the lower bound onnpn\_\{p\}, letμ\\muhave densityνp​\(x\)​fg​\(x\)/mp​\(g\)\\nu\_\{p\}\(x\)f\_\{g\}\(x\)/m\_\{p\}\(g\)\. Binary scaling givesmp​\(g/4\)=mp​\(g\)/4m\_\{p\}\(g/4\)=m\_\{p\}\(g\)/4\. Together withfg/4\(x\)=2fg\(x\)e−3x2/\(2g\)f\_\{g/4\}\(x\)=2f\_\{g\}\(x\)e^\{\-3x^\{2\}/\(2g\)\}, this yields𝔼μe−3X2/\(2g\)=1/8\\mathbb\{E\}\_\{\\mu\}e^\{\-3X^\{2\}/\(2g\)\}=1/8\. Jensen’s inequality implies𝔼μ​\(X2/g\)≥2​log⁡2\\mathbb\{E\}\_\{\\mu\}\(X^\{2\}/g\)\\geq 2\\log 2, ornp​\(g\)≥2​log⁡2​mp​\(g\)n\_\{p\}\(g\)\\geq 2\\log 2\\,m\_\{p\}\(g\)\.

For the matrix comparison, Gaussian regression givesz=\(G​ei/γi\)​zi\+ξz=\(Ge\_\{i\}/\\gamma\_\{i\}\)z\_\{i\}\+\\xi, whereξ\\xiis independent ofziz\_\{i\}and has covarianceG−G​ei​ei⊤​G/γiG\-Ge\_\{i\}e\_\{i\}^\{\\top\}G/\\gamma\_\{i\}\. HenceKp,i=mp​\(γi\)​G\+np​\(γi\)−mp​\(γi\)γi​G​ei​ei⊤​GK\_\{p,i\}=m\_\{p\}\(\\gamma\_\{i\}\)G\+\\frac\{n\_\{p\}\(\\gamma\_\{i\}\)\-m\_\{p\}\(\\gamma\_\{i\}\)\}\{\\gamma\_\{i\}\}Ge\_\{i\}e\_\{i\}^\{\\top\}G\. Whitening byG−1/2G^\{\-1/2\}gives eigenvaluesmp​\(γi\)m\_\{p\}\(\\gamma\_\{i\}\)on the orthogonal complement of the unit vectorG1/2​ei/γiG^\{1/2\}e\_\{i\}/\\sqrt\{\\gamma\_\{i\}\}andnp​\(γi\)n\_\{p\}\(\\gamma\_\{i\}\)on that vector\. The scalar bounds on these eigenvalues prove \([43](https://arxiv.org/html/2610.41074#A2.E43)\)\. ∎

###### Lemma B\.2\(Joint cell variance\)\.

For centered jointly Gaussian\(X,Y\)\(X,Y\)with variancesg,h\>0g,h\>0, arbitrary correlation in\[−1,1\]\[\-1,1\]and finite precisionsp,p′p,p^\{\\prime\}, one has

c×​qp​qp′​g​h≤𝔼⁡\[νp​\(X\)​νp′​\(Y\)\]≤316​qp​qp′​g​h,c×=38​\(δ⁡\(1−δ\)64\)2,δ=2−8\.c\_\{\\times\}q\_\{p\}q\_\{p^\{\\prime\}\}gh\\leq\\mathbb\{E\}\[\\nu\_\{p\}\(X\)\\nu\_\{p^\{\\prime\}\}\(Y\)\]\\leq\\tfrac\{3\}\{16\}q\_\{p\}q\_\{p^\{\\prime\}\}gh,\\qquad c\_\{\\times\}=\\tfrac\{3\}\{8\}\\left\(\\frac\{\\delta\(1\-\\delta\)\}\{64\}\\right\)^\{2\},\\quad\\delta=2^\{\-8\}\.Exact precisions or zero variances give zero on both sides\.

###### Proof\.

ForX∼N⁡\(0,g\)X\\sim N\(0,g\), letEXE\_\{X\}be the event thatg/4≤\|X\|≤4​g\\sqrt\{g\}/4\\leq\|X\|\\leq 4\\sqrt\{g\}and\|X\|\|X\|lies in the middle\[δ,1−δ\]\[\\delta,1\-\\delta\]fraction of its cell\. The folded Gaussian density is at most4/\(5​g\)4/\(5\\sqrt\{g\}\)\. The cells achieving this magnitude range have total width at most8​g8\\sqrt\{g\}, so their excluded edges have probability at most\(64/5\)​δ=1/20\(64/5\)\\delta=1/20\. The two magnitude exclusions have probabilities at most1/51/5and1/161/16, by the same density bound and Markov’s inequality\. HencePr⁡\(EX\)≥11/16\\Pr\(E\_\{X\}\)\\geq 11/16\. OnEXE\_\{X\}, the cell width satisfiesΔ≥2−p​g/8\\Delta\\geq 2^\{\-p\}\\sqrt\{g\}/8, and \([3](https://arxiv.org/html/2610.41074#S2.E3)\) givesνp​\(X\)≥δ⁡\(1−δ\)​Δ2≥δ⁡\(1−δ\)64​qp​g\.\\nu\_\{p\}\(X\)\\geq\\delta\(1\-\\delta\)\\Delta^\{2\}\\geq\\frac\{\\delta\(1\-\\delta\)\}\{64\}q\_\{p\}g\.The analogous event forYYintersectsEXE\_\{X\}with probability at least3/83/8, without any independence assumption\. Multiplying the variance bounds on this intersection proves the lower estimate\. For the upper estimate, use \([3](https://arxiv.org/html/2610.41074#S2.E3)\) and the Gaussian identity𝔼​X2​Y2=g​h​\(1\+2​Corr⁡\(X,Y\)2\)≤3​g​h\\mathbb\{E\}X^\{2\}Y^\{2\}=gh\(1\+2\\operatorname\{Corr\}\(X,Y\)^\{2\}\)\\leq 3gh\. Exact precisions and zero variances are immediate\. ∎

Forz∼N⁡\(0,G\)z\\sim N\(0,G\)andr∈\{fw,bw\}r\\in\\\{\\mathrm\{fw\},\\mathrm\{bw\}\\\}, define the diagonal conditional rounding covarianceDr,t​\(z\)=diag⁡\(νpr,t​\(zi\)\)D\_\{r,t\}\(z\)=\\operatorname\{diag\}\(\\nu\_\{p\_\{r,t\}\}\(z\_\{i\}\)\)and the linear maps

ℱt​\(W\)\\displaystyle\\mathcal\{F\}\_\{t\}\(W\)=𝔼⁡\[z​z⊤​tr⁡\(Dfw,t​\(z\)​W\)\],\\displaystyle=\\mathbb\{E\}\[zz^\{\\top\}\\operatorname\{tr\}\(D\_\{\\rm fw,t\}\(z\)W\)\],\(44\)ℬt​\(C,s\)\\displaystyle\\mathcal\{B\}\_\{t\}\(C,s\)=𝔼⁡\[Dbw,t​\(z\)​\(z⊤​C​z\+s\)\],ℐt​\(W\)\\displaystyle=\\mathbb\{E\}\[D\_\{\\rm bw,t\}\(z\)\(z^\{\\top\}Cz\+s\)\],\\quad\\mathcal\{I\}\_\{t\}\(W\)=𝔼⁡\[Dbw,t​\(z\)​tr⁡\(Dfw,t​\(z\)​W\)\]\.\\displaystyle=\\mathbb\{E\}\[D\_\{\\rm bw,t\}\(z\)\\operatorname\{tr\}\(D\_\{\\rm fw,t\}\(z\)W\)\]\.These maps are positive forC,W⪰0C,W\\succeq 0,s≥0s\\geq 0\. WithJJfrom \([35](https://arxiv.org/html/2610.41074#A2.E35)\),U=tr⁡\(J​W\)U=\\operatorname\{tr\}\(JW\)andR=s\+tr⁡\(G​C\)R=s\+\\operatorname\{tr\}\(GC\), the preceding lemmas give

ft−​U​G⪯ℱt​\(W\)⪯ft\+​U​G,βt−​R​J⪯ℬt​\(C,s\)⪯βt\+​R​J,mt−​U​J⪯ℐt​\(W\)⪯mt\+​U​J,f\_\{t\}^\{\-\}UG\\preceq\\mathcal\{F\}\_\{t\}\(W\)\\preceq f\_\{t\}^\{\+\}UG,\\quad\\beta\_\{t\}^\{\-\}RJ\\preceq\\mathcal\{B\}\_\{t\}\(C,s\)\\preceq\\beta\_\{t\}^\{\+\}RJ,\\quad m\_\{t\}^\{\-\}UJ\\preceq\\mathcal\{I\}\_\{t\}\(W\)\\preceq m\_\{t\}^\{\+\}UJ,\(45\)where the upper coefficients are

ft\+=3​θG4​M​qfw,t,βt\+=3​θG4​M​qbw,t,mt\+=3​θG216​M2​qfw,t​qbw,t,f\_\{t\}^\{\+\}=\\frac\{3\\theta\_\{G\}\}\{4M\}q\_\{\\mathrm\{fw\},t\},\\qquad\\beta\_\{t\}^\{\+\}=\\frac\{3\\theta\_\{G\}\}\{4M\}q\_\{\\mathrm\{bw\},t\},\\qquad m\_\{t\}^\{\+\}=\\frac\{3\\theta\_\{G\}^\{2\}\}\{16M^\{2\}\}q\_\{\\mathrm\{fw\},t\}q\_\{\\mathrm\{bw\},t\},\(46\)and the lower coefficients are

ft−\\displaystyle f\_\{t\}^\{\-\}=qfw,t​θG/\(192​M\),βt−=qbw,t​θG/\(192​M\),mt−=c×​qfw,t​qbw,t​θG2/M2\.\\displaystyle=q\_\{\\rm fw,t\}\\theta\_\{G\}/\(192M\),\\qquad\\beta\_\{t\}^\{\-\}=q\_\{\\rm bw,t\}\\theta\_\{G\}/\(192M\),\\quad m\_\{t\}^\{\-\}=c\_\{\\times\}q\_\{\\rm fw,t\}q\_\{\\rm bw,t\}\\theta\_\{G\}^\{2\}/M^\{2\}\.\(47\)Indeedℱt​\(W\)=∑iWi​i​Kpfw,t,i\\mathcal\{F\}\_\{t\}\(W\)=\\sum\_\{i\}W\_\{ii\}K\_\{p\_\{\\rm fw,t\},i\}, theiith diagonal entry ofℬt\\mathcal\{B\}\_\{t\}istr⁡\(C​Kpbw,t,i\)\+s​mpbw,t​\(Gi​i\)\\operatorname\{tr\}\(CK\_\{p\_\{\\rm bw,t\},i\}\)\+sm\_\{p\_\{\\rm bw,t\}\}\(G\_\{ii\}\), and that ofℐt\\mathcal\{I\}\_\{t\}is∑kWk​k​𝔼​\[νpbw,t​\(zi\)​νpfw,t​\(zk\)\]\\sum\_\{k\}W\_\{kk\}\\mathbb\{E\}\[\\nu\_\{p\_\{\\rm bw,t\}\}\(z\_\{i\}\)\\nu\_\{p\_\{\\rm fw,t\}\}\(z\_\{k\}\)\]\. SinceW⪰0W\\succeq 0, everyWk​kW\_\{kk\}is nonnegative\. Applying the preceding moment bounds and using∑kGk​k​Wk​k=θG​U/M\\sum\_\{k\}G\_\{kk\}W\_\{kk\}=\\theta\_\{G\}U/MandGi​i=θG​Ji​i/MG\_\{ii\}=\\theta\_\{G\}J\_\{ii\}/Mproves \([45](https://arxiv.org/html/2610.41074#A2.E45)\)\.

###### Proposition B\.3\(Actual moments and matched\-reference comparison\)\.

Consider deterministic precision schedules and nonnegative learning rates in \([5](https://arxiv.org/html/2610.41074#S2.E5)\), withhth\_\{t\}as in \([7](https://arxiv.org/html/2610.41074#S2.E7)\)\. Write

Ht=I−ηt​G,Pu→t=∏v=ut−1Hv,Pt=P0→t,Pt→t=I\.H\_\{t\}=I\-\\eta\_\{t\}G,\\qquad P\_\{u\\to t\}=\\prod\_\{v=u\}^\{t\-1\}H\_\{v\},\\qquad P\_\{t\}=P\_\{0\\to t\},\\quad P\_\{t\\to t\}=I\.\(48\)The matricesHtH\_\{t\}commute, so each productPu→tP\_\{u\\to t\}is symmetric\. With the moments \([40](https://arxiv.org/html/2610.41074#A2.E40)\), gap \([41](https://arxiv.org/html/2610.41074#A2.E41)\) and rounding maps \([44](https://arxiv.org/html/2610.41074#A2.E44)\), all finite\-time moments exist, andC0=V,C\_\{0\}=V,

Ct\+1\\displaystyle C\_\{t\+1\}=Ht​Ct​Ht\+ht​\{\(s\+tr⁡\(G​Ct\)\)​G\+G​Ct​G\+ℱt​\(Wt\)\+ℬt​\(Ct,s\)\+ℐt​\(Wt\)\},\\displaystyle=H\_\{t\}C\_\{t\}H\_\{t\}\+h\_\{t\}\\\{\(s\+\\operatorname\{tr\}\(GC\_\{t\}\)\)G\+GC\_\{t\}G\+\\mathcal\{F\}\_\{t\}\(W\_\{t\}\)\+\\mathcal\{B\}\_\{t\}\(C\_\{t\},s\)\+\\mathcal\{I\}\_\{t\}\(W\_\{t\}\)\\\},\(49\)Wt\\displaystyle W\_\{t\}=Ct\+V−Pt​V−V​Pt\.\\displaystyle=C\_\{t\}\+V\-P\_\{t\}V\-VP\_\{t\}\.\(50\)For exact\-arithmetic SGD, the same recursion holds with all three rounding maps set to zero\. Moreover,Xt=Wt−Wt0=Ct−Ct0X\_\{t\}=W\_\{t\}\-W\_\{t\}^\{0\}=C\_\{t\}\-C\_\{t\}^\{0\}satisfiesX0=0,X\_\{0\}=0,

Xt\+1=Ht​Xt​Ht\+ht​Qt,Qt=G​Xt​G\+xt​G\+ℱt​\(Wt\)\+ℬt​\(Ct,s\)\+ℐt​\(Wt\),X\_\{t\+1\}=H\_\{t\}X\_\{t\}H\_\{t\}\+h\_\{t\}Q\_\{t\},\\quad Q\_\{t\}=GX\_\{t\}G\+x\_\{t\}G\+\\mathcal\{F\}\_\{t\}\(W\_\{t\}\)\+\\mathcal\{B\}\_\{t\}\(C\_\{t\},s\)\+\\mathcal\{I\}\_\{t\}\(W\_\{t\}\),\(51\)andXt,Qt⪰0X\_\{t\},Q\_\{t\}\\succeq 0at every finite time\.

###### Proof\.

The bound in \([42](https://arxiv.org/html/2610.41074#A2.E42)\) implies

‖vt\+1‖≤\(1\+ηt​B−1​∑ℓ‖bt​ℓ‖​‖at​ℓ‖\)​‖vt‖\+ηt​B−1​∑ℓ‖bt​ℓ‖​\|yt​ℓ\|\.\\\|v\_\{t\+1\}\\\|\\leq\(1\+\\eta\_\{t\}B^\{\-1\}\\textstyle\\sum\_\{\\ell\}\\\|b\_\{t\\ell\}\\\|\\\|a\_\{t\\ell\}\\\|\)\\\|v\_\{t\}\\\|\+\\eta\_\{t\}B^\{\-1\}\\textstyle\\sum\_\{\\ell\}\\\|b\_\{t\\ell\}\\\|\|y\_\{t\\ell\}\|\.Hereat​ℓ,bt​ℓa\_\{t\\ell\},b\_\{t\\ell\}are the rounded features from \([4](https://arxiv.org/html/2610.41074#S2.E4)\)\. Repeatedly applying this inequality bounds each finite\-time state by a polynomial in finitely many Gaussian magnitudes, including the teacher\. All its moments therefore exist\.

Letℱt\\mathscr\{F\}\_\{t\}contain the teacher and the history through updatet−1t\-1, withSSstill fixed\. Given this history, pute=vt−v∗e=v\_\{t\}\-v\_\{\*\}and use the prediction residualϵ~\\widetilde\{\\epsilon\}defined in Appendix[B\.1](https://arxiv.org/html/2610.41074#A2.SS1)\. Conditional onℱt\\mathscr\{F\}\_\{t\},zzandϵ~\\widetilde\{\\epsilon\}are jointly centered Gaussian with cross\-covarianceS​Σ​θ∗−G​v∗=0S\\Sigma\\theta\_\{\*\}\-Gv\_\{\*\}=0, hence are independent\. Writes~=σ2\+2​AS\\widetilde\{s\}=\\sigma^\{2\}\+2A\_\{S\}for the residual variance at this conditioning stage\. Writea=z\+ξa=z\+\\xi,b=z\+ζb=z\+\\zeta,r=z⊤​e−ϵ~r=z^\{\\top\}e\-\\widetilde\{\\epsilon\}andΓ=b⁡\(r\+ξ⊤​vt\)\\Gamma=b\(r\+\\xi^\{\\top\}v\_\{t\}\)\. Conditional on\(ℱt,x,ϵ\)\(\\mathscr\{F\}\_\{t\},x,\\epsilon\), the rounding errorsξ,ζ\\xi,\\zetaare centered and independent, with covariancesDfw,t​\(z\),Dbw,t​\(z\)D\_\{\\rm fw,t\}\(z\),D\_\{\\rm bw,t\}\(z\)\. Thus

𝔼​Γ=z​r,𝔼⁡\[Γ​Γ⊤\]=\(z​z⊤\+Dbw,t​\(z\)\)​\(r2\+vt⊤​Dfw,t​\(z\)​vt\)\.\\mathbb\{E\}\\Gamma=zr,\\qquad\\mathbb\{E\}\[\\Gamma\\Gamma^\{\\top\}\]=\(zz^\{\\top\}\+D\_\{\\rm bw,t\}\(z\)\)\(r^\{2\}\+v\_\{t\}^\{\\top\}D\_\{\\rm fw,t\}\(z\)v\_\{t\}\)\.The Gaussian fourth\-moment identity gives𝔼⁡\[z​z⊤​\(z⊤​e\)2\]=\(e⊤​G​e\)​G\+2​G​e​e⊤​G\\mathbb\{E\}\[zz^\{\\top\}\(z^\{\\top\}e\)^\{2\}\]=\(e^\{\\top\}Ge\)G\+2Gee^\{\\top\}G\. The residual cross term vanishes, so averaging the last display gives

𝔼⁡\[Γ∣ℱt\]=G​e,𝔼⁡\[Γ​Γ⊤∣ℱt\]=\(s~\+e⊤​G​e\)​G\+2​G​e​e⊤​G\+ℱt​\(vt​vt⊤\)\+ℬt​\(e​e⊤,s~\)\+ℐt​\(vt​vt⊤\)\.\\mathbb\{E\}\[\\Gamma\\mid\\mathscr\{F\}\_\{t\}\]=Ge,\\quad\\mathbb\{E\}\[\\Gamma\\Gamma^\{\\top\}\\mid\\mathscr\{F\}\_\{t\}\]=\(\\widetilde\{s\}\+e^\{\\top\}Ge\)G\+2Gee^\{\\top\}G\+\\mathcal\{F\}\_\{t\}\(v\_\{t\}v\_\{t\}^\{\\top\}\)\+\\mathcal\{B\}\_\{t\}\(ee^\{\\top\},\\widetilde\{s\}\)\+\\mathcal\{I\}\_\{t\}\(v\_\{t\}v\_\{t\}^\{\\top\}\)\.The batch samples are conditionally independent\. Consequently,𝔼⁡\[et\+1∣ℱt\]=Ht​e\\mathbb\{E\}\[e\_\{t\+1\}\\mid\\mathscr\{F\}\_\{t\}\]=H\_\{t\}e, and the conditional second moment equalsHt​e​e⊤​HtH\_\{t\}ee^\{\\top\}H\_\{t\}plushth\_\{t\}times the conditional covariance ofΓ\\Gamma\. This covariance is the preceding second moment ofΓ\\GammaminusG​e​e⊤​GGee^\{\\top\}G\. Averaging over the teacher uses𝔼⁡\[s~∣S\]=s\\mathbb\{E\}\[\\widetilde\{s\}\\mid S\]=sand linearity of the maps, proving \([49](https://arxiv.org/html/2610.41074#A2.E49)\)\. Iterating the conditional mean gives

𝔼\[euew⊤∣S\]=Pw→uCw\(u≥w\),𝔼\[vt∣S,θ∗\]=\(I−Pt\)v∗\.\\mathbb\{E\}\[e\_\{u\}e\_\{w\}^\{\\top\}\\mid S\]=P\_\{w\\to u\}C\_\{w\}\\quad\(u\\geq w\),\\qquad\\mathbb\{E\}\[v\_\{t\}\\mid S,\\theta\_\{\*\}\]=\(I\-P\_\{t\}\)v\_\{\*\}\.\(52\)Sincee0=−v∗e\_\{0\}=\-v\_\{\*\}, the mean recursion also gives𝔼⁡\[et​v∗⊤∣S\]=−Pt​V\\mathbb\{E\}\[e\_\{t\}v\_\{\*\}^\{\\top\}\\mid S\]=\-P\_\{t\}V\. Expandingvt​vt⊤=\(et\+v∗\)​\(et\+v∗\)⊤v\_\{t\}v\_\{t\}^\{\\top\}=\(e\_\{t\}\+v\_\{\*\}\)\(e\_\{t\}\+v\_\{\*\}\)^\{\\top\}proves \([50](https://arxiv.org/html/2610.41074#A2.E50)\); no commutation betweenVVandPtP\_\{t\}is needed\. Subtracting the exact recurrence proves \([51](https://arxiv.org/html/2610.41074#A2.E51)\)\. Positivity of the maps and induction from zero proveXt,Qt⪰0X\_\{t\},Q\_\{t\}\\succeq 0\. ∎

To compare the rounded and exact\-arithmetic runs, forZ⪰0Z\\succeq 0define the order\-preserving affine maps

Qt±​\(Z\)=G​Z​G\+\[tr⁡\(G​Z\)\+ft±​\(Ut\+tr⁡\(J​Z\)\)\]​G\+\[βt±​\(Rt\+tr⁡\(G​Z\)\)\+mt±​\(Ut\+tr⁡\(J​Z\)\)\]​J\.Q\_\{t\}^\{\\pm\}\(Z\)=GZG\+\[\\operatorname\{tr\}\(GZ\)\+f\_\{t\}^\{\\pm\}\(U\_\{t\}\+\\operatorname\{tr\}\(JZ\)\)\]G\+\[\\beta\_\{t\}^\{\\pm\}\(R\_\{t\}\+\\operatorname\{tr\}\(GZ\)\)\+m\_\{t\}^\{\\pm\}\(U\_\{t\}\+\\operatorname\{tr\}\(JZ\)\)\]J\.\(53\)Then \([45](https://arxiv.org/html/2610.41074#A2.E45)\) givesQt−​\(Xt\)⪯Qt⪯Qt\+​\(Xt\)Q\_\{t\}^\{\-\}\(X\_\{t\}\)\\preceq Q\_\{t\}\\preceq Q\_\{t\}^\{\+\}\(X\_\{t\}\)\. Starting fromX0±=0X\_\{0\}^\{\\pm\}=0, letXt\+1±=Ht​Xt±​Ht\+ht​Qt±​\(Xt±\)X\_\{t\+1\}^\{\\pm\}=H\_\{t\}X\_\{t\}^\{\\pm\}H\_\{t\}\+h\_\{t\}Q\_\{t\}^\{\\pm\}\(X\_\{t\}^\{\\pm\}\)\. The map bounds and induction in the positive semidefinite order give

0⪯Xt−⪯Xt⪯Xt\+,Qt−​\(Xt−\)⪯Qt⪯Qt\+​\(Xt\+\)\.0\\preceq X\_\{t\}^\{\-\}\\preceq X\_\{t\}\\preceq X\_\{t\}^\{\+\},\\qquad Q\_\{t\}^\{\-\}\(X\_\{t\}^\{\-\}\)\\preceq Q\_\{t\}\\preceq Q\_\{t\}^\{\+\}\(X\_\{t\}^\{\+\}\)\.\(54\)

### B\.3One sketch event and constants

We now prove Proposition[B\.9](https://arxiv.org/html/2610.41074#Thmtheorem9), constructing one event, independent of the learning\-rate and precision schedules, on which all geometric comparisons hold\. Throughout this construction,c,C\>0c,C\>0may change from line to line and depend only on the fixed population family in \([1](https://arxiv.org/html/2610.41074#S2.E1)\)\. The numerical thresholds used later to define the admissible schedules are recorded explicitly\.

Split the population coordinates afternH=⌊κgeo​M⌋n\_\{H\}=\\lfloor\\kappa\_\{\\rm geo\}M\\rfloor:S=\(SH,SR\)S=\(S\_\{H\},S\_\{R\}\), with matching diagonal blocksΣH,ΣR\\Sigma\_\{H\},\\Sigma\_\{R\}ofΣ\\Sigma\. Choose the fixed constants

κgeo=14min\(1,cd−1\),δgeo=34\(cd−1\),Mgeo=⌈κgeo−1⌉,cH=14,rgeo=12\(1\+δgeo−1\),pgeo=min\(1/32,rgeo2/2\),K0,geo=cd\+1\+2,KR,geo=cd\+1\+1\+rgeo\.\\begin\{gathered\}\\kappa\_\{\\rm geo\}=\\tfrac\{1\}\{4\}\\min\(1,c\_\{d\}\-1\),\\quad\\delta\_\{\\rm geo\}=\\tfrac\{3\}\{4\}\(c\_\{d\}\-1\),\\\\ M\_\{\\rm geo\}=\\lceil\\kappa\_\{\\rm geo\}^\{\-1\}\\rceil,\\quad c\_\{H\}=\\tfrac\{1\}\{4\},\\\\ r\_\{\\rm geo\}=\\tfrac\{1\}\{2\}\(\\sqrt\{1\+\\delta\_\{\\rm geo\}\}\-1\),\\quad p\_\{\\rm geo\}=\\min\(1/32,r\_\{\\rm geo\}^\{2\}/2\),\\\\ K\_\{0,\\rm geo\}=\\sqrt\{c\_\{d\}\+1\}\+2,\\quad K\_\{R,\\rm geo\}=\\sqrt\{c\_\{d\}\+1\}\+1\+r\_\{\\rm geo\}\.\\end\{gathered\}\(55\)Choose a fixed minimum widthMwM\_\{\\rm w\}satisfying

Mw≥Mgeo≥4\.M\_\{\\rm w\}\\geq M\_\{\\rm geo\}\\geq 4\.\(56\)ForM≥MgeoM\\geq M\_\{\\rm geo\}define event

ℰM=\{∥S∥≤K0,geo,smin\(SH\)≥cH,rgeo≤smin\(SR⊤\)≤smax\(SR⊤\)≤KR,geo\}\.\\mathcal\{E\}\_\{M\}=\\\{\\\|S\\\|\\leq K\_\{0,\\rm geo\},\\ s\_\{\\min\}\(S\_\{H\}\)\\geq c\_\{H\},\\ r\_\{\\rm geo\}\\leq s\_\{\\min\}\(S\_\{R\}^\{\\top\}\)\\leq s\_\{\\max\}\(S\_\{R\}^\{\\top\}\)\\leq K\_\{R,\\rm geo\}\\\}\.\(57\)
###### Lemma B\.4\(Spectral and approximation geometry\)\.

The eventℰM\\mathcal\{E\}\_\{M\}satisfiesPr⁡\(ℰMc\)≤6​e−pgeo​M\\Pr\(\\mathcal\{E\}\_\{M\}^\{c\}\)\\leq 6e^\{\-p\_\{\\rm geo\}M\}\. On it, with𝖳=SR​ΣR​SR⊤\\mathsf\{T\}=S\_\{R\}\\Sigma\_\{R\}S\_\{R\}^\{\\top\},

cM−aI⪯𝖳⪯CM−aI,∥ΣR∥≤CM−a,cj−a≤gj≤Cj−a,c≤θG≤C,∥LS∥≤C,cAM1−b≤A¯≤CAM1−b,cA=12cλ​ϑ\(cd−1\)\(cd\+1\)−b\.\\begin\{gathered\}cM^\{\-a\}I\\preceq\\mathsf\{T\}\\preceq CM^\{\-a\}I,\\quad\\\|\\Sigma\_\{R\}\\\|\\leq CM^\{\-a\},\\quad cj^\{\-a\}\\leq g\_\{j\}\\leq Cj^\{\-a\},\\\\ c\\leq\\theta\_\{G\}\\leq C,\\quad\\\|L\_\{S\}\\\|\\leq C,\\quad c\_\{A\}M^\{1\-b\}\\leq\\bar\{A\}\\leq C\_\{A\}M^\{1\-b\},\\qquad c\_\{A\}=\\tfrac\{1\}\{2\}c\_\{\\lambda\\vartheta\}\(c\_\{d\}\-1\)\(c\_\{d\}\+1\)^\{\-b\}\.\\end\{gathered\}\(58\)The lower approximation bound holds for every full\-row\-rankSS\. MoreoverLS​S⊤=IL\_\{S\}S^\{\\top\}=IandG−1/2SΣ1/2G^\{\-1/2\}S\\Sigma^\{1/2\}is a coisometry\.

###### Proof\.

The split satisfiesnH≍Mn\_\{H\}\\asymp M,1≤nH≤M/41\\leq n\_\{H\}\\leq M/4andd−nH≥\(1\+δgeo\)​Md\-n\_\{H\}\\geq\(1\+\\delta\_\{\\rm geo\}\)M\. Apply[Vershynin \(2011, Corollary 5\.35\)](https://arxiv.org/html/2610.41074#bib.bib68)toM​S⊤\\sqrt\{M\}S^\{\\top\},M​SH\\sqrt\{M\}S\_\{H\}andM​SR⊤\\sqrt\{M\}S\_\{R\}^\{\\top\}, with deviationsM\\sqrt\{M\},M/4\\sqrt\{M\}/4andrgeo​Mr\_\{\\rm geo\}\\sqrt\{M\}\. The union bound gives the stated failure probability\. OnℰM\\mathcal\{E\}\_\{M\}, the population envelopes and tail singular\-value bounds giveΣR≍M−a​I\\Sigma\_\{R\}\\asymp M^\{\-a\}I,𝖳≍M−a​I\\mathsf\{T\}\\asymp M^\{\-a\}IandG⪰𝖳≻0G\\succeq\\mathsf\{T\}\\succ 0\.

On the orthogonal complement of the firstj−1j\-1columns ofSS,u⊤​G​u≤C​j−a​‖u‖2u^\{\\top\}Gu\\leq Cj^\{\-a\}\\\|u\\\|^\{2\}\. Forj≤nHj\\leq n\_\{H\}, the firstjjcolumns have smallest singular value at leastcHc\_\{H\}, so their range satisfiesu⊤​G​u≥c​j−a​‖u‖2u^\{\\top\}Gu\\geq cj^\{\-a\}\\\|u\\\|^\{2\}\. Forj\>nHj\>n\_\{H\}, useG⪰𝖳G\\succeq\\mathsf\{T\}andj\>κgeo​Mj\>\\kappa\_\{\\rm geo\}M\. Min–max therefore givesgj≍j−ag\_\{j\}\\asymp j^\{\-a\}, and summation givesθG≍1\\theta\_\{G\}\\asymp 1\. PutDH=ΣH−1\+SH⊤​𝖳−1​SH⪰c​Ma​ID\_\{H\}=\\Sigma\_\{H\}^\{\-1\}\+S\_\{H\}^\{\\top\}\\mathsf\{T\}^\{\-1\}S\_\{H\}\\succeq cM^\{a\}I\. The identityG−1​SH​ΣH=𝖳−1​SH​DH−1G^\{\-1\}S\_\{H\}\\Sigma\_\{H\}=\\mathsf\{T\}^\{\-1\}S\_\{H\}D\_\{H\}^\{\-1\}bounds the head block ofLSL\_\{S\}, while‖G−1​SR​ΣR‖≤‖G−1‖​‖SR‖​‖ΣR‖≤C\\\|G^\{\-1\}S\_\{R\}\\Sigma\_\{R\}\\\|\\leq\\\|G^\{\-1\}\\\|\\\|S\_\{R\}\\\|\\\|\\Sigma\_\{R\}\\\|\\leq Cbounds its tail block\. Thus‖LS‖≤C\\\|L\_\{S\}\\\|\\leq C\. The right\-inverse and coisometry identities follow directly fromG=S​Σ​S⊤G=S\\Sigma S^\{\\top\}\.

PutP=I−Σ1/2​S⊤​G−1​S​Σ1/2P=I\-\\Sigma^\{1/2\}S^\{\\top\}G^\{\-1\}S\\Sigma^\{1/2\}andDθ=diag⁡\(λj​ϑj\)D\_\{\\theta\}=\\operatorname\{diag\}\(\\lambda\_\{j\}\\vartheta\_\{j\}\)\. For every full\-row\-rankSS,PPis a rank\-\(d−M\)\(d\-M\)orthogonal projector, so2​A¯=tr⁡\(P​Dθ\)≥cλ​ϑ​d−b​\(d−M\)≥2​cA​M1−b2\\bar\{A\}=\\operatorname\{tr\}\(PD\_\{\\theta\}\)\\geq c\_\{\\lambda\\vartheta\}d^\{\-b\}\(d\-M\)\\geq 2c\_\{A\}M^\{1\-b\}\. The Woodbury identity gives

PH​H=ΣH−1/2DH−1ΣH−1/2⪯CM−aΣH−1,P\_\{HH\}=\\Sigma\_\{H\}^\{\-1/2\}D\_\{H\}^\{\-1\}\\Sigma\_\{H\}^\{\-1/2\}\\preceq CM^\{\-a\}\\Sigma\_\{H\}^\{\-1\},wherePH​HP\_\{HH\}is the head block ofPP\. UsingPj​j≤1P\_\{jj\}\\leq 1on the tail and1<b<a1<b<a,

2​A¯≤C​M−a​∑j≤nHja−b\+C​∑j\>nHj−b≤C​M1−b\.2\\bar\{A\}\\leq CM^\{\-a\}\\sum\_\{j\\leq n\_\{H\}\}j^\{a\-b\}\+C\\sum\_\{j\>n\_\{H\}\}j^\{\-b\}\\leq CM^\{1\-b\}\.Fix one such upper constantCAC\_\{A\}uniformly over widths and sketches on the event\. ∎

###### Corollary B\.5\(Deterministic effective\-noise scale\)\.

On the event of Lemma[B\.4](https://arxiv.org/html/2610.41074#Thmtheorem4),

σ2\+2​cA​M1−b≤s⁡\(S\)≤σ2\+2​CA​M1−b\.\\sigma^\{2\}\+2c\_\{A\}M^\{1\-b\}\\leq s\(S\)\\leq\\sigma^\{2\}\+2C\_\{A\}M^\{1\-b\}\.\(59\)UsesMs\_\{M\}from \([19](https://arxiv.org/html/2610.41074#S3.E19)\) and putcs=min⁡\(1,2​cA\)c\_\{s\}=\\min\(1,2c\_\{A\}\),Cs=max⁡\(1,2​CA\)C\_\{s\}=\\max\(1,2C\_\{A\}\)\. Then

cs​sM≤s⁡\(S\)≤Cs​sM,0≤s⁡\(S\)/σ2−1≤2​CA​M1−b/σ2\.c\_\{s\}s\_\{M\}\\leq s\(S\)\\leq C\_\{s\}s\_\{M\},\\qquad 0\\leq s\(S\)/\\sigma^\{2\}\-1\\leq 2C\_\{A\}M^\{1\-b\}/\\sigma^\{2\}\.\(60\)The same constants compares\(S\)\+τ−\(b−1\)/as\(S\)\+\\tau^\{\-\(b\-1\)/a\}andsM\+τ−\(b−1\)/as\_\{M\}\+\\tau^\{\-\(b\-1\)/a\}for everyτ\>0\\tau\>0\.

###### Proof\.

Addσ2\\sigma^\{2\}to twice the approximation bounds and usecs≤1≤Csc\_\{s\}\\leq 1\\leq C\_\{s\}\. Subtractingσ2\\sigma^\{2\}and dividing by it proves the second inequality; addingτ−\(b−1\)/a\\tau^\{\-\(b\-1\)/a\}preserves the first comparison\. ∎

###### Lemma B\.6\(Full\-matrix teacher source\)\.

SupposeSShas full row rank,‖S‖≤Ksk\\\|S\\\|\\leq K\_\{\\rm sk\},‖LS‖≤Kproj\\\|L\_\{S\}\\\|\\leq K\_\{\\rm proj\}, and0<ω<10<\\omega<1\. If positive constantsχ−,χ\+\\chi\_\{\-\},\\chi\_\{\+\}satisfyχ−​Σ−ω⪯Cθ⪯χ\+​Σ−ω\\chi\_\{\-\}\\Sigma^\{\-\\omega\}\\preceq C\_\{\\theta\}\\preceq\\chi\_\{\+\}\\Sigma^\{\-\\omega\}, then

χ−​Ksk−2​\(1−ω\)​G−ω⪯V⪯χ\+​Kproj2​\(1−ω\)​G−ω\.\\chi\_\{\-\}K\_\{\\rm sk\}^\{\-2\(1\-\\omega\)\}G^\{\-\\omega\}\\preceq V\\preceq\\chi\_\{\+\}K\_\{\\rm proj\}^\{2\(1\-\\omega\)\}G^\{\-\\omega\}\.\(61\)For \([1](https://arxiv.org/html/2610.41074#S2.E1)\),ω=a−ba\\omega=\\frac\{a\-b\}\{a\}andχ−=cλ​ϑCλ−b/a\\chi\_\{\-\}=c\_\{\\lambda\\vartheta\}C\_\{\\lambda\}^\{\-b/a\},χ\+=Cλ​ϑcλ−b/a\\chi\_\{\+\}=C\_\{\\lambda\\vartheta\}c\_\{\\lambda\}^\{\-b/a\}\. ThuscG−\(a−b\)/a⪯V⪯CG−\(a−b\)/acG^\{\-\(a\-b\)/a\}\\preceq V\\preceq CG^\{\-\(a\-b\)/a\}onℰM\\mathcal\{E\}\_\{M\}\.

###### Proof\.

Consider the analytic matrix familiesF\(z\)=Gz/2LSΣ−z/2F\(z\)=G^\{z/2\}L\_\{S\}\\Sigma^\{\-z/2\}andR\(z\)=Σz/2S⊤G−z/2R\(z\)=\\Sigma^\{z/2\}S^\{\\top\}G^\{\-z/2\}on0≤ℜ⁡z≤10\\leq\\Re z\\leq 1\. Both are bounded on this strip\. The boundary bounds forFFareKprojK\_\{\\rm proj\}atℜ⁡z=0\\Re z=0and one atℜ⁡z=1\\Re z=1; those forRRareKskK\_\{\\rm sk\}and one, respectively\. Indeed, imaginary powers are unitary,F\(1\)=G−1/2SΣ1/2F\(1\)=G^\{\-1/2\}S\\Sigma^\{1/2\}is the coisometry from Lemma[B\.4](https://arxiv.org/html/2610.41074#Thmtheorem4), andR⁡\(1\)=F​\(1\)∗R\(1\)=F\(1\)^\{\*\}\. Apply the scalar three\-lines theorem to matrix coefficients\([Tropp, 2022](https://arxiv.org/html/2610.41074#bib.bib69), Theorem 7\.13\): it gives‖F⁡\(ω\)‖≤Kproj1−ω\\\|F\(\\omega\)\\\|\\leq K\_\{\\rm proj\}^\{1\-\\omega\}and‖R⁡\(ω\)‖≤Ksk1−ω\\\|R\(\\omega\)\\\|\\leq K\_\{\\rm sk\}^\{1\-\\omega\}\. SinceF⁡\(ω\)​R​\(ω\)=IF\(\\omega\)R\(\\omega\)=I,‖u‖2≤‖F​\(ω\)∗​u‖​‖R⁡\(ω\)​u‖\\\|u\\\|^\{2\}\\leq\\\|F\(\\omega\)^\{\*\}u\\\|\\\|R\(\\omega\)u\\\|impliesKsk−2​\(1−ω\)​I⪯F⁡\(ω\)​F​\(ω\)∗⪯Kproj2​\(1−ω\)​IK\_\{\\rm sk\}^\{\-2\(1\-\\omega\)\}I\\preceq F\(\\omega\)F\(\\omega\)^\{\*\}\\preceq K\_\{\\rm proj\}^\{2\(1\-\\omega\)\}I\. SinceF⁡\(ω\)​F​\(ω\)∗=Gω/2​LS​Σ−ω​LS⊤​Gω/2F\(\\omega\)F\(\\omega\)^\{\*\}=G^\{\\omega/2\}L\_\{S\}\\Sigma^\{\-\\omega\}L\_\{S\}^\{\\top\}G^\{\\omega/2\}, congruence byG−ω/2G^\{\-\\omega/2\}and the assumed bounds onCθC\_\{\\theta\}prove \([61](https://arxiv.org/html/2610.41074#A2.E61)\)\. Finallyϑjλjω=\(λjϑj\)λj−b/a∈\[χ−,χ\+\]\\vartheta\_\{j\}\\lambda\_\{j\}^\{\\omega\}=\(\\lambda\_\{j\}\\vartheta\_\{j\}\)\\lambda\_\{j\}^\{\-b/a\}\\in\[\\chi\_\{\-\},\\chi\_\{\+\}\]proves the model specialization without commutingVVandGG\. ∎

###### Lemma B\.7\(Simultaneous rounding\-coordinate geometry\)\.

Almost surelyGGhas simple positive spectrum\. Its ordered orthonormal frameF⁡\(G\)=\(ϕ1,…,ϕM\)F\(G\)=\(\\phi\_\{1\},\\ldots,\\phi\_\{M\}\), with each column’s first nonzero coordinate positive and extended byIIoff this set, is Borel\. For independent Rademacher signsϵi\\epsilon\_\{i\}, also independent ofSS,O=F⁡\(G\)​diag⁡\(ϵ\)O=F\(G\)\\operatorname\{diag\}\(\\epsilon\)is Haar, independent of the spectrum and every rotation\-invariant Borel sketch statistic\. ForO∈O⁡\(M\)O\\in O\(M\)define the exact tests

ℋM=⋂i,j\{12​M≤∑kOk​i2Ok​j2≤16M,∑kOk​i2Ok​j4≤16M2\}\.\\mathcal\{H\}\_\{M\}=\\bigcap\_\{i,j\}\\left\\\{\\frac\{1\}\{2M\}\\leq\\sum\_\{k\}O\_\{ki\}^\{2\}O\_\{kj\}^\{2\}\\leq\\frac\{16\}\{M\},\\quad\\sum\_\{k\}O\_\{ki\}^\{2\}O\_\{kj\}^\{4\}\\leq\\frac\{16\}\{M^\{2\}\}\\right\\\}\.\(62\)The eventΩMH=\{G​simple positive,F⁡\(G\)∈ℋM\}\\Omega\_\{M\}^\{H\}=\\\{G\\text\{ simple positive\},F\(G\)\\in\\mathcal\{H\}\_\{M\}\\\}is Borel\. For eachD\>0D\>0it fails with probability at mostKD​M−DK\_\{D\}M^\{\-D\}\. On it, for every positive spectrum,

1/2≤ϕj⊤Jϕj≤16,ϕj⊤J2ϕj≤16\(1≤j≤M\)\.1/2\\leq\\phi\_\{j\}^\{\\top\}J\\phi\_\{j\}\\leq 16,\\qquad\\phi\_\{j\}^\{\\top\}J^\{2\}\\phi\_\{j\}\\leq 16\\quad\(1\\leq j\\leq M\)\.\(63\)

###### Proof\.

A leadingMM\-column minor ofSSand the spectral discriminant ofGGare nonzero polynomials, as shown byS=\[diag⁡\(j/λj\)j≤M​0\]S=\[\\operatorname\{diag\}\(\\sqrt\{j/\\lambda\_\{j\}\}\)\_\{j\\leq M\}\\ 0\]\. HenceGGhas simple positive spectrum almost surely\. On the simple positive set, the continuous spectral projectorPj​\(A\)=∏i≠j\(A−gi​\(A\)​I\)/\(gj​\(A\)−gi​\(A\)\)P\_\{j\}\(A\)=\\prod\_\{i\\neq j\}\(A\-g\_\{i\}\(A\)I\)/\(g\_\{j\}\(A\)\-g\_\{i\}\(A\)\)gives the Borel columnPj​\(A\)​ek/Pj​\(A\)k​kP\_\{j\}\(A\)e\_\{k\}/\\sqrt\{P\_\{j\}\(A\)\_\{kk\}\}, choosing its first positive diagonal entrykk\. For each orthogonalUU,F⁡\(U​G​U⊤\)=U​F​\(G\)​DU​\(G\)F\(UGU^\{\\top\}\)=UF\(G\)D\_\{U\}\(G\)for diagonal signsDUD\_\{U\}\. The Gaussian density is left\-rotation invariant\. Averaging the column signs gives\(Z⁡\(S\),U​O\)​=𝑑​\(Z⁡\(S\),O\)\(Z\(S\),UO\)\\overset\{d\}\{=\}\(Z\(S\),O\)for every invariantZZ\. IntegratingUUagainst Haar probability proves the Haar law and independence\([Meckes, 2019](https://arxiv.org/html/2610.41074#bib.bib70), Theorem 1\.4\)\.

In the Gaussian Gram–Schmidt construction\([Meckes, 2019](https://arxiv.org/html/2610.41074#bib.bib70), Section 1\.2\), distinct Haar columns areu=x/‖x‖u=x/\\\|x\\\|,v=\(y−γ​x\)/‖y−γ​x‖v=\(y\-\\gamma x\)/\\\|y\-\\gamma x\\\|, for independent standard Gaussian vectorsx,yx,y\. SetAr​s=M−1​∑kxkr​yksA\_\{rs\}=M^\{\-1\}\\sum\_\{k\}x\_\{k\}^\{r\}y\_\{k\}^\{s\},γ=A11/A20\\gamma=A\_\{11\}/A\_\{20\}andb⟂=A02−A112/A20b\_\{\\perp\}=A\_\{02\}\-A\_\{11\}^\{2\}/A\_\{20\}\. Let𝒫M\\mathcal\{P\}\_\{M\}be the event that\|Ar​s−\(𝔼​Zr\)​\(𝔼​Zs\)\|≤1/100\|A\_\{rs\}\-\(\\mathbb\{E\}Z^\{r\}\)\(\\mathbb\{E\}Z^\{s\}\)\|\\leq 1/100for everyr\+s≤6r\+s\\leq 6,Z∼N⁡\(0,1\)Z\\sim N\(0,1\)\. For any integerr≥1r\\geq 1and independent copies of a centeredY1Y\_\{1\}with𝔼​\|Y1\|2​r<∞\\mathbb\{E\}\|Y\_\{1\}\|^\{2r\}<\\infty, expansion and Hölder give

𝔼​\|M−1​∑kYk\|2​r≤\(2​r\)2​r​𝔼​\|Y1\|2​r​M−r\.\\mathbb\{E\}\\left\|M^\{\-1\}\\sum\_\{k\}Y\_\{k\}\\right\|^\{2r\}\\leq\(2r\)^\{2r\}\\mathbb\{E\}\|Y\_\{1\}\|^\{2r\}M^\{\-r\}\.\(64\)In the expansion, singleton indices vanish; at most\(2​r\)2​r​Mr\(2r\)^\{2r\}M^\{r\}terms remain, each bounded by𝔼​\|Y1\|2​r\\mathbb\{E\}\|Y\_\{1\}\|^\{2r\}by Hölder\. Gaussian monomials have every finite moment\. Markov and a union bound over 28 such monomials therefore give arbitrarily large polynomial decay and

KD=1\+supn≥2nD\+2​Pr⁡\(𝒫nc\)<∞\.K\_\{D\}=1\+\\sup\_\{n\\geq 2\}n^\{D\+2\}\\Pr\(\\mathcal\{P\}\_\{n\}^\{c\}\)<\\infty\.\(65\)On𝒫M\\mathcal\{P\}\_\{M\},A20,A02∈\[\.99,1\.01\]A\_\{20\},A\_\{02\}\\in\[\.99,1\.01\],\|γ\|≤1/99\|\\gamma\|\\leq 1/99andb⟂∈\[98/99,1\.01\]b\_\{\\perp\}\\in\[98/99,1\.01\]\. The exact ratios are

M​∑kuk2​vk2\\displaystyle M\\sum\_\{k\}u\_\{k\}^\{2\}v\_\{k\}^\{2\}=A22−2​γ​A31\+γ2​A40A20​b⟂,\\displaystyle=\\frac\{A\_\{22\}\-2\\gamma A\_\{31\}\+\\gamma^\{2\}A\_\{40\}\}\{A\_\{20\}b\_\{\\perp\}\},M2​∑kuk2​vk4\\displaystyle M^\{2\}\\sum\_\{k\}u\_\{k\}^\{2\}v\_\{k\}^\{4\}=A24−4​γ​A33\+6​γ2​A42−4​γ3​A51\+γ4​A60A20​b⟂2\.\\displaystyle=\\frac\{A\_\{24\}\-4\\gamma A\_\{33\}\+6\\gamma^\{2\}A\_\{42\}\-4\\gamma^\{3\}A\_\{51\}\+\\gamma^\{4\}A\_\{60\}\}\{A\_\{20\}b\_\{\\perp\}^\{2\}\}\.Substituting the bounds defining𝒫M\\mathcal\{P\}\_\{M\}shows that the first ratio lies in\(1/2,2\)\(1/2,2\)and the second is below44\. For a repeated column, the ratios areA40/A202∈\(1/2,4\)A\_\{40\}/A\_\{20\}^\{2\}\\in\(1/2,4\)andA60/A203<16A\_\{60\}/A\_\{20\}^\{3\}<16\. Thus the corresponding pair tests hold\. A union bound overM2M^\{2\}pairs with \([65](https://arxiv.org/html/2610.41074#A2.E65)\) proves the failure bound; forM=1M=1the tests hold identically\. They are invariant under column signs, so the event is Borel in the sketch alone\.

Withπi=gi/θG\\pi\_\{i\}=g\_\{i\}/\\theta\_\{G\},ϕj⊤​J​ϕj=M​∑iπi​∑kOk​i2​Ok​j2\\phi\_\{j\}^\{\\top\}J\\phi\_\{j\}=M\\sum\_\{i\}\\pi\_\{i\}\\sum\_\{k\}O\_\{ki\}^\{2\}O\_\{kj\}^\{2\}\. Jensen also givesϕj⊤​J2​ϕj≤M2​∑iπi​∑kOk​j2​Ok​i4\\phi\_\{j\}^\{\\top\}J^\{2\}\\phi\_\{j\}\\leq M^\{2\}\\sum\_\{i\}\\pi\_\{i\}\\sum\_\{k\}O\_\{kj\}^\{2\}O\_\{ki\}^\{4\}\. The pair tests prove \([63](https://arxiv.org/html/2610.41074#A2.E63)\)\. Because the eigenframe is independent of rotation\-invariant sketch statistics, the same probability bound holds conditional on any such statistic, including the eventℰM\\mathcal\{E\}\_\{M\}\. ∎

Define

ΩM=ℰM∩ΩMH,𝒥M±=\{\(S,ϵ\):S∈ℰM,G​simple positive,F⁡\(G\)​diag⁡\(ϵ\)∈ℋM,ϵ∈\{−1,1\}M\}\.\\begin\{gathered\}\\Omega\_\{M\}=\\mathcal\{E\}\_\{M\}\\cap\\Omega\_\{M\}^\{H\},\\quad\\mathcal\{J\}\_\{M\}^\{\\pm\}=\\left\\\{\(S,\\epsilon\):\\begin\{gathered\}S\\in\\mathcal\{E\}\_\{M\},\\quad G\\text\{ simple positive\},\\\\ F\(G\)\\operatorname\{diag\}\(\\epsilon\)\\in\\mathcal\{H\}\_\{M\},\\quad\\epsilon\\in\\\{\-1,1\\\}^\{M\}\\end\{gathered\}\\right\\\}\.\\end\{gathered\}\(66\)All tests are invariant under column signs\. Thus the consequences of the Haar calculation hold for the actual eigenvectors of every fixedS∈ΩMS\\in\\Omega\_\{M\}\. The training proof introduces no additional random frame\.

###### Definition B\.8\(Fixed constants\)\.

For the fixed population family andMwM\_\{\\rm w\}, let𝒲\\mathcal\{W\}be the class of positive tuplesW=\(cg,Cg,Cθtr,cV,CV,CAsch,cJ,CJ\)W=\(c\_\{g\},C\_\{g\},C\_\{\\theta\}^\{\\rm tr\},c\_\{V\},C\_\{V\},C\_\{A\}^\{\\rm sch\},c\_\{J\},C\_\{J\}\), withCθtr,CJ≥1C\_\{\\theta\}^\{\\rm tr\},C\_\{J\}\\geq 1, depending only on the fixed population family\. A tuple is valid when, for everyM≥MwM\\geq M\_\{\\rm w\}and every\(S,ϵ\)∈𝒥M±\(S,\\epsilon\)\\in\\mathcal\{J\}\_\{M\}^\{\\pm\},

cgj−a≤gj≤Cgj−a,θG≤Cθtr,cVG−\(a−b\)/a⪯V⪯CVG−\(a−b\)/a,0≤A¯≤CAschM1−b,cJ≤ϕj⊤Jϕj≤CJ,ϕj⊤J2ϕj≤CJ,ϕj=F⁡\(G\)​diag⁡\(ϵ\)​ej\.\\begin\{gathered\}c\_\{g\}j^\{\-a\}\\leq g\_\{j\}\\leq C\_\{g\}j^\{\-a\},\\quad\\theta\_\{G\}\\leq C\_\{\\theta\}^\{\\rm tr\},\\\\ c\_\{V\}G^\{\-\(a\-b\)/a\}\\preceq V\\preceq C\_\{V\}G^\{\-\(a\-b\)/a\},\\\\ 0\\leq\\bar\{A\}\\leq C\_\{A\}^\{\\rm sch\}M^\{1\-b\},\\quad c\_\{J\}\\leq\\phi\_\{j\}^\{\\top\}J\\phi\_\{j\}\\leq C\_\{J\},\\quad\\phi\_\{j\}^\{\\top\}J^\{2\}\\phi\_\{j\}\\leq C\_\{J\},\\\\ \\phi\_\{j\}=F\(G\)\\operatorname\{diag\}\(\\epsilon\)e\_\{j\}\.\\end\{gathered\}\(67\)Fix one valid tupleW∈𝒲W\\in\\mathcal\{W\}before choosing the learning\-rate and precision schedules, and writeΘ=Cθtr\\Theta=C\_\{\\theta\}^\{\\rm tr\}\. The constants below refer to this chosen tuple\.

The preceding lemmas show that𝒲\\mathcal\{W\}is nonempty: combine their spectral and teacher bounds withcJ=1/2c\_\{J\}=1/2,CJ=16C\_\{J\}=16, increasing the upper constants to at least one where required\. The geometric constantCAC\_\{A\}and the selected tuple componentCAschC\_\{A\}^\{\\rm sch\}need not be equal\.

###### Proposition B\.9\(Common sketch event\)\.

For everyM≥MwM\\geq M\_\{\\rm w\},ΩM\\Omega\_\{M\}in \([66](https://arxiv.org/html/2610.41074#A2.E66)\) is independent of the learning\-rate and precision schedules\. Its definition does not depend onDD, and every valid tuple satisfies \([67](https://arxiv.org/html/2610.41074#A2.E67)\) on it\. For each fixedD\>0D\>0,

Pr⁡\(ΩMc\)≤6​e−pgeo​M\+KD​M−D≤\[KD\+6​\(D/\(e​pgeo\)\)D\]​M−D\.\\Pr\(\\Omega\_\{M\}^\{c\}\)\\leq 6e^\{\-p\_\{\\rm geo\}M\}\+K\_\{D\}M^\{\-D\}\\leq\[K\_\{D\}\+6\(D/\(ep\_\{\\rm geo\}\)\)^\{D\}\]M^\{\-D\}\.\(68\)

###### Proof\.

The Borel property follows from the preceding constructions\. A union bound combines their failure probabilities; independence of the two events is unnecessary\. The final inequality follows fromsupx\>0xD​e−pgeo​x=\(D/\(e​pgeo\)\)D\\sup\_\{x\>0\}x^\{D\}e^\{\-p\_\{\\rm geo\}x\}=\(D/\(ep\_\{\\rm geo\}\)\)^\{D\}\. ∎

Finally, the unconditional lower approximation estimate and the chosen upper comparison give

cA​M1−b≤A¯≤CAsch​M1−b\.c\_\{A\}M^\{1\-b\}\\leq\\bar\{A\}\\leq C\_\{A\}^\{\\rm sch\}M^\{1\-b\}\.\(69\)

## Appendix CGeneral learning\-rate and precision schedules

This appendix bounds the second moments that determine prediction risk for general learning\-rate and precision schedules\. Theorem[C\.3](https://arxiv.org/html/2610.41074#Thmtheorem3a)gives risk identities and exact\-arithmetic moment bounds, and Lemma[C\.4](https://arxiv.org/html/2610.41074#Thmtheorem4a)controls the additional state moments caused by rounding\. Theorem[C\.6](https://arxiv.org/html/2610.41074#Thmtheorem6a)and Proposition[C\.7](https://arxiv.org/html/2610.41074#Thmtheorem7a)give matching bounds for the SR\-induced output risk, which Appendix[D\.1](https://arxiv.org/html/2610.41074#A4.SS1)converts into Theorem[1](https://arxiv.org/html/2610.41074#Thmtheorem1)\. Appendix[C\.6](https://arxiv.org/html/2610.41074#A3.SS6)constructs the admissibility thresholds used throughout\. In this appendix, condition on a sketchSSin the event of Proposition[B\.9](https://arxiv.org/html/2610.41074#Thmtheorem9)\. WriteΘ=Cθtr\\Theta=C\_\{\\theta\}^\{\\rm tr\}andτ∗=τ∗sch\\tau\_\{\*\}=\\tau\_\{\*\}^\{\\rm sch\}\. The symbolsc,Cc,Cdenote positive constants depending only on the fixed model, the comparison tuple of Definition[B\.8](https://arxiv.org/html/2610.41074#Thmtheorem8),c0c\_\{0\}, the minimum cumulative learning rateτ∗\\tau\_\{\*\}and the positive noise variance\. They do not depend on width, horizon, batch size or the learning\-rate and precision schedules\. Their construction is given in Appendix[C\.6](https://arxiv.org/html/2610.41074#A3.SS6)\.

### C\.1Schedules, output responses and admissibility

We define the output responses and feedback bounds used in Assumption[C\.1](https://arxiv.org/html/2610.41074#Thmtheorem1b)\. Use the model and independence assumptions of Section[2](https://arxiv.org/html/2610.41074#S2)and the auxiliary matrices and moments in \([34](https://arxiv.org/html/2610.41074#A2.E34)\), \([35](https://arxiv.org/html/2610.41074#A2.E35)\), \([40](https://arxiv.org/html/2610.41074#A2.E40)\) and \([41](https://arxiv.org/html/2610.41074#A2.E41)\)\. Define the exponents and propagation products along the eigenvectors ofGGby

ωV=\(a−b\)/a,α=1/a,ν=\(a−b\+1\)/a,\\displaystyle\\omega\_\{V\}=\(a\-b\)/a,\\qquad\\alpha=1/a,\\qquad\\nu=\(a\-b\+1\)/a,\(70\)πj​\(t,u\)=∏v=ut−1\(1−ηv​gj\),πj​\(t,t\)=1\.\\displaystyle\\pi\_\{j\}\(t,u\)=\\prod\_\{v=u\}^\{t\-1\}\(1\-\\eta\_\{v\}g\_\{j\}\),\\qquad\\pi\_\{j\}\(t,t\)=1\.\(71\)The parameter range1<b<a1<b<agives0<ωV<10<\\omega\_\{V\}<1and0<α<ν<10<\\alpha<\\nu<1\.

##### Mean propagation and output responses\.

UseHu,Pu→t,PtH\_\{u\},P\_\{u\\to t\},P\_\{t\}from \([48](https://arxiv.org/html/2610.41074#A2.E48)\)\. Using the eigenpairs of \([34](https://arxiv.org/html/2610.41074#A2.E34)\) and the output weights of \([39](https://arxiv.org/html/2610.41074#A2.E39)\), define

dout,j=∑t=1Twtoutπj\(t,0\),ℓout,j,u=∑t=u\+1Twtoutπj\(t,u\+1\)\.\\displaystyle d\_\{\{\\rm out\},j\}=\\sum\_\{t=1\}^\{T\}w\_\{t\}^\{\{\\rm out\}\}\\pi\_\{j\}\(t,0\),\\qquad\\ell\_\{\{\\rm out\},j,u\}=\\sum\_\{t=u\+1\}^\{T\}w\_\{t\}^\{\{\\rm out\}\}\\pi\_\{j\}\(t,u\+1\)\.\(72\)Empty products equal one\. An error introduced at updateuufirst enters stateu\+1u\+1\. For positive semidefiniteAA, its direct output response is

ℛuout​\(A\)=12​∑j=1Mgj​ℓout,j,u2​ϕj⊤​A​ϕj\.\\mathcal\{R\}\_\{u\}^\{\{\\rm out\}\}\(A\)=\\tfrac\{1\}\{2\}\\sum\_\{j=1\}^\{M\}g\_\{j\}\\ell\_\{\{\\rm out\},j,u\}^\{2\}\\phi\_\{j\}^\{\\top\}A\\phi\_\{j\}\.\(73\)This is the prediction variance produced by covarianceAAunder the subsequent mean updates and output averaging\. Stochastic feedback is accounted for separately below\. WritePout=∑t=1Twtout​PtP\_\{\\rm out\}=\\sum\_\{t=1\}^\{T\}w\_\{t\}^\{\\rm out\}P\_\{t\}andLT,u=∑t=u\+1Twtout​Pu\+1→tL\_\{T,u\}=\\sum\_\{t=u\+1\}^\{T\}w\_\{t\}^\{\\rm out\}P\_\{u\+1\\to t\}\. Equivalently,

ℛuout​\(A\)=12​tr⁡\(G​LT,u​A​LT,u𝖳\)\.\\mathcal\{R\}\_\{u\}^\{\\rm out\}\(A\)=\\tfrac\{1\}\{2\}\\operatorname\{tr\}\(GL\_\{T,u\}AL\_\{T,u\}^\{\\mathsf\{T\}\}\)\.\(74\)The response is nonnegative forA⪰0A\\succeq 0\. Considerout∈\{last,avg\}\{\\rm out\}\\in\\\{\\mathrm\{last\},\\mathrm\{avg\}\\\}\. Define the bias and SGD\-variance functionals by

ℬout=12​∑j=1Mgjb/a​dout,j2,𝒱out=∑u<Thu​Ru​ℛuout​\(G\),\\mathcal\{B\}^\{\{\\rm out\}\}=\\tfrac\{1\}\{2\}\\sum\_\{j=1\}^\{M\}g\_\{j\}^\{b/a\}d\_\{\{\\rm out\},j\}^\{2\},\\qquad\\mathcal\{V\}^\{\{\\rm out\}\}=\\sum\_\{u<T\}h\_\{u\}R\_\{u\}\\mathcal\{R\}\_\{u\}^\{\{\\rm out\}\}\(G\),\(75\)wherehu,Ruh\_\{u\},R\_\{u\}are given in \([7](https://arxiv.org/html/2610.41074#S2.E7)\) and \([40](https://arxiv.org/html/2610.41074#A2.E40)\)\. On the sketch event, Definition[B\.8](https://arxiv.org/html/2610.41074#Thmtheorem8)comparesℬout\\mathcal\{B\}^\{\{\\rm out\}\}with the exact mean bias12​∑jgj​dout,j2​ϕj⊤​V​ϕj\\frac\{1\}\{2\}\\sum\_\{j\}g\_\{j\}d\_\{\{\\rm out\},j\}^\{2\}\\phi\_\{j\}^\{\\top\}V\\phi\_\{j\}\.

##### Rounding sources and covariance comparisons\.

Use the lower and upper rounding coefficientsfuϵ,βuϵ,muϵf\_\{u\}^\{\\epsilon\},\\beta\_\{u\}^\{\\epsilon\},m\_\{u\}^\{\\epsilon\}of \([46](https://arxiv.org/html/2610.41074#A2.E46)\) and \([47](https://arxiv.org/html/2610.41074#A2.E47)\)\. For brevity, writecf=cb=3/4c\_\{f\}=c\_\{b\}=3/4andcm=3/16c\_\{m\}=3/16for coefficients in \([46](https://arxiv.org/html/2610.41074#A2.E46)\)\. The exact\-arithmetic comparison uses the same width, learning rates, batch size and output\. No commutation ofG,V,JG,V,Jis assumed\. Forϵ∈\{−,\+\}\\epsilon\\in\\\{\-,\+\\\}, writextϵ=tr⁡\(G​Xtϵ\)x\_\{t\}^\{\\epsilon\}=\\operatorname\{tr\}\(GX\_\{t\}^\{\\epsilon\}\)andytϵ=tr⁡\(J​Xtϵ\)y\_\{t\}^\{\\epsilon\}=\\operatorname\{tr\}\(JX\_\{t\}^\{\\epsilon\}\), whereXtϵX\_\{t\}^\{\\epsilon\}is the comparison second\-moment gap from Appendix[B](https://arxiv.org/html/2610.41074#A2)\. WriteQuϵ=Quϵ​\(Xuϵ\)Q\_\{u\}^\{\\epsilon\}=Q\_\{u\}^\{\\epsilon\}\(X\_\{u\}^\{\\epsilon\}\), with the affine map defined in \([53](https://arxiv.org/html/2610.41074#A2.E53)\)\. Its direct source is

Duϵ=fuϵ​Uu​G\+\(βuϵ​Ru\+muϵ​Uu\)​J\.D\_\{u\}^\{\\epsilon\}=f\_\{u\}^\{\\epsilon\}U\_\{u\}G\+\(\\beta\_\{u\}^\{\\epsilon\}R\_\{u\}\+m\_\{u\}^\{\\epsilon\}U\_\{u\}\)J\.\(76\)For the exact\-arithmetic recursion, write

Nu=Ru​G\+G​Cu0​G\.N\_\{u\}=R\_\{u\}G\+GC\_\{u\}^\{0\}G\.\(77\)A missing comparison superscript means plus\. These definitions also apply at zero learning rates and at exact precision, since they involve no division byhuh\_\{u\}or a rounding coefficient\. Foru<tu<t, define the propagation kernels in aGG\-eigenbasis:

𝒜u​\(Z\)=Hu​Z​Hu\+hu​G​Z​G,ai​j,u=\(1−ηu​gi\)​\(1−ηu​gj\)\+hu​gi​gj,\\displaystyle\\mathcal\{A\}\_\{u\}\(Z\)=H\_\{u\}ZH\_\{u\}\+h\_\{u\}GZG,\\qquad a\_\{ij,u\}=\(1\-\\eta\_\{u\}g\_\{i\}\)\(1\-\\eta\_\{u\}g\_\{j\}\)\+h\_\{u\}g\_\{i\}g\_\{j\},Di​j​\(t,u\)=∏v=u\+1t−1ai​j,v,dj​\(t,u\)=Dj​j​\(t,u\),ℓt,u=τt−τu\+1,\\displaystyle D\_\{ij\}\(t,u\)=\\prod\_\{v=u\+1\}^\{t\-1\}a\_\{ij,v\},\\quad d\_\{j\}\(t,u\)=D\_\{jj\}\(t,u\),\\quad\\ell\_\{t,u\}=\\tau\_\{t\}\-\\tau\_\{u\+1\},Sℓ𝒜​\(t,u\)=∑jgjℓ​dj​\(t,u\),SℓH​\(t,u\)=∑jgjℓ​πj​\(t,u\+1\)2,\\displaystyle S\_\{\\ell\}^\{\\mathcal\{A\}\}\(t,u\)=\\sum\_\{j\}g\_\{j\}^\{\\ell\}d\_\{j\}\(t,u\),\\quad S\_\{\\ell\}^\{H\}\(t,u\)=\\sum\_\{j\}g\_\{j\}^\{\\ell\}\\pi\_\{j\}\(t,u\+1\)^\{2\},KP​Q𝒜\(t,u\)=tr\(P𝒜t−1⋯𝒜u\+1\(Q\)\),\\displaystyle K\_\{PQ\}^\{\\mathcal\{A\}\}\(t,u\)=\\operatorname\{tr\}\(P\\mathcal\{A\}\_\{t\-1\}\\cdots\\mathcal\{A\}\_\{u\+1\}\(Q\)\),KP​QH​\(t,u\)=tr⁡\(P​Pu\+1→t​Q​Pu\+1→t\),P,Q∈\{G,J\}\.\\displaystyle K\_\{PQ\}^\{H\}\(t,u\)=\\operatorname\{tr\}\(PP\_\{u\+1\\to t\}QP\_\{u\+1\\to t\}\),\\qquad P,Q\\in\\\{G,J\\\}\.\(78\)Empty products of scalar factors and of linear maps are, respectively, one and the identity map\. PutAuϵ=xuϵ\+fuϵ​\(Uu\+yuϵ\)A\_\{u\}^\{\\epsilon\}=x\_\{u\}^\{\\epsilon\}\+f\_\{u\}^\{\\epsilon\}\(U\_\{u\}\+y\_\{u\}^\{\\epsilon\}\)andBuϵ=βuϵ​\(Ru\+xuϵ\)\+muϵ​\(Uu\+yuϵ\)B\_\{u\}^\{\\epsilon\}=\\beta\_\{u\}^\{\\epsilon\}\(R\_\{u\}\+x\_\{u\}^\{\\epsilon\}\)\+m\_\{u\}^\{\\epsilon\}\(U\_\{u\}\+y\_\{u\}^\{\\epsilon\}\)\. Thus the comparison recursion isXu\+1ϵ=𝒜u​\(Xuϵ\)\+hu​\(Auϵ​G\+Buϵ​J\)X\_\{u\+1\}^\{\\epsilon\}=\\mathcal\{A\}\_\{u\}\(X\_\{u\}^\{\\epsilon\}\)\+h\_\{u\}\(A\_\{u\}^\{\\epsilon\}G\+B\_\{u\}^\{\\epsilon\}J\)\. The two sources coupling different rounding locations are

Ztb,ϵ=∑u<thu​βuϵ​Ru​KJ​JH​\(t,u\),Ytf,ϵ=∑u<thu​fuϵ​Uu​S2H​\(t,u\)\.Z\_\{t\}^\{b,\\epsilon\}=\\sum\_\{u<t\}h\_\{u\}\\beta\_\{u\}^\{\\epsilon\}R\_\{u\}K\_\{JJ\}^\{H\}\(t,u\),\\qquad Y\_\{t\}^\{f,\\epsilon\}=\\sum\_\{u<t\}h\_\{u\}f\_\{u\}^\{\\epsilon\}U\_\{u\}S\_\{2\}^\{H\}\(t,u\)\.\(79\)The first measures the parameter second moment contributed by earlier backward rounding; the second measures the residual second moment contributed by earlier forward rounding\. Multiplying them by a later forward or backward rounding coefficient produces the two temporal interaction terms\. All sums are strictly causal: a source introduced at updateuufirst enters stateu\+1u\+1\.

##### Risk contributions from rounding\.

DefineΨϵout=∑u<Thu​ℛuout​\(Duϵ\),\\Psi\_\{\\epsilon\}^\{\\rm out\}=\\sum\_\{u<T\}h\_\{u\}\\mathcal\{R\}\_\{u\}^\{\\rm out\}\(D\_\{u\}^\{\\epsilon\}\),

Convb→fϵ,out\\displaystyle\\operatorname\{Conv\}\_\{b\\to f\}^\{\\epsilon,\\rm out\}=∑t<Tht​ftϵ​Ztb,ϵ​ℛtout​\(G\),Convf→bϵ,out=∑t<Tht​βtϵ​Ytf,ϵ​ℛtout​\(J\),\\displaystyle=\\sum\_\{t<T\}h\_\{t\}f\_\{t\}^\{\\epsilon\}Z\_\{t\}^\{b,\\epsilon\}\\mathcal\{R\}\_\{t\}^\{\\rm out\}\(G\),\\quad\\operatorname\{Conv\}\_\{f\\to b\}^\{\\epsilon,\\rm out\}=\\sum\_\{t<T\}h\_\{t\}\\beta\_\{t\}^\{\\epsilon\}Y\_\{t\}^\{f,\\epsilon\}\\mathcal\{R\}\_\{t\}^\{\\rm out\}\(J\),\(80\)𝒩out\\displaystyle\\mathcal\{N\}^\{\\rm out\}=Ψ\+out\+Convb→f\+,out\+Convf→b\+,out\.\\displaystyle=\\Psi\_\{\+\}^\{\\rm out\}\+\\operatorname\{Conv\}\_\{b\\to f\}^\{\+,\\rm out\}\+\\operatorname\{Conv\}\_\{f\\to b\}^\{\+,\\rm out\}\.\(81\)The direct termΨϵ\\Psi\_\{\\epsilon\}contains forward rounding, backward rounding and their same\-update interaction throughDuϵD\_\{u\}^\{\\epsilon\}\. The other two terms account for interactions across updates\. The sum𝒩out\\mathcal\{N\}^\{\\rm out\}is a covariance comparison functional for the SR\-induced error, rather than an exact variance identity\. A missing comparison superscript means the upper coefficient; all terms are nonnegative and strictly causal\. For positive cumulative learning rates, define the residual comparison

rt=s\+τt−\(b−1\)/a,τt\>0,t≠0\.r\_\{t\}=s\+\\tau\_\{t\}^\{\-\(b\-1\)/a\},\\qquad\\tau\_\{t\}\>0,\\quad t\\neq 0\.\(82\)The eventΩM\\Omega\_\{M\}and width thresholdMwM\_\{\\rm w\}are defined in Proposition[B\.9](https://arxiv.org/html/2610.41074#Thmtheorem9)\. On this event, the fixed tuple of Definition[B\.8](https://arxiv.org/html/2610.41074#Thmtheorem8)implies \([67](https://arxiv.org/html/2610.41074#A2.E67)\) and the approximation bounds \([69](https://arxiv.org/html/2610.41074#A2.E69)\)\. Appendix[C\.6](https://arxiv.org/html/2610.41074#A3.SS6)fixesτ∗,ρsch,κsch\\tau\_\{\*\},\\rho\_\{\\rm sch\},\\kappa\_\{\\rm sch\}from this tuple,c0c\_\{0\}and the model, before any schedule is selected\. Writeη¯=c0/cg\\bar\{\\eta\}=c\_\{0\}/c\_\{g\},ρsrc=1/\(4​Θ\)\\rho\_\{\\rm src\}=1/\(4\\Theta\)andκ=κsch\\kappa=\\kappa\_\{\\rm sch\}\.

##### Feedback quantities\.

For any finite schedule at a full\-row\-rank sketch define, even at zero clock,

Γstab​\(T\)=∑u<Thu​qfw,u​qbw,u/M,\\displaystyle\\Gamma\_\{\\rm stab\}\(T\)=\\sum\_\{u<T\}h\_\{u\}q\_\{\{\\rm fw\},u\}q\_\{\{\\rm bw\},u\}/M,Γf→b\(T\)=maxt<T:qbw,t\>0∑u<thu​qfw,u​Uu​S2H​\(t,u\)M​Rt\+qfw,t​Ut,\\displaystyle\\Gamma\_\{f\\to b\}\(T\)=\\max\_\{t<T:q\_\{\{\\rm bw\},t\}\>0\}\\frac\{\\sum\_\{u<t\}h\_\{u\}q\_\{\{\\rm fw\},u\}U\_\{u\}S\_\{2\}^\{H\}\(t,u\)\}\{MR\_\{t\}\+q\_\{\{\\rm fw\},t\}U\_\{t\}\},\(83\)withmax⁡∅=0\.\\max\\varnothing=0\.For a time horizon withτT≥τ∗\\tau\_\{T\}\\geq\\tau\_\{\*\}, userT≠0r\_\{T\}\\neq 0from \([82](https://arxiv.org/html/2610.41074#A3.E82)\) and define

Γ×​\(T\)=maxt<T⁡qfw,tM​rT​∑u<thu​qbw,u​Ru,\\displaystyle\\Gamma\_\{\\times\}\(T\)=\\max\_\{t<T\}\\frac\{q\_\{\{\\rm fw\},t\}\}\{Mr\_\{T\}\}\\sum\_\{u<t\}h\_\{u\}q\_\{\{\\rm bw\},u\}R\_\{u\},Γconv​\(T\)=max⁡\(Γ×​\(T\),Γf→b​\(T\)\)\.\\displaystyle\\Gamma\_\{\\rm conv\}\(T\)=\\max\(\\Gamma\_\{\\times\}\(T\),\\Gamma\_\{f\\to b\}\(T\)\)\.\(84\)All maxima and source sums are finite and all denominators are positive\.

###### Assumption C\.1\(Admissible learning\-rate and precision schedules\)\.

The learning\-rate and precision schedules are deterministic and chosen before all random draws\. For the horizonTT, they satisfy the rate and clock conditions \([16](https://arxiv.org/html/2610.41074#S3.E16)\) for0≤u<T0\\leq u<T\(withτ∗=τ∗sch\\tau\_\{\*\}=\\tau\_\{\*\}^\{\\rm sch\}\), together with

Γstab​\(T\)≤κ,Γconv​\(T\)≤κ\.\\Gamma\_\{\\rm stab\}\(T\)\\leq\\kappa,\\qquad\\Gamma\_\{\\rm conv\}\(T\)\\leq\\kappa\.\(85\)Such a schedule and time horizon are called admissible\.

##### Consequences of admissibility\.

Chooseρsch≤ρsrc\\rho\_\{\\rm sch\}\\leq\\rho\_\{\\rm src\}\. OnΩM\\Omega\_\{M\},g1≥cgg\_\{1\}\\geq c\_\{g\}andθG≤Θ\\theta\_\{G\}\\leq\\Theta, so admissibility gives

ηu≤η¯,ρu≤ρsch≤ρsrc,ρu​θG≤14\.\\eta\_\{u\}\\leq\\bar\{\\eta\},\\qquad\\rho\_\{u\}\\leq\\rho\_\{\\rm sch\}\\leq\\rho\_\{\\rm src\},\\qquad\\rho\_\{u\}\\theta\_\{G\}\\leq\\tfrac\{1\}\{4\}\.\(86\)ForMa≥τ∗M^\{a\}\\geq\\tau\_\{\*\}, sufficiently many small, equal exact\-arithmetic steps give an admissible schedule with total learning rateτ∗\\tau\_\{\*\}\. Unrolling the comparison recursion gives

\(xtϵytϵ\)=∑u<thu​\(KG​G𝒜KG​J𝒜KJ​G𝒜KJ​J𝒜\)​\(t,u\)​\(AuϵBuϵ\)\.\\binom\{x\_\{t\}^\{\\epsilon\}\}\{y\_\{t\}^\{\\epsilon\}\}=\\sum\_\{u<t\}h\_\{u\}\\begin\{pmatrix\}K\_\{GG\}^\{\\mathcal\{A\}\}&K\_\{GJ\}^\{\\mathcal\{A\}\}\\\\ K\_\{JG\}^\{\\mathcal\{A\}\}&K\_\{JJ\}^\{\\mathcal\{A\}\}\\end\{pmatrix\}\(t,u\)\\binom\{A\_\{u\}^\{\\epsilon\}\}\{B\_\{u\}^\{\\epsilon\}\}\.\(87\)All kernels and sources are nonnegative\.

### C\.2Spectral propagation and causal compositions

Lemma[C\.2](https://arxiv.org/html/2610.41074#Thmtheorem2a)compares propagation and spectral sums with scalar decay functions used in the subsequent risk bounds\. Use the propagation products, kernels and spectral sums in \([71](https://arxiv.org/html/2610.41074#A3.E71)\) and \([78](https://arxiv.org/html/2610.41074#A3.E78)\)\. Forv≥0v\\geq 0, writeφ0​\(v\)=M\\varphi\_\{0\}\(v\)=Mandφi​\(v\)=\(1\+v\)α−i\\varphi\_\{i\}\(v\)=\(1\+v\)^\{\\alpha\-i\}fori=1,2i=1,2\. SetΛ0=\(1−c0\)−1\\Lambda\_\{0\}=\(1\-c\_\{0\}\)^\{\-1\}, and abbreviatedjH=πj​\(t,u\+1\)2d\_\{j\}^\{H\}=\\pi\_\{j\}\(t,u\+1\)^\{2\}anddj𝒜=dj​\(t,u\)d\_\{j\}^\{\\mathcal\{A\}\}=d\_\{j\}\(t,u\)\. Foru<min⁡\(v,w\)u<\\min\(v,w\)andi=1,2i=1,2, define the paired sums

Λipair​\(u,v,w\)=∑jgji​πj​\(v,u\+1\)​πj​\(w,u\+1\),d=τv\+τw−2​τu\+1\.\\Lambda\_\{i\}^\{\\rm pair\}\(u;v,w\)=\\sum\_\{j\}g\_\{j\}^\{i\}\\pi\_\{j\}\(v,u\+1\)\\pi\_\{j\}\(w,u\+1\),\\qquad d=\\tau\_\{v\}\+\\tau\_\{w\}\-2\\tau\_\{u\+1\}\.\(88\)Letaj​\(x\)a\_\{j\}\(x\)denote eithere−gj​xe^\{\-g\_\{j\}x\}or a product of two propagation factors along the same eigenvector, whose intervals have total cumulative learning ratexx\. Empty sums and maxima are zero\.

###### Lemma C\.2\(Propagation, spectral sums and causal composition\)\.

Assume \([67](https://arxiv.org/html/2610.41074#A2.E67)\) and0≤ηu​g1≤c00\\leq\\eta\_\{u\}g\_\{1\}\\leq c\_\{0\}\. The constants below depend only on the fixed model, comparison bounds andc0c\_\{0\}\.

1. \(i\)*Products and kernel geometry\.*For every cumulative\-time interval, e−Λ0​gj​\(τt−τu\)≤πj​\(t,u\)≤e−gj​\(τt−τu\),1−πj​\(t,0\)≤min⁡\(1,gj​τt\)\.e^\{\-\\Lambda\_\{0\}g\_\{j\}\(\\tau\_\{t\}\-\\tau\_\{u\}\)\}\\leq\\pi\_\{j\}\(t,u\)\\leq e^\{\-g\_\{j\}\(\\tau\_\{t\}\-\\tau\_\{u\}\)\},\\qquad 1\-\\pi\_\{j\}\(t,0\)\\leq\\min\(1,g\_\{j\}\\tau\_\{t\}\)\.\(89\)For𝖷∈\{H,𝒜\}\\mathsf\{X\}\\in\\\{H,\\mathcal\{A\}\\\}, suppressing\(t,u\)\(t,u\), KG​G𝖷=S2𝖷,KG​J𝖷=KJ​G𝖷=∑jgj​Jj​j​dj𝖷,\\displaystyle K\_\{GG\}^\{\\mathsf\{X\}\}=S\_\{2\}^\{\\mathsf\{X\}\},\\qquad K\_\{GJ\}^\{\\mathsf\{X\}\}=K\_\{JG\}^\{\\mathsf\{X\}\}=\\sum\_\{j\}g\_\{j\}J\_\{jj\}d\_\{j\}^\{\\mathsf\{X\}\},cJ​S1𝖷≤KG​J𝖷≤CJ​S1𝖷,cJ2​S0𝖷≤KJ​J𝖷≤CJ​S0𝖷\.\\displaystyle c\_\{J\}S\_\{1\}^\{\\mathsf\{X\}\}\\leq K\_\{GJ\}^\{\\mathsf\{X\}\}\\leq C\_\{J\}S\_\{1\}^\{\\mathsf\{X\}\},\\qquad c\_\{J\}^\{2\}S\_\{0\}^\{\\mathsf\{X\}\}\\leq K\_\{JJ\}^\{\\mathsf\{X\}\}\\leq C\_\{J\}S\_\{0\}^\{\\mathsf\{X\}\}\.\(90\)
2. \(ii\)*Spectral comparisons\.*There are0<cP≤1≤CP0<c\_\{P\}\\leq 1\\leq C\_\{P\}such that cP​φi​\(ℓt,u\)≤Si𝖷​\(t,u\)≤CP​φi​\(ℓt,u\),\\displaystyle c\_\{P\}\\varphi\_\{i\}\(\\ell\_\{t,u\}\)\\leq S\_\{i\}^\{\\mathsf\{X\}\}\(t,u\)\\leq C\_\{P\}\\varphi\_\{i\}\(\\ell\_\{t,u\}\),\(91\)KP​QH≤KP​Q𝒜≤C3​KP​QH\(P,Q∈\{G,J\}\),\\displaystyle K\_\{PQ\}^\{H\}\\leq K\_\{PQ\}^\{\\mathcal\{A\}\}\\leq C\_\{3\}K\_\{PQ\}^\{H\}\\quad\(P,Q\\in\\\{G,J\\\}\),\(92\)for0≤ℓt,u≤Ma0\\leq\\ell\_\{t,u\}\\leq M^\{a\}andi=0,1,2i=0,1,2\. The upper profile bounds hold for all nonnegative gaps\. Forr∈\{1,2,b/a,2−ωV\}r\\in\\\{1,2,b/a,2\-\\omega\_\{V\}\\\}, there are0<ℓr≤ur<∞0<\\ell\_\{r\}\\leq u\_\{r\}<\\inftywith ℓr​\(1\+x\)α−r≤∑jgjr​aj​\(x\)≤ur​\(1\+x\)α−r\.\\ell\_\{r\}\(1\+x\)^\{\\alpha\-r\}\\leq\\sum\_\{j\}g\_\{j\}^\{r\}a\_\{j\}\(x\)\\leq u\_\{r\}\(1\+x\)^\{\\alpha\-r\}\.\(93\)Its lower bound requiresx≤2​Max\\leq 2M^\{a\}and its upper bound is global\. Set KS=max⁡\{22−α​CP,cP−1\}\.K\_\{S\}=\\max\\\{2^\{2\-\\alpha\}C\_\{P\},c\_\{P\}^\{\-1\}\\\}\.\(94\)Then, fori=1,2i=1,2, KS−1​φi​\(d\)≤Λipair​\(u,v,w\)≤KS​φi​\(d\)K\_\{S\}^\{\-1\}\\varphi\_\{i\}\(d\)\\leq\\Lambda\_\{i\}^\{\\rm pair\}\(u;v,w\)\\leq K\_\{S\}\\varphi\_\{i\}\(d\)\(95\)where the lower bound requiresd≤2​Mad\\leq 2M^\{a\}\. In particular, forℓt,u≤Ma\\ell\_\{t,u\}\\leq M^\{a\}, cJ2​cP​M≤KJ​JH​\(t,u\)≤CJ​M\.c\_\{J\}^\{2\}c\_\{P\}M\\leq K\_\{JJ\}^\{H\}\(t,u\)\\leq C\_\{J\}M\.\(96\)
3. \(iii\)*Telescoping and weighted sums\.*At every finite cumulative time, ∑u<tηu​gj​πj​\(t,u\+1\)=1−πj​\(t,0\),∑u<tηu​gj​πj​\(t,u\+1\)2≤1−πj​\(t,0\),\\displaystyle\\sum\_\{u<t\}\\eta\_\{u\}g\_\{j\}\\pi\_\{j\}\(t,u\+1\)=1\-\\pi\_\{j\}\(t,0\),\\qquad\\sum\_\{u<t\}\\eta\_\{u\}g\_\{j\}\\pi\_\{j\}\(t,u\+1\)^\{2\}\\leq 1\-\\pi\_\{j\}\(t,0\),\(97\)∑u<thu​S2H​\(t,u\)≤θG​maxu<t​ρu\.\\displaystyle\\sum\_\{u<t\}h\_\{u\}S\_\{2\}^\{H\}\(t,u\)\\leq\\theta\_\{G\}\\max\_\{u<t\}\\rho\_\{u\}\.\(98\)
4. \(iv\)*Causal sums and compositions\.*LetD=ℓt,u≤MaD=\\ell\_\{t,u\}\\leq M^\{a\}and𝖷,𝖸,𝖹∈\{H,𝒜\}\\mathsf\{X\},\\mathsf\{Y\},\\mathsf\{Z\}\\in\\\{H,\\mathcal\{A\}\\\}\. For\(i,j,m\)∈\{\(2,2,2\),\(2,1,1\),\(1,2,1\)\}\(i,j,m\)\\in\\\{\(2,2,2\),\(2,1,1\),\(1,2,1\)\\\}, ∑u<v<tηv​S1𝖷​\(t,v\)≤CK​M,∑u<v<tηv​S1𝖷​\(v,u\)≤CK​M,\\displaystyle\\sum\_\{u<v<t\}\\eta\_\{v\}S\_\{1\}^\{\\mathsf\{X\}\}\(t,v\)\\leq C\_\{K\}M,\\qquad\\sum\_\{u<v<t\}\\eta\_\{v\}S\_\{1\}^\{\\mathsf\{X\}\}\(v,u\)\\leq C\_\{K\}M,\(99\)∑u<v<tηv​S2𝖷​\(t,v\)≤CK,∑u<v<tηv​Si𝖷​\(t,v\)​Sj𝖸​\(v,u\)≤CK​Sm𝖹​\(t,u\),\\displaystyle\\sum\_\{u<v<t\}\\eta\_\{v\}S\_\{2\}^\{\\mathsf\{X\}\}\(t,v\)\\leq C\_\{K\},\\quad\\sum\_\{u<v<t\}\\eta\_\{v\}S\_\{i\}^\{\\mathsf\{X\}\}\(t,v\)S\_\{j\}^\{\\mathsf\{Y\}\}\(v,u\)\\leq C\_\{K\}S\_\{m\}^\{\\mathsf\{Z\}\}\(t,u\),\(100\)∑u<v<tηv​S1𝖷​\(t,v\)​S1𝖸​\(v,u\)≤CK​min⁡\{M​S1𝖹,S0𝖹\}​\(t,u\)\.\\displaystyle\\sum\_\{u<v<t\}\\eta\_\{v\}S\_\{1\}^\{\\mathsf\{X\}\}\(t,v\)S\_\{1\}^\{\\mathsf\{Y\}\}\(v,u\)\\leq C\_\{K\}\\min\\\{MS\_\{1\}^\{\\mathsf\{Z\}\},S\_\{0\}^\{\\mathsf\{Z\}\}\\\}\(t,u\)\.\(101\)The fixed\-terminal one\-kernel bounds also hold over the full historyv<tv<twhenτt≤Ma\\tau\_\{t\}\\leq M^\{a\}\. Replacingηv\\eta\_\{v\}byhvh\_\{v\}multiplies each bound by the maximumρv\\rho\_\{v\}over its summation indices\.

###### Proof\.

In eigenbasis ofGG, diagonal action gives

𝒜t−1⋯𝒜u\+1\(Q\)\\displaystyle\\mathcal\{A\}\_\{t\-1\}\\cdots\\mathcal\{A\}\_\{u\+1\}\(Q\)=∑i,jQi​jDi​j\(t,u\)ϕiϕj𝖳,KP​Q𝒜=∑i,jPi​jQi​jDi​j\.\\displaystyle=\\sum\_\{i,j\}Q\_\{ij\}D\_\{ij\}\(t,u\)\\phi\_\{i\}\\phi\_\{j\}^\{\\mathsf\{T\}\},\\quad K\_\{PQ\}^\{\\mathcal\{A\}\}=\\sum\_\{i,j\}P\_\{ij\}Q\_\{ij\}D\_\{ij\}\.\(102\)For mean propagation, replaceDi​jD\_\{ij\}byπi​\(t,u\+1\)​πj​\(t,u\+1\)\\pi\_\{i\}\(t,u\+1\)\\pi\_\{j\}\(t,u\+1\)\. The vectors\(1−ηv​gi,hv​gi\)\(1\-\\eta\_\{v\}g\_\{i\},\\sqrt\{h\_\{v\}\}g\_\{i\}\)give, by Cauchy–Schwarz,

0≤πi​πj≤Di​j≤di​dj≤\(di\+dj\)/2\.0\\leq\\pi\_\{i\}\\pi\_\{j\}\\leq D\_\{ij\}\\leq\\sqrt\{d\_\{i\}d\_\{j\}\}\\leq\(d\_\{i\}\+d\_\{j\}\)/2\.\(103\)Forx=ηv​gj∈\[0,c0\]x=\\eta\_\{v\}g\_\{j\}\\in\[0,c\_\{0\}\],e−Λ0​x≤1−x≤e−xe^\{\-\\Lambda\_\{0\}x\}\\leq 1\-x\\leq e^\{\-x\}and\(1−x\)2≤aj​j,v=1−2​x\+\(1\+B−1\)​x2≤e−2​\(1−c0\)​x\(1\-x\)^\{2\}\\leq a\_\{jj,v\}=1\-2x\+\(1\+B^\{\-1\}\)x^\{2\}\\leq e^\{\-2\(1\-c\_\{0\}\)x\}\. Multiplication and1−∏v\(1−xv\)≤∑vxv1\-\\prod\_\{v\}\(1\-x\_\{v\}\)\\leq\\sum\_\{v\}x\_\{v\}give \([89](https://arxiv.org/html/2610.41074#A3.E89)\) and

e−2​Λ0​gj​ℓt,u≤dj​\(t,u\)≤e−2​\(1−c0\)​gj​ℓt,u\.e^\{\-2\\Lambda\_\{0\}g\_\{j\}\\ell\_\{t,u\}\}\\leq d\_\{j\}\(t,u\)\\leq e^\{\-2\(1\-c\_\{0\}\)g\_\{j\}\\ell\_\{t,u\}\}\.\(104\)The first kernel identities follow by taking traces\. ForJ​JJJ, use

cJ2​∑jdj≤∑jJj​j2​dj≤∑i,jJi​j2​Di​j≤∑i\(J2\)i​i​di≤CJ​∑idi\.c\_\{J\}^\{2\}\\sum\_\{j\}d\_\{j\}\\leq\\sum\_\{j\}J\_\{jj\}^\{2\}d\_\{j\}\\leq\\sum\_\{i,j\}J\_\{ij\}^\{2\}D\_\{ij\}\\leq\\sum\_\{i\}\(J^\{2\}\)\_\{ii\}d\_\{i\}\\leq C\_\{J\}\\sum\_\{i\}d\_\{i\}\.This argument also applies to mean propagation\.

All required powers satisfy0<r≤20<r\\leq 2anda​r\>1ar\>1\. Forx≥1x\\geq 1, boundgjr​e−x​gj≤min⁡\{x−r,Cgr​j−a​r\}g\_\{j\}^\{r\}e^\{\-xg\_\{j\}\}\\leq\\min\\\{x^\{\-r\},C\_\{g\}^\{r\}j^\{\-ar\}\\\}and compare the decreasing sum with its integral\. Forx≤1x\\leq 1, sumCgr​j−a​rC\_\{g\}^\{r\}j^\{\-ar\}\. These calculations give the global upper order\(1\+x\)α−r\(1\+x\)^\{\\alpha\-r\}\. For a lower bound withx≤2​Max\\leq 2M^\{a\}, setk=max⁡\{1,\(x/2\)α\}≤Mk=\\max\\\{1,\(x/2\)^\{\\alpha\}\\\}\\leq M\. The interval\[k/2,k\]\[k/2,k\]contains at leastk/4k/4integer indices \(use the first index ifk<2k<2\), so

∑jgjr​e−Λ0​x​gj≥cgr​e−2a\+1​Cg​Λ04​\(1\+x/2\)α−r\.\\sum\_\{j\}g\_\{j\}^\{r\}e^\{\-\\Lambda\_\{0\}xg\_\{j\}\}\\geq\\frac\{c\_\{g\}^\{r\}e^\{\-2^\{a\+1\}C\_\{g\}\\Lambda\_\{0\}\}\}\{4\}\(1\+x/2\)^\{\\alpha\-r\}\.The last half of the eigenvalue indices gives the zero\-order lower coefficient12​e−2a\+1​Cg​Λ0\\frac\{1\}\{2\}e^\{\-2^\{a\+1\}C\_\{g\}\\Lambda\_\{0\}\}\. The product bounds now give \([93](https://arxiv.org/html/2610.41074#A3.E93)\) and \([91](https://arxiv.org/html/2610.41074#A3.E91)\), with the common constantscP,CPc\_\{P\},C\_\{P\}\. Together with the bounds in the eigenbasis ofGG, the factors1,CJ/cJ,CJ/cJ21,C\_\{J\}/c\_\{J\},C\_\{J\}/c\_\{J\}^\{2\}giveC3=\(CP/cP\)​max⁡\(1,CJ/cJ,CJ/cJ2\)C\_\{3\}=\(C\_\{P\}/c\_\{P\}\)\\max\(1,C\_\{J\}/c\_\{J\},C\_\{J\}/c\_\{J\}^\{2\}\)and \([96](https://arxiv.org/html/2610.41074#A3.E96)\)\. A paired product lies betweene−Λ0​gj​de^\{\-\\Lambda\_\{0\}g\_\{j\}d\}ande−gj​de^\{\-g\_\{j\}d\}\. Apply the same sum and integral estimates to these exponential sums at clockd/2d/2, and useφi​\(d\)≤φi​\(d/2\)≤2i−α​φi​\(d\)\\varphi\_\{i\}\(d\)\\leq\\varphi\_\{i\}\(d/2\)\\leq 2^\{i\-\\alpha\}\\varphi\_\{i\}\(d\)\. This proves \([95](https://arxiv.org/html/2610.41074#A3.E95)\) with \([94](https://arxiv.org/html/2610.41074#A3.E94)\)\.

The identityηu​gj​πj​\(t,u\+1\)=πj​\(t,u\+1\)−πj​\(t,u\)\\eta\_\{u\}g\_\{j\}\\pi\_\{j\}\(t,u\+1\)=\\pi\_\{j\}\(t,u\+1\)\-\\pi\_\{j\}\(t,u\)gives \([97](https://arxiv.org/html/2610.41074#A3.E97)\), since0<πj≤10<\\pi\_\{j\}\\leq 1\. Multiply its inequality bygjg\_\{j\}, sum over eigenvector indices and usehu≤\(maxv<t⁡ρv\)​ηuh\_\{u\}\\leq\(\\max\_\{v<t\}\\rho\_\{v\}\)\\eta\_\{u\}to obtain \([98](https://arxiv.org/html/2610.41074#A3.E98)\)\.

PutA=1\+η¯A=1\+\\bar\{\\eta\}and choose the fixed coefficientsF1=A/α,F2=A2−α/\(1−α\),Csc=8​A2​max⁡\(F1,F2\),F\_\{1\}=A/\\alpha,F\_\{2\}=A^\{2\-\\alpha\}/\(1\-\\alpha\),C\_\{\\rm sc\}=8A^\{2\}\\max\(F\_\{1\},F\_\{2\}\),and

CK=max⁡\{CP​F2,2α​CP​F1,2α​CP2​Csc/cP\}\.\\displaystyle C\_\{K\}=\\max\\\{C\_\{P\}F\_\{2\},2^\{\\alpha\}C\_\{P\}F\_\{1\},2^\{\\alpha\}C\_\{P\}^\{2\}C\_\{\\rm sc\}/c\_\{P\}\\\}\.\(105\)On\[b,b\+η\]\[b,b\+\\eta\],φi​\(b\)≤Ai−α​φi​\(s\)\\varphi\_\{i\}\(b\)\\leq A^\{i\-\\alpha\}\\varphi\_\{i\}\(s\)\. Integrating over disjoint intervals indicates

∑vηv​φ1​\(bv\)≤F1​\(1\+D\)α,∑vηv​φ2​\(bv\)≤F2\.\\sum\_\{v\}\\eta\_\{v\}\\varphi\_\{1\}\(b\_\{v\}\)\\leq F\_\{1\}\(1\+D\)^\{\\alpha\},\\qquad\\sum\_\{v\}\\eta\_\{v\}\\varphi\_\{2\}\(b\_\{v\}\)\\leq F\_\{2\}\.\(106\)For a composition, letav=τt−τv\+1a\_\{v\}=\\tau\_\{t\}\-\\tau\_\{v\+1\}andbv=τv−τu\+1b\_\{v\}=\\tau\_\{v\}\-\\tau\_\{u\+1\}\. Thenav\+bv\+ηv=Da\_\{v\}\+b\_\{v\}\+\\eta\_\{v\}=D, hence1\+max⁡\(av,bv\)≥\(1\+D\)/\(2​A\)1\+\\max\(a\_\{v\},b\_\{v\}\)\\geq\(1\+D\)/\(2A\)\. Whenav≥bva\_\{v\}\\geq b\_\{v\}, useφi​\(av\)≤\(2​A\)i−α​φi​\(D\)\\varphi\_\{i\}\(a\_\{v\}\)\\leq\(2A\)^\{i\-\\alpha\}\\varphi\_\{i\}\(D\)and apply \([106](https://arxiv.org/html/2610.41074#A3.E106)\) to the other factor; interchange the roles otherwise\. The resulting bounds are

∑u<v<tηv​φi​\(av\)​φj​\(bv\)≤Csc​\{φ2​\(D\),\(i,j\)=\(2,2\),φ1​\(D\),\(i,j\)=\(2,1\),\(1,2\),\(1\+D\)2​α−1,\(i,j\)=\(1,1\)\.\\sum\_\{u<v<t\}\\eta\_\{v\}\\varphi\_\{i\}\(a\_\{v\}\)\\varphi\_\{j\}\(b\_\{v\}\)\\leq C\_\{\\rm sc\}\\begin\{cases\}\\varphi\_\{2\}\(D\),&\(i,j\)=\(2,2\),\\\\ \\varphi\_\{1\}\(D\),&\(i,j\)=\(2,1\),\(1,2\),\\\\ \(1\+D\)^\{2\\alpha\-1\},&\(i,j\)=\(1,1\)\.\\end\{cases\}\(107\)The mixed cases use2​α−2≤α−12\\alpha\-2\\leq\\alpha\-1\. Substitute the common spectral profiles\. For the last case, use\(1\+D\)α≤2α​M\(1\+D\)^\{\\alpha\}\\leq 2^\{\\alpha\}MandcP​M≤S0c\_\{P\}M\\leq S\_\{0\}to obtain both alternatives in \([101](https://arxiv.org/html/2610.41074#A3.E101)\)\. The same cell integrals give the one\-kernel bounds and their full\-history versions\. Finallyhv=ρv​ηvh\_\{v\}=\\rho\_\{v\}\\eta\_\{v\}gives the weighted bounds, including zero\-rate steps\. ∎

### C\.3Exact\-arithmetic risk and source moments

We use standard second\-moment methods for linear SGD\([Wu et al\., 2022](https://arxiv.org/html/2610.41074#bib.bib23);[Lin et al\., 2024](https://arxiv.org/html/2610.41074#bib.bib14)\)\. In the eigenbasis ofGG, the diagonal entries of the exact\-arithmetic recursion in Proposition[B\.3](https://arxiv.org/html/2610.41074#Thmtheorem3)obey the recurrence of[Bordelon and Mori \(2026, Theorem 1\)](https://arxiv.org/html/2610.41074#bib.bib2)\. Theorem[C\.3](https://arxiv.org/html/2610.41074#Thmtheorem3a)expresses prediction risk through propagated contributions from the initial error, minibatch sampling and rounding\. It also bounds the exact\-arithmetic moments that enter the subsequent rounding\-error analysis\.

###### Theorem C\.3\(Risk identities and exact\-arithmetic source bounds\)\.

For \([8](https://arxiv.org/html/2610.41074#S2.E8)\), the following hold\.

1. 1\.*Risk identities\.*At a fixed full\-row\-rank sketch, every finite deterministic schedule satisfies E0out=12​tr⁡\(G​Pout​V​Pout𝖳\)\+∑u<Thu​ℛuout​\(Nu\),\\displaystyle E\_\{0\}^\{\\rm out\}=\\tfrac\{1\}\{2\}\\operatorname\{tr\}\(GP\_\{\\rm out\}VP\_\{\\rm out\}^\{\\mathsf\{T\}\}\)\+\\sum\_\{u<T\}h\_\{u\}\\mathcal\{R\}\_\{u\}^\{\\rm out\}\(N\_\{u\}\),\(108\)Δout=∑u<Thu​ℛuout​\(Qu\)≥0\.\\displaystyle\\Delta^\{\\rm out\}=\\sum\_\{u<T\}h\_\{u\}\\mathcal\{R\}\_\{u\}^\{\\rm out\}\(Q\_\{u\}\)\\geq 0\.\(109\)The first identity separates the squared conditional mean error from the variance caused by random SGD updates, with the teacher averaged afterward\.E0outE\_\{0\}^\{\\rm out\}is defined by \([38](https://arxiv.org/html/2610.41074#A2.E38)\)\.
2. 2\.*Risk comparison\.*IfcV​G−ωV⪯V⪯CV​G−ωVc\_\{V\}G^\{\-\\omega\_\{V\}\}\\preceq V\\preceq C\_\{V\}G^\{\-\\omega\_\{V\}\}, then cV​ℬout\+𝒱out≤E0out≤CV​ℬout\+2​𝒱out\.c\_\{V\}\\mathcal\{B\}^\{\\rm out\}\+\\mathcal\{V\}^\{\\rm out\}\\leq E\_\{0\}^\{\\rm out\}\\leq C\_\{V\}\\mathcal\{B\}^\{\\rm out\}\+2\\mathcal\{V\}^\{\\rm out\}\.\(110\)
3. 3\.*Source bounds\.*Under \([67](https://arxiv.org/html/2610.41074#A2.E67)\),0≤ηu​g1≤c0<10\\leq\\eta\_\{u\}g\_\{1\}\\leq c\_\{0\}<1andρu≤ρsrc\\rho\_\{u\}\\leq\\rho\_\{\\rm src\}, exact\-arithmetic moments satisfy s≤Ru≤Ksref​s,Ksref=CRraw​\(1\+σ−2\),\\displaystyle s\\leq R\_\{u\}\\leq K\_\{s\}^\{\\rm ref\}s,\\qquad K\_\{s\}^\{\\rm ref\}=C\_\{R\}^\{\\rm raw\}\(1\+\\sigma^\{\-2\}\),\(111\)Ut≤CUraw​\(1\+τt\)ν,\\displaystyle U\_\{t\}\\leq C\_\{U\}^\{\\rm raw\}\(1\+\\tau\_\{t\}\)^\{\\nu\},\(112\)Ut≥cU​τtν\(1≤τt≤Ma\),\\displaystyle U\_\{t\}\\geq c\_\{U\}\\tau\_\{t\}^\{\\nu\}\\qquad\(1\\leq\\tau\_\{t\}\\leq M^\{a\}\),\(113\)Uu≤KUUt\(0≤u≤t,τt≤Ma\)\.\\displaystyle U\_\{u\}\\leq K\_\{U\}U\_\{t\}\\qquad\(0\\leq u\\leq t,\\ \\tau\_\{t\}\\leq M^\{a\}\)\.\(114\)The constants are independent of width, time horizon, batch size and schedules\.

###### Proof\.

Leteu=vu−v∗e\_\{u\}=v\_\{u\}\-v\_\{\*\}andℱu\\mathscr\{F\}\_\{u\}contain the sketch, teacher and history before updateuu\. Proposition[B\.3](https://arxiv.org/html/2610.41074#Thmtheorem3)shows thatζu\+1=eu\+1−Hu​eu\\zeta\_\{u\+1\}=e\_\{u\+1\}\-H\_\{u\}e\_\{u\}and𝔼⁡\[ζu\+1∣ℱu\]=0\.\\mathbb\{E\}\[\\zeta\_\{u\+1\}\\mid\\mathscr\{F\}\_\{u\}\]=0\.These zero\-mean update fluctuations are orthogonal across times and tov∗v\_\{\*\}\. Then,vout−v∗=−Pout​v∗\+∑u<TLT,u​ζu\+1v^\{\\rm out\}\-v\_\{\*\}=\-P\_\{\\rm out\}v\_\{\*\}\+\\sum\_\{u<T\}L\_\{T,u\}\\zeta\_\{u\+1\}\. Their second moments arehu​Nuh\_\{u\}N\_\{u\}in exact arithmetic andhu​\(Nu\+Qu\)h\_\{u\}\(N\_\{u\}\+Q\_\{u\}\)with SR\. Orthogonality proves \([108](https://arxiv.org/html/2610.41074#A3.E108)\)–\([109](https://arxiv.org/html/2610.41074#A3.E109)\)\. The state recursion and \([50](https://arxiv.org/html/2610.41074#A2.E50)\) also indicates

Ct0=Pt​V​Pt\+∑u<thu​Pu\+1→t​Nu​Pu\+1→t,\\displaystyle C\_\{t\}^\{0\}=P\_\{t\}VP\_\{t\}\+\\sum\_\{u<t\}h\_\{u\}P\_\{u\+1\\to t\}N\_\{u\}P\_\{u\+1\\to t\},Wt0=\(I−Pt\)​V​\(I−Pt\)\+∑u<thu​Pu\+1→t​Nu​Pu\+1→t\.\\displaystyle W\_\{t\}^\{0\}=\(I\-P\_\{t\}\)V\(I\-P\_\{t\}\)\+\\sum\_\{u<t\}h\_\{u\}P\_\{u\+1\\to t\}N\_\{u\}P\_\{u\+1\\to t\}\.\(115\)ForC⪰0C\\succeq 0,G​C​G⪯tr⁡\(G​C\)​GGCG\\preceq\\operatorname\{tr\}\(GC\)G, so

Ru​G⪯Nu⪯2​Ru​G\.R\_\{u\}G\\preceq N\_\{u\}\\preceq 2R\_\{u\}G\.\(116\)The positive output response and teacher sandwich prove \([110](https://arxiv.org/html/2610.41074#A3.E110)\)\.

Sincetr⁡\(G​Pt​V​Pt\)≤tr⁡\(G​V\)≤Cbias\\operatorname\{tr\}\(GP\_\{t\}VP\_\{t\}\)\\leq\\operatorname\{tr\}\(GV\)\\leq C\_\{\\rm bias\}, \([115](https://arxiv.org/html/2610.41074#A3.E115)\) and \([116](https://arxiv.org/html/2610.41074#A3.E116)\) giveRt≤s\+Cbias\+2​∑u<thu​Ru​S2H​\(t,u\)R\_\{t\}\\leq s\+C\_\{\\rm bias\}\+2\\sum\_\{u<t\}h\_\{u\}R\_\{u\}S\_\{2\}^\{H\}\(t,u\)\. Taking the finite maximum and applying \([97](https://arxiv.org/html/2610.41074#A3.E97)\) withρsrc​θG≤1/4\\rho\_\{\\rm src\}\\theta\_\{G\}\\leq 1/4absorbs half that maximum:

s≤Rt≤2​\(s\+Cbias\)≤CRraw​\(1\+s\)\.s\\leq R\_\{t\}\\leq 2\(s\+C\_\{\\rm bias\}\)\\leq C\_\{R\}^\{\\rm raw\}\(1\+s\)\.\(117\)Nows≥σ2\>0s\\geq\\sigma^\{2\}\>0leads to \([111](https://arxiv.org/html/2610.41074#A3.E111)\)\.

LetUtdet,UtnoiseU\_\{t\}^\{\\rm det\},U\_\{t\}^\{\\rm noise\}be theJJ\-traces of the mean and fluctuation parts in \([115](https://arxiv.org/html/2610.41074#A3.E115)\)\. Then

Utdet≤CV​CJ​∑jgj−ωV​min⁡\(1,τt​gj\)2,Utnoise≤2​CJ​CRraw​\(1\+s\)​∑j,u<thu​gj​πj​\(t,u\+1\)2\.U\_\{t\}^\{\\rm det\}\\leq C\_\{V\}C\_\{J\}\\sum\_\{j\}g\_\{j\}^\{\-\\omega\_\{V\}\}\\min\(1,\\tau\_\{t\}g\_\{j\}\)^\{2\},\\qquad U\_\{t\}^\{\\rm noise\}\\leq 2C\_\{J\}C\_\{R\}^\{\\rm raw\}\(1\+s\)\\sum\_\{j,u<t\}h\_\{u\}g\_\{j\}\\pi\_\{j\}\(t,u\+1\)^\{2\}\.Telescoping bounds the inner time sum byρsrc​min⁡\(1,τt​gj\)\\rho\_\{\\rm src\}\\min\(1,\\tau\_\{t\}g\_\{j\}\)\. ForZ=K0​\(1\+τt\)Z=K\_\{0\}\(1\+\\tau\_\{t\}\), split the following sums atZαZ^\{\\alpha\}:

∑j≥1ja−b​min⁡\(1,Z​j−a\)2≤2​a\+2​b−1a\+b−1​Zν,∑j≥1min⁡\(1,Z​j−a\)≤2​a−1a−1​Zα\.\\displaystyle\\sum\_\{j\\geq 1\}j^\{a\-b\}\\min\(1,Zj^\{\-a\}\)^\{2\}\\leq\\frac\{2a\+2b\-1\}\{a\+b\-1\}Z^\{\\nu\},\\quad\\sum\_\{j\\geq 1\}\\min\(1,Zj^\{\-a\}\)\\leq\\frac\{2a\-1\}\{a\-1\}Z^\{\\alpha\}\.Use the integral plus maximum for the first sum and the integral for the second\. Withs≤σ2\+CA¯s\\leq\\sigma^\{2\}\+C\_\{\\bar\{A\}\}andα<ν\\alpha<\\nu, these give \([112](https://arxiv.org/html/2610.41074#A3.E112)\)\. For1≤τt≤Ma1\\leq\\tau\_\{t\}\\leq M^\{a\}, sum the mean contribution over indicesj≤n=⌊τtα⌋j\\leq n=\\lfloor\\tau\_\{t\}^\{\\alpha\}\\rfloor\. They satisfy1−πj​\(t,0\)≥\(1−e−1\)​min⁡\(1,cg\)1\-\\pi\_\{j\}\(t,0\)\\geq\(1\-e^\{\-1\}\)\\min\(1,c\_\{g\}\), and∑j≤nja−b≥2−\(a−b\+1\)​τtν/\(a−b\+1\)\\sum\_\{j\\leq n\}j^\{a\-b\}\\geq 2^\{\-\(a\-b\+1\)\}\\tau\_\{t\}^\{\\nu\}/\(a\-b\+1\)\. The lower teacher, spectral and eigenbasis bounds onJJimply \([113](https://arxiv.org/html/2610.41074#A3.E113)\)\.

For the early trajectory, set

R∗=CRraw​\(1\+σ2\+CA¯\),DV=Cg2−ωV​a\+ba\+b−1,u∗=cV​cJ​cg2−ωV​e−2​Cg,\\displaystyle R\_\{\*\}=C\_\{R\}^\{\\rm raw\}\(1\+\\sigma^\{2\}\+C\_\{\\bar\{A\}\}\),\\qquad D\_\{V\}=C\_\{g\}^\{2\-\\omega\_\{V\}\}\\frac\{a\+b\}\{a\+b\-1\},\\qquad u\_\{\*\}=c\_\{V\}c\_\{J\}c\_\{g\}^\{2\-\\omega\_\{V\}\}e^\{\-2C\_\{g\}\},Kpre=CJ​\(CV​DV\+2​R∗​Θ\)u∗,KU=max⁡\{1,2ν​CUraw/cU,Kpre\}\.\\displaystyle K\_\{\\rm pre\}=\\frac\{C\_\{J\}\(C\_\{V\}D\_\{V\}\+2R\_\{\*\}\\Theta\)\}\{u\_\{\*\}\},\\qquad K\_\{U\}=\\max\\\{1,2^\{\\nu\}C\_\{U\}^\{\\rm raw\}/c\_\{U\},K\_\{\\rm pre\}\\\}\.\(118\)For0<τt≤10<\\tau\_\{t\}\\leq 1,1−πj​\(u,0\)≤gj​τu1\-\\pi\_\{j\}\(u,0\)\\leq g\_\{j\}\\tau\_\{u\}and∑v<uhv≤τu2\\sum\_\{v<u\}h\_\{v\}\\leq\\tau\_\{u\}^\{2\}yieldUu≤CJ​\(CV​DV\+2​R∗​Θ\)​τt2U\_\{u\}\\leq C\_\{J\}\(C\_\{V\}D\_\{V\}\+2R\_\{\*\}\\Theta\)\\tau\_\{t\}^\{2\}foru≤tu\\leq t\. The contribution from the leading eigenvector givesUt≥u∗​τt2U\_\{t\}\\geq u\_\{\*\}\\tau\_\{t\}^\{2\}\. Together with \([112](https://arxiv.org/html/2610.41074#A3.E112)\)–\([113](https://arxiv.org/html/2610.41074#A3.E113)\) forτt≥1\\tau\_\{t\}\\geq 1andUu=Ut=0U\_\{u\}=U\_\{t\}=0atτt=0\\tau\_\{t\}=0, this proves \([114](https://arxiv.org/html/2610.41074#A3.E114)\)\.

Finally,s≤rt≤\(1\+σ−2\)​ss\\leq r\_\{t\}\\leq\(1\+\\sigma^\{\-2\}\)sforτt≥1\\tau\_\{t\}\\geq 1\. Forτ∗≤τT≤Ma\\tau\_\{\*\}\\leq\\tau\_\{T\}\\leq M^\{a\}, these bounds implycsrc​rt≤Rt≤Csrc​rtc\_\{\\rm src\}r\_\{t\}\\leq R\_\{t\}\\leq C\_\{\\rm src\}r\_\{t\}forτt≥τ∗,\\tau\_\{t\}\\geq\\tau\_\{\*\},and

csrc​rT≤Ru≤Csrc​\(1\+s\)\(0≤u≤T\),\\displaystyle c\_\{\\rm src\}r\_\{T\}\\leq R\_\{u\}\\leq C\_\{\\rm src\}\(1\+s\)\\quad\(0\\leq u\\leq T\),\(119\)Uu≤Csrc​Ut\(u≤t,τt≥τ∗\),Ut≤Csrc​\(1\+τt\)​Rt\(0≤t≤T\)\.\\displaystyle U\_\{u\}\\leq C\_\{\\rm src\}U\_\{t\}\\quad\(u\\leq t,\\ \\tau\_\{t\}\\geq\\tau\_\{\*\}\),\\quad U\_\{t\}\\leq C\_\{\\rm src\}\(1\+\\tau\_\{t\}\)R\_\{t\}\\quad\(0\\leq t\\leq T\)\.The constantscsrc,Csrcc\_\{\\rm src\},C\_\{\\rm src\}are defined in \([164](https://arxiv.org/html/2610.41074#A3.E164)\)\. ∎

### C\.4Bounds for rounding\-induced second moments

Rounding changes the residual and parameter second moments, which in turn determine the variance of later rounding errors\. The quantitiesxt\+x\_\{t\}^\{\+\}andyt\+y\_\{t\}^\{\+\}bound these increases, respectively\. Lemma[C\.4](https://arxiv.org/html/2610.41074#Thmtheorem4a)controls these increases using the exact\-arithmetic moments and an explicit interaction term\.

Usek2=S2𝒜k\_\{2\}=S\_\{2\}^\{\\mathcal\{A\}\},k1=CJ​S1𝒜k\_\{1\}=C\_\{J\}S\_\{1\}^\{\\mathcal\{A\}\},k0=CJ​S0𝒜k\_\{0\}=C\_\{J\}S\_\{0\}^\{\\mathcal\{A\}\}and the causal operators\(𝖫i​z\)t=∑u<thu​ki​\(t,u\)​zu\(\\mathsf\{L\}\_\{i\}z\)\_\{t\}=\\sum\_\{u<t\}h\_\{u\}k\_\{i\}\(t,u\)z\_\{u\}forz≥0z\\geq 0\. Operator products act from right to left\. In this subsection suppress the upper\-comparison superscript: writex=x\+x=x^\{\+\},y=y\+y=y^\{\+\}andf=f\+f=f^\{\+\},β=β\+\\beta=\\beta^\{\+\},m=m\+m=m^\{\+\}\. Thusm=f​β/3m=f\\beta/3, including when a coefficient is zero\. WithB0=β​R\+m​UB^\{0\}=\\beta R\+mU, define the source parts

Xf=𝖫2​\(f​U\),XB=𝖫1​B0,Yf=𝖫1​\(f​U\),Yb=𝖫0​\(β​R\),Ym=𝖫0​\(m​U\),\\displaystyle X\_\{f\}=\\mathsf\{L\}\_\{2\}\(fU\),\\quad X\_\{B\}=\\mathsf\{L\}\_\{1\}B^\{0\},\\quad Y\_\{f\}=\\mathsf\{L\}\_\{1\}\(fU\),\\quad Y\_\{b\}=\\mathsf\{L\}\_\{0\}\(\\beta R\),\\quad Y\_\{m\}=\\mathsf\{L\}\_\{0\}\(mU\),\(120\)Ψx=Xf\+XB,Ψy=Yf\+Yb\+Ym\.\\displaystyle\\Psi\_\{x\}=X\_\{f\}\+X\_\{B\},\\qquad\\Psi\_\{y\}=Y\_\{f\}\+Y\_\{b\}\+Y\_\{m\}\.\(121\)We use the bounds of Theorem[C\.3](https://arxiv.org/html/2610.41074#Thmtheorem3a)and the constantCKC\_\{K\}from Lemma[C\.2](https://arxiv.org/html/2610.41074#Thmtheorem2a)\. Consider

cR=min⁡\(csrc,1\),CUfb=max⁡\{Csrc,Csrc​\(1\+τ∗\)ν,1\},\\displaystyle c\_\{R\}=\\min\(c\_\{\\rm src\},1\),\\quad C\_\{U\}^\{\\rm fb\}=\\max\\\{C\_\{\\rm src\},C\_\{\\rm src\}\(1\+\\tau\_\{\*\}\)^\{\\nu\},1\\\},Cm​β=cm​Θ/\(cb​cR​σ2\),Cf​r=cf​Θ/σ2,Cy​A=CJ​cf​Θ​CUfb​CK,\\displaystyle C\_\{m\\beta\}=c\_\{m\}\\Theta/\(c\_\{b\}c\_\{R\}\\sigma^\{2\}\),\\quad C\_\{fr\}=c\_\{f\}\\Theta/\\sigma^\{2\},\\quad C\_\{yA\}=C\_\{J\}c\_\{f\}\\Theta C\_\{U\}^\{\\rm fb\}C\_\{K\},C8=\(CJ​cf​cb​Θ2\+Cy​A\+CJ​cm​Θ2​CUfb\)​\(1\+Cf​r\),\\displaystyle C\_\{8\}=\(C\_\{J\}c\_\{f\}c\_\{b\}\\Theta^\{2\}\+C\_\{yA\}\+C\_\{J\}c\_\{m\}\\Theta^\{2\}C\_\{U\}^\{\\rm fb\}\)\(1\+C\_\{fr\}\),C¯0=cf​max⁡\(Θ,cb/cm\),CΘ=2​CJ​cf​Θ​CK/cP\+Cy​A​max⁡\(1,Cm​β\),\\displaystyle\\bar\{C\}\_\{0\}=c\_\{f\}\\max\(\\Theta,c\_\{b\}/c\_\{m\}\),\\quad C\_\{\\Theta\}=2C\_\{J\}c\_\{f\}\\Theta C\_\{K\}/c\_\{P\}\+C\_\{yA\}\\max\(1,C\_\{m\\beta\}\),CΘ′=CJ​cm​Θ2/cP,C7=cb​Θ​CJ​CK/cP\.\\displaystyle C^\{\\prime\}\_\{\\Theta\}=C\_\{J\}c\_\{m\}\\Theta^\{2\}/c\_\{P\},\\qquad C\_\{7\}=c\_\{b\}\\Theta C\_\{J\}C\_\{K\}/c\_\{P\}\.\(122\)These coefficients depend only on the fixed model, comparison bounds and minimum cumulative learning rateτ∗\\tau\_\{\*\}, before the rate and feedback thresholds are chosen\.

Assume Assumption[C\.1](https://arxiv.org/html/2610.41074#Thmtheorem1b)\. Then all cumulative\-time gaps are at mostMaM^\{a\}, and Theorem[C\.3](https://arxiv.org/html/2610.41074#Thmtheorem3a)givesRu≥cR​rTR\_\{u\}\\geq c\_\{R\}r\_\{T\}throughout the history, withcR=min⁡\(csrc,1\)c\_\{R\}=\\min\(c\_\{\\rm src\},1\)as in \([122](https://arxiv.org/html/2610.41074#A3.E122)\)\. Here sequence products, such asf​UfUorβ​R\\beta R, are evaluated pointwise in time\. The kernel comparison \([87](https://arxiv.org/html/2610.41074#A3.E87)\) gives

𝖣=𝖫2\+𝖫1​β,𝖤=𝖫2​f\+𝖫1​m,𝖥=𝖫1\+𝖫0​β,𝖧=𝖫1​f\+𝖫0​m,\\displaystyle\\mathsf\{D\}=\\mathsf\{L\}\_\{2\}\+\\mathsf\{L\}\_\{1\}\\beta,\\quad\\mathsf\{E\}=\\mathsf\{L\}\_\{2\}f\+\\mathsf\{L\}\_\{1\}m,\\quad\\mathsf\{F\}=\\mathsf\{L\}\_\{1\}\+\\mathsf\{L\}\_\{0\}\\beta,\\quad\\mathsf\{H\}=\\mathsf\{L\}\_\{1\}f\+\\mathsf\{L\}\_\{0\}m,\(123\)Ξ=𝖤​Ψy,\\displaystyle\\Xi=\\mathsf\{E\}\\Psi\_\{y\},\(124\)x\+≤Ψx\+𝖣​x\+\+𝖤​y\+,y\+≤Ψy\+𝖥​x\+\+𝖧​y\+\.\\displaystyle x^\{\+\}\\leq\\Psi\_\{x\}\+\\mathsf\{D\}x^\{\+\}\+\\mathsf\{E\}y^\{\+\},\\qquad y^\{\+\}\\leq\\Psi\_\{y\}\+\\mathsf\{F\}x^\{\+\}\+\\mathsf\{H\}y^\{\+\}\.\(125\)
###### Lemma C\.4\(Bounds for rounding\-induced second moments\)\.

On the common event, every admissible deterministic schedule satisfies, simultaneously at every0≤t≤T0\\leq t\\leq T,

yt\+≤4​Ψy​\(t\),xt\+≤4​Ψx​\(t\)\+8​Ξ​\(t\)\.y\_\{t\}^\{\+\}\\leq 4\\Psi\_\{y\}\(t\),\\qquad x\_\{t\}^\{\+\}\\leq 4\\Psi\_\{x\}\(t\)\+8\\Xi\(t\)\.\(126\)

###### Proof\.

Let us considerwt=max⁡\(Ut,1\)w\_\{t\}=\\max\(U\_\{t\},1\)and define

A×=CJ​cf​cb​Θ2,Am=CJ​cm​Θ2​CUfb,δ=max⁡\(ρsch,Γstab,Γ×\),\\displaystyle A\_\{\\times\}=C\_\{J\}c\_\{f\}c\_\{b\}\\Theta^\{2\},\\quad A\_\{m\}=C\_\{J\}c\_\{m\}\\Theta^\{2\}C\_\{U\}^\{\\rm fb\},\\quad\\delta=\\max\(\\rho\_\{\\rm sch\},\\Gamma\_\{\\rm stab\},\\Gamma\_\{\\times\}\),λ=C8​δ,γ=C3​C¯0​Γf→b,\\displaystyle\\lambda=C\_\{8\}\\delta,\\qquad\\gamma=C\_\{3\}\\bar\{C\}\_\{0\}\\Gamma\_\{f\\to b\},\(127\)ϵ0=CK​ρsch,ϵ=CJ​CK​ρsch,ζ=C7​ρsch,e=ϵ0/cR,ϵH=CΘ​ρsch\+CΘ′​Γstab\.\\displaystyle\\epsilon\_\{0\}=C\_\{K\}\\rho\_\{\\rm sch\},\\quad\\epsilon=C\_\{J\}C\_\{K\}\\rho\_\{\\rm sch\},\\quad\\zeta=C\_\{7\}\\rho\_\{\\rm sch\},\\quad e=\\epsilon\_\{0\}/c\_\{R\},\\quad\\epsilon\_\{H\}=C\_\{\\Theta\}\\rho\_\{\\rm sch\}\+C^\{\\prime\}\_\{\\Theta\}\\Gamma\_\{\\rm stab\}\.\(128\)The fixed admissibility thresholds are chosen so that

λ≤12,γ≤1128,ϵ0≤ϵ≤164,ζ≤164,e≤1128,ϵH≤18\.\\displaystyle\\lambda\\leq\\tfrac\{1\}\{2\},\\quad\\gamma\\leq\\tfrac\{1\}\{128\},\\quad\\epsilon\_\{0\}\\leq\\epsilon\\leq\\tfrac\{1\}\{64\},\\quad\\zeta\\leq\\tfrac\{1\}\{64\},\\quad e\\leq\\tfrac\{1\}\{128\},\\quad\\epsilon\_\{H\}\\leq\\tfrac\{1\}\{8\}\.\(129\)The exact\-arithmetic moment bounds, kernel masses in Lemma[C\.2](https://arxiv.org/html/2610.41074#Thmtheorem2a)and the definitions of the feedback gains give

Uu≤CUfb​wt​\(u≤t\),Yf​\(t\)≤Cy​A​ρsch​wt,Ym​\(t\)≤Am​Γstab​wt,\\displaystyle U\_\{u\}\\leq C\_\{U\}^\{\\rm fb\}w\_\{t\}\\ \(u\\leq t\),\\qquad Y\_\{f\}\(t\)\\leq C\_\{yA\}\\rho\_\{\\rm sch\}w\_\{t\},\\qquad Y\_\{m\}\(t\)\\leq A\_\{m\}\\Gamma\_\{\\rm stab\}w\_\{t\},\(130\)ft​Yb​\(t\)≤A×​Γ×​rT\.\\displaystyle f\_\{t\}Y\_\{b\}\(t\)\\leq A\_\{\\times\}\\Gamma\_\{\\times\}r\_\{T\}\.\(131\)Usingft​wt≤ft​Ut\+Cf​r​rTf\_\{t\}w\_\{t\}\\leq f\_\{t\}U\_\{t\}\+C\_\{fr\}r\_\{T\}and the definition ofC8C\_\{8\}gives the first bound below\. For the second, the spectral comparison and forward\-to\-backward gain giveβt​Xf​\(t\)≤C3​Γf→b​\[cf​θG​βt​Rt\+\(cf​cb/cm\)​mt​Ut\]≤γ​Bt0\\beta\_\{t\}X\_\{f\}\(t\)\\leq C\_\{3\}\\Gamma\_\{f\\to b\}\[c\_\{f\}\\theta\_\{G\}\\beta\_\{t\}R\_\{t\}\+\(c\_\{f\}c\_\{b\}/c\_\{m\}\)m\_\{t\}U\_\{t\}\]\\leq\\gamma B\_\{t\}^\{0\}; atβt=0\\beta\_\{t\}=0it follows from nonnegativity\. Thus

ft​Ψy​\(t\)≤λ⁡\(ft​Ut\+rT\),βt​Xf​\(t\)≤γ​Bt0\.f\_\{t\}\\Psi\_\{y\}\(t\)\\leq\\lambda\(f\_\{t\}U\_\{t\}\+r\_\{T\}\),\\qquad\\beta\_\{t\}X\_\{f\}\(t\)\\leq\\gamma B\_\{t\}^\{0\}\.\(132\)Reordering the causal sums and applying Lemma[C\.2](https://arxiv.org/html/2610.41074#Thmtheorem2a)gives, for everyz≥0z\\geq 0,

𝖫2​𝖫2​z≤ϵ0​𝖫2​z,𝖫2​𝖫1​z≤ϵ0​𝖫1​z,𝖫1​𝖫2​z≤ϵ0​𝖫1​z,𝖫1​𝖫1​z≤ϵ​𝖫0​z,\\displaystyle\\mathsf\{L\}\_\{2\}\\mathsf\{L\}\_\{2\}z\\leq\\epsilon\_\{0\}\\mathsf\{L\}\_\{2\}z,\\quad\\mathsf\{L\}\_\{2\}\\mathsf\{L\}\_\{1\}z\\leq\\epsilon\_\{0\}\\mathsf\{L\}\_\{1\}z,\\quad\\mathsf\{L\}\_\{1\}\\mathsf\{L\}\_\{2\}z\\leq\\epsilon\_\{0\}\\mathsf\{L\}\_\{1\}z,\\quad\\mathsf\{L\}\_\{1\}\\mathsf\{L\}\_\{1\}z\\leq\\epsilon\\mathsf\{L\}\_\{0\}z,\(133\)𝖫1​\(β​𝖫1​z\)≤ζ​𝖫1​z,𝖫0​\(β​𝖫1​z\)≤ζ​𝖫0​z\.\\displaystyle\\mathsf\{L\}\_\{1\}\(\\beta\\mathsf\{L\}\_\{1\}z\)\\leq\\zeta\\mathsf\{L\}\_\{1\}z,\\qquad\\mathsf\{L\}\_\{0\}\(\\beta\\mathsf\{L\}\_\{1\}z\)\\leq\\zeta\\mathsf\{L\}\_\{0\}z\.\(134\)Use \([100](https://arxiv.org/html/2610.41074#A3.E100)\) and \([101](https://arxiv.org/html/2610.41074#A3.E101)\) for the first line\. For the second, useβv≤cb​Θ/M\\beta\_\{v\}\\leq c\_\{b\}\\Theta/M, theM​S1MS\_\{1\}composition bound and theS1S\_\{1\}mass bound;S0≥cP​MS\_\{0\}\\geq c\_\{P\}Mgives the𝖫0\\mathsf\{L\}\_\{0\}comparison\. The same bounds yield𝖫1​\(f​Ψy\)≤\(CJ​cf​Θ​CK/cP\)​ρsch​Ψy\\mathsf\{L\}\_\{1\}\(f\\Psi\_\{y\}\)\\leq\(C\_\{J\}c\_\{f\}\\Theta C\_\{K\}/c\_\{P\}\)\\rho\_\{\\rm sch\}\\Psi\_\{y\}and𝖫0​\(m⁡\(Yb\+Ym\)\)≤CΘ′​Γstab​\(Yb\+Ym\)\\mathsf\{L\}\_\{0\}\(m\(Y\_\{b\}\+Y\_\{m\}\)\)\\leq C^\{\\prime\}\_\{\\Theta\}\\Gamma\_\{\\rm stab\}\(Y\_\{b\}\+Y\_\{m\}\)\. Moreovermt≤Cm​β​βt​Rtm\_\{t\}\\leq C\_\{m\\beta\}\\beta\_\{t\}R\_\{t\}givesmt​Yf​\(t\)≤Cy​A​ρsch​max⁡\(1,Cm​β\)​\(mt​Ut\+βt​Rt\)m\_\{t\}Y\_\{f\}\(t\)\\leq C\_\{yA\}\\rho\_\{\\rm sch\}\\max\(1,C\_\{m\\beta\}\)\(m\_\{t\}U\_\{t\}\+\\beta\_\{t\}R\_\{t\}\)\. Applying𝖫0\\mathsf\{L\}\_\{0\}proves

𝖧​Ψy≤ϵH​Ψy\.\\mathsf\{H\}\\Psi\_\{y\}\\leq\\epsilon\_\{H\}\\Psi\_\{y\}\.\(135\)Apply these compositions and \([132](https://arxiv.org/html/2610.41074#A3.E132)\) to the source profiles to obtain

𝖣​Ψx≤\(ϵ\+ζ\+γ\)​Ψx,𝖥​Ψx≤\(ϵ\+ζ\+γ\)​Ψy\.\\displaystyle\\mathsf\{D\}\\Psi\_\{x\}\\leq\(\\epsilon\+\\zeta\+\\gamma\)\\Psi\_\{x\},\\qquad\\mathsf\{F\}\\Psi\_\{x\}\\leq\(\\epsilon\+\\zeta\+\\gamma\)\\Psi\_\{y\}\.\(136\)LetΞ=Ef\+Em\\Xi=E\_\{f\}\+E\_\{m\}withEf=𝖫2​\(f​Ψy\)E\_\{f\}=\\mathsf\{L\}\_\{2\}\(f\\Psi\_\{y\}\)andEm=𝖫1​\(m​Ψy\)E\_\{m\}=\\mathsf\{L\}\_\{1\}\(m\\Psi\_\{y\}\)\. The local source bound andS2S\_\{2\}giveEf≤λ​Xf\+λ​ϵ0​rTE\_\{f\}\\leq\\lambda X\_\{f\}\+\\lambda\\epsilon\_\{0\}r\_\{T\}andβ​Ef≤λ⁡\(γ\+e\)​B0\\beta E\_\{f\}\\leq\\lambda\(\\gamma\+e\)B^\{0\}, sinceRt≥cR​rTR\_\{t\}\\geq c\_\{R\}r\_\{T\}\. With \([133](https://arxiv.org/html/2610.41074#A3.E133)\)–\([135](https://arxiv.org/html/2610.41074#A3.E135)\), these imply

𝖣​Ξ≤\(ϵ\+ζ\)​Ξ\+λ⁡\(γ\+e\)​XB,𝖥​Ξ≤\[\(ϵ\+ζ\)​ϵH\+λ⁡\(γ\+e\)\]​Ψy\.\\displaystyle\\mathsf\{D\}\\Xi\\leq\(\\epsilon\+\\zeta\)\\Xi\+\\lambda\(\\gamma\+e\)X\_\{B\},\\qquad\\mathsf\{F\}\\Xi\\leq\[\(\\epsilon\+\\zeta\)\\epsilon\_\{H\}\+\\lambda\(\\gamma\+e\)\]\\Psi\_\{y\}\.\(137\)All profiles vanish initially and depend only on earlier indices\. Substituting the induction bounds into \([125](https://arxiv.org/html/2610.41074#A3.E125)\) and using \([129](https://arxiv.org/html/2610.41074#A3.E129)\) givesxt\+≤3932​Ψx​\(t\)\+174​Ξ​\(t\)≤4​Ψx​\(t\)\+8​Ξ​\(t\)x\_\{t\}^\{\+\}\\leq\\tfrac\{39\}\{32\}\\Psi\_\{x\}\(t\)\+\\tfrac\{17\}\{4\}\\Xi\(t\)\\leq 4\\Psi\_\{x\}\(t\)\+8\\Xi\(t\)andyt\+≤74​Ψy​\(t\)≤4​Ψy​\(t\)\.y\_\{t\}^\{\+\}\\leq\\tfrac\{7\}\{4\}\\Psi\_\{y\}\(t\)\\leq 4\\Psi\_\{y\}\(t\)\.∎

### C\.5From state moments to output prediction risk

Squared output filters pair two state indices\. Their total cumulative\-time gap can therefore reach2​Ma2M^\{a\}, even thoughτT≤Ma\\tau\_\{T\}\\leq M^\{a\}\. Lemma[C\.5](https://arxiv.org/html/2610.41074#Thmtheorem5a)propagates the state estimates through these filters, preparing the output\-risk bound in Theorem[C\.6](https://arxiv.org/html/2610.41074#Thmtheorem6a)\.

###### Lemma C\.5\(Finite output\-filter compositions\)\.

Suppose the sketch satisfies \([67](https://arxiv.org/html/2610.41074#A2.E67)\), and the schedule satisfies0≤ηt​g1≤c0<10\\leq\\eta\_\{t\}g\_\{1\}\\leq c\_\{0\}<1,τT≤Ma\\tau\_\{T\}\\leq M^\{a\}andρt≤ρsch\\rho\_\{t\}\\leq\\rho\_\{\\rm sch\}, with the notation of \([7](https://arxiv.org/html/2610.41074#S2.E7)\)\. There is a constantKR\>0K\_\{R\}\>0, depending only on the fixed model and comparison bounds andc0c\_\{0\}, such that for either output in \([39](https://arxiv.org/html/2610.41074#A2.E39)\) and everyu<Tu<T, the kernels in \([78](https://arxiv.org/html/2610.41074#A3.E78)\) and responses in \([73](https://arxiv.org/html/2610.41074#A3.E73)\) satisfy

∑u<t<Tht​S2H​\(t,u\)​ℛtout​\(G\)≤KR​ρsch​ℛuout​\(G\),\\displaystyle\\sum\_\{u<t<T\}h\_\{t\}S\_\{2\}^\{H\}\(t,u\)\\mathcal\{R\}\_\{t\}^\{\\rm out\}\(G\)\\leq K\_\{R\}\\rho\_\{\\rm sch\}\\mathcal\{R\}\_\{u\}^\{\\rm out\}\(G\),\(138\)∑u<t<Tht​S1H​\(t,u\)​ℛtout​\(G\)≤KR​ρsch​ℛuout​\(J\),\\displaystyle\\sum\_\{u<t<T\}h\_\{t\}S\_\{1\}^\{H\}\(t,u\)\\mathcal\{R\}\_\{t\}^\{\\rm out\}\(G\)\\leq K\_\{R\}\\rho\_\{\\rm sch\}\\mathcal\{R\}\_\{u\}^\{\\rm out\}\(J\),\(139\)M−1​∑u<t<Tht​S1H​\(t,u\)​ℛtout​\(J\)≤KR​ρsch​ℛuout​\(J\)\.\\displaystyle M^\{\-1\}\\sum\_\{u<t<T\}h\_\{t\}S\_\{1\}^\{H\}\(t,u\)\\mathcal\{R\}\_\{t\}^\{\\rm out\}\(J\)\\leq K\_\{R\}\\rho\_\{\\rm sch\}\\mathcal\{R\}\_\{u\}^\{\\rm out\}\(J\)\.\(140\)The same constant applies to all such schedules and both outputs\.

###### Proof\.

Use the paired bounds of Lemma[C\.2](https://arxiv.org/html/2610.41074#Thmtheorem2a), keeping its exactKSK\_\{S\}and2​Ma2M^\{a\}window\. Foru<t<n=min⁡\(v,w\)u<t<n=\\min\(v,w\)putbt=τt−τu\+1b\_\{t\}=\\tau\_\{t\}\-\\tau\_\{u\+1\}andat=τv\+τw−2​τt\+1a\_\{t\}=\\tau\_\{v\}\+\\tau\_\{w\}\-2\\tau\_\{t\+1\}, andd=τv\+τw−2​τu\+1d=\\tau\_\{v\}\+\\tau\_\{w\}\-2\\tau\_\{u\+1\}\. Then

at\+2​bt\+2​ηt=d\.a\_\{t\}\+2b\_\{t\}\+2\\eta\_\{t\}=d\.\(141\)The cells\[bt,bt\+ηt\]\[b\_\{t\},b\_\{t\}\+\\eta\_\{t\}\]tile\[0,τn−τu\+1\]\[0,\\tau\_\{n\}\-\\tau\_\{u\+1\}\];\[at,at\+2​ηt\]\[a\_\{t\},a\_\{t\}\+2\\eta\_\{t\}\]tile a subset of\[0,d\]\[0,d\]in reverse\. Thebtb\_\{t\}sums obey \([106](https://arxiv.org/html/2610.41074#A3.E106)\)\. Integrating on the reversed cells of length2​ηt2\\eta\_\{t\}gives∑ηt​φ1​\(at\)≤F1pair​\(1\+d\)α\\sum\\eta\_\{t\}\\varphi\_\{1\}\(a\_\{t\}\)\\leq F\_\{1\}^\{\\rm pair\}\(1\+d\)^\{\\alpha\}and∑ηt​φ2​\(at\)≤F2pair\\sum\\eta\_\{t\}\\varphi\_\{2\}\(a\_\{t\}\)\\leq F\_\{2\}^\{\\rm pair\}, whereA2=1\+2​η¯A\_\{2\}=1\+2\\bar\{\\eta\},Hpair=3​\(1\+η¯\)H\_\{\\rm pair\}=3\(1\+\\bar\{\\eta\}\)and

F1pair=A2/\(2​α\),F2pair=A22−α/\[2​\(1−α\)\],\\displaystyle F\_\{1\}^\{\\rm pair\}=A\_\{2\}/\(2\\alpha\),\\qquad F\_\{2\}^\{\\rm pair\}=A\_\{2\}^\{2\-\\alpha\}/\[2\(1\-\\alpha\)\],Fmax=max⁡\(F1,F2,F1pair,F2pair\),Cpair=2​Hpair2​Fmax\.\\displaystyle F\_\{\\max\}=\\max\(F\_\{1\},F\_\{2\},F\_\{1\}^\{\\rm pair\},F\_\{2\}^\{\\rm pair\}\),\\qquad C\_\{\\rm pair\}=2H\_\{\\rm pair\}^\{2\}F\_\{\\max\}\.\(142\)By \([141](https://arxiv.org/html/2610.41074#A3.E141)\),1\+d≤1\+3​max⁡\(at,bt\)\+2​η¯≤Hpair​\[1\+max⁡\(at,bt\)\]1\+d\\leq 1\+3\\max\(a\_\{t\},b\_\{t\}\)\+2\\bar\{\\eta\}\\leq H\_\{\\rm pair\}\[1\+\\max\(a\_\{t\},b\_\{t\}\)\]\. Split atat≥bta\_\{t\}\\geq b\_\{t\}\. Summing the smaller\-argument profile yields coefficientsHpair2−α​\(F2\+F2pair\)H\_\{\\rm pair\}^\{2\-\\alpha\}\(F\_\{2\}\+F\_\{2\}^\{\\rm pair\}\),Hpair2−α​F1\+Hpair1−α​F2pairH\_\{\\rm pair\}^\{2\-\\alpha\}F\_\{1\}\+H\_\{\\rm pair\}^\{1\-\\alpha\}F\_\{2\}^\{\\rm pair\}, andHpair1−α​\(F1\+F1pair\)H\_\{\\rm pair\}^\{1\-\\alpha\}\(F\_\{1\}\+F\_\{1\}^\{\\rm pair\}\), respectively, all bounded byCpairC\_\{\\rm pair\}; here2​α−2≤α−12\\alpha\-2\\leq\\alpha\-1\. Since\(1\+d\)α/M≤3α\(1\+d\)^\{\\alpha\}/M\\leq 3^\{\\alpha\}on the pair window,

∑ηt​φ2​\(bt\)​φ2​\(at\)≤Cpair​φ2​\(d\),∑ηt​φ1​\(bt\)​φ2​\(at\)≤Cpair​φ1​\(d\),\\displaystyle\\sum\\eta\_\{t\}\\varphi\_\{2\}\(b\_\{t\}\)\\varphi\_\{2\}\(a\_\{t\}\)\\leq C\_\{\\rm pair\}\\varphi\_\{2\}\(d\),\\quad\\sum\\eta\_\{t\}\\varphi\_\{1\}\(b\_\{t\}\)\\varphi\_\{2\}\(a\_\{t\}\)\\leq C\_\{\\rm pair\}\\varphi\_\{1\}\(d\),M−1​∑ηt​φ1​\(bt\)​φ1​\(at\)≤3α​Cpair​φ1​\(d\)\.\\displaystyle M^\{\-1\}\\sum\\eta\_\{t\}\\varphi\_\{1\}\(b\_\{t\}\)\\varphi\_\{1\}\(a\_\{t\}\)\\leq 3^\{\\alpha\}C\_\{\\rm pair\}\\varphi\_\{1\}\(d\)\.\(143\)
SubstituteSiH≤CP​φiS\_\{i\}^\{H\}\\leq C\_\{P\}\\varphi\_\{i\}, the paired bounds atat,da\_\{t\},d, andht≤ρsch​ηth\_\{t\}\\leq\\rho\_\{\\rm sch\}\\eta\_\{t\}:

∑u<t<min⁡\(v,w\)ht​SiH​\(t,u\)​Λjpair​\(t,v,w\)≤Kpair​ρsch​Λmpair​\(u,v,w\),\\sum\_\{u<t<\\min\(v,w\)\}h\_\{t\}S\_\{i\}^\{H\}\(t,u\)\\Lambda\_\{j\}^\{\\rm pair\}\(t;v,w\)\\leq K\_\{\\rm pair\}\\rho\_\{\\rm sch\}\\Lambda\_\{m\}^\{\\rm pair\}\(u;v,w\),for\(i,j,m\)=\(2,2,2\),\(1,2,1\),\(1,1,1\)\(i,j,m\)=\(2,2,2\),\(1,2,1\),\(1,1,1\), with an extraM−1M^\{\-1\}on the last left side, where

Kpair=3α​CP​KS2​Cpair,KR=Kpair​max⁡\(1,1/cJ,CJ/cJ\)\.K\_\{\\rm pair\}=3^\{\\alpha\}C\_\{P\}K\_\{S\}^\{2\}C\_\{\\rm pair\},\\qquad K\_\{R\}=K\_\{\\rm pair\}\\max\(1,1/c\_\{J\},C\_\{J\}/c\_\{J\}\)\.\(144\)DefineΛJpair=∑jgj​Jj​j​πj​\(v,u\+1\)​πj​\(w,u\+1\)\\Lambda\_\{J\}^\{\\rm pair\}=\\sum\_\{j\}g\_\{j\}J\_\{jj\}\\pi\_\{j\}\(v,u\+1\)\\pi\_\{j\}\(w,u\+1\)\. The bounds in the eigenbasis ofGGgivecJ​Λ1pair≤ΛJpair≤CJ​Λ1pairc\_\{J\}\\Lambda\_\{1\}^\{\\rm pair\}\\leq\\Lambda\_\{J\}^\{\\rm pair\}\\leq C\_\{J\}\\Lambda\_\{1\}^\{\\rm pair\}, and

ℛuout​\(G\)=12​∑v,w\>uwvout​wwout​Λ2pair,ℛuout​\(J\)=12​∑v,w\>uwvout​wwout​ΛJpair\.\\mathcal\{R\}\_\{u\}^\{\\rm out\}\(G\)=\\tfrac\{1\}\{2\}\\sum\_\{v,w\>u\}w\_\{v\}^\{\\rm out\}w\_\{w\}^\{\\rm out\}\\Lambda\_\{2\}^\{\\rm pair\},\\qquad\\mathcal\{R\}\_\{u\}^\{\\rm out\}\(J\)=\\tfrac\{1\}\{2\}\\sum\_\{v,w\>u\}w\_\{v\}^\{\\rm out\}w\_\{w\}^\{\\rm out\}\\Lambda\_\{J\}^\{\\rm pair\}\.Multiply bywvout​wwout/2w\_\{v\}^\{\\rm out\}w\_\{w\}^\{\\rm out\}/2and reorder\. The factors1,1/cJ,CJ/cJ1,1/c\_\{J\},C\_\{J\}/c\_\{J\}giveKRK\_\{R\}, also for empty sums and zero cells\. ∎

##### The SR\-induced risk and its remainder\.

The direct and temporal\-interaction contributions are defined in \([80](https://arxiv.org/html/2610.41074#A3.E80)\); their sum is𝒩out\\mathcal\{N\}^\{\\rm out\}in \([81](https://arxiv.org/html/2610.41074#A3.E81)\)\. The risk increaseΔout\\Delta^\{\\rm out\}is defined in \([10](https://arxiv.org/html/2610.41074#S2.E10)\)\. For the comparison tracesxt\+,yt\+x\_\{t\}^\{\+\},y\_\{t\}^\{\+\}of Appendix[C\.1](https://arxiv.org/html/2610.41074#A3.SS1), define

Remout=∑t<Tht​\[\(2​xt\+\+ft​yt\+\)​ℛtout​\(G\)\+\(βt​xt\+\+mt​yt\+\)​ℛtout​\(J\)\]\.\\operatorname\{Rem\}^\{\\rm out\}=\\sum\_\{t<T\}h\_\{t\}\[\(2x\_\{t\}^\{\+\}\+f\_\{t\}y\_\{t\}^\{\+\}\)\\mathcal\{R\}\_\{t\}^\{\\rm out\}\(G\)\+\(\\beta\_\{t\}x\_\{t\}^\{\+\}\+m\_\{t\}y\_\{t\}^\{\+\}\)\\mathcal\{R\}\_\{t\}^\{\\rm out\}\(J\)\]\.\(145\)Here and below,f,β,mf,\\beta,mare the upper coefficients in \([46](https://arxiv.org/html/2610.41074#A2.E46)\)\.

The following theorem bounds prediction risk using the state\-moment estimates\. Under admissibility, repeated rounding interactions contribute at most a fixed multiple of the leading terms\. Proposition[C\.7](https://arxiv.org/html/2610.41074#Thmtheorem7a)then supplies the matching lower bound for the full rounding expression\.

###### Theorem C\.6\(Bounds for the SR\-induced output risk\)\.

OnΩM\\Omega\_\{M\}, under Assumption[C\.1](https://arxiv.org/html/2610.41074#Thmtheorem1b), there is a fixed constantC13\>0C\_\{13\}\>0such that, for either output in \([39](https://arxiv.org/html/2610.41074#A2.E39)\),

Ψ−out\\displaystyle\\Psi\_\{\-\}^\{\\rm out\}≤Δout≤\(1\+C13\)​𝒩out,\\displaystyle\\leq\\Delta^\{\\rm out\}\\leq\(1\+C\_\{13\}\)\\mathcal\{N\}^\{\\rm out\},\(146\)Remout\\displaystyle\\operatorname\{Rem\}^\{\\rm out\}≤C13​\[Ψ\+out\+Convb→fout\]\.\\displaystyle\\leq C\_\{13\}\[\\Psi\_\{\+\}^\{\\rm out\}\+\\operatorname\{Conv\}\_\{b\\to f\}^\{\\rm out\}\]\.\(147\)The bound onΓf→b\\Gamma\_\{f\\to b\}in \([85](https://arxiv.org/html/2610.41074#A3.E85)\), defined in \([83](https://arxiv.org/html/2610.41074#A3.E83)\), controls the forward\-to\-backward interaction by a fixed multiple of the direct contributionΨ\+out\\Psi\_\{\+\}^\{\\rm out\}\.

###### Proof\.

Use the source profiles in \([120](https://arxiv.org/html/2610.41074#A3.E120)\)–\([121](https://arxiv.org/html/2610.41074#A3.E121)\) and the causal operators defined immediately before them, withΞ\\Xifrom \([124](https://arxiv.org/html/2610.41074#A3.E124)\)\. WriteZtb=Ztb,\+Z\_\{t\}^\{b\}=Z\_\{t\}^\{b,\+\}andYtf,H=Ytf,\+Y\_\{t\}^\{f,H\}=Y\_\{t\}^\{f,\+\}from \([79](https://arxiv.org/html/2610.41074#A3.E79)\)\. For this proof set

EU=KU​\[CJ​cf​Θ​CK​ρsch\+CJ​cm​Θ2​κ\],Kb=CJ​C3/cJ2,\\displaystyle E\_\{U\}=K\_\{U\}\[C\_\{J\}c\_\{f\}\\Theta C\_\{K\}\\rho\_\{\\rm sch\}\+C\_\{J\}c\_\{m\}\\Theta^\{2\}\\kappa\],\\quad K\_\{b\}=C\_\{J\}C\_\{3\}/c\_\{J\}^\{2\},χb=A×​κ3​cR,χf=A×​κ​CK​ρschcR,Dout=C3​KR​ρsch,\\displaystyle\\chi\_\{b\}=\\frac\{A\_\{\\times\}\\kappa\}\{3c\_\{R\}\},\\quad\\chi\_\{f\}=\\frac\{A\_\{\\times\}\\kappa C\_\{K\}\\rho\_\{\\rm sch\}\}\{c\_\{R\}\},\\quad D\_\{\\rm out\}=C\_\{3\}K\_\{R\}\\rho\_\{\\rm sch\},\(148\)whereA×A\_\{\\times\}is defined in \([127](https://arxiv.org/html/2610.41074#A3.E127)\)\. Abbreviate⟨z,Q⟩out=∑t<Tht​zt​ℛtout​\(Q\)\\langle z,Q\\rangle\_\{\\rm out\}=\\sum\_\{t<T\}h\_\{t\}z\_\{t\}\\mathcal\{R\}\_\{t\}^\{\\rm out\}\(Q\)and

Pf=⟨f​U,G⟩out,Pb=⟨β​R,J⟩out,Pm=⟨m​U,J⟩out,P=Pf\+Pb\+Pm=Ψ\+out\.P\_\{f\}=\\langle fU,G\\rangle\_\{\\rm out\},\\quad P\_\{b\}=\\langle\\beta R,J\\rangle\_\{\\rm out\},\\quad P\_\{m\}=\\langle mU,J\\rangle\_\{\\rm out\},\\quad P=P\_\{f\}\+P\_\{b\}\+P\_\{m\}=\\Psi\_\{\+\}^\{\\rm out\}\.The parameter comparison \([114](https://arxiv.org/html/2610.41074#A3.E114)\) boundsYf\+YmY\_\{f\}\+Y\_\{m\}byEU​UE\_\{U\}U\. The kernel comparison in Lemma[C\.2](https://arxiv.org/html/2610.41074#Thmtheorem2a)givesCJ​S0𝒜≤\(CJ/cJ2\)​KJ​J𝒜≤Kb​KJ​JHC\_\{J\}S\_\{0\}^\{\\mathcal\{A\}\}\\leq\(C\_\{J\}/c\_\{J\}^\{2\}\)K\_\{JJ\}^\{\\mathcal\{A\}\}\\leq K\_\{b\}K\_\{JJ\}^\{H\}\. Together with \([131](https://arxiv.org/html/2610.41074#A3.E131)\),m=f​β/3m=f\\beta/3andRt≥cR​rTR\_\{t\}\\geq c\_\{R\}r\_\{T\}, these yield

Ψy​\(t\)≤Yb​\(t\)\+EU​Ut,Yb​\(t\)≤Kb​Ztb,ft​Yb​\(t\)≤A×​κ​rT,mt​Yb​\(t\)≤χb​βt​Rt\.\\displaystyle\\Psi\_\{y\}\(t\)\\leq Y\_\{b\}\(t\)\+E\_\{U\}U\_\{t\},\\quad Y\_\{b\}\(t\)\\leq K\_\{b\}Z\_\{t\}^\{b\},\\quad f\_\{t\}Y\_\{b\}\(t\)\\leq A\_\{\\times\}\\kappa r\_\{T\},\\quad m\_\{t\}Y\_\{b\}\(t\)\\leq\\chi\_\{b\}\\beta\_\{t\}R\_\{t\}\.\(149\)DefineXm=𝖫1​\(m​U\)X\_\{m\}=\\mathsf\{L\}\_\{1\}\(mU\),Efb=𝖫2​\(f​Yb\)E\_\{f\}^\{b\}=\\mathsf\{L\}\_\{2\}\(fY\_\{b\}\)andEmb=𝖫1​\(m​Yb\)E\_\{m\}^\{b\}=\\mathsf\{L\}\_\{1\}\(mY\_\{b\}\)\. The definition ofΞ\\Xiand the first bound above imply

Ξ≤Efb\+Emb\+EU​\(Xf\+Xm\),βt​Efb​\(t\)≤χf​βt​Rt\.\\displaystyle\\Xi\\leq E\_\{f\}^\{b\}\+E\_\{m\}^\{b\}\+E\_\{U\}\(X\_\{f\}\+X\_\{m\}\),\\qquad\\beta\_\{t\}E\_\{f\}^\{b\}\(t\)\\leq\\chi\_\{f\}\\beta\_\{t\}R\_\{t\}\.\(150\)For the second inequality, useEfb≤A×​κ​rT​CK​ρschE\_\{f\}^\{b\}\\leq A\_\{\\times\}\\kappa r\_\{T\}C\_\{K\}\\rho\_\{\\rm sch\}andRt≥cR​rTR\_\{t\}\\geq c\_\{R\}r\_\{T\}\.

Lemma[C\.5](https://arxiv.org/html/2610.41074#Thmtheorem5a)and the kernel comparison in Lemma[C\.2](https://arxiv.org/html/2610.41074#Thmtheorem2a)give, for everyz≥0z\\geq 0,

⟨𝖫2​z,G⟩out≤Dout​⟨z,G⟩out,⟨𝖫1​z,G⟩out≤CJ​Dout​⟨z,J⟩out,\\displaystyle\\langle\\mathsf\{L\}\_\{2\}z,G\\rangle\_\{\\rm out\}\\leq D\_\{\\rm out\}\\langle z,G\\rangle\_\{\\rm out\},\\quad\\langle\\mathsf\{L\}\_\{1\}z,G\\rangle\_\{\\rm out\}\\leq C\_\{J\}D\_\{\\rm out\}\\langle z,J\\rangle\_\{\\rm out\},\(151\)⟨β​𝖫1​z,J⟩out≤cb​Θ​CJ​Dout​⟨z,J⟩out\.\\displaystyle\\langle\\beta\\mathsf\{L\}\_\{1\}z,J\\rangle\_\{\\rm out\}\\leq c\_\{b\}\\Theta C\_\{J\}D\_\{\\rm out\}\\langle z,J\\rangle\_\{\\rm out\}\.The last inequality also usesβt≤cb​Θ/M\\beta\_\{t\}\\leq c\_\{b\}\\Theta/M\. Applying these estimates to the source profiles gives,

⟨Ψx,G⟩out\+⟨β​Ψx,J⟩out\\displaystyle\\langle\\Psi\_\{x\},G\\rangle\_\{\\rm out\}\+\\langle\\beta\\Psi\_\{x\},J\\rangle\_\{\\rm out\}≤C⁡\(P\+Convf→bout\),\\displaystyle\\leq C\(P\+\\operatorname\{Conv\}\_\{f\\to b\}^\{\\rm out\}\),⟨f​Ψy,G⟩out\+⟨m​Ψy,J⟩out\\displaystyle\\langle f\\Psi\_\{y\},G\\rangle\_\{\\rm out\}\+\\langle m\\Psi\_\{y\},J\\rangle\_\{\\rm out\}≤C⁡\(P\+Convb→fout\),\\displaystyle\\leq C\(P\+\\operatorname\{Conv\}\_\{b\\to f\}^\{\\rm out\}\),⟨Ξ,G⟩out\+⟨β​Ξ,J⟩out\\displaystyle\\langle\\Xi,G\\rangle\_\{\\rm out\}\+\\langle\\beta\\Xi,J\\rangle\_\{\\rm out\}≤C⁡\(P\+Convb→fout\+Convf→bout\),\\displaystyle\\leq C\(P\+\\operatorname\{Conv\}\_\{b\\to f\}^\{\\rm out\}\+\\operatorname\{Conv\}\_\{f\\to b\}^\{\\rm out\}\),for a fixedCC\. The first usesXf≤C3​Yf,HX\_\{f\}\\leq C\_\{3\}Y^\{f,H\}; the second uses \([149](https://arxiv.org/html/2610.41074#A3.E149)\)\. For the third, apply \([151](https://arxiv.org/html/2610.41074#A3.E151)\) to the decomposition \([150](https://arxiv.org/html/2610.41074#A3.E150)\), using its pointwise bound for⟨β​Efb,J⟩out≤χf​Pb\\langle\\beta E\_\{f\}^\{b\},J\\rangle\_\{\\rm out\}\\leq\\chi\_\{f\}P\_\{b\}\. Substitute into \([145](https://arxiv.org/html/2610.41074#A3.E145)\) and use the boundsx\+≤4​Ψx\+8​Ξx^\{\+\}\\leq 4\\Psi\_\{x\}\+8\\Xi,y\+≤4​Ψyy^\{\+\}\\leq 4\\Psi\_\{y\}of Lemma[C\.4](https://arxiv.org/html/2610.41074#Thmtheorem4a)\. This gives a fixedC∗out≥1C\_\{\*\}^\{\\rm out\}\\geq 1with

Remout≤C∗out​\[P\+Convb→fout\+Convf→bout\]\.\\operatorname\{Rem\}^\{\\rm out\}\\leq C\_\{\*\}^\{\\rm out\}\[P\+\\operatorname\{Conv\}\_\{b\\to f\}^\{\\rm out\}\+\\operatorname\{Conv\}\_\{f\\to b\}^\{\\rm out\}\]\.\(152\)Substituting the gain definition \([83](https://arxiv.org/html/2610.41074#A3.E83)\) and the rounding coefficients gives

βt​Ytf,H≤Γf→b​\[cf​θG​βt​Rt\+\(cf​cb/cm\)​mt​Ut\]≤C¯0​Γf→b​Bt0,\\beta\_\{t\}Y\_\{t\}^\{f,H\}\\leq\\Gamma\_\{f\\to b\}\[c\_\{f\}\\theta\_\{G\}\\beta\_\{t\}R\_\{t\}\+\(c\_\{f\}c\_\{b\}/c\_\{m\}\)m\_\{t\}U\_\{t\}\]\\leq\\bar\{C\}\_\{0\}\\Gamma\_\{f\\to b\}B\_\{t\}^\{0\},whereB0=β​R\+m​UB^\{0\}=\\beta R\+mUas in \([120](https://arxiv.org/html/2610.41074#A3.E120)\), andC¯0\\bar\{C\}\_\{0\}is defined in \([122](https://arxiv.org/html/2610.41074#A3.E122)\)\. This holds also atβt=0\\beta\_\{t\}=0\. Multiply byht​ℛtout​\(J\)h\_\{t\}\\mathcal\{R\}\_\{t\}^\{\\rm out\}\(J\)and sum, usingBt0​J⪯Dt\+B\_\{t\}^\{0\}J\\preceq D\_\{t\}^\{\+\}andΓf→b≤κ\\Gamma\_\{f\\to b\}\\leq\\kappa:

Convf→bout≤C¯0​κ​Ψ\+out\.\\operatorname\{Conv\}\_\{f\\to b\}^\{\\rm out\}\\leq\\bar\{C\}\_\{0\}\\kappa\\Psi\_\{\+\}^\{\\rm out\}\.\(153\)TakingC∗outC\_\{\*\}^\{\\rm out\}large enough for both outputs, define

C13=\(1\+C¯0​κ\)​C∗out\.C\_\{13\}=\(1\+\\bar\{C\}\_\{0\}\\kappa\)C\_\{\*\}^\{\\rm out\}\.\(154\)Then \([152](https://arxiv.org/html/2610.41074#A3.E152)\) proves \([147](https://arxiv.org/html/2610.41074#A3.E147)\) and

Remout≤C13​𝒩out\.\\operatorname\{Rem\}^\{\\rm out\}\\leq C\_\{13\}\\mathcal\{N\}^\{\\rm out\}\.\(155\)All coefficients are fixed independently of the schedule and output;C13C\_\{13\}is not used to choose admissibility thresholds\.

SinceG​Xt\+​G⪯xt\+​GGX\_\{t\}^\{\+\}G\\preceq x\_\{t\}^\{\+\}G, the upper source in \([53](https://arxiv.org/html/2610.41074#A2.E53)\) is bounded byDt\+D\_\{t\}^\{\+\}plus the four terms in \([145](https://arxiv.org/html/2610.41074#A3.E145)\)\. The covariance sandwich \([54](https://arxiv.org/html/2610.41074#A2.E54)\) and output identity \([109](https://arxiv.org/html/2610.41074#A3.E109)\) therefore giveΔout≤Ψ\+out\+Remout\\Delta^\{\\rm out\}\\leq\\Psi\_\{\+\}^\{\\rm out\}\+\\operatorname\{Rem\}^\{\\rm out\}\. The same positivity argument givesΔout≥Ψ−out\\Delta^\{\\rm out\}\\geq\\Psi\_\{\-\}^\{\\rm out\}, sinceQt−⪰Dt−Q\_\{t\}^\{\-\}\\succeq D\_\{t\}^\{\-\}\. Equation \([155](https://arxiv.org/html/2610.41074#A3.E155)\) now proves \([146](https://arxiv.org/html/2610.41074#A3.E146)\)\. ∎

###### Proposition C\.7\(Matching lower bound for SR\-induced error\)\.

For every full\-row\-rankSS, finite deterministic schedule and either output in \([8](https://arxiv.org/html/2610.41074#S2.E8)\),

Δout≥Ψ−out\+Convb→f−,out\+Convf→b−,out≥cnative​𝒩out,\\Delta^\{\\rm out\}\\geq\\Psi\_\{\-\}^\{\\rm out\}\+\\operatorname\{Conv\}\_\{b\\to f\}^\{\-,\\rm out\}\+\\operatorname\{Conv\}\_\{f\\to b\}^\{\-,\\rm out\}\\geq c\_\{\\rm native\}\\mathcal\{N\}^\{\\rm out\},\(156\)whererf=rb=1/144,rm=\(16/3\)​c×,r\_\{f\}=r\_\{b\}=1/144,r\_\{m\}=\(16/3\)c\_\{\\times\},and

cnative=min⁡\(rf,rb,rm,rf​rb\)=rm\>0\.\\displaystyle c\_\{\\rm native\}=\\min\(r\_\{f\},r\_\{b\},r\_\{m\},r\_\{f\}r\_\{b\}\)=r\_\{m\}\>0\.\(157\)

###### Proof\.

Unroll the separate sourcesβu−​Ru​J\\beta\_\{u\}^\{\-\}R\_\{u\}Jandfu−​Uu​Gf\_\{u\}^\{\-\}U\_\{u\}Gto getyt−≥Ztb,−y\_\{t\}^\{\-\}\\geq Z\_\{t\}^\{b,\-\}andxt−≥Ytf,−x\_\{t\}^\{\-\}\\geq Y\_\{t\}^\{f,\-\}\. Distinct terms giveQt−⪰Dt−\+ft−​Ztb,−​G\+βt−​Ytf,−​JQ\_\{t\}^\{\-\}\\succeq D\_\{t\}^\{\-\}\+f\_\{t\}^\{\-\}Z\_\{t\}^\{b,\-\}G\+\\beta\_\{t\}^\{\-\}Y\_\{t\}^\{f,\-\}J\. Apply the positive response, sum and useQt⪰Qt−Q\_\{t\}\\succeq Q\_\{t\}^\{\-\}\. The coefficient ratios in \([47](https://arxiv.org/html/2610.41074#A2.E47)\) arerf,rb,rmr\_\{f\},r\_\{b\},r\_\{m\}, soΨ−≥min⁡\(rf,rb,rm\)​Ψ\+\\Psi\_\{\-\}\\geq\\min\(r\_\{f\},r\_\{b\},r\_\{m\}\)\\Psi\_\{\+\}and each conversion has ratiorf​rbr\_\{f\}r\_\{b\}\. Lemma[B\.2](https://arxiv.org/html/2610.41074#Thmtheorem2)inducesrm<rf​rbr\_\{m\}<r\_\{f\}r\_\{b\}\. ∎

##### Corollary: covariance and full\-risk comparisons\.

SupposeS∈ΩMS\\in\\Omega\_\{M\}, Assumption[C\.1](https://arxiv.org/html/2610.41074#Thmtheorem1b)holds,

M≥Mw,Ma≥τ∗\.M\\geq M\_\{\\rm w\},\\qquad M^\{a\}\\geq\\tau\_\{\*\}\.\(158\)For either output in \([39](https://arxiv.org/html/2610.41074#A2.E39)\), the SR\-induced risk satisfies

cnative​𝒩out≤Δout≤\(1\+C13\)​𝒩out\.c\_\{\\rm native\}\\mathcal\{N\}^\{\\rm out\}\\leq\\Delta^\{\\rm out\}\\leq\(1\+C\_\{13\}\)\\mathcal\{N\}^\{\\rm out\}\.\(159\)Withℬout,𝒱out\\mathcal\{B\}^\{\\rm out\},\\mathcal\{V\}^\{\\rm out\}from \([75](https://arxiv.org/html/2610.41074#A3.E75)\) and𝒩out\\mathcal\{N\}^\{\\rm out\}from \([81](https://arxiv.org/html/2610.41074#A3.E81)\), the full prediction risk satisfies

ℛout\\displaystyle\\mathscr\{R\}^\{\\rm out\}≥cA​M1−b\+cV​ℬout\+𝒱out\+cnative​𝒩out,\\displaystyle\\geq c\_\{A\}M^\{1\-b\}\+c\_\{V\}\\mathcal\{B\}^\{\\rm out\}\+\\mathcal\{V\}^\{\\rm out\}\+c\_\{\\rm native\}\\mathcal\{N\}^\{\\rm out\},ℛout\\displaystyle\\mathscr\{R\}^\{\\rm out\}≤CAsch​M1−b\+CV​ℬout\+2​𝒱out\+\(1\+C13\)​𝒩out\.\\displaystyle\\leq C\_\{A\}^\{\\rm sch\}M^\{1\-b\}\+C\_\{V\}\\mathcal\{B\}^\{\\rm out\}\+2\\mathcal\{V\}^\{\\rm out\}\+\(1\+C\_\{13\}\)\\mathcal\{N\}^\{\\rm out\}\.\(160\)In particular, common lower and upper constants are

cfull=min⁡\{cA,cV,1,cnative\}\>0,Cfull=max⁡\{CAsch,CV,2,1\+C13\}<∞\.\\displaystyle c\_\{\\rm full\}=\\min\\\{c\_\{A\},c\_\{V\},1,c\_\{\\rm native\}\\\}\>0,\\qquad C\_\{\\rm full\}=\\max\\\{C\_\{A\}^\{\\rm sch\},C\_\{V\},2,1\+C\_\{13\}\\\}<\\infty\.\(161\)
###### Proof\.

Theorem[C\.6](https://arxiv.org/html/2610.41074#Thmtheorem6a)and Proposition[C\.7](https://arxiv.org/html/2610.41074#Thmtheorem7a)give \([159](https://arxiv.org/html/2610.41074#A3.E159)\)\. Add the exact\-arithmetic comparison of Theorem[C\.3](https://arxiv.org/html/2610.41074#Thmtheorem3a)and the approximation bounds \([69](https://arxiv.org/html/2610.41074#A2.E69)\), using the risk decomposition \([38](https://arxiv.org/html/2610.41074#A2.E38)\), to obtain \([160](https://arxiv.org/html/2610.41074#A3.E160)\)\. Taking the minimum and maximum of its coefficients gives \([161](https://arxiv.org/html/2610.41074#A3.E161)\)\. The exact\-arithmetic part also givesℛ0out≍M1−b\+ℬout\+𝒱out\\mathscr\{R\}\_\{0\}^\{\\rm out\}\\asymp M^\{1\-b\}\+\\mathcal\{B\}^\{\\rm out\}\+\\mathcal\{V\}^\{\\rm out\}\. All bounds hold deterministically onΩM\\Omega\_\{M\}for every admissible schedule chosen before the random draws; admissibility is checked at the realized sketch\. The event’s probability guarantee is for each width separately, as in Proposition[B\.9](https://arxiv.org/html/2610.41074#Thmtheorem9)\. ∎

##### Replacing the exact\-arithmetic residual by its noise scale\.

OnΩM\\Omega\_\{M\}under the cap and source rate, define𝒱out,\(s\)=12​∑u<Thu​s​∑jgj2​ℓout,j,u2\\mathcal\{V\}^\{\\rm out,\(s\)\}=\\tfrac\{1\}\{2\}\\sum\_\{u<T\}h\_\{u\}s\\sum\_\{j\}g\_\{j\}^\{2\}\\ell\_\{\{\\rm out\},j,u\}^\{2\}\. Multiplying \([111](https://arxiv.org/html/2610.41074#A3.E111)\) by each nonnegative squared\-filter term gives𝒱out,\(s\)≤𝒱out≤Ksref​𝒱out,\(s\)\\mathcal\{V\}^\{\\rm out,\(s\)\}\\leq\\mathcal\{V\}^\{\\rm out\}\\leq K\_\{s\}^\{\\rm ref\}\\mathcal\{V\}^\{\\rm out,\(s\)\}for any finite schedule and prescribed output\. Thus \([160](https://arxiv.org/html/2610.41074#A3.E160)\) allows this replacement, with unchanged lower constant and𝒩\\mathcal\{N\}, and upper constantmax⁡\{CAsch,CV,2​Ksref,1\+C13\}\\max\\\{C\_\{A\}^\{\\rm sch\},C\_\{V\},2K\_\{s\}^\{\\rm ref\},1\+C\_\{13\}\\\}\. The factorKsrefK\_\{s\}^\{\\rm ref\}is distinct from the applications’KsK\_\{s\}\.

### C\.6Construction of the admissibility thresholds

Fix the population family, its positive noise varianceσ2\\sigma^\{2\}, a valid comparison tupleWWfrom Definition[B\.8](https://arxiv.org/html/2610.41074#Thmtheorem8), and anyc0∈\(0,1\)c\_\{0\}\\in\(0,1\)\. RecallΘ=Cθtr\\Theta=C\_\{\\theta\}^\{\\rm tr\}and the exponentsα,ν,ωV\\alpha,\\nu,\\omega\_\{V\}from \([70](https://arxiv.org/html/2610.41074#A3.E70)\)\. The choices have a definite order: first choose the cumulative\-time thresholdτ∗\\tau\_\{\*\}below, then compute the auxiliary bounds, and finally define the feedback thresholdκsch\\kappa\_\{\\rm sch\}and rate ceilingρsch\\rho\_\{\\rm sch\}in \([166](https://arxiv.org/html/2610.41074#A3.E166)\)\. This order ensures that the thresholds depend only on fixed model quantities, with no circular dependence on the schedules to be compared\.

##### Cumulative optimization time threshold\.

Setη¯=c0/cg\\bar\{\\eta\}=c\_\{0\}/c\_\{g\},CA¯=2​CAschC\_\{\\bar\{A\}\}=2C\_\{A\}^\{\\rm sch\}andΛ0=\(1−c0\)−1\\Lambda\_\{0\}=\(1\-c\_\{0\}\)^\{\-1\}\. Choose and fixτ∗=τ∗sch=τ∗,sch\\tau\_\{\*\}=\\tau\_\{\*\}^\{\\rm sch\}=\\tau\_\{\*,\\rm sch\}satisfying

τ∗≥32​\(1\+η¯\+cg−1\),c′=2​\(1−c0\),C′=2/\(1−c0\)\.\\tau\_\{\*\}\\geq 32\(1\+\\bar\{\\eta\}\+c\_\{g\}^\{\-1\}\),\\qquad c^\{\\prime\}=2\(1\-c\_\{0\}\),\\quad C^\{\\prime\}=2/\(1\-c\_\{0\}\)\.\(162\)The chosenτ∗\\tau\_\{\*\}may exceed this lower bound\. It is used throughout the scheduled theorem and its constant\-learning\-rate corollaries\. The coefficientsCfbC\_\{\\rm fb\}andCUfbC\_\{U\}^\{\\rm fb\}below depend on this choice, so the rate and feedback thresholds are computed afterwards\.

##### Bounds for propagation and spectral sums\.

The following coefficients give the spectral bounds and the comparison factorC3C\_\{3\}in Lemma[C\.2](https://arxiv.org/html/2610.41074#Thmtheorem2a)\. Forℓ=1,2\\ell=1,2define

CU,ℓ=2ℓ​\(1\+Cgℓ​aa−1\),cL,ℓ​\(Λ\)=14​Λ−α​cgℓ​e−2a\+1​Cg​Λ,\\displaystyle C\_\{U,\\ell\}=2^\{\\ell\}\\left\(1\+C\_\{g\}^\{\\ell\}\\frac\{a\}\{a\-1\}\\right\),\\quad c\_\{L,\\ell\}\(\\Lambda\)=\\tfrac\{1\}\{4\}\\Lambda^\{\-\\alpha\}c\_\{g\}^\{\\ell\}e^\{\-2^\{a\+1\}C\_\{g\}\\Lambda\},cL,0​\(Λ\)=12​e−2a\+1​Cg​Λ,L0=cL,0​\(Λ0\),U0sp=1,\\displaystyle c\_\{L,0\}\(\\Lambda\)=\\tfrac\{1\}\{2\}e^\{\-2^\{a\+1\}C\_\{g\}\\Lambda\},\\quad L\_\{0\}=c\_\{L,0\}\(\\Lambda\_\{0\}\),\\quad U\_\{0\}^\{\\rm sp\}=1,Lℓ=cL,ℓ​\(Λ0\)​\(1−c0\)ℓ−α,Uℓsp=CU,ℓ​\(1−c0\)α−ℓ,\\displaystyle L\_\{\\ell\}=c\_\{L,\\ell\}\(\\Lambda\_\{0\}\)\(1\-c\_\{0\}\)^\{\\ell\-\\alpha\},\\quad U\_\{\\ell\}^\{\\rm sp\}=C\_\{U,\\ell\}\(1\-c\_\{0\}\)^\{\\alpha\-\\ell\},cP=min⁡\(1,L0,L1,L2\),CP=max⁡\(1,U0sp,U1sp,U2sp\),C3=CPcP​max⁡\(1,CJ/cJ,CJ/cJ2\)\.\\displaystyle c\_\{P\}=\\min\(1,L\_\{0\},L\_\{1\},L\_\{2\}\),\\quad C\_\{P\}=\\max\(1,U\_\{0\}^\{\\rm sp\},U\_\{1\}^\{\\rm sp\},U\_\{2\}^\{\\rm sp\}\),\\quad C\_\{3\}=\\frac\{C\_\{P\}\}\{c\_\{P\}\}\\max\(1,C\_\{J\}/c\_\{J\},C\_\{J\}/c\_\{J\}^\{2\}\)\.\(163\)The superscript “sp” distinguishes fixed spectral constants from the exact\-arithmetic sourceUtU\_\{t\}\.

##### Bounds for the exact\-arithmetic case\.

The next coefficients bound the residual and parameter histories in Theorem[C\.3](https://arxiv.org/html/2610.41074#Thmtheorem3a)\. The additional coefficientCfbC\_\{\\rm fb\}bounds the forward\-to\-backward gain for constant learning rates and precisions in \([187](https://arxiv.org/html/2610.41074#A4.E187)\)\.

K0=max⁡\(1,Cg/\(1−c0\)\),cLsrc=\(1−e−1\)2​min⁡\(1,cg\)2​Cg−ωV​2−\(a−b\+1\)a−b\+1,\\displaystyle K\_\{0\}=\\max\(1,C\_\{g\}/\(1\-c\_\{0\}\)\),\\quad c\_\{L\}^\{\\rm src\}=\(1\-e^\{\-1\}\)^\{2\}\\min\(1,c\_\{g\}\)^\{2\}C\_\{g\}^\{\-\\omega\_\{V\}\}\\frac\{2^\{\-\(a\-b\+1\)\}\}\{a\-b\+1\},Cbias=CV​Cgb/a​bb−1,ρsrc=14​Θ,CRraw=2​max⁡\(1,Cbias\),\\displaystyle C\_\{\\rm bias\}=C\_\{V\}C\_\{g\}^\{b/a\}\\frac\{b\}\{b\-1\},\\quad\\rho\_\{\\rm src\}=\\frac\{1\}\{4\\Theta\},\\quad C\_\{R\}^\{\\rm raw\}=2\\max\(1,C\_\{\\rm bias\}\),CUraw=CV​CJ​cg−ωV​2​a\+2​b−1a\+b−1​K0ν\+2​CJ​CRraw​\(1\+σ2\+CA¯\)​ρsrc​2​a−1a−1​K0α,\\displaystyle\\begin\{aligned\} C\_\{U\}^\{\\rm raw\}=\{\}&C\_\{V\}C\_\{J\}c\_\{g\}^\{\-\\omega\_\{V\}\}\\frac\{2a\+2b\-1\}\{a\+b\-1\}K\_\{0\}^\{\\nu\}\+2C\_\{J\}C\_\{R\}^\{\\rm raw\}\(1\+\\sigma^\{2\}\+C\_\{\\bar\{A\}\}\)\\rho\_\{\\rm src\}\\frac\{2a\-1\}\{a\-1\}K\_\{0\}^\{\\alpha\},\\end\{aligned\}cU=cV​cJ​cLsrc,csrc=min⁡\{σ2/\(1\+σ2\),cU\},\\displaystyle c\_\{U\}=c\_\{V\}c\_\{J\}c\_\{L\}^\{\\rm src\},\\quad c\_\{\\rm src\}=\\min\\\{\\sigma^\{2\}/\(1\+\\sigma^\{2\}\),c\_\{U\}\\\},Csrc=max⁡\{CRraw​\(1\+σ−2\),CRraw​\(1\+σ2\+CA¯\),CUraw,2ν​CUraw/cU,CUraw/σ2\},\\displaystyle C\_\{\\rm src\}=\\max\\\{C\_\{R\}^\{\\rm raw\}\(1\+\\sigma^\{\-2\}\),C\_\{R\}^\{\\rm raw\}\(1\+\\sigma^\{2\}\+C\_\{\\bar\{A\}\}\),C\_\{U\}^\{\\rm raw\},2^\{\\nu\}C\_\{U\}^\{\\rm raw\}/c\_\{U\},C\_\{U\}^\{\\rm raw\}/\\sigma^\{2\}\\\},\(164\)cRglob=csrc,CRglob=Csrc​\(1\+σ−2\),Cfb=max⁡\{Csrc,Csrc​\(1\+τ∗\)ν/\(csrc​σ2\)\}\.\\displaystyle c\_\{R\}^\{\\rm glob\}=c\_\{\\rm src\},\\quad C\_\{R\}^\{\\rm glob\}=C\_\{\\rm src\}\(1\+\\sigma^\{\-2\}\),\\quad C\_\{\\rm fb\}=\\max\\\{C\_\{\\rm src\},C\_\{\\rm src\}\(1\+\\tau\_\{\*\}\)^\{\\nu\}/\(c\_\{\\rm src\}\\sigma^\{2\}\)\\\}\.\(165\)K0K\_\{0\}is distinct from the sketch norm boundK0,geoK\_\{0,\\rm geo\}in \([55](https://arxiv.org/html/2610.41074#A2.E55)\)\. The exact\-arithmetic moment bounds are established underηu/B≤ρsrc\\eta\_\{u\}/B\\leq\\rho\_\{\\rm src\}; the same constants apply under the smaller ceilingρsch\\rho\_\{\\rm sch\}chosen below\.

##### Bounds for rounding\-induced errors\.

The coefficients for Lemma[C\.4](https://arxiv.org/html/2610.41074#Thmtheorem4a)are defined in \([122](https://arxiv.org/html/2610.41074#A3.E122)\)\. They depend on the fixed source and kernel bounds andτ∗\\tau\_\{\*\}, independently of the thresholds below\. With the preceding coefficients fixed, define

ρfb=min⁡\{ρsrc,\(64​CJ​CK\)−1,\(64​C7\)−1,\(16​CΘ\)−1,\(2​C8\)−1,cR/\(128​CK\)\},\\displaystyle\\rho\_\{\\rm fb\}=\\min\\\{\\rho\_\{\\rm src\},\(64C\_\{J\}C\_\{K\}\)^\{\-1\},\(64C\_\{7\}\)^\{\-1\},\(16C\_\{\\Theta\}\)^\{\-1\},\(2C\_\{8\}\)^\{\-1\},c\_\{R\}/\(128C\_\{K\}\)\\\},κsch=min⁡\{\(128​C3​C¯0\)−1,\(16​CΘ′\)−1,\(2​C8\)−1,cR/\(8​cf​cb​Θ2\)\},\\displaystyle\\kappa\_\{\\rm sch\}=\\min\\\{\(128C\_\{3\}\\bar\{C\}\_\{0\}\)^\{\-1\},\(16C^\{\\prime\}\_\{\\Theta\}\)^\{\-1\},\(2C\_\{8\}\)^\{\-1\},c\_\{R\}/\(8c\_\{f\}c\_\{b\}\\Theta^\{2\}\)\\\},ρsch=min⁡\{ρfb,κsch/\(Cfb​Θ\)\}\.\\displaystyle\\rho\_\{\\rm sch\}=\\min\\\{\\rho\_\{\\rm fb\},\\kappa\_\{\\rm sch\}/\(C\_\{\\rm fb\}\\Theta\)\\\}\.\(166\)Together with the imposed feedback bounds, these minima give \([129](https://arxiv.org/html/2610.41074#A3.E129)\), as required by Lemma[C\.4](https://arxiv.org/html/2610.41074#Thmtheorem4a); the two summands ofϵH\\epsilon\_\{H\}are each at most1/161/16\. Assumption[C\.1](https://arxiv.org/html/2610.41074#Thmtheorem1b)requires each of the three feedback gains to be at mostκsch\\kappa\_\{\\rm sch\}\. The construction does not derive those conditions from the learning\-rate cap alone\. The final minimum also ensuresCfb​Θ​ρsch≤κschC\_\{\\rm fb\}\\Theta\\rho\_\{\\rm sch\}\\leq\\kappa\_\{\\rm sch\}, as used in the constant\-learning\-rate gain bound \([187](https://arxiv.org/html/2610.41074#A4.E187)\)\. Writeκ=κsch\\kappa=\\kappa\_\{\\rm sch\}only as an abbreviation\. These are sufficient thresholds, with no claim of sharpness\. All denominators are positive\. The resulting constants are finite and independent of width, horizon, batch, learning\-rate and precision schedules, and the realized sketch\.

## Appendix DProofs of the neural and compute scaling laws

This appendix proves the scaling laws stated in Section[3](https://arxiv.org/html/2610.41074#S3)\. Appendix[D\.1](https://arxiv.org/html/2610.41074#A4.SS1)proves Theorem[1](https://arxiv.org/html/2610.41074#Thmtheorem1)by converting the covariance bounds of Appendix[C](https://arxiv.org/html/2610.41074#A3)into explicit scalar expressions\. Corollaries[D\.3](https://arxiv.org/html/2610.41074#Thmtheorem3b)and[D\.4](https://arxiv.org/html/2610.41074#Thmtheorem4b)give the constant\-learning\-rate laws\. Proposition[D\.5](https://arxiv.org/html/2610.41074#Thmtheorem5b)proves the compute orders in Table[1](https://arxiv.org/html/2610.41074#S3.T1); Propositions[D\.6](https://arxiv.org/html/2610.41074#Thmtheorem6b)and[D\.7](https://arxiv.org/html/2610.41074#Thmtheorem7b)allow learning rate and batch size to vary across budgets and treat the two outputs\. Let

ℒSout=M1−b\+ℬout\+𝒱out\+𝒩out,cF=cfull,CF=Cfull,\\mathcal\{L\}\_\{S\}^\{\{\\rm out\}\}=M^\{1\-b\}\+\\mathcal\{B\}^\{\{\\rm out\}\}\+\\mathcal\{V\}^\{\{\\rm out\}\}\+\\mathcal\{N\}^\{\{\\rm out\}\},\\qquad c\_\{F\}=c\_\{\\rm full\},\\quad C\_\{F\}=C\_\{\\rm full\},\(167\)where the terms are defined in \([75](https://arxiv.org/html/2610.41074#A3.E75)\), \([81](https://arxiv.org/html/2610.41074#A3.E81)\), and \([161](https://arxiv.org/html/2610.41074#A3.E161)\)\.

### D\.1Scalar comparisons and proof of Theorem[1](https://arxiv.org/html/2610.41074#Thmtheorem1)

We convert Appendix[C](https://arxiv.org/html/2610.41074#A3)’s covariance comparisons into the scalar law of Section[3\.1](https://arxiv.org/html/2610.41074#S3.SS1)\. Recall Assumption[C\.1](https://arxiv.org/html/2610.41074#Thmtheorem1b)and exponentsα,ν,ωV\\alpha,\\nu,\\omega\_\{V\}from \([70](https://arxiv.org/html/2610.41074#A3.E70)\)\.

##### Effective noise and exact\-arithmetic residual bounds\.

The approximation bound andM≥1M\\geq 1give

Ks=1\+2​CAsch/σ2,σ2≤s⁡\(S\)≤Ks​σ2\.K\_\{s\}=1\+2C\_\{A\}^\{\\rm sch\}/\\sigma^\{2\},\\qquad\\sigma^\{2\}\\leq s\(S\)\\leq K\_\{s\}\\sigma^\{2\}\.\(168\)This normalization differs fromKsrefK\_\{s\}^\{\\rm ref\}in \([111](https://arxiv.org/html/2610.41074#A3.E111)\), which bounds the exact\-arithmetic residual throughout training\. Equation \([117](https://arxiv.org/html/2610.41074#A3.E117)\) supplies the fixed choice

Cσ=2​\(1\+Cbias/σ2\),s≤Rt≤Cσ​s\.C\_\{\\sigma\}=2\(1\+C\_\{\\rm bias\}/\\sigma^\{2\}\),\\qquad s\\leq R\_\{t\}\\leq C\_\{\\sigma\}s\.\(169\)Thusσ2≤Rt≤Cσ​Ks​σ2\\sigma^\{2\}\\leq R\_\{t\}\\leq C\_\{\\sigma\}K\_\{s\}\\sigma^\{2\}throughout training\.

##### Decomposing the scalar rounding scale\.

ReuseKrout,B^out,V^outK\_\{r\}^\{\{\\rm out\}\},\\widehat\{B\}^\{\{\\rm out\}\},\\widehat\{V\}^\{\{\\rm out\}\}andU^t\\widehat\{U\}\_\{t\}from \([11](https://arxiv.org/html/2610.41074#S3.E11)\)–\([14](https://arxiv.org/html/2610.41074#S3.E14)\)\. For a deterministic training configurationπ\\pi, writeΦout​\(π\)\\Phi^\{\{\\rm out\}\}\(\\pi\)for the scalar objective defined below\. For a fixed output, suppress its superscript onB^,V^\\widehat\{B\},\\widehat\{V\}and define the auxiliary quantities needed to separate the two orders of interaction:

kr​\(t,u\)\\displaystyle k\_\{r\}\(t,u\)=\[1\+2\(τt−τu\+1\)\]α−r\(u<t,r=1,2\),\\displaystyle=\[1\+2\(\\tau\_\{t\}\-\\tau\_\{u\+1\}\)\]^\{\\alpha\-r\}\\quad\(u<t,\\ r=1,2\),\(170\)Z^t\\displaystyle\\widehat\{Z\}\_\{t\}=σ2∑u<thuqbw,u,Y^t=∑u<thuqfw,uMU^uk2\(t,u\)\.\\displaystyle=\\sigma^\{2\}\\sum\_\{u<t\}h\_\{u\}q\_\{\\mathrm\{bw\},u\},\\qquad\\widehat\{Y\}\_\{t\}=\\sum\_\{u<t\}h\_\{u\}\\frac\{q\_\{\\mathrm\{fw\},u\}\}\{M\}\\widehat\{U\}\_\{u\}k\_\{2\}\(t,u\)\.LetΨ^\\widehat\{\\Psi\}denote the first sum in \([15](https://arxiv.org/html/2610.41074#S3.E15)\)\. The two terms in its second sum are

C^b​f\\displaystyle\\widehat\{C\}\_\{bf\}=12∑t<Thtqfw,tMZ^tK2out\(t\),C^f​b=12∑t<Thtqbw,tMY^tK1out\(t\),\\displaystyle=\\tfrac\{1\}\{2\}\\sum\_\{t<T\}h\_\{t\}\\frac\{q\_\{\\mathrm\{fw\},t\}\}\{M\}\\widehat\{Z\}\_\{t\}K\_\{2\}^\{\{\\rm out\}\}\(t\),\\quad\\widehat\{C\}\_\{fb\}=\\tfrac\{1\}\{2\}\\sum\_\{t<T\}h\_\{t\}\\frac\{q\_\{\\mathrm\{bw\},t\}\}\{M\}\\widehat\{Y\}\_\{t\}K\_\{1\}^\{\{\\rm out\}\}\(t\),\(171\)N^out\\displaystyle\\widehat\{N\}^\{\{\\rm out\}\}=Ψ^\+C^b​f\+C^f​b\.\\displaystyle=\\widehat\{\\Psi\}\+\\widehat\{C\}\_\{bf\}\+\\widehat\{C\}\_\{fb\}\.This is an exact decomposition of the scalar expression, with no new approximation\. For the optimization arguments below, write

Φout=M1−b\+B^out\+V^out\+N^out\.\\Phi^\{\{\\rm out\}\}=M^\{1\-b\}\+\\widehat\{B\}^\{\{\\rm out\}\}\+\\widehat\{V\}^\{\{\\rm out\}\}\+\\widehat\{N\}^\{\{\\rm out\}\}\.\(172\)We also suppress the superscript onΦ\\Phiwhen the output is fixed\.

The following theorem replaces the matrix expressions from Appendix[C](https://arxiv.org/html/2610.41074#A3)by explicit scalar sums\. The comparison is uniform over admissible configurations, so minimizing the scalar objective controls prediction risk within a fixed factor\.

###### Theorem D\.1\(Actual\-risk comparison with a finite discrete objective\)\.

Under Assumption[C\.1](https://arxiv.org/html/2610.41074#Thmtheorem1b), with both width conditions and the fixedCσC\_\{\\sigma\}in \([169](https://arxiv.org/html/2610.41074#A4.E169)\), forout∈\{last,avg\}\{\\rm out\}\\in\\\{\\mathrm\{last\},\\mathrm\{avg\}\\\}, we have

aΦ​Φout​\(π\)≤ℛout​\(π\)≤AΦ​Φout​\(π\),a\_\{\\Phi\}\\Phi^\{\\rm out\}\(\\pi\)\\leq\\mathscr\{R\}^\{\\rm out\}\(\\pi\)\\leq A\_\{\\Phi\}\\Phi^\{\\rm out\}\(\\pi\),\(173\)for fixed0<aΦ≤AΦ<∞0<a\_\{\\Phi\}\\leq A\_\{\\Phi\}<\\inftydepending only on the fixed model, the comparison bounds in Definition[B\.8](https://arxiv.org/html/2610.41074#Thmtheorem8),c0c\_\{0\},τ∗\\tau\_\{\*\},CσC\_\{\\sigma\}andKsK\_\{s\}, uniformly inS,M,T,B,FS,M,T,B,F, precision choices and learning rates\.

##### Interpretation of the scalar quantities\.

The objectiveΦ\\Phiseparates approximation error, bias, minibatch variance and SR\-induced error\. The scaleU^t\\widehat\{U\}\_\{t\}estimates the exact\-arithmetic parameter momentUtU\_\{t\}, the source of direct forward\-rounding error\. Its two terms describe learning the target and fluctuations from minibatch noise\. WithinN^\\widehat\{N\},Ψ^\\widehat\{\\Psi\}contains direct forward and backward rounding and their interaction\. Each contribution is weighted byhuh\_\{u\}and its output responseK1K\_\{1\}orK2K\_\{2\}\. The termC^b​f\\widehat\{C\}\_\{bf\}captures earlier backward rounding changing the parameter moment entering later forward rounding;C^f​b\\widehat\{C\}\_\{fb\}captures earlier forward rounding changing the residual variance entering later backward rounding\. These interactions can occur even with independent, unbiased rounding because later error variances depend on the current state\. All joint terms vanish if either rounding location uses exact arithmetic throughout training\.

###### Proof\.

We compare each nonnegative term separately\. The exact\-arithmetic parameter source is the only term requiring both its deterministic and stochastic parts to be estimated before the scalar responses can be substituted\. Putr∗=2−ωVr\_\{\*\}=2\-\\omega\_\{V\},Dν=22−ν​max⁡\(1,3ν/ν\)D\_\{\\nu\}=2^\{2\-\\nu\}\\max\(1,3^\{\\nu\}/\\nu\), and define

u−\\displaystyle u\_\{\-\}=min⁡\{cV​cJ​ℓr∗​2ν−2,cJ​ℓ1\},\\displaystyle=\\min\\\{c\_\{V\}c\_\{J\}\\ell\_\{r\_\{\*\}\}2^\{\\nu\-2\},\\ c\_\{J\}\\ell\_\{1\}\\\},u\+\\displaystyle u\_\{\+\}=max⁡\{CV​CJ​ur∗​\(1−c0\)−2​Dν,2​Cσ​Ks​CJ​u1\},R\+=Cσ​Ks\.\\displaystyle=\\max\\\{C\_\{V\}C\_\{J\}u\_\{r\_\{\*\}\}\(1\-c\_\{0\}\)^\{\-2\}D\_\{\\nu\},2C\_\{\\sigma\}K\_\{s\}C\_\{J\}u\_\{1\}\\\},\\quad R\_\{\+\}=C\_\{\\sigma\}K\_\{s\}\.\(174\)We first proveu−​U^t≤Ut≤u\+​U^tu\_\{\-\}\\widehat\{U\}\_\{t\}\\leq U\_\{t\}\\leq u\_\{\+\}\\widehat\{U\}\_\{t\}for every state, including zero clock\. For the mean part of \([115](https://arxiv.org/html/2610.41074#A3.E115)\), the product bounds give1−e−gj​τt≤1−πj​\(t,0\)≤\(1−c0\)−1​\(1−e−gj​τt\)1\-e^\{\-g\_\{j\}\\tau\_\{t\}\}\\leq 1\-\\pi\_\{j\}\(t,0\)\\leq\(1\-c\_\{0\}\)^\{\-1\}\(1\-e^\{\-g\_\{j\}\\tau\_\{t\}\}\)\. Moreover,

∑jgj−ωV​\(1−e−gj​τ\)2=∫0τ∫0τ∑jgj2−ωV​e−gj​\(v\+w\)​𝑑v​𝑑w\.\\sum\_\{j\}g\_\{j\}^\{\-\\omega\_\{V\}\}\(1\-e^\{\-g\_\{j\}\\tau\}\)^\{2\}=\\int\_\{0\}^\{\\tau\}\\int\_\{0\}^\{\\tau\}\\sum\_\{j\}g\_\{j\}^\{2\-\\omega\_\{V\}\}e^\{\-g\_\{j\}\(v\+w\)\}\\,dv\\,dw\.The teacher\-source sandwich can be traced against\(I−Pt\)​J​\(I−Pt\)\(I\-P\_\{t\}\)J\(I\-P\_\{t\}\)without assuming thatJJandVVcommute withGG\. Lemma[C\.2](https://arxiv.org/html/2610.41074#Thmtheorem2a)applies to the displayed integral sincev\+w≤2​Mav\+w\\leq 2M^\{a\}\. The integral of\(1\+v\+w\)ν−2\(1\+v\+w\)^\{\\nu\-2\}lies between2ν−2​τ2​\(1\+τ\)ν−22^\{\\nu\-2\}\\tau^\{2\}\(1\+\\tau\)^\{\\nu\-2\}andDν​τ2​\(1\+τ\)ν−2D\_\{\\nu\}\\tau^\{2\}\(1\+\\tau\)^\{\\nu\-2\}: usev\+w≤2​τv\+w\\leq 2\\taufor the lower; for the upper use integrand≤1\\leq 1atτ≤1\\tau\\leq 1and∫02​τz​\(1\+z\)ν−2​𝑑z≤\(1\+2​τ\)ν/ν\\int\_\{0\}^\{2\\tau\}z\(1\+z\)^\{\\nu\-2\}dz\\leq\(1\+2\\tau\)^\{\\nu\}/\\nuotherwise\. These bounds give the first entries in the minimum and maximum definingu−u\_\{\-\}andu\+u\_\{\+\}\. For the stochastic part, useσ2​G⪯Nu⪯2​R\+​σ2​G\\sigma^\{2\}G\\preceq N\_\{u\}\\preceq 2R\_\{\+\}\\sigma^\{2\}Gin \([115](https://arxiv.org/html/2610.41074#A3.E115)\), then trace againstJJ\. The boundscJ≤Jj​j≤CJc\_\{J\}\\leq J\_\{jj\}\\leq C\_\{J\}and \([93](https://arxiv.org/html/2610.41074#A3.E93)\) atx=2​\(τt−τu\+1\)x=2\(\\tau\_\{t\}\-\\tau\_\{u\+1\}\)give the second entries\. In particular, the noise contribution contains∑u<thu​k1​\(t,u\)\\sum\_\{u<t\}h\_\{u\}k\_\{1\}\(t,u\)even before clock one\. At zero clock both the mean and noise parts vanish\.

Expand each output filter into its pair of state indices\. The weights are nonnegative for both outputs under consideration, so the spectral comparison can be applied to every pair and summed\.

12​ℓ2​K2out​\(u\)≤ℛuout​\(G\)≤12​u2​K2out​\(u\),12​cJ​ℓ1​K1out​\(u\)≤ℛuout​\(J\)≤12​CJ​u1​K1out​\(u\)\.\\tfrac\{1\}\{2\}\\ell\_\{2\}K\_\{2\}^\{\\rm out\}\(u\)\\leq\\mathcal\{R\}\_\{u\}^\{\\rm out\}\(G\)\\leq\\tfrac\{1\}\{2\}u\_\{2\}K\_\{2\}^\{\\rm out\}\(u\),\\qquad\\tfrac\{1\}\{2\}c\_\{J\}\\ell\_\{1\}K\_\{1\}^\{\\rm out\}\(u\)\\leq\\mathcal\{R\}\_\{u\}^\{\\rm out\}\(J\)\\leq\\tfrac\{1\}\{2\}C\_\{J\}u\_\{1\}K\_\{1\}^\{\\rm out\}\(u\)\.All paired clocks lie in\[0,2​Ma\]\[0,2M^\{a\}\]\. The same argument applied to the bias gives

ℓb/a​B^≤ℬout≤ub/a​B^,ℓ2​V^≤𝒱out≤R\+​u2​V^\.\\ell\_\{b/a\}\\widehat\{B\}\\leq\\mathcal\{B\}^\{\\rm out\}\\leq u\_\{b/a\}\\widehat\{B\},\\qquad\\ell\_\{2\}\\widehat\{V\}\\leq\\mathcal\{V\}^\{\\rm out\}\\leq R\_\{\+\}u\_\{2\}\\widehat\{V\}\.\(175\)The second comparison also usesσ2≤Ru≤R\+​σ2\\sigma^\{2\}\\leq R\_\{u\}\\leq R\_\{\+\}\\sigma^\{2\}\. The boundsθG∈\[cg,Θ\]\\theta\_\{G\}\\in\[c\_\{g\},\\Theta\]and \([46](https://arxiv.org/html/2610.41074#A2.E46)\) imply

fu\+≍qfw,u/M,βu\+≍qbw,u/M,mu\+≍qfw,u​qbw,u/M2\.f\_\{u\}^\{\+\}\\asymp q\_\{\\mathrm\{fw\},u\}/M,\\qquad\\beta\_\{u\}^\{\+\}\\asymp q\_\{\\mathrm\{bw\},u\}/M,\\qquad m\_\{u\}^\{\+\}\\asymp q\_\{\\mathrm\{fw\},u\}q\_\{\\mathrm\{bw\},u\}/M^\{2\}\.Together with the preceding source and output comparisons, these bounds identify the three terms ofΨ^\\widehat\{\\Psi\}\. For the ordered terms, estimate the inner sources before the outer response\. Equation \([96](https://arxiv.org/html/2610.41074#A3.E96)\) givesZtb,\+≍Z^tZ\_\{t\}^\{b,\+\}\\asymp\\widehat\{Z\}\_\{t\}: the factorMMinKJ​JHK\_\{JJ\}^\{H\}cancels the1/M1/Minβu\+\\beta\_\{u\}^\{\+\}\. Similarly,Uu≍U^uU\_\{u\}\\asymp\\widehat\{U\}\_\{u\}and \([93](https://arxiv.org/html/2610.41074#A3.E93)\) giveYtf,\+≍Y^tY\_\{t\}^\{f,\+\}\\asymp\\widehat\{Y\}\_\{t\}\. Multiplying each by its outer rounding coefficient and output response givesC^b​f\\widehat\{C\}\_\{bf\}andC^f​b\\widehat\{C\}\_\{fb\}, respectively, with the strict time ordering unchanged\. Hence fixed coefficientslN,uNl\_\{N\},u\_\{N\}satisfy

lN​N^out≤𝒩out≤uN​N^out\.l\_\{N\}\\widehat\{N\}^\{\\rm out\}\\leq\\mathcal\{N\}^\{\\rm out\}\\leq u\_\{N\}\\widehat\{N\}^\{\\rm out\}\.\(176\)ChooselΦ\>0l\_\{\\Phi\}\>0below all the component lower coefficients anduΦ<∞u\_\{\\Phi\}<\\inftyabove all the upper coefficients, including the approximation coefficient one\. ThenlΦ​Φ≤ℒSout≤uΦ​Φl\_\{\\Phi\}\\Phi\\leq\\mathcal\{L\}\_\{S\}^\{\\rm out\}\\leq u\_\{\\Phi\}\\Phiunder the cap, rate and clock conditions\. The comparisons in this step use no gain condition\.

On the full admissible class, \([160](https://arxiv.org/html/2610.41074#A3.E160)\) bounds actual risk above and below by fixed multiples ofℒSout\\mathcal\{L\}\_\{S\}^\{\\rm out\}\. This proves \([173](https://arxiv.org/html/2610.41074#A4.E173)\), withaΦ=cF​lΦa\_\{\\Phi\}=c\_\{F\}l\_\{\\Phi\},AΦ=CF​uΦA\_\{\\Phi\}=C\_\{F\}u\_\{\\Phi\}andQΦ=AΦ/aΦQ\_\{\\Phi\}=A\_\{\\Phi\}/a\_\{\\Phi\}\. ∎

###### Proof of Theorem[1](https://arxiv.org/html/2610.41074#Thmtheorem1)\.

Work on the sketch event of Proposition[B\.9](https://arxiv.org/html/2610.41074#Thmtheorem9)under the full admissible class\. For exact\-arithmetic SGD, combine the estimation\-risk bounds \([110](https://arxiv.org/html/2610.41074#A3.E110)\) with the approximation bounds \([69](https://arxiv.org/html/2610.41074#A2.E69)\), using \([38](https://arxiv.org/html/2610.41074#A2.E38)\)\. Substitute \([175](https://arxiv.org/html/2610.41074#A4.E175)\)\. The constants

c0ref=min⁡\{cA,cV​ℓb/a,ℓ2\},C0ref=max⁡\{CAsch,CV​ub/a,2​R\+​u2\}c\_\{0\}^\{\\rm ref\}=\\min\\\{c\_\{A\},c\_\{V\}\\ell\_\{b/a\},\\ell\_\{2\}\\\},\\qquad C\_\{0\}^\{\\rm ref\}=\\max\\\{C\_\{A\}^\{\\rm sch\},C\_\{V\}u\_\{b/a\},2R\_\{\+\}u\_\{2\}\\\}then boundℛ0out\\mathscr\{R\}\_\{0\}^\{\\rm out\}by the corresponding multiples ofM1−b\+B^\+V^M^\{1\-b\}\+\\widehat\{B\}\+\\widehat\{V\}\. For the rounding error, the comparison \([159](https://arxiv.org/html/2610.41074#A3.E159)\) and \([176](https://arxiv.org/html/2610.41074#A4.E176)\) imply

cnative​lN​N^out≤Δout≤\(1\+C13\)​uN​N^out\.c\_\{\\rm native\}l\_\{N\}\\widehat\{N\}^\{\\rm out\}\\leq\\Delta^\{\\rm out\}\\leq\(1\+C\_\{13\}\)u\_\{N\}\\widehat\{N\}^\{\\rm out\}\.The decomposition \([171](https://arxiv.org/html/2610.41074#A4.E171)\) is exactly \([15](https://arxiv.org/html/2610.41074#S3.E15)\) for each output\. These are the first two comparisons in \([17](https://arxiv.org/html/2610.41074#S3.E17)\)\. The decomposition of full risk follows separately from \([38](https://arxiv.org/html/2610.41074#A2.E38)\)\.

∎

### D\.2Constant\-learning\-rate scaling laws

We evaluate the general\-schedule expressions at constant learning rates and precisions, obtaining Corollaries[D\.3](https://arxiv.org/html/2610.41074#Thmtheorem3b)and[D\.4](https://arxiv.org/html/2610.41074#Thmtheorem4b)\. Fixη\>0\\eta\>0,η​g1≤c0\\eta g\_\{1\}\\leq c\_\{0\},ρ=η/B≤ρsch\\rho=\\eta/B\\leq\\rho\_\{\\rm sch\}andτ=η​T∈\[τ∗,Ma\]\\tau=\\eta T\\in\[\\tau\_\{\*\},M^\{a\}\]\. ThenT,τ≥32T,\\tau\\geq 32by \([162](https://arxiv.org/html/2610.41074#A3.E162)\)\. Define

dlast=1,davg=τ1/a−1\.d\_\{\\rm last\}=1,\\qquad d\_\{\\rm avg\}=\\tau^\{1/a\-1\}\.\(177\)The scalardoutd\_\{\\rm out\}describes the output dependence of the variance scales below\. Herec,C\>0c,C\>0have the fixed dependencies of Appendix[C](https://arxiv.org/html/2610.41074#A3)and are uniform inM,T,B,ηM,T,B,\\eta, precisions and output\.

###### Lemma D\.2\.

Withrj=1−η​gjr\_\{j\}=1\-\\eta g\_\{j\}, coefficientsdout,jd\_\{\{\\rm out\},j\}andℓout,j,u\\ell\_\{\{\\rm out\},j,u\}in \([72](https://arxiv.org/html/2610.41074#A3.E72)\) are

dlast,j=rjT,davg,j=rj​\(1−rjT\)τ​gj,\\displaystyle d\_\{\{\\rm last\},j\}=r\_\{j\}^\{T\},\\qquad d\_\{\{\\rm avg\},j\}=\\frac\{r\_\{j\}\(1\-r\_\{j\}^\{T\}\)\}\{\\tau g\_\{j\}\},ℓlast,j,u=rjT−1−u,ℓavg,j,u=1−rjT−uτ​gj\.\\displaystyle\\ell\_\{\{\\rm last\},j,u\}=r\_\{j\}^\{T\-1\-u\},\\qquad\\ell\_\{\{\\rm avg\},j,u\}=\\frac\{1\-r\_\{j\}^\{T\-u\}\}\{\\tau g\_\{j\}\}\.\(178\)In particularℬout≍τ−\(b−1\)/a\\mathcal\{B\}^\{\\rm out\}\\asymp\\tau^\{\-\(b\-1\)/a\}\. For constant precisions, writeΨ\+,F,Ψ\+,B,Ψ\+,M\\Psi\_\{\+,F\},\\Psi\_\{\+,B\},\\Psi\_\{\+,M\}for the three direct terms of \([80](https://arxiv.org/html/2610.41074#A3.E80)\)\. Each component lies between fixed positive multiples of its listed scale\. For𝒱out\\mathcal\{V\}^\{\\rm out\},Ψ\+,Bout\\Psi\_\{\+,B\}^\{\\rm out\}andConvb→fout\\operatorname\{Conv\}\_\{b\\to f\}^\{\\rm out\}, the upper bound isC​KsrefCK\_\{s\}^\{\\rm ref\}times that scale, usings≤Ru≤Ksref​ss\\leq R\_\{u\}\\leq K\_\{s\}^\{\\rm ref\}sfrom \([111](https://arxiv.org/html/2610.41074#A3.E111)\)\.

###### Proof\.

Putk=τ1/a≤Mk=\\tau^\{1/a\}\\leq M\. Geometric summation gives \([178](https://arxiv.org/html/2610.41074#A4.E178)\); \([89](https://arxiv.org/html/2610.41074#A3.E89)\) gives1−rjn≍min⁡\(1,n​η​gj\)1\-r\_\{j\}^\{n\}\\asymp\\min\(1,n\\eta g\_\{j\}\)and two\-sided exponential bounds forrjnr\_\{j\}^\{n\}\. Hencedavg,j≍min⁡\(1,\(τ​gj\)−1\)d\_\{\{\\rm avg\},j\}\\asymp\\min\(1,\(\\tau g\_\{j\}\)^\{\-1\}\)\. Split the bias sum atkk, using\(τ​gj\)2​rj2​T≤1\(\\tau g\_\{j\}\)^\{2\}r\_\{j\}^\{2T\}\\leq 1for the last iterate\. Indicesj∈\[k/2,k\]j\\in\[k/2,k\]give the lower bound \(usej=1j=1ifk<2k<2\):

ℬout≤C⁡\[τ−2​∑j≤kj2​a−b\+∑j\>kj−b\]≤C​k1−b,ℬout≥c​k1−b\.\\mathcal\{B\}^\{\\rm out\}\\leq C\\left\[\\tau^\{\-2\}\\sum\_\{j\\leq k\}j^\{2a\-b\}\+\\sum\_\{j\>k\}j^\{\-b\}\\right\]\\leq Ck^\{1\-b\},\\qquad\\mathcal\{B\}^\{\\rm out\}\\geq ck^\{1\-b\}\.\(179\)The same split gives

∑j≤Mgjℓ−2min\(1,τgj\)2≍τ2−ℓ\+1/a,ℓ=1,2\.\\sum\_\{j\\leq M\}g\_\{j\}^\{\\ell\-2\}\\min\(1,\\tau g\_\{j\}\)^\{2\}\\asymp\\tau^\{2\-\\ell\+1/a\},\\qquad\\ell=1,2\.\(180\)The head and tail are bounded byC​ka⁡\(2−ℓ\)\+1Ck^\{a\(2\-\\ell\)\+1\}andC​τ2​k1−a​ℓC\\tau^\{2\}k^\{1\-a\\ell\}; the indices nearkkgive the lower bound, includingk=Mk=M\. DefineKℓ,out=η​∑u<T∑jgjℓ​ℓout,j,u2K\_\{\\ell,\\rm out\}=\\eta\\sum\_\{u<T\}\\sum\_\{j\}g\_\{j\}^\{\\ell\}\\ell\_\{\{\\rm out\},j,u\}^\{2\}\. Direct summation gives

Kℓ,last=∑jgjℓ−1​1−rj2​T2−η​gj,Kℓ,avg=ητ2​∑n=1T∑jgjℓ−2​\(1−rjn\)2\.\\displaystyle K\_\{\\ell,\\rm last\}=\\sum\_\{j\}g\_\{j\}^\{\\ell\-1\}\\frac\{1\-r\_\{j\}^\{2T\}\}\{2\-\\eta g\_\{j\}\},\\quad K\_\{\\ell,\\rm avg\}=\\frac\{\\eta\}\{\\tau^\{2\}\}\\sum\_\{n=1\}^\{T\}\\sum\_\{j\}g\_\{j\}^\{\\ell\-2\}\(1\-r\_\{j\}^\{n\}\)^\{2\}\.\(181\)Use the two intervalsF=\[⌈T/2⌉,⌈3​T/4⌉\)F=\[\\lceil T/2\\rceil,\\lceil 3T/4\\rceil\)andLT=\[⌈3​T/4⌉,T\)L\_\{T\}=\[\\lceil 3T/4\\rceil,T\), each containing at leastT/8T/8updates\. The split atkkgivesK2,last≤ΘK\_\{2,\\rm last\}\\leq\\ThetaandK1,last≤C​kK\_\{1,\\rm last\}\\leq Ck\. The spectral lower bound \([91](https://arxiv.org/html/2610.41074#A3.E91)\) onFFgivesη​∑u∈FS1H​\(T,u\)≥c​k\\eta\\sum\_\{u\\in F\}S\_\{1\}^\{H\}\(T,u\)\\geq ck\. The first eigenvector givesη​∑u∈LTS2H​\(T,u\)≥c\\eta\\sum\_\{u\\in L\_\{T\}\}S\_\{2\}^\{H\}\(T,u\)\\geq c, sinceη⁡⌊T/4⌋≥τ/8≥4\\eta\\lfloor T/4\\rfloor\\geq\\tau/8\\geq 4\. For the average, \([180](https://arxiv.org/html/2610.41074#A4.E180)\) divided byτ\\taugives the full upper bounds and matching lower bounds onFF\. For source weights0≤wu≤Aw​τξ0\\leq w\_\{u\}\\leq A\_\{w\}\\tau^\{\\xi\}throughout training andwu≥aw​τξw\_\{u\}\\geq a\_\{w\}\\tau^\{\\xi\}onLTL\_\{T\}for last/GG, or onFFotherwise, \([73](https://arxiv.org/html/2610.41074#A3.E73)\) andcJ≤Jj​j≤CJc\_\{J\}\\leq J\_\{jj\}\\leq C\_\{J\}imply

∑u<Thu​wu​ℛuout​\(G\)≍ρ​τξ​dout,∑u<Thu​wu​ℛuout​\(J\)≍ρ​τξ​k\.\\sum\_\{u<T\}h\_\{u\}w\_\{u\}\\mathcal\{R\}\_\{u\}^\{\\rm out\}\(G\)\\asymp\\rho\\tau^\{\\xi\}d\_\{\\rm out\},\\qquad\\sum\_\{u<T\}h\_\{u\}w\_\{u\}\\mathcal\{R\}\_\{u\}^\{\\rm out\}\(J\)\\asymp\\rho\\tau^\{\\xi\}k\.\(182\)Hereξ≥0\\xi\\geq 0andaw,Aw\>0a\_\{w\},A\_\{w\}\>0are fixed, and the comparison constants may depend onaw,Awa\_\{w\},A\_\{w\}\. A common nonnegative prefactor may multiply both weight bounds and conclusions\. Use the residual bound \([111](https://arxiv.org/html/2610.41074#A3.E111)\) and parameter bounds \([112](https://arxiv.org/html/2610.41074#A3.E112)\)–\([113](https://arxiv.org/html/2610.41074#A3.E113)\):Uu≤C​τνU\_\{u\}\\leq C\\tau^\{\\nu\}throughout training andUu≥c​τνU\_\{u\}\\geq c\\tau^\{\\nu\}whenu≥⌈T/4⌉u\\geq\\lceil T/4\\rceil\. Writef=f\+f=f^\{\+\},β=β\+\\beta=\\beta^\{\+\}andm=m\+m=m^\{\+\}\. The causal sources in \([79](https://arxiv.org/html/2610.41074#A3.E79)\) satisfy

c​ρ​β​M​s​τt≤Ztb,\+≤C​Ksref​ρ​β​M​s​τt,\\displaystyle c\\rho\\beta Ms\\tau\_\{t\}\\leq Z\_\{t\}^\{b,\+\}\\leq CK\_\{s\}^\{\\rm ref\}\\rho\\beta Ms\\tau\_\{t\},0≤Ytf,\+≤Cρfτν,Ytf,\+≥cρfτν\(t≥⌈T/4⌉\)\.\\displaystyle 0\\leq Y\_\{t\}^\{f,\+\}\\leq C\\rho f\\tau^\{\\nu\},\\qquad Y\_\{t\}^\{f,\+\}\\geq c\\rho f\\tau^\{\\nu\}\\quad\(t\\geq\\lceil T/4\\rceil\)\.\(183\)ForZb,\+Z^\{b,\+\}, sum the residual bounds againstKJ​JH≍MK\_\{JJ\}^\{H\}\\asymp Mfrom \([96](https://arxiv.org/html/2610.41074#A3.E96)\)\. ForYf,\+Y^\{f,\+\}, telescoping bounds its kernel mass byη​∑u<tS2H​\(t,u\)≤Θ\\eta\\sum\_\{u<t\}S\_\{2\}^\{H\}\(t,u\)\\leq\\Theta\. For its lower bound, sum over the last quarter of updates beforett\. Thereτu≥3​τt/4≥3​τ/16\\tau\_\{u\}\\geq 3\\tau\_\{t\}/4\\geq 3\\tau/16, soUu≥c​τνU\_\{u\}\\geq c\\tau^\{\\nu\}; the same sum using the first eigenvector at horizonttis bounded below sinceτt≥τ/4≥8\\tau\_\{t\}\\geq\\tau/4\\geq 8\. Finally apply \([182](https://arxiv.org/html/2610.41074#A4.E182)\) toRR\(variance and backward\),UU\(forward and same\-update\),Zb,\+Z^\{b,\+\}\(backward\-to\-forward\), andYf,\+Y^\{f,\+\}\(forward\-to\-backward\), multiplying by the coefficients in \([80](https://arxiv.org/html/2610.41074#A3.E80)\)\. Using \([46](https://arxiv.org/html/2610.41074#A2.E46)\) andcg≤θG≤Θc\_\{g\}\\leq\\theta\_\{G\}\\leq\\Thetagives the table\. Under Assumption[C\.1](https://arxiv.org/html/2610.41074#Thmtheorem1b), \([159](https://arxiv.org/html/2610.41074#A3.E159)\) gives the comparison with actual SR\-induced error\. ∎

###### Corollary D\.3\(Constant\-learning\-rate exact\-arithmetic risk\)\.

Assume the model and comparison bounds of Definition[B\.8](https://arxiv.org/html/2610.41074#Thmtheorem8),M≥MwM\\geq M\_\{\\rm w\},Ma≥τ∗M^\{a\}\\geq\\tau\_\{\*\},S∈ΩMS\\in\\Omega\_\{M\}and integerB≥1B\\geq 1\. Ifη\>0\\eta\>0,η​g1≤c0\\eta g\_\{1\}\\leq c\_\{0\},ρ=η/B≤ρsch\\rho=\\eta/B\\leq\\rho\_\{\\rm sch\}andτ=η​T∈\[τ∗,Ma\]\\tau=\\eta T\\in\[\\tau\_\{\*\},M^\{a\}\], both prescribed outputs satisfy

c\(τ−\(b−1\)/a\+ρs\)\\displaystyle c\\bigl\(\\tau^\{\-\(b\-1\)/a\}\+\\rho s\\bigr\)≤E0last≤CKsref\(τ−\(b−1\)/a\+ρs\),\\displaystyle\\leq E\_\{0\}^\{\\rm last\}\\leq CK\_\{s\}^\{\\rm ref\}\\bigl\(\\tau^\{\-\(b\-1\)/a\}\+\\rho s\\bigr\),\(184\)c\(τ−\(b−1\)/a\+ρsτ1/a−1\)\\displaystyle c\\bigl\(\\tau^\{\-\(b\-1\)/a\}\+\\rho s\\tau^\{1/a\-1\}\\bigr\)≤E0avg≤CKsref\(τ−\(b−1\)/a\+ρsτ1/a−1\)\.\\displaystyle\\leq E\_\{0\}^\{\\rm avg\}\\leq CK\_\{s\}^\{\\rm ref\}\\bigl\(\\tau^\{\-\(b\-1\)/a\}\+\\rho s\\tau^\{1/a\-1\}\\bigr\)\.AddingA¯\\bar\{A\}gives full exact\-arithmetic prediction risk\. The exact\-arithmetic residual bound is \([111](https://arxiv.org/html/2610.41074#A3.E111)\), distinct from the sketch\-noise normalizationKsK\_\{s\}\. No rounding\-gain condition is needed\.

###### Proof\.

Theorem[C\.3](https://arxiv.org/html/2610.41074#Thmtheorem3a)bounds exact\-arithmetic by its bias and variance functionals\. Lemma[D\.2](https://arxiv.org/html/2610.41074#Thmtheorem2b)evaluates both, withKsref≥1K\_\{s\}^\{\\rm ref\}\\geq 1explicit in the upper\. Substitutingdlast=1d\_\{\\rm last\}=1anddavg=τ1/a−1d\_\{\\rm avg\}=\\tau^\{1/a\-1\}completes the proof\. ∎

###### Corollary D\.4\(Constant\-learning\-rate scaling law\)\.

Under the hypotheses of Corollary[D\.3](https://arxiv.org/html/2610.41074#Thmtheorem3b), take constant allowedqfw,qbwq\_\{\\mathrm\{fw\}\},q\_\{\\mathrm\{bw\}\}, including exact arithmetic at either location\. Under Assumption[C\.1](https://arxiv.org/html/2610.41074#Thmtheorem1b), the SR\-induced errors satisfy \([20](https://arxiv.org/html/2610.41074#S3.E20)\) and \([21](https://arxiv.org/html/2610.41074#S3.E21)\); adding \([18](https://arxiv.org/html/2610.41074#S3.E18)\) or \([19](https://arxiv.org/html/2610.41074#S3.E19)\) gives full risk\. Within the same cap, rate and clock domain, withCRglobC\_\{R\}^\{\\rm glob\}from \([165](https://arxiv.org/html/2610.41074#A3.E165)\),

\{Γstab≤κ/max\(1,CRglob\)\}⊆\{admissible schedules\}⊆\{Γstab≤κ\}\.\\\{\\Gamma\_\{\\rm stab\}\\leq\\kappa/\\max\(1,C\_\{R\}^\{\\rm glob\}\)\\\}\\subseteq\\\{\\text\{admissible schedules\}\\\}\\subseteq\\\{\\Gamma\_\{\\rm stab\}\\leq\\kappa\\\}\.\(185\)Thus \([22](https://arxiv.org/html/2610.41074#S3.E22)\) is sufficient with

κc=κschmax⁡\(1,CRglob\)\.\\kappa\_\{\\rm c\}=\\frac\{\\kappa\_\{\\rm sch\}\}\{\\max\(1,C\_\{R\}^\{\\rm glob\}\)\}\.\(186\)

###### Proof\.

For constant rates and precisions, the gain definitions \([83](https://arxiv.org/html/2610.41074#A3.E83)\)–\([84](https://arxiv.org/html/2610.41074#A3.E84)\) give

Γstab=ρ​qfw​qbw​τ/M,Γ×=qfw​qbw​η2BM\(s\+τ−\(b−1\)/a\)​∑u<T−1Ru,\\displaystyle\\Gamma\_\{\\rm stab\}=\\rho q\_\{\\mathrm\{fw\}\}q\_\{\\mathrm\{bw\}\}\\tau/M,\\qquad\\Gamma\_\{\\times\}=\\frac\{q\_\{\\mathrm\{fw\}\}q\_\{\\mathrm\{bw\}\}\\eta^\{2\}\}\{BM\(s\+\\tau^\{\-\(b\-1\)/a\}\)\}\\sum\_\{u<T\-1\}R\_\{u\},12​cRglob​Γstab≤Γ×≤CRglob​Γstab,Γf→b≤Cfb​ρ​θG≤κ\.\\displaystyle\\tfrac\{1\}\{2\}c\_\{R\}^\{\\rm glob\}\\Gamma\_\{\\rm stab\}\\leq\\Gamma\_\{\\times\}\\leq C\_\{R\}^\{\\rm glob\}\\Gamma\_\{\\rm stab\},\\qquad\\Gamma\_\{f\\to b\}\\leq C\_\{\\rm fb\}\\rho\\theta\_\{G\}\\leq\\kappa\.\(187\)The residual prefix sum is largest atT−1T\-1; the residual bounds and\(T−1\)/T∈\[1/2,1\)\(T\-1\)/T\\in\[1/2,1\)give the comparison ofΓ×\\Gamma\_\{\\times\}withΓstab\\Gamma\_\{\\rm stab\}\. ForΓf→b\\Gamma\_\{f\\to b\}, the maximum is empty ifqbw=0q\_\{\\mathrm\{bw\}\}=0, and its numerator is zero att=0t=0\. Otherwise write its numerator and denominator asNt,DtN\_\{t\},D\_\{t\}\. Telescoping givesNt≤ρ​qfw​θG​maxu<t​UuN\_\{t\}\\leq\\rho q\_\{\\mathrm\{fw\}\}\\theta\_\{G\}\\max\_\{u<t\}U\_\{u\}\. Forτt≥τ∗\\tau\_\{t\}\\geq\\tau\_\{\*\}, \([119](https://arxiv.org/html/2610.41074#A3.E119)\) givesNt/Dt≤Csrc​ρ​θGN\_\{t\}/D\_\{t\}\\leq C\_\{\\rm src\}\\rho\\theta\_\{G\}; forτt<τ∗\\tau\_\{t\}<\\tau\_\{\*\}, useUu≤Csrc​\(1\+τ∗\)νU\_\{u\}\\leq C\_\{\\rm src\}\(1\+\\tau\_\{\*\}\)^\{\\nu\}andDt≥σ2D\_\{t\}\\geq\\sigma^\{2\}\. The constantCfbC\_\{\\rm fb\}in \([165](https://arxiv.org/html/2610.41074#A3.E165)\) covers both cases, giving

Γf→b​\(T\)≤Cfb​ρ​θG\.\\Gamma\_\{f\\to b\}\(T\)\\leq C\_\{\\rm fb\}\\rho\\theta\_\{G\}\.\(188\)Sinceρsch≤κ/\(Cfb​Θ\)\\rho\_\{\\rm sch\}\\leq\\kappa/\(C\_\{\\rm fb\}\\Theta\), this gain is at mostκ\\kappa\. The bounds forΓ×\\Gamma\_\{\\times\}then prove \([185](https://arxiv.org/html/2610.41074#A4.E185)\) and the stated sufficient threshold\.

Apply \([159](https://arxiv.org/html/2610.41074#A3.E159)\) to Lemma[D\.2](https://arxiv.org/html/2610.41074#Thmtheorem2b)\. The forward\-to\-backward term isρ\\rhotimes the joint scale\. For the last iterate, the joint term is bounded by the forward term usingqbw​τ1/a/M≤1q\_\{\\mathrm\{bw\}\}\\tau^\{1/a\}/M\\leq 1\. For the uniform average, the backward\-to\-forward term is bounded by the backward term usingρ​qfw≤ρsch\\rho q\_\{\\mathrm\{fw\}\}\\leq\\rho\_\{\\rm sch\}\. These multiplicative comparisons include zero rounding coefficients\. Substituting \([177](https://arxiv.org/html/2610.41074#A4.E177)\) yields the SR laws withs⁡\(S\)s\(S\)\. Uses≤s\+τ−\(b−1\)/a≤\(1\+σ−2\)ss\\leq s\+\\tau^\{\-\(b\-1\)/a\}\\leq\(1\+\\sigma^\{\-2\}\)sand Corollary[B\.5](https://arxiv.org/html/2610.41074#Thmtheorem5)to obtain their stated forms withsMs\_\{M\}\. Adding the exact\-arithmetic and approximation bounds completes the full\-risk comparison\. ∎

### D\.3Compute\-optimal risk with fixed precisions

Proposition[D\.5](https://arxiv.org/html/2610.41074#Thmtheorem5b)proves the compute orders in Table[1](https://arxiv.org/html/2610.41074#S3.T1)by balancing width and training duration\. Use the cost \([23](https://arxiv.org/html/2610.41074#S3.E23)\) and Corollary[D\.4](https://arxiv.org/html/2610.41074#Thmtheorem4b)\. Precision is fixed within each run and across budgets\. Exact precision is allowed, with4−∞=04^\{\-\\infty\}=0\. For a fixed precision pair writeW=c⁡\(pfw\)\+c⁡\(pbw\)\>0W=c\(p\_\{\\mathrm\{fw\}\}\)\+c\(p\_\{\\mathrm\{bw\}\}\)\>0; thenf=\(W/ρ\)​M​τf=\(W/\\rho\)M\\tau, whereτ=η​T\\tau=\\eta Tandρ=η/B\\rho=\\eta/B\.

###### Proposition D\.5\(Fixed\-precision compute orders\)\.

FixB∈ℕB\\in\\mathbb\{N\},0<η≤min⁡\{c0/Cg,B​ρsch\}0<\\eta\\leq\\min\\\{c\_\{0\}/C\_\{g\},B\\rho\_\{\\rm sch\}\\\}and an allowed precision pair independently ofFF\. For all sufficiently largeFF, the optimal uniformly averaged risk in the full constant\-rate class withf≤Ff\\leq Fhas the order in Table[1](https://arxiv.org/html/2610.41074#S3.T1)\.

###### Proof\.

Putβ=\(b−1\)/a\\beta=\(b\-1\)/a\. The bias lower bound is a fixed multiple ofτ−β\\tau^\{\-\\beta\}\. Ifqbw=0q\_\{\\mathrm\{bw\}\}=0, the windowτ≤Ma\\tau\\leq M^\{a\}, budget and rate ceiling give

F≥\(W/ρ\)​τ1\+1/a≥\(W/ρsch\)​τ1\+1/a,F\\geq\(W/\\rho\)\\tau^\{1\+1/a\}\\geq\(W/\\rho\_\{\\rm sch\}\)\\tau^\{1\+1/a\},\(189\)proving the first lower bound\. For backward\-only rounding, the constant\-rate law gives

ℛavg≳τ−β\+ρ​qbw​σ2​τ1/aM≳τ−β\+W​qbw​σ2F​τ1\+1/a\.\\mathscr\{R\}^\{\\rm avg\}\\gtrsim\\tau^\{\-\\beta\}\+\\frac\{\\rho q\_\{\\mathrm\{bw\}\}\\sigma^\{2\}\\tau^\{1/a\}\}\{M\}\\gtrsim\\tau^\{\-\\beta\}\+\\frac\{Wq\_\{\\mathrm\{bw\}\}\\sigma^\{2\}\}\{F\}\\tau^\{1\+1/a\}\.\(190\)Splitting atτ=Fa/\(a\+b\)\\tau=F^\{a/\(a\+b\)\}proves the second lower bound\. When both scales are positive, the necessary stability\-gain condition isρ​qfw​qbw​τ/M≤κsch\\rho q\_\{\\mathrm\{fw\}\}q\_\{\\mathrm\{bw\}\}\\tau/M\\leq\\kappa\_\{\\rm sch\}\. Thus

F≥\(W​qfw​qbw/κsch\)​τ2,F\\geq\(Wq\_\{\\mathrm\{fw\}\}q\_\{\\mathrm\{bw\}\}/\\kappa\_\{\\rm sch\}\)\\tau^\{2\},\(191\)which yields the third lower bound\.

For the three upper bounds, respectively setr=1,b,ar=1,b,a\. Choose a fixed largeA≥1A\\geq 1, also satisfyingA≥ρ​qfw​qbw/κcA\\geq\\rho q\_\{\\mathrm\{fw\}\}q\_\{\\mathrm\{bw\}\}/\\kappa\_\{\\rm c\}in the third case, and take

TF=⌊d​Fa/\(a\+r\)⌋,τF=η​TF,MF=⌈A​τFr/a⌉\.T\_\{F\}=\\left\\lfloor dF^\{a/\(a\+r\)\}\\right\\rfloor,\\qquad\\tau\_\{F\}=\\eta T\_\{F\},\\qquad M\_\{F\}=\\left\\lceil A\\tau\_\{F\}^\{r/a\}\\right\\rceil\.A sufficiently small fixedd\>0d\>0makesf≤Ff\\leq Feventually\. Both width thresholds and the minimum cumulative learning rate are met; the cap and rate ceiling were fixed above, andMF≥τF1/aM\_\{F\}\\geq\\tau\_\{F\}^\{1/a\}verifies the clock window\. The mixed gain is zero in the first two cases and at mostκc\\kappa\_\{\\rm c\}in the third, so \([22](https://arxiv.org/html/2610.41074#S3.E22)\) verifies all feedback bounds\.

It remains to check risk\. Approximation error, bias and exact\-SGD variance areO⁡\(τF−β\)O\(\\tau\_\{F\}^\{\-\\beta\}\), sincer≥1r\\geq 1anda\>ba\>b\. Forward rounding contributes at mostO⁡\(τF−β\)O\(\\tau\_\{F\}^\{\-\\beta\}\)\. Backward rounding is absent in the first case, isO⁡\(τF−β\)O\(\\tau\_\{F\}^\{\-\\beta\}\)in the second, and isO⁡\(τF\(1−a\)/a\)=O⁡\(τF−β\)O\(\\tau\_\{F\}^\{\(1\-a\)/a\}\)=O\(\\tau\_\{F\}^\{\-\\beta\}\)in the third\. The third case’s joint term isO⁡\(τF\(2−a−b\)/a\)=O⁡\(τF−β\)O\(\\tau\_\{F\}^\{\(2\-a\-b\)/a\}\)=O\(\\tau\_\{F\}^\{\-\\beta\}\)\. Corollary[D\.4](https://arxiv.org/html/2610.41074#Thmtheorem4b)therefore gives all upper bounds, withτF≍Fa/\(a\+r\)\\tau\_\{F\}\\asymp F^\{a/\(a\+r\)\}\. ∎

### D\.4Compute\-optimal scaling with optimized learning rate and batch size

Proposition[D\.5](https://arxiv.org/html/2610.41074#Thmtheorem5b)fixes the learning rate and batch size across budgets\. We now optimize both as functions ofFF, while keeping them constant within each run\. Uniform averaging has Table[1](https://arxiv.org/html/2610.41074#S3.T1)’s risk exponents; the last\-iterate result also accounts for reducing the SGD\-noise contribution\.

Use the same fixed model with widths satisfying \([158](https://arxiv.org/html/2610.41074#A3.E158)\)\. Fix one allowed precision pair independently ofFF, and writeβ0=\(b−1\)/a\\beta\_\{0\}=\(b\-1\)/aandWp=c⁡\(pfw\)\+c⁡\(pbw\)\>0W\_\{p\}=c\(p\_\{\\mathrm\{fw\}\}\)\+c\(p\_\{\\mathrm\{bw\}\}\)\>0\. A configurationπ\\pispecifies integersM,T,B≥1M,T,B\\geq 1and a constant rateη\>0\\eta\>0, all chosen before the random draws\. Let𝒞p​\(F\)\\mathcal\{C\}\_\{p\}\(F\)contain those with nonemptyΩM\\Omega\_\{M\}, both width conditions, the full constant\-schedule assumptions of Corollary[D\.4](https://arxiv.org/html/2610.41074#Thmtheorem4b)for everyS∈ΩMS\\in\\Omega\_\{M\}, andf=B​T​M​Wp=\(Wp/ρ\)​M​τ≤Ff=BTMW\_\{p\}=\(W\_\{p\}/\\rho\)M\\tau\\leq F\. All three feedback bounds in Assumption[C\.1](https://arxiv.org/html/2610.41074#Thmtheorem1b)remain required\.

Belowc,C,F0\>0c,C,F\_\{0\}\>0depend only on the fixed model, comparison bounds, admissibility thresholds and precision pair, includingKs,KsrefK\_\{s\},K\_\{s\}^\{\\rm ref\}\. They are uniform inM,T,B,η,SM,T,B,\\eta,S, but need not remain uniform as a positive rounding scale tends to zero\. Statements hold forF≥F0F\\geq F\_\{0\}\.

###### Proposition D\.6\(Averaged compute\-optimal risk\)\.

Letγavg\\gamma\_\{\\rm avg\}be the risk exponent in the corresponding row of Table[1](https://arxiv.org/html/2610.41074#S3.T1)\. Everyπ∈𝒞p​\(F\)\\pi\\in\\mathcal\{C\}\_\{p\}\(F\)satisfiesℛSavg​\(π\)≥c​F−γavg\\mathscr\{R\}\_\{S\}^\{\\rm avg\}\(\\pi\)\\geq cF^\{\-\\gamma\_\{\\rm avg\}\}onΩM⁡\(π\)\\Omega\_\{M\(\\pi\)\}\. Conversely, a deterministic configurationπF∈𝒞p​\(F\)\\pi\_\{F\}\\in\\mathcal\{C\}\_\{p\}\(F\)attainsℛSavg​\(πF\)≤C​F−γavg\\mathscr\{R\}\_\{S\}^\{\\rm avg\}\(\\pi\_\{F\}\)\\leq CF^\{\-\\gamma\_\{\\rm avg\}\}uniformly on its event\. Thus optimizing the constant learning rate and batch size leaves Table[1](https://arxiv.org/html/2610.41074#S3.T1)’s optimal risk orders unchanged\.

###### Proof\.

The lower\-bound argument of Proposition[D\.5](https://arxiv.org/html/2610.41074#Thmtheorem5b)is uniform in the actual admissibleρ\\rho\. Forqbw=0q\_\{\\mathrm\{bw\}\}=0, \([189](https://arxiv.org/html/2610.41074#A4.E189)\) uses only its fixed ceiling\. For backward\-only rounding, \([190](https://arxiv.org/html/2610.41074#A4.E190)\) eliminatesρ\\rhousing the cost identity; for joint rounding, \([191](https://arxiv.org/html/2610.41074#A4.E191)\) eliminates it using the necessary stability gain\. Together with the common bias lower bound, these same inequalities prove all three lower orders even when the constant rate and batch vary withFF\.

For the uppers, fixB=1B=1and any sufficiently small fixedη0≤min⁡\{c0/Cg,ρsch\}\\eta\_\{0\}\\leq\\min\\\{c\_\{0\}/C\_\{g\},\\rho\_\{\\rm sch\}\\\}\. The integer policies of Proposition[D\.5](https://arxiv.org/html/2610.41074#Thmtheorem5b)belong to𝒞p​\(F\)\\mathcal\{C\}\_\{p\}\(F\)and attain precisely the three orders in Table[1](https://arxiv.org/html/2610.41074#S3.T1)\. Finally,f≥Wp​Mf\\geq W\_\{p\}Mleaves only finitely many budget\-feasible widths\. The pointwise lower and uniform attainer prove the infimum statement without a joint\-width event or any coupling of sketches across widths\. ∎

###### Proposition D\.7\(Last\-iterate compute orders with variable constant rate\)\.

For the same fixed precisions and class𝒞p​\(F\)\\mathcal\{C\}\_\{p\}\(F\), the preceding pointwise lower, uniform integer\-attainment and conditional\-infimum conclusions hold for the last iterate with exponent

γlast=\{\(b−1\)/\(a\+b\),qfw=0,\(b−1\)/\(2​a\),qfw\>0\.\\gamma\_\{\\rm last\}=\\begin\{cases\}\(b\-1\)/\(a\+b\),&q\_\{\\mathrm\{fw\}\}=0,\\\\ \(b\-1\)/\(2a\),&q\_\{\\mathrm\{fw\}\}\>0\.\\end\{cases\}\(192\)Both exact and backward\-only arithmetic have the first exponent; forward\-only and joint rounding have the second\. Constants have the same fixed\-dependence and nonuniformity convention as above\. This statement optimizes constant\-within\-run rates and batches, not arbitrary learning\-rate schedules or iterate weights\.

###### Proof\.

The exact\-arithmetic SGD risk lower bound and capacity/cost inequalities give

ℛlast≥c⁡\{τ−β0\+ρ​σ2\},ρ≥Wp​M​τF≥WpF​τ1\+1/a\.\\mathscr\{R\}^\{\\rm last\}\\geq c\\\{\\tau^\{\-\\beta\_\{0\}\}\+\\rho\\sigma^\{2\}\\\},\\qquad\\rho\\geq\\frac\{W\_\{p\}M\\tau\}\{F\}\\geq\\frac\{W\_\{p\}\}\{F\}\\tau^\{1\+1/a\}\.\(193\)Splitting atτ=Fa/\(a\+b\)\\tau=F^\{a/\(a\+b\)\}proves the first lower order\. Whenqfw\>0q\_\{\\mathrm\{fw\}\}\>0, the direct forward term also gives

ℛlast≥c​τ−β0\+c′​ρ​qfwM​τ1−β0≥c​τ−β0\+c′​Wp​qfwF​τ2−β0\.\\mathscr\{R\}^\{\\rm last\}\\geq c\\tau^\{\-\\beta\_\{0\}\}\+c^\{\\prime\}\\frac\{\\rho q\_\{\\mathrm\{fw\}\}\}\{M\}\\tau^\{1\-\\beta\_\{0\}\}\\geq c\\tau^\{\-\\beta\_\{0\}\}\+c^\{\\prime\}\\frac\{W\_\{p\}q\_\{\\mathrm\{fw\}\}\}\{F\}\\tau^\{2\-\\beta\_\{0\}\}\.\(194\)Here0<β0<10<\\beta\_\{0\}<1, since1<b<a1<b<a\. Splitting atτ=F1/2\\tau=F^\{1/2\}proves the second lower order, including the forward\-only case in which the mixed gain vanishes\.

We give one integer construction for both upper orders\. Setr=br=bifqfw=0q\_\{\\mathrm\{fw\}\}=0andr=ar=aotherwise\. Withκc\\kappa\_\{\\rm c\}from \([186](https://arxiv.org/html/2610.41074#A4.E186)\), choose a fixedd\>0d\>0satisfying

d≤12​min⁡\{1,ρsch,c0/Cg,κc/max⁡\(1,qfw​qbw\)\}\.d\\leq\\tfrac\{1\}\{2\}\\min\\left\\\{1,\\rho\_\{\\rm sch\},c\_\{0\}/C\_\{g\},\\kappa\_\{\\rm c\}/\\max\(1,q\_\{\\mathrm\{fw\}\}q\_\{\\mathrm\{bw\}\}\)\\right\\\}\.\(195\)Before any random draws, choose

MF=⌊\(d​F2​Wp\)1/\(a\+r\)⌋,BF=1,ηF=d​MF1−r,TF=⌊MFaηF⌋\.M\_\{F\}=\\left\\lfloor\\left\(\\frac\{dF\}\{2W\_\{p\}\}\\right\)^\{1/\(a\+r\)\}\\right\\rfloor,\\quad B\_\{F\}=1,\\quad\\eta\_\{F\}=dM\_\{F\}^\{1\-r\},\\quad T\_\{F\}=\\left\\lfloor\\frac\{M\_\{F\}^\{a\}\}\{\\eta\_\{F\}\}\\right\\rfloor\.\(196\)For sufficiently largeFF,MFM\_\{F\}exceeds the width conditions and has a nonempty event, andMFa/2≤τF=ηF​TF≤MFaM\_\{F\}^\{a\}/2\\leq\\tau\_\{F\}=\\eta\_\{F\}T\_\{F\}\\leq M\_\{F\}^\{a\}verifies the lower and upper bounds on cumulative learning rate\. The event boundg1≤Cgg\_\{1\}\\leq C\_\{g\}and \([195](https://arxiv.org/html/2610.41074#A4.E195)\) verify the step and rate caps\. Ifqfw=0q\_\{\\mathrm\{fw\}\}=0, all mixed terms vanish\. Otherwiser=ar=aandΓstab≤κc\\Gamma\_\{\\rm stab\}\\leq\\kappa\_\{\\rm c\}whereΓstab=ρF​qfw​qbw​τF/MF\.\\Gamma\_\{\\rm stab\}=\\rho\_\{F\}q\_\{\\mathrm\{fw\}\}q\_\{\\mathrm\{bw\}\}\\tau\_\{F\}/M\_\{F\}\.The verifications in \([185](https://arxiv.org/html/2610.41074#A4.E185)\) hold for everyS∈ΩMFS\\in\\Omega\_\{M\_\{F\}\}\. Moreover,fF=Wp​MF​TF≤Wpd​MFa\+r≤F/2\.f\_\{F\}=W\_\{p\}M\_\{F\}T\_\{F\}\\leq\\frac\{W\_\{p\}\}\{d\}M\_\{F\}^\{a\+r\}\\leq F/2\.

At matched clock, approximation and bias have orderMF1−bM\_\{F\}^\{1\-b\};σ2≤s⁡\(S\)≤Ks​σ2\\sigma^\{2\}\\leq s\(S\)\\leq K\_\{s\}\\sigma^\{2\}ands⁡\(S\)\+τF−β0≤Cs\(S\)\+\\tau\_\{F\}^\{\-\\beta\_\{0\}\}\\leq Cuniformly\. Forr=br=bthe exact\-arithmetic minibatch\-noise and backward\-rounding terms areO⁡\(ρF\)=O⁡\(MF1−b\)O\(\\rho\_\{F\}\)=O\(M\_\{F\}^\{1\-b\}\), and forward/joint errors are zero\. Forr=ar=athe forward error isO⁡\(ρF​qfw​MFa−b\)=O⁡\(MF1−b\)O\(\\rho\_\{F\}q\_\{\\mathrm\{fw\}\}M\_\{F\}^\{a\-b\}\)=O\(M\_\{F\}^\{1\-b\}\); the exact\-arithmetic minibatch\-noise and backward\-rounding terms areO⁡\(MF1−a\)O\(M\_\{F\}^\{1\-a\}\)\. The last\-iterate joint error isO⁡\(ρF2​qfw​qbw​MFa−1\)=O⁡\(MF1−a\)O\(\\rho\_\{F\}^\{2\}q\_\{\\mathrm\{fw\}\}q\_\{\\mathrm\{bw\}\}M\_\{F\}^\{a\-1\}\)=O\(M\_\{F\}^\{1\-a\}\)\. Becausea\>ba\>b, all terms areO⁡\(MF1−b\)O\(M\_\{F\}^\{1\-b\}\)\. Corollary[D\.4](https://arxiv.org/html/2610.41074#Thmtheorem4b)therefore givesℛSlast\(πF\)≤CMF1−b≍F−\(b−1\)/\(a\+r\)\\mathscr\{R\}\_\{S\}^\{\\rm last\}\(\\pi\_\{F\}\)\\leq CM\_\{F\}^\{1\-b\}\\asymp F^\{\-\(b\-1\)/\(a\+r\)\}uniformly on the event\. The conditional\-infimum statement follows as in the preceding proof\. ∎

## Appendix EImplications for precision and training design

This appendix derives the design consequences discussed in Section[4](https://arxiv.org/html/2610.41074#S4)\. Proposition[E\.1](https://arxiv.org/html/2610.41074#Thmtheorem1d)compares rounding locations and outputs; Propositions[E\.2](https://arxiv.org/html/2610.41074#Thmtheorem2c)–[E\.4](https://arxiv.org/html/2610.41074#Thmtheorem4c)address precision allocation across training; and Corollary[E\.5](https://arxiv.org/html/2610.41074#Thmtheorem5c)gives bit requirements for a prescribed risk tolerance\. Lemma[E\.6](https://arxiv.org/html/2610.41074#Thmtheorem6c)and Theorem[E\.7](https://arxiv.org/html/2610.41074#Thmtheorem7c)develop learning\-rate schedules, while Proposition[E\.8](https://arxiv.org/html/2610.41074#Thmtheorem8a)and Theorem[E\.9](https://arxiv.org/html/2610.41074#Thmtheorem9a)establish the compute consequences of growing finite precisions and allowing exact arithmetic\. Only the last iterate and uniform average are considered with Assumption[C\.1](https://arxiv.org/html/2610.41074#Thmtheorem1b)\. Use the constant\-schedule notation of Section[3\.2](https://arxiv.org/html/2610.41074#S3.SS2)\. Corollary[B\.5](https://arxiv.org/html/2610.41074#Thmtheorem5)justifies replacing sketch\-dependent noise by its deterministic scale, including when the bias term is added\.

### E\.1Where: rounding location and iterate averaging

Proposition[E\.1](https://arxiv.org/html/2610.41074#Thmtheorem1d)compares the two outputs and identifies which feature computation is more sensitive to rounding\. Fix commonM,η,B,TM,\\eta,B,Tandq=4−p\>0q=4^\{\-p\}\>0,p∈ℕ0p\\in\\mathbb\{N\}\_\{0\}, such that both rounding choices\(qfw,qbw\)=\(q,0\)\(q\_\{\\mathrm\{fw\}\},q\_\{\\mathrm\{bw\}\}\)=\(q,0\)and\(0,q\)\(0,q\)satisfy Corollary[D\.4](https://arxiv.org/html/2610.41074#Thmtheorem4b)\.

###### Proposition E\.1\(Comparisons with rounding at one location\)\.

The two outputs satisfy

Δlast​\(q,0\)Δavg​\(q,0\)≍τ\(a−1\)/a,Δlast​\(0,q\)≍Δavg​\(0,q\)\.\\frac\{\\Delta^\{\\rm last\}\(q,0\)\}\{\\Delta^\{\\rm avg\}\(q,0\)\}\\asymp\\tau^\{\(a\-1\)/a\},\\qquad\\Delta^\{\\rm last\}\(0,q\)\\asymp\\Delta^\{\\rm avg\}\(0,q\)\.\(197\)Ifcτ​Ma≤τ≤Mac\_\{\\tau\}M^\{a\}\\leq\\tau\\leq M^\{a\}for fixedcτ∈\(0,1\]c\_\{\\tau\}\\in\(0,1\], \([24](https://arxiv.org/html/2610.41074#S4.E24)\) holds\. Constants in comparisons at this matched clock may also depend oncτc\_\{\\tau\}\. For backward\-only rounding,

Δlast​\(0,qbw\)ℛ0last​\(T\)≲qbw​τ1/aM≤qbw\.\\frac\{\\Delta^\{\\rm last\}\(0,q\_\{\\mathrm\{bw\}\}\)\}\{\\mathscr\{R\}\_\{0\}^\{\\rm last\}\(T\)\}\\lesssim q\_\{\\mathrm\{bw\}\}\\frac\{\\tau^\{1/a\}\}\{M\}\\leq q\_\{\\mathrm\{bw\}\}\.\(198\)At a fixed\-factor matched clock, the averaged counterpart is

Δavg​\(0,qbw\)ℛ0avg​\(T\)≍ρ​qbw​\(sM\+M1−b\)M1−b\+ρ​sM​M1−a\.\\frac\{\\Delta^\{\\rm avg\}\(0,q\_\{\\mathrm\{bw\}\}\)\}\{\\mathscr\{R\}\_\{0\}^\{\\rm avg\}\(T\)\}\\asymp\\frac\{\\rho q\_\{\\mathrm\{bw\}\}\(s\_\{M\}\+M^\{1\-b\}\)\}\{M^\{1\-b\}\+\\rho s\_\{M\}M^\{1\-a\}\}\.\(199\)

###### Proof\.

Set the unused rounding scale to zero in \([20](https://arxiv.org/html/2610.41074#S3.E20)\) and \([21](https://arxiv.org/html/2610.41074#S3.E21)\)\. Dividing the forward\-only laws across the two outputs gives the first comparison in \([197](https://arxiv.org/html/2610.41074#A5.E197)\); the backward\-only expressions coincide up to fixed factors\. Dividing the two location laws and then substitutingcτ​Ma≤τ≤Mac\_\{\\tau\}M^\{a\}\\leq\\tau\\leq M^\{a\}andsM\+M1−b≍σ2\+M1−bs\_\{M\}\+M^\{1\-b\}\\asymp\\sigma^\{2\}\+M^\{1\-b\}proves \([24](https://arxiv.org/html/2610.41074#S4.E24)\)\.

For \([198](https://arxiv.org/html/2610.41074#A5.E198)\), the exact\-arithmetic SGD risk lower bound in Corollary[D\.3](https://arxiv.org/html/2610.41074#Thmtheorem3b)is a fixed multiple ofτ−\(b−1\)/a\+ρsM\\tau^\{\-\(b\-1\)/a\}\+\\rho s\_\{M\}\. The quotient

ρ\(sM\+τ−\(b−1\)/a\)τ−\(b−1\)/a\+ρsM≤max⁡\{1,ρ\}≤max⁡\{1,ρsch\}\\frac\{\\rho\(s\_\{M\}\+\\tau^\{\-\(b\-1\)/a\}\)\}\{\\tau^\{\-\(b\-1\)/a\}\+\\rho s\_\{M\}\}\\leq\\max\\\{1,\\rho\\\}\\leq\\max\\\{1,\\rho\_\{\\rm sch\}\\\}proves the first inequality, andτ≤Ma\\tau\\leq M^\{a\}proves the second\. Finally, adding the approximation term to \([184](https://arxiv.org/html/2610.41074#A4.E184)\) and evaluating at the matched clock gives the denominator in \([199](https://arxiv.org/html/2610.41074#A5.E199)\)\. ∎

### E\.2When: allocating precision across updates

We first quantify sensitivity to rounding at each update in Proposition[E\.2](https://arxiv.org/html/2610.41074#Thmtheorem2c), then allocate a precision budget in Propositions[E\.3](https://arxiv.org/html/2610.41074#Thmtheorem3c)and[E\.4](https://arxiv.org/html/2610.41074#Thmtheorem4c)\. Fix the width, batch, horizon, deterministic learning rates and output\. Keep one rounding location exact throughout training and allocate finite precisions at the other\. Use the coefficients and scalar objective in \([25](https://arxiv.org/html/2610.41074#S4.E25)\); for nonconstant learning rates, replaceη2/B\\eta^\{2\}/Bthere byhuh\_\{u\}\. Thus the allocation uses the exact\-arithmetic source and output responses already defined in Section[3\.1](https://arxiv.org/html/2610.41074#S3.SS1)\.

###### Proposition E\.2\(Single\-location sensitivity and timing\)\.

For either output, \([25](https://arxiv.org/html/2610.41074#S4.E25)\) holds with the replacement above\. The coefficients are nonnegative and independent of the allocated precisions\. At constant positive learning rate, the backward coefficientsAbw,uA\_\{\\mathrm\{bw\},u\}are nondecreasing inuufor the last iterate and nonincreasing inuufor the uniform average\. Thus the real\-bit optimizer of Proposition[E\.3](https://arxiv.org/html/2610.41074#Thmtheorem3c)has the corresponding ordering of backward precisions\.

###### Proof\.

Setting one precision scale to zero throughout \([15](https://arxiv.org/html/2610.41074#S3.E15)\) removes every joint term and the direct term at that location\. The remaining expression is \([25](https://arxiv.org/html/2610.41074#S4.E25)\); apply Theorem[1](https://arxiv.org/html/2610.41074#Thmtheorem1)for its risk comparison\. For the last iterate, the exponent1/a−1<01/a\-1<0in \([11](https://arxiv.org/html/2610.41074#S3.E11)\) makesK1last​\(u\)K\_\{1\}^\{\\rm last\}\(u\)increase as the remaining cumulative rate decreases\. For the uniform average, reindex its double sum byi=v−u−1i=v\-u\-1andj=w−u−1j=w\-u\-1\. At constant rate it is

K1avg​\(u\)=1T2​∑i,j=0T−u−1\[1\+η⁡\(i\+j\)\]1/a−1\.K\_\{1\}^\{\\rm avg\}\(u\)=\\frac\{1\}\{T^\{2\}\}\\sum\_\{i,j=0\}^\{T\-u\-1\}\[1\+\\eta\(i\+j\)\]^\{1/a\-1\}\.Increasinguuremoves nonnegative terms\. The prefactorhu​σ2/\(2​M\)h\_\{u\}\\sigma^\{2\}/\(2M\)is constant, proving the two coefficient orderings\. The optimizer is a nondecreasing function of each coefficient, which proves its precision ordering\. Forward coefficients also contain the changing parameter sourceU^u\\widehat\{U\}\_\{u\}, so this argument does not assert the same monotonicity for them\. ∎

#### E\.2\.1A shared precision budget

With one exact computation and one finite\-precision computation, the cost \([23](https://arxiv.org/html/2610.41074#S3.E23)\) is equivalent to

∑u<Tpr,u≤P,P=\(pref\+p0\)​FB​M−T⁡\(pref\+2​p0\)\.\\sum\_\{u<T\}p\_\{r,u\}\\leq P,\\qquad P=\\frac\{\(p\_\{\\rm ref\}\+p\_\{0\}\)F\}\{BM\}\-T\(p\_\{\\rm ref\}\+2p\_\{0\}\)\.The fixed term charges the exact computation and the sign/exponent costs\. Feasibility requiresP≥0P\\geq 0\. Zero mantissa bits meanq=1q=1; they do not mean exact arithmetic\. Exact arithmetic hasq=0q=0and is charged separately by the compute definition\.

WriteN^​\(p\)=∑iAi​4−pi\\widehat\{N\}\(p\)=\\sum\_\{i\}A\_\{i\}4^\{\-p\_\{i\}\}for \([25](https://arxiv.org/html/2610.41074#S4.E25)\), withAi≥0A\_\{i\}\\geq 0and budgetP\>0P\>0\. Letnncount the positive coefficients and, whenn\>0n\>0, setGM\(A\)=\(∏i:Ai\>0Ai\)1/n\\operatorname\{GM\}\(A\)=\(\\prod\_\{i:A\_\{i\}\>0\}A\_\{i\}\)^\{1/n\}\.

###### Proposition E\.3\(Optimal allocation for an additive SR\-error objective\)\.

Ifn\>0n\>0, the unique real minimizer underpi≥0p\_\{i\}\\geq 0and∑ipi≤P\\sum\_\{i\}p\_\{i\}\\leq Pis \([26](https://arxiv.org/html/2610.41074#S4.E26)\), withiireplacing\(r,u\)\(r,u\)andλ\>0\\lambda\>0determined by∑ipi=P\\sum\_\{i\}p\_\{i\}=P\. If every coefficient is positive and every optimalpi\>0p\_\{i\}\>0, this gives

pi=Pn\+log4⁡AiGM⁡\(A\),∑iAi​4−pi∑iAi4−P/n=GM⁡\(A\)n−1​∑iAi\.p\_\{i\}=\\frac\{P\}\{n\}\+\\log\_\{4\}\\frac\{A\_\{i\}\}\{\\operatorname\{GM\}\(A\)\},\\qquad\\frac\{\\sum\_\{i\}A\_\{i\}4^\{\-p\_\{i\}\}\}\{\\sum\_\{i\}A\_\{i\}4^\{\-P/n\}\}=\\frac\{\\operatorname\{GM\}\(A\)\}\{n^\{\-1\}\\sum\_\{i\}A\_\{i\}\}\.\(200\)For integer mantissas, use budget⌊P⌋\\lfloor P\\rfloorand assign each successive bit where it decreasesN^\\widehat\{N\}most; this is optimal when all nonnegative integers are allowed\. Flooring a feasible real allocation increasesN^\\widehat\{N\}by a factor of at most four\.

On a nonempty admissible family with uniform boundscΔ​N^≤Δ≤CΔ​N^c\_\{\\Delta\}\\widehat\{N\}\\leq\\Delta\\leq C\_\{\\Delta\}\\widehat\{N\},0<cΔ≤CΔ0<c\_\{\\Delta\}\\leq C\_\{\\Delta\}, an exact scalar minimizer achieves SR\-induced error within a factorCΔ/cΔC\_\{\\Delta\}/c\_\{\\Delta\}of the minimum in that family\. The same factor bounds full risk when exact\-arithmetic risk is common\.

###### Proof\.

On positive\-weight coordinates the objective is strictly convex and strictly decreasing\. Consequently its minimizer exhausts the budget\. The optimality conditions giveAi​4−pi=λA\_\{i\}4^\{\-p\_\{i\}\}=\\lambdawhenpi\>0p\_\{i\}\>0andAi≤λA\_\{i\}\\leq\\lambdawhenpi=0p\_\{i\}=0, which yields \([26](https://arxiv.org/html/2610.41074#S4.E26)\)\. The sum∑i:Ai\>0\[log4\(Ai/λ\)\]\+\\sum\_\{i:A\_\{i\}\>0\}\[\\log\_\{4\}\(A\_\{i\}/\\lambda\)\]\_\{\+\}decreases continuously from infinity to zero and is strictly decreasing when positive, proving uniqueness of the multiplier and the allocation\. Zero\-weight coordinates receive no bits\. If the lower constraints are inactive, summing the logarithms yields \([200](https://arxiv.org/html/2610.41074#A5.E200)\), with minimumnGM\(A\)4−P/nn\\operatorname\{GM\}\(A\)4^\{\-P/n\}\.

The gain from increasing an integer mantissa fromkktok\+1k\+1is3​Ai​4−\(k\+1\)3A\_\{i\}4^\{\-\(k\+1\)\}\. These gains decrease along each coordinate\. Selecting the largest remaining gain at each step therefore selects the largestPPgains overall, with all predecessor gains already selected; this proves integer optimality\. Since4−⌊pi⌋≤4 4−pi4^\{\-\\lfloor p\_\{i\}\\rfloor\}\\leq 4\\,4^\{\-p\_\{i\}\}, flooring increases the objective by at most four\.

Finally, for any feasible comparatorppand an objective minimizerp∗p\_\{\*\},Δ⁡\(p∗\)≤CΔ​N^​\(p∗\)≤CΔ​N^​\(p\)≤\(CΔ/cΔ\)​Δ​\(p\)\\Delta\(p\_\{\*\}\)\\leq C\_\{\\Delta\}\\widehat\{N\}\(p\_\{\*\}\)\\leq C\_\{\\Delta\}\\widehat\{N\}\(p\)\\leq\(C\_\{\\Delta\}/c\_\{\\Delta\}\)\\Delta\(p\)\. Taking the infimum proves the claim\. Adding the common nonnegative exact\-arithmetic risk preserves the same factor\. The boundary caseP=0P=0forces zero allocation; if every coefficient vanishes, zero allocation is optimal\. ∎

This proposition optimizes the scalar comparison expression, rather than identifying an exact minimizer of actual binary\-SR risk\. With one location exact, all joint feedback gains vanish\. Remaining rate, width and clock assumptions still apply\. Restricted precision menus require a separate feasible search rather than unrestricted integer greedy allocation\.

#### E\.2\.2Joint precision allocation across training phases

We minimize the joint SR\-error objective over predetermined training phases and a finite precision menu, subject to the compute budget and a sufficient feedback test\. The guarantee compares schedules within this same feasible family\.

Fix eligible widthMM, batchBB, horizonTTand deterministic learning rates, and use either the last iterate or the uniformly averaged iterate\. Require the pointwise cap, rate ceiling and clock conditions of Assumption[C\.1](https://arxiv.org/html/2610.41074#Thmtheorem1b)throughoutΩM\\Omega\_\{M\}, with the same fixed comparison bounds and both width conditions\. All quantities below are chosen before the random draws\. UseU^\\widehat\{U\},KroutK\_\{r\}^\{\{\\rm out\}\}andhuh\_\{u\}from Section[3](https://arxiv.org/html/2610.41074#S3), and the direct coefficientsAr,uA\_\{r,u\}in \([25](https://arxiv.org/html/2610.41074#S4.E25)\) \(withη2/B\\eta^\{2\}/Breplaced byhuh\_\{u\}\)\. Define

Cu​v=\{hu​U^u2​M2​K1out​\(u\),u=v,hu​hv​U^u2​M2​\[1\+2​\(τv−τu\+1\)\]1/a−2​K1out​\(v\),u<v,σ2​hu​hv2​M​K2out​\(u\),v<u\.C\_\{uv\}=\\begin\{cases\}\\dfrac\{h\_\{u\}\\widehat\{U\}\_\{u\}\}\{2M^\{2\}\}K\_\{1\}^\{\{\\rm out\}\}\(u\),&u=v,\\\\\[2\.0pt\] \\dfrac\{h\_\{u\}h\_\{v\}\\widehat\{U\}\_\{u\}\}\{2M^\{2\}\}\[1\+2\(\\tau\_\{v\}\-\\tau\_\{u\+1\}\)\]^\{1/a\-2\}K\_\{1\}^\{\{\\rm out\}\}\(v\),&u<v,\\\\\[2\.0pt\] \\dfrac\{\\sigma^\{2\}h\_\{u\}h\_\{v\}\}\{2M\}K\_\{2\}^\{\{\\rm out\}\}\(u\),&v<u\.\\end\{cases\}\(201\)The row index ofCCis the forward\-rounding update and its column index is the backward\-rounding update\. In particular,u<vu<vrepresents earlier forward and later backward rounding;v<uv<urepresents the reverse order\. All coefficients are nonnegative and independent of the precision choices\. In this appendix,ppdenotes the pair of precision schedules\{pfw,u,pbw,u\}u<T\\\{p\_\{\\mathrm\{fw\},u\},p\_\{\\mathrm\{bw\},u\}\\\}\_\{u<T\}\. Collecting \([15](https://arxiv.org/html/2610.41074#S3.E15)\) gives the exact scalar identity with4−∞=04^\{\-\\infty\}=0:

N^out​\(p\)=\\displaystyle\\widehat\{N\}^\{\{\\rm out\}\}\(p\)=\{\}∑u<T\[Afw,u​4−pfw,u\+Abw,u​4−pbw,u\]\+∑u,v<TCu​v​4−\(pfw,u\+pbw,v\)\.\\displaystyle\\sum\_\{u<T\}\\left\[A\_\{\\mathrm\{fw\},u\}4^\{\-p\_\{\\mathrm\{fw\},u\}\}\+A\_\{\\mathrm\{bw\},u\}4^\{\-p\_\{\\mathrm\{bw\},u\}\}\\right\]\+\\sum\_\{u,v<T\}C\_\{uv\}4^\{\-\(p\_\{\\mathrm\{fw\},u\}\+p\_\{\\mathrm\{bw\},v\}\)\}\.\(202\)
##### A deterministic sufficient feasibility certificate\.

UseR\+R\_\{\+\}andu\+u\_\{\+\}from \([174](https://arxiv.org/html/2610.41074#A4.E174)\), and letu2u\_\{2\}be the upper spectral coefficient in \([93](https://arxiv.org/html/2610.41074#A3.E93)\)\. They are fixed uniform constants on the sketch event\. Use the deterministicΓstab​\(T\)\\Gamma\_\{\\rm stab\}\(T\)of \([83](https://arxiv.org/html/2610.41074#A3.E83)\)\. For a precision schedulepp, define the two additional upper bounds

Γ¯f→b​\(p\)\\displaystyle\\overline\{\\Gamma\}\_\{f\\to b\}\(p\)=u\+​u2M​σ2maxt<T:qbw,t\>0∑u<thuqfw,uU^u\[1\+2\(τt−τu\+1\)\]1/a−2,\\displaystyle=\\frac\{u\_\{\+\}u\_\{2\}\}\{M\\sigma^\{2\}\}\\max\_\{t<T:q\_\{\\mathrm\{bw\},t\}\>0\}\\sum\_\{u<t\}h\_\{u\}q\_\{\\mathrm\{fw\},u\}\\widehat\{U\}\_\{u\}\[1\+2\(\\tau\_\{t\}\-\\tau\_\{u\+1\}\)\]^\{1/a\-2\},Γ¯×​\(p\)\\displaystyle\\overline\{\\Gamma\}\_\{\\times\}\(p\)=R\+​σ2M\[σ2\+τT−\(b−1\)/a\]​maxt<T​qfw,t​∑u<thu​qbw,u\.\\displaystyle=\\frac\{R\_\{\+\}\\sigma^\{2\}\}\{M\[\\sigma^\{2\}\+\\tau\_\{T\}^\{\-\(b\-1\)/a\}\]\}\\max\_\{t<T\}q\_\{\\mathrm\{fw\},t\}\\sum\_\{u<t\}h\_\{u\}q\_\{\\mathrm\{bw\},u\}\.\(203\)The maximum over an empty set is zero\. The certificate requiresΓstab​\(T\)\\Gamma\_\{\\rm stab\}\(T\)and both displayed upper bounds to be at mostκsch\\kappa\_\{\\rm sch\}\. It is sufficient, not necessary, for the original feedback conditions\. Keeping features exact at either rounding location throughout training makes all three quantities zero\.

Fix a partition\{Ij\}j=1J\\\{I\_\{j\}\\\}\_\{j=1\}^\{J\}of the update indices into nonempty predetermined phases and a finite menuℳ⊂ℕ0∪\{∞\}\\mathcal\{M\}\\subset\\mathbb\{N\}\_\{0\}\\cup\\\{\\infty\\\}, allowing exact arithmetic only when the menu contains it\. Let𝒫Jsafe​\(F\)\\mathcal\{P\}\_\{J\}^\{\\rm safe\}\(F\)be the assignmentspr,u=pr,jp\_\{r,u\}=p\_\{r,j\}foru∈Iju\\in I\_\{j\},pr,j∈ℳp\_\{r,j\}\\in\\mathcal\{M\}, which obey the literal charge \([23](https://arxiv.org/html/2610.41074#S3.E23)\) and the certificate above, including the stability bound and both conversion upper bounds in \([203](https://arxiv.org/html/2610.41074#A5.E203)\)\. For finite precision at both rounding locations, the charge is equivalent to

∑j=1J\|Ij\|\(pfw,j\+pbw,j\)≤\(pref\+p0\)​FB​M−2​T​p0\.\\sum\_\{j=1\}^\{J\}\|I\_\{j\}\|\(p\_\{\\mathrm\{fw\},j\}\+p\_\{\\mathrm\{bw\},j\}\)\\leq\\frac\{\(p\_\{\\rm ref\}\+p\_\{0\}\)F\}\{BM\}\-2Tp\_\{0\}\.\(204\)For configurations combining exact arithmetic and finite precision, use \([23](https://arxiv.org/html/2610.41074#S3.E23)\)\. Assume𝒫Jsafe​\(F\)≠∅\\mathcal\{P\}\_\{J\}^\{\\rm safe\}\(F\)\\neq\\varnothing\.

###### Proposition E\.4\(Certified finite\-family precision design\)\.

For the fixed data above, \([202](https://arxiv.org/html/2610.41074#A5.E202)\) holds identically for the scalar objective\. It can be evaluated on phases by aggregatingAr,j=∑u∈IjAr,uA\_\{r,j\}=\\sum\_\{u\\in I\_\{j\}\}A\_\{r,u\}andCj​kphase=∑u∈Ij,v∈IkCu​vC\_\{jk\}^\{\\rm phase\}=\\sum\_\{u\\in I\_\{j\},v\\in I\_\{k\}\}C\_\{uv\}\. Exact enumeration returns a minimizerp∗p\_\{\*\}ofN^out\\widehat\{N\}^\{\{\\rm out\}\}on𝒫Jsafe​\(F\)\\mathcal\{P\}\_\{J\}^\{\\rm safe\}\(F\)after examining at most\|ℳ\|2​J\|\\mathcal\{M\}\|^\{2J\}assignments\. Every returned schedule belongs to the full admissible class throughoutΩM\\Omega\_\{M\}\. Let0<cΔ≤CΔ0<c\_\{\\Delta\}\\leq C\_\{\\Delta\}be the uniform constants forcΔ​N^out≤Δout≤CΔ​N^outc\_\{\\Delta\}\\widehat\{N\}^\{\{\\rm out\}\}\\leq\\Delta^\{\{\\rm out\}\}\\leq C\_\{\\Delta\}\\widehat\{N\}^\{\{\\rm out\}\}in Theorem[1](https://arxiv.org/html/2610.41074#Thmtheorem1)\. Then, for everyS∈ΩMS\\in\\Omega\_\{M\},

ΔSout​\(p∗\)\\displaystyle\\Delta\_\{S\}^\{\{\\rm out\}\}\(p\_\{\*\}\)≤CΔcΔ​minp∈𝒫Jsafe​\(F\)​ΔSout​\(p\),\\displaystyle\\leq\\frac\{C\_\{\\Delta\}\}\{c\_\{\\Delta\}\}\\min\_\{p\\in\\mathcal\{P\}\_\{J\}^\{\\rm safe\}\(F\)\}\\Delta\_\{S\}^\{\{\\rm out\}\}\(p\),ℛSout​\(p∗\)\\displaystyle\\mathscr\{R\}\_\{S\}^\{\{\\rm out\}\}\(p\_\{\*\}\)≤CΔcΔ​minp∈𝒫Jsafe​\(F\)​ℛSout​\(p\)\.\\displaystyle\\leq\\frac\{C\_\{\\Delta\}\}\{c\_\{\\Delta\}\}\\min\_\{p\\in\\mathcal\{P\}\_\{J\}^\{\\rm safe\}\(F\)\}\\mathscr\{R\}\_\{S\}^\{\{\\rm out\}\}\(p\)\.\(205\)

###### Proof\.

Collecting the direct and ordered interaction terms in \([15](https://arxiv.org/html/2610.41074#S3.E15)\) gives \([202](https://arxiv.org/html/2610.41074#A5.E202)\)\. Summing their coefficients over the phase indices gives the stated aggregation\.

The exact\-arithmetic bounds \([169](https://arxiv.org/html/2610.41074#A4.E169)\), \([168](https://arxiv.org/html/2610.41074#A4.E168)\), the parameter comparison established using \([174](https://arxiv.org/html/2610.41074#A4.E174)\) in the proof of Theorem[D\.1](https://arxiv.org/html/2610.41074#Thmtheorem1c), and \([93](https://arxiv.org/html/2610.41074#A3.E93)\) use only the rate, clock and spectral hypotheses\. Substituting these bounds into \([83](https://arxiv.org/html/2610.41074#A3.E83)\)–\([84](https://arxiv.org/html/2610.41074#A3.E84)\), withM​Rt\+qfw,t​Ut≥M​σ2MR\_\{t\}\+q\_\{\\mathrm\{fw\},t\}U\_\{t\}\\geq M\\sigma^\{2\}andrT≥σ2\+τT−\(b−1\)/ar\_\{T\}\\geq\\sigma^\{2\}\+\\tau\_\{T\}^\{\-\(b\-1\)/a\}, givesΓf→b≤Γ¯f→b\\Gamma\_\{f\\to b\}\\leq\\overline\{\\Gamma\}\_\{f\\to b\}andΓ×≤Γ¯×\\Gamma\_\{\\times\}\\leq\\overline\{\\Gamma\}\_\{\\times\}\. Thus the certificate implies full admissibility throughoutΩM\\Omega\_\{M\}without assuming the feedback conditions being verified\. The feasible family is nonempty and has at most\|ℳ\|2​J\|\\mathcal\{M\}\|^\{2J\}members, so enumeration finds a scalar\-objective minimizer\. Apply Proposition[E\.3](https://arxiv.org/html/2610.41074#Thmtheorem3c)with the uniform constants of Theorem[1](https://arxiv.org/html/2610.41074#Thmtheorem1)\. The exact\-arithmetic risk is common because width, batch, horizon, rates and output are fixed, proving both bounds in \([205](https://arxiv.org/html/2610.41074#A5.E205)\)\. ∎

### E\.3How many bits: controlling the risk increase

Corollary[E\.5](https://arxiv.org/html/2610.41074#Thmtheorem5c)proves \([29](https://arxiv.org/html/2610.41074#S4.E29)\) for the relative\-risk requirement \([27](https://arxiv.org/html/2610.41074#S4.E27)\), using \([18](https://arxiv.org/html/2610.41074#S3.E18)\)–\([21](https://arxiv.org/html/2610.41074#S3.E21)\)\. Fix either output under the hypotheses of Corollary[D\.4](https://arxiv.org/html/2610.41074#Thmtheorem4b), and suppress its superscript\. LetDDbe the sum on the right of \([18](https://arxiv.org/html/2610.41074#S3.E18)\) or \([19](https://arxiv.org/html/2610.41074#S3.E19)\), including approximation error\. LetAfw,Abw,A×A\_\{\\mathrm\{fw\}\},A\_\{\\mathrm\{bw\}\},A\_\{\\times\}be the coefficients ofqfw,qbw,qfw​qbwq\_\{\\mathrm\{fw\}\},q\_\{\\mathrm\{bw\}\},q\_\{\\mathrm\{fw\}\}q\_\{\\mathrm\{bw\}\}in \([20](https://arxiv.org/html/2610.41074#S3.E20)\) or \([21](https://arxiv.org/html/2610.41074#S3.E21)\), and writeN=Afw​qfw\+Abw​qbw\+A×​qfw​qbwN=A\_\{\\mathrm\{fw\}\}q\_\{\\mathrm\{fw\}\}\+A\_\{\\mathrm\{bw\}\}q\_\{\\mathrm\{bw\}\}\+A\_\{\\times\}q\_\{\\mathrm\{fw\}\}q\_\{\\mathrm\{bw\}\}\. Then

cR​D≤ℛ0​\(T\)≤CR​D,cΔ​N≤Δ⁡\(T\)≤CΔ​N\.c\_\{\\rm R\}D\\leq\\mathscr\{R\}\_\{0\}\(T\)\\leq C\_\{\\rm R\}D,\\qquad c\_\{\\Delta\}N\\leq\\Delta\(T\)\\leq C\_\{\\Delta\}N\.\(206\)The constants have the fixed dependencies and uniformity of Corollaries[D\.3](https://arxiv.org/html/2610.41074#Thmtheorem3b)–[D\.4](https://arxiv.org/html/2610.41074#Thmtheorem4b)\. Atcτ​Ma≤τ≤Mac\_\{\\tau\}M^\{a\}\\leq\\tau\\leq M^\{a\}for fixedcτ∈\(0,1\]c\_\{\\tau\}\\in\(0,1\], Table[2](https://arxiv.org/html/2610.41074#S4.T2)gives equivalent scales\.

###### Corollary E\.5\(Constant\-learning\-rate precision thresholds\)\.

Forϑ\>0\\vartheta\>0and at least one rounded location, \([29](https://arxiv.org/html/2610.41074#S4.E29)\) is sufficient for \([27](https://arxiv.org/html/2610.41074#S4.E27)\) withC=n​CΔ/cRC=nC\_\{\\Delta\}/c\_\{\\rm R\}and necessary withC=cΔ/CRC=c\_\{\\Delta\}/C\_\{\\rm R\}\. Heren=1n=1when one location is rounded andn=3n=3when both are rounded\.

###### Proof\.

Corollaries[D\.3](https://arxiv.org/html/2610.41074#Thmtheorem3b)–[D\.4](https://arxiv.org/html/2610.41074#Thmtheorem4b), the approximation bounds onΩM\\Omega\_\{M\}, and the uniform noise replacement prove \([206](https://arxiv.org/html/2610.41074#A5.E206)\)\. Atcτ​Ma≤τ≤Mac\_\{\\tau\}M^\{a\}\\leq\\tau\\leq M^\{a\}, each clock power differs from itsτ=Ma\\tau=M^\{a\}value by fixed factors depending oncτc\_\{\\tau\}\. This proves the same comparisons for Table[2](https://arxiv.org/html/2610.41074#S4.T2), including its exact\-arithmetic SGD risk row\. WriteC−=cΔ/CRC\_\{\-\}=c\_\{\\Delta\}/C\_\{\\rm R\}andC\+=CΔ/cRC\_\{\+\}=C\_\{\\Delta\}/c\_\{\\rm R\}\. The central joint test is

C\+​N≤ϑ​D⟹Δ≤ϑ​ℛ0,Δ≤ϑ​ℛ0⟹C−​N≤ϑ​D\.C\_\{\+\}N\\leq\\vartheta D\\quad\\Longrightarrow\\quad\\Delta\\leq\\vartheta\\mathscr\{R\}\_\{0\},\\qquad\\Delta\\leq\\vartheta\\mathscr\{R\}\_\{0\}\\quad\\Longrightarrow\\quad C\_\{\-\}N\\leq\\vartheta D\.\(207\)The sufficient bound allows arbitrary tradeoffs among the nonnegative terms\. Bounding each active term byϑ​D/\(n​C\+\)\\vartheta D/\(nC\_\{\+\}\)is one conservative way to enforce it; it is not required that the contributions be equal\. Conversely, the necessary bound on the sum requires every active term to be at mostϑ​D/C−\\vartheta D/C\_\{\-\}\. Substituteqfw=4−pfwq\_\{\\mathrm\{fw\}\}=4^\{\-p\_\{\\mathrm\{fw\}\}\}andqbw=4−pbwq\_\{\\mathrm\{bw\}\}=4^\{\-p\_\{\\mathrm\{bw\}\}\}in the active bounds and take base\-four logarithms\. Nonnegative mantissas give the positive parts in \([29](https://arxiv.org/html/2610.41074#S4.E29)\), withC=n​C\+=n​CΔ/cRC=nC\_\{\+\}=nC\_\{\\Delta\}/c\_\{\\rm R\}for sufficiency andC=C−=cΔ/CRC=C\_\{\-\}=c\_\{\\Delta\}/C\_\{\\rm R\}for necessity\. For sufficiency, round real bounds upward to allowed integer mantissas and verify admissibility\. If both computations are exact,N=Δ=0N=\\Delta=0\. ∎

##### A necessary backward precision that grows with width\.

For the uniform average atτ≍Ma\\tau\\asymp M^\{a\}, fix positiveρ,σ2,ϑ\\rho,\\sigma^\{2\},\\varthetaand keep forward features exact\. ThenDavg≍M1−bD^\{\\rm avg\}\\asymp M^\{1\-b\}andAbw≍ρ​σ2A\_\{\\mathrm\{bw\}\}\\asymp\\rho\\sigma^\{2\}\. The lower bound forΔ\\Deltaand upper bound forℛ0\\mathscr\{R\}\_\{0\}, rather than the sufficient tests alone, imply

qbw≤C​ϑ​M1−bρ​σ2,pbw≥\[\(b−1\)​log4​M\+log4⁡c​ρ​σ2ϑ\]\+\.q\_\{\\mathrm\{bw\}\}\\leq C\\frac\{\\vartheta M^\{1\-b\}\}\{\\rho\\sigma^\{2\}\},\\qquad p\_\{\\mathrm\{bw\}\}\\geq\\left\[\(b\-1\)\\log\_\{4\}M\+\\log\_\{4\}\\frac\{c\\rho\\sigma^\{2\}\}\{\\vartheta\}\\right\]\_\{\+\}\.This is the necessary condition stated in Section[4\.3](https://arxiv.org/html/2610.41074#S4.SS3)\. With both locations rounded, the joint term also gives a condition onpfw\+pbwp\_\{\\mathrm\{fw\}\}\+p\_\{\\mathrm\{bw\}\}; separate one\-location accuracy tests do not control that additional term\.

##### Consistency versus preserving the exact\-arithmetic rate\.

Fix positiveρ,σ2\\rho,\\sigma^\{2\}and letM,τ→∞M,\\tau\\to\\inftyin the full constant class\. The window anda\>ba\>bgiveℛ0avg≍τ−\(b−1\)/a\\mathscr\{R\}\_\{0\}^\{\\rm avg\}\\asymp\\tau^\{\-\(b\-1\)/a\}, so the general\-clock relative comparison is

Δavgℛ0avg≍\\displaystyle\\frac\{\\Delta^\{\\rm avg\}\}\{\\mathscr\{R\}\_\{0\}^\{\\rm avg\}\}\\asymp\{\}ρ​qfw​τ1/aM\+ρqbw\(sM\+τ−\(b−1\)/a\)τb/aM\+ρ​qfw​qbw​τ1\+1/aM2\.\\displaystyle\\frac\{\\rho q\_\{\\mathrm\{fw\}\}\\tau^\{1/a\}\}\{M\}\+\\frac\{\\rho q\_\{\\mathrm\{bw\}\}\(s\_\{M\}\+\\tau^\{\-\(b\-1\)/a\}\)\\tau^\{b/a\}\}\{M\}\+\\frac\{\\rho q\_\{\\mathrm\{fw\}\}q\_\{\\mathrm\{bw\}\}\\tau^\{1\+1/a\}\}\{M^\{2\}\}\.\(208\)The first term is bounded by the rate ceiling\. The third is bounded by a fixed multiple of the stability gain, becauseτ1/a≤M\\tau^\{1/a\}\\leq M\. Their absolute contributions vanish with the exact\-arithmetic risk\. SincesM\+τ−\(b−1\)/a→σ2\>0s\_\{M\}\+\\tau^\{\-\(b\-1\)/a\}\\to\\sigma^\{2\}\>0, the remaining term yields

ℛavg→0\\displaystyle\\mathscr\{R\}^\{\\rm avg\}\\to 0⟺qbwτ1/a/M→0,\\displaystyle\\Longleftrightarrow\\quad q\_\{\\mathrm\{bw\}\}\\tau^\{1/a\}/M\\to 0,\(209\)Δavg=O⁡\(ℛ0avg\)\\displaystyle\\Delta^\{\\rm avg\}=O\(\\mathscr\{R\}\_\{0\}^\{\\rm avg\}\)⟺qbwτb/a/M=O\(1\)\.\\displaystyle\\Longleftrightarrow\\quad q\_\{\\mathrm\{bw\}\}\\tau^\{b/a\}/M=O\(1\)\.Nonnegativity gives necessity and the preceding bounds give sufficiency\. These distinctions explain Section[3\.2](https://arxiv.org/html/2610.41074#S3.SS2)’s learning\-curve discussion\.

##### Sufficient width tests for a desired training duration\.

For largeτ\\tauand fixed positive noise, one sufficient construction bounds each of the three terms inCΔ​NC\_\{\\Delta\}NbyϑcRτ−\(b−1\)/a/3\\vartheta c\_\{\\rm R\}\\tau^\{\-\(b\-1\)/a\}/3\. This is a conservative way to enforce the joint budget \([207](https://arxiv.org/html/2610.41074#A5.E207)\)\. It gives

M\\displaystyle M≥τ1/a,\\displaystyle\\geq\\tau^\{1/a\},M\\displaystyle M≥3​CΔ​ρ​qfwϑ​cR​τ1/a,\\displaystyle\\geq\\frac\{3C\_\{\\Delta\}\\rho q\_\{\\mathrm\{fw\}\}\}\{\\vartheta c\_\{\\rm R\}\}\\tau^\{1/a\},\(210\)M\\displaystyle M≥3​CΔ​ρ​qbwϑ​cR​\(sM\+τ−b−1a\)​τb/a,\\displaystyle\\geq\\frac\{3C\_\{\\Delta\}\\rho q\_\{\\mathrm\{bw\}\}\}\{\\vartheta c\_\{\\rm R\}\}\(s\_\{M\}\+\\tau^\{\-\\frac\{b\-1\}\{a\}\}\)\\tau^\{b/a\},M2\\displaystyle M^\{2\}≥3​CΔ​ρ​qfw​qbwϑ​cR​τ1\+1/a,\\displaystyle\\geq\\frac\{3C\_\{\\Delta\}\\rho q\_\{\\mathrm\{fw\}\}q\_\{\\mathrm\{bw\}\}\}\{\\vartheta c\_\{\\rm R\}\}\\tau^\{1\+1/a\},M\\displaystyle M≥ρ​qfw​qbwκc​τ\.\\displaystyle\\geq\\frac\{\\rho q\_\{\\mathrm\{fw\}\}q\_\{\\mathrm\{bw\}\}\}\{\\kappa\_\{\\rm c\}\}\\tau\.The first is the clock window, the next three follow by substituting the coefficients of \([21](https://arxiv.org/html/2610.41074#S3.E21)\) into the component budgets, and the last verifies the sufficient feedback test\. Together with the remaining width, minimum cumulative learning rate and rate conditions they ensure the desired relative accuracy\. They are sufficient construction rules, not all necessary width restrictions\. Deleting an exact location deletes its associated terms\. These are the complete checks behind the leading width scales in \([233](https://arxiv.org/html/2610.41074#A5.E233)\)\. The last condition is simply \([22](https://arxiv.org/html/2610.41074#S3.E22)\) rearranged as a lower bound onMM\.

### E\.4Learning\-rate design with stochastic rounding

Lemma[E\.6](https://arxiv.org/html/2610.41074#Thmtheorem6c)solves a continuous relaxation at fixed total learning rate\. Theorem[E\.7](https://arxiv.org/html/2610.41074#Thmtheorem7c)then chooses that total and constructs discrete SGD rates with a fixed\-factor risk guarantee\. The inverse\-square\-root calculation follows[Bordelon and Mori \(2026, Appendix B\.1\)](https://arxiv.org/html/2610.41074#bib.bib2)\. No continuous\-time limit of SGD is taken\.

Fixτ,h,T\>0\\tau,h,T\>0withτ≤h​T\\tau\\leq hT, and letK:\[0,τ\]→\(0,∞\)K:\[0,\\tau\]\\to\(0,\\infty\)be continuous\. For measurable rates0≤η≤h0\\leq\\eta\\leq hwith∫0Tη=τ\\int\_\{0\}^\{T\}\\eta=\\tau, setχ⁡\(t\)=∫tTη⁡\(s\)​𝑑s\\chi\(t\)=\\int\_\{t\}^\{T\}\\eta\(s\)\\,ds\. WriteH⁡\(x\)=∫0xK⁡\(v\)​𝑑vH\(x\)=\\int\_\{0\}^\{x\}\\sqrt\{K\(v\)\}\\,dvand consider

𝒱⁡\[η\]=∫0Tη​\(t\)2​K​\(χ⁡\(t\)\)​𝑑t\.\\mathcal\{V\}\[\\eta\]=\\int\_\{0\}^\{T\}\\eta\(t\)^\{2\}K\(\\chi\(t\)\)\\,dt\.\(211\)
###### Lemma E\.6\(Optimal learning rate at fixed total rate\)\.

ForT\>τ/hT\>\\tau/h, the unique minimizer isη⁡\(t\)=a⁡\(χ⁡\(t\)\)\\eta\(t\)=a\(\\chi\(t\)\), where

a⁡\(χ\)=min⁡\{h,λ/K⁡\(χ\)\},∫0τd​χa⁡\(χ\)=T\.a\(\\chi\)=\\min\\\{h,\\sqrt\{\\lambda/K\(\\chi\)\}\\\},\\qquad\\int\_\{0\}^\{\\tau\}\\frac\{d\\chi\}\{a\(\\chi\)\}=T\.\(212\)The time constraint determinesλ\>0\\lambda\>0uniquely\. AtT=τ/hT=\\tau/h, the only feasible rate ishhalmost everywhere\. The uncapped optimum is feasible exactly whenH⁡\(τ\)≤h​T​min\[0,τ\]⁡KH\(\\tau\)\\leq hT\\sqrt\{\\min\_\{\[0,\\tau\]\}K\}; then

a⁡\(χ\)=H⁡\(τ\)T​K⁡\(χ\),H⁡\(χ⁡\(t\)\)H⁡\(τ\)=1−tT,min⁡𝒱=H​\(τ\)2T\.a\(\\chi\)=\\frac\{H\(\\tau\)\}\{T\\sqrt\{K\(\\chi\)\}\},\\qquad\\frac\{H\(\\chi\(t\)\)\}\{H\(\\tau\)\}=1\-\\frac\{t\}\{T\},\\qquad\\min\\mathcal\{V\}=\\frac\{H\(\\tau\)^\{2\}\}\{T\}\.\(213\)Every feasible profile satisfies

𝒱⁡\[η\]≥H​\(τ\)2/T\.\\mathcal\{V\}\[\\eta\]\\geq H\(\\tau\)^\{2\}/T\.\(214\)IfKKis strictly decreasing andT\>τ/hT\>\\tau/h, the optimal rate starts at the cap if needed and then decreases\.

###### Proof\.

Forλ\>0\\lambda\>0, minimizingz​K​\(x\)\+λ/zzK\(x\)\+\\lambda/zover0<z≤h0<z\\leq hgives \([212](https://arxiv.org/html/2610.41074#A5.E212)\)\. The time integral∫0τmax⁡\{h−1,K⁡\(x\)/λ\}​𝑑x\\int\_\{0\}^\{\\tau\}\\max\\\{h^\{\-1\},\\sqrt\{K\(x\)/\\lambda\}\\\}\\,dxis continuous, tends to infinity asλ↓0\\lambda\\downarrow 0, and decreases strictly above its minimumτ/h\\tau/h\. ThusT\>τ/hT\>\\tau/hdetermines one multiplier\.

Putmλ​\(x\)=a⁡\(x\)​K​\(x\)\+λ/a⁡\(x\)m\_\{\\lambda\}\(x\)=a\(x\)K\(x\)\+\\lambda/a\(x\)\. Pointwise minimality givesz2​K​\(x\)\+λ≥z​mλ​\(x\)z^\{2\}K\(x\)\+\\lambda\\geq zm\_\{\\lambda\}\(x\)for0≤z≤h0\\leq z\\leq h, including zero\. Sinceχ˙=−η\\dot\{\\chi\}=\-\\eta,

𝒱⁡\[η\]≥∫0τmλ​\(x\)​𝑑x−λ​T\.\\mathcal\{V\}\[\\eta\]\\geq\\int\_\{0\}^\{\\tau\}m\_\{\\lambda\}\(x\)\\,dx\-\\lambda T\.Equality holds for the profile defined byt=∫χ⁡\(t\)τ𝑑x/a⁡\(x\)t=\\int\_\{\\chi\(t\)\}^\{\\tau\}dx/a\(x\)and forcesη=a⁡\(χ\)\\eta=a\(\\chi\)almost everywhere, proving uniqueness even against zero\-rate intervals\. AtT=τ/hT=\\tau/h, feasibility forcesη=h\\eta=halmost everywhere\.

Without clipping,λ=H⁡\(τ\)/T\\sqrt\{\\lambda\}=H\(\\tau\)/Tand the maximum rate isH⁡\(τ\)/\(T​min⁡K\)H\(\\tau\)/\(T\\sqrt\{\\min K\}\), giving the cap criterion and \([213](https://arxiv.org/html/2610.41074#A5.E213)\)\. For every feasible rate,H⁡\(τ\)=∫0Tη⁡\(t\)​K⁡\(χ⁡\(t\)\)​𝑑tH\(\\tau\)=\\int\_\{0\}^\{T\}\\eta\(t\)\\sqrt\{K\(\\chi\(t\)\)\}\\,dt; Cauchy–Schwarz gives \([214](https://arxiv.org/html/2610.41074#A5.E214)\)\. Finally, decreasingKKmakesa⁡\(χ\)a\(\\chi\)increase inχ\\chi\. Asχ\\chidecreases during training, clipping can occur only initially\. ∎

#### E\.4\.1From the continuous optimum to discrete SGD

##### Setup and comparison class\.

Fix an eligible widthMM, integersT,B≥1T,B\\geq 1, and a constant backward mantissapbw∈ℕ0∪\{∞\}p\_\{\\mathrm\{bw\}\}\\in\\mathbb\{N\}\_\{0\}\\cup\\\{\\infty\\\}\. Forward features remain exact and the output is the last iterate\. Letq=4−pbwq=4^\{\-p\_\{\\mathrm\{bw\}\}\}, withq=0q=0at exact precision, and use the common cap

ηmax=min⁡\{B​ρsch,c0/Cg\},\\eta\_\{\\max\}=\\min\\\{B\\rho\_\{\\rm sch\},c\_\{0\}/C\_\{g\}\\\},\(215\)whereg1​\(S\)≤Cgg\_\{1\}\(S\)\\leq C\_\{g\}uniformly onΩM\\Omega\_\{M\}\. This cap is chosen before the sketch draw\. Assume

I=\[τ∗sch,min⁡\{Ma,ηmax​T\}\]≠∅,B​T​M​\[1\+c⁡\(pbw\)\]≤F\.I=\[\\tau\_\{\*\}^\{\\rm sch\},\\min\\\{M^\{a\},\\eta\_\{\\max\}T\\\}\]\\neq\\varnothing,\\qquad BTM\[1\+c\(p\_\{\\mathrm\{bw\}\}\)\]\\leq F\.\(216\)The cost is fixed for these data and does not depend on the rate shape\. All joint feedback gains vanish with exact forward features; the rate ceiling, clock window and both width conditions still apply\.

Writeβ=\(b−1\)/a\\beta=\(b\-1\)/aand define the fixed\-time bias/approximation part and the combined noise response by

BM​\(τ\)=M1−b\+12​\(1\+2​τ\)−β,Kq​\(x\)=\(1\+2​x\)1/a−2​\[1\+qM​\(1\+2​x\)\]\.B\_\{M\}\(\\tau\)=M^\{1\-b\}\+\\tfrac\{1\}\{2\}\(1\+2\\tau\)^\{\-\\beta\},\\qquad K\_\{q\}\(x\)=\(1\+2x\)^\{1/a\-2\}\\left\[1\+\\frac\{q\}\{M\}\(1\+2x\)\\right\]\.\(217\)Let𝒞q\\mathcal\{C\}\_\{q\}be the measurable rates0≤η≤ηmax0\\leq\\eta\\leq\\eta\_\{\\max\}on\[0,T\]\[0,T\]with total rateτ∈I\\tau\\in I\. Use the remaining clockχ\\chifrom \([211](https://arxiv.org/html/2610.41074#A5.E211)\) and define

Jq​\[η\]=BM​\(τ\)\+σ22​B​∫0Tη​\(t\)2​Kq​\(χ⁡\(t\)\)​𝑑t\.J\_\{q\}\[\\eta\]=B\_\{M\}\(\\tau\)\+\\frac\{\\sigma^\{2\}\}\{2B\}\\int\_\{0\}^\{T\}\\eta\(t\)^\{2\}K\_\{q\}\(\\chi\(t\)\)\\,dt\.\(218\)This is a finite\-horizon relaxation of the discrete scalar risk\. One time unit represents one update; we take no diffusion or vanishing\-step limit\. The discrete rates associated with a continuous profile are its cell averages\(D0​η\)i=∫ii\+1η⁡\(t\)​𝑑t\(D\_\{0\}\\eta\)\_\{i\}=\\int\_\{i\}^\{i\+1\}\\eta\(t\)\\,dt,0≤i<T0\\leq i<T\.

For each event sketchSS, let𝒫q​\(S\)\\mathcal\{P\}\_\{q\}\(S\)contain all deterministicTT\-step learning\-rate schedules with these fixedM,B,pbwM,B,p\_\{\\mathrm\{bw\}\}, exact forward features, and the full scheduled assumptions atSS\. The pointwise spectral cap in this comparator class may be less restrictive than the common cap \([215](https://arxiv.org/html/2610.41074#A5.E215)\)\.

##### Learning\-rate construction\.

For eachτ∈I\\tau\\in Iwithτ<ηmax​T\\tau<\\eta\_\{\\max\}T, letaτa\_\{\\tau\}andλ⁡\(τ\)\\lambda\(\\tau\)be the profile and unique multiplier in \([212](https://arxiv.org/html/2610.41074#A5.E212)\), withh=ηmaxh=\\eta\_\{\\max\}and the kernel \([217](https://arxiv.org/html/2610.41074#A5.E217)\)\. Atτ=ηmax​T\\tau=\\eta\_\{\\max\}T, use the constant\-cap profile\. Choose the smallest minimizerτopt\\tau\_\{\\rm opt\}onIIof

jq​\(τ\)=BM​\(τ\)\+σ22​B​∫0τaτ​\(x\)​Kq​\(x\)​𝑑x\.j\_\{q\}\(\\tau\)=B\_\{M\}\(\\tau\)\+\\frac\{\\sigma^\{2\}\}\{2B\}\\int\_\{0\}^\{\\tau\}a\_\{\\tau\}\(x\)K\_\{q\}\(x\)\\,dx\.Defineχ∗​\(t\)\\chi\_\{\*\}\(t\)byt=∫χ∗​\(t\)τopt𝑑x/aτopt​\(x\)t=\\int\_\{\\chi\_\{\*\}\(t\)\}^\{\\tau\_\{\\rm opt\}\}dx/a\_\{\\tau\_\{\\rm opt\}\}\(x\)and setη∗​\(t\)=aτopt​\(χ∗​\(t\)\)\\eta\_\{\*\}\(t\)=a\_\{\\tau\_\{\\rm opt\}\}\(\\chi\_\{\*\}\(t\)\)\.

The following theorem converts the continuous solution into an admissible discrete SGD schedule by averaging over update intervals\. Its prediction risk is within a fixed factor of the best schedule in the stated comparison class\.

###### Theorem E\.7\(Learning\-rate construction and risk bound\)\.

The profileη∗\\eta\_\{\*\}minimizesJqJ\_\{q\}over𝒞q\\mathcal\{C\}\_\{q\}\. Its cell averages are admissible discrete SGD rates and satisfy

ℛSlast​\(D0​η∗\)≤Clr​infπ∈𝒫q​\(S\)ℛSlast​\(π\),S∈ΩM\.\\mathscr\{R\}\_\{S\}^\{\\rm last\}\(D\_\{0\}\\eta\_\{\*\}\)\\leq C\_\{\\rm lr\}\\inf\_\{\\pi\\in\\mathcal\{P\}\_\{q\}\(S\)\}\\mathscr\{R\}\_\{S\}^\{\\rm last\}\(\\pi\),\\qquad S\\in\\Omega\_\{M\}\.\(219\)The factorClrC\_\{\\rm lr\}depends only on the fixed model and constants, admissibility thresholds and positive noise variance, uniformly inM,T,B,pbw,FM,T,B,p\_\{\\mathrm\{bw\}\},FandSS\.

###### Proof\.

*Continuous optimization\.*Lemma[E\.6](https://arxiv.org/html/2610.41074#Thmtheorem6c)minimizesJqJ\_\{q\}at each fixedτ\\tau\. The unique multiplier depends continuously onτ<ηmax​T\\tau<\\eta\_\{\\max\}T\. At the cap endpoint,∫0τ\(ηmax−aτ​\(x\)\)​𝑑x≤ηmax2​\(T−τ/ηmax\)→0\\int\_\{0\}^\{\\tau\}\(\\eta\_\{\\max\}\-a\_\{\\tau\}\(x\)\)\\,dx\\leq\\eta\_\{\\max\}^\{2\}\(T\-\\tau/\\eta\_\{\\max\}\)\\to 0\. SinceKqK\_\{q\}is bounded,jqj\_\{q\}converges to the constant\-cap value\. Thus it is continuous on compactII, soτopt\\tau\_\{\\rm opt\}exists andη∗\\eta\_\{\*\}minimizesJqJ\_\{q\}\.

*Discrete comparison\.*For a discrete rate arrayπ=\(ηi\)i<T\\pi=\(\\eta\_\{i\}\)\_\{i<T\}, the scalar risk \([172](https://arxiv.org/html/2610.41074#A4.E172)\) is

Φ⁡\(π\)=BM​\(τ\)\+σ22​B​∑i<Tηi2​Kq​\(τT−τi\+1\)\.\\Phi\(\\pi\)=B\_\{M\}\(\\tau\)\+\\frac\{\\sigma^\{2\}\}\{2B\}\\sum\_\{i<T\}\\eta\_\{i\}^\{2\}K\_\{q\}\(\\tau\_\{T\}\-\\tau\_\{i\+1\}\)\.LetE0​πE\_\{0\}\\pibe the piecewise\-constant extension of a discrete schedule capped byηmax\\eta\_\{\\max\}\. On a unit interval, a capped profile’s remaining clock varies by at mostηmax\\eta\_\{\\max\}\. The two negative powers inKqK\_\{q\}therefore giveD−1​Kq​\(χ⁡\(i\+1\)\)≤Kq​\(χ⁡\(t\)\)≤Kq​\(χ⁡\(i\+1\)\)D^\{\-1\}K\_\{q\}\(\\chi\(i\+1\)\)\\leq K\_\{q\}\(\\chi\(t\)\)\\leq K\_\{q\}\(\\chi\(i\+1\)\), whereD=\(1\+2​ηmax\)2−1/aD=\(1\+2\\eta\_\{\\max\}\)^\{2\-1/a\}\. Integration and Cauchy–Schwarz on each interval yield

D−1​Φ​\(π\)≤Jq​\(E0​π\)≤Φ⁡\(π\),Φ⁡\(D0​η\)≤D​Jq​\[η\]\.D^\{\-1\}\\Phi\(\\pi\)\\leq J\_\{q\}\(E\_\{0\}\\pi\)\\leq\\Phi\(\\pi\),\\qquad\\Phi\(D\_\{0\}\\eta\)\\leq DJ\_\{q\}\[\\eta\]\.\(220\)Cell averaging preserves cumulative rate at integer times and the rate cap\. With zero joint gains and \([216](https://arxiv.org/html/2610.41074#A5.E216)\), this proves admissibility and unchanged cost forD0​η∗D\_\{0\}\\eta\_\{\*\}\. AlsoD≤Dmax=\(1\+2​c0/Cg\)2−1/aD\\leq D\_\{\\max\}=\(1\+2c\_\{0\}/C\_\{g\}\)^\{2\-1/a\}\.

*Comparison with all admissible schedules\.*Putd=min⁡\{1,cg/Cg\}\>0d=\\min\\\{1,c\_\{g\}/C\_\{g\}\\\}\>0\. Sinceg1​\(S\)≥cgg\_\{1\}\(S\)\\geq c\_\{g\}, multiplying any pointwise\-admissible schedule byddenforces the common cap\. Fromd⁡\(1\+2​x\)≤1\+2​d​x≤1\+2​xd\(1\+2x\)\\leq 1\+2dx\\leq 1\+2x,

BM​\(d​τ\)≤d−β​BM​\(τ\),d2​Kq​\(d​x\)≤Kq​\(x\)\.B\_\{M\}\(d\\tau\)\\leq d^\{\-\\beta\}B\_\{M\}\(\\tau\),\\qquad d^\{2\}K\_\{q\}\(dx\)\\leq K\_\{q\}\(x\)\.HenceΦ⁡\(d​π\)≤d−β​Φ​\(π\)\\Phi\(d\\pi\)\\leq d^\{\-\\beta\}\\Phi\(\\pi\)\. Ifτ≥τ∗sch/d\\tau\\geq\\tau\_\{\*\}^\{\\rm sch\}/d, the scaled schedule stays in the clock window; its piecewise\-constant extension belongs to𝒞q\\mathcal\{C\}\_\{q\}and hasJq≤d−β​Φ​\(π\)J\_\{q\}\\leq d^\{\-\\beta\}\\Phi\(\\pi\)by \([220](https://arxiv.org/html/2610.41074#A5.E220)\)\.

For smallerτ\\tau, use insteadηi=τ∗sch/T\\eta\_\{i\}=\\tau\_\{\*\}^\{\\rm sch\}/T, feasible by \([216](https://arxiv.org/html/2610.41074#A5.E216)\)\. Its scalar risk is at most

E∗=32\+σ2​ρsch​τ∗sch\.E\_\{\*\}=\\tfrac\{3\}\{2\}\+\\sigma^\{2\}\\rho\_\{\\rm sch\}\\tau\_\{\*\}^\{\\rm sch\}\.\(221\)IndeedBM≤3/2B\_\{M\}\\leq 3/2,Kq≤2K\_\{q\}\\leq 2and∑iηi2/B≤ρsch​τ∗sch\\sum\_\{i\}\\eta\_\{i\}^\{2\}/B\\leq\\rho\_\{\\rm sch\}\\tau\_\{\*\}^\{\\rm sch\}\. The comparator’s bias is at least12​\(1\+2​τ∗sch/d\)−β\\tfrac\{1\}\{2\}\(1\+2\\tau\_\{\*\}^\{\\rm sch\}/d\)^\{\-\\beta\}\. Thus both cases give a profileE​π∈𝒞qE\\pi\\in\\mathcal\{C\}\_\{q\}with

Jq​\[E​π\]≤Ccap​Φ​\(π\),Ccap=max⁡\{d−β,2​E∗​\(1\+2​τ∗sch/d\)β\}\.J\_\{q\}\[E\\pi\]\\leq C\_\{\\rm cap\}\\Phi\(\\pi\),\\qquad C\_\{\\rm cap\}=\\max\\\{d^\{\-\\beta\},2E\_\{\*\}\(1\+2\\tau\_\{\*\}^\{\\rm sch\}/d\)^\{\\beta\}\\\}\.\(222\)Continuous optimality and \([220](https://arxiv.org/html/2610.41074#A5.E220)\) now imply

Φ⁡\(D0​η∗\)≤Dmax​infη∈𝒞qJq​\[η\]≤Dmax​Ccap​Φ​\(π\)\(π∈𝒫q​\(S\)\)\.\\Phi\(D\_\{0\}\\eta\_\{\*\}\)\\leq D\_\{\\max\}\\inf\_\{\\eta\\in\\mathcal\{C\}\_\{q\}\}J\_\{q\}\[\\eta\]\\leq D\_\{\\max\}C\_\{\\rm cap\}\\Phi\(\\pi\)\\qquad\(\\pi\\in\\mathcal\{P\}\_\{q\}\(S\)\)\.\(223\)The two\-sided risk comparison in Theorem[D\.1](https://arxiv.org/html/2610.41074#Thmtheorem1c)proves \([219](https://arxiv.org/html/2610.41074#A5.E219)\) withClr=QΦ​Dmax​CcapC\_\{\\rm lr\}=Q\_\{\\Phi\}D\_\{\\max\}C\_\{\\rm cap\}\. The construction uses only the fixed bounds and training specifications before the random draws, so it is independent of the realized sketch\. ∎

##### Comparing exact and rounded objectives\.

For the kernel \([217](https://arxiv.org/html/2610.41074#A5.E217)\), writeHqH\_\{q\}for the functionHHin Lemma[E\.6](https://arxiv.org/html/2610.41074#Thmtheorem6c)\. Its fixed\-total\-rate uncapped noise value is given by \([213](https://arxiv.org/html/2610.41074#A5.E213)\)\. For exact arithmetic,

H0​\(τ\)=a⁡\[\(1\+2​τ\)1/\(2​a\)−1\]\.H\_\{0\}\(\\tau\)=a\\left\[\(1\+2\\tau\)^\{1/\(2a\)\}\-1\\right\]\.\(224\)On the common continuous class𝒞q\\mathcal\{C\}\_\{q\}, putτmax=max⁡I\\tau\_\{\\max\}=\\max Iandεq=q⁡\(1\+2​τmax\)/M\\varepsilon\_\{q\}=q\(1\+2\\tau\_\{\\max\}\)/M\. The pointwise boundsK0≤Kq≤\(1\+εq\)​K0K\_\{0\}\\leq K\_\{q\}\\leq\(1\+\\varepsilon\_\{q\}\)K\_\{0\}and common nonnegative bias/approximation terms imply

inf𝒞qJ0≤inf𝒞qJq≤\(1\+εq\)​inf𝒞qJ0\.\\inf\_\{\\mathcal\{C\}\_\{q\}\}J\_\{0\}\\leq\\inf\_\{\\mathcal\{C\}\_\{q\}\}J\_\{q\}\\leq\(1\+\\varepsilon\_\{q\}\)\\inf\_\{\\mathcal\{C\}\_\{q\}\}J\_\{0\}\.\(225\)Thus a uniformly small correction changes the best scalar\-objective value only slightly\. The discrete actual\-risk comparison still carries Theorem[E\.7](https://arxiv.org/html/2610.41074#Thmtheorem7c)’s fixed factor\. Separately, nonnegativity of SR\-induced error givesinfℛ0last≤infℛqlast\\inf\\mathscr\{R\}\_\{0\}^\{\\rm last\}\\leq\\inf\\mathscr\{R\}\_\{q\}^\{\\rm last\}on the same admissible discrete class\. A common\-budget comparison requires both computations to fit, namelyF≥B​T​M​max⁡\{2,1\+c⁡\(pbw\)\}F\\geq BTM\\max\\\{2,1\+c\(p\_\{\\mathrm\{bw\}\}\)\\\}\.

#### E\.4\.2Interpreting the explicit learning\-rate schedules

Substituting \([217](https://arxiv.org/html/2610.41074#A5.E217)\) into \([212](https://arxiv.org/html/2610.41074#A5.E212)\) gives the cumulative\-learning\-rate expression \([31](https://arxiv.org/html/2610.41074#S4.E31)\), withχ⁡\(t\)=τ−τt\\chi\(t\)=\\tau\-\\tau\_\{t\}andλbw=λ⁡\(τ\)\\lambda\_\{\\mathrm\{bw\}\}=\\lambda\(\\tau\)forτ<ηmax​T\\tau<\\eta\_\{\\max\}T\. Settingq=0q=0gives \([30](https://arxiv.org/html/2610.41074#S4.E30)\), with its own normalizationλ0\\lambda\_\{0\}\. Atτ=ηmax​T\\tau=\\eta\_\{\\max\}T, both profiles are constant at the cap and a multiplier need not be unique\. For the same prescribedτ\\tau, the two profiles have area∫0Tη⁡\(t\)​𝑑t=τ\\int\_\{0\}^\{T\}\\eta\(t\)\\,dt=\\tau; their multipliers are chosen separately\. The theorem then minimizes the risk overτ\\tau, so optimized exact and rounded runs need not have the same area\.

Without clipping, the fixed\-total\-rate objective value follows by multiplying the minimum in \([213](https://arxiv.org/html/2610.41074#A5.E213)\) byσ2/\(2​B\)\\sigma^\{2\}/\(2B\)and addingBM​\(τ\)B\_\{M\}\(\\tau\)\. This is an identity for the continuous scalar objective; the discrete actual\-risk guarantee includes the factor in Theorem[E\.7](https://arxiv.org/html/2610.41074#Thmtheorem7c)\.

##### Exact arithmetic and the finite endpoint\.

For exact arithmetic, \([224](https://arxiv.org/html/2610.41074#A5.E224)\) evaluates the integral\. Writingc=λ0c=\\sqrt\{\\lambda\_\{0\}\}on an uncapped segment and usingd​χ/d​t=−ηd\\chi/dt=\-\\etagives

dd​t​\(1\+2​χ\)1/\(2​a\)=−ca\.\\frac\{d\}\{dt\}\(1\+2\\chi\)^\{1/\(2a\)\}=\-\\frac\{c\}\{a\}\.The segment ending atχ⁡\(T\)=0\\chi\(T\)=0therefore has the explicit time form

ηexact​\(t\)=c​\[1\+ca​\(T−t\)\]2​a−1\.\\eta\_\{\\rm exact\}\(t\)=c\\left\[1\+\\frac\{c\}\{a\}\(T\-t\)\\right\]^\{2a\-1\}\.\(226\)Wherec⁡\(T−t\)/a≫1c\(T\-t\)/a\\gg 1, it follows the power\(T−t\)2​a−1\(T\-t\)^\{2a\-1\}\. This is the decay exponent of[Bordelon and Mori \(2026, Result 2\)](https://arxiv.org/html/2610.41074#bib.bib2)in the regime1<b<a1<b<a, after translating their notation\. The added one in our scalar response gives a finite endpoint: the continuous rate tends tocc, rather than extrapolating the bulk power to zero\. Thus the cited power\-law asymptotics and our regularized scalar expression agree in their common bulk regime, rather than being identical finite\-horizon formulas\.

##### The backward\-rounding correction\.

The ratioq⁡\(1\+2​χ\)/Mq\(1\+2\\chi\)/Mis the backward\-rounding response relative to the exact\-SGD noise response\. It is largest at the beginning, whenχ=τ\\chi=\\tau, and decreases as training progresses\. This is a relative response comparison, not a claim that the earliest updates contribute the largest absolute error\. Ifq⁡\(1\+2​τ\)/M≪1q\(1\+2\\tau\)/M\\ll 1, the correction is uniformly small\. If it is much larger than one on an uncapped interval, then

η⁡\(t\)≈λbw​M/q​\[1\+2​χ​\(t\)\]\(a−1\)/\(2​a\)\.\\eta\(t\)\\approx\\sqrt\{\\lambda\_\{\\mathrm\{bw\}\}M/q\}\\,\[1\+2\\chi\(t\)\]^\{\(a\-1\)/\(2a\)\}\.For large remaining cumulative learning rate, integratingd​χ/d​t=−ηd\\chi/dt=\-\\etamakesχ\(a\+1\)/\(2​a\)\\chi^\{\(a\+1\)/\(2a\)\}approximately affine in training time\. Thus the corresponding rate has decay power\(a−1\)/\(a\+1\)\(a\-1\)/\(a\+1\)\. The zero point extrapolated by this bulk approximation need not coincide with the actual training endpoint\. Both dominance and absence of clipping are required\. The approximation stops applying as the response ratio approaches one; the full expression \([31](https://arxiv.org/html/2610.41074#S4.E31)\) describes that transition and the endpoint\. The normalization and selected total rate also change with precision, so this argument does not imply a universally lower peak or an earlier decay start\.

### E\.5Compute allocation: fixed and growing precisions

The compute\-optimal fixed\-precision laws are proved in Proposition[D\.5](https://arxiv.org/html/2610.41074#Thmtheorem5b), with variable\-constant\-rate extensions in Propositions[D\.6](https://arxiv.org/html/2610.41074#Thmtheorem6b)and[D\.7](https://arxiv.org/html/2610.41074#Thmtheorem7b)\. Here Proposition[E\.8](https://arxiv.org/html/2610.41074#Thmtheorem8a)gives the growing\-precision construction, and Theorem[E\.9](https://arxiv.org/html/2610.41074#Thmtheorem9a)treats exact precision\. We also explain the allocation comparisons used in Section[4\.5](https://arxiv.org/html/2610.41074#S4.SS5)\. The cost is \([23](https://arxiv.org/html/2610.41074#S3.E23)\); for constant schedules, use its width–clock form from Section[4\.5](https://arxiv.org/html/2610.41074#S4.SS5)andB​T=τ/ρBT=\\tau/\\rho\. Producing featuresS​xSxby dense multiplication would separately cost orderM​dMdper example\. The declared linear\-width charge assumes this work is supplied, precomputed or accounted for separately; including it can change the compute exponents\.

Fix an integerB≥1B\\geq 1,0<η≤min⁡\{c0/Cg,B​ρsch\}0<\\eta\\leq\\min\\\{c\_\{0\}/C\_\{g\},B\\rho\_\{\\rm sch\}\\\}and a relative\-risk toleranceϑ\>0\\vartheta\>0\. Use uniform averaging andTM=⌊Ma/η⌋T\_\{M\}=\\lfloor M^\{a\}/\\eta\\rfloor\. Both mantissas are finite integers, constant within a run but allowed to depend onMM\. WritefMf\_\{M\}for the charge \([23](https://arxiv.org/html/2610.41074#S3.E23)\)\.

###### Proposition E\.8\(Growing precisions for uniform averaging\)\.

For sufficiently large eligibleMM, an admissible run satisfyingΔavg≤ϑ​ℛ0avg\\Delta^\{\\rm avg\}\\leq\\vartheta\\mathscr\{R\}\_\{0\}^\{\\rm avg\}necessarily obeys

pbw≥b−12​log2​M−O⁡\(1\),pfw\+pbw≥a−12​log2​M−O⁡\(1\)\.p\_\{\\mathrm\{bw\}\}\\geq\\frac\{b\-1\}\{2\}\\log\_\{2\}M\-O\(1\),\\qquad p\_\{\\mathrm\{fw\}\}\+p\_\{\\mathrm\{bw\}\}\\geq\\frac\{a\-1\}\{2\}\\log\_\{2\}M\-O\(1\)\.\(227\)For sufficiently large fixedCfw,CbwC\_\{\\mathrm\{fw\}\},C\_\{\\mathrm\{bw\}\}, the choice

pbw=⌈b−12​log2​M\+Cbw⌉,pfw=⌈a−b2​log2​M\+Cfw⌉p\_\{\\mathrm\{bw\}\}=\\left\\lceil\\frac\{b\-1\}\{2\}\\log\_\{2\}M\+C\_\{\\mathrm\{bw\}\}\\right\\rceil,\\qquad p\_\{\\mathrm\{fw\}\}=\\left\\lceil\\frac\{a\-b\}\{2\}\\log\_\{2\}M\+C\_\{\\mathrm\{fw\}\}\\right\\rceil\(228\)satisfies this tolerance and the full constant\-rate conditions\. Among admissible finite\-bit runs satisfying the tolerance, the least charge at widthMMis≍Ma\+1​log⁡M\\asymp M^\{a\+1\}\\log M\. This split has full risk≍M1−b\\asymp M^\{1\-b\}and attains \([32](https://arxiv.org/html/2610.41074#S4.E32)\) under budgetFF\.

###### Proof\.

For largeMM,Ma/2≤η​TM≤MaM^\{a\}/2\\leq\\eta T\_\{M\}\\leq M^\{a\}, so \([19](https://arxiv.org/html/2610.41074#S3.E19)\) givesℛ0avg≍M1−b\\mathscr\{R\}\_\{0\}^\{\\rm avg\}\\asymp M^\{1\-b\}\. The backward and joint lower comparisons in Corollary[E\.5](https://arxiv.org/html/2610.41074#Thmtheorem5c)requireqbw≲M1−bq\_\{\\mathrm\{bw\}\}\\lesssim M^\{1\-b\}andqfw​qbw≲M1−aq\_\{\\mathrm\{fw\}\}q\_\{\\mathrm\{bw\}\}\\lesssim M^\{1\-a\}\. Taking logarithms proves \([227](https://arxiv.org/html/2610.41074#A5.E227)\)\. For \([228](https://arxiv.org/html/2610.41074#A5.E228)\),qbw≤4−Cbw​M1−bq\_\{\\mathrm\{bw\}\}\\leq 4^\{\-C\_\{\\mathrm\{bw\}\}\}M^\{1\-b\},qfw≤4−Cfw​Mb−aq\_\{\\mathrm\{fw\}\}\\leq 4^\{\-C\_\{\\mathrm\{fw\}\}\}M^\{b\-a\}andqfw​qbw≤4−\(Cfw\+Cbw\)​M1−aq\_\{\\mathrm\{fw\}\}q\_\{\\mathrm\{bw\}\}\\leq 4^\{\-\(C\_\{\\mathrm\{fw\}\}\+C\_\{\\mathrm\{bw\}\}\)\}M^\{1\-a\}\. Substitution in \([21](https://arxiv.org/html/2610.41074#S3.E21)\) gives the desired tolerance once the offsets are large enough\. Enlarging their sum also makesρ​qfw​qbw​τ/M≤ρ​4−\(Cfw\+Cbw\)≤κc\\rho q\_\{\\mathrm\{fw\}\}q\_\{\\mathrm\{bw\}\}\\tau/M\\leq\\rho 4^\{\-\(C\_\{\\mathrm\{fw\}\}\+C\_\{\\mathrm\{bw\}\}\)\}\\leq\\kappa\_\{\\rm c\}\. Thus \([22](https://arxiv.org/html/2610.41074#S3.E22)\), the stipulated rate and sufficiently large widths verify full admissibility\. The necessary summed\-bit bound gives the cost lower bound\. The displayed split matches it: its total mantissa length isO⁡\(log⁡M\)O\(\\log M\), the finite\-precision cost is affine, andB​TM≍Ma/ρBT\_\{M\}\\asymp M^\{a\}/\\rho\. Hence the least charge is≍Ma\+1​log⁡M\\asymp M^\{a\+1\}\\log M, with full risk≍M1−b\\asymp M^\{1\-b\}for this split\. ChooseMMat a sufficiently small fixed multiple of\(F/log⁡F\)1/\(a\+1\)\(F/\\log F\)^\{1/\(a\+1\)\}, rounded down to an integer, and useTMT\_\{M\}above\. The charge is at mostFFand the risk has order \([32](https://arxiv.org/html/2610.41074#S4.E32)\)\. ∎

#### E\.5\.1The optimum when exact precision is allowed

##### Comparison classes\.

A policy chooses integerM,T,B≥1M,T,B\\geq 1, deterministic nonnegative rates, precisions inℕ0∪\{∞\}\\mathbb\{N\}\_\{0\}\\cup\\\{\\infty\\\}, and either the last iterate or uniform average, before all random draws\. Require both width thresholds, a nonempty eventΩM\\Omega\_\{M\}and cost at mostFF\. Let𝔓F\\mathfrak\{P\}\_\{F\}consist of these policies satisfying the full scheduled class throughoutΩM\\Omega\_\{M\}\. For the lower bound only, enlarge this class by keeping the pointwise capηu​g1​\(S\)≤c0\\eta\_\{u\}g\_\{1\}\(S\)\\leq c\_\{0\}and dropping the rate ceiling, the lower and upper bounds on cumulative learning rate, and feedback restrictions\. We call this the capped comparison class; no upper scaling law is needed on the enlarged class\.

##### Integer construction\.

LetπF\\pi\_\{F\}use uniform averaging, exact precision at both locations,B=1B=1and constant learning rateη0\\eta\_\{0\}, with

η0=12​min⁡\{c0/Cg,ρsch\},MF=⌊\(η0​F/4\)1/\(a\+1\)⌋,TF=⌊MFa/η0⌋\.\\eta\_\{0\}=\\tfrac\{1\}\{2\}\\min\\\{c\_\{0\}/C\_\{g\},\\rho\_\{\\rm sch\}\\\},\\qquad M\_\{F\}=\\left\\lfloor\(\\eta\_\{0\}F/4\)^\{1/\(a\+1\)\}\\right\\rfloor,\\qquad T\_\{F\}=\\left\\lfloor M\_\{F\}^\{a\}/\\eta\_\{0\}\\right\\rfloor\.\(229\)
The following theorem identifies the best possible risk exponent under the declared compute model when exact precision is allowed\. Its lower bound covers both outputs, and a uniform\-average construction attains the order\. Exact backward features with finite forward precision can also attain the same order\.

###### Theorem E\.9\(Compute\-optimal risk when exact precision is allowed\)\.

Letγ=\(b−1\)/\(a\+1\)\\gamma=\(b\-1\)/\(a\+1\)\. There are fixedc,C\>0c,C\>0such that, for sufficiently largeFF, every policy in the capped comparison class has conditional risk at leastc​F−γcF^\{\-\\gamma\}on its sketch event, for either output\. The construction \([229](https://arxiv.org/html/2610.41074#A5.E229)\) belongs to𝔓F\\mathfrak\{P\}\_\{F\}and attains the matching upper bound\. Thus, for any fixed choiceSM∈ΩMS\_\{M\}\\in\\Omega\_\{M\}at each feasible width,

c​F−γ≤infπ∈𝔓FℛSM⁡\(π\)​\(π\)≤ℛSMF​\(πF\)≤C​F−γ\.cF^\{\-\\gamma\}\\leq\\inf\_\{\\pi\\in\\mathfrak\{P\}\_\{F\}\}\\mathscr\{R\}\_\{S\_\{M\(\\pi\)\}\}\(\\pi\)\\leq\\mathscr\{R\}\_\{S\_\{M\_\{F\}\}\}\(\\pi\_\{F\}\)\\leq CF^\{\-\\gamma\}\.\(230\)Fixed finite forward precision with exact backward features also attains this order: withW=1\+c⁡\(pfw\)W=1\+c\(p\_\{\\mathrm\{fw\}\}\), replaceMFM\_\{F\}by⌊\(η0​F/\(2​W\)\)1/\(a\+1\)⌋\\lfloor\(\\eta\_\{0\}F/\(2W\)\)^\{1/\(a\+1\)\}\\rfloorand useTF=⌊MFa/η0⌋T\_\{F\}=\\lfloor M\_\{F\}^\{a\}/\\eta\_\{0\}\\rfloorat that width\. A sufficiently large fixedpfwp\_\{\\mathrm\{fw\}\}meets any fixed positive relative SR\-error tolerance\.

The same order holds for every class containing this attainer and contained in the capped comparison class\. All comparisons are conditional on the event at each width, without a joint event over widths\.

###### Proof\.

Setm0=min⁡\{M,\(1\+τT\)1/a\}m\_\{0\}=\\min\\\{M,\(1\+\\tau\_\{T\}\)^\{1/a\}\\\}\. Unbiasedness, conditional Jensen inequality and \([67](https://arxiv.org/html/2610.41074#A2.E67)\) imply

ℛout≥A¯\+12​tr⁡\(G​Pout​V​Pout\)≥A¯\+cV2​∑jgjb/a​dout,j2\.\\mathscr\{R\}^\{\{\\rm out\}\}\\geq\\bar\{A\}\+\\tfrac\{1\}\{2\}\\operatorname\{tr\}\(GP\_\{\{\\rm out\}\}VP\_\{\{\\rm out\}\}\)\\geq\\bar\{A\}\+\\tfrac\{c\_\{V\}\}\{2\}\\sum\_\{j\}g\_\{j\}^\{b/a\}d\_\{\{\\rm out\},j\}^\{2\}\.\(231\)For either output, the propagation coefficients in \([72](https://arxiv.org/html/2610.41074#A3.E72)\) satisfydout,j≥πj​\(T,0\)d\_\{\{\\rm out\},j\}\\geq\\pi\_\{j\}\(T,0\)\. We claim

ℛout≥c∗​m01−b\\mathscr\{R\}^\{\{\\rm out\}\}\\geq c\_\{\*\}m\_\{0\}^\{1\-b\}\(232\)for fixedc∗\>0c\_\{\*\}\>0\. Ifm0≥M/4m\_\{0\}\\geq M/4, useA¯≥cA​M1−b\\bar\{A\}\\geq c\_\{A\}M^\{1\-b\}from \([69](https://arxiv.org/html/2610.41074#A2.E69)\)\. Otherwisem0=\(1\+τT\)1/am\_\{0\}=\(1\+\\tau\_\{T\}\)^\{1/a\}\. The block⌈m0⌉≤j<2​⌈m0⌉\\lceil m\_\{0\}\\rceil\\leq j<2\\lceil m\_\{0\}\\rceillies belowMMand contains at leastm0m\_\{0\}indices\. Theregj​τT≤Cgg\_\{j\}\\tau\_\{T\}\\leq C\_\{g\}andgjb/a≳m0−bg\_\{j\}^\{b/a\}\\gtrsim m\_\{0\}^\{\-b\}\. The global product bound \([89](https://arxiv.org/html/2610.41074#A3.E89)\) makes their sum in \([231](https://arxiv.org/html/2610.41074#A5.E231)\) at least a fixed multiple ofm01−bm\_\{0\}^\{1\-b\}, including whenτT=0\\tau\_\{T\}=0\.

Setw∗=2​min⁡\{1,p0/\(pref\+p0\)\}\>0w\_\{\*\}=2\\min\\\{1,p\_\{0\}/\(p\_\{\\rm ref\}\+p\_\{0\}\)\\\}\>0\. The cost and rate cap givef≥w∗​M​Tf\\geq w\_\{\*\}MTandτT≤η¯​T\\tau\_\{T\}\\leq\\bar\{\\eta\}T, whereη¯=c0/cg\\bar\{\\eta\}=c\_\{0\}/c\_\{g\}\. Consequently,

m0a\+1≤M⁡\(1\+τT\)≤\(1\+η¯\)​M​T≤\(1\+η¯\)​F/w∗\.m\_\{0\}^\{a\+1\}\\leq M\(1\+\\tau\_\{T\}\)\\leq\(1\+\\bar\{\\eta\}\)MT\\leq\(1\+\\bar\{\\eta\}\)F/w\_\{\*\}\.Since1−b<01\-b<0, \([232](https://arxiv.org/html/2610.41074#A5.E232)\) proves the lower order\.

For sufficiently largeFF,MFa/2≤η0​TF≤MFaM\_\{F\}^\{a\}/2\\leq\\eta\_\{0\}T\_\{F\}\\leq M\_\{F\}^\{a\}\. Both width thresholds, the minimum cumulative learning rate, the rate cap and rate ceiling hold; the feedback gains vanish\. The sketch events are nonempty by \([68](https://arxiv.org/html/2610.41074#A2.E68)\), and2​MF​TF≤2​MFa\+1/η0≤F/22M\_\{F\}T\_\{F\}\\leq 2M\_\{F\}^\{a\+1\}/\\eta\_\{0\}\\leq F/2\. Equation \([19](https://arxiv.org/html/2610.41074#S3.E19)\) gives full averaged riskO⁡\(MF1−b\)=O⁡\(F−γ\)O\(M\_\{F\}^\{1\-b\}\)=O\(F^\{\-\\gamma\}\)\.

With finite forward and exact backward precision, the gains still vanish and the modified width gives cost at mostF/2F/2\. Equation \([21](https://arxiv.org/html/2610.41074#S3.E21)\) gives forward SR\-induced errorO⁡\(qfw​MF1−b\)O\(q\_\{\\mathrm\{fw\}\}M\_\{F\}^\{1\-b\}\), so a sufficiently small fixedqfwq\_\{\\mathrm\{fw\}\}meets the prescribed tolerance\. The pointwise lower bound and uniform attainment prove \([230](https://arxiv.org/html/2610.41074#A5.E230)\) and the intermediate\-class claim\. ∎

##### Unequal precisions and the width requirement\.

The sufficient checks \([210](https://arxiv.org/html/2610.41074#A5.E210)\) give \([233](https://arxiv.org/html/2610.41074#A5.E233)\) by using its three leading width demands\.

M≳max⁡\{τ1/a,ρ​σ2​qbwϑ​τb/a,ρ​qfw​qbwκc​τ\}\.M\\gtrsim\\max\\left\\\{\\tau^\{1/a\},\\ \\frac\{\\rho\\sigma^\{2\}q\_\{\\mathrm\{bw\}\}\}\{\\vartheta\}\\tau^\{b/a\},\\ \\frac\{\\rho q\_\{\\mathrm\{fw\}\}q\_\{\\mathrm\{bw\}\}\}\{\\kappa\_\{\\rm c\}\}\\tau\\right\\\}\.\(233\)At fixed total mantissa length, the affine finite\-precision cost andqfw​qbwq\_\{\\mathrm\{fw\}\}q\_\{\\mathrm\{bw\}\}stay unchanged\. Reallocating one bit from forward to backward precision divides the direct backward comparison term by four and multiplies the direct forward term by four\.

The two inequalities in \([227](https://arxiv.org/html/2610.41074#A5.E227)\) do not require both mantissas to grow separately\. A sufficiently large fixed forward mantissa controls its direct term; growing the backward mantissa as\(a−1\)​log4​M\+O⁡\(1\)\(a\-1\)\\log\_\{4\}M\+O\(1\)then meets both the backward and summed\-bit tests\. The split \([228](https://arxiv.org/html/2610.41074#A5.E228)\) is another sufficient choice\. These matched, constant\-rate constructions do not establish the global optimality of the logarithmic factor for all\-finite policies\. The exact\-allowed benchmark and growing\-finite construction use different precision families, and normalized exact cost is a modeling choice rather than a hardware claim\. Finite precision costs less than the unit\-priced exact option only whenc⁡\(p\)<1c\(p\)<1, equivalentlyp<prefp<p\_\{\\rm ref\}\. Growing mantissas eventually cost more than that option, so the construction establishes an achievable finite\-only rate rather than an asymptotic cost advantage over exact arithmetic\.

##### Reducing the learning\-rate\-to\-batch\-size ratio\.

The integer construction \([196](https://arxiv.org/html/2610.41074#A4.E196)\) also gives averaged attainers when evaluated using \([19](https://arxiv.org/html/2610.41074#S3.E19)\) and \([21](https://arxiv.org/html/2610.41074#S3.E21)\)\. With backward\-only rounding taker=br=b:ρ≍M1−b\\rho\\asymp M^\{1\-b\}, risk≍M1−b\\asymp M^\{1\-b\}andf≍Ma\+bf\\asymp M^\{a\+b\}\. With both scales positive taker=ar=a:ρ≍M1−a\\rho\\asymp M^\{1\-a\}, risk≍M1−b\\asymp M^\{1\-b\}andf≍M2​af\\asymp M^\{2a\}\. To check the latter, the averaged backward, forward and joint terms are respectivelyO⁡\(M1−a\)O\(M^\{1\-a\}\),O⁡\(M2−a−b\)O\(M^\{2\-a\-b\}\)andO⁡\(M1−b\)O\(M^\{1\-b\}\)\. The cited construction verifies all width, clock, rate and feedback conditions\. These policies reduce the rate ratio at matched clock; the fixed\-rate attainers instead increase width for a given clock\. SinceB​T=τ/ρBT=\\tau/\\rho, neither is a free reduction of stochastic noise\. At fixedM,τM,\\tauand precisions, halvingρ\\rhodoubles both the number of sampled examples and the modeled compute cost\.

#### E\.5\.2Prediction risk at a fixed compute budget

Figure[3](https://arxiv.org/html/2610.41074#S4.F3)\(c\) fixesd=1024d=1024,λj=j−2\\lambda\_\{j\}=j^\{\-2\},λjϑj=j−3/2\\lambda\_\{j\}\\vartheta\_\{j\}=j^\{\-3/2\},σ2=0\.1\\sigma^\{2\}=0\.1,B=32B=32andη=0\.2\\eta=0\.2, with zero initialization\. The input distribution and teacher distribution are common across widths\. We evaluate the prediction risk of the uniformly averaged iterate, including approximation error, conditional on each sketch and averaged over the teacher, training data and SR, using Proposition[B\.3](https://arxiv.org/html/2610.41074#Thmtheorem3)and \([52](https://arxiv.org/html/2610.41074#A2.E52)\)\.

Use \([23](https://arxiv.org/html/2610.41074#S3.E23)\) withp0=9p\_\{0\}=9,pref=7p\_\{\\rm ref\}=7and budgetF=218,103,808F=218\{,\}103\{,\}808\. For each precision pair, we test various widths\. At every width, minimizing risk over positive integer durations satisfyingf≤Ff\\leq Fandη​T≤M2\\eta T\\leq M^\{2\}selects the longest feasible duration for all three sketches\. Minimizing the mean risk across these widths, we can get the following table\.

Each entry in last column averages the three trials\. All selected training configurations meet the same budget\. Joint one\-bit SR\(1,1\)\(1,1\)permits9\.1%9\.1\\%greater width and46\.7%46\.7\\%more updates, reducing mean risk by9\.7%9\.7\\%relative to the selected exact\-arithmetic training configuration\. This illustrates the principle of Section[4\.5](https://arxiv.org/html/2610.41074#S4.SS5): select precision jointly with width and training duration\. The comparison underlying \([33](https://arxiv.org/html/2610.41074#S4.E33)\) extends to different widths by evaluating each exact\-arithmetic risk and SR\-induced error at its own width and duration\.

## Appendix FExperimental details

This appendix gives experimental settings, risk estimators and uncertainty calculations\. Appendices[F\.1](https://arxiv.org/html/2610.41074#A6.SS1)–[F\.2](https://arxiv.org/html/2610.41074#A6.SS2)describe Figures[1](https://arxiv.org/html/2610.41074#S1.F1)–[2](https://arxiv.org/html/2610.41074#S4.F2); Appendix[F\.3](https://arxiv.org/html/2610.41074#A6.SS3.SSS0.Px1)describes the compute and data\-reuse comparisons in Figures[3](https://arxiv.org/html/2610.41074#S4.F3)–[4](https://arxiv.org/html/2610.41074#S5.F4); Appendix[F\.4](https://arxiv.org/html/2610.41074#A6.SS4)gives the supplementary controls in Figure[6](https://arxiv.org/html/2610.41074#A6.F6); and Appendix[F\.5](https://arxiv.org/html/2610.41074#A6.SS5)describes the neural\-network experiments in Figure[5](https://arxiv.org/html/2610.41074#S5.F5)\.

### F\.1Rounding location, timing, and precision requirements

We use \([5](https://arxiv.org/html/2610.41074#S2.E5)\) witha=2a=2,b=3/2b=3/2,σ2=0\.1\\sigma^\{2\}=0\.1,B=32B=32,d=4​Md=4M, fresh data and independent feature rounding unless stated otherwise\. The exact\-arithmetic evaluation is \([10](https://arxiv.org/html/2610.41074#S2.E10)\), including approximation error\. Below we first show the experimental detials of Figure[1](https://arxiv.org/html/2610.41074#S1.F1)\.

##### Rounding location\.

Figure[1](https://arxiv.org/html/2610.41074#S1.F1)\(a\) uses one\-bit precision at one location and exact computation at the other, evaluated at the recorded step nearestτT=Ma\\tau\_\{T\}=M^\{a\}\. We present SR\-induced error of each output divided by its own exact\-arithmetic risk\. Risks average over teachers and training randomness for each sketch; curves show medians across three sketches per width\.

##### Precision timing\.

Figure[1](https://arxiv.org/html/2610.41074#S1.F1)\(b\) fixesM=128M=128andT=31,715T=31\{,\}715, switching between one and three bits after update15,85715\{,\}857\. One bit followed by three uses63,43163\{,\}431mantissa bit\-updates; the reverse order uses63,42963\{,\}429, so average precisions are nearly equal\. Forward\-rounding tests keep backward precision at four bits; backward\-rounding tests keep forward features exact\. We present the SR\-induced error with three bits early divided by that with three bits late\.

##### Precision requirements\.

Figure[1](https://arxiv.org/html/2610.41074#S1.F1)\(c\) usesM=16,32,64,128,256,512M=16,32,64,128,256,512andpfw=pbw∈\{1,2,3,4\}p\_\{\\mathrm\{fw\}\}=p\_\{\\mathrm\{bw\}\}\\in\\\{1,2,3,4\\\}\. At each width, precisions share the same learning rate and training time\. At the step nearestτT=M2\\tau\_\{T\}=M^\{2\}, we select the smallest tested precision satisfying accuracy tolerance in \([27](https://arxiv.org/html/2610.41074#S4.E27)\) for the last iterate withϑ=1%\\vartheta=1\\%\. Training duration grows with width\. The table also gives requirements for two other tolerances; “None” means no tested precision succeeds\.

### F\.2Joint precision allocation and learning\-rate design

Figure[2](https://arxiv.org/html/2610.41074#S4.F2)\(a\) fixesM=128M=128,d=512d=512,B=32B=32andη=0\.51660418\\eta=0\.51660418\. Choose stepT=31,424T=31\{,\}424, nearestτT=M2\\tau\_\{T\}=M^\{2\}, we compare\(pfw,pbw\)=\(1,4\),\(2,3\),\(3,2\),\(4,1\)\(p\_\{\\mathrm\{fw\}\},p\_\{\\mathrm\{bw\}\}\)=\(1,4\),\(2,3\),\(3,2\),\(4,1\)at equal compute under \([23](https://arxiv.org/html/2610.41074#S3.E23)\)\. We show SR\-induced error of tested precisions divided by its exact\-arithmetic risk for each case\.

Figure[2](https://arxiv.org/html/2610.41074#S4.F2)\(b,c\) usesM=32M=32,d=128d=128,B=4B=4,T=4096T=4096, last\-iterate output, exact forward features andpbw∈\{1,2,4,∞\}p\_\{\\mathrm\{bw\}\}\\in\\\{1,2,4,\\infty\\\}\. The learning\-rate upper boundηmax=0\.2\\eta\_\{\\max\}=0\.2is chosen before drawing sketches\. We average \([30](https://arxiv.org/html/2610.41074#S4.E30)\) and \([31](https://arxiv.org/html/2610.41074#S4.E31)\) over unit update intervals\. Figure[2](https://arxiv.org/html/2610.41074#S4.F2)\(b\) shows the one\-bit precision\-dependent schedule, the exact\-SGD schedule design, and the constant learning\-rate baselineηt=τT/T=512/4096=1/8\\eta\_\{t\}=\\tau\_\{T\}/T=512/4096=1/8, all atτT=512\\tau\_\{T\}=512\. The exact\-SGD design is constructed using \([30](https://arxiv.org/html/2610.41074#S4.E30)\); its name describes the schedule, rather than the training precision\.

Figure[2](https://arxiv.org/html/2610.41074#S4.F2)\(c\) compares the two learning\-rate schedules illustrated in Figure[2](https://arxiv.org/html/2610.41074#S4.F2)\(b\), with both schedules evaluated at the same backward precision\. A regression problem consists of one randomly drawn sketch and teacher, which remain fixed during repeated training\. For each schedule and precision, we testτT∈\{16,32,64,128,256,512\}\\tau\_\{T\}\\in\\\{16,32,64,128,256,512\\\}on four independent problems, using 16 training runs per problem for each candidate\. We choose the total learning rate with the smallest mean last\-iterate prediction risk across these problems and runs\.

For the plotted comparison, we generate eight new independent problems that were not used for tuning\. For each problem, we train with each selected schedule 64 times\. Each comparison matches one run of each schedule, using the same batches, label noises and random numbers for rounding\. Each repetition uses new training data and random draws\. The randomness for data, label noise and rounding is mutually independent\. For each comparison, we subtract the final prediction risk of the exact\-SGD schedule from that of the precision\-dependent schedule, both measured by \([9](https://arxiv.org/html/2610.41074#S2.E9)\)\. We average the 64 differences to obtain one result for each problem\. The plotted point is the average of these eight results\. We compute the standard error as their sample standard deviation divided by8\\sqrt\{8\}; bars extend twice this amount above and below the point\. Positive values favor the exact\-SGD schedule\. These experiments lie outside the conservative learning\-rate class of Assumption[C\.1](https://arxiv.org/html/2610.41074#Thmtheorem1b), and do not directly test Theorem[E\.7](https://arxiv.org/html/2610.41074#Thmtheorem7c)or establish a global optimum\.

### F\.3Compute\-budget comparisons

We describe the width and duration comparisons in Figure[3](https://arxiv.org/html/2610.41074#S4.F3)and the fixed\-width data\-reuse comparisons in Figure[4](https://arxiv.org/html/2610.41074#S5.F4)\. All compute comparisons use \([23](https://arxiv.org/html/2610.41074#S3.E23)\) withp0=9p\_\{0\}=9andpref=7p\_\{\\rm ref\}=7\.

##### Width and training duration\.

Figure[3](https://arxiv.org/html/2610.41074#S4.F3)\(a,b\) usesd=4​Md=4M,η=0\.5/g1​\(S\)\\eta=0\.5/g\_\{1\}\(S\)and training to approximatelyτT=8​M2\\tau\_\{T\}=8M^\{2\}\. Colored curves show uniformly averaged prediction risk as training continues\. At each budget, the black curve selects the smallest risk among tested widths and recorded durations within that budget\. The red dashed curve in Figure[3](https://arxiv.org/html/2610.41074#S4.F3)\(a\) fits the exact\-arithmetic compute scaling,F−1/6F^\{\-1/6\}, in Table[1](https://arxiv.org/html/2610.41074#S3.T1), adjusting only its multiplicative coefficient\. Figure[3](https://arxiv.org/html/2610.41074#S4.F3)\(b\) reuses that coefficient and accounts for the update cost falling from2​B​M2BMto5​B​M/45BM/4, while ignoring SR\-induced error\. Both curves are presented6%6\\%lower for readability\.

Figure[3](https://arxiv.org/html/2610.41074#S4.F3)\(c\) uses the fixed task and three sketches of Appendix[E\.5\.2](https://arxiv.org/html/2610.41074#A5.SS5.SSS2)\. Widths and durations are selected using their mean prediction risk\. At some small widths,η​T≤M2\\eta T\\leq M^\{2\}ends training before the budget is used\. Forward\-only and backward\-only one\-bit SR give nearly coincident curves in this comparison\.

##### Fresh and reused data at fixed width\.

Figure[4](https://arxiv.org/html/2610.41074#S5.F4)fixesM=256M=256,d=1024d=1024,B=32B=32andη=0\.2\\eta=0\.2\. We compare exact arithmetic with independent SR for forward and backward features atpfw=pbw=2p\_\{\\mathrm\{fw\}\}=p\_\{\\mathrm\{bw\}\}=2; all other computations are exact\. Each of eight independent sketch–teacher–data tasks has 16 SR replicas sharing its data and access order\. Rounding streams are matched across policies and data conditions within a task\. Training uses fresh online batches or a fixed dataset of 2,048 or 16,384 examples\. For the reused\-data setting, after the first pass, existing examples are reshuffled without regenerating inputs or labels\. All plotted reused\-data runs complete the first pass\. Thus fresh and reused training differ in data availability as well as repeated access\.

All risks use the uniform average outputs\. Under \([23](https://arxiv.org/html/2610.41074#S3.E23)\), each exact\-arithmetic update costs1638416384compute, and each update with two\-bit SR at both feature locations costs1126411264compute\. We useFFas a total compute ceiling\. Prediction risks were saved only at selected update counts, so for each method we take the latest saved evaluation whose total cost is at mostFF\. The table gives the selected update counts and the percentage ofFFactually used:

For example, atF=227F=2^\{27\}, exact arithmetic training uses81928192updates and exhausts the budget\. SR can afford11,91511\{,\}915updates, but its latest saved evaluation within the budget is at99619961updates, using only83\.60%83\.60\\%\. The unused budget reflects the spacing of saved evaluations\. Figure[4](https://arxiv.org/html/2610.41074#S5.F4)therefore compares available checkpoints under a the same compute budget, with unequal actual compute use\.

### F\.4Additional rounding locations and precision allocation

Figure[6](https://arxiv.org/html/2610.41074#A6.F6)provides additional rounding\-location controls and an integer\-precision allocation comparison, with protocols given below\.

Figure 6:Supplementary precision comparisons\. Figure[6](https://arxiv.org/html/2610.41074#A6.F6)\(a,b\) shows SR\-induced risk increases for last\-iterate and uniformly averaged outputs at fixed duration, with fresh or reused data\. The four categories use feature rounding alone, add temporary prediction\-weight rounding, add stored\-parameter rounding, or round at all six locations, as defined below\. The vertical axes use different logarithmic ranges; bars show twice the standard error across eight task means\. Figure[6](https://arxiv.org/html/2610.41074#A6.F6)\(c\) compares four integer precision schedules with constant two\-bit precision at the same feature location\. Mean bit counts are shown; the realized budgets differ\.##### Rounding locations and update order\.

Figure[6](https://arxiv.org/html/2610.41074#A6.F6)\(a,b\) uses the tasks and three data conditions of Figure[4](https://arxiv.org/html/2610.41074#S5.F4), withM=256M=256,d=1024d=1024,B=32B=32,ηu=0\.2\\eta\_\{u\}=0\.2andT=16384T=16384\. Training uses fresh batches or fixed datasets of8​M=20488M=2048or64​M=1638464M=16384samples, reshuffled for each subsequent pass with inputs and labels retained\. Each of eight independent sketch–teacher–data tasks has 16 SR replicas\.

Recallvuv\_\{u\},au​ℓa\_\{u\\ell\},bu​ℓb\_\{u\\ell\}andQpQ\_\{p\}from \([4](https://arxiv.org/html/2610.41074#S2.E4)\)–\([5](https://arxiv.org/html/2610.41074#S2.E5)\), withv0=0v\_\{0\}=0andQ∞​\(x\)=xQ\_\{\\infty\}\(x\)=x\. Four additional precisionsppred,pres,pgrad,pstorep\_\{\\rm pred\},p\_\{\\rm res\},p\_\{\\rm grad\},p\_\{\\rm store\}specify rounding of prediction weights, residuals, minibatch gradients and stored parameters, respectively\. All five training consider the following quantizations:

au​ℓ\\displaystyle a\_\{u\\ell\}=Qpfw,u​\(S​xu​ℓ\),\\displaystyle=Q\_\{p\_\{\\mathrm\{fw\},u\}\}\(Sx\_\{u\\ell\}\),bu​ℓ\\displaystyle b\_\{u\\ell\}=Qpbw,u​\(S​xu​ℓ\),\\displaystyle=Q\_\{p\_\{\\mathrm\{bw\},u\}\}\(Sx\_\{u\\ell\}\),\(234\)v~u\\displaystyle\\widetilde\{v\}\_\{u\}=Qppred​\(vu\),\\displaystyle=Q\_\{p\_\{\\rm pred\}\}\(v\_\{u\}\),ru​ℓ\\displaystyle r\_\{u\\ell\}=Qpres​\(au​ℓ⊤​v~u−yu​ℓ\),\\displaystyle=Q\_\{p\_\{\\rm res\}\}\(a\_\{u\\ell\}^\{\\top\}\\widetilde\{v\}\_\{u\}\-y\_\{u\\ell\}\),g^u\\displaystyle\\widehat\{g\}\_\{u\}=Qpgrad​\(1B​∑ℓ=1Bbu​ℓ​ru​ℓ\),\\displaystyle=Q\_\{p\_\{\\rm grad\}\}\\\!\\left\(\\frac\{1\}\{B\}\\sum\_\{\\ell=1\}^\{B\}b\_\{u\\ell\}r\_\{u\\ell\}\\right\),vu\+1\\displaystyle v\_\{u\+1\}=Qpstore​\(vu−ηu​g^u\)\.\\displaystyle=Q\_\{p\_\{\\rm store\}\}\(v\_\{u\}\-\\eta\_\{u\}\\widehat\{g\}\_\{u\}\)\.The table lists the precision tuple\(pfw,u,pbw,u,ppred,pres,pgrad,pstore\)\(p\_\{\\mathrm\{fw\},u\},p\_\{\\mathrm\{bw\},u\},p\_\{\\rm pred\},p\_\{\\rm res\},p\_\{\\rm grad\},p\_\{\\rm store\}\)in each case:

The temporary copyv~u\\widetilde\{v\}\_\{u\}is formed once per update, shared by theBBpredictions, and discarded afterward; the update starts fromvuv\_\{u\}\. Storage rounding is applied after the update, and its result is carried to the next step\. Distinct rounding locations use independent draws, renewed each update; draws at the same location are matched across policies and data conditions\. Bit counts are mantissa precisions as in Section[2](https://arxiv.org/html/2610.41074#S2)\. Remaining arithmetic, accumulation and evaluation use float64\. No compute savings are assigned to the four additional rounding locations\.

##### Outputs and plotted risk increases\.

For every case we usevTlastv\_\{T\}^\{\\rm last\}andvTavgv\_\{T\}^\{\\rm avg\}from \([8](https://arxiv.org/html/2610.41074#S2.E8)\), without rounding the output average\. LetvTout,\(0\)v\_\{T\}^\{\{\\rm out\},\(0\)\}denote the exact\-arithmetic output on the same data and access order\. For each rounded output, the plotted estimate averages12​‖vTout−vTout,\(0\)‖G2,\\frac\{1\}\{2\}\\bigl\\\|v\_\{T\}^\{\{\\rm out\}\}\-v\_\{T\}^\{\{\\rm out\},\(0\)\}\\bigr\\\|\_\{G\}^\{2\},forout∈\{last,avg\}\{\\rm out\}\\in\\\{\\mathrm\{last\},\\mathrm\{avg\}\\\}, first over the 16 replicas within each task and then over the eight tasks\. This quantity estimates the rounding\-induced risk increase in \([10](https://arxiv.org/html/2610.41074#S2.E10)\)\. Risks are evaluated by \([9](https://arxiv.org/html/2610.41074#S2.E9)\)\. Smaller SR\-induced error across data conditions need not imply smaller full risk\.

##### Integer precision allocation\.

Figure[6](https://arxiv.org/html/2610.41074#A6.F6)\(c\) illustrates the precision\-allocation idea in \([26](https://arxiv.org/html/2610.41074#S4.E26)\)\. WithM=64M=64,B=32B=32,η=0\.5/g1\\eta=0\.5/g\_\{1\}andT=8869T=8869, teacher\-averaged second\-moment recursions for one fixed sketch compare time\-varying integer precision with constant two\-bit precision at either feature location, keeping the other exact, for both outputs\. The plotted ratios compare SR\-induced risk increases; realized bit budgets differ, so these comparisons do not establish equal\-budget optimality\.

### F\.5Neural\-network rounding locations and evaluation

Figure[5](https://arxiv.org/html/2610.41074#S5.F5)reports final transformer validation loss and MLP test loss, both cross\-entropy using natural logarithms\. Training without SR disables only the added experimental stochastic\-rounding operations\. Evaluation means making predictions and calculating validation or test loss after training, with parameters fixed and no further updates\. It disables the added SR operations unless stated otherwise; the underlying arithmetic described below remains unchanged\. Bit counts refer to mantissa precision, with unrestricted exponents\.

##### Models and training\.

The transformer has six layers, six heads, width 384, context 256 and 30,041,856 parameters\. It trains on approximately 200 million FineWeb\-Edu tokens for 12,207 updates, with batches of64×25664\\times 256tokens\. AdamW uses peak learning rate6×10−46\\times 10^\{\-4\}, coefficients\(0\.9,0\.95\)\(0\.9,0\.95\), weight decay0\.10\.1, gradient clipping at one and 100 warmup updates\. We use three seeds and 40 fixed validation batches\. All transformer runs use BF16/FP32 mixed\-precision arithmetic, with BF16 autocast enabled during both training and evaluation, including when added SR is disabled\. The CIFAR\-10 MLP has architecture30723072–10241024–1010with ReLU and FP32 computation\. It trains for 25 epochs \(9750 updates\), batch size 128, using SGD with momentum0\.90\.9, weight decay5×10−45\\times 10^\{\-4\}, peak learning rate0\.010\.01and 200 warmup updates\. We use 16 seeds and the full CIFAR\-10 test set\. Both models decrease the learning rate linearly during the final20%20\\%of training and report last\-iterate performance\.

##### Rounding locations\.

Figure[5](https://arxiv.org/html/2610.41074#S5.F5)\(a\) compares independent forward/backward activation rounding at two bits, incoming weight\-gradient signals at two bits, temporary forward weights at two bits, and parameter storage at seven bits\. The combined case uses two bits at all four operation locations and seven\-bit storage\. Optimizer states and the input\-gradient product use their original arithmetic\. Operation rounding affects linear layers except the tied language\-model head\. Storage rounding affects all parameters, including embeddings, normalization parameters and biases; disabling SR at evaluation does not undo earlier storage rounding\. These cases therefore differ in parameter coverage as well as rounding location\.

Table 3:Final transformer validation loss with added SR disabled at evaluation\. The table covers Figure[5](https://arxiv.org/html/2610.41074#S5.F5)\(a,b\) and a three\-bit storage comparison\.
##### Shared activation rounding\.

Figure[5](https://arxiv.org/html/2610.41074#S5.F5)\(b\) uses one or two bits at both activation computations\. Independent SR rounds the activation separately for forward and weight\-gradient products; shared SR reuses the forward\-rounded activation\. No other experimental SR is active\. Seeds match initialization, data order and evaluation examples\. Rounding draws differ because sharing uses fewer casts; the data stream is independent of those draws\. Each plotted loss difference subtracts same\-seed training without SR\. The direct shared\-minus\-independent transformer differences are−0\.02140±0\.00311\-0\.02140\\pm 0\.00311at one bit and−0\.00400±0\.00502\-0\.00400\\pm 0\.00502at two bits, with twice the paired standard error\. The two\-bit comparison does not clearly distinguish the methods\. Sharing changes dependence between rounding errors and can change the mean update, beyond Theorem[1](https://arxiv.org/html/2610.41074#Thmtheorem1)\.

##### Training precision and rounding at test time\.

Figure[5](https://arxiv.org/html/2610.41074#S5.F5)\(c\) uses one\-, two\- or three\-bit forward activation rounding during training\. Each trained MLP makes predictions on the same test examples twice, keeping its parameters fixed and making no updates: with forward\-activation SR disabled, and with SR enabled at its training precision\. Both tests use the MLP’s underlying FP32 arithmetic\. The dashed line uses training and evaluation without SR\. Accuracy in Table[4](https://arxiv.org/html/2610.41074#A6.T4)uses evaluation without SR\.

Table 4:Final MLP test performance for Figure[5](https://arxiv.org/html/2610.41074#S5.F5)\(b,c\)\. Entries are means±\\pmtwo standard errors across 16 seeds\. The two loss columns evaluate the same final parameters without or with SR; accuracy uses evaluation without SR\.