LEAN: Loss-Engineered Modulation of the Model Weight-Sensitivity Distribution for Efficient, Error-Tolerant Neural Networks

# LEAN: Loss-Engineered Modulation of the Model Weight-Sensitivity Distribution for Efficient, Error-Tolerant Neural Networks
Source: [https://arxiv.org/html/2610.28872](https://arxiv.org/html/2610.28872)
Habibur Rahaman and Swarup BhuniaAffiliation:Electrical and Computer Engineering, University of Florida Gainesville, Florida, USA rahaman\.habibur@ufl\.edu, swarup@ece\.ufl\.edu

###### Abstract

The weights of a trained network do not have uniform sensitivity to model accuracy: the 1% most sensitive weights carry about a fifth of the damage a bit flip can do\. This distribution decides the bit\-flip tolerance of a model, the fraction of weights a code must cover and the bits each weight needs; existing schemes work with the distribution training leaves\. We show that we can systematically modulate or alter the weight sensitivity distribution of a trained model, thereby shifting weights from one sensitivity bin to another\. The sensitivity of a stored weight is the product of its gradient and the step a flip takes in the storage format, andLEANadds a composite regularizer on both factors, a range term that bounds the flip step and a tail term that lowers the gradient of the most sensitive weights, with one coefficientλ\\lambdasetting how far the distribution moves\. Ten epochs with the term, as a fine\-tune or as the last epochs of training, shrink the sensitive tail 5 to 13 times on nine vision models \(ResNet\-18, ResNet\-50, VGG\-16 on CIFAR\-10, CIFAR\-100, ImageNet\) within 0\.5 points of accuracy; real flips confirm that the reshaped weights do less damage, a plain gradient\-norm penalty does not reproduce the effect, and a preliminary extension to three small language models shows the same tail shaping\. With a BCH code over 1% of the weights the model then survives 300 progressive bit\-search flips in every training seed and 500 in two of three, where three of four published defenses with the same code fail by 300 and WCAT falls to 75% at 500\. The reshaped distribution also cuts the parity for a fixed guarantee 3\.7 times and makes a sensitivity\-ordered 8/6/4/3\-bit format affordable, which stores ImageNet ResNet\-50 at a 3\.29\-bit weight payload, 2\.4 times smaller than INT8, above the accuracy of LSQ at 3 bits\.

###### Index Terms:

model parameter sensitivity distribution, weight sensitivity, shifting distribution, bit\-flip attack, error correcting codes, quantization, mixed precision, model efficiency, DNN accelerators, neural network, machine learning models

![Refer to caption](https://arxiv.org/html/2610.28872v1/figs/fig_teaser.png)Fig\. 1:Effect ofLEANon ResNet\-18, CIFAR\-10\. \(a\) The raw weight\-sensitivity distribution: number of weights per sensitivity bin \(40 log\-spaced bins of the sign\-flip damagedid\_\{i\}, as a percentage of the baseline maximum\); the bulk of the weights sits between 0\.01 and 1% and the HIGH band is a thin tail, whichLEANmoves to the left\. \(b\) The cumulative distribution, counted from the tail: the number of weights whose damage is at least a thresholdss; one ten\-epoch fine\-tune leaves 9 times fewer weights above the baseline HIGH threshold at the same accuracy, while Hessian\-aware training leaves the tail where normal training left it \(1\.01% against 1\.00% above the threshold\)\. \(c\) Accuracy under progressive bit search when the same 1% BCH code protects each model: the normally trained and the Hessian\-aware model fall to chance by 300 flips, theLEANmodel \(λ=40\\lambda=40\) keeps 94% after 500 in two of three training seeds and after 300 in all \(Table[V](https://arxiv.org/html/2610.28872#S4.T5)\)\.![Refer to caption](https://arxiv.org/html/2610.28872v1/figs/fig_overview.png)Fig\. 2:Overview\. Step 1: a normally trained network has a heavy\-tailed weight\-sensitivity distribution; the top 1% of weights \(HIGH\) carry about 20% of the first\-order damage of sign\-bit flips\. Step 2: ten epochs with the extra termλ​PLEAN\\lambda P\_\{\\mathrm\{LEAN\}\}, as a fine\-tune or as the last epochs of training, move weights out of HIGH and MID into LOW;λ\\lambdasets how far, and accuracy changes by at most 0\.5 points\. Step 3: the deployed model needs 3\.7 times less parity for the same guarantee, survives 300 to 500 progressive bit\-search flips with a 1% BCH code, and stores at 3\.3 bits per weight in a sensitivity\-ordered format\.## IIntroduction

Modern neural networks have tens of millions to billions of weights and they do not impact model functional behavior uniformly\. If a single stored weight is corrupted, by a memory fault, a bit\-flip attack or a coarse quantization step, the loss change depends on which weight it is, and it differs by a factor of 10,000 or more between the median weight and the most sensitive one\. On ResNet\-18 trained on CIFAR\-10, the 1% of weights with the largest first\-order sensitivity account for 19\.5% of the total loss change under sign\-bit flips, the next 8% for most of the rest, and the remaining 91% contribute little under this metric \(Figure[1](https://arxiv.org/html/2610.28872#S0.F1)a,b\)\. Hence, weight sensitivity is strongly non\-uniform, and its distribution has a large spread between the highest and the lowest sensitivity\.

The shape of that distribution influences the cost of the stored model in terms of error\-tolerance and computation efficiency\. It sets bit\-flip tolerance, because Rowhammer\[[1](https://arxiv.org/html/2610.28872#bib.bib1),[2](https://arxiv.org/html/2610.28872#bib.bib2)\], progressive bit search\[[3](https://arxiv.org/html/2610.28872#bib.bib3)\], its targeted variant\[[4](https://arxiv.org/html/2610.28872#bib.bib4)\]and low\-voltage SRAM errors\[[5](https://arxiv.org/html/2610.28872#bib.bib5)\]all do their damage through the tail \(the attack surface of AI hardware is surveyed in\[[6](https://arxiv.org/html/2610.28872#bib.bib30)\]\), and an adaptive attacker only needs to hit it\. It sets the price of protection, because a selective code\[[7](https://arxiv.org/html/2610.28872#bib.bib12)\]has to cover the tail and its parity grows with the tail’s size, while a code over everything costs 12 to 20% of memory\. And it sets the price of storage, because a variable\-precision format must keep the tail at full width and can be cheap only on the rest\. A heavier tail means more weights to protect, more parity, more bits and a shorter survival; a lighter one means the opposite\. Robust training\[[8](https://arxiv.org/html/2610.28872#bib.bib6),[9](https://arxiv.org/html/2610.28872#bib.bib7),[10](https://arxiv.org/html/2610.28872#bib.bib8),[11](https://arxiv.org/html/2610.28872#bib.bib9)\]changes the distribution as a side effect, and selective protection and mixed precision live with whatever distribution they are given; none of them optimizes the size of the tail, measured in the format the weights are stored in\.

This paper treats the sensitivity distribution as something to be chosen\.LEANis a training\-time procedure that reshapes it\. The damage of a flip is the product of two factors, the gradient of the weight and the step the flip takes in the storage format, andLEANregularizes both: a range term bounds the step of every layer, and a tail term, the mean sensitivity of the top 1% of weights differentiated through the gradient, lowers the gradient where it is largest\. A coefficientλ\\lambdasets how far the tail is pushed \(Figure[2](https://arxiv.org/html/2610.28872#S0.F2)\)\. The change is to the whole distribution, not to a handful of weights, and it costs essentially no accuracy\. The paper includes two checks: real flips confirm that the reshaped weights do less damage, not only that their gradients read smaller, and an ablation separates the two terms\. At INT8 the range term does most of the work; the tail term adds to it but cannot replace it, and alone it lowers the gradient without lowering the damage\. Figure[1](https://arxiv.org/html/2610.28872#S0.F1)previews the effect on one model: the tail shrinks by an order of magnitude, and under the same 1% code the model survives an attack that reduces the normally trained one to chance\.

The contributions of this paper are the following\.

- •A systematic way to modulate the weight\-sensitivity distribution of a network during training\.One composite regularizer on the two factors of the damage, with a coefficientλ\\lambdathat sets the amount, an ablation that attributes the effect to each term, a real\-flip check that the effect is not gradient masking \(Table[IV](https://arxiv.org/html/2610.28872#S4.T4)\), and a plain gradient\-norm baseline that does not reproduce it\. Measured on nine vision model\-dataset configurations \(ResNet\-18, ResNet\-50 and VGG\-16 on CIFAR\-10, CIFAR\-100 and ImageNet\) and three language models \(Qwen2\-0\.5B, Llama\-3\.2\-1B and Qwen2\-1\.5B\): the sensitive tail shrinks 5 to 13 times on the vision models within 0\.5 points of accuracy, and 1\.7 to 5\.3 times on the language models, after one fine\-tune and monotonically inλ\\lambda\. It is applied as a ten\-epoch fine\-tune of a trained network or switched on in the last epochs of training; both routes reach the same floor\.
- •Consequences of the reshaped distribution for protection, precision and storage\.Bit\-flip protection: a 1% code that fails on the normal model by 200 flips holds through 300 in every run and 94% through 500 in two of three seeds, and the parity for a fixed guarantee drops 3\.7 times\. Precision and storage: a sensitivity\-ordered 8/6/4/3\-bit format is 1\.2 points above LSQ on ImageNet ResNet\-50 at an exactly matched 4 bits per weight, and at 3\.29 bits reaches 77\.2% against 74\.4% for LSQ at 3 bits, 2\.4 times less weight payload than INT8 and an estimated 59% less weight\-fetch energy; most of that gain comes from ordering the bits by sensitivity, which the small tail makes affordable, andλ\\lambdaadds the rest\. With a small interleaved code the CIFAR low\-bit models survive 500 flips as well\. The evaluation uses adaptive bit\-flip attacks as the hardest case of a stored\-weight fault\.

The rest of the paper is organized as follows\. Section[II](https://arxiv.org/html/2610.28872#S2)explains why the sensitivity distribution matters, through the attacks, the codes and the formats that depend on it\. Section[III](https://arxiv.org/html/2610.28872#S3)presents the objective that reshapes it\. Section[IV](https://arxiv.org/html/2610.28872#S4)measures the reshaping, checks what each term does and that the damage really falls, and measures the effects on protection, precision and storage; Section[V](https://arxiv.org/html/2610.28872#S5)discusses what that means for the systems that would use it and extends the measurement to language models; Section[VI](https://arxiv.org/html/2610.28872#S6)reviews related work and Section[VII](https://arxiv.org/html/2610.28872#S7)concludes\.

## IIBackground

This section explains why the sensitivity distribution matters, through the three things that depend on it: the attacks that exploit its tail, the codes that must cover it, and the formats that must store it\. It also introduces the baselines\.

### Bit\-flip attacks on stored weights

Stored weights are reached by Rowhammer\-style faults, by low\-voltage operation and by hardware or infrastructure Trojans\[[12](https://arxiv.org/html/2610.28872#bib.bib29),[6](https://arxiv.org/html/2610.28872#bib.bib30)\]; whatever the vector, the damage depends on which weight is hit\. Progressive bit search \(PBS\)\[[3](https://arxiv.org/html/2610.28872#bib.bib3)\]is the standard adaptive attacker against a quantized network: at each step it ranks stored bits by the gradient of the loss, evaluates the loss after flipping each of the top candidates, commits the worst, and repeats; a handful of flips take ImageNet ResNets to chance\. T\-BFA\[[4](https://arxiv.org/html/2610.28872#bib.bib4)\]uses the same search to send one source class to a chosen target class and is reported by its attack success rate\. Rowhammer makes such flips practical in DRAM\[[1](https://arxiv.org/html/2610.28872#bib.bib1),[2](https://arxiv.org/html/2610.28872#bib.bib2)\], and low\-voltage SRAM produces random ones\[[5](https://arxiv.org/html/2610.28872#bib.bib5)\]\. Both attacks succeed by finding the tail of the sensitivity distribution; in all our experiments the attacker is white\-box and sees the weights, the gradients and, where a code is present, the code\.

### The sensitivity distribution, tolerance and cost

For a weight stored as an 8\-bit two’s complement integer with a per\-tensor scale, flipping the sign bit changes the integer by 128, which is half of the representable range\. Every weight in a layer therefore moves by the same large amount when its sign bit flips, and the damage is decided by the gradient alone\. Gradients are heavy\-tailed: in a trained ResNet\-18 the 1% of weights with the largest first\-order damage account for about 20% of the total, and the next 8% for most of the rest; the curves in Figure[6](https://arxiv.org/html/2610.28872#S4.F6)fall by a factor of 10,000 or more between the median weight and the most sensitive one\. This non\-uniformity is the fact every selective scheme builds on, and it is also what PBS exploits, since the gradient ranking that PBS uses is the same ranking that defines the sensitive set\. It is worth being explicit about the two roles the distribution plays\. The first is error tolerance: a random flip usually lands in the LOW mass and does little, while an adaptive one is aimed at the tail and does much, so the tolerance of a model to a given number of flips is set by how many weights sit above the damage level at which a flip matters, not by the size of the model\. The second is efficiency: a selective code must cover the tail, a mixed\-precision format must store it at full width, and a budget of bits or parity buys protection in proportion to how small the tail is\. A model whose sensitivity distribution has a short tail is therefore both more tolerant and cheaper to protect, for the same code and the same format\. Robust training methods change the distribution as a side effect; the method in Section[III](https://arxiv.org/html/2610.28872#S3)changes it on purpose and by a chosen amount\.

### Selective error correction

A code over every weight bit stops flips outright but costs 12 to 20% of model memory in parity, which embedded codes such as E3\-CODE\[[13](https://arxiv.org/html/2610.28872#bib.bib31)\]reduce by folding parity into the weight storage\. A sensitivity\-aware selective code protects selectively: it sorts weights into HIGH, MID and LOW bins by first\-order damage, encodes the four most significant bit planes of the HIGH weights with a BCH\(127,43\) code that correctst=12t=12errors per 127\-bit codeword, and stores the parity in the low\-order bit planes of LOW weights, where it costs nothing visible in accuracy\. Only four of the eight planes are coded, so the parity is at most4/8×84/43≈0\.98%4/8\\times 84/43\\approx 0\.98\\%of the weight bits for every 1% of weights covered, and 0\.91% as packed in Table[VII](https://arxiv.org/html/2610.28872#S4.T7), and it occupies low\-order bits of LOW weights rather than extra memory; its cost is a parity budget inside the model, not bits added to it\. Defeating one codeword takes more thantterrors in that word\. In our attack evaluation a flip on a covered bit is modelled as corrected, with no persistent effect; this models an interleaved deployment under the assumption that the attacker cannot accumulate more thantterrors in one codeword before it is corrected\. The code itself is not simulated, and the guarantee rests on that assumption\. In\-place zero\-space protection\[[7](https://arxiv.org/html/2610.28872#bib.bib12)\]and similar schemes make the same trade\. For all of them the cost is set by one number, the size of the HIGH bin\. The method in the next section is built to make that number small\.

### Quantization baselines

For storage we compare against LSQ\[[14](https://arxiv.org/html/2610.28872#bib.bib13)\], which learns the quantization step size by gradient descent and is the standard strong baseline for low\-bit uniform quantization, and against KURE\[[15](https://arxiv.org/html/2610.28872#bib.bib14)\], which regularizes the kurtosis of the weight distribution so that one model quantizes well at several bit widths\. Both are retrained in our pipeline with the same schedule as our own models\.

## IIIMethod

We first write down what a flip costs, then turn that cost into a training term, show what one gradient step does to it, and finally describe the storage format that a small sensitive set makes affordable\.

### Sensitivity of a stored weight

We store weights as INT8 with one scale per tensor: layerℓ\\ellhassℓ=max⁡\|Wℓ\|/127s\_\{\\ell\}=\\max\|W\_\{\\ell\}\|/127, and a weightwwis kept as the integerq=round⁡\(w/sℓ\)q=\\mathrm\{round\}\(w/s\_\{\\ell\}\)\. In two’s complement, flipping the sign bit of an 8\-bit code changesqqby 128, so every weight in the layer moves by the same amountΔℓ=128​sℓ\\Delta\_\{\\ell\}=128\\,s\_\{\\ell\}, whatever its own magnitude\. The first\-order change in the loss produced by that flip is

di=\|∂L∂wi\|​Δℓ⁡\(i\),d\_\{i\}=\\Big\|\\frac\{\\partial L\}\{\\partial w\_\{i\}\}\\Big\|\\,\\Delta\_\{\\ell\(i\)\},\(1\)which we call the damage, or sensitivity, of weightii\. We fix two thresholds once, from the baseline model: the HIGH threshold is the damage of the weight at the top 1% of the baseline distribution and the MID threshold that at the top 9% \(so HIGH holds 1% of the weights and MID the next 8%\), so the baseline has 1% HIGH, 8% MID and 91% LOW\. A trained model is always measured against these fixed thresholds, so “fewer HIGH weights” means fewer weights above the baseline’s own cut, and the three bins of any model sum to the same total\.

### Objective

LetTTbe the set of thek=⌈0\.01​N⌉k=\\lceil 0\.01N\\rceilweights with the largest damage\.LEANminimizes

Lλ\\displaystyle L\_\{\\lambda\}=LCE\+λPLEAN,PLEAN=Ptail\+βPclip,\\displaystyle=L\_\{\\mathrm\{CE\}\}\+\\lambda\\,P\_\{\\mathrm\{LEAN\}\},\\qquad P\_\{\\mathrm\{LEAN\}\}=P\_\{\\mathrm\{tail\}\}\+\\beta\\,P\_\{\\mathrm\{clip\}\},\(2\)Ptail\\displaystyle P\_\{\\mathrm\{tail\}\}=1k​∑i∈Tdi,\\displaystyle=\\tfrac\{1\}\{k\}\\textstyle\\sum\_\{i\\in T\}d\_\{i\},wherePclip=∑ℓmean​\(relu⁡\(\|Wℓ\|−q99​\(\|Wℓ\|\)\)\)2P\_\{\\mathrm\{clip\}\}=\\sum\_\{\\ell\}\\mathrm\{mean\}\\big\(\\mathrm\{relu\}\(\|W\_\{\\ell\}\|\-q\_\{99\}\(\|W\_\{\\ell\}\|\)\)\\big\)^\{2\}bounds the INT8 range of each layer, and with it the flip stepΔℓ\\Delta\_\{\\ell\};β=100\\beta=100in every experiment and is never tuned, so the effective weight of the range term is100​λ100\\lambdaand grows with the knob\. The two terms act on the two factors of Eq\.[1](https://arxiv.org/html/2610.28872#S3.E1):PclipP\_\{\\mathrm\{clip\}\}shrinksΔℓ\\Delta\_\{\\ell\}where a layer has outlying weights, which is what sets the sign\-flip step, andPtailP\_\{\\mathrm\{tail\}\}lowers\|gi\|\|g\_\{i\}\|on the current tail\. Section[IV](https://arxiv.org/html/2610.28872#S4)separates the two; at INT8 the range term does most of the reshaping and the tail term adds to it\.PtailP\_\{\\mathrm\{tail\}\}is a function of the gradient, so its gradient is a Hessian\-vector product\. We compute it by double backpropagation on every fourth step, with the tail term scaled by four on that step so that its average weight over training isλ\\lambda; the fine\-tune then costs about 1\.5 times a plain one\. In the double backward pass the stepΔℓ\\Delta\_\{\\ell\}\(and with it the scalesℓs\_\{\\ell\}\), the quantile that selectsTTand the quantileq99q\_\{99\}are detached, so the second\-order term differentiates only throughgg, as in the proposition below\. All experiments use ten epochs from the baseline checkpoint with SGD, learning rate 0\.002 on CIFAR and 0\.0005 on ImageNet, batch size 128 and 32\.

Algorithm[1](https://arxiv.org/html/2610.28872#alg1)lists the steps, and Figure[3](https://arxiv.org/html/2610.28872#S3.F3)shows the same procedure as a flow, from a trained network to a deployed one\. The INT8 pipeline has oneLEANstage of ten epochs, a fine\-tune or the last epochs of training: the baseline is measured, trained with theLEANterm, re\-measured against the thresholds fixed from the baseline, and packed, either with a code over a fixed budget \(the top 1% by damage, as in all attack tables\) or over everything aboveτH\\tau\_\{H\}\(the threshold mode of the parity experiment\)\. The low\-bit format of the last paragraph of this section is a second deployment and uses a second, quantization\-aware fine\-tune\.

Algorithm 1LEAN: shaping the sensitivity distribution1:trained

ww; sign\-flip step

Δℓ\\Delta\_\{\\ell\}of the storage format;

λ\\lambda; probe batch

BB
2:

di←\|∂LCE/∂wi\|​Δℓ⁡\(i\)d\_\{i\}\\leftarrow\|\\partial L\_\{\\mathrm\{CE\}\}/\\partial w\_\{i\}\|\\,\\Delta\_\{\\ell\(i\)\}on

BB⊳\\trianglerightbaseline

3:

τH,τM←\\tau\_\{H\},\\tau\_\{M\}\\leftarrowtop\-1% and top\-9% quantiles of

dd⊳\\trianglerightfixed

4:foreach fine\-tuning step with batch

\(x,y\)\(x,y\)do

5:

g←∇wLCE​\(x,y\)g\\leftarrow\\nabla\_\{w\}L\_\{\\mathrm\{CE\}\}\(x,y\)
6:

u←0u\\leftarrow 0
7:ifstep mod 4

=0=0then⊳\\trianglerightgraph retained

8:

di←\|gi\|​Δℓ⁡\(i\)d\_\{i\}\\leftarrow\|g\_\{i\}\|\\,\\Delta\_\{\\ell\(i\)\};

T←T\\leftarrowtop

k=⌈0\.01​N⌉k=\\lceil 0\.01N\\rceilof

dd
9:

Ptail←1k​∑i∈TdiP\_\{\\mathrm\{tail\}\}\\leftarrow\\frac\{1\}\{k\}\\sum\_\{i\\in T\}d\_\{i\}⊳\\trianglerightΔℓ\\Delta\_\{\\ell\},TTdetached

10:

u←4​∇wPtailu\\leftarrow 4\\,\\nabla\_\{w\}P\_\{\\mathrm\{tail\}\}by double backprop⊳\\triangleright=4​H​D​σ=4HD\\sigma

11:endif

12:

w←w−η⁡\[g\+λ⁡\(u\+β​∇wPclip\)\]w\\leftarrow w\-\\eta\\,\\Big\[\\,g\+\\lambda\\,\\big\(u\+\\beta\\,\\nabla\_\{w\}P\_\{\\mathrm\{clip\}\}\\big\)\\Big\]⊳\\trianglerightEq\.[2](https://arxiv.org/html/2610.28872#S3.E2)

13:endfor

14:re\-measure

ddon

BB; shares above

τH\\tau\_\{H\},

τM\\tau\_\{M\}⊳\\trianglerightHIGH, MID

15:pack: BCH over the top 1% by

dd\(fixed budget\) or over

\{i:di\>τH\}\\\{i:d\_\{i\}\>\\tau\_\{H\}\\\}\(threshold\)

![Refer to caption](https://arxiv.org/html/2610.28872v1/figs/fig_flow_drawio.png)Fig\. 3:TheLEANprocedure\. The trained network is quantized and its sensitivity distribution is measured once, which fixes the HIGH and MID thresholds; ten epochs with theLEANterm follow, as a fine\-tune or as the last epochs of training, with the tailTTre\-selected and the tail term applied, through double backpropagation, every fourth step; the model is then re\-measured against the fixed thresholds and packed\.λ\\lambdais the only knob; the low\-bit format uses a separate quantization\-aware fine\-tune \(last paragraph of this section\)\.###### Proposition 1\(Observation\)\.

Letg=∇wLCEg=\\nabla\_\{w\}L\_\{\\mathrm\{CE\}\},H=∇w2LCEH=\\nabla^\{2\}\_\{w\}L\_\{\\mathrm\{CE\}\},D=diag⁡\(Δℓ⁡\(i\)\)D=\\mathrm\{diag\}\(\\Delta\_\{\\ell\(i\)\}\)andσi=sign\(gi\)1\[i∈T\]/k\\sigma\_\{i\}=\\mathrm\{sign\}\(g\_\{i\}\)\\,\\mathbf\{1\}\[i\\in T\]/k\. HoldTT, the signs ofggonTTandDDfixed\. A gradient step of sizeη\\etaonλ​Ptail\\lambda P\_\{\\mathrm\{tail\}\}alone changes the tail sensitivity byΔ​Ptail=−η​λ​‖H​D​σ‖2\+O⁡\(η2\)\\Delta P\_\{\\mathrm\{tail\}\}=\-\\eta\\lambda\\,\\\|HD\\sigma\\\|^\{2\}\+O\(\\eta^\{2\}\)\.

WithTTand the signs held fixed over the step,Ptail=σ⊤​D​gP\_\{\\mathrm\{tail\}\}=\\sigma^\{\\top\}Dg, so∇wPtail=H​D​σ\\nabla\_\{w\}P\_\{\\mathrm\{tail\}\}=HD\\sigma; the step isw←w−η​λ​H​D​σw\\leftarrow w\-\\eta\\lambda HD\\sigma, and the first\-order change ofPtailP\_\{\\mathrm\{tail\}\}is−η​λ​\(H​D​σ\)⊤​\(H​D​σ\)\-\\eta\\lambda\\,\(HD\\sigma\)^\{\\top\}\(HD\\sigma\)\. This is the descent lemma applied toPtailP\_\{\\mathrm\{tail\}\}and we claim nothing more for it; it is local to the tail\-penalty step, andTTcan change between steps\. Its use is to say what the tail term can and cannot do: it moveswwalong the Hessian rows of the tail weights and lowers their gradients, which is the quantity a gradient\-guided attacker reads, and a detached proxy such as\|wi\|\|w\_\{i\}\|or a frozen\|gi\|\|g\_\{i\}\|has noHHterm and cannot do that\. It does not act onΔℓ\\Delta\_\{\\ell\}, which is the other factor of the damage and, as Section[IV](https://arxiv.org/html/2610.28872#S4)shows, the larger lever at INT8\. Whether lowering the gradient also lowers the real damage of a flip is an empirical question, and we answer it with real flips rather than with the proxy\.

### Sensitivity\-ordered low\-bit storage

The proposition covers the training side\. On the deployment side, a cheaper code and a longer survival need no change to the storage format, but the third consequence, a smaller model, does\. Once HIGH and MID are small, a mixed format can afford to store them well\. We keep the HIGH tier at 8 bits with a per\-tensor scale, the MID tier at 6 bits per channel, fill a 4\-bit tier with the next most sensitive weights until an average\-bit budget is met, and store the remaining weights at 3 bits per channel\. This is a second, ten\-epoch quantization\-aware fine\-tune, started from theLEANINT8 checkpoint, that carries the sameλ\\lambdaterm, now evaluated with each weight’s own deployed flip step2b−1​s2^\{b\-1\}s; the tiers are refreshed every 50 of its steps\. The bits\-per\-weight figures we report count the weight payload only; the per\-channel scales and the tier map are not included, and code parity is added only where stated\. The map is the larger of the two: tiers are assigned per weight, so an uncompressed map costs 2 bits per weight, an entropy\-coded one about 1 bit at the tier shares of Table[VIII](https://arxiv.org/html/2610.28872#S4.T8), and a block\-wise assignment would cost a fraction of a bit; we leave its layout to the deployment and compare payloads\. Two details decide whether this low\-bit model can be attacked cheaply\. The scale of the 6/4/3\-bit tiers is set by the 0\.99 quantile of the tier rather than its maximum, so that a sign flip in the 3\-bit tier moves a weight 2 to 4 times less than it otherwise would; and tiers are ranked by deployed\-format damage, so that the 8\-bit tier is the most damaging 1% of the model that is actually stored, not of the floating\-point model it came from\.

## IVResults

The experiments are in two parts\. The first part establishes that the procedure reshapes the distribution on every model, by an amount thatλ\\lambdacontrols, without hurting accuracy\. The second part measures the consequences of the reshaped distribution for bit\-flip protection, for precision and storage, and for the low\-bit model’s own resilience\. Language models are in Section[V](https://arxiv.org/html/2610.28872#S5)\.

### Experimental setup

We use ResNet\-18, ResNet\-50\[[16](https://arxiv.org/html/2610.28872#bib.bib16)\]and VGG\-16 with batch normalization\[[17](https://arxiv.org/html/2610.28872#bib.bib17)\]on CIFAR\-10, CIFAR\-100\[[18](https://arxiv.org/html/2610.28872#bib.bib18)\]and ImageNet\[[19](https://arxiv.org/html/2610.28872#bib.bib19)\]; ResNet\-34 is added in the storage and low\-bit experiments \(Tables[XI](https://arxiv.org/html/2610.28872#S5.T11),[IX](https://arxiv.org/html/2610.28872#S4.T9)and[X](https://arxiv.org/html/2610.28872#S4.T10)\) as a fourth deployment target\. CIFAR baselines are trained from scratch\. ImageNet baselines are the torchvision pretrained models, and every ImageNet fine\-tune and evaluation in this paper uses one fixed random split of the 50k validation set \(seed 0\), 40k images for fine\-tuning and 10k held out for evaluation; ImageNet accuracies are therefore on this 10k held\-out set and are not directly comparable to published 50k\-set numbers\. EveryLEANINT8 model is a single ten\-epoch fine\-tune from its baseline with the objective of Eq\.[2](https://arxiv.org/html/2610.28872#S3.E2)\(Figure[4](https://arxiv.org/html/2610.28872#S4.F4)shows one such run\); the published defenses and the quantization baselines are fine\-tuned with the same schedule\. Unless a table says otherwise, each entry is one training run and one attack run with a fixed seed; Table[VIII](https://arxiv.org/html/2610.28872#S4.T8)uses three seeds, and the ResNet\-18 CIFAR\-10λ=40\\lambda=40fine\-tune was repeated with three seeds: clean accuracy 94\.84±\\pm0\.06, HIGH 0\.113±\\pm0\.003%, MID 1\.84±\\pm0\.17%, and with the 1% code every seed holds 94% through 300 PBS flips while two of three hold 94% at 500 \(the third falls to chance there\)\. The tail fraction is 1% throughout; 0\.5% and 2% give HIGH 0\.115% and 0\.118% at 94\.7% and 94\.9% accuracy, so the result is not sensitive to it\. The protection experiments instantiate the selective code with the layout of Section[II](https://arxiv.org/html/2610.28872#S2), butLEANdoes not depend on it: any scheme whose cost grows with the number of high\-sensitivity weights benefits from the same tail reduction\. Sensitivity is measured by Eq\.[1](https://arxiv.org/html/2610.28872#S3.E1)on 20 batches of 128 images, against thresholds fixed from the baseline: the thresholds used during training come from training batches, and the reported distributions, counts and shares \(Table[I](https://arxiv.org/html/2610.28872#S4.T1), Figures[5](https://arxiv.org/html/2610.28872#S4.F5)and[6](https://arxiv.org/html/2610.28872#S4.F6)\) from held\-out batches\. Attacks are real PBS with 50 candidate bits per step and two attack batches of 128 images, reported as top\-1 accuracy on the held\-out set afternncommitted flips; T\-BFA is reported as accuracy and attack success rate after 30 flips\. Clean accuracy is top\-1 on the held\-out set\. All experiments were run in PyTorch on NVIDIA RTX A6000 GPUs \(48 GB of memory each\), one GPU per run; a ten\-epochLEANfine\-tune takes about 9 minutes for ResNet\-18 on CIFAR\-10 and 1 to 3 hours on ImageNet, and a 500\-flip PBS attack takes 25 minutes on CIFAR and 1 to 3 hours on ImageNet\.

![Refer to caption](https://arxiv.org/html/2610.28872v1/figs/fig_loss.png)Fig\. 4:OneLEANfine\-tune, ResNet\-18 CIFAR\-10,λ=40\\lambda=40: \(a\) the regularizer falls 28 times while the cross\-entropy stays flat; \(b\) test accuracy dips after the first epoch and recovers to within 0\.2 points of the baseline\.
### The distribution moves on every model

Table[I](https://arxiv.org/html/2610.28872#S4.T1)counts the weights above the baseline HIGH threshold before and afterLEANon nine models; Figure[5](https://arxiv.org/html/2610.28872#S4.F5)shows the raw distribution and Figure[6](https://arxiv.org/html/2610.28872#S4.F6)the cumulative distribution, counted from the tail\. The count falls 5 to 13 times\. The share of the total first\-order damage carried by the top 1% of weights falls 1\.4 to 2\.1 times on all nine models \(Table[I](https://arxiv.org/html/2610.28872#S4.T1), from 9 to 28% down to 5 to 18%\); a uniform rescaling of the flip steps would leave this share unchanged, so the distribution is reshaped, not only scaled\. The same share computed from the gradients alone, with no flip step, is unchanged byLEAN\(8\.2 against 8\.7% on ResNet\-18 CIFAR\-10, and alike elsewhere\), so the reshaping acts through the per\-layer flip steps, which the range term lowers where the tail lives, while the tail term’s contribution shows in the real\-flip damage of Table[IV](https://arxiv.org/html/2610.28872#S4.T4)rather than in this share\. Both figures use logarithmic axes, which is why the change looks modest at first sight: the bulk of the distribution spans six decades of count and is untouched, while the tail that moves is one decade lower, a factor of 10 in the number of weights that the code has to cover\. The clean\-accuracy change is at most 0\.5 points \(ResNet\-50 on CIFAR\-100\) and is usually 0\.1 or less; on ImageNet ResNet\-18 it is zero\. The curves make clear that the change is not confined to the very top of the distribution: theLEANcurve leaves the baseline curve at the MID boundary and runs about one decade below it through the whole HIGH band, so the entire tail has moved to lower sensitivity\. Where the weights go is also visible: since every weight is counted somewhere, a lower curve at the MID line is the same statement as a larger LOW bin, and on ResNet\-18 CIFAR\-10 LOW grows from 91% to about 92% while HIGH drops from 1% to 0\.11%\.

TABLE I:HIGH\-bin weights before and afterLEAN, the share of total first\-order damage carried by the top 1% of weights \(%, scale\-free, measured on 20 held\-out batches; the ablation of Table[IV](https://arxiv.org/html/2610.28872#S4.T4)measures the same quantity on its own probe batches, 20\.4→\\to11\.7%\) and clean accuracy \(%\), each as base→\\toLEAN;λ=40\\lambda=40unless noted\.![Refer to caption](https://arxiv.org/html/2610.28872v1/figs/fig_sens9_raw.png)Fig\. 5:Raw weight\-sensitivity distribution of the nine models: number of weights per sensitivity bin \(40 log\-spaced bins ofdid\_\{i\}as a percentage of the baseline maximum; the first bin collects every weight below10−310^\{\-3\}\), baseline solid,LEANdashed, with the baseline LOW, MID and HIGH bands\. Figure[6](https://arxiv.org/html/2610.28872#S4.F6)shows the same data as a cumulative distribution\. The bulk of the distribution is unchanged; the HIGH tail moves left into the MID and LOW bands\.![Refer to caption](https://arxiv.org/html/2610.28872v1/figs/fig_sens9.png)Fig\. 6:Cumulative weight\-sensitivity distribution of the nine models, counted from the tail \(number of weights whose sensitivity is at leastss\), on log\-log axes; baseline solid,LEANdashed\. The shaded bands are the baseline LOW, MID and HIGH bins\. TheLEANcurve separates from the baseline at the MID boundary and stays about one decade lower across the HIGH band\. The box in each panel gives the count above the HIGH threshold before and after\.
### λ\\lambdais a knob

Table[I](https://arxiv.org/html/2610.28872#S4.T1)uses oneλ\\lambdaper model; Figure[7](https://arxiv.org/html/2610.28872#S4.F7)plots the HIGH share and the clean accuracy againstλ\\lambdafor every value we trained on six models\. The HIGH share falls monotonically withλ\\lambdaand saturates near 0\.1% of the weights; accuracy stays within 0\.5 points of the baseline up toλ=40\\lambda=40and begins to fall atλ=100\\lambda=100\(ResNet\-50 ImageNet goes from 80\.7 to 79\.7\)\. We therefore useλ=40\\lambda=40on CIFAR andλ=30\\lambda=30to4040on ImageNet unless stated otherwise\. The two ImageNet models are the ones that shrink least \(5 to 6 times\); their sensitive sets are the largest to begin with and need the strongest penalty\.

![Refer to caption](https://arxiv.org/html/2610.28872v1/figs/fig_knob2.png)Fig\. 7:The knob\. HIGH\-bin share \(bars, log scale\) and clean accuracy \(line\) againstλ\\lambdafor six models, at the values trained for the sweep\. The HIGH bin falls monotonically and saturates near 0\.1%; accuracy stays within 0\.5 points up toλ=40\\lambda=40and drops atλ=100\\lambda=100\.
### What sets the floor

The knob does not go to zero, and Figure[8](https://arxiv.org/html/2610.28872#S4.F8)shows why\. The damage of a weight is\|gi\|​Δℓ\|g\_\{i\}\|\\Delta\_\{\\ell\}, so the tail can be lowered only as far as its two factors allow\. The range term shrinksΔℓ\\Delta\_\{\\ell\}down to the level of a layer’s ordinary weights and no further without clipping them, so the flip step of a layer falls by at most its outlier ratio,

Δℓ≥128127​q99​\(\|Wℓ\|\),ΔℓbeforeΔℓafter≤ρℓ=max⁡\|Wℓ\|q99​\(\|Wℓ\|\),\\Delta\_\{\\ell\}\\geq\\tfrac\{128\}\{127\}\\,q\_\{99\}\(\|W\_\{\\ell\}\|\),\\qquad\\frac\{\\Delta\_\{\\ell\}^\{\\mathrm\{before\}\}\}\{\\Delta\_\{\\ell\}^\{\\mathrm\{after\}\}\}\\leq\\rho\_\{\\ell\}=\\frac\{\\max\|W\_\{\\ell\}\|\}\{q\_\{99\}\(\|W\_\{\\ell\}\|\)\},\(3\)which is a large gain in a layer with outlying weights and none in a layer without them\. The tail term lowers the top\-kkgradients at a speed set by the curvature term‖H​D​σ‖\\\|HD\\sigma\\\|of the Observation, and because the setTTis re\-selected every step, what it does in practice is flatten the top of the distribution into a plateau rather than remove it \(Figure[6](https://arxiv.org/html/2610.28872#S4.F6)\)\. Where the two stop is the joint optimumλ​∇wPLEAN=−∇wLCE\\lambda\\nabla\_\{w\}P\_\{\\mathrm\{LEAN\}\}=\-\\nabla\_\{w\}L\_\{\\mathrm\{CE\}\}, beyond which every further reduction of the tail is paid for in task loss\. On ResNet\-18 CIFAR\-10 the HIGH bin falls monotonically withλ\\lambdaand flattens near 0\.1% fromλ=30\\lambda=30on, while accuracy begins to drop \(Figure[8](https://arxiv.org/html/2610.28872#S4.F8)a\); the penalty itself plateaus by epoch 6 to 8 of the fine\-tune \(Figure[4](https://arxiv.org/html/2610.28872#S4.F4)\)\. Where the remaining weights sit is instructive \(Figure[8](https://arxiv.org/html/2610.28872#S4.F8)b\): the deep layers empty completely, and four fifths of the 12\.6k weights still above the threshold are in the first convolution and the first stage, small layers that every input passes through, whose weights have no outliers to clip \(the first convolution hasmax⁡\|W\|/q99​\|W\|=1\.7\\max\|W\|/q\_\{99\}\|W\|=1\.7, the lowest in the network, and keeps 92% of its HIGH weights\) and whose gradients cannot be lowered without changing what the network computes\. The floor is therefore a property of the model, set by its per\-layer outlier ratiosρℓ\\rho\_\{\\ell\}\(Eq\.[3](https://arxiv.org/html/2610.28872#S4.E3)\) and by the curvature of the loss around its early weights, not byλ\\lambdaor the number of epochs;λ\\lambdachooses how far toward it to go\. Across models the reduction is largest where there are few weights and large outliers, 9 to 13 times on CIFAR, and smallest where many weights sit near the threshold, 5 to 7 times on ImageNet and 1\.7 to 3\.4 times on the language models \(Figure[8](https://arxiv.org/html/2610.28872#S4.F8)c\)\.

![Refer to caption](https://arxiv.org/html/2610.28872v1/figs/fig_floor.png)Fig\. 8:The limit of the reshaping\. \(a\) HIGH share againstλ\\lambda\(solid, left axis\) and change in clean accuracy \(dotted, right axis\) for ResNet\-18 CIFAR\-10 and ResNet\-50 ImageNet; the share flattens near 0\.1% while accuracy begins to fall\. \(b\) HIGH weights per layer before and afterLEAN, ResNet\-18 CIFAR\-10,λ=40\\lambda=40: the deep layers empty, the first convolution and first stage keep most of what remains\. \(c\) HIGH reduction factor for the twelve models, largest to smallest: CIFAR, then ImageNet, then the language models\.
### Fine\-tune or last epochs of training

LEANis a loss term, so it can be switched on inside normal training instead of run as a separate fine\-tune\. Figure[9](https://arxiv.org/html/2610.28872#S4.F9)and Tables[II](https://arxiv.org/html/2610.28872#S4.T2)and[III](https://arxiv.org/html/2610.28872#S4.T3)test this on six model/dataset pairs: one 200\-epoch run from scratch per pair, forked at epoch 190 into a plain control and branches with the term on for the last epochs\. Switched on as is, in the last ten epochs of the cosine schedule, the term works but reaches less far than the fine\-tune, HIGH 0\.25 to 0\.50% against 0\.11%, and the penalty is still falling at epoch 200 \(Figure[9](https://arxiv.org/html/2610.28872#S4.F9)b\)\. The cause is the learning rate, which in those epochs is 0\.0005 falling to zero, a quarter of the fine\-tune’s\. Holding it at the fine\-tune’s 0\.002 for the same ten epochs reaches the floor on every pair, HIGH 0\.04 to 0\.14% and MID 0\.7 to 2\.9%, at the fine\-tune’s accuracy; twenty epochs orλ=100\\lambda=100at the dying rate get halfway, HIGH 0\.05 to 0\.20%\. Reaching the floor costs 0\.1 to 0\.7 points except on VGG\-16 CIFAR\-100 \(1\.3 points, where a smallerλ\\lambdawould be used\), stopping halfway costs 0\.1 or nothing\. Under attack the routes line up in the same order \(Table[V](https://arxiv.org/html/2610.28872#S4.T5), bottom block, ResNet\-18 CIFAR\-10 with the 1% code\): the control breaks at 200 flips, the last\-epochs model at 300, the rate\-held model holds 94% at 300 and breaks at 500; the targeted attack with the code stays at 3 to 6% success for all of them, as for the fine\-tune\. Giving the last ten epochs the fine\-tune’s full schedule, 0\.002 decaying to zero \(rate held, cos\.\), closes the remaining gap: 93% at 500 flips at 95\.0% clean accuracy, the fine\-tune’s protection at a slightly higher accuracy, and on VGG\-16 the same schedule reaches HIGH 0\.04% at no accuracy cost\. The decay matters more than the last few hundredths of HIGH: twenty epochs at a constant 0\.002 \(rate held, 20 ep\.\) leave a smaller tail by count \(0\.13%, MID 0\.6%\) yet break at 400 flips, because the undecayed rate leaves the surviving weights less settled\. Both routes are the same technique; we use the fine\-tune because it gives the term that schedule by construction and because the ImageNet and language\-model baselines are pretrained checkpoints with no training of ours to switch it on in\.

TABLE II:LEANin the last epochs of a 200\-epoch run, CIFAR\-10; HIGH and MID against the thresholds of the matching control of the same run\. The fine\-tune rows are the paper’s models, measured against their own baseline\.TABLE III:The same recipes on four more pairs \(C\-10, C\-100: CIFAR\),λ=40\\lambda=40unless stated, ten epochs unless stated\. HIGH and MID in % of weights\.![Refer to caption](https://arxiv.org/html/2610.28872v1/figs/fig_inset.png)Fig\. 9:LEANinside training,λ=40\\lambda=40; ResNet\-18 CIFAR\-10 in the top row, VGG\-16 CIFAR\-10 below\. \(a, d\) Test accuracy for epochs 180 to 200 of a 200\-epoch run; at epoch 190 the run is forked into a plain control and a branch with theLEANterm on\. \(b, e\) The penalty over the tenLEANepochs inside training \(learning rate 0\.0005 to 0\) and in the fine\-tune \(0\.002, cosine\)\. \(c, f\) HIGH share and accuracy of the control, the last\-epochs model, the last\-epochs model with the rate held at 0\.002, and the fine\-tune\. \(g\) HIGH share per recipe on four more pairs, accuracy inside the bars\.
### Which term does the work, and is the damage real

Two questions have to be settled before the consequences are measured\. The first is whether the proxydi=\|gi\|​Δℓ⁡\(i\)d\_\{i\}=\|g\_\{i\}\|\\Delta\_\{\\ell\(i\)\}can be gamed: an attacker ranks weights by the gradient, and a term that lowers the gradient could hide weights without making them safer\. The second is which of the two terms of Eq\.[2](https://arxiv.org/html/2610.28872#S3.E2)moves the distribution\. Table[IV](https://arxiv.org/html/2610.28872#S4.T4)answers both on ResNet\-18 CIFAR\-10\. HIGH and the top\-1% damage share are from the proxy\. For the first question we flip real sign bits one at a time, in the INT8 model, for the 1000 weights the proxy ranks highest and for 1000 random weights, and record the true loss increase; the table gives the mean over the 100 worst weights found \(worst 100\) and over the baseline’s own top 1000 re\-flipped inside each fine\-tuned model \(base top\), and random weights cause no measurable change in any model\.Δ\\Deltaratio is the mean per\-layer flip step relative to the baseline\. In theLEANmodel the worst 100 weights cost 0\.009 per flip against 0\.031 in the baseline, and the baseline’s top 1000 cost 0\.0014 against 0\.0069, so the damage really falls; the proxy’s share of total damage held by the top 1%, which does not depend on the scale ofΔℓ\\Delta\_\{\\ell\}, falls from 20% to 12%\. For the second, the range term alone reaches almost the same point \(HIGH 0\.13%, worst\-100 damage 0\.012,Δℓ\\Delta\_\{\\ell\}halved on average\) at no accuracy cost, and the tail term on top of it lowers the damage a further 25% \(0\.012 to 0\.009, and 0\.0019 to 0\.0014 on the fixed set\)\. The tail term alone does what the first question feared: atλ=400\\lambda=400it reports a 3 times smaller HIGH bin while the real damage of the worst weights is unchanged and accuracy drops a point\. At INT8, then, the lever is the flip step, which the range term controls, and the tail term is a useful addition that must not be used on its own\. The last row is the plain gradient\-norm penalty, the squared norm of the weight gradient, under the same schedule, the nearest generic regularizer: atλ=0\.01\\lambda=0\.01, the largest value that keeps accuracy within a point, it leaves HIGH at 0\.64% and the real damage of the worst weights at 0\.026 against 0\.031, and atλ=0\.03\\lambda=0\.03accuracy falls to 88% with the tail unchanged; without the storage\-format weighting and the tail selection the gradient is smoothed everywhere and the tail is not where the pressure goes\.

TABLE IV:Ablation and real\-flip check, ResNet\-18 CIFAR\-10,λ=40\\lambda=40unless noted\. Columns are defined in the text\.
### Effect on bit\-flip protection

The first consequence of a smaller tail is that a small code becomes enough\. We attack the INT8 model with real progressive bit search\[[3](https://arxiv.org/html/2610.28872#bib.bib3)\]over all layers and protect it, where stated, with the selective BCH\(127,43\) code \(t=12t=12\) on the top four bit planes of the HIGH weights\. Table[V](https://arxiv.org/html/2610.28872#S4.T5)placesLEANnext to four published robust\-training defenses retrained in the same pipeline: piecewise clustering\[[8](https://arxiv.org/html/2610.28872#bib.bib6)\], WCAT\[[9](https://arxiv.org/html/2610.28872#bib.bib7)\], Forget\-and\-Rewire\[[10](https://arxiv.org/html/2610.28872#bib.bib8)\]and Hessian\-aware training\[[11](https://arxiv.org/html/2610.28872#bib.bib9)\]\. Without a code every method,LEANincluded, is at chance within 50 to 100 flips: an attacker that may flip anything always finds the next most damaging weight\. With the same 1% code the methods separate, and the HIGH column gives the first reason: the defenses leave 0\.89 to 1\.05% of the weights above the threshold \(WCAT 0\.17%\), so a 1% code is exactly full and PBS finds the next weight\. A small tail is necessary but not sufficient; what also counts is how damaging the weights just below the code are, the worst\-100 column of Table[IV](https://arxiv.org/html/2610.28872#S4.T4): the range\-only model, WCAT and the rate\-held 20\-epoch model all leave 0\.13 to 0\.17% HIGH yet end at 49, 75 and 16% at 500 flips\. In what follows a model breaks at the first flip count at which its accuracy falls below 90%\. The normal model breaks at 200 flips, piecewise clustering and Hessian\-aware training at 200, Forget\-and\-Rewire at 300, WCAT at 500 \(75%\);LEANatλ=40\\lambda=40holds 94% at 500 \(in two of three training seeds; the third holds 94% at 300 and breaks at 500, the experimental setup of Section[IV](https://arxiv.org/html/2610.28872#S4)\), and theλ\\lambdasweep \(λ=10\\lambda=10failing at 500,λ=30\\lambda=30holding 90%\) shows the dependence directly\. The ablation rows settle what each term buys under attack: the range term alone \(HIGH 0\.13%\) holds at 200 and breaks at 300, short of WCAT, and the tail term alone breaks at 200 like the normal model, or at 500 when pushed toλ=400\\lambda=400; only the two together pass WCAT\. The plain gradient\-norm fine\-tune breaks at 200\. The code removes the weights the attacker wants, andλ\\lambdamakes sure that nothing of comparable sensitivity is left behind them\. On ImageNet ResNet\-18 with the 1% code theLEANmodel holds 83% at 100 flips and 72% at 200 against 69% and 41% for the normal model\. Table[VI](https://arxiv.org/html/2610.28872#S4.T6)repeats the exercise with the targeted attack\[[4](https://arxiv.org/html/2610.28872#bib.bib4)\]on all nine models, 30 flips, three class pairs each\. CIFAR uses the held\-out test set\. ImageNet uses all 50 validation images of each class \(25 for the attack, 25 for the success rate\), 40 of which were in theLEANfine\-tuning split while the normal model is the torchvision checkpoint that never saw them, so the overlap favours theLEANside and the ImageNet rows should be read with that in mind; the CIFAR rows have no such overlap\. Accuracy on the non\-source classes stays within one point of clean on CIFAR; on ImageNet VGG\-16 without a code it falls to 22 to 27%\. The same rule holds, and it also shows where the rule stops\. Where the targeted attack goes through weights that the mixed\-data sensitivity ranks highly \(the CIFAR\-10 models and ImageNet VGG\-16, whose classifier holds most of its tail\),LEANwith the code ends it: 1 to 5% success against 30 to 93% for the normal model with the same code\. Where the attack lives in the last layer’s row for one of 100 or 1000 classes, weights whose mixed\-data sensitivity is modest,LEANlowers the success rate on CIFAR\-100 and changes nothing on the ImageNet ResNets\. The reason is thatLEANreshapes the sensitivity it is trained on, the damage under the mixed\-class training objective, and robustness does not transfer by itself to a class\-conditional sensitivity that this objective does not rank\. A tier chosen by the maximum damage over target classes closes that route for the normal model too; it is a packing\-time choice that does not separateLEANfrom the baseline, so we report the plain ranking and count the ImageNet ResNets as a limit of the mixed\-data objective rather than of the code\.

TABLE V:ResNet\-18 CIFAR\-10, INT8, accuracy \(%\) afternnprogressive bit\-search flips, without and with the same 1% BCH code; HIGH in %\. Clean accuracy 95\. Bottom block: the models trained withLEANin the last epochs\.TABLE VI:T\-BFA, 30\-flip budget, mean attack success rate \(%\) over three class pairs, without and with the 1% code;LEANat theλ\\lambdaof Table[I](https://arxiv.org/html/2610.28872#S4.T1)\. Data and caveats in the text\.TABLE VII:Parity for a fixed guarantee, ResNet\-18 CIFAR\-10, INT8 with the selective code; coverage in % of weights, parity in % of weight bits, accuracy \(%\) at 6% BER on the covered and on the uncovered weights, and accuracy drop \(points\) at 25 flips\. The 6% BER figure on the uncovered weights is not reported for the coverage sweep, since at one seed it depends on which layers the random errors land in\.
### Effect on parity, precision, storage and energy

The second consequence of a smaller tail is cost: less parity, fewer bits and less energy for the same model\. Figure[10](https://arxiv.org/html/2610.28872#S4.F10)collects the three cost numbers\. Protection first\. The guarantee is fixed in advance and has a random and an adaptive part: 95% accuracy at a 6% random bit error rate on the protected weights, and no accuracy drop at 25 progressive flips \(Table[VII](https://arxiv.org/html/2610.28872#S4.T7); the code is BCHt=12t=12over four planes with the parity embedded in LOW weights, parity in % of weight bits; BER prot\. and BER unprot\. are the accuracy when the 6% error rate hits the covered weights with the code on, and the uncovered weights with no code\)\. The 6% rate is a stress level, far above the10−310^\{\-3\}to10−210^\{\-2\}of low\-voltage SRAM, chosen so that the guarantee is conservative; the 25\-flip point sizes the parity, and the 500\-flip survival of Table[V](https://arxiv.org/html/2610.28872#S4.T5)is the separate, stronger result for theλ=40\\lambda=40model with the 1% code\. The top 1% by damage is not enough for the normal ResNet\-18, because PBS moves on to the weights just below the cut, which are nearly as damaging; it needs the code on 20% of the weights, which is 18% of the weight bits as parity: coverages of 12\.5, 15\.2, 16\.8 and 19\.2% all lose about 0\.5 points at 25 flips, so the minimum lies between 19\.2 and 20%\. TheLEANmodel withλ=10\\lambda=10meets the same guarantee with the code on 5\.3% of the weights, 4\.8% parity, because the weights below its cut are far less damaging; the parity is 3\.7 times smaller\. We useλ=10\\lambda=10in this table because it is the smallestλ\\lambdathat meets the zero\-drop guarantee; the paper’sλ=40\\lambda=40model at the same 5\.3% coverage misses it by 0\.16 points at 25 flips, and loses one point when the 6% error rate hits its uncovered weights\. The same table gives the random\-fault side: with no code on the uncovered weights, a 6% bit error rate there costs the normal model seven points at 20% coverage and theLEANmodel four points at 11\.9% and one point at 5\.3%\. Since the HIGH bin shrinks with negligible change in accuracy \(Table[I](https://arxiv.org/html/2610.28872#S4.T1)\), the parity budget of any selective scheme shrinks with it\.

Storage next\. Table[VIII](https://arxiv.org/html/2610.28872#S4.T8)compares formats for ImageNet ResNet\-50 at exactly matched \(4\.00\) and comparable \(3\.00 against 3\.29\) bits per weight under identical ten\-epoch training\. At a uniform 3 bits our quantization\-aware training reaches 74\.3%, the same as LSQ\[[14](https://arxiv.org/html/2610.28872#bib.bib13)\]\(74\.4%\), soλ\\lambdaby itself is not a better quantizer\. The gain decomposes as follows: sensitivity\-ordered tiering at 3\.29 bits per weight adds 2\.4 points over the uniform 3\-bit model, andλ\\lambdaadds 0\.4 more, consistently over three seeds\. Most of the improvement is therefore from the ordering; whatLEANcontributes is a smaller 8\-bit and 6\-bit tier, which is what makes the ordering affordable at this budget, plus the last 0\.4 points\. At an exactly matched 4\.00 bits the tieredLEANmodel is 1\.2 points above LSQ and 3\.7 above KURE\[[15](https://arxiv.org/html/2610.28872#bib.bib14)\]\. In bytes, the 25\.6 MB INT8 model becomes 10\.5 MB of weight payload, or 11\.6 MB with the codes of the next paragraph \(at 3 bits per weight there are no LOW low\-order bits to hide parity in, so there it is counted\), not counting scales and the tier map\.

Energy last\. This is an analytical estimate, not a measurement: fewer bits per weight means fewer bits cross the memory interface, and we scale the weight traffic by published access energies\. Taking the 45 nm access energies of Han et al\.\[[20](https://arxiv.org/html/2610.28872#bib.bib20)\], 640 pJ for a 32\-bit DRAM access and 5 pJ for a 32\-bit SRAM access, the weight fetch for one ResNet\-50 inference falls from 4\.1 mJ to 1\.7 mJ from DRAM and from 32 to 13μ\\muJ from on\-chip SRAM, an estimated 59% less weight\-fetch energy in either case; tier lookup, scale and metadata fetch and decoder activity are outside this estimate\. The same reduction applies to the SRAM weight buffer an accelerator must provision, which shrinks 2\.2 to 2\.4 times\. Table[XI](https://arxiv.org/html/2610.28872#S5.T11)gives the same two numbers for every model in the paper: INT8 with the 1% code stays at 8 bits per weight because the parity is embedded in LOW weights; the low\-bit format with both codes is 3\.63 bits per weight \(CIFAR, Table[IX](https://arxiv.org/html/2610.28872#S4.T9)\) or 4\.09 with the 3% tier and thet=79t=79code \(ImageNet, Table[X](https://arxiv.org/html/2610.28872#S4.T10), given for the CIFAR models for comparison\); the CIFAR and ImageNet ResNets of the same depth differ only in the classifier, so one row covers both; decoder and tier\-lookup energy are not included\. The ratio is fixed by the format \(2\.2 times at 3\.63 bits per weight, 2\.0 times at 4\.09, with the INT8 parity embedded\), so what changes from model to model is the absolute saving, from 1 mJ per inference on a CIFAR ResNet\-18 to 12\.5 mJ on ImageNet VGG\-16\.

![Refer to caption](https://arxiv.org/html/2610.28872v1/figs/fig_storage.png)Fig\. 10:Cost of protection, storage and weight fetch before and afterLEAN\. \(a\) Parity for the same protection guarantee \(Table[VII](https://arxiv.org/html/2610.28872#S4.T7)\), ResNet\-18 CIFAR\-10, measured; coverage in brackets\. \(b\) Weight memory of ImageNet ResNet\-50 at INT8, where the 1% code’s parity is embedded in LOW weights, and in the tiered 3\.29\-bit format without and with the 1% BCH andt=19t=19sign\-plane codes \(3\.63 bits per weight\); the 4\.09\-bit ImageNet hardening mode of Table[X](https://arxiv.org/html/2610.28872#S4.T10)is not shown\. \(c\) DRAM weight\-fetch energy per inference with the 45 nm access energies of Han et al\.\[[20](https://arxiv.org/html/2610.28872#bib.bib20)\]; on\-chip SRAM scales identically \(32 to 13μ\\muJ\)\.TABLE VIII:ImageNet ResNet\-50 top\-1 \(%\) at matched \(4\.00\) and comparable \(3\.00 vs 3\.29\) bits per weight; tiered rows are mean of three seeds\.
### Effect on the low\-bit model’s own resilience

The INT8 model and the 3\.29\-bit model are different deployments; the natural question is whether the low\-bit one can be protected by the 1% code as well\. It cannot, and the reason is itself a statement about the distribution: at 3 bits per channel almost every weight is one sign flip away from a change of its full magnitude, so the tail the code must cover is the whole model \(Table[IX](https://arxiv.org/html/2610.28872#S4.T9), left block, chance by 100 flips\)\. Three changes, none of them retraining, close the gap, at a price in clean accuracy of about five points on ImageNet \(Table[X](https://arxiv.org/html/2610.28872#S4.T10), 71\.8% against 77\.2% for the storage\-only format of Table[VIII](https://arxiv.org/html/2610.28872#S4.T8)\), which is the cost of the smaller scales and the deployed\-format ranking\. The 6/4/3\-bit scales are set by the 0\.99 quantile instead of the maximum, which shrinks every low\-bit flip 2 to 4 times; the tiers are ranked by deployed\-format damage; and a long interleaved BCH code over the sign\-bit plane \(1023 sign bits per codeword,t=19t=19, 0\.19 bits per weight\) spreads each codeword over random addresses, so an adaptive attacker must spendt\+1t\+1flips to gain one\. With these, the CIFAR models hold 100 to 500 flips within 2 to 7 points of clean at 3\.63 bits per weight \(Table[IX](https://arxiv.org/html/2610.28872#S4.T9), right block\)\. ImageNet needs more:t=79t=79and a 3% tier obtained by promoting the 2% most damaging uncovered weights at packing time, at 4\.09 bits per weight, after which VGG\-16 \(72\.5% clean after promotion\) is unchanged by 500 flips and ResNet\-34 loses one point, while ResNet\-18 and ResNet\-50 lose 14 and 13 \(Table[X](https://arxiv.org/html/2610.28872#S4.T10)\)\. The general lesson for the code is that correction over neighbouring words does not help against an adaptive flipper, since neighbouring weights are as useful to it as one; interleaving is what makes the attacker’s extra flips worthless\.

TABLE IX:CIFAR models at 3\.29 bits per weight under progressive bit search, accuracy \(%\): BCH on the 1% tier only \(3\.45 b/w\) and with the interleaved sign\-plane code \(3\.63 b/w\)\.TABLE X:ImageNet models at 3\.29 bits per weight under progressive bit search, accuracy \(%\) after 100 / 200 / 500 flips, with the sign\-plane code att=19t=19\(3\.63 b/w\) ort=79t=79\(3\.68\), and with the 3% tier andt=79t=79\(4\.09\)\. No retraining\.

## VApplications

This section says what a reshaped distribution means for the systems that would use it, and extends the measurement to language models\.

TABLE XI:Weight memory \(MB\) and estimated DRAM weight\-fetch energy per inference \(mJ\) for the deployed formats; formats and assumptions in the text\.†Millions of weights\.### Accelerators and deployment

A selective code hides its parity in the low\-order bits of LOW weights, so its cost is the number of HIGH weights times the code rate and nothing else\.LEANreduces that number 5 to 13 times at training time without touching the memory system: the decoder, the bin table and the parity embedding are reused unchanged, and the only difference the hardware sees is a shorter list of protected addresses\. Two deployments follow from the two fine\-tunes\. TheLEANINT8 checkpoint with the 1% code is a security mode, surviving 300 flips in every seed and 500 in two of three, with its parity embedded in LOW weights at no added memory; the quantization\-aware checkpoint in the 8/6/4/3\-bit format is a storage mode, 2\.4 times smaller in weight payload and, by the access\-energy model of Section[IV](https://arxiv.org/html/2610.28872#S4), an estimated 59% cheaper to fetch, and with the sign\-plane code it is a security mode too at 3\.63 to 4\.09 bits per weight\. The two checkpoints share the baseline, the thresholds and theλ\\lambdaterm; a device picks the mode that fits its memory\.

TABLE XII:LEANon language models: HIGH and MID shares against the control’s thresholds, and WikiText\-2 perplexity at fp32 and after per\-channel INT4 \(lower is better\)\.TABLE XIII:Preliminary: progressive sign\-bit attack on INT8 language models, perplexity afternnflips, without and with a 1% code by each model’s own ranking; “dead” is above10610^\{6\}\.![Refer to caption](https://arxiv.org/html/2610.28872v1/figs/fig_llm.png)Fig\. 11:LEANon three language models \(Table[XII](https://arxiv.org/html/2610.28872#S5.T12)in bars\): \(a\) HIGH and \(b\) MID shares against the control’s thresholds, with the reduction factor; \(c\) WikiText\-2 perplexity of the control and theLEANmodel, with the cost\.λ=10\\lambda=10for Qwen2\-0\.5B, 3 for the other two\.
### Effect on language models

This part is a preliminary extension, smaller in scale than the vision results: three models of 0\.5 to 1\.5 billion weights, one epoch on one million WikiText\-2 tokens, perplexity as the metric\. Weight sensitivity in transformers is skewed in the same way, and AttentionBreaker\[[21](https://arxiv.org/html/2610.28872#bib.bib10)\]and SilentStriker\[[22](https://arxiv.org/html/2610.28872#bib.bib11)\]exploit it with a few dozen flips\. Table[XII](https://arxiv.org/html/2610.28872#S5.T12)and Figure[11](https://arxiv.org/html/2610.28872#S5.F11)apply the same objective to the linear layers of Qwen2\-0\.5B, Llama\-3\.2\-1B and Qwen2\-1\.5B, fine\-tuned for one epoch on one million WikiText\-2 tokens against a control fine\-tuned identically withλ=0\\lambda=0; HIGH and MID are measured against the control’s top\-1% and top\-9% thresholds, as for the CNNs\. The HIGH set shrinks 1\.7 to 5\.3 times \(5\.3 is Llama atλ=10\\lambda=10, at a 2\.5 perplexity cost; 1\.7 to 3\.4 at theλ\\lambdawe would deploy, Figure[8](https://arxiv.org/html/2610.28872#S4.F8)c\) and the MID set by about the same factor, and the cost is 0\.6 perplexity at theλ\\lambdawe would deploy \(λ=10\\lambda=10on Qwen2,λ=3\\lambda=3on Llama; the penalty is summed over weights, so a larger model needs a smaller coefficient for the same pressure\)\. Post\-training INT4 improves on the two smaller models, from 21\.1 to 19\.3 on Qwen2\-0\.5B and from 21\.4 to 18\.9 on Llama, for the same reason as in Section[IV](https://arxiv.org/html/2610.28872#S4): the weights the penalty removes from the tail are the ones that set the quantization grid\. Qwen2\-1\.5B shows the other side of the knob: the penalty is summed over 1\.3 billion weights, soλ=3\\lambda=3there is as strong asλ=10\\lambda=10on Llama, the tail shrinks 3\.4 times but perplexity rises by 2\.1 and INT4 does not improve; a smallerλ\\lambdais the right setting for a model of that size\. Table[XIII](https://arxiv.org/html/2610.28872#S5.T13)puts the three models under the same progressive sign\-bit attack as the CNNs, with and without a 1% code chosen by each model’s own damage ranking\. Without a code all three are destroyed within 10 flips\. With the code the picture depends on size\. On Llama\-3\.2\-1B theLEANmodel is worse than the control for the first 20 flips \(21 against 12\.5 perplexity at 10 flips\) and 3 times better from 50 flips on \(42 against 122 at 50, 61 against 210 at 100\), which is the regime a coded attacker has to work in\. On the smaller Qwen2\-0\.5B a 1% code is not enough for either model beyond 20 flips;LEANis at its clean perplexity after 10 flips where the control has already doubled, and still usable at 20 where the control is gone\. On Qwen2\-1\.5B, whereλ=3\\lambda=3was already too strong, the attack shows the same thing: both models hold 20 flips with the code and both are gone at 50, and theLEANmodel is slightly worse at 20 \(16\.3 against 11\.2\), so an over\-strongλ\\lambdabuys nothing\. Whereλ\\lambdais set so that perplexity moves by less than one point, the gain is the same kind as on the CNNs, the code removes the weights the attacker wants andλ\\lambdathins what is left\.

### Limitations

LEANneeds training data and about 1\.5 times the cost of a plain fine\-tune, and the low\-bit format needs a second one\. Our adaptive attacker is white\-box and knows the code, but it is greedy; a search that plans across codewords could do better, and the protection results rest on the interleaving assumption of Section[II](https://arxiv.org/html/2610.28872#S2)\. Most entries are single runs; the storage comparison and the ResNet\-18 CIFAR\-10 reference model have three seeds \(the experimental setup of Section[IV](https://arxiv.org/html/2610.28872#S4)\), and the accuracy differences of 0\.1 to 0\.5 points in Table[I](https://arxiv.org/html/2610.28872#S4.T1)should be read with that in mind; the 500\-flip outcome of the coded ResNet\-18 sits at a threshold and varies with the training seed, while the 300\-flip outcome does not\. The evaluation is of adaptive flips; random faults are measured at one bit error rate only \(Table[VII](https://arxiv.org/html/2610.28872#S4.T7)\)\. The robust\-training baselines of Table[V](https://arxiv.org/html/2610.28872#S4.T5)were fine\-tuned with our ten\-epoch schedule rather than their original recipes\. The tail term is not a defense on its own: Table[IV](https://arxiv.org/html/2610.28872#S4.T4)shows that alone it lowers the gradient without lowering the damage, which is whyLEANalways pairs it with the range term, and why the real\-flip check should accompany any proxy of this kind\. The ImageNet models cost more bits in the low\-bit security mode: ResNet\-50 needs the 3% tier and thet=79t=79code, 4\.09 bits per weight, to hold 500 flips, and ResNet\-18 at that cost is still 14 points below clean\. Aλ\\lambdafine\-tune that targets the deployed format more directly, or a wider tier chosen during training rather than at packing, are the natural next steps\. In this paperλ\\lambdais the only knob, and it moves the whole tail at once\. The objective has other parameters that could shape the distribution more selectively and that we have not explored: the tail fraction that defines the setTT\(0\.5 to 2% gives the same HIGH bin, but a per\-bin target could act on MID and HIGH separately\), the relative weightβ\\betaof the range and tail terms, the clipping quantileq99q\_\{99\}, and the learning rate and number of epochs, which Table[II](https://arxiv.org/html/2610.28872#S4.T2)already shows to set how far the tail moves for a fixedλ\\lambda\. A family of such controls, each tied to one part of the distribution, is the natural extension of the one\-knob procedure presented here\.

## VIRelated Work

LEANsits between three lines of work, and takes a different position from each\.

### Robust training against bit flips

Piecewise clustering\[[8](https://arxiv.org/html/2610.28872#bib.bib6)\]and quantization\-range learning \(WCAT,\[[9](https://arxiv.org/html/2610.28872#bib.bib7)\]\) make the stored values of a network tolerant to flips by constraining weight magnitudes; Forget\-and\-Rewire\[[10](https://arxiv.org/html/2610.28872#bib.bib8)\]reroutes the most important connections so that single flips matter less; Hessian\-aware training\[[11](https://arxiv.org/html/2610.28872#bib.bib9)\]regularizes curvature\. All of these change the network so that a flip hurts less, and all of them, likeLEAN, are at chance within 50 flips of an adaptive attacker when no code is present \(Table[V](https://arxiv.org/html/2610.28872#S4.T5)\)\. WCAT is the closest to the range term ofLEAN, since both bound the per\-layer quantization range; Table[V](https://arxiv.org/html/2610.28872#S4.T5)separates them under the same code \(WCAT, one run, 75% at 500 flips;LEAN94% in two of three seeds\), and the real\-flip rows of Table[IV](https://arxiv.org/html/2610.28872#S4.T4)show what the tail term adds on top of range control\.LEANdiffers from all four in what it optimizes: not robustness to flips in general but the size of the set that a selective code has to cover, measured in the storage format\.

### Gradient\-norm regularization

PtailP\_\{\\mathrm\{tail\}\}belongs to an old family: double backpropagation\[[23](https://arxiv.org/html/2610.28872#bib.bib21)\]penalizes the input gradient, gradient\-norm regularization and sharpness\-aware minimization\[[24](https://arxiv.org/html/2610.28872#bib.bib22)\]penalize the weight gradient or the loss in a neighbourhood\.LEANdiffers in two details that the ablation shows are the ones that matter for stored weights: the penalty is top\-kk, so only the tail is pushed, and it is weighted by the storage\-format stepΔℓ\\Delta\_\{\\ell\}, which ties it to the range term\. A plain gradient\-norm fine\-tune under the same budget is the natural further baseline, and Tables[IV](https://arxiv.org/html/2610.28872#S4.T4)and[V](https://arxiv.org/html/2610.28872#S4.T5)include it: at the largest coefficient the accuracy tolerates it leaves the tail at 0\.64% and the coded model breaks at 200 flips like the normal one, so the tail\-selective, storage\-weighted form is not a generic gradient smoother\.

### Selective and in\-place protection

Selective and embedded codes such as in\-place zero\-space protection\[[7](https://arxiv.org/html/2610.28872#bib.bib12)\]and E3\-CODE\[[13](https://arxiv.org/html/2610.28872#bib.bib31)\]reduce the parity cost of error correction by protecting only the weights that matter or by hiding the parity in the weights themselves\. Our contribution is upstream of them: it makes the set they protect smaller, and is compatible with any of them\. Detection schemes such as RADAR\[[25](https://arxiv.org/html/2610.28872#bib.bib27)\]and Aegis\[[26](https://arxiv.org/html/2610.28872#bib.bib28)\]catch flips at run time instead of correcting them; a correcting code was chosen here because it needs no response path, but the smaller sensitive set would lower the cost of a checksum in the same way\.

### Low\-bit quantization

LSQ\[[14](https://arxiv.org/html/2610.28872#bib.bib13)\]learns the quantizer step; KURE\[[15](https://arxiv.org/html/2610.28872#bib.bib14)\]shapes the weight distribution for robustness across bit widths; HAWQ\[[27](https://arxiv.org/html/2610.28872#bib.bib15)\]assigns bit widths per layer from Hessian information; SpQR\[[28](https://arxiv.org/html/2610.28872#bib.bib23)\]and SqueezeLLM\[[29](https://arxiv.org/html/2610.28872#bib.bib24)\]keep a small set of outlier or sensitive weights at high precision inside a low\-bit model, which is per\-weight precision of the same kind as our tiers\. Our tiered format assigns bit widths per weight by stored\-format sensitivity, and its difference from those methods is upstream:λ\\lambdamakes the high\-precision set smaller before the format is chosen, and the per\-channel INT4 we use as the LLM baseline is a deliberately simple one \(GPTQ\[[30](https://arxiv.org/html/2610.28872#bib.bib25)\]and AWQ\[[31](https://arxiv.org/html/2610.28872#bib.bib26)\]would lower both rows of Table[XII](https://arxiv.org/html/2610.28872#S5.T12)\)\. At a uniform bit widthLEANmatches LSQ rather than beating it \(Table[VIII](https://arxiv.org/html/2610.28872#S4.T8)\); the gain comes from the ordering, which the shrunken sensitive set makes affordable\. Generating weights from a compact seed\[[32](https://arxiv.org/html/2610.28872#bib.bib32)\]is an orthogonal route to storage efficiency that a reshaped sensitivity distribution does not preclude\.

### Attacks on large language models

AttentionBreaker\[[21](https://arxiv.org/html/2610.28872#bib.bib10)\]and SilentStriker\[[22](https://arxiv.org/html/2610.28872#bib.bib11)\]carry gradient\-guided bit flips to transformers, where a few dozen flips suffice\. The sensitivity structure they exploit is the oneLEANshapes, and Section[V](https://arxiv.org/html/2610.28872#S5)reports a first adaptation\.

## VIIConclusion

The weights of a trained neural network model are not equally sensitive with respect to its functional behavior\. The shape of their sensitivity distribution sets what a bit\-flip attacker needs, what a code must cover, and how many bits each weight has to keep\. Existing methods do not optimize the shape of the weight sensitivity distribution\. We have presentedLEANthat allows us to modulate the weight sensitivity distribution, leading to varying numbers of weights in different weight sensitivity bins\. For example, it helps greatly reduce the number of weights in the highest sensitivity bins by moving them to lower sensitivity bins\. Such a shift in weight sensitivity distribution may result in lower cost for protection against bit\-flip attacks as well as increased efficiency in model weight storage \(e\.g\., sensitivity\-guided variable precision quantization of weights\) and energy/performance for computing activation at each node\.

One composite regularizer on the two factors of the damage, the flip step and the gradient, controlled by a single coefficient, moves the sensitive tail 5 to 13 times lower on nine vision models for an accuracy change under 0\.5 points and 1\.7 to 5\.3 times on three language models, and the amount followsλ\\lambdamonotonically\. The term works as a ten\-epoch fine\-tune or in the last epochs of training, and it stops at a floor set by the model itself, its per\-layer outlier ratios and the curvature around its early layers\. The reshaped distribution then does the rest\. A 1% code holds ResNet\-18 through 300 to 500 adaptive flips where three of four published defenses fail by 300 and WCAT falls to 75% at 500, the parity for a fixed guarantee drops 3\.7 times, a 3\.29\-bit sensitivity\-ordered format, which the small tail makes affordable, stores the ImageNet ResNet\-50 weight payload 2\.4 times smaller than INT8 above the accuracy of LSQ, and a small interleaved code lets the low\-bit models survive too\. The targeted attack on the ImageNet ResNets, which moves through weights the mixed\-data sensitivity does not rank, is the case the current objective does not reach, and shaping the distribution under a class\-aware sensitivity is the next step, as is control through parameters other thanλ\\lambda, such as a per\-bin target or the balance of the two terms, so that different parts of the distribution can be moved independently\.

## References

- \[1\]\(2014\)Flipping bits in memory without accessing them: an experimental study of DRAM disturbance errors\.InProceedings of the 41st International Symposium on Computer Architecture \(ISCA\),pp\. 361–372\.Cited by:[§I](https://arxiv.org/html/2610.28872#S1.p2.1),[§II](https://arxiv.org/html/2610.28872#S2.SS0.SSS0.Px1.p1.1)\.
- \[2\]F\. Yao, A\. S\. Rakin, and D\. Fan\(2020\)DeepHammer: depleting the intelligence of deep neural networks through targeted chain of bit flips\.InProceedings of the 29th USENIX Security Symposium,pp\. 1463–1480\.Cited by:[§I](https://arxiv.org/html/2610.28872#S1.p2.1),[§II](https://arxiv.org/html/2610.28872#S2.SS0.SSS0.Px1.p1.1)\.
- \[3\]A\. S\. Rakin, Z\. He, and D\. Fan\(2019\)Bit\-flip attack: crushing neural network with progressive bit search\.InProceedings of the IEEE/CVF International Conference on Computer Vision \(ICCV\),pp\. 1211–1220\.Cited by:[§I](https://arxiv.org/html/2610.28872#S1.p2.1),[§II](https://arxiv.org/html/2610.28872#S2.SS0.SSS0.Px1.p1.1),[§IV](https://arxiv.org/html/2610.28872#S4.SS0.SSS0.Px7.p1.1)\.
- \[4\]A\. S\. Rakin, Z\. He, J\. Li, F\. Yao, C\. Chakrabarti, and D\. Fan\(2022\)T\-BFA: targeted bit\-flip adversarial weight attack\.IEEE Transactions on Pattern Analysis and Machine Intelligence44\(11\),pp\. 7928–7939\.Cited by:[§I](https://arxiv.org/html/2610.28872#S1.p2.1),[§II](https://arxiv.org/html/2610.28872#S2.SS0.SSS0.Px1.p1.1),[§IV](https://arxiv.org/html/2610.28872#S4.SS0.SSS0.Px7.p1.1)\.
- \[5\]D\. Stutz, N\. Chandramoorthy, M\. Hein, and B\. Schiele\(2021\)Bit error robustness for energy\-efficient DNN accelerators\.InProceedings of Machine Learning and Systems \(MLSys\),Vol\.3,pp\. 569–598\.Cited by:[§I](https://arxiv.org/html/2610.28872#S1.p2.1),[§II](https://arxiv.org/html/2610.28872#S2.SS0.SSS0.Px1.p1.1)\.
- \[6\]H\. Rahaman, S\. Paria, A\. Chatterjee, and S\. Bhunia\(2026\)Evolving landscape of attacks on AI hardware and robust defenses\.In2026 27th International Symposium on Quality Electronic Design \(ISQED\),pp\. 1–6\.Cited by:[§I](https://arxiv.org/html/2610.28872#S1.p2.1),[§II](https://arxiv.org/html/2610.28872#S2.SS0.SSS0.Px1.p1.1)\.
- \[7\]H\. Guan, L\. Ning, Z\. Lin, X\. Shen, H\. Zhou, and S\. Lim\(2019\)In\-place zero\-space memory protection for CNN\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Vol\.32\.Cited by:[§I](https://arxiv.org/html/2610.28872#S1.p2.1),[§II](https://arxiv.org/html/2610.28872#S2.SS0.SSS0.Px3.p1.1),[§VI](https://arxiv.org/html/2610.28872#S6.SS0.SSS0.Px3.p1.1)\.
- \[8\]Z\. He, A\. S\. Rakin, J\. Li, C\. Chakrabarti, and D\. Fan\(2020\)Defending and harnessing the bit\-flip based adversarial weight attack\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),pp\. 14095–14103\.Cited by:[§I](https://arxiv.org/html/2610.28872#S1.p2.1),[§IV](https://arxiv.org/html/2610.28872#S4.SS0.SSS0.Px7.p1.1),[TABLE V](https://arxiv.org/html/2610.28872#S4.T5.4.4.1.1),[§VI](https://arxiv.org/html/2610.28872#S6.SS0.SSS0.Px1.p1.1)\.
- \[9\]K\. Chitsaz, G\. Mordido, J\. David, and F\. Leduc\-Primeau\(2023\)Training DNNs resilient to adversarial and random bit\-flips by learning quantization ranges\.Transactions on Machine Learning Research\.Cited by:[§I](https://arxiv.org/html/2610.28872#S1.p2.1),[§IV](https://arxiv.org/html/2610.28872#S4.SS0.SSS0.Px7.p1.1),[TABLE V](https://arxiv.org/html/2610.28872#S4.T5.4.7.1.1),[§VI](https://arxiv.org/html/2610.28872#S6.SS0.SSS0.Px1.p1.1)\.
- \[10\]N\. Nazari, H\. M\. Makrani, C\. Fang, H\. Sayadi, S\. Rafatirad, K\. N\. Khasawneh, and H\. Homayoun\(2024\)Forget and rewire: enhancing the resilience of transformer\-based models against bit\-flip attacks\.InProceedings of the 33rd USENIX Security Symposium,Cited by:[§I](https://arxiv.org/html/2610.28872#S1.p2.1),[§IV](https://arxiv.org/html/2610.28872#S4.SS0.SSS0.Px7.p1.1),[TABLE V](https://arxiv.org/html/2610.28872#S4.T5.4.6.1.1),[§VI](https://arxiv.org/html/2610.28872#S6.SS0.SSS0.Px1.p1.1)\.
- \[11\]T\. H\. Pranto, S\. Kim, L\. Chen, and S\. Hong\(2025\)Hessian\-aware training for enhancing DNNs resilience to parameter corruptions\.arXiv preprint arXiv:2504\.01933\.Cited by:[§I](https://arxiv.org/html/2610.28872#S1.p2.1),[§IV](https://arxiv.org/html/2610.28872#S4.SS0.SSS0.Px7.p1.1),[TABLE V](https://arxiv.org/html/2610.28872#S4.T5.4.5.1.1),[§VI](https://arxiv.org/html/2610.28872#S6.SS0.SSS0.Px1.p1.1)\.
- \[12\]H\. Rahaman, Q\. Xu, Z\. Haider, P\. Chakraborty, S\. Bhunia, and F\. Suya\(2026\)\(A\)iSpy: parasitic Trojans for machine learning infrastructure\.arXiv preprint arXiv:2607\.17550\.Cited by:[§II](https://arxiv.org/html/2610.28872#S2.SS0.SSS0.Px1.p1.1)\.
- \[13\]V\. N\. Kamineni, H\. Rahaman, O\. Sen, B\. Chatterjee, S\. Bhunia, and R\. Ewetz\(2026\)E3\-CODE: embedded and efficient error\-correcting code for error\-resilient neural networks\.Cited by:[§II](https://arxiv.org/html/2610.28872#S2.SS0.SSS0.Px3.p1.1),[§VI](https://arxiv.org/html/2610.28872#S6.SS0.SSS0.Px3.p1.1)\.
- \[14\]S\. K\. Esser, J\. L\. McKinstry, D\. Bablani, R\. Appuswamy, and D\. S\. Modha\(2020\)Learned step size quantization\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§II](https://arxiv.org/html/2610.28872#S2.SS0.SSS0.Px4.p1.1),[§IV](https://arxiv.org/html/2610.28872#S4.SS0.SSS0.Px8.p2.1),[TABLE VIII](https://arxiv.org/html/2610.28872#S4.T8.2.3.1.1),[§VI](https://arxiv.org/html/2610.28872#S6.SS0.SSS0.Px4.p1.1)\.
- \[15\]M\. Shkolnik, B\. Chmiel, R\. Banner, G\. Shomron, Y\. Nahshan, A\. Bronstein, and U\. Weiser\(2020\)Robust quantization: one model to rule them all\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Vol\.33,pp\. 5308–5317\.Cited by:[§II](https://arxiv.org/html/2610.28872#S2.SS0.SSS0.Px4.p1.1),[§IV](https://arxiv.org/html/2610.28872#S4.SS0.SSS0.Px8.p2.1),[TABLE VIII](https://arxiv.org/html/2610.28872#S4.T8.2.4.1.1),[§VI](https://arxiv.org/html/2610.28872#S6.SS0.SSS0.Px4.p1.1)\.
- \[16\]K\. He, X\. Zhang, S\. Ren, and J\. Sun\(2016\)Deep residual learning for image recognition\.InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition \(CVPR\),pp\. 770–778\.Cited by:[§IV](https://arxiv.org/html/2610.28872#S4.SS0.SSS0.Px1.p1.1)\.
- \[17\]K\. Simonyan and A\. Zisserman\(2015\)Very deep convolutional networks for large\-scale image recognition\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§IV](https://arxiv.org/html/2610.28872#S4.SS0.SSS0.Px1.p1.1)\.
- \[18\]A\. Krizhevsky\(2009\)Learning multiple layers of features from tiny images\.Technical reportUniversity of Toronto\.Cited by:[§IV](https://arxiv.org/html/2610.28872#S4.SS0.SSS0.Px1.p1.1)\.
- \[19\]J\. Deng, W\. Dong, R\. Socher, L\. Li, K\. Li, and L\. Fei\-Fei\(2009\)ImageNet: a large\-scale hierarchical image database\.InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition \(CVPR\),pp\. 248–255\.Cited by:[§IV](https://arxiv.org/html/2610.28872#S4.SS0.SSS0.Px1.p1.1)\.
- \[20\]S\. Han, X\. Liu, H\. Mao, J\. Pu, A\. Pedram, M\. A\. Horowitz, and W\. J\. Dally\(2016\)EIE: efficient inference engine on compressed deep neural network\.InProceedings of the 43rd International Symposium on Computer Architecture \(ISCA\),pp\. 243–254\.Cited by:[Fig\. 10](https://arxiv.org/html/2610.28872#S4.F10),[§IV](https://arxiv.org/html/2610.28872#S4.SS0.SSS0.Px8.p3.1)\.
- \[21\]S\. Das, S\. Bhattacharya, S\. Kundu, S\. Kundu, A\. Menon, A\. Raha, and K\. Basu\(2024\)AttentionBreaker: adaptive evolutionary optimization for unmasking vulnerabilities in LLMs through bit\-flip attacks\.arXiv preprint arXiv:2411\.13757\.Cited by:[§V](https://arxiv.org/html/2610.28872#S5.SS0.SSS0.Px2.p1.1),[§VI](https://arxiv.org/html/2610.28872#S6.SS0.SSS0.Px5.p1.1)\.
- \[22\]H\. Xu, Q\. Peng, J\. Shi, H\. Zheng, Y\. Li, and C\. Zhuo\(2025\)SilentStriker: toward stealthy bit\-flip attacks on large language models\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§V](https://arxiv.org/html/2610.28872#S5.SS0.SSS0.Px2.p1.1),[§VI](https://arxiv.org/html/2610.28872#S6.SS0.SSS0.Px5.p1.1)\.
- \[23\]H\. Drucker and Y\. Le Cun\(1992\)Improving generalization performance using double backpropagation\.IEEE Transactions on Neural Networks3\(6\),pp\. 991–997\.Cited by:[§VI](https://arxiv.org/html/2610.28872#S6.SS0.SSS0.Px2.p1.1)\.
- \[24\]P\. Foret, A\. Kleiner, H\. Mobahi, and B\. Neyshabur\(2021\)Sharpness\-aware minimization for efficiently improving generalization\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§VI](https://arxiv.org/html/2610.28872#S6.SS0.SSS0.Px2.p1.1)\.
- \[25\]J\. Li, A\. S\. Rakin, Z\. He, D\. Fan, and C\. Chakrabarti\(2021\)RADAR: run\-time adversarial weight attack detection and accuracy recovery\.InDesign, Automation and Test in Europe Conference \(DATE\),Cited by:[§VI](https://arxiv.org/html/2610.28872#S6.SS0.SSS0.Px3.p1.1)\.
- \[26\]J\. Wang, Z\. Zhang, M\. Wang, H\. Qiu, T\. Zhang, Q\. Li, Z\. Li, T\. Wei, and C\. Zhang\(2023\)Aegis: mitigating targeted bit\-flip attacks against deep neural networks\.InProceedings of the 32nd USENIX Security Symposium,Cited by:[§VI](https://arxiv.org/html/2610.28872#S6.SS0.SSS0.Px3.p1.1)\.
- \[27\]Z\. Dong, Z\. Yao, A\. Gholami, M\. W\. Mahoney, and K\. Keutzer\(2019\)HAWQ: hessian AWare quantization of neural networks with mixed\-precision\.InProceedings of the IEEE/CVF International Conference on Computer Vision \(ICCV\),pp\. 293–302\.Cited by:[§VI](https://arxiv.org/html/2610.28872#S6.SS0.SSS0.Px4.p1.1)\.
- \[28\]T\. Dettmers, R\. Svirschevski, V\. Egiazarian, D\. Kuznedelev, E\. Frantar, S\. Ashkboos, A\. Borzunov, T\. Hoefler, and D\. Alistarh\(2023\)SpQR: a sparse\-quantized representation for near\-lossless LLM weight compression\.arXiv preprint arXiv:2306\.03078\.Cited by:[§VI](https://arxiv.org/html/2610.28872#S6.SS0.SSS0.Px4.p1.1)\.
- \[29\]S\. Kim, C\. Hooper, A\. Gholami, Z\. Dong, X\. Li, S\. Shen, M\. W\. Mahoney, and K\. Keutzer\(2023\)SqueezeLLM: dense\-and\-sparse quantization\.arXiv preprint arXiv:2306\.07629\.Cited by:[§VI](https://arxiv.org/html/2610.28872#S6.SS0.SSS0.Px4.p1.1)\.
- \[30\]E\. Frantar, S\. Ashkboos, T\. Hoefler, and D\. Alistarh\(2023\)GPTQ: accurate post\-training quantization for generative pre\-trained transformers\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§VI](https://arxiv.org/html/2610.28872#S6.SS0.SSS0.Px4.p1.1)\.
- \[31\]J\. Lin, J\. Tang, H\. Tang, S\. Yang, W\. Chen, W\. Wang, G\. Xiao, X\. Dang, C\. Gan, and S\. Han\(2024\)AWQ: activation\-aware weight quantization for on\-device LLM compression and acceleration\.InProceedings of Machine Learning and Systems \(MLSys\),Cited by:[§VI](https://arxiv.org/html/2610.28872#S6.SS0.SSS0.Px4.p1.1)\.
- \[32\]H\. Rahaman, A\. Chatterjee, and S\. Bhunia\(2025\)Secure and storage\-efficient deep learning models for edge AI using automatic weight generation\.arXiv preprint arXiv:2507\.06380\.Cited by:[§VI](https://arxiv.org/html/2610.28872#S6.SS0.SSS0.Px4.p1.1)\.