Quantization Damage Is Multiplicative, Not Additive

# Quantization Damage Is Multiplicative, Not Additive Preprint. Under review at the Third Workshop on Uncertainty-Aware NLP (UncertaiNLP), EMNLP 2026 (non-archival).
Source: [https://arxiv.org/html/2608.06564](https://arxiv.org/html/2608.06564)
Zekun Wu1,2Swati Dhiman1,2Adriano Koshiyama1 1Holistic AI2UCL Centre for Artificial Intelligence, University College London \{zekun\.wu, swati\.dhiman, adriano\.koshiyama\}@holisticai\.com zekun\.wu\.19@ucl\.ac\.ukswati\.dhiman\.25@ucl\.ac\.uk

###### Abstract

Quantization, storing a model’s weights in fewer bits, is how large language models are actually deployed, and below four bits it is known to hurt\. What nobody can currently say is*how*it hurts: which of the model’s decisions will change when it is compressed to a given bit\-width\. The damage is silent\. A compressed agent stops calling its tools; one bit lower, it loses roughly half its safety refusals, yet benchmark scores barely move\. Prior work assumes quantization adds noise of a roughly fixed size, which would make confident decisions safe\. We measure the decision itself instead\. For each two\-way decision, the*margin*is the score of the option the model picks minus the score of its best alternative; we track it before and after quantization across 16 models from 8 model families, three quantization methods, and bit\-widths from 8 down to 2\. Quantization does not add fixed\-size noise to the margin\. It multiplies the margin by a factor that collapses with bit\-width \(median0\.860\.86at 4 bits,0\.330\.33at 3,0\.000\.00at 2\); we call thismargin shrinkage\. This contraction reduces the protection a large margin affords, letting the model’s own small biases and residual noise push the margin across zero and pick the direction of failure: at 3 bits the decision to call a tool collapses toward inaction while the choice of which tool is untouched\. When competing descriptions of the damage are fitted and compared statistically, additive noise never wins on the damaged tool and safety decisions\. The fitted relation predicts flip rates within a median of 1\.8 percentage points on held\-out decisions, though no flip was used in the fit; per decision, the predicted flip probabilities are calibrated uncertainty estimates \(expected calibration error 0\.004 over 131,758 predictions\)\. The same form holds in every model we measure, but the constants are each model’s own and do not transfer\. A small paired margin set, measured per model and bit\-width, estimates which decisions break without full generative evaluation; under our cost\-matched tests, nothing repairs damage more cheaply than one more bit\.

Quantization Damage Is Multiplicative, Not Additive††thanks:Preprint\. Under review at the Third Workshop on Uncertainty\-Aware NLP \(UncertaiNLP\), EMNLP 2026 \(non\-archival\)\.

## 1Introduction

Post\-training quantization is how large language models are actually deployed\. Quantization to four bits is usually close to lossless, three is risky, two is not usable\(Frantar et al\.,[2022](https://arxiv.org/html/2608.06564#bib.bib6); Dettmers et al\.,[2022](https://arxiv.org/html/2608.06564#bib.bib3)\)\. While the engineering is settled, the nature of the damage is not:*what exactly breaks?*

The answer assumed by most of the literature is that quantization adds noise\(Lin et al\.,[2024](https://arxiv.org/html/2608.06564#bib.bib9); Jang et al\.,[2026](https://arxiv.org/html/2608.06564#bib.bib7)\)\. Round\-off error in the weights propagates to the logits \(the model’s raw output scores\), the argument goes, arriving as a perturbation of roughly fixed size that flips a decision when it exceeds the margin between the model’s top two options\. The clean consequence is that a confidently\-made decision is essentially safe, since a fixed perturbation cannot cross a large margin\(Jang et al\.,[2026](https://arxiv.org/html/2608.06564#bib.bib7)\)\. It also motivates the dominant repair strategy of finding the weights whose perturbation matters most and spending more bits on them\(Lin et al\.,[2024](https://arxiv.org/html/2608.06564#bib.bib9)\)\.

We show the assumption is wrong in a specific and consequential way, and that correcting it makes damage predictable\. The scope, stated up front: the multiplicative*form*holds in every damaged condition we measure, the*constants*do not transfer between models, and we measure both the shared form and the per\-model constants\.

The measurement:We measure individual decisions rather than benchmark scores\. Each item presents a binary choice whose two options have identifiable first tokens; the margin is the logit difference between them, its sign the decision, its magnitude the accumulated evidence\. We record it at full precision and after quantization, for the same item, with the alternative fixed at the full\-precision model’s highest\-scoring wrong option: a paired per\-decision measurement, the unit everything below is built on\.

The finding:Plotting quantized margins against full\-precision ones \(Figure[1](https://arxiv.org/html/2608.06564#S1.F1)\) does not give a cloud of fixed width around the diagonal, which is what additive noise predicts\. It gives a line through the origin with a slope below one\. Damage is*multiplicative*: a decision won by 20 logits and one won by 2 logits lose the same*fraction*of their margin, so the second crosses zero and the first does not\. Writing the fitted relation as

m′=c⋅m\+b\+ε,ε∼𝒩​\(0,σ2\),m^\{\\prime\}=c\\cdot m\+b\+\\varepsilon,\\qquad\\varepsilon\\sim\\mathcal\{N\}\(0,\\sigma^\{2\}\),\(1\)the surviving fractionccis the quantity that collapses with bit\-width,bbis a push specific to each decision family \(the task types of §[2](https://arxiv.org/html/2608.06564#S2)\), andσ\\sigmais what is left over\. We call the phenomenon carried bycc*margin shrinkage*, and Eq\.[1](https://arxiv.org/html/2608.06564#S1.E1)itself the law; both names are used throughout\. Flips are produced jointly by shrinkage and the push\. Shrinkage removes the protection a large margin would otherwise give; the push picks the direction\. The two effects are estimated separately: the additive competitor with a free push \(A1, §[3\.2](https://arxiv.org/html/2608.06564#S3.SS2)\) loses in every damaged tool\-call and safety condition\.

![Refer to caption](https://arxiv.org/html/2608.06564v1/x1.png)Figure 1:Quantized against full\-precision margins \(Qwen3\-4B, whether\-to\-call, round\-to\-nearest\)\. Additive noise predicts a constant\-width band around the identity; the data is a fan through the origin whose slope collapses with bit\-width\.The one fact:Shrinkage is a rate: what survives is a fractionccof the margin, not the margin minus an amount\. Additive noise has a scale of its own\. A proportional loss has none; its size depends on the margin it acts on\. Any defense that relies on the size of the margin is therefore in doubt, and the three this paper can test all fail: a confident margin \(§[3](https://arxiv.org/html/2608.06564#S3)\), protected important weights \(§[6\.2](https://arxiv.org/html/2608.06564#S6.SS2)\), relying on an aggregate benchmark to sound the alarm \(§[5](https://arxiv.org/html/2608.06564#S5)\)\. The interventions that repair anything are the ones that move the rate itself\.

More than a curve fit:Read as a statement about the distribution ofm′m^\{\\prime\}givenmm, the law becomes a working model of the effect of quantization on a measured decision: within this description, everything is carried by\(c,b,σ\)\(c,b,\\sigma\), and the flip\-probability formula follows rather than being assumed\. So the formula is testable against a quantity that played no part in fitting it, the flip rate, which it matches to a median of 1\.8 points on held\-out halves of 868 cells, where each cell is one decision family of one model\-and\-quantizer run at one bit\-width \(1\.5 points scored on the fitting decisions\)\. Where this description fails, the paper measures the failure: families whose noise grows with the margin \(§[4](https://arxiv.org/html/2608.06564#S4)\), one cell whose mean curve is convex rather than linear \(§[3\.2](https://arxiv.org/html/2608.06564#S3.SS2)\), and, at the lowest bit\-widths, the point where the two options stop being what the model would actually generate \(Limitations\)\.

The practical consequence:If damage were additive, protecting the weights that contribute most to the perturbation would help\. If damage is a proportional loss of the whole margin, there is no specific set of weights whose protection should help, so importance\-based protection should not beat a cost\-matched control\. In the four forms we tested, it does not \(§[6\.2](https://arxiv.org/html/2608.06564#S6.SS2)\)\. One exclusion needs no experiment at all: any post\-hoc rescaling of the logits leaves every decision unchanged, because it scales the margin without moving its sign\.

##### Contributions:

1. 1\.Measure and characterize\.Per\-decision margin measurement across bit\-widths, methods and model families establishes*margin shrinkage*: damage is a multiplicative loss of the margin plus a decision\-family\-specific push, and no damaged tool\-call or safety condition in 12 models prefers an additive account \(§[3\.2](https://arxiv.org/html/2608.06564#S3.SS2)\)\.
2. 2\.Predict and bound\.Read conditionally, the law yields a flip\-probability formula with nothing fitted to flips, accurate to a median 1\.8 points on held\-out decisions, and it predicts its own failing family in advance\. A separate bound with no fitted parameters rules out independent error accumulation \(§[4](https://arxiv.org/html/2608.06564#S4), §[3\.3](https://arxiv.org/html/2608.06564#S3.SS3)\)\.
3. 3\.Apply and compare\.At deployment bit\-widths the decision to act collapses one\-directionally while the choice of action survives, invisible to aggregate benchmarks, and among the repairs we tested nothing beats spending one more bit \(§[5](https://arxiv.org/html/2608.06564#S5), §[6\.2](https://arxiv.org/html/2608.06564#S6.SS2)\)\.

##### What this paper does not claim:

The steep drop below 4 bits, instruction\-tuned fragility, and attention re\-routing under compression are published phenomena; new here are the per\-family parameterization, its reading as a model of the conditional distribution of the quantized margin, and the sweep linking re\-routing to decision margins \(§[7](https://arxiv.org/html/2608.06564#S7)spells out what is new against each prior result\)\.

## 2Measurement setup

The margin:Each item presents the model with a choice between two continuations; the margin is the difference between the logits of the two choices’ first tokens\. One real item \(Qwen3\-4B, tool test set \#52\): the next token is either the special token that begins a tool call, or the wordI, which begins a text reply\. At full precision the margin is\+19\.25\+19\.25; at 4 bits,\+16\.25\+16\.25, so 84% survives and the decision stands; at 3 bits,−13\.88\-13\.88, pushed past zero, and the model no longer acts\.

The first token:The decision is made once, where the model commits to a continuation type, so the first token gives a paired, cheap measurement; margin shifts reproduce in free\-running greedy generation \(60 prompts: 9\.3% of positions diverge at 4 bits, 38\.6% at 3; the margin’s size predicts which positions diverge, AUC \(area under the ROC curve\) 0\.918/0\.816\)\.

The quantization methods:Round\-to\-nearest \(RTN\) weight rounding with group size 64, applied in place; GPTQ checkpoints at 4, 3 and 2 bits, calibrated and exported with AutoRound; and llama\.cpp’s released GGUF builds \(q8\_0 as reference, q4\_k\_m, iq4\_xs, iq3\_xxs, iq2\_m\)\.

The test sets:Four author\-constructed batteries, released with the artifact: tool calling, 280 items \(160 whether\-to\-call, 40 which\-tool, 40 argument filling, 40 tool\-result use\); safety, 400 items \(comply or refuse, measured by two instruments: 200 forced binary choice, 200 scored on the reply’s natural first token — the instrument called the*safety opener*below\); general knowledge, 220 items \(60 multiple choice, 60 yes/no, 60 arithmetic, 40 code tokens\); and social bias, 192 items drawn from the BBQ benchmark\(Parrish et al\.,[2022](https://arxiv.org/html/2608.06564#bib.bib11)\), the one external set\. These are the decision families used below \(setup matrix: Appendix Table[3](https://arxiv.org/html/2608.06564#A1.T3)\)\.

The alternative:The “best alternative” is whichever wrong option the full\-precision model scored highest, held fixed for every quantized condition \(tool and safety items have exactly one wrong option\)\.

The estimator:Every table uses one fit: least squares with an intercept, per \(result file, decision family, bit\-width\) cell, one result file being one model’s margins under one quantization method or damage axis \(§[5\.2](https://arxiv.org/html/2608.06564#S5.SS2)\)\. A slope needs the full\-precision margins in a cell to vary; in a family whose decisions are all won by about the same amount it is not identifiable, and we report the flip rate alone \(a cell is estimable when its slope is identifiable with standard error at most 0\.10\)\. This affects 196 of 781 cells \(25\.1%\), structurally: which\-tool selection is 78% unestimable, whether\-to\-call 3%, the safety opener 0%; dropping them would discard the*worst*damage \(median flip rate 0\.33 against 0\.11, Appendix Fig\.[6](https://arxiv.org/html/2608.06564#A1.F6)\)\.

Two sources of uncertainty:The±\\pmvalues onccare regression standard errors, given one particular quantization; five replicates under stochastic rounding \(a quantizer that rounds up or down at random, so each seed yields a different rounding realization\) put the spread across realizations at 3–15×\\timesthe regression standard errors\. Applied throughout:differences inccbelow 0\.04 are not interpretable\.The regressor itself carries no such noise: the full\-precision forward pass is deterministic, so the slope cannot suffer the attenuation toward zero that a noisily measured input inflicts on a regression \(errors\-in\-variables\); run\-to\-run variation exists only on the quantized side, where the replicates bound it\. With the unit fixed, the first question is what damage looks like in it\.

## 3Multiplicative shrinkage

### 3\.1Model\-free evidence

Figure[1](https://arxiv.org/html/2608.06564#S1.F1)plots quantized margins against full\-precision ones for one model’s whether\-to\-call decisions at three bit\-widths\. Additive noise predicts a cloud of roughly constant width hugging the identity line\. At 4 bits the points lie on a line of slope0\.830\.83, already below identity\. At 3 bits the slope is0\.130\.13and the entire should\-call cloud, every point that was positive, has been pushed*below zero*\. At 2 bits the line is flat: the surviving margin no longer depends on the original at all\. The evidence has been erased, not perturbed\. A full\-vocabulary check agrees: the 2\-bit distribution over next tokens is near\-uniform \(median entropy 8\.0 nats against 0\.01 at full precision; uniform over the vocabulary would be about 12\)\. At that bit\-width the measurement reflects which of the two designated continuations the model prefers, not what it would freely generate \(see Limitations\)\. Nor is the shrinkage a global change of logit scale\. At the same condition the surviving fraction is family\-specific \(0\.87 for code tokens against 0\.13 for whether\-to\-call at 3 bits\), and a uniform rescaling of the logits changes no decision at all\. Across the full matrix the medians are 0\.86 at 4 bits, 0\.33 at 3, 0\.00 at 2: the fan pattern is the rule across models, not a quirk of this one\. We treat the 2\-bit floor as the point where the instrument itself stops measuring; the evidence for the law comes from the 4\- and 3\-bit regime\.

### 3\.2Comparison with additive models

A slope below one is suggestive, not conclusive, since an additive model whose noise grows with the margin can produce the same appearance\. We fit four accounts by maximum likelihood and let BIC choose \(the Bayesian information criterion: better likelihood is rewarded, every extra parameter is charged\): A0, constant noise \(m′=m\+εm^\{\\prime\}=m\+\\varepsilon\); A1, with drift \(m′=m\+b\+εm^\{\\prime\}=m\+b\+\\varepsilon\); A2, margin\-scaled noise \(sd​\(ε\)=s0\+s1​\|m\|\\mathrm\{sd\}\(\\varepsilon\)=s\_\{0\}\+s\_\{1\}\|m\|\); and M, the law of Eq\.[1](https://arxiv.org/html/2608.06564#S1.E1)\. A0 and A1 are special cases of M;A2 is the competitor that matters, the only additive form generating a positive correlation between margin size and change size, and not nested in M\. These are competing statistical descriptions of the quantized margin given the clean one, not claims about the weight error’s physical form\. Multiplicative against additive is not a distinction of algebraic form: Eq\.[1](https://arxiv.org/html/2608.06564#S1.E1)can be rewritten as an additive error\(c−1\)​m\+b\+ε\(c\-1\)m\+b\+\\varepsilonwhose size grows with the margin; what the comparison rejects is error whose scale does not depend on the margin\.

The comparison is run*per decision family*, not pooled, because families differ in mean margin and a pooled slope reads between\-family differences as shrinkage\. Table[1](https://arxiv.org/html/2608.06564#S3.T1)summarizes the comparison\. Where damage is real \(fittedc<0\.70c<0\.70\), the multiplicative account wins 85% of damaged cells over 12 models \(23 of 27 in the main example model alone\), every cell in the families the paper’s claims rest on \(whether\-to\-call and both safety instruments — hereafter the*core families*\), and the majority within every model with three or more damaged cells\. The constant\-noise and drift\-only accounts win no damaged cell anywhere; per\-cell BIC differences in the core families run\+28\.8\+28\.8to\+1031\.2\+1031\.2\(above 10 is conventionally decisive\)\. The verdict does not depend on the 0\.70 cutoff \(83\.2% to 85\.0% at every cutoff from 0\.50 to 0\.90\)\. Where quantization is near\-lossless \(c≥0\.70c\\geq 0\.70\) it wins only 9 of 24, as expected: with nothing to explain, the accounts coincide\.One family is a genuine exception:tool\-result use prefers the margin\-scaled\-noise account in every condition, including severely damaged ones \(A2 ahead by 87 to 243 BIC\)\. The exception recurs; the same family, for the same reason, is where the flip forecast of §[4](https://arxiv.org/html/2608.06564#S4)fails\.

Table 1:Damaged\-cell wins in the four\-account comparison over 12 models \(fittedc<0\.70c<0\.70, slope estimable\)\. Entries are cells won, of 569 damaged and 332 core\-family cells; A0–A2 are the additive accounts; their 87 wins all sit outside the core families \(§[3\.2](https://arxiv.org/html/2608.06564#S3.SS2)\)\.A rerun with more flexible competitors added \(an affine form with margin\-growing noise, a power\-curve mean, and their combination\) strengthens the result: over the 27 damaged cells*no*winner has an additive mean, and with a multiplicative\-mean, margin\-scaled\-noise form among the candidates the exception family prefers it to the additive version in every damaged condition \(by 38–173 BIC\), so its mean is multiplicative like everyone else’s and the failing assumption is constant variance alone\. One cell deviates from linearity: the safety forced choice at 3 bits prefers a mean that bows upward \(a convex power form\) by 28\.8 BIC; the other eight damaged core\-family cells are won by the law outright\. Appendix Table[5](https://arxiv.org/html/2608.06564#A1.T5)gives the pooled view; the same comparison on safety\-refusal margins across four models and on social\-bias margins reaches the same result wherever damage is real\. Model comparison of this kind selects the best predictive description among the stated alternatives; it does not prove a generative mechanism, and the competitor set, though built adversarially, is finite\.

### 3\.3A fit\-free bound on error accumulation

![Refer to caption](https://arxiv.org/html/2608.06564v1/x2.png)Figure 2:A parameter\-free upper bound: no error\-accumulation model can lose more than half the margin per bit\. 107 of 183 measured steps sit above it, none where damage is mild\.Before any mechanism, one thing can be settled with no free parameters\. The natural account of the collapse is that each layer contributes a near\-independent error, so error variances add\. If each layer’s output is also rescaled to keep its overall size fixed \(renormalized\), the surviving fraction takes the one\-parameter form on the left, and differentiating it bounds every member of the class, on the right:

c​\(B\)=\(1\+A​γ−B\)−1/2,\|d​ln⁡c/d​B\|<\(ln⁡γ\)/2,c\(B\)=\(1\+A\\gamma^\{\-B\}\)^\{\-1/2\},\\;\|d\\ln c/dB\|<\(\\ln\\gamma\)/2,withBBthe bit\-width\. For weight quantizationγ=4\\gamma=4, because a quantizer’s step halves per bit and its error variance therefore falls fourfold\. Outlier weights scale the prefactorAA, not the exponent, so the bound is insensitive to them\.No member of this independent\-accumulation class can lose more than half its margin per bit removed\. Figure[2](https://arxiv.org/html/2608.06564#S3.F2)draws that ceiling against every measured step\. Across 183 adjacent\-bit steps the data splits exactly at the collapse: at8→68\{\\to\}6and6→56\{\\to\}5bits the median factor lost per bit is×1\.01\\times 1\.01and 0% of steps exceed the bound; at4→34\{\\to\}3it is×2\.43\\times 2\.43with 56% exceeding; at3→23\{\\to\}2,×15\.27\\times 15\.27with 85%\. Overall 107 of 183 steps violate the bound, and the violations are not driven by a few unusual models: every one of the ten measured models violates individually, the median within\-model violating share is 61%, and none of the 32 steps whose endpoints are both mild \(c≥0\.7c\\geq 0\.7\) violates anywhere\. This excludes the class without fitting any member of it, including a closed\-form curve we had fitted in an earlier stage of this work; accounts with signal\-correlated errors or bit\-dependent scale parameters sit outside it\. Ruling out accumulation says what the collapse is not; the next section says what the law is, and what follows from it\.

## 4From a law to a forecast

### 4\.1The conditional characterization

Fix a decision family and a quantization condition\. The claim is about the conditional distribution ofm′m^\{\\prime\}givenmm: ifm′∣m∼𝒩​\(c​m\+b,σ2\)m^\{\\prime\}\\mid m\\sim\\mathcal\{N\}\(c\\,m\+b,\\ \\sigma^\{2\}\)withc,b,σc,b,\\sigmaconstant inmm, two things follow that a regression would not give\. First, the flip probability isforced:

P​\(flip∣m\)=Φ​\(−\(c​m\+b\)​sign​\(m\)/σ\)P\(\\mathrm\{flip\}\\mid m\)=\\Phi\\bigl\(\-\(cm\+b\)\\,\\mathrm\{sign\}\(m\)/\\sigma\\bigr\)\(2\)whereΦ\\Phiis the standard normal CDF\. The formula is a corollary, not an extra assumption\. Second, under the model,\(c,b,σ\)\(c,b,\\sigma\)is thecomplete parameterization of the channel, the random mapping from clean margin to quantized margin: everything the model permits quantization to do to a decision given its full\-precision margin\. Where an assumption fails, the paper reports it \(families whose noise grows with the margin, §[4](https://arxiv.org/html/2608.06564#S4); one bowed mean, §[3\.2](https://arxiv.org/html/2608.06564#S3.SS2)\)\.

Joint Gaussianity of\(m,m′\)\(m,m^\{\\prime\}\)is deliberately not claimed; the full\-precision margins are strongly bimodal, and Appendix[A\.5](https://arxiv.org/html/2608.06564#A1.SS5)contains the test\.

The Gaussian channel is also the generic expectation \(a margin sums many weakly dependent contributions, a central\-limit intuition\), so the characterization predicts its own failure mode: a family whose margin is dominated by a few discrete computations, such as copying a value from context; that is the family where the forecast fails worst \(§[4\.3](https://arxiv.org/html/2608.06564#S4.SS3)\)\.

### 4\.2Forecast accuracy

What the test is:\(c,b,σ\)\(c,b,\\sigma\)are fitted to*margins*, and Eq\.[2](https://arxiv.org/html/2608.06564#S4.E2)is scored against a*flip rate*that played no part in the fit; each cell uses its own parameters, so this tests the law’s shape, not whether its constants transfer\. Over 585 estimable cells the prediction lands within amedian of 1\.5 percentage pointsof the observed rate \(mean absolute error 4\.3pp\)\. Scored on held\-out halves instead, the median is 1\.8 points over 868 cells \(Appendix[A\.6](https://arxiv.org/html/2608.06564#A1.SS6)\)\. Read as a per\-item uncertainty estimate, the formula is well calibrated: over 131,758 predictions from 1,113 cells, decisions assigned flip probabilityppactually flip at a rate within 0\.02 ofpp\(calibration error 0\.004; Brier 0\.081 against 0\.168 for guessing the average\)\. It does not degrade as damage grows, and its signed error \(\+2\.5\+2\.5pp\) is conservative\. The distinction: the margin fails as a*certificate*\(no size is safe under contraction\) but succeeds as a*predictive feature*once the contraction is calibrated\. Cells are not independent \(93 files; these cells span 14 models\); scored per model instead \(median within each model, then across models\) the error is 0\.88pp\.

The limits appear when constants are transferred across models\. Applying another model’s fitted constants to the two transfer\-test models gives 0\.2–8\.5pp of error at 4 bits and 18–33pp at 3 \(Gemma\-3\-4B fully held out; Granite\-8B appears elsewhere in the result matrix, a weaker test\)\. The forecast is accurate where nothing happens and wrong where damage begins \(Appendix Figure[8](https://arxiv.org/html/2608.06564#A1.F8)\)\. Recalibrating on the target model helps in four of six evaluable cells and hurts in two; extrapolating from one measured bit\-width is worse than measuring every bit\-width \(mean absolute error 36\.5 against 14\.9pp\)\.

### 4\.3The exception family

Two families miss: tool\-result use by\+18\.9\+18\.9points and code tokens by\+7\.0\+7\.0\. The first is the interesting one and the theory predicts it in advance\. The audit locates which assumption fails: its residual margins are approximately normal like everyone else’s, so the Gaussian\-residual assumption is intact, and the §[3\.2](https://arxiv.org/html/2608.06564#S3.SS2)rerun with more flexible competing models shows its conditional mean is multiplicative as well; what fails is constant variance alone, with margins atc≈0c\\approx 0still preserving rank order \(Spearman up to 0\.86\), which a constant\-σ\\sigmachannel cannot represent\. A forecast built on constantσ\\sigmamust over\-predict exactly whereσ\\sigmagrows with the margin, and an explicit per\-cell audit confirms the boundary: constant variance is rejected \(a Glejser test, which asks whether the residual’s size grows with the margin; false\-discovery\-rate controlled,q<\.05q<\.05\) in 93% of tool\-result cells and 58% of code cells, the two families where the forecast misses, against 17% for whether\-to\-call and 8% for argument\-fill\. The slopes themselves do not depend on the assumption: refitting every estimable cell by weighted least squares, each item weighted by its own estimated noise, movesccby a median of 0\.0035, and only 7% of cells cross the 0\.04 floor of §[2](https://arxiv.org/html/2608.06564#S2), led by tool\-result use, whose slope we had already declined to interpret\. The one failing family is not an anomaly; it is the theory’s predicted boundary, failing in the predicted way\. The forecast says how many decisions flip; the next section asks which ones, and in which direction\.

## 5Damage at deployment bit\-widths

Before anything new is measured, margin shrinkage explains three published observations that were reported separately\. Benchmark scores staying flat while agents fail\(Jang et al\.,[2026](https://arxiv.org/html/2608.06564#bib.bib7)\)happens because individual margins shrink across zero, item by item, while the aggregate averages over them \(Figure[3](https://arxiv.org/html/2608.06564#S5.F3)\)\. The survival at 4 bits of the refusal direction, an internal activation direction that drives refusals\(Chhabra and Khalili,[2025](https://arxiv.org/html/2608.06564#bib.bib2)\), follows from the law at bit\-widths above the collapse \(c≈1c\\approx 1on both safety instruments\); the same instruments lose half their refusals at 2 bits\. And tool use surviving 4 bits in agentic benchmarks\(Dong et al\.,[2025](https://arxiv.org/html/2608.06564#bib.bib4)\)is consistent with the whether\-to\-call collapse sitting one bit below the bit\-width they tested\. One relation explains all three; and on the three released builds measured both ways, benchmark score and flipped share rank the builds in the same order \(the live function\-calling benchmark ofYan et al\.,[2024](https://arxiv.org/html/2608.06564#bib.bib14); score deltas\+2\.7\+2\.7,−6\.2\-6\.2,−10\.5\-10\.5; flipped shares \.02, \.03, \.10; full grid in Appendix Table[6](https://arxiv.org/html/2608.06564#A1.T6)\)\.

![Refer to caption](https://arxiv.org/html/2608.06564v1/x3.png)Figure 3:One model’s released GGUF builds, end to end: \(a\) benchmark scores barely move; \(b\) the decision margins underneath \(lowest\-bit build against the 8\-bit reference\); \(c\) the flipped share, the should\-not\-call side already losing 29% one\-directionally, five times the benchmark drift\. One bit lower, round\-to\-nearest shows the full collapse \(Table[2](https://arxiv.org/html/2608.06564#S5.T2)\); the full score\-against\-flips grid is Appendix Table[6](https://arxiv.org/html/2608.06564#A1.T6)\.### 5\.1The invocation–selection split

At the bit\-width where damage begins, the two halves of a tool\-calling decision come apart\.*Which*tool to call is untouched: in 32 of 34 model\-and\-method conditions, none of the forty selections changes\.*Whether*to call one has collapsed, and in one direction only: 96% of should\-call decisions flip while 80% of should\-not\-call decisions survive, a 4\.8\-fold asymmetry \(Table[2](https://arxiv.org/html/2608.06564#S5.T2)\)\. The model stops acting; it does not start acting wrongly\.

Table 2:The one\-directional collapse of “should I call a tool?” \(Qwen3\-4B\)\. Read by survivalcc\(the surviving fraction\), not nominal bits:c≳0\.5c\\gtrsim 0\.5leans the other way, the collapse \(bold\) breaks one\-directionally,c≈0c\\approx 0flips both directions\. SC/SNC: fraction of should\-call / should\-not\-call items flipped; ratio==SC/SNC; b/w in the row labels are weight bits;†\\daggermarks exported released checkpoints\.Appendix Figure[4](https://arxiv.org/html/2608.06564#A1.F4)shows the separation opening at 3 bits and closing at 2\.

It isnot a margin\-size artifact: within a common margin band the asymmetry keeps non\-overlapping Wilson confidence intervals in 3 of 7 models at 3 bits and 4 of 7 at 2\. Itsurvives on a released checkpoint\(Table[2](https://arxiv.org/html/2608.06564#S5.T2): GPTQ w2 gives\.963/\.200\.963/\.200at equal measured damage; at w3,c=0\.79c=0\.79, no asymmetry has appeared yet\)\. And it tracksmeasured damage, not the method; every condition withccabove about 0\.5 leans the other way, including all three llama\.cpp settings \(the most aggressive holdsc=0\.59c=0\.59, short of the collapse\)\. The asymmetry belongs to the early stage of damage, where the directional shiftbboutweighs the residual noise; at 2 bits both directions flip and the asymmetry largely fades \(Table[2](https://arxiv.org/html/2608.06564#S5.T2), RTN b2\)\. The pattern is not simply explained by model size\. Qwen3\.5\-9B shows no asymmetry, and the two sub\-2B models flip in inconsistent directions\.

The sharpest form of the claim is at the item level\. Where 96% of items flip, overlap between algorithms is near\-guaranteed; the informative side is where only a fifth flip\. The exported GPTQ w2 model and a simulated 3\-bit condition from an activation\-aware quantizer \(rounding set from observed activations\) are genuinely different models \(mean\|m′\|\|m^\{\\prime\}\|17\.38 against 14\.98\)\. On the should\-call side, where 96% of items flip, both flip the same 77 of 77; the collapse reproduces exactly, though saturation cannot distinguish items\. On the should\-not\-call side only a fifth of items flip, yet the two models flipthe same 16 of 16, just 7 of them among the 16 smallest\|m\|\|m\|\. Across the two algorithms tested, which decision breaks is a property of the item, not of the algorithm\. A flip is a behavioural event, a changed output token, so the sharpness cannot be manufactured by metric choice\(Schaeffer et al\.,[2023](https://arxiv.org/html/2608.06564#bib.bib13)\); no score\-level artifact reproduces item\-for\-item across two algorithms\.

### 5\.2Refusals and the activation axis

The same measurement applied to refusals shows a failure mode aggregate accuracy cannot report\. At 4 bits every model keeps its refusal behaviour on both instruments, including the exported calibrated checkpoint \(refuse side0\.0000\.000on both\)\. One bit lower, model size decides: at 3 bits a 1\.7B model has lost almost all refusals on the more sensitive instrument \(0\.92\), a 4B model a little \(0\.13\), a 32B model none \(0\.00\)\. At 2 bits roughly half the refusals are gone in every model, and one case where the two instruments disagree qualitatively \(Granite: 0\.03 against 0\.64\) is reported unresolved\. A free\-generation check validates the measurement\. On the natural\-opener items, 64\-token greedy completions deliver explicit refusals at an unchanged rate at 4 bits \(70% against 73% at full precision\), and 49% at 3 bits while first\-token margins still prefer refusal \(the lost refusals deflect rather than comply\); at 2 bits the completions are incoherent, neither refusing nor complying\. Where the two diverge, the margin is the slower of the pair to change, so damage read from margins is a lower bound on the behavioural loss\.

Quantizing the activations as well as the weights is a second axis with its own breaking point, and it dominates the first \(Appendix Figure[5](https://arxiv.org/html/2608.06564#A1.F5)\)\. Across 72 conditions on five models from four model families: eight\-bit activations are effectively free and stack with four\-bit weights \(median surviving margin 0\.933\), while 4\-bit activations destroy the decision marginregardless of weight precision\(0\.073 with 8\-bit weights against 0\.085 with 4\-bit weights, inside the rounding spread\)\.

A block\-diagonal Hadamard rotation \(an orthogonal transform that spreads outlier coordinates across a block;Ashkboos et al\.,[2024](https://arxiv.org/html/2608.06564#bib.bib1)\) moves the collapse about two bits lower, a shift predictable to within 0\.21 bits before running any quantization, and at equal measured damage changes nothing else, the sameccand the same flipped items \(77 of 77, 16 of 16\)\. Where the collapse sits is a design choice; what happens at it did not change in anything we measured \(Appendix[A\.8](https://arxiv.org/html/2608.06564#A1.SS8)\)\.

The flat benchmark and the mirage argument summarize the same quantity, the margin \(Figure[3](https://arxiv.org/html/2608.06564#S5.F3)\):Schaeffer et al\. \([2023](https://arxiv.org/html/2608.06564#bib.bib13)\)show apparent sharpness can be a discontinuous metric’s artifact over smooth improvement; we show the reverse, smoothness as an aggregation artifact over sharply flipping decisions\.

## 6Mechanism and open questions

### 6\.1The measured chain

Four measurements explain where the collapse sits and why it is abrupt\. \(1\) The internal disturbance doubles per bit removed: the relative deviation of the residual stream \(the network’s running sum of contributions\) rises from 3\.5–7\.2% at 8 bits to 151–345% at 2 across five models\. \(2\) It grows through depth at a rate independent of bit\-width \(per\-layer ratio 1\.02–1\.07 everywhere\), ruling out our own preferred explanation, an amplification transition \(a bit\-dependent growth rate\)\. \(3\) The map from disturbance to surviving margin, measured by injecting weight noise, is flat below about a third of the stream’s norm and falls off at 70–120%\. \(4\) A quantity that doubles per bit crosses any threshold within one bit; the fitted width is 1\.2 bits\.

Past that threshold the network is not a noisier version of itself\. The fraction of \(layer, head, query\) triples whose top attended position changes accelerates beyond smooth drift crossing 3 to 2 bits in three of four architectures \(the fourth has not collapsed\)\. If the residual stream had merely been rotated, distances preserved, the surviving fraction could not drop below 0\.72 \(a bound from the disturbance size\); the measured 0\.30 means the state is genuinely restructured\. This is a within\-model consistency account, not a causal one; the attention\-gap measure \(a head’s top\-position lead\) does not predict cross\-model robustness \(Limitations\)\.

### 6\.2Tested repairs

One repair class isimpossible on paper: positive rescaling and offset of the logits moves no sign, so no recalibration recovers a flip\.Five fail when measured, importance\-based weight protection among them, none beating a cost\-matched control\.Bias subtraction without labels is a diagnostic, not a fix: it recovers flips only where damage is bias\-dominated \(net\+0\.65\+0\.65there\), 0\.0 at the median over 554 damaged cells\.Three help, as the law predicts: an activation\-aware quantizer, which helps only at the collapse; moving bits from the weights to theKV cache\(16 to 8 bits free at 4\-bit weights; 3 destroys the decision\); and non\-uniform per\-layer bit allocation\. The baseline for every entry:one more bit recovers a median 0\.305 of flipped decisionsover 501 severely damaged conditions; nothing tested beats it \(Appendix[A\.9](https://arxiv.org/html/2608.06564#A1.SS9)\)\.

## 7Related work

GPTQ, AWQ and rotation schemes are standard\(Frantar et al\.,[2022](https://arxiv.org/html/2608.06564#bib.bib6); Lin et al\.,[2024](https://arxiv.org/html/2608.06564#bib.bib9); Ashkboos et al\.,[2024](https://arxiv.org/html/2608.06564#bib.bib1)\); the drop below 4 bits is well documented\(Ouyang et al\.,[2025](https://arxiv.org/html/2608.06564#bib.bib10)\)\. Precision\-aware scaling laws\(Kumar et al\.,[2025](https://arxiv.org/html/2608.06564#bib.bib8)\)predict the aggregate*loss*cost of a bit\-width; ours predicts*which decisions*the cost lands on, one bit lower\.

A concurrent agent study\(Jang et al\.,[2026](https://arxiv.org/html/2608.06564#bib.bib7)\)explains quantized tool\-calling failures with an additive thin\-margin account; we agree on where and correct how \(§[3\.2](https://arxiv.org/html/2608.06564#S3.SS2)\)\.Chhabra and Khalili \([2025](https://arxiv.org/html/2608.06564#bib.bib2)\)report that the activation direction driving refusals is preserved; we agree at 4 bits, not at 3 and 2 \(a loss an adversary can trigger;Egashira et al\.,[2024](https://arxiv.org/html/2608.06564#bib.bib5)\)\. KV\-cache work\(Salfati,[2026](https://arxiv.org/html/2608.06564#bib.bib12)\)states the threshold argument at one bit\-width; we add the sweep\.

## 8Conclusion

Quantization damages decisions by contracting every margin while a family\-specific push picks the direction, not by adding fixed\-size noise\. Contraction has no scale, so scale\-based defenses fail together; acting collapses while choosing survives; no tested repair beats one more bit\.

## Limitations

Scope and support:Post\-training quantization; decisions read at the first token, single\-step; five models on the activation axis, sixteen on the weight axis\. A quarter of the result matrix is not slope\-estimable and is reported as flip rates; those are the worst\-damaged cells, not the quiet ones\. Benjamini–Hochberg correction \(the standard false\-discovery\-rate control\) covers the per\-cell tests ofc=1c=1andb=0b=0across 781 cells and*not*the downstream descriptive tallies \(23 of 27, 107 of 183, and the rest\), which are uncorrected counts over a fixed set of conditions and should be read as such; the two largest counts are also reported clustered by model in the sections where they appear, and Appendix Table[7](https://arxiv.org/html/2608.06564#A1.T7)reconciles every denominator \(the bound violations in §[3\.3](https://arxiv.org/html/2608.06564#S3.SS3), the forecast accuracy in §[4](https://arxiv.org/html/2608.06564#S4)\)\. The forecast needs one calibration pass per model: borrowed constants are accurate at 4 bits and wrong by 18–33pp at 3, and extrapolating from one measured bit\-width is worse than the measured table\. The refusal\-against\-comply split cannot be measured on GGUF builds, whose interface returns log probabilities only for the 200 most likely next tokens \(only about 45% of items are covered, and the covered items skew toward close decisions\), and is reported as unmeasured\. A full\-vocabulary check also locates the instrument’s boundary: at 4 bits the model’s single most likely next token, read over the whole vocabulary, is one of the item’s two options in 85% of tool items, so the margin comparison also reflects what the model actually generates; at round\-to\-nearest 3 and 2 bits that share falls to 30% and zero, with the preferred option’s median vocabulary rank near 3,500 and 91,000\. Severely damaged rows therefore describe the conditional preference between the designated continuations, which is what grammar\-constrained decoding \(decoding held to a legal output format\) executes and what reproduces item\-for\-item across algorithms, not the unconstrained emission\. The free\-generation check of §[5\.2](https://arxiv.org/html/2608.06564#S5.SS2)bounds the direction of the error: where first\-token margins and 64\-token behaviour diverge, the margin under\-reports the change, so damage read from margins is conservative\. Our round\-to\-nearest grid leaves one of the2b2^\{b\}quantization levels unused, slightly harsher than an optimal symmetric quantizer\.

The mechanism’s limits:The attention\-gap measure that explains the collapse within a model does not predict which models are more robust than others, a synthetic layer stack built to reproduce the collapse predicts a scaling with network depth that real models violate, and why large models preserve a larger fraction of their margin is open\. The candidate explanation left standing — decorrelation, the standardized margin losing its correlation with its clean counterpart rather than its scale, which fits rank order persisting atc≈0c\\approx 0— is untested here\. One result from an earlier draft is permanently withdrawn: the claim that a directly measured disturbance predicts cross\-model robustness better than the constant derived from the amplification hypothesis of §[6\.1](https://arxiv.org/html/2608.06564#S6.SS1)\. Re\-measuring with definitions fixed in advance showed our results cannot supply the cross\-model quantity that claim needs; the two predictors are also correlated at 0\.97\. Whether the one model whose tool\-call decision never responds to quantization \(Llama\-3\.1\-8B\) reflects this particular checkpoint or its whole model family is left open: the model chosen to settle it does not call tools at full precision\.

## References

- Ashkboos et al\. \(2024\)Saleh Ashkboos, Amirkeivan Mohtashami, Maximilian L\. Croci, Bo Li, Martin Jaggi, Dan Alistarh, Torsten Hoefler, and James Hensman\. 2024\.QuaRot: Outlier\-free 4\-bit inference in rotated LLMs\.*Advances in Neural Information Processing Systems*\.
- Chhabra and Khalili \(2025\)Vishnu Kabir Chhabra and Mohammad Mahdi Khalili\. 2025\.Towards understanding and improving refusal in compressed models via mechanistic interpretability\.*arXiv preprint arXiv:2504\.04215*\.
- Dettmers et al\. \(2022\)Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer\. 2022\.LLM\.int8\(\): 8\-bit matrix multiplication for transformers at scale\.*Advances in Neural Information Processing Systems*\.
- Dong et al\. \(2025\)Peijie Dong, Zhenheng Tang, Xiang Liu, Lujun Li, Xiaowen Chu, and Bo Li\. 2025\.Can compressed LLMs truly act? an empirical evaluation of agentic capabilities in LLM compression\.*arXiv preprint arXiv:2505\.19433*\.
- Egashira et al\. \(2024\)Kazuki Egashira, Mark Vero, Robin Staab, Jingxuan He, and Martin Vechev\. 2024\.Exploiting LLM quantization\.*Advances in Neural Information Processing Systems*\.ArXiv:2405\.18137\.
- Frantar et al\. \(2022\)Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh\. 2022\.GPTQ: Accurate post\-training quantization for generative pre\-trained transformers\.*arXiv preprint arXiv:2210\.17323*\.
- Jang et al\. \(2026\)Jiwon Jang, Kisu Yang, Heuiseok Lim, and Hyunwoo Park\. 2026\.Flat score, amplified failures: How the error budget masks damage in quantized LLM agents\.*arXiv preprint arXiv:2607\.27275*\.
- Kumar et al\. \(2025\)Tanishq Kumar, Zachary Ankner, Benjamin F\. Spector, Blake Bordelon, Niklas Muennighoff, Mansheej Paul, Cengiz Pehlevan, Christopher Ré, and Aditi Raghunathan\. 2025\.Scaling laws for precision\.In*International Conference on Learning Representations*\.ArXiv:2411\.04330\.
- Lin et al\. \(2024\)Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei\-Ming Chen, Wei\-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han\. 2024\.AWQ: Activation\-aware weight quantization for LLM compression and acceleration\.In*Proceedings of Machine Learning and Systems \(MLSys\)*\.
- Ouyang et al\. \(2025\)Xu Ouyang, Tao Ge, Thomas Hartvigsen, Zhisong Zhang, Haitao Mi, and Dong Yu\. 2025\.Low\-bit quantization favors undertrained LLMs: Scaling laws for quantized LLMs with 100t training tokens\.In*Proceedings of ACL*\.ArXiv:2411\.17691\.
- Parrish et al\. \(2022\)Alicia Parrish, Angelica Chen, Nikita Nangia, Vishakh Padmakumar, Jason Phang, Jana Thompson, Phu Mon Htut, and Samuel R\. Bowman\. 2022\.BBQ: A hand\-built bias benchmark for question answering\.In*Findings of ACL*\.
- Salfati \(2026\)Samuel Salfati\. 2026\.Quantization dominates rank reduction for KV\-cache compression\.*arXiv preprint arXiv:2604\.11501*\.
- Schaeffer et al\. \(2023\)Rylan Schaeffer, Brando Miranda, and Sanmi Koyejo\. 2023\.Are emergent abilities of large language models a mirage?*Advances in Neural Information Processing Systems*\.
- Yan et al\. \(2024\)Fanjia Yan, Huanzhi Mao, Charlie Cheng\-Jie Ji, Tianjun Zhang, Shishir G\. Patil, Ion Stoica, and Joseph E\. Gonzalez\. 2024\.Berkeley function\-calling leaderboard\.[https://gorilla\.cs\.berkeley\.edu/leaderboard\.html](https://gorilla.cs.berkeley.edu/leaderboard.html)\.

## Appendix AAppendix

### A\.1The experimental setup at a glance

Table 3:The setup matrix: quantization streams, their bit\-widths, model coverage, and the test sets they are scored on\. Battery sizes: tool 280 items \(whether\-to\-call 160, which\-tool 40, argument filling 40, tool\-result use 40\); safety 400 \(forced choice 200, safety opener 200\); general knowledge 220; BBQ social bias 192, the one external set\.Table[3](https://arxiv.org/html/2608.06564#A1.T3)summarizes the experimental grid whose prose description is §[2](https://arxiv.org/html/2608.06564#S2); every stream is scored with the same margin instrument and the same estimator\. Table[4](https://arxiv.org/html/2608.06564#A1.T4)documents the custom calibration set behind the calibrated paths\.

Table 4:The custom calibration set used for the calibrated quantization paths\. General text and tool\-calling dialogues are mixed at roughly 60/40 by tokens; dialogues are rendered through the target model’s chat template so the calibration stream contains the real tool\-call special tokens\. Documents are truncated at 2048 tokens\. Released with the artifact\.
### A\.2Figures and tables referenced from the main text

The figures and tables below are referenced from §[3\.3](https://arxiv.org/html/2608.06564#S3.SS3), §[4](https://arxiv.org/html/2608.06564#S4)and §[5](https://arxiv.org/html/2608.06564#S5); they appear in the order the body cites them\.

![Refer to caption](https://arxiv.org/html/2608.06564v1/x4.png)Figure 4:The invocation/selection split at 3 bits \(selection untouched in 32 of 34 conditions while whether\-to\-call collapses\) and its closure at 2, where selection collapses too\.![Refer to caption](https://arxiv.org/html/2608.06564v1/x5.png)Figure 5:The activation\-axis grid: 72 conditions, five models, both test sets; at four\-bit activations the weight setting stops mattering; the activation axis dominates the weight axis\.
### A\.3Unestimable cells

Figure[6](https://arxiv.org/html/2608.06564#A1.F6)shows that the quarter of cells whose slope cannot be estimated are the worst\-damaged ones, which is why they are reported as flip rates rather than dropped\.

![Refer to caption](https://arxiv.org/html/2608.06564v1/x6.png)Figure 6:The 25% of cells whose slope cannot be estimated are the*most*damaged ones, not the quiet ones: median flip rate 0\.33 against 0\.11, and 54% of them lose more than a quarter of their decisions against 37%\. Dropping them would discard the worst damage in the matrix, which is why they are reported as flip rates rather than omitted\.
### A\.4The pooled comparison

Table[5](https://arxiv.org/html/2608.06564#A1.T5)gives the pooled per\-condition view of the noise\-against\-shrinkage comparison; the per\-family comparison in §[3\.2](https://arxiv.org/html/2608.06564#S3.SS2)is what the paper relies on\.

Table 5:Does quantization add noise or shrink the signal? Pooled fits per condition \(Qwen3\-4B; the per\-family comparison is described in the text\)\.Δ\\DeltaBIC\>10\>10is decisive for shrinkage over the best additive account; the correlation column is evidence against constant\-noise accounts only;R2R^\{2\}is fit quality only\. RTN is round\-to\-nearest\.
### A\.5Joint Gaussianity, tested and not claimed

Joint Gaussianity of\(m,m′\)\(m,m^\{\\prime\}\)would be the tempting stronger claim; it holds exactly when the conditional holds*and*mmis itself Gaussian, and that extra requirement fails: the full\-precision margins have strongly negative excess kurtosis in every core family \(−1\.71\-1\.71to−0\.61\-0\.61\), the signature of bimodality, two humps rather than one bell, which is what a two\-sided decision family must look like\. The forecast is computed per decision from its own observed margin, so it never assumed a Gaussian input\.

### A\.6One law over every axis of damage

Figure[7](https://arxiv.org/html/2608.06564#A1.F7)puts all of it on one panel: 868 cells from weight rounding under two calibrated methods, activation quantization with and without the rotation, and exported checkpoints, each cell’s flip rate predicted from parameters fitted on the*other half*of its own decisions, never on flips\. Median error 1\.8 points; if any damage axis sat off the diagonal, the parameterization would not cover it; none does\.

![Refer to caption](https://arxiv.org/html/2608.06564v1/x7.png)Figure 7:Forecast accuracy across the full result matrix: 868 cells, every axis of damage, one parameterization; flip rates predicted on held\-out items, never fitted on flips\. Median error 1\.8 points \(in\-sample error on these 868 cells is 1\.2 points; the held\-out version is plotted to avoid overstating accuracy\)\.![Refer to caption](https://arxiv.org/html/2608.06564v1/x8.png)Figure 8:The forecast on two models never used to build the table \(circles: Gemma\-3\-4B; squares: Granite\-3\.3\-8B; colour gives the bit\-width; the five worst misses are labelled\)\. At 4 bits it transfers \(median error 0\.5pp\); at 3 bits and below, borrowed constants are wrong by 18–33pp\. The shape holds, the constants do not transfer\.
### A\.7Benchmark scores against the margin instrument

Table[6](https://arxiv.org/html/2608.06564#A1.T6)expands the three\-build comparison of §[5](https://arxiv.org/html/2608.06564#S5)into the full grid: one model’s released GGUF builds scored on four categories of the live function\-calling benchmark\(Yan et al\.,[2024](https://arxiv.org/html/2608.06564#bib.bib14)\)beside the margin instrument’s flip shares on the same builds\. The single\-turn categories drift within a few points, and irrelevance detection even rises at the 2\-bit\-class build, consistent with the collapse pushing whether\-to\-call decisions toward not calling; the multi\-turn category falls from 25\.5 to 7\.5; and the margin instrument sees the flip mass grow tenfold over the same span\. Mac\- and cluster\-scored categories are mixed here; the same artifact scores agree to 0\.5pp across hosts\.

Table 6:Released GGUF builds \(Qwen3\-4B\): benchmark score against measured decision flips\. BFCL category accuracies \(%\); flip shares against the q8\_0 reference over 226 paired decisions \(all families / whether\-to\-call side\)\.
### A\.8The rotation intervention

Those bit\-widths are properties of a quantization scheme, and we identify which scheme by measuring an alternative one\. The activation quantizer above is per\-token min–max with no outlier handling, the regime that a substantial line of systems work is designed to escape\(Ashkboos et al\.,[2024](https://arxiv.org/html/2608.06564#bib.bib1)\)\. Re\-running the same sweep with a block\-diagonal Hadamard rotation around the quantizer moves the collapseabout two bits lower: the plain scheme fails between 8\-bit and 6\-bit activations, the rotated one holds to 4 bits and fails between 4 and 3, and in all four model×\\timestest\-set cells the 6\-bit setting is rescued outright \(0\.17–0\.53 to 0\.91–0\.96\)\. That two\-bit figure is derivable before any quantization run\. A per\-token min–max quantizer’s step is the token’s range over2B2^\{B\}, and an orthogonal rotation preserves the norm while shrinking the range, so a range compression ofRRis worth exactlylog2⁡R\\log\_\{2\}Rbits\. Measured from forward passes alone, the prediction lands at 2\.24 against an observed 2\.41 \(Qwen3\-4B tool\), 2\.18 against 1\.96, 2\.46 against 2\.47, and 2\.40 against 2\.82: a mean absolute error of0\.21 bits with nothing fitted\. The predictor must be the 90th\-percentile input to the MLP down\-projection \(down\_proj\), not the median tensor \(which would predict 0\.96 bits\): the collapse is set by the worst\-conditioned tensor\. What the rotation does to that tensor is exactly what “spreading the outliers” should mean; its excess kurtosis falls from 141 to 3\.

What the rotation does*not*do is remove the collapse, and that is the part that matters here \(Figure[9](https://arxiv.org/html/2608.06564#A1.F9)\)\. At equal measured damage the two schemes are indistinguishable: plain 5\-bit activations and rotated 3\-bit activations both sit atc=0\.12c=0\.12, both flip\.963\.963of should\-call decisions against\.200\.200of should\-not\-call ones, and the flipped items overlap 77 of 77 and 16 of 16\. Where the collapse sits is a design choice worth about two bits\. What happens once you reach it is not a design choice at all\.

![Refer to caption](https://arxiv.org/html/2608.06564v1/x9.png)Figure 9:The intervention: a block\-Hadamard rotation around the activation quantizer moves the collapse about two bits lower \(left\), a shift derivable to 0\.21 bits from the range compression of the worst tensor, and at equal measured damage changes nothing else: samecc, same flip rates, same items \(right\)\.#### Reproducibility statement

Every number in this paper is generated from a named result file by a checked\-in script; a check collects the result files from every machine used and verifies that each artifact is cited, and a second check verifies every numeric literal in the source against stored values\. Code, test sets, and the full result matrix will be released\.

Further appendix material \(the full result matrix; survival curves across bit\-widths; the base\-against\-instruct comparison inside the collapse region; the map from injected noise to margin loss; controls that ablate the directional push; the grid over KV\-cache bit allocations; and the repair\-test table with its cost\-matched controls\) is contained in the remaining figures and tables of the artifact release\.

### A\.9The repair tests in full

One repair class isanalytically impossible: any post\-hoc transformℓ→α​ℓ\+β\\ell\\to\\alpha\\ell\+\\betawithα\>0\\alpha\>0scales every margin without moving its sign, so no temperature or bias recalibration recovers a single flipped decision\.Five fail when measured: protecting “important” weights \(four attempts, including a control protecting an equal byte budget of random weights, which performs identically\), keeping trailing blocks at full precision, margin\-gradient rounding, dithering \(noise added before rounding\), and attention\-only protection at matched cost, which wins in only three of five model families — too unstable to count as a success\.One requires knowing each item’s correct answer \(an oracle\), and its variant without ground\-truth labels is a diagnostic rather than a fix: subtracting the per\-item measured offset recovers a median 0\.325 of flipped decisions when told which answer is correct\. Estimatingbbper family on a calibration split and subtracting it, which needs no labels, recovers flips exactly where the law says damage is bias\-dominated \(35 of 46 on the largest one\-directional condition, net\+0\.65\+0\.65\) and nets0\.0 at the medianover 554 damaged cells, because where decorrelation dominates there is no bias to remove\. The practical results are two: importance\-based protection consistently fails against cost\-matched controls, and the usual bit allocation between weights and cache should be inverted\.

Three help, in the way the law predicts: a proof\-of\-concept activation\-aware quantizer helps exactly at the collapse and nowhere else \(15 of 19 matched conditions at 3 bits, chance\-level at 4 and 2; evidence about our implementation, not the method class\); moving bits from the weights to theKV cache\(the stored attention keys and values\) inverts the usual allocation \(cache 16 to 8 bits is free at 4\-bit weights, cache 3 bits is fatal\); and non\-uniform bit allocation across layers helps while the sensitivity*ranking*justifying it does not\. Every entry is measured against the same baseline:one more bit recovers a median 0\.305 of flipped decisionsover the 501 severely damaged conditions\. Nothing tested beats it\.

### A\.10Cell accounting

The paper’s counts come from sweeps with different scopes and filters; this table reconciles them\.

Table 7:Where each denominator comes from\. “Estimable” always means the slope’s standard error is at most 0\.10\.