QAM-W: Joint 2D Codebook Quantization for LLM Weights via Hadamard Rotation and Activation-Aware Scaling

# Joint 2D Codebook Quantization for LLM Weights via Hadamard Rotation and Activation-Aware Scaling
Source: [https://arxiv.org/html/2605.26339](https://arxiv.org/html/2605.26339)
Preetam Sharma Independent Research preetam@manifoldlab\.ai&Kacper Dobek Institute of Computing Science Poznan University of Technology, Poznan, Poland kacper\.dobek@cs\.put\.poznan\.pl

###### Abstract

Scalar post\-training quantizers discard pairwise coordinate structure within weight rows\. We introduce QAM\-W \(Quadrature Amplitude Modulation for Weights\), a codec that recovers this structure: each row is L2\-normalized, block\-Hadamard rotated, paired into 2D coordinates, and quantized against a single Lloyd\-Max codebook trained on the unit circular Gaussian, with activation\-aware per\-channel scaling\. In a cross\-model study spanning five LLMs from four families \(1\.1B–13B parameters\) and eight quantized configurations, the activation\-aware variant at≈5\.5\\approx 5\.5bpw stays within±0\.4%\\pm 0\.4\\%of BF16 WikiText\-2 perplexity on every model, matching the SmoothQuant W8A8 quality envelope at32%32\\%fewer weight bits\. Joint 2D coding outperforms polar \(amplitude×\\timesphase\) coding by 2–15 ppΔ\\DeltaPPL at equal bitrate, and paired KL against BF16 tracksΔ\\DeltaPPL% at Spearmanρ=0\.99\\rho=0\.99across 37 \(method, model\) rows, consistent with a monotone composite bound from codec distortion to KL divergence\. A 3\.5 bpw variant is competitive on quantization\-tolerant architectures\. At strict 4 bpw, the rotated\-codebook frontier method QTIP outperforms QAM\-W; the contribution is the quality\-preserving 5–6 bpw band\.

QAM\-W: Joint 2D Codebook Quantization for LLM Weights via Hadamard Rotation and Activation\-Aware Scaling

Preetam SharmaIndependent Researchpreetam@manifoldlab\.aiKacper DobekInstitute of Computing SciencePoznan University of Technology, Poznan, Polandkacper\.dobek@cs\.put\.poznan\.pl

## 1Introduction

Most post\-training quantizers in current use are scalar: each weight is mapped independently to a small integer index, with per\-group or per\-channel scales\. Scalar coding is simple but leaves rate\-distortion slack for sources whose coordinates are correlated\. Within a weight row, coordinates are typically correlated; after an orthogonal rotation they approximate a 2D circular Gaussian\. Quantizing pairs jointly – as points in the plane rather than as two independent scalars – recovers pair\-level structure that a scalar codebook discards\.

The empirical claim of this work is bounded:*in the≈5\\approx 5–66bpw quality\-preserving regime*, an activation\-aware joint\-2D codec stays within±0\.4%\\pm 0\.4\\%of BF16 perplexity across five LLMs from four families spanning1\.11\.1B–1313B parameters, matching SmoothQuant\(Xiaoet al\.,[2023](https://arxiv.org/html/2605.26339#bib.bib1)\)W8A8’s quality at≈32%\\approx 32\\%fewer*weight*bits\. At strict44bpw, QTIP\(Tsenget al\.,[2024b](https://arxiv.org/html/2605.26339#bib.bib27)\)outperforms QAM\-W \([section˜5\.2](https://arxiv.org/html/2605.26339#S5.SS2)\); the contribution here is the 5–6 bpw band\.

Rotation\-based methods such as QuIP\(Cheeet al\.,[2023](https://arxiv.org/html/2605.26339#bib.bib5)\)and QuIP\#\(Tsenget al\.,[2024a](https://arxiv.org/html/2605.26339#bib.bib6)\)apply incoherent rotations to homogenize coordinate magnitudes before scalar or lattice coding\. Activation\-aware methods such as AWQ\(Linet al\.,[2024](https://arxiv.org/html/2605.26339#bib.bib2)\)exploit the per\-channel activation scale but leave joint coordinate structure untouched\. The codec presented here combines all three ideas: a deterministic block\-Hadamard rotation, joint 2D Lloyd\-Max coding against a single codebook trained on the unit circular Gaussian\(Lloyd,[1982](https://arxiv.org/html/2605.26339#bib.bib9)\), and AWQ\-style per\-channel scaling driven by the layer\-output trace identity\.

Throughout, reported bits\-per\-weight \(bpw\) refers to weight\-storage and memory\-bandwidth footprint\. The current pipeline dequantizes to BF16 before matmul; latency claims would require a fused kernel and are out of scope \(see[Limitations](https://arxiv.org/html/2605.26339#Sx1)\)\.

##### Contributions:

- •Codec and analysis\.A row\-norm\-factored, block\-Hadamard\-rotated, joint 2D Lloyd\-Max codec for LLM weights with activation\-aware scaling, backed by a composite monotone bound from codec distortion to KL divergence \([sections˜3](https://arxiv.org/html/2605.26339#S3)and[4](https://arxiv.org/html/2605.26339#S4)\)\.
- •Cross\-model study\.Eight quantized configurations on five LLMs under a unified perplexity, paired\-KL, and six\-task harness protocol \([sections˜5\.1](https://arxiv.org/html/2605.26339#S5.SS1)and[5\.2](https://arxiv.org/html/2605.26339#S5.SS2)\)\.
- •KL as rank diagnostic\.Across3737\(method, model\) rows, paired KL andΔ\\DeltaPPL% are rank\-correlated at Spearmanρ=0\.99\\rho=0\.99\.

## 2Related Work

##### Scalar post\-training quantization\.

Round\-to\-nearest \(RTN\) quantization\(Krishnamoorthi,[2018](https://arxiv.org/html/2605.26339#bib.bib13)\)with per\-group scaling is the simplest baseline\. GPTQ\(Frantaret al\.,[2023](https://arxiv.org/html/2605.26339#bib.bib3)\)compensates quantization error column\-by\-column using approximate second\-order information\. AWQ\(Linet al\.,[2024](https://arxiv.org/html/2605.26339#bib.bib2)\)protects high\-activation channels via per\-channel scaling; AutoRound\(Chenget al\.,[2023](https://arxiv.org/html/2605.26339#bib.bib14)\)refines rounding with SignSGD; HQQ\(Badri and Shaji,[2023](https://arxiv.org/html/2605.26339#bib.bib36)\)treats rounding as a half\-quadratic problem without calibration data\. SmoothQuant\(Xiaoet al\.,[2023](https://arxiv.org/html/2605.26339#bib.bib1)\)redistributes quantization difficulty from activations to weights through a diagonal scaling transform, targeting joint W8A8\. OmniQuant\(Shaoet al\.,[2024](https://arxiv.org/html/2605.26339#bib.bib31)\)jointly learns clipping and smoothing transforms\. LLM\.int8\(\)\(Dettmerset al\.,[2022](https://arxiv.org/html/2605.26339#bib.bib32)\)keeps outlier activation channels in 16\-bit mixed precision\. SqueezeLLM\(Kimet al\.,[2024](https://arxiv.org/html/2605.26339#bib.bib35)\)couples a non\-uniform scalar codebook with a dense\-and\-sparse decomposition; QLoRA’s NF4\(Dettmerset al\.,[2023](https://arxiv.org/html/2605.26339#bib.bib4)\)places levels on the Gaussian distribution\.

##### Rotated and vector quantization\.

QuIP\(Cheeet al\.,[2023](https://arxiv.org/html/2605.26339#bib.bib5)\)and QuIP\#\(Tsenget al\.,[2024a](https://arxiv.org/html/2605.26339#bib.bib6)\)apply random or structured orthogonal rotations to decorrelate weight coordinates before quantization\. QuaRot\(Ashkbooset al\.,[2024](https://arxiv.org/html/2605.26339#bib.bib33)\)extends rotation through the residual stream for end\-to\-end 4\-bit inference; SpinQuant\(Liuet al\.,[2025](https://arxiv.org/html/2605.26339#bib.bib34)\)replaces the fixed rotation with a learned one\. QAM\-W shares the rotation principle but opts for a deterministic block\-Hadamard, which requires no calibration on the rotation itself\. Iso\-bpw comparison against the learned\-rotation family is deferred to future work\.

On the VQ side, AQLM\(Egiazarianet al\.,[2024](https://arxiv.org/html/2605.26339#bib.bib7)\)uses additive codebooks and pushes below2\.52\.5bpw\. QTIP\(Tsenget al\.,[2024b](https://arxiv.org/html/2605.26339#bib.bib27)\)combines anE8E\_\{8\}\-lattice trellis with Hadamard incoherence and is the strongest published rotated\-codebook method; it and QAM\-W occupy different operating points, with QTIP stronger at strict 4 bpw and QAM\-W stronger in the 5–6 bpw band \([section˜5\.2](https://arxiv.org/html/2605.26339#S5.SS2)\)\. VPTQ\(Liuet al\.,[2024](https://arxiv.org/html/2605.26339#bib.bib29)\)stacks residual codebooks down to∼2\\sim\\\!2bpw; GPTVQ\(van Baalenet al\.,[2024](https://arxiv.org/html/2605.26339#bib.bib30)\)shows the rate\-distortion gain from higher codebook dimension saturates aroundd=8d\\\!=\\\!8, placing QAM\-W’sd=2d\\\!=\\\!2choice at the small end of that spectrum by design\.

##### Concurrent work\.

PolarQuant\(Vicentino,[2026](https://arxiv.org/html/2605.26339#bib.bib38)\)couples a Walsh\-Hadamard rotation with a*scalar*\(d=1d\\\!=\\\!1\) Lloyd\-Max codebook, sharing QAM\-W’s first two pipeline stages but differing in codebook dimension\. Thed=1d\\\!=\\\!1vs\.d=2d\\\!=\\\!2ablation is the highest\-priority follow\-up\.

##### Signal\-processing origins\.

Quadrature amplitude modulation \(QAM\) originates in digital communications\(Proakis and Salehi,[2008](https://arxiv.org/html/2605.26339#bib.bib8)\); QAM\-W adapts this geometric view to weight compression\. The codebook training follows the Lloyd\-Max tradition\(Lloyd,[1982](https://arxiv.org/html/2605.26339#bib.bib9); Max,[1960](https://arxiv.org/html/2605.26339#bib.bib12)\)extended to 2D sources, and can be seen as a Hadamard\-rotated generalization of thekk\-means weight clustering in Deep Compression\(Hanet al\.,[2016](https://arxiv.org/html/2605.26339#bib.bib37)\)\.

## 3QAM\-W Codec Pipeline

This section describes the codec as an encoder/decoder pair\. The mathematical properties of the rotation and polar baseline are separated into[appendix˜A](https://arxiv.org/html/2605.26339#A1)\.

### 3\.1Encoder

QAM\-W quantizes a weight matrixW∈ℝdout×dinW\\in\\mathbb\{R\}^\{d\_\{\\mathrm\{out\}\}\\times d\_\{\\mathrm\{in\}\}\}row\-by\-row\. For each rowww, the encoder performs the following steps:

1. 1\.Separate scale from direction\.Record the row normr=‖w‖2r=\\\|w\\\|\_\{2\}as separate metadata and normalize the row tou=w/ru=w/rwhenr\>0r\>0\. Ifr=0r=0, the row is encoded as a zero row\.
2. 2\.Rotate the direction\.Apply a deterministic sign\-masked block\-Hadamard rotation, givingy=Rfwd​uy=R\_\{\\mathrm\{fwd\}\}u\.
3. 3\.Pair coordinates\.Group adjacent rotated coordinates into complex valueszk=y2​k\+i​y2​k\+1z\_\{k\}=y\_\{2k\}\+iy\_\{2k\+1\}fork=0,…,d/2−1k=0,\\ldots,d/2\-1\.
4. 4\.Normalize each pair\.Divide each pair by its calibrated scaleσk\\sigma\_\{k\}\.
5. 5\.Quantize\.Encode each normalized pair either with the polar baseline codebooks or with the joint 2D QAM\-W codebook\.
6. 6\.Pack\.Store the resulting codebook indices in a bit\-packed byte stream\.

### 3\.2Decoder

The decoder reverses the encoder: unpack codebook indices, look up the corresponding centroids, rescale each pair byσk\\sigma\_\{k\}, reassemble pairs into a rotated real vector, apply the inverse rotationRinvR\_\{\\mathrm\{inv\}\}, and multiply by the stored row normrr\. The benchmark charges 16 bits per row for norm metadata; a fully serialized f16 row norm would add a small radial quantization term not part of the directional analysis in[appendix˜A](https://arxiv.org/html/2605.26339#A1)\.

### 3\.3Rotation Choice

The rotation used by the encoder is a block\-Hadamard transform with a deterministic sign mask\. It is cheap to apply with a butterfly algorithm and requires no stored dense matrix\. The implementation chooses the block sizebbas the largest power of two dividing the row dimensiondind\_\{\\text\{in\}\}, capped atbmax=1024b\_\{\\max\}=1024\. For the weight matrices in the experiments in this paper,din=2048d\_\{\\text\{in\}\}=2048usesb=1024b=1024anddin=5632d\_\{\\text\{in\}\}=5632usesb=512b=512\. The rotation’s exact isometry property is proved in[lemma˜1](https://arxiv.org/html/2605.26339#Thmlemma1)\.

### 3\.4Pair Calibration

After rotation, adjacent coordinates are modeled empirically as approximately circular Gaussian pairs with pair\-specific scaleσk\\sigma\_\{k\}\. Under the idealized model that the two coordinates in pairkkare zero\-mean Gaussian variables with equal varianceσk2\\sigma\_\{k\}^\{2\}and zero covariance, the amplitude\|zk\|\|z\_\{k\}\|is Rayleigh distributed and𝔼​\|zk\|=σk​π/2\\mathbb\{E\}\|z\_\{k\}\|=\\sigma\_\{k\}\\sqrt\{\\pi/2\}\. Calibration therefore estimates

σ^k=mean⁡\(\|zk\|\)π/2\\hat\{\\sigma\}\_\{k\}=\\frac\{\\operatorname\{mean\}\(\|z\_\{k\}\|\)\}\{\\sqrt\{\\pi/2\}\}\(1\)from up to 1024 unit\-normalized rows per weight matrix\.

### 3\.5Polar Baseline

The polar baseline quantizes amplitude and phase independently\. The amplitude\|zk\|/σk\|z\_\{k\}\|/\\sigma\_\{k\}is quantized with anNa=2BaN\_\{a\}=2^\{B\_\{a\}\}level Lloyd\-Max codebook for the unit Rayleigh densityf​\(r\)=r​e−r2/2f\(r\)=re^\{\-r^\{2\}/2\}\. The phasearg⁡\(zk\)\\arg\(z\_\{k\}\)is rounded to one ofNp=2BpN\_\{p\}=2^\{B\_\{p\}\}uniform bins\. The total per\-pair budget isB=Ba\+BpB=B\_\{a\}\+B\_\{p\}bits\. The pairwise distortion model for this baseline is analyzed in[theorem˜1](https://arxiv.org/html/2605.26339#Thmtheorem1)\.

### 3\.6Joint 2D QAM

The main QAM\-W variant replaces the two independent polar codebooks with a single 2D codebook𝒞=\{c0,…,c2B−1\}⊂ℝ2\\mathcal\{C\}=\\\{c\_\{0\},\\ldots,c\_\{2^\{B\}\-1\}\\\}\\subset\\mathbb\{R\}^\{2\}\. The codebook is trained by Lloyd iterations on the unit circular Gaussian

f​\(x,y\)=12​π​exp⁡\(−x2\+y22\)\.f\(x,y\)=\\frac\{1\}\{2\\pi\}\\exp\\left\(\-\\frac\{x^\{2\}\+y^\{2\}\}\{2\}\\right\)\.\(2\)Encoding maps each normalized pairzk/σkz\_\{k\}/\\sigma\_\{k\}to its nearest codebook entry by Euclidean distance\. Decoding is a table lookup followed by multiplication byσk\\sigma\_\{k\}\.

The current experiments useB=11B=11bits per pair, matching the5\+65\{\+\}6polar configuration in nominal pair payload\. Native byte alignment and stream layout can differ between polar and joint\-2D implementations; the comparison is therefore equal\-bits\-per\-pair rather than byte\-identical storage\.

The choiced=2d=2is also deliberate at the decode end: reconstruction is a single lookup into a2B2^\{B\}\-entry table that, atB≤11B\\leq 11, occupies at most1616KB and stays L1\-resident\. This is structurally simpler than the multi\-codebook lookups and vector additions of additive methods such as AQLM\(Egiazarianet al\.,[2024](https://arxiv.org/html/2605.26339#bib.bib7)\), or the sequential trellis decoding of QTIP\(Tsenget al\.,[2024b](https://arxiv.org/html/2605.26339#bib.bib27)\)\. Given that rate\-distortion gains saturate byd≈8d\\approx 8\([section˜2](https://arxiv.org/html/2605.26339#S2)\),d=2d=2deliberately trades part of the asymptotic ceiling for a decode path that is one cache\-resident table read\.

### 3\.7Bitrate Accounting

The bitrate per weight includesBBbits per pair for the codebook index and 16 bits per row for row\-norm metadata\. Since each pair covers two scalar coordinates,

bpw=B2\+16din\.\\operatorname\{bpw\}=\\frac\{B\}\{2\}\+\\frac\{16\}\{d\_\{\\mathrm\{in\}\}\}\.\(3\)Fordin=2048d\_\{\\mathrm\{in\}\}=2048andB=11B=11, this is approximately 5\.51 bpw before byte\-alignment overhead\. Reported bitrates include the row\-level packing and alignment used by the implementation\.

### 3\.8The QAM\-W\-3\.5 Low\-Bit Variant

To probe the sub\-4 bpw regime, QAM\-W is instantiated withB=7B=7bits per pair, giving a27=1282^\{7\}=128\-entry 2D Lloyd\-Max codebook trained on the same unit circular Gaussian\. Activation\-aware per\-channel scaling \([appendix˜B](https://arxiv.org/html/2605.26339#A2)\) remains applicable\. Fordin=2048d\_\{\\mathrm\{in\}\}=2048the per\-pair budget plus row\-norm metadata givesbpw≈3\.51\\operatorname\{bpw\}\\approx 3\.51; the AWQ\-aware variant adds∼0\.003\\sim 0\.003bpw of per\-channel scale metadata forα=0\.3\\alpha=0\.3\. This is the configuration analyzed in[section˜5\.3\.1](https://arxiv.org/html/2605.26339#S5.SS3.SSS1)\.

## 4Analysis

QAM\-W’s design rests on four analytical results\. We state them here and defer the formal statements, proofs, and supporting empirical checks to[appendices˜A](https://arxiv.org/html/2605.26339#A1),[B](https://arxiv.org/html/2605.26339#A2)and[C](https://arxiv.org/html/2605.26339#A3)\.

##### The rotation is distortion\-free\.

The sign\-masked block\-Hadamard rotation is an exact isometry in real arithmetic \([lemma˜1](https://arxiv.org/html/2605.26339#Thmlemma1)\): it adds no reconstruction error and only redistributes energy within each block before quantization\. It does not by itself make the rotated coordinates Gaussian; that the post\-rotation complex pairs concentrate near a circular Gaussian is an empirical property, checked by QQ plots against fitted Rayleigh and Gaussian marginals \([section˜A\.4](https://arxiv.org/html/2605.26339#A1.SS4)\)\.

##### Joint 2D coding dominates polar at fixed rate\.

For the polar baseline, the per\-pair distortion splits into an amplitude term and a phase term with a closed form under the Rayleigh–Lloyd source model \([theorems˜1](https://arxiv.org/html/2605.26339#Thmtheorem1)and[1](https://arxiv.org/html/2605.26339#Thmcorollary1)\)\. Because that factorization codes amplitude and phase independently, it cannot capture the pair\-level correlation that a single 2D codebook exploits\.

##### Activation\-weighted error is the right objective\.

The layer\-output error obeys the trace identity‖X​Δ​W⊤‖F2=Tr⁡\(Δ​W​M^x​Δ​W⊤\)\\\|X\\Delta W^\{\\top\}\\\|\_\{F\}^\{2\}=\\operatorname\{Tr\}\(\\Delta W\\hat\{M\}\_\{x\}\\Delta W^\{\\top\}\)\([proposition˜B\.1](https://arxiv.org/html/2605.26339#A2.Thmproposition1)\), so what a quantized layer actually contributes is*activation\-weighted*error, not raw weight MSE\. This motivates AWQ\-style per\-channel scaling, which attenuates the contribution of high\-RMS input channels under a diagonal\-MxM\_\{x\}model \([corollaries˜3](https://arxiv.org/html/2605.26339#Thmcorollary3)and[B\.2](https://arxiv.org/html/2605.26339#A2.Thmproposition2)\)\. Empirically, the amplification of weight\-domain Frobenius error into layer\-output RMSE varies by0\.550\.55–5\.5×5\.5\\timesacross layers and methods \([table˜4](https://arxiv.org/html/2605.26339#A2.T4)\)\.

##### A monotone codec\-to\-KL bound\.

Chaining the rotation isometry, the pairwise identity, the layer\-output trace identity, and a softmax\-Fisher local KL expansion \([corollary˜4](https://arxiv.org/html/2605.26339#Thmcorollary4)\) yields an upper bound on the paired KL between the BF16 and quantized models that is monotone in two scalars: the per\-pair Lloyd distortionDBD\_\{B\}\(codec\) and a model\-fixed constantCWC\_\{W\}\([theorem˜2](https://arxiv.org/html/2605.26339#Thmtheorem2)\)\. The bound motivates, but does not prove, that realized KL should rank methods the same wayDBD\_\{B\}does; across the3737\(method, model\) rows in this study, paired KL andΔ\\DeltaPPL% are rank\-correlated at Spearmanρ=0\.99\\rho=0\.99\([fig\.˜1](https://arxiv.org/html/2605.26339#S5.F1)\)\.

## 5Experiments

Eight quantized configurations are evaluated against a BF16 reference on three instruction\-tuned LLMs: TinyLlama\-1\.1B\-Chat\(Zhanget al\.,[2024](https://arxiv.org/html/2605.26339#bib.bib23)\), Qwen2\.5\-3B\-Instruct\(Yanget al\.,[2024](https://arxiv.org/html/2605.26339#bib.bib26)\), and Mistral\-7B\-Instruct\-v0\.3\(Jianget al\.,[2023](https://arxiv.org/html/2605.26339#bib.bib25)\)\. These are dense decoder\-only transformers with standard attention, chosen so the comparison rests on mature, audited loading and evaluation tooling \(see Limitations\)\. Only MLP gate, up, and down projections are quantized; attention weights and the LM head remain BF16\.

##### Baselines\.

Five external methods: RTN\(Krishnamoorthi,[2018](https://arxiv.org/html/2605.26339#bib.bib13)\), GPTQ\(Frantaret al\.,[2023](https://arxiv.org/html/2605.26339#bib.bib3)\), AWQ\(Linet al\.,[2024](https://arxiv.org/html/2605.26339#bib.bib2)\), and AutoRound\(Chenget al\.,[2023](https://arxiv.org/html/2605.26339#bib.bib14)\), all at W4A16 g128 \(≈4\.1\\approx 4\.1bpw\); SmoothQuant\(Xiaoet al\.,[2023](https://arxiv.org/html/2605.26339#bib.bib1)\)W8A8 \(≈8\.1\\approx 8\.1bpw\)\.*SmoothQuant eval mode:*weights are smoothed and fake\-quantized to int8 then restored to BF16; activations stay BF16\. All comparisons against SmoothQuant in this paper are on weight storage and memory bandwidth, not matmul arithmetic\.

##### QAM\-W variants\.

QAM\-W\-polar \(polar baseline,≈4\.0\\approx 4\.0bpw\), QAM\-W\-4 \(joint 2D,B=8B\\\!=\\\!8,≈4\.0\\approx 4\.0bpw\), and QAM\-W\-5\.5 \(joint 2D,B=11B\\\!=\\\!11, AWQ\-style scalingα=0\.3\\alpha\\\!=\\\!0\.3,≈5\.5\\approx 5\.5bpw\)\. A sub\-4 bpw variant QAM\-W\-3\.5 \(B=7B\\\!=\\\!7,≈3\.5\\approx 3\.5bpw\) is reported in[section˜5\.3\.1](https://arxiv.org/html/2605.26339#S5.SS3.SSS1)\.

##### Metrics\.

WikiText\-2\(Merityet al\.,[2017](https://arxiv.org/html/2605.26339#bib.bib15)\)stride perplexity \(seq\_len 2048, stride 1024, 16K scored tokens\); paired KL divergence against BF16 \(4K scored tokens\); six\-task lm\-evaluation\-harness\(Gaoet al\.,[2024](https://arxiv.org/html/2605.26339#bib.bib22)\)panel \(PIQA\(Bisket al\.,[2020](https://arxiv.org/html/2605.26339#bib.bib16)\), HellaSwag\(Zellerset al\.,[2019](https://arxiv.org/html/2605.26339#bib.bib17)\), COPA\(Roemmeleet al\.,[2011](https://arxiv.org/html/2605.26339#bib.bib18)\), RTE\(Wanget al\.,[2018](https://arxiv.org/html/2605.26339#bib.bib19)\), OpenBookQA\(Mihaylovet al\.,[2018](https://arxiv.org/html/2605.26339#bib.bib20)\), LAMBADA\-OpenAI\(Papernoet al\.,[2016](https://arxiv.org/html/2605.26339#bib.bib21)\)\)\. Full reproducibility details including model SHAs, calibration setup, and hardware are in[appendix˜J](https://arxiv.org/html/2605.26339#A10)\.

### 5\.1Cross\-Model Quality Results

![Refer to caption](https://arxiv.org/html/2605.26339v1/plots/day8/kl_vs_dppl.png)Figure 1:Empirical check of[theorem˜2](https://arxiv.org/html/2605.26339#Thmtheorem2): measured paired KL \(logxx\) vs\. measuredΔ\\DeltaPPL% \(symlogyy\) for every \(method, model\) row\. Only the headline points and the two large outlier points are labelled to avoid clutter\. The pooled rank correlation across all 37 \(method, model\) rows spanning six model variants is Spearmanρ=0\.99\\rho=0\.99\.[Table˜1](https://arxiv.org/html/2605.26339#S5.T1)summarizes the cross\-model quality story\.

Table 1:Cross\-architecture quantization sensitivity\. Each column reportsΔ\\DeltaPPL% versus that model’s BF16 reference \(anchor PPL in the column header\); rows are sorted by ascending meanΔ\\Deltaacross models\. QAM\-W\-5\.5 is the lowest\-bpw method in the BF16\-quality envelope: the only sub\-8 bpw method that stays within±0\.4%\\pm 0\.4\\%of BF16 on every model\. Detailed per\-model breakdowns with paired KL and harness average are in[appendix˜H](https://arxiv.org/html/2605.26339#A8)\.
Reader key:↓\\downarrowlower is better\. Shaded rows \(⋆\\star\) mark QAM\-W \(ours\)\.##### QAM\-W\-5\.5 reaches the BF16 envelope at the lowest bitrate\.

At≈5\.5\\approx 5\.5bpw, QAM\-W\-5\.5 stays within±0\.4%\\pm 0\.4\\%of BF16 perplexity on all three models\. SmoothQuant W8A8 reaches the same envelope but at≈8\.1\\approx 8\.1bpw — about47%47\\%more weight bits\.

Calibration narrows the cross\-model spread\.Uncalibrated RTN costs \+1\.9% on Mistral 7B but \+24\.6% on Qwen 2\.5 3B — a 12×\\timesspread driven by architecture alone\. Among calibrated methods \(GPTQ, AWQ, AutoRound, QAM\-W\-5\.5\) the cross\-model spread is much tighter\. Single\-model comparisons among uncalibrated quantizers can therefore mislead in a way that calibrated comparisons typically do not\.

##### Joint 2D coding beats polar at equal bitrate\.

At≈4\\approx 4bpw, QAM\-W\-4 beats QAM\-W\-polar by 1–3 pp ofΔ\\DeltaPPL on the quantization\-tolerant models \(TinyLlama, Mistral\) and by 14\.6 pp on the quantization\-sensitive Qwen 2\.5\. The joint codebook captures pair\-level correlations that the polar factorization discards; the gain is largest precisely where the model is most sensitive to codec choice\.

#### 5\.1\.1Cross\-Model Slope

![Refer to caption](https://arxiv.org/html/2605.26339v1/plots/day8/slope_dppl.png)Figure 2:Cross\-modelΔ\\DeltaPPL% slope\. One line per method; the shaded band at the bottom is the<1%<\\\!1\\%BF16\-envelope zone\. Two methods are drawn thicker as the headline: QAM\-W\-5\.5 \(⋆\\star,≈5\.5\\approx 5\.5bpw\) sits inside the envelope on all three architectures; SmoothQuant \(♢\\diamondsuit,≈8\.1\\approx 8\.1bpw\) reaches the same envelope but at∼47%\\sim 47\\%higher bitrate\.Across the3737\(method, model\) rows assembled in this study, paired KL andΔ\\DeltaPPL% are rank\-correlated at Spearmanρ=0\.99\\rho=0\.99\([fig\.˜1](https://arxiv.org/html/2605.26339#S5.F1)\), consistent with the composite bound in[section˜4](https://arxiv.org/html/2605.26339#S4)\. Multi\-seed stride\-offset rescoring confirms the ranking is robust \(std0\.030\.03–0\.080\.08pp;[appendix˜G](https://arxiv.org/html/2605.26339#A7)\)\.

### 5\.2Iso\-bpw Frontier Comparison

To position QAM\-W against the rotated\-codebook frontier, we run an iso\-bpw comparison on Llama\-2\-7B\-base\(Touvronet al\.,[2023](https://arxiv.org/html/2605.26339#bib.bib24)\)against QTIP\(Tsenget al\.,[2024b](https://arxiv.org/html/2605.26339#bib.bib27)\)\(4\-bit and 2\-bit\) and AQLM\(Egiazarianet al\.,[2024](https://arxiv.org/html/2605.26339#bib.bib7)\)\(2\-bit\), using published HuggingFace checkpoints dequantized to BF16 before scoring under our protocol\.

Table 2:Frontier comparison on Llama\-2\-7B\-base, sorted by ascending bpw\. SmoothQuant W8A8 bpw is over MLP weights only; QTIP, AQLM, and the QAM\-W rows quantize the full weight pool per their published configurations\.
Reader key:↓\\downarrowlower is better,↑\\uparrowhigher is better\. Shaded rows \(⋆\\star\) are QAM\-W \(ours\)\.##### QAM\-W\-5\.5 transfers; QTIP leads at 4 bpw\.

QAM\-W\-5\.5 on Llama\-2\-7B\-base reaches\+0\.29%\+0\.29\\%Δ\\DeltaPPL, within the same envelope as the cross\-model rows\. At iso\-4\-bpw, QTIP\-4Bit \(\+1\.32%\+1\.32\\%, KL0\.0100\.010\) beats QAM\-W\-4 \(\+2\.15%\+2\.15\\%, KL0\.0290\.029\): QTIP’sE8E\_\{8\}lattice trellis and BlockLDLQ rounding outpace QAM\-W’sd=2d\\\!=\\\!2Lloyd codebook at a strict 4\-bit ceiling\. QAM\-W\-5\.5 improves over QTIP\-4Bit only at∼1\.4×\\sim\\\!1\.4\\timesthe bit budget — a different operating point, not a Pareto win\.

##### Low\-bpw frontier\.

At≈2\\approx 2bpw, QTIP and AQLM take comparable PPL hits \(\+17\+17–19%19\\%\)\. QAM\-W has no 2\-bpw operating point; its closest variant is QAM\-W\-3\.5 at≈3\.5\\approx 3\.5bpw \([section˜5\.3\.1](https://arxiv.org/html/2605.26339#S5.SS3.SSS1)\)\.

### 5\.3Operating Points and Downstream Behavior

#### 5\.3\.1Low\-Bit Operating Point: QAM\-W\-3\.5

AB=7B\\\!=\\\!7codebook \(128128codewords\) drops QAM\-W to≈3\.5\\approx 3\.5bpw\. On TinyLlama, activation\-aware scaling recovers≈0\.5\\approx 0\.5bits of effective precision: QAM\-W\-3\.5 \(\+4\.1%\+4\.1\\%Δ\\DeltaPPL, KL0\.0370\.037\) essentially ties the unscaled QAM\-W\-4 at4\.04\.0bpw \(\+4\.2%\+4\.2\\%, KL0\.0340\.034\)\. Without scaling the 7\-bit codec costs\+7\.6%\+7\.6\\%\. On Qwen 2\.5 the 128\-entry codec is not yet Pareto\-competitive \(\+7\.5%\+7\.5\\%, worse than GPTQ\(Frantaret al\.,[2023](https://arxiv.org/html/2605.26339#bib.bib3)\)at\+2\.9%\+2\.9\\%\), indicating∼4\\sim\\\!4bpw as the lower bound at which the 2D codec geometry pays off on sensitive architectures\. On Mistral 7B, 3\.5 bpw costs only\+1\.98%\+1\.98\\%\. Full results in[appendix˜E](https://arxiv.org/html/2605.26339#A5)\.

#### 5\.3\.2Downstream Task Behavior

A six\-task lm\-evaluation\-harness\(Gaoet al\.,[2024](https://arxiv.org/html/2605.26339#bib.bib22)\)panel corroborates the PPL story\. LAMBADA is the most quantization\-sensitive task; PIQA\(Bisket al\.,[2020](https://arxiv.org/html/2605.26339#bib.bib16)\)and COPA\(Roemmeleet al\.,[2011](https://arxiv.org/html/2605.26339#bib.bib18)\)are nearly flat across methods within each model; per\-task 95% CI is±2\\pm 2–33pp at the evaluation limits used, so within\-model cross\-method differences inside that interval should be read as ties\. Per\-task breakdowns and Pareto plots are in[appendix˜H](https://arxiv.org/html/2605.26339#A8)\.

##### MMLU\.

QAM\-W\-5\.5 and BF16 scored on the full 57\-subject MMLU\(Hendryckset al\.,[2021](https://arxiv.org/html/2605.26339#bib.bib28)\)panel match within0\.20\.2pp on all three models \([table˜3](https://arxiv.org/html/2605.26339#S5.T3)\), confirming the codec’s quality envelope extends beyond the perplexity metric\.

Table 3:MMLU macro\-average accuracy \(57 subjects,limit=300\)\. QAM\-W\-5\.5 matches BF16 within0\.20\.2pp on all three models\.Reader key:↑\\uparrowhigher is better\.

## 6Conclusion

The activation\-aware joint\-2D variant of QAM\-W reaches the BF16 quality envelope at≈5\.5\\approx\\\!5\.5bpw on all five models tested, withΔ\\DeltaPPL≤0\.4%\\leq 0\.4\\%on each\. Three assumptions underpin the codec: \(i\) weight\-row coordinates are correlated and admit a homogenizing orthogonal rotation; \(ii\) the rotated pairs concentrate near a circular Gaussian suitable for joint 2D Lloyd coding; \(iii\) per\-channel activation magnitudes provide a rescaling that aligns codec error with the low\-eigenvalue directions ofMxM\_\{x\}\. When any of these weakens — as on Qwen 2\.5 3B — all three are needed to recover quality\.

In practice, QAM\-W\-5\.5 \(≈5\.5\\approx\\\!5\.5bpw\) is the quality\-preserving choice, matching SmoothQuant\(Xiaoet al\.,[2023](https://arxiv.org/html/2605.26339#bib.bib1)\)W8A8 at≈32%\\approx\\\!32\\%fewer weight bits\. QAM\-W\-4 \(≈4\\approx\\\!4bpw\) serves as the iso\-bitrate comparison against W4A16 baselines and directly demonstrates that joint 2D coding outperforms polar by 1–3 pp on quantization\-tolerant models and by over 14 pp on the most quantization\-sensitive one\. QAM\-W\-3\.5 \(≈3\.5\\approx\\\!3\.5bpw\) is viable on quantization\-tolerant architectures but not yet Pareto\-competitive on sensitive ones\.

More broadly, among uncalibrated quantizers, sensitivity is dominated by the model architecture: RTN costs \+1\.9% on Mistral but \+24\.6% on Qwen, a 12×\\timesdifference from changing the model alone\. Among calibrated methods, the spread is much smaller\. Single\-model comparisons can therefore mislead\. Natural follow\-ups include a learned 2D codebook fit to each checkpoint’s empirical pair distribution and a fused dequantization\-matmul kernel to translate the bitrate saving into inference latency gains\.

## Limitations

##### Scope of quantized parameters\.

The cross\-model headline tables \([section˜5\.1](https://arxiv.org/html/2605.26339#S5.SS1)\) report bpw over MLP gate/up/down projections only; attention weights and the LM head remain BF16\. The MLP \+ attention extension in[section˜F\.1](https://arxiv.org/html/2605.26339#A6.SS1)adds the four self\-attention projections \(qq,kk,vv,oo\) for the BF16 reference and the four strongest sub\-8 bpw quantized methods on Mistral 7B and shows the ranking is preserved, but the cross\-model tables themselves do not yet include an MLP \+ attention row for TinyLlama and Qwen; the LM head remains untouched in every reported configuration\.

##### Codebook training cost\.

Lloyd\-Max codebook training for the joint 2D codec is a one\-time cost of∼5\\sim 5s per bitrate level\. This is negligible compared to inference, but it is non\-zero and is performed on a synthetic unit circular Gaussian rather than per\-checkpoint — a learned codebook conditioned on the empirical post\-rotation pair distribution might buy additional precision\.

##### Activation\-awareα\\alphais grid\-tuned, not learned\.

The AWQ\-style per\-channel scaling exponentα=0\.3\\alpha=0\.3was chosen from a small grid\{0\.0,0\.3,0\.5,0\.8\}\\\{0\.0,0\.3,0\.5,0\.8\\\}on TinyLlama and held fixed across models\. A per\-model or per\-layerα\\alphaschedule would likely improve quality on Qwen 2\.5, where the 7\-bit codec is mostα\\alpha\-sensitive\.

##### Architecture coverage\.

The cross\-model tables rest on three instruction\-tuned models from three families \(TinyLlama, Qwen 2\.5, Mistral\) plus two Llama\-2 base models at 7B and 13B \([sections˜5\.2](https://arxiv.org/html/2605.26339#S5.SS2)and[F\.2](https://arxiv.org/html/2605.26339#A6.SS2)\) — five models from four families across1\.11\.1B–1313B parameters\. Beyond this range,7070B and contemporary families \(Llama 3, Gemma 2, DeepSeek\) would further tighten the architecture\-sensitivity claim, particularly to test whether the≈5\.5\\approx\\\!5\.5bpw envelope continues to hold at the7070B parameter scale where activation statistics diverge further from the smaller\-model distributions calibrated against here\. The choice to evaluate on these architecturally conventional models is deliberate: all five are dense decoder\-only transformers with standard multi\-head or grouped\-query attention, for which mature weight\-loading and evaluation libraries make the codec comparison free of model\-loading confounds\. Contemporary \(2026\-era\) architectures are not yet reliably supported end\-to\-end by stock libraries — e\.g\. Gemma\-4 combines per\-layer input embeddings with interleaved local/global sliding\-window attention and must be loaded through a forked model implementation\. Extending QAM\-W to such architectures, via a dedicated inference engine, is left to future work\.

##### Iso\-4\-bpw quality trails the QTIP frontier\.

[Section˜5\.2](https://arxiv.org/html/2605.26339#S5.SS2)reports QTIP\-4Bit ahead of QAM\-W’s joint\-2D codec at iso\-4\-bpw on Llama\-2\-7B\-base; the joint\-2D Lloyd codebook trades some rate\-distortion ceiling for training cheapness and decode simplicity \(a single2B2^\{B\}\-entry table lookup rather than a lattice nearest\-point search\)\. Closing the iso\-4\-bpw gap is the most natural extension: candidate paths are a richer codebook training distribution per checkpoint \([section˜6](https://arxiv.org/html/2605.26339#S6)\) and tighter Lloyd optimization with QTIP\-style BlockLDLQ rounding\.

##### Harness precision\.

The six\-task harness panel useslimit=300\(Qwen, Mistral\) andlimit=500\(TinyLlama\)\. The resulting per\-task accuracy noise floor is on the order of one to two percentage points; cross\-method differences smaller than that should be read as ties\.

##### No fused dequantization\-matmul kernel\.

The reported bpw is a storage metric\. The current pipeline dequantizes weights to BF16 before the linear\-layer matmul, so the bitrate saving shows up in weight storage and host\-to\-device memory bandwidth, not in matmul arithmetic\. The bpw numbers should be read as memory\-footprint claims until a fused kernel is available\. Building this kernel — and with it support for architectures beyond the conventional dense transformers studied here — is the primary engineering follow\-up\.

##### Comparison scope\.

The evaluation covers five models from four families and seven baselines under one unified protocol\. Notably absent are iso\-bpw comparisons against learned\-rotation methods \(QuaRot, SpinQuant\) and a directd=1d\\\!=\\\!1vs\.d=2d\\\!=\\\!2ablation against PolarQuant\(Vicentino,[2026](https://arxiv.org/html/2605.26339#bib.bib38)\), which would isolate the codebook\-dimension contribution from the shared rotation\-plus\-scaling pipeline\. Scale beyond 13B and broader downstream benchmarks \(MMLU\-Pro, BBH, IFEval, GSM8K\) are also not covered\.

## References

- S\. Ashkboos, A\. Mohtashami, M\. L\. Croci, B\. Li, P\. Cameron, M\. Jaggi, D\. Alistarh, T\. Hoefler, and J\. Hensman \(2024\)QuaRot: outlier\-free 4\-bit inference in rotated LLMs\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§2](https://arxiv.org/html/2605.26339#S2.SS0.SSS0.Px2.p1.1)\.
- Half\-quadratic quantization of large machine learning models\.Note:[https://mobiusml\.github\.io/hqq\_blog/](https://mobiusml.github.io/hqq_blog/)Cited by:[§2](https://arxiv.org/html/2605.26339#S2.SS0.SSS0.Px1.p1.1)\.
- Y\. Bisk, R\. Zellers, R\. L\. Bras, J\. Gao, and Y\. Choi \(2020\)PIQA: reasoning about physical commonsense in natural language\.InAAAI Conference on Artificial Intelligence,Cited by:[§5](https://arxiv.org/html/2605.26339#S5.SS0.SSS0.Px3.p1.1),[§5\.3\.2](https://arxiv.org/html/2605.26339#S5.SS3.SSS2.p1.2)\.
- J\. Chee, Y\. Cai, V\. Kuleshov, and C\. De Sa \(2023\)QuIP: 2\-bit quantization of large language models with guarantees\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§1](https://arxiv.org/html/2605.26339#S1.p3.1),[§2](https://arxiv.org/html/2605.26339#S2.SS0.SSS0.Px2.p1.1)\.
- W\. Cheng, W\. Zhang, H\. Shen, Y\. Cai, X\. He, and K\. Lv \(2023\)Optimize weight rounding via signed gradient descent for the quantization of llms\.arXiv preprint arXiv:2309\.05516\.Cited by:[§2](https://arxiv.org/html/2605.26339#S2.SS0.SSS0.Px1.p1.1),[§5](https://arxiv.org/html/2605.26339#S5.SS0.SSS0.Px1.p1.2)\.
- T\. Dettmers, M\. Lewis, Y\. Belkada, and L\. Zettlemoyer \(2022\)LLM\.int8\(\): 8\-bit matrix multiplication for transformers at scale\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§2](https://arxiv.org/html/2605.26339#S2.SS0.SSS0.Px1.p1.1)\.
- T\. Dettmers, A\. Pagnoni, A\. Holtzman, and L\. Zettlemoyer \(2023\)QLoRA: efficient finetuning of quantized llms\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§2](https://arxiv.org/html/2605.26339#S2.SS0.SSS0.Px1.p1.1)\.
- V\. Egiazarian, A\. Panferov, D\. Kuznedelev, E\. Frantar, A\. Babenko, and D\. Alistarh \(2024\)Extreme compression of large language models via additive quantization\.InProceedings of the 41st International Conference on Machine Learning \(ICML\),Cited by:[§2](https://arxiv.org/html/2605.26339#S2.SS0.SSS0.Px2.p2.5),[§3\.6](https://arxiv.org/html/2605.26339#S3.SS6.p3.6),[§5\.2](https://arxiv.org/html/2605.26339#S5.SS2.p1.1)\.
- E\. Frantar, S\. Ashkboos, T\. Hoefler, and D\. Alistarh \(2023\)GPTQ: accurate post\-training quantization for generative pre\-trained transformers\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§2](https://arxiv.org/html/2605.26339#S2.SS0.SSS0.Px1.p1.1),[§5](https://arxiv.org/html/2605.26339#S5.SS0.SSS0.Px1.p1.2),[§5\.3\.1](https://arxiv.org/html/2605.26339#S5.SS3.SSS1.p1.15)\.
- L\. Gao, J\. Tow, B\. Abbasi, S\. Biderman, S\. Black, A\. DiPofi, C\. Foster, L\. Golding, J\. Hsu, A\. Le Noac’h, H\. Li, K\. McDonell, N\. Muennighoff, C\. Ociepa, J\. Phang, L\. Reynolds, H\. Schoelkopf, A\. Skowron, L\. Sutawika, E\. Tang, A\. Thite, B\. Wang, K\. Wang, and A\. Zou \(2024\)A framework for few\-shot language model evaluation \(lm\-evaluation\-harness\)\.Note:Zenodo, version 0\.4\.xExternal Links:[Document](https://dx.doi.org/10.5281/zenodo.10256836)Cited by:[§5](https://arxiv.org/html/2605.26339#S5.SS0.SSS0.Px3.p1.1),[§5\.3\.2](https://arxiv.org/html/2605.26339#S5.SS3.SSS2.p1.2)\.
- A\. Gersho and R\. M\. Gray \(1991\)Vector quantization and signal compression\.Kluwer Academic Publishers\.Cited by:[item \(C1\)](https://arxiv.org/html/2605.26339#A3.I1.ix1.p1.10),[§C\.2](https://arxiv.org/html/2605.26339#A3.SS2.p2.1)\.
- S\. Han, H\. Mao, and W\. J\. Dally \(2016\)Deep compression: compressing deep neural networks with pruning, trained quantization and Huffman coding\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§2](https://arxiv.org/html/2605.26339#S2.SS0.SSS0.Px4.p1.1)\.
- D\. Hendrycks, C\. Burns, S\. Basart, A\. Zou, M\. Mazeika, D\. Song, and J\. Steinhardt \(2021\)Measuring massive multitask language understanding\.InICLR,Cited by:[§5\.3\.2](https://arxiv.org/html/2605.26339#S5.SS3.SSS2.Px1.p1.1)\.
- A\. Q\. Jiang, A\. Sablayrolles, A\. Mensch, C\. Bamford, D\. S\. Chaplot, D\. de las Casas, F\. Bressand, G\. Lengyel, G\. Lample, L\. Saulnier, L\. Renard Lavaud, M\. Lachaux, P\. Stock, T\. Le Scao, T\. Lavril, T\. Wang, T\. Lacroix, and W\. El Sayed \(2023\)Mistral 7B\.arXiv preprint arXiv:2310\.06825\.Cited by:[§5](https://arxiv.org/html/2605.26339#S5.p1.1)\.
- S\. Kim, C\. Hooper, A\. Gholami, Z\. Dong, X\. Li, S\. Shen, M\. W\. Mahoney, and K\. Keutzer \(2024\)SqueezeLLM: dense\-and\-sparse quantization\.InProceedings of the 41st International Conference on Machine Learning \(ICML\),Cited by:[§2](https://arxiv.org/html/2605.26339#S2.SS0.SSS0.Px1.p1.1)\.
- R\. Krishnamoorthi \(2018\)Quantizing deep convolutional networks for efficient inference: a whitepaper\.External Links:1806\.08342,[Link](https://arxiv.org/abs/1806.08342)Cited by:[§2](https://arxiv.org/html/2605.26339#S2.SS0.SSS0.Px1.p1.1),[§5](https://arxiv.org/html/2605.26339#S5.SS0.SSS0.Px1.p1.2)\.
- J\. Lin, J\. Tang, H\. Tang, S\. Yang, W\. Chen, W\. Wang, G\. Xiao, X\. Dang, C\. Gan, and S\. Han \(2024\)AWQ: activation\-aware weight quantization for LLM compression and acceleration\.InProceedings of Machine Learning and Systems \(MLSys\),Note:Best Paper AwardCited by:[§1](https://arxiv.org/html/2605.26339#S1.p3.1),[§2](https://arxiv.org/html/2605.26339#S2.SS0.SSS0.Px1.p1.1),[§5](https://arxiv.org/html/2605.26339#S5.SS0.SSS0.Px1.p1.2)\.
- Y\. Liu, J\. Wen, Y\. Wang, S\. Ye, L\. L\. Zhang, T\. Cao, C\. Li, and M\. Yang \(2024\)VPTQ: extreme low\-bit vector post\-training quantization for large language models\.InEmpirical Methods in Natural Language Processing \(EMNLP\),Cited by:[§2](https://arxiv.org/html/2605.26339#S2.SS0.SSS0.Px2.p2.5)\.
- Z\. Liu, C\. Zhao, I\. Fedorov, B\. Soran, D\. Choudhary, R\. Krishnamoorthi, V\. Chandra, Y\. Tian, and T\. Blankevoort \(2025\)SpinQuant: LLM quantization with learned rotations\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§2](https://arxiv.org/html/2605.26339#S2.SS0.SSS0.Px2.p1.1)\.
- S\. P\. Lloyd \(1982\)Least squares quantization in pcm\.IEEE Transactions on Information Theory28\(2\),pp\. 129–137\.Note:Originally published as Bell Labs Technical Memorandum, 1957Cited by:[§C\.2](https://arxiv.org/html/2605.26339#A3.SS2.p2.1),[§1](https://arxiv.org/html/2605.26339#S1.p3.1),[§2](https://arxiv.org/html/2605.26339#S2.SS0.SSS0.Px4.p1.1)\.
- J\. Max \(1960\)Quantizing for minimum distortion\.IRE Transactions on Information Theory6\(1\),pp\. 7–12\.Cited by:[§C\.2](https://arxiv.org/html/2605.26339#A3.SS2.p2.1),[§2](https://arxiv.org/html/2605.26339#S2.SS0.SSS0.Px4.p1.1)\.
- S\. Merity, C\. Xiong, J\. Bradbury, and R\. Socher \(2017\)Pointer sentinel mixture models\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§5](https://arxiv.org/html/2605.26339#S5.SS0.SSS0.Px3.p1.1)\.
- T\. Mihaylov, P\. Clark, T\. Khot, and A\. Sabharwal \(2018\)Can a suit of armor conduct electricity? a new dataset for open book question answering\.InEmpirical Methods in Natural Language Processing \(EMNLP\),Cited by:[§5](https://arxiv.org/html/2605.26339#S5.SS0.SSS0.Px3.p1.1)\.
- D\. Paperno, G\. Kruszewski, A\. Lazaridou, N\. Q\. Pham, R\. Bernardi, S\. Pezzelle, M\. Baroni, G\. Boleda, and R\. Fernández \(2016\)The LAMBADA dataset: word prediction requiring a broad discourse context\.InAnnual Meeting of the Association for Computational Linguistics \(ACL\),Cited by:[§5](https://arxiv.org/html/2605.26339#S5.SS0.SSS0.Px3.p1.1)\.
- J\. G\. Proakis and M\. Salehi \(2008\)Digital communications\.McGraw\-Hill\.Cited by:[§2](https://arxiv.org/html/2605.26339#S2.SS0.SSS0.Px4.p1.1)\.
- M\. Roemmele, C\. A\. Bejan, and A\. S\. Gordon \(2011\)Choice of plausible alternatives: an evaluation of commonsense causal reasoning\.InAAAI Spring Symposium,Cited by:[§5](https://arxiv.org/html/2605.26339#S5.SS0.SSS0.Px3.p1.1),[§5\.3\.2](https://arxiv.org/html/2605.26339#S5.SS3.SSS2.p1.2)\.
- W\. Shao, M\. Chen, Z\. Zhang, P\. Xu, L\. Zhao, Z\. Li, K\. Zhang, P\. Gao, Y\. Qiao, and P\. Luo \(2024\)OmniQuant: omnidirectionally calibrated quantization for large language models\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§2](https://arxiv.org/html/2605.26339#S2.SS0.SSS0.Px1.p1.1)\.
- H\. Touvron, L\. Martin, K\. Stone, P\. Albert, A\. Almahairi, Y\. Babaei, N\. Bashlykov, S\. Batra, P\. Bhargava, S\. Bhosale, D\. Bikel, L\. Blecher, C\. C\. Ferrer, M\. Chen, G\. Cucurull, D\. Esiobu, J\. Fernandes, J\. Fu, W\. Fu, B\. Fuller, C\. Gao, V\. Goswami, N\. Goyal, A\. Hartshorn, S\. Hosseini, R\. Hou, H\. Inan, M\. Kardas, V\. Kerkez, M\. Khabsa, I\. Kloumann, A\. Korenev, P\. S\. Koura, M\. Lachaux, T\. Lavril, J\. Lee, D\. Liskovich, Y\. Lu, Y\. Mao, X\. Martinet, T\. Mihaylov, P\. Mishra, I\. Molybog, Y\. Nie, A\. Poulton, J\. Reizenstein, R\. Rungta, K\. Saladi, A\. Schelten, R\. Silva, E\. M\. Smith, R\. Subramanian, X\. E\. Tan, B\. Tang, R\. Taylor, A\. Williams, J\. X\. Kuan, P\. Xu, Z\. Yan, I\. Zarov, Y\. Zhang, A\. Fan, M\. Kambadur, S\. Narang, A\. Rodriguez, R\. Stojnic, S\. Edunov, and T\. Scialom \(2023\)Llama 2: open foundation and fine\-tuned chat models\.arXiv preprint arXiv:2307\.09288\.Cited by:[§5\.2](https://arxiv.org/html/2605.26339#S5.SS2.p1.1)\.
- A\. Tseng, J\. Chee, Q\. Sun, V\. Kuleshov, and C\. De Sa \(2024a\)QuIP\#: even better LLM quantization with Hadamard incoherence and lattice codebooks\.InProceedings of the 41st International Conference on Machine Learning \(ICML\),Cited by:[§1](https://arxiv.org/html/2605.26339#S1.p3.1),[§2](https://arxiv.org/html/2605.26339#S2.SS0.SSS0.Px2.p1.1)\.
- A\. Tseng, Q\. Sun, D\. Hou, and C\. De Sa \(2024b\)QTIP: quantization with trellises and incoherence processing\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Note:SpotlightCited by:[§1](https://arxiv.org/html/2605.26339#S1.p2.7),[§2](https://arxiv.org/html/2605.26339#S2.SS0.SSS0.Px2.p2.5),[§3\.6](https://arxiv.org/html/2605.26339#S3.SS6.p3.6),[§5\.2](https://arxiv.org/html/2605.26339#S5.SS2.p1.1)\.
- M\. van Baalen, A\. Kuzmin, M\. Nagel, P\. Couperus, C\. Bastoul, E\. Mahurin, T\. Blankevoort, and P\. Whatmough \(2024\)GPTVQ: the blessing of dimensionality for LLM quantization\.InICML Workshop on Efficient Systems for Foundation Models \(ES\-FoMo\),Cited by:[§2](https://arxiv.org/html/2605.26339#S2.SS0.SSS0.Px2.p2.5)\.
- C\. Vicentino \(2026\)PolarQuant: optimal Gaussian weight quantization via Hadamard rotation for LLM compression\.Note:arXiv preprint arXiv:2603\.29078 \(withdrawn; later rebranded as HLWQ\)Cited by:[§2](https://arxiv.org/html/2605.26339#S2.SS0.SSS0.Px3.p1.3),[Comparison scope\.](https://arxiv.org/html/2605.26339#Sx1.SS0.SSS0.Px8.p1.2)\.
- A\. Wang, A\. Singh, J\. Michael, F\. Hill, O\. Levy, and S\. R\. Bowman \(2018\)GLUE: a multi\-task benchmark and analysis platform for natural language understanding\.InEMNLP BlackboxNLP Workshop,Note:RTE recognized as part of the GLUE benchmark\.Cited by:[§5](https://arxiv.org/html/2605.26339#S5.SS0.SSS0.Px3.p1.1)\.
- G\. Xiao, J\. Lin, M\. Seznec, H\. Wu, J\. Demouth, and S\. Han \(2023\)SmoothQuant: accurate and efficient post\-training quantization for large language models\.InProceedings of the 40th International Conference on Machine Learning \(ICML\),Cited by:[§1](https://arxiv.org/html/2605.26339#S1.p2.7),[§2](https://arxiv.org/html/2605.26339#S2.SS0.SSS0.Px1.p1.1),[§5](https://arxiv.org/html/2605.26339#S5.SS0.SSS0.Px1.p1.2),[§6](https://arxiv.org/html/2605.26339#S6.p2.4)\.
- A\. Yang, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Li, D\. Liu, F\. Huang, H\. Wei,et al\.\(2024\)Qwen2\.5 technical report\.arXiv preprint arXiv:2412\.15115\.Cited by:[§5](https://arxiv.org/html/2605.26339#S5.p1.1)\.
- P\. L\. Zador \(1982\)Asymptotic quantization error of continuous signals and the quantization dimension\.IEEE Transactions on Information Theory28\(2\),pp\. 139–149\.Cited by:[item \(C1\)](https://arxiv.org/html/2605.26339#A3.I1.ix1.p1.10),[§C\.2](https://arxiv.org/html/2605.26339#A3.SS2.p2.1)\.
- R\. Zellers, A\. Holtzman, Y\. Bisk, A\. Farhadi, and Y\. Choi \(2019\)HellaSwag: can a machine really finish your sentence?\.InAnnual Meeting of the Association for Computational Linguistics \(ACL\),Cited by:[§5](https://arxiv.org/html/2605.26339#S5.SS0.SSS0.Px3.p1.1)\.
- P\. Zhang, G\. Zeng, T\. Wang, and W\. Lu \(2024\)TinyLlama: an open\-source small language model\.arXiv preprint arXiv:2401\.02385\.Cited by:[§5](https://arxiv.org/html/2605.26339#S5.p1.1)\.

## Appendix ACodec\-Level Analysis

With the encoder/decoder fixed \([section˜3](https://arxiv.org/html/2605.26339#S3)\), the two codec operations that determine directional reconstruction error are the block\-Hadamard rotation and the per\-pair quantizer\. Each is analyzed in turn: the rotation as an exact isometry, and the polar amplitude\-phase decomposition as the baseline against which the joint 2D codebook is judged\. Layer\-output effects — how this directional error translates to perturbations under real activations — are deferred to[appendix˜B](https://arxiv.org/html/2605.26339#A2)\.

### A\.1Analysis I: Rotation Isometry

The normalized row is transformed by a sign\-masked block\-Hadamard rotation\. This rotation is an exact isometry in real arithmetic, so it cannot add mathematical distortion by itself\. Its practical role is to reduce coordinate coherence and spread energy within each block before quantization; the circular\-Gaussian model used by the pair codebooks is calibrated empirically and does not follow from orthogonality alone\.

###### Definition A\.1\(Unnormalized Walsh\-Hadamard matrix\)\.

For any integerb=2mb=2^\{m\}withm≥0m\\geq 0, the unnormalized Walsh\-Hadamard matrixHb∈\{−1,\+1\}b×bH\_\{b\}\\in\\\{\-1,\+1\\\}^\{b\\times b\}is defined recursively byH1=\[1\]H\_\{1\}=\[1\]and

H2​b=\[HbHbHb−Hb\]\.H\_\{2b\}=\\begin\{bmatrix\}H\_\{b\}&H\_\{b\}\\\\ H\_\{b\}&\-H\_\{b\}\\end\{bmatrix\}\.\(4\)It satisfiesHb⊤​Hb=b​IH\_\{b\}^\{\\top\}H\_\{b\}=bIandHb=Hb⊤H\_\{b\}=H\_\{b\}^\{\\top\}\.

###### Definition A\.2\(Block\-normalized Hadamard transform\)\.

Letbbbe a power of two dividingdind\_\{\\text\{in\}\}\. The normalized block\-Hadamard matrixF∈ℝdin×dinF\\in\\mathbb\{R\}^\{d\_\{\\text\{in\}\}\\times d\_\{\\text\{in\}\}\}is block diagonal withdin/bd\_\{\\text\{in\}\}/bidentical blocks, each equal to\(1/b\)​Hb\(1/\\sqrt\{b\}\)H\_\{b\}:

F=1b​Hb⊕⋯⊕Hb⏟d/b​blocks\.F=\\frac\{1\}\{\\sqrt\{b\}\}\\,\\underbrace\{H\_\{b\}\\oplus\\cdots\\oplus H\_\{b\}\}\_\{d/b\\text\{ blocks\}\}\.\(5\)

###### Definition A\.3\(Sign mask\)\.

Lets1,…,sdin∈\{±1\}s\_\{1\},\\dots,s\_\{d\_\{\\text\{in\}\}\}\\in\\\{\\pm 1\\\}be a deterministic sequence generated from a fixed 64\-bit seed\. The sign mask is the diagonal matrixS=diag​\(s1,…,sdin\)S=\\mathrm\{diag\}\(s\_\{1\},\\dots,s\_\{d\_\{\\text\{in\}\}\}\)\.

###### Lemma 1\(Block\-Hadamard rotation isometry\)\.

Assumedin=m​bd\_\{\\text\{in\}\}=mbfor integersm≥1m\\geq 1andb=2qb=2^\{q\}withq≥1q\\geq 1\. LetF=Im⊗b−1/2​HbF=I\_\{m\}\\otimes b^\{\-1/2\}H\_\{b\}be the block\-normalized Hadamard matrix andS=diag​\(s1,…,sdin\)S=\\mathrm\{diag\}\(s\_\{1\},\\dots,s\_\{d\_\{\\text\{in\}\}\}\)withsj∈\{±1\}s\_\{j\}\\in\\\{\\pm 1\\\}\. WithRfwd=F​SR\_\{\\mathrm\{fwd\}\}=FSandRinv=S​FR\_\{\\mathrm\{inv\}\}=SF, both maps are orthogonal in exact real arithmetic \(Rfwd⊤​Rfwd=Rinv⊤​Rinv=IR\_\{\\mathrm\{fwd\}\}^\{\\top\}R\_\{\\mathrm\{fwd\}\}=R\_\{\\mathrm\{inv\}\}^\{\\top\}R\_\{\\mathrm\{inv\}\}=I\), mutually inverse \(Rinv​Rfwd=IR\_\{\\mathrm\{inv\}\}R\_\{\\mathrm\{fwd\}\}=I\), and preserve norms, distances, inner products, and per\-block energy\.

###### Proof\.

Hb⊤​Hb=b​IH\_\{b\}^\{\\top\}H\_\{b\}=bIandHb=Hb⊤H\_\{b\}=H\_\{b\}^\{\\top\}, so each normalized blockb−1/2​Hbb^\{\-1/2\}H\_\{b\}is orthogonal and symmetric\.FFis block diagonal with these blocks, givingF⊤​F=IF^\{\\top\}F=IandF⊤=FF^\{\\top\}=F\.SSis diagonal with±1\\pm 1entries, soS⊤=SS^\{\\top\}=SandS2=IS^\{2\}=I\. ThenRfwd⊤​Rfwd=S​F⊤​F​S=IR\_\{\\mathrm\{fwd\}\}^\{\\top\}R\_\{\\mathrm\{fwd\}\}=SF^\{\\top\}FS=IandRinv​Rfwd=S​F⋅F​S=S​F2​S=IR\_\{\\mathrm\{inv\}\}R\_\{\\mathrm\{fwd\}\}=SF\\cdot FS=SF^\{2\}S=I\. Norm, distance, and inner\-product preservation follow from orthogonality; per\-block energy preservation follows becauseFFandSSact independently within each block of the partition\. ∎

### A\.2Analysis II: Polar Pair Distortion

The second codec\-level theorem analyzes the polar baseline from[section˜3](https://arxiv.org/html/2605.26339#S3)\. The theorem gives a deterministic amplitude\-phase decomposition; the following corollary adds the Rayleigh/Lloyd source model used to compare independent amplitude\-phase coding against joint 2D QAM\.

###### Theorem 1\(Polar QAM pair distortion decomposition\)\.

For a complex pairz=a​ei​θz=ae^\{i\\theta\}and reconstructionz^=a^​ei​θ^\\hat\{z\}=\\hat\{a\}e^\{i\\hat\{\\theta\}\},

‖z−z^‖2=\(a−a^\)2\+2​a​a^​\(1−cos⁡\(θ−θ^\)\)\.\\\|z\-\\hat\{z\}\\\|^\{2\}=\(a\-\\hat\{a\}\)^\{2\}\+2a\\hat\{a\}\\bigl\(1\-\\cos\(\\theta\-\\hat\{\\theta\}\)\\bigr\)\.\(6\)Ifa,a^,θ,θ^a,\\hat\{a\},\\theta,\\hat\{\\theta\}are random variables with finite second moments andδ=wrap⁡\(θ−θ^\)\\delta=\\operatorname\{wrap\}\(\\theta\-\\hat\{\\theta\}\), then

𝔼​‖z−z^‖2=𝔼​\(a−a^\)2\+2​𝔼​\[a​a^​\(1−cos⁡δ\)\]\.\\mathbb\{E\}\\\|z\-\\hat\{z\}\\\|^\{2\}=\\mathbb\{E\}\(a\-\\hat\{a\}\)^\{2\}\+2\\mathbb\{E\}\\left\[a\\hat\{a\}\\bigl\(1\-\\cos\\delta\\bigr\)\\right\]\.\(7\)

###### Proof\.

The deterministic identity follows by expanding

\|a​ei​θ−a^​ei​θ^\|2\\displaystyle\|ae^\{i\\theta\}\-\\hat\{a\}e^\{i\\hat\{\\theta\}\}\|^\{2\}=a2\+a^2−2​a​a^​cos⁡\(θ−θ^\)\\displaystyle=a^\{2\}\+\\hat\{a\}^\{2\}\-2a\\hat\{a\}\\cos\(\\theta\-\\hat\{\\theta\}\)=\(a−a^\)2\\displaystyle=\(a\-\\hat\{a\}\)^\{2\}\+2​a​a^​\(1−cos⁡\(θ−θ^\)\)\.\\displaystyle\\quad\+2a\\hat\{a\}\\bigl\(1\-\\cos\(\\theta\-\\hat\{\\theta\}\)\\bigr\)\.Because cosine is2​π2\\pi\-periodic, replacingθ−θ^\\theta\-\\hat\{\\theta\}by the wrapped errorδ\\deltadoes not change the expression\. Taking expectations gives the second claim\. ∎

###### Corollary 1\(Rayleigh\-Lloyd polar distortion bound\)\.

Assume the circular\-Gaussian model for pairkk:a=σk​Ua=\\sigma\_\{k\}UwithU∼Rayleigh⁡\(1\)U\\sim\\operatorname\{Rayleigh\}\(1\),θ\\thetais uniform on\(−π,π\]\(\-\\pi,\\pi\], andaaandθ\\thetaare independent\. LetQaQ\_\{a\}be anNaN\_\{a\}\-level Lloyd\-Max quantizer forUUunder squared error, with normalized distortionCLM​\(Na\)=𝔼​\(U−Qa​\(U\)\)2C\_\{\\mathrm\{LM\}\}\(N\_\{a\}\)=\\mathbb\{E\}\(U\-Q\_\{a\}\(U\)\)^\{2\}, and seta^=σk​Qa​\(U\)\\hat\{a\}=\\sigma\_\{k\}Q\_\{a\}\(U\)\. Let the phase quantizer roundθ\\thetato the nearest one ofNpN\_\{p\}uniform phase levels, with bin centers aligned to the grid\{\(2​k\+1\)​h:k=0,1,…,Np−1\}\\\{\(2k\+1\)h:k=0,1,\\ldots,N\_\{p\}\-1\\\}modulo2​π2\\pi, whereh=π/Nph=\\pi/N\_\{p\}\(so that adjacent phase bin boundaries are spaced by2​h2hand bin centers sit at midpoints\)\. Then

𝔼​‖z−z^‖2=\\displaystyle\\mathbb\{E\}\\\|z\-\\hat\{z\}\\\|^\{2\}=\{\}σk2​CLM​\(Na\)\\displaystyle\\sigma\_\{k\}^\{2\}C\_\{\\mathrm\{LM\}\}\(N\_\{a\}\)\(8\)\+2​σk2​Ma​\(Na\)​η​\(Np\),\\displaystyle\+2\\sigma\_\{k\}^\{2\}M\_\{a\}\(N\_\{a\}\)\\eta\(N\_\{p\}\),where

Ma​\(Na\)=𝔼​\[U​Qa​\(U\)\],η​\(Np\)=1−sin⁡\(π/Np\)π/Np\.\\begin\{gathered\}M\_\{a\}\(N\_\{a\}\)=\\mathbb\{E\}\\bigl\[UQ\_\{a\}\(U\)\\bigr\],\\\\ \\eta\(N\_\{p\}\)=1\-\\frac\{\\sin\(\\pi/N\_\{p\}\)\}\{\\pi/N\_\{p\}\}\.\\end\{gathered\}\(9\)For a centroid Lloyd\-Max quantizer,Ma​\(Na\)=2−CLM​\(Na\)M\_\{a\}\(N\_\{a\}\)=2\-C\_\{\\mathrm\{LM\}\}\(N\_\{a\}\), hence

𝔼∥z−z^∥2=σk2\[\\displaystyle\\mathbb\{E\}\\\|z\-\\hat\{z\}\\\|^\{2\}=\\sigma\_\{k\}^\{2\}\\Bigl\[CLM​\(Na\)\\displaystyle C\_\{\\mathrm\{LM\}\}\(N\_\{a\}\)\(10\)\+2\(2−CLM\(Na\)\)η\(Np\)\]\\displaystyle\+2\\bigl\(2\-C\_\{\\mathrm\{LM\}\}\(N\_\{a\}\)\\bigr\)\\eta\(N\_\{p\}\)\\Bigr\]and

𝔼∥z−z^∥2≤σk2\[\\displaystyle\\mathbb\{E\}\\\|z\-\\hat\{z\}\\\|^\{2\}\\leq\\sigma\_\{k\}^\{2\}\\Bigl\[CLM​\(Na\)\\displaystyle C\_\{\\mathrm\{LM\}\}\(N\_\{a\}\)\(11\)\+\(2−CLM\(Na\)\)π23​Np2\]\.\\displaystyle\+\\bigl\(2\-C\_\{\\mathrm\{LM\}\}\(N\_\{a\}\)\\bigr\)\\frac\{\\pi^\{2\}\}\{3N\_\{p\}^\{2\}\}\\Bigr\]\.

###### Proof\.

The amplitude contribution is𝔼​\(a−a^\)2=σk2​CLM​\(Na\)\\mathbb\{E\}\(a\-\\hat\{a\}\)^\{2\}=\\sigma\_\{k\}^\{2\}C\_\{\\mathrm\{LM\}\}\(N\_\{a\}\)by the definition of the normalized Lloyd\-Max distortion\. Under the circular\-Gaussian model, amplitude and phase are independent\. Becauseθ\\thetais uniform on\(−π,π\]\(\-\\pi,\\pi\]and the phase quantizer is uniform with bin centers aligned to multiples of2​h2h\(the hypothesis on the phase grid\), nearest\-level rounding produces a wrapped phase errorδ\\deltathat is uniform on\[−h,h\]\[\-h,h\]withh=π/Nph=\\pi/N\_\{p\}, independent ofaaanda^\\hat\{a\}\. Hence

𝔼​\(1−cos⁡δ\)\\displaystyle\\mathbb\{E\}\(1\-\\cos\\delta\)=1−12​h​∫−hhcos⁡t​d​t\\displaystyle=1\-\\frac\{1\}\{2h\}\\int\_\{\-h\}^\{h\}\\cos t\\,dt\(12\)=1−sin⁡hh=η​\(Np\)\.\\displaystyle=1\-\\frac\{\\sin h\}\{h\}=\\eta\(N\_\{p\}\)\.The phase contribution is therefore2​𝔼​\[a​a^\]​η​\(Np\)=2​σk2​Ma​\(Na\)​η​\(Np\)2\\mathbb\{E\}\[a\\hat\{a\}\]\\eta\(N\_\{p\}\)=2\\sigma\_\{k\}^\{2\}M\_\{a\}\(N\_\{a\}\)\\eta\(N\_\{p\}\)\.

For a centroid Lloyd\-Max quantizer,Qa​\(U\)=𝔼​\[U∣Qa​\(U\)\]Q\_\{a\}\(U\)=\\mathbb\{E\}\[U\\mid Q\_\{a\}\(U\)\]on each cell\. By the tower property and the centroid identity, and noting thatQa​\(U\)Q\_\{a\}\(U\)isσ​\(Qa​\(U\)\)\\sigma\(Q\_\{a\}\(U\)\)\-measurable so it pulls inside the conditional expectation,

Ma​\(Na\)\\displaystyle M\_\{a\}\(N\_\{a\}\)=𝔼​\[U​Qa​\(U\)\]\\displaystyle=\\mathbb\{E\}\[UQ\_\{a\}\(U\)\]\(13\)=𝔼​\[𝔼​\[U∣Qa​\(U\)\]⋅Qa​\(U\)\]\\displaystyle=\\mathbb\{E\}\\bigl\[\\mathbb\{E\}\[U\\mid Q\_\{a\}\(U\)\]\\cdot Q\_\{a\}\(U\)\\bigr\]=𝔼​\[Qa​\(U\)2\]\.\\displaystyle=\\mathbb\{E\}\[Q\_\{a\}\(U\)^\{2\}\]\.Since𝔼​U2=2\\mathbb\{E\}U^\{2\}=2for the unit Rayleigh law,

CLM​\(Na\)\\displaystyle C\_\{\\mathrm\{LM\}\}\(N\_\{a\}\)=𝔼​\(U−Qa​\(U\)\)2\\displaystyle=\\mathbb\{E\}\(U\-Q\_\{a\}\(U\)\)^\{2\}\(14\)=𝔼​U2−𝔼​\[Qa​\(U\)2\]=2−Ma​\(Na\),\\displaystyle=\\mathbb\{E\}U^\{2\}\-\\mathbb\{E\}\[Q\_\{a\}\(U\)^\{2\}\]=2\-M\_\{a\}\(N\_\{a\}\),which givesMa​\(Na\)=2−CLM​\(Na\)M\_\{a\}\(N\_\{a\}\)=2\-C\_\{\\mathrm\{LM\}\}\(N\_\{a\}\)\. Finally, the bound1−sin⁡h/h≤h2/61\-\\sin h/h\\leq h^\{2\}/6follows from the Taylor expansionsin⁡h=h−h3/6\+O​\(h5\)\\sin h=h\-h^\{3\}/6\+O\(h^\{5\}\), valid forh∈\[0,π\]h\\in\[0,\\pi\]\(the remainder is non\-negative on this interval since the next term is\+h5/120\+h^\{5\}/120and the series is alternating with decreasing magnitude\)\. Substitutingh=π/Nph=\\pi/N\_\{p\}gives2​\(2−CLM​\(Na\)\)​η​\(Np\)≤\(2−CLM​\(Na\)\)​π2/\(3​Np2\)2\(2\-C\_\{\\mathrm\{LM\}\}\(N\_\{a\}\)\)\\eta\(N\_\{p\}\)\\leq\(2\-C\_\{\\mathrm\{LM\}\}\(N\_\{a\}\)\)\\pi^\{2\}/\(3N\_\{p\}^\{2\}\)\. ∎

### A\.3Rotated\-Domain Error Accounting

Each complex pair contains two scalar coordinates\. Because the inverse rotation is an isometry, the squared error of the unnormalized inverse\-rotated direction is the sum of pairwise errors:

‖u−u~‖22=‖Rfwd​u−y^‖22=∑k‖zk−z^k‖2,u~=Rinv​y^\.\\begin\{gathered\}\\\|u\-\\tilde\{u\}\\\|\_\{2\}^\{2\}=\\\|R\_\{\\mathrm\{fwd\}\}u\-\\hat\{y\}\\\|\_\{2\}^\{2\}=\\sum\_\{k\}\\\|z\_\{k\}\-\\hat\{z\}\_\{k\}\\\|^\{2\},\\\\ \\tilde\{u\}=R\_\{\\mathrm\{inv\}\}\\hat\{y\}\.\\end\{gathered\}\(15\)If the decoder subsequently normalizesu~\\tilde\{u\}before multiplying by the stored row norm, that final radial projection is a separate step; the equality above applies to the linear inverse\-rotation stage\.

###### Corollary 2\(Direction and per\-coordinate reporting\)\.

Let each pair error beek=‖zk−z^k‖2e\_\{k\}=\\\|z\_\{k\}\-\\hat\{z\}\_\{k\}\\\|^\{2\}\. Before the optional final direction renormalization,

‖u−u~‖22=∑kek,MSEcoord=1d​∑kek=12​meank⁡\(ek\)\.\\begin\{gathered\}\\\|u\-\\tilde\{u\}\\\|\_\{2\}^\{2\}=\\sum\_\{k\}e\_\{k\},\\\\ \\operatorname\{MSE\}\_\{\\mathrm\{coord\}\}=\\frac\{1\}\{d\}\\sum\_\{k\}e\_\{k\}=\\frac\{1\}\{2\}\\operatorname\{mean\}\_\{k\}\(e\_\{k\}\)\.\\end\{gathered\}\(16\)Ifuuandu^\\hat\{u\}are both unit vectors, then the cosine loss is exactly

1−⟨u,u^⟩=12​‖u−u^‖22\.1\-\\langle u,\\hat\{u\}\\rangle=\\frac\{1\}\{2\}\\\|u\-\\hat\{u\}\\\|\_\{2\}^\{2\}\.\(17\)

###### Proof\.

The first identity is the rotated\-domain error equality above, and the per\-coordinate statement only divides by thed=2​\#​\{k\}d=2\\,\\\#\\\{k\\\}scalar coordinates\. For unit vectors,‖u−u^‖22=‖u‖22\+‖u^‖22−2​⟨u,u^⟩=2​\(1−⟨u,u^⟩\)\\\|u\-\\hat\{u\}\\\|\_\{2\}^\{2\}=\\\|u\\\|\_\{2\}^\{2\}\+\\\|\\hat\{u\}\\\|\_\{2\}^\{2\}\-2\\langle u,\\hat\{u\}\\rangle=2\(1\-\\langle u,\\hat\{u\}\\rangle\)\. ∎

### A\.4Empirical Check of the Circular\-Gaussian Model

[Corollary˜1](https://arxiv.org/html/2605.26339#Thmcorollary1)and the Lloyd codebook training in[section˜3\.4](https://arxiv.org/html/2605.26339#S3.SS4)both invoke the circular\-Gaussian model for the post\-rotation pair distribution\. Whether this model is a useful idealization of empirical transformer weights is a separable question that can be checked directly\. For each of the three target architectures, one representative MLPdown\_projmatrix is processed with the same pipeline used by the codec: a sign\-masked block\-Hadamard rotation atb=1024b=1024block size, per\-row unit normalization, and sampling of50,00050\{,\}000rotated complex pairs\. The marginal distributions of the magnitude\|z\|\|z\|are then compared to a fitted Rayleigh law, and of the real partℜ⁡\(z\)\\Re\(z\)to a fitted zero\-mean Gaussian, via QQ plots \([fig\.˜3](https://arxiv.org/html/2605.26339#A1.F3)\)\.

![Refer to caption](https://arxiv.org/html/2605.26339v1/plots/day8/qq_rotated_pairs.png)Figure 3:QQ plots for the rotated complex\-pair marginals of a representativemlp\.down\_projmatrix on each of the three target architectures\. Top row, per architecture: empirical\|z\|\|z\|quantiles against a Rayleigh law fitted on the same sample\. Bottom row: empiricalℜ⁡\(z\)\\Re\(z\)quantiles against a zero\-mean Gaussian fitted on the same sample\. Sample size per panel is50,00050\{,\}000rotated pairs collected from one representative MLPdown\_projafter a sign\-masked block\-Hadamard rotation at the sameb=1024b\{=\}1024block size used by the codec\. The traces lie on they=xy\{=\}xidentity through the body of the distribution and depart only in the far tail\. The circular\-Gaussian model used by[corollary˜1](https://arxiv.org/html/2605.26339#Thmcorollary1)is therefore a reasonable working idealization in the regime that dominates the Lloyd objective\.[Figure˜3](https://arxiv.org/html/2605.26339#A1.F3)shows the empirical check: one representativemlp\.down\_projmatrix per architecture,50,00050\{,\}000rotated complex pairs after the same sign\-masked block\-Hadamard rotation the codec uses\. The body of the distribution tracks the Rayleigh \(magnitude\) and zero\-mean Gaussian \(real\-part\) marginals closely; departures appear only in the far tail, which the Lloyd training implicitly down\-weights through the density\-weighted MSE objective\. The post\-rotation distribution is therefore close enough to the circular\-Gaussian model that the analytical results based on it are an informative guide to codec behaviour, not a placeholder assumption\.

## Appendix BLayer Output Error and Activation Awareness

[appendix˜A](https://arxiv.org/html/2605.26339#A1)established that the codec’s reconstruction error is directional and controlled, to first order, by the per\-pair Lloyd distortion\. Quality at the model level, however, is not measured in weight space directly: a quantized layer is judged by how its output deviates from the BF16 output under the activations it actually sees\. This section bridges weight error to layer\-output error via a closed\-form trace identity, then uses that identity to motivate AWQ\-style per\-channel scaling\.

##### Notation\.

Throughout this and the following sections,M^x=X⊤​X\\hat\{M\}\_\{x\}=X^\{\\top\}Xis the unnormalized activation second\-moment matrix \(X∈ℝn×dinX\\in\\mathbb\{R\}^\{n\\times d\_\{\\mathrm\{in\}\}\}, withnnactivation rows in the calibration batch\), andMx=M^x/nM\_\{x\}=\\hat\{M\}\_\{x\}/nis its per\-token\-normalized form\. Both forms appear;M^x\\hat\{M\}\_\{x\}in finite\-sample empirical identities andMxM\_\{x\}when stating per\-sample expectations or when referring to a random activation row’s second moment\.

### B\.1Activation\-Weighted Layer Error

###### Proposition B\.1\(Layer output error identity\)\.

LetW,W^∈ℝdout×dinW,\\hat\{W\}\\in\\mathbb\{R\}^\{d\_\{\\mathrm\{out\}\}\\times d\_\{\\mathrm\{in\}\}\}withΔ​W=W−W^\\Delta W=W\-\\hat\{W\}, and letX∈ℝn×dinX\\in\\mathbb\{R\}^\{n\\times d\_\{\\mathrm\{in\}\}\}be a row\-major matrix of layer\-input activation vectors withn≥1n\\geq 1\. The identity conditions on the same input activationsXXbeing supplied to the original and quantized layer; if earlier quantized layers perturbXX, that propagation effect is outside this single\-layer statement\. Define the unnormalized activation second\-moment matrixM^x=X⊤​X\\hat\{M\}\_\{x\}=X^\{\\top\}X\. Then

‖X​W⊤−X​W^⊤‖F2\\displaystyle\\\|XW^\{\\top\}\-X\\hat\{W\}^\{\\top\}\\\|\_\{F\}^\{2\}=‖X​Δ​W⊤‖F2\\displaystyle=\\\|X\\Delta W^\{\\top\}\\\|\_\{F\}^\{2\}\(18\)=Tr⁡\(Δ​W​M^x​Δ​W⊤\)\.\\displaystyle=\\operatorname\{Tr\}\(\\Delta W\\hat\{M\}\_\{x\}\\Delta W^\{\\top\}\)\.WithMx=1n​X⊤​XM\_\{x\}=\\frac\{1\}\{n\}X^\{\\top\}X, the mean per\-sample squared output error is

1n​‖X​Δ​W⊤‖F2=Tr⁡\(Δ​W​Mx​Δ​W⊤\)\.\\frac\{1\}\{n\}\\\|X\\Delta W^\{\\top\}\\\|\_\{F\}^\{2\}=\\operatorname\{Tr\}\(\\Delta WM\_\{x\}\\Delta W^\{\\top\}\)\.\(19\)Equivalently, for a random activation rowx∈ℝdinx\\in\\mathbb\{R\}^\{d\_\{\\mathrm\{in\}\}\}with finite second momentMx=𝔼​\[x⊤​x\]M\_\{x\}=\\mathbb\{E\}\[x^\{\\top\}x\],

𝔼​\[‖x​W⊤−x​W^⊤‖22\]=Tr⁡\(Δ​W​Mx​Δ​W⊤\)\.\\mathbb\{E\}\\left\[\\\|xW^\{\\top\}\-x\\hat\{W\}^\{\\top\}\\\|\_\{2\}^\{2\}\\right\]=\\operatorname\{Tr\}\(\\Delta WM\_\{x\}\\Delta W^\{\\top\}\)\.\(20\)

###### Proof\.

‖X​Δ​W⊤‖F2\\displaystyle\\\|X\\Delta W^\{\\top\}\\\|\_\{F\}^\{2\}=Tr⁡\[\(X​Δ​W⊤\)​\(X​Δ​W⊤\)⊤\]\\displaystyle=\\operatorname\{Tr\}\\left\[\(X\\Delta W^\{\\top\}\)\(X\\Delta W^\{\\top\}\)^\{\\top\}\\right\]=Tr⁡\(X​Δ​W⊤​Δ​W​X⊤\)\\displaystyle=\\operatorname\{Tr\}\(X\\Delta W^\{\\top\}\\Delta WX^\{\\top\}\)=Tr⁡\(Δ​W​X⊤​X​Δ​W⊤\)\.\\displaystyle=\\operatorname\{Tr\}\(\\Delta WX^\{\\top\}X\\Delta W^\{\\top\}\)\.Dividing bynngives the empirical mean\. The random\-activation statement is the same identity withMx=𝔼​\[x⊤​x\]M\_\{x\}=\\mathbb\{E\}\[x^\{\\top\}x\]\. ∎

This identity shows that the relevant layer objective is not unweighted weight MSE,‖Δ​W‖F2\\\|\\Delta W\\\|\_\{F\}^\{2\}, but activation\-weighted error on the calibration activations\. Unweighted MSE is recovered only whenMxM\_\{x\}is proportional to the identity\. The matrixMxM\_\{x\}is an uncentered second moment; it equals the covariance only when activations have zero mean\. The implementation reports the relative layer\-output RMSE

‖X​W⊤−X​W^⊤‖F‖X​W⊤‖F,\\frac\{\\\|XW^\{\\top\}\-X\\hat\{W\}^\{\\top\}\\\|\_\{F\}\}\{\\\|XW^\{\\top\}\\\|\_\{F\}\},\(21\)which is the proposition’s numerator normalized by the BF16 layer\-output norm\.

###### Corollary 3\(Diagonal activation\-RMS surrogate\)\.

If the normalized activation second moment is diagonal,Mx=diag⁡\(r12,…,rdin2\)M\_\{x\}=\\operatorname\{diag\}\(r\_\{1\}^\{2\},\\ldots,r\_\{d\_\{\\mathrm\{in\}\}\}^\{2\}\), then

1n​‖X​Δ​W⊤‖F2=∑i=1dout∑j=1dinrj2​\(Δ​Wi​j\)2\.\\frac\{1\}\{n\}\\\|X\\Delta W^\{\\top\}\\\|\_\{F\}^\{2\}=\\sum\_\{i=1\}^\{d\_\{\\mathrm\{out\}\}\}\\sum\_\{j=1\}^\{d\_\{\\mathrm\{in\}\}\}r\_\{j\}^\{2\}\(\\Delta W\_\{ij\}\)^\{2\}\.\(22\)For empirical calibration activations,rj2=n−1​∑t=1nXt​j2r\_\{j\}^\{2\}=n^\{\-1\}\\sum\_\{t=1\}^\{n\}X\_\{tj\}^\{2\}is the squared RMS of input channeljj\.

###### Proof\.

Substitute the diagonalMxM\_\{x\}into[proposition˜B\.1](https://arxiv.org/html/2605.26339#A2.Thmproposition1):

Tr⁡\(Δ​W​Mx​Δ​W⊤\)\\displaystyle\\operatorname\{Tr\}\(\\Delta WM\_\{x\}\\Delta W^\{\\top\}\)=∑i\(Δ​W​Mx​Δ​W⊤\)i​i\\displaystyle=\\sum\_\{i\}\(\\Delta WM\_\{x\}\\Delta W^\{\\top\}\)\_\{ii\}=∑i,jrj2​\(Δ​Wi​j\)2\.\\displaystyle=\\sum\_\{i,j\}r\_\{j\}^\{2\}\(\\Delta W\_\{ij\}\)^\{2\}\.The empirical formula forrj2r\_\{j\}^\{2\}is thejjth diagonal entry ofn−1​X⊤​Xn^\{\-1\}X^\{\\top\}X\. ∎

WhenMxM\_\{x\}is not diagonal, this expression is a diagonal approximation that ignores cross\-channel activation correlations in the current input basis\. It still gives the operative AWQ\-style intuition: errors in high\-RMS channels matter more to the layer output than equal\-sized errors in low\-RMS channels\.

### B\.2Activation\-Aware Scaling

QAM\-W uses AWQ\-inspired per\-input\-channel scaling before quantization\. For scaling factorssj\>0s\_\{j\}\>0andDs=diag⁡\(s1,…,sdin\)D\_\{s\}=\\operatorname\{diag\}\(s\_\{1\},\\ldots,s\_\{d\_\{\\mathrm\{in\}\}\}\),

Wscaled\\displaystyle W\_\{\\mathrm\{scaled\}\}=W​Ds,\\displaystyle=WD\_\{s\},\(23\)W^scaled\\displaystyle\\hat\{W\}\_\{\\mathrm\{scaled\}\}=Quantize⁡\(Wscaled\),\\displaystyle=\\operatorname\{Quantize\}\(W\_\{\\mathrm\{scaled\}\}\),\(24\)W^\\displaystyle\\hat\{W\}=W^scaled​Ds−1\.\\displaystyle=\\hat\{W\}\_\{\\mathrm\{scaled\}\}D\_\{s\}^\{\-1\}\.\(25\)The implementation setssj∝rjαs\_\{j\}\\propto r\_\{j\}^\{\\alpha\}withα∈\{0\.3,0\.5,0\.8\}\\alpha\\in\\\{0\.3,0\.5,0\.8\\\}, normalizes the geometric mean of the scales to one, and clamps scales to\[1/16,16\]\[1/16,16\]\.

###### Proposition B\.2\(Sufficient\-condition model for activation\-aware scaling\)\.

LetEs=Wscaled−W^scaledE\_\{s\}=W\_\{\\mathrm\{scaled\}\}\-\\hat\{W\}\_\{\\mathrm\{scaled\}\}be the quantization error in the scaled domain, with expectation taken over the quantizer randomness alone for a fixed \(deterministic\) weight matrixWW\. Assume:

- \(A1\)*Scale\-invariant distortion:*the per\-coordinate quantization error satisfies𝔼​\[\(Es\)i​j2\]≤ci​j\\mathbb\{E\}\[\(E\_\{s\}\)\_\{ij\}^\{2\}\]\\leq c\_\{ij\}for bounded constantsci​jc\_\{ij\}that do not depend on the scaling factorss1,…,sdins\_\{1\},\\ldots,s\_\{d\_\{\\mathrm\{in\}\}\}\.
- \(A2\)*Diagonal activation second moment:*Mx=diag⁡\(r12,…,rdin2\)M\_\{x\}=\\operatorname\{diag\}\(r\_\{1\}^\{2\},\\ldots,r\_\{d\_\{\\mathrm\{in\}\}\}^\{2\}\), as in[corollary˜3](https://arxiv.org/html/2605.26339#Thmcorollary3)\.
- \(A3\)*Unclamped power\-law scaling:*sj∝rjαs\_\{j\}\\propto r\_\{j\}^\{\\alpha\}for someα∈\[0,1\]\\alpha\\in\[0,1\], with the global geometric\-mean normalization absorbed intoWWand no per\-channel clipping\.

Then the de\-scaled errorΔ​W\(s\)=W−W^=Es​Ds−1\\Delta W^\{\(s\)\}=W\-\\hat\{W\}=E\_\{s\}D\_\{s\}^\{\-1\}satisfies𝔼​\[\(Δ​Wi​j\(s\)\)2\]=𝔼​\[\(Es\)i​j2\]/sj2≤ci​j/sj2\\mathbb\{E\}\[\(\\Delta W\_\{ij\}^\{\(s\)\}\)^\{2\}\]=\\mathbb\{E\}\[\(E\_\{s\}\)\_\{ij\}^\{2\}\]/s\_\{j\}^\{2\}\\leq c\_\{ij\}/s\_\{j\}^\{2\}, where the equality usesΔ​Wi​j\(s\)=\(Es\)i​j/sj\\Delta W\_\{ij\}^\{\(s\)\}=\(E\_\{s\}\)\_\{ij\}/s\_\{j\}withsjs\_\{j\}deterministic and the inequality is \(A1\)\. Under \(A2\) and[corollary˜3](https://arxiv.org/html/2605.26339#Thmcorollary3), the activation\-weighted error satisfies

ℰact​\(s\)=∑i,jrj2​𝔼​\[\(Δ​Wi​j\(s\)\)2\]≤∑i,jci​j​rj2sj2\.\\mathcal\{E\}\_\{\\mathrm\{act\}\}\(s\)=\\sum\_\{i,j\}r\_\{j\}^\{2\}\\,\\mathbb\{E\}\\bigl\[\(\\Delta W\_\{ij\}^\{\(s\)\}\)^\{2\}\\bigr\]\\leq\\sum\_\{i,j\}c\_\{ij\}\\frac\{r\_\{j\}^\{2\}\}\{s\_\{j\}^\{2\}\}\.\(26\)Under \(A3\) the channel factor changes fromrj2r\_\{j\}^\{2\}torj2​\(1−α\)r\_\{j\}^\{2\(1\-\\alpha\)\}, attenuating the activation\-weighted contribution of high\-RMS channels\.

###### Proof\.

WithWWfixed and the expectation taken over the quantizer randomness only,Δ​Wi​j\(s\)=\(Es\)i​j/sj\\Delta W\_\{ij\}^\{\(s\)\}=\(E\_\{s\}\)\_\{ij\}/s\_\{j\}withsjs\_\{j\}deterministic, so𝔼​\[\(Δ​Wi​j\(s\)\)2\]=𝔼​\[\(Es\)i​j2\]/sj2\\mathbb\{E\}\[\(\\Delta W\_\{ij\}^\{\(s\)\}\)^\{2\}\]=\\mathbb\{E\}\[\(E\_\{s\}\)\_\{ij\}^\{2\}\]/s\_\{j\}^\{2\}\. Assumption \(A1\)—independence ofci​jc\_\{ij\}fromss—is what makes this division valid as a uniform bound across the choice of scales:𝔼​\[\(Δ​Wi​j\(s\)\)2\]≤ci​j/sj2\\mathbb\{E\}\[\(\\Delta W\_\{ij\}^\{\(s\)\}\)^\{2\}\]\\leq c\_\{ij\}/s\_\{j\}^\{2\}\. Substituting into the diagonal\-MxM\_\{x\}identity from[corollary˜3](https://arxiv.org/html/2605.26339#Thmcorollary3)\(which is \(A2\)\) givesℰact​\(s\)=∑i,jrj2​𝔼​\[\(Δ​Wi​j\(s\)\)2\]≤∑i,jci​j​rj2/sj2\\mathcal\{E\}\_\{\\mathrm\{act\}\}\(s\)=\\sum\_\{i,j\}r\_\{j\}^\{2\}\\mathbb\{E\}\[\(\\Delta W\_\{ij\}^\{\(s\)\}\)^\{2\}\]\\leq\\sum\_\{i,j\}c\_\{ij\}r\_\{j\}^\{2\}/s\_\{j\}^\{2\}\. Under \(A3\),sj∝rjαs\_\{j\}\\propto r\_\{j\}^\{\\alpha\}with global normalization absorbed intoWWand no clipping, so the channel factorrj2/sj2r\_\{j\}^\{2\}/s\_\{j\}^\{2\}is proportional torj2​\(1−α\)r\_\{j\}^\{2\(1\-\\alpha\)\}\. ∎

The proposition is a design model rather than a universal guarantee; it describes how \(A1\)–\(A3\) interact, and the deployed quantizer satisfies all three only approximately\. Real quantizers are nonlinear: codebook boundaries, row renormalization, and non\-uniform Lloyd levels all change the effective source distribution, so \(A1\) fails when the constantsci​jc\_\{ij\}shift withssunder aggressive scaling\.[Appendix˜D](https://arxiv.org/html/2605.26339#A4)probes \(A1\) directly on a representative MLPdown\_projlayer of Mistral\-7B and finds that the mediancjc\_\{j\}is essentially flat acrossα∈\[0,0\.8\]\\alpha\\in\[0,0\.8\]but the maximumcjc\_\{j\}grows∼50×\\sim\\\!50\\times, so \(A1\) holds in the bulk and fails in the tail; the deployedα=0\.3\\alpha=0\.3sits well below the failure regime\. Activation\-aware scaling is therefore treated as an objective\-motivated variant whose behaviour is checked empirically via layer\-output RMSE and paired KL rather than asserted from the design model\.

### B\.3Empirical Check of the Layer\-Output Identity

[Proposition˜B\.1](https://arxiv.org/html/2605.26339#A2.Thmproposition1)is exact, so it does not need empirical validation as a mathematical statement\. The operational claim that follows from it — that unweighted weight Frobenius‖Δ​W‖F/‖W‖F\\\|\\Delta W\\\|\_\{F\}/\\\|W\\\|\_\{F\}is a worse proxy for what the model actually computes than the activation\-weighted layer\-output RMSETr⁡\(Δ​W​Mx​Δ​W⊤\)/Tr⁡\(W​Mx​W⊤\)\\sqrt\{\\operatorname\{Tr\}\(\\Delta WM\_\{x\}\\Delta W^\{\\top\}\)/\\operatorname\{Tr\}\(WM\_\{x\}W^\{\\top\}\)\}— does\.[Table˜4](https://arxiv.org/html/2605.26339#A2.T4)reports both quantities \(averaged across the 96 MLP linear layers of Mistral\-7B\-Instruct\-v0\.3\) for four methods spanning two orders of magnitude inΔ\\DeltaPPL%\. The Mx matrices are computed from forward\-pass activations of the BF16 reference on eight WikiText\-2 sequences of length 2048\.

Table 4:Empirical anchor for[proposition˜B\.1](https://arxiv.org/html/2605.26339#A2.Thmproposition1): per\-layer unweighted weight Frobenius and activation\-weighted layer\-output RMSE, averaged over the 96 MLP linear layers of Mistral 7B \(checkpoints from the MLP \+ attention quantization extension,[section˜F\.1](https://arxiv.org/html/2605.26339#A6.SS1)\)\. Theamp\.column is the median per\-layer ratioρo/ρw\\rho\_\{o\}/\\rho\_\{w\}, whereρw=‖Δ​W‖F/‖W‖F\\rho\_\{w\}=\\\|\\Delta W\\\|\_\{F\}/\\\|W\\\|\_\{F\}andρo2=Tr⁡\(Δ​W​Mx​Δ​W⊤\)/Tr⁡\(W​Mx​W⊤\)\\rho\_\{o\}^\{2\}=\\operatorname\{Tr\}\(\\Delta WM\_\{x\}\\Delta W^\{\\top\}\)/\\operatorname\{Tr\}\(WM\_\{x\}W^\{\\top\}\)\. Mean KL andΔ\\DeltaPPL% are from[table˜6](https://arxiv.org/html/2605.26339#A6.T6)\.
Reader key:↓\\downarrowlower is better\. Shaded rows \(⋆\\star\) are QAM\-W \(ours\)\.Two observations from[table˜4](https://arxiv.org/html/2605.26339#A2.T4)support the operational reading of[proposition˜B\.1](https://arxiv.org/html/2605.26339#A2.Thmproposition1)\.

##### Method\-dependent amplification\.

The activation\-weighting amplification factor amp\. med\. varies by nearly2×2\\timesacross the four methods\. GPTQ’s Hessian\-aware optimization aligns its residual error with low\-MxM\_\{x\}directions and earns the largest discount \(medianρo/ρw=0\.57\\rho\_\{o\}/\\rho\_\{w\}=0\.57\): the activation\-weighted layer\-output RMSE is∼1\.8×\\sim\\\!1\.8\\timessmaller than the unweighted weight Frobenius would suggest\. AWQ does not earn the same discount \(median0\.950\.95\): its weight error is roughly isotropic with respect toMxM\_\{x\}, so the activation weighting barely changes the apparent magnitude\. SmoothQuant and the activation\-aware joint\-2D codec sit in between \(0\.870\.87and0\.830\.83respectively\)\. At equal raw weight Frobenius, two methods can therefore differ substantially on what the model actually computes; weight Frobenius alone is not enough to rank them\.

##### Per\-layer spread\.

The amplification factor also has substantial*within*\-method spread\. For AWQ the per\-layer ratio ranges from0\.550\.55to5\.535\.53across the 96 layers \(std\.0\.500\.50\); the maximum corresponds to the embedding\-adjacent MLP block where the AWQ error happens to project disproportionately onto high\-MxM\_\{x\}directions, and the model output inherits the full hit\. GPTQ’s per\-layer ratio, by contrast, sits in a tight\[0\.24,0\.69\]\[0\.24,0\.69\]band \(std\.0\.120\.12\)\. This is direct evidence that the layer\-output identity is doing real work: two layers with the same‖Δ​W‖F\\\|\\Delta W\\\|\_\{F\}can produce layer\-output error that differs by an order of magnitude depending on how that error aligns with the calibrationMxM\_\{x\}\. The codec that explicitly accounts forMxM\_\{x\}via per\-channel scaling — whether at calibration time \(GPTQ\) or at quantization time \(QAM\-W withα=0\.3\\alpha=0\.3\) — earns a measurable reduction inρo\\rho\_\{o\}that unweighted Frobenius cannot see\.

## Appendix CFrom Codec Error to Model Behavior

[Proposition˜B\.1](https://arxiv.org/html/2605.26339#A2.Thmproposition1)maps weight error to layer\-output error\.[Appendix˜A](https://arxiv.org/html/2605.26339#A1)earlier derived per\-pair codec distortion\. This section closes the chain to model behavior with a local logit\-KL bridge, then composes the three steps into a single monotone upper bound whose codec\-dependent ingredient is the per\-pair Lloyd distortionDBD\_\{B\}on the unit circular Gaussian\. The bound is tested empirically in[sections˜5\.1](https://arxiv.org/html/2605.26339#S5.SS1)and[5\.3\.2](https://arxiv.org/html/2605.26339#S5.SS3.SSS2): if informative, measured paired KL should be rank\-correlated withDBD\_\{B\}across methods at fixed architecture\.

### C\.1Logit Perturbation and Local KL Bridge

Letf​\(x\)f\(x\)be the BF16 logits andf^​\(x\)\\hat\{f\}\(x\)the logits under quantized weights, withδ=f^​\(x\)−f​\(x\)\\delta=\\hat\{f\}\(x\)\-f\(x\)\. If the transformer computation after the perturbed linear layers is locally Lipschitz around the evaluation activation trajectories, then the logit perturbation is controlled by the accumulated layer\-output perturbations,

‖δ‖2≲L​∑ℓ‖Xℓ​Δ​Wℓ⊤‖F\.\\\|\\delta\\\|\_\{2\}\\;\\lesssim\\;L\\sum\_\{\\ell\}\\\|X\_\{\\ell\}\\Delta W\_\{\\ell\}^\{\\top\}\\\|\_\{F\}\.\(27\)The constantLLis not estimated here because norm\-only Lipschitz bounds for transformers are usually too loose to be useful\. The structural point is that activation\-weighted layer error is the quantity that propagates, not raw weight MSE\.

###### Corollary 4\(Local logit\-KL bridge\)\.

Letp=softmax⁡\(f\)p=\\operatorname\{softmax\}\(f\)andq=softmax⁡\(f\+δ\)q=\\operatorname\{softmax\}\(f\+\\delta\)with logit HessianFp=diag⁡\(p\)−p​p⊤F\_\{p\}=\\operatorname\{diag\}\(p\)\-pp^\{\\top\}\. Then

DKL​\(p∥q\)=12​δ⊤​Fp​δ\+r​\(δ\),D\_\{\\mathrm\{KL\}\}\(p\\\|q\)=\\tfrac\{1\}\{2\}\\delta^\{\\top\}F\_\{p\}\\delta\+r\(\\delta\),\(28\)where the third\-order Lagrange remainder satisfies\|r​\(δ\)\|≤13​‖δ‖23\|r\(\\delta\)\|\\leq\\tfrac\{1\}\{3\}\\\|\\delta\\\|\_\{2\}^\{3\}\(proof below; the softmax cumulant tensor’s operator\-3\-norm on the unit sphere is bounded by2/2≤1\\sqrt\{2\}/2\\leq 1\)\. Since‖Fp‖2≤1/2\\\|F\_\{p\}\\\|\_\{2\}\\leq 1/2, the local approximationDKL​\(p∥q\)≈14​‖δ‖22D\_\{\\mathrm\{KL\}\}\(p\\\|q\)\\approx\\tfrac\{1\}\{4\}\\\|\\delta\\\|\_\{2\}^\{2\}holds with absolute error≤13​‖δ‖23\\leq\\tfrac\{1\}\{3\}\\\|\\delta\\\|\_\{2\}^\{3\}\. Combining with \([27](https://arxiv.org/html/2605.26339#A3.E27)\) gives

DKL​\(p∥q\)≲L24​\(∑ℓ‖Xℓ​Δ​Wℓ⊤‖F\)2,D\_\{\\mathrm\{KL\}\}\(p\\\|q\)\\;\\lesssim\\;\\frac\{L^\{2\}\}\{4\}\\Bigl\(\\sum\_\{\\ell\}\\\|X\_\{\\ell\}\\Delta W\_\{\\ell\}^\{\\top\}\\\|\_\{F\}\\Bigr\)^\{2\},\(29\)valid in the regime where the cubic remainder is small relative to the quadratic term\.

###### Proof\.

Taylor expansion of softmax cross\-entropy around logitsff\. The first\-order term vanishes because KL is minimized atq=pq=p, and the Hessian isFpF\_\{p\}\. The operator\-norm bound‖Fp‖2≤1/2\\\|F\_\{p\}\\\|\_\{2\}\\leq 1/2follows from Popoviciu’s inequality applied to a unit\-vector random variable underpp\. For the cubic remainder, the third\-cumulant tensor of the softmax log\-partition isTi​j​k=κ3​\(p\)i​j​kT\_\{ijk\}=\\kappa\_\{3\}\(p\)\_\{ijk\}\. For a unit vectorvvandI∼pI\\sim p,T​\(v,v,v\)=𝔼p​\[\(vI−𝔼p​vI\)3\]T\(v,v,v\)=\\mathbb\{E\}\_\{p\}\[\(v\_\{I\}\-\\mathbb\{E\}\_\{p\}v\_\{I\}\)^\{3\}\]\. Since‖v‖2=1\\\|v\\\|\_\{2\}=1we havemax⁡v−min⁡v≤2\\max v\-\\min v\\leq\\sqrt\{2\}; combining\|𝔼​\[\(Y−𝔼​Y\)3\]\|≤\(max−min\)⋅Var​\(Y\)\|\\mathbb\{E\}\[\(Y\-\\mathbb\{E\}Y\)^\{3\}\]\|\\leq\(\\max\-\\min\)\\cdot\\mathrm\{Var\}\(Y\)with Popoviciu’s variance boundVar​\(Y\)≤\(max−min\)2/4\\mathrm\{Var\}\(Y\)\\leq\(\\max\-\\min\)^\{2\}/4gives\|T​\(v,v,v\)\|≤\(2\)3/4=2/2≤1\|T\(v,v,v\)\|\\leq\(\\sqrt\{2\}\)^\{3\}/4=\\sqrt\{2\}/2\\leq 1\. The Lagrange remainder isr​\(δ\)=16​T​\(ξ\)​\(δ,δ,δ\)r\(\\delta\)=\\tfrac\{1\}\{6\}T\(\\xi\)\(\\delta,\\delta,\\delta\)for someξ\\xion the segment, so\|r​\(δ\)\|≤16​‖δ‖3<13​‖δ‖3\|r\(\\delta\)\|\\leq\\tfrac\{1\}\{6\}\\,\\\|\\delta\\\|^\{3\}<\\tfrac\{1\}\{3\}\\\|\\delta\\\|^\{3\}\. Substituting \([27](https://arxiv.org/html/2605.26339#A3.E27)\) into the quadratic term gives the final display\. ∎

### C\.2Composite Bound

LetDBD\_\{B\}denote the per\-pair Lloyd\-Max MSE of the2B2^\{B\}\-point planar codebook on the unit circular Gaussian \([theorem˜1](https://arxiv.org/html/2605.26339#Thmtheorem1)\)\. For a single weight roww∈ℝdinw\\in\\mathbb\{R\}^\{d\_\{\\mathrm\{in\}\}\}with paired coordinates\{zk\}\\\{z\_\{k\}\\\}, each pair has calibrated scaleσk\\sigma\_\{k\}\([section˜3\.4](https://arxiv.org/html/2605.26339#S3.SS4)\)\. Codebook lookup is performed on each pair after rescaling toward the codebook’s source domain\.

###### Assumption C\.1\(Linear codec response\)\.

For pairkkwith per\-coordinate calibration varianceσk2\\sigma\_\{k\}^\{2\}from[section˜3\.4](https://arxiv.org/html/2605.26339#S3.SS4)\(so𝔼​‖zk‖2=2​σk2\\mathbb\{E\}\\,\\\|z\_\{k\}\\\|^\{2\}=2\\sigma\_\{k\}^\{2\}under the circular\-Gaussian model\), the per\-pair squared reconstruction error scales withσk2\\sigma\_\{k\}^\{2\}:𝔼​‖Δ​zk‖2≤σk2​DB\\mathbb\{E\}\\,\\\|\\Delta z\_\{k\}\\\|^\{2\}\\leq\\sigma\_\{k\}^\{2\}\\,D\_\{B\}, whereDBD\_\{B\}is the per\-pair Lloyd\-Max MSE on the unit circular Gaussian \(per\-coordinate variance11\) and the expectation is over the quantizer applied to a row drawn from the calibration sample\.

This is exact for high\-rate Lloyd codebooks applied to scaled copies of a fixed source\(Lloyd,[1982](https://arxiv.org/html/2605.26339#bib.bib9); Max,[1960](https://arxiv.org/html/2605.26339#bib.bib12); Gersho and Gray,[1991](https://arxiv.org/html/2605.26339#bib.bib10); Zador,[1982](https://arxiv.org/html/2605.26339#bib.bib11)\)\. For empirical post\-rotation pairs the assumption is an idealization; the QQ check \([fig\.˜3](https://arxiv.org/html/2605.26339#A1.F3)\) supports its accuracy in the bulk and[fig\.˜4](https://arxiv.org/html/2605.26339#A4.F4)probes its failure modes in the tail\.

Summing[Assumption˜C\.1](https://arxiv.org/html/2605.26339#A3.Thmassumption1)over pairs of a single row and using Parseval \(the block\-Hadamard rotation is an isometry, so∑k‖zk‖2=‖w‖2\\sum\_\{k\}\\\|z\_\{k\}\\\|^\{2\}=\\\|w\\\|^\{2\}deterministically; in expectation∑k2​σk2=𝔼​‖w‖2\\sum\_\{k\}2\\sigma\_\{k\}^\{2\}=\\mathbb\{E\}\\,\\\|w\\\|^\{2\}, hence∑kσk2=12​𝔼​‖w‖2\\sum\_\{k\}\\sigma\_\{k\}^\{2\}=\\tfrac\{1\}\{2\}\\mathbb\{E\}\\,\\\|w\\\|^\{2\}\) gives

𝔼​‖Δ​W‖F2≤12​DB​𝔼​‖W‖F2,\\mathbb\{E\}\\,\\\|\\Delta W\\\|\_\{F\}^\{2\}\\;\\leq\\;\\tfrac\{1\}\{2\}\\,D\_\{B\}\\,\\mathbb\{E\}\\,\\\|W\\\|\_\{F\}^\{2\},\(30\)where the expectation on both sides is over the calibration sample and the quantizer\. Plugging \([30](https://arxiv.org/html/2605.26339#A3.E30)\) into the activation\-weighted trace identity \([proposition˜B\.1](https://arxiv.org/html/2605.26339#A2.Thmproposition1), random\-activation form,Mx:=𝔼​\[x⊤​x\]M\_\{x\}:=\\mathbb\{E\}\[x^\{\\top\}x\]\) and applying the PSD quadratic\-form boundTr⁡\(A​M​A⊤\)≤λmax​\(M\)​‖A‖F2\\operatorname\{Tr\}\(A\\,M\\,A^\{\\top\}\)\\leq\\lambda\_\{\\max\}\(M\)\\,\\\|A\\\|\_\{F\}^\{2\},

ρℓ2:=𝔼x​\[‖x​Δ​Wℓ⊤‖22\]≤12​DB​λmax\(ℓ\)​‖Wℓ‖F2,\\rho\_\{\\ell\}^\{2\}:=\\mathbb\{E\}\_\{x\}\\\!\\left\[\\\|x\\,\\Delta W\_\{\\ell\}^\{\\top\}\\\|\_\{2\}^\{2\}\\right\]\\;\\leq\\;\\tfrac\{1\}\{2\}\\,D\_\{B\}\\,\\lambda\_\{\\max\}^\{\(\\ell\)\}\\,\\\|W\_\{\\ell\}\\\|\_\{F\}^\{2\},\(31\)whereλmax\(ℓ\):=λmax​\(Mx\(ℓ\)\)\\lambda\_\{\\max\}^\{\(\\ell\)\}:=\\lambda\_\{\\max\}\(M\_\{x\}^\{\(\\ell\)\}\)andρℓ\\rho\_\{\\ell\}is the per\-token layer\-output RMS at quantized layerℓ\\ell\. Summing over theLqL\_\{q\}quantized layers and applying Cauchy–Schwarz,

∑ℓ=1Lqρℓ≤12​DB​Lq​CW,CW:=∑ℓ=1Lqλmax\(ℓ\)​‖Wℓ‖F2\.\\begin\{gathered\}\\sum\_\{\\ell=1\}^\{L\_\{q\}\}\\rho\_\{\\ell\}\\;\\leq\\;\\sqrt\{\\tfrac\{1\}\{2\}\\,D\_\{B\}\\,L\_\{q\}\\,C\_\{W\}\},\\\\ C\_\{W\}:=\\sum\_\{\\ell=1\}^\{L\_\{q\}\}\\lambda\_\{\\max\}^\{\(\\ell\)\}\\,\\\|W\_\{\\ell\}\\\|\_\{F\}^\{2\}\.\\end\{gathered\}\(32\)Composing with[corollary˜4](https://arxiv.org/html/2605.26339#Thmcorollary4)gives the composite bound below\.

###### Theorem 2\(Composite codec\-to\-KL upper bound is monotone inDBD\_\{B\}andCWC\_\{W\}\)\.

Under the assumptions of[propositions˜B\.1](https://arxiv.org/html/2605.26339#A2.Thmproposition1)and[4](https://arxiv.org/html/2605.26339#Thmcorollary4), the upper bound on the realized paired KL divergence between the BF16 and quantized models obtained by chaining \([31](https://arxiv.org/html/2605.26339#A3.E31)\), \([32](https://arxiv.org/html/2605.26339#A3.E32)\), and[corollary˜4](https://arxiv.org/html/2605.26339#Thmcorollary4)is non\-decreasing in two scalars:DBD\_\{B\}\(the per\-pair Lloyd distortion of the codebook on the unit circular Gaussian\) at fixed architecture, andCWC\_\{W\}\(a model\-fixed constant set by the calibration activations and weight norms\) at fixed codec\.

𝔼​DKL​\(p∥q\)≲U​\(DB,CW\),∂U/∂DB≥0,∂U/∂CW≥0,\\begin\{gathered\}\\mathbb\{E\}\\,D\_\{\\mathrm\{KL\}\}\(p\\,\\\|\\,q\)\\;\\lesssim\\;U\\\!\\bigl\(D\_\{B\},\\,C\_\{W\}\\bigr\),\\\\ \\partial U/\\partial D\_\{B\}\\geq 0,\\;\\partial U/\\partial C\_\{W\}\\geq 0,\\end\{gathered\}\(33\)in the local regime where the cubic Taylor remainder of[corollary˜4](https://arxiv.org/html/2605.26339#Thmcorollary4)is small relative to the quadratic term\.

###### Proof\.

Each of \([31](https://arxiv.org/html/2605.26339#A3.E31)\), \([32](https://arxiv.org/html/2605.26339#A3.E32)\), and[corollary˜4](https://arxiv.org/html/2605.26339#Thmcorollary4)is monotone non\-decreasing in its codec\-dependent argument\. Composition of monotone non\-decreasing maps is monotone non\-decreasing, which establishes the partial\-derivative sign ofUU\. ∎

Three structural consequences of[theorem˜2](https://arxiv.org/html/2605.26339#Thmtheorem2)are testable\.

- \(C1\)*Codec dependence is linear inDBD\_\{B\}\.*At fixed model, the upper bound on𝔼​DKL\\mathbb\{E\}\\,D\_\{\\mathrm\{KL\}\}scales linearly inDBD\_\{B\}\. For a2B2^\{B\}\-point 2D Lloyd codebook at high rate on the unit circular Gaussian source, Zador’s formula\(Zador,[1982](https://arxiv.org/html/2605.26339#bib.bib11); Gersho and Gray,[1991](https://arxiv.org/html/2605.26339#bib.bib10)\)givesDB∼2​G​\(2\)​2−BD\_\{B\}\\sim 2\\,G\(2\)\\,2^\{\-B\}with hexagonal\-lattice constantG​\(2\)=5/\(36​3\)≈0\.0802G\(2\)=5/\(36\\sqrt\{3\}\)\\approx 0\.0802, so per additional bit*per pair*,ln⁡𝔼​DKL\\ln\\mathbb\{E\}\\,D\_\{\\mathrm\{KL\}\}should decrease by≈ln⁡2≈0\.69\\approx\\ln 2\\approx 0\.69; per additional bit*per weight*\(two pair\-bits each\), the predicted decrement is≈2​ln⁡2≈1\.39\\approx 2\\ln 2\\approx 1\.39\(≈0\.60\\approx 0\.60dex\)\.
- \(C2\)*Architecture enters throughCWC\_\{W\}\.*The only architecture\-dependent factor in the envelope isCWC\_\{W\}, which suggests the cross\-model sensitivity gradient in[table˜1](https://arxiv.org/html/2605.26339#S5.T1)should trackCWC\_\{W\}\.
- \(C3\)*Activation\-aware scaling reduces effectiveDBD\_\{B\}\.*AWQ\-style per\-channel rescaling shrinks the effective Lloyd distortion by a multiplicative factor determined by the alignment betweenMxM\_\{x\}and the codec’s directional error \([proposition˜B\.2](https://arxiv.org/html/2605.26339#A2.Thmproposition2)\)\. The composite bound’s first\-order dependence onDBD\_\{B\}implies that this scaling enters the KL upper bound multiplicatively at the same order\.

### C\.3Empirical Validation

The composite upper bound implies one quantitative and two qualitative properties of the envelope; this subsection tests whether the realized𝔼​DKL\\mathbb\{E\}\\,D\_\{\\mathrm\{KL\}\}tracks them\.

##### \(C1\) Rate\-distortion slope\.

Reading the per\-method paired KL across bpw levels \([tables˜9](https://arxiv.org/html/2605.26339#A8.T9),[10](https://arxiv.org/html/2605.26339#A8.T10)and[11](https://arxiv.org/html/2605.26339#A8.T11)\), paired KL drops from∼10−1\\sim\\\!10^\{\-1\}at4\.04\.0bpw \(polar\) to∼3×10−3\\sim\\\!3\\\!\\times\\\!10^\{\-3\}at5\.55\.5bpw \(activation\-aware joint\-2D\) to∼10−3\\sim\\\!10^\{\-3\}at8\.18\.1bpw \(SmoothQuant W8A8\) — roughly two orders of magnitude over four bits of weight, or≈0\.50\\approx\\\!0\.50dex per bit per weight\. The upper\-bound prediction is2​log10⁡2≈0\.602\\log\_\{10\}2\\approx 0\.60dex per bit\. The measured slope is shallower than the asymptote, which matches the bitrates in this study \(44–88bpw\) being below the high\-rate regime whereD∼2​σ2​G​\(2\)​2−BD\\sim 2\\sigma^\{2\}G\(2\)\\,2^\{\-B\}becomes tight\.

##### \(C2\) Architecture\-fixed ranking\.

For each fixed architecture,[table˜1](https://arxiv.org/html/2605.26339#S5.T1)shows the same method ranking \(SmoothQuant W8A8≻\\succQAM\-W\-5\.5≻\\succcalibrated W4≻\\succuncalibrated W4 / polar\)\. The cross\-modelΔ\\DeltaPPL spread, by contrast, varies by an order of magnitude \([fig\.˜2](https://arxiv.org/html/2605.26339#S5.F2)\)\.[Theorem˜2](https://arxiv.org/html/2605.26339#Thmtheorem2)motivates this factorization: within\-model dependence is onDBD\_\{B\}alone; between\-model spread enters only throughCWC\_\{W\}\.

##### \(C3\) Activation\-aware shift\.

QAM\-W\-5\.5 \(5\.5 bpw, with activation\-aware scaling\) sits at lower KL than QAM\-W\-4 \(4\.0 bpw, no scaling\) in[tables˜9](https://arxiv.org/html/2605.26339#A8.T9),[10](https://arxiv.org/html/2605.26339#A8.T10)and[11](https://arxiv.org/html/2605.26339#A8.T11); the two methods differ both in bitrate and in scaling, so the comparison is not iso\-bpw, but the gap between them is largest on Qwen 2\.5 — the model with the largestλmax\(ℓ\)\\lambda\_\{\\max\}^\{\(\\ell\)\}values per[proposition˜B\.2](https://arxiv.org/html/2605.26339#A2.Thmproposition2)\.

##### Joint check\.

[Figure˜1](https://arxiv.org/html/2605.26339#S5.F1)\(main text\) plots measured paired KL against measuredΔ\\DeltaPPL% for every \(method, model\) row available in the study:2424rows from the three instruction\-tuned models,44from the Stage D Mistral extension \([section˜F\.1](https://arxiv.org/html/2605.26339#A6.SS1)\),66from the Llama\-2\-7B\-base frontier comparison \([section˜5\.2](https://arxiv.org/html/2605.26339#S5.SS2)\), and33from the Llama\-2\-13B\-base scale extension \([section˜F\.2](https://arxiv.org/html/2605.26339#A6.SS2)\) —3737rows total across six model variants spanning1\.11\.1B–1313B parameters and bitrates from22to8\.18\.1bpw\. The pooled Spearman correlation isρ=\+0\.99\\rho=\+0\.99\. The envelope being monotone does not prove the realized quantity is, so the observedρ\\rhois evidence that𝔼​DKL\\mathbb\{E\}\\,D\_\{\\mathrm\{KL\}\}tracks the envelope’s ranking on this dataset rather than a guarantee that it must\.

## Appendix DEmpirical Check of Proposition[B\.2](https://arxiv.org/html/2605.26339#A2.Thmproposition2)\(Assumption A1\)

[proposition˜B\.2](https://arxiv.org/html/2605.26339#A2.Thmproposition2)’s assumption \(A1\) is that the per\-coordinate squared quantization error in the*scaled*domain,𝔼​\[\(Es\)i​j2\]\\mathbb\{E\}\[\(E\_\{s\}\)\_\{ij\}^\{2\}\], is bounded by a constantci​jc\_\{ij\}that does not depend on the per\-channel scaling factorssjs\_\{j\}\. The proposition uses this to argue that activation\-aware scaling shrinks the activation\-weighted error from arj2r\_\{j\}^\{2\}channel weighting to arj2​\(1−α\)r\_\{j\}^\{2\(1\-\\alpha\)\}weighting\. The discussion at the end of[appendix˜B](https://arxiv.org/html/2605.26339#A2)flags A1 as the load\-bearing assumption and notes that the deployed quantizer satisfies it only approximately\.

This appendix tests A1 empirically\. The probe is onmodel\.layers\.16\.mlp\.down\_projof Mistral\-7B\-Instruct\-v0\.3 \(din=14,336d\_\{\\mathrm\{in\}\}=14\{,\}336channels\), and the codec is the same QAM\-W joint\-2D Lloyd codebook atB=11B=11used in the deployed QAM\-W\-5\.5 configuration\. For eachα∈\{0\.0,0\.3,0\.5,0\.8\}\\alpha\\in\\\{0\.0,0\.3,0\.5,0\.8\\\}, the per\-channel scaling factorsj∝rjαs\_\{j\}\\propto r\_\{j\}^\{\\alpha\}is computed \(with the geometric\-mean normalization and clamp range\[1/16,16\]\[1/16,16\]used at deployment\), the weight matrix is scaled bydiag⁡\(sj\)\\operatorname\{diag\}\(s\_\{j\}\), quantized, and the per\-channel mean\-squared scaled\-domain residual is measured ascj​\(α\)=𝔼i​\[\(Es\)i​j2\]c\_\{j\}\(\\alpha\)=\\mathbb\{E\}\_\{i\}\\,\[\(E\_\{s\}\)\_\{ij\}^\{2\}\]\.

![Refer to caption](https://arxiv.org/html/2605.26339v1/plots/day9/prop_aware_c_vs_alpha.png)Figure 4:Empirical check of[proposition˜B\.2](https://arxiv.org/html/2605.26339#A2.Thmproposition2)\(A1\)\. Left: per\-channelcjc\_\{j\}for eachα\\alphain the deployed grid, channels sorted by activation RMS \(highest\-RMS on the left\)\. The curves overlap closely through the bulk of the distribution and diverge at the high\-activation tail\. Right: median, p99, and max ofcjc\_\{j\}as a function ofα\\alphaacross all 14,336 input channels of the probeddown\_proj\. The median is essentially flat \(≤1\.3×\\leq\\\!1\.3\\timesspread across the grid\), p99 grows∼10×\\sim\\\!10\\times, and the maximum grows∼50×\\sim\\\!50\\timesfromα=0\\alpha=0toα=0\.8\\alpha=0\.8\. The deployedα=0\.3\\alpha=0\.3\(dotted vertical\) sits well below the regime where the worst\-case channel begins to dominate\.##### Reading\.

A1’s “bounded constant independent ofss” claim holds approximately for the typical channel: the mediancjc\_\{j\}varies by less than1\.3×1\.3\\timesacross the full\[0,0\.8\]\[0,0\.8\]grid, well inside the natural Lloyd\-rounding variance\. A1*fails*in the worst\-case tail: the maximumcjc\_\{j\}grows roughly50×50\\timesfromα=0\\alpha=0toα=0\.8\\alpha=0\.8, because row renormalization concentrates more energy into a few high\-magnitude channels asα\\alphagrows\. The deployedα=0\.3\\alpha=0\.3is conservative relative to this tail growth \(maxcjc\_\{j\}atα=0\.3\\alpha=0\.3is an order of magnitude smaller than atα=0\.8\\alpha=0\.8\), and the per\-channel clamp range\[1/16,16\]\[1/16,16\]is never reached at anyα\\alphain the tested grid \(clamp hit rate is 0% at all four points\)\. So the proposition’s design model remains a reasonable guide for the deployed range, with the caveat that its bound is not uniformly tight against the high\-RMS tail\.

##### Scope\.

The probe is on a single representative MLPdown\_projlayer of one model\. Whether the same50×50\\timestail\-growth pattern holds across layers and architectures is left as empirical follow\-up\.

## Appendix ELow\-Bit Operating Point Details

Table 5:QAM\-W\-3\.5 frontier\. PPL is WikiText\-2 stride; mean KL is paired against the BF16 reference \(4K scored tokens\)\. Bpw from each method’s manifest\.
Reader key:↓\\downarrowlower is better\. Shaded rows \(⋆\\star\) are QAM\-W \(ours\)\.
## Appendix FMLP \+ Self\-Attention and 13B\-Scale Extensions

### F\.1MLP \+ Self\-Attention Quantization Extension

The cross\-model study reports bpw over MLP weights only — attention projections and the LM head remain BF16\. The natural follow\-up: if QAM\-W and the strongest baselines are extended to also quantize the four self\-attention projections \(q\_proj,k\_proj,v\_proj,o\_proj\), does the bpw\-efficiency ranking persist?

The BF16 reference and the four strongest sub\-8 bpw configurations from[table˜11](https://arxiv.org/html/2605.26339#A8.T11)\(SmoothQuant W8A8, QAM\-W\-5\.5, GPTQ, AWQ\) are re\-run on Mistral\-7B\-Instruct\-v0\.3 with stageall\_linear, quantizing both MLP and self\-attention\. Everything else is unchanged: identical calibration corpus \(128 WikiText\-2 sequences for the linear\-codec baselines; per\-input\-channel RMS over the same 128 sequences for QAM\-W\-5\.5\), identical PPL / KL protocol, and the same six\-task harness panel\. The bpw column now reports a substantially larger fraction of the total model\.

Table 6:MLP \+ self\-attention quantization on Mistral\-7B\-Instruct\-v0\.3\.Bpwnow spans the full MLP and attention weight pool;lm\_headand embeddings remain BF16\. Rows sorted by ascendingΔ\\DeltaPPL%\.
Reader key:↓\\downarrowlower is better,↑\\uparrowhigher is better\. Shaded row \(⋆\\star\) is QAM\-W \(ours\)\.##### Ranking is preserved\.

The method ordering from[table˜1](https://arxiv.org/html/2605.26339#S5.T1)\(MLP only\) carries over: SmoothQuant W8A8≻\\succQAM\-W\-5\.5≻\\succGPTQ≻\\succAWQ\. The two low\-distortion methods stay within the BF16 envelope \(Δ\\DeltaPPL≤0\.15%\\leq 0\.15\\%\) and SmoothQuant remains tied with BF16 on the harness average\.

##### Where the extra cost goes\.

Comparing against the MLP\-only Mistral row of[table˜11](https://arxiv.org/html/2605.26339#A8.T11), the principal change is in the 4\-bit calibrated methods: GPTQ moves from\+0\.9%\+0\.9\\%to\+1\.43%\+1\.43\\%\(∼0\.5\\sim\\\!0\.5pp worse from quantizing attention\), and AWQ from\+1\.8%\+1\.8\\%to\+2\.88%\+2\.88\\%\(∼1\.1\\sim\\\!1\.1pp worse\)\. The two low\-distortion methods barely change\. Attention projections are slightly more sensitive than MLP projections under per\-channel scalar quantization, but the 2D joint codebook and 8\-bit per\-channel calibration both absorb this extra sensitivity without loss\.

##### Implication for bpw accounting\.

The bpw column of[table˜6](https://arxiv.org/html/2605.26339#A6.T6)is the more representative comparison for memory\-bound deployment, since it spans the full weight pool that the inference runtime loads\. QAM\-W\-5\.5 retains its≈1\.5×\\approx\\\!1\.5\\timesbpw advantage over SmoothQuant when both are extended to all linear layers, and its quality gap to BF16 remains within the multi\-seed noise floor \([table˜8](https://arxiv.org/html/2605.26339#A7.T8)\)\.

### F\.2Scale Extension: Llama\-2\-13B\-base

The natural follow\-up question is whether the≈5\.5\\approx 5\.5bpw envelope holds at the next scale rung\. Base model is Llama\-2\-13B, accessed asNousResearch/Llama\-2\-13b\-hf\(sha4b54e9fd\.\.\.\)\. Protocol is identical to the rest of this paper: MLP\-only quantization, stride PPL atseq\_len=2048\\texttt\{seq\\\_len\}=2048,stride=1024\\texttt\{stride\}=1024, 16K scored tokens; paired KL on 4K scored tokens; six\-task harness atlimit=300\\texttt\{limit\}=300\.

Table 7:Llama\-2\-13B\-base, four configurations\. QAM\-W\-5\.5 lands at\+0\.25%\+0\.25\\%Δ\\DeltaPPL, in the same envelope as[table˜1](https://arxiv.org/html/2605.26339#S5.T1)and[table˜2](https://arxiv.org/html/2605.26339#S5.T2)\.
Reader key:↓\\downarrowlower is better,↑\\uparrowhigher is better\. Shaded row \(⋆\\star\) is QAM\-W \(ours\)\.##### The 5\.5 bpw envelope holds at 13B\.

QAM\-W\-5\.5 at 5\.5 bpw lands at\+0\.25%\+0\.25\\%Δ\\DeltaPPL with mean KL0\.0030\.003and a harness average0\.0030\.003below BF16 — inside the∼2\\sim\\\!2percentage\-point harness noise floor atlimit=300\. Combining across the study, the same codec on five models from four families spans\+0\.4%\+0\.4\\%\(TinyLlama\) /\+0\.3%\+0\.3\\%\(Qwen\-2\.5\-3B\) /\+0\.1%\+0\.1\\%\(Mistral\) /\+0\.29%\+0\.29\\%\(Llama\-2\-7B\-base\) /\+0\.25%\+0\.25\\%\(Llama\-2\-13B\-base\) — std0\.100\.10pp across the five\.

##### GPTQ W4A16 g128 is unusually weak at 13B\.

GPTQ at 4 bpw lands at\+5\.05%\+5\.05\\%on Llama\-2\-13B\-base — substantially worse than on the smaller panel \(TinyLlama\+2\.6%\+2\.6\\%, Qwen\+2\.9%\+2\.9\\%, Mistral\+0\.9%\+0\.9\\%\)\. This is consistent with GPTQ’s per\-channel Hessian\-aware optimization being sensitive to particular activation outlier patterns; the Llama\-2\-13B\-base appears to have an unusually hostile pattern at W4A16 g128\. The harness average for GPTQ \(0\.73630\.7363\) is*higher*than BF16’s0\.73040\.7304, which is inside the harness noise floor and should be read as a tie\.

##### SmoothQuant is effectively lossless at 13B\.

At≈8\.1\\approx 8\.1bpw SmoothQuant W8A8 \(eval mode,[section˜5](https://arxiv.org/html/2605.26339#S5)\) lands at\+0\.02%\+0\.02\\%Δ\\DeltaPPL and exactly matches the BF16 harness average\. QAM\-W\-5\.5 achieves comparable quality at≈32%\\approx 32\\%fewer weight bits\.

## Appendix GStride\-Offset Robustness of WikiText\-2 Perplexity

The single\-seedΔ\\DeltaPPL numbers in[table˜1](https://arxiv.org/html/2605.26339#S5.T1)are deterministic given the manifest’s weight hash and the corpus, but they sample only one stride\-window grid over the WikiText\-2 test split\. To characterize the noise floor of the evaluation itself, the two principal configurations — BF16 and QAM\-W\-5\.5 \(the lowest\-bpw method inside the BF16\-quality envelope on this panel\) — are re\-scored under three stride\-window starting offsets\{0,4000,8000\}\\\{0,4000,8000\\\}\. Each offset shifts the window grid by a different amount and produces a near\-disjoint sample of the corpus while preserving the 16K scored\-token budget\. Models and checkpoints are otherwise identical;qam\_awareis re\-trained from the deployed config so the off=0 row reproduces the single\-seed[table˜1](https://arxiv.org/html/2605.26339#S5.T1)value within numerical jitter\.

Table 8:Multi\-seed PPL for the two headline configurations across three stride\-window starting offsets on each model\. The absolute\-PPL std is dominated by corpus\-position variation \(different offsets land in regions of WikiText\-2 test with different intrinsic difficulty\); theΔ\\DeltaPPL% std is an order of magnitude smaller because that corpus\-position variation cancels between the two rows\. The within\-offsetΔ\\DeltaPPL% is the right quality measure for the codec\.
Reader key:↓\\downarrowlower is better\. Shaded rows \(⋆\\star\) are QAM\-W \(ours\)\.Two readings of[table˜8](https://arxiv.org/html/2605.26339#A7.T8)matter for how[table˜1](https://arxiv.org/html/2605.26339#S5.T1)should be interpreted\.

##### Absolute PPL is noisy;Δ\\DeltaPPL is not\.

The absolute PPL has std0\.110\.11–0\.390\.39across offsets — 1–6% of the mean — because different stride\-window starts land in different parts of WikiText\-2 test, which have different intrinsic difficulty\. But theΔ\\DeltaPPL% between BF16 and QAM\-W\-5\.5 at the*same*offset has std of only0\.030\.03–0\.080\.08percentage points, about an order of magnitude smaller\. The codec\-quality measurement factors out the corpus\-position variation, which is the right behaviour for a method comparison\.

##### MistralΔ\\DeltaPPL is at the boundary of stride noise\.

The single\-seed\+0\.1%\+0\.1\\%Δ\\DeltaPPL for Mistral in[table˜1](https://arxiv.org/html/2605.26339#S5.T1)sits inside the0\.18±0\.08%0\.18\\pm 0\.08\\%multi\-seed mean envelope: the codec degradation is real but at the same order of magnitude as the WikiText\-2 stride\-sampling noise floor on this model\. On TinyLlama \(\+0\.33±0\.03%\+0\.33\\pm 0\.03\\%\) and Qwen \(\+0\.26±0\.03%\+0\.26\\pm 0\.03\\%\) theΔ\\DeltaPPL is several standard deviations above zero and the relative ranking from[table˜1](https://arxiv.org/html/2605.26339#S5.T1)is robust\.

## Appendix HPer\-Model Detail Tables

Tables[tables˜9](https://arxiv.org/html/2605.26339#A8.T9),[10](https://arxiv.org/html/2605.26339#A8.T10)and[11](https://arxiv.org/html/2605.26339#A8.T11)expand the headline cross\-model summary in[table˜1](https://arxiv.org/html/2605.26339#S5.T1)into per\-architecture detail: bits\-per\-weight, absolute WikiText\-2 perplexity,Δ\\DeltaPPL% vs\. BF16, paired mean KL on 4K scored tokens, and the six\-task harness average\. Rows are sorted by ascendingΔ\\DeltaPPL%; the BF16 reference row is always first\. Throughout:↓\\downarrowlower is better,↑\\uparrowhigher is better\. Shaded rows \(⋆\\star\) are QAM\-W \(ours\)\.

Table 9:TinyLlama\-1\.1B\-Chat\. Bits\-per\-weight \(bpw, MLP\-only\), WikiText\-2 perplexity,Δ\\DeltaPPL% vs\. BF16, paired mean KL, and six\-task harness average\. Bitrate range4\.0064\.006–8\.1298\.129bpw \+ BF16 reference\. Rows are sorted by descending harness average within each model\. All task columns are accuracy \(↑\\uparrowhigher is better\); shaded rows \(⋆\\star\) are QAM\-W \(ours\)\.Table 10:Qwen2\.5\-3B\-Instruct\. Same columns as[table˜9](https://arxiv.org/html/2605.26339#A8.T9); bitrate range4\.0064\.006–8\.1298\.129bpw\.Table 11:Mistral\-7B\-Instruct\-v0\.3\. Same columns as[table˜9](https://arxiv.org/html/2605.26339#A8.T9); bitrate range4\.0034\.003–8\.1278\.127bpw\.
## Appendix IPer\-Task Harness Breakdowns

![Refer to caption](https://arxiv.org/html/2605.26339v1/plots/day8/harness_tasks_grid_v2.png)Figure 5:Six\-subplot grid \(PIQA, HellaSwag, COPA, RTE, OpenBookQA, LAMBADA\) with method on thexx\-axis and accuracy on theyy\-axis\. Dashed horizontal lines are the BF16 reference per model\. LAMBADA is the most quantization\-sensitive task; PIQA and COPA are nearly flat across methods\. Method ranking is stable across all six tasks within each model\.![Refer to caption](https://arxiv.org/html/2605.26339v1/plots/day7/pareto_harness.png)Figure 6:Bits\-per\-weight vs\. harness average across the six\-task panel, one panel per model\. The shape mirrors the PPL slope in[fig\.˜2](https://arxiv.org/html/2605.26339#S5.F2): the harness story confirms the WikiText\-2 story\.The headline cross\-model harness picture is summarised by the harness\-average column in[tables˜9](https://arxiv.org/html/2605.26339#A8.T9),[10](https://arxiv.org/html/2605.26339#A8.T10)and[11](https://arxiv.org/html/2605.26339#A8.T11)and visualised by[fig\.˜5](https://arxiv.org/html/2605.26339#A9.F5)\. For completeness, this appendix records the per\-task accuracy on each of the six lm\-eval\-harness tasks \(PIQA, HellaSwag, COPA, RTE, OpenBookQA, LAMBADA\-OpenAI\)\.

Table 12:TinyLlama\-1\.1B\-Chat per\-task harness accuracy atlimit=500\.Table 13:Qwen2\.5\-3B\-Instruct per\-task harness accuracy atlimit=300\.Table 14:Mistral\-7B\-Instruct\-v0\.3 per\-task harness accuracy atlimit=300\.
## Appendix JReproducibility Checklist

This appendix maps the present paper against the NeurIPS reproducibility checklist and provides the exact pointers a third party would need to re\-run any reported row\.

### J\.1Data and Calibration

##### Evaluation corpus\.

WikiText\-2 raw test split, obtained via the HuggingFacedatasetslibrary; the version is pinned by the SHA\-256 of the local cached\*\.txtfile, recorded in each eval JSON’s corpus metadata \(wikitext\-2\-raw\-v1\.test\.meta\.json\)\.

##### Calibration corpus\.

GPTQ, AWQ, AutoRound, SmoothQuant: 128 sequences of length 2048 from the WikiText\-2 train split\. QAM\-W activation\-aware variants compute per\-input\-channel RMS over 16 sequences of length 512 from the same WikiText\-2 split, via forward\-pass capture \([section˜5](https://arxiv.org/html/2605.26339#S5),§\\S“Reproducibility details”\)\. QAM\-W weight\-side pair\-scale calibration uses up to 1024 unit\-normalized rows per matrix and no activations\.

##### Downstream task panel\.

lm\-evaluation\-harness at versions0\.4\.110\.4\.11and0\.4\.120\.4\.12across the study \(per\-run version recorded in each eval JSON’sharness\.framework\_version\), no chat template, zero\-shot\. Batch size1616for the cross\-model main study,88for the Stage D extension and Llama\-2\-7B frontier comparison, and44for the Llama\-2\-13B scale extension \(per\-run value inharness\.batch\_size\)\. Six tasks \([section˜5\.3\.2](https://arxiv.org/html/2605.26339#S5.SS3.SSS2)\); MMLU \(57 subjects\) added in[section˜5\.3\.2](https://arxiv.org/html/2605.26339#S5.SS3.SSS2.Px1)for the two principal configurations\.

### J\.2Model Provenance

The three instruction\-tuned models and the two Llama\-2 base models used in this paper are pinned to specific HuggingFace commit SHAs in[section˜5](https://arxiv.org/html/2605.26339#S5),§\\S“Reproducibility details”\. The two additional pins introduced by the Llama\-2 sections:

- •Llama\-2\-7B\-base:NousResearch/Llama\-2\-7b\-hfat8efe6c9b93655b934e27bd9981e3ec13e55aee9d\.
- •Llama\-2\-13B\-base:NousResearch/Llama\-2\-13b\-hfat4b54e9fdd2ebd8db0901d37b98d3a53bbfaa4503\.

The published QTIP and AQLM baselines \([section˜5\.2](https://arxiv.org/html/2605.26339#S5.SS2)\) are pinned via their HuggingFace repo paths:relaxml/Llama\-2\-7b\-QTIP\-4Bit\(03aad3088415e5ec98b1e35924719b11e29c7e45\),relaxml/Llama\-2\-7b\-QTIP\-2Bit, andISTA\-DASLab/Llama\-2\-7b\-AQLM\-2Bit\-1x16\-hf\(9746e511b4eecf972af0a2bb3ebd93f2c5f84ecc\)\.

### J\.3Software Versions

- •Rust quantization binary \(qam\-llm\-bench\): builds are reproducible fromcargo build \-\-releaseagainst the includedCargo\.toml\(see[section˜J\.7](https://arxiv.org/html/2605.26339#A10.SS7)\)\.
- •Python evaluation pipeline \(qambench\): version0\.1\.0, installed viapip install \-e qambench\[eval\]\.
- •transformersversion:4\.45\.xfor the main run;aqlmversion installed viapip install aqlmresolves to the latest compatible release at run time\.
- •lm\-evaluation\-harness: versions0\.4\.110\.4\.11and0\.4\.120\.4\.12across the study \(drift reflects upstream package updates between the cross\-model main study and the later Llama\-2 extensions; the exact version is recorded per\-row inharness\.framework\_version\)\.
- •torch:2\.4\.0\+cu121for the QTIP dequant bridge \([section˜5\.2](https://arxiv.org/html/2605.26339#S5.SS2)\); the rest of the pipeline runs against whatever torch is installed in the active venv\.
- •quip\-sharp / qtip\-kernels: built from theCornell\-RelaxML/qtiprepository at thetrain\-fixWforward path; thescripts/day10b\_qtip\_dequant\.pyscript in the companion repo dequantizes QTIP checkpoints to BF16 via the identity\-matrix probe described in[section˜5\.2](https://arxiv.org/html/2605.26339#S5.SS2),§\\S“Setup”\.

### J\.4Hardware

Every row in the paper was produced on a single NVIDIA A100 80 GB on a rented Vast\.ai host\. The total GPU time consumed across the study — including cross\-model main run, Stage D, multi\-seed, per\-layer diagnostic, prop:aware A1 empirical check, MMLU, Llama\-2\-7B frontier comparison \(including QTIP dequant\), and Llama\-2\-13B scale extension — is approximately2020A100\-hours\. The total Vast\.ai cost at the host’s hourly rate was approximately USD 34\.

### J\.5Per\-Method Hyperparameters

Reported in[section˜5](https://arxiv.org/html/2605.26339#S5)§\\S“Configurations” and[table˜7](https://arxiv.org/html/2605.26339#A6.T7),[table˜2](https://arxiv.org/html/2605.26339#S5.T2),[table˜1](https://arxiv.org/html/2605.26339#S5.T1), etc\., per method\. Concretely:

- •RTN / GPTQ / AWQ / AutoRound:W​4​A​16W4A16group sizeg=128g=128, symmetric, f16 per\-group scales\. AutoRound runs 200 SignSGD iterations\.
- •SmoothQuant:W​8​A​8W8A8, smoothing strengthαsq=0\.5\\alpha\_\{\\mathrm\{sq\}\}=0\.5, 128 calibration sequences\.
- •QAM\-W: see[section˜3](https://arxiv.org/html/2605.26339#S3)\. Block\-Hadamard block sizeb=min⁡\(2⌊log2⁡din⌋,1024\)b=\\min\(2^\{\\lfloor\\log\_\{2\}d\_\{\\mathrm\{in\}\}\\rfloor\},1024\)\. Activation\-aware exponentα=0\.3\\alpha=0\.3, clamped to\[1/16,16\]\[1/16,16\], geometric\-mean normalized to 1 \([appendix˜B](https://arxiv.org/html/2605.26339#A2)\)\.
- •AQLM 2\-bit 1×\\times16: published checkpoint;in\_group\_size=8\\texttt\{in\\\_group\\\_size\}=8,nbits\_per\_codebook=16\\texttt\{nbits\\\_per\\\_codebook\}=16,num\_codebooks=1\\texttt\{num\\\_codebooks\}=1\.
- •QTIP\-4Bit / QTIP\-2Bit: published checkpoint; trellisL=16L=16,K∈\{4,2\}K\\in\\\{4,2\\\},V=2V=2\(per QTIP authors’ release\)\.

### J\.6Determinism

Per\-row PPL and KL are deterministic given the manifest’sweights\_sha256hash, the corpus SHA\-256, the eval\-timetorch\.bfloat16dtype, and the harness version\. TheΔ\\DeltaPPL% spread across stride\-window starting offsets is reported in[table˜8](https://arxiv.org/html/2605.26339#A7.T8)on three offsets\{0,4000,8000\}\\\{0,4000,8000\\\}for two headline configurations on each instruction\-tuned model\. TheΔ\\DeltaPPL% std observed there is0\.030\.03–0\.080\.08percentage points, which is the noise floor for the cross\-model rows\.

### J\.7Code Release

- •The Rustqam\-codeclibrary andqam\-llm\-benchCLI binary\.
- •The Pythonqambenchpackage \(loaders, perplexity, KL, lm\-evaluation\-harness adapter\)\.
- •Per\-experiment scripts underscripts/that exactly reproduce each results directory:run\_cross\_model\_main\.sh\(cross\-model main study\),run\_mlp\_attention\_mistral\.sh\(Stage D Mistral\),run\_multiseed\_and\_lowbit\_fill\.sh\(multi\-seed \+ 3\.5 bpw Mistral fill\),analyze\_layer\_output\_identity\.py\(Prop\.[B\.1](https://arxiv.org/html/2605.26339#A2.Thmproposition1)anchor\),run\_mmlu\_panel\.sh\(MMLU\),check\_activation\_aware\_assumption\.py\(Prop\. 1 A1 check\),run\_llama2\_7b\_frontier\.sh\(Llama\-2\-7B \+ AQLM frontier\),dequant\_qtip\_checkpoint\.py\(QTIP loader bridge\),run\_llama2\_13b\_scale\.sh\(Llama\-2\-13B scale\)\.
- •Reduce scripts that render the LaTeX rows of each table from the raw eval JSONs:reduce\_results\_to\_tables\.py\(Stage D, multi\-seed, low\-bit fill\)\.
- •Plot scripts inscripts/make\_plots\.pyandscripts/plot\_kl\_vs\_dppl\.py\.
- •ADockerfileandMakefile\(make run\) that build the full Rust \+ Python environment for one\-command end\-to\-end reproduction\.

The repository is provided under a proprietary, all\-rights\-reserved license \(seeLICENSEin the companion repo\)\.