ShamAN-Q: Shampoo Augmented NanoQuant for Sub-1-bit LLM Weights

# ShamAN-Q: Shampoo Augmented NanoQuant for Sub-1-bit LLM Weights
Source: [https://arxiv.org/html/2609.38521](https://arxiv.org/html/2609.38521)
###### Abstract

We introduce ShamAN\-Q, a sub\-1\-bit post\-training quantization method that extends NanoQuant by replacing each its diagonal reconstruction geometry with a tractable dense curvature metric, using a general paradigm popularized by the Shampoo optimizer\. For each linear weight, ShamAN\-Q fits a Kronecker product to the empirical Fisher information matrix of a small calibration set by Kullback–Leibler minimization, forming a Mahalanobis reconstruction loss from the result\. The continuous ADMM updates from NanoQuant become solutions to Sylvester equations, while its discrete projection and deployment format remain unchanged\. Because the curvature is local to a given set of weights, ShamAN\-Q re\-measures the input curvature statistic for each layer immediately before layer factorization, periodically refreshing all statistics on the partially quantized model\. ShamAN\-Q also redistributes the uniform rank from NanoQuant across layers at the same total number of bits\. On Qwen3\-Base, ShamAN\-Q lowers WikiText\-2 perplexity at≈\\approx1 bpw from 27\.56 to 22\.96 \(0\.6B\), 19\.21 to 16\.72 \(1\.7B\), and 14\.29 to 13\.80 \(4B\) while matching or improving zero\-shot accuracy on the Eleuther LM Evaluation Harness\. On 0\.6B, ShamAN\-Q at≈\\approx0\.8 bpw matches the published perplexity of NanoQuant at≈\\approx1\.0 bpw\.

## 1Introduction

Figure 1:WikiText\-2 perplexity \(PPL\) \(left, lower is better\) and zero\-shot mean on reasoning tasks in Eleuther LM Evaluation Harness \(right, higher is better\) vs\. Qwen3\-Base size at a 1\.0 bpw budget\. ShamAN\-Q matches or improves on NanoQuant in both PPL and zero\-shot mean at every size\. All points are from Table[1](https://arxiv.org/html/2609.38521#S5.T1); only reported results are plotted\.The energy consumption of language models is an omnipresent, growing concern, affecting all scales from those of frontier data centers down to small edge devices\. Awareness of this cost grows alongside demand for data sovereignty, independence from model providers, and fast local intelligence\. Beyond the cost of the computation itself, an often overlooked source of language model energy cost is the repeated movement of data\. For causal language model inference, processing a prompt in parallel during prefill is often compute\-bound\. A long prompt produces a large effective batch sizeBB, and assumingn×nn\\times nmatrices, the resulting matrix–matrix multiplication movesO⁡\(n2\)O\(n^\{2\}\)data forO⁡\(B​n2\)O\(Bn^\{2\}\)computation\. Decoding, by contrast, is often bandwidth\-bound, since each generated token requires transferring the entire active set of parameters from memory to the compute cores, before counting any host–device or device–device transfers required by models that do not fit on a single accelerator\. In the extreme, batch sizeB=1B=1reduces to matrix–vector multiplication withO⁡\(n2\)O\(n^\{2\}\)data movement for onlyO⁡\(n2\)O\(n^\{2\}\)computation\.

Shrinking the stored weights therefore directly attacks the dominant cost of decoding, and large language models are increasingly deployed under memory and bandwidth budgets that their weights alone exceed\. Post\-training quantization \(PTQ\) to 4 bits per weight \(bpw\) is now well established\([Frantar et al\., 2023](https://arxiv.org/html/2609.38521#bib.bib2)\), and even 2 bpw is maturing\([Lin et al\., 2024](https://arxiv.org/html/2609.38521#bib.bib3)\), but the*sub\-1\-bit*regime requires a structural change, since a scalar codebook cannot spend less than one bit on every weight\. NanoQuant\([Chong et al\., 2026](https://arxiv.org/html/2609.38521#bib.bib1)\)instead represents each decoder linear layer by two low\-rank binary factors and two short FP16 scale vectors \(Eq\.[1](https://arxiv.org/html/2609.38521#S3.E1)\)\. By doing so, NanoQuant changes the dominant storage from one bit per matrix entry tor⁡\(m\+n\)r\(m\+n\)bits per layer at factor rankr≫16r\\gg 16\. At this compression level, nearly every weight is perturbed, and as a result accuracy depends not only on the size of the reconstruction error but also on which input and output directions absorb it\.

NanoQuant handles unequal channel scales through diagonal preconditioning, after which its optimization process minimizes a Frobenius reconstruction loss\. This diagonal preconditioning acts as a proxy for second\-order information specific to each channel, but it does not capture second\-order relationships between channels\. ShamAN\-Q extends NanoQuant by replacing the channel\-wise scaling with a dense metric that captures these channel correlations\. In this regard, ShamAN\-Q is to NanoQuant what Shampoo\([Gupta et al\., 2018](https://arxiv.org/html/2609.38521#bib.bib8)\)is to diagonal AdaGrad\([Duchi et al\., 2011](https://arxiv.org/html/2609.38521#bib.bib7)\)\. One\-sided Hessian\-aware PTQ methods such as GPTQ\([Frantar et al\., 2023](https://arxiv.org/html/2609.38521#bib.bib2)\)model input sensitivity while treating all output directions alike\. The Fisher Information matrix of a weight matrix captures both, but itsm​n×m​nmn\\times mnsize is prohibitive for anm×nm\\times nweight\. Approximating it with ann×nn\\times ninput statistic and anm×mm\\times moutput statistic makes this information tractable\([Martens and Grosse, 2015](https://arxiv.org/html/2609.38521#bib.bib4)\)\(§[3\.2](https://arxiv.org/html/2609.38521#S3.SS2)\)\.

ShamAN\-Q estimates dense curvature statistics from a small calibration set while limiting the influence of massive\-activation tokens\. As quantization changes the model, these statistics are regularized and refreshed; they simultaneously serve to guide low\-rank reconstruction and rank allocation at NanoQuant’s total bit budget, thus preserving its binary representation and deployment format\. At this budget, ShamAN\-Q lowers WikiText\-2 perplexity on Qwen3\-Base from 27\.56 to 22\.96 at 0\.6B, 19\.21 to 16\.72 at 1\.7B, and 14\.29 to 13\.80 at 4B while matching or improving zero\-shot accuracy on the Eleuther LM Eval Harness; at 0\.8 bpw, ShamAN\-Q at 0\.6B matches NanoQuant’s published 1\.0 bpw perplexity \(§[5](https://arxiv.org/html/2609.38521#S5)\)\. We now describe the primary contributions of this paper:

- •A dense two\-sided reconstruction metric extension of NanoQuant for sub\-1\-bit PTQ: KL\-Shampoo fit, and a fresh input statistic with periodic full refresh \(§[4\.1](https://arxiv.org/html/2609.38521#S4.SS1), §[4\.3](https://arxiv.org/html/2609.38521#S4.SS3)\), resulting in a modified ADMM whose continuous updates are solutions to Sylvester equations\. Other components, such as the Sign\-Value Independent Decomposition \(SVID\) projection\([Pouransari et al\., 2020](https://arxiv.org/html/2609.38521#bib.bib23);[Xu et al\., 2020](https://arxiv.org/html/2609.38521#bib.bib22)\), dual updates, scale extraction, and packed format structures remain unchanged \(§[4\.3](https://arxiv.org/html/2609.38521#S4.SS3)\)\.
- •A redistribution of NanoQuant’s uniform rank budget at bit parity from a combination of depth, layer type, and a measurement of reconstruction loss for each layer \(§[4\.2](https://arxiv.org/html/2609.38521#S4.SS2)\)\.
- •Consistent decreases in perplexity paired with increases in zero\-shot reasoning performance over NanoQuant on Qwen3\-Base from 0\.6B to 14B at identical≲1\\lesssim 1\-bit budgets, attributed to each component \(§[5](https://arxiv.org/html/2609.38521#S5)\)\.

## 2Related Work

Binary and Sub\-1\-bit Quantization of LLMs\.Weight\-only PTQ is already an established paradigm at 4 and 2 bpw\([Frantar et al\., 2023](https://arxiv.org/html/2609.38521#bib.bib2);[Lin et al\., 2024](https://arxiv.org/html/2609.38521#bib.bib3)\)\. Pushing further, binary methods either train with quantization\-aware objectives, as in the cases of BitNet\([Wang et al\., 2023](https://arxiv.org/html/2609.38521#bib.bib16)\)and OneBit\([Xu et al\., 2024](https://arxiv.org/html/2609.38521#bib.bib17)\), or partially binarize and keep salient weights at higher precision, as in the case of PB\-LLM\([Shang et al\., 2023](https://arxiv.org/html/2609.38521#bib.bib18)\);[Tu et al\. \(2026\)](https://arxiv.org/html/2609.38521#bib.bib12), in contrast, revisits 1\-bit optimization from pre\-trained weights\. Generally, in\-place binarization with full\-precision scales is bounded below by one bit per weight, and grouping metadata pushes the effective rate higher\. NanoQuant\([Chong et al\., 2026](https://arxiv.org/html/2609.38521#bib.bib1)\)breaks this bound in the PTQ setting by factorizing each weight into two low\-rank binary matrices and two scale vectors, initialized by a latent\-binary ADMM under diagonal preconditioning and refined block by block \(§[3\.1](https://arxiv.org/html/2609.38521#S3.SS1)\)\.

Hessian\-aware PTQ and Kronecker Curvature Statistics\.GPTQ\([Frantar et al\., 2023](https://arxiv.org/html/2609.38521#bib.bib2)\)demonstrates a form of Hessian\-aware PTQ by weighting the reconstruction loss with the input second moment, a one\-sided Hessian surrogate that models input sensitivity while treating all output directions alike\. The second order optimizers K\-FAC\([Martens and Grosse, 2015](https://arxiv.org/html/2609.38521#bib.bib4)\)and Shampoo\([Gupta et al\., 2018](https://arxiv.org/html/2609.38521#bib.bib8)\)approximate the layer Fisher by a Kronecker product of input and output statistics, and BaKron\([Birnick and Saab, 2026](https://arxiv.org/html/2609.38521#bib.bib11)\)applies similar Kronecker\-structured Hessians to weight quantization\. Two estimators fit the Kronecker product to the empirical Fisher directly, the nearest Kronecker product \(NKP\) in Frobenius norm\([Mei et al\., 2023](https://arxiv.org/html/2609.38521#bib.bib5);[Morwani et al\., 2025](https://arxiv.org/html/2609.38521#bib.bib6)\)and the matrix\-normal maximum\-likelihood fit of KL\-Shampoo\([Lin et al\., 2026](https://arxiv.org/html/2609.38521#bib.bib9);[Sun and Wei, 2026](https://arxiv.org/html/2609.38521#bib.bib10)\)\. §[3\.2](https://arxiv.org/html/2609.38521#S3.SS2)presents both, and in ShamAN\-Q we adopt the KL fit because of how it weights massive\-activation tokens\([Sun et al\., 2024](https://arxiv.org/html/2609.38521#bib.bib21)\)\.

ADMM for Structured Compression\.The alternating direction method of multipliers\([Boyd et al\., 2011](https://arxiv.org/html/2609.38521#bib.bib14)\)suits objectives that pair a continuous reconstruction loss with a hard discrete constraint\. NanoQuant introduces LB\-ADMM, which decouples the low\-rank reconstruction from the binary structure by using SVID\-projected consensus variables\. ShamAN\-Q retains this effective decoupling and its Euclidean consensus penalties, so that only the continuous subproblems change\. These subproblems change from right\-sided normal equations \(Eq\.[5](https://arxiv.org/html/2609.38521#S3.E5)\) to two\-sided Sylvester equations\.

## 3Background

### 3\.1NanoQuant

Representation and bit cost\.NanoQuant\([Chong et al\., 2026](https://arxiv.org/html/2609.38521#bib.bib1)\)deploys each linear layer, with weight𝐖∈ℝm×n\\mathbf\{W\}\\in\\mathbb\{R\}^\{m\\times n\}and factor rankrr, in the two\-scale form

𝐖^=Diag⁡\(𝐬1\)​𝐔±1​𝐕±1⊤​Diag⁡\(𝐬2\),𝐔±1∈\{±1\}m×r,𝐕±1∈\{±1\}n×r,\\widehat\{\\mathbf\{W\}\}=\\operatorname\{Diag\}\(\\mathbf\{s\}\_\{1\}\)\\,\\mathbf\{U\}\_\{\\pm 1\}\\,\\mathbf\{V\}\_\{\\pm 1\}^\{\\top\}\\,\\operatorname\{Diag\}\(\\mathbf\{s\}\_\{2\}\),\\qquad\\mathbf\{U\}\_\{\\pm 1\}\\in\\\{\\pm 1\\\}^\{m\\times r\},\\quad\\mathbf\{V\}\_\{\\pm 1\}\\in\\\{\\pm 1\\\}^\{n\\times r\},\(1\)using FP16 scale vectors of lengthsmmandnn\. Since the signs requirer⁡\(m\+n\)r\(m\+n\)bits, and the scales16​\(m\+n\)16\(m\+n\)bits, a rank\-rrlayer costs

bits⁡\(r\)=\(r\+16\)​\(m\+n\),BPW=\(r\+16\)​\(m\+n\)/m​n\.\\operatorname\{bits\}\(r\)=\(r\+16\)\(m\+n\),\\qquad\\operatorname\{BPW\}=\(r\+16\)\(m\+n\)/mn\.\(2\)
Pipeline\.Phase 1,*global calibration*, runs a calibration set through the full\-precision teacherℳ\\mathcal\{M\}and collects, per linear layer and tokentt, the layer input𝐱t∈ℝn\\mathbf\{x\}\_\{t\}\\in\\mathbb\{R\}^\{n\}and output gradient𝜹t∈ℝm\\bm\{\\delta\}\_\{t\}\\in\\mathbb\{R\}^\{m\}\. Phase 2,*block reconstruction*, processes decoder blocks in order\. Within a block, Step 1 \(TuneFP\) tunes the remaining full\-precision weights to reproduce the teacher’s block output from the already\-quantized prefix’s activations, and is repeated between layer factorizations\. Step 2 initializes each linear layer’s binary factors and scales by the LB\-ADMM below\. Step 3 \(TuneLatentSTE\) jointly tunes the block’s continuous latents𝒰,𝒱\\mathcal\{U\},\\mathcal\{V\}and scales𝐬1,𝐬2\\mathbf\{s\}\_\{1\},\\mathbf\{s\}\_\{2\}under a weighted MSE block loss with the straight\-through estimator\([Bengio et al\., 2013](https://arxiv.org/html/2609.38521#bib.bib13)\), after which𝐔±1=sign⁡\(𝒰\)\\mathbf\{U\}\_\{\\pm 1\}=\\sign\(\\mathcal\{U\}\)and𝐕±1=sign⁡\(𝒱\)\\mathbf\{V\}\_\{\\pm 1\}=\\sign\(\\mathcal\{V\}\)are packed\. Phase 3 \(TuneScalesKD\), tuning only the scale vectors of every layer, minimizes the forward KL divergence between the next\-token distributions of the teacher and quantization models\.

Diagonal preconditioning \(Step 2\-1\)\.NanoQuant performs one measurement of the per\-input\-feature activation second moments and the per\-output\-feature output gradient second moments for the full\-precision model, after robust token clipping, and shrinks them toward their mean,

𝐝in2=1N​∑t𝐱t⊙𝐱t,𝐝out2=1N​∑t𝜹t⊙𝜹t,𝐝2←\(1−γ\)​𝐝2\+γ​mean⁡\(𝐝2\)​1,\\mathbf\{d\}\_\{\\mathrm\{in\}\}^\{2\}=\\frac\{1\}\{N\}\\sum\_\{t\}\\mathbf\{x\}\_\{t\}\\odot\\mathbf\{x\}\_\{t\},\\qquad\\mathbf\{d\}\_\{\\mathrm\{out\}\}^\{2\}=\\frac\{1\}\{N\}\\sum\_\{t\}\\bm\{\\delta\}\_\{t\}\\odot\\bm\{\\delta\}\_\{t\},\\qquad\\mathbf\{d\}^\{2\}\\leftarrow\(1\-\\gamma\)\\,\\mathbf\{d\}^\{2\}\+\\gamma\\,\\operatorname\{mean\}\(\\mathbf\{d\}^\{2\}\)\\,\\mathbf\{1\},\(3\)withγ∈\[0,1\]\\gamma\\in\[0,1\]\. These shrunken moments then produce𝐃~in=Diag⁡\(𝐝in\)\\widetilde\{\\mathbf\{D\}\}\_\{\\mathrm\{in\}\}=\\operatorname\{Diag\}\(\\mathbf\{d\}\_\{\\mathrm\{in\}\}\),𝐃~out=Diag⁡\(𝐝out\)\\widetilde\{\\mathbf\{D\}\}\_\{\\mathrm\{out\}\}=\\operatorname\{Diag\}\(\\mathbf\{d\}\_\{\\mathrm\{out\}\}\), the transformed target𝐖~=𝐃~out​𝐖​𝐃~in\\widetilde\{\\mathbf\{W\}\}=\\widetilde\{\\mathbf\{D\}\}\_\{\\mathrm\{out\}\}\\mathbf\{W\}\\widetilde\{\\mathbf\{D\}\}\_\{\\mathrm\{in\}\}, and the loss∥𝐄~∥F2=tr⁡\(𝐄⊤​𝐃~out2​𝐄​𝐃~in2\)\\lVert\\widetilde\{\\mathbf\{E\}\}\\rVert\_\{F\}^\{2\}=\\operatorname\{tr\}\\big\(\\mathbf\{E\}^\{\\top\}\\widetilde\{\\mathbf\{D\}\}\_\{\\mathrm\{out\}\}^\{2\}\\mathbf\{E\}\\widetilde\{\\mathbf\{D\}\}\_\{\\mathrm\{in\}\}^\{2\}\\big\)where𝐄=𝐖−𝐖^\\mathbf\{E\}=\\mathbf\{W\}\-\\widehat\{\\mathbf\{W\}\}and𝐄~=𝐃~out​𝐄​𝐃~in\\widetilde\{\\mathbf\{E\}\}=\\widetilde\{\\mathbf\{D\}\}\_\{\\mathrm\{out\}\}\\mathbf\{E\}\\widetilde\{\\mathbf\{D\}\}\_\{\\mathrm\{in\}\}\. This loss comprises the two\-sided metric of §[3\.2](https://arxiv.org/html/2609.38521#S3.SS2)with diagonal statistics\.

LB\-ADMM \(Step 2\-2\)\.The continuous optimization variables are𝐔∈ℝm×r\\mathbf\{U\}\\in\\mathbb\{R\}^\{m\\times r\}and𝐕∈ℝn×r\\mathbf\{V\}\\in\\mathbb\{R\}^\{n\\times r\}, and they correspond with consensus variables𝐙U\\mathbf\{Z\}\_\{U\}and𝐙V\\mathbf\{Z\}\_\{V\}, which are restricted to the SVID family𝒞d,r=\{Diag\(𝐚\)𝐒Diag\(𝐛\):𝐚∈ℝ\+d,𝐛∈ℝ\+r,𝐒∈\{±1\}d×r\}\\mathcal\{C\}\_\{d,r\}=\\\{\\operatorname\{Diag\}\(\\mathbf\{a\}\)\\mathbf\{S\}\\operatorname\{Diag\}\(\\mathbf\{b\}\):\\mathbf\{a\}\\in\\mathbb\{R\}\_\{\+\}^\{d\},\\ \\mathbf\{b\}\\in\\mathbb\{R\}\_\{\+\}^\{r\},\\ \\mathbf\{S\}\\in\\\{\\pm 1\\\}^\{d\\times r\}\\\}\. NanoQuant then solves

min𝐔,𝐕,𝐙U,𝐙V𝐔=𝐙U,𝐕=𝐙V⁡12​∥𝐖~−𝐔𝐕⊤∥F2\+λ2​\(∥𝐔∥F2\+∥𝐕∥F2\)\+ℐ𝒞m,r​\(𝐙U\)\+ℐ𝒞n,r​\(𝐙V\)\\min\_\{\\begin\{subarray\}\{c\}\\mathbf\{U\},\\mathbf\{V\},\\mathbf\{Z\}\_\{U\},\\mathbf\{Z\}\_\{V\}\\\\ \\mathbf\{U\}=\\mathbf\{Z\}\_\{U\},\\ \\mathbf\{V\}=\\mathbf\{Z\}\_\{V\}\\end\{subarray\}\}\\;\\frac\{1\}\{2\}\\lVert\\widetilde\{\\mathbf\{W\}\}\-\\mathbf\{U\}\\mathbf\{V\}^\{\\top\}\\rVert\_\{F\}^\{2\}\+\\frac\{\\lambda\}\{2\}\\left\(\\lVert\\mathbf\{U\}\\rVert\_\{F\}^\{2\}\+\\lVert\\mathbf\{V\}\\rVert\_\{F\}^\{2\}\\right\)\+\\mathcal\{I\}\_\{\\mathcal\{C\}\_\{m,r\}\}\(\\mathbf\{Z\}\_\{U\}\)\+\\mathcal\{I\}\_\{\\mathcal\{C\}\_\{n,r\}\}\(\\mathbf\{Z\}\_\{V\}\)\(4\)by a scaled\-dual ADMM, with ridgeλ\\lambdaand a penaltyρk\\rho\_\{k\}that grows linearly overKKiterations\. Takingαk=ρk\+λ\\alpha\_\{k\}=\\rho\_\{k\}\+\\lambda, with the consensus target𝐂Uk=𝐙Uk−𝚲Uk\\mathbf\{C\}\_\{U\}^\{k\}=\\mathbf\{Z\}\_\{U\}^\{k\}\-\\bm\{\\Lambda\}\_\{U\}^\{k\}being formed using the scaled dual𝚲Uk\\bm\{\\Lambda\}\_\{U\}^\{k\}, the𝐔\\mathbf\{U\}update is the right\-sidedr×rr\\times rnormal equation

𝐔k\+1​\(\(𝐕k\)⊤​𝐕k\+αk​𝐈r\)=𝐖~​𝐕k\+ρk​𝐂Uk,\\mathbf\{U\}^\{k\+1\}\\big\(\(\\mathbf\{V\}^\{k\}\)^\{\\top\}\\mathbf\{V\}^\{k\}\+\\alpha\_\{k\}\\mathbf\{I\}\_\{r\}\\big\)=\\widetilde\{\\mathbf\{W\}\}\\mathbf\{V\}^\{k\}\+\\rho\_\{k\}\\mathbf\{C\}\_\{U\}^\{k\},\(5\)while the𝐕\\mathbf\{V\}update is analogous\. As shown for𝐙U\\mathbf\{Z\}\_\{U\}, the consensus variables are projected according to𝐙Uk\+1=SVID​\(𝐔k\+1\+𝚲Uk\)\\mathbf\{Z\}\_\{U\}^\{k\+1\}=\\textsc\{SVID\}\(\\mathbf\{U\}^\{k\+1\}\+\\bm\{\\Lambda\}\_\{U\}^\{k\}\), where SVID takes the sign of its argument and the three largest singular values of its absolute value, and the scaled duals are updated as𝚲Uk\+1=𝚲Uk\+𝐔k\+1−𝐙Uk\+1\\bm\{\\Lambda\}\_\{U\}^\{k\+1\}=\\bm\{\\Lambda\}\_\{U\}^\{k\}\+\\mathbf\{U\}^\{k\+1\}\-\\mathbf\{Z\}\_\{U\}^\{k\+1\}\(Appendix[C](https://arxiv.org/html/2609.38521#A3)\)\.

Scale extraction \(Step 2\-3\)\.The final proxies𝐔K\+𝚲UK\\mathbf\{U\}^\{K\}\+\\bm\{\\Lambda\}\_\{U\}^\{K\}and𝐕K\+𝚲VK\\mathbf\{V\}^\{K\}\+\\bm\{\\Lambda\}\_\{V\}^\{K\}are magnitude\-balanced\. Their signs are become𝐔±1,𝐕±1\\mathbf\{U\}\_\{\\pm 1\},\\mathbf\{V\}\_\{\\pm 1\}, their row\-wise mean absolute values give the scales, and the inverse of the diagonal maps return the scales to the original coordinates \(Appendix[C](https://arxiv.org/html/2609.38521#A3)\)\.

### 3\.2Kronecker Curvature Statistics: Shampoo and KL\-Shampoo

Empirical Fisher and the two\-sided metric\.For calibration tokentt, the weight gradient is the rank\-one matrix𝜹t​𝐱t⊤\\bm\{\\delta\}\_\{t\}\\mathbf\{x\}\_\{t\}^\{\\top\}, and the empirical layer Fisher information, in the original coordinates ofvec⁡\(𝐖\)\\operatorname\{vec\}\(\\mathbf\{W\}\), is given by𝐇^cal=∑tvec⁡\(𝜹t​𝐱t⊤\)​vec⁡\(𝜹t​𝐱t⊤\)⊤∈ℝm​n×m​n\\widehat\{\\mathbf\{H\}\}\_\{\\mathrm\{cal\}\}=\\sum\_\{t\}\\operatorname\{vec\}\(\\bm\{\\delta\}\_\{t\}\\mathbf\{x\}\_\{t\}^\{\\top\}\)\\operatorname\{vec\}\(\\bm\{\\delta\}\_\{t\}\\mathbf\{x\}\_\{t\}^\{\\top\}\)^\{\\top\}\\in\\mathbb\{R\}^\{mn\\times mn\}\. A Kronecker approximation can estimate this matrix as𝐇^cal≈𝐀⊗𝐆\\widehat\{\\mathbf\{H\}\}\_\{\\mathrm\{cal\}\}\\approx\\mathbf\{A\}\\otimes\\mathbf\{G\}using an input\-side statistic𝐀∈ℝn×n\\mathbf\{A\}\\in\\mathbb\{R\}^\{n\\times n\}and an output\-side statistic𝐆∈ℝm×m\\mathbf\{G\}\\in\\mathbb\{R\}^\{m\\times m\}\([Martens and Grosse, 2015](https://arxiv.org/html/2609.38521#bib.bib4);[Gupta et al\., 2018](https://arxiv.org/html/2609.38521#bib.bib8)\)\. We call these*curvature statistics*and reserve*factors*for the low\-rank weight matrices𝐔\\mathbf\{U\}and𝐕\\mathbf\{V\}\. Under column\-major vectorization,𝐀\\mathbf\{A\}and𝐆\\mathbf\{G\}act respectively on input and output coordinates, and the metric induced on an error𝐄\\mathbf\{E\}is

‖𝐄‖𝐆,𝐀2:⁣=tr\(𝐄⊤𝐆𝐄𝐀\)=vec\(𝐄\)⊤\(𝐀⊗𝐆\)vec\(𝐄\)\.\\left\\lVert\\mathbf\{E\}\\right\\rVert\_\{\\mathbf\{G\},\\mathbf\{A\}\}^\{2\}\\mathrel\{:=\}\\operatorname\{tr\}\\\!\\left\(\\mathbf\{E\}^\{\\top\}\\mathbf\{G\}\\mathbf\{E\}\\mathbf\{A\}\\right\)=\\operatorname\{vec\}\(\\mathbf\{E\}\)^\{\\top\}\(\\mathbf\{A\}\\otimes\\mathbf\{G\}\)\\operatorname\{vec\}\(\\mathbf\{E\}\)\.\(6\)All computations use the two statistics directly, so the fullm​n×m​nmn\\times mnKronecker product never needs to be formed\. NanoQuant uses as its loss the diagonal case𝐀=𝐃~in2\\mathbf\{A\}=\\widetilde\{\\mathbf\{D\}\}\_\{\\mathrm\{in\}\}^\{2\},𝐆=𝐃~out2\\mathbf\{G\}=\\widetilde\{\\mathbf\{D\}\}\_\{\\mathrm\{out\}\}^\{2\}, which is equivalent to𝐀=𝐈n\\mathbf\{A\}=\\mathbf\{I\}\_\{n\},𝐆=𝐈m\\mathbf\{G\}=\\mathbf\{I\}\_\{m\}in the transformed coordinates\.

Shampoo\.Shampoo\([Gupta et al\., 2018](https://arxiv.org/html/2609.38521#bib.bib8)\)preconditions a matrix gradient∇t=𝜹t​𝐱t⊤\\nabla\_\{t\}=\\bm\{\\delta\}\_\{t\}\\mathbf\{x\}\_\{t\}^\{\\top\}with two accumulated statistics,

𝐆Sh=∑t∇t∇t⊤=∑t∥𝐱t∥2​𝜹t​𝜹t⊤,𝐀Sh=∑t∇t⊤∇t=∑t∥𝜹t∥2​𝐱t​𝐱t⊤,\\mathbf\{G\}\_\{\\mathrm\{Sh\}\}=\\sum\_\{t\}\\nabla\_\{t\}\\nabla\_\{t\}^\{\\top\}=\\sum\_\{t\}\\lVert\\mathbf\{x\}\_\{t\}\\rVert^\{2\}\\,\\bm\{\\delta\}\_\{t\}\\bm\{\\delta\}\_\{t\}^\{\\top\},\\qquad\\mathbf\{A\}\_\{\\mathrm\{Sh\}\}=\\sum\_\{t\}\\nabla\_\{t\}^\{\\top\}\\nabla\_\{t\}=\\sum\_\{t\}\\lVert\\bm\{\\delta\}\_\{t\}\\rVert^\{2\}\\,\\mathbf\{x\}\_\{t\}\\mathbf\{x\}\_\{t\}^\{\\top\},\(7\)and updates according to𝐖←𝐖−η𝐆Sh−1/4∇𝐀Sh−1/4\\mathbf\{W\}\\leftarrow\\mathbf\{W\}\-\\eta\\,\\mathbf\{G\}\_\{\\mathrm\{Sh\}\}^\{\-1/4\}\\,\\nabla\\,\\mathbf\{A\}\_\{\\mathrm\{Sh\}\}^\{\-1/4\}, which corresponds in vectorized form with the preconditioner\(𝐀Sh1/2⊗𝐆Sh1/2\)−1/2\(\\mathbf\{A\}\_\{\\mathrm\{Sh\}\}^\{1/2\}\\otimes\\mathbf\{G\}\_\{\\mathrm\{Sh\}\}^\{1/2\}\)^\{\-1/2\}\.[Morwani et al\. \(2025\)](https://arxiv.org/html/2609.38521#bib.bib6)show that the square of Shampoo’s approximation is one power\-iteration step toward the optimal Kronecker approximation of𝐇^cal\\widehat\{\\mathbf\{H\}\}\_\{\\mathrm\{cal\}\}, so the exponent1/41/4per statistic is the inverse square root of an approximate curvature\.

Estimators as token\-weighted second moments\.Kronecker fits of𝐇^cal\\widehat\{\\mathbf\{H\}\}\_\{\\mathrm\{cal\}\}all share the form𝐆∝∑twtx​𝜹t​𝜹t⊤\\mathbf\{G\}\\propto\\sum\_\{t\}w^\{x\}\_\{t\}\\,\\bm\{\\delta\}\_\{t\}\\bm\{\\delta\}\_\{t\}^\{\\top\},𝐀∝∑twtδ​𝐱t​𝐱t⊤\\mathbf\{A\}\\propto\\sum\_\{t\}w^\{\\delta\}\_\{t\}\\,\\mathbf\{x\}\_\{t\}\\mathbf\{x\}\_\{t\}^\{\\top\}, differing only in their token weights\. The nearest\-Kronecker\-product \(NKP\) fit\([Mei et al\., 2023](https://arxiv.org/html/2609.38521#bib.bib5);[Morwani et al\., 2025](https://arxiv.org/html/2609.38521#bib.bib6)\)minimizes∥𝐇^cal−𝐀⊗𝐆∥F\\lVert\\widehat\{\\mathbf\{H\}\}\_\{\\mathrm\{cal\}\}\-\\mathbf\{A\}\\otimes\\mathbf\{G\}\\rVert\_\{F\}over positive semidefinite statistics, with fixed\-point weightswtx=𝐱t⊤​𝐀​𝐱tw^\{x\}\_\{t\}=\\mathbf\{x\}\_\{t\}^\{\\top\}\\mathbf\{A\}\\,\\mathbf\{x\}\_\{t\},wtδ=𝜹t⊤​𝐆​𝜹tw^\{\\delta\}\_\{t\}=\\bm\{\\delta\}\_\{t\}^\{\\top\}\\mathbf\{G\}\\,\\bm\{\\delta\}\_\{t\}; Shampoo’s statistics coem from its first alternation from𝐀=𝐈n\\mathbf\{A\}=\\mathbf\{I\}\_\{n\},𝐆=𝐈m\\mathbf\{G\}=\\mathbf\{I\}\_\{m\}\. TheKL\-Shampoofit\([Lin et al\., 2026](https://arxiv.org/html/2609.38521#bib.bib9);[Sun and Wei, 2026](https://arxiv.org/html/2609.38521#bib.bib10)\)is the matrix\-normal maximum\-likelihood estimate

\(𝐀,𝐆\)∈arg⁡min𝐀,𝐆≻𝟎​DKL​\(𝒩⁡\(𝟎,𝐇^cal\)∥𝒩⁡\(𝟎,𝐀⊗𝐆\)\),tr⁡\(𝐀\)=n,\(\\mathbf\{A\},\\mathbf\{G\}\)\\in\\arg\\min\_\{\\mathbf\{A\},\\mathbf\{G\}\\succ\\mathbf\{0\}\}D\_\{\\mathrm\{KL\}\}\\\!\\left\(\\mathcal\{N\}\(\\mathbf\{0\},\\widehat\{\\mathbf\{H\}\}\_\{\\mathrm\{cal\}\}\)\\,\\middle\\\|\\,\\mathcal\{N\}\(\\mathbf\{0\},\\mathbf\{A\}\\otimes\\mathbf\{G\}\)\\right\),\\qquad\\operatorname\{tr\}\(\\mathbf\{A\}\)=n,\(8\)where the trace constraint removes the scale ambiguity\(c​𝐀\)⊗\(c−1​𝐆\)=𝐀⊗𝐆\(c\\mathbf\{A\}\)\\otimes\(c^\{\-1\}\\mathbf\{G\}\)=\\mathbf\{A\}\\otimes\\mathbf\{G\}\. Stationarity then gives the coupled fixed point

𝐆=1n​∑t\(𝐱t⊤​𝐀−1​𝐱t\)​𝜹t​𝜹t⊤,𝐀=1m​∑t\(𝜹t⊤​𝐆−1​𝜹t\)​𝐱t​𝐱t⊤,\\mathbf\{G\}=\\frac\{1\}\{n\}\\sum\_\{t\}\\big\(\\mathbf\{x\}\_\{t\}^\{\\top\}\\mathbf\{A\}^\{\-1\}\\mathbf\{x\}\_\{t\}\\big\)\\,\\bm\{\\delta\}\_\{t\}\\bm\{\\delta\}\_\{t\}^\{\\top\},\\qquad\\mathbf\{A\}=\\frac\{1\}\{m\}\\sum\_\{t\}\\big\(\\bm\{\\delta\}\_\{t\}^\{\\top\}\\mathbf\{G\}^\{\-1\}\\bm\{\\delta\}\_\{t\}\\big\)\\,\\mathbf\{x\}\_\{t\}\\mathbf\{x\}\_\{t\}^\{\\top\},\(9\)so the token weights are the*inverse*quadratic forms\. The KL\-Shampoo optimizer tracks these statistics by an exponential moving average and updates𝐖←𝐖−η𝐆−1/2∇𝐀−1/2\\mathbf\{W\}\\leftarrow\\mathbf\{W\}\-\\eta\\,\\mathbf\{G\}^\{\-1/2\}\\,\\nabla\\,\\mathbf\{A\}^\{\-1/2\}, with exponent1/21/2per statistic because𝐀⊗𝐆\\mathbf\{A\}\\otimes\\mathbf\{G\}now approximates𝐇^cal\\widehat\{\\mathbf\{H\}\}\_\{\\mathrm\{cal\}\}directly\.

## 4ShamAN\-Q

Figure 2:ShamAN\-Q end to end\. Phase 1 runs the calibration set through the frozen teacher, fits and shrinks the curvature statistics\(𝐀γ,𝐆γ\)\(\\mathbf\{A\}\_\{\\gamma\},\\mathbf\{G\}\_\{\\gamma\}\)of every linear layer, and allocates ranks at bit parity\. Phase 2 processes decoder blocks in order\. The factorized prefix and the untouched full\-precision suffix are frozen while blockbbis tuned \(TuneFP\), factorized layer by layer by Kron–LB–ADMM under the transported statistics\(𝐀¯,𝐆¯\)\(\\overline\{\\mathbf\{A\}\},\\overline\{\\mathbf\{G\}\}\)\(Figure[3](https://arxiv.org/html/2609.38521#S4.F3)\), and refined by latent STE\. The statistics of the unfactorized layers are refreshed on the partially quantized model after everyΔ\\Deltablocks\. Phase 3 distills only the scale vectors against the teacher, with the sign matrices frozen\. Flames mark trained parameters, snowflakes frozen ones, and blue blocks denote differences between ShamAN\-Q and NanoQuant\.ShamAN\-Q keeps the same representation as NanoQuant and similarly adopts a three\-phase pipeline \(§[3\.1](https://arxiv.org/html/2609.38521#S3.SS1)\), but changes the geometry used to fit the low\-rank factors at four interfaces\. Curvature is estimated during the initial calibration \(§[4\.1](https://arxiv.org/html/2609.38521#S4.SS1)\), used during rank allocation \(§[4\.2](https://arxiv.org/html/2609.38521#S4.SS2)\), applied to the continuous factor updates, and refreshed with respect to the partially quantized model \(§[4\.3](https://arxiv.org/html/2609.38521#S4.SS3)\)\. All components related to SVID, scale extraction, latent tuning, packing, and scale distillation are unchanged\. Figure[2](https://arxiv.org/html/2609.38521#S4.F2)shows the three phases and marks their changed stages, while Figure[3](https://arxiv.org/html/2609.38521#S4.F3)details the curvature\-aware ADMM fitting of one layer, which is the primary extension over NanoQuant\. The complete pipeline is given in Appendix[C](https://arxiv.org/html/2609.38521#A3)\.

Figure 3:ShamAN\-Q pipeline for one layer\. Calibration fits and shrinks the curvature statistics, yielding the transported metric\(𝐀¯,𝐆¯\)\(\\overline\{\\mathbf\{A\}\},\\overline\{\\mathbf\{G\}\}\)and diagonal maps\(𝐃~in,𝐃~out\)\(\\widetilde\{\\mathbf\{D\}\}\_\{\\mathrm\{in\}\},\\widetilde\{\\mathbf\{D\}\}\_\{\\mathrm\{out\}\}\)\. Preconditioning maps𝐖\\mathbf\{W\}to𝐖~\\widetilde\{\\mathbf\{W\}\}\. Kron–LB–ADMM returns the continuous weight factors\(𝒰,𝒱\)\(\\mathcal\{U\},\\mathcal\{V\}\)and scale vectors\(𝐬1,𝐬2\)\(\\mathbf\{s\}\_\{1\},\\mathbf\{s\}\_\{2\}\)\. The transported statistics alter only the continuous𝐔/𝐕\\mathbf\{U\}/\\mathbf\{V\}optimization component; Euclidean consensus leaves SVID, dual updates, scale extraction, and sign extraction unchanged from NanoQuant\.### 4\.1Dense Kronecker Curvature from Global Calibration

#### Curvature estimation\.

Where NanoQuant keeps only the diagonal of𝐇^cal\\widehat\{\\mathbf\{H\}\}\_\{\\mathrm\{cal\}\}\(Eq\.[3](https://arxiv.org/html/2609.38521#S3.E3)\), ShamAN\-Q keeps its full Kronecker structure;\(𝐀,𝐆\)\(\\mathbf\{A\},\\mathbf\{G\}\)are fitted for each linear layer by three alternating passes of the KL\-Shampoo fixed point \(Eq\.[9](https://arxiv.org/html/2609.38521#S3.E9)\) over the calibration set, with Cholesky inverses\. The KL fit is preferred to NKP for its token weighting \(§[3\.2](https://arxiv.org/html/2609.38521#S3.SS2)\)\. The NKP weights𝐱t⊤​𝐀​𝐱t\\mathbf\{x\}\_\{t\}^\{\\top\}\\mathbf\{A\}\\,\\mathbf\{x\}\_\{t\}are quartic in activation magnitude, so the few tokens corresponding with massive activations\([Sun et al\., 2024](https://arxiv.org/html/2609.38521#bib.bib21)\)dominate the leading eigenvalues\. The KL weights𝐱t⊤​𝐀−1​𝐱t\\mathbf\{x\}\_\{t\}^\{\\top\}\\mathbf\{A\}^\{\-1\}\\mathbf\{x\}\_\{t\}, in contrast, are bounded leverage scores, and the resulting statistics have condition numbers of order 40 instead of several hundred\. The estimator consumes only the pairs\(𝐱t,𝜹t\)\(\\mathbf\{x\}\_\{t\},\\bm\{\\delta\}\_\{t\}\)and never materializes𝐇^cal\\widehat\{\\mathbf\{H\}\}\_\{\\mathrm\{cal\}\}\.

#### Scale\-matched shrinkage\.

The fitted statistics are regularized before any coordinate change,

𝐀γ=\(1−γ\)​𝐀\+γ​tr⁡\(𝐀\)n​𝐈n,𝐆γ=\(1−γ\)​𝐆\+γ​tr⁡\(𝐆\)m​𝐈m,γ∈\[0,1\]\.\\mathbf\{A\}\_\{\\gamma\}=\(1\-\\gamma\)\\mathbf\{A\}\+\\gamma\\,\\tfrac\{\\operatorname\{tr\}\(\\mathbf\{A\}\)\}\{n\}\\,\\mathbf\{I\}\_\{n\},\\qquad\\mathbf\{G\}\_\{\\gamma\}=\(1\-\\gamma\)\\mathbf\{G\}\+\\gamma\\,\\tfrac\{\\operatorname\{tr\}\(\\mathbf\{G\}\)\}\{m\}\\,\\mathbf\{I\}\_\{m\},\\qquad\\gamma\\in\[0,1\]\.\(10\)Each identity has the same mean eigenvalue as the statistic it regularizes, so shrinkage changes anisotropy without arbitrarily changing its scale: there is no additional damping term\. The diagonal of the shrunken statistic is NanoQuant’s shrunken𝐝2\\mathbf\{d\}^\{2\}of Eq\.[3](https://arxiv.org/html/2609.38521#S3.E3), and NanoQuant’s robust token clipping is kept when the statistics are accumulated\. The fit and shrinkage of this section form the calibration stage of Figure[3](https://arxiv.org/html/2609.38521#S4.F3)\. They run once at calibration and at each periodic refresh, while the diagonal maps, transport, and tempering of §[4\.3](https://arxiv.org/html/2609.38521#S4.SS3)run per layer\.

### 4\.2Non\-uniform Rank Allocation at Bit Parity

NanoQuant gives every layer the same bits per weight: settingBPW=β\\operatorname\{BPW\}=\\betain Eq\.[2](https://arxiv.org/html/2609.38521#S3.E2)yields

rℓ⋆=β​mℓ​nℓ/\(mℓ\+nℓ\)−16,β=1,r^\{\\star\}\_\{\\ell\}=\\beta\\,m\_\{\\ell\}n\_\{\\ell\}/\(m\_\{\\ell\}\+n\_\{\\ell\}\)\-16,\\qquad\\beta=1,\(11\)givingrℓunir^\{\\mathrm\{uni\}\}\_\{\\ell\}when floored to a multiple of 32\. ShamAN\-Q keeps the totalNbits=∑ℓbitsℓ⁡\(rℓuni\)N\_\{\\mathrm\{bits\}\}=\\sum\_\{\\ell\}\\operatorname\{bits\}\_\{\\ell\}\(r^\{\\mathrm\{uni\}\}\_\{\\ell\}\)\(bit parity\) and redistributes it across layers in steps of 32 with rank ceilingmin⁡\(mℓ,nℓ\)\\min\(m\_\{\\ell\},n\_\{\\ell\}\)\. Givenbℓ∈\{1,…,B\}b\_\{\\ell\}\\in\\\{1,\\dots,B\\\}, the block of layerℓ\\elland its typet⁡\(ℓ\)t\(\\ell\)\(q, k, v, o, gate, up, down\), both allocators use the depth priorωℓ=exp⁡\(κ⁡\(bℓ−1B−1−12\)\)​wt⁡\(ℓ\)\\omega\_\{\\ell\}=\\exp\\big\(\\kappa\\big\(\\tfrac\{b\_\{\\ell\}\-1\}\{B\-1\}\-\\tfrac\{1\}\{2\}\\big\)\\big\)\\,w\_\{t\(\\ell\)\}, withκ=0\.6\\kappa=0\.6, so the last block gets more bits than the first by a factor ofe0\.6≈1\.8e^\{0\.6\}\\approx 1\.8\. The*hand table*sets budgetsβℓ∝β​ωℓ\\beta\_\{\\ell\}\\propto\\beta\\,\\omega\_\{\\ell\}—where the type weightswtw\_\{t\}are contained in a range from 0\.85 \(q\) to 1\.15 \(down\), taken from the block\-loss jump each layer type caused when binarized in the 4B logs—and rounds them to ranks meeting exactlyNbitsN\_\{\\mathrm\{bits\}\}\. The*measured sensitivity*allocator instead probes each layer with a short dense\-curvature LB\-ADMM \(§[4\.3](https://arxiv.org/html/2609.38521#S4.SS3)\) at three ranks, scores the result by the Gauss–Newton errorJℓ​\(r\)=∥𝐖ℓ−𝐖^ℓ​\(r\)∥𝐆γ,ℓ,𝐀γ,ℓ2J\_\{\\ell\}\(r\)=\\lVert\\mathbf\{W\}\_\{\\ell\}\-\\widehat\{\\mathbf\{W\}\}\_\{\\ell\}\(r\)\\rVert^\{2\}\_\{\\mathbf\{G\}\_\{\\gamma,\\ell\},\\mathbf\{A\}\_\{\\gamma,\\ell\}\}using the layer’s own metric, fits a power law, multiplies its level by the depth prior with allwt=1w\_\{t\}=1, and hands out 32\-rank steps greedily by predicted loss decrease per bit\. The probes rank the seven layer types within a block well, but under\-value late blocks, which the prior supplies\. Appendix[B](https://arxiv.org/html/2609.38521#A2)gives the type weights, rounding, and probe protocol\.

### 4\.3Block Reconstruction Pipeline

Blocks are processed in order, and within each block every linear layer is factorized according to NanoQuant’s per\-layer schedule of §[3\.1](https://arxiv.org/html/2609.38521#S3.SS1)\. Step 1 \(TuneFP\) and Step 3 \(TuneLatentSTE, sign extraction, and packing\) are unchanged; ShamAN\-Q changes only the preconditioning and the continuous solves inside Step 2\.

Periodic curvature refresh\.The Fisher information is local to the full\-precision weights, and the model drifts from that point as blocks are binarized\. ShamAN\-Q corrects for this at two time scales: a per\-layer fresh input statistic inside Step 2\-1, and a periodic refresh at block boundaries\. For everyΔ\\Deltablocks, the statistics of all*not\-yet\-factorized*layers are re\-estimated with one warm\-started pass of Eq\.[9](https://arxiv.org/html/2609.38521#S3.E9)through the current partially quantized modelℳ^\\widehat\{\\mathcal\{M\}\}, so that later blocks see the statistics they will actually receive, and are shrunk withγ\\gammaas before\.

Step 2\-1: Dense\-Curvature Preconditioning\.NanoQuant measures𝐃~in\\widetilde\{\\mathbf\{D\}\}\_\{\\mathrm\{in\}\}once, on the full\-precision model \(Eq\.[3](https://arxiv.org/html/2609.38521#S3.E3)\)\. Under the block\-wise schedule, by the time layerℓ\\ellis factorized, its inputs come from an already quantized prefix and from full\-precision layers thatTuneFPhas just modified\. Immediately before ADMM for each layer, we therefore re\-measure its input second moment on exactly those inputs,𝐀fresh=1N​∑t=1N𝐱t​𝐱t⊤\\mathbf\{A\}\_\{\\mathrm\{fresh\}\}=\\frac\{1\}\{N\}\\sum\_\{t=1\}^\{N\}\\mathbf\{x\}\_\{t\}\\mathbf\{x\}\_\{t\}^\{\\top\}, tkaen over theNNcalibration tokens with no gradient weighting\. Shrunken by Eq\.[10](https://arxiv.org/html/2609.38521#S4.E10), it then takes the place of𝐀γ\\mathbf\{A\}\_\{\\gamma\}for that layer, while the output\-side statistic𝐆γ\\mathbf\{G\}\_\{\\gamma\}is that of either the calibration time or most recent refresh\.

The diagonal preconditioners are the square roots of the diagonals of the shrunken statistics, floored atϵ\>0\\epsilon\>0to keep both maps invertible \(Eq\.[18](https://arxiv.org/html/2609.38521#A3.E18)\), while𝐖~\\widetilde\{\\mathbf\{W\}\}and𝐄~\\widetilde\{\\mathbf\{E\}\}are taken as in §[3\.1](https://arxiv.org/html/2609.38521#S3.SS1)\. Rather than refitting in these coordinates, we transport the same shrunken statistics by congruence, which preserves the Mahalanobis error exactly:

𝐀¯=𝐃~in−1​𝐀γ​𝐃~in−1,𝐆¯=𝐃~out−1​𝐆γ​𝐃~out−1,‖𝐄‖𝐆γ,𝐀γ2=‖𝐄~‖𝐆¯,𝐀¯2,\\overline\{\\mathbf\{A\}\}=\\widetilde\{\\mathbf\{D\}\}\_\{\\mathrm\{in\}\}^\{\-1\}\\mathbf\{A\}\_\{\\gamma\}\\widetilde\{\\mathbf\{D\}\}\_\{\\mathrm\{in\}\}^\{\-1\},\\qquad\\overline\{\\mathbf\{G\}\}=\\widetilde\{\\mathbf\{D\}\}\_\{\\mathrm\{out\}\}^\{\-1\}\\mathbf\{G\}\_\{\\gamma\}\\widetilde\{\\mathbf\{D\}\}\_\{\\mathrm\{out\}\}^\{\-1\},\\qquad\\left\\lVert\\mathbf\{E\}\\right\\rVert\_\{\\mathbf\{G\}\_\{\\gamma\},\\mathbf\{A\}\_\{\\gamma\}\}^\{2\}=\\left\\lVert\\widetilde\{\\mathbf\{E\}\}\\right\\rVert\_\{\\overline\{\\mathbf\{G\}\},\\overline\{\\mathbf\{A\}\}\}^\{2\},\(12\)which is obtained by substituting𝐄=𝐃~out−1​𝐄~​𝐃~in−1\\mathbf\{E\}=\\widetilde\{\\mathbf\{D\}\}\_\{\\mathrm\{out\}\}^\{\-1\}\\widetilde\{\\mathbf\{E\}\}\\widetilde\{\\mathbf\{D\}\}\_\{\\mathrm\{in\}\}^\{\-1\}into Eq\.[6](https://arxiv.org/html/2609.38521#S3.E6)and cycling the trace\. Notwithstanding the floor, the transported statistics are unit\-diagonal correlation matrices, so the dense metric combines NanoQuant’s diagonal preconditioning with\(𝐆¯,𝐀¯\)\(\\overline\{\\mathbf\{G\}\},\\overline\{\\mathbf\{A\}\}\)\. Thus NanoQuant is recovered exactly by𝐀¯=𝐈n\\overline\{\\mathbf\{A\}\}=\\mathbf\{I\}\_\{n\}and𝐆¯=𝐈m\\overline\{\\mathbf\{G\}\}=\\mathbf\{I\}\_\{m\}\.

The one\-bit reconstruction error is large \(∥𝐄∥≈0\.4\\lVert\\mathbf\{E\}\\rVert\\approx 0\.4–0\.5​∥𝐖∥0\.5\\,\\lVert\\mathbf\{W\}\\rVert\), well outside the neighborhood in which the Fisher is an accurate curvature, so before the ADMM solver is applied, each transported statistic is replaced by its trace\-preserving power \(*spectral tempering*\)\. Writing𝐀¯=𝐐​Diag⁡\(𝐜\)​𝐐⊤\\overline\{\\mathbf\{A\}\}=\\mathbf\{Q\}\\operatorname\{Diag\}\(\\mathbf\{c\}\)\\mathbf\{Q\}^\{\\top\}, we then have

𝐀¯⟵∑ici∑icip​𝐐​Diag⁡\(𝐜p\)​𝐐⊤,p=12,\\overline\{\\mathbf\{A\}\}\\;\\longleftarrow\\;\\frac\{\\sum\_\{i\}c\_\{i\}\}\{\\sum\_\{i\}c\_\{i\}^\{\\,p\}\}\\;\\mathbf\{Q\}\\operatorname\{Diag\}\(\\mathbf\{c\}^\{\\,p\}\)\\mathbf\{Q\}^\{\\top\},\\qquad p=\\tfrac\{1\}\{2\},\(13\)and likewise for𝐆¯\\overline\{\\mathbf\{G\}\}\. This halves the log\-condition number, and as a result, the data term trusts the dominant curvature directions less\. The square root is the regret\-optimal metric under an uncertain curvature\. Tempering is free given the cached eigendecomposition\. It leaves the identity unchanged, and beats bothp=1p=1and Shampoo’sp=14p=\\tfrac\{1\}\{4\}\(§[5\.4](https://arxiv.org/html/2609.38521#S5.SS4)\)\. All uses of𝐀¯,𝐆¯\\overline\{\\mathbf\{A\}\},\\overline\{\\mathbf\{G\}\}below refer to the tempered statistics; their eigensystems and𝐌:⁣=𝐆¯𝐖~𝐀¯\\mathbf\{M\}\\mathrel\{:=\}\\overline\{\\mathbf\{G\}\}\\,\\widetilde\{\\mathbf\{W\}\}\\,\\overline\{\\mathbf\{A\}\}are cached per layer \(Algorithm[1](https://arxiv.org/html/2609.38521#alg1)in Appendix[C](https://arxiv.org/html/2609.38521#A3)\)\.

Step 2\-2: Dense\-Curvature LB\-ADMM\.Relative to NanoQuant’s objective \(Eq\.[4](https://arxiv.org/html/2609.38521#S3.E4)\), ShamAN\-Q replaces the Frobenius reconstruction term by the transported metric and keeps everything else\. In the transformed coordinates it solves

min𝐔,𝐕,𝐙U,𝐙V\\displaystyle\\min\_\{\\mathbf\{U\},\\mathbf\{V\},\\mathbf\{Z\}\_\{U\},\\mathbf\{Z\}\_\{V\}\}\\quad12​‖𝐖~−𝐔𝐕⊤‖𝐆¯,𝐀¯2\+λ2​\(‖𝐔‖F2\+‖𝐕‖F2\)\+ℐ𝒞m,r​\(𝐙U\)\+ℐ𝒞n,r​\(𝐙V\)\\displaystyle\\frac\{1\}\{2\}\\left\\lVert\\widetilde\{\\mathbf\{W\}\}\-\\mathbf\{U\}\\mathbf\{V\}^\{\\top\}\\right\\rVert\_\{\\overline\{\\mathbf\{G\}\},\\overline\{\\mathbf\{A\}\}\}^\{2\}\+\\frac\{\\lambda\}\{2\}\\left\(\\left\\lVert\\mathbf\{U\}\\right\\rVert\_\{F\}^\{2\}\+\\left\\lVert\\mathbf\{V\}\\right\\rVert\_\{F\}^\{2\}\\right\)\+\\mathcal\{I\}\_\{\\mathcal\{C\}\_\{m,r\}\}\(\\mathbf\{Z\}\_\{U\}\)\+\\mathcal\{I\}\_\{\\mathcal\{C\}\_\{n,r\}\}\(\\mathbf\{Z\}\_\{V\}\)subject to𝐔=𝐙U,𝐕=𝐙V\.\\displaystyle\\text\{subject to\}\\quad\\mathbf\{U\}=\\mathbf\{Z\}\_\{U\},\\quad\\mathbf\{V\}=\\mathbf\{Z\}\_\{V\}\.\(14\)with𝐙U\\mathbf\{Z\}\_\{U\}and𝐙V\\mathbf\{Z\}\_\{V\}in the SVID family of §[3\.1](https://arxiv.org/html/2609.38521#S3.SS1)\. The reconstruction term is Mahalanobis, but the two augmented consensus penalties are Euclidean\. This keeps SVID, dual updates, theρk\\rho\_\{k\}schedule, and residuals identical to NanoQuant, so only the continuous solves change\. Figure[3](https://arxiv.org/html/2609.38521#S4.F3)follows one layer from calibration through preconditioning, the continuous Kron–LB–ADMM solve, and scale and sign extraction; Algorithm[2](https://arxiv.org/html/2609.38521#alg2)in Appendix[C](https://arxiv.org/html/2609.38521#A3)lists the solver in full\.

Takeαk=ρk\+λ\\alpha\_\{k\}=\\rho\_\{k\}\+\\lambdaand let𝐂Uk=𝐙Uk−𝚲Uk\\mathbf\{C\}\_\{U\}^\{k\}=\\mathbf\{Z\}\_\{U\}^\{k\}\-\\bm\{\\Lambda\}\_\{U\}^\{k\}be the consensus target formed with the scaled dual𝚲Uk\\bm\{\\Lambda\}\_\{U\}^\{k\}\. Holding𝐕k\\mathbf\{V\}^\{k\}fixed, stationarity of Eq\.[14](https://arxiv.org/html/2609.38521#S4.E14)gives

𝐆¯​𝐔k\+1​𝐑Vk\+αk​𝐔k\+1=𝐌𝐕k\+ρk​𝐂Uk,𝐑Vk=\(𝐕k\)⊤​𝐀¯​𝐕k,\\overline\{\\mathbf\{G\}\}\\,\\mathbf\{U\}^\{k\+1\}\\mathbf\{R\}\_\{V\}^\{k\}\+\\alpha\_\{k\}\\mathbf\{U\}^\{k\+1\}=\\mathbf\{M\}\\mathbf\{V\}^\{k\}\+\\rho\_\{k\}\\mathbf\{C\}\_\{U\}^\{k\},\\qquad\\mathbf\{R\}\_\{V\}^\{k\}=\(\\mathbf\{V\}^\{k\}\)^\{\\top\}\\overline\{\\mathbf\{A\}\}\\mathbf\{V\}^\{k\},\(15\)and the𝐕\\mathbf\{V\}update is obtained analogously, by exchanging the roles of\(𝐆¯,𝐔\)\(\\overline\{\\mathbf\{G\}\},\\mathbf\{U\}\)and\(𝐀¯,𝐕\)\(\\overline\{\\mathbf\{A\}\},\\mathbf\{V\}\)and using𝐌⊤\\mathbf\{M\}^\{\\top\}in place of𝐌\\mathbf\{M\}\. These are Mahalanobis Sylvester equations, rather than NanoQuant’s right\-sidedr×rr\\times rnormal equations \(Eq\.[5](https://arxiv.org/html/2609.38521#S3.E5)\)\. Because the augmented penalties are Euclidean, the left curvature statistic does not cancel\. Both have the form𝐂𝐗𝐑\+α​𝐗=𝐁\\mathbf\{C\}\\mathbf\{X\}\\mathbf\{R\}\+\\alpha\\mathbf\{X\}=\\mathbf\{B\}, with𝐂\\mathbf\{C\}a tempered, transported curvature statistic and𝐑∈ℝr×r\\mathbf\{R\}\\in\\mathbb\{R\}^\{r\\times r\}positive semidefinite\. With the curvature eigensystem𝐂=𝐐​Diag⁡\(𝐜\)​𝐐⊤\\mathbf\{C\}=\\mathbf\{Q\}\\operatorname\{Diag\}\(\\mathbf\{c\}\)\\mathbf\{Q\}^\{\\top\}cached once per layer and𝐑=𝐒​Diag⁡\(𝝂\)​𝐒⊤\\mathbf\{R\}=\\mathbf\{S\}\\operatorname\{Diag\}\(\\bm\{\\nu\}\)\\mathbf\{S\}^\{\\top\}recomputed for each update, the solution in the joint eigenbasis is

𝐗=𝐐𝐘𝐒⊤,𝐘i,j=\[𝐐⊤​𝐁𝐒\]i,j/\(ci​νj\+α\)\.\\mathbf\{X\}=\\mathbf\{Q\}\\mathbf\{Y\}\\mathbf\{S\}^\{\\top\},\\qquad\\mathbf\{Y\}\_\{i,j\}=\\left\[\\mathbf\{Q\}^\{\\top\}\\mathbf\{B\}\\mathbf\{S\}\\right\]\_\{i,j\}/\(c\_\{i\}\\nu\_\{j\}\+\\alpha\)\.\(16\)Becauseα=ρ\+λ\>0\\alpha=\\rho\+\\lambda\>0, every denominator is positive even for semidefinite statistics\. We observe that taking𝐆¯=𝐈m\\overline\{\\mathbf\{G\}\}=\\mathbf\{I\}\_\{m\}and𝐀¯=𝐈n\\overline\{\\mathbf\{A\}\}=\\mathbf\{I\}\_\{n\}recovers Eq\.[5](https://arxiv.org/html/2609.38521#S3.E5)\. One eigendecomposition per statistic per layer comprises the added cost \(Appendix[D](https://arxiv.org/html/2609.38521#A4)\), and the SVID projection and its inner iteration count are independent of𝐀¯,𝐆¯\\overline\{\\mathbf\{A\}\},\\overline\{\\mathbf\{G\}\}\(Appendix[C](https://arxiv.org/html/2609.38521#A3)\)\.

Step 2\-3: Unchanged Scale Extraction\.The magnitude balancing and scale extraction that NanoQuant uses are applied verbatim to the final proxies in the transformed coordinates, followed by its un\-preconditioning through the inverse diagonal maps \(Appendix[C](https://arxiv.org/html/2609.38521#A3)\)\. The resulting sign matrices initialize the latents𝒰,𝒱\\mathcal\{U\},\\mathcal\{V\}of Step 3, and the block loss, packing, andTuneScalesKDall come from NanoQuant as well \(§[3\.1](https://arxiv.org/html/2609.38521#S3.SS1)\)\.

## 5Experiments

### 5\.1Experimental Setup

Implementation and Environment\.ShamAN\-Q is implemented on top of the released NanoQuant code and follows its protocol wherever §[4](https://arxiv.org/html/2609.38521#S4)does not change it\. The 0\.6B to 4B results use a single NVIDIA A100\-80GB GPU with fp64 eigendecompositions\. The 8B and 14B results use fp32 eigendecompositions on two to four GPUs, which was verified to be lossless at 1\.7B\. Hyperparameters and the engineering measures that keep the dense recipe within budget are given in Appendix[A](https://arxiv.org/html/2609.38521#A1)\.

Models and Datasets\.We quantize Qwen3\-Base at 0\.6B, 1\.7B, and 4B, 8B, and 14B parameters\([Qwen Team, 2025](https://arxiv.org/html/2609.38521#bib.bib15)\), calibrating on 128 WikiText\-2\([Merity et al\., 2016](https://arxiv.org/html/2609.38521#bib.bib19)\)sequences of 2048 tokens \(seed 0\)\. At 8B and 14B the curvature statistics and their refreshes use 512 sequences, while block reconstruction and distillation still use 128\. We report both WikiText\-2 perplexity at context 2048 and the mean zero\-shot accuracy from the Eleuther LM Evaluation Harness\([Gao et al\., 2024](https://arxiv.org/html/2609.38521#bib.bib20)\)\. All numbers are single runs\.

Baselines\.The baseline is NanoQuant at the same total number of bits, presented both as published\([Chong et al\., 2026](https://arxiv.org/html/2609.38521#bib.bib1)\)and as our own paper\-faithful reproduction\. Every ShamAN\-Q model spends the*same total number of bits*as NanoQuant\. That is, at the 1 bpw budget, 0\.973 bpw for 0\.6B and 0\.986 bpw for 1\.7B and 4B, and at 0\.8 bpw, 0\.773 / 0\.783 / 0\.793 / 0\.794 bpw for 0\.6B / 1\.7B / 4B / 8B\. At 0\.55 bpw the actual values are 0\.527 / 0\.535 / 0\.544 / 0\.545 bpw, and the 8B curvature statistics use 128 sequences\. Only the rank allocator \(§[4\.2](https://arxiv.org/html/2609.38521#S4.SS2)\) differs across sizes\.

Next Token Prediction\.Table[1](https://arxiv.org/html/2609.38521#S5.T1)compares ShamAN\-Q with NanoQuant at identical bit budgets\. PPL is lower at every model size, and the gap narrows as the model grows\. At 0\.8 bpw the PPL margin over NanoQuant widens at 1\.7B and 4B \(13 to 26% and 3 to 11%\), narrows to 0\.4% at 8B, and at 0\.6B and 1\.7B ShamAN\-Q at 0\.8 bpw matches or beats NanoQuant at 1\.0 bpw\. Figure[1](https://arxiv.org/html/2609.38521#S1.F1)shows both metrics across model sizes through 14B\. The right block reports the zero\-shot mean\. NanoQuant publishes zero\-shot accuracies for Qwen3 only at 8B, and at 4B the rank allocator trades perplexity against zero\-shot accuracy \(§[5\.2](https://arxiv.org/html/2609.38521#S5.SS2)\)\.

Table 1:WikiText\-2 perplexity and zero\-shot mean across a suite of reasoning tasks for<<1 bpw Qwen3\-Base models at the same total bits as NanoQuant\([Chong et al\., 2026](https://arxiv.org/html/2609.38521#bib.bib1)\), with both reported and reproduced values\. The upper block uses a 1\.0 bpw budget, the middle block a 0\.8 bpw budget, and the lower block a 0\.55 bpw budget\. Unreported results are marked with –\.
### 5\.2Rank Allocation at Bit Parity

Table[4](https://arxiv.org/html/2609.38521#S5.T4)compares the allocators of §[4\.2](https://arxiv.org/html/2609.38521#S4.SS2)at the same total bits\. At 0\.6B the measured sensitivity×\\timesdepth prior is best, 2\.5 perplexity below uniform ranks\. At 4B the measured allocator gives 13\.80 against 13\.55 from the hand table but restores the zero\-shot mean \(0\.463 vs\. 0\.450\); both are single runs\. The depth prior alone thus does not recover what the type weights in the hand table yields at that width, and the two allocators trade perplexity against zero\-shot accuracy\.

Table 2:Rank allocators at the same bpw\. WikiText\-2 PPL with zero\-shot mean\. “Measured” is probed sensitivity×\\timesdepth prior\.Table 3:End\-to\-end wall\-clock time for reproduced NanoQuant and ShamAN\-Q on Qwen3\-Base at11bpw on 1×\\timesNVIDIA A100 80 GB\.
Table 4:Cumulative attribution at Qwen3\-1\.7B\-Base, uniform ranks, 0\.986 bpw, WikiText\-2 perplexity\.ConfigurationPPLNanoQuant, published19\.21NanoQuant, our reproduction18\.76NKP statistics, untempered19\.21\+\+temperingp=12p=\\tfrac\{1\}\{2\}17\.46\+\+fresh input statistic and refresh17\.28KL\-Shampoo statistics, untempered17\.04\+\+temperingp=12p=\\tfrac\{1\}\{2\}\(ShamAN\-Q\)16\.72

### 5\.3Quantization Cost

ShamAN\-Q uses the same deployment format as NanoQuant, so inference kernels, memory footprint, and decoding throughput are unchanged\. The cost is incurred during quantization, as shown in Table[4](https://arxiv.org/html/2609.38521#S5.T4)\. Appendix[D](https://arxiv.org/html/2609.38521#A4)gives the asymptotic accounting and Appendix[A](https://arxiv.org/html/2609.38521#A1)the timing breakdown and cost measures\.

### 5\.4Ablation Studies

Table[4](https://arxiv.org/html/2609.38521#S5.T4)adds the modifications of §[4](https://arxiv.org/html/2609.38521#S4)cumulatively at 1\.7B with uniform ranks\. Our reproduction of NanoQuant is already 0\.45 below the published number\. A dense NKP fit without tempering gives nothing over the diagonal baseline\. Tempering the same statistics comprises the single largest step, and replacing the NKP fit by the KL\-Shampoo fit improves both the untempered and the tempered models\. Only the tempering and rank\-allocation steps are individually outside noise\. The fresh input statistic and periodic refresh of §[4\.3](https://arxiv.org/html/2609.38521#S4.SS3)are a consistent gain at every block boundary, though their aggregate effect in Table[4](https://arxiv.org/html/2609.38521#S5.T4)is within single\-run noise\. For the tempering exponent,p=1p=1\(none\) andp=1/4p=1/4\(Shampoo\) were both worse thanp=1/2p=1/2at 1\.7B\.

## 6Conclusion

We introduced ShamAN\-Q, which replaces the diagonal reconstruction geometry in NanoQuant with a dense metric estimated by a KL\-Shampoo\-style Kronecker fit of the empirical Fisher Information matrix\. We also identify useful processing techniques for robust application of the curvature estimated under uncertainty, including shrinkage, tempering, and periodic refreshes\. The continuous ADMM updates of NanoQuant then become Sylvester equation solutions, which can be computed in a cached eigenbasis, while other components of NanoQuant, including crucially the packed format, remain unchanged, so inference can inherit the speed of the customized kernel\. At the same total number of bits, ShamAN\-Q lowers WikiText\-2 PPL and increases zero\-shot reasoning performance of≲1\\lesssim 1bpw Qwen3\-Base models from 0\.6B to 14B\. Future work includes parallelizing the computation and using further approximations to the dense curvature statistics that yield fast matrix multiplication, both to reduce wall clock time\. In addition, as the depth prior and calibration budget were tuned at 0\.6B, tuning would likely help, and understanding scaling behavior is left as a promising direction for future investigation\.

## AI use statement

In this work, we used generative AI tools for implementing methods\. We have not used generative AI tools for proposing or refining hypotheses, designing or providing feedback on research methodology or experiments, or interpreting results\. Generating synthetic data sets, helping develop theoretical models or conceptual frameworks, formulating mathematical claims, providing critical ingredients for proving mathematical claims, assisting in the writing of proofs, assisting with translation, cleaning and reformating datasets, and supporting qualitative and thematic data analysis are not applicable to this work\.

Additionally, we used generative AI tools for creating scientific figures, suggesting experimental parameters, brainstorming, identifying some relevant literature, and some editing\. We have reviewed all AI\-assisted work\. LLM\-generated code was verified via tests reviewed by humans prior to implementation\. AI\-generated figures were produced as TikZ that was manually revised by humans, and AI\-generated writing was manually reviewed by humans\. We take responsibility for the final content of this work, including text, claims or artifacts produced with the aid of generative AI\.

## Ethics statement

This work is intended to reduce the energy consumption required to run AI models, which has been a globally increasing concern due to scarcity of resources, limited production capacity, and environmental impact\. On the surface, we might expect that reducing the energy cost of model inference would have a positive impact on these issues\. However, this should be considered in the context of Jevon’s paradox, where a decreasing energy cost for using an item results in increased aggregate usage due to more widespread access and adoption to the point that total energy consumption due to the item increases\. This effect, while not guaranteed, remains a possibility here\.

## Reproducibility statement

To aid with reproducibility, we provide source code with configuration files to run experiments found in the paper, which we intend to open source as part of the publication\. The implementation details of the algorithms and relevant hyperparameters are contained within §[4](https://arxiv.org/html/2609.38521#S4)and appendices\. In addition, in the appendices we have included some discussion of directions that were unsuccessful in our explorations\.

## References

- Bengioet al\.\(2013\)Y\. Bengio, N\. Léonard, and A\. CourvilleEstimating or propagating gradients through stochastic neurons for conditional computation\.arXiv preprint arXiv:1308\.3432\.Cited by:[§3\.1](https://arxiv.org/html/2609.38521#S3.SS1.p2.1)\.
- Birnick and Saab \(2026\)J\. Birnick and R\. SaabBaKron: efficient quantization with kronecker\-factored hessians\.arXiv preprint arXiv:2608\.06291\.Cited by:[§2](https://arxiv.org/html/2609.38521#S2.p2.1)\.
- Boydet al\.\(2011\)S\. Boyd, N\. Parikh, E\. Chu, B\. Peleato, and J\. EcksteinDistributed optimization and statistical learning via the alternating direction method of multipliers\.Foundations and Trends in Machine Learning3\(1\),pp\. 1–122\.Cited by:[§2](https://arxiv.org/html/2609.38521#S2.p3.1)\.
- Chonget al\.\(2026\)H\. Chong, D\. Kim, C\. Kim, and M\. ChoiNanoQuant: Efficient Sub\-1\-Bit Quantization of Large Language Models\.InForty\-third International Conference on Machine Learning,External Links:[Link](https://openreview.net/forum?id=qiZDlnvWTR)Cited by:[§1](https://arxiv.org/html/2609.38521#S1.p2.1),[§2](https://arxiv.org/html/2609.38521#S2.p1.1),[§3\.1](https://arxiv.org/html/2609.38521#S3.SS1.p1.1),[§5\.1](https://arxiv.org/html/2609.38521#S5.SS1.p3.1),[Table 1](https://arxiv.org/html/2609.38521#S5.T1)\.
- Duchiet al\.\(2011\)J\. Duchi, E\. Hazan, and Y\. SingerAdaptive subgradient methods for online learning and stochastic optimization\.Journal of Machine Learning Research12\(61\),pp\. 2121–2159\.External Links:[Link](https://jmlr.org/papers/v12/duchi11a.html)Cited by:[§1](https://arxiv.org/html/2609.38521#S1.p3.1)\.
- Frantaret al\.\(2023\)E\. Frantar, S\. Ashkboos, T\. Hoefler, and D\. AlistarhGPTQ: accurate post\-training quantization for generative pre\-trained transformers\.InInternational Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2609.38521#S1.p2.1),[§1](https://arxiv.org/html/2609.38521#S1.p3.1),[§2](https://arxiv.org/html/2609.38521#S2.p1.1),[§2](https://arxiv.org/html/2609.38521#S2.p2.1)\.
- Gaoet al\.\(2024\)L\. Gao, J\. Tow, B\. Abbasi, S\. Biderman, S\. Black, A\. DiPofi, C\. Foster, L\. Golding, J\. Hsu, A\. Le Noac’h, H\. Li, K\. McDonell, N\. Muennighoff, C\. Ociepa, J\. Phang, L\. Reynolds, H\. Schoelkopf, A\. Skowron, L\. Sutawika, E\. Tang, A\. Thite, B\. Wang, K\. Wang, and A\. ZouThe language model evaluation harness\.Zenodo\.External Links:[Document](https://dx.doi.org/10.5281/zenodo.12608602),[Link](https://zenodo.org/records/12608602)Cited by:[§5\.1](https://arxiv.org/html/2609.38521#S5.SS1.p2.1)\.
- Guptaet al\.\(2018\)V\. Gupta, T\. Koren, and Y\. SingerShampoo: preconditioned stochastic tensor optimization\.InInternational Conference on Machine Learning,Cited by:[§1](https://arxiv.org/html/2609.38521#S1.p3.1),[§2](https://arxiv.org/html/2609.38521#S2.p2.1.2),[§3\.2](https://arxiv.org/html/2609.38521#S3.SS2.p1.1),[§3\.2](https://arxiv.org/html/2609.38521#S3.SS2.p2.1)\.
- Linet al\.\(2024\)J\. Lin, J\. Tang, H\. Tang, S\. Yang, W\. Chen, W\. Wang, G\. Xiao, X\. Dang, C\. Gan, and S\. HanAWQ: activation\-aware weight quantization for LLM compression and acceleration\.InProceedings of Machine Learning and Systems \(MLSys\),Cited by:[§1](https://arxiv.org/html/2609.38521#S1.p2.1),[§2](https://arxiv.org/html/2609.38521#S2.p1.1)\.
- Linet al\.\(2026\)W\. Lin, S\. C\. Lowe, F\. Dangel, R\. Eschenhagen, Z\. Xu, and R\. GrosseUnderstanding and improving Shampoo and SOAP via Kullback\-Leibler Minimization\.InInternational Conference on Learning Representations,C\. Vondrick, B\. Hariharan, C\. Raffel, L\. Pinto, D\. Yang, and A\. Faust \(Eds\.\),Vol\.2026,pp\. 92799–92826\.External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2026/file/95a704bd2fdf8ef8242b4adcc7ce3c93-Paper-Conference.pdf)Cited by:[§2](https://arxiv.org/html/2609.38521#S2.p2.1),[§3\.2](https://arxiv.org/html/2609.38521#S3.SS2.p3.1)\.
- Martens and Grosse \(2015\)J\. Martens and R\. GrosseOptimizing neural networks with kronecker\-factored approximate curvature\.InInternational Conference on Machine Learning,Cited by:[§1](https://arxiv.org/html/2609.38521#S1.p3.1),[§2](https://arxiv.org/html/2609.38521#S2.p2.1),[§3\.2](https://arxiv.org/html/2609.38521#S3.SS2.p1.1)\.
- Meiet al\.\(2023\)J\. Mei, A\. Moreno, and L\. WaltersKradagrad: kronecker approximation\-domination gradient preconditioned stochastic optimization\.InUncertainty in Artificial Intelligence,pp\. 1412–1422\.Cited by:[§2](https://arxiv.org/html/2609.38521#S2.p2.1),[§3\.2](https://arxiv.org/html/2609.38521#S3.SS2.p3.1)\.
- Merityet al\.\(2016\)S\. Merity, C\. Xiong, J\. Bradbury, and R\. SocherPointer sentinel mixture models\.arXiv preprint arXiv:1609\.07843\.Cited by:[§5\.1](https://arxiv.org/html/2609.38521#S5.SS1.p2.1)\.
- Morwaniet al\.\(2025\)D\. Morwani, I\. Shapira, N\. Vyas, E\. Malach, S\. Kakade, and L\. JansonA new perspective on shampoo’s preconditioner\.International Conference on Learning Representations2025,pp\. 5802–5822\(en\)\.Cited by:[§2](https://arxiv.org/html/2609.38521#S2.p2.1),[§3\.2](https://arxiv.org/html/2609.38521#S3.SS2.p2.2),[§3\.2](https://arxiv.org/html/2609.38521#S3.SS2.p3.1)\.
- Pouransariet al\.\(2020\)H\. Pouransari, Z\. Tu, and O\. TuzelLeast squares binary quantization of neural networks\.In2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops \(CVPRW\),pp\. 2986–2996\.Cited by:[1st item](https://arxiv.org/html/2609.38521#S1.I1.i1.p1.1)\.
- Qwen Team \(2025\)Qwen TeamQwen3 technical report\.arXiv preprint arXiv:2505\.09388\.Cited by:[§5\.1](https://arxiv.org/html/2609.38521#S5.SS1.p2.1)\.
- Shanget al\.\(2023\)Y\. Shang, Z\. Yuan, Q\. Wu, and Z\. DongPB\-LLM: partially binarized large language models\.arXiv preprint arXiv:2310\.00034\.Cited by:[§2](https://arxiv.org/html/2609.38521#S2.p1.1)\.
- Sunet al\.\(2024\)M\. Sun, X\. Chen, J\. Z\. Kolter, and Z\. LiuMassive activations in large language models\.InFirst Conference on Language Modeling \(COLM\),Note:arXiv:2402\.17762Cited by:[§2](https://arxiv.org/html/2609.38521#S2.p2.1),[§4\.1](https://arxiv.org/html/2609.38521#S4.SS1.SSS0.Px1.p1.1)\.
- Sun and Wei \(2026\)R\. Sun and E\. WeiPro\-KLShampoo: Projected KL\-Shampoo with Whitening Recovered by Orthogonalization\.arXiv preprint arXiv:2605\.06316\.Cited by:[§2](https://arxiv.org/html/2609.38521#S2.p2.1),[§3\.2](https://arxiv.org/html/2609.38521#S3.SS2.p3.1)\.
- Tuet al\.\(2026\)Z\. Tu, J\. Li, Y\. Xi, S\. Liu, C\. Liu, H\. Chen, J\. Hu, and Y\. WangRethinking 1\-bit optimization leveraging pre\-trained large language models\.InProceedings of the 43rd International Conference on Machine Learning \(ICML\),External Links:[Link](https://openreview.net/forum?id=xdS18gI4e8)Cited by:[§2](https://arxiv.org/html/2609.38521#S2.p1.1)\.
- Wanget al\.\(2023\)H\. Wang, S\. Ma, L\. Dong, S\. Huang, H\. Wang, L\. Ma, F\. Yang, R\. Wang, Y\. Wu, and F\. WeiBitNet: scaling 1\-bit transformers for large language models\.arXiv preprint arXiv:2310\.11453\.Cited by:[§2](https://arxiv.org/html/2609.38521#S2.p1.1)\.
- Xuet al\.\(2020\)J\. Xu, X\. Chen, S\. Hu, J\. Yu, X\. Liu, and H\. MengLow\-bit quantization of recurrent neural network language models using alternating direction methods of multipliers\.InICASSP 2020\-2020 IEEE International Conference on Acoustics, Speech and Signal Processing \(ICASSP\),pp\. 7939–7943\.Cited by:[1st item](https://arxiv.org/html/2609.38521#S1.I1.i1.p1.1)\.
- Xuet al\.\(2024\)Y\. Xu, X\. Han, Z\. Yang, S\. Wang, Q\. Zhu, Z\. Liu, W\. Liu, and W\. CheOneBit: towards extremely low\-bit large language models\.InAdvances in Neural Information Processing Systems,Cited by:[§2](https://arxiv.org/html/2609.38521#S2.p1.1)\.

## Appendix AImplementation Details

#### Hyperparameters\.

All runs follow the protocol used by NanoQuant wherever §[4](https://arxiv.org/html/2609.38521#S4)does not change it: 128 WikiText\-2 calibration sequences of 2048 tokens selected with seed 0 \(512 for the curvature statistics at 8B and 14B\), shrinkageγ=0\.2\\gamma=0\.2, NanoQuant’s robust token clipping,K=400K=400LB\-ADMM iterations with the linearρk\\rho\_\{k\}schedule, two FP16 scale vectors per layer, andTuneFP/TuneLatentSTE/TuneScalesKDwith learning rates10−4/10−5/10−610^\{\-4\}/10^\{\-5\}/10^\{\-6\}, batch sizes4/1/14/1/1, 8 epochs each \(at 8B and 14B the first two stages use 2, 4, 6, and 8 epochs for the q and k, v and o, gate, and up and down layers\) with cosine decay, NanoQuant’s weighted MSE block loss, and forward\-KL scale\-only distillation\. ShamAN\-Q adds three KL\-Shampoo passes at calibration, the tempering exponentp=12p=\\tfrac\{1\}\{2\}, the refresh periodΔ=7\\Delta=7for the 28\-block 0\.6B and 1\.7B models andΔ=9\\Delta=9for the 36\-block 4B model and 8B models, andΔ=10\\Delta=10for the 40\-block 14B model, the fresh input statistic, and the rank allocator listed in Table[1](https://arxiv.org/html/2609.38521#S5.T1)\. The measured allocator probes each layer at\{0\.5,1,1\.5\}×rℓ⋆\\\{0\.5,1,1\.5\\\}\\times r^\{\\star\}\_\{\\ell\}at 0\.6B and 1\.7B and at\{0\.6,1,1\.4\}×rℓ⋆\\\{0\.6,1,1\.4\\\}\\times r^\{\\star\}\_\{\\ell\}from 4B upward, withK=50K=50\. Everyn×nn\\times nandm×mm\\times meigendecomposition in the reported runs is computed in fp64\.

#### Keeping the dense recipe within budget\.

The recipe costs more than NanoQuant per block: dense statistics per layer, one eigendecomposition per statistic per ADMM call, the fresh input measurement, the refresh passes, the rank probe, and late blocks whose allocated ranks reach 2560 at 4B \(§[5\.3](https://arxiv.org/html/2609.38521#S5.SS3)\)\. With allocated ranks at 4B, ADMM accounts for 260 s of a 473 s block at block 7, and for about 40 % of the block with uniform ranks\. Four measures keep this within budget\. \(i\) The fp32 statistics of all layers do not fit next to the model at 4B \(about 36 GB\), so the calibration passes accumulate the Kronecker statistics on the GPU over layer groups sized to a memory budget instead of streaming to the CPU, reducing 4B calibration from about 5 h to about 35 min\. The KL fit’s per\-pass inverses are Cholesky solves on the accumulation device, reducing 1\.7B calibration from 55 to 9 min\. \(ii\) The fresh input statistic of §[4\.3](https://arxiv.org/html/2609.38521#S4.SS3)is measured once per shared\-input group and its eigendecomposition cached for the group’s ADMM calls, saving three of seven measurements and three large eigendecompositions per block\. \(iii\) Then×nn\\times neigendecompositions can run in fp32: the eigenvalues are clamped and the result feeds a sign projection, and 1\.7B perplexity was unchanged \(16\.66 vs\. 16\.72 with fp64\)\. The reported 1\.7B and 4B results used fp64, so this is a cost measure only\. \(iv\) Calibration statistics, the rank probe, every per\-layer ADMM solution \(content\-addressed by weight bytes, curvature tensors, and ADMM settings\), per\-block checkpoints, and the pre\-distillation model are cached, so an interrupted run resumes from its last block in minutes\. Refresh costs are small, as demonstrated by how three refreshes cost 4 min in total at 1\.7B\. The rank probe cost is given in Appendix[B](https://arxiv.org/html/2609.38521#A2)\.

## Appendix BRank Allocation Details

This appendix completes §[4\.2](https://arxiv.org/html/2609.38521#S4.SS2)\. Both allocators start from the uniform ranksrℓunir^\{\\mathrm\{uni\}\}\_\{\\ell\}of Eq\.[11](https://arxiv.org/html/2609.38521#S4.E11), keep their totalNbitsN\_\{\\mathrm\{bits\}\}, and move ranks in steps of 32 with ceilingmin⁡\(mℓ,nℓ\)\\min\(m\_\{\\ell\},n\_\{\\ell\}\)\.

#### Hand table\.

The per\-layer budget isβℓ=β​ζ​ωℓ\\beta\_\{\\ell\}=\\beta\\,\\zeta\\,\\omega\_\{\\ell\}with the depth priorωℓ\\omega\_\{\\ell\}of §[4\.2](https://arxiv.org/html/2609.38521#S4.SS2)and type weightsw=\(q​0\.85,k​0\.90,v​1\.10,o​1\.00,gate​1\.00,up​1\.05,down​1\.15\)w=\(\\text\{q\}\\ 0\.85,\\ \\text\{k\}\\ 0\.90,\\ \\text\{v\}\\ 1\.10,\\ \\text\{o\}\\ 1\.00,\\ \\text\{gate\}\\ 1\.00,\\ \\text\{up\}\\ 1\.05,\\ \\text\{down\}\\ 1\.15\), set from the median relative block\-loss jump each layer type caused when binarized in the 4B logs, andζ\\zetachosen so that∑ℓβℓ​mℓ​nℓ=β​∑ℓmℓ​nℓ\\sum\_\{\\ell\}\\beta\_\{\\ell\}m\_\{\\ell\}n\_\{\\ell\}=\\beta\\sum\_\{\\ell\}m\_\{\\ell\}n\_\{\\ell\}\. Ranks are obtained fromβℓ\\beta\_\{\\ell\}by Eq\.[11](https://arxiv.org/html/2609.38521#S4.E11), floored to multiples of 32, and then moved in±32\\pm 32steps, largest relative deviation fromβℓ\\beta\_\{\\ell\}first, until the total equalsNbitsN\_\{\\mathrm\{bits\}\}\.

#### Measured sensitivity times depth prior\.

After calibration, every layer is solved by a short run of the dense\-curvature LB\-ADMM of §[4\.3](https://arxiv.org/html/2609.38521#S4.SS3)\(K=50K=50\) at the three candidate ranks\{0\.5,1,1\.5\}×rℓ⋆\\\{0\.5,1,1\.5\\\}\\times r^\{\\star\}\_\{\\ell\}at 0\.6B and 1\.7B and\{0\.6,1,1\.4\}×rℓ⋆\\\{0\.6,1,1\.4\\\}\\times r^\{\\star\}\_\{\\ell\}from 4B upward, and the deployed matrix is scored by the Gauss–Newton weight error

Jℓ​\(r\)=‖𝐖ℓ−𝐖^ℓ​\(r\)‖𝐆γ,ℓ,𝐀γ,ℓ2,J\_\{\\ell\}\(r\)=\\left\\lVert\\mathbf\{W\}\_\{\\ell\}\-\\widehat\{\\mathbf\{W\}\}\_\{\\ell\}\(r\)\\right\\rVert\_\{\\mathbf\{G\}\_\{\\gamma,\\ell\},\\mathbf\{A\}\_\{\\gamma,\\ell\}\}^\{2\},\(17\)with the layer’s own shrunken statistics before the trace normalization, so that values are comparable across layers\. A power lawJℓ​\(r\)=eaℓ​r−ηℓJ\_\{\\ell\}\(r\)=e^\{a\_\{\\ell\}\}r^\{\-\\eta\_\{\\ell\}\}is fitted through the three probes and its level multiplied by the depth prior of §[4\.2](https://arxiv.org/html/2609.38521#S4.SS2)with allwt=1w\_\{t\}=1\. Starting from rank 32 everywhere, 32\-rank steps are handed to the layer with the largest predicted loss decrease per added bit,\(Jℓ​\(r\)−Jℓ​\(r\+32\)\)/\(32​\(mℓ\+nℓ\)\)\\big\(J\_\{\\ell\}\(r\)\-J\_\{\\ell\}\(r\+32\)\\big\)/\\big\(32\(m\_\{\\ell\}\+n\_\{\\ell\}\)\\big\), untilNbitsN\_\{\\mathrm\{bits\}\}is reached\. The measured curves rank the seven layer types within a block well \(Spearman0\.80\.8–0\.960\.96against logged block\-loss jumps\) but under\-value late blocks, which is what the prior supplies\. The probe shares each statistic’s eigendecomposition across its three candidate ranks and is cached, taking 707 s at 0\.6B\.

## Appendix CAlgorithms

This appendix lists the procedures of §[4\.3](https://arxiv.org/html/2609.38521#S4.SS3)and the unchanged NanoQuant components they call \(§[3\.1](https://arxiv.org/html/2609.38521#S3.SS1)\), written in the transformed coordinates\.

#### Diagonal maps and transformed target \(Step 2\-1\)\.

With the maximum applied elementwise, the diagonal preconditioners are

𝐝in\\displaystyle\\mathbf\{d\}\_\{\\mathrm\{in\}\}=max⁡\{diag⁡\(𝐀γ\),ϵ​𝟏n\},\\displaystyle=\\sqrt\{\\max\\\{\\operatorname\{diag\}\(\\mathbf\{A\}\_\{\\gamma\}\),\\epsilon\\mathbf\{1\}\_\{n\}\\\}\},𝐃~in\\displaystyle\\widetilde\{\\mathbf\{D\}\}\_\{\\mathrm\{in\}\}=Diag⁡\(𝐝in\),\\displaystyle=\\operatorname\{Diag\}\(\\mathbf\{d\}\_\{\\mathrm\{in\}\}\),𝐝out\\displaystyle\\mathbf\{d\}\_\{\\mathrm\{out\}\}=max⁡\{diag⁡\(𝐆γ\),ϵ​𝟏m\},\\displaystyle=\\sqrt\{\\max\\\{\\operatorname\{diag\}\(\\mathbf\{G\}\_\{\\gamma\}\),\\epsilon\\mathbf\{1\}\_\{m\}\\\}\},𝐃~out\\displaystyle\\widetilde\{\\mathbf\{D\}\}\_\{\\mathrm\{out\}\}=Diag⁡\(𝐝out\),\\displaystyle=\\operatorname\{Diag\}\(\\mathbf\{d\}\_\{\\mathrm\{out\}\}\),\(18\)and the transformed target, transformed error, and cached metric target are

𝐖~=𝐃~out𝐖𝐃~in,𝐄~=𝐃~out𝐄𝐃~in,𝐌:⁣=𝐆¯𝐖~𝐀¯,\\widetilde\{\\mathbf\{W\}\}=\\widetilde\{\\mathbf\{D\}\}\_\{\\mathrm\{out\}\}\\mathbf\{W\}\\widetilde\{\\mathbf\{D\}\}\_\{\\mathrm\{in\}\},\\qquad\\widetilde\{\\mathbf\{E\}\}=\\widetilde\{\\mathbf\{D\}\}\_\{\\mathrm\{out\}\}\\mathbf\{E\}\\widetilde\{\\mathbf\{D\}\}\_\{\\mathrm\{in\}\},\\qquad\\mathbf\{M\}\\mathrel\{:=\}\\overline\{\\mathbf\{G\}\}\\,\\widetilde\{\\mathbf\{W\}\}\\,\\overline\{\\mathbf\{A\}\},\(19\)where𝐌\\mathbf\{M\}uses the tempered transported statistics of Eq\.[13](https://arxiv.org/html/2609.38521#S4.E13)and is cached together with their eigensystems\. Algorithm[1](https://arxiv.org/html/2609.38521#alg1)lists the step\.

Algorithm 1PrepareMetricTargetfor one layer \(Step 2\-1\)\.0:

𝐖\\mathbf\{W\}, shrunken statistics

\(𝐀γ,𝐆γ\)\(\\mathbf\{A\}\_\{\\gamma\},\\mathbf\{G\}\_\{\\gamma\}\), with

𝐀γ\\mathbf\{A\}\_\{\\gamma\}from the fresh measurement of §[4\.3](https://arxiv.org/html/2609.38521#S4.SS3), floor

ϵ\\epsilon, tempering exponent

pp
1:Form

\(𝐃~in,𝐃~out\)\(\\widetilde\{\\mathbf\{D\}\}\_\{\\mathrm\{in\}\},\\widetilde\{\\mathbf\{D\}\}\_\{\\mathrm\{out\}\}\)by Eq\.[18](https://arxiv.org/html/2609.38521#A3.E18)

2:

𝐖~←𝐃~out​𝐖​𝐃~in\\widetilde\{\\mathbf\{W\}\}\\leftarrow\\widetilde\{\\mathbf\{D\}\}\_\{\\mathrm\{out\}\}\\mathbf\{W\}\\widetilde\{\\mathbf\{D\}\}\_\{\\mathrm\{in\}\}
3:Transport to

\(𝐀¯,𝐆¯\)\(\\overline\{\\mathbf\{A\}\},\\overline\{\\mathbf\{G\}\}\)by Eq\.[12](https://arxiv.org/html/2609.38521#S4.E12), do not refit

4:Eigendecompose

𝐀¯,𝐆¯\\overline\{\\mathbf\{A\}\},\\overline\{\\mathbf\{G\}\}and temper both by Eq\.[13](https://arxiv.org/html/2609.38521#S4.E13)

5:

𝐌←𝐆¯​𝐖~​𝐀¯\\mathbf\{M\}\\leftarrow\\overline\{\\mathbf\{G\}\}\\widetilde\{\\mathbf\{W\}\}\\overline\{\\mathbf\{A\}\}, cache the tempered eigensystems

6:return

𝐖~\\widetilde\{\\mathbf\{W\}\}, both diagonal maps, and the tempered metric state

#### Continuous block updates \(Step 2\-2\)\.

Setαk=ρk\+λ\\alpha\_\{k\}=\\rho\_\{k\}\+\\lambda\. Holding𝐕k\\mathbf\{V\}^\{k\}fixed, define

𝐑Vk=\(𝐕k\)⊤​𝐀¯​𝐕k,𝐁Uk=𝐌𝐕k\+ρk​𝐂Uk,\\mathbf\{R\}\_\{V\}^\{k\}=\(\\mathbf\{V\}^\{k\}\)^\{\\top\}\\overline\{\\mathbf\{A\}\}\\mathbf\{V\}^\{k\},\\qquad\\mathbf\{B\}\_\{U\}^\{k\}=\\mathbf\{M\}\\mathbf\{V\}^\{k\}\+\\rho\_\{k\}\\mathbf\{C\}\_\{U\}^\{k\},\(20\)so that Eq\.[15](https://arxiv.org/html/2609.38521#S4.E15)reads𝐆¯​𝐔k\+1​𝐑Vk\+αk​𝐔k\+1=𝐁Uk\\overline\{\\mathbf\{G\}\}\\,\\mathbf\{U\}^\{k\+1\}\\mathbf\{R\}\_\{V\}^\{k\}\+\\alpha\_\{k\}\\mathbf\{U\}^\{k\+1\}=\\mathbf\{B\}\_\{U\}^\{k\}\. After this update, define

𝐑Uk\+1=\(𝐔k\+1\)⊤​𝐆¯​𝐔k\+1,𝐁Vk=𝐌⊤​𝐔k\+1\+ρk​𝐂Vk,\\mathbf\{R\}\_\{U\}^\{k\+1\}=\(\\mathbf\{U\}^\{k\+1\}\)^\{\\top\}\\overline\{\\mathbf\{G\}\}\\mathbf\{U\}^\{k\+1\},\\qquad\\mathbf\{B\}\_\{V\}^\{k\}=\\mathbf\{M\}^\{\\top\}\\mathbf\{U\}^\{k\+1\}\+\\rho\_\{k\}\\mathbf\{C\}\_\{V\}^\{k\},\(21\)which yields

𝐀¯​𝐕k\+1​𝐑Uk\+1\+αk​𝐕k\+1=𝐁Vk\.\\overline\{\\mathbf\{A\}\}\\,\\mathbf\{V\}^\{k\+1\}\\mathbf\{R\}\_\{U\}^\{k\+1\}\+\\alpha\_\{k\}\\mathbf\{V\}^\{k\+1\}=\\mathbf\{B\}\_\{V\}^\{k\}\.\(22\)Both systems are solved by Eq\.[16](https://arxiv.org/html/2609.38521#S4.E16), and these two solves are denoted byKronUSolveandKronVSolvein Algorithm[2](https://arxiv.org/html/2609.38521#alg2)\.

#### Euclidean proxy projection\.

For a proxy𝐏∈ℝd×r\\mathbf\{P\}\\in\\mathbb\{R\}^\{d\\times r\}, SVID computes

𝐒=sign⁡\(𝐏\),\(𝐚,σ,𝐛\)=TopSingularTriplet​\(\|𝐏\|\),\\mathbf\{S\}=\\sign\(\\mathbf\{P\}\),\\qquad\(\\mathbf\{a\},\\sigma,\\mathbf\{b\}\)=\\textsc\{TopSingularTriplet\}\(\|\\mathbf\{P\}\|\),\(23\)with nonnegative unit vectors𝐚,𝐛\\mathbf\{a\},\\mathbf\{b\}and zeros in𝐏\\mathbf\{P\}mapped to\+1\+1bysign\\sign\. Defining𝐩=σ​𝐚\\mathbf\{p\}=\\sqrt\{\\sigma\}\\mathbf\{a\}and𝐪=σ​𝐛\\mathbf\{q\}=\\sqrt\{\\sigma\}\\mathbf\{b\}, the projected proxy is

SVID​\(𝐏\)=\(σ​𝐚𝐛⊤\)⊙𝐒=Diag⁡\(𝐩\)​𝐒​Diag⁡\(𝐪\)\.\\textsc\{SVID\}\(\\mathbf\{P\}\)=\(\\sigma\\mathbf\{a\}\\mathbf\{b\}^\{\\top\}\)\\odot\\mathbf\{S\}=\\operatorname\{Diag\}\(\\mathbf\{p\}\)\\mathbf\{S\}\\operatorname\{Diag\}\(\\mathbf\{q\}\)\.\(24\)The triplet is obtained by the same power iteration as NanoQuant\. Since the consensus penalties in Eq\.[14](https://arxiv.org/html/2609.38521#S4.E14)are Euclidean, neither Eq\.[24](https://arxiv.org/html/2609.38521#A3.E24)nor its inner iteration count depends on𝐀¯\\overline\{\\mathbf\{A\}\}or𝐆¯\\overline\{\\mathbf\{G\}\}\.

#### Scale extraction \(Step 2\-3\)\.

The final proxies𝐏U,𝐏V\\mathbf\{P\}\_\{U\},\\mathbf\{P\}\_\{V\}live in the transformed coordinates of Eq\.[19](https://arxiv.org/html/2609.38521#A3.E19)\. NanoQuant’s magnitude balancing \(its Eq\. 7\) and scale extraction \(its Eq\. 8\) are applied to them verbatim, yielding sign matrices𝐒U,𝐒V\\mathbf\{S\}\_\{U\},\\mathbf\{S\}\_\{V\}and transformed scale vectors𝐬~1,𝐬~2\\widetilde\{\\mathbf\{s\}\}\_\{1\},\\widetilde\{\\mathbf\{s\}\}\_\{2\}\. The return to the original weight coordinates is

𝐬1=𝐝out−1⊙𝐬~1,𝐬2=𝐝in−1⊙𝐬~2,\\mathbf\{s\}\_\{1\}=\\mathbf\{d\}\_\{\\mathrm\{out\}\}^\{\-1\}\\odot\\widetilde\{\\mathbf\{s\}\}\_\{1\},\\qquad\\mathbf\{s\}\_\{2\}=\\mathbf\{d\}\_\{\\mathrm\{in\}\}^\{\-1\}\\odot\\widetilde\{\\mathbf\{s\}\}\_\{2\},\(25\)which is exactly NanoQuant’s un\-preconditioning step with the diagonal maps of Eq\.[18](https://arxiv.org/html/2609.38521#A3.E18)\.Finalizein Algorithm[2](https://arxiv.org/html/2609.38521#alg2)denotes these operations\.

Algorithm 2Kron\-LB\-ADMMfor one transformed layer \(Step 2\-2\)\.0:Prepared state from Algorithm[1](https://arxiv.org/html/2609.38521#alg1), rank

rr, iterations

KK, penalty schedule

ρk\\rho\_\{k\}, ridge

λ\\lambda
1:Initialize

𝐔0,𝐕0\\mathbf\{U\}^\{0\},\\mathbf\{V\}^\{0\}as in NanoQuant

2:

𝐙U0←SVID​\(𝐔0\)\\mathbf\{Z\}\_\{U\}^\{0\}\\leftarrow\\textsc\{SVID\}\(\\mathbf\{U\}^\{0\}\),

𝐙V0←SVID​\(𝐕0\)\\mathbf\{Z\}\_\{V\}^\{0\}\\leftarrow\\textsc\{SVID\}\(\\mathbf\{V\}^\{0\}\)
3:Initialize scaled duals

𝚲U0,𝚲V0\\bm\{\\Lambda\}\_\{U\}^\{0\},\\bm\{\\Lambda\}\_\{V\}^\{0\}
4:for

k=0,…,K−1k=0,\\ldots,K\-1do

5:

𝐂Uk←𝐙Uk−𝚲Uk\\mathbf\{C\}\_\{U\}^\{k\}\\leftarrow\\mathbf\{Z\}\_\{U\}^\{k\}\-\\bm\{\\Lambda\}\_\{U\}^\{k\},

𝐂Vk←𝐙Vk−𝚲Vk\\mathbf\{C\}\_\{V\}^\{k\}\\leftarrow\\mathbf\{Z\}\_\{V\}^\{k\}\-\\bm\{\\Lambda\}\_\{V\}^\{k\}
6:

𝐔k\+1←KronUSolve​\(𝐕k,𝐂Uk,𝐆¯,𝐀¯,𝐌,ρk,λ\)\\mathbf\{U\}^\{k\+1\}\\leftarrow\\textsc\{KronUSolve\}\(\\mathbf\{V\}^\{k\},\\mathbf\{C\}\_\{U\}^\{k\};\\overline\{\\mathbf\{G\}\},\\overline\{\\mathbf\{A\}\},\\mathbf\{M\},\\rho\_\{k\},\\lambda\)
7:

𝐕k\+1←KronVSolve​\(𝐔k\+1,𝐂Vk,𝐆¯,𝐀¯,𝐌,ρk,λ\)\\mathbf\{V\}^\{k\+1\}\\leftarrow\\textsc\{KronVSolve\}\(\\mathbf\{U\}^\{k\+1\},\\mathbf\{C\}\_\{V\}^\{k\};\\overline\{\\mathbf\{G\}\},\\overline\{\\mathbf\{A\}\},\\mathbf\{M\},\\rho\_\{k\},\\lambda\)
8:

𝐙Uk\+1←SVID​\(𝐔k\+1\+𝚲Uk\)\\mathbf\{Z\}\_\{U\}^\{k\+1\}\\leftarrow\\textsc\{SVID\}\(\\mathbf\{U\}^\{k\+1\}\+\\bm\{\\Lambda\}\_\{U\}^\{k\}\)
9:

𝐙Vk\+1←SVID​\(𝐕k\+1\+𝚲Vk\)\\mathbf\{Z\}\_\{V\}^\{k\+1\}\\leftarrow\\textsc\{SVID\}\(\\mathbf\{V\}^\{k\+1\}\+\\bm\{\\Lambda\}\_\{V\}^\{k\}\)
10:

𝚲Uk\+1←𝚲Uk\+𝐔k\+1−𝐙Uk\+1\\bm\{\\Lambda\}\_\{U\}^\{k\+1\}\\leftarrow\\bm\{\\Lambda\}\_\{U\}^\{k\}\+\\mathbf\{U\}^\{k\+1\}\-\\mathbf\{Z\}\_\{U\}^\{k\+1\}
11:

𝚲Vk\+1←𝚲Vk\+𝐕k\+1−𝐙Vk\+1\\bm\{\\Lambda\}\_\{V\}^\{k\+1\}\\leftarrow\\bm\{\\Lambda\}\_\{V\}^\{k\}\+\\mathbf\{V\}^\{k\+1\}\-\\mathbf\{Z\}\_\{V\}^\{k\+1\}, update Euclidean residuals

12:endfor

13:

𝐏U←𝐔K\+𝚲UK\\mathbf\{P\}\_\{U\}\\leftarrow\\mathbf\{U\}^\{K\}\+\\bm\{\\Lambda\}\_\{U\}^\{K\},

𝐏V←𝐕K\+𝚲VK\\mathbf\{P\}\_\{V\}\\leftarrow\\mathbf\{V\}^\{K\}\+\\bm\{\\Lambda\}\_\{V\}^\{K\}
14:return

Finalize​\(𝐏U,𝐏V,𝐃~out,𝐃~in\)\\textsc\{Finalize\}\(\\mathbf\{P\}\_\{U\},\\mathbf\{P\}\_\{V\},\\widetilde\{\\mathbf\{D\}\}\_\{\\mathrm\{out\}\},\\widetilde\{\\mathbf\{D\}\}\_\{\\mathrm\{in\}\}\)

#### End\-to\-end pipeline\.

Algorithm[3](https://arxiv.org/html/2609.38521#alg3)shows the complete pipeline\. Calibration statistics are first collected from the full\-precision teacher, so every initial KL\-Shampoo fit is defined in the original layer coordinates, and ranks are allocated once\. AfterTuneFPand the fresh measurement,𝐖~\\widetilde\{\\mathbf\{W\}\}, the transported and tempered statistics, and𝐌\\mathbf\{M\}are recomputed for the current target\. Latent tuning and final knowledge distillation act on𝒰,𝒱,𝐬1,𝐬2\\mathcal\{U\},\\mathcal\{V\},\\mathbf\{s\}\_\{1\},\\mathbf\{s\}\_\{2\}exactly as in NanoQuant\.

Algorithm 3Dense\-curvature ShamAN\-Q\.0:Full\-precision teacher

ℳ\\mathcal\{M\}, calibration set

𝒳cal\\mathcal\{X\}\_\{\\mathrm\{cal\}\}, bit budget

β\\betaand allocator \(§[4\.2](https://arxiv.org/html/2609.38521#S4.SS2)\),

γ,ϵ,p,Δ\\gamma,\\epsilon,p,\\Delta, ADMM parameters

K,ρ,λK,\\rho,\\lambda, tuning budgets

Tpre,Tpost,TglobT\_\{\\mathrm\{pre\}\},T\_\{\\mathrm\{post\}\},T\_\{\\mathrm\{glob\}\}
1:// Phase 1: global calibration and rank allocation

2:Collect

\(𝐱t,𝜹t\)\(\\mathbf\{x\}\_\{t\},\\bm\{\\delta\}\_\{t\}\)for every linear layer on

𝒳cal\\mathcal\{X\}\_\{\\mathrm\{cal\}\}through

ℳ\\mathcal\{M\}
3:foreach linear layer

ℓ\\elldo

4:

\(𝐀,𝐆\)\(ℓ\)←KL\-Shampoo\(\\mathbf\{A\},\\mathbf\{G\}\)^\{\(\\ell\)\}\\leftarrow\\textsc\{KL\-Shampoo\}by Eq\.[9](https://arxiv.org/html/2609.38521#S3.E9), three passes, shrink by Eq\.[10](https://arxiv.org/html/2609.38521#S4.E10)

5:endfor

6:Allocate ranks

\{rℓ\}\\\{r\_\{\\ell\}\\\}at bit parity \(§[4\.2](https://arxiv.org/html/2609.38521#S4.SS2)\), probing with

K=50K=50if the measured allocator is used

7:// Phase 2: block reconstruction

8:

ℳ^←ℳ\\widehat\{\\mathcal\{M\}\}\\leftarrow\\mathcal\{M\}
9:forblocks

b=1,…,Bb=1,\\ldots,Bdo

10:if

b\>1b\>1and

\(b−1\)modΔ=0\(b\-1\)\\bmod\\Delta=0then

11:Refresh

\(𝐀,𝐆\)\(ℓ\)\(\\mathbf\{A\},\\mathbf\{G\}\)^\{\(\\ell\)\}of all unfactorized layers with one warm\-started pass of Eq\.[9](https://arxiv.org/html/2609.38521#S3.E9)through

ℳ^\\widehat\{\\mathcal\{M\}\}, shrink

12:endif

13:

𝐗b←ℳ^<b​\(𝒳cal\)\\mathbf\{X\}\_\{b\}\\leftarrow\\widehat\{\\mathcal\{M\}\}\_\{<b\}\(\\mathcal\{X\}\_\{\\mathrm\{cal\}\}\),

𝐘b←ℬbFP​\(𝐗b\)\\mathbf\{Y\}\_\{b\}\\leftarrow\\mathcal\{B\}\_\{b\}^\{\\mathrm\{FP\}\}\(\\mathbf\{X\}\_\{b\}\)
14:

TuneFP​\(b,𝐗b,𝐘b,Tpre\)\\textsc\{TuneFP\}\(b,\\mathbf\{X\}\_\{b\},\\mathbf\{Y\}\_\{b\};T\_\{\\mathrm\{pre\}\}\)// Step 1

15:forshared\-input groups

g∈bg\\in b\(q k v, o, gate up, down\)do

16:Measure

𝐀fresh\(g\)\\mathbf\{A\}\_\{\\mathrm\{fresh\}\}^\{\(g\)\}\(§[4\.3](https://arxiv.org/html/2609.38521#S4.SS3)\) on the current inputs of

gg, shrink by Eq\.[10](https://arxiv.org/html/2609.38521#S4.E10)

17:forlinear layers

ℓ∈g\\ell\\in gwith weight

𝐖\(ℓ\)\\mathbf\{W\}^\{\(\\ell\)\}do

18:PrepareMetricTarget\(Algorithm[1](https://arxiv.org/html/2609.38521#alg1)\) with

𝐀γ←𝐀fresh,γ\(g\)\\mathbf\{A\}\_\{\\gamma\}\\leftarrow\\mathbf\{A\}\_\{\\mathrm\{fresh\},\\gamma\}^\{\(g\)\}and the current

𝐆γ\(ℓ\)\\mathbf\{G\}\_\{\\gamma\}^\{\(\\ell\)\}// Step 2\-1

19:

\(𝒰,𝒱,𝐬1,𝐬2\)\(ℓ\)←Kron\-LB\-ADMM\(\\mathcal\{U\},\\mathcal\{V\},\\mathbf\{s\}\_\{1\},\\mathbf\{s\}\_\{2\}\)^\{\(\\ell\)\}\\leftarrow\\textsc\{Kron\-LB\-ADMM\}at rank

rℓr\_\{\\ell\}by Algorithm[2](https://arxiv.org/html/2609.38521#alg2)// Steps 2\-2, 2\-3

20:endfor

21:endfor

22:

TuneLatentSTE​\(b,𝐗b,𝐘b,Tpost\)\\textsc\{TuneLatentSTE\}\(b,\\mathbf\{X\}\_\{b\},\\mathbf\{Y\}\_\{b\};T\_\{\\mathrm\{post\}\}\)over

𝒰,𝒱,𝐬1,𝐬2\\mathcal\{U\},\\mathcal\{V\},\\mathbf\{s\}\_\{1\},\\mathbf\{s\}\_\{2\}// Step 3

23:forlinear layers

ℓ∈b\\ell\\in bdo

24:

𝐔±1\(ℓ\)←sign⁡\(𝒰\(ℓ\)\)\\mathbf\{U\}\_\{\\pm 1\}^\{\(\\ell\)\}\\leftarrow\\sign\(\\mathcal\{U\}^\{\(\\ell\)\}\),

𝐕±1\(ℓ\)←sign⁡\(𝒱\(ℓ\)\)\\mathbf\{V\}\_\{\\pm 1\}^\{\(\\ell\)\}\\leftarrow\\sign\(\\mathcal\{V\}^\{\(\\ell\)\}\)
25:

PackBinary​\(𝐔±1\(ℓ\),𝐕±1\(ℓ\)\)\\textsc\{PackBinary\}\(\\mathbf\{U\}\_\{\\pm 1\}^\{\(\\ell\)\},\\mathbf\{V\}\_\{\\pm 1\}^\{\(\\ell\)\}\)
26:endfor

27:endfor

28:// Phase 3: model reconstruction

29:

TuneScalesKD​\(ℳ^,ℳ,𝒳cal,Tglob\)\\textsc\{TuneScalesKD\}\(\\widehat\{\\mathcal\{M\}\},\\mathcal\{M\},\\mathcal\{X\}\_\{\\mathrm\{cal\}\};T\_\{\\mathrm\{glob\}\}\)over

𝐬1,𝐬2\\mathbf\{s\}\_\{1\},\\mathbf\{s\}\_\{2\}
30:return

ℳ^\\widehat\{\\mathcal\{M\}\}

## Appendix DComplexity Analysis

Table[5](https://arxiv.org/html/2609.38521#A4.T5)separates calibration, preparation, and one outer ADMM sweep for a layer withNNcalibration tokens\. The dense setup includes forming𝐌\\mathbf\{M\}and eigendecomposing the two transported statistics\.

Table 5:Per\-layer costs for anm×nm\\times nweight, factor rankrr, andNNcalibration tokens\.Each layer performs two continuous solves per outer iteration and two SVID projections at initialization plus two per iteration\. For representationx∈\{diagonal,dense\}x\\in\\\{\\mathrm\{diagonal\},\\mathrm\{dense\}\\\}, its reconstruction runtime can be decomposed as

Tx,ℓ=Sx,ℓ\+Kout,ℓ​Cx,ℓouter\+\(Kout,ℓ\+1\)​CℓSVID\+Ndiag,ℓ​Dx,ℓ,T\_\{x,\\ell\}=S\_\{x,\\ell\}\+K\_\{\\mathrm\{out\},\\ell\}C\_\{x,\\ell\}^\{\\mathrm\{outer\}\}\+\(K\_\{\\mathrm\{out\},\\ell\}\+1\)C\_\{\\ell\}^\{\\mathrm\{SVID\}\}\+N\_\{\\mathrm\{diag\},\\ell\}D\_\{x,\\ell\},\(26\)whereSSis setup,CouterC^\{\\mathrm\{outer\}\}is one pair of continuous solves,CSVIDC^\{\\mathrm\{SVID\}\}is one pair of proxy projections, andDDcovers diagnostic evaluations\. This accounting makes the one\-time dense setup explicit rather than hiding it in the iteration cost\. In practice, the\(m2\+n2\)​r\(m^\{2\}\+n^\{2\}\)rterm does not dominate: at 4B, where allocated ranks reachr≈1500r\\approx 1500–25002500and each layer performs 800r×rr\\times reigendecompositions overK=400K=400iterations, removing everyn2​rn^\{2\}rproduct changes per\-layer ADMM time by only 5–8 %, so the per\-iterationr×rr\\times reigendecomposition dominates instead \(Appendix[E](https://arxiv.org/html/2609.38521#A5)\)\.

## Appendix EDesign Alternatives Not Adopted

The following variants were evaluated and are not part of ShamAN\-Q\.

Variants within noise\.A dense Mahalanobis*block*loss forTuneFPandTuneLatentSTE\(NanoQuant’s weighted MSE is kept\), a spike\-plus\-flat projection of the curvature statistics, distilling the latent binaries in the KD stage, feature distillation, trainable norm weights, and best\-epoch selection in the KD stage, a logit\-level objective for the last blocks, lifting the rank ceiling, and spending the rounding remainder to reach exactly 1\.0 bpw were all within single\-run noise\. An explicit per\-rank middle scale vector between the two sign matrices \(§[4\.3](https://arxiv.org/html/2609.38521#S4.SS3)\), carried from ADMM through tuning and KD, was 3\.5–5 perplexity*worse*at 0\.6B: a mean\-one rebalanced export of the same vector was within noise, and a sweep of its learning rate was flat to negative\. On the 0\.6B four\-block screen used for tuning\-budget changes \(block\-3 perplexity 14\.15 for the control, single\-run noise about 0\.2\), a plateau stop that ends an epoch loop once the block loss improves by less than10−210^\{\-2\}was within noise \(14\.23\), while running oneTuneFPround per shared\-input group \(four rounds per block instead of seven\) cost\+0\.28\+0\.28, outside noise, so every round matters, including those between layers that read the same activation\.

Speed variants\.An inexact Sylvester solve \(a stale eigenbasis as preconditioner with Rayleigh, QR, and conjugate\-gradient rungs\) saved about 10 % of ADMM time at 1\.7B and nothing at 4B, and early stopping of ADMM never fires: at iteration 400 every layer still flips between 30 and 20,000 signs per iteration under the linearρk\\rho\_\{k\}schedule\. A low\-rank\-plus\-identity Sylvester step for spectrally projected statistics, which keeps theksk\_\{\\mathrm\{s\}\}largest andkdk\_\{\\mathrm\{d\}\}smallest eigenvalues of each transported statistic exactly and replaces the rest by their geometric mean so that each update costsO⁡\(n​k​r\+n​r2\)O\(nkr\+nr^\{2\}\)instead ofO⁡\(n2​r\)O\(n^\{2\}r\)withk=ks\+kdk=k\_\{\\mathrm\{s\}\}\+k\_\{\\mathrm\{d\}\}, saved only 5–8 % of 4B ADMM time, because ther×rr\\times reigendecomposition and not then2​rn^\{2\}rproducts dominates at those widths \(Appendix[D](https://arxiv.org/html/2609.38521#A4)\): the projection itself cost\+0\.10\+0\.10to\+0\.34\+0\.34perplexity at block 3 withk=64k=64per side and was within noise atk=256k=256\. It is not used\.