Understanding LLM Quantization through Activation-Guided Compensation and Orthogonal Residuals

# Understanding LLM Quantization through Activation-Guided Compensation and Orthogonal Residuals
Source: [https://arxiv.org/html/2609.21450](https://arxiv.org/html/2609.21450)
Issei SatoDepartment of Computer Science, The University of Tokyo\{narita\-yamato644sato\}@g\.ecc\.u\-tokyo\.ac\.jp

###### Abstract

Post\-training weight\-activation quantization reduces the memory and inference costs of large language models, but aggressive W4A4 quantization remains difficult because activation outliers degrade effective quantization resolution\. Although weight optimization, channel\-wise scaling, and orthogonal rotation mitigate this problem, the error components they address and their relationship remain unclear\. Using an exact decomposition of local weight\-activation quantization error into an activation\-guided weight compensation term and an orthogonal residual, we bound the residual using persistent channel\-wise outlier and regular activation quantities\. This decomposition clarifies which error components can be addressed by weight compensation and which require transformation design\. We then use the residual bounds to derive practical guidelines for applying randomized Hadamard rotation, sign selection, and channel scaling\. In particular, the analysis explains how random signs suppress constructive interference among persistent outlier channels, how sampling multiple sign patterns can improve transformation selection, and how second\-moment balancing leads to anL2L\_\{2\}scaling rule while a further relaxation recovers SmoothQuant\-styleL∞L\_\{\\infty\}scaling\. We evaluate these guidelines through backpropagation\-free configurations across eight Llama and Mistral models, obtaining performance competitive with gradient\-trained SpinQuant\.

## 1Introduction

Large language models \(LLMs\) achieve strong performance across a wide range of tasks, but their growing memory and inference costs hinder practical deployment\. Post\-training quantization \(PTQ\) reduces these costs using only a small calibration set, yet aggressive 4\-bit weight and 4\-bit activation quantization \(W4A4\) remains challenging because large activation values enlarge quantization ranges and reduce the effective resolution available for ordinary values\([Frantar et al\., 2023](https://arxiv.org/html/2609.21450#bib.bib11);[Dettmers et al\., 2022](https://arxiv.org/html/2609.21450#bib.bib2);[Xiao et al\., 2023](https://arxiv.org/html/2609.21450#bib.bib1)\)\. Existing methods address this difficulty in different ways: GPTQ\-type methods optimize quantized weights to preserve full\-precision layer outputs, scaling\-based methods such as SmoothQuant redistribute quantization difficulty between activations and weights, and rotation\-based methods such as QuaRot and SpinQuant transform activation coordinates to improve low\-bit quantization\([Frantar et al\., 2023](https://arxiv.org/html/2609.21450#bib.bib11);[Xiao et al\., 2023](https://arxiv.org/html/2609.21450#bib.bib1);[Ashkboos et al\., 2024](https://arxiv.org/html/2609.21450#bib.bib4);[Liu et al\., 2025](https://arxiv.org/html/2609.21450#bib.bib9)\)\. Other hybrid and learned approaches combine or optimize these transformations in different ways\([Czakó et al\., 2025](https://arxiv.org/html/2609.21450#bib.bib13);[Shao et al\., 2024](https://arxiv.org/html/2609.21450#bib.bib8);[Sun et al\., 2025](https://arxiv.org/html/2609.21450#bib.bib10)\)\. Despite their effectiveness, it remains unclear which components of the quantization error these operations address and how their roles are related\.

In this paper, we address this gap through a local error\-decomposition framework for weight\-activation PTQ\. We exactly decompose the local quantization error into an activation\-guided weight compensation \(AGWC\) term and an orthogonal residual\. For a fixed transformation, weight optimization can reduce the AGWC term but cannot change the orthogonal residual\. This decomposition therefore separates the part of the quantization error that can be addressed by weight compensation from the part that requires transformation design\. By further analyzing the orthogonal residual, we clarify how rotation and scaling act on different error structures and provide theoretical guidance for how existing weight optimization, rotation, and scaling techniques should be combined\.

Our analysis further connects several design choices that are otherwise treated separately in existing quantization methods\. It explains the role of randomized rotation in controlling error associated with persistent activation outliers, and provides a principled basis for selecting among rotation patterns\. It also characterizes channel scaling through the balance between activation\-side and weight\-side quantities, relating second\-moment\-based scaling to SmoothQuant\-style scaling\. In this way, rotation, sign selection, and scaling can be understood as complementary operations guided by the structure of the residual error rather than as independent heuristics\.

To examine the practical implications of these theoretical guidelines, we evaluate backpropagation\-free configurations that follow the analysis, including signed online Hadamard rotation, calibration\-based sign selection, andL2L\_\{2\}channel scaling\. Across eight Llama and Mistral models, these theoretically guided choices achieve performance competitive with gradient\-trained SpinQuant without backpropagation\. These results demonstrate that the proposed error\-decomposition framework can provide practical guidance for configuring existing quantization techniques\.

To summarize, our contributions are as follows:

- •Using an exact decomposition of local weight\-activation quantization error into an activation\-guided weight compensation term and an orthogonal residual, we bound the residual through persistent channel\-wise outlier and regular activation quantities, clarifying how transformation design complements weight compensation\.
- •We analyze the orthogonal residual to provide theoretical guidance for randomized rotation, sign selection, and channel scaling, placing these operations within a common framework\.
- •We derive a second\-moment bound on the regular quantity and obtain anL2L\_\{2\}scaling rule from its Frobenius surrogate, relating it to SmoothQuant\-styleL∞L\_\{\\infty\}scaling through a further relaxation\.
- •We validate these design principles in backpropagation\-free quantization configurations across multiple Llama and Mistral models, achieving performance competitive with gradient\-trained SpinQuant\.

## 2Related Work

#### Post\-training weight\-activation quantization\.

Post\-training quantization \(PTQ\) reduces the memory and inference costs of large language models using only a small calibration set, without requiring full retraining\([Frantar et al\., 2023](https://arxiv.org/html/2609.21450#bib.bib11);[Lin et al\., 2024](https://arxiv.org/html/2609.21450#bib.bib12);[Xiao et al\., 2023](https://arxiv.org/html/2609.21450#bib.bib1)\)\. Reconstruction\-based methods such as GPTQ optimize quantized weights to preserve full\-precision layer outputs\([Frantar et al\., 2023](https://arxiv.org/html/2609.21450#bib.bib11)\), while activation\-aware methods such as AWQ use activation statistics to identify and protect salient weight channels\([Lin et al\., 2024](https://arxiv.org/html/2609.21450#bib.bib12)\)\. Compared with weight\-only quantization, aggressive low\-bit weight\-activation quantization is more challenging because activation quantization introduces input\-dependent error, and large activation values expand the quantization range and reduce the effective resolution available for ordinary values\([Dettmers et al\., 2022](https://arxiv.org/html/2609.21450#bib.bib2);[Xiao et al\., 2023](https://arxiv.org/html/2609.21450#bib.bib1)\)\. In particular, persistent channel\-wise outliers, namely channels that remain consistently large across tokens, can repeatedly dominate per\-token activation quantization scales\([Xiao et al\., 2023](https://arxiv.org/html/2609.21450#bib.bib1);[Raman et al\., 2025](https://arxiv.org/html/2609.21450#bib.bib3)\)\. Raman et al\.\([Raman et al\., 2025](https://arxiv.org/html/2609.21450#bib.bib3)\)further provide an explicit statistical criterion for detecting such channels and empirically analyze their formation through normalization and weight\-matrix multiplication\. We therefore focus on persistent channel\-wise outliers as a principal activation\-side source of error in weight\-activation quantization\.

#### Output\-preserving transformations\.

Outlier\-aware quantization often applies transformations that preserve the full\-precision output while modifying activations and weights before quantization\. SmoothQuant transfers activation magnitudes to weights through output\-preserving channel\-wise scaling\([Xiao et al\., 2023](https://arxiv.org/html/2609.21450#bib.bib1)\)\. Rotation\-based methods include QuIP, which uses structured random orthogonal transformations to spread large weight entries and locally sensitive directions more evenly across coordinates before quantization\([Chee et al\., 2023](https://arxiv.org/html/2609.21450#bib.bib7)\), and QuIP\#, which efficiently implements a related preprocessing strategy using randomized Hadamard transforms\([Tseng et al\., 2024](https://arxiv.org/html/2609.21450#bib.bib5)\)\. QuaRot applies Hadamard rotations throughout the Transformer\([Ashkboos et al\., 2024](https://arxiv.org/html/2609.21450#bib.bib4)\), while SmoothRot combines Hadamard rotation with calibrated channel scaling\([Czakó et al\., 2025](https://arxiv.org/html/2609.21450#bib.bib13)\)\. DFRot empirically compares randomized orthogonal and Hadamard rotations under per\-token activation quantization\([Xiang and Zhang, 2025](https://arxiv.org/html/2609.21450#bib.bib24)\)\. SpinQuant learns quantization\-aware rotations through gradient\-based calibration\([Liu et al\., 2025](https://arxiv.org/html/2609.21450#bib.bib9)\)\. OmniQuant\([Shao et al\., 2024](https://arxiv.org/html/2609.21450#bib.bib8)\)and FlatQuant\([Sun et al\., 2025](https://arxiv.org/html/2609.21450#bib.bib10)\)learn more general equivalent transformations through gradient\-based calibration\. Our work develops a common analytical basis for these transformations through local quantization\-error decomposition\. The residual analysis explains how rotation and scaling complement weight compensation and guides their design without backpropagation\.

## 3Problem Setting

We first define the local reconstruction problem for a linear layer under an output\-preserving transformation\. We then introduce a simple decomposition of activations that separates persistent outlier structure from the remaining variation, which will be used to analyze rotation and scaling\.

### 3\.1Local Weight\-Activation Quantization

Consider a linear layer𝐘=𝐗𝐖⊤\\mathbf\{Y\}=\\mathbf\{X\}\\mathbf\{W\}^\{\\top\}, where𝐗∈ℝT×d\\mathbf\{X\}\\in\\mathbb\{R\}^\{T\\times d\}and𝐖∈ℝd′×d\\mathbf\{W\}\\in\\mathbb\{R\}^\{d^\{\\prime\}\\times d\}\. For an invertible transformation𝐏∈ℝd×d\\mathbf\{P\}\\in\\mathbb\{R\}^\{d\\times d\}, define𝐙=𝐗𝐏\\mathbf\{Z\}=\\mathbf\{X\}\\mathbf\{P\}and𝐕=𝐖𝐏−⁣⊤\\mathbf\{V\}=\\mathbf\{W\}\\mathbf\{P\}^\{\-\\top\}, so that𝐗𝐖⊤=𝐙𝐕⊤\\mathbf\{X\}\\mathbf\{W\}^\{\\top\}=\\mathbf\{Z\}\\mathbf\{V\}^\{\\top\}\. Thus,𝐏\\mathbf\{P\}changes the representation seen by the quantizers without changing the full\-precision output\. We isolate activation quantization at the current layer, excluding upstream errors, to clarify the effects of rotation and scaling\.

For activation quantization, letbAb\_\{A\}denote the quantized activation bit\-width, and setqA=2bA−1−1q\_\{A\}=2^\{b\_\{A\}\-1\}\-1, the maximum value after quantization\. For analytical tractability, we consider no\-clipping dynamic per\-token symmetric uniform quantization\. For each tokentt, define the dynamic scale byΔt=∥\(𝐙\)t,:∥∞/qA\\Delta\_\{t\}=\\\|\(\\mathbf\{Z\}\)\_\{t,:\}\\\|\_\{\\infty\}/q\_\{A\}\. Then the quantized activation𝐙~=𝒬A​\(𝐙\)=𝐙\+𝐀\\widetilde\{\\mathbf\{Z\}\}=\\mathcal\{Q\}\_\{A\}\(\\mathbf\{Z\}\)=\\mathbf\{Z\}\+\\mathbf\{A\}is given element\-wise by

\(𝐙~\)t,j=𝒬A​\(\(𝐙\)t,j\)=Δt​round⁡\(\(𝐙\)t,jΔt\),\(𝐀\)t,j=\(𝐙~\)t,j−\(𝐙\)t,j\.\\displaystyle\(\\widetilde\{\\mathbf\{Z\}\}\)\_\{t,j\}=\\mathcal\{Q\}\_\{A\}\(\(\\mathbf\{Z\}\)\_\{t,j\}\)=\\Delta\_\{t\}\\operatorname\{round\}\\left\(\\frac\{\(\\mathbf\{Z\}\)\_\{t,j\}\}\{\\Delta\_\{t\}\}\\right\),\\qquad\(\\mathbf\{A\}\)\_\{t,j\}=\(\\widetilde\{\\mathbf\{Z\}\}\)\_\{t,j\}\-\(\\mathbf\{Z\}\)\_\{t,j\}\.\(1\)For weight quantization, we write𝐕~=𝒬W​\(𝐕\)\\widetilde\{\\mathbf\{V\}\}=\\mathcal\{Q\}\_\{W\}\(\\mathbf\{V\}\)\. Unlike the activation quantizer above,𝒬W\\mathcal\{Q\}\_\{W\}is not restricted to a fixed rounding rule\. It may itself be the result of an optimization procedure, such as GPTQ, or any other feasible weight quantization method\.

We investigate the local reconstruction objective

𝒥⁡\(𝐏,𝐕~\)=1T​‖𝐙~​𝐕~⊤−𝐙𝐕⊤‖F2\.\\displaystyle\\mathcal\{J\}\(\\mathbf\{P\},\\widetilde\{\\mathbf\{V\}\}\)=\\frac\{1\}\{T\}\\left\\\|\\widetilde\{\\mathbf\{Z\}\}\\widetilde\{\\mathbf\{V\}\}^\{\\top\}\-\\mathbf\{Z\}\\mathbf\{V\}^\{\\top\}\\right\\\|\_\{F\}^\{2\}\.\(2\)
This objective measures the output reconstruction error of a single linear layer due to quantization of its input activations and weights\.

### 3\.2Persistent Channel\-Wise Outlier Decomposition

Under the per\-token activation quantizer introduced above, the quantization scale for tokenttis determined by∥\(𝐙\)t,:∥∞\\\|\(\\mathbf\{Z\}\)\_\{t,:\}\\\|\_\{\\infty\}, so a small number of consistently large activation channels can repeatedly dominate the quantization range and reduce the effective resolution available for the remaining coordinates\. Prior work has observed that activation outliers in LLMs often exhibit such a channel\-wise persistent structure across tokens\([Xiao et al\., 2023](https://arxiv.org/html/2609.21450#bib.bib1);[Raman et al\., 2025](https://arxiv.org/html/2609.21450#bib.bib3)\)\. Motivated by this observation, we represent each persistent outlier channel by a token\-shared level and assign the remaining token\-dependent variation to a regular component\. This decomposition serves as an analytical abstraction that separates a sparse persistent component, whose coordinates may coherently interfere after rotation, from the remaining activation variation, thereby enabling separate analyses of Hadamard rotation and channel\-wise scaling\.

To formalize this abstraction, let𝒞co​\(𝐗\)⊆\[d\]\\mathcal\{C\}\_\{\\mathrm\{co\}\}\{\}\(\\mathbf\{X\}\)\\subseteq\[d\]be the set of persistent channel\-wise outlier channels detected on calibration data\. For each CO channelk∈𝒞co​\(𝐗\)k\\in\\mathcal\{C\}\_\{\\mathrm\{co\}\}\{\}\(\\mathbf\{X\}\), letLkL\_\{k\}denote its persistent CO level\. We decompose the activation as𝐗=𝐗co\+𝐗reg\\mathbf\{X\}=\\mathbf\{X\}^\{\\mathrm\{co\}\}\+\\mathbf\{X\}^\{\\mathrm\{reg\}\}, where𝐗co\\mathbf\{X\}^\{\\mathrm\{co\}\}is defined by

\(𝐗co\)t,k=\{Lk,k∈𝒞co​\(𝐗\),0,k∉𝒞co​\(𝐗\),t∈\[T\],\\displaystyle\(\\mathbf\{X\}^\{\\mathrm\{co\}\}\)\_\{t,k\}=\\begin\{cases\}L\_\{k\},&k\\in\\mathcal\{C\}\_\{\\mathrm\{co\}\}\{\}\(\\mathbf\{X\}\),\\\\ 0,&k\\notin\\mathcal\{C\}\_\{\\mathrm\{co\}\}\{\}\(\\mathbf\{X\}\),\\end\{cases\}\\qquad t\\in\[T\],\(3\)and𝐗reg=𝐗−𝐗co\\mathbf\{X\}^\{\\mathrm\{reg\}\}=\\mathbf\{X\}\-\\mathbf\{X\}^\{\\mathrm\{co\}\}\. We refer to this decomposition as the persistent sparse CO model\. Under this model,𝐗co\\mathbf\{X\}^\{\\mathrm\{co\}\}captures the persistent activation component shared across tokens, while𝐗reg\\mathbf\{X\}^\{\\mathrm\{reg\}\}contains both the token\-dependent deviations of the CO channels and the activations of the remaining channels\.

We consider transformations of the form𝐏=𝚲−1​𝐃𝐇⊤\\mathbf\{P\}=\\mathbf\{\\Lambda\}^\{\-1\}\\mathbf\{D\}\\mathbf\{H\}^\{\\top\}, where𝚲=diag⁡\(λ1,…,λd\)\\mathbf\{\\Lambda\}=\\operatorname\{diag\}\(\\lambda\_\{1\},\\ldots,\\lambda\_\{d\}\)is a positive diagonal scaling matrix, with eachλk\>0\\lambda\_\{k\}\>0denoting the scaling coefficient for channelkk,𝐃\\mathbf\{D\}is a diagonal sign matrix, and𝐇\\mathbf\{H\}is the normalized Hadamard matrix\. The normalized Hadamard matrix is orthogonal, has entries of equal magnitude1/d1/\\sqrt\{d\}, and admits anO⁡\(d​log⁡d\)O\(d\\log d\)fast transform; combining it with random diagonal signs gives the randomized Hadamard transform commonly used for coordinate flattening and incoherence processing\([Ailon and Chazelle, 2006](https://arxiv.org/html/2609.21450#bib.bib23);[Tseng et al\., 2024](https://arxiv.org/html/2609.21450#bib.bib5)\)\. The diagonal scaling redistributes channel magnitudes between activations and weights, while the signed Hadamard transformation disperses activation coordinates across the transformed channel dimension\. Since the corresponding inverse transformation is absorbed into the weights, these operations preserve the full\-precision linear\-layer output\.

## 4Error Decomposition and Residual Bounds for Quantization

We first derive an exact decomposition of the local objective in Section[3](https://arxiv.org/html/2609.21450#S3)to separate the roles of weight compensation and transformation design\. We then use residual bounds to guide practical configurations: CO bounds motivate signed Hadamard rotation and sign sampling, while a second\-moment surrogate of the regular bound yields anL2L\_\{2\}scaling rule\.

### 4\.1Local Quantization Error Decomposition and Residual Bound

First, we exactly decompose the local quantization error and then bound the orthogonal residual to clarify the distinct roles of weight optimization, rotation, and scaling\.

###### Theorem 1\(Local quantization error decomposition and residual bound\)\.

Define\(𝐕⋆\)⊤:=𝐕⊤−𝐙~†​𝐀𝐕⊤\(\\mathbf\{V\}^\{\\star\}\)^\{\\top\}:=\\mathbf\{V\}^\{\\top\}\-\\widetilde\{\\mathbf\{Z\}\}^\{\\dagger\}\\mathbf\{A\}\\mathbf\{V\}^\{\\top\}and𝚷𝐙~:=𝐙~​𝐙~†\\mathbf\{\\Pi\}\_\{\\widetilde\{\\mathbf\{Z\}\}\}:=\\widetilde\{\\mathbf\{Z\}\}\\widetilde\{\\mathbf\{Z\}\}^\{\\dagger\}, where𝚷𝐙~\\mathbf\{\\Pi\}\_\{\\widetilde\{\\mathbf\{Z\}\}\}is the orthogonal projector onto the column space of𝐙~\\widetilde\{\\mathbf\{Z\}\}\. Then, the local quantization error𝒥⁡\(𝐏,𝐕~\)\\mathcal\{J\}\(\\mathbf\{P\},\\widetilde\{\\mathbf\{V\}\}\)in Eq\. \([2](https://arxiv.org/html/2609.21450#S3.E2)\) admits the following exact decomposition and upper bound:

𝒥⁡\(𝐏,𝐕~\)\\displaystyle\\mathcal\{J\}\(\\mathbf\{P\},\\widetilde\{\\mathbf\{V\}\}\)=𝒥AGWC​\(𝐏,𝐕~\)\+1T​‖\(𝐈−𝚷𝐙~\)​𝐀𝐕⊤‖F2\\displaystyle=\\mathcal\{J\}\_\{\\mathrm\{AGWC\}\}\(\\mathbf\{P\},\\widetilde\{\\mathbf\{V\}\}\)\+\\frac\{1\}\{T\}\\left\\\|\\left\(\\mathbf\{I\}\-\\mathbf\{\\Pi\}\_\{\\widetilde\{\\mathbf\{Z\}\}\}\\right\)\\mathbf\{A\}\\mathbf\{V\}^\{\\top\}\\right\\\|\_\{F\}^\{2\}\(4\)≤𝒥AGWC​\(𝐏,𝐕~\)\+d4​qA2​T​\(‖𝐖​𝚲‖22​𝒥co​\(𝚲,𝐃\)\+𝒥reg​\(𝚲,𝐃\)\)2,\\displaystyle\\leq\\mathcal\{J\}\_\{\\mathrm\{AGWC\}\}\(\\mathbf\{P\},\\widetilde\{\\mathbf\{V\}\}\)\+\\frac\{d\}\{4q\_\{A\}^\{2\}T\}\\left\(\\sqrt\{\\left\\\|\\mathbf\{W\}\\mathbf\{\\Lambda\}\\right\\\|\_\{2\}^\{2\}\\mathcal\{J\}\_\{\\mathrm\{co\}\}\(\\mathbf\{\\Lambda\},\\mathbf\{D\}\)\}\+\\sqrt\{\\mathcal\{J\}\_\{\\mathrm\{reg\}\}\(\\mathbf\{\\Lambda\},\\mathbf\{D\}\)\}\\right\)^\{2\},\(5\)where

𝒥AGWC​\(𝐏,𝐕~\)\\displaystyle\\mathcal\{J\}\_\{\\mathrm\{AGWC\}\}\(\\mathbf\{P\},\\widetilde\{\\mathbf\{V\}\}\)=1T​‖𝐙~​\(𝐕~−𝐕⋆\)⊤‖F2,\\displaystyle=\\frac\{1\}\{T\}\\left\\\|\\widetilde\{\\mathbf\{Z\}\}\\left\(\\widetilde\{\\mathbf\{V\}\}\-\\mathbf\{V\}^\{\\star\}\\right\)^\{\\top\}\\right\\\|\_\{F\}^\{2\},\(6\)𝒥co​\(𝚲,𝐃\)\\displaystyle\\mathcal\{J\}\_\{\\mathrm\{co\}\}\(\\mathbf\{\\Lambda\},\\mathbf\{D\}\)=∑t=1T‖\(𝐗co\)t,:𝚲−1𝐃𝐇⊤‖∞2,\\displaystyle=\\sum\_\{t=1\}^\{T\}\\left\\\|\(\\mathbf\{X\}^\{\\mathrm\{co\}\}\)\_\{t,:\}\\mathbf\{\\Lambda\}^\{\-1\}\\mathbf\{D\}\\mathbf\{H\}^\{\\top\}\\right\\\|\_\{\\infty\}^\{2\},\(7\)𝒥reg​\(𝚲,𝐃\)\\displaystyle\\mathcal\{J\}\_\{\\mathrm\{reg\}\}\(\\mathbf\{\\Lambda\},\\mathbf\{D\}\)=‖𝐖𝚲‖22∑t=1T‖\(𝐗reg\)t,:𝚲−1𝐃𝐇⊤‖∞2\.\\displaystyle=\\left\\\|\\mathbf\{W\}\\mathbf\{\\Lambda\}\\right\\\|\_\{2\}^\{2\}\\sum\_\{t=1\}^\{T\}\\left\\\|\(\\mathbf\{X\}^\{\\mathrm\{reg\}\}\)\_\{t,:\}\\mathbf\{\\Lambda\}^\{\-1\}\\mathbf\{D\}\\mathbf\{H\}^\{\\top\}\\right\\\|\_\{\\infty\}^\{2\}\.\(8\)

The proof is provided in Appendix[B\.1](https://arxiv.org/html/2609.21450#A2.SS1)\.

For full\-column\-rank𝐙~\\widetilde\{\\mathbf\{Z\}\}, this projection identity matches CoreQ’s mismatch decomposition\([Cha et al\., 2026](https://arxiv.org/html/2609.21450#bib.bib6)\)\. CoreQ uses this decomposition to limit overfitting to calibration data by adjusting how much the weights compensate for input errors\. We instead bound the transformation\-dependent residual through persistent CO and regular activation quantities to clarify how rotation and scaling complement weight compensation\.

We call𝒥AGWC\\mathcal\{J\}\_\{\\mathrm\{AGWC\}\}in Eq\. \([6](https://arxiv.org/html/2609.21450#S4.E6)\) the*activation\-guided weight compensation*\(AGWC\) term: the activation discrepancy𝐀\\mathbf\{A\}guides the continuous target𝐕⋆\\mathbf\{V\}^\{\\star\}, and𝒥AGWC\\mathcal\{J\}\_\{\\mathrm\{AGWC\}\}measures the reconstruction error of the quantized weights relative to this target\. Geometrically, this term lies in the column space of𝐙~\\widetilde\{\\mathbf\{Z\}\}and can therefore be affected by changing the quantized weights, whereas the second term lies in the orthogonal complement of this space\. Thus, for a fixed transformation, weight optimization can reduce the AGWC term but cannot change the orthogonal residual\. The CO and regular quantities are not additional terms in the exact decomposition; they appear only in an upper bound on the orthogonal residual\. Theorem[1](https://arxiv.org/html/2609.21450#Thmtheorem1)thus separates the scope of weight compensation from the residual error requiring transformation design\. This distinction guides how weight optimization, rotation, and scaling can be combined\.

### 4\.2Weight Compensation through the Lens of Error Decomposition

Theorem[1](https://arxiv.org/html/2609.21450#Thmtheorem1)provides a common perspective on existing weight\-reconstruction and compensation methods\. When𝐀=𝟎\\mathbf\{A\}=\\bm\{0\},𝐕⋆=𝐕\\mathbf\{V\}^\{\\star\}=\\mathbf\{V\}and𝒥AGWC\\mathcal\{J\}\_\{\\mathrm\{AGWC\}\}reduces to the standard GPTQ\-style reconstruction objective\([Frantar et al\., 2023](https://arxiv.org/html/2609.21450#bib.bib11)\)\. Thus, shared\-input reconstruction can be viewed as the special case in which no activation discrepancy needs to be compensated\. When the input discrepancy is nonzero, however, shared\-input reconstruction, even with quantized activations, does not explicitly compensate for this discrepancy\.

QEP\([Arai and Ichikawa, 2025](https://arxiv.org/html/2609.21450#bib.bib25)\)corrects weights before applying a quantizer such as GPTQ; with full compensation \(α=1\\alpha=1\), matching inputs and reference outputs, and an undamped invertible Hessian, its target equals𝐕⋆\\mathbf\{V\}^\{\\star\}\. With GPTQ, the corrected weights are quantized sequentially, updating the remaining weights\. GPTAQ\([Li et al\., 2025](https://arxiv.org/html/2609.21450#bib.bib26)\)incorporates input\-error compensation into sequential updates derived from asymmetric reconstruction, using channel\-wise residual components for efficiency\. For fixed inputs and𝐏\\mathbf\{P\}, Eq\. \([4](https://arxiv.org/html/2609.21450#S4.E4)\) makes𝒥\\mathcal\{J\}and𝒥AGWC\\mathcal\{J\}\_\{\\mathrm\{AGWC\}\}equivalent up to a weight\-independent constant over the same feasible set, without implying identical algorithmic outputs\. The orthogonal residual remains unchanged by weight compensation, so the CO and regular quantities in its bound guide complementary rotation and scaling\.

In practice, we adopt backpropagation\-free GPTAQ with activation quantization enabled before weight calibration to account for local and propagated input errors against full\-precision reference outputs\([Li et al\., 2025](https://arxiv.org/html/2609.21450#bib.bib26)\)\.

### 4\.3Randomized Hadamard Rotation for𝒥co\\mathcal\{J\}\_\{\\mathrm\{co\}\}

We now analyze the rotation\-dependent CO quantity𝒥co​\(𝚲,𝐃\)\\mathcal\{J\}\_\{\\mathrm\{co\}\}\(\\mathbf\{\\Lambda\},\\mathbf\{D\}\)in the residual upper bound of Theorem[1](https://arxiv.org/html/2609.21450#Thmtheorem1)and clarify how randomized Hadamard rotation controls this quantity\.

###### Theorem 2\(Hadamard rotation bounds\)\.

Under the persistent sparse CO model defined in Eq\. \([3](https://arxiv.org/html/2609.21450#S3.E3)\), let

Nco​\(𝐗\)=\|𝒞co​\(𝐗\)\|,LΛ,max=maxk∈𝒞co​\(𝐗\)⁡\|Lk\|λk\.\\displaystyle N\_\{\\mathrm\{co\}\}\{\}\(\\mathbf\{X\}\)=\\left\|\\mathcal\{C\}\_\{\\mathrm\{co\}\}\{\}\(\\mathbf\{X\}\)\\right\|,\\qquad L\_\{\\Lambda,\\max\}=\\max\_\{k\\in\\mathcal\{C\}\_\{\\mathrm\{co\}\}\{\}\(\\mathbf\{X\}\)\}\\frac\{\|L\_\{k\}\|\}\{\\lambda\_\{k\}\}\.\(9\)Then, the following three bounds hold:

#### Fixed rotation\.

If𝐃=𝐈\\mathbf\{D\}=\\mathbf\{I\}, then

𝒥co​\(𝚲,𝐈\)≤T​Nco​\(𝐗\)2d​LΛ,max2\.\\displaystyle\\mathcal\{J\}\_\{\\mathrm\{co\}\}\(\\mathbf\{\\Lambda\},\\mathbf\{I\}\)\\leq\\frac\{TN\_\{\\mathrm\{co\}\}\{\}\(\\mathbf\{X\}\)^\{2\}\}\{d\}L\_\{\\Lambda,\\max\}^\{2\}\.\(10\)

#### Randomized rotation\.

If𝐃=diag⁡\(ϵ1,…,ϵd\)\\mathbf\{D\}=\\operatorname\{diag\}\(\\epsilon\_\{1\},\\ldots,\\epsilon\_\{d\}\)with independent Rademacher signs, then for anyδ∈\(0,1\)\\delta\\in\(0,1\), with probability at least1−δ1\-\\delta,

𝒥co​\(𝚲,𝐃\)≤2​T​Nco​\(𝐗\)d​LΛ,max2​\(log⁡\(2​d\)\+log⁡1δ\)\.\\displaystyle\\mathcal\{J\}\_\{\\mathrm\{co\}\}\(\\mathbf\{\\Lambda\},\\mathbf\{D\}\)\\leq\\frac\{2TN\_\{\\mathrm\{co\}\}\{\}\(\\mathbf\{X\}\)\}\{d\}L\_\{\\Lambda,\\max\}^\{2\}\\left\(\\log\(2d\)\+\\log\\frac\{1\}\{\\delta\}\\right\)\.\(11\)

#### Sampled rotation\.

Let𝒟Ns=\{𝐃\(1\),…,𝐃\(Ns\)\}\\mathcal\{D\}\_\{N\_\{s\}\}=\\\{\\mathbf\{D\}^\{\(1\)\},\\ldots,\\mathbf\{D\}^\{\(N\_\{s\}\)\}\\\}be a set ofNsN\_\{s\}independently sampled random sign matrices\. Then, with probability at least1−δ1\-\\delta,

min𝐃∈𝒟Ns⁡𝒥co​\(𝚲,𝐃\)≤2​T​Nco​\(𝐗\)d​LΛ,max2​\(log⁡\(2​d\)\+1Ns​log⁡1δ\)\.\\displaystyle\\min\_\{\\mathbf\{D\}\\in\\mathcal\{D\}\_\{N\_\{s\}\}\}\\mathcal\{J\}\_\{\\mathrm\{co\}\}\(\\mathbf\{\\Lambda\},\\mathbf\{D\}\)\\leq\\frac\{2TN\_\{\\mathrm\{co\}\}\{\}\(\\mathbf\{X\}\)\}\{d\}L\_\{\\Lambda,\\max\}^\{2\}\\left\(\\log\(2d\)\+\\frac\{1\}\{N\_\{s\}\}\\log\\frac\{1\}\{\\delta\}\\right\)\.\(12\)
The proof is provided in Appendix[B\.2](https://arxiv.org/html/2609.21450#A2.SS2)\.

Theorem[2](https://arxiv.org/html/2609.21450#Thmtheorem2)identifies interference among persistent outlier channels as a mechanism governing the CO quantity in the residual bound\. Its fixed and randomized cases quantify this mechanism for existing Hadamard rotation schemes\([Tseng et al\., 2024](https://arxiv.org/html/2609.21450#bib.bib5);[Ashkboos et al\., 2024](https://arxiv.org/html/2609.21450#bib.bib4)\); the sampled case bounds the best CO quantity in a candidate set and motivates calibration\-based sign selection\. Because each entry of a normalized Hadamard matrix has magnitude1/d1/\\sqrt\{d\}, each CO channel contributes to every rotated coordinate at this scale; however, under a fixed rotation, contributions from multiple CO channels can constructively align, yielding the worst\-case dependenceNco​\(𝐗\)2/dN\_\{\\mathrm\{co\}\}\{\}\(\\mathbf\{X\}\)^\{2\}/d\. Random signs convert these contributions into Rademacher sums and reduce this dependence toNco​\(𝐗\)​log⁡d/dN\_\{\\mathrm\{co\}\}\{\}\(\\mathbf\{X\}\)\\log d/dup to logarithmic factors, while sampling multiple sign matrices increases the probability that the candidate set contains a favorable interference pattern\. Appendix[A](https://arxiv.org/html/2609.21450#A1)provides an interference\-based interpretation of the𝒪⁡\(Nco​\(𝐗\)2\)\\mathcal\{O\}\(N\_\{\\mathrm\{co\}\}\{\}\(\\mathbf\{X\}\)^\{2\}\)dependence under fixed rotation and its reduction to𝒪​\(Nco​\(𝐗\)\)\\mathcal\{O\}\(N\_\{\\mathrm\{co\}\}\{\}\(\\mathbf\{X\}\)\)under random signs\. Since both the number of persistent CO channelsNco​\(𝐗\)N\_\{\\mathrm\{co\}\}\{\}\(\\mathbf\{X\}\)and their scaled maximum magnitudeLΛ,maxL\_\{\\Lambda,\\max\}depend on the model, layer, and activation distribution, these quantities provide one possible explanation for why different models exhibit different levels of accuracy degradation under low\-bit quantization\.

### 4\.4DerivingL2L\_\{2\}Scaling from a Bound on𝒥reg\\mathcal\{J\}\_\{\\mathrm\{reg\}\}

We next consider the regular quantity𝒥reg\\mathcal\{J\}\_\{\\mathrm\{reg\}\}in the residual bound\. Unlike persistent COs, the regular component is not dominated by a small number of shared large coordinates\. After random\-sign Hadamard rotation, its maximum coordinate can be controlled by its second\-moment energy\.

###### Proposition 1\(L2L\_\{2\}scaling from second\-moment balancing\)\.

Let𝐃\\mathbf\{D\}be a random sign matrix with independent Rademacher signs\. For anyδ∈\(0,1\)\\delta\\in\(0,1\), with probability at least1−δ1\-\\delta,

𝒥reg​\(𝚲,𝐃\)≤2​log⁡\(2​d​T/δ\)d​‖𝐖​𝚲‖22​‖𝐗reg​𝚲−1‖F2\.\\displaystyle\\mathcal\{J\}\_\{\\mathrm\{reg\}\}\(\\mathbf\{\\Lambda\},\\mathbf\{D\}\)\\leq\\frac\{2\\log\(2dT/\\delta\)\}\{d\}\\left\\\|\\mathbf\{W\}\\mathbf\{\\Lambda\}\\right\\\|\_\{2\}^\{2\}\\left\\\|\\mathbf\{X\}^\{\\mathrm\{reg\}\}\\mathbf\{\\Lambda\}^\{\-1\}\\right\\\|\_\{F\}^\{2\}\.\(13\)Consequently, the regular quantity𝒥reg\\mathcal\{J\}\_\{\\mathrm\{reg\}\}is controlled by‖𝐖​𝚲‖22​‖𝐗reg​𝚲−1‖F2\\left\\\|\\mathbf\{W\}\\mathbf\{\\Lambda\}\\right\\\|\_\{2\}^\{2\}\\left\\\|\\mathbf\{X\}^\{\\mathrm\{reg\}\}\\mathbf\{\\Lambda\}^\{\-1\}\\right\\\|\_\{F\}^\{2\}\. Using the Frobenius surrogateℛ2​\(𝚲\)=‖𝐖​𝚲‖F2​‖𝐗reg​𝚲−1‖F2\\mathcal\{R\}\_\{2\}\(\\mathbf\{\\Lambda\}\)=\\left\\\|\\mathbf\{W\}\\mathbf\{\\Lambda\}\\right\\\|\_\{F\}^\{2\}\\left\\\|\\mathbf\{X\}^\{\\mathrm\{reg\}\}\\mathbf\{\\Lambda\}^\{\-1\}\\right\\\|\_\{F\}^\{2\}, we denote byλk\(2\)\\lambda\_\{k\}^\{\(2\)\}the channel\-wise scaling coefficient obtained by minimizingℛ2​\(𝚲\)\\mathcal\{R\}\_\{2\}\(\\mathbf\{\\Lambda\}\)\. This gives

λk\(2\)=‖\(𝐗reg\):,k‖2‖\(𝐖\):,k‖2\.\\displaystyle\\lambda\_\{k\}^\{\(2\)\}=\\sqrt\{\\frac\{\\left\\\|\(\\mathbf\{X\}^\{\\mathrm\{reg\}\}\)\_\{:,k\}\\right\\\|\_\{2\}\}\{\\left\\\|\(\\mathbf\{W\}\)\_\{:,k\}\\right\\\|\_\{2\}\}\}\.\(14\)

The proof is provided in Appendix[B\.3](https://arxiv.org/html/2609.21450#A2.SS3)\.

The resulting rule has the sameL2L\_\{2\}balancing form as QuIP’s rescaling\([Chee et al\., 2023](https://arxiv.org/html/2609.21450#bib.bib7)\)\. Note that QuIP motivates this scaling through a heuristic surrogate for output error in weight\-only quantization, whereas we derive it from a bound on the activation\-quantization residual\. This derivation also enables a common interpretation ofL2L\_\{2\}and SmoothQuant\-styleL∞L\_\{\\infty\}scaling, as discussed in Section[4\.5](https://arxiv.org/html/2609.21450#S4.SS5)\.

Proposition[1](https://arxiv.org/html/2609.21450#Thmproposition1)explains whyL2L\_\{2\}statistics are natural for the regular component\. The second\-moment bound on𝒥reg\\mathcal\{J\}\_\{\\mathrm\{reg\}\}combines the activation\-side energy‖𝐗reg​𝚲−1‖F2\\\|\\mathbf\{X\}^\{\\mathrm\{reg\}\}\\mathbf\{\\Lambda\}^\{\-1\}\\\|\_\{F\}^\{2\}with the weight\-side amplification factor‖𝐖​𝚲‖22\\\|\\mathbf\{W\}\\mathbf\{\\Lambda\}\\\|\_\{2\}^\{2\}\. In particular, the weight\-side spectral norm‖𝐖​𝚲‖2\\\|\\mathbf\{W\}\\mathbf\{\\Lambda\}\\\|\_\{2\}measures the worst\-case amplification induced by the scaled weight matrix and therefore reflects not only individual channel magnitudes but also cross\-channel interactions through the geometry of the weight columns\. Directly optimizing the resulting objective can therefore account for such interactions in principle\. However, the objective is not separable across channels, making it difficult to obtain a simple closed\-form channel\-wise scaling rule\.

To obtain a tractable channel\-wise rule, we use the Frobenius surrogateℛ2​\(𝚲\)\\mathcal\{R\}\_\{2\}\(\\mathbf\{\\Lambda\}\)\. This relaxation replaces the worst\-case coupled amplification by the total second\-moment energy of the scaled weight columns\. The surrogate is a product of sums of squared channel\-wise quantities:λk∥\(𝐖\):,k∥2\\lambda\_\{k\}\\\|\(\\mathbf\{W\}\)\_\{:,k\}\\\|\_\{2\}on the weight side and∥\(𝐗reg\):,k∥2/λk\\\|\(\\mathbf\{X\}^\{\\mathrm\{reg\}\}\)\_\{:,k\}\\\|\_\{2\}/\\lambda\_\{k\}on the activation side\. The equality condition in the Cauchy–Schwarz inequality yields theL2L\_\{2\}scaling rule in Proposition[1](https://arxiv.org/html/2609.21450#Thmproposition1)\. The proposition is stated in terms of the regular component𝐗reg\\mathbf\{X\}^\{\\mathrm\{reg\}\}to characterize the error component targeted by the scaling rule\. In practice, since𝐗reg\\mathbf\{X\}^\{\\mathrm\{reg\}\}is not explicitly identified, we approximate its channel\-wise second\-moment statistics using the empirical full activations, as described in Section[4\.6](https://arxiv.org/html/2609.21450#S4.SS6)\.

### 4\.5ComparingL2L\_\{2\}Scaling with SmoothQuant\-StyleL∞L\_\{\\infty\}

Our residual analysis connectsL2L\_\{2\}and SmoothQuant\-styleL∞L\_\{\\infty\}scaling through successive relaxations of the same regular\-term bound, providing a common interpretation not established by QuIP’s heuristic derivation\.

From Proposition[1](https://arxiv.org/html/2609.21450#Thmproposition1), after random\-sign Hadamard rotation, the regular quantity satisfies𝒥reg​\(𝚲,𝐃\)≲‖𝐖​𝚲‖22​‖𝐗reg​𝚲−1‖F2\\mathcal\{J\}\_\{\\mathrm\{reg\}\}\(\\mathbf\{\\Lambda\},\\mathbf\{D\}\)\\lesssim\\left\\\|\\mathbf\{W\}\\mathbf\{\\Lambda\}\\right\\\|\_\{2\}^\{2\}\\left\\\|\\mathbf\{X\}^\{\\mathrm\{reg\}\}\\mathbf\{\\Lambda\}^\{\-1\}\\right\\\|\_\{F\}^\{2\}\. OurL2L\_\{2\}scaling is obtained from a Frobenius surrogate of this second\-moment bound\. In contrast, a SmoothQuant\-style interpretation can be obtained by further upper bounding the same quantity usingL∞L\_\{\\infty\}channel statistics:

‖𝐖​𝚲‖22​‖𝐗reg​𝚲−1‖F2\\displaystyle\\left\\\|\\mathbf\{W\}\\mathbf\{\\Lambda\}\\right\\\|\_\{2\}^\{2\}\\left\\\|\\mathbf\{X\}^\{\\mathrm\{reg\}\}\\mathbf\{\\Lambda\}^\{\-1\}\\right\\\|\_\{F\}^\{2\}≤‖𝐖​𝚲‖F2​‖𝐗reg​𝚲−1‖F2\\displaystyle\\leq\\left\\\|\\mathbf\{W\}\\mathbf\{\\Lambda\}\\right\\\|\_\{F\}^\{2\}\\left\\\|\\mathbf\{X\}^\{\\mathrm\{reg\}\}\\mathbf\{\\Lambda\}^\{\-1\}\\right\\\|\_\{F\}^\{2\}≤d′T\(∑k=1dλk2‖\(𝐖\):,k‖∞2\)\(∑k=1d‖\(𝐗reg\):,k‖∞2λk2\)\.\\displaystyle\\leq d^\{\\prime\}T\\left\(\\sum\_\{k=1\}^\{d\}\\lambda\_\{k\}^\{2\}\\left\\\|\(\\mathbf\{W\}\)\_\{:,k\}\\right\\\|\_\{\\infty\}^\{2\}\\right\)\\left\(\\sum\_\{k=1\}^\{d\}\\frac\{\\left\\\|\(\\mathbf\{X\}^\{\\mathrm\{reg\}\}\)\_\{:,k\}\\right\\\|\_\{\\infty\}^\{2\}\}\{\\lambda\_\{k\}^\{2\}\}\\right\)\.\(15\)This relaxation is looser because it first replaces the spectral norm of the scaled weight matrix by its Frobenius norm and then replaces channel\-wise second\-moment quantities by their maximum magnitudes\. We denote byλk\(∞\)\\lambda\_\{k\}^\{\(\\infty\)\}the channel\-wise scaling coefficient obtained by minimizing the resultingL∞L\_\{\\infty\}surrogate\. Then, up to a common positive scalar,

λk\(∞\)=‖\(𝐗reg\):,k‖∞‖\(𝐖\):,k‖∞\.\\displaystyle\\lambda\_\{k\}^\{\(\\infty\)\}=\\sqrt\{\\frac\{\\left\\\|\(\\mathbf\{X\}^\{\\mathrm\{reg\}\}\)\_\{:,k\}\\right\\\|\_\{\\infty\}\}\{\\left\\\|\(\\mathbf\{W\}\)\_\{:,k\}\\right\\\|\_\{\\infty\}\}\}\.\(16\)This rule is stated in terms of the regular component𝐗reg\\mathbf\{X\}^\{\\mathrm\{reg\}\}to characterize the error component targeted by the scaling\. In practice, since𝐗reg\\mathbf\{X\}^\{\\mathrm\{reg\}\}is not explicitly identified, we approximate its channel\-wise maximum statistics using the empirical full activations, recovering the familiar SmoothQuant\-style scaling rule\.

Our framework thus explains both scaling rules through their retained statistics: theL2L\_\{2\}surrogate preserves channel\-wise second moments, whereas the SmoothQuant\-styleL∞L\_\{\\infty\}surrogate further relaxes them to channel maxima\. This provides a bound\-based rationale for usingL2L\_\{2\}statistics after randomized Hadamard rotation\.

### 4\.6From Theory to Practice: L2\-SmoothRot

The preceding analysis leads to L2\-SmoothRot, a simple configuration with three components: add random signs to online Hadamard rotations, select sign patterns using calibration data, and applyL2L\_\{2\}channel scaling\. Theorem[2](https://arxiv.org/html/2609.21450#Thmtheorem2)motivates the first two, while Proposition[1](https://arxiv.org/html/2609.21450#Thmproposition1)derives the scaling rule from a second\-moment surrogate\. We implement these choices without backpropagation within the QuaRot framework, following SmoothRot for scaling placement and fusion\.

#### Signed online rotation \(SOR\)\.

Theorem[2](https://arxiv.org/html/2609.21450#Thmtheorem2)shows that the sign structure of a Hadamard rotation is itself important: a fixed Hadamard transform can retain constructive interference among persistent CO channels, whereas random signs suppress this interference through Rademacher cancellation\. This suggests that the random\-sign principle should be applied not only to offline rotations, but also to online Hadamard rotations that directly act on activations before quantization\. In the original QuaRot, random signs are used only for offline rotation, whereas the online Hadamard rotations are fixed\. Motivated by this distinction, we insert sign flips before the online Hadamard rotations at the attention output and FFN down projections\. We also apply the same orthogonal signed rotation𝐑=𝐃𝐇⊤\\mathbf\{R\}=\\mathbf\{D\}\\mathbf\{H\}^\{\\top\}to queries and keys within each head after RoPE, before key quantization, preserving unquantized attention scores since\(𝐐𝐑\)​\(𝐊𝐑\)⊤=𝐐𝐊⊤\(\\mathbf\{Q\}\\mathbf\{R\}\)\(\\mathbf\{K\}\\mathbf\{R\}\)^\{\\top\}=\\mathbf\{Q\}\\mathbf\{K\}^\{\\top\}\.

#### Sign sampling \(SS\)\.

The sampled bound in Theorem[2](https://arxiv.org/html/2609.21450#Thmtheorem2)motivates searching multiple sign patterns by controlling the best CO quantity in the candidate set\. Together with observed rotation variability\([Liu et al\., 2025](https://arxiv.org/html/2609.21450#bib.bib9)\), this motivates calibration\-based selection without backpropagation\. We jointly select sign patterns for offline, attention\-output, FFN\-down, and QK rotations, generated from a single seed per candidate\. Each category shares its sign vector across layers, and QK additionally shares its vector across all query and key heads\. We screen candidates with RTN and select among the GPTAQ\-reevaluated finalists by validation perplexity \(Section[5](https://arxiv.org/html/2609.21450#S5)\)\.

#### L2L\_\{2\}scaling \(L2S\)\.

We use the scaling rule derived by minimizing the second\-moment surrogate in Proposition[1](https://arxiv.org/html/2609.21450#Thmproposition1), with empirical activation statistics as a proxy for the regular component\. Following SmoothRot, we apply scaling only to the FFN down\-projection input\. For the down\-projection input𝐗down\\mathbf\{X\}\_\{\\mathrm\{down\}\}and weight𝐖down\\mathbf\{W\}\_\{\\mathrm\{down\}\}, we use the transformation𝐗down​𝐖down⊤=\(𝐗down​𝚲−1\)​\(𝐖down​𝚲\)⊤\\mathbf\{X\}\_\{\\mathrm\{down\}\}\\mathbf\{W\}\_\{\\mathrm\{down\}\}^\{\\top\}=\(\\mathbf\{X\}\_\{\\mathrm\{down\}\}\\mathbf\{\\Lambda\}^\{\-1\}\)\(\\mathbf\{W\}\_\{\\mathrm\{down\}\}\\mathbf\{\\Lambda\}\)^\{\\top\}\. Since the regular component is not explicitly separated in practice, we use the empirical full\-activation norm as a readily computable proxy:

λk\(2\)=‖\(𝐗down\):,k‖2‖\(𝐖down\):,k‖2\.\\displaystyle\\lambda\_\{k\}^\{\(2\)\}=\\sqrt\{\\frac\{\\left\\\|\(\\mathbf\{X\}\_\{\\mathrm\{down\}\}\)\_\{:,k\}\\right\\\|\_\{2\}\}\{\\left\\\|\(\\mathbf\{W\}\_\{\\mathrm\{down\}\}\)\_\{:,k\}\\right\\\|\_\{2\}\}\}\.\(17\)As in SmoothRot, the scaling factors can be merged into the corresponding up\- and down\-projection weights and therefore introduce no additional inference\-time overhead\.

Overall, the decomposition in Theorem[1](https://arxiv.org/html/2609.21450#Thmtheorem1)and the bounds in Theorem[2](https://arxiv.org/html/2609.21450#Thmtheorem2)and Proposition[1](https://arxiv.org/html/2609.21450#Thmproposition1)provide a unified rationale for combining weight compensation with rotation and scaling\. Their effects are coupled, since changing the transformation can alter both terms of the exact decomposition\.

Table 1:W4A4 results with GPTAQ weight quantization and KV4 across model families and scales\. We report WikiText\-2 perplexity \(PPL\) and average zero\-shot accuracy over six tasks\.

## 5Experiments

#### Tasks and baselines\.

We evaluate Llama\-7B/13B\([Touvron et al\., 2023a](https://arxiv.org/html/2609.21450#bib.bib20)\), Llama2\-7B/13B\([Touvron et al\., 2023b](https://arxiv.org/html/2609.21450#bib.bib21)\), Llama3\-8B\([Grattafiori et al\., 2024](https://arxiv.org/html/2609.21450#bib.bib19)\), Llama3\.2\-1B/3B\([Meta AI, 2024](https://arxiv.org/html/2609.21450#bib.bib29)\), and Mistral\-7B\-v0\.3\([Jiang et al\., 2023](https://arxiv.org/html/2609.21450#bib.bib27);[Mistral AI, 2024](https://arxiv.org/html/2609.21450#bib.bib28)\)under W4A4 quantization\. We evaluate perplexity \(PPL\) on6464samples each from WikiText\-2 \(WT2\) test and a C4 validation subset\. For zero\-shot evaluation, we report accuracy on PIQA\([Bisk et al\., 2020](https://arxiv.org/html/2609.21450#bib.bib14)\), ARC\-Easy \(ARC\-E\) and ARC\-Challenge \(ARC\-C\)\([Clark et al\., 2018](https://arxiv.org/html/2609.21450#bib.bib15)\), HellaSwag \(HS\)\([Zellers et al\., 2019](https://arxiv.org/html/2609.21450#bib.bib16)\), WinoGrande \(WG\)\([Sakaguchi et al\., 2021](https://arxiv.org/html/2609.21450#bib.bib17)\), and LAMBADA \(LB\)\([Paperno et al\., 2016](https://arxiv.org/html/2609.21450#bib.bib18)\)\. We compare L2\-SmoothRot with QuaRot, SmoothRot, and SpinQuant\([Liu et al\., 2025](https://arxiv.org/html/2609.21450#bib.bib9)\), applying the same GPTAQ weight\-quantization settings to the transformed models produced by all four methods\.

#### Quantization and calibration\.

We simulate W4A4 decoder linear layers with KV4: weights use per\-output\-channel symmetric quantization, activations per\-token symmetric quantization, and K/V per\-token, per\-head asymmetric quantization\. Activation and K/V clipping ratios are0\.90\.9and0\.950\.95\. GPTAQ uses128128WikiText\-2 train samples\. Calibration includes local activation and propagated weight, activation, and K/V errors against corresponding unquantized\-path outputs\. Scaling statistics use the untransformed, unquantized model with512512WikiText\-2 train samples\. Following the rotation\-training protocol of SpinQuant\([Liu et al\., 2025](https://arxiv.org/html/2609.21450#bib.bib9)\), we train rotations on WikiText\-2 train for100100steps with frozen 16\-bit weights and 4\-bit activations and K/V, then quantize the rotated weights with GPTAQ\.

#### Rotation selection\.

For L2\-SmoothRot, we screen1010candidates with RTN, reevaluate the top33with GPTAQ, and select the lowest PPL on WikiText\-2 validation\.

#### Main results\.

Table[1](https://arxiv.org/html/2609.21450#S4.T1)summarizes the main W4A4 results across eight models from two families and from 1B to 13B parameters; Appendix[C](https://arxiv.org/html/2609.21450#A3)reports C4 perplexity and per\-task accuracies\. L2\-SmoothRot achieves the lowest WikiText\-2 perplexity on all eight models and the highest average zero\-shot accuracy on five\. It also achieves the lowest C4 perplexity on every model, indicating that the perplexity improvements are not limited to WikiText\-2, which is used for calibration and rotation selection\. On Llama3\-8B and Mistral\-7B\-v0\.3, its average accuracy is within0\.260\.26and0\.170\.17percentage points of SpinQuant, respectively; on Llama3\.2\-3B, it improves perplexity over SpinQuant while trailing its average accuracy by1\.091\.09points\. Overall, these results demonstrate the practical value of the transformation design guided by our analysis, with consistent perplexity gains and competitive downstream accuracy across model families and scales without backpropagation\.

Figure 1:RTN\-based seed\-selection ablation on Llama3\-8B with GPTAQ\.MethodLlama3\-8BQuaRotSOR \(𝒥co\\mathcal\{J\}\_\{\\mathrm\{co\}\}\)SS \(𝒥co\\mathcal\{J\}\_\{\\mathrm\{co\}\}\)L2S \(𝒥reg\\mathcal\{J\}\_\{\\mathrm\{reg\}\}\)PPL↓\\downarrow✓7\.45\[2pt/1pt\] ✓✓7\.30✓✓7\.40✓✓7\.32\[2pt/1pt\] ✓✓✓✓7\.20

Table 2:Component ablation on Llama3\-8B with GPTAQ \(W4A4KV4\)\. Parentheses indicate the residual\-bound term motivating each component\.

#### Component ablation\.

Table[2](https://arxiv.org/html/2609.21450#S5.T2)ablates the proposed components on Llama3\-8B under W4A4 quantization with GPTAQ\. Starting from QuaRot \(7\.457\.45\), adding SOR \(including QK rotation\), SS, or L2S individually reduces WikiText\-2 perplexity to7\.307\.30,7\.407\.40, and7\.327\.32, respectively, with SOR providing the largest individual gain\. Combining all components further reduces perplexity to7\.207\.20\. These gains are consistent with their complementary motivations: SOR and SS primarily target𝒥co\\mathcal\{J\}\_\{\\mathrm\{co\}\}, whereas L2S targets𝒥reg\\mathcal\{J\}\_\{\\mathrm\{reg\}\}, although their effects on quantization error need not be independent\.

#### Seed\-selection ablation\.

On Llama3\-8B with W4A4, KV4, and GPTAQ, we precompute RTN validation and GPTAQ validation/test perplexities on WikiText\-2 for100100random rotation seeds\. For eachN∈\{4,10,16,32\}N\\in\\\{4,10,16,32\\\}, we repeat10001000trials, samplingNNseeds without replacement and applying both selection rules to the same subset\. Top\-11selects the RTN validation winner; top\-33reranks the three best RTN candidates by GPTAQ validation perplexity\. Figure[1](https://arxiv.org/html/2609.21450#S5.F1)shows that both rules improve mean GPTAQ test perplexity over uniform selection \(the all\-seed mean\) and benefit from larger candidate sets\. Top\-33consistently outperforms top\-11, whose gains diminish betweenN=16N=16andN=32N=32\. Thus, RTN screening offers a low\-cost selection criterion: top\-11needs one GPTAQ run, while top\-33trades three runs for better selection\.

## 6Conclusion

In this paper, we analyzed local weight\-activation quantization error through an exact decomposition into an activation\-guided weight compensation term and an orthogonal residual\. Bounding the residual through persistent channel\-wise outlier and regular activation quantities clarifies how rotation and scaling complement weight optimization\. Within this framework, our bounds characterize outlier interference under fixed, randomized, and sampled Hadamard rotations, explaining the role of random signs in suppressing constructive interference and motivating calibration\-based sign selection\. For scaling, minimizing a second\-moment surrogate of the regular\-term bound yields anL2L\_\{2\}rule, while a further relaxation recovers a SmoothQuant\-styleL∞L\_\{\\infty\}rule\. We evaluated these design principles through backpropagation\-free configurations combining signed online rotation, sign sampling, andL2L\_\{2\}channel scaling\. Experiments across eight Llama and Mistral models demonstrated performance competitive with gradient\-trained SpinQuant without backpropagation\. Our analysis provides a principled basis for designing output\-preserving transformations for low\-bit weight\-activation quantization\.

## AI Use Statement

Generative AI tools were used to assist with writing and polishing the manuscript\. We reviewed and revised all AI\-assisted text\.

## References

- Ailon and Chazelle \(2006\)N\. Ailon and B\. ChazelleApproximate nearest neighbors and the fast johnson\-lindenstrauss transform\.InProceedings of the Thirty\-Eighth Annual ACM Symposium on Theory of Computing,STOC ’06,New York, NY, USA,pp\. 557–563\.External Links:ISBN 1595931341,[Link](https://doi.org/10.1145/1132516.1132597),[Document](https://dx.doi.org/10.1145/1132516.1132597)Cited by:[§3\.2](https://arxiv.org/html/2609.21450#S3.SS2.p3.1)\.
- Arai and Ichikawa \(2025\)Y\. Arai and Y\. IchikawaQuantization error propagation: revisiting layer\-wise post\-training quantization\.InAdvances in Neural Information Processing Systems,Vol\.38\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2025/hash/df2034a516cbd617a96492cc476276c9-Abstract-Conference.html)Cited by:[§4\.2](https://arxiv.org/html/2609.21450#S4.SS2.p2.1)\.
- Ashkbooset al\.\(2024\)S\. Ashkboos, A\. Mohtashami, M\. L\. Croci, B\. Li, P\. Cameron, M\. Jaggi, D\. Alistarh, T\. Hoefler, and J\. HensmanQuaRot: outlier\-free 4\-bit inference in rotated llms\.InAdvances in Neural Information Processing Systems,A\. Globerson, L\. Mackey, D\. Belgrave, A\. Fan, U\. Paquet, J\. Tomczak, and C\. Zhang \(Eds\.\),Vol\.37,pp\. 100213–100240\.External Links:[Document](https://dx.doi.org/10.52202/079017-3180),[Link](https://proceedings.neurips.cc/paper_files/paper/2024/file/b5b939436789f76f08b9d0da5e81af7c-Paper-Conference.pdf)Cited by:[§1](https://arxiv.org/html/2609.21450#S1.p1.1),[§2](https://arxiv.org/html/2609.21450#S2.SS0.SSS0.Px2.p1.1),[§4\.3](https://arxiv.org/html/2609.21450#S4.SS3.SSS0.Px3.p3.1.1)\.
- Bisket al\.\(2020\)Y\. Bisk, R\. Zellers, J\. Gao, Y\. Choi,et al\.Piqa: reasoning about physical commonsense in natural language\.InProceedings of the AAAI conference on artificial intelligence,Vol\.34,pp\. 7432–7439\.Cited by:[§5](https://arxiv.org/html/2609.21450#S5.SS0.SSS0.Px1.p1.1)\.
- Chaet al\.\(2026\)S\. Cha, H\. Chen, D\. Kim, H\. Zhang, K\. Chan, G\. de Veciana, and H\. VikaloCoreQ: learning\-free mismatch correction and successive rounding for quantization\.Note:Version 2External Links:2602\.05902,[Link](https://arxiv.org/abs/2602.05902v2)Cited by:[§4\.1](https://arxiv.org/html/2609.21450#S4.SS1.p3.1)\.
- Cheeet al\.\(2023\)J\. Chee, Y\. Cai, V\. Kuleshov, and C\. De SaQuIP: 2\-bit quantization of large language models with guarantees\.InAdvances in Neural Information Processing Systems,A\. Oh, T\. Naumann, A\. Globerson, K\. Saenko, M\. Hardt, and S\. Levine \(Eds\.\),Vol\.36,pp\. 4396–4429\.External Links:[Document](https://dx.doi.org/10.52202/075280-0196),[Link](https://proceedings.neurips.cc/paper_files/paper/2023/file/0df38cd13520747e1e64e5b123a78ef8-Paper-Conference.pdf)Cited by:[§2](https://arxiv.org/html/2609.21450#S2.SS0.SSS0.Px2.p1.1),[§4\.4](https://arxiv.org/html/2609.21450#S4.SS4.p3.1)\.
- Clarket al\.\(2018\)P\. Clark, I\. Cowhey, O\. Etzioni, T\. Khot, A\. Sabharwal, C\. Schoenick, and O\. TafjordThink you have solved question answering? try arc, the ai2 reasoning challenge\.External Links:1803\.05457,[Link](https://arxiv.org/abs/1803.05457)Cited by:[§5](https://arxiv.org/html/2609.21450#S5.SS0.SSS0.Px1.p1.1)\.
- Czakóet al\.\(2025\)P\. Czakó, G\. Kertész, and S\. SzénásiSmoothRot: combining channel\-wise scaling and rotation for quantization\-friendly llms\.In2025 IEEE International Conference on Systems, Man, and Cybernetics \(SMC\),Vol\.,pp\. 6461–6466\.External Links:[Document](https://dx.doi.org/10.1109/SMC58881.2025.11342731)Cited by:[§1](https://arxiv.org/html/2609.21450#S1.p1.1),[§2](https://arxiv.org/html/2609.21450#S2.SS0.SSS0.Px2.p1.1)\.
- Dettmerset al\.\(2022\)T\. Dettmers, M\. Lewis, Y\. Belkada, and L\. ZettlemoyerGPT3\.int8\(\): 8\-bit matrix multiplication for transformers at scale\.InAdvances in Neural Information Processing Systems,S\. Koyejo, S\. Mohamed, A\. Agarwal, D\. Belgrave, K\. Cho, and A\. Oh \(Eds\.\),Vol\.35,pp\. 30318–30332\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2022/file/c3ba4962c05c49636d4c6206a97e9c8a-Paper-Conference.pdf)Cited by:[§1](https://arxiv.org/html/2609.21450#S1.p1.1),[§2](https://arxiv.org/html/2609.21450#S2.SS0.SSS0.Px1.p1.1)\.
- Frantaret al\.\(2023\)E\. Frantar, S\. Ashkboos, T\. Hoefler, and D\. AlistarhGPTQ: accurate post\-training quantization for generative pre\-trained transformers\.External Links:2210\.17323,[Link](https://arxiv.org/abs/2210.17323)Cited by:[§1](https://arxiv.org/html/2609.21450#S1.p1.1),[§2](https://arxiv.org/html/2609.21450#S2.SS0.SSS0.Px1.p1.1),[§4\.2](https://arxiv.org/html/2609.21450#S4.SS2.p1.1)\.
- Grattafioriet al\.\(2024\)A\. Grattafiori, A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Vaughan, A\. Yang, A\. Fan, A\. Goyal, A\. Hartshorn, A\. Yang, A\. Mitra, A\. Sravankumar, A\. Korenev, A\. Hinsvark, A\. Rao, A\. Zhang, A\. Rodriguez, A\. Gregerson, A\. Spataru, B\. Roziere, B\. Biron, B\. Tang, B\. Chern, C\. Caucheteux, C\. Nayak, C\. Bi, C\. Marra, C\. McConnell, C\. Keller, C\. Touret, C\. Wu, C\. Wong, C\. C\. Ferrer, C\. Nikolaidis, D\. Allonsius, D\. Song, D\. Pintz, D\. Livshits, D\. Wyatt, D\. Esiobu, D\. Choudhary, D\. Mahajan, D\. Garcia\-Olano, D\. Perino, D\. Hupkes, E\. Lakomkin, E\. AlBadawy, E\. Lobanova, E\. Dinan, E\. M\. Smith, F\. Radenovic, F\. Guzmán, F\. Zhang, G\. Synnaeve, G\. Lee, G\. L\. Anderson, G\. Thattai, G\. Nail, G\. Mialon, G\. Pang, G\. Cucurell, H\. Nguyen, H\. Korevaar, H\. Xu, H\. Touvron, I\. Zarov, I\. A\. Ibarra, I\. Kloumann, I\. Misra, I\. Evtimov, J\. Zhang, J\. Copet, J\. Lee, J\. Geffert, J\. Vranes, J\. Park, J\. Mahadeokar, J\. Shah, J\. van der Linde, J\. Billock, J\. Hong, J\. Lee, J\. Fu, J\. Chi, J\. Huang, J\. Liu, J\. Wang, J\. Yu, J\. Bitton, J\. Spisak, J\. Park, J\. Rocca, J\. Johnstun, J\. Saxe, J\. Jia, K\. V\. Alwala, K\. Prasad, K\. Upasani, K\. Plawiak, K\. Li, K\. Heafield, K\. Stone, K\. El\-Arini, K\. Iyer, K\. Malik, K\. Chiu, K\. Bhalla, K\. Lakhotia, L\. Rantala\-Yeary, L\. van der Maaten, L\. Chen, L\. Tan, L\. Jenkins, L\. Martin, L\. Madaan, L\. Malo, L\. Blecher, L\. Landzaat, L\. de Oliveira, M\. Muzzi, M\. Pasupuleti, M\. Singh, M\. Paluri, M\. Kardas, M\. Tsimpoukelli, M\. Oldham, M\. Rita, M\. Pavlova, M\. Kambadur, M\. Lewis, M\. Si, M\. K\. Singh, M\. Hassan, N\. Goyal, N\. Torabi, N\. Bashlykov, N\. Bogoychev, N\. Chatterji, N\. Zhang, O\. Duchenne, O\. Çelebi, P\. Alrassy, P\. Zhang, P\. Li, P\. Vasic, P\. Weng, P\. Bhargava, P\. Dubal, P\. Krishnan, P\. S\. Koura, P\. Xu, Q\. He, Q\. Dong, R\. Srinivasan, R\. Ganapathy, R\. Calderer, R\. S\. Cabral, R\. Stojnic, R\. Raileanu, R\. Maheswari, R\. Girdhar, R\. Patel, R\. Sauvestre, R\. Polidoro, R\. Sumbaly, R\. Taylor, R\. Silva, R\. Hou, R\. Wang, S\. Hosseini, S\. Chennabasappa, S\. Singh, S\. Bell, S\. S\. Kim, S\. Edunov, S\. Nie, S\. Narang, S\. Raparthy, S\. Shen, S\. Wan, S\. Bhosale, S\. Zhang, S\. Vandenhende, S\. Batra, S\. Whitman, S\. Sootla, S\. Collot, S\. Gururangan, S\. Borodinsky, T\. Herman, T\. Fowler, T\. Sheasha, T\. Georgiou, T\. Scialom, T\. Speckbacher, T\. Mihaylov, T\. Xiao, U\. Karn, V\. Goswami, V\. Gupta, V\. Ramanathan, V\. Kerkez, V\. Gonguet, V\. Do, V\. Vogeti, V\. Albiero, V\. Petrovic, W\. Chu, W\. Xiong, W\. Fu, W\. Meers, X\. Martinet, X\. Wang, X\. Wang, X\. E\. Tan, X\. Xia, X\. Xie, X\. Jia, X\. Wang, Y\. Goldschlag, Y\. Gaur, Y\. Babaei, Y\. Wen, Y\. Song, Y\. Zhang, Y\. Li, Y\. Mao, Z\. D\. Coudert, Z\. Yan, Z\. Chen, Z\. Papakipos, A\. Singh, A\. Srivastava, A\. Jain, A\. Kelsey, A\. Shajnfeld, A\. Gangidi, A\. Victoria, A\. Goldstand, A\. Menon, A\. Sharma, A\. Boesenberg, A\. Baevski, A\. Feinstein, A\. Kallet, A\. Sangani, A\. Teo, A\. Yunus, A\. Lupu, A\. Alvarado, A\. Caples, A\. Gu, A\. Ho, A\. Poulton, A\. Ryan, A\. Ramchandani, A\. Dong, A\. Franco, A\. Goyal, A\. Saraf, A\. Chowdhury, A\. Gabriel, A\. Bharambe, A\. Eisenman, A\. Yazdan, B\. James, B\. Maurer, B\. Leonhardi, B\. Huang, B\. Loyd, B\. D\. Paola, B\. Paranjape, B\. Liu, B\. Wu, B\. Ni, B\. Hancock, B\. Wasti, B\. Spence, B\. Stojkovic, B\. Gamido, B\. Montalvo, C\. Parker, C\. Burton, C\. Mejia, C\. Liu, C\. Wang, C\. Kim, C\. Zhou, C\. Hu, C\. Chu, C\. Cai, C\. Tindal, C\. Feichtenhofer, C\. Gao, D\. Civin, D\. Beaty, D\. Kreymer, D\. Li, D\. Adkins, D\. Xu, D\. Testuggine, D\. David, D\. Parikh, D\. Liskovich, D\. Foss, D\. Wang, D\. Le, D\. Holland, E\. Dowling, E\. Jamil, E\. Montgomery, E\. Presani, E\. Hahn, E\. Wood, E\. Le, E\. Brinkman, E\. Arcaute, E\. Dunbar, E\. Smothers, F\. Sun, F\. Kreuk, F\. Tian, F\. Kokkinos, F\. Ozgenel, F\. Caggioni, F\. Kanayet, F\. Seide, G\. M\. Florez, G\. Schwarz, G\. Badeer, G\. Swee, G\. Halpern, G\. Herman, G\. Sizov, Guangyi, Zhang, G\. Lakshminarayanan, H\. Inan, H\. Shojanazeri, H\. Zou, H\. Wang, H\. Zha, H\. Habeeb, H\. Rudolph, H\. Suk, H\. Aspegren, H\. Goldman, H\. Zhan, I\. Damlaj, I\. Molybog, I\. Tufanov, I\. Leontiadis, I\. Veliche, I\. Gat, J\. Weissman, J\. Geboski, J\. Kohli, J\. Lam, J\. Asher, J\. Gaya, J\. Marcus, J\. Tang, J\. Chan, J\. Zhen, J\. Reizenstein, J\. Teboul, J\. Zhong, J\. Jin, J\. Yang, J\. Cummings, J\. Carvill, J\. Shepard, J\. McPhie, J\. Torres, J\. Ginsburg, J\. Wang, K\. Wu, K\. H\. U, K\. Saxena, K\. Khandelwal, K\. Zand, K\. Matosich, K\. Veeraraghavan, K\. Michelena, K\. Li, K\. Jagadeesh, K\. Huang, K\. Chawla, K\. Huang, L\. Chen, L\. Garg, L\. A, L\. Silva, L\. Bell, L\. Zhang, L\. Guo, L\. Yu, L\. Moshkovich, L\. Wehrstedt, M\. Khabsa, M\. Avalani, M\. Bhatt, M\. Mankus, M\. Hasson, M\. Lennie, M\. Reso, M\. Groshev, M\. Naumov, M\. Lathi, M\. Keneally, M\. Liu, M\. L\. Seltzer, M\. Valko, M\. Restrepo, M\. Patel, M\. Vyatskov, M\. Samvelyan, M\. Clark, M\. Macey, M\. Wang, M\. J\. Hermoso, M\. Metanat, M\. Rastegari, M\. Bansal, N\. Santhanam, N\. Parks, N\. White, N\. Bawa, N\. Singhal, N\. Egebo, N\. Usunier, N\. Mehta, N\. P\. Laptev, N\. Dong, N\. Cheng, O\. Chernoguz, O\. Hart, O\. Salpekar, O\. Kalinli, P\. Kent, P\. Parekh, P\. Saab, P\. Balaji, P\. Rittner, P\. Bontrager, P\. Roux, P\. Dollar, P\. Zvyagina, P\. Ratanchandani, P\. Yuvraj, Q\. Liang, R\. Alao, R\. Rodriguez, R\. Ayub, R\. Murthy, R\. Nayani, R\. Mitra, R\. Parthasarathy, R\. Li, R\. Hogan, R\. Battey, R\. Wang, R\. Howes, R\. Rinott, S\. Mehta, S\. Siby, S\. J\. Bondu, S\. Datta, S\. Chugh, S\. Hunt, S\. Dhillon, S\. Sidorov, S\. Pan, S\. Mahajan, S\. Verma, S\. Yamamoto, S\. Ramaswamy, S\. Lindsay, S\. Lindsay, S\. Feng, S\. Lin, S\. C\. Zha, S\. Patil, S\. Shankar, S\. Zhang, S\. Zhang, S\. Wang, S\. Agarwal, S\. Sajuyigbe, S\. Chintala, S\. Max, S\. Chen, S\. Kehoe, S\. Satterfield, S\. Govindaprasad, S\. Gupta, S\. Deng, S\. Cho, S\. Virk, S\. Subramanian, S\. Choudhury, S\. Goldman, T\. Remez, T\. Glaser, T\. Best, T\. Koehler, T\. Robinson, T\. Li, T\. Zhang, T\. Matthews, T\. Chou, T\. Shaked, V\. Vontimitta, V\. Ajayi, V\. Montanez, V\. Mohan, V\. S\. Kumar, V\. Mangla, V\. Ionescu, V\. Poenaru, V\. T\. Mihailescu, V\. Ivanov, W\. Li, W\. Wang, W\. Jiang, W\. Bouaziz, W\. Constable, X\. Tang, X\. Wu, X\. Wang, X\. Wu, X\. Gao, Y\. Kleinman, Y\. Chen, Y\. Hu, Y\. Jia, Y\. Qi, Y\. Li, Y\. Zhang, Y\. Zhang, Y\. Adi, Y\. Nam, Yu, Wang, Y\. Zhao, Y\. Hao, Y\. Qian, Y\. Li, Y\. He, Z\. Rait, Z\. DeVito, Z\. Rosnbrick, Z\. Wen, Z\. Yang, Z\. Zhao, and Z\. MaThe llama 3 herd of models\.External Links:2407\.21783,[Link](https://arxiv.org/abs/2407.21783)Cited by:[§5](https://arxiv.org/html/2609.21450#S5.SS0.SSS0.Px1.p1.1)\.
- Hoeffding \(1963\)W\. HoeffdingProbability inequalities for sums of bounded random variables\.Journal of the American Statistical Association58\(301\),pp\. 13–30\.External Links:ISSN 01621459, 1537274X,[Link](http://www.jstor.org/stable/2282952)Cited by:[§B\.2](https://arxiv.org/html/2609.21450#A2.SS2.SSS0.Px2.p1.2)\.
- Jianget al\.\(2023\)A\. Q\. Jiang, A\. Sablayrolles, A\. Mensch, C\. Bamford, D\. S\. Chaplot, D\. de las Casas, F\. Bressand, G\. Lengyel, G\. Lample, L\. Saulnier, L\. R\. Lavaud, M\. Lachaux, P\. Stock, T\. Le Scao, T\. Lavril, T\. Wang, T\. Lacroix, and W\. El SayedMistral 7b\.arXiv preprint arXiv:2310\.06825\.External Links:[Link](https://arxiv.org/abs/2310.06825)Cited by:[§5](https://arxiv.org/html/2609.21450#S5.SS0.SSS0.Px1.p1.1)\.
- Liet al\.\(2025\)Y\. Li, R\. Yin, D\. Lee, S\. Xiao, and P\. PandaGPTAQ: efficient finetuning\-free quantization for asymmetric calibration\.InProceedings of the 42nd International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.267,pp\. 36690–36706\.External Links:[Link](https://proceedings.mlr.press/v267/li25dn.html)Cited by:[§4\.2](https://arxiv.org/html/2609.21450#S4.SS2.p2.1),[§4\.2](https://arxiv.org/html/2609.21450#S4.SS2.p3.1)\.
- Linet al\.\(2024\)J\. Lin, J\. Tang, H\. Tang, S\. Yang, W\. Chen, W\. Wang, G\. Xiao, X\. Dang, C\. Gan, and S\. HanAWQ: activation\-aware weight quantization for on\-device llm compression and acceleration\.InProceedings of Machine Learning and Systems,P\. Gibbons, G\. Pekhimenko, and C\. D\. Sa \(Eds\.\),Vol\.6,pp\. 87–100\.External Links:[Link](https://proceedings.mlsys.org/paper_files/paper/2024/file/42a452cbafa9dd64e9ba4aa95cc1ef21-Paper-Conference.pdf)Cited by:[§2](https://arxiv.org/html/2609.21450#S2.SS0.SSS0.Px1.p1.1)\.
- Liuet al\.\(2025\)Z\. Liu, C\. Zhao, I\. Fedorov, B\. Soran, D\. Choudhary, R\. Krishnamoorthi, V\. Chandra, Y\. Tian, and T\. BlankevoortSpinQuant: llm quantization with learned rotations\.InInternational Conference on Learning Representations,Y\. Yue, A\. Garg, N\. Peng, F\. Sha, and R\. Yu \(Eds\.\),Vol\.2025,pp\. 92009–92032\.External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2025/file/e5b1c0d4866f72393c522c8a00eed4eb-Paper-Conference.pdf)Cited by:[§1](https://arxiv.org/html/2609.21450#S1.p1.1),[§2](https://arxiv.org/html/2609.21450#S2.SS0.SSS0.Px2.p1.1),[§4\.6](https://arxiv.org/html/2609.21450#S4.SS6.SSS0.Px2.p1.1),[§5](https://arxiv.org/html/2609.21450#S5.SS0.SSS0.Px1.p1.1),[§5](https://arxiv.org/html/2609.21450#S5.SS0.SSS0.Px2.p1.1)\.
- Meta AI \(2024\)Meta AILlama 3\.2: revolutionizing edge ai and vision with open, customizable models\.Note:[https://ai\.meta\.com/blog/llama\-3\-2\-connect\-2024\-vision\-edge\-mobile\-devices/](https://ai.meta.com/blog/llama-3-2-connect-2024-vision-edge-mobile-devices/)Cited by:[§5](https://arxiv.org/html/2609.21450#S5.SS0.SSS0.Px1.p1.1)\.
- Mistral AI \(2024\)Mistral AIMistral 7b v0\.3\.Note:[https://docs\.mistral\.ai/models/mistral\-7b\-0\-3](https://docs.mistral.ai/models/mistral-7b-0-3)Model documentationCited by:[§5](https://arxiv.org/html/2609.21450#S5.SS0.SSS0.Px1.p1.1)\.
- Papernoet al\.\(2016\)D\. Paperno, G\. Kruszewski, A\. Lazaridou, N\. Q\. Pham, R\. Bernardi, S\. Pezzelle, M\. Baroni, G\. Boleda, and R\. FernándezThe LAMBADA dataset: word prediction requiring a broad discourse context\.InProceedings of the 54th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),K\. Erk and N\. A\. Smith \(Eds\.\),Berlin, Germany,pp\. 1525–1534\.External Links:[Link](https://aclanthology.org/P16-1144/),[Document](https://dx.doi.org/10.18653/v1/P16-1144)Cited by:[§5](https://arxiv.org/html/2609.21450#S5.SS0.SSS0.Px1.p1.1)\.
- Ramanet al\.\(2025\)R\. Raman, K\. Sharma, and S\. Q\. ZhangRethinking the outlier distribution in large language models: an in\-depth study\.External Links:2505\.21670,[Link](https://arxiv.org/abs/2505.21670)Cited by:[§2](https://arxiv.org/html/2609.21450#S2.SS0.SSS0.Px1.p1.1),[§3\.2](https://arxiv.org/html/2609.21450#S3.SS2.p1.1)\.
- Sakaguchiet al\.\(2021\)K\. Sakaguchi, R\. L\. Bras, C\. Bhagavatula, and Y\. ChoiWinoGrande: an adversarial winograd schema challenge at scale\.Commun\. ACM64\(9\),pp\. 99–106\.External Links:ISSN 0001\-0782,[Link](https://doi.org/10.1145/3474381),[Document](https://dx.doi.org/10.1145/3474381)Cited by:[§5](https://arxiv.org/html/2609.21450#S5.SS0.SSS0.Px1.p1.1)\.
- Shaoet al\.\(2024\)W\. Shao, M\. Chen, Z\. Zhang, P\. Xu, L\. Zhao, Z\. Li, K\. Zhang, G\. Peng, Y\. Qiao, and P\. LuoOmniQuant: omnidirectionally calibrated quantization for large language models\.InInternational Conference on Learning Representations,B\. Kim, Y\. Yue, S\. Chaudhuri, K\. Fragkiadaki, M\. Khan, and Y\. Sun \(Eds\.\),Vol\.2024,pp\. 45472–45496\.External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2024/file/c6483c8a68083af3383f91ee0dc6db95-Paper-Conference.pdf)Cited by:[§1](https://arxiv.org/html/2609.21450#S1.p1.1),[§2](https://arxiv.org/html/2609.21450#S2.SS0.SSS0.Px2.p1.1)\.
- Sunet al\.\(2025\)Y\. Sun, R\. Liu, H\. Bai, H\. Bao, K\. Zhao, Y\. Li, J\. Hu, X\. Yu, L\. Hou, C\. Yuan, X\. Jiang, W\. Liu, and J\. YaoFlatQuant: flatness matters for LLM quantization\.InProceedings of the 42nd International Conference on Machine Learning,A\. Singh, M\. Fazel, D\. Hsu, S\. Lacoste\-Julien, F\. Berkenkamp, T\. Maharaj, K\. Wagstaff, and J\. Zhu \(Eds\.\),Proceedings of Machine Learning Research, Vol\.267,pp\. 57587–57613\.External Links:[Link](https://proceedings.mlr.press/v267/sun25l.html)Cited by:[§1](https://arxiv.org/html/2609.21450#S1.p1.1),[§2](https://arxiv.org/html/2609.21450#S2.SS0.SSS0.Px2.p1.1)\.
- Touvronet al\.\(2023a\)H\. Touvron, T\. Lavril, G\. Izacard, X\. Martinet, M\. Lachaux, T\. Lacroix, B\. Rozière, N\. Goyal, E\. Hambro, F\. Azhar, A\. Rodriguez, A\. Joulin, E\. Grave, and G\. LampleLLaMA: open and efficient foundation language models\.External Links:2302\.13971,[Link](https://arxiv.org/abs/2302.13971)Cited by:[§5](https://arxiv.org/html/2609.21450#S5.SS0.SSS0.Px1.p1.1)\.
- Touvronet al\.\(2023b\)H\. Touvron, L\. Martin, K\. Stone, P\. Albert, A\. Almahairi, Y\. Babaei, N\. Bashlykov, S\. Batra, P\. Bhargava, S\. Bhosale, D\. Bikel, L\. Blecher, C\. C\. Ferrer, M\. Chen, G\. Cucurull, D\. Esiobu, J\. Fernandes, J\. Fu, W\. Fu, B\. Fuller, C\. Gao, V\. Goswami, N\. Goyal, A\. Hartshorn, S\. Hosseini, R\. Hou, H\. Inan, M\. Kardas, V\. Kerkez, M\. Khabsa, I\. Kloumann, A\. Korenev, P\. S\. Koura, M\. Lachaux, T\. Lavril, J\. Lee, D\. Liskovich, Y\. Lu, Y\. Mao, X\. Martinet, T\. Mihaylov, P\. Mishra, I\. Molybog, Y\. Nie, A\. Poulton, J\. Reizenstein, R\. Rungta, K\. Saladi, A\. Schelten, R\. Silva, E\. M\. Smith, R\. Subramanian, X\. E\. Tan, B\. Tang, R\. Taylor, A\. Williams, J\. X\. Kuan, P\. Xu, Z\. Yan, I\. Zarov, Y\. Zhang, A\. Fan, M\. Kambadur, S\. Narang, A\. Rodriguez, R\. Stojnic, S\. Edunov, and T\. ScialomLlama 2: open foundation and fine\-tuned chat models\.External Links:2307\.09288,[Link](https://arxiv.org/abs/2307.09288)Cited by:[§5](https://arxiv.org/html/2609.21450#S5.SS0.SSS0.Px1.p1.1)\.
- Tsenget al\.\(2024\)A\. Tseng, J\. Chee, Q\. Sun, V\. Kuleshov, and C\. De SaQuIP\#: even better LLM quantization with hadamard incoherence and lattice codebooks\.InProceedings of the 41st International Conference on Machine Learning,R\. Salakhutdinov, Z\. Kolter, K\. Heller, A\. Weller, N\. Oliver, J\. Scarlett, and F\. Berkenkamp \(Eds\.\),Proceedings of Machine Learning Research, Vol\.235,pp\. 48630–48656\.External Links:[Link](https://proceedings.mlr.press/v235/tseng24a.html)Cited by:[§2](https://arxiv.org/html/2609.21450#S2.SS0.SSS0.Px2.p1.1),[§3\.2](https://arxiv.org/html/2609.21450#S3.SS2.p3.1),[§4\.3](https://arxiv.org/html/2609.21450#S4.SS3.SSS0.Px3.p3.1.1)\.
- Xiang and Zhang \(2025\)J\. Xiang and S\. Q\. ZhangDFRot: achieving outlier\-free and massive activation\-free for rotated LLMs with refined rotation\.InSecond Conference on Language Modeling,External Links:[Link](https://openreview.net/forum?id=WzGypILLDb)Cited by:[§2](https://arxiv.org/html/2609.21450#S2.SS0.SSS0.Px2.p1.1)\.
- Xiaoet al\.\(2023\)G\. Xiao, J\. Lin, M\. Seznec, H\. Wu, J\. Demouth, and S\. HanSmoothQuant: accurate and efficient post\-training quantization for large language models\.InProceedings of the 40th International Conference on Machine Learning,A\. Krause, E\. Brunskill, K\. Cho, B\. Engelhardt, S\. Sabato, and J\. Scarlett \(Eds\.\),Proceedings of Machine Learning Research, Vol\.202,pp\. 38087–38099\.External Links:[Link](https://proceedings.mlr.press/v202/xiao23c.html)Cited by:[§1](https://arxiv.org/html/2609.21450#S1.p1.1),[§2](https://arxiv.org/html/2609.21450#S2.SS0.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2609.21450#S2.SS0.SSS0.Px2.p1.1),[§3\.2](https://arxiv.org/html/2609.21450#S3.SS2.p1.1)\.
- Zellerset al\.\(2019\)R\. Zellers, A\. Holtzman, Y\. Bisk, A\. Farhadi, and Y\. ChoiHellaSwag: can a machine really finish your sentence?\.InProceedings of the 57th Annual Meeting of the Association for Computational Linguistics,A\. Korhonen, D\. Traum, and L\. Màrquez \(Eds\.\),Florence, Italy,pp\. 4791–4800\.External Links:[Link](https://aclanthology.org/P19-1472/),[Document](https://dx.doi.org/10.18653/v1/P19-1472)Cited by:[§5](https://arxiv.org/html/2609.21450#S5.SS0.SSS0.Px1.p1.1)\.

## Appendix AAn Interference View of Hadamard Rotation Bounds

In this section, we provide additional analysis supporting the randomized Hadamard rotation arguments in Section[4\.3](https://arxiv.org/html/2609.21450#S4.SS3)\. We make explicit how contributions from multiple persistent CO channels can constructively interfere under a fixed Hadamard rotation and explain how random signs remove this deterministic alignment in expectation\. This interpretation complements the high\-probability bounds in Theorem[2](https://arxiv.org/html/2609.21450#Thmtheorem2)\.

Let𝐇∈ℝd×d\\mathbf\{H\}\\in\\mathbb\{R\}^\{d\\times d\}be the normalized Hadamard matrix\. Ford=2nd=2^\{n\}, identifying the indicesj,kj,kwith binary vectors in\{0,1\}n\\\{0,1\\\}^\{n\}, we write

\(𝐇\)j,k=1d​\(−1\)⟨j,k⟩,\\displaystyle\(\\mathbf\{H\}\)\_\{j,k\}=\\frac\{1\}\{\\sqrt\{d\}\}\(\-1\)^\{\\langle j,k\\rangle\},\(18\)where⟨j,k⟩\\langle j,k\\rangledenotes the bit\-wise inner product modulo22\.

To simplify the notation, we omit channel scaling; the same argument applies by replacingLkL\_\{k\}withLk/λkL\_\{k\}/\\lambda\_\{k\}\. For the fixed Hadamard rotation𝐗hadco=𝐗co​𝐇⊤\\mathbf\{X\}^\{\\mathrm\{co\}\}\_\{\\mathrm\{had\}\}=\\mathbf\{X\}^\{\\mathrm\{co\}\}\\mathbf\{H\}^\{\\top\}, each output coordinate is

\(𝐗hadco\)t,j=1d​∑k∈𝒞co​\(𝐗\)Lk​\(−1\)⟨j,k⟩\.\\displaystyle\(\\mathbf\{X\}^\{\\mathrm\{co\}\}\_\{\\mathrm\{had\}\}\)\_\{t,j\}=\\frac\{1\}\{\\sqrt\{d\}\}\\sum\_\{k\\in\\mathcal\{C\}\_\{\\mathrm\{co\}\}\{\}\(\\mathbf\{X\}\)\}L\_\{k\}\(\-1\)^\{\\langle j,k\\rangle\}\.\(19\)Thus, although each CO channel is spread across all output coordinates, the rotated coordinate is a deterministic signed sum of the CO magnitudes\.

Its squared magnitude is

Ej\\displaystyle E\_\{j\}:=\|\(𝐗hadco\)t,j\|2\\displaystyle:=\\left\|\(\\mathbf\{X\}^\{\\mathrm\{co\}\}\_\{\\mathrm\{had\}\}\)\_\{t,j\}\\right\|^\{2\}\(20\)=1d​∑k∈𝒞co​\(𝐗\)Lk2⏟individual​\-​channel​energy\+1d​∑k,m∈𝒞co​\(𝐗\)k≠mLk​Lm​\(−1\)⟨j,k⊕m⟩⏟cross​\-​channel​interference,\\displaystyle=\\underbrace\{\\frac\{1\}\{d\}\\sum\_\{k\\in\\mathcal\{C\}\_\{\\mathrm\{co\}\}\{\}\(\\mathbf\{X\}\)\}L\_\{k\}^\{2\}\}\_\{\\mathrm\{individual\\text\{\-\}channel\\ energy\}\}\+\\underbrace\{\\frac\{1\}\{d\}\\sum\_\{\\begin\{subarray\}\{c\}k,m\\in\\mathcal\{C\}\_\{\\mathrm\{co\}\}\{\}\(\\mathbf\{X\}\)\\\\ k\\neq m\\end\{subarray\}\}L\_\{k\}L\_\{m\}\(\-1\)^\{\\langle j,k\\oplus m\\rangle\}\}\_\{\\mathrm\{cross\\text\{\-\}channel\\ interference\}\},\(21\)where⊕\\oplusdenotes bit\-wise XOR\. The first term is independent ofjj, whereas the second term depends on the fixed Hadamard sign pattern\. When several CO contributions have the same sign at an output coordinate, the cross terms are positive and the contributions add constructively\.

The individual\-channel term containsNco​\(𝐗\)N\_\{\\mathrm\{co\}\}\{\}\(\\mathbf\{X\}\)contributions and therefore scales linearly withNco​\(𝐗\)N\_\{\\mathrm\{co\}\}\{\}\(\\mathbf\{X\}\)\. In contrast, the cross\-channel term containsNco​\(𝐗\)​\(Nco​\(𝐗\)−1\)N\_\{\\mathrm\{co\}\}\{\}\(\\mathbf\{X\}\)\(N\_\{\\mathrm\{co\}\}\{\}\(\\mathbf\{X\}\)\-1\)ordered pairs\. Thus, when many of these pairwise terms align positively, the cross\-channel interference can grow on the order ofNco​\(𝐗\)2N\_\{\\mathrm\{co\}\}\{\}\(\\mathbf\{X\}\)^\{2\}\. This provides an interference\-based interpretation of the quadratic dependence on the number of CO channels in the fixed\-rotation bound of Theorem[2](https://arxiv.org/html/2609.21450#Thmtheorem2)\.

Now consider a random sign matrix𝐃=diag⁡\(ε1,…,εd\)\\mathbf\{D\}=\\operatorname\{diag\}\(\\varepsilon\_\{1\},\\dots,\\varepsilon\_\{d\}\), where theεk\\varepsilon\_\{k\}are independent Rademacher variables\. The corresponding coordinate energy is

Ej​\(𝐃\)=1d​\(∑k∈𝒞co​\(𝐗\)εk​Lk​\(−1\)⟨j,k⟩\)2\.\\displaystyle E\_\{j\}\(\\mathbf\{D\}\)=\\frac\{1\}\{d\}\\left\(\\sum\_\{k\\in\\mathcal\{C\}\_\{\\mathrm\{co\}\}\{\}\(\\mathbf\{X\}\)\}\\varepsilon\_\{k\}L\_\{k\}\(\-1\)^\{\\langle j,k\\rangle\}\\right\)^\{2\}\.\(22\)Expanding the square gives

Ej​\(𝐃\)=1d​∑k∈𝒞co​\(𝐗\)Lk2\+1d​∑k,m∈𝒞co​\(𝐗\)k≠mεk​εm​Lk​Lm​\(−1\)⟨j,k⊕m⟩\.\\displaystyle E\_\{j\}\(\\mathbf\{D\}\)=\\frac\{1\}\{d\}\\sum\_\{k\\in\\mathcal\{C\}\_\{\\mathrm\{co\}\}\{\}\(\\mathbf\{X\}\)\}L\_\{k\}^\{2\}\+\\frac\{1\}\{d\}\\sum\_\{\\begin\{subarray\}\{c\}k,m\\in\\mathcal\{C\}\_\{\\mathrm\{co\}\}\{\}\(\\mathbf\{X\}\)\\\\ k\\neq m\\end\{subarray\}\}\\varepsilon\_\{k\}\\varepsilon\_\{m\}L\_\{k\}L\_\{m\}\(\-1\)^\{\\langle j,k\\oplus m\\rangle\}\.\(23\)Since𝔼𝐃​\[εk​εm\]=0​\(k≠m\),\\mathbb\{E\}\_\{\\mathbf\{D\}\}\\left\[\\varepsilon\_\{k\}\\varepsilon\_\{m\}\\right\]=0\\,\(k\\neq m\),all cross\-channel interference terms vanish in expectation:

𝔼𝐃​\[Ej​\(𝐃\)\]=1d​∑k∈𝒞co​\(𝐗\)Lk2\.\\displaystyle\\mathbb\{E\}\_\{\\mathbf\{D\}\}\\left\[E\_\{j\}\(\\mathbf\{D\}\)\\right\]=\\frac\{1\}\{d\}\\sum\_\{k\\in\\mathcal\{C\}\_\{\\mathrm\{co\}\}\{\}\(\\mathbf\{X\}\)\}L\_\{k\}^\{2\}\.\(24\)Hence, after randomization, only the individual\-channel energy remains in expectation, reducing the dependence on the number of CO channels from𝒪⁡\(Nco​\(𝐗\)2\)\\mathcal\{O\}\(N\_\{\\mathrm\{co\}\}\{\}\(\\mathbf\{X\}\)^\{2\}\)to𝒪​\(Nco​\(𝐗\)\)\\mathcal\{O\}\(N\_\{\\mathrm\{co\}\}\{\}\(\\mathbf\{X\}\)\)\. This provides an interference\-based interpretation of the improvement from the fixed to the randomized bound in Theorem[2](https://arxiv.org/html/2609.21450#Thmtheorem2)\.

Sampling multiple sign matrices can further be interpreted as increasing the chance of obtaining a favorable sign assignment in which the CO contributions cancel more effectively, thereby reducing the largest rotated coordinate\.

## Appendix BProofs for Section[4](https://arxiv.org/html/2609.21450#S4)

### B\.1Proof of Theorem[1](https://arxiv.org/html/2609.21450#Thmtheorem1)

###### Proof\.

We first prove the projection\-based identity\. By definition,𝐙~=𝐙\+𝐀\\widetilde\{\\mathbf\{Z\}\}=\\mathbf\{Z\}\+\\mathbf\{A\}\. Therefore,

𝐙~​𝐕~⊤−𝐙𝐕⊤\\displaystyle\\widetilde\{\\mathbf\{Z\}\}\\widetilde\{\\mathbf\{V\}\}^\{\\top\}\-\\mathbf\{Z\}\\mathbf\{V\}^\{\\top\}=𝐙~​𝐕~⊤−𝐙~​𝐕⊤\+𝐙~​𝐕⊤−𝐙𝐕⊤\\displaystyle=\\widetilde\{\\mathbf\{Z\}\}\\widetilde\{\\mathbf\{V\}\}^\{\\top\}\-\\widetilde\{\\mathbf\{Z\}\}\\mathbf\{V\}^\{\\top\}\+\\widetilde\{\\mathbf\{Z\}\}\\mathbf\{V\}^\{\\top\}\-\\mathbf\{Z\}\\mathbf\{V\}^\{\\top\}\(25\)=𝐙~​\(𝐕~−𝐕\)⊤\+\(𝐙~−𝐙\)​𝐕⊤\\displaystyle=\\widetilde\{\\mathbf\{Z\}\}\\left\(\\widetilde\{\\mathbf\{V\}\}\-\\mathbf\{V\}\\right\)^\{\\top\}\+\\left\(\\widetilde\{\\mathbf\{Z\}\}\-\\mathbf\{Z\}\\right\)\\mathbf\{V\}^\{\\top\}\(26\)=𝐙~​\(𝐕~−𝐕\)⊤\+𝐀𝐕⊤\.\\displaystyle=\\widetilde\{\\mathbf\{Z\}\}\\left\(\\widetilde\{\\mathbf\{V\}\}\-\\mathbf\{V\}\\right\)^\{\\top\}\+\\mathbf\{A\}\\mathbf\{V\}^\{\\top\}\.\(27\)Let𝚷𝐙~:=𝐙~​𝐙~†\\mathbf\{\\Pi\}\_\{\\widetilde\{\\mathbf\{Z\}\}\}:=\\widetilde\{\\mathbf\{Z\}\}\\widetilde\{\\mathbf\{Z\}\}^\{\\dagger\}denote the orthogonal projector onto the column space of𝐙~\\widetilde\{\\mathbf\{Z\}\}, where𝐙~†\\widetilde\{\\mathbf\{Z\}\}^\{\\dagger\}is the Moore–Penrose pseudoinverse\. We decompose𝐀𝐕⊤\\mathbf\{A\}\\mathbf\{V\}^\{\\top\}into its projection onto this column space and its orthogonal complement:

𝐀𝐕⊤=𝐙~​𝐙~†​𝐀𝐕⊤\+\(𝐈−𝚷𝐙~\)​𝐀𝐕⊤\.\\displaystyle\\mathbf\{A\}\\mathbf\{V\}^\{\\top\}=\\widetilde\{\\mathbf\{Z\}\}\\widetilde\{\\mathbf\{Z\}\}^\{\\dagger\}\\mathbf\{A\}\\mathbf\{V\}^\{\\top\}\+\\left\(\\mathbf\{I\}\-\\mathbf\{\\Pi\}\_\{\\widetilde\{\\mathbf\{Z\}\}\}\\right\)\\mathbf\{A\}\\mathbf\{V\}^\{\\top\}\.\(28\)Thus,

𝐙~​𝐕~⊤−𝐙𝐕⊤\\displaystyle\\widetilde\{\\mathbf\{Z\}\}\\widetilde\{\\mathbf\{V\}\}^\{\\top\}\-\\mathbf\{Z\}\\mathbf\{V\}^\{\\top\}=𝐙~​\(\(𝐕~−𝐕\)⊤\+𝐙~†​𝐀𝐕⊤\)\+\(𝐈−𝚷𝐙~\)​𝐀𝐕⊤\.\\displaystyle=\\widetilde\{\\mathbf\{Z\}\}\\left\(\\left\(\\widetilde\{\\mathbf\{V\}\}\-\\mathbf\{V\}\\right\)^\{\\top\}\+\\widetilde\{\\mathbf\{Z\}\}^\{\\dagger\}\\mathbf\{A\}\\mathbf\{V\}^\{\\top\}\\right\)\+\\left\(\\mathbf\{I\}\-\\mathbf\{\\Pi\}\_\{\\widetilde\{\\mathbf\{Z\}\}\}\\right\)\\mathbf\{A\}\\mathbf\{V\}^\{\\top\}\.\(29\)By the definition\(𝐕⋆\)⊤=𝐕⊤−𝐙~†​𝐀𝐕⊤\(\\mathbf\{V\}^\{\\star\}\)^\{\\top\}=\\mathbf\{V\}^\{\\top\}\-\\widetilde\{\\mathbf\{Z\}\}^\{\\dagger\}\\mathbf\{A\}\\mathbf\{V\}^\{\\top\}, this becomes

𝐙~​𝐕~⊤−𝐙𝐕⊤=𝐙~​\(𝐕~−𝐕⋆\)⊤\+\(𝐈−𝚷𝐙~\)​𝐀𝐕⊤\.\\displaystyle\\widetilde\{\\mathbf\{Z\}\}\\widetilde\{\\mathbf\{V\}\}^\{\\top\}\-\\mathbf\{Z\}\\mathbf\{V\}^\{\\top\}=\\widetilde\{\\mathbf\{Z\}\}\\left\(\\widetilde\{\\mathbf\{V\}\}\-\\mathbf\{V\}^\{\\star\}\\right\)^\{\\top\}\+\\left\(\\mathbf\{I\}\-\\mathbf\{\\Pi\}\_\{\\widetilde\{\\mathbf\{Z\}\}\}\\right\)\\mathbf\{A\}\\mathbf\{V\}^\{\\top\}\.\(30\)The first term belongs to the column space of𝐙~\\widetilde\{\\mathbf\{Z\}\}, while the second term belongs to its orthogonal complement\. Hence, by orthogonality,

𝒥⁡\(𝐏,𝐕~\)=1T​‖𝐙~​\(𝐕~−𝐕⋆\)⊤‖F2\+1T​‖\(𝐈−𝚷𝐙~\)​𝐀𝐕⊤‖F2\.\\displaystyle\\mathcal\{J\}\(\\mathbf\{P\},\\widetilde\{\\mathbf\{V\}\}\)=\\frac\{1\}\{T\}\\left\\\|\\widetilde\{\\mathbf\{Z\}\}\\left\(\\widetilde\{\\mathbf\{V\}\}\-\\mathbf\{V\}^\{\\star\}\\right\)^\{\\top\}\\right\\\|\_\{F\}^\{2\}\+\\frac\{1\}\{T\}\\left\\\|\\left\(\\mathbf\{I\}\-\\mathbf\{\\Pi\}\_\{\\widetilde\{\\mathbf\{Z\}\}\}\\right\)\\mathbf\{A\}\\mathbf\{V\}^\{\\top\}\\right\\\|\_\{F\}^\{2\}\.\(31\)
The first squared norm is the activation\-guided weight compensation term𝒥AGWC\\mathcal\{J\}\_\{\\mathrm\{AGWC\}\}defined in Eq\. \([6](https://arxiv.org/html/2609.21450#S4.E6)\)\. The second is independent of𝐕~\\widetilde\{\\mathbf\{V\}\}for fixed inputs and𝐏\\mathbf\{P\}, and is attained as the minimum local error when unrestricted weights are set to𝐕⋆\\mathbf\{V\}^\{\\star\}\.

We next upper bound this orthogonal residual using the CO and regular quantities; they are not separate terms in the exact identity\. For no\-clipping dynamic per\-token symmetric quantization, let

Δt=‖\(𝐙\)t,:‖∞qA,qA=2bA−1−1\.\\displaystyle\\Delta\_\{t\}=\\frac\{\\left\\\|\(\\mathbf\{Z\}\)\_\{t,:\}\\right\\\|\_\{\\infty\}\}\{q\_\{A\}\},\\qquad q\_\{A\}=2^\{b\_\{A\}\-1\}\-1\.\(32\)For each entry,

\|\(𝐀\)t,j\|=\|𝒬A​\(\(𝐙\)t,j\)−\(𝐙\)t,j\|≤Δt2\.\\displaystyle\\left\|\(\\mathbf\{A\}\)\_\{t,j\}\\right\|=\\left\|\\mathcal\{Q\}\_\{A\}\(\(\\mathbf\{Z\}\)\_\{t,j\}\)\-\(\\mathbf\{Z\}\)\_\{t,j\}\\right\|\\leq\\frac\{\\Delta\_\{t\}\}\{2\}\.\(33\)Therefore,

‖𝐀‖F2≤∑t=1T∑j=1dΔt24=d4​qA2∑t=1T‖\(𝐙\)t,:‖∞2\.\\displaystyle\\left\\\|\\mathbf\{A\}\\right\\\|\_\{F\}^\{2\}\\leq\\sum\_\{t=1\}^\{T\}\\sum\_\{j=1\}^\{d\}\\frac\{\\Delta\_\{t\}^\{2\}\}\{4\}=\\frac\{d\}\{4q\_\{A\}^\{2\}\}\\sum\_\{t=1\}^\{T\}\\left\\\|\(\\mathbf\{Z\}\)\_\{t,:\}\\right\\\|\_\{\\infty\}^\{2\}\.\(34\)Since orthogonal projection does not increase the Frobenius norm,

1T​‖\(𝐈−𝚷𝐙~\)​𝐀𝐕⊤‖F2\\displaystyle\\frac\{1\}\{T\}\\left\\\|\\left\(\\mathbf\{I\}\-\\mathbf\{\\Pi\}\_\{\\widetilde\{\\mathbf\{Z\}\}\}\\right\)\\mathbf\{A\}\\mathbf\{V\}^\{\\top\}\\right\\\|\_\{F\}^\{2\}≤1T​‖𝐀𝐕⊤‖F2\\displaystyle\\leq\\frac\{1\}\{T\}\\left\\\|\\mathbf\{A\}\\mathbf\{V\}^\{\\top\}\\right\\\|\_\{F\}^\{2\}\(35\)≤‖𝐕‖22T​‖𝐀‖F2\.\\displaystyle\\leq\\frac\{\\left\\\|\\mathbf\{V\}\\right\\\|\_\{2\}^\{2\}\}\{T\}\\left\\\|\\mathbf\{A\}\\right\\\|\_\{F\}^\{2\}\.\(36\)For𝐏=𝚲−1​𝐃𝐇⊤\\mathbf\{P\}=\\mathbf\{\\Lambda\}^\{\-1\}\\mathbf\{D\}\\mathbf\{H\}^\{\\top\}, we have𝐕=𝐖𝐏−⁣⊤=𝐖​𝚲​𝐃𝐇⊤\\mathbf\{V\}=\\mathbf\{W\}\\mathbf\{P\}^\{\-\\top\}=\\mathbf\{W\}\\mathbf\{\\Lambda\}\\mathbf\{D\}\\mathbf\{H\}^\{\\top\}\. Because𝐃\\mathbf\{D\}and𝐇⊤\\mathbf\{H\}^\{\\top\}are orthogonal matrices,‖𝐕‖2=‖𝐖​𝚲‖2\\\|\\mathbf\{V\}\\\|\_\{2\}=\\\|\\mathbf\{W\}\\mathbf\{\\Lambda\}\\\|\_\{2\}\. Thus,

1T‖\(𝐈−𝚷𝐙~\)𝐀𝐕⊤‖F2≤d​‖𝐖​𝚲‖224​qA2​T∑t=1T‖\(𝐗\)t,:𝚲−1𝐃𝐇⊤‖∞2\.\\displaystyle\\frac\{1\}\{T\}\\left\\\|\\left\(\\mathbf\{I\}\-\\mathbf\{\\Pi\}\_\{\\widetilde\{\\mathbf\{Z\}\}\}\\right\)\\mathbf\{A\}\\mathbf\{V\}^\{\\top\}\\right\\\|\_\{F\}^\{2\}\\leq\\frac\{d\\left\\\|\\mathbf\{W\}\\mathbf\{\\Lambda\}\\right\\\|\_\{2\}^\{2\}\}\{4q\_\{A\}^\{2\}T\}\\sum\_\{t=1\}^\{T\}\\left\\\|\(\\mathbf\{X\}\)\_\{t,:\}\\mathbf\{\\Lambda\}^\{\-1\}\\mathbf\{D\}\\mathbf\{H\}^\{\\top\}\\right\\\|\_\{\\infty\}^\{2\}\.\(37\)
Finally, decompose\(𝐗\)t,:=\(𝐗co\)t,:\+\(𝐗reg\)t,:\(\\mathbf\{X\}\)\_\{t,:\}=\(\\mathbf\{X\}^\{\\mathrm\{co\}\}\)\_\{t,:\}\+\(\\mathbf\{X\}^\{\\mathrm\{reg\}\}\)\_\{t,:\}\. By the triangle inequality inℓ∞\\ell\_\{\\infty\}and then inℓ2\\ell\_\{2\}over tokens,

\(∑t=1T‖\(𝐗\)t,:𝚲−1𝐃𝐇⊤‖∞2\)1/2\\displaystyle\\left\(\\sum\_\{t=1\}^\{T\}\\left\\\|\(\\mathbf\{X\}\)\_\{t,:\}\\mathbf\{\\Lambda\}^\{\-1\}\\mathbf\{D\}\\mathbf\{H\}^\{\\top\}\\right\\\|\_\{\\infty\}^\{2\}\\right\)^\{1/2\}≤\(∑t=1T‖\(𝐗co\)t,:𝚲−1𝐃𝐇⊤‖∞2\)1/2\\displaystyle\\leq\\left\(\\sum\_\{t=1\}^\{T\}\\left\\\|\(\\mathbf\{X\}^\{\\mathrm\{co\}\}\)\_\{t,:\}\\mathbf\{\\Lambda\}^\{\-1\}\\mathbf\{D\}\\mathbf\{H\}^\{\\top\}\\right\\\|\_\{\\infty\}^\{2\}\\right\)^\{1/2\}\(38\)\+\(∑t=1T‖\(𝐗reg\)t,:𝚲−1𝐃𝐇⊤‖∞2\)1/2\.\\displaystyle\\quad\+\\left\(\\sum\_\{t=1\}^\{T\}\\left\\\|\(\\mathbf\{X\}^\{\\mathrm\{reg\}\}\)\_\{t,:\}\\mathbf\{\\Lambda\}^\{\-1\}\\mathbf\{D\}\\mathbf\{H\}^\{\\top\}\\right\\\|\_\{\\infty\}^\{2\}\\right\)^\{1/2\}\.\(39\)Therefore,

∑t=1T‖\(𝐗\)t,:𝚲−1𝐃𝐇⊤‖∞2≤\(𝒥co​\(𝚲,𝐃\)\+𝒥reg​\(𝚲,𝐃\)‖𝐖​𝚲‖22\)2\.\\displaystyle\\sum\_\{t=1\}^\{T\}\\left\\\|\(\\mathbf\{X\}\)\_\{t,:\}\\mathbf\{\\Lambda\}^\{\-1\}\\mathbf\{D\}\\mathbf\{H\}^\{\\top\}\\right\\\|\_\{\\infty\}^\{2\}\\leq\\left\(\\sqrt\{\\mathcal\{J\}\_\{\\mathrm\{co\}\}\(\\mathbf\{\\Lambda\},\\mathbf\{D\}\)\}\+\\sqrt\{\\frac\{\\mathcal\{J\}\_\{\\mathrm\{reg\}\}\(\\mathbf\{\\Lambda\},\\mathbf\{D\}\)\}\{\\left\\\|\\mathbf\{W\}\\mathbf\{\\Lambda\}\\right\\\|\_\{2\}^\{2\}\}\}\\right\)^\{2\}\.\(40\)Combining the above inequalities gives

𝒥⁡\(𝐏,𝐕~\)≤𝒥AGWC​\(𝐏,𝐕~\)\+d4​qA2​T​\(‖𝐖​𝚲‖22​𝒥co​\(𝚲,𝐃\)\+𝒥reg​\(𝚲,𝐃\)\)2\.\\displaystyle\\mathcal\{J\}\(\\mathbf\{P\},\\widetilde\{\\mathbf\{V\}\}\)\\leq\\mathcal\{J\}\_\{\\mathrm\{AGWC\}\}\(\\mathbf\{P\},\\widetilde\{\\mathbf\{V\}\}\)\+\\frac\{d\}\{4q\_\{A\}^\{2\}T\}\\left\(\\sqrt\{\\left\\\|\\mathbf\{W\}\\mathbf\{\\Lambda\}\\right\\\|\_\{2\}^\{2\}\\mathcal\{J\}\_\{\\mathrm\{co\}\}\(\\mathbf\{\\Lambda\},\\mathbf\{D\}\)\}\+\\sqrt\{\\mathcal\{J\}\_\{\\mathrm\{reg\}\}\(\\mathbf\{\\Lambda\},\\mathbf\{D\}\)\}\\right\)^\{2\}\.\(41\)∎

### B\.2Proof of Theorem[2](https://arxiv.org/html/2609.21450#Thmtheorem2)

###### Proof\.

Let the normalized Hadamard matrix be written as

\(𝐇⊤\)k,j=hk,jd,hk,j∈\{−1,\+1\}\.\\displaystyle\(\\mathbf\{H\}^\{\\top\}\)\_\{k,j\}=\\frac\{h\_\{k,j\}\}\{\\sqrt\{d\}\},\\qquad h\_\{k,j\}\\in\\\{\-1,\+1\\\}\.\(42\)

#### Fixed rotation\.

Under the persistent sparse CO model,

\(\(𝐗co\)t,:𝚲−1𝐇⊤\)j=1d∑k∈𝒞co​\(𝐗\)Lkλkhk,j\.\\displaystyle\\left\(\(\\mathbf\{X\}^\{\\mathrm\{co\}\}\)\_\{t,:\}\\mathbf\{\\Lambda\}^\{\-1\}\\mathbf\{H\}^\{\\top\}\\right\)\_\{j\}=\\frac\{1\}\{\\sqrt\{d\}\}\\sum\_\{k\\in\\mathcal\{C\}\_\{\\mathrm\{co\}\}\{\}\(\\mathbf\{X\}\)\}\\frac\{L\_\{k\}\}\{\\lambda\_\{k\}\}h\_\{k,j\}\.\(43\)Therefore,

\|\(\(𝐗co\)t,:𝚲−1𝐇⊤\)j\|≤1d∑k∈𝒞co​\(𝐗\)\|Lk\|λk≤Nco​\(𝐗\)dLΛ,max\.\\displaystyle\\left\|\\left\(\(\\mathbf\{X\}^\{\\mathrm\{co\}\}\)\_\{t,:\}\\mathbf\{\\Lambda\}^\{\-1\}\\mathbf\{H\}^\{\\top\}\\right\)\_\{j\}\\right\|\\leq\\frac\{1\}\{\\sqrt\{d\}\}\\sum\_\{k\\in\\mathcal\{C\}\_\{\\mathrm\{co\}\}\{\}\(\\mathbf\{X\}\)\}\\frac\{\|L\_\{k\}\|\}\{\\lambda\_\{k\}\}\\leq\\frac\{N\_\{\\mathrm\{co\}\}\{\}\(\\mathbf\{X\}\)\}\{\\sqrt\{d\}\}L\_\{\\Lambda,\\max\}\.\(44\)Taking the maximum overjjand summing overTTtokens gives

𝒥co​\(𝚲,𝐈\)≤T​Nco​\(𝐗\)2d​LΛ,max2\.\\displaystyle\\mathcal\{J\}\_\{\\mathrm\{co\}\}\(\\mathbf\{\\Lambda\},\\mathbf\{I\}\)\\leq\\frac\{TN\_\{\\mathrm\{co\}\}\{\}\(\\mathbf\{X\}\)^\{2\}\}\{d\}L\_\{\\Lambda,\\max\}^\{2\}\.\(45\)

#### Randomized rotation\.

Consider a random sign matrix𝐃=diag⁡\(ε1,…,εd\)\\mathbf\{D\}=\\operatorname\{diag\}\(\\varepsilon\_\{1\},\\ldots,\\varepsilon\_\{d\}\), whereεk\\varepsilon\_\{k\}are independent Rademacher variables\. For a fixed coordinatejj,

Yj=\(\(𝐗co\)t,:𝚲−1𝐃𝐇⊤\)j=1d∑k∈𝒞co​\(𝐗\)Lkλkεkhk​j\.\\displaystyle Y\_\{j\}=\\left\(\(\\mathbf\{X\}^\{\\mathrm\{co\}\}\)\_\{t,:\}\\mathbf\{\\Lambda\}^\{\-1\}\\mathbf\{D\}\\mathbf\{H\}^\{\\top\}\\right\)\_\{j\}=\\frac\{1\}\{\\sqrt\{d\}\}\\sum\_\{k\\in\\mathcal\{C\}\_\{\\mathrm\{co\}\}\{\}\(\\mathbf\{X\}\)\}\\frac\{L\_\{k\}\}\{\\lambda\_\{k\}\}\\varepsilon\_\{k\}h\_\{kj\}\.\(46\)This is a centered Rademacher sum\. By Hoeffding’s inequality\([Hoeffding, 1963](https://arxiv.org/html/2609.21450#bib.bib22)\),

Pr⁡\(\|Yj\|\>u\)≤2​exp⁡\(−d​u22​RΛ2\),\\displaystyle\\Pr\\left\(\|Y\_\{j\}\|\>u\\right\)\\leq 2\\exp\\left\(\-\\frac\{du^\{2\}\}\{2R\_\{\\Lambda\}^\{2\}\}\\right\),\(47\)whereRΛ2=∑k∈𝒞co​\(𝐗\)Lk2/λk2R\_\{\\Lambda\}^\{2\}=\\sum\_\{k\\in\\mathcal\{C\}\_\{\\mathrm\{co\}\}\{\}\(\\mathbf\{X\}\)\}L\_\{k\}^\{2\}/\\lambda\_\{k\}^\{2\}\. Taking a union bound overddcoordinates gives

Pr\(‖\(𝐗co\)t,:𝚲−1𝐃𝐇⊤‖∞\>u\)≤2dexp\(−d​u22​RΛ2\)\.\\displaystyle\\Pr\\left\(\\left\\\|\(\\mathbf\{X\}^\{\\mathrm\{co\}\}\)\_\{t,:\}\\mathbf\{\\Lambda\}^\{\-1\}\\mathbf\{D\}\\mathbf\{H\}^\{\\top\}\\right\\\|\_\{\\infty\}\>u\\right\)\\leq 2d\\exp\\left\(\-\\frac\{du^\{2\}\}\{2R\_\{\\Lambda\}^\{2\}\}\\right\)\.\(48\)Choosing

u2=2​RΛ2d​\(log⁡\(2​d\)\+log⁡1δ\)\\displaystyle u^\{2\}=\\frac\{2R\_\{\\Lambda\}^\{2\}\}\{d\}\\left\(\\log\\left\(2d\\right\)\+\\log\\frac\{1\}\{\\delta\}\\right\)\(49\)gives, with probability at least1−δ1\-\\delta,

‖\(𝐗co\)t,:𝚲−1𝐃𝐇⊤‖∞2≤2​RΛ2d\(log\(2d\)\+log1δ\)\\displaystyle\\left\\\|\(\\mathbf\{X\}^\{\\mathrm\{co\}\}\)\_\{t,:\}\\mathbf\{\\Lambda\}^\{\-1\}\\mathbf\{D\}\\mathbf\{H\}^\{\\top\}\\right\\\|\_\{\\infty\}^\{2\}\\leq\\frac\{2R\_\{\\Lambda\}^\{2\}\}\{d\}\\left\(\\log\\left\(2d\\right\)\+\\log\\frac\{1\}\{\\delta\}\\right\)\(50\)Because the persistent sparse CO vector is shared across tokens, summing overTTtokens yields

𝒥co​\(𝚲,𝐃\)≤2​Td​RΛ2​\(log⁡\(2​d\)\+log⁡1δ\)\\displaystyle\\mathcal\{J\}\_\{\\mathrm\{co\}\}\(\\mathbf\{\\Lambda\},\\mathbf\{D\}\)\\leq\\frac\{2T\}\{d\}R\_\{\\Lambda\}^\{2\}\\left\(\\log\\left\(2d\\right\)\+\\log\\frac\{1\}\{\\delta\}\\right\)\(51\)SinceRΛ2≤Nco​\(𝐗\)​LΛ,max2R\_\{\\Lambda\}^\{2\}\\leq N\_\{\\mathrm\{co\}\}\{\}\(\\mathbf\{X\}\)L\_\{\\Lambda,\\max\}^\{2\}, the statedNco​\(𝐗\)N\_\{\\mathrm\{co\}\}\{\}\(\\mathbf\{X\}\)\-dependent bound follows\.

#### Sampled rotation\.

ConsiderNsN\_\{s\}independently sampled sign matrices𝒟Ns=\{𝐃\(1\),…,𝐃\(Ns\)\}\\mathcal\{D\}\_\{N\_\{s\}\}=\\\{\\mathbf\{D\}^\{\(1\)\},\\ldots,\\mathbf\{D\}^\{\(N\_\{s\}\)\}\\\}\. For a single sample,

Pr⁡\(𝒥co​\(𝚲,𝐃\)\>T​u2\)≤2​d​exp⁡\(−d​u22​RΛ2\)\.\\displaystyle\\Pr\\left\(\\mathcal\{J\}\_\{\\mathrm\{co\}\}\(\\mathbf\{\\Lambda\},\\mathbf\{D\}\)\>Tu^\{2\}\\right\)\\leq 2d\\exp\\left\(\-\\frac\{du^\{2\}\}\{2R\_\{\\Lambda\}^\{2\}\}\\right\)\.\(52\)Hence,

Pr⁡\(min𝐃∈𝒟Ns⁡𝒥co​\(𝚲,𝐃\)\>T​u2\)≤\[2​d​exp⁡\(−d​u22​RΛ2\)\]Ns\.\\displaystyle\\Pr\\left\(\\min\_\{\\mathbf\{D\}\\in\\mathcal\{D\}\_\{N\_\{s\}\}\}\\mathcal\{J\}\_\{\\mathrm\{co\}\}\(\\mathbf\{\\Lambda\},\\mathbf\{D\}\)\>Tu^\{2\}\\right\)\\leq\\left\[2d\\exp\\left\(\-\\frac\{du^\{2\}\}\{2R\_\{\\Lambda\}^\{2\}\}\\right\)\\right\]^\{N\_\{s\}\}\.\(53\)Setting the right\-hand side toδ\\deltagives

u2=2​RΛ2d​\(log⁡\(2​d\)\+1Ns​log⁡1δ\)\.\\displaystyle u^\{2\}=\\frac\{2R\_\{\\Lambda\}^\{2\}\}\{d\}\\left\(\\log\(2d\)\+\\frac\{1\}\{N\_\{s\}\}\\log\\frac\{1\}\{\\delta\}\\right\)\.\(54\)Therefore, with probability at least1−δ1\-\\delta,

min𝐃∈𝒟Ns⁡𝒥co​\(𝚲,𝐃\)≤2​Td​RΛ2​\(log⁡\(2​d\)\+1Ns​log⁡1δ\)\.\\displaystyle\\min\_\{\\mathbf\{D\}\\in\\mathcal\{D\}\_\{N\_\{s\}\}\}\\mathcal\{J\}\_\{\\mathrm\{co\}\}\(\\mathbf\{\\Lambda\},\\mathbf\{D\}\)\\leq\\frac\{2T\}\{d\}R\_\{\\Lambda\}^\{2\}\\left\(\\log\(2d\)\+\\frac\{1\}\{N\_\{s\}\}\\log\\frac\{1\}\{\\delta\}\\right\)\.\(55\)Using againRΛ2≤Nco​\(𝐗\)​LΛ,max2R\_\{\\Lambda\}^\{2\}\\leq N\_\{\\mathrm\{co\}\}\{\}\(\\mathbf\{X\}\)L\_\{\\Lambda,\\max\}^\{2\}gives the stated bound\. ∎

### B\.3Proof of Proposition[1](https://arxiv.org/html/2609.21450#Thmproposition1)

###### Proof\.

Let𝒓t=\(𝐗reg\)t,:𝚲−1\\bm\{r\}\_\{t\}=\(\\mathbf\{X\}^\{\\mathrm\{reg\}\}\)\_\{t,:\}\\mathbf\{\\Lambda\}^\{\-1\}\. For a fixed tokenttand coordinatejj,

Yt,j=\(𝒓t​𝐃𝐇⊤\)j=1d​∑k=1drt,k​εk​hk,j\.\\displaystyle Y\_\{t,j\}=\\left\(\\bm\{r\}\_\{t\}\\mathbf\{D\}\\mathbf\{H\}^\{\\top\}\\right\)\_\{j\}=\\frac\{1\}\{\\sqrt\{d\}\}\\sum\_\{k=1\}^\{d\}r\_\{t,k\}\\varepsilon\_\{k\}h\_\{k,j\}\.\(56\)This is a centered Rademacher sum\. By Hoeffding’s inequality,

Pr⁡\(\|Yt,j\|\>u\)≤2​exp⁡\(−d​u22​‖𝒓t‖22\)\.\\displaystyle\\Pr\\left\(\|Y\_\{t,j\}\|\>u\\right\)\\leq 2\\exp\\left\(\-\\frac\{du^\{2\}\}\{2\\left\\\|\\bm\{r\}\_\{t\}\\right\\\|\_\{2\}^\{2\}\}\\right\)\.\(57\)Taking a union bound over allTTtokens andddcoordinates, with probability at least1−δ1\-\\delta,

‖𝒓t​𝐃𝐇⊤‖∞2≤2​log⁡\(2​d​T/δ\)d​‖𝒓t‖22\\displaystyle\\left\\\|\\bm\{r\}\_\{t\}\\mathbf\{D\}\\mathbf\{H\}^\{\\top\}\\right\\\|\_\{\\infty\}^\{2\}\\leq\\frac\{2\\log\(2dT/\\delta\)\}\{d\}\\left\\\|\\bm\{r\}\_\{t\}\\right\\\|\_\{2\}^\{2\}\(58\)simultaneously for alltt\. Summing overttgives

∑t=1T‖\(𝐗reg\)t,:𝚲−1𝐃𝐇⊤‖∞2\\displaystyle\\sum\_\{t=1\}^\{T\}\\left\\\|\(\\mathbf\{X\}^\{\\mathrm\{reg\}\}\)\_\{t,:\}\\mathbf\{\\Lambda\}^\{\-1\}\\mathbf\{D\}\\mathbf\{H\}^\{\\top\}\\right\\\|\_\{\\infty\}^\{2\}≤2​log⁡\(2​d​T/δ\)d∑t=1T‖\(𝐗reg\)t,:𝚲−1‖22\\displaystyle\\leq\\frac\{2\\log\(2dT/\\delta\)\}\{d\}\\sum\_\{t=1\}^\{T\}\\left\\\|\(\\mathbf\{X\}^\{\\mathrm\{reg\}\}\)\_\{t,:\}\\mathbf\{\\Lambda\}^\{\-1\}\\right\\\|\_\{2\}^\{2\}\(59\)=2​log⁡\(2​d​T/δ\)d​‖𝐗reg​𝚲−1‖F2\.\\displaystyle=\\frac\{2\\log\(2dT/\\delta\)\}\{d\}\\left\\\|\\mathbf\{X\}^\{\\mathrm\{reg\}\}\\mathbf\{\\Lambda\}^\{\-1\}\\right\\\|\_\{F\}^\{2\}\.\(60\)Multiplying both sides by‖𝐖​𝚲‖22\\left\\\|\\mathbf\{W\}\\mathbf\{\\Lambda\}\\right\\\|\_\{2\}^\{2\}yields

𝒥reg​\(𝚲,𝐃\)≤2​log⁡\(2​d​T/δ\)d​‖𝐖​𝚲‖22​‖𝐗reg​𝚲−1‖F2\.\\displaystyle\\mathcal\{J\}\_\{\\mathrm\{reg\}\}\(\\mathbf\{\\Lambda\},\\mathbf\{D\}\)\\leq\\frac\{2\\log\(2dT/\\delta\)\}\{d\}\\left\\\|\\mathbf\{W\}\\mathbf\{\\Lambda\}\\right\\\|\_\{2\}^\{2\}\\left\\\|\\mathbf\{X\}^\{\\mathrm\{reg\}\}\\mathbf\{\\Lambda\}^\{\-1\}\\right\\\|\_\{F\}^\{2\}\.\(61\)Thus, the regular quantity𝒥reg\\mathcal\{J\}\_\{\\mathrm\{reg\}\}is controlled by‖𝐖​𝚲‖22​‖𝐗reg​𝚲−1‖F2\\\|\\mathbf\{W\}\\mathbf\{\\Lambda\}\\\|\_\{2\}^\{2\}\\\|\\mathbf\{X\}^\{\\mathrm\{reg\}\}\\mathbf\{\\Lambda\}^\{\-1\}\\\|\_\{F\}^\{2\}up to constants and logarithmic factors\.

We derive a channel\-wise scaling rule using the Frobenius surrogate

ℛ2​\(𝚲\)=‖𝐖​𝚲‖F2​‖𝐗reg​𝚲−1‖F2\.\\displaystyle\\mathcal\{R\}\_\{2\}\(\\mathbf\{\\Lambda\}\)=\\left\\\|\\mathbf\{W\}\\mathbf\{\\Lambda\}\\right\\\|\_\{F\}^\{2\}\\left\\\|\\mathbf\{X\}^\{\\mathrm\{reg\}\}\\mathbf\{\\Lambda\}^\{\-1\}\\right\\\|\_\{F\}^\{2\}\.\(62\)Letak=∥\(𝐗reg\):,k∥22a\_\{k\}=\\\|\(\\mathbf\{X\}^\{\\mathrm\{reg\}\}\)\_\{:,k\}\\\|\_\{2\}^\{2\}andbk=∥\(𝐖\):,k∥22b\_\{k\}=\\\|\(\\mathbf\{W\}\)\_\{:,k\}\\\|\_\{2\}^\{2\}\. Then,

ℛ2​\(𝚲\)=\(∑k=1dλk2​bk\)​\(∑k=1dakλk2\)\.\\displaystyle\\mathcal\{R\}\_\{2\}\(\\mathbf\{\\Lambda\}\)=\\left\(\\sum\_\{k=1\}^\{d\}\\lambda\_\{k\}^\{2\}b\_\{k\}\\right\)\\left\(\\sum\_\{k=1\}^\{d\}\\frac\{a\_\{k\}\}\{\\lambda\_\{k\}^\{2\}\}\\right\)\.\(63\)By Cauchy–Schwarz,

\(∑k=1dλk2​bk\)​\(∑k=1dakλk2\)≥\(∑k=1dak​bk\)2\.\\displaystyle\\left\(\\sum\_\{k=1\}^\{d\}\\lambda\_\{k\}^\{2\}b\_\{k\}\\right\)\\left\(\\sum\_\{k=1\}^\{d\}\\frac\{a\_\{k\}\}\{\\lambda\_\{k\}^\{2\}\}\\right\)\\geq\\left\(\\sum\_\{k=1\}^\{d\}\\sqrt\{a\_\{k\}b\_\{k\}\}\\right\)^\{2\}\.\(64\)Equality holds whenλk2​bk∝ak/λk2\\lambda\_\{k\}^\{2\}b\_\{k\}\\propto a\_\{k\}/\\lambda\_\{k\}^\{2\}, or equivalently,λk4∝ak/bk\\lambda\_\{k\}^\{4\}\\propto a\_\{k\}/b\_\{k\}\. Thus, up to a common positive scalar,

λk=\(akbk\)1/4=‖\(𝐗reg\):,k‖2‖\(𝐖\):,k‖2\.\\displaystyle\\lambda\_\{k\}=\\left\(\\frac\{a\_\{k\}\}\{b\_\{k\}\}\\right\)^\{1/4\}=\\sqrt\{\\frac\{\\left\\\|\(\\mathbf\{X\}^\{\\mathrm\{reg\}\}\)\_\{:,k\}\\right\\\|\_\{2\}\}\{\\left\\\|\(\\mathbf\{W\}\)\_\{:,k\}\\right\\\|\_\{2\}\}\}\.\(65\)This proves the statedL2L\_\{2\}scaling rule\. ∎

## Appendix CFull Results Across Models

Table[3](https://arxiv.org/html/2609.21450#A3.T3)reports C4 perplexity and the individual zero\-shot task accuracies omitted from Table[1](https://arxiv.org/html/2609.21450#S4.T1)\. All settings and baselines are identical to those in Section[5](https://arxiv.org/html/2609.21450#S5)\.

Table 3:Full W4A4 results with GPTAQ weight quantization and KV4 across model families and scales\.ModelBitsMethodPPL↓\\downarrowAccuracy \(%\)↑\\uparrowWT2C4PIQAARC\-EARC\-CHSWGLBAvg\.Llama\-7BW16A16N/A5\.678\.0379\.1672\.8544\.6276\.1970\.0173\.5169\.39\[2pt/1pt\]W4A4QuaRot6\.128\.8477\.5368\.0141\.3872\.8266\.7770\.5466\.17SmoothRot6\.138\.8777\.5867\.7240\.2772\.6667\.8870\.8366\.16SpinQuant6\.198\.9576\.4469\.0742\.3272\.7567\.0169\.5166\.18L2\-SmoothRot6\.098\.7677\.1569\.6541\.3073\.0167\.6471\.2866\.67Llama\-13BW16A16N/A5\.057\.4680\.2074\.7547\.8779\.0872\.6176\.1971\.78\[2pt/1pt\]W4A4QuaRot5\.428\.0078\.5172\.0146\.1676\.6669\.8573\.7869\.49SmoothRot5\.428\.0378\.6270\.5843\.9476\.6770\.8873\.8669\.09SpinQuant5\.448\.1378\.6271\.7646\.3376\.8969\.1473\.1269\.31L2\-SmoothRot5\.377\.9678\.2972\.1847\.0176\.6870\.7274\.6069\.91Llama2\-7BW16A16N/A5\.477\.9279\.1174\.5846\.2576\.0069\.0673\.9069\.82\[2pt/1pt\]W4A4QuaRot5\.968\.8376\.5570\.5441\.9872\.7865\.4370\.6666\.32SmoothRot5\.988\.8776\.3971\.0041\.7272\.8966\.1470\.5666\.45SpinQuant6\.069\.0376\.8269\.3241\.9872\.2864\.8070\.7065\.98L2\-SmoothRot5\.948\.8077\.8071\.0941\.0472\.3166\.7770\.5266\.59Llama2\-13BW16A16N/A4\.867\.3680\.5277\.5349\.0679\.3872\.1476\.7772\.57\[2pt/1pt\]W4A4QuaRot5\.248\.0178\.7373\.4445\.1476\.3170\.7274\.8969\.87SmoothRot5\.268\.0478\.4075\.5146\.9376\.2569\.1474\.6770\.15SpinQuant5\.268\.1477\.8075\.3846\.8476\.7969\.7774\.8170\.23L2\-SmoothRot5\.237\.9978\.5174\.5447\.7076\.9370\.6474\.6970\.50Llama3\-8BW16A16N/A5\.948\.8080\.8577\.7453\.4179\.1672\.6176\.0573\.30\[2pt/1pt\]W4A4QuaRot7\.4512\.1676\.6668\.7741\.7273\.5165\.5967\.9865\.71SmoothRot7\.5712\.4575\.3067\.0543\.0073\.3469\.0667\.3865\.85SpinQuant7\.3312\.0476\.9374\.9246\.5075\.4567\.8868\.5068\.36L2\-SmoothRot7\.2011\.7177\.0973\.3245\.5674\.8568\.3569\.4568\.10Llama3\.2\-1BW16A16N/A9\.4113\.0574\.5460\.6136\.3563\.6860\.6962\.9559\.80\[2pt/1pt\]W4A4QuaRot13\.8121\.5267\.7450\.8830\.2053\.1953\.9141\.5749\.58SmoothRot14\.1322\.0567\.1950\.7630\.3852\.4253\.5137\.5148\.63SpinQuant13\.3320\.4666\.8751\.4730\.8055\.1952\.8845\.0250\.37L2\-SmoothRot12\.9219\.8768\.5551\.8131\.8354\.1753\.0444\.8350\.71Llama3\.2\-3BW16A16N/A7\.5510\.4677\.4871\.7245\.9973\.5870\.0170\.4468\.20\[2pt/1pt\]W4A4QuaRot9\.2714\.2974\.0561\.9938\.1467\.5363\.0661\.9361\.12SmoothRot9\.3514\.5272\.8060\.1938\.3166\.4261\.0960\.6659\.91SpinQuant9\.1914\.3673\.7263\.9341\.2168\.2263\.7761\.3262\.03L2\-SmoothRot9\.0814\.0472\.4758\.5439\.3367\.9464\.4062\.9760\.94Mistral\-7B\-v0\.3W16A16N/A5\.358\.2882\.2678\.2452\.2280\.4773\.8875\.3373\.73\[2pt/1pt\]W4A4QuaRot5\.738\.9480\.1475\.6747\.7077\.7069\.4672\.6070\.55SmoothRot5\.778\.9980\.4173\.9545\.5677\.4068\.7572\.7069\.80SpinQuant5\.748\.9580\.4175\.8849\.1577\.9670\.0173\.6171\.17L2\-SmoothRot5\.718\.8779\.4975\.7648\.8978\.0470\.0973\.7671\.00