当更低的重建损失反而有害时：低比特 LLM 量化的分布鲁棒优化

# When Lower Reconstruction Loss Hurts:Distributionally Robust Refinement for Low-Bit LLM Quantization
Source: [https://arxiv.org/html/2610.11226](https://arxiv.org/html/2610.11226)
Yanlong Zhao††thanks:Equal Contribution:[Yanlong Zhao](mailto:[email protected]),[Xiaoyuan Cheng](mailto:[email protected])and[Huihang Liu](mailto:[email protected])\.Xiaoyuan Cheng11footnotemark:1Affiliation:University College LondonHuihang Liu11footnotemark:1Affiliation:Shanghai University of Finance and EconomicsBaihua HeAffiliation:University of Science and Technology of ChinaXinyu ZhangAffiliation:University of Science and Technology of ChinaAffiliation:AMSS, Chinese Academy of SciencesHarrison Bo Hua ZhuAffiliation:University of CopenhagenAffiliation:Imperial College LondonAffiliation:Technical University of DenmarkWenlong ChenAffiliation:Imperial College LondonLi ZengAffiliation:Shenzhen Research Institute of Big DataZhuo Sun††thanks:Correspondence Author\. Correspondence to[Zhuo Sun](mailto:[email protected])\.Affiliation:Shanghai University of Finance and EconomicsAffiliation:Imperial College London

###### Abstract

Weight\-only post\-training quantization \(PTQ\) relies heavily on reconstruction loss minimization to preserve model quality at low precision\. We show that the weights favored by minimizing this loss need not yield better model performance on new tasks\. In fact, we find that lower reconstruction loss can even degrade model performance on the same calibration data\. Our analysis further shows that weights with lower reconstruction loss on calibration data can have higher loss than other weights when the distribution of input activations changes\. Motivated by these observations and our analysis, we propose Distributionally Robust Quantization \(DRQ\), a post\-hoc refinement process that minimizes worst\-case reconstruction loss over a constrained set of input activation distributions\. DRQ refines the integer codes representing quantized weights within the existing quantization grid, keeping quantization parameters and inference operators unchanged\. Extensive experiments show that DRQ improves models quantized by six representative PTQ methods, including AWQ, GPTQ, and ParoQuant, and delivers gains across both dense and mixture\-of\-experts large language models\. These results establish DRQ as a general post\-hoc refinement framework for weight\-only PTQ, achieving better downstream performance without adding inference overhead\.

## 1Introduction

Large language models \(LLMs\) have demonstrated strong capabilities in language understanding, reasoning, and code generation\([OpenAI et al\., 2023](https://arxiv.org/html/2610.11226#bib.bib11);[Grattafiori et al\., 2024](https://arxiv.org/html/2610.11226#bib.bib7);[Chen et al\., 2021](https://arxiv.org/html/2610.11226#bib.bib12);[DeepSeek\-AI, 2026](https://arxiv.org/html/2610.11226#bib.bib3)\)\. Their increasing memory footprint and bandwidth demands, however, make deployment costly on resource\-constrained devices\. Weight\-only post\-training quantization \(PTQ\) addresses this problem by compressing pretrained weights into low\-bit formats such as INT4 while retaining the original activation precision\. It reduces weight storage and memory traffic to accelerate memory\-bound autoregressive decoding\([Frantar et al\., 2023](https://arxiv.org/html/2610.11226#bib.bib4);[Lin et al\., 2024](https://arxiv.org/html/2610.11226#bib.bib6);[Ashkboos et al\., 2024](https://arxiv.org/html/2610.11226#bib.bib1)\)\.

Most weight\-only PTQ methods rely on minimizing reconstruction loss on calibration data to preserve model quality\([Frantar et al\., 2023](https://arxiv.org/html/2610.11226#bib.bib4);[Lin et al\., 2024](https://arxiv.org/html/2610.11226#bib.bib6);[Shao et al\., 2024](https://arxiv.org/html/2610.11226#bib.bib8);[Tseng et al\., 2024](https://arxiv.org/html/2610.11226#bib.bib27);[Li et al\., 2025](https://arxiv.org/html/2610.11226#bib.bib5);[Sun et al\., 2025](https://arxiv.org/html/2610.11226#bib.bib23);[Liang et al\., 2026](https://arxiv.org/html/2610.11226#bib.bib22)\)\. For a linear layer, this means matching the output of the quantized weights to that of the full\-precision weights on the calibration data,

min𝐖^∈𝒬⁡ℒ⁡\(𝐖^\),ℒ⁡\(𝐖^\)=1n​‖𝐖^​𝐗−𝐖𝐗‖F2,\\min\_\{\\widehat\{\\mathbf\{W\}\}\\in\\mathcal\{Q\}\}\\;\\mathcal\{L\}\(\\widehat\{\\mathbf\{W\}\}\),\\qquad\\mathcal\{L\}\(\\widehat\{\\mathbf\{W\}\}\)=\\frac\{1\}\{n\}\\left\\lVert\\widehat\{\\mathbf\{W\}\}\\mathbf\{X\}\-\\mathbf\{W\}\\mathbf\{X\}\\right\\rVert\_\{\\rm F\}^\{2\},\(1\)where𝐖\\mathbf\{W\}and𝐖^\\widehat\{\\mathbf\{W\}\}denote the full\-precision and quantized weights, respectively, the quantization grid𝒬\\mathcal\{Q\}denotes the set of all possible quantized weights, and𝐗\\mathbf\{X\}denotes the associated input activations of the calibration data\. Methods differ in how they minimize this loss\. For example, GPTQ compensates for rounding errors\([Frantar et al\., 2023](https://arxiv.org/html/2610.11226#bib.bib4)\), AWQ searches for scaling and clipping parameters\([Lin et al\., 2024](https://arxiv.org/html/2610.11226#bib.bib6)\), and ParoQuant learns transformations\([Liang et al\., 2026](https://arxiv.org/html/2610.11226#bib.bib22)\)\.

We show that reducing reconstruction loss on calibration data can degrade the performance of the quantized model\. We first quantize Llama\-3\.2\-1B\-Instruct with 3\-bit AWQ using WikiText\-2 \(WT2\)\([Merity et al\., 2017](https://arxiv.org/html/2610.11226#bib.bib10)\)as the calibration dataset\. Starting from the weights quantized by AWQ, we further optimize one linear layer at a time to reduce reconstruction loss on the calibration data\. Specifically, we search over the integer codes representing its weights, while keeping the scales, zero points, and all other layers’ weights fixed\. We repeat this procedure at 16 layers of the model using two different calibration sets sampled from WT2, producing 32 modified weight matrices\. In 18 of 32 cases, the modified weights yield lower reconstruction loss but higher model perplexity \(PPL\) on the same calibration data \([Figure1](https://arxiv.org/html/2610.11226#S1.F1)a\)\. This mismatch also extends to held\-out data\. In a separate experiment, we examine all 112 linear layers of Llama\-3\.2\-1B\-Instruct quantized with 3\-bit AWQ\. For each layer, we test integer\-code replacements while keeping the scales and zero points fixed\. In each row of the weight matrix, we retain the single\-code change that most reduces reconstruction loss on calibration data\. We sort these changes from the largest loss reduction to the smallest\. Starting from the original quantized weights each time, we create five modified weight matrices by applying the top1/2561/256,1/1281/128,1/641/64,1/321/32, and1/161/16of the changes in this list, respectively\. This finally yields 560 modified weight matrices, all with lower reconstruction loss on the calibration data than the corresponding original quantized weights\. We then evaluate each matrix by replacing only the corresponding layer’s weights and measuring model PPL on WT2 data reserved for evaluation\. Of these 560 replacements, 413 \(73\.8%\) increase PPL relative to the initial quantized model\.

Figure 1:Lower reconstruction loss need not improve quantized model performance\. Results are from Llama\-3\.2\-1B\-Instruct with 3\-bit quantization\. \(a\) Among 32 modifications to AWQ\-quantized weights, 18 reduce reconstruction loss but increase model PPL on the same WT2 calibration data\. \(b\) Among 560 modified weight matrices per method that reduce reconstruction loss on calibration data, 413 \(73\.8%\) for AWQ and 264 \(47\.1%\) for GPTQ increase model PPL on held\-out WT2 data\. \(c\) Further minimizing reconstruction loss increases held\-out C4 PPL by 0\.69%, whereas DRQ accepts slightly higher reconstruction loss on calibration data and lowers C4 PPL by 0\.32%\. Each evaluation modifies one linear layer, keeping all other weights fixed\. In \(a,c\), reconstruction loss is divided by its initial value, and PPL changes are relative to the initial quantized model\.These results expose a mismatch between reducing reconstruction loss on calibration data and improving the performance of the quantized model\. Ideally, minimizing reconstruction loss brings the outputs of quantized layers closer to those of their full\-precision counterparts on calibration data\. Yet this closer match need not generalize to other datasets or improve model performance even on the same calibration data\. Optimizing reconstruction loss alone can therefore degrade the performance of the quantized model\.

To figure out this problem, we study how this loss depends on the input activations observed during calibration\. For example, when an input channel of a linear layer stays close to zero, even large differences between the quantized and full\-precision weights in the corresponding column can have little effect on the layer outputs\. Yet new data could produce larger activations in this channel, so the same weight differences can cause larger output differences and thus higher reconstruction loss\. Our analysis identifies when a quantized weight matrix with lower reconstruction loss on calibration data can have higher loss than another under constrained changes in the distribution of input activations \([Theorem3\.1](https://arxiv.org/html/2610.11226#S3.Thmtheorem1)\)\.

Motivated by this analysis, we propose*Distributionally Robust Quantization \(DRQ\)*, a general refinement framework for weight\-only PTQ\. DRQ minimizes the worst\-case reconstruction loss over a set of input activation distributions, with constraints on how these distributions differ from the calibration distribution\. This objective combines reconstruction loss on calibration data with its largest possible increase under these constraints\. A real illustrative example in[Figure1](https://arxiv.org/html/2610.11226#S1.F1)c shows that further minimizing reconstruction loss on calibration data increases model PPL on held\-out C4\. DRQ instead accepts a small increase in reconstruction loss on calibration data and lowers PPL on held\-out C4 relative to the initial model quantized by AWQ\. The resulting quantized weights therefore perform better on data not used for calibration\. DRQ updates the integer codes of quantized weights while keeping the bit width, grouping, scales, zero points, and inference operators unchanged\. All refinement is offline, so DRQ adds no inference overhead\.

#### Contributions\.

We uncover a counterintuitive phenomenon in weight\-only PTQ\. Lower reconstruction loss on calibration data does not necessarily lead to better performance of the quantized model, even on the same data\. Motivated by this finding, we analyze how changes in input activation distributions affect reconstruction loss\. This analysis leads to DRQ, which refines quantized weights by minimizing the worst\-case reconstruction loss over a constrained set of input activation distributions\. DRQ updates the integer codes representing quantized weights while preserving the quantization format and inference operators, so these improvements add no inference overhead\. Experiments show that DRQ improves models quantized by six representative PTQ methods, with gains across both dense and mixture\-of\-experts \(MoE\) architectures\.

## 2Preliminaries

### 2\.1Weight\-only quantization on a fixed grid

Quantization approximates high\-precision values using a finite set of values that can be stored with fewer bits\. Weight\-only post\-training quantization applies this to pretrained weights while retaining the original activation precision\. Consider a linear layer with weights𝐖∈ℝm×d\\mathbf\{W\}\\in\\mathbb\{R\}^\{m\\times d\}, whereddandmmare the input and output dimensions\. Abb\-bit uniform quantizer represents each weightwwby an integer codeq∈𝒵b=\{0,…,2b−1\}q\\in\\mathcal\{Z\}\_\{b\}=\\\{0,\\ldots,2^\{b\}\-1\\\}\. With round\-to\-nearest \(RTN\), we compute the codeqqand its corresponding quantized weightw^\\hat\{w\}as

q=clip⁡\(round⁡\(w/s\)\+z,0,2b−1\),w^=s⁡\(q−z\)\.q=\\operatorname\{clip\}\\\!\\left\(\\operatorname\{round\}\(w/s\)\+z,\\,0,\\,2^\{b\}\-1\\right\),\\qquad\\hat\{w\}=s\(q\-z\)\.
Heres\>0s\>0is the scale, which sets the spacing between adjacent quantized values, and the zero pointzzis the integer code corresponding to zero\. The rounding operation selects the nearest integer, and clipping keeps the code between00and2b−12^\{b\}\-1\. Following common practice in most PTQ work, we use group\-wise quantization\([Dettmers and Zettlemoyer, 2023](https://arxiv.org/html/2610.11226#bib.bib46);[Liang et al\., 2026](https://arxiv.org/html/2610.11226#bib.bib22)\), which divides each row of𝐖\\mathbf\{W\}into groups ofggconsecutive weights along the input dimension\. Each group shares a scalessand a zero pointzz\. Large weights in one group therefore do not force other groups to use a larger quantization step, which helps represent smaller weights more accurately\. Withssandzzfixed,w^=s⁡\(q−z\)\\hat\{w\}=s\(q\-z\)shows that each quantized weight is determined by its integer codeqq\. DRQ therefore refines the weights by updating only their integer codes\.

### 2\.2Reconstruction on calibration data

Calibration uses sample text to collect the input activations reaching each layer\. Let the input activation matrix𝐗=\[x1,…,xn\]∈ℝd×n\\mathbf\{X\}=\[x\_\{1\},\\ldots,x\_\{n\}\]\\in\\mathbb\{R\}^\{d\\times n\}contain these vectors, with one token per column\. The full\-precision and quantized outputs are𝐖𝐗\\mathbf\{W\}\\mathbf\{X\}and𝐖^​𝐗\\widehat\{\\mathbf\{W\}\}\\mathbf\{X\}\. Their squared difference, summed over output channels and averaged over tokens, is the reconstruction loss

ℒ⁡\(𝐖^\)=1n​‖𝐖^​𝐗−𝐖𝐗‖F2=tr⁡\(\(𝐖^−𝐖\)​𝐇​\(𝐖^−𝐖\)⊤\),\\mathcal\{L\}\(\\widehat\{\\mathbf\{W\}\}\)=\\frac\{1\}\{n\}\\left\\lVert\\widehat\{\\mathbf\{W\}\}\\mathbf\{X\}\-\\mathbf\{W\}\\mathbf\{X\}\\right\\rVert\_\{\\rm F\}^\{2\}=\\operatorname\{tr\}\\\!\\left\(\(\\widehat\{\\mathbf\{W\}\}\-\\mathbf\{W\}\)\\mathbf\{H\}\(\\widehat\{\\mathbf\{W\}\}\-\\mathbf\{W\}\)^\{\\top\}\\right\),\(2\)where𝐇=n−1​𝐗𝐗⊤\\mathbf\{H\}=n^\{\-1\}\\mathbf\{X\}\\mathbf\{X\}^\{\\top\}is the activation second\-moment matrix\. The squared Frobenius norm sums the squared differences between the two output matrices, andtr\\operatorname\{tr\}sums matrix diagonal entries\. Most PTQ methods such as GPTQ, AWQ, and OmniQuant\([Frantar et al\., 2023](https://arxiv.org/html/2610.11226#bib.bib4);[Lin et al\., 2024](https://arxiv.org/html/2610.11226#bib.bib6);[Shao et al\., 2024](https://arxiv.org/html/2610.11226#bib.bib8)\)minimize reconstruction loss in different ways to obtain quantized weights𝐖^\\widehat\{\\mathbf\{W\}\}\.

## 3DRQ: Distributionally Robust Quantization

![Refer to caption](https://arxiv.org/html/2610.11226v1/code_selection_schematic.png)Figure 2:An illustrative example of integer\-code refinement with DRQ on Llama\-3\.2\-1B\-Instruct quantized with 2\-bit GPTQ\. We select two integer codes,q1q\_\{1\}andq2q\_\{2\}, from the quantized weights and plot the loss contours over different pairs of code values\. The percentages indicate changes in reconstruction loss relative to the initial quantized weights\. Panel \(a\) shows reconstruction loss on the WikiText\-2 \(WT2\) calibration data, panel \(b\) illustrates how DRQ refines the two integer codes, and panel \(c\) shows reconstruction loss on C4\. Although DRQ increases reconstruction loss on the calibration data by 1\.78%, it reduces reconstruction loss on C4 by 15\.11%, both relative to the initial quantized weights\.In this section, we first analyze how changes in the distribution of input activations affect reconstruction loss \([Section3\.1](https://arxiv.org/html/2610.11226#S3.SS1)\)\. Based on this analysis, we formulate a worst\-case reconstruction objective and derive an equivalent form for optimizing quantized weights \([Section3\.2](https://arxiv.org/html/2610.11226#S3.SS2)\)\. Finally, we describe how DRQ optimizes this objective by updating the integer codes of weights produced by an existing PTQ method, while keeping the quantization parameters fixed \([Section3\.3](https://arxiv.org/html/2610.11226#S3.SS3)\)\.

### 3\.1Reconstruction under changing input distributions

New data can produce input activations that differ from those observed during calibration\. Given the quantized weights𝐖^\\widehat\{\\mathbf\{W\}\},[Equation2](https://arxiv.org/html/2610.11226#S2.E2)shows that the effect of input activations on reconstruction loss is determined by the second\-moment matrix𝐇\\mathbf\{H\}\. We can therefore study how new data affect this loss by examining changes in that matrix\. To describe these changes, we introduce a perturbation matrix𝚫\\boldsymbol\{\\Delta\}and write the second\-moment matrix of the new input activations as𝐇\+𝚫\\mathbf\{H\}\+\\boldsymbol\{\\Delta\}\. Here,𝐇\\mathbf\{H\}is computed from the calibration data, and𝚫\\boldsymbol\{\\Delta\}is the difference in second\-moment matrices between the new and the calibration data\.

To examine how changes in the input distribution can increase reconstruction loss, we restrict𝚫\\boldsymbol\{\\Delta\}to be positive semidefinite, written𝚫⪰0\\boldsymbol\{\\Delta\}\\succeq 0\. Under this constraint, the expected reconstruction loss of any fixed quantized weights can increase or remain unchanged\. One example is adding noise of the formε​u\\varepsilon uto an input activation vectorxx, whereε\\varepsilonhas mean zero and is independent ofxx\. We introduce an*input direction*u∈ℝdu\\in\\mathbb\{R\}^\{d\}to describe how the input channels change and measure the resulting increase in reconstruction loss\. In the noise example above, its entries specify the sign and relative size of the noise added to each channel\. For two input channels,u=\(1,1\)⊤u=\(1,1\)^\{\\top\}means addingε\\varepsilonto both channels, so they increase or decrease together\. Withu=\(1,−1\)⊤u=\(1,\-1\)^\{\\top\}, we addε\\varepsilonto the first channel and subtract it from the second, so one increases while the other decreases\. In this example, the second\-moment matrix changes by𝚫=𝔼⁡\[ε2\]​u​u⊤⪰0\\boldsymbol\{\\Delta\}=\\mathbb\{E\}\[\\varepsilon^\{2\}\]uu^\{\\top\}\\succeq 0\. For fixed quantized weights, this change increases the expected reconstruction loss by𝔼⁡\[ε2\]​‖\(𝐖^−𝐖\)​u‖22\\mathbb\{E\}\[\\varepsilon^\{2\}\]\\left\\lVert\(\\widehat\{\\mathbf\{W\}\}\-\\mathbf\{W\}\)u\\right\\rVert\_\{2\}^\{2\}\. Comparing input directions under the same distribution\-change budget therefore lets us identify the changes that increase this loss the most\.

The loss increase in the example above depends on both the input direction and the size of the added noise\. Without a limit on the noise, reconstruction loss can grow arbitrarily large\. We therefore imposetr⁡\(𝐂​𝚫\)≤α\\operatorname\{tr\}\(\\mathbf\{C\}\\boldsymbol\{\\Delta\}\)\\leq\\alpha, whereα≥0\\alpha\\geq 0is the distribution\-change budget\. The positive\-definite matrix𝐂\\mathbf\{C\}determines how much change is permitted along each input direction\. For a change𝚫=t​u​u⊤\\boldsymbol\{\\Delta\}=tuu^\{\\top\}witht≥0t\\geq 0, the constraint becomest​u⊤​𝐂​u≤αt\\,u^\{\\top\}\\mathbf\{C\}u\\leq\\alpha\. Thus, a largeru⊤​𝐂​uu^\{\\top\}\\mathbf\{C\}upermits a smallerttunder the same budget\. To compare all directions at the same budget, we rescale eachuuto satisfyu⊤​𝐂​u=1u^\{\\top\}\\mathbf\{C\}u=1\. Then𝚫=α​u​u⊤\\boldsymbol\{\\Delta\}=\\alpha uu^\{\\top\}uses the full budgetα\\alpha\.

Panels \(a\) and \(c\) of[Figure2](https://arxiv.org/html/2610.11226#S3.F2.fig1)illustrate how weights with lower reconstruction loss on calibration data can have higher loss on new inputs\. Consider two quantized weight matrices withℒ⁡\(𝐖^1\)<ℒ⁡\(𝐖^2\)\\mathcal\{L\}\(\\widehat\{\\mathbf\{W\}\}\_\{1\}\)<\\mathcal\{L\}\(\\widehat\{\\mathbf\{W\}\}\_\{2\}\)\. Input changes can increase both losses, but increase the loss of𝐖^1\\widehat\{\\mathbf\{W\}\}\_\{1\}more\. For its loss to remain no greater than that of𝐖^2\\widehat\{\\mathbf\{W\}\}\_\{2\}, the difference between these increases must not exceed the original loss differenceℒ⁡\(𝐖^2\)−ℒ⁡\(𝐖^1\)\\mathcal\{L\}\(\\widehat\{\\mathbf\{W\}\}\_\{2\}\)\-\\mathcal\{L\}\(\\widehat\{\\mathbf\{W\}\}\_\{1\}\)\. The following theorem gives the exact condition\.

###### Theorem 3\.1\(Reconstruction loss under distribution changes\)\.

Let𝐖^1\\widehat\{\\mathbf\{W\}\}\_\{1\}and𝐖^2\\widehat\{\\mathbf\{W\}\}\_\{2\}satisfyℒ⁡\(𝐖^1\)<ℒ⁡\(𝐖^2\)\\mathcal\{L\}\(\\widehat\{\\mathbf\{W\}\}\_\{1\}\)<\\mathcal\{L\}\(\\widehat\{\\mathbf\{W\}\}\_\{2\}\)\. Consider input activation distributions with second\-moment matrix𝐇\+𝚫\\mathbf\{H\}\+\\boldsymbol\{\\Delta\}, where𝚫⪰0\\boldsymbol\{\\Delta\}\\succeq 0andtr⁡\(𝐂​𝚫\)≤α\\operatorname\{tr\}\(\\mathbf\{C\}\\boldsymbol\{\\Delta\}\)\\leq\\alpha\. The expected reconstruction loss of𝐖^1\\widehat\{\\mathbf\{W\}\}\_\{1\}is no greater than that of𝐖^2\\widehat\{\\mathbf\{W\}\}\_\{2\}for every such distribution if and only if every directionuunormalized byu⊤​𝐂​u=1u^\{\\top\}\\mathbf\{C\}u=1satisfies

α⁡\(‖\(𝐖^1−𝐖\)​u‖22−‖\(𝐖^2−𝐖\)​u‖22\)≤ℒ⁡\(𝐖^2\)−ℒ⁡\(𝐖^1\)\.\\alpha\\\!\\left\(\\left\\lVert\(\\widehat\{\\mathbf\{W\}\}\_\{1\}\-\\mathbf\{W\}\)u\\right\\rVert\_\{2\}^\{2\}\-\\left\\lVert\(\\widehat\{\\mathbf\{W\}\}\_\{2\}\-\\mathbf\{W\}\)u\\right\\rVert\_\{2\}^\{2\}\\right\)\\leq\\mathcal\{L\}\(\\widehat\{\\mathbf\{W\}\}\_\{2\}\)\-\\mathcal\{L\}\(\\widehat\{\\mathbf\{W\}\}\_\{1\}\)\.\(3\)If the inequality fails for a directionuuwithu⊤​𝐂​u=1u^\{\\top\}\\mathbf\{C\}u=1, the change𝚫=α​u​u⊤\\boldsymbol\{\\Delta\}=\\alpha uu^\{\\top\}satisfies both constraints and makes𝐖^1\\widehat\{\\mathbf\{W\}\}\_\{1\}have higher expected reconstruction loss than𝐖^2\\widehat\{\\mathbf\{W\}\}\_\{2\}\.

See Appendix[B\.3](https://arxiv.org/html/2610.11226#A2.SS3)for the proof\.[Theorem3\.1](https://arxiv.org/html/2610.11226#S3.Thmtheorem1)motivates a refinement objective that accounts for both reconstruction loss on calibration data and its largest possible increase under the distribution constraints above\. DRQ, introduced next, follows this approach\. In the schematic in[Figure2](https://arxiv.org/html/2610.11226#S3.F2.fig1), panel \(b\) illustrates how DRQ refines codes to accept slightly higher reconstruction loss on calibration data in panel \(a\) and achieve lower loss on held\-out inputs in panel \(c\)\.

### 3\.2Robust reconstruction objective

The preceding analysis compares two weight matrices under the same distribution changes\. For refinement, we need an objective for evaluating each weight matrix\. The proposed method, Distributionally Robust Quantization \(DRQ\), therefore considers the largest reconstruction loss that each matrix can have under the stated constraints and seeks weights that make this loss as small as possible\. Using the same distribution\-change budgetα\\alpha, we minimize

min𝐖^∈𝒬⁡max𝚫⪰0,tr⁡\(𝐂​𝚫\)≤α​tr⁡\(\(𝐖^−𝐖\)​\(𝐇\+𝚫\)​\(𝐖^−𝐖\)⊤\),\\min\_\{\\widehat\{\\mathbf\{W\}\}\\in\\mathcal\{Q\}\}\\;\\max\_\{\\boldsymbol\{\\Delta\}\\succeq 0,\\,\\operatorname\{tr\}\(\\mathbf\{C\}\\boldsymbol\{\\Delta\}\)\\leq\\alpha\}\\operatorname\{tr\}\\\!\\left\(\(\\widehat\{\\mathbf\{W\}\}\-\\mathbf\{W\}\)\(\\mathbf\{H\}\+\\boldsymbol\{\\Delta\}\)\(\\widehat\{\\mathbf\{W\}\}\-\\mathbf\{W\}\)^\{\\top\}\\right\)~,\(4\)
whereℒDRQ​\(𝐖^\):=max𝚫⪰0,tr⁡\(𝐂​𝚫\)≤α⁡tr⁡\(\(𝐖^−𝐖\)​\(𝐇\+𝚫\)​\(𝐖^−𝐖\)⊤\)\\mathcal\{L\}\_\{\\rm DRQ\}\(\\widehat\{\\mathbf\{W\}\}\):=\\max\_\{\\boldsymbol\{\\Delta\}\\succeq 0,\\,\\operatorname\{tr\}\(\\mathbf\{C\}\\boldsymbol\{\\Delta\}\)\\leq\\alpha\}\\operatorname\{tr\}\\\!\(\(\\widehat\{\\mathbf\{W\}\}\-\\mathbf\{W\}\)\(\\mathbf\{H\}\+\\boldsymbol\{\\Delta\}\)\(\\widehat\{\\mathbf\{W\}\}\-\\mathbf\{W\}\)^\{\\top\}\)\. The outer minimization selects quantized weights on the existing quantization grid that minimize this loss\. To evaluateℒDRQ​\(𝐖^\)\\mathcal\{L\}\_\{\\rm DRQ\}\(\\widehat\{\\mathbf\{W\}\}\), the following theorem reduces the inner maximization to finding the input directionuualong which reconstruction loss increases the most under the same distribution\-change budget\. The resulting expression is reconstruction loss on calibration data plus this worst\-case increase\.

###### Theorem 3\.2\(Worst\-case reconstruction loss\)\.

For any𝐖^∈ℝm×d\\widehat\{\\mathbf\{W\}\}\\in\\mathbb\{R\}^\{m\\times d\},𝐂≻0\\mathbf\{C\}\\succ 0, andα≥0\\alpha\\geq 0, the worst\-case loss in[Equation4](https://arxiv.org/html/2610.11226#S3.E4)satisfies

ℒDRQ​\(𝐖^\)=ℒ⁡\(𝐖^\)\+α​𝒮𝐂​\(𝐖^\),\\mathcal\{L\}\_\{\\rm DRQ\}\(\\widehat\{\\mathbf\{W\}\}\)=\\mathcal\{L\}\(\\widehat\{\\mathbf\{W\}\}\)\+\\alpha\\mathcal\{S\}\_\{\\mathbf\{C\}\}\(\\widehat\{\\mathbf\{W\}\}\),\(5\)where the*worst\-case directional error*is

𝒮𝐂​\(𝐖^\)=maxu⊤​𝐂​u=1⁡‖\(𝐖^−𝐖\)​u‖22\.\\mathcal\{S\}\_\{\\mathbf\{C\}\}\(\\widehat\{\\mathbf\{W\}\}\)=\\max\_\{u^\{\\top\}\\mathbf\{C\}u=1\}\\left\\lVert\(\\widehat\{\\mathbf\{W\}\}\-\\mathbf\{W\}\)u\\right\\rVert\_\{2\}^\{2\}\.\(6\)

For an input vectoruu,\(𝐖^−𝐖\)​u=𝐖^​u−𝐖​u\(\\widehat\{\\mathbf\{W\}\}\-\\mathbf\{W\}\)u=\\widehat\{\\mathbf\{W\}\}u\-\\mathbf\{W\}uis the difference between the quantized and full\-precision layer outputs\. Among directions rescaled to satisfyu⊤​𝐂​u=1u^\{\\top\}\\mathbf\{C\}u=1, the*worst\-case input direction*gives the largest squared output difference,𝒮𝐂​\(𝐖^\)\\mathcal\{S\}\_\{\\mathbf\{C\}\}\(\\widehat\{\\mathbf\{W\}\}\)\. A distribution change𝚫=α​u​u⊤\\boldsymbol\{\\Delta\}=\\alpha uu^\{\\top\}along this direction uses the full budget and increases reconstruction loss byα​𝒮𝐂​\(𝐖^\)\\alpha\\mathcal\{S\}\_\{\\mathbf\{C\}\}\(\\widehat\{\\mathbf\{W\}\}\)\. This is the largest increase among all distribution changes satisfying the constraints\.

In[Equation4](https://arxiv.org/html/2610.11226#S3.E4),α\\alphasets the budget for changes in the input activation distribution\. In the equivalent objective in[Equation5](https://arxiv.org/html/2610.11226#S3.E5), it weights the worst\-case directional error relative to reconstruction loss on calibration data\. DRQ can accept slightly higher reconstruction loss on calibration data when this reduces the largest possible increase enough to lower the total worst\-case loss\. Appendix[B\.5](https://arxiv.org/html/2610.11226#A2.SS5)gives the proof and a distribution change that attains this loss\.

We introduce𝐂\\mathbf\{C\}to account for input activations that remain small during calibration\. When an input channel stays near zero, even large differences between the quantized and full\-precision weights in the corresponding column can have little effect on the layer outputs\. We therefore consider larger possible increases in these activations when evaluating the worst\-case loss\. To apply the same idea to combinations of channels, we set𝐂=\(1−β\)​𝐇\+β​𝐈\\mathbf\{C\}=\(1\-\\beta\)\\mathbf\{H\}\+\\beta\\mathbf\{I\}, with0<β≤10<\\beta\\leq 1\. For directions with the same Euclidean norm, a smalleru⊤​𝐇​uu^\{\\top\}\\mathbf\{H\}ugives a smalleru⊤​𝐂​uu^\{\\top\}\\mathbf\{C\}uwhenβ<1\\beta<1, permitting a larger increase under the same budget\. Atβ=1\\beta=1, all directions with the same Euclidean norm are treated equally\. The identity term keeps𝐂\\mathbf\{C\}positive definite, so changes remain bounded in every direction\.

### 3\.3Refinement on a fixed quantization grid

DRQ processes one Transformer block at a time and refines each linear layer after the base PTQ method quantizes its weights\. It changes only integer codes, keeping the bit width, grouping, scales, and zero points fixed\.

\{wrapalgorithm\}

r0\.66

DRQ refinement of one linear layer

1:

Full\-precision weights𝐖\\mathbf\{W\}, initial quantized weights𝐖^\\widehat\{\\mathbf\{W\}\},calibration activations𝐗\\mathbf\{X\},α,β\\alpha,\\beta, maximum roundsKK
2:

𝐖^DRQ←𝐖^\\widehat\{\\mathbf\{W\}\}\_\{\\rm DRQ\}\\leftarrow\\widehat\{\\mathbf\{W\}\},

𝐇←𝐗𝐗⊤/n,𝐂←\(1−β\)​𝐇\+β​𝐈\\mathbf\{H\}\\leftarrow\\mathbf\{X\}\\mathbf\{X\}^\{\\top\}/n,\\quad\\mathbf\{C\}\\leftarrow\(1\-\\beta\)\\mathbf\{H\}\+\\beta\\mathbf\{I\}
3:for

r=1,…,Kr=1,\\ldots,Kdo

4:

u←arg​maxu⊤​𝐂​u=1⁡‖\(𝐖^DRQ−𝐖\)​u‖22u\\leftarrow\\argmax\_\{u^\{\\top\}\\mathbf\{C\}u=1\}\\left\\lVert\(\\widehat\{\\mathbf\{W\}\}\_\{\\rm DRQ\}\-\\mathbf\{W\}\)u\\right\\rVert\_\{2\}^\{2\}
5:Rank integer\-code updates by their reduction in

6:

ℒ⁡\(𝐖^DRQ\)\+α​‖\(𝐖^DRQ−𝐖\)​u‖22\\mathcal\{L\}\(\\widehat\{\\mathbf\{W\}\}\_\{\\rm DRQ\}\)\+\\alpha\\left\\lVert\(\\widehat\{\\mathbf\{W\}\}\_\{\\rm DRQ\}\-\\mathbf\{W\}\)u\\right\\rVert\_\{2\}^\{2\}\.

7:Apply the first

kkranked code updates to

𝐖^DRQ\\widehat\{\\mathbf\{W\}\}\_\{\\rm DRQ\}for several values of

kkto generate candidates

𝒫\\mathcal\{P\}\.

8:

ℒDRQ​\(𝐖^k\)←ℒ⁡\(𝐖^k\)\+α​maxu⊤​𝐂​u=1​‖\(𝐖^k−𝐖\)​u‖22,∀𝐖^k∈𝒫\\begin\{aligned\} \\mathcal\{L\}\_\{\\rm DRQ\}\(\\widehat\{\\mathbf\{W\}\}\_\{k\}\)&\\leftarrow\\mathcal\{L\}\(\\widehat\{\\mathbf\{W\}\}\_\{k\}\)\\\\ &\\quad\+\\alpha\\max\_\{u^\{\\top\}\\mathbf\{C\}u=1\}\\left\\lVert\(\\widehat\{\\mathbf\{W\}\}\_\{k\}\-\\mathbf\{W\}\)u\\right\\rVert\_\{2\}^\{2\},\\quad\\forall\\widehat\{\\mathbf\{W\}\}\_\{k\}\\in\\mathcal\{P\}\\end\{aligned\}
9:

𝒫←\{𝐖^k∈𝒫:ℒDRQ​\(𝐖^k\)<ℒDRQ​\(𝐖^DRQ\)\}\\mathcal\{P\}\\leftarrow\\\{\\widehat\{\\mathbf\{W\}\}\_\{k\}\\in\\mathcal\{P\}:\\ \\mathcal\{L\}\_\{\\rm DRQ\}\(\\widehat\{\\mathbf\{W\}\}\_\{k\}\)<\\mathcal\{L\}\_\{\\rm DRQ\}\(\\widehat\{\\mathbf\{W\}\}\_\{\\rm DRQ\}\)\\\}
10:if

𝒫=∅\\mathcal\{P\}=\\varnothingthen break

11:

𝐖^DRQ←arg​min𝐖^k∈𝒫⁡ℒDRQ​\(𝐖^k\)\\widehat\{\\mathbf\{W\}\}\_\{\\rm DRQ\}\\leftarrow\\argmin\_\{\\widehat\{\\mathbf\{W\}\}\_\{k\}\\in\\mathcal\{P\}\}\\mathcal\{L\}\_\{\\rm DRQ\}\(\\widehat\{\\mathbf\{W\}\}\_\{k\}\)
12:endfor

13:return

𝐖^DRQ\\widehat\{\\mathbf\{W\}\}\_\{\\rm DRQ\}

Algorithm[3\.3](https://arxiv.org/html/2610.11226#S3.SS3)summarizes how we optimize[Equation5](https://arxiv.org/html/2610.11226#S3.E5), using numerical estimates of the worst\-case directions and losses\. The set𝒫\\mathcal\{P\}contains proposed weight matrices\. To preserve the fit to calibration data, reconstruction loss may increase only by a preset percentage relative to the initial weights, on both the search subset and separate checking subsets\. All refinement is offline and leaves inference operators unchanged, so DRQ adds no inference overhead\. Appendix[C](https://arxiv.org/html/2610.11226#A3)gives the search procedure and parameter settings, and Appendix[D\.3](https://arxiv.org/html/2610.11226#A4.SS3)covers sparsely activated experts\.

## 4Experimental Evaluation

### 4\.1Experimental setup

#### Models and tasks\.

We apply DRQ to both dense and mixture\-of\-experts \(MoE\) models, including Llama\-3\.2\-Instruct \(1B, 3B\), Llama\-3\.1\-Instruct \(8B, 70B\)\([Grattafiori et al\., 2024](https://arxiv.org/html/2610.11226#bib.bib7)\), and Qwen3\-30B\-A3B\([Yang et al\., 2025](https://arxiv.org/html/2610.11226#bib.bib9)\)\. We evaluate model quality after quantization using two types of metrics: \(1\) perplexity on WikiText\-2 \(WT2\)\([Merity et al\., 2017](https://arxiv.org/html/2610.11226#bib.bib10)\)and C4\([Raffel et al\., 2020](https://arxiv.org/html/2610.11226#bib.bib26)\); \(2\) zero\-shot accuracy on seven downstream tasks, including PIQA\([Bisk et al\., 2020](https://arxiv.org/html/2610.11226#bib.bib16)\), HellaSwag\([Zellers et al\., 2019](https://arxiv.org/html/2610.11226#bib.bib17)\), WinoGrande\([Sakaguchi et al\., 2019](https://arxiv.org/html/2610.11226#bib.bib18)\), ARC\-Easy and ARC\-Challenge\([Clark et al\., 2018](https://arxiv.org/html/2610.11226#bib.bib19)\), BoolQ\([Clark et al\., 2019](https://arxiv.org/html/2610.11226#bib.bib20)\), and MMLU\([Hendrycks et al\., 2021](https://arxiv.org/html/2610.11226#bib.bib21)\)\.

#### Implementation\.

We evaluate DRQ at W4A16, W3A16, and W2A16 to study how well it preserves model quality across different weight bit widths\. All three settings use group\-wise weight quantization with a group size of 128\([Frantar et al\., 2023](https://arxiv.org/html/2610.11226#bib.bib4);[Lin et al\., 2024](https://arxiv.org/html/2610.11226#bib.bib6)\)\. Unless otherwise specified, we calibrate with 128 WikiText\-2 training sequences of 2,048 tokens and select DRQ hyperparameters using 64 validation samples from Pile\([Gao et al\., 2020](https://arxiv.org/html/2610.11226#bib.bib24)\)\. Experiments run on NVIDIA A800 80GB GPUs unless otherwise specified\. Further implementation details are provided in Appendix[C](https://arxiv.org/html/2610.11226#A3)\.

#### Baselines\.

We use GPTQ\([Frantar et al\., 2023](https://arxiv.org/html/2610.11226#bib.bib4)\)and AWQ\([Lin et al\., 2024](https://arxiv.org/html/2610.11226#bib.bib6)\), two widely used weight\-only PTQ methods, as the main baselines for most experiments\. To demonstrate that DRQ generalizes across quantization methods, we also include round\-to\-nearest \(RTN\), OmniQuant\([Shao et al\., 2024](https://arxiv.org/html/2610.11226#bib.bib8)\), GPTAQ\([Li et al\., 2025](https://arxiv.org/html/2610.11226#bib.bib5)\), and ParoQuant\([Liang et al\., 2026](https://arxiv.org/html/2610.11226#bib.bib22)\)\.

Table 1:Perplexity and average task accuracy \(%\) of dense Llama models across model sizes and weight bit widths\. FP denotes the unquantized full\-precision model\.

### 4\.2Dense\-model results across sizes and bit widths

[Table1](https://arxiv.org/html/2610.11226#S4.T1)shows that DRQ improves average task accuracy across both GPTQ and AWQ at every evaluated model size and bit width, while reducing perplexity in most cases\. The gains are particularly pronounced when aggressive quantization causes substantial quality loss\. For Llama\-3\.1\-70B with GPTQ W2, DRQ lowers C4 PPL from 31\.77 to 27\.11 and raises accuracy from 48\.23% to 54\.56%\. Similarly, for Llama\-3\.2\-3B with GPTQ W3, C4 PPL falls from 30\.34 to 22\.11 and accuracy increases from 55\.83% to 60\.24%\. These results show that refinement can recover part of the quality lost at low precision, although severe AWQ\-W2 degradation persists\. Its benefits also extend to stronger initial quantized models\. At W4, where the baselines already retain much of their full\-precision accuracy, DRQ improves accuracy in all eight comparisons\. Appendix[D\.1](https://arxiv.org/html/2610.11226#A4.SS1)provides the detailed results\.

### 4\.3Extension to mixture\-of\-experts models

[Table2](https://arxiv.org/html/2610.11226#S4.T2)extends the evaluation to Qwen3\-30B\-A3B at W3A16\. DRQ lowers perplexity on both WT2 and C4 and improves accuracy on five of the seven downstream tasks\. Together with the dense\-model results, these improvements demonstrate DRQ’s applicability across dense and MoE architectures\. Further details on MoE quantization and expert refinement are provided in Appendix[D\.3](https://arxiv.org/html/2610.11226#A4.SS3)\.

Table 2:Perplexity and seven\-task accuracy \(%\) for Qwen3\-30B\-A3B W3A16\.
### 4\.4Versatility across six PTQ methods

DRQ improves models quantized by all six PTQ methods, reducing both WT2 and C4 perplexity and increasing average accuracy on Llama\-3\.2\-3B W3 \([Figure3](https://arxiv.org/html/2610.11226#S4.F3)\)\. ParoQuant achieves the best results before refinement and retains its lead on all three metrics after applying DRQ, showing that even a strong initial quantized model can benefit from refinement\. Detailed task\-level results and computation costs are provided in Appendices[D\.2](https://arxiv.org/html/2610.11226#A4.SS2)and[E\.1](https://arxiv.org/html/2610.11226#A5.SS1)\.

Figure 3:Six\-method comparison on Llama\-3\.2\-3B W3\.
### 4\.5Calibration size and the choice of objective

We vary calibration size from 16 to 256 sequences of 2,048 tokens from WT2 or C4 for Llama\-3\.2\-3B\-Instruct with GPTQ W4\. Reconstruction\-only refinement \(rec\-only\) uses the same code search as DRQ withα=0\\alpha=0\.

Figure 4:Reconstruction loss and PPL across calibration sizes for Llama\-3\.2\-3B\-Instruct with GPTQ W4\. Loss ratios divide the reconstruction loss summed across layers after refinement by that before refinement, using each run’s own calibration activations\.The first column of[Figure4](https://arxiv.org/html/2610.11226#S4.F4)shows larger relative loss reductions from rec\-only as calibration size grows\. This further reduction is possible because GPTQ does not fully minimize reconstruction loss over the larger calibration sets\. Yet even as rec\-only reduces the loss further relative to GPTQ, model performance worsens, with PPL increasing by orders of magnitude\. DRQ instead keeps reconstruction loss close to GPTQ’s while achieving lower PPL on both datasets\. These results show that more calibration data need not resolve this mismatch, as also established theoretically in[TheoremB\.2](https://arxiv.org/html/2610.11226#A2.Thmtheorem2)\.[Figure7](https://arxiv.org/html/2610.11226#A4.F7)in Appendix[D\.4](https://arxiv.org/html/2610.11226#A4.SS4)examinesα\\alphaandβ\\betaat W3\.

### 4\.6Efficiency of DRQ

[Section4\.5](https://arxiv.org/html/2610.11226#S4.SS5)shows the offline cost of DRQ on Llama\-3\.1\-8B at W3\. We time the loop that quantizes successive Transformer blocks, including forward passes on calibration data\. Across 20 configurations, refinement takes almost the same time with GPTQ and AWQ, averaging 6\.01 and 5\.96 minutes, respectively\. It is because DRQ applies the same code refinement within the existing quantization grid in both cases\. Further efficiency analysis appears in Appendix[E](https://arxiv.org/html/2610.11226#A5)\.

Figure 5:Quantization time forLlama\-3\.1\-8B W3\.

## 5Related Work

#### Post\-training quantization of LLMs\.

PTQ methods improve low\-bit approximation by reducing the effect of outliers and optimizing quantized weights\. Scaling and clipping control channel magnitudes and quantization ranges\([Lin et al\., 2024](https://arxiv.org/html/2610.11226#bib.bib6);[Xiao et al\., 2023](https://arxiv.org/html/2610.11226#bib.bib28);[Shao et al\., 2024](https://arxiv.org/html/2610.11226#bib.bib8)\), while rotations and affine transformations redistribute values across channels\([Ashkboos et al\., 2024](https://arxiv.org/html/2610.11226#bib.bib1);[Liu et al\., 2025](https://arxiv.org/html/2610.11226#bib.bib14);[Liang et al\., 2026](https://arxiv.org/html/2610.11226#bib.bib22);[Sun et al\., 2025](https://arxiv.org/html/2610.11226#bib.bib23);[Zhao et al\., 2026](https://arxiv.org/html/2610.11226#bib.bib15)\)\. QuIP and QTIP combine randomized transformations with adaptive rounding and trellis coding, respectively\([Chee et al\., 2023](https://arxiv.org/html/2610.11226#bib.bib2);[Tseng et al\., 2024](https://arxiv.org/html/2610.11226#bib.bib27)\)\. For weight optimization, GPTQ uses second\-order error compensation during rounding\([Frantar et al\., 2023](https://arxiv.org/html/2610.11226#bib.bib4)\)\. Complementary work changes the reconstruction objective to better preserve model performance\. GPTAQ accounts for errors from preceding quantized layers when matching full\-precision layer outputs, and CoreQ adjusts the strength of this correction across layers\([Li et al\., 2025](https://arxiv.org/html/2610.11226#bib.bib5);[Cha et al\., 2026](https://arxiv.org/html/2610.11226#bib.bib29)\)\. GuidedQuant uses gradients of the model’s final loss to weight reconstruction loss\([Kim et al\., 2025](https://arxiv.org/html/2610.11226#bib.bib13)\), while LFQ matches the model’s token probabilities to those of the full\-precision model when quantizing the final Transformer block\([Lee et al\., 2026](https://arxiv.org/html/2610.11226#bib.bib25)\)\.

#### Distributionally robust optimization\.

Distributionally robust optimization \(DRO\) minimizes the worst expected loss over a set of plausible distributions\. Moment constraints, divergence bounds, and transport distances provide different ways to define this uncertainty set\([Delage and Ye, 2010](https://arxiv.org/html/2610.11226#bib.bib30);[Mohajerin Esfahani and Kuhn, 2018](https://arxiv.org/html/2610.11226#bib.bib31);[Levy et al\., 2020](https://arxiv.org/html/2610.11226#bib.bib37)\)\. Group DRO focuses on shifts among predefined groups, while other approaches learn or constrain how an adversary reweights individual samples\([Sagawa et al\., 2020](https://arxiv.org/html/2610.11226#bib.bib32);[Michel et al\., 2022](https://arxiv.org/html/2610.11226#bib.bib42);[Setlur et al\., 2023](https://arxiv.org/html/2610.11226#bib.bib43)\)\. The choice of set determines which distribution changes the objective protects against\. Allowing too much freedom can lead to overly conservative solutions or amplify the influence of outliers\. Geometry\-aware approaches use data structure to constrain these changes or calibrate robust risks\([Liu et al\., 2022](https://arxiv.org/html/2610.11226#bib.bib35);[Liu et al\., 2024](https://arxiv.org/html/2610.11226#bib.bib36)\), while outlier\-resistant formulations address contamination in the data\([Zhai et al\., 2021](https://arxiv.org/html/2610.11226#bib.bib40)\)\. Connections to regularization further explain how uncertainty sets induce penalties on the learning objective\([Husain, 2020](https://arxiv.org/html/2610.11226#bib.bib33);[Cranko et al\., 2021](https://arxiv.org/html/2610.11226#bib.bib34)\)\. Research on these formulations also establishes population\-risk guarantees for Wasserstein DRO\([Azizian et al\., 2023](https://arxiv.org/html/2610.11226#bib.bib44);[Le and Malick, 2025](https://arxiv.org/html/2610.11226#bib.bib39)\)and develops scalable solvers\([Haddadpour et al\., 2022](https://arxiv.org/html/2610.11226#bib.bib41);[Mehta et al\., 2024](https://arxiv.org/html/2610.11226#bib.bib45);[Zhang et al\., 2025](https://arxiv.org/html/2610.11226#bib.bib38)\)\.

## 6Conclusion

The weights that minimize the reconstruction loss on calibration data need not minimize reconstruction loss on new inputs or yield the best model performance\. We establish this gap between calibration fit and performance at deployment and show why exact optimization and more calibration data need not close it both theoretically and empirically\. Our analysis characterizes how input changes reverse reconstruction rankings and motivates a distributionally robust objective that accounts for those changes\. The proposed method, DRQ, combines reconstruction loss on calibration data with a worst\-case directional error term to refine existing integer codes of quantized weights\. This improves the selection criterion while preserving the quantization grid, scales, and zero points\. Experiments across multiple PTQ methods and dense and mixture\-of\-experts models demonstrate better perplexity and average task accuracy\. By retaining the original quantization format and inference operators, DRQ turns robust reconstruction into better downstream performance without adding extra computational overhead during inference\.

## References

- Ashkbooset al\.\(2024\)S\. Ashkboos, A\. Mohtashami, M\. L\. Croci, B\. Li, P\. Cameron, M\. Jaggi, D\. Alistarh, T\. Hoefler, and J\. HensmanQuaRot: outlier\-free 4\-bit inference in rotated LLMs\.InAdvances in Neural Information Processing Systems,Vol\.37,pp\. 100213–100240\.External Links:[Document](https://dx.doi.org/10.52202/079017-3180),[Link](https://proceedings.neurips.cc/paper_files/paper/2024/hash/b5b939436789f76f08b9d0da5e81af7c-Abstract-Conference.html)Cited by:[§D\.2](https://arxiv.org/html/2610.11226#A4.SS2.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2610.11226#S1.p1.1),[§5](https://arxiv.org/html/2610.11226#S5.SS0.SSS0.Px1.p1.1)\.
- Azizianet al\.\(2023\)W\. Azizian, F\. Iutzeler, and J\. MalickExact generalization guarantees for \(regularized\) Wasserstein distributionally robust models\.InAdvances in Neural Information Processing Systems,Vol\.36,pp\. 14584–14596\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2023/hash/2f060912eacace9ce61ef339205ec54c-Abstract-Conference.html)Cited by:[§5](https://arxiv.org/html/2610.11226#S5.SS0.SSS0.Px2.p1.1)\.
- Bisket al\.\(2020\)Y\. Bisk, R\. Zellers, R\. Le Bras, J\. Gao, and Y\. ChoiPIQA: reasoning about physical commonsense in natural language\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.34,pp\. 7432–7439\.External Links:[Link](https://arxiv.org/abs/1911.11641)Cited by:[§4\.1](https://arxiv.org/html/2610.11226#S4.SS1.SSS0.Px1.p1.1)\.
- Chaet al\.\(2026\)S\. Cha, H\. Chen, D\. Kim, H\. Zhang, K\. Chan, G\. de Veciana, and H\. VikaloCoreQ: learning\-free mismatch correction and successive rounding for quantization\.External Links:2602\.05902,[Document](https://dx.doi.org/10.48550/arXiv.2602.05902),[Link](https://arxiv.org/abs/2602.05902)Cited by:[§5](https://arxiv.org/html/2610.11226#S5.SS0.SSS0.Px1.p1.1)\.
- Cheeet al\.\(2023\)J\. Chee, Y\. Cai, V\. Kuleshov, and C\. De SaQuIP: 2\-bit quantization of large language models with guarantees\.InAdvances in Neural Information Processing Systems,Vol\.36,pp\. 4396–4429\.External Links:2307\.13304,[Link](https://arxiv.org/abs/2307.13304)Cited by:[§5](https://arxiv.org/html/2610.11226#S5.SS0.SSS0.Px1.p1.1)\.
- Chenet al\.\(2021\)M\. Chen, J\. Tworek, H\. Jun, Q\. Yuan, H\. P\. de Oliveira Pinto,et al\.Evaluating large language models trained on code\.External Links:2107\.03374,[Document](https://dx.doi.org/10.48550/arXiv.2107.03374)Cited by:[§1](https://arxiv.org/html/2610.11226#S1.p1.1)\.
- Clarket al\.\(2019\)C\. Clark, K\. Lee, M\. Chang, T\. Kwiatkowski, M\. Collins, and K\. ToutanovaBoolQ: exploring the surprising difficulty of natural yes/no questions\.InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1,pp\. 2924–2936\.External Links:[Document](https://dx.doi.org/10.18653/v1/N19-1300),[Link](https://aclanthology.org/N19-1300/)Cited by:[§4\.1](https://arxiv.org/html/2610.11226#S4.SS1.SSS0.Px1.p1.1)\.
- Clarket al\.\(2018\)P\. Clark, I\. Cowhey, O\. Etzioni, T\. Khot, A\. Sabharwal, C\. Schoenick, and O\. TafjordThink you have solved question answering? try ARC, the AI2 reasoning challenge\.External Links:1803\.05457,[Link](https://arxiv.org/abs/1803.05457)Cited by:[§4\.1](https://arxiv.org/html/2610.11226#S4.SS1.SSS0.Px1.p1.1)\.
- Crankoet al\.\(2021\)Z\. Cranko, Z\. Shi, X\. Zhang, R\. Nock, and S\. KornblithGeneralised Lipschitz regularisation equals distributional robustness\.InProceedings of the 38th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.139,pp\. 2178–2188\.External Links:[Link](https://proceedings.mlr.press/v139/cranko21a.html)Cited by:[§5](https://arxiv.org/html/2610.11226#S5.SS0.SSS0.Px2.p1.1)\.
- DeepSeek\-AI \(2026\)DeepSeek\-AIDeepSeek\-V4\.1\-Flash: pushing the limits of KV cache compression\.External Links:2609\.19969,[Link](https://arxiv.org/abs/2609.19969)Cited by:[§1](https://arxiv.org/html/2610.11226#S1.p1.1)\.
- Delage and Ye \(2010\)E\. Delage and Y\. YeDistributionally robust optimization under moment uncertainty with application to data\-driven problems\.Operations Research58\(3\),pp\. 595–612\.External Links:[Document](https://dx.doi.org/10.1287/opre.1090.0741),[Link](https://pubsonline.informs.org/doi/10.1287/opre.1090.0741)Cited by:[§5](https://arxiv.org/html/2610.11226#S5.SS0.SSS0.Px2.p1.1)\.
- Dettmers and Zettlemoyer \(2023\)T\. Dettmers and L\. ZettlemoyerThe case for 4\-bit precision: k\-bit inference scaling laws\.InProceedings of the 40th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.202,pp\. 7750–7774\.External Links:[Link](https://proceedings.mlr.press/v202/dettmers23a.html)Cited by:[§2\.1](https://arxiv.org/html/2610.11226#S2.SS1.p2.1)\.
- Frantaret al\.\(2023\)E\. Frantar, S\. Ashkboos, T\. Hoefler, and D\. AlistarhGPTQ: accurate post\-training quantization for generative pre\-trained transformers\.External Links:2210\.17323,[Document](https://dx.doi.org/10.48550/arXiv.2210.17323),[Link](https://arxiv.org/abs/2210.17323)Cited by:[§D\.2](https://arxiv.org/html/2610.11226#A4.SS2.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2610.11226#S1.p1.1),[§1](https://arxiv.org/html/2610.11226#S1.p2.1),[§1](https://arxiv.org/html/2610.11226#S1.p2.2),[§2\.2](https://arxiv.org/html/2610.11226#S2.SS2.p1.2),[§4\.1](https://arxiv.org/html/2610.11226#S4.SS1.SSS0.Px2.p1.1),[§4\.1](https://arxiv.org/html/2610.11226#S4.SS1.SSS0.Px3.p1.1),[§5](https://arxiv.org/html/2610.11226#S5.SS0.SSS0.Px1.p1.1)\.
- Gaoet al\.\(2020\)L\. Gao, S\. Biderman, S\. Black, L\. Golding, T\. Hoppe, C\. Foster, J\. Phang, H\. He, A\. Thite, N\. Nabeshima, S\. Presser, and C\. LeahyThe pile: an 800gb dataset of diverse text for language modeling\.External Links:2101\.00027,[Document](https://dx.doi.org/10.48550/arXiv.2101.00027),[Link](https://arxiv.org/abs/2101.00027)Cited by:[Appendix C](https://arxiv.org/html/2610.11226#A3.SS0.SSS0.Px5.p1.1),[§4\.1](https://arxiv.org/html/2610.11226#S4.SS1.SSS0.Px2.p1.1)\.
- Grattafioriet al\.\(2024\)A\. Grattafiori, A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian,et al\.The Llama 3 herd of models\.External Links:2407\.21783,[Document](https://dx.doi.org/10.48550/arXiv.2407.21783),[Link](https://arxiv.org/abs/2407.21783)Cited by:[§1](https://arxiv.org/html/2610.11226#S1.p1.1),[§4\.1](https://arxiv.org/html/2610.11226#S4.SS1.SSS0.Px1.p1.1)\.
- Haddadpouret al\.\(2022\)F\. Haddadpour, M\. M\. Kamani, M\. Mahdavi, and A\. KarbasiLearning distributionally robust models at scale via composite optimization\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=To-R742x7se)Cited by:[§5](https://arxiv.org/html/2610.11226#S5.SS0.SSS0.Px2.p1.1)\.
- Hendryckset al\.\(2021\)D\. Hendrycks, C\. Burns, S\. Basart, A\. Zou, M\. Mazeika, D\. Song, and J\. SteinhardtMeasuring massive multitask language understanding\.InInternational Conference on Learning Representations,External Links:[Link](https://arxiv.org/abs/2009.03300)Cited by:[§4\.1](https://arxiv.org/html/2610.11226#S4.SS1.SSS0.Px1.p1.1)\.
- Husain \(2020\)H\. HusainDistributional robustness with IPMs and links to regularization and GANs\.InAdvances in Neural Information Processing Systems,Vol\.33,pp\. 11816–11827\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2020/hash/8929c70f8d710e412d38da624b21c3c8-Abstract.html)Cited by:[§5](https://arxiv.org/html/2610.11226#S5.SS0.SSS0.Px2.p1.1)\.
- Kimet al\.\(2025\)J\. Kim, M\. El Halabi, W\. Park, C\. J\. S\. Schaefer, D\. Lee, Y\. Park, J\. W\. Lee, and H\. O\. SongGuidedQuant: large language model quantization via exploiting end loss guidance\.InProceedings of the 42nd International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.267,pp\. 30011–30037\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2505.07004),2505\.07004,[Link](https://proceedings.mlr.press/v267/kim25d.html)Cited by:[§5](https://arxiv.org/html/2610.11226#S5.SS0.SSS0.Px1.p1.1)\.
- Le and Malick \(2025\)T\. Le and J\. MalickUniversal generalization guarantees for Wasserstein distributionally robust models\.InInternational Conference on Learning Representations,External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2025/hash/35b5c175e139bff5f22a5361270fce87-Abstract-Conference.html)Cited by:[§5](https://arxiv.org/html/2610.11226#S5.SS0.SSS0.Px2.p1.1)\.
- Leeet al\.\(2026\)J\. H\. Lee, J\. Y\. Yang, J\. Choi, and E\. YangLFQ: logit\-aware final\-block quantization for boosting the generation quality of low\-bit quantized LLMs\.InProceedings of the 43rd International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.306\.External Links:[Link](https://openreview.net/forum?id=ykdc70h1ND)Cited by:[§5](https://arxiv.org/html/2610.11226#S5.SS0.SSS0.Px1.p1.1)\.
- Levyet al\.\(2020\)D\. Levy, Y\. Carmon, J\. C\. Duchi, and A\. SidfordLarge\-scale methods for distributionally robust optimization\.InAdvances in Neural Information Processing Systems,Vol\.33,pp\. 8847–8860\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2020/hash/64986d86a17424eeac96b08a6d519059-Abstract.html)Cited by:[§5](https://arxiv.org/html/2610.11226#S5.SS0.SSS0.Px2.p1.1)\.
- Liet al\.\(2025\)Y\. Li, R\. Yin, D\. Lee, S\. Xiao, and P\. PandaGPTAQ: efficient finetuning\-free quantization for asymmetric calibration\.InProceedings of the 42nd International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.267,pp\. 36690–36706\.External Links:[Link](https://proceedings.mlr.press/v267/li25dn.html)Cited by:[§1](https://arxiv.org/html/2610.11226#S1.p2.1),[§4\.1](https://arxiv.org/html/2610.11226#S4.SS1.SSS0.Px3.p1.1),[§5](https://arxiv.org/html/2610.11226#S5.SS0.SSS0.Px1.p1.1)\.
- Lianget al\.\(2026\)Y\. Liang, H\. Chen, Z\. Zhang, S\. Han, and Z\. LiuParoQuant: pairwise rotation quantization for efficient reasoning LLM inference\.InInternational Conference on Learning Representations,External Links:[Link](https://github.com/z-lab/paroquant)Cited by:[§1](https://arxiv.org/html/2610.11226#S1.p2.1),[§1](https://arxiv.org/html/2610.11226#S1.p2.2),[§2\.1](https://arxiv.org/html/2610.11226#S2.SS1.p2.1),[§4\.1](https://arxiv.org/html/2610.11226#S4.SS1.SSS0.Px3.p1.1),[§5](https://arxiv.org/html/2610.11226#S5.SS0.SSS0.Px1.p1.1)\.
- Linet al\.\(2024\)J\. Lin, J\. Tang, H\. Tang, S\. Yang, W\. Chen, W\. Wang, G\. Xiao, X\. Dang, C\. Gan, and S\. HanAWQ: activation\-aware weight quantization for on\-device LLM compression and acceleration\.InProceedings of Machine Learning and Systems,Vol\.6,pp\. 87–100\.External Links:[Link](https://proceedings.mlsys.org/paper_files/paper/2024/file/42a452cbafa9dd64e9ba4aa95cc1ef21-Paper-Conference.pdf)Cited by:[§1](https://arxiv.org/html/2610.11226#S1.p1.1),[§1](https://arxiv.org/html/2610.11226#S1.p2.1),[§1](https://arxiv.org/html/2610.11226#S1.p2.2),[§2\.2](https://arxiv.org/html/2610.11226#S2.SS2.p1.2),[§4\.1](https://arxiv.org/html/2610.11226#S4.SS1.SSS0.Px2.p1.1),[§4\.1](https://arxiv.org/html/2610.11226#S4.SS1.SSS0.Px3.p1.1),[§5](https://arxiv.org/html/2610.11226#S5.SS0.SSS0.Px1.p1.1)\.
- Liuet al\.\(2022\)J\. Liu, J\. Wu, B\. Li, and P\. CuiDistributionally robust optimization with data geometry\.InAdvances in Neural Information Processing Systems,Vol\.35,pp\. 33689–33701\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2022/hash/da535999561b932f56efdd559498282e-Abstract-Conference.html)Cited by:[§5](https://arxiv.org/html/2610.11226#S5.SS0.SSS0.Px2.p1.1)\.
- Liuet al\.\(2024\)J\. Liu, J\. Wu, T\. Wang, H\. Zou, B\. Li, and P\. CuiGeometry\-calibrated DRO: combating over\-pessimism with free energy implications\.InProceedings of the 41st International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.235,pp\. 32184–32200\.External Links:[Link](https://proceedings.mlr.press/v235/liu24br.html)Cited by:[§5](https://arxiv.org/html/2610.11226#S5.SS0.SSS0.Px2.p1.1)\.
- Liuet al\.\(2025\)Z\. Liu, C\. Zhao, I\. Fedorov, B\. Soran, D\. Choudhary, R\. Krishnamoorthi, V\. Chandra, Y\. Tian, and T\. BlankevoortSpinQuant: LLM quantization with learned rotations\.InThe Thirteenth International Conference on Learning Representations,External Links:[Document](https://dx.doi.org/10.48550/arXiv.2405.16406),2405\.16406,[Link](https://openreview.net/forum?id=ogO6DGE6FZ)Cited by:[§5](https://arxiv.org/html/2610.11226#S5.SS0.SSS0.Px1.p1.1)\.
- Mehtaet al\.\(2024\)R\. Mehta, J\. Diakonikolas, and Z\. HarchaouiDrago: primal\-dual coupled variance reduction for faster distributionally robust optimization\.InAdvances in Neural Information Processing Systems,Vol\.37,pp\. 134770–134825\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2024/hash/f2c5a0d8e445cc359eb446521f438ee3-Abstract-Conference.html)Cited by:[§5](https://arxiv.org/html/2610.11226#S5.SS0.SSS0.Px2.p1.1)\.
- Merityet al\.\(2017\)S\. Merity, C\. Xiong, J\. Bradbury, and R\. SocherPointer sentinel mixture models\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=Byj72udxe)Cited by:[§1](https://arxiv.org/html/2610.11226#S1.p3.1),[§4\.1](https://arxiv.org/html/2610.11226#S4.SS1.SSS0.Px1.p1.1)\.
- Michelet al\.\(2022\)P\. Michel, T\. Hashimoto, and G\. NeubigDistributionally robust models with parametric likelihood ratios\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=a34GrNaYEcS)Cited by:[§5](https://arxiv.org/html/2610.11226#S5.SS0.SSS0.Px2.p1.1)\.
- Mohajerin Esfahani and Kuhn \(2018\)P\. Mohajerin Esfahani and D\. KuhnData\-driven distributionally robust optimization using the Wasserstein metric: performance guarantees and tractable reformulations\.Mathematical Programming171\(1–2\),pp\. 115–166\.External Links:[Document](https://dx.doi.org/10.1007/s10107-017-1172-1),[Link](https://link.springer.com/article/10.1007/s10107-017-1172-1)Cited by:[§5](https://arxiv.org/html/2610.11226#S5.SS0.SSS0.Px2.p1.1)\.
- OpenAIet al\.\(2023\)OpenAI, J\. Achiam, S\. Adler, S\. Agarwal, L\. Ahmad,et al\.GPT\-4 technical report\.External Links:2303\.08774,[Document](https://dx.doi.org/10.48550/arXiv.2303.08774)Cited by:[§1](https://arxiv.org/html/2610.11226#S1.p1.1)\.
- Raffelet al\.\(2020\)C\. Raffel, N\. Shazeer, A\. Roberts, K\. Lee, S\. Narang, M\. Matena, Y\. Zhou, W\. Li, and P\. J\. LiuExploring the limits of transfer learning with a unified text\-to\-text transformer\.Journal of Machine Learning Research21\(140\),pp\. 1–67\.External Links:[Link](https://jmlr.org/papers/v21/20-074.html)Cited by:[§4\.1](https://arxiv.org/html/2610.11226#S4.SS1.SSS0.Px1.p1.1)\.
- Sagawaet al\.\(2020\)S\. Sagawa, P\. W\. Koh, T\. B\. Hashimoto, and P\. LiangDistributionally robust neural networks for group shifts: on the importance of regularization for worst\-case generalization\.InInternational Conference on Learning Representations,External Links:[Link](https://arxiv.org/abs/1911.08731)Cited by:[§5](https://arxiv.org/html/2610.11226#S5.SS0.SSS0.Px2.p1.1)\.
- Sakaguchiet al\.\(2019\)K\. Sakaguchi, R\. Le Bras, C\. Bhagavatula, and Y\. ChoiWinoGrande: an adversarial winograd schema challenge at scale\.External Links:1907\.10641,[Link](https://arxiv.org/abs/1907.10641)Cited by:[§4\.1](https://arxiv.org/html/2610.11226#S4.SS1.SSS0.Px1.p1.1)\.
- Setluret al\.\(2023\)A\. Setlur, D\. K\. Dennis, B\. Eysenbach, A\. Raghunathan, C\. Finn, V\. Smith, and S\. LevineBitrate\-constrained DRO: beyond worst case robustness to unknown group shifts\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=2QzNuaRHn4Z)Cited by:[§5](https://arxiv.org/html/2610.11226#S5.SS0.SSS0.Px2.p1.1)\.
- Shaoet al\.\(2024\)W\. Shao, M\. Chen, Z\. Zhang, P\. Xu, L\. Zhao, Z\. Li, K\. Zhang, P\. Gao, Y\. Qiao, and P\. LuoOmniQuant: omnidirectionally calibrated quantization for large language models\.InThe Twelfth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=8Wuvhh0LYW)Cited by:[§1](https://arxiv.org/html/2610.11226#S1.p2.1),[§2\.2](https://arxiv.org/html/2610.11226#S2.SS2.p1.2),[§4\.1](https://arxiv.org/html/2610.11226#S4.SS1.SSS0.Px3.p1.1),[§5](https://arxiv.org/html/2610.11226#S5.SS0.SSS0.Px1.p1.1)\.
- Sunet al\.\(2025\)Y\. Sun, R\. Liu, H\. Bai, H\. Bao, K\. Zhao, Y\. Li, J\. Hu, X\. Yu, L\. Hou, C\. Yuan, X\. Jiang, W\. Liu, and J\. YaoFlatQuant: flatness matters for LLM quantization\.InProceedings of the 42nd International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.267,pp\. 57587–57613\.External Links:[Link](https://proceedings.mlr.press/v267/sun25l.html)Cited by:[§1](https://arxiv.org/html/2610.11226#S1.p2.1),[§5](https://arxiv.org/html/2610.11226#S5.SS0.SSS0.Px1.p1.1)\.
- Tsenget al\.\(2024\)A\. Tseng, Q\. Sun, D\. Hou, and C\. De SaQTIP: quantization with trellises and incoherence processing\.InAdvances in Neural Information Processing Systems,Vol\.37\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2024/hash/6de2e84b8da47bb2eb5e2ac96c63d2b0-Abstract-Conference.html)Cited by:[§1](https://arxiv.org/html/2610.11226#S1.p2.1),[§5](https://arxiv.org/html/2610.11226#S5.SS0.SSS0.Px1.p1.1)\.
- Xiaoet al\.\(2023\)G\. Xiao, J\. Lin, M\. Seznec, H\. Wu, J\. Demouth, and S\. HanSmoothQuant: accurate and efficient post\-training quantization for large language models\.InProceedings of the 40th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.202,pp\. 38087–38099\.External Links:[Link](https://proceedings.mlr.press/v202/xiao23c.html)Cited by:[§5](https://arxiv.org/html/2610.11226#S5.SS0.SSS0.Px1.p1.1)\.
- Yanget al\.\(2025\)A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui,et al\.Qwen3 technical report\.External Links:2505\.09388,[Document](https://dx.doi.org/10.48550/arXiv.2505.09388),[Link](https://arxiv.org/abs/2505.09388)Cited by:[§4\.1](https://arxiv.org/html/2610.11226#S4.SS1.SSS0.Px1.p1.1)\.
- Zellerset al\.\(2019\)R\. Zellers, A\. Holtzman, Y\. Bisk, A\. Farhadi, and Y\. ChoiHellaSwag: can a machine really finish your sentence?\.InProceedings of the 57th Annual Meeting of the Association for Computational Linguistics,pp\. 4791–4800\.External Links:[Document](https://dx.doi.org/10.18653/v1/P19-1472),[Link](https://aclanthology.org/P19-1472/)Cited by:[§4\.1](https://arxiv.org/html/2610.11226#S4.SS1.SSS0.Px1.p1.1)\.
- Zhaiet al\.\(2021\)R\. Zhai, C\. Dan, Z\. Kolter, and P\. RavikumarDORO: distributional and outlier robust optimization\.InProceedings of the 38th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.139,pp\. 12345–12355\.External Links:[Link](https://proceedings.mlr.press/v139/zhai21a.html)Cited by:[§5](https://arxiv.org/html/2610.11226#S5.SS0.SSS0.Px2.p1.1)\.
- Zhanget al\.\(2025\)Q\. Zhang, Y\. Zhou, S\. Khan, A\. Prater\-Bennette, L\. Shen, and S\. ZouRevisiting large\-scale non\-convex distributionally robust optimization\.InInternational Conference on Learning Representations,External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2025/hash/bdbb5a9dd174d3f8230b33269a9ccf56-Abstract-Conference.html)Cited by:[§5](https://arxiv.org/html/2610.11226#S5.SS0.SSS0.Px2.p1.1)\.
- Zhaoet al\.\(2026\)Y\. Zhao, X\. Cheng, H\. Liu, B\. He, X\. Zhang, H\. B\. H\. Zhu, W\. Chen, L\. Zeng, and Z\. SunSaliency\-aware regularized quantization calibration for large language models\.arXiv preprint arXiv:2605\.05693\.Cited by:[§5](https://arxiv.org/html/2610.11226#S5.SS0.SSS0.Px1.p1.1)\.

## Appendix ANotation

Table 3:Summary of notation\.
## Appendix BTheoretical Results and Proofs

This appendix supports the analysis and objective in[Section3](https://arxiv.org/html/2610.11226#S3)\. We first explain how a fixed quantization grid can make reconstruction loss on calibration data favor weights with higher loss on new inputs, and why more calibration data need not resolve this difference\. We then prove the results for changes in the input activation distribution and explain the guarantees of the DRQ objective\. Throughout, the full\-precision weights and quantization grid remain fixed\. The matrix𝐇\\mathbf\{H\}is the second\-moment matrix of the calibration input activations, computed from samples or from their distribution as specified below\.

### B\.1The role of the quantization grid

Without a quantization constraint, retaining the full\-precision weights gives zero reconstruction loss for any input distribution\. Quantization usually removes this option\. The following result makes this distinction explicit\.

###### Proposition B\.1\(Reconstruction when full\-precision weights are representable\)\.

For every activation second\-moment matrix𝐇⪰0\\mathbf\{H\}\\succeq 0, the unconstrained problemmin𝐖^∈ℝm×d⁡‖\(𝐖^−𝐖\)​𝐇1/2‖F2\\min\_\{\\widehat\{\\mathbf\{W\}\}\\in\\mathbb\{R\}^\{m\\times d\}\}\\left\\lVert\(\\widehat\{\\mathbf\{W\}\}\-\\mathbf\{W\}\)\\mathbf\{H\}^\{1/2\}\\right\\rVert\_\{\\rm F\}^\{2\}has𝐖\\mathbf\{W\}as a global minimizer\. This minimizer is unique when𝐇≻0\\mathbf\{H\}\\succ 0\. If𝐖∈𝒬\\mathbf\{W\}\\in\\mathcal\{Q\}, it also minimizes reconstruction loss over the quantization grid for every input distribution with a finite second moment\. Thus two positive\-definite second\-moment matrices can select different unique minimizers only if the grid excludes𝐖\\mathbf\{W\}\.

###### Proof of[PropositionB\.1](https://arxiv.org/html/2610.11226#A2.Thmtheorem1)\.

For any𝐖^\\widehat\{\\mathbf\{W\}\}and𝐇⪰0\\mathbf\{H\}\\succeq 0,‖\(𝐖^−𝐖\)​𝐇1/2‖F2≥0\\left\\lVert\(\\widehat\{\\mathbf\{W\}\}\-\\mathbf\{W\}\)\\mathbf\{H\}^\{1/2\}\\right\\rVert\_\{\\rm F\}^\{2\}\\geq 0\. Setting𝐖^=𝐖\\widehat\{\\mathbf\{W\}\}=\\mathbf\{W\}gives zero loss\. If𝐇≻0\\mathbf\{H\}\\succ 0, equality requires𝐖^=𝐖\\widehat\{\\mathbf\{W\}\}=\\mathbf\{W\}, so the minimizer is unique\. The same argument applies when optimization is restricted to a grid containing𝐖\\mathbf\{W\}\. ∎

Excluding𝐖\\mathbf\{W\}makes different choices possible, but does not force them\. The fixed\-grid example in[TheoremB\.3](https://arxiv.org/html/2610.11226#A2.Thmtheorem3)shows that the weights minimizing reconstruction loss on calibration data can have larger loss on a new input\.

### B\.2Why more calibration data may not resolve the mismatch

More calibration data can estimate reconstruction loss more accurately\. This does not necessarily improve the selected weights if minimizing the expected reconstruction loss already favors weights with a higher evaluation loss\. The following result separates this mismatch between objectives from error due to a limited calibration sample\.

Let𝒥⁡\(𝐖^\)\\mathcal\{J\}\(\\widehat\{\\mathbf\{W\}\}\)be a fixed, finite evaluation loss, with smaller values indicating better performance\. It may be reconstruction loss on new input activations or the model’s token negative log\-likelihood \(NLL\) after replacing one layer’s weights while keeping all other weights fixed\. The evaluation rule and distribution of evaluation data remain unchanged as the number of calibration samples increases\.

###### Theorem B\.2\(More calibration data need not improve the selected weights\)\.

Let𝒬\\mathcal\{Q\}be a fixed finite quantization grid, and letx1,x2,…x\_\{1\},x\_\{2\},\\ldotsbe independent samples from the same calibration input activation distribution with𝔼​‖x‖22<∞\\mathbb\{E\}\\left\\lVert x\\right\\rVert\_\{2\}^\{2\}<\\infty\. Suppose𝐖^∈𝒬\\widehat\{\\mathbf\{W\}\}\\in\\mathcal\{Q\}uniquely minimizes𝔼​‖\(𝐖^−𝐖\)​x‖22\\mathbb\{E\}\\left\\lVert\(\\widehat\{\\mathbf\{W\}\}\-\\mathbf\{W\}\)x\\right\\rVert\_\{2\}^\{2\}over this grid, while another matrix𝐖^′∈𝒬\\widehat\{\\mathbf\{W\}\}^\{\\prime\}\\in\\mathcal\{Q\}satisfies𝒥⁡\(𝐖^′\)<𝒥⁡\(𝐖^\)\\mathcal\{J\}\(\\widehat\{\\mathbf\{W\}\}^\{\\prime\}\)<\\mathcal\{J\}\(\\widehat\{\\mathbf\{W\}\}\)\. With probability one, every exact minimizer of reconstruction loss on the firstnnsamples equals𝐖^\\widehat\{\\mathbf\{W\}\}for all sufficiently largenn\. Its evaluation loss therefore remains worse than that of𝐖^′\\widehat\{\\mathbf\{W\}\}^\{\\prime\}by the fixed positive amount𝒥⁡\(𝐖^\)−𝒥⁡\(𝐖^′\)\\mathcal\{J\}\(\\widehat\{\\mathbf\{W\}\}\)\-\\mathcal\{J\}\(\\widehat\{\\mathbf\{W\}\}^\{\\prime\}\)\.

###### Proof of[TheoremB\.2](https://arxiv.org/html/2610.11226#A2.Thmtheorem2)\.

For each quantized weight matrix𝐕∈𝒬\\mathbf\{V\}\\in\\mathcal\{Q\}, write the reconstruction loss on the firstnncalibration inputs as

ℒ^n​\(𝐕\)=1n​∑k=1n‖\(𝐕−𝐖\)​xk‖22\.\\widehat\{\\mathcal\{L\}\}\_\{n\}\(\\mathbf\{V\}\)=\\frac\{1\}\{n\}\\sum\_\{k=1\}^\{n\}\\left\\lVert\(\\mathbf\{V\}\-\\mathbf\{W\}\)x\_\{k\}\\right\\rVert\_\{2\}^\{2\}\.\(7\)The finite\-second\-moment assumption ensures that this squared loss has a finite expectation\. The strong law of large numbers givesℒ^n​\(𝐕\)→𝔼​‖\(𝐕−𝐖\)​x‖22\\widehat\{\\mathcal\{L\}\}\_\{n\}\(\\mathbf\{V\}\)\\to\\mathbb\{E\}\\left\\lVert\(\\mathbf\{V\}\-\\mathbf\{W\}\)x\\right\\rVert\_\{2\}^\{2\}almost surely\. Because the grid is finite, the largest absolute difference between empirical and expected loss over all matrices in𝒬\\mathcal\{Q\}also converges to zero\.

The unique minimizer𝐖^\\widehat\{\\mathbf\{W\}\}of expected reconstruction loss is separated from every other matrix by a positive gap

γ=min𝐕∈𝒬,𝐕≠𝐖^⁡\{𝔼​‖\(𝐕−𝐖\)​x‖22−𝔼​‖\(𝐖^−𝐖\)​x‖22\}\>0\.\\gamma=\\min\_\{\\mathbf\{V\}\\in\\mathcal\{Q\},\\,\\mathbf\{V\}\\neq\\widehat\{\\mathbf\{W\}\}\}\\left\\\{\\mathbb\{E\}\\left\\lVert\(\\mathbf\{V\}\-\\mathbf\{W\}\)x\\right\\rVert\_\{2\}^\{2\}\-\\mathbb\{E\}\\left\\lVert\(\\widehat\{\\mathbf\{W\}\}\-\\mathbf\{W\}\)x\\right\\rVert\_\{2\}^\{2\}\\right\\\}\>0\.\(8\)With probability one, all empirical losses eventually lie withinγ/3\\gamma/3of their expectations\. Every other matrix then has empirical loss at leastγ/3\\gamma/3larger than that of𝐖^\\widehat\{\\mathbf\{W\}\}\. The exact empirical minimizer therefore equals𝐖^\\widehat\{\\mathbf\{W\}\}for all sufficiently largenn\.

The selected weights then retain the evaluation\-loss gap

𝒥⁡\(𝐖^\)−𝒥⁡\(𝐖^′\)\>0,\\mathcal\{J\}\(\\widehat\{\\mathbf\{W\}\}\)\-\\mathcal\{J\}\(\\widehat\{\\mathbf\{W\}\}^\{\\prime\}\)\>0,\(9\)which proves the claim\. ∎

The theorem assumes that the expected reconstruction loss and evaluation loss favor different weights\. It shows that more samples from the same distribution cannot remove this existing mismatch simply by estimating reconstruction loss more accurately\. It does not require evaluation on a different dataset, since𝒥\\mathcal\{J\}can also measure model performance on the calibration distribution\.[TheoremB\.3](https://arxiv.org/html/2610.11226#A2.Thmtheorem3)gives a concrete instance with reconstruction loss on a new input as𝒥\\mathcal\{J\}\.

### B\.3When input changes reverse the reconstruction loss comparison

We now compare two quantized weight matrices withℒ⁡\(𝐖^1\)<ℒ⁡\(𝐖^2\)\\mathcal\{L\}\(\\widehat\{\\mathbf\{W\}\}\_\{1\}\)<\\mathcal\{L\}\(\\widehat\{\\mathbf\{W\}\}\_\{2\}\)\. As in[Section3\.1](https://arxiv.org/html/2610.11226#S3.SS1), new input activations have second\-moment matrix𝐇\+𝚫\\mathbf\{H\}\+\\boldsymbol\{\\Delta\}, where𝚫⪰0\\boldsymbol\{\\Delta\}\\succeq 0andtr⁡\(𝐂​𝚫\)≤α\\operatorname\{tr\}\(\\mathbf\{C\}\\boldsymbol\{\\Delta\}\)\\leq\\alpha, with fixed𝐂≻0\\mathbf\{C\}\\succ 0\. The proof of[Theorem3\.1](https://arxiv.org/html/2610.11226#S3.Thmtheorem1)finds the largest possible increase in the loss of𝐖^1\\widehat\{\\mathbf\{W\}\}\_\{1\}relative to that of𝐖^2\\widehat\{\\mathbf\{W\}\}\_\{2\}under these constraints\.

For the proof, we use two auxiliary quantities to express the difference between the reconstruction losses of𝐖^1\\widehat\{\\mathbf\{W\}\}\_\{1\}and𝐖^2\\widehat\{\\mathbf\{W\}\}\_\{2\},

𝐁12\\displaystyle\\mathbf\{B\}\_\{12\}=\(𝐖^1−𝐖\)⊤​\(𝐖^1−𝐖\)−\(𝐖^2−𝐖\)⊤​\(𝐖^2−𝐖\),\\displaystyle=\(\\widehat\{\\mathbf\{W\}\}\_\{1\}\-\\mathbf\{W\}\)^\{\\top\}\(\\widehat\{\\mathbf\{W\}\}\_\{1\}\-\\mathbf\{W\}\)\-\(\\widehat\{\\mathbf\{W\}\}\_\{2\}\-\\mathbf\{W\}\)^\{\\top\}\(\\widehat\{\\mathbf\{W\}\}\_\{2\}\-\\mathbf\{W\}\),\(10\)e12​\(𝐂\)\\displaystyle e\_\{12\}\(\\mathbf\{C\}\)=\[λmax\(𝐂−1/2𝐁12𝐂−1/2\)\]\+\.\\displaystyle=\\left\[\\lambda\_\{\\max\}\\\!\\left\(\\mathbf\{C\}^\{\-1/2\}\\mathbf\{B\}\_\{12\}\\mathbf\{C\}^\{\-1/2\}\\right\)\\right\]\_\{\+\}\.For an input vectoruu, each squared output error measures the difference between a quantized layer’s output and the full\-precision output𝐖​u\\mathbf\{W\}u\. The quantityu⊤​𝐁12​uu^\{\\top\}\\mathbf\{B\}\_\{12\}usubtracts this error for𝐖^2\\widehat\{\\mathbf\{W\}\}\_\{2\}from that for𝐖^1\\widehat\{\\mathbf\{W\}\}\_\{1\}\. The shorthande12​\(𝐂\)e\_\{12\}\(\\mathbf\{C\}\)gives the largest increase in their reconstruction\-loss difference under a distribution\-change budget of one\. Here\[a\]\+=max⁡\{a,0\}\[a\]\_\{\+\}=\\max\\\{a,0\\\}, since𝚫=0\\boldsymbol\{\\Delta\}=0satisfies the constraints and leaves the difference unchanged\. We will show that the loss of𝐖^1\\widehat\{\\mathbf\{W\}\}\_\{1\}remains no larger than that of𝐖^2\\widehat\{\\mathbf\{W\}\}\_\{2\}exactly whenℒ⁡\(𝐖^2\)−ℒ⁡\(𝐖^1\)≥α​e12​\(𝐂\)\\mathcal\{L\}\(\\widehat\{\\mathbf\{W\}\}\_\{2\}\)\-\\mathcal\{L\}\(\\widehat\{\\mathbf\{W\}\}\_\{1\}\)\\geq\\alpha e\_\{12\}\(\\mathbf\{C\}\)\.

###### Proof of[Theorem3\.1](https://arxiv.org/html/2610.11226#S3.Thmtheorem1)\.

Under the second moment𝐇\+𝚫\\mathbf\{H\}\+\\boldsymbol\{\\Delta\}, the difference between the two weight matrices’ reconstruction losses is

tr⁡\[\(𝐖^1−𝐖\)​\(𝐇\+𝚫\)​\(𝐖^1−𝐖\)⊤\]\\displaystyle\\operatorname\{tr\}\\\!\\left\[\(\\widehat\{\\mathbf\{W\}\}\_\{1\}\-\\mathbf\{W\}\)\(\\mathbf\{H\}\+\\boldsymbol\{\\Delta\}\)\(\\widehat\{\\mathbf\{W\}\}\_\{1\}\-\\mathbf\{W\}\)^\{\\top\}\\right\]\(11\)−tr⁡\[\(𝐖^2−𝐖\)​\(𝐇\+𝚫\)​\(𝐖^2−𝐖\)⊤\]\\displaystyle\-\\operatorname\{tr\}\\\!\\left\[\(\\widehat\{\\mathbf\{W\}\}\_\{2\}\-\\mathbf\{W\}\)\(\\mathbf\{H\}\+\\boldsymbol\{\\Delta\}\)\(\\widehat\{\\mathbf\{W\}\}\_\{2\}\-\\mathbf\{W\}\)^\{\\top\}\\right\]=ℒ⁡\(𝐖^1\)−ℒ⁡\(𝐖^2\)\+tr⁡\(𝐁12​𝚫\),\\displaystyle=\{\\mathcal\{L\}\(\\widehat\{\\mathbf\{W\}\}\_\{1\}\)\-\\mathcal\{L\}\(\\widehat\{\\mathbf\{W\}\}\_\{2\}\)\}\+\\operatorname\{tr\}\(\\mathbf\{B\}\_\{12\}\\boldsymbol\{\\Delta\}\),The first term is the loss difference on calibration data\. The second is the change in that difference caused by𝚫\\boldsymbol\{\\Delta\}\.

Set𝐄=𝐂1/2​𝚫​𝐂1/2\\mathbf\{E\}=\\mathbf\{C\}^\{1/2\}\\boldsymbol\{\\Delta\}\\mathbf\{C\}^\{1/2\}and𝐃=𝐂−1/2𝐁12𝐂−1/2\\mathbf\{D\}=\\mathbf\{C\}^\{\-1/2\}\\mathbf\{B\}\_\{12\}\\mathbf\{C\}^\{\-1/2\}\. The constraints on𝚫\\boldsymbol\{\\Delta\}are equivalent to𝐄⪰0\\mathbf\{E\}\\succeq 0andtr⁡\(𝐄\)≤α\\operatorname\{tr\}\(\\mathbf\{E\}\)\\leq\\alpha\. Hence

max𝚫⪰0,tr⁡\(𝐂​𝚫\)≤α⁡tr⁡\(𝐁12​𝚫\)=max𝐄⪰0,tr⁡\(𝐄\)≤α⁡tr⁡\(𝐃𝐄\)=α​\[λmax​\(𝐃\)\]\+\.\\begin\{split\}\\max\_\{\\boldsymbol\{\\Delta\}\\succeq 0,\\,\\operatorname\{tr\}\(\\mathbf\{C\}\\boldsymbol\{\\Delta\}\)\\leq\\alpha\}\\operatorname\{tr\}\(\\mathbf\{B\}\_\{12\}\\boldsymbol\{\\Delta\}\)&=\\max\_\{\\mathbf\{E\}\\succeq 0,\\,\\operatorname\{tr\}\(\\mathbf\{E\}\)\\leq\\alpha\}\\operatorname\{tr\}\(\\mathbf\{D\}\\mathbf\{E\}\)\\\\ &=\\alpha\[\\lambda\_\{\\max\}\(\\mathbf\{D\}\)\]\_\{\+\}\.\\end\{split\}\(12\)The last equality follows by setting𝐄=α​v​v⊤\\mathbf\{E\}=\\alpha vv^\{\\top\}for a unit leading eigenvectorvvwhen its eigenvalue is positive, and setting𝐄=0\\mathbf\{E\}=0otherwise\. The largest loss difference in[Equation11](https://arxiv.org/html/2610.11226#A2.E11)is thereforeℒ⁡\(𝐖^1\)−ℒ⁡\(𝐖^2\)\+α​e12​\(𝐂\)\\mathcal\{L\}\(\\widehat\{\\mathbf\{W\}\}\_\{1\}\)\-\\mathcal\{L\}\(\\widehat\{\\mathbf\{W\}\}\_\{2\}\)\+\\alpha e\_\{12\}\(\\mathbf\{C\}\)\. The loss of𝐖^1\\widehat\{\\mathbf\{W\}\}\_\{1\}remains no larger under every change satisfying the two constraints exactly when this expression is nonpositive\. For𝚫=α​u​u⊤\\boldsymbol\{\\Delta\}=\\alpha uu^\{\\top\}withu⊤​𝐂​u=1u^\{\\top\}\\mathbf\{C\}u=1, the change in the loss difference isα​u⊤​𝐁12​u\\alpha u^\{\\top\}\\mathbf\{B\}\_\{12\}u\. Requiring it to be at mostℒ⁡\(𝐖^2\)−ℒ⁡\(𝐖^1\)\\mathcal\{L\}\(\\widehat\{\\mathbf\{W\}\}\_\{2\}\)\-\\mathcal\{L\}\(\\widehat\{\\mathbf\{W\}\}\_\{1\}\)for every suchuugives[Equation3](https://arxiv.org/html/2610.11226#S3.E3)\. Sinceℒ⁡\(𝐖^2\)−ℒ⁡\(𝐖^1\)\>0\\mathcal\{L\}\(\\widehat\{\\mathbf\{W\}\}\_\{2\}\)\-\\mathcal\{L\}\(\\widehat\{\\mathbf\{W\}\}\_\{1\}\)\>0and a maximizing𝚫\\boldsymbol\{\\Delta\}has rank at most one, the directional condition is equivalent to the condition over all matrices𝚫\\boldsymbol\{\\Delta\}satisfying the constraints\.

If the inequality fails, take a unit eigenvectorvvcorresponding to the largest eigenvalue of𝐃\\mathbf\{D\}and set𝚫∗=α𝐂−1/2vv⊤𝐂−1/2\\boldsymbol\{\\Delta\}^\{\*\}=\\alpha\\mathbf\{C\}^\{\-1/2\}vv^\{\\top\}\\mathbf\{C\}^\{\-1/2\}\. This rank\-one change satisfies both constraints and makes𝐖^1\\widehat\{\\mathbf\{W\}\}\_\{1\}have higher expected reconstruction loss than𝐖^2\\widehat\{\\mathbf\{W\}\}\_\{2\}on the new inputs\. ∎

Whene12​\(𝐂\)\>0e\_\{12\}\(\\mathbf\{C\}\)\>0,\(ℒ⁡\(𝐖^2\)−ℒ⁡\(𝐖^1\)\)/e12​\(𝐂\)\\bigl\(\\mathcal\{L\}\(\\widehat\{\\mathbf\{W\}\}\_\{2\}\)\-\\mathcal\{L\}\(\\widehat\{\\mathbf\{W\}\}\_\{1\}\)\\bigr\)/e\_\{12\}\(\\mathbf\{C\}\)is the largest distribution\-change budget for which𝐖^1\\widehat\{\\mathbf\{W\}\}\_\{1\}is guaranteed to have no larger reconstruction loss than𝐖^2\\widehat\{\\mathbf\{W\}\}\_\{2\}\. Ife12​\(𝐂\)=0e\_\{12\}\(\\mathbf\{C\}\)=0, no positive semidefinite change in the second\-moment matrix can give𝐖^1\\widehat\{\\mathbf\{W\}\}\_\{1\}higher reconstruction loss than𝐖^2\\widehat\{\\mathbf\{W\}\}\_\{2\}\. The result compares layer\-output reconstruction losses, not model PPL\.

### B\.4A fixed\-grid example of higher loss on a new input

The following example gives two quantized weight vectors with arbitrarily small reconstruction losses on calibration data\. The vector with the smaller loss nevertheless has a larger squared output error on a new input\. Both vectors use the same fixed scale, and the new input has unit length, so the difference does not arise from changing the scale or increasing the input magnitude\.

###### Theorem B\.3\(Lower reconstruction loss on calibration data need not generalize\)\.

Consider a linear layer with two inputs and one output, with reconstruction lossℒ⁡\(𝐰^\)=\(𝐰^−𝐰\)​𝐇​\(𝐰^−𝐰\)⊤\\mathcal\{L\}\(\\widehat\{\\mathbf\{w\}\}\)=\(\\widehat\{\\mathbf\{w\}\}\-\\mathbf\{w\}\)\\mathbf\{H\}\(\\widehat\{\\mathbf\{w\}\}\-\\mathbf\{w\}\)^\{\\top\}\. Let𝒵b\\mathcal\{Z\}\_\{b\}be any finite set of integer codes containing00and11, and fix the quantization scales\>0s\>0in𝒬b\(s\)=\{s\(k1,k2\):k1,k2∈𝒵b\}\\mathcal\{Q\}\_\{b\}\(s\)=\\\{s\(k\_\{1\},k\_\{2\}\):k\_\{1\},k\_\{2\}\\in\\mathcal\{Z\}\_\{b\}\\\}\. For everyε\>0\\varepsilon\>0, there exist a full\-precision weight vector𝐰∉𝒬b​\(s\)\\mathbf\{w\}\\notin\\mathcal\{Q\}\_\{b\}\(s\), a calibration second\-moment matrix𝐇≻0\\mathbf\{H\}\\succ 0withtr⁡\(𝐇\)=1\\operatorname\{tr\}\(\\mathbf\{H\}\)=1, and two quantized weight vectors𝐰^rec,𝐰^alt∈𝒬b​\(s\)\\widehat\{\\mathbf\{w\}\}\_\{\\rm rec\},\\widehat\{\\mathbf\{w\}\}\_\{\\rm alt\}\\in\\mathcal\{Q\}\_\{b\}\(s\)such that𝐰^rec\\widehat\{\\mathbf\{w\}\}\_\{\\rm rec\}uniquely minimizes reconstruction loss over the grid and0<ℒ⁡\(𝐰^rec\)<ℒ⁡\(𝐰^alt\)≤ε0<\\mathcal\{L\}\(\\widehat\{\\mathbf\{w\}\}\_\{\\rm rec\}\)<\\mathcal\{L\}\(\\widehat\{\\mathbf\{w\}\}\_\{\\rm alt\}\)\\leq\\varepsilon\. On the unit\-length input𝐱∗=\(0,1\)⊤\\mathbf\{x\}\_\{\*\}=\(0,1\)^\{\\top\}, their squared output errors ares2s^\{2\}and00, respectively\. The full\-precision weight vector𝐰\\mathbf\{w\}is the unique unconstrained minimizer of the same reconstruction loss\.

###### Corollary B\.4\(Reconstruction\-only refinement can increase loss on a new input\)\.

In the example of[TheoremB\.3](https://arxiv.org/html/2610.11226#A2.Thmtheorem3), take𝐰^alt\\widehat\{\\mathbf\{w\}\}\_\{\\rm alt\}as the initial quantized weights\. Exactly minimizing reconstruction loss over the same grid returns𝐰^rec\\widehat\{\\mathbf\{w\}\}\_\{\\rm rec\}\. This reduces the loss on calibration data but increases the squared output error on𝐱∗\\mathbf\{x\}\_\{\*\}by exactlys2s^\{2\}\.

###### Proof of[TheoremB\.3](https://arxiv.org/html/2610.11226#A2.Thmtheorem3)\.

Fix a finite set of integer codes𝒵b\\mathcal\{Z\}\_\{b\}containing00and11\. Choose an irrationalδ∈\(0,1\)\\delta\\in\(0,1\)such that

k1\+δ​k2≠0for all​\(k1,k2\)∈𝒵b2∖\{\(0,0\)\}\.k\_\{1\}\+\\delta k\_\{2\}\\neq 0\\quad\\text\{for all \}\(k\_\{1\},k\_\{2\}\)\\in\\mathcal\{Z\}\_\{b\}^\{2\}\\setminus\\\{\(0,0\)\\\}\.\(13\)An irrationalδ\\deltaensures that the expression above is nonzero for every nonzero integer pair in𝒵b2\\mathcal\{Z\}\_\{b\}^\{2\}\.

Define𝐰=\(−δ​s,s\)\\mathbf\{w\}=\(\-\\delta s,s\),𝐱=\(1,δ\)⊤\\mathbf\{x\}=\(1,\\delta\)^\{\\top\}, and𝐇~η=𝐱𝐱⊤\+η​𝐈2\\widetilde\{\\mathbf\{H\}\}\_\{\\eta\}=\\mathbf\{x\}\\mathbf\{x\}^\{\\top\}\+\\eta\\mathbf\{I\}\_\{2\}\. LetZη=1\+δ2\+2​ηZ\_\{\\eta\}=1\+\\delta^\{2\}\+2\\etaand𝐇η=𝐇~η/Zη\\mathbf\{H\}\_\{\\eta\}=\\widetilde\{\\mathbf\{H\}\}\_\{\\eta\}/Z\_\{\\eta\}, sotr⁡\(𝐇η\)=1\\operatorname\{tr\}\(\\mathbf\{H\}\_\{\\eta\}\)=1\. We first use the unnormalized second\-moment matrix𝐇~η\\widetilde\{\\mathbf\{H\}\}\_\{\\eta\}, since dividing by the positive scalarZηZ\_\{\\eta\}preserves the loss comparison between all weight vectors in the grid\. Because𝐰𝐱=0\\mathbf\{w\}\\mathbf\{x\}=0, the reconstruction loss of𝐰^=s⁡\(k1,k2\)\\widehat\{\\mathbf\{w\}\}=s\(k\_\{1\},k\_\{2\}\)becomes

Lη​\(𝐰^\)=s2​\(k1\+δ​k2\)2\+η​‖𝐰^−𝐰‖22\.L\_\{\\eta\}\(\\widehat\{\\mathbf\{w\}\}\)=s^\{2\}\(k\_\{1\}\+\\delta k\_\{2\}\)^\{2\}\+\\eta\\left\\lVert\\widehat\{\\mathbf\{w\}\}\-\\mathbf\{w\}\\right\\rVert\_\{2\}^\{2\}\.\(14\)Atη=0\\eta=0,[Equation13](https://arxiv.org/html/2610.11226#A2.E13)makes𝐰^rec=\(0,0\)\\widehat\{\\mathbf\{w\}\}\_\{\\mathrm\{rec\}\}=\(0,0\)the only quantized weight vector with zero loss\. The finite grid gives a positive reconstruction\-loss gap

r=min𝐰^∈𝒬b​\(s\),𝐰^≠𝐰^rec⁡L0​\(𝐰^\)\>0\.r=\\min\_\{\\widehat\{\\mathbf\{w\}\}\\in\\mathcal\{Q\}\_\{b\}\(s\),\\,\\widehat\{\\mathbf\{w\}\}\\neq\\widehat\{\\mathbf\{w\}\}\_\{\\mathrm\{rec\}\}\}L\_\{0\}\(\\widehat\{\\mathbf\{w\}\}\)\>0\.\(15\)LetD=max𝐰^∈𝒬b​\(s\),𝐰^≠𝐰^rec⁡\|‖𝐰^−𝐰‖22−‖𝐰‖22\|D=\\max\_\{\\widehat\{\\mathbf\{w\}\}\\in\\mathcal\{Q\}\_\{b\}\(s\),\\,\\widehat\{\\mathbf\{w\}\}\\neq\\widehat\{\\mathbf\{w\}\}\_\{\\mathrm\{rec\}\}\}\|\\left\\lVert\\widehat\{\\mathbf\{w\}\}\-\\mathbf\{w\}\\right\\rVert\_\{2\}^\{2\}\-\\left\\lVert\\mathbf\{w\}\\right\\rVert\_\{2\}^\{2\}\|\. Setη0=1\\eta\_\{0\}=1ifD=0D=0andη0=r/\(2​D\)\\eta\_\{0\}=r/\(2D\)otherwise\. For every nonzero quantized weight vector and0<η<η00<\\eta<\\eta\_\{0\},

Lη​\(𝐰^\)−Lη​\(𝐰^rec\)\\displaystyle L\_\{\\eta\}\(\\widehat\{\\mathbf\{w\}\}\)\-L\_\{\\eta\}\(\\widehat\{\\mathbf\{w\}\}\_\{\\mathrm\{rec\}\}\)=L0​\(𝐰^\)\+η⁡\(‖𝐰^−𝐰‖22−‖𝐰‖22\)\\displaystyle=L\_\{0\}\(\\widehat\{\\mathbf\{w\}\}\)\+\\eta\\left\(\\left\\lVert\\widehat\{\\mathbf\{w\}\}\-\\mathbf\{w\}\\right\\rVert\_\{2\}^\{2\}\-\\left\\lVert\\mathbf\{w\}\\right\\rVert\_\{2\}^\{2\}\\right\)\(16\)≥r−η​D\>r/2\>0\.\\displaystyle\\geq r\-\\eta D\>r/2\>0\.\(17\)Thus𝐰^rec\\widehat\{\\mathbf\{w\}\}\_\{\\mathrm\{rec\}\}remains the unique minimizer of reconstruction loss over the grid when the second\-moment matrix is positive definite\.

Take𝐰^alt=\(0,s\)\\widehat\{\\mathbf\{w\}\}\_\{\\mathrm\{alt\}\}=\(0,s\)\. The two weight vectors differ from𝐰\\mathbf\{w\}by\(δ​s,−s\)\(\\delta s,\-s\)and\(δ​s,0\)\(\\delta s,0\), respectively\. Substitution gives

Lη​\(𝐰^rec\)=η​s2​\(1\+δ2\),Lη​\(𝐰^alt\)=δ2​s2​\(1\+η\),L\_\{\\eta\}\(\\widehat\{\\mathbf\{w\}\}\_\{\\mathrm\{rec\}\}\)=\\eta s^\{2\}\(1\+\\delta^\{2\}\),\\qquad L\_\{\\eta\}\(\\widehat\{\\mathbf\{w\}\}\_\{\\mathrm\{alt\}\}\)=\\delta^\{2\}s^\{2\}\(1\+\\eta\),\(18\)whose difference iss2​\(δ2−η\)s^\{2\}\(\\delta^\{2\}\-\\eta\)\. The reconstruction minimizer has strictly smaller loss on calibration data wheneverη<δ2\\eta<\\delta^\{2\}\.

Givenε\>0\\varepsilon\>0, chooseδ\\deltasatisfying[Equation13](https://arxiv.org/html/2610.11226#A2.E13)andδ2​s2≤ε/4\\delta^\{2\}s^\{2\}\\leq\\varepsilon/4\. Then choose

0<η<min⁡\{η0,δ2/2,1,ε4​s2​\(1\+δ2\)\}\.0<\\eta<\\min\\left\\\{\\eta\_\{0\},\\delta^\{2\}/2,1,\\frac\{\\varepsilon\}\{4s^\{2\}\(1\+\\delta^\{2\}\)\}\\right\\\}\.\(19\)These choices imply0<Lη​\(𝐰^rec\)<Lη​\(𝐰^alt\)≤ε0<L\_\{\\eta\}\(\\widehat\{\\mathbf\{w\}\}\_\{\\mathrm\{rec\}\}\)<L\_\{\\eta\}\(\\widehat\{\\mathbf\{w\}\}\_\{\\mathrm\{alt\}\}\)\\leq\\varepsilonfor the unnormalized second moment\. Normalizing its trace to one reduces both losses by the common factorZη−1Z\_\{\\eta\}^\{\-1\}and preserves the strict ordering\. For the unit input vector𝐱∗=\(0,1\)⊤\\mathbf\{x\}\_\{\*\}=\(0,1\)^\{\\top\},

\|\(𝐰^rec−𝐰\)​𝐱∗\|2=s2,\|\(𝐰^alt−𝐰\)​𝐱∗\|2=0\.\|\(\\widehat\{\\mathbf\{w\}\}\_\{\\mathrm\{rec\}\}\-\\mathbf\{w\}\)\\mathbf\{x\}\_\{\*\}\|^\{2\}=s^\{2\},\\qquad\|\(\\widehat\{\\mathbf\{w\}\}\_\{\\mathrm\{alt\}\}\-\\mathbf\{w\}\)\\mathbf\{x\}\_\{\*\}\|^\{2\}=0\.\(20\)Finally,𝐇η≻0\\mathbf\{H\}\_\{\\eta\}\\succ 0becauseη\>0\\eta\>0\. The unconstrained loss is zero only at𝐰^=𝐰\\widehat\{\\mathbf\{w\}\}=\\mathbf\{w\}, which the grid cannot represent because−δ\-\\deltais not an integer\. This second moment can also be realized by input activations\. For example, a distribution that assigns probability1/21/2to each column of𝐗=2​𝐇η1/2\\mathbf\{X\}=\\sqrt\{2\}\\,\\mathbf\{H\}\_\{\\eta\}^\{1/2\}has second moment𝐇η\\mathbf\{H\}\_\{\\eta\}\. This completes the example with a fixed scale, a trace\-one second moment, and a unit\-length new input\. ∎

###### Proof of[CorollaryB\.4](https://arxiv.org/html/2610.11226#A2.Thmtheorem4)\.

By[TheoremB\.3](https://arxiv.org/html/2610.11226#A2.Thmtheorem3),𝐰^rec\\widehat\{\\mathbf\{w\}\}\_\{\\rm rec\}uniquely minimizes reconstruction loss over the entire grid and has strictly lower loss on calibration data than𝐰^alt\\widehat\{\\mathbf\{w\}\}\_\{\\rm alt\}\. Exact minimization must therefore return𝐰^rec\\widehat\{\\mathbf\{w\}\}\_\{\\rm rec\}\. The squared output errors of these two vectors on𝐱∗\\mathbf\{x\}\_\{\*\}ares2s^\{2\}and zero, respectively\. ∎

Sampling independently from the two\-point distribution above satisfies the assumptions of[TheoremB\.2](https://arxiv.org/html/2610.11226#A2.Thmtheorem2)\. Taking𝒥⁡\(𝐰^\)=\|\(𝐰^−𝐰\)​𝐱∗\|2\\mathcal\{J\}\(\\widehat\{\\mathbf\{w\}\}\)=\|\(\\widehat\{\\mathbf\{w\}\}\-\\mathbf\{w\}\)\\mathbf\{x\}\_\{\*\}\|^\{2\}, the theorem gives an evaluation\-loss gap ofs2s^\{2\}for the exact empirical minimizer for all sufficiently large sample sizes, with probability one\. This is a counterexample for linear\-layer reconstruction, rather than a bound on model PPL\. It permits a general change in input distribution, whereas the following DRQ result uses the positive semidefinite changes specified in[Section3\.1](https://arxiv.org/html/2610.11226#S3.SS1)\.

### B\.5Worst\-case reconstruction loss

DRQ evaluates each quantized weight matrix by its largest expected reconstruction loss over input activation distributions whose second\-moment matrices are𝐇\+𝚫\\mathbf\{H\}\+\\boldsymbol\{\\Delta\}, with𝚫⪰0\\boldsymbol\{\\Delta\}\\succeq 0andtr⁡\(𝐂​𝚫\)≤α\\operatorname\{tr\}\(\\mathbf\{C\}\\boldsymbol\{\\Delta\}\)\\leq\\alpha\. We compute𝐇\\mathbf\{H\}from the search subset and keep both𝐇\\mathbf\{H\}and𝐂\\mathbf\{C\}fixed during refinement\.[Theorem3\.2](https://arxiv.org/html/2610.11226#S3.Thmtheorem2)shows that a change along one input direction attains the largest loss increase\. Finding this direction reduces to an eigenvalue problem, as shown below\.

###### Proof of[Theorem3\.2](https://arxiv.org/html/2610.11226#S3.Thmtheorem2)\.

Set𝐄=𝐂1/2​𝚫​𝐂1/2\\mathbf\{E\}=\\mathbf\{C\}^\{1/2\}\\boldsymbol\{\\Delta\}\\mathbf\{C\}^\{1/2\}and

𝐁=𝐂−1/2\(𝐖^−𝐖\)⊤\(𝐖^−𝐖\)𝐂−1/2\.\\mathbf\{B\}=\\mathbf\{C\}^\{\-1/2\}\(\\widehat\{\\mathbf\{W\}\}\-\\mathbf\{W\}\)^\{\\top\}\(\\widehat\{\\mathbf\{W\}\}\-\\mathbf\{W\}\)\\mathbf\{C\}^\{\-1/2\}\.This change of variables is invertible, and the constraints become𝐄⪰0\\mathbf\{E\}\\succeq 0andtr⁡\(𝐄\)≤α\\operatorname\{tr\}\(\\mathbf\{E\}\)\\leq\\alpha\. The expected reconstruction loss under the second\-moment matrix𝐇\+𝚫\\mathbf\{H\}\+\\boldsymbol\{\\Delta\}is

tr⁡\[\(𝐖^−𝐖\)​\(𝐇\+𝚫\)​\(𝐖^−𝐖\)⊤\]=ℒ⁡\(𝐖^\)\+tr⁡\(𝐁𝐄\)\.\\operatorname\{tr\}\\\!\\left\[\(\\widehat\{\\mathbf\{W\}\}\-\\mathbf\{W\}\)\(\\mathbf\{H\}\+\\boldsymbol\{\\Delta\}\)\(\\widehat\{\\mathbf\{W\}\}\-\\mathbf\{W\}\)^\{\\top\}\\right\]=\\mathcal\{L\}\(\\widehat\{\\mathbf\{W\}\}\)\+\\operatorname\{tr\}\(\\mathbf\{B\}\\mathbf\{E\}\)\.\(21\)Since both matrices are positive semidefinite,tr⁡\(𝐁𝐄\)≤λmax​\(𝐁\)​tr⁡\(𝐄\)≤α​λmax​\(𝐁\)\\operatorname\{tr\}\(\\mathbf\{B\}\\mathbf\{E\}\)\\leq\\lambda\_\{\\max\}\(\\mathbf\{B\}\)\\operatorname\{tr\}\(\\mathbf\{E\}\)\\leq\\alpha\\lambda\_\{\\max\}\(\\mathbf\{B\}\)\.

Letvvbe a unit eigenvector for the largest eigenvalue of𝐁\\mathbf\{B\}, and choose𝐄∗=α​v​v⊤\\mathbf\{E\}^\{\*\}=\\alpha vv^\{\\top\}\. Thentr⁡\(𝐄∗\)=α\\operatorname\{tr\}\(\\mathbf\{E\}^\{\*\}\)=\\alphaand the upper bound is attained\. The resulting change in the second\-moment matrix

𝚫∗=α𝐂−1/2vv⊤𝐂−1/2\\boldsymbol\{\\Delta\}^\{\*\}=\\alpha\\mathbf\{C\}^\{\-1/2\}vv^\{\\top\}\\mathbf\{C\}^\{\-1/2\}\(22\)satisfies both constraints and uses the full distribution\-change budgetα\\alpha\. Finally,λmax​\(𝐁\)=𝒮𝐂​\(𝐖^\)\\lambda\_\{\\max\}\(\\mathbf\{B\}\)=\\mathcal\{S\}\_\{\\mathbf\{C\}\}\(\\widehat\{\\mathbf\{W\}\}\)by settingu=𝐂−1/2vu=\\mathbf\{C\}^\{\-1/2\}v, which givesu⊤​𝐂​u=1u^\{\\top\}\\mathbf\{C\}u=1and\(𝐖^−𝐖\)⊤​\(𝐖^−𝐖\)​u=λmax​\(𝐁\)​𝐂​u\(\\widehat\{\\mathbf\{W\}\}\-\\mathbf\{W\}\)^\{\\top\}\(\\widehat\{\\mathbf\{W\}\}\-\\mathbf\{W\}\)u=\\lambda\_\{\\max\}\(\\mathbf\{B\}\)\\mathbf\{C\}u\. This proves[Theorem3\.2](https://arxiv.org/html/2610.11226#S3.Thmtheorem2)\. ∎

Ifα=0\\alpha=0or𝐖^=𝐖\\widehat\{\\mathbf\{W\}\}=\\mathbf\{W\},𝚫=0\\boldsymbol\{\\Delta\}=0attains the maximum\. Otherwise the construction above has rank one and can be realized by adding an independent perturbation that equalsα​u\\sqrt\{\\alpha\}\\,uor−α​u\-\\sqrt\{\\alpha\}\\,u, each with probability1/21/2, to the calibration input activations\. Its mean is zero and its second moment isα​u​u⊤\\alpha uu^\{\\top\}, so the resulting distribution has second moment𝐇\+𝚫∗\\mathbf\{H\}\+\\boldsymbol\{\\Delta\}^\{\*\}\.

The maximizing direction depends on both the weight difference𝐖^−𝐖\\widehat\{\\mathbf\{W\}\}\-\\mathbf\{W\}and𝐂\\mathbf\{C\}\. Changing integer codes can therefore change the worst\-case direction, which is why DRQ estimates it again before accepting proposed weights\.

### B\.6Comparing refined and initial quantized weights

Two weight matrices can attain their worst\-case losses under different input distributions\. A smaller worst\-case loss therefore does not imply smaller loss under every distribution in the set\. To check this stronger property, we compare both matrices under the same𝚫\\boldsymbol\{\\Delta\}\. Write𝐖^0\\widehat\{\\mathbf\{W\}\}\_\{0\}for the initial quantized weights\. Usinge12​\(𝐂\)e\_\{12\}\(\\mathbf\{C\}\)from[Equation10](https://arxiv.org/html/2610.11226#A2.E10), define

Γα​\(𝐖^1,𝐖^2\)=ℒ⁡\(𝐖^1\)−ℒ⁡\(𝐖^2\)\+α​e12​\(𝐂\)\.\\Gamma\_\{\\alpha\}\(\\widehat\{\\mathbf\{W\}\}\_\{1\},\\widehat\{\\mathbf\{W\}\}\_\{2\}\)=\\mathcal\{L\}\(\\widehat\{\\mathbf\{W\}\}\_\{1\}\)\-\\mathcal\{L\}\(\\widehat\{\\mathbf\{W\}\}\_\{2\}\)\+\\alpha e\_\{12\}\(\\mathbf\{C\}\)\.\(23\)
###### Proposition B\.5\(Comparing reconstruction losses under the same input distribution\)\.

For any𝐖^1,𝐖^2∈ℝm×d\\widehat\{\\mathbf\{W\}\}\_\{1\},\\widehat\{\\mathbf\{W\}\}\_\{2\}\\in\\mathbb\{R\}^\{m\\times d\},𝐂≻0\\mathbf\{C\}\\succ 0, andα≥0\\alpha\\geq 0, the quantity in[Equation23](https://arxiv.org/html/2610.11226#A2.E23)satisfies

Γα​\(𝐖^1,𝐖^2\)\\displaystyle\\Gamma\_\{\\alpha\}\(\\widehat\{\\mathbf\{W\}\}\_\{1\},\\widehat\{\\mathbf\{W\}\}\_\{2\}\)=max𝚫⪰0,tr⁡\(𝐂​𝚫\)≤α⁡tr⁡\(𝐁12​\(𝐇\+𝚫\)\)\\displaystyle=\\max\_\{\\boldsymbol\{\\Delta\}\\succeq 0,\\,\\operatorname\{tr\}\(\\mathbf\{C\}\\boldsymbol\{\\Delta\}\)\\leq\\alpha\}\\operatorname\{tr\}\\\!\\left\(\\mathbf\{B\}\_\{12\}\(\\mathbf\{H\}\+\\boldsymbol\{\\Delta\}\)\\right\)\(24\)≤ℒDRQ​\(𝐖^1\)−ℒ⁡\(𝐖^2\)\.\\displaystyle\\leq\\mathcal\{L\}\_\{\\rm DRQ\}\(\\widehat\{\\mathbf\{W\}\}\_\{1\}\)\-\\mathcal\{L\}\(\\widehat\{\\mathbf\{W\}\}\_\{2\}\)\.Consequently,Γα​\(𝐖^1,𝐖^0\)<0\\Gamma\_\{\\alpha\}\(\\widehat\{\\mathbf\{W\}\}\_\{1\},\\widehat\{\\mathbf\{W\}\}\_\{0\}\)<0certifies that𝐖^1\\widehat\{\\mathbf\{W\}\}\_\{1\}has strictly smaller reconstruction loss than𝐖^0\\widehat\{\\mathbf\{W\}\}\_\{0\}for every second moment satisfying these constraints\.

###### Proof\.

The equality follows from[Equation12](https://arxiv.org/html/2610.11226#A2.E12)\. Since𝚫⪰0\\boldsymbol\{\\Delta\}\\succeq 0, the reconstruction loss of𝐖^2\\widehat\{\\mathbf\{W\}\}\_\{2\}is at leastℒ⁡\(𝐖^2\)\\mathcal\{L\}\(\\widehat\{\\mathbf\{W\}\}\_\{2\}\), while that of𝐖^1\\widehat\{\\mathbf\{W\}\}\_\{1\}is at mostℒDRQ​\(𝐖^1\)\\mathcal\{L\}\_\{\\rm DRQ\}\(\\widehat\{\\mathbf\{W\}\}\_\{1\}\)\. Subtracting these bounds and maximizing proves[Equation24](https://arxiv.org/html/2610.11226#A2.E24)\. A negative maximum makes the pairwise difference negative for every second moment satisfying the constraints\. ∎

The guarantee uses the exact value ofΓα\\Gamma\_\{\\alpha\}\. DRQ estimates this quantity for reporting when its selected weights differ from the initial weights\. A negative estimate alone does not establish the guarantee, since a Rayleigh quotient can underestimate the largest eigenvalue\. The acceptance rule uses the reconstruction tolerances and a decrease in the estimated worst\-case reconstruction loss, as described in Appendix[C](https://arxiv.org/html/2610.11226#A3)\.

### B\.7Equivalent optimization over input directions

For a matrix,‖⋅‖2\\left\\lVert\\cdot\\right\\rVert\_\{2\}denotes its largest singular value\. The identity‖\(𝐖^−𝐖\)𝐂−1/2‖22=maxu⊤​𝐂​u=1‖\(𝐖^−𝐖\)u‖22\\left\\lVert\(\\widehat\{\\mathbf\{W\}\}\-\\mathbf\{W\}\)\\mathbf\{C\}^\{\-1/2\}\\right\\rVert\_\{2\}^\{2\}=\\max\_\{u^\{\\top\}\\mathbf\{C\}u=1\}\\left\\lVert\(\\widehat\{\\mathbf\{W\}\}\-\\mathbf\{W\}\)u\\right\\rVert\_\{2\}^\{2\}gives the directional penalty in[Equation5](https://arxiv.org/html/2610.11226#S3.E5)\. Introducing a scalarttthat bounds this error gives an equivalent form of the minimization in[Equation4](https://arxiv.org/html/2610.11226#S3.E4),

min𝐖^∈𝒬,t\\displaystyle\\min\_\{\\widehat\{\\mathbf\{W\}\}\\in\\mathcal\{Q\},t\}ℒ⁡\(𝐖^\)\+α​t\\displaystyle\\mathcal\{L\}\(\\widehat\{\\mathbf\{W\}\}\)\+\{\\alpha\}t\(25\)s\.t\.\\displaystyle\\text\{s\.t\.\}‖\(𝐖^−𝐖\)u‖22≤t,∀u:u⊤𝐂u=1\.\\displaystyle\\left\\lVert\(\\widehat\{\\mathbf\{W\}\}\-\\mathbf\{W\}\)u\\right\\rVert\_\{2\}^\{2\}\\leq t,\\quad\\forall u:u^\{\\top\}\\mathbf\{C\}u=1\.For fixed weights andα\>0\\alpha\>0, minimizing overttsets it to the largest directional error\. Atα=0\\alpha=0, this value ofttis still feasible and optimal, although larger values are also optimal\. The formulation therefore preserves the minimum and the minimizing weight matrices for everyα≥0\\alpha\\geq 0\. It is called semi\-infinite because it has finitely many optimization variables but one constraint for each of infinitely many input directions\.

### B\.8A reconstruction bound using Wasserstein distance

Another way to restrict a change in the input activation distribution is to pair original and new activation vectors and bound their average squared difference\. Using the weighted 2\-Wasserstein distance to make this comparison gives a reconstruction\-loss bound involving the same worst\-case directional error as DRQ\.

LetP^n\\widehat\{P\}\_\{n\}assign probability1/n1/nto each calibration input vector used for code search, so its second moment is𝐇\\mathbf\{H\}\. For a distributionQQwith a finite second moment, define the weighted 2\-Wasserstein distance by

𝖶2,𝐂2​\(Q,P^n\)=infπ∈Π⁡\(P^n,Q\)𝔼\(x,x′\)∼π​‖𝐂1/2​\(x′−x\)‖22\.\\mathsf\{W\}\_\{2,\\mathbf\{C\}\}^\{2\}\(Q,\\widehat\{P\}\_\{n\}\)=\\inf\_\{\\pi\\in\\Pi\(\\widehat\{P\}\_\{n\},Q\)\}\\mathbb\{E\}\_\{\(x,x^\{\\prime\}\)\\sim\\pi\}\\left\\lVert\\mathbf\{C\}^\{1/2\}\(x^\{\\prime\}\-x\)\\right\\rVert\_\{2\}^\{2\}\.Each joint distribution inΠ⁡\(P^n,Q\)\\Pi\(\\widehat\{P\}\_\{n\},Q\)pairs an original activation vectorxxwith a new vectorx′x^\{\\prime\}, while preserving their distributionsP^n\\widehat\{P\}\_\{n\}andQQ\. The squared distance is the smallest average value of‖𝐂1/2​\(x′−x\)‖22\\left\\lVert\\mathbf\{C\}^\{1/2\}\(x^\{\\prime\}\-x\)\\right\\rVert\_\{2\}^\{2\}over these pairings\. A changeh=x′−xh=x^\{\\prime\}\-xcontributesh⊤​𝐂​hh^\{\\top\}\\mathbf\{C\}hto this average\. Forε≥0\\varepsilon\\geq 0andζ\>0\\zeta\>0,

supQ:𝖶2,𝐂​\(Q,P^n\)≤ε𝔼Q‖\(𝐖^−𝐖\)x‖22\\displaystyle\\sup\_\{Q:\\mathsf\{W\}\_\{2,\\mathbf\{C\}\}\(Q,\\widehat\{P\}\_\{n\}\)\\leq\\varepsilon\}\\mathbb\{E\}\_\{Q\}\\left\\lVert\(\\widehat\{\\mathbf\{W\}\}\-\\mathbf\{W\}\)x\\right\\rVert\_\{2\}^\{2\}\(26\)≤\(1\+ζ\)\[ℒ\(𝐖^\)\+ε2ζ‖\(𝐖^−𝐖\)𝐂−1/2‖22\]\.\\displaystyle\\leq\(1\+\\zeta\)\\left\[\\mathcal\{L\}\(\\widehat\{\\mathbf\{W\}\}\)\+\\frac\{\\varepsilon^\{2\}\}\{\\zeta\}\\left\\lVert\(\\widehat\{\\mathbf\{W\}\}\-\\mathbf\{W\}\)\\mathbf\{C\}^\{\-1/2\}\\right\\rVert\_\{2\}^\{2\}\\right\]\.The bound combines reconstruction loss on calibration data with the same worst\-case directional error used by DRQ\. It is an upper bound for distributions within a Wasserstein distance, whereas[Theorem3\.2](https://arxiv.org/html/2610.11226#S3.Thmtheorem2)gives an equality for DRQ’s second\-moment constraints\. These are two different sets of distributions\.

###### Proof of[Equation26](https://arxiv.org/html/2610.11226#A2.E26)\.

FixQQwith𝖶2,𝐂​\(Q,P^n\)≤ε\\mathsf\{W\}\_\{2,\\mathbf\{C\}\}\(Q,\\widehat\{P\}\_\{n\}\)\\leq\\varepsilonand let\(x,x′\)\(x,x^\{\\prime\}\)be drawn from a joint distributionπ∈Π⁡\(P^n,Q\)\\pi\\in\\Pi\(\\widehat\{P\}\_\{n\},Q\)\. Minkowski’s inequality inL2L\_\{2\}gives

\(𝔼Q​‖\(𝐖^−𝐖\)​x′‖22\)1/2\\displaystyle\\left\(\\mathbb\{E\}\_\{Q\}\\left\\lVert\(\\widehat\{\\mathbf\{W\}\}\-\\mathbf\{W\}\)x^\{\\prime\}\\right\\rVert\_\{2\}^\{2\}\\right\)^\{1/2\}\(27\)≤\(𝔼P^n​‖\(𝐖^−𝐖\)​x‖22\)1/2\+\(𝔼π​‖\(𝐖^−𝐖\)​\(x′−x\)‖22\)1/2\.\\displaystyle\\leq\\left\(\\mathbb\{E\}\_\{\\widehat\{P\}\_\{n\}\}\\left\\lVert\(\\widehat\{\\mathbf\{W\}\}\-\\mathbf\{W\}\)x\\right\\rVert\_\{2\}^\{2\}\\right\)^\{1/2\}\+\\left\(\\mathbb\{E\}\_\{\\pi\}\\left\\lVert\(\\widehat\{\\mathbf\{W\}\}\-\\mathbf\{W\}\)\(x^\{\\prime\}\-x\)\\right\\rVert\_\{2\}^\{2\}\\right\)^\{1/2\}\.For every displacementhh,‖\(𝐖^−𝐖\)h‖2≤‖\(𝐖^−𝐖\)𝐂−1/2‖2‖𝐂1/2h‖2\\left\\lVert\(\\widehat\{\\mathbf\{W\}\}\-\\mathbf\{W\}\)h\\right\\rVert\_\{2\}\\leq\\left\\lVert\(\\widehat\{\\mathbf\{W\}\}\-\\mathbf\{W\}\)\\mathbf\{C\}^\{\-1/2\}\\right\\rVert\_\{2\}\\left\\lVert\\mathbf\{C\}^\{1/2\}h\\right\\rVert\_\{2\}\. Taking the infimum over joint distributionsπ\\pibounds the second term byε‖\(𝐖^−𝐖\)𝐂−1/2‖2\\varepsilon\\left\\lVert\(\\widehat\{\\mathbf\{W\}\}\-\\mathbf\{W\}\)\\mathbf\{C\}^\{\-1/2\}\\right\\rVert\_\{2\}\. The first term isℒ​\(𝐖^\)1/2\\mathcal\{L\}\(\\widehat\{\\mathbf\{W\}\}\)^\{1/2\}\. Applying\(a\+b\)2≤\(1\+ζ\)​a2\+\(1\+ζ−1\)​b2\(a\+b\)^\{2\}\\leq\(1\+\\zeta\)a^\{2\}\+\(1\+\\zeta^\{\-1\}\)b^\{2\}and taking the supremum over distributionsQQwithin the specified Wasserstein distance proves[Equation26](https://arxiv.org/html/2610.11226#A2.E26)\. ∎

## Appendix CAlgorithmic Details

#### Blockwise integration\.

DRQ follows Algorithm[3\.3](https://arxiv.org/html/2610.11226#S3.SS3), refining each linear layer after the base PTQ method quantizes it\. The bit width, grouping, scales, and zero points remain fixed\. DRQ uses cached calibration activations and the corresponding full\-precision weights after any scaling or rotation by the base method\. In GPTQ, refined layers supply inputs for subsequent layer groups within the block\. In AWQ, refinement follows the block’s scaling and clipping\. The refined block then produces calibration inputs for the next block\.

#### Calibration subsets\.

We split the selected activation vectors into a search subset𝐗d\\mathbf\{X\}\_\{\\rm d\}and two separate checking subsets𝐗c\(1\)\\mathbf\{X\}\_\{\\rm c\}^\{\(1\)\}and𝐗c\(2\)\\mathbf\{X\}\_\{\\rm c\}^\{\(2\)\}\. The search subset determines𝐇\\mathbf\{H\},𝐂\\mathbf\{C\}, and reconstruction loss used to score code changes\. These subsets and matrices stay fixed during refinement\. Appendix[D\.3](https://arxiv.org/html/2610.11226#A4.SS3)describes the adjustments for experts receiving few calibration tokens\.

#### Proposing code changes\.

Each round estimates the current weights’ worst\-case input directionuu\. Holdinguufixed, we score code replacements by their change inℒ⁡\(𝐖^\)\+α​‖\(𝐖^−𝐖\)​u‖22\\mathcal\{L\}\(\\widehat\{\\mathbf\{W\}\}\)\+\\alpha\\left\\lVert\(\\widehat\{\\mathbf\{W\}\}\-\\mathbf\{W\}\)u\\right\\rVert\_\{2\}^\{2\}\. In each row, we retain the eight lowest\-scoring single\-code replacements and test pairs that change two different weights\. We keep the single replacement or pair with the largest decrease in this objective, provided the decrease is positive\. We then rank these row updates by their decreases and apply different fractions of the list to the current weights, starting with the largest decreases\.

#### Accepting new weights\.

For each proposed matrix, we estimate its worst\-case direction again\. Its estimatedℒDRQ\\mathcal\{L\}\_\{\\rm DRQ\}must decrease relative to the current weights\. On the search subset and each checking subset, reconstruction loss must also stay within the specified percentage above that of the initial PTQ weights𝐖^0\\widehat\{\\mathbf\{W\}\}\_\{0\}\. Proposals that fail these conditions or the numerical checks are rejected\. We continue with the passing matrix having the lowest estimatedℒDRQ\\mathcal\{L\}\_\{\\rm DRQ\}, or stop if none passes\. We return the initial or accepted weights with the lowest estimated loss\.

#### Main parameters\.

For the dense\-model and six\-method comparisons, we selectα\\alphaandβ\\betausing 64 validation samples from Pile\([Gao et al\., 2020](https://arxiv.org/html/2610.11226#bib.bib24)\)\. The settings below use the standard128×2,048128\\times 2\{,\}048calibration setup\. Other comparisons retain their stated settings\. Reconstruction\-only refinement uses the same procedure withα=0\\alpha=0\.

## Appendix DMore Experimental Results

### D\.1Detailed dense Llama results

[Tables4](https://arxiv.org/html/2610.11226#A4.T4)to[7](https://arxiv.org/html/2610.11226#A4.T7)report accuracy on each of the seven downstream tasks before and after DRQ refinement\. Average task accuracy improves in every reported comparison with the initial quantized model\. The size of the improvement at lower bit widths depends on the base PTQ method\. For the 70B model quantized with GPTQ at W2, DRQ improves all seven task scores and raises average accuracy by 6\.33 percentage points\.

Table 4:Llama\-3\.2\-1B\-Instruct accuracy \(%\) on each downstream task\.Table 5:Llama\-3\.2\-3B\-Instruct accuracy \(%\) on each downstream task\.Table 6:Llama\-3\.1\-8B\-Instruct accuracy \(%\) on each downstream task\.Table 7:Llama\-3\.1\-70B\-Instruct accuracy \(%\) on each downstream task\.
### D\.2Results across PTQ methods

[Table8](https://arxiv.org/html/2610.11226#A4.T8)reports WT2 and C4 perplexity and accuracy on each of the seven downstream tasks for the initial and DRQ\-refined Llama\-3\.2\-3B\-Instruct W3A16 models in[Figure3](https://arxiv.org/html/2610.11226#S4.F3)\. The OmniQuant runs use learned weight clipping without learned equivalent transformations\. Across the six selected comparisons, DRQ lowers both perplexities and raises average task accuracy\. For GPTQ and AWQ, accuracy improves on six of the seven tasks\. For RTN, gains on PIQA, BoolQ, and ARC\-Easy outweigh declines on the other four tasks\.

Table 8:Perplexity and seven\-task accuracy \(%\) for Llama\-3\.2\-3B W3\.#### QuaRot with GPTQ\.

We further evaluate DRQ on Llama\-3\.1\-8B\-Instruct with QuaRot’s Hadamard rotations followed by GPTQ\([Ashkboos et al\., 2024](https://arxiv.org/html/2610.11226#bib.bib1);[Frantar et al\., 2023](https://arxiv.org/html/2610.11226#bib.bib4)\)\. DRQ refines the integer codes of the quantized weights in the rotated representation while retaining the rotations and quantization parameters\. We test W2A16 and W3A16 with a group size of 128\. Each calibration set contains 128 WT2 sequences of 2,048 tokens\. The reconstruction tolerances on the search and checking subsets are both 0\.01\. We use\(α,β\)=\(1,0\.005\)\(\\alpha,\\beta\)=\(1,0\.005\)for W2 and\(0\.1,0\.005\)\(0\.1,0\.005\)for W3\. As is shown in[Table9](https://arxiv.org/html/2610.11226#A4.T9), at W3, DRQ improves accuracy on all seven tasks, raising mean accuracy from 68\.44% to 69\.72% while lowering WT2 and C4 PPL\. This shows that DRQ can further improve a model quantized with QuaRot and GPTQ\.

Table 9:QuaRot \+ GPTQ with DRQ on Llama\-3\.1\-8B\-Instruct\.

### D\.3Calibration tokens and expert refinement

We examine how the number of calibration tokens reaching each expert affects DRQ refinement in the Qwen3\-30B\-A3B run reported in[Table2](https://arxiv.org/html/2610.11226#S4.T2)\. We quantize 18,432 expert linear layers and 192 attention linear layers across 48 Transformer blocks, leaving the routers unquantized\. Each expert contains three linear layers\.

#### Routing distributes calibration tokens unevenly\.

[Figure6](https://arxiv.org/html/2610.11226#A4.F6)\(a\) shows that 4,794 of 18,624 linear layers \(25\.74%\) receive fewer than the 1,024 input activation vectors used for the full search and checking subsets\. These include 399 linear layers with one or two routed tokens and 201 with none, all belonging to experts\. The router sends different numbers of calibration tokens to different experts\. Some experts therefore receive too few tokens to form separate checking subsets\.

Figure 6:Calibration token counts and integer\-code changes in Qwen3\-30B\-A3B W3A16\. \(a\) The 18,624 quantized linear layers grouped by the number of calibration tokens received, with layer counts shown beside the bars\. \(b\) The fraction of linear layers with at least one changed integer code in each token\-count group\. The 0–2\-token groups contain expert linear layers exclusively, while the≥3\\geq 3group includes both expert and attention linear layers\.
#### Refinement uses the available tokens\.

DRQ adjusts its reconstruction checks to the number of available tokens\. With at least three tokens, it forms three disjoint subsets, one for proposing and scoring integer\-code changes and two for checking reconstruction loss\. It reduces their sizes when needed\. With one or two tokens, all input activation vectors are used for search\. Linear layers receiving no tokens use RTN initialization and set𝐇\\mathbf\{H\}and𝐂\\mathbf\{C\}to the identity matrix\. In this case, the reconstruction term is the squared Frobenius distance between the quantized and full\-precision weights\. Layers with fewer than three tokens have no separate checking subsets\. The reconstruction\-loss limits on both checking subsets therefore apply to layers with at least three tokens\. This procedure uses the activations routed by the model without adding artificial tokens or changing which experts the router selects\.

In[Figure6](https://arxiv.org/html/2610.11226#A4.F6)\(b\), DRQ modifies 266 of the 399 linear layers receiving one or two tokens \(66\.7%\), compared with 8,657 of 18,024 linear layers in the≥3\\geq 3group \(48\.0%\)\. Layers receiving one or two tokens are modified more often but contribute only 199,289 of the 5,425,802 integer\-code changes \(3\.67%\)\. All 201 zero\-token linear layers retain their initial integer codes\. Overall, DRQ changes 0\.01815% of the weights included in these counts, compared with the codes before each layer is refined, while retaining scales and zero points\. These changes affect many linear layers but a small fraction of their weights, with most code changes occurring in the≥3\\geq 3group\.

### D\.4Parameter sensitivity

[Figure7](https://arxiv.org/html/2610.11226#A4.F7)evaluates the roles ofα\\alphaandβ\\betaon Llama\-3\.2\-3B\-Instruct W3\. In[Equation5](https://arxiv.org/html/2610.11226#S3.E5),α\\alphaweights the worst\-case directional error, whileβ\\betasets the weight on the identity matrix in𝐂\\mathbf\{C\}\. We test all combinations ofα∈\{0\.1,0\.3,0\.5,1\}\\alpha\\in\\\{0\.1,0\.3,0\.5,1\\\}andβ∈\{0\.005,0\.1,0\.25,0\.75,1\}\\beta\\in\\\{0\.005,0\.1,0\.25,0\.75,1\\\}\. Among the displayed runs, 17 of 20 GPTQ configurations and 11 of 20 AWQ configurations improve average task accuracy and both WT2 and C4 PPL relative to the initial quantized model\. Improvements on all three metrics occur across more GPTQ configurations, while the favorable AWQ settings concentrate aroundα=0\.3\\alpha=0\.3and0\.50\.5\.

Atα=0\.3\\alpha=0\.3, both methods improve all three metrics across all fiveβ\\betavalues\. Increasingα\\alphato 1 worsens WT2 PPL in three GPTQ and two AWQ configurations\. A stronger penalty on worst\-case directional error therefore does not consistently improve model performance in this sweep\. The objective balances this penalty against reconstruction loss on calibration data, and the parameter settings that improve model performance depend on the base PTQ method\.

Figure 7:Performance across differentα\\alphaandβ\\betacombinations on Llama\-3\.2\-3B\-Instruct at W3\.

## Appendix EComputational Efficiency

### E\.1Refinement time across PTQ methods

[Table10](https://arxiv.org/html/2610.11226#A5.T10)compares DRQ refinement time for the settings in Appendix[D\.2](https://arxiv.org/html/2610.11226#A4.SS2)\. The layer\-refinement time sums 196 DRQ calls, one per linear layer\. The time including integration also covers the surrounding code that calls DRQ within the base PTQ method or passes stored calibration activations to it\. Both columns exclude base\-method quantization and hyperparameter search\. RTN, GPTQ, AWQ, OmniQuant, and GPTAQ were remeasured on the same A800 80GB GPU\. ParoQuant uses its earlier timing measurement\.

Table 10:DRQ refinement time for Llama\-3\.2\-3B W3\.
### E\.2Scaling with model size and bit width

[Figure8](https://arxiv.org/html/2610.11226#A5.F8)compares DRQ refinement time at W2–W4 across 1B–70B Llama models\. We sum the time spent refining linear layers\. Each 1B–8B point averages the runs for that setting, with 86 runs in total at\(α,β\)=\(0\.3,0\.25\)\(\\alpha,\\beta\)=\(0\.3,0\.25\)\. The 1B–8B models use one A800 80GB GPU, and the 70B model uses two\. At W3, refinement takes 2\.24, 4\.05, 5\.95, and 28\.05 minutes with GPTQ for 1B, 3B, 8B, and 70B, respectively\. The corresponding AWQ times are 2\.00, 3\.90, 5\.40, and 28\.96 minutes\. The two 70B W3 runs therefore spend similar time on DRQ\. At 70B, W2/W4 take 36\.55/26\.85 minutes with GPTQ and 76\.03/28\.20 minutes with AWQ\. The AWQ W2 and W4 runs use\(α,β\)=\(1,1\)\(\\alpha,\\beta\)=\(1,1\)and\(0\.3,1\)\(0\.3,1\), respectively\. For every 1B–70B model and base PTQ method, W2 takes longer than W3 and W4, including when comparing runs with the same seeds\. Refinement time depends on the code replacements tested and the number of search and checking steps performed\.

Figure 8:DRQ refinement time across model sizes and weight bit widths\.
### E\.3Search complexity and deployment

DRQ limits search cost by retaining a small set of integer\-code changes and testing selected combinations\. Consider a weight matrix withmmrows, one per output channel, andddcolumns, one per input channel\. Withndn\_\{\\rm d\}calibration input vectors in the search subset andB=\|𝒵b\|B=\|\\mathcal\{Z\}\_\{b\}\|available integer codes, scoring all single\-code changes costsO⁡\(m​d​nd\+m​d​B\)O\(mdn\_\{\\rm d\}\+mdB\)per round\. Only thek=8k=8shortlisted changes in each output row enter pair scoring, giving at most\(k2\)=28\\binom\{k\}\{2\}=28candidate pairs per row\. Computing their activation inner products costsO⁡\(m​k2​nd\)O\(mk^\{2\}n\_\{\\rm d\}\)\. This shortlist limits the cost of scoring pairs even for wide linear layers while accounting for interactions that single\-code scores miss\.

DRQ retains at most one update per output row, replacing one or two integer codes\. It ranks these row updates by the decrease in the objective with the input directionuuheld fixed inO⁡\(m​log⁡m\)O\(m\\log m\)time and checksp≤6p\\leq 6distinct nonzero prefixes\. Each prefix applies the first part of this ranked list to the current weights\. Increasing its length changes more rows\. Direct checks of reconstruction loss between quantized and full\-precision layer outputs costO⁡\(p​m​d​\(nd\+nc\)\)O\(pmd\(n\_\{\\rm d\}\+n\_\{\\rm c\}\)\), wherencn\_\{\\rm c\}counts all input vectors in the calibration subsets reserved for checking reconstruction loss\. Wider layers and larger activation subsets make each check more expensive, but the prefix limit bounds how many checks are needed\.

We also bound the number of numerical solves used to estimate the worst\-case input direction\. LetTeigT\_\{\\rm eig\}denote the time for one complete numerical solve, including matrix products, linear solves involving𝐂\\mathbf\{C\}, Krylov basis operations, and Rayleigh–Ritz extraction\. Evaluating the initial quantized weights, the current weights at each round, and candidate prefixes costs at most\[1\+∑r=1R\(1\+pe,r\)\]​Teig\[1\+\\sum\_\{r=1\}^\{R\}\(1\+p\_\{\{\\rm e\},r\}\)\]T\_\{\\rm eig\}overR≤3R\\leq 3rounds, wherepe,r≤pp\_\{\{\\rm e\},r\}\\leq pcounts candidate solves in roundrr\. Even if all six prefixes reach this stage in every round, at most 22 solves are needed per linear layer\. Early rejection and stopping reduce this number\. Initial reconstruction checks and the optional final comparison with the initial quantized weights in[Equation23](https://arxiv.org/html/2610.11226#A2.E23)add one\-time costs\.

All of this computation takes place during offline refinement\. DRQ changes the integer codes representing quantized weights while preserving the bit width, quantization grid, dequantization rule, and parameter layout\. It therefore adds no inference operators or additional stored quantization parameters\. Appendix[E\.4](https://arxiv.org/html/2610.11226#A5.SS4)measures the resulting latency, throughput, and memory use\.

### E\.4Inference overhead

We benchmark Llama\-3\.1\-8B\-Instruct at W4A16 with a group size of 128 on one NVIDIA A800 80GB GPU, using vLLM 0\.6\.3 in eager mode\. Batch sizes are\{1,4,8\}\\\{1,4,8\\\}, input lengths are\{128,2048\}\\\{128,2048\\\}, and output length is 128 tokens\. We evaluate each initial quantized model and its DRQ\-refined counterpart on the same workload and physical GPU, with 10 warm\-up runs and 30 timed repeats per round\. We time GPU\-synchronized engine steps, including scheduling and sampling\. Prefill latency covers the first step that processes the input sequence, while decoding latency averages the remaining 127 token\-generation steps\. Throughput accounts for both prefill and decoding\. For latency and throughput, we take the median within each round and average these medians over four retained pairs of benchmark rounds for GPTQ and AWQ\. Peak memory is the maximum PyTorch allocated memory across retained rounds\.

[Table11](https://arxiv.org/html/2610.11226#A5.T11)reports all six workloads\. Across these workloads, prefill and decoding times change by−0\.80%\-0\.80\\%to\+0\.82%\+0\.82\\%after refinement, computed from the geometric mean of paired latency ratios across rounds\. Model tensors occupy the same amount of memory\. DRQ changes the integer codes representing weights while preserving tensor layouts and inference operators, which explains the unchanged memory footprint under matched workloads\. These results demonstrate that DRQ adds no extra inference computation or memory overhead\.

Table 11:Inference cost across batch sizes and input lengths on Llama\-3\.1\-8B\-Instruct, W4A16\. Output length is fixed at 128 tokens\.