Language-Conditional Dequantization: Recovering What Quantization Steals from Non-English Languages

# Recovering What Quantization Steals from Non-English Languages
Source: [https://arxiv.org/html/2608.11786](https://arxiv.org/html/2608.11786)
## Language\-Conditional Dequantization: Recovering What Quantization Steals from Non\-English Languages

###### Abstract

Aggressive quantization disproportionately harms multilingual capability: in the sub\-4B INT3 GPTQ regime, we measure 2–4×\\timeslarger perplexity degradation on non\-English languages than on English\. We propose Language\-Conditional Dequantization \(LCD\), a post\-hoc method that attaches per\-language rank\-2 LoRA corrections to the linear layers of an already\-quantized model, adding 0\.12% parameters per language and training in under 20 minutes on a single GPU\. Across Qwen2\.5\-3B and Llama\-3\.2\-3B, LCD recovers 70–83% of the perplexity gap for non\-Latin\-script languages and 17–28% of the GlobalMMLU accuracy gap, outperforming a language\-agnostic correction of equal capacity by 3–9 points on typologically distant languages and a data\-free low\-rank baseline \(LQER\) by an order of magnitude\. We further identify a*perplexity–accuracy disconnect*and trace it to*where*quantization concentrates damage: early\-depth errors \(Llama\) propagate downstream and resist local correction, while late\-depth errors \(Qwen\) do not\. A layer\-restricted variant of LCD validates this mechanism directly\.

## 1Introduction

Quantized large language models serve billions of queries daily across the world’s languages\. When a Korean user queries a 3\-bit quantized model, they receive measurably worse outputs than an English user, not because the original model lacked Korean ability, but because the quantization process was calibrated exclusively on English data\. In this paper, we quantify the severity of this gap in the regime where quantization is most needed: on Qwen2\.5\-3B, INT3 GPTQ degrades Arabic perplexity by 4\.37×\\timesand Japanese by 3\.04×\\times, compared to just 1\.35×\\timesfor English; on Llama\-3\.2\-3B the pattern repeats \(Arabic 3\.65×\\times, English 1\.39×\\times\)\. In the sub\-4B INT3 regime, this is a practically consequential, language\-dependent quality gap\.

Multilingual calibration[2](https://arxiv.org/html/2608.11786#bib.bib2)addresses the root cause but requires re\-quantization, which is impractical for already\-deployed models\. Static error correction methods \(LQER[16](https://arxiv.org/html/2608.11786#bib.bib5), RILQ[9](https://arxiv.org/html/2608.11786#bib.bib6), ResQ[13](https://arxiv.org/html/2608.11786#bib.bib7)\) add low\-rank residuals post\-hoc but apply the same correction regardless of input language\. Input\-conditional methods like BinaryMoS[8](https://arxiv.org/html/2608.11786#bib.bib4)adapt weights to the input but target binarization and ignore language identity\.

We propose Language\-Conditional Dequantization \(LCD\), built on a simple insight:*if quantization error is language\-dependent, the correction should be too*\. For each linear layer in a quantized model, LCD attaches a per\-language rank\-rradditive correction via forward hooks:

y^=Wq​x\+1r​\(Aℓ⋅Bℓ\)⋅x\\hat\{y\}=W\_\{q\}x\+\\frac\{1\}\{r\}\(A\_\{\\ell\}\\cdot B\_\{\\ell\}\)\\cdot x\(1\)whereAℓ∈ℝdout×rA\_\{\\ell\}\\in\\mathbb\{R\}^\{d\_\{\\text\{out\}\}\\times r\}andBℓ∈ℝr×dinB\_\{\\ell\}\\in\\mathbb\{R\}^\{r\\times d\_\{\\text\{in\}\}\}are language\-specific correction matrices\. With rankr=2r=2, this adds only 0\.12% parameters per language\. Corrections are trained on 256 samples of language\-specific text in under 20 minutes per language on a single GPU\. At inference time, the appropriate correction is selected based on the input language, requiring no modification to the base model architecture\.

Our contributions are:

1. 1\.We characterize the multilingual degradation caused by English\-calibrated INT3 GPTQ on two sub\-4B models \(Qwen2\.5\-3B, Llama\-3\.2\-3B\), quantifying per\-language perplexity ratios from 1\.35×\\times\(English\) to 4\.37×\\times\(Arabic\)\. While the broad phenomenon is known[11](https://arxiv.org/html/2608.11786#bib.bib1), we pin down its magnitude in the specific regime where aggressive quantization is most needed for deployment, motivating the need for language\-conditional correction\.
2. 2\.We propose LCD, a post\-hoc method that recovers 70–83% of the perplexity gap for non\-Latin\-script languages at 0\.12% parameter cost per language, without re\-quantization\.
3. 3\.We show that the per\-language conditioning captures real language\-specific signal: a language\-agnostic rank\-2 baseline matches per\-LCD on average but trails by 3–9 points on the typologically distant languages where per\-language corrections have the most signal to exploit\.
4. 4\.We reveal a*perplexity–accuracy disconnect*and trace it to a structural cause: a per\-layer error analysis shows that quantization concentrates damage in different network depths across models \(Qwen: late; Llama: early\-middle\)\. We*validate and probe*this mechanism: LCD restricted to Llama’s bottom half of transformer blocks outperforms uniform correction by 10 pp at half the parameter cost \(Table[6](https://arxiv.org/html/2608.11786#S5.T6)\); precision\-targeting of the exact worst\-error layers \(7\-\-14\) does not further improve over uniform, indicating that broad early\-depth coverage, rather than narrow layer allocation, is the operative mechanism\.111Code:[https://anonymous\.4open\.science/r/LCD\-2466](https://anonymous.4open.science/r/LCD-2466)

## 2Related Work

#### Multilingual Quantization Harm\.

[11](https://arxiv.org/html/2608.11786#bib.bib1)documented that quantization disproportionately degrades non\-English performance, particularly for non\-Latin\-script languages\.[1](https://arxiv.org/html/2608.11786#bib.bib3)found no such harm for k\-quantized Llama\-3\.3\-70B, but their 70B moderate\-quantization regime differs qualitatively from the sub\-4B INT3 setting where redundancy is scarce\.[2](https://arxiv.org/html/2608.11786#bib.bib2)addressed the root cause by using multilingual calibration data during GPTQ, but this requires re\-quantization and cannot be applied to already\-deployed models\. LCD is complementary: it operates post\-hoc\.

#### Quantization Error Correction\.

Prior work on low\-rank error correction splits on a single axis:*at\-quantization\-time*vs\.*post\-hoc*\. LRQ[10](https://arxiv.org/html/2608.11786#bib.bib10), OmniQuant[14](https://arxiv.org/html/2608.11786#bib.bib11), LQER[16](https://arxiv.org/html/2608.11786#bib.bib5), QERA[17](https://arxiv.org/html/2608.11786#bib.bib8), RILQ[9](https://arxiv.org/html/2608.11786#bib.bib6), and ResQ[13](https://arxiv.org/html/2608.11786#bib.bib7)all require access to the quantization pipeline and produce language\-agnostic corrections\. Recover\-LoRA[3](https://arxiv.org/html/2608.11786#bib.bib12)is the closest prior work: it trains LoRA adapters post\-hoc on a pre\-quantized model to recover lost accuracy\. LCD extends this post\-hoc paradigm with language\-*conditional*selection: each language receives a dedicated adapter that specializes to its quantization\-error profile\. To our knowledge, LCD is the only post\-hoc method that is also language\-conditional\.

#### Input\-Conditional Adaptation\.

BinaryMoS[8](https://arxiv.org/html/2608.11786#bib.bib4)uses token\-adaptive scaling for binarized models; MLAS\-LoRA[4](https://arxiv.org/html/2608.11786#bib.bib9)applies language\-aware LoRA for multilingual fine\-tuning \(not quantization repair\)\. LCD occupies the language\-level position on the input\-conditionality spectrum and targets a distinct problem: correcting systematic quantization error concentrated in specific language distributions\.

## 3Method

### 3\.1Language\-Conditional Correction

For each linear layer and each languageℓ\\ell, we define a rank\-rradditive correction:

ycorrected=Wq​x\+1r​Aℓ​Bℓ​xy\_\{\\text\{corrected\}\}=W\_\{q\}x\+\\frac\{1\}\{r\}A\_\{\\ell\}B\_\{\\ell\}x\(2\)whereAℓ∈ℝdout×rA\_\{\\ell\}\\in\\mathbb\{R\}^\{d\_\{\\text\{out\}\}\\times r\}andBℓ∈ℝr×dinB\_\{\\ell\}\\in\\mathbb\{R\}^\{r\\times d\_\{\\text\{in\}\}\}are learnable parameters and1/r1/ris a fixed scaling constant\. We initializeAℓ=𝟎A\_\{\\ell\}=\\mathbf\{0\}andBℓ∼𝒩⁡\(0,0\.01\)B\_\{\\ell\}\\sim\\mathcal\{N\}\(0,0\.01\), ensuring the correction is exactly zero at initialization\. This follows the LoRA parameterization[7](https://arxiv.org/html/2608.11786#bib.bib14)but serves a fundamentally different purpose: rather than adapting the model to a new task, we are*correcting systematic quantization error*for a specific language distribution\.

Corrections are implemented asforward\_hooks on each linear layer \(excludinglm\_headandembed\_tokens\), requiring no modification to the model architecture or forward pass\.

### 3\.2Training

Each language is trained independently\. Given a set ofN=256N=256text samples from the target language \(sourced from mC4/C4\), we minimize the standard language modeling loss with only the correction parameters\{Aℓ,Bℓ\}\\\{A\_\{\\ell\},B\_\{\\ell\}\\\}trainable:

ℒℓ=−∑tlogP\(xt\|x<t;Wq,Aℓ,Bℓ\)\\mathcal\{L\}\_\{\\ell\}=\-\\sum\_\{t\}\\log P\(x\_\{t\}\|x\_\{<t\};W\_\{q\},A\_\{\\ell\},B\_\{\\ell\}\)\(3\)
We use AdamW with learning rate5×10−45\\times 10^\{\-4\}, cosine decay with 10% warmup, weight decay 0\.01, gradient clipping at 1\.0, and train for 500 steps per language\. All base model parameters remain frozen\. Training takes approximately 15–20 minutes per language on a single L4 GPU, so covering all eight non\-English languages costs under 3 GPU\-hours in total\. Each trained correction ships as a∼\\sim7 MB delta on top of the*unchanged*deployed checkpoint; by contrast, re\-quantizing with multilingual calibration requires curating per\-language calibration data, re\-running the quantization pipeline, and redistributing the full multi\-gigabyte model artifact\.

### 3\.3Inference

At inference time, the input language is identified \(via locale, automatic detection, or explicit specification\) and the corresponding correction slot is activated via a single index\. In many deployments the language is already known from the user locale or product surface; when it is not, off\-the\-shelf language identification \(e\.g\., fastText LID\) classifies a prompt in well under a millisecond, negligible relative to a single decoding step of the LLM itself\. The base quantized model resides in memory once; language switching requires only updating the active adapter index, with no model reloading\. ForKKlanguages with rankr=2r=2, total overhead isK×0\.12%K\\times 0\.12\\%additional parameters\.

## 4Experimental Setup

#### Models\.

Qwen2\.5\-3B[12](https://arxiv.org/html/2608.11786#bib.bib16)and Llama\-3\.2\-3B[6](https://arxiv.org/html/2608.11786#bib.bib17), two sub\-4B families with distinct tokenizers and training distributions\.

#### Quantization\.

GPTQ[5](https://arxiv.org/html/2608.11786#bib.bib13)W3A16, group size 128, calibrated on 128 English C4 samples, replicating the standard English\-only deployment practice\.

#### Languages\.

9 languages: English \(baseline\), Arabic, Japanese, Chinese, Hindi, Russian, French, Spanish, Korean\.

#### Metrics\.

\(1\)*Perplexity*on held\-out mC4/C4 \(32 samples/language\); degradation ratio and recovery %\. \(2\)*GlobalMMLU accuracy*[15](https://arxiv.org/html/2608.11786#bib.bib15)via log\-likelihood over≈\\approx14,040 items per language \(57 subjects\)\.

#### Baselines\.

FP16 \(unquantized\), INT3 \(uncorrected\), INT3\+LCD \(corrected\)\. English is the calibration language and excluded from correction training\.

## 5Results

### 5\.1INT3 Quantization Disproportionately Harms Non\-English

Table[1](https://arxiv.org/html/2608.11786#S5.T1)presents the perplexity degradation ratios across languages; the underlying absolute perplexities for every condition are given in Appendix[A](https://arxiv.org/html/2608.11786#A1)\. On Qwen2\.5\-3B, English degrades by 1\.35×\\times, while Arabic degrades by 4\.37×\\times, Korean by 3\.49×\\times, and Japanese by 3\.04×\\times\(a relative disparity of 2\.3–3\.2×\\times\)\. The pattern is consistent on Llama\-3\.2\-3B, confirming that the effect is not model\-specific\. Languages with non\-Latin scripts and greater typological distance from English exhibit the largest degradation; Romance languages \(French, Spanish\) suffer only modest additional harm beyond English\.

Table 1:Perplexity degradation ratios after INT3 GPTQ quantization\. I/F = INT3/FP16 ratio \(mC4/C4 held\-out, 32 samples/lang\); “vs EN” is the ratio of each language’s degradation to English’s\.
### 5\.2LCD Recovers Most of the Perplexity Gap

Table[2](https://arxiv.org/html/2608.11786#S5.T2)shows the perplexity recovery results\. LCD recovers 70–83% of the quantization gap for non\-Latin\-script languages \(Arabic, Japanese, Chinese, Korean\) on both models\. Recovery is lower for Latin\-script languages close to English \(French 38%/54%, Spanish 36%/52%\), consistent with these languages suffering less language\-specific quantization error for LCD to correct\. These figures are robust to the random seed: retraining all corrections under three seeds and re\-evaluating on a fixed 256\-sample held\-out set changes per\-language recovery by at most±1\.3\\pm 1\.3percentage points \(standard deviation\), so the results are not an artifact of a single seed\.

LangQwenLlamaScriptArabic78\.880\.1ArabicJapanese69\.772\.9CJKChinese71\.276\.4CJKKorean74\.876\.7HangulHindi82\.668\.6DevanagariRussian57\.071\.7CyrillicFrench38\.253\.8LatinSpanish35\.652\.2LatinAvg \(non\-EN\)63\.569\.1Avg \(non\-Latin\)72\.474\.4Table 2:Perplexity recovery \(%\): fraction of the INT3→\\toFP16 gap closed by LCD\. Green:≥\\geq70%\. English is excluded from correction training as it is the calibration language\.
### 5\.3Language\-Specific vs\. Language\-Agnostic Correction

A natural question is whether per\-language conditioning is necessary, or whether a single shared rank\-2 LoRA trained on mixed multilingual data would suffice\. Table[3](https://arxiv.org/html/2608.11786#S5.T3)compares LCD against a language\-agnostic baseline \(same rank, same training budget, training data pooled across four target languages\) on Qwen2\.5\-3B\.

Table 3:Per\-language vs\. language\-agnostic rank\-2 LoRA on Qwen2\.5\-3B \(perplexity recovery %\)\. Averages are near\-identical, but per\-language wins by 3–9 points on typologically distant languages and loses 16 points on French, where no distinct per\-language signal exists\.Δ\\Delta= per\-language−\-agnostic \(percentage points\)\.On average the two methods are nearly identical \(65\.4% vs\. 64\.6%\), but the per\-language breakdown is revealing: the per\-language correction wins by 3–9 points on Arabic, Japanese, and Korean \(precisely the languages whose distribution is most distant from the English calibration set\) and loses 16 points on French\. This pattern confirms that per\-language conditioning captures genuine language\-specific signal where such signal exists in the data; on French, where the quantization error is closer to English’s, a shared correction trained on a multilingual mixture transfers better than a French\-specific one estimated from 256 samples of French text alone\. We interpret the near\-tie on average as a sharpening, not a weakening, of the central claim: LCD is a faithful correction of language\-specific quantization error, not a free lunch that helps every language\.

#### Comparison to data\-free error reconstruction \(LQER\)\.

A stronger, principled baseline is LQER[16](https://arxiv.org/html/2608.11786#bib.bib5), which reconstructs the quantization error with a rank\-rrtruncated SVD ofWfp16−WqW\_\{\\text\{fp16\}\}\-W\_\{q\}at each layer: low\-rank like LCD, but*data\-free*,*untrained*, and language\-agnostic\. At the identical rank\-2 budget on Qwen2\.5\-3B \(Table[4](https://arxiv.org/html/2608.11786#S5.T4)\), LQER recovers only 5\.5% of the perplexity gap and 13\.0% of the GlobalMMLU gap \(non\-English averages\), versus 63\.5% and 27\.8% for LCD; LQER is even negative on several languages \(e\.g\. Russian,−40%\-40\\%perplexity\)\. A static SVD of the weight error cannot capture the input\-dependent, language\-specific component that LCD learns from data\. This is the sharpest statement of our claim: at matched parameter budget, a*trained, language\-conditional*correction recovers an order of magnitude more of the perplexity gap than a*data\-free, agnostic*one\.

Table 4:LCD vs\. LQER[16](https://arxiv.org/html/2608.11786#bib.bib5)on Qwen2\.5\-3B at matched rank\-2 budget: non\-English average recovery \(%\) of the INT3→\\toFP16 gap, for perplexity \(PPL\) and GlobalMMLU \(MMLU\)\.

### 5\.4Recovery Correlates with Typological Distance

Recovery correlates with typological distance from English: non\-Latin\-script languages \(Arabic, Japanese, Chinese, Korean\) cluster at the top \(70–83%\), while Latin\-script languages close to English \(French, Spanish\) cluster at the bottom \(36–54%\), with Hindi and Russian in between\. This ordering is consistent across both models and confirms that LCD corrects calibration–distribution mismatch: the further a language’s activation statistics lie from English, the more concentrated the quantization error and the more LCD has to recover\.

### 5\.5Rank Ablation

We trained LCD corrections on Qwen2\.5\-3B at ranks 1, 2, and 4 \(Arabic and Japanese, 500 steps,lr=×10−4\\text\{lr\}=5\\\!\\times\\\!10^\{\-4\}\)\. The average recovery across the two languages was 76\.92% \(r=1\), 76\.98% \(r=2\), and 76\.84% \(r=4\)\. Rank is not the bottleneck: the gap from rank 1 to rank 4 is within 0\.1 points, indicating that the correction subspace is effectively one\-dimensional on these languages\. We adoptr=2r=2as a small safety margin; a rank\-1 deployment would halve parameter overhead to 0\.06% per language without measurable degradation\.

### 5\.6Downstream Transfer: The Perplexity–Accuracy Disconnect

Table[5](https://arxiv.org/html/2608.11786#S5.T5)presents GlobalMMLU accuracy across conditions\. INT3 quantization severely degrades accuracy on both models \(Qwen: 64\.7→\\to41\.8% English; Llama: 55\.5→\\to39\.9%\)\.

Table 5:GlobalMMLU accuracy \(%\) across nine languages\. Subscripts show binomial SE \(p⁡\(1−p\)/n\\sqrt\{p\(1\-p\)/n\},n≈14,040n\\\!\\approx\\\!14\{,\}040\)\. Significance vs\. INT3:†p<0\.05\{\}^\{\\dagger\}p\\\!<\\\!0\.05,‡p<0\.001\{\}^\{\\ddagger\}p\\\!<\\\!0\.001\(pairedtt\-test across 57 subjects, df = 56\)\.Bold: LCD gain≥\\geq5 pp over INT3\. Qwen closes 28% of the FP16→\\toINT3 gap \(non\-EN avg\); Llama closes 17%\.On Qwen2\.5\-3B, LCD delivers consistent accuracy gains across every non\-English language\. The largest improvements are Chinese \(\+\+9\.1 points\), Spanish \(\+\+6\.7\), Russian \(\+\+6\.2\), and Japanese \(\+\+6\.0\); Arabic, French, and Korean also gain between 3\.5 and 7\.8 points\. Across the eight non\-English languages, LCD closes an average of 28% of the FP16→\\toINT3 accuracy gap\.

On Llama\-3\.2\-3B, LCD also improves every non\-English language, but the gains are smaller, between 1\.1 points \(Chinese, Korean\) and 3\.5 points \(Russian\), closing an average of only 17% of the gap\. This*perplexity–accuracy disconnect*is the paper’s most scientifically informative finding: Llama’s perplexity recovery \(avg 69%\) exceeds Qwen’s \(avg 63%\), yet its MMLU recovery lags substantially\. Section[5\.7](https://arxiv.org/html/2608.11786#S5.SS7)diagnoses this gap using a third measurement, distinct from both perplexity and task accuracy: per\-layer activation error, which reveals*where*in the network quantization concentrates its damage and why that location determines whether perplexity recovery translates into downstream gains\.

### 5\.7Diagnosing the Disconnect: Where Quantization Hurts

To understand the perplexity–accuracy disconnect mechanistically, we measure the relative Frobenius error‖yfp16−yint3‖F/‖yfp16‖F\\\|y\_\{\\text\{fp16\}\}\-y\_\{\\text\{int3\}\}\\\|\_\{F\}/\\\|y\_\{\\text\{fp16\}\}\\\|\_\{F\}for every linear layer’s output activations, separately for each language\. Figure[1](https://arxiv.org/html/2608.11786#S5.F1)shows the resulting heatmaps\.

![Refer to caption](https://arxiv.org/html/2608.11786v1/figures/layerwise_qwen.png)

![Refer to caption](https://arxiv.org/html/2608.11786v1/figures/layerwise_llama.png)

Figure 1:Per\-layer relative INT3 quantization error by language\. Top: Qwen2\.5\-3B\. Bottom: Llama\-3\.2\-3B\. Each column is one linear layer; columns are sorted attention\-then\-MLP, shallow\-to\-deep within each block\. Color indicates‖yfp16−yint3‖F/‖yfp16‖F\\\|y\_\{\\text\{fp16\}\}\-y\_\{\\text\{int3\}\}\\\|\_\{F\}/\\\|y\_\{\\text\{fp16\}\}\\\|\_\{F\}averaged over 32 samples per language\. The worst\-error layers are the “compression” projections \(o\_proj,down\_proj\) in both models, but their*depth*differs: Qwen’s hotspots cluster in layers 18–30 \(out of 36\), while Llama’s cluster in layers 7–14 \(out of 28\)\.Two structural findings explain the disconnect\. First, the highest\-error layers in both models areo\_projanddown\_proj, the compression bottlenecks that project from a larger intermediate space back to the hidden dimension, where GPTQ’s per\-channel grid is least faithful\. Second, and more consequentially, the*depth*of these worst\-error layers differs sharply: Qwen2\.5\-3B’s top\-10 lie in indices 18–30 of 36 \(late\); Llama\-3\.2\-3B’s lie in indices 7–14 of 28 \(early\-middle\), with 14\+ downstream layers consuming their corrupted output\.

This depth asymmetry explains the disconnect directly\. Late\-layer errors \(Qwen\) primarily distort final next\-token logits, which perplexity measures and LCD fixes locally\. Early\-middle errors \(Llama\) distort intermediate representations that feed into downstream attention and MLP operations, exactly the multi\-step computation MMLU requires\. A rank\-2 hook corrects a layer’s immediate output but cannot undo propagated upstream corruption\. Quantization damage thus operates at two levels:*distributional*\(perplexity, correctable regardless of depth\) and*representational*\(downstream accuracy, correctable only when error is near the output\)\. The representational reading also predicts a scale effect: larger models carry more redundancy, so INT3 should inflict less representational damage and leave less for LCD to recover; our Qwen2\.5\-7B experiment \(Limitations\) bears this out, with non\-English MMLU recovery falling to 8\.5% and higher\-capacity corrections overfitting\.

#### Direct validation: targeting the worst\-error depth band\.

The depth\-asymmetry hypothesis predicts that, for each model, restricting LCD’s correction to the half of the network containing the worst\-error layers should match or exceed uniform correction; restricting it to the opposite half should underperform\. We test this by training LCD \(rankr=2r=2, identical hyperparameters\) under three layer\-coverage configurations on the four languages with the largest INT3 degradation: \(i\)alllayers \(the default\); \(ii\)bottom\-halfof transformer blocks \(covering Llama’s worst\-error band at indices 7–14\); \(iii\)top\-halfof transformer blocks \(covering Qwen’s worst\-error band at indices 18–30\)\. Table[6](https://arxiv.org/html/2608.11786#S5.T6)reports the resulting perplexity recovery\.

Table 6:Perplexity recovery \(%\) for LCD under three layer\-coverage configurations, all at rankr=2r=2and identical training\.bottom\-halfcovers Llama’s worst\-error band \(indices 7–14\) and recovers more than uniformallon every Llama language despite using half the parameters\. On Qwen, whose worst layers \(18–30\) span a larger fraction of the network,alldominates and the two halves are not differentiated\.On Llama,bottom\-halfexceeds uniformallby 10 pp and beatstop\-halfby 43 pp on average, obtaining better recovery at half the parameters\. On Qwen, the two halves are indistinguishable and both trailall, consistent with its worst\-error layers spanning a broad 40% band where any half\-subset misses some hotspots\. The depth asymmetry is visible precisely because Llama’s worst layers \(25% of the network\) are entirely contained in one half and absent from the other\.

Critically, this perplexity advantage does*not*reach downstream accuracy\. Evaluated on GlobalMMLU, Llama’sbottom\-halfLCD recovers only 9\.5% of the accuracy gap, versus 17% for uniformall\(the*reverse*of the perplexity ordering\)\. Concentrating capacity where the*distributional*error is largest \(the early compression band\) actively trades away*representational*recovery\. This double dissociation \(bottom\-half wins on perplexity but loses on MMLU\) is the strongest evidence that the two levels of quantization damage are governed by distinct mechanisms, and that perplexity recovery is an unreliable proxy for downstream gains\.

#### Negative result: narrow layer targeting does not improve further\.

We follow up by testing whether concentrating corrections*specifically*on layers 7–14 with additional rank yields further gains\. Three configurations are evaluated:uniform\-r2\(rank\-2, all layers\),targeted\-r4\(rank\-4, layers 7–14 only\), andtargeted\-r2\(rank\-2, layers 7–14 only\)\.uniform\-r2achieves 45\.0% average recovery; both targeted variants reach only 37%, andtargeted\-r4≈\\approxtargeted\-r2\(37\.0% vs\. 37\.1%\) rules out rank as an explanatory factor\. The bottom\-half advantage thus comes from broad coverage of the early depth band \(layers 0–13 as a whole\) rather than precise targeting of the identified hotspot\. When correction capacity is limited,*broad*early\-depth coverage dominates narrow layer allocation\.

## 6Conclusion

We have shown that English\-calibrated INT3 quantization creates a systematic, language\-dependent quality gap that disproportionately harms non\-English users\. LCD demonstrates that this gap is largely correctable: rank\-2 corrections at 0\.12% parameter cost per language recover 70–83% of the perplexity degradation for non\-Latin\-script languages across two model families, and 17–28% of the GlobalMMLU accuracy gap\. A language\-agnostic baseline with the same capacity matches per\-LCD on average but trails by 3–9 points on the typologically distant languages where language\-specific signal is concentrated: evidence that LCD is a faithful correction of language\-specific quantization error rather than a generic fine\-tune\.

Our analysis further reveals that perplexity recovery does not reliably translate to downstream accuracy: Llama’s higher perplexity recovery yields smaller MMLU gains than Qwen’s\. A per\-layer error analysis traces this to depth asymmetry: Llama’s worst\-error layers sit early\-middle in the network, propagating corruption downstream, while Qwen’s sit late\. Restricting LCD to Llama’s bottom half confirms this \(\+10 pp over uniform\), yet precision\-targeting of layers 7–14 specifically does not help further \(37% vs\. 45%\), establishing that broad early\-depth coverage, not narrow layer allocation, is the operative mechanism\.

The broader implication is practical:*quantization need not discriminate*\. With trivial overhead, deployed quantized models can serve non\-English users more equitably\.

## Limitations

#### Language identification at inference\.

LCD requires the input language to be known at inference time so that the correct per\-language adapter is activated\. In controlled deployment \(e\.g\., a localized product surface with known user locale\) this is straightforward, but code\-switched inputs, low\-quality language ID, or mixed\-language prompts are not directly handled by the current design\. A soft gating mechanism over adapters, or falling back to the language\-agnostic adapter when confidence is low, is a natural extension but is not evaluated here\.

#### Residual MMLU gap on early\-error architectures\.

Section[5\.7](https://arxiv.org/html/2608.11786#S5.SS7)attributes LCD’s smaller MMLU gains on Llama\-3\.2\-3B to the depth of its worst\-error layers \(indices 7–14 of 28\), whose corrupted outputs propagate through many downstream blocks\. We evaluate a layer\-targeted variant that concentrates rank\-4 corrections exclusively on these layers, but it*underperforms*uniform rank\-2 correction \(37% vs\. 45% perplexity recovery\), indicating that the MMLU gap reflects a structural representational deficit that narrow layer targeting cannot overcome\. Allocating higher rank specifically to the compression projections \(o\_proj,down\_proj\), which carry the highest per\-layer error in both models, rather than to entire transformer blocks, remains an open direction\.

#### Narrow quantization regime\.

We study INT3 GPTQ with W3A16, group size 128, English C4 calibration\. Other quantization schemes \(AWQ, GGUF k\-quants, SmoothQuant\), other bit\-widths \(INT2, INT4, FP4\), and other calibration corpora may change both the magnitude of the language\-specific harm and the degree to which a rank\-2 correction suffices\. We expect the qualitative story to hold but make no claim beyond the evaluated setting\.

#### Model scale and family coverage\.

Our main evaluation uses two sub\-4B\-parameter models \(Qwen2\.5\-3B, Llama\-3\.2\-3B\)\. To probe scale, we additionally quantized Qwen2\.5\-7B to INT3 and trained LCD corrections, evaluating on GlobalMMLU\. The result is the opposite of a clean scale\-up: at 7B, rank\-2 LCD recovers only 8\.5% of the non\-English accuracy gap \(vs\. 28% at 3B\), and raising capacity to rank\-4 with 1000 steps makes*every*language worse \(average−3\.9%\-3\.9\\%\), as the corrections overfit the 256\-sample training slice\. Crucially, the languages quantization damages most still recover at 7B \(Korean\+4\.7\+4\.7pts, 26\.5%; Arabic\+2\.4\+2\.4pts, 17\.0%\), while languages with small INT3 gaps \(Russian, Spanish, French\) go flat or slightly negative\. We read this as consistent with, not counter to, our central claim: a larger model carries more redundancy, so INT3 inflicts less*representational*damage on high\-resource languages, leaving little language\-specific error for a correction to recover, and additional capacity then fits noise\. LCD’s downstream benefit is therefore damage\-dependent and does not automatically grow with scale; characterizing exactly where on the \(scale×\\timesbit\-width\) plane the benefit persists is open\. We make no claim about MoE architectures or instruction\-tuned variants\.

#### Language coverage and data\.

Perplexity recovery is measured on 32 mC4/C4 held\-out samples per language, and MMLU on the GlobalMMLU test set \(≈\\approx14,040 items per language\)\. The eight non\-English languages we evaluate span four scripts, but they are all relatively high\-resource\. Recovery behavior on truly low\-resource languages, on dialectal variation, and on domain\-shifted text \(legal, medical, code\-mixed\) is an open question\.

#### Adapter training data\.

Each per\-language adapter is trained on 256 samples of monolingual text from that language for 500 steps\. This is deliberately small to match the “post\-hoc, cheap” framing, but it also means the correction is estimated from a narrow slice of each language’s distribution\. Larger and more diverse per\-language data may shift the per\-language vs\. agnostic comparison, particularly for languages like French where the agnostic baseline currently wins\.

#### Ethical and fairness considerations\.

LCD is intended to*reduce*a fairness gap introduced by English\-centric calibration\. However, because the correction is trained on web\-scraped multilingual text, it inherits any biases and quality issues present in that corpus\. We do not audit the adapters for language\-specific toxicity, factuality, or stereotype changes introduced by the correction itself; such an audit is necessary before deploying LCD in user\-facing settings\.

## Ethics Statement

#### Societal benefit\.

English\-calibrated INT3 quantization imposes a systematic quality penalty on non\-English users: perplexity degrades 2\.6–3\.2×\\timesmore for Arabic and Korean than for English on the same model, with no corresponding reduction in model size for those users\. LCD is designed to reduce this disparity\. Because corrections train on 256 samples in under 20 minutes on a single consumer GPU, the method is accessible to researchers without large\-scale compute resources\. Because it operates post\-hoc on already\-quantized weights, it offers a practical correction path for deployed models where re\-quantization is not an option\.

#### Known limitations and risks\.

LCD requires the input language to be identified at inference time\. Misidentification activates the wrong per\-language correction; because the adapters are small \(0\.12% of parameters, initialized near zero\), the magnitude of harm from a mismatched adapter is bounded, but it is not zero and is not characterized in this work\. Each adapter is trained on web\-scraped monolingual text \(C4 or mC4\)\. These corpora carry societal biases, quality skew, and domain imbalance\. We do not audit whether the correction process amplifies or introduces language\-specific toxicity, stereotyping, or factual error; such an audit is necessary before deploying LCD in user\-facing systems\. Our evaluation covers eight non\-English languages, all relatively high\-resource; generalization to endangered or very\-low\-resource languages is untested\. Finally, practitioners should not treat perplexity recovery as a proxy for full task recovery: on Llama\-3\.2\-3B, 69% average perplexity recovery translates to only 17% MMLU gap recovery\.

#### Data and compute\.

All training and evaluation data \(C4, mC4, GlobalMMLU\) are publicly available\. No human subjects were involved and no personally identifiable information was used\. Total compute for the experiments reported in this paper was approximately 42 GPU\-hours on L4/L40S hardware\.

## References

- Borgersen and Goodwin \(2025\)N\. Borgersen and M\. GoodwinEnglish k\-quantization does not disproportionately diminish multilingual performance\.Note:arXiv preprint arXiv:2503\.03592Cited by:[§2](https://arxiv.org/html/2608.11786#S2.SS0.SSS0.Px1.p1.1)\.
- Chimotoet al\.\(2026\)E\. A\. Chimoto, M\. Elhoushi, and B\. BassettCalibrating beyond English: addressing multilingual bias in quantization\.InProceedings of EACL,Cited by:[§1](https://arxiv.org/html/2608.11786#S1.p2.1),[§2](https://arxiv.org/html/2608.11786#S2.SS0.SSS0.Px1.p1.1)\.
- Daset al\.\(2025\)D\. Das, R\. Patwari, and A\. SirasaoRecover\-LoRA: data\-free accuracy recovery of degraded language models via low\-rank adaptation\.InProceedings of EMNLP \(Industry Track\),Note:arXiv:2510\.08600Cited by:[§2](https://arxiv.org/html/2608.11786#S2.SS0.SSS0.Px2.p1.1)\.
- Donget al\.\(2025\)T\. Dong, B\. Li, J\. Liu, S\. Zhu, and D\. XiongMLAS\-LoRA: language\-aware parameters detection and LoRA\-based knowledge transfer for multilingual machine translation\.InProceedings of ACL,Cited by:[§2](https://arxiv.org/html/2608.11786#S2.SS0.SSS0.Px3.p1.1)\.
- Frantaret al\.\(2023\)E\. Frantar, S\. Ashkboos, T\. Hoefler, and D\. AlistarhGPTQ: accurate post\-training quantization for generative pre\-trained transformers\.ICLR\.Cited by:[§4](https://arxiv.org/html/2608.11786#S4.SS0.SSS0.Px2.p1.1)\.
- Grattafioriet al\.\(2024\)A\. Grattafioriet al\.The Llama 3 herd of models\.Note:arXiv preprint arXiv:2407\.21783Cited by:[§4](https://arxiv.org/html/2608.11786#S4.SS0.SSS0.Px1.p1.1)\.
- Huet al\.\(2022\)E\. J\. Hu, Y\. Shen, P\. Wallis, Z\. Allen\-Zhu, Y\. Li, S\. Wang, L\. Wang, and W\. ChenLoRA: low\-rank adaptation of large language models\.ICLR\.Cited by:[§3\.1](https://arxiv.org/html/2608.11786#S3.SS1.p1.2)\.
- Joet al\.\(2024\)D\. Jo, T\. Kim, Y\. Kim, and J\. KimMixture of scales: memory\-efficient token\-adaptive binarization for large language models\.InNeurIPS,Note:arXiv:2406\.12311Cited by:[§1](https://arxiv.org/html/2608.11786#S1.p2.1),[§2](https://arxiv.org/html/2608.11786#S2.SS0.SSS0.Px3.p1.1)\.
- Leeet al\.\(2025a\)G\. Lee, J\. Lee, S\. Hong, M\. Kim, E\. Ahn, D\. Chang, and J\. ChoiRILQ: rank\-insensitive LoRA\-based quantization error compensation for boosting 2\-bit large language model accuracy\.InAAAI,Note:arXiv:2412\.01129Cited by:[§1](https://arxiv.org/html/2608.11786#S1.p2.1),[§2](https://arxiv.org/html/2608.11786#S2.SS0.SSS0.Px2.p1.1)\.
- Leeet al\.\(2025b\)J\. H\. Lee, J\. Kim, J\. Y\. Yang, S\. J\. Kwon, E\. Yang, K\. M\. Yoo, and D\. LeeLRQ: optimizing post\-training quantization for large language models by learning low\-rank weight\-scaling matrices\.InProceedings of NAACL\-HLT,Note:arXiv:2407\.11534Cited by:[§2](https://arxiv.org/html/2608.11786#S2.SS0.SSS0.Px2.p1.1)\.
- Marchisioet al\.\(2024\)K\. Marchisio, S\. Dash, H\. Chen, D\. Aumiller, A\. Üstün, S\. Hooker, and S\. RuderHow does quantization affect multilingual LLMs?\.InFindings of EMNLP,Note:arXiv:2407\.03211Cited by:[item 1](https://arxiv.org/html/2608.11786#S1.I1.i1.p1.1),[§2](https://arxiv.org/html/2608.11786#S2.SS0.SSS0.Px1.p1.1)\.
- Qwen Team \(2024\)Qwen TeamQwen2\.5 technical report\.Note:arXiv preprint arXiv:2412\.15115Cited by:[§4](https://arxiv.org/html/2608.11786#S4.SS0.SSS0.Px1.p1.1)\.
- Saxenaet al\.\(2025\)U\. Saxena, S\. Sharify, K\. Roy, and X\. WangResQ: mixed\-precision quantization of large language models with low\-rank residuals\.InICML,Note:arXiv:2412\.14363Cited by:[§1](https://arxiv.org/html/2608.11786#S1.p2.1),[§2](https://arxiv.org/html/2608.11786#S2.SS0.SSS0.Px2.p1.1)\.
- Shaoet al\.\(2024\)W\. Shao, M\. Chen, Z\. Zhang, P\. Xu, L\. Zhao, Z\. Li, K\. Zhang, P\. Gao, Y\. Qiao, and P\. LuoOmniQuant: omnidirectionally calibrated quantization for large language models\.InICLR,Note:arXiv:2308\.13137Cited by:[§2](https://arxiv.org/html/2608.11786#S2.SS0.SSS0.Px2.p1.1)\.
- Singhet al\.\(2024\)S\. Singh, A\. Romanou, C\. Fourrier,et al\.Global MMLU: understanding and addressing cultural and linguistic biases in multilingual evaluation\.Note:arXiv preprint arXiv:2412\.03304Cited by:[§4](https://arxiv.org/html/2608.11786#S4.SS0.SSS0.Px4.p1.1)\.
- Zhanget al\.\(2024\)C\. Zhang, J\. Cheng, G\. A\. Constantinides, and Y\. ZhaoLQER: low\-rank quantization error reconstruction for LLMs\.InICML,Note:arXiv:2402\.02446Cited by:[§1](https://arxiv.org/html/2608.11786#S1.p2.1),[§2](https://arxiv.org/html/2608.11786#S2.SS0.SSS0.Px2.p1.1),[§5\.3](https://arxiv.org/html/2608.11786#S5.SS3.SSS0.Px1.p1.1),[Table 4](https://arxiv.org/html/2608.11786#S5.T4)\.
- Zhanget al\.\(2025\)C\. Zhang, J\. T\. H\. Wong, C\. Xiao, G\. A\. Constantinides, and Y\. ZhaoQERA: an analytical framework for quantization error reconstruction\.InICLR,Note:arXiv:2410\.06040Cited by:[§2](https://arxiv.org/html/2608.11786#S2.SS0.SSS0.Px2.p1.1)\.

## Appendix AAbsolute Perplexities

Tables[1](https://arxiv.org/html/2608.11786#S5.T1)and[2](https://arxiv.org/html/2608.11786#S5.T2)report degradation ratios and recovery percentages to normalize across languages with very different base perplexities\. Table[7](https://arxiv.org/html/2608.11786#A1.T7)provides the underlying absolute values \(mC4/C4 held\-out, 32 samples per language\) for all three conditions\.

Table 7:Absolute perplexity per language under each condition \(mC4/C4 held\-out, 32 samples/language\)\. English receives no correction, as it is the calibration language\.