ExTernD: Expanded-Rank Ternary Decomposition Ternary LLM PTQ with Accuracy Approaching Any Quantization Level

# Expanded-Rank Ternary Decomposition Ternary LLM PTQ with Accuracy Approaching Any Quantization Level
Source: [https://arxiv.org/html/2607.13511](https://arxiv.org/html/2607.13511)
\(July 2026 \(early preprint\)\)

###### Abstract

We introduceExTernD\(Expanded\-rank Ternary Decomposition\), a post\-training factorization of each LLM weight matrixA∈ℝm×nA\\in\\mathbb\{R\}^\{m\\times n\}intoA≈B​diag⁡\(D\)​CA\\approx B\\,\\operatorname\{diag\}\(D\)\\,Cwith ternary factorsB∈\{−1,0,\+1\}m×kB\\in\\\{\-1,0,\+1\\\}^\{m\\times k\},C∈\{−1,0,\+1\}k×nC\\in\\\{\-1,0,\+1\\\}^\{k\\times n\}and a real scale vectorD∈ℝkD\\in\\mathbb\{R\}^\{k\}\. The*inner rank*k=μ​min⁡\(m,n\)k=\\mu\\min\(m,n\)is deliberately expanded beyond full rank \(μ\>1\\mu\>1\), so that components past full rank correct the quantization error of earlier ones\. We prove the residual decreases monotonically inkkand can be driven below anyε\>0\\varepsilon\>0: ExTernD approaches bf16 accuracy arbitrarily closely, which no ternary scheme with a fixed plane count can do\. Memory and compute scale continuously withμ\\mu, and factor sparsity continuously with a thresholdτ\\tau, so an accuracy target is hit exactly rather than rounded to the next bit\-width\. ExTernD matches Q4\_K’s per\-matrix accuracy at 5\.2–5\.5 effective bpw \(5\.1–5\.5 with importance weighting\) on Gemma\-4\-E2B and Qwen3\.5\-4B, and a full Qwen3\.5\-4B conversion atμ=3\\mu=3reaches 10\.10 wikitext\-2 perplexity against 9\.78 for bf16 \(\+3\.2%\+3\.2\\%\), placing it near the Q4\_K/Q5\_K accuracy band at∼5\.7\{\\sim\}5\.7effective bpw\.

## 1Introduction

Ternary weights are attractive: matrix multiplication degenerates into additions and subtractions, and storage approacheslog2⁡3≈1\.58\\log\_\{2\}3\\approx 1\.58bits per weight\. BitNet\[[1](https://arxiv.org/html/2607.13511#bib.bib1),[2](https://arxiv.org/html/2607.13511#bib.bib2)\]shows that 1\-bit and 1\.58\-bit LLMs can be*trained from scratch*; ternary weight networks\[[3](https://arxiv.org/html/2607.13511#bib.bib3)\]and trained ternary quantization\[[4](https://arxiv.org/html/2607.13511#bib.bib4)\]established the format earlier for vision models\. But a single ternary plane carries at most 1\.58 bits of information per weight, so*post\-training*projection of a full\-precision matrix onto one ternary plane loses too much: reasoning collapses\[[5](https://arxiv.org/html/2607.13511#bib.bib5)\]\.

PTQTP\[[5](https://arxiv.org/html/2607.13511#bib.bib5)\]takes the important step of using*two*trit\-planes, a superposition of two ternary matrices with row\-wise scales, and shows this restores much of the lost expressiveness without retraining\. But two planes is still a fixed capacity budget with a ceiling it cannot get past, and there is nothing special about two, or about any fixed number of planes\. ExTernD replaces a fixed count of full\-size planes with a*factored*representation whose inner dimensionkkis a free parameter:A≈B​diag⁡\(D\)​CA\\approx B\\,\\operatorname\{diag\}\(D\)\\,Cwith both factors ternary\. Settingk=μ​min⁡\(m,n\)k=\\mu\\min\(m,n\)\(typicallyμ=2\\mu=2–3\.53\.5, but unbounded\) gives the representation as much capacity as needed; each additional component fits the residual left by all previous ones, so capacity is spent exactly where the ternary constraint hurt and accuracy is not capped \(Sec\.[2\.1](https://arxiv.org/html/2607.13511#S2.SS1), proof in Appendix[A](https://arxiv.org/html/2607.13511#A1)\)\. All of this is strictly post\-training: the factorization sees one weight matrix at a time and never a gradient\.

Mainstream post\-training quantization \(GPTQ\[[7](https://arxiv.org/html/2607.13511#bib.bib7)\], AWQ\[[8](https://arxiv.org/html/2607.13511#bib.bib8)\], llama\.cpp k\-quants\) sits at a small number of discrete accuracy/cost points: 3, 4, 5 bits per weight\. Becausekkis a real\-valued rank rather than a bit\-width, ExTernD’s accuracy/cost trade\-off is instead continuous: the inner\-rank multiplierμ\\mu\(storage and compute both scale linearly with it, Sec\.[2\.6](https://arxiv.org/html/2607.13511#S2.SS6)\) and a sparsity thresholdτ\\tau\(which sets the fraction of zeros inBBandCC, e\.g\.τ=0\.7→∼41%\\tau\{=\}0\.7\\to\{\\sim\}41\\%zeros,τ=1\.0→∼57%\\tau\{=\}1\.0\\to\{\\sim\}57\\%,τ=2\.0→∼87%\\tau\{=\}2\.0\\to\{\\sim\}87\\%\) can each be set per matrix to any real value, letting a target accuracy be hit exactly rather than rounded up to the next bit\-width\.

Contributions: \(i\) ExTernD and a greedy ALS algorithm for it, with a proof \(Appendix[A](https://arxiv.org/html/2607.13511#A1)\) that its error falls below anyε\>0\\varepsilon\>0asμ\\mugrows, making itto our knowledge the only ternary quantization that can provably reach any target accuracy, up to bf16, and hence beat any fixed\-plane\-count method at large enoughμ\\mu; \(ii\) a batched block\-ALS GPU algorithm with decorrelated targets, matching sequential quality at∼16×\{\\sim\}16\\timesspeed; \(iii\) an importance\-weighted variant using llama\.cpp imatrix statistics, including a non\-obvious correction to the V\-step; \(iv\) an empirical study on Gemma\-4\-E2B, Qwen3\.5\-4B and Granite\-4\.0\-h\-tiny, including a full end\-to\-end model conversion\.

## 2Method

### 2\.1Expanded inner rank

GivenA∈ℝm×nA\\in\\mathbb\{R\}^\{m\\times n\}, we seek

A≈A^=B​diag⁡\(D\)​C,B∈\{−1,0,1\}m×k,C∈\{−1,0,1\}k×n,D∈ℝk\.A\\;\\approx\\;\\hat\{A\}\\;=\\;B\\,\\operatorname\{diag\}\(D\)\\,C,\\qquad B\\in\\\{\-1,0,1\\\}^\{m\\times k\},\\;C\\in\\\{\-1,0,1\\\}^\{k\\times n\},\\;D\\in\\mathbb\{R\}^\{k\}\.\(1\)Quality is reported asenergy preserved,E=1−‖A−A^‖F2/‖A‖F2E=1\-\\\|A\-\\hat\{A\}\\\|\_\{F\}^\{2\}/\\\|A\\\|\_\{F\}^\{2\}\.

With real factors,k=min⁡\(m,n\)k=\\min\(m,n\)suffices \(SVD\)\. With ternary factors each rank\-one componentdi​bi​ci⊤d\_\{i\}\\,b\_\{i\}c\_\{i\}^\{\\top\}is a crude, quantized object; atk=min⁡\(m,n\)k=\\min\(m,n\)the representation hits an information ceiling well short of full precision\. The core idea is to keep adding components*past*full rank: componentiiis fitted to the residualR=A−∑j<idj​bj​cj⊤R=A\-\\sum\_\{j<i\}d\_\{j\}b\_\{j\}c\_\{j\}^\{\\top\}, i\.e\. to the accumulated quantization error of its predecessors\. The multiplierμ=k/min⁡\(m,n\)\\mu=k/\\min\(m,n\)becomes the accuracy dial\. Crucially this only works through*sequential deflation*: solving for allk\>min⁡\(m,n\)k\>\\min\(m,n\)components jointly is underdetermined and fails completely \(0–50% energy in our ablation, vs\. 96–100% with deflation\)\.

Becauseμ\\muis unbounded and the residual is monotone in it \(Appendix[A](https://arxiv.org/html/2607.13511#A1)\), the accuracy ceiling that a fixed plane count imposes does not exist here: any target accuracy is reachable by construction rather than by luck of the fit\.

### 2\.2Ternarization operator and continuous sparsity

Foru∈ℝmu\\in\\mathbb\{R\}^\{m\}, the adaptive mean\-threshold ternarization with scaleτ\\tauis

Tτ​\(u\)i=sign⁡\(ui\)⋅𝟏​\[\|ui\|\>τ⋅1m​∑j\|uj\|\],T\_\{\\tau\}\(u\)\_\{i\}=\\operatorname\{sign\}\(u\_\{i\}\)\\cdot\\mathbf\{1\}\\\!\\Big\[\\,\|u\_\{i\}\|\>\\tau\\cdot\\tfrac\{1\}\{m\}\\textstyle\\sum\_\{j\}\|u\_\{j\}\|\\,\\Big\],\(2\)keeping the single largest entry if the result would be all\-zero\.τ\\taucontrols factor sparsity continuously and almost matrix\-independently\. The optimal ternary projectionarg⁡mint,α⁡‖u−α​t‖2\\arg\\min\_\{t,\\alpha\}\\\|u\-\\alpha t\\\|\_\{2\}has a closed form \(sort\|u\|\|u\|, keep the top\-s⋆s^\{\\star\}prefix maximizing1s​∑i≤s\|u\|\(i\)\\frac\{1\}\{\\sqrt\{s\}\}\\sum\_\{i\\leq s\}\|u\|\_\{\(i\)\}\) but empirically tiesT0\.7T\_\{0\.7\}, so we use the cheaper threshold form\.

### 2\.3Greedy sequential ALS

With residualR←AR\\leftarrow A, fori=1,…,ki=1,\\dots,k:

1. 1\.Alternating ternary fit\(15 iterations\):v←Tτ​\(R⊤​u\)v\\leftarrow T\_\{\\tau\}\(R^\{\\top\}u\),u←Tτ​\(R​v\)u\\leftarrow T\_\{\\tau\}\(Rv\)\.
2. 2\.Optimal scale:di=u⊤​R​v‖u‖22​‖v‖22d\_\{i\}=\\dfrac\{u^\{\\top\}Rv\}\{\\\|u\\\|\_\{2\}^\{2\}\\,\\\|v\\\|\_\{2\}^\{2\}\}, the least\-squares minimizer of‖R−d​u​v⊤‖F\\\|R\-d\\,uv^\{\\top\}\\\|\_\{F\}\.
3. 3\.Deflate:R←R−di​u​v⊤R\\leftarrow R\-d\_\{i\}\\,uv^\{\\top\}; storeB:,i=uB\_\{:,i\}=u,Ci,:=v⊤C\_\{i,:\}=v^\{\\top\}\.

Step 2 guarantees‖R‖F2\\\|R\\\|\_\{F\}^\{2\}never increases \(Appendix[A](https://arxiv.org/html/2607.13511#A1)\), which is what makesμ\\mua well\-behaved dial: more components monotonically means less error, all the way to exact recovery\.

### 2\.4Batched block ALS \(GPU\)

Extracting components in blocks ofbb\(default 256, capped at⌊min⁡\(m,n\)/8⌋\\lfloor\\min\(m,n\)/8\\rfloor\) turns per\-component vector work into block matmuls\. The naive batching, ternarizingR⊤​UR^\{\\top\}Ucolumn\-wise, collapses \(99\.5%→\\to50% energy atb=256b\{=\}256\): every column chases the same dominant residual direction\. The fix is to ternarize the*jointly decorrelated least\-squares target*:

V←Tτ​\(\[\(U⊤​U\+ϵ​I\)−1​U⊤​R\]⊤\),U←Tτ​\(R​V​\(V⊤​V\+ϵ​I\)−1\),V\\leftarrow T\_\{\\tau\}\\\!\\Big\(\\big\[\(U^\{\\top\}U\+\\epsilon I\)^\{\-1\}U^\{\\top\}R\\big\]^\{\\top\}\\Big\),\\qquad U\\leftarrow T\_\{\\tau\}\\\!\\Big\(R\\,V\\,\(V^\{\\top\}V\+\\epsilon I\)^\{\-1\}\\Big\),\(3\)so each component is fitted against the residual minus its block\-mates’ contributions \(implicit within\-block deflation; an exact block solve in place of Gauss–Seidel\)\. After the alternating iterations, allbbscales are solved jointly and exactly:

\[\(U⊤​U\)⊙\(V⊤​V\)\]​d=diag⁡\(U⊤​R​V\),\\big\[\(U^\{\\top\}U\)\\odot\(V^\{\\top\}V\)\\big\]\\,d=\\operatorname\{diag\}\(U^\{\\top\}RV\),\(4\)thenR←R−U​diag⁡\(d\)​V⊤R\\leftarrow R\-U\\operatorname\{diag\}\(d\)V^\{\\top\}\. Deflation*across*blocks stays fully sequential\. Block width must respectb≲min⁡\(m,n\)/8b\\lesssim\\min\(m,n\)/8or the ternary Gram matrices become ill\-conditioned\.

Refinement sweeps\.After extraction, revisiting each block \(add its contribution back toRR, re\-run the block fit, re\-deflate\) improves energy monotonically:≈\+0\.4\{\\approx\}\{\+\}0\.4points on hard matrices after 10 sweeps, and a 4–8% reduction in the multiplier needed for fixed energy \(larger at highτ\\tau\)\.

### 2\.5Importance\-weighted ALS

Frobenius error weights all input channels equally; activations do not\. Given per\-input\-channel second momentshhfrom a llama\.cpp importance matrix \(imatrix\), we minimize∑i​jh~j​\(Ai​j−A^i​j\)2\\sum\_\{ij\}\\tilde\{h\}\_\{j\}\\,\(A\_\{ij\}\-\\hat\{A\}\_\{ij\}\)^\{2\}withh~=h/max⁡\(h\)\+λ\\tilde\{h\}=h/\\max\(h\)\+\\lambda\. The U\-step and the joint scale solve generalize directly \(W=R⊙h~W=R\\odot\\tilde\{h\}as target,V⊤​diag⁡\(h~\)​VV^\{\\top\}\\operatorname\{diag\}\(\\tilde\{h\}\)VGram matrices\); they are the exact weighted normal equations\. The V\-step does*not*: solving it againstWWloses 0\.3–1\.8 points of weighted energy versus the unweighted algorithm, because theh~\\tilde\{h\}rescaling starves low\-importance channels of support under a threshold that spans the whole component\. In a column\-weighted objectiveh~\\tilde\{h\}cancels in the V\-step, so the correct V\-target is the plain residualRR\.λ\\lambdainterpolates continuously between the uniform \(λ→∞\\lambda\\to\\infty\) and pure\-imatrix \(λ=0\\lambda=0\) objectives;λ=0\\lambda=0matches what llama\.cpp imatrix quants optimize and is stable in practice\.

### 2\.6Cost model

Inference computesy=B​\(diag⁡\(D\)​\(C​x\)\)y=B\(\\operatorname\{diag\}\(D\)\(Cx\)\):k​\(m\+n\)k\(m\+n\)ternary add/subtracts pluskkmultiplies, versusm​nmnmultiply\-adds, a ratio ofμ​\(m\+n\)/max⁡\(m,n\)\\mu\\,\(m\{\+\}n\)/\\max\(m,n\)that is linear \(hence continuous\) inμ\\mu\. With sparse mask\+sign packing \(a 1\-bit zero/nonzero mask per stored element plus one sign bit per nonzero,bpw=2−sparsity\\mathrm\{bpw\}=2\-\\mathrm\{sparsity\}\), storage per*original*weight is

bpweff=μ⋅m\+nmax⁡\(m,n\)⋅\(2−sparsity\),\\mathrm\{bpw\}\_\{\\mathrm\{eff\}\}=\\mu\\cdot\\frac\{m\+n\}\{\\max\(m,n\)\}\\cdot\(2\-\\mathrm\{sparsity\}\),\(5\)again linear inμ\\muand continuous inτ\\tauthrough the sparsity\. This packing is near the entropy floor for unstructured supports \(mask entropyH​\(0\.43\)≈0\.99H\(0\.43\)\\approx 0\.99bits; signs incompressible\)\.

## 3Results

Setup: matrices from google/gemma\-4\-E2B, Qwen/Qwen3\.5\-4B, and IBM Granite\-4\.0\-h\-tiny\. Every experiment in this paper \(decompositions, multiplier searches, the full\-model conversion, and all perplexity evaluations\) was run on a single AMD MI50 32 GB GPU \(torch/ROCm\)\. The Q4\_K baseline is a faithful torch port of llama\.cpp’squantize\_row\_q4\_K\_ref\(4\.5 bpw\)\.

### 3\.1Energy vs\. multiplier; batched = sequential

Table 1:Energy preserved \(%\) atτ=0\.7\\tau=0\.7\. Attention projections need much less inner rank than MLPs\. The batched algorithm \(Sec\.[2\.4](https://arxiv.org/html/2607.13511#S2.SS4)\) matches the sequential one within±0\.2\\pm 0\.2points on every matrix while running the six\-matrix suite in 10\.5 s vs\. 170\.9 s \(∼16×\{\\sim\}16\\times\)\.On Qwen3\.5\-4B atμ=2\.5\\mu\{=\}2\.5,τ=0\.7\\tau\{=\}0\.7, 10 sweeps: MLPs 99\.21–99\.26%, v\_proj 99\.52%, o\_proj 99\.78%\. Same algorithm, easier model; achievable energy at fixedμ\\muis strongly model\-dependent\.

### 3\.2The two dials interact favorably

Sweepingτ\\tauat a fixed 99%\-energy target: sparsity rises continuously \(41% atτ=0\.7\\tau\{=\}0\.7to 87% atτ=2\.0\\tau\{=\}2\.0\) while the requiredμ\\murises \(up\_proj: 2\.82→\\to7\.21\)\. The*nonzero budget*μ×\\mu\\timesdensity falls monotonically \(1\.65→\\to0\.87\): sparser factors with more components need fewer total nonzeros for equal energy, up to a convergence cliff nearτ=2\.5\\tau=2\.5\. Under mask\+sign packing \(Eq\.[5](https://arxiv.org/html/2607.13511#S2.E5)\), where zeros also cost a mask bit, the optimum is interior atτ=1\.0\\tau=1\.0\(∼\\sim57% sparsity\)\.

### 3\.3Effective bits at matched Q4\_K accuracy

Q4\_K achieves a strikingly uniform 99\.4–99\.5% energy on every matrix of both models\. Choosingμ\\muper matrix to match it exactly:

Table 2:Effective bits per original weight at Q4\_K\-matched energy, no sweeps\. The∼20%\{\\sim\}20\\%gap is uniform across shapes and*models*: Qwen is easier in absolute energy, but Q4\_K improves by the same margin, so the gap is a property of the algorithm, not the test model\. Attention consistently needs far less rank \(μ≈2\.0\\mu\\approx 2\.0–2\.72\.7\) than MLPs \(≈3\.0\\approx 3\.0–3\.43\.4\)\.Because the packing is near the entropy floor, closing this gap is a*multiplier*problem, not a coding problem: parity requires∼17%\{\\sim\}17\\%lower iso\-qualityμ\\mu\. Refinement sweeps give∼5%\{\\sim\}5\\%; importance weighting \(next\) gives another∼4%\{\\sim\}4\\%independently\.

### 3\.4Importance weighting

With the corrected V\-step \(Sec\.[2\.5](https://arxiv.org/html/2607.13511#S2.SS5)\) atλ=0\\lambda=0, weighting helps most where the imatrix is skewed: on Granite\-4\.0\-h\-tiny, v\_proj weighted energy 96\.55%→\\to98\.95% \(\+2\.40\+2\.40\); flat\-imatrix tensors are unchanged\. On Qwen3\.5\-4B, the rank multiplier needed for 99\.5% weighted energy drops 2–8% on MLP gate/up and v\_proj, and 4\.5–7\.4% on gated\-DeltaNet projections, the best territory, since their qkv/out input importance is strongly skewed\. Redoing the iso\-Q4\_K accounting under the weighted metric: mean bpweff5\.49→\\to5\.26 \(−4\.2%\-4\.2\\%\), best tensors 5\.08–5\.14, shrinking the Q4\_K gap from∼22%\{\\sim\}22\\%to∼17%\{\\sim\}17\\%overall and 13–15% on the best tensors\.

### 3\.5End\-to\-end model conversion

All 200 language\-model linear layers of Qwen3\.5\-4B \(96 MLP, 32 attention, 72 gated\-DeltaNet\) were decomposed with the weighted algorithm at a fixedμ=3\\mu=3,τ=1\.0\\tau=1\.0,λ=0\\lambda=0, no sweeps, in 19\.9 minutes total on one GPU\. Per\-tensor energy: plain min/mean 98\.52/99\.45%, weighted 99\.34/99\.65%\.

Table 3:First full end\-to\-end validation \(llama\-perplexity, 580 chunks, same imatrix for all calibrated rows\)\. The decomposed model is coherent; at matched bits and calibration it currently trails pure Q5\_K by∼\\sim1\.5 points of relative PPL\. None of the known levers were applied to this conversion: refinement sweeps, per\-matrixμ\\muallocation \(iso\-qualityμ\\muspreads 2\.3–3\.0 across tensors while this run used a flatμ=3\\mu=3\), orτ/λ\\tau/\\lambdatuned on PPL\.The pure ladder also shows that the tuned Q4\_K\_M mix beats pure Q5\_K despite fewer bits: per\-tensor budget allocation is worth more than 0\.6 uniform bpw for k\-quants, and per\-matrixμ\\muallocation, trivially expressible here becauseμ\\muis continuous, is the analogous untapped lever for the decomposition\.

### 3\.6Comparison with Fixed and Stacked Ternary Formats

To isolate the structural capacity of ExTernD, we compare its weight\-reconstruction energy against two recent post\-training ternarization methods: PT2\-LLM\[[6](https://arxiv.org/html/2607.13511#bib.bib6)\], which uses a fixed single ternary plane \(∼1\.58\{\\sim\}1\.58bpw\), and Progressive Trit\-Plane Approximation \(PTQTP\)\[[5](https://arxiv.org/html/2607.13511#bib.bib5)\], which uses a dense superposition ofKKternary planes\. To match bit budgets precisely, we evaluate PTQTP at group sizeG=128G=128and set the ExTernD component count to match the number of ternary elements \(k=K​m​nm\+nk=K\\frac\{mn\}\{m\+n\}\)\.

#### The 1\.58\-bit Bottleneck\.

PT2\-LLM uses an iterative ternary fitting \(ITF\) algorithm to optimize a single ternary matrix with row\-wise and block\-wise scales\. When applied purely to the weight matrix, it preserves only∼78​–​81%\{\\sim\}78\\text\{\-\-\}81\\%of the original energy\. This confirms our hypothesis: a fixed single ternary plane lacks the structural capacity to accurately represent a dense LLM weight matrix, forcing such methods to heavily rely on activation\-aware scaling to mask the underlying quantization error\.

#### Scaling to Stacked Planes\.

PTQTP addresses this bottleneck by stacking multiple dense ternary planes\. At low bit\-budgets \(K=2K=2,∼3\.4\{\\sim\}3\.4bpw\), PTQTP significantly outperforms ExTernD by 1\.8–2\.6 percentage points of energy preservation, as direct superposition preserves dense structures better than a rank\-deficient \(μ≈1\.5\\mu\\approx 1\.5\) factorization\. However, as the bit budget scales toK=4K=4\(∼6\.8\{\\sim\}6\.8bpw\), ExTernD’s inner rank expands toward and past the matrix’s full rank \(μ\>3\\mu\>3\), structurally approaching perfect reconstruction\. At this regime, ExTernD slightly edges out PTQTP in energy \(∼99\.72%\{\\sim\}99\.72\\%vs99\.59%99\.59\\%\), while PTQTP hits diminishing returns due to its rigid stacked\-discrete\-value formulation\.

#### The Sparsity Advantage\.

Crucially, ExTernD provides a continuous sparsity thresholdτ\\tau\. When forcing ExTernD to favor sparsity \(τ=1\.0\\tau=1\.0\), it achieves a massive57% sparsitycompared to PTQTP’s natural∼34%\{\\sim\}34\\%sparsity and PT2\-LLM’s∼46%\{\\sim\}46\\%sparsity, while maintaining highly competitive accuracy \(only a∼0\.1%\{\\sim\}0\.1\\%drop in energy compared to PTQTP atK=4K=4\)\.

#### Imatrix\-Weighted Objective\.

When minimizing the imatrix\-weighted Frobenius error while strictly matching the number of ternary elements to PTQTP’sK=4K=4, PTQTP shifts its dense assignments to perfectly fit the most important channels, spiking its weighted energy to∼99\.85%\{\\sim\}99\.85\\%\. Under this naive element\-matching, ExTernD sits at a competitive∼99\.58%\{\\sim\}99\.58\\%when constrained toτ=1\.0\\tau=1\.0\.

#### Iso\-BPW Parity\.

Matching the number of ternary elements penalizes ExTernD by ignoring its massive sparsity advantage\. A fairer comparison matches the*effective bits\-per\-weight \(BPW\)*, utilizing Shannon entropy to account for zero\-density\. PTQTP atK=4K=4\(∼33%\\sim 33\\%sparsity\) requires6\.686\.68effective BPW\. When we increase ExTernD’s inner rankμ\\muto precisely match this6\.686\.68BPW footprint, ExTernD achieves virtually identical energy preservation \(∼99\.84%\{\\sim\}99\.84\\%atτ=1\.0\\tau=1\.0,56%56\\%sparsity\) while maintaining a significantly sparser and more hardware\-friendly format \(Table[5](https://arxiv.org/html/2607.13511#S3.T5)\)\. This decisively proves that ExTernD’s factorization structurally matches or exceeds dense stacked planes when normalized for actual information capacity, eliminating the ceiling of fixed\-width methods\.

Table 4:Comparison of weight energy preservation and sparsity\. PT2\-LLM demonstrates the∼81%\{\\sim\}81\\%hard limit of a single ternary plane\. PTQTP scales accuracy by stacking dense planes, while ExTernD scales by expanding the inner rank of sparse ternary factors\. At high capacity \(K=4K=4\), ExTernD matches dense stacked accuracy while unlocking57%57\\%sparsity\.Table 5:Iso\-BPW Comparison\. When allowing ExTernD to expand its inner rankμ\\muto match the actual information\-theoretic bit footprint \(6\.68 BPW\) of PTQTPK=4K=4, ExTernD achieves exact parity in imatrix\-weighted energy preservation, despite being nearly twice as sparse \(56%56\\%vs33%33\\%\)\.

## 4Deployment notes

The factors store today in llama\.cpp’s TQ2\_0 \(2\.06 bpw; its per\-block fp16 scales absorbDDintoCC’s row scales for free\), and bitnet\.cpp\-style LUT kernels\[[9](https://arxiv.org/html/2607.13511#bib.bib9)\]apply directly to both\.

Emerging multiplication\-free ASICs stand to exploit this the most, and the inner\-rank expansion is designed with them in mind: the3\.43\.4–6×6\\timesop\-count growth reads as slowdown only on multiplier\-based datapaths, where a ternary add buys nothing over a multiply–add\.

## 5Limitations

Everything here ran on a single AMD MI50 32 GB GPU, which bounds the evaluation\. We have not tested above 4B or on matrices larger than∼9216×2560\{\\sim\}9216\\times 2560, so the behaviour ofμ\\muat 30B–70B\+ is unknown\. Evaluation is perplexity\-only: no downstream benchmarks, no KLD, no long\-context checks\. We compare only against fixed\-capacity baselines at the tensor level, but full end\-to\-end evaluation against established ternary schemes\[[5](https://arxiv.org/html/2607.13511#bib.bib5),[2](https://arxiv.org/html/2607.13511#bib.bib2)\]is the most valuable missing experiment: our structural and representational claims against them are proven \(Sec\.[2\.1](https://arxiv.org/html/2607.13511#S2.SS1)\), but end\-to\-end task accuracy is not yet measured\. The known levers \(sweeps∼5%\{\\sim\}5\\%, weighting∼4%\{\\sim\}4\\%, per\-matrix rank allocation\) are unstacked, and no fused kernels exist\.

QAT is untried and is the obvious next lever: a short straight\-through pass on an ExTernD initialization should recover accuracy, plausibly faster than QAT on a conventionally quantized model, since rank expansion hands the optimizerμ​\(m\+n\)/max⁡\(m,n\)\\mu\\,\(m\{\+\}n\)/\\max\(m,n\)times as many trainable ternary entries per layer, already initialized near target\. Whether STE gradients behave through two chained ternary factors is untested\.

Matching Q4\_K or Q5\_K bit\-for\-bit is not the objective; those are a familiar yardstick\. The objective is a viable ternary LLM, where accuracy is set byμ\\murather than capped by the format, and this is best read as a step*towards*highly accurate ternary quantization rather than a finished recipe\.

## 6Conclusion

Expanding the inner rank turns ternary representation from a fixed, lossy format with a hard accuracy ceiling into a family that provably converges to full precision askkgrows\. This is what makes ternary LLMs viable: accuracy becomes a dial \(μ\\mu, continuous, withτ\\taufor sparsity\) rather than a ceiling, so multiplication\-free kernels and multiplication\-free hardware can be pointed at any accuracy target, to our knowledge the only known way to get there\. A simple greedy ALS already lands within∼\\sim17–20% of Q4\_K’s bit efficiency at matched accuracy, with identified, unstacked levers of comparable total size, while keeping inference multiplication\-free\.

## Appendix AMonotone residual decrease and convergence

###### Proposition 1\(Monotone decrease\)\.

LetRiR\_\{i\}be the residual before extracting componentii, and letu,vu,vbe any nonzero ternary vectors produced by the fit\. With the optimal scaledi=u⊤​Ri​v/\(‖u‖2​‖v‖2\)d\_\{i\}=u^\{\\top\}R\_\{i\}v/\(\\\|u\\\|^\{2\}\\\|v\\\|^\{2\}\),

‖Ri\+1‖F2=‖Ri−di​u​v⊤‖F2=‖Ri‖F2−\(u⊤​Ri​v\)2‖u‖22​‖v‖22≤‖Ri‖F2,\\\|R\_\{i\+1\}\\\|\_\{F\}^\{2\}=\\\|R\_\{i\}\-d\_\{i\}uv^\{\\top\}\\\|\_\{F\}^\{2\}=\\\|R\_\{i\}\\\|\_\{F\}^\{2\}\-\\frac\{\(u^\{\\top\}R\_\{i\}v\)^\{2\}\}\{\\\|u\\\|\_\{2\}^\{2\}\\,\\\|v\\\|\_\{2\}^\{2\}\}\\;\\leq\\;\\\|R\_\{i\}\\\|\_\{F\}^\{2\},with strict decrease wheneveru⊤​Ri​v≠0u^\{\\top\}R\_\{i\}v\\neq 0\.

###### Proof\.

Expand‖R−d​u​v⊤‖F2=‖R‖F2−2​d​u⊤​R​v\+d2​‖u‖2​‖v‖2\\\|R\-d\\,uv^\{\\top\}\\\|\_\{F\}^\{2\}=\\\|R\\\|\_\{F\}^\{2\}\-2d\\,u^\{\\top\}Rv\+d^\{2\}\\\|u\\\|^\{2\}\\\|v\\\|^\{2\}and substitute the minimizingdd\. ∎

###### Proposition 2\(Convergence to arbitrary accuracy\)\.

Augment each step to take whichever of \(a\) the ALS solution and \(b\) the best single\-entry pairu=ei⋆,v=sign⁡\(\(R\)i⋆​j⋆\)​ej⋆u=e\_\{i^\{\\star\}\},v=\\operatorname\{sign\}\(\(R\)\_\{i^\{\\star\}j^\{\\star\}\}\)\\,e\_\{j^\{\\star\}\}at the largest\-magnitude residual entry gives the larger decrease\. Then

‖Rk‖F2≤\(1−1m​n\)k​‖A‖F2⟶0\.\\\|R\_\{k\}\\\|\_\{F\}^\{2\}\\;\\leq\\;\\Big\(1\-\\tfrac\{1\}\{mn\}\\Big\)^\{k\}\\,\\\|A\\\|\_\{F\}^\{2\}\\;\\longrightarrow\\;0\.

###### Proof\.

Option \(b\) with‖u‖=‖v‖=1\\\|u\\\|=\\\|v\\\|=1removes\(u⊤​R​v\)2=maxi​j⁡Ri​j2≥‖R‖F2/m​n\(u^\{\\top\}Rv\)^\{2\}=\\max\_\{ij\}R\_\{ij\}^\{2\}\\geq\\\|R\\\|\_\{F\}^\{2\}/mn\. The augmented step removes at least this much, giving the geometric bound\. ∎

###### Corollary 1\.

For everyε\>0\\varepsilon\>0there is a finite inner rankkkwith‖A−B​diag⁡\(D\)​C‖F<ε\\\|A\-B\\operatorname\{diag\}\(D\)C\\\|\_\{F\}<\\varepsilon: the decomposition reaches the bf16 weights arbitrarily closely, with error monotone inkk\. No fixed number of ternary planes has this property; the expansion is what removes the information bottleneck\.

In practice the safeguard never activates, since the ALS decrease is far larger than the single\-entry bound, and empirical convergence is much faster than geometric inm​nmn\(99% energy atμ≈2\.5\\mu\\approx 2\.5–33\)\.

## References

- \[1\]H\. Wang, S\. Ma, L\. Dong, S\. Huang, H\. Wang, L\. Ma, F\. Yang, R\. Wang, Y\. Wu, F\. Wei\. BitNet: Scaling 1\-bit Transformers for Large Language Models\.*arXiv:2310\.11453*, 2023\.
- \[2\]S\. Ma, H\. Wang, L\. Ma, L\. Wang, W\. Wang, S\. Huang, L\. Dong, R\. Wang, J\. Xue, F\. Wei\. The Era of 1\-bit LLMs: All Large Language Models are in 1\.58 Bits\.*arXiv:2402\.17764*, 2024\.
- \[3\]F\. Li, B\. Liu, X\. Wang, B\. Zhang, J\. Yan\. Ternary Weight Networks\.*arXiv:1605\.04711*, 2016\.
- \[4\]C\. Zhu, S\. Han, H\. Mao, W\. J\. Dally\. Trained Ternary Quantization\.*ICLR*, 2017\. arXiv:1612\.01064\.
- \[5\]H\. Xiao, R\. Yang, Q\. Yang, W\. Xu, Z\. Li, Y\. Su, Z\. Liu, H\. Yang, N\. Wong\. PTQTP: Post\-Training Quantization to Trit\-Planes for Large Language Models\.*arXiv:2509\.16989*, 2025\.
- \[6\]X\. Yan, W\. Wu, Z\. Wu, S\. Yang, Z\. Gao, M\. Huang\. PT2\-LLM: Post\-Training Ternarization for Large Language Models\.*arXiv:2510\.03267*, 2025\.
- \[7\]E\. Frantar, S\. Ashkboos, T\. Hoefler, D\. Alistarh\. GPTQ: Accurate Post\-Training Quantization for Generative Pre\-trained Transformers\.*arXiv:2210\.17323*, 2022\.
- \[8\]J\. Lin, J\. Tang, H\. Tang, S\. Yang, W\.\-M\. Chen, W\.\-C\. Wang, G\. Xiao, X\. Dang, C\. Gan, S\. Han\. AWQ: Activation\-aware Weight Quantization for LLM Compression and Acceleration\.*MLSys*, 2024\. arXiv:2306\.00978\.
- \[9\]J\. Wang et al\. Bitnet\.cpp: Efficient Inference Framework for 1\-bit LLMs\.*https://github\.com/microsoft/BitNet*, 2024\.