每个权重都是一条由共享的离散决策构成的路径

# Every Weight Is a Path of Shared Discrete Decisions
Source: [https://arxiv.org/html/2610.31931](https://arxiv.org/html/2610.31931)
Stamatis LefkimmiatisAffiliation:LEAD Lab, ITMO University

October 2026

###### Abstract

We introduce TWIG \(Tree\-structured Weights with Inherited Gates\), a representation in which every weight of a neural network is the end of a path of small discrete decisions: a sign, a shift by a power of two, a zero gate, an index into a shared table\. Weights that belong together share the upper part of their paths, and each decision is inherited by every weight below it, so it is stored once and applied once\. This simple change has three consequences\. Any trained network can be written as a TWIG, and when the decisions are powers of two its weight layers become circuits made almost entirely of wires and adders, suited to chips whose weights are fixed at fabrication\. Pruning, quantization, low\-rank factorization, codebooks and weight sharing become edits of one object, so a single optimizer can choose among all of them for a whole network\. And choosing the decisions is the ground\-state problem of a higher\-order Boltzmann machine whose interactions follow the paths of the tree, the problem class that annealers and quantum optimizers are built to solve\. Today these questions are handled apart: each compression method brings its own format and solver, a circuit with fixed weights needs a dedicated constant multiplier for every weight, and discrete solvers reach only narrow subproblems such as rounding\. Weight formats have long been moving toward shared decisions, through group scales, shared exponents, microexponents and sparse weights; TWIG takes this trend to its end, and every common format is a shallow TWIG at no extra cost\. On five weight matrices of a large language model, a flat tree whose signed power\-of\-two terms are chosen per weight for circuit cost, with no group scales, needs 9–15% fewer additions than GPTQ at equal layer error at 3\- and 4\-bit accuracy on typical matrices, and 20–55% fewer on matrices with strong outliers\. The properties of TWIG are proved; the measurements we report are first evidence on selected models and layers\.

## 1Introduction

Modern neural networks are trained in the continuum\. Gradient descent needs a smooth landscape to follow, so weights are modelled as real numbers and learning is described as motion through a high\-dimensional real space; much of what we understand about optimization and generalization rests on this view\. Deployment, however, takes place in a discrete world\. On digital hardware, every weight is a finite bit pattern drawn from a fixed format, every product is computed by a circuit of logic gates, and every byte has to be moved through a memory hierarchy\. The passage from the continuous model that was learned to the discrete object that is executed is therefore unavoidable, and it has grown into a research field of its own\.

That field has grown in pieces\. Pruning removes weights, quantization restricts them to finite grids, low\-rank factorization replaces matrices by products of thinner ones, codebook methods map blocks of weights to entries of a shared table, and weight sharing reuses parameters across layers\. Each family brings its own storage format, its own solver and often its own kernel, and combining them \(four bits in one layer, half of the weights pruned in another, a low\-rank correction in a third\) remains largely a matter of manual design, without a common yardstick\. Two further questions sit almost entirely outside this picture\. The first is the cost of*computing*with the weights rather than storing them: in digital logic a multiplier is considerably larger and more energy\-hungry than an adder\[[30](https://arxiv.org/html/2610.31931#bib.bib30)\], and in circuits whose weights are fixed at fabrication, such as printed classifiers\[[57](https://arxiv.org/html/2610.31931#bib.bib57)\]or chips that hardwire an entire language model\[[71](https://arxiv.org/html/2610.31931#bib.bib71)\], every weight calls for a constant multiplier of its own\. The second is the reach of discrete optimization: choosing discrete weights is a combinatorial problem, yet annealers, samplers and quantum optimizers, which are built for exactly such problems, have so far been applied only to narrow parts of it, such as the rounding of a single layer\[[59](https://arxiv.org/html/2610.31931#bib.bib59)\]\. What is missing, we argue, is a representation of weights that is discrete by construction, that makes the structure shared by weights explicit, and that exposes the cost of applying them\.

The formats used in practice have been moving toward such a representation for some time \([Figure1](https://arxiv.org/html/2610.31931#S1.F1)\)\. Group quantization lets the weights of a group share one scale\[[22](https://arxiv.org/html/2610.31931#bib.bib22),[47](https://arxiv.org/html/2610.31931#bib.bib47)\]; block floating point and the microscaling formats let a block of weights share one exponent\[[65](https://arxiv.org/html/2610.31931#bib.bib65),[66](https://arxiv.org/html/2610.31931#bib.bib66)\]; sparse formats let the description of a weight end as soon as it is known to be zero; and codebook methods describe a whole block of weights by one or a few entries of a shared table\[[16](https://arxiv.org/html/2610.31931#bib.bib16),[77](https://arxiv.org/html/2610.31931#bib.bib77)\]\. Each of these steps records a decision once for many weights instead of once for every weight, and each has paid off in memory, bandwidth and speed\. A few formats already stack two or three levels of shared scales, such as shared microexponents\[[67](https://arxiv.org/html/2610.31931#bib.bib67)\]and HiFloat4\[[50](https://arxiv.org/html/2610.31931#bib.bib50)\]\. Yet each format shares one fixed kind of decision, at levels fixed in advance, in a format of its own\.

We take this trend to its natural end\. Our proposal can be stated in a single sentence:

> Write every weight as the end of a path of small discrete decisions, and let weights that belong together share the upper part of the path\.

The result is a tree over the entries of a weight matrix \([Figure1](https://arxiv.org/html/2610.31931#S1.F1)d\)\. Each internal node holds one small decision \(a shift by a power of two, a sign, a zero gate, an index into a shared table\), which is inherited by every weight below it; each leaf holds the little that remains private to a single weight; and the value of a weight is the product of the decisions along its path\. We call this representation TWIG, for*Tree\-structured Weights with Inherited Gates*, and refer to an individual tree as a*state tree*\. A state tree still describes an ordinary real matrix, so the learned model and its deployed form remain directly comparable; what the tree adds is an explicit account of how the matrix is stored, which decisions its entries share, and what it costs to apply\.

signexp\.mant\.signexp\.mant\.signexp\.mant\.signexp\.mant\.group scaleq1q\_\{1\}q2q\_\{2\}q3q\_\{3\}q4q\_\{4\}shared exponent±m\\pm m±m\\pm m0000scale×12\\times\\frac\{1\}\{2\}00\+1\+1−12\-\\frac\{1\}\{2\}w1w\_\{1\}w2w\_\{2\}w3w\_\{3\}w4w\_\{4\}w1w\_\{1\}w2w\_\{2\}w3w\_\{3\}w4w\_\{4\}w1w\_\{1\}w2w\_\{2\}w3w\_\{3\}w4w\_\{4\}w1w\_\{1\}w2w\_\{2\}w3w\_\{3\}w4w\_\{4\}\(a\) 16\-bit float\(b\) group scale\(c\) shared exponent\(d\) TWIG state treeevery weight ownsits whole paththe first decisionis shared by the groupa zero ends thepath earlydecisions are inheritedat every levelFigure 1:From formats in use to TWIG\. Each column of boxes is the description of one weight, read from top to bottom; a box that spans several weights is a decision they share, stored once\. \(a\) A 16\-bit float stores a sign, an exponent and a mantissa for every weight\. \(b\) Group quantization shares one scale among the weights of a group\. \(c\) Block formats share an exponent,and a zero ends a weight’s description\. \(d\) A state tree shares decisions at every level, and each decision is inherited by every weight below it: one scale for all four weights, a halving for the first pair, an “off” switch for the second pair, after which nothing more is stored \(dashed\)\. Blue: shared decisions; orange: private decisions; grey: zero\.This single change addresses the three questions above at once, because the same tree can be read in three ways \([Table1](https://arxiv.org/html/2610.31931#S2.T1)\)\. Read as a*compressed model*, a state tree contains today’s formats as shallow special cases, and pruning, quantization, factorization, codebooks and sharing all become edits of the same object; methods developed separately become comparable and combinable, and a single budgeted optimizer can choose among all of them for a whole network \([Section5](https://arxiv.org/html/2610.31931#S5)\)\. Read as a*circuit*, a tree whose decisions are powers of two multiplies by shifting, which in fixed hardware is merely a way of routing wires; its weight layers become circuits made almost entirely of wires and adders, and precision and circuit size become a single quantity \([Section6](https://arxiv.org/html/2610.31931#S6)\)\. Read as an*optimization problem*, choosing the decisions of a tree is the search for the lowest\-energy state of a higher\-order Boltzmann machine whose couplings are supplied by the data and whose higher\-order terms follow the paths of the tree, the problem class that classical samplers, annealers and quantum optimizers are built to solve \([Section7](https://arxiv.org/html/2610.31931#S7)\)\. Finally, because the decisions are explicit, a network could be trained directly in this discrete space, so that the trained object is already the deployed one; we outline this direction in[Section8](https://arxiv.org/html/2610.31931#S8)\.

#### Contributions\.

- •We introduce TWIG, a representation of neural network weights as trees of shared discrete decisions, together with its two costs: storage bits and circuit additions \([Section2](https://arxiv.org/html/2610.31931#S2)\)\.
- •We establish its basic properties: it contains the number formats in use today at no extra cost, it loses nothing in the worst case and uses fewer bits where weights share structure, and any matrix can be approximated, to any precision, by a sum of power\-of\-two trees that is applied with shifts and additions alone \([Section3](https://arxiv.org/html/2610.31931#S3)\)\.
- •We show that the main families of compression methods are edits of one object, chosen by a single budgeted optimizer that is exact under additive costs \([Section5](https://arxiv.org/html/2610.31931#S5)\)\.
- •We show how a state tree maps onto a circuit of wires and adders, and report a first measurement on five weight matrices of a large language model: a flat tree whose signed power\-of\-two terms are chosen per weight saves 9–15% of the additions of GPTQ at equal layer error at 3\- and 4\-bit accuracy on typical matrices, and 20–55% on matrices with strong outliers \([Section6](https://arxiv.org/html/2610.31931#S6)\)\.
- •We show that choosing the states of a tree is the ground\-state problem of a higher\-order Boltzmann machine whose higher\-order terms follow the paths of the tree, which connects weight selection to classical and quantum discrete optimizers \([Section7](https://arxiv.org/html/2610.31931#S7)\)\.

Several of the properties restate classical results in the language of trees, as marked in[AppendixA](https://arxiv.org/html/2610.31931#A1); the results specific to state trees are the reach of shared shifts, the equivalence of two encodings, the exact optimizer with shared subtrees and the higher\-order Boltzmann machine\.[Section9](https://arxiv.org/html/2610.31931#S9)collects first evidence on real models and beyond them, and[Section10](https://arxiv.org/html/2610.31931#S10)relates TWIG to number formats, decision diagrams and multiplier\-free networks\. Precise definitions, statements and proofs are collected in[AppendixA](https://arxiv.org/html/2610.31931#A1)\.

## 2From numbers to state trees

### 2\.1A number is a path of decisions

Consider a weight stored in 16\-bit floating point,w1=0\.15625w\_\{1\}=0\.15625\([Figure2](https://arxiv.org/html/2610.31931#S2.F2)a\)\. Its sixteen bits are not read as one number but as three decisions: a sign bitss, a five\-bit exponenteeand a ten\-bit mantissamm\. On the normal range the value they encode is a product of three factors, one per decision,

w=\(−1\)s⋅2e−15⋅\(1\+m1024\),w\\;=\\;\(\-1\)^\{s\}\\cdot 2^\{\\,e\-15\}\\cdot\\Big\(1\+\\frac\{m\}\{1024\}\\Big\),so thatw1=\(\+1\)⋅2−3⋅1\.25w\_\{1\}=\(\+1\)\\cdot 2^\{\-3\}\\cdot 1\.25\. Read from left to right, the decisions form a path: the sign selects a half\-line, the exponent an octave within it, and the mantissa a point within the octave\. Now consider a second weight from the same matrix,w2=−0\.1875=\(−1\)⋅2−3⋅1\.5w\_\{2\}=\-0\.1875=\(\-1\)\\cdot 2^\{\-3\}\\cdot 1\.5\. Its sign and mantissa differ from those ofw1w\_\{1\}, but its exponent is the same: both weights lie in the same octave of magnitudes\. In a standard format each weight owns its entire path, so this common exponent is stored twice\.

ssexponenteemantissamm00110001000000001011001000000000w1=0\.15625w\_\{1\}=0\.15625w2=−0\.1875w\_\{2\}=\-0\.1875=\+1⋅2−3⋅1\.25=\\;\+1\\cdot 2^\{\-3\}\\cdot 1\.25=−1⋅2−3⋅1\.5=\\;\-1\\cdot 2^\{\-3\}\\cdot 1\.5identical in both weights\(a\)Two weights in FP16: each stores its own sign, exponent and mantissa \(16 bits each\)\.exponent0110001100×2−3\\times\\,2^\{\-3\}𝟎\|0100000000\\mathbf\{0\}\\,\|\\,0100000000\+1\.25\+1\.25𝟏\|1000000000\\mathbf\{1\}\\,\|\\,1000000000−1\.5\-1\.5w1=2−3⋅\(\+1\.25\)w\_\{1\}=2^\{\-3\}\\cdot\(\+1\.25\)w2=2−3⋅\(−1\.5\)w\_\{2\}=2^\{\-3\}\\cdot\(\-1\.5\)stored once\(b\)The shared exponent is stored once; each leaf keeps its sign and mantissa\. 27 bits instead of 32, with exactly the same values\.x1x\_\{1\}x2x\_\{2\}\+\+\+\+≫2\\gg 2≫1\\gg 1−\-yy≫3\\gg 3shared nodeleaf ofw1w\_\{1\}:1\.25=1\+2−21\.25=1\+2^\{\-2\}leaf ofw2w\_\{2\}:1\.5=1\+2−11\.5=1\+2^\{\-1\}\(c\)Computingy=w1​x1\+w2​x2y=w\_\{1\}x\_\{1\}\+w\_\{2\}x\_\{2\}from the tree: three additions, no multiplication\.
Figure 2:A first example\. \(a\) Two neighbouring weights stored in 16\-bit floating point; their exponents are identical, yet the format stores them twice\. \(b\) Because a weight is the*product*of its decisions, the shared exponent can be stored once in a node above both weights; each weight is recovered exactly as the product along its path\. \(c\) The same tree computesw1​x1\+w2​x2w\_\{1\}x\_\{1\}\+w\_\{2\}x\_\{2\}without forming either weight: each leaf becomes shifted copies of its own input \(≫k\\gg kshifts bykkbinary positions, i\.e\. multiplies by2−k2^\{\-k\}\), and the shared exponent is applied once, to the sum, as a shift by three positions\. Three additions replace two multiplications and one addition\.
### 2\.2Sharing a decision at the same precision

Because the value of a weight is a product of its decisions, a factor that two weights have in common can be taken out of both and stored once, in a node placed above them \([Figure2](https://arxiv.org/html/2610.31931#S2.F2)b\)\. The node holds the exponent,2−32^\{\-3\}; the two leaves below it hold what remains private to each weight, a sign and a mantissa\. Each weight is recovered as the product of the decisions along its path from the node to its leaf, so both values are reproduced exactly, at the same precision as before, with5\+2×11=275\+2\\times 11=27bits instead of3232\. The saving grows with the size of the group:ggweights that share an exponent need5\+11​g5\+11gbits instead of16​g16g, a reduction that approaches31%31\\%for large groups without changing a single value\. The weights of a trained layer rarely share an exponent exactly, but neighbouring weights tend to lie within a few octaves of one another\. A second kind of shared decision covers this case: a node that holds a single bit and either leaves everything below it unchanged or halves it\. Placed at several levels, such bits let weights of different but nearby magnitudes share most of their path, and the private part left at each leaf can then be shortened where accuracy allows\. This is the general state tree of[Section2\.4](https://arxiv.org/html/2610.31931#S2.SS4)\.

### 2\.3Multiplying without rebuilding the weight

The same structure changes how the weights are used\. A layer never needs the weights themselves, only their products with its inputs, herey=w1​x1\+w2​x2y=w\_\{1\}x\_\{1\}\+w\_\{2\}x\_\{2\}\. Written along the tree,

y=2−3​\[\(1\+2−2\)​x1−\(1\+2−1\)​x2\]\.y\\;=\\;2^\{\-3\}\\Big\[\\big\(1\+2^\{\-2\}\\big\)\\,x\_\{1\}\\;\-\\;\\big\(1\+2^\{\-1\}\\big\)\\,x\_\{2\}\\Big\]\.Each leaf acts on its own input, and the shared node acts once, on the sum of its subtree \([Figure2](https://arxiv.org/html/2610.31931#S2.F2)c\)\. Every factor is a power of two, so every multiplication is a shift:x1\+\(x1≫2\)x\_\{1\}\+\(x\_\{1\}\\gg 2\)andx2\+\(x2≫1\)x\_\{2\}\+\(x\_\{2\}\\gg 1\)take one addition each, a subtraction combines them, and the shared exponent becomes a shift of the result by three positions, which in a circuit with fixed weights is only a way of routing wires\. Three additions replace two multiplications and one addition, and neither weight is ever formed as a number\. In general a mantissa has more nonzero digits than in this example, and each costs one addition;[Section3](https://arxiv.org/html/2610.31931#S3)shows that a few signed power\-of\-two terms per weight suffice for any desired precision, and[Section6](https://arxiv.org/html/2610.31931#S6)measures how many are needed on real weights\.

### 2\.4State trees in general

The example has a single shared node\. A state tree applies the same idea at every level of a hierarchy over the entries of a matrix\.[Figure3](https://arxiv.org/html/2610.31931#S2.F3)shows the tree we use as a running example\. It describes one column of a weight matrix with eight rows, arranged as the leaves of a balanced binary tree\. Every internal node holds one bit, which either leaves everything below it unchanged or halves it, and every leaf holds one of five states,\{0,±1,±12\}\\\{0,\\pm 1,\\pm\\frac\{1\}\{2\}\\\}: a sign and a small magnitude, possibly zero\. The value of an entry is the product of the states on its path from the root, multiplied by a scale for its row and a base value for its column\. For row 6, the path drawn in red gives14​ρ6​βj\\frac\{1\}\{4\}\\rho\_\{6\}\\beta\_\{j\}\.

000011\+1\+10000−12\-\\frac\{1\}\{2\}\+1\+11100−1\-1\+12\+\\frac\{1\}\{2\}1100\+1\+1column baseβj\\beta\_\{j\}ρ1\\rho\_\{1\}ρ2\\rho\_\{2\}ρ3\\rho\_\{3\}ρ4\\rho\_\{4\}ρ5\\rho\_\{5\}ρ6\\rho\_\{6\}ρ7\\rho\_\{7\}ρ8\\rho\_\{8\}node bitb∈\{0,1\}b\\in\\\{0,1\\\}: multiply everythingbelow the node by\(12\)b\(\\frac\{1\}\{2\}\)^\{b\};leaf statea∈\{0,±1,±12\}a\\in\\\{0,\\pm 1,\\pm\\frac\{1\}\{2\}\\\}\.W^6​j=ρ6​βj⋅1⋅12⋅1⋅\(\+12\)\\widehat\{W\}\_\{6j\}=\\rho\_\{6\}\\,\\beta\_\{j\}\\cdot 1\\cdot\\tfrac\{1\}\{2\}\\cdot 1\\cdot\\big\(\+\\tfrac\{1\}\{2\}\\big\)=14​ρ6​βj\\phantom\{\\widehat\{W\}\_\{6j\}\}=\\tfrac\{1\}\{4\}\\,\\rho\_\{6\}\\,\\beta\_\{j\}rowsFigure 3:The running example: a deeper state tree for one columnjjof a matrix with eight rows\. Each internal node holds one bit that halves everything below it or leaves it unchanged; each leaf holds a sign and a magnitude, possibly zero\. The value of an entry is the product of the states on its path \(red\), times a row scale and a column base,[Equation1](https://arxiv.org/html/2610.31931#S2.E1)\. Apart fromρi​βj\\rho\_\{i\}\\beta\_\{j\}, every factor is a power of two or zero\. The column stores 7 node bits \(6 if the root bit is folded intoβj\\beta\_\{j\}\) and 8 five\-state leaves, about 3\.2 bits per weight\.In general, a state tree is a tree whose leaves are the entries of a matrix and whose nodes hold*states*, each drawn from a small alphabet and each standing for a value; choosing the state of a node is one decision, and it is inherited by every entry below the node\. The entry in rowiiand columnjjis

W^i​j=ρi​βj​∏nodesvon the path to\(i,j\)\(value of the state at​v\),\\widehat\{W\}\_\{ij\}\\;=\\;\\rho\_\{i\}\\,\\beta\_\{j\}\\\!\\\!\\prod\_\{\\text\{nodes $v$ on the path to $\(i,j\)$\}\}\\\!\\\!\\big\(\\text\{value of the state at \}v\\big\),\(1\)where the row scalesρi\\rho\_\{i\}and column basesβj\\beta\_\{j\}carry the overall scale of the matrix and may be stored in any format, for example FP16\. Internal nodes stand for groups of entries \(a block of a row, a pair of neighbouring rows, a channel, an attention head, the whole matrix\), and a decision at a node is shared by its whole group\. A tree can be as shallow as the single shared exponent of[Figure2](https://arxiv.org/html/2610.31931#S2.F2)or as deep as the running example\. Because neighbours share decisions, the order of the rows matters: in the running example, rows with similar magnitudes are placed next to each other\. A tree may group the entries of a row or of a column\. The tree of[Figure2](https://arxiv.org/html/2610.31931#S2.F2)groups two entries of one row, so its shared shift acts once on a sum of inputs; the running example groups the entries of one column, so a shared shift acts once on an input before it fans out to several outputs\. The two orientations cost the same bits, but they give different circuits, and, as[Section7](https://arxiv.org/html/2610.31931#S7)will show, they couple different leaves when the states are optimized\. A handful of kinds of states do most of the work:

- •a*shift*has a power of two as its value and rescales everything below it, like the shared exponent of[Figure2](https://arxiv.org/html/2610.31931#S2.F2);
- •a*gate*has the value zero and deletes everything below it, so nothing below a closed gate needs to be stored;
- •a*transparent*state has the value one, so a node can always be present without changing anything;
- •*signs and small magnitudes*complete the description at the leaves;
- •a*lookup*is a state whose value depends on the states above it; for example, the entries below an index into a shared table read their values from the table\.

### 2\.5Combining trees

A single tree describes every weight by one path\. Three operations combine trees into richer descriptions, and each of them has a familiar meaning\. The first is the*sum*\. Look again at the leaf ofw1w\_\{1\}in[Figure2](https://arxiv.org/html/2610.31931#S2.F2): its mantissa,1\.25=1\+2−21\.25=1\+2^\{\-2\}, is already a sum of two powers of two, and each of the two terms became one wire into the adder\. In general, a sum adds two trees entry by entry and expresses “a coarse description plus a correction”: a first tree gives every weight its leading power of two, a second tree adds the next one where it is needed, and each additional term costs one addition\. Sums are how a state tree buys precision\. The second is the*product*, which multiplies two trees as matrices\. It expresses “a product of two thin matrices”, as in low\-rank factorization, or “a matrix placed between two fixed transforms”, as in methods that rotate weights before quantizing them\. The third is the*reference*, which points to a tree defined elsewhere\. It expresses “look it up in a shared table”: the referenced tree is stored once and may be read by many entries, many matrices or many layers, as a codebook or a shared dictionary is\. With references, the tree becomes a directed acyclic graph, which we call a*state\-tree graph*; we keep the shorter name when the distinction does not matter\. Sums, products and references reappear in[Section3](https://arxiv.org/html/2610.31931#S3)as the dials that make the representation universal, and in[Section5](https://arxiv.org/html/2610.31931#S5)as the edits behind familiar compression methods\.

### 2\.6Two prices: bits and additions

The example of[Figure2](https://arxiv.org/html/2610.31931#S2.F2)already showed that a state tree has two costs, and both can be read from the tree itself\. The*storage cost*is the number of bits needed to write down the states: the logarithm of the alphabet size for every stored state, with shared parts counted once and nothing counted below a closed gate\. In[Figure2](https://arxiv.org/html/2610.31931#S2.F2)it is 27 bits for two weights, against 32 in FP16\. The column of[Figure3](https://arxiv.org/html/2610.31931#S2.F3)stores seven node bits and eight five\-state leaves,7\+8​log2​5≈25\.67\+8\\log\_\{2\}5\\approx 25\.6bits or about 3\.2 bits per weight; for long columns the cost approaches1\+log2⁡5≈3\.31\+\\log\_\{2\}5\\approx 3\.3bits per weight, plus the row scales and column bases\. Folding the root bit into the column base saves one more bit per column \([PropositionA\.10](https://arxiv.org/html/2610.31931#A1.Thmtheorem10)\), and a fractional cost such aslog2⁡5\\log\_\{2\}5is reached by coding leaves jointly, for example three five\-state leaves in 7 bits, since53<275^\{3\}<2^\{7\}\. The*circuit cost*is the number of additions needed to apply the matrix to an input when every state is a power of two, a short sum of powers of two, or zero\. A power of two only shifts an input, a sign only negates it, and a zero removes it, so all that remains is adding up the shifted inputs: one addition for every term beyond the first in each output\. In[Figure2](https://arxiv.org/html/2610.31931#S2.F2)this gives three additions\. In the running example, every nonzero entry contributes a single shifted copy of its input to its output, and the two zero entries contribute nothing\.[Section6](https://arxiv.org/html/2610.31931#S6)develops this view for whole networks\. The two prices need not agree, and the example shows why\. Sharing the exponent ofw1w\_\{1\}andw2w\_\{2\}saved five bits but not a single addition: a shift by three positions costs nothing in a circuit with fixed weights, whether it is applied once to the sum or separately to each term\. The three additions came entirely from the nonzero digits at the leaves\. In general, shared decisions save bits, while additions are saved only by having fewer nonzero terms, so the best tree for storage and the best tree for circuits can differ \([Section6](https://arxiv.org/html/2610.31931#S6)\)\. Because the representation makes both prices explicit, an optimizer can trade one against the other\.

### 2\.7One tree, three readings

The vocabulary is now complete, and it can be read in three ways \([Table1](https://arxiv.org/html/2610.31931#S2.T1)\)\. In compression, the states are what is stored, and choosing them is the compression method\. In a circuit, the states become wires and adders, and choosing them is the circuit design\. In discrete optimization, the states are binary units, and choosing them is the search for the configuration of lowest energy\. The rest of the paper follows these three readings in turn, after first establishing, in[Section3](https://arxiv.org/html/2610.31931#S3), what state trees can and cannot express\.

Table 1:One tree, three readings\. Each part of a state tree has a meaning in compression \([Section5](https://arxiv.org/html/2610.31931#S5)\), in circuits \([Section6](https://arxiv.org/html/2610.31931#S6)\) and in discrete optimization \([Section7](https://arxiv.org/html/2610.31931#S7)\); this is what lets one representation serve all three\.

## 3What state trees can express

The example of[Figure2](https://arxiv.org/html/2610.31931#S2.F2)raises four natural questions\. Does writing weights as state trees lose anything that today’s formats offer? Can every matrix be described in this way, and at what price in bits? Can multiplications always be avoided without giving up precision? And what does it cost to share a decision? This section answers them in turn\. We state each answer informally, as a property; the precise statements and their short proofs are collected in[AppendixA](https://arxiv.org/html/2610.31931#A1)\. The properties refer to two variants of the model\. In the*core*model, the value of every state is fixed in advance\. The general model also admits lookups, whose values depend on the states above them, as a mantissa depends on whether its exponent is subnormal\. With lookups, any code that is decoded from the root down is a state tree, so[Property1](https://arxiv.org/html/2610.31931#Thmproperty1)is a statement about costs rather than about expressiveness\. It is the only property that needs lookups \(marked in[Table2](https://arxiv.org/html/2610.31931#S3.T2)\); all others hold in the core model\.

### 3\.1Nothing is lost: today’s formats are shallow state trees

The shared exponent of[Figure2](https://arxiv.org/html/2610.31931#S2.F2)is not an exotic construction: it is what block floating point and the microscaling formats already do for blocks of weights, and a group scale in integer quantization plays the same role\. Every format in common use turns out to be a state tree with only a few levels\.

###### Property 1\(Existing formats\)\.

Floating\-point formats, integers with group scales and zero points, block floating point and microscaling formats \(including shared microexponents\), ternary weights, vector codebooks andN:MN\{:\}Msparsity are all shallow state trees, with at most five levels of decisions on any path\. Each costs at most as many bits as the original format, and exactly as many for the dense formats \([Table2](https://arxiv.org/html/2610.31931#S3.T2);[PropositionA\.4](https://arxiv.org/html/2610.31931#A1.Thmtheorem4)\)\.

Adopting state trees is therefore a conservative step: whatever a format in use today can store, a state tree stores at no extra cost\. Refining a tree never removes options either\. A new node can always be given the transparent state, and an alphabet can always be enlarged, so a deeper tree contains every shallower one with the same leaves \([PropositionA\.5](https://arxiv.org/html/2610.31931#A1.Thmtheorem5)\)\. Most formats of[Table2](https://arxiv.org/html/2610.31931#S3.T2)occupy only the top and bottom of the tree; shared microexponents and HiFloat4 already use one or two levels in between, and these levels are where new structure can go\.

Table 2:Formats in use are shallow state trees, at no more than their original cost \([Property1](https://arxiv.org/html/2610.31931#Thmproperty1)\)\.†Uses lookups: subnormal mantissas, codes read relative to a zero point, entries read from a codebook, and entries kept by anN:MN\{:\}Mmask; the other rows are core trees\. Additive codebooks and rotations are built from these pieces with sums and products; trellis codes, whose overlapping windows share bits without nesting, fit only as a codebook over a whole block, with a table that is generated rather than stored\. Deeper trees add levels of shared decisions between the two columns\.
### 3\.2Nothing is lost in the worst case

In[Figure2](https://arxiv.org/html/2610.31931#S2.F2), sharing the exponent did not change which values the two weights could take; it only stopped the format from paying twice for the same decision\. This is the general picture\. What a tree can express is governed by the number of states it stores, and where those states sit decides how much the description costs\.

###### Property 2\(Completeness\)\.

A tree withKKpossible values at each leaf represents every matrix whose entries come from a set ofKKvalues, usinglog2⁡K\\log\_\{2\}Kbits per entry, and no encoding that can represent all such matrices uses fewer bits on every one of them\. With finely spaced leaf values, state trees approximate every matrix to any accuracy, within a vanishing margin of the number of bits that any encoding needs in the worst case \([TheoremsA\.6](https://arxiv.org/html/2610.31931#A1.Thmtheorem6)and[A\.7](https://arxiv.org/html/2610.31931#A1.Thmtheorem7)\)\.

A tree can always spend its states at the leaves and behave as an ordinary quantizer, so it never does worse than a flat format\. Spending states higher up, where one state serves many weights, pays off when the weights have the corresponding structure: similar magnitudes within a group, as in[Figure2](https://arxiv.org/html/2610.31931#S2.F2), runs of zeros, or repeated patterns\. For matrices without any such structure, no representation can do better on all of them at once, and a state tree costs the same as a flat format\. State trees do not create structure; they give it a place to go\.

### 3\.3No multiplications: shifts and additions suffice

In[Figure2](https://arxiv.org/html/2610.31931#S2.F2), every factor was a power of two, so every multiplication became a shift\. The two mantissas helped:1\.251\.25and1\.51\.5each need only two power\-of\-two terms\. A single tree whose states are powers of two can only produce values of the form±2k\\pm 2^\{k\}\([Figure4](https://arxiv.org/html/2610.31931#S3.F4)a, middle\)\. This is a logarithmic grid, and its worst\-case relative error, one third, does not shrink as the tree grows\. The remedy is the sum of[Section2](https://arxiv.org/html/2610.31931#S2): add a second tree for the next term, and a third if needed\.

###### Property 3\(Shifts and additions suffice\)\.

A sum oftttrees whose states are powers of two approximates every matrix to a relative error of3−t3^\{\-t\}in every entry, within the range of exponents its states allow\. It represents every matrix of integers of magnitude below2b2^\{b\}exactly with⌈\(b\+1\)/2⌉\\lceil\(b\+1\)/2\\rceilterms per entry, and with aboutb/3b/3nonzero terms on average\. Applying such a sum to an input needs shifts, additions, and onlym\+nm\+ngenuine multiplications, by the row scales and the column bases that thetttrees share \([TheoremA\.8](https://arxiv.org/html/2610.31931#A1.Thmtheorem8)\)\.

Each additional power\-of\-two term in a weight costs one addition and divides the worst\-case error by three \([Figure4](https://arxiv.org/html/2610.31931#S3.F4)a, bottom\)\. Precision and circuit size thus become one and the same quantity, the number of terms, which[Section6](https://arxiv.org/html/2610.31931#S6)turns into circuits\. The remainingm\+nm\+nmultiplications can themselves be expanded into shifts and additions, or folded into neighbouring layers\.

### 3\.4The price of sharing

Sharing a decision saves bits, but it constrains the weights that share it\. In[Figure2](https://arxiv.org/html/2610.31931#S2.F2)the shared exponent required both weights to lie in the same octave\. The halving bits of the running example relax this requirement in a precise way\.

###### Property 4\(What shared shifts allow\)\.

In a binary tree of shift bits, two nonzero entries whose closest common node lieshhlevels above them can differ in magnitude by at most a factor of2h2^\{h\}\. Every pattern of magnitudes that obeys this rule at every node, within the range that the depth of the tree allows, can be reached \([PropositionA\.9](https://arxiv.org/html/2610.31931#A1.Thmtheorem9)\)\.

Neighbours in the tree must therefore have similar magnitudes, which is why the order of the rows matters, and why a richer leaf alphabet or a second tree is used where neighbours differ a lot\. The order matters for bits only; additions do not depend on it, since shifts are free in a fixed circuit\.[PropositionA\.9](https://arxiv.org/html/2610.31931#A1.Thmtheorem9)also states what a good order must achieve: within every node, the exponents of the nonzero entries may spread by no more than the height of the node\. Splitting the rows recursively into balanced halves of similar magnitude is a natural way to approach it\. The rule is the same whether the shift bits sit on the edges of the tree, with leaves\{0,±1\}\\\{0,\\pm 1\\\}, or on its nodes, with leaves\{0,±1,±12\}\\\{0,\\pm 1,\\pm\\frac\{1\}\{2\}\\\}as in the running example: the two encodings describe exactly the same columns, up to one factor per column \([PropositionA\.10](https://arxiv.org/html/2610.31931#A1.Thmtheorem10)\)\. Yet the second costs about 3\.3 instead of 3\.6 bits per weight, with leaves coded jointly in both\. The saving comes from merging the shift on the edge into each leaf with the leaf’s own alphabet: a zero needs no shift, so the six combinations of a shift bit and a three\-state leaf collapse into five values\. The same matrices can thus be described at different prices, and finding the cheaper description is part of choosing a tree\. As a small proof of concept, we used the tree of the running example as a post\-training quantizer for the MLP layers of LLaMA\-3\-8B, with one balanced tree per column over rows ordered by recursive bisection of their magnitude profiles, and states chosen by an exact dynamic program per column with GPTQ\-style error feedback: at 3\.33 bits per weight it reached a WikiText\-2 perplexity of 7\.78, against 8\.03 for GPTQ at 3\.25 bits,111Here only the MLP layers are quantized, and GPTQ uses groups of 128 weights\.[Table6](https://arxiv.org/html/2610.31931#S9.T6)quantizes all linear layers of the transformer blocks with the strongest GPTQ configuration, which gives 8\.09\.and the edge\-bit encoding produced identical weights at 3\.60 bits, as the equivalence of the two encodings predicts\.

### 3\.5Beyond states: rank and sharing

The number of states is one dial along which a state tree grows\. The other two operations of[Section2](https://arxiv.org/html/2610.31931#S2)provide two more \([Figure4](https://arxiv.org/html/2610.31931#S3.F4)b,c\)\.*Rank\.*A sum ofrrproducts of a column and a row describes exactly the matrices of rank at mostrr, so full rank describes every matrix\. When the error is measured on the outputs of the layer, the best low\-rank tree has a closed form, a singular value decomposition in a reweighted space, a classical result of reduced\-rank regression\[[32](https://arxiv.org/html/2610.31931#bib.bib32)\]that SVD\-LLM\[[84](https://arxiv.org/html/2610.31931#bib.bib84)\]recovers \([TheoremA\.11](https://arxiv.org/html/2610.31931#A1.Thmtheorem11)\)\.*Sharing\.*Several matrices with the same input size, in one layer or in different layers, can refer to one shared subtree that is stored once \([Figure4](https://arxiv.org/html/2610.31931#S3.F4)c\)\. The more the shared part contains, the more the matrices have in common, and a large enough shared part describes them exactly \([PropositionA\.12](https://arxiv.org/html/2610.31931#A1.Thmtheorem12)\)\. With real\-valued factors, each of these two dials on its own reaches every matrix\. When the factors are themselves written as state trees, the three dials combine, and a single description can trade states, rank and sharing against one another\.

valueflat 3\-bit grid8 states−1\-10011shift tree, depth 3values±2−k\\pm 2^\{\-k\}and00sum of 2 shift trees17 levels,≤2\\leq 2terms\(a\)States: the values one entry can take\.\+\+×\\timesu1u\_\{1\}v1⊤v\_\{1\}^\{\\\!\\top\}×\\timesu2u\_\{2\}v2⊤v\_\{2\}^\{\\\!\\top\}×\\timesuru\_\{r\}vr⊤v\_\{r\}^\{\\\!\\top\}⋯\\cdots\(b\)Rank:W^=∑k≤ruk​vk⊤\\widehat\{W\}=\\sum\_\{k\\leq r\}u\_\{k\}v\_\{k\}^\{\\top\}\.×\\timesW1W\_\{1\}\(layer 3\)×\\timesW2W\_\{2\}\(layer 3\)×\\timesW3W\_\{3\}\(layer 17\)C1C\_\{1\}C2C\_\{2\}C3C\_\{3\}shared subtree \(stored once\)\(c\)Sharing: several matrices refer to one stored subtree\.
Figure 4:Three dials along which state trees grow\. \(a\) Adding states: a flat grid; a shift tree, whose values form a logarithmic grid; and a sum of two shift trees, which reaches the uniform 17\-level grid of all multiples of18\\frac\{1\}\{8\}in\[−1,1\]\[\-1,1\], since each of them is a sum of at most two signed powers of two with exponents down to−3\-3\([Property3](https://arxiv.org/html/2610.31931#Thmproperty3)\)\. \(b\) Adding rank\. \(c\) Sharing a subtree among several matrices, possibly in different layers\. With real\-valued factors, rank or sharing alone reaches every matrix\.#### Summary\.

State trees lose nothing that today’s formats offer, they lose nothing in the worst case, they reach any precision with shifts and additions, and they make the price of sharing explicit\. What they offer in return is a single object on which the many methods of model compression can be expressed and compared, which is the subject of section 5\.

## 4Building a tree, step by step

The previous sections described state trees in general\. In this section, we illustrate how to build one in full, on a small matrix, so that every decision can be followed\. The tree we build is the one behind our lossless format \([Section9](https://arxiv.org/html/2610.31931#S9)\): it stores BF16 weights exactly, in fewer bits, and a GPU decodes it directly into the registers of its tensor cores\. Every choice below is made with that decoder in mind\.

0\.0391\\phantom\{\-\}0\.0391−0\.0234\-0\.02340\.0352\\phantom\{\-\}0\.0352−0\.0137\-0\.0137−0\.0703\-0\.07030\.0215\\phantom\{\-\}0\.02150\.1055\\phantom\{\-\}0\.1055−0\.0352\-0\.03520\.0195\\phantom\{\-\}0\.0195−0\.0088\-0\.0088−0\.0469\-0\.04690\.0283\\phantom\{\-\}0\.02830\.1875\\phantom\{\-\}0\.1875−0\.0156\-0\.01560\.0410\\phantom\{\-\}0\.0410−0\.0625\-0\.0625g1g\_\{1\}g2g\_\{2\}g3g\_\{3\}g4g\_\{4\}inputsj⟶j\\;\\longrightarrowoutputsii2−32^\{\-3\}2−42^\{\-4\}2−52^\{\-5\}2−62^\{\-6\}2−72^\{\-7\}octave\(a\)A4×44\\times 4matrix in BF16 \(values rounded\); shading shows each weight’s octave\.\(b\)Row 1: each weight’s three BF16 decisions, and what the tree keeps of them\.
0\+1\.25\+1\.251−1\.5\-1\.50\+1\.125\+1\.1252−1\.75\-1\.750−1\.125\-1\.1252\+1\.375\+1\.3750\+1\.6875\+1\.68751−1\.125\-1\.1251\+1\.25\+1\.252−1\.125\-1\.1250−1\.5\-1\.51\+1\.8125\+1\.81250\+1\.5\+1\.53−1\.0\-1\.02\+1\.3125\+1\.31251−1\.0\-1\.02−52^\{\-5\}row 1==groupg1g\_\{1\}2−42^\{\-4\}row 2==groupg2g\_\{2\}2−52^\{\-5\}row 3==groupg3g\_\{3\}2−32^\{\-3\}row 4==groupg4g\_\{4\}WWpath ofw14w\_\{14\}:2−5⋅2−2⋅\(−1\.75\)=−0\.013671875\\;2^\{\-5\}\\cdot 2^\{\-2\}\\cdot\(\-1\.75\)=\-0\.013671875shape ofWW: fixed, not storedshared exponentEE: 8 bits per groupstateDD, a shift by2−D2^\{\-D\}: 4 bits per weightleaf: sign and mantissa, 8 bits per weight\(c\)The state tree of the whole matrix\. The path ofw14w\_\{14\}\(red\) multiplies the shared exponent, the shift of its state and its leaf, and returns the BF16 value exactly\.
Figure 5:Building the tree of our lossless format\. \(a\) Weights of a trained layer lie within a few octaves of one another, but rarely share an exponent exactly\. \(b\) Each group keeps its largest exponent once; every weight keeps only its distanceDDbelow it, and its sign and mantissa\. \(c\) The resulting tree has a fixed shape: one node per group, one state and one leaf per weight\. Blue: shared decisions; orange: private decisions\. The example uses groups of 4 weights; the format uses groups of 64\.#### Step 1: every weight is a path\.

[Figure5](https://arxiv.org/html/2610.31931#S4.F5)a shows a4×44\\times 4matrix in BF16\. As in[Section2](https://arxiv.org/html/2610.31931#S2), each weight is read as three decisions: a signss, an 8\-bit exponent fieldeeand a 7\-bit mantissamm, withw=\(−1\)s⋅2e−127⋅\(1\+m/128\)w=\(\-1\)^\{s\}\\cdot 2^\{\\,e\-127\}\\cdot\(1\+m/128\)\. The shading shows the octave2e−1272^\{e\-127\}of every weight\. Within a row the weights lie within a few octaves of one another, as trained weights usually do, but their exponents rarely coincide, so the single shared exponent of[Figure2](https://arxiv.org/html/2610.31931#S2.F2)cannot be used as it stands\.

#### Step 2: choose the groups\.

The tree groups the four weights of every row in their natural order; in the format a group holds 64 consecutive weights of a row\. Groups run along the input dimension for two reasons\. Their weights are multiplied by consecutive inputs and summed into the same output, so in a circuit the shared factor of a group could be applied once to its partial sum, as in[Figure2](https://arxiv.org/html/2610.31931#S2.F2)c\. And they lie next to one another in memory, in the order in which a tensor core reads them\. Because the order is the natural one, no permutation has to be stored or applied\.

#### Step 3: lift the shared decision\.

Every group receives a node that holds its largest exponent fieldEE, stored once in 8 bits\. Below it, every weight keeps only its distance to that exponent, the stateD=E−eD=E\-e, which stands for a shift by2−D2^\{\-D\}and takes 4 bits, and a leaf that holds its sign and mantissa in 8 bits \([Figure5](https://arxiv.org/html/2610.31931#S4.F5)b\)\. Along every path the product is unchanged,

w=2E−127⋅2−D⋅\(−1\)s​\(1\+m128\),w\\;=\\;2^\{\\,E\-127\}\\cdot 2^\{\-D\}\\cdot\(\-1\)^\{s\}\\Big\(1\+\\frac\{m\}\{128\}\\Big\),so the tree reproduces every weight exactly\. In row 1,EEencodes2−52^\{\-5\}and the four states are 0, 1, 0 and 2; the red path in[Figure5](https://arxiv.org/html/2610.31931#S4.F5)c decodesw14w\_\{14\}\. Unlike the shared exponent of[Section2](https://arxiv.org/html/2610.31931#S2), this node does not require the exponents of a group to be equal: the states absorb their differences, which is the second kind of shared decision anticipated there\.

#### Step 4: set aside the rare outliers\.

A 4\-bit state distinguishes 16 values\. The states 0 to 14 are distances, so the weights of a group may spread over 15 octaves\. A weight that lies further below the largest of its group, an exact zero for instance, takes the state 15 as an escape: its leaf is left empty, and its 16 bits are kept exactly in a short side list\. No weight of the example needs an escape; in the models we tested, about one weight in ten thousand does \([SectionA\.4](https://arxiv.org/html/2610.31931#A1.SS4)\)\.

#### Step 5: count the bits\.

A group stores8\+4×\(4\+8\)=568\+4\\times\(4\+8\)=56bits for its four weights, against 64 in BF16\. As with the shared exponent of[Section2](https://arxiv.org/html/2610.31931#S2), the saving grows with the group: with 64 weights a group needs8/64\+12=12\.1258/64\+12=12\.125bits per weight, and with the escapes, whole models take 12\.13 bits per weight, 24% less than BF16, with every weight bit\-exact\.

0111101000000001000000100 01000001 10000000 001000011100000EED1D\_\{1\}L1L\_\{1\}D2D\_\{2\}L2L\_\{2\}D3D\_\{3\}L3L\_\{3\}D4D\_\{4\}L4L\_\{4\}byte 0shared exponentbytes 1–2states, 4 bits eachbytes 3–6leaves: sign and mantissa, 8 bits each−\-122−2=120122\-2=1201011110001100000BF16 pattern ofw14w\_\{14\}=0xBC60=−2−7⋅1\.75=−0\.013671875=\\texttt\{0xBC60\}=\-2^\{\-7\}\\cdot 1\.75=\-0\.013671875copiedfromL4L\_\{4\}bf16 = \(s << 15\) \| \(\(E \- D\) << 7\) \| m, written directly into the tensor\-core registers;DjD\_\{j\}lies at byte1\+⌊\(j−1\)/2⌋1\+\\lfloor\(j\-1\)/2\\rfloorandLjL\_\{j\}at byte2\+j2\+j: fixed offsets, no pointers\.Figure 6:The stored record of groupg1g\_\{1\}and the decoding ofw14w\_\{14\}\. Every field has a fixed width at a fixed offset, so the address of any weight’s state and leaf follows from its index alone\. Decoding subtracts the state from the shared exponent and copies the sign and mantissa from the leaf\. In the format, a block of 16 rows and 64 columns, the shape the kernel feeds to the tensor cores, is one record of16×97=1,55216\\times 97=1\{,\}552bytes\.
#### Step 6: decode without pointers\.

[Figure6](https://arxiv.org/html/2610.31931#S4.F6)shows how the first group is stored\. Every field has a fixed width at a fixed offset: the shared exponent in the first byte, the four states in the next two and the four leaves in the last four\. A thread that needs weightjjof a group computes the addresses ofDjD\_\{j\}andLjL\_\{j\}fromjjalone; it never reads a pointer and never branches, except at the rare escape\. Rebuilding the BF16 pattern takes a subtraction and two copies: the exponent field isE−DjE\-D\_\{j\}, and the sign and mantissa come from the leaf\. The result is written directly into the registers from which the tensor cores read, so the weights never travel through memory in BF16\. This is what lets the kernel of[Section9](https://arxiv.org/html/2610.31931#S9)run at the speed of memory\.

#### Why a tree, and not something smaller\.

The tree we have built is not the shortest possible description of the matrix\. Groups whose subtrees coincide could be stored once, turning the tree into a graph; prefixes shared by different groups could be merged, as in a trie; weights could be grouped by magnitude rather than by position; and the states could be entropy\-coded, since small distances are far more frequent than large ones\. Each of these would lower the bits further, but each needs pointers, indices or codes of variable length that a decoder must follow before it can locate a weight, and that breaks the fixed addressing that lets a tensor core read its weights at full speed\. We therefore keep the fixed\-shape tree and leave these richer representations to future work\. We do not claim that this tree, or trees in general, are the optimal representation of a weight matrix\. We claim that the tree improves on the representations in use in several respects at once: it stores exactly the same values in fewer bits, it decodes at the speed of memory, and it makes explicit the structure that the following sections edit, wire and optimize\.

#### From exact to compressed\.

The same tree also shows what compression does\. Keeping only the leading three bits of every mantissa shrinks the leaf alphabet from 256 to 16 values and the leaf from 8 to 4 bits\. The tree keeps its shape, but now describes an approximation of the matrix rather than the matrix itself\. Every compression method can be read as an edit of this kind, which is where the next section begins\.

## 5Compression is editing one tree

Model compression has grown into several families \(pruning, quantization, low\-rank factorization, codebooks and weight sharing\), each with its own format, its own solver and often its own kernel\. Combining them is hard precisely because their formats differ\. In a state tree they become something simpler: different ways of editing one object\.

### 5\.1Every method is an edit

The two weights of[Figure2](https://arxiv.org/html/2610.31931#S2.F2)already show most of the edits\. Replacing the leaf ofw2w\_\{2\}by the state00prunesw2w\_\{2\}; a zero state at the shared node removes both weights at once, which is structured pruning in miniature\. Shortening the mantissas is quantization:1\.251\.25and1\.51\.5use only two of their ten mantissa bits, so for these two weights a two\-bit alphabet loses nothing\. The shared exponent itself is an inserted node, the same edit that produces group scales and block formats\. Writing1\.251\.25as1\+2−21\+2^\{\-2\}splits a leaf into a sum, which is how residual corrections and additive codebooks work\. The remaining edits act on whole matrices: factoring a tree into a product gives low rank and rotations, a reference to a shared part gives codebooks and weight sharing across layers, and a selector that chooses among several subtrees gives mixed precision and the choice of a method for each matrix\.[Table3](https://arxiv.org/html/2610.31931#S5.T3)lists these seven edits, and[Figure7](https://arxiv.org/html/2610.31931#S5.F7)draws the most common ones\. Common compression methods, including all those cited in the table, are combinations of them, together with a rule for choosing the new states\.

Table 3:Seven edits of a state\-tree graph\. Common compression methods, including those cited here, are combinations of these edits, together with a rule for choosing the new states\.sgs\_\{g\}3399001414\(a\)Quantization: group scale node,2b2^\{b\}\-level leaves\.11w1w\_\{1\}0000w4w\_\{4\}\(b\)Unstructured pruning: zero leaf states\.1111wwww00\(c\)Structured pruning: one zero state removes a subtree\.2:42\{:\}400wwww00\(d\)N:MN\{:\}M: the group node chooses which entries are nonzero\.\+\+×\\timesu1u\_\{1\}v1⊤v\_\{1\}^\{\\\!\\top\}×\\timesuru\_\{r\}vr⊤v\_\{r\}^\{\\\!\\top\}\(e\)Low rank: sum of rank\-one product nodes\.\+\+×\\timesUUVVss00qq\(f\)Low rank plus a sparse or quantized residual\.×\\times×\\timesC1C\_\{1\}C2C\_\{2\}sharedW1W\_\{1\}W2W\_\{2\}\(g\)Weight sharing: two matrices refer to one stored subtree\.ss\#​5\\\#5\#​2\\\#2\#​5\\\#5codebook\{ck\}⊂ℝd\\\{c\_\{k\}\\\}\\subset\\mathbb\{R\}^\{d\}\(h\)Vector quantization: block indices into a shared codebook\.
Figure 7:Compression methods as edits of a state\-tree graph\. Grey: zero states; green: sum \(\+\+\) and product \(×\\times\) nodes; purple: references to shared subtrees\.
### 5\.2Methods differ only in how they choose states

Seen this way, methods within one family share a representation and differ only in how they choose the states\. Magnitude pruning, Wanda\[[70](https://arxiv.org/html/2610.31931#bib.bib70)\], SparseGPT\[[23](https://arxiv.org/html/2610.31931#bib.bib23)\]and ALPS\[[54](https://arxiv.org/html/2610.31931#bib.bib54)\]all produce zero states at the leaves; they differ in their criterion and their solver\. Round\-to\-nearest, GPTQ\[[22](https://arxiv.org/html/2610.31931#bib.bib22)\]and AWQ\[[47](https://arxiv.org/html/2610.31931#bib.bib47)\]all choose leaf states under group\-scale nodes; again, only the rule for choosing differs\. Two consequences follow\. Methods of the same family can be compared on equal terms, since they produce the same kind of object\. And methods from different families can be combined without inventing a new format: a quantized low\-rank factor, or a pruned tree that reads from a shared codebook, is simply a deeper tree\.

### 5\.3Pricing an edit

To choose among edits, each one needs a price in accuracy\. Because every edit changes the same kind of object, a single error measure can price all of them\. The standard choice measures the error on the outputs of the layer: with calibration inputsx1,…,xN∈ℝnx\_\{1\},\\dots,x\_\{N\}\\in\\mathbb\{R\}^\{n\}, taken as column vectors, their second momentH=1N​∑νxν​xν⊤H=\\frac\{1\}\{N\}\\sum\_\{\\nu\}x\_\{\\nu\}x\_\{\\nu\}^\{\\top\}and the errorE=W−W^E=W\-\\widehat\{W\},

fH​\(W^\)=tr⁡\(E​H​E⊤\)=1N​∑ν=1N‖E​xν‖2\.f\_\{H\}\(\\widehat\{W\}\)\\;=\\;\\operatorname\{tr\}\\big\(EHE^\{\\top\}\\big\)\\;=\\;\\frac\{1\}\{N\}\\sum\_\{\\nu=1\}^\{N\}\\big\\\|Ex\_\{\\nu\}\\big\\\|^\{2\}\.\(2\)Correlated inputs make the errors in different columns interact, which is what solvers such as GPTQ exploit\. The outputs of a layer do not matter equally to the loss either: the second momentGGof the loss gradients at the outputs records which output directions matter, and it adds an interaction between rows in the Kronecker\-factored formtr⁡\(G​E​H​E⊤\)\\operatorname\{tr\}\(GEHE^\{\\top\}\)\[[53](https://arxiv.org/html/2610.31931#bib.bib53),[78](https://arxiv.org/html/2610.31931#bib.bib78)\]\. When such proxies are not accurate enough, the price of an edit can be measured directly, by running the network with the edit applied\.

### 5\.4One choice per matrix, one budget for the network

With every method expressed as an edit and every edit priced, compressing a network becomes a single problem\. Each matrix receives a menu of edits, covering all methods and all settings, each with a size and an accuracy cost, and the task is to choose one edit per matrix so that the total size fits a budget \([Figure8](https://arxiv.org/html/2610.31931#S5.F8)\)\.

###### Property 5\(One optimizer\)\.

If the accuracy costs of different matrices add up, the best choice is found by dynamic programming over the Pareto sets of the menus, exactly up to the resolution of the size grid, and it is never worse than applying any single method, or any uniform setting, that fits the same budget\. When the candidate groups for shared subtrees are nested or disjoint, a recursion over the groups keeps the search exact \([TheoremsA\.13](https://arxiv.org/html/2610.31931#A1.Thmtheorem13)and[A\.14](https://arxiv.org/html/2610.31931#A1.Thmtheorem14)\)\.

Restricted to one family, this is the per\-layer allocation that SPDY solves by dynamic programming\[[21](https://arxiv.org/html/2610.31931#bib.bib21)\]and HAWQ\-V3 by integer programming\[[87](https://arxiv.org/html/2610.31931#bib.bib87)\]; what the tree adds is that one menu can hold edits of every family\. The guarantee holds with respect to the cost model\. In practice, the costs of edits in different matrices are not exactly additive, and aggressive edits interact\. A simple procedure keeps the exact search honest: build and measure the chosen edits together, update their costs, and solve again until the choice stops changing;[SectionA\.4](https://arxiv.org/html/2610.31931#A1.SS4)describes the version used in our experiments\.

sizeaccuracy costmenu of one matrix:∙\\bulletPareto,∘\\circdominated2\-bit4\-bit8\-bitpruned\+3\-bitvvW1W\_\{1\}W2W\_\{2\}W3W\_\{3\}W4W\_\{4\}menu ofvv: its children’s menus combined,or one subtree shared belowvvcandidate group for a shared subtreeFigure 8:Left: the menu of one matrix in the \(size, accuracy cost\) plane; only its Pareto set enters the optimizer \([Property5](https://arxiv.org/html/2610.31931#Thmproperty5)\)\. Right: nested candidate groups for shared subtrees; every group offers either its children’s combined menus or one subtree shared by all of its matrices, and keeping Pareto sets at every node keeps the search exact\. Points are illustrative\.Finally, the budget need not be counted in bytes\. Additions, latency or energy can take its place without changing the algorithm, which turns the optimizer into a compiler for a given hardware target\. For state trees the most natural of these budgets is the number of additions, and it leads to the second reading of the tree: as a circuit\.

## 6From trees to circuits

[Figure2](https://arxiv.org/html/2610.31931#S2.F2)c computed two weighted inputs with three additions and no multiplication\. This section scales that picture up: first to a layer, then to a whole network, and finally to a first measurement on real weights\. The motivation is simple\. In digital logic a multiplier takes considerably more area and energy than an adder of the same width\[[30](https://arxiv.org/html/2610.31931#bib.bib30)\], and a state tree whose states are powers of two needs no multiplier for its weights at all\.

### 6\.1A layer becomes wires and adders

Suppose every state of a tree is a power of two, a short sum of powers of two, or zero\. Then[Equation1](https://arxiv.org/html/2610.31931#S2.E1)turns the producty=W^​xy=\\widehat\{W\}xinto

yi=ρi∑j=1n∑τ=1ti​j±\(\(βjxj\)≫ki​j​τ\),y\_\{i\}\\;=\\;\\rho\_\{i\}\\sum\_\{j=1\}^\{n\}\\ \\sum\_\{\\tau=1\}^\{t\_\{ij\}\}\\ \\pm\\big\(\(\\beta\_\{j\}x\_\{j\}\)\\gg k\_\{ij\\tau\}\\big\),\(3\)whereti​jt\_\{ij\}is the number of power\-of\-two terms in entry\(i,j\)\(i,j\)and, as in[Figure2](https://arxiv.org/html/2610.31931#S2.F2),≫k\\gg kshifts a number bykkbinary positions, that is, multiplies it by2−k2^\{\-k\}\(a negativekkshifts to the left\)\. In a circuit whose weights are fixed,[Equation3](https://arxiv.org/html/2610.31931#S6.E3)is a network of wires and adders \([Figure9](https://arxiv.org/html/2610.31931#S6.F9)\)\. A shift is a re\-labelling of wires and a sign is an inverted adder input, so both are free; a zero state is simply a missing wire; and every term beyond the first in an output costs one adder\. The only genuine multiplications left are thennby the column bases and themmby the row scales of anm×nm\\times nmatrix\. They can be folded into neighbouring layers where no residual addition or nonlinearity intervenes, and otherwise remain;[Section6\.5](https://arxiv.org/html/2610.31931#S6.SS5)charges them in full\.

x1x\_\{1\}x2x\_\{2\}x3x\_\{3\}β1\\beta\_\{1\}β2\\beta\_\{2\}β3\\beta\_\{3\}\+\+\+\+y1y\_\{1\}y2y\_\{2\}ρ1\\rho\_\{1\}ρ2\\rho\_\{2\}≫1\\gg 1−\-≫2\\gg 2≫1\\gg 1≫3\\gg 3W^=diag⁡\(ρ\)​\(2−1−1002−22−1\+2−3\)​diag⁡\(β\)\\widehat\{W\}=\\operatorname\{diag\}\(\\rho\)\\begin\{pmatrix\}2^\{\-1\}&\-1&0\\\\ 0&2^\{\-2\}&2^\{\-1\}\+2^\{\-3\}\\end\{pmatrix\}\\operatorname\{diag\}\(\\beta\)solid: one power\-of\-two term per entry;dashed: second term of entry\(2,3\)\(2,3\);W^13=0\\widehat\{W\}\_\{13\}=0: no wire;adders: one per arrow beyond the firstinto each sum; multipliers: onlyβj\\beta\_\{j\},ρi\\rho\_\{i\}\.Figure 9:A layer as a circuit\. When the states are powers of two, multiplications by weights become shifts and signs, which are wiring; zeros are missing wires; and each additional power\-of\-two term costs one adder\. The only multiplications left are by the column bases and the row scales\.
### 6\.2Where the circuits pay off

The gain is largest where weights are fixed at fabrication: printed and bespoke classifiers\[[57](https://arxiv.org/html/2610.31931#bib.bib57)\], and chips that hardwire an entire language model into silicon\[[71](https://arxiv.org/html/2610.31931#bib.bib71)\]\. There, a weight with few nonzero digits is literally a few wires, and the tree states which ones\. It is worth being precise about the source of this gain\. A general multiplier is itself a shift\-and\-add circuit: Booth recoding writes one operand in signed digits and adds shifted copies of the other\. A state tree makes the same structure explicit and tailors it to the actual weights: in a circuit with fixed weights, zero digits disappear entirely, and the tree can choose weights with few nonzero digits wherever accuracy allows\. The same structure serves other targets as well\. In programmable hardware, stored states select shifts and signs through small multiplexers, which take area of their own, so the saving over a general multiplier is smaller there, although a shift stored at a node is selected once for its whole group rather than for every weight, so deeper trees also save multiplexers; in software, lookup\-table kernels\[[61](https://arxiv.org/html/2610.31931#bib.bib61),[85](https://arxiv.org/html/2610.31931#bib.bib85)\]replace multiplications by table reads\. Throughout, we assume fixed\-point activations\. With floating\-point activations, adders must first align exponents, which remains cheap when the activations share an exponent per block, as in the microscaling formats\.

### 6\.3A whole network

[Table4](https://arxiv.org/html/2610.31931#S6.T4)goes through the components of common networks\. Fully connected layers are exactly[Equation3](https://arxiv.org/html/2610.31931#S6.E3), with biases as constant adder inputs\. A convolution applies one kernel matrix at every position\[[8](https://arxiv.org/html/2610.31931#bib.bib8)\], so it is one shared subtree referenced at every position; its circuit is either replicated per position or built once and reused over time\. Fast convolution algorithms are products with fixed transforms, and for Winograd’sF⁡\(2×2,3×3\)F\(2\\times 2,3\\times 3\)\[[42](https://arxiv.org/html/2610.31931#bib.bib42)\]all three transforms have entries in\{0,±12,±1\}\\\{0,\\pm\\frac\{1\}\{2\},\\pm 1\\\}, so they are shift trees themselves\. The four projections of attention\[[80](https://arxiv.org/html/2610.31931#bib.bib80)\]are fully connected layers, and the query, key and value projections read the same input, so they can share partial sums\. Batch normalization folds into neighbouring layers at inference\[[33](https://arxiv.org/html/2610.31931#bib.bib33)\], and the gains of layer and RMS normalization fold into the column bases of the next layer\. What remains outside the tree is small and well delimited: products of two activations, such as attention scores and the gate of a gated MLP, which are not weights and still need multipliers; and a few nonlinear functions, for which integer\-only approximations are well studied\[[36](https://arxiv.org/html/2610.31931#bib.bib36)\]\.

Table 4:How the components of common networks map onto state trees and circuits\. Every product of a weight and an activation becomes shifts and additions; what remains are products of two activations and a few nonlinear functions\.
### 6\.4Sharing additions, and their limit

Different outputs often need the same partial sum of shifted inputs, and can then share the adder that forms it\. Finding such shared sums is the constant matrix–vector multiplication problem, a generalization of multiple constant multiplication\[[81](https://arxiv.org/html/2610.31931#bib.bib81)\]\. Entries below a common node already share their shifts, so the tree is a natural starting point for this search\. There is also a limit, and it mirrors the one for storage in[Section3](https://arxiv.org/html/2610.31931#S3)\. For matrices without structure, any circuit of additions and bounded shifts needs on the order ofn2​b/log⁡\(n​b\)n^\{2\}b/\\log\(nb\)operations for mostn×nn\\times nmatrices ofbb\-bit integers, and, whenbbis not exponentially larger thannn, a classical bucketing construction comes within a constant factor of this\[[4](https://arxiv.org/html/2610.31931#bib.bib4),[45](https://arxiv.org/html/2610.31931#bib.bib45)\]\([PropositionA\.15](https://arxiv.org/html/2610.31931#A1.Thmtheorem15)\)\. Large savings in additions, as in bits, come from structure, and the tree is where structure is recorded\.

### 6\.5A first measurement

Does choosing states for circuit cost pay off on real weights? We compared state trees with GPTQ\[[22](https://arxiv.org/html/2610.31931#bib.bib22)\]on five linear layers of LLaMA\-3\-8B, using 128 calibration sequences from FineWeb\. GPTQ is designed for storage at a fixed bit width, not for additions, so the comparison asks how many additions a tree saves when it is allowed to target them\. We use the flattest tree on purpose: a*flat signed\-digit tree*has one scale per row and leaves that are short sums of signed powers of two\. This isolates the circuit price; deeper trees mainly pay off in bits\. Both methods are costed by the same rule, one adder per nonzero signed digit of a weight and four adders per real\-valued scale, which counts one adder per output more than the circuit of[Equation3](https://arxiv.org/html/2610.31931#S6.E3), equally for both\. For GPTQ the rule is applied to its quantized weights and group scales\. For the tree, we choose the states to minimize the layer errorfHf\_\{H\}of[Equation2](https://arxiv.org/html/2610.31931#S5.E2)plus a price for every nonzero digit, and adjust the price until the layer error equals that of GPTQ at 4, 3 and 2 bits\. The states are chosen in a single GPTQ\-style pass: column by column, every weight becomes zero or one to three terms±2e\\pm 2^\{e\}times its row scale, witheefrom11down to−11\-11, whichever adds least to the layer error plus the price of its digits, and the remaining columns receive the usual error update\. The choice separates over the weights of a column, so each column is solved exactly given the columns before it\. The row scale starts at the largest magnitude in its row and is refitted by least squares in the metric ofHHafter the pass\. The weights of the tree are hardwired, so its storage is not counted\. GPTQ is run in its plain form, with an asymmetric min–max grid for every group of 128 weights, no clipping or grid search, natural column order and 1% damping; its digits are counted in the non\-adjacent form of the signed codeq−zq\-z\.

Table 5:Adders per weight at equal layer error on LLaMA\-3\-8B: GPTQ→\\toa flat state tree whose states were chosen for circuit cost\. The last two matrices have strong outliers\. Counts are rounded to two decimals; percentages are computed from the unrounded counts\.[Table5](https://arxiv.org/html/2610.31931#S6.T5)shows the result\. At equal layer error, the tree needs 9–15% fewer adders than GPTQ at 3\- and 4\-bit accuracy on the three typical matrices, and 30–35% fewer at 2\-bit accuracy, a regime in which GPTQ itself already degrades the model severely, so that the 3\- and 4\-bit columns are the practical operating points\. The gains are largest on the two matrices with strong outliers, up to 55% at 3\-bit and 63% at 2\-bit accuracy on the down projection of the first layer\. Part of the saving comes from the scales, since GPTQ needs a scale for every group of 128 weights and the tree one per row\. Counting weight digits alone, the tree still needs 6–7% fewer adders on the typical matrices at 4\-bit accuracy, 9–11% at 3\-bit and 23–29% at 2\-bit accuracy, and up to 53% and 59% on the outlier matrices at 3\- and 2\-bit accuracy\. We attribute this to where the digits go: a uniform grid gives every weight the same number of digits, whereas the tree spends digits where they reduce the error most\. A second observation concerns the shape of the tree, and it confirms what the two\-weight example of[Section2](https://arxiv.org/html/2610.31931#S2)suggested\. In our trials, the flattest tree was also the best one for additions: the shared exponents of deeper trees saved stored bits but not additions, because a shift costs nothing in a circuit with fixed weights\. Bits and additions are two different prices, and the representation makes both visible, so that the optimizer of[Section5](https://arxiv.org/html/2610.31931#S5)can trade one against the other\. Counting additions leaves out their width, which grows with the spread of the shifts that an adder combines, and the truncation of low\-order bits by right shifts in fixed point\. The counts are therefore a first\-order proxy for circuit size, not a measurement of area\. The flat tree also differs from earlier multiplier\-free quantizers\. APoT\[[44](https://arxiv.org/html/2610.31931#bib.bib44)\]draws its levels from a fixed additive power\-of\-two grid shared by a whole layer and learned with quantization\-aware training, and ShiftAddLLM\[[89](https://arxiv.org/html/2610.31931#bib.bib89)\]writes weights as sums of binary matrices with power\-of\-two group scales; the flat tree chooses the number and position of the signed digits separately for every weight and prices each digit against the layer error, so a zero costs nothing and an outlier costs one digit\. A full study of circuits for whole networks, including area, energy, these effects and measured comparisons with quantizers that also target additions\[[44](https://arxiv.org/html/2610.31931#bib.bib44),[18](https://arxiv.org/html/2610.31931#bib.bib18),[89](https://arxiv.org/html/2610.31931#bib.bib89)\], is left to a separate paper\.

## 7Choosing states is finding a ground state

In the previous two sections the states of a tree were chosen by some solver, either for accuracy at a given size or for accuracy at a given number of additions\. It is worth asking what kind of problem these solvers face\. The answer is precise and well studied: choosing the states of a tree is the problem of finding the lowest\-energy configuration of a Boltzmann machine\.

### 7\.1Boltzmann machines

A Boltzmann machine\[[1](https://arxiv.org/html/2610.31931#bib.bib1)\]consists ofNNbinary unitsz1,…,zNz\_\{1\},\\dots,z\_\{N\}, each either off \(zp=0z\_\{p\}=0\) or on \(zp=1z\_\{p\}=1\); a*configuration*z=\(z1,…,zN\)z=\(z\_\{1\},\\dots,z\_\{N\}\)fixes the state of every unit\. Each configuration has an energy

ℰ⁡\(z\)=∑p<qJp​q​zp​zq\+∑php​zp,\\mathcal\{E\}\(z\)\\;=\\;\\sum\_\{p<q\}J\_\{pq\}\\,z\_\{p\}z\_\{q\}\+\\sum\_\{p\}h\_\{p\}\\,z\_\{p\},\(4\)where the indicesppandqqrun over the units\. The*coupling*Jp​qJ\_\{pq\}is the energy added when unitsppandqqare on together: a positive coupling discourages them from being on at the same time, and a negative one encourages it\. The*field*hph\_\{p\}is the energy added when unitppis on by itself\. The machine assigns each configuration the probabilityP\(z\)=exp\(−ℰ\(z\)/T\)/ZP\(z\)=\\exp\(\-\\mathcal\{E\}\(z\)/T\)/Z, where the*temperature*T\>0T\>0sets how strongly low energies are preferred, andZZ, the sum ofexp\(−ℰ\(z\)/T\)\\exp\(\-\\mathcal\{E\}\(z\)/T\)over all configurations, makes the probabilities add up to one\. AsTTfalls, the probability concentrates on the configurations of lowest energy, the*ground states*\. A higher\-order machine\[[68](https://arxiv.org/html/2610.31931#bib.bib68)\]also has terms that couple three or more units, such asJp​q​r​zp​zq​zrJ\_\{pqr\}\\,z\_\{p\}z\_\{q\}z\_\{r\}\. Finding a ground state of[Equation4](https://arxiv.org/html/2610.31931#S7.E4)is the problem known as QUBO \(quadratic unconstrained binary optimization\); written with spinsσp=2​zp−1∈\{−1,\+1\}\\sigma\_\{p\}=2z\_\{p\}\-1\\in\\\{\-1,\+1\\\}instead of bits, it is the Ising model\[[49](https://arxiv.org/html/2610.31931#bib.bib49)\]\.

### 7\.2A machine with two weights

Return to the two weights of[Figure2](https://arxiv.org/html/2610.31931#S2.F2), and suppose that their mantissas must be shortened, so that each weight has to choose between its two nearest representable values\. Each choice is a binary unit:z1z\_\{1\}forw1w\_\{1\}andz2z\_\{2\}forw2w\_\{2\}\. The natural energy is the layer error of[Equation2](https://arxiv.org/html/2610.31931#S5.E2)\. For the two inputsx1,x2x\_\{1\},x\_\{2\}it reads

fH=e12​H11\+2​e1​e2​H12\+e22​H22,f\_\{H\}\\;=\\;e\_\{1\}^\{2\}H\_\{11\}\\;\+\\;2\\,e\_\{1\}e\_\{2\}\\,H\_\{12\}\\;\+\\;e\_\{2\}^\{2\}H\_\{22\},wheree1e\_\{1\}ande2e\_\{2\}are the errors of the two weights, each affine in its own unit\. The first and last terms depend on one unit each\. The middle term couples them, with a strength set byH12H\_\{12\}, the correlation between the two inputs\. If the inputs are uncorrelated,H12=0H\_\{12\}=0and each weight can be rounded on its own, which is exactly what rounding to the nearest value does\. If they are correlated, the best choice for one weight depends on the choice made for the other: the two units are coupled, and the error is a small Boltzmann machine\. Now let the shared node take part as well, with a bitbbthat either keeps or halves both weights\. Each weight is then the product of the node’s factor and its leaf, so the error contains products ofbbwithz1z\_\{1\}andz2z\_\{2\}: terms that couple three units at once\. These higher\-order terms follow the path from the shared node to its leaves\. These two observations, couplings from the data and higher\-order terms from the tree, are all there is to the general case\.

### 7\.3The Boltzmann machine of a tree

Every state of a tree can be written with binary units\. A leaf withKKpossible values becomes a group ofKKunits of which exactly one is on\. A shift bit with multiplierrbr^\{b\}is affine in its bit,rb=1\+\(r−1\)​br^\{b\}=1\+\(r\-1\)b\. A gate multiplies its subtree by a bit, and a lookup index is again a group of units of which exactly one is on\. Every weight is then a polynomial in the units on its path, and the layer error becomes an energy\. One\-hot groups need a penalty that keeps exactly one unit on, but many alphabets need none: a leaf\{0,±1\}\\\{0,\\pm 1\\\}isz−z′z\-z^\{\\prime\}with two free units, zero having two codes, which leaves the energy flat between them and does no harm to the search, and an integer range has a binary encoding that is linear in its units\.

###### Property 6\(The Boltzmann machine of a tree\)\.

With the node states fixed, the layer error is a quadratic function of the leaf units: a pairwise Boltzmann machine in which leaves of the same row are coupled through the input statisticsHH, and, under the Kronecker\-factored error, leaves of different rows are also coupled through the output statisticsGG\. With the node states free as well, the error is a higher\-order Boltzmann machine whose higher\-order terms follow the paths of the tree; auxiliary units reduce it to a pairwise machine\[[6](https://arxiv.org/html/2610.31931#bib.bib6),[31](https://arxiv.org/html/2610.31931#bib.bib31)\]\. In both cases, for given row scales and column bases, the ground state is the best matrix the tree can represent \([TheoremA\.16](https://arxiv.org/html/2610.31931#A1.Thmtheorem16)\)\.

z1​jz\_\{1j\}z1​kz\_\{1k\}z2​jz\_\{2j\}z2​kz\_\{2k\}z3​jz\_\{3j\}z3​kz\_\{3k\}z4​jz\_\{4j\}z4​kz\_\{4k\}Hj​kH\_\{jk\}bbbbbbbbbbbbG23​Hj​kG\_\{23\}H\_\{jk\}columnjjcolumnkkorangeleaf units \(rows 1–4 of two columns\)circlesshared shift bits of each columnshadedone higher\-order term: the unitson the paths to\(1,j\)\(1,j\)and\(1,k\)\(1,k\)bluesame row, two columns: fromHHred dashedtwo rows: fromGG\(two\-sided error\)Figure 10:The Boltzmann machine of a state tree \([Property6](https://arxiv.org/html/2610.31931#Thmproperty6)\), drawn for two columns of a four\-row version of the running example\. The tree decides which units exist and where the higher\-order terms are; the data decide how strongly the units interact\.[Figure10](https://arxiv.org/html/2610.31931#S7.F10)shows the same division of labour as the two\-weight machine, now for a larger tree: the tree decides which units exist and where the higher\-order terms are, and the data decide how strongly the units interact\.

### 7\.4Familiar solvers, and new ones

Seen through this machine, familiar solvers are particular ways of lowering its energy\. Rounding to the nearest value ignores all couplings, as in the uncorrelated case above\. GPTQ visits the columns in turn and fixes each one given the columns already fixed, carrying their errors forward to the columns that remain; it is a single sweep at zero temperature that never revisits a decision, and, run in reverse column order and without clipping, it coincides with Babai’s nearest\-plane algorithm for lattices\[[5](https://arxiv.org/html/2610.31931#bib.bib5),[10](https://arxiv.org/html/2610.31931#bib.bib10)\]\. Adaptive rounding writes the rounding of a layer as a QUBO\[[59](https://arxiv.org/html/2610.31931#bib.bib59)\]; in the present language, this is a flat tree whose leaves choose between rounding up and rounding down, exactly the two\-weight machine without its shared node\.

The machine also suggests solvers that are rarely used for this problem: Gibbs sampling, simulated annealing\[[38](https://arxiv.org/html/2610.31931#bib.bib38)\], parallel tempering, and sampling at a finite temperature to obtain many good solutions instead of one\. Because the energy covers every kind of decision, including zeros, shared shifts, codebook indices and selectors, the same solver can choose all of them together\.

### 7\.5Quantum optimizers

Quantum annealers\[[34](https://arxiv.org/html/2610.31931#bib.bib34)\]and the quantum approximate optimization algorithm\[[19](https://arxiv.org/html/2610.31931#bib.bib19)\]search for low\-energy states of Ising models\. After the substitutionσ=2​z−1\\sigma=2z\-1and, when node states are free, a standard reduction to pairwise terms, the energy of a tree has exactly this form; running it on a device still requires an embedding of the couplings and their representation at finite precision\. A whole weight matrix has far more units than present devices have qubits, but the tree also supplies a decomposition: one group of rows and one block of columns at a time, with GPTQ\-style error feedback carrying the effect of each solved block to the rest\. Each block is a dense machine of tens to a few hundred units, the size handled by current annealers and small variational circuits, inside a classical loop that walks the tree\. These sizes hold when the node states of a block are fixed, so that its machine is pairwise from the start; freeing them adds auxiliary units and penalty couplings, which enlarge the machine and widen the range of couplings a device must represent\. A single large matrix splits into a very large number of such blocks, and whether quantum solvers will outperform classical ones on them is an open question\. What the representation provides is the problem in the form these devices accept, together with a decomposition that matches their size\.

The same tree helps in the opposite direction, when a network has to be evaluated inside a quantum algorithm\. There the weights are classical constants, and multiplying a quantum register by a constant is compiled into shifted additions, whose cost grows with the number of nonzero digits of the constant, while a shift is only a relabelling of qubits\[[11](https://arxiv.org/html/2610.31931#bib.bib11)\]\. A state tree controls exactly this number\.

Finally, the temperature of the machine points to a different use of the tree: not only to choose the states of a network that has already been trained, but to learn them in the first place\. This is the subject of the next section\.

## 8Learning in the space of states

So far a tree was fitted to a network trained in the usual way\. The representation also invites training the tree directly\. Ternary networks trained from scratch\[[51](https://arxiv.org/html/2610.31931#bib.bib51),[52](https://arxiv.org/html/2610.31931#bib.bib52)\]already do this for the simplest tree, one scale above leaves\{−1,0,1\}\\\{\-1,0,1\\\}, and quantization\-aware training does it for group\-quantization trees\. Deeper trees can be trained in the same way, through a relaxation of the discrete choices: every alphabet receives a soft distribution over its states, which is sharpened as a temperature is lowered\. Training then moves through the space of decisions, and the trained object is already the deployed one: its storage cost, its circuit and its format are fixed by the tree\. The temperature plays the same role as in the Boltzmann machine of[Section7](https://arxiv.org/html/2610.31931#S7), and connecting the two precisely for training losses is a natural direction for future work\.

## 9Early evidence and directions

The previous sections developed one idea from several sides\. This section shows that it already delivers, on real models and beyond them \([Tables6](https://arxiv.org/html/2610.31931#S9.T6)and[8](https://arxiv.org/html/2610.31931#S9.T8)\)\. We present five results, each followed by what it means in practice\. They are selected examples rather than a general claim, and each will be reported in full in a separate paper; together, they show that a single representation brings concrete gains in places as different as model serving, compression, model tuning, hardware design and image coding\.

ReadingSetting and metricBaselineState treeStorage and
servingBF16 weights stored exactly; bits per weightBF16: 16 bits12\.13 bits \(−24%\-24\\%\), bit\-exactLLaMA\-3\-8B, layer 15; matrix–vector speed\-up, batch 1–8cuBLAS \(A100\);
PyTorch \(CPU\)1\.05–1\.51×\\times; 1\.24–1\.27×\\timessame matrices and batches, format alone \(bound1\.32×1\.32\\times\)our kernel on BF16 \(A100\)1\.03–1\.27×\\timesvLLM end\-to\-end serving, Llama\-3\.2\-1B and Qwen3\.5\-9B; speed\-up at batch 1 / 8 / 32vLLM, BF161\.23×\\times/ 1\.17–1\.21×\\times/≈\\approx1\.0×\\timesCompressionLLaMA\-3\-8B at 3\.25 bits per weight, no training; perplexity on WikiText\-2 / C4 / FineWebstrongest GPTQ:
8\.09 / 12\.72 / 11\.816\.95 / 10\.93 / 9\.76Discrete
optimizationQwen2\.5\-1\.5B\-Instruct at 2\.44 bits per weight, only states change; validation KL, GSM8K, MMLUbefore tuning:
0\.491, 13\.95, 48\.30\.371, 21\.83, 49\.6LLaMA\-3\-8B, layer 1 down projection, 3\.3\-bit tree; relative output errorbefore sampling: 0\.02670\.0153CircuitsLLaMA\-3\-8B, adders per weight at equal layer error \([Table5](https://arxiv.org/html/2610.31931#S6.T5)\)GPTQ at 4 / 3 / 2 bits9–32% / 14–55% / 30–63% fewerTable 6:One result for each reading of the tree\. The strongest GPTQ uses act\-order and an MSE\-optimized grid in the same pipeline\. Perplexity at context 2048 \(FP16: 6\.14 / 9\.44 / 7\.85\); bits per weight cover all linear layers of the transformer blocks, scales included\.#### Lossless storage: less memory and faster serving, for free\.

The simplest use of the tree involves no approximation at all\.[Section2](https://arxiv.org/html/2610.31931#S2)showed that two neighbouring weights can share their exponent and still be reproduced exactly\. At scale, a BF16 matrix stored with one shared exponent per 64 weights and, per weight, a 4\-bit state and an 8\-bit leaf needs 12\.13 instead of 16 bits per weight over whole models, and every weight decodes to exactly its BF16 value\. Weights whose exponent lies 15 or more below the largest in their group, about 0\.01% of them and including exact zeros, are kept exactly in a short side list, whose cost is included \([SectionA\.4](https://arxiv.org/html/2610.31931#A1.SS4)\)\. Fewer bytes matter because generating tokens is limited by memory traffic rather than by arithmetic\. Our CUDA kernel decodes the tree with byte\-parallel integer operations, feeds the tensor cores directly and runs at 97% of the bandwidth floor of the tree bytes\. Reading 12\.13 instead of 16 bits per weight bounds the gain from the format alone at16/12\.13≈1\.32×16/12\.13\\approx 1\.32\\times\.[Table7](https://arxiv.org/html/2610.31931#S9.T7)separates the two sources of speed on layer 15 of LLaMA\-3\-8B, with every kernel timed in the same run\. Against our own kernel reading plain BF16, the tree makes matrix–vector products 1\.03–1\.27×\\timesfaster at batch sizes 1 to 8, below the bound; this is the gain of the format\. Against cuBLAS the gain is 1\.05–1\.51×\\times, because on the large MLP matrices our kernel design is itself up to 1\.27×\\timesfaster than cuBLAS, while on the attention projections cuBLAS is already as fast and the whole gain comes from the format\. At batch 32 the product becomes compute\-bound, and the tree is up to 15% slower than our BF16 kernel\. On a CPU, products become 1\.24–1\.27×\\timesfaster than PyTorch\. In end\-to\-end serving with vLLM, generation becomes 1\.23×\\timesfaster at batch size 1 for both Llama\-3\.2\-1B\[[48](https://arxiv.org/html/2610.31931#bib.bib48)\]and Qwen3\.5\-9B\[[62](https://arxiv.org/html/2610.31931#bib.bib62)\], and 1\.17–1\.21×\\timesfaster at batch size 8\. When serving becomes compute\-bound, at batch size 32, the tree runs at a speed comparable to BF16\. These end\-to\-end gains include both the format and our kernel design; separating them requires serving the same model with our kernel reading BF16, which we leave to the companion paper\.

Impact\.A BF16 model can be stored with 24% less disk and served with 24% less weight memory and up to 23% higher throughput at the small batch sizes of latency\-sensitive serving, with tiny slowdown in the compute\-bound regime\. Because the weights are exactly the same, all of this comes without any loss of accuracy, calibration or retraining, and the memory it frees can hold a longer key\-\-value cache or a larger batch\. The vLLM plugin is enabled with a single flag and is publicly available\.222Code and vLLM plugin:[https://github\.com/ammarali32/StateTree\-Representation](https://github.com/ammarali32/StateTree-Representation)\.

Table 7:Matrix–vector products of layer 15 of LLaMA\-3\-8B on an A100, all three kernels timed in the same run\. “Own BF16” is our kernel with the same structure reading plain BF16, so*kernel*isolates kernel engineering and*format*isolates the 12\.13\-bit tree \(bounded by16/12\.13≈1\.32×16/12\.13\\approx 1\.32\\times\); their product is the total gain over cuBLAS\. At batch 32 the product is compute\-bound and decoding costs more than the saved bytes gain\.
#### Compression: better low\-bit models, without training\.

[Section5](https://arxiv.org/html/2610.31931#S5)argued that once every compression method is an edit of the same object, a single optimizer can choose among all of them for every matrix\. A dynamic program over the whole of LLaMA\-3\-8B does exactly this: at 3\.25 bits per weight and without any training, it lowers perplexity by 14–17% relative to the strongest GPTQ configuration in the same pipeline, and closes 58% of GPTQ’s gap to FP16 on WikiText\-2 \(from 1\.95 to 0\.81\)\. The accuracy cost of every option is measured on the calibration lossLL, as12​\[L⁡\(W\+Δ\)\+L⁡\(W−Δ\)\]−L⁡\(W\)\\frac\{1\}\{2\}\[L\(W\+\\Delta\)\+L\(W\-\\Delta\)\]\-L\(W\)withΔ\\Deltathe error of the option, which removes the first\-order term and puts all matrices and methods on one scale\. The menu of every matrix combines quantization, pruning and low\-rank edits, with no rotation and no trellis\-coded quantizer, and will be described in a follow\-up paper; every option the solver picks is built and measured, and the problem is solved again until the whole plan has been measured \([SectionA\.4](https://arxiv.org/html/2610.31931#A1.SS4)\)\. Instead of one method being forced on every matrix, each matrix receives the edit that buys the most accuracy for its bits\.

Impact\.At a fifth of the size of FP16, the compressed model is much closer to the original than GPTQ at the same size, with no training at all\. And because every method is just one more edit on the menu, a new compression method does not have to compete with the existing ones: it can be added to the optimizer and combined with all of them\.

#### Discrete optimization: a better model in the same bytes\.

A compressed model written as a tree is a set of discrete states, and[Section7](https://arxiv.org/html/2610.31931#S7)showed that choosing them is a ground\-state problem\. A model can therefore keep improving after compression without changing its format or its size\. In*Ising healing*, every step models the KL divergence to the original model by its gradient and its Kronecker factors, estimated on about 45k tokens, and gives every movable code one unit that either keeps it or moves it by one level within its own grid\. The energy is then the Kronecker\-factored machine of[Property6](https://arxiv.org/html/2610.31931#Thmproperty6)with an added linear term; it is solved with ballistic simulated bifurcation, a quantum\-inspired classical solver\[[26](https://arxiv.org/html/2610.31931#bib.bib26),[27](https://arxiv.org/html/2610.31931#bib.bib27)\], started from a greedy update, and a step is kept only if the KL on a validation set improves\. As an illustration, we compressed Qwen2\.5\-1\.5B\-Instruct without training to 2\.44 bits per weight with a wider menu, whose details will follow in a separate paper, and let only its GPTQ\-family leaves, 6\.8% of the weights, move\. Forty such steps changed 46,239 stored codes, about 0\.004% of the model’s codes, and lowered the validation KL to the original model by 24%; since this KL guided the search, we report it as a validation result only\. On benchmarks that play no part in the search, healing improved perplexity on all three corpora \(on WikiText\-2 from 13\.74 to 12\.85\) and raised GSM8K accuracy by 7\.9 points, from 13\.95 to 21\.83\. GSM8K has 1,319 problems, so each of these accuracies carries a 95% binomial interval of about±2\\pm 2points and the gain is about five standard errors; it comes from a single run\. The solver reached lower energies than a greedy update on 371 of 516 real instances and higher ones on none, but end to end the choice of solver matters little\. The reported run keeps, at every step, the best of no move, the greedy update and simulated bifurcation; run alone in a separate ablation, greedy updates reach a validation KL of 0\.378 and GSM8K 20\.3, and simulated bifurcation 0\.371 and 21\.0\. With the same number of tokens, tuning only the continuous scales of the same model helps more \(KL 0\.325, GSM8K 27\.5\); healing changes the discrete codes instead, so the two act on different parameters\. Annealed Gibbs sampling over the states of a 3\.3\-bit tree, six sweeps followed by two of coordinate descent, reduced the output error of the down projection of the first layer of LLaMA\-3\-8B by 43%\.

Impact\.A compressed model can be repaired after the fact with the same file size, the same format and the same kernels; the repair touches a few tens of thousands of codes and can be shipped as a tiny patch\. The Boltzmann reading turns such repairs into a standard discrete problem, open to solvers from statistical physics today and to quantum annealers as they mature, although on this first instance simple solvers already capture most of the gain\.

#### Hardware: fewer adders for the same accuracy\.

At equal layer error, choosing the states of a tree for circuit cost needs 14–55% fewer adders than GPTQ at 3\-bit accuracy and 30–63% fewer at 2\-bit accuracy, the upper ends on the two matrices with strong outliers \([Section6\.5](https://arxiv.org/html/2610.31931#S6.SS5)\)\. The saving is structural: every weight carries its own power of two, so an outlier costs one adder instead of forcing extra precision on its whole group; zeros and shifts are free; and no scale multipliers per group remain\.

Impact\.For chips that hardwire a model, the number of adders is a first\-order measure of silicon area and energy\. The tree lets a designer choose the weights for the circuit rather than for storage, and spend adders only where accuracy needs them\.

Table 8:Lossless image coding on Kodak\[[39](https://arxiv.org/html/2610.31931#bib.bib39)\]; every result was checked to be bit\-exact\. GPU decoding of JPEG 2000 and HTJ2K uses nvImageCodec with the nvJPEG2000 backend\. All GPU timings decode the 24 images in batched calls; one image per call, JPEG 2000 and HTJ2K reach 0\.03 and 0\.39 Gpixel/s\. GPU timings run from compressed bytes in host memory to the decoded image in GPU memory \(median of 10 runs, A100\-SXM4\-40GB\); the CPU timing, on 16 cores of a shared AMD EPYC 7742, is indicative\. For a single small image, GPU HTJ2K keeps an advantage that the batched timings do not show\.
#### Beyond neural networks: fast lossless images\.

Nothing in the representation is specific to weights: any array of integers can be written as paths of decisions\. We tested this on lossless image coding, using the same path that[Section2](https://arxiv.org/html/2610.31931#S2)applied to a weight\. After a reversible colour transform chosen per image, each pixel is predicted coarse to fine from four already decoded neighbours, in twelve passes per channel, each of which decodes fully in parallel\. The prediction residual, mapped to a non\-negative integer, is then written as a path of two decisions: its*state*is its bit length, the counterpart of an exponent, and its*leaf*holds the bits below the leading one, the counterpart of a mantissa\. There is one twist\. Instead of storing a shared node for a group of pixels, the decoder infers it: a context computed from the decoded neighbours, their range and their recent residual activity, plays the role of the node and selects the probability model under which the state and the first leaf bit are coded with interleaved rANS\[[14](https://arxiv.org/html/2610.31931#bib.bib14),[25](https://arxiv.org/html/2610.31931#bib.bib25)\]; the remaining leaf bits are stored as they are\. The shared decision thus becomes shared knowledge rather than shared storage\. The codec is therefore an analogue of a state tree rather than an instance of[DefinitionA\.1](https://arxiv.org/html/2610.31931#A1.Thmtheorem1): its decisions are entropy\-coded under inferred contexts instead of being stored as states\. The probability models come from a small codebook trained once on 40 DIV2K images\[[2](https://arxiv.org/html/2610.31931#bib.bib2)\], disjoint from the test sets\.

The result is a codec whose main strength is speed \([Table8](https://arxiv.org/html/2610.31931#S9.T8)\)\. On Kodak it needs 9\.47 bits per pixel, less than lossless WebP \(9\.50\), JPEG 2000 \(9\.57\) and HTJ2K \(10\.53\), though 9% more than JPEG XL at its strongest setting \(8\.66\); on the CLIC professional validation set\[[75](https://arxiv.org/html/2610.31931#bib.bib75)\]it needs 8\.51, against 7\.88 for JPEG XL and 8\.81 for lossless WebP \([SectionA\.4](https://arxiv.org/html/2610.31931#A1.SS4)\)\. Decoding batched images on an A100 runs at 2\.9 Gpixel/s on both datasets, against 0\.45 for HTJ2K and 0\.10 for JPEG 2000 on the same GPU, and about 0\.04 for JPEG 2000 on sixteen CPU cores\. For a single small image, GPU HTJ2K keeps an advantage that the batched timings do not show, and the strongest context\-modelling coders, JPEG XL and paq8px, still compress photographs better\.

Impact\.Exact images decode about6×6\\timesfaster than with HTJ2K and almost30×30\\timesfaster than with JPEG 2000 on the same GPU, at a better compression ratio than both\. Training pipelines in which data loading is the bottleneck, and the streaming of images and textures, gain directly\. More broadly, the result shows that the representation is a property of numbers, not of neural networks\.

#### What comes next\.

These results cover only part of what the representation opens, and each suggests a direction of its own\.

- •*Learning in tree space\.*Training deep trees from scratch \([Section8](https://arxiv.org/html/2610.31931#S8)\) would let networks discover which decisions to share, instead of having the sharing imposed after training\.
- •*From model to silicon\.*Compiling whole models into hardwired adder circuits would turn[Section6](https://arxiv.org/html/2610.31931#S6)into a design flow with explicit trade\-offs between area, energy and accuracy\.
- •*Sparse model updates\.*Since tuning moves only a tiny fraction of the states, updates and fine\-tuning deltas can be shipped as small edits of a tree\.
- •*Beyond weights\.*Activations and the key–value cache could be written as trees on the fly, and the image result suggests that other numerical data, such as scientific arrays, audio and video, can benefit from the same structure\.
- •*Quantum hardware\.*The block subproblems of[Section7](https://arxiv.org/html/2610.31931#S7)are natural first workloads for quantum annealers and variational circuits as they grow\.

In every case the gain has the same source: the decisions that numbers share are made explicit, stored once, and optimized as discrete states\.

## 10Related work

TWIG brings together ideas from several lines of work that have so far developed separately\.

#### Number formats with shared parts\.

Group quantization shares a scale, and block floating point and the microscaling formats share an exponent across a block of weights\[[65](https://arxiv.org/html/2610.31931#bib.bib65),[66](https://arxiv.org/html/2610.31931#bib.bib66)\]\. These are state trees with two levels of decisions \([Table2](https://arxiv.org/html/2610.31931#S3.T2)\)\. Shared microexponents\[[67](https://arxiv.org/html/2610.31931#bib.bib67)\]and HiFloat4\[[50](https://arxiv.org/html/2610.31931#bib.bib50)\]go deeper, with one or two further levels of one\-bit shifts shared by sub\-blocks, and HiFloat4 reports higher accuracy than NVFP4 on several language models at the same 4\.5 bits per weight\. TWIG allows sharing at any level and of any kind of decision, and makes its cost in bits and in additions explicit\.

#### Path\-structured representations\.

Describing a function or a matrix by paths through a graph with values on its edges, with identical parts stored once, is a classical idea in circuit verification and quantum computing\. Edge values are additive in edge\-valued decision diagrams\[[40](https://arxiv.org/html/2610.31931#bib.bib40)\]and multiplicative in binary moment diagrams\[[7](https://arxiv.org/html/2610.31931#bib.bib7)\]and in the decision diagrams that store the matrices of quantum circuits\[[55](https://arxiv.org/html/2610.31931#bib.bib55)\]; weighted automata use the same principle to compress images\[[12](https://arxiv.org/html/2610.31931#bib.bib12)\]\. In numerical linear algebra, hierarchical matrices store low\-rank blocks over a tree of index sets\[[28](https://arxiv.org/html/2610.31931#bib.bib28)\], and quantized tensor trains write an entry as a product of factors indexed by the binary digits of its position\[[60](https://arxiv.org/html/2610.31931#bib.bib60),[35](https://arxiv.org/html/2610.31931#bib.bib35)\], close to the product along a path in[Equation1](https://arxiv.org/html/2610.31931#S2.E1)\. Tree\-structured and residual vector quantizers\[[24](https://arxiv.org/html/2610.31931#bib.bib24)\], and binary\-coded quantization, which writes a matrix as a sum of scaled binary matrices\[[86](https://arxiv.org/html/2610.31931#bib.bib86),[61](https://arxiv.org/html/2610.31931#bib.bib61)\], share decisions in related ways\. TWIG adopts this path structure for the weights of neural networks\. What is new is the cost model of bits and additions, the treatment of compression methods as edits of one object, and the reading of state selection as a Boltzmann machine\.

#### Multiplier\-free networks\.

Logarithmic and power\-of\-two weights\[[56](https://arxiv.org/html/2610.31931#bib.bib56),[90](https://arxiv.org/html/2610.31931#bib.bib90),[17](https://arxiv.org/html/2610.31931#bib.bib17)\], sums of power\-of\-two terms\[[44](https://arxiv.org/html/2610.31931#bib.bib44)\], and networks built from additions\[[9](https://arxiv.org/html/2610.31931#bib.bib9)\]or from shifts and additions\[[88](https://arxiv.org/html/2610.31931#bib.bib88),[89](https://arxiv.org/html/2610.31931#bib.bib89)\]avoid multipliers by choosing a multiplier\-free family of weights, during or after training\. In the language of this paper these families are particular state trees, and[Property3](https://arxiv.org/html/2610.31931#Thmproperty3)explains why sums of power\-of\-two terms lose no expressive power\. Signed\-power\-of\-two coefficients and multiplierless constant multiplication have long been used in digital filter design, where the number of nonzero digits is a cost separate from the word length\[[46](https://arxiv.org/html/2610.31931#bib.bib46),[81](https://arxiv.org/html/2610.31931#bib.bib81)\]\. Reconfigurable constant\-coefficient multipliers bring this to neural networks on FPGAs, restricting weights to coefficients that adders, shifts and multiplexers can apply\[[18](https://arxiv.org/html/2610.31931#bib.bib18)\], and lookup\-table kernels bring related ideas to processors\[[61](https://arxiv.org/html/2610.31931#bib.bib61),[85](https://arxiv.org/html/2610.31931#bib.bib85)\]\.

#### Unified and budgeted compression\.

Optimal Brain Compression treats pruning and quantization in one framework\[[20](https://arxiv.org/html/2610.31931#bib.bib20)\], SPDY chooses per\-layer settings under a speed budget by dynamic programming\[[21](https://arxiv.org/html/2610.31931#bib.bib21)\], and mixed\-precision methods allocate bits per layer, for instance by integer programming\[[87](https://arxiv.org/html/2610.31931#bib.bib87)\], and ROCKET allocates low\-rank truncation across layers as a knapsack problem\[[91](https://arxiv.org/html/2610.31931#bib.bib91)\]\. TWIG supplies a common object for such choices, so that a single menu can contain edits from every family \([Section5](https://arxiv.org/html/2610.31931#S5)\)\.

#### Discrete optimization of weights\.

Adaptive rounding writes the rounding of a layer as a QUBO\[[59](https://arxiv.org/html/2610.31931#bib.bib59)\]; GPTQ has been identified with Babai’s nearest\-plane algorithm, under conditions on column order and clipping\[[10](https://arxiv.org/html/2610.31931#bib.bib10)\]; Kronecker\-factored curvature has been used to round with both input and output statistics\[[78](https://arxiv.org/html/2610.31931#bib.bib78)\]; and quantization and pruning have been formulated for simulated and quantum annealing\[[58](https://arxiv.org/html/2610.31931#bib.bib58),[82](https://arxiv.org/html/2610.31931#bib.bib82)\]\.[Property6](https://arxiv.org/html/2610.31931#Thmproperty6)extends the QUBO view to all decisions of a tree, including zeros, shared shifts, codebook indices and selectors\.

## 11Conclusion

Neural networks are trained in the continuum but run in a discrete world, and the passage between the two has so far been handled one method at a time\. We have proposed to make this passage explicit with a single representation\. In TWIG, every weight is the end of a path of small discrete decisions, and weights that belong together share the upper part of their paths, so that each shared decision is stored once and applied once\. The idea is simple, and its consequences reach in several directions\. The formats in use today are shallow state trees; nothing is lost in the worst case, and shared structure lowers the cost in bits\. Read as a compressed model, a tree turns pruning, quantization, factorization, codebooks and sharing into edits of one object, chosen by one optimizer\. Read as a circuit, it applies a layer with shifts and additions, apart from one scale per row and per column, with a size bounded by a count read off the tree\. Read as an optimization problem, choosing its states is the ground\-state problem of a higher\-order Boltzmann machine whose interactions follow the paths of the tree, the problem class that annealers and quantum optimizers are built to solve\. And because its decisions are explicit, a tree could be trained directly, so that the trained object is already the deployed one, a direction we leave open\.

## Appendix ADefinitions, statements and proofs

Throughout,W∈ℝm×nW\\in\\mathbb\{R\}^\{m\\times n\}is a weight matrix \(rows are outputs, columns are inputs\),I=\[m\]×\[n\]I=\[m\]\\times\[n\]is the set of its entries, and𝒫=\{0\}∪\{±2k:k∈ℤ\}\\mathcal\{P\}=\\\{0\\\}\\cup\\\{\\pm 2^\{k\}:k\\in\\mathbb\{Z\}\\\}is the set of signed powers of two together with zero\. Several statements below are known results restated for state trees: the counting bounds of[TheoremsA\.6](https://arxiv.org/html/2610.31931#A1.Thmtheorem6)and[A\.7](https://arxiv.org/html/2610.31931#A1.Thmtheorem7), the non\-adjacent form in[TheoremA\.8](https://arxiv.org/html/2610.31931#A1.Thmtheorem8), reduced\-rank regression in[TheoremA\.11](https://arxiv.org/html/2610.31931#A1.Thmtheorem11)\[[32](https://arxiv.org/html/2610.31931#bib.bib32)\], the multiple\-choice knapsack problem in[TheoremA\.13](https://arxiv.org/html/2610.31931#A1.Thmtheorem13)\[[69](https://arxiv.org/html/2610.31931#bib.bib69)\], the bucketing construction in[PropositionA\.15](https://arxiv.org/html/2610.31931#A1.Thmtheorem15), and the quadratic form of rounding in[TheoremA\.16](https://arxiv.org/html/2610.31931#A1.Thmtheorem16)\(i\)\[[59](https://arxiv.org/html/2610.31931#bib.bib59)\]\. We include them, with their sources, to keep the paper self\-contained\. The statements specific to state trees are mainly[PropositionsA\.9](https://arxiv.org/html/2610.31931#A1.Thmtheorem9)and[A\.10](https://arxiv.org/html/2610.31931#A1.Thmtheorem10),[TheoremA\.14](https://arxiv.org/html/2610.31931#A1.Thmtheorem14)and[TheoremA\.16](https://arxiv.org/html/2610.31931#A1.Thmtheorem16)\(ii\)\.

### A\.1Definitions

###### Definition A\.1\(State tree\)\.

A*state tree*𝒯\\mathcal\{T\}overIIconsists of \(a\) a rooted tree whose leaves are in bijection withII; an internal node may have a single child, so that one group of entries can carry several decisions in sequence; \(b\) for every nodevv, a finite alphabetSvS\_\{v\}and a value mapcvc\_\{v\}that assigns a real value to each state ofvv, possibly depending on the states of the ancestors ofvv\(a node whose value depends on the states of its ancestors is a*lookup*\); \(c\) row scalesρ∈ℝm\\rho\\in\\mathbb\{R\}^\{m\}and column basesβ∈ℝn\\beta\\in\\mathbb\{R\}^\{n\}, stored in fixed formats \(for example FP16\)\. For an assignments=\(sv\)vs=\(s\_\{v\}\)\_\{v\}of states, the represented matrix isW^i​j​\(s\)=ρi​βj​∏v∈π⁡\(i,j\)cv​\(s\)\\widehat\{W\}\_\{ij\}\(s\)=\\rho\_\{i\}\\beta\_\{j\}\\prod\_\{v\\in\\pi\(i,j\)\}c\_\{v\}\(s\), whereπ⁡\(i,j\)\\pi\(i,j\)is the path from the root to the leaf of\(i,j\)\(i,j\)\. The set of matrices representable by𝒯\\mathcal\{T\}isℳ​\(𝒯\)=\{W^​\(s\)\}\\mathcal\{M\}\(\\mathcal\{T\}\)=\\\{\\widehat\{W\}\(s\)\\\}\. The leaf sets of the subtrees of𝒯\\mathcal\{T\}form a laminar family of groups of entries, the*hierarchy*of𝒯\\mathcal\{T\}\. The*depth*of𝒯\\mathcal\{T\}is the largest number of edges on a path from the root to a leaf, and the*height*h⁡\(v\)h\(v\)of a node is the largest number of edges fromvvdown to a leaf of its subtree\.

The definition is deliberately broad: because a value map may depend on the states of the ancestors, any code that is decoded from the root down can be written as a state tree, so[PropositionA\.4](https://arxiv.org/html/2610.31931#A1.Thmtheorem4)is a statement about costs rather than about expressiveness\. Some results use the narrower*core*model, in which no value map depends on the ancestors:[PropositionsA\.9](https://arxiv.org/html/2610.31931#A1.Thmtheorem9)and[A\.10](https://arxiv.org/html/2610.31931#A1.Thmtheorem10)are stated for it, and[TheoremA\.16](https://arxiv.org/html/2610.31931#A1.Thmtheorem16)\(ii\) for node values that are affine in one bit or linear in one one\-hot group\. Trellis codes, whose overlapping windows share bits without nesting, are state trees only as a codebook over a whole block\.

###### Definition A\.2\(State\-tree graph\)\.

State\-tree expressions are generated by𝖤::=𝒯​∣𝖤\+𝖤∣​𝖤⋅𝖤\|ref⁡\(𝖤\)\\mathsf\{E\}::=\\mathcal\{T\}\\mid\\mathsf\{E\}\+\\mathsf\{E\}\\mid\\mathsf\{E\}\\cdot\\mathsf\{E\}\\mid\\mathrm\{ref\}\(\\mathsf\{E\}\): a state tree, a*sum node*\(entrywise sum of operands of equal shape\), a*product node*\(matrix product of compatible operands\) and a*reference*to an expression defined elsewhere\. An expression in which some part is referenced more than once is a*state\-tree graph*, and the referenced part is a*shared subtree*\. The tables read by lookups are shared subtrees\.

###### Definition A\.3\(Costs\)\.

Under an assignmentss, a node is*inactive*if one of its ancestors has value00or is inactive, or if its own value map is identically zero given the states of its ancestors; otherwise it is*active*\. The*storage cost*isbits⁡\(s\)=∑v​activelog2⁡\|Sv\|\\operatorname\{bits\}\(s\)=\\sum\_\{v\\ \\text\{active\}\}\\log\_\{2\}\|S\_\{v\}\|, with each shared subtree counted once, plus the cost ofρ\\rhoandβ\\betain their formats\. A decoder that reads the states from the root down knows which nodes are active, so this is the length of a decodable code, up to rounding that joint coding of several states makes negligible\. The shape of the tree, including the order of rows and columns, is fixed per matrix shape and is not counted; a data\-dependent order of the rows costs at mostlog2⁡m\!\\log\_\{2\}m\!further bits, and can be folded into the neighbouring layer when every matrix that reads or writes the same dimension uses the same order, as all matrices that read or write the residual stream of a transformer must, including the embedding and the output head\. Folding gives one order per dimension, shared by every matrix that reads or writes it \(the gate and up projections of a gated MLP, for instance, share theirs\); a separate order for each matrix instead costslog2⁡m\!/\(m​n\)<log2⁡m/n\\log\_\{2\}m\!/\(mn\)<\\log\_\{2\}m/nbits per weight, which is negligible for large layers\. If every entry is a sum ofti​jt\_\{ij\}elements of𝒫\\mathcal\{P\}timesρi​βj\\rho\_\{i\}\\beta\_\{j\}, the*circuit cost*is the number of two\-input additions used to evaluatex↦W^​xx\\mapsto\\widehat\{W\}xby[Equation3](https://arxiv.org/html/2610.31931#S6.E3), which is at most∑i\(∑jti​j−1\)\+\\sum\_\{i\}\\big\(\\sum\_\{j\}t\_\{ij\}\-1\\big\)^\{\+\}; sharing partial sums between outputs can reduce it\[[81](https://arxiv.org/html/2610.31931#bib.bib81)\]\.

### A\.2What state trees express

###### Proposition A\.4\(Existing formats;[Property1](https://arxiv.org/html/2610.31931#Thmproperty1)\)\.

Each of the following is a state tree with at most five levels of decisions on any path \(at most three for \(i\)–\(v\), except four for HiFloat4\) whose storage cost is at most that of the format, and equal to it for \(i\)–\(v\): \(i\) binary floating\-point formats, including subnormals; \(ii\)bb\-bit integers with a scale per group, and with a scale and a zero point per group; \(iii\) block floating point and microscaling formats, including shared microexponents \(MX6, MX9\)\[[67](https://arxiv.org/html/2610.31931#bib.bib67)\]and HiFloat4\[[50](https://arxiv.org/html/2610.31931#bib.bib50)\]; \(iv\) ternary weights with a scale; \(v\) codebook formats that store, for every block ofddentries, an index into a table ofdd\-dimensional vectors; \(vi\)N:MN\{:\}Msparsity, with the kept values in any of the formats above\.

###### Proof\.

\(i\) Give each entry a chain of three nodes: a sign node with values±1\\pm 1; an exponent node with values2e−e02^\{e\-e\_\{0\}\}\(and21−e02^\{1\-e\_\{0\}\}for the subnormal exponent code\); and a mantissa leaf that is a lookup on the exponent, with value1\+m​2−p1\+m2^\{\-p\}for normal exponent codes andm​2−pm2^\{\-p\}for the subnormal code\. Codes reserved for infinities and NaN do not occur as weights, but stay in the alphabet, so the exponent node costs the full exponent field\. \(ii\) A group node whose alphabet is the scale format, above leaves\{0,…,2b−1\}\\\{0,\\dots,2^\{b\}\-1\\\}or a symmetric integer range\. For a zero point, insert below the scale node a node of value11whose state is the zero pointzz, and make each leaf a lookup with valueq−zq\-z\. \(iii\) A block node with values2e2^\{e\}\(or the block scale\) above leaves in the element format; shared microexponents and HiFloat4 insert one or two levels of one\-bit shift nodes, one node per sub\-block, between the block node and the leaves\. \(iv\) One node holding the scale above leaves\{−1,0,1\}\\\{\-1,0,1\\\}\. \(v\) A block node whose statekkindexes the table; each of itsddleaves has a single state and is a lookup that returns its coordinate of thekk\-th table vector; the table is a shared subtree\. \(vi\) A group node overMMentries whose state is one of the\(MN\)\\binom\{M\}\{N\}admissible masks; the leaves outside the mask have value00and are inactive, and the others carry the value format; the mask costslog2⁡\(MN\)\\log\_\{2\}\\binom\{M\}\{N\}bits, no more than the mask field of the format\. In cases \(i\)–\(v\) the stored states are exactly the fields of the format\. ∎

###### Proposition A\.5\(Refinement and sharing\)\.

\(i\) If𝒯′\\mathcal\{T\}^\{\\prime\}is obtained from𝒯\\mathcal\{T\}by inserting nodes whose alphabets contain a state of value11\(independent of the ancestors\), or by enlarging alphabets, thenℳ⁡\(𝒯\)⊆ℳ⁡\(𝒯′\)\\mathcal\{M\}\(\\mathcal\{T\}\)\\subseteq\\mathcal\{M\}\(\\mathcal\{T\}^\{\\prime\}\)\. \(ii\) A tree is contained in any sum node that has it as a summand, provided the other summands can represent00\. \(iii\) Replacing several copies of a subtree by references to one shared subtree yields a subset of the original set of matrices, and saves the storage of all copies but one\.

###### Proof\.

\(i\) Give the inserted nodes their value\-one state and copy all other states; the represented matrix is unchanged\. \(ii\) Set the other summands to zero\. \(iii\) A shared assignment is an unshared assignment in which all copies have equal states\. ∎

###### Theorem A\.6\(Counting;[Property2](https://arxiv.org/html/2610.31931#Thmproperty2)\)\.

LetQ⊂ℝQ\\subset\\mathbb\{R\}haveKKelements\. \(i\) The flat tree \(a root of value11above leaves with alphabetQQ, andρ=β=𝟏\\rho=\\beta=\\mathbf\{1\}\) represents every matrix inQm×nQ^\{m\\times n\}withm​n​log2⁡Kmn\\log\_\{2\}Kbits\. \(ii\) Any encoding, of fixed or variable length, whose codewords have at mostBBbits represents at most2B\+1−12^\{B\+1\}\-1matrices\. Representing all ofQm×nQ^\{m\\times n\}therefore needsB≥m​n​log2⁡K−1B\\geq mn\\log\_\{2\}K\-1, andB≥m​n​log2⁡KB\\geq mn\\log\_\{2\}Kfor fixed\-length codes\.

###### Proof\.

\(i\) is immediate from the definition\. \(ii\) There are∑ℓ=0B2ℓ=2B\+1−1\\sum\_\{\\ell=0\}^\{B\}2^\{\\ell\}=2^\{B\+1\}\-1binary strings of length at mostBB, and\|Qm×n\|=Km​n=2m​n​log2⁡K\|Q^\{m\\times n\}\|=K^\{mn\}=2^\{mn\\log\_\{2\}K\}\. ∎

###### Theorem A\.7\(Density at the optimal price;[Property2](https://arxiv.org/html/2610.31931#Thmproperty2)\)\.

Letε∈\(0,1\]\\varepsilon\\in\(0,1\]andGε=\{−1\+\(2k−1\)ε:k=1,…,⌈1/ε⌉\}G\_\{\\varepsilon\}=\\\{\-1\+\(2k\-1\)\\varepsilon:\\ k=1,\\dots,\\lceil 1/\\varepsilon\\rceil\\\}\. \(i\) For everyWWwith‖W‖max≤1\\\|W\\\|\_\{\\max\}\\leq 1, the flat tree with leaf alphabetGεG\_\{\\varepsilon\}achieves‖W−W^‖max≤ε\\\|W\-\\widehat\{W\}\\\|\_\{\\max\}\\leq\\varepsilonwithm​n​log2​⌈1/ε⌉mn\\log\_\{2\}\\lceil 1/\\varepsilon\\rceilbits\. \(ii\) Any encoding whose codewords have at mostBBbits and which achieves maximum errorε\\varepsilonon all suchWWhasB≥m​n​log2⁡\(1/ε\)−1B\\geq mn\\log\_\{2\}\(1/\\varepsilon\)\-1\. \(iii\) The gap between \(i\) and \(ii\) is at mostm​n​log2⁡\(1\+ε\)\+1mn\\log\_\{2\}\(1\+\\varepsilon\)\+1bits\. \(iv\) For a generalWW, a root node with a power\-of\-two value2e2^\{e\}, where2e−1<‖W‖max≤2e2^\{e\-1\}<\\\|W\\\|\_\{\\max\}\\leq 2^\{e\}, gives maximum error at most2​ε​‖W‖max2\\varepsilon\\\|W\\\|\_\{\\max\}, at the additional cost of storingee\.

###### Proof\.

\(i\) Consecutive points ofGεG\_\{\\varepsilon\}are2​ε2\\varepsilonapart, the first lies at−1\+ε\-1\+\\varepsilonand the last at least at1−ε1\-\\varepsilon, so everyu∈\[−1,1\]u\\in\[\-1,1\]is withinε\\varepsilonofGεG\_\{\\varepsilon\}\. \(ii\) The max\-norm balls of radiusε\\varepsilonaround the codewords cover\[−1,1\]m​n\[\-1,1\]^\{mn\}; each has volume\(2​ε\)m​n\(2\\varepsilon\)^\{mn\}, so there are at leastε−m​n\\varepsilon^\{\-mn\}codewords, and the bound follows as in[TheoremA\.6](https://arxiv.org/html/2610.31931#A1.Thmtheorem6)\. \(iii\)log2⁡⌈1/ε⌉≤log2⁡\(1/ε\+1\)=log2⁡\(1/ε\)\+log2⁡\(1\+ε\)\\log\_\{2\}\\lceil 1/\\varepsilon\\rceil\\leq\\log\_\{2\}\(1/\\varepsilon\+1\)=\\log\_\{2\}\(1/\\varepsilon\)\+\\log\_\{2\}\(1\+\\varepsilon\)\. \(iv\) Apply \(i\) to2−e​W2^\{\-e\}W\. ∎

###### Theorem A\.8\(Sums of shift trees;[Property3](https://arxiv.org/html/2610.31931#Thmproperty3)\)\.

\(i\) For every realx≠0x\\neq 0andt≥1t\\geq 1there arep1,…,pt∈𝒫p\_\{1\},\\dots,p\_\{t\}\\in\\mathcal\{P\}with\|x−∑τpτ\|≤3−t​\|x\|\|x\-\\sum\_\{\\tau\}p\_\{\\tau\}\|\\leq 3^\{\-t\}\|x\|\. \(ii\) Every integerzzwith\|z\|<2b\|z\|<2^\{b\}is a sum of at most⌈\(b\+1\)/2⌉\\lceil\(b\+1\)/2\\rceilnonzero elements of𝒫\\mathcal\{P\}, and ofb/3\+O⁡\(1\)b/3\+O\(1\)on average over uniformly randomzz\[[63](https://arxiv.org/html/2610.31931#bib.bib63)\]\. \(iii\) Hence for everyWWandt≥1t\\geq 1there is a sum ofttflat trees with values in𝒫\\mathcal\{P\}\(the finitely many terms that occur form their alphabets\),W^=diag⁡\(ρ\)​\(∑τ=1tPτ\)​diag⁡\(β\)\\widehat\{W\}=\\operatorname\{diag\}\(\\rho\)\\big\(\\sum\_\{\\tau=1\}^\{t\}P\_\{\\tau\}\\big\)\\operatorname\{diag\}\(\\beta\), such that\|Wi​j−W^i​j\|≤3−t​\|Wi​j\|\|W\_\{ij\}\-\\widehat\{W\}\_\{ij\}\|\\leq 3^\{\-t\}\|W\_\{ij\}\|for all entries; every matrixdiag⁡\(ρ\)​Z​diag⁡\(β\)\\operatorname\{diag\}\(\\rho\)Z\\operatorname\{diag\}\(\\beta\)with integer entries\|Zi​j\|<2b\|Z\_\{ij\}\|<2^\{b\}is represented exactly witht=⌈\(b\+1\)/2⌉t=\\lceil\(b\+1\)/2\\rceil; andx↦W^​xx\\mapsto\\widehat\{W\}xis computed with shifts, at most∑i\(∑jti​j−1\)\+\\sum\_\{i\}\(\\sum\_\{j\}t\_\{ij\}\-1\)^\{\+\}additions, whereti​j≤tt\_\{ij\}\\leq tis the number of nonzero terms of entry\(i,j\)\(i,j\), andm\+nm\+nmultiplications\.

###### Proof\.

\(i\) If\|x\|∈\[2k,2k\+1\]\|x\|\\in\[2^\{k\},2^\{k\+1\}\], the nearer of2k2^\{k\}and2k\+12^\{k\+1\}, with the sign ofxx, is within2k−12^\{k\-1\}ofxx, and the distance relative to\|x\|\|x\|is largest,13\\frac\{1\}\{3\}, at\|x\|=3⋅2k−1\|x\|=3\\cdot 2^\{k\-1\}\. So one term leaves a residual of at most\|x\|/3\|x\|/3; repeat on the residual, and use the term00once the residual vanishes\. \(ii\) The non\-adjacent form ofzzhas at mostb\+1b\+1signed binary digits, no two adjacent ones nonzero, and its average density of nonzero digits is13\\frac\{1\}\{3\}\[[63](https://arxiv.org/html/2610.31931#bib.bib63)\]\. \(iii\) Apply \(i\) or \(ii\) to every entry ofdiag⁡\(ρ\)−1​W​diag⁡\(β\)−1\\operatorname\{diag\}\(\\rho\)^\{\-1\}W\\operatorname\{diag\}\(\\beta\)^\{\-1\}\(ρ=β=𝟏\\rho=\\beta=\\mathbf\{1\}is allowed\); zero entries take the term00\. In the product, every term is a shifted and signed copy ofβj​xj\\beta\_\{j\}x\_\{j\}added into the accumulator ofyiy\_\{i\}\. ∎

###### Proposition A\.9\(Reach of shared shifts;[Property4](https://arxiv.org/html/2610.31931#Thmproperty4)\)\.

Consider one column of a perfect binary tree of depthDD, with a shift bitbe∈\{0,1\}b\_\{e\}\\in\\\{0,1\\\}\(multiplier2−be2^\{\-b\_\{e\}\}\) on every edge and leaves in\{0,±1\}\\\{0,\\pm 1\\\}\. For a leafℓ\\ellletkℓ=∑e∈π⁡\(ℓ\)bek\_\{\\ell\}=\\sum\_\{e\\in\\pi\(\\ell\)\}b\_\{e\}, and letLLbe the set of nonzero leaves\. Integer exponents\(kℓ\)ℓ∈L\(k\_\{\\ell\}\)\_\{\\ell\\in L\}are realizable if and only if0≤kℓ≤D0\\leq k\_\{\\ell\}\\leq Dfor everyℓ∈L\\ell\\in Land, for every nodevvwith subtreeTvT\_\{v\}and heighth⁡\(v\)h\(v\),

maxℓ∈Tv∩L⁡kℓ−minℓ∈Tv∩L⁡kℓ≤h⁡\(v\)\.\\max\_\{\\ell\\in T\_\{v\}\\cap L\}k\_\{\\ell\}\-\\min\_\{\\ell\\in T\_\{v\}\\cap L\}k\_\{\\ell\}\\;\\leq\\;h\(v\)\.

###### Proof\.

Letpvp\_\{v\}be the number of active bits on the path from the root tovv\.*Necessity:*every leafℓ\\ellbelowvvhaspv≤kℓ≤pv\+h⁡\(v\)p\_\{v\}\\leq k\_\{\\ell\}\\leq p\_\{v\}\+h\(v\)\.*Sufficiency:*choose thepvp\_\{v\}from the top, keepingmaxTv∩L⁡k−h⁡\(v\)≤pv≤minTv∩L⁡k\\max\_\{T\_\{v\}\\cap L\}k\-h\(v\)\\leq p\_\{v\}\\leq\\min\_\{T\_\{v\}\\cap L\}k\(no condition ifTv∩LT\_\{v\}\\cap Lis empty\); this holds at the root withp=0p=0because0≤kℓ≤D0\\leq k\_\{\\ell\}\\leq D\. For a childccofvvwithTc∩L≠∅T\_\{c\}\\cap L\\neq\\emptyset, the admissible interval\[maxTc∩L⁡k−h⁡\(c\),minTc∩L⁡k\]\[\\max\_\{T\_\{c\}\\cap L\}k\-h\(c\),\\ \\min\_\{T\_\{c\}\\cap L\}k\]is nonempty by the condition atcc; its left end is at mostmaxTv∩L⁡k−h⁡\(v\)\+1≤pv\+1\\max\_\{T\_\{v\}\\cap L\}k\-h\(v\)\+1\\leq p\_\{v\}\+1and its right end at leastminTv∩L⁡k≥pv\\min\_\{T\_\{v\}\\cap L\}k\\geq p\_\{v\}\. Hence it containspvp\_\{v\}orpv\+1p\_\{v\}\+1; take that value aspcp\_\{c\}and setb\(v,c\)=pc−pvb\_\{\(v,c\)\}=p\_\{c\}\-p\_\{v\}\. At a leaf the interval is the single pointkℓk\_\{\\ell\}\. Bits on edges into subtrees without nonzero leaves are arbitrary\. ∎

###### Proposition A\.10\(Two equivalent encodings;[Property4](https://arxiv.org/html/2610.31931#Thmproperty4)\)\.

For one column of a perfect binary tree withmmleaves, the encoding of[PropositionA\.9](https://arxiv.org/html/2610.31931#A1.Thmtheorem9)\(edge bits, leaves\{0,±1\}\\\{0,\\pm 1\\\}\) and the encoding with one shift bit per internal node and leaves\{0,±1,±12\}\\\{0,\\pm 1,\\pm\\frac\{1\}\{2\}\\\}represent the same set of columns, up to a factor12\\frac\{1\}\{2\}per column that can be absorbed intoβj\\beta\_\{j\}\. The first stores2​\(m−1\)\+m​log2​32\(m\-1\)\+m\\log\_\{2\}3bits and the second\(m−1\)\+m​log2​5\(m\-1\)\+m\\log\_\{2\}5, about3\.583\.58and3\.323\.32bits per weight for largemm, with leaves coded jointly\.

###### Proof\.

A bit on the edge into an internal nodeccmultiplies exactly the subtree ofcc, as the node bit ofccdoes; a bit on the edge into a leaf multiplies only that leaf, which is the leaf magnitude12\\frac\{1\}\{2\}\. This maps every edge\-bit assignment to a node\-bit assignment with root bit00that represents the same column; conversely, a node\-bit assignment with root bit11is one with root bit00times12\\frac\{1\}\{2\}\. The map is onto but not one\-to\-one, because a zero leaf ignores the bit on its edge\. ∎

###### Theorem A\.11\(Rank;[Section3](https://arxiv.org/html/2610.31931#S3)\)\.

\(i\) A sum node ofrrproduct nodesuk​vk⊤u\_\{k\}v\_\{k\}^\{\\top\}represents exactly the matrices of rank at mostrr\. \(ii\) LetH≻0H\\succ 0withH=L​L⊤H=LL^\{\\top\}\. The minimizer offHf\_\{H\}over matrices of rank at mostrrisW^=\[W​L\]r​L−1\\widehat\{W\}=\[WL\]\_\{r\}L^\{\-1\}, where\[⋅\]r\[\\cdot\]\_\{r\}keeps therrlargest singular values\. \(iii\) Let alsoG≻0G\\succ 0withG=LG​LG⊤G=L\_\{G\}L\_\{G\}^\{\\top\}\. The minimizer oftr⁡\(G⁡\(W−W^\)​H​\(W−W^\)⊤\)\\operatorname\{tr\}\\big\(G\(W\-\\widehat\{W\}\)H\(W\-\\widehat\{W\}\)^\{\\top\}\\big\)over matrices of rank at mostrrisW^=LG−⁣⊤​\[LG⊤​W​L\]r​L−1\\widehat\{W\}=L\_\{G\}^\{\-\\top\}\[L\_\{G\}^\{\\top\}WL\]\_\{r\}L^\{\-1\}\.

###### Proof\.

\(i\) is the rank decomposition\. \(iii\) The objective equals‖LG⊤​W​L−LG⊤​W^​L‖F2\\\|L\_\{G\}^\{\\top\}WL\-L\_\{G\}^\{\\top\}\\widehat\{W\}L\\\|\_\{F\}^\{2\}, andW^↦LG⊤​W^​L\\widehat\{W\}\\mapsto L\_\{G\}^\{\\top\}\\widehat\{W\}Lis a linear bijection that preserves rank, so the Eckart–Young theorem\[[15](https://arxiv.org/html/2610.31931#bib.bib15)\]gives the truncated singular value decomposition\. \(ii\) is the caseG=IG=I\. SingularHHorGGare handled by damping\. ∎

###### Proposition A\.12\(Shared subtrees;[Section3](https://arxiv.org/html/2610.31931#S3)\)\.

LetW1,…,WgW\_\{1\},\\dots,W\_\{g\}havenncolumns and input statisticsH1,…,Hg⪰0H\_\{1\},\\dots,H\_\{g\}\\succeq 0, and writeWi≈Ci​DW\_\{i\}\\approx C\_\{i\}Dwith one sharedD∈ℝr×nD\\in\\mathbb\{R\}^\{r\\times n\}\. \(i\) Exact representation is possible if and only ifr≥rank⁡\[W1;…;Wg\]r\\geq\\operatorname\{rank\}\[W\_\{1\};\\dots;W\_\{g\}\]\. \(ii\) For fixedCiC\_\{i\}, the minimizers of∑ifHi​\(Ci​D\)\\sum\_\{i\}f\_\{H\_\{i\}\}\(C\_\{i\}D\)are the solutions of the generalized Sylvester equation∑i\(Ci⊤​Ci\)​D​Hi=∑iCi⊤​Wi​Hi\\sum\_\{i\}\(C\_\{i\}^\{\\top\}C\_\{i\}\)\\,D\\,H\_\{i\}=\\sum\_\{i\}C\_\{i\}^\{\\top\}W\_\{i\}H\_\{i\}\. \(iii\) Forg=2g=2, write this equation asA​D​B\+C​D​E=KADB\+CDE=K\. IfA,B≻0A,B\\succ 0\(for instanceC1C\_\{1\}of full column rank andH1≻0H\_\{1\}\\succ 0\), letP⊤​A​P=IP^\{\\top\}AP=I,P⊤​C​P=diag⁡\(λ\)P^\{\\top\}CP=\\operatorname\{diag\}\(\\lambda\),Q⊤​B​Q=IQ^\{\\top\}BQ=IandQ⊤​E​Q=diag⁡\(σ\)Q^\{\\top\}EQ=\\operatorname\{diag\}\(\\sigma\)be the generalized eigendecompositions\. ThenD=P⁡\[\(P⊤​K​Q\)⊘\(𝟏𝟏⊤\+λ​σ⊤\)\]​Q⊤D=P\\big\[\(P^\{\\top\}KQ\)\\oslash\(\\mathbf\{1\}\\mathbf\{1\}^\{\\top\}\+\\lambda\\sigma^\{\\top\}\)\\big\]Q^\{\\top\}, with⊘\\oslashthe entrywise division\.

###### Proof\.

\(i\)Wi=Ci​DW\_\{i\}=C\_\{i\}Dfor alliiif and only if every row of everyWiW\_\{i\}lies in the row space ofDD\. \(ii\) The objective is a convex quadratic function ofDD; set its gradient to zero\. \(iii\) SubstituteD=P​X​Q⊤D=PXQ^\{\\top\}and multiply byP⊤P^\{\\top\}on the left and byQQon the right to obtainX\+diag⁡\(λ\)​X​diag⁡\(σ\)=P⊤​K​QX\+\\operatorname\{diag\}\(\\lambda\)X\\operatorname\{diag\}\(\\sigma\)=P^\{\\top\}KQ, which is solved entrywise; the denominators are at least11becauseλ,σ≥0\\lambda,\\sigma\\geq 0, asCCandEEare positive semidefinite\. ∎

### A\.3Compression, circuits and Boltzmann machines

###### Theorem A\.13\(One optimizer;[Property5](https://arxiv.org/html/2610.31931#Thmproperty5)\)\.

Let every matrixμ\\muhave a finite menu𝒪μ\\mathcal\{O\}\_\{\\mu\}of options with sizesbμ​\(o\)b\_\{\\mu\}\(o\)and accuracy costscμ​\(o\)c\_\{\\mu\}\(o\), and considermin∑μcμ\(oμ\)\\min\\sum\_\{\\mu\}c\_\{\\mu\}\(o\_\{\\mu\}\)subject to∑μbμ​\(oμ\)≤B\\sum\_\{\\mu\}b\_\{\\mu\}\(o\_\{\\mu\}\)\\leq B\. \(i\) Replacing every menu by its Pareto set, the options not dominated in both size and cost, does not change the optimum\. \(ii\) Dynamic programming over a grid ofUUsize units solves the problem exactly, with sizes rounded up to the grid so that its solutions are feasible, inO⁡\(U​∑μ\|Pareto⁡\(𝒪μ\)\|\)O\\big\(U\\sum\_\{\\mu\}\|\\operatorname\{Pareto\}\(\\mathcal\{O\}\_\{\\mu\}\)\|\\big\)time\[[69](https://arxiv.org/html/2610.31931#bib.bib69)\]; its value lies between the optima of the unrounded problem for the budgetBBand forBBreduced by one grid unit per matrix\. \(iii\) The optimum is at most the cost of any single method or uniform setting that fits the budget\.

###### Proof\.

\(i\) Replacing an option by one that dominates it keeps the budget and does not increase the cost\. \(ii\) The recursionDPμ​\(u\)=mino⁡\{DPμ−1​\(u−⌈bμ​\(o\)⌉\)\+cμ​\(o\)\}\\mathrm\{DP\}\_\{\\mu\}\(u\)=\\min\_\{o\}\\\{\\mathrm\{DP\}\_\{\\mu\-1\}\(u\-\\lceil b\_\{\\mu\}\(o\)\\rceil\)\+c\_\{\\mu\}\(o\)\\\}enumerates all choices\. Its solutions are feasible forBB, and a choice that fitsBBreduced by one unit per matrix still fits after rounding up, since each size grows by less than one unit\. \(iii\) One method applied to every matrix is one feasible choice\. ∎

###### Theorem A\.14\(Shared subtrees in the optimizer;[Property5](https://arxiv.org/html/2610.31931#Thmproperty5)\)\.

Let the candidate groups for shared subtrees form a forest whose leaves are single matrices, and let every internal nodevvoffer a set𝒮v\\mathcal\{S\}\_\{v\}of \(size, cost\) options, each sharing one subtree among all matrices belowvv\. With additive costs, the recursionPareto⁡\(v\)=Pareto⁡\(\(⨁c∈ch⁡\(v\)Pareto⁡\(c\)\)∪𝒮v\)\\operatorname\{Pareto\}\(v\)=\\operatorname\{Pareto\}\\big\(\(\\bigoplus\_\{c\\in\\mathrm\{ch\}\(v\)\}\\operatorname\{Pareto\}\(c\)\)\\cup\\mathcal\{S\}\_\{v\}\\big\), with⊕\\oplusthe Minkowski sum of \(size, cost\) sets andPareto⁡\(μ\)=Pareto⁡\(𝒪μ\)\\operatorname\{Pareto\}\(\\mu\)=\\operatorname\{Pareto\}\(\\mathcal\{O\}\_\{\\mu\}\)at the leaves, followed by[TheoremA\.13](https://arxiv.org/html/2610.31931#A1.Thmtheorem13)over the roots, is exact over all configurations in which the shared groups are pairwise disjoint\.

###### Proof\.

In an admissible configuration, the matrices belowvvare either covered by one option atvvor split among the children ofvvwith independent configurations whose sizes and costs add\. By induction, the attainable pairs belowvvare\(⨁c𝒞c\)∪𝒮v\(\\bigoplus\_\{c\}\\mathcal\{C\}\_\{c\}\)\\cup\\mathcal\{S\}\_\{v\}, where𝒞c\\mathcal\{C\}\_\{c\}is the set of attainable pairs belowcc, and by[TheoremA\.13](https://arxiv.org/html/2610.31931#A1.Thmtheorem13)\(i\) it suffices to keep Pareto sets at every step\. With sizes rounded up to the grid of[TheoremA\.13](https://arxiv.org/html/2610.31931#A1.Thmtheorem13)once, in the menus, all sums stay on the grid and every Pareto set has at mostU\+1U\+1points\. ∎

###### Proposition A\.15\(Limits of shift\-and\-add circuits;[Section6](https://arxiv.org/html/2610.31931#S6)\)\.

\(i\) Consider straight\-line programs whose instructions add or subtract two earlier values \(inputs or results\), each shifted by at mostSSpositions\. If such a program of lengthLLcomputesx↦W​xx\\mapsto Wxexactly, then for all but a vanishing fraction ofW∈\{0,…,2b−1\}N×NW\\in\\\{0,\\dots,2^\{b\}\-1\\\}^\{N\\times N\},L​log⁡\(N​L​S\)=Ω⁡\(N2​b\)L\\log\(NLS\)=\\Omega\(N^\{2\}b\); forSSpolynomial inNNandbbthis givesL=Ω⁡\(N2​b/log⁡\(N​b\)\)L=\\Omega\\big\(N^\{2\}b/\\log\(Nb\)\\big\)\. \(ii\)O⁡\(N2​b/log⁡N\)O\(N^\{2\}b/\\log N\)additions suffice: splitWWintobbbinary matrices, groupg≈log2⁡N−log2⁡log2​Ng\\approx\\log\_\{2\}N\-\\log\_\{2\}\\log\_\{2\}Nrows, add every input once into the bucket indexed by itsgg\-bit column pattern, combine the bucket sums into the outputs of the group, and combine the bit planes with shifts\[[4](https://arxiv.org/html/2610.31931#bib.bib4),[45](https://arxiv.org/html/2610.31931#bib.bib45)\]\.

###### Proof\.

\(i\) An instruction is described by two operand indices among at mostN\+LN\+Lvalues, two shifts and a sign, i\.e\. byO⁡\(log⁡\(N​L​S\)\)O\(\\log\(NLS\)\)bits, and the program also names itsNNoutputs\. So there are2O⁡\(L​log⁡\(N​L​S\)\)\+N​log⁡\(N\+L\)2^\{O\(L\\log\(NLS\)\)\+N\\log\(N\+L\)\}programs, while there are2N2​b2^\{N^\{2\}b\}matrices, and different matrices need different programs, since a program determines the map it computes\. IfL≥N2​bL\\geq N^\{2\}bthe second bound holds trivially; otherwiselog⁡\(N​L​S\)=O⁡\(log⁡\(N​b\)\)\\log\(NLS\)=O\(\\log\(Nb\)\)forSSpolynomial inNNandbb\. \(ii\) For each of thebbbit planes and each of theN/gN/grow groups, adding theNNinputs into2g2^\{g\}buckets takesNNadditions, and each of theggoutputs sums at most2g−12^\{g\-1\}buckets, so the group costs at mostN\+g​2g≤2​NN\+g2^\{g\}\\leq 2Nadditions by the choice ofgg\. In total this isO⁡\(N2​b/g\)=O⁡\(N2​b/log⁡N\)O\(N^\{2\}b/g\)=O\(N^\{2\}b/\\log N\), plusO⁡\(N​b\)O\(Nb\)additions for the bit planes\. ∎

###### Theorem A\.16\(The Boltzmann machine of a tree;[Property6](https://arxiv.org/html/2610.31931#Thmproperty6)\)\.

Fixρ\\rhoandβ\\beta, and encode the states of a state tree forWWwith binary units: a leaf with valuesa1,…,aKa\_\{1\},\\dots,a\_\{K\}as a one\-hot groupzi​j,1,…,zi​j,Kz\_\{ij,1\},\\dots,z\_\{ij,K\}; a shift bit with multiplierrbr^\{b\}as the affine factor1\+\(r−1\)​b1\+\(r\-1\)b; a gate as a bit that multiplies its subtree; and an index into a fixed table as a one\-hot group\. As the energy takefHf\_\{H\}of[Equation2](https://arxiv.org/html/2610.31931#S5.E2), orfG,H=tr⁡\(G​E​H​E⊤\)f\_\{G,H\}=\\operatorname\{tr\}\(GEHE^\{\\top\}\), plus a penaltyλ​\(∑kzk−1\)2\\lambda\(\\sum\_\{k\}z\_\{k\}\-1\)^\{2\}for every one\-hot group\. \(i\) With the node states fixed, the energy is quadratic in the leaf units\. WriteW^i​j=∑kvi​j,k​zi​j,k\\widehat\{W\}\_\{ij\}=\\sum\_\{k\}v\_\{ij,k\}z\_\{ij,k\}withvi​j,k=ρi​βj​πi​j​akv\_\{ij,k\}=\\rho\_\{i\}\\beta\_\{j\}\\pi\_\{ij\}a\_\{k\}, whereπi​j\\pi\_\{ij\}is the product of the fixed node values on the path\. UnderfHf\_\{H\}, units of entries\(i,j\)\(i,j\)and\(i,l\)\(i,l\)withj≠lj\\neq lare coupled withJ=2​vi​j,k​vi​l,k′​Hj​lJ=2v\_\{ij,k\}v\_\{il,k^\{\\prime\}\}H\_\{jl\}, the coefficient ofzi​j,k​zi​l,k′z\_\{ij,k\}z\_\{il,k^\{\\prime\}\}in[Equation4](https://arxiv.org/html/2610.31931#S7.E4), and units in different rows are not coupled\. UnderfG,Hf\_\{G,H\}, units of entries\(i,j\)≠\(i′,l\)\(i,j\)\\neq\(i^\{\\prime\},l\)are coupled withJ=2​vi​j,k​vi′​l,k′​Gi​i′​Hj​lJ=2v\_\{ij,k\}v\_\{i^\{\\prime\}l,k^\{\\prime\}\}G\_\{ii^\{\\prime\}\}H\_\{jl\}\. Two units of the same leaf \(k≠k′k\\neq k^\{\\prime\}\) are coupled withJ=2​vi​j,k​vi​j,k′​Hj​j\+2​λJ=2v\_\{ij,k\}v\_\{ij,k^\{\\prime\}\}H\_\{jj\}\+2\\lambda, withGi​i​Hj​jG\_\{ii\}H\_\{jj\}in place ofHj​jH\_\{jj\}underfG,Hf\_\{G,H\}\. Every unit has the fieldhi​j,k=vi​j,k2​Hj​j−2​vi​j,k​\(W​H\)i​j−λh\_\{ij,k\}=v\_\{ij,k\}^\{2\}H\_\{jj\}\-2v\_\{ij,k\}\(WH\)\_\{ij\}\-\\lambda, orvi​j,k2​Gi​i​Hj​j−2​vi​j,k​\(G​W​H\)i​j−λv\_\{ij,k\}^\{2\}G\_\{ii\}H\_\{jj\}\-2v\_\{ij,k\}\(GWH\)\_\{ij\}\-\\lambdaunderfG,Hf\_\{G,H\}, up to a constant\. \(ii\) With the node states free, and every node value affine in one bit or linear in one one\-hot group, the energy is a polynomial of degree at most2​D\+22D\+2for a tree of depthDD, and each of its terms involves the units of at most two root\-to\-leaf paths\. Auxiliary units and penalties reduce it to a quadratic function\[[6](https://arxiv.org/html/2610.31931#bib.bib6),[31](https://arxiv.org/html/2610.31931#bib.bib31)\]\. \(iii\) Ifλ\\lambdaexceeds the range of the error over all binary assignments, every ground state satisfies the one\-hot constraints, and the matrix it decodes to minimizes the error overℳ⁡\(𝒯\)\\mathcal\{M\}\(\\mathcal\{T\}\)\.

###### Proof\.

\(i\) Witheie\_\{i\}theii\-th row ofEE,fH=∑iei​H​ei⊤f\_\{H\}=\\sum\_\{i\}e\_\{i\}He\_\{i\}^\{\\top\}andfG,H=∑i,i′Gi​i′​ei​H​ei′⊤f\_\{G,H\}=\\sum\_\{i,i^\{\\prime\}\}G\_\{ii^\{\\prime\}\}e\_\{i\}He\_\{i^\{\\prime\}\}^\{\\top\}\. SubstituteW^i​j\\widehat\{W\}\_\{ij\}, which is linear in the leaf units, and usez2=zz^\{2\}=z\. \(ii\) By assumption, each node value is affine in one bit or linear in a one\-hot group, soW^i​j\\widehat\{W\}\_\{ij\}is a polynomial of degree at mostD\+1D\+1in the units on its path, and the error is quadratic in the entries\. A lookup whose value depends on its own state and on the state of an ancestor is bilinear in their units, and raises the degree by at most one for each ancestor it reads\. \(iii\) An assignment that violates a one\-hot constraint pays a penalty of at leastλ\\lambda, which then exceeds the difference between the error of any feasible assignment and the smallest error overall\. ∎

#### Convolutions\.

A convolution computesW​patch​\(p\)W\\,\\mathrm\{patch\}\(p\)at every positionppwith the same kernel matrixWW\. Written as one matrix acting on the whole input, its blocks for different positions are copies ofWWat shifted columns, and each copy is a reference to one shared subtree\. Grouped and depthwise convolutions are block\-diagonal over channel groups, that is, disjoint channel subtrees\.

### A\.4Experimental details

#### Common setup\.

Perplexity is measured at context 2048 on WikiText\-2, C4 and FineWeb; FP16 LLaMA\-3\-8B gives 6\.14, 9\.44 and 7\.85 in the same pipeline\. Matrices are compressed one block after another, with Hessians computed afresh from 128 FineWeb calibration sequences and keeping the attention\-sink token\. MMLU and GSM8K are evaluated 5\-shot with lm\-eval, with the same settings for every model\. Compression and healing use PyTorch 2\.7\.1 and Transformers 5; serving uses vLLM 0\.32\.0 with PyTorch 2\.13\.0 and CUDA 12\.9\. All GPU measurements are on an A100\-SXM4\-40GB\.

#### Lossless storage\.

Each row is split into groups of 64 consecutive weights\. The group node stores the largest BF16 exponentEEof the group \(8 bits\); each weight stores the stateD=E−e∈\{0,…,14\}D=E\-e\\in\\\{0,\\dots,14\\\}of its exponentee\(4 bits\) and a leaf with its sign and 7\-bit mantissa \(8 bits\), 12\.125 bits per weight\. The state 15 is an escape: a weight withD≥15D\\geq 15, which includes exact zeros and subnormals, has its leaf set to zero and is stored in a side list with its 16 BF16 bits and its row and column, 80 bits in all\. Escapes are rare, 0\.0095% of the weights of Llama\-3\.2\-1B and 0\.0097% of those of Qwen3\.5\-9B and at most 0\.014% in any layer, so whole models, embeddings and output head included, take 12\.133 bits per weight on disk\. At runtime every escape is repacked as one 32\-bit entry, and the kernel layout of a single LLaMA\-3\-8B matrix takes 12\.137 bits per weight\. The fused kernel patches escaped weights in registers from a 64\-byte block per work unit, holding a bit mask, a count and up to 13 entries, and reads any further escapes from a global list\. Every weight of every matrix tested decodes to its exact BF16 bits\.

#### Whole\-model optimizer\.

Every block linear layer receives the same menu of quantization, pruning and low\-rank edits, alone and combined, with no rotation and no trellis\-coded quantizer; the menus will be described in a follow\-up paper\. The cost of an option is the measured curvature cost of[Section9](https://arxiv.org/html/2610.31931#S9), on 16 calibration sequences\. GPTQ at 4, 3 and 2 bits is measured for all 224 matrices; the costs of the other options are first interpolated along each matrix’s curve of cost againstfHf\_\{H\}\. Every option the solver picks is then built and measured, and the knapsack is solved again until the whole plan has been measured; in the 4\.25\-bit run \(26\.6% of the FP16 bytes\) this took nine rounds, and the 3\.25\-bit run of[Section9](https://arxiv.org/html/2610.31931#S9)uses 20\.3% of the bytes of the block weights\. Three further rounds re\-measure the costs with the whole plan applied\.

#### Ising healing\.

The 2\.44\-bit model, an illustration, is Qwen2\.5\-1\.5B\-Instruct compressed without training by the optimizer with a wider menu, to 15\.2% of the bytes of its block weights; the menu will be described in a follow\-up paper\. Healing starts from a rebuild that reruns the whole pipeline with a recorder of the leaf states\. Its re\-measured costs give a slightly different plan, with perplexities of 13\.74, 20\.95 and 20\.88 on WikiText\-2, C4 and FineWeb against 13\.63, 20\.75 and 20\.71 for the first run; a second rebuild gave 13\.72, 20\.84 and 20\.85, so runs differ by about 0\.1\. All gains are measured against the rebuild\. Only leaves of the GPTQ family, 6\.8% of the weights, can move; all other leaves stay fixed\. Each step estimates the gradient of the KL divergence and its Kronecker factorsH=𝔼⁡\[x​x⊤\]H=\\mathbb\{E\}\[xx^\{\\top\}\]andG=𝔼⁡\[g​g⊤\]G=\\mathbb\{E\}\[gg^\{\\top\}\]on about 45k tokens, with labels sampled from the student\. A move must lower the energy by more than a threshold, set so that a fractionqqof the weights qualifies; the lowest energy among no move, the greedy update and simulated bifurcation started from it is kept; and the step is accepted only if the validation KL improves, after whichqqgrows by a factor of 1\.5, or else shrinks by a factor of 0\.3\. There are 40 steps\. After healing, the perplexities are 12\.85, 19\.51 and 20\.02\. In a solver ablation with the same model, data and number of steps, greedy updates alone reach a validation KL of 0\.378 \(GSM8K 20\.32, MMLU 49\.49\), simulated bifurcation 0\.371 \(21\.00, 49\.29\), local search 0\.369 and random moves 0\.383; the reported run keeps the best of no move, the greedy update and simulated bifurcation at every step\.

#### Image coding on CLIC\.

On the 41 images of the CLIC 2020 professional validation set, the GPU codec needs 8\.51 bits per pixel, against 7\.88 for JPEG XL at effort 9 \(7\.97 at effort 7\), 8\.81 for lossless WebP and 11\.71 for PNG, all run through imagecodecs 2026\.3\.6 on CPU \(libjxl 0\.11\.2; libwebp 1\.6\.0 at method 4, against method 6 on Kodak; libpng 1\.6\.55 with zlib at level 9\)\. The whole set decodes at 2\.86 Gpixel/s end to end, 3\.46 Gpixel/s for the kernels alone, and one image at a time at 0\.59–0\.66 Gpixel/s\. JPEG 2000 and HTJ2K were measured on Kodak only\.

## References

- \[1\]D\. H\. Ackley, G\. E\. Hinton, T\. J\. Sejnowski\. A learning algorithm for Boltzmann machines\.*Cognitive Science*, 9\(1\):147–169, 1985\.
- \[2\]E\. Agustsson, R\. Timofte\. NTIRE 2017 challenge on single image super\-resolution: dataset and study\.*CVPR Workshops*, 2017\.
- \[3\]J\. Alakuijala et al\. JPEG XL next\-generation image compression architecture and coding tools\.*Proc\. SPIE 11137, Applications of Digital Image Processing XLII*, 2019\.
- \[4\]V\. L\. Arlazarov, E\. A\. Dinic, M\. A\. Kronrod, I\. A\. Faradzev\. On economical construction of the transitive closure of an oriented graph\.*Soviet Mathematics Doklady*, 1970\.
- \[5\]L\. Babai\. On Lovász’ lattice reduction and the nearest lattice point problem\.*Combinatorica*, 6\(1\):1–13, 1986\.
- \[6\]E\. Boros, P\. L\. Hammer\. Pseudo\-Boolean optimization\.*Discrete Applied Mathematics*, 123:155–225, 2002\.
- \[7\]R\. E\. Bryant, Y\.\-A\. Chen\. Verification of arithmetic circuits with binary moment diagrams\.*Design Automation Conference \(DAC\)*, 1995\.
- \[8\]K\. Chellapilla, S\. Puri, P\. Simard\. High performance convolutional neural networks for document processing\.*International Workshop on Frontiers in Handwriting Recognition*, 2006\.
- \[9\]H\. Chen, Y\. Wang, C\. Xu, B\. Shi, C\. Xu, Q\. Tian, C\. Xu\. AdderNet: do we really need multiplications in deep learning?*CVPR*, 2020\.
- \[10\]J\. Chen, Y\. Shabanzadeh, E\. Crnčević, T\. Hoefler, D\. Alistarh\. The geometry of LLM quantization: GPTQ as Babai’s nearest plane algorithm\.*ICLR*, 2026\.
- \[11\]S\. A\. Cuccaro, T\. G\. Draper, S\. A\. Kutin, D\. P\. Moulton\. A new quantum ripple\-carry addition circuit\. arXiv:quant\-ph/0410184, 2004\.
- \[12\]K\. Culik II, J\. Kari\. Image compression using weighted finite automata\.*Computers & Graphics*, 17\(3\):305–313, 1993\.
- \[13\]T\. Dao, A\. Gu, M\. Eichhorn, A\. Rudra, C\. Ré\. Learning fast algorithms for linear transforms using butterfly factorizations\.*ICML*, 2019\.
- \[14\]J\. Duda\. Asymmetric numeral systems: entropy coding combining speed of Huffman coding with compression rate of arithmetic coding\. arXiv:1311\.2540, 2013\.
- \[15\]C\. Eckart, G\. Young\. The approximation of one matrix by another of lower rank\.*Psychometrika*, 1936\.
- \[16\]V\. Egiazarian, A\. Panferov, D\. Kuznedelev, E\. Frantar, A\. Babenko, D\. Alistarh\. Extreme compression of large language models via additive quantization\.*ICML*, 2024\.
- \[17\]M\. Elhoushi, Z\. Chen, F\. Shafiq, Y\. H\. Tian, J\. Y\. Li\. DeepShift: towards multiplication\-less neural networks\.*CVPR Workshops*, 2021\.
- \[18\]J\. Faraone, M\. Kumm, M\. Hardieck, P\. Zipf, X\. Liu, D\. Boland, P\. H\. W\. Leong\. AddNet: deep neural networks using FPGA\-optimized multipliers\.*IEEE Transactions on Very Large Scale Integration \(VLSI\) Systems*, 28\(1\), 2020\.
- \[19\]E\. Farhi, J\. Goldstone, S\. Gutmann\. A quantum approximate optimization algorithm\. arXiv:1411\.4028, 2014\.
- \[20\]E\. Frantar, S\. P\. Singh, D\. Alistarh\. Optimal Brain Compression: a framework for accurate post\-training quantization and pruning\.*NeurIPS*, 2022\.
- \[21\]E\. Frantar, D\. Alistarh\. SPDY: accurate pruning with speedup guarantees\.*ICML*, 2022\.
- \[22\]E\. Frantar, S\. Ashkboos, T\. Hoefler, D\. Alistarh\. GPTQ: accurate post\-training quantization for generative pre\-trained transformers\.*ICLR*, 2023\.
- \[23\]E\. Frantar, D\. Alistarh\. SparseGPT: massive language models can be accurately pruned in one\-shot\.*ICML*, 2023\.
- \[24\]A\. Gersho, R\. M\. Gray\.*Vector Quantization and Signal Compression*\. Kluwer, 1992\.
- \[25\]F\. Giesen\. Interleaved entropy coders\. arXiv:1402\.3392, 2014\.
- \[26\]H\. Goto, K\. Tatsumura, A\. R\. Dixon\. Combinatorial optimization by simulating adiabatic bifurcations in nonlinear Hamiltonian systems\.*Science Advances*, 5\(4\):eaav2372, 2019\.
- \[27\]H\. Goto, K\. Endo, M\. Suzuki, Y\. Sakai, T\. Kanao, Y\. Hamakawa, R\. Hidaka, M\. Yamasaki, K\. Tatsumura\. High\-performance combinatorial optimization based on classical mechanics\.*Science Advances*, 7\(6\):eabe7953, 2021\.
- \[28\]W\. Hackbusch\. A sparse matrix arithmetic based onℋ\\mathcal\{H\}\-matrices\. Part I: Introduction toℋ\\mathcal\{H\}\-matrices\.*Computing*, 62\(2\):89–108, 1999\.
- \[29\]B\. Hassibi, D\. G\. Stork\. Second order derivatives for network pruning: optimal brain surgeon\.*NeurIPS*, 1993\.
- \[30\]M\. Horowitz\. Computing’s energy problem \(and what we can do about it\)\.*IEEE International Solid\-State Circuits Conference \(ISSCC\)*, 2014\.
- \[31\]H\. Ishikawa\. Transformation of general binary MRF minimization to the first\-order case\.*IEEE TPAMI*, 33\(6\):1234–1249, 2011\.
- \[32\]A\. J\. Izenman\. Reduced\-rank regression for the multivariate linear model\.*Journal of Multivariate Analysis*, 5\(2\):248–264, 1975\.
- \[33\]B\. Jacob, S\. Kligys, B\. Chen, M\. Zhu, M\. Tang, A\. Howard, H\. Adam, D\. Kalenichenko\. Quantization and training of neural networks for efficient integer\-arithmetic\-only inference\.*CVPR*, 2018\.
- \[34\]T\. Kadowaki, H\. Nishimori\. Quantum annealing in the transverse Ising model\.*Physical Review E*, 58:5355, 1998\.
- \[35\]B\. N\. Khoromskij\.O⁡\(d​log⁡N\)O\(d\\log N\)\-quantics approximation ofNN\-ddtensors in high\-dimensional numerical modeling\.*Constructive Approximation*, 34\(2\):257–280, 2011\.
- \[36\]S\. Kim, A\. Gholami, Z\. Yao, M\. W\. Mahoney, K\. Keutzer\. I\-BERT: integer\-only BERT quantization\.*ICML*, 2021\.
- \[37\]S\. Kim, C\. Hooper, A\. Gholami, Z\. Dong, X\. Li, S\. Shen, M\. W\. Mahoney, K\. Keutzer\. SqueezeLLM: dense\-and\-sparse quantization\.*ICML*, 2024\.
- \[38\]S\. Kirkpatrick, C\. D\. Gelatt, M\. P\. Vecchi\. Optimization by simulated annealing\.*Science*, 220\(4598\):671–680, 1983\.
- \[39\]Eastman Kodak\. Kodak Lossless True Color Image Suite \(PhotoCD PCD0992\)\.[http://r0k\.us/graphics/kodak/](http://r0k.us/graphics/kodak/)\.
- \[40\]Y\.\-T\. Lai, S\. Sastry\. Edge\-valued binary decision diagrams for multi\-level hierarchical verification\.*Design Automation Conference \(DAC\)*, 1992\.
- \[41\]Z\. Lan, M\. Chen, S\. Goodman, K\. Gimpel, P\. Sharma, R\. Soricut\. ALBERT: a lite BERT for self\-supervised learning of language representations\.*ICLR*, 2020\.
- \[42\]A\. Lavin, S\. Gray\. Fast algorithms for convolutional neural networks\.*CVPR*, 2016\.
- \[43\]Y\. LeCun, J\. S\. Denker, S\. A\. Solla\. Optimal brain damage\.*NeurIPS*, 1990\.
- \[44\]Y\. Li, X\. Dong, W\. Wang\. Additive powers\-of\-two quantization: an efficient non\-uniform discretization for neural networks\.*ICLR*, 2020\.
- \[45\]E\. Liberty, S\. W\. Zucker\. The Mailman algorithm: a note on matrix–vector multiplication\.*Information Processing Letters*, 2009\.
- \[46\]Y\. C\. Lim, R\. Yang, D\. Li, J\. Song\. Signed power\-of\-two term allocation scheme for the design of digital filters\.*IEEE Transactions on Circuits and Systems II*, 46\(5\):577–584, 1999\.
- \[47\]J\. Lin, J\. Tang, H\. Tang, S\. Yang, et al\. AWQ: activation\-aware weight quantization for on\-device LLM compression and acceleration\.*MLSys*, 2024\.
- \[48\]Llama Team, AI @ Meta\. The Llama 3 herd of models\. arXiv:2407\.21783, 2024\.
- \[49\]A\. Lucas\. Ising formulations of many NP problems\.*Frontiers in Physics*, 2:5, 2014\.
- \[50\]Y\. Luo, J\. Huang, Y\. Cheng, et al\. HiFloat4 format for language model inference\. arXiv:2602\.11287, 2026\.
- \[51\]S\. Ma, H\. Wang, L\. Ma, et al\. The era of 1\-bit LLMs: all large language models are in 1\.58 bits\. arXiv:2402\.17764, 2024\.
- \[52\]S\. Ma, H\. Wang, et al\. BitNet b1\.58 2B4T technical report\. arXiv:2504\.12285, 2025\.
- \[53\]J\. Martens, R\. Grosse\. Optimizing neural networks with Kronecker\-factored approximate curvature\.*ICML*, 2015\.
- \[54\]X\. Meng, K\. Behdin, H\. Wang, R\. Mazumder\. ALPS: improved optimization for highly sparse one\-shot pruning for large language models\.*NeurIPS*, 2024\.
- \[55\]D\. M\. Miller, M\. A\. Thornton\. QMDD: a decision diagram structure for reversible and quantum circuits\.*International Symposium on Multiple\-Valued Logic \(ISMVL\)*, 2006\.
- \[56\]D\. Miyashita, E\. H\. Lee, B\. Murmann\. Convolutional neural networks using logarithmic data representation\. arXiv:1603\.01025, 2016\.
- \[57\]M\. H\. Mubarik, D\. D\. Weller, N\. Bleier, M\. Tomei, J\. Aghassi\-Hagmann, M\. B\. Tahoori, R\. Kumar\. Printed machine learning classifiers\.*IEEE/ACM MICRO*, 2020\.
- \[58\]S\. Muñiz Subiñas, M\. L\. González, J\. Ruiz Gómez, A\. Mata Ali, J\. Martínez Martín, M\. Franco Hernando, Á\. M\. García\-Vico\. Optimization of the quantization of dense neural networks from an exact QUBO formulation\. arXiv:2510\.16075, 2025\.
- \[59\]M\. Nagel, R\. A\. Amjad, M\. van Baalen, C\. Louizos, T\. Blankevoort\. Up or down? Adaptive rounding for post\-training quantization\.*ICML*, 2020\.
- \[60\]I\. V\. Oseledets\. Tensor\-train decomposition\.*SIAM Journal on Scientific Computing*, 33\(5\):2295–2317, 2011\.
- \[61\]G\. Park, B\. Park, M\. Kim, S\. Lee, J\. Kim, B\. Kwon, S\. J\. Kwon, B\. Kim, Y\. Lee, D\. Lee\. LUT\-GEMM: quantized matrix multiplication based on LUTs for efficient inference in large\-scale generative language models\.*ICLR*, 2024\.
- \[62\]Qwen Team\. Qwen3\.5: towards native multimodal agents\.[https://qwen\.ai/blog?id=qwen3\.5](https://qwen.ai/blog?id=qwen3.5), February 2026\.
- \[63\]G\. W\. Reitwiesner\. Binary arithmetic\.*Advances in Computers*, vol\. 1, 1960\.
- \[64\]WebP Image Format\. IETF RFC 9649, 2024\.
- \[65\]B\. Darvish Rouhani, D\. Lo, R\. Zhao, et al\. Pushing the limits of narrow precision inferencing at cloud scale with Microsoft floating point\.*NeurIPS*, 2020\.
- \[66\]B\. Darvish Rouhani, R\. Zhao, A\. More, et al\. Microscaling data formats for deep learning\. arXiv:2310\.10537, 2023\.
- \[67\]B\. Darvish Rouhani, R\. Zhao, V\. Elango, R\. Shafipour, M\. Hall, et al\. With shared microexponents, a little shifting goes a long way\.*International Symposium on Computer Architecture \(ISCA\)*, 2023\.
- \[68\]T\. J\. Sejnowski\. Higher\-order Boltzmann machines\.*AIP Conference Proceedings*, 151:398–403, 1986\.
- \[69\]P\. Sinha, A\. A\. Zoltners\. The multiple\-choice knapsack problem\.*Operations Research*, 1979\.
- \[70\]M\. Sun, Z\. Liu, A\. Bair, J\. Z\. Kolter\. A simple and effective pruning approach for large language models\.*ICLR*, 2024\.
- \[71\]Taalas\. HC1: a technology demonstrator with Llama 3\.1 8B hardwired in silicon\. Product announcement, February 2026\.[https://taalas\.com/products/](https://taalas.com/products/), accessed October 2026\.
- \[72\]D\. S\. Taubman, M\. W\. Marcellin\.*JPEG2000: Image Compression Fundamentals, Standards and Practice*\. Kluwer, 2002\.
- \[73\]D\. Taubman, A\. Naman, R\. Mathew\. High throughput block coding in the HTJ2K compression standard\.*IEEE ICIP*, 2019\.
- \[74\]M\. Thoma, M\. Groezinger, M\. Forstenhäusler, et al\. Advancing SVD\-based LLM compression via layer\-wise error model search\.*ICML*, 2026\.
- \[75\]G\. Toderici et al\. Workshop and Challenge on Learned Image Compression \(CLIC2020\)\.*CVPR Workshops*, 2020\.
- \[76\]A\. Tseng, Q\. Sun, D\. Hou, C\. De Sa\. QTIP: quantization with trellises and incoherence processing\.*NeurIPS*, 2024\.
- \[77\]A\. Tseng, J\. Chee, Q\. Sun, V\. Kuleshov, C\. De Sa\. QuIP\#: even better LLM quantization with Hadamard incoherence and lattice codebooks\.*ICML*, 2024\.
- \[78\]A\. Tseng, Z\. Sun, C\. De Sa\. Model\-preserving adaptive rounding\.*ICML*, 2026\.
- \[79\]M\. van Baalen, A\. Kuzmin, I\. Koryakovskiy, M\. Nagel, P\. Couperus, C\. Bastoul, E\. Mahurin, T\. Blankevoort, P\. Whatmough\. GPTVQ: the blessing of dimensionality for LLM quantization\. arXiv:2402\.15319, 2024\.
- \[80\]A\. Vaswani, N\. Shazeer, N\. Parmar, J\. Uszkoreit, L\. Jones, A\. N\. Gomez, Ł\. Kaiser, I\. Polosukhin\. Attention is all you need\.*NeurIPS*, 2017\.
- \[81\]Y\. Voronenko, M\. Püschel\. Multiplierless multiple constant multiplication\.*ACM Transactions on Algorithms*, 2007\.
- \[82\]Z\. Wang, B\. C\. M\. Choong, T\. Huang, D\. Gerlinghoff, R\. S\. M\. Goh, C\. Liu, T\. Luo\. Is quantum optimization ready? An effort towards neural network compression using adiabatic quantum computing\. arXiv:2505\.16332, 2025\.
- \[83\]J\. Wang, Y\. Chen, I\. Lin, B\. Li, G\. L\. Zhang\. Basis sharing: cross\-layer parameter sharing for large language model compression\.*ICLR*, 2025\.
- \[84\]X\. Wang, Y\. Zheng, Z\. Wan, M\. Zhang\. SVD\-LLM: truncation\-aware singular value decomposition for large language model compression\.*ICLR*, 2025\.
- \[85\]J\. Wei, S\. Cao, T\. Cao, L\. Ma, L\. Wang, Y\. Zhang, M\. Yang\. T\-MAC: CPU renaissance via table lookup for low\-bit LLM deployment on edge\. arXiv:2407\.00088, 2024\.
- \[86\]C\. Xu, J\. Yao, Z\. Lin, W\. Ou, Y\. Cao, Z\. Wang, H\. Zha\. Alternating multi\-bit quantization for recurrent neural networks\.*ICLR*, 2018\.
- \[87\]Z\. Yao, Z\. Dong, Z\. Zheng, A\. Gholami, J\. Yu, E\. Tan, L\. Wang, Q\. Huang, Y\. Wang, M\. W\. Mahoney, K\. Keutzer\. HAWQ\-V3: dyadic neural network quantization\.*ICML*, 2021\.
- \[88\]H\. You, X\. Chen, Y\. Zhang, C\. Li, S\. Li, Z\. Liu, Z\. Wang, Y\. Lin\. ShiftAddNet: a hardware\-inspired deep network\.*NeurIPS*, 2020\.
- \[89\]H\. You, Y\. Guo, Y\. Fu, et al\. ShiftAddLLM: accelerating pretrained LLMs via post\-training multiplication\-less reparameterization\.*NeurIPS*, 2024\.
- \[90\]A\. Zhou, A\. Yao, Y\. Guo, L\. Xu, Y\. Chen\. Incremental network quantization: towards lossless CNNs with low\-precision weights\.*ICLR*, 2017\.
- \[91\]A\. Ali, B\. Mohammad, D\. Makhov, D\. Shopkhoev, M\. Zhussip, S\. Lefkimmiatis\. ROCKET: rapid optimization via calibration\-guided knapsack enhanced truncation for efficient model compression\. arXiv:2602\.11008, 2026\.
- \[92\]D\. Makhov, D\. Shopkhoev, M\. Zhussip, A\. Ali, S\. Lefkimmiatis\. CoSpaDi: compressing LLMs via calibration\-guided sparse dictionary learning\. arXiv:2509\.22075, 2025\.
- \[93\]D\. Makhov, D\. Shopkhoev, M\. Zhussip, A\. Ali, B\. Mohammad, S\. Lefkimmiatis\. COMPOT: calibration\-optimized matrix Procrustes orthogonalization for transformers compression\. arXiv:2602\.15200, 2026\.
- \[94\]B\. Mohammad, A\. Ali, S\. Lefkimmiatis\. GeoPair: geometry\-preserving cross\-layer factorization for training\-free transformer compression\. arXiv:2609\.25963, 2026\.
- \[95\]D\. Shopkhoev, A\. Ali, M\. Zhussip, V\. Malykh, S\. Lefkimmiatis, N\. Komodakis, S\. Zagoruyko\. ReplaceMe: network simplification via depth pruning and transformer block linearization\.*NeurIPS*, 2025\.
- \[96\]M\. Zhussip, D\. Shopkhoev, A\. Ali, S\. Lefkimmiatis\. Share your attention: transformer weight sharing via matrix\-based dictionary learning\.*AAAI*, 2026\.