What Survives When You Compress a Recursive Reasoner for the Edge?

# What Survives When You Compress a Recursive Reasoner for the Edge?
Source: [https://arxiv.org/html/2606.26488](https://arxiv.org/html/2606.26488)
Pearse Jim1,Steven Kolawole2,111footnotemark:1,Opegbemi M\. Busoye1,Glory Bagai1,Virginia Smith2

1ML Collective,2Carnegie Mellon University

###### Abstract

Recursive reasoning models can solve complex structured tasks with only a few million parameters by repeatedly updating a latent state\. Deploying these models on edge hardware requires significant compression, but unlike conventional sequence models, quantization errors compound across recursive reasoning cycles rather than across output tokens\. As a result, standard intuitions about compression fail to apply\. In this work, we ask what survives when recursive reasoners are compressed\. Across a full precision sweep, three tasks, and two recursive architectures, we find that aggressive compression preserves*local*prediction but destroys*global*reasoning: cell accuracy holds while puzzle\-exact accuracy collapses to zero under naïve INT4, pruning, distillation, and linear attention alike\. Token\-level objectives, including quantization\-aware training, cannot repair it\. The collapse is architectural – it strikes MLP\-mixing recursion but not attention on the same task – and we reverse it with per\-channel calibrated INT4 without retraining\. We also introduce carry\-trajectory fidelity, the cosine similarity to the full\-precision reasoning path, as a label\-free signal that predicts this damage and its recovery before a task evaluation\. The combined result is a deployment recipe: flash\-streamed embeddings remove a 99\.4 MB bottleneck, INT8 at one cycle matches full\-depth accuracy at6×6\\timesfewer FLOPs \(8 MB SoC\), and calibrated INT4 fits a 4 MB microcontroller\.

What Survives When You Compress a Recursive Reasoner for the Edge?

Pearse Jim1††thanks:Equal Contribution\., Steven Kolawole2,111footnotemark:1, Opegbemi M\. Busoye1, Glory Bagai1, Virginia Smith21ML Collective,2Carnegie Mellon University

![Refer to caption](https://arxiv.org/html/2606.26488v1/x1.png)Figure 1:Memory footprint by configuration\. The FP32 embedding table \(99\.4 MB\) accounts for 79% of the∼\\sim126 MB baseline\. Flash loading reduces the active embedding to 2 KB, isolating backbone compression\. INT8 \+ flash fits the 8 MB SoC target; INT4 calibrated \+ flash fits the 4 MB MCU target\.## 1Introduction

The Tiny Recursive Model \(TRM\) achieves 36\.00% puzzle exact match on ARC\-2024 using 6\.83M parameters\(Jolicoeur\-Martineau,[2025](https://arxiv.org/html/2606.26488#bib.bib24)\)—competitive with larger autoregressive systems on a benchmark designed to resist pattern matching\(Chollet,[2019](https://arxiv.org/html/2606.26488#bib.bib25)\)\. It does so by iterating a latent carry state across recursive cycles, a mechanism shared by Universal Transformers\(Dehghaniet al\.,[2019](https://arxiv.org/html/2606.26488#bib.bib9)\)and depth\-recurrent language models\(Geipinget al\.,[2025](https://arxiv.org/html/2606.26488#bib.bib31); Wanget al\.,[2025](https://arxiv.org/html/2606.26488#bib.bib8); Haoet al\.,[2025](https://arxiv.org/html/2606.26488#bib.bib10); Kohliet al\.,[2026](https://arxiv.org/html/2606.26488#bib.bib36)\)\. Yet TRM weighs∼\\sim126 MB \(15×15\\timesmobile SoC SRAM\)\. Deployment on resource\-constrained edge hardware requires compression\. When such a model is squeezed, what survives and what breaks?

While single\-pass transformer quantization is well\-studied \(noise accumulates over output tokens, and INT8 deployment is standard\)\(Nagelet al\.,[2021](https://arxiv.org/html/2606.26488#bib.bib16); Dettmerset al\.,[2022](https://arxiv.org/html/2606.26488#bib.bib15); Hanet al\.,[2016](https://arxiv.org/html/2606.26488#bib.bib26)\), recursive models introduce a different failure mode: quantization noise compounds acrossHHrecursive cycles at each carry\-state update\. Whether this compounding degrades reasoning has not been studied\. While quantization harms single\-pass reasoning chains\(Liet al\.,[2025](https://arxiv.org/html/2606.26488#bib.bib19)\)and emergent abilities collapse below 4\-bit\(Liuet al\.,[2024](https://arxiv.org/html/2606.26488#bib.bib20)\), depth and precision are independent in those systems\. In recursive models, they are not: the optimal operating point depends jointly on bit\-width*and*depth\.

We present the first systematic compression study of a recursive reasoning model under hard SRAM constraints\(Linet al\.,[2020](https://arxiv.org/html/2606.26488#bib.bib27); Abushahlaet al\.,[2025](https://arxiv.org/html/2606.26488#bib.bib17)\)\. We sweep precision \(FP32, FP16, INT8, INT4\) and depth across three tasks \(ARC\-2024, Maze\-Hard, Sudoku\-Extreme\) and two architectures \(TRM and HRM\)\. As shown in Figure[1](https://arxiv.org/html/2606.26488#S0.F1), the dominant memory bottleneck is the 99\.4 MB puzzle embedding table, which we resolve via 2 KB flash loading, isolating backbone compression\. Our primary contributions are:

1. 1\.We establish adepth–precision crossover: INT8 at a single recursive cycle matches full\-depth FP32 accuracy at6×6\\timesfewer FLOPs, an interaction absent from single\-pass quantization \(§[4\.2](https://arxiv.org/html/2606.26488#S4.SS2)\)\.
2. 2\.We identify acompositional collapse: under naïve INT4, pruning, distillation, and QAT, local token accuracy survives while global puzzle accuracy falls to zero; an attention\-vs\-MLP\-mixing ablation shows this is architectural, not task\-intrinsic \(§[4\.4](https://arxiv.org/html/2606.26488#S4.SS4)\)\.
3. 3\.We introducecarry\-trajectory fidelity, a label\-free detector that grades this damage and predicts its recovery from the model’s own hidden states, with no task labels \(§[4\.3](https://arxiv.org/html/2606.26488#S4.SS3)\)\.
4. 4\.We derive and benchmark adeployment recipe: per\-channel calibrated INT4 fits a 4 MB MCU and INT8 with flash\-streamed embeddings fits an 8 MB SoC, validated with on\-device latency on three Snapdragon platforms \(§[4\.5](https://arxiv.org/html/2606.26488#S4.SS5),[4\.7](https://arxiv.org/html/2606.26488#S4.SS7)\)\.

## 2Related Work

#### Overthinking in recursive models is named but not detectable\.

Recurrent and weight\-tied architectures are a parameter\-efficient path to algorithmic reasoning\(Dehghaniet al\.,[2019](https://arxiv.org/html/2606.26488#bib.bib9)\), and recent depth\-recurrent language models show iterative latent refinement scales favorably\(Geipinget al\.,[2025](https://arxiv.org/html/2606.26488#bib.bib31); Wanget al\.,[2025](https://arxiv.org/html/2606.26488#bib.bib8); Haoet al\.,[2025](https://arxiv.org/html/2606.26488#bib.bib10); Kohliet al\.,[2026](https://arxiv.org/html/2606.26488#bib.bib36)\)\.Bansalet al\.\([2022](https://arxiv.org/html/2606.26488#bib.bib11)\)name their central failure—overthinking, where too many iterations corrupt the instance—but detect it only by full task evaluation;Duet al\.\([2026](https://arxiv.org/html/2606.26488#bib.bib21)\)andLuet al\.\([2025](https://arxiv.org/html/2606.26488#bib.bib23)\)find correctness signals in latent trajectories, yet none give a signal usable*during inference*without labels\. Our carry\-trajectory fidelity, grounded inJianget al\.\([2025](https://arxiv.org/html/2606.26488#bib.bib22)\)’s link between hidden\-state cosine similarity and representational saturation, fills this gap: it predicts collapse under quantization with no task evaluation\.

#### Quantization\-as\-regularization has an uncharacterized depth dimension\.

Quantization noise can act as a regulariser – pushing toward flatter minima with a model\-specific optimal level\(AskariHemmatet al\.,[2024](https://arxiv.org/html/2606.26488#bib.bib12),[2022](https://arxiv.org/html/2606.26488#bib.bib29)\), robust under distribution shift and limited data\(Javedet al\.,[2025](https://arxiv.org/html/2606.26488#bib.bib13); Tallecet al\.,[2023](https://arxiv.org/html/2606.26488#bib.bib14)\), sometimes letting INT8 beat FP32\(Zhanget al\.,[2025](https://arxiv.org/html/2606.26488#bib.bib30)\)– which explains INT8 parity in single\-pass models\. None predicts recursive behavior, where the same noise is applied acrossHHcarry\-state updates and beneficial regularization at shallow depth becomes destructive compounding at depth\. We characterize this depth–precision interaction directly\.

#### Compressed reasoning fails early; QAT recovery targets the wrong objective\.

On reasoning chains, PTQ induces early failures that cascade\(Liet al\.,[2025](https://arxiv.org/html/2606.26488#bib.bib19)\), with a precision floor near 4\-bit\(Liuet al\.,[2024](https://arxiv.org/html/2606.26488#bib.bib20)\)and W8A8 as the lossless threshold for small models\(Liuet al\.,[2025](https://arxiv.org/html/2606.26488#bib.bib33)\); consistent with our INT8 finding, but for single\-pass decoding\. Standard QAT for reasoning uses distillation and task\-level objectives\(Lvet al\.,[2026](https://arxiv.org/html/2606.26488#bib.bib35)\); we show cross\-entropy QAT fails not from poor calibration but from objective misalignment, recovering cell\-level but not puzzle\-level accuracy\(Nagelet al\.,[2021](https://arxiv.org/html/2606.26488#bib.bib16)\)\.

#### Edge NLP deployment motivates but does not resolve these questions\.

Foundational compression\(Hanet al\.,[2016](https://arxiv.org/html/2606.26488#bib.bib26)\), the finding that pruning degrades small LMs more than quantization\(Zhouet al\.,[2025](https://arxiv.org/html/2606.26488#bib.bib34)\)\(consistent with our pruning results\), INT8 as the deployment\-native format\(Dettmerset al\.,[2022](https://arxiv.org/html/2606.26488#bib.bib15); Kimet al\.,[2021](https://arxiv.org/html/2606.26488#bib.bib28)\), sub\-megabyte MCU inference via hardware co\-design\(Linet al\.,[2020](https://arxiv.org/html/2606.26488#bib.bib27)\), MCU\-quantization surveys at our 4–8 MB constraints\(Abushahlaet al\.,[2025](https://arxiv.org/html/2606.26488#bib.bib17)\), and on\-device NLP pipelines\(Bohdalet al\.,[2025](https://arxiv.org/html/2606.26488#bib.bib32)\): none studies recursive reasoning models or the depth–precision interplay\. No prior work compresses a recursive reasoning model; the closest\(Liet al\.,[2025](https://arxiv.org/html/2606.26488#bib.bib19)\)addresses single\-pass decoding and lacks the recursive carry\-state structure we study\.

## 3Background and Experimental Setup

### 3\.1Tiny Recursive Model \(TRM\)

TRM\(Jolicoeur\-Martineau,[2025](https://arxiv.org/html/2606.26488#bib.bib24)\)is a weight\-tied recursive transformer that solves structured reasoning tasks through iterative latent refinement rather than autoregressive generation\. Let𝒱\\mathcal\{V\}be a finite vocabulary andX∈𝒱TxX\\in\\mathcal\{V\}^\{T\_\{x\}\}be an input sequence\. TRM maintains a latent carry state updated across two nested loops\.

Given an initial answer embeddingY0∈ℝTy×dY\_\{0\}\\in\\mathbb\{R\}^\{T\_\{y\}\\times d\}, the model maintains a carry stateZh∈ℝTz×dZ\_\{h\}\\in\\mathbb\{R\}^\{T\_\{z\}\\times d\}across outer cyclesh∈\{1,…,H\}h\\in\\\{1,\\dots,H\\\}\. At each outer cycle, an inner loop executesLLrefinement steps\. A shared two\-layer transformer blockfθf\_\{\\theta\}updates the state:

Zh\(l\)=fθ​\(\[X;Yh−1;Zh\(l−1\)\]\)Z\_\{h\}^\{\(l\)\}=f\_\{\\theta\}\\\!\\left\(\[X\\,;\\,Y\_\{h\-1\}\\,;\\,Z\_\{h\}^\{\(l\-1\)\}\]\\right\)\(1\)
where\[⋅;⋅\]\[\\cdot;\\cdot\]denotes sequence concatenation\. AfterLLinner steps, the outer loop advances via:

Yh,Zh\+1\(0\)=gϕ​\(Zh\(L\)\)Y\_\{h\},\\;Z\_\{h\+1\}^\{\(0\)\}=g\_\{\\phi\}\\\!\\left\(Z\_\{h\}^\{\(L\)\}\\right\)\(2\)A one\-step gradient approximation during training keeps memory𝒪​\(1\)\\mathcal\{O\}\(1\)with respect to recursive depth\. The total number of refinement steps isH×nsupH\\times n\_\{\\text\{sup\}\}, wherensupn\_\{\\text\{sup\}\}is the number of outer\-loop supervision steps\. At the default ARC\-2024 configuration \(H=3H=3,nsup=16n\_\{\\text\{sup\}\}=16\), TRM executes 48 refinement steps, incurring∼\\sim3,000 GFLOPs per puzzle over a 900\-token sequence \(𝒪​\(n2\)\\mathcal\{O\}\(n^\{2\}\)attention\)\.

We study two architectural variants:TRM\-Attention, which uses multi\-head self\-attention over the full token sequence, andTRM\-MLP\-Mixing, which replaces attention with token\-mixing MLP layers\. These variants differ in how relational structure is encoded across refinement steps, which has consequences for compression sensitivity \(§[4\.4](https://arxiv.org/html/2606.26488#S4.SS4)\)\.

### 3\.2Tasks and Datasets

#### ARC\-2024\.

Abstract visual reasoning requiring detection of transformation rules across grid inputs\(Chollet,[2019](https://arxiv.org/html/2606.26488#bib.bib25)\)\. TRM uses TRM\-Attention with a 50,911\-entry puzzle embedding table \(99\.4 MB in FP32\) that provides puzzle\-specific context, Test\-Time Augmentation \(1,000 augmentations per puzzle\), puzzle\-identifier adaptation, and multi\-pass majority voting\. Parameters: 6\.83M\.

#### Maze\-Hard\.

Path\-finding requiring exact end\-to\-end traversal over topological maps\. Uses TRM\-Attention without an embedding table or TTA\. Parameters: 6\.82M\.

#### Sudoku\-Extreme\.

Constraint satisfaction over9×99\\times 9grids with hard row, column, and block exclusions\. Uses TRM\-MLP\-Mixing without an embedding table\. Parameters: 5\.03M\.

We report two metrics throughout\.Cell accuracyis the fraction of correct grid cells predictions\.Puzzle exact matchis the fraction of puzzles solved with zero errors\. These metrics can diverge substantially: a model may predict nearly correct tokens \(high cell accuracy\) while satisfying zero global constraints \(zero puzzle exact\), as observed under several compression interventions \(§[4\.6](https://arxiv.org/html/2606.26488#S4.SS6)\)\.

### 3\.3Deployment Targets

The compression goal is on\-device inference with no cloud dependency under hard SRAM constraints\. We target two hardware classes:Mobile SoC \(e\.g\. Cortex\-A55\)with 8 MB SRAM and a 4 MBMCU \(e\.g\. Cortex\-M55\)as a stretch goal \. The FP32 baseline \(∼\\sim126 MB total: 26\.7 MB backbone \+ 99\.4 MB embeddings\) exceeds both targets by an order of magnitude\. On\-device latency, memory, and model size in §[4\.7](https://arxiv.org/html/2606.26488#S4.SS7)are measured directly on three Snapdragon edge platforms via Qualcomm AI Hub \(App\.[G](https://arxiv.org/html/2606.26488#A7)\)\.

### 3\.4Compression Methods

#### Post\-training quantization \(PTQ\)\.

We apply PTQ directly to the pretrained FP32 checkpoint, without retraining\.FP16halves memory via standard half\-precision casting\.INT8uses per\-channel scale calibration viabitsandbytes\(Dettmerset al\.,[2022](https://arxiv.org/html/2606.26488#bib.bib15)\)\- the standard format for hardware with integer acceleration units\.INT4 naïveapplies per\-tensor PTQ without calibration\.INT4 calibratedapplies per\-channel asymmetric PTQ \(min/max range per output channel\), again with no fine\-tuning; it recovers most of the accuracy that naïve INT4 destroys \(§[4\.5](https://arxiv.org/html/2606.26488#S4.SS5)\)\.

Structured pruning and knowledge distillation were also evaluated\. Both collapse puzzle\-level accuracy across all tasks while partially preserving cell accuracy \(§[4\.6](https://arxiv.org/html/2606.26488#S4.SS6)\), consistent with the model’s compactness \(5–7M parameters\): at this scale, parameter redundancy is insufficient to absorb pruning, and a single\-pass student cannot capture the recursive reasoning\.

#### Embedding compression\.

For ARC\-2024, we separately evaluate three strategies for the 99\.4 MB puzzle embedding table: INT8 quantization of the full table, SVD rank\-16 factorization, and single\-puzzle flash loading \(streaming only the 2 KB row for the current puzzle at inference time\)\.

### 3\.5Cycle Reduction

HH\(outer cycles\) andnsupn\_\{\\text\{sup\}\}\(the number of outer\-loop supervision steps\) together determine total recursion depth:H=3H=3,nsup=16n\_\{\\text\{sup\}\}=16means 48 sequential refinement steps on ARC\-2024\. We sweep\(H,nsup\)∈\{\(1,1​–​16\),\(3,1​–​16\)\}\(H,n\_\{\\text\{sup\}\}\)\\in\\\{\(1,1\\text\{\-\-\}16\),\\,\(3,1\\text\{\-\-\}16\)\\\}for ARC\-2024, and\(H,nsup\)∈\{\(1​–​4,1​–​10\)\}\(H,n\_\{\\text\{sup\}\}\)\\in\\\{\(1\\text\{\-\-\}4,1\\text\{\-\-\}10\)\\\}for Maze\-Hard and Sudoku\-Extreme\. FLOPs scale linearly with total steps: atH=1H=1,nsup=8n\_\{\\text\{sup\}\}=8, the per\-puzzle cost falls from∼\\sim3,000 GFLOPs to∼\\sim500 GFLOPs on ARC\-2024\.

### 3\.6Carry\-State Diagnostics

We use the recursive carry state to build a label\-free signal for reasoning collapse under compression\. LetZh∈ℝTz×dZ\_\{h\}\\in\\mathbb\{R\}^\{T\_\{z\}\\times d\}be the carry state after recursive callhh, withzh,iz\_\{h,i\}itsii\-th token vector\.

#### Consecutive similarity \(baseline\)\.

The cosine similarity between successive carry states,

sh=1Tz​∑i=1Tzzh,i⋅zh−1,i‖zh,i‖2​‖zh−1,i‖2,s\_\{h\}=\\frac\{1\}\{T\_\{z\}\}\\sum\_\{i=1\}^\{T\_\{z\}\}\\frac\{z\_\{h,i\}\\cdot z\_\{h\-1,i\}\}\{\\\|z\_\{h,i\}\\\|\_\{2\}\\,\\\|z\_\{h\-1,i\}\\\|\_\{2\}\},\(3\)measures per\-step refinement \(lowshs\_\{h\}\) versus convergence to a near\-fixed point \(highshs\_\{h\}\), followingJianget al\.\([2025](https://arxiv.org/html/2606.26488#bib.bib22)\)’s finding that hidden\-state cosine similarity tracks representational saturation\. We find this signal is nearly invariant to precision \(§[4\.3](https://arxiv.org/html/2606.26488#S4.SS3)\) and thus a poor failure detector\.

#### Carry\-trajectory fidelity \(primary\)\.

The signal we recommend compares the compressed trajectory to the full\-precision one\. LetZHqZ\_\{H\}^\{q\}andZHfp32Z\_\{H\}^\{\\text\{fp32\}\}be the final carry states of the quantized and FP32 models on the same input\. Fidelity is

ϕ=1Tz​∑i=1TzzH,iq⋅zH,ifp32‖zH,iq‖2​‖zH,ifp32‖2\.\\phi=\\frac\{1\}\{T\_\{z\}\}\\sum\_\{i=1\}^\{T\_\{z\}\}\\frac\{z\_\{H,i\}^\{q\}\\cdot z\_\{H,i\}^\{\\text\{fp32\}\}\}\{\\\|z\_\{H,i\}^\{q\}\\\|\_\{2\}\\,\\\|z\_\{H,i\}^\{\\text\{fp32\}\}\\\|\_\{2\}\}\.\(4\)ϕ\\phiquantifies how far quantization pushes the reasoning trajectory off the full\-precision path\. It requires only the FP32 ref \(available before deployment\) and no task labels, and—unlikeshs\_\{h\}—is graded and monotonic with the eventual accuracy loss \(§[4\.3](https://arxiv.org/html/2606.26488#S4.SS3)\)\.

### 3\.7Quantization\-Aware Training and Recovery

As one probe of whether a token\-level objective can repair compression\-induced reasoning loss, we run QAT from the naïve INT4 checkpoint \(100 steps,lr=10−5\\text\{lr\}\{=\}10^\{\-5\}, cross\-entropy over token predictions; §[4\.6](https://arxiv.org/html/2606.26488#S4.SS6)\)\. As the alternative recovery path we apply per\-channel calibrated INT4 \(a post\-training quantizer with no fine\-tuning\) and report both its accuracy and its carry\-trajectory fidelity \(§[4\.5](https://arxiv.org/html/2606.26488#S4.SS5)\)\.

## 4Results

### 4\.1The Embedding Table, Not the Backbone, Is the Memory Bottleneck

The ARC\-2024 puzzle embedding table occupies 99\.4 MB of the∼\\sim126 MB FP32 footprint \(79% of total model memory\) while the transformer backbone requires only 26\.7 MB \(Table[1](https://arxiv.org/html/2606.26488#S4.T1)\)\. Three strategies were evaluated for embedding compression\.

Table 1:Embedding compression strategies on ARC\-2024\. Only flash loading fits within the 4–8 MB deployment targets while preserving accuracy\.SVD rank\-16 factorization reduces the table to 3\.2 MB but degrades the embedding space: cosine similarity between original and factorized embeddings falls to 0\.94, introducing lossy approximation\. INT8 quantization of the full table preserves accuracy but reaches only 24\.9 MB, which is insufficient for either target\. Single\-puzzle flash loading resolves the bottleneck: at inference time, the 2 KB row for the current puzzle is streamed from flash storage, and accuracy is unchanged\. The embedding problem is thereby decoupled from the backbone compression problem\. Figure[1](https://arxiv.org/html/2606.26488#S0.F1)visualizes the resulting memory footprint across configurations\.

### 4\.2INT8 with Reduced Recursion Matches Full\-Depth FP32

Table[3](https://arxiv.org/html/2606.26488#A1.T3)in the Appendix shows full\-system accuracy across all precision levels\. INT8 is lossless across both architectural variants and all three tasks: per\-channel calibration holds quantization error within what the recursive refinement process tolerates\. The key efficiency gain therefore comes from cycle reduction, not precision loss; INT8 atH=1H\{=\}1,nsup=8n\_\{\\text\{sup\}\}\{=\}8achieves 35\.25% puzzle exact on ARC\-2024 \(with test\-time augmentation\),111ARC accuracies use the standard TRM augment\-and\-vote protocol\. Single forward\-pass accuracy—the setting whose on\-device latency we measure \(Table[11](https://arxiv.org/html/2606.26488#A7.T11)\)—is lower \(e\.g\. 26\.0% for this INT8 configuration\); augmentation recovers the gap at a proportional latency cost\.within 0\.75 pp of the 36\.00% baseline at6×6\\timesfewer FLOPs \(Figure[2](https://arxiv.org/html/2606.26488#S4.F2)\)\.

![Refer to caption](https://arxiv.org/html/2606.26488v1/x2.png)Figure 2:ARC\-2024 efficiency frontier\. INT8 reaches full\-depth FP32 accuracy at one recursive cycle \(⋆\\star, 500 GFLOPs\)—a6×6\\timesFLOPs reduction\.Across the full depth sweep \(Appendix[B](https://arxiv.org/html/2606.26488#A2)\),H=1H\{=\}1brackets the 36\.00% baseline from 34\.25% \(nsup=4n\_\{\\text\{sup\}\}\{=\}4\) to 36\.25% \(nsup=16n\_\{\\text\{sup\}\}\{=\}16\)\. The reduction holds on Maze\-Hard \(INT8 atH=1H\{=\}1,nsup=8n\_\{\\text\{sup\}\}\{=\}8stays within 0\.10 pp at3\.75×3\.75\\timesfewer FLOPs\)\. Sudoku\-Extreme is the exception: its accuracy keeps climbing with depth \(best atH=4H\{=\}4\) and INT8 stays lossless there, so its full recursion cannot be cheaply cut \(Appendices[C](https://arxiv.org/html/2606.26488#A3),[D](https://arxiv.org/html/2606.26488#A4)\)\.

#### Static vs\. Dynamic Quantization\.

While dynamic quantization consistently preserves full\-precision accuracy across tasks \(recovering almost all performance loss\), naive static quantization exhibits extreme task\-specific fragility\. Out of the box, naive static INT8 is lossless only for Maze\-Hard; it incurs a significant 10\.5 pp puzzle\-exact penalty on ARC\-2024, and leads to a catastrophic collapse on Sudoku\-Extreme \(dropping to 5\.30% exact match\)\. Recovering Sudoku’s performance under static quantization requires selective operator exclusion \(specifically leaving activation and carry state updates in FP32 and only quantizingMatMul/Gemmweights\)\. Thus, while dynamic quantization represents the most robust out\-of\-the\-box post\-training quantization path for recurrent transformer models, it is incompatible with NPU hardware accelerators on edge devices that require static, pre\-allocated tensor boundaries\.

### 4\.3Carry\-Trajectory Fidelity Grades Reasoning Collapse

We want a label\-free signal that, before any task evaluation, predicts whether a compression preserves or destroys reasoning\. The obvious candidate \(cosine similarity between consecutive carry states\) does not work: it is task\-specific but nearly invariant to precision, changing by≤0\.04\\leq 0\.04from FP32 to INT4 on every task \(App\.[E](https://arxiv.org/html/2606.26488#A5)\), so it cannot separate harmless from harmful compression\.

The signal that does work is*carry\-trajectory fidelity*: the cosine similarity between the quantized model’s final carry state and the full\-precision model’s final carry state on the same input \(§[3\.6](https://arxiv.org/html/2606.26488#S3.SS6)\)\. It measures how far quantization pushes the reasoning trajectory off the full\-precision path, requires only the FP32 reference \(available before deployment\), and uses no task labels\. Fidelity is graded and monotonic with damage \(Figure[3](https://arxiv.org/html/2606.26488#S4.F3); full data Appendix[E](https://arxiv.org/html/2606.26488#A5)\): it stays≥0\.99\\geq 0\.99wherever INT4 is harmless \(Maze, and INT8 on every task\) and falls in proportion to the accuracy loss—moderately on ARC \(0\.690\.69,−10\.5\-10\.5pp puzzle exact\) and catastrophically on Sudoku \(0\.350\.35,−63\.8\-63\.8pp\)\. It also tracks*recovery*under calibration \(§[4\.5](https://arxiv.org/html/2606.26488#S4.SS5)\)\.

![Refer to caption](https://arxiv.org/html/2606.26488v1/x3.png)Figure 3:Carry\-trajectory fidelity is a label\-free pass/fail signal: a single threshold \(∼\\sim0\.8, dashed\) separates compression that preserves reasoning \(green\) from compression that breaks it \(red\); naïve INT4 on MLP\-mixing \(Sudoku\) and ARC are flagged, and per\-channel calibration moves Sudoku back across the line\.
### 4\.4Compression Fragility Is Architectural, Not Task\-Intrinsic

Table[3](https://arxiv.org/html/2606.26488#A1.T3)shows task\-level accuracy in compression: the TRM\-Attention tasks \(ARC, Maze\) tolerate naïve INT4, while TRM\-MLP\-Mixing \(Sudoku\) collapses\. This looks like a task\-precision effect; but a same\-task ablation shows it is not \(Figure[4](https://arxiv.org/html/2606.26488#S4.F4)\)\.

![Refer to caption](https://arxiv.org/html/2606.26488v1/x4.png)Figure 4:Architecture ablation on Sudoku\-Extreme: full precision vs\. naïve INT4 puzzle\-exact for three recursive architectures on the*same*task\. Only TRM\-MLP\-Mixing collapses \(−64\-64pp\); TRM\-Attention \(at the same FP32 accuracy\) and HRM \(a different model family\) survive; the fragility is the MLP\-mixing token mixer, not the constraint\-satisfaction task\.Under naïve INT4, Maze\-Hard loses 0\.40 pp and ARC\-2024 loses 10\.5 pp puzzle exact while keeping 84\.8% cell accuracy, but Sudoku\-Extreme collapses by 63\.8 pp\. Since Sudoku was tested only with MLP\-mixing, we re\-run it with attention: TRM\-Attention, at the*same*full\-precision accuracy \(73\.1% vs\. 73\.8%\), retains 69\.5% under naïve INT4, and HRM\(Wanget al\.,[2025](https://arxiv.org/html/2606.26488#bib.bib8)\)is also robust, while only MLP\-mixing collapses \(Table[10](https://arxiv.org/html/2606.26488#A6.T10)\)\. The accuracy match rules out a headroom confound: the fragility is the token mixer, not the task\.

### 4\.5Calibrated INT4 Recovers Constraint Reasoning

The catastrophic Sudoku\-INT4 result is specific to*naïve per\-tensor*quantization\. Replacing it with per\-channel calibrated INT4 \(still 4\-bit weights, still post\-training, no fine\-tuning\) recovers almost all of the lost accuracy \(full table in App\.[E](https://arxiv.org/html/2606.26488#A5)\)\. Under an identical evaluation harness, naïve INT4 drops Sudoku to 10\.2% puzzle exact while calibrated INT4 reaches 71\.9%, within 1\.9 pp of the 73\.8% FP32 baseline measured the same way\. The carry\-trajectory fidelity diagnostic tracks this recovery with no task evaluation: fidelity rises from0\.350\.35\(naïve, broken\) to0\.890\.89\(calibrated, recovered\)\.

This reframes the deployment floor: for constraint\-satisfaction reasoning it is not INT8 but*calibrated*INT4, provided the quantizer is per\-channel\. Naïve INT4 remains unusable, and the diagnostic distinguishes the two before deployment\.

### 4\.6Token\-Level Recovery Does Not Restore Reasoning

A pattern recurs across our compression interventions: cell accuracy and puzzle\-exact accuracy decouple\. A model can predict most individual cells correctly while satisfying zero global constraints—the signature of a*compositional\-correctness*breakdown that leaves local token prediction intact\. We observe it under four distinct interventions:

Onknowledge distillation, a single\-cycle student \(13–15% of teacher parameters\) reaches 87\.3% \(Maze\) / 54\.8% \(Sudoku\) cell accuracy but 0\.0% puzzle exact \(Appendix[K](https://arxiv.org/html/2606.26488#A11)\)\. Withstructured pruningat 25% sparsity, cell accuracy partially survives \(86\.4% Maze, 50\.0% Sudoku\) while puzzle exact is 0\.0% on every task\. Withlinear\-attention approximationon Maze, replacing softmax attention holds cell accuracy at 87\.5% but collapses puzzle exact to 0\.0% \(Appendix[J](https://arxiv.org/html/2606.26488#A10)\)\. Also,QATvia naïve INT4 \(100 steps,lr=10−5\\text\{lr\}\{=\}10^\{\-5\}, cross\-entropy\) does not converge \(0\.0% cell*and*0\.0% puzzle exact\) so a token\-level objective gives no useful gradient toward global constraint satisfaction\.

The common cause is the objective: cross\-entropy and token\-distillation losses reward each cell independently, so a model that emits plausible cells without satisfying the puzzle is scored identically to one that solves it\. This is not specific to TRM—it applies to any structured\-prediction setting where correctness is compositional\. The practical implication is that the reliable recovery path for compression\-induced reasoning loss is*not*a better token\-level training objective but a better quantizer: per\-channel calibration recovers Sudoku with no retraining at all \(§[4\.5](https://arxiv.org/html/2606.26488#S4.SS5)\)\.

### 4\.7Optimal Deployment Configuration

Table[2](https://arxiv.org/html/2606.26488#A1.T2)in the Appendix summarizes the deployment configurations\. Rather than relying solely on analytical projections, we measure physical on\-device step latencies, puzzle latencies, and peak memory usage directly on physical hardware \(Samsung Galaxy S24, Samsung Galaxy S22 5G, and Dragonwing RB3 Gen 2\) via Qualcomm AI Hub \(Table[11](https://arxiv.org/html/2606.26488#A7.T11)\)\. INT8 with single\-puzzle flash loading and reduced recursion \(H=1H\{=\}1,nsup=8n\_\{\\text\{sup\}\}\{=\}8\) occupies 6\.51 MB SRAM \(which is within the 8 MB mobile SoC target\) and runs at∼\\sim500 GFLOPs per ARC\-2024 puzzle\.

Maze\-Hard is the cleanest deployable case: with no test\-time augmentation and no embedding table, its per\-pass latency is its per\-puzzle latency, and INT4 is nearly lossless \.

Per\-channel calibrated INT4 fits the 4 MB MCU target \(3\.26 MB\) and recovers near\-FP32 accuracy with no retraining—demonstrated directly on Sudoku\-Extreme \(§[4\.5](https://arxiv.org/html/2606.26488#S4.SS5)\) and consistent with its high carry\-trajectory fidelity on ARC \(0\.94\)\. The practical compression floor is therefore calibrated INT4, not INT8; naïve INT4 remains unusable, and the fidelity diagnostic separates the two before deployment\. The6×6\\timesFLOPs reduction also lowers energy proportionally \(∼\\sim3,000 to∼\\sim500 mJ/puzzle; Appendix[H](https://arxiv.org/html/2606.26488#A8)\), and there is no cheaper shortcut through the input: truncating the context destroys accuracy on every task—halving it collapses Maze and Sudoku to 0% puzzle exact \(Appendix[I](https://arxiv.org/html/2606.26488#A9)\)\.

## 5Discussion

#### Why the architecture split exists\.

The split has a plausible mechanism: attention encodes sparse relational updates that tolerate coarse noise, whereas MLP\-mixing encodes denser token\-mixing that needs per\-channel scales to survive\. Architecture choice in recursive models is therefore a deployment decision, best made at design time\.

#### Fidelity should generalize beyond TRM\.

Because carry\-trajectory fidelity needs only a full\-precision reference and the model’s own hidden states \(no labels, no task run\) it should transfer to any iterative\-latent system with accessible state: Universal Transformers\(Dehghaniet al\.,[2019](https://arxiv.org/html/2606.26488#bib.bib9)\), deep equilibrium models, and state\-space models\.

## Limitations

Our study centers on the TRM family across three tasks, with a second architecture \(HRM\) evaluated only on Sudoku\-Extreme; broader generalization across architectures and tasks is untested\. The calibrated\-INT4 recovery is measured by full\-system accuracy on Sudoku; for ARC we report only carry\-trajectory fidelity \(0\.94\), not re\-measured accuracy\. The fidelity diagnostic requires a full\-precision reference, so it supports a pre\-deployment decision rather than monitoring a live model\. ARC’s accuracy relies on test\-time augmentation, so its reported single\-pass latency understates the full per\-puzzle deployment cost; Maze and Sudoku need no augmentation and are the configurations whose latency is directly deployable\. Finally, we evaluate no autoregressive baseline of comparable size, so we cannot fully separate the compression dynamics from effects of model compactness\.

## References

- H\. A\. Abushahla, D\. Varam, A\. J\. N\. Panopio, and M\. I\. AlHajri \(2025\)Neural network quantization for microcontrollers: a comprehensive survey of methods, platforms, and applications\.arXiv preprint arXiv:2508\.15008\.Cited by:[§1](https://arxiv.org/html/2606.26488#S1.p3.1),[§2](https://arxiv.org/html/2606.26488#S2.SS0.SSS0.Px4.p1.1)\.
- M\. AskariHemmat, R\. A\. Hemmat, A\. Hoffman, I\. Lazarevich, E\. Saboori, O\. Mastropietro, S\. Sah, Y\. Savaria, and J\. David \(2022\)QReg: on regularization effects of quantization\.arXiv preprint arXiv:2206\.12372\.Cited by:[§2](https://arxiv.org/html/2606.26488#S2.SS0.SSS0.Px2.p1.1)\.
- M\. AskariHemmat, A\. Jeddi, R\. A\. Hemmat, I\. Lazarevich, A\. Hoffman, S\. Sah, E\. Saboori, Y\. Savaria, and J\. David \(2024\)QGen: on the ability to generalize in quantization aware training\.arXiv preprint arXiv:2404\.11769\.Cited by:[§2](https://arxiv.org/html/2606.26488#S2.SS0.SSS0.Px2.p1.1)\.
- A\. Bansal, A\. Schwarzschild, E\. Borgnia, Z\. Emam, F\. Huang, M\. Goldblum, and T\. Goldstein \(2022\)End\-to\-end algorithm synthesis with recurrent networks: logical extrapolation without overthinking\.InAdvances in Neural Information Processing Systems,Vol\.35,pp\. 20232–20242\.Cited by:[§2](https://arxiv.org/html/2606.26488#S2.SS0.SSS0.Px1.p1.1)\.
- O\. Bohdal, K\. Theodosiadis, A\. Mpatziakas, D\. Filippidis, I\. Spyrou, C\. Zonios, A\. Drosou, D\. Ioannidis, K\. Lee, J\. Moon,et al\.\(2025\)On\-device system of compositional multi\-tasking in large language models\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: Industry Track,pp\. 416–424\.Cited by:[§2](https://arxiv.org/html/2606.26488#S2.SS0.SSS0.Px4.p1.1)\.
- F\. Chollet \(2019\)On the measure of intelligence\.arXiv preprint arXiv:1911\.01547\.Cited by:[§1](https://arxiv.org/html/2606.26488#S1.p1.2),[§3\.2](https://arxiv.org/html/2606.26488#S3.SS2.SSS0.Px1.p1.1)\.
- M\. Dehghani, S\. Gouws, O\. Vinyals, J\. Uszkoreit, and L\. Kaiser \(2019\)Universal transformers\.InInternational Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2606.26488#S1.p1.2),[§2](https://arxiv.org/html/2606.26488#S2.SS0.SSS0.Px1.p1.1),[§5](https://arxiv.org/html/2606.26488#S5.SS0.SSS0.Px2.p1.1)\.
- T\. Dettmers, M\. Lewis, Y\. Belkada, and L\. Zettlemoyer \(2022\)LLM\.int8\(\): 8\-bit matrix multiplication for transformers at scale\.Advances in neural information processing systems35,pp\. 30318–30332\.Cited by:[Appendix L](https://arxiv.org/html/2606.26488#A12.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2606.26488#S1.p2.1),[§2](https://arxiv.org/html/2606.26488#S2.SS0.SSS0.Px4.p1.1),[§3\.4](https://arxiv.org/html/2606.26488#S3.SS4.SSS0.Px1.p1.1)\.
- H\. Du, Y\. Dong, and X\. Ning \(2026\)Latent thinking optimization: your latent reasoning language model secretly encodes reward signals in its latent thoughts\.InInternational Conference on Learning Representations,Cited by:[§2](https://arxiv.org/html/2606.26488#S2.SS0.SSS0.Px1.p1.1)\.
- J\. Geiping, S\. McLeish, N\. Jain, J\. Kirchenbauer, S\. Singh, B\. Bartoldson, B\. Kailkhura, A\. Bhatele, and T\. Goldstein \(2025\)Scaling up test\-time compute with latent reasoning: a recurrent depth approach\.Advances in Neural Information Processing Systems38,pp\. 41340–41391\.Cited by:[§1](https://arxiv.org/html/2606.26488#S1.p1.2),[§2](https://arxiv.org/html/2606.26488#S2.SS0.SSS0.Px1.p1.1)\.
- S\. Han, H\. Mao, and W\. J\. Dally \(2016\)Deep compression: compressing deep neural networks with pruning, trained quantization and Huffman coding\.InInternational Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2606.26488#S1.p2.1),[§2](https://arxiv.org/html/2606.26488#S2.SS0.SSS0.Px4.p1.1)\.
- S\. Hao, S\. Sukhbaatar, D\. Su, X\. Li, Z\. Hu, J\. E\. Weston, and Y\. Tian \(2025\)Training large language models to reason in a continuous latent space\.InSecond Conference on Language Modeling,Cited by:[§1](https://arxiv.org/html/2606.26488#S1.p1.2),[§2](https://arxiv.org/html/2606.26488#S2.SS0.SSS0.Px1.p1.1)\.
- S\. Javed, H\. Le, and M\. Salzmann \(2025\)QT\-DoG: quantization\-aware training for domain generalization\.InProceedings of the 42nd International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.267,pp\. 26981–27004\.Cited by:[§2](https://arxiv.org/html/2606.26488#S2.SS0.SSS0.Px2.p1.1)\.
- J\. Jiang, J\. Zhou, and Z\. Zhu \(2025\)Tracing representation progression: analyzing and enhancing layer\-wise similarity\.InInternational Conference on Learning Representations,Vol\.2025,pp\. 1118–1143\.Cited by:[§2](https://arxiv.org/html/2606.26488#S2.SS0.SSS0.Px1.p1.1),[§3\.6](https://arxiv.org/html/2606.26488#S3.SS6.SSS0.Px1.p1.2)\.
- A\. Jolicoeur\-Martineau \(2025\)Less is more: recursive reasoning with tiny networks\.arXiv preprint arXiv:2510\.04871\.Cited by:[§1](https://arxiv.org/html/2606.26488#S1.p1.2),[§3\.1](https://arxiv.org/html/2606.26488#S3.SS1.p1.2)\.
- S\. Kim, A\. Gholami, Z\. Yao, M\. W\. Mahoney, and K\. Keutzer \(2021\)I\-BERT: integer\-only BERT quantization\.InInternational conference on machine learning,pp\. 5506–5518\.Cited by:[§2](https://arxiv.org/html/2606.26488#S2.SS0.SSS0.Px4.p1.1)\.
- H\. Kohli, S\. Parthasarathy, H\. Sun, and Y\. Yao \(2026\)Loop, think, & generalize: implicit reasoning in recurrent\-depth transformers\.arXiv preprint arXiv:2604\.07822\.Cited by:[§1](https://arxiv.org/html/2606.26488#S1.p1.2),[§2](https://arxiv.org/html/2606.26488#S2.SS0.SSS0.Px1.p1.1)\.
- Z\. Li, Y\. Su, S\. Wang, R\. Yang, C\. Xie, A\. Liu, M\. Li, J\. Cao, Y\. Xie, N\. Wong,et al\.\(2025\)Quantization meets reasoning: exploring and mitigating degradation of low\-bit LLMs in mathematical reasoning\.arXiv preprint arXiv:2505\.11574\.Cited by:[§1](https://arxiv.org/html/2606.26488#S1.p2.1),[§2](https://arxiv.org/html/2606.26488#S2.SS0.SSS0.Px3.p1.1),[§2](https://arxiv.org/html/2606.26488#S2.SS0.SSS0.Px4.p1.1)\.
- J\. Lin, W\. Chen, Y\. Lin, C\. Gan, S\. Han,et al\.\(2020\)MCUNet: tiny deep learning on IoT devices\.Advances in neural information processing systems33,pp\. 11711–11722\.Cited by:[§1](https://arxiv.org/html/2606.26488#S1.p3.1),[§2](https://arxiv.org/html/2606.26488#S2.SS0.SSS0.Px4.p1.1)\.
- P\. Liu, Z\. Liu, Z\. Gao, D\. Gao, W\. X\. Zhao, Y\. Li, B\. Ding, and J\. Wen \(2024\)Do emergent abilities exist in quantized large language models: an empirical study\.InProceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation \(LREC\-COLING 2024\),pp\. 5174–5190\.Cited by:[§1](https://arxiv.org/html/2606.26488#S1.p2.1),[§2](https://arxiv.org/html/2606.26488#S2.SS0.SSS0.Px3.p1.1)\.
- R\. Liu, Y\. Sun, M\. Zhang, H\. Bai, X\. Yu, T\. Yu, C\. Yuan, and L\. Hou \(2025\)Quantization hurts reasoning? an empirical study on quantized reasoning models\.InProceedings of the Conference on Language Modeling,Note:arXiv:2504\.04823Cited by:[§2](https://arxiv.org/html/2606.26488#S2.SS0.SSS0.Px3.p1.1)\.
- W\. Lu, Y\. Yang, K\. Lee, Y\. Li, and E\. Liu \(2025\)Latent chain\-of\-thought? decoding the depth\-recurrent transformer\.arXiv preprint arXiv:2507\.02199\.Note:COLM 2025 Workshop on LLM Explainability for Reasoning and PlanningCited by:[§2](https://arxiv.org/html/2606.26488#S2.SS0.SSS0.Px1.p1.1)\.
- K\. Lv, M\. Zhang, X\. Xia, J\. Ni, S\. Yan, X\. Yu, L\. Hou, C\. Yuan, and H\. Bai \(2026\)What makes low\-bit quantization\-aware training work for reasoning llms? a systematic study\.arXiv preprint arXiv:2601\.14888\.Cited by:[§2](https://arxiv.org/html/2606.26488#S2.SS0.SSS0.Px3.p1.1)\.
- M\. Nagel, M\. Fournarakis, R\. A\. Amjad, Y\. Bondarenko, M\. Van Baalen, and T\. Blankevoort \(2021\)A white paper on neural network quantization\.arXiv preprint arXiv:2106\.08295\.Cited by:[§1](https://arxiv.org/html/2606.26488#S1.p2.1),[§2](https://arxiv.org/html/2606.26488#S2.SS0.SSS0.Px3.p1.1)\.
- G\. Tallec, E\. Yvinec, A\. Dapogny, and K\. Bailly \(2023\)Fighting over\-fitting with quantization for learning deep neural networks on noisy labels\.In2023 IEEE International Conference on Image Processing \(ICIP\),pp\. 575–579\.Cited by:[§2](https://arxiv.org/html/2606.26488#S2.SS0.SSS0.Px2.p1.1)\.
- G\. Wang, J\. Li, Y\. Sun, X\. Chen, C\. Liu, Y\. Wu, M\. Lu, S\. Song, and Y\. A\. Yadkori \(2025\)Hierarchical reasoning model\.arXiv preprint arXiv:2506\.21734\.Cited by:[Appendix F](https://arxiv.org/html/2606.26488#A6.p1.7),[§1](https://arxiv.org/html/2606.26488#S1.p1.2),[§2](https://arxiv.org/html/2606.26488#S2.SS0.SSS0.Px1.p1.1),[§4\.4](https://arxiv.org/html/2606.26488#S4.SS4.p2.1)\.
- M\. S\. Zhang, R\. A\. Ruia, A\. Kewalram, S\. Dharmapuram, U\. Sharma, and K\. Zhu \(2025\)When less is more: 8\-bit quantization improves continual learning in large language models\.arXiv preprint arXiv:2512\.18934\.Cited by:[§2](https://arxiv.org/html/2606.26488#S2.SS0.SSS0.Px2.p1.1)\.
- Z\. Zhou, S\. Kurz, and Z\. Zhao \(2025\)Revisiting pruning vs quantization for small language models\.InFindings of the Association for Computational Linguistics: EMNLP 2025,pp\. 12055–12070\.Cited by:[§2](https://arxiv.org/html/2606.26488#S2.SS0.SSS0.Px4.p1.1)\.

## Appendix AFull Precision × Task Results

Table 2:Deployment configurations per ARC\-2024 puzzle\. Bold marks viable targets\. SRAM shows backbone\-only footprint \(ARC embedding handled via flash loading\)\. On\-device latencies are measured directly on physical hardware \(Table[11](https://arxiv.org/html/2606.26488#A7.T11)\)\. Per\-channel calibrated INT4 recovers near\-FP32 accuracy without retraining \(§[4\.5](https://arxiv.org/html/2606.26488#S4.SS5)\), making the 3\.26 MB configuration a viable 4 MB MCU target\.Table[3](https://arxiv.org/html/2606.26488#A1.T3)reports full\-system accuracy across all precision levels and tasks at default recursive depth\.

Table 3:Full\-system precision × task accuracy results at default recursive depth\. ARC\-2024:H=3H\{=\}3,nsup=16n\_\{\\text\{sup\}\}\{=\}16with TTA\. Maze\-Hard and Sudoku\-Extreme:H=3H\{=\}3,nsup=10n\_\{\\text\{sup\}\}\{=\}10\. SRAM shows backbone\-only footprint \(ARC embedding handled via flash loading\)\.†FP32 baseline loaded with BF16 training weights; SRAM at FP32 precision\.
## Appendix BDepth × Quantization Sweep \(ARC\-2024\)

Table[4](https://arxiv.org/html/2606.26488#A2.T4)reports key ARC\-2024 depth configurations \(full\-system, with TTA\): INT8 matches FP32 at every depth tested\.

Table 4:ARC\-2024 depth–precision sweep \(full\-system, TTA\)\. INT8 atH=1H\{=\}1,nsup=8n\_\{\\text\{sup\}\}\{=\}8reaches 35\.25% at 500 GFLOPs—within 0\.75 pp of the 36\.00% FP32 peak at6×6\\timesfewer FLOPs\.
## Appendix CDepth × Quantization Sweep \(Maze\-Hard\)

Table[5](https://arxiv.org/html/2606.26488#A3.T5)shows key configurations from the Maze\-Hard depth sweep, confirming rapid saturation\.

Table 5:Maze\-Hard depth–precision sweep\.†Default\. INT8 atH=1H\{=\}1,n=8n\{=\}8matches default FP32 at3\.75×3\.75\\timesfewer FLOPs\.
## Appendix DDepth × Quantization Sweep \(Sudoku\-Extreme\)

Table[6](https://arxiv.org/html/2606.26488#A4.T6)shows key configurations from the Sudoku\-Extreme depth sweep, confirming that depth never saturates\.

Table 6:Sudoku\-Extreme depth–precision sweep\.†Default\. INT8 atH=4H\{=\}4,n=10n\{=\}10matches FP32 \(70\.20% vs 70\.30%\); reducing depth toH=1H\{=\}1causes major accuracy loss\. Full depth is genuinely needed\.
## Appendix ECarry\-State Diagnostic: Full Data

We measure two carry\-state quantities, computed by hooking the shared reasoning module across all recursive calls within one inference pass\. Both are computed on real test puzzles \(MazeN=64N\{=\}64, ARCN=32N\{=\}32, SudokuN=256N\{=\}256\); the ARC checkpoint and Maze/Sudoku checkpoints are the public TRM releases, and all numbers below are reproduced from those checkpoints \(§[L](https://arxiv.org/html/2606.26488#A12)\)\.

#### Consecutive carry similarity \(single\-model\)\.

The mean cosine similarity between consecutive carry states \(Table[7](https://arxiv.org/html/2606.26488#A5.T7)\) is task\-specific but*insensitive*to compression: it is essentially identical under FP32, INT8, and INT4 on every task \(changes≤0\.04\\leq 0\.04\), and is therefore a weak failure signal\. We report it for completeness; the diagnostic below is the one we recommend\.

Table 7:Mean consecutive carry\-state cosine similarity\. The change under INT4 is small on every task \(≤0\.04\\leq 0\.04\); the signal does not reliably separate compression that preserves reasoning from compression that destroys it\. \(Maze/Sudoku reproduced; ARC from the source run\.\)
#### Carry\-trajectory fidelity \(vs\. full\-precision reference\)\.

The recommended diagnostic is the cosine similarity between the quantized model’s final carry state and the FP32 model’s final carry state on the same input—how far quantization pushes the reasoning trajectory off the full\-precision path \(Table[8](https://arxiv.org/html/2606.26488#A5.T8)\)\. It requires only the FP32 reference \(available before deployment\) and no task labels\. Unlike the consecutive signal, fidelity is*graded and monotonic with severity*: it stays≈\\approx1\.0 wherever INT4 is harmless \(Maze, all INT8\) and falls in proportion to the accuracy damage—moderately on ARC \(0\.690\.69,−10\-10pp\) and catastrophically on Sudoku \(0\.350\.35,−64\-64pp\)\. Per\-channel calibrated INT4*recovers*fidelity in lockstep with accuracy, so the diagnostic predicts both failure and recovery\.

Table 8:Carry\-trajectory fidelity \(final\-state cosine vs\. FP32\)\. Monotonic with INT4 severity \(Maze→\\toARC→\\toSudoku\); INT8 is never flagged; calibrated INT4 recovers both fidelity and accuracy\. Last row: INT4\-naïve puzzle\-exact change vs\. FP32\.
#### Calibrated INT4 recovers Sudoku\.

Measured under an identical evaluation harness \(N=256N\{=\}256, 16 ACT steps, no TTA\), naïve per\-tensor INT4 collapses Sudoku to10\.210\.2% puzzle\-exact while per\-channel calibrated INT4 recovers to71\.971\.9% \(vs\.73\.873\.8% FP32 under the same harness\)—i\.e\., the catastrophic INT4 result is specific to naïve quantization, and the 4 MB MCU target is reachable with calibration \(Table[9](https://arxiv.org/html/2606.26488#A5.T9)\)\.

Table 9:Sudoku\-Extreme under an identical harness \(N=256N\{=\}256, no TTA; FP32 reads73\.873\.8here vs\.69\.169\.1full\-set/TTA elsewhere\)\. Fidelity tracks accuracy across all four variants\.

## Appendix FArchitecture Ablation: MLP\-Mixing vs\. Attention

To separate architecture from task precision we evaluate three recursive architectures on the*same*task \(Sudoku\-Extreme\) under an identical harness \(N=256N\{=\}256, 16 ACT steps, no TTA; Table[10](https://arxiv.org/html/2606.26488#A6.T10)\)\. TRM\-MLP\-Mixing and TRM\-Attention are the same model family at the same full\-precision accuracy \(73\.873\.8vs\.73\.173\.1% puzzle exact\), differing only in the token mixer; HRM\(Wanget al\.,[2025](https://arxiv.org/html/2606.26488#bib.bib8)\)is a different recursive architecture \(separate high\- and low\-level modules, attention\-based\)\. Under naïve INT4, TRM\-MLP\-Mixing collapses to10\.210\.2% while both attention architectures retain their accuracy\. Since TRM\-Attention matches TRM\-MLP at full precision, this is not an accuracy\-headroom effect—the INT4 fragility is specific to MLP\-mixing, not the task\. The carry\-trajectory fidelity diagnostic grades all three correctly \(0\.350\.35/0\.870\.87/0\.980\.98\), confirming it generalizes across recursive architectures\.

Table 10:Same task \(Sudoku\-Extreme\), three recursive architectures: naïve INT4 puzzle\-exact \(%\) and final\-state carry fidelity\. Only MLP\-mixing collapses; both attention architectures—including a different model family \(HRM\)—survive\. TRM\-Attention matches TRM\-MLP at FP32, ruling out an accuracy\-headroom confound\.
## Appendix GOn\-Device Hardware Benchmarks

Rather than relying purely on first\-order analytical projections, we compile physical, on\-device step and puzzle latencies, peak memory usage, and file sizes directly on three physical edge devices:

- •Samsung Galaxy S24: Featuring the flagship Qualcomm Snapdragon 8 Gen 3 chipset \(SM8650, Hexagon v75 NPU\)\.
- •Samsung Galaxy S22 5G: Featuring the Qualcomm Snapdragon 8 Gen 1 chipset \(SM8450, Hexagon v70 NPU\)\.
- •Dragonwing RB3 Gen 2: Featuring the Qualcomm QCS6490 chipset \(Hexagon NPU\)\.

Evaluations are compiled and run via Qualcomm AI Hub targeting native hardware execution\. Table[11](https://arxiv.org/html/2606.26488#A7.T11)reports the full set of consolidated benchmark results\.

As discussed in Section[4\.2](https://arxiv.org/html/2606.26488#S4.SS2), while static INT8 quantization delivers substantial latency gains for compute\-bound tasks with long sequences \(like ARC\-2024 and Maze\-Hard\), it incurs a step\-latency penalty on the computationally light Sudoku\-Extreme task \(which is instead dominated by NPU driver/dispatch overhead and format conversion casting\)\.

Table 11:On\-device hardware benchmark results compiled via Qualcomm AI Hub\. All evaluations are run on physical devices: Galaxy S24 \(Snapdragon 8 Gen 3\), Galaxy S22 \(Snapdragon 8 Gen 1\), and Dragonwing \(Qualcomm QCS6490\)\.### G\.1Sequence\-Length Dependent Quantization Penalty

The latency benchmark in Table[11](https://arxiv.org/html/2606.26488#A7.T11)reveals an interesting and counter\-intuitive result: while static INT8 quantization accelerates the ARC\-2024 and Maze\-Hard models, it actually leads to a latency slowdown for the Sudoku\-Extreme model per step \(from 11\.38 ms to 27\.23 ms on the S24\)\. This behaviour is a classic*small\-matrix INT8 penalty*governed by the sequence length \(TT\):

- •Compute\-Bound Regime \(ARC & Maze\):ARC and Maze operate on a sequence lengthT=900T=900\. At this scale, the weight projection matrices are large \(900×512900\\times 512by512×2048512\\times 2048\), which translates to heavy arithmetic compute \(∼\\sim187 GFLOPs/step\)\. The Snapdragon Hexagon HTP and CPU vector units \(NEON SIMD\) can tile these matrices efficiently\. Moving this workload to the NPU’s INT8 tensor cores reduces execution time significantly, easily overriding the fixed NPU driver/dispatch overhead\.
- •Overhead\-Bound Regime \(Sudoku\):Sudoku has a sequence length of onlyT=81T=81\(9797with padding and embeddings\)\. The matrix multiplications are tiny \(e\.g\.,97×51297\\times 512by512×2048512\\times 2048\), requiring only∼\\sim25 MFLOPs/step \(7,500×\\timesless compute than Maze\)\. A97×51297\\times 512matrix multiplication requires only∼\\sim100K multiply\-accumulates, which is too small to saturate the INT8 parallel execution blocks\. Consequently, the fixed overhead of the Quantize/Dequantize \(QDQ\) operators—which convert tensors back and forth at every node boundary—dominates the execution time\. While the FP32 model runs in a single, unbroken compiler\-fused pipeline in 11\.38 ms, the INT8 model is penalized by a constant∼\\sim25 ms NPU invocation and marshalling overhead\.

Furthermore, Sudoku relies on token\-mixing MLP layers \(mlp\_tlayers of 388 KB/194 KB\)\. These represent extremely small matrices\. The Qualcomm Hexagon tensor processor has a minimum efficient tile size threshold; matrix dimensions below this limit fail to trigger accelerated execution paths and fall back to slower scalar execution, further exacerbating the INT8 penalty\.

The crossover point where static INT8 quantization becomes beneficial for this recursive transformer architecture on mobile chipsets is approximatelyT≈200​–​300T\\approx 200\\text\{\-\-\}300\. Below this threshold, floating\-point execution remains faster\.

## Appendix HEnergy per Inference

Estimated energy follows the FLOPs counts under a Cortex\-M55 power model \(∼\\sim1 GFLOP/mW\): ARC\-2024 at full depth \(∼\\sim3,000 GFLOPs/puzzle\) draws∼\\sim3,000 mJ/puzzle, versus∼\\sim500 mJ at the reduced configuration \(H=1H\{=\}1,nsup=8n\_\{\\text\{sup\}\}\{=\}8\)\. Sudoku\-Extreme \(∼\\sim256 GFLOPs at default depth\) draws∼\\sim256 mJ/puzzle\. These are analytical estimates under the same FLOPs\-based model, and motivate the6×6\\timesFLOPs reduction as a6×6\\timesenergy reduction at no accuracy cost\.

## Appendix IContext\-Length Truncation

Because attention cost scales as𝒪​\(T2\)\\mathcal\{O\}\(T^\{2\}\), we tested zero\-padding the input beyond a truncated context\. Truncation is universally destructive \(Table[12](https://arxiv.org/html/2606.26488#A9.T12)\): even halving the sequence length collapses puzzle\-exact accuracy on every task\. There is no cheap context shortcut—the full window is load\-bearing\.

Table 12:Puzzle\-exact accuracy under input context truncation\. \(Maze/Sudoku collapse to 0 at 50%; ARC degrades gracefully but severely\.\)
## Appendix JLinear\-Attention Approximation \(Maze\)

Replacing softmax attention with ELU feature\-map linear attention \(𝒪​\(n\)\\mathcal\{O\}\(n\)vs\.𝒪​\(n2\)\\mathcal\{O\}\(n^\{2\}\)\) on Maze illustrates the same cell\-vs\-puzzle split as quantization: the swap preserves token\-level accuracy but destroys puzzle\-level reasoning\. Zero\-shot, cell accuracy holds at 87\.3% while puzzle\-exact falls to 0\.00% \(from 78\.50% / 99\.40%\); brief retraining \(200 steps\) recovers cell accuracy \(87\.5%\) but not puzzle\-exact \(0\.00%\)\. This is a fourth intervention—alongside INT4, pruning, and distillation—under which local prediction survives while global constraint satisfaction collapses\. Sudoku\-Extreme \(MLP\-mixing\) has no attention layers and is not applicable\.

## Appendix KStructured Pruning and Knowledge Distillation

#### Structured pruning\.

Magnitude\-based structured pruning was evaluated at 25% and 50% sparsity targets across all three tasks\.

Table 13:Structured pruning results\. Even 25% sparsity destroys puzzle\-level reasoning across all tasks\. Cell accuracy partially survives on Maze \(86\.40%\) and Sudoku \(50\.01%\), indicating that token prediction survives while global constraint satisfaction does not\.
#### Knowledge distillation\.

Student models \(hidden=256, 1 layer, 1\-cycle\) were trained via KL\-divergence distillation from the full teacher\.

Table 14:Knowledge distillation results\. Student models \(12–15% of teacher params\) learn token\-level patterns but fail entirely at puzzle\-level reasoning, suggesting the recursive structure is functionally necessary\.

## Appendix LImplementation Details

#### Quantization\.

INT8 and INT4 PTQ were applied usingbitsandbytesv0\.41\(Dettmerset al\.,[2022](https://arxiv.org/html/2606.26488#bib.bib15)\)withload\_in\_8bit=Trueandload\_in\_4bit=Truerespectively\. Per\-channel calibration for INT4 calibrated used 128 calibration puzzles drawn from the training set\.

#### Carry\-state extraction\.

shs\_\{h\}was computed by registering a forward hook on the carry\-state update functiongϕg\_\{\\phi\}to captureZhZ\_\{h\}at each outer cycle\. Cosine similarity was computed token\-wise and averaged across the sequence dimension and across all puzzles in the evaluation set\.

#### QAT\.

QAT was initialized from the naïve INT4 checkpoint and fine\-tuned for 100 steps atlr=10−5\\text\{lr\}=10^\{\-5\}with cross\-entropy loss over the token prediction head, using a batch size of 4 puzzles\. Training and evaluation used the Modal cloud platform\.

#### ONNX export and edge deployment\.

Models were exported to ONNX format usingtorch\.onnx\.exportfor deployment via ONNX Runtime and Qualcomm AI Hub\. The ONNX export produces a single\-step model \(TRMBackboneStep\) that executes one H\-cycle step, with the outer recursion loop managed by the inference harness\.

## Appendix MPeak SRAM Memory Footprint and Host\-Side Compression

For on\-device edge deployment, we evaluate the peak memory footprint under two execution scenarios: a*Full Load*scenario \(where the entire backbone and intermediate activations/carries are stored in SRAM\) and a*Streaming*scenario \(where only the current recursive step is loaded, leveraging single\-puzzle flash loading of embedding tables\)\.

Table[15](https://arxiv.org/html/2606.26488#A13.T15)compiles these peak SRAM requirements against a theoretical 8 MB SRAM hardware budget limit\.

Table 15:Peak memory footprint \(SRAM\) for deployable ONNX models\. Embedding layers are excluded \(no\-embed\) as they are streamed via flash loading\. The 8 MB target assumes streaming mode\.By default, only the Sudoku \(INT8\) model fits within the 8 MB SRAM budget \(requiring 3\.06 MB streaming peak\)\. However, we can applyHost\-side compressionto enable ARC and Maze deployment\. Host\-side compression is a deployment runtime decision that is completely orthogonal to the ONNX graph itself:

- •The ONNX model backbone remains unchanged, executing in high\-precision FP32\.
- •However, between recursive steps, the host runtime quantizes the FP32 intermediate activation and carry state tensors to INT8 for storage in host memory \(SRAM\)\.
- •Before the next step is fed, the runtime dequantizes them back to FP32\.

This dynamic host\-side compression eliminates the SRAM memory overhead of storing high\-precision intermediate states, bringing the ARC and Maze streaming peak memory footprint from 9\.51 MB down to∼\\sim6\.82 MB—successfully fitting both models within the 8 MB SRAM edge budget\.

## Appendix NDetailed Recursive Sweep Grids

We report the complete multi\-variant validation sweeps across combinations of recursive steps \(nn\) and precision modes on GPU \(CUDA\) for all three architectures\. Tables[16](https://arxiv.org/html/2606.26488#A14.T16),[17](https://arxiv.org/html/2606.26488#A14.T17), and[18](https://arxiv.org/html/2606.26488#A14.T18)present these complete grids, including exact match accuracy and cell accuracy\.

Table 16:Detailed recursive sweep grid for ARC\-2024 \(seq\_len = 900\) acrossHH\(outer cycles\),nn\(supervision steps\), and precision variants\.Table 17:Detailed recursive sweep grid for Maze\-Hard \(seq\_len = 900\) acrossHH\(outer cycles\),nn\(supervision steps\), and precision variants\.Table 18:Detailed recursive sweep grid for Sudoku\-Extreme \(seq\_len = 81\) acrossHH\(outer cycles\),nn\(supervision steps\), and precision variants\.