Reasoning with Neural Cellular Automata

# Reasoning with Neural Cellular Automata
Source: [https://arxiv.org/html/2609.36126](https://arxiv.org/html/2609.36126)
\\uselogo

Pietro MiottiAffiliation:Google Paradigms of Intelligence TeamAidan SirbuAffiliation:Google Paradigms of Intelligence TeamAffiliation:School of Computer Science, McGill UniversityAffiliation:Mila \- Quebec AI InstituteKonstantin SchürholtAffiliation:Google Paradigms of Intelligence TeamMariia DrozdovaAffiliation:Google Paradigms of Intelligence TeamAffiliation:University of GenevaArna GhoshAffiliation:Google Paradigms of Intelligence TeamBlaise Agüera y ArcasAffiliation:Google Paradigms of Intelligence TeamJames ManyikaAffiliation:Google Paradigms of Intelligence TeamBlake RichardsAffiliation:Google Paradigms of Intelligence TeamAffiliation:School of Computer Science, McGill UniversityAffiliation:Mila \- Quebec AI InstituteAffiliation:Department of Neurology and Neurosurgery, McGill UniversityAffiliation:Montreal Neurological Institute, McGill UniversityAffiliation:Learning in Machines and Brains Program, CIFARAffiliation:Equal supervisionEyvind NiklassonAffiliation:Google Paradigms of Intelligence TeamAffiliation:Equal supervision

###### Abstract

Modern AI architectures used to solve visual reasoning tasks typically rely heavily on global connectivity and synchronization\. As biological systems demonstrate, though, sophisticated computation can be performed in a more decentralized fashion\. In this work, we test the reasoning capabilities of Neural Cellular Automata \(NCAs\), networks of recurrent cells that use strictly local connectivity and asynchronous updates\. NCAs have been extensively studied in artificial life experiments, but it is unclear whether they can perform complex multi\-step reasoning\. We show that NCAs produce spatio\-temporal dynamics capable of solving challenging visual reasoning tasks, including large mazes, Sudoku, and ARC\-AGI\-1\. Furthermore, we provide evidence that NCAs generalize out\-of\-distribution when running with larger grids, longer rollouts, or parallel trials; and that the latter can be made more efficient via pruning of redundant trajectories\. We find that these generalization capabilities depend on training with sample replay and stochastic perturbations, and that stochasticity remains beneficial at test time\. Finally, we show that NCAs are robust reasoners capable of dynamically modulating compute to recover efficiently from damage, and that they can scale to solve reasoning in raw pixel space\.

###### keywords

Neural Cellular Automata \(NCAs\), Self\-Organizing Systems, Visual Reasoning

## 1Introduction

Self\-organization, the process by which low\-level units interact locally to produce sophisticated global patterns, is a staple of biological life and intelligence\([Camazine et al\., 2003](https://arxiv.org/html/2609.36126#bib.bib35);[Karsenti, 2008](https://arxiv.org/html/2609.36126#bib.bib34)\)\. The degree of self\-organization in a computational system can range across a spectrum, from purely centralized architectures to highly localized, decentralized interactions\. Most modern artificial intelligence \(AI\) systems use distributed representations, but they still rely heavily on global connectivity and synchronized interaction between units, placing them farther along the centralization spectrum\. Moreover, while architectures such as transformers and their looped variants have been widely used in recent years, their all\-to\-all connectivity and associated data\-movement dictate energy cost, and fundamentally set limits on their underlying physical computing paradigms\. Enforcing stronger locality constraints from the ground up could offer a principled alternative for AI models that run on decentralized, fully self\-organizing computing substrates\. However, it remains an open question to what extent locality constrained architectures can actually be scaled, and whether they can really perform complex tasks to address modern AI challenges\. In particular, it remains unclear whether low\-level, localized communication protocols can solve complex reasoning tasks\. Reasoning is a cornerstone of general intelligence that presents a notorious challenge even for modern deep learning architectures\. Classical tasks such as pathfinding \(e\.g\., mazes\) and constraint satisfaction problems \(e\.g\., Sudoku\), along with recent benchmarks like ARC\-AGI, are being extensively used to benchmark the ability of AI systems to solve logical problems and generalize out\-of\-distribution\. Solving these tasks requires AI models to jointly learn to infer the underlying logic and to execute the multi\-step procedures necessary to generate valid solutions, all while being trained solely on input\-output observations\. Current research typically follows one of two approaches: prompting large, generalist LLMs with in\-context task information; or using recursive reasoning approaches with compact specialist models such as HRM\([Wang et al\., 2025](https://arxiv.org/html/2609.36126#bib.bib11)\), TRM\([Jolicoeur\-Martineau, 2025](https://arxiv.org/html/2609.36126#bib.bib12)\)or LoopViT\([Shu et al\., 2026](https://arxiv.org/html/2609.36126#bib.bib2)\)\. Yet, most reasoning models developed to date still rely heavily on dense, longe\-range connectivity and synchronized execution, allowing every part of the context to be attended to at each update step\.

Here we explore whether there is truly a need for global connectivity and coordination in order to engage in effective reasoning\. Specifically, we investigate whether Neural Cellular Automata \(NCA\), a recent neural network architecture that uses fully local connectivity between a distributed grid of recurrent cells\([Mordvintsev et al\., 2020](https://arxiv.org/html/2609.36126#bib.bib1)\), can support distributed reasoning\. At the intersection of Artificial Life and AI, and inspired by research on self\-organization and morphogenesis, NCAs can be viewed as specific instances of recurrent convolutional or graph neural networks with a strong locality constraint\. They are also closely related to recent recursive reasoning architectures, with the central difference being their use oflocal connectivityandasynchronous updates\. NCAs have already been applied across diverse domains, from pattern generation and repair\([Mordvintsev et al\., 2020](https://arxiv.org/html/2609.36126#bib.bib1);[Kim et al\., 2026](https://arxiv.org/html/2609.36126#bib.bib5)\)to classification\([Randazzo et al\., 2020](https://arxiv.org/html/2609.36126#bib.bib7)\), segmentation\([Sandler et al\., 2020](https://arxiv.org/html/2609.36126#bib.bib6)\), control\([Variengien et al\., 2021](https://arxiv.org/html/2609.36126#bib.bib8)\), and as ViT adaptor layers\([Xu et al\., 2024](https://arxiv.org/html/2609.36126#bib.bib21)\), showing compelling properties such as high parameter efficiency and robustness\. Although recent studies demonstrated initial success applying NCAs to mazes\([Earle et al\., 2023](https://arxiv.org/html/2609.36126#bib.bib20)\)and ARC\-AGI\-1\([Etienne et al\., 2025](https://arxiv.org/html/2609.36126#bib.bib19)\), their scaling behavior and potential for reasoning remain largely unexplored\.

In this work, we demonstrate the reasoning capabilities of NCAs across a suite of reasoning benchmarks: pathfinding on large out\-of\-distribution mazes\([Schwarzschild et al\., 2021](https://arxiv.org/html/2609.36126#bib.bib13)\)and multi\-solution ones\([Wang et al\., 2025](https://arxiv.org/html/2609.36126#bib.bib11)\); constraint\-satisfaction on out\-of\-distribution\([Miyato et al\., 2024](https://arxiv.org/html/2609.36126#bib.bib15)\)and extremely difficult\([Wang et al\., 2025](https://arxiv.org/html/2609.36126#bib.bib11)\)Sudoku boards; program induction and few\-shot visual reasoning on the ARC\-AGI\-1 benchmark\([Chollet, 2019](https://arxiv.org/html/2609.36126#bib.bib3)\); and joint perception\-reasoning directly in pixel space on Visual Sudoku\([Wang et al\., 2019](https://arxiv.org/html/2609.36126#bib.bib18)\)\. Our main contributions are fourfold\. First, we show that, despite their strong locality constraints, NCAs perform well on all these benchmarks using compact architectures\. With few parameters, cells locally coordinate to solve complex multi\-step reasoning tasks, with the spatial communication bottleneck giving rise to structured, observable reasoning dynamics that unfold across both space and time\. On ARC\-AGI\-1, we show that a single NCA can solve multiple tasks and transfer across them when provided with appropriate contextual information\. Second, we demonstrate that NCAs can generalize to harder, out\-of\-distribution tasks when compute is expanded at test\-time across either space or time, or via parallelization\. To mitigate the cost of running multiple, parallel rollouts, we propose a pruning strategy that shows more efficient scaling for low compute budgets\. We show that the generalization capabilities of these recursive, distributed reasoners depend on training with sample replay and stochastic perturbations, and that stochasticity remains beneficial at test time\. Third, we highlight compelling properties of NCAs, revealing that NCAs are robust and adaptive reasoners that can dynamically modulate their compute to solve tasks and repair from damage more efficiently\. Finally, we show that NCAs can reason in larger, harder problem modalities and solve Sudokus directly in pixel space\.

![Refer to caption](https://arxiv.org/html/2609.36126v1/figures/Figure_1_v3.png)Figure 1:Overview of reasoning with Neural Cellular Automata \(NCA\)\.Cells maintain structured states \(CinC\_\{\\text\{in\}\},CoutC\_\{\\text\{out\}\},ChidC\_\{\\text\{hid\}\}\) on anH×WH\\times Wgrid and update iteratively using local3×33\\times 3perception and a shared update module, applied residually under a stochastic firing mask\. Training unrolls grids from the dataset or a sample replay buffer forNNsteps, optimizing the NCA to solve the task\.
## 2Related Work: Recurrent Visual Reasoning \(RvR\)

##### RvR Architectures\.

Most recent RvR models use variants of looped transformers with global attention\([Miyato et al\., 2024](https://arxiv.org/html/2609.36126#bib.bib15);[Wang et al\., 2025](https://arxiv.org/html/2609.36126#bib.bib11);[Jolicoeur\-Martineau, 2025](https://arxiv.org/html/2609.36126#bib.bib12);[Baek et al\., 2026](https://arxiv.org/html/2609.36126#bib.bib25)\)\. LoopViT uses a hybrid architecture that alternates global and local attention layers\([Shu et al\., 2026](https://arxiv.org/html/2609.36126#bib.bib2)\)\. Some prior works use fully\-local NCAs for pathfinding\([Endo and Yasuoka,](https://arxiv.org/html/2609.36126#bib.bib24);[Earle et al\., 2023](https://arxiv.org/html/2609.36126#bib.bib20)\)with some degree of length generalization using handcoding\-inspired variants, and for ARC\-AGI\-1 achieving 12\.9% on a subset of 262 fixed\-size tasks by training one NCA per task\([Etienne et al\., 2025](https://arxiv.org/html/2609.36126#bib.bib19)\)\. Sheaf\-ADMM\([Seely et al\., 2026](https://arxiv.org/html/2609.36126#bib.bib26)\)uses a NCA\-like architecture with local agents for visual reasoning, but relies on non\-strict locality for Sudoku \(i\.e\. agents can see full rows, columns or blocks\) and shows very limited length\-generalization on mazes\. In our work, cells coordinate under strictly local observability to solve complex problems including Sudoku\-Extreme and multi\-task ARC\.

##### RvR Training\.

Training NCAs withsample replay\([Mordvintsev et al\., 2020](https://arxiv.org/html/2609.36126#bib.bib1);[Du and Mordatch, 2019](https://arxiv.org/html/2609.36126#bib.bib10)\)parallels strategies like progressive loss\([Bansal et al\., 2022](https://arxiv.org/html/2609.36126#bib.bib14)\)and deep supervision\([Jolicoeur\-Martineau, 2025](https://arxiv.org/html/2609.36126#bib.bib12)\)used in the recent RvR literature to emulate extended rollouts and encourage convergence to stable fixed\-points without deep unrolling\. Exploitingstochasticityis also proposed by other recent RvR work\([Baek et al\., 2026](https://arxiv.org/html/2609.36126#bib.bib25)\), but they do so by injecting some noise in part of the state used for prediction, whereas we use asynchronous updates and perturbations across the full state\. Other concurrent works\([Helbling et al\., 2026](https://arxiv.org/html/2609.36126#bib.bib28);[Drozdova et al\., 2026](https://arxiv.org/html/2609.36126#bib.bib29);[Suleymanzade et al\., 2026](https://arxiv.org/html/2609.36126#bib.bib27)\)propose that diffusion\-inspired progressive noise and local denoising objectives may provide another effective alternative for regularizing the convergence landscape of RvR models\.

##### RvR Test\-Time Scaling\.

RvR models are dynamical systems with complex state spaces\([Lai et al\., 2026](https://arxiv.org/html/2609.36126#bib.bib23)\)\. Several works, including ours, leverage stochastic parallel trials for exploring the state\-space at test\-time, effectively increasing the chance of discovering the correct solution with some variants in the candidate selection mechanism\([Miyato et al\., 2024](https://arxiv.org/html/2609.36126#bib.bib15);[Sghaier et al\., 2026](https://arxiv.org/html/2609.36126#bib.bib16);[Baek et al\., 2026](https://arxiv.org/html/2609.36126#bib.bib25)\)\. In addition, we proposeNiche\-Capped Diversity Pruningto discard redundant trajectories and optimize test\-time scaling for constrained compute budgets \(see section[4\.3](https://arxiv.org/html/2609.36126#S4.SS3)\)\.

##### RvR with Adaptive Compute\.

Several RvR models use global convergence metrics as early\-exit mechanisms, such as learned halting\([Jolicoeur\-Martineau, 2025](https://arxiv.org/html/2609.36126#bib.bib12)\)or entropy monitoring\([Shu et al\., 2026](https://arxiv.org/html/2609.36126#bib.bib2)\)\. While these methods act as global on/off switches that stop all compute at once, our approach decentralizes adaptive compute: cells modulate their update frequency independently based on self\-predicted confidence\. This creates an organic allocation of compute that naturally concentrates on unresolved or perturbed sub\-regions, while stable cells idle and save resources \(see section[4\.4](https://arxiv.org/html/2609.36126#S4.SS4)\)\.

## 3Method

In this section, we introduce the NCA architecture \(Figure[1](https://arxiv.org/html/2609.36126#S1.F1)\) and the associated training regimes and test\-time scaling procedures that we use for recursive reasoning\. Complete architectural, training, and testing details are provided in Appendix[B](https://arxiv.org/html/2609.36126#A2), and hyperparameters in Appendix[C](https://arxiv.org/html/2609.36126#A3)\.

### 3\.1Architecture

##### NCA Cell State\.

The grid state is represented asX∈ℝH×W×CX\\in\\mathbb\{R\}^\{H\\times W\\times C\}, where spatial dimensionsH×WH\\times Wmirror the problem topology \(e\.g\.,9×99\\times 9for Sudoku\) and each cell maintains a state vectorxi,j∈ℝCx\_\{i,j\}\\in\\mathbb\{R\}^\{C\}acrossCCchannels\. The state is partitioned into functional slices: output slice \(CoutC\_\{\\text\{out\}\}\) for token predictions, hidden slice \(ChidC\_\{\\text\{hid\}\}\) for latent reasoning, an immutable input slice \(CinC\_\{\\text\{in\}\}\) for task constraints, and an additional task slice \(CtaskC\_\{\\text\{task\}\}\) for ARC\-AGI\-1\. Unlike prior NCA works operating directly on RGB pixel values, our tasks use discrete vocabularies \(e\.g\. digits for Sudoku and color symbols for ARC\) mapped to fixed orthogonal embedding vectors that are loaded directly intoCinC\_\{\\text\{in\}\}at initialization \(t=0t=0\)\. Depending on the task, the inputCinC\_\{\\text\{in\}\}slice acts as environmental constraints \(e\.g\., spanning all channels in mazes to block information flow through walls\) or as environmental clues \(e\.g\., overlapping withCoutC\_\{\\text\{out\}\}in Sudoku to clamp known clues on the prediction channels, or forming a dedicated read\-only slice in ARC\-AGI\-1\)\. For ARC, the mutableCtaskC\_\{\\text\{task\}\}slice is initialized att=0t=0with a learned global task embedding to condition cells on the target task\.

##### NCA Cell Update\.

At each step, cells update asynchronously in two stages: \(i\)Perception, where a multi\-head perception module maps immediate3×33\\times 3neighbor states \(Moore neighborhood\) into a vectorzi,jz\_\{i,j\}; and \(ii\)Update, where a shared, weight\-tied MLP maps the perception vectorzi,jz\_\{i,j\}to an updateΔ​xi,j\\Delta x\_\{i,j\}, applied residually with a stochastic update maskmi,j∼Bernoulli​\(pfire\)m\_\{i,j\}\\sim\\text\{Bernoulli\}\(p\_\{\\text\{fire\}\}\):xi,j\(t\+1\)=xi,j\(t\)\+mi,j​Δ​xi,jx\_\{i,j\}^\{\(t\+1\)\}=x\_\{i,j\}^\{\(t\)\}\+m\_\{i,j\}\\Delta x\_\{i,j\}\. For perception, we use learned convolutions for Maze and Visual Sudoku, and a position\-dependent variant of attention called“fixed attention”for Sudoku and ARC\. While standard self\-attention is more expressive, we observed that attention weights converged to depend purely on relative positions rather than cell states \(Figure S[14](https://arxiv.org/html/2609.36126#A4.F14)\); replacing it with fixed attention improved performance \(Figure S[13](https://arxiv.org/html/2609.36126#A4.F13)\)\. After the update, we apply local\-only normalization to cells for Sudoku and ARC: channels are split into groups and normalized to unit length similar to[Miyato et al\. \(2024\)](https://arxiv.org/html/2609.36126#bib.bib15)\.

##### NCA Cell Prediction\.

At any step, the output slicexi,j,outx\_\{i,j,\\text\{out\}\}can be mapped to logit predictions via cosine similarity to token embeddings \(Maze, Sudoku\) or using a lightweight MLP head \(ARC\)\. A per\-cell confidenceci,j∈\[0,1\]c\_\{i,j\}\\in\[0,1\]is derived from the output distribution \(e\.g\., maximum probability\)\.

### 3\.2Training

##### Training with Sample Replay\.

Following the original NCA training pipeline\([Mordvintsev et al\., 2020](https://arxiv.org/html/2609.36126#bib.bib1)\), models are trained using Backpropagation Through Time \(BPTT\) coupled with asample replay bufferandtraining perturbations\. Training batches mix previously evolved grid states drawn from the buffer with fresh initializations from the dataset, with stochastic perturbations \(Gaussian noise, damage, and target swap\)\. The batch is then unrolled under the NCA update rule forNNsteps to compute gradients via BPTT, and post\-rollout states are written back into the buffer in place\.

##### Training with Time Encodings \(ARC\)\.

For ARC, we adapt the pre\-training and test\-time\-training \(TTT\) pipeline from recent VARC\([Hu et al\., 2025](https://arxiv.org/html/2609.36126#bib.bib4)\)and LoopViT\([Shu et al\., 2026](https://arxiv.org/html/2609.36126#bib.bib2)\)works\. Models are trained via BPTT forN=64N=64steps, using the RE\-ARC dataset and online augmentations \(reflections, color permutations, translations\)\. A learned time embedding is added to the hidden state of every cell at every step\. At inference, we replace this global step with individual cell clocks: each cell maintains an internal, locally\-incremented counter that advances only when the cell fires\.

### 3\.3Test\-time Scaling

##### Parallel Trials Scaling\.

NCA rollouts are inherently stochastic due to asynchronous updates and random initializations\. Consequently, runningKKparallel rollouts for the same input task produces distinct trajectories that increase the chance of converging to a valid solution \(parallel trials\)\. We then select the final prediction among theKKcandidates via either highest board\-level confidence \(average of all per\-cell confidencesci,jc\_\{i,j\}for Maze and Sudoku\) or via majority voting \(ARC\)\.

##### Niche\-Capped Diversity Pruning\.

To encourage parallel compute to be allocated across structurally diverse solution hypotheses rather than wasted on duplicate paths, we introduceNiche\-Capped Diversity Pruningfor parallel trials scaling\. At scheduled rollout checkpoints, active trajectories are grouped into discrete solution“niches”based on their current predictions; and the ensemble size is progressively halved by capping each niche at a maximum capacity \(see Appendix[B\.6](https://arxiv.org/html/2609.36126#A2.SS6)\)\.

##### Spatial Substrate Scaling\.

Because the cells that compose NCAs rely strictly on local perception they are agnostic to grid size; the computational substrate can be expanded at test time simply by tiling additional cells\. We exploit this form of scaling for Maze\-OOD, where cell perception is position\-independent and the grid can be seamlessly expanded with no weight updates\.

## 4Experimental Results

### 4\.1Experimental Setup

##### Benchmarks\.

We evaluate NCAs across various spatial reasoning tasks detailed in Appendix[A](https://arxiv.org/html/2609.36126#A1): \(i\)Maze path\-finding, testing out\-of\-distribution size generalization \(Maze\-OOD;[Bansal et al\., 2022](https://arxiv.org/html/2609.36126#bib.bib14)\) and solution ambiguity on hard mazes \(Maze\-Hard;[Wang et al\., 2025](https://arxiv.org/html/2609.36126#bib.bib11)\); \(ii\)Sudoku distributed constraint satisfaction problem \(DiSCP\), testing out\-of\-distribution difficulty generalization \(Sudoku\-OOD;[Miyato et al\., 2024](https://arxiv.org/html/2609.36126#bib.bib15)\) and challenging search and backtracks on very hard boards \(Sudoku\-Extreme;[Wang et al\., 2025](https://arxiv.org/html/2609.36126#bib.bib11)\); \(iii\)Few\-shot abstraction, testing task\-conditioned program induction from few input\-output demonstrations and augmentations \(ARC\-AGI\-1;[Chollet, 2019](https://arxiv.org/html/2609.36126#bib.bib3)\); and \(iv\)Pixel\-space reasoning, testing end\-to\-end perception and reasoning on much larger raw image inputs \(Visual\-Sudoku\)\.

##### Evaluation\.

For each benchmark, we train a NCA using the architecture and training pipelines detailed in Appendix[B](https://arxiv.org/html/2609.36126#A2), and experimental settings provided in Appendix[C](https://arxiv.org/html/2609.36126#A3)\. Inference runsKKparallel rollouts of the trained model, each forDDiterations\. The values ofD,KD,K, and grid size \(S=H×WS=H\\times W\) used for each benchmark are specified in Table[1](https://arxiv.org/html/2609.36126#S4.T1)\.

##### Baselines\.

Our objective is not to establish new state\-of\-the\-art results, but to demonstrate that NCAs can execute complex reasoning tasks in a strictly local, fully distributed fashion\. Nonetheless, we include top\-performing RvR baselines as reference points to contextualize NCA results: DeepThink\([Bansal et al\., 2022](https://arxiv.org/html/2609.36126#bib.bib14)\)for Maze\-OOD; PTRM\([Sghaier et al\., 2026](https://arxiv.org/html/2609.36126#bib.bib16)\)for Maze\-Hard; AKOrN\([Miyato et al\., 2024](https://arxiv.org/html/2609.36126#bib.bib15)\)for Sudoku\-OOD; PTRM for Sudoku\-Extreme; and TRM and LoopViT\([Jolicoeur\-Martineau, 2025](https://arxiv.org/html/2609.36126#bib.bib12);[Shu et al\., 2026](https://arxiv.org/html/2609.36126#bib.bib2)\)for ARC\-AGI\-1\.

Table 1:Overview of NCA performances and efficiency across reasoning benchmarks \(see section[B\.5](https://arxiv.org/html/2609.36126#A2.SS5)for FLOPs estimation\)\.![Refer to caption](https://arxiv.org/html/2609.36126v1/figures/figure_reasoning_skills.png)Figure 2:Emergent reasoning dynamics in NCAs\.\(A\) Backtracking in mazes:cell activations explore paths in parallel, then locally back\-propagate“waves”to prune dead\-end paths, until converging to the final solution path\.\(B\) Iterative trajectory refinement in Maze\-Hard:cells rapidly resolve unambiguous segments \(t=15t=15\) until settling first on a valid, sub\-optimal solution \(1, blue\), before discovering a shorter valid alternative \(2\), and ultimately stabilizing into an optimal solution \(3, green\)\.\(C\) Local spatial propagation of objects and colors in ARC\-AGI\-1:solving the task of coloring gray shapes \(target\) according to the top\-left reference pattern \(source\) shows a continuous, step\-by\-step spatial diffusion of color features from source to target\.\(D\) Backtracking in Sudoku\-Extreme:on the hardest Sudoku boards \(tdokudifficulty 10; Figure S[16](https://arxiv.org/html/2609.36126#A4.F16)B\), cells demonstrate collective trial\-and\-error: they first reach a near\-valid grid but with a few conflicting digits \(t=60t=60\), they escape that local minimum by temporarily increasing constraint violations \(t=116t=116\), and finally converging to a correct global consensus \(t=250t=250\)\.

### 4\.2NCAs can solve complex reasoning tasks with a compact, fully local architecture

On all the evaluated visual reasoning benchmarks \(Mazes, Sudoku, ARC\-AGI\-1\), NCAs demonstrate strong accuracy using a highly compact, local architecture with few parameters \(Table[1](https://arxiv.org/html/2609.36126#S4.T1)\)\. Parameter efficiency is achieved by weight sharing: identical, local rules are executed recurrently over space and time to produce complex, multi\-step reasoning\. Because of the locality constraints, information can only spread between neighboring cells, which introduces spatial propagation delays\. As such, NCAs necessarily require more iterations \(higherDD\) to reach the correct solution than their globally connected counterparts\. Yet, we find that they maintain a modest total FLOP budget \(Table[1](https://arxiv.org/html/2609.36126#S4.T1)\) while supporting parallel, asynchronous execution at inference\.

In tasks where local information is key to reasoning, such as mazes, we find that NCAs engage in a form of distributed spatio\-temporal reasoning over different potential paths \(Figure[2](https://arxiv.org/html/2609.36126#S4.F2)A,B\), qualitatively similar to exploration by slime molds\([Nakagaki et al\., 2000](https://arxiv.org/html/2609.36126#bib.bib22)\)or breadth\-first search\([Earle et al\., 2023](https://arxiv.org/html/2609.36126#bib.bib20)\)\. Interestingly, recurrent models without locality constraints, such as TRM, do not appear to follow a localized path \(Figure S[15](https://arxiv.org/html/2609.36126#A4.F15)\) and typically converge in fewer steps using non\-local jumps \(Table S[7](https://arxiv.org/html/2609.36126#A4.T7)\)\. NCAs instead generate a form of interpretable“spatial chain\-of\-thought”through visible, step\-by\-step traces of spatial reasoning in mazes\. This local information flow in reasoning can also be observed in the ARC\-AGI\-1 tasks, where NCAs appear to propagate image information gradually across the grid to solve the task \(Figure[2](https://arxiv.org/html/2609.36126#S4.F2)C\)\.

Even for tasks like Sudoku, where global constraints are critical, we find that the locality bottleneck does not prevent NCAs from discovering valid solutions\. In fact, even on Sudoku\-Extreme boards requiring extensive search and backtracks, NCAs are able to escape states where local constraints are satisfied but global ones are not, and to iterate until arriving at a solution that eventually respects the global constraints \(Figure[2](https://arxiv.org/html/2609.36126#S4.F2)D\)\.

![Refer to caption](https://arxiv.org/html/2609.36126v1/figures/figure_arc_task_embedding_i_o_h.png)Figure 3:Multi\-task execution in a single NCA\.When evaluating the pre\-trained NCA on an unseen input \(“CA”in blue\), conditioned under different task embeddings, the model applies the specific transformation corresponding to each task \(filling enclosed and/or open regions with target color\), demonstrating context\-dependent execution rather than input memorization\.We find that the locality constraints of NCAs do not limit them to a single mode of spatio\-temporal reasoning dynamics, either\. A single NCA model can transfer across ARC\-AGI\-1 tasks, engaging in distinct forms of spatio\-temporal reasoning when conditioned on unique, learnable task embeddings\. When tested on novel, unseen inputs, the task embedding acts as a high\-level program instruction, such that the NCA executes the relevant spatial rule until achieving the desired input\-output transformation \(Figure S[3](https://arxiv.org/html/2609.36126#S4.F3)\)\.

### 4\.3Scaling compute in NCAs enables generalization to harder tasks

We next examined whether NCAs can generalize out\-of\-distribution to harder versions of the tasks they were trained on if we scale up compute at test\-time\. We used multiple ways to scale compute: launching independent, stochastic rollouts with varying initialization \(parallel trials scaling\), extending rollout length \(temporal scaling\), and expanding grid size \(spatial substrate scaling\)\.

![Refer to caption](https://arxiv.org/html/2609.36126v1/figures/figure_ood_generalization.png)Figure 4:Test\-time scaling improves NCA generalization on out\-of\-distribution \(OOD\) boards\.\(A\) Parallel state\-space exploration:running parallel, stochastic rollouts significantly improves OOD generalization on Sudoku\.\(B\) Spatial substrate scaling:NCAs trained on9×99\\times 9mazes generalize to up to500×500\\timeslarger mazes given additional substrate and iterations\. Mean curves over 3 test seeds\.First, similar to other works in the literature, we find that running parallel, longer trials and selecting candidates via scoring significantly improves performance and OOD generalization\. For instance, in Sudoku\-OOD, it unlocks solutions to Sudokus with as few as 17 clues despite being trained only on boards with at least 31 clues \(Figure[4](https://arxiv.org/html/2609.36126#S4.F4)A\)\. Here test\-time scaling acts as parallel state\-space exploration: independent rollouts navigate distinct regions of the state space until convergence, generating several candidate solutions which the scoring mechanism can then select from\. This simple exploration strategy shows similar improvements in performance for solving hard instances of Sudoku\-Extreme \(Figure S[16](https://arxiv.org/html/2609.36126#A4.F16)B\), Maze\-Hard \(Figure S[17](https://arxiv.org/html/2609.36126#A4.F17)\), and ARC\-AGI\-1 \(Figure[2](https://arxiv.org/html/2609.36126#footnote2)\)\.

![Refer to caption](https://arxiv.org/html/2609.36126v1/figures/figure_arc_ttt_cropped.png)Figure 5:Test\-time compute scaling on ARC\-AGI public evaluation set\.Pass@KKscaling under the offline\-first\-augmentation policy \(see Appendix[B\.3](https://arxiv.org/html/2609.36126#A2.SS3)and[B\.4](https://arxiv.org/html/2609.36126#A2.SS4)\), reaching 60\.3% Pass@64222Using an NCA ensemble \(NCA E\) of 3 models independently fine\-tuned further boosts performances to 63\.0%\.\.Running parallel stochastic rollouts increases compute costs, of course, proportionally toK×DK\\times D\. Under limited compute budgets, however, test\-time scaling can be made much more efficient by pruning redundant rollouts\. UsingNiche\-Capped Diversity Pruningon Sudoku\-Extreme, we find that the empirical compute\-accuracy Pareto frontiers significantly shift to the left as the number of halving steps \(mm\) increases: across the low\-to\-mid compute regimes, pruned rollouts achieve similar accuracy with up to4×4\\timeslower compute compared to unpruned baselines \(Figure[6](https://arxiv.org/html/2609.36126#S4.F6)A\)\. Similarly, at equivalent FLOP budgets, deeper rollouts compressed via pruning outperform standard baselines in compute\-constrained regimes: at a base budgetDbase=64D\_\{\\text\{base\}\}=64, quadrupling depth \(D=256,m=8D=256,m=8\) boosts accuracy from25\.6%25\.6\\%to81\.7%81\.7\\%\(Figure[6](https://arxiv.org/html/2609.36126#S4.F6)B\)\. Past a certain budget \(Dbase=256D\_\{\\text\{base\}\}=256\), pruning yields only small differences over full parallel sampling\.

![Refer to caption](https://arxiv.org/html/2609.36126v1/figures/figure_tts_pruning.png)Figure 6:Scaling laws and iso\-compute allocation of test\-time pruning on Sudoku\-Extreme\. \(A\) Exact accuracy versus total FLOPs across rollout depthsDDfor unpruned baselines and pruning divisorsmm\(shaded bands indicate±1\\pm 1SD\)\. \(B\) Allocation of fixed compute budgets to breadth \(unprunedDbaseD\_\{\\text\{base\}\}\) versus depth \(2⋅Dbase2\\cdot D\_\{\\text\{base\}\}withm=4m=4;4⋅Dbase4\\cdot D\_\{\\text\{base\}\}withm=8m=8\)\. Reallocating compute into depth yields accuracy gains up toDbase=128D\_\{\\text\{base\}\}=128\.On Maze\-OOD, we also investigate length generalization via a second form of test\-time scaling: spatial substrate scaling\. With neither weight updates nor architectural modifications, NCAs generalize on out\-of\-distribution mazes up to500×500\\timeslarger \(Figure[4](https://arxiv.org/html/2609.36126#S4.F4)B\)\. Interestingly, the NCAs learned a seemingly exact, generalizable, and interpretable pathfinding algorithm with wavefront expansion and backtracking that scales efficiently across the expanded substrates\. This leads to impressive length generalization despite being trained solely on small9×99\\times 9mazes \(3 minutes training on TPU\)\.

![Refer to caption](https://arxiv.org/html/2609.36126v1/figures/figure_noise_generalization.png)Figure 7:Stochasticity and perturbations are beneficial both at training and inference for generalization\.\(A\) Training ingredients ablation study:the use of asynchronous updates \(in particular for Sudoku\), stochastic perturbations \(noise, damage and target swap\) as well as sample replay during training are critical ingredients for generalization to out\-of\-distribution instances\. Mean\-std curves over 3 train seeds are displayed\.\(B\) Noise\-injection at test time:when adding noise of varying magnitude at test time, not only do the learned NCA rules show perfect robustness but noise even boosts performances across all benchmarks and noise scales\. Metrics are averaged over 3 test seeds\.To better understand how NCAs can generalize to harder tasks we ablated various components of the training and test\-time pipeline\. We find that training perturbations combined with sample replay act as critical regularizers during training to achieve generalization \(Figure[7](https://arxiv.org/html/2609.36126#S4.F7)A\)\. By exposing the model to perturbed states outside standard solution paths and varying rollout depthsDD, these strategies generate a rich distribution of dynamic trajectories during training and foster the learning of general solutions\. Interestingly, asynchronous updates are also critical for generalization in Sudoku, likely because updating cells non\-simultaneously helps to break symmetries and escape local minima during constraint resolution\. We also find that injecting random noise to cell states at test\-time is not only tolerated \(i\.e\. does not degrade performance\) but even improves reasoning accuracy across benchmarks \(Figure[7](https://arxiv.org/html/2609.36126#S4.F7)B\), suggesting noise is a beneficial feature for NCAs, not a vulnerability\.

### 4\.4NCAs are robust adaptive reasoners

![Refer to caption](https://arxiv.org/html/2609.36126v1/figures/figure_maze_xl.png)Figure 8:Robust adaptive compute on extra\-large mazes\.\(A\) Compute savings:lowering the fire rate of confident cells \(pfirep\_\{\\text\{fire\}\}=0\.4 ifci,jc\_\{i,j\}\>0\.95, else 0\.8\) requires more steps to solve the mazes \(reach y=0\.95 with1\.29×1\.29\\timessteps\), but reduces total cell updates to0\.71×0\.71\\times\.\(B\) Localized activity:10 steps after adding damage mid\-rollout \(cyan circles\), cells automatically activate either on damaged zones or on the yet\-unsolved backtracking front \(pink\), while stable path segments remain largely dormant\.\(C\) Enhanced damage recovery:When injecting damage att=6000t=6000, the adaptive strategy recovers faster \(0\.94×0\.94\\timessteps\) and cuts cumulative cell operations nearly in half \(0\.52×0\.52\\times\)\. Metrics are averaged over 3 test seeds\.One defining characteristic of NCAs is asynchronous execution: cells update independently without a shared global clock\. While the experiments above apply a uniform update rate across the grid, the decentralized design of NCAs allows cells to decide locally when and how often to update\. We explored the use of such adaptive compute within NCA models trained on the Maze\-OOD benchmark when deployed on extra large201×201201\\times 201grids\. Specifically, rather than updating uniformly, each cell’s update probability correlates with local certainty: the probability to update is low if the cell is“confident”\(pfire=0\.4p\_\{\\text\{fire\}\}=0\.4ifci,j\>0\.95c\_\{i,j\}\>0\.95\), while uncertain cells continue to update at the default rate \(pfire=0\.8p\_\{\\text\{fire\}\}=0\.8\)\. We found that with this form of adaptive compute the NCAs required more global iterations to converge to the correct solutions, but, because each iteration involved fewer cell updates, the total number of cell updates required to converge was reduced by about 30% \(Figure[8](https://arxiv.org/html/2609.36126#S4.F8)A\)\. This simple strategy shows that compute can be effectively concentrated on active, unsolved regions of the problem space while saving resources elsewhere\.

We then examined how adaptive compute impacted the robustness of NCAs in response to damage\. Mid\-rollout \(t=6000t=6000\), we damaged the NCAs by zeroing out cell states in random circular patches of the maze \(Figure[8](https://arxiv.org/html/2609.36126#S4.F8)B, cyan circles\)\. In response, adaptive updates naturally focused on either the damaged zones or on the end of the yet\-unsolved backtracking front \(pink segments\)\. Stable path segments however remained largely dormant, saving compute resources when not needed\. Interestingly, this adaptive compute accelerated damage repair: whether measured in iterations or total cell updates, adaptive NCAs repaired damaged paths and converged to the correct solution more efficiently, cutting the total repair\-and\-solve time in half \(0\.52×0\.52\\times; Figure[8](https://arxiv.org/html/2609.36126#S4.F8)C\)\. These results suggest that fault tolerance \(recovering from damage at a scale never seen during training\) and efficiency go hand in hand, pointing to adaptive compute strategies as another potential path towards more efficient, large\-scale reasoning\.

### 4\.5NCAs can scale to reason in pixel space

![Refer to caption](https://arxiv.org/html/2609.36126v1/figures/visual_sudoku_arxiv.png)Figure 9:256×256256\\times 256NCA solving an out\-of\-distributionhardsample from Visual Sudoku\.Our results thus far have demonstrated that NCAs can solve tasks where task\-specific semantic information is encoded into the inputs and outputs of the model, and where training occurs on limited grid sizes \(under32×3232\\times 32pixels\)\. We now demonstrate that we can relax these constraints, removing task\-specific semantic information, making the models perform iterative reasoning directly in pixel\-space over very large grids\. Specifically, we train NCAs on Visual Sudoku, a pure“image\-to\-image”reasoning task: presented with a256×256256\\times 256image of an incomplete Sudoku board with randomly sampled MNIST digits as input, NCAs must return a completed, rendered image of the solved Sudoku\. This task differs significantly from the previous ones, presenting a more general challenge as it requires \(i\) learning effective communication and coordination across hundreds of cells at training time; and \(ii\) simultaneously solving three separate tasks \(image classification, Sudoku solving, and image rendering\), all three at scales far beyond what an individual cell can perceive locally\.

To that end, we scale an“off\-the\-shelf”NCA architecture, with pixel inputs/outputs and fixed Sobel perception filters, trained with MSE loss against random MNIST images whose categories match the reference solution\. We relax task constraints and provide input clues via the initial image only \(t=0\), leaving these pixels mutable by the NCA\. The model is trained oneasypuzzles from the Visual\-Sudoku dataset, and tested on botheasyandhard\. On novel, unseen Sudoku images, it achieves an 87\.8% solve rate oneasyand up to 18\.9% onhard, using test\-time scaling via longer\-rollouts than those seen at training \(see Figure S[19](https://arxiv.org/html/2609.36126#A4.F19)\)\. Interestingly, qualitatively investigating the model’s behavior shows some form of distributed“chain\-of\-thought”process through time over the pixel space \(Figure[9](https://arxiv.org/html/2609.36126#S4.F9)\)\. At first, cells appear to perform classification of input digits by converting each clue \(given as new, unseen MNIST images\) into a canonical form \(resembling the mean of its digit class\), while filling blank slots with what looks like the average of all MNIST digits \(t=128t=128\)\. Then, cells proceed to iteratively solve the Sudoku, with partial guesses visible in the intermediate steps as rendered superpositions of possible digits \(t=1024t=1024\), until reaching a consensus on the final correct guesses likewise rendered in canonical form \(t=3072t=3072\)\.

While our training pipeline on Visual\-Sudoku is memory\-intensive and leaves room for optimization \(Appendix[C\.4](https://arxiv.org/html/2609.36126#A3.SS4)\), these results provide a proof\-of\-concept that fully local, compact models can scale to much harder tasks while executing very efficiently at inference in a fully decentralized fashion\.

## 5Discussion

Our main contributions in this paper were fourfold\. First, we have shown that spatial locality is not a barrier for complex multi\-step visual reasoning: evaluated on a suite of challenging benchmarks including mazes, Sudoku, and ARC\-AGI\-1, compact NCAs successfully generate structured reasoning dynamics through local, asynchronous updates \([4\.2](https://arxiv.org/html/2609.36126#S4.SS2)\)\. Second, we have shown that our core training recipe for NCAs \(using sample replay and state perturbations\), combined with the three axes of compute expansion for NCAs \(spatial substrate, temporal dimension, and stochastic trials\), enable strong out\-of\-distribution generalization on hard tasks \([4\.3](https://arxiv.org/html/2609.36126#S4.SS3)\)\. Third, we highlighted compelling properties of NCAs, demonstrating that self\-repair can be coupled with dynamic compute allocation to concentrate computational resources along active reasoning fronts \([4\.4](https://arxiv.org/html/2609.36126#S4.SS4)\)\. Finally, our results on Visual Sudoku suggest that this decentralized framework can scale to larger\-scale visual reasoning directly in raw pixel space, requiring cells to communicate across long distances \([4\.5](https://arxiv.org/html/2609.36126#S4.SS5)\)\.

Taken together, these results suggest that enforcing stronger locality constraints from the ground up could offer a principled alternative for designing AI systems that naturally map to decentralized, self\-organizing hardware and its critical requirements, such as low\-power communication and fault tolerance\. There are, however, several limitations to the present work that could be addressed in future works\. First, while inference is completely decentralized, training remains non\-local as we use truncated backpropagation through time, requiring global loss aggregation and error propagation\. Future work could explore training over shorter truncated horizons, for instance by decomposing tasks into local denoising objectives via diffusion\-based curricula\([Drozdova et al\., 2026](https://arxiv.org/html/2609.36126#bib.bib29)\), or by exploring local learning rules that eliminate global gradient backpropagation altogether\([Ernoult et al\., 2019](https://arxiv.org/html/2609.36126#bib.bib32);[Bellec et al\., 2020](https://arxiv.org/html/2609.36126#bib.bib33)\)\. Second, while exploring the NCA state space at test\-time via stochastic rollouts significantly increased the likelihood of converging to the correct solution, this exploration strategy remains a simple random search\. Future work may investigate active exploration strategies, such as curiosity\-driven diversity search, to uncover the diverse reachable attractors of these systems more efficiently than random search\([Reinke et al\., 2019](https://arxiv.org/html/2609.36126#bib.bib30);[Etcheverry, 2023](https://arxiv.org/html/2609.36126#bib.bib31)\)\. Finally, while strict locality comes with various useful properties regarding perspectives for hardware co\-design, it inevitably introduces information propagation delays and bottlenecks to resolve long\-range dependencies\. Future work could investigate NCA\-like architectures that are predominantly local but augmented with sparse, low\-bandwidth, longer\-range channels of communication, alongside evaluations of their physical trade\-offs\.

While the exact trajectory of future computing hardware remains uncertain, emerging paradigms such as neuromorphic and distributed spatial processors highlight a need for AI architectures that can operate in a low\-power, fault\-tolerant, and inherently decentralized manner\. We hope the insights from this study provide a modest step toward bridging that algorithmic and physical divide\.

## References

- Baeket al\.\(2026\)J\. Baek, M\. Jo, M\. Kim, M\. Ren, Y\. Bengio, and S\. AhnGenerative recursive reasoning\.arXiv \[cs\.AI\]\.Cited by:[§2](https://arxiv.org/html/2609.36126#S2.SS0.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2609.36126#S2.SS0.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2609.36126#S2.SS0.SSS0.Px3.p1.1)\.
- Bansalet al\.\(2022\)A\. Bansal, A\. Schwarzschild, E\. Borgnia, Z\. Emam, F\. Huang, M\. Goldblum, and T\. GoldsteinEnd\-to\-end algorithm synthesis with recurrent networks: logical extrapolation without overthinking\.arXiv \[cs\.LG\]\.Cited by:[§A\.1](https://arxiv.org/html/2609.36126#A1.SS1.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2609.36126#S2.SS0.SSS0.Px2.p1.1),[§4\.1](https://arxiv.org/html/2609.36126#S4.SS1.SSS0.Px1.p1.1),[§4\.1](https://arxiv.org/html/2609.36126#S4.SS1.SSS0.Px3.p1.1)\.
- Bellecet al\.\(2020\)G\. Bellec, F\. Scherr, A\. Subramoney, E\. Hajek, D\. Salaj, R\. Legenstein, and W\. MaassA solution to the learning dilemma for recurrent networks of spiking neurons\.Nat\. Commun\.11\(1\),pp\. 3625\(en\)\.Cited by:[§5](https://arxiv.org/html/2609.36126#S5.p2.1)\.
- Camazineet al\.\(2003\)S\. Camazine, J\. Deneubourg, N\. R\. Franks, J\. Sneyd, G\. Theraula, and E\. BonabeauSelf\-organization in biological systems\.Princeton Studies in Complexity,Princeton University Press,Princeton, NJ\(en\)\.Cited by:[§1](https://arxiv.org/html/2609.36126#S1.p1.1)\.
- Chollet \(2019\)F\. CholletOn the measure of intelligence\.External Links:1911\.01547Cited by:[§A\.3](https://arxiv.org/html/2609.36126#A1.SS3.p1.1),[§1](https://arxiv.org/html/2609.36126#S1.p3.1),[§4\.1](https://arxiv.org/html/2609.36126#S4.SS1.SSS0.Px1.p1.1)\.
- Drozdovaet al\.\(2026\)M\. Drozdova, A\. Sirbu, P\. Miotti, R\. Obryk, M\. Etcheverry, E\. Niklasson, and B\. RichardsDiffusion as a training curriculum for timestep\-free iterative reasoning\.arXiv \[cs\.LG\]\.Cited by:[§2](https://arxiv.org/html/2609.36126#S2.SS0.SSS0.Px2.p1.1),[§5](https://arxiv.org/html/2609.36126#S5.p2.1)\.
- Du and Mordatch \(2019\)Y\. Du and I\. MordatchImplicit generation and modeling with energy based models\.InAdvances in Neural Information Processing Systems,Vol\.32,pp\.\.Cited by:[§2](https://arxiv.org/html/2609.36126#S2.SS0.SSS0.Px2.p1.1)\.
- Earleet al\.\(2023\)S\. Earle, O\. Yildiz, J\. Togelius, and C\. HegdePathfinding neural cellular automata\.arXiv \[cs\.LG\]\.Cited by:[§1](https://arxiv.org/html/2609.36126#S1.p2.1),[§2](https://arxiv.org/html/2609.36126#S2.SS0.SSS0.Px1.p1.1),[§4\.2](https://arxiv.org/html/2609.36126#S4.SS2.p2.1)\.
- \[9\]K\. Endo and K\. YasuokaNeural cellular maze solver\.External Links:[Link](https://umu1729.github.io/pages-neural-cellular-maze-solver/)Cited by:[§2](https://arxiv.org/html/2609.36126#S2.SS0.SSS0.Px1.p1.1)\.
- Ernoultet al\.\(2019\)M\. Ernoult, J\. Grollier, D\. Querlioz, Y\. Bengio, and B\. ScellierUpdates of equilibrium prop match gradients of backprop through time in an RNN with static input\.arXiv \[cs\.LG\]\.Cited by:[§5](https://arxiv.org/html/2609.36126#S5.p2.1)\.
- Etcheverry \(2023\)M\. EtcheverryCuriosity\-driven AI for Science : Automated Discovery of Self\-Organized Structures\.Theses,Université de Bordeaux\.External Links:[Link](https://theses.hal.science/tel-04504878),[Document](https://dx.doi.org/10.70675/5041450bz7b8az4fcez966azf89b0a4e7e48)Cited by:[§5](https://arxiv.org/html/2609.36126#S5.p2.1)\.
- Etienneet al\.\(2025\)G\. Etienne, R\. Felix, K\. Mia, L\. Mikkel, and N\. StefanoARC\-NCA: towards developmental solutions to the abstraction and reasoning corpus\.arXiv \[cs\.AI\]\.Cited by:[§1](https://arxiv.org/html/2609.36126#S1.p2.1),[§2](https://arxiv.org/html/2609.36126#S2.SS0.SSS0.Px1.p1.1)\.
- Helblinget al\.\(2026\)A\. Helbling, A\. Bryutkin, M\. Martino, D\. H\. Chau, N\. Dehmamy, and H\. StrobeltFlow reasoning models: turning flows into efficient recurrent reasoners\.arXiv \[cs\.AI\]\.Cited by:[§2](https://arxiv.org/html/2609.36126#S2.SS0.SSS0.Px2.p1.1)\.
- Hodel \(2024\)M\. HodelAddressing the abstraction and reasoning corpus via procedural example generation\.External Links:2404\.07353Cited by:[§B\.3](https://arxiv.org/html/2609.36126#A2.SS3.SSS0.Px2.p1.1)\.
- Huet al\.\(2025\)K\. Hu, A\. Cy, L\. Qiu, X\. D\. Ding, R\. Wang, Y\. E\. Zhu, J\. Andreas, and K\. HeARC is a vision problem\!\.External Links:2511\.14761Cited by:[§A\.3](https://arxiv.org/html/2609.36126#A1.SS3.p1.1),[§B\.3](https://arxiv.org/html/2609.36126#A2.SS3.SSS0.Px3.p1.1),[§3\.2](https://arxiv.org/html/2609.36126#S3.SS2.SSS0.Px2.p1.1)\.
- Jolicoeur\-Martineau \(2025\)A\. Jolicoeur\-MartineauLess is more: recursive reasoning with tiny networks\.arXiv \[cs\.LG\]\.Cited by:[§B\.5](https://arxiv.org/html/2609.36126#A2.SS5.SSS0.Px2.p1.1),[§D\.2](https://arxiv.org/html/2609.36126#A4.SS2.p1.1),[§1](https://arxiv.org/html/2609.36126#S1.p1.1),[§2](https://arxiv.org/html/2609.36126#S2.SS0.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2609.36126#S2.SS0.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2609.36126#S2.SS0.SSS0.Px4.p1.1),[§4\.1](https://arxiv.org/html/2609.36126#S4.SS1.SSS0.Px3.p1.1)\.
- Karsenti \(2008\)E\. KarsentiSelf\-organization in cell biology: a brief history\.Nat\. Rev\. Mol\. Cell Biol\.9\(3\),pp\. 255–262\(en\)\.Cited by:[§1](https://arxiv.org/html/2609.36126#S1.p1.1)\.
- Kimet al\.\(2026\)H\. Kim, E\. Pajouheshgar, S\. Süsstrunk, W\. Jakob, and J\. ParkNeural particle automata: learning self\-organizing particle dynamics\.arXiv \[cs\.NE\]\.Cited by:[§1](https://arxiv.org/html/2609.36126#S1.p2.1)\.
- Laiet al\.\(2026\)J\. Lai, A\. Bao, J\. Quinn, and W\. GilpinFractal basins trap latent reasoning\.arXiv \[cs\.LG\]\.Cited by:[§2](https://arxiv.org/html/2609.36126#S2.SS0.SSS0.Px3.p1.1)\.
- Miyatoet al\.\(2024\)T\. Miyato, S\. Löwe, A\. Geiger, and M\. WellingArtificial kuramoto oscillatory neurons\.arXiv \[cs\.LG\]\.Cited by:[§A\.2](https://arxiv.org/html/2609.36126#A1.SS2.SSS0.Px1.p1.1),[§B\.1](https://arxiv.org/html/2609.36126#A2.SS1.SSS0.Px6.p1.1),[§1](https://arxiv.org/html/2609.36126#S1.p3.1),[§2](https://arxiv.org/html/2609.36126#S2.SS0.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2609.36126#S2.SS0.SSS0.Px3.p1.1),[§3\.1](https://arxiv.org/html/2609.36126#S3.SS1.SSS0.Px2.p1.1),[§4\.1](https://arxiv.org/html/2609.36126#S4.SS1.SSS0.Px1.p1.1),[§4\.1](https://arxiv.org/html/2609.36126#S4.SS1.SSS0.Px3.p1.1)\.
- Mordvintsevet al\.\(2020\)A\. Mordvintsev, E\. Randazzo, E\. Niklasson, and M\. LevinGrowing neural cellular automata\.Distill5\(2\),pp\. e23\.Cited by:[item –](https://arxiv.org/html/2609.36126#A2.I3.ix1.p1.2),[§B\.2](https://arxiv.org/html/2609.36126#A2.SS2.p1.1),[§1](https://arxiv.org/html/2609.36126#S1.p2.1),[§2](https://arxiv.org/html/2609.36126#S2.SS0.SSS0.Px2.p1.1),[§3\.2](https://arxiv.org/html/2609.36126#S3.SS2.SSS0.Px1.p1.1)\.
- Nakagakiet al\.\(2000\)T\. Nakagaki, H\. Yamada, and A\. TóthMaze\-solving by an amoeboid organism\.Nature407\(6803\),pp\. 470\(en\)\.Cited by:[§4\.2](https://arxiv.org/html/2609.36126#S4.SS2.p2.1)\.
- Palmet al\.\(2017\)R\. B\. Palm, U\. Paquet, and O\. WintherRecurrent relational networks\.arXiv \[cs\.AI\]\.Cited by:[§A\.2](https://arxiv.org/html/2609.36126#A1.SS2.SSS0.Px1.p1.1)\.
- Randazzoet al\.\(2020\)E\. Randazzo, A\. Mordvintsev, E\. Niklasson, M\. Levin, and S\. GreydanusSelf\-classifying MNIST digits\.Distill5\(8\),pp\. e00027\.002\(en\)\.Cited by:[§1](https://arxiv.org/html/2609.36126#S1.p2.1)\.
- Reinkeet al\.\(2019\)C\. Reinke, M\. Etcheverry, and P\. OudeyerIntrinsically motivated discovery of diverse patterns in self\-organizing systems\.arXiv \[cs\.LG\]\.Cited by:[§5](https://arxiv.org/html/2609.36126#S5.p2.1)\.
- Sandleret al\.\(2020\)M\. Sandler, A\. Zhmoginov, L\. Luo, A\. Mordvintsev, E\. Randazzo, and B\. A\. y\. ArcasImage segmentation via cellular automata\.arXiv \[cs\.CV\]\.Cited by:[§1](https://arxiv.org/html/2609.36126#S1.p2.1)\.
- Schwarzschildet al\.\(2021\)A\. Schwarzschild, E\. Borgnia, A\. Gupta, F\. Huang, U\. Vishkin, M\. Goldblum, and T\. GoldsteinCan you learn an algorithm? generalizing from easy to hard problems with recurrent networks\.arXiv \[cs\.LG\]\.Cited by:[§1](https://arxiv.org/html/2609.36126#S1.p3.1)\.
- Seelyet al\.\(2026\)J\. Seely, B\. Cupiał, and L\. JonesLearning multi\-agent coordination via sheaf\-ADMM\.arXiv \[cs\.LG\]\.Cited by:[§2](https://arxiv.org/html/2609.36126#S2.SS0.SSS0.Px1.p1.1)\.
- Sghaieret al\.\(2026\)A\. Sghaier, A\. Parviz, and A\. Jolicoeur\-MartineauProbabilistic tiny recursive model\.arXiv \[cs\.AI\]\.Cited by:[§2](https://arxiv.org/html/2609.36126#S2.SS0.SSS0.Px3.p1.1),[§4\.1](https://arxiv.org/html/2609.36126#S4.SS1.SSS0.Px3.p1.1)\.
- Shuet al\.\(2026\)W\. Shu, X\. Qiu, R\. Zhu, H\. H\. Chen, Y\. Liu, and H\. YangLoopViT: scaling visual ARC with looped transformers\.External Links:2602\.02156Cited by:[§A\.3](https://arxiv.org/html/2609.36126#A1.SS3.p1.1),[§B\.3](https://arxiv.org/html/2609.36126#A2.SS3.SSS0.Px1.p1.1),[§B\.3](https://arxiv.org/html/2609.36126#A2.SS3.SSS0.Px2.p1.1),[§B\.3](https://arxiv.org/html/2609.36126#A2.SS3.SSS0.Px3.p1.1),[§B\.3](https://arxiv.org/html/2609.36126#A2.SS3.p1.1),[§1](https://arxiv.org/html/2609.36126#S1.p1.1),[§2](https://arxiv.org/html/2609.36126#S2.SS0.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2609.36126#S2.SS0.SSS0.Px4.p1.1),[§3\.2](https://arxiv.org/html/2609.36126#S3.SS2.SSS0.Px2.p1.1),[§4\.1](https://arxiv.org/html/2609.36126#S4.SS1.SSS0.Px3.p1.1)\.
- Suleymanzadeet al\.\(2026\)A\. Suleymanzade, C\. Lee, F\. Eijkelboom, N\. M\. Boffi, I\. I\. Ceylan, and J\. KimThinking with looped flows\.arXiv \[cs\.LG\]\.Cited by:[§2](https://arxiv.org/html/2609.36126#S2.SS0.SSS0.Px2.p1.1)\.
- Variengienet al\.\(2021\)A\. Variengien, S\. Nichele, T\. Glover, and S\. Pontes\-FilhoTowards self\-organized control: using neural cellular automata to robustly control a cart\-pole agent\.arXiv \[cs\.NE\]\.Cited by:[§1](https://arxiv.org/html/2609.36126#S1.p2.1)\.
- Wanget al\.\(2025\)G\. Wang, J\. Li, Y\. Sun, X\. Chen, C\. Liu, Y\. Wu, M\. Lu, S\. Song, and Y\. A\. YadkoriHierarchical reasoning model\.arXiv \[cs\.AI\]\.Cited by:[§A\.1](https://arxiv.org/html/2609.36126#A1.SS1.SSS0.Px2.p1.1),[§A\.2](https://arxiv.org/html/2609.36126#A1.SS2.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2609.36126#S1.p1.1),[§1](https://arxiv.org/html/2609.36126#S1.p3.1),[§2](https://arxiv.org/html/2609.36126#S2.SS0.SSS0.Px1.p1.1),[§4\.1](https://arxiv.org/html/2609.36126#S4.SS1.SSS0.Px1.p1.1)\.
- Wanget al\.\(2019\)P\. Wang, P\. L\. Donti, B\. Wilder, and Z\. KolterSATNet: bridging deep learning and logical reasoning using a differentiable satisfiability solver\.arXiv \[cs\.LG\]\.Cited by:[§A\.2](https://arxiv.org/html/2609.36126#A1.SS2.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2609.36126#S1.p3.1)\.
- Xuet al\.\(2024\)Y\. Xu, T\. Zhang, and S\. SüsstrunkAdaNCA: neural cellular automata as adaptors for more robust vision transformer\.arXiv \[cs\.CV\]\.Cited by:[§1](https://arxiv.org/html/2609.36126#S1.p2.1)\.

Appendix

[A](https://arxiv.org/html/2609.36126#A1)Benchmarks\.[A](https://arxiv.org/html/2609.36126#A1) [A\.1](https://arxiv.org/html/2609.36126#A1.SS1)Maze\.[A\.1](https://arxiv.org/html/2609.36126#A1.SS1) [A\.2](https://arxiv.org/html/2609.36126#A1.SS2)Sudoku\.[A\.2](https://arxiv.org/html/2609.36126#A1.SS2) [A\.3](https://arxiv.org/html/2609.36126#A1.SS3)ARC\-AGI\-1\.[A\.3](https://arxiv.org/html/2609.36126#A1.SS3) [A\.4](https://arxiv.org/html/2609.36126#A1.SS4)Visual\-Sudoku\.[A\.4](https://arxiv.org/html/2609.36126#A1.SS4) [B](https://arxiv.org/html/2609.36126#A2)Method Details\.[B](https://arxiv.org/html/2609.36126#A2) [B\.1](https://arxiv.org/html/2609.36126#A2.SS1)Architecture\.[B\.1](https://arxiv.org/html/2609.36126#A2.SS1) [B\.2](https://arxiv.org/html/2609.36126#A2.SS2)Training with Sample Replay\.[B\.2](https://arxiv.org/html/2609.36126#A2.SS2) [B\.3](https://arxiv.org/html/2609.36126#A2.SS3)Training with Time Encoding \(ARC\)\.[B\.3](https://arxiv.org/html/2609.36126#A2.SS3) [B\.4](https://arxiv.org/html/2609.36126#A2.SS4)Evaluation Protocol\.[B\.4](https://arxiv.org/html/2609.36126#A2.SS4) [B\.5](https://arxiv.org/html/2609.36126#A2.SS5)FLOPs Estimation\.[B\.5](https://arxiv.org/html/2609.36126#A2.SS5) [B\.6](https://arxiv.org/html/2609.36126#A2.SS6)TTS Pruning Method\.[B\.6](https://arxiv.org/html/2609.36126#A2.SS6) [C](https://arxiv.org/html/2609.36126#A3)Experimental Settings\.[C](https://arxiv.org/html/2609.36126#A3) [C\.1](https://arxiv.org/html/2609.36126#A3.SS1)Maze\.[C\.1](https://arxiv.org/html/2609.36126#A3.SS1) [C\.2](https://arxiv.org/html/2609.36126#A3.SS2)Sudoku\.[C\.2](https://arxiv.org/html/2609.36126#A3.SS2) [C\.3](https://arxiv.org/html/2609.36126#A3.SS3)ARC\-AGI\-1\.[C\.3](https://arxiv.org/html/2609.36126#A3.SS3) [C\.4](https://arxiv.org/html/2609.36126#A3.SS4)Visual\-Sudoku\.[C\.4](https://arxiv.org/html/2609.36126#A3.SS4) [D](https://arxiv.org/html/2609.36126#A4)Additional Results\.[D](https://arxiv.org/html/2609.36126#A4) [D\.1](https://arxiv.org/html/2609.36126#A4.SS1)Fixed Attention Analysis\.[D\.1](https://arxiv.org/html/2609.36126#A4.SS1) [D\.2](https://arxiv.org/html/2609.36126#A4.SS2)Impact of the Locality Bottleneck\.[D\.2](https://arxiv.org/html/2609.36126#A4.SS2) [D\.3](https://arxiv.org/html/2609.36126#A4.SS3)Extended Test\-Time Scaling Results\.[D\.3](https://arxiv.org/html/2609.36126#A4.SS3) [D\.4](https://arxiv.org/html/2609.36126#A4.SS4)Extended TTS Pruning Results\.[D\.4](https://arxiv.org/html/2609.36126#A4.SS4)

## Appendix ABenchmarks

### A\.1Maze

We first evaluate the NCA’spath\-findingabilities on two maze benchmarks: theMaze\-OODbenchmark to test generalization to out\-of\-distribution maze sizes, and theMaze\-Hardbenchmark to test on hard mazes with multiple solutions\.

##### Maze\-OOD\.

Dataset from[Bansal et al\. \(2022\)](https://arxiv.org/html/2609.36126#bib.bib14)generated using theeasy\-to\-hardpython package data, which tests out\-of\-distribution spatial generalization\. The training dataset contains 50K examples of small9×99\\times 9grids \(each with a unique solution\)\. The test set contains larger mazes of sizes ranging from9×99\\times 9to37×3737\\times 37\(in increments of 2, 10K each\), as well as extreme sizes59×5959\\times 59\(10K\) and201×201201\\times 201\(1K\) with very long dead\-ends and deceptive paths\. The vocabulary consists ofV=4V=4tokens: empty cell \(00\), solution path \(11\), wall \(22\), and endpoints \(33\)\. Inputs specify wall and endpoint locations, and the NCA must complete the rest by predicting either empty or solution path\. Training uses 8\-way dihedral data augmentation \(rotations and reflections\)\.

![Refer to caption](https://arxiv.org/html/2609.36126v1/figures/figure_mhard_illustration.png)Figure 10:Maze\-Hard task ambiguity\. While trained on A\* targets \(exact, left\), there are multiple othervalid\(middle, blue\) andoptimal\(right, green\) solutions\.
##### Maze\-Hard\.

Dataset from[Wang et al\. \(2025\)](https://arxiv.org/html/2609.36126#bib.bib11)that tests the ability of a model to solve hard30×3030\\times 30mazes requiring solution paths of at least length 110\. Unlike Maze\-OOD, these mazes frequently contain multiple valid and optimal paths, introducing ambiguity against the single referenceA∗\\text\{A\}^\{\*\}solution \(Figure[10](https://arxiv.org/html/2609.36126#A1.F10)\)\. The dataset contains 1000 training samples and 1000 test samples, with targets generated by theA∗A^\{\*\}algorithm\. The vocabulary consists ofV=5V=5tokens: empty cell \(00\), solution path \(11\), wall \(22\), start \(33\), and goal \(44\) endpoints\. Start and goal endpoints are explicitly distinguished this time because A\* solutions \(which this benchmark evaluates against\) is asymmetric under start\-goal swapping\. Note that given this dataset models are trained to matchA∗A^\{\*\}solutions as opposed to learninganyoptimal policy\. We include this benchmark, in part, to test whether NCAs can learnA∗A^\{\*\}\-like solutions via purely local dynamics, despite cells lacking the global distance\-to\-goal heuristic used inA∗A^\{\*\}\. We also compare with training on the same dataset but with BFS\-generated targets, with results also reported in Table[1](https://arxiv.org/html/2609.36126#S4.T1)\. Data augmentation is disabled forA∗A^\{\*\}\(as it makes the supervised targets inconsistent\) but used for BFS \(with 8 dihedral transformations computed on the fly\)\. We assess accuracy of the solutions based on whether the generated path isvalid\(single continuous path which connects start and goal without crossing walls or branching\),optimal\(valid path with minimal length\) orexact\(matches the providedA∗A^\{\*\}/BFS solution\)\.

### A\.2Sudoku

We then evaluate the ability of NCAs to solve adistributed constraint satisfaction problem\(DisCSP\) on two Sudoku benchmarks: theSudoku\-OODbenchmark to test generalization to out\-of\-distribution Sudoku difficulties, and theSudoku\-Extremebenchmark to test on very hard Sudoku boards that require extensive“guesses”and“backtracks”to be solved\.

![Refer to caption](https://arxiv.org/html/2609.36126v1/figures/figure_sood_illustration.png)Figure 11:Sudoku\-OOD dataset: the test set \(47–64 cells to fill\) presents a severe OOD shift relative to training \(39–50 cells to fill\)##### Sudoku\-OOD\.

Benchmark by[Miyato et al\. \(2024\)](https://arxiv.org/html/2609.36126#bib.bib15)to test sudoku solving and out\-of\-distribution difficulty generalization\. The training set contains 9K easy boards \(and 1K validation boards\) which were used in the SAT\-Net paper\([Wang et al\., 2019](https://arxiv.org/html/2609.36126#bib.bib18)\)\. The test set contains harder boards across 18 difficulty levels \(1K examples per level\) which were used in the RRN paper\([Palm et al\., 2017](https://arxiv.org/html/2609.36126#bib.bib17)\), with a severe OOD shift relative to training \(Figure[11](https://arxiv.org/html/2609.36126#A1.F11)\)\. Difficulty here is defined by the number of empty cells, ranging from 47 to 64 empty cells\. The vocabulary consists ofV=10V=10tokens: empty cell \(00\) and digits1​–​91\\text\{\-\-\}9\. Input cells specify given prefilled digits, and the NCA must complete the board by predicting digits for all remaining empty cells\. Training uses Sudoku\-invariant symmetry data augmentation: transposition, random permutation of digits1​–​91\\text\{\-\-\}9, and random permutations of rows/columns within3×33\\times 3blocks and of the blocks themselves\.

![Refer to caption](https://arxiv.org/html/2609.36126v1/figures/figure_sext_illustration.png)Figure 12:Sudoku Extreme Difficulty\. \(A\) We group test samples in bins of increasing difficulties to plot results across difficulties in Figure[16](https://arxiv.org/html/2609.36126#A4.F16)\. \(B\) Statistics of difficulties\.
##### Sudoku\-Extreme\.

Benchmark by[Wang et al\. \(2025\)](https://arxiv.org/html/2609.36126#bib.bib11)to test sudoku solving on extremely challenging boards requiring extensive search and backtracking\. The training set contains only 1000 examples, and the test set contains 422,780 boards with difficulty scores ranging from 0 to 289, defined here as the number of backtracks needed by the logic\-basedtdokusolver333[https://t\-dillon\.github\.io/tdoku/](https://t-dillon.github.io/tdoku/)\(Figure[12](https://arxiv.org/html/2609.36126#A1.F12)\)\. Difficulties, i\.e\. numbers of backtracks needed to solve these boards, are significantly higher than the ones of other Sudoku datasets used in the literature\([Wang et al\., 2025](https://arxiv.org/html/2609.36126#bib.bib11)\)\. As in Sudoku\-OOD, the vocabulary consists ofV=10V=10tokens \(00for empty cells,1​–​91\\text\{\-\-\}9for digits\) and training uses the same data augmentations\.

### A\.3ARC\-AGI\-1

In order to investigate whether conditioning enables a single NCA model to generalize across multiple tasks, we evaluate NCAs on theAbstraction and Reasoning Corpus\(ARC\-AGI\-1,[Chollet \(2019\)](https://arxiv.org/html/2609.36126#bib.bib3)\), a benchmark designed for few\-shot visual reasoning\. Each task consists of a small set of input\-output grid demonstrations \(typically 2–5 examples\) requiring the model to infer an underlying transformation rule and apply it to a novel test input\. Following the training pipeline of recent works\([Hu et al\., 2025](https://arxiv.org/html/2609.36126#bib.bib4);[Shu et al\., 2026](https://arxiv.org/html/2609.36126#bib.bib2)\), we complement the original training set with synthetic tasks generated by RE\-ARC, which expands the set of available input\-output pairs for the training tasks while preserving the core visual logic\.

### A\.4Visual Sudoku

In Visual Sudoku, we investigate whether NCAs can be scaled to more complex problems on a much larger grids and performreasoning in raw pixel space\. Visual\-Sudoku is purely an image\-to\-image task, requiring to parse raw image pixels, classify the given digits, solve the DisCSP implementing the Sudoku, and then render an image representation of the correct digit as output\. More specifically, Visual\-Sudoku is a dataset derived from theeasysubset of Sudoku\-OOD, rendering each puzzle pair as a pair ofimageswhere each digit is represented as an independently randomly sampled28×2828\\times 28MNIST image from the corresponding class\. In other words, a11Sudoku clue is represented by randomly sampling an MNIST image of a11\. Since each digit image is sampled independently, a11in one part of the board appears differently from a11in another, and likewise a55given as an input clue will appear differently from the corresponding55in the same position in the solved image\. Thus, the pair of images will be one image for the unsolved state with clues, and one image for the solved state with all digits filled\. This yields a256×256256\\times 256input image \(adding 4 pixels of zero\-padding to each side\), with no additional semantic information provided\.

The training set uses the9,0009,000puzzles from theeasyset of Sudoku\-OOD, and we test on both the in\-distributioneasytest\-set \(10001000puzzles\) and thehardtest\-set \(10001000puzzles sampled randomly\)\. The digit images during training are sampled from the MNISTtestset, and the digit images during testing are sampled from the MNISTtrainset \(accidental inversion which we left as is, since both subsets remain strictly disjoint\)\. To check Sudoku correctness at inference, we read out predictions using a non\-learned classifier, which simply computes the pixel\-wiseL2L\_\{2\}distance between each28×2828\\times 28output patch and the mean image of each MNIST digit class, and predicts the closest class\.

## Appendix BMethod

This section describes the model architecture, training procedure and test\-time scaling evaluation protocol\.

### B\.1Architecture

The NCA operates on a 2D grid of sizeH×WH\\times W\. Each grid cell at spatial coordinates\(i,j\)\(i,j\)contains a continuous state vectorxi,j∈ℝCx\_\{i,j\}\\in\\mathbb\{R\}^\{C\}, whereCCis the total number of channels\. The overall grid state at discrete time stepttis denoted byX\(t\)∈ℝH×W×CX^\{\(t\)\}\\in\\mathbb\{R\}^\{H\\times W\\times C\}\.

##### Task Embedding\.

To project the discrete task representation \(2D grid of tokens \+ task id for ARC\) into the continuous latent space of the NCA we construct the following embeddings:

- –Token Embedding:Each discrete tokenv∈\{0,1,…,V−1\}v\\in\\\{0,1,\\dots,V\-1\\\}\(whereVVis the vocabulary size\) is represented by aCinC\_\{\\text\{in\}\}\-dimensional vectorev∈ℝCine\_\{v\}\\in\\mathbb\{R\}^\{C\_\{\\text\{in\}\}\}\. These embeddings are fixed and predefined as orthonormal vectors for Maze, Sudoku, and ARC\. For Visual Sudoku we directly use pixel values \(Cin=1C\_\{\\text\{in\}\}=1\)\.
- –Task Embedding \(ARC\):a global task embeddingetask∈ℝCtaske\_\{\\text\{task\}\}\\in\\mathbb\{R\}^\{C\_\{\\text\{task\}\}\}is learned to condition the initialization state of NCA cells on the underlying task\.

##### State Initialization\.

The initial continuous grid stateX\(0\)∈ℝH×W×CX^\{\(0\)\}\\in\\mathbb\{R\}^\{H\\times W\\times C\}is constructed as follows:

- –Input Cells:For cells where tokens are provided in the initial board \(e\.g\., walls in Maze or given digits in Sudoku\), the firstCinC\_\{\\text\{in\}\}channels are set to the corresponding token embeddingeve\_\{v\}, while the remaining channels are initialized with Gaussian noise𝒩⁡\(0,σinit2\)\\mathcal\{N\}\(0,\\sigma\_\{\\text\{init\}\}^\{2\}\)\. During rollout, theCinC\_\{\\text\{in\}\}channels remain strictly fixed to their initial token embeddings \(Δ​xi,j,in\(t\)=0\\Delta x\_\{i,j,\\text\{in\}\}^\{\(t\)\}=0\), acting as immutable elements\.
- –Empty/Unsolved Cells:For empty cells, allCCstate channels are initialized with Gaussian noise𝒩⁡\(0,σinit2\)\\mathcal\{N\}\(0,\\sigma\_\{\\text\{init\}\}^\{2\}\)\.
- –Task Conditioning \(ARC\):For ARC, the global task embeddingetaske\_\{\\text\{task\}\}is injected into a dedicated channel slice of every cell across the grid\.
- –Input and Output padding \(ARC\):To handle variable task dimensions, grids are padded into a32×3232\\times 32canvas using a new padding token which is explicitly masked out of the loss function\.

##### Perception Module\.

At every steptt, each cell\(i,j\)\(i,j\)inspects its local neighborhood𝒩⁡\(i,j\)\\mathcal\{N\}\(i,j\)\(a3×33\\times 3Moore neighborhood containing 9 cells\) to extract a perception vectorzi,j\(t\)z\_\{i,j\}^\{\(t\)\}:

- –Convolutional Sensing \(Maze, Visual Sudoku\):A set ofKheadsK\_\{\\text\{heads\}\}3×33\\times 3convolution kernels\{Wperc\(k\)\}k=1Kheads\\\{W\_\{\\text\{perc\}\}^\{\(k\)\}\\\}\_\{k=1\}^\{K\_\{\\text\{heads\}\}\}is applied to the neighbor states\. For each headkk, the 9 neighbor states are linearly combined: zi,j\(k\)=∑n=19Wperc,n\(k\)​xn\(t\)∈ℝCz\_\{i,j\}^\{\(k\)\}=\\sum\_\{n=1\}^\{9\}W\_\{\\text\{perc\},n\}^\{\(k\)\}x\_\{n\}^\{\(t\)\}\\in\\mathbb\{R\}^\{C\}wherexn\(t\)∈𝒩⁡\(i,j\)x\_\{n\}^\{\(t\)\}\\in\\mathcal\{N\}\(i,j\)\. Concatenating allKheadsK\_\{\\text\{heads\}\}outputs produces the perception vectorzi,j∈ℝKheads⋅Cz\_\{i,j\}\\in\\mathbb\{R\}^\{K\_\{\\text\{heads\}\}\\cdot C\}\. For Maze, these kernels are learned\. For Visual\-Sudoku, they are kept fixed as the identity, vertical, and horizontal Sobel filters as in[Mordvintsev et al\. \(2020\)](https://arxiv.org/html/2609.36126#bib.bib1)\.
- –Fixed\-Attention Sensing \(Sudoku, ARC\):A set ofKheadsK\_\{\\text\{heads\}\}position\-specific3×33\\times 3attention matrices\{Ai,j\(k\)\}k=1Kheads\\\{A\_\{i,j\}^\{\(k\)\}\\\}\_\{k=1\}^\{K\_\{\\text\{heads\}\}\}and value projection matrices\{V\(k\)\}k=1Kheads\\\{V^\{\(k\)\}\\\}\_\{k=1\}^\{K\_\{\\text\{heads\}\}\}\(whereV\(k\)∈ℝCKheads×CV^\{\(k\)\}\\in\\mathbb\{R\}^\{\\frac\{C\}\{K\_\{\\text\{heads\}\}\}\\times C\}\) is applied to the neighbor states\. Each headkkprojects and linearly combines neighbor states: zi,j\(k\)=∑n=19Ai,j,n\(k\)​\(V\(k\)​xn\(t\)\)∈ℝCKheadsz\_\{i,j\}^\{\(k\)\}=\\sum\_\{n=1\}^\{9\}A\_\{i,j,n\}^\{\(k\)\}\\left\(V^\{\(k\)\}x\_\{n\}^\{\(t\)\}\\right\)\\in\\mathbb\{R\}^\{\\frac\{C\}\{K\_\{\\text\{heads\}\}\}\}wherexn\(t\)∈𝒩⁡\(i,j\)x\_\{n\}^\{\(t\)\}\\in\\mathcal\{N\}\(i,j\)\. Concatenating allKheadsK\_\{\\text\{heads\}\}outputs produces the perception vectorzi,j∈ℝCz\_\{i,j\}\\in\\mathbb\{R\}^\{C\}\.

##### Update Module\.

The update module is shared across all cells and maps the perception vectorzi,j\(t\)z\_\{i,j\}^\{\(t\)\}to a proposed state updateΔ​xi,j\(t\)\\Delta x\_\{i,j\}^\{\(t\)\}\.

- –MLP \(Maze, Sudoku, Visual Sudoku\):A standard two\-layer MLP withReLU\\operatorname\{ReLU\}activation: Δ​xi,j\(t\)=W2​ReLU⁡\(W1​zi,j\(t\)\+b1\)\+b2\\Delta x\_\{i,j\}^\{\(t\)\}=W\_\{2\}\\operatorname\{ReLU\}\\left\(W\_\{1\}z\_\{i,j\}^\{\(t\)\}\+b\_\{1\}\\right\)\+b\_\{2\}
- –SwiGLU \(ARC\):A gated activation layer that splits the projected representation into gateggand valueuu:=W1​zi,j\(t\)\+b1,Δ​xi,j\(t\)=W2​\(swish⁡\(g\)⊙u\)\+b2\\begin\{aligned\} &=W\_\{1\}z\_\{i,j\}^\{\(t\)\}\+b\_\{1\},\\\\ \\Delta x\_\{i,j\}^\{\(t\)\}&=W\_\{2\}\\left\(\\operatorname\{swish\}\(g\)\\odot u\\right\)\+b\_\{2\}\\end\{aligned\}whereswish⁡\(g\)=g⋅σ⁡\(g\)\\operatorname\{swish\}\(g\)=g\\cdot\\sigma\(g\), and⊙\\odotdenotes element\-wise multiplication\.

W1W\_\{1\}expands the perception vector by an expansion factorEE, andW2W\_\{2\}projects back toCCchannels\.

##### Asynchronous Stochastic Execution\.

To simulate asynchronous cellular behavior, cells update stochastically according to a binary maskmi,j\(t\)∈\{0,1\}m\_\{i,j\}^\{\(t\)\}\\in\\\{0,1\\\}\. Ifmi,j\(t\)=0m\_\{i,j\}^\{\(t\)\}=0, cell\(i,j\)\(i,j\)does not update at steptt\. We consider two firing strategies:

- –Uniform Firing \(Default\):Cells fire with a constant cell fire ratepfirep\_\{\\text\{fire\}\}: mi,j\(t\)∼Bernoulli⁡\(pfire\)m\_\{i,j\}^\{\(t\)\}\\sim\\operatorname\{Bernoulli\}\(p\_\{\\text\{fire\}\}\)
- –Adaptive Firing \(Section[4\.4](https://arxiv.org/html/2609.36126#S4.SS4)\):The fire rate drops toplow<pfirep\_\{\\text\{low\}\}<p\_\{\\text\{fire\}\}once a cell’s confidenceci,j\(t\)c\_\{i,j\}^\{\(t\)\}exceeds thresholdτhalt\\tau\_\{\\text\{halt\}\}: mi,j\(t\)∼Bernoulli⁡\(\{plowif​ci,j\(t\)\>τhalt,pfireotherwise\)m\_\{i,j\}^\{\(t\)\}\\sim\\operatorname\{Bernoulli\}\\left\(\\begin\{cases\}p\_\{\\text\{low\}\}&\\text\{if \}c\_\{i,j\}^\{\(t\)\}\>\\tau\_\{\\text\{halt\}\},\\\\ p\_\{\\text\{fire\}\}&\\text\{otherwise\}\\end\{cases\}\\right\)

##### Residual State Update\.

The cell state is updated via a residual connection:xi,j\(t\+1\)=xi,j\(t\)\+mi,j\(t\)⋅Δ​xi,j\(t\)x\_\{i,j\}^\{\(t\+1\)\}=x\_\{i,j\}^\{\(t\)\}\+m\_\{i,j\}^\{\(t\)\}\\cdot\\Delta x\_\{i,j\}^\{\(t\)\}\. For Sudoku and ARC, we also useper\-cell group normalizationfollowing[Miyato et al\. \(2024\)](https://arxiv.org/html/2609.36126#bib.bib15): cell states are seen asoscillators, i\.e\.UUindependent unit vectors of sizeC/UC/Uthat rotate on a sphere, and each unit is re\-normalized after each update to unitL2L\_\{2\}norm\.

##### Prediction Head\.

At any steptt, the predicted tokenv^i,j\\hat\{v\}\_\{i,j\}of cell\(i,j\)\(i,j\)is inferred from itsCoutC\_\{\\text\{out\}\}\-dimensional readout slicexi,j,outx\_\{i,j,\\text\{out\}\}by computing a scoreℓi,j​\(v\)\\ell\_\{i,j\}\(v\)for each candidate tokenvv:

- –L2L\_\{2\}Similarity \(Maze\):Computes the inverse Euclidean distance between the cell readout slice and token embeddings: ℓi,j​\(v\)=11\+‖xi,j,out−ev‖2\\ell\_\{i,j\}\(v\)=\\frac\{1\}\{1\+\\\|x\_\{i,j,\\text\{out\}\}\-e\_\{v\}\\\|\_\{2\}\}
- –Cosine Similarity \(Sudoku\):Computes the normalized dot product with token embeddings: ℓi,j​\(v\)=xi,j,out⋅ev‖xi,j,out‖2​‖ev‖2\\ell\_\{i,j\}\(v\)=\\frac\{x\_\{i,j,\\text\{out\}\}\\cdot e\_\{v\}\}\{\\\|x\_\{i,j,\\text\{out\}\}\\\|\_\{2\}\\\|e\_\{v\}\\\|\_\{2\}\}
- –MLP Head \(ARC\):Projects the readout slice to class logits via a learned linear transformation: ℓi,j​\(v\)=\(Wpred​xi,j,out\+bpred\)v\\ell\_\{i,j\}\(v\)=\\left\(W\_\{\\text\{pred\}\}x\_\{i,j,\\text\{out\}\}\+b\_\{\\text\{pred\}\}\\right\)\_\{v\}
- –Parameter\-free classifier \(Visual Sudoku\):The output image patch for Sudoku cell\(i,j\)\(i,j\)isIi,j=X28​i:28​\(i\+1\),28​j:28​\(j\+1\),outI\_\{i,j\}=X\_\{28i:28\(i\+1\),\\,28j:28\(j\+1\),\\,\\text\{out\}\}\. We compute predictions against empirical class\-mean prototypesPv=1Nv​∑k=1NvIv\(k\)P\_\{v\}=\\frac\{1\}\{N\_\{v\}\}\\sum\_\{k=1\}^\{N\_\{v\}\}I\_\{v\}^\{\(k\)\}, computed across the MNIST test set\. Thus, score is: ℓi,j​\(v\)=−‖Ii,j−Pv‖F2\\ell\_\{i,j\}\(v\)=\-\\\|I\_\{i,j\}\-P\_\{v\}\\\|\_\{F\}^\{2\}

The predicted token isv^i,j=arg⁡maxv​ℓi,j​\(v\)\\hat\{v\}\_\{i,j\}=\\arg\\max\_\{v\}\\ell\_\{i,j\}\(v\)\.

##### Confidence Head\.

A per\-cell scalar confidenceci,j∈\[0,1\]c\_\{i,j\}\\in\[0,1\]can also be extracted at inference at any steptt, using the maximum score across all candidate tokens:

ci,j=maxv⁡ℓi,j​\(v\)c\_\{i,j\}=\\max\_\{v\}\\ell\_\{i,j\}\(v\)The global board confidence is the mean cell confidence across the grid:

c⁡\(X\)=1H⋅W​∑i=1H∑j=1Wci,jc\(X\)=\\frac\{1\}\{H\\cdot W\}\\sum\_\{i=1\}^\{H\}\\sum\_\{j=1\}^\{W\}c\_\{i,j\}

##### Loss Function\.

- –Mean Squared Error \(Maze, Sudoku, Visual Sudoku\):Measures the distance between the cell readout slice and target token embeddings: ℒMSE=1\|𝒲\|⋅H⋅W​∑t∈𝒲∑i=1H∑j=1W‖xi,j,out\(t\)−eyi,j‖22\\mathcal\{L\}\_\{\\text\{MSE\}\}=\\frac\{1\}\{\|\\mathcal\{W\}\|\\cdot H\\cdot W\}\\sum\_\{t\\in\\mathcal\{W\}\}\\sum\_\{i=1\}^\{H\}\\sum\_\{j=1\}^\{W\}\\left\\\|x\_\{i,j,\\text\{out\}\}^\{\(t\)\}\-e\_\{y\_\{i,j\}\}\\right\\\|\_\{2\}^\{2\}
- –Cross\-Entropy \(ARC\):Standard categorical cross\-entropy computed on the predicted class logitsℓi,j\(t\)\\ell\_\{i,j\}^\{\(t\)\}against target tokens: ℒCE=−1\|𝒲\|⋅H⋅W∑t∈𝒲∑i=1H∑j=1Wlogsoftmax\(ℓi,j\(t\)\)yi,j\\mathcal\{L\}\_\{\\text\{CE\}\}=\-\\frac\{1\}\{\|\\mathcal\{W\}\|\\cdot H\\cdot W\}\\sum\_\{t\\in\\mathcal\{W\}\}\\sum\_\{i=1\}^\{H\}\\sum\_\{j=1\}^\{W\}\\log\\operatorname\{softmax\}\\left\(\\ell\_\{i,j\}^\{\(t\)\}\\right\)\_\{y\_\{i,j\}\}

The set of steps𝒲\\mathcal\{W\}that we compare with target within the chunk are either:

- –All\-step supervision \(Maze, Sudoku\-OOD, Visual Sudoku\): The loss is averaged across allNchunkN\_\{\\text\{chunk\}\}steps of the sub\-window \(\|𝒲\|=Nchunk\|\\mathcal\{W\}\|=N\_\{\\text\{chunk\}\}\), providing stronger gradient signal but penalizing intermediate states along the trajectory\.
- –Window\-step supervision \(ARC\): The loss is evaluated across the finalwwsteps of the chunk \(1<\|𝒲\|=w<Nchunk1<\|\\mathcal\{W\}\|=w<N\_\{\\text\{chunk\}\}\), allowing early exploration while reinforcing stable convergence over the final trajectory\.
- –Last\-step Supervision \(Sudoku\-Extreme\): The loss is evaluated only at the final step of the chunk \(\|𝒲\|=1\|\\mathcal\{W\}\|=1\), giving intermediate steps more freedom to explore solution space\.

### B\.2Training with Sample Replay

Following the original NCA training methodology\([Mordvintsev et al\., 2020](https://arxiv.org/html/2609.36126#bib.bib1)\), we train our models using Backpropagation Through Time \(BPTT\) combined with a buffer of previously\-inferred samples as well as training perturbations\. The use of the replay buffer stabilizes long\-horizon recurrent dynamics by emulating long execution trajectories without incurring the memory footprint of backpropagating through thousands of steps\.

##### Training pipeline\.

A fixed\-capacity buffer maintainsMMstate grids initialized with fresh board states \(age 0\)\. At each training iteration:

1. –A batch ofBBstates is drawn uniformly at random from the sample replay buffer\.
2. –A fractionrseedr\_\{\\text\{seed\}\}of the sampled batch is replaced with fresh initial states \(age 0\)\.
3. –Perturbations \(target swapping, state noise, and damage\) are applied to the batch\.
4. –The batch is unrolled forNNsteps using the current NCA parameters\.
5. –A random sub\-window of lengthNchunkN\_\{\\text\{chunk\}\}is selected\.
6. –The loss is evaluated across the subwindow, gradients are computed with truncated BPTT on the chunk, and the NCA parameters are updated\.
7. –The final post\-rollout states and their incremented ages are written back into the pool\.

##### Training Perturbations\.

Three types of perturbations are applied during training:

- –State Noise \(During Rollout\):At each rollout step, Gaussian noise𝒩⁡\(0,σnoise2\)\\mathcal\{N\}\(0,\\sigma\_\{\\text\{noise\}\}^\{2\}\)is injected into cell states with temporal probabilityptp\_\{t\}and spatial probabilitypsp\_\{s\}\(excluding the frozen input channels of input cells\)\.
- –State Damage \(Pre\-Rollout\):With probabilitypdamagep\_\{\\text\{damage\}\},NdamageN\_\{\\text\{damage\}\}circular masks with radiir∈\[0\.1,0\.4\]r\\in\[0\.1,0\.4\]\(normalized to grid size\) reset cell channels to zero\.
- –Target Swapping \(Pre\-Rollout\):With probabilitypswapp\_\{\\text\{swap\}\}, a sample’s input clues and target solution are replaced with those of another board from the dataset, forcing the NCA to constantly sense its environment and dynamically adapt its internal states toward the new task when such swap occurs\.

### B\.3Training with time encoding \(ARC\)

Following prior work\([Shu et al\., 2026](https://arxiv.org/html/2609.36126#bib.bib2)\), we condition model execution by injecting step\-based temporal encodings\. We hypothesize that solving ARC tasks requires multi\-phase computation and that this temporal signal helps this staged reasoning\. At steptt, the step embeddingestep\(t\)e\_\{\\text\{step\}\}^\{\(t\)\}is learned and broadcast across the grid and added directly to the hidden state slice of every cell prior to neighborhood sensing:

xi,j,hid\(t\)←xi,j,hid\(t\)\+estep\(t\),x\_\{i,j,\\text\{hid\}\}^\{\(t\)\}\\leftarrow x\_\{i,j,\\text\{hid\}\}^\{\(t\)\}\+e\_\{\\text\{step\}\}^\{\(t\)\},\(1\)where the time\-encodingestep\(t\)e\_\{\\text\{step\}\}^\{\(t\)\}is a vector carrying no spatial coordinate or relational information\.

At inference the global clock becomes an internal*local clock*: each cell indexes the step embedding by the number of steps in which it has itself fired, keeping its state and counter frozen otherwise\.

##### Canvas and online augmentation\.

All grids are padded on a fixed32×3232\\times 32canvas\([Shu et al\., 2026](https://arxiv.org/html/2609.36126#bib.bib2)\)\. At every optimization step the original input–target pair is rescaled by a random integer and placed at a random offset in the canvas, where a new border token marks the extent of the target grid\. The rest of the canvas is filled with pad tokens that are masked out of the loss\. Thisonlineaugmentation is identical in both the pre\-training and the fine\-tuning, so each pair is seen at a new scale and position every time it is sampled\.

##### Pre\-training\.

Following LoopViT\([Shu et al\., 2026](https://arxiv.org/html/2609.36126#bib.bib2)\), we pre\-train on training tasks augmented with the pairs from the RE\-ARC dataset\([Hodel, 2024](https://arxiv.org/html/2609.36126#bib.bib9)\)\. At each training step we proceed with the following pipeline: 1\) sample a batch of input–target pairs; 2\) render them on the canvas with fresh random scale and offset; 3\) rolloutNNsteps and perform a training step\.

##### Test\-time training \(per task\)\.

For each evaluation task we build5151augmentations \(offlineaugmentations\): the original plus five dihedral transforms, each with ten color permutations\. The augmentation pipeline was built on previous work\([Shu et al\., 2026](https://arxiv.org/html/2609.36126#bib.bib2);[Hu et al\., 2025](https://arxiv.org/html/2609.36126#bib.bib4)\)\. Every augmented variant receives its own task id, whose embedding is randomly initialized, while all remaining weights are loaded from the pre\-trained model\. We then fine\-tune with the same loop as pre\-training, sampling batches from the demonstration pairs pooled across the5151variants\. The task embeddings and the model weights are updated jointly\.

### B\.4Evaluation Protocol

At test time, inference consists of rolling out the trained NCA forDDiterations from an initial stateX\(0\)X^\{\(0\)\}initialized with Gaussian noise of scaleσinit\\sigma\_\{\\text\{init\}\}\. For test\-time scaling experiments \(section[4\.3](https://arxiv.org/html/2609.36126#S4.SS3)\), we scale computation along two axes: increasing the rollout horizonDD\(temporal scaling\) and launchingKKparallel rollouts with an automated selection mechanism \(parallel trials scaling\)\. Note that for Maze\-OOD depth scaling is combined with scaling substrate sizeSS\(spatial substrate scaling\)\.

##### Depth Scaling \(DD\)\.

The model is unrolled forDDsteps\. Because information propagates locally at each step, scaling the iteration budgetDDallows signals to traverse larger boards and perform iterative reasoning to solve more complicated tasks than the ones seen during training\.

##### Width Scaling \(KK\)\.

To explore diverse solution trajectories for a given board, we runKKparallel rollouts starting from different initial states and running with stochastic asynchronous updates \(pfire<1p\_\{\\text\{fire\}\}<1\):

- –Stochastic Initialization \(Maze, Sudoku\):Each of theKKtrials is initialized with independent Gaussian noise𝒩⁡\(0,σinit2\)\\mathcal\{N\}\(0,\\sigma\_\{\\text\{init\}\}^\{2\}\)\.
- –Augmented Ensembles \(ARC\):For each test image we aggregate predictions over a budget ofK=E×A×RK=E\\times A\\times Rrollouts: - –Offline augmentations \(AA*task augmented variants*\): these are the same variants used for test\-time training, so each test image is evaluated with its own fine\-tuned task embedding; - –Online augmentations \-RRstochastic rollouts per variant, which differ through the random hidden\-state initialization, the asynchronous cell\-firing mask, and a fresh random scaling and translation of the test input on the32×3232\\times 32canvas drawn independently for every rollout; - –Model ensemble \-EEindependently fine\-tuned models generateA×RA\\times Rpredictions which are aggregated in the same pool\. Every generated image is mapped back to the canonical frame, creating a set ofKKcandidate predictions, the most\-voted grids form the two attempts pass@1 and pass@2\.

##### Test\-Time Noise Injection \(Maze, Sudoku\)\.

We additionally inject noise during exploration, parametrized byΘnoise=\(rnoise,pt,ps,σ\)\\Theta\_\{\\text\{noise\}\}=\(r\_\{\\text\{noise\}\},p\_\{t\},p\_\{s\},\\sigma\), where:

- –rnoiser\_\{\\text\{noise\}\}is the active noise duration \(typically the initial fraction of the rollout\), after which noise is disabled;
- –ptp\_\{t\}andpsp\_\{s\}are the temporal and spatial Bernoulli probabilities of noise injection;
- –σ\\sigmais the standard deviation of the injected Gaussian noise𝒩⁡\(0,σ2\)\\mathcal\{N\}\(0,\\sigma^\{2\}\)\.

##### Final Selection\.

GivenKKcandidate solutions\{Y^k\}k=1K\\\{\\hat\{Y\}\_\{k\}\\\}\_\{k=1\}^\{K\}, we use these aggregation strategies:

- –Conf@K \(Maze\-Hard, Sudoku\):Without access to ground truthYY, the model autonomously selects the candidate with the highest board confidence: k∗=arg⁡maxk∈\{1,…,K\}⁡c⁡\(Xk\(D\)\),Conf@K=𝕀⁡\(Y^k∗=Y\)\.k^\{\*\}=\\arg\\max\_\{k\\in\\\{1,\\dots,K\\\}\}c\\left\(X\_\{k\}^\{\(D\)\}\\right\),\\text\{Conf@K\}=\\mathbb\{I\}\\left\(\\hat\{Y\}\_\{k^\{\*\}\}=Y\\right\)\.
- –Pass@M via Majority Voting \(ARC\):From theKKparallel rollouts, we select theMMmost frequently predicted unique board configurations𝒮M=\{Y^\(1\),…,Y^\(M\)\}\\mathcal\{S\}\_\{M\}=\\\{\\hat\{Y\}\_\{\(1\)\},\\dots,\\hat\{Y\}\_\{\(M\)\}\\\}\. The prediction is considered successful if any of theMMsubmissions matches the ground truth: Pass@M=𝕀\(∃Y^∈𝒮M:Y^=Y∗\)\.\\text\{Pass@M\}=\\mathbb\{I\}\\left\(\\exists\\hat\{Y\}\\in\\mathcal\{S\}\_\{M\}:\\hat\{Y\}=Y^\{\*\}\\right\)\.

### B\.5FLOPs Estimation

##### Method\.

The floating\-point operations \(FLOPs\) reported in the main paper correspond to inference on asingleboard of sizeSS, accounting the recurrence depthDDand the number of parallel trialsKKused for test\-time scaling\.

For PyTorch\-based baselines \(DeepThink,AKOrN,\(P\)TRM,HRM, andLoopViT\), models were instantiated directly from their official codebase repositories using published configurations and evaluation settings \(S,D,KS,D,K\)\. Single\-step FLOPs were measured using PyTorch’sFlopCounterMode\. For allNCAmodels, implemented in JAX with configurations detailed in section[C](https://arxiv.org/html/2609.36126#A3), single\-step FLOPs were extracted from the compiled computation graph using the XLA cost analysis interface on CPU\.

The total test\-time compute is then calculated asFLOPstotal=FLOPsstep×D×K\\text\{FLOPs\}\_\{\\text\{total\}\}=\\text\{FLOPs\}\_\{\\text\{step\}\}\\times D\\times K\.

All baseline configurations, model instanciations, parameter counts, and FLOPS estimation scripts are documented in the accompanying notebookflops\_baselines\.ipynb\.

Table 2:TRM FLOPs per supervision step\.
##### Validation\.

FLOP estimates can vary depending on framework\-level operator definitions and the target accelerator backend\. To quantify these variations, we reimplemented the TRM architecture\([Jolicoeur\-Martineau, 2025](https://arxiv.org/html/2609.36126#bib.bib12)\)in pure JAX and benchmarked the supervision\-step FLOPs across four profiling configurations \(Table[2](https://arxiv.org/html/2609.36126#A2.T2)\)\.

In Table[1](https://arxiv.org/html/2609.36126#S4.T1), we reportPyTorchFlopCounterModefor baselines \(to capture all operations and activations without kernel\-level omissions\) andJAX compiled on CPUfor Reasoning NCA \(to evaluate exact mathematical operations without GPU/TPU hardware tile padding or compiler rewrites\)\. As shown in Table[2](https://arxiv.org/html/2609.36126#A2.T2), both chosen estimators align consistently with less than3%3\\%relative difference \(ΔFC, CPU<3%\\Delta\_\{\\text\{FC, CPU\}\}<3\\%\), with JAX reporting slightly higher counts\.

##### Limitations\.

While FLOPs offer a hardware\-agnostic proxy for computational complexity, some limitations must be acknowledged\. First, our estimation isimplementation\-dependent\. For instance, our NCA implementation computes updates for allH×WH\\times Wcells before applying a binary maskmi,j∼Bernoulli⁡\(pfire\)m\_\{i,j\}\\sim\\operatorname\{Bernoulli\}\(p\_\{\\text\{fire\}\}\)to emulate asynchronous updates\. Forpfire=0\.8p\_\{\\text\{fire\}\}=0\.8, this dense execution incurs a1\.25×1\.25\\timesarithmetic overhead, resulting in higher reported FLOPs than an implementation where dormant cells perform zero operations\. Second, FLOP counts measure raw arithmetic operations but do not account formemory access patterns,parallelizability, orenergy footprint\. Despite being hardware\-dependent, we believe that such metrics could provide better insights into the scalability and energy advantages on current and/or future distributed hardware that FLOP counts alone do not capture\.

### B\.6TTS Pruning Method

When scaling test\-time compute by running an ensemble ofKKparallel stochastic rollouts on a single puzzle, independent instances often converge to identical intermediate trajectories\. Running duplicate trajectories to completion incurs substantial compute overhead without expanding exploratory breadth\. To eliminate this redundancy, we introduceNiche\-Capped Diversity Pruning, an algorithm that periodically identifies congruent solution branches, caps the number of duplicate instances, and progressively concentrates compute on diverse hypotheses\.

##### Algorithm Formulation\.

Consider an initial ensemble ofKKparallel rollout trajectoriesℰ\(0\)=\{X1\(0\),…,XK\(0\)\}\\mathcal\{E\}^\{\(0\)\}=\\\{X\_\{1\}^\{\(0\)\},\\dots,X\_\{K\}^\{\(0\)\}\\\}initialized with Gaussian noiseσinit\\sigma\_\{\\text\{init\}\}and rolled out for a total horizon ofDDsteps\. We partition the trajectory intommequidistant checkpoints spaced by strideΔ​t=D/m\\Delta t=D/m, corresponding to evaluation intervalst∈\{Δ​t,2​Δ​t,…,\(m−1\)​Δ​t\}t\\in\\\{\\Delta t,2\\Delta t,\\dots,\(m\-1\)\\Delta t\\\}\.

At each checkpointtt, pruning proceeds in four steps:

1. –Discrete State Extraction:For each active trajectorys∈ℰ\(t\)s\\in\\mathcal\{E\}^\{\(t\)\}, the current board configurationY^s\(t\)∈\{0,…,V−1\}H×W\\hat\{Y\}\_\{s\}^\{\(t\)\}\\in\\\{0,\\dots,V\-1\\\}^\{H\\times W\}is decoded from the cell readout slices: Y^s,i,j\(t\)=arg⁡maxv​ℓs,i,j\(t\)​\(v\)\\hat\{Y\}\_\{s,i,j\}^\{\(t\)\}=\\arg\\max\_\{v\}\\ell\_\{s,i,j\}^\{\(t\)\}\(v\)whereℓs,i,j\(t\)​\(v\)\\ell\_\{s,i,j\}^\{\(t\)\}\(v\)is the cosine similarity score between the cell statexs,i,j,out\(t\)x\_\{s,i,j,\\text\{out\}\}^\{\(t\)\}and token embeddingeve\_\{v\}\(Section[B\.1](https://arxiv.org/html/2609.36126#A2.SS1)\)\.
2. –Niche Partitioning:Trajectories are clustered into discrete equivalence classes orniches𝒩c\\mathcal\{N\}\_\{c\}sharing identical predicted board states: 𝒩c=\{s∈ℰ\(t\)\|Y^s\(t\)=Yc\}\\mathcal\{N\}\_\{c\}=\\left\\\{s\\in\\mathcal\{E\}^\{\(t\)\}\\;\\middle\|\\;\\hat\{Y\}\_\{s\}^\{\(t\)\}=Y\_\{c\}\\right\\\}whereYcY\_\{c\}denotes a unique candidate board configuration\.
3. –Niche\-Cap Filtering:To prevent over\-representation of any single attractor basin, each niche𝒩c\\mathcal\{N\}\_\{c\}is capped at a maximum capacity ofncapn\_\{\\text\{cap\}\}seeds \(we usencap=3n\_\{\\text\{cap\}\}=3\)\. For niches with\|𝒩c\|\>ncap\|\\mathcal\{N\}\_\{c\}\|\>n\_\{\\text\{cap\}\}, we retain thencapn\_\{\\text\{cap\}\}instances with the highest global board confidencec⁡\(Xs\(t\)\)c\(X\_\{s\}^\{\(t\)\}\)\(or uniform sampling\) and discard the remainder: 𝒩~c=Top−⁡ncap​\(𝒩c,by​c​\(Xs\(t\)\)\)\.\\tilde\{\\mathcal\{N\}\}\_\{c\}=\\operatorname\{Top\-\}n\_\{\\text\{cap\}\}\\left\(\\mathcal\{N\}\_\{c\},\\;\\text\{by \}c\(X\_\{s\}^\{\(t\)\}\)\\right\)\.
4. –Ensemble Halving:The total active ensemble is halved to target size⌊K/2⌋\\lfloor K/2\\rfloorby pooling surviving seeds from the capped niches, prioritizing representation across distinct niches to maximize hypothesis diversity before resuming rollouts\.

##### Exploration Noise Schedule\.

As in unpruned exploration \(Section[B\.4](https://arxiv.org/html/2609.36126#A2.SS4)\), test\-time Gaussian perturbations,𝒩⁡\(0,σ2\)\\mathcal\{N\}\(0,\\sigma^\{2\}\), are injected during the initial quarter of the trajectory \(rnoise=0\.25r\_\{\\text\{noise\}\}=0\.25\) with temporal probabilitypt=0\.2p\_\{t\}=0\.2, spatial probabilityps=0\.8p\_\{s\}=0\.8, and standard deviationσ=0\.1\\sigma=0\.1\. This ensures rollouts explore divergent state\-space regions before the first pruning interval culls duplicate attractors\.

##### Theoretical FLOPs Reduction\.

For an initial ensembleKKunrolled overDDsteps and halved atm∈\{2,4,8\}m\\in\\\{2,4,8\\\}equidistant intervals, total compute scales as:

FLOPs​\(m\)\\displaystyle\\text\{FLOPs\}\(m\)=∑i=0m−1\(K2i⋅Dm\)\\displaystyle=\\sum\_\{i=0\}^\{m\-1\}\\left\(\\frac\{K\}\{2^\{i\}\}\\cdot\\frac\{D\}\{m\}\\right\)=2−2−\(m−1\)m⋅\(K⋅D\)\.\\displaystyle=\\frac\{2\-2^\{\-\(m\-1\)\}\}\{m\}\\cdot\(K\\cdot D\)\.The relative FLOP savings are1−2−2−\(m−1\)m1\-\\frac\{2\-2^\{\-\(m\-1\)\}\}\{m\}, yielding exact savings of25\.0%form=2m=2,53\.1%form=4m=4, and75\.1%form=8m=8, with minimal effect on final accuracy across 110,000 Sudoku\-Extreme boards\.

## Appendix CExperimental Settings

### C\.1Maze

Detailed architecture and training configurations are listed in Table[3](https://arxiv.org/html/2609.36126#A3.T3)\. Training took approximately 3 minutes for Maze\-OOD and 2h30 for Maze\-Hard \(TPU v5lite\)\.

At test time, states are initialized randomly \(σinit\\sigma\_\{\\text\{init\}\}as in training\)\. For Maze\-Hard, we perform parallel rollouts with noise injection for test\-time scaling experiments\. We use the same noise regime that in training \(pt=0\.1,ps=0\.3p\_\{t\}=0\.1,p\_\{s\}=0\.3\) but vary noise magnitudeσ\\sigmabetween 0 and 1 \(as reported in Figure[7](https://arxiv.org/html/2609.36126#S4.F7)\) and apply it for the first quarter of the rollout \(rnoise=0\.25r\_\{\\texttt\{noise\}\}=0\.25\)\.

### C\.2Sudoku

Detailed architecture and training configurations are listed in Table[3](https://arxiv.org/html/2609.36126#A3.T3)\. Training took approximately 3h for Sudoku\-OOD \(TPU v5lite\) and 22h for Sudoku\-Extreme \(TPU7x\)\.

At test time, states are initialized randomly \(σinit\\sigma\_\{\\text\{init\}\}as in training\)\. We perform parallel rollouts with noise injection for test\-time scaling experiments\. We use the same noise regime that in training \(pt=0\.1,ps=0\.2p\_\{t\}=0\.1,p\_\{s\}=0\.2\) but vary noise magnitudeσ\\sigmabetween 0 and 1 \(as reported in Figure[7](https://arxiv.org/html/2609.36126#S4.F7)\) and apply it for the first quarter of the rollout \(rnoise=0\.25r\_\{\\texttt\{noise\}\}=0\.25\)\.

Table 3:Training hyperparameters for the Maze and Sudoku benchmarks\.
### C\.3ARC\-AGI\-1

Detailed architecture and training configurations are listed in Tables[5](https://arxiv.org/html/2609.36126#A3.T5)and[5](https://arxiv.org/html/2609.36126#A3.T5)\. Training took approximately 24h for ARC\-AGI\-1 \(TPU 7x\)\.

At test time, we useA=51A=51offline augmentations andR=64R=64online augmentations per task of the public evaluation set \(KK=3264, as reported in Figure[2](https://arxiv.org/html/2609.36126#footnote2)\)\.

Table 4:NCA architecture hyperparameters for ARC\-AGI\-1\.
Table 5:Pre\-training and TTT optimization hyperparameters for ARC\-AGI\-1\.

### C\.4Visual Sudoku

Detailed architecture and training configurations are listed in Table[6](https://arxiv.org/html/2609.36126#A3.T6)\. The architecture and approach served to simply validate that an NCA model could in principle solve a large, complex task like Visual Sudoku\. This was done by taking an“off\-the\-shelf”NCA and associated training pipeline, and directly applying to the Visual Sudoku image\-to\-image task\. As a result, the training is not optimised for this task, and incorporates many of the assumptions made for the small model, such as back\-propagation through full unrolls of the entire trajectory of 1024 steps\. This results in a compute intensive training regime which we believe could be significantly optimised\. Training took approximately three days on 64 v5p TPUs machines\. The training also occasionally suffered from instabilities, where the loss would spike\. When this happened, training was resumed from a previous, clean, checkpoint, with a lower learning rate, to move past the spike, then resumed with the original learning rate\.

Table 6:Training hyperparameters for Visual Sudoku\.

## Appendix DAdditional Results

### D\.1Fixed Attention Analysis

We tested an attention\-based perception module to provide cells with a more expressive, position\-dependent, and data\-dependent way to attend to their neighbors compared to a convolution perception module\. We first implemented a traditional self\-attention perception module with key\-query dot products within the3×33\\times 3neighborhood to obtain attention scores\. Yet, inspecting the learned weights revealed that self\-attention converged to static, spatially symmetric patterns independent of time step \(Figure[14](https://arxiv.org/html/2609.36126#A4.F14)\); rather than routing information dynamically based on cell states\.

![Refer to caption](https://arxiv.org/html/2609.36126v1/figures/fixed_vs_self_attention_curves.png)Figure 13:Fixed\-attention ablation study\.Training loss \(A\) and board accuracy \(B\) show that fixed attention consistently trains faster and achieves higher solve rates than the standard self\-attention on Sudoku\-OOD and Sudoku\-Extreme\. Mean\-std curves over 3 training seeds are shown\.Motivated by this observation, we replaced dynamic self\-attention with“fixed attention”, directly parameterizing static, position\-dependent mixing weights across the3×33\\times 3neighborhood\. This change eliminates the need to compute query\-key dot products at every rollout step\. Beyond computational savings, we found that directly parameterizing this fixed attention significantly improved training, achieving lower training loss and higher final board accuracy across both Sudoku\-OOD and Sudoku\-Extreme benchmarks \(Figure[13](https://arxiv.org/html/2609.36126#A4.F13)\)\.

![Refer to caption](https://arxiv.org/html/2609.36126v1/figures/fixed_vs_self_attention_viz.png)Figure 14:Self\-attention converges to static spatial patterns\.Attention weights across the 16 perception heads for fixed attention \(left\) and self\-attention at rollout stepst∈\{50,100,150\}t\\in\\\{50,100,150\\\}\(right\)\. Each panel shows the9×99\\times 9Sudoku grid, with each cell displaying 9 dots showing attention weight to its3×33\\times 3Moore neighborhood \(size indicates weight magnitude; green is positive, pink is negative\)\. Self\-attention weights, derived from query\-key dot product at rollout steptt, have converged to static spatial patterns that do not depend on cell states nor time step\. Fixed attention parameterizes this inter\-cell coupling directly, leading to more efficient learning\.
### D\.2Impact of the Locality Bottleneck

To study how spatial locality shapes iterative reasoning dynamics, we compare NCAs with TRM\([Jolicoeur\-Martineau, 2025](https://arxiv.org/html/2609.36126#bib.bib12)\)\. Unlike NCAs, TRM uses all\-to\-all connectivity and synchronous updates, allowing every grid token to attend to all other tokens via global self\-attention at each step\.

We evaluate both models on the 1000 test instances of Maze\-Hard \(30×3030\\times 30\) overt=1,…,Dt=1,\\dots,Dsteps, wherettdenotes each time step at which the output prediction is updated\. In TRM, the internal state consists of two components: a high\-level answer stateyyfrom which predictions are decoded \(analogous to ourCoutC\_\{\\text\{out\}\}channels\) and a low\-level latent reasoning statezz\(analogous to ourChidC\_\{\\text\{hid\}\}channels\)\. The answer stateyyis updated once everyLcycles=4L\_\{\\text\{cycles\}\}=4updates of the latent statezz\. TRM is evaluated withNsup=16N\_\{\\text\{sup\}\}=16supervision steps andHcycles=3H\_\{\\text\{cycles\}\}=3prediction updates per supervision step, resulting in a maximum rollout depth ofD=Nsup×Hcycles=48D=N\_\{\\text\{sup\}\}\\times H\_\{\\text\{cycles\}\}=48prediction steps\. For the NCA, we useD=200D=200steps\. As TRM did not release official checkpoints, we use the reproduction by[https://github\.com/gaoxin492/TinyRecursiveModels](https://github.com/gaoxin492/TinyRecursiveModels)\.

Table 7:The locality bottleneck enforces iterative reasoning\.Time\-to\-Solve \(mean±\\pmstd\) on the mutually solved subset of Maze\-Hard\.We definetsolvet\_\{\\text\{solve\}\}as the earliest step at which the decoded prediction matches the exact ground\-truth \(A∗A^\{\*\}\) path and remains stably correct for all remaining steps through the end of the rollout\. Within their respective budgets, both models achieve similar solve rates: 787/1000 for TRM and 790/1000 for NCA\. Table[7](https://arxiv.org/html/2609.36126#A4.T7)reports the averagetsolvet\_\{\\text\{solve\}\}computed over the subset of 679 mazes that both NCA and TRM solved\. TRM converges in only∼3\\sim 3prediction steps \(corresponding to∼12\\sim 12updates of the latent vectorzz\), whereas the NCA requires∼42\\sim 42steps\.

![Refer to caption](https://arxiv.org/html/2609.36126v1/figures/figure_trm_solving.png)Figure 15:TRM converges in very few steps\.Visualizing the intermediate predictions on an example maze \(Figure S[15](https://arxiv.org/html/2609.36126#A4.F15)\) shows that TRM“jumps”to the solution very quickly \(in 2 prediction steps\)\. In contrast, NCAs show contiguous, wave\-like path exploration \(Figure[2](https://arxiv.org/html/2609.36126#S4.F2)\)\. While NCAs require more steps for solving mazes due to this locality constraint, it makes the NCA’s reasoning process quite interpretable with an observable spatial“chain\-of\-thought”that can be visually tracked on the grid\.

![Refer to caption](https://arxiv.org/html/2609.36126v1/figures/figure_sudoku_full_tts.png)Figure 16:Extended test\-time scaling results on Sudoku benchmarks\.Evaluation on\(A\)Sudoku\-OOD and\(B\)Sudoku\-Extreme across, rollout iterations \(left sub\-panel\) and parallel trials \(right sub\-panel\)\. Colored curves show test accuracy for different difficulties without noise \(solid lines\) and with noise injection \(dashed lines\)\. Metrics are averaged over 3 test seeds\.
### D\.3Extended Test\-Time Scaling Results

#### D\.3\.1Sudoku

We find that parallel trials significantly improve performance on Sudoku benchmarks \(Figure[16](https://arxiv.org/html/2609.36126#A4.F16)\)\. Sampling multiple stochastic trajectories increases the likelihood of finding the correct solution, and our confidence\-based selection mechanism reliably identifies it among theKKcandidates\. On Sudoku\-OOD, this strategy boosts accuracy from∼\\sim50% \(K=1, D=48\) to 97% \(K=2048, D=256\); and on Sudoku\-Extreme from∼\\sim50% \(K=1, D=96\) to 91\.9% \(K=512, D=2048\)\.

We also observe that injecting noise at test\-time is beneficial: while causing an initial accuracy drop when applied, adding noise at inference eventually slightly exceeds the performance of the no\-noise variant by few percents rising accuracy to 98\.5% for Sudoku\-OOD and 92\.7% for Sudoku\-Extreme\.

Interestingly, we find that the difficulty ordering in Sudoku\-Extreme differs from empirical difficulty: NCAs struggle most with intermediate difficulty bins \(\[6​–​11\]\[6\\text\{\-\-\}11\]and\[11​–​16\]\[11\\text\{\-\-\}16\]backtracks intdoku\) rather than the highest\-backtrack bins \(\[54​–​290\]\[54\\text\{\-\-\}290\]\)\. This suggests that intermediate boards contain more densely entangled candidate constraints across multiple cells, creating local minima that are harder for NCAs to resolve\.

![Refer to caption](https://arxiv.org/html/2609.36126v1/figures/figure_maze_full_tts.png)Figure 17:Extended test\-time scaling results on Maze benchmarks\.Evaluation on Maze\-Hard\(A\)withA∗A^\{\*\}targets and\(B\)with BFS targets, across rollout iterations \(left sub\-panel\) and parallel trials \(right sub\-panel\)\. Colored curves show test accuracy for different difficulties without noise \(solid lines\) and with noise injection \(dashed lines\)\. Metrics are averaged over 3 test seeds\.
#### D\.3\.2Maze

On Maze\-Hard, test\-time scaling is also beneficial but provides smaller gains, pushing results from∼\\sim85% \(K=1, D=128\) to 88\.1% \(K=128, D=128\) when trained onA∗A^\{\*\}targets, and from∼\\sim92% \(K=1, D=128\) to 96\.1% \(K=128, D=128\) when trained on BFS targets; with more than 98% of predictions being valid solutions in both cases\. We see again small gains with noise\-injection rising accuracy to 89\.2% on the model trained onA∗A^\{\*\}targets\. Overall, training on BFS targets in Maze\-Hard instead ofA∗A^\{\*\}targets significantly improves performances\.

#### D\.3\.3ARC\-AGI\-1

![Refer to caption](https://arxiv.org/html/2609.36126v1/figures/figure_arc_ttt.png)Figure 18:Test\-time compute scaling on ARC\-AGI public evaluation set\.\(A\)Pass@1 majority\-voting accuracy across increasing numbers of offline and online augmentations\.\(B\)Pass@KKscaling under the offline\-first\-augmentation policy, reaching 60\.3% Pass@64\.In ARC\-AGI\-1, the use of parallel stochastic rollouts from different augmented inputs similarly increases pass@2 performances on the evaluation set from 28\.25% \(K=1, D=64\) to 48\.75% \(K=3264, D=64\)\. Moreover, considering Pass@64 pushes performance further to 60\.3%, suggesting that some of the stochastic rollouts successfully discover valid solutions, but the present candidate selection mechanism is unable to reliably select it\. Using an NCA ensemble \(NCA E\) of 3 models independently fine\-tuned further boosts performances to 63\.0% Pass@64\.

![Refer to caption](https://arxiv.org/html/2609.36126v1/figures/test_time_scaling_hard_combined.png)Figure 19:Test\-time scaling via extended rollouts on Visual Sudokuhardpuzzles, rendered both with unseen and seen MNIST digits\. The accuracy improves from approximately 10% to 23% with additional test time compute with seen, and from approximately 8% to 19% with unseen\.
#### D\.3\.4Visual\-Sudoku

We observe that running the model for longer rollouts than used during training improves accuracy \(Figure[19](https://arxiv.org/html/2609.36126#A4.F19)\)\.

### D\.4Extended TTS Pruning Results

![Refer to caption](https://arxiv.org/html/2609.36126v1/figures/figure_tts_pruning_inference_time.png)Figure 20:End\-to\-end wall\-clock inference runtime and speedup of test\-time pruning on Sudoku\-Extreme\. \(A\) Total inference runtime \(hours on NVIDIA H100 GPUs across110,000110\{,\}000test puzzles\) for the unpruned baseline \(K=512K=512\) and pruning divisorsm∈\{2,4,8\}m\\in\\\{2,4,8\\\}across rollout horizonsD∈\{64,…,2048\}D\\in\\\{64,\\dots,2048\\\}\(error bars indicate±1\\pm 1SD over 3 seeds\)\. \(B\) Empirical wall\-clock speedup multiplier \(Timebase/Timepruned\\text\{Time\}\_\{\\text\{base\}\}/\\text\{Time\}\_\{\\text\{pruned\}\}\) versus rollout horizonDD\. Horizontal dashed lines indicate the theoretical FLOP\-reduction ceilingsm2−2−\(m−1\)\\frac\{m\}\{2\-2^\{\-\(m\-1\)\}\}\(1\.33×1\.33\\times,2\.13×2\.13\\times, and4\.01×4\.01\\timesform=2,4,8m=2,4,8\); measured speedups closely track the theoretical limits across all horizons \(<1%<1\\%sorting and compaction overhead\)\.To evaluate the practical serving efficiency and hardware translation of niche\-capped pruning, we measure the end\-to\-end wall\-clock inference duration across all horizonsD∈\[64,…,2048\]D\\in\[64,\.\.\.,2048\]on NVIDIA H100 GPUs \(Figure[20](https://arxiv.org/html/2609.36126#A4.F20)A\)\. Unpruned baseline rollouts at peak horizon \(D=2048D=2048\) require18\.42​hours18\.42\\text\{ hours\}per shard \(110,000 Sudoku\-Extreme test puzzles\)\. Progressive niche\-capping reduces this duration dramatically to13\.82​hours13\.82\\text\{ hours\}\(m=2m=2\),8\.66​hours8\.66\\text\{ hours\}\(m=4m=4\), and4\.63​hours4\.63\\text\{ hours\}\(m=8m=8\), cutting evaluation turnaround time and cloud serving costs by up to13\.79​hours13\.79\\text\{ hours\}\(3\.98×3\.98\\times\)\. We also measure the empirical speedup \(Timebase/Timepruned\\text\{Time\}\_\{\\text\{base\}\}/\\text\{Time\}\_\{\\text\{pruned\}\}\) against the theoretical ceiling\[m2−2−\(m−1\)\]\\left\[\\frac\{m\}\{2\-2^\{\-\(m\-1\)\}\}\\right\]\. Across all evaluated rollout horizons \(D=64​…​2048D=64\\dots 2048\), the measured acceleration closely tracks theoretical limits \(Figure[20](https://arxiv.org/html/2609.36126#A4.F20)B\)\. This confirms that our vectorized JAX implementation incurs<0\.9%<0\.9\\%sorting overhead, allowing theoretical FLOP formulations to serve as good predictors of real\-world accelerator limits\.