Algorithmic Information Dynamics of Learning: A Certified, Differentiable Complexity Controller for Grokking

# Algorithmic Information Dynamics of Learning:A Certified, Differentiable Complexity Controller for Grokking
Source: [https://arxiv.org/html/2609.13197](https://arxiv.org/html/2609.13197)
Luan OzelimAffiliation:Oxford Immune Algorithmics, Oxford University Innovation & London Institute for Healthcare Engineering, U\.KHector ZenilThanks:Corresponding author: hector\.zenil@kcl\.ac\.ukAffiliation:Oxford Immune Algorithmics, Oxford University Innovation & London Institute for Healthcare Engineering, U\.KAffiliation:Department of Biomedical Computing, School of Biomedical Engineering and Imaging Sciences & King’s Institute for AI, King’s College London, U\.K

###### Abstract

Algorithmic Information Dynamics \(AID\) studies systems by perturbing them and measuring changes in algorithmic complexity, but its usual estimator, the Block Decomposition Method, is piecewise constant, restricting the calculus to finite differences\. We useKs​FCDMK^\{\\mathrm\{CDM\}\}\_\{\{\\mathrm\{s\}F\}\}, a certified,*differentiable*estimator, to bring the calculus into learning dynamics: grokking, where a complexity*order parameter*is known but has not been made to*act*\. As a transient loss kick, the estimator becomes a*controller*that accelerates grokking in Levin’s description\-length–versus\-time sense, within a data\-dependent Occam boundary whose finite\-size trend,fc∼ln⁡p/pf\_\{c\}\\sim\\ln p/p, is consistent with a coupon\-collector interpretation\. Ablations show that a complexity gate matches a train\-loss gate in rescuing failing seeds with27%27\\%less intervention; among the tested signals, only map complexity marks the transition’s*completion*; the certified prior and the per\-parameter∇K\\nabla Kattribution are both fungible \(a uniform\-prior sensor makes bit\-identical gate decisions, and random supports match∇K\\nabla K\-selected ones above a sparsity threshold\); and direct field perturbation shows a*nucleation\-like*response to the Occam field \(no linear regime is resolved over the probed amplitudes, so these measurements do not justify a fluctuation–dissipation surrogate\), with a finite\-field response growing by orders of magnitude toward the transition\. These measurements account for the empirically tuned staircase: bang–bang pulses, stall\-fired and released on yield, whose iteration plausibly builds the response it exploits\. The kick transfers to sparse parity and to a transformer; a sustained weight\-space loss fails\. The algorithmic estimator’s distinct contribution is*timing*\(when to fire and when to release\), not attribution\.

Highlights

- •A certified, differentiable complexity estimator is used to*control*, not only describe, grokking\.
- •Delivered as a transient kick, it roughly doubles grokking speed and rescues failing seeds\.
- •Acceleration is bounded by a data\-dependent Occam boundary whose measured threshold recedes approximately asln⁡p/p\\ln p/p\.
- •The response to the Occam field is nucleation\-like, with no linear regime resolved over the probed amplitudes\.
- •The complexity gate matches a loss gate using27%27\\%less intervention\.

Keywords:algorithmic information dynamics; algorithmic complexity; grokking; control of learning dynamics; order parameter; finite\-size scaling; nucleation

## 1Introduction

Algorithmic Information Dynamics \(AID\)\[[7](https://arxiv.org/html/2609.13197#bib.bib7),[8](https://arxiv.org/html/2609.13197#bib.bib8)\]studies a system through the response of its algorithmic \(Kolmogorov\) complexity\[[3](https://arxiv.org/html/2609.13197#bib.bib3),[6](https://arxiv.org/html/2609.13197#bib.bib6)\]to perturbation: for an elementeeof an objectGG, the signed quantityΔ​K​\(e\)=K⁡\(G∖e\)−K⁡\(G\)\\Delta K\(e\)=K\(G\\setminus e\)\-K\(G\)separates elements that inject algorithmic randomness \(their removal lowers complexity\) from elements that carry the object’s program \(their removal raises it\)\. Steering a system by acting selectively on these elements is AID’s*algorithmic causal calculus*, and guiding machine learning by algorithmic probability is its natural extension\[[12](https://arxiv.org/html/2609.13197#bib.bib12)\]\.

Two things have limited the calculus\. First, its estimator: the Block Decomposition Method\[[11](https://arxiv.org/html/2609.13197#bib.bib11)\], built on the Coding Theorem Method’s enumeration of small machines\[[9](https://arxiv.org/html/2609.13197#bib.bib9),[10](https://arxiv.org/html/2609.13197#bib.bib10)\], composes block complexities under a block\-independence assumption and is piecewise constant in its input, soΔ​K\\Delta Kmust be obtained by finite differences over a combinatorial set of perturbations, and no gradient exists\. Second, its domain: the calculus has been applied to*objects*\(strings, networks\), not to the*dynamics*that produce them\.

This paper addresses both limits, in the setting where the connection between algorithmic complexity and learning dynamics is sharpest: grokking, the delayed transition from memorisation to generalisation\[[13](https://arxiv.org/html/2609.13197#bib.bib13)\], whose mechanism has been studied through circuit efficiency\[[19](https://arxiv.org/html/2609.13197#bib.bib19)\], representation\-learning phases\[[16](https://arxiv.org/html/2609.13197#bib.bib16)\], optimiser dynamics\[[18](https://arxiv.org/html/2609.13197#bib.bib18)\], and mechanistic progress measures\[[15](https://arxiv.org/html/2609.13197#bib.bib15)\]\. That a complexity measure tracks this transition is established: DeMoss et al\.\[[14](https://arxiv.org/html/2609.13197#bib.bib14)\]show a compression\-based complexity of the weights rises and falls across it; Sakabe et al\.\[[1](https://arxiv.org/html/2609.13197#bib.bib1)\]show, with the block decomposition method, that binarised networks move toward algorithmic simplicity over training and that this tracks the loss more closely than entropy does; and the companion paper\[[2](https://arxiv.org/html/2609.13197#bib.bib2)\]shows the certified estimatorKs​FCDMK^\{\\mathrm\{CDM\}\}\_\{\{\\mathrm\{s\}F\}\}, read label\-free from the network’s own input–output map, is a clean order parameter for it\. The*descriptor*is therefore prior art, and we treat it as our starting point\.

The claim of this paper is the*controller*\. We takeKs​FCDMK^\{\\mathrm\{CDM\}\}\_\{\{\\mathrm\{s\}F\}\}, the certified, differentiable, multidimensional estimator of\[[2](https://arxiv.org/html/2609.13197#bib.bib2)\], whose relaxation to the real\-valued hypercube is exact on the binary corners; because it is differentiable, AID’s perturbation calculus becomes a gradient,Δ​K→∇K\\Delta K\\to\\nabla K\. We then use the measure to act on the training trajectory: to accelerate the transition \(as a transient kick\), to time other interventions \(as a gate with a release\), and to delimit when such control can work at all \(the Occam boundary and its finite\-size scaling\)\. Each constructive result is paired with an ablation that asks whether the algorithmic apparatus is load\-bearing, and we give the resulting negative answers the same weight as the positive ones\.

#### Scope of the claim\.

Classical AID perturbs the*state*; the controller below perturbs the*flow*\(the loss\)\. These are not the same intervention, and the bridge \(that in grokking the object of interest is the network’s own input–output map, which the trajectory moves through function space\) is an extension of AID rather than a direct application\. We state it as such\.

#### Roadmap\.

Section[2](https://arxiv.org/html/2609.13197#S2)builds the transient\-kick controller, states the condition under which complexity control can recover a function at all \(Condition[1](https://arxiv.org/html/2609.13197#Thmcondition1)\), and studies that condition’s dependence on the amount of data and on system size\. Section[3](https://arxiv.org/html/2609.13197#S3)dissects the controller by gating known accelerators on the order parameter and ablating each ingredient in turn \(the gate, the release, and the sensor’s certified prior\), and tests whether the controller survives a change of task\. Section[4](https://arxiv.org/html/2609.13197#S4)uses the calculus itself: it applies∇K\\nabla Kas a per\-parameter classification, measures the system’s response to a complexity field directly, and asks whether the measured response accounts for the schedule that was tuned by hand\. Sections[5](https://arxiv.org/html/2609.13197#S5)and[6](https://arxiv.org/html/2609.13197#S6)weigh what certification buys against what the ablations found dispensable, and set out what remains open\.

## 2Complexity as a controller

We train a two\-layer MLP on modular arithmetic, the canonical grokking task\[[13](https://arxiv.org/html/2609.13197#bib.bib13)\], and readKs​FCDMK^\{\\mathrm\{CDM\}\}\_\{\{\\mathrm\{s\}F\}\}of the learned input–output map \(thep×pp\\times poutput table\) from the network’s own predictions, using no labels\. That this reading is a clean, label\-free*order parameter*for the memorisation\-to\-generalisation transition is established in the companion paper\[[2](https://arxiv.org/html/2609.13197#bib.bib2)\]; here we ask what happens when the order parameter is used to act\. Following\[[2](https://arxiv.org/html/2609.13197#bib.bib2)\]the estimator is built on the symmetrised reference machines​F\{\\mathrm\{s\}F\}, so thatKs​FCDMK^\{\\mathrm\{CDM\}\}\_\{\{\\mathrm\{s\}F\}\}is exactly invariant to the arbitrary labelling of the two output\-bit values\. This choice does not affect the control results below: because the object scored is a large, structuredp×pp\\times pmap, the raw and symmetrised estimates of every map that appears \(true, memorised, random\) differ by under4%4\\%and induce the same ordering\. We adopt it for consistency and because a controller read from a tape\-dependent convention would be suspect\. On the raw machineFFthe corresponding numbers are, in each case, within a few percent of those reported\.

#### The estimator, in brief\.

For a binary field \(here, each of the⌈log2⁡p⌉\\lceil\\log\_\{2\}p\\rceilbit\-planes of thep×pp\\times pmap\)Ks​FCDMK^\{\\mathrm\{CDM\}\}\_\{\{\\mathrm\{s\}F\}\}codes the cells in raster order by a chain rule\. Each cell’s bit is predicted by a Bayesian mixture of two experts: a*causal\-context expert*, a count table over the pattern formed by the cell’s four causal neighbours, seeded with unit pseudocount by thes​F\{\\mathrm\{s\}F\}pattern law distilled from the exhaustive enumeration of the reference machine; and a Krichevsky–Trofimov frequency expert\. The estimate is the total codelength∑i−log2Q\(bi∣b<i\)\\sum\_\{i\}\-\\log\_\{2\}Q\(b\_\{i\}\\mid b\_\{<i\}\)in bits, summed over bit\-planes\. It is*certified*in the sense of\[[2](https://arxiv.org/html/2609.13197#bib.bib2)\]: the value is an exact prefix\-codelength \(so2−Ks​FCDM2^\{\-K^\{\\mathrm\{CDM\}\}\_\{\{\\mathrm\{s\}F\}\}\}is a semimeasure\), and it is exactly invariant under global bit complementation by construction ofs​F\{\\mathrm\{s\}F\}\. The differentiable relaxationKs​FsoftK^\{\\mathrm\{soft\}\}\_\{\{\\mathrm\{s\}F\}\}evaluates the same code on the expected bit\-planes under the network’s softmax output; it coincides withKs​FCDMK^\{\\mathrm\{CDM\}\}\_\{\{\\mathrm\{s\}F\}\}on binary corners and is differentiable in the logits; every gradient of complexity used in this paper is a gradient ofKs​FsoftK^\{\\mathrm\{soft\}\}\_\{\{\\mathrm\{s\}F\}\}\. For brevity, references below to “Ks​FCDMK^\{\\mathrm\{CDM\}\}\_\{\{\\mathrm\{s\}F\}\}pressure” or a “Ks​FCDMK^\{\\mathrm\{CDM\}\}\_\{\{\\mathrm\{s\}F\}\}loss” mean pressure applied through this differentiable relaxation; reported map complexities are discreteKs​FCDMK^\{\\mathrm\{CDM\}\}\_\{\{\\mathrm\{s\}F\}\}values unless stated otherwise\.

### 2\.1Experimental protocol and reporting conventions

Unless otherwise noted, the primary experiments use modular addition atp=31p\{=\}31with a seeded random40%40\\%of thep2p^\{2\}input pairs for training and the complement for testing\. The model is a two\-layerReLU\\mathrm\{ReLU\}MLP with width\-128128operand embeddings, hidden width256256, and a linear readout toppclasses\. We train with full\-batch AdamW at learning rate10−310^\{\-3\}and weight decay1\.01\.0\. The discreteKs​FCDMK^\{\\mathrm\{CDM\}\}\_\{\{\\mathrm\{s\}F\}\}readout is evaluated on the network’s complete argmax input–output map every250250steps; the gated ablation uses5050\-step checks\. A run is said to grok at the first check for which held\-out accuracy exceeds0\.90\.9\.

All reported grok\-step means are conditional on reaching that criterion within the stated budget, so every mean is accompanied by the number of successful seeds\. Comparisons use matched seed pools within a shared code path unless explicitly stated otherwise\. These small\-sample summaries are descriptive, not population\-level estimates; per\-seed results for the two principal ablation grids and full controller constants appear in Appendix[A](https://arxiv.org/html/2609.13197#A1)\. Task\-, architecture\-, and actuator\-specific departures from this protocol are given where each experiment is introduced\.

### 2\.2From monitor to controller, and the Occam boundary

The order parameter is also actionable, but only in a specific form and only for a specific class of tasks\. As a*sustained*loss,Ks​FsoftK^\{\\mathrm\{soft\}\}\_\{\{\\mathrm\{s\}F\}\}fails: minimising it traps training in a simple but incorrect solution\. Once the data are fit, the cross\-entropy gradient vanishes and the complexity term drags the function toward simpler maps, of which a constant map is the global minimum\. Pinning the fitted train logits to prevent this instead freezes the network: train and test inputs share weights, and grokking is a global reorganisation that transiently perturbs the train logits\. This is why a purpose\-built weight measure such as spectral entropy\[[14](https://arxiv.org/html/2609.13197#bib.bib14)\]is used as a training loss instead\. A*transient*kick escapes this: a short pulse ofKs​FCDMK^\{\\mathrm\{CDM\}\}\_\{\{\\mathrm\{s\}F\}\}pressure, removed before it can change the fixed point, breaks the memorisation basin, after which plain dynamics coast to the low\-complexity solution\. Iterating short, self\-releasing kicks \(each released after a modest relative fall inKs​FCDMK^\{\\mathrm\{CDM\}\}\_\{\{\\mathrm\{s\}F\}\}or a step cap, and re\-fired when complexity stalls above the best achieved\) forms a push–relax–push staircase \(Fig\.[1](https://arxiv.org/html/2609.13197#S2.F1)\)\. It approximately halves the time to grok: on modular*addition*the staircase reaches full generalisation after a mean of7,7197\{,\}719vs\.17,83317\{,\}833steps atp=31p\{=\}31\(means over successful runs; kick8/88/8seeds, plain6/86/8\) and5,1255\{,\}125vs9,3759\{,\}375atp=41p\{=\}41\(6/66/6\)\. It lands on the true\-map complexity every time, and rescues the seeds plain training misses\. The three established dials act on other quantities: weight\-decay scheduling\[[19](https://arxiv.org/html/2609.13197#bib.bib19)\], gradient filtering\[[20](https://arxiv.org/html/2609.13197#bib.bib20)\]and weight\-norm control\[[21](https://arxiv.org/html/2609.13197#bib.bib21)\]\.

Figure 1:Iterated*transient*complexity kicks \(shaded\) accelerate grokking\. Each kick lowersKs​FCDMK^\{\\mathrm\{CDM\}\}\_\{\{\\mathrm\{s\}F\}\}of the map; the network partially relapses while coasting \(the bounces\), and the controller re\-fires when complexity stalls\. After a few such steps a kick breaks the memorisation basin: test accuracy jumps andKs​FCDMK^\{\\mathrm\{CDM\}\}\_\{\{\\mathrm\{s\}F\}\}collapses to the true\-map value \(dashed\)\. A*sustained*Ks​FCDMK^\{\\mathrm\{CDM\}\}\_\{\{\\mathrm\{s\}F\}\}loss instead collapses or freezes \(text\); only the transient pulse drives the transition\. One representative seed; multi\-seed statistics appear in the text and §[4](https://arxiv.org/html/2609.13197#S4), and per\-seed values for the two ablation grids in Appendix[A](https://arxiv.org/html/2609.13197#A1)\.The acceleration is conditional, and the condition is Occam’s razor\. The kick presupposes that the correct solution is the minimum\-complexity map consistent with the data\. Modular addition satisfies this; modular*multiplication*does not \(a lower\-complexity map fits the same training cells\), and there the kick over\-simplifies below the true complexity to a wrong map \(0/60/6grok,Ks​FCDMK^\{\\mathrm\{CDM\}\}\_\{\{\\mathrm\{s\}F\}\}driven to≈3,300\{\\approx\}3\{,\}300against a true\-map≈4,330\{\\approx\}4\{,\}330\)\. This lower\-Ks​FCDMK^\{\\mathrm\{CDM\}\}\_\{\{\\mathrm\{s\}F\}\}map is concrete \(Fig\.[2](https://arxiv.org/html/2609.13197#S2.F2)\): it keeps the true products on the training cells but fills the unconstrained cells with large low\-complexity patches of a few repeated values, so it fits every training cell yet is wrong on almost all held\-out ones\. Penalising only the excess above the true value,relu⁡\(Ks​FCDM−K∗\)\\mathrm\{relu\}\(K^\{\\mathrm\{CDM\}\}\_\{\{\\mathrm\{s\}F\}\}\-K^\{\\ast\}\), repairs it \(0/→2/30/3\\\!\\to\\\!2/3\) and leaves addition untouched; butK∗K^\{\\ast\}cannot be recovered from the training data, because the true and the simpler\-wrong map agree on every training cell, the same fact that makes minimisation over\-simplify\.

Figure 2:The Occam boundary, made concrete \(p=31p\{=\}31,40%40\\%training data\)\.*Rows:*addition \(top\) and multiplication \(bottom\)\.*Columns:*the true table; the memorised map before the kick; the map after the kick; and its agreement with truth: grey marks the40%40\\%training cells \(all fit in both cases\), green/red the held\-out cells that are correct/wrong\. The same procedure recovers addition \(test100%100\\%; after\-kickKs​FCDM≈501K^\{\\mathrm\{CDM\}\}\_\{\{\\mathrm\{s\}F\}\}\\\!\\approx\\\!501, the true value\) but over\-simplifies multiplication to a lower\-complexity patchwork \(test7%7\\%;Ks​FCDM≈3,308K^\{\\mathrm\{CDM\}\}\_\{\{\\mathrm\{s\}F\}\}\\\!\\approx\\\!3\{,\}308against a true4,3304\{,\}330\): at this data fraction a simpler map provides a completion consistent with the multiplication training cells but not with the addition ones, so the razor points at the truth for one and away from it for the other\. With more data the multiplication gap closes \(text\)\.The boundary is thus not merely a matter of tuning but an identifiability constraint\.

###### Condition 1\(the Occam boundary\)\.

A necessary condition for algorithmic\-complexity control to recover the true function is that no lower\-complexity function be consistent with the same training sample\. If the true function is the unique minimum\-complexity completion and the controlled optimisation reaches that completion, complexity pressure is aligned with recovery\.

Crucially, the condition is on the sample, so it is data\-dependent, and more data expands it\. Sweeping the training fraction on multiplication, the over\-simplification vanishes at a threshold amount of data\. The kick’s recoveredKs​FCDMK^\{\\mathrm\{CDM\}\}\_\{\{\\mathrm\{s\}F\}\}climbs3,→3,→4,→4,3353\{,\}308\\\!\\to\\\!3\{,\}605\\\!\\to\\\!4\{,\}330\\\!\\to\\\!4\{,\}335and the test accuracy jumps→→→0\.990\.07\\\!\\to\\\!0\.09\\\!\\to\\\!1\.00\\\!\\to\\\!0\.99as the fraction goes→→→0\.80\.4\\\!\\to\\\!0\.5\\\!\\to\\\!0\.6\\\!\\to\\\!0\.8\. At fraction0\.60\.6the recovered complexity reaches the true\-map value \(4,3304\{,\}330\) exactly as the test accuracy snaps to11: the true multiplication table is then recovered at its own complexity, as predicted when the sample makes complexity pressure point toward the true completion\.

This behaviour echoes the Solomonoff principle on which the estimator is built\[[4](https://arxiv.org/html/2609.13197#bib.bib4)\]: with increasing data, simpler consistent hypotheses that do not represent the generating rule are progressively excluded\. Addition’s threshold is negligible \(its smooth ramp is the simplest completion of even a small sample\); multiplication’s is higher \(its intricate structure must be pinned by more data\)\. The razor accelerates learning when, and as soon as, the data make it point at the truth\.

For monitoring, the order parameter is unconditional and label\-free; for control it is Occam\-conditional, and where the condition holds the certified estimator both reads the generalisation transition and, as a transient pulse, hastens it toward the correct solution\.

### 2\.3The acceleration is Levin\-style, not a data–time trade

It is tempting to read the kick as buying speed with data, but a joint data×\\timestime sweep shows otherwise \(Fig\.[3](https://arxiv.org/html/2609.13197#S2.F3)\)\. On modular*addition*the kick reaches full generalisation at the same data threshold as plain training but in fewer iterations, and the saving is largest at the data\-scarce margin \(2\.3×2\.3\\timesat40%40\\%training data,17,33317\{,\}333vs\.7,4177\{,\}417steps, shrinking to a tie by70%70\\%\): same data, less time\. On*multiplication*, where Occam is misaligned below the threshold fraction, the kick is worse on both axes: it needs more data to recover at all \(threshold≈0\.6\{\\approx\}0\.6vs\.≈0\.4\{\\approx\}0\.4; at50%50\\%the baseline groks in all three seeds and the kick in one\) and, where both eventually grok, it is slower \(8,5008\{,\}500vs\.3,2503\{,\}250steps at60%60\\%\)\. All figures here are means over three seeds\. So the kick is not a resource exchange; it is a Levin\-style simplicity\-biased accelerator that shortens the search for the short\-program solution\. This is the operational content of Levin’sK​t=ℓ\+log⁡tKt=\\ell\+\\log t\[[5](https://arxiv.org/html/2609.13197#bib.bib5)\]: biasing the dynamics toward low description length shrinks the time to find the low\-complexity solution, a free time\-saving when the target is the minimum\-complexity hypothesis the data support, and a penalty in both time and data when it is not\. The intrinsic grokking data–time trade \(more data→\\toshorter delay, visible in both baselines\) belongs to grokking; the kick’s role is to slide the time axis down for Occam\-aligned tasks\.

Figure 3:Data×\\timestime: the kick accelerates without buying speed with data\.*Top:*final test accuracy vs\. training fraction;*bottom:*iterations to grok\. On*addition*\(left\) baseline and kick share the recovery threshold \(top curves coincide\) but the kick takes fewer iterations, the gap largest at the scarce margin \(40%40\\%\)\. On*multiplication*\(right\) the kick recovers only above a higher data threshold and, where both grok, takes more iterations\. The kick is a Levin\-style simplicity accelerator: faster when the target is the minimum\-complexity hypothesis, costlier in both data and time when it is not\. Each point is a mean over three seeds; iteration means are taken over the seeds that grok within the30,00030\{,\}000\-step budget \(all three everywhere except multiplication with the kick at50%50\\%, where one of three groks\); fractions at which no seed groks are left blank\.
### 2\.4Finite\-size scaling of the Occam boundary

Because the boundary is a property of the sample, the recovery of the true rule as the training fraction crossesfcf\_\{c\}forms a transition\-like crossover in a control parameter\. Its dependence on system sizeppexhibits an empirical finite\-size\-scaling pattern \(Fig\.[4](https://arxiv.org/html/2609.13197#S2.F4)\)\. Sweepingp∈\{17,23,31,41,53,61,71\}p\\in\\\{17,23,31,41,53,61,71\\\}on multiplication, the threshold fraction falls monotonically,fc≈0\.71,0\.63,0\.53,0\.44,0\.38,0\.29,0\.26f\_\{c\}\\approx 0\.71,\\,0\.63,\\,0\.53,\\,0\.44,\\,0\.38,\\,0\.29,\\,0\.26, while the transition sharpens\. On a grid refined to steps of0\.0250\.025nearfcf\_\{c\}at every modulus, the window over which recovery accuracy lies strictly between0\.050\.05and0\.950\.95narrows monotonically withpp:0\.600\.60,0\.480\.48,0\.380\.38,0\.180\.18,0\.130\.13,0\.030\.03, and a single sampled fraction atp=71p\{=\}71\. Atp=17p\{=\}17the accuracy climbs slowly across the whole sweep, from0\.050\.05atf=0\.15f\{=\}0\.15to0\.230\.23atf=0\.70f\{=\}0\.70, before rising; atp=71p\{=\}71it moves from0\.140\.14to0\.990\.99across one0\.0250\.025\-wide step\. The systematic shift and sharpening are consistent with finite\-size scaling, although these finite systems and seed counts do not by themselves establish a thermodynamic critical point\.

Figure 4:Empirical finite\-size scaling of the Occam boundary \(modular multiplication\), over seven modulip∈\{17,…,71\}p\\in\\\{17,\\dots,71\\\}\.*Left:*kick recovery accuracy vs\. training fraction; the transition shifts left and sharpens asppgrows \(each mean is a22\-seed mean, refined to1212seeds across the transition region, with the recovered plateau above it filled by monotonicity\)\.*Middle:*the threshold fractionfcf\_\{c\}\(interpolated0\.50\.5crossing\) againstpp, with the empirical fitfc=4\.6​ln⁡p/pf\_\{c\}=4\.6\\,\\ln p/p, not a straight line to zero\.*Right:*the diagnostic collapse: the threshold sample countnc=fc​p2n\_\{c\}=f\_\{c\}\\,p^\{2\}divided byp​ln⁡pp\\ln pis flat at4\.6±7%4\.6\\pm 7\\%across all seven moduli\. This normalisation is motivated by the coupon\-collector scalep​ln⁡pp\\ln pfor coveringppresidues and supports, but does not uniquely establish, that interpretation\.Over the tested range, the decline is better described byfc∼ln⁡p/pf\_\{c\}\\sim\\ln p/pthan by a linear extrapolation\. A possible sample\-complexity explanation is that recovery requires the training cells to constrain the multiplicative structure across allppresidues\. Random coverage ofppcategories has the coupon\-collector scalep​ln⁡pp\\ln p, motivating this scaling ansatz rather than deriving it from the present experiments\. The number of training constraints isn=f​p2n=f\\,p^\{2\}, so writing the threshold count asnc=fc​p2n\_\{c\}=f\_\{c\}\\,p^\{2\}and dividing byp​ln⁡pp\\ln pnormalises all seven moduli to a constant,nc/\(p​ln⁡p\)=4\.6±7%n\_\{c\}/\(p\\ln p\)=4\.6\\pm 7\\%\(Fig\.[4](https://arxiv.org/html/2609.13197#S2.F4), right\), flatter than the collapse under a fixed absolute sample \(nc=constn\_\{c\}=\\mathrm\{const\}, spread52%52\\%\), a fixed fraction \(nc∝p2n\_\{c\}\\propto p^\{2\},34%34\\%\), or a fixed count per residue \(nc∝pn\_\{c\}\\propto p,15%15\\%\)\. These observations supportfc=nc/p2∼ln⁡p/pf\_\{c\}=n\_\{c\}/p^\{2\}\\sim\\ln p/pwithin the measured range; under that ansatz the threshold fraction vanishes only asymptotically\. \(A straight line through the first three sizes appears to reachfc=0f\_\{c\}=0nearp≈73p\\approx 73\. Over seven sizes that reading is an artefact of fitting a short arc ofln⁡p/p\\ln p/p, which tends to zero only asp→∞p\\to\\infty\.\) The resulting interpretation is Solomonoff\-like: a fixed fraction supplies more absolute evidence asppgrows, and simpler incorrect completions are excluded once a sufficiently broad sample is seen\. Establishing coupon collection as the mechanism, rather than one explanation of the observed scaling, remains open\.

## 3Dissecting the controller

The kick is our own actuator\. To separate what the complexity signal contributes from what any intervention would contribute, we now gate known accelerators on the order parameter and ablate every ingredient: the gate, the release, and the sensor’s certified prior\. In AID terms this is an element classification applied to the training procedure itself: the policies below range from a blind, always\-on intervention through a gate on a cheap non\-algorithmic signal \(the train loss\) to a gate on the algorithmic response, and the experiment asks which classification of when to act carries the effect\.

### 3\.1Gating a known accelerator on the order parameter

#### Setup\.

Two actuators on modular addition \(p=31p\{=\}31, training fraction0\.40\.4, full\-batch AdamW,25,00025\{,\}000\-step budget, six seeds\): Grokfast slow\-gradient amplification\[[20](https://arxiv.org/html/2609.13197#bib.bib20)\]\(g^=g\+λ​EMAα​\(g\)\\hat\{g\}=g\+\\lambda\\,\\mathrm\{EMA\}\_\{\\alpha\}\(g\),α=0\.9\\alpha\{=\}0\.9,λ=2\\lambda\{=\}2\), and a weight\-decay schedule in the spirit of the circuit\-efficiency account\[[19](https://arxiv.org/html/2609.13197#bib.bib19)\]\(decay0\.10\.1normally, boosted to1\.01\.0while the gate is active\)\. Five gate policies each:*none*\(never\),*always*,*fixed*\(from step500500; Grokfast’s own two\-stage variant\),*loss*\(once the train cross\-entropy falls below0\.050\.05; a gate any cheap signal can implement, with no release\), and*Ks​FCDMK^\{\\mathrm\{CDM\}\}\_\{\{\\mathrm\{s\}F\}\}*\(on once fit, released when the map’sKs​FCDMK^\{\\mathrm\{CDM\}\}\_\{\{\\mathrm\{s\}F\}\}has fallen to40%40\\%of its post\-memorisation peak\)\. Table[1](https://arxiv.org/html/2609.13197#S3.T1)reports the outcome; the code path is shared, and indeed the blind\-on weight\-decay row reproduces the plain baseline seed for seed, as it must \(both are AdamW at decay1\.01\.0\)\.

Table 1:Gated\-vs\-blind ablation \(p=31p\{=\}31, fraction0\.40\.4, six seeds,25,00025\{,\}000\-step budget\)\. “Grokked” counts seeds reaching test accuracy\>0\.9\>0\.9; “grok step” is the mean over those seeds; “intervention steps” is the mean number of steps the actuator was active\. TheKs​FCDMK^\{\\mathrm\{CDM\}\}\_\{\{\\mathrm\{s\}F\}\}gate is also run with the sensor’s certified prior replaced by a uniform base measure \(indented row\): its decisions are bit\-identical on every seed\.
#### The actuators buy reliability, not speed\.

In this regime Grokfast actuation buys reliability, while the choice of gate determines how long the intervention remains active\. Plain training groks only4/64/6seeds \(the two failures sit at test≈0\.5\{\\approx\}0\.5for the whole budget\); the always\-on, loss\-gated, andKs​FCDMK^\{\\mathrm\{CDM\}\}\_\{\{\\mathrm\{s\}F\}\}\-gated Grokfast policies rescue all six seeds \(the fixed\-step gate rescues five\), at mean grok steps inside the baseline’s own seed spread \(14,05014\{,\}050–20,35020\{,\}350; per\-seed values in Table[5](https://arxiv.org/html/2609.13197#A1.T5)\)\. Grokfast’s headline accelerations arise in minibatch training, where the gradient has genuinely fast stochastic components for the low\-pass filter to separate; in full batch the gradient is the slow component, sog\+λ​EMA​\(g\)≈\(1\+λ\)​gg\+\\lambda\\,\\mathrm\{EMA\}\(g\)\\approx\(1\+\\lambda\)g, a rescaling that Adam’s per\-coordinate normalisation largely absorbs\. We verified this is not a tuning accident by probing the other three cells of theα∈\{0\.9,0\.98\}×λ∈\{2,5\}\\alpha\\in\\\{0\.9,0\.98\\\}\\times\\lambda\\in\\\{2,5\\\}grid \(two seeds each; the fourth cell is the always\-on row of Table[1](https://arxiv.org/html/2609.13197#S3.T1)\)\. No cell shows systematic acceleration: probe grok steps span13,75013\{,\}750–21,40021\{,\}400, straddling the plain baseline’s own14,05014\{,\}050–20,35020\{,\}350seed spread rather than shifting below it\. The informative comparison is therefore not speed but who rescues, and at what intervention budget\.

#### The complexity gate matches the loss gate at27%27\\%less intervention\.

It does so because the order parameter carries a signal the loss cannot: completion\. The loss gate and theKs​FCDMK^\{\\mathrm\{CDM\}\}\_\{\{\\mathrm\{s\}F\}\}gate both rescue all six seeds with nearly equal mean grok steps \(18,83318\{,\}833vs18,76618\{,\}766; final test1\.0001\.000vs0\.9860\.986\)\. But the train loss is pinned near zero from the moment of memorisation onward: it can switch the actuator on, never off\. The loss gate therefore intervenes for essentially the whole budget \(24,88424\{,\}884steps on average\)\.Ks​FCDMK^\{\\mathrm\{CDM\}\}\_\{\{\\mathrm\{s\}F\}\}falls to the true\-map value exactly when the reorganisation completes, and the release fires on that collapse: mean18,20818\{,\}208intervention steps,27%27\\%fewer, with the release trailing the test\-accuracy crossing on every seed \(by1\.91\.9–3\.83\.8k steps, mean≈2\.6\{\\approx\}2\.6k\)\. In the AID classification, the loss is an*onset*observable; the algorithmic response is the only observable in the set that also marks*completion*\.

### 3\.2The release, and which actuators tolerate it

#### Transient gating suits flow\-shaping actuators; objective\-shaping actuators must persist\.

For the weight\-decay actuator the same release is harmful:3/63/6grokked, mean final test0\.8310\.831: released seeds relapse, because dropping the decay back to0\.10\.1restores the memorisation optimum rather than merely slowing the dynamics\. The loss\-gated \(never released\) schedule is instead the best weight\-decay policy \(6/66/6, mean17,85817\{,\}858\)\. This is the mirror image of the kick section’s dichotomy: a*sustained*Ks​FCDMK^\{\\mathrm\{CDM\}\}\_\{\{\\mathrm\{s\}F\}\}loss fails where the*transient*kick succeeds because the sustained version moves the fixed point; here a*transient*decay boost fails where the*sustained*one succeeds because the decay is part of the objective\. The rule that unifies all four cells: interventions that reshape the*dynamics*\(gradient filtering, the kick\) should be gated transiently and released at completion; interventions that reshape the*objective*\(weight decay\) must persist\. The order parameter times both correctly: it tells the first class when to let go, and the second class when to switch on\.

#### Weight\-norm control is a second objective\-shaping actuator\.

The third known dial obeys the same rule\. Omnigrok\[[21](https://arxiv.org/html/2609.13197#bib.bib21)\]identifies the weight norm as the variable controlling the grokking delay, operating on the initialisation scale\. Turned into an in\-training actuator \(project the full parameter vector back to a sphere of radiusρ​‖θ0‖\\rho\\,\\\|\\theta\_\{0\}\\\|while the gate is active\), it separates sharply by decay regime \(Table[2](https://arxiv.org/html/2609.13197#S3.T2), three seeds throughout\)\. At the paper’s strong decay \(1\.01\.0\) the projection is simply neutral: every radius groks in all three seeds, and the three radius means \(16,50016\{,\}500,20,63320\{,\}633,17,36717\{,\}367\) bracket the plain baseline’s own \(17,81717\{,\}817\)\. At weak decay \(0\.010\.01\), where Omnigrok’s effect lives, plain training never groks and ends at the floor \(0\.0010\.001mean final test accuracy\), and so doρ=0\.5\\rho\{=\}0\.5andρ=0\.7\\rho\{=\}0\.7\. One radius engages, and does so sharply: atρ=0\.3\\rho\{=\}0\.3all three seeds reach0\.790\.79–0\.820\.82final test accuracy within≈3,000\{\\approx\}3\{,\}000steps, two of them crossing the0\.90\.9grok criterion\. The dial is real but narrow: the response is not monotone inρ\\rho, and no setting here reaches the full generalisation the strong\-decay runs attain\.

That operating point makes the dichotomy’s prediction testable after all \(Table[3](https://arxiv.org/html/2609.13197#S3.T3)\)\. A norm constraint shapes the*objective*, so by the rule above its release should harm, and it does\. TheKs​FCDMK^\{\\mathrm\{CDM\}\}\_\{\{\\mathrm\{s\}F\}\}gate and the train\-loss gate reach the same grok step on every seed \(mean3,0503\{,\}050\), but the released version ends at0\.8080\.808against the never\-released0\.9120\.912\. The two seeds on which the release actually fires make the trade explicit: the constrained budget falls from14,25114\{,\}251and13,90013\{,\}900steps to1,7001\{,\}700and1,2501\{,\}250, factors of8\.48\.4and11\.111\.1, and final test accuracy falls with it, from0\.9580\.958and0\.9640\.964to0\.8210\.821and0\.7900\.790\. On the third seedKs​FCDMK^\{\\mathrm\{CDM\}\}\_\{\{\\mathrm\{s\}F\}\}never collapses, the release never fires, and the two policies coincide exactly\. Always\-on control is less reliable and more expensive than theKs​FCDMK^\{\\mathrm\{CDM\}\}\_\{\{\\mathrm\{s\}F\}\}\-gated policy \(1/31/3vs\.2/32/3grokked and25,00125\{,\}001vs\.5,4175\{,\}417constrained steps\)\. Weight\-norm control therefore joins weight decay as a second objective\-shaping actuator rather than standing as an exception\. The order parameter identifies a low\-cost release point, but the subsequent loss of accuracy shows that release is not the correct action for this actuator\. As with the decay schedule, the effective policy is to switch on and maintain the constraint\.

Table 2:Weight\-norm constraint as an in\-training actuator, by decay regime\. Grok step is the mean over the seeds crossing0\.90\.9test accuracy, with the count; final test accuracy is the mean over all three seeds\. Neutral at strong decay; at weak decay onlyρ=0\.3\\rho\{=\}0\.3engages\.Table 3:Gated vs\. blind weight\-norm control at the one engaging operating point \(ρ=0\.3\\rho\{=\}0\.3, weak decay\), three seeds\. TheKs​FCDMK^\{\\mathrm\{CDM\}\}\_\{\{\\mathrm\{s\}F\}\}gate matches the train\-loss gate’s grok step at2\.6×2\.6\\timesless intervention overall \(5,4175\{,\}417vs\.13,81713\{,\}817constrained steps\), but releasing an objective\-shaping constraint costs final accuracy\.

### 3\.3Ablating the sensor’s certified prior

#### The certified prior is not load\-bearing for gate timing\.

The companion paper measures theFFprior’s contribution to the estimator’s codelength at∼0\.5%\{\\sim\}0\.5\\%on its corpus, anO⁡\(1\)O\(1\)\-bit effect\. A controller claim built on a “certified estimator” therefore owes an ablation: we reran theKs​FCDMK^\{\\mathrm\{CDM\}\}\_\{\{\\mathrm\{s\}F\}\}\-gated Grokfast experiment with the sensor’s base measure replaced by the uniform law \(G⁡\[⋅\]=1/2G\[\\cdot\]=1/2for every causal pattern, in place of thes​F\{\\mathrm\{s\}F\}pattern law\), leaving the count\-based context expert otherwise untouched\. The gate’s decisions are bit\-identical on all six seeds \(Table[1](https://arxiv.org/html/2609.13197#S3.T1)\): every amplification window opens and closes at the same check\. The mechanism is plain: on the true addition map thes​F\{\\mathrm\{s\}F\}base measure shifts the sensor by1\.81\.8bits out of501501\(0\.36%0\.36\\%, matching the companion measurement\), and on a random map by0\.00\.0bits out of4,8294\{,\}829; against pattern counts accumulated over961961cells, a unit pseudocount of base measure cannot move a60%60\\%\-collapse threshold\.

What the timing signal actually uses is the estimator’s structure \(the causal\-context mixture that measures the map’s own self\-similarity\), not the enumeration\-derived prior that seeds it\. The claim we can support is therefore narrower: the estimator is certified in the stated sense \(its discrete values are exact prefix\-codelengths, its relaxation agrees on binary corners, and its invariances are proved\)\. This licenses reading its collapse as a reduction in description length under the specified code; but the control\-relevant information would survive an uncertified base measure, so the controller programme stands on the mixture construction, not on theFFenumeration\. This is consistent with, and sharpens, the companion’s finding that the algorithmic prior’s benefit isO⁡\(1\)O\(1\)bits: at control granularity,O⁡\(1\)O\(1\)bits is below the actuation threshold\.

### 3\.4What transfers, and where the domain ends

#### The staircase kick transfers to a second task family: sparse parity\.

Everything above is modular arithmetic, so we port the controller, unchanged, to sparse parity, the canonical hard case for gradient learning\[[17](https://arxiv.org/html/2609.13197#bib.bib17)\]:n=10n\{=\}10input bits, label the XOR of the firstk=5k\{=\}5\(the rest distractors\), an MLP trained on a fraction of the2102^\{10\}inputs, andKs​FCDMK^\{\\mathrm\{CDM\}\}\_\{\{\\mathrm\{s\}F\}\}read from the network’s predicted output map reshaped to a32×3232\\times 32binary field\. We apply the same staircase kick \(fire once fit, release on a relative complexity fall, re\-fire on a stall\), with identical staircase hyperparameters \(release fraction, stall rule, kick cap, pressure schedule\); only the fit tolerance changed with the loss \(BCE<0\.03\\mathrm\{BCE\}<0\.03for the single\-bit output vsCE<0\.01\\mathrm\{CE\}<0\.01\)\. At the data\-scarce margin \(20%20\\%of inputs,40,00040\{,\}000\-step budget, four seeds\) the baseline groks with a heavy\-tailed delay \(steps15,50015\{,\}500,1,5001\{,\}500,6,5006\{,\}500,1,7501\{,\}750across seeds\), while the kicked runs grok at2,7502\{,\}750,1,0001\{,\}000,1,7501\{,\}750,1,0001\{,\}000: every seed accelerates, the slowest by5\.6×5\.6\\times, and the mean falls from∼6,300\{\\sim\}6\{,\}300to∼1,600\{\\sim\}1\{,\}600steps \(Fig\.[5](https://arxiv.org/html/2609.13197#S3.F5)\)\.

Just above the margin \(25%25\\%of inputs\) the baseline delay vanishes and the kick simply matches it \(750750–1,5001\{,\}500vs500500–1,2501\{,\}250steps\), the same largest\-at\-the\-scarce\-margin profile as modular addition’s data×\\timestime sweep, and never a cost\.

The complexity readout repeats the modular\-arithmetic signature: the true parity map scores59\.459\.4bits under the sensor, and the kicked runs drive the map toKs​FCDM=59K^\{\\mathrm\{CDM\}\}\_\{\{\\mathrm\{s\}F\}\}=59bits on three of four seeds \(8282on the fourth\), so the controller again lands on the true\-mapKs​FCDMK^\{\\mathrm\{CDM\}\}\_\{\{\\mathrm\{s\}F\}\}value, while the two slow baseline seeds stall for thousands of steps at intermediate maps of77–10×10\\timesthat complexity \(medianKs​FCDMK^\{\\mathrm\{CDM\}\}\_\{\{\\mathrm\{s\}F\}\}over each seed’s pre\-grok window,430430and602602bits\)\. Parity is, like addition, a task whose true rule is the minimum\-complexity completion of a modest sample \(thek=5k\{=\}5XOR map is5959bits against hundreds for the memorised patchworks\), so Condition[1](https://arxiv.org/html/2609.13197#Thmcondition1)predicts the kick helps, and it does\. The transfer required changing the task, the architecture \(a plainReLU\\mathrm\{ReLU\}MLP on bit vectors\), the output geometry \(a32×3232\\times 32single\-bit field instead of⌈log2⁡p⌉\\lceil\\log\_\{2\}p\\rceilbit\-planes\), and the label structure, and the controller’s behaviour is unchanged: this is evidence the mechanism is the measure, not an artefact of modular tables\.

#### And to a second architecture: a transformer\.

Returning to modular addition atp=31p\{=\}31, a two\-block causal transformer \(token embeddings for the two operands, learned positions,44heads, width128128\) replaces the MLP on the same task, with the same map readout, staircase, and hyperparameters\. Plain training is slower and less reliable than the MLP’s\. By seed, the baseline/kick grok steps for seeds00,11, and22are\>30,000/13,750\>\{\}30\{,\}000/13\{,\}750,36,250/11,50036\{,\}250/11\{,\}500, and15,250/13,75015\{,\}250/13\{,\}750; the second baseline was observed in a run extended to40,00040\{,\}000steps, while the first ended its30,00030\{,\}000\-step run at test0\.820\.82\. Every kicked run drives the map toKs​FCDM=501K^\{\\mathrm\{CDM\}\}\_\{\{\\mathrm\{s\}F\}\}=501bits, the true\-map value\. One caveat: the first seed, rescued from a non\-grokking baseline, relapses partway after release \(final test0\.760\.76after touching0\.9\+0\.9\{\+\}\), a coast\-stability difference between architectures that the MLP does not show; the other two kicked seeds end at0\.990\.99\. The relapse is curable: adding a small validation gate \(200200points reserved from the original held\-out pool and excluded from the reported test set; snapshot the best\-validation state, revert and stop kicking on a0\.150\.15drop\) stabilises every seed \(finals0\.930\.93,1\.01\.0,1\.01\.0\), at the cost of a later grok on the fragile seed \(27,50027\{,\}500\)\. The complexity sensor remains label\-free, but this guard consumes additional labelled validation data\. The order parameter, the kick, and the landing on the true\-mapKs​FCDMK^\{\\mathrm\{CDM\}\}\_\{\{\\mathrm\{s\}F\}\}value survive the change of architecture on all three transformer seeds; the post\-release coast does not, and the guard that fixes it is the one component that needs labels beyond the supervised training set\.

Figure 5:The staircase kick on sparse parity \(n=10n\{=\}10,k=5k\{=\}5,20%20\\%training data; the seed with the slowest baseline\)\.*Left:*test accuracy; the kicked run \(blue\) groks at2,7502\{,\}750steps where the baseline \(grey\) crawls to its transition near15,50015\{,\}500\.*Right:*the same runs’ map complexity: the kick \(orange; kick windows shaded\) drivesKs​FCDMK^\{\\mathrm\{CDM\}\}\_\{\{\\mathrm\{s\}F\}\}down to the true parity map’s5959bits, while the baseline lingers at intermediate\-complexity maps throughout its pre\-grok interval\. Same controller, sensor and schedule as modular arithmetic; only the task changed\.
#### A sustained weight\-spaceKs​FCDMK^\{\\mathrm\{CDM\}\}\_\{\{\\mathrm\{s\}F\}\}loss fails: the controller’s domain is function space\.

DeMoss et al\.\[[14](https://arxiv.org/html/2609.13197#bib.bib14)\]regularise a*weight\-space*complexity \(spectral entropy\) as a training loss and report it induces grokking\. Running their setup with our measure \(softKs​FCDMK^\{\\mathrm\{CDM\}\}\_\{\{\\mathrm\{s\}F\}\}of the min–max–normalised embedding matrices, the only differentiable weight readout available to us\) fails at every amplitude tried: acrossλ∈\{10−3,10−2,10−1\}\\lambda\\in\\\{10^\{\-3\},10^\{\-2\},10^\{\-1\}\\\}\(three seeds each\) no run grokked within the budget, and the final test accuracy sits at the1/31≈0\.031/31\\approx 0\.03chance floor to within a few percentage points \(means0\.040\.04,0\.060\.06,0\.010\.01\), against1/31/3grokked and a mean final accuracy of0\.640\.64for the plain baseline in the same budget\. This is the expected outcome \(the companion paper already found min–max weight readouts to be weak\-to\-anti\-correlated order parameters\), and we report it as the boundary it is:Ks​FCDMK^\{\\mathrm\{CDM\}\}\_\{\{\\mathrm\{s\}F\}\}’s controller role is in*function space*, read from the network’s input–output map, where its collapse marks the transition; pushed into weight space through a readout that is not an order parameter, the same pressure carries no usable signal\. The comparison with spectral entropy is thus not measure\-vs\-measure on common ground but domain\-vs\-domain: purpose\-built weight functionals can act in weight space; our map functional acts on the representation for which its code is defined\.

## 4The calculus, used

Two ingredients of AID’s programme remain: use∇K\\nabla Kas a per\-element classification \(not merely as a loss term\), and explain the controller’s schedule using the system’s measured response rather than tuning alone\. This section does both, and then settles the question the measurements raise on the way, the kick’s amplitude; the answers reshape the programme in the same direction as §[3](https://arxiv.org/html/2609.13197#S3): the fine structure of the algorithmic signal is largely fungible; its*dynamics*\(when the response is large, when it saturates\) is where the control information lives\.

### 4\.1∇K\\nabla Kas a per\-parameter classification

#### The element classification is fungible above a support threshold\.

The differentiable estimator assigns every parameter an algorithmic responsesi=\|∂Ks​Fsoft/∂θi\|s\_\{i\}=\|\\partial K^\{\\mathrm\{soft\}\}\_\{\{\\mathrm\{s\}F\}\}/\\partial\\theta\_\{i\}\|, AID’s element classification applied to the network itself\. We test whether it is load\-bearing: at each kick start the parameters are scored, and the kick’sKK\-gradient is applied only through the top\-qqfraction bysis\_\{i\}, against a uniformly random mask of the same size \(Table[4](https://arxiv.org/html/2609.13197#S4.T4); six seeds per cell, twelve at3%3\\%, same staircase as §[2](https://arxiv.org/html/2609.13197#S2)\)\. Three facts emerge\. First, the kick survives drastic sparsification: at30%30\\%support either selection reproduces the full kick \(8,6258\{,\}625and9,5419\{,\}541vs7,8757\{,\}875steps,6/66/6each, against18,00018\{,\}000,5/65/6, unkicked\)\. Second, the controlling variable is the size of the support, not the selection: at10%10\\%the∇K\\nabla Kand random masks are indistinguishable \(12,16612\{,\}166vs12,20812\{,\}208\)\. The classification earns a visible margin only in the sparse regime, and there its value is chiefly reliability: extending the3%3\\%cells to twelve seeds, the∇K\\nabla Kmask groks11/1211/12against random’s8/128/12, with a modest speed edge on top \(faster in6/76/7seed pairs where both grok, by1,5361\{,\}536steps on average\); at1%1\\%both are weak\.

Third, the classification is coarse and unstable\. It is coarse in that the top\-1%1\\%set lies almost entirely in one module:77%77\\%of its mass falls in the final readout matrix \(mean over1717kicks across six seeds\), so at parameter granularity it says little beyond “the readout layer”\. It is unstable in that consecutive kicks select nearly disjoint sets: Jaccard overlap∼5%\{\\sim\}5\\%on average,1\.51\.5–8\.2%8\.2\\%across the1111consecutive pairs\. The map’s complexity is a function\-space quantity; pulled back to parameter space it attributes broadly and transiently, and almost any sufficiently wide channel transmits the pressure\. This is the parameter\-space twin of the uniform\-prior result of §[3](https://arxiv.org/html/2609.13197#S3): neither the sensor’s certified prior nor the actuator’s∇K\\nabla K\-selected support is what the controller runs on\. What it runs on is measured next\.

Table 4:Sparse algorithmic perturbations \(modular addition,p=31p\{=\}31, fraction0\.40\.4,20,00020\{,\}000\-step budget; six seeds per cell, the3%3\\%cells extended to twelve\): mean grok step \(seeds grokked\) when the kick’sKK\-gradient is restricted to a support of fractionqq, selected by\|∂Ks​Fsoft/∂θi\|\|\\partial K^\{\\mathrm\{soft\}\}\_\{\{\\mathrm\{s\}F\}\}/\\partial\\theta\_\{i\}\|\(top row\) or at random \(middle\)\. Acceleration tracks the support size; the∇K\\nabla Kselection matters only in the sparse regime, where it buys reliability\.

### 4\.2The measured response to the Occam field

#### The response to the Occam field is nucleation\-like\.

No linear regime is resolved over the amplitudes probed\. We apply the field directly rather than inferring its effect from the controller: from a snapshot at timettof a plain trajectory, run300300steps with and without a constant pressureλ\\lambdaonKs​FsoftK^\{\\mathrm\{soft\}\}\_\{\{\\mathrm\{s\}F\}\}\(the “Occam field”\), read the released complexityΔ​K​\(t,λ\)\\Delta K\(t,\\lambda\), and restore the snapshot \(the plain branch is deterministic, so theλ=0\\lambda\{=\}0control is exact\)\. Fig\.[6](https://arxiv.org/html/2609.13197#S4.F6)a showsΔ​K​\(λ\)\\Delta K\(\\lambda\)at three times along a memorised trajectory: att=3,000t\{=\}3\{,\}000the probe releases2\.72\.7bits atλ=10−8\\lambda\{=\}10^\{\-8\},6969at10−710^\{\-7\}, and saturates near2,0002\{,\}000bits above10−510^\{\-5\}, a threshold\-and\-saturation curve with no window in whichΔ​K∝λ\\Delta K\\propto\\lambda\. Byt=13,000t\{=\}13\{,\}000the response is already saturated atλ=10−7\\lambda\{=\}10^\{\-7\}\(≈3,800\\approx 3\{,\}800bits, the map’s entire excess over the true value\), with3,6003\{,\}600released at3×10−83\\times 10^\{\-8\}\. These measurements therefore do not support substituting a fluctuation–dissipation surrogate forχ\\chi: over the tested range, the response is inconsistent with linear response and instead resembles nucleation—below threshold almost nothing; above it the basin breaks and the descent self\-accelerates to completion\.

#### The finite\-field response grows by orders of magnitude toward the transition\.

The optimal kick time rides its plateau\. Fixing a probe in the onset regime \(λ=10−7\\lambda\{=\}10^\{\-7\},300300steps\) defines the operational finite\-field response coefficientχ⁡\(t\)=Δ​K/λ\\chi\(t\)=\\Delta K/\\lambda\. This is a finite ratio, not the zero\-field derivative of linear\-response theory\. We measure it every500500steps along six plain trajectories \(Fig\.[6](https://arxiv.org/html/2609.13197#S4.F6)b\)\. The released complexity climbs from order11–4040bits att=1,000t\{=\}1\{,\}000\(seed\-dependent\) to3,8003\{,\}800–4,2004\{,\}200bits at a peak or plateau\. The location of that peak is likewise seed\-dependent:t≈9,000t\\approx 9\{,\}000for the seed that groks spontaneously at15,25015\{,\}250, as late ast≈16,000t\\approx 16\{,\}000for the slowest\. It always arrives before the visible test\-accuracy rise \(test0\.090\.09–0\.150\.15at every seed’s peak\), and falls once the transition completes, the excess complexity having been spent\. Since the probe saturates near the transition, the measured growth is a lower bound\.

For the operational test \(Fig\.[6](https://arxiv.org/html/2609.13197#S4.F6)c\), we fire a single kick at timeTTand measure the delay from kick start to grok over the same six seeds\. The delay falls broadly withTTand collapses to a single check interval \(250250steps\) onceTTreaches that seed’sχ\\chiplateau: five of six seeds collapse within the sweep, and the sixth is exactly the seed whose plateau arrives last \(t≈16,000t\\approx 16\{,\}000\), its delay down to500500at the sweep’s edge\. Fired there, the kick tips the transition essentially instantly\. Kicks fired early, whereχ\\chiis orders of magnitude smaller, precondition the trajectory but must wait thousands of steps for the yield, and on two seeds a mid\-trajectory single kick fails outright within the budget: a lone pulse with no re\-fire can strand the trajectory, which is precisely the failure mode the staircase’s stall rule exists to catch\. WritingT∗T^\{\\ast\}for the start time that minimises the total step count to grok \(kick start plus delay, the quantity a practitioner would optimise\),T∗T^\{\\ast\}lies between8,0008\{,\}000and14,00014\{,\}000and sits at the front edge of each seed’s plateau: the seed with the earliestχ\\chipeak has the earliestT∗T^\{\\ast\}and the seed with the latest peak the latest, and across the six seedsT∗T^\{\\ast\}tracks theχ\\chi\-peak time at Spearmanρ=0\.84\\rho=0\.84\(p=0\.04p=0\.04\), as the finite\-field\-response picture predicts\.

#### The staircase, explained by the measured response\.

These two measurements account for the empirically tuned schedule\. \(i\) The observed nucleation\-like response provides no measured linear regime from which to design smooth annealing: minimal\-dissipation \(thermodynamic\-length\) protocols presuppose a linear\-response metric, which the present measurements do not establish\. For a threshold response under the standing constraint that the data must stay fit \(a sustained pressure collapses or freezes, §[2](https://arxiv.org/html/2609.13197#S2)\), the natural protocol is bang–bang: full pulses separated by releases, which is the staircase’s shape\. \(ii\) The pulse should fire whereχ\\chiis large, but the plateau’s location is seed\-dependent and not observable in advance; the staircase’s stall rule \(re\-fire whenKs​FCDMK^\{\\mathrm\{CDM\}\}\_\{\{\\mathrm\{s\}F\}\}stagnates above its best\) serves as a causal estimate of “the trajectory is stuck and responsive”, and its release rule \(a relative fall inKs​FCDMK^\{\\mathrm\{CDM\}\}\_\{\{\\mathrm\{s\}F\}\}\) detects that the yield has been realised\. \(iii\) Iterating dominates waiting: the full staircase reaches6,5006\{,\}500–12,00012\{,\}000steps across the six seeds \(mean7,8757\{,\}875\), faster on every seed than that seed’s best single kick at anyTT\(9,2509\{,\}250–14,50014\{,\}500\)\. Because each pulse lowers the map’s complexity, one possible explanation is that it also raises the response available to the next pulse: the staircase may build the plateau rather than wait for it\. We have not, however, measuredχ\\chialong a kicked trajectory\. The schedule that was tuned by hand in §[2](https://arxiv.org/html/2609.13197#S2)is thus consistent with a measured nucleation\-like response whose finite\-field coefficient is seed\-dependent\.

### 4\.3The kick’s amplitude

#### The Occam boundary is a misalignment boundary, not an under\-actuation one\.

The finite\-size\-scaling picture suggested a companion prediction: the minimum effective kick amplitude on multiplication should vanish asf→fc\+f\\to f\_\{c\}^\{\+\}\. Measured, the question turns out to be ill\-posed\. Comparingβmax\\beta\_\{\\max\}three decades apart \(10−810^\{\-8\}and10−510^\{\-5\}, againstβ=0\\beta\{=\}0\) atf∈\[0\.55,0\.8\]f\\in\[0\.55,0\.8\], two seeds each: every amplitude recovers the true table\. The reason is that at these fractions the unkicked dynamics already groks in1,2501\{,\}250–4,7504\{,\}750steps; atβmax=10−8\\beta\_\{\\max\}\{=\}10^\{\-8\}the kicked trajectory is check\-for\-check identical toβ=0\\beta\{=\}0at every fraction and seed\. Larger amplitudes only slow it, monotonically: atf=0\.55f\{=\}0\.55the two\-seed mean rises from4,0004\{,\}000steps plain to6,6256\{,\}625atβmax=10−5\\beta\_\{\\max\}\{=\}10^\{\-5\}, consistent with the data×\\timestime sweep’s kick penalty\. Abovefcf\_\{c\}there is no metastable memorised plateau on multiplication for a kick to tip, soβmin\\beta\_\{\\min\}is trivially zero; belowfcf\_\{c\}no amplitude recovers \(the kick over\-simplifies, §[2](https://arxiv.org/html/2609.13197#S2)\): the failure is misalignment of the razor, not insufficient actuation\. Amplitude thresholds exist only where there is a basin to escape: on*addition*at scarce data, where the response function of Fig\.[6](https://arxiv.org/html/2609.13197#S4.F6)a shows a time\-dependent threshold that falls to nothing as the trajectory approaches its own transition\.

Figure 6:Finite\-field algorithmic response, measured by direct perturbation \(modular addition,p=31p\{=\}31, fraction0\.40\.4\)\.*\(a\)*Complexity released by a300300\-step probe of amplitudeλ\\lambda, at three times along one memorised trajectory: threshold and saturation, with no linear regime resolved over the probed amplitudes\. The response is nucleation\-like, so these measurements do not support a fluctuation–dissipation surrogate forχ\\chi\.*\(b\)*The same probe at fixedλ=10−7\\lambda\{=\}10^\{\-7\}swept along six seeds’ trajectories \(solid, log scale; dotted: test accuracy\):χ\\chigrows by orders of magnitude and peaks/plateaus just before each seed’s spontaneous transition\.*\(c\)*Single kicks fired at timeTT, same six seeds: the delay from kick start to grok collapses to one check interval onceTTreaches that seed’sχ\\chiplateau \(missing points: the kick failed to grok within the budget; a lone pulse can strand the trajectory, which the staircase’s re\-fire rule catches\)\. The staircase \(iterated pulses\) beats every seed’s best single\-TTkick, consistent with, but not proving, the hypothesis that earlier pulses increase the response available to later ones\.

## 5Discussion

#### Timing, not attribution\.

Four independent checks failed to find the controller’s power where the natural reading of “certified algorithmic control” would place it\. The certified prior is not load\-bearing \(a uniform base measure yields bit\-identical gate decisions\); the per\-parameter∇K\\nabla Kattribution is not load\-bearing above a support threshold \(random supports match\); the attribution that does exist is diffuse and transient \(77%77\\%readout,∼5%\{\\sim\}5\\%overlap between kicks\); and the measure did not act through the one weight\-space readout available to us\. None of the cheaper signals tested replicated the temporal structure of the algorithmic signal: the train loss can say “memorised” but never “reorganised”, a fixed step can say neither, and, among these observables, only the complexity collapse marks completion: it is what licenses the release that saves27%27\\%of the intervention budget, what separates dynamics\-shaping from objective\-shaping actuators, and, read as a finite\-field response, what predicts when a kick will tip the transition instantly\. For AID this is a lesson about extending the calculus from objects to dynamics: for objects the calculus classifies*elements*; for dynamics, what it classifies profitably is*moments*\.

#### What, then, does certification buy?

Not the observed gating advantage: the tested uniform\-prior mixture produced the same gate decisions\. It buys the semantics of the readings\. BecauseKs​FCDMK^\{\\mathrm\{CDM\}\}\_\{\{\\mathrm\{s\}F\}\}is an exact prefix\-codelength whose relaxation agrees with it on binary corners and whose invariances are proved\[[2](https://arxiv.org/html/2609.13197#bib.bib2)\], the statement “the kicked run landed at the true\-map complexity” is an equality of description lengths under a specified prefix code, rather than merely an equality of unconstrained scores\. Likewise, interpreting Condition[1](https://arxiv.org/html/2609.13197#Thmcondition1)as an algorithmic Occam condition depends on a measure with a defensible claim to estimate complexity\. Certification makes the reading a falsifiable statement about simplicity under that code, and the ablations then tell us which parts of the apparatus the engineering actually consumes\.

#### Two axes, two scenarios\.

For the nonlinear\-dynamics reader the system exhibits two distinct scenarios in different control parameters\. Along the data axis the Occam boundary exhibits a finite\-size\-scaling pattern:fcf\_\{c\}recedes approximately asln⁡p/p\\ln p/p, the recovery crossover sharpens with system size, and the normalisationnc∼p​ln⁡pn\_\{c\}\\sim p\\ln pis consistent with a coupon\-collector scale \(§[2](https://arxiv.org/html/2609.13197#S2)\)\. Along the*training flow*at fixed data, the memorised state behaves as a*metastable*state: the response to the Occam field is threshold\-and\-saturation, with no linear regime resolved over the probed amplitudes; escape is nucleation\-like and self\-accelerating; and the finite\-field response grows by orders of magnitude as the spontaneous transition approaches \(§[4](https://arxiv.org/html/2609.13197#S4)\)\. The controller’s form follows from which scenario it faces: against metastability the natural protocol is bang–bang pulses, not annealing\. Against misalignment \(belowfcf\_\{c\}\) no protocol helps, because the failure is in the target, not the actuation\.

## 6Limitations and open directions

- •Statistics\.The dissection experiments run11–1212seeds per cell, most cells six \(per\-seed values for the two ablation grids appear in Appendix[A](https://arxiv.org/html/2609.13197#A1)\)\. The headline contrasts include rescue versus non\-rescue, delay collapse on theχ\\chiplateau, and bit\-identical uniform\-prior decisions; however, the close comparisons \(3%3\\%top\-qqvs\. random and single\-TToptima\) require a wider seed pool before fine quantitative claims are warranted\. No inferential uncertainty is claimed from these small samples\. Grok\-step means are computed over grokked seeds only; the grokked counts are always reported alongside so the survivorship is visible\.
- •From measuredχ\\chito a closed\-loop schedule\.§[4](https://arxiv.org/html/2609.13197#S4)accounts for the staircase’s form \(pulses, stall\-fired, release\-on\-yield\) from the measured response, but does not yet optimise its constants: an online controller that estimates the local response from its own recent probe history and sets pulse amplitude and window accordingly is the natural next step\.
- •A label\-free coast guard\.The transformer’s post\-release relapse is cured by a small labelled validation gate \(§[3](https://arxiv.org/html/2609.13197#S3)\), the one component that consumes labels beyond the supervised training set\. Whether a label\-free guard \(e\.g\. re\-fire on aKs​FCDMK^\{\\mathrm\{CDM\}\}\_\{\{\\mathrm\{s\}F\}\}rise above the achieved floor\) can replace it is untested\.
- •Scale of the readout\.All maps here are small enough to enumerate \(p2p^\{2\}cells,2102^\{10\}parity inputs\)\. For large systems the map must be subsampled andKs​FCDMK^\{\\mathrm\{CDM\}\}\_\{\{\\mathrm\{s\}F\}\}read from a patch ensemble; nothing in the estimator forbids this, but the control results have not yet been reproduced in that regime\.
- •Flow, not state\.The controller perturbs the training*flow*\(the loss\), whereas classical AID perturbs the*state*\(§[1](https://arxiv.org/html/2609.13197#S1)\)\. Whether the same timing signal can be recovered from state perturbations, resetting or reinitialising selected parameters and readingΔ​Ks​FCDM\\Delta K^\{\\mathrm\{CDM\}\}\_\{\{\\mathrm\{s\}F\}\}of the resulting map, is untested here\.
- •Weight\-norm control\.It converts into a gateable actuator only in a narrow, non\-monotone window \(ρ=0\.3\\rho\{=\}0\.3under weak decay\), where it confirms the objective\-shaping prediction \(§[3](https://arxiv.org/html/2609.13197#S3)\)\. Why the response is non\-monotone inρ\\rho, and why no setting reaches the full generalisation the strong\-decay runs attain, are untested here; a version acting on the initialisation scale between restarts, closer to Omnigrok’s own, may behave differently\.

## Code and data availability

The complete grokking harness \(tasks, models, controllers\), the drivers for every sweep, and cached results \(JSON\) for the ablation, susceptibility, scaling, parity, transformer and data×\\timestime experiments accompany the manuscript, together with the generators that produce every table and figure\. Two sets of reported numbers are produced by drivers that re\-run rather than cache, and reproducing them needs a GPU: the headline timing means of §[2](https://arxiv.org/html/2609.13197#S2)\(grok\_timing\.py\) and the data\-fraction complexity climb of §[2](https://arxiv.org/html/2609.13197#S2)\(grok\_climb\.py\)\. The six tables and three of the six figures regenerate from the caches without a GPU; the remaining three figures re\-run their trajectories, and the two structural identities of Table[1](https://arxiv.org/html/2609.13197#S3.T1)\(blind\-on weight decay≡\\equivplain baseline; uniform\-prior sensor≡\\equivs​F\{\\mathrm\{s\}F\}gate decisions, both seed for seed\) are asserted by the table generator itself\. The estimator is thepycdmpackage of the companion paper\[[2](https://arxiv.org/html/2609.13197#bib.bib2)\]; setting one environment variable \(AIDCONTROL\_RAW=1\) reproduces every number on the raw machineFFfor the invariance checks quoted in §[2](https://arxiv.org/html/2609.13197#S2)\. The package is archived at10\.5281/zenodo\.0000000\(final DOI to be disclosed\) and released under the MIT licence at the release tagged for this manuscript; the repository URL is to be disclosed\.

## Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper\.

## CRediT authorship contribution statement

Luan Ozelim:Conceptualization, Methodology, Software, Validation, Investigation, Data curation, Formal analysis, Visualization, Writing – original draft, Writing – review & editing\.Hector Zenil:Conceptualization, Supervision, Funding acquisition\.

## Acknowledgements

The authors gratefully acknowledge the support of Oxford Immune Algorithmics and of King’s College London\.

## Appendix AExperimental setup and per\-seed results

#### Tasks and models\.

Modular arithmetic: inputs\(a,b\)∈ℤp2\(a,b\)\\in\\mathbb\{Z\}\_\{p\}^\{2\}, label\(a\+b\)modp\(a\{\+\}b\)\\bmod por\(a⋅b\)modp\(a\\cdot b\)\\bmod p; the training set is a uniformly random fractionffof thep2p^\{2\}cells \(seeded\), the rest held out\. MLP: two embeddings of widthd=128d\{=\}128concatenated, two hiddenReLU\\mathrm\{ReLU\}layers of width256256, linear readout toppclasses\. Transformer \(§[3](https://arxiv.org/html/2609.13197#S3)\): token embeddings for the two operands, learned positional embeddings, two pre\-norm causal blocks \(4 heads, width128128, GELU MLP of width512512\), readout at the last position\. Sparse parity:n=10n\{=\}10input bits, label the XOR of the firstkk; MLP with two hidden layers of width256256\. All training is full\-batch AdamW, learning rate10−310^\{\-3\}, weight decay1\.01\.0unless a schedule or the weak\-decay probe says otherwise\.

#### Readout and grok criterion\.

Ks​FCDMK^\{\\mathrm\{CDM\}\}\_\{\{\\mathrm\{s\}F\}\}is evaluated on the argmax output map over all inputs \(bit\-planes of thep×pp\\times ptable; a32×3232\\times 32field for parity\), every250250steps \(5050in the gated ablation\)\. “Grok step” is the first check at which held\-out accuracy exceeds0\.90\.9; means are over grokked seeds with the grokked count reported alongside\.

#### Staircase constants \(kick, parity, transformer\)\.

Fit toleranceCE<0\.01\\mathrm\{CE\}<0\.01\(BCE<0\.03\\mathrm\{BCE\}<0\.03for parity\); pressure rampβ←β\+2×10−5​\(ε−CE\)\\beta\\\!\\leftarrow\\\!\\beta\+2\\times 10^\{\-5\}\(\\varepsilon\-\\mathrm\{CE\}\)clipped to\[0,3×10−4\]\[0,3\\times 10^\{\-4\}\]; release whenKs​FCDMK^\{\\mathrm\{CDM\}\}\_\{\{\\mathrm\{s\}F\}\}falls to0\.60\.6of the kick\-start value or after3,0003\{,\}000steps; re\-fire after44stalled checks withKCDMs​F\>1\.05×K^\{\\mathrm\{CDM\}\}\_\{\{\\mathrm\{s\}F\}\}\>1\.05\\timesthe best achieved\. Gated ablation \(§[3](https://arxiv.org/html/2609.13197#S3)\): fit gateCE<0\.05\\mathrm\{CE\}<0\.05, release at0\.40\.4of the post\-memorisation peak, fixed gate from step500500; Grokfastg^=g\+λ​EMAα​\(g\)\\hat\{g\}=g\+\\lambda\\,\\mathrm\{EMA\}\_\{\\alpha\}\(g\)withα=0\.9\\alpha\{=\}0\.9,λ=2\\lambda\{=\}2; weight\-decay schedule0\.1/1\.00\.1/1\.0; weight\-norm projection toρ​‖θ0‖\\rho\\,\\\|\\theta\_\{0\}\\\|\.

#### Susceptibility probes\.

Branches restart from a full snapshot \(parameters and optimiser moments\); each branch runs300300steps with constant pressureλ\\lambdaonKs​FsoftK^\{\\mathrm\{soft\}\}\_\{\{\\mathrm\{s\}F\}\}\(λ=0\\lambda\{=\}0: plain\); theλ=0\\lambda\{=\}0branch is deterministic, so the control is exact\. Theχ⁡\(t\)\\chi\(t\)scan probes every500500steps atλ=10−7\\lambda\{=\}10^\{\-7\}\.

#### Budgets, seed pools, and one reconciliation\.

The headline kick numbers of §[2](https://arxiv.org/html/2609.13197#S2)come from dedicated timing runs \(88and66seeds, larger step budgets\); the ablations of §[3](https://arxiv.org/html/2609.13197#S3)–[4](https://arxiv.org/html/2609.13197#S4)use66seeds with20,00020\{,\}000–25,00025\{,\}000\-step budgets \(the3%3\\%cells of Table[4](https://arxiv.org/html/2609.13197#S4.T4): twelve\), and the single\-kick sweep uses66seeds with a22,00022\{,\}000\-step budget\. Because full\-batch trajectories are chaotically sensitive to float\-level reordering, identical configurations run through different code paths can shift individual grok steps by𝒪⁡\(103\)\\mathcal\{O\}\(10^\{3\}\)steps without changing any ordering; all comparisons in this paper are therefore made within one script’s code path \(and the two cross\-path identities we do rely on are asserted exactly; see Code availability\)\. Every experiment runs on one consumer GPU in minutes per trajectory\.

#### Per\-seed grok steps\.

Tables[5](https://arxiv.org/html/2609.13197#A1.T5)and[6](https://arxiv.org/html/2609.13197#A1.T6)list the per\-seed grok steps behind Tables[1](https://arxiv.org/html/2609.13197#S3.T1)and[4](https://arxiv.org/html/2609.13197#S4.T4)\.

Table 5:Per\-seed grok steps for the gated\-vs\-blind ablation of Table[1](https://arxiv.org/html/2609.13197#S3.T1)\(“–”: did not reach test0\.90\.9within the25,00025\{,\}000\-step budget\)\.Table 6:Per\-seed grok steps for the sparse\-∇K\\nabla Kgrid of Table[4](https://arxiv.org/html/2609.13197#S4.T4)\(20,00020\{,\}000\-step budget\)\.

## References

- \[1\]E\. Y\. Sakabe, F\. S\. Abrahão, A\. Simões, E\. Colombini, P\. Costa, R\. Gudwin, and H\. Zenil\.Binarized neural networks converge toward algorithmic simplicity: empirical support for the learning\-as\-compression hypothesis\.*Frontiers in Computational Neuroscience*20, 1791546, 2026\.doi:10\.3389/fncom\.2026\.1791546\.
- \[2\]H\. Zenil and L\. Ozelim\.The Chain Decomposition Method: A Differentiable, Multidimensional Solomonoff Estimator of Kolmogorov Complexity on the Real\-Valued Hypercube\.Preprint, arXiv \(2026\); arXiv identifier to be disclosed\.
- \[3\]A\. N\. Kolmogorov\.Three Approaches to the Quantitative Definition of Information\.*Problems of Information Transmission*, 1\(1\):1–7, 1965\.
- \[4\]R\. J\. Solomonoff\.A Formal Theory of Inductive Inference\. Parts I and II\.*Information and Control*, 7\(1\):1–22 and 7\(2\):224–254, 1964\.
- \[5\]L\. A\. Levin\.Universal Sequential Search Problems\.*Problems of Information Transmission*, 9\(3\):265–266, 1973\.
- \[6\]M\. Li and P\. M\. B\. Vitányi\.*An Introduction to Kolmogorov Complexity and Its Applications*\.Springer, 4th edition, 2019\.
- \[7\]H\. Zenil, N\. A\. Kiani, F\. Marabita, Y\. Deng, S\. Elias, A\. Schmidt, G\. Ball, and J\. Tegnér\.An Algorithmic Information Calculus for Causal Discovery and Reprogramming Systems\.*iScience*, 19:1160–1172, 2019\.
- \[8\]H\. Zenil, N\. A\. Kiani, and J\. Tegnér\.*Algorithmic Information Dynamics: A Computational Approach to Causality with Applications to Living Systems*\.Cambridge University Press, 2023\.
- \[9\]J\.\-P\. Delahaye and H\. Zenil\.Numerical Evaluation of Algorithmic Complexity for Short Strings: A Glance into the Innermost Structure of Randomness\.*Applied Mathematics and Computation*, 219\(1\):63–77, 2012\.
- \[10\]F\. Soler\-Toscano, H\. Zenil, J\.\-P\. Delahaye, and N\. Gauvrit\.Calculating Kolmogorov Complexity from the Output Frequency Distributions of Small Turing Machines\.*PLoS ONE*, 9\(5\):e96223, 2014\.
- \[11\]H\. Zenil, S\. Hernández\-Orozco, N\. A\. Kiani, F\. Soler\-Toscano, A\. Rueda\-Toicen, and J\. Tegnér\.A Decomposition Method for Global Evaluation of Shannon Entropy and Local Estimations of Algorithmic Complexity\.*Entropy*, 20\(8\):605, 2018\.
- \[12\]S\. Hernández\-Orozco, H\. Zenil, J\. Riedel, A\. Uccello, N\. A\. Kiani, and J\. Tegnér\.Algorithmic Probability\-Guided Machine Learning on Non\-Differentiable Spaces\.*Frontiers in Artificial Intelligence*, 3:567356, 2021\.
- \[13\]A\. Power, Y\. Burda, H\. Edwards, I\. Babuschkin, and V\. Misra\.Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets\.*arXiv:2201\.02177*, 2022\.
- \[14\]B\. DeMoss, S\. Sapora, J\. Foerster, N\. Hawes, and I\. Posner\.The Complexity Dynamics of Grokking\.*Physica D: Nonlinear Phenomena*,482, 134859, 2025\.doi:10\.1016/j\.physd\.2025\.134859\.
- \[15\]N\. Nanda, L\. Chan, T\. Lieberum, J\. Smith, and J\. Steinhardt\.Progress Measures for Grokking via Mechanistic Interpretability\.In*The Eleventh International Conference on Learning Representations \(ICLR\)*, 2023\.
- \[16\]Z\. Liu, O\. Kitouni, N\. Nolte, E\. J\. Michaud, M\. Tegmark, and M\. Williams\.Towards Understanding Grokking: An Effective Theory of Representation Learning\.In*Advances in Neural Information Processing Systems 35 \(NeurIPS\)*, 2022\.
- \[17\]B\. Barak, B\. L\. Edelman, S\. Goel, S\. Kakade, E\. Malach, and C\. Zhang\.Hidden Progress in Deep Learning: SGD Learns Parities Near the Computational Limit\.In*Advances in Neural Information Processing Systems 35 \(NeurIPS\)*, 2022\.
- \[18\]V\. Thilak, E\. Littwin, S\. Zhai, O\. Saremi, R\. Paiss, and J\. Susskind\.The Slingshot Mechanism: An Empirical Study of Adaptive Optimizers and the Grokking Phenomenon\.*arXiv:2206\.04817*, 2022\.
- \[19\]V\. Varma, R\. Shah, Z\. Kenton, J\. Kramár, and R\. Kumar\.Explaining Grokking Through Circuit Efficiency\.*arXiv:2309\.02390*, 2023\.
- \[20\]J\. Lee, B\. G\. Kang, K\. Kim, and K\. M\. Lee\.Grokfast: Accelerated Grokking by Amplifying Slow Gradients\.*arXiv:2405\.20233*, 2024\.
- \[21\]Z\. Liu, E\. J\. Michaud, and M\. Tegmark\.Omnigrok: Grokking Beyond Algorithmic Data\.In*The Eleventh International Conference on Learning Representations \(ICLR\)*, 2023\.