The Quantum Shortcut: Complex Phase-State Dynamics Reduce the Optimization Steps of Sequence Models

# Complex Phase-State Dynamics Reduce the Optimization Steps of Sequence Models
Source: [https://arxiv.org/html/2608.14691](https://arxiv.org/html/2608.14691)
Ahmed Nebli cAI Technology GmbH ahmed\.nebli@cai\-technology\.ai&Hadi Saadatdoorabi cAI Technology GmbH hadi\.saadatdoorabi@cai\-technology\.ai&Christopher Keibel Independent Researcher christopher\.keibel\.90@gmail\.com&Kevin Yam Yam Technology Consulting kevin@yam\-consulting\.com

###### Abstract

Sequence models are conventionally distinguished by their backbone, the mechanism that routes information across positions, such as attention or recurrence\. This paper varies a choice that is prior to the backbone and shared by nearly all current models: the*substrate*, the number system in which the hidden state is represented together with the form of the map from state to prediction\. The prevailing substrate is a real\-valued state with an affine–softmax readout; we study a complex\-valued alternative drawn from the mathematics of quantum theory, in which information is carried by the phases of the state and scores are quadratic Born forms\. Prior work proved an idealized version of this substrate representationally stronger than any real model with a linear readout; we ask whether it also trains faster\. Relaxing the two properties that block deployment, exact unitarity and the Born vocabulary readout, we instantiate it in the Mamba state\-space model and an attention\-based Transformer\. At 253M parameters, matched to within0\.02%0\.02\\%and trained under one fixed protocol on three byte\-level corpora, the complex models reach every measured validation loss in approximately one third \(state\-space\) and one half \(attention\) of the optimization steps of their real counterparts\. The two backbones then diverge\. Once the learning\-rate warmup ends, the state\-space advantage continues to widen, from0\.3210\.321to0\.3540\.354bits per character on OpenWebText and from0\.3680\.368to0\.3960\.396on FineWeb, which an artifact of the warmup ramp would not do; the attention advantage instead decays toward zero on every corpus, and is therefore an effect of early training\. The comparison counts optimization steps at equal tokens per step, and for the attention pair the current kernel overhead cancels the step advantage in wall\-clock time\. We analyze the two candidate mechanisms and release all code and training logs\.

*K*eywordssequence modeling⋅\\cdotstate\-space models⋅\\cdotcomplex\-valued networks⋅\\cdotquantum\-inspired methods⋅\\cdotsample efficiency⋅\\cdotoptimization

## 1Introduction

Sequence modeling, the task of predicting the next element of a sequence from the elements that precede it, underlies modern language models and much of contemporary machine learning\. The design of a sequence model is conventionally described by its backbone, the rule by which information from earlier positions reaches later ones\. Transformers aggregate by global self\-attention\[[55](https://arxiv.org/html/2608.14691#bib.bib1)\]; state\-space models propagate a fixed\-dimensional linear recurrence\[[17](https://arxiv.org/html/2608.14691#bib.bib3),[16](https://arxiv.org/html/2608.14691#bib.bib2)\]; long\-convolution models mix positions through learned kernels\[[44](https://arxiv.org/html/2608.14691#bib.bib15)\]\. The trade\-offs among these mechanisms, in expressivity, parallelism, and long\-range retention, have been characterized in detail\[[53](https://arxiv.org/html/2608.14691#bib.bib40),[52](https://arxiv.org/html/2608.14691#bib.bib41),[17](https://arxiv.org/html/2608.14691#bib.bib3)\]\. A second design choice, made implicitly by nearly all of these models, has been varied only rarely \(the exceptions are reviewed in[Section˜4](https://arxiv.org/html/2608.14691#S4)\): the number system in which the hidden state is represented, together with the functional form of the map from state to next\-token distribution\. In the prevailing design the state is a vector inℝd\\mathbb\{R\}^\{d\}and the output map is an affine transformation followed by a softmax\. We call this pair, the field of the state and the form of the readout, the model’s*substrate*, and we treat it as an axis of design distinct from the backbone; whether it is also independent of the backbone is precisely what the experiments test \([Section˜9\.1](https://arxiv.org/html/2608.14691#S9.SS1)\)\.

The substrate matters because of how forgetting works\. A real\-valued model forgets by shrinking the entries of its state, and shrinking a real number degrades the information it carries; attenuation of one percent per position, compounded over a thousand positions, leaves less than0\.005%0\.005\\%of the signal from the earliest positions\. A complex number carries two separable quantities, a magnitude and a phase\. The substrate studied here stores information in the phases, implements forgetting as a contraction of the magnitudes alone, and reads predictions from phase differences through a quadratic readout, under which two contributions cancel when their phases oppose; this is destructive interference, the same phenomenon exploited in acoustic noise cancellation\. Contraction leaves phases unchanged, so stored information fades in amplitude without being distorted, and interference permits evidence to count against a hypothesis rather than merely attenuate it, with a strength set by the participating magnitudes\. The empirical finding is that models built on this substrate reach every quality level we measure in approximately one third \(state\-space backbone\) to one half \(attention backbone\) of the optimization steps of real\-valued models of identical size trained identically, on corpora spanning 100 MB to 15 GB\. The substrate borrows only the mathematics of quantum theory; the models are classical and run on standard accelerators\.

The real, affine–softmax substrate carries two documented limitations, and both are independent of the backbone\. First, an affine map from add\-dimensional state followed by a softmax realizes log\-probability matrices of rank at mostd\+1d\+1\[[61](https://arxiv.org/html/2608.14691#bib.bib13)\]\. When the conditional structure of the data has higher rank, the readout is misspecified no matter how the state is computed\. Second, a real linear recurrence with spectral radiusρ<1\\rho<1attenuates a propagated gradient byρT\\rho^\{T\}overTTsteps, the classical vanishing\-gradient regime\[[2](https://arxiv.org/html/2608.14691#bib.bib42),[40](https://arxiv.org/html/2608.14691#bib.bib43)\]\. At operating points typical of deep state\-space models, sayρ=0\.99\\rho=0\.99andT=1024T=1024, this factor isρT≈3\.4×10−5\\rho^\{T\}\\approx 3\.4\\times 10^\{\-5\}: the gradient path to early positions is effectively severed\. Neither limitation is removed by changing how the model routes information\. The two limitations also indicate where a remedy must act: the readout must escape the rank ceiling, and the recurrence must carry information in a quantity that contraction does not attenuate\. The complex substrate, which pairs a quadratic readout with information stored in phase, does both, and[Section˜6](https://arxiv.org/html/2608.14691#S6)states each point precisely\.

Both limitations bear on training as well as on representation, and they motivate the quantity we measure: the number of optimization steps required to reach a target loss\. This number determines the computation expended and, at a fixed batch size, the volume of data consumed in reaching a given quality; a substrate that reduces it improves the training cost and the sample efficiency of every backbone that adopts it\. As the results will show, the difference between substrates is established in the earliest steps of training and is largest there\. This per\-step quantity, rather than the terminal loss alone, is the primary measure of this study\.

The motivation for changing the field fromℝ\\mathbb\{R\}toℂ\\mathbb\{C\}is a specific structural property: interference\. In a real vector space the combination of two state components is monotone in their magnitudes\. In a complex vector space it is additionally governed by their relative phase, so that two components of fixed magnitude can reinforce or cancel\. A simple example, to which we return throughout the paper, illustrates the consequence for language modeling\. A byte\-level model that has read the characters1 9 4must maintain two readings of its context: the prefix of a year, as in1945, and the prefix of a price, as in19\.4\. A real model holds both readings as magnitudes and, once a later byte settles the question, suppresses the losing reading with a learned gate that shrinks its magnitude toward zero\. A complex model can instead advance the relative phase of the two readings so that, under a quadratic readout, the losing reading interferes destructively with the incoming evidence while the winning one interferes constructively\. The distinction is substantive because cancellation strictly extends attenuation: a gate can only drive a component toward zero, whereas interference allows a component that is still present to count against an outcome\. The same enrichment is what renders alternating\-current analysis, Fourier optics, and quantum mechanics tractable over the complex field\.

The complex substrate was developed in idealized form byNebliet al\.\[[37](https://arxiv.org/html/2608.14691#bib.bib12)\]\. There the hidden state is a unit\-norm vector inℂN\\mathbb\{C\}^\{N\}; its evolution is generated by a learned Hamiltonian and is exactly unitary, discretized by a Cayley \(Crank–Nicolson\) map that preserves the state norm for any step size; and the next\-token distribution is the Born rulepv=\|⟨ev\|ψ⟩\|2p\_\{v\}=\|\\left\\langle e\_\{v\}\\,\\middle\|\\,\\psi\\right\\rangle\|^\{2\}, a quadratic form in the state\. That work proved a representational separation: a family of phase\-disambiguation tasks solved exactly by a complex unitary model of dimensionNNrequires dimensionΩ​\(N2\)\\Omega\(N^\{2\}\)from any real orthogonal model with an affine–softmax readout, because the quadratic readout accesses theΘ​\(N2\)\\Theta\(N^\{2\}\)pairwise phase correlations of the state’s density matrix, which a linear functional of the state cannot resolve; these correlations areΘ​\(N2\)\\Theta\(N^\{2\}\)measurement directions, notΘ​\(N2\)\\Theta\(N^\{2\}\)independent state coordinates, since the pure\-state variety has real dimension2​N−12N\-1\. The separation is representational: it says what each class can express at its optimum and is silent on how training gets there\. Whether the substrate helps a*trained*model, at scale, on natural data, was left open\.

Two properties of the idealized substrate obstruct direct deployment, and this paper studies the consequences of relaxing them\. Exact unitarity admits no forgetting: a norm\-preserving recurrence cannot damp a state component that has ceased to matter, whereas gating has served as the forgetting mechanism of recurrent models since the LSTM\[[25](https://arxiv.org/html/2608.14691#bib.bib44),[13](https://arxiv.org/html/2608.14691#bib.bib45)\], and the ablations ofGu and Dao\[[16](https://arxiv.org/html/2608.14691#bib.bib2)\]attribute the gains of selective state\-space models over their time\-invariant predecessors to exactly this input\-dependent forgetting\. And a Born readout over a vocabulary of sizeVVcostsO​\(N​V\)O\(NV\)per step and ties the state width toV\\sqrt\{V\}if its rank advantage is to be realized, which is unfavorable at subword vocabularies\. We therefore construct a*deployable*substrate: unitarity is relaxed to a contractive, input\-dependent recurrence that keeps the phase of every transition eigenvalue on the unit circle, and the Born rule is applied where it is inexpensive, as the scoring function of a linear\-attention kernel, while the vocabulary readout remains a tied softmax\.[Section˜5](https://arxiv.org/html/2608.14691#S5)states precisely what each relaxation surrenders and what it retains\.

The principal empirical result is that this deployable substrate, at 253M parameters and under a single training protocol, reaches every measured validation loss in approximately one third of the optimization steps of the real Mamba and one half of those of the real Transformer, on three byte\-level corpora spanning a150×150\\timesrange of size\. We refer to this reduction as the*quantum shortcut*\. It is established early in training: H\-Mamba passes 2\.5 bits per character near step 78, a level the real Mamba reaches near step 252, and the ratio of steps remains nearly constant as the target descends from 2\.5 to 2\.0 bits per character\. Because the measured targets are crossed during the learning\-rate warmup, we examine separately what happens once the warmup ends \([Section˜8\.2](https://arxiv.org/html/2608.14691#S8.SS2)\)\. The two backbones behave differently there, and the difference is part of the result: the state\-space gap continues to widen after the ramp stops, while the attention gap decays toward zero\. The acceleration is thus backbone\-independent in the early phase, and, on the present evidence, persistent only for the recurrence\. The design of the comparison addresses the most immediate alternative explanations\. The members of each pair are matched in parameter count to within0\.02%0\.02\\%, which rules out raw parameter count as the driver; the training protocol is identical across models and follows the published recipe at this scale\[[3](https://arxiv.org/html/2608.14691#bib.bib48)\], which argues against a tuning artifact; the stability of the ratio across the descending range of the loss is inconsistent with a transient advantage of initialization; and the recurrence of the effect across two backbones that share almost no computational structure makes a purely backbone\-specific explanation unlikely\. One qualification applies\. The fused kernel of the complex attention currently runs at half the throughput of its real counterpart, so for that pair the reduction in steps yields parity, not savings, in wall\-clock time, and the state\-space comparison lacks an optimized real baseline kernel; the established value of the shortcut is accordingly in optimization steps and data efficiency rather than in wall\-clock time \([Section˜8\.4](https://arxiv.org/html/2608.14691#S8.SS4)\)\.

This paper makes three contributions\. First, we define the deployable complex substrate and characterize its two relaxations, contraction in place of unitarity and Born scoring in place of a Born vocabulary head, stating for each what is surrendered and what is retained \([Section˜5](https://arxiv.org/html/2608.14691#S5)\)\. Second, in the central result of the paper, we establish the quantum shortcut: at matched capacity and under an identical protocol, the substrate reduces the number of steps to every measured target by a factor of approximately three for a state\-space backbone and two for an attention backbone, uniformly over three corpora \([Sections˜7](https://arxiv.org/html/2608.14691#S7)and[8](https://arxiv.org/html/2608.14691#S8);[Figures˜1](https://arxiv.org/html/2608.14691#S1.F1)and[3](https://arxiv.org/html/2608.14691#S8.F3);[Tables˜5](https://arxiv.org/html/2608.14691#S8.T5)and[6](https://arxiv.org/html/2608.14691#S8.T6)\)\. Third, we account for the effect from both directions: an optimization floor relates the rank of an affine–softmax readout to the loss it can attain, an exact\-transport lemma shows that the phase coordinate crosses the contracting recurrence with unit sensitivity, while its observable effect remains scaled by the participating magnitudes \([Section˜6](https://arxiv.org/html/2608.14691#S6)\), and per\-layer gradient measurements exhibit the predicted behavior in the trained models \([Section˜8\.4](https://arxiv.org/html/2608.14691#S8.SS4)\); together these attribute the state\-space acceleration to the recurrence rather than to the readout\. Complete specifications of the architecture, the parallel\-scan and fused\-kernel implementations, and all training logs accompany the paper, so that every reported quantity is re\-derivable \([Appendix˜D](https://arxiv.org/html/2608.14691#A4)\)\. The paper is organized as follows\.[Section˜2](https://arxiv.org/html/2608.14691#S2)fixes notation and defines the three objects used throughout, complex states, unitary evolution, and the Born rule; it assumes no familiarity with quantum mechanics\.[Section˜3](https://arxiv.org/html/2608.14691#S3)recalls the idealized substrate and its representational guarantee, and[Section˜4](https://arxiv.org/html/2608.14691#S4)reviews the six lines of related work\.[Section˜5](https://arxiv.org/html/2608.14691#S5)specifies the deployable substrate one relaxation at a time and closes with a self\-contained summary of the full construction\.[Section˜6](https://arxiv.org/html/2608.14691#S6)states the two theoretical results: an optimization floor for rank\-limited readouts and an exact\-transport lemma for relative phase\.[Sections˜7](https://arxiv.org/html/2608.14691#S7)and[8](https://arxiv.org/html/2608.14691#S8)describe the experimental design and the measurements,[Section˜9](https://arxiv.org/html/2608.14691#S9)examines mechanism, cost, and scaling, and[Section˜10](https://arxiv.org/html/2608.14691#S10)states each limitation together with the experiment that would remove it\. Readers primarily interested in the empirical result may proceed directly to[Sections˜7](https://arxiv.org/html/2608.14691#S7)and[8](https://arxiv.org/html/2608.14691#S8)\.

![Refer to caption](https://arxiv.org/html/2608.14691v1/x1.png)Figure 1:The substrate substitution and its effect\.Left: the deployable complex substrate replaces the real state and affine–softmax scoring of a classical block with a complex state, a diagonal Cayley phase recurrence, and Born\-rule scoring; the rest of the block is unchanged\. Right: validation bits per character against optimization step for the four models on three corpora\. Within the measured range of targets \(2\.52\.5to2\.02\.0bits per character\), each complex variant \(solid\) reaches every value of the loss in one half to one third of the steps of its real counterpart \(dashed\), and the separation is established within the first200200steps\. On enwik8, the smallest corpus, the complex models saturate the data and overfit beyond the budget, and the rotary Transformer baseline finishes below H\-Transformer \([Section˜8\.3](https://arxiv.org/html/2608.14691#S8.SS3)\)\.
## 2Preliminaries and Notation

We fix notation and recall the algebraic objects used throughout\. The material is standard and is developed at length inNebliet al\.\[[37](https://arxiv.org/html/2608.14691#bib.bib12)\]; we restate only what makes the paper self\-contained, and we define every object before using it\. Readers familiar with Dirac notation and density matrices may proceed to[Table˜1](https://arxiv.org/html/2608.14691#S2.T1), which summarizes the correspondence between the idealized substrate of[Section˜3](https://arxiv.org/html/2608.14691#S3)and the deployable substrate of[Section˜5](https://arxiv.org/html/2608.14691#S5)\.

#### Complex states and the density matrix\.

The state space is the complex Hilbert spaceℂN\\mathbb\{C\}^\{N\}with the Hermitian inner product⟨ϕ\|ψ⟩=∑kϕ¯k​ψk\\left\\langle\\phi\\,\\middle\|\\,\\psi\\right\\rangle=\\sum\_\{k\}\\bar\{\\phi\}\_\{k\}\\psi\_\{k\}\. A state\|ψ⟩\\left\|\\psi\\right\\rangleis a column vector and⟨ψ\|\\left\\langle\\psi\\right\|its conjugate transpose\. Each state has a density matrixρ=\|ψ⟩​⟨ψ\|∈ℂN×N\\rho=\\left\|\\psi\\right\\rangle\\left\\langle\\psi\\right\|\\in\\mathbb\{C\}^\{N\\times N\}: a rank\-one, Hermitian, positive semidefinite operator withtr⁡ρ=‖ψ‖2\\operatorname\{tr\}\\rho=\\\|\\psi\\\|^\{2\}\. Its diagonal entriesρk​k=\|ψk\|2\\rho\_\{kk\}=\|\\psi\_\{k\}\|^\{2\}are per\-coordinate occupation magnitudes\. Its off\-diagonal entriesρj​k=ψ¯j​ψk\\rho\_\{jk\}=\\bar\{\\psi\}\_\{j\}\\psi\_\{k\}carry the pairwise relative phases\. The off\-diagonals are the coordinates that a quadratic functional of the state can read and a linear one cannot, a fact that recurs throughout the paper\.

#### Unitary evolution and the Cayley transform\.

An operatorU∈ℂN×NU\\in\\mathbb\{C\}^\{N\\times N\}is unitary ifU†​U=IU^\{\\dagger\}U=I; it then preserves inner products and norms, the complex analogue of a rotation\. The Cayley transform parameterizes the unitary group by skew\-Hermitian matrices \(A†=−AA^\{\\dagger\}=\-A\),

U=\(I−A\)−1​\(I\+A\),U=\(I\-A\)^\{\-1\}\(I\+A\),\(1\)and is defined and unitary for every skew\-HermitianAA: the spectrum ofAAis purely imaginary, soI−AI\-Ais always invertible \(an exclusion condition arises only for the inverse map, which requires−1\-1not to be an eigenvalue ofUU\)\. ForA=−i2​H​Δ​tA=\-\\tfrac\{i\}\{2\}H\\,\\Delta twithHHHermitian, it coincides with the Crank–Nicolson \(implicit midpoint\) discretization\[[9](https://arxiv.org/html/2608.14691#bib.bib7)\]of the Schrödinger evolution\|ψ⟩˙=−i​H​\|ψ⟩\\dot\{\\left\|\\psi\\right\\rangle\}=\-iH\\left\|\\psi\\right\\rangle\. Unlike an explicit integrator, the Cayley transform preserves the norm exactly for any step sizeΔ​t\\Delta t; the phase trajectory is second\-order accurate inΔ​t\\Delta t\(local errorO​\(Δ​t3\)O\(\\Delta t^\{3\}\)\), the standard accuracy of the implicit midpoint rule\. WhenAAis diagonal, the transform acts coordinatewise and maps each imaginary eigenvalue ofAAonto the unit circle by a Möbius map\.

#### The Born rule\.

Given measurement vectors\{\|ev⟩\}v=1V⊂ℂN\\\{\\left\|e\_\{v\}\\right\\rangle\\\}\_\{v=1\}^\{V\}\\subset\\mathbb\{C\}^\{N\}, the Born rule\[[38](https://arxiv.org/html/2608.14691#bib.bib25)\]assigns outcomevvthe probabilitypv=\|⟨ev\|ψ⟩\|2=tr⁡\(\|ev⟩​⟨ev\|​ρ\)p\_\{v\}=\|\\left\\langle e\_\{v\}\\,\\middle\|\\,\\psi\\right\\rangle\|^\{2\}=\\operatorname\{tr\}\\\!\\big\(\\left\|e\_\{v\}\\right\\rangle\\left\\langle e\_\{v\}\\right\|\\,\\rho\\big\)\. As a function of the state this is a real quadratic form; as a function ofρ\\rhoit is linear\. That a linear functional ofρ\\rhois a quadratic functional of\|ψ⟩\\left\|\\psi\\right\\rangleis the observation on which the separation of[Section˜3](https://arxiv.org/html/2608.14691#S3)rests\. Throughout,σ​\(⋅\)\\sigma\(\\cdot\)is the logistic function,⊙\\odotthe Hadamard product, andtr\\operatorname\{tr\}the trace;TTis the sequence length,NNthe per\-head state width,HHthe number of heads, andVVthe vocabulary size\. These three objects, the complex state, the unitary map, and the quadratic measurement, are the entire mathematical inventory of the paper\.

Table 1:Correspondence between the idealized wave\-function substrate ofNebliet al\.\[[37](https://arxiv.org/html/2608.14691#bib.bib12)\], the deployable substrate of this paper, and their roles in the model\.

## 3The Wave\-Function Substrate and Its Idealization

[Section˜1](https://arxiv.org/html/2608.14691#S1)claimed that storing information in phase and reading it out quadratically yields a training advantage\. Both ingredients were first assembled, in exact form, in the idealized model ofNebliet al\.\[[37](https://arxiv.org/html/2608.14691#bib.bib12)\]; the deployable substrate of this paper is a controlled relaxation of that model, and its relaxations are best understood against the model they relax\. This section states the idealized substrate and the guarantee it enjoys, in the notation of[Section˜2](https://arxiv.org/html/2608.14691#S2), and refers to the cited work for constructions and proofs\.

The idealized substrate couples three components\. The state is a unit\-norm vector\|ψt⟩∈ℂN\\left\|\\psi\_\{t\}\\right\\rangle\\in\\mathbb\{C\}^\{N\}\. Its evolution is\|ψt\+1⟩=Ut​\|ψt⟩\\left\|\\psi\_\{t\+1\}\\right\\rangle=U\_\{t\}\\left\|\\psi\_\{t\}\\right\\rangle, whereUtU\_\{t\}is generated by a learned Hermitian HamiltonianHt=H0\+Hint​\(xt\)H\_\{t\}=H\_\{0\}\+H\_\{\\mathrm\{int\}\}\(x\_\{t\}\): a diagonal free termH0H\_\{0\}fixes a spectrum of baseline oscillation frequencies, and an input\-dependent interactionHint​\(xt\)H\_\{\\mathrm\{int\}\}\(x\_\{t\}\)couples the input to the dynamics\. The Cayley discretization \([1](https://arxiv.org/html/2608.14691#S2.E1)\) makes eachUtU\_\{t\}exactly unitary, so‖ψt‖≡1\\\|\\psi\_\{t\}\\\|\\equiv 1at every step and for every sequence length\. The readout is the Born rule over a learned measurement basis\. BecauseHtH\_\{t\}is Hermitian by construction, norm preservation needs no penalties or projections, and the backward operator inherits the same unit modulus: gradients are neither amplified nor attenuated as they traverse the recurrence\.

What distinguishes the idealized substrate is not norm preservation alone but the interaction between the complex state and the quadratic readout\. Sincepv=tr⁡\(\|ev⟩​⟨ev\|​ρ\)p\_\{v\}=\\operatorname\{tr\}\(\\left\|e\_\{v\}\\right\\rangle\\left\\langle e\_\{v\}\\right\|\\rho\)is linear inρ\\rho, the family of Born readouts spans the fullN2N^\{2\}\-dimensional real vector space of Hermitian measurement functionals, whereas linear readouts of the state span only a2​N2N\-real\-dimensional family\. Two precisions apply: the pure\-state image\{\|ψ⟩​⟨ψ\|\}\\\{\\left\|\\psi\\right\\rangle\\left\\langle\\psi\\right\|\\\}is a variety of real dimension2​N−12N\-1, so theN2N^\{2\}entries ofρ\\rhoare algebraically dependent as features, and the countN2N^\{2\}refers to independent measurement directions, not independent state coordinates\.Nebliet al\.\[[37](https://arxiv.org/html/2608.14691#bib.bib12)\]convert this dimensional gap into a lower bound\. A family of phase\-disambiguation tasks realized exactly by a complex unitary model of dimensionNNrequires state dimensionΩ​\(N2\)\\Omega\(N^\{2\}\)from any real model with orthogonal dynamics and an affine–softmax readout; the real model must spend an explicit state coordinate on each pairwise phase correlation\. The bound is representational\. It concerns what each class expresses at its optimum and says nothing about how the optimum is reached; moreover, it relies on assumptions, exact unitarity, a Born vocabulary readout, and orthogonal real dynamics, that the deployable model of this paper does not satisfy\. We therefore use it strictly as motivation, and no part of the guarantee transfers to the architectures trained here\. This paper addresses the training side, and[Section˜6](https://arxiv.org/html/2608.14691#S6)supplies the corresponding optimization statement\.

The mechanism the guarantee formalizes is interference, and the two\-reading example of[Section˜1](https://arxiv.org/html/2608.14691#S1)is an instance of it; the word\-level analogue, disambiguating an occurrence of “bank” by a later token, opensNebliet al\.\[[37](https://arxiv.org/html/2608.14691#bib.bib12)\]\. Two admissible readings of a context are encoded in two components of the state, and a later token is compatible with one reading but not the other\. A real model suppresses the losing reading with a gate, a learned multiplicative mask that shrinks the disfavored component toward zero\. The complex substrate suppresses it through the evolution itself: the recurrence advances the relative phase of the two components so that, under the quadratic readout, the losing reading interferes destructively with the incoming evidence while the winning one interferes constructively\. Attenuation can only shrink a component, whereas interference can reverse its sign at the readout; the latter is a strictly larger repertoire of operations, available without a dedicated gating module\.

With the idealized substrate in place, two questions arise: where this construction stands relative to prior work on complex, unitary, and quantum\-inspired models, and what must change before it can train inside a production backbone\.[Section˜4](https://arxiv.org/html/2608.14691#S4)answers the first question,[Section˜5](https://arxiv.org/html/2608.14691#S5)the second\.

## 4Related Work

The substrate assembled in[Section˜5](https://arxiv.org/html/2608.14691#S5)borrows from several literatures: its recurrence and scan come from the state\-space line, its parameterization from the unitary line, its second host architecture from linear attention, its integrator from structure\-preserving numerics, and its readout from quantum\-inspired modeling\. This section reviews each in enough detail to make the borrowings, and the departures, exact\.

### 4\.1Structured state\-space models

The state\-space line grew out of the memory problem for recurrent networks\. HiPPO\[[18](https://arxiv.org/html/2608.14691#bib.bib50)\]derived state matrices from online polynomial approximation of the input history, giving fixed\-dimensional recurrences with provable memory of the past\. S4\[[17](https://arxiv.org/html/2608.14691#bib.bib3)\]made these systems trainable at depth by parameterizing a stable linear time\-invariant system and evaluating its sequence map as a convolution, setting the standard on long\-range benchmarks such as the Long Range Arena\[[52](https://arxiv.org/html/2608.14691#bib.bib41)\]\. Two simplifications followed\. DSS\[[20](https://arxiv.org/html/2608.14691#bib.bib51)\]and S4D\[[19](https://arxiv.org/html/2608.14691#bib.bib52)\]showed that diagonal state matrices recover the performance of the full structured system once the initialization is chosen correctly, and S5\[[47](https://arxiv.org/html/2608.14691#bib.bib14)\]showed that a diagonal recurrence evaluated by an associative parallel scan suffices, at a fraction of the implementation complexity\. The linear recurrent unit\[[39](https://arxiv.org/html/2608.14691#bib.bib22)\]completed this reduction by asking which ingredients are essential: it parameterizes complex\-diagonal eigenvalues by log\-magnitude and phase, initializes them on a ring close to the unit circle, and identifies the input normalization that makes deep stacks of bare linear recurrences trainable\.

A second thread carried the line to language\. H3\[[12](https://arxiv.org/html/2608.14691#bib.bib53)\]diagnosed the gap between state\-space models and attention on associative\-recall tasks and closed part of it with a hybrid design; Hyena\[[44](https://arxiv.org/html/2608.14691#bib.bib15)\]replaced attention with long implicit convolutions; Mamba\[[16](https://arxiv.org/html/2608.14691#bib.bib2)\]made the recurrence parameters functions of the current token, so that the model can selectively write, retain, or discard, and paired this selective recurrence with a hardware\-aware scan, reporting language modeling competitive with Transformers at linear\-time inference\. Mamba\-2\[[11](https://arxiv.org/html/2608.14691#bib.bib26)\]exhibited the selective recurrence as a form of structured masked attention, unifying the two families\.

The recurrence \([2](https://arxiv.org/html/2608.14691#S5.E2)\) is a member of this family: a diagonal, input\-dependent linear recurrence, evaluated by the same associative scan \([Section˜5\.3](https://arxiv.org/html/2608.14691#S5.SS3)\), hosted in Mamba’s own block \([Section˜5\.7](https://arxiv.org/html/2608.14691#S5.SS7)\)\. The departure is confined to a single design decision\. Every model above keeps its eigenvalues strictly inside the unit disc, so stability, forgetting, and memory all hang on one quantity,\|λ\|<1\|\\lambda\|<1: the recurrence forgets by attenuating, and the same attenuation erodes what it remembers\. The tension is visible within the line itself, in the LRU’s finding that eigenvalues must crowd the unit circle for long memory yet stay inside it for stability\. The substrate studied here dissolves the tension by splitting the roles: the Cayley map holds the phase of every eigenvalue exactly on the circle, a separate sigmoid gate supplies the decay, and the information rides in the phase, which the decay never touches\. The remaining difference is the readout: the models above read their state linearly, which leaves relative phases unobservable, whereas the Born score of[Section˜5\.5](https://arxiv.org/html/2608.14691#S5.SS5)reads them directly\. H\-Mamba is Mamba with this substitution and nothing else\.

### 4\.2Complex\-valued and unitary networks

Complex\-valued networks predate deep learning; the signal\-processing and neural\-network literature developed complex arithmetic, activation functions, and training rules for them over several decades\[[36](https://arxiv.org/html/2608.14691#bib.bib29),[24](https://arxiv.org/html/2608.14691#bib.bib30)\]\. Within deep learning, the unitary RNN\[[1](https://arxiv.org/html/2608.14691#bib.bib4)\]introduced complex recurrences to solve a conditioning problem: constraining the recurrent operator to a product of structured unitary factors \(diagonal phases, Householder reflections, Fourier transforms\) fixes the spectral norm of the recurrent Jacobian at one and eliminates exploding and vanishing gradients by construction\. EUNN\[[28](https://arxiv.org/html/2608.14691#bib.bib54)\]made the parameterization tunable in capacity, the full\-capacity uRNN\[[56](https://arxiv.org/html/2608.14691#bib.bib5)\]optimized over the entire unitary group, scoRNN\[[23](https://arxiv.org/html/2608.14691#bib.bib55)\]obtained orthogonal recurrent matrices through a scaled Cayley transform, andLezcano\-Casado and Martínez\-Rubio\[[33](https://arxiv.org/html/2608.14691#bib.bib20)\]gave exact, inexpensive trivializations of the orthogonal and unitary groups; geometric numerical integration\[[22](https://arxiv.org/html/2608.14691#bib.bib8)\]supplies the discretizations that keep learned dynamics on the group\. Deep complex networks\[[54](https://arxiv.org/html/2608.14691#bib.bib6)\]extended convolution, batch normalization, and initialization toℂ\\mathbb\{C\}, complex gated RNNs\[[57](https://arxiv.org/html/2608.14691#bib.bib56)\]combined complex arithmetic with gating, and complex attention has been developed for inherently complex\-valued data such as audio and radar\[[59](https://arxiv.org/html/2608.14691#bib.bib38)\]; tensor\-network models\[[49](https://arxiv.org/html/2608.14691#bib.bib39)\]apply the adjacent multilinear algebra to supervised learning\.

This line settled two questions on which our construction depends: complex arithmetic trains stably at depth, and unitarity removes gradient pathologies\. It also left two questions open, and they define what we do here\. First, exact unitarity forbids forgetting, and the models of this line were accordingly validated on memory benchmarks, copying, adding, and pixel\-by\-pixel classification, rather than on competitive language modeling; the constraint that stabilizes gradients also freezes stale state\. The substrate here keeps the unit\-modulus phase but moves the modulus into a separate learned gate, so that forgetting returns without sacrificing the transport property, which[Lemma˜6\.2](https://arxiv.org/html/2608.14691#S6.Thmtheorem2)states exactly\. Second, every model in this line reads its state through a linear map, so theΘ​\(N2\)\\Theta\(N^\{2\}\)pairwise phase correlations a complex state carries are not resolved at the output; the complex structure works as a conditioning device, not as an information channel\. Even the Cayley transform, where it appears\[[23](https://arxiv.org/html/2608.14691#bib.bib55),[33](https://arxiv.org/html/2608.14691#bib.bib20)\], serves to keep a real recurrent matrix orthogonal; here the same map is applied coordinatewise to keep individual phases on the circle while the Born score turns those phases into predictions\.

### 4\.3Linear attention and relative position encodings

Linear attention\[[31](https://arxiv.org/html/2608.14691#bib.bib10)\]replaces the softmax kernel with a positive feature map, which factorizes the attention matrix and turns autoregressive attention into a linear recurrence withO​\(T\)O\(T\)inference; Performers\[[8](https://arxiv.org/html/2608.14691#bib.bib57)\]construct random features that approximate the softmax kernel without bias\. A subsequent generation attached forgetting to this recurrence: RetNet\[[51](https://arxiv.org/html/2608.14691#bib.bib28)\]with a fixed exponential decay, RWKV\[[43](https://arxiv.org/html/2608.14691#bib.bib27)\]with an attention\-free mixing rule in the same spirit, and gated linear attention\[[60](https://arxiv.org/html/2608.14691#bib.bib31)\]with data\-dependent gates and a hardware\-efficient training algorithm\. In parallel, rotary position embeddings\[[50](https://arxiv.org/html/2608.14691#bib.bib32)\]became the standard way to encode relative position: queries and keys are rotated in two\-dimensional subspaces through angles proportional to absolute position, so that their inner product depends on position only through differences\. A rotation of a two\-dimensional subspace is multiplication by a unit\-modulus complex number, so every model with rotary embeddings already stores position in phase\.

These two threads each contain half of our attention realization\. The gated linear\-attention thread maintains a recurrent state with a learned decay gate, exactly the role the gateα\\alphaplays in \([2](https://arxiv.org/html/2608.14691#S5.E2)\), but scores with nonnegative feature maps, which can only accumulate evidence\. The rotary thread stores information in phase, but with frequencies fixed by schedule, phases that encode position alone, and a score that is, in complex notation, the real part of a complex inner product, a signed but linear functional of the query–key overlaps\. Neither thread scores with a quadratic form, so in neither can one overlap cancel another\. The realization of[Section˜5\.5](https://arxiv.org/html/2608.14691#S5.SS5)combines the halves and adds the missing nonlinearity: learned frequencies whose phases participate in the computation, a decay gate for forgetting, and the squared\-modulus score\|⟨qi\|kj⟩\|2\|\\left\\langle q\_\{i\}\\,\\middle\|\\,k\_\{j\}\\right\\rangle\|^\{2\}, whose cross terms make interference available in the attention weights themselves, at a strength scaled by the participating magnitudes\.

### 4\.4Hamiltonian, oscillatory, and energy\-preserving networks

A fourth line builds conservation and oscillation into network dynamics\. Hamiltonian neural networks\[[15](https://arxiv.org/html/2608.14691#bib.bib34)\]learn a scalar energy whose induced flow the model follows, conserving the learned energy along trajectories; symplectic recurrent networks\[[7](https://arxiv.org/html/2608.14691#bib.bib35)\]integrate learned separable Hamiltonians with leapfrog integrators and show that the integrator, not the Hamiltonian form alone, secures stable learned dynamics\. On the recurrent side,Haber and Ruthotto\[[21](https://arxiv.org/html/2608.14691#bib.bib36)\]pose forward propagation as a well\-posed differential equation whose stability is controlled through the spectrum of the transition operator; AntisymmetricRNN\[[5](https://arxiv.org/html/2608.14691#bib.bib37)\]instantiates this with an antisymmetric weight matrix, whose purely imaginary spectrum is the real\-valued analogue of unitary evolution; and coRNN\[[46](https://arxiv.org/html/2608.14691#bib.bib58)\]builds the recurrence from a network of controlled, damped oscillators, with proven bounds on gradient growth and strong results on long\-dependency benchmarks\.

The oscillatory members of this line come closest to the substrate studied here: coRNN, like the recurrence here, uses oscillation as the vehicle of memory, and our per\-channel frequenciesωk\\omega\_\{k\}have their counterpart in its oscillator frequencies\. The difference is what a real oscillator can carry\. A real oscillation stores its state in a position–velocity pair whose amplitude decays under damping, so information and decay share coordinates, and a linear readout of those coordinates sees amplitudes, not phase relations\. A complex coordinate is the canonical pair packaged as one number: the gate damps its modulus while the argument advances undisturbed, and the quadratic readout of[Section˜5\.5](https://arxiv.org/html/2608.14691#S5.SS5)converts the arguments’ differences into probabilities\. Conservation likewise changes role\. In this line the conserved quantity is a real energy or norm, conserved as physical fidelity or as a stability device; here the conserved quantity is the phase alone, the norm is deliberately not conserved so that the model can forget, and the conserved quantity is what the model predicts with\.

### 4\.5Continuous\-time and quantum\-inspired models

Neural ordinary differential equations\[[6](https://arxiv.org/html/2608.14691#bib.bib33)\]formalize deep networks as discretizations of continuous\-time systems and make explicit that a learned dynamical system is shaped jointly by its vector field and its integrator\. That lesson fixes a choice made here: the Crank–Nicolson discretization\[[9](https://arxiv.org/html/2608.14691#bib.bib7)\]of[Section˜2](https://arxiv.org/html/2608.14691#S2)preserves the unitary structure of the continuous flow exactly, at every step size, where an explicit integrator would preserve it only to first order\.

Quantum\-inspired modeling of language predates deep learning as well\. Quantum language models for information retrieval\[[48](https://arxiv.org/html/2608.14691#bib.bib59)\]represented queries and documents as density matrices and scored them with Born\-type traces, capturing term dependencies that bag\-of\-words models miss; this is, to our knowledge, the earliest use in language processing of the density\-matrix lift that[Section˜3](https://arxiv.org/html/2608.14691#S3)exploits\. Quantum cognition\[[4](https://arxiv.org/html/2608.14691#bib.bib60)\]documented systematic violations of classical probability in human judgment that quantum probability describes, motivating interference as an inductive bias for ambiguity; and a separate engineering line develops variational circuits for quantum hardware\[[45](https://arxiv.org/html/2608.14691#bib.bib21)\], which shares formalism but not purpose with classical quantum\-inspired models\. The framework this paper builds on\[[37](https://arxiv.org/html/2608.14691#bib.bib12)\]belongs to the classical line and assembled the full package for sequence modeling: a complex state, norm\-preserving Hamiltonian evolution, a Born\-rule readout, theΩ​\(N2\)\\Omega\(N^\{2\}\)separation theorem recalled in[Section˜3](https://arxiv.org/html/2608.14691#S3), and conserved probability currents for tracing information flow\. What that work left open, and what we supply here, is the training side: whether the substrate, once relaxed enough to live inside production backbones, changes how fast such models learn\. Everything from[Section˜5](https://arxiv.org/html/2608.14691#S5)onward addresses that question at 253M parameters\.

### 4\.6The softmax bottleneck

Yanget al\.\[[61](https://arxiv.org/html/2608.14691#bib.bib13)\]proved that an affine–softmax head of widthddproduces log\-probability matrices of rank at mostd\+1d\+1, so a model whose true conditional structure has higher rank is misspecified no matter how its backbone computes the state; their mixture of softmaxes restores rank by combiningKKsoftmax components at aKK\-fold readout cost\. Follow\-up work varied the fix while keeping the diagnosis: Sigsoftmax\[[29](https://arxiv.org/html/2608.14691#bib.bib61)\]modifies the output nonlinearity, and Mixtape\[[62](https://arxiv.org/html/2608.14691#bib.bib62)\]replaces the mixture with an efficient gating construction\. All of these remedies operate on a real state and raise rank by complicating the output function\.

In the vocabulary of this paper, the bottleneck is a substrate property, the first of the two limitations with which[Section˜1](https://arxiv.org/html/2608.14691#S1)opened, and the line bears on this paper twice\.[Proposition˜6\.1](https://arxiv.org/html/2608.14691#S6.Thmtheorem1)sharpens the diagnosis from expressibility to optimization: where the head is narrower than the rank of the target, no training trajectory through an affine–softmax head reaches the optimum, by a margin the tail spectrum quantifies\. That condition is not met at the byte vocabulary of our experiments \([Section˜6\.1](https://arxiv.org/html/2608.14691#S6.SS1)\), so the sharpened statement is offered as general motivation\. And the mixture of softmaxes supplies the natural control for our readout mechanism: it raises rank without complex numbers, so if it closed the gap we measure on the state\-space backbone, the readout rather than the recurrence would be implicated;[Section˜9\.2](https://arxiv.org/html/2608.14691#S9.SS2)places this control alongside the factorial ablation\. The Born readout of the idealized substrate raises the accessible rank to orderN2N^\{2\}by changing the functional form of the readout instead of mixing copies of it\.

Across these six literatures, the individual ingredients of the deployable substrate all appear: diagonal scans, Cayley parameterizations, decay gates, phase\-encoded structure, density\-matrix readouts, and rank\-raising output layers\. Their combination, a contractive recurrence whose eigenvalue phases are exactly unit\-modulus, feeding a quadratic score, at parameters matched to a production baseline, does not, and neither does a measurement of what that combination changes about optimization\. Those are the subjects of the next four sections\.

## 5The Deployable Complex Substrate

Embedding the idealized substrate in a contemporary backbone requires relaxing its two most restrictive properties: exact unitarity and the Born vocabulary readout\. Each is in tension with a mechanism that trained sequence models rely on\. This section specifies the relaxed substrate and states, for each relaxation, what is surrendered and what is retained\. The substrate is defined per head; a model usesH=16H=16heads of widthN=16N=16, and the two architectural realizations that host it are given in[Section˜5\.7](https://arxiv.org/html/2608.14691#S5.SS7)\. The subsections follow the order of a forward pass, from the state through its evolution to the readout, and[Section˜5\.8](https://arxiv.org/html/2608.14691#S5.SS8)closes with a summary of the full specification\.

### 5\.1State

In place of the real hidden vector𝐡t∈ℝd\\mathbf\{h\}\_\{t\}\\in\\mathbb\{R\}^\{d\}, each head carries a complex vector\|ψt⟩∈ℂN\\left\|\\psi\_\{t\}\\right\\rangle\\in\\mathbb\{C\}^\{N\}, stored as real and imaginary parts:2​N2Nreal numbers per head\. The state is initialized from a learned complex prior normalized to unit modulus, and the choice is deliberate\. Were the state initialized on the real axis, every off\-diagonal entryρj​k=ψ¯j​ψk\\rho\_\{jk\}=\\bar\{\\psi\}\_\{j\}\\psi\_\{k\}of the density matrix would be real at the first step\. The interference cross terms would vanish identically, and early optimization would be spent rotating the state off the real axis before any phase\-dependent computation could begin\. A complex prior places nontrivial phase in the state from the first token\. The idealized substrate keeps‖ψt‖=1\\\|\\psi\_\{t\}\\\|=1at every step; the deployable substrate normalizes only the prior and lets the norm vary thereafter\. That is the first relaxation, and it is the subject of the next subsection\.

### 5\.2Evolution and the relaxation of unitarity

The state evolves by a diagonal, input\-dependent linear recurrence,

\|ψt\+1⟩=at⊙\|ψt⟩\+bt,at=αt⊙λt∈ℂN,\\left\|\\psi\_\{t\+1\}\\right\\rangle=a\_\{t\}\\odot\\left\|\\psi\_\{t\}\\right\\rangle\+b\_\{t\},\\qquad a\_\{t\}=\\alpha\_\{t\}\\odot\\lambda\_\{t\}\\in\\mathbb\{C\}^\{N\},\(2\)wherebt∈ℂNb\_\{t\}\\in\\mathbb\{C\}^\{N\}is a complex input projection and each transition coefficientat,ka\_\{t,k\}is a unit\-modulus phaseλt,k\\lambda\_\{t,k\}times a real gateαt,k∈\(0,1\)\\alpha\_\{t,k\}\\in\(0,1\)\. The phase is the diagonal Cayley transform of a real parameterϕt,k\\phi\_\{t,k\},

λt,k=1\+i​ϕt,k1−i​ϕt,k,\|λt,k\|=1,arg⁡λt,k=2​arctan⁡ϕt,k,\\lambda\_\{t,k\}=\\frac\{1\+i\\phi\_\{t,k\}\}\{1\-i\\phi\_\{t,k\}\},\\qquad\|\\lambda\_\{t,k\}\|=1,\\qquad\\arg\\lambda\_\{t,k\}=2\\arctan\\phi\_\{t,k\},\(3\)so without the gate the transition spectrum lies on the unit circle\. The phase parameter is assembled from an input\-dependent step sizeΔ​tt,k=softplus⁡\(⋅\)\\Delta t\_\{t,k\}=\\operatorname\{softplus\}\(\\cdot\), an input\-dependent shiftδt,k\\delta\_\{t,k\}, and a per\-channel base frequencyωk\\omega\_\{k\}, asϕt,k=12​Δ​tt,k​\(δt,k\+ωk\)\\phi\_\{t,k\}=\\tfrac\{1\}\{2\}\\Delta t\_\{t,k\}\(\\delta\_\{t,k\}\+\\omega\_\{k\}\)\. This reproduces, coordinatewise, the free\-plus\-interaction decomposition of the idealized Hamiltonian:ωk\\omega\_\{k\}supplies fixed baseline frequencies,δt,k\\delta\_\{t,k\}the input\-dependent modulation\.

The exactness of the parameterization matters at this sequence length, and it is the reason we prefer the Cayley map to approximate norm control by spectral normalization, penalty terms, or projection steps\. The Cayley map holds\|λt,k\|=1\|\\lambda\_\{t,k\}\|=1at machine precision with two arithmetic operations per coordinate and nothing to tune\. A scheme that controls the modulus only approximately, to within10−310^\{\-3\}say, accumulates a modulus drift of0\.9991024≈0\.360\.999^\{1024\}\\approx 0\.36over the sequence, losing64%64\\%of the transported signal to the attenuation the phase channel is designed to remove\. Approximate orthogonality suffices as a stability device; exact unit modulus is what makes the phase transport of[Lemma˜6\.2](https://arxiv.org/html/2608.14691#S6.Thmtheorem2)distortion\-free\.

The gateαt,k=σ​\(⋅\)\\alpha\_\{t,k\}=\\sigma\(\\cdot\)constitutes the relaxation of unitarity\. It sets the modulus of the transition coefficient,\|at,k\|=αt,k\|a\_\{t,k\}\|=\\alpha\_\{t,k\}, and hence the rate at which coordinatekkforgets its history: a value near one preserves the coordinate, a value near zero discards it\. This restores the selective, input\-dependent forgetting on which gated state\-space models depend\[[16](https://arxiv.org/html/2608.14691#bib.bib2)\]and which an exactly unitary recurrence cannot express; in the example of[Section˜1](https://arxiv.org/html/2608.14691#S1), it is the gate that allows the model to discard both readings once the passage has moved on\. The cost of the relaxation is that the recurrence becomes contractive, and the exact gradient guarantee of the idealized substrate is forfeited\. Two properties are retained, and they are the reason the relaxation is admissible\. The phaseλt,k\\lambda\_\{t,k\}remains on the unit circle, so the argument of a coordinate is transported without distortion even while its magnitude decays; and the sign of interference is unaffected, because constructive against destructive is decided by relative phase, although the strength of the cross term scales with the participating magnitudes\. The residual gradient benefit this predicts is formalized in[Lemma˜6\.2](https://arxiv.org/html/2608.14691#S6.Thmtheorem2)and measured in[Section˜8\.4](https://arxiv.org/html/2608.14691#S8.SS4)\.

The contrast that motivates the phase channel is quantitative\. A real diagonal recurrence with per\-coordinate factorρ<1\\rho<1transports a signal acrossTTsteps with gainρT\\rho^\{T\}; atρ=0\.99\\rho=0\.99andT=1024T=1024this gain is3\.4×10−53\.4\\times 10^\{\-5\}\. The complex recurrence is likewise contractive, with magnitude gain∏tαt,k\\prod\_\{t\}\\alpha\_\{t,k\}, but the phase coordinate itself is transported without distortion, so the information encoded in the argument arrives undistorted at the end of the sequence; its observable strength there is set by the accumulated magnitudes \([Remark˜6\.3](https://arxiv.org/html/2608.14691#S6.Thmtheorem3)\)\. The complex recurrence does not avoid contraction; it separates a decaying channel, the magnitude, from a preserved channel, the phase, and places the transported information in the preserved one\.

### 5\.3Evaluation by parallel scan

Equation \([2](https://arxiv.org/html/2608.14691#S5.E2)\) is a first\-order linear recurrence and admits an associative\-scan solution\. Writing the transition in polar form and accumulating over a chunk of lengthCC,

\|ψs\+m⟩=Pm⊙\(\|ψs⟩\+∑r=1mPr−1⊙bs\+r\),Pm=∏r=1mas\+r,\\left\|\\psi\_\{s\+m\}\\right\\rangle=P\_\{m\}\\odot\\Big\(\\left\|\\psi\_\{s\}\\right\\rangle\+\\sum\_\{r=1\}^\{m\}P\_\{r\}^\{\-1\}\\odot b\_\{s\+r\}\\Big\),\\qquad P\_\{m\}=\\prod\_\{r=1\}^\{m\}a\_\{s\+r\},\(4\)where the cumulative products and sums are prefix scans over the chunk, and the final state of each chunk seeds the next\. The cumulative product is evaluated in the logarithmic domain, as a prefix sum oflog⁡\|at,k\|\\log\|a\_\{t,k\}\|clamped from below so that the reciprocalPr−1P\_\{r\}^\{\-1\}cannot overflow, together with a prefix sum ofarg⁡at,k\\arg a\_\{t,k\}, and is reassembled as a magnitude at an angle\. The construction computes the exact trajectory inO​\(T\)O\(T\)arithmetic withO​\(log⁡C\)O\(\\log C\)sequential depth\. Because the recurrence is diagonal, each step costsO​\(N\)O\(N\), not theO​\(N2\)O\(N^\{2\}\)of a dense transition\. The substrate is therefore cheaper per step than the dense real recurrence it replaces, and complex arithmetic enters only as a constant factor on theO​\(N\)O\(N\)term\.

### 5\.4Relation to real diagonal state\-space models

Structurally, the recurrence \([2](https://arxiv.org/html/2608.14691#S5.E2)\) is a diagonal linear state\-space model of the kind studied by S5\[[47](https://arxiv.org/html/2608.14691#bib.bib14)\]and the linear recurrent unit\[[39](https://arxiv.org/html/2608.14691#bib.bib22)\], and it is worth stating exactly where it departs from them\. A real diagonal model places each transition eigenvalue on the real line or inside the unit disc, and the information carried across time sits in the*magnitude*of the state, which the eigenvalue attenuates geometrically\. Our recurrence places the phase of each eigenvalue on the unit circle by \([3](https://arxiv.org/html/2608.14691#S5.E3)\), and the carried information sits in the*argument*of the state, which the eigenvalue rotates without distortion; the magnitude, which sets how strongly that argument is observed downstream, is governed separately by the gate, whose sole role is forgetting\. The two designs make opposite choices about which component of a complex number transports and which forgets\. The real diagonal model transports in magnitude and cannot forget without attenuating the transported signal\. Ours transports in phase and forgets in magnitude, so transport and forgetting are decoupled\. The gradient measurements of[Section˜8\.4](https://arxiv.org/html/2608.14691#S8.SS4)are the empirical trace of this decoupling\.

### 5\.5Readout and the relaxation of the Born vocabulary head

The second relaxation concerns the Born rule\. In the idealized substrate the Born rule is the vocabulary readout, and its density\-matrix lift is the source of theΩ​\(N2\)\\Omega\(N^\{2\}\)advantage\. But a Born vocabulary head evaluates\|⟨ev\|ψ⟩\|2\|\\left\\langle e\_\{v\}\\,\\middle\|\\,\\psi\\right\\rangle\|^\{2\}against one measurement vector per token, costingO​\(N​V\)O\(NV\)per step, and realizing the full rank advantage requiresN=Ω​\(V\)N=\\Omega\(\\sqrt\{V\}\)\. At subword vocabularies the state width and readout cost would grow asV\\sqrt\{V\}andVV\. Rather than pay this cost before the substrate has proven useful, we apply the Born rule where it is inexpensive, as the scoring function of a linear\-attention kernel, and keep a conventional tied\-softmax vocabulary head\. The two backbones consequently read the complex state differently\.

The choice of the squared modulus over some other nonlinear projection is likewise not arbitrary\. The Born score is the rank\-one case of the general quadratic readouttr⁡\(M​ρ\)\\operatorname\{tr\}\(M\\rho\)withMMpositive semidefinite: it is nonnegative, it is normalizable without exponentials, which is what permits the running\-sum attention normalization in \([5](https://arxiv.org/html/2608.14691#S5.E5)\), and it is linear in the density matrixρ\\rho\. The linearity inρ\\rhois the property on which the representational separation of[Section˜3](https://arxiv.org/html/2608.14691#S3)and the rank accounting of[Section˜6](https://arxiv.org/html/2608.14691#S6)rest; a generic nonlinearity of the state would surrender both the probabilistic interpretation and the analysis\. The remaining alternatives, a complex recurrence read by a plain softmax and a real recurrence read by a Born score, are exactly the factorial cells specified in[Section˜9\.2](https://arxiv.org/html/2608.14691#S9.SS2)\.

In the state\-space realization the recurrence output isyt=C¯t⊙\|ψt⟩y\_\{t\}=\\bar\{C\}\_\{t\}\\odot\\left\|\\psi\_\{t\}\\right\\ranglefor a learned complex projectionCtC\_\{t\}; real and imaginary parts are concatenated, mapped to the inner width, gated by a SiLU branch, and added to the residual stream, from which the tied\-softmax head produces the distribution\. The Born rule does not enter this readout at all: the complex substrate acts on the state\-space backbone entirely through the recurrence\. The attention realization scores positioniiagainst positionjjby

si​j=\|⟨qi\|kj⟩\|2D​τi,\|ψi⟩=∑j≤isi​j​\|vj⟩∑j≤isi​j,s\_\{ij\}=\\frac\{\|\\left\\langle q\_\{i\}\\,\\middle\|\\,k\_\{j\}\\right\\rangle\|^\{2\}\}\{D\\,\\tau\_\{i\}\},\\qquad\\left\|\\psi\_\{i\}\\right\\rangle=\\frac\{\\sum\_\{j\\leq i\}s\_\{ij\}\\left\|v\_\{j\}\\right\\rangle\}\{\\sum\_\{j\\leq i\}s\_\{ij\}\},\(5\)a causal, sum\-normalized attention whose weight is the squared modulus of a complex inner product rather than the softmax of a real dot product, with temperatureτi\\tau\_\{i\}and head widthD=ND=N\. Queries and keys are complex and are rotated before scoring by a per\-position phaseθt,k=t​ωk\\theta\_\{t,k\}=t\\,\\omega\_\{k\}with learnable base frequencies, a complex relative encoding in the rotary family\. Because the weight is a squared modulus, overlaps between query and key components reinforce or cancel according to their relative phase, with a cross\-term strength proportional to the participating magnitudes\|⟨qi\|kj⟩\|\|\\left\\langle q\_\{i\}\\,\\middle\|\\,k\_\{j\}\\right\\rangle\|; the Born rule thus supplies interference at the level of attention even though the vocabulary readout remains a tied softmax\.

A two\-state computation exhibits the mechanism shared by both readouts and formalizes the example of[Section˜1](https://arxiv.org/html/2608.14691#S1)\. Let\|ϕ1⟩,\|ϕ2⟩\\left\|\\phi\_\{1\}\\right\\rangle,\\left\|\\phi\_\{2\}\\right\\ranglebe orthonormal states encoding the two readings and let\|ψ⟩=12​\(\|ϕ1⟩\+ei​θ​\|ϕ2⟩\)\\left\|\\psi\\right\\rangle=\\tfrac\{1\}\{\\sqrt\{2\}\}\\big\(\\left\|\\phi\_\{1\}\\right\\rangle\+e^\{i\\theta\}\\left\|\\phi\_\{2\}\\right\\rangle\\big\)\. A measurement vector\|e⟩=12​\(\|ϕ1⟩\+\|ϕ2⟩\)\\left\|e\\right\\rangle=\\tfrac\{1\}\{\\sqrt\{2\}\}\\big\(\\left\|\\phi\_\{1\}\\right\\rangle\+\\left\|\\phi\_\{2\}\\right\\rangle\\big\)aligned with their sum returns

p=\|⟨e\|ψ⟩\|2=12​\(1\+cos⁡θ\),p=\|\\left\\langle e\\,\\middle\|\\,\\psi\\right\\rangle\|^\{2\}=\\tfrac\{1\}\{2\}\\,\(1\+\\cos\\theta\),which falls from11atθ=0\\theta=0to0atθ=π\\theta=\\pias the relative phase alone advances, both magnitudes held fixed\. The recurrence controlsθ\\thetathrough the Cayley phase, so a downstream token can extinguish one reading by rotating it into phase opposition\. A real readout, which does not representθ\\theta, can only attenuate a component toward zero; it cannot make a present component count with negative sign\. This is the operational content of the separation recalled in[Section˜3](https://arxiv.org/html/2608.14691#S3)\.

### 5\.6Initialization

Initialization places nontrivial phase in the computation from the first step and spreads the per\-channel frequencies across timescales\. The complex prior and the complex measurement vectors of the attention readout are drawn from a circularly symmetric complex normal distribution and normalized to unit modulus; a real initialization would zero every interference cross term at the first step \([Section˜5\.1](https://arxiv.org/html/2608.14691#S5.SS1)\) and empirically delays the onset of learning\. The base frequenciesωk\\omega\_\{k\}are initialized on a geometric grid spanning roughly two decades, with alternating sign across channels, so the substrate starts with a spectrum of oscillation timescales instead of one characteristic rate; this mirrors the diagonal free HamiltonianH0H\_\{0\}of the idealized model\. Input projections use the Xavier rule, and the per\-head parameter projection starts with a small standard deviation, so the input\-dependent modulation of phase and gate begins near zero and grows over training\. These choices are held fixed across all models and corpora\.

### 5\.7Architectural realizations

Both complex realizations inherit the block structure of Mamba and modify only the substrate\. The shared block applies RMS normalization; an input projectionℝd→ℝ2​dinner\\mathbb\{R\}^\{d\}\\to\\mathbb\{R\}^\{2d\_\{\\mathrm\{inner\}\}\}that separates a signal branch from a SiLU gate branch; a depthwise causal convolution of kernel width 4 with a SiLU nonlinearity on the signal branch; a projection to the per\-head substrate parameters; the substrate itself; a projection back to the inner width; the gate; and an output projection with a residual connection\. The input embedding is tied to the output head\. All models share one configuration:dmodel=1024d\_\{\\mathrm\{model\}\}=1024,dinner=2048d\_\{\\mathrm\{inner\}\}=2048,L=24L=24layers,H=16H=16heads,N=16N=16, convolution width 4, sequence length10241024, byte vocabulary256256, and 253M total parameters \([Table˜3](https://arxiv.org/html/2608.14691#S5.T3)\)\.

The state\-space realization, H\-Mamba \(H for the Hilbert\-space state\), obtains its substrate parameters from a per\-head projection emitting6​N\+16N\+1real numbers: the phase shiftδ\\delta\(NNvalues\), a logit for the step sizeΔ​t\\Delta t\(1 value\), a logit for the gateα\\alpha\(NNvalues\), and the complex input and output projectionsbbandCC\(2​N2Nvalues each\)\. The step size is thesoftplus\\operatorname\{softplus\}of its logit, clamped at0\.10\.1; the gate is the logistic of its logit; the phase follows \([3](https://arxiv.org/html/2608.14691#S5.E3)\); and the recurrence \([2](https://arxiv.org/html/2608.14691#S5.E2)\) is evaluated by the chunked scan of[Section˜5\.3](https://arxiv.org/html/2608.14691#S5.SS3)with chunk length 64\. Every component downstream of the recurrence \(the gate, the residual connection, and the tied\-softmax head\) is identical to that of the real Mamba\.

The attention realization, H\-Transformer, obtains complex queries, keys, values, and a temperature from the same per\-head projection, hosted in the same shared block\. Queries and keys are phase\-rotated, scored by \([5](https://arxiv.org/html/2608.14691#S5.E5)\) under the causal running\-sum normalization, and read out bilinearly against the query into the same tied\-softmax head\. The attention is computed by a single fused kernel, implemented in Triton with a compiled fallback, that evaluates the causal Born attention in one pass with single\-precision accumulation; this lets the complex attention train in half precision without instability\. Its real counterpart is the standard pre\-norm Transformer block, rotary dot\-product attention\[[50](https://arxiv.org/html/2608.14691#bib.bib32)\]followed by a SiLU feed\-forward layer, at its published configuration\.

#### Parameter matching\.

For the comparison to isolate the substrate, each complex model and its real counterpart must have equal capacity\. We match each pair tensor by tensor to within0\.02%0\.02\\%of the total, letting the real baselines absorb the small surplus introduced by the complex projections in their feed\-forward capacity\. Concretely, each real\-Mamba block carries a small SwiGLU branch of hidden width12581258whose only function is parameter matching, and the real Transformer’s feed\-forward hidden width is30963096; each value is chosen so that the pair totals 253M\. No other dimension differs, so the results below are not attributable to a difference in total parameter count, though the two members of a pair allocate those parameters differently\.

### 5\.8Parameter and computational cost

The substrate is inexpensive relative to the block that hosts it\. Per head, the recurrence adds the6​N\+16N\+1parameters of the per\-head projection and runs the scan inO​\(N\)O\(N\)arithmetic per step, againstO​\(N2\)O\(N^\{2\}\)for a dense real transition of the same width\. The complex state doubles state storage fromNNto2​N2Nreals per head, a negligible fraction of block memory, which the shared input and output projections dominate\.[Table˜2](https://arxiv.org/html/2608.14691#S5.T2)summarizes per\-component parameters and per\-step arithmetic\. Because the extra arithmetic of complex numbers is a constant factor on a non\-dominant term, the complex models are not materially more expensive per step than the real ones; the measured throughput of[Section˜8\.4](https://arxiv.org/html/2608.14691#S8.SS4)is consistent with this accounting\.

#### Summary of the deployable substrate\.

Each head carries a complex state of widthN=16N=16, stored as2​N2Nreal numbers and initialized from a unit\-modulus complex prior\. The state evolves by the diagonal recurrence \([2](https://arxiv.org/html/2608.14691#S5.E2)\)\. Each transition coefficient is a unit\-modulus Cayley phase \([3](https://arxiv.org/html/2608.14691#S5.E3)\), which carries the information, multiplied by a sigmoid gate, which does the forgetting; the recurrence is evaluated exactly by a chunked scan inO​\(N\)O\(N\)arithmetic per step\. The attention realization scores positions by the Born rule \([5](https://arxiv.org/html/2608.14691#S5.E5)\); the state\-space realization does not use the Born rule at all\. Both realizations end in an ordinary tied\-softmax head\. Relative to the idealized substrate, two properties are surrendered: exact norm preservation, since the recurrence contracts, and theΩ​\(N2\)\\Omega\(N^\{2\}\)vocabulary readout, since the head is a softmax\. Two properties are retained: the unit\-modulus phase and the interference it supports, and[Lemma˜6\.2](https://arxiv.org/html/2608.14691#S6.Thmtheorem2)will make the retention exact\. Relative to the real baselines, the only changes are the complex state, the phase–gate transition, and, in the attention model, the Born score; parameters match to within0\.02%0\.02\\%\.[Section˜6](https://arxiv.org/html/2608.14691#S6)gives the two retained properties their formal statements\.

Table 2:Per\-head parameters and per\-step arithmetic of the substrate and the shared block components\. HereN=16N=16is the state width,dinner=2048d\_\{\\mathrm\{inner\}\}=2048the inner width,dmodel=1024d\_\{\\mathrm\{model\}\}=1024,V=256V=256the vocabulary, andTTthe sequence length\.![Refer to caption](https://arxiv.org/html/2608.14691v1/x2.png)Figure 2:The substitution is local\.The state\-space realization \(left\) and the attention realization \(right\) share the block skeleton of the classical backbone \(unshaded\): convolution branch, gate branch, residual stream, tied output head\. The shaded components are the substrate: the complex state, the diagonal Cayley phase recurrence, and, in the attention realization, Born\-rule scoring\. Every unshaded tensor matches its real\-valued counterpart in shape and initialization\.Table 3:Configuration shared by all four models\. Real and complex counterparts are matched to within0\.02%0\.02\\%of parameters; the real baselines absorb the difference in feed\-forward capacity \(a parameter\-matching SwiGLU branch of hidden width12581258in each Mamba block; feed\-forward hidden width30963096in the Transformer\)\.

## 6Theoretical Analysis

The separation recalled in[Section˜3](https://arxiv.org/html/2608.14691#S3)is about representation: it bounds the state dimension a real model needs to*express*a task family\. Our measurements are about optimization: they count the gradient steps a model needs to*reach*a loss\. This section provides two statements that connect the two\. The first is a floor on the loss attainable by a rank\-limited readout\. It is stated for general model classes;[Section˜6\.1](https://arxiv.org/html/2608.14691#S6.SS1)shows that it is inactive at the vocabulary head of every model trained here, so it motivates the present results without explaining them\. The second is an exact\-transport lemma for relative phase, which formalizes the advantage available to the recurrence and predicts the gradient measurements of[Section˜8\.4](https://arxiv.org/html/2608.14691#S8.SS4)\.

### 6\.1An optimization floor for the affine–softmax readout

The floor rests on a simple observation: an affine–softmax head produces only logit matrices of low rank, so the set of predictors it can ever visit is a thin slice of the space in which the target lies\. If the target carries spectral mass outside that slice, no trajectory through the slice approaches it; the loss is bounded away from the optimum at every step by an amount proportional to that mass, and per\-step progress must stall as the trajectory nears the bound\. The proposition below makes this quantitative, and its proof is elementary\.

Fix a conditioning distribution supported onMMcontexts, taken uniform for simplicity, and letL⋆∈ℝM×VL^\{\\star\}\\in\\mathbb\{R\}^\{M\\times V\}be the matrix whose rowccholds the optimal next\-token logits for contextcc\. Because the softmax is invariant to adding a constant to a row, we pass to row\-centered representatives: for anyZ∈ℝM×VZ\\in\\mathbb\{R\}^\{M\\times V\}writeZ~=Z​\(I−1V​𝟏𝟏⊤\)\\widetilde\{Z\}=Z\(I\-\\tfrac\{1\}\{V\}\\mathbf\{1\}\\mathbf\{1\}^\{\\top\}\), which projects each row orthogonal to the all\-ones vector\. An affine–softmax head of widthddproduces stacked logitsZ=H​W⊤\+𝟏​b⊤Z=HW^\{\\top\}\+\\mathbf\{1\}b^\{\\top\}withH∈ℝM×dH\\in\\mathbb\{R\}^\{M\\times d\}, sorank⁡\(Z\)≤d\+1\\operatorname\{rank\}\(Z\)\\leq d\+1\[[61](https://arxiv.org/html/2608.14691#bib.bib13)\], and centering cannot raise rank:rank⁡\(Z~\)≤d\+1\\operatorname\{rank\}\(\\widetilde\{Z\}\)\\leq d\+1for every parameter setting\. Writeσ1≥σ2≥⋯\\sigma\_\{1\}\\geq\\sigma\_\{2\}\\geq\\cdotsfor the singular values ofL~⋆\\widetilde\{L\}^\{\\star\}, and letpmin\>0p\_\{\\min\}\>0be a floor such that every conditional probability produced along the segment between the realized and the optimal logits is at leastpminp\_\{\\min\}\(an assumption about the region training traverses\)\.

###### Proposition 6\.1\(Optimization floor for the affine–softmax readout\)\.

Under the assumptions above, for every parameter setting of the backbone and head, and hence after any number of gradient steps, the expected excess cross\-entropy obeys

𝔼c​\[ℒc−ℒc⋆\]≥pmin2​M​∑i\>d\+1σi2​\(L~⋆\)\.\\mathbb\{E\}\_\{c\}\\\!\\left\[\\mathcal\{L\}\_\{c\}\-\\mathcal\{L\}\_\{c\}^\{\\star\}\\right\]\\;\\geq\\;\\frac\{p\_\{\\min\}\}\{2M\}\\sum\_\{i\>d\+1\}\\sigma\_\{i\}^\{2\}\\big\(\\widetilde\{L\}^\{\\star\}\\big\)\.\(6\)The floor is strictly positive wheneverrank⁡\(L~⋆\)\>d\+1\\operatorname\{rank\}\(\\widetilde\{L\}^\{\\star\}\)\>d\+1and vanishes otherwise\.

###### Proof\.

The excess loss at contextccisKL​\(pc⋆∥p^c\)\\mathrm\{KL\}\(p^\{\\star\}\_\{c\}\\,\\\|\\,\\hat\{p\}\_\{c\}\)\. WithA​\(z\)=log​∑vezvA\(z\)=\\log\\sum\_\{v\}e^\{z\_\{v\}\}, a direct computation givesKL​\(pc⋆∥p^c\)=A​\(z^c\)−A​\(zc⋆\)−⟨∇A​\(zc⋆\),z^c−zc⋆⟩\\mathrm\{KL\}\(p^\{\\star\}\_\{c\}\\\|\\hat\{p\}\_\{c\}\)=A\(\\hat\{z\}\_\{c\}\)\-A\(z^\{\\star\}\_\{c\}\)\-\\langle\\nabla A\(z^\{\\star\}\_\{c\}\),\\hat\{z\}\_\{c\}\-z^\{\\star\}\_\{c\}\\rangle, the Bregman divergence ofAA; both sides are invariant to row shifts, so takeΔc=z^~c−z~c⋆\\Delta\_\{c\}=\\widetilde\{\\hat\{z\}\}\_\{c\}\-\\widetilde\{z\}^\{\\star\}\_\{c\}\. Taylor’s theorem with integral remainder yieldsKL=∫01\(1−s\)​Δc⊤​∇2A​\(zc⋆\+s​Δc\)​Δc​𝑑s\\mathrm\{KL\}=\\int\_\{0\}^\{1\}\(1\-s\)\\,\\Delta\_\{c\}^\{\\top\}\\nabla^\{2\}A\\big\(z^\{\\star\}\_\{c\}\+s\\Delta\_\{c\}\\big\)\\Delta\_\{c\}\\,dswith∇2A​\(u\)=diag⁡\(q\)−q​q⊤\\nabla^\{2\}A\(u\)=\\operatorname\{diag\}\(q\)\-qq^\{\\top\},q=softmax⁡\(u\)q=\\operatorname\{softmax\}\(u\)\. For any probability vectorqqand anyvvwith∑kvk=0\\sum\_\{k\}v\_\{k\}=0, the variance identity givesv⊤​\(diag⁡\(q\)−q​q⊤\)​v=∑kqk​\(vk−m\)2v^\{\\top\}\(\\operatorname\{diag\}\(q\)\-qq^\{\\top\}\)v=\\sum\_\{k\}q\_\{k\}\(v\_\{k\}\-m\)^\{2\}withm=q⊤​vm=q^\{\\top\}v, and

∑kqk​\(vk−m\)2≥pmin​∑k\(vk−m\)2=pmin​\(‖v‖2\+V​m2\)≥pmin​‖v‖2\.\\sum\_\{k\}q\_\{k\}\(v\_\{k\}\-m\)^\{2\}\\;\\geq\\;p\_\{\\min\}\\sum\_\{k\}\(v\_\{k\}\-m\)^\{2\}\\;=\\;p\_\{\\min\}\\big\(\\\|v\\\|^\{2\}\+Vm^\{2\}\\big\)\\;\\geq\\;p\_\{\\min\}\\\|v\\\|^\{2\}\.HenceKL​\(pc⋆∥p^c\)≥pmin2​‖Δc‖2\\mathrm\{KL\}\(p^\{\\star\}\_\{c\}\\\|\\hat\{p\}\_\{c\}\)\\geq\\tfrac\{p\_\{\\min\}\}\{2\}\\\|\\Delta\_\{c\}\\\|^\{2\}, and averaging over theMMcontexts,𝔼c​\[ℒc−ℒc⋆\]≥pmin2​M​‖Z~−L~⋆‖F2\\mathbb\{E\}\_\{c\}\[\\mathcal\{L\}\_\{c\}\-\\mathcal\{L\}\_\{c\}^\{\\star\}\]\\geq\\tfrac\{p\_\{\\min\}\}\{2M\}\\\|\\widetilde\{Z\}\-\\widetilde\{L\}^\{\\star\}\\\|\_\{F\}^\{2\}\. Sincerank⁡\(Z~\)≤d\+1\\operatorname\{rank\}\(\\widetilde\{Z\}\)\\leq d\+1at every parameter setting, the Eckart–Young theorem bounds the Frobenius distance below by the tail of the spectrum:‖Z~−L~⋆‖F2≥∑i\>d\+1σi2​\(L~⋆\)\\\|\\widetilde\{Z\}\-\\widetilde\{L\}^\{\\star\}\\\|\_\{F\}^\{2\}\\geq\\sum\_\{i\>d\+1\}\\sigma\_\{i\}^\{2\}\(\\widetilde\{L\}^\{\\star\}\)\. Combining the two displays gives \([6](https://arxiv.org/html/2608.14691#S6.E6)\)\. The tail sum is positive exactly whenrank⁡\(L~⋆\)\>d\+1\\operatorname\{rank\}\(\\widetilde\{L\}^\{\\star\}\)\>d\+1\. ∎

Three consequences bear on the experiments, and the first is a delimitation\. The floor is*not active at the vocabulary head of the models trained here*: the head width isd=dmodel=1024d=d\_\{\\mathrm\{model\}\}=1024while the byte vocabulary gives centered target rank at mostV−1=255V\-1=255, sod\+1d\+1exceeds the maximal possible rank and the vocabulary floor is zero for every model in this study\. The proposition therefore does not explain the present byte\-level results at the vocabulary, and we state this plainly; it serves as general motivation, and it delimits where readout rank could matter: heads narrower than their vocabulary, as arise at subword scale \([Section˜9\.4](https://arxiv.org/html/2608.14691#S9.SS4)\), and, by analogy, ranking operations whose width is far below the number of alternatives they rank\. Attention is such an operation: each head scoresT=1024T=1024positions through queries and keys of dimensionD=16D=16, so an attention\-level analogue of the rank constraint is far from vacuous there\. Second, where a floor is active it constrains the whole trajectory, not one step: no schedule, optimizer, or backbone takes a rank\-limited head below it, so per\-step progress must decay as training approaches it, while a scorer of higher effective rank faces no such ceiling\. Third, the proposition should not be read as a claim that Born attention removes the softmax bottleneck: both H\-models retain the tied softmax vocabulary head, with the same \(inactive\) vocabulary\-rank situation as their baselines\. The Born score is better described as a richer*internal interaction rule*, a quadratic score with signed cross terms, whose relation to the floor is the attention\-level analogy above rather than an instance of the proposition\. H\-Mamba uses neither a Born score nor a modified head, so no readout\-rank mechanism of any kind is available to it, and its acceleration is most naturally attributed to the recurrence \([Section˜5\.2](https://arxiv.org/html/2608.14691#S5.SS2)\); we return to this division of labor in[Section˜9\.2](https://arxiv.org/html/2608.14691#S9.SS2)\. The fullΩ​\(N2\)\\Omega\(N^\{2\}\)mechanism concerns the idealized Born vocabulary readout, which the deployable substrate does not instantiate\.

### 6\.2Exact transport of relative phase

The second statement concerns the recurrence\. The quantity that the quadratic readout consumes is relative phase: by the two\-state computation of[Section˜5\.5](https://arxiv.org/html/2608.14691#S5.SS5), the score depends on the off\-diagonal density\-matrix coordinatesρj​k=ψj​ψ¯k\\rho\_\{jk\}=\\psi\_\{j\}\\bar\{\\psi\}\_\{k\}through their arguments\. The following lemma is deliberately narrow: it shows that for an isolated homogeneous component, the recurrence \([2](https://arxiv.org/html/2608.14691#S5.E2)\) accumulates phase additively and transports the phase*coordinate*without distortion, at unit sensitivity, regardless of the gates\. What it does not assert, and what[Remark˜6\.3](https://arxiv.org/html/2608.14691#S6.Thmtheorem3)delimits, is that loss gradients through the phase are constant: the observable effect of a phase, and the gradient that returns through it, are scaled by the participating magnitudes\.

###### Lemma 6\.2\(Exact transport of relative phase\)\.

Consider the homogeneous recurrence\|ψt\+1⟩=at⊙\|ψt⟩\\left\|\\psi\_\{t\+1\}\\right\\rangle=a\_\{t\}\\odot\\left\|\\psi\_\{t\}\\right\\ranglewithat,k=αt,k​λt,ka\_\{t,k\}=\\alpha\_\{t,k\}\\lambda\_\{t,k\},αt,k∈\(0,1\)\\alpha\_\{t,k\}\\in\(0,1\),\|λt,k\|=1\|\\lambda\_\{t,k\}\|=1, and writeθt,k=arg⁡λt,k\\theta\_\{t,k\}=\\arg\\lambda\_\{t,k\}\. Letρj​k,t=ψt,j​ψ¯t,k\\rho\_\{jk,t\}=\\psi\_\{t,j\}\\bar\{\\psi\}\_\{t,k\}be an off\-diagonal coordinate of the density matrix\. Then for every horizonTT,

arg⁡ρj​k,T=arg⁡ρj​k,0\+∑t<T\(θt,j−θt,k\),\|ρj​k,T\|=\|ρj​k,0\|​∏t<Tαt,j​αt,k\.\\arg\\rho\_\{jk,T\}=\\arg\\rho\_\{jk,0\}\+\\sum\_\{t<T\}\\big\(\\theta\_\{t,j\}\-\\theta\_\{t,k\}\\big\),\\qquad\|\\rho\_\{jk,T\}\|=\|\\rho\_\{jk,0\}\|\\prod\_\{t<T\}\\alpha\_\{t,j\}\\,\\alpha\_\{t,k\}\.\(7\)In particular,∂arg⁡ρj​k,T/∂θs,j=1\\partial\\arg\\rho\_\{jk,T\}\\big/\\partial\\theta\_\{s,j\}=1for everys<Ts<T, independently of the horizon and of the gates, whereas every derivative of the magnitude carries the accumulated contraction∏tα\\prod\_\{t\}\\alpha\.

###### Proof\.

Unrolling the recurrence givesψT,k=ψ0,k​∏t<Tat,k\\psi\_\{T,k\}=\\psi\_\{0,k\}\\prod\_\{t<T\}a\_\{t,k\}, and thereforeρj​k,T=ψ0,j​ψ¯0,k​∏t<Tat,j​a¯t,k\\rho\_\{jk,T\}=\\psi\_\{0,j\}\\bar\{\\psi\}\_\{0,k\}\\prod\_\{t<T\}a\_\{t,j\}\\bar\{a\}\_\{t,k\}\. Sinceat,j​a¯t,k=αt,j​αt,k​ei​\(θt,j−θt,k\)a\_\{t,j\}\\bar\{a\}\_\{t,k\}=\\alpha\_\{t,j\}\\alpha\_\{t,k\}\\,e^\{i\(\\theta\_\{t,j\}\-\\theta\_\{t,k\}\)\}, taking arguments and moduli yields \([7](https://arxiv.org/html/2608.14691#S6.E7)\); the argument is additive in the phases and free of the gates\. Differentiation is immediate\. By linearity of the recurrence, the same identities apply to each input contributionPr−1⊙bs\+rP^\{\-1\}\_\{r\}\\odot b\_\{s\+r\}of the inhomogeneous solution \([4](https://arxiv.org/html/2608.14691#S5.E4)\)*separately*; the phase of their sum is a nonlinear function of the parts and can change through interference, so the transport identity governs the parts, not the aggregate\. ∎

It is worth stating plainly what the two results do and do not contribute, since both have just been delimited\. Neither is a derivation of the measured step ratios\.[Proposition˜6\.1](https://arxiv.org/html/2608.14691#S6.Thmtheorem1)is inactive at the vocabulary of these experiments and bears on the design only through the analogy to narrow ranking operations;[Lemma˜6\.2](https://arxiv.org/html/2608.14691#S6.Thmtheorem2)concerns an isolated component and yields a decoupling statement rather than a gradient guarantee\. What they establish is the*direction*of two candidate mechanisms and, more importantly, the reason the substrate was built as it was: a score whose reachable set is not confined by the width of the state, and a recurrence whose information coordinate is not the coordinate that contraction attenuates\. The empirical claims of this paper rest on the measurements of[Section˜8](https://arxiv.org/html/2608.14691#S8), not on these statements, and the two are kept separate deliberately\. With that scope fixed, the results still make complementary predictions: a score\-side advantage wherever the rank of a ranking operation binds, which the attention realization tests, and a transport\-side advantage wherever gradients must cross many positions, which the state\-space realization tests\.[Section˜7](https://arxiv.org/html/2608.14691#S7)describes that comparison\.

## 7Experimental Setup

The goal of the experiments follows from[Section˜6](https://arxiv.org/html/2608.14691#S6): to measure the per\-step advantage of the substrate while excluding capacity, tuning, corpus, and backbone as alternative explanations\. We evaluate four models, the real and complex realizations of two backbones, on three byte\-level corpora under a single training protocol\. The design is intended to make the substrate the principal systematic difference within each pair, and to distinguish substrate effects from effects of dataset scale; the respects in which the paired models nonetheless differ are recorded in[Section˜10](https://arxiv.org/html/2608.14691#S10)\.

#### Models\.

The four models are the real Mamba and its complex realization H\-Mamba, and a real Transformer and its complex realization H\-Transformer\. The real Transformer uses standard softmax dot\-product attention; the complex realization replaces it with the sum\-normalized Born score of \([5](https://arxiv.org/html/2608.14691#S5.E5)\), a linear\-attention\-style kernel, so the attention pair differs in the functional form of the score, including its normalization, which is the readout half of the substrate axis \([Section˜10](https://arxiv.org/html/2608.14691#S10)records the caveat that the pair is parameter\-matched but not architecturally identical\)\. The real Mamba is a selective state\-space model with state dimension 16 and aΔ\\Delta\-projection rank of 64; the real Transformer is a pre\-normalized decoder with rotary position embeddings and dot\-product attention\. Both follow their published configurations\[[16](https://arxiv.org/html/2608.14691#bib.bib2),[50](https://arxiv.org/html/2608.14691#bib.bib32)\], and neither is weakened: the real Transformer attains the lowest terminal loss of any model in this study on enwik8 \(1\.3841\.384bpc,[Table˜8](https://arxiv.org/html/2608.14691#S8.T8)\)\. All four share the configuration of[Table˜3](https://arxiv.org/html/2608.14691#S5.T3), read out through a weight\-tied byte embedding, and are matched in parameter count to within0\.02%0\.02\\%as described in[Section˜5\.7](https://arxiv.org/html/2608.14691#S5.SS7)\.

#### Data\.

We model text at the byte level\[[58](https://arxiv.org/html/2608.14691#bib.bib49)\]: the vocabulary is the 256 byte values, and performance is reported as bits per character \(bpc\), the cross\-entropy in nats divided byln⁡2\\ln 2\. This is the average number of bits the model needs to encode the next byte, and lower is better\. Byte\-level modeling removes the tokenizer as a confound\. We note for clarity that it also renders the vocabulary\-level floor of[Proposition˜6\.1](https://arxiv.org/html/2608.14691#S6.Thmtheorem1)inactive, since the head width exceeds the byte vocabulary; the rank considerations of[Section˜6](https://arxiv.org/html/2608.14691#S6)pertain, if anywhere, to the attention scores, whose per\-head widthD=16D=16is far below theT=1024T=1024positions they rank\. The three corpora span150×150\\timesin size: enwik8\[[27](https://arxiv.org/html/2608.14691#bib.bib16)\], 100 MB of Wikipedia markup; OpenWebText\[[14](https://arxiv.org/html/2608.14691#bib.bib17)\], a 3 GB stream of web text; and FineWeb\[[42](https://arxiv.org/html/2608.14691#bib.bib18)\], a 15 GB subset of CommonCrawl\. Each corpus is read as raw bytes, partitioned90/5/590/5/5into training, validation, and test, and segmented into non\-overlapping windows of 1024 bytes, the target at each position being the next byte\. The test portions are held out and untouched; every number in this paper is computed on the validation portions, and the released logs permit test evaluation\.

#### Optimization\.

A single protocol is applied without modification to all four models, so no element of the schedule can favor one substrate\. The optimizer is AdamW\[[32](https://arxiv.org/html/2608.14691#bib.bib11),[35](https://arxiv.org/html/2608.14691#bib.bib46)\]withβ1=0\.9\\beta\_\{1\}=0\.9,β2=0\.95\\beta\_\{2\}=0\.95, and decoupled weight decay0\.10\.1on tensors of rank at least 2; the learning rate follows a cosine schedule\[[34](https://arxiv.org/html/2608.14691#bib.bib47)\]from6×10−46\\times 10^\{\-4\}to6×10−56\\times 10^\{\-5\}with 750 warmup steps; gradients are clipped to unit norm\[[40](https://arxiv.org/html/2608.14691#bib.bib43)\]; computation isbf16with per\-layer activation checkpointing\. These values follow the published recipe for language models of this size\[[3](https://arxiv.org/html/2608.14691#bib.bib48)\]\. None was tuned to either substrate, and we deliberately performed no per\-model search: unequal tuning effort would confound the substrate comparison, whereas a common published recipe favors neither side by construction\. The residual possibility that the shared recipe suits one substrate better than the other is discussed in[Section˜10](https://arxiv.org/html/2608.14691#S10)\. Training is data\-parallel over four A100\-80GB accelerators at 256 sequences each, an effective batch of 1024 sequences, or1,048,5761\{,\}048\{,\}576tokens, per step\. Validation loss is evaluated every 20 steps and the best checkpoint retained\.[Table˜4](https://arxiv.org/html/2608.14691#S7.T4)lists the protocol in full\.

Table 4:Complete optimization protocol, applied without modification to all twelve runs\.
#### Budget\.

The optimization budget is 1000 steps on OpenWebText and FineWeb and 1500 on enwik8; unless stated otherwise, models are compared at the common budget of 1000 steps\. This is the appropriate basis for a claim about per\-step optimization, and the choice is consequential on the smallest corpus\. Beyond roughly 1000 steps the complex models, which extract more from each step, begin to overfit the 100 MB of enwik8; their validation loss climbs toward2\.22\.2–2\.32\.3bpc by step 1500 while the slower real Mamba is still improving\.[Section˜8\.3](https://arxiv.org/html/2608.14691#S8.SS3)treats this behavior, which is a consequence of the substrate’s sample efficiency rather than a defect of it\.

#### Runs, controls, and reproducibility\.

Three properties make the comparison a test of the substrate rather than of tuning: the protocol is identical across models; the counterparts are parameter\-matched to within0\.02%0\.02\\%; and every training log, together with the parser that converts logs into the tables and figures below, is released, so each quantity is re\-derivable, not asserted\. Twelve runs cover the four models on the three corpora\. The real\-Mamba run on FineWeb stopped immediately after its step\-1000 validation evaluation, which is the common comparison point, so it enters every comparison below; we note the single place where its extended trajectory is missing\. The methodological caveat is that each cell is a single run;[Section˜10](https://arxiv.org/html/2608.14691#S10)states which conclusions this qualifies and calibrates, against the smallest effects in the paper, which effects seed variance could plausibly explain\.

## 8Results

Four models, three corpora, and one protocol yield twelve runs\. The per\-step comparison comes first \([Section˜8\.1](https://arxiv.org/html/2608.14691#S8.SS1)\), then the behavior after the learning\-rate warmup ends \([Section˜8\.2](https://arxiv.org/html/2608.14691#S8.SS2)\), terminal quality and the behavior at the data limit \([Section˜8\.3](https://arxiv.org/html/2608.14691#S8.SS3)\), and finally throughput and the gradient measurements \([Section˜8\.4](https://arxiv.org/html/2608.14691#S8.SS4)\)\.

### 8\.1Per\-step convergence

[Figure˜4](https://arxiv.org/html/2608.14691#S8.F4)plots validation loss against optimization step for all runs\. We summarize each pair by the ratio of steps needed to reach a common loss\. For a targetℓ\\ell, the ratio is the step at which the real model first attainsℓ\\ell, divided by the step at which the complex model first attains it; each step is obtained by linear interpolation between the validation evaluations \(every 20 steps\) that bracket the target\.[Table˜5](https://arxiv.org/html/2608.14691#S8.T5)reports this ratio at three targets;[Table˜6](https://arxiv.org/html/2608.14691#S8.T6)reports the underlying step counts, and[Figure˜3](https://arxiv.org/html/2608.14691#S8.F3)draws them\. The targets2\.52\.5,2\.22\.2, and2\.02\.0bpc lie in the interval over which all four models are still descending, so the ratio measures the rate of convergence, not the eventual separation of terminal losses\. Lower targets are excluded because the slower models do not attain them within the budget, which would turn a measurement into a lower bound\. As a representative instance: on enwik8, H\-Mamba first reaches2\.02\.0bpc at step 191; the real Mamba reaches it at step 575\.

Table 5:Optimization steps to reach a target validation loss, real divided by complex\.A value of3\.03\.0indicates that the real model requires three times as many steps as its complex counterpart to reach the stated bits per character\. Ratios are read from the curves of[Figure˜4](https://arxiv.org/html/2608.14691#S8.F4); a larger value is a larger per\-step advantage for the substrate\.Table 6:Step at which each model first attains a target validation loss \(linear interpolation between evaluations\)\. The ratios of[Table˜5](https://arxiv.org/html/2608.14691#S8.T5)are the real counts divided by the complex counts\.![Refer to caption](https://arxiv.org/html/2608.14691v1/x3.png)Figure 3:Steps to first reach each validation target, drawn from the counts of[Table˜6](https://arxiv.org/html/2608.14691#S8.T6)\. The complex model \(blue\) requires fewer steps than its real counterpart \(gray\) in all 18 comparisons; annotated ratios are those of[Table˜5](https://arxiv.org/html/2608.14691#S8.T5)\. The complex state\-space model passes2\.52\.5bpc within 75–79 steps on every corpus, a level the real model reaches after 237–252 steps\.Three features of[Table˜5](https://arxiv.org/html/2608.14691#S8.T5)determine the interpretation of the result\.

#### Stability across thresholds\.

Across the three columns of the table, the ratio is nearly constant as the target descends from2\.52\.5to2\.02\.0bpc\. A transient advantage of initialization would appear as a ratio decaying toward one with descending target; instead, the separation is maintained through the steep portion of the learning curve, so the complex models do not merely start faster and get overtaken as training proceeds\.

#### Stability across corpora\.

Within each backbone, the rows of the table agree across corpora that differ by150×150\\timesin size: approximately 3 for the state\-space pair on enwik8, OpenWebText, and FineWeb alike, and approximately 2 for the attention pair\. An effect tied to the statistics of a particular corpus would vary across these rows, and it does not\.

#### Stability across backbones\.

The four models constitute a2×22\\times 2design over backbone and substrate, and both complex cells improve on their real cells by a stable, backbone\-characteristic factor\. The most informative cell is H\-Mamba\. Its vocabulary readout is an ordinary tied softmax, so no readout mechanism is available to it, and its acceleration, the larger of the two, is most naturally attributed to the recurrence, where[Section˜5\.2](https://arxiv.org/html/2608.14691#S5.SS2)locates the retained benefit of the substrate\.

#### Crossings\.

Within the common budget, the complex curve lies below its real counterpart at every recorded evaluation for five of the six pairs\. The exception is the attention pair on enwik8, where the rotary Transformer, which attains the lowest terminal loss of any model in this study on this corpus, crosses below H\-Transformer within the budget as the complex model begins to saturate the 100 MB corpus; it finishes at1\.3841\.384against1\.5161\.516bpc \([Table˜8](https://arxiv.org/html/2608.14691#S8.T8)\)\. Beyond the budget, and again on enwik8 only, the real Mamba eventually overtakes the by\-then overfitting H\-Mamba \([Section˜8\.3](https://arxiv.org/html/2608.14691#S8.SS3)\)\. Both crossings occur in the regime where the sample efficiency of the substrate exhausts the smallest corpus; on the web corpora no curve crosses at any point\. The measured targets are, however, all crossed during the learning\-rate warmup, and[Section˜8\.2](https://arxiv.org/html/2608.14691#S8.SS2)examines what happens once it ends\.

![Refer to caption](https://arxiv.org/html/2608.14691v1/x4.png)Figure 4:Per\-step convergence on the three corpora\.Validation bpc against optimization step\. Within the measured range of targets, each complex realization \(solid\) reaches every loss value in fewer steps than its real counterpart \(dashed\); the horizontal gap at a fixed loss is the ratio tabulated in[Table˜5](https://arxiv.org/html/2608.14691#S8.T5)\. On enwik8 the complex curves turn upward beyond roughly step 1000 as the 100 MB corpus is overfit \([Section˜8\.3](https://arxiv.org/html/2608.14691#S8.SS3)\)\.

### 8\.2Behavior after the warmup phase

Every target in[Table˜5](https://arxiv.org/html/2608.14691#S8.T5)is crossed during the learning\-rate warmup, which occupies the first 750 of the 1000 steps of the budget \([Table˜4](https://arxiv.org/html/2608.14691#S7.T4)\); the complex models cross all measured targets between steps 75 and 247, and the real models cross the2\.52\.5\-bpc target in the same interval\. A comparison confined to that interval invites a specific alternative explanation: that the ramp of the schedule, rather than the substrate, produces the separation\. The schedule is identical across all four models, so this would have to be an interaction between the ramp and the substrate rather than a difference in treatment; but the interaction is not excluded by the consistency of the ratios, since the same schedule is used in all twelve runs and an interaction would be equally consistent\.

The released logs settle the question directly, because they record validation loss on both sides of the warmup boundary\.[Figure˜5](https://arxiv.org/html/2608.14691#S8.F5)plots the validation gap, real minus complex, against optimization step, and[Table˜7](https://arxiv.org/html/2608.14691#S8.T7)reports it at the boundary and at the end of the budget\. An advantage manufactured by the ramp should contract once the ramp stops\.

Table 7:Validation gap \(real−\-complex, bpc\) at the end of warmup and at the end of the budget\.A positive value means the complex model leads\. The state\-space gap widens after the warmup ends on both web corpora; the attention gap contracts on all three\. On enwik8 the complex models are at the data limit by this point \([Section˜8\.3](https://arxiv.org/html/2608.14691#S8.SS3)\), which confounds the comparison there\.![Refer to caption](https://arxiv.org/html/2608.14691v1/x5.png)Figure 5:Validation gap against optimization step, on both sides of the warmup boundary\.Shaded region: the 750\-step learning\-rate warmup; dashed line: its end\. Positive values indicate the complex model leads\. The state\-space gap \(blue\) is flat to increasing after the warmup ends on the two web corpora; the attention gap \(orange\) decays monotonically throughout on all three corpora\. On enwik8 the complex models reach the data limit within the budget, so its late behavior reflects overfitting \([Section˜8\.3](https://arxiv.org/html/2608.14691#S8.SS3)\) rather than the schedule\.The two pairs behave differently, and we report the difference as part of the result\. For the state\-space pair, the gap increases after the warmup ends on both web corpora, from\+0\.321\+0\.321to\+0\.354\+0\.354bpc on OpenWebText and from\+0\.368\+0\.368to\+0\.396\+0\.396on FineWeb\. This is the opposite of what an artifact of the ramp would produce, and it is observed on precisely the corpora where the data limit is not reached within the budget; the one corpus where the gap contracts, enwik8, is the one where the complex models have exhausted a 100 MB training split \([Section˜8\.3](https://arxiv.org/html/2608.14691#S8.SS3)\)\. We therefore regard the warmup explanation as inconsistent with the state\-space measurements, within the budget examined\. Two limits of this evidence should be stated with it\. The interval after the boundary is short, 250 steps of the 1000\-step budget, though the trend is monotone across the twelve validation evaluations it contains; and it is not a constant learning rate but the cosine decay phase, so what the measurement excludes is an artifact of the*ramp*, not every possible interaction between the schedule and the substrate\. A run at constant learning rate remains the control that would exclude the wider class, and we identify it as such in[Section˜10](https://arxiv.org/html/2608.14691#S10)\.

For the attention pair the picture is different: the gap contracts on all three corpora, on both sides of the boundary, reaching\+0\.026\+0\.026and\+0\.034\+0\.034bpc at step 1000 on the two web corpora, and extrapolation of the trend would place it near zero shortly beyond the budget\. The attention advantage measured in[Table˜5](https://arxiv.org/html/2608.14691#S8.T5)is therefore concentrated in early training, and we do not claim that it persists\. This asymmetry is consistent with the mechanism decomposition of[Section˜9\.2](https://arxiv.org/html/2608.14691#S9.SS2): the conditioning mechanism, which the state\-space pair exercises, addresses a structural attenuation that is present at every step, whereas the score mechanism operates against a real attention baseline that is already well conditioned and that closes the gap as training proceeds\.

### 8\.3Terminal performance

Faster convergence is informative only if it is not obtained at the expense of the converged loss\.[Table˜8](https://arxiv.org/html/2608.14691#S8.T8)reports validation loss at the common 1000\-step budget\. H\-Mamba improves on the real Mamba on all three corpora, by0\.3150\.315to0\.3960\.396bpc, equivalently a2020to24%24\\%reduction in per\-byte perplexity\. H\-Transformer improves on the real Transformer on both web corpora, by0\.0260\.026and0\.0340\.034bpc, and trails it on enwik8 for the reason given above\. The per\-step advantage of[Table˜5](https://arxiv.org/html/2608.14691#S8.T5)is the principal finding; the present table indicates that, within the measured budget and in the regime where data is not the binding constraint, the acceleration is not obtained at the expense of the converged loss\. These step\-1000 values are measured 250 steps after the warmup ends, and[Table˜7](https://arxiv.org/html/2608.14691#S8.T7)shows that the state\-space margins are still growing at that point while the attention margins are contracting\.

Table 8:Validation bpc at the 1000\-step common budget\(lower is better\)\. Daggers mark the complex realizations; the FineWeb real\-Mamba entry is its step\-1000 evaluation, the last its log records\. On enwik8 the complex realizations overfit beyond the budget \([Section˜8\.3](https://arxiv.org/html/2608.14691#S8.SS3)\)\.#### Dependence on corpus size\.

The terminal margin of H\-Mamba over the real Mamba increases monotonically with corpus size:0\.3150\.315bpc on enwik8,0\.3540\.354on OpenWebText, and0\.3960\.396on FineWeb\. The attention margin follows the same ordering, from negative on enwik8 to0\.0260\.026on OpenWebText and0\.0340\.034on FineWeb\. Were the substrate’s benefit a fixed representational offset, one would expect the margins to contract on larger corpora as the models approach the achievable loss; the observed widening is in the opposite direction\. A mechanism for the widening is not established by this paper: the vocabulary\-level floor of[Proposition˜6\.1](https://arxiv.org/html/2608.14691#S6.Thmtheorem1)is inactive here \([Section˜6\.1](https://arxiv.org/html/2608.14691#S6.SS1)\), so it cannot supply one, and the reading we consider most natural, that more diverse data exercises more of the phase capacity the recurrence transports, is a hypothesis\. The ordering is nonetheless informative for a different question: the step\-1000 margin is smallest on the corpus where the complex models have exhausted the data and largest on the largest corpus, which is the ordering expected if the contraction of the enwik8 margin reflects the data limit and not the schedule \([Section˜8\.2](https://arxiv.org/html/2608.14691#S8.SS2)\)\. Because each margin is a single run and the increments lie within what a second seed could perturb, the monotone widening itself should also be read as directional rather than established\.

#### Behavior at the data limit\.

On enwik8 the complex models lead at every point of the common budget, reach their minimum near step 1000, and subsequently overfit; the still\-descending real Mamba overtakes them late in the extended budget\. This behavior is not in tension with the central result but is a corollary of it\. A model that extracts more from each step also reaches, in fewer steps, the point at which a finite corpus ceases to constrain it, and 100 MB affords a 253M\-parameter model few enough distinct contexts that this point falls within the budget\. The web corpora, larger by factors of 30 and 150, contain no such point within the budget, and the complex models improve monotonically throughout\. Two practical consequences follow\. At small data scales, the sample efficiency of the substrate translates into an earlier need for regularization or early stopping, the standard treatments; it does not call for a modification of the substrate itself\. For evaluation methodology, a fixed large budget on a small corpus penalizes the fastest learner for exhausting the data first, so comparisons between substrates should be made at matched loss or matched data in addition to matched steps\. The training curves \([Appendix˜C](https://arxiv.org/html/2608.14691#A3)\) coincide with the validation curves up to the point of overfitting, which is consistent with the advantage on the web corpora reflecting learning rather than memorization\.

### 8\.4Throughput and gradient propagation

Optimization steps are one of the two factors in wall\-clock training time, the other being the cost of a step, and we report measured throughput while distinguishing the fair comparisons from the unfair\. The fused Born\-attention kernel of H\-Transformer processes approximately80,00080\{,\}000tokens per second, against approximately160,000160\{,\}000for the real Transformer\. Both employ fused attention kernels \(cf\.[10](https://arxiv.org/html/2608.14691#bib.bib9)\), so the comparison is fair, and the factor of two in per\-step cost offsets the factor of two in per\-step convergence: at present, the two attention models reach a given loss in comparable wall\-clock time, and the net value of the substrate for that pair lies in the data consumed rather than in the time elapsed\. The diagonal scan of H\-Mamba processes approximately48,00048\{,\}000tokens per second; the reference implementation of the real Mamba, which evaluates its scan without kernel fusion, processes approximately7,0007\{,\}000\. We do not report that ratio as a speedup, since it reflects the absence of an optimized baseline kernel rather than a property of the substrate\. Combining step counts with measured throughput yields elapsed\-time estimates: on OpenWebText at the2\.02\.0bpc target, approximately0\.80\.8hours of training for H\-Transformer \(219 steps at80,00080\{,\}000tokens per second\) against approximately0\.90\.9hours for the real Transformer \(484 steps at160,000160\{,\}000\), consistent with the parity stated above; the corresponding state\-space estimate awaits an optimized baseline kernel\. The per\-step result of[Section˜8\.1](https://arxiv.org/html/2608.14691#S8.SS1)is independent of these considerations, as it counts optimization steps under a common protocol and is unaffected by kernel throughput\. A per\-step arithmetic accounting is given in[Appendix˜B](https://arxiv.org/html/2608.14691#A2)\.

The gradient measurement anticipated in[Section˜5\.2](https://arxiv.org/html/2608.14691#S5.SS2), and consistent with[Lemma˜6\.2](https://arxiv.org/html/2608.14691#S6.Thmtheorem2), provides supporting evidence for the conditioning mechanism\. \(Global\-norm clipping rescales all layers by a common factor and therefore does not affect the per\-layer*spread*analyzed here\.\)[Figure˜6](https://arxiv.org/html/2608.14691#S8.F6)reports per\-layer gradient norms over training for H\-Mamba and the real Mamba\. Across the 24 layers of the real Mamba, the per\-layer gradient norm spreads by more than an order of magnitude, the early layers receiving one to two orders of magnitude less signal than the late layers, which is the signature of geometric attenuation over the 1024\-step sequence\. Across the same 24 layers of H\-Mamba the spread is a factor of approximately two\. This pattern is what the decoupling account predicts when trained carrier channels hold their gates near one \([Remark˜6\.3](https://arxiv.org/html/2608.14691#S6.Thmtheorem3)\); it is a supporting diagnostic for the conditioning mechanism, not direct proof of long\-range gradient transmission\. Gradient conditioning and the score’s interaction rule are nonetheless distinct mechanisms that act jointly in the attention realization, and their clean separation requires the factorial ablation specified in[Section˜9\.2](https://arxiv.org/html/2608.14691#S9.SS2); the present measurement is consistent with the retained mechanism being operative, and does not indicate that it acts alone\.

![Refer to caption](https://arxiv.org/html/2608.14691v1/x6.png)Figure 6:Per\-layer gradient norm over trainingfor H\-Mamba and the real Mamba\. H\-Mamba maintains all 24 layers within a factor of approximately two of one another, whereas the real Mamba exhibits a spread of more than an order of magnitude between its early and late layers, consistent with the geometric attenuation of a contractive real recurrence over the 1024\-step sequence\.

## 9Discussion

The experiments show that a single substitution, the deployable complex substrate in place of the real one, is accompanied by a reduction in the number of steps to every measured target, by a factor of approximately three for the state\-space backbone and two for the attention backbone, across corpora spanning a150×150\\timesrange of size\. We examine the locus of the effect \([Section˜9\.1](https://arxiv.org/html/2608.14691#S9.SS1)\), the decomposition of the mechanisms that plausibly produce it \([Section˜9\.2](https://arxiv.org/html/2608.14691#S9.SS2)\), its consequences for training cost \([Section˜9\.3](https://arxiv.org/html/2608.14691#S9.SS3)\), its expected behavior under the scaling variables held fixed here \([Section˜9\.4](https://arxiv.org/html/2608.14691#S9.SS4)\), and its relation to the idealized substrate \([Section˜9\.5](https://arxiv.org/html/2608.14691#S9.SS5)\)\.

### 9\.1The locus of the effect

The two backbones are dissimilar in precisely the respect that matters for an optimization claim\. The state\-space backbone maintains a fixed\-dimensional summary of the past and updates it multiplicatively, so a gradient reaching an early position must traverse the entire recurrence\. The attention backbone recomputes a weighted combination of all past positions at every step, so an early position is reached directly, in one hop\. Any explanation located in the routing, whether in how gradients thread the recurrence or in what attention can express, applies to one backbone and not the other\. An explanation that survives the swap must live in what the two models share, and after the parameter matching of[Section˜5\.7](https://arxiv.org/html/2608.14691#S5.SS7)they share exactly one non\-standard component: the substrate\.

The substrate contributes two properties that are indifferent to routing\. The first is phase transport: the argument of a state coordinate moves through time without distortion, whether the path is a 1024\-step recurrence or an attention sum, because the Cayley phase has unit modulus by construction; the strength at which it is observed remains scaled by the accumulated magnitudes \([Remark˜6\.3](https://arxiv.org/html/2608.14691#S6.Thmtheorem3)\)\. The measurements of[Section˜8\.4](https://arxiv.org/html/2608.14691#S8.SS4)show this property operating in the backbone where its absence is most damaging\. The second is interference: the scoring operation can combine two present components with opposite signs, at a strength set by their magnitudes, which enlarges the function class of the score without a dedicated gating module \(the cancellation mechanism of[Section˜5\.5](https://arxiv.org/html/2608.14691#S5.SS5)\)\. Neither property refers to the manner in which positions are connected\. This is the sense in which substrate and backbone are close to orthogonal axes of design, and the stable, backbone\-characteristic ratios of three and two are what one would expect when a common cause meets two different amounts of headroom\. The independence claim requires one qualification, which[Section˜8\.2](https://arxiv.org/html/2608.14691#S8.SS2)supplies: the early\-phase acceleration appears in both backbones, whereas an advantage that persists past the warmup and continues to widen is observed, within this budget, only for the state\-space backbone\. What transfers across backbones is therefore the early effect; the persistence of the effect is established here only where the conditioning mechanism operates\.

Two experiments would test the attribution directly\. If the effect is substrate\-borne, a third backbone given the same complex state and Born score, for instance a long\-convolution or gated linear\-attention model, should accelerate by its own stable factor, whereas a backbone\-specific account predicts no acceleration there\. If phase transport is the operative property for recurrent backbones, the state\-space ratio should grow with sequence length, since the attenuationρT\\rho^\{T\}of the real baseline worsens geometrically inTTwhile the phase channel does not\. Neither experiment requires new theory, and either could falsify the substrate attribution\.

### 9\.2Decomposition of the mechanism

Two mechanisms could produce the acceleration, and our experiments separate them only partially\. The first is the interaction rule of the score: the Born score is a quadratic interaction with signed cross terms, and, by the attention\-level analogy of[Section˜6\.1](https://arxiv.org/html/2608.14691#S6.SS1)\(the vocabulary\-level floor itself is inactive here\), a richer score can escape rank\-type ceilings that aD=16D=16\-dimensional real dot product faces when ranking10241024positions\. The second is optimization conditioning: the recurrence transports gradients through the phase \([Lemma˜6\.2](https://arxiv.org/html/2608.14691#S6.Thmtheorem2),[Section˜8\.4](https://arxiv.org/html/2608.14691#S8.SS4)\) and supplies interference to the state dynamics\. The pattern of results assigns the mechanisms at two of the four corners of the design\. The vocabulary readout of H\-Mamba is a tied softmax identical to its baseline’s, and it uses no Born score, so no readout mechanism is available to it; its threefold acceleration is accordingly attributed to conditioning\. The Born score of H\-Transformer is a genuinely richer interaction rule, so the score mechanism is available to it in addition to conditioning\.

Surprisingly, the backbone in which both mechanisms are available gains less: a factor of two rather than three\. The resolution lies in the headroom available to each baseline\. The real Transformer is already well conditioned; its gradient paths are one hop long, and no attenuation of the formρT≈3\.4×10−5\\rho^\{T\}\\approx 3\.4\\times 10^\{\-5\}afflicts it, so the conditioning mechanism, the larger of the two, has little left to repair\. The real Mamba, whose contractive recurrence attenuates propagated signal by factors of order10−510^\{\-5\}over its 1024\-step sequences \([Section˜5\.2](https://arxiv.org/html/2608.14691#S5.SS2)\), gives the conditioning mechanism the most room to operate\. This account yields the prediction noted in[Section˜9\.1](https://arxiv.org/html/2608.14691#S9.SS1): the state\-space ratio should grow with sequence length while the attention ratio should not, since only the former baseline deteriorates withTT\. It also anticipates the post\-warmup trajectories of[Section˜8\.2](https://arxiv.org/html/2608.14691#S8.SS2)\. A conditioning defect of the real recurrence is present at every step of training, so an advantage that repairs it should not expire when the schedule stops ramping, which is what the widening state\-space gap shows; a score\-side advantage over a baseline that is already well conditioned has no comparable structural deficit to keep exploiting, and the contracting attention gap is what that predicts\. We note this as a consistency between the mechanism decomposition and an independent measurement, not as a confirmation of the decomposition, which the factorial cells below would test\.

What remains unresolved is the interaction of the two mechanisms, and the experiment that resolves it is a factorial design over two binary factors: the field of the recurrence, real or complex, and the score, affine–softmax or Born\. Two of the four cells are the models of this paper\. The two missing cells are a complex recurrence with a purely softmax pipeline, which isolates conditioning, and a real recurrence with a Born score, which isolates rank; comparing each missing cell with the real baseline measures one mechanism alone, and the amount by which the full substrate exceeds the sum of the two measures their interaction\. A complementary control requires no complex arithmetic at all: a mixture\-of\-softmaxes head\[[61](https://arxiv.org/html/2608.14691#bib.bib13)\]raises the rank of a real readout, and if it closed the gap on the state\-space backbone the readout would be implicated after all, although the fact that H\-Mamba attains its acceleration with an ordinary softmax head already argues otherwise\. We specify these cells in detail because they are the most informative experiments the study leaves undone; the available compute did not permit them, and[Section˜10](https://arxiv.org/html/2608.14691#S10)records this as the principal open question of the paper\.

### 9\.3Steps, tokens, and wall\-clock time

At a fixed batch size, a reduction in optimization steps is an equal reduction in the tokens consumed\. Each step processes1,048,5761\{,\}048\{,\}576tokens \([Table˜4](https://arxiv.org/html/2608.14691#S7.T4)\), so reaching2\.02\.0bpc on FineWeb costs H\-Mamba approximately0\.230\.23B tokens against approximately0\.790\.79B for the real Mamba \([Table˜6](https://arxiv.org/html/2608.14691#S8.T6)\), a saving of roughly0\.570\.57B tokens for that single target\. This is the most direct value of the substrate, and it accrues where training is data\-bound rather than compute\-bound: specialized domains, private corpora, low\-resource languages, and byte\-level settings in which the corpus, and not the accelerator budget, is the scarce resource\. The overfitting of[Section˜8\.3](https://arxiv.org/html/2608.14691#S8.SS3)is the same property observed at the opposite extreme: on a 100 MB corpus, reaching a given loss on one third of the data means exhausting the data three times sooner\.

Wall\-clock time is governed by the product of the step count and the step cost, and the two pairs stand differently in this respect\. For the attention pair, the measured throughputs \(80,00080\{,\}000against160,000160\{,\}000tokens per second\) exactly offset the step ratio of two, so the substrate at present purchases data efficiency at wall\-clock parity\. For the state\-space pair the comparison requires an optimized real baseline kernel, which does not exist in our setup; a comparison against the unfused reference scan would overstate the substrate’s contribution, and we do not make it\. The break\-even condition is nonetheless fixed by the step ratio alone\. With a step ratio of three, the complex scan at48,00048\{,\}000tokens per second reaches any fixed target sooner in wall\-clock time than any real kernel running below approximately144,000144\{,\}000tokens per second at this configuration\. Whether an optimized real scan exceeds that figure is an engineering question, not a scientific one; the accounting of[Appendix˜B](https://arxiv.org/html/2608.14691#A2)indicates that the complex kernel, at present bound by memory bandwidth, not arithmetic, has unexploited headroom of its own\. Until such kernels exist, the established value of the substrate is in steps and tokens, and its value in hours remains conditional\.

### 9\.4Behavior under scaling

Three quantities held fixed in this study govern the extrapolation of the result: the vocabulary size, the state width, and the model scale\. The vocabulary is the 256 byte values, which happens to equalN2=162N^\{2\}=16^\{2\}\. The coincidence would matter for the idealized Born vocabulary readout, whose rank advantage binds whenN2≥VN^\{2\}\\geq V; it is immaterial to the models actually trained, whose vocabulary head is a tied softmax that scales as any language\-model head does\. Because the measured advantage originates in the recurrence and the attention score, it carries no substrate\-specific dependence onVV, and we expect it to persist at subword vocabularies\. This expectation is untested, and verifying it is the first experiment we would undertake\. The component that scales unfavorably is the idealized readout: at vocabularies of 30k to 128k tokens it demands a state width of orderV\\sqrt\{V\}and a readout cost ofO​\(N​V\)O\(NV\), or else the surrender of the exact rank advantage to a softmax\.

The state widthN=16N=16is narrow\. The expressivity of the substrate and the cost of the scan grow linearly inNN, while the rank ceiling of the idealized readout grows asN2N^\{2\}; whether the per\-step advantage grows, saturates, or shrinks withNNis not determined here and bears directly on the value of the substrate at scale\. The model scale is fixed at 253M parameters and the budget at 1000 to 1500 steps\. The regime in which the idealized readout becomes decisive, a vocabulary far exceedingN2N^\{2\}, as with a 50k subword vocabulary against a few hundred state dimensions, is precisely the regime this study cannot reach\. The scaling of the effect inNN,VV, and parameter count, and the point at which restoring the idealized readout becomes worthwhile, are the principal quantities left undetermined; settling them requires experiments at the several\-hundred\-million to billion\-parameter scale, for which the established scaling methodology applies directly\[[30](https://arxiv.org/html/2608.14691#bib.bib23),[26](https://arxiv.org/html/2608.14691#bib.bib24)\]\.

### 9\.5Relation to the idealized substrate

The substrate evaluated here is deliberately the conservative member of a family, and the family has three axes\. The first is the generator of the evolution, ranging from the diagonal phase used here, through block\-diagonal and banded Hermitian generators that couple groups of coordinates, to the dense input\-dependent Hamiltonian of the idealized model; the diagonal choice obtains theO​\(N\)O\(N\)scan at the price of inter\-coordinate coupling within the recurrence, which the readout must then supply\. The second is the norm behavior of the recurrence, ranging from the contractive gate used here to exact unitarity; the gate obtains selective forgetting at the price of the exact gradient guarantee\. The third is the placement of the Born rule, ranging from the attention score used here to the vocabulary readout; the attention placement obtains a cost independent of vocabulary size at the price of theΩ​\(N2\)\\Omega\(N^\{2\}\)readout advantage\. On all three axes we occupy the vertex nearest a production system, chosen so that the substrate could be hosted by an existing backbone and trained without incident\.

The relaxations define the remaining experiments, which are the specific ones the theory ofNebliet al\.\[[37](https://arxiv.org/html/2608.14691#bib.bib12)\]designates rather than generic future work\. Restoring exact unitarity, together with a forgetting mechanism compatible with it, such as unitary dynamics on an extended state with explicit sink coordinates, would test whether norm preservation adds anything beyond the phase transport that the relaxed recurrence already provides; the measurements of[Section˜8\.4](https://arxiv.org/html/2608.14691#S8.SS4)suggest the residual is small, and a null result would itself be informative, licensing the cheaper contractive form\. Moving the Born rule to the vocabulary readout at subword scale would test whether theΩ​\(N2\)\\Omega\(N^\{2\}\)rank advantage materializes as a training advantage where the rank ceiling actually binds\. Since the conservative vertex already yields a stable factor of two to three in optimization steps, a systematic traversal of this space, and the identification of the vertex that is optimal at a given scale, appears at least as well posed a research program as a further iteration on the backbone, whose axis is by now densely explored while the substrate axis remains nearly untouched\.

## 10Limitations

Several limitations bound the conclusions, and we state each together with the experiment that would remove it\. Every cell of the experimental design is a single training run, so no error bars are available, and we calibrate what seed variance could plausibly explain against the smallest effects we report\. The attention margins on the web corpora,0\.0260\.026to0\.0340\.034bpc, and the monotone widening of the margin with corpus size are effects we ourselves treat as potentially seed\-level; they should be read as directional until replicated\. The state\-space margins,0\.3150\.315to0\.3960\.396bpc, are an order of magnitude larger than that self\-declared noise scale, and the corresponding step ratios of approximately three recur across nine independent corpus\-target cells \([Table˜5](https://arxiv.org/html/2608.14691#S8.T5)\); we therefore regard their direction as established\. Three runs per cell on a single corpus would settle both questions\. Four further measurement caveats apply\. Most target crossings occur during or near the 750\-step learning\-rate warmup\. The complex models cross every measured target inside it, between steps 75 and 247, and the real models cross the 2\.5\-bpc target there as well\. The step ratios of[Table˜5](https://arxiv.org/html/2608.14691#S8.T5)are therefore measured in a regime where the schedule is nonstationary, though identical across models\.[Section˜8\.2](https://arxiv.org/html/2608.14691#S8.SS2)addresses this for the state\-space pair, whose advantage widens after the warmup ends on both web corpora, which an artifact of the ramp would not do; it does not address it for the attention pair, whose advantage contracts throughout and which we accordingly describe as an early\-training effect of untested persistence\. A run at constant learning rate, with no warmup and no decay, would settle the question for both pairs directly, and is the control we would add first\. Crossing steps are obtained by linear interpolation between validation evaluations spaced 20 steps apart, which bounds their resolution\. The paired models are parameter\-matched but not functionally identical: the real Mamba carries a parameter\-matching SwiGLU branch, and the attention pair differs in the functional form of the score, including its normalization\. And the per\-layer gradient plots of[Section˜8\.4](https://arxiv.org/html/2608.14691#S8.SS4)are supporting diagnostics for the conditioning mechanism, not direct measurements of long\-range gradient transport\. All models, moreover, share one size, 253M parameters, and one budget regime, 1000 to 1500 steps, which is far from convergence on the web corpora\. The result is therefore a statement about the measured range of the loss, and the persistence of the per\-step advantage under budgets and models one to two orders of magnitude larger is the most consequential question the study leaves open; extending a single pair on one web corpus by an order of magnitude in steps would answer it directly\.

Two further limitations concern interpretation, not measurement\. The deployable substrate relaxes both properties on which the representational theory ofNebliet al\.\[[37](https://arxiv.org/html/2608.14691#bib.bib12)\]rests, so theΩ​\(N2\)\\Omega\(N^\{2\}\)separation describes the idealized model and not the models trained here, and[Proposition˜6\.1](https://arxiv.org/html/2608.14691#S6.Thmtheorem1)bounds the readout in isolation, under an assumption on the probabilities traversed during training, without accounting for the magnitude of the full effect\. The two mechanisms that plausibly produce the acceleration, the conditioning of the optimization by the recurrence and the richer interaction rule of the Born score, are likewise confounded in the attention model, which possesses both; the factorial cells that would separate them, a complex recurrence with a softmax score, a real recurrence with a Born score, and the mixture\-of\-softmaxes control, were not run for want of compute \([Section˜9\.2](https://arxiv.org/html/2608.14691#S9.SS2)\)\. Finally, the throughput comparison for the state\-space backbone is uninformative, because the real baseline runs an unfused reference scan, and all wall\-clock figures are specific to A100\-80GB hardware in half precision; an optimized real\-Mamba kernel at matched configuration would allow the break\-even condition of[Section˜9\.3](https://arxiv.org/html/2608.14691#S9.SS3)to be tested rather than stated\.

A third group of limitations concerns cost, engineering, and evaluation scope\. Complex arithmetic carries a constant\-factor overhead, and current accelerators provide no native complex tensor\-core path, so every complex operation decomposes into real ones; the fused Born\-attention kernel consequently runs at half the throughput of its real counterpart, and the step advantage of the attention pair does not yet translate into wall\-clock savings\. The implementation depends on two nonstandard components for numerical stability, the clamped log\-domain scan of[Section˜5\.3](https://arxiv.org/html/2608.14691#S5.SS3)and the single\-precision accumulation of the fused kernel, without whichbf16training of the substrate is unstable in our experience; adopting the substrate therefore carries a real, if modest, engineering cost\. The training protocol, while identical across models, was not re\-tuned per model, and a per\-model learning\-rate sweep could move the reported ratios in either direction; such a sweep is inexpensive relative to the factorial ablation and is worth performing alongside it\. The theoretical account is partial in a specific sense:[Proposition˜6\.1](https://arxiv.org/html/2608.14691#S6.Thmtheorem1)is inactive at the vocabulary scale used here and[Lemma˜6\.2](https://arxiv.org/html/2608.14691#S6.Thmtheorem2)concerns an isolated component, so together they indicate the direction of each mechanism rather than accounting for the measured magnitudes; a convergence\-rate or conditioning analysis of the joint recurrence\-plus\-readout system, for instance through the Fisher information, is not attempted\. Lastly, the evaluation is confined to language\-modeling loss; calibration, robustness, out\-of\-distribution behavior, and analyses of the learned representations, including the distribution of learned phases and the singular\-value spectra of the states, are natural companion studies that our results do not address\.

## 11Conclusion

Nebliet al\.\[[37](https://arxiv.org/html/2608.14691#bib.bib12)\]established, for an idealized model class on synthetic tasks, that a complex substrate is representationally more powerful than a real one\. This paper establishes the corresponding statement about training\. A deployable form of the substrate, contractive where the idealized model is unitary and with the Born rule confined to the attention score, reaches every measured validation loss in approximately one third of the optimization steps of the real Mamba and one half of those of the real Transformer, from the first hundred steps of training, across corpora spanning a150×150\\timesrange of size, at matched parameter counts\. The two backbones differ in what happens next: the state\-space advantage continues to widen after the learning\-rate warmup ends, while the attention advantage contracts toward zero, so the early acceleration is common to both backbones and its persistence is established here only for the recurrence\. Stated compactly: replacing the number system of the state and the form of the score, while leaving the routing mechanism untouched, reduced the optimization steps of two structurally unrelated models by factors of two to three, and the indifference of the effect to the routing supports attributing it to the substrate\. The relaxations that made the substrate deployable define the experiments that remain: restoring exact unitarity, and returning the Born rule to the vocabulary readout at the scale where its rank advantage is predicted to bind\. On the present evidence, the field over which a model computes is a design decision of the same standing as the mechanism by which it attends\.

## References

- \[1\]M\. Arjovsky, A\. Shah, and Y\. Bengio\(2016\)Unitary evolution recurrent neural networks\.InInternational Conference on Machine Learning \(ICML\),pp\. 1120–1128\.Cited by:[§4\.2](https://arxiv.org/html/2608.14691#S4.SS2.p1.1)\.
- \[2\]Y\. Bengio, P\. Simard, and P\. Frasconi\(1994\)Learning long\-term dependencies with gradient descent is difficult\.IEEE Transactions on Neural Networks5\(2\),pp\. 157–166\.Cited by:[§1](https://arxiv.org/html/2608.14691#S1.p3.8)\.
- \[3\]T\. Brown, B\. Mann, N\. Ryder, M\. Subbiah, J\. Kaplan, P\. Dhariwal, A\. Neelakantan, P\. Shyam, G\. Sastry, A\. Askell,et al\.\(2020\)Language models are few\-shot learners\.InAdvances in Neural Information Processing Systems,Vol\.33,pp\. 1877–1901\.Cited by:[§1](https://arxiv.org/html/2608.14691#S1.p8.2),[§7](https://arxiv.org/html/2608.14691#S7.SS0.SSS0.Px3.p1.6)\.
- \[4\]J\. R\. Busemeyer and P\. D\. Bruza\(2012\)Quantum models of cognition and decision\.Cambridge University Press\.Cited by:[§4\.5](https://arxiv.org/html/2608.14691#S4.SS5.p2.1)\.
- \[5\]B\. Chang, M\. Chen, E\. Haber, and E\. H\. Chi\(2019\)AntisymmetricRNN: a dynamical system view on recurrent neural networks\.InInternational Conference on Learning Representations,Cited by:[§4\.4](https://arxiv.org/html/2608.14691#S4.SS4.p1.1)\.
- \[6\]R\. T\. Q\. Chen, Y\. Rubanova, J\. Bettencourt, and D\. Duvenaud\(2018\)Neural ordinary differential equations\.InAdvances in Neural Information Processing Systems,Cited by:[§4\.5](https://arxiv.org/html/2608.14691#S4.SS5.p1.1)\.
- \[7\]Z\. Chen, J\. Zhang, M\. Arjovsky, and L\. Bottou\(2020\)Symplectic recurrent neural networks\.InInternational Conference on Learning Representations,Cited by:[§4\.4](https://arxiv.org/html/2608.14691#S4.SS4.p1.1)\.
- \[8\]K\. Choromanski, V\. Likhosherstov, D\. Dohan, X\. Song, A\. Gane, T\. Sarlos, P\. Hawkins, J\. Davis, A\. Mohiuddin, L\. Kaiser,et al\.\(2021\)Rethinking attention with Performers\.InInternational Conference on Learning Representations,Cited by:[§4\.3](https://arxiv.org/html/2608.14691#S4.SS3.p1.1)\.
- \[9\]J\. Crank and P\. Nicolson\(1947\)A practical method for numerical evaluation of solutions of partial differential equations of the heat\-conduction type\.Mathematical Proceedings of the Cambridge Philosophical Society43\(1\),pp\. 50–67\.Cited by:[§2](https://arxiv.org/html/2608.14691#S2.SS0.SSS0.Px2.p1.16),[§4\.5](https://arxiv.org/html/2608.14691#S4.SS5.p1.1)\.
- \[10\]T\. Dao, D\. Fu, S\. Ermon, A\. Rudra, and C\. Ré\(2022\)FlashAttention: fast and memory\-efficient exact attention with IO\-awareness\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§8\.4](https://arxiv.org/html/2608.14691#S8.SS4.p1.9)\.
- \[11\]T\. Dao and A\. Gu\(2024\)Transformers are SSMs: generalized models and efficient algorithms through structured state space duality\.InInternational Conference on Machine Learning \(ICML\),Cited by:[§4\.1](https://arxiv.org/html/2608.14691#S4.SS1.p2.1)\.
- \[12\]D\. Y\. Fu, T\. Dao, K\. K\. Saab, A\. W\. Thomas, A\. Rudra, and C\. Ré\(2023\)Hungry hungry hippos: towards language modeling with state space models\.InInternational Conference on Learning Representations,Cited by:[§4\.1](https://arxiv.org/html/2608.14691#S4.SS1.p2.1)\.
- \[13\]F\. A\. Gers, J\. Schmidhuber, and F\. Cummins\(2000\)Learning to forget: continual prediction with LSTM\.Neural Computation12\(10\),pp\. 2451–2471\.Cited by:[§1](https://arxiv.org/html/2608.14691#S1.p7.3)\.
- \[14\]A\. Gokaslan and V\. Cohen\(2019\)OpenWebText corpus\.External Links:[Link](http://skylion007.github.io/OpenWebTextCorpus)Cited by:[§7](https://arxiv.org/html/2608.14691#S7.SS0.SSS0.Px2.p1.5)\.
- \[15\]S\. Greydanus, M\. Dzamba, and J\. Yosinski\(2019\)Hamiltonian neural networks\.InAdvances in Neural Information Processing Systems,Vol\.32\.Cited by:[§4\.4](https://arxiv.org/html/2608.14691#S4.SS4.p1.1)\.
- \[16\]A\. Gu and T\. Dao\(2023\)Mamba: linear\-time sequence modeling with selective state spaces\.arXiv preprint arXiv:2312\.00752\.Cited by:[§1](https://arxiv.org/html/2608.14691#S1.p1.1),[§1](https://arxiv.org/html/2608.14691#S1.p7.3),[§4\.1](https://arxiv.org/html/2608.14691#S4.SS1.p2.1),[§5\.2](https://arxiv.org/html/2608.14691#S5.SS2.p3.4),[§7](https://arxiv.org/html/2608.14691#S7.SS0.SSS0.Px1.p1.3)\.
- \[17\]A\. Gu, K\. Goel, and C\. Ré\(2022\)Efficiently modeling long sequences with structured state spaces\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§1](https://arxiv.org/html/2608.14691#S1.p1.1),[§4\.1](https://arxiv.org/html/2608.14691#S4.SS1.p1.1)\.
- \[18\]A\. Gu, T\. Dao, S\. Ermon, A\. Rudra, and C\. Ré\(2020\)HiPPO: recurrent memory with optimal polynomial projections\.InAdvances in Neural Information Processing Systems,Vol\.33,pp\. 1474–1487\.Cited by:[§4\.1](https://arxiv.org/html/2608.14691#S4.SS1.p1.1)\.
- \[19\]A\. Gu, K\. Goel, A\. Gupta, and C\. Ré\(2022\)On the parameterization and initialization of diagonal state space models\.InAdvances in Neural Information Processing Systems,Vol\.35,pp\. 35971–35983\.Cited by:[§4\.1](https://arxiv.org/html/2608.14691#S4.SS1.p1.1)\.
- \[20\]A\. Gupta, A\. Gu, and J\. Berant\(2022\)Diagonal state spaces are as effective as structured state spaces\.InAdvances in Neural Information Processing Systems,Vol\.35,pp\. 22982–22994\.Cited by:[§4\.1](https://arxiv.org/html/2608.14691#S4.SS1.p1.1)\.
- \[21\]E\. Haber and L\. Ruthotto\(2017\)Stable architectures for deep neural networks\.Inverse Problems34\(1\),pp\. 014004\.Cited by:[§4\.4](https://arxiv.org/html/2608.14691#S4.SS4.p1.1)\.
- \[22\]E\. Hairer, C\. Lubich, and G\. Wanner\(2006\)Geometric numerical integration: structure\-preserving algorithms for ordinary differential equations\.Springer\.Cited by:[§4\.2](https://arxiv.org/html/2608.14691#S4.SS2.p1.1)\.
- \[23\]K\. Helfrich, D\. Willmott, and Q\. Ye\(2018\)Orthogonal recurrent neural networks with scaled Cayley transform\.InInternational Conference on Machine Learning,pp\. 1969–1978\.Cited by:[§4\.2](https://arxiv.org/html/2608.14691#S4.SS2.p1.1),[§4\.2](https://arxiv.org/html/2608.14691#S4.SS2.p2.1)\.
- \[24\]A\. Hirose\(2012\)Complex\-valued neural networks\.2nd edition,Springer\.Cited by:[§4\.2](https://arxiv.org/html/2608.14691#S4.SS2.p1.1)\.
- \[25\]S\. Hochreiter and J\. Schmidhuber\(1997\)Long short\-term memory\.Neural Computation9\(8\),pp\. 1735–1780\.Cited by:[§1](https://arxiv.org/html/2608.14691#S1.p7.3)\.
- \[26\]J\. Hoffmann, S\. Borgeaud, A\. Mensch,et al\.\(2022\)Training compute\-optimal large language models\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§9\.4](https://arxiv.org/html/2608.14691#S9.SS4.p2.7)\.
- \[27\]M\. Hutter\(2006\)The hutter prize and the enwik8 dataset\.External Links:[Link](http://prize.hutter1.net/)Cited by:[§7](https://arxiv.org/html/2608.14691#S7.SS0.SSS0.Px2.p1.5)\.
- \[28\]L\. Jing, Y\. Shen, T\. Dubček, J\. Peurifoy, S\. Skirlo, Y\. LeCun, M\. Tegmark, and M\. Soljačić\(2017\)Tunable efficient unitary neural networks \(EUNN\) and their application to RNNs\.InInternational Conference on Machine Learning,pp\. 1733–1741\.Cited by:[§4\.2](https://arxiv.org/html/2608.14691#S4.SS2.p1.1)\.
- \[29\]S\. Kanai, Y\. Fujiwara, Y\. Yamanaka, and S\. Adachi\(2018\)Sigsoftmax: reanalysis of the softmax bottleneck\.InAdvances in Neural Information Processing Systems,Vol\.31\.Cited by:[§4\.6](https://arxiv.org/html/2608.14691#S4.SS6.p1.4)\.
- \[30\]J\. Kaplan, S\. McCandlish, T\. Henighan, T\. B\. Brown, B\. Chess, R\. Child, S\. Gray, A\. Radford, J\. Wu, and D\. Amodei\(2020\)Scaling laws for neural language models\.arXiv preprint arXiv:2001\.08361\.Cited by:[§9\.4](https://arxiv.org/html/2608.14691#S9.SS4.p2.7)\.
- \[31\]A\. Katharopoulos, A\. Vyas, N\. Pappas, and F\. Fleuret\(2020\)Transformers are RNNs: fast autoregressive transformers with linear attention\.InInternational Conference on Machine Learning \(ICML\),Cited by:[§4\.3](https://arxiv.org/html/2608.14691#S4.SS3.p1.1)\.
- \[32\]D\. P\. Kingma and J\. Ba\(2015\)Adam: a method for stochastic optimization\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§7](https://arxiv.org/html/2608.14691#S7.SS0.SSS0.Px3.p1.6)\.
- \[33\]M\. Lezcano\-Casado and D\. Martínez\-Rubio\(2019\)Cheap orthogonal constraints in neural networks: a simple parametrization of the orthogonal and unitary group\.InInternational Conference on Machine Learning \(ICML\),Cited by:[§4\.2](https://arxiv.org/html/2608.14691#S4.SS2.p1.1),[§4\.2](https://arxiv.org/html/2608.14691#S4.SS2.p2.1)\.
- \[34\]I\. Loshchilov and F\. Hutter\(2017\)SGDR: stochastic gradient descent with warm restarts\.InInternational Conference on Learning Representations,Cited by:[§7](https://arxiv.org/html/2608.14691#S7.SS0.SSS0.Px3.p1.6)\.
- \[35\]I\. Loshchilov and F\. Hutter\(2019\)Decoupled weight decay regularization\.InInternational Conference on Learning Representations,Cited by:[§7](https://arxiv.org/html/2608.14691#S7.SS0.SSS0.Px3.p1.6)\.
- \[36\]D\. P\. Mandic and V\. S\. L\. Goh\(2009\)Complex valued nonlinear adaptive filters: noncircularity, widely linear and neural models\.Wiley\.Cited by:[§4\.2](https://arxiv.org/html/2608.14691#S4.SS2.p1.1)\.
- \[37\]A\. Nebli, H\. Saadatdoorabi, and K\. Yam\(2026\)Deep sequence modeling with quantum dynamics: language as a wave function\.Note:arXiv:2602\.22255External Links:[Link](https://arxiv.org/abs/2602.22255)Cited by:[§1](https://arxiv.org/html/2608.14691#S1.p6.8),[§10](https://arxiv.org/html/2608.14691#S10.p2.1),[§11](https://arxiv.org/html/2608.14691#S11.p1.1),[Table 1](https://arxiv.org/html/2608.14691#S2.T1),[§2](https://arxiv.org/html/2608.14691#S2.p1.1),[§3](https://arxiv.org/html/2608.14691#S3.p1.1),[§3](https://arxiv.org/html/2608.14691#S3.p3.11),[§3](https://arxiv.org/html/2608.14691#S3.p4.1),[§4\.5](https://arxiv.org/html/2608.14691#S4.SS5.p2.1),[Remark 5\.1](https://arxiv.org/html/2608.14691#S5.Thmtheorem1.p1.3),[§9\.5](https://arxiv.org/html/2608.14691#S9.SS5.p2.1)\.
- \[38\]M\. A\. Nielsen and I\. L\. Chuang\(2000\)Quantum computation and quantum information\.Cambridge University Press\.Cited by:[§2](https://arxiv.org/html/2608.14691#S2.SS0.SSS0.Px3.p1.13)\.
- \[39\]A\. Orvieto, S\. L\. Smith, A\. Gu, A\. Fernando, C\. Gulcehre, R\. Pascanu, and S\. De\(2023\)Resurrecting recurrent neural networks for long sequences\.InInternational Conference on Machine Learning \(ICML\),Cited by:[§4\.1](https://arxiv.org/html/2608.14691#S4.SS1.p1.1),[§5\.4](https://arxiv.org/html/2608.14691#S5.SS4.p1.1)\.
- \[40\]R\. Pascanu, T\. Mikolov, and Y\. Bengio\(2013\)On the difficulty of training recurrent neural networks\.InInternational Conference on Machine Learning,pp\. 1310–1318\.Cited by:[§1](https://arxiv.org/html/2608.14691#S1.p3.8),[§7](https://arxiv.org/html/2608.14691#S7.SS0.SSS0.Px3.p1.6)\.
- \[41\]A\. Paszkeet al\.\(2019\)PyTorch: an imperative style, high\-performance deep learning library\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[Table 4](https://arxiv.org/html/2608.14691#S7.T4.14.20.6.2)\.
- \[42\]G\. Penedo, H\. Kydlíček, L\. von Werra, and T\. Wolf\(2024\)The FineWeb datasets: decanting the web for the finest text data at scale\.Note:arXiv:2406\.17557Cited by:[§7](https://arxiv.org/html/2608.14691#S7.SS0.SSS0.Px2.p1.5)\.
- \[43\]B\. Peng, E\. Alcaide, Q\. Anthony, A\. Albalak, S\. Arcadinho, H\. Cao, X\. Cheng, M\. Chung, M\. Grella, K\. GV,et al\.\(2023\)RWKV: reinventing RNNs for the transformer era\.Note:arXiv:2305\.13048Cited by:[§4\.3](https://arxiv.org/html/2608.14691#S4.SS3.p1.1)\.
- \[44\]M\. Poli, S\. Massaroli, E\. Nguyen, D\. Y\. Fu, T\. Dao, S\. Baccus, Y\. Bengio, S\. Ermon, and C\. Ré\(2023\)Hyena hierarchy: towards larger convolutional language models\.InInternational Conference on Machine Learning \(ICML\),Cited by:[§1](https://arxiv.org/html/2608.14691#S1.p1.1),[§4\.1](https://arxiv.org/html/2608.14691#S4.SS1.p2.1)\.
- \[45\]J\. Preskill\(2018\)Quantum computing in the NISQ era and beyond\.Quantum2,pp\. 79\.Cited by:[§4\.5](https://arxiv.org/html/2608.14691#S4.SS5.p2.1)\.
- \[46\]T\. K\. Rusch and S\. Mishra\(2021\)Coupled oscillatory recurrent neural network \(coRNN\): an accurate and \(gradient\) stable architecture for learning long time dependencies\.InInternational Conference on Learning Representations,Cited by:[§4\.4](https://arxiv.org/html/2608.14691#S4.SS4.p1.1)\.
- \[47\]J\. T\. H\. Smith, A\. Warrington, and S\. W\. Linderman\(2023\)Simplified state space layers for sequence modeling\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§4\.1](https://arxiv.org/html/2608.14691#S4.SS1.p1.1),[§5\.4](https://arxiv.org/html/2608.14691#S5.SS4.p1.1)\.
- \[48\]A\. Sordoni, J\. Nie, and Y\. Bengio\(2013\)Modeling term dependencies with quantum language models for IR\.InProceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval,pp\. 653–662\.Cited by:[§4\.5](https://arxiv.org/html/2608.14691#S4.SS5.p2.1)\.
- \[49\]E\. M\. Stoudenmire and D\. J\. Schwab\(2016\)Supervised learning with tensor networks\.InAdvances in Neural Information Processing Systems,Vol\.29\.Cited by:[§4\.2](https://arxiv.org/html/2608.14691#S4.SS2.p1.1)\.
- \[50\]J\. Su, Y\. Lu, S\. Pan, A\. Murtadha, B\. Wen, and Y\. Liu\(2021\)RoFormer: enhanced transformer with rotary position embedding\.arXiv preprint arXiv:2104\.09864\.Cited by:[§4\.3](https://arxiv.org/html/2608.14691#S4.SS3.p1.1),[§5\.7](https://arxiv.org/html/2608.14691#S5.SS7.p3.1),[§7](https://arxiv.org/html/2608.14691#S7.SS0.SSS0.Px1.p1.3)\.
- \[51\]Y\. Sun, L\. Dong, S\. Huang, S\. Ma, Y\. Xia, J\. Xue, J\. Wang, and F\. Wei\(2023\)Retentive network: a successor to Transformer for large language models\.Note:arXiv:2307\.08621Cited by:[§4\.3](https://arxiv.org/html/2608.14691#S4.SS3.p1.1)\.
- \[52\]Y\. Tay, M\. Dehghani, S\. Abnar, Y\. Shen, D\. Bahri, P\. Pham, J\. Rao, L\. Yang, S\. Ruder, and D\. Metzler\(2021\)Long range arena: a benchmark for efficient transformers\.InInternational Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2608.14691#S1.p1.1),[§4\.1](https://arxiv.org/html/2608.14691#S4.SS1.p1.1)\.
- \[53\]Y\. Tay, M\. Dehghani, D\. Bahri, and D\. Metzler\(2022\)Efficient transformers: a survey\.ACM Computing Surveys55\(6\),pp\. 1–28\.Cited by:[§1](https://arxiv.org/html/2608.14691#S1.p1.1)\.
- \[54\]C\. Trabelsi, O\. Bilaniuk, Y\. Zhang, D\. Serdyuk, S\. Subramanian, J\. F\. Santos, S\. Mehri, N\. Rostamzadeh, Y\. Bengio, and C\. J\. Pal\(2018\)Deep complex networks\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§4\.2](https://arxiv.org/html/2608.14691#S4.SS2.p1.1)\.
- \[55\]A\. Vaswani, N\. Shazeer, N\. Parmar, J\. Uszkoreit, L\. Jones, A\. N\. Gomez, L\. Kaiser, and I\. Polosukhin\(2017\)Attention is all you need\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§1](https://arxiv.org/html/2608.14691#S1.p1.1)\.
- \[56\]S\. Wisdom, T\. Powers, J\. Hershey, J\. Le Roux, and L\. Atlas\(2016\)Full\-capacity unitary recurrent neural networks\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§4\.2](https://arxiv.org/html/2608.14691#S4.SS2.p1.1)\.
- \[57\]M\. Wolter and A\. Yao\(2018\)Complex gated recurrent neural networks\.InAdvances in Neural Information Processing Systems,Vol\.31\.Cited by:[§4\.2](https://arxiv.org/html/2608.14691#S4.SS2.p1.1)\.
- \[58\]L\. Xue, A\. Barua, N\. Constant, R\. Al\-Rfou, S\. Narang, M\. Kale, A\. Roberts, and C\. Raffel\(2022\)ByT5: towards a token\-free future with pre\-trained byte\-to\-byte models\.Transactions of the Association for Computational Linguistics10,pp\. 291–306\.Cited by:[§7](https://arxiv.org/html/2608.14691#S7.SS0.SSS0.Px2.p1.5)\.
- \[59\]M\. Yang, M\. Q\. Ma, D\. Li, Y\. H\. Tsai, and R\. Salakhutdinov\(2020\)Complex transformer: a framework for modeling complex\-valued sequence\.InIEEE International Conference on Acoustics, Speech and Signal Processing \(ICASSP\),pp\. 4232–4236\.Cited by:[§4\.2](https://arxiv.org/html/2608.14691#S4.SS2.p1.1)\.
- \[60\]S\. Yang, B\. Wang, Y\. Shen, R\. Panda, and Y\. Kim\(2024\)Gated linear attention transformers with hardware\-efficient training\.InInternational Conference on Machine Learning \(ICML\),Cited by:[§4\.3](https://arxiv.org/html/2608.14691#S4.SS3.p1.1)\.
- \[61\]Z\. Yang, Z\. Dai, R\. Salakhutdinov, and W\. W\. Cohen\(2018\)Breaking the softmax bottleneck: a high\-rank RNN language model\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§1](https://arxiv.org/html/2608.14691#S1.p3.8),[§4\.6](https://arxiv.org/html/2608.14691#S4.SS6.p1.4),[§6\.1](https://arxiv.org/html/2608.14691#S6.SS1.p2.15),[§9\.2](https://arxiv.org/html/2608.14691#S9.SS2.p3.1)\.
- \[62\]Z\. Yang, T\. Luong, R\. Salakhutdinov, and Q\. V\. Le\(2019\)Mixtape: breaking the softmax bottleneck efficiently\.InAdvances in Neural Information Processing Systems,Vol\.32\.Cited by:[§4\.6](https://arxiv.org/html/2608.14691#S4.SS6.p1.4)\.

## Appendix ASubstrate Pseudocode

[Algorithms˜1](https://arxiv.org/html/2608.14691#alg1)and[2](https://arxiv.org/html/2608.14691#alg2)implement the recurrence \([2](https://arxiv.org/html/2608.14691#S5.E2)\) of[Section˜5\.2](https://arxiv.org/html/2608.14691#S5.SS2)without sequential evaluation, and[Algorithm˜3](https://arxiv.org/html/2608.14691#alg3)implements the attention score \([5](https://arxiv.org/html/2608.14691#S5.E5)\) of[Section˜5\.5](https://arxiv.org/html/2608.14691#S5.SS5)\.

Algorithm 1Diagonal Cayley transition coefficient0:phase parameters

ϕ∈ℝN\\boldsymbol\{\\phi\}\\in\\mathbb\{R\}^\{N\}, gate logits

𝐮∈ℝN\\mathbf\{u\}\\in\\mathbb\{R\}^\{N\}
1:

𝝀←\(1\+i​ϕ\)⊘\(1−i​ϕ\)\\boldsymbol\{\\lambda\}\\leftarrow\(1\+i\\boldsymbol\{\\phi\}\)\\oslash\(1\-i\\boldsymbol\{\\phi\}\)\{

\|λk\|=1\|\\lambda\_\{k\}\|=1,

arg⁡λk=2​arctan⁡ϕk\\arg\\lambda\_\{k\}=2\\arctan\\phi\_\{k\}\}

2:

𝜶←σ​\(𝐮\)\\boldsymbol\{\\alpha\}\\leftarrow\\sigma\(\\mathbf\{u\}\)\{selective gate,

αk∈\(0,1\)\\alpha\_\{k\}\\in\(0,1\)\}

3:return

𝐚←𝜶⊙𝝀\\mathbf\{a\}\\leftarrow\\boldsymbol\{\\alpha\}\\odot\\boldsymbol\{\\lambda\}\{

\|ak\|=αk\|a\_\{k\}\|=\\alpha\_\{k\}\}

Algorithm 2Chunked parallel scan forψt\+1=at⊙ψt\+bt\\psi\_\{t\+1\}=a\_\{t\}\\odot\\psi\_\{t\}\+b\_\{t\}0:initial state

x0∈ℂNx\_\{0\}\\in\\mathbb\{C\}^\{N\}, and

log⁡\|𝐚\|\\log\|\\mathbf\{a\}\|,

arg⁡𝐚\\arg\\mathbf\{a\},

𝐛\\mathbf\{b\}over a chunk of length

CC
1:

ℓ←clamp⁡\(cumsum⁡\(log⁡\|𝐚\|\),−30,0\)\\ell\\leftarrow\\operatorname\{clamp\}\(\\operatorname\{cumsum\}\(\\log\|\\mathbf\{a\}\|\),\-30,0\);

ϑ←cumsum⁡\(arg⁡𝐚\)\\vartheta\\leftarrow\\operatorname\{cumsum\}\(\\arg\\mathbf\{a\}\)
2:

P←polar⁡\(eℓ,ϑ\)P\\leftarrow\\operatorname\{polar\}\(e^\{\\ell\},\\vartheta\);

P−1←polar⁡\(e−ℓ,−ϑ\)P^\{\-1\}\\leftarrow\\operatorname\{polar\}\(e^\{\-\\ell\},\-\\vartheta\)
3:return

P⊙\(x0\+cumsum⁡\(𝐛⊙P−1\)\)P\\odot\\big\(x\_\{0\}\+\\operatorname\{cumsum\}\(\\mathbf\{b\}\\odot P^\{\-1\}\)\\big\)\{final state carried to next chunk\}

Algorithm 3Causal Born attention, per head0:complex queries, keys, values

q,k,vq,k,v; temperatures

τ\\tau; head width

DD
1:

si​j←\|⟨qi\|kj⟩\|2/\(D​τi\)s\_\{ij\}\\leftarrow\|\\left\\langle q\_\{i\}\\,\\middle\|\\,k\_\{j\}\\right\\rangle\|^\{2\}/\(D\\,\\tau\_\{i\}\)for

j≤ij\\leq i\{squared modulus, not softmax\}

2:

ψi←\(∑j≤isi​j​vj\)/\(∑j≤isi​j\)\\psi\_\{i\}\\leftarrow\\big\(\\textstyle\\sum\_\{j\\leq i\}s\_\{ij\}v\_\{j\}\\big\)\\big/\\big\(\\sum\_\{j\\leq i\}s\_\{ij\}\\big\)\{causal running\-sum normalization\}

3:return

ψ\\psi\{fused kernel, single\-precision accumulation\}

## Appendix BPer\-Step Arithmetic Accounting

The diagonal Cayley recurrence needsO​\(N\)O\(N\)arithmetic per step per head: forming the transition coefficient and updating the state each cost about6​N6Nreal operations, roughly12​N12Nin total, against the2​N22N^\{2\}floating\-point operations of a dense real transition, a ratio of approximately 3 atN=16N=16\. This ratio is not realized in wall\-clock time, for three reasons\. AtN=16N=16the recurrence kernel is bound by memory bandwidth, not arithmetic: the state resides in registers, and the cost is moving the projections through high\-bandwidth memory\. Complex arithmetic carries a constant\-factor overhead in our implementation\. And the projection costs are shared with the real model, so they cancel in the comparison\. We therefore report measured throughput in[Section˜8\.4](https://arxiv.org/html/2608.14691#S8.SS4)and claim no kernel speedup for the state\-space backbone, whose real baseline runs an unfused reference scan\.

## Appendix CTraining Loss

[Figure˜7](https://arxiv.org/html/2608.14691#A3.F7)reports training loss against optimization step, the counterpart of the validation curves of[Figure˜4](https://arxiv.org/html/2608.14691#S8.F4)\. The two agree: the complex realizations lead on the training objective by the same per\-step margins they show on validation, and the separation appears at the same early step\. This argues against the reading that the validation acceleration is an artifact of the split, since a model that reduced training loss no faster than its counterpart would be unlikely to lead on validation by the observed margin\. Training and validation diverge only on enwik8 beyond the budget, at the overfitting point identified in[Section˜8\.3](https://arxiv.org/html/2608.14691#S8.SS3)\. The real Mamba is omitted from this figure because its training loop logs a sequence\-summed rather than token\-mean loss, which puts its training curve on an incomparable scale; its validation figures, computed by the shared evaluation path, are unaffected and appear throughout the main text\.

![Refer to caption](https://arxiv.org/html/2608.14691v1/x7.png)Figure 7:Training loss against optimization stepfor the complex realizations and the real Transformer\. The per\-step ordering reproduces that of the validation curves in[Figure˜4](https://arxiv.org/html/2608.14691#S8.F4), establishing that the acceleration is a property of learning, not of the validation split\.
## Appendix DReproducibility

Each corpus is read as raw bytes, partitioned90/5/590/5/5into training, validation, and test, and segmented into non\-overlapping windows of 1024 bytes, the target being the input advanced by one byte\. Bits per character is the cross\-entropy in nats divided byln⁡2\\ln 2\. The training loop records the loss every 10 steps and the validation loss every 20, over 20 validation batches during training and over the full validation split at completion\. All twelve training logs are released, together with the parser that produces every table and figure from them; every quantity reported in[Tables˜5](https://arxiv.org/html/2608.14691#S8.T5),[6](https://arxiv.org/html/2608.14691#S8.T6)and[8](https://arxiv.org/html/2608.14691#S8.T8)is re\-derivable from the logs\. The FineWeb real\-Mamba log ends at its step\-1000 evaluation, which is the value[Table˜8](https://arxiv.org/html/2608.14691#S8.T8)reports\. For reference, the components a reimplementation requires are specified as follows: the state initialization and frequency grid in[Section˜5\.6](https://arxiv.org/html/2608.14691#S5.SS6), the Cayley parameterization and gate in[Section˜5\.2](https://arxiv.org/html/2608.14691#S5.SS2), the scan in[Section˜5\.3](https://arxiv.org/html/2608.14691#S5.SS3)and[Appendix˜A](https://arxiv.org/html/2608.14691#A1), the block structure and normalization in[Section˜5\.7](https://arxiv.org/html/2608.14691#S5.SS7), the parameter matching in[Section˜5\.7](https://arxiv.org/html/2608.14691#S5.SS7), and the optimizer, schedule, and precision in[Table˜4](https://arxiv.org/html/2608.14691#S7.T4)\. The training code and the fused kernels accompany the release\.