How Wrong Can a Good Predictor Be? Diverging Updates with Vanishing Predictive KL

# How Wrong Can a Good Predictor Be?Diverging Updates with Vanishing Predictive KL
Source: [https://arxiv.org/html/2609.11132](https://arxiv.org/html/2609.11132)
Qifu WenAffiliation:Boston University, Boston, MA, USAAffiliation:Shanghai Jiao Tong University, Shanghai, ChinaEmail:[qfwen@bu\.edu](mailto:)Shuaijun LiuAffiliation:The Hong Kong University of Science and Technology \(Guangzhou\), Guangzhou, China\*Co\-first authors\.Xi ZengAffiliation:Boston University, Boston, MA, USANingxin SuAffiliation:The Hong Kong University of Science and Technology \(Guangzhou\), Guangzhou, China\*Co\-first authors\.

###### Abstract

Accurate posterior prediction need not require accurate approximation of Bayesian updates\. We prove that an unbounded gap between the update maps can coexist with vanishing predictive KL for every fixed finiteK≥2K\\geq 2in a stationary symmetric Gaussian HMM\. Exact Bayesian mixing and an explicit deterministic radial filter act on the sameK−1K\-1belief coordinates\. Asq→0\+q\\to 0^\{\+\}, their separation in centered logits in the worst case grows at least linearly in the natural confidence scaleLK​\(q\)L\_\{K\}\(q\), while their categoricalDKL\(exact∥radial\)D\_\{\\mathrm\{KL\}\}\(\\mathrm\{exact\}\\\|\\mathrm\{radial\}\)vanishes at the same explicit witness\. Along stationary HMM trajectories, the expected terminal KL between filtered posteriors also converges to zero atH\(q\)=⌈−log\(q\)/c⌉\+1H\(q\)=\\lceil\-\\log\(q\)/c\\rceil\+1\. Typical blocks without switches drive both filters into a common confidence cone, where softmax curvature suppresses their disagreement; a single Gaussian maximal event controls adaptive noise\. A sweep with equally spaced Gaussians overK∈\{2,4,8\}K\\in\\\{2,4,8\\\}illustrates the opposing trends, and binary controls at long horizons compare saturating and nonsaturating recurrences\. The result isolates two missing links between internal update gaps and predictive cost: the contribution of separating states to expected loss and decoder sensitivity\. Thus even an unbounded internal update gap does not by itself certify predictive failure\. The construction is fixed inKKand does not provide a universal criterion for when compression is harmless or characterize when internal gaps must incur task loss\.

## 1Introduction

Transformers retain a growing context, whereas state space and recurrent alternatives compress history into a deterministic state of fixed size\. Recurrence expressivity and state tracking are therefore active design constraints\[[36](https://arxiv.org/html/2609.11132#bib.bib36),[34](https://arxiv.org/html/2609.11132#bib.bib34)\], with formal comparisons also separating simplified selective SSMs from linear attention\[[10](https://arxiv.org/html/2609.11132#bib.bib10)\]\. But these architectural results do not answer the question at the task level: when does a strict computational mismatch force worse prediction? In a finite HMM, exact Bayes itself occupies onlyK−1K\-1belief coordinates; the issue is whether a different update geometry with a fixed state must pay predictive loss\.

Pointwise representability, learnability, and task cost weighted by the distribution are distinct\. The predictive cost of a pointwise separation depends on the loss the decoder can see under the evaluation path law, including the loss magnitude on rare paths\. Impossibility theorems at the architecture level establish representability gaps in specified models\[[56](https://arxiv.org/html/2609.11132#bib.bib56),[54](https://arxiv.org/html/2609.11132#bib.bib54)\], whereas learnability can fail even for representable functions\[[27](https://arxiv.org/html/2609.11132#bib.bib27)\]\. Neither distinction alone determines this task cost\. We study this boundary in a sequential model with finitely many states where both can be calculated\.

Our result has two levels\. At the level of the update maps, for every fixed finiteK≥2K\\geq 2an explicit moving witness makes the quotient distance in centered logits between exact Bayesian mixing and a radial filter grow asΩ​\(LK​\(q\)\)\\Omega\(L\_\{K\}\(q\)\), while their categoricalDKL\(exact∥radial\)D\_\{\\mathrm\{KL\}\}\(\\mathrm\{exact\}\\\|\\mathrm\{radial\}\)at that same witness tends to zero\. At the level of the path law, with pairwise distinct scalar Gaussian means and slow symmetric switching, the filters consume the same stationary HMM observations and their expected terminal KL between filtered posteriors atH\(q\)=⌈−log\(q\)/c⌉\+1H\(q\)=\\lceil\-\\log\(q\)/c\\rceil\+1also tends to zero\. Thus even an unbounded internal separation need not induce a nonvanishing performance gap\.

The result at the level of the maps is decoder geometry: both updates become increasingly confident in the same state, so softmax erases a growing logit discrepancy\. The theorem under the path law adds a distributional argument\. On a block with no latent switch, which occurs with high probability, one maximal event controls all adaptive Gaussian noise weights, radial saturation keeps the approximation in a confidence cone with exact Bayes, and a uniform KL envelope absorbs the remaining paths\.

A sweep with equally spaced Gaussians overK∈\{2,4,8\}K\\in\\\{2,4,8\\\}illustrates the two opposing trends at finite scale: centered states separate while predictive KL shrinks\. The binary case is a microscope rather than the theorem’s scope: it visualizes the geometry and separates saturation from tanh specifically\. The training experiments do not establish transfer to learned models\.

We prove that, for fixedKK, internal separation can diverge while predictive KL vanishes at the same input\. We also prove convergence of expected predictive KL under the stationary law as the horizon grows in the limit of rare switching\. The proof explains how decoder sensitivity and the contribution of separating states to expected loss control predictive error\. Controlled numerical experiments illustrate the two results; simpler recurrences also achieve small KL at the tested short horizons \(Table[7](https://arxiv.org/html/2609.11132#A6.T7)\)\. We do not claim uniformity in growingKK, an impossibility result for a class of architectures, or learnability of the radial rule\.

Figure[1](https://arxiv.org/html/2609.11132#S1.F1)distinguishes the statements at the common input and under the path law\.

Figure 1:Two distinct fixed\-KKseparations\.\(A\) At a common moving input, exact and radial mixing have diverging separation in centered logits but vanishing categorical KL\. \(B\) On shared stationary observations from the Gaussian HMM, their separate recurrences have vanishing expected terminal posterior KL atH⁡\(q\)H\(q\)\. Both statements fixKKand emission parameters beforeq→0\+q\\to 0^\{\+\}; the schematic does not assert that \(A\) implies \(B\)\.
## 2Related Work

##### Expressivity, learning, and task cost\.

Efficient sequence models make recurrence expressivity an explicit design variable\. Recent work strengthens selective state space updates, analyzes their limits on tracking the state, and compares simplified selective layers with linear attention\[[36](https://arxiv.org/html/2609.11132#bib.bib36),[34](https://arxiv.org/html/2609.11132#bib.bib34),[10](https://arxiv.org/html/2609.11132#bib.bib10),[56](https://arxiv.org/html/2609.11132#bib.bib56),[1](https://arxiv.org/html/2609.11132#bib.bib1)\]\. Results on the worst case and on formal languages identify computations that specified architectures cannot represent\[[43](https://arxiv.org/html/2609.11132#bib.bib43),[54](https://arxiv.org/html/2609.11132#bib.bib54)\], while complementary results show that expressible functions can still be hard to learn\[[27](https://arxiv.org/html/2609.11132#bib.bib27)\]\. These results determine representation or optimization, not the predictive cost weighted by the distribution of a mismatch\. Predictive cost additionally depends on decoder sensitivity and the contribution of the separating states weighted by loss under the evaluation path law\. With unbounded KL, vanishing event probability alone does not control this contribution\. An expressivity analysis for a specific task for GNNs makes a related diagnosis\[[35](https://arxiv.org/html/2609.11132#bib.bib35)\]; our contribution is an explicit sequential counterexample with a proved loss under the path law\.

##### Harmless information loss and internal divergence\.

Compression that is aware of the task already shows that discarded information can be harmless relative to a downstream predictor family or distortion\[[16](https://arxiv.org/html/2609.11132#bib.bib16),[15](https://arxiv.org/html/2609.11132#bib.bib15),[26](https://arxiv.org/html/2609.11132#bib.bib26)\]\. We do not optimize a representation or bit rate: the state dimension and prescribed update family are fixed, while their numerical maps depend onqqand their distance in centered logits diverges\. The generic mathematical shape is not new in isolation\. On separable data, gradient descent can send linear predictor norms to infinity while logistic loss tends to zero and the normalized direction converges\[[62](https://arxiv.org/html/2609.11132#bib.bib62)\]\. Our conjunction is different: two explicit filtering maps diverge at the same centered input, their categorical KL vanishes there, and a second theorem proves predictive convergence under a stationary sequential law\.

##### Approximate Bayesian filtering and finite memory\.

RNNs can approximate Bayesian filtering statistics on fixed finite horizons and, under additional conditions, uniformly in time\[[5](https://arxiv.org/html/2609.11132#bib.bib5)\]\. Neural Bayesian Filtering instead learns belief embeddings of fixed length and proves particle consistency over a finite horizon under its stated assumptions\[[61](https://arxiv.org/html/2609.11132#bib.bib61)\]\. Prediction with short memory gives a different benchmark: an order\-ℓ\\ellMarkov predictor attains average KL at mostI⁡\(ℳ\)/ℓI\(\\mathcal\{M\}\)/\\elland averageℓ1\\ell\_\{1\}error at mostI⁡\(ℳ\)/ℓ\\sqrt\{I\(\\mathcal\{M\}\)/\\ell\}\[[57](https://arxiv.org/html/2609.11132#bib.bib57)\]\. For annn\-state HMM,I⁡\(ℳ\)≤log⁡nI\(\\mathcal\{M\}\)\\leq\\log n, independently of mixing time\. This benchmark concerns distributions of the next observation, whereas our theorem measures categorical filtered posteriors\. Our predictor maintainsK−1K\-1belief coordinates\. Streaming projection instead retains a truncated mixture over latent paths with a fixed budget through a deterministic recurrence\[[17](https://arxiv.org/html/2609.11132#bib.bib17)\]\. The distinction is the maintained object and approximation operation, not recurrence itself; our result couples diverging centered map distance with vanishing categorical KL\.

##### Filter stability and robustness\.

Classical filter stability asks whether filters with different initial conditions merge while running the correct model\[[46](https://arxiv.org/html/2609.11132#bib.bib46),[9](https://arxiv.org/html/2609.11132#bib.bib9),[41](https://arxiv.org/html/2609.11132#bib.bib41)\]\. Robustness theory instead perturbs model kernels and controls policy costs\[[31](https://arxiv.org/html/2609.11132#bib.bib31),[33](https://arxiv.org/html/2609.11132#bib.bib33)\]or error in the filter kernel and policy performance\[[14](https://arxiv.org/html/2609.11132#bib.bib14)\]under the respective assumptions\. Control policies with a finite window can also become nearly optimal as the window grows\[[32](https://arxiv.org/html/2609.11132#bib.bib32)\]\. Our recurrence is neither a differently initialized exact filter nor a sequence of kernels converging to the true one; its pointwise update mismatch grows on the stated witness\.

##### Detection delay and the binary controls\.

The affine control at the long endpoint is adjacent to quickest change detection\. Page introduced the cumulative sum scheme\[[48](https://arxiv.org/html/2609.11132#bib.bib48)\]; Lorden introduced a criterion on conditional delay in the worst case and proved asymptotic optimality, Moustakides established exact CUSUM optimality under it, and Pollak studied nearly minimax detection under a different criterion on conditional delay\[[40](https://arxiv.org/html/2609.11132#bib.bib40),[44](https://arxiv.org/html/2609.11132#bib.bib44),[52](https://arxiv.org/html/2609.11132#bib.bib52)\]\. Pollak’s irreversible model with a single change is adjacent to but not identical to our HMM that switches repeatedly\. These works calibrate the binary diagnostic but do not imply the fixed\-KKradial theorem\.

##### Why state space models are the motivating case\.

The design pattern of a deterministic state spans structured SSMs, linear RNNs, gated linear attention, and recurrent competitors\[[24](https://arxiv.org/html/2609.11132#bib.bib24),[25](https://arxiv.org/html/2609.11132#bib.bib25),[60](https://arxiv.org/html/2609.11132#bib.bib60),[47](https://arxiv.org/html/2609.11132#bib.bib47),[67](https://arxiv.org/html/2609.11132#bib.bib67),[68](https://arxiv.org/html/2609.11132#bib.bib68),[49](https://arxiv.org/html/2609.11132#bib.bib49),[3](https://arxiv.org/html/2609.11132#bib.bib3),[12](https://arxiv.org/html/2609.11132#bib.bib12)\]\. Selective SSM theory and studies of state tracking show how transition structure, composition, precision, or normalization alter representational capability\[[45](https://arxiv.org/html/2609.11132#bib.bib45),[59](https://arxiv.org/html/2609.11132#bib.bib59),[7](https://arxiv.org/html/2609.11132#bib.bib7),[64](https://arxiv.org/html/2609.11132#bib.bib64),[58](https://arxiv.org/html/2609.11132#bib.bib58),[38](https://arxiv.org/html/2609.11132#bib.bib38)\]\. They motivate the logical question studied here\. We do not prove that any named architecture implements, or fails to implement, either filter in our theorem\.

## 3Problem Setup: Gaussian HMM with a Fixed Finite State and Radial Filter

###### Definition 1\(StationaryKK\-state Gaussian HMM and coupled filters\)\.

Fix an integerK≥2K\\geq 2, pairwise distinct meansμ0,…,μK−1∈ℝ\\mu\_\{0\},\\ldots,\\mu\_\{K\-1\}\\in\\mathbb\{R\}, andσ\>0\\sigma\>0, with0<q<\(K−1\)/K0<q<\(K\-1\)/Kso thatLK​\(q\)\>0L\_\{K\}\(q\)\>0andαK​\(q\)\>0\\alpha\_\{K\}\(q\)\>0\. Let\(St\)t≥0\(S\_\{t\}\)\_\{t\\geq 0\}be stationary and uniform on\{0,…,K−1\}\\\{0,\\ldots,K\-1\\\}, with

Pr\(St\+1=i∣St=i\)=1−q,Pr\(St\+1=j∣St=i\)=qK−1\(j≠i\)\.\\Pr\(S\_\{t\+1\}=i\\mid S\_\{t\}=i\)=1\-q,\\qquad\\Pr\(S\_\{t\+1\}=j\\mid S\_\{t\}=i\)=\\frac\{q\}\{K\-1\}\\quad\(j\\neq i\)\.Conditional onSt=sS\_\{t\}=s, letXt\+1∼𝒩⁡\(μs,σ2\)X\_\{t\+1\}\\sim\\mathcal\{N\}\(\\mu\_\{s\},\\sigma^\{2\}\)\. Write𝒞​z=z−K−1​\(𝟏⊤​z\)​𝟏\\mathcal\{C\}z=z\-K^\{\-1\}\(\\mathbf\{1\}^\{\\top\}z\)\\mathbf\{1\}for centering and

g​\(x\)s=μs​xσ2−μs22​σ2g\(x\)\_\{s\}=\\frac\{\\mu\_\{s\}x\}\{\\sigma^\{2\}\}\-\\frac\{\\mu\_\{s\}^\{2\}\}\{2\\sigma^\{2\}\}for the Gaussian score, understood modulo common shifts\. For centered logitszz, define exact transition mixing

Φq​\(z\)=𝒞​log⁡\(Pq⊤​softmax⁡\(z\)\),\\Phi\_\{q\}\(z\)=\\mathcal\{C\}\\log\(P\_\{q\}^\{\\top\}\\operatorname\{softmax\}\(z\)\),where the logarithm is componentwise\. Let

r⁡\(z\)=2​‖𝒞​z‖2,αK​\(q\)=1−K​qK−1,LK​\(q\)=log⁡\(K−1\)​\(1−q\)q,r\(z\)=\\sqrt\{2\}\\,\\\|\\mathcal\{C\}z\\\|\_\{2\},\\qquad\\alpha\_\{K\}\(q\)=1\-\\frac\{Kq\}\{K\-1\},\\qquad L\_\{K\}\(q\)=\\log\\frac\{\(K\-1\)\(1\-q\)\}\{q\},and define the radial map

Rq​\(z\)=LK​\(q\)​tanh⁡\(αK​\(q\)​r​\(z\)/LK​\(q\)\)r⁡\(z\)​𝒞​z,R\_\{q\}\(z\)=\\frac\{L\_\{K\}\(q\)\\tanh\(\\alpha\_\{K\}\(q\)r\(z\)/L\_\{K\}\(q\)\)\}\{r\(z\)\}\\,\\mathcal\{C\}z,with continuous valueRq​\(0\)=0R\_\{q\}\(0\)=0and scaleαK​\(q\)\\alpha\_\{K\}\(q\)at the origin\. Starting fromz0E=z0R=0z^\{E\}\_\{0\}=z^\{R\}\_\{0\}=0, the coupled filters use the same observations:

zt\+1E=𝒞⁡\(Φq​\(ztE\)\+g⁡\(Xt\+1\)\),zt\+1R=𝒞⁡\(Rq​\(ztR\)\+g⁡\(Xt\+1\)\)\.z^\{E\}\_\{t\+1\}=\\mathcal\{C\}\\\!\\left\(\\Phi\_\{q\}\(z^\{E\}\_\{t\}\)\+g\(X\_\{t\+1\}\)\\right\),\\qquad z^\{R\}\_\{t\+1\}=\\mathcal\{C\}\\\!\\left\(R\_\{q\}\(z^\{R\}\_\{t\}\)\+g\(X\_\{t\+1\}\)\\right\)\.Their terminal categorical outputs areptE=softmax⁡\(ztE\)p^\{E\}\_\{t\}=\\operatorname\{softmax\}\(z^\{E\}\_\{t\}\)andptR=softmax⁡\(ztR\)p^\{R\}\_\{t\}=\\operatorname\{softmax\}\(z^\{R\}\_\{t\}\)\. Fort≥1t\\geq 1,ptEp^\{E\}\_\{t\}is the filtered posterior ofSt−1S\_\{t\-1\}givenX1:tX\_\{1:t\};ptRp^\{R\}\_\{t\}approximates that posterior\. These outputs precede any additional transition or emission channel\.

Define

dmin=mini≠j⁡\(μi−μj\)22​σ2\.d\_\{\\min\}=\\min\_\{i\\neq j\}\\frac\{\(\\mu\_\{i\}\-\\mu\_\{j\}\)^\{2\}\}\{2\\sigma^\{2\}\}\.Fix0<c<dmin0<c<d\_\{\\min\}and setH⁡\(q\)=⌈\(−log⁡q\)/c⌉\+1H\(q\)=\\lceil\(\-\\log q\)/c\\rceil\+1\. Constants below may depend on the fixedKK, the full mean vector,σ\\sigma, andcc\. Appendix[B](https://arxiv.org/html/2609.11132#A2)and Table[4](https://arxiv.org/html/2609.11132#A2.T4)translate these centered coordinates into a geometric example with three states\.

## 4Main Result and Proof Mechanism

###### Definition 2\(Centered separation between the update maps\)\.

For centeredz∈ℝKz\\in\\mathbb\{R\}^\{K\}, define the quotient distance

𝒟q​\(z\)=maxi,j⁡\|\[\(Φq​\(z\)\)i−\(Φq​\(z\)\)j\]−\[\(Rq​\(z\)\)i−\(Rq​\(z\)\)j\]\|\.\\mathcal\{D\}\_\{q\}\(z\)=\\max\_\{i,j\}\\left\|\[\(\\Phi\_\{q\}\(z\)\)\_\{i\}\-\(\\Phi\_\{q\}\(z\)\)\_\{j\}\]\-\[\(R\_\{q\}\(z\)\)\_\{i\}\-\(R\_\{q\}\(z\)\)\_\{j\}\]\\right\|\.It ignores common logit shifts, which do not change a categorical prediction\.

###### Theorem 3\(Diverging updates with vanishing predictive KL\)\.

Fix any finiteK≥2K\\geq 2, letL=LK​\(q\)L=L\_\{K\}\(q\), and set

zq=L\(1/4,−1/4,0,…,0\),c⋆=12−tanh12\>0\.z\_\{q\}=L\(1/4,\-1/4,0,\\ldots,0\),\\qquad c\_\{\\star\}=\\frac\{1\}\{2\}\-\\tanh\\frac\{1\}\{2\}\>0\.Then

\[\(Φq​\(zq\)\)0−\(Φq​\(zq\)\)1\]−\[\(Rq​\(zq\)\)0−\(Rq​\(zq\)\)1\]L⟶c⋆\(q→0\+\)\.\\frac\{\[\(\\Phi\_\{q\}\(z\_\{q\}\)\)\_\{0\}\-\(\\Phi\_\{q\}\(z\_\{q\}\)\)\_\{1\}\]\-\[\(R\_\{q\}\(z\_\{q\}\)\)\_\{0\}\-\(R\_\{q\}\(z\_\{q\}\)\)\_\{1\}\]\}\{L\}\\longrightarrow c\_\{\\star\}\\qquad\(q\\to 0^\{\+\}\)\.Consequently, for all sufficiently smallqq, with the threshold allowed to depend on fixedKK,

supz:𝒞​z=z𝒟q\(z\)≥𝒟q\(zq\)≥c⋆2LK\(q\)⟶∞\.\\sup\_\{z:\\mathcal\{C\}z=z\}\\mathcal\{D\}\_\{q\}\(z\)\\geq\\mathcal\{D\}\_\{q\}\(z\_\{q\}\)\\geq\\frac\{c\_\{\\star\}\}\{2\}L\_\{K\}\(q\)\\longrightarrow\\infty\.Nevertheless, at the same witness,

DKL​\(softmax⁡Φq​\(zq\)∥softmax⁡Rq​\(zq\)\)⟶0\.D\_\{\\mathrm\{KL\}\}\\\!\\left\(\\operatorname\{softmax\}\\Phi\_\{q\}\(z\_\{q\}\)\\,\\middle\\\|\\,\\operatorname\{softmax\}R\_\{q\}\(z\_\{q\}\)\\right\)\\longrightarrow 0\.

###### Theorem 4\(Predictive convergence with a fixed finite state\)\.

Setting\.For every fixed finiteK≥2K\\geq 2, consider Definition[1](https://arxiv.org/html/2609.11132#Thmtheorem1)with pairwise distinct means,σ\>0\\sigma\>0, and0<c<dmin0<c<d\_\{\\min\}\. The two filters share the stationary HMM observation path and are evaluated atH⁡\(q\)=⌈\(−log⁡q\)/c⌉\+1H\(q\)=\\lceil\(\-\\log q\)/c\\rceil\+1\.

Conclusion\.

𝔼⁡\[DKL​\(pH⁡\(q\)E∥pH⁡\(q\)R\)\]⟶0as​q→0\+\.\\mathbb\{E\}\\\!\\left\[D\_\{\\mathrm\{KL\}\}\\\!\\left\(p^\{E\}\_\{H\(q\)\}\\,\\middle\\\|\\,p^\{R\}\_\{H\(q\)\}\\right\)\\right\]\\longrightarrow 0\\qquad\\text\{as \}q\\to 0^\{\+\}\.The expectation is under the stationary HMM path law\.

##### Logical separation\.

Theorem[3](https://arxiv.org/html/2609.11132#Thmtheorem3)is a statement about the maps in the worst case, with a matched decoder statement: the internal distance diverges at an explicit moving witness, yet the two decoded predictions agree there\. Theorem[4](https://arxiv.org/html/2609.11132#Thmtheorem4)is a distributional statement: predictive agreement also holds in expectation along trajectories generated by the stationary HMM\. The first rules out the explanation that the update maps merely become uniformly close; the second shows that the phenomenon is not confined to one logit vector chosen by hand\.

##### Mechanism\.

At the explicit witness, both maps place the same coordinate ahead by a margin proportional toLK​\(q\)L\_\{K\}\(q\), so categorical softmax curvature suppresses their growing internal discrepancy\. Along HMM paths, with probability tending to one the logarithmic window contains no latent switch\. Conditional on a fixed state, the centered score decomposes into a deterministic direction and one scalar Gaussian noise direction\. Because every radial mixing coefficient lies in\(0,1\)\(0,1\), Abel summation controls the full adaptive noise process by one maximum of a Gaussian partial sum\. ForK≥3K\\geq 3, a radial barrier keeps the state at orderLK​\(q\)2/3L\_\{K\}\(q\)^\{2/3\}and a matching lower bound drives every margin of the true state over a wrong state to the same order after an initial transient\. Exact Bayes has an even larger margin\. The separate binary argument supplies a common logarithmic margin\. The curvature bound in the common confidence cone makes the KL vanish on this event, while a2​LK​\(q\)2L\_\{K\}\(q\)envelope absorbs switch paths and failures of the maximal event\. Appendix[D](https://arxiv.org/html/2609.11132#A4)explains these proof steps and the dependence of the constants onKK\.

##### Scope\.

Both results fixKKbefore takingq→0\+q\\to 0^\{\+\}\. The quantitative witness moves outward withLK​\(q\)L\_\{K\}\(q\); it does not imply a gap bounded away from zero on every fixed bounded input\. Neither theorem is uniform in growingKKor nearly colliding means, and neither is an impossibility theorem for an entire SSM architecture class or a claim about what training by gradient descent will learn\. Table[1](https://arxiv.org/html/2609.11132#S4.T1)separates proved statements, evidence at finite scale, and exclusions\.

Table 1:Summary of results\.The first three rows are mathematical results\. The fourth is an illustration at finite scale, and the last row records exclusions rather than negative results\.

## 5Evidence at Finite Scale for Fixed K

Theorem[4](https://arxiv.org/html/2609.11132#Thmtheorem4)is asymptotic and does not specify a universal finite\-qqonset\. We therefore use a deliberately simple numerical illustration that matches its Gaussian assumptions\. ForK∈\{2,4,8\}K\\in\\\{2,4,8\\\}, the state means are equally spaced, centered, and rescaled to minimum pairwise distance one\. We evaluateL∈\{8,16,32,64,128\}L\\in\\\{8,16,32,64,128\\\}with 4,096 paired stationary paths per cell\. Exact Bayes and the radial filter consume the same observations\. The two displayed quantities are the centered distance between terminal states and the categoricalDKL\(exact∥radial\)D\_\{\\mathrm\{KL\}\}\(\\mathrm\{exact\}\\\|\\mathrm\{radial\}\)after the observation\.

Figure[2](https://arxiv.org/html/2609.11132#S5.F2)shows increasing sampled state distance and decreasing categorical KL across all three displayed values ofKK: internal distance grows with the scale of rare switching while predictive KL falls\. The curves are not used to establish the theorem or to claim a uniform onset inKK\. Appendix[F](https://arxiv.org/html/2609.11132#A6)gives the estimands, coupling, uncertainty calculation, and exact summary statistics\.

Figure 2:Separation at finite scale and decoder saturation\.Left pair: centered distance between terminal states grows while categoricalDKL\(exact∥radial\)D\_\{\\mathrm\{KL\}\}\(\\mathrm\{exact\}\\\|\\mathrm\{radial\}\)falls for Gaussian HMMs withK=2,4,8K=2,4,8\(4,096 paired paths per cell; 95% bootstrap intervals\)\. Right pair: KL of the binary controls over ten seeds atH⁡\(q\)H\(q\)and⌈8/q⌉\\lceil 8/q\\rceil\(relative SE≤3\.1%\\leq 3\.1\\%\), followed by an analytic illustration with a logistic decoder, not sampled trajectories\. Nearly equal endpoints are displaced for legibility\. Table[7](https://arxiv.org/html/2609.11132#A6.T7)and Appendix[F](https://arxiv.org/html/2609.11132#A6)give values, clipping conventions, and estimation details\. Measurements illustrate, rather than prove, the asymptotic theorem\.
## 6Binary Specialization and Geometry in the Finite Regime

ForK=2K=2, translating the two means by their midpoint reduces the model to\(−μ,\+μ\)\(\-\\mu,\+\\mu\)without changing likelihood ratios\. Centered logits then collapse to one scalarhh\. Exact mixing becomesFq​\(h\)F\_\{q\}\(h\)and the radial witness becomes the tanh clampGq​\(h\)G\_\{q\}\(h\)below\. Theorem[3](https://arxiv.org/html/2609.11132#Thmtheorem3)already supplies an explicit, diverging separation at the level of the maps for every fixedKK\. This binary slice instead serves as a visualization in the finite regime and isolates why an affine map cannot represent exact mixing; the stationary fixed\-KKtheorem does not depend on the affine obstruction proved here\.

###### Proposition 5\(Geometry of exact mixing\)\.

For every0<q<120<q<\\tfrac\{1\}\{2\}the exact mixing mapFqF\_\{q\}of Definition[1](https://arxiv.org/html/2609.11132#Thmtheorem1)satisfies:

1. \(i\)Saturation:\|Fq​\(h\)\|<L​\(q\)\|F\_\{q\}\(h\)\|<L\(q\)for every finitehh\.
2. \(ii\)Contraction:FqF\_\{q\}contracts logit perturbations with global factor at most1−2​q1\-2q, attained at the midpoint\.
3. \(iii\)Not affine: withx=log⁡2x=\\log 2and Δq:=\|Fq​\(0\)−2​Fq​\(x\)\+Fq​\(2​x\)\|\>0,\\Delta\_\{q\}:=\\bigl\|F\_\{q\}\(0\)\-2F\_\{q\}\(x\)\+F\_\{q\}\(2x\)\\bigr\|\>0,every affine mapAAsatisfiesmaxz∈\{0,x,2​x\}⁡\|Fq​\(z\)−A⁡\(z\)\|≥Δq/4\\max\_\{z\\in\\\{0,x,2x\\\}\}\|F\_\{q\}\(z\)\-A\(z\)\|\\geq\\Delta\_\{q\}/4\.

*Proof\.*See Appendix[E](https://arxiv.org/html/2609.11132#A5)\.□\\square

Moreover, predictive loss is evaluated after the logistic decoder under the HMM path law, not in the logit norm in the worst case\. The approximate mapGqG\_\{q\}preserves the exact saturation scale and midpoint slope, and the shared envelope\|Fq​\(h\)\|,\|Gq​\(h\)\|<L⁡\(q\)\|F\_\{q\}\(h\)\|,\|G\_\{q\}\(h\)\|<L\(q\)gives global control without requiring uniform closeness\.

The exact, radial, and affine transition geometry is shown in Figure[3](https://arxiv.org/html/2609.11132#A3.F3)in the appendix\.

## 7Binary Controls and Learned Transfer

The experiment with learned models asks whether recurrences faithful to the architecture recover the proved mechanism across six switch probabilities\. When trained end to end, the selective SSM arm closes at least half the reference gap at four of sixqqvalues, below the prespecified criterion of five out of six; these results do not establish transfer to learned models\. Under distillation, every nonreference comparison arm meets the threshold of half closure at all six values\.

Closure is defined as\(KLS6−KLarm\)/\(KLS6−KLtanh\)\(\\mathrm\{KL\}\_\{\\mathrm\{S6\}\}\-\\mathrm\{KL\}\_\{\\mathrm\{arm\}\}\)/\(\\mathrm\{KL\}\_\{\\mathrm\{S6\}\}\-\\mathrm\{KL\}\_\{\\tanh\}\), so the constrained scalar S6 arm sits at00, the analytic tanh recurrence sits at11, and values above11mean the learned arm beats the proved mechanism\. Here every architecture loss isDKL\(exact Bayes∥model\)D\_\{\\mathrm\{KL\}\}\(\\text\{exact Bayes\}\\\|\\text\{model\}\), evaluated after converting logits to float32 probabilities and clipping both arguments to\[ε32,1−ε32\]\[\\varepsilon\_\{32\},1\-\\varepsilon\_\{32\}\], whereε32=torch\.finfo\(torch\.float32\)\.eps≈1\.1920929×10−7\\varepsilon\_\{32\}=\\texttt\{torch\.finfo\(torch\.float32\)\.eps\}\\approx 1\.1920929\\times 10^\{\-7\}\. This differs from the metric for the scalar controls in Table[7](https://arxiv.org/html/2609.11132#A6.T7), which uses10−1510^\{\-15\}clipping\. Table[2](https://arxiv.org/html/2609.11132#S7.T2)gives the breakdown at the horizon matched to the theoremH⁡\(q\)=⌈−ln⁡q⌉\+1H\(q\)=\\lceil\-\\ln q\\rceil\+1\.

Table 2:Closure perqqwhen trained end to end, mean±\\pmstandard error over ten evaluation seeds\.Closure places the constrained scalar S6 arm at00and the analytic tanh recurrence at11, so the arm faithful to the architecture meets the criterion of half closure only atq∈\{2−5,2−6,2−7,2−8\}q\\in\\\{2^\{\-5\},2^\{\-6\},2^\{\-7\},2^\{\-8\}\\\}, the four empirically passing points\. The asymptotic theorem supplies no finite\-qqcutoff\.The learned arm passes at the four smaller testedqqvalues, but this empirical split does not identify the theorem’s asymptotic onset\. Atq=2−3q=2^\{\-3\}the reference gapKLS6−KLtanh\\mathrm\{KL\}\_\{\\mathrm\{S6\}\}\-\\mathrm\{KL\}\_\{\\tanh\}is only1\.5×10−31\.5\\times 10^\{\-3\}, so closure is scale sensitive and seed dispersion is widest at the two largestqq\. Even so, the two points below half closure lie5\.45\.4and2\.82\.8standard errors below the threshold, whereas the four passing points lie at least2626standard errors above it using the unrounded results across seeds\. The outcome of four out of six is therefore not explained by marginal uncertainty\. Under distillation, the closure computed from the ratio of mean losses ranges from1\.161\.16to3\.973\.97and every nonreference comparison arm passes at all sixqqvalues\. Table[2](https://arxiv.org/html/2609.11132#S7.T2)instead reports the mean of closure ratios for each seed\.

##### Generalization at long horizons separates the learned architectures\.

The secondary endpoint on long paths evaluates the same trained models on sequences of length⌈8/q⌉\\lceil 8/q\\rceil, up to20482048steps, against a training horizon matched to the theorem that is at most eight\. Here the ranking inverts\. The analytic tanh recurrence is horizon stable, its predictive KL staying between1\.7×10−41\.7\\times 10^\{\-4\}and6\.4×10−36\.4\\times 10^\{\-3\}at the two tested switch probabilities, while the scalar S6 arm degrades to1\.9×10−11\.9\\times 10^\{\-1\}\. The GRU inherits that stability, passing four of sixqqwhen trained end to end and all six under distillation\. The official Mamba blocks do not: when trained end to end, two stacked blocks pass only two of six, and a single block fails at everyqq, with a worst per\-qqmean closure at long horizons of−32\.65\-32\.65atq=2−3q=2^\{\-3\}when trained end to end\. Under distillation, both Mamba depths pass zero of six at this long endpoint\. We therefore do not claim that a canonical selective SSM acquires the mechanism\. Matching the exact filter at the theorem’s horizon does not ensure stability at longer horizons\.

##### Scalar controls test a simpler explanation\.

A naive recurrence could also achieve low loss at the horizon matched to the theorem, making the exhibited tanh geometry unnecessary\. We therefore measured three controls against exact Bayes with the same generator \(μ=σ=1\\mu=\\sigma=1, 200,000 paths per point atH⁡\(q\)H\(q\), 20,000 at⌈8/q⌉\\lceil 8/q\\rceil, split across ten fixed seeds\)\. They are a plain affine recurrenceh←\(1−2​q\)​h\+ℓ⁡\(x\)h\\\!\\leftarrow\\\!\(1\-2q\)h\+\\ell\(x\)with no saturation, the class considered by the pointwise lower bound; a hard clip of that map at±L⁡\(q\)\\pm L\(q\); and the identity accumulator\. The third panel of Figure[2](https://arxiv.org/html/2609.11132#S5.F2)shows the endpoint pattern; Appendix[F](https://arxiv.org/html/2609.11132#A6), Table[7](https://arxiv.org/html/2609.11132#A6.T7), reports the exact cells\.

The controls support three conclusions\. First, both saturating recurrences are horizon stable at the two tested switch probabilities, while the affine and identity controls degrade at the long endpoint\. Their relative ordering is not uniform: the hard clip has lower predictive KL atq=2−8q=2^\{\-8\}, whereas the tanh clamp has lower predictive KL atq=2−3q=2^\{\-3\}\. These two points do not identify a crossover threshold\. The theorem should therefore be read as proving one saturating instance, not the optimal one\. Second, the endpoint matched to the theorem does not establish a separation among scalar recurrence classes; the binary empirical separation among these scalar controls occurs at the long endpoint and is measured, not proved\. At that endpoint the affine map reaches5\.535\.53nats atq=2−8q=2^\{\-8\}, while both saturating controls remain stable\. Third, the closure scale of Table[2](https://arxiv.org/html/2609.11132#S7.T2)anchors11to one saturating witness among several; the readings from the learned arm remain meaningful as calibrated distances to an analytic reference with a guarantee at the stated logarithmic horizon\. Its stability at longer horizons is measured here, not established by that theorem\.

Hyperparameters were selected by mean validation loss over three tuning seeds before confirmation, with ties broken deterministically and incomplete settings excluded\. Tuning and confirmation seeds are disjoint, all final evaluation seeds are included, and the supplement gives the procedure for model selection and implementation details\.

## 8Limitations

Both theorems fix finiteKKbefore takingq→0\+q\\to 0^\{\+\}\. Their constants can deteriorate withKKand with nearly colliding means; no rate uniform inK=K⁡\(q\)K=K\(q\)is proved\. The quantitative witness moves withLK​\(q\)L\_\{K\}\(q\), so it gives neither a gap on every bounded input nor a lower bound on the state dimension\. The stationary theorem further assumes pairwise distinct scalar Gaussian means, symmetric switching, stationary initialization, and the stated logarithmic horizon\.

The sweep with equally spaced Gaussians is an illustration at finite scale, not evidence for a uniform onset rate\. The comparison concerns two explicit maps, not every SSM, and does not show that training learns the radial rule\. Binary controls are diagnostics in the finite regime: hard clipping can beat tanh, and the training experiments do not establish transfer to learned models\. The result is a counterexample, not a universal criterion for pruning, quantization, distillation, low rank adaptation, or other compression\. The converse question, when an internal mismatch must incur nonvanishing task loss, remains open\.

## 9Conclusion

For every fixed finite latent state space, exact Bayesian mixing and our radial filter can diverge linearly in centered logits at an explicit witness while their decoded categorical KL vanishes\. Predictive agreement also holds in stationary expectation in the symmetric scalar Gaussian HMM, so it is neither explained by uniform internal approximation nor confined to one input chosen by hand\. A controlled sweep with equally spaced Gaussians overK∈\{2,4,8\}K\\in\\\{2,4,8\\\}illustrates both trends at finite scale, while binary controls compare behavior at short and long horizons\. At the common input witness, softmax suppresses a growing map discrepancy; on typical HMM paths, a shared confidence cone suppresses predictive error\. The result therefore identifies two missing links from internal separation to predictive loss: its contribution weighted by loss under the evaluation path law, and the decoder’s sensitivity there\. It does not show that this failure of gap transfer is prevalent in learned models or provide a converse criterion; uniform growing\-KKrates and extensions to learned sequence models remain open\.

### Reproducibility statement

The assumptions and complete paper proofs for the stated results are given in the appendices\. Appendix[F](https://arxiv.org/html/2609.11132#A6)specifies the empirical estimands, aggregation, model configurations, hyperparameters, seeds, and clipping conventions\. The anonymous supplementary package contains the simulation code and numerical results used for the reported experiments\.

## References

- \[1\]Eric Alsmann, Lowejatan Noori, and Martin Lange\.On the expressiveness of state space models via temporal logics\.In*International Conference on Learning Representations*, 2026\.
- \[2\]Simran Arora, Sabri Eyuboglu, Michael Zhang, Aman Timalsina, Silas Alberti, Dylan Zinsley, James Zou, Atri Rudra, and Christopher Ré\.Simple linear attention language models balance the recall\-throughput tradeoff\.*arXiv preprint arXiv:2402\.18668*, 2024\.
- \[3\]Maximilian Beck, Korbinian Pöppel, Markus Spanring, Andreas Auer, Oleksandra Prudnikova, Michael Kopp, Günter Klambauer, Johannes Brandstetter, and Sepp Hochreiter\.xLSTM: Extended long short\-term memory\.In*Advances in Neural Information Processing Systems*, 2024\.arXiv:2405\.04517\.
- \[4\]Philipp Becker, Niklas Freymuth, and Gerhard Neumann\.KalMamba: Towards efficient probabilistic state space models for RL under uncertainty\.*arXiv preprint arXiv:2406\.15131*, 2024\.
- \[5\]Adrian N\. Bishop and Edwin V\. Bonilla\.Recurrent neural networks and universal approximation of bayesian filters\.In*Proceedings of the 26th International Conference on Artificial Intelligence and Statistics*, volume 206 of*Proceedings of Machine Learning Research*, pp\. 6956–6967\. PMLR, 2023\.URL[https://proceedings\.mlr\.press/v206/bishop23a\.html](https://proceedings.mlr.press/v206/bishop23a.html)\.
- \[6\]David Blackwell\.The entropy of functions of finite\-state markov chains\.In*Transactions of the First Prague Conference on Information Theory, Statistical Decision Functions, Random Processes*, pp\. 13–20\. Publishing House of the Czechoslovak Academy of Sciences, 1957\.
- \[7\]Marco Bondaschi, Nived Rajaraman, Xiuying Wei, Kannan Ramchandran, Razvan Pascanu, Caglar Gulcehre, Michael Gastpar, and Ashok Vardhan Makkuva\.From markov to laplace: How mamba in\-context learns markov chains, 2025\.arXiv:2502\.10178\.
- \[8\]Loïc Cabannes, Pierre\-Emmanuel Mazaré, Gergely Szilvasy, Matthijs Douze, Maria Lomeli, Ilze Amanda Auzina, Justin Carpentier, Gabriel Synnaeve, and Hervé Jégou\.Sparse delta memory: Scaling the state of linear RNNs through sparsity\.*arXiv preprint arXiv:2607\.07386*, 2026\.
- \[9\]Pavel Chigansky, Robert Liptser, and Ramon van Handel\.Intrinsic methods in filter stability\.In*The Oxford Handbook of Nonlinear Filtering*, pp\. 319–351\. Oxford University Press, 2011\.
- \[10\]Edo Cohen\-Karlik, Itamar Zimerman, Liane Galanti, Ido Andrew Atad, Amir Globerson, and Lior Wolf\.On the expressivity of selective state\-space layers: A multivariate polynomial approach\.In*Proceedings of the 29th International Conference on Artificial Intelligence and Statistics*, volume 300 of*Proceedings of Machine Learning Research*, pp\. 136–144\. PMLR, 2026\.URL[https://proceedings\.mlr\.press/v300/cohen\-karlik26a\.html](https://proceedings.mlr.press/v300/cohen-karlik26a.html)\.
- \[11\]Adrien Corenflos, James Thornton, George Deligiannidis, and Arnaud Doucet\.Differentiable particle filtering via entropy\-regularized optimal transport\.In*International Conference on Machine Learning*, 2021\.arXiv:2102\.07850\.
- \[12\]Tri Dao and Albert Gu\.Transformers are SSMs: Generalized models and efficient algorithms through structured state space duality\.In*International Conference on Machine Learning*, 2024\.arXiv:2405\.21060\.
- \[13\]Grégoire Delétang, Anian Ruoss, Jordi Grau\-Moya, Tim Genewein, Li Kevin Wenliang, Elliot Catt, Chris Cundy, Marcus Hutter, Shane Legg, Joel Veness, and Pedro A\. Ortega\.Neural networks and the chomsky hierarchy\.In*International Conference on Learning Representations*, 2023\.arXiv:2207\.02098\.
- \[14\]Yunus Emre Demirci, Ali Devran Kara, and Serdar Yüksel\.Sensitivity of filter kernels and robustness bounds to transition and measurement kernel perturbations in partially observable stochastic control\.*arXiv preprint arXiv:2508\.10658*, 2025\.
- \[15\]Yann Dubois, Douwe Kiela, David J\. Schwab, and Ramakrishna Vedantam\.Learning optimal representations with the decodable information bottleneck\.In*Advances in Neural Information Processing Systems*, 2020\.arXiv:2009\.12789\.
- \[16\]Yann Dubois, Benjamin Bloem\-Reddy, Karen Ullrich, and Chris J\. Maddison\.Lossy compression for lossless prediction\.In*Advances in Neural Information Processing Systems*, 2021\.arXiv:2106\.10800\.
- \[17\]Gerardo Duran\-Martin\.A predictive view on streaming hidden markov models, 2026\.arXiv:2604\.09208\.
- \[18\]Meir Feder and Yury Polyanskiy\.Sequential prediction under log\-loss and misspecification\.*arXiv preprint arXiv:2102\.00050*, 2021\.
- \[19\]Daniel Y\. Fu, Tri Dao, Khaled K\. Saab, Armin W\. Thomas, Atri Rudra, and Christopher Ré\.Hungry hungry hippos: Towards language modeling with state space models\.In*International Conference on Learning Representations*, 2023\.arXiv:2212\.14052\.
- \[20\]Aydin Ghojogh, M\. Hadi Sepanj, and Benyamin Ghojogh\.On the relation of state space models and hidden markov models\.*arXiv preprint arXiv:2601\.13357*, 2026\.
- \[21\]Tingnan Gong, Junghwan Lee, Xiuyuan Cheng, and Yao Xie\.Neural network\-based CUSUM for online change\-point detection\.*arXiv preprint arXiv:2210\.17312*, 2022\.
- \[22\]Riccardo Grazzi, Julien Siems, Arber Zela, Jörg K\. H\. Franke, Frank Hutter, and Massimiliano Pontil\.Unlocking state\-tracking in linear RNNs through negative eigenvalues\.*arXiv preprint arXiv:2411\.12537*, 2024\.
- \[23\]Albert Gu and Tri Dao\.Mamba: Linear\-time sequence modeling with selective state spaces\.In*First Conference on Language Modeling*, 2024\.arXiv:2312\.00752\.
- \[24\]Albert Gu, Tri Dao, Stefano Ermon, Atri Rudra, and Christopher Ré\.HiPPO: Recurrent memory with optimal polynomial projections\.In*Advances in Neural Information Processing Systems*, 2020\.arXiv:2008\.07669\.
- \[25\]Albert Gu, Karan Goel, and Christopher Ré\.Efficiently modeling long sequences with structured state spaces\.In*International Conference on Learning Representations*, 2022\.arXiv:2111\.00396\.
- \[26\]Hassan Hafez\-Kolahi, Behrad Moniri, Shohreh Kasaei, and Mahdieh Soleymani Baghshah\.Rate\-distortion analysis of minimum excess risk in bayesian learning\.In*International Conference on Machine Learning*, 2021\.arXiv:2105\.04180\.
- \[27\]Michael Hahn and Mark Rofin\.Why are sensitive functions hard for transformers?In*Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics*, 2024\.arXiv:2402\.09963\.
- \[28\]Ramin Hasani, Mathias Lechner, Tsun\-Hsuan Wang, Makram Chahine, Alexander Amini, and Daniela Rus\.Liquid structural state\-space models\.In*International Conference on Learning Representations*, 2023\.arXiv:2209\.12951\.
- \[29\]Vishesh Jain, Frederic Koehler, Jingbo Liu, and Elchanan Mossel\.Accuracy\-memory tradeoffs and phase transitions in belief propagation\.*arXiv preprint arXiv:1905\.10031*, 2019\.
- \[30\]Sham M\. Kakade, Akshay Krishnamurthy, Gaurav Mahajan, and Cyril Zhang\.Learning hidden markov models using conditional samples\.*arXiv preprint arXiv:2302\.14753*, 2023\.
- \[31\]Ali Devran Kara and Serdar Yüksel\.Robustness to incorrect system models in stochastic control\.*SIAM Journal on Control and Optimization*, 2020\.arXiv:1803\.06046\.
- \[32\]Ali Devran Kara and Serdar Yüksel\.Near optimality of finite memory feedback policies in partially observed markov decision processes\.*Journal of Machine Learning Research*, 23\(11\):1–46, 2022\.
- \[33\]Ali Devran Kara, Maxim Raginsky, and Serdar Yüksel\.Robustness to incorrect models and data\-driven learning in average\-cost optimal stochastic control\.*Automatica*, 2022\.arXiv:2003\.05769\.
- \[34\]Arjun Karuvally, Franz Nowak, T\. Anderson Keller, Carmen Amo Alonso, Terrence J\. Sejnowski, and Hava T\. Siegelmann\.Bridging expressivity and scalability with adaptive unitary SSMs\.*arXiv preprint arXiv:2507\.05238*, 2025\.
- \[35\]Niklas Kemper, Tom Wollschläger, and Stephan Günnemann\.What expressivity theory misses: Message passing complexity for GNNs\.In*Advances in Neural Information Processing Systems*, 2025\.arXiv:2509\.01254\.
- \[36\]Aakash Lahoti, Kevin Y\. Li, Berlin Chen, Caitlin Wang, Aviv Bick, J\. Zico Kolter, Tri Dao, and Albert Gu\.Mamba\-3: Improved sequence modeling using state space principles\.In*International Conference on Learning Representations*, 2026\.arXiv:2603\.15569\.
- \[37\]François Le Gland and Nadia Oudjane\.Stability and uniform approximation of nonlinear filters using the hilbert metric and application to particle filters\.*The Annals of Applied Probability*, 14\(1\), 2004\.doi:10\.1214/aoap/1075828050\.
- \[38\]Jiaoda Li and Ryan Cotterell\.Characterizing the expressivity of fixed\-precision transformer language models\.*arXiv preprint arXiv:2505\.23623*, 2025\.
- \[39\]Marten Lienen, Abdullah Saydemir, and Stephan Günnemann\.UnHiPPO: Uncertainty\-aware initialization for state space models\.*arXiv preprint arXiv:2506\.05065*, 2025\.
- \[40\]G\. Lorden\.Procedures for reacting to a change in distribution\.*The Annals of Mathematical Statistics*, 42\(6\), 1971\.doi:10\.1214/aoms/1177693055\.
- \[41\]Curtis McDonald and Serdar Yüksel\.Exponential filter stability via dobrushin’s coefficient\.*Electronic Communications in Probability*, 2020\.arXiv:1910\.08463\.
- \[42\]Arian Mehrfard, Bharanidhar Duraisamy, Stefan Haag, Florian Geiss, and Mirko Mählisch\.Adaptive learned state estimation based on KalmanNet\.*arXiv preprint arXiv:2604\.02441*, 2026\.
- \[43\]William Merrill, Jackson Petty, and Ashish Sabharwal\.The illusion of state in state\-space models\.In*International Conference on Machine Learning*, 2024\.
- \[44\]George V\. Moustakides\.Optimal stopping times for detecting changes in distributions\.*The Annals of Statistics*, 14\(4\), 1986\.doi:10\.1214/aos/1176350164\.
- \[45\]Nicola Muca Cirone, Antonio Orvieto, Benjamin Walker, Cristopher Salvi, and Terry Lyons\.Theoretical foundations of deep selective state\-space models\.In*Advances in Neural Information Processing Systems*, 2024\.
- \[46\]Daniel Ocone and Etienne Pardoux\.Asymptotic stability of the optimal filter with respect to its initial condition\.*SIAM Journal on Control and Optimization*, 34\(1\):226–243, 1996\.doi:10\.1137/S0363012993256617\.
- \[47\]Antonio Orvieto, Samuel L\. Smith, Albert Gu, Anushan Fernando, Caglar Gulcehre, Razvan Pascanu, and Soham De\.Resurrecting recurrent neural networks for long sequences\.In*International Conference on Machine Learning*, 2023\.arXiv:2303\.06349\.
- \[48\]E\. S\. Page\.Continuous inspection schemes\.*Biometrika*, 41\(1\-2\):100–115, 1954\.doi:10\.1093/biomet/41\.1\-2\.100\.
- \[49\]Bo Peng, Eric Alcaide, Quentin Anthony, Alon Albalak, Samuel Arcadinho, Stella Biderman, Huanqi Cao, Xin Cheng, Michael Chung, Xingjian Du, Matteo Grella, Kranthi Kiran GV, Xuzheng He, Haowen Hou, Jiaju Lin, Przemysław Kazienko, Jan Kocoń, Jiaming Kong, Bartłomiej Koptyra, Hayden Lau, Krishna Sri Ipsit Mantri, Ferdinand Mom, Atsushi Saito, Guangyu Song, Xiangru Tang, Bolun Wang, Johan S\. Wind, Stanisław Woźniak, Ruichong Zhang, Zhenyuan Zhang, Qihang Zhao, Peng Zhou, Qinghua Zhou, Jian Zhu, and Rui\-Jie Zhu\.RWKV: Reinventing RNNs for the transformer era\.*arXiv preprint arXiv:2305\.13048*, 2023\.
- \[50\]Liangzu Peng, Aditya Chattopadhyay, Luca Zancato, Elvis Nunez, Wei Xia, and Stefano Soatto\.Gated KalmaNet: A fading memory layer through test\-time ridge regression\.*arXiv preprint arXiv:2511\.21016*, 2025\.
- \[51\]Michael Poli, Stefano Massaroli, Eric Nguyen, Daniel Y\. Fu, Tri Dao, Stephen Baccus, Yoshua Bengio, Stefano Ermon, and Christopher Ré\.Hyena hierarchy: Towards larger convolutional language models\.In*International Conference on Machine Learning*, 2023\.arXiv:2302\.10866\.
- \[52\]Moshe Pollak\.Optimal detection of a change in distribution\.*The Annals of Statistics*, 13\(1\), 1985\.doi:10\.1214/aos/1176346587\.
- \[53\]Lawrence R\. Rabiner\.A tutorial on hidden markov models and selected applications in speech recognition\.*Proceedings of the IEEE*, 77\(2\):257–286, 1989\.
- \[54\]Yash Sarrof, Yana Veitsman, and Michael Hahn\.The expressive capacity of state space models: A formal language perspective\.In*Advances in Neural Information Processing Systems*, 2024\.
- \[55\]Vaisakh Shaj, Cameron Barker, Aidan Scannell, Andras Szecsenyi, Elliot J\. Crowley, and Amos Storkey\.Kalman linear attention: Parallel bayesian filtering for efficient language modelling and state tracking\.In*International Conference on Machine Learning*, 2026\.
- \[56\]Mehran Shakerinava, Behnoush Khavari, Siamak Ravanbakhsh, and Sarath Chandar\.The expressive limits of diagonal ssms for state\-tracking\.In*International Conference on Learning Representations*, 2026\.
- \[57\]Vatsal Sharan, Sham Kakade, Percy Liang, and Gregory Valiant\.Prediction with a short memory, 2018\.arXiv:1612\.02526\.
- \[58\]Julien Siems, Timur Carstensen, Arber Zela, Frank Hutter, Massimiliano Pontil, and Riccardo Grazzi\.DeltaProduct: Improving state\-tracking in linear RNNs via householder products\.*arXiv preprint arXiv:2502\.10297*, 2025\.
- \[59\]Julien Siems, Riccardo Grazzi, Korbinian Pöppel, Kirill Kalinin, Hitesh Ballani, and Babak Rahmani\.Learning state\-tracking from code using linear rnns\.In*International Conference on Learning Representations*, 2026\.
- \[60\]Jimmy T\.H\. Smith, Andrew Warrington, and Scott W\. Linderman\.Simplified state space layers for sequence modeling\.In*International Conference on Learning Representations*, 2023\.arXiv:2208\.04933\.
- \[61\]Christopher Solinas, Radovan Haluška, David Sychrovský, Finbarr Timbers, Nolan Bard, Michael Buro, Martin Schmid, Nathan R\. Sturtevant, and Michael Bowling\.Neural bayesian filtering, 2025\.arXiv:2510\.03614\.
- \[62\]Daniel Soudry, Elad Hoffer, Mor Shpigel Nacson, and Nathan Srebro\.The implicit bias of gradient descent on separable data\.In*International Conference on Learning Representations*, 2018\.URL[https://openreview\.net/forum?id=r1q7n9gAb](https://openreview.net/forum?id=r1q7n9gAb)\.
- \[63\]Alex Tang, M\. Emrullah Ildiz, Batin Kurt, Samet Oymak, and Necmiye Ozay\.On the generalization properties of selective state\-space models for filtering tasks for unknown systems\.*arXiv preprint arXiv:2604\.23818*, 2026\.
- \[64\]Aleksandar Terzić, Nicolas Menet, Michael Hersche, Thomas Hofmann, and Abbas Rahimi\.Structured sparse transition matrices to enable state tracking in state\-space models\.In*Advances in Neural Information Processing Systems*, 2025\.arXiv:2509\.22284\.
- \[65\]Benjamin Walker and Terry Lyons\.Chess\-world\-model: A 10M\-game benchmark for exact state tracking from chess move sequences\.*arXiv preprint arXiv:2605\.30100*, 2026\.
- \[66\]Geoffrey Wolfer and Aryeh Kontorovich\.Estimating the mixing time of ergodic markov chains\.*arXiv preprint arXiv:1902\.01224*, 2019\.
- \[67\]Songlin Yang, Bailin Wang, Yikang Shen, Rameswar Panda, and Yoon Kim\.Gated linear attention transformers with hardware\-efficient training\.In*International Conference on Machine Learning*, 2024a\.arXiv:2312\.06635\.
- \[68\]Songlin Yang, Bailin Wang, Yu Zhang, Yikang Shen, and Yoon Kim\.Parallelizing linear transformers with the delta rule over sequence length\.In*Advances in Neural Information Processing Systems*, 2024b\.arXiv:2406\.06484\.
- \[69\]Di Zhang and Jiaqi Xing\.Bayesian optimality of in\-context learning with selective state spaces, 2026\.arXiv:2602\.17744\.

## Appendix AOpen Affine Control Lower Bound

This appendix concerns a different question from Theorem[4](https://arxiv.org/html/2609.11132#Thmtheorem4): whether the nonsaturating binary affine control has a nonvanishing lower bound at the much longer endpoint⌈8/q⌉\\lceil 8/q\\rceil\. The section on learned models establishes that separation only by measurement; the final analytic step for the joint event below remains open\. The mechanism is*sluggishness*: after a latent switch, the nonsaturating affine recurrence carries an unclamped state near its AR\(1\) fixed pointd/\(2​q\)d/\(2q\), withd=2​μ2/σ2d=2\\mu^\{2\}/\\sigma^\{2\}, and needsΘ⁡\(1/q\)\\Theta\(1/q\)steps to change sign, while exact Bayes recovers on a shorter timescale for building confidence\. Simulation localizes the cost accordingly: the fraction on the wrong side relative to the truth is between0\.180\.18and0\.200\.20across everyqqtested\. The subset on which Exact Bayes is confident is reported separately below\. Disagreement with the exact filter was not measured\. Table[3](https://arxiv.org/html/2609.11132#A1.T3)separates the proved ingredients from the remaining step for the terminal joint event\. For the concentration calculation, the centered oriented evidence has variance proxyv=2​dv=2d, and at horizonttwe choose the deterministic prefix budgetBt=t​dB\_\{t\}=td\.

Table 3:The lower bound per step, step by step\.Every deterministic and probabilistic ingredient is proved; the banded final row is the one step still open, and the separation at long horizons is not claimed without it\.StepStatementStatus*Deterministic core*Cost of a step on the wrong sidea confident exact predictive against a model on the wrong side costs≥3/80\\geq 3/80natsprovedAffine stateht=at​h0\+∑i<tat−1−i​eih\_\{t\}=a^\{t\}h\_\{0\}\+\\sum\_\{i<t\}a^\{t\-1\-i\}e\_\{i\}provedSummation by parts\|Sj\|≤Bt\\lvert S\_\{j\}\\rvert\\leq B\_\{t\}gives weighted sum≥−2​Bt\\geq\-2B\_\{t\}provedCombining the boundsno crossing by steptt, and that step costs≥3/80\\geq 3/80conditional on the margin*Probabilistic tail*Gaussian sub\-Gaussianitya centered real Gaussian is sub\-Gaussian with constant its varianceproved herePrefix tail on a blockon a block with fixed latent signs, some prefix exceedsBtB\_\{t\}with measure at most2\(t\+1\)e−Bt2/\(2tv\)2\(t\{\+\}1\)e^\{\-B\_\{t\}^\{2\}/\(2tv\)\}proved, discharges the tailTerminal joint eventuniform probability that the terminal step is on the wrong side while exact Bayes is confidentopenWithv=2​dv=2dandBt=t​dB\_\{t\}=td, the charge from the prefix tail is2\(t\+1\)e−td/42\(t\{\+\}1\)e^\{\-td/4\}\. Instantiating the affine recurrence ata=1−2​qa=1\-2qandh0=d/\(2​q\)h\_\{0\}=d/\(2q\)reduces the condition for no crossing2​Bt<at​h02B\_\{t\}<a^\{t\}h\_\{0\}to

4​q​t<\(1−2​q\)t\.4qt<\(1\-2q\)^\{t\}\.Writingy=q​ty=qtand takingq→0q\\to 0gives the boundary equation4​y=e−2​y4y=e^\{\-2y\}, whose unique positive root isy⋆=0\.175866​…y\_\{\\star\}=0\.175866\\ldots\. Hencet∗​q→y⋆t^\{\\ast\}q\\to y\_\{\\star\}, and stationarity gives limiting mass of the run age1−e−y⋆=0\.161270​…1\-e^\{\-y\_\{\\star\}\}=0\.161270\\ldots\. Both constants are analytic, independent ofqq, and carry no sampling error\. The latter is close to the measured fraction on the wrong side relative to the truth,0\.180\.18to0\.200\.20acrossq=2−5q=2^\{\-5\}to2−122^\{\-12\}under the convention after mixing of Table[2](https://arxiv.org/html/2609.11132#S7.T2); the bound from summation by parts loses a factor of two, so the analytic constant need not match the observed fraction\.

##### What the open step actually requires\.

The estimand at the long endpoint is the predictive KL at the*final step*, so no integration along a trajectory is needed: it suffices to lower bound, uniformly inqq, the probability that the terminal step is a step on the wrong side, since the cost per step above then gives𝔼⁡\[KLT\]≥\(3/80\)​ℙ​\(Wjoint\)\\mathbb\{E\}\[\\mathrm\{KL\}\_\{T\}\]\\geq\(3/80\)\\,\\mathbb\{P\}\(W\_\{\\mathrm\{joint\}\}\)\. Stationarity supplies that bridge exactly\. WritingATA\_\{T\}for the age of the current latent run, memorylessness givesℙ⁡\(AT≥k\)=\(1−q\)k\\mathbb\{P\}\(A\_\{T\}\\geq k\)=\(1\-q\)^\{k\}, henceℙ⁡\(AT≤t∗\)→1−e−0\.176\\mathbb\{P\}\(A\_\{T\}\\leq t^\{\\ast\}\)\\to 1\-e^\{\-0\.176\}, which is the0\.1610\.161already reported\. What remains open is therefore a uniform bound on the joint event that the run is young enough for the affine state not to have crossed and old enough for the exact filter to be confident, not an integration\.

##### Why a stepwise union bound is insufficient\.

Charging the worst case at every step forces the tolerance per step to grow likelog⁡\(1/q\)\\sqrt\{\\log\(1/q\)\}, shrinking the guaranteed horizon before crossing toΘ⁡\(1/\(q​log⁡\(1/q\)\)\)\\Theta\(1/\(q\\sqrt\{\\log\(1/q\)\}\)\)and making the resulting probability of the switch window tend to zero\. This decay conflicts with the measured nearly constant fraction on the wrong side\. Controlling the partial sum instead preserves a constant probability of the switch window\.

## Appendix BGeometry of Centered Logits and an Example with Three States

This section gives a geometric reading of Definition[1](https://arxiv.org/html/2609.11132#Thmtheorem1)\. It is not an additional assumption and is not used as a substitute for the proof\. Its purpose is to make clear what the state dimension, centering operation, and radial nonlinearity mean whenK\>2K\>2\.

### B\.1WhatKKcounts

The integerKKcounts latent modes, not observations, layers, or time steps\. A machine with normal, overheated, and failed modes hasK=3K=3even if it is monitored for a million steps\. At timett, a filter assigns a probability vector

pt=\(pt​\(0\),…,pt​\(K−1\)\),pt​\(s\)≥0,∑spt​\(s\)=1\.p\_\{t\}=\(p\_\{t\}\(0\),\\ldots,p\_\{t\}\(K\-1\)\),\\qquad p\_\{t\}\(s\)\\geq 0,\\quad\\sum\_\{s\}p\_\{t\}\(s\)=1\.The vector hasKKentries but onlyK−1K\-1degrees of freedom because its entries sum to one\. Logits make the same redundancy explicit: for every scalarbb,

softmax⁡\(z\+b​𝟏\)=softmax⁡\(z\)\.\\operatorname\{softmax\}\(z\+b\\mathbf\{1\}\)=\\operatorname\{softmax\}\(z\)\.Thuszzandz\+b​𝟏z\+b\\mathbf\{1\}represent the same belief\. Centering chooses one representative from each equivalence class,

𝒞​z=z−𝟏⊤​zK​𝟏,𝟏⊤​𝒞​z=0\.\\mathcal\{C\}z=z\-\\frac\{\\mathbf\{1\}^\{\\top\}z\}\{K\}\\mathbf\{1\},\\qquad\\mathbf\{1\}^\{\\top\}\\mathcal\{C\}z=0\.The meaningful filter state therefore lies in the\(K−1\)\(K\-1\)\-dimensional hyperplane orthogonal to𝟏\\mathbf\{1\}\. Pairwise log oddszi−zjz\_\{i\}\-z\_\{j\}are coordinates on this quotient space\. This is why every mismatch in Definition[2](https://arxiv.org/html/2609.11132#Thmtheorem2)is stated through pairwise differences rather than raw logits\.

It is useful to keep two scale parameters separate\. The state countKKfixes the dimension of the belief geometry\. The quantity

LK​\(q\)=log⁡\(K−1\)​\(1−q\)qL\_\{K\}\(q\)=\\log\\frac\{\(K\-1\)\(1\-q\)\}\{q\}is a confidence scale determined by the switch probability\. For fixedKK, rare switching meansq→0\+q\\to 0^\{\+\}and henceLK​\(q\)→∞L\_\{K\}\(q\)\\to\\infty\. The theorem takes this limit after fixingKK; it does not let the number of modes grow with rarity\.

### B\.2Direction records preference; radius records confidence

Within the centered hyperplane, regardzzas an arrow\. Its direction records the pattern of relative preferences among states\. Its length records how strongly the filter prefers that pattern\. Our conventionr⁡\(z\)=2​‖𝒞​z‖2r\(z\)=\\sqrt\{2\}\\\|\\mathcal\{C\}z\\\|\_\{2\}makes this length agree with the absolute log odds in the binary specialization\.

For a concreteK=3K=3belief, take

z=\(2,0,−2\),softmax⁡\(z\)≈\(0\.867,0\.117,0\.016\)\.z=\(2,0,\-2\),\\qquad\\operatorname\{softmax\}\(z\)\\approx\(0\.867,0\.117,0\.016\)\.State00leads state11, which leads state22\. If a radial operation halves the arrow, it produces\(1,0,−1\)\(1,0,\-1\)and probabilities approximately\(0\.665,0\.245,0\.090\)\(0\.665,0\.245,0\.090\)\. The ranking and all ratios between pairwise gaps are unchanged, but the confidence is lower\. This example illustrates the geometry only; the actual radial factor is chosen byqqand by the current radius\.

More precisely, the radial transition can be written

Rq​\(z\)=βq​\(z\)​𝒞​z,βq​\(z\)=LK​\(q\)​tanh⁡\(αK​\(q\)​r​\(z\)/LK​\(q\)\)r⁡\(z\)\.R\_\{q\}\(z\)=\\beta\_\{q\}\(z\)\\mathcal\{C\}z,\\qquad\\beta\_\{q\}\(z\)=\\frac\{L\_\{K\}\(q\)\\tanh\(\\alpha\_\{K\}\(q\)r\(z\)/L\_\{K\}\(q\)\)\}\{r\(z\)\}\.It preserves the direction of every nonzero centered state during the transition mixing step and changes only its radius\. It is not an orthogonal projection: there is no fixed subspace onto which the state is dropped, and the multiplier depends nonlinearly on the current radius\. Near the origin,Rq​\(z\)≈αK​\(q\)​𝒞​zR\_\{q\}\(z\)\\approx\\alpha\_\{K\}\(q\)\\mathcal\{C\}z; far from the origin, its radius saturates belowLK​\(q\)L\_\{K\}\(q\)\. The next observation then adds the centered score𝒞​g​\(x\)\\mathcal\{C\}g\(x\), which can change both direction and length\. A trajectory can therefore turn many times even though each isolated radial mixing step preserves direction\.

### B\.3What Exact Bayes does differently

Exact transition mixing acts on probabilities before returning to centered logits\. It adjusts the coordinates according to the full categorical mixturePq⊤​pP\_\{q\}^\{\\top\}p, not through one shared radial multiplier\. Except in special directions, it need not preserve the centered arrow’s direction\. The radial filter deliberately discards this geometry specific to each coordinate and retains only a common confidence contraction followed by the same observation score\.

The two main theorems test different consequences of that difference\. Theorem[3](https://arxiv.org/html/2609.11132#Thmtheorem3)selects an explicit moving logitzqz\_\{q\}and proves that the exact and radial maps separate by orderLK​\(q\)L\_\{K\}\(q\)in centered coordinates\. This rules out the explanation that the two update maps merely converge to one another\. Theorem[4](https://arxiv.org/html/2609.11132#Thmtheorem4)then couples both filters to the same stationary observation path and proves that their decoded predictive distributions nevertheless converge in expected KL at the logarithmic horizon\.

The apparent paradox is resolved by the softmax decoder\. A centered displacement is expensive near a decision boundary, where probabilities are sensitive to logits\. It can be cheap deep inside a shared confidence cone, where both filters put almost all mass on the same state\. In the example with three states, moving from\(10,0,−10\)\(10,0,\-10\)to\(6,0,−6\)\(6,0,\-6\)is a large internal change, yet both decoded vectors are overwhelmingly concentrated on state00\. The proof formalizes precisely this regime inside the common cone rather than asserting that every large logit difference is harmless\.

### B\.4A dictionary for one step

The following dictionary separates objects that are easy to conflate\.

Table 4:Geometric dictionary for the fixed\-KKresult\.Observation scores are added after the transition map and may rotate the state\.

## Appendix CProof Roadmap for Fixed K

This section is a reading guide to Appendix[E](https://arxiv.org/html/2609.11132#A5)\. The quantitative theorem at the level of the maps and the stationary theorem under the path law share an endpoint set by softmax curvature but answer different questions\. The first follows from an explicit centered witness and direct asymptotics\. The second has a probabilistic spine and two recovery arguments specific to the dimension\.

##### Dependency order\.

Theorem[3](https://arxiv.org/html/2609.11132#Thmtheorem3)first compares the exact and radial gaps for the chosen pair atzqz\_\{q\}, then invokes the curvature lemma for the common cone\. For Theorem[4](https://arxiv.org/html/2609.11132#Thmtheorem4)andK≥3K\\geq 3, Lemma[8](https://arxiv.org/html/2609.11132#Thmtheorem8)and the Gaussian maximal event feed the radial barriers and Lemma[13](https://arxiv.org/html/2609.11132#Thmtheorem13); separately, the calculation over path mixtures yields Lemma[16](https://arxiv.org/html/2609.11132#Thmtheorem16)\. These two margins meet only in Lemma[19](https://arxiv.org/html/2609.11132#Thmtheorem19), with the deterministic envelope controlling the complement\. The binary branch replaces the radial barrier step with common scalar recovery and then rejoins the same curvature and envelope argument\.

Figure 3:Binary transition geometry atq=1/8q=1/8\.A: signed residual of exact minus tanh\. B: the maps at true scale share saturation levels and midpoint slope\. C: values at0,log⁡2,2​log⁡20,\\log 2,2\\log 2certify that the map is not affine; their second differenceΔq\\Delta\_\{q\}forces every affine approximation to incur maximum error at leastΔq/4\\Delta\_\{q\}/4\. This is a fixed\-qq,K=2K=2diagnostic\.
### C\.1Proof outline

Both filters start in the same centered belief coordinates and consume the same observations\. The proof under the path law first conditions on a constant latent block\. It then shows that exact Bayes and the radial recurrence identify the true state with diverging margins\. A deterministic lemma on the categorical KL converts that common confidence cone into an exponentially small loss on the good event, while a2​LK​\(q\)2L\_\{K\}\(q\)envelope controls every remaining path\.

### C\.2TheK≥3K\\geq 3branch

On a block with no switch in statess, the centered Gaussian score isas\+w​Zta\_\{s\}\+wZ\_\{t\}\. Because radial mixing uses one adaptive scalar coefficient, the approximate state has the exact representationztR=Ut​as\+Vt​wz^\{R\}\_\{t\}=U\_\{t\}a\_\{s\}\+V\_\{t\}w\. Abel summation bounds everyVtV\_\{t\}by twice one maximum of a Gaussian partial sum, so a single maximal inequality controls the whole horizon without a temporal union bound\. Projection ontow⟂w^\{\\perp\}supplies a deterministic signal direction; upper and lower radial barriers then give margins for the true state of orderLK​\(q\)2/3L\_\{K\}\(q\)^\{2/3\}\. Recovery of the exact filter is established separately by comparing the constant path in the true state with constant wrong paths and the aggregate Bayes factor of all alternatives that contain a switch\.

### C\.3The binary branch

ForK=2K=2, the signal and noise directions are collinear, so the projection step is unavailable\. Midpoint translation reduces arbitrary distinct means to the symmetric model\. An event on the prefix evidence forces both scalar recurrences to hit a common margin, and a geometric sum over possible hit locations controls suffix persistence\. Figure[3](https://arxiv.org/html/2609.11132#A3.F3)visualizes the transition maps used by this branch\. The argument unions suffix lengths, not transition errors at one step, and its failure probability remains negligible after multiplication by the KL envelope\.

### C\.4Conditioning argument

There are three changes of probability law\. First, the event that no switch occurs costs at mostH⁡\(q\)​qH\(q\)q\. Second, conditional on that event, Gaussian concentration applies to independent emissions from the fixed latent state\. Third, the exact posterior still sums over every latent path; candidate paths that contain a switch are included and controlled through their aggregate Bayes factor\. The final expectation is recovered by averaging the finitely many conditional bounds over the stationary uniform starting state\.

## Appendix DProof Anatomy, Constants, and Quantifiers

This section explains the main steps in the proof of Theorem[4](https://arxiv.org/html/2609.11132#Thmtheorem4)and the dependence of its constants on the model parameters\. Full proofs are given in Appendix[E](https://arxiv.org/html/2609.11132#A5)\.

### D\.1Quantifier order

FixKK, the distinct meansμ0,…,μK−1\\mu\_\{0\},\\ldots,\\mu\_\{K\-1\}, the varianceσ2\\sigma^\{2\}, and a horizon constantc<dminc<d\_\{\\min\}, where

dmin=mini≠j⁡\(μi−μj\)22​σ2\>0\.d\_\{\\min\}=\\min\_\{i\\neq j\}\\frac\{\(\\mu\_\{i\}\-\\mu\_\{j\}\)^\{2\}\}\{2\\sigma^\{2\}\}\>0\.Only then sendq→0\+q\\to 0^\{\+\}\. Constants denotedCK,cs,κC\_\{K\},c\_\{s\},\\kappa, or a small\-qqthreshold may depend on these fixed parameters\. This permits a finite union over states and competitors, but it does not provide a bound uniform inKKor in a family whose means approach one another\. The theorem has the logical form

∀K<∞∀\(μ0,…,μK−1\)distinct,limq→0\+𝔼DKL\(pHE∥pHR\)=0,\\forall K<\\infty\\;\\forall\(\\mu\_\{0\},\\ldots,\\mu\_\{K\-1\}\)\\text\{ distinct\},\\qquad\\lim\_\{q\\to 0^\{\+\}\}\\mathbb\{E\}D\_\{\\rm KL\}\(p\_\{H\}^\{E\}\\\|p\_\{H\}^\{R\}\)=0,not a joint limit overKKandqq\.

### D\.2The probabilistic argument forK≥3K\\geq 3

Condition on a block with no switch whose true state isss\. The centered Gaussian score has the exact form with two vectors

𝒞​g​\(Xt\+1\)=as\+w​Zt\+1,\\mathcal\{C\}g\(X\_\{t\+1\}\)=a\_\{s\}\+wZ\_\{t\+1\},so the radial recurrence stays in the random plane spanned byasa\_\{s\}andww:

ztR=Ut​as\+Vt​w,Ut\+1=βt​Ut\+1,Vt\+1=βt​Vt\+Zt\+1\.z\_\{t\}^\{R\}=U\_\{t\}a\_\{s\}\+V\_\{t\}w,\\qquad U\_\{t\+1\}=\\beta\_\{t\}U\_\{t\}\+1,\\quad V\_\{t\+1\}=\\beta\_\{t\}V\_\{t\}\+Z\_\{t\+1\}\.This does not reduce the filter to two states\. It reduces the analysis of one conditioned block to one accumulated signal coefficientUtU\_\{t\}and one accumulated noise coefficientVtV\_\{t\}\.

The multiplierβt\\beta\_\{t\}depends on the past, so the noise is not a deterministic weighted Gaussian sum\. The key observation is order, not independence: for fixed terminaltt, the products of positive multipliers attached to later observations are nondecreasing\. Abel summation therefore gives the pathwise inequality

\|Vt\|≤2​maxu≤H​\|∑k=1uZk\|\|V\_\{t\}\|\\leq 2\\max\_\{u\\leq H\}\\left\|\\sum\_\{k=1\}^\{u\}Z\_\{k\}\\right\|simultaneously for alltt\. One maximal bound from an exponential martingale controls this maximum\. The proof does not apply a separate tail bound at every time and then union bound over the horizon\.

On the resulting event,\|Vt\|=O⁡\(L​log⁡L\)\|V\_\{t\}\|=O\(\\sqrt\{L\\log L\}\)\. The tanh expansion supplies an upper barrierUt=O⁡\(L2/3\)U\_\{t\}=O\(L^\{2/3\}\)and a matching lower bound after an initial transient of orderL2/3L^\{2/3\}\. The exponent2/32/3is the balance at which the unit signal increment competes with the cubic correction intanh⁡x=x−Θ⁡\(x3\)\\tanh x=x\-\\Theta\(x^\{3\}\)\. It is sufficient for recovery because

zt,sR−zt,jR=ds​j​Ut\+ηs​j​Vt,ds​j=\(μs−μj\)22​σ2\>0,ηs​j=μs−μjσ,z^\{R\}\_\{t,s\}\-z^\{R\}\_\{t,j\}=d\_\{sj\}U\_\{t\}\+\\eta\_\{sj\}V\_\{t\},\\qquad d\_\{sj\}=\\frac\{\(\\mu\_\{s\}\-\\mu\_\{j\}\)^\{2\}\}\{2\\sigma^\{2\}\}\>0,\\quad\\eta\_\{sj\}=\\frac\{\\mu\_\{s\}\-\\mu\_\{j\}\}\{\\sigma\},andL2/3L^\{2/3\}dominatesL​log⁡L\\sqrt\{L\\log L\}\.

Exact Bayes is handled separately\. The constant\-sslatent path is the likelihood reference\. Constant wrong paths lose exponentially inLL, while the aggregate Bayes factor of all switched paths has expectation of orderH​qHq\. This is a calculation over path mixtures; it does not assume independence between posterior coordinates\. BecauseH=Θ⁡\(L\)H=\\Theta\(L\)andq=Θ⁡\(e−L\)q=\\Theta\(e^\{\-L\}\)for fixedKK, mass of the switched paths vanishes\.

### D\.3Bounding predictive KL

The radial and exact arguments meet only after both filters have entered a common confidence cone\. On the good event, every margin of the true state over a wrong state is at leastc∗​L2/3c\_\{\*\}L^\{2/3\}\. Along the line segment between the two terminal logits, softmax curvature is then exponentially small\. Combining this curvature with the deterministic terminal range bound yields

DKL\(pHE∥pHR\)≤CKL2e−c∗​L2/3\.D\_\{\\rm KL\}\(p\_\{H\}^\{E\}\\\|p\_\{H\}^\{R\}\)\\leq C\_\{K\}L^\{2\}e^\{\-c\_\{\*\}L^\{2/3\}\}\.The bound on this event alone is not enough because KL is unbounded in arbitrary logits\. Lemma[18](https://arxiv.org/html/2609.11132#Thmtheorem18)supplies a pathwise2​L2Lenvelope, which is multiplied by the vanishing probability of the complement\. Combining the bound on this event with the uniform bound on its complement turns recovery with high probability into convergence of expected KL\.

### D\.4Why the binary branch is separate

ForK≥3K\\geq 3, distinct scalar means imply thatasa\_\{s\}cannot be collinear with the common noise directionww: otherwise a nonzero quadratic would agree with an affine function at at least three distinct points\. This makes the radial radius control available\. With two points, that polynomial contradiction disappears\. The binary proof instead translates arbitrary means to the symmetric pair\(−δ,\+δ\)\(\-\\delta,\+\\delta\)and controls both scalar recurrences through a common recovery event\. It then rejoins the same KL curvature and envelope lemmas\. The separate branch is an issue of proof geometry, not a restriction to symmetric models with two states\.

### D\.5Rates proved and rates not claimed

The witness at the level of the maps and stationary path result have different rates and should not be merged\. At the explicit witness, centered map separation is asymptoticallyc⋆​Lc\_\{\\star\}L, and decoder KL is bounded by a polynomial inLLtimese−L/5e^\{\-L/5\}\. Along typical blocks with no switch in the stationary proof, the radial margin for the true state is only required to grow asL2/3L^\{2/3\}, which still makes the decoder cost on the good event vanish\. Neither statement identifies an optimal exponent, a sharp finite\-qqonset, or constants uniform over state geometries\. The curves with equally spaced Gaussians in Appendix[F](https://arxiv.org/html/2609.11132#A6)are therefore illustrations rather than rate estimates\.

## Appendix EProofs of the Main Results

This appendix proves Theorems[3](https://arxiv.org/html/2609.11132#Thmtheorem3)and[4](https://arxiv.org/html/2609.11132#Thmtheorem4)for every fixed finiteK≥2K\\geq 2\. The quantitative theorem at the level of the maps uses an explicit centered witness and a direct asymptotic calculation\. The stationary theorem uses a representation in two scalars of the radial filter, one Gaussian maximal event, and an analysis over path mixtures of exact Bayes forK≥3K\\geq 3; the binary case uses the same KL bounds after midpoint translation and a recovery argument in one dimension\.

### E\.1Preliminaries

WriteT=−log⁡qT=\-\\log q,α=αK​\(q\)\\alpha=\\alpha\_\{K\}\(q\), andL=LK​\(q\)L=L\_\{K\}\(q\)\. For fixedKK,

L=T\+log⁡\(K−1\)\+log⁡\(1−q\)=T\+O⁡\(1\)\.L=T\+\\log\(K\-1\)\+\\log\(1\-q\)=T\+O\(1\)\.Consequently, for all sufficiently smallqq,L\>1L\>1,α≥1/2\\alpha\\geq 1/2, andH≤CH​LH\\leq C\_\{H\}L\. We will also use

q=K−1eL\+K−1≤\(K−1\)​e−L,H​q⟶0,\(1−q\)−\(H−1\)⟶1\.q=\\frac\{K\-1\}\{e^\{L\}\+K\-1\}\\leq\(K\-1\)e^\{\-L\},\\qquad Hq\\longrightarrow 0,\\qquad\(1\-q\)^\{\-\(H\-1\)\}\\longrightarrow 1\.
Fix a statessand let

𝒩s=\{S0=⋯=SH−1=s\}\.\\mathcal\{N\}\_\{s\}=\\\{S\_\{0\}=\\cdots=S\_\{H\-1\}=s\\\}\.Conditional on𝒩s\\mathcal\{N\}\_\{s\},Xt\+1=μs\+σ​Zt\+1X\_\{t\+1\}=\\mu\_\{s\}\+\\sigma Z\_\{t\+1\}fort=0,…,H−1t=0,\\ldots,H\-1, where theZtZ\_\{t\}are independent standard Gaussians\. With

as=𝒞⁡\(μs​𝝁σ2−𝝁⊙22​σ2\),w=𝒞⁡\(𝝁σ\),a\_\{s\}=\\mathcal\{C\}\\left\(\\frac\{\\mu\_\{s\}\\boldsymbol\{\\mu\}\}\{\\sigma^\{2\}\}\-\\frac\{\\boldsymbol\{\\mu\}^\{\\odot 2\}\}\{2\\sigma^\{2\}\}\\right\),\\qquad w=\\mathcal\{C\}\\left\(\\frac\{\\boldsymbol\{\\mu\}\}\{\\sigma\}\\right\),the centered score isg⁡\(Xt\+1\)=as\+w​Zt\+1g\(X\_\{t\+1\}\)=a\_\{s\}\+wZ\_\{t\+1\}\.

###### Lemma 7\(Radial dynamics in two scalars\)\.

On𝒩s\\mathcal\{N\}\_\{s\}, adapted scalar processesUt,VtU\_\{t\},V\_\{t\}satisfy

ztR=Ut​as\+Vt​w,U0=V0=0,z^\{R\}\_\{t\}=U\_\{t\}a\_\{s\}\+V\_\{t\}w,\\qquad U\_\{0\}=V\_\{0\}=0,Ut\+1=βt​Ut\+1,Vt\+1=βt​Vt\+Zt\+1,U\_\{t\+1\}=\\beta\_\{t\}U\_\{t\}\+1,\\qquad V\_\{t\+1\}=\\beta\_\{t\}V\_\{t\}\+Z\_\{t\+1\},where

βt=L​tanh⁡\(α​r​\(ztR\)/L\)r⁡\(ztR\)\\beta\_\{t\}=\\frac\{L\\tanh\(\\alpha r\(z^\{R\}\_\{t\}\)/L\)\}\{r\(z^\{R\}\_\{t\}\)\}with continuous valueα\\alphaat the origin\. Moreover,0<βt≤α<10<\\beta\_\{t\}\\leq\\alpha<1andUt≥0U\_\{t\}\\geq 0\.

*Proof\.*The radial transition multiplies every centered coordinate by the same scalarβt\\beta\_\{t\}\. Substitution of \(11\) proves \(12\) by induction\. Positivity is immediate, whiletanh⁡x≤x\\tanh x\\leq xgivesβt≤α\\beta\_\{t\}\\leq\\alpha\.□\\square

### E\.2One maximal event controls the adaptive noise

LetSu=∑k=1uZkS\_\{u\}=\\sum\_\{k=1\}^\{u\}Z\_\{k\}\.

###### Lemma 8\(Adaptive Abel bound\)\.

For every realized path and everyt≤Ht\\leq H,

\|Vt\|≤2​maxu≤H​\|Su\|\.\|V\_\{t\}\|\\leq 2\\max\_\{u\\leq H\}\|S\_\{u\}\|\.

*Proof\.*Expanding \(12\) gives

Vt=∑k=1tγk,t​Zk,γk,t=∏ℓ=kt−1βℓ,γt,t=1\.V\_\{t\}=\\sum\_\{k=1\}^\{t\}\\gamma\_\{k,t\}Z\_\{k\},\\qquad\\gamma\_\{k,t\}=\\prod\_\{\\ell=k\}^\{t\-1\}\\beta\_\{\\ell\},\\qquad\\gamma\_\{t,t\}=1\.Although the weights are adaptive, they obey0≤γ1,t≤⋯≤γt,t=10\\leq\\gamma\_\{1,t\}\\leq\\cdots\\leq\\gamma\_\{t,t\}=1pathwise\. Abel summation therefore gives

Vt=St−∑k=1t−1\(γk\+1,t−γk,t\)​Sk\.V\_\{t\}=S\_\{t\}\-\\sum\_\{k=1\}^\{t\-1\}\(\\gamma\_\{k\+1,t\}\-\\gamma\_\{k,t\}\)S\_\{k\}\.The coefficients in the second term are nonnegative and sum to at most one, proving \(13\)\. This pathwise bound also applies to adaptive weights\.□\\square

###### Lemma 9\(Gaussian maximal event\)\.

Set

xL=2​H​log⁡\(2​L3\),ℰL=\{maxu≤H\|Su\|≤xL\}\.x\_\{L\}=\\sqrt\{2H\\log\(2L^\{3\}\)\},\\qquad\\mathcal\{E\}\_\{L\}=\\left\\\{\\max\_\{u\\leq H\}\|S\_\{u\}\|\\leq x\_\{L\}\\right\\\}\.ThenPr⁡\(ℰLc\)≤L−3\\Pr\(\\mathcal\{E\}\_\{L\}^\{c\}\)\\leq L^\{\-3\}\. OnℰL\\mathcal\{E\}\_\{L\},

maxt≤H⁡\|Vt\|≤ML:=2​xL=O⁡\(L​log⁡L\)\.\\max\_\{t\\leq H\}\|V\_\{t\}\|\\leq M\_\{L\}:=2x\_\{L\}=O\(\\sqrt\{L\\log L\}\)\.

*Proof\.*For everyλ\>0\\lambda\>0,exp⁡\(λ​Su−u​λ2/2\)\\exp\(\\lambda S\_\{u\}\-u\\lambda^\{2\}/2\)is a nonnegative martingale\. Doob’s maximal inequality, optimized atλ=x/H\\lambda=x/H, yields

Pr\(maxu≤HSu\>x\)≤e−x2/\(2H\)\.\\Pr\\\!\\left\(\\max\_\{u\\leq H\}S\_\{u\}\>x\\right\)\\leq e^\{\-x^\{2\}/\(2H\)\}\.Apply the same argument to−Su\-S\_\{u\}and union only the two signs\. Substitutingx=xLx=x\_\{L\}and applying Lemma[8](https://arxiv.org/html/2609.11132#Thmtheorem8)proves the claim simultaneously over the full horizon\.□\\square

### E\.3Radial recovery whenK≥3K\\geq 3

AssumeK≥3K\\geq 3, and letPPbe Euclidean projection ontow⟂w^\{\\perp\}\.

###### Lemma 10\(Noncollinear signal\)\.

For every statess,P​as≠0Pa\_\{s\}\\neq 0\.

*Proof\.*Distinct means implyw≠0w\\neq 0\. IfP​as=0Pa\_\{s\}=0, centeredness would giveas=λ​wa\_\{s\}=\\lambda w\. Hence a constantbbwould satisfy

μs​μiσ2−μi22​σ2=λ​μiσ\+b\\frac\{\\mu\_\{s\}\\mu\_\{i\}\}\{\\sigma^\{2\}\}\-\\frac\{\\mu\_\{i\}^\{2\}\}\{2\\sigma^\{2\}\}=\\frac\{\\lambda\\mu\_\{i\}\}\{\\sigma\}\+bfor everyii\. After clearing denominators, a quadratic with nonzero quadratic coefficient would vanish at theK≥3K\\geq 3distinct valuesμi\\mu\_\{i\}, a contradiction\.□\\square

SetDs=2​‖P​as‖2\>0D\_\{s\}=\\sqrt\{2\}\\\|Pa\_\{s\}\\\|\_\{2\}\>0,As=r⁡\(as\)A\_\{s\}=r\(a\_\{s\}\), andW=r⁡\(w\)W=r\(w\)\. From Lemma[7](https://arxiv.org/html/2609.11132#Thmtheorem7),

Ds​Ut≤r⁡\(ztR\)≤As​Ut\+W​\|Vt\|\.D\_\{s\}U\_\{t\}\\leq r\(z^\{R\}\_\{t\}\)\\leq A\_\{s\}U\_\{t\}\+W\|V\_\{t\}\|\.
###### Lemma 11\(Upper radial barrier\)\.

For eachss, constantsCU,Cr,qs\>0C\_\{U\},C\_\{r\},q\_\{s\}\>0exist such that, onℰL\\mathcal\{E\}\_\{L\}and forq<qsq<q\_\{s\},

Ut≤CU​L2/3,r⁡\(ztR\)≤Cr​L2/3U\_\{t\}\\leq C\_\{U\}L^\{2/3\},\\qquad r\(z^\{R\}\_\{t\}\)\\leq C\_\{r\}L^\{2/3\}for allt≤Ht\\leq H\.

*Proof\.*The functionh⁡\(x\)=tanh⁡\(x\)/xh\(x\)=\\tanh\(x\)/x, continuously extended at zero, decreases on\[0,∞\)\[0,\\infty\)\. Using the lower bound in \(15\),

βt​Ut=α​h​\(α​r​\(ztR\)/L\)​Ut≤LDs​tanh⁡\(α​Ds​Ut/L\)\.\\beta\_\{t\}U\_\{t\}=\\alpha h\(\\alpha r\(z^\{R\}\_\{t\}\)/L\)U\_\{t\}\\leq\\frac\{L\}\{D\_\{s\}\}\\tanh\(\\alpha D\_\{s\}U\_\{t\}/L\)\.LetfL​\(u\)=1\+\(L/Ds\)​tanh⁡\(α​Ds​u/L\)f\_\{L\}\(u\)=1\+\(L/D\_\{s\}\)\\tanh\(\\alpha D\_\{s\}u/L\)\. ChooseCUC\_\{U\}so thatDs2​CU3\>64D\_\{s\}^\{2\}C\_\{U\}^\{3\}\>64, and putR=CU​L2/3R=C\_\{U\}L^\{2/3\}\. For largeLL,α​Ds​R/L≤1\\alpha D\_\{s\}R/L\\leq 1\. Sincetanh⁡x≤x−x3/8\\tanh x\\leq x\-x^\{3\}/8on\[0,1\]\[0,1\],

fL​\(R\)−R≤1−\(1−α\)​R−α3​Ds2​CU38≤1−Ds2​CU364<0\.f\_\{L\}\(R\)\-R\\leq 1\-\(1\-\\alpha\)R\-\\frac\{\\alpha^\{3\}D\_\{s\}^\{2\}C\_\{U\}^\{3\}\}\{8\}\\leq 1\-\\frac\{D\_\{s\}^\{2\}C\_\{U\}^\{3\}\}\{64\}<0\.The mapfLf\_\{L\}is increasing,U0≤RU\_\{0\}\\leq R, andUt\+1≤fL​\(Ut\)U\_\{t\+1\}\\leq f\_\{L\}\(U\_\{t\}\), so induction on the first crossing yieldsUt≤RU\_\{t\}\\leq R\. The upper bound in \(15\), together with \(14\) andML=o⁡\(L2/3\)M\_\{L\}=o\(L^\{2/3\}\), gives the radius bound\.□\\square

###### Lemma 12\(Lower memory after the initial transient\)\.

There are constantscU,C0,qs′\>0c\_\{U\},C\_\{0\},q\_\{s\}^\{\\prime\}\>0such that, onℰL\\mathcal\{E\}\_\{L\}and forq<qs′q<q\_\{s\}^\{\\prime\},

Ut≥cU​L2/3U\_\{t\}\\geq c\_\{U\}L^\{2/3\}wheneverC0​L2/3≤t≤HC\_\{0\}L^\{2/3\}\\leq t\\leq H\.

*Proof\.*The global inequalitytanh⁡x≥x−x3/3\\tanh x\\geq x\-x^\{3\}/3and Lemma[11](https://arxiv.org/html/2609.11132#Thmtheorem11)imply

βt≥α−α3​r​\(ztR\)23​L2≥1−εL,εL=K​qK−1\+Cr23​L2/3\.\\beta\_\{t\}\\geq\\alpha\-\\frac\{\\alpha^\{3\}r\(z^\{R\}\_\{t\}\)^\{2\}\}\{3L^\{2\}\}\\geq 1\-\\varepsilon\_\{L\},\\qquad\\varepsilon\_\{L\}=\\frac\{Kq\}\{K\-1\}\+\\frac\{C\_\{r\}^\{2\}\}\{3L^\{2/3\}\}\.For smallqq,0<εL<10<\\varepsilon\_\{L\}<1andb1L−2/3≤εL≤b2L−2/3b\_\{1\}L^\{\-2/3\}\\leq\\varepsilon\_\{L\}\\leq b\_\{2\}L^\{\-2/3\}for positive constantsb1,b2b\_\{1\},b\_\{2\}\. Iterating \(12\) gives

Ut≥1−\(1−εL\)tεL\.U\_\{t\}\\geq\\frac\{1\-\(1\-\\varepsilon\_\{L\}\)^\{t\}\}\{\\varepsilon\_\{L\}\}\.Fort≥⌈\(log⁡2\)/εL⌉t\\geq\\lceil\(\\log 2\)/\\varepsilon\_\{L\}\\rceil, the numerator is at least1/21/2, which proves \(17\)\. This transient isO⁡\(L2/3\)O\(L^\{2/3\}\)and is smaller thanH=Θ⁡\(L\)H=\\Theta\(L\)\.□\\square

###### Lemma 13\(Simultaneous radial recovery\)\.

For everyss, constantscs,qs′′\>0c\_\{s\},q\_\{s\}^\{\\prime\\prime\}\>0exist such that, onℰL\\mathcal\{E\}\_\{L\}and forq<qs′′q<q\_\{s\}^\{\\prime\\prime\},

zt,sR−zt,jR≥cs​L2/3z^\{R\}\_\{t,s\}\-z^\{R\}\_\{t,j\}\\geq c\_\{s\}L^\{2/3\}for everyj≠sj\\neq sandC0​L2/3≤t≤HC\_\{0\}L^\{2/3\}\\leq t\\leq H\.

*Proof\.*Direct calculation gives

\(as\)s−\(as\)j=ds​j:=\(μs−μj\)22​σ2\>0,ws−wj=ηs​j:=μs−μjσ\.\(a\_\{s\}\)\_\{s\}\-\(a\_\{s\}\)\_\{j\}=d\_\{sj\}:=\\frac\{\(\\mu\_\{s\}\-\\mu\_\{j\}\)^\{2\}\}\{2\\sigma^\{2\}\}\>0,\\qquad w\_\{s\}\-w\_\{j\}=\\eta\_\{sj\}:=\\frac\{\\mu\_\{s\}\-\\mu\_\{j\}\}\{\\sigma\}\.Thuszt,sR−zt,jR=ds​j​Ut\+ηs​j​Vtz^\{R\}\_\{t,s\}\-z^\{R\}\_\{t,j\}=d\_\{sj\}U\_\{t\}\+\\eta\_\{sj\}V\_\{t\}\. The first term is bounded below by a positive multiple ofL2/3L^\{2/3\}, uniformly over the finitely many competitors, whereas the second isO⁡\(L​log⁡L\)=o⁡\(L2/3\)O\(\\sqrt\{L\\log L\}\)=o\(L^\{2/3\}\)by \(14\)\.□\\square

### E\.4Recovery of the exact filter by path mixtures

Continue to condition on𝒩s\\mathcal\{N\}\_\{s\}\. The exact posterior afterHHobservations is the normalized sum of prior times likelihood over latent paths\. Use the constant path\(s,…,s\)\(s,\\ldots,s\)as reference; its prior mass is

π∗=1K​\(1−q\)H−1\.\\pi\_\{\*\}=\\frac\{1\}\{K\}\(1\-q\)^\{H\-1\}\.
###### Lemma 14\(Constant wrong paths\)\.

There areκ,qs\>0\\kappa,q\_\{s\}\>0such that, forq<qsq<q\_\{s\}, with conditional probability at least1−\(K−1\)​e−κ​L1\-\(K\-1\)e^\{\-\\kappa L\}, every constant pathj≠sj\\neq shas likelihood ratio relative to the reference at moste−L/2e^\{\-L/2\}\.

*Proof\.*Forj≠sj\\neq s, let

Yts,j=log⁡φμs,σ​\(Xt\)φμj,σ​\(Xt\)\.Y\_\{t\}^\{s,j\}=\\log\\frac\{\\varphi\_\{\\mu\_\{s\},\\sigma\}\(X\_\{t\}\)\}\{\\varphi\_\{\\mu\_\{j\},\\sigma\}\(X\_\{t\}\)\}\.Under𝒩s\\mathcal\{N\}\_\{s\}, these variables are independent Gaussians with meands​jd\_\{sj\}and variance2​ds​j2d\_\{sj\}\. SinceH​ds​j≥\(dmin/c\)​THd\_\{sj\}\\geq\(d\_\{\\min\}/c\)Twithdmin/c\>1d\_\{\\min\}/c\>1, whileL=T\+O⁡\(1\)L=T\+O\(1\), a bound on the Gaussian lower tail gives

Pr⁡\(∑t=1HYts,j<L/2\)≤e−κs​j​L\\Pr\\\!\\left\(\\sum\_\{t=1\}^\{H\}Y\_\{t\}^\{s,j\}<L/2\\right\)\\leq e^\{\-\\kappa\_\{sj\}L\}for smallqq\. A finite union overj≠sj\\neq sproves the claim\.□\\square

###### Lemma 15\(Bayes factor of a switched path\)\.

Let𝒫sw\\mathcal\{P\}\_\{\\rm sw\}be the latent paths containing at least one switch, letπ⁡\(p\)\\pi\(p\)denote their prior masses, and letLRp\\mathrm\{LR\}\_\{p\}be likelihood relative to the constant\-sspath\. For

Bsw=∑p∈𝒫swπ⁡\(p\)​LRpπ∗,B\_\{\\rm sw\}=\\frac\{\\sum\_\{p\\in\\mathcal\{P\}\_\{\\rm sw\}\}\\pi\(p\)\\mathrm\{LR\}\_\{p\}\}\{\\pi\_\{\*\}\},𝔼\[Bsw∣𝒩s\]≤K​H​q\(1−q\)H−1,Pr\(Bsw\>e−L/2∣𝒩s\)≤CLe−L/2\.\\mathbb\{E\}\[B\_\{\\rm sw\}\\mid\\mathcal\{N\}\_\{s\}\]\\leq\\frac\{KHq\}\{\(1\-q\)^\{H\-1\}\},\\qquad\\Pr\(B\_\{\\rm sw\}\>e^\{\-L/2\}\\mid\\mathcal\{N\}\_\{s\}\)\\leq CLe^\{\-L/2\}\.

*Proof\.*Under the reference observation law, every likelihood ratio has expectation one\. The numerator in the first expectation is therefore the prior mass of paths that contain a switch, at mostH​qHq\. Division by \(20\) gives the first bound\. Markov’s inequality at thresholde−L/2e^\{\-L/2\}, followed by \(10\), gives the second\.□\\square

###### Lemma 16\(Exact posterior margin\)\.

Conditional on𝒩s\\mathcal\{N\}\_\{s\}, outside an event of probability at most

\(K−1\)e−κ​L\+CLe−L/2,\(K\-1\)e^\{\-\\kappa L\}\+CLe^\{\-L/2\},the exact terminal logits satisfy

zH,sE−zH,jE≥L/2−log⁡Kz^\{E\}\_\{H,s\}\-z^\{E\}\_\{H,j\}\\geq L/2\-\\log Kfor everyj≠sj\\neq s\.

*Proof\.*On the intersection of the events in Lemmas[14](https://arxiv.org/html/2609.11132#Thmtheorem14)and[15](https://arxiv.org/html/2609.11132#Thmtheorem15), total nonreference weight divided by reference weight is at mostKe−L/2Ke^\{\-L/2\}\. HencepHE​\(s\)/pHE​\(j\)≥eL/2/Kp^\{E\}\_\{H\}\(s\)/p^\{E\}\_\{H\}\(j\)\\geq e^\{L/2\}/Kfor everyj≠sj\\neq s, and taking logarithms gives \(23\)\.□\\square

### E\.5Categorical KL bounds

For vectorsu,vu,v, putΔ=u−v\\Delta=u\-vandrange⁡\(Δ\)=maxi⁡Δi−mini⁡Δi\\operatorname\{range\}\(\\Delta\)=\\max\_\{i\}\\Delta\_\{i\}\-\\min\_\{i\}\\Delta\_\{i\}\.

###### Lemma 17\(Range envelope\)\.

For everyu,v∈ℝKu,v\\in\\mathbb\{R\}^\{K\},

DKL\(softmaxu∥softmaxv\)≤range\(u−v\)\.D\_\{\\rm KL\}\(\\operatorname\{softmax\}u\\\|\\operatorname\{softmax\}v\)\\leq\\operatorname\{range\}\(u\-v\)\.

*Proof\.*LetA\(z\)=log∑ieziA\(z\)=\\log\\sum\_\{i\}e^\{z\_\{i\}\}\. The incrementA⁡\(u\)−A⁡\(v\)A\(u\)\-A\(v\)lies betweenmini⁡Δi\\min\_\{i\}\\Delta\_\{i\}andmaxi⁡Δi\\max\_\{i\}\\Delta\_\{i\}\. Thus every log probability ratio is at mostrange⁡\(Δ\)\\operatorname\{range\}\(\\Delta\), and averaging undersoftmax⁡\(u\)\\operatorname\{softmax\}\(u\)proves \(24\)\.□\\square

###### Lemma 18\(Terminal envelope\)\.

For every observation path,

DKL\(softmaxzHE∥softmaxzHR\)≤2L\.D\_\{\\rm KL\}\(\\operatorname\{softmax\}z^\{E\}\_\{H\}\\\|\\operatorname\{softmax\}z^\{R\}\_\{H\}\)\\leq 2L\.

*Proof\.*LetmE=Φq​\(zH−1E\)m^\{E\}=\\Phi\_\{q\}\(z^\{E\}\_\{H\-1\}\)andmR=Rq​\(zH−1R\)m^\{R\}=R\_\{q\}\(z^\{R\}\_\{H\-1\}\)be the logits before the observation at the terminal step\. The final shared score cancels, sozHE−zHR=mE−mRz^\{E\}\_\{H\}\-z^\{R\}\_\{H\}=m^\{E\}\-m^\{R\}\. Exact mixing places every probability coordinate in\[q/\(K−1\),1−q\]\[q/\(K\-1\),1\-q\], hence every exact pairwise score before the observation difference has magnitude at mostLL\. The radial image has pairwise diameter strictly belowLLbecause its radial norm is belowLL\. Thereforerange⁡\(mE−mR\)≤2​L\\operatorname\{range\}\(m^\{E\}\-m^\{R\}\)\\leq 2L, and \(24\) applies\.□\\square

###### Lemma 19\(Curvature on a common cone\)\.

Suppose a statessandm≥0m\\geq 0satisfy

us−uj≥m,vs−vj≥mu\_\{s\}\-u\_\{j\}\\geq m,\\qquad v\_\{s\}\-v\_\{j\}\\geq mfor everyj≠sj\\neq s, andrange⁡\(u−v\)≤R\\operatorname\{range\}\(u\-v\)\\leq R\. Then

DKL\(softmaxu∥softmaxv\)≤K\(K−1\)R2e−m\.D\_\{\\rm KL\}\(\\operatorname\{softmax\}u\\\|\\operatorname\{softmax\}v\)\\leq K\(K\-1\)R^\{2\}e^\{\-m\}\.

*Proof\.*Every pointuτ=\(1−τ\)​u\+τ​vu\_\{\\tau\}=\(1\-\\tau\)u\+\\tau vhas the same margin lower bound\. Ifpτ=softmax⁡\(uτ\)p\_\{\\tau\}=\\operatorname\{softmax\}\(u\_\{\\tau\}\), then1−pτ​\(s\)≤\(K−1\)​e−m1\-p\_\{\\tau\}\(s\)\\leq\(K\-1\)e^\{\-m\}\. The Hessian of log\-sum\-exp isdiag⁡\(pτ\)−pτ​pτ⊤\\operatorname\{diag\}\(p\_\{\\tau\}\)\-p\_\{\\tau\}p\_\{\\tau\}^\{\\top\}, so its operator norm is at most2​\(K−1\)​e−m2\(K\-1\)e^\{\-m\}\. The Hessian annihilates common shifts, andrange⁡\(u−v\)≤R\\operatorname\{range\}\(u\-v\)\\leq Rimplies‖𝒞⁡\(u−v\)‖22≤K​R2\\\|\\mathcal\{C\}\(u\-v\)\\\|\_\{2\}^\{2\}\\leq KR^\{2\}\. The integral Taylor formula for the corresponding Bregman divergence now gives \(26\)\.□\\square

### E\.6Binary specialization

WhenK=2K=2, let

m0=μ0\+μ12,δ=\|μ1−μ0\|2\.m\_\{0\}=\\frac\{\\mu\_\{0\}\+\\mu\_\{1\}\}\{2\},\\qquad\\delta=\\frac\{\|\\mu\_\{1\}\-\\mu\_\{0\}\|\}\{2\}\.After relabeling if needed and translating observations bym0m\_\{0\}, the emission means are−δ,\+δ\-\\delta,\+\\delta\. This preserves the likelihood ratio, the latent law, and both recurrences for the pairwise logits\. Their evidence drift and variance are

d=2​δ2σ2=\(μ1−μ0\)22​σ2=dmin,Var⁡\(Yt\)=2​d\.d=\\frac\{2\\delta^\{2\}\}\{\\sigma^\{2\}\}=\\frac\{\(\\mu\_\{1\}\-\\mu\_\{0\}\)^\{2\}\}\{2\\sigma^\{2\}\}=d\_\{\\min\},\\qquad\\operatorname\{Var\}\(Y\_\{t\}\)=2d\.In pairwise log odds the two transition maps are

Fq​\(h\)=2​artanh⁡\(\(1−2​q\)​tanh⁡\(h/2\)\),Gq​\(h\)=L​tanh⁡\(\(1−2​q\)​h/L\)\.F\_\{q\}\(h\)=2\\operatorname\{artanh\}\\\!\\left\(\(1\-2q\)\\tanh\(h/2\)\\right\),\\qquad G\_\{q\}\(h\)=L\\tanh\(\(1\-2q\)h/L\)\.
###### Lemma 20\(Binary common recovery\)\.

Letr=⌈T/c⌉r=\\lceil T/c\\rceil,H=r\+1H=r\+1,a=3​log⁡Ta=3\\log T, andA=B=5​aA=B=5a\. There are eventsℬq\\mathcal\{B\}\_\{q\}such that

Pr⁡\(ℬq\)​L2⟶0,\\Pr\(\\mathcal\{B\}\_\{q\}\)L^\{2\}\\longrightarrow 0,and onℬqc\\mathcal\{B\}\_\{q\}^\{c\}the exact and radial posterior log odds at timeHH, oriented toward the block’s initial state, are both at leastaa\.

*Proof\.*Forx≥0x\\geq 0, define the common budget for one step for the confidence loss

ρq​\(x\)=max⁡\{log⁡\(1−q\+q​ex\)−log⁡\(1−q\),2​q​x\+x2L\}\.\\rho\_\{q\}\(x\)=\\max\\left\\\{\\log\(1\-q\+qe^\{x\}\)\-\\log\(1\-q\),\\ 2qx\+\\frac\{x^\{2\}\}\{L\}\\right\\\}\.The two entries bound the losses ofFqF\_\{q\}andGqG\_\{q\}, respectively, on a positive interval\. Conditional on a block with no switch and its initial sign, the signed evidence variablesY0,…,YrY\_\{0\},\\ldots,Y\_\{r\}are independent𝒩⁡\(d,2​d\)\\mathcal\{N\}\(d,2d\)\. PutN=r\+1N=r\+1and

Q=L\+A\+r​ρq​\(A\),Σpre=exp⁡\[−\(N​d−Q\)24​N​d\]\.Q=L\+A\+r\\rho\_\{q\}\(A\),\\qquad\\Sigma\_\{\\rm pre\}=\\exp\\left\[\-\\frac\{\(Nd\-Q\)^\{2\}\}\{4Nd\}\\right\]\.If∑i=0rYi\>Q\\sum\_\{i=0\}^\{r\}Y\_\{i\}\>Q, both recurrences must hitAA: otherwise the lower bounds for one stepFq​\(x\)≥x−ρq​\(A\)F\_\{q\}\(x\)\\geq x\-\\rho\_\{q\}\(A\)andGq​\(x\)≥x−ρq​\(A\)G\_\{q\}\(x\)\\geq x\-\\rho\_\{q\}\(A\)sum from the uniform reset to a terminal value strictly aboveAA, a contradiction\.

After a hit, monotonicity and saturation give, for either mapMqM\_\{q\},

Mq​\(x\)≥min⁡\{x,B\}−ρq​\(B\)\.M\_\{q\}\(x\)\\geq\\min\\\{x,B\\\}\-\\rho\_\{q\}\(B\)\.Thus the hit persists to marginaawhenever every nonempty suffix obeys

∑i=jr\(Yi−ρq​\(B\)\)\>−\(A−a\)\.\\sum\_\{i=j\}^\{r\}\\bigl\(Y\_\{i\}\-\\rho\_\{q\}\(B\)\\bigr\)\>\-\(A\-a\)\.For a suffix of lengthℓ\\ell, the bound on the Gaussian lower tail is at most

exp⁡\[−\{A−a\+ℓ⁡\[d−ρq​\(B\)\]\}24​ℓ​d\]\.\\exp\\left\[\-\\frac\{\\\{A\-a\+\\ell\[d\-\\rho\_\{q\}\(B\)\]\\\}^\{2\}\}\{4\\ell d\}\\right\]\.Summing over possible hit locations, and usingρq​\(5​a\)→0\\rho\_\{q\}\(5a\)\\to 0, we have eventuallyd−ρq​\(B\)≥d/2d\-\\rho\_\{q\}\(B\)\\geq d/2\. WithA−a=4​aA\-a=4a,

\(4​a\+ℓ​d/2\)24​ℓ​d≥a\+ℓ​d16\.\\frac\{\(4a\+\\ell d/2\)^\{2\}\}\{4\\ell d\}\\geq a\+\\frac\{\\ell d\}\{16\}\.The entire suffix sum is therefore at mostCd​e−a=Cd​T−3C\_\{d\}e^\{\-a\}=C\_\{d\}T^\{\-3\}\.

For the prefix, choosed0=\(c\+d\)/2d\_\{0\}=\(c\+d\)/2\. EventuallyQ≤N​d0Q\\leq Nd\_\{0\}, and

L2​Σpre≤T2​exp⁡\[−r​\(d−c\)216​d\]⟶0\.L^\{2\}\\Sigma\_\{\\rm pre\}\\leq T^\{2\}\\exp\\left\[\-\\frac\{r\(d\-c\)^\{2\}\}\{16d\}\\right\]\\longrightarrow 0\.Removing the conditioning on no switch separately from the prefix and suffix bounds costs at most2​N​q2Nq, and2​N​q​L2=O⁡\(T3​e−T\)→02NqL^\{2\}=O\(T^\{3\}e^\{\-T\}\)\\to 0\. Combining these three terms proves \(29\) and the deterministic hit and persist argument proves the common margin\.□\\square

### E\.7Completion of the main theorem

First letK≥3K\\geq 3\. Conditional onS0=sS\_\{0\}=s, intersect the actual event of no switch𝒩s\\mathcal\{N\}\_\{s\}, the maximal eventℰL\\mathcal\{E\}\_\{L\}, and the two events for exact recovery from Lemmas[14](https://arxiv.org/html/2609.11132#Thmtheorem14)and[15](https://arxiv.org/html/2609.11132#Thmtheorem15)\. On this good event, Lemmas[13](https://arxiv.org/html/2609.11132#Thmtheorem13)and[16](https://arxiv.org/html/2609.11132#Thmtheorem16)give a common terminal margin for the true state at leastc∗​L2/3c\_\{\*\}L^\{2/3\}for somec∗\>0c\_\{\*\}\>0\. Lemmas[18](https://arxiv.org/html/2609.11132#Thmtheorem18)and[19](https://arxiv.org/html/2609.11132#Thmtheorem19)then yield

DKL\(pHE∥pHR\)≤CKL2e−c∗​L2/3\.D\_\{\\rm KL\}\(p^\{E\}\_\{H\}\\\|p^\{R\}\_\{H\}\)\\leq C\_\{K\}L^\{2\}e^\{\-c\_\{\*\}L^\{2/3\}\}\.The complement has conditional probability at most

Hq\+L−3\+\(K−1\)e−κ​L\+CLe−L/2\.Hq\+L^\{\-3\}\+\(K\-1\)e^\{\-\\kappa L\}\+CLe^\{\-L/2\}\.This unions event types only; temporal control is already contained in the single maximal event\. Applying the deterministic2​L2Lenvelope \(25\) on the complement shows that the conditional expected KL tends to zero\. The stationary starting state is uniform, so averaging the finitely many conditional bounds overssproves Theorem[4](https://arxiv.org/html/2609.11132#Thmtheorem4)forK≥3K\\geq 3\.

ForK=2K=2, Lemma[20](https://arxiv.org/html/2609.11132#Thmtheorem20)and \(26\), withR=2​LR=2L, give onℬqc\\mathcal\{B\}\_\{q\}^\{c\}

DKL\(pHE∥pHR\)≤CL2e−3​log⁡T⟶0\.D\_\{\\rm KL\}\(p^\{E\}\_\{H\}\\\|p^\{R\}\_\{H\}\)\\leq CL^\{2\}e^\{\-3\\log T\}\\longrightarrow 0\.Onℬq\\mathcal\{B\}\_\{q\}, \(25\) and \(29\) give𝔼⁡\[DKL;ℬq\]≤2​L​Pr⁡\(ℬq\)→0\\mathbb\{E\}\[D\_\{\\rm KL\};\\mathcal\{B\}\_\{q\}\]\\leq 2L\\Pr\(\\mathcal\{B\}\_\{q\}\)\\to 0\. This proves the binary case for arbitrary distinct means and completes the proof of Theorem[4](https://arxiv.org/html/2609.11132#Thmtheorem4)\.

### E\.8Proof of diverging updates with vanishing predictive KL

*Proof of Theorem[3](https://arxiv.org/html/2609.11132#Thmtheorem3)\.*Fix finiteK≥2K\\geq 2and abbreviate

α=1−K​qK−1,L=LK​\(q\),b=qK−1\.\\alpha=1\-\\frac\{Kq\}\{K\-1\},\\qquad L=L\_\{K\}\(q\),\\qquad b=\\frac\{q\}\{K\-1\}\.Adding a common constant to the input leavessoftmax⁡\(z\)\\operatorname\{softmax\}\(z\), and thereforelog⁡\(Pq⊤​softmax⁡\(z\)\)\\log\(P\_\{q\}^\{\\top\}\\operatorname\{softmax\}\(z\)\), unchanged; the radial output is also unchanged because its definition centers the input\. Pairwise output differences are therefore gauge invariant, and the displayed witnesszq=L\(1/4,−1/4,0,…,0\)z\_\{q\}=L\(1/4,\-1/4,0,\\ldots,0\)is centered\.

Put

Zq=eL/4\+e−L/4\+K−2\.Z\_\{q\}=e^\{L/4\}\+e^\{\-L/4\}\+K\-2\.The definition ofLLgivesb=\(1−q\)​e−Lb=\(1\-q\)e^\{\-L\}\. Fori∈\{0,1\}i\\in\\\{0,1\\\}, let

ηi=b​Zqα​e\(zq\)i\.\\eta\_\{i\}=\\frac\{bZ\_\{q\}\}\{\\alpha e^\{\(z\_\{q\}\)\_\{i\}\}\}\.SinceZq≤K​eL/4Z\_\{q\}\\leq Ke^\{L/4\},

0≤η0≤K⁡\(1−q\)αe−L,0≤η1≤K⁡\(1−q\)αe−L/2\.0\\leq\\eta\_\{0\}\\leq\\frac\{K\(1\-q\)\}\{\\alpha\}e^\{\-L\},\\qquad 0\\leq\\eta\_\{1\}\\leq\\frac\{K\(1\-q\)\}\{\\alpha\}e^\{\-L/2\}\.For fixedKK, both vanish asq→0\+q\\to 0^\{\+\}\. Common output centering cancels in the difference for the chosen pair, so

\(Φq​\(zq\)\)0−\(Φq​\(zq\)\)1\\displaystyle\(\\Phi\_\{q\}\(z\_\{q\}\)\)\_\{0\}\-\(\\Phi\_\{q\}\(z\_\{q\}\)\)\_\{1\}=L2\+log⁡\(1\+η0\)−log⁡\(1\+η1\)\\displaystyle=\\frac\{L\}\{2\}\+\\log\(1\+\\eta\_\{0\}\)\-\\log\(1\+\\eta\_\{1\}\)=L2\+o⁡\(1\),\\displaystyle=\\frac\{L\}\{2\}\+o\(1\),where the absolute remainder is at mostη0\+η1\\eta\_\{0\}\+\\eta\_\{1\}\.

Becauser⁡\(zq\)=L/2r\(z\_\{q\}\)=L/2, the radial gap for the chosen pair is exactly

\(Rq​\(zq\)\)0−\(Rq​\(zq\)\)1=L​tanh⁡\(α/2\)\.\(R\_\{q\}\(z\_\{q\}\)\)\_\{0\}\-\(R\_\{q\}\(z\_\{q\}\)\)\_\{1\}=L\\tanh\(\\alpha/2\)\.Subtracting \(38\) from \(37\), dividing byLL, and usingα→1\\alpha\\to 1yields

\[\(Φq​\(zq\)\)0−\(Φq​\(zq\)\)1\]−\[\(Rq​\(zq\)\)0−\(Rq​\(zq\)\)1\]L⟶12−tanh⁡12=c⋆\>0\.\\frac\{\[\(\\Phi\_\{q\}\(z\_\{q\}\)\)\_\{0\}\-\(\\Phi\_\{q\}\(z\_\{q\}\)\)\_\{1\}\]\-\[\(R\_\{q\}\(z\_\{q\}\)\)\_\{0\}\-\(R\_\{q\}\(z\_\{q\}\)\)\_\{1\}\]\}\{L\}\\longrightarrow\\frac\{1\}\{2\}\-\\tanh\\frac\{1\}\{2\}=c\_\{\\star\}\>0\.The positivity follows fromtanh⁡x<x\\tanh x<xforx\>0x\>0\. The eventual lower bound and divergence of the centered supremum follow directly from \(39\)\.

It remains to compare the decoded distributions at the same witness\. Coordinate00is the unique maximizer of both outputs\. ForK≥3K\\geq 3, every zero coordinate has

ηzero=b​Zqα≤K⁡\(1−q\)αe−3L/4⟶0\.\\eta\_\{\\rm zero\}=\\frac\{bZ\_\{q\}\}\{\\alpha\}\\leq\\frac\{K\(1\-q\)\}\{\\alpha\}e^\{\-3L/4\}\\longrightarrow 0\.Hence the exact margin of the top over zero isL/4\+o⁡\(1\)L/4\+o\(1\), while \(37\) gives the exact margin of the top over coordinate11asL/2\+o⁡\(1\)L/2\+o\(1\)\. WhenK=2K=2there are no zero coordinates, and \(37\) is the only margin over a wrong state\. Every exact margin of the top over a wrong state is therefore eventually at leastL/5L/5\.

Under the radial map, every margin of the top over a wrong state is at least\(L/2\)​tanh⁡\(α/2\)≥L/5\(L/2\)\\tanh\(\\alpha/2\)\\geq L/5eventually\. The exact transition probabilities lie strictly betweenbband1−q1\-q, so its pairwise logit diameter is belowLL; the radial pairwise diameter is also belowLL\. Thus the range ofΦq​\(zq\)−Rq​\(zq\)\\Phi\_\{q\}\(z\_\{q\}\)\-R\_\{q\}\(z\_\{q\}\)is below2​L2L\. Applying Lemma[19](https://arxiv.org/html/2609.11132#Thmtheorem19)withm=L/5m=L/5andR=2​LR=2Lgives

DKL\(softmaxΦq\(zq\)∥softmaxRq\(zq\)\)≤4K\(K−1\)L2e−L/5⟶0\.D\_\{\\rm KL\}\\\!\\left\(\\operatorname\{softmax\}\\Phi\_\{q\}\(z\_\{q\}\)\\,\\middle\\\|\\,\\operatorname\{softmax\}R\_\{q\}\(z\_\{q\}\)\\right\)\\leq 4K\(K\-1\)L^\{2\}e^\{\-L/5\}\\longrightarrow 0\.This proves all claims\.□\\square

### E\.9Proof of the binary transition geometry

*Proof of Proposition[5](https://arxiv.org/html/2609.11132#Thmtheorem5)\.*Writingu=ehu=e^\{h\}andDq​\(u\)=\(\(1−q\)​u\+q\)​\(q​u\+1−q\)D\_\{q\}\(u\)=\(\(1\-q\)u\+q\)\(qu\+1\-q\)gives

Fq′​\(h\)=\(1−2​q\)​uDq​\(u\),Fq′′​\(h\)=\(1−2​q\)​q​\(1−q\)​u​\(1−u2\)Dq​\(u\)2\.F\_\{q\}^\{\\prime\}\(h\)=\\frac\{\(1\-2q\)u\}\{D\_\{q\}\(u\)\},\\qquad F\_\{q\}^\{\\prime\\prime\}\(h\)=\\frac\{\(1\-2q\)q\(1\-q\)u\(1\-u^\{2\}\)\}\{D\_\{q\}\(u\)^\{2\}\}\.The first formula, equivalentlyFq′​\(h\)=\(1−2​q\)​p​\(1−p\)/\[p′​\(1−p′\)\]F\_\{q\}^\{\\prime\}\(h\)=\(1\-2q\)p\(1\-p\)/\[p^\{\\prime\}\(1\-p^\{\\prime\}\)\]forp=σ⁡\(h\)p=\\sigma\(h\)andp′=q\+\(1−2​q\)​pp^\{\\prime\}=q\+\(1\-2q\)p, is at most1−2​q1\-2q, with equality only ath=0h=0; the rangep′∈\(q,1−q\)p^\{\\prime\}\\in\(q,1\-q\)proves saturation\. The second formula is strictly negative forh\>0h\>0, so strict concavity on\[0,2​x\]\[0,2x\]andFq​\(0\)=0F\_\{q\}\(0\)=0giveΔq=2​Fq​\(x\)−Fq​\(2​x\)\>0\\Delta\_\{q\}=2F\_\{q\}\(x\)\-F\_\{q\}\(2x\)\>0\. For residualsez=Fq​\(z\)−A⁡\(z\)e\_\{z\}=F\_\{q\}\(z\)\-A\(z\), the affine second difference vanishes and hence

Δq=\|e0−2​ex\+e2​x\|≤4​maxz​\|ez\|,\\Delta\_\{q\}=\|e\_\{0\}\-2e\_\{x\}\+e\_\{2x\}\|\\leq 4\\max\_\{z\}\|e\_\{z\}\|,which proves the quantitative obstruction\.□\\square

## Appendix FEmpirical Estimands and Aggregation

This appendix defines the illustration with equally spaced Gaussians at fixedKK, experiments with learned models, and estimands for the scalar controls\. These experiments illustrate the results and are not used in the proof of Theorem[4](https://arxiv.org/html/2609.11132#Thmtheorem4)\. The displayed subset with equally spaced Gaussians was selected after the broader prespecified grid had been evaluated, to illustrate the two directions rather than test a uniform onset at finite scale\. On the original Gaussian grid, four of five configurations pass the joint directional criterion; theK=8K=8skewed configuration has an unresolved slope of the log KL over the prespecified tail\. The supplement includes results for the full original grid and identifies the additional experiments conducted afterward; the displayed subset does not replace the original outcome\.

### F\.1Illustration with equally spaced Gaussians at fixedKK

#### F\.1\.1Design, coupling, and estimands

ForK∈\{2,4,8\}K\\in\\\{2,4,8\\\}, the scalar Gaussian means begin as the arithmetic progression0,1,…,K−10,1,\\ldots,K\-1, are centered, and are rescaled so that the minimum pairwise distance is one\. The observation variance is one, hence

dmin=mini≠j⁡\(μi−μj\)22=12,c=0\.45​dmin=0\.225\.d\_\{\\min\}=\\min\_\{i\\neq j\}\\frac\{\(\\mu\_\{i\}\-\\mu\_\{j\}\)^\{2\}\}\{2\}=\\frac\{1\}\{2\},\\qquad c=0\.45d\_\{\\min\}=0\.225\.At eachL∈\{8,16,32,64,128\}L\\in\\\{8,16,32,64,128\\\}we set

q=K−1eL\+K−1,H=⌈L0\.225⌉\+1\.q=\\frac\{K\-1\}\{e^\{L\}\+K\-1\},\\qquad H=\\left\\lceil\\frac\{L\}\{0\.225\}\\right\\rceil\+1\.This implemented horizon differs from the theorem’s⌈−log\(q\)/c⌉\+1\\lceil\-\\log\(q\)/c\\rceil\+1byOK​\(1\)O\_\{K\}\(1\)steps, sinceL=−log⁡q\+log⁡\(K−1\)\+log⁡\(1−q\)L=\-\\log q\+\\log\(K\-1\)\+\\log\(1\-q\); it shares the logarithmic scale rather than the exact schedule\. Four seed blocks contribute 1,024 stationary trajectories per cell, for 4,096 paired paths\. Exact Bayes and the tanh radial filter consume the same observations\. Every displayed path and filter state is finite\.

The internal quantity isGH=‖𝒞⁡\(zHE−zHR\)‖2G\_\{H\}=\\\|\\mathcal\{C\}\(z\_\{H\}^\{E\}\-z\_\{H\}^\{R\}\)\\\|\_\{2\}and the task quantity isDKL\(pHE∥pHR\)D\_\{\\rm KL\}\(p\_\{H\}^\{E\}\\\|p\_\{H\}^\{R\}\)\. The normalized gapGH/LG\_\{H\}/Ldistinguishes growth proportional to the confidence scale from a merely positive absolute gap\. Internal slopes regress meanGHG\_\{H\}onLL; predictive slopes regress the logarithm of mean predictive KL onLL\. Both use the predeclared tail windowL∈\{32,64,128\}L\\in\\\{32,64,128\\\}\.

Uncertainty is computed by resampling complete paths\. Each of 2,000 bootstrap replicates independently resamples path indices within each scale cell, then fits the statistic across those resampled cell means\. Filters remain paired on each path, but independently simulated scales are not paired\. Table[5](https://arxiv.org/html/2609.11132#A6.T5)reports the three displayed configurations\.

Table 5:Equally spaced Gaussian illustration at fixed scale\.Internal slopes regress centered separation from exact to radial onLL; slopes of the log KL regresslogDKL\(exact∥radial\)\\log D\_\{\\rm KL\}\(\\mathrm\{exact\}\\\|\\mathrm\{radial\}\)onLL\. These values illustrate the theorem’s two directions for three fixed state counts; they are not used in its proof\.
#### F\.1\.2Controls and implementation details

Hard clipping and the affine update are comparisons rather than assumptions of the theorem\. Hard clipping may outperform tanh at finite scales because the theorem claims existence and asymptotic convergence, not optimality of the smooth map\. The affine control has lower mean KL than tanh in every displayed tail cellL≥32L\\geq 32\. All 4,096 sampled paths in each such cell contain no switch even by2​H2H, so these curves do not test rare costs in the switch tail or identify saturation as necessary\. The binary⌈8/q⌉\\lceil 8/q\\rceilcontrols address a separate recovery question at the longer endpoint\.

The supplementary code includes the data generator, simulation parameters, seed construction, measurements per trajectory, and the 2,000\-replicate bootstrap procedure used to produce the curves and table\. Numerical checks verify the KL direction and that all simulated trajectories are complete and finite\. Table values are rounded to four decimal places; the accompanying data provide full precision\.

### F\.2Experiments with learned models

The experiment covers six switch probabilitiesq∈\{2−3,…,2−8\}q\\in\\\{2^\{\-3\},\\ldots,2^\{\-8\}\\\}, two training stages \(G1 distilled and G2 trained end to end\), five learned arms and an additional analytic tanh reference, three tuning seeds, and ten disjoint confirmation seeds\. For each stage and model, the selected hyperparameter setting minimizes mean validation loss over all three tuning seeds, with ties broken deterministically; a setting missing any tuning seed is ineligible\. This selection rule was fixed before confirmation\. All180180tuning runs and600600final evaluation runs completed successfully\. Every confirmation seed is included in the reported aggregates\. The experimental protocol also specified test excess NLL on the next observation and calibration error, but neither was measured\. Table[6](https://arxiv.org/html/2609.11132#A6.T6)specifies the model architectures and selected optimizer settings; the official Mamba implementation ismamba\-ssm==2\.3\.2\.post1\.

Table 6:Model architectures and training settings\.All arms use AdamW, batch size256256, at most10,00010\{,\}000updates, validation every250250updates, and patience2,0002\{,\}000;η\\etaandλ\\lambdadenote learning rate and weight decay\. G1 distills the exact predictive distribution, whereas G2 minimizes negative log likelihood of the next observation\. Parameter counts are matched within5%5\\%\. The arm with a finite window usesH⁡\(q\)=⌈−log⁡q⌉\+1H\(q\)=\\lceil\-\\log q\\rceil\+1; the Sharan existence boundlog⁡2/H⁡\(q\)\\log 2/H\(q\)ranges from0\.1730\.173to0\.0990\.099over this grid and is not a guarantee for the trained MLP\.The closure statistic in Table[2](https://arxiv.org/html/2609.11132#S7.T2)is computed within a common stage,qq, seed, and endpoint:

Ca=DS6−DaDS6−Dtanh\.C\_\{a\}=\\frac\{D\_\{\\mathrm\{S6\}\}\-D\_\{a\}\}\{D\_\{\\mathrm\{S6\}\}\-D\_\{\\tanh\}\}\.Thus S6 is00and the analytic tanh arm is11before averaging\. The table reports the mean and standard error of these ten closure values, one for each seed, not a standard error obtained by treating paths within a seed as independent replicates\.

### F\.3Predictive KL: orientation and clipping

The experiments with learned models and the scalar controls also use different probability clipping settings\. Table[2](https://arxiv.org/html/2609.11132#S7.T2)usesDKL\(exact Bayes∥model\)D\_\{\\mathrm\{KL\}\}\(\\text\{exact Bayes\}\\\|\\text\{model\}\)after float32 sigmoid conversion and clipping atε32=torch\.finfo\(torch\.float32\)\.eps≈1\.1920929×10−7\\varepsilon\_\{32\}=\\texttt\{torch\.finfo\(torch\.float32\)\.eps\}\\approx 1\.1920929\\times 10^\{\-7\}\. The scalar controls below use float64 probabilities clipped at10−1510^\{\-15\}\. These settings compute KL after different clipping operations, so the reported values are not directly interchangeable\.

The fixed\-KKtheorems, Table[7](https://arxiv.org/html/2609.11132#A6.T7), and Figure[2](https://arxiv.org/html/2609.11132#S5.F2)all use the directionDKL\(exact∥approximation\)D\_\{\\mathrm\{KL\}\}\(\\text\{exact\}\\\|\\text\{approximation\}\), although the experiments use their separately disclosed probability clipping settings\. An additional diagnostic reverses the direction and reportsDKL\(tanh∥exact\)D\_\{\\mathrm\{KL\}\}\(\\tanh\\\|\\mathrm\{exact\}\)\. Because KL is asymmetric, those values in the reverse orientation are not compared numerically with the theorem or the comparison from exact to control\.

That auxiliary diagnostic evaluates the tanh recurrence atH⁡\(q\)H\(q\)without probability clipping\. It computes Bernoulli KL directly in logit space, uses the same ten seeds and20,00020\{,\}000paths per seed, and is reported separately from the comparison from exact to control\.

The largest relative standard error is0\.40%0\.40\\%\. The loss in the reverse orientation increases across this finite grid\. It is a diagnostic in the finite regime, not an estimate of either proved limit from exact to radial or its asymptotic rate\.

Ifeeis the exact predictive logit,mmis a model predictive logit,p=σ⁡\(e\)p=\\sigma\(e\), andr=σ⁡\(m\)r=\\sigma\(m\), the corresponding untruncated empirical estimand is

Dtrue\(e∥m\)=DKL\(Ber\(p\)∥Ber\(r\)\)=p\(e−m\)\+sp\(m\)−sp\(e\),sp\(z\)=log\(1\+ez\)\.D\_\{\\mathrm\{true\}\}\(e\\\|m\)=D\_\{\\mathrm\{KL\}\}\\\!\\left\(\\operatorname\{Ber\}\(p\)\\middle\\\|\\operatorname\{Ber\}\(r\)\\right\)=p\(e\-m\)\+\\operatorname\{sp\}\(m\)\-\\operatorname\{sp\}\(e\),\\qquad\\operatorname\{sp\}\(z\)=\\log\(1\+e^\{z\}\)\.The last expression is a stable evaluation in logit space of the ordinary Bernoulli KL; implementations may evaluatesp\\operatorname\{sp\}bylogaddexpwithout clipping either probability\.

The experiment with scalar controls evaluates a distinct, clipped quantity\. Forcε​\(u\)=min⁡\{1−ε,max⁡\{ε,u\}\}c\_\{\\varepsilon\}\(u\)=\\min\\\{1\-\\varepsilon,\\max\\\{\\varepsilon,u\\\}\\\}withε=10−15\\varepsilon=10^\{\-15\}, it records

Dclip,ε\(e∥m\)=DKL\(Ber\(cε\(σ\(e\)\)\)∥Ber\(cε\(σ\(m\)\)\)\)\.D\_\{\\mathrm\{clip\},\\varepsilon\}\(e\\\|m\)=D\_\{\\mathrm\{KL\}\}\\\!\\left\(\\operatorname\{Ber\}\(c\_\{\\varepsilon\}\(\\sigma\(e\)\)\)\\middle\\\|\\operatorname\{Ber\}\(c\_\{\\varepsilon\}\(\\sigma\(m\)\)\)\\right\)\.Accordingly, every scalar value currently printed in Table[7](https://arxiv.org/html/2609.11132#A6.T7)and plotted in the third panel of Figure[2](https://arxiv.org/html/2609.11132#S5.F2)is clipped predictive KL with orientation*exact Bayes∥\\\|control*\. Clipping is nearly inactive for the two saturating controls but can substantially cap a nonsaturating control in an extreme tail\. The clipped and untruncated estimands must therefore be named separately\. The reported control results use clipped KL, not the untruncated quantity\.

### F\.4Scalar controls and endpoints

All four scalar recurrences consume the same simulated observation paths as the exact filter\. AfterTTobservations, lethTh\_\{T\}be the exact posterior logit andh~Ta\\widetilde\{h\}\_\{T\}^\{a\}the state of controlaa\. The final logits compared by the scalar evaluator are

eT=Fq​\(hT\),mTa=Mqa​\(h~Ta\),a∈\{tanh,clip,affine,identity\}\.e\_\{T\}=F\_\{q\}\(h\_\{T\}\),\\qquad m\_\{T\}^\{a\}=M\_\{q\}^\{a\}\(\\widetilde\{h\}\_\{T\}^\{a\}\),\\qquad a\\in\\\{\\tanh,\\mathrm\{clip\},\\mathrm\{affine\},\\mathrm\{identity\}\\\}\.The two endpoints are the horizon matched to the theoremH⁡\(q\)=⌈−log⁡q⌉\+1H\(q\)=\\lceil\-\\log q\\rceil\+1and the long endpoint⌈8/q⌉\\lceil 8/q\\rceil\. The experiment in Table[7](https://arxiv.org/html/2609.11132#A6.T7)and Figure[2](https://arxiv.org/html/2609.11132#S5.F2)usesq=2−3q=2^\{\-3\}andq=2−8q=2^\{\-8\}, ten public seeds1080,…,10891080,\\ldots,1089,20,00020\{,\}000paths per seed atH⁡\(q\)H\(q\), and2,0002\{,\}000paths per seed at the long endpoint\. Thus each short cell contains200,000200\{,\}000paths and each long cell contains20,00020\{,\}000paths\.

Table 7:Controls separate saturation from geometry\.Expected KL at the final step,KL\(exact Bayes∥control\)\\mathrm\{KL\}\(\\text\{exact Bayes\}\\\|\\text\{control\}\), at the endpoint matched to the theoremH⁡\(q\)=⌈−log⁡q⌉\+1H\(q\)=\\lceil\-\\log q\\rceil\+1and the long endpoint⌈8/q⌉\\lceil 8/q\\rceil\. At the long endpoint, the saturating losses remain below6\.4×10−36\.4\\times 10^\{\-3\}, while the tested affine and identity losses reach5\.535\.53and14\.5414\.54nats, respectively \(bold\)\.For a seedsswithNsN\_\{s\}paths, the stored replicate is

D^sa=1Ns∑i=1NsDclip,10−15\(eT,i∥mT,ia\)\.\\widehat\{D\}\_\{s\}^\{a\}=\\frac\{1\}\{N\_\{s\}\}\\sum\_\{i=1\}^\{N\_\{s\}\}D\_\{\\mathrm\{clip\},10^\{\-15\}\}\(e\_\{T,i\}\\\|m\_\{T,i\}^\{a\}\)\.WithS=10S=10, the manuscript mean and uncertainty are

D¯a=1S​∑s=1SD^sa,SE⁡\(D¯a\)=1S⁡\(S−1\)​∑s=1S\(D^sa−D¯a\)2\.\\overline\{D\}^\{a\}=\\frac\{1\}\{S\}\\sum\_\{s=1\}^\{S\}\\widehat\{D\}\_\{s\}^\{a\},\\qquad\\operatorname\{SE\}\(\\overline\{D\}^\{a\}\)=\\sqrt\{\\frac\{1\}\{S\(S\-1\)\}\\sum\_\{s=1\}^\{S\}\(\\widehat\{D\}\_\{s\}^\{a\}\-\\overline\{D\}^\{a\}\)^\{2\}\}\.The random stream for oneqq/endpoint/seed cell is derived from the displayed seed, the exponent ofqq, and the horizon; all four controls reuse that cell\. The manuscript aggregates seed means rather than treating paths within a seed as independent replicates\. The largest relative standard error among the sixteen plotted quantities is at most3\.1%3\.1\\%\.

The right pair in Figure[2](https://arxiv.org/html/2609.11132#S5.F2)separates measurement from illustration\. The third panel reports the mean losses of the scalar controls\. The fourth panel is not a sampled trajectory: it evaluates the logistic decoder on a fixed grid, with schematic guide logitsv=3v=3andu=5\.25u=5\.25\.

### F\.5Errors relative to the latent state

LetZT\+1∈\{0,1\}Z\_\{T\+1\}\\in\\\{0,1\\\}be the latent state after the terminal transition,sZ=2​ZT\+1−1s\_\{Z\}=2Z\_\{T\+1\}\-1,eTe\_\{T\}the exact predictive logit, andmTaffm\_\{T\}^\{\\rm aff\}the affine predictive logit\. The event that the affine prediction disagrees with the latent state is

Wtruth=𝟏\{sZmTaff≤0\}\.W\_\{\\rm truth\}=\\mathbf\{1\}\\\{s\_\{Z\}m\_\{T\}^\{\\rm aff\}\\leq 0\\\}\.The associated confidence event used by the formal diagnostic for the lower bound is

Wjoint=Wtruth1\{σ\(sZeT\)≥15/16\}\.W\_\{\\rm joint\}=W\_\{\\rm truth\}\\,\\mathbf\{1\}\\\{\\sigma\(s\_\{Z\}e\_\{T\}\)\\geq 15/16\\\}\.Both indicators are defined relative to the true latent state\. Disagreement with the exact filter is a different quantity and was not measured in this experiment\.

The experiment usesq=2−5,2−8,2−10,2−12q=2^\{\-5\},2^\{\-8\},2^\{\-10\},2^\{\-12\}, the same ten public seeds, and1,0001\{,\}000paths per seed\. Means across seeds±\\pmstandard errors forWtruthW\_\{\\rm truth\}are respectively0\.1982±0\.00640\.1982\\pm 0\.0064,0\.1821±0\.00420\.1821\\pm 0\.0042,0\.1829±0\.00310\.1829\\pm 0\.0031, and0\.1868±0\.00360\.1868\\pm 0\.0036\. The correspondingWjointW\_\{\\rm joint\}values, qualified by confidence, are0\.0794±0\.00270\.0794\\pm 0\.0027,0\.1649±0\.00460\.1649\\pm 0\.0046,0\.1781±0\.00280\.1781\\pm 0\.0028, and0\.1856±0\.00360\.1856\\pm 0\.0036\. The reported range describes errors relative to the latent state, rather than disagreement with the exact filter\.

### F\.6Scope of the empirical conclusions

The experiments support two deliberately limited conclusions\. First, at the long endpoint the saturating controls remain stable while the two nonsaturating controls degrade under the disclosed clipped metric\. Second, matching the short theorem horizon does not by itself establish stability at long horizons for a learned architecture\. Neither observation proves the open lower bound for the terminal joint event qualified by confidence, identifies tanh as the unique stable recurrence, or establishes a positive result on transfer between architectures\.

## Appendix GExtended Background

This appendix places the result in the literature on recurrent models, approximate filtering, and sequential prediction\.

### G\.1The family with a deterministic state

The recurrence studied here is deliberately the smallest object that exhibits the property in question: a state of fixed size updated deterministically from the previous state and the current observation\. For recurrent implementations, this property permits inference in linear time and memory independent of sequence length\. Configurations with long convolutions or attention hybrids need not share that memory bound\.

The lineage begins with the question of how a state of fixed size should summarize a growing history at all\. Optimal polynomial projection gives one answer with an explicit approximation guarantee\[[24](https://arxiv.org/html/2609.11132#bib.bib24)\], and structured state spaces turn that answer into a trainable layer\[[25](https://arxiv.org/html/2609.11132#bib.bib25)\], later simplified\[[60](https://arxiv.org/html/2609.11132#bib.bib60)\]and given variants in continuous time\[[28](https://arxiv.org/html/2609.11132#bib.bib28)\]\. Initialization of these models remains an active question in its own right\[[39](https://arxiv.org/html/2609.11132#bib.bib39)\]\. A parallel line showed that carefully parameterized linear recurrences recover much of the benefit without the state space machinery\[[47](https://arxiv.org/html/2609.11132#bib.bib47)\], and designs with long convolutions and hybrid designs traded recurrence for structured mixing\[[51](https://arxiv.org/html/2609.11132#bib.bib51),[19](https://arxiv.org/html/2609.11132#bib.bib19)\]\. Gated linear attention and its delta rule successors reached the same destination from the attention side\[[67](https://arxiv.org/html/2609.11132#bib.bib67),[68](https://arxiv.org/html/2609.11132#bib.bib68)\], with sparsity now used to enlarge the state without paying for it densely\[[8](https://arxiv.org/html/2609.11132#bib.bib8)\], while recurrent architectures competitive at language scale arrived independently\[[49](https://arxiv.org/html/2609.11132#bib.bib49),[3](https://arxiv.org/html/2609.11132#bib.bib3)\]\. The selective mechanism that makes the transition depend on the input\[[23](https://arxiv.org/html/2609.11132#bib.bib23)\]and the duality that exposes such models as a restricted form of attention\[[12](https://arxiv.org/html/2609.11132#bib.bib12)\]complete the picture\.

These models motivate asking what predictive cost follows from a prescribed update geometry\. Shared motivation does not extend our existence theorem to every member of this family: the theorem concerns the explicit radial filter, not its implementation by a named neural architecture\.

### G\.2What is already known about the gap, and what is not

The representational side of the question is comparatively well mapped\. Formal placements bound what specified architectures can express\[[54](https://arxiv.org/html/2609.11132#bib.bib54)\], while empirical studies on formal languages probe generalization outside the training distribution\[[13](https://arxiv.org/html/2609.11132#bib.bib13)\]\. Arguments from circuit complexity place models of fixed depth inside classes that cannot perform certain sequential computations\[[43](https://arxiv.org/html/2609.11132#bib.bib43)\], and analyses with limited precision show how much of the apparent capacity survives finite arithmetic\[[38](https://arxiv.org/html/2609.11132#bib.bib38)\]\. On the constructive side, changing the spectrum or the sparsity pattern of the transition recovers ability to track the state that diagonal models lack\[[22](https://arxiv.org/html/2609.11132#bib.bib22),[64](https://arxiv.org/html/2609.11132#bib.bib64),[58](https://arxiv.org/html/2609.11132#bib.bib58)\], and the frontier between recall and throughput has been characterized directly\[[2](https://arxiv.org/html/2609.11132#bib.bib2)\]\. Benchmarks now target exact state tracking as a primary object\[[65](https://arxiv.org/html/2609.11132#bib.bib65)\]\.

What this literature does not settle is the step our paper takes\. A proved inability to represent an update is not by itself a proved cost under the data distribution, because the distribution need not visit the region where the representation fails\. The closest existing work asks about generalization of selective state space models on filtering tasks\[[63](https://arxiv.org/html/2609.11132#bib.bib63)\], which shares our setting but not our question: we fix the recurrence and ask what its representational shortfall costs in expected predictive divergence\.

### G\.3Filtering, stability, and their relation to this result

Hidden Markov filtering\[[53](https://arxiv.org/html/2609.11132#bib.bib53)\]and entropy questions for functions of chains with finitely many states\[[6](https://arxiv.org/html/2609.11132#bib.bib6)\]have a long history\. Exact filtering of aKK\-state HMM admits a deterministic continuous belief state withK−1K\-1coordinates\. Finite dimension is not finite cardinality or bounded precision; our restriction concerns update geometry, not the existence of a sufficient state of finite dimension\. The stability literature asks a superficially similar question to ours and a materially different one: whether a filter initialized incorrectly but running the correct kernels forgets its error\[[46](https://arxiv.org/html/2609.11132#bib.bib46)\], surveyed by[Chigansky et al\. \[9\]](https://arxiv.org/html/2609.11132#bib.bib9), with quantitative forms via contraction coefficients\[[41](https://arxiv.org/html/2609.11132#bib.bib41)\]\. Robustness results perturb the kernels instead, controlling policy costs\[[31](https://arxiv.org/html/2609.11132#bib.bib31),[33](https://arxiv.org/html/2609.11132#bib.bib33)\]or error in the filter kernel and policy performance\[[14](https://arxiv.org/html/2609.11132#bib.bib14)\]\. Policies with a finite window can become nearly optimal as window length grows\[[32](https://arxiv.org/html/2609.11132#bib.bib32)\]\. Our state dimension is fixed, butRqR\_\{q\}changes numerically withqq, and its map discrepancy in the worst case grows rather than vanishing as a perturbation parameter\. How fast a correct filter forgets is itself quantified\[[37](https://arxiv.org/html/2609.11132#bib.bib37)\], and estimating the mixing time is a developed subject\[[66](https://arxiv.org/html/2609.11132#bib.bib66)\]\.

Learned and structured filters form a third strand\. Probabilistic state space models have been fused with the selective architecture\[[4](https://arxiv.org/html/2609.11132#bib.bib4)\], particle filters made differentiable\[[11](https://arxiv.org/html/2609.11132#bib.bib11)\], belief embeddings learned with consistency guarantees\[[61](https://arxiv.org/html/2609.11132#bib.bib61)\], and Updates in the style of the Kalman filter recast as attention or as regression at test time\[[55](https://arxiv.org/html/2609.11132#bib.bib55),[50](https://arxiv.org/html/2609.11132#bib.bib50)\], including adaptive variants\[[42](https://arxiv.org/html/2609.11132#bib.bib42)\]\. Learning from ordinary HMM observation sequences can be cryptographically hard, whereas query access to conditional probabilities permits efficient learning; guarantees from conditional samples also depend on a fidelity parameter\[[30](https://arxiv.org/html/2609.11132#bib.bib30)\]\. Reconstruction from finite messages on trees exhibits phase transitions between memory and accuracy\[[29](https://arxiv.org/html/2609.11132#bib.bib29)\], and a recent synthesis compares state space models and HMMs\[[20](https://arxiv.org/html/2609.11132#bib.bib20)\]\.

### G\.4Prediction under a wrong model

Our theorem measures the categorical KL between filtered posteriors after the observation, not the KL between densities of the next observation, and binary empirical endpoints additionally apply transition mixing\. These losses score decoded distributions rather than internal parameters\. Prediction with short memory supplies a conceptual benchmark on the next observation\[[57](https://arxiv.org/html/2609.11132#bib.bib57)\]; sequential prediction under log loss with a misspecified model class is studied directly\[[18](https://arxiv.org/html/2609.11132#bib.bib18)\]; and streaming projection onto a mixture over paths with a fixed budget is another deterministic recurrence with a different maintained object and truncation\[[17](https://arxiv.org/html/2609.11132#bib.bib17)\]\. Read as detection delay, our controls at long horizons belong to quickest change detection, from the original cumulative sum scheme\[[48](https://arxiv.org/html/2609.11132#bib.bib48)\]through its optimality theory\[[40](https://arxiv.org/html/2609.11132#bib.bib40),[44](https://arxiv.org/html/2609.11132#bib.bib44),[52](https://arxiv.org/html/2609.11132#bib.bib52)\]to learned detectors\[[21](https://arxiv.org/html/2609.11132#bib.bib21)\]\. Behavior in context on Markov sources is a further contact point\[[7](https://arxiv.org/html/2609.11132#bib.bib7),[69](https://arxiv.org/html/2609.11132#bib.bib69)\]\.

### G\.5Comparison by mathematical object

Nearby literatures can sound interchangeable when summarized as “approximating a sequence model,” but they differ in what may change with the accuracy target and in where error is measured\. Table[8](https://arxiv.org/html/2609.11132#A7.T8)records the distinction needed here\.

Table 8:Nearby questions separated by object and limit\.The comparison is conceptual; it does not claim that the cited settings are special cases of ours\.The comparison also limits the architectural interpretation: a selective SSM, a gated recurrent network, and the radial filter all keep a fixed state, but the theorem concerns the explicit radial map, not a class containing every such recurrence\. Exact Bayes and the radial witness are mathematical filters, so their separation predates training or parameterization\. The result sits between architecture expressivity and statistical decision theory: an internal separation can fail to induce predictive loss under a specified sequential law, leaving the converse criterion and learned realization open\.