The Attention Within: Consensus Dynamics in Selective State Space Models

# The Attention Within: Consensus Dynamics in Selective State Space Models
Source: [https://arxiv.org/html/2609.17997](https://arxiv.org/html/2609.17997)
João Pedro SilvestreAffiliation:João Pedro Silvestre and Paulo Tabuada are with the Electrical and Computer Engineering Department, University of California at Los Angeles, Los Angeles, CA 90095 USA \(e\-mail:\{joaosilvestre, tabuada\}@ucla\.edu\)\.Álvaro Rodríguez AbellaAffiliation:Álvaro Rodríguez Abella is with the Department of Applied Mathematics, Comillas Pontifical University, Madrid, 28015 \- Madrid, Spain \(e\-mail:arabella@comillas\.edu\)\.Paulo Tabuada††thanks:This work was partially supported by the NSF award 2502536 and the Air Force Office of Scientific Research under the Multidisciplinary University Research Initiative grant Hybrid Dynamics \- Deconstruction and Aggregation \(HyDDRA\)\. J\.P\.S was partially supported by the PhD fellowship 2023\.01843\.BD from the Fundação para a Ciência e a Tecnologia \(FCT\), Portugal\. A\.R\.A\. was partially supported by grant PID2024\-156578NB\-I00 funded by MICIU/AEI/10\.13039/501100011033/FEDER, EU\.Affiliation:João Pedro Silvestre and Paulo Tabuada are with the Electrical and Computer Engineering Department, University of California at Los Angeles, Los Angeles, CA 90095 USA \(e\-mail:\{joaosilvestre, tabuada\}@ucla\.edu\)\.

###### Abstract

Selective state space models \(SSMs\) have recently emerged as a compelling alternative to transformers, combining competitive performance with substantially improved inference efficiency\. At each SSM layer, a sequence of hidden states are propagated by a recurrence, mixing information of different tokens\. Despite using a different mechanism, this mixing plays a role analogous to attention in transformers\. In fact, recent works have shown that the two architectures may be closer than they first appear, as this recurrence admits a formulation akin to linear attention\. In transformers, attention is known to drive the tokens to cluster, i\.e\., to reach consensus, collapsing in the limit to a single direction\. Thus, we ask: does the recurrence at the core of SSMs drive the tokens to consensus, as attention does in transformers?

To answer this question, we take a dynamical systems perspective on SSMs, modeling the evolution of tokens across layers as an ordinary differential equation\. By exploiting input\-to\-state stability arguments, we establish local exponential stability of the consensus equilibria and characterize their domain of attraction for time\-varying weight matrices, a setting not addressed by previous results\. We thereby show that the resemblance between SSMs and transformers does run deeper: the recurrence at the core of SSMs aggregates tokens just as attention does\. Numerical experiments on a pretrained Mamba\-2 model point to the output gate as the component that regulates the extent of this consensus, preventing the tokens from reaching it in full\.

## IIntroduction

In recent years, large language models \(LLMs\) have seen widespread adoption across a rapidly expanding range of tasks\[[1](https://arxiv.org/html/2609.17997#bib.bib22)\]\. Chief among these models is the transformer\[[2](https://arxiv.org/html/2609.17997#bib.bib15)\], which has emerged as the dominant architectural paradigm: it underpins the foundation models in widest use today, including popular models such as ChatGPT\[[3](https://arxiv.org/html/2609.17997#bib.bib25)\]\. The reach of the architecture, however, extends well beyond language\. Attention, the mechanism at its core, first introduced for neural machine translation\[[4](https://arxiv.org/html/2609.17997#bib.bib29)\], has since proved to be a powerful mechanism, allowing the introduction of transformers into domains such as vision\[[5](https://arxiv.org/html/2609.17997#bib.bib23)\]and protein structure prediction\[[6](https://arxiv.org/html/2609.17997#bib.bib24)\]\.

However, transformers are not without shortcomings\. The first is computational: the cost of attention grows quadratically with the sequence length\[[7](https://arxiv.org/html/2609.17997#bib.bib26)\]\. The second emerges with depth\. As models grow deeper, now reaching hundreds of layers\[[8](https://arxiv.org/html/2609.17997#bib.bib14),[9](https://arxiv.org/html/2609.17997#bib.bib16)\], the returns diminish and the expressive power of the network saturates beyond a certain point\[[10](https://arxiv.org/html/2609.17997#bib.bib17)\], while the tokens grow alike as more and more layers are traversed\[[11](https://arxiv.org/html/2609.17997#bib.bib5),[12](https://arxiv.org/html/2609.17997#bib.bib27)\]\. What to make of this last effect is still contested\. Some works see it as a defect\[[11](https://arxiv.org/html/2609.17997#bib.bib5)\], since tokens that become indistinguishable can no longer carry distinct information\. Others see it as the very mechanism by which the model groups related tokens\[[13](https://arxiv.org/html/2609.17997#bib.bib6)\], and it has been used directly to solve language tasks, by clustering the tokens of a sentence around the ones that carry most of its meaning\[[14](https://arxiv.org/html/2609.17997#bib.bib30)\]\.

Selective state space models, first introduced in\[[15](https://arxiv.org/html/2609.17997#bib.bib3),[16](https://arxiv.org/html/2609.17997#bib.bib4)\], were designed to reduce the quadratic cost of attention at inference time\. In doing so, they retain the benefits of recurrent neural networks\[[17](https://arxiv.org/html/2609.17997#bib.bib19)\]while avoiding their classical shortcomings, such as vanishing gradients and the lack of parallelism during training\.

The differences between SSMs and transformers, however, may be more subtle than they first appear\. Recent works have shown that the core of an SSM admits a formulation akin to linear attention\[[16](https://arxiv.org/html/2609.17997#bib.bib4),[18](https://arxiv.org/html/2609.17997#bib.bib2)\], exposing intrinsic similarities between the two architectures\. However, that similarity is structural, and does not by itself determine how the tokens behave across layers\. A natural question thus arises: do the models share fundamental dynamical properties?

Several works have shown that SSMs already mirror transformers in some of their expressiveness barriers\[[19](https://arxiv.org/html/2609.17997#bib.bib20)\]; empirical studies suggest that the parallel may extend to the consensus phenomenon, where SSM tokens can cluster and become increasingly indistinguishable across layers\[[20](https://arxiv.org/html/2609.17997#bib.bib11),[21](https://arxiv.org/html/2609.17997#bib.bib12)\]\. For transformers, the phenomenon is by now well documented: as layers accumulate, the tokens cluster together and drift toward a common direction\[[22](https://arxiv.org/html/2609.17997#bib.bib7)\], an effect that has been analysed with time\-varying weights, multiple heads, and in the autoregressive setting\[[23](https://arxiv.org/html/2609.17997#bib.bib8),[24](https://arxiv.org/html/2609.17997#bib.bib9)\]\. For SSMs, the comparable theory is far more limited, confined to time\-invariant parameters and no normalization\[[25](https://arxiv.org/html/2609.17997#bib.bib10),[26](https://arxiv.org/html/2609.17997#bib.bib13)\], a setting too narrow to capture the models used in practice\.

In this paper, we show that, in fact, the resemblance does run deeper: it extends to the dynamical evolution of tokens across layers\. We do so by taking a dynamical systems perspective on selective SSMs: we model the evolution of tokens across the layers of the Mamba\-2 model as an ordinary differential equation, and exploit its causal cascade structure to establish consensus through an input\-to\-state stability \(ISS\) argument\[[27](https://arxiv.org/html/2609.17997#bib.bib18),[28](https://arxiv.org/html/2609.17997#bib.bib1)\]\. To the best of our knowledge, these are the first such results for SSMs with time\-varying weight matrices, the regime that faithfully reflects how their parameters vary across layers\. We then analyse the Mamba\-2 model experimentally, confirming that its core clusters the tokens and identifying the output gate as the component that prevents full consensus\.

Our contributions are threefold:

1. 1\.We derive a continuous\-time model of token evolution in Mamba\-2 capturing multi\-dimensional tokens, time\-varying weights, and layer normalization\.
2. 2\.We prove local exponential stability of the consensus equilibria, under persistence of excitation\.
3. 3\.We characterize the domain of attraction of the consensus equilibria\.

### Notations

Letr,s,ℓ∈ℕ=\{1,2,…\}r,s,\\ell\\in\\mathbb\{N\}=\\\{1,2,\\ldots\\\}\. The space ofr×sr\\times sreal matrices is denoted byℝr×s\\mathbb\{R\}^\{r\\times s\}\. In particular,𝕀r∈ℝr×r\\mathbb\{I\}\_\{r\}\\in\\mathbb\{R\}^\{r\\times r\}denotes the identity matrix\. The transpose and Frobenius norm of a matrixA∈ℝr×sA\\in\\mathbb\{R\}^\{r\\times s\}are denoted byA⊤A^\{\\top\}and‖A‖\\\|A\\\|, respectively\. Givenai∈ℝa\_\{i\}\\in\\mathbb\{R\},1≤i≤r1\\leq i\\leq r, the diagonal matrix with entriesa1,…,ara\_\{1\},\\ldots,a\_\{r\}is denoted bydiag⁡\(a1,…,ar\)∈ℝr×r\\operatorname\{diag\}\(a\_\{1\},\\ldots,a\_\{r\}\)\\in\\mathbb\{R\}^\{r\\times r\}\. Points in the Euclidean spaceℝr\\mathbb\{R\}^\{r\}are regarded as column vectors and denoted byx=\(x1,…,xr\)∈ℝr≡ℝr×1x=\(x^\{1\},\\dots,x^\{r\}\)\\in\\mathbb\{R\}^\{r\}\\equiv\\mathbb\{R\}^\{r\\times 1\}\. Tuples ofℓ\\ellpoints are denoted byX=\(x1,…,xℓ\)∈\(ℝr\)ℓX=\(x\_\{1\},\\ldots,x\_\{\\ell\}\)\\in\(\\mathbb\{R\}^\{r\}\)^\{\\ell\}\. Open intervals are denoted by\]a,b\[\]a,b\[\\,, while closed intervals are denoted by\[a,b\]\[a,b\]\. In particular, we denoteℝ0\+=\[0,∞\[\\mathbb\{R\}\_\{0\}^\{\+\}=\[0,\\infty\[andℝ\+=\]0,∞\[\\mathbb\{R\}^\{\+\}=\\,\]0,\\infty\[\\,\. The tangent space of a smooth manifoldMMatp∈Mp\\in Mand its elements are denoted byTp​MT\_\{p\}MandXp∈Tp​MX\_\{p\}\\in T\_\{p\}M, respectively\. Given another smooth manifoldNNand a smooth mapϕ:M→N\\phi:M\\to N,i\.e\.,ϕ∈C∞​\(M,N\)\\phi\\in C^\{\\infty\}\(M,N\), the corresponding tangent map is denoted byT​ϕ:T​M→T​NT\\phi:TM\\to TN\.

## IIDynamics of selective state space models

In this section, we introduce the Mamba\-2 model and derive a continuous\-time approximation of its dynamics\. Our objective is to derive a model that directly relates the input and output of each layer, allowing us to then analyse the model as a dynamical system\. In these models the input is typically considered to be a token,i\.e\., a numerical representation of a word or sub\-word, and the output sequence is used to predict the next word or sub\-word in the sequence\.

While input\-output equations for the Mamba\-2 have been derived in\[[18](https://arxiv.org/html/2609.17997#bib.bib2)\]in discrete time and in\[[25](https://arxiv.org/html/2609.17997#bib.bib10)\]in continuous time, our formulation differs from both by incorporating layer normalization, which projects the tokens onto the unit sphere𝕊n−1\\mathbb\{S\}^\{n\-1\}after each layer update\. This constraint is natural due to the use of the RMSNorm\[[29](https://arxiv.org/html/2609.17997#bib.bib21)\], the normalization used in both Mamba and Mamba\-2, which projects each token to a sphere\.

### II\-AConfiguration space

Letn∈ℕn\\in\\mathbb\{N\}and consider the Euclidean inner product onℝn\\mathbb\{R\}^\{n\},i\.e\.,⟨x1,x2⟩=x1⊤​x2\\langle x\_\{1\},x\_\{2\}\\rangle=x\_\{1\}^\{\\top\}\\,x\_\{2\}forx1,x2∈ℝnx\_\{1\},x\_\{2\}\\in\\mathbb\{R\}^\{n\}\. The corresponding norm is denoted by\|x\|=⟨x,x⟩1/2\|x\|=\\langle x,x\\rangle^\{1/2\}\. The points ofℝn\\mathbb\{R\}^\{n\}of unit norm define the\(n−1\)\(n\-1\)\-dimensional sphere:

𝕊n−1=\{y∈ℝn∣⟨y,y⟩=1\}\.\\mathbb\{S\}^\{n\-1\}=\\\{y\\in\\mathbb\{R\}^\{n\}\\mid\\langle y,y\\rangle=1\\\}\.As we consider a model consisting ofℓ\\elltokens, the resulting state space is the Cartesian product ofℓ\\ellcopies of thenn\-sphere:

\(𝕊n−1\)ℓ=𝕊n−1×…×𝕊n−1⏟ℓ​\-times\.\\displaystyle\(\\mathbb\{S\}^\{n\-1\}\)^\{\\ell\}=\\underbrace\{\\mathbb\{S\}^\{n\-1\}\\times\{\\ldots\}\\times\\mathbb\{S\}^\{n\-1\}\}\_\{\\ell\\text\{\-times\}\}\.Similarly, consider the sphere projection:

π:ℝn−\{0\}→𝕊n−1,x↦π⁡\(x\)=x​\|x\|−1\.\\pi:\\mathbb\{R\}^\{n\}\-\\\{0\\\}\\to\\mathbb\{S\}^\{n\-1\},\\quad x\\mapsto\\pi\(x\)=x\\,\|x\|^\{\-1\}\.Its tangent map at eachx∈ℝn−\{0\}x\\in\\mathbb\{R\}^\{n\}\-\\\{0\\\},Tx​π:Tx​\(ℝn−\{0\}\)→Tπ⁡\(x\)​𝕊n−1T\_\{x\}\\pi:T\_\{x\}\(\\mathbb\{R\}^\{n\}\-\\\{0\\\}\)\\to T\_\{\\pi\(x\)\}\\mathbb\{S\}^\{n\-1\}, is given by:

Tx​π⋅Xx=\|x\|−1​\(𝕀n−x​x⊤​\|x\|−2\)⋅Xx,T\_\{x\}\\pi\\cdot X\_\{x\}=\|x\|^\{\-1\}\\left\(\\mathbb\{I\}\_\{n\}\-x\\,x^\{\\top\}\\,\|x\|^\{\-2\}\\right\)\\cdot X\_\{x\},for eachXx∈Tx​\(ℝn−\{0\}\)X\_\{x\}\\in T\_\{x\}\(\\mathbb\{R\}^\{n\}\-\\\{0\\\}\)\. In particular, forz∈𝕊n−1z\\in\\mathbb\{S\}^\{n\-1\}, it readsTz​π⋅Xz=\(𝕀n−z​z⊤\)⋅XzT\_\{z\}\\pi\\cdot X\_\{z\}=\\left\(\\mathbb\{I\}\_\{n\}\-z\\,z^\{\\top\}\\right\)\\cdot X\_\{z\}\.

### II\-BThe Mamba\-2 model

Analogously to transformers, the Mamba\-2 model can be described as a sequence\-to\-sequence map: givenn∈ℕn\\in\\mathbb\{N\}, the model takes an input sequence ofℓ∈ℕ\\ell\\in\\mathbb\{N\}tokens:

Z⁡\(1\)=\(z1​\(1\),…,zℓ​\(1\)\)∈\(𝕊n−1\)ℓ,\\displaystyle Z\(1\)=\(z\_\{1\}\(1\),\\dots,z\_\{\\ell\}\(1\)\)\\in\(\\mathbb\{S\}^\{n\-1\}\)^\{\\ell\},and produces an output sequence:

Z⁡\(κ\)=\(z1​\(κ\),…,zℓ​\(κ\)\)∈\(𝕊n−1\)ℓ,\\displaystyle Z\(\\kappa\)=\(z\_\{1\}\(\\kappa\),\\dots,z\_\{\\ell\}\(\\kappa\)\)\\in\(\\mathbb\{S\}^\{n\-1\}\)^\{\\ell\},whereκ∈ℕ\\kappa\\in\\mathbb\{N\}is the depth of the model,i\.e\., the number of layers, andzi​\(k\)z\_\{i\}\(k\)denotes theii\-th token at layerk∈\{1,…,κ\}k\\in\\\{1,\\ldots,\\kappa\\\}\. Moreover, the output of each layer is dependent on its input and the layer indexkk, described asZ⁡\(k\+1\)=f⁡\(k,Z⁡\(k\)\)Z\(k\+1\)=f\(k,Z\(k\)\),1≤k≤κ−11\\leq k\\leq\\kappa\-1\. In this work, we examine the asymptotic behavior of the model,i\.e\., its evolution as the number of layers increases indefinitelyκ→∞\\kappa\\to\\infty\.

Since the Mamba\-2 model operates with a separate set of parameters for each of thenncomponents of a token, we useμ∈\{1,…,n\}\\mu\\in\\\{1,\\dots,n\\\}to index quantities that vary across components111In the machine learning literature, each componentμ\\muis commonly referred to as a*channel*\.\. Thus, the scalarziμ​\(k\)z\_\{i\}^\{\\mu\}\(k\)denotes theμ\\mu\-th entry of theii\-th token at layerkk, whereμ∈\{1,…,n\}\\mu\\in\\\{1,\\ldots,n\\\},i∈\{1,…,ℓ\}i\\in\\\{1,\\ldots,\\ell\\\}andk∈ℕk\\in\\mathbb\{N\}\.

Similarly to older models, such as Recurrent Neural Networks, the model maintains a hidden state indexed by each token component,i\.e\., a vectorhiμ​\(k\)∈ℝmh\_\{i\}^\{\\mu\}\(k\)\\in\\mathbb\{R\}^\{m\}that acts as a compressed memory of the precedingi−1i\-1tokens\. This hidden statehiμ​\(k\)h\_\{i\}^\{\\mu\}\(k\)is computed fromzi​\(k\)z\_\{i\}\(k\)andhi−1μ​\(k\)h\_\{i\-1\}^\{\\mu\}\(k\)by the recurrence relation \([1](https://arxiv.org/html/2609.17997#S2.E1)\), which is a function of the following input\-dependent matrices:

Aiμ​\(k\)=exp⁡\(−αμ​\(k\)​Δiμ​\(k\)\)​𝕀m∈ℝm×m,\\displaystyle A\_\{i\}^\{\\mu\}\(k\)=\\exp\(\-\\alpha^\{\\mu\}\(k\)\\,\\Delta^\{\\mu\}\_\{i\}\(k\)\)\\,\\mathbb\{I\}\_\{m\}\\in\\mathbb\{R\}^\{m\\times m\},Biμ​\(k\)=Δiμ​\(k\)​SB​\(k\)​zi​\(k\)∈ℝm,\\displaystyle B\_\{i\}^\{\\mu\}\(k\)=\\Delta^\{\\mu\}\_\{i\}\(k\)\\,S\_\{B\}\(k\)\\,z\_\{i\}\(k\)\\in\\mathbb\{R\}^\{m\},whereαμ​\(k\)∈ℝ\\alpha^\{\\mu\}\(k\)\\in\\mathbb\{R\}is the learnable decay rate, andΔiμ​\(k\)=softplus⁡\(WΔμ​\(k\)​zi​\(k\)\+bΔμ​\(k\)\)\\Delta\_\{i\}^\{\\mu\}\(k\)=\\operatorname\{softplus\}\(W\_\{\\Delta\}^\{\\mu\}\(k\)\\,z\_\{i\}\(k\)\+b\_\{\\Delta\}^\{\\mu\}\(k\)\), withSB​\(k\)∈ℝm×nS\_\{B\}\(k\)\\in\\mathbb\{R\}^\{m\\times n\},WΔμ​\(k\)∈ℝ1×nW\_\{\\Delta\}^\{\\mu\}\(k\)\\in\\mathbb\{R\}^\{1\\times n\}andbΔμ​\(k\)∈ℝb\_\{\\Delta\}^\{\\mu\}\(k\)\\in\\mathbb\{R\}consisting of trainable weights\. Recall thatsoftplus⁡\(a\)=ln⁡\(exp⁡\(a\)\+1\)\\operatorname\{softplus\}\(a\)=\\ln\(\\exp\(a\)\+1\)for eacha∈ℝa\\in\\mathbb\{R\}\.

The recurrence equations for the Mamba\-2 consist of two coupled updates operating along different axes: one propagates the hidden statehiμh^\{\\mu\}\_\{i\}along the sequence \(indexed byii\), and the other propagates the tokenziz\_\{i\}across layers \(indexed bykk\)\. The hidden state evolves independently along each componentμ\\mu:

hiμ​\(k\)=Aiμ​\(k\)​hi−1μ​\(k\)\+Biμ​\(k\)​ziμ​\(k\),\\displaystyle\\boxed\{h\_\{i\}^\{\\mu\}\(k\)=A\_\{i\}^\{\\mu\}\(k\)\\,h\_\{i\-1\}^\{\\mu\}\(k\)\+B\_\{i\}^\{\\mu\}\(k\)\\,z\_\{i\}^\{\\mu\}\(k\),\}\(1\)for eachμ∈\{1,…,n\}\\mu\\in\\\{1,\\ldots,n\\\},i∈\{1,…,ℓ\}i\\in\\\{1,\\ldots,\\ell\\\}andk∈ℕk\\in\\mathbb\{N\}, whereh0μ​\(k\)=0h\_\{0\}^\{\\mu\}\(k\)=0\. The token update then assembles all components and projects onto the sphere:

zi​\(k\+1\)=π⁡\(zi​\(k\)\+τ​ri​\(k\)\),\\displaystyle\\boxed\{z\_\{i\}\(k\+1\)=\\pi\\,\\big\(z\_\{i\}\(k\)\+\\tau\\,r\_\{i\}\(k\)\\big\),\}\(2\)whereri​\(k\)∈ℝnr\_\{i\}\(k\)\\in\\mathbb\{R\}^\{n\}is given byriμ​\(k\)=zi⊤​\(k\)​SC⊤​\(k\)​hiμ​\(k\)r\_\{i\}^\{\\mu\}\(k\)=z\_\{i\}^\{\\top\}\(k\)\\,S\_\{C\}^\{\\top\}\(k\)\\,h\_\{i\}^\{\\mu\}\(k\),μ∈\{1,…,n\}\\mu\\in\\\{1,\\ldots,n\\\}, withSC​\(k\)∈ℝm×nS\_\{C\}\(k\)\\in\\mathbb\{R\}^\{m\\times n\}consisting of trainable weights, andτ∈ℝ\+\\tau\\in\\mathbb\{R\}^\{\+\}being a small scale parameter\.

Note that the first equation propagates the hidden statehiμ​\(k\)h\_\{i\}^\{\\mu\}\(k\)along the sequence: for a fixed layerkk, it accumulates a compressed representation of tokens11throughii, withAiμ​\(k\)A\_\{i\}^\{\\mu\}\(k\)controlling how past information decays andBiμ​\(k\)B\_\{i\}^\{\\mu\}\(k\)injecting information about the current token\. The second equation propagates the tokens across layers: it updateszi​\(k\)z\_\{i\}\(k\)tozi​\(k\+1\)z\_\{i\}\(k\+1\)by reading out from the hidden state via the output matrixCi​\(k\)=zi⊤​\(k\)​SC⊤​\(k\)C\_\{i\}\(k\)=z\_\{i\}^\{\\top\}\(k\)\\,S\_\{C\}^\{\\top\}\(k\), adding the skip connection, and normalizing onto𝕊n−1\\mathbb\{S\}^\{n\-1\}\. The parameterτ∈ℝ\+\\tau\\in\\mathbb\{R\}^\{\+\}is a small training weight that is usually absorbed into the output matrix, as both are learned during training\. Here we keepτ\\tauexplicit, as it plays the role of a step size in the derivation of the continuous\-time model in section[II\-D](https://arxiv.org/html/2609.17997#S2.SS4)\.

The model presented so far excludes two nonlinearities of the Mamba\-2 architecture, which we now make explicit\. The first is the output gate\. In the architecture, the readoutri​\(k\)r\_\{i\}\(k\)in \([2](https://arxiv.org/html/2609.17997#S2.E2)\) is multiplied elementwise by a gateg⁡\(zi​\(k\)\)∈ℝng\(z\_\{i\}\(k\)\)\\in\\mathbb\{R\}^\{n\}before the skip connection and normalization, so that the update readszi​\(k\+1\)=π⁡\(zi​\(k\)\+τ​g​\(zi​\(k\)\)⊙ri​\(k\)\),z\_\{i\}\(k\+1\)=\\pi\\big\(z\_\{i\}\(k\)\+\\tau\\,g\(z\_\{i\}\(k\)\)\\odot r\_\{i\}\(k\)\\big\),where⊙\\odotdenotes elementwise multiplication\. Similarly to\[[25](https://arxiv.org/html/2609.17997#bib.bib10)\], we exclude the gate in order to isolate the recurrence, which is the component through which the tokens interact and the one that plays the role of attention\. As we show in Section[VI](https://arxiv.org/html/2609.17997#S6), the gate is the main component that attenuates the consensus induced by the recurrence\.

The second nonlinearity concerns the matricesBiμ​\(k\)B\_\{i\}^\{\\mu\}\(k\)andCi​\(k\)C\_\{i\}\(k\)defined in Section[II\-B](https://arxiv.org/html/2609.17997#S2.SS2)\. Variants of the architecture differ in this respect, and one common choice, present in the Mamba\-2 model considered in Section[VI](https://arxiv.org/html/2609.17997#S6), is to pass the branches producingBBandCCthrough a SiLU nonlinearity\. In this case,SB​\(k\)​zi​\(k\)S\_\{B\}\(k\)z\_\{i\}\(k\)andSC​\(k\)​zi​\(k\)S\_\{C\}\(k\)z\_\{i\}\(k\)are replaced byς⁡\(SB​\(k\)​zi​\(k\)\)\\varsigma\(S\_\{B\}\(k\)z\_\{i\}\(k\)\)andς⁡\(SC​\(k\)​zi​\(k\)\)\\varsigma\(S\_\{C\}\(k\)z\_\{i\}\(k\)\), respectively, whereς\\varsigmadenotes the elementwise SiLU, i\.e\.,ς​\(x\)μ=xμ​\(1\+e−xμ\)−1,\\varsigma\(x\)^\{\\mu\}=x^\{\\mu\}\(1\+e^\{\-x^\{\\mu\}\}\)^\{\-1\},for eachμ∈\{1,…,n\}\\mu\\in\\\{1,\\dots,n\\\}\. Since the particular nonlinearity used depends on the specific architecture considered, we exclude it from the model and address its effect separately in Lemma[IV\.1](https://arxiv.org/html/2609.17997#S4.Thmlemma1)\.

### II\-CDiscrete time input\-output model

The recurrence presented in the previous section is written per\-component and depends on the hidden state, which prevents a direct analysis of the full token dynamics\. To obtain a closed\-form expression for the token update, we unroll the hidden\-state recurrence\. To this end, we require some prior definitions that simplify the notation\.

#### Notations

For eachi,j∈\{1,…,ℓ\}i,j\\in\\\{1,\\ldots,\\ell\\\}withj≤ij\\leq iandk∈ℕk\\in\\mathbb\{N\}, letDi​j​\(k\)∈ℝn×nD\_\{ij\}\(k\)\\in\\mathbb\{R\}^\{n\\times n\}be the diagonal matrix:

Di​j​\(k\)=diag⁡\(λi​j1​\(k\),…,λi​jn​\(k\)\),\\displaystyle D\_\{ij\}\(k\)=\\operatorname\{diag\}\\\!\\left\(\\lambda\_\{ij\}^\{1\}\(k\),\\dots,\\lambda\_\{ij\}^\{n\}\(k\)\\right\),where:

⊳\\displaystyle\\triangleright\\quadλi​jμ​\(k\)=Δjμ​\(k\)​exp⁡\(di​jμ​\(k\)\),\\displaystyle\\lambda\_\{ij\}^\{\\mu\}\(k\)=\\Delta\_\{j\}^\{\\mu\}\(k\)\\,\\exp\(d\_\{ij\}^\{\\mu\}\(k\)\),⊳\\displaystyle\\triangleright\\quaddi​jμ\(k\)=−\(1−δi​j\)αμ\(k\)∑l=j\+1iΔlμ\(k\),\\displaystyle d\_\{ij\}^\{\\mu\}\(k\)=\-\(1\-\\delta\_\{ij\}\)\\,\\alpha^\{\\mu\}\(k\)\\sum\_\{l=j\+1\}^\{i\}\\Delta\_\{l\}^\{\\mu\}\(k\),withδi​j\\delta\_\{ij\}denoting the Kronecker’s delta\. The eigenvalues ofDi​j​\(k\)D\_\{ij\}\(k\)areλi​jμ​\(k\)∈ℝ\+\\lambda\_\{ij\}^\{\\mu\}\(k\)\\in\\mathbb\{R\}^\{\+\},1≤μ≤n1\\leq\\mu\\leq n, with associated unit eigenvectors:

𝔢μ=\(0,…,0,1⏟μ​\-th,0,…,0\)∈ℝn\.\\displaystyle\\mathfrak\{e\}^\{\\mu\}=\(0,\\dots,0,\\underbrace\{1\}\_\{\\mu\\text\{\-th\}\},0,\\dots,0\)\\in\\mathbb\{R\}^\{n\}\.We also define the*interaction kernel*as:

SB​C​\(k\)=SC⊤​\(k\)​SB​\(k\)∈ℝn×n\.\\displaystyle S\_\{BC\}\(k\)=S\_\{C\}^\{\\top\}\(k\)\\,S\_\{B\}\(k\)\\in\\mathbb\{R\}^\{n\\times n\}\.

#### Input\-output equation

With these definitions in hand, we eliminate the hidden state from \([1](https://arxiv.org/html/2609.17997#S2.E1)\) and \([2](https://arxiv.org/html/2609.17997#S2.E2)\), and express the token update in a closed\-form function of the tokens alone\.

###### Lemma II\.1\(Input\-output formulation of Mamba\-2 model\)

A sequenceZ⁡\(k\)=\(z1​\(k\),…,zℓ​\(k\)\)∈\(𝕊n−1\)ℓZ\(k\)=\(z\_\{1\}\(k\),\\ldots,z\_\{\\ell\}\(k\)\)\\in\(\\mathbb\{S\}^\{n\-1\}\)^\{\\ell\},k∈ℕk\\in\\mathbb\{N\}, satisfies \([1](https://arxiv.org/html/2609.17997#S2.E1)\) and \([2](https://arxiv.org/html/2609.17997#S2.E2)\), for some hidden states, if and only if it satisfies the following recurrence equations:

zi​\(k\+1\)=π⁡\(zi​\(k\)\+τ​∑j=1iβi​j​\(k\)​zj​\(k\)\),\\displaystyle\\boxed\{z\_\{i\}\(k\+1\)=\\pi\\bigg\(z\_\{i\}\(k\)\+\\tau\\sum\_\{j=1\}^\{i\}\\beta\_\{ij\}\(k\)\\,z\_\{j\}\(k\)\\bigg\),\}for eachi∈\{1,…,ℓ\}i\\in\\\{1,\\ldots,\\ell\\\}andk∈ℕk\\in\\mathbb\{N\}, where:

βi​j​\(k\)=\(zi⊤​\(k\)​SB​C​\(k\)​zj​\(k\)\)​Di​j​\(k\)\.\\displaystyle\\beta\_\{ij\}\(k\)=\\left\(z\_\{i\}^\{\\top\}\(k\)\\,S\_\{BC\}\(k\)\\,z\_\{j\}\(k\)\\right\)D\_\{ij\}\(k\)\.

###### Proof:

For eachi,j∈\{1,…,ℓ\}i,j\\in\\\{1,\\ldots,\\ell\\\}withj≤ij\\leq i, a straightforward computation yields:

∏l=j\+1iAlμ\\displaystyle\\prod\_\{l=j\+1\}^\{i\}A\_\{l\}^\{\\mu\}=∏l=j\+1iexp⁡\(−αμ​Δlμ​𝕀m\)\\displaystyle=\\prod\_\{l=j\+1\}^\{i\}\\exp\(\-\\alpha^\{\\mu\}\\,\\Delta\_\{l\}^\{\\mu\}\\,\\mathbb\{I\}\_\{m\}\)=∏l=j\+1iexp⁡\(−αμ​Δlμ\)​𝕀m\\displaystyle=\\prod\_\{l=j\+1\}^\{i\}\\exp\(\-\\alpha^\{\\mu\}\\,\\Delta\_\{l\}^\{\\mu\}\)\\,\\mathbb\{I\}\_\{m\}=exp\(−αμ∑l=j\+1iΔlμ\)𝕀m=exp\(di​jμ\)𝕀m,\\displaystyle=\\exp\\left\(\-\\alpha^\{\\mu\}\\sum\_\{l=j\+1\}^\{i\}\\Delta\_\{l\}^\{\\mu\}\\right\)\\mathbb\{I\}\_\{m\}=\\exp\(d\_\{ij\}^\{\\mu\}\)\\,\\mathbb\{I\}\_\{m\},where we omitted the layer indexk∈ℕk\\in\\mathbb\{N\}for brevity\. Hence, sinceh0μ=0h\_\{0\}^\{\\mu\}=0, by unrolling \([1](https://arxiv.org/html/2609.17997#S2.E1)\) we obtain:

hiμ\\displaystyle h\_\{i\}^\{\\mu\}=∑j=1i−1\(∏l=j\+1iAlμ\)​Bjμ​zjμ\+Biμ​ziμ=∑j=1iexp⁡\(di​jμ\)​Bjμ​zjμ,\\displaystyle=\\sum\_\{j=1\}^\{i\-1\}\\left\(\\prod\_\{l=j\+1\}^\{i\}A\_\{l\}^\{\\mu\}\\right\)B\_\{j\}^\{\\mu\}\\,z\_\{j\}^\{\\mu\}\+B\_\{i\}^\{\\mu\}\\,z\_\{i\}^\{\\mu\}=\\sum\_\{j=1\}^\{i\}\\exp\(d\_\{ij\}^\{\\mu\}\)B\_\{j\}^\{\\mu\}\\,z\_\{j\}^\{\\mu\},\(3\)where we used thatexp⁡\(di​jμ\)=1\\exp\(d\_\{ij\}^\{\\mu\}\)=1wheni=ji=j\. On the other hand, for each1≤μ≤n1\\leq\\mu\\leq n, we have:

Ci​exp⁡\(di​jμ\)​Bjμ​zjμ\\displaystyle C\_\{i\}\\,\\exp\(d\_\{ij\}^\{\\mu\}\)\\,B\_\{j\}^\{\\mu\}\\,z\_\{j\}^\{\\mu\}=zi⊤​SC⊤​exp⁡\(di​jμ\)​Δjμ​SB​zj​zjμ\\displaystyle=z\_\{i\}^\{\\top\}\\,S\_\{C\}^\{\\top\}\\,\\exp\(d\_\{ij\}^\{\\mu\}\)\\,\\Delta\_\{j\}^\{\\mu\}\\,S\_\{B\}\\,z\_\{j\}\\,z\_\{j\}^\{\\mu\}=\(zi⊤​SC⊤​SB​zj\)​Δjμ​exp⁡\(di​jμ\)​zjμ\\displaystyle=\\left\(z\_\{i\}^\{\\top\}\\,S\_\{C\}^\{\\top\}\\,S\_\{B\}\\,z\_\{j\}\\right\)\\Delta\_\{j\}^\{\\mu\}\\,\\exp\(d\_\{ij\}^\{\\mu\}\)\\,z\_\{j\}^\{\\mu\}\\,=\(zi⊤​SB​C​zj\)​Δjμ​exp⁡\(di​jμ\)​zjμ\.\\displaystyle=\\left\(z\_\{i\}^\{\\top\}\\,S\_\{BC\}\\,z\_\{j\}\\right\)\\Delta\_\{j\}^\{\\mu\}\\,\\exp\(d\_\{ij\}^\{\\mu\}\)\\,z\_\{j\}^\{\\mu\}\.\(4\)
By substituting \([3](https://arxiv.org/html/2609.17997#S2.E3)\) into \([2](https://arxiv.org/html/2609.17997#S2.E2)\) and taking \([4](https://arxiv.org/html/2609.17997#S2.E4)\) into account, we obtain:

Ci​\(k\)​hiμ​\(k\)\\displaystyle C\_\{i\}\(k\)\\,h\_\{i\}^\{\\mu\}\(k\)=Ci​\(k\)​∑j=1iexp⁡\(di​jμ​\(k\)\)​Bjμ​\(k\)​zjμ​\(k\)\\displaystyle=C\_\{i\}\(k\)\\sum\_\{j=1\}^\{i\}\\exp\(d\_\{ij\}^\{\\mu\}\(k\)\)B\_\{j\}^\{\\mu\}\(k\)\\,z\_\{j\}^\{\\mu\}\(k\)=∑j=1izi⊤​\(k\)​SB​C​\(k\)​zj​\(k\)​Δjμ​exp⁡\(di​jμ​\(k\)\)​zjμ\.\\displaystyle=\\sum\_\{j=1\}^\{i\}z\_\{i\}^\{\\top\}\(k\)\\,S\_\{BC\}\(k\)\\,z\_\{j\}\(k\)\\,\\Delta\_\{j\}^\{\\mu\}\\,\\exp\(d\_\{ij\}^\{\\mu\}\(k\)\)\\,z\_\{j\}^\{\\mu\}\.The result now follows from \([2](https://arxiv.org/html/2609.17997#S2.E2)\) as well as the definition ofDi​j​\(k\)D\_\{ij\}\(k\)\. ∎

### II\-DContinuous\-time state space model

The next objective is to derive a continuous\-time counterpart of the discrete model in Lemma[II\.1](https://arxiv.org/html/2609.17997#S2.Thmlemma1), which will allow us to analyze the dynamics of the Mamba\-2 model with classical control\-theoretic tools\. To this end, we recall how continuous\-time models are obtained from discrete\-time ones on manifolds\.

For a compact and connected Riemannian manifold\(M,g\)\(M,g\), let𝐝g:M×M→ℝ0\+\\mathbf\{d\}\_\{g\}:M\\times M\\to\\mathbb\{R\}\_\{0\}^\{\+\}denote the induced geodesic distance\. The flow of a vector fieldf∈𝔛⁡\(M\)f\\in\\mathfrak\{X\}\(M\)is denoted byfτ:M→Mf^\{\\tau\}:M\\to M,τ∈ℝ0\+\\tau\\in\\mathbb\{R\}\_\{0\}^\{\+\}\. Recall that all vector fields on a compact manifold are \(forward\) complete\. A mapϕ:M×ℝ→M\\phi:M\\times\\mathbb\{R\}\\to Mis a first order approximation offτ​\(Z\)f^\{\\tau\}\(Z\)if there existT∈ℝ\+T\\in\\mathbb\{R\}^\{\+\}andσ:M→ℝ0\+\\sigma:M\\to\\mathbb\{R\}\_\{0\}^\{\+\}such that𝐝g​\(fτ​\(Z\),ϕ⁡\(Z,τ\)\)≤σ⁡\(Z\)​τ2\\mathbf\{d\}\_\{g\}\(f^\{\\tau\}\(Z\),\\phi\(Z,\\tau\)\)\\leq\\sigma\(Z\)\\,\\tau^\{2\}for eachτ∈\[0,T\]\\tau\\in\[0,T\]andZ∈MZ\\in M\.

LetM=\(𝕊n−1\)ℓM=\(\\mathbb\{S\}^\{n\-1\}\)^\{\\ell\}be equipped with the Riemannian metric induced by the Euclidean metric on\(ℝn\)ℓ\(\\mathbb\{R\}^\{n\}\)^\{\\ell\}\. Our objective is to construct a vector fieldffon\(𝕊n−1\)ℓ\(\\mathbb\{S\}^\{n\-1\}\)^\{\\ell\}whose flow best approximates the discrete recurrence in Lemma[II\.1](https://arxiv.org/html/2609.17997#S2.Thmlemma1)\. Given the discrete update on Lemma[II\.1](https://arxiv.org/html/2609.17997#S2.Thmlemma1), the best first\-order approximation inτ\\tauyields:

z˙i\\displaystyle\\dot\{z\}\_\{i\}=dd​τ\|τ=0​π​\(zi\+τ​∑j=1iβi​j​zj\)=Tzi​π⋅∑j=1iβi​j​zj\.\\displaystyle=\\left\.\\frac\{d\}\{d\\tau\}\\right\|\_\{\\tau=0\}\\pi\\left\(z\_\{i\}\+\\tau\\,\\sum\_\{j=1\}^\{i\}\\beta\_\{ij\}\\,z\_\{j\}\\right\)=T\_\{z\_\{i\}\}\\pi\\cdot\\sum\_\{j=1\}^\{i\}\\beta\_\{ij\}\\,z\_\{j\}\.By replacing the layer indexk∈ℕk\\in\\mathbb\{N\}by a continuous variablet∈ℝ0\+t\\in\\mathbb\{R\}\_\{0\}^\{\+\}, which may be regarded as time, the continuous\-time SSM dynamics reads:

z˙i=Tziπ⋅∑j=1iβi​j\(t\)zj,1≤i≤ℓ,\\displaystyle\\boxed\{\\dot\{z\}\_\{i\}=T\_\{z\_\{i\}\}\\pi\\cdot\\sum\_\{j=1\}^\{i\}\\beta\_\{ij\}\(t\)\\,z\_\{j\},\\quad 1\\leq i\\leq\\ell,\}\(5\)forZ=\(z1,…,zℓ\)∈\(𝕊n−1\)ℓZ=\(z\_\{1\},\\ldots,z\_\{\\ell\}\)\\in\(\\mathbb\{S\}^\{n\-1\}\)^\{\\ell\}, with:

βi​j​\(t\)\\displaystyle\\beta\_\{ij\}\(t\)=\(zi⊤​SB​C​\(t\)​zj\)​Di​j​\(t\)\.\\displaystyle=\\left\(z\_\{i\}^\{\\top\}\\,S\_\{BC\}\(t\)\\,z\_\{j\}\\right\)D\_\{ij\}\(t\)\.
The weight matrices, together with all quantities defined from them, are now functions of time:

αμ:ℝ0\+→ℝ,\\displaystyle\\alpha^\{\\mu\}:\\mathbb\{R\}\_\{0\}^\{\+\}\\to\\mathbb\{R\},SB,SC:ℝ0\+→ℝm×n,\\displaystyle S\_\{B\},~S\_\{C\}:\\mathbb\{R\}\_\{0\}^\{\+\}\\to\\mathbb\{R\}^\{m\\times n\},WΔμ:ℝ0\+→ℝ1×n,\\displaystyle W\_\{\\Delta\}^\{\\mu\}:\\mathbb\{R\}\_\{0\}^\{\+\}\\to\\mathbb\{R\}^\{1\\times n\},bΔμ:ℝ0\+→ℝ\.\\displaystyle b\_\{\\Delta\}^\{\\mu\}:\\mathbb\{R\}\_\{0\}^\{\+\}\\to\\mathbb\{R\}\.
Note that, by compactness of𝕊n−1\\mathbb\{S\}^\{n\-1\}, the continuous SSM dynamics is forward complete,i\.e\., solutions are defined for all positive times under mild regularity assumptions on the parameter functions\.

## IIIAssumptions on the parameters

Here we introduce the working assumptions on the parameters of the model\. Roughly speaking, they are boundedness of the parameters, uniqueness of the principal eigenvalue ofDi​jD\_\{ij\}and strict positivity of the spectral gap\.

The*spectral gap*is the smallest distance from the principal eigenvalue ofDi​jD\_\{ij\}to the rest of them:

γi​j=inf\(t,Z\)∈ℝ0\+×\(𝕊n−1\)ℓminμ≠μ~⁡\(λi​jμ~​\(t,Z\)−λi​jμ​\(t,Z\)\),\\displaystyle\\gamma\_\{ij\}=\\inf\_\{\(t,Z\)\\in\\mathbb\{R\}\_\{0\}^\{\+\}\\times\(\\mathbb\{S\}^\{n\-1\}\)^\{\\ell\}\}~\\min\_\{\\mu\\neq\\tilde\{\\mu\}\}\\left\(\\lambda\_\{ij\}^\{\\tilde\{\\mu\}\}\(t,Z\)\-\\lambda\_\{ij\}^\{\\mu\}\(t,Z\)\\right\),for eachi,j∈\{1,…,ℓ\}i,j\\in\\\{1,\\ldots,\\ell\\\}withj≤ij\\leq i, whereλi​jμ~​\(t,Z\)\\lambda\_\{ij\}^\{\\tilde\{\\mu\}\}\(t,Z\)is the largest eigenvalue ofDi​j​\(t,Z\)D\_\{ij\}\(t,Z\)for each\(t,Z\)∈ℝ0\+×\(𝕊n−1\)ℓ\(t,Z\)\\in\\mathbb\{R\}\_\{0\}^\{\+\}\\times\(\\mathbb\{S\}^\{n\-1\}\)^\{\\ell\}\. Note thatγi​j∈ℝ0\+\\gamma\_\{ij\}\\in\\mathbb\{R\}\_\{0\}^\{\+\}\. In general,μ~\\tilde\{\\mu\}may depend on the time and state,\(t,Z\)\(t,Z\)\. Our assumption below says that, in fact, it is independent of them\.

###### Assumption III\.1

The weight matrices of the continuous model \([5](https://arxiv.org/html/2609.17997#S2.E5)\) satisfy:

1. 1\.There existsμ~∈\{1,…,n\}\\tilde\{\\mu\}\\in\\\{1,\\ldots,n\\\}such that, for each\(t,Z\)∈ℝ0\+×\(𝕊n−1\)ℓ\(t,Z\)\\in\\mathbb\{R\}\_\{0\}^\{\+\}\\times\(\\mathbb\{S\}^\{n\-1\}\)^\{\\ell\}andi,j∈\{1,…,ℓ\}i,j\\in\\\{1,\\ldots,\\ell\\\}withj≤ij\\leq i, the principal eigenvalue ofDi​j​\(t,Z\)D\_\{ij\}\(t,Z\)isλi​jμ~​\(t,Z\)\\lambda\_\{ij\}^\{\\tilde\{\\mu\}\}\(t,Z\)\.
2. 2\.γ=min1≤j≤i≤ℓ⁡γi​j∈ℝ\+\\gamma=\\min\_\{1\\leq j\\leq i\\leq\\ell\}~\\gamma\_\{ij\}\\in\\mathbb\{R\}^\{\+\}\.
3. 3\.max⁡supt∈ℝ0\+1≤μ≤n⁡\|αμ​\(t\)\|∈ℝ\+\\max\_\{1\\leq\\mu\\leq n\}~\\sup\_\{t\\in\\mathbb\{R\}\_\{0\}^\{\+\}\}\|\\alpha^\{\\mu\}\(t\)\|\\in\\mathbb\{R\}^\{\+\}\.
4. 4\.max⁡supt∈ℝ0\+1≤μ≤n⁡‖WΔμ​\(t\)‖∈ℝ\+\\max\_\{1\\leq\\mu\\leq n\}~\\sup\_\{t\\in\\mathbb\{R\}\_\{0\}^\{\+\}\}\\\|W\_\{\\Delta\}^\{\\mu\}\(t\)\\\|\\in\\mathbb\{R\}^\{\+\}\.
5. 5\.max⁡supt∈ℝ0\+1≤μ≤n⁡‖bΔμ​\(t\)‖∈ℝ\+\\max\_\{1\\leq\\mu\\leq n\}~\\sup\_\{t\\in\\mathbb\{R\}\_\{0\}^\{\+\}\}\\\|b\_\{\\Delta\}^\{\\mu\}\(t\)\\\|\\in\\mathbb\{R\}^\{\+\}\.
6. 6\.min⁡inft∈ℝ0\+1≤μ≤n⁡‖bΔμ​\(t\)‖∈ℝ\+\\min\_\{1\\leq\\mu\\leq n\}~\\inf\_\{t\\in\\mathbb\{R\}\_\{0\}^\{\+\}\}\\\|b\_\{\\Delta\}^\{\\mu\}\(t\)\\\|\\in\\mathbb\{R\}^\{\+\}\.
7. 7\.supt∈ℝ0\+‖SB​C​\(t\)‖∈ℝ\+\\sup\_\{t\\in\\mathbb\{R\}\_\{0\}^\{\+\}\}\\\|S\_\{BC\}\(t\)\\\|\\in\\mathbb\{R\}^\{\+\}\.

Compactness of the sphere, combined with continuity and monotonicity of the exponential and softplus functions, together with the bounds on the weightsWΔμW\_\{\\Delta\}^\{\\mu\},bΔμb\_\{\\Delta\}^\{\\mu\}andαμ\\alpha^\{\\mu\}, lead to bounds on the matricesDi​jD\_\{ij\}\.

###### Lemma III\.1

Under Assumption[III\.1](https://arxiv.org/html/2609.17997#S3.Thmassumption1), for eachi,j∈\{1,…,ℓ\}i,j\\in\\\{1,\\ldots,\\ell\\\}withj≤ij\\leq iand1≤μ≤n1\\leq\\mu\\leq n, we have:

sup\(t,Z\)∈ℝ0\+×\(𝕊n−1\)ℓλi​jμ​\(t,Z\)∈ℝ\+,\\displaystyle\\sup\_\{\(t,Z\)\\in\\mathbb\{R\}\_\{0\}^\{\+\}\\times\(\\mathbb\{S\}^\{n\-1\}\)^\{\\ell\}\}~\\lambda\_\{ij\}^\{\\mu\}\(t,Z\)\\in\\mathbb\{R\}^\{\+\},inf\(t,Z\)∈ℝ0\+×\(𝕊n−1\)ℓλi​jμ​\(t,Z\)∈ℝ\+\.\\displaystyle\\inf\_\{\(t,Z\)\\in\\mathbb\{R\}\_\{0\}^\{\+\}\\times\(\\mathbb\{S\}^\{n\-1\}\)^\{\\ell\}\}~\\lambda\_\{ij\}^\{\\mu\}\(t,Z\)\\in\\mathbb\{R\}^\{\+\}\.In particular,sup\(t,Z\)∈ℝ0\+×𝕊n−1‖Di​j​\(t,Z\)‖∈ℝ\+\\sup\_\{\(t,Z\)\\in\\mathbb\{R\}\_\{0\}^\{\+\}\\times\\mathbb\{S\}^\{n\-1\}\}\\\|D\_\{ij\}\(t,Z\)\\\|\\in\\mathbb\{R\}^\{\+\}\.

To conclude this section, we introduce three sets that will be used to study the asymptotic stability of the consensus equilibria and their domains of attraction\.

###### Definition III\.1

For eachμ∈\{1,…,n\}\\mu\\in\\\{1,\\dots,n\\\}, we define:

1. 1\.The*consensus set*: 𝒞ℓ\(μ\)=⋃Σ∈\{−1,\+1\}ℓ\{\(σ1𝔢μ,…,σℓ𝔢μ\)∈\(𝕊n−1\)ℓ\},\\displaystyle\\mathcal\{C\}\_\{\\ell\}\(\\mu\)=\\bigcup\_\{\\Sigma\\in\\\{\-1,\+1\\\}^\{\\ell\}\}\\left\\\{\(\\sigma\_\{1\}\\,\\mathfrak\{e\}^\{\\mu\},\\ldots,\\sigma\_\{\\ell\}\\,\\mathfrak\{e\}^\{\\mu\}\)\\in\(\\mathbb\{S\}^\{n\-1\}\)^\{\\ell\}\\right\\\},whereΣ=\(σ1,…,σℓ\)\\Sigma=\(\\sigma\_\{1\},\\dots,\\sigma\_\{\\ell\}\)\.
2. 2\.The*spherical cap*: Ωcσ​\(μ\)=\{z∈𝕊n−1∣σ​z⊤​𝔢μ=σ​zμ\>c\},\\displaystyle\\Omega\_\{c\}^\{\\sigma\}\(\\mu\)=\\\{z\\in\\mathbb\{S\}^\{n\-1\}\\mid\\sigma\\,z^\{\\top\}\\mathfrak\{e\}^\{\\mu\}=\\sigma\\,z^\{\\mu\}\>c\\\},wherec∈\[0,1\[c\\in\[0,1\[andσ∈\{−1,\+1\}\\sigma\\in\\\{\-1,\+1\\\}\.
3. 3\.The*equator*:ℰ⁡\(μ\)=\{z∈𝕊n−1∣z⊤​𝔢μ=0\}\\mathcal\{E\}\(\\mu\)=\\\{z\\in\\mathbb\{S\}^\{n\-1\}\\mid z^\{\\top\}\\mathfrak\{e\}^\{\\mu\}=0\\\}\.

The consensus set𝒞ℓ​\(μ\)\\mathcal\{C\}\_\{\\ell\}\(\\mu\)consists of2ℓ2^\{\\ell\}isolated points in\(𝕊n−1\)ℓ\(\\mathbb\{S\}^\{n\-1\}\)^\{\\ell\}, each corresponding to a sign patternΣ∈\{−1,\+1\}ℓ\\Sigma\\in\\\{\-1,\+1\\\}^\{\\ell\}\. At any consensus equilibrium, every token is aligned with the eigenvector𝔢μ\\mathfrak\{e\}^\{\\mu\}, up to sign: tokens need not all point in the same direction, but each must lie at one of the two poles222Here we use the term pole to refer to the antipodal pair±𝔢μ\\pm\\mathfrak\{e\}^\{\\mu\}, that is, the two points where the axis spanned by𝔢μ\\mathfrak\{e\}^\{\\mu\}meets𝕊n−1\\mathbb\{S\}^\{n\-1\}\.±𝔢μ\\pm\\mathfrak\{e\}^\{\\mu\}\. In Section[IV](https://arxiv.org/html/2609.17997#S4), we prove that each of these equilibria is locally exponentially stable whenμ=μ~\\mu=\\tilde\{\\mu\}corresponds to the principal eigenvector ofDi​jD\_\{ij\}\. For comparison, analogous results for transformers\[[23](https://arxiv.org/html/2609.17997#bib.bib8),[22](https://arxiv.org/html/2609.17997#bib.bib7)\]typically establish attractivity or asymptotic stability on an open hemisphere\. Hence, in that context consensus is understood as all tokens pointing to the same direction\. To distinguish both scenarios, the consensus points in𝒞ℓ​\(μ\)\\mathcal\{C\}\_\{\\ell\}\(\\mu\)are sometimes called*bipartite consensus*, but we will avoid such nomenclature\.

## IVPersistency of Excitation of the interaction kernel

In this section, we prove local exponential stability of the consensus equilibria under Persistency of Excitation \(PE\)\. The remaining cascade terms are handled by an ISS argument and a recursive choice of sufficiently small caps\.

Following Assumption[III\.1](https://arxiv.org/html/2609.17997#S3.Thmassumption1), for eachi,j∈\{1,…,ℓ\}i,j\\in\\\{1,\\ldots,\\ell\\\}withj≤ij\\leq i, we define:

Γi​j\\displaystyle\\Gamma\_\{ij\}=sup\(t,Z\)∈ℝ0\+×\(𝕊n−1\)ℓmaxμ≠μ~⁡\(λi​jμ~​\(t,Z\)−λi​jμ​\(t,Z\)\),\\displaystyle=\\sup\_\{\(t,Z\)\\in\\mathbb\{R\}\_\{0\}^\{\+\}\\times\(\\mathbb\{S\}^\{n\-1\}\)^\{\\ell\}\}~\\max\_\{\\mu\\neq\\tilde\{\\mu\}\}\\bigl\(\\lambda\_\{ij\}^\{\\tilde\{\\mu\}\}\(t,Z\)\-\\lambda\_\{ij\}^\{\\mu\}\(t,Z\)\\bigr\),Note thatΓi​j∈ℝ\+\\Gamma\_\{ij\}\\in\\mathbb\{R\}^\{\+\}thanks to Lemma[III\.1](https://arxiv.org/html/2609.17997#S3.Thmlemma1)\. This allows for introducing the following parameters:

Γ\\displaystyle\\Gamma=max1≤j≤i≤ℓ⁡Γi​j∈ℝ\+,\\displaystyle=\\max\_\{1\\leq j\\leq i\\leq\\ell\}\\Gamma\_\{ij\}\\in\\mathbb\{R\}^\{\+\},Λ\\displaystyle\\Lambda=max2≤i≤ℓsup\(t,Z\)∈ℝ0\+×\(𝕊n−1\)ℓ∑j=1i−1λi​jμ~\(t,Z\)∈ℝ\+\.\\displaystyle=\\max\_\{2\\leq i\\leq\\ell\}\\,\\sup\_\{\(t,Z\)\\in\\mathbb\{R\}\_\{0\}^\{\+\}\\times\(\\mathbb\{S\}^\{n\-1\}\)^\{\\ell\}\}\\,\\sum\_\{j=1\}^\{i\-1\}\\lambda\_\{ij\}^\{\\tilde\{\\mu\}\}\(t,Z\)\\in\\mathbb\{R\}^\{\+\}\.
Letc∈\]0,1\[c\\in\\,\]0,1\[\. For each\(t,z\)∈ℝ0\+×Ωcσ​\(μ~\)\(t,z\)\\in\\mathbb\{R\}\_\{0\}^\{\+\}\\times\\Omega\_\{c\}^\{\\sigma\}\(\\tilde\{\\mu\}\), whereσ∈\{−1,\+1\}\\sigma\\in\\\{\-1,\+1\\\}, we define:

ac​\(t,z\)=\{γ​c​\(1\+c\)​\(z⊤​SB​C​\(t\)​z\),z⊤​SB​C​\(t\)​z∈ℝ0\+,2​\(Γ\+Λ\)​\(z⊤​SB​C​\(t\)​z\),z⊤​SB​C​\(t\)​z∈ℝ−\.\\displaystyle a\_\{c\}\(t,z\)=\\begin\{cases\}\\gamma c\(1\+c\)\\,\(z^\{\\top\}S\_\{BC\}\(t\)z\),&z^\{\\top\}S\_\{BC\}\(t\)z\\in\\mathbb\{R\}\_\{0\}^\{\+\},\\\\\[3\.99994pt\] 2\(\\Gamma\+\\Lambda\)\\,\(z^\{\\top\}S\_\{BC\}\(t\)z\),&z^\{\\top\}S\_\{BC\}\(t\)z\\in\\mathbb\{R\}^\{\-\}\.\\end\{cases\}\(6\)
We now require thataca\_\{c\}persistently excites the dynamics: there is someT∈ℝ\+T\\in\\mathbb\{R\}^\{\+\}such that, on average over any interval of sizeTT,aca\_\{c\}is positive\. In this section we will prove that this is sufficient to establish local exponential stability of the consensus equilibria: intuitively, persistency of excitation allows the first token to converge, and the cascade structure then propagates this convergence through the remaining tokens\.

###### Assumption IV\.1\(Persistency of excitation\)

There existc∈\]0,1\[c\\in\\,\]0,1\[andT,ρ∈ℝ\+T,\\rho\\in\\mathbb\{R\}^\{\+\}such that∫tt\+Tαc​\(s\)​𝑑s≥ρ​T\\int\_\{t\}^\{t\+T\}\\alpha\_\{c\}\(s\)\\,ds\\geq\\rho Tfor eacht∈ℝ0\+t\\in\\mathbb\{R\}\_\{0\}^\{\+\}, whereαc​\(t\)=min⁡infz∈Ωcσ​\(μ~\)σ∈\{−1,\+1\}⁡ac​\(t,z\)\\alpha\_\{c\}\(t\)=\\min\_\{\\sigma\\in\\\{\-1,\+1\\\}\}~\\inf\_\{z\\in\\Omega\_\{c\}^\{\\sigma\}\(\\tilde\{\\mu\}\)\}a\_\{c\}\(t,z\)\.

Assumption[IV\.1](https://arxiv.org/html/2609.17997#S4.Thmassumption1)requires the interaction term to be sufficiently positive on average, rather than pointwise\. Whether this property holds depends on the particular weights of the model and architecture considered\. For the Mamba\-2 architecture studied in Section[VI](https://arxiv.org/html/2609.17997#S6), which has a SiLU nonlinearity, the following result provides some motivation for this assumption\. In particular, under independently drawn zero\-mean weights, the SiLU\-modified interaction term has strictly positive expectation, whereas its bilinear counterpart has zero expectation\. Thus, while the next result does not establish Assumption[IV\.1](https://arxiv.org/html/2609.17997#S4.Thmassumption1), it shows that, for this particular architectural choice, the SiLU nonlinearity biases the interaction term toward positive values\.

###### Lemma IV\.1\(Positive bias induced by the nonlinearity\)

LetSB,SCS\_\{B\},S\_\{C\}be independent random matrices with𝔼⁡\[SB\]=𝔼⁡\[SC\]=0\\mathbb\{E\}\[S\_\{B\}\]=\\mathbb\{E\}\[S\_\{C\}\]=0, and letzi,zj∈𝕊n−1z\_\{i\},z\_\{j\}\\in\\mathbb\{S\}^\{n\-1\}be such that, for someμ∈\{1,…,m\}\\mu\\in\\\{1,\\dots,m\\\}, it holds that:

ℙ\[\(SCzi\)μ=0\]<1andℙ\[\(SBzj\)μ=0\]<1\.\\displaystyle\\mathbb\{P\}\\big\[\(S\_\{C\}z\_\{i\}\)^\{\\mu\}=0\\big\]<1\\quad\\text\{and\}\\quad\\mathbb\{P\}\\big\[\(S\_\{B\}z\_\{j\}\)^\{\\mu\}=0\\big\]<1\.Then:

1. 1\.𝔼⁡\[zi⊤​SC⊤​SB​zj\]=0\\mathbb\{E\}\\big\[z\_\{i\}^\{\\top\}S\_\{C\}^\{\\top\}S\_\{B\}\\,z\_\{j\}\\big\]=0\.
2. 2\.𝔼⁡\[ς​\(SC​zi\)⊤​ς​\(SB​zj\)\]\>0\\mathbb\{E\}\\big\[\\varsigma\(S\_\{C\}z\_\{i\}\)^\{\\top\}\\varsigma\(S\_\{B\}z\_\{j\}\)\\big\]\>0\.

###### Proof:

Result 1\) follows from independence, since𝔼⁡\[SC⊤​SB\]=𝔼​\[SC\]⊤​𝔼​\[SB\]=0\\mathbb\{E\}\[S\_\{C\}^\{\\top\}S\_\{B\}\]=\\mathbb\{E\}\[S\_\{C\}\]^\{\\top\}\\mathbb\{E\}\[S\_\{B\}\]=0\. For 2\), the vectorsSC​ziS\_\{C\}z\_\{i\}andSB​zjS\_\{B\}z\_\{j\}are independent, and therefore it follows:

𝔼⁡\[ς​\(SC​zi\)⊤​ς​\(SB​zj\)\]=𝔼​\[ς⁡\(SC​zi\)\]⊤​𝔼​\[ς⁡\(SB​zj\)\]\.\\mathbb\{E\}\\big\[\\varsigma\(S\_\{C\}z\_\{i\}\)^\{\\top\}\\varsigma\(S\_\{B\}z\_\{j\}\)\\big\]=\\mathbb\{E\}\\big\[\\varsigma\(S\_\{C\}z\_\{i\}\)\\big\]^\{\\top\}\\mathbb\{E\}\\big\[\\varsigma\(S\_\{B\}z\_\{j\}\)\\big\]\.Since it holds that\(1\+e−a\)−1=12\+12​tanh⁡\(a2\)\(1\+e^\{\-a\}\)^\{\-1\}=\\tfrac\{1\}\{2\}\+\\tfrac\{1\}\{2\}\\tanh\(\\tfrac\{a\}\{2\}\)for eacha∈ℝa\\in\\mathbb\{R\}, the SiLU decomposes asς⁡\(x\)=x2\+ϱ⁡\(x\)\\varsigma\(x\)=\\tfrac\{x\}\{2\}\+\\varrho\(x\), whereϱ​\(x\)μ=xμ2​tanh⁡\(xμ2\)≥0\\varrho\(x\)^\{\\mu\}=\\tfrac\{x^\{\\mu\}\}\{2\}\\tanh\(\\tfrac\{x^\{\\mu\}\}\{2\}\)\\geq 0, astanh\\tanhis odd\. Computing componentwise, we obtain:

𝔼⁡\[ς​\(SC​zi\)μ\]\\displaystyle\\mathbb\{E\}\[\\varsigma\(S\_\{C\}z\_\{i\}\)^\{\\mu\}\]=12​𝔼​\[\(SC​zi\)μ\]\+𝔼⁡\[ϱ​\(SC​zi\)μ\]\\displaystyle=\\tfrac\{1\}\{2\}\\mathbb\{E\}\[\(S\_\{C\}z\_\{i\}\)^\{\\mu\}\]\+\\mathbb\{E\}\[\\varrho\(S\_\{C\}z\_\{i\}\)^\{\\mu\}\]=𝔼⁡\[ϱ​\(SC​zi\)μ\]≥0,\\displaystyle=\\mathbb\{E\}\[\\varrho\(S\_\{C\}z\_\{i\}\)^\{\\mu\}\]\\geq 0,for eachμ∈\{1,…,m\}\\mu\\in\\\{1,\\dots,m\\\}, since𝔼⁡\[SC\]=0\\mathbb\{E\}\[S\_\{C\}\]=0\. The same holds forSBS\_\{B\}, so the inner product above is a sum of products of nonnegative terms\. Moreover,ϱ⁡\(a\)=0\\varrho\(a\)=0if and only ifa=0a=0, soℙ\[\(SCzi\)μ=0\]<1\\mathbb\{P\}\\big\[\(S\_\{C\}z\_\{i\}\)^\{\\mu\}=0\\big\]<1yields𝔼⁡\[ϱ​\(SC​zi\)μ\]\>0\\mathbb\{E\}\[\\varrho\(S\_\{C\}z\_\{i\}\)^\{\\mu\}\]\>0, and likewise forSBS\_\{B\}, so the result follows\. ∎

In Section[VI](https://arxiv.org/html/2609.17997#S6), we show the consequence of the positive bias: with weights drawn at random, the tokens cluster when the nonlinearity is present, and spread out when it is removed\.

Our next step is to introduce the inner product between each token and the candidate equilibrium, and derive its dynamics\. This reduces the analysis to a scalar differential equation for each token, in which convergence to consensus amounts to that product reaching one\. For convenience, givenΣ=\(σ1,…,σℓ\)∈\{−1,\+1\}ℓ\\Sigma=\(\\sigma\_\{1\},\\ldots,\\sigma\_\{\\ell\}\)\\in\\\{\-1,\+1\\\}^\{\\ell\}, tokens are projected to the real line by defining:

bi=σi​\(𝔢μ~\)⊤​zi=σi​ziμ~,i∈\{1,…,ℓ\}\.\\displaystyle b\_\{i\}=\\sigma\_\{i\}\\,\(\\mathfrak\{e\}^\{\\tilde\{\\mu\}\}\)^\{\\top\}\\,z\_\{i\}=\\sigma\_\{i\}\\,z\_\{i\}^\{\\tilde\{\\mu\}\},\\qquad i\\in\\\{1,\\ldots,\\ell\\\}\.\(7\)The dynamics is obtained from \([5](https://arxiv.org/html/2609.17997#S2.E5)\) asb˙i=σi​\(𝔢μ~\)⊤​z˙i=σi​z˙iμ~\\dot\{b\}\_\{i\}=\\sigma\_\{i\}\\,\(\\mathfrak\{e\}^\{\\tilde\{\\mu\}\}\)^\{\\top\}\\,\\dot\{z\}\_\{i\}=\\sigma\_\{i\}\\,\\dot\{z\}\_\{i\}^\{\\tilde\{\\mu\}\}\. Note thatzi∈Ωcσi​\(μ~\)z\_\{i\}\\in\\Omega\_\{c\}^\{\\sigma\_\{i\}\}\(\\tilde\{\\mu\}\)corresponds tobi∈\]c,1\]b\_\{i\}\\in\\,\]c,1\], andzi=σi​𝔢μ~z\_\{i\}=\\sigma\_\{i\}\\mathfrak\{e\}^\{\\tilde\{\\mu\}\}corresponds tobi=1b\_\{i\}=1\. For convenience, we will also use the following maps as Lyapunov functions:

Vi=1−bi=1−σi​𝔢μ~,i∈\{1,…,ℓ\}\.\\displaystyle V\_\{i\}=1\-b\_\{i\}=1\-\\sigma\_\{i\}\\,\\mathfrak\{e\}^\{\\tilde\{\\mu\}\},\\quad i\\in\\\{1,\\ldots,\\ell\\\}\.\(8\)Lastly, we define the errors as:

ei=zi−σi​𝔢μ~,i∈\{1,…,ℓ\}\.\\displaystyle e\_\{i\}=z\_\{i\}\-\\sigma\_\{i\}\\,\\mathfrak\{e\}^\{\\tilde\{\\mu\}\},\\qquad i\\in\\\{1,\\ldots,\\ell\\\}\.\(9\)An easy check shows that\|ei\|2=2​\(1−bi\)\|e\_\{i\}\|^\{2\}=2\\,\(1\-b\_\{i\}\)\.

The following result, whose proof can be found in the Appendix, is used to show local exponential stability of the consensus equilibria\.

###### Proposition IV\.1\(Bounds for the scalar dynamics\)

LetΣ=\(σ1,…,σℓ\)∈\{−1,\+1\}ℓ\\Sigma=\(\\sigma\_\{1\},\\ldots,\\sigma\_\{\\ell\}\)\\in\\\{\-1,\+1\\\}^\{\\ell\}and consider the projected variables \([7](https://arxiv.org/html/2609.17997#S4.E7)\) under Assumptions[III\.1](https://arxiv.org/html/2609.17997#S3.Thmassumption1)and[IV\.1](https://arxiv.org/html/2609.17997#S4.Thmassumption1)\. Then:

1. 1\.Fori=1i=1, we haveb˙1≥ac​\(t,z1\)​\(1−b1\)\\dot\{b\}\_\{1\}\\geq a\_\{c\}\(t,z\_\{1\}\)\(1\-b\_\{1\}\)for each\(t,b1\)∈ℝ0\+×\]c,1\]\(t,b\_\{1\}\)\\in\\mathbb\{R\}\_\{0\}^\{\+\}\\times\\,\]c,1\]\.
2. 2\.For eachi∈\{2,…,ℓ\}i\\in\\\{2,\\ldots,\\ell\\\}andε∈ℝ\+\\varepsilon\\in\\mathbb\{R\}^\{\+\}, there existdε∈\]0,1−c\]d\_\{\\varepsilon\}\\in\\,\]0,1\-c\]andM∈ℝ\+M\\in\\mathbb\{R\}^\{\+\}such thatb˙i≥\(ac​\(t,zi\)−ε\)​\(1−bi\)−M​∑j=1i−1\|ej\|\\dot\{b\}\_\{i\}\\geq\(a\_\{c\}\(t,z\_\{i\}\)\-\\varepsilon\)\\,\(1\-b\_\{i\}\)\-M\\sum\_\{j=1\}^\{i\-1\}\|e\_\{j\}\|for each\(t,bi\)∈ℝ0\+×\[1−dε,1\]\(t,b\_\{i\}\)\\in\\mathbb\{R\}\_\{0\}^\{\+\}\\times\[1\-d\_\{\\varepsilon\},1\]\.

The first result explaining the asymptotic behavior of \([7](https://arxiv.org/html/2609.17997#S4.E7)\) concerns the first token and it establishes exponential convergence to the principal eigenvector ofDi​jD\_\{ij\}\.

###### Lemma IV\.2\(Asymptotic behavior of the first token\)

Under Assumptions[III\.1](https://arxiv.org/html/2609.17997#S3.Thmassumption1)and[IV\.1](https://arxiv.org/html/2609.17997#S4.Thmassumption1), for eachσ1∈\{−1,\+1\}\\sigma\_\{1\}\\in\\\{\-1,\+1\\\}, the pointzσ1∗=σ1​𝔢μ~∈𝒞1​\(μ~\)z\_\{\\sigma\_\{1\}\}^\{\*\}=\\sigma\_\{1\}\\,\\mathfrak\{e\}^\{\\tilde\{\\mu\}\}\\in\\mathcal\{C\}\_\{1\}\(\\tilde\{\\mu\}\)is a locally exponentially stable equilibrium for the dynamics of the first token in model \([5](https://arxiv.org/html/2609.17997#S2.E5)\) and its domain of attraction contains the setΩc1σ1​\(μ~\)\\Omega\_\{c\_\{1\}\}^\{\\sigma\_\{1\}\}\(\\tilde\{\\mu\}\), wherec1=1−\(1−c\)​exp⁡\(η0​T\)∈\]c,1\[c\_\{1\}=1\-\(1\-c\)\\exp\(\\eta\_\{0\}T\)\\in\\,\]c,1\[andη0=min⁡\{0,inf\(t,z\)∈ℝ0\+×Ωcσ1​\(μ~\)ac​\(t,z\)\}∈ℝ0−\\eta\_\{0\}=\\min\\left\\\{0,~\\inf\_\{\(t,z\)\\in\\mathbb\{R\}\_\{0\}^\{\+\}\\times\\Omega\_\{c\}^\{\\sigma\_\{1\}\}\(\\tilde\{\\mu\}\)\}~a\_\{c\}\(t,z\)\\right\\\}\\in\\mathbb\{R\}\_\{0\}^\{\-\}\.

###### Proof:

Note that, using the projected variables \([7](https://arxiv.org/html/2609.17997#S4.E7)\), it is enough to show thatb1∗=1b\_\{1\}^\{\*\}=1is an exponentially stable equilibrium for the dynamicsb˙1=σ1​z˙1μ~\\dot\{b\}\_\{1\}=\\sigma\_\{1\}\\dot\{z\}\_\{1\}^\{\\tilde\{\\mu\}\}with domain of attraction containing the interval\]c1,1\]\]c\_\{1\},1\]\.

Given thatac​\(t,z\)≥η0a\_\{c\}\(t,z\)\\geq\\eta\_\{0\}by definition, from the first part of Proposition[IV\.1](https://arxiv.org/html/2609.17997#S4.Thmproposition1), and using \([8](https://arxiv.org/html/2609.17997#S4.E8)\), we conclude thatV˙1≤−η0​V1\\dot\{V\}\_\{1\}\\leq\-\\eta\_\{0\}\\,V\_\{1\}for each\(t,V1\)∈ℝ0\+×\[0,1−c\[\(t,V\_\{1\}\)\\in\\mathbb\{R\}\_\{0\}^\{\+\}\\times\[0,1\-c\[\\,and, in particular, for each\(t,V1\)∈ℝ0\+×\[0,1−c1\[\(t,V\_\{1\}\)\\in\\mathbb\{R\}\_\{0\}^\{\+\}\\times\[0,1\-c\_\{1\}\[\\,\. As a result, for eacht∈ℝ0\+t\\in\\mathbb\{R\}\_\{0\}^\{\+\}andτ∈\[0,T\]\\tau\\in\[0,T\]such thatV1\(s\)∈\[0,1−c1\[V\_\{1\}\(s\)\\in\[0,1\-c\_\{1\}\[for eachs∈\[t,t\+τ\]s\\in\[t,t\+\\tau\], we have:

V1​\(t\+τ\)≤exp⁡\(−η0​τ\)​V1​\(t\)\.\\displaystyle V\_\{1\}\(t\+\\tau\)\\leq\\exp\(\-\\eta\_\{0\}\\tau\)\\,V\_\{1\}\(t\)\.\(10\)
Next, we show thatV1\(t\)∈\[0,1−c\[V\_\{1\}\(t\)\\in\[0,1\-c\[for eacht∈ℝ0\+t\\in\\mathbb\{R\}\_\{0\}^\{\+\}providedV1\(0\)∈\[0,1−c1\[V\_\{1\}\(0\)\\in\[0,1\-c\_\{1\}\[\\,\. By contradiction, suppose thatt0=inf\{t∈ℝ0\+∣V1​\(t\)=1−c\}∈ℝ\+t\_\{0\}=\\inf\\\{t\\in\\mathbb\{R\}\_\{0\}^\{\+\}\\mid V\_\{1\}\(t\)=1\-c\\\}\\in\\mathbb\{R\}^\{\+\}, and writet0=\(k−1\)​T\+τt\_\{0\}=\(k\-1\)T\+\\taufor somek∈ℕk\\in\\mathbb\{N\}andτ∈\[0,T\[\\tau\\in\[0,T\[\\,\. Given thatV1\(0\)∈\[0,1−c1\[⊂\[0,1−c\[V\_\{1\}\(0\)\\in\[0,1\-c\_\{1\}\[\\,\\subset\[0,1\-c\[\\,, we know thatV1\(t\)∈\[0,1−c\[V\_\{1\}\(t\)\\in\[0,1\-c\[by construction for eacht∈\[0,t0\[t\\in\[0,t\_\{0\}\[\\,\. Hence,\(1\)\(1\)of Proposition[IV\.1](https://arxiv.org/html/2609.17997#S4.Thmproposition1)ensures thatV˙1​\(t\)≤−ac​\(t,z1​\(t\)\)​V1​\(t\)\\dot\{V\}\_\{1\}\(t\)\\leq\-a\_\{c\}\(t,z\_\{1\}\(t\)\)\\,V\_\{1\}\(t\)for eacht∈\[0,t0\[t\\in\[0,t\_\{0\}\[\\,\. From this and Assumption[IV\.1](https://arxiv.org/html/2609.17997#S4.Thmassumption1), we obtain:

V1​\(\(k−1\)​T\)\\displaystyle V\_\{1\}\(\(k\-1\)T\)≤exp\(−∫\(k−2\)​T\(k−1\)​Tαc\(t\)dt\)V1\(\(k−2\)T\)\\displaystyle\\leq\\exp\\left\(\-\\int\_\{\(k\-2\)T\}^\{\(k\-1\)T\}\\alpha\_\{c\}\(t\)\\,dt\\right\)V\_\{1\}\(\(k\-2\)T\)≤exp⁡\(−ρ​T\)​V1​\(\(k−2\)​T\),\\displaystyle\\leq\\exp\(\-\\rho T\)\\,V\_\{1\}\(\(k\-2\)T\),for eachk∈ℕk\\in\\mathbb\{N\}\. By iterating the previous inequality, we obtain:

V1​\(\(k−1\)​T\)\\displaystyle V\_\{1\}\(\(k\-1\)T\)≤exp⁡\(−ρ⁡\(k−1\)​T\)​V1​\(0\),k∈ℕ\.\\displaystyle\\leq\\exp\(\-\\rho\(k\-1\)T\)\\,V\_\{1\}\(0\),\\qquad k\\in\\mathbb\{N\}\.\(11\)Thus,V1​\(\(k−1\)​T\)<V1​\(0\)<1−c1V\_\{1\}\(\(k\-1\)T\)<V\_\{1\}\(0\)<1\-c\_\{1\}\. From this and \([10](https://arxiv.org/html/2609.17997#S4.E10)\) witht=\(k−1\)​Tt=\(k\-1\)T, we obtain a contradiction:

1−c=V1​\(t0\)\\displaystyle 1\-c=V\_\{1\}\(t\_\{0\}\)≤exp⁡\(−η0​τ\)​V1​\(\(k−1\)​T\)\\displaystyle\\leq\\exp\(\-\\eta\_\{0\}\\tau\)\\,V\_\{1\}\(\(k\-1\)T\)<exp⁡\(−η0​T\)​\(1−c1\)\\displaystyle<\\exp\(\-\\eta\_\{0\}T\)\(1\-c\_\{1\}\)=exp⁡\(−η0​T\)​\(1−c\)​exp⁡\(η0​T\)=1−c\.\\displaystyle=\\exp\(\-\\eta\_\{0\}T\)\(1\-c\)\\exp\(\\eta\_\{0\}T\)=1\-c\.As a result,V1\(t\)∈\[0,1−c\[V\_\{1\}\(t\)\\in\[0,1\-c\[for eacht∈ℝ0\+t\\in\\mathbb\{R\}\_\{0\}^\{\+\}\.

Lastly, for eacht∈ℝ0\+t\\in\\mathbb\{R\}\_\{0\}^\{\+\}, letk∈ℕk\\in\\mathbb\{N\}be such thatt∈\[\(k−1\)T,kT\[t\\in\[\(k\-1\)T,kT\[\\,\. From \([10](https://arxiv.org/html/2609.17997#S4.E10)\) and \([11](https://arxiv.org/html/2609.17997#S4.E11)\), we conclude:

V1​\(t\)\\displaystyle V\_\{1\}\(t\)≤exp⁡\(−η0​\(t−\(k−1\)​T\)\)​V1​\(\(k−1\)​T\)\\displaystyle\\leq\\exp\(\-\\eta\_\{0\}\(t\-\(k\-1\)T\)\)\\,V\_\{1\}\(\(k\-1\)T\)<exp⁡\(−η0​\(t−\(k−1\)​T\)\)​exp⁡\(−ρ​T​\(k−1\)\)​V1​\(0\)\\displaystyle<\\exp\(\-\\eta\_\{0\}\(t\-\(k\-1\)T\)\)\\exp\(\-\\rho T\(k\-1\)\)\\,V\_\{1\}\(0\)<exp⁡\(−η0​T\)​exp⁡\(−ρ⁡\(T⁡\(k−1\)−t\)\)​exp⁡\(−ρ​t\)​V1​\(0\)\\displaystyle<\\exp\(\-\\eta\_\{0\}T\)\\exp\(\-\\rho\(T\(k\-1\)\-t\)\)\\exp\(\-\\rho t\)\\,V\_\{1\}\(0\)<exp⁡\(−η0​T\)​exp⁡\(ρ​T\)​exp⁡\(−ρ​t\)​V1​\(0\)\\displaystyle<\\exp\(\-\\eta\_\{0\}T\)\\exp\(\\rho T\)\\exp\(\-\\rho t\)\\,V\_\{1\}\(0\)=M1​exp⁡\(−ρ​t\)​V1​\(0\),\\displaystyle=M\_\{1\}\\exp\(\-\\rho t\)\\,V\_\{1\}\(0\),whereM1=exp⁡\(\(ρ−η0\)​T\)∈ℝ\+M\_\{1\}=\\exp\(\(\\rho\-\\eta\_\{0\}\)T\)\\in\\mathbb\{R\}^\{\+\}\. ∎

The next result leverages Lemma[IV\.2](https://arxiv.org/html/2609.17997#S4.Thmlemma2)and Proposition[IV\.1](https://arxiv.org/html/2609.17997#S4.Thmproposition1)to establish local exponential stability of the consensus set\.

###### Theorem IV\.1\(Local exponential stability of consensus\)

Under Assumptions[III\.1](https://arxiv.org/html/2609.17997#S3.Thmassumption1)and[IV\.1](https://arxiv.org/html/2609.17997#S4.Thmassumption1), for eachΣ=\(σ1,…,σℓ\)∈\{−1,\+1\}ℓ\\Sigma=\(\\sigma\_\{1\},\\ldots,\\sigma\_\{\\ell\}\)\\in\\\{\-1,\+1\\\}^\{\\ell\}, the consensus equilibriumZΣ∗=\(σ1​𝔢μ~,…,σℓ​𝔢μ~\)∈𝒞ℓ​\(μ~\)Z\_\{\\Sigma\}^\{\*\}=\(\\sigma\_\{1\}\\,\\mathfrak\{e\}^\{\\tilde\{\\mu\}\},\\ldots,\\sigma\_\{\\ell\}\\mathfrak\{e\}^\{\\tilde\{\\mu\}\}\)\\in\\mathcal\{C\}\_\{\\ell\}\(\\tilde\{\\mu\}\)is locally exponentially stable for the continuous SSM dynamics \([5](https://arxiv.org/html/2609.17997#S2.E5)\)\.

###### Proof:

Givenε∈\]0,ρ\[\\varepsilon\\in\\,\]0,\\rho\[\\,, letdε∈\]0,1−c\]d\_\{\\varepsilon\}\\in\\,\]0,1\-c\]andM∈ℝ\+M\\in\\mathbb\{R\}^\{\+\}as in Proposition[IV\.1](https://arxiv.org/html/2609.17997#S4.Thmproposition1)\. In addition, let:

ηε=min⁡\{ε,min⁡inf\(t,z\)∈ℝ0\+×Ωcσ​\(μ~\)σ∈\{−1,\+1\}⁡ac​\(t,z\)\}−ε∈ℝ0−\.\\displaystyle\\eta\_\{\\varepsilon\}=\\min\\left\\\{\\varepsilon,~\\min\_\{\\sigma\\in\\\{\-1,\+1\\\}\}~\\inf\_\{\(t,z\)\\in\\mathbb\{R\}\_\{0\}^\{\+\}\\times\\Omega\_\{c\}^\{\\sigma\}\(\\tilde\{\\mu\}\)\}~a\_\{c\}\(t,z\)\\right\\\}\-\\varepsilon\\in\\mathbb\{R\}\_\{0\}^\{\-\}\.Letθ=exp⁡\(−\(ρ−ε\)​T\)∈\]0,1\[\\theta=\\exp\(\-\(\\rho\-\\varepsilon\)T\)\\in\\,\]0,1\[\\,,r=exp\(−ηεT\)∈\[1,∞\[r=\\exp\(\-\\eta\_\{\\varepsilon\}T\)\\in\[1,\\infty\[,R∈\]r,∞\[R\\in\\,\]r,\\infty\[andH=T​M​exp⁡\(−ηε​T\)∈ℝ\+H=T\\,M\\exp\(\-\\eta\_\{\\varepsilon\}T\)\\in\\mathbb\{R\}^\{\+\}\. Following Lemma[IV\.2](https://arxiv.org/html/2609.17997#S4.Thmlemma2), letM1∈\[1,∞\[M\_\{1\}\\in\[1,\\infty\[andρ1∈ℝ\+\\rho\_\{1\}\\in\\mathbb\{R\}^\{\+\}be such that:

V1​\(t\)≤M1​exp⁡\(−ρ1​t\)​V1​\(0\),\\displaystyle V\_\{1\}\(t\)\\leq M\_\{1\}\\,\\exp\(\-\\rho\_\{1\}t\)\\,V\_\{1\}\(0\),\(12\)for eachV1\(0\)∈\[0,d1\[V\_\{1\}\(0\)\\in\[0,d\_\{1\}\[\\,, whered1=1−c1d\_\{1\}=1\-c\_\{1\}andc1=1−\(1−c\)​exp⁡\(η0​T\)∈\]c,1\[c\_\{1\}=1\-\(1\-c\)\\exp\(\\eta\_\{0\}T\)\\in\\,\]c,1\[\\,\. Fori∈\{2,…,ℓ\}i\\in\\\{2,\\ldots,\\ell\\\}, we recursively define:

νi=12​min⁡\{ρ1,…,ρi−1\},\\displaystyle\\nu\_\{i\}=\\frac\{1\}\{2\}\\min\\left\\\{\\rho\_\{1\},\\ldots,\\rho\_\{i\-1\}\\right\\\},\(13\)ρi=12​min⁡\{−log⁡\(θ\)T,νi\}∈ℝ\+\.\\displaystyle\\rho\_\{i\}=\\frac\{1\}\{2\}\\min\\left\\\{\-\\frac\{\\log\(\\theta\)\}\{T\},\\nu\_\{i\}\\right\\\}\\in\\mathbb\{R\}^\{\+\}\.\(14\)In addition, letCi=C\(T,θ,ρi\)∈\[1,∞\[C\_\{i\}=C\(T,\\theta,\\rho\_\{i\}\)\\in\[1,\\infty\[be as in Lemma[\.2](https://arxiv.org/html/2609.17997#A0.Thmlemma2)and:

Mi∈\[max\{1,exp\(ρiT\)\(Cir\+1−θ\)\},∞\[\.\\displaystyle M\_\{i\}\\in\[\\max\\\{1,\\exp\(\\rho\_\{i\}T\)\(C\_\{i\}r\+1\-\\theta\)\\\},\\infty\[\\,\.\(15\)Lastly, let\(d1,…,dℓ\)\(d\_\{1\},\\ldots,d\_\{\\ell\}\)be the sequence given by Lemma[\.1](https://arxiv.org/html/2609.17997#A0.Thmlemma1), and denoteci=1−di∈\]c,1\[c\_\{i\}=1\-d\_\{i\}\\in\\,\]c,1\[fori∈\{1,…,ℓ\}i\\in\\\{1,\\ldots,\\ell\\\}\.

For eachi∈\{1,…,ℓ\}i\\in\\\{1,\\ldots,\\ell\\\}, the dynamics of theii\-th token only depends on the previous onesj∈\{1,…,i\}j\\in\\\{1,\\ldots,i\\\}\. Thus, we proceed by induction on the token index\.

Base case\.From Lemma[IV\.2](https://arxiv.org/html/2609.17997#S4.Thmlemma2), the equilibriumZΣ∗​\(1\)=σ1​𝔢μ~∈𝒞1​\(μ~\)Z\_\{\\Sigma\}^\{\*\}\(1\)=\\sigma\_\{1\}\\,\\mathfrak\{e\}^\{\\tilde\{\\mu\}\}\\in\\mathcal\{C\}\_\{1\}\(\\tilde\{\\mu\}\)is locally exponentially stable for the subsystem of \([5](https://arxiv.org/html/2609.17997#S2.E5)\) given by the first token, and its domain of attraction contains the setΩc1σ1​\(μ~\)\\Omega\_\{c\_\{1\}\}^\{\\sigma\_\{1\}\}\(\\tilde\{\\mu\}\)\. Specifically, \([12](https://arxiv.org/html/2609.17997#S4.E12)\) holds\.

Induction hypothesis\.Giveni∈\{2,…,ℓ\}i\\in\\\{2,\\ldots,\\ell\\\}, the equilibriumZΣ∗​\(i−1\)=\(σ1​𝔢μ~,…,σi−1​𝔢μ~\)∈𝒞i−1​\(μ~\)Z\_\{\\Sigma\}^\{\*\}\(i\-1\)=\(\\sigma\_\{1\}\\,\\mathfrak\{e\}^\{\\tilde\{\\mu\}\},\\ldots,\\sigma\_\{i\-1\}\\,\\mathfrak\{e\}^\{\\tilde\{\\mu\}\}\)\\in\\mathcal\{C\}\_\{i\-1\}\(\\tilde\{\\mu\}\), is exponentially stable for the subsystem of \([5](https://arxiv.org/html/2609.17997#S2.E5)\) given by the firsti−1i\-1tokens and its domain of attraction contains the setΩc1σ1​\(μ~\)×…×Ωci−1σi−1​\(μ~\)\\Omega\_\{c\_\{1\}\}^\{\\sigma\_\{1\}\}\(\\tilde\{\\mu\}\)\\times\\ldots\\times\\Omega\_\{c\_\{i\-1\}\}^\{\\sigma\_\{i\-1\}\}\(\\tilde\{\\mu\}\)\. In terms of \([8](https://arxiv.org/html/2609.17997#S4.E8)\), the previous condition reads:

Vj\(t\)≤Mjdjexp\(−ρjt\),Vj\(0\)∈\[0,dj\[,\\displaystyle V\_\{j\}\(t\)\\leq M\_\{j\}\\,d\_\{j\}\\,\\exp\(\-\\rho\_\{j\}\\,t\),\\qquad V\_\{j\}\(0\)\\in\[0,d\_\{j\}\[\\,,for eachj∈\{1,…,i−1\}j\\in\\\{1,\\ldots,i\-1\\\}\. Analogously, in terms of the error \([9](https://arxiv.org/html/2609.17997#S4.E9)\), we may write:

\|ej​\(t\)\|≤Mj​exp⁡\(−ρj2​t\)​\|ej​\(0\)\|,\\displaystyle\|e\_\{j\}\(t\)\|\\leq\\sqrt\{M\_\{j\}\}\\,\\exp\\left\(\-\\frac\{\\rho\_\{j\}\}\{2\}t\\right\)\\,\|e\_\{j\}\(0\)\|,\(16\)for each\|ej\(0\)\|∈\[0,2​dj\[\|e\_\{j\}\(0\)\|\\in\[0,\\sqrt\{2d\_\{j\}\}\[\\,\.

Inductive step\.We need to show thatbi∗=1b\_\{i\}^\{\*\}=1is a locally exponentially stable equilibrium for the projected dynamics \([7](https://arxiv.org/html/2609.17997#S4.E7)\) and its domain of attraction contains the set\]ci,1\]\]c\_\{i\},1\]\. To that end, we use the following claim, whose proof can be found in the Appendix\.

###### Claim IV\.1

Letk∈ℕ∪\{0\}k\\in\\mathbb\{N\}\\cup\\\{0\\\}and denoteK=∑j=1i−1M​2​Mj​dj​exp⁡\(−ρj2​\(k−1\)​T\)K=\\sum\_\{j=1\}^\{i\-1\}M\\sqrt\{2M\_\{j\}d\_\{j\}\}\\exp\\left\(\-\\frac\{\\rho\_\{j\}\}\{2\}\(k\-1\)T\\right\)\. IfVi\(\(k−1\)T\)∈\[0,di\[V\_\{i\}\(\(k\-1\)T\)\\in\[0,d\_\{i\}\[\\,, then:

1. 1\.Vi\(t\)∈\[0,Rdi\[V\_\{i\}\(t\)\\in\[0,Rd\_\{i\}\[for eacht∈\[\(k−1\)T,kT\[t\\in\[\(k\-1\)T,kT\[\\,\.
2. 2\.Vi​\(k​T\)≤θ​Vi​\(\(k−1\)​T\)\+K​T​exp⁡\(−ηε​T\)V\_\{i\}\(kT\)\\leq\\theta\\,V\_\{i\}\(\(k\-1\)T\)\+KT\\exp\(\-\\eta\_\{\\varepsilon\}T\)\.
3. 3\.Vi\(kT\)∈\[0,di\[V\_\{i\}\(kT\)\\in\[0,d\_\{i\}\[\\,\.

An induction argument using Claim[IV\.1](https://arxiv.org/html/2609.17997#S4.Thmclaim1)shows that, ifVi\(0\)∈\[0,di\[V\_\{i\}\(0\)\\in\[0,d\_\{i\}\[\\,, then:

Vi​\(k​T\)≤\\displaystyle V\_\{i\}\(kT\)\\leqθ​Vi​\(\(k−1\)​T\)\\displaystyle\\theta\\,V\_\{i\}\(\(k\-1\)T\)\(17\)\+H∑j=1i−12​Mj​djexp\(−ρj2\(k−1\)T\),\\displaystyle\+H\\sum\_\{j=1\}^\{i\-1\}\\sqrt\{2M\_\{j\}d\_\{j\}\}\\exp\\left\(\-\\frac\{\\rho\_\{j\}\}\{2\}\(k\-1\)T\\right\),for eachk∈ℕ∪\{0\}k\\in\\mathbb\{N\}\\cup\\\{0\\\}\.

On the other hand, using \([13](https://arxiv.org/html/2609.17997#S4.E13)\), \([14](https://arxiv.org/html/2609.17997#S4.E14)\) and\(1−θ\)​di≥H​∑j=1i−12​Mj​dj\(1\-\\theta\)d\_\{i\}\\geq H\\sum\_\{j=1\}^\{i\-1\}\\sqrt\{2M\_\{j\}d\_\{j\}\}\(recall Lemma[\.1](https://arxiv.org/html/2609.17997#A0.Thmlemma1)\), we obtain:

∑j=1i−12​Mj​dj​exp⁡\(−ρj2​\(k−1\)​T\)\\displaystyle\\sum\_\{j=1\}^\{i\-1\}\\sqrt\{2M\_\{j\}d\_\{j\}\}\\,\\exp\\left\(\-\\frac\{\\rho\_\{j\}\}\{2\}\(k\-1\)T\\right\)≤∑j=1i−12​Mj​dj​exp⁡\(−νi​\(k−1\)​T\)\\displaystyle\\leq\\sum\_\{j=1\}^\{i\-1\}\\sqrt\{2M\_\{j\}d\_\{j\}\}\\exp\(\-\\nu\_\{i\}\(k\-1\)T\)≤1−θH​exp⁡\(−νi​\(k−1\)​T\)​di,\\displaystyle\\leq\\frac\{1\-\\theta\}\{H\}\\exp\(\-\\nu\_\{i\}\(k\-1\)T\)\\,d\_\{i\},for eachk∈ℕk\\in\\mathbb\{N\}\. From this and \([17](https://arxiv.org/html/2609.17997#S4.E17)\), we obtain:

Vi​\(k​T\)≤\\displaystyle V\_\{i\}\(kT\)\\leqOPENθ​Vi​\(\(k−1\)​T\)\)\+\(1−θ\)​di​exp⁡\(−νi​\(k−1\)​T\),\\displaystyle\\theta\\,V\_\{i\}\(\(k\-1\)T\)\)\+\(1\-\\theta\)\\,d\_\{i\}\\,\\exp\(\-\\nu\_\{i\}\(k\-1\)T\),for eachk∈ℕk\\in\\mathbb\{N\}\. Hence, Lemma[\.2](https://arxiv.org/html/2609.17997#A0.Thmlemma2)withν=νi\\nu=\\nu\_\{i\},d=did=d\_\{i\},ϱ=ρi\\varrho=\\rho\_\{i\}andyk=1−bi​\(k​T\)y\_\{k\}=1\-b\_\{i\}\(kT\)fork∈ℕ∪\{0\}k\\in\\mathbb\{N\}\\cup\\\{0\\\}, leads to:

Vi​\(k​T\)≤C⁡\(T,θ,ρi\)​di​exp⁡\(−ρi​k​T\),k∈ℕ\.\\displaystyle V\_\{i\}\(kT\)\\leq C\(T,\\theta,\\rho\_\{i\}\)\\,d\_\{i\}\\,\\exp\(\-\\rho\_\{i\}kT\),\\qquad k\\in\\mathbb\{N\}\.\(18\)
To conclude, letVi\(0\)∈\[0,ci\[V\_\{i\}\(0\)\\in\[0,c\_\{i\}\[andt∈ℝ0\+t\\in\\mathbb\{R\}\_\{0\}^\{\+\}\. For convenience, we writet=\(k−1\)​T\+τt=\(k\-1\)T\+\\taufor somek∈ℕk\\in\\mathbb\{N\}andτ∈\[0,T\[\\tau\\in\[0,T\[\\,\. As above, fromOPEN1\)1\)of Claim[IV\.1](https://arxiv.org/html/2609.17997#S4.Thmclaim1)and Proposition[IV\.1](https://arxiv.org/html/2609.17997#S4.Thmproposition1), we obtain:

V˙i≤−ηε​Vi\+M​∑j=1i−12​Mj​dj​exp⁡\(−ρj2​\(k−1\)​T\),\\displaystyle\\dot\{V\}\_\{i\}\\leq\-\\eta\_\{\\varepsilon\}\\,V\_\{i\}\+M\\sum\_\{j=1\}^\{i\-1\}\\sqrt\{2M\_\{j\}d\_\{j\}\}\\exp\\left\(\-\\frac\{\\rho\_\{j\}\}\{2\}\(k\-1\)T\\right\),for eachs∈\[\(k−1\)​T,t\]s\\in\[\(k\-1\)T,t\], where \([16](https://arxiv.org/html/2609.17997#S4.E16)\) has been used\. By integrating the previous expression between\(k−1\)​T\(k\-1\)Tandtt, we obtain:

Vi​\(t\)≤\\displaystyle V\_\{i\}\(t\)\\leqr​Vi​\(\(k−1\)​T\)\\displaystyle r\\,V\_\{i\}\(\(k\-1\)T\)\+H∑j=1i−12​Mj​djexp\(−ρj2\(k−1\)T\)\.\\displaystyle\+H\\,\\sum\_\{j=1\}^\{i\-1\}\\sqrt\{2M\_\{j\}d\_\{j\}\}\\exp\\left\(\-\\frac\{\\rho\_\{j\}\}\{2\}\(k\-1\)T\\right\)\.
By gathering the previous expression and \([18](https://arxiv.org/html/2609.17997#S4.E18)\), we conclude:

Vi​\(t\)\\displaystyle V\_\{i\}\(t\)≤\(C⁡\(T,θ,ρi\)​r\+1−θ\)​di​exp⁡\(−ρi​\(k−1\)​T\)\\displaystyle\\leq\\big\(C\(T,\\theta,\\rho\_\{i\}\)\\,r\+1\-\\theta\\big\)\\,d\_\{i\}\\,\\exp\(\-\\rho\_\{i\}\(k\-1\)T\)≤Mi​di​exp⁡\(−ρi​t\),\\displaystyle\\leq M\_\{i\}\\,d\_\{i\}\\,\\exp\(\-\\rho\_\{i\}t\),where we used \([13](https://arxiv.org/html/2609.17997#S4.E13)\), \([14](https://arxiv.org/html/2609.17997#S4.E14)\) andH​∑j=1i−12​Mj​dj≤\(1−θ\)​diH\\sum\_\{j=1\}^\{i\-1\}\\sqrt\{2M\_\{j\}d\_\{j\}\}\\leq\(1\-\\theta\)d\_\{i\}, as well as \([15](https://arxiv.org/html/2609.17997#S4.E15)\)\. ∎

As mentioned in Remark[IV\.1](https://arxiv.org/html/2609.17997#S4.Thmremark1), whenSB​C​\(t\)S\_\{BC\}\(t\)is locally positive definite, Assumption[IV\.1](https://arxiv.org/html/2609.17997#S4.Thmassumption1)is automatically satisfied, which yields the following particular case of the previous result\.

###### Corollary IV\.1

Suppose that Assumption[III\.1](https://arxiv.org/html/2609.17997#S3.Thmassumption1)holds and there existsc∈\]0,1\[c\\in\\,\]0,1\[such that:

inf\(t,z\)∈ℝ0\+×Ωc\+1​\(μ~\)z⊤​SB​C​\(t\)​z∈ℝ\+\.\\displaystyle\\inf\_\{\(t,z\)\\in\\mathbb\{R\}\_\{0\}^\{\+\}\\times\\Omega\_\{c\}^\{\+1\}\(\\tilde\{\\mu\}\)\}~z^\{\\top\}\\,S\_\{BC\}\(t\)\\,z\\in\\mathbb\{R\}^\{\+\}\.Then, for eachΣ=\(σ1,…,σℓ\)∈\{−1,\+1\}ℓ\\Sigma=\(\\sigma\_\{1\},\\ldots,\\sigma\_\{\\ell\}\)\\in\\\{\-1,\+1\\\}^\{\\ell\}, the consensus equilibriumZΣ∗=\(σ1​𝔢μ~,…,σℓ​𝔢μ~\)∈𝒞ℓ​\(μ~\)Z\_\{\\Sigma\}^\{\*\}=\(\\sigma\_\{1\}\\,\\mathfrak\{e\}^\{\\tilde\{\\mu\}\},\\ldots,\\sigma\_\{\\ell\}\\mathfrak\{e\}^\{\\tilde\{\\mu\}\}\)\\in\\mathcal\{C\}\_\{\\ell\}\(\\tilde\{\\mu\}\)is locally exponentially stable for the continuous SSM dynamics \([5](https://arxiv.org/html/2609.17997#S2.E5)\)\.

As mentioned in Section[II\-B](https://arxiv.org/html/2609.17997#S2.SS2), different Mamba\-2 variants employ different nonlinearities, and the model adopted in this paper does not include the SiLU nonlinearity present in the Mamba\-2 model considered in Section[VI](https://arxiv.org/html/2609.17997#S6)\. In that architecture, SiLU is applied to the branches producingBBandCC, so thatSB​\(k\)​zj​\(k\)S\_\{B\}\(k\)z\_\{j\}\(k\)andSC​\(k\)​zi​\(k\)S\_\{C\}\(k\)z\_\{i\}\(k\)are replaced byς⁡\(SB​\(k\)​zj​\(k\)\)\\varsigma\(S\_\{B\}\(k\)z\_\{j\}\(k\)\)andς⁡\(SC​\(k\)​zi​\(k\)\)\\varsigma\(S\_\{C\}\(k\)z\_\{i\}\(k\)\), respectively\. Assumption[IV\.1](https://arxiv.org/html/2609.17997#S4.Thmassumption1)and Theorem[IV\.1](https://arxiv.org/html/2609.17997#S4.Thmtheorem1)can be suitably reformulated to account for this modification, and the corresponding arguments carry over*mutatis mutandis*after replacing the bilinear interaction term by its nonlinear counterpart\.

## VGlobal positive definiteness of the interaction kernel

In this section, the objective is to describe the domain of attraction of the consensus equilibria\. An attentive reader may note that Assumption[IV\.1](https://arxiv.org/html/2609.17997#S4.Thmassumption1)is in practice difficult to verify, as it requires computing an integral of an infimum over the embedding space in which the tokens lie\. To obtain a more practical characterization, we strengthen the assumption to global positive definiteness\. A PE variant could be considered as well, with qualitatively similar results, nonetheless we adopt the pointwise condition here for cleaner exposition and ease of comparison with the standard positive\-definite setting\.

###### Assumption V\.1

There existsα∈ℝ\+\\alpha\\in\\mathbb\{R\}^\{\+\}such thatSB​C​\(t\)≻α​𝕀nS\_\{BC\}\(t\)\\succ\\alpha\\,\\mathbb\{I\}\_\{n\}for eacht∈ℝ0\+t\\in\\mathbb\{R\}\_\{0\}^\{\+\}\.

Let us introduce some parameters that will be helpful to characterize the domain of attraction of the consensus equilibria:

1. 1\.*Anisotropy measure*: c⋆=supt∈ℝ0\+‖SB​C​\(t\)−\(\(𝔢μ~\)⊤​SB​C​\(t\)​𝔢μ~\)​𝕀n‖inf\(t,z\)∈ℝ0\+×𝕊n−1z⊤​SB​C​\(t\)​z\.\\displaystyle c\_\{\\star\}=\\frac\{\\sup\_\{t\\in\\mathbb\{R\}\_\{0\}^\{\+\}\}\\\|S\_\{BC\}\(t\)\-\(\(\\mathfrak\{e\}^\{\\tilde\{\\mu\}\}\)^\{\\top\}\\,S\_\{BC\}\(t\)\\,\\mathfrak\{e\}^\{\\tilde\{\\mu\}\}\)\\mathbb\{I\}\_\{n\}\\\|\}\{\\inf\_\{\(t,z\)\\in\\mathbb\{R\}\_\{0\}^\{\+\}\\times\\mathbb\{S\}^\{n\-1\}\}z^\{\\top\}\\,S\_\{BC\}\(t\)\\,z\}\.
2. 2\.*Uniform eigenvalue upper bound*: λmax=max⁡sup\(t,Z\)∈ℝ0\+×\(𝕊n−1\)ℓ1≤j≤i≤ℓ⁡λi​jμ~​\(t,Z\)\.\\displaystyle\\lambda\_\{\\max\}=\\max\_\{1\\leq j\\leq i\\leq\\ell\}~\\sup\_\{\(t,Z\)\\in\\mathbb\{R\}\_\{0\}^\{\+\}\\times\(\\mathbb\{S\}^\{n\-1\}\)^\{\\ell\}\}~\\lambda\_\{ij\}^\{\\tilde\{\\mu\}\}\(t,Z\)\.
3. 3\.*Uniform eigenvalue lower bound*: λmin=min⁡inf\(t,Z\)∈ℝ0\+×\(𝕊n−1\)ℓ1≤j≤i≤ℓ⁡λi​jμ~​\(t,Z\)\.\\displaystyle\\lambda\_\{\\min\}=\\min\_\{1\\leq j\\leq i\\leq\\ell\}~\\inf\_\{\(t,Z\)\\in\\mathbb\{R\}\_\{0\}^\{\+\}\\times\(\\mathbb\{S\}^\{n\-1\}\)^\{\\ell\}\}~\\lambda\_\{ij\}^\{\\tilde\{\\mu\}\}\(t,Z\)\.
4. 4\.*Effective threshold*: ceff=c⋆​λmax\(γ\+\(ℓ−1\)​λmin\)2\+c⋆2​λmax2​\(ℓ−1\)2\\displaystyle c\_\{\\textrm\{eff\}\}=\\frac\{c\_\{\\star\}\\,\\lambda\_\{\\max\}\}\{\\sqrt\{\(\\gamma\+\(\\ell\-1\)\\,\\lambda\_\{\\min\}\)^\{2\}\+c\_\{\\star\}^\{2\}\\,\\lambda\_\{\\max\}^\{2\}\\,\(\\ell\-1\)^\{2\}\}\}

From Lemma[III\.1](https://arxiv.org/html/2609.17997#S3.Thmlemma1), it is clear thatc⋆∈ℝ0\+c\_\{\\star\}\\in\\mathbb\{R\}\_\{0\}^\{\+\},λmax,λmin∈ℝ\+\\lambda\_\{\\max\},\\lambda\_\{\\min\}\\in\\mathbb\{R\}^\{\+\}andceff∈\[0,1\[c\_\{\\textrm\{eff\}\}\\in\\,\[0,1\[\\,\.

Let us introduce the*trapping set*as:

ℬℓ​\(μ~\)=\\displaystyle\\mathcal\{B\}\_\{\\ell\}\(\\tilde\{\\mu\}\)=\{Z0∈\(𝕊n−1\)ℓ∣∃i∈\{1,…,ℓ\}\\displaystyle\\\{Z\_\{0\}\\in\(\\mathbb\{S\}^\{n\-1\}\)^\{\\ell\}\\mid~\\exists\\,i\\in\\\{1,\\ldots,\\ell\\\}such thatlim supt→∞\|ziμ~\(t,Z0\)\|≤ceff\},\\displaystyle~\\text\{such that\}~\\limsup\_\{t\\to\\infty\}\|z\_\{i\}^\{\\tilde\{\\mu\}\}\(t,Z\_\{0\}\)\|\\leq c\_\{\\textrm\{eff\}\}\\\},whereZ⁡\(t,Z0\)=\(z1​\(⋅,Z0\),…,zℓ​\(⋅,Z0\)\):ℝ0\+→\(𝕊n−1\)ℓZ\(t,Z\_\{0\}\)=\(z\_\{1\}\(\\cdot,Z\_\{0\}\),\\ldots,z\_\{\\ell\}\(\\cdot,Z\_\{0\}\)\):\\mathbb\{R\}\_\{0\}^\{\+\}\\to\(\\mathbb\{S\}^\{n\-1\}\)^\{\\ell\}denotes the solution of \([5](https://arxiv.org/html/2609.17997#S2.E5)\) with initial conditionZ0Z\_\{0\}att=0t=0\.

As in section[IV](https://arxiv.org/html/2609.17997#S4), we work with the scalar variables \([7](https://arxiv.org/html/2609.17997#S4.E7)\), the functions \([8](https://arxiv.org/html/2609.17997#S4.E8)\) and the errors \([9](https://arxiv.org/html/2609.17997#S4.E9)\)\. Analogous to Proposition[IV\.1](https://arxiv.org/html/2609.17997#S4.Thmproposition1), the following result, whose proof can be found in the Appendix, provides bounds that will be useful in the proof of Theorem[V\.1](https://arxiv.org/html/2609.17997#S5.Thmtheorem1)\.

###### Proposition V\.1

LetZ0∈\(𝕊n−1\)ℓ−ℬℓ​\(μ~\)Z\_\{0\}\\in\(\\mathbb\{S\}^\{n\-1\}\)^\{\\ell\}\-\\mathcal\{B\}\_\{\\ell\}\(\\tilde\{\\mu\}\),Σ∈\{−1,\+1\}ℓ\\Sigma\\in\\\{\-1,\+1\\\}^\{\\ell\}, andc∈\]ceff,bmax\[c\\in\\,\]c\_\{\\textrm\{eff\}\},b^\{\\textrm\{max\}\}\[\\,, wherebmax=min⁡lim supt→∞i∈\{1,…,ℓ\}⁡\|bi​\(t\)\|b^\{\\textrm\{max\}\}=\\min\_\{i\\in\\\{1,\\ldots,\\ell\\\}\}\\limsup\_\{t\\to\\infty\}\|b\_\{i\}\(t\)\|\. Then, under Assumptions[III\.1](https://arxiv.org/html/2609.17997#S3.Thmassumption1)and[V\.1](https://arxiv.org/html/2609.17997#S5.Thmassumption1), there existκ,M∈ℝ\+\\kappa,M\\in\\mathbb\{R\}^\{\+\}such that, for eacht∈Ic,iΣ​\(Z0,μ~\)=\{t∈ℝ0\+∣bi​\(t\)∈\[c,1\]\}t\\in I\_\{c,i\}^\{\\Sigma\}\(Z\_\{0\},\\tilde\{\\mu\}\)=\\\{t\\in\\mathbb\{R\}\_\{0\}^\{\+\}\\mid b\_\{i\}\(t\)\\in\[c,1\]\\\}, we have:

b˙i\\displaystyle\\dot\{b\}\_\{i\}≥κ⁡\(1−bi\)−M​∑j=1i−1\|ej​\(t\)\|,i∈\{1,…,ℓ\}\.\\displaystyle\\geq\\kappa\\,\(1\-b\_\{i\}\)\-M\\sum\_\{j=1\}^\{i\-1\}\|e\_\{j\}\(t\)\|,\\qquad i\\in\\\{1,\\ldots,\\ell\\\}\.

As in section[IV](https://arxiv.org/html/2609.17997#S4), we start by studying the dynamics of the first token\.

###### Lemma V\.1\(Asymptotic behavior of first token\)

Under Assumptions[III\.1](https://arxiv.org/html/2609.17997#S3.Thmassumption1)and[V\.1](https://arxiv.org/html/2609.17997#S5.Thmassumption1), for eachσ1∈\{−1,\+1\}\\sigma\_\{1\}\\in\\\{\-1,\+1\\\}, the pointzσ1∗=σ1​𝔢μ~∈𝒞1​\(μ~\)z\_\{\\sigma\_\{1\}\}^\{\*\}=\\sigma\_\{1\}\\,\\mathfrak\{e\}^\{\\tilde\{\\mu\}\}\\in\\mathcal\{C\}\_\{1\}\(\\tilde\{\\mu\}\)is an exponentially stable equilibrium for the dynamics of the first token of model \([5](https://arxiv.org/html/2609.17997#S2.E5)\) with domain of attractionΩ0σ1​\(μ~\)\\Omega\_\{0\}^\{\\sigma\_\{1\}\}\(\\tilde\{\\mu\}\)\.

###### Proof:

In terms of the scalar variable \([7](https://arxiv.org/html/2609.17997#S4.E7)\), the dynamics of the first token reads:

b˙1=\(z1⊤​SB​C​\(t\)​z1\)​\(λ11μ~​\(t,z1\)−z1⊤​D11​\(t\)​z1\)​b1,\\displaystyle\\dot\{b\}\_\{1\}=\(z\_\{1\}^\{\\top\}\\,S\_\{BC\}\(t\)\\,z\_\{1\}\)\\left\(\\lambda\_\{11\}^\{\\tilde\{\\mu\}\}\(t,z\_\{1\}\)\-z\_\{1\}^\{\\top\}\\,D\_\{11\}\(t\)\\,z\_\{1\}\\right\)b\_\{1\},\(19\)for each\(t,b1\)∈ℝ0\+×\[0,1\]\(t,b\_\{1\}\)\\in\\mathbb\{R\}\_\{0\}^\{\+\}\\times\\,\[0,1\]\. The only equilibria areb1∗=−1b\_\{1\}^\{\*\}=\-1,b1∗=0b\_\{1\}^\{\*\}=0, andb1∗=1b\_\{1\}^\{\*\}=1, which correspond toz1∗=−zσ1∗z\_\{1\}^\{\*\}=\-z\_\{\\sigma\_\{1\}\}^\{\*\},z1∗∈ℰ⁡\(μ~\)z\_\{1\}^\{\*\}\\in\\mathcal\{E\}\(\\tilde\{\\mu\}\)andz1∗=zσ1∗z\_\{1\}^\{\*\}=z\_\{\\sigma\_\{1\}\}^\{\*\}, respectively\. Given that:

λ11μ~​\(t,z1\)−z1⊤​D11​\(t,z1\)​z1\\displaystyle\\lambda\_\{11\}^\{\\tilde\{\\mu\}\}\(t,z\_\{1\}\)\-z\_\{1\}^\{\\top\}\\,D\_\{11\}\(t,z\_\{1\}\)\\,z\_\{1\}=∑μ=1n\(λ11μ~​\(t,z1\)−λ11μ​\(t,z1\)\)​\(z1μ\)2\\displaystyle\\quad=\\sum\_\{\\mu=1\}^\{n\}\\left\(\\lambda\_\{11\}^\{\\tilde\{\\mu\}\}\(t,z\_\{1\}\)\-\\lambda\_\{11\}^\{\\mu\}\(t,z\_\{1\}\)\\right\)\(z\_\{1\}^\{\\mu\}\)^\{2\}=∑μ≠μ~\(λ11μ~​\(t,z1\)−λ11μ​\(t,z1\)\)​\(z1μ\)2\\displaystyle\\quad=\\sum\_\{\\mu\\neq\\tilde\{\\mu\}\}\\left\(\\lambda\_\{11\}^\{\\tilde\{\\mu\}\}\(t,z\_\{1\}\)\-\\lambda\_\{11\}^\{\\mu\}\(t,z\_\{1\}\)\\right\)\(z\_\{1\}^\{\\mu\}\)^\{2\}≥γ⁡\(1−b12\)\.\\displaystyle\\quad\\geq\\gamma\(1\-b\_\{1\}^\{2\}\)\.\(20\)where we used\|z1\|=1\|z\_\{1\}\|=1, we conclude thatb˙1≥α​γ​\(1−b12\)​b1\\dot\{b\}\_\{1\}\\geq\\alpha\\,\\gamma\\,\(1\-b\_\{1\}^\{2\}\)\\,b\_\{1\}, for eachb1∈\[0,1\]b\_\{1\}\\in\[0,1\]\. Given thata1∗=1a\_\{1\}^\{\*\}=1is an exponentially stable equilibrium of the systema˙1=\(1−a12\)​a1\\dot\{a\}\_\{1\}=\(1\-a\_\{1\}^\{2\}\)\\,a\_\{1\},a1∈\[0,1\]a\_\{1\}\\in\[0,1\], with domain of attraction\]0,1\]\]0,1\], the Grönwall–Bellman inequality ensures thatb1⋆=1b\_\{1\}^\{\\star\}=1is an exponentially stable equilibrium of \([19](https://arxiv.org/html/2609.17997#S5.E19)\) with domain of attraction\]0,1\]\]0,1\]\. This corresponds toz1⋆=zσ1⋆z\_\{1\}^\{\\star\}=z\_\{\\sigma\_\{1\}\}^\{\\star\}being an exponentially stable equilibrium of the dynamics of the first token of \([5](https://arxiv.org/html/2609.17997#S2.E5)\) with domain of attractionΩ0σ1​\(μ~\)\\Omega\_\{0\}^\{\\sigma\_\{1\}\}\(\\tilde\{\\mu\}\)\. ∎

We are ready to find the domain of attraction and study the stability of the consensus equilibria for the SSM dynamics\.

###### Theorem V\.1\(Asymptotic stability of consensus\)

Under Assumptions[III\.1](https://arxiv.org/html/2609.17997#S3.Thmassumption1)and[V\.1](https://arxiv.org/html/2609.17997#S5.Thmassumption1), the consensus set𝒞ℓ​\(μ~\)\\mathcal\{C\}\_\{\\ell\}\(\\tilde\{\\mu\}\)is exponentially stable for the continuous SSM dynamics \([5](https://arxiv.org/html/2609.17997#S2.E5)\) with domain of attraction:

𝒟ℓ​\(μ~\)=\(𝕊n−1\)ℓ−ℬℓ​\(μ~\)\\displaystyle\\mathcal\{D\}\_\{\\ell\}\(\\tilde\{\\mu\}\)=\(\\mathbb\{S\}^\{n\-1\}\)^\{\\ell\}\-\\mathcal\{B\}\_\{\\ell\}\(\\tilde\{\\mu\}\)=\{Z0∈\(𝕊n−1\)ℓ∣lim supt→∞\|ziμ~\(t,Z0\)\|\>ceff,1≤i≤ℓ\}\.\\displaystyle=\\\{Z\_\{0\}\\in\(\\mathbb\{S\}^\{n\-1\}\)^\{\\ell\}\\mid\\limsup\_\{t\\to\\infty\}\|z\_\{i\}^\{\\tilde\{\\mu\}\}\(t,Z\_\{0\}\)\|\>\{c\_\{\\textrm\{eff\}\}\},~1\\leq i\\leq\\ell\\\}\.

###### Proof:

Given that Assumption[V\.1](https://arxiv.org/html/2609.17997#S5.Thmassumption1)is stronger than Assumption[IV\.1](https://arxiv.org/html/2609.17997#S4.Thmassumption1), Theorem[IV\.1](https://arxiv.org/html/2609.17997#S4.Thmtheorem1)holds, whence𝒞ℓ​\(μ~\)\\mathcal\{C\}\_\{\\ell\}\(\\tilde\{\\mu\}\)is locally exponentially stable\. Thus, it is enough to show that its domain of attraction is𝒟ℓ​\(μ~\)=\(𝕊n−1\)ℓ−ℬℓ​\(μ~\)\\mathcal\{D\}\_\{\\ell\}\(\\tilde\{\\mu\}\)=\(\\mathbb\{S\}^\{n\-1\}\)^\{\\ell\}\-\\mathcal\{B\}\_\{\\ell\}\(\\tilde\{\\mu\}\),i\.e\.,

limt→∞Z⁡\(t,Z0\)∈𝒞ℓ​\(μ~\)⇔Z0∈𝒟ℓ​\(μ~\)\.\\displaystyle\\lim\_\{t\\to\\infty\}Z\(t,Z\_\{0\}\)\\in\\mathcal\{C\}\_\{\\ell\}\(\\tilde\{\\mu\}\)~\\Leftrightarrow~Z\_\{0\}\\in\\mathcal\{D\}\_\{\\ell\}\(\\tilde\{\\mu\}\)\.
To specify the dependence on the initial conditionZ0∈\(𝕊n−1\)ℓZ\_\{0\}\\in\(\\mathbb\{S\}^\{n\-1\}\)^\{\\ell\}, as well on the signsΣ=\(σ1,…,σℓ\)∈\{−1,\+1\}ℓ\\Sigma=\(\\sigma\_\{1\},\\ldots,\\sigma\_\{\\ell\}\)\\in\\\{\-1,\+1\\\}^\{\\ell\}, the scalar functions \([7](https://arxiv.org/html/2609.17997#S4.E7)\) and error functions \([9](https://arxiv.org/html/2609.17997#S4.E9)\) corresponding to theii\-th component of the solutionZ⁡\(t,Z0\)Z\(t,Z\_\{0\}\)of \([5](https://arxiv.org/html/2609.17997#S2.E5)\) are denoted bybiΣ​\(t,Z0\)b\_\{i\}^\{\\Sigma\}\(t,Z\_\{0\}\)andeiΣ​\(t,Z0\)e\_\{i\}^\{\\Sigma\}\(t,Z\_\{0\}\), respectively, for eachi∈\{1,…,ℓ\}i\\in\\\{1,\\ldots,\\ell\\\}\.

\(⇒\)\(\\Rightarrow\)LetZ0∈\(𝕊n−1\)ℓZ\_\{0\}\\in\(\\mathbb\{S\}^\{n\-1\}\)^\{\\ell\}be such thatlimt→∞Z⁡\(t,Z0\)∈𝒞ℓ​\(μ~\)\\lim\_\{t\\to\\infty\}Z\(t,Z\_\{0\}\)\\in\\mathcal\{C\}\_\{\\ell\}\(\\tilde\{\\mu\}\),i\.e\.,limt→∞\|biΣ​\(t,Z0\)\|=1\\lim\_\{t\\to\\infty\}\|b\_\{i\}^\{\\Sigma\}\(t,Z\_\{0\}\)\|=1for each1≤i≤ℓ1\\leq i\\leq\\ellandΣ∈\{−1,\+1\}ℓ\\Sigma\\in\\\{\-1,\+1\\\}^\{\\ell\}\. By contradiction, suppose thatZ0∈ℬℓ​\(μ~\)Z\_\{0\}\\in\\mathcal\{B\}\_\{\\ell\}\(\\tilde\{\\mu\}\)\. Then there existsi∈\{1,…,ℓ\}i\\in\\\{1,\\ldots,\\ell\\\}such thatlim supt→∞zi​\(t,Z0\)∉Ωceffσ​\(μ~\)\\limsup\_\{t\\to\\infty\}z\_\{i\}\(t,Z\_\{0\}\)\\not\\in\\Omega\_\{c\_\{\\textrm\{eff\}\}\}^\{\\sigma\}\(\\tilde\{\\mu\}\), i\.e\.:

lim supt→∞\|biΣ​\(t,Z0\)\|\\displaystyle\\limsup\_\{t\\to\\infty\}\|b\_\{i\}^\{\\Sigma\}\(t,Z\_\{0\}\)\|≤ceff<1=limt→∞\|biΣ​\(t,Z0\)\|\.\\displaystyle\\leq c\_\{\\textrm\{eff\}\}<1=\\lim\_\{t\\to\\infty\}\|b\_\{i\}^\{\\Sigma\}\(t,Z\_\{0\}\)\|\.As a result,Z0∉ℬℓ​\(μ~\)Z\_\{0\}\\not\\in\\mathcal\{B\}\_\{\\ell\}\(\\tilde\{\\mu\}\),i\.e\.,Z0∈𝒟ℓ​\(μ~\)Z\_\{0\}\\in\\mathcal\{D\}\_\{\\ell\}\(\\tilde\{\\mu\}\)\.

\(⇐\)\(\\Leftarrow\)Let us proceed by induction in the number of tokens\. For each1≤i≤ℓ1\\leq i\\leq\\ell, we denoteZ\(i\)=\(z1,…,zi\)∈\(𝕊n−1\)iZ^\{\(i\)\}=\(z\_\{1\},\\ldots,z\_\{i\}\)\\in\(\\mathbb\{S\}^\{n\-1\}\)^\{i\}andΣ\(i\)=\(σ1,…,σi\)∈\{−1,\+1\}i\\Sigma^\{\(i\)\}=\(\\sigma\_\{1\},\\ldots,\\sigma\_\{i\}\)\\in\\\{\-1,\+1\\\}^\{i\}\. Similarly, the solution of the subsystem of \([5](https://arxiv.org/html/2609.17997#S2.E5)\) given by the firstiitokens with initial conditionZ0\(i\)∈\(𝕊n−1\)iZ\_\{0\}^\{\(i\)\}\\in\(\\mathbb\{S\}^\{n\-1\}\)^\{i\}is denoted byZ\(i\)​\(t,Z0\(i\)\)=\(z1\(i\)​\(t,Z0\(i\)\),…,zi\(i\)​\(t,Z0\(i\)\)\)Z^\{\(i\)\}\(t,Z\_\{0\}^\{\(i\)\}\)=\(z\_\{1\}^\{\(i\)\}\(t,Z\_\{0\}^\{\(i\)\}\),\\ldots,z\_\{i\}^\{\(i\)\}\(t,Z\_\{0\}^\{\(i\)\}\)\), for eacht∈ℝ0\+t\\in\\mathbb\{R\}\_\{0\}^\{\+\}\.

Base case\.From Lemma[V\.1](https://arxiv.org/html/2609.17997#S5.Thmlemma1),𝒞1​\(μ~\)\\mathcal\{C\}\_\{1\}\(\\tilde\{\\mu\}\)is exponentially stable for the subsystem of \([5](https://arxiv.org/html/2609.17997#S2.E5)\) given by the first token, and its domain of attraction isΩ0​\(μ~\)\\Omega\_\{0\}\(\\tilde\{\\mu\}\)\. In particular,ℬ1​\(μ~\)=ℰ​\(μ~\)\\mathcal\{B\}\_\{1\}\(\\tilde\{\\mu\}\)=\\mathcal\{E\}\(\\tilde\{\\mu\}\)\.

Induction hypothesis\.Let1<i≤ℓ1<i\\leq\\ell\. For eachZ0\(i−1\)∈𝒟i−1​\(μ~\)Z\_\{0\}^\{\(i\-1\)\}\\in\\mathcal\{D\}\_\{i\-1\}\(\\tilde\{\\mu\}\), we have:

limt→∞Z\(i−1\)​\(t,Z0\(i−1\)\)∈𝒞i−1​\(μ~\)\.\\displaystyle\\lim\_\{t\\to\\infty\}Z^\{\(i\-1\)\}\(t,Z\_\{0\}^\{\(i\-1\)\}\)\\in\\mathcal\{C\}\_\{i\-1\}\(\\tilde\{\\mu\}\)\.Equivalently, there existsΣ\(i−1\)=\(σ1,…,σi−1\)∈\{−1,\+1\}i−1\\Sigma^\{\(i\-1\)\}=\(\\sigma\_\{1\},\\ldots,\\sigma\_\{i\-1\}\)\\in\\\{\-1,\+1\\\}^\{i\-1\}such that:

limt→∞ejΣ\(j\)​\(t,Z0\(j\)\)=0,1≤j≤i−1,\\displaystyle\\lim\_\{t\\to\\infty\}e\_\{j\}^\{\\Sigma^\{\(j\)\}\}\(t,Z\_\{0\}^\{\(j\)\}\)=0,~1\\leq j\\leq i\-1,\(21\)whereZ0\(j\)=\(z1,0,…,zj,0\)∈𝒟j​\(μ~\)Z\_\{0\}^\{\(j\)\}=\(z\_\{1,0\},\\ldots,z\_\{j,0\}\)\\in\\mathcal\{D\}\_\{j\}\(\\tilde\{\\mu\}\)andΣ\(j\)=\(σ1,…,σj\)∈\{−1,\+1\}j\\Sigma^\{\(j\)\}=\(\\sigma\_\{1\},\\ldots,\\sigma\_\{j\}\)\\in\\\{\-1,\+1\\\}^\{j\}\.

Inductive step\.For eachZ0\(i\)=\(z1,0,…,zi,0\)∈𝒟i​\(μ~\)Z\_\{0\}^\{\(i\)\}=\(z\_\{1,0\},\\ldots,z\_\{i,0\}\)\\in\\mathcal\{D\}\_\{i\}\(\\tilde\{\\mu\}\), we can pickc∈\]ceff,bimax\[c\\in\\,\]c\_\{\\textrm\{eff\}\},b\_\{i\}^\{\\max\}\[\\,, wherebimax=lim supt→∞\|ziμ~​\(t,Z0\)\|b\_\{i\}^\{\\max\}=\\limsup\_\{t\\to\\infty\}\|z\_\{i\}^\{\\tilde\{\\mu\}\}\(t,Z\_\{0\}\)\|\. In addition, thanks to the induction hypothesis and the fact thatZ0\(i−1\)∈𝒟i−1​\(μ~\)Z\_\{0\}^\{\(i\-1\)\}\\in\\mathcal\{D\}\_\{i\-1\}\(\\tilde\{\\mu\}\), there existsΣ\(i−1\)=\(σ1,…,σi−1\)∈\{−1,\+1\}i−1\\Sigma^\{\(i\-1\)\}=\(\\sigma\_\{1\},\\ldots,\\sigma\_\{i\-1\}\)\\in\\\{\-1,\+1\\\}^\{i\-1\}such that \([21](https://arxiv.org/html/2609.17997#S5.E21)\) holds\. By takingκ,M∈ℝ\+\\kappa,M\\in\\mathbb\{R\}^\{\+\}as in Proposition[V\.1](https://arxiv.org/html/2609.17997#S5.Thmproposition1), there existst0∈ℝ0\+t\_\{0\}\\in\\mathbb\{R\}\_\{0\}^\{\+\}such that:

M∑j=1i−1\|ejΣ\(j\)\(t,Z0j\)\|<κ\(1−c\),t∈\[t0,∞\[\.\\displaystyle M\\sum\_\{j=1\}^\{i\-1\}\|e\_\{j\}^\{\\Sigma^\{\(j\)\}\}\(t,Z\_\{0\}^\{j\}\)\|<\\kappa\\,\(1\-c\),\\qquad t\\in\[t\_\{0\},\\infty\[\\,\.\(22\)Moreover, given thatbimax\>cb\_\{i\}^\{\\max\}\>c, there existst1≥t0t\_\{1\}\\geq t\_\{0\}such that\|ziμ~​\(t1,Z0\)\|\>c\|z\_\{i\}^\{\\tilde\{\\mu\}\}\(t\_\{1\},Z\_\{0\}\)\|\>c\. Letσi=sign⁡\(ziμ~​\(t1,Z0\)\)\\sigma\_\{i\}=\\operatorname\{sign\}\(z\_\{i\}^\{\\tilde\{\\mu\}\}\(t\_\{1\},Z\_\{0\}\)\)and setΣ\(i\)=\(σ1,…,σi−1,σi\)∈\{−1,\+1\}i\\Sigma^\{\(i\)\}=\(\\sigma\_\{1\},\\ldots,\\sigma\_\{i\-1\},\\sigma\_\{i\}\)\\in\\\{\-1,\+1\\\}^\{i\}\. With the notation of Proposition[V\.1](https://arxiv.org/html/2609.17997#S5.Thmproposition1),t1∈Ic,iΣ\(i\)​\(Z0\(i\),μ~\)t\_\{1\}\\in I\_\{c,i\}^\{\\Sigma^\{\(i\)\}\}\(Z\_\{0\}^\{\(i\)\},\\tilde\{\\mu\}\), wherebiΣ\(i\)​\(t1,Z0\(i\)\)=σi​ziμ~​\(t1,Z0\(i\)\)b\_\{i\}^\{\\Sigma^\{\(i\)\}\}\(t\_\{1\},Z\_\{0\}^\{\(i\)\}\)=\\sigma\_\{i\}\\,z\_\{i\}^\{\\tilde\{\\mu\}\}\(t\_\{1\},Z\_\{0\}^\{\(i\)\}\)\. Let us show that, in fact,Ic,iΣ\(i\)\(Z0\(i\),μ~\)=\[t1,∞\[I\_\{c,i\}^\{\\Sigma^\{\(i\)\}\}\(Z\_\{0\}^\{\(i\)\},\\tilde\{\\mu\}\)=\[t\_\{1\},\\infty\[\\,\. By contradiction, suppose that:

t⋆=inf\{t≥t1∣biΣ\(i\)​\(t,Z0\(i\)\)≤c\}∈ℝ\+\.\\displaystyle t\_\{\\star\}=\\inf\\\{t\\geq t\_\{1\}\\mid b\_\{i\}^\{\\Sigma^\{\(i\)\}\}\(t,Z\_\{0\}^\{\(i\)\}\)\\leq c\\\}\\in\\mathbb\{R\}^\{\+\}\.By continuity,biΣ\(i\)​\(t⋆,Z0\(i\)\)=cb\_\{i\}^\{\\Sigma^\{\(i\)\}\}\(t\_\{\\star\},Z\_\{0\}^\{\(i\)\}\)=c\. Hence, from Proposition[V\.1](https://arxiv.org/html/2609.17997#S5.Thmproposition1)and \([22](https://arxiv.org/html/2609.17997#S5.E22)\), we obtain \(the dependence on the initial conditionZ0\(i\)Z\_\{0\}^\{\(i\)\}is dropped for simplicity\):

b˙iΣ\(i\)​\(t⋆\)\\displaystyle\\dot\{b\}\_\{i\}^\{\\Sigma^\{\(i\)\}\}\(t\_\{\\star\}\)≥κ⁡\(1−biΣ\(i\)​\(t⋆\)\)−M​∑j=1i−1\|ejΣ\(j\)​\(t⋆\)\|\\displaystyle\\geq\\kappa\\,\(1\-b\_\{i\}^\{\\Sigma^\{\(i\)\}\}\(t\_\{\\star\}\)\)\-M\\sum\_\{j=1\}^\{i\-1\}\|e\_\{j\}^\{\\Sigma^\{\(j\)\}\}\(t\_\{\\star\}\)\|\>κ⁡\(1−c\)−κ⁡\(1−c\)=0\.\\displaystyle\>\\kappa\\,\(1\-c\)\-\\kappa\(1\-c\)=0\.This contradicts thatbiΣ\(i\)​\(t\)\>cb\_\{i\}^\{\\Sigma^\{\(i\)\}\}\(t\)\>cfor eacht∈\[t1,t⋆\[t\\in\[t\_\{1\},t\_\{\\star\}\[\\,\.

As a result, the bound in Proposition[V\.1](https://arxiv.org/html/2609.17997#S5.Thmproposition1)is valid onIc,iΣ\(i\)\(Z0\(i\),μ~\)=\[t1,∞\[I\_\{c,i\}^\{\\Sigma^\{\(i\)\}\}\(Z\_\{0\}^\{\(i\)\},\\tilde\{\\mu\}\)=\[t\_\{1\},\\infty\[\\,\. This, together with Lemma[\.3](https://arxiv.org/html/2609.17997#A0.Thmlemma3)and \([21](https://arxiv.org/html/2609.17997#S5.E21)\), gives the result\.

∎

For each token1≤i≤ℓ1\\leq i\\leq\\ell, the effective thresholdceffc\_\{\\textrm\{eff\}\}separates two regimes on𝕊n−1\\mathbb\{S\}^\{n\-1\}\. When\|ziμ~\|\>ceff\|z\_\{i\}^\{\\tilde\{\\mu\}\}\|\>c\_\{\\textrm\{eff\}\}, both the self\-drift and the isotropic part of the cross\-terms drivebi​\(t\)b\_\{i\}\(t\)towards±1\\pm 1ast→∞t\\to\\infty\. When\|ziμ~\|≤ceff\|z\_\{i\}^\{\\tilde\{\\mu\}\}\|\\leq c\_\{\\textrm\{eff\}\}, however, the anisotropic cross\-terms oppose the self\-drift\. The dominating term at the current time will determine the sign ofb˙i​\(t\)\\dot\{b\}\_\{i\}\(t\), which will change within time\.

From Lemma[V\.1](https://arxiv.org/html/2609.17997#S5.Thmlemma1), the trapping set of the first token is the equatorℰ⁡\(μ~\)\\mathcal\{E\}\(\\tilde\{\\mu\}\), which has zero measure \(as it has co\-dimension 1\)\. In other words, the domain of attraction𝒟1​\(μ~\)\\mathcal\{D\}\_\{1\}\(\\tilde\{\\mu\}\)is co\-null\. This follows from the fact that the dynamics of the first token has no cross\-terms\.

Let us analyze the size of the domain of attraction for the remaining tokens\. We can distinguish the two cases:

1. 1\.Isotropic case\.WhenSB​C​\(t\)=g⁡\(t\)​𝕀nS\_\{BC\}\(t\)=g\(t\)\\,\\mathbb\{I\}\_\{n\}for some continuousg:ℝ0\+→\[α,∞\[g:\\mathbb\{R\}\_\{0\}^\{\+\}\\to\[\\alpha,\\infty\[\\,, withα∈ℝ\+\\alpha\\in\\mathbb\{R\}^\{\+\}, the effective threshold vanishes:ceff=0c\_\{\\textrm\{eff\}\}=0\. In this case, the trapping set is defined by an equality and, thus, it has zero measure \(as it has positive co\-dimension\)\. Therefore, the domain of attraction𝒟ℓ​\(μ~\)\\mathcal\{D\}\_\{\\ell\}\(\\tilde\{\\mu\}\)is co\-null\.
2. 2\.Anisotropic case\.WhenSB​C​\(t\)S\_\{BC\}\(t\)is not a multiple of the identity,c⋆∈ℝ0\+c\_\{\\star\}\\in\\mathbb\{R\}\_\{0\}^\{\+\}and the trapping set may have positive measure\. Furthermore, there might be locally asymptotically stable equilibria inside the trapping set\. The size of the domain of attraction is a decreasing function of the effective threshold\.

## VIMamba\-2 Experiments

In this section, our objective is to verify whether the consensus phenomenon can be observed in the Mamba\-2 model as the number of layers increases\. To do so, we use themamba2\-130mmodel \(n=768n=768,2424layers,6464channels per head\) from\[[30](https://arxiv.org/html/2609.17997#bib.bib28)\]\. Since our results are asymptotic, we extend the depth by cycling the2424available layers: once the tokens have passed through the last layer, they re\-enter the first, so that the setting remains time\-varying\. We takeℓ=50\\ell=50tokens, initialised by uniformly sampling from the dictionary, and iterate the layer update for a depth of200200\. Each experiment is repeated for5050independent draws of the initial tokens, and we report the mean over those runs\.

WritingZ=\(z1,…,zℓ\)Z=\(z\_\{1\},\\dots,z\_\{\\ell\}\), letλ¯1≥⋯≥λ¯ℓ\\bar\{\\lambda\}\_\{1\}\\geq\\dots\\geq\\bar\{\\lambda\}\_\{\\ell\}denote the eigenvalues of the Gram matrix of the normalised tokenszi/\|zi\|z\_\{i\}/\|z\_\{i\}\|, scaled so that∑kλ¯k=1\\sum\_\{k\}\\bar\{\\lambda\}\_\{k\}=1\. We measure two quantities:

- •λ¯1\\bar\{\\lambda\}\_\{1\}, the share of the total energy carried by the leading direction\. It equals11exactly when all tokens are collinear, and1/ℓ1/\\ellwhen they are mutually orthogonal\.
- •r=\(∑kλ¯k2\)−1r=\\big\(\\sum\_\{k\}\\bar\{\\lambda\}\_\{k\}^\{2\}\\big\)^\{\-1\}, the participation ratio of that spectrum: the effective number of directions occupied by the tokens, ranging from11\(collinear\) toℓ\\ell\(isotropic\)\.

Both detect alignment along a common direction irrespective of the signsσi\\sigma\_\{i\},i\.e, at any consensus configuration in𝒞ℓ​\(μ~\)\\mathcal\{C\}\_\{\\ell\}\(\\tilde\{\\mu\}\)they satisfyλ¯1=1\\bar\{\\lambda\}\_\{1\}=1andr=1r=1\.

### VI\-AResults

![Refer to caption](https://arxiv.org/html/2609.17997v1/Normal.png)Fig\. 1:Weights of themamba2\-130mmodel, with and without the output gate\. Top:λ¯1\\bar\{\\lambda\}\_\{1\}\. Bottom:rr\.Figure[1](https://arxiv.org/html/2609.17997#S6.F1)reports the values ofrrandλ¯1\\bar\{\\lambda\}\_\{1\}obtained using the original Mamba\-2 weights and the SiLU nonlinearity, both with and without the output gate\. Recall from Section[II\-B](https://arxiv.org/html/2609.17997#S2.SS2)that the output gate is a nonlinear functiong⁡\(zi\)g\(z\_\{i\}\)that multiplies the output of the recurrence elementwise and is excluded from our model\. The results with and without the output gate follow the same qualitative trajectory: the tokens aggregate along a single direction over the first layers, after which the two quantities reach a plateau and remain there for the rest of the run\. What differs is how far the aggregation proceeds before settling\. Without the gate,λ¯1\\bar\{\\lambda\}\_\{1\}rises to approximately0\.80\.8andrrfalls to about1\.41\.4, against roughly0\.450\.45and44when the gate is retained\. This supports the claim that the gate is what keeps the tokens from converging to a single direction\.

![Refer to caption](https://arxiv.org/html/2609.17997v1/Singlelayer.png)Fig\. 2:A single layer of themamba2\-130mmodel applied repeatedly, with the output gate removed\. Top:λ¯1\\bar\{\\lambda\}\_\{1\}\. Bottom:rr\.In the second experiment, our objective was to investigate how the consensus phenomenon depends on the time\-varying nature of the weights\. To do so, we repeated the previous experiment without the gate, using the same layer at every depth, so that the weight matrices are time\-invariant and, in particular, the dominant eigenvalue ofDi​jD\_\{ij\}is attained at the same indexμ~\\tilde\{\\mu\}throughout\. In Figure[2](https://arxiv.org/html/2609.17997#S6.F2), it can be seen that the behaviour changes qualitatively:λ¯1\\bar\{\\lambda\}\_\{1\}exceeds0\.950\.95within the first few layers andrrdrops to approximately1\.051\.05, both remaining there for the rest of the run, so that the tokens are aligned along a single direction up to numerical accuracy\.

Comparing the two experiments, the weaker consensus in Figure[1](https://arxiv.org/html/2609.17997#S6.F1)can be attributed to the time variation of the weights\. When the weights vary with the layer, the indexμ~\\tilde\{\\mu\}of the dominant eigenvalue ofDi​jD\_\{ij\}need not remain constant, and as mentioned in Remark[III\.1](https://arxiv.org/html/2609.17997#S3.Thmassumption1), each time the index changes the tokens begin realigning towards a new principal eigenvector\. The convergence established on each interval of constancy is therefore interrupted before the tokens reach the corresponding equilibrium\. Holding the weights fixed removes these transitions, and the consensus phenomenon can be observed in full\.

![Refer to caption](https://arxiv.org/html/2609.17997v1/Random.png)Fig\. 3:Randomly drawn weight matrices, redrawn at every layer and with the output gate removed, with and without the nonlinearity on the branch producingBBandCC\. Top:λ¯1\\bar\{\\lambda\}\_\{1\}\. Bottom:rr\.In the third experiment our objective was to isolate the effect of the SiLU, the elementwise nonlinearityς⁡\(a\)=a​\(1\+e−a\)−1\\varsigma\(a\)=a\\,\(1\+e^\{\-a\}\)^\{\-1\},a∈ℝa\\in\\mathbb\{R\}, present in the Mamba\-2 architecture used in this section\. To do so, we draw the weight matrices independently at every layer, so that the layers are not periodic, and remove the gate, running the model with and without the SiLU\. Figure[3](https://arxiv.org/html/2609.17997#S6.F3)reports the values ofrrandλ¯1\\bar\{\\lambda\}\_\{1\}for the random model with and without SiLU\.

It can be seen that, for bothλ¯1\\bar\{\\lambda\}\_\{1\}andrr, the curves with and without the SiLU separate within the first few layers and then move in opposite directions\. With the nonlinearity,λ¯1\\bar\{\\lambda\}\_\{1\}increases steadily to approximately0\.880\.88whilerrdecreases to about1\.31\.3, so the tokens concentrate along a single direction\. Without it,λ¯1\\bar\{\\lambda\}\_\{1\}falls to roughly0\.230\.23whilerrgrows past1414and is still increasing at the end of the run: the tokens do not merely fail to reach consensus, they spread over an increasing number of directions\.

Lemma[IV\.1](https://arxiv.org/html/2609.17997#S4.Thmlemma1)accounts for the difference between the experiments with and without the SiLU\. With matrices drawn independently at every layer, which is the setting of this experiment, the interaction term is centered at00without the nonlinearity, whereas the SiLU shifts its mean to be strictly positive\. While this does not establish that Assumption[IV\.1](https://arxiv.org/html/2609.17997#S4.Thmassumption1)is satisfied, and hence does not guarantee that the tokens converge, it indicates that the interaction term is biased towards positive values, which is what drives the tokens together\.

### VI\-BPersistency of Excitation

Our results rely on Assumption[IV\.1](https://arxiv.org/html/2609.17997#S4.Thmassumption1)and in this section we report on experiments designed to test it\. We therefore turn to the assumption itself\. For the first token, item 1\) of Proposition[IV\.1](https://arxiv.org/html/2609.17997#S4.Thmproposition1)readsb˙1≥ac​\(t,z1\)​\(1−b1\)\\dot\{b\}\_\{1\}\\geq a\_\{c\}\(t,z\_\{1\}\)\(1\-b\_\{1\}\), so the quantity that governs its convergence isaca\_\{c\}evaluated along the trajectory\. We therefore record, at every layer and for each of the5050runs, the interaction term of each token:

qi​\(t\)=ς​\(SC​\(t\)​zi​\(t\)\)⊤​ς​\(SB​\(t\)​zi​\(t\)\),q\_\{i\}\(t\)\\;=\\;\\varsigma\\big\(S\_\{C\}\(t\)\\,z\_\{i\}\(t\)\\big\)^\{\\\!\\top\}\\varsigma\\big\(S\_\{B\}\(t\)\\,z\_\{i\}\(t\)\\big\),which reduces tozi​\(t\)⊤​SB​C​\(t\)​zi​\(t\)z\_\{i\}\(t\)^\{\\top\}S\_\{BC\}\(t\)\\,z\_\{i\}\(t\)when the nonlinearity is removed, and which agrees withac​\(t,zi​\(t\)\)a\_\{c\}\(t,z\_\{i\}\(t\)\)up to a positive constant, for eachtt\. For a window lengthTTwe compute the worst\-case average ofqiq\_\{i\}over all windows of that length contained in the run, and callTTadmissible when it is positive\. In the first two results we look for the smallest admissible window:

T⋆=min⁡\{1≤T≤κ\|min0≤t≤κ−T⁡1T​∑s=t\+1t\+Tqi​\(s\)\>0\},T^\{\\star\}\\;=\\;\\min\\left\\\{\\,1\\leq T\\leq\\kappa\\;\\middle\|\\;\\min\_\{0\\leq t\\leq\\kappa\-T\}\\;\\frac\{1\}\{T\}\\sum\_\{s=t\+1\}^\{t\+T\}q\_\{i\}\(s\)\\;\>\\;0\\,\\right\\\},\(23\)where the inner minimum is taken over all windows contained in the run, andT⋆T^\{\\star\}does not exist when no window length is admissible\. Note that the existence ofT∗T^\{\*\}does not establish Assumption[IV\.1](https://arxiv.org/html/2609.17997#S4.Thmassumption1), which requires the averaged condition at every point of the cap rather than along the sampled trajectories\. The two are nonetheless related: wheneverzi​\(t\)z\_\{i\}\(t\)lies in the cap,qi​\(t\)q\_\{i\}\(t\)is an upper bound forαc​\(t\)\\alpha\_\{c\}\(t\)up to a positive constant, so a negative average refutes the assumption, while a positive one across many trajectories is supporting evidence for it, without establishing it\.

Figures[4](https://arxiv.org/html/2609.17997#S6.F4)and[5](https://arxiv.org/html/2609.17997#S6.F5)illustrate the measurement on the first token of one run, with the original weights of the Mamba\-2 and for randomly drawn weights respectively, both without the output gate\. The sign ofq1q\_\{1\}alternates in both figures, so although pointwise positivity fails,q1​\(t\)q\_\{1\}\(t\)remains positive on average\. The minimum average is negative for short windows, but increases with the window length, and crosses zero atT=8T=8for the model andT=34T=34for the random weights\. With windows of at least that length the interaction term is positive on average along these trajectories, and hence so is the upper bound it provides forαc\\alpha\_\{c\}\.

Figure[6](https://arxiv.org/html/2609.17997#S6.F6)collects the four configurations, taking the largestT∗T^\{\*\}of \([23](https://arxiv.org/html/2609.17997#S6.E23)\) over all tokens and all runs\. With the original Mamba\-2 weights the smallest admissible window isT=10T=10, and removing the nonlinearity leaves it essentially unchanged atT=11T=11\. For matrices drawn independently at every layer, the smallest admissible window isT=35T=35with the nonlinearity, and no window length up to the length of the run is admissible without it,i\.e\.i\.e\.along these trajectories Assumption[IV\.1](https://arxiv.org/html/2609.17997#S4.Thmassumption1)is violated\. This behaviour is consistent with Lemma[IV\.1](https://arxiv.org/html/2609.17997#S4.Thmlemma1)\. The nonlinearity therefore accounts for the clustering seen with random matrices, while for the trained matrices the interaction term is positive on average along these trajectories whether or not the nonlinearity is present\.

Overall, along the trajectories we sample, the interaction term of the trained Mamba\-2 is positive on average over short windows compared with the depth simulated\. This does not establish Assumption[IV\.1](https://arxiv.org/html/2609.17997#S4.Thmassumption1), but it is consistent with it, and indicates that the mechanism analysed in Section[IV](https://arxiv.org/html/2609.17997#S4)is at work in Mamba\-2 itself and not only in the setting under which our results were proved\.

![Refer to caption](https://arxiv.org/html/2609.17997v1/AvgOrIl.png)Fig\. 4:Interaction term of the first token along one trajectory of themamba2\-130mmodel\. Top: its sign at each layer\. Middle: its average over windows of lengthT=8T=8\. Bottom: the worst\-case window average as a function of the window length, with the smallest admissibleTTmarked\.![Refer to caption](https://arxiv.org/html/2609.17997v1/AvgRandil.png)Fig\. 5:As in Figure[4](https://arxiv.org/html/2609.17997#S6.F4), for weight matrices drawn independently at every layer and with the output gate removed\. The smallest admissible window is longer,T=34T=34\.![Refer to caption](https://arxiv.org/html/2609.17997v1/AllRuns.png)Fig\. 6:Worst\-case window average of the interaction term of the first token against the window length, for the four configurations considered: themamba2\-130mweights and independently drawn weights, each with and without the nonlinearity on the branches producingBBandCC\. The smallest admissible window is marked where it exists; for independently drawn weights without the nonlinearity no window length is admissible\.

## VIIConclusion

In this paper, we showed that the relationship between selective state space models and transformers extends beyond their input\-output representations and reaches the dynamics induced by depth\. By deriving a continuous\-time model for Mamba\-2 with time\-varying weight matrices, we recast the evolution of tokens as a dynamical system on the sphere\. In this framework, using the causal cascade structure of selective SSMs and input\-to\-state stability, we proved local exponential stability of the consensus equilibria under a persistency of excitation condition\. Then, under a stronger global condition, we described the corresponding domain of attraction\. Our experiments further indicate that the output gate is the component that attenuates the consensus phenomenon, which suggests that gating is fundamental in preventing all tokens from converging to a single cluster\. These results provide, to the best of our knowledge, the first consensus analysis for selective SSMs with time\-varying weight matrices, showing that the mechanisms underlying their efficient recurrent structure also lead to the loss of token diversity\.

## References

- \[1\]\(2026\)A survey of large language models\.Frontiers of Computer Science20\(12\),pp\. 2012627\.Cited by:[§I](https://arxiv.org/html/2609.17997#S1.p1.1)\.
- \[2\]A\. Vaswani, N\. Shazeer, N\. Parmar, J\. Uszkoreit, L\. Jones, A\. N\. Gomez, Ł\. Kaiser, and I\. Polosukhin\(2017\)Attention is all you need\.Advances in neural information processing systems30\.Cited by:[§I](https://arxiv.org/html/2609.17997#S1.p1.1)\.
- \[3\]J\. Achiam, S\. Adler, S\. Agarwal, L\. Ahmad, I\. Akkaya, F\. L\. Aleman, D\. Almeida, J\. Altenschmidt, S\. Altman, S\. Anadkat,et al\.\(2023\)Gpt\-4 technical report\.arXiv preprint arXiv:2303\.08774\.Cited by:[§I](https://arxiv.org/html/2609.17997#S1.p1.1)\.
- \[4\]D\. Bahdanau, K\. Cho, and Y\. Bengio\(2014\)Neural machine translation by jointly learning to align and translate\.arXiv preprint arXiv:1409\.0473\.Cited by:[§I](https://arxiv.org/html/2609.17997#S1.p1.1)\.
- \[5\]A\. Dosovitskiy, L\. Beyer, A\. Kolesnikov, D\. Weissenborn, X\. Zhai, T\. Unterthiner, M\. Dehghani, M\. Minderer, G\. Heigold, S\. Gelly,et al\.\(2020\)An image is worth 16x16 words: transformers for image recognition at scale\.arXiv preprint arXiv:2010\.11929\.Cited by:[§I](https://arxiv.org/html/2609.17997#S1.p1.1)\.
- \[6\]J\. Jumper, R\. Evans, A\. Pritzel, T\. Green, M\. Figurnov, O\. Ronneberger, K\. Tunyasuvunakool, R\. Bates, A\. Žídek, A\. Potapenko,et al\.\(2021\)Highly accurate protein structure prediction with alphafold\.nature596\(7873\),pp\. 583–589\.Cited by:[§I](https://arxiv.org/html/2609.17997#S1.p1.1)\.
- \[7\]Y\. Tay, M\. Dehghani, D\. Bahri, and D\. Metzler\(2022\)Efficient transformers: a survey\.ACM Computing Surveys55\(6\),pp\. 1–28\.Cited by:[§I](https://arxiv.org/html/2609.17997#S1.p2.1)\.
- \[8\]R\. Waleffe, W\. Byeon, D\. Riber,et al\.\(2024\)An empirical study of Mamba\-based language models\.arXiv preprint arXiv:2406\.07887\.External Links:[Link](https://arxiv.org/abs/2406.07887)Cited by:[§I](https://arxiv.org/html/2609.17997#S1.p2.1)\.
- \[9\]A\. Grattafiori, A\. Dubey, A\. Jauhri,et al\.\(2024\)The LLaMA 3 herd of models\.arXiv preprint arXiv:2407\.21783\.External Links:[Link](https://arxiv.org/abs/2407.21783)Cited by:[§I](https://arxiv.org/html/2609.17997#S1.p2.1)\.
- \[10\]Y\. Levine, N\. Wies, O\. Sharir, H\. Bata, and A\. Shashua\(2020\)Limits to depth efficiencies of self\-attention\.Advances in Neural Information Processing Systems33,pp\. 22640–22651\.Cited by:[§I](https://arxiv.org/html/2609.17997#S1.p2.1)\.
- \[11\]Y\. Dong, J\. Cordonnier, and A\. Loukas\(2021\)Attention is not all you need: pure attention loses rank doubly exponentially with depth\.InInternational Conference on Machine Learning,pp\. 2793–2803\.Cited by:[§I](https://arxiv.org/html/2609.17997#S1.p2.1)\.
- \[12\]L\. Noci, S\. Anagnostidis, L\. Biggio, A\. Orvieto, S\. P\. Singh, and A\. Lucchi\(2022\)Signal propagation in transformers: theoretical perspectives and the role of rank collapse\.Advances in Neural Information Processing Systems35,pp\. 27198–27211\.Cited by:[§I](https://arxiv.org/html/2609.17997#S1.p2.1)\.
- \[13\]B\. Geshkovski, C\. Letrouit, Y\. Polyanskiy, and P\. Rigollet\(2023\)The emergence of clusters in self\-attention dynamics\.InAdvances in Neural Information Processing Systems,Vol\.36,pp\. 57026–57037\.Cited by:[§I](https://arxiv.org/html/2609.17997#S1.p2.1)\.
- \[14\]A\. Alcalde, G\. Fantuzzi, and E\. Zuazua\(2024\)Clustering in pure\-attention hardmax transformers and its role in sentiment analysis\.arXiv preprint arXiv:2407\.01602\.Cited by:[§I](https://arxiv.org/html/2609.17997#S1.p2.1)\.
- \[15\]A\. Gu and T\. Dao\(2024\)Mamba: linear\-time sequence modeling with selective state spaces\.arXiv preprint arXiv:2312\.00752\.External Links:[Link](https://arxiv.org/abs/2312.00752)Cited by:[§I](https://arxiv.org/html/2609.17997#S1.p3.1)\.
- \[16\]T\. Dao and A\. Gu\(2024\)Transformers are SSMs: generalized models and efficient algorithms through structured state space duality\.InInternational Conference on Machine Learning,Cited by:[§I](https://arxiv.org/html/2609.17997#S1.p3.1),[§I](https://arxiv.org/html/2609.17997#S1.p4.1)\.
- \[17\]S\. Hochreiter and J\. Schmidhuber\(1997\)Long short\-term memory\.Neural Computation9\(8\),pp\. 1735–1780\.Cited by:[§I](https://arxiv.org/html/2609.17997#S1.p3.1)\.
- \[18\]A\. A\. Ali, I\. Zimerman, and L\. Wolf\(2025\)The hidden attention of mamba models\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 1516–1534\.Cited by:[§I](https://arxiv.org/html/2609.17997#S1.p4.1),[§II](https://arxiv.org/html/2609.17997#S2.p2.1)\.
- \[19\]W\. Merrill, J\. Petty, and A\. Sabharwal\(2024\)The illusion of state in state\-space models\.arXiv preprint arXiv:2404\.08819\.External Links:[Link](https://arxiv.org/abs/2404.08819)Cited by:[§I](https://arxiv.org/html/2609.17997#S1.p5.1)\.
- \[20\]P\. Wang, R\. Zheng, X\. Liu, S\. Mao, X\. Chen, Z\. Lin, and Z\. Wang\(2025\)Understanding and mitigating bottlenecks of state space models through the lens of recency and over\-smoothing\.InInternational Conference on Learning Representations,Cited by:[§I](https://arxiv.org/html/2609.17997#S1.p5.1)\.
- \[21\]O\. Skean, U\. Utkarsh, and J\. Z\. Kolter\(2025\)A comparative analysis of contextual representation flow in state\-space and transformer architectures\.arXiv preprint arXiv:2510\.06640\.External Links:[Link](https://arxiv.org/abs/2510.06640)Cited by:[§I](https://arxiv.org/html/2609.17997#S1.p5.1)\.
- \[22\]B\. Geshkovski, C\. Letrouit, Y\. Polyanskiy, and P\. Rigollet\(2025\)A mathematical perspective on transformers\.Bulletin of the American Mathematical Society62\(3\),pp\. 427–479\.Cited by:[§I](https://arxiv.org/html/2609.17997#S1.p5.1),[§III](https://arxiv.org/html/2609.17997#S3.p5.1)\.
- \[23\]Á\. Rodríguez Abella, J\. P\. Silvestre, and P\. Tabuada\(2024\)The asymptotic behavior of attention in transformers\.arXiv preprint arXiv:2412\.02682\.External Links:[Link](https://arxiv.org/abs/2412.02682)Cited by:[§I](https://arxiv.org/html/2609.17997#S1.p5.1),[§III](https://arxiv.org/html/2609.17997#S3.p5.1)\.
- \[24\]Á\. Rodríguez Abella, J\. P\. Silvestre, and P\. Tabuada\(2025\)Consensus is all you get: the role of attention in transformers\.InForty\-second International Conference on Machine Learning,Cited by:[§I](https://arxiv.org/html/2609.17997#S1.p5.1),[Remark II\.3](https://arxiv.org/html/2609.17997#S2.Thmremark3.p1.1.1)\.
- \[25\]T\. N\. Vo, D\. Pham, X\. T\. Tong, and T\. M\. Nguyen\(2025\)Demystifying the token dynamics of deep selective state space models\.InInternational Conference on Learning Representations,Note:SpotlightCited by:[§I](https://arxiv.org/html/2609.17997#S1.p5.1),[§II\-B](https://arxiv.org/html/2609.17997#S2.SS2.p6.1),[§II](https://arxiv.org/html/2609.17997#S2.p2.1)\.
- \[26\]F\. A\. Joseph, J\. Sieber, M\. N\. Zeilinger, and C\. Amo Alonso\(2025\)Lambda\-skip connections: the architectural component that prevents rank collapse\.InInternational Conference on Learning Representations,Cited by:[§I](https://arxiv.org/html/2609.17997#S1.p5.1)\.
- \[27\]E\. D\. Sontag\(2008\)Input to state stability: basic concepts and results\.InNonlinear and Optimal Control Theory,pp\. 163–220\.Cited by:[§I](https://arxiv.org/html/2609.17997#S1.p6.1)\.
- \[28\]H\. K\. Khalil\(2002\)Nonlinear systems\.3rd edition,Prentice Hall,Upper Saddle River, NJ\.External Links:ISBN 978\-0130673893Cited by:[§I](https://arxiv.org/html/2609.17997#S1.p6.1)\.
- \[29\]B\. Zhang and R\. Sennrich\(2019\)Root mean square layer normalization\.Advances in Neural Information Processing Systems32\.Cited by:[§II](https://arxiv.org/html/2609.17997#S2.p2.1)\.
- \[30\]A\. Gu and T\. Dao\(2024\)Mamba2\-130m\.Hugging Face\.Note:[https://huggingface\.co/state\-spaces/mamba2\-130m](https://huggingface.co/state-spaces/mamba2-130m)Model checkpoint, accessed September 16, 2026Cited by:[§VI](https://arxiv.org/html/2609.17997#S6.p1.1)\.

We begin by showing the bounds for the projected variables in section[IV](https://arxiv.org/html/2609.17997#S4)\.

###### Proof:

For each\(t,Z\)∈ℝ0\+×\(𝕊n−1\)ℓ\(t,Z\)\\in\\mathbb\{R\}\_\{0\}^\{\+\}\\times\(\\mathbb\{S\}^\{n\-1\}\)^\{\\ell\}andi,j∈\{1,…,ℓ\}i,j\\in\\\{1,\\ldots,\\ell\\\}withj≤ij\\leq i, we may write:

zi⊤​SB​C​\(t\)​\(ej\+σj​𝔢μ~\)=zi⊤​SB​C​\(t\)​ej\+σj​zi⊤​SB​C​\(t\)​𝔢μ~\.\\displaystyle z\_\{i\}^\{\\top\}\\,S\_\{BC\}\(t\)\\,\(e\_\{j\}\+\\sigma\_\{j\}\\mathfrak\{e\}^\{\\tilde\{\\mu\}\}\)=z\_\{i\}^\{\\top\}\\,S\_\{BC\}\(t\)\\,e\_\{j\}\+\\sigma\_\{j\}\\,z\_\{i\}^\{\\top\}\\,S\_\{BC\}\(t\)\\,\\mathfrak\{e\}^\{\\tilde\{\\mu\}\}\.This allows for expressing the dynamics \([5](https://arxiv.org/html/2609.17997#S2.E5)\) of theii\-th token as follows:

z˙i=\\displaystyle\\dot\{z\}\_\{i\}=\\,Tzi​π⋅\(zi⊤​SB​C​\(t\)​zi\)​Di​i​\(t,Z\)​zi\\displaystyle T\_\{z\_\{i\}\}\\pi\\cdot\(z\_\{i\}^\{\\top\}\\,S\_\{BC\}\(t\)\\,z\_\{i\}\)D\_\{ii\}\(t,Z\)\\,z\_\{i\}\+Tziπ⋅∑j=1i−1\(zi⊤SB​C\(t\)𝔢μ~\)Di​j\(t,Z\)𝔢μ~\\displaystyle\+T\_\{z\_\{i\}\}\\pi\\cdot\\sum\_\{j=1\}^\{i\-1\}\(z\_\{i\}^\{\\top\}S\_\{BC\}\(t\)\\,\\mathfrak\{e\}^\{\\tilde\{\\mu\}\}\)D\_\{ij\}\(t,Z\)\\,\\mathfrak\{e\}^\{\\tilde\{\\mu\}\}\+Tziπ⋅∑j=1i−1σj\(zi⊤SB​C\(t\)𝔢μ~\)Di​j\(t,Z\)ej\\displaystyle\+T\_\{z\_\{i\}\}\\pi\\cdot\\sum\_\{j=1\}^\{i\-1\}\\sigma\_\{j\}\\,\(z\_\{i\}^\{\\top\}S\_\{BC\}\(t\)\\,\\mathfrak\{e\}^\{\\tilde\{\\mu\}\}\)D\_\{ij\}\(t,Z\)\\,e\_\{j\}\+Tziπ⋅∑j=1i−1σj\(zi⊤SB​C\(t\)ej\)Di​j\(t,Z\)𝔢μ~\\displaystyle\+T\_\{z\_\{i\}\}\\pi\\cdot\\sum\_\{j=1\}^\{i\-1\}\\sigma\_\{j\}\\,\(z\_\{i\}^\{\\top\}S\_\{BC\}\(t\)\\,e\_\{j\}\)D\_\{ij\}\(t,Z\)\\,\\mathfrak\{e\}^\{\\tilde\{\\mu\}\}\+Tziπ⋅∑j=1i−1\(zi⊤SB​C\(t\)ej\)Di​j\(t,Z\)ej,\\displaystyle\+T\_\{z\_\{i\}\}\\pi\\cdot\\sum\_\{j=1\}^\{i\-1\}\(z\_\{i\}^\{\\top\}S\_\{BC\}\(t\)\\,e\_\{j\}\)D\_\{ij\}\(t,Z\)\\,e\_\{j\},where the sums are zero wheni=1i=1and we used thatσj2=1\\sigma\_\{j\}^\{2\}=1for eachj∈\{1,…,i−1\}j\\in\\\{1,\\ldots,i\-1\\\}\. Hence \(we omit the argumentsttandZZfor brevity\):

b˙i=\\displaystyle\\dot\{b\}\_\{i\}=~\(zi⊤​SB​C​zi\)​\(λi​iμ~−zi⊤​Di​i​zi\)​bi\\displaystyle\(z\_\{i\}^\{\\top\}\\,S\_\{BC\}\\,z\_\{i\}\)\\,\(\\lambda\_\{ii\}^\{\\tilde\{\\mu\}\}\-z\_\{i\}^\{\\top\}\\,D\_\{ii\}\\,z\_\{i\}\)\\,b\_\{i\}\(24\)\+∑j=1i−1\(zi⊤SB​C𝔢μ~\)\(σiλi​jμ~−\(zi⊤Di​j𝔢μ~\)bi\)\\displaystyle\+\\sum\_\{j=1\}^\{i\-1\}\(z\_\{i\}^\{\\top\}\\,S\_\{BC\}\\,\\mathfrak\{e\}^\{\\tilde\{\\mu\}\}\)\(\\sigma\_\{i\}\\lambda\_\{ij\}^\{\\tilde\{\\mu\}\}\-\(z\_\{i\}^\{\\top\}\\,D\_\{ij\}\\,\\mathfrak\{e\}^\{\\tilde\{\\mu\}\}\)\\,b\_\{i\}\)\(25\)\+∑j=1i−1σj\(zi⊤SB​C𝔢μ~\)\(σiλi​jμ~ejμ~−\(zi⊤Di​jej\)bi\)\\displaystyle\+\\sum\_\{j=1\}^\{i\-1\}\\sigma\_\{j\}\\,\(z\_\{i\}^\{\\top\}\\,S\_\{BC\}\\,\\mathfrak\{e\}^\{\\tilde\{\\mu\}\}\)\\,\(\\sigma\_\{i\}\\lambda\_\{ij\}^\{\\tilde\{\\mu\}\}\\,e\_\{j\}^\{\\tilde\{\\mu\}\}\-\(z\_\{i\}^\{\\top\}\\,D\_\{ij\}\\,e\_\{j\}\)\\,b\_\{i\}\)\(26\)\+∑j=1i−1σj\(zi⊤SB​Cej\)\(σiλi​jμ~−\(zi⊤Di​j𝔢μ~\)bi\)\\displaystyle\+\\sum\_\{j=1\}^\{i\-1\}\\sigma\_\{j\}\\,\(z\_\{i\}^\{\\top\}\\,S\_\{BC\}\\,e\_\{j\}\)\(\\sigma\_\{i\}\\lambda\_\{ij\}^\{\\tilde\{\\mu\}\}\-\(z\_\{i\}^\{\\top\}\\,D\_\{ij\}\\,\\mathfrak\{e\}^\{\\tilde\{\\mu\}\}\)\\,b\_\{i\}\)\(27\)\+∑j=1i−1\(zi⊤SB​Cej\)\(σiλi​jμ~ejμ~−\(zi⊤Di​jej\)bi\)\.\\displaystyle\+\\sum\_\{j=1\}^\{i\-1\}\(z\_\{i\}^\{\\top\}\\,S\_\{BC\}\\,e\_\{j\}\)\(\\sigma\_\{i\}\\lambda\_\{ij\}^\{\\tilde\{\\mu\}\}\\,e\_\{j\}^\{\\tilde\{\\mu\}\}\-\(z\_\{i\}^\{\\top\}\\,D\_\{ij\}\\,e\_\{j\}\)\\,b\_\{i\}\)\.\(28\)
From the fact that\|zi\|=1\|z\_\{i\}\|=1, we obtainγ⁡\(1−bi2\)≤λi​iμ~−zi⊤​Di​i​zi≤Γ⁡\(1−bi2\)\\gamma\(1\-b\_\{i\}^\{2\}\)\\leq\\lambda\_\{ii\}^\{\\tilde\{\\mu\}\}\-z\_\{i\}^\{\\top\}D\_\{ii\}z\_\{i\}\\leq\\Gamma\(1\-b\_\{i\}^\{2\}\)\. Hence, given\(t,bi\)∈ℝ0\+×\]c,1\]\(t,b\_\{i\}\)\\in\\mathbb\{R\}\_\{0\}^\{\+\}\\times\\,\]c,1\], we distinguish two cases:

1. 1\.zi⊤​SB​C​zi∈ℝ0\+z\_\{i\}^\{\\top\}\\,S\_\{BC\}\\,z\_\{i\}\\in\\mathbb\{R\}\_\{0\}^\{\+\}, then: \([24](https://arxiv.org/html/2609.17997#A0.E24)\)≥\(zi⊤​SB​C​zi\)​γ​\(1\+bi\)​\(1−bi\)​bi\\displaystyle\\geq\(z\_\{i\}^\{\\top\}\\,S\_\{BC\}\\,z\_\{i\}\)\\gamma\(1\+b\_\{i\}\)\(1\-b\_\{i\}\)b\_\{i\}≥γ​c​\(1\+c\)​\(zi⊤​SB​C​zi\)​\(1−bi\)=ac​\(t,zi\)​\(1−bi\)\.\\displaystyle\\geq\\gamma c\(1\+c\)\(z\_\{i\}^\{\\top\}\\,S\_\{BC\}\\,z\_\{i\}\)\\,\(1\-b\_\{i\}\)=a\_\{c\}\(t,z\_\{i\}\)\\,\(1\-b\_\{i\}\)\.
2. 2\.zi⊤​SB​C​zi∈ℝ−z\_\{i\}^\{\\top\}\\,S\_\{BC\}\\,z\_\{i\}\\in\\mathbb\{R\}^\{\-\}, then: \([24](https://arxiv.org/html/2609.17997#A0.E24)\)≥\(zi⊤​SB​C​zi\)​Γ​\(1\+bi\)​\(1−bi\)​bi\\displaystyle\\geq\(z\_\{i\}^\{\\top\}\\,S\_\{BC\}\\,z\_\{i\}\)\\Gamma\(1\+b\_\{i\}\)\(1\-b\_\{i\}\)b\_\{i\}≥2​\(Γ\+Λ\)​\(zi⊤​SB​C​zi\)​\(1−bi\)=ac​\(t,zi\)​\(1−bi\),\\displaystyle\\geq 2\(\\Gamma\+\\Lambda\)\(z\_\{i\}^\{\\top\}\\,S\_\{BC\}\\,z\_\{i\}\)\\,\(1\-b\_\{i\}\)=a\_\{c\}\(t,z\_\{i\}\)\\,\(1\-b\_\{i\}\),where we used that\(1\+bi\)​bi≤2\(1\+b\_\{i\}\)b\_\{i\}\\leq 2andΛ∈ℝ\+\\Lambda\\in\\mathbb\{R\}^\{\+\}\.

Fori=1i=1, \([25](https://arxiv.org/html/2609.17997#A0.E25)\), \([26](https://arxiv.org/html/2609.17997#A0.E26)\), \([27](https://arxiv.org/html/2609.17997#A0.E27)\) and \([28](https://arxiv.org/html/2609.17997#A0.E28)\) vanish, so we conclude thatb˙1≥ac​\(t,z1\)​\(1−b1\)\\dot\{b\}\_\{1\}\\geq a\_\{c\}\(t,z\_\{1\}\)\\,\(1\-b\_\{1\}\)\.

Fori∈\{2,…,ℓ\}i\\in\\\{2,\\ldots,\\ell\\\}, we need to bound the remaining terms\. Using thatσi​𝔢μ~=zi−ei\\sigma\_\{i\}\\,\\mathfrak\{e\}^\{\\tilde\{\\mu\}\}=z\_\{i\}\-e\_\{i\}, we obtain:

\([25](https://arxiv.org/html/2609.17997#A0.E25)\)=∑j=1i−1σi​\(zi⊤​SB​C​𝔢μ~\)​λi​jμ~​\(1−bi2\)\\displaystyle=\\sum\_\{j=1\}^\{i\-1\}\\sigma\_\{i\}\(z\_\{i\}^\{\\top\}\\,S\_\{BC\}\\,\\mathfrak\{e\}^\{\\tilde\{\\mu\}\}\)\\,\\lambda\_\{ij\}^\{\\tilde\{\\mu\}\}\\,\(1\-b\_\{i\}^\{2\}\)=∑j=1i−1\(zi⊤​SB​C​zi−zi⊤​SB​C​ei\)​λi​jμ~​\(1−bi2\)\\displaystyle=\\sum\_\{j=1\}^\{i\-1\}\(z\_\{i\}^\{\\top\}\\,S\_\{BC\}\\,z\_\{i\}\-z\_\{i\}^\{\\top\}\\,S\_\{BC\}\\,e\_\{i\}\)\\,\\lambda\_\{ij\}^\{\\tilde\{\\mu\}\}\\,\(1\-b\_\{i\}^\{2\}\)=∑j=1i−1λi​jμ~​\(zi⊤​SB​C​zi−zi⊤​SB​C​ei\)​\(1−bi2\)\.\\displaystyle=\\sum\_\{j=1\}^\{i\-1\}\\lambda\_\{ij\}^\{\\tilde\{\\mu\}\}\\,\(z\_\{i\}^\{\\top\}\\,S\_\{BC\}\\,z\_\{i\}\-z\_\{i\}^\{\\top\}\\,S\_\{BC\}\\,e\_\{i\}\)\\,\(1\-b\_\{i\}^\{2\}\)\.As for the first token, given\(t,bi\)∈ℝ0\+×\]c,1\]\(t,b\_\{i\}\)\\in\\mathbb\{R\}\_\{0\}^\{\+\}\\times\\,\]c,1\], we distinguish two cases:

1. 1\.zi⊤​SB​C​zi∈ℝ0\+z\_\{i\}^\{\\top\}\\,S\_\{BC\}\\,z\_\{i\}\\in\\mathbb\{R\}\_\{0\}^\{\+\}, then\([25](https://arxiv.org/html/2609.17997#A0.E25)\)≥−Λ⁡\(zi⊤​SB​C​ei\)​\(1−bi2\)\\eqref\{PE:dotbi2\}\\geq\-\\Lambda\\,\(z\_\{i\}^\{\\top\}\\,S\_\{BC\}\\,e\_\{i\}\)\\,\(1\-b\_\{i\}^\{2\}\)\.
2. 2\.zi⊤​SB​C​zi∈ℝ−z\_\{i\}^\{\\top\}\\,S\_\{BC\}\\,z\_\{i\}\\in\\mathbb\{R\}^\{\-\}, then\([25](https://arxiv.org/html/2609.17997#A0.E25)\)≥2​Λ​\(zi⊤​SB​C​zi\)​\(1−bi\)−Λ⁡\(zi⊤​SB​C​ei\)​\(1−bi2\)\\eqref\{PE:dotbi2\}\\geq 2\\Lambda\\,\(z\_\{i\}^\{\\top\}\\,S\_\{BC\}\\,z\_\{i\}\)\\,\(1\-b\_\{i\}\)\-\\Lambda\\,\(z\_\{i\}^\{\\top\}\\,S\_\{BC\}\\,e\_\{i\}\)\\,\(1\-b\_\{i\}^\{2\}\), where we used that1\+bi≤21\+b\_\{i\}\\leq 2andΛ∈ℝ\+\\Lambda\\in\\mathbb\{R\}^\{\+\}\.

Therefore, in both cases we obtain:

\([24](https://arxiv.org/html/2609.17997#A0.E24)\)\+\([25](https://arxiv.org/html/2609.17997#A0.E25)\)≥ac​\(t,zi\)​\(1−bi\)−Λ⁡\(zi⊤​SB​C​ei\)​\(1−bi2\)\.\\displaystyle\\eqref\{PE:dotbi1\}\+\\eqref\{PE:dotbi2\}\\geq a\_\{c\}\(t,z\_\{i\}\)\\,\(1\-b\_\{i\}\)\-\\Lambda\\,\(z\_\{i\}^\{\\top\}\\,S\_\{BC\}\\,e\_\{i\}\)\\,\(1\-b\_\{i\}^\{2\}\)\.Let us pickdε∈\]0,1−c\]d\_\{\\varepsilon\}\\in\\,\]0,1\-c\]such that2​2​dε​Λ​supt∈ℝ0\+‖SB​C​\(t\)‖≤ε2\\sqrt\{2d\_\{\\varepsilon\}\}\\Lambda\\sup\_\{t\\in\\mathbb\{R\}\_\{0\}^\{\+\}\}\\\|S\_\{BC\}\(t\)\\\|\\leq\\varepsilon\. Hence, for each\(t,bi\)∈ℝ0\+×\]1−dε,1\]\(t,b\_\{i\}\)\\in\\mathbb\{R\}\_\{0\}^\{\+\}\\times\\,\]1\-d\_\{\\varepsilon\},1\], we have\|ei\|=2​\(1−bi\)<2​dε\|e\_\{i\}\|=\\sqrt\{2\(1\-b\_\{i\}\)\}<\\sqrt\{2d\_\{\\varepsilon\}\}, whence:

−Λ⁡\(zi⊤​SB​C​ei\)​\(1−bi2\)\\displaystyle\-\\Lambda\\,\(z\_\{i\}^\{\\top\}\\,S\_\{BC\}\\,e\_\{i\}\)\\,\(1\-b\_\{i\}^\{2\}\)≥−22​dεΛsupt∈ℝ0\+∥SB​C\(t\)∥\(1−bi\)\\displaystyle\\geq\-2\\sqrt\{2d\_\{\\varepsilon\}\}\\Lambda\\,\\sup\_\{t\\in\\mathbb\{R\}\_\{0\}^\{\+\}\}\\\|S\_\{BC\}\(t\)\\\|\\,\(1\-b\_\{i\}\)≥−ε⁡\(1−bi\),\\displaystyle\\geq\-\\varepsilon\\,\(1\-b\_\{i\}\),where we used that1\+bi≤21\+b\_\{i\}\\leq 2and\|zi\|=1\|z\_\{i\}\|=1\. As a result:

\([24](https://arxiv.org/html/2609.17997#A0.E24)\)\+\([25](https://arxiv.org/html/2609.17997#A0.E25)\)≥\(ac​\(t,zi\)−ε\)​\(1−bi\)\.\\displaystyle\\eqref\{PE:dotbi1\}\+\\eqref\{PE:dotbi2\}\\geq\(a\_\{c\}\(t,z\_\{i\}\)\-\\varepsilon\)\\,\(1\-b\_\{i\}\)\.
Lastly, from Assumption[III\.1](https://arxiv.org/html/2609.17997#S3.Thmassumption1)and Lemma[III\.1](https://arxiv.org/html/2609.17997#S3.Thmlemma1), as well as compactness of the sphere, the error terms are all bounded,i\.e\., there existsM∈ℝ\+M\\in\\mathbb\{R\}^\{\+\}such that:

\([26](https://arxiv.org/html/2609.17997#A0.E26)\)\+\([27](https://arxiv.org/html/2609.17997#A0.E27)\)\+\([28](https://arxiv.org/html/2609.17997#A0.E28)\)≥−M∑j=1i−1\|ej\|\.\\displaystyle\\eqref\{PE:dotbi3\}\+\\eqref\{PE:dotbi4\}\+\\eqref\{PE:dotbi5\}\\geq\-M\\sum\_\{j=1\}^\{i\-1\}\|e\_\{j\}\|\.\(29\)for each\(t,bi\)∈ℝ0\+×\]c,1\]\(t,b\_\{i\}\)\\in\\mathbb\{R\}\_\{0\}^\{\+\}\\times\\,\]c,1\], and in particular, for each\(t,bi\)∈ℝ0\+×\]1−dε,1\]\(t,b\_\{i\}\)\\in\\mathbb\{R\}\_\{0\}^\{\+\}\\times\\,\]1\-d\_\{\\varepsilon\},1\]\.

By gathering the previous bounds, we conclude\. ∎

The following two technical lemmas are useful in the proof of Theorem[IV\.1](https://arxiv.org/html/2609.17997#S4.Thmtheorem1)\.

###### Lemma \.1

Letdε,H,Mi,θ,r,R∈ℝ\+d\_\{\\varepsilon\},H,M\_\{i\},\\theta,r,R\\in\\mathbb\{R\}^\{\+\},i∈\{1,…,ℓ\}i\\in\\\{1,\\ldots,\\ell\\\}, be such thatθ<1\\theta<1andR\>r≥1R\>r\\geq 1\. Then there exists a sequence:

\(d1,…,dℓ\)∈∏i=1ℓ\]dimin,dmax\[,\\displaystyle\(d\_\{1\},\\ldots,d\_\{\\ell\}\)\\in\\prod\_\{i=1\}^\{\\ell\}\\,\]d\_\{i\}^\{\\min\},d^\{\\max\}\[\\,,wheredmax=dε/Rd^\{\\max\}=d\_\{\\varepsilon\}/R,d1min=0d\_\{1\}^\{\\min\}=0and:

dimin=max⁡\{di−1,Hμ​∑j=1i−12​Mj​dj\},\\displaystyle d\_\{i\}^\{\\min\}=\\max\\left\\\{d\_\{i\-1\},~\\frac\{H\}\{\\mu\}\\sum\_\{j=1\}^\{i\-1\}\\sqrt\{2M\_\{j\}d\_\{j\}\}\\right\\\},fori∈\{2,…,ℓ\}i\\in\\\{2,\\ldots,\\ell\\\}, withμ=min⁡\{1−θ,R−r\}∈ℝ\+\\mu=\\min\\\{1\-\\theta,R\-r\\\}\\in\\mathbb\{R\}^\{\+\}\.

###### Proof:

Givend1∈\]0,dmax\[d\_\{1\}\\in\\,\]0,d^\{\\max\}\[\\,, we recursively define:

di=2​di−1\+2​Hμ​∑j=1i−12​Mj​dj,\\displaystyle d\_\{i\}=2d\_\{i\-1\}\+\\frac\{2H\}\{\\mu\}\\sum\_\{j=1\}^\{i\-1\}\\sqrt\{2M\_\{j\}d\_\{j\}\},for eachi∈\{2,…,ℓ\}i\\in\\\{2,\\ldots,\\ell\\\}\. It is clear thatdi\>dimind\_\{i\}\>d\_\{i\}^\{\\min\}by construction\.

To conclude, we need to show thatd1d\_\{1\}can be chosen so thatdi<dmaxd\_\{i\}<d^\{\\max\}for eachi∈\{2,…,ℓ\}i\\in\\\{2,\\ldots,\\ell\\\}\. To that end, we regarddi=di​\(d1\)d\_\{i\}=d\_\{i\}\(d\_\{1\}\), and note that they are continuous functions on\]0,dmax\[\]0,d^\{\\max\}\[\\,\. Hence,limd1→0\+di​\(d1\)=di​\(0\)=0\\lim\_\{d\_\{1\}\\to 0^\{\+\}\}d\_\{i\}\(d\_\{1\}\)=d\_\{i\}\(0\)=0\. Hence,d1∈\]0,dmax\[d\_\{1\}\\in\\,\]0,d^\{\\max\}\[can be chosen so thatdi<dmaxd\_\{i\}<d^\{\\max\}for eachi∈\{2,…​ℓ\}i\\in\\\{2,\\ldots\\ell\\\}\. ∎

###### Lemma \.2\(Discrete ISS\)

Letθ∈\]0,1\[\\theta\\in\\,\]0,1\[\\,,ν,d∈ℝ\+\\nu,d\\in\\mathbb\{R\}^\{\+\}, andϱ=12min\{−log\(θ\)/T,ν\}\\varrho=\\frac\{1\}\{2\}\\min\\\{\-\\log\(\\theta\)/T,\\nu\\\}\. There existsC\(T,θ,ϱ\)∈\[1,∞\[C\(T,\\theta,\\varrho\)\\in\[1,\\infty\[such that, for each real sequence\(yk\)k∈ℕ\(y\_\{k\}\)\_\{k\\in\\mathbb\{N\}\}andy0∈\[0,d\[y\_\{0\}\\in\[0,d\[satisfying:

yk≤θ​yk−1\+\(1−θ\)​d​exp⁡\(−ν⁡\(k−1\)​T\),k∈ℕ,\\displaystyle y\_\{k\}\\leq\\theta\\,y\_\{k\-1\}\+\(1\-\\theta\)\\,d\\,\\exp\(\-\\nu\(k\-1\)T\),\\qquad k\\in\\mathbb\{N\},thenyk≤C⁡\(T,θ,ϱ\)​d​exp⁡\(−ϱ​k​T\)y\_\{k\}\\leq C\(T,\\theta,\\varrho\)\\,d\\,\\exp\(\-\\varrho kT\)for eachk∈ℕk\\in\\mathbb\{N\}\.

###### Proof:

Given thatϱ∈\]0,−log\(θ\)/T\[\\varrho\\in\\,\]0,\-\\log\(\\theta\)/T\[\\,, we haveexp⁡\(−ϱ​T\)∈\]θ,1\[\\exp\(\-\\varrho T\)\\in\\,\]\\theta,1\[\. Thus, we define:

C\(T,θ,ϱ\)=1−θexp⁡\(−ϱ​T\)−θ∈\[1,∞\[\.\\displaystyle C\(T,\\theta,\\varrho\)=\\frac\{1\-\\theta\}\{\\exp\(\-\\varrho T\)\-\\theta\}\\in\[1,\\infty\[\\,\.For brevity, we denoteC=C⁡\(T,θ,ϱ\)C=C\(T,\\theta,\\varrho\)\. Let us proceed by induction ink∈ℕk\\in\\mathbb\{N\}\.

Base case\. Fork=1k=1, we have:

y1\\displaystyle y\_\{1\}≤θ​y0\+\(1−θ\)​d\\displaystyle\\leq\\theta\\,y\_\{0\}\+\(1\-\\theta\)\\,d≤\(θ\+C⁡\(exp⁡\(−ϱ​T\)−θ\)\)​d\\displaystyle\\leq\(\\theta\+C\(\\exp\(\-\\varrho T\)\-\\theta\)\)\\,d≤C​d​exp⁡\(−ϱ​T\)\.\\displaystyle\\leq C\\,d\\,\\exp\(\-\\varrho T\)\.
Induction hypothesis\. Fork∈ℕk\\in\\mathbb\{N\}withk≥2k\\geq 2, we haveyk−1≤C​d​exp⁡\(−ϱ⁡\(k−1\)​T\)y\_\{k\-1\}\\leq C\\,d\\,\\exp\(\-\\varrho\(k\-1\)T\)\.

Inductive step\. Note thatexp⁡\(\(ϱ−ν\)​\(k−1\)​T\)∈\]0,1\[\\exp\(\(\\varrho\-\\nu\)\(k\-1\)T\)\\in\\,\]0,1\[sinceϱ∈\]0,ν\[\\varrho\\in\\,\]0,\\nu\[\\,\. Hence:

exp⁡\(−ν⁡\(k−1\)​T\+ϱ​k​T\)\\displaystyle\\exp\(\-\\nu\(k\-1\)T\+\\varrho kT\)=exp⁡\(ϱ​T\+\(ϱ−ν\)​\(k−1\)​T\)\\displaystyle=\\exp\(\\varrho T\+\(\\varrho\-\\nu\)\(k\-1\)T\)<exp⁡\(ϱ​T\)\.\\displaystyle<\\exp\(\\varrho T\)\.From this, the recurrence inequality, the induction hypothesis and the definition ofC=C⁡\(T,θ,ϱ\)C=C\(T,\\theta,\\varrho\), we conclude:

yk\\displaystyle y\_\{k\}≤θ​yk−1\+\(1−θ\)​d​exp⁡\(−ν⁡\(k−1\)​T\)\\displaystyle\\leq\\theta\\,y\_\{k\-1\}\+\(1\-\\theta\)\\,d\\,\\exp\(\-\\nu\(k\-1\)T\)≤θ​C​d​exp⁡\(−ϱ⁡\(k−1\)​T\)\+\(1−θ\)​d​exp⁡\(−ν⁡\(k−1\)​T\)\\displaystyle\\leq\\theta\\,C\\,d\\,\\exp\(\-\\varrho\(k\-1\)T\)\+\(1\-\\theta\)\\,d\\,\\exp\(\-\\nu\(k\-1\)T\)=\(θ​C​exp⁡\(ϱ​T\)\+\(1−θ\)​exp⁡\(ϱ​T\)\)​d​exp⁡\(−ϱ​k​T\)\\displaystyle=\\left\(\\theta\\,C\\,\\exp\(\\varrho T\)\+\(1\-\\theta\)\\exp\(\\varrho T\)\\right\)d\\exp\(\-\\varrho kT\)=C​d​exp⁡\(−ϱ​k​T\)\.\\displaystyle=C\\,d\\,\\exp\(\-\\varrho kT\)\.∎

Now we prove Claim[IV\.1](https://arxiv.org/html/2609.17997#S4.Thmclaim1), which was used in the proof of Theorem[IV\.1](https://arxiv.org/html/2609.17997#S4.Thmtheorem1)\.

###### Proof:

Let us show each statement:

1. 1\.By contradiction, suppose thatt0=inf\{t∈\[\(k−1\)T,kT\[∣Vi\(t\)=Rdi\}∈\[\(k−1\)T,kT\[t\_\{0\}=\\inf\\\{t\\in\[\(k\-1\)T,kT\[\\,\\mid V\_\{i\}\(t\)=Rd\_\{i\}\\\}\\in\[\(k\-1\)T,kT\[\. Given thatR​di<dεRd\_\{i\}<d\_\{\\varepsilon\}by construction, we haveVi​\(t\)≤dεV\_\{i\}\(t\)\\leq d\_\{\\varepsilon\}for eacht∈\[\(k−1\)​T,t0\]t\\in\[\(k\-1\)T,t\_\{0\}\]\. Hence, from Proposition[IV\.1](https://arxiv.org/html/2609.17997#S4.Thmproposition1)and \([16](https://arxiv.org/html/2609.17997#S4.E16)\), we obtain: V˙i\\displaystyle\\dot\{V\}\_\{i\}≤−\(ac​\(t,zi\)−ε\)​Vi\+M​∑j=1i−1\|ej\|\\displaystyle\\leq\-\(a\_\{c\}\(t,z\_\{i\}\)\-\\varepsilon\)\\,V\_\{i\}\+M\\sum\_\{j=1\}^\{i\-1\}\|e\_\{j\}\|≤−\(ac​\(t,zi\)−ε\)​Vi\+M​∑j=1i−12​Mj​dj​exp⁡\(−ρj​t2\)\\displaystyle\\leq\-\(a\_\{c\}\(t,z\_\{i\}\)\-\\varepsilon\)\\,V\_\{i\}\+M\\sum\_\{j=1\}^\{i\-1\}\\sqrt\{2M\_\{j\}d\_\{j\}\}\\exp\\left\(\-\\frac\{\\rho\_\{j\}t\}\{2\}\\right\)≤−ηε​Vi\+M​∑j=1i−12​Mj​dj,\\displaystyle\\leq\-\\eta\_\{\\varepsilon\}\\,V\_\{i\}\+M\\sum\_\{j=1\}^\{i\-1\}\\sqrt\{2M\_\{j\}d\_\{j\}\},\(30\)for eacht∈\[\(k−1\)​T,t0\]t\\in\[\(k\-1\)T,t\_\{0\}\], where we used thatac​\(t,zi\)−ε≥ηεa\_\{c\}\(t,z\_\{i\}\)\-\\varepsilon\\geq\\eta\_\{\\varepsilon\}by definition\. For brevity, we writeτ=t0−\(k−1\)T∈\[0,T\[\\tau=t\_\{0\}\-\(k\-1\)T\\in\[0,T\[\\,\. Moreover, recall thatVi​\(\(k−1\)​T\)<diV\_\{i\}\(\(k\-1\)T\)<d\_\{i\},r=exp⁡\(−ηε​T\)r=\\exp\(\-\\eta\_\{\\varepsilon\}T\)andH=T​M​exp⁡\(−ηε​T\)H=T\\,M\\exp\(\-\\eta\_\{\\varepsilon\}T\)\. By integrating \([30](https://arxiv.org/html/2609.17997#A0.E30)\) betweent=\(k−1\)​Tt=\(k\-1\)Tandt=t0t=t\_\{0\}and using Lemma[\.1](https://arxiv.org/html/2609.17997#A0.Thmlemma1), we arrive at a contradiction\. Indeed, we denoteK=M​∑j=1i−12​Mj​djK=M\\sum\_\{j=1\}^\{i\-1\}\\sqrt\{2M\_\{j\}d\_\{j\}\}and we distinguish two cases: 1. \(a\)ηε=0\\eta\_\{\\varepsilon\}=0: R​di=Vi​\(t0\)\\displaystyle Rd\_\{i\}=V\_\{i\}\(t\_\{0\}\)≤Vi​\(\(k−1\)​T\)\+τ​K\\displaystyle\\leq V\_\{i\}\(\(k\-1\)T\)\+\\tau\\,K<di\+T​K<R​di\.\\displaystyle<d\_\{i\}\+T\\,K<Rd\_\{i\}\. 2. \(b\)ηε∈ℝ−\\eta\_\{\\varepsilon\}\\in\\mathbb\{R\}^\{\-\}: V˙i≤−ηε​Vi\+K,\\displaystyle\\dot\{V\}\_\{i\}\\leq\-\\eta\_\{\\varepsilon\}V\_\{i\}\+K,for eacht∈\[\(k−1\)​T,t0\]t\\in\[\(k\-1\)T,t\_\{0\}\]\. By using the integrating factorexp⁡\(ηε​t\)\\exp\(\\eta\_\{\\varepsilon\}t\), we obtain: dd​t​\(exp⁡\(ηε​t\)​Vi\)≤K​exp⁡\(ηε​t\),\\displaystyle\\frac\{d\}\{dt\}\\left\(\\exp\(\\eta\_\{\\varepsilon\}t\)V\_\{i\}\\right\)\\leq K\\exp\(\\eta\_\{\\varepsilon\}t\),for eacht∈\[\(k−1\)​T,t0\]t\\in\[\(k\-1\)T,t\_\{0\}\]\. By writingτ=t0−\(k−1\)T∈\[0,T\[\\tau=t\_\{0\}\-\(k\-1\)T\\in\[0,T\[\\,, we obtain: R​di=Vi​\(t0\)\\displaystyle Rd\_\{i\}=V\_\{i\}\(t\_\{0\}\)≤exp⁡\(−ηε​τ\)​Vi​\(\(k−1\)​T\)\+Kηε​\(1−exp⁡\(−ηε​τ\)\)\\displaystyle\\leq\\exp\(\-\\eta\_\{\\varepsilon\}\\tau\)\\,V\_\{i\}\(\(k\-1\)T\)\+\\frac\{K\}\{\\eta\_\{\\varepsilon\}\}\(1\-\\exp\(\-\\eta\_\{\\varepsilon\}\\tau\)\)=exp⁡\(−ηε​τ\)​Vi​\(\(k−1\)​T\)\+K​∫0τexp⁡\(−ηε​s\)​𝑑s\\displaystyle=\\exp\(\-\\eta\_\{\\varepsilon\}\\tau\)\\,V\_\{i\}\(\(k\-1\)T\)\+K\\int\_\{0\}^\{\\tau\}\\exp\(\-\\eta\_\{\\varepsilon\}s\)\\,ds≤exp⁡\(−ηε​T\)​\(di\+T​M​∑j=1i−12​Mj​dj\)<R​di\.\\displaystyle\\leq\\exp\(\-\\eta\_\{\\varepsilon\}T\)\(d\_\{i\}\+TM\\sum\_\{j=1\}^\{i\-1\}\\sqrt\{2M\_\{j\}d\_\{j\}\}\)<Rd\_\{i\}\. Hence,Vi\(t\)∈\[0,Rdi\[V\_\{i\}\(t\)\\in\[0,Rd\_\{i\}\[for eacht∈\[\(k−1\)T,kT\[t\\in\[\(k\-1\)T,kT\[\\,\.
2. 2\.Given thatdi∈\]0,dε/R\[d\_\{i\}\\in\\,\]0,d\_\{\\varepsilon\}/R\[\\,, from the previous item we conclude thatVi\(t\)∈\[0,dε\[V\_\{i\}\(t\)\\in\[0,d\_\{\\varepsilon\}\[for eacht∈\[\(k−1\)T,kT\[t\\in\[\(k\-1\)T,kT\[\\,\. Moreover,dε∈\]0,1−c\]d\_\{\\varepsilon\}\\in\\,\]0,1\-c\], whenceVi\(t\)∈\[0,1−c\[V\_\{i\}\(t\)\\in\[0,1\-c\[\\,,i\.e\.,zi​\(t\)∈Ωcσi​\(μ~\)z\_\{i\}\(t\)\\in\\Omega\_\{c\}^\{\\sigma\_\{i\}\}\(\\tilde\{\\mu\}\), for eacht∈\[\(k−1\)T,kT\[t\\in\[\(k\-1\)T,kT\[\\,\. Thus,ac​\(t,zi​\(t\)\)≥αc​\(t\)a\_\{c\}\(t,z\_\{i\}\(t\)\)\\geq\\alpha\_\{c\}\(t\)for eacht∈\[\(k−1\)T,kT\[t\\in\[\(k\-1\)T,kT\[\\,\. Thus, Assumption[IV\.1](https://arxiv.org/html/2609.17997#S4.Thmassumption1)gives: ∫\(k−1\)​Tk​T\(ac​\(s,zi​\(s\)\)−ε\)​𝑑s≥\(ρ−ε\)​T\.\\displaystyle\\int\_\{\(k\-1\)T\}^\{kT\}\(a\_\{c\}\(s,z\_\{i\}\(s\)\)\-\\varepsilon\)\\,ds\\geq\(\\rho\-\\varepsilon\)T\.\(31\)In addition, Proposition[IV\.1](https://arxiv.org/html/2609.17997#S4.Thmproposition1)holds, yielding: V˙i\\displaystyle\\dot\{V\}\_\{i\}≤−\(ac​\(t,zi\)−ε\)​Vi\+M​∑j=1i−1\|ej\|\\displaystyle\\leq\-\(a\_\{c\}\(t,z\_\{i\}\)\-\\varepsilon\)\\,V\_\{i\}\+M\\sum\_\{j=1\}^\{i\-1\}\|e\_\{j\}\|≤−\(αc​\(t\)−ε\)​Vi\+∑j=1i−1Kj​exp⁡\(−ρj2​t\),\\displaystyle\\leq\-\(\\alpha\_\{c\}\(t\)\-\\varepsilon\)\\,V\_\{i\}\+\\sum\_\{j=1\}^\{i\-1\}K\_\{j\}\\exp\\left\(\-\\frac\{\\rho\_\{j\}\}\{2\}t\\right\),for eacht∈\[\(k−1\)T,kT\[t\\in\[\(k\-1\)T,kT\[\\,, where we used \([16](https://arxiv.org/html/2609.17997#S4.E16)\) and denotedKj=M​2​Mj​djK\_\{j\}=M\\sqrt\{2M\_\{j\}d\_\{j\}\}for brevity\. The integrating factorϕ⁡\(t\)=exp⁡\(∫\(k−1\)​Tt\(αc​\(s\)−ε\)​𝑑s\)\\phi\(t\)=\\exp\\left\(\\int\_\{\(k\-1\)T\}^\{t\}\(\\alpha\_\{c\}\(s\)\-\\varepsilon\)\\,ds\\right\)gives: dd​t​\(ϕ​\(t\)​Vi\)\\displaystyle\\frac\{d\}\{dt\}\(\\phi\(t\)V\_\{i\}\)≤∑j=1i−1Kj​exp⁡\(−ρj2​t\)​ϕ​\(t\)\\displaystyle\\leq\\sum\_\{j=1\}^\{i\-1\}K\_\{j\}\\exp\\left\(\-\\frac\{\\rho\_\{j\}\}\{2\}t\\right\)\\phi\(t\)≤∑j=1i−1Kj​exp⁡\(−ρj2​\(k−1\)​T\)​ϕ​\(t\),\\displaystyle\\leq\\sum\_\{j=1\}^\{i\-1\}K\_\{j\}\\exp\\left\(\-\\frac\{\\rho\_\{j\}\}\{2\}\(k\-1\)T\\right\)\\phi\(t\),\(32\)for eacht∈\[\(k−1\)T,kT\[t\\in\[\(k\-1\)T,kT\[\\,\. Note that: ϕ​\(k​T\)−1\\displaystyle\\phi\(kT\)^\{\-1\}=exp\(−∫\(k−1\)​Tk​T\(αc\(s\)−ε\)ds\)\\displaystyle=\\exp\\left\(\-\\int\_\{\(k\-1\)T\}^\{kT\}\(\\alpha\_\{c\}\(s\)\-\\varepsilon\)\\,ds\\right\)≤exp⁡\(−\(ρ−ε\)​T\)=θ,\\displaystyle\\leq\\exp\(\-\(\\rho\-\\varepsilon\)T\)=\\theta,where we used \([31](https://arxiv.org/html/2609.17997#A0.E31)\)\. Thus, by integrating \([32](https://arxiv.org/html/2609.17997#A0.E32)\) betweent=\(k−1\)​Tt=\(k\-1\)Tandt=k​Tt=kT, we obtain: Vi​\(k​T\)\\displaystyle V\_\{i\}\(kT\)≤Vi​\(\(k−1\)​T\)​ϕ​\(k​T\)−1\\displaystyle\\leq V\_\{i\}\(\(k\-1\)T\)\\phi\(kT\)^\{\-1\}\+K∫\(k−1\)​Tk​Texp\(−∫tk​T\(αc\(s\)−ε\)ds\)dt\\displaystyle\+K\\int\_\{\(k\-1\)T\}^\{kT\}\\exp\\left\(\-\\int\_\{t\}^\{kT\}\(\\alpha\_\{c\}\(s\)\-\\varepsilon\)\\,ds\\right\)dt≤θ​Vi​\(\(k−1\)​T\)\+K​T​exp⁡\(−ηε​T\),\\displaystyle\\leq\\theta\\,V\_\{i\}\(\(k\-1\)T\)\+KT\\exp\(\-\\eta\_\{\\varepsilon\}\\,T\),where we used thatαc​\(t\)−ε≥ηε\\alpha\_\{c\}\(t\)\-\\varepsilon\\geq\\eta\_\{\\varepsilon\}andϕ⁡\(\(k−1\)​T\)=1\\phi\(\(k\-1\)T\)=1\.
3. 3\.From the previous item, as well asVi\(\(k−1\)T\)∈\[0,di\[V\_\{i\}\(\(k\-1\)T\)\\in\[0,d\_\{i\}\[and Lemma[\.1](https://arxiv.org/html/2609.17997#A0.Thmlemma1), we conclude: Vi​\(k​T\)≤θ​di\+H​∑j=1i−12​Mj​dj<di,\\displaystyle V\_\{i\}\(kT\)\\leq\\theta d\_\{i\}\+H\\sum\_\{j=1\}^\{i\-1\}\\sqrt\{2M\_\{j\}d\_\{j\}\}<d\_\{i\},where we recall thatH=T​M​exp⁡\(−ηε​T\)H=TM\\,\\exp\(\-\\eta\_\{\\varepsilon\}T\)\.

∎

Now we write the proofs for section[V](https://arxiv.org/html/2609.17997#S5)\. We begin by showing the bounds for the projected variables\.

###### Proof:

Firstly, note that\]ceff,bimax\[≠∅\]c\_\{\\textrm\{eff\}\},b\_\{i\}^\{\\textrm\{max\}\}\[\\,\\neq\\emptysetasZ0∉ℬℓ​\(μ~\)Z\_\{0\}\\not\\in\\mathcal\{B\}\_\{\\ell\}\(\\tilde\{\\mu\}\)\. Note thatVi\|\]ci,1\[\>0V\_\{i\}\|\_\{\]c\_\{i\},1\[\}\>0andVi​\(1\)=0V\_\{i\}\(1\)=0\. Hence, its dynamics readily follows from \([24](https://arxiv.org/html/2609.17997#A0.E24)\), \([25](https://arxiv.org/html/2609.17997#A0.E25)\), \([26](https://arxiv.org/html/2609.17997#A0.E26)\), \([27](https://arxiv.org/html/2609.17997#A0.E27)\) and \([28](https://arxiv.org/html/2609.17997#A0.E28)\)\. Using thatV˙i=−b˙i\\dot\{V\}\_\{i\}=\-\\dot\{b\}\_\{i\}, let us bound each term separately onIc,iΣ​\(Z0,μ~\)I\_\{c,i\}^\{\\Sigma\}\(Z\_\{0\},\\tilde\{\\mu\}\)\.

For the first term, we have \(for brevity, we drop the argumenttt\):

−\([24](https://arxiv.org/html/2609.17997#A0.E24)\)\\displaystyle\-\\eqref\{PE:dotbi1\}≤−α​γ​c​\(1−bi2\),\\displaystyle\\leq\-\\alpha\\,\\gamma\\,c\\,\(1\-b\_\{i\}^\{2\}\),\(33\)where we used \([20](https://arxiv.org/html/2609.17997#S5.E20)\) andα=inf\(t,z\)∈ℝ0\+×𝕊n−1z⊤​SB​C​\(t\)​z\\alpha=\\inf\_\{\(t,z\)\\in\\mathbb\{R\}\_\{0\}^\{\+\}\\times\\mathbb\{S\}^\{n\-1\}\}z^\{\\top\}\\,S\_\{BC\}\(t\)\\,z\.

For the second term, by writingzi=bi​𝔢μ~\+1−bi2​ϵμ~z\_\{i\}=b\_\{i\}\\,\\mathfrak\{e\}^\{\\tilde\{\\mu\}\}\+\\sqrt\{1\-b\_\{i\}^\{2\}\}\\,\\epsilon\_\{\\tilde\{\\mu\}\}, withϵμ~∈𝕊n−1\\epsilon\_\{\\tilde\{\\mu\}\}\\in\\mathbb\{S\}^\{n\-1\}such thatϵμ~⊤​𝔢μ~=0\\epsilon\_\{\\tilde\{\\mu\}\}^\{\\top\}\\,\\mathfrak\{e\}^\{\\tilde\{\\mu\}\}=0, we havezi⊤​SB​C​σi​𝔢μ~=bi​\(\(𝔢μ~\)⊤​SB​C​𝔢μ~\)\+ri​\(t\)z\_\{i\}^\{\\top\}S\_\{BC\}\\,\\sigma\_\{i\}\\,\\mathfrak\{e\}^\{\\tilde\{\\mu\}\}=b\_\{i\}\\left\(\(\\mathfrak\{e\}^\{\\tilde\{\\mu\}\}\)^\{\\top\}\\,S\_\{BC\}\\,\\mathfrak\{e\}^\{\\tilde\{\\mu\}\}\\right\)\+r\_\{i\}\(t\), where:

ri=1−bi2​ϵμ~⊤​\(SB​C−\(\(𝔢μ~\)⊤​SB​C​𝔢μ~\)​𝕀n\)​𝔢μ~\.\\displaystyle r\_\{i\}=\\sqrt\{1\-b\_\{i\}^\{2\}\}\\,\\epsilon\_\{\\tilde\{\\mu\}\}^\{\\top\}\\,\(S\_\{BC\}\-\(\(\\mathfrak\{e\}^\{\\tilde\{\\mu\}\}\)^\{\\top\}S\_\{BC\}\\,\\mathfrak\{e\}^\{\\tilde\{\\mu\}\}\)\\mathbb\{I\}\_\{n\}\)\\,\\mathfrak\{e\}^\{\\tilde\{\\mu\}\}\.Note that\|ri\|≤α​c⋆​1−bi2\|r\_\{i\}\|\\leq\\alpha\\,c\_\{\\star\}\\,\\sqrt\{1\-b\_\{i\}^\{2\}\}\. Hence, we have:

−\([25](https://arxiv.org/html/2609.17997#A0.E25)\)\\displaystyle\-\\eqref\{PE:dotbi2\}=−∑j=1i−1σiλi​jμ~\(zi⊤SB​C𝔢μ~\)\(1−bi2\)\\displaystyle=\-\\sum\_\{j=1\}^\{i\-1\}\\sigma\_\{i\}\\,\\lambda\_\{ij\}^\{\\tilde\{\\mu\}\}\\,\(z\_\{i\}^\{\\top\}S\_\{BC\}\\,\\mathfrak\{e\}^\{\\tilde\{\\mu\}\}\)\(1\-b\_\{i\}^\{2\}\)=−∑j=1i−1λi​jμ~\(bi\(\(𝔢μ~\)⊤SB​C𝔢μ~\)\+ri\(t\)\)\(1−bi2\)\\displaystyle=\-\\sum\_\{j=1\}^\{i\-1\}\\lambda\_\{ij\}^\{\\tilde\{\\mu\}\}\\left\(b\_\{i\}\\left\(\(\\mathfrak\{e\}^\{\\tilde\{\\mu\}\}\)^\{\\top\}\\,S\_\{BC\}\\,\\mathfrak\{e\}^\{\\tilde\{\\mu\}\}\\right\)\+r\_\{i\}\(t\)\\right\)\(1\-b\_\{i\}^\{2\}\)≤−\(i−1\)​α​\(λmin​c−λmax​c⋆​1−c2\)​\(1−bi2\)\.\\displaystyle\\leq\-\(i\-1\)\\,\\alpha\\left\(\\lambda\_\{\\min\}\\,c\-\\lambda\_\{\\max\}\\,c\_\{\\star\}\\sqrt\{1\-c^\{2\}\}\\right\)\(1\-b\_\{i\}^\{2\}\)\.\(34\)
From \([33](https://arxiv.org/html/2609.17997#A0.E33)\) and \([34](https://arxiv.org/html/2609.17997#A0.E34)\), we obtain:

−\([24](https://arxiv.org/html/2609.17997#A0.E24)\)−\([25](https://arxiv.org/html/2609.17997#A0.E25)\)≤\\displaystyle\-\\eqref\{PE:dotbi1\}\-\\eqref\{PE:dotbi2\}\\leq−α⁡\(γ​c\+\(i−1\)​λmin​cCLOSE\\displaystyle\-\\alpha\\Big\(\\gamma\\,c\+\(i\-1\)\\,\\lambda\_\{\\min\}\\,cOPEN−\(i−1\)​λmax​c⋆​1−c2\)​\(1−bi2\)\\displaystyle\-\(i\-1\)\\,\\lambda\_\{\\max\}\\,c\_\{\\star\}\\sqrt\{1\-c^\{2\}\}\\Big\)\(1\-b\_\{i\}^\{2\}\)≤\\displaystyle\\leq−α​β​\(1\+c\)​Vi,\\displaystyle\-\\alpha\\,\\beta\\,\(1\+c\)\\,V\_\{i\},\(35\)whereβ=c⁡\(γ\+\(i−1\)​λmin\)−c⋆​λmax​\(i−1\)​1−c2\\beta=c\\,\(\\gamma\+\(i\-1\)\\,\\lambda\_\{\\min\}\)\-c\_\{\\star\}\\,\\lambda\_\{\\max\}\\,\(i\-1\)\\,\\sqrt\{1\-c^\{2\}\}\. Note thatβ∈ℝ\+\\beta\\in\\mathbb\{R\}^\{\+\}\. Indeed, by squaring:

c⁡\(γ\+\(i−1\)​λmin\)\>c⋆​λmax​\(i−1\)​1−c2\\displaystyle c\\,\(\\gamma\+\(i\-1\)\\,\\lambda\_\{\\min\}\)\>c\_\{\\star\}\\,\\lambda\_\{\\max\}\\,\(i\-1\)\\,\\sqrt\{1\-c^\{2\}\}and rearranging terms, we obtain:

c2​\(\(γ\+\(ℓ−1\)​λmin\)2\+c⋆2​λmax2​\(ℓ−1\)2\)\\displaystyle c^\{2\}\\left\(\(\\gamma\+\(\\ell\-1\)\\,\\lambda\_\{\\min\}\)^\{2\}\+c\_\{\\star\}^\{2\}\\,\\lambda\_\{\\max\}^\{2\}\\,\(\\ell\-1\)^\{2\}\\right\)≥c2​\(\(γ\+\(i−1\)​λmin\)2\+c⋆2​λmax2​\(i−1\)2\)\\displaystyle\\quad\\geq c^\{2\}\\left\(\(\\gamma\+\(i\-1\)\\,\\lambda\_\{\\min\}\)^\{2\}\+c\_\{\\star\}^\{2\}\\,\\lambda\_\{\\max\}^\{2\}\\,\(i\-1\)^\{2\}\\right\)\>c⋆2​λmax2​\(i−1\)2≥c⋆2​λmax2\.\\displaystyle\\quad\>c\_\{\\star\}^\{2\}\\,\\lambda\_\{\\max\}^\{2\}\\,\(i\-1\)^\{2\}\\geq c\_\{\\star\}^\{2\}\\,\\lambda\_\{\\max\}^\{2\}\.This holds provided:

c\>c⋆​λmax\(γ\+\(ℓ−1\)​λmin\)2\+c⋆2​λmax2​\(ℓ−1\)2=ceff\.c\>\\frac\{c\_\{\\star\}\\,\\lambda\_\{\\max\}\}\{\\sqrt\{\(\\gamma\+\(\\ell\-1\)\\,\\lambda\_\{\\min\}\)^\{2\}\+c\_\{\\star\}^\{2\}\\,\\lambda\_\{\\max\}^\{2\}\\,\(\\ell\-1\)^\{2\}\}\}=c\_\{\\textrm\{eff\}\}\.
Lastly, from Assumption[III\.1](https://arxiv.org/html/2609.17997#S3.Thmassumption1)and Lemma[III\.1](https://arxiv.org/html/2609.17997#S3.Thmlemma1), as well as compactness of the sphere, the error terms are all bounded:

−\([26](https://arxiv.org/html/2609.17997#A0.E26)\)−\([27](https://arxiv.org/html/2609.17997#A0.E27)\)−\([28](https://arxiv.org/html/2609.17997#A0.E28)\)≤M​∑j=1i−1\|ej\|\.\\displaystyle\-\\eqref\{PE:dotbi3\}\-\\eqref\{PE:dotbi4\}\-\\eqref\{PE:dotbi5\}\\leq M\\sum\_\{j=1\}^\{i\-1\}\|e\_\{j\}\|\.\(36\)
By gathering \([35](https://arxiv.org/html/2609.17997#A0.E35)\) and \([36](https://arxiv.org/html/2609.17997#A0.E36)\), and denotingκ=α​β​\(1\+c\)∈ℝ\+\\kappa=\\alpha\\,\\beta\\,\(1\+c\)\\in\\mathbb\{R\}^\{\+\}, we conclude:

V˙i\\displaystyle\\dot\{V\}\_\{i\}≤−κ​Vi\+M​∑j=1i−1\|ej\|\.\\displaystyle\\leq\-\\kappa\\,V\_\{i\}\+M\\sum\_\{j=1\}^\{i\-1\}\|e\_\{j\}\|\.
∎

Lastly, the next result is readily obtained by an ISS argument\. It guarantees convergence to consensus of the last token at the inductive step in the proof of Theorem[V\.1](https://arxiv.org/html/2609.17997#S5.Thmtheorem1)\.

###### Lemma \.3

Letb:ℝ0\+→\[−1,1\]b:\\mathbb\{R\}\_\{0\}^\{\+\}\\to\[\-1,1\]be continuously differentiable ande:ℝ0\+→ℝ0\+e:\\mathbb\{R\}\_\{0\}^\{\+\}\\to\\mathbb\{R\}\_\{0\}^\{\+\}be continuous withlimt→∞e⁡\(t\)=0\\lim\_\{t\\to\\infty\}e\(t\)=0\. If there existκ∈ℝ\+\\kappa\\in\\mathbb\{R\}^\{\+\}andt0∈ℝ0\+t\_\{0\}\\in\\mathbb\{R\}\_\{0\}^\{\+\}such thatb˙≥κ⁡\(1−b\)−e⁡\(t\)\\dot\{b\}\\geq\\kappa\\,\(1\-b\)\-e\(t\)for eacht∈\[t0,∞\[t\\in\[t\_\{0\},\\infty\[\\,, thenlimt→∞b⁡\(t\)=1\\lim\_\{t\\to\\infty\}b\(t\)=1\.