Drive vs. Decay: On the Training Dynamics of Joint-Embedding Predictive Architectures

# On the Training Dynamics ofJoint-Embedding Predictive Architectures
Source: [https://arxiv.org/html/2610.02344](https://arxiv.org/html/2610.02344)
## Drive vs\. Decay: On the Training Dynamics of Joint\-Embedding Predictive Architectures

Seong Woo AhnFabrice PopineauAffiliation:Arpad Rimmel Bich\-Liên DoanAffiliation:Université Paris\-Saclay, CNRS, CentraleSupélec, LISN, Orsay, FranceEmail:[jose\.lucas\-de\-melo\-costa@universite\-paris\-saclay\.fr](mailto:)

###### Abstract

Joint\-Embedding Predictive Architectures \(JEPAs\) are prone to representation collapse, typically mitigated through empirical heuristics\. We develop an early\-training stability theory that unifies these heuristics\. Linearising the coupled JEPA gradient flow around the trivial fixed point reveals two competing effects: a driving force \(γ\\gamma\) and a decay effect \(σ\\sigma\)\. Under approximate spectral decoupling, a per\-mode stability ratioμi=γi/σi\\mu\_\{i\}=\\gamma\_\{i\}/\\sigma\_\{i\}factorises into independent data\-side and predictor\-side terms and the count of unstable modes tracks the rank of representations that can emerge\. The framework predicts a phase boundary, which we confirm empirically across more than 800 Tabular\-JEPA configurations\. It also unifies predictor scaling, masking ratio, and EMA as distinct mechanisms for shiftingμ\\mu\. Guided by this analysis, we introduce ResidualPred, a transformer predictor whose attention is biased toward the identity at initialisation; it improves both effective rank and downstream accuracy on tabular benchmarks and in I\-JEPA pretraining on CIFAR\-10, CIFAR\-100, STL\-10, and ImageNet\. Our framework connects empirical collapse\-avoidance heuristics to an explicit dynamical picture, yielding theory\-driven stabilizers\. Code is available at[https://github\.com/jose\-melo/drive\-vs\-decay](https://github.com/jose-melo/drive-vs-decay)\.

## 1Introduction

Figure 1:Phase boundary\.Learning where the drive wins \(μ=γ/σ\>1\\mu=\\gamma/\\sigma\>1\), collapse where the decay wins \(μ<1\\mu<1\)\.Self\-Supervised Learning \(SSL\) has emerged as a powerful approach for representation learning, enabling models to learn meaningful features from vast amounts of unlabeled data\([Shwartz\-Ziv and LeCun, 2024](https://arxiv.org/html/2610.02344#bib.bib27)\)\. Among SSL techniques, non\-contrastive methods have gained popularity as they avoid the need for explicit negative sampling\([Grill et al\., 2020](https://arxiv.org/html/2610.02344#bib.bib17);[Bardes et al\., 2022](https://arxiv.org/html/2610.02344#bib.bib6);[Zhang et al\., 2022](https://arxiv.org/html/2610.02344#bib.bib37)\)and Joint Embedding Predictive Architectures \(JEPAs\) are a notable variant within this family, operating by predicting the latent representation of a target view of the input \(e\.g\., a masked portion\) from the representation of a context view \(e\.g\., the remaining portion\)\([Assran et al\., 2023](https://arxiv.org/html/2610.02344#bib.bib1);[Fei et al\., 2024](https://arxiv.org/html/2610.02344#bib.bib14);[Dong et al\., 2024](https://arxiv.org/html/2610.02344#bib.bib12)\)Despite the empirical success of JEPAs, a key concern is the potential for representational collapse, where the learned embeddings converge to a trivial solution or low\-rank subspace, rendering the representations low\-informative for the downstream task\.

How can we prevent representational collapse when the collapsed solution can also minimize the very mean\-squared prediction objective we optimize? Most existing answers have been empirical: the asymmetric predictor, a stop\-gradient on the target branch, and slowly evolving targets via an exponential moving average \(EMA\)[Assran et al\. \(2023\)](https://arxiv.org/html/2610.02344#bib.bib1)\. A growing line of work has begun to formalize these stabilizers and to propose alternatives, including carefully designed predictor initializations[Bagatella et al\. \(2026\)](https://arxiv.org/html/2610.02344#bib.bib3), frozen\-teacher formulations[Li et al\. \(2026\)](https://arxiv.org/html/2610.02344#bib.bib22), and auxiliary regularizations that enforce embedding diversity\. These solutions are largely justified by empirical heuristics, underscoring the need for a theory of JEPA training dynamics\. Existing theoretical analyses have primarily focused on architectures augmented with explicit diversity\-inducing regularizers[Balestriero and LeCun \(2025\)](https://arxiv.org/html/2610.02344#bib.bib4);[Yu et al\. \(2025\)](https://arxiv.org/html/2610.02344#bib.bib36)\. In parallel, the seminal study of[Tian et al\. \(2021\)](https://arxiv.org/html/2610.02344#bib.bib31)dissected several anti\-collapse mechanisms in Siamese non\-contrastive frameworks, offering key dynamical insights\. Yet, a substantial gap remains: we still lack a complete characterization of the original JEPA objective and under realistic update rules\.

Therefore, this work proposes a theoretical framework for the training dynamics of JEPAs that yields principled mechanisms for avoiding early\-stage collapse\. Our analysis reveals that representation learning is governed by the spectral competition between two operators: a*driving*force, induced by view construction \(e\.g\., masking\) and predictor\-target alignment, and a*decay*force, induced by predictor capacity and the data covariance structure \(Figure[1](https://arxiv.org/html/2610.02344#S1.F1)\)\. This perspective unifies several empirical observations in the JEPA literature, and it suggests new architectural and initialization strategies that improve stability\. We validate the theory through a large\-scale study of800800Tabular\-JEPA variants on standard tabular benchmarks, and we further demonstrate its relevance beyond the linear regime using transformer\-based encoders\. Our main contributions are:

- •A drive–decay stability theory for JEPAs\.We show that collapse stability is governed by a per\-mode ratioμi=γi/σi\\mu\_\{i\}=\\gamma\_\{i\}/\\sigma\_\{i\}\. The thresholdμ\>1\\mu\>1predicts escape from collapse, while the number of super\-critical modes yields a rank budget for the emerging representation\.
- •A unified account of collapse controls\.Across more than800800configurations, we show that predictor scale and context masking act as control knobs forμ\\mu, while EMA acts as a timescale parameter\. Together, these mechanisms explain transitions in effective rank and downstream performance\. In the linear model, seven collapse\-prevention mechanisms reduce to doses onμ\\mu\.
- •A theory\-guided transformer predictor\.We introduceResidualPred, an identity\-biased transformer predictor with diagonal attention bias\. It improves effective rank and linear\-probe accuracy over standard predictors on brittle tabular settings and I\-JEPA image benchmarks\.

## 2Related Work

##### JEPAs and collapse\-avoidance techniques\.

Joint Embedding Predictive Architectures \(JEPAs\) are self\-supervised models that learn by predicting high\-level representations rather than reconstructing raw inputs\([Assran et al\., 2023](https://arxiv.org/html/2610.02344#bib.bib1);[Bardhan et al\., 2025](https://arxiv.org/html/2610.02344#bib.bib7);[Dong et al\., 2024](https://arxiv.org/html/2610.02344#bib.bib12);[Skenderi et al\., 2025](https://arxiv.org/html/2610.02344#bib.bib28)\)\. A standard JEPA contains a*context encoder*, a*target encoder*, and a*predictor*: the context encoder processes a masked view of the input, the target encoder encodes a target view, and the predictor is trained to match the target representation\. To avoid trivial solutions, where all inputs are mapped to the same representation, JEPAs usually rely on anti\-collapse mechanisms such as EMA target updates\([Grill et al\., 2020](https://arxiv.org/html/2610.02344#bib.bib17)\), variance–covariance regularization\([Mo and Tong, 2024](https://arxiv.org/html/2610.02344#bib.bib24);[Drozdov et al\., 2024](https://arxiv.org/html/2610.02344#bib.bib13)\), frozen teachers\([Li et al\., 2026](https://arxiv.org/html/2610.02344#bib.bib22)\), or distributional constraints such as SIGReg\([Balestriero and LeCun, 2025](https://arxiv.org/html/2610.02344#bib.bib4)\)\. Recent probabilistic and theoretical perspectives further connect these regularizers to embedding\-distribution control and density estimation\([Huang, 2026](https://arxiv.org/html/2610.02344#bib.bib21);[Balestriero et al\., 2025](https://arxiv.org/html/2610.02344#bib.bib5)\)\. Thus, while the JEPA principle is simple, its practical success depends critically on mechanisms that prevent representational collapse\.

##### Training Dynamics of Non\-Contrastive SSL

Understanding how non\-contrastive SSL methods, including JEPAs, avoid collapse without explicit negative pairs remains an active research topic\.[Tian et al\. \(2021\)](https://arxiv.org/html/2610.02344#bib.bib31)studied SimSiam\-like dynamics with linear predictors, showing that the interaction between the predictor and stop\-gradient prevents gradient alignment toward collapse, acting as an implicit regularizer\.[Zhang et al\. \(2022\)](https://arxiv.org/html/2610.02344#bib.bib37)extended this view through a unified gradient\-based analysis, identifying the gradient components that preserve representation diversity across non\-contrastive methods\. Related work further highlights predictor networks as implicit variance regularizers\([Halvagal et al\., 2023](https://arxiv.org/html/2610.02344#bib.bib18)\), reads EMA and stop\-gradient as orthonormalisation of a linear predictor\([Richemond et al\., 2023](https://arxiv.org/html/2610.02344#bib.bib26)\), while JEPA\-specific analyses suggest that these architectures may also benefit from an inherent learning bias against collapse\([Littwin et al\., 2024](https://arxiv.org/html/2610.02344#bib.bib23)\)\.

##### Identity\-biased initialisation\.

Biasing residual blocks toward the identity at initialisation is well established outside JEPAs, notably in Fixup\([Zhang et al\., 2019](https://arxiv.org/html/2610.02344#bib.bib38)\), ReZero\([Bachlechner et al\., 2021](https://arxiv.org/html/2610.02344#bib.bib2)\), SkipInit\([De and Smith, 2020](https://arxiv.org/html/2610.02344#bib.bib10)\)and IDInit\([Pan et al\., 2025](https://arxiv.org/html/2610.02344#bib.bib25)\)\. In these works, identity\-biased initialisation is mainly used to stabilise signal propagation in deep networks\. This work studies it in a different setting and for a different purpose\. Rather than focusing on signal propagation, we examine how such initialisation affects the training dynamics of joint\-embedding predictive models and their tendency to collapse\. Closer still, mimetic initialisation\([Trockman and Kolter, 2023](https://arxiv.org/html/2610.02344#bib.bib32)\)sets the query–key product of self\-attention near the identity, and[Wen and Li \(2022\)](https://arxiv.org/html/2610.02344#bib.bib35)analyse how a trainable, possibly identity\-initialised prediction head shapes what non\-contrastive training learns; we claim no architectural novelty for identity\-biasing attention, and none of these works contains a per\-mode stability ratio defined through the predictor at the JEPA fixed point, a supercriticality count, or a rank budget\. We are closest in spirit to TD\-JEPA\([Bagatella et al\., 2026](https://arxiv.org/html/2610.02344#bib.bib3)\), which also connects predictor initialisation to collapse prevention, but in a TD\-RL setting; this work provides a more general approach derived from the new drive\-decay framework\.

## 3Drive vs\. Decay: Theoretical Analysis of Coupled JEPA Dynamics

We present a theoretical analysis of the Joint Embedding Predictive Architecture \(JEPA\) training dynamics\. Unlike previous approaches that assume a static relationship between encoders, we model the system as a coupled dynamical system governed by the interaction between the gradient descent updates and the Exponential Moving Average \(EMA\) mechanism\.

### 3\.1Linear Model: Coupled Dynamical System Formulation

Let the input data be represented by a vector𝐱∈ℝN\\mathbf\{x\}\\in\\mathbb\{R\}^\{N\}, whereNNis the input dimensionality\. The context encoderfθf\_\{\\theta\}is represented by a weight matrix𝐖∈ℝd×N\\mathbf\{W\}\\in\\mathbb\{R\}^\{d\\times N\}\. The target encoderfθ¯f\_\{\\bar\{\\theta\}\}is represented by𝐖a∈ℝd×N\\mathbf\{W\}\_\{a\}\\in\\mathbb\{R\}^\{d\\times N\}\. The predictorgϕg\_\{\\phi\}is represented by𝐖p∈ℝd×d\\mathbf\{W\}\_\{p\}\\in\\mathbb\{R\}^\{d\\times d\}\.

𝐱\\mathbf\{x\}m1m\_\{1\}𝐱1\\mathbf\{x\}\_\{1\}fθf\_\{\\theta\}gϕg\_\{\\phi\}fθ¯f\_\{\\bar\{\\theta\}\}m2m\_\{2\}ℒ\\mathcal\{L\}loss

Figure 2:Linear JEPA architecture\.Masking operations define the context and target views\. A context mask yields𝐱1∈ℝN\\mathbf\{x\}\_\{1\}\\in\\mathbb\{R\}^\{N\}, which serves as input to the context encoder\. This masking operation can be conceptualized as zeroing out elements of𝐱\\mathbf\{x\}by an element\-wise multiplication with a random binary feature mask𝐦1\\mathbf\{m\}\_\{1\}\. The target encoder receives the original input𝐱\\mathbf\{x\}and a target feature mask𝐦2∈\{0,1\}d\\mathbf\{m\}\_\{2\}\\in\\\{0,1\\\}^\{d\}is applied to the output of the target encoder, as presented in Figure[2](https://arxiv.org/html/2610.02344#S3.F2)\. We further consider𝐌𝟐=diag​\(𝐦𝟐\)\\mathbf\{M\_\{2\}\}=\\text\{diag\}\(\\mathbf\{m\_\{2\}\}\), withm2i∼Bernoulli​\(pt​g​t\)m\_\{2\}^\{i\}\\sim\\text\{Bernoulli\}\(p\_\{tgt\}\), where now we assume that the selected target features are masked for each row\.

The JEPA objective is to minimize the expected squared Euclidean distance between the predicted representation and the masked target representation:

ℒ⁡\(𝐖,𝐖p,𝐖a\)=12​𝔼​\[‖𝐖p​𝐖𝐱1−sg​\(𝐌2​𝐖a​𝐱\)‖22\]\\mathcal\{L\}\(\\mathbf\{W\},\\mathbf\{W\}\_\{p\};\\mathbf\{W\}\_\{a\}\)=\\frac\{1\}\{2\}\\mathbb\{E\}\\left\[\\\|\\mathbf\{W\}\_\{p\}\\mathbf\{W\}\\mathbf\{x\}\_\{1\}\-\\texttt\{sg\}\(\\mathbf\{M\}\_\{2\}\\mathbf\{W\}\_\{a\}\\mathbf\{x\}\)\\\|\_\{2\}^\{2\}\\right\]\(1\)wheresgis an operation that prevents the backpropagation of gradients through its argument\. In the context of this loss function, it means that the term𝐌2​𝐖a​𝐱\\mathbf\{M\}\_\{2\}\\mathbf\{W\}\_\{a\}\\mathbf\{x\}is treated as a fixed target or a constant when computing the gradients for updating the parameters𝐖\\mathbf\{W\}and𝐖p\\mathbf\{W\}\_\{p\}\. Note that𝐖a\\mathbf\{W\}\_\{a\}is typically updated offline, not directly by minimizing this loss\.

The system evolves according to continuous\-time dynamics where𝐖\\mathbf\{W\}and𝐖p\\mathbf\{W\}\_\{p\}follow the negative gradient ofℒ\\mathcal\{L\}, while𝐖a\\mathbf\{W\}\_\{a\}follows the context weights via EMA\. The Lemma[3\.1](https://arxiv.org/html/2610.02344#S3.Thmtheorem1)states their relationship \(we refer the reader to Appendix[A](https://arxiv.org/html/2610.02344#A1)for the details\)\.

###### Lemma 3\.1\(Coupled JEPA Dynamics\)\.

Letβ\>0\\beta\>0represent the EMA update speed\. The state evolution is governed by:

𝐖˙=−𝐖p⊤​𝐖p​𝐖​𝚺11\+𝐖p⊤​𝐌¯2​𝐖a​𝚺X​1,𝐖˙a=β⁡\(𝐖−𝐖a\),\\dot\{\\mathbf\{W\}\}=\-\\mathbf\{W\}\_\{p\}^\{\\top\}\\mathbf\{W\}\_\{p\}\\mathbf\{W\}\\mathbf\{\\Sigma\}\_\{11\}\+\\mathbf\{W\}\_\{p\}^\{\\top\}\\bar\{\\mathbf\{M\}\}\_\{2\}\\mathbf\{W\}\_\{a\}\\mathbf\{\\Sigma\}\_\{X1\},\\qquad\\quad\\dot\{\\mathbf\{W\}\}\_\{a\}=\\beta\(\\mathbf\{W\}\-\\mathbf\{W\}\_\{a\}\),\(2\)where𝚺11=𝔼⁡\[𝐱1​𝐱1⊤\]\\mathbf\{\\Sigma\}\_\{11\}=\\mathbb\{E\}\[\\mathbf\{x\}\_\{1\}\\mathbf\{x\}\_\{1\}^\{\\top\}\]and𝚺X​1=𝔼⁡\[𝐱𝐱1⊤\]\\mathbf\{\\Sigma\}\_\{X1\}=\\mathbb\{E\}\[\\mathbf\{x\}\\mathbf\{x\}\_\{1\}^\{\\top\}\]are data covariance matrices\. For the stability analysis𝐖p\\mathbf\{W\}\_\{p\}is taken as given: the framework governs encoder\-side escape given the predictor’s state, and Lemma[3\.4](https://arxiv.org/html/2610.02344#S3.Thmtheorem4)below says what that costs when the predictor is trained jointly\.

### 3\.2Jacobian Stability Analysis

To analyze representational collapse, we examine the stability of the trivial fixed point𝐖=𝟎,𝐖a=𝟎\\mathbf\{W\}=\\mathbf\{0\},\\mathbf\{W\}\_\{a\}=\\mathbf\{0\}\. We define the full state vector of the system as𝐳=\[𝐯𝐞𝐜​\(𝐖\)⊤,𝐯𝐞𝐜​\(𝐖a\)⊤\]⊤∈ℝ2​d​N\\mathbf\{z\}=\[\\mathbf\{vec\}\(\\mathbf\{W\}\)^\{\\top\},\\mathbf\{vec\}\(\\mathbf\{W\}\_\{a\}\)^\{\\top\}\]^\{\\top\}\\in\\mathbb\{R\}^\{2dN\}\.

The linearized dynamics around the origin are given by𝐳˙=𝒥​𝐳\\dot\{\\mathbf\{z\}\}=\\mathcal\{J\}\\mathbf\{z\}, where𝒥\\mathcal\{J\}is the Jacobian matrix:

𝒥=\[−\(𝚺11⊗𝐖p⊤​𝐖p\)\(𝚺X​1⊤⊗𝐖p⊤​𝐌¯2\)β​𝐈d​N−β​𝐈d​N\]\\mathcal\{J\}=\\begin\{bmatrix\}\-\(\\mathbf\{\\Sigma\}\_\{11\}\\otimes\\mathbf\{W\}\_\{p\}^\{\\top\}\\mathbf\{W\}\_\{p\}\)&\(\\mathbf\{\\Sigma\}\_\{X1\}^\{\\top\}\\otimes\\mathbf\{W\}\_\{p\}^\{\\top\}\\bar\{\\mathbf\{M\}\}\_\{2\}\)\\\\ \\beta\\mathbf\{I\}\_\{dN\}&\-\\beta\\mathbf\{I\}\_\{dN\}\\end\{bmatrix\}\(3\)
The spectral properties of𝒥\\mathcal\{J\}dictate the system’s fate: if all eigenvalues satisfyRe​\(λ\)<0\\text\{Re\}\(\\lambda\)<0, the origin is a stable attractor, leading to model collapse\. Conversely, learning requires an unstable origin \(∃λ:Re​\(λ\)\>0\\exists\\lambda:\\text\{Re\}\(\\lambda\)\>0\) to enable the growth of non\-trivial representations\.

###### Assumption 3\.2\(Approximate spectral decoupling\)\.

For analysis, we work in a basis where the*data\-side*covariances and the*feature\-side*predictor operators are \(approximately\) jointly diagonalizable\. Concretely, we assume𝚺11\\mathbf\{\\Sigma\}\_\{11\}and𝚺X​1\\mathbf\{\\Sigma\}\_\{X1\}share an eigenbasis, and𝐏≔𝐖p⊤​𝐖p\\mathbf\{P\}\\coloneqq\\mathbf\{W\}\_\{p\}^\{\\top\}\\mathbf\{W\}\_\{p\}and𝐐≔𝐖p⊤​𝐌¯2\\mathbf\{Q\}\\coloneqq\\mathbf\{W\}\_\{p\}^\{\\top\}\\bar\{\\mathbf\{M\}\}\_\{2\}share an eigenbasis\. This lets the coupled dynamics decompose into independent scalar modes\.

The Assumption[3\.2](https://arxiv.org/html/2610.02344#S3.Thmtheorem2)is mild in the regimes we study\. On the*data side*, when the context view𝐱1\\mathbf\{x\}\_\{1\}is produced by mild masking, the cross\-covariance𝚺X​1\\mathbf\{\\Sigma\}\_\{X1\}is typically close to a scaled version of𝚺11\\mathbf\{\\Sigma\}\_\{11\}; under common preprocessing \(normalization\), covariances are often near\-diagonal\. On the*feature side*,𝐏=𝐖p⊤​𝐖p\\mathbf\{P\}=\\mathbf\{W\}\_\{p\}^\{\\top\}\\mathbf\{W\}\_\{p\}and𝐐=𝐖p⊤​𝐌¯2\\mathbf\{Q\}=\\mathbf\{W\}\_\{p\}^\{\\top\}\\bar\{\\mathbf\{M\}\}\_\{2\}are close to commuting when𝐌¯2\\bar\{\\mathbf\{M\}\}\_\{2\}is diagonal and𝐖p\\mathbf\{W\}\_\{p\}is near\-orthogonal at initialization or kept so by regularization\.

By exploiting the block structure \(see Appendix[A](https://arxiv.org/html/2610.02344#A1)for step\-by\-step derivation\), we find that:

γi\\displaystyle\\gamma\_\{i\}∈Spec​\(\(𝚺X​1⊤⊗𝐖p⊤​𝐌¯2\)CLOSE\\displaystyle\\in\\text\{Spec\}\(\(\\mathbf\{\\Sigma\}\_\{X1\}^\{\\top\}\\otimes\\mathbf\{W\}\_\{p\}^\{\\top\}\\bar\{\\mathbf\{M\}\}\_\{2\}\)driving operator\(4\)σi\\displaystyle\\sigma\_\{i\}∈Spec​\(𝚺11⊗𝐖p⊤​𝐖p\)\\displaystyle\\in\\text\{Spec\}\(\\mathbf\{\\Sigma\}\_\{11\}\\otimes\\mathbf\{W\}\_\{p\}^\{\\top\}\\mathbf\{W\}\_\{p\}\)decay operator\(5\)μi\\displaystyle\\mu\_\{i\}≔γiσi\\displaystyle\\coloneqq\\frac\{\\gamma\_\{i\}\}\{\\sigma\_\{i\}\}stability variable\(6\)
As a result, each spatial eigenmodeiievolves according to the characteristic equationλ2\+\(β\+σi\)​λ\+β​σi​\(1−μi\)=0\\lambda^\{2\}\+\(\\beta\+\\sigma\_\{i\}\)\\lambda\+\\beta\\sigma\_\{i\}\(1\-\\mu\_\{i\}\)=0, which fully determines the local stability of the trivial solution along that mode\. This form exposes two qualitatively distinct behaviors\. First, whenμi\>1\\mu\_\{i\}\>1, the constant term becomes negative, yielding a positive real eigenvalue\. The origin is then unstable and the corresponding mode exhibits exponential growth, corresponding to the learning regime\. In general, the driving operator need not be symmetric, and its eigenvalues may be complex\. A detailed discussion of these conditions and alternative real\-valued formulations based on singular values is provided in Appendix[A\.4](https://arxiv.org/html/2610.02344#A1.SS4)\. Appendix[A\.5](https://arxiv.org/html/2610.02344#A1.SS5)gives certificates without symmetry or decoupling:κ⁡\(𝚺11\)​κ​\(𝐖p⊤​𝐖p\)​μ~11<1\\kappa\(\\mathbf\{\\Sigma\}\_\{11\}\)\\,\\kappa\(\\mathbf\{W\}\_\{p\}^\{\\top\}\\mathbf\{W\}\_\{p\}\)\\,\\tilde\{\\mu\}\_\{11\}<1forces collapse, a determinant condition forces escape, and under Assumption[3\.2](https://arxiv.org/html/2610.02344#S3.Thmtheorem2)both reduce to the per\-mode criterion below\.

JEPA dynamics are governed by the spectral ratioγ/σ\\gamma/\\sigmabetween drive and decay\.•Collapse Regime \(μ≪1\\mu\\ll 1\):Decay dominates\. The trivial fixed point is astable attractor\.•Learning Regime \(μ≫1\\mu\\gg 1\):Driving force dominates\. The trivial solution becomesunstable, facilitating the learning of non\-trivial features\.

Under Assumption[3\.2](https://arxiv.org/html/2610.02344#S3.Thmtheorem2), the ratio factorizes as

μk​ℓ=\(ckak\)⋅\(qℓpℓ\),\\mu\_\{k\\ell\}\\;=\\;\\Big\(\\frac\{c\_\{k\}\}\{a\_\{k\}\}\\Big\)\\cdot\\Big\(\\frac\{q\_\{\\ell\}\}\{p\_\{\\ell\}\}\\Big\),\(7\)whereaka\_\{k\},ckc\_\{k\}are the eigenvalues of𝚺11\\mathbf\{\\Sigma\}\_\{11\}and𝚺X​1\\mathbf\{\\Sigma\}\_\{X1\}along data modekk, andpℓp\_\{\\ell\},qℓq\_\{\\ell\}are the eigenvalues of𝐖p⊤​𝐖p\\mathbf\{W\}\_\{p\}^\{\\top\}\\mathbf\{W\}\_\{p\}and𝐖p⊤​𝐌¯2\\mathbf\{W\}\_\{p\}^\{\\top\}\\bar\{\\mathbf\{M\}\}\_\{2\}along predictor modeℓ\\ell\. The two factors are independent:ck/akc\_\{k\}/a\_\{k\}is a*cross\-view retention coefficient*governed by data structure and masking, whileqℓ/pℓq\_\{\\ell\}/p\_\{\\ell\}is a*predictor alignment ratio*governed by the predictor geometry\.

### 3\.3Rank budget and joint training

The factorisation \([7](https://arxiv.org/html/2610.02344#S3.E7)\) promotes the scalar boundaryμ\>1\\mu\{\>\}1to a count of unstable directions\. Let𝒰=\{\(k,ℓ\):μk​ℓ\>1\}\\mathcal\{U\}=\\\{\(k,\\ell\):\\mu\_\{k\\ell\}\{\>\}1\\\}\. The proof and the loose\-count corollary are in Appendix[C](https://arxiv.org/html/2610.02344#A3)\.

###### Corollary 3\.3\(Super\-criticality count bounds the post\-escape rank\)\.

Under the exact form of Assumption[3\.2](https://arxiv.org/html/2610.02344#S3.Thmtheorem2)with orthonormal eigenbases and real spectra \(so that𝚺X​1\\mathbf\{\\Sigma\}\_\{X1\}and𝐖p⊤​𝐌¯2\\mathbf\{W\}\_\{p\}^\{\\top\}\\bar\{\\mathbf\{M\}\}\_\{2\}are symmetric\) and everyak​pℓ\>0a\_\{k\}p\_\{\\ell\}\>0, the linearised JEPA flow leaves the trivial fixed point along exactly the modes in𝒰\\mathcal\{U\}where the initial state has a non\-zero unstable component, and the growing component𝐖\+​\(t\)\\mathbf\{W\}\_\{\+\}\(t\)of𝐖⁡\(t\)=𝐖\+​\(t\)\+𝐖−​\(t\)\\mathbf\{W\}\(t\)=\\mathbf\{W\}\_\{\+\}\(t\)\+\\mathbf\{W\}\_\{\-\}\(t\)obeys

rank⁡\(𝐖\+​\(t\)\)≤r𝑡𝑖𝑔ℎ𝑡:=min⁡\(\|πℓ​𝒰\|,\|πk​𝒰\|,d,N\)for all​t≥0,\\rank\\\!\\big\(\\mathbf\{W\}\_\{\+\}\(t\)\\big\)\\;\\leq\\;r\_\{\\mathit\{tight\}\}\\;:=\\;\\min\\\!\\big\(\|\\pi\_\{\\ell\}\\mathcal\{U\}\|,\\;\|\\pi\_\{k\}\\mathcal\{U\}\|,\\;d,\\;N\\big\)\\qquad\\text\{for all \}t\\geq 0,\(8\)whereπℓ𝒰=\{ℓ:∃k,\(k,ℓ\)∈𝒰\}\\pi\_\{\\ell\}\\mathcal\{U\}=\\\{\\ell:\\exists k,\(k,\\ell\)\\in\\mathcal\{U\}\\\}and similarly forπk\\pi\_\{k\}\. If no mode hasμk​ℓ=1\\mu\_\{k\\ell\}=1or is defective, thenσr𝑡𝑖𝑔ℎ𝑡\+1​\(𝐖⁡\(t\)\)≤K¯​Z0​e−η​t\\sigma\_\{r\_\{\\mathit\{tight\}\}\+1\}\(\\mathbf\{W\}\(t\)\)\\leq\\bar\{K\}Z\_\{0\}e^\{\-\\eta t\}withη\\etathe slowest stable rate andK¯\\bar\{K\},Z0Z\_\{0\}computable at initialisation, so for any thresholdε\>0\\varepsilon\>0at mostr𝑡𝑖𝑔ℎ𝑡r\_\{\\mathit\{tight\}\}singular values of𝐖⁡\(t\)\\mathbf\{W\}\(t\)exceedε\\varepsilononcet≥η−1​log⁡\(K¯​Z0/ε\)t\\geq\\eta^\{\-1\}\\log\(\\bar\{K\}Z\_\{0\}/\\varepsilon\); and, in the escape regime \(𝒰≠∅\\mathcal\{U\}\\neq\\emptysetwith a non\-zero component along some escaping mode\), past an explicit initial timeERank⁡\(𝐖⁡\(t\)\)≤r𝑡𝑖𝑔ℎ𝑡\+δ⁡\(t\)\\Erank\(\\mathbf\{W\}\(t\)\)\\leq r\_\{\\mathit\{tight\}\}\+\\delta\(t\)withδ⁡\(t\)→0\\delta\(t\)\\to 0exponentially\.

The bound concerns the growing component, not the generically full algebraic rank of𝐖⁡\(t\)\\mathbf\{W\}\(t\); in the linear model it passes to the effective rank of the embeddings𝐙=𝐖𝐗\\mathbf\{Z\}=\\mathbf\{W\}\\mathbf\{X\}, the quantity every experiment tracks \(Appendix[C\.9](https://arxiv.org/html/2610.02344#A3.SS9)\)\.

A JEPA trains the predictor with the encoder, at one learning rate in this paper’s recipes; near collapse the fixed\-predictor analysis also describes that system\.

###### Lemma 3\.4\(Exact local criterion under joint training\)\.

The predictor gradient is quadratic in the encoder state and vanishes at𝐖=𝐖a=𝟎\\mathbf\{W\}=\\mathbf\{W\}\_\{a\}=\\mathbf\{0\}, so the collapsed set is a manifold of equilibria of the full\(𝐖,𝐖a,𝐖p\)\(\\mathbf\{W\},\\mathbf\{W\}\_\{a\},\\mathbf\{W\}\_\{p\}\)gradient flow, whose Jacobian there is Eq\. \([3](https://arxiv.org/html/2610.02344#S3.E3)\) on top of a zero block at any ratio of learning rates\. Under the conditions of Appendix[F\.6](https://arxiv.org/html/2610.02344#A6.SS6), the predictor moves by at mostC​δ2C\\delta^\{2\}while the encoder state grows to normδ\\delta, and if every mode with real roots has margin\|μk​ℓ−1\|\>C′​δ2\|\\mu\_\{k\\ell\}\-1\|\>C^\{\\prime\}\\delta^\{2\}the Jacobian keeps exactly\|𝒰\|\|\\mathcal\{U\}\|unstable eigenvalues throughout, each withinO⁡\(δ2\)O\(\\delta^\{2\}\)of an escape rate computed at initialisation\.

Quasi\-staticity of the predictor near the collapsed set is therefore a consequence of the initialisation scale rather than of a learning\-rate choice \(Appendix[F\.6](https://arxiv.org/html/2610.02344#A6.SS6)\)\. How the predictor adapts after escape is measured in Section[6\.2](https://arxiv.org/html/2610.02344#S6.SS2)\.

## 4Methodology

In order to validate our theoretical results, we start by evaluating the linear Tabular\-JEPA on real world tabular benchmarks to verify our assumptions and explain the expected result\. Then, we proceed by evaluating the standard JEPA models on both tabular and image datasets, verifying the generalization of our approach\. Code for all experiments is available at[https://github\.com/jose\-melo/drive\-vs\-decay](https://github.com/jose-melo/drive-vs-decay)\.

##### Datasets and Training Protocol\.

Eight datasets were used: three tabular datasets \(aloi,helena,jannis\) and five image datasets \(CIFAR\-10,CIFAR\-100,STL\-10,ImageNet\-100, andImageNet\-1k\)\. The tabular datasets were stratified into80%80\\%train,10%10\\%validation, and10%10\\%test splits; continuous features are scaled with min–max normalization, categorical columns are label\-encoded\. Image datasets followed the training protocol of[Assran et al\. \(2023\)](https://arxiv.org/html/2610.02344#bib.bib1), except that the ImageNet runs use a matched\-budget protocol, at128128pixels unless stated \(Appendix[I\.5](https://arxiv.org/html/2610.02344#A9.SS5)\)\. We refer the reader to Appendix[G](https://arxiv.org/html/2610.02344#A7)for details\. For the original T\-JEPA and I\-JEPA models, the training protocol followed the recommendations for each model\. Considering the Tabular\-JEPA variants, the optimization uses AdamW with learning rate10−310^\{\-3\}, weight decay10−510^\{\-5\}, cosine annealing to10−510^\{\-5\}over4040epochs, and gradient clipping at norm1\.01\.0\. Hyperparameters were found using Bayesian optimization search\.

##### Models\.

For the evaluation, we considered several variants of Joint\-Embedding Predictive Architectures \(JEPAs\)\. We includeT\-JEPA[Thimonier et al\. \(2025\)](https://arxiv.org/html/2610.02344#bib.bib29), which adapts the JEPA framework to tabular data\. In addition, we evaluate a family ofTabular\-JEPA variantsinspired by T\-JEPA but using linear encoders and a scaled predictor\. These variants are specifically introduced to isolate and demonstrate the different stabilization and collapse mechanisms studied under the proposed drive\-decay framework with a simpler architecture\. For image experiments, we useI\-JEPA[Assran et al\. \(2023\)](https://arxiv.org/html/2610.02344#bib.bib1)with a ViT\-S/4 encoder on CIFAR\-10/100, ViT\-S/8 on STL\-10 and ViT\-S/16 on ImageNet\. Downstream accuracy is evaluated using a linear probe; the ImageNet runs add a parameter\-free nearest\-neighbour classifier and an attentive probe, with exact permutation tests over seeds \(Appendix[I\.5](https://arxiv.org/html/2610.02344#A9.SS5)\)\.

##### Evaluation Metrics

We evaluate the spectral diversity of learned embeddings with the effective rank to directly quantify whether representations concentrate in a low\-dimensional subspace\. Let𝐙∈ℝn×d\\mathbf\{Z\}\\in\\mathbb\{R\}^\{n\\times d\}be a matrix of embeddings with singular values\{σi\}i=1r\\\{\\sigma\_\{i\}\\\}\_\{i=1\}^\{r\}\. Defining the normalized spectrumpi=σi∑j=1rσjp\_\{i\}\\;=\\;\\frac\{\\sigma\_\{i\}\}\{\\sum\_\{j=1\}^\{r\}\\sigma\_\{j\}\}, we measure intrinsic dimensionality via the*effective rank*\(we refer the reader to the Appendix[H](https://arxiv.org/html/2610.02344#A8)for more details\):ERank\(𝐙\)=exp\(−∑i=1rpilogpi\)\\mathrm\{ERank\}\(\\mathbf\{Z\}\)\\;=\\;\\exp\\\!\\left\(\-\\sum\_\{i=1\}^\{r\}p\_\{i\}\\log p\_\{i\}\\right\)\.

## 5Controlling the Training Dynamics: Example of Mechanisms

The theoretical condition for preventing collapse depends on the spectral interplay between the predictor geometry and the data statistics\. Based on the definition ofμi=γi/σi\\mu\_\{i\}=\\gamma\_\{i\}/\\sigma\_\{i\}, we identify three primary mechanisms that allow practical control over the training regime\.

##### Mechanism 1: Predictor scaling \(Inverse proportionality\)

Figure 3:Inverseμ⁡\(α\)\\mu\(\\alpha\)scaling\.Empiricalμ\\muvs\. predictor scaleα\\alphaforpm1∈\{0\.2,…,0\.7\}p\_\{m\_\{1\}\}\\\!\\in\\\!\\\{0\.2,\\dots,0\.7\\\}\.The most direct method to assureμ\>1\\mu\>1is via the initialization scale of the predictor weights𝐖p\\mathbf\{W\}\_\{p\}\. Consider the scaling of the decay and driving operators with respect to a scalar scaling factorα\\alphaapplied to𝐖p\\mathbf\{W\}\_\{p\}\(i\.e\.,𝐖p←α​𝐖p\\mathbf\{W\}\_\{p\}\\leftarrow\\alpha\\mathbf\{W\}\_\{p\}\), the parameterμ\\muscales inversely with the predictor norm:

μ⁡\(α\)=γ⁡\(α\)σ⁡\(α\)∝αα2=1α\\mu\(\\alpha\)=\\frac\{\\gamma\(\\alpha\)\}\{\\sigma\(\\alpha\)\}\\propto\\frac\{\\alpha\}\{\\alpha^\{2\}\}=\\frac\{1\}\{\\alpha\}\(9\)To guarantee the supercritical regime \(μ≫1\\mu\\gg 1\), one must initialize the predictor with a sufficiently small variance\. Large initialization values increase the quadratic penalty \(self\-regularization\) faster than the linear driving force, forcingμ≪1\\mu\\ll 1and causing collapse\.

Figure[3](https://arxiv.org/html/2610.02344#S5.F3)provides a direct empirical validation of the theoretical scaling\. For small and moderate context masking ratios \(pm1≤0\.4p\_\{m\_\{1\}\}\\leq 0\.4\), the measuredμ\\muclosely tracks the dashed theoretical curve across more than an order of magnitude inα\\alpha, confirming the dominant role of predictor\-induced contraction in setting the stability margin\. As the masking ratio increases \(pm1≥0\.5p\_\{m\_\{1\}\}\\geq 0\.5\), the empirical curves systematically shift downward and deviate from the ideal inverse law\. This effect is expected from the theory, as presented next\.

##### Mechanism 2: View Correlation \(Masking Strategy\)

The second critical control lever forμ\\muis the correlation structure between the context view𝐱1\\mathbf\{x\}\_\{1\}and the target input𝐱\\mathbf\{x\}, governed by the context ratiopm1∈\[0,1\)p\_\{m\_\{1\}\}\\in\[0,1\)\. Recall that the driving forceγi\\gamma\_\{i\}is derived from the spectrum of the cross\-covariance𝚺X​1\\mathbf\{\\Sigma\}\_\{X1\}, while the decay forceσi\\sigma\_\{i\}is driven by the context auto\-covariance𝚺11\\mathbf\{\\Sigma\}\_\{11\}\.

When𝐱1\\mathbf\{x\}\_\{1\}is formed by feature\-wise masking𝐱1=𝐦1⊙𝐱\\mathbf\{x\}\_\{1\}=\\mathbf\{m\}\_\{1\}\\odot\\mathbf\{x\}with keep probabilitypm1≔ℙ⁡\(m1​j=1\)p\_\{m\_\{1\}\}\\coloneqq\\mathbb\{P\}\(m\_\{1j\}=1\), both operators vary withpm1p\_\{m\_\{1\}\}\. In expectation, the cross\-covariance scales as a single factor while the context auto\-covariance does not,

𝚺X​1=𝔼⁡\[𝐱​\(𝐦1⊙𝐱\)⊤\]≈pm1​𝚺X​X,𝚺11=𝔼⁡\[\(𝐦1⊙𝐱\)​\(𝐦1⊙𝐱\)⊤\],\\mathbf\{\\Sigma\}\_\{X1\}=\\mathbb\{E\}\[\\mathbf\{x\}\(\\mathbf\{m\}\_\{1\}\\odot\\mathbf\{x\}\)^\{\\top\}\]\\approx p\_\{m\_\{1\}\}\\,\\mathbf\{\\Sigma\}\_\{XX\},\\qquad\\mathbf\{\\Sigma\}\_\{11\}=\\mathbb\{E\}\[\(\\mathbf\{m\}\_\{1\}\\odot\\mathbf\{x\}\)\(\\mathbf\{m\}\_\{1\}\\odot\\mathbf\{x\}\)^\{\\top\}\],\(10\)since diagonal entries scale aspm1p\_\{m\_\{1\}\}but off\-diagonal entries scale aspm12p\_\{m\_\{1\}\}^\{2\}\(sincem1​i​m1​jm\_\{1i\}m\_\{1j\}appears fori≠ji\\neq j\)\. Therefore, the spectral ratioμi=γi/σi\\mu\_\{i\}=\\gamma\_\{i\}/\\sigma\_\{i\}depends on how the data covariance is structured\.

##### Mechanism 3: EMA as a timescale

The EMA decay rateβ\\betais often viewed as a heuristic stabilizer\. Our dynamical analysis provides a precise spectral interpretation\. In the characteristic equation,β\\betamultiplies the constant term without changing its sign, so for a real drive spectrum it cannot decide whether a mode escapes; it acts as a*timescale separator*: forμi≪1\\mu\_\{i\}\\ll 1, smallerβ\\betaslows collapse, while forμi≫1\\mu\_\{i\}\\gg 1, it throttles feature growth\. Prior analyses, notably[Tian et al\. \(2021\)](https://arxiv.org/html/2610.02344#bib.bib31), often assume that the target weights𝐖a\\mathbf\{W\}\_\{a\}remain proportional to the encoder weights𝐖\\mathbf\{W\}\. That proportional\-target assumption is theβ→∞\\beta\\to\\inftylimit of the coupled analysis, in which𝐖a\\mathbf\{W\}\_\{a\}tracks𝐖\\mathbf\{W\}almost instantaneously; the resulting adiabatic stability condition, its critical couplingτcrit\\tau\_\{\\text\{crit\}\}, the bifurcation it produces and the region of better downstream performance it delimits are derived in Appendix[F](https://arxiv.org/html/2610.02344#A6)\.

![Refer to caption](https://arxiv.org/html/2610.02344v1/images/contours_image.png)Figure 4:Empirical phase diagram of Tabular JEPA stability predicted by the drive\-decay framework\.Each panel reports a two–dimensional sweep over the*predictor scale*log10⁡\(α\)\\log\_\{10\}\(\\alpha\)and the*context masking ratio*pm1p\_\{m\_\{1\}\}, for three tabular datasets\. Colored heatmaps show the downstream normalized*linear probe score*\. The thick red curve highlights the critical boundaryμ=1\\mu=1, which our theory identifies as the transition between a*learning*regime and a*collapsing*regime\.Table 1:Aggregated results for Tabular–JEPA variants \(aggregated over ALOI, Helena, and Jannis\) and T\-JEPA \(Jannis\)\. For Tabular–JEPA, only test accuracy is normalized within each dataset before aggregation\. Metrics are reported as mean values over 5 runs\.
##### Gathering Mechanisms: Empirical Phase Transitions\.

Figure[4](https://arxiv.org/html/2610.02344#S5.F4)illustrates a phase transition linking downstream performance to theμ\\mustability coefficient\. This relationship is further demonstrated by the results in Table[1](https://arxiv.org/html/2610.02344#S5.T1), which reports mean values over five random seeds, aggregated by context\-ratio and predictor\-scale subgroups\.

Moving along the predictor scale axis, we observe a sharp degradation in test accuracy as the scaleα\\alphaincreases\. For instance, in theTabular\-JEPAvariants with high context ratios, increasing the predictor scale results in a drop in normalized test accuracy from0\.8720\.872to0\.5860\.586\. This aligns with the theoretical scalingμ⁡\(α\)∝1/α\\mu\(\\alpha\)\\propto 1/\\alpha\. The role of the context masking ratiopm1p\_\{m\_\{1\}\}is also critical\. Low context ratios, corresponding to higher masking, reduce the correlation between the context and target views, weakening the cross\-covariance contribution to the growth factorγ\\gammaand subsequently loweringμ\\mu\. Table[1](https://arxiv.org/html/2610.02344#S5.T1)shows that forT\-JEPA, configurations with low context ratios and medium\-to\-high predictor scales exhibit medianμ\\muvalues near zero,μ50∈\[0\.05,0\.11\]\\mu\_\{50\}\\in\[0\.05,0\.11\], and collapsed rank,Erank≈4\.6\\mathrm\{Erank\}\\approx 4\.6, leading to a failure to learn meaningful representations, with Test Acc\.≈0\.464\\approx 0\.464\. We refer the reader to Appendix[I](https://arxiv.org/html/2610.02344#A9)for a detailed analysis\.

##### Empirical validation of the rank\-control corollary\.

![Refer to caption](https://arxiv.org/html/2610.02344v1/helena_main_scatter.png)

Figure 5:ERank\\Erankvsr~𝑡𝑖𝑔ℎ𝑡\\tilde\{r\}\_\{\\mathit\{tight\}\}onhelena\(one point per run\); colour islog10⁡maxk​ℓ​μ~k​ℓ\\log\_\{10\}\\max\_\{k\\ell\}\\tilde\{\\mu\}\_\{k\\ell\}\.Table 2:Correlation per dataset\.The weakerjannisvalue comes with a near\-degenerate top spectrum \(Appendix[B\.4](https://arxiv.org/html/2610.02344#A2.SS4)\)\.
The metric of interest isr~𝑡𝑖𝑔ℎ𝑡\\tilde\{r\}\_\{\\mathit\{tight\}\}\(Corollary[3\.3](https://arxiv.org/html/2610.02344#S3.Thmtheorem3)\), computed via the singular\-value formμ~k​ℓ\\tilde\{\\mu\}\_\{k\\ell\}\. For more details on the limits of the Assumption[3\.2](https://arxiv.org/html/2610.02344#S3.Thmtheorem2)and effects of singular\-value calculations, we refer the reader to Appendix[B](https://arxiv.org/html/2610.02344#A2)and[C\.8](https://arxiv.org/html/2610.02344#A3.SS8)\. The count of super\-critical modes is converted into a discrete rank budget\. Across810810runs, this budget tracks the representation ERank withρ=\+0\.91\\rho=\{\+\}0\.91onhelena\(Figure[5](https://arxiv.org/html/2610.02344#S5.F5)\)\.

The rank\-budget view refines the scalar criterionμ\>1\\mu\>1: rather than only predicting escape from collapse, it estimates how many rank\-one directions are available after escape\. Table[2](https://arxiv.org/html/2610.02344#S5.T2)shows that the prediction is not specific to a single dataset: the correlation betweenr~𝑡𝑖𝑔ℎ𝑡\\tilde\{r\}\_\{\\mathit\{tight\}\}and the observed effective rank exceeds0\.90\.9onhelenaandaloi, and remains positive, though weaker, onjannis\. The measured data\-side commutator onjannisis no larger than onhelena, which is predicted well; what distinguishesjannisis a near\-degenerate top spectrum, where mode identities mix under a small violation while the count survives, the signature that Appendix[B\.4](https://arxiv.org/html/2610.02344#A2.SS4)measures\. Together with theμ50\\mu\_\{50\}phase diagnostics and Figure[4](https://arxiv.org/html/2610.02344#S5.F4), these results support the role ofμ\\muboth as a stability threshold and as a quantitative proxy for the number of active rank\-one learning directions\.

##### Other mechanisms as doses onμ\\mu\.

The three mechanisms above are not special cases\. Appendix[J](https://arxiv.org/html/2610.02344#A10)takes seven anti\-collapse devices from the literature and writes each as a term of the linear model whose effect onμ\\muis derived; Table[3](https://arxiv.org/html/2610.02344#S5.T3)shows the five beyond predictor scaling and EMA: encoder weight decay raises the decay and the loss\-side regularisers SIGReg and VICReg lower its predictor factor, mimetic initialisation moves the predictor factors, and a frozen identity predictor fixes them, so that escape becomes a condition on the data spectrum\. For each mechanism the number of modes that escape is predicted from the initialisation, and we verified the prediction numerically by integrating the linear model with the mechanism added\.

Table 3:Five further anti\-collapse mechanisms as doses onμ\\mu, derived in Appendix[J](https://arxiv.org/html/2610.02344#A10)\.ppis the context keep probability and\{sk\}\\\{s\_\{k\}\\\}the spectrum of the unit\-diagonal data covariance\.

## 6ResidualPred: The Residual Transformer Predictor

Large predictor weights can make decay dominate drive \(μ≪1\\mu\\ll 1\)\. We avoid this by parametrizing the predictor as a residual correction to the identity, so early dynamics are driven by feature alignment rather than arbitrary initialization scale\.

### 6\.1Theoretical Motivation

Table 4:ResidualPred across tabular and image datasets\.Linear\-probe accuracy \(%\)\. More ImageNet results: Appendix[I\.5](https://arxiv.org/html/2610.02344#A9.SS5), Figure[6\(c\)](https://arxiv.org/html/2610.02344#S6.F6.sf3)\.We parametrize the predictor as a residual map,𝐖p=𝐈\+Δ​𝐖p,Δ​𝐖p≈𝟎​at init\.\\mathbf\{W\}\_\{p\}\\;=\\;\\mathbf\{I\}\+\\Delta\\mathbf\{W\}\_\{p\},\\Delta\\mathbf\{W\}\_\{p\}\\approx\\mathbf\{0\}\\ \\text\{at init\.\}Hence the predictor Gram matrix satisfies𝐖p⊤​𝐖p=𝐈\+𝒪⁡\(‖Δ​𝐖p‖\)\\mathbf\{W\}\_\{p\}^\{\\top\}\\mathbf\{W\}\_\{p\}=\\mathbf\{I\}\+\\mathcal\{O\}\(\\\|\\Delta\\mathbf\{W\}\_\{p\}\\\|\), preventing an arbitrary predictor\-induced contraction at initialization\. In our spectral characterization, with𝐖p≈𝐈\\mathbf\{W\}\_\{p\}\\approx\\mathbf\{I\}, the initialization regime becomes*view\-limited*:σi\\sigma\_\{i\}is set by the masked\-view energy𝚺11\\mathbf\{\\Sigma\}\_\{11\}andγi\\gamma\_\{i\}by the view alignment𝚺X​1\\mathbf\{\\Sigma\}\_\{X1\}, rather than by predictor scale\. To extend this identity\-preserving strategy to the general transformer architecture[Vaswani et al\. \(2017\)](https://arxiv.org/html/2610.02344#bib.bib34), we must ensure the predictor block acts as an identity map intoken space\. Standard self\-attention initialization typically yields uniform maps \(𝐀≈1N​𝟏𝟏⊤\\mathbf\{A\}\\approx\\frac\{1\}\{N\}\\mathbf\{1\}\\mathbf\{1\}^\{\\top\}\), which averages out spatial signals and collapsesμ\\mu\. We proposeResidualPred, which enforces𝐀≈𝐈\\mathbf\{A\}\\approx\\mathbf\{I\}via a Diagonal Attention Bias\. To prevent the internal attention mechanism from diffusing gradients uniformly, we add a diagonal logit bias:Attention​\(𝐐,𝐊,𝐕\)=Softmax​\(𝐐𝐊⊤d\+βi​n​i​t⋅𝐈t​o​k​e​n\)​𝐕\\text\{Attention\}\(\\mathbf\{Q\},\\mathbf\{K\},\\mathbf\{V\}\)=\\text\{Softmax\}\\left\(\\frac\{\\mathbf\{Q\}\\mathbf\{K\}^\{\\top\}\}\{\\sqrt\{d\}\}\+\\beta\_\{init\}\\cdot\\mathbf\{I\}\_\{token\}\\right\)\\mathbf\{V\}whereβi​n​i​t≫1\\beta\_\{init\}\\gg 1\(e\.g\.,10\.010\.0, see Appendix[D](https://arxiv.org/html/2610.02344#A4)\)\. This forces the softmax output to approximate the identity matrix, ensuring that tokeniiattends strictly to itself at the start of training\.

\(a\)Aggregated training diagnostics for T\-JEPA onjannisandhelena, averaged over context ratios and predictor scales\. Columns separate non\-brittle \(no\) and brittle \(yes\)\. ERank0is the ERank at epoch 0\.
![Refer to caption](https://arxiv.org/html/2610.02344v1/images/neurips_300ep_probe_residualpred_vs_standard.png)\(b\)I\-JEPA on CIFAR\-100, 300 epochs: linear\-probe accuracy\.
\(c\)ImageNet\-1k pilot at a matched budget: final linear\-probe top\-1 per seed\.
Figure 6:ResidualPred– aggregated brittle\-vs\-non\-brittle diagnostics, image\-domain training trajectory, and the ImageNet\-1k pilot\.Panel \(a\) carries the general tabular claim: the predictors start matched and Standard finishes below its own initialisation in both regimes\. Panel \(c\): ViT\-S/16, predictor depth66,3030epochs, five seeds each, mean±\\pms\.d\.; everyResidualPredseed lies above every standard seed at both resolutions\.
### 6\.2ResidualPredResults

ResidualPredimproves accuracy in every setting of Table[4](https://arxiv.org/html/2610.02344#S6.T4), most on Helena\. In brittle \(μ<1\\mu<1\) and non\-brittle configurations, Standard T\-JEPA collapses whileResidualPredincreases rank \(Figure[6](https://arxiv.org/html/2610.02344#S6.F6)\(a\)\), and onhelenakeeps it above Standard’s throughout training \(Figure[13](https://arxiv.org/html/2610.02344#A9.F13)\)\.

Table 5:Against SIGReg on the ImageNet\-1k pilot\.Mean over five seeds\.Final training loss is therefore a poor diagnostic: near\-zero MSE can accompany degenerate embeddings\. On CIFAR\-100,ResidualPredleads throughout 300 epochs \(Figure[6\(b\)](https://arxiv.org/html/2610.02344#S6.F6.sf2)\)\. Its gain holds on ImageNet\-100 at predictor depth22and on ViT\-B/16, and on a thousand\-class ImageNet\-1k pilot at a matched budget, at128128and224224pixels \(Figure[6\(c\)](https://arxiv.org/html/2610.02344#S6.F6.sf3)\); tripling the schedule to9090epochs attenuates the margin \(Appendix[I\.5](https://arxiv.org/html/2610.02344#A9.SS5)\)\.

The criterion is proven in the linear regime, while its extension to images is an empirical prediction whose mediating variable is measured after predictor equilibration, not at initialisation\. At initialisation, a whole\-block estimator does not separate the predictors; the target\-aligned drive after a few predictor\-only steps is higher forResidualPredin99of99matched comparisons, evidence for the first link of the mechanism only \(Appendix[K](https://arxiv.org/html/2610.02344#A11)\)\. Frozen at initialisation and denied learning, theResidualPredpredictor collapses the representation and loses the entire gain \(Appendix[I\.8](https://arxiv.org/html/2610.02344#A9.SS8)\)\. Effective rank on images is necessary for accuracy without being sufficient\.

ResidualPredbeats SIGReg\([Balestriero and LeCun, 2025](https://arxiv.org/html/2610.02344#bib.bib4)\)and the two compose \(Table[5](https://arxiv.org/html/2610.02344#S6.T5), Appendix[I\.9](https://arxiv.org/html/2610.02344#A9.SS9)\)\. Mimetic initialisation\([Trockman and Kolter, 2023](https://arxiv.org/html/2610.02344#bib.bib32)\)gains1\.61\.6points with seed overlap and trailsResidualPredby4\.94\.9\.ResidualPredhas one hyperparameter, unchanged everywhere, and no loss term or per\-step cost\.

## 7Discussion

These results support the drive–decay view of JEPA stability: representation learning is governed by a spectral competition between an alignment\-driven forceγ\\gammaand a predictor\-induced decay forceσ\\sigma\. This single mechanism explains why collapse can occur despite loss minimization and unifies several empirical anti\-collapse heuristics\. The phase diagrams show a sharp transition aligned with the theoretical boundary: whenμ≪1\\mu\\ll 1, representations collapse to low rank and downstream performance degrades\. Viewed through this lens, existing methods in the literature can be interpreted as indirect ways of manipulatingμ\\mu\. For example, explicit regularization terms that enforce embedding diversity\([Balestriero and LeCun, 2025](https://arxiv.org/html/2610.02344#bib.bib4)\)effectively modify the Jacobian spectrum, shifting eigenvalues away from the stable collapsed regime and variance–covariance regularisers\([Bardes et al\., 2022](https://arxiv.org/html/2610.02344#bib.bib6)\)act as an explicit modification of the decay operatorσ\\sigmain our framework\.ResidualPredprovides a structural alternative: by initializing the predictor near the identity, it lets view alignment dominate early training, allowing informative modes to grow before nonlinear effects take over\.

Table[3](https://arxiv.org/html/2610.02344#S5.T3)makes that reinterpretation precise for seven published mechanisms, each a derived dose onμ\\muwhose predicted escaping count the linear model reproduces\. DirectPred, IsoLoss, the rate equation of[Halvagal et al\. \(2023\)](https://arxiv.org/html/2610.02344#bib.bib18)and predictor orthonormalisation are recovered as special cases\.

##### Limitations\.

Our theory is derived in a linear setting and relies on the spectral\-decoupling approximation of Assumption[3\.2](https://arxiv.org/html/2610.02344#S3.Thmtheorem2), whose violation we measure \(Appendix[B\.4](https://arxiv.org/html/2610.02344#A2.SS4)\); onjannis, whose top spectrum is near\-degenerate, per\-mode identification degrades while the count\-level prediction survives\. The criterion is proven in the linear regime, while its extension to images is an empirical prediction whose mediating variable is measured after predictor equilibration, not at initialisation \(Section[6\.2](https://arxiv.org/html/2610.02344#S6.SS2), Appendix[K](https://arxiv.org/html/2610.02344#A11)\); the quantitative relation between the initialisation\-time budget and the effective rank is claimed on tabular data only\. The framework governs encoder\-side escape given the predictor’s state, whose equilibration the frozen\-predictor control tests\. On images, effective rank is necessary for accuracy without being sufficient \(Section[6\.2](https://arxiv.org/html/2610.02344#S6.SS2)\)\. At deeply subcritical tabular settings,ResidualPredwidens the escape basin without guaranteeing a large rank budget: escape there is partial and seed\-dependent \(Appendix[I\.3](https://arxiv.org/html/2610.02344#A9.SS3)\)\.ResidualPredextends the identity\-bias principle to attention by analogy rather than formal derivation\. Finally,μ=1\\mu=1is a local initialisation criterion, and the ImageNet\-1k results are matched\-budget pilots; the official full\-resolution, full\-schedule I\-JEPA recipe is untested\.

## 8Conclusion

This work establishes a dynamical systems perspective on JEPAs, identifying the spectral ratioμ=γ/σ\\mu=\\gamma/\\sigmaas the critical coefficient governing the trade\-off between representation growth and collapse\. Guided by this theory, we introducedResidualPred, an identity\-biased predictor that structurally enforces non\-trivial learning dynamics by tying initial stability to view correlation\. Beyond JEPAs, the drive\-decay framework suggests a broader viewpoint on non\-contrastive self\-supervised learning: collapse is governed by an explicit spectral competition between alignment\-driven growth and predictor\-induced decay\. Framing stability in these terms clarifies the role of EMA, stop\-gradient, and predictor design as mechanisms for inducing timescale separation and amplifying informative modes\. This perspective opens the door to theory\-driven architecture design, replacing ad hoc collapse\-avoidance heuristics with analyzable structural controls\.

## Acknowledgements and Disclosures

This work was performed using HPC resources from the “Mésocentre” computing center of CentraleSupélec and École Normale Supérieure Paris\-Saclay supported by CNRS and Région Île\-de\-France \([http://mesocentre\.centralesupelec\.fr/](http://mesocentre.centralesupelec.fr/)\)\. This project was provided with computing AI and storage resources by GENCI at IDRIS thanks to the grant 2026\-AD011011828R6 on the supercomputer Jean Zay’s H100 partition\. This research benefited from the support of the Chair “Artificial Intelligence applied to the banking domain and to global health” led by CentraleSupélec and sponsored by Lusis, and from a CIFRE fellowship\. Competing interests: the first author is employed by Lusis under a CIFRE convention; the authors declare no other financial relationships that could be perceived as influencing this work\.

## References

- Assran et al\. \(2023\)Mahmoud Assran, Quentin Duval, Ishan Misra, Piotr Bojanowski, Pascal Vincent, Michael Rabbat, Yann LeCun, and Nicolas Ballas\.Self\-supervised learning from images with a joint\-embedding predictive architecture\.In*Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\)*, pages 15619–15629, 2023\.doi:10\.1109/CVPR52729\.2023\.01499\.
- Bachlechner et al\. \(2021\)Thomas Bachlechner, Bodhisattwa Prasad Majumder, Henry Mao, Gary Cottrell, and Julian McAuley\.ReZero is all you need: Fast convergence at large depth\.In*Proceedings of the Thirty\-Seventh Conference on Uncertainty in Artificial Intelligence*, volume 161 of*Proceedings of Machine Learning Research*, pages 1352–1361\. PMLR, 2021\.URL[https://proceedings\.mlr\.press/v161/bachlechner21a\.html](https://proceedings.mlr.press/v161/bachlechner21a.html)\.
- Bagatella et al\. \(2026\)Marco Bagatella, Matteo Pirotta, Ahmed Touati, Alessandro Lazaric, and Andrea Tirinzoni\.TD\-JEPA: Latent\-predictive representations for zero\-shot reinforcement learning\.In*International Conference on Learning Representations \(ICLR\)*, 2026\.URL[https://openreview\.net/forum?id=SzXDuBN8M1](https://openreview.net/forum?id=SzXDuBN8M1)\.
- Balestriero and LeCun \(2025\)Randall Balestriero and Yann LeCun\.LeJEPA: Provable and Scalable Self\-Supervised Learning Without the Heuristics, November 2025\.URL[http://arxiv\.org/abs/2511\.08544](http://arxiv.org/abs/2511.08544)\.arXiv:2511\.08544 \[cs\]\.
- Balestriero et al\. \(2025\)Randall Balestriero, Nicolas Ballas, Mike Rabbat, and Yann LeCun\.Gaussian Embeddings: How JEPAs Secretly Learn Your Data Density, October 2025\.URL[http://arxiv\.org/abs/2510\.05949](http://arxiv.org/abs/2510.05949)\.
- Bardes et al\. \(2022\)Adrien Bardes, Jean Ponce, and Yann LeCun\.VICReg: Variance\-invariance\-covariance regularization for self\-supervised learning\.In*International Conference on Learning Representations*, 2022\.URL[https://openreview\.net/forum?id=xm6YD62D1Ub](https://openreview.net/forum?id=xm6YD62D1Ub)\.
- Bardhan et al\. \(2025\)Jai Bardhan, Radhikesh Agrawal, Abhiram Tilak, Cyrin Neeraj, and Subhadip Mitra\.HEP\-JEPA: A foundation model for collider physics\.In*ICLR 2025 Workshop on Agentic AI for Scientific Discovery*, 2025\.URL[https://openreview\.net/forum?id=ZKl1a1Dmvk](https://openreview.net/forum?id=ZKl1a1Dmvk)\.
- Bauer and Fike \(1960\)F\. L\. Bauer and C\. T\. Fike\.Norms and exclusion theorems\.*Numerische Mathematik*, 2:137–141, 1960\.doi:10\.1007/BF01386217\.
- Cover and Thomas \(2006\)Thomas M\. Cover and Joy A\. Thomas\.*Elements of Information Theory*\.Wiley, 2 edition, 2006\.
- De and Smith \(2020\)Soham De and Samuel L\. Smith\.Batch normalization biases residual blocks towards the identity function in deep networks\.In*Advances in Neural Information Processing Systems*, volume 33, pages 19964–19975, 2020\.URL[https://proceedings\.neurips\.cc/paper/2020/hash/e6b738eca0e6792ba8a9cbcba6c1881d\-Abstract\.html](https://proceedings.neurips.cc/paper/2020/hash/e6b738eca0e6792ba8a9cbcba6c1881d-Abstract.html)\.
- Deng et al\. \(2009\)Jia Deng, Wei Dong, Richard Socher, Li\-Jia Li, Kai Li, and Li Fei\-Fei\.ImageNet: A large\-scale hierarchical image database\.In*2009 IEEE Conference on Computer Vision and Pattern Recognition*, pages 248–255, 2009\.doi:10\.1109/CVPR\.2009\.5206848\.
- Dong et al\. \(2024\)Zijian Dong, Ruilin Li, Yilei Wu, Thuan Tinh Nguyen, Joanna Su Xian Chong, Fang Ji, Nathanael Ren Jie Tong, Christopher Li Hsian Chen, and Juan Helen Zhou\.Brain\-JEPA: Brain dynamics foundation model with gradient positioning and spatiotemporal masking\.In*Advances in Neural Information Processing Systems*, volume 37, pages 86048–86073, 2024\.doi:10\.52202/079017\-2732\.
- Drozdov et al\. \(2024\)Katrina Drozdov, Ravid Shwartz\-Ziv, and Yann LeCun\.Video Representation Learning with Joint\-Embedding Predictive Architectures, December 2024\.URL[http://arxiv\.org/abs/2412\.10925](http://arxiv.org/abs/2412.10925)\.arXiv:2412\.10925 \[cs\]\.
- Fei et al\. \(2024\)Zhengcong Fei, Mingyuan Fan, and Junshi Huang\.A\-JEPA: Joint\-Embedding Predictive Architecture Can Listen, January 2024\.URL[http://arxiv\.org/abs/2311\.15830](http://arxiv.org/abs/2311.15830)\.arXiv:2311\.15830 \[cs\]\.
- Garrido et al\. \(2023\)Quentin Garrido, Randall Balestriero, Laurent Najman, and Yann LeCun\.RankMe: Assessing the downstream performance of pretrained self\-supervised representations by their rank\.In*Proceedings of the 40th International Conference on Machine Learning*, volume 202 of*Proceedings of Machine Learning Research*, pages 10929–10974\. PMLR, 2023\.
- Ghosh et al\. \(2024\)Arna Ghosh, Kumar Krishna Agrawal, Shagun Sodhani, Adam M\. Oberman, and Blake A\. Richards\.Harnessing small projectors and multiple views for efficient vision pretraining\.In*Advances in Neural Information Processing Systems*, volume 37, pages 39837–39868\. Curran Associates, Inc\., 2024\.doi:10\.52202/079017\-1258\.
- Grill et al\. \(2020\)Jean\-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre H\. Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Daniel Guo, Mohammad Gheshlaghi Azar, Bilal Piot, Koray Kavukcuoglu, Rémi Munos, and Michal Valko\.Bootstrap your own latent: A new approach to self\-supervised learning\.In*Advances in Neural Information Processing Systems*, volume 33, pages 21271–21284\. Curran Associates, Inc\., 2020\.URL[https://proceedings\.neurips\.cc/paper\_files/paper/2020/file/f3ada80d5c4ee70142b17b8192b2958e\-Paper\.pdf](https://proceedings.neurips.cc/paper_files/paper/2020/file/f3ada80d5c4ee70142b17b8192b2958e-Paper.pdf)\.
- Halvagal et al\. \(2023\)Manu Srinath Halvagal, Axel Laborieux, and Friedemann Zenke\.Implicit variance regularization in non\-contrastive SSL\.In*Advances in Neural Information Processing Systems*, volume 36, 2023\.doi:10\.52202/075280\-2768\.
- Horn and Johnson \(1991\)Roger A\. Horn and Charles R\. Johnson\.*Topics in Matrix Analysis*\.Cambridge University Press, 1991\.doi:10\.1017/CBO9780511840371\.
- Horn and Johnson \(2013\)Roger A\. Horn and Charles R\. Johnson\.*Matrix Analysis*\.Cambridge University Press, 2 edition, 2013\.
- Huang \(2026\)Yongchao Huang\.VJEPA: Variational joint embedding predictive architectures as probabilistic world models\.In*Proceedings of the 43rd International Conference on Machine Learning \(ICML\)*, Proceedings of Machine Learning Research\. PMLR, 2026\.URL[https://icml\.cc/virtual/2026/poster/61579](https://icml.cc/virtual/2026/poster/61579)\.
- Li et al\. \(2026\)Xianhang Li, Chen Huang, Chun\-Liang Li, Eran Malach, Joshua Susskind, Vimal Thilak, and Etai Littwin\.Rethinking JEPA: Compute\-efficient video self\-supervised learning with frozen teachers\.In*International Conference on Learning Representations*, 2026\.URL[https://openreview\.net/forum?id=3cB9243E9i](https://openreview.net/forum?id=3cB9243E9i)\.
- Littwin et al\. \(2024\)Etai Littwin, Omid Saremi, Madhu Advani, Vimal Thilak, Preetum Nakkiran, Chen Huang, and Joshua Susskind\.How JEPA avoids noisy features: The implicit bias of deep linear self distillation networks\.In*Advances in Neural Information Processing Systems*, volume 37, 2024\.doi:10\.52202/079017\-2897\.
- Mo and Tong \(2024\)Shentong Mo and Shengbang Tong\.Connecting joint\-embedding predictive architecture with contrastive self\-supervised learning\.In*Advances in Neural Information Processing Systems*, volume 37, pages 2348–2377\. Curran Associates, Inc\., 2024\.doi:10\.52202/079017\-0077\.
- Pan et al\. \(2025\)Yu Pan, Chaozheng Wang, Zekai Wu, Qifan Wang, Min Zhang, and Zenglin Xu\.IDInit: A universal and stable initialization method for neural network training\.In*International Conference on Learning Representations*, 2025\.URL[https://openreview\.net/forum?id=LFiaoYnP6T](https://openreview.net/forum?id=LFiaoYnP6T)\.
- Richemond et al\. \(2023\)Pierre H\. Richemond, Allison Tam, Yunhao Tang, Florian Strub, Bilal Piot, and Felix Hill\.The edge of orthogonality: A simple view of what makes BYOL tick\.In*Proceedings of the 40th International Conference on Machine Learning*, volume 202 of*Proceedings of Machine Learning Research*, pages 29063–29081\. PMLR, 2023\.URL[https://proceedings\.mlr\.press/v202/richemond23a\.html](https://proceedings.mlr.press/v202/richemond23a.html)\.
- Shwartz\-Ziv and LeCun \(2024\)Ravid Shwartz\-Ziv and Yann LeCun\.To compress or not to compress—self\-supervised learning and information theory: A review\.*Entropy*, 26\(3\):252, 2024\.doi:10\.3390/e26030252\.
- Skenderi et al\. \(2025\)Geri Skenderi, Hang Li, Jiliang Tang, and Marco Cristani\.Graph\-level representation learning with joint\-embedding predictive architectures\.*Transactions on Machine Learning Research*, 2025\.URL[https://openreview\.net/forum?id=v47f4DwYZb](https://openreview.net/forum?id=v47f4DwYZb)\.
- Thimonier et al\. \(2025\)Hugo Thimonier, José Lucas De Melo Costa, Fabrice Popineau, Arpad Rimmel, and Bich\-Liên Doan\.T\-JEPA: Augmentation\-free self\-supervised learning for tabular data\.In*International Conference on Learning Representations \(ICLR\)*, pages 58743–58768, 2025\.URL[https://proceedings\.iclr\.cc/paper\_files/paper/2025/file/93f01c8d9b355d7bbe3f353b44ccde66\-Paper\-Conference\.pdf](https://proceedings.iclr.cc/paper_files/paper/2025/file/93f01c8d9b355d7bbe3f353b44ccde66-Paper-Conference.pdf)\.
- Tian et al\. \(2020\)Yonglong Tian, Dilip Krishnan, and Phillip Isola\.Contrastive multiview coding\.In*Computer Vision – ECCV 2020*, pages 776–794\. Springer, 2020\.doi:10\.1007/978\-3\-030\-58621\-8\_45\.
- Tian et al\. \(2021\)Yuandong Tian, Xinlei Chen, and Surya Ganguli\.Understanding self\-supervised learning dynamics without contrastive pairs\.In*Proceedings of the 38th International Conference on Machine Learning*, volume 139 of*Proceedings of Machine Learning Research*, pages 10268–10278\. PMLR, 2021\.URL[https://proceedings\.mlr\.press/v139/tian21a\.html](https://proceedings.mlr.press/v139/tian21a.html)\.
- Trockman and Kolter \(2023\)Asher Trockman and J\. Zico Kolter\.Mimetic initialization of self\-attention layers\.In*Proceedings of the 40th International Conference on Machine Learning*, volume 202 of*Proceedings of Machine Learning Research*, pages 34456–34468\. PMLR, 2023\.URL[https://proceedings\.mlr\.press/v202/trockman23a\.html](https://proceedings.mlr.press/v202/trockman23a.html)\.
- Turrisi da Costa et al\. \(2022\)Victor Guilherme Turrisi da Costa, Enrico Fini, Moin Nabi, Nicu Sebe, and Elisa Ricci\.solo\-learn: A library of self\-supervised methods for visual representation learning\.*Journal of Machine Learning Research*, 23\(56\):1–6, 2022\.URL[http://jmlr\.org/papers/v23/21\-1155\.html](http://jmlr.org/papers/v23/21-1155.html)\.
- Vaswani et al\. \(2017\)Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N\. Gomez, Łukasz Kaiser, and Illia Polosukhin\.Attention is all you need\.In*Advances in Neural Information Processing Systems*, volume 30\. Curran Associates, Inc\., 2017\.URL[https://proceedings\.neurips\.cc/paper\_files/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa\-Paper\.pdf](https://proceedings.neurips.cc/paper_files/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf)\.
- Wen and Li \(2022\)Zixin Wen and Yuanzhi Li\.The mechanism of prediction head in non\-contrastive self\-supervised learning\.In*Advances in Neural Information Processing Systems*, volume 35, 2022\.URL[https://proceedings\.neurips\.cc/paper\_files/paper/2022/hash/9d276b0a087efdd2404f3295b26c24c1\-Abstract\-Conference\.html](https://proceedings.neurips.cc/paper_files/paper/2022/hash/9d276b0a087efdd2404f3295b26c24c1-Abstract-Conference.html)\.
- Yu et al\. \(2025\)Jiacan Yu, Siyi Chen, Mingrui Liu, Nono Horiuchi, Vladimir Braverman, Zicheng Xu, Dan Haramati, and Randall Balestriero\.Why and How Auxiliary Tasks Improve JEPA Representations, September 2025\.URL[http://arxiv\.org/abs/2509\.12249](http://arxiv.org/abs/2509.12249)\.arXiv:2509\.12249 \[cs\]\.
- Zhang et al\. \(2022\)Chaoning Zhang, Kang Zhang, Chenshuang Zhang, Trung X\. Pham, Chang D\. Yoo, and In So Kweon\.How does SimSiam avoid collapse without negative samples? A unified understanding with self\-supervised contrastive learning\.In*International Conference on Learning Representations*, 2022\.URL[https://openreview\.net/forum?id=bwq6O4Cwdl](https://openreview.net/forum?id=bwq6O4Cwdl)\.
- Zhang et al\. \(2019\)Hongyi Zhang, Yann N\. Dauphin, and Tengyu Ma\.Fixup initialization: Residual learning without normalization\.In*International Conference on Learning Representations*, 2019\.URL[https://openreview\.net/forum?id=H1gsz30cKX](https://openreview.net/forum?id=H1gsz30cKX)\.

## Appendix ADetailed Derivation of Dynamics and Stability

This appendix provides a comprehensive derivation of the coupled dynamical system, the vectorization of the matrix differential equations, and the spectral analysis that leads to the stability conditions presented in the main text\.

### A\.1Derivation of the Coupled Gradient Dynamics

We begin with the JEPA loss function defined in Eq\. \([1](https://arxiv.org/html/2610.02344#S3.E1)\)\. The objective is to minimize the expected squared error between the predictor output and the target projection:

ℒ=12​𝔼​\[‖𝐖p​𝐖𝐱1−sg​\(𝐌2​𝐖a​𝐱\)‖22\]\\mathcal\{L\}=\\frac\{1\}\{2\}\\mathbb\{E\}\\left\[\\\|\\mathbf\{W\}\_\{p\}\\mathbf\{W\}\\mathbf\{x\}\_\{1\}\-\\texttt\{sg\}\(\\mathbf\{M\}\_\{2\}\\mathbf\{W\}\_\{a\}\\mathbf\{x\}\)\\\|\_\{2\}^\{2\}\\right\]\(11\)
To determine the training dynamics of the context encoder𝐖\\mathbf\{W\}, we compute the gradient∇𝐖ℒ\\nabla\_\{\\mathbf\{W\}\}\\mathcal\{L\}\. Let𝐞=𝐖p​𝐖𝐱1−𝐌2​𝐖a​𝐱\\mathbf\{e\}=\\mathbf\{W\}\_\{p\}\\mathbf\{W\}\\mathbf\{x\}\_\{1\}\-\\mathbf\{M\}\_\{2\}\\mathbf\{W\}\_\{a\}\\mathbf\{x\}be the error vector\. The gradient of the squared norm12​‖𝐞‖22\\frac\{1\}\{2\}\\\|\\mathbf\{e\}\\\|\_\{2\}^\{2\}with respect to𝐖\\mathbf\{W\}is given by chain rule:

∇𝐖\(12​‖𝐞‖22\)=𝐖p⊤​𝐞𝐱1⊤\\nabla\_\{\\mathbf\{W\}\}\\left\(\\frac\{1\}\{2\}\\\|\\mathbf\{e\}\\\|\_\{2\}^\{2\}\\right\)=\\mathbf\{W\}\_\{p\}^\{\\top\}\\mathbf\{e\}\\mathbf\{x\}\_\{1\}^\{\\top\}\(12\)Note that the term𝐌2​𝐖a​𝐱\\mathbf\{M\}\_\{2\}\\mathbf\{W\}\_\{a\}\\mathbf\{x\}is treated as a constant due to the stop\-gradient \(sg\) operator\. Taking the expectation over the data distribution:

∇𝐖ℒ\\displaystyle\\nabla\_\{\\mathbf\{W\}\}\\mathcal\{L\}=𝔼⁡\[𝐖p⊤​\(𝐖p​𝐖𝐱1−𝐌2​𝐖a​𝐱\)​𝐱1⊤\]\\displaystyle=\\mathbb\{E\}\\left\[\\mathbf\{W\}\_\{p\}^\{\\top\}\(\\mathbf\{W\}\_\{p\}\\mathbf\{W\}\\mathbf\{x\}\_\{1\}\-\\mathbf\{M\}\_\{2\}\\mathbf\{W\}\_\{a\}\\mathbf\{x\}\)\\mathbf\{x\}\_\{1\}^\{\\top\}\\right\]\(13\)=𝐖p⊤​𝐖p​𝐖​𝔼​\[𝐱1​𝐱1⊤\]−𝐖p⊤​𝐌¯2​𝐖a​𝔼​\[𝐱𝐱1⊤\]\\displaystyle=\\mathbf\{W\}\_\{p\}^\{\\top\}\\mathbf\{W\}\_\{p\}\\mathbf\{W\}\\mathbb\{E\}\[\\mathbf\{x\}\_\{1\}\\mathbf\{x\}\_\{1\}^\{\\top\}\]\-\\mathbf\{W\}\_\{p\}^\{\\top\}\\bar\{\\mathbf\{M\}\}\_\{2\}\\mathbf\{W\}\_\{a\}\\mathbb\{E\}\[\\mathbf\{x\}\\mathbf\{x\}\_\{1\}^\{\\top\}\]\(14\)Substituting the covariance matrices𝚺11=𝔼⁡\[𝐱1​𝐱1⊤\]\\mathbf\{\\Sigma\}\_\{11\}=\\mathbb\{E\}\[\\mathbf\{x\}\_\{1\}\\mathbf\{x\}\_\{1\}^\{\\top\}\]and𝚺X​1=𝔼⁡\[𝐱𝐱1⊤\]\\mathbf\{\\Sigma\}\_\{X1\}=\\mathbb\{E\}\[\\mathbf\{x\}\\mathbf\{x\}\_\{1\}^\{\\top\}\]yields the gradient flow𝐖˙∝−∇𝐖ℒ\\dot\{\\mathbf\{W\}\}\\propto\-\\nabla\_\{\\mathbf\{W\}\}\\mathcal\{L\}:

𝐖˙=−𝐖p⊤​𝐖p​𝐖​𝚺11\+𝐖p⊤​𝐌¯2​𝐖a​𝚺X​1\\dot\{\\mathbf\{W\}\}=\-\\mathbf\{W\}\_\{p\}^\{\\top\}\\mathbf\{W\}\_\{p\}\\mathbf\{W\}\\mathbf\{\\Sigma\}\_\{11\}\+\\mathbf\{W\}\_\{p\}^\{\\top\}\\bar\{\\mathbf\{M\}\}\_\{2\}\\mathbf\{W\}\_\{a\}\\mathbf\{\\Sigma\}\_\{X1\}\(15\)
Simultaneously, the target encoder𝐖a\\mathbf\{W\}\_\{a\}is updated via an Exponential Moving Average \(EMA\) of the context encoder𝐖\\mathbf\{W\}\. In discrete time with momentummm, this is𝐖a\(t\+1\)=m​𝐖a\(t\)\+\(1−m\)​𝐖\(t\)\\mathbf\{W\}\_\{a\}^\{\(t\+1\)\}=m\\mathbf\{W\}\_\{a\}^\{\(t\)\}\+\(1\-m\)\\mathbf\{W\}^\{\(t\)\}\. In the continuous\-time limit, taking the update step size to zero, this becomes the differential equation:

𝐖˙a=β⁡\(𝐖−𝐖a\)\\dot\{\\mathbf\{W\}\}\_\{a\}=\\beta\(\\mathbf\{W\}\-\\mathbf\{W\}\_\{a\}\)\(16\)whereβ\\betais to the update speed\(1−m\)\(1\-m\)\.

### A\.2Vectorization and Jacobian Construction

To analyze the stability of the zero solution \(𝐖=𝟎,𝐖a=𝟎\\mathbf\{W\}=\\mathbf\{0\},\\mathbf\{W\}\_\{a\}=\\mathbf\{0\}\), we must represent the system as a standard linear ODE𝐳˙=𝒥​𝐳\\dot\{\\mathbf\{z\}\}=\\mathcal\{J\}\\mathbf\{z\}\. We define the state vector𝐳\\mathbf\{z\}by stacking the vectorized weight matrices:

𝐳=\[𝐯𝐞𝐜⁡\(𝐖\)𝐯𝐞𝐜⁡\(𝐖a\)\]∈ℝ2​d​N\\mathbf\{z\}=\\begin\{bmatrix\}\\mathbf\{vec\}\(\\mathbf\{W\}\)\\\\ \\mathbf\{vec\}\(\\mathbf\{W\}\_\{a\}\)\\end\{bmatrix\}\\in\\mathbb\{R\}^\{2dN\}\(17\)
We utilize the fundamental property of the Kronecker product: for any matrices𝐀,𝐗,𝐁\\mathbf\{A\},\\mathbf\{X\},\\mathbf\{B\}of compatible dimensions,

𝐯𝐞𝐜⁡\(𝐀𝐗𝐁\)=\(𝐁⊤⊗𝐀\)​𝐯𝐞𝐜​\(𝐗\)\\mathbf\{vec\}\(\\mathbf\{A\}\\mathbf\{X\}\\mathbf\{B\}\)=\(\\mathbf\{B\}^\{\\top\}\\otimes\\mathbf\{A\}\)\\mathbf\{vec\}\(\\mathbf\{X\}\)\(18\)
Step 1: Vectorizing𝐖˙\\dot\{\\mathbf\{W\}\}\(Eq\.[15](https://arxiv.org/html/2610.02344#A1.E15)\)

- •First term:𝐖p⊤​𝐖p​𝐖​𝚺11\\mathbf\{W\}\_\{p\}^\{\\top\}\\mathbf\{W\}\_\{p\}\\mathbf\{W\}\\mathbf\{\\Sigma\}\_\{11\}\. Here𝐀=𝐖p⊤​𝐖p\\mathbf\{A\}=\\mathbf\{W\}\_\{p\}^\{\\top\}\\mathbf\{W\}\_\{p\},𝐗=𝐖\\mathbf\{X\}=\\mathbf\{W\}, and𝐁=𝚺11\\mathbf\{B\}=\\mathbf\{\\Sigma\}\_\{11\}\. Using the identity:𝐯𝐞𝐜⁡\(𝐖p⊤​𝐖p​𝐖​𝚺11\)=\(𝚺11⊤⊗𝐖p⊤​𝐖p\)​𝐯𝐞𝐜​\(𝐖\)\\mathbf\{vec\}\(\\mathbf\{W\}\_\{p\}^\{\\top\}\\mathbf\{W\}\_\{p\}\\mathbf\{W\}\\mathbf\{\\Sigma\}\_\{11\}\)=\(\\mathbf\{\\Sigma\}\_\{11\}^\{\\top\}\\otimes\\mathbf\{W\}\_\{p\}^\{\\top\}\\mathbf\{W\}\_\{p\}\)\\mathbf\{vec\}\(\\mathbf\{W\}\)\. Since𝚺11\\mathbf\{\\Sigma\}\_\{11\}is symmetric,𝚺11⊤=𝚺11\\mathbf\{\\Sigma\}\_\{11\}^\{\\top\}=\\mathbf\{\\Sigma\}\_\{11\}\.
- •Second term:𝐖p⊤​𝐌¯2​𝐖a​𝚺X​1\\mathbf\{W\}\_\{p\}^\{\\top\}\\bar\{\\mathbf\{M\}\}\_\{2\}\\mathbf\{W\}\_\{a\}\\mathbf\{\\Sigma\}\_\{X1\}\. Here𝐀=𝐖p⊤​𝐌¯2\\mathbf\{A\}=\\mathbf\{W\}\_\{p\}^\{\\top\}\\bar\{\\mathbf\{M\}\}\_\{2\},𝐗=𝐖a\\mathbf\{X\}=\\mathbf\{W\}\_\{a\}, and𝐁=𝚺X​1\\mathbf\{B\}=\\mathbf\{\\Sigma\}\_\{X1\}\. Using the identity:𝐯𝐞𝐜⁡\(𝐖p⊤​𝐌¯2​𝐖a​𝚺X​1\)=\(𝚺X​1⊤⊗𝐖p⊤​𝐌¯2\)​𝐯𝐞𝐜​\(𝐖a\)\\mathbf\{vec\}\(\\mathbf\{W\}\_\{p\}^\{\\top\}\\bar\{\\mathbf\{M\}\}\_\{2\}\\mathbf\{W\}\_\{a\}\\mathbf\{\\Sigma\}\_\{X1\}\)=\(\\mathbf\{\\Sigma\}\_\{X1\}^\{\\top\}\\otimes\\mathbf\{W\}\_\{p\}^\{\\top\}\\bar\{\\mathbf\{M\}\}\_\{2\}\)\\mathbf\{vec\}\(\\mathbf\{W\}\_\{a\}\)\.

Thus, the vectorized equation for𝐖\\mathbf\{W\}is:

𝐯𝐞𝐜⁡\(𝐖˙\)=−\(𝚺11⊗𝐖p⊤​𝐖p\)​𝐯𝐞𝐜​\(𝐖\)\+\(𝚺X​1⊤⊗𝐖p⊤​𝐌¯2\)​𝐯𝐞𝐜​\(𝐖a\)\\mathbf\{vec\}\(\\dot\{\\mathbf\{W\}\}\)=\-\(\\mathbf\{\\Sigma\}\_\{11\}\\otimes\\mathbf\{W\}\_\{p\}^\{\\top\}\\mathbf\{W\}\_\{p\}\)\\mathbf\{vec\}\(\\mathbf\{W\}\)\+\(\\mathbf\{\\Sigma\}\_\{X1\}^\{\\top\}\\otimes\\mathbf\{W\}\_\{p\}^\{\\top\}\\bar\{\\mathbf\{M\}\}\_\{2\}\)\\mathbf\{vec\}\(\\mathbf\{W\}\_\{a\}\)\(19\)
Step 2: Vectorizing𝐖˙a\\dot\{\\mathbf\{W\}\}\_\{a\}\(Eq\.[16](https://arxiv.org/html/2610.02344#A1.E16)\)The EMA equation is linear and acts element\-wise\.

𝐯𝐞𝐜⁡\(𝐖˙a\)=β​𝐯𝐞𝐜​\(𝐖\)−β​𝐯𝐞𝐜​\(𝐖a\)\\mathbf\{vec\}\(\\dot\{\\mathbf\{W\}\}\_\{a\}\)=\\beta\\mathbf\{vec\}\(\\mathbf\{W\}\)\-\\beta\\mathbf\{vec\}\(\\mathbf\{W\}\_\{a\}\)\(20\)In Kronecker notation \(where𝐈d​N=𝐈N⊗𝐈d\\mathbf\{I\}\_\{dN\}=\\mathbf\{I\}\_\{N\}\\otimes\\mathbf\{I\}\_\{d\}\), this is:

𝐯𝐞𝐜⁡\(𝐖˙a\)=\(β​𝐈d​N\)​𝐯𝐞𝐜​\(𝐖\)−\(β​𝐈d​N\)​𝐯𝐞𝐜​\(𝐖a\)\\mathbf\{vec\}\(\\dot\{\\mathbf\{W\}\}\_\{a\}\)=\(\\beta\\mathbf\{I\}\_\{dN\}\)\\mathbf\{vec\}\(\\mathbf\{W\}\)\-\(\\beta\\mathbf\{I\}\_\{dN\}\)\\mathbf\{vec\}\(\\mathbf\{W\}\_\{a\}\)\(21\)
Step 3: Constructing the Block JacobianCombining these results into the matrix form𝐳˙=𝒥​𝐳\\dot\{\\mathbf\{z\}\}=\\mathcal\{J\}\\mathbf\{z\}:

\[𝐯𝐞𝐜⁡\(𝐖˙\)𝐯𝐞𝐜⁡\(𝐖˙a\)\]=\[−\(𝚺11⊗𝐖p⊤​𝐖p\)\(𝚺X​1⊤⊗𝐖p⊤​𝐌¯2\)β​𝐈d​N−β​𝐈d​N\]​\[𝐯𝐞𝐜⁡\(𝐖\)𝐯𝐞𝐜⁡\(𝐖a\)\]\\begin\{bmatrix\}\\mathbf\{vec\}\(\\dot\{\\mathbf\{W\}\}\)\\\\ \\mathbf\{vec\}\(\\dot\{\\mathbf\{W\}\}\_\{a\}\)\\end\{bmatrix\}=\\begin\{bmatrix\}\-\(\\mathbf\{\\Sigma\}\_\{11\}\\otimes\\mathbf\{W\}\_\{p\}^\{\\top\}\\mathbf\{W\}\_\{p\}\)&\(\\mathbf\{\\Sigma\}\_\{X1\}^\{\\top\}\\otimes\\mathbf\{W\}\_\{p\}^\{\\top\}\\bar\{\\mathbf\{M\}\}\_\{2\}\)\\\\ \\beta\\mathbf\{I\}\_\{dN\}&\-\\beta\\mathbf\{I\}\_\{dN\}\\end\{bmatrix\}\\begin\{bmatrix\}\\mathbf\{vec\}\(\\mathbf\{W\}\)\\\\ \\mathbf\{vec\}\(\\mathbf\{W\}\_\{a\}\)\\end\{bmatrix\}\(22\)This confirms Eq\. \([3](https://arxiv.org/html/2610.02344#S3.E3)\) in the main text\.

### A\.3Characteristic Equation and Eigenvalue Analysis

To determine stability, we solve for the eigenvaluesλ\\lambdaof𝒥\\mathcal\{J\}using the characteristic equationdet\(𝒥−λ​𝐈\)=0\\det\(\\mathcal\{J\}\-\\lambda\\mathbf\{I\}\)=0\.

det\(−\(𝚺11⊗𝐖p⊤​𝐖p\)−λ​𝐈\(𝚺X​1⊤⊗𝐖p⊤​𝐌¯2\)β​𝐈−β​𝐈−λ​𝐈\)=0\\det\\begin\{pmatrix\}\-\(\\mathbf\{\\Sigma\}\_\{11\}\\otimes\\mathbf\{W\}\_\{p\}^\{\\top\}\\mathbf\{W\}\_\{p\}\)\-\\lambda\\mathbf\{I\}&\(\\mathbf\{\\Sigma\}\_\{X1\}^\{\\top\}\\otimes\\mathbf\{W\}\_\{p\}^\{\\top\}\\bar\{\\mathbf\{M\}\}\_\{2\}\)\\\\ \\beta\\mathbf\{I\}&\-\\beta\\mathbf\{I\}\-\\lambda\\mathbf\{I\}\\end\{pmatrix\}=0\(23\)
We simplify this by assuming the system can be decomposed into eigenmodes\. Let us assume we are working in a basis where the spatial operators are diagonal\. Letσi\\sigma\_\{i\}be an eigenvalue of the ”decay” operator\(𝚺11⊗𝐖p⊤​𝐖p\)\(\\mathbf\{\\Sigma\}\_\{11\}\\otimes\\mathbf\{W\}\_\{p\}^\{\\top\}\\mathbf\{W\}\_\{p\}\)\. Letγi\\gamma\_\{i\}be the corresponding eigenvalue of the ”driving” operator\(𝚺X​1⊤⊗𝐖p⊤​𝐌¯2\)\(\\mathbf\{\\Sigma\}\_\{X1\}^\{\\top\}\\otimes\\mathbf\{W\}\_\{p\}^\{\\top\}\\bar\{\\mathbf\{M\}\}\_\{2\}\)\. For a specific modeii, the determinant of the2×22\\times 2block becomes:

det\(−σi−λγiβ−\(β\+λ\)\)=0\\det\\begin\{pmatrix\}\-\\sigma\_\{i\}\-\\lambda&\\gamma\_\{i\}\\\\ \\beta&\-\(\\beta\+\\lambda\)\\end\{pmatrix\}=0\(24\)Expanding the determinant\(a​d−b​c\)\(ad\-bc\):

\(−σi−λ\)​\(−\(β\+λ\)\)−\(β\)​\(γi\)\\displaystyle\(\-\\sigma\_\{i\}\-\\lambda\)\(\-\(\\beta\+\\lambda\)\)\-\(\\beta\)\(\\gamma\_\{i\}\)=0\\displaystyle=0\(25\)\(λ\+σi\)​\(λ\+β\)−β​γi\\displaystyle\(\\lambda\+\\sigma\_\{i\}\)\(\\lambda\+\\beta\)\-\\beta\\gamma\_\{i\}=0\\displaystyle=0\(26\)λ2\+\(β\+σi\)​λ\+β​σi−β​γi\\displaystyle\\lambda^\{2\}\+\(\\beta\+\\sigma\_\{i\}\)\\lambda\+\\beta\\sigma\_\{i\}\-\\beta\\gamma\_\{i\}=0\\displaystyle=0\(27\)
To relate this to the paper’s main theorem, we defineμi\\mu\_\{i\}as the ratio of driving force to decay force for that mode:μi=γi/σi\\mu\_\{i\}=\\gamma\_\{i\}/\\sigma\_\{i\}\. Substitutingγi=μi​σi\\gamma\_\{i\}=\\mu\_\{i\}\\sigma\_\{i\}:

λ2\+\(β\+σi\)​λ\+β​σi​\(1−μi\)=0\\lambda^\{2\}\+\(\\beta\+\\sigma\_\{i\}\)\\lambda\+\\beta\\sigma\_\{i\}\(1\-\\mu\_\{i\}\)=0\(28\)

### A\.4Reality and Positivity of the Driving Spectrum

Our stability analysis defines the driving and decay spectral quantities as

γi∈Spec⁡\(𝚺X​1⊤⊗𝐖p⊤​𝐌¯2\),σi∈Spec⁡\(𝚺11⊗𝐖p⊤​𝐖p\),μi=γi/σi\.\\gamma\_\{i\}\\in\\mathrm\{Spec\}\\\!\\left\(\\mathbf\{\\Sigma\}\_\{X1\}^\{\\top\}\\otimes\\mathbf\{W\}\_\{p\}^\{\\top\}\\bar\{\\mathbf\{M\}\}\_\{2\}\\right\),\\qquad\\sigma\_\{i\}\\in\\mathrm\{Spec\}\\\!\\left\(\\mathbf\{\\Sigma\}\_\{11\}\\otimes\\mathbf\{W\}\_\{p\}^\{\\top\}\\mathbf\{W\}\_\{p\}\\right\),\\qquad\\mu\_\{i\}=\\gamma\_\{i\}/\\sigma\_\{i\}\.\(29\)By construction,𝚺11\\mathbf\{\\Sigma\}\_\{11\}is positive semidefinite and𝐖p⊤​𝐖p\\mathbf\{W\}\_\{p\}^\{\\top\}\\mathbf\{W\}\_\{p\}is positive semidefinite, which impliesσi≥0\\sigma\_\{i\}\\geq 0andσi∈ℝ\\sigma\_\{i\}\\in\\mathbb\{R\}\.

In contrast, the driving operator𝚺X​1⊤⊗𝐖p⊤​𝐌¯2\\mathbf\{\\Sigma\}\_\{X1\}^\{\\top\}\\otimes\\mathbf\{W\}\_\{p\}^\{\\top\}\\bar\{\\mathbf\{M\}\}\_\{2\}is not symmetric in general\. The cross\-covariance𝚺X​1\\mathbf\{\\Sigma\}\_\{X1\}need not be symmetric \(it is under the feature\-wise masking model of Appendix[B](https://arxiv.org/html/2610.02344#A2), where𝚺X​1=pm1​𝚺X​X\\mathbf\{\\Sigma\}\_\{X1\}=p\_\{m\_\{1\}\}\\mathbf\{\\Sigma\}\_\{XX\}\), and the matrix𝐖p⊤​𝐌¯2\\mathbf\{W\}\_\{p\}^\{\\top\}\\bar\{\\mathbf\{M\}\}\_\{2\}is not symmetric unless additional structural constraints are imposed\. As a result,γi\\gamma\_\{i\}may be negative or complex in general\. Consequently, the scalar stability ratioμi=γi/σi\\mu\_\{i\}=\\gamma\_\{i\}/\\sigma\_\{i\}is not guaranteed to be real or ordered, and the threshold conditionμi\>1\\mu\_\{i\}\>1is not, in full generality, a well\-defined universal stability boundary\.

##### When the scalar criterion is exact\.

The per\-mode criterionμi\>1\\mu\_\{i\}\>1used in the main text is exact when the following two conditions hold together:

1. 1\.the driving operator is symmetric, that is,𝚺X​1\\mathbf\{\\Sigma\}\_\{X1\}and𝐖p⊤​𝐌¯2\\mathbf\{W\}\_\{p\}^\{\\top\}\\bar\{\\mathbf\{M\}\}\_\{2\}are both symmetric, so that the Kronecker product is symmetric and everyγi\\gamma\_\{i\}is real;
2. 2\.the exact form of Assumption[3\.2](https://arxiv.org/html/2610.02344#S3.Thmtheorem2)holds with orthonormal eigenbases, so that the coupled dynamics split into independent modes and eachγi\\gamma\_\{i\}is paired with theσi\\sigma\_\{i\}of the same mode\.

Neither condition suffices alone\. A symmetric driving operator whose eigenbasis differs from that of the decay operator has real eigenvalues, but the flow does not split into modes, and pairing thekk\-th largestγ\\gammawith thekk\-th largestσ\\sigmacan miscount the unstable directions\. Conversely, a shared eigenbasis does not make the drive real\. With𝐌¯2=pt​g​t​𝐈\\bar\{\\mathbf\{M\}\}\_\{2\}=p\_\{tgt\}\\mathbf\{I\}and a scaled rotation𝐖p=s​𝐑θ\\mathbf\{W\}\_\{p\}=s\\,\\mathbf\{R\}\_\{\\theta\}, the matrix𝐖p⊤​𝐖p=s2​𝐈\\mathbf\{W\}\_\{p\}^\{\\top\}\\mathbf\{W\}\_\{p\}=s^\{2\}\\mathbf\{I\}commutes with𝐖p⊤​𝐌¯2=pt​g​t​s​𝐑θ⊤\\mathbf\{W\}\_\{p\}^\{\\top\}\\bar\{\\mathbf\{M\}\}\_\{2\}=p\_\{tgt\}\\,s\\,\\mathbf\{R\}\_\{\\theta\}^\{\\top\}, so Assumption[3\.2](https://arxiv.org/html/2610.02344#S3.Thmtheorem2)holds exactly, yetqℓ=pt​g​t​s​e±i​θq\_\{\\ell\}=p\_\{tgt\}\\,s\\,e^\{\\pm\\mathrm\{i\}\\theta\}is complex\. Under the masking model of Appendix[B](https://arxiv.org/html/2610.02344#A2), where𝚺X​1\\mathbf\{\\Sigma\}\_\{X1\}is symmetric and𝐌¯2=pt​g​t​𝐈\\bar\{\\mathbf\{M\}\}\_\{2\}=p\_\{tgt\}\\mathbf\{I\}, the two conditions amount to a symmetric predictor𝐖p\\mathbf\{W\}\_\{p\}together with commutingdiag⁡\(𝚺X​X\)\\mathrm\{diag\}\(\\mathbf\{\\Sigma\}\_\{XX\}\)and𝚺X​X\\mathbf\{\\Sigma\}\_\{XX\}\.

When the modes decouple but a driving eigenvalueγi\\gamma\_\{i\}is complex, the characteristic equation above still governs modeii, and it has a root with positive real part if and only if

Re⁡γi\+β​\(Im⁡γi\)2\(β\+σi\)2\>σi,\\Real\\gamma\_\{i\}\+\\frac\{\\beta\\,\(\\operatorname\{Im\}\\gamma\_\{i\}\)^\{2\}\}\{\(\\beta\+\\sigma\_\{i\}\)^\{2\}\}\\;\>\\;\\sigma\_\{i\},\(30\)which reduces toγi\>σi\\gamma\_\{i\}\>\\sigma\_\{i\}, that isμi\>1\\mu\_\{i\}\>1, for a real drive\. To see this, writeb=β\+σib=\\beta\+\\sigma\_\{i\}andΔ=b2−4​β​\(σi−γi\)\\Delta=b^\{2\}\-4\\beta\(\\sigma\_\{i\}\-\\gamma\_\{i\}\)\. The roots are12​\(−b±Δ\)\\frac\{1\}\{2\}\(\-b\\pm\\sqrt\{\\Delta\}\), so the larger real part is positive exactly whenRe⁡Δ\>b\\Real\\sqrt\{\\Delta\}\>b, which is\|Δ\|\>b2\+4​β​\(σi−Re⁡γi\)\|\\Delta\|\>b^\{2\}\+4\\beta\(\\sigma\_\{i\}\-\\Real\\gamma\_\{i\}\)\. If the right\-hand side is negative, thenRe⁡γi\>σi\\Real\\gamma\_\{i\}\>\\sigma\_\{i\}and both conditions hold; otherwise squaring givesβ​\(Im⁡γi\)2\>b2​\(σi−Re⁡γi\)\\beta\(\\operatorname\{Im\}\\gamma\_\{i\}\)^\{2\}\>b^\{2\}\(\\sigma\_\{i\}\-\\Real\\gamma\_\{i\}\), which is Eq\. \([30](https://arxiv.org/html/2610.02344#A1.E30)\)\. Sinceβ​σi/\(β\+σi\)2≤1/4\\beta\\sigma\_\{i\}/\(\\beta\+\\sigma\_\{i\}\)^\{2\}\\leq 1/4, it follows forσi\>0\\sigma\_\{i\}\>0thatRe⁡μi\>1\\Real\\mu\_\{i\}\>1is sufficient for escape and\|μi\|\>1\|\\mu\_\{i\}\|\>1is necessary, while neither is the criterion\. Without shared eigenbases, Appendix[A\.5](https://arxiv.org/html/2610.02344#A1.SS5)gives certificates that need neither condition\.

##### Alternative real\-valued formulation\.

A fully real and nonnegative driving strength can be obtained by replacing eigenvalues with*singular values*\(which are well\-defined for any matrix\)\. For each pair of indices\(k,ℓ\)\(k,\\ell\), define

γ~k​ℓ:=σ\(k\)​\(𝚺X​1\)⋅σ\(ℓ\)​\(𝐖p⊤​𝐌¯2\),μ~k​ℓ:=γ~k​ℓak​pℓ,\\tilde\{\\gamma\}\_\{k\\ell\}\\;:=\\;\\sigma^\{\(k\)\}\\\!\\bigl\(\\mathbf\{\\Sigma\}\_\{X1\}\\bigr\)\\,\\cdot\\,\\sigma^\{\(\\ell\)\}\\\!\\bigl\(\\mathbf\{W\}\_\{p\}^\{\\top\}\\bar\{\\mathbf\{M\}\}\_\{2\}\\bigr\),\\qquad\\tilde\{\\mu\}\_\{k\\ell\}\\;:=\\;\\frac\{\\tilde\{\\gamma\}\_\{k\\ell\}\}\{a\_\{k\}\\,p\_\{\\ell\}\},\(31\)whereσ\(k\)​\(𝐀\)\\sigma^\{\(k\)\}\(\\mathbf\{A\}\)denotes thekk\-th singular value of𝐀\\mathbf\{A\}in non\-increasing order,ak=σ\(k\)​\(𝚺11\)a\_\{k\}=\\sigma^\{\(k\)\}\(\\mathbf\{\\Sigma\}\_\{11\}\), andpℓ=σ\(ℓ\)​\(𝐖p⊤​𝐖p\)p\_\{\\ell\}=\\sigma^\{\(\\ell\)\}\(\\mathbf\{W\}\_\{p\}^\{\\top\}\\mathbf\{W\}\_\{p\}\)\. By constructionγ~k​ℓ,μ~k​ℓ∈ℝ≥0\\tilde\{\\gamma\}\_\{k\\ell\},\\tilde\{\\mu\}\_\{k\\ell\}\\in\\mathbb\{R\}\_\{\\geq 0\}and the threshold conditionμ~k​ℓ\>1\\tilde\{\\mu\}\_\{k\\ell\}\>1is rigorously well\-defined regardless of the symmetry of𝚺X​1\\mathbf\{\\Sigma\}\_\{X1\}or𝐖p⊤​𝐌¯2\\mathbf\{W\}\_\{p\}^\{\\top\}\\bar\{\\mathbf\{M\}\}\_\{2\}\. When both operators are symmetric and share the orthonormal eigenbases of𝚺11\\mathbf\{\\Sigma\}\_\{11\}and𝐖p⊤​𝐖p\\mathbf\{W\}\_\{p\}^\{\\top\}\\mathbf\{W\}\_\{p\}, their singular values are the\|ck\|\|c\_\{k\}\|and\|qℓ\|\|q\_\{\\ell\}\|, but Eq\. \([31](https://arxiv.org/html/2610.02344#A1.E31)\) sorts them independently of theaka\_\{k\}andpℓp\_\{\\ell\}\. It therefore agrees with the eigenvector\-paired absolute\-value form\(\|ck\|/ak\)⋅\(\|qℓ\|/pℓ\)\(\|c\_\{k\}\|/a\_\{k\}\)\\cdot\(\|q\_\{\\ell\}\|/p\_\{\\ell\}\), which the rank budget of App\.[C](https://arxiv.org/html/2610.02344#A3)uses, only when the two orderings coincide\. For example,𝚺11=diag⁡\(2,1\)\\mathbf\{\\Sigma\}\_\{11\}=\\mathrm\{diag\}\(2,1\)and𝚺X​1=diag⁡\(0\.5,1\.5\)\\mathbf\{\\Sigma\}\_\{X1\}=\\mathrm\{diag\}\(0\.5,1\.5\)with a scalar predictor give one supercritical mode in the paired form, whose flow escapes, and none in Eq\. \([31](https://arxiv.org/html/2610.02344#A1.E31)\)\.

### A\.5Stability of the full nonsymmetric block Jacobian

The discussion above states what the scalar criterion needs and stops there\. This section proves what the operator class admits without any symmetry or decoupling assumption, and shows how the statements collapse to the per\-mode criterion of Section[3](https://arxiv.org/html/2610.02344#S3)when Assumption[3\.2](https://arxiv.org/html/2610.02344#S3.Thmtheorem2)holds exactly\. Throughout,n:=d​Nn:=dN,

𝐒:=𝚺11⊗𝐖p⊤​𝐖p,𝐆:=𝚺X​1⊤⊗𝐖p⊤​𝐌¯2,𝒥=\(−𝐒𝐆β​𝐈n−β​𝐈n\),\\mathbf\{S\}:=\\mathbf\{\\Sigma\}\_\{11\}\\otimes\\mathbf\{W\}\_\{p\}^\{\\top\}\\mathbf\{W\}\_\{p\},\\qquad\\mathbf\{G\}:=\\mathbf\{\\Sigma\}\_\{X1\}^\{\\top\}\\otimes\\mathbf\{W\}\_\{p\}^\{\\top\}\\bar\{\\mathbf\{M\}\}\_\{2\},\\qquad\\mathcal\{J\}=\\begin\{pmatrix\}\-\\mathbf\{S\}&\\mathbf\{G\}\\\\ \\beta\\mathbf\{I\}\_\{n\}&\-\\beta\\mathbf\{I\}\_\{n\}\\end\{pmatrix\},\(32\)so that𝐒\\mathbf\{S\}is real symmetric positive semi\-definite, a Kronecker product of two such factors, while𝐆\\mathbf\{G\}is a general real matrix\. We writeα⁡\(𝒥\):=max⁡\{Re⁡λ:λ∈Spec⁡\(𝒥\)\}\\alpha\(\\mathcal\{J\}\):=\\max\\\{\\Real\\lambda:\\lambda\\in\\Spec\(\\mathcal\{J\}\)\\\}for the spectral abscissa, so that the linearised flow collapses whenα⁡\(𝒥\)<0\\alpha\(\\mathcal\{J\}\)<0and escapes whenα⁡\(𝒥\)\>0\\alpha\(\\mathcal\{J\}\)\>0, andr⁡\(𝐆\):=max‖w‖=1⁡\|w∗​𝐆​w\|r\(\\mathbf\{G\}\):=\\max\_\{\\\|w\\\|=1\}\|w^\{\\ast\}\\mathbf\{G\}w\|for the numerical radius\. All statements are local at the collapsed fixed point, within the scope of Lemma[3\.1](https://arxiv.org/html/2610.02344#S3.Thmtheorem1); they say nothing about non\-trivial equilibria elsewhere\.

###### Theorem A\.1\(Quadratic\-pencil reduction; no assumptions\)\.

LetQ⁡\(λ\):=λ2​𝐈n\+λ⁡\(𝐒\+β​𝐈n\)\+β⁡\(𝐒−𝐆\)Q\(\\lambda\):=\\lambda^\{2\}\\mathbf\{I\}\_\{n\}\+\\lambda\(\\mathbf\{S\}\+\\beta\\mathbf\{I\}\_\{n\}\)\+\\beta\(\\mathbf\{S\}\-\\mathbf\{G\}\)\. Then, as an identity of polynomials inλ\\lambda,det\(λ​𝐈2​n−𝒥\)=detQ⁡\(λ\)\\det\(\\lambda\\mathbf\{I\}\_\{2n\}\-\\mathcal\{J\}\)=\\det Q\(\\lambda\), soSpec⁡\(𝒥\)\\Spec\(\\mathcal\{J\}\), with algebraic multiplicities, is the solution set of the quadratic eigenvalue problemdetQ⁡\(λ\)=0\\det Q\(\\lambda\)=0\. Moreover: \(i\) forλ≠−β\\lambda\\neq\-\\beta, pencil eigenpairs\(λ,w\)\(\\lambda,w\)withQ⁡\(λ\)​w=0Q\(\\lambda\)w=0,w≠0w\\neq 0, correspond bijectively to eigenpairs\(λ,\[w;βλ\+β​w\]\)\(\\lambda,\[w;\\tfrac\{\\beta\}\{\\lambda\+\\beta\}w\]\)of𝒥\\mathcal\{J\}; \(ii\)−β∈Spec⁡\(𝒥\)\-\\beta\\in\\Spec\(\\mathcal\{J\}\)iff𝐆\\mathbf\{G\}is singular; \(iii\)det𝒥=βn​det\(𝐒−𝐆\)\\det\\mathcal\{J\}=\\beta^\{n\}\\det\(\\mathbf\{S\}\-\\mathbf\{G\}\), hence0∈Spec⁡\(𝒥\)0\\in\\Spec\(\\mathcal\{J\}\)iffdet\(𝐒−𝐆\)=0\\det\(\\mathbf\{S\}\-\\mathbf\{G\}\)=0\.

###### Theorem A\.2\(Unconditional abscissa control and collapse certificate\)\.

\(i\) Letλ∈Spec⁡\(𝒥\)\\lambda\\in\\Spec\(\\mathcal\{J\}\)haveRe⁡λ≥0\\Real\\lambda\\geq 0, with eigenvector\[w;wa\]\[w;w\_\{a\}\]normalised so that‖w‖=1\\\|w\\\|=1\. Then

Re⁡λ≤\|w∗​𝐆​w\|−w∗​𝐒​w≤r⁡\(𝐆\)−λmin​\(𝐒\)≤σmax​\(𝐆\)−λmin​\(𝐒\)\.\\Real\\lambda\\;\\leq\\;\|w^\{\\ast\}\\mathbf\{G\}w\|\-w^\{\\ast\}\\mathbf\{S\}w\\;\\leq\\;r\(\\mathbf\{G\}\)\-\\lambda\_\{\\min\}\(\\mathbf\{S\}\)\\;\\leq\\;\\sigma\_\{\\max\}\(\\mathbf\{G\}\)\-\\lambda\_\{\\min\}\(\\mathbf\{S\}\)\.\(33\)\(ii\) Henceα⁡\(𝒥\)≤max⁡\{0,σmax​\(𝐆\)−λmin​\(𝐒\)\}\\alpha\(\\mathcal\{J\}\)\\leq\\max\\\{0,\\,\\sigma\_\{\\max\}\(\\mathbf\{G\}\)\-\\lambda\_\{\\min\}\(\\mathbf\{S\}\)\\\}, and by the Kronecker structureσmax​\(𝐆\)=σ\(1\)​\(𝚺X​1\)​σ\(1\)​\(𝐖p⊤​𝐌¯2\)=γ~11\\sigma\_\{\\max\}\(\\mathbf\{G\}\)=\\sigma^\{\(1\)\}\(\\mathbf\{\\Sigma\}\_\{X1\}\)\\,\\sigma^\{\(1\)\}\(\\mathbf\{W\}\_\{p\}^\{\\top\}\\bar\{\\mathbf\{M\}\}\_\{2\}\)=\\tilde\{\\gamma\}\_\{11\}andλmin​\(𝐒\)=aN​pd\\lambda\_\{\\min\}\(\\mathbf\{S\}\)=a\_\{N\}p\_\{d\}, in the notation of Eq\. \([31](https://arxiv.org/html/2610.02344#A1.E31)\)\. \(iii\) \(Collapse certificate\.\) Ifσmax​\(𝐆\)<λmin​\(𝐒\)\\sigma\_\{\\max\}\(\\mathbf\{G\}\)<\\lambda\_\{\\min\}\(\\mathbf\{S\}\), equivalently, whenλmin​\(𝐒\)\>0\\lambda\_\{\\min\}\(\\mathbf\{S\}\)\>0, if

μˇ:=σmax​\(𝐆\)λmin​\(𝐒\)=γ~11aN​pd=κ⁡\(𝚺11\)​κ​\(𝐖p⊤​𝐖p\)​μ~11<1,\\check\{\\mu\}\\;:=\\;\\frac\{\\sigma\_\{\\max\}\(\\mathbf\{G\}\)\}\{\\lambda\_\{\\min\}\(\\mathbf\{S\}\)\}\\;=\\;\\frac\{\\tilde\{\\gamma\}\_\{11\}\}\{a\_\{N\}p\_\{d\}\}\\;=\\;\\kappa\(\\mathbf\{\\Sigma\}\_\{11\}\)\\,\\kappa\(\\mathbf\{W\}\_\{p\}^\{\\top\}\\mathbf\{W\}\_\{p\}\)\\,\\tilde\{\\mu\}\_\{11\}\\;<\\;1,\(34\)thenα⁡\(𝒥\)<0\\alpha\(\\mathcal\{J\}\)<0: the origin is exponentially stable for the linearised flow, with no symmetry, normality or decoupling assumption\. Moreoverμ~k​ℓ≤μˇ\\tilde\{\\mu\}\_\{k\\ell\}\\leq\\check\{\\mu\}for every\(k,ℓ\)\(k,\\ell\), so the certificate is built from the singular values that define the proxy of Eq\. \([31](https://arxiv.org/html/2610.02344#A1.E31)\) and dominates every per\-modeμ~k​ℓ\\tilde\{\\mu\}\_\{k\\ell\}\. \(iv\) \(Explicit rate\.\) For anyc∈\[0,β\)c\\in\[0,\\beta\)withβ​σmax​\(𝐆\)/\(β−c\)−λmin​\(𝐒\)<−c\\beta\\sigma\_\{\\max\}\(\\mathbf\{G\}\)/\(\\beta\-c\)\-\\lambda\_\{\\min\}\(\\mathbf\{S\}\)<\-c, one hasα⁡\(𝒥\)<−c\\alpha\(\\mathcal\{J\}\)<\-c\. \(v\) \(Non\-necessity\.\) The converse of \(iii\) is false:μˇ≥1\\check\{\\mu\}\\geq 1does not imply instability\. With𝐒=𝐈n\\mathbf\{S\}=\\mathbf\{I\}\_\{n\}and𝐆\\mathbf\{G\}whose sole non\-zero entry isG12=100G\_\{12\}=100, so thatμˇ=100\\check\{\\mu\}=100, one hasdetQ⁡\(λ\)=\(\(λ\+1\)​\(λ\+β\)\)n\\det Q\(\\lambda\)=\(\(\\lambda\+1\)\(\\lambda\+\\beta\)\)^\{n\}, henceSpec⁡\(𝒥\)=\{−1,−β\}\\Spec\(\\mathcal\{J\}\)=\\\{\-1,\-\\beta\\\}andα⁡\(𝒥\)<0\\alpha\(\\mathcal\{J\}\)<0\. Singular values alone cannot decide the stability of𝒥\\mathcal\{J\}; the gap is a non\-normality effect\.

###### Theorem A\.3\(Escape certificates\)\.

\(i\) \(General𝐆\\mathbf\{G\}\.\) Ifdet\(𝐒−𝐆\)<0\\det\(\\mathbf\{S\}\-\\mathbf\{G\}\)<0, then𝒥\\mathcal\{J\}has a real eigenvalueλ∗\>0\\lambda^\{\\ast\}\>0and the origin is exponentially unstable\. \(ii\) \(Symmetric drive: a sharp trichotomy\.\) Assume𝐆=𝐆⊤\\mathbf\{G\}=\\mathbf\{G\}^\{\\top\}, which holds under sufficient condition 1 of Appendix[A\.4](https://arxiv.org/html/2610.02344#A1.SS4)and in particular under the exact form of Assumption[3\.2](https://arxiv.org/html/2610.02344#S3.Thmtheorem2)with orthonormal shared bases\. Call a unitw∈ℝnw\\in\\mathbb\{R\}^\{n\}admissible ifδw:=−w⊤​\(𝐒−𝐆\)​w\>0\\delta\_\{w\}:=\-w^\{\\top\}\(\\mathbf\{S\}\-\\mathbf\{G\}\)w\>0\. Then: \(a\)𝐒−𝐆≻0\\mathbf\{S\}\-\\mathbf\{G\}\\succ 0iffα⁡\(𝒥\)<0\\alpha\(\\mathcal\{J\}\)<0; \(b\)λmin​\(𝐒−𝐆\)=0\\lambda\_\{\\min\}\(\\mathbf\{S\}\-\\mathbf\{G\}\)=0impliesα⁡\(𝒥\)=0\\alpha\(\\mathcal\{J\}\)=0; \(c\)λmin​\(𝐒−𝐆\)<0\\lambda\_\{\\min\}\(\\mathbf\{S\}\-\\mathbf\{G\}\)<0implies that𝒥\\mathcal\{J\}has a real eigenvalueλ∗\>0\\lambda^\{\\ast\}\>0, and for every admissiblewwsome real eigenvalue of𝒥\\mathcal\{J\}lies in\[λ\+​\(w\),Λ\]\[\\lambda\_\{\+\}\(w\),\\Lambda\], where

λ\+​\(w\)\\displaystyle\\lambda\_\{\+\}\(w\):=12​\[\(w⊤​𝐒​w\+β\)2\+4​β​δw−\(w⊤​𝐒​w\+β\)\]\>0,\\displaystyle:=\\tfrac\{1\}\{2\}\\Bigl\[\\sqrt\{\(w^\{\\top\}\\mathbf\{S\}w\+\\beta\)^\{2\}\+4\\beta\\delta\_\{w\}\}\-\(w^\{\\top\}\\mathbf\{S\}w\+\\beta\)\\Bigr\]\>0,\(35\)Λ\\displaystyle\\Lambda:=12​\(−β\+β2\+4​β​‖𝐒−𝐆‖2\);\\displaystyle:=\\tfrac\{1\}\{2\}\\bigl\(\-\\beta\+\\sqrt\{\\beta^\{2\}\+4\\beta\\\|\\mathbf\{S\}\-\\mathbf\{G\}\\\|\_\{2\}\}\\bigr\);every real non\-negative eigenvalue of𝒥\\mathcal\{J\}is at mostΛ\\Lambda, andα⁡\(𝒥\)≥supwλ\+​\(w\)\\alpha\(\\mathcal\{J\}\)\\geq\\sup\_\{w\}\\lambda\_\{\+\}\(w\)over admissibleww\. In particular, for symmetric drive,α⁡\(𝒥\)\>0\\alpha\(\\mathcal\{J\}\)\>0iffλmin​\(𝐒−𝐆\)<0\\lambda\_\{\\min\}\(\\mathbf\{S\}\-\\mathbf\{G\}\)<0iff some direction hasw⊤​𝐆​w\>w⊤​𝐒​ww^\{\\top\}\\mathbf\{G\}w\>w^\{\\top\}\\mathbf\{S\}w: drive beats decay as quadratic forms\. This is a complete stability statement with no decoupling assumption\. Under symmetry, \(ii\) refines \(i\), since an even number of negative eigenvalues of𝐒−𝐆\\mathbf\{S\}\-\\mathbf\{G\}has positive determinant and is detected only by \(ii\)\.

###### Corollary A\.4\(Exact Assumption[3\.2](https://arxiv.org/html/2610.02344#S3.Thmtheorem2)recovers the per\-mode criterion\)\.

Assume Assumption[3\.2](https://arxiv.org/html/2610.02344#S3.Thmtheorem2)exactly, in the form of Step 1 of Appendix[C](https://arxiv.org/html/2610.02344#A3): orthogonal𝐔\\mathbf\{U\},𝐕\\mathbf\{V\}with𝚺11=𝐔​diag⁡\(ak\)​𝐔⊤\\mathbf\{\\Sigma\}\_\{11\}=\\mathbf\{U\}\\diag\(a\_\{k\}\)\\mathbf\{U\}^\{\\top\},𝚺X​1=𝐔​diag⁡\(ck\)​𝐔⊤\\mathbf\{\\Sigma\}\_\{X1\}=\\mathbf\{U\}\\diag\(c\_\{k\}\)\\mathbf\{U\}^\{\\top\},𝐖p⊤​𝐖p=𝐕​diag⁡\(pℓ\)​𝐕⊤\\mathbf\{W\}\_\{p\}^\{\\top\}\\mathbf\{W\}\_\{p\}=\\mathbf\{V\}\\diag\(p\_\{\\ell\}\)\\mathbf\{V\}^\{\\top\},𝐖p⊤​𝐌¯2=𝐕​diag⁡\(qℓ\)​𝐕⊤\\mathbf\{W\}\_\{p\}^\{\\top\}\\bar\{\\mathbf\{M\}\}\_\{2\}=\\mathbf\{V\}\\diag\(q\_\{\\ell\}\)\\mathbf\{V\}^\{\\top\}\. This forces𝐆=𝐆⊤\\mathbf\{G\}=\\mathbf\{G\}^\{\\top\}and\[𝐒,𝐆\]=0\[\\mathbf\{S\},\\mathbf\{G\}\]=0\. Withσk​ℓ:=ak​pℓ\\sigma\_\{k\\ell\}:=a\_\{k\}p\_\{\\ell\}andγk​ℓ:=ck​qℓ\\gamma\_\{k\\ell\}:=c\_\{k\}q\_\{\\ell\}:

1. 1\.det\(λ​𝐈2​n−𝒥\)=∏k,ℓ\[λ2\+\(β\+σk​ℓ\)​λ\+β⁡\(σk​ℓ−γk​ℓ\)\]\\det\(\\lambda\\mathbf\{I\}\_\{2n\}\-\\mathcal\{J\}\)=\\prod\_\{k,\\ell\}\\bigl\[\\lambda^\{2\}\+\(\\beta\+\\sigma\_\{k\\ell\}\)\\lambda\+\\beta\(\\sigma\_\{k\\ell\}\-\\gamma\_\{k\\ell\}\)\\bigr\], the per\-mode characteristic polynomial of Section[3](https://arxiv.org/html/2610.02344#S3), with constant termβ​σk​ℓ​\(1−μk​ℓ\)\\beta\\sigma\_\{k\\ell\}\(1\-\\mu\_\{k\\ell\}\)wheneverσk​ℓ\>0\\sigma\_\{k\\ell\}\>0\.
2. 2\.λmin​\(𝐒−𝐆\)=mink​ℓ⁡\(σk​ℓ−γk​ℓ\)\\lambda\_\{\\min\}\(\\mathbf\{S\}\-\\mathbf\{G\}\)=\\min\_\{k\\ell\}\(\\sigma\_\{k\\ell\}\-\\gamma\_\{k\\ell\}\), so Theorem[A\.3](https://arxiv.org/html/2610.02344#A1.Thmtheorem3)\(ii\) reduces mode by mode to the paper’s criterion:𝒥\\mathcal\{J\}is Hurwitz iffσk​ℓ\>γk​ℓ\\sigma\_\{k\\ell\}\>\\gamma\_\{k\\ell\}for all modes, i\.e\.maxk​ℓ⁡μk​ℓ<1\\max\_\{k\\ell\}\\mu\_\{k\\ell\}<1when allσk​ℓ\>0\\sigma\_\{k\\ell\}\>0; escape iffγk​ℓ\>σk​ℓ\\gamma\_\{k\\ell\}\>\\sigma\_\{k\\ell\}for some mode\. Takingw=𝐮k⊗𝐯ℓw=\\mathbf\{u\}\_\{k\}\\otimes\\mathbf\{v\}\_\{\\ell\}givesλ\+​\(w\)=λk​ℓ\+\\lambda\_\{\+\}\(w\)=\\lambda^\{\+\}\_\{k\\ell\}, the per\-mode escape rate, and α\(𝒥\)=max\{\(k,ℓ\):γk​ℓ\>σk​ℓ\}λk​ℓ\+\\alpha\(\\mathcal\{J\}\)\\;=\\;\\max\_\{\\\{\(k,\\ell\)\\,:\\,\\gamma\_\{k\\ell\}\>\\sigma\_\{k\\ell\}\\\}\}\\lambda^\{\+\}\_\{k\\ell\}\(36\)whenever a supercritical mode exists: the variational lower bound is attained with equality\. The index set is written in the division\-free form so that degenerate escaping modes,σk​ℓ=0<γk​ℓ\\sigma\_\{k\\ell\}=0<\\gamma\_\{k\\ell\}, for whichμk​ℓ\\mu\_\{k\\ell\}is undefined, are included\.
3. 3\.μ~k​ℓ≤μˇ=κ⁡\(𝚺11\)​κ​\(𝐖p⊤​𝐖p\)​μ~11\\tilde\{\\mu\}\_\{k\\ell\}\\leq\\check\{\\mu\}=\\kappa\(\\mathbf\{\\Sigma\}\_\{11\}\)\\kappa\(\\mathbf\{W\}\_\{p\}^\{\\top\}\\mathbf\{W\}\_\{p\}\)\\,\\tilde\{\\mu\}\_\{11\}for all\(k,ℓ\)\(k,\\ell\): the unconditional certificate is conservative relative to the per\-mode threshold by exactly the condition numbers of the decay factors, and tightens to theμ~\\tilde\{\\mu\}criterion as those factors become well conditioned\.
4. 4\.\(Adiabatic tightness\.\)max‖u‖=1⁡\(\|u∗​𝐆​u\|−u∗​𝐒​u\)=maxk​ℓ⁡\(\|γk​ℓ\|−σk​ℓ\)\\max\_\{\\\|u\\\|=1\}\(\|u^\{\\ast\}\\mathbf\{G\}u\|\-u^\{\\ast\}\\mathbf\{S\}u\)=\\max\_\{k\\ell\}\(\|\\gamma\_\{k\\ell\}\|\-\\sigma\_\{k\\ell\}\), and on a supercritical modeλk​ℓ\+↑γk​ℓ−σk​ℓ\\lambda^\{\+\}\_\{k\\ell\}\\uparrow\\gamma\_\{k\\ell\}\-\\sigma\_\{k\\ell\}monotonically asβ→∞\\beta\\to\\infty\. Hence the bound of Theorem[A\.2](https://arxiv.org/html/2610.02344#A1.Thmtheorem2)\(i\) is asymptotically attained in the adiabatic limit provided the maximiser of\|γk​ℓ\|−σk​ℓ\|\\gamma\_\{k\\ell\}\|\-\\sigma\_\{k\\ell\}hasγk​ℓ\>0\\gamma\_\{k\\ell\}\>0, in particular whenever allck​qℓ≥0c\_\{k\}q\_\{\\ell\}\\geq 0; with negative drive eigenvalues it can be strictly conservative for everyβ\\beta\(modes\(σ,γ\)=\(1,−5\)\(\\sigma,\\gamma\)=\(1,\-5\)and\(2,0\.5\)\(2,0\.5\)give the bound44while𝒥\\mathcal\{J\}is Hurwitz for everyβ\\beta\)\.

Together, the three theorems bracketα⁡\(𝒥\)\\alpha\(\\mathcal\{J\}\)between computable singular\-value and quadratic\-form quantities, and the bracket closes to the exact per\-mode answer as Assumption[3\.2](https://arxiv.org/html/2610.02344#S3.Thmtheorem2)becomes exact\.

#### Proofs

###### Proof of Theorem[A\.1](https://arxiv.org/html/2610.02344#A1.Thmtheorem1)\.

Fixλ∈ℂ\\lambda\\in\\mathbb\{C\}withλ≠−β\\lambda\\neq\-\\betaand writeλ​𝐈2​n−𝒥=\(λ​𝐈\+𝐒−𝐆−β​𝐈\(λ\+β\)​𝐈\)\\lambda\\mathbf\{I\}\_\{2n\}\-\\mathcal\{J\}=\\begin\{pmatrix\}\\lambda\\mathbf\{I\}\+\\mathbf\{S\}&\-\\mathbf\{G\}\\\\ \-\\beta\\mathbf\{I\}&\(\\lambda\+\\beta\)\\mathbf\{I\}\\end\{pmatrix\}\. The lower\-right block is invertible, so the Schur determinant formula\[[Horn and Johnson, 2013](https://arxiv.org/html/2610.02344#bib.bib20)\]givesdet\(λ​𝐈2​n−𝒥\)=\(λ\+β\)n​det\(λ​𝐈\+𝐒−βλ\+β​𝐆\)=det\(\(λ\+β\)​\(λ​𝐈\+𝐒\)−β​𝐆\)=detQ⁡\(λ\)\\det\(\\lambda\\mathbf\{I\}\_\{2n\}\-\\mathcal\{J\}\)=\(\\lambda\+\\beta\)^\{n\}\\det\\bigl\(\\lambda\\mathbf\{I\}\+\\mathbf\{S\}\-\\tfrac\{\\beta\}\{\\lambda\+\\beta\}\\mathbf\{G\}\\bigr\)=\\det\\bigl\(\(\\lambda\+\\beta\)\(\\lambda\\mathbf\{I\}\+\\mathbf\{S\}\)\-\\beta\\mathbf\{G\}\\bigr\)=\\det Q\(\\lambda\), using\(λ\+β\)​\(λ​𝐈\+𝐒\)=λ2​𝐈\+λ​𝐒\+β​λ​𝐈\+β​𝐒\(\\lambda\+\\beta\)\(\\lambda\\mathbf\{I\}\+\\mathbf\{S\}\)=\\lambda^\{2\}\\mathbf\{I\}\+\\lambda\\mathbf\{S\}\+\\beta\\lambda\\mathbf\{I\}\+\\beta\\mathbf\{S\}\. Both sides are polynomials inλ\\lambdaagreeing onℂ∖\{−β\}\\mathbb\{C\}\\setminus\\\{\-\\beta\\\}, hence identically, and both are monic of degree2​n2n, so the spectrum with multiplicities is the root set ofdetQ\\det Q\. \(i\) Let𝒥​z=λ​z\\mathcal\{J\}z=\\lambda zwithz=\[w;wa\]≠0z=\[w;w\_\{a\}\]\\neq 0andλ≠−β\\lambda\\neq\-\\beta\. The second block row readsβ​w=\(λ\+β\)​wa\\beta w=\(\\lambda\+\\beta\)w\_\{a\}, sowa=βλ\+β​ww\_\{a\}=\\tfrac\{\\beta\}\{\\lambda\+\\beta\}wandw≠0w\\neq 0; substituting into the first row and multiplying by\(λ\+β\)\(\\lambda\+\\beta\)givesQ⁡\(λ\)​w=0Q\(\\lambda\)w=0\. The steps reverse\. \(ii\) Ifλ=−β\\lambda=\-\\beta, the second row forcesw=0w=0and the first reduces to𝐆​wa=0\\mathbf\{G\}w\_\{a\}=0; consistentlyQ⁡\(−β\)=−β​𝐆Q\(\-\\beta\)=\-\\beta\\mathbf\{G\}\. \(iii\) Settingλ=0\\lambda=0:det\(−𝒥\)=detQ⁡\(0\)=βn​det\(𝐒−𝐆\)\\det\(\-\\mathcal\{J\}\)=\\det Q\(0\)=\\beta^\{n\}\\det\(\\mathbf\{S\}\-\\mathbf\{G\}\), anddet\(−𝒥\)=det𝒥\\det\(\-\\mathcal\{J\}\)=\\det\\mathcal\{J\}because2​n2nis even\. ∎

###### Proof of Theorem[A\.2](https://arxiv.org/html/2610.02344#A1.Thmtheorem2)\.

\(i\) Letλ∈Spec⁡\(𝒥\)\\lambda\\in\\Spec\(\\mathcal\{J\}\)withRe⁡λ≥0\\Real\\lambda\\geq 0and eigenvector\[w;wa\]\[w;w\_\{a\}\]\. SinceRe⁡\(λ\+β\)≥β\>0\\Real\(\\lambda\+\\beta\)\\geq\\beta\>0,λ≠−β\\lambda\\neq\-\\beta, sowa=βλ\+β​ww\_\{a\}=\\tfrac\{\\beta\}\{\\lambda\+\\beta\}wwithw≠0w\\neq 0; normalise‖w‖=1\\\|w\\\|=1\. The inner product of the first block row withwwgivesλ=−w∗​𝐒​w\+βλ\+β​w∗​𝐆​w\\lambda=\-w^\{\\ast\}\\mathbf\{S\}w\+\\tfrac\{\\beta\}\{\\lambda\+\\beta\}w^\{\\ast\}\\mathbf\{G\}w\. For real symmetric𝐒\\mathbf\{S\},w∗​𝐒​w∈ℝw^\{\\ast\}\\mathbf\{S\}w\\in\\mathbb\{R\}andw∗​𝐒​w≥λmin​\(𝐒\)≥0w^\{\\ast\}\\mathbf\{S\}w\\geq\\lambda\_\{\\min\}\(\\mathbf\{S\}\)\\geq 0by the Rayleigh bound; also\|λ\+β\|≥β\|\\lambda\+\\beta\|\\geq\\beta, so\|β/\(λ\+β\)\|≤1\|\\beta/\(\\lambda\+\\beta\)\|\\leq 1\. Taking real parts,Re⁡λ≤−w∗​𝐒​w\+\|w∗​𝐆​w\|\\Real\\lambda\\leq\-w^\{\\ast\}\\mathbf\{S\}w\+\|w^\{\\ast\}\\mathbf\{G\}w\|, and the chain follows from\|w∗​𝐆​w\|≤r⁡\(𝐆\)≤σmax​\(𝐆\)\|w^\{\\ast\}\\mathbf\{G\}w\|\\leq r\(\\mathbf\{G\}\)\\leq\\sigma\_\{\\max\}\(\\mathbf\{G\}\)\. \(ii\) Ifα⁡\(𝒥\)≥0\\alpha\(\\mathcal\{J\}\)\\geq 0the maximising eigenvalue satisfies \(i\)\. Singular values of𝐀⊗𝐁\\mathbf\{A\}\\otimes\\mathbf\{B\}are the productsσi​\(𝐀\)​σj​\(𝐁\)\\sigma\_\{i\}\(\\mathbf\{A\}\)\\sigma\_\{j\}\(\\mathbf\{B\}\)and eigenvalues of a Kronecker product of symmetric matrices are the products of eigenvalues\[[Horn and Johnson, 1991](https://arxiv.org/html/2610.02344#bib.bib19)\], and transposition preserves singular values; henceσmax​\(𝐆\)=γ~11\\sigma\_\{\\max\}\(\\mathbf\{G\}\)=\\tilde\{\\gamma\}\_\{11\}and, both factors of𝐒\\mathbf\{S\}being positive semi\-definite,λmin​\(𝐒\)=aN​pd\\lambda\_\{\\min\}\(\\mathbf\{S\}\)=a\_\{N\}p\_\{d\}\. \(iii\) Ifσmax​\(𝐆\)<λmin​\(𝐒\)\\sigma\_\{\\max\}\(\\mathbf\{G\}\)<\\lambda\_\{\\min\}\(\\mathbf\{S\}\), an eigenvalue withRe⁡λ≥0\\Real\\lambda\\geq 0would satisfy0≤Re⁡λ<00\\leq\\Real\\lambda<0; the spectrum is finite, soα⁡\(𝒥\)<0\\alpha\(\\mathcal\{J\}\)<0strictly\. The identityμˇ=κ⁡\(𝚺11\)​κ​\(𝐖p⊤​𝐖p\)​μ~11\\check\{\\mu\}=\\kappa\(\\mathbf\{\\Sigma\}\_\{11\}\)\\kappa\(\\mathbf\{W\}\_\{p\}^\{\\top\}\\mathbf\{W\}\_\{p\}\)\\tilde\{\\mu\}\_\{11\}follows from \(ii\) andμ~11=γ~11/\(a1​p1\)\\tilde\{\\mu\}\_\{11\}=\\tilde\{\\gamma\}\_\{11\}/\(a\_\{1\}p\_\{1\}\)\. For the domination,σ\(k\)​\(𝚺X​1\)≤σ\(1\)​\(𝚺X​1\)\\sigma^\{\(k\)\}\(\\mathbf\{\\Sigma\}\_\{X1\}\)\\leq\\sigma^\{\(1\)\}\(\\mathbf\{\\Sigma\}\_\{X1\}\),σ\(ℓ\)​\(𝐖p⊤​𝐌¯2\)≤σ\(1\)​\(𝐖p⊤​𝐌¯2\)\\sigma^\{\(\\ell\)\}\(\\mathbf\{W\}\_\{p\}^\{\\top\}\\bar\{\\mathbf\{M\}\}\_\{2\}\)\\leq\\sigma^\{\(1\)\}\(\\mathbf\{W\}\_\{p\}^\{\\top\}\\bar\{\\mathbf\{M\}\}\_\{2\}\),ak≥aNa\_\{k\}\\geq a\_\{N\}andpℓ≥pdp\_\{\\ell\}\\geq p\_\{d\}, henceμ~k​ℓ≤γ~11/\(aN​pd\)=μˇ\\tilde\{\\mu\}\_\{k\\ell\}\\leq\\tilde\{\\gamma\}\_\{11\}/\(a\_\{N\}p\_\{d\}\)=\\check\{\\mu\}\. \(iv\) If some eigenvalue hadRe⁡λ≥−c\\Real\\lambda\\geq\-c, thenRe⁡\(λ\+β\)≥β−c\>0\\Real\(\\lambda\+\\beta\)\\geq\\beta\-c\>0, the elimination of \(i\) is valid with\|β/\(λ\+β\)\|≤β/\(β−c\)\|\\beta/\(\\lambda\+\\beta\)\|\\leq\\beta/\(\\beta\-c\), and the real\-part estimate givesRe⁡λ≤−λmin​\(𝐒\)\+ββ−c​σmax​\(𝐆\)<−c\\Real\\lambda\\leq\-\\lambda\_\{\\min\}\(\\mathbf\{S\}\)\+\\tfrac\{\\beta\}\{\\beta\-c\}\\sigma\_\{\\max\}\(\\mathbf\{G\}\)<\-c, a contradiction\. \(v\) With𝐒=𝐈n\\mathbf\{S\}=\\mathbf\{I\}\_\{n\}and𝐆\\mathbf\{G\}as stated, strictly upper triangular with a single non\-zero entry,Q⁡\(λ\)Q\(\\lambda\)is upper triangular with diagonal entriesλ2\+\(1\+β\)​λ\+β=\(λ\+1\)​\(λ\+β\)\\lambda^\{2\}\+\(1\+\\beta\)\\lambda\+\\beta=\(\\lambda\+1\)\(\\lambda\+\\beta\), sodetQ⁡\(λ\)=\(\(λ\+1\)​\(λ\+β\)\)n\\det Q\(\\lambda\)=\(\(\\lambda\+1\)\(\\lambda\+\\beta\)\)^\{n\}andSpec⁡\(𝒥\)=\{−1,−β\}\\Spec\(\\mathcal\{J\}\)=\\\{\-1,\-\\beta\\\}by Theorem[A\.1](https://arxiv.org/html/2610.02344#A1.Thmtheorem1), whileμˇ=\|G12\|\\check\{\\mu\}=\|G\_\{12\}\|is arbitrary\. ∎

###### Proof of Theorem[A\.3](https://arxiv.org/html/2610.02344#A1.Thmtheorem3)\.

\(i\)p⁡\(λ\):=det\(λ​𝐈2​n−𝒥\)p\(\\lambda\):=\\det\(\\lambda\\mathbf\{I\}\_\{2n\}\-\\mathcal\{J\}\)is a monic real polynomial withp⁡\(0\)=βn​det\(𝐒−𝐆\)<0p\(0\)=\\beta^\{n\}\\det\(\\mathbf\{S\}\-\\mathbf\{G\}\)<0by Theorem[A\.1](https://arxiv.org/html/2610.02344#A1.Thmtheorem1); sincep⁡\(λ\)→\+∞p\(\\lambda\)\\to\+\\inftyalong the positive reals, the intermediate value theorem gives a real rootλ∗\>0\\lambda^\{\\ast\}\>0\. \(ii\) Assume𝐆=𝐆⊤\\mathbf\{G\}=\\mathbf\{G\}^\{\\top\}, so that the coefficient matrices ofQQare symmetric\.*Case \(a\)\.*Letλ∈Spec⁡\(𝒥\)\\lambda\\in\\Spec\(\\mathcal\{J\}\); by Theorem[A\.1](https://arxiv.org/html/2610.02344#A1.Thmtheorem1)there isx≠0x\\neq 0withQ⁡\(λ\)​x=0Q\(\\lambda\)x=0, henceλ2​mx\+λ​c\+k=0\\lambda^\{2\}m\_\{x\}\+\\lambda c\+k=0withmx:=‖x‖2\>0m\_\{x\}:=\\\|x\\\|^\{2\}\>0,c:=x∗​\(𝐒\+β​𝐈\)​x≥β​mx\>0c:=x^\{\\ast\}\(\\mathbf\{S\}\+\\beta\\mathbf\{I\}\)x\\geq\\beta m\_\{x\}\>0andk:=β​x∗​\(𝐒−𝐆\)​x\>0k:=\\beta x^\{\\ast\}\(\\mathbf\{S\}\-\\mathbf\{G\}\)x\>0, the last two real because the forms are Hermitian\. A real quadratic with positive coefficients has both roots in the open left half\-plane, soα⁡\(𝒥\)<0\\alpha\(\\mathcal\{J\}\)<0; the converse is Cases \(b\) and \(c\)\.*Case \(b\)\.*The same computation withk≥0k\\geq 0givesRe⁡λ≤0\\Real\\lambda\\leq 0for every eigenvalue, anddet\(𝐒−𝐆\)=0\\det\(\\mathbf\{S\}\-\\mathbf\{G\}\)=0puts00in the spectrum by Theorem[A\.1](https://arxiv.org/html/2610.02344#A1.Thmtheorem1)\(iii\)\.*Case \(c\)\.*Leth⁡\(λ\):=λmin​\(Q⁡\(λ\)\)h\(\\lambda\):=\\lambda\_\{\\min\}\(Q\(\\lambda\)\)on\[0,∞\)\[0,\\infty\), continuous becauseλmin\\lambda\_\{\\min\}is11\-Lipschitz in the spectral norm by Weyl’s inequality\. For an admissible unitwwthe scalar quadraticfw​\(λ\):=w⊤​Q​\(λ\)​w=λ2\+\(w⊤​𝐒​w\+β\)​λ−β​δwf\_\{w\}\(\\lambda\):=w^\{\\top\}Q\(\\lambda\)w=\\lambda^\{2\}\+\(w^\{\\top\}\\mathbf\{S\}w\+\\beta\)\\lambda\-\\beta\\delta\_\{w\}has exactly one positive root,λ\+​\(w\)\\lambda\_\{\+\}\(w\), and the variational characterisation givesh⁡\(λ\+​\(w\)\)≤fw​\(λ\+​\(w\)\)=0h\(\\lambda\_\{\+\}\(w\)\)\\leq f\_\{w\}\(\\lambda\_\{\+\}\(w\)\)=0\. For any unitvvandλ≥0\\lambda\\geq 0,v⊤​Q​\(λ\)​v≥λ2\+β​λ−β​‖𝐒−𝐆‖2v^\{\\top\}Q\(\\lambda\)v\\geq\\lambda^\{2\}\+\\beta\\lambda\-\\beta\\\|\\mathbf\{S\}\-\\mathbf\{G\}\\\|\_\{2\}, soh⁡\(λ\)\>0h\(\\lambda\)\>0forλ\>Λ\\lambda\>\\Lambda: every real non\-negative eigenvalue lies in\[0,Λ\]\[0,\\Lambda\], and the intermediate value theorem gives a real eigenvalue in\[λ\+​\(w\),Λ\]\[\\lambda\_\{\+\}\(w\),\\Lambda\]\. The three cases are exhaustive and exclusive inλmin​\(𝐒−𝐆\)\\lambda\_\{\\min\}\(\\mathbf\{S\}\-\\mathbf\{G\}\)\. For the refinement,det\(𝐒−𝐆\)<0\\det\(\\mathbf\{S\}\-\\mathbf\{G\}\)<0iff the number of negative eigenvalues is odd, which forcesλmin​\(𝐒−𝐆\)<0\\lambda\_\{\\min\}\(\\mathbf\{S\}\-\\mathbf\{G\}\)<0\. ∎

###### Proof of Corollary[A\.4](https://arxiv.org/html/2610.02344#A1.Thmtheorem4)\.

\(1\) Under the hypothesis𝚺X​1\\mathbf\{\\Sigma\}\_\{X1\}and𝐖p⊤​𝐌¯2\\mathbf\{W\}\_\{p\}^\{\\top\}\\bar\{\\mathbf\{M\}\}\_\{2\}are symmetric, so𝐆\\mathbf\{G\}is symmetric; with𝚷:=𝐔⊗𝐕\\mathbf\{\\Pi\}:=\\mathbf\{U\}\\otimes\\mathbf\{V\}the mixed\-product rule gives𝐒=𝚷​diag⁡\(σk​ℓ\)​𝚷⊤\\mathbf\{S\}=\\mathbf\{\\Pi\}\\diag\(\\sigma\_\{k\\ell\}\)\\mathbf\{\\Pi\}^\{\\top\}and𝐆=𝚷​diag⁡\(γk​ℓ\)​𝚷⊤\\mathbf\{G\}=\\mathbf\{\\Pi\}\\diag\(\\gamma\_\{k\\ell\}\)\\mathbf\{\\Pi\}^\{\\top\}, hence𝚷⊤​Q​\(λ\)​𝚷\\mathbf\{\\Pi\}^\{\\top\}Q\(\\lambda\)\\mathbf\{\\Pi\}is diagonal with the per\-mode quadratics anddetQ\\det Qfactors; Theorem[A\.1](https://arxiv.org/html/2610.02344#A1.Thmtheorem1)identifies the factorisation with the spectrum\. \(2\)𝐒−𝐆\\mathbf\{S\}\-\\mathbf\{G\}is diagonal in the joint basis, soλmin​\(𝐒−𝐆\)=mink​ℓ⁡\(σk​ℓ−γk​ℓ\)\\lambda\_\{\\min\}\(\\mathbf\{S\}\-\\mathbf\{G\}\)=\\min\_\{k\\ell\}\(\\sigma\_\{k\\ell\}\-\\gamma\_\{k\\ell\}\)and Theorem[A\.3](https://arxiv.org/html/2610.02344#A1.Thmtheorem3)\(ii\) gives the mode\-wise equivalence;w=𝐮k⊗𝐯ℓw=\\mathbf\{u\}\_\{k\}\\otimes\\mathbf\{v\}\_\{\\ell\}hasw⊤​𝐒​w=σk​ℓw^\{\\top\}\\mathbf\{S\}w=\\sigma\_\{k\\ell\}andδw=γk​ℓ−σk​ℓ\\delta\_\{w\}=\\gamma\_\{k\\ell\}\-\\sigma\_\{k\\ell\}, soλ\+​\(w\)\\lambda\_\{\+\}\(w\)is the positive root of the mode’s quadratic\. Since the spectrum is exactly the union of per\-mode roots, and each quadratic withb=β\+σk​ℓ\>0b=\\beta\+\\sigma\_\{k\\ell\}\>0has both roots withRe⁡λ≤0\\Real\\lambda\\leq 0when its constant term is non\-negative and a unique positive root otherwise, the maximum over supercritical modes is attained\. \(3\) is proved inside Theorem[A\.2](https://arxiv.org/html/2610.02344#A1.Thmtheorem2)\(iii\)\. \(4\) In joint coordinates\|w∗​𝐆​w\|−w∗​𝐒​w≤∑k​ℓ\(\|γk​ℓ\|−σk​ℓ\)​\|w^k​ℓ\|2≤maxk​ℓ⁡\(\|γk​ℓ\|−σk​ℓ\)\|w^\{\\ast\}\\mathbf\{G\}w\|\-w^\{\\ast\}\\mathbf\{S\}w\\leq\\sum\_\{k\\ell\}\(\|\\gamma\_\{k\\ell\}\|\-\\sigma\_\{k\\ell\}\)\|\\hat\{w\}\_\{k\\ell\}\|^\{2\}\\leq\\max\_\{k\\ell\}\(\|\\gamma\_\{k\\ell\}\|\-\\sigma\_\{k\\ell\}\), with equality at the maximising mode\. For a supercritical mode,F⁡\(λ,β\):=λ2\+\(β\+σ\)​λ−β⁡\(γ−σ\)F\(\\lambda,\\beta\):=\\lambda^\{2\}\+\(\\beta\+\\sigma\)\\lambda\-\\beta\(\\gamma\-\\sigma\)satisfiesF⁡\(γ−σ,β\)=\(γ−σ\)​γ\>0F\(\\gamma\-\\sigma,\\beta\)=\(\\gamma\-\\sigma\)\\gamma\>0, soλ\+<γ−σ\\lambda^\{\+\}<\\gamma\-\\sigma; implicit differentiation givesd​λ\+/d​β\>0d\\lambda^\{\+\}/d\\beta\>0; and dividingF=0F=0byβ\\betaand lettingβ→∞\\beta\\to\\inftygivesλ\+→γ−σ\\lambda^\{\+\}\\to\\gamma\-\\sigma\. A mode whose\|γk​ℓ\|−σk​ℓ\|\\gamma\_\{k\\ell\}\|\-\\sigma\_\{k\\ell\}is maximal but hasγk​ℓ<0\\gamma\_\{k\\ell\}<0is stable and contributes nothing to the abscissa, which is why the sign qualifier is needed\. ∎

## Appendix BSpectral\-Decoupling Regime: Analytical and Empirical Evidence

We complement the discussion of Assumption[3\.2](https://arxiv.org/html/2610.02344#S3.Thmtheorem2)with a closed\-form algebraic identity and a quantitative measurement of the resulting commutators on real data\. The picture that emerges is a clean separation between a*masking\-side factor*that we control through the context ratiopm1p\_\{m\_\{1\}\}and a*data\-side factor*that depends on the spectral geometry of each benchmark\. Throughout, we keep the discussion constructive: the identity below makes the regime of validity*operationally falsifiable*, and the measurements show that all three datasets used in the main text sit comfortably inside it\.

### B\.1Commutator definition and what it controls

Assumption[3\.2](https://arxiv.org/html/2610.02344#S3.Thmtheorem2)requires that the data\-side covariances𝚺11\\mathbf\{\\Sigma\}\_\{11\},𝚺X​1\\mathbf\{\\Sigma\}\_\{X1\}and the predictor\-side operators𝐏:=𝐖p⊤​𝐖p\\mathbf\{P\}:=\\mathbf\{W\}\_\{p\}^\{\\top\}\\mathbf\{W\}\_\{p\},𝐐:=𝐖p⊤​𝐌¯2\\mathbf\{Q\}:=\\mathbf\{W\}\_\{p\}^\{\\top\}\\bar\{\\mathbf\{M\}\}\_\{2\}admit \(approximately\) a common eigenbasis on each side\. The natural quantitative proxy is the relative Frobenius commutator,

CΣ:=‖𝚺11​𝚺X​1−𝚺X​1​𝚺11‖F‖𝚺11‖F​‖𝚺X​1‖F,CP:=‖𝐏𝐐−𝐐𝐏‖F‖𝐏‖F​‖𝐐‖F,C\_\{\\Sigma\}\\;:=\\;\\frac\{\\bigl\\\|\\mathbf\{\\Sigma\}\_\{11\}\\mathbf\{\\Sigma\}\_\{X1\}\-\\mathbf\{\\Sigma\}\_\{X1\}\\mathbf\{\\Sigma\}\_\{11\}\\bigr\\\|\_\{F\}\}\{\\\|\\mathbf\{\\Sigma\}\_\{11\}\\\|\_\{F\}\\,\\\|\\mathbf\{\\Sigma\}\_\{X1\}\\\|\_\{F\}\}\\,,\\qquad C\_\{P\}\\;:=\\;\\frac\{\\\|\\mathbf\{P\}\\mathbf\{Q\}\-\\mathbf\{Q\}\\mathbf\{P\}\\\|\_\{F\}\}\{\\\|\\mathbf\{P\}\\\|\_\{F\}\\,\\\|\\mathbf\{Q\}\\\|\_\{F\}\}\\,,\(37\)which vanish exactly when joint diagonalisation holds and grow continuously as the assumption tightens or relaxes\. The scalarμ\\mu\-prediction is exact whenCΣ=CP=0C\_\{\\Sigma\}=C\_\{P\}=0; the per\-mode rank\-control corollary degrades smoothly inCΣ\+CPC\_\{\\Sigma\}\+C\_\{P\}and is recovered as soon as either side becomes small\.

### B\.2Closed\-form factorisation of the data\-side commutator

Under the masking model used in the paper,𝐱1=𝐦1⊙𝐱\\mathbf\{x\}\_\{1\}=\\mathbf\{m\}\_\{1\}\\odot\\mathbf\{x\}withm1​j​∼iid​Bernoulli​\(pm1\)m\_\{1j\}\\overset\{\\text\{iid\}\}\{\\sim\}\\mathrm\{Bernoulli\}\(p\_\{m\_\{1\}\}\), the data covariances admit the explicit decomposition

𝚺X​1=pm1​𝚺X​X,𝚺11=pm1​diag​\(𝚺X​X\)\+pm12​offdiag​\(𝚺X​X\),\\mathbf\{\\Sigma\}\_\{X1\}\\;=\\;p\_\{m\_\{1\}\}\\,\\mathbf\{\\Sigma\}\_\{XX\}\\,,\\qquad\\mathbf\{\\Sigma\}\_\{11\}\\;=\\;p\_\{m\_\{1\}\}\\,\\mathrm\{diag\}\(\\mathbf\{\\Sigma\}\_\{XX\}\)\\,\+\\,p\_\{m\_\{1\}\}^\{2\}\\,\\mathrm\{offdiag\}\(\\mathbf\{\\Sigma\}\_\{XX\}\)\\,,\(38\)wherediag⁡\(⋅\)\\mathrm\{diag\}\(\\cdot\)extracts the diagonal andoffdiag⁡\(𝐀\):=𝐀−diag⁡\(𝐀\)\\mathrm\{offdiag\}\(\\mathbf\{A\}\):=\\mathbf\{A\}\-\\mathrm\{diag\}\(\\mathbf\{A\}\)\. Substituting \([38](https://arxiv.org/html/2610.02344#A2.E38)\) into the data\-side commutator and using\[𝚺X​X,𝚺X​X\]=𝟎\[\\mathbf\{\\Sigma\}\_\{XX\},\\mathbf\{\\Sigma\}\_\{XX\}\]=\\mathbf\{0\}gives the closed\-form identity

\[𝚺11,𝚺X​1\]=pm12​\(1−pm1\)​\[diag⁡\(𝚺X​X\),𝚺X​X\]\\boxed\{\\;\\bigl\[\\mathbf\{\\Sigma\}\_\{11\},\\mathbf\{\\Sigma\}\_\{X1\}\\bigr\]\\;=\\;p\_\{m\_\{1\}\}^\{2\}\\,\(1\-p\_\{m\_\{1\}\}\)\\;\\bigl\[\\,\\mathrm\{diag\}\(\\mathbf\{\\Sigma\}\_\{XX\}\),\\,\\mathbf\{\\Sigma\}\_\{XX\}\\,\\bigr\]\\;\}\(39\)The masking\-side and data\-side dependences factorise\. Two observations follow immediately\.

##### Masking factor is mild at the operating point\.

The scalarf⁡\(p\):=p2​\(1−p\)f\(p\):=p^\{2\}\(1\-p\)is maximised atp⋆=2/3p^\{\\star\}=2/3withf⁡\(p⋆\)=4/27≈0\.148f\(p^\{\\star\}\)=4/27\\approx 0\.148, and decays forp→0p\\to 0andp→1p\\to 1\. At the main image operating pointpm1=0\.85p\_\{m\_\{1\}\}=0\.85used in the I\-JEPA experiments,f⁡\(0\.85\)=0\.108f\(0\.85\)=0\.108, i\.e\. already below3/43/4of the worst case; atpm1=0\.95p\_\{m\_\{1\}\}=0\.95it falls to0\.0450\.045\. The masking\-side multiplier therefore never amplifies the data\-side commutator and, in the limit of mild masking, drives it to zero independently of𝚺X​X\\mathbf\{\\Sigma\}\_\{XX\}\.

##### Data factor is a property of𝚺X​X\\mathbf\{\\Sigma\}\_\{XX\}alone\.

The remaining factor𝐃:=\[diag⁡\(𝚺X​X\),𝚺X​X\]\\mathbf\{D\}:=\\bigl\[\\,\\mathrm\{diag\}\(\\mathbf\{\\Sigma\}\_\{XX\}\),\\,\\mathbf\{\\Sigma\}\_\{XX\}\\,\\bigr\]depends only on how far the input covariance is from being diagonal in the canonical \(feature\-token\) basis\. It is invariant under rescaling of𝚺X​X\\mathbf\{\\Sigma\}\_\{XX\}, which means that the relative commutatorCΣC\_\{\\Sigma\}in \([37](https://arxiv.org/html/2610.02344#A2.E37)\) is determined by the*shape*of the spectrum, not its overall magnitude\.

### B\.3Empirical commutators across spectra

Using the closed\-form identity \([39](https://arxiv.org/html/2610.02344#A2.E39)\), we evaluateCΣC\_\{\\Sigma\}on five spectra spanning the regimes the framework needs to cover: an isotropic baseline \(𝚺X​X=𝐈\\mathbf\{\\Sigma\}\_\{XX\}=\\mathbf\{I\}\), a diagonal power\-law \(canonical\-basis\-aligned\), a ‘cifar\-like’ synthetic with random rotation and a power\-law spectrum, a strongly rotated rank\-decay spectrum, and the empirical patch covariance of CIFAR\-10 \(n=128,000n=128\{,\}000patches of size4×4×34\\times 4\\times 3, channel\-standardised;D=48D=48\)\. Sweepingpm1p\_\{m\_\{1\}\}from0\.050\.05to0\.990\.99in5050steps yields Figure[7](https://arxiv.org/html/2610.02344#A2.F7)\.

Figure 7:Relative data\-side commutatorCΣC\_\{\\Sigma\}as a function of context ratiopm1p\_\{m\_\{1\}\}\.Left: rawCΣC\_\{\\Sigma\}\.Right:‖\[𝚺11,𝚺X​1\]‖F/\(pm12​\(1−pm1\)\)\\\|\[\\mathbf\{\\Sigma\}\_\{11\},\\mathbf\{\\Sigma\}\_\{X1\}\]\\\|\_\{F\}/\\bigl\(p\_\{m\_\{1\}\}^\{2\}\(1\-p\_\{m\_\{1\}\}\)\\bigr\)– flat inpm1p\_\{m\_\{1\}\}\(residual<10−12<10^\{\-12\}relative\), confirming the closed\-form identity \([39](https://arxiv.org/html/2610.02344#A2.E39)\)\. The CIFAR\-10 raw\-patch curve sits well below the synthetic benchmarks at all operating points\.Table 6:Relative data\-side commutatorCΣC\_\{\\Sigma\}across spectra atpm1=0\.85p\_\{m\_\{1\}\}=0\.85, and the maximum overpm1∈\[0\.05,0\.99\]p\_\{m\_\{1\}\}\\in\[0\.05,0\.99\]\.The masking factorf⁡\(0\.85\)=0\.108f\(0\.85\)=0\.108\. Real CIFAR\-10 raw\-patch covariance givesCΣC\_\{\\Sigma\}on the order of10−410^\{\-4\}, two to three orders of magnitude tighter than rotated power\-law surrogates\.Three points stand out from Table[6](https://arxiv.org/html/2610.02344#A2.T6)and Figure[7](https://arxiv.org/html/2610.02344#A2.F7)\.

1. 1\.Real\-image regime is well inside the decoupled limit\.On real CIFAR\-10 raw patches at the main operating point,CΣ≈10−4C\_\{\\Sigma\}\\approx 10^\{\-4\}\. Pixel\-level correlation structure aligns the dominant directions of𝚺X​X\\mathbf\{\\Sigma\}\_\{XX\}with the canonical basis closely enough that𝐃\\mathbf\{D\}is small in absolute terms, even though the off\-diagonal energy of𝚺X​X\\mathbf\{\\Sigma\}\_\{XX\}itself is substantial \(the off\-diagonal\-to\-total Frobenius ratio is0\.980\.98\)\. Assumption[3\.2](https://arxiv.org/html/2610.02344#S3.Thmtheorem2)therefore holds at the10−410^\{\-4\}level on this benchmark, much tighter than typical ‘approximate’ assumptions in linear analyses\.
2. 2\.Synthetic surrogates trace the boundary of the regime\.Rotated power\-law covariances giveCΣC\_\{\\Sigma\}in the10−310^\{\-3\}–10−210^\{\-2\}band, defining a soft envelope above which the per\-mode story would need to be re\-cast in the eigenbasis of𝐃\\mathbf\{D\}rather than the canonical one\.
3. 3\.The masking knob always works in our favour\.Because of thepm12​\(1−pm1\)p\_\{m\_\{1\}\}^\{2\}\(1\-p\_\{m\_\{1\}\}\)factor, increasing the context retention probability towards11drivesCΣC\_\{\\Sigma\}to zero at a quadratic rate – regardless of𝚺X​X\\mathbf\{\\Sigma\}\_\{XX\}\. The high\-ctx regime studied in the main paper is therefore the regime in which the assumption is sharpest\.

### B\.4Tabular benchmarks:aloi,helena,jannis

The same closed\-form identity applies feature\-wise to the three tabular benchmarks, so the assumption can be checked before training; we measure the violation on the exact features the sweep trains on, after the min–max preprocessing used throughout the paper\.

The commutator alone cannot single outjannis:helenacarries a near\-identical violation and is predicted well\. What distinguishesjannisis spectral degeneracy: its minimum relative gap among the ten largest eigenvalues of𝚺X​X\\mathbf\{\\Sigma\}\_\{XX\}is roughly fifteen times smaller thanhelena’s\. This predicts the observed signature, in which per\-mode identification degrades while the count\-level prediction survives: onjannisthe per\-mode correlation falls to\+0\.619\+0\.619, against\+0\.906\+0\.906onhelenaand\+0\.905\+0\.905onaloi, while the signed and loose\-count correlations stay at\+0\.648\+0\.648and\+0\.697\+0\.697with positive confidence intervals \(Table[7](https://arxiv.org/html/2610.02344#A3.T7)\)\. With three datasets we state this as a measured, mechanism\-consistent hypothesis and not as a fitted law\.

##### Incorporating the interaction\.

Write the linearised generator as𝒥=𝐃\+ϵ​𝐑\\mathcal\{J\}=\\mathbf\{D\}\+\\epsilon\\mathbf\{R\}, with𝐃\\mathbf\{D\}decoupled and the commutator of Eq\. \([37](https://arxiv.org/html/2610.02344#A2.E37)\) as the measurable small parameterϵ\\epsilon\. First\-order terms shift eachμi\\mu\_\{i\}through the diagonal of𝐑\\mathbf\{R\}, and modes mix through inverse\-eigengap\-weighted off\-diagonals\. Near\-degenerate blocks are therefore where interaction first affects mode identity, which is thejanniscase\. Low\-dimensional projectors with multiple views, as in[Ghosh et al\. \[2024\]](https://arxiv.org/html/2610.02344#bib.bib16), act on exactly that quantity, since they limit how many modes are simultaneously active and thereby shrink the effective cross\-talk; their setting is projector\-based vision SSL rather than JEPA, so we take the mechanism and not the system\. We do not claim the coupled derivation here\.

## Appendix CFull proof of the rank\-control corollary \(Corollary[3\.3](https://arxiv.org/html/2610.02344#S3.Thmtheorem3)\)

We prove the corollary in five steps; Appendix[C\.6](https://arxiv.org/html/2610.02344#A3.SS6)gives the finite\-time statements\.

### C\.1Step 1 – Joint diagonalisation in the shared basis

Let𝐔=\[𝐮1⋯𝐮N\]∈ℝN×N\\mathbf\{U\}=\[\\mathbf\{u\}\_\{1\}\\,\\cdots\\,\\mathbf\{u\}\_\{N\}\]\\in\\mathbb\{R\}^\{N\\times N\}and𝐕=\[𝐯1⋯𝐯d\]∈ℝd×d\\mathbf\{V\}=\[\\mathbf\{v\}\_\{1\}\\,\\cdots\\,\\mathbf\{v\}\_\{d\}\]\\in\\mathbb\{R\}^\{d\\times d\}be orthonormal eigenbases as in the exact form of Assumption[3\.2](https://arxiv.org/html/2610.02344#S3.Thmtheorem2), taken with real spectra as in Corollary[3\.3](https://arxiv.org/html/2610.02344#S3.Thmtheorem3)\(equivalently,𝚺X​1\\mathbf\{\\Sigma\}\_\{X1\}and𝐖p⊤​𝐌¯2\\mathbf\{W\}\_\{p\}^\{\\top\}\\bar\{\\mathbf\{M\}\}\_\{2\}symmetric\), so that

𝚺11\\displaystyle\\mathbf\{\\Sigma\}\_\{11\}=𝐔​diag​\(ak\)​𝐔⊤,\\displaystyle=\\mathbf\{U\}\\,\\mathrm\{diag\}\(a\_\{k\}\)\\,\\mathbf\{U\}^\{\\top\},𝚺X​1\\displaystyle\\quad\\mathbf\{\\Sigma\}\_\{X1\}=𝐔​diag​\(ck\)​𝐔⊤,\\displaystyle=\\mathbf\{U\}\\,\\mathrm\{diag\}\(c\_\{k\}\)\\,\\mathbf\{U\}^\{\\top\},𝐖p⊤​𝐖p\\displaystyle\\mathbf\{W\}\_\{p\}^\{\\top\}\\mathbf\{W\}\_\{p\}=𝐕​diag​\(pℓ\)​𝐕⊤,\\displaystyle=\\mathbf\{V\}\\,\\mathrm\{diag\}\(p\_\{\\ell\}\)\\,\\mathbf\{V\}^\{\\top\},𝐖p⊤​𝐌¯2\\displaystyle\\quad\\mathbf\{W\}\_\{p\}^\{\\top\}\\bar\{\\mathbf\{M\}\}\_\{2\}=𝐕​diag​\(qℓ\)​𝐕⊤\.\\displaystyle=\\mathbf\{V\}\\,\\mathrm\{diag\}\(q\_\{\\ell\}\)\\,\\mathbf\{V\}^\{\\top\}\.Define𝐖~:=𝐕⊤​𝐖𝐔\\widetilde\{\\mathbf\{W\}\}:=\\mathbf\{V\}^\{\\top\}\\mathbf\{W\}\\mathbf\{U\}and𝐖~a:=𝐕⊤​𝐖a​𝐔\\widetilde\{\\mathbf\{W\}\}\_\{a\}:=\\mathbf\{V\}^\{\\top\}\\mathbf\{W\}\_\{a\}\\mathbf\{U\}\. Substituting in the linearised flow \(Eq\. \([2](https://arxiv.org/html/2610.02344#S3.E2)\)\),

𝐖~˙\\displaystyle\\dot\{\\widetilde\{\\mathbf\{W\}\}\}=−diag⁡\(pℓ\)​𝐖~​diag​\(ak\)\+diag⁡\(qℓ\)​𝐖~a​diag​\(ck\),\\displaystyle=\-\\mathrm\{diag\}\(p\_\{\\ell\}\)\\,\\widetilde\{\\mathbf\{W\}\}\\,\\mathrm\{diag\}\(a\_\{k\}\)\+\\mathrm\{diag\}\(q\_\{\\ell\}\)\\,\\widetilde\{\\mathbf\{W\}\}\_\{a\}\\,\\mathrm\{diag\}\(c\_\{k\}\),\(40\)𝐖~˙a\\displaystyle\\dot\{\\widetilde\{\\mathbf\{W\}\}\}\_\{a\}=β⁡\(𝐖~−𝐖~a\)\.\\displaystyle=\\beta\(\\widetilde\{\\mathbf\{W\}\}\-\\widetilde\{\\mathbf\{W\}\}\_\{a\}\)\.\(41\)The\(ℓ,k\)\(\\ell,k\)component of \([40](https://arxiv.org/html/2610.02344#A3.E40)\)–\([41](https://arxiv.org/html/2610.02344#A3.E41)\) depends only on the\(ℓ,k\)\(\\ell,k\)entries\. Writingwℓ​k:=\[𝐖~\]ℓ​kw\_\{\\ell k\}:=\[\\widetilde\{\\mathbf\{W\}\}\]\_\{\\ell k\}andwℓ​ka:=\[𝐖~a\]ℓ​kw^\{a\}\_\{\\ell k\}:=\[\\widetilde\{\\mathbf\{W\}\}\_\{a\}\]\_\{\\ell k\},

\[w˙ℓ​kw˙ℓ​ka\]=\[−ak​pℓck​qℓβ−β\]⏟=:𝐉k​ℓ\[wℓ​kwℓ​ka\],\\begin\{bmatrix\}\\dot\{w\}\_\{\\ell k\}\\\\ \\dot\{w\}^\{a\}\_\{\\ell k\}\\end\{bmatrix\}=\\underbrace\{\\begin\{bmatrix\}\-a\_\{k\}p\_\{\\ell\}&c\_\{k\}q\_\{\\ell\}\\\\ \\beta&\-\\beta\\end\{bmatrix\}\}\_\{=:\\,\\mathbf\{J\}\_\{k\\ell\}\}\\begin\{bmatrix\}w\_\{\\ell k\}\\\\ w^\{a\}\_\{\\ell k\}\\end\{bmatrix\},\(42\)so the2​d​N2dN\-dimensional flow splits intod​NdNindependent planar flows\.

### C\.2Step 2 – The2×22\\times 2spectrum and saddle condition

The characteristic polynomial of𝐉k​ℓ\\mathbf\{J\}\_\{k\\ell\}isλ2\+\(β\+ak​pℓ\)​λ\+β​ak​pℓ​\(1−μk​ℓ\)=0\\lambda^\{2\}\+\(\\beta\+a\_\{k\}p\_\{\\ell\}\)\\lambda\+\\beta a\_\{k\}p\_\{\\ell\}\(1\-\\mu\_\{k\\ell\}\)=0, withμk​ℓ:=\(ck​qℓ\)/\(ak​pℓ\)\\mu\_\{k\\ell\}:=\(c\_\{k\}q\_\{\\ell\}\)/\(a\_\{k\}p\_\{\\ell\}\), and its discriminantΔk​ℓ=\(β−ak​pℓ\)2\+4​β​ck​qℓ\\Delta\_\{k\\ell\}=\(\\beta\-a\_\{k\}p\_\{\\ell\}\)^\{2\}\+4\\beta c\_\{k\}q\_\{\\ell\}is non\-negative wheneverck​qℓ≥0c\_\{k\}q\_\{\\ell\}\\geq 0\. The eigenvalues areλk​ℓ±=12​\(−\(β\+ak​pℓ\)±Δk​ℓ\)\\lambda^\{\\pm\}\_\{k\\ell\}=\\tfrac\{1\}\{2\}\\big\(\-\(\\beta\+a\_\{k\}p\_\{\\ell\}\)\\pm\\sqrt\{\\Delta\_\{k\\ell\}\}\\big\), with productβ​σk​ℓ​\(1−μk​ℓ\)\\beta\\sigma\_\{k\\ell\}\(1\-\\mu\_\{k\\ell\}\), so

μk​ℓ\>1⇔λk​ℓ\+​λk​ℓ−<0⇔𝐉k​ℓ​is a saddle\.\\mu\_\{k\\ell\}\>1\\iff\\lambda^\{\+\}\_\{k\\ell\}\\lambda^\{\-\}\_\{k\\ell\}<0\\iff\\mathbf\{J\}\_\{k\\ell\}\\text\{ is a saddle\.\}Super\-critical modes haveλk​ℓ\+\>0\>λk​ℓ−\\lambda^\{\+\}\_\{k\\ell\}\>0\>\\lambda^\{\-\}\_\{k\\ell\}; for sub\-critical modes both eigenvalues have negative real parts\.

### C\.3Step 3 – Mode\-wise unstable manifolds

For a saddle block the unstable eigenvector is𝐞k​ℓ\+=\(ck​qℓ,λk​ℓ\+\+ak​pℓ\)⊤\\mathbf\{e\}^\{\+\}\_\{k\\ell\}=\(c\_\{k\}q\_\{\\ell\},\\,\\lambda^\{\+\}\_\{k\\ell\}\+a\_\{k\}p\_\{\\ell\}\)^\{\\top\}, and𝐞k​ℓ−\\mathbf\{e\}^\{\-\}\_\{k\\ell\}is its analogue withλk​ℓ−\\lambda^\{\-\}\_\{k\\ell\}\. Every solution is

\[wℓ​k​\(t\)wℓ​ka​\(t\)\]=αk​ℓ\+​eλk​ℓ\+​t​𝐞k​ℓ\+\+αk​ℓ−​eλk​ℓ−​t​𝐞k​ℓ−,\\begin\{bmatrix\}w\_\{\\ell k\}\(t\)\\\\ w^\{a\}\_\{\\ell k\}\(t\)\\end\{bmatrix\}=\\alpha^\{\+\}\_\{k\\ell\}e^\{\\lambda^\{\+\}\_\{k\\ell\}t\}\\mathbf\{e\}^\{\+\}\_\{k\\ell\}\+\\alpha^\{\-\}\_\{k\\ell\}e^\{\\lambda^\{\-\}\_\{k\\ell\}t\}\\mathbf\{e\}^\{\-\}\_\{k\\ell\},sowℓ​k​\(t\)≃αk​ℓ\+​\(ck​qℓ\)​eλk​ℓ\+​tw\_\{\\ell k\}\(t\)\\simeq\\alpha^\{\+\}\_\{k\\ell\}\(c\_\{k\}q\_\{\\ell\}\)\\,e^\{\\lambda^\{\+\}\_\{k\\ell\}t\}for\(k,ℓ\)∈𝒰\(k,\\ell\)\\in\\mathcal\{U\}whenαk​ℓ\+≠0\\alpha^\{\+\}\_\{k\\ell\}\\neq 0, andwℓ​k​\(t\)→0w\_\{\\ell k\}\(t\)\\to 0for sub\-critical modes\.

### C\.4Step 4 – Reassembly into the original basis

Since𝐖⁡\(t\)=𝐕​𝐖~​\(t\)​𝐔⊤=∑ℓ,kwℓ​k​\(t\)​𝐯ℓ​𝐮k⊤\\mathbf\{W\}\(t\)=\\mathbf\{V\}\\widetilde\{\\mathbf\{W\}\}\(t\)\\mathbf\{U\}^\{\\top\}=\\sum\_\{\\ell,k\}w\_\{\\ell k\}\(t\)\\mathbf\{v\}\_\{\\ell\}\\mathbf\{u\}\_\{k\}^\{\\top\}, Step 3 gives

𝐖⁡\(t\)≃∑\(k,ℓ\)∈𝒰ak​ℓ​eλk​ℓ\+​t​𝐯ℓ​𝐮k⊤,ak​ℓ=αk​ℓ\+​ck​qℓ\.\\mathbf\{W\}\(t\)\\simeq\\sum\_\{\(k,\\ell\)\\in\\mathcal\{U\}\}a\_\{k\\ell\}\\,e^\{\\lambda^\{\+\}\_\{k\\ell\}t\}\\,\\mathbf\{v\}\_\{\\ell\}\\mathbf\{u\}\_\{k\}^\{\\top\},\\qquad a\_\{k\\ell\}=\\alpha^\{\+\}\_\{k\\ell\}c\_\{k\}q\_\{\\ell\}\.\(43\)

### C\.5Step 5 – The rank bound \(unstable\-component statement\)

Split𝐖⁡\(t\)=𝐖\+​\(t\)\+𝐖−​\(t\)\\mathbf\{W\}\(t\)=\\mathbf\{W\}^\{\+\}\(t\)\+\\mathbf\{W\}^\{\-\}\(t\)along the unstable and stable invariant subspaces of the block\-diagonal Jacobian\. Eq\. \([43](https://arxiv.org/html/2610.02344#A3.E43)\) is exact for𝐖\+​\(t\)\\mathbf\{W\}^\{\+\}\(t\)\(Lemma[C\.7](https://arxiv.org/html/2610.02344#A3.Thmtheorem7)\), and𝐖−​\(t\)\\mathbf\{W\}^\{\-\}\(t\)decays at the slowest stable rateη\\etaof Eq\. \([45](https://arxiv.org/html/2610.02344#A3.E45)\)\. For a generic initialisation𝐖−​\(t\)≠0\\mathbf\{W\}^\{\-\}\(t\)\\neq 0, so the algebraic rank of𝐖⁡\(t\)\\mathbf\{W\}\(t\)is generically full at every finitett; the corollary therefore bounds𝐖\+​\(t\)\\mathbf\{W\}^\{\+\}\(t\)\.

Grouping the terms of𝐖\+​\(t\)\\mathbf\{W\}^\{\+\}\(t\)by predictor index writes it as a sum of\|πℓ​𝒰\|\|\\pi\_\{\\ell\}\\mathcal\{U\}\|rank\-one matrices𝐯ℓ​𝐫ℓ​\(t\)⊤\\mathbf\{v\}\_\{\\ell\}\\mathbf\{r\}\_\{\\ell\}\(t\)^\{\\top\}, whereπℓ\(𝒰\)=\{ℓ:∃k,\(k,ℓ\)∈𝒰\}\\pi\_\{\\ell\}\(\\mathcal\{U\}\)=\\\{\\ell:\\exists k,\(k,\\ell\)\\in\\mathcal\{U\}\\\}; grouping by data index gives\|πk​𝒰\|\|\\pi\_\{k\}\\mathcal\{U\}\|terms\. Hence

rank⁡\(𝐖\+​\(t\)\)≤rtight:=min⁡\(\|πℓ​𝒰\|,\|πk​𝒰\|,d,N\)≤min⁡\(\|𝒰\|,d,N\),\\rank\(\\mathbf\{W\}^\{\+\}\(t\)\)\\;\\leq\\;r\_\{\\mathrm\{tight\}\}\\;:=\\;\\min\\big\(\|\\pi\_\{\\ell\}\\mathcal\{U\}\|,\\,\|\\pi\_\{k\}\\mathcal\{U\}\|,\\,d,\\,N\\big\)\\;\\leq\\;\\min\(\|\\mathcal\{U\}\|,d,N\),\(44\)which is Eq\. \([8](https://arxiv.org/html/2610.02344#S3.E8)\) of the main text\. The experiments measureERank\\Erankof the embeddings rather than an algebraic rank; Theorems[C\.3](https://arxiv.org/html/2610.02344#A3.Thmtheorem3)and[C\.11](https://arxiv.org/html/2610.02344#A3.Thmtheorem11)give the corresponding finite\-time bounds for𝐖⁡\(t\)\\mathbf\{W\}\(t\)and for the embeddings\.□\\square

### C\.6Finite\-time restatement of the rank budget

With𝐖p\\mathbf\{W\}\_\{p\}held fixed, the flow of Eq\. \([2](https://arxiv.org/html/2610.02344#S3.E2)\) is linear in\(𝐖,𝐖a\)\(\\mathbf\{W\},\\mathbf\{W\}\_\{a\}\), so the statements below hold for alltt\. The assumptions are: \(A1\) the fixed\-predictor flow withβ\>0\\beta\>0; \(A2\) the exact form of Assumption[3\.2](https://arxiv.org/html/2610.02344#S3.Thmtheorem2)with orthogonal𝐔\\mathbf\{U\},𝐕\\mathbf\{V\}and real spectra, as in Step 1; \(A3\) non\-degeneracy, namelyak\>0a\_\{k\}\>0andpℓ\>0p\_\{\\ell\}\>0,μk​ℓ≠1\\mu\_\{k\\ell\}\\neq 1andΔk​ℓ≠0\\Delta\_\{k\\ell\}\\neq 0for every mode, whereΔk​ℓ=\(β−σk​ℓ\)2\+4​β​γk​ℓ\\Delta\_\{k\\ell\}=\(\\beta\-\\sigma\_\{k\\ell\}\)^\{2\}\+4\\beta\\gamma\_\{k\\ell\}\(the last two fail only on a measure\-zero set\)\. The approximate version of the assumption, with small commutators, is not covered here\.

##### Notation\.

m:=min⁡\(d,N\)m:=\\min\(d,N\); in this sectionr:=r𝑡𝑖𝑔ℎ𝑡r:=r\_\{\\mathit\{tight\}\}of Eq\. \([8](https://arxiv.org/html/2610.02344#S3.E8)\), not the loose count\|𝒰\|\|\\mathcal\{U\}\|of Step 5;rankε⁡\(𝐌\)\\operatorname\{rank\}\_\{\\varepsilon\}\(\\mathbf\{M\}\)counts the singular values of𝐌\\mathbf\{M\}strictly aboveε\\varepsilon;ERank\\Erankis the effective rank of Appendix[H](https://arxiv.org/html/2610.02344#A8)\. The decay rate is

η:=min\{\|Reλ\|:λ∈Spec\(𝒥\),Reλ<0\}\>0,\\eta:=\\min\\\{\|\\Real\\lambda\|:\\lambda\\in\\Spec\(\\mathcal\{J\}\),\\ \\Real\\lambda<0\\\}\>0,\(45\)the slowest stable rate: per mode,\|Re⁡λk​ℓ\+\|\|\\Real\\lambda^\{\+\}\_\{k\\ell\}\|on subcritical modes and\|λk​ℓ−\|\|\\lambda^\{\-\}\_\{k\\ell\}\|on supercritical ones\. The transient\-growth constants are

𝐍k​ℓ:=𝐉k​ℓ−12​tr⁡\(𝐉k​ℓ\)​𝐈2,Kk​ℓ:=1\+2​‖𝐍k​ℓ‖2\|Δk​ℓ\|,K¯:=max⁡\(1,maxk,ℓ⁡Kk​ℓ\),\\mathbf\{N\}\_\{k\\ell\}:=\\mathbf\{J\}\_\{k\\ell\}\-\\tfrac\{1\}\{2\}\\operatorname\{tr\}\(\\mathbf\{J\}\_\{k\\ell\}\)\\mathbf\{I\}\_\{2\},\\qquad K\_\{k\\ell\}:=1\+\\frac\{2\\\|\\mathbf\{N\}\_\{k\\ell\}\\\|\_\{2\}\}\{\\sqrt\{\|\\Delta\_\{k\\ell\}\|\}\},\\qquad\\bar\{K\}:=\\max\\bigl\(1,\\max\_\{k,\\ell\}K\_\{k\\ell\}\\bigr\),\(46\)withZ0:=\(‖𝐖⁡\(0\)‖F2\+‖𝐖a​\(0\)‖F2\)1/2Z\_\{0\}:=\(\\\|\\mathbf\{W\}\(0\)\\\|\_\{F\}^\{2\}\+\\\|\\mathbf\{W\}\_\{a\}\(0\)\\\|\_\{F\}^\{2\}\)^\{1/2\},wk​ℓ​\(0\):=𝐯ℓ⊤​𝐖​\(0\)​𝐮kw\_\{k\\ell\}\(0\):=\\mathbf\{v\}\_\{\\ell\}^\{\\top\}\\mathbf\{W\}\(0\)\\mathbf\{u\}\_\{k\},yk​ℓ​\(0\):=𝐯ℓ⊤​𝐖a​\(0\)​𝐮ky\_\{k\\ell\}\(0\):=\\mathbf\{v\}\_\{\\ell\}^\{\\top\}\\mathbf\{W\}\_\{a\}\(0\)\\mathbf\{u\}\_\{k\}, and, for\(k,ℓ\)∈𝒰\(k,\\ell\)\\in\\mathcal\{U\}, the amplitudeak​ℓa\_\{k\\ell\}of the expansion of Step 4, written in initialisation\-computable form,

ak​ℓ:=\(β\+λk​ℓ\+\)​\[β​wk​ℓ​\(0\)−\(β\+λk​ℓ−\)​yk​ℓ​\(0\)\]β⁡\(λk​ℓ\+−λk​ℓ−\)\.a\_\{k\\ell\}:=\\frac\{\(\\beta\+\\lambda^\{\+\}\_\{k\\ell\}\)\\bigl\[\\beta\\,w\_\{k\\ell\}\(0\)\-\(\\beta\+\\lambda^\{\-\}\_\{k\\ell\}\)\\,y\_\{k\\ell\}\(0\)\\bigr\]\}\{\\beta\\,\(\\lambda^\{\+\}\_\{k\\ell\}\-\\lambda^\{\-\}\_\{k\\ell\}\)\}\.\(47\)With𝐖⁡\(t\)=𝐖\+​\(t\)\+𝐖−​\(t\)\\mathbf\{W\}\(t\)=\\mathbf\{W\}\_\{\+\}\(t\)\+\\mathbf\{W\}\_\{\-\}\(t\)the split of Step 5, Lemma[C\.7](https://arxiv.org/html/2610.02344#A3.Thmtheorem7)below gives

𝐖\+​\(t\)=∑\(k,ℓ\)∈𝒰ak​ℓ​eλk​ℓ\+​t​𝐯ℓ​𝐮k⊤for all​t\.\\mathbf\{W\}\_\{\+\}\(t\)=\\sum\_\{\(k,\\ell\)\\in\\mathcal\{U\}\}a\_\{k\\ell\}\\,e^\{\\lambda^\{\+\}\_\{k\\ell\}t\}\\,\\mathbf\{v\}\_\{\\ell\}\\mathbf\{u\}\_\{k\}^\{\\top\}\\qquad\\text\{for all \}t\.\(48\)
###### Theorem C\.1\(Exact rank budget, all times\)\.

Under \(A1\)–\(A3\), for everyt≥0t\\geq 0,rank⁡𝐖\+​\(t\)≤r𝑡𝑖𝑔ℎ𝑡\\rank\\mathbf\{W\}\_\{\+\}\(t\)\\leq r\_\{\\mathit\{tight\}\}\. If some axis projection \(πℓ\\pi\_\{\\ell\}orπk\\pi\_\{k\}\) is injective on𝒰\\mathcal\{U\}and the corresponding amplitudesak​ℓa\_\{k\\ell\}are all non\-zero, the bound is attained for allt≥0t\\geq 0\.

###### Theorem C\.2\(Finite\-time thresholded rank\)\.

Under \(A1\)–\(A3\), for allt≥0t\\geq 0,σr\+1​\(𝐖⁡\(t\)\)≤‖𝐖−​\(t\)‖F≤K¯​Z0​e−η​t\\sigma\_\{r\+1\}\(\\mathbf\{W\}\(t\)\)\\leq\\\|\\mathbf\{W\}\_\{\-\}\(t\)\\\|\_\{F\}\\leq\\bar\{K\}Z\_\{0\}e^\{\-\\eta t\}, and consequently, for every thresholdε\>0\\varepsilon\>0,rankε⁡\(𝐖⁡\(t\)\)≤r𝑡𝑖𝑔ℎ𝑡\\operatorname\{rank\}\_\{\\varepsilon\}\(\\mathbf\{W\}\(t\)\)\\leq r\_\{\\mathit\{tight\}\}for allt≥T⁡\(ε\):=max⁡\{0,η−1​log⁡\(K¯​Z0/ε\)\}t\\geq T\(\\varepsilon\):=\\max\\\{0,\\eta^\{\-1\}\\log\(\\bar\{K\}Z\_\{0\}/\\varepsilon\)\\\}\.

###### Theorem C\.3\(Finite\-time effective\-rank bound\)\.

Assume \(A1\)–\(A3\),𝒰≠∅\\mathcal\{U\}\\neq\\emptyset, and some\(k∗,ℓ∗\)∈𝒰\(k^\{\\ast\},\\ell^\{\\ast\}\)\\in\\mathcal\{U\}withA0:=\|ak∗​ℓ∗\|\>0A\_\{0\}:=\|a\_\{k^\{\\ast\}\\ell^\{\\ast\}\}\|\>0, which holds almost surely for an absolutely continuous initialisation; writeλ∗:=λk∗​ℓ∗\+\\lambda\_\{\\ast\}:=\\lambda^\{\+\}\_\{k^\{\\ast\}\\ell^\{\\ast\}\}\. Ifr=mr=mthenERank⁡\(𝐖⁡\(t\)\)≤r\\Erank\(\\mathbf\{W\}\(t\)\)\\leq rtrivially\. Otherwise define

θ¯​\(t\)\\displaystyle\\bar\{\\theta\}\(t\):=2​\(m−r\)​K¯​Z0A0e−\(λ∗\+η\)​t,T0:=max\{0,1λ∗\+ηlog4​\(m−r\)​K¯​Z0A0\},\\displaystyle:=\\frac\{2\(m\-r\)\\bar\{K\}Z\_\{0\}\}\{A\_\{0\}\}\\,e^\{\-\(\\lambda\_\{\\ast\}\+\\eta\)t\},\\qquad T\_\{0\}:=\\max\\Bigl\\\{0,\\frac\{1\}\{\\lambda\_\{\\ast\}\+\\eta\}\\log\\frac\{4\(m\-r\)\\bar\{K\}Z\_\{0\}\}\{A\_\{0\}\}\\Bigr\\\},\(49\)φ⁡\(x\)\\displaystyle\\varphi\(x\):=x⁡\(1\+log⁡1x\+log\+⁡m−rr\)\.\\displaystyle:=x\\Bigl\(1\+\\log\\frac\{1\}\{x\}\+\\log\_\{\+\}\\frac\{m\-r\}\{r\}\\Bigr\)\.Then for allt≥T0t\\geq T\_\{0\},ERank⁡\(𝐖⁡\(t\)\)≤r𝑡𝑖𝑔ℎ𝑡​eφ​\(θ¯​\(t\)\)\\Erank\(\\mathbf\{W\}\(t\)\)\\leq r\_\{\\mathit\{tight\}\}\\,e^\{\\varphi\(\\bar\{\\theta\}\(t\)\)\}, and whenever additionallyφ​\(θ¯​\(t\)\)≤1\\varphi\(\\bar\{\\theta\}\(t\)\)\\leq 1,

ERank⁡\(𝐖⁡\(t\)\)≤r𝑡𝑖𝑔ℎ𝑡\+δ⁡\(t\),δ⁡\(t\):=\(e−1\)​r​φ​\(θ¯​\(t\)\)=O⁡\(\(1\+t\)​e−\(λ∗\+η\)​t\)→0\.\\Erank\(\\mathbf\{W\}\(t\)\)\\leq r\_\{\\mathit\{tight\}\}\+\\delta\(t\),\\qquad\\delta\(t\):=\(e\-1\)\\,r\\,\\varphi\(\\bar\{\\theta\}\(t\)\)=O\\bigl\(\(1\+t\)e^\{\-\(\\lambda\_\{\\ast\}\+\\eta\)t\}\\bigr\)\\to 0\.\(50\)

###### Proposition C\.4\(Transfer to the singular\-value form\)\.

With the eigenvector\-pairedμ~k​ℓ=\(\|ck\|/ak\)​\(\|qℓ\|/pℓ\)\\tilde\{\\mu\}\_\{k\\ell\}=\(\|c\_\{k\}\|/a\_\{k\}\)\(\|q\_\{\\ell\}\|/p\_\{\\ell\}\)one hasμk​ℓ≤μ~k​ℓ\\mu\_\{k\\ell\}\\leq\\tilde\{\\mu\}\_\{k\\ell\}, hence𝒰⊆𝒰~\\mathcal\{U\}\\subseteq\\widetilde\{\\mathcal\{U\}\}andr𝑡𝑖𝑔ℎ𝑡≤r~𝑡𝑖𝑔ℎ𝑡r\_\{\\mathit\{tight\}\}\\leq\\tilde\{r\}\_\{\\mathit\{tight\}\}\. Theorems[C\.1](https://arxiv.org/html/2610.02344#A3.Thmtheorem1)–[C\.3](https://arxiv.org/html/2610.02344#A3.Thmtheorem3)hold with the leadingr𝑡𝑖𝑔ℎ𝑡r\_\{\\mathit\{tight\}\}replaced byr~𝑡𝑖𝑔ℎ𝑡\\tilde\{r\}\_\{\\mathit\{tight\}\}, all other constants still computed from the signed spectra and the signedr𝑡𝑖𝑔ℎ𝑡r\_\{\\mathit\{tight\}\}\. Substitutingr~\\tilde\{r\}insideφ\\varphi,δ\\deltaorT0T\_\{0\}is not proved\.

#### Proofs

###### Lemma C\.5\(Mode decoupling and classification\)\.

Under \(A1\)–\(A3\), the coordinateswk​ℓ=𝐯ℓ⊤​𝐖𝐮kw\_\{k\\ell\}=\\mathbf\{v\}\_\{\\ell\}^\{\\top\}\\mathbf\{W\}\\mathbf\{u\}\_\{k\}andyk​ℓ=𝐯ℓ⊤​𝐖a​𝐮ky\_\{k\\ell\}=\\mathbf\{v\}\_\{\\ell\}^\{\\top\}\\mathbf\{W\}\_\{a\}\\mathbf\{u\}\_\{k\}decouple the flow intod​NdNplanar systemsζ˙k​ℓ=𝐉k​ℓ​ζk​ℓ\\dot\{\\zeta\}\_\{k\\ell\}=\\mathbf\{J\}\_\{k\\ell\}\\zeta\_\{k\\ell\}with𝐉k​ℓ\\mathbf\{J\}\_\{k\\ell\}as in Eq\. \([42](https://arxiv.org/html/2610.02344#A3.E42)\), and‖𝐖‖F2=∑k​ℓwk​ℓ2\\\|\\mathbf\{W\}\\\|\_\{F\}^\{2\}=\\sum\_\{k\\ell\}w\_\{k\\ell\}^\{2\}\. Ifμk​ℓ\>1\\mu\_\{k\\ell\}\>1thenΔk​ℓ\>\(β\+σk​ℓ\)2\>0\\Delta\_\{k\\ell\}\>\(\\beta\+\\sigma\_\{k\\ell\}\)^\{2\}\>0andλk​ℓ\+\>0\>−\(β\+σk​ℓ\)\>λk​ℓ−\\lambda^\{\+\}\_\{k\\ell\}\>0\>\-\(\\beta\+\\sigma\_\{k\\ell\}\)\>\\lambda^\{\-\}\_\{k\\ell\}\. Ifμk​ℓ<1\\mu\_\{k\\ell\}<1then both eigenvalues have negative real part; they are real iffΔk​ℓ≥0\\Delta\_\{k\\ell\}\\geq 0, and the slow stable rate is\|Re⁡λk​ℓ\+\|\|\\Real\\lambda^\{\+\}\_\{k\\ell\}\|\. Henceη\>0\\eta\>0and𝒥\\mathcal\{J\}is hyperbolic\.

###### Proof\.

The matrices𝐯ℓ​𝐮k⊤\\mathbf\{v\}\_\{\\ell\}\\mathbf\{u\}\_\{k\}^\{\\top\}are Frobenius\-orthonormal, and substituting𝐖=∑wk​ℓ​𝐯ℓ​𝐮k⊤\\mathbf\{W\}=\\sum w\_\{k\\ell\}\\mathbf\{v\}\_\{\\ell\}\\mathbf\{u\}\_\{k\}^\{\\top\}into the flow givesw˙k​ℓ=−σk​ℓ​wk​ℓ\+γk​ℓ​yk​ℓ\\dot\{w\}\_\{k\\ell\}=\-\\sigma\_\{k\\ell\}w\_\{k\\ell\}\+\\gamma\_\{k\\ell\}y\_\{k\\ell\},y˙k​ℓ=β⁡\(wk​ℓ−yk​ℓ\)\\dot\{y\}\_\{k\\ell\}=\\beta\(w\_\{k\\ell\}\-y\_\{k\\ell\}\)\. Forμ\>1\\mu\>1the constant term of the characteristic polynomial is negative, so the roots are real with opposite signs, andΔ=\(β\+σ\)2\+4​β​\(γ−σ\)\>\(β\+σ\)2\\Delta=\(\\beta\+\\sigma\)^\{2\}\+4\\beta\(\\gamma\-\\sigma\)\>\(\\beta\+\\sigma\)^\{2\}givesλ−<−\(β\+σ\)\\lambda^\{\-\}<\-\(\\beta\+\\sigma\)\. Forμ<1\\mu<1the root sum is−\(β\+σ\)<0\-\(\\beta\+\\sigma\)<0and the product is positive: both real negative, or a complex pair with real part−12​\(β\+σ\)\-\\tfrac\{1\}\{2\}\(\\beta\+\\sigma\)\. By \(A3\) no eigenvalue lies on the imaginary axis\. ∎

###### Lemma C\.6\(Planar semigroup bound\)\.

Let𝐌∈ℝ2×2\\mathbf\{M\}\\in\\mathbb\{R\}^\{2\\times 2\}have eigenvaluesλ1≠λ2\\lambda\_\{1\}\\neq\\lambda\_\{2\}, spectral abscissaaa, and𝐍:=𝐌−12​tr⁡\(𝐌\)​𝐈\\mathbf\{N\}:=\\mathbf\{M\}\-\\tfrac\{1\}\{2\}\\operatorname\{tr\}\(\\mathbf\{M\}\)\\mathbf\{I\}\. Then‖et​𝐌‖2≤\(1\+2​‖𝐍‖2/\|λ1−λ2\|\)​ea​t\\\|e^\{t\\mathbf\{M\}\}\\\|\_\{2\}\\leq\(1\+2\\\|\\mathbf\{N\}\\\|\_\{2\}/\|\\lambda\_\{1\}\-\\lambda\_\{2\}\|\)\\,e^\{at\}for allt≥0t\\geq 0\.

###### Proof\.

𝐍\\mathbf\{N\}is traceless with eigenvalues±δ\\pm\\delta,δ:=12​\(λ1−λ2\)≠0\\delta:=\\tfrac\{1\}\{2\}\(\\lambda\_\{1\}\-\\lambda\_\{2\}\)\\neq 0, so𝐍2=δ2​𝐈\\mathbf\{N\}^\{2\}=\\delta^\{2\}\\mathbf\{I\}andet​𝐍=cosh⁡\(δ​t\)​𝐈\+sinh⁡\(δ​t\)δ​𝐍e^\{t\\mathbf\{N\}\}=\\cosh\(\\delta t\)\\mathbf\{I\}\+\\tfrac\{\\sinh\(\\delta t\)\}\{\\delta\}\\mathbf\{N\}, withet​𝐌=e12​tr⁡\(𝐌\)​t​et​𝐍e^\{t\\mathbf\{M\}\}=e^\{\\frac\{1\}\{2\}\\operatorname\{tr\}\(\\mathbf\{M\}\)t\}e^\{t\\mathbf\{N\}\}\. In the real casea=12​tr⁡\(𝐌\)\+δa=\\tfrac\{1\}\{2\}\\operatorname\{tr\}\(\\mathbf\{M\}\)\+\\deltaandcosh⁡\(δ​t\)≤eδ​t\\cosh\(\\delta t\)\\leq e^\{\\delta t\},sinh⁡\(δ​t\)≤12​eδ​t\\sinh\(\\delta t\)\\leq\\tfrac\{1\}\{2\}e^\{\\delta t\}; in the complex caseδ=i​ω\\delta=i\\omega,a=12​tr⁡\(𝐌\)a=\\tfrac\{1\}\{2\}\\operatorname\{tr\}\(\\mathbf\{M\}\),\|cos⁡ω​t\|≤1\|\\cos\\omega t\|\\leq 1and\|sin⁡\(ω​t\)/ω\|≤2/\|λ1−λ2\|\|\\sin\(\\omega t\)/\\omega\|\\leq 2/\|\\lambda\_\{1\}\-\\lambda\_\{2\}\|\. ∎

For𝐌=𝐉k​ℓ\\mathbf\{M\}=\\mathbf\{J\}\_\{k\\ell\},\|λ1−λ2\|=\|Δk​ℓ\|\|\\lambda\_\{1\}\-\\lambda\_\{2\}\|=\\sqrt\{\|\\Delta\_\{k\\ell\}\|\}and the constant isKk​ℓK\_\{k\\ell\}\.

###### Lemma C\.7\(Saddle splitting with initialisation\-computable coefficients\)\.

For\(k,ℓ\)∈𝒰\(k,\\ell\)\\in\\mathcal\{U\}, the eigenvectors of𝐉k​ℓ\\mathbf\{J\}\_\{k\\ell\}aree±=\(1,b±\)⊤e\_\{\\pm\}=\(1,b\_\{\\pm\}\)^\{\\top\}withb±:=β/\(β\+λ±\)b\_\{\\pm\}:=\\beta/\(\\beta\+\\lambda^\{\\pm\}\), well defined and independent becauseχ⁡\(−β\)=\(β\+λ\+\)​\(β\+λ−\)=−β​γ<0\\chi\(\-\\beta\)=\(\\beta\+\\lambda^\{\+\}\)\(\\beta\+\\lambda^\{\-\}\)=\-\\beta\\gamma<0andb−−b\+=−Δ/γ≠0b\_\{\-\}\-b\_\{\+\}=\-\\sqrt\{\\Delta\}/\\gamma\\neq 0\. Every solution splits uniquely asζ⁡\(t\)=a​eλ\+​t​e\+\+a−​eλ−​t​e−\\zeta\(t\)=a\\,e^\{\\lambda^\{\+\}t\}e\_\{\+\}\+a^\{\-\}e^\{\\lambda^\{\-\}t\}e\_\{\-\}witha=\(b−​w​\(0\)−y⁡\(0\)\)/\(b−−b\+\)a=\(b\_\{\-\}w\(0\)\-y\(0\)\)/\(b\_\{\-\}\-b\_\{\+\}\)equal to Eq\. \([47](https://arxiv.org/html/2610.02344#A3.E47)\),a−=\(y⁡\(0\)−b\+​w​\(0\)\)/\(b−−b\+\)a^\{\-\}=\(y\(0\)\-b\_\{\+\}w\(0\)\)/\(b\_\{\-\}\-b\_\{\+\}\), and\|a−\|≤2​\(γ/Δ\)​‖ζ⁡\(0\)‖≤Kk​ℓ​‖ζ⁡\(0\)‖\|a^\{\-\}\|\\leq\\sqrt\{2\}\\,\(\\gamma/\\sqrt\{\\Delta\}\)\\,\\\|\\zeta\(0\)\\\|\\leq K\_\{k\\ell\}\\\|\\zeta\(0\)\\\|; since the first component ofe\+e\_\{\+\}is one,aais the amplitude of the𝐖\\mathbf\{W\}\-coordinate along the unstable eigenline\.

###### Proof\.

The second row of\(𝐉k​ℓ−λ​𝐈\)​e=0\(\\mathbf\{J\}\_\{k\\ell\}\-\\lambda\\mathbf\{I\}\)e=0readsβ​w=\(β\+λ\)​y\\beta w=\(\\beta\+\\lambda\)y, givinge±e\_\{\\pm\}\. Evaluating the monic characteristic polynomial at−β\-\\betagivesχ⁡\(−β\)=−β​σ​μ=−β​γ<0\\chi\(\-\\beta\)=\-\\beta\\sigma\\mu=\-\\beta\\gamma<0on𝒰\\mathcal\{U\}, and alsoχ⁡\(−β\)=\(β\+λ\+\)​\(β\+λ−\)\\chi\(\-\\beta\)=\(\\beta\+\\lambda^\{\+\}\)\(\\beta\+\\lambda^\{\-\}\), whenceb−−b\+=β\(λ\+−λ−\)/\(\(β\+λ\+\)\(β\+λ−\)\)=−Δ/γb\_\{\-\}\-b\_\{\+\}=\\beta\(\\lambda^\{\+\}\-\\lambda^\{\-\}\)/\(\(\\beta\+\\lambda^\{\+\}\)\(\\beta\+\\lambda^\{\-\}\)\)=\-\\sqrt\{\\Delta\}/\\gamma\. Inverting the2×22\\times 2system of eigenvector coordinates gives the coefficients; Cauchy–Schwarz gives\|a−\|≤1\+b\+2​‖ζ⁡\(0\)‖​γ/Δ\|a^\{\-\}\|\\leq\\sqrt\{1\+b\_\{\+\}^\{2\}\}\\,\\\|\\zeta\(0\)\\\|\\,\\gamma/\\sqrt\{\\Delta\}, withb\+∈\(0,1\)b\_\{\+\}\\in\(0,1\)sinceλ\+\>0\\lambda^\{\+\}\>0, and2​γ≤2​‖𝐍k​ℓ‖2\\sqrt\{2\}\\gamma\\leq 2\\\|\\mathbf\{N\}\_\{k\\ell\}\\\|\_\{2\}becauseγ\\gammais an entry of𝐍k​ℓ\\mathbf\{N\}\_\{k\\ell\}\. The two forms ofaaagree after multiplying numerator and denominator by\(β\+λ−\)\(\\beta\+\\lambda^\{\-\}\)and using−γ/\(β\+λ−\)=\(β\+λ\+\)/β\-\\gamma/\(\\beta\+\\lambda^\{\-\}\)=\(\\beta\+\\lambda^\{\+\}\)/\\beta; with the eigenvector normalisation of Step 3 this amplitude readsαk​ℓ\+​ck​qℓ\\alpha^\{\+\}\_\{k\\ell\}c\_\{k\}q\_\{\\ell\}, which is theak​ℓa\_\{k\\ell\}of Step 4\. ∎

With𝐖−:=𝐖−𝐖\+\\mathbf\{W\}\_\{\-\}:=\\mathbf\{W\}\-\\mathbf\{W\}\_\{\+\}, mode\-wisewk​ℓ−​\(t\)=ak​ℓ−​eλk​ℓ−​tw^\{\-\}\_\{k\\ell\}\(t\)=a^\{\-\}\_\{k\\ell\}e^\{\\lambda^\{\-\}\_\{k\\ell\}t\}for\(k,ℓ\)∈𝒰\(k,\\ell\)\\in\\mathcal\{U\}andwk​ℓ−​\(t\)=wk​ℓ​\(t\)w^\{\-\}\_\{k\\ell\}\(t\)=w\_\{k\\ell\}\(t\)otherwise\.

###### Proof of Theorem[C\.1](https://arxiv.org/html/2610.02344#A3.Thmtheorem1)\.

Every term of𝐖\+​\(t\)\\mathbf\{W\}\_\{\+\}\(t\)has column space inspan⁡\{𝐯ℓ:ℓ∈πℓ​𝒰\}\\operatorname\{span\}\\\{\\mathbf\{v\}\_\{\\ell\}:\\ell\\in\\pi\_\{\\ell\}\\mathcal\{U\}\\\}and row space inspan⁡\{𝐮k:k∈πk​𝒰\}\\operatorname\{span\}\\\{\\mathbf\{u\}\_\{k\}:k\\in\\pi\_\{k\}\\mathcal\{U\}\\\}, sorank⁡𝐖\+​\(t\)≤min⁡\(\|πℓ​𝒰\|,\|πk​𝒰\|\)≤min⁡\(d,N\)\\rank\\mathbf\{W\}\_\{\+\}\(t\)\\leq\\min\(\|\\pi\_\{\\ell\}\\mathcal\{U\}\|,\|\\pi\_\{k\}\\mathcal\{U\}\|\)\\leq\\min\(d,N\)for everytt\. Ifπℓ\\pi\_\{\\ell\}is injective on𝒰\\mathcal\{U\}, eachℓ\\elloccurs in exactly one pair\(k⁡\(ℓ\),ℓ\)\(k\(\\ell\),\\ell\)and𝐖\+​\(t\)=∑ℓ𝐯ℓ​gℓ​\(t\)⊤\\mathbf\{W\}\_\{\+\}\(t\)=\\sum\_\{\\ell\}\\mathbf\{v\}\_\{\\ell\}g\_\{\\ell\}\(t\)^\{\\top\}withgℓ​\(t\)=ak⁡\(ℓ\)​ℓ​eλ\+​t​𝐮k⁡\(ℓ\)g\_\{\\ell\}\(t\)=a\_\{k\(\\ell\)\\ell\}e^\{\\lambda^\{\+\}t\}\\mathbf\{u\}\_\{k\(\\ell\)\}; the𝐯ℓ\\mathbf\{v\}\_\{\\ell\}are orthonormal and thegℓ​\(t\)g\_\{\\ell\}\(t\)are non\-zero multiples of distinct orthonormal vectors at everytt, since exponentials never vanish, so the rank equals\|πk​𝒰\|=r𝑡𝑖𝑔ℎ𝑡\|\\pi\_\{k\}\\mathcal\{U\}\|=r\_\{\\mathit\{tight\}\}for alltt\. Symmetrically forπk\\pi\_\{k\}\. ∎

###### Proof of Theorem[C\.2](https://arxiv.org/html/2610.02344#A3.Thmtheorem2)\.

For\(k,ℓ\)∉𝒰\(k,\\ell\)\\notin\\mathcal\{U\}, Lemma[C\.5](https://arxiv.org/html/2610.02344#A3.Thmtheorem5)places the spectral abscissa of𝐉k​ℓ\\mathbf\{J\}\_\{k\\ell\}at−ηk​ℓ\-\\eta\_\{k\\ell\}withηk​ℓ≥η\\eta\_\{k\\ell\}\\geq\\eta, and Lemma[C\.6](https://arxiv.org/html/2610.02344#A3.Thmtheorem6)gives\|wk​ℓ−​\(t\)\|≤K¯​e−η​t​‖ζk​ℓ​\(0\)‖\|w^\{\-\}\_\{k\\ell\}\(t\)\|\\leq\\bar\{K\}e^\{\-\\eta t\}\\\|\\zeta\_\{k\\ell\}\(0\)\\\|; for\(k,ℓ\)∈𝒰\(k,\\ell\)\\in\\mathcal\{U\}, Lemma[C\.7](https://arxiv.org/html/2610.02344#A3.Thmtheorem7)gives\|wk​ℓ−​\(t\)\|=\|ak​ℓ−\|​eλk​ℓ−​t≤Kk​ℓ​‖ζk​ℓ​\(0\)‖​e−η​t\|w^\{\-\}\_\{k\\ell\}\(t\)\|=\|a^\{\-\}\_\{k\\ell\}\|e^\{\\lambda^\{\-\}\_\{k\\ell\}t\}\\leq K\_\{k\\ell\}\\\|\\zeta\_\{k\\ell\}\(0\)\\\|e^\{\-\\eta t\}since\|λk​ℓ−\|≥η\|\\lambda^\{\-\}\_\{k\\ell\}\|\\geq\\eta\. Thewk​ℓ−w^\{\-\}\_\{k\\ell\}are the entries of𝐕⊤​𝐖−​𝐔\\mathbf\{V\}^\{\\top\}\\mathbf\{W\}\_\{\-\}\\mathbf\{U\}with orthogonal factors, so‖𝐖−​\(t\)‖F2=∑k​ℓ\|wk​ℓ−​\(t\)\|2≤K¯2​Z02​e−2​η​t\\\|\\mathbf\{W\}\_\{\-\}\(t\)\\\|\_\{F\}^\{2\}=\\sum\_\{k\\ell\}\|w^\{\-\}\_\{k\\ell\}\(t\)\|^\{2\}\\leq\\bar\{K\}^\{2\}Z\_\{0\}^\{2\}e^\{\-2\\eta t\}\. Ifr=mr=mthe singular\-value claim is trivial\. Otherwise the singular\-value perturbation inequalityσi\+j−1​\(𝐀\+𝐁\)≤σi​\(𝐀\)\+σj​\(𝐁\)\\sigma\_\{i\+j\-1\}\(\\mathbf\{A\}\+\\mathbf\{B\}\)\\leq\\sigma\_\{i\}\(\\mathbf\{A\}\)\+\\sigma\_\{j\}\(\\mathbf\{B\}\)\[[Horn and Johnson, 1991](https://arxiv.org/html/2610.02344#bib.bib19)\]withi=r\+1i=r\+1,j=1j=1,𝐀=𝐖\+​\(t\)\\mathbf\{A\}=\\mathbf\{W\}\_\{\+\}\(t\),𝐁=𝐖−​\(t\)\\mathbf\{B\}=\\mathbf\{W\}\_\{\-\}\(t\)andσr\+1​\(𝐖\+​\(t\)\)=0\\sigma\_\{r\+1\}\(\\mathbf\{W\}\_\{\+\}\(t\)\)=0\(Theorem[C\.1](https://arxiv.org/html/2610.02344#A3.Thmtheorem1)\) givesσr\+1​\(𝐖⁡\(t\)\)≤‖𝐖−​\(t\)‖2≤‖𝐖−​\(t\)‖F\\sigma\_\{r\+1\}\(\\mathbf\{W\}\(t\)\)\\leq\\\|\\mathbf\{W\}\_\{\-\}\(t\)\\\|\_\{2\}\\leq\\\|\\mathbf\{W\}\_\{\-\}\(t\)\\\|\_\{F\}\. Fort≥T⁡\(ε\)t\\geq T\(\\varepsilon\),σr\+1​\(𝐖⁡\(t\)\)≤ε\\sigma\_\{r\+1\}\(\\mathbf\{W\}\(t\)\)\\leq\\varepsilon, and singular values are non\-increasing, so at mostrrof them exceedε\\varepsilon\. ∎

###### Proof of Theorem[C\.3](https://arxiv.org/html/2610.02344#A3.Thmtheorem3)\.

Assumer<mr<mand writeϱ⁡\(t\):=K¯​Z0​e−η​t\\varrho\(t\):=\\bar\{K\}Z\_\{0\}e^\{\-\\eta t\}andσi:=σi​\(𝐖⁡\(t\)\)\\sigma\_\{i\}:=\\sigma\_\{i\}\(\\mathbf\{W\}\(t\)\)\.*Head\.*Since𝐮k∗\\mathbf\{u\}\_\{k^\{\\ast\}\}and𝐯ℓ∗\\mathbf\{v\}\_\{\\ell^\{\\ast\}\}are unit vectors,σ1≥\|𝐯ℓ∗⊤​𝐖​\(t\)​𝐮k∗\|≥A0​eλ∗​t−ϱ⁡\(t\)\\sigma\_\{1\}\\geq\|\\mathbf\{v\}\_\{\\ell^\{\\ast\}\}^\{\\top\}\\mathbf\{W\}\(t\)\\mathbf\{u\}\_\{k^\{\\ast\}\}\|\\geq A\_\{0\}e^\{\\lambda\_\{\\ast\}t\}\-\\varrho\(t\)by Lemma[C\.7](https://arxiv.org/html/2610.02344#A3.Thmtheorem7); fort≥T0t\\geq T\_\{0\},θ¯​\(t\)≤12\\bar\{\\theta\}\(t\)\\leq\\tfrac\{1\}\{2\}givesϱ⁡\(t\)≤14​A0​eλ∗​t\\varrho\(t\)\\leq\\tfrac\{1\}\{4\}A\_\{0\}e^\{\\lambda\_\{\\ast\}t\}, henceσ1≥12​A0​eλ∗​t\>0\\sigma\_\{1\}\\geq\\tfrac\{1\}\{2\}A\_\{0\}e^\{\\lambda\_\{\\ast\}t\}\>0\.*Tail\.*By Theorem[C\.2](https://arxiv.org/html/2610.02344#A3.Thmtheorem2),∑i\>rσi≤\(m−r\)​ϱ​\(t\)\\sum\_\{i\>r\}\\sigma\_\{i\}\\leq\(m\-r\)\\varrho\(t\), so the tail massθ⁡\(t\):=∑i\>rσi/∑iσi\\theta\(t\):=\\sum\_\{i\>r\}\\sigma\_\{i\}/\\sum\_\{i\}\\sigma\_\{i\}satisfiesθ⁡\(t\)≤θ¯​\(t\)≤12\\theta\(t\)\\leq\\bar\{\\theta\}\(t\)\\leq\\tfrac\{1\}\{2\}\.*Entropy grouping\.*Ifθ=0\\theta=0thenERank≤r\\Erank\\leq rdirectly\. Otherwise group the normalised spectrum into head and tail blocks of masses\(1−θ,θ\)\(1\-\\theta,\\theta\); the chain rule for entropy and the maximum\-entropy bound on each block giveH⁡\(p\)≤Hb​\(θ\)\+\(1−θ\)​log⁡r\+θ​log⁡\(m−r\)H\(p\)\\leq H\_\{b\}\(\\theta\)\+\(1\-\\theta\)\\log r\+\\theta\\log\(m\-r\), soERank≤r​eu⁡\(θ\)\\Erank\\leq r\\,e^\{u\(\\theta\)\}withu⁡\(θ\):=Hb​\(θ\)\+θ​log⁡m−rru\(\\theta\):=H\_\{b\}\(\\theta\)\+\\theta\\log\\frac\{m\-r\}\{r\}\[[Cover and Thomas, 2006](https://arxiv.org/html/2610.02344#bib.bib9)\]\.*Envelope\.*On\(0,1\]\(0,1\],−\(1−θ\)​log⁡\(1−θ\)≤θ\-\(1\-\\theta\)\\log\(1\-\\theta\)\\leq\\thetagivesHb​\(θ\)≤θ⁡\(1−log⁡θ\)H\_\{b\}\(\\theta\)\\leq\\theta\(1\-\\log\\theta\)andu≤φu\\leq\\varphi;φ′​\(x\)=log⁡1x\+log\+⁡m−rr≥0\\varphi^\{\\prime\}\(x\)=\\log\\frac\{1\}\{x\}\+\\log\_\{\+\}\\frac\{m\-r\}\{r\}\\geq 0on\(0,1\]\(0,1\], soφ\\varphiis non\-decreasing andu​\(θ\)≤φ​\(θ¯​\(t\)\)u\(\\theta\)\\leq\\varphi\(\\bar\{\\theta\}\(t\)\)\.*Chord\.*ex≤1\+\(e−1\)​xe^\{x\}\\leq 1\+\(e\-1\)xon\[0,1\]\[0,1\], which gives the additive form whenφ​\(θ¯​\(t\)\)≤1\\varphi\(\\bar\{\\theta\}\(t\)\)\\leq 1\.*Rate\.*Withθ¯=A​e−b​t\\bar\{\\theta\}=Ae^\{\-bt\},b=λ∗\+ηb=\\lambda\_\{\\ast\}\+\\eta,δ⁡\(t\)=\(e−1\)​r​A​e−b​t​\(1\+b​t−log⁡A\+log\+⁡m−rr\)\\delta\(t\)=\(e\-1\)rAe^\{\-bt\}\(1\+bt\-\\log A\+\\log\_\{\+\}\\frac\{m\-r\}\{r\}\), which isO⁡\(\(1\+t\)​e−b​t\)O\(\(1\+t\)e^\{\-bt\}\)\. ∎

###### Proof of Proposition[C\.4](https://arxiv.org/html/2610.02344#A3.Thmtheorem4)\.

μk​ℓ=ck​qℓ/\(ak​pℓ\)≤\|ck\|​\|qℓ\|/\(ak​pℓ\)=μ~k​ℓ\\mu\_\{k\\ell\}=c\_\{k\}q\_\{\\ell\}/\(a\_\{k\}p\_\{\\ell\}\)\\leq\|c\_\{k\}\|\|q\_\{\\ell\}\|/\(a\_\{k\}p\_\{\\ell\}\)=\\tilde\{\\mu\}\_\{k\\ell\}, so𝒰⊆𝒰~\\mathcal\{U\}\\subseteq\\widetilde\{\\mathcal\{U\}\}and the projections are nested, givingr𝑡𝑖𝑔ℎ𝑡≤r~𝑡𝑖𝑔ℎ𝑡r\_\{\\mathit\{tight\}\}\\leq\\tilde\{r\}\_\{\\mathit\{tight\}\}\. All three theorems are upper bounds monotone in the leadingrr, so they hold withr~𝑡𝑖𝑔ℎ𝑡\\tilde\{r\}\_\{\\mathit\{tight\}\}substituted in that position only\. ∎

These statements concern the linear flow with a fixed predictor, in which the fastest escaping mode eventually dominates and the effective rank relaxes toward one \(Appendix[I\.8](https://arxiv.org/html/2610.02344#A9.SS8)\);r𝑡𝑖𝑔ℎ𝑡r\_\{\\mathit\{tight\}\}is therefore an upper bound and not a prediction ofERank\\Erank\. The case of a trained predictor and a non\-linear encoder is measured in Appendix[I\.5](https://arxiv.org/html/2610.02344#A9.SS5)\.

### C\.7Singular\-value formulation and per\-dataset results

#### Setup

The sweep is a3×9×10×33\{\\times\}9\{\\times\}10\{\\times\}3grid \(datasets∈\{\\in\\\{aloi,helena,jannis\}\\\};99predictor scalesα∈\{10−4,10−3,10−2,10−1,0\.5,1,2,5,10\}\\alpha\\in\\\{10^\{\-4\},10^\{\-3\},10^\{\-2\},10^\{\-1\},0\.5,1,2,5,10\\\};1010context ratiospm1∈\{10−3,10−2,0\.1,0\.2,…,0\.8\}p\_\{m\_\{1\}\}\\in\\\{10^\{\-3\},10^\{\-2\},0\.1,0\.2,\\ldots,0\.8\\\};33seeds\),810810cells,804804completed\. Linear T\-JEPA,4040epochs, AdamW, embed\_dimd=64d\{=\}64, no residual predictor; final\-checkpoint values reported\.

#### Singular\-value formulationμ~k​ℓ\\tilde\{\\mu\}\_\{k\\ell\}

The ratioμk​ℓ=\(ck/ak\)​\(qℓ/pℓ\)\\mu\_\{k\\ell\}=\(c\_\{k\}/a\_\{k\}\)\(q\_\{\\ell\}/p\_\{\\ell\}\)needs𝚺X​1\\mathbf\{\\Sigma\}\_\{X1\}and𝐖p⊤​𝐌¯2\\mathbf\{W\}\_\{p\}^\{\\top\}\\bar\{\\mathbf\{M\}\}\_\{2\}to have real spectra; in generalμk​ℓ\\mu\_\{k\\ell\}may be negative or complex\. The eigenvector\-paired absolute\-value form

μ~k​ℓ:=\|ck\|ak⋅\|qℓ\|pℓ≥0\\tilde\{\\mu\}\_\{k\\ell\}\\;:=\\;\\frac\{\|c\_\{k\}\|\}\{a\_\{k\}\}\\cdot\\frac\{\|q\_\{\\ell\}\|\}\{p\_\{\\ell\}\}\\;\\geq 0\(51\)is real and non\-negative and equalsμk​ℓ\\mu\_\{k\\ell\}whenck,qℓ\>0c\_\{k\},q\_\{\\ell\}\>0\. With𝒰~=\{\(k,ℓ\):μ~k​ℓ\>1\}\\widetilde\{\\mathcal\{U\}\}=\\\{\(k,\\ell\):\\tilde\{\\mu\}\_\{k\\ell\}\{\>\}1\\\},

r~𝑡𝑖𝑔ℎ𝑡:=min⁡\(\|πℓ​𝒰~\|,\|πk​𝒰~\|,d,N\)\.\\tilde\{r\}\_\{\\mathit\{tight\}\}\\;:=\\;\\min\\\!\\big\(\|\\pi\_\{\\ell\}\\widetilde\{\\mathcal\{U\}\}\|,\\;\|\\pi\_\{k\}\\widetilde\{\\mathcal\{U\}\}\|,\\;d,\\;N\\big\)\.\(52\)Sinceμk​ℓ≤μ~k​ℓ\\mu\_\{k\\ell\}\\leq\\tilde\{\\mu\}\_\{k\\ell\},r𝑡𝑖𝑔ℎ𝑡≤r~𝑡𝑖𝑔ℎ𝑡r\_\{\\mathit\{tight\}\}\\leq\\tilde\{r\}\_\{\\mathit\{tight\}\}, and the bounds of Cor\.[3\.3](https://arxiv.org/html/2610.02344#S3.Thmtheorem3)hold withr~𝑡𝑖𝑔ℎ𝑡\\tilde\{r\}\_\{\\mathit\{tight\}\}in the leading position \(Proposition[C\.4](https://arxiv.org/html/2610.02344#A3.Thmtheorem4)\)\.

Table 7:Per\-dataset Spearmanρ⁡\(ERank,⋅\)\\rho\(\\Erank,\\cdot\)for the absolute\-value and signed formulations and the loose counts, with10001000\-resample bootstrap95%95\\%CIs\.

### C\.8Signed versus absolute\-value formulation

The two formulations coincide whenckc\_\{k\}andqℓq\_\{\\ell\}are positive on every active mode\. Otherwise near\-zero eigenvalues can change sign across nearby configurations, which makes the signed count unstable, whileμ~k​ℓ\\tilde\{\\mu\}\_\{k\\ell\}stays defined and bounds it from above \(Proposition[C\.4](https://arxiv.org/html/2610.02344#A3.Thmtheorem4)\)\. Onhelena, whose smallest gap among the ten leading eigenvalues is about fifteen times that ofjannis\(Appendix[B\.4](https://arxiv.org/html/2610.02344#A2.SS4)\), the two agree:ρ⁡\(ERank,r~𝑡𝑖𝑔ℎ𝑡\)=\+0\.906\\rho\(\\Erank,\\tilde\{r\}\_\{\\mathit\{tight\}\}\)=\+0\.906andρ⁡\(ERank,r𝑡𝑖𝑔ℎ𝑡\)=\+0\.908\\rho\(\\Erank,r\_\{\\mathit\{tight\}\}\)=\+0\.908\(Figure[8](https://arxiv.org/html/2610.02344#A3.F8)\)\. Onaloithe absolute\-value form keepsρ=\+0\.905\\rho=\+0\.905where the signed form drops to\+0\.672\+0\.672; onjannisboth soften \(\+0\.62\+0\.62to\+0\.65\+0\.65\)\. We therefore reportμ~k​ℓ\\tilde\{\\mu\}\_\{k\\ell\}in the main text\.

Figure 8:Onhelenathe two formulations agree\.The signedμk​ℓ\\mu\_\{k\\ell\}and the absolute\-valueμ~k​ℓ\\tilde\{\\mu\}\_\{k\\ell\}give essentially identical scatter againstERank\\Erank\.
### C\.9Transfer of the budget to the embedding spectrum

The experiments read the effective rank of embeddings\. This section transfers the budget of Corollary[3\.3](https://arxiv.org/html/2610.02344#S3.Thmtheorem3)from𝐖\\mathbf\{W\}to𝐙\\mathbf\{Z\}in the linear model, under the assumptions of Appendix[C\.6](https://arxiv.org/html/2610.02344#A3.SS6)\. For non\-linear encoders the transfer is an empirical prediction, tested in Section[6](https://arxiv.org/html/2610.02344#S6)and Appendix[I\.5](https://arxiv.org/html/2610.02344#A9.SS5)\.

#### C\.9\.1The covariance as a readout

Write𝐳=\[vec​\(𝐖\)⊤,vec​\(𝐖a\)⊤\]⊤\\mathbf\{z\}=\[\\mathrm\{vec\}\(\\mathbf\{W\}\)^\{\\top\},\\mathrm\{vec\}\(\\mathbf\{W\}\_\{a\}\)^\{\\top\}\]^\{\\top\}for the state and𝐂⁡\(t\)=𝐖⁡\(t\)​𝚺​𝐖​\(t\)⊤\\mathbf\{C\}\(t\)=\\mathbf\{W\}\(t\)\\mathbf\{\\Sigma\}\\mathbf\{W\}\(t\)^\{\\top\}for the embedding covariance under a fixed positive semi\-definite second moment𝚺\\mathbf\{\\Sigma\}, the population𝚺X​X\\mathbf\{\\Sigma\}\_\{XX\}or the empirical second moment of a fixed evaluation set\.

###### Proposition C\.8\(Closure one level up\)\.

Differentiating𝐂\\mathbf\{C\}along Eq\. \([2](https://arxiv.org/html/2610.02344#S3.E2)\) gives

𝐂˙=−\(𝐏𝐖​𝚺11​𝚺​𝐖⊤\+𝐖​𝚺​𝚺11​𝐖⊤​𝐏\)\+\(𝐐𝐖a​𝚺X​1​𝚺​𝐖⊤\+𝐖​𝚺​𝚺X​1⊤​𝐖a⊤​𝐐⊤\),\\dot\{\\mathbf\{C\}\}=\-\\bigl\(\\mathbf\{P\}\\mathbf\{W\}\\mathbf\{\\Sigma\}\_\{11\}\\mathbf\{\\Sigma\}\\mathbf\{W\}^\{\\top\}\+\\mathbf\{W\}\\mathbf\{\\Sigma\}\\mathbf\{\\Sigma\}\_\{11\}\\mathbf\{W\}^\{\\top\}\\mathbf\{P\}\\bigr\)\+\\bigl\(\\mathbf\{Q\}\\mathbf\{W\}\_\{a\}\\mathbf\{\\Sigma\}\_\{X1\}\\mathbf\{\\Sigma\}\\mathbf\{W\}^\{\\top\}\+\\mathbf\{W\}\\mathbf\{\\Sigma\}\\mathbf\{\\Sigma\}\_\{X1\}^\{\\top\}\\mathbf\{W\}\_\{a\}^\{\\top\}\\mathbf\{Q\}^\{\\top\}\\bigr\),\(53\)which is not a function of𝐂\\mathbf\{C\}alone\. The joint second moment𝚵:=𝐳𝐳⊤\\mathbf\{\\Xi\}:=\\mathbf\{z\}\\mathbf\{z\}^\{\\top\}is closed,

𝚵˙=𝒥​𝚵\+𝚵​𝒥⊤,𝚵⁡\(t\)=e𝒥​t​𝚵​\(0\)​e𝒥⊤​t,\\dot\{\\mathbf\{\\Xi\}\}=\\mathcal\{J\}\\mathbf\{\\Xi\}\+\\mathbf\{\\Xi\}\\mathcal\{J\}^\{\\top\},\\qquad\\mathbf\{\\Xi\}\(t\)=e^\{\\mathcal\{J\}t\}\\mathbf\{\\Xi\}\(0\)e^\{\\mathcal\{J\}^\{\\top\}t\},\(54\)and𝐂⁡\(t\)\\mathbf\{C\}\(t\)is the fixed linear readout𝐂⁡\(t\)=TrN​\[\(𝚺⊗𝐈d\)​𝚵W​W​\(t\)\]\\mathbf\{C\}\(t\)=\\mathrm\{Tr\}\_\{N\}\\bigl\[\(\\mathbf\{\\Sigma\}\\otimes\\mathbf\{I\}\_\{d\}\)\\,\\mathbf\{\\Xi\}\_\{WW\}\(t\)\\bigr\]\.

###### Proof\.

The first display is the product rule applied to𝐂=𝐖​𝚺​𝐖⊤\\mathbf\{C\}=\\mathbf\{W\}\\mathbf\{\\Sigma\}\\mathbf\{W\}^\{\\top\}with Eq\. \([2](https://arxiv.org/html/2610.02344#S3.E2)\) substituted\. It involves𝐖​𝚺11​𝚺​𝐖⊤\\mathbf\{W\}\\mathbf\{\\Sigma\}\_\{11\}\\mathbf\{\\Sigma\}\\mathbf\{W\}^\{\\top\}and𝐖a​𝚺X​1​𝚺​𝐖⊤\\mathbf\{W\}\_\{a\}\\mathbf\{\\Sigma\}\_\{X1\}\\mathbf\{\\Sigma\}\\mathbf\{W\}^\{\\top\}, neither of which is a function of𝐖​𝚺​𝐖⊤\\mathbf\{W\}\\mathbf\{\\Sigma\}\\mathbf\{W\}^\{\\top\}\. Vectorising withvec⁡\(𝐀𝐗𝐁\)=\(𝐁⊤⊗𝐀\)​vec​\(𝐗\)\\mathrm\{vec\}\(\\mathbf\{A\}\\mathbf\{X\}\\mathbf\{B\}\)=\(\\mathbf\{B\}^\{\\top\}\\otimes\\mathbf\{A\}\)\\mathrm\{vec\}\(\\mathbf\{X\}\)turns the coupled flow into𝐳˙=𝒥​𝐳\\dot\{\\mathbf\{z\}\}=\\mathcal\{J\}\\mathbf\{z\}, whence𝚵˙=𝒥​𝚵\+𝚵​𝒥⊤\\dot\{\\mathbf\{\\Xi\}\}=\\mathcal\{J\}\\mathbf\{\\Xi\}\+\\mathbf\{\\Xi\}\\mathcal\{J\}^\{\\top\}\. For the readout, the\(m,n\)\(m,n\)block of𝚵W​W\\mathbf\{\\Xi\}\_\{WW\}is\(𝐖𝐞m\)​\(𝐖𝐞n\)⊤\(\\mathbf\{W\}\\mathbf\{e\}\_\{m\}\)\(\\mathbf\{W\}\\mathbf\{e\}\_\{n\}\)^\{\\top\}, so∑m,nΣm​n​\(𝐖𝐞m\)​\(𝐖𝐞n\)⊤=𝐖​𝚺​𝐖⊤=𝐂\\sum\_\{m,n\}\\Sigma\_\{mn\}\(\\mathbf\{W\}\\mathbf\{e\}\_\{m\}\)\(\\mathbf\{W\}\\mathbf\{e\}\_\{n\}\)^\{\\top\}=\\mathbf\{W\}\\mathbf\{\\Sigma\}\\mathbf\{W\}^\{\\top\}=\\mathbf\{C\}\. ∎

#### C\.9\.2Growth at twice the rate

The generatorℒ:𝐗↦𝒥​𝐗\+𝐗​𝒥⊤\\mathcal\{L\}:\\mathbf\{X\}\\mapsto\\mathcal\{J\}\\mathbf\{X\}\+\\mathbf\{X\}\\mathcal\{J\}^\{\\top\}has spectrum\{λi\+λj:λi,λj∈Spec\(𝒥\)\}\\\{\\lambda\_\{i\}\+\\lambda\_\{j\}:\\lambda\_\{i\},\\lambda\_\{j\}\\in\\Spec\(\\mathcal\{J\}\)\\\}\. When𝒥\\mathcal\{J\}is diagonalisable, as under the hypotheses of Theorem[C\.11](https://arxiv.org/html/2610.02344#A3.Thmtheorem11), the outer products of its eigenvectors form an eigenbasis ofℒ\\mathcal\{L\}, and the top growth rate is2​max⁡Re​Spec⁡\(𝒥\)2\\max\\mathrm\{Re\}\\,\\Spec\(\\mathcal\{J\}\)\.

Per mode, a super\-critical pair evolves as𝐳k​ℓ​\(t\)=αk​ℓ​eλ\+​t​ξk​ℓ\+\+⋯\\mathbf\{z\}\_\{k\\ell\}\(t\)=\\alpha\_\{k\\ell\}e^\{\\lambda^\{\+\}t\}\\mathbf\{\\xi\}^\{\+\}\_\{k\\ell\}\+\\cdots, where the𝐖\\mathbf\{W\}\-component ofξk​ℓ\+\\mathbf\{\\xi\}^\{\+\}\_\{k\\ell\}is one, soW~ℓ​k​\(t\)2∼αk​ℓ2​e2​λk​ℓ\+​t\\widetilde\{W\}\_\{\\ell k\}\(t\)^\{2\}\\sim\\alpha\_\{k\\ell\}^\{2\}e^\{2\\lambda^\{\+\}\_\{k\\ell\}t\}for a generic initialisation \(αk​ℓ≠0\\alpha\_\{k\\ell\}\\neq 0\)\. If𝚺=𝐔​diag​\(sk\)​𝐔⊤\\mathbf\{\\Sigma\}=\\mathbf\{U\}\\mathrm\{diag\}\(s\_\{k\}\)\\mathbf\{U\}^\{\\top\}withsk\>0s\_\{k\}\>0, then\(𝐕⊤​𝐂𝐕\)ℓ​ℓ=∑ksk​W~ℓ​k2\(\\mathbf\{V\}^\{\\top\}\\mathbf\{C\}\\mathbf\{V\}\)\_\{\\ell\\ell\}=\\sum\_\{k\}s\_\{k\}\\widetilde\{W\}\_\{\\ell k\}^\{2\}, so the embedding variance along𝐯ℓ\\mathbf\{v\}\_\{\\ell\}grows at rate2​max⁡\{λk​ℓ\+:μk​ℓ\>1\}2\\max\\\{\\lambda^\{\+\}\_\{k\\ell\}:\\mu\_\{k\\ell\}\>1\\\}if and only if somekkhasμk​ℓ\>1\\mu\_\{k\\ell\}\>1, and decays otherwise: the collapsed covariance is unstable exactly when𝒰≠∅\\mathcal\{U\}\\neq\\emptyset\. This per\-mode form needs𝚺\\mathbf\{\\Sigma\}to share the data eigenbasis, which a generic empirical second moment does not; the transfer below does not need it\.

#### C\.9\.3The spectral transfer, two\-sided

###### Lemma C\.9\(Weights to embeddings\)\.

For any𝐖∈ℝd×N\\mathbf\{W\}\\in\\mathbb\{R\}^\{d\\times N\}and positive semi\-definite𝚺\\mathbf\{\\Sigma\},σi​\(𝐖​𝚺1/2\)≤σmax​\(𝚺\)1/2​σi​\(𝐖\)\\sigma\_\{i\}\(\\mathbf\{W\}\\mathbf\{\\Sigma\}^\{1/2\}\)\\leq\\sigma\_\{\\max\}\(\\mathbf\{\\Sigma\}\)^\{1/2\}\\sigma\_\{i\}\(\\mathbf\{W\}\)for everyii; if𝚺≻0\\mathbf\{\\Sigma\}\\succ 0then alsoσi​\(𝐖​𝚺1/2\)≥σmin​\(𝚺\)1/2​σi​\(𝐖\)\\sigma\_\{i\}\(\\mathbf\{W\}\\mathbf\{\\Sigma\}^\{1/2\}\)\\geq\\sigma\_\{\\min\}\(\\mathbf\{\\Sigma\}\)^\{1/2\}\\sigma\_\{i\}\(\\mathbf\{W\}\)\. Hence, for𝚺≻0\\mathbf\{\\Sigma\}\\succ 0,rank⁡\(𝐖​𝚺1/2\)=rank⁡\(𝐖\)\\rank\(\\mathbf\{W\}\\mathbf\{\\Sigma\}^\{1/2\}\)=\\rank\(\\mathbf\{W\}\), the eigenvalues of𝐂\\mathbf\{C\}are sandwiched betweenλmin​\(𝚺\)​σi​\(𝐖\)2\\lambda\_\{\\min\}\(\\mathbf\{\\Sigma\}\)\\sigma\_\{i\}\(\\mathbf\{W\}\)^\{2\}andλmax​\(𝚺\)​σi​\(𝐖\)2\\lambda\_\{\\max\}\(\\mathbf\{\\Sigma\}\)\\sigma\_\{i\}\(\\mathbf\{W\}\)^\{2\}, and for everyε\>0\\varepsilon\>0the thresholded ranks obeyrankε/σmin​\(𝚺\)1/2⁡\(𝐖\)≤rankε⁡\(𝐖​𝚺1/2\)≤rankε/σmax​\(𝚺\)1/2⁡\(𝐖\)\\rank\_\{\\varepsilon/\\sigma\_\{\\min\}\(\\mathbf\{\\Sigma\}\)^\{1/2\}\}\(\\mathbf\{W\}\)\\leq\\rank\_\{\\varepsilon\}\(\\mathbf\{W\}\\mathbf\{\\Sigma\}^\{1/2\}\)\\leq\\rank\_\{\\varepsilon/\\sigma\_\{\\max\}\(\\mathbf\{\\Sigma\}\)^\{1/2\}\}\(\\mathbf\{W\}\)\.

###### Proof\.

The upper bound is the singular\-value product inequalityσi\+j−1​\(𝐀𝐁\)≤σi​\(𝐀\)​σj​\(𝐁\)\\sigma\_\{i\+j\-1\}\(\\mathbf\{A\}\\mathbf\{B\}\)\\leq\\sigma\_\{i\}\(\\mathbf\{A\}\)\\sigma\_\{j\}\(\\mathbf\{B\}\)atj=1j=1\[[Horn and Johnson, 1991](https://arxiv.org/html/2610.02344#bib.bib19)\]\. The lower bound applies the same inequality to𝐖=\(𝐖𝚺1/2\)𝚺−1/2\\mathbf\{W\}=\(\\mathbf\{W\}\\mathbf\{\\Sigma\}^\{1/2\}\)\\mathbf\{\\Sigma\}^\{\-1/2\}\. The three consequences follow index by index\. ∎

The two constants differ byκ​\(𝚺\)1/2\\kappa\(\\mathbf\{\\Sigma\}\)^\{1/2\}, withκ⁡\(𝐀\):=σmax​\(𝐀\)/σmin​\(𝐀\)\\kappa\(\\mathbf\{A\}\):=\\sigma\_\{\\max\}\(\\mathbf\{A\}\)/\\sigma\_\{\\min\}\(\\mathbf\{A\}\)\. This ratio enters only the head\-to\-tail ratio of the theorem below, neverr𝑡𝑖𝑔ℎ𝑡r\_\{\\mathit\{tight\}\}\.

#### C\.9\.4The budget, at the embedding level

Letm:=min⁡\(d,N\)m:=\\min\(d,N\), writer:=r𝑡𝑖𝑔ℎ𝑡r:=r\_\{\\mathit\{tight\}\}with1≤r<m1\\leq r<m, letq⁡\(t\)q\(t\)be the fraction of the singular\-value mass of𝐖⁡\(t\)​𝚺1/2\\mathbf\{W\}\(t\)\\mathbf\{\\Sigma\}^\{1/2\}carried by the indices beyondrr, and setψ⁡\(q\):=q⁡\[log⁡\(1/q\)\+1\+log⁡\(m−r\)\]\\psi\(q\):=q\\bigl\[\\log\(1/q\)\+1\+\\log\(m\-r\)\\bigr\]\.

###### Lemma C\.10\(Entropy grouping\)\.

Lets1≥⋯≥sm≥0s\_\{1\}\\geq\\dots\\geq s\_\{m\}\\geq 0, not all zero,pi=si/∑jsjp\_\{i\}=s\_\{i\}/\\sum\_\{j\}s\_\{j\}, andq:=∑i\>rpi∈\[0,1\)q:=\\sum\_\{i\>r\}p\_\{i\}\\in\[0,1\)for some1≤r<m1\\leq r<m\. ThenERank=exp\(−∑ipilogpi\)≤reψ⁡\(q\)\\Erank=\\exp\\bigl\(\-\\sum\_\{i\}p\_\{i\}\\log p\_\{i\}\\bigr\)\\leq r\\,e^\{\\psi\(q\)\}, andψ\\psiis increasing on\(0,1\)\(0,1\)withψ⁡\(q\)→0\\psi\(q\)\\to 0asq↓0q\\downarrow 0\.

###### Proof\.

Group the indices into the head\{1,…,r\}\\\{1,\\dots,r\\\}, of mass1−q1\-q, and the tail, of massqq\. The chain rule for entropy givesH⁡\(p\)=h⁡\(q\)\+\(1−q\)​Hhead\+q​HtailH\(p\)=h\(q\)\+\(1\-q\)H\_\{\\mathrm\{head\}\}\+qH\_\{\\mathrm\{tail\}\}, withhhthe binary entropy andHhead,HtailH\_\{\\mathrm\{head\}\},H\_\{\\mathrm\{tail\}\}the entropies of the renormalised head and tail\. Bounding each by the uniform value givesH⁡\(p\)≤h⁡\(q\)\+\(1−q\)​log⁡r\+q​log⁡\(m−r\)H\(p\)\\leq h\(q\)\+\(1\-q\)\\log r\+q\\log\(m\-r\), henceERank≤r​exp⁡\(h⁡\(q\)\+q​log⁡m−rr\)\\Erank\\leq r\\exp\\bigl\(h\(q\)\+q\\log\\tfrac\{m\-r\}\{r\}\\bigr\)\. Sincelog⁡\(1\+x\)≤x\\log\(1\+x\)\\leq xatx=q/\(1−q\)x=q/\(1\-q\),−\(1−q\)​log⁡\(1−q\)≤q\-\(1\-q\)\\log\(1\-q\)\\leq q, soh⁡\(q\)≤q​log⁡\(1/q\)\+qh\(q\)\\leq q\\log\(1/q\)\+q; andq​log⁡m−rr≤q​log⁡\(m−r\)q\\log\\tfrac\{m\-r\}\{r\}\\leq q\\log\(m\-r\)becauser≥1r\\geq 1\. Finallyψ′​\(q\)=log⁡\(\(m−r\)/q\)\>0\\psi^\{\\prime\}\(q\)=\\log\\bigl\(\(m\-r\)/q\\bigr\)\>0forq<1≤m−rq<1\\leq m\-r\. ∎

###### Theorem C\.11\(Effective rank obeys the budget past an explicit time\)\.

Under the exact form of Assumption[3\.2](https://arxiv.org/html/2610.02344#S3.Thmtheorem2), in the escape regime𝒰≠∅\\mathcal\{U\}\\neq\\emptysetwith a generic initialisation, with no critical or defective mode \(μk​ℓ≠1\\mu\_\{k\\ell\}\\neq 1andΔk​ℓ≠0\\Delta\_\{k\\ell\}\\neq 0for all\(k,ℓ\)\(k,\\ell\)\), and with𝚺≻0\\mathbf\{\\Sigma\}\\succ 0andr<mr<m, the constants

D:=2​\(m−r\)​κ​\(𝚺\)1/2​κ⋆​‖𝐳⁡\(0\)‖α⋆,T1:=max⁡\(T0,log⁡\(2​D\)/\(λmax\+\+η\)\),D:=\\frac\{2\(m\-r\)\\,\\kappa\(\\mathbf\{\\Sigma\}\)^\{1/2\}\\,\\kappa\_\{\\star\}\\,\\\|\\mathbf\{z\}\(0\)\\\|\}\{\\alpha\_\{\\star\}\},\\qquad T\_\{1\}:=\\max\\bigl\(T\_\{0\},\\log\(2D\)/\(\\lambda^\{\+\}\_\{\\max\}\+\\eta\)\\bigr\),computable at initialisation, with the conditioningκ⋆\\kappa\_\{\\star\}of the mode blocks, the amplitudeα⋆\\alpha\_\{\\star\}of the initial state along the fastest escaping mode and the timeT0T\_\{0\}defined in the proof, are such that

q⁡\(t\)≤D​e−\(λmax\+\+η\)​t\(t≥T0\)andERank⁡\(𝐖⁡\(t\)​𝚺1/2\)≤r​eψ⁡\(q⁡\(t\)\)\(t≥T1\),q\(t\)\\leq D\\,e^\{\-\(\\lambda^\{\+\}\_\{\\max\}\+\\eta\)t\}\\quad\(t\\geq T\_\{0\}\)\\qquad\\text\{and\}\\qquad\\Erank\\bigl\(\\mathbf\{W\}\(t\)\\mathbf\{\\Sigma\}^\{1/2\}\\bigr\)\\leq r\\,e^\{\\psi\(q\(t\)\)\}\\quad\(t\\geq T\_\{1\}\),\(55\)solim supt→∞ERank≤r𝑡𝑖𝑔ℎ𝑡\\limsup\_\{t\\to\\infty\}\\Erank\\leq r\_\{\\mathit\{tight\}\}, with excessψ⁡\(q⁡\(t\)\)≤C′​\(1\+t\)​e−\(λmax\+\+η\)​t\\psi\(q\(t\)\)\\leq C^\{\\prime\}\(1\+t\)e^\{\-\(\\lambda^\{\+\}\_\{\\max\}\+\\eta\)t\}\.

###### Proof\.

Recall from Step 5 of this appendix the split𝐖⁡\(t\)=𝐖\+​\(t\)\+𝐖−​\(t\)\\mathbf\{W\}\(t\)=\\mathbf\{W\}\_\{\+\}\(t\)\+\\mathbf\{W\}\_\{\-\}\(t\)along the unstable and stable invariant subspaces, andη\\eta, the slowest stable rate of Eq\. \([45](https://arxiv.org/html/2610.02344#A3.E45)\), which is positive because no mode is critical\. Let𝐓k​ℓ\\mathbf\{T\}\_\{k\\ell\}be the matrix of unit\-norm eigenvectors of𝐉k​ℓ\\mathbf\{J\}\_\{k\\ell\}in Eq\. \([42](https://arxiv.org/html/2610.02344#A3.E42)\), which exists becauseΔk​ℓ≠0\\Delta\_\{k\\ell\}\\neq 0, and setκ⋆:=maxk,ℓ⁡κ⁡\(𝐓k​ℓ\)\\kappa\_\{\\star\}:=\\max\_\{k,\\ell\}\\kappa\(\\mathbf\{T\}\_\{k\\ell\}\)\. Letλmax\+:=max𝒰⁡λk​ℓ\+\\lambda^\{\+\}\_\{\\max\}:=\\max\_\{\\mathcal\{U\}\}\\lambda^\{\+\}\_\{k\\ell\}, attained at\(k⋆,ℓ⋆\)\(k\_\{\\star\},\\ell\_\{\\star\}\), andα⋆:=\|αk⋆​ℓ⋆\|\\alpha\_\{\\star\}:=\|\\alpha\_\{k\_\{\\star\}\\ell\_\{\\star\}\}\|withαk​ℓ\\alpha\_\{k\\ell\}the amplitude above\. Generic initialisation meansα⋆≠0\\alpha\_\{\\star\}\\neq 0, which fails only on a hyperplane of initial states\. Writeςi:=σi​\(𝐖⁡\(t\)​𝚺1/2\)\\varsigma\_\{i\}:=\\sigma\_\{i\}\(\\mathbf\{W\}\(t\)\\mathbf\{\\Sigma\}^\{1/2\}\)andc:=λmax\+\+ηc:=\\lambda^\{\+\}\_\{\\max\}\+\\eta\.

First, the stable part\. On a saddle block the stable component evolves aseλk​ℓ−​t​𝚷k​ℓ−​𝐳k​ℓ​\(0\)e^\{\\lambda^\{\-\}\_\{k\\ell\}t\}\\mathbf\{\\Pi\}^\{\-\}\_\{k\\ell\}\\mathbf\{z\}\_\{k\\ell\}\(0\), with𝚷k​ℓ−\\mathbf\{\\Pi\}^\{\-\}\_\{k\\ell\}the spectral projector onto the stable eigenvector and‖𝚷k​ℓ−‖2≤κ⁡\(𝐓k​ℓ\)\\\|\\mathbf\{\\Pi\}^\{\-\}\_\{k\\ell\}\\\|\_\{2\}\\leq\\kappa\(\\mathbf\{T\}\_\{k\\ell\}\), and on a stable block‖e𝐉k​ℓ​t‖2≤κ⁡\(𝐓k​ℓ\)​max±​eRe​λk​ℓ±​t\\\|e^\{\\mathbf\{J\}\_\{k\\ell\}t\}\\\|\_\{2\}\\leq\\kappa\(\\mathbf\{T\}\_\{k\\ell\}\)\\max\_\{\\pm\}e^\{\\mathrm\{Re\}\\,\\lambda^\{\\pm\}\_\{k\\ell\}t\}\. Every stable rate is at most−η\-\\eta, and the rotation to𝐖~\\widetilde\{\\mathbf\{W\}\}is orthogonal, so summing the blocks gives‖𝐖−​\(t\)‖2≤‖𝐖−​\(t\)‖F≤κ⋆​e−η​t​‖𝐳⁡\(0\)‖\\\|\\mathbf\{W\}\_\{\-\}\(t\)\\\|\_\{2\}\\leq\\\|\\mathbf\{W\}\_\{\-\}\(t\)\\\|\_\{F\}\\leq\\kappa\_\{\\star\}e^\{\-\\eta t\}\\\|\\mathbf\{z\}\(0\)\\\|\.

Second, the tail\. By Corollary[3\.3](https://arxiv.org/html/2610.02344#S3.Thmtheorem3),rank⁡\(𝐖\+​\(t\)​𝚺1/2\)≤r\\rank\(\\mathbf\{W\}\_\{\+\}\(t\)\\mathbf\{\\Sigma\}^\{1/2\}\)\\leq r, soσr\+1​\(𝐖\+​𝚺1/2\)=0\\sigma\_\{r\+1\}\(\\mathbf\{W\}\_\{\+\}\\mathbf\{\\Sigma\}^\{1/2\}\)=0\. The additive inequalityσi\+j−1​\(𝐀\+𝐁\)≤σi​\(𝐀\)\+σj​\(𝐁\)\\sigma\_\{i\+j\-1\}\(\\mathbf\{A\}\+\\mathbf\{B\}\)\\leq\\sigma\_\{i\}\(\\mathbf\{A\}\)\+\\sigma\_\{j\}\(\\mathbf\{B\}\)ati=r\+1i=r\+1,j=1j=1then givesςr\+1≤σmax​\(𝚺\)1/2​‖𝐖−​\(t\)‖2≤σmax​\(𝚺\)1/2​κ⋆​‖𝐳⁡\(0\)‖​e−η​t\\varsigma\_\{r\+1\}\\leq\\sigma\_\{\\max\}\(\\mathbf\{\\Sigma\}\)^\{1/2\}\\\|\\mathbf\{W\}\_\{\-\}\(t\)\\\|\_\{2\}\\leq\\sigma\_\{\\max\}\(\\mathbf\{\\Sigma\}\)^\{1/2\}\\kappa\_\{\\star\}\\\|\\mathbf\{z\}\(0\)\\\|e^\{\-\\eta t\}\.

Third, the head\. Since\{𝐯ℓ​𝐮k⊤\}\\\{\\mathbf\{v\}\_\{\\ell\}\\mathbf\{u\}\_\{k\}^\{\\top\}\\\}is orthonormal and the𝐖\\mathbf\{W\}\-component ofξk​ℓ\+\\mathbf\{\\xi\}^\{\+\}\_\{k\\ell\}is one,𝐯ℓ⋆⊤​𝐖\+​\(t\)​𝐮k⋆=αk⋆​ℓ⋆​eλmax\+​t\\mathbf\{v\}\_\{\\ell\_\{\\star\}\}^\{\\top\}\\mathbf\{W\}\_\{\+\}\(t\)\\mathbf\{u\}\_\{k\_\{\\star\}\}=\\alpha\_\{k\_\{\\star\}\\ell\_\{\\star\}\}e^\{\\lambda^\{\+\}\_\{\\max\}t\}, soσ1​\(𝐖\+​\(t\)\)≥α⋆​eλmax\+​t\\sigma\_\{1\}\(\\mathbf\{W\}\_\{\+\}\(t\)\)\\geq\\alpha\_\{\\star\}e^\{\\lambda^\{\+\}\_\{\\max\}t\}\. The same additive inequality and the lower bound of Lemma[C\.9](https://arxiv.org/html/2610.02344#A3.Thmtheorem9)giveς1≥σmin​\(𝚺\)1/2​\(α⋆​eλmax\+​t−κ⋆​‖𝐳⁡\(0\)‖​e−η​t\)\\varsigma\_\{1\}\\geq\\sigma\_\{\\min\}\(\\mathbf\{\\Sigma\}\)^\{1/2\}\\bigl\(\\alpha\_\{\\star\}e^\{\\lambda^\{\+\}\_\{\\max\}t\}\-\\kappa\_\{\\star\}\\\|\\mathbf\{z\}\(0\)\\\|e^\{\-\\eta t\}\\bigr\)\.

Finally, setT0:=max⁡\(0,log⁡\(2​κ⋆​‖𝐳⁡\(0\)‖/α⋆\)/c\)T\_\{0\}:=\\max\\bigl\(0,\\log\(2\\kappa\_\{\\star\}\\\|\\mathbf\{z\}\(0\)\\\|/\\alpha\_\{\\star\}\)/c\\bigr\)\. Fort≥T0t\\geq T\_\{0\}the stable term is at most half the head term, soς1≥σmin​\(𝚺\)1/2​α⋆2​eλmax\+​t\>0\\varsigma\_\{1\}\\geq\\sigma\_\{\\min\}\(\\mathbf\{\\Sigma\}\)^\{1/2\}\\tfrac\{\\alpha\_\{\\star\}\}\{2\}e^\{\\lambda^\{\+\}\_\{\\max\}t\}\>0\. At mostm−rm\-rtail values are non\-zero and the total mass is at leastς1\\varsigma\_\{1\}, soq⁡\(t\)≤\(m−r\)​ςr\+1/ς1≤D​e−c​tq\(t\)\\leq\(m\-r\)\\varsigma\_\{r\+1\}/\\varsigma\_\{1\}\\leq De^\{\-ct\}withDDas stated\. Fort≥T1t\\geq T\_\{1\}this givesq⁡\(t\)≤12q\(t\)\\leq\\tfrac\{1\}\{2\}, and Lemma[C\.10](https://arxiv.org/html/2610.02344#A3.Thmtheorem10)yieldsERank≤r​eψ⁡\(q⁡\(t\)\)\\Erank\\leq r\\,e^\{\\psi\(q\(t\)\)\}\. Becauseψ\\psiis increasing on\(0,1\)\(0,1\),ψ⁡\(q⁡\(t\)\)≤ψ⁡\(D​e−c​t\)=D​e−c​t​\[log⁡\(1/D\)\+c​t\+1\+log⁡\(m−r\)\]≤C′​\(1\+t\)​e−c​t\\psi\(q\(t\)\)\\leq\\psi\(De^\{\-ct\}\)=De^\{\-ct\}\\bigl\[\\log\(1/D\)\+ct\+1\+\\log\(m\-r\)\\bigr\]\\leq C^\{\\prime\}\(1\+t\)e^\{\-ct\}withC′:=D⁡\[1\+\|log⁡D\|\+log⁡\(m−r\)\+c\]C^\{\\prime\}:=D\\bigl\[1\+\|\\log D\|\+\\log\(m\-r\)\+c\\bigr\], and thelim sup\\limsupfollows\. ∎

The additive form of Theorem[C\.3](https://arxiv.org/html/2610.02344#A3.Thmtheorem3)needs the extra conditionφ​\(θ¯​\(t\)\)≤1\\varphi\(\\bar\{\\theta\}\(t\)\)\\leq 1; the multiplicative form here does not\.

###### Corollary C\.12\(The quantity the experiments measure\)\.

Fix an evaluation set ofnninputs, the same at everytt, stacked as the columns of𝐗∈ℝN×n\\mathbf\{X\}\\in\\mathbb\{R\}^\{N\\times n\}, and let𝐙⁡\(t\)=𝐖⁡\(t\)​𝐗∈ℝd×n\\mathbf\{Z\}\(t\)=\\mathbf\{W\}\(t\)\\mathbf\{X\}\\in\\mathbb\{R\}^\{d\\times n\}and𝚺^:=1n​𝐗𝐗⊤\\hat\{\\mathbf\{\\Sigma\}\}:=\\tfrac\{1\}\{n\}\\mathbf\{X\}\\mathbf\{X\}^\{\\top\}\. Thenσi​\(𝐙\)=n​σi​\(𝐖​𝚺^1/2\)\\sigma\_\{i\}\(\\mathbf\{Z\}\)=\\sqrt\{n\}\\,\\sigma\_\{i\}\(\\mathbf\{W\}\\hat\{\\mathbf\{\\Sigma\}\}^\{1/2\}\)andERank⁡\(𝐙⁡\(t\)\)=ERank⁡\(𝐖⁡\(t\)​𝚺^1/2\)\\Erank\(\\mathbf\{Z\}\(t\)\)=\\Erank\(\\mathbf\{W\}\(t\)\\hat\{\\mathbf\{\\Sigma\}\}^\{1/2\}\)\. If𝐗\\mathbf\{X\}has rankNN\(son≥Nn\\geq N\),𝚺^≻0\\hat\{\\mathbf\{\\Sigma\}\}\\succ 0and Theorem[C\.11](https://arxiv.org/html/2610.02344#A3.Thmtheorem11)holds with𝚺^\\hat\{\\mathbf\{\\Sigma\}\}in place of𝚺\\mathbf\{\\Sigma\}; no compatibility of𝚺^\\hat\{\\mathbf\{\\Sigma\}\}with Assumption[3\.2](https://arxiv.org/html/2610.02344#S3.Thmtheorem2)is needed\.

###### Proof\.

𝐙𝐙⊤=n​𝐖​𝚺^​𝐖⊤\\mathbf\{Z\}\\mathbf\{Z\}^\{\\top\}=n\\,\\mathbf\{W\}\\hat\{\\mathbf\{\\Sigma\}\}\\mathbf\{W\}^\{\\top\}gives the singular\-value identity, and the effective rank is invariant under the factorn\\sqrt\{n\}and under zero padding of the spectrum\.𝐯⊤​𝚺^​𝐯=1n​‖𝐗⊤​𝐯‖2\\mathbf\{v\}^\{\\top\}\\hat\{\\mathbf\{\\Sigma\}\}\\mathbf\{v\}=\\tfrac\{1\}\{n\}\\\|\\mathbf\{X\}^\{\\top\}\\mathbf\{v\}\\\|^\{2\}vanishes for some𝐯≠𝟎\\mathbf\{v\}\\neq\\mathbf\{0\}exactly whenrank⁡\(𝐗\)<N\\rank\(\\mathbf\{X\}\)<N\. The proof of Theorem[C\.11](https://arxiv.org/html/2610.02344#A3.Thmtheorem11)uses only that the weighting matrix is fixed and positive definite, so it applies with𝚺^\\hat\{\\mathbf\{\\Sigma\}\}\. ∎

## Appendix DChoice of the Diagonal\-Attention Biasβinit\\beta\_\{\\text\{init\}\}

ResidualPred\(Section[6](https://arxiv.org/html/2610.02344#S6)\) introduces a single new hyperparameter, the diagonal\-attention biasβinit\\beta\_\{\\text\{init\}\}, applied at initialisation as

Attention⁡\(𝐐,𝐊,𝐕\)=Softmax⁡\(𝐐𝐊⊤d\+β⋅𝐁\)​𝐕\.\\mathrm\{Attention\}\(\\mathbf\{Q\},\\mathbf\{K\},\\mathbf\{V\}\)\\;=\\;\\mathrm\{Softmax\}\\\!\\left\(\\frac\{\\mathbf\{Q\}\\mathbf\{K\}^\{\\top\}\}\{\\sqrt\{d\}\}\\;\+\\;\\beta\\cdot\\mathbf\{B\}\\right\)\\mathbf\{V\}\\,\.\(56\)
\(a\)ERank vs\.βinit\\beta\_\{\\text\{init\}\}\.\(b\)Linear\-probe accuracy vs\.βinit\\beta\_\{\\text\{init\}\}\.
Figure 9:Monotone effect ofβinit\\beta\_\{\\text\{init\}\}on ERank and downstream accuracy\.The image and tabular pipelines instantiate Eq\. \([56](https://arxiv.org/html/2610.02344#A4.E56)\) differently due to their distinct sequence lengths and token structure\.

##### Images: learnable per\-headβ\\beta, context\-only identity\.

Each head has a learnableβh\\beta\_\{h\}initialised atβinit\\beta\_\{\\text\{init\}\}\(no weight decay\), affecting only the initial bias\. The matrix𝐁\\mathbf\{B\}is the identity on context tokens \(i=j≤nctxi=j\\leq n\_\{\\text\{ctx\}\}\) and zero elsewhere\. Thus, context tokens are biased toward identity, while mask tokens use standard attention to read from context\. Per\-headβ\\betacaptures ViT head specialisation, and restricting the bias to context ensures target tokens can be predicted from context at init\.

##### Tabular: fixed sharedβ\\beta, full identity\.

We use a fixed scalarβ=10\\beta=10shared across heads\. This simpler design is sufficient because: \(i\) smallNNalready yields near\-saturated diagonals, leaving no benefit for learningβ\\beta; \(ii\) tokens are homogeneous, so per\-head variation is unnecessary\. Both variants achieve the same objective \(identity predictor, near\-zero targets\) via different mechanisms\. The fixedβ\\betaalso keeps the tabular sweep lightweight\.

### D\.1Saturation criterion

For a context\-token row, the diagonal entry of the row\-stochastic attention matrix at initialisation is

Ai​i​\(βinit\)=11\+\(nctx−1\)​e−βinit,A\_\{ii\}\(\\beta\_\{\\text\{init\}\}\)\\;=\\;\\frac\{1\}\{1\+\(n\_\{\\text\{ctx\}\}\-1\)\\,e^\{\-\\beta\_\{\\text\{init\}\}\}\}\\,,\(57\)under the standard zero\-mean assumption on the off\-diagonal logits𝐐𝐊⊤/d\\mathbf\{Q\}\\mathbf\{K\}^\{\\top\}/\\sqrt\{d\}at initialisation\. The thresholdβinit≈log⁡nctx\+5\\beta\_\{\\text\{init\}\}\\approx\\log n\_\{\\text\{ctx\}\}\+5guaranteesAi​i\>0\.99A\_\{ii\}\>0\.99, i\.e\. a softmax operationally identical to the identity\. With keep probabilitypm1=0\.85p\_\{m\_\{1\}\}=0\.85, the context\-block size isnctx≤54n\_\{\\text\{ctx\}\}\\\!\\leq\\\!54on CIFAR\-10/100 \(ViT\-S/4,N=64N=64\) andnctx≤122n\_\{\\text\{ctx\}\}\\\!\\leq\\\!122on STL\-10 \(ViT\-S/8,N=144N=144\); for both,βinit=10\\beta\_\{\\text\{init\}\}=10saturates Eq\. \([57](https://arxiv.org/html/2610.02344#A4.E57)\) above0\.9970\.997\.

## Appendix EZero\-Init MLP Output: An Ablated Design Choice

TheResidualPredpredictor of Section[6](https://arxiv.org/html/2610.02344#S6)adapts the identity\-bias principle to the transformer architecture through a single structural ingredient: a diagonal\-attention biasβinit\\beta\_\{\\text\{init\}\}at initialisation\. During the design ofResidualPredwe considered a second, complementary candidate – a zero\-initialisation of the MLP output projection \(𝐖fc2←𝟎\\mathbf\{W\}\_\{\\text\{fc2\}\}\\\!\\leftarrow\\\!\\mathbf\{0\}\) – which extends the identity\-bias intuition to the second residual sub\-block of the transformer\. This appendix documents the candidate design, the2×22\\times 2ablation that disentangles its effect from the diagonal\-attention bias, and the reason why we ultimately retain only the diagonal bias in the main paper\. The exposition is in two registers: the formula and theoretical motivation, and the empirical evidence that informs the design choice\.

### E\.1Motivation: a natural extension of the identity\-bias principle

In the linear analysis of Section[3](https://arxiv.org/html/2610.02344#S3), the predictor enters the dynamics through𝐏:=𝐖p⊤​𝐖p\\mathbf\{P\}:=\\mathbf\{W\}\_\{p\}^\{\\top\}\\mathbf\{W\}\_\{p\}and𝐐:=𝐖p⊤​𝐌¯2\\mathbf\{Q\}:=\\mathbf\{W\}\_\{p\}^\{\\top\}\\bar\{\\mathbf\{M\}\}\_\{2\}, with the per\-mode decay rateσi\\sigma\_\{i\}controlled by the spectrum of𝐏\\mathbf\{P\}\. Initialising𝐖p≈𝐈\+𝚫\\mathbf\{W\}\_\{p\}\\approx\\mathbf\{I\}\+\\mathbf\{\\Delta\}with‖𝚫‖→0\\\|\\mathbf\{\\Delta\}\\\|\\to 0places the predictor in the regime in which view alignment dominates the early dynamics\.

A standard transformer\-based predictor block \(pre\-LN convention\) is composed of*two*residual sub\-blocks,

block⁡\(𝐱\)=𝐱\+Attn⁡\(LN1​\(𝐱\)\)\+MLP⁡\(LN2​\(𝐱\+Attn⁡\(LN1​\(𝐱\)\)\)\),\\mathrm\{block\}\(\\mathbf\{x\}\)\\;=\\;\\mathbf\{x\}\\,\+\\,\\mathrm\{Attn\}\(\\mathrm\{LN\}\_\{1\}\(\\mathbf\{x\}\)\)\\,\+\\,\\mathrm\{MLP\}\\\!\\bigl\(\\mathrm\{LN\}\_\{2\}\(\\mathbf\{x\}\+\\mathrm\{Attn\}\(\\mathrm\{LN\}\_\{1\}\(\\mathbf\{x\}\)\)\)\\bigr\)\\,,\(58\)so an exact𝐖p≈𝐈\\mathbf\{W\}\_\{p\}\\approx\\mathbf\{I\}att=0t=0would in principle require*both*sub\-blocks to behave as the identity\. The diagonal\-attention bias addresses the attention sub\-block \(Appendix[D](https://arxiv.org/html/2610.02344#A4)\)\. Independently, one can address the MLP sub\-block by zeroing its output projection at initialisation, which is the route followed by ReZero\[[Bachlechner et al\., 2021](https://arxiv.org/html/2610.02344#bib.bib2)\], Fixup\[[Zhang et al\., 2019](https://arxiv.org/html/2610.02344#bib.bib38)\], SkipInit\[[De and Smith, 2020](https://arxiv.org/html/2610.02344#bib.bib10)\], and IDInit\[[Pan et al\., 2025](https://arxiv.org/html/2610.02344#bib.bib25)\]for deep residual signal propagation\. Whether this route adds measurable value when applied on top of the diagonal\-attention bias*specifically for collapse avoidance*is an empirical question we test below\.

### E\.2The candidate formula

In a standard transformer, the MLP sub\-block is

MLP⁡\(𝐳\)=𝐖fc2​ϕ​\(𝐖fc1​𝐳\+𝐛fc1\)\+𝐛fc2,\\mathrm\{MLP\}\(\\mathbf\{z\}\)\\;=\\;\\mathbf\{W\}\_\{\\text\{fc2\}\}\\,\\phi\\\!\\bigl\(\\mathbf\{W\}\_\{\\text\{fc1\}\}\\,\\mathbf\{z\}\+\\mathbf\{b\}\_\{\\text\{fc1\}\}\\bigr\)\\,\+\\,\\mathbf\{b\}\_\{\\text\{fc2\}\}\\,,\(59\)withϕ⁡\(⋅\)=GELU⁡\(⋅\)\\phi\(\\cdot\)=\\mathrm\{GELU\}\(\\cdot\)\. The candidate ‘zero\-init’ modification is the structural choice

𝐖fc2←0,𝐛fc2←0att=0\.\\boxed\{\\quad\\mathbf\{W\}\_\{\\text\{fc2\}\}\\;\\leftarrow\\;\\mathbf\{0\}\\,,\\qquad\\mathbf\{b\}\_\{\\text\{fc2\}\}\\;\\leftarrow\\;\\mathbf\{0\}\\quad\\text\{at \}t=0\\,\.\\quad\}\(60\)Substituting \([60](https://arxiv.org/html/2610.02344#A5.E60)\) into \([59](https://arxiv.org/html/2610.02344#A5.E59)\) givesMLP⁡\(𝐳\)≡𝟎\\mathrm\{MLP\}\(\\mathbf\{z\}\)\\equiv\\mathbf\{0\}att=0t=0for every input𝐳\\mathbf\{z\}, and therefore

block⁡\(𝐱\)=𝐱\+Attn⁡\(LN1​\(𝐱\)\)at​t=0\.\\mathrm\{block\}\(\\mathbf\{x\}\)\\;=\\;\\mathbf\{x\}\\,\+\\,\\mathrm\{Attn\}\(\\mathrm\{LN\}\_\{1\}\(\\mathbf\{x\}\)\)\\quad\\text\{at \}t=0\\,\.\(61\)
##### Scope\.

Zero\-init applies only to\(𝐖fc2,𝐛fc2\)\(\\mathbf\{W\}\_\{\\text\{fc2\}\},\\mathbf\{b\}\_\{\\text\{fc2\}\}\)\. The attention output projectionattn\.proj\\mathrm\{attn\.proj\}is left at its standard initialisation: zeroing it would sever the path that lets context tokens inform mask tokens through attention, which is essential for a sequence\-to\-sequence predictor\. The candidate is therefore the appropriate analogue of the ReZero / Fixup recipe for an attention\-based predictor\.

##### Gradients are preserved\.

Although the parameter is zero,∂ℒ/∂𝐖fc2\\partial\\mathcal\{L\}/\\partial\\mathbf\{W\}\_\{\\text\{fc2\}\}is non\-zero at the first optimisation step \(it depends on the post\-activation output and the upstream error signal, not on𝐖fc2\\mathbf\{W\}\_\{\\text\{fc2\}\}itself\), so the MLP becomes active after a single update\. The candidate is a delayed\-engagement scheme, not a frozen\-parameter scheme\.

### E\.3What zero\-init does to the predictor block att=0t=0

The remaining attention residual in \([61](https://arxiv.org/html/2610.02344#A5.E61)\) depends onβinit\\beta\_\{\\text\{init\}\}\. Let𝐀⁡\(βinit\):=Softmax⁡\(𝐐𝐊⊤/d\+βinit​𝐈\)\\mathbf\{A\}\(\\beta\_\{\\text\{init\}\}\):=\\mathrm\{Softmax\}\\bigl\(\\mathbf\{Q\}\\mathbf\{K\}^\{\\top\}/\\sqrt\{d\}\+\\beta\_\{\\text\{init\}\}\\mathbf\{I\}\\bigr\)denote the attention map\. With per\-token value projection𝐕​LN1​\(𝐱\)\\mathbf\{V\}\\,\\mathrm\{LN\}\_\{1\}\(\\mathbf\{x\}\)and output projection𝐎\\mathbf\{O\}, the block under zero\-init MLP becomes:

- •Diagonal\-bias regime \(βinit≫log⁡L\\beta\_\{\\text\{init\}\}\\\!\\gg\\\!\\log L\)\.𝐀⁡\(βinit\)→𝐈token\\mathbf\{A\}\(\\beta\_\{\\text\{init\}\}\)\\to\\mathbf\{I\}\_\{\\text\{token\}\}\(Eq\. \([57](https://arxiv.org/html/2610.02344#A4.E57)\) of Appendix[D](https://arxiv.org/html/2610.02344#A4)\), so each token’s output depends on its own input only: block​\(𝐱\)i=𝐱i\+𝐎​𝐕​LN1​\(𝐱\)i\.\\mathrm\{block\}\(\\mathbf\{x\}\)\_\{i\}\\;=\\;\\mathbf\{x\}\_\{i\}\\,\+\\,\\mathbf\{O\}\\,\\mathbf\{V\}\\,\\mathrm\{LN\}\_\{1\}\(\\mathbf\{x\}\)\_\{i\}\\,\.\(62\)A per\-token linear transform of𝐱i\\mathbf\{x\}\_\{i\}, full\-rank in token space\.
- •Diffuse regime \(βinit=0\\beta\_\{\\text\{init\}\}=0\)\.𝐀⁡\(βinit\)≈\(1/L\)​11⊤\\mathbf\{A\}\(\\beta\_\{\\text\{init\}\}\)\\approx\(1/L\)\\,\\mathbf\{1\}\\mathbf\{1\}^\{\\top\}\(uniform attention at random init\), and each token’s output depends on the average of all tokens’ inputs only: block​\(𝐱\)i=𝐱i\+𝐎​𝐕​LN1​\(𝐱\)¯,𝐳¯:=1L​∑j=1L𝐳j\.\\mathrm\{block\}\(\\mathbf\{x\}\)\_\{i\}\\;=\\;\\mathbf\{x\}\_\{i\}\\,\+\\,\\mathbf\{O\}\\,\\mathbf\{V\}\\,\\overline\{\\mathrm\{LN\}\_\{1\}\(\\mathbf\{x\}\)\}\\,,\\qquad\\overline\{\\mathbf\{z\}\}:=\\tfrac\{1\}\{L\}\\sum\_\{j=1\}^\{L\}\\mathbf\{z\}\_\{j\}\\,\.\(63\)The attention contribution is rank\-one in token space: a single direction is broadcast to every token\.

##### The decisive factor att=0t=0is the support of the predictor’s deviation from identity\.

View the predictor att=0t=0as a linear map on the stacked token–feature spaceℝL​N\\mathbb\{R\}^\{LN\}\. With zero\-init MLP, the deviation from identity is the attention residual itself:

𝐖p=𝐈L​N\+𝐓⊗\(𝐎𝐕\),𝐓=\{𝐈L\(diagonal\-bias regime\)\(1/L\)​1L​𝟏L⊤\(diffuse regime\)\\mathbf\{W\}\_\{p\}\\;=\\;\\mathbf\{I\}\_\{LN\}\\,\+\\,\\mathbf\{T\}\\\!\\otimes\\\!\(\\mathbf\{O\}\\mathbf\{V\}\)\\,,\\qquad\\mathbf\{T\}\\;=\\;\\begin\{cases\}\\mathbf\{I\}\_\{L\}&\\text\{\(diagonal\-bias regime\)\}\\\\\[2\.0pt\] \(1/L\)\\,\\mathbf\{1\}\_\{L\}\\mathbf\{1\}\_\{L\}^\{\\top\}&\\text\{\(diffuse regime\)\}\\end\{cases\}\(64\)The two regimes differ in the*token\-space support*of𝐓\\mathbf\{T\}: full rank in the diagonal\-bias case, rank one in the diffuse case\. Computing𝐏=𝐖p⊤​𝐖p=𝐈L​N\+𝐓⊗\(\(𝐎𝐕\)⊤\+𝐎𝐕\+\(𝐎𝐕\)⊤​\(𝐎𝐕\)\)\\mathbf\{P\}=\\mathbf\{W\}\_\{p\}^\{\\top\}\\mathbf\{W\}\_\{p\}=\\mathbf\{I\}\_\{LN\}\+\\mathbf\{T\}\\otimes\\bigl\(\(\\mathbf\{O\}\\mathbf\{V\}\)^\{\\top\}\+\\mathbf\{O\}\\mathbf\{V\}\+\(\\mathbf\{O\}\\mathbf\{V\}\)^\{\\top\}\(\\mathbf\{O\}\\mathbf\{V\}\)\\bigr\)shows that𝐏\\mathbf\{P\}inherits this structure mode\-by\-mode:

- •In the diagonal\-bias regime \(𝐓=𝐈L\\mathbf\{T\}=\\mathbf\{I\}\_\{L\}\), every\(k,ℓ\)\(k,\\ell\)pair receives the predictor enhancement:pℓp\_\{\\ell\}takes the form1\+λℓ​\(𝐒\)1\+\\lambda\_\{\\ell\}\(\\mathbf\{S\}\)with𝐒=\(𝐎𝐕\)⊤\+𝐎𝐕\+\(𝐎𝐕\)⊤​\(𝐎𝐕\)\\mathbf\{S\}=\(\\mathbf\{O\}\\mathbf\{V\}\)^\{\\top\}\+\\mathbf\{O\}\\mathbf\{V\}\+\(\\mathbf\{O\}\\mathbf\{V\}\)^\{\\top\}\(\\mathbf\{O\}\\mathbf\{V\}\), on*all*L​NLNmodes\. Likewiseqℓq\_\{\\ell\}inherits a per\-token enhancement on all modes from𝐖p⊤​𝐌¯2\\mathbf\{W\}\_\{p\}^\{\\top\}\\bar\{\\mathbf\{M\}\}\_\{2\}\.
- •In the diffuse regime \(𝐓=\(1/L\)​𝟏𝟏⊤\\mathbf\{T\}=\(1/L\)\\mathbf\{1\}\\mathbf\{1\}^\{\\top\}, eigenvalues\{1,0,…,0\}\\\{1,0,\\dots,0\\\}\), the predictor enhancement is supported only on the*all\-ones*direction in token space:NNof theL​NLNmodes receive the full enhancement1\+λℓ​\(𝐒\)1\+\\lambda\_\{\\ell\}\(\\mathbf\{S\}\), while the remainingL​N−NLN\-Nmodes havepℓ=1p\_\{\\ell\}=1andqℓq\_\{\\ell\}at the trivial baseline given by𝐌¯2\\bar\{\\mathbf\{M\}\}\_\{2\}alone\.

The drive–decay ratio along these unaffectedL​N−NLN\-Nmodes therefore reduces to the data\-side baselineμk​ℓ=\(ck/ak\)⋅\(qℓ\(0\)/pℓ\(0\)\)\\mu\_\{k\\ell\}=\(c\_\{k\}/a\_\{k\}\)\\cdot\(q\_\{\\ell\}^\{\(0\)\}/p\_\{\\ell\}^\{\(0\)\}\)withpℓ\(0\)=1p\_\{\\ell\}^\{\(0\)\}=1andqℓ\(0\)q\_\{\\ell\}^\{\(0\)\}given by𝐌¯2\\bar\{\\mathbf\{M\}\}\_\{2\}\. In other words, in the diffuse regime the predictor amplifiesμk​ℓ\\mu\_\{k\\ell\}on at mostNNof theL​NLNtoken\-feature modes; in the diagonal\-bias regime it amplifies on allL​NLN\. Zero\-init the MLP output suppresses an*additional*, smaller deviation from identity on top of the attention residual \([64](https://arxiv.org/html/2610.02344#A5.E64)\), but does not change the support of the dominant attention deviation; in the diffuse regime, the attention contribution alone already determines which token\-feature modes the predictor can lift above threshold\. The theoretical prediction is therefore that zero\-init alone \(without the diagonal bias\) leaves the bulk of theL​NLNmodes at their unenhanced baseline, where the data\-side ratiock/akc\_\{k\}/a\_\{k\}has to do all of the work; its expected effect on accuracy in the diffuse regime is small\.

### E\.4Empirical results

We run the four cells of the grid on the I\-JEPA CIFAR\-100 benchmark with ViT\-Small/4,L=64L=64,pm1=0\.85p\_\{m\_\{1\}\}=0\.85, at100100and300300epoch budgets\.

Table 8:Linear\-probe accuracy across the2×22\\times 2grid, CIFAR\-100, ViT\-Small/4,pm1=0\.85p\_\{m\_\{1\}\}=0\.85\.Mean±\\pmstandard deviation over seeds\.Δ\\Deltaand two\-sidedpp\-values are computed against the Standard cell\.ResidualPred\(the design retained in the main paper\) andResidualPred \+ ZeroInitare statistically indistinguishable in mean\.

## Appendix FAdiabatic case

### F\.1Derivation of Linear JEPA Dynamics \(Expected Loss\)

We derive the gradients of the expected loss functionℒ\\mathcal\{L\}\(Eq\. \([1](https://arxiv.org/html/2610.02344#S3.E1)\)\) with respect to𝐖\\mathbf\{W\}and𝐖p\\mathbf\{W\}\_\{p\}\.

ℒ=12​𝔼​\[‖𝐖p​𝐖𝐱1−sg​\(𝐌2​𝐖a​𝐱\)‖22\]\\mathcal\{L\}=\\frac\{1\}\{2\}\\mathbb\{E\}\\left\[\\\|\\mathbf\{W\}\_\{p\}\\mathbf\{W\}\\mathbf\{x\}\_\{1\}\-\\texttt\{sg\}\(\\mathbf\{M\}\_\{2\}\\mathbf\{W\}\_\{a\}\\mathbf\{x\}\)\\\|\_\{2\}^\{2\}\\right\]
Gradient with respect to𝐖\\mathbf\{W\}:

We need to compute∇𝐖ℒ\\nabla\_\{\\mathbf\{W\}\}\\mathcal\{L\}\. We can move the gradient operator inside the expectation, as expectation is a linear operator:

∇𝐖ℒ=𝔼⁡\[∇𝐖12​‖𝐖p​𝐖𝐱1−𝐭‖22\]\\nabla\_\{\\mathbf\{W\}\}\\mathcal\{L\}=\\mathbb\{E\}\\left\[\\nabla\_\{\\mathbf\{W\}\}\\frac\{1\}\{2\}\\\|\\mathbf\{W\}\_\{p\}\\mathbf\{W\}\\mathbf\{x\}\_\{1\}\-\\mathbf\{t\}\\\|\_\{2\}^\{2\}\\right\]Let the term inside the expectation beLs=12​‖𝐖p​𝐖𝐱1−𝐭‖22L\_\{s\}=\\frac\{1\}\{2\}\\\|\\mathbf\{W\}\_\{p\}\\mathbf\{W\}\\mathbf\{x\}\_\{1\}\-\\mathbf\{t\}\\\|\_\{2\}^\{2\}\. This is a standard least\-squares type objective\. The gradient ofLsL\_\{s\}with respect to𝐖\\mathbf\{W\}can be found using matrix calculus identities\. Thus, the gradient is:

∇𝐖Ls=𝐖p⊤​\(𝐖p​𝐖𝐱1−𝐭\)​𝐱1⊤\\nabla\_\{\\mathbf\{W\}\}L\_\{s\}=\\mathbf\{W\}\_\{p\}^\{\\top\}\(\\mathbf\{W\}\_\{p\}\\mathbf\{W\}\\mathbf\{x\}\_\{1\}\-\\mathbf\{t\}\)\\mathbf\{x\}\_\{1\}^\{\\top\}Substituting this back into the expression for∇𝐖ℒ\\nabla\_\{\\mathbf\{W\}\}\\mathcal\{L\}and replacing𝐭\\mathbf\{t\}with its definition:

∇𝐖ℒ\\displaystyle\\nabla\_\{\\mathbf\{W\}\}\\mathcal\{L\}=𝔼⁡\[𝐖p⊤​\(𝐖p​𝐖𝐱1−𝐌2​𝐖a​𝐱\)​𝐱1⊤\]\\displaystyle=\\mathbb\{E\}\\left\[\\mathbf\{W\}\_\{p\}^\{\\top\}\(\\mathbf\{W\}\_\{p\}\\mathbf\{W\}\\mathbf\{x\}\_\{1\}\-\\mathbf\{M\}\_\{2\}\\mathbf\{W\}\_\{a\}\\mathbf\{x\}\)\\mathbf\{x\}\_\{1\}^\{\\top\}\\right\]=𝔼⁡\[𝐖p⊤​𝐖p​𝐖𝐱1​𝐱1⊤−𝐖p⊤​𝐌2​𝐖a​𝐱𝐱1⊤\]\\displaystyle=\\mathbb\{E\}\\left\[\\mathbf\{W\}\_\{p\}^\{\\top\}\\mathbf\{W\}\_\{p\}\\mathbf\{W\}\\mathbf\{x\}\_\{1\}\\mathbf\{x\}\_\{1\}^\{\\top\}\-\\mathbf\{W\}\_\{p\}^\{\\top\}\\mathbf\{M\}\_\{2\}\\mathbf\{W\}\_\{a\}\\mathbf\{x\}\\mathbf\{x\}\_\{1\}^\{\\top\}\\right\]Using the linearity of expectation and the definitions𝚺11=𝔼⁡\[𝐱1​𝐱1⊤\]\\mathbf\{\\Sigma\}\_\{11\}=\\mathbb\{E\}\[\\mathbf\{x\}\_\{1\}\\mathbf\{x\}\_\{1\}^\{\\top\}\]and𝚺X​1=𝔼⁡\[𝐱𝐱1⊤\]\\mathbf\{\\Sigma\}\_\{X1\}=\\mathbb\{E\}\[\\mathbf\{x\}\\mathbf\{x\}\_\{1\}^\{\\top\}\]:

∇𝐖ℒ\\displaystyle\\nabla\_\{\\mathbf\{W\}\}\\mathcal\{L\}=𝐖p⊤​𝐖p​𝐖​𝔼​\[𝐱1​𝐱1⊤\]−𝔼⁡\[𝐖p⊤​𝐌2​𝐖a​𝐱𝐱1⊤\]\\displaystyle=\\mathbf\{W\}\_\{p\}^\{\\top\}\\mathbf\{W\}\_\{p\}\\mathbf\{W\}\\mathbb\{E\}\[\\mathbf\{x\}\_\{1\}\\mathbf\{x\}\_\{1\}^\{\\top\}\]\-\\mathbb\{E\}\[\\mathbf\{W\}\_\{p\}^\{\\top\}\\mathbf\{M\}\_\{2\}\\mathbf\{W\}\_\{a\}\\mathbf\{x\}\\mathbf\{x\}\_\{1\}^\{\\top\}\]=𝐖p⊤𝐖p𝐖𝚺11−𝔼\[𝐖p⊤𝐌2𝐖a𝐱𝐱1⊤\]\(∗\)\\displaystyle=\\mathbf\{W\}\_\{p\}^\{\\top\}\\mathbf\{W\}\_\{p\}\\mathbf\{W\}\\mathbf\{\\Sigma\}\_\{11\}\-\\mathbb\{E\}\[\\mathbf\{W\}\_\{p\}^\{\\top\}\\mathbf\{M\}\_\{2\}\\mathbf\{W\}\_\{a\}\\mathbf\{x\}\\mathbf\{x\}\_\{1\}^\{\\top\}\]\\quad\(\*\)
If we further assume that the masking strategy𝐌2\\mathbf\{M\}\_\{2\}is independent of𝐱,𝐱1\\mathbf\{x\},\\mathbf\{x\}\_\{1\}, let𝐌¯2=𝔼⁡\[𝐌2\]\\bar\{\\mathbf\{M\}\}\_\{2\}=\\mathbb\{E\}\[\\mathbf\{M\}\_\{2\}\], and assume𝐖a\\mathbf\{W\}\_\{a\}is deterministic \(e\.g\., an Exponential Moving Average update, treated as constant within the expectation\), the second term simplifies:

𝔼⁡\[𝐖p⊤​𝐌2​𝐖a​𝐱𝐱1⊤\]=𝐖p⊤​𝔼​\[𝐌2\]​𝐖a​𝔼​\[𝐱𝐱1⊤\]=𝐖p⊤​𝐌¯2​𝐖a​𝚺X​1\\mathbb\{E\}\[\\mathbf\{W\}\_\{p\}^\{\\top\}\\mathbf\{M\}\_\{2\}\\mathbf\{W\}\_\{a\}\\mathbf\{x\}\\mathbf\{x\}\_\{1\}^\{\\top\}\]=\\mathbf\{W\}\_\{p\}^\{\\top\}\\mathbb\{E\}\[\\mathbf\{M\}\_\{2\}\]\\mathbf\{W\}\_\{a\}\\mathbb\{E\}\[\\mathbf\{x\}\\mathbf\{x\}\_\{1\}^\{\\top\}\]=\\mathbf\{W\}\_\{p\}^\{\\top\}\\bar\{\\mathbf\{M\}\}\_\{2\}\\mathbf\{W\}\_\{a\}\\mathbf\{\\Sigma\}\_\{X1\}Substituting this into\(∗\)\(\*\):

∇𝐖ℒ=𝐖p⊤​𝐖p​𝐖​𝚺11−𝐖p⊤​𝐌¯2​𝐖a​𝚺X​1\\nabla\_\{\\mathbf\{W\}\}\\mathcal\{L\}=\\mathbf\{W\}\_\{p\}^\{\\top\}\\mathbf\{W\}\_\{p\}\\mathbf\{W\}\\mathbf\{\\Sigma\}\_\{11\}\-\\mathbf\{W\}\_\{p\}^\{\\top\}\\bar\{\\mathbf\{M\}\}\_\{2\}\\mathbf\{W\}\_\{a\}\\mathbf\{\\Sigma\}\_\{X1\}
Gradient with respect to𝐖p\\mathbf\{W\}\_\{p\}:We find the gradient inside the expectation:

∇𝐖p‖𝐖p​\(𝐖𝐱1\)−\(𝐌2​𝐖a​𝐱\)‖22=2​\(𝐖p​𝐖𝐱1−𝐌2​𝐖a​𝐱\)​\(𝐖𝐱1\)⊤\\nabla\_\{\\mathbf\{W\}\_\{p\}\}\\\|\\mathbf\{W\}\_\{p\}\(\\mathbf\{W\}\\mathbf\{x\}\_\{1\}\)\-\(\\mathbf\{M\}\_\{2\}\\mathbf\{W\}\_\{a\}\\mathbf\{x\}\)\\\|\_\{2\}^\{2\}=2\(\\mathbf\{W\}\_\{p\}\\mathbf\{W\}\\mathbf\{x\}\_\{1\}\-\\mathbf\{M\}\_\{2\}\\mathbf\{W\}\_\{a\}\\mathbf\{x\}\)\(\\mathbf\{W\}\\mathbf\{x\}\_\{1\}\)^\{\\top\}Taking the expectation:

∇𝐖pℒ\\displaystyle\\nabla\_\{\\mathbf\{W\}\_\{p\}\}\\mathcal\{L\}=12​𝔼​\[2​\(𝐖p​𝐖𝐱1−𝐌2​𝐖a​𝐱\)​\(𝐖𝐱1\)⊤\]\\displaystyle=\\frac\{1\}\{2\}\\mathbb\{E\}\\left\[2\(\\mathbf\{W\}\_\{p\}\\mathbf\{W\}\\mathbf\{x\}\_\{1\}\-\\mathbf\{M\}\_\{2\}\\mathbf\{W\}\_\{a\}\\mathbf\{x\}\)\(\\mathbf\{W\}\\mathbf\{x\}\_\{1\}\)^\{\\top\}\\right\]=𝔼⁡\[𝐖p​\(𝐖𝐱1\)​\(𝐖𝐱1\)⊤−\(𝐌2​𝐖a​𝐱\)​\(𝐖𝐱1\)⊤\]\\displaystyle=\\mathbb\{E\}\\left\[\\mathbf\{W\}\_\{p\}\(\\mathbf\{W\}\\mathbf\{x\}\_\{1\}\)\(\\mathbf\{W\}\\mathbf\{x\}\_\{1\}\)^\{\\top\}\-\(\\mathbf\{M\}\_\{2\}\\mathbf\{W\}\_\{a\}\\mathbf\{x\}\)\(\\mathbf\{W\}\\mathbf\{x\}\_\{1\}\)^\{\\top\}\\right\]=𝐖p​𝔼​\[\(𝐖𝐱1\)​\(𝐖𝐱1\)⊤\]−𝔼⁡\[\(𝐌2​𝐖a​𝐱\)​\(𝐖𝐱1\)⊤\]\\displaystyle=\\mathbf\{W\}\_\{p\}\\mathbb\{E\}\[\(\\mathbf\{W\}\\mathbf\{x\}\_\{1\}\)\(\\mathbf\{W\}\\mathbf\{x\}\_\{1\}\)^\{\\top\}\]\-\\mathbb\{E\}\[\(\\mathbf\{M\}\_\{2\}\\mathbf\{W\}\_\{a\}\\mathbf\{x\}\)\(\\mathbf\{W\}\\mathbf\{x\}\_\{1\}\)^\{\\top\}\]=𝐖p​\(𝐖​𝔼​\[𝐱1​𝐱1⊤\]​𝐖⊤\)−𝔼⁡\[𝐌2​𝐖a​𝐱𝐱1⊤​𝐖⊤\]\\displaystyle=\\mathbf\{W\}\_\{p\}\(\\mathbf\{W\}\\mathbb\{E\}\[\\mathbf\{x\}\_\{1\}\\mathbf\{x\}\_\{1\}^\{\\top\}\]\\mathbf\{W\}^\{\\top\}\)\-\\mathbb\{E\}\[\\mathbf\{M\}\_\{2\}\\mathbf\{W\}\_\{a\}\\mathbf\{x\}\\mathbf\{x\}\_\{1\}^\{\\top\}\\mathbf\{W\}^\{\\top\}\]=𝐖p​\(𝐖​𝚺11​𝐖⊤\)−𝔼⁡\[𝐌2​𝐖a​𝚺X​1​𝐖⊤\]\\displaystyle=\\mathbf\{W\}\_\{p\}\(\\mathbf\{W\}\\mathbf\{\\Sigma\}\_\{11\}\\mathbf\{W\}^\{\\top\}\)\-\\mathbb\{E\}\[\\mathbf\{M\}\_\{2\}\\mathbf\{W\}\_\{a\}\\mathbf\{\\Sigma\}\_\{X1\}\\mathbf\{W\}^\{\\top\}\]Assuming𝐌2\\mathbf\{M\}\_\{2\}is independent and𝐖a\\mathbf\{W\}\_\{a\}is fixed:

∇𝐖pℒ=𝐖p​\(𝐖​𝚺11​𝐖⊤\)−𝐌¯2​𝐖a​𝚺X​1​𝐖⊤\\nabla\_\{\\mathbf\{W\}\_\{p\}\}\\mathcal\{L\}=\\mathbf\{W\}\_\{p\}\(\\mathbf\{W\}\\mathbf\{\\Sigma\}\_\{11\}\\mathbf\{W\}^\{\\top\}\)\-\\bar\{\\mathbf\{M\}\}\_\{2\}\\mathbf\{W\}\_\{a\}\\mathbf\{\\Sigma\}\_\{X1\}\\mathbf\{W\}^\{\\top\}

### F\.2Stationary Point Equation Derivation

We derive the fixed\-point equation for non\-trivial stationary points𝐖∗≠𝟎\\mathbf\{W\}^\{\*\}\\neq\\mathbf\{0\}\. Start from the stationary condition for𝐖\\mathbf\{W\},∇𝐖ℒ=𝟎\\nabla\_\{\\mathbf\{W\}\}\\mathcal\{L\}=\\mathbf\{0\}, using the simplified form:

𝐖p⊤​𝐖p​𝐖​𝚺11−𝐖p⊤​𝐌¯2​𝐖a​𝚺X​1=𝟎\\mathbf\{W\}\_\{p\}^\{\\top\}\\mathbf\{W\}\_\{p\}\\mathbf\{W\}\\mathbf\{\\Sigma\}\_\{11\}\-\\mathbf\{W\}\_\{p\}^\{\\top\}\\bar\{\\mathbf\{M\}\}\_\{2\}\\mathbf\{W\}\_\{a\}\\mathbf\{\\Sigma\}\_\{X1\}=\\mathbf\{0\}Substitute𝐖a=τ​𝐖\\mathbf\{W\}\_\{a\}=\\tau\\mathbf\{W\}:

𝐖p⊤​𝐖p​𝐖​𝚺11=τ​𝐖p⊤​𝐌¯2​𝐖​𝚺X​1\\mathbf\{W\}\_\{p\}^\{\\top\}\\mathbf\{W\}\_\{p\}\\mathbf\{W\}\\mathbf\{\\Sigma\}\_\{11\}=\\tau\\mathbf\{W\}\_\{p\}^\{\\top\}\\bar\{\\mathbf\{M\}\}\_\{2\}\\mathbf\{W\}\\mathbf\{\\Sigma\}\_\{X1\}Assume𝐖p\\mathbf\{W\}\_\{p\}is invertible\. Left\-multiply by\(𝐖p⊤\)−1=\(𝐖p−1\)⊤\(\\mathbf\{W\}\_\{p\}^\{\\top\}\)^\{\-1\}=\(\\mathbf\{W\}\_\{p\}^\{\-1\}\)^\{\\top\}:

𝐖p​𝐖​𝚺11=τ​𝐌¯2​𝐖​𝚺X​1\\mathbf\{W\}\_\{p\}\\mathbf\{W\}\\mathbf\{\\Sigma\}\_\{11\}=\\tau\\bar\{\\mathbf\{M\}\}\_\{2\}\\mathbf\{W\}\\mathbf\{\\Sigma\}\_\{X1\}Assume𝚺11\\mathbf\{\\Sigma\}\_\{11\}is invertible\. Right\-multiply by𝚺11−1\\mathbf\{\\Sigma\}\_\{11\}^\{\-1\}:

𝐖p​𝐖=τ​𝐌¯2​𝐖​\(𝚺X​1​𝚺11−1\)\\mathbf\{W\}\_\{p\}\\mathbf\{W\}=\\tau\\bar\{\\mathbf\{M\}\}\_\{2\}\\mathbf\{W\}\(\\mathbf\{\\Sigma\}\_\{X1\}\\mathbf\{\\Sigma\}\_\{11\}^\{\-1\}\)Define𝐀R=𝚺X​1​𝚺11−1∈ℝN×N\\mathbf\{A\}\_\{R\}=\\mathbf\{\\Sigma\}\_\{X1\}\\mathbf\{\\Sigma\}\_\{11\}^\{\-1\}\\in\\mathbb\{R\}^\{N\\times N\}\.

𝐖p​𝐖=τ​𝐌¯2​𝐖𝐀R\\mathbf\{W\}\_\{p\}\\mathbf\{W\}=\\tau\\bar\{\\mathbf\{M\}\}\_\{2\}\\mathbf\{W\}\\mathbf\{A\}\_\{R\}Assume𝐖p\\mathbf\{W\}\_\{p\}is invertible\. Left\-multiply by𝐖p−1\\mathbf\{W\}\_\{p\}^\{\-1\}:

𝐖=τ​𝐖p−1​𝐌¯2​𝐖𝐀R\\mathbf\{W\}=\\tau\\mathbf\{W\}\_\{p\}^\{\-1\}\\bar\{\\mathbf\{M\}\}\_\{2\}\\mathbf\{W\}\\mathbf\{A\}\_\{R\}Define𝐀L=𝐖p−1​𝐌¯2∈ℝd×d\\mathbf\{A\}\_\{L\}=\\mathbf\{W\}\_\{p\}^\{\-1\}\\bar\{\\mathbf\{M\}\}\_\{2\}\\in\\mathbb\{R\}^\{d\\times d\}\.

𝐖=τ​𝐀L​𝐖𝐀R\\mathbf\{W\}=\\tau\\mathbf\{A\}\_\{L\}\\mathbf\{W\}\\mathbf\{A\}\_\{R\}\(65\)
To transform this into the eigenvalue problem, apply the vectorization operatorvec⁡\(⋅\)\\vecop\(\\cdot\):

vec⁡\(𝐖\)=τ​vec⁡\(𝐀L​𝐖𝐀R\)\\vecop\(\\mathbf\{W\}\)=\\tau\\vecop\(\\mathbf\{A\}\_\{L\}\\mathbf\{W\}\\mathbf\{A\}\_\{R\}\)Using the Kronecker product propertyvec⁡\(𝐀𝐗𝐁\)=\(𝐁⊤⊗𝐀\)​vec⁡\(𝐗\)\\vecop\(\\mathbf\{A\}\\mathbf\{X\}\\mathbf\{B\}\)=\(\\mathbf\{B\}^\{\\top\}\\otimes\\mathbf\{A\}\)\\vecop\(\\mathbf\{X\}\):

vec⁡\(𝐖\)=τ⁡\(𝐀R⊤⊗𝐀L\)​vec⁡\(𝐖\)\\vecop\(\\mathbf\{W\}\)=\\tau\(\\mathbf\{A\}\_\{R\}^\{\\top\}\\otimes\\mathbf\{A\}\_\{L\}\)\\vecop\(\\mathbf\{W\}\)Rearranging yields the linear system forvec⁡\(𝐖\)\\vecop\(\\mathbf\{W\}\):

\[𝐈d​N−τ⁡\(𝐀R⊤⊗𝐀L\)\]​vec⁡\(𝐖\)=𝟎\[\\mathbf\{I\}\_\{dN\}\-\\tau\(\\mathbf\{A\}\_\{R\}^\{\\top\}\\otimes\\mathbf\{A\}\_\{L\}\)\]\\vecop\(\\mathbf\{W\}\)=\\mathbf\{0\}This system has non\-trivial solutionsvec⁡\(𝐖\)≠𝟎\\vecop\(\\mathbf\{W\}\)\\neq\\mathbf\{0\}if and only if the matrix\[𝐈d​N−τ⁡\(𝐀R⊤⊗𝐀L\)\]\[\\mathbf\{I\}\_\{dN\}\-\\tau\(\\mathbf\{A\}\_\{R\}^\{\\top\}\\otimes\\mathbf\{A\}\_\{L\}\)\]is singular\. This occurs when11is an eigenvalue ofτ⁡\(𝐀R⊤⊗𝐀L\)\\tau\(\\mathbf\{A\}\_\{R\}^\{\\top\}\\otimes\\mathbf\{A\}\_\{L\}\), or equivalently:

1τ∈Spec⁡\(𝐀R⊤⊗𝐀L\)\\frac\{1\}\{\\tau\}\\in\\Spec\(\\mathbf\{A\}\_\{R\}^\{\\top\}\\otimes\\mathbf\{A\}\_\{L\}\)This confirms the condition stated in Theorem[F\.1](https://arxiv.org/html/2610.02344#A6.Thmtheorem1)for the existence of non\-trivial stationary points\.

Conditions for Real Eigenvalues:The eigenvaluesλ\\lambdaof𝐊=𝐀R⊤⊗𝐀L\\mathbf\{K\}=\\mathbf\{A\}\_\{R\}^\{\\top\}\\otimes\\mathbf\{A\}\_\{L\}are, in general, complex numbers\. They are the products of the eigenvalues of𝐀R⊤\\mathbf\{A\}\_\{R\}^\{\\top\}\(which are the same as those of𝐀R\\mathbf\{A\}\_\{R\}\) and the eigenvalues of𝐀L\\mathbf\{A\}\_\{L\}\. The eigenvalues of𝐊\\mathbf\{K\}are guaranteed to be real if both𝐀R\\mathbf\{A\}\_\{R\}and𝐀L\\mathbf\{A\}\_\{L\}are Hermitian \(symmetric in the real case\)\.

- •𝐀R=𝚺X​1​𝚺11−1=𝔼⁡\[𝐱𝐱1⊤\]​\(𝔼⁡\[𝐱1​𝐱1⊤\]\)−1\\mathbf\{A\}\_\{R\}=\\mathbf\{\\Sigma\}\_\{X1\}\\mathbf\{\\Sigma\}\_\{11\}^\{\-1\}=\\mathbb\{E\}\[\\mathbf\{x\}\\mathbf\{x\}\_\{1\}^\{\\top\}\]\(\\mathbb\{E\}\[\\mathbf\{x\}\_\{1\}\\mathbf\{x\}\_\{1\}^\{\\top\}\]\)^\{\-1\}\.𝚺11\\mathbf\{\\Sigma\}\_\{11\}is symmetric positive semi\-definite \(assumed positive definite\)\. However,𝚺X​1\\mathbf\{\\Sigma\}\_\{X1\}is generally not symmetric, as it encodes the cross\-correlation between the full input and the masked input\. Thus,𝐀R\\mathbf\{A\}\_\{R\}is generally not symmetric\.
- •𝐀L=𝐖p−1​𝐌¯2\\mathbf\{A\}\_\{L\}=\\mathbf\{W\}\_\{p\}^\{\-1\}\\bar\{\\mathbf\{M\}\}\_\{2\}\.𝐌¯2\\bar\{\\mathbf\{M\}\}\_\{2\}is diagonal, hence symmetric\. However, the predictor matrix𝐖p\\mathbf\{W\}\_\{p\}is generally not symmetric, nor is its inverse\. Thus,𝐀L\\mathbf\{A\}\_\{L\}is generally not symmetric\.

Therefore, we must generally consider complex eigenvaluesλ\\lambda\. The stability analysis relies on the magnitude\|λ\|\|\\lambda\|, andτc​r​i​t\\tau\_\{crit\}is defined using the eigenvalue with the largest magnitude,\|λmax\|\|\\lambda\_\{\\max\}\|\. If, under specific simplifying assumptions \(e\.g\., particular data structures or predictor forms\), both𝐀L\\mathbf\{A\}\_\{L\}and𝐀R\\mathbf\{A\}\_\{R\}were symmetric, then all eigenvalues would be real, andτc​r​i​t=1/max⁡\|λi\|=1/\|λdominant\|\\tau\_\{crit\}=1/\\max\|\\lambda\_\{i\}\|=1/\|\\lambda\_\{\\text\{dominant\}\}\|, whereλdominant\\lambda\_\{\\text\{dominant\}\}is the eigenvalue with the largest absolute value\.

### F\.3The adiabatic stability theorem

Prior analyses, notably[Tian et al\. \[2021\]](https://arxiv.org/html/2610.02344#bib.bib31), often assume that the target weights𝐖a\\mathbf\{W\}\_\{a\}remain proportional to the encoder weights𝐖\\mathbf\{W\}\. This corresponds to an adiabatic limit of the Jacobian analysis of Section[3](https://arxiv.org/html/2610.02344#S3), where EMA is much faster than encoder dynamics \(β→∞\\beta\\to\\infty\)\. In this regime, illustrated in Figure[10](https://arxiv.org/html/2610.02344#A6.F10),𝐖a\\mathbf\{W\}\_\{a\}tracks𝐖\\mathbf\{W\}almost instantaneously, yielding𝐯𝐞𝐜⁡\(𝐖a\)←τ​𝐯𝐞𝐜​\(𝐖\)\\mathbf\{vec\}\(\\mathbf\{W\}\_\{a\}\)\\leftarrow\\tau\\,\\mathbf\{vec\}\(\\mathbf\{W\}\)along unstable modes\. This approximation leads to an analogous stability threshold, formalized below and derived in Appendix[F\.2](https://arxiv.org/html/2610.02344#A6.SS2)\. This bridge to prior Siamese analyses sits in the appendix because it does not motivate the predictor of Section[6](https://arxiv.org/html/2610.02344#S6)\.

![Refer to caption](https://arxiv.org/html/2610.02344v1/images/fig4_accuracy_vs_tau_sidebyside.png)Figure 10:Adiabatic stability\.Assuming the target encoder tracks the context encoder proportionally, the parameterτ\\taucan be used to avoid representational collapse also yielding better performance; see the analysis in Appendix[F\.4](https://arxiv.org/html/2610.02344#A6.SS4)\.###### Theorem F\.1\(Adiabatic Stability Condition\)\.

In the adiabatic limit, non\-trivial stationary points𝐖∗≠𝟎\\mathbf\{W\}^\{\*\}\\neq\\mathbf\{0\}exist if and only if the effective couplingτ\\tausatisfies:

1τ∈Spec⁡\(𝐀R⊤⊗𝐀L\)\\frac\{1\}\{\\tau\}\\in\\Spec\(\\mathbf\{A\}\_\{R\}^\{\\top\}\\otimes\\mathbf\{A\}\_\{L\}\)\(66\)

with𝐀R=𝚺X​1​𝚺11−1\\mathbf\{A\}\_\{R\}=\\mathbf\{\\Sigma\}\_\{X1\}\\mathbf\{\\Sigma\}\_\{11\}^\{\-1\},𝐀L=𝐖p−1​𝐌¯2\\mathbf\{A\}\_\{L\}=\\mathbf\{W\}\_\{p\}^\{\-1\}\\bar\{\\mathbf\{M\}\}\_\{2\}and assuming𝐖𝐚←τ​𝐖\\mathbf\{W\_\{a\}\}\\leftarrow\\tau\\mathbf\{W\}\. This allows us to define the critical valueτcrit:=1\|λmax​\(𝐀L\)\|​\|λmax​\(𝐀R\)\|\\tau\_\{\\text\{crit\}\}:=\\frac\{1\}\{\|\\lambda\_\{\\max\}\(\\mathbf\{A\}\_\{L\}\)\|\|\\lambda\_\{\\max\}\(\\mathbf\{A\}\_\{R\}\)\|\}that also defines a region of better downstream performance, as the next subsection shows\.

### F\.4The Breaking Point: When JEPA Escapes Collapse

#### F\.4\.1Bifurcation Phenomenon

The bifurcation behavior in JEPA training dynamics stems from the spectral condition stated in Theorem[F\.1](https://arxiv.org/html/2610.02344#A6.Thmtheorem1), which shows that non\-trivial stationary points emerge if and only if1τ∈Spec⁡\(𝐀R⊤⊗𝐀L\)\\frac\{1\}\{\\tau\}\\in\\Spec\(\\mathbf\{A\}\_\{R\}^\{\\top\}\\otimes\\mathbf\{A\}\_\{L\}\)\. Since the spectrum of the Kronecker product satisfiesSpec⁡\(𝐀R⊤⊗𝐀L\)=\{λi​μj\}\\Spec\(\\mathbf\{A\}\_\{R\}^\{\\top\}\\otimes\\mathbf\{A\}\_\{L\}\)=\\\{\\lambda\_\{i\}\\mu\_\{j\}\\\}, whereλi∈Spec⁡\(𝐀R⊤\)\\lambda\_\{i\}\\in\\Spec\(\\mathbf\{A\}\_\{R\}^\{\\top\}\)andμj∈Spec⁡\(𝐀L\)\\mu\_\{j\}\\in\\Spec\(\\mathbf\{A\}\_\{L\}\), the threshold for bifurcation is governed by the largest eigenvalue of𝐀L\\mathbf\{A\}\_\{L\}and𝐀R\\mathbf\{A\}\_\{R\}\. We define the critical value:

τcrit:=1\|λmax​\(𝐀L\)\|​\|λmax​\(𝐀R\)\|,\\tau\_\{\\text\{crit\}\}\\;:=\\;\\frac\{1\}\{\|\\lambda\_\{\\max\}\(\\mathbf\{A\}\_\{L\}\)\|\|\\lambda\_\{\\max\}\(\\mathbf\{A\}\_\{R\}\)\|\}\\,,\(67\)which marks the bifurcation point: forτ<τcrit\\tau<\\tau\_\{\\text\{crit\}\}, the only stationary point is the trivial solution𝐖=𝟎\\mathbf\{W\}=\\mathbf\{0\}, while forτ\>τcrit\\tau\>\\tau\_\{\\text\{crit\}\}, non\-trivial solutions𝐖≠𝟎\\mathbf\{W\}\\neq\\mathbf\{0\}emerge, indicating a qualitative change in the structure of the loss landscape\.

![Refer to caption](https://arxiv.org/html/2610.02344v1/images/jepa_collapse_metrics_vs_tau_fixed_predictor_mean_std.png)Figure 11:Bifurcation Phenomenon\.Empirical analysis of JEPA collapse metrics as a function of the parameterτ\\tau\. The plot displays the mean Frobenius Norm \(right axis\) and mean Condition Number \(left axis, log scale\) of the encoder weights𝐖\\mathbf\{W\}, averaged over 20 experimental runs\. The shaded area represents the standard deviation\. A critical threshold, empirically determined atτc​r​i​t≈0\.551\\tau\_\{crit\}\\approx 0\.551, delineates the transition between the collapsed and non\-collapsed operational regimes\.Empirical evidence supporting this theoretical framework is presented in Figure[11](https://arxiv.org/html/2610.02344#A6.F11)\. This figure illustrates the behavior of key collapse metrics, specifically, the Frobenius norm‖𝐖‖F\|\|\\mathbf\{W\}\|\|\_\{F\}and the condition numberκ⁡\(𝐖\)\\kappa\(\\mathbf\{W\}\)\(the ratio of the largest to smallest singular value,σmax​\(𝐖∗\)/σmin​\(𝐖∗\)\\sigma\_\{\\text\{max\}\}\(\\mathbf\{W\}^\{\*\}\)/\\sigma\_\{\\text\{min\}\}\(\\mathbf\{W\}^\{\*\}\)\) of the encoder weights𝐖\\mathbf\{W\}, as a function of the parameterτ\\tau\.

The behavior of these metrics across different regimes ofτ\\taucan be delineated as follows:

- •Subcritical Regime \(τ<τc​r​i​t\\tau<\\tau\_\{crit\}\):The encoder converges to the trivial solution𝐖≈𝟎\\mathbf\{W\}\\approx\\mathbf\{0\}\. - –Frobenius norm remains close to zero\(‖𝐖‖F≈0\)\\big\(\|\|\\mathbf\{W\}\|\|\_\{F\}\\approx 0\\big\), indicating no meaningful transformation is learned\. - –Condition number is extremely high or undefined, reflecting that𝐖\\mathbf\{W\}is ill\-conditioned or nearly singular\.
- •Supercritical Regime \(τ\>τc​r​i​t\\tau\>\\tau\_\{crit\}\):Non\-trivial solutions𝐖∗≠𝟎\\mathbf\{W\}^\{\*\}\\neq\\mathbf\{0\}emerge and the encoder escapes collapse\. - –Frobenius norm increases sharply, signaling effective learning of structured representations\. - –Condition number stabilizes at moderate values, indicating a well\-conditioned matrix that preserves feature diversity\.

### F\.5Loss Landscape Transformation

To further elucidate the structural changes in the JEPA loss landscape as a consequence of the bifurcation atτcrit\\tau\_\{\\text\{crit\}\}\(Eq\.[67](https://arxiv.org/html/2610.02344#A6.E67)\), Figure[12](https://arxiv.org/html/2610.02344#A6.F12)provides a conceptual, two\-dimensional projection\. These visualizations illustrate the loss functionℒ\\mathcal\{L\}over a simplified parameter space of the context encoder matrix𝐖\\mathbf\{W\}\. This space is spanned byr1r\_\{1\}andr2r\_\{2\}\. These factors scale two distinct groups of eigenvalues of the encoder matrix𝐖\\mathbf\{W\}:r1r\_\{1\}scales the group of eigenvalues that are inherently larger, whiler2r\_\{2\}scales the group of eigenvalues that are inherently smaller\. This parameterization allows for exploration of changes in𝐖\\mathbf\{W\}’s overall norm and its spectral shape\. The figure contrasts the landscape topology for parameter regimes below and above an empirically determined critical valueτcrit≈0\.400\\tau\_\{\\text\{crit\}\}\\approx 0\.400for this specific JEPA\-like setup\.

![Refer to caption](https://arxiv.org/html/2610.02344v1/images/loss_landscape.png)Figure 12:Loss landscape projection\.Two\-dimensional projection of a JEPA\-like loss landscape in the Jannis dataset\. An empirical critical value ofτcrit≈0\.400\\tau\_\{\\text\{crit\}\}\\approx 0\.400is noted for this illustration\. \(Left\) Subcritical regime \(τ=0\.200<τcrit\\tau=0\.200<\\tau\_\{\\text\{crit\}\}\): The loss landscape exhibits a global minimum closer to the origin \(r1≈0,r2≈0r\_\{1\}\\approx 0,r\_\{2\}\\approx 0\), indicating the stability of the collapsed solution𝐖≈𝟎\\mathbf\{W\}\\approx\\mathbf\{0\}\. \(Right\) Supercritical regime \(τ=0\.600\>τcrit\\tau=0\.600\>\\tau\_\{\\text\{crit\}\}\): The origin has transformed into a region of higher loss, and new, distinct minima withℒ\>0\\mathcal\{L\}\>0have emerged away from the origin\.The key observations from these illustrative landscapes are:

- •In the subcritical regime \(Figure[12](https://arxiv.org/html/2610.02344#A6.F12), Left\), whereτ=0\.200<τcrit\\tau=0\.200<\\tau\_\{\\text\{crit\}\}: The loss functionℒ\\mathcal\{L\}displays a distinct global minimum at the origin \(r1≈0,r2≈0r\_\{1\}\\approx 0,r\_\{2\}\\approx 0\)\. This topology signifies that any optimization trajectory, including one starting from the originalWW\(r1=1,r2=1r\_\{1\}=1,r\_\{2\}=1\), is naturally guided towards the collapsed state where the online encoder𝐖≈𝟎\\mathbf\{W\}\\approx\\mathbf\{0\}\. The landscape is characteristic of a system where this trivial solution is the most stable configuration\.
- •In the supercritical regime \(Figure[12](https://arxiv.org/html/2610.02344#A6.F12), Right\), whereτ=0\.600\>τcrit\\tau=0\.600\>\\tau\_\{\\text\{crit\}\}: A qualitative transformation of the loss landscape is evident\. The origin \(r1≈0,r2≈0r\_\{1\}\\approx 0,r\_\{2\}\\approx 0\) no longer corresponds to a minimum loss value; instead, it has acquired characteristics of an unstable point, exhibiting a significantly higher loss compared to newly formed minima\. These new minima, corresponding to non\-trivial solutions𝐖∗≠𝟎\\mathbf\{W\}^\{\*\}\\neq\\mathbf\{0\}, are now located at finite distances from the origin\.

These visualizations effectively demonstrate the bifurcation phenomenon’s impact on the loss landscape\. The transition from a landscape dominated by a single minimum at the origin to one featuring multiple, off\-origin minima asτ\\taucrossesτcrit\\tau\_\{\\text\{crit\}\}is fundamental to understanding how JEPA and similar architectures can avoid representational collapse and learn useful features\.

### F\.6The criterion under joint predictor training

A JEPA trains the predictor together with the encoder\. This section shows that the fixed\-predictor criterion remains the exact local criterion under joint training, and bounds the predictor’s drift until the encoder escapes\.

#### F\.6\.1The collapsed set is a manifold of equilibria

Write the full flow on\(𝐖,𝐖a,𝐖p\)\(\\mathbf\{W\},\\mathbf\{W\}\_\{a\},\\mathbf\{W\}\_\{p\}\)with predictor learning rateηp\\eta\_\{p\}, in units of the encoder learning rate\. The predictor gradient of Appendix[F](https://arxiv.org/html/2610.02344#A6)is

∇𝐖pℒ=𝐖p​\(𝐖​𝚺11​𝐖⊤\)−𝐌¯2​𝐖a​𝚺X​1​𝐖⊤\.\\nabla\_\{\\mathbf\{W\}\_\{p\}\}\\mathcal\{L\}=\\mathbf\{W\}\_\{p\}\\bigl\(\\mathbf\{W\}\\mathbf\{\\Sigma\}\_\{11\}\\mathbf\{W\}^\{\\top\}\\bigr\)\-\\bar\{\\mathbf\{M\}\}\_\{2\}\\mathbf\{W\}\_\{a\}\\mathbf\{\\Sigma\}\_\{X1\}\\mathbf\{W\}^\{\\top\}\.\(68\)
###### Lemma F\.2\(Equilibria, block structure, and speed\)\.

Letℰ:=\{\(𝐖,𝐖a,𝐖p\):𝐖=𝟎,𝐖a=𝟎\}\\mathcal\{E\}:=\\\{\(\\mathbf\{W\},\\mathbf\{W\}\_\{a\},\\mathbf\{W\}\_\{p\}\):\\mathbf\{W\}=\\mathbf\{0\},\\ \\mathbf\{W\}\_\{a\}=\\mathbf\{0\}\\\}\. Then

1. \(i\)ℰ\\mathcal\{E\}is ad2d^\{2\}\-dimensional manifold of equilibria of the full flow;
2. \(ii\)the linearisation at any point ofℰ\\mathcal\{E\}is block diagonal,\(𝒥⁡\(𝐖p0\)00𝟎d2×d2\)\\begin\{pmatrix\}\\mathcal\{J\}\(\\mathbf\{W\}\_\{p\}^\{0\}\)&0\\\\ 0&\\mathbf\{0\}\_\{d^\{2\}\\times d^\{2\}\}\\end\{pmatrix\}, for everyηp\\eta\_\{p\}, so Eq\. \([3](https://arxiv.org/html/2610.02344#S3.E3)\) is the exact linearisation of the coupled system and thed2d^\{2\}zero eigenvalues are the neutral directions tangent toℰ\\mathcal\{E\};
3. \(iii\)at any state with‖𝐖p‖2≤w\\\|\\mathbf\{W\}\_\{p\}\\\|\_\{2\}\\leq wandmax⁡\(‖𝐖‖F,‖𝐖a‖F\)≤ε\\max\(\\\|\\mathbf\{W\}\\\|\_\{F\},\\\|\\mathbf\{W\}\_\{a\}\\\|\_\{F\}\)\\leq\\varepsilonthe predictor’s speed obeys‖𝐖˙p‖F≤ηp​\(w​sΣ\+pt​g​t​sX\)​ε2\\\|\\dot\{\\mathbf\{W\}\}\_\{p\}\\\|\_\{F\}\\leq\\eta\_\{p\}\\,\(w\\,s\_\{\\Sigma\}\+p\_\{tgt\}s\_\{X\}\)\\,\\varepsilon^\{2\}, against‖𝐖˙‖F≤w⁡\(w​sΣ\+pt​g​t​sX\)​ε\\\|\\dot\{\\mathbf\{W\}\}\\\|\_\{F\}\\leq w\(ws\_\{\\Sigma\}\+p\_\{tgt\}s\_\{X\}\)\\varepsilon, wheresΣ=‖𝚺11‖2s\_\{\\Sigma\}=\\\|\\mathbf\{\\Sigma\}\_\{11\}\\\|\_\{2\},sX=‖𝚺X​1‖2s\_\{X\}=\\\|\\mathbf\{\\Sigma\}\_\{X1\}\\\|\_\{2\}andpt​g​t=‖𝐌¯2‖2p\_\{tgt\}=\\\|\\bar\{\\mathbf\{M\}\}\_\{2\}\\\|\_\{2\}\.

###### Proof\.

\(i\) Every monomial of the three fields contains at least one factor𝐖\\mathbf\{W\}or𝐖a\\mathbf\{W\}\_\{a\}, so all three vanish at𝐖=𝐖a=𝟎\\mathbf\{W\}=\\mathbf\{W\}\_\{a\}=\\mathbf\{0\}for every𝐖p\\mathbf\{W\}\_\{p\}\.

\(ii\) Differentiate the three components at\(𝟎,𝟎,𝐖p0\)\(\\mathbf\{0\},\\mathbf\{0\},\\mathbf\{W\}\_\{p\}^\{0\}\)\. The partial of the encoder field in\(𝐖,𝐖a\)\(\\mathbf\{W\},\\mathbf\{W\}\_\{a\}\)vectorises to𝒥⁡\(𝐖p0\)\\mathcal\{J\}\(\\mathbf\{W\}\_\{p\}^\{0\}\); its partial in𝐖p\\mathbf\{W\}\_\{p\}is−\(δ​𝐖p⊤​𝐖p0\+\(𝐖p0\)⊤​δ​𝐖p\)​𝐖​𝚺11\+δ​𝐖p⊤​𝐌¯2​𝐖a​𝚺X​1\-\(\\delta\\mathbf\{W\}\_\{p\}^\{\\top\}\\mathbf\{W\}\_\{p\}^\{0\}\+\(\\mathbf\{W\}\_\{p\}^\{0\}\)^\{\\top\}\\delta\\mathbf\{W\}\_\{p\}\)\\mathbf\{W\}\\mathbf\{\\Sigma\}\_\{11\}\+\\delta\\mathbf\{W\}\_\{p\}^\{\\top\}\\bar\{\\mathbf\{M\}\}\_\{2\}\\mathbf\{W\}\_\{a\}\\mathbf\{\\Sigma\}\_\{X1\}, which vanishes at the base point, killing the upper\-right block\. The target field isβ⁡\(δ​𝐖−δ​𝐖a\)\\beta\(\\delta\\mathbf\{W\}\-\\delta\\mathbf\{W\}\_\{a\}\)and carries no𝐖p\\mathbf\{W\}\_\{p\}dependence\. Every term of the predictor field is at least bilinear in\(𝐖,𝐖a\)\(\\mathbf\{W\},\\mathbf\{W\}\_\{a\}\)by Eq\. \([68](https://arxiv.org/html/2610.02344#A6.E68)\), so every term of its derivative keeps a factor𝐖\\mathbf\{W\}or𝐖a\\mathbf\{W\}\_\{a\}and vanishes; both of its partials are zero, for everyηp\\eta\_\{p\}, which kills the bottom row\.

\(iii\) Both summands of Eq\. \([68](https://arxiv.org/html/2610.02344#A6.E68)\) are bilinear in\(𝐖,𝐖a\)\(\\mathbf\{W\},\\mathbf\{W\}\_\{a\}\), and bounding each in the spectral norm gives the stated inequality\. ∎

#### F\.6\.2How far the predictor drifts before the encoder escapes

Write𝐳=\[vec​\(𝐖\)⊤,vec​\(𝐖a\)⊤\]⊤\\mathbf\{z\}=\[\\mathrm\{vec\}\(\\mathbf\{W\}\)^\{\\top\},\\mathrm\{vec\}\(\\mathbf\{W\}\_\{a\}\)^\{\\top\}\]^\{\\top\}with‖𝐳‖2=‖𝐖‖F2\+‖𝐖a‖F2\\\|\\mathbf\{z\}\\\|^\{2\}=\\\|\\mathbf\{W\}\\\|\_\{F\}^\{2\}\+\\\|\\mathbf\{W\}\_\{a\}\\\|\_\{F\}^\{2\},ε:=‖𝐳⁡\(0\)‖\\varepsilon:=\\\|\\mathbf\{z\}\(0\)\\\|,𝒥0:=𝒥⁡\(𝐖p0\)\\mathcal\{J\}\_\{0\}:=\\mathcal\{J\}\(\\mathbf\{W\}\_\{p\}^\{0\}\),𝐃⁡\(t\):=𝐖p​\(t\)−𝐖p0\\mathbf\{D\}\(t\):=\\mathbf\{W\}\_\{p\}\(t\)\-\\mathbf\{W\}\_\{p\}^\{0\}andρ⁡\(t\):=‖𝐃⁡\(t\)‖F\\rho\(t\):=\\\|\\mathbf\{D\}\(t\)\\\|\_\{F\}\. Then𝐳˙=𝒥​\(𝐖p​\(t\)\)​𝐳\\dot\{\\mathbf\{z\}\}=\\mathcal\{J\}\(\\mathbf\{W\}\_\{p\}\(t\)\)\\,\\mathbf\{z\}exactly\. The result below assumes the exact form of Assumption[3\.2](https://arxiv.org/html/2610.02344#S3.Thmtheorem2)with a real drive, at initialisation only: orthogonal𝐔\\mathbf\{U\}and𝐕\\mathbf\{V\}with𝚺11=𝐔​diag​\(ak\)​𝐔⊤\\mathbf\{\\Sigma\}\_\{11\}=\\mathbf\{U\}\\mathrm\{diag\}\(a\_\{k\}\)\\mathbf\{U\}^\{\\top\},𝚺X​1=𝐔​diag​\(ck\)​𝐔⊤\\mathbf\{\\Sigma\}\_\{X1\}=\\mathbf\{U\}\\mathrm\{diag\}\(c\_\{k\}\)\\mathbf\{U\}^\{\\top\},𝐖p0⊤​𝐖p0=𝐕​diag​\(pℓ\)​𝐕⊤\\mathbf\{W\}\_\{p\}^\{0\\top\}\\mathbf\{W\}\_\{p\}^\{0\}=\\mathbf\{V\}\\mathrm\{diag\}\(p\_\{\\ell\}\)\\mathbf\{V\}^\{\\top\}and𝐖p0⊤​𝐌¯2=𝐕​diag​\(qℓ\)​𝐕⊤\\mathbf\{W\}\_\{p\}^\{0\\top\}\\bar\{\\mathbf\{M\}\}\_\{2\}=\\mathbf\{V\}\\mathrm\{diag\}\(q\_\{\\ell\}\)\\mathbf\{V\}^\{\\top\}, whereak\>0a\_\{k\}\>0,pℓ\>0p\_\{\\ell\}\>0andck,qℓc\_\{k\},q\_\{\\ell\}are real\. Then𝒥0\\mathcal\{J\}\_\{0\}splits into the2×22\\times 2blocks\(−σk​ℓγk​ℓβ−β\)\\begin\{pmatrix\}\-\\sigma\_\{k\\ell\}&\\gamma\_\{k\\ell\}\\\\ \\beta&\-\\beta\\end\{pmatrix\}withσk​ℓ=ak​pℓ\\sigma\_\{k\\ell\}=a\_\{k\}p\_\{\\ell\},γk​ℓ=ck​qℓ\\gamma\_\{k\\ell\}=c\_\{k\}q\_\{\\ell\}and discriminantΔk​ℓ=\(β−σk​ℓ\)2\+4​β​γk​ℓ\\Delta\_\{k\\ell\}=\(\\beta\-\\sigma\_\{k\\ell\}\)^\{2\}\+4\\beta\\gamma\_\{k\\ell\}\. Decoupling is not imposed att\>0t\>0, so the statements below concern the spectrum of𝒥​\(𝐖p​\(t\)\)\\mathcal\{J\}\(\\mathbf\{W\}\_\{p\}\(t\)\)\.

###### Theorem F\.3\(Drift bound\)\.

Assume exact decoupling at initialisation as above,Δk​ℓ≠0\\Delta\_\{k\\ell\}\\neq 0for every mode, and𝒰≠∅\\mathcal\{U\}\\neq\\emptyset\. Letλ∗\>0\\lambda\_\{\*\}\>0be the spectral abscissa of𝒥0\\mathcal\{J\}\_\{0\}\. Let𝐎¯\\bar\{\\mathbf\{O\}\}be the orthogonal change of basis that block\-diagonalises𝒥0\\mathcal\{J\}\_\{0\}, let𝐗^\\widehat\{\\mathbf\{X\}\}be the block\-diagonal similarity that brings each block to diagonal form with unit\-norm columns \(real blocks\) or to real rotation form \(complex blocks\), and setκ:=‖𝐗^‖2​‖𝐗^−1‖2\\kappa:=\\\|\\widehat\{\\mathbf\{X\}\}\\\|\_\{2\}\\\|\\widehat\{\\mathbf\{X\}\}^\{\-1\}\\\|\_\{2\}\. With𝐲⁡\(0\):=𝐗^−1​𝐎¯⊤​𝐳​\(0\)\\mathbf\{y\}\(0\):=\\widehat\{\\mathbf\{X\}\}^\{\-1\}\\bar\{\\mathbf\{O\}\}^\{\\top\}\\mathbf\{z\}\(0\)and𝚷∗\\mathbf\{\\Pi\}\_\{\*\}the coordinate projection onto the eigenvalueλ∗\\lambda\_\{\*\}, letα:=‖𝚷∗​𝐲​\(0\)‖/‖𝐲⁡\(0\)‖\>0\\alpha:=\\\|\\mathbf\{\\Pi\}\_\{\*\}\\mathbf\{y\}\(0\)\\\|/\\\|\\mathbf\{y\}\(0\)\\\|\>0be the alignment of the initialisation\. SetCp:=2​w0​sΣ\+pt​g​t​sXC\_\{p\}:=2w\_\{0\}s\_\{\\Sigma\}\+p\_\{tgt\}s\_\{X\}andLE:=3​w0​sΣ\+pt​g​t​sXL\_\{E\}:=3w\_\{0\}s\_\{\\Sigma\}\+p\_\{tgt\}s\_\{X\}withw0=‖𝐖p0‖2w\_\{0\}=\\\|\\mathbf\{W\}\_\{p\}^\{0\}\\\|\_\{2\}andpt​g​t=‖𝐌¯2‖2p\_\{tgt\}=\\\|\\bar\{\\mathbf\{M\}\}\_\{2\}\\\|\_\{2\},Lα:=log⁡\(4​κ2/α\)L\_\{\\alpha\}:=\\log\(4\\kappa^\{2\}/\\alpha\), and

ρ∞:=ηp​Cp​\(1\+Lα\)λ∗​δ2,T¯:=1λ∗​log⁡4​κ​δα​ε\.\\rho\_\{\\infty\}:=\\frac\{\\eta\_\{p\}\\,C\_\{p\}\\,\(1\+L\_\{\\alpha\}\)\}\{\\lambda\_\{\*\}\}\\,\\delta^\{2\},\\qquad\\bar\{T\}:=\\frac\{1\}\{\\lambda\_\{\*\}\}\\log\\frac\{4\\kappa\\delta\}\{\\alpha\\varepsilon\}\.\(69\)If \(C0\)κ​ε≤δ\\kappa\\varepsilon\\leq\\delta, \(C1\)2​ρ∞≤w02\\rho\_\{\\infty\}\\leq w\_\{0\}and \(C2\)2​κ​LE​ρ∞​T¯≤log⁡\(1\+α/2\)2\\kappa L\_\{E\}\\rho\_\{\\infty\}\\bar\{T\}\\leq\\log\(1\+\\alpha/2\), then the first timeTδT\_\{\\delta\}at which‖𝐳‖\\\|\\mathbf\{z\}\\\|exceedsδ\\deltasatisfiesTδ≤T¯T\_\{\\delta\}\\leq\\bar\{T\}, and

‖𝐖p​\(t\)−𝐖p0‖F≤ρ∞for all​t≤Tδ\.\\bigl\\\|\\mathbf\{W\}\_\{p\}\(t\)\-\\mathbf\{W\}\_\{p\}^\{0\}\\bigr\\\|\_\{F\}\\;\\leq\\;\\rho\_\{\\infty\}\\qquad\\text\{for all \}t\\leq T\_\{\\delta\}\.\(70\)The bound does not depend on the initialisation scaleε\\varepsilon\.

###### Proof\.

Writeρ¯:=2​ρ∞\\bar\{\\rho\}:=2\\rho\_\{\\infty\}andc:=κ​LE​ρ¯c:=\\kappa L\_\{E\}\\bar\{\\rho\}\. We use‖𝐀𝐁𝐂‖F≤‖𝐀‖2​‖𝐁‖F​‖𝐂‖2\\\|\\mathbf\{A\}\\mathbf\{B\}\\mathbf\{C\}\\\|\_\{F\}\\leq\\\|\\mathbf\{A\}\\\|\_\{2\}\\\|\\mathbf\{B\}\\\|\_\{F\}\\\|\\mathbf\{C\}\\\|\_\{2\}and‖𝐀⊗𝐁‖2=‖𝐀‖2​‖𝐁‖2\\\|\\mathbf\{A\}\\otimes\\mathbf\{B\}\\\|\_\{2\}=\\\|\\mathbf\{A\}\\\|\_\{2\}\\\|\\mathbf\{B\}\\\|\_\{2\}\.

Step 0, coordinates\. Let𝚲:=𝐗^−1​𝐎¯⊤​𝒥0​𝐎¯​𝐗^\\mathbf\{\\Lambda\}:=\\widehat\{\\mathbf\{X\}\}^\{\-1\}\\bar\{\\mathbf\{O\}\}^\{\\top\}\\mathcal\{J\}\_\{0\}\\bar\{\\mathbf\{O\}\}\\widehat\{\\mathbf\{X\}\}\. Every block of𝚲\\mathbf\{\\Lambda\}is normal and the largest real part of its spectrum isλ∗\\lambda\_\{\*\}, so‖e𝚲​t‖2=eλ∗​t\\\|e^\{\\mathbf\{\\Lambda\}t\}\\\|\_\{2\}=e^\{\\lambda\_\{\*\}t\}fort≥0t\\geq 0\. With𝐲⁡\(t\):=𝐗^−1​𝐎¯⊤​𝐳​\(t\)\\mathbf\{y\}\(t\):=\\widehat\{\\mathbf\{X\}\}^\{\-1\}\\bar\{\\mathbf\{O\}\}^\{\\top\}\\mathbf\{z\}\(t\)and𝐄⁡\(t\):=𝒥⁡\(𝐖p​\(t\)\)−𝒥0\\mathbf\{E\}\(t\):=\\mathcal\{J\}\(\\mathbf\{W\}\_\{p\}\(t\)\)\-\\mathcal\{J\}\_\{0\}and𝐄~​\(t\):=𝐗^−1​𝐎¯⊤​𝐄​\(t\)​𝐎¯​𝐗^\\widetilde\{\\mathbf\{E\}\}\(t\):=\\widehat\{\\mathbf\{X\}\}^\{\-1\}\\bar\{\\mathbf\{O\}\}^\{\\top\}\\mathbf\{E\}\(t\)\\bar\{\\mathbf\{O\}\}\\widehat\{\\mathbf\{X\}\},

𝐲˙=𝚲​𝐲\+𝐄~​\(t\)​𝐲,‖𝐄~​\(t\)‖2≤κ​‖𝐄⁡\(t\)‖2,ε/‖𝐗^‖2≤‖𝐲⁡\(0\)‖≤‖𝐗^−1‖2​ε\.\\dot\{\\mathbf\{y\}\}=\\mathbf\{\\Lambda\}\\mathbf\{y\}\+\\widetilde\{\\mathbf\{E\}\}\(t\)\\mathbf\{y\},\\qquad\\\|\\widetilde\{\\mathbf\{E\}\}\(t\)\\\|\_\{2\}\\leq\\kappa\\\|\\mathbf\{E\}\(t\)\\\|\_\{2\},\\qquad\\varepsilon/\\\|\\widehat\{\\mathbf\{X\}\}\\\|\_\{2\}\\leq\\\|\\mathbf\{y\}\(0\)\\\|\\leq\\\|\\widehat\{\\mathbf\{X\}\}^\{\-1\}\\\|\_\{2\}\\,\\varepsilon\.\(71\)
Step 1, perturbation of the Jacobian\. Ifρ⁡\(t\)≤ρ¯≤w0\\rho\(t\)\\leq\\bar\{\\rho\}\\leq w\_\{0\}, then‖𝐖p​\(t\)‖2≤2​w0\\\|\\mathbf\{W\}\_\{p\}\(t\)\\\|\_\{2\}\\leq 2w\_\{0\}, and since𝐖p⊤​𝐖p−𝐖p0⊤​𝐖p0=𝐖p⊤​𝐃\+𝐃⊤​𝐖p0\\mathbf\{W\}\_\{p\}^\{\\top\}\\mathbf\{W\}\_\{p\}\-\\mathbf\{W\}\_\{p\}^\{0\\top\}\\mathbf\{W\}\_\{p\}^\{0\}=\\mathbf\{W\}\_\{p\}^\{\\top\}\\mathbf\{D\}\+\\mathbf\{D\}^\{\\top\}\\mathbf\{W\}\_\{p\}^\{0\},

‖𝐄⁡\(t\)‖2≤sΣ​‖𝐖p⊤​𝐖p−𝐖p0⊤​𝐖p0‖2\+sX​pt​g​t​‖𝐃‖2≤LE​ρ​\(t\)\.\\\|\\mathbf\{E\}\(t\)\\\|\_\{2\}\\leq s\_\{\\Sigma\}\\,\\\|\\mathbf\{W\}\_\{p\}^\{\\top\}\\mathbf\{W\}\_\{p\}\-\\mathbf\{W\}\_\{p\}^\{0\\top\}\\mathbf\{W\}\_\{p\}^\{0\}\\\|\_\{2\}\+s\_\{X\}\\,p\_\{tgt\}\\,\\\|\\mathbf\{D\}\\\|\_\{2\}\\leq L\_\{E\}\\,\\rho\(t\)\.\(72\)
Step 2, drift generator\. Under the same condition, Lemma[F\.2](https://arxiv.org/html/2610.02344#A6.Thmtheorem2)\(iii\) withw=2​w0w=2w\_\{0\}andε\\varepsilonreplaced by‖𝐳⁡\(t\)‖\\\|\\mathbf\{z\}\(t\)\\\|, which bounds both‖𝐖‖F\\\|\\mathbf\{W\}\\\|\_\{F\}and‖𝐖a‖F\\\|\\mathbf\{W\}\_\{a\}\\\|\_\{F\}, gives‖𝐖˙p‖F≤ηp​Cp​‖𝐳⁡\(t\)‖2\\\|\\dot\{\\mathbf\{W\}\}\_\{p\}\\\|\_\{F\}\\leq\\eta\_\{p\}C\_\{p\}\\\|\\mathbf\{z\}\(t\)\\\|^\{2\}, henceρ⁡\(t\)≤ηp​Cp​∫0t‖𝐳⁡\(s\)‖2​𝑑s\\rho\(t\)\\leq\\eta\_\{p\}C\_\{p\}\\int\_\{0\}^\{t\}\\\|\\mathbf\{z\}\(s\)\\\|^\{2\}\\,ds\.

Step 3, bootstrap\. Letτ:=min⁡\(Tδ,T¯\)\\tau:=\\min\(T\_\{\\delta\},\\bar\{T\}\)and letℬ\\mathcal\{B\}be the set ofT∈\[0,τ\]T\\in\[0,\\tau\]withsup\[0,T\]ρ≤ρ¯\\sup\_\{\[0,T\]\}\\rho\\leq\\bar\{\\rho\}\. It contains00and is closed by continuity\. FixT∈ℬT\\in\\mathcal\{B\}\. On\[0,T\]\[0,T\], Steps 1 and 2 apply and‖𝐄~‖2≤c\\\|\\widetilde\{\\mathbf\{E\}\}\\\|\_\{2\}\\leq c\. By Duhamel’s formula,ϕ⁡\(t\):=e−λ∗​t​‖𝐲⁡\(t\)‖\\phi\(t\):=e^\{\-\\lambda\_\{\*\}t\}\\\|\\mathbf\{y\}\(t\)\\\|obeysϕ⁡\(t\)≤ϕ⁡\(0\)\+c​∫0tϕ⁡\(s\)​𝑑s\\phi\(t\)\\leq\\phi\(0\)\+c\\int\_\{0\}^\{t\}\\phi\(s\)\\,ds, and Grönwall’s inequality gives‖𝐲⁡\(t\)‖≤‖𝐲⁡\(0\)‖​eλ~​t\\\|\\mathbf\{y\}\(t\)\\\|\\leq\\\|\\mathbf\{y\}\(0\)\\\|e^\{\\tilde\{\\lambda\}t\}withλ~:=λ∗\+c\\tilde\{\\lambda\}:=\\lambda\_\{\*\}\+c, so‖𝐳⁡\(t\)‖≤κ​ε​eλ~​t\\\|\\mathbf\{z\}\(t\)\\\|\\leq\\kappa\\varepsilon e^\{\\tilde\{\\lambda\}t\}\. Also‖𝐳‖≤δ\\\|\\mathbf\{z\}\\\|\\leq\\deltaon\[0,T\]\[0,T\]\. PutL:=log⁡\(δ/\(κ​ε\)\)≥0L:=\\log\(\\delta/\(\\kappa\\varepsilon\)\)\\geq 0by \(C0\) ands∗:=L/λ~s^\{\*\}:=L/\\tilde\{\\lambda\}, and split the drift integral ats∗s^\{\*\}:

∫0t‖𝐳‖2​𝑑s≤∫0s∗κ2​ε2​e2​λ~​s​𝑑s\+δ2​\(t−s∗\)\+≤δ22​λ~\+δ2​\(t−s∗\)\+\.\\int\_\{0\}^\{t\}\\\|\\mathbf\{z\}\\\|^\{2\}\\,ds\\leq\\int\_\{0\}^\{s^\{\*\}\}\\kappa^\{2\}\\varepsilon^\{2\}e^\{2\\tilde\{\\lambda\}s\}\\,ds\+\\delta^\{2\}\(t\-s^\{\*\}\)\_\{\+\}\\leq\\frac\{\\delta^\{2\}\}\{2\\tilde\{\\lambda\}\}\+\\delta^\{2\}\(t\-s^\{\*\}\)\_\{\+\}\.\(73\)SinceT¯=\(L\+Lα\)/λ∗\\bar\{T\}=\(L\+L\_\{\\alpha\}\)/\\lambda\_\{\*\}and \(C2\) givesc≤λ∗​log⁡\(1\+α/2\)/\(L\+Lα\)c\\leq\\lambda\_\{\*\}\\log\(1\+\\alpha/2\)/\(L\+L\_\{\\alpha\}\),

\(t−s∗\)\+≤T¯−s∗=L​cλ∗​λ~\+Lαλ∗≤log⁡\(1\+α/2\)λ~\+Lαλ∗≤1/2\+Lαλ∗,\(t\-s^\{\*\}\)\_\{\+\}\\leq\\bar\{T\}\-s^\{\*\}=\\frac\{L\\,c\}\{\\lambda\_\{\*\}\\tilde\{\\lambda\}\}\+\\frac\{L\_\{\\alpha\}\}\{\\lambda\_\{\*\}\}\\leq\\frac\{\\log\(1\+\\alpha/2\)\}\{\\tilde\{\\lambda\}\}\+\\frac\{L\_\{\\alpha\}\}\{\\lambda\_\{\*\}\}\\leq\\frac\{1/2\+L\_\{\\alpha\}\}\{\\lambda\_\{\*\}\},\(74\)usingλ~≥λ∗\\tilde\{\\lambda\}\\geq\\lambda\_\{\*\}andlog⁡\(1\+α/2\)≤log⁡32<12\\log\(1\+\\alpha/2\)\\leq\\log\\tfrac\{3\}\{2\}<\\tfrac\{1\}\{2\}forα≤1\\alpha\\leq 1\. Thereforeρ⁡\(t\)≤ηp​Cp​δ2​\(1\+Lα\)/λ∗=ρ∞<ρ¯\\rho\(t\)\\leq\\eta\_\{p\}C\_\{p\}\\delta^\{2\}\(1\+L\_\{\\alpha\}\)/\\lambda\_\{\*\}=\\rho\_\{\\infty\}<\\bar\{\\rho\}on\[0,T\]\[0,T\], and by continuity the strict inequality persists slightly beyondTT\. Soℬ\\mathcal\{B\}is open in\[0,τ\]\[0,\\tau\], hence equal to it, and the drift bound holds on\[0,τ\]\[0,\\tau\]\.

Step 4, the encoder escapes\. SupposeTδ\>T¯T\_\{\\delta\}\>\\bar\{T\}, so that Step 3 holds on\[0,T¯\]\[0,\\bar\{T\}\]\. The coordinates selected by𝚷∗\\mathbf\{\\Pi\}\_\{\*\}are real eigendirections with𝚲​𝚷∗=λ∗​𝚷∗\\mathbf\{\\Lambda\}\\mathbf\{\\Pi\}\_\{\*\}=\\lambda\_\{\*\}\\mathbf\{\\Pi\}\_\{\*\}, becauseμk​ℓ\>1\\mu\_\{k\\ell\}\>1forcesΔk​ℓ\>0\\Delta\_\{k\\ell\}\>0\. By Duhamel’s formula and the growth bound of Step 3,

‖𝚷∗​𝐲​\(t\)‖≥eλ∗​t​α\|𝐲⁡\(0\)​‖−c‖​𝐲​\(0\)\|∫0teλ∗​\(t−s\)​eλ~​s​𝑑s=eλ∗​t​‖𝐲⁡\(0\)‖​\[α−\(ec​t−1\)\]\.\\\|\\mathbf\{\\Pi\}\_\{\*\}\\mathbf\{y\}\(t\)\\\|\\geq e^\{\\lambda\_\{\*\}t\}\\alpha\\\|\\mathbf\{y\}\(0\)\\\|\-c\\,\\\|\\mathbf\{y\}\(0\)\\\|\\int\_\{0\}^\{t\}e^\{\\lambda\_\{\*\}\(t\-s\)\}e^\{\\tilde\{\\lambda\}s\}\\,ds=e^\{\\lambda\_\{\*\}t\}\\\|\\mathbf\{y\}\(0\)\\\|\\bigl\[\\alpha\-\(e^\{ct\}\-1\)\\bigr\]\.\(75\)By \(C2\),ec​t−1≤α/2e^\{ct\}\-1\\leq\\alpha/2fort≤T¯t\\leq\\bar\{T\}, so‖𝚷∗​𝐲​\(t\)‖≥\(α/2\)​‖𝐲⁡\(0\)‖​eλ∗​t\\\|\\mathbf\{\\Pi\}\_\{\*\}\\mathbf\{y\}\(t\)\\\|\\geq\(\\alpha/2\)\\\|\\mathbf\{y\}\(0\)\\\|e^\{\\lambda\_\{\*\}t\}\. Combining this with‖𝐳⁡\(t\)‖≥‖𝐲⁡\(t\)‖/‖𝐗^−1‖2\\\|\\mathbf\{z\}\(t\)\\\|\\geq\\\|\\mathbf\{y\}\(t\)\\\|/\\\|\\widehat\{\\mathbf\{X\}\}^\{\-1\}\\\|\_\{2\}and the lower bound on‖𝐲⁡\(0\)‖\\\|\\mathbf\{y\}\(0\)\\\|from Step 0 gives‖𝐳⁡\(t\)‖≥\(α​ε/2​κ\)​eλ∗​t\\\|\\mathbf\{z\}\(t\)\\\|\\geq\(\\alpha\\varepsilon/2\\kappa\)\\,e^\{\\lambda\_\{\*\}t\}\. Att=T¯t=\\bar\{T\}the right\-hand side equals2​δ2\\delta, which contradictsTδ\>T¯T\_\{\\delta\}\>\\bar\{T\}\. HenceTδ≤T¯T\_\{\\delta\}\\leq\\bar\{T\},τ=Tδ\\tau=T\_\{\\delta\}, and the drift bound holds up toTδT\_\{\\delta\}\. ∎

###### Corollary F\.4\(The classification survives\)\.

Under the hypotheses of Theorem[F\.3](https://arxiv.org/html/2610.02344#A6.Thmtheorem3), letg0:=min⁡\{\|Re​λ\|:λ∈Spec⁡\(𝒥0\)\}g\_\{0\}:=\\min\\\{\|\\mathrm\{Re\}\\,\\lambda\|:\\lambda\\in\\Spec\(\\mathcal\{J\}\_\{0\}\)\\\}be the distance of the initial spectrum to the imaginary axis, and letλk​ℓ\+\\lambda^\{\+\}\_\{k\\ell\}denote the larger root of mode\(k,ℓ\)\(k,\\ell\)\. If in additionκ​LE​ρ∞<g0\\kappa L\_\{E\}\\rho\_\{\\infty\}<g\_\{0\}, then for allt≤Tδt\\leq T\_\{\\delta\}the Jacobian𝒥​\(𝐖p​\(t\)\)\\mathcal\{J\}\(\\mathbf\{W\}\_\{p\}\(t\)\)has no eigenvalue on the imaginary axis, has exactly\|𝒰\|\|\\mathcal\{U\}\|eigenvalues in the open right half plane, and each of them lies withinκ​LE​ρ∞\\kappa L\_\{E\}\\rho\_\{\\infty\}of someλk​ℓ\+\\lambda^\{\+\}\_\{k\\ell\}with\(k,ℓ\)∈𝒰\(k,\\ell\)\\in\\mathcal\{U\}\. The conditionκ​LE​ρ∞<g0\\kappa L\_\{E\}\\rho\_\{\\infty\}<g\_\{0\}holds wheneverκ​LE​ρ∞<\(β\+σk​ℓ\)/2\\kappa L\_\{E\}\\rho\_\{\\infty\}<\(\\beta\+\\sigma\_\{k\\ell\}\)/2on every mode withΔk​ℓ<0\\Delta\_\{k\\ell\}<0and\|μk​ℓ−1\|\>C′​δ2\|\\mu\_\{k\\ell\}\-1\|\>C^\{\\prime\}\\delta^\{2\}on every mode withΔk​ℓ\>0\\Delta\_\{k\\ell\}\>0, where

C′:=ηp​κ​LE​Cp​\(1\+Lα\)λ∗​maxk​ℓ​β\+σk​ℓ\+β​σk​ℓ​\(μk​ℓ\)\+β​σk​ℓ\.C^\{\\prime\}:=\\frac\{\\eta\_\{p\}\\,\\kappa\\,L\_\{E\}\\,C\_\{p\}\\,\(1\+L\_\{\\alpha\}\)\}\{\\lambda\_\{\*\}\}\\,\\max\_\{k\\ell\}\\frac\{\\beta\+\\sigma\_\{k\\ell\}\+\\sqrt\{\\beta\\sigma\_\{k\\ell\}\(\\mu\_\{k\\ell\}\)\_\{\+\}\}\}\{\\beta\\sigma\_\{k\\ell\}\}\.\(76\)

###### Proof\.

By Theorem[F\.3](https://arxiv.org/html/2610.02344#A6.Thmtheorem3)and Step 1 of its proof,‖𝐄⁡\(t\)‖2≤LE​ρ∞=:emax\\\|\\mathbf\{E\}\(t\)\\\|\_\{2\}\\leq L\_\{E\}\\rho\_\{\\infty\}=:e\_\{\\max\}fort≤Tδt\\leq T\_\{\\delta\}, and the hypothesis readsκ​emax<g0\\kappa e\_\{\\max\}<g\_\{0\}\. The matrix𝒥0\\mathcal\{J\}\_\{0\}is diagonalisable, and on complex blocks the real rotation form differs from a complex eigenbasis by1/21/\\sqrt\{2\}times a unitary matrix, soκ\\kappais the condition number of an eigenvector matrix of𝒥0\\mathcal\{J\}\_\{0\}\. The Bauer–Fike theorem\[[Bauer and Fike, 1960](https://arxiv.org/html/2610.02344#bib.bib8)\]then places every eigenvalueλ^\\hat\{\\lambda\}of𝒥⁡\(𝐖p​\(t\)\)=𝒥0\+𝐄⁡\(t\)\\mathcal\{J\}\(\\mathbf\{W\}\_\{p\}\(t\)\)=\\mathcal\{J\}\_\{0\}\+\\mathbf\{E\}\(t\)withinκ​emax\\kappa e\_\{\\max\}of some eigenvalueλj\\lambda\_\{j\}of𝒥0\\mathcal\{J\}\_\{0\}\.

First, no eigenvalue lies on the axis: ifλ^\\hat\{\\lambda\}were purely imaginary, then\|λ^−λj\|≥\|Re​λj\|≥g0\>κ​emax\|\\hat\{\\lambda\}\-\\lambda\_\{j\}\|\\geq\|\\mathrm\{Re\}\\,\\lambda\_\{j\}\|\\geq g\_\{0\}\>\\kappa e\_\{\\max\}for everyjj, a contradiction\.

Second, the count in the right half plane is constant\. The eigenvalues of𝒥​\(𝐖p​\(t\)\)\\mathcal\{J\}\(\\mathbf\{W\}\_\{p\}\(t\)\)depend continuously onttand stay in the disc of radiusR:=supt≤Tδ‖𝒥⁡\(𝐖p​\(t\)\)‖2R:=\\sup\_\{t\\leq T\_\{\\delta\}\}\\\|\\mathcal\{J\}\(\\mathbf\{W\}\_\{p\}\(t\)\)\\\|\_\{2\}\. None of them meets the boundary of\{Reλ\>0\}∩\{\|λ\|<R\+1\}\\\{\\mathrm\{Re\}\\,\\lambda\>0\\\}\\cap\\\{\|\\lambda\|<R\+1\\\}, so the number inside is locally constant inttand hence constant on\[0,Tδ\]\[0,T\_\{\\delta\}\]\. Att=0t=0a mode withμk​ℓ\>1\\mu\_\{k\\ell\}\>1has roots of productβ​σk​ℓ​\(1−μk​ℓ\)<0\\beta\\sigma\_\{k\\ell\}\(1\-\\mu\_\{k\\ell\}\)<0and contributes one positive root; a real mode withμk​ℓ<1\\mu\_\{k\\ell\}<1has positive product and negative sum−\(β\+σk​ℓ\)\-\(\\beta\+\\sigma\_\{k\\ell\}\)and contributes none; a complex mode has real part−\(β\+σk​ℓ\)/2\-\(\\beta\+\\sigma\_\{k\\ell\}\)/2and contributes none\. The count is therefore\|𝒰\|\|\\mathcal\{U\}\|\.

Third, localisation\. IfRe​λ^\>0\\mathrm\{Re\}\\,\\hat\{\\lambda\}\>0, its Bauer–Fike partner satisfiesRe​λj\>−κ​emax\>−g0\\mathrm\{Re\}\\,\\lambda\_\{j\}\>\-\\kappa e\_\{\\max\}\>\-g\_\{0\}, henceRe​λj≥g0\>0\\mathrm\{Re\}\\,\\lambda\_\{j\}\\geq g\_\{0\}\>0, and the only eigenvalues of𝒥0\\mathcal\{J\}\_\{0\}in the right half plane are theλk​ℓ\+\\lambda^\{\+\}\_\{k\\ell\}with\(k,ℓ\)∈𝒰\(k,\\ell\)\\in\\mathcal\{U\}\.

Finally, the margin form\. Fix a mode\. IfΔk​ℓ<0\\Delta\_\{k\\ell\}<0, both roots have real part−\(β\+σk​ℓ\)/2\-\(\\beta\+\\sigma\_\{k\\ell\}\)/2\. IfΔk​ℓ\>0\\Delta\_\{k\\ell\}\>0, both roots are real with productβ​σk​ℓ​\(1−μk​ℓ\)\\beta\\sigma\_\{k\\ell\}\(1\-\\mu\_\{k\\ell\}\)\. SinceΔk​ℓ≤\|β−σk​ℓ\|\+2​β​σk​ℓ​\(μk​ℓ\)\+\\sqrt\{\\Delta\_\{k\\ell\}\}\\leq\|\\beta\-\\sigma\_\{k\\ell\}\|\+2\\sqrt\{\\beta\\sigma\_\{k\\ell\}\(\\mu\_\{k\\ell\}\)\_\{\+\}\}, the larger modulus is at mostβ\+σk​ℓ\+β​σk​ℓ​\(μk​ℓ\)\+\\beta\+\\sigma\_\{k\\ell\}\+\\sqrt\{\\beta\\sigma\_\{k\\ell\}\(\\mu\_\{k\\ell\}\)\_\{\+\}\}, and the smaller one is at least

gk​ℓ:=β​σk​ℓ​\|1−μk​ℓ\|β\+σk​ℓ\+β​σk​ℓ​\(μk​ℓ\)\+\.g\_\{k\\ell\}:=\\frac\{\\beta\\sigma\_\{k\\ell\}\\,\|1\-\\mu\_\{k\\ell\}\|\}\{\\beta\+\\sigma\_\{k\\ell\}\+\\sqrt\{\\beta\\sigma\_\{k\\ell\}\(\\mu\_\{k\\ell\}\)\_\{\+\}\}\}\.\(77\)Sog0g\_\{0\}is at least the minimum ofgk​ℓg\_\{k\\ell\}over real modes and of\(β\+σk​ℓ\)/2\(\\beta\+\\sigma\_\{k\\ell\}\)/2over complex ones\. On a real mode,\|μk​ℓ−1\|\>C′​δ2\|\\mu\_\{k\\ell\}\-1\|\>C^\{\\prime\}\\delta^\{2\}and the definition ofC′C^\{\\prime\}givegk​ℓ\>κ​LE​ρ∞g\_\{k\\ell\}\>\\kappa L\_\{E\}\\rho\_\{\\infty\}, becauseρ∞=ηp​Cp​\(1\+Lα\)​δ2/λ∗\\rho\_\{\\infty\}=\\eta\_\{p\}C\_\{p\}\(1\+L\_\{\\alpha\}\)\\delta^\{2\}/\\lambda\_\{\*\}\. Together with the condition on complex modes this yieldsg0\>κ​LE​ρ∞g\_\{0\}\>\\kappa L\_\{E\}\\rho\_\{\\infty\}\. ∎

Condition \(C2\) holds whenε≥\(4κδ/α\)e−K2/δ2\\varepsilon\\geq\(4\\kappa\\delta/\\alpha\)\\,e^\{\-K\_\{2\}/\\delta^\{2\}\}withK2:=λ∗2​log⁡\(1\+α/2\)/\(2​κ​LE​ηp​Cp​\(1\+Lα\)\)K\_\{2\}:=\\lambda\_\{\*\}^\{2\}\\log\(1\+\\alpha/2\)/\(2\\kappa L\_\{E\}\\eta\_\{p\}C\_\{p\}\(1\+L\_\{\\alpha\}\)\):ε\\varepsilonmay be exponentially small in1/δ21/\\delta^\{2\}but not zero at fixedδ\\delta\.

#### F\.6\.3Numerical check with the predictor trained concurrently

With the predictor trained concurrently from its initialisation, the sign of the leading eigenvalue of the frozen\-predictor Jacobian of Eq\. \([3](https://arxiv.org/html/2610.02344#S3.E3)\) at initialisation classified escape against collapse in104104of104104runs at equal learning rates,104104of104104at three times the predictor rate, and102102of104104at ten times; both disagreements lie at\|λinit\|≤0\.004\|\\lambda\_\{\\mathrm\{init\}\}\|\\leq 0\.004\.

## Appendix GTraining Details

This section presents the implementation details of the project\.

### G\.1Programming environment

The experimental codebase, available at[https://github\.com/jose\-melo/drive\-vs\-decay](https://github.com/jose-melo/drive-vs-decay), was implemented in Python and relies on several widely used scientific computing and deep learning libraries\. Table[9](https://arxiv.org/html/2610.02344#A7.T9)summarizes the main dependencies and their corresponding versions\. All experiments were conducted on a single NVIDIA HGX A100 GPU with 40 GB of memory\.

Table 9:Main libraries used in the project\.
### G\.2Tabular\-domain experiments

We conduct experiments on three standard tabular benchmarks:Jannis,Helena, andALOI, which cover a broad range of dataset scales, feature dimensionalities, and classification complexities\.Helenais a medium\-scale, high\-dimensional dataset with 100 classes and purely numerical features, testing multi\-class discrimination under moderate sample sizes\.ALOIis a large\-scale, high\-cardinality classification benchmark with 1,000 classes and 128 numerical features, posing a challenging regime for representation learning and scalability\. Together, these datasets enable a systematic evaluation across increasing dimensionality, class cardinality, and computational cost\.

Table 10:Dataset characteristics
### G\.3Image\-domain experiments

For the image experiments, we pretrain I\-JEPA\[[Assran et al\., 2023](https://arxiv.org/html/2610.02344#bib.bib1)\]on three image benchmarks at modest resolution:

- •CIFAR\-10\(×3232\\\!\\times\\\!32, 10 classes, 50k train images\);
- •CIFAR\-100\(×3232\\\!\\times\\\!32, 100 classes, 50k train images\);
- •STL\-10\(×9696\\\!\\times\\\!96, 10 classes, 5k labeled \+ 100k unlabeled\)\.

## Appendix HEvaluation Metrics

### H\.1Quantifying Representational Collapse\.

A central failure mode of Joint Embedding Predictive Architectures and related self\-supervised systems is*representational collapse*, where learned embeddings concentrate in a low\-dimensional subspace or degenerate toward rank\-deficient solutions\. Such collapse is not necessarily reflected by the training loss alone, as non\-contrastive objectives may admit low\-loss solutions with severely impoverished geometry\. Consequently, evaluation must explicitly probe the*intrinsic dimensionality*and spectral diversity of the learned representations\. To this end, we adopt a spectral measure that captures how information is distributed across orthogonal directions in the embedding space, thereby providing a principled and architecture\-agnostic proxy for collapse\.

##### Effective Rank\.

Let𝐙∈ℝn×d\\mathbf\{Z\}\\in\\mathbb\{R\}^\{n\\times d\}denote a matrix of embeddings, obtained by stackingnnrepresentation vectors of dimensiondd\. Let

𝐙=𝐔​diag⁡\(σ1,…,σr\)​𝐕⊤\\mathbf\{Z\}=\\mathbf\{U\}\\,\\diag\(\\sigma\_\{1\},\\ldots,\\sigma\_\{r\}\)\\,\\mathbf\{V\}^\{\\top\}\(78\)be the singular value decomposition of𝐙\\mathbf\{Z\}, wherer=rank⁡\(𝐙\)r=\\operatorname\{rank\}\(\\mathbf\{Z\}\)andσ1≥⋯≥σr≥0\\sigma\_\{1\}\\geq\\cdots\\geq\\sigma\_\{r\}\\geq 0are the non\-zero singular values\. We define a normalized spectral distribution

pi≔σi∑j=1rσj,i=1,…,r,p\_\{i\}\\;\\coloneqq\\;\\frac\{\\sigma\_\{i\}\}\{\\sum\_\{j=1\}^\{r\}\\sigma\_\{j\}\},\\qquad i=1,\\ldots,r,\(79\)which can be interpreted as a probability mass function over singular modes, reflecting how representation energy is allocated across orthogonal directions\.

The*effective rank*is then defined as the exponential of the Shannon entropy of this distribution:

ERank\(𝐙\)≔exp\(−∑i=1rpilogpi\)\.\\mathrm\{ERank\}\(\\mathbf\{Z\}\)\\;\\coloneqq\\;\\exp\\\!\\left\(\-\\sum\_\{i=1\}^\{r\}p\_\{i\}\\log p\_\{i\}\\right\)\.\(80\)

##### Interpretation and Theoretical Properties\.

The effective rank admits several desirable properties for diagnosing collapse:

- •Upper and lower bounds\.One has1≤ERank⁡\(𝐙\)≤r1\\leq\\mathrm\{ERank\}\(\\mathbf\{Z\}\)\\leq r\. The lower bound is achieved when all mass concentrates on a single singular value \(complete collapse to a one\-dimensional subspace\), while the upper bound is achieved when the spectrum is uniform,pi=1/rp\_\{i\}=1/r, corresponding to maximally spread representations\.
- •Sensitivity to spectral concentration\.Unlike the algebraic rank, which is binary and insensitive to magnitude,ERank\\mathrm\{ERank\}continuously penalizes dominance by a small number of singular modes\. This makes it well\-suited to detect*soft collapse*, where representations remain formally full\-rank but are effectively low\-dimensional\.
- •Scale invariance\.Since\{pi\}\\\{p\_\{i\}\\\}is normalized,ERank\\mathrm\{ERank\}is invariant to global rescaling of representations, isolating geometric collapse from trivial norm effects\.

##### Relation to Collapse Dynamics\.

From a dynamical perspective, collapse corresponds to a regime in which training amplifies only a small subset of eigendirections of the representation covariance while suppressing others\. In this regime, the spectral distribution\{pi\}\\\{p\_\{i\}\\\}becomes sharply peaked, reducing entropy and drivingERank\\mathrm\{ERank\}toward its minimum\. Conversely, stable learning that preserves multiple informative modes yields a flatter spectrum and higher effective rank\. As such,ERank\\mathrm\{ERank\}provides a direct quantitative probe of whether training dynamics maintain a rich, multi\-dimensional representation geometry or drift toward degenerate fixed points\.

Throughout our experiments, we report effective rank as a primary diagnostic for representational collapse, complementing downstream performance metrics and enabling a fine\-grained analysis of how different stabilization strategies affect the intrinsic dimensionality of learned embeddings\.

## Appendix IFull Results

The aggregated grid over context ratio and predictor scale is Table[1](https://arxiv.org/html/2610.02344#S5.T1)in the main text; the per\-dataset tables follow\.

### I\.1Tabular\-JEPA Variants

Table 11:Detailed results for Tabular–JEPA variants across ALOI, Helena, and Jannis datasets\. Each row shows results for a specific context ratio and predictor scale configuration, with means and standard errors \(SEM\) aggregated over seeds\. Collapse statistics \(μ\\muquartiles\), effective rank \(Erank\), final training loss, and test accuracy are reported\.Table 12:Detailed results for Tabular–JEPA variants across ALOI, Helena, and Jannis datasets \(continued\)\.Table 13:Detailed results for Tabular–JEPA variants across ALOI, Helena, and Jannis datasets \(continued\)\.Table 14:Detailed results for Tabular–JEPA variants across ALOI, Helena, and Jannis datasets \(continued\)\.Table 15:Detailed results for Tabular–JEPA variants across ALOI, Helena, and Jannis datasets \(continued\)\.
### I\.2T\-JEPA

Table 16:Detailed results for T\-JEPA on Jannis dataset\. Each row shows results for a specific context ratio and predictor scale configuration, with means aggregated over seeds\.
### I\.3T\-JEPA with ResidualPred – per\-run details

Tables[17](https://arxiv.org/html/2610.02344#A9.T17)and[18](https://arxiv.org/html/2610.02344#A9.T18)expand the T\-JEPA×\\timesResidualPred ablation that backs the tabular columns of Table[4](https://arxiv.org/html/2610.02344#S6.T4)in the main text\. Table[17](https://arxiv.org/html/2610.02344#A9.T17)is deliberately the subcritical stress grid: its five configurations sit where the criterion places both predictors below threshold, so no per\-seed prevention claim is made there; the criterion widens the escape basin without guaranteeing escape\. The recurring probe values are class priors\. Onjannis,46\.01%46\.01\\%is the frequency of class33and34\.38%34\.38\\%that of class11in the training split, so a collapsed representation yields a near\-constant probe and the same value recurs to two decimals across unrelated runs\. Escapes are discontinuous, at65\.36%65\.36\\%onjannisand32\.58%32\.58\\%onhelena; probe accuracy and effective rank co\-sign on every row; no Standard run escapes; and onhelenathe separation is complete, six against six, with a ResidualPred minimum of15\.3915\.39against a Standard maximum of11\.5811\.58on the probe\.

Figure 13:Helena ERank trajectoriesat deliberately brittle configurations, where the criterion predicts collapse \(mean across six runs; per\-seed thin lines\)\. The observable range is about\[1\.1,9\]\[1\.1,9\], since min–max preprocessing leaves a large common mean \(collapse attractor near1\.11\.1, fresh encoder1\.31\.3to1\.91\.9, raw features near8\.88\.8\); Standard lands at1\.111\.11to1\.121\.12on all six seeds, everyResidualPredseed above, at1\.141\.14to3\.033\.03, and the probe moves from8\.6%8\.6\\%to21\.3%21\.3\\%\.Table 17:Per\-seed T\-JEPA \+ ResidualPred results onjannisandhelena, on the subcritical stress grid, where the criterion places both predictors below threshold: no per\-seed prevention is claimed, and the recurring probe values are class priors \(46\.01%46\.01\\%and34\.38%34\.38\\%onjannis\)\. ERank values are reported at epoch00and at the last training epoch; final loss is the training loss at the last epoch; probe accuracy is the linear\-probe test score evaluated on the frozen encoder at the last epoch\.VariantDatasetpm1p\_\{m\_\{1\}\}α\\alphaSeedERank0ERanklastFinal lossProbe acc\. \(%\)Standardjannis0\.0750\.0750\.200\.20001\.711\.711\.271\.27×10−74\.78\\\!\\times\\\!10^\{\-7\}46\.0146\.01Standardjannis0\.0750\.0750\.200\.20111\.791\.791\.291\.29×10−52\.54\\\!\\times\\\!10^\{\-5\}34\.3834\.38Standardjannis0\.0750\.0750\.200\.20221\.791\.791\.291\.29×10−96\.33\\\!\\times\\\!10^\{\-9\}46\.0146\.01Standardjannis0\.1500\.1500\.500\.50001\.651\.651\.281\.28×10−92\.37\\\!\\times\\\!10^\{\-9\}46\.0146\.01Standardjannis0\.1500\.1500\.500\.50111\.711\.711\.281\.28×10−81\.09\\\!\\times\\\!10^\{\-8\}34\.3834\.38Standardjannis0\.1500\.1500\.500\.50221\.661\.661\.271\.27×10−95\.32\\\!\\times\\\!10^\{\-9\}46\.0146\.01Standardjannis0\.1500\.1501\.001\.00001\.781\.781\.271\.27×10−122\.12\\\!\\times\\\!10^\{\-12\}46\.0146\.01Standardjannis0\.1500\.1501\.001\.00111\.681\.681\.281\.28×10−93\.18\\\!\\times\\\!10^\{\-9\}46\.0146\.01Standardjannis0\.1500\.1501\.001\.00221\.761\.761\.281\.28×10−81\.18\\\!\\times\\\!10^\{\-8\}46\.0146\.01\+ResPredjannis0\.0750\.0750\.200\.20001\.701\.701\.251\.25×10−84\.75\\\!\\times\\\!10^\{\-8\}34\.3834\.38\+ResPredjannis0\.0750\.0750\.200\.20111\.691\.692\.902\.90×10−26\.27\\\!\\times\\\!10^\{\-2\}65\.36\\mathbf\{65\.36\}\+ResPredjannis0\.0750\.0750\.200\.20221\.691\.691\.331\.33×10−96\.13\\\!\\times\\\!10^\{\-9\}34\.3834\.38\+ResPredjannis0\.1500\.1500\.500\.50001\.791\.791\.271\.27×10−61\.38\\\!\\times\\\!10^\{\-6\}46\.0146\.01\+ResPredjannis0\.1500\.1500\.500\.50111\.811\.811\.261\.26×10−82\.14\\\!\\times\\\!10^\{\-8\}34\.3834\.38\+ResPredjannis0\.1500\.1500\.500\.50221\.901\.904\.034\.03×10−11\.40\\\!\\times\\\!10^\{\-1\}61\.65\\mathbf\{61\.65\}\+ResPredjannis0\.1500\.1501\.001\.00001\.771\.771\.271\.27×10−52\.49\\\!\\times\\\!10^\{\-5\}46\.0146\.01\+ResPredjannis0\.1500\.1501\.001\.00111\.791\.791\.241\.24×10−41\.29\\\!\\times\\\!10^\{\-4\}46\.4646\.46\+ResPredjannis0\.1500\.1501\.001\.00221\.851\.851\.271\.27×10−83\.83\\\!\\times\\\!10^\{\-8\}46\.0146\.01Standardhelena0\.1500\.1500\.500\.50001\.541\.541\.111\.11×10−42\.05\\\!\\times\\\!10^\{\-4\}6\.14\\phantom\{0\}6\.14Standardhelena0\.1500\.1500\.500\.50111\.371\.371\.121\.12×10−37\.56\\\!\\times\\\!10^\{\-3\}10\.0510\.05Standardhelena0\.1500\.1500\.500\.50221\.341\.341\.121\.12×10−32\.18\\\!\\times\\\!10^\{\-3\}11\.5811\.58Standardhelena0\.1500\.1501\.001\.00001\.371\.371\.121\.12×10−43\.23\\\!\\times\\\!10^\{\-4\}6\.14\\phantom\{0\}6\.14Standardhelena0\.1500\.1501\.001\.00111\.291\.291\.111\.11×10−32\.08\\\!\\times\\\!10^\{\-3\}11\.2711\.27Standardhelena0\.1500\.1501\.001\.00221\.371\.371\.121\.12×10−42\.31\\\!\\times\\\!10^\{\-4\}6\.14\\phantom\{0\}6\.14\+ResPredhelena0\.1500\.1500\.500\.50001\.531\.533\.033\.03×10−11\.62\\\!\\times\\\!10^\{\-1\}32\.58\\mathbf\{32\.58\}\+ResPredhelena0\.1500\.1500\.500\.50111\.451\.451\.861\.86×10−21\.67\\\!\\times\\\!10^\{\-2\}27\.63\\mathbf\{27\.63\}\+ResPredhelena0\.1500\.1500\.500\.50221\.381\.381\.311\.31×10−21\.19\\\!\\times\\\!10^\{\-2\}15\.4815\.48\+ResPredhelena0\.1500\.1501\.001\.00001\.541\.541\.911\.91×10−21\.07\\\!\\times\\\!10^\{\-2\}21\.0621\.06\+ResPredhelena0\.1500\.1501\.001\.00111\.471\.471\.141\.14×10−42\.03\\\!\\times\\\!10^\{\-4\}15\.4815\.48\+ResPredhelena0\.1500\.1501\.001\.00221\.381\.381\.171\.17×10−44\.24\\\!\\times\\\!10^\{\-4\}15\.3915\.39

Table 18:Per\-configuration aggregates \(mean±\\pmstandard deviation across33seeds\) for the runs of Table[17](https://arxiv.org/html/2610.02344#A9.T17)\.
### I\.4I\-JEPA Detailed Results on CIFAR\-10, CIFAR\-100, and STL\-10

Tables[19](https://arxiv.org/html/2610.02344#A9.T19)and[20](https://arxiv.org/html/2610.02344#A9.T20)expand the image columns of Table[4](https://arxiv.org/html/2610.02344#S6.T4)in the main text\. All runs use the I\-JEPA configuration of Section[4](https://arxiv.org/html/2610.02344#S4)with ViT\-Small/44on CIFAR\-10/CIFAR\-100 \(L=64L=64tokens\) and ViT\-Small/88on STL\-10 \(L=144L=144tokens\), context retentionpm1=0\.85p\_\{m\_\{1\}\}=0\.85, AdamW optimiser, and100100\-epoch cosine schedule\.

Table 19:Per\-seed I\-JEPA results on CIFAR\-10, CIFAR\-100, and STL\-10 atpm1=0\.85p\_\{m\_\{1\}\}=0\.85,100100epochs\.μ50\(0\)\\mu\_\{50\}^\{\(0\)\}is the median per\-mode initialisation stability ratio;ERank0\\mathrm\{ERank\}\_\{0\},ERank100\\mathrm\{ERank\}\_\{100\}are the effective rank at epoch00and epoch100100; Probe is the linear\-probe test accuracy at epoch100100\.Table 20:Per\-cell aggregates of Table[19](https://arxiv.org/html/2610.02344#A9.T19)\(mean±\\pmstandard deviation across seeds\)\. The Probe column reproduces the image rows of Table[4](https://arxiv.org/html/2610.02344#S6.T4)of the main text exactly to two decimal places\.
### I\.5Image experiments beyond CIFAR: protocol

All runs use one I\-JEPA pipeline\[[Assran et al\., 2023](https://arxiv.org/html/2610.02344#bib.bib1)\]with identical loss, optimiser, EMA teacher and masking, a ViT\-S/16 encoder at128128pixels unless stated otherwise, and differ only in the predictor; the loss\-side arms of Appendix[I\.9](https://arxiv.org/html/2610.02344#A9.SS9)add one loss term\. ImageNet\-100\[[Tian et al\., 2020](https://arxiv.org/html/2610.02344#bib.bib30)\]:150150epochs, predictor depths22and66, three seeds per predictor; the backbone check compares ViT\-S/16 and ViT\-B/16 at epoch100100, depth66\. ImageNet\-1k\[[Deng et al\., 2009](https://arxiv.org/html/2610.02344#bib.bib11)\]: a matched\-budget pilot of3030epochs at depth66with five seeds per predictor, and two stresses that change one variable each,224224pixels at3030epochs \(five seeds\) and9090epochs at128128pixels \(three seeds\)\.

Readouts on the frozen encoder are a linear probe, a parameter\-free nearest\-neighbour classifier \(k=20k=20\) and an attentive probe; effective rank is RankMe\[[Garrido et al\., 2023](https://arxiv.org/html/2610.02344#bib.bib15)\], logged every five epochs\. On ImageNet\-1k the nearest\-neighbour and attentive readouts use a class\-balanced bank of200200images per class, identical for every arm, a deviation from the full\-split protocol\.

### I\.6ImageNet\-100 at two depths and three probes

Table 21:ImageNet\-100, I\-JEPA ViT\-S/16,128128pixels,150150epochs, three seeds per predictor:ResidualPredminus the standard predictor\. EveryResidualPredseed exceeds every standard seed on all three probes at both depths; exact paired permutationp=0\.031p=0\.031per probe\.The two predictors start matched in effective rank, andResidualPred’s is higher at every logged epoch from the fifth onward in every matched seed\. At the matched100100\-epoch, depth\-66setting the effect holds on a backbone with four times the parameters: ViT\-B/16 goes from0\.3390\.339to0\.3980\.398on the linear probe,\+6\.0\+6\.0points with no seed overlap, and effective rank rises by23\.923\.9; ViT\-S/16 gains4\.44\.4points at the same setting\. Across the1818ImageNet\-100 and ViT\-B runs, no effective\-rank trajectory falls more than0\.3%0\.3\\%below its running maximum after escape\.

### I\.7A thousand\-class pilot and its two stresses

Table 22:The thousand\-class pilot \(ImageNet\-1k, ViT\-S/16,128128pixels, depth66,3030epochs, matched budget\), seed means over five seeds per predictor\. On the linear probe and effective rank everyResidualPredseed exceeds every standard seed \(p=0\.008p=0\.008\); the nearest\-neighbour and attentive readouts use the class\-balanced bank\.The predictors start rank\-matched, separate by epoch five, and the gap widens in seed means\. The parameter\-free margin is larger at a thousand classes than at a hundred \(\+8\.58\+8\.58against\+5\.5\+5\.5points at depth66\)\. This is a matched\-budget ablation at128128pixels; nothing is claimed about the official full\-resolution, full\-schedule recipe\.

Table 23:The two stresses, recipe otherwise unchanged\. Linear\-probe top\-1, seed mean and range\.224224pixels: five seeds per predictor\.9090epochs: three seeds per predictor, complete separation\.At224224pixels the margin is kept, with theResidualPredseed spread widening to2\.62\.6points\. Over9090epochs the margin attenuates to\+3\.45\+3\.45points; the standard predictor’s effective rank rises from111111at epoch3030to179179at epoch9090in seed means, against200200to251251forResidualPred, and matchingResidualPred’s3030\-epoch accuracy costs the standard predictor up to three times the compute\.

### I\.8Controls: frozen and skip\-only predictors

Two controls at the ImageNet\-100 depth\-66setting, three seeds each\. A skip\-only predictor is degenerate in a JEPA: with no path from context to target the encoder gradient is identically zero, so it serves only as a null anchor\. Freezing the exactResidualPredpredictor at its initialisation, with everything else trained, collapses the representation instead of reverting to the standard predictor: the linear probe falls from the trainedResidualPred’s0\.4270\.427to0\.0990\.099and effective rank from82\.182\.1to1\.671\.67, every frozen seed below every standard seed\. On two seeds with identical data order the frozen and trained runs start at the same effective rank \(24\.744424\.7444and25\.253625\.2536\) and are separated by epoch five \(1\.151\.15to2\.732\.73against25\.925\.9to30\.630\.6across the three seeds\)\. TheResidualPredpredictor denied learning loses the entire gain\.

In the linear model with𝐖p\\mathbf\{W\}\_\{p\}fixed, the regime of Lemma[3\.1](https://arxiv.org/html/2610.02344#S3.Thmtheorem1), the fastest super\-critical mode dominates and effective rank relaxes to1\.001\.00to1\.041\.04in all1717settings screened; in the trained system,λmax​\(𝐖p⊤​𝐖p\)\\lambda\_\{\\max\}\(\\mathbf\{W\}\_\{p\}^\{\\top\}\\mathbf\{W\}\_\{p\}\)re\-equilibrates to0\.800\.80to1\.101\.10from initialisations spanning a500×500\\timesrange\. In the public solo\-learn BYOL implementation\[[Turrisi da Costa et al\., 2022](https://arxiv.org/html/2610.02344#bib.bib33)\]\(CIFAR\-100, ResNet\-18, no moving\-average teacher\), effective rank and accuracy increase with the ratio of predictor to encoder learning rate over\{0,0\.1,1,10\}\\\{0,0\.1,1,10\\\}in both seeds, the frozen predictor being the worst\.

### I\.9Comparison with loss\-side mechanisms at the pilot setting

SIGReg is implemented as the Epps–Pulley statistic of LeJEPA\[[Balestriero and LeCun, 2025](https://arxiv.org/html/2610.02344#bib.bib4)\], with4848resampled slices per step computed per rank, and added to the I\-JEPA loss with weightλ\\lambda\. This isolates SIGReg as an anti\-collapse term and is not a reproduction of LeJEPA, which removes the EMA teacher and the predictor\.λ=0\.05\\lambda=0\.05is the authors’ recommended default, bracketed byλ∈\{0\.02,0\.15\}\\lambda\\in\\\{0\.02,0\.15\\\}\. The weighting is additive; the convex form of the released LeJEPA algorithm differs from it by a rescaling of the prediction term by about5%5\\%at this dose\.

Table 24:Loss\-side comparison at the pilot setting \(ImageNet\-1k,128128pixels,3030epochs, depth66\);nnis the number of seeds\. SIGReg beats the standard predictor by3\.983\.98points andResidualPredbeats SIGReg by2\.502\.50points, each with complete separation; all3535pairwise seed differences betweenResidualPredand SIGReg are positive, from\+1\.42\+1\.42to\+3\.73\+3\.73points\.SIGReg is active in this pipeline: its penalty falls from0\.0330\.033at initialisation to0\.00110\.0011by the final epoch, and it raises effective rank from107107to186186\.ResidualPredbeats it in this setting, and the two compose: combining them adds18\.218\.2to the effective rank ofResidualPredalone \(90%90\\%interval\+15\.0\+15\.0to\+21\.4\+21\.4\)\. Mimetic attention initialisation\[[Trockman and Kolter, 2023](https://arxiv.org/html/2610.02344#bib.bib32)\]improves on the standard predictor by1\.61\.6points with seed overlap and trailsResidualPredby4\.94\.9points\. A VICReg arm at the coefficient ratio of the original paper destabilised in both seeds \(linear probe0\.0260\.026and0\.0620\.062\) and is excluded; the C\-JEPA integration\[[Mo and Tong, 2024](https://arxiv.org/html/2610.02344#bib.bib24)\]differs from it, so these runs say nothing about VICReg itself\.ResidualPredadds one hyperparameter,βi​n​i​t=10\\beta\_\{init\}=10, unchanged on every dataset, and no loss term or per\-step computation\.

## Appendix JSeven anti\-collapse mechanisms as doses on one ratio: verification on the linear model

For each mechanism below we derive its effect onμ=γ/σ\\mu=\\gamma/\\sigmaunder Assumption[3\.2](https://arxiv.org/html/2610.02344#S3.Thmtheorem2), predict from the initialisation how many modes escape, and count the modes that escape when the linear model, Eq\. \([2](https://arxiv.org/html/2610.02344#S3.E2)\) with the mechanism added, is integrated\. Across152152\(mechanism, dose, seed\) runs the predicted count equals the measured count in all152152\. All runs are simulations of the linear model: the table verifies the derivations against the model they concern and is not a test of the criterion against JEPA training, which is Section[6](https://arxiv.org/html/2610.02344#S6)and Appendix[I\.5](https://arxiv.org/html/2610.02344#A9.SS5)\.

Table 25:Seven anti\-collapse mechanisms as doses on one ratio, verified on the linear model: number of escaping modes predicted from the derivedγ\\gammaandσ\\sigmaat initialisation against the number that grows when Eq\. \([2](https://arxiv.org/html/2610.02344#S3.E2)\) is integrated with the mechanism added\. All152152runs are simulations of the linear model, not deep networks\.### J\.1The seven doses

Throughout,ak,cka\_\{k\},c\_\{k\}are the eigenvalues of𝚺11\\mathbf\{\\Sigma\}\_\{11\}and𝚺X​1\\mathbf\{\\Sigma\}\_\{X1\}along data modekk, andpℓ,qℓp\_\{\\ell\},q\_\{\\ell\}those of𝐏=𝐖p⊤​𝐖p\\mathbf\{P\}=\\mathbf\{W\}\_\{p\}^\{\\top\}\\mathbf\{W\}\_\{p\}and𝐐=𝐖p⊤​𝐌¯2\\mathbf\{Q\}=\\mathbf\{W\}\_\{p\}^\{\\top\}\\bar\{\\mathbf\{M\}\}\_\{2\}along predictor modeℓ\\ell, so thatγk​ℓ=ck​qℓ\\gamma\_\{k\\ell\}=c\_\{k\}q\_\{\\ell\},σk​ℓ=ak​pℓ\\sigma\_\{k\\ell\}=a\_\{k\}p\_\{\\ell\}andμk​ℓ=γk​ℓ/σk​ℓ\\mu\_\{k\\ell\}=\\gamma\_\{k\\ell\}/\\sigma\_\{k\\ell\}\.

###### Proposition J\.1\(Encoder weight decay is an additive floor\)\.

Adding−ηw​d​𝐖\-\\eta\_\{wd\}\\mathbf\{W\}to Eq\. \([2](https://arxiv.org/html/2610.02344#S3.E2)\) replaces the decay operator by𝚺11⊗𝐏\+ηw​d​𝐈\\mathbf\{\\Sigma\}\_\{11\}\\otimes\\mathbf\{P\}\+\\eta\_\{wd\}\\mathbf\{I\}, soσk​ℓ′=σk​ℓ\+ηw​d\\sigma^\{\\prime\}\_\{k\\ell\}=\\sigma\_\{k\\ell\}\+\\eta\_\{wd\}withγk​ℓ\\gamma\_\{k\\ell\}unchanged, and mode\(k,ℓ\)\(k,\\ell\)escapes if and only ifγk​ℓ−σk​ℓ\>ηw​d\\gamma\_\{k\\ell\}\-\\sigma\_\{k\\ell\}\>\\eta\_\{wd\}\. This assumes the decay acts on the context encoder only\.

###### Proposition J\.2\(Predictor scaling divides the ratio\)\.

Under𝐖p↦α​𝐖p\\mathbf\{W\}\_\{p\}\\mapsto\\alpha\\mathbf\{W\}\_\{p\},pℓp\_\{\\ell\}scales asα2\\alpha^\{2\}andqℓq\_\{\\ell\}asα\\alpha, soμk​ℓ​\(α\)=μk​ℓ​\(1\)/α\\mu\_\{k\\ell\}\(\\alpha\)=\\mu\_\{k\\ell\}\(1\)/\\alphaand nothing escapes aboveαe​x​t=maxk​ℓ⁡μk​ℓ​\(1\)\\alpha\_\{ext\}=\\max\_\{k\\ell\}\\mu\_\{k\\ell\}\(1\)\.

###### Proposition J\.3\(An isotropy penalty is negative weight decay in embedding space\)\.

Let the auxiliary term contribute\+λ​κ0​𝐖​𝚺11\+\\lambda\\kappa\_\{0\}\\mathbf\{W\}\\mathbf\{\\Sigma\}\_\{11\}to the flow as𝐖→𝟎\\mathbf\{W\}\\to\\mathbf\{0\}, withκ0\\kappa\_\{0\}twice the magnitude of the estimator’s slope at zero embedding variance divided by the embedding dimension\. The decay block becomes−𝚺11⊗\[𝐏−ρ𝐈\]\-\\mathbf\{\\Sigma\}\_\{11\}\\otimes\[\\mathbf\{P\}\-\\rho\\mathbf\{I\}\]withρ:=λ​κ0/\(1−λ\)\\rho:=\\lambda\\kappa\_\{0\}/\(1\-\\lambda\)\. Forpℓ\>ρp\_\{\\ell\}\>\\rhoescape holds if and only ifμk​ℓ\>1−ρ/pℓ\\mu\_\{k\\ell\}\>1\-\\rho/p\_\{\\ell\}; the mode also escapes forpℓ≤ρp\_\{\\ell\}\\leq\\rhowithγk​ℓ\>0\\gamma\_\{k\\ell\}\>0, and forpℓ<ρp\_\{\\ell\}<\\rhowithγk​ℓ=0\\gamma\_\{k\\ell\}=0\. The linearisation uses only the second moment of a centred distribution, andκ0\\kappa\_\{0\}is calibrated per setup\.

###### Proposition J\.4\(The variance hinge mirrors weight decay\)\.

With a hingeλv​∑imax⁡\(0,g−si\)\\lambda\_\{v\}\\sum\_\{i\}\\max\(0,g\-s\_\{i\}\)on embedding standard deviationssis\_\{i\}, the drive is unchanged and, while the hinge is active, the decay becomesσk​ℓ​\(s\)=ak​\(pℓ−λv/sℓ\)\\sigma\_\{k\\ell\}\(s\)=a\_\{k\}\(p\_\{\\ell\}\-\\lambda\_\{v\}/s\_\{\\ell\}\)\. In the small\-weight limit this isak​\(pℓ−λ~\)a\_\{k\}\(p\_\{\\ell\}\-\\tilde\{\\lambda\}\)withλ~=λv/εv\\tilde\{\\lambda\}=\\lambda\_\{v\}/\\sqrt\{\\varepsilon\_\{v\}\}, so forpℓ\>λ~p\_\{\\ell\}\>\\tilde\{\\lambda\}escape holds if and only ifμk​ℓ\>1−λ~/pℓ\\mu\_\{k\\ell\}\>1\-\\tilde\{\\lambda\}/p\_\{\\ell\}\(εv\\varepsilon\_\{v\}is the variance offset inside the hinge’s square root\)\. The covariance term is cubic in the weights and does not enter the linearisation\.

###### Proposition J\.5\(Mimetic initialisation is a decaying dose on the predictor factor\)\.

For a normal predictor withℓ\\ell\-th eigenvalue\|λℓ\|​ei​ϕℓ\|\\lambda\_\{\\ell\}\|e^\{\\mathrm\{i\}\\phi\_\{\\ell\}\},Reμk​ℓ=\(ck/ak\)\(pt​g​t/\|λℓ\|\)cosϕℓ\\mathrm\{Re\}\\,\\mu\_\{k\\ell\}=\(c\_\{k\}/a\_\{k\}\)\(p\_\{tgt\}/\|\\lambda\_\{\\ell\}\|\)\\cos\\phi\_\{\\ell\}\. For𝐖p​\(t\)∝m⁡\(t\)​𝐈\+𝐎\\mathbf\{W\}\_\{p\}\(t\)\\propto m\(t\)\\mathbf\{I\}\+\\mathbf\{O\}at fixed Frobenius norm, with𝐎\\mathbf\{O\}orthogonal with rotation anglesθℓ\\theta\_\{\\ell\}and identity dosem⁡\(t\)=m0​e−ζ​tm\(t\)=m\_\{0\}e^\{\-\\zeta t\},cos⁡ϕℓ​\(t\)=\(m⁡\(t\)\+cos⁡θℓ\)/m\(t\)2\+2m\(t\)cosθℓ\+1\\cos\\phi\_\{\\ell\}\(t\)=\\bigl\(m\(t\)\+\\cos\\theta\_\{\\ell\}\\bigr\)/\\sqrt\{m\(t\)^\{2\}\+2m\(t\)\\cos\\theta\_\{\\ell\}\+1\}\. In the adiabatic approximation a mode escapes when∫0Tλk\+​\(t\)​𝑑t\>log⁡\(103\)\\int\_\{0\}^\{T\}\\lambda^\{\+\}\_\{k\}\(t\)\\,dt\>\\log\(10^\{3\}\)\.

###### Proposition J\.6\(A frozen identity predictor is a condition on the data spectrum\)\.

With𝐖p=𝐈\\mathbf\{W\}\_\{p\}=\\mathbf\{I\}fixed,𝐏=𝐈\\mathbf\{P\}=\\mathbf\{I\}and𝐐=pt​g​t​𝐈\\mathbf\{Q\}=p\_\{tgt\}\\mathbf\{I\}\. Under Bernoulli feature masking with keep probabilityppon a unit\-diagonal data covariance with spectrum\{sk\}\\\{s\_\{k\}\\\},μk=pt​g​t​sk/\(\(1−p\)\+p​sk\)\\mu\_\{k\}=p\_\{tgt\}s\_\{k\}/\\bigl\(\(1\-p\)\+p\\,s\_\{k\}\\bigr\), so modekkescapes if and only ifsk\>\(1−p\)/\(pt​g​t−p\)s\_\{k\}\>\(1\-p\)/\(p\_\{tgt\}\-p\), which requirespt​g​t\>pp\_\{tgt\}\>p\. Under symmetric additive view noise instead of masking \(the BYOL\-style setting\),ck/ak<1c\_\{k\}/a\_\{k\}<1for every mode and nothing escapes\.

###### Proposition J\.7\(Target\-encoder momentum sets speed, not membership\)\.

Inλ2\+\(β\+σk​ℓ\)​λ\+β​σk​ℓ​\(1−μk​ℓ\)\\lambda^\{2\}\+\(\\beta\+\\sigma\_\{k\\ell\}\)\\lambda\+\\beta\\sigma\_\{k\\ell\}\(1\-\\mu\_\{k\\ell\}\)the rateβ\>0\\beta\>0does not change the sign of the constant term, so for a real drive the escaping set is independent ofβ\\beta, while the escape rateλ\+​\(β\)=12​\[−\(β\+σ\)\+\(β−σ\)2\+4​β​γ\]\\lambda^\{\+\}\(\\beta\)=\\tfrac\{1\}\{2\}\\bigl\[\-\(\\beta\+\\sigma\)\+\\sqrt\{\(\\beta\-\\sigma\)^\{2\}\+4\\beta\\gamma\}\\bigr\]is strictly increasing inβ\\betafor a super\-critical mode\.

### J\.2Four published results recovered as limiting cases

DirectPred\[[Tian et al\., 2021](https://arxiv.org/html/2610.02344#bib.bib31)\]sets the predictor from a moving average of the embedding second moment, which makes𝐏\\mathbf\{P\}and𝐐\\mathbf\{Q\}commute; with predictor eigenvaluep^k=sk\+ε\\hat\{p\}\_\{k\}=\\sqrt\{s\_\{k\}\}\+\\varepsilon, every surviving mode is held atμk=1\\mu\_\{k\}=1at its fixed point, and their growth conditionΔj\>0\\Delta\_\{j\}\>0isμj\>1\\mu\_\{j\}\>1in the adiabatic limit where the target is a fixed multipleτ\\tauof the context encoder\. With weight decayη\\eta, modekkhas ratioτ​ck​pt​g​t​p^/\(ak​p^2\+η\)\\tau c\_\{k\}p\_\{tgt\}\\hat\{p\}/\(a\_\{k\}\\hat\{p\}^\{2\}\+\\eta\), whose supremum overp^\\hat\{p\}gives the death thresholdη\>τ2​maxk⁡\(ck2​pt​g​t2/ak\)/4\\eta\>\\tau^\{2\}\\max\_\{k\}\(c\_\{k\}^\{2\}p\_\{tgt\}^\{2\}/a\_\{k\}\)/4, which reproduces their isotropic threshold with a relative error of4\.1×10−94\.1\\times 10^\{\-9\}\. The rate equation of[Halvagal et al\. \[2023\]](https://arxiv.org/html/2610.02344#bib.bib18)is the flow\-level case of the criterion under predictor slaving, and IsoLoss, from the same paper and in the same regime, equalises escape rates: on the tabular sweep it compresses the spread of escape epochs from7373to77\. Orthonormalising the predictor\[[Richemond et al\., 2023](https://arxiv.org/html/2610.02344#bib.bib26)\]pins𝐖p⊤​𝐖p=𝐈\\mathbf\{W\}\_\{p\}^\{\\top\}\\mathbf\{W\}\_\{p\}=\\mathbf\{I\}, a projection on the decay side, soσk​ℓ=ak\\sigma\_\{k\\ell\}=a\_\{k\}andqℓ=pt​g​t​e−i​θℓq\_\{\\ell\}=p\_\{tgt\}e^\{\-\\mathrm\{i\}\\theta\_\{\\ell\}\}; the exact escape boundary isgcosθℓ\+βg2sin2θℓ/\(β\+ak\)2\>akg\\cos\\theta\_\{\\ell\}\+\\beta g^\{2\}\\sin^\{2\}\\theta\_\{\\ell\}/\(\\beta\+a\_\{k\}\)^\{2\}\>a\_\{k\}withg:=pt​g​t​ckg:=p\_\{tgt\}c\_\{k\}, which matched a direct eigensolve in all of20,00020\{,\}000random draws\.

## Appendix KMeasuring the drive on images after the predictor equilibrates

This appendix reports the two measurements behind the statement of Section[6\.2](https://arxiv.org/html/2610.02344#S6.SS2)that, on images, the mediating variable is measured after predictor equilibration and not at initialisation\.

### K\.1The initialisation\-time estimator is null, on the whole predictor block

The estimator of Section[5](https://arxiv.org/html/2610.02344#S5)reads the predictor’s two linear maps, which are initialised identically with and without the diagonal bias, so on images it cannot see the intervention\. We therefore linearised the whole predictor block \(attention with the diagonal bias, feed\-forward and both projections\) with respect to its input at initialisation, on real masked batches and identical encoders, and formedμ~\\tilde\{\\mu\}from the singular values of that Jacobian, at predictor depths22,66and1212with three seeds each\. It does not separate the two predictors: the seed ranges overlap at every depth, at depth66from0\.3130\.313to0\.3260\.326forResidualPredagainst0\.3120\.312to0\.3150\.315for the standard predictor\. A magnitude functional discards the sign of the drive, and a large singular value can coexist with no growth \(Theorem[A\.2](https://arxiv.org/html/2610.02344#A1.Thmtheorem2)\(v\)\)\.

### K\.2The target\-aligned drive after a few predictor\-only steps

We probe along the data eigenmodesbℓb\_\{\\ell\}and read the signed driveγ^ℓ=⟨r¯ℓ,bℓ⟩\\hat\{\\gamma\}\_\{\\ell\}=\\langle\\bar\{r\}\_\{\\ell\},b\_\{\\ell\}\\rangle, the drive side of Eq\. \([2](https://arxiv.org/html/2610.02344#S3.E2)\) with𝐖p\\mathbf\{W\}\_\{p\}replaced by the block’s input Jacobian, afterKKpredictor\-only steps with the encoder fixed at its initialisation, the initial encoder as teacher, and the recipe’s optimiser and learning rate\. This functional was introduced after the initialisation\-time result above\.

In every matched initialisation at every depth,ResidualPred’s aligned drive atK=10K=10is above the standard predictor’s,99of99, with per\-seed ratios1\.971\.97,1\.801\.80,1\.701\.70at depth22,2\.602\.60,2\.142\.14,1\.171\.17at depth66, and5\.435\.43,4\.534\.53,9\.299\.29at depth1212\. The leading real eigenvalue of the block’s transmission operator, which keeps the sign structure the singular\-value form of Appendix[A\.4](https://arxiv.org/html/2610.02344#A1.SS4)discards, is also higher in the same99of99, with ratios from1\.061\.06to8\.568\.56\. OverK∈\{1,2,3,5,7,10\}K\\in\\\{1,2,3,5,7,10\\\}the paired ratio is above one in all3636\(seed, depth,KK\) cells at depths22and66, peaking nearK=5K=5to77\.

These measurements support two statements: the identity bias raises the target\-aligned drive the predictor establishes in its first equilibration steps, and the initialisation\-time quantity is null on images\. Embedding effective rank is the only quantity claimed to track the accuracy gain, as necessary rather than sufficient and as depth\-limited: on CIFAR\-100 at predictor depth1212, SIGReg raises effective rank by34\.534\.5to61\.161\.1while accuracy does not move\. No estimator ofμ\\muis claimed to mediate the accuracy gain\.

## NeurIPS Paper Checklist

1. 1\.Claims
2. Question: Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
3. Answer:\[Yes\]
4. Justification: Abstract and Sec\.[1](https://arxiv.org/html/2610.02344#S1)state the drive–decay theory, theμ\\mu\-threshold, the800\+800\{\+\}\-configuration validation, and theResidualPredgain on tabular and image benchmarks; each claim maps to a section \(Sec\.[3](https://arxiv.org/html/2610.02344#S3), Sec\.[5](https://arxiv.org/html/2610.02344#S5), Sec\.[6](https://arxiv.org/html/2610.02344#S6)\)\.
5. Guidelines: - •The answer\[N/A\]means that the abstract and introduction do not include the claims made in the paper\. - •The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations\. A\[No\]or\[N/A\]answer to this question will not be perceived well by the reviewers\. - •The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings\. - •It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper\.
6. 2\.Limitations
7. Question: Does the paper discuss the limitations of the work performed by the authors?
8. Answer:\[Yes\]
9. Justification: Sec\.[7](https://arxiv.org/html/2610.02344#S7)\(Limitations paragraph\) discusses linearity, the spectral\-decoupling assumption, dataset\-dependent tightness, and the local nature ofμ=1\\mu\{=\}1\.
10. Guidelines: - •The answer\[N/A\]means that the paper has no limitation while the answer\[No\]means that the paper has limitations, but those are not discussed in the paper\. - •The authors are encouraged to create a separate “Limitations” section in their paper\. - •The paper should point out any strong assumptions and how robust the results are to violations of these assumptions \(e\.g\., independence assumptions, noiseless settings, model well\-specification, asymptotic approximations only holding locally\)\. The authors should reflect on how these assumptions might be violated in practice and what the implications would be\. - •The authors should reflect on the scope of the claims made, e\.g\., if the approach was only tested on a few datasets or with a few runs\. In general, empirical results often depend on implicit assumptions, which should be articulated\. - •The authors should reflect on the factors that influence the performance of the approach\. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting\. Or a speech\-to\-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon\. - •The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size\. - •If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness\. - •While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren’t acknowledged in the paper\. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community\. Reviewers will be specifically instructed to not penalize honesty concerning limitations\.
11. 3\.Theory assumptions and proofs
12. Question: For each theoretical result, does the paper provide the full set of assumptions and a complete \(and correct\) proof?
13. Answer:\[Yes\]
14. Justification: Lemma[3\.1](https://arxiv.org/html/2610.02344#S3.Thmtheorem1), Assumption[3\.2](https://arxiv.org/html/2610.02344#S3.Thmtheorem2), Corollary[3\.3](https://arxiv.org/html/2610.02344#S3.Thmtheorem3), and Theorem[F\.1](https://arxiv.org/html/2610.02344#A6.Thmtheorem1)are stated with assumptions; full proofs are in App\.[A](https://arxiv.org/html/2610.02344#A1), App\.[C](https://arxiv.org/html/2610.02344#A3), App\.[B](https://arxiv.org/html/2610.02344#A2)\.
15. Guidelines: - •The answer\[N/A\]means that the paper does not include theoretical results\. - •All the theorems, formulas, and proofs in the paper should be numbered and cross\-referenced\. - •All assumptions should be clearly stated or referenced in the statement of any theorems\. - •The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition\. - •Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material\. - •Theorems and Lemmas that the proof relies upon should be properly referenced\.
16. 4\.Experimental result reproducibility
17. Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper \(regardless of whether the code and data are provided or not\)?
18. Answer:\[Yes\]
19. Justification: Sec\.[4](https://arxiv.org/html/2610.02344#S4)and App\.[G](https://arxiv.org/html/2610.02344#A7)report datasets, splits, optimiser, schedule, and seed protocol; supplementary material ships the full pipeline with reproduction scripts\.
20. Guidelines: - •The answer\[N/A\]means that the paper does not include experiments\. - •If the paper includes experiments, a\[No\]answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not\. - •If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable\. - •Depending on the contribution, reproducibility can be accomplished in various ways\. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model\. In general\. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model \(e\.g\., in the case of a large language model\), releasing of a model checkpoint, or other means that are appropriate to the research performed\. - •While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution\. For example 1. \(a\)If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm\. 2. \(b\)If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully\. 3. \(c\)If the contribution is a new model \(e\.g\., a large language model\), then there should either be a way to access this model for reproducing the results or a way to reproduce the model \(e\.g\., with an open\-source dataset or instructions for how to construct the dataset\)\. 4. \(d\)We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility\. In the case of closed\-source models, it may be that access to the model is limited in some way \(e\.g\., to registered users\), but it should be possible for other researchers to have some path to reproducing or verifying the results\.
21. 5\.Open access to data and code
22. Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?
23. Answer:\[Yes\]
24. Justification: Anonymised code, configs, and per\-run results are in the supplementary, withreproduce\_all\.sh,reproduce\_subset\.sh,test\.sh; tabular data via OpenML, image data viatorchvision\.
25. Guidelines: - •The answer\[N/A\]means that paper does not include experiments requiring code\. - • - •While we encourage the release of code and data, we understand that this might not be possible, so\[No\]is an acceptable answer\. Papers cannot be rejected simply for not including code, unless this is central to the contribution \(e\.g\., for a new open\-source benchmark\)\. - •The instructions should contain the exact command and environment needed to run to reproduce the results\. See the NeurIPS code and data submission guidelines \([https://neurips\.cc/public/guides/CodeSubmissionPolicy](https://neurips.cc/public/guides/CodeSubmissionPolicy)\) for more details\. - •The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc\. - •The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines\. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why\. - •At submission time, to preserve anonymity, the authors should release anonymized versions \(if applicable\)\. - •Providing as much information as possible in supplemental material \(appended to the paper\) is recommended, but including URLs to data and code is permitted\.
26. 6\.Experimental setting/details
27. Question: Does the paper specify all the training and test details \(e\.g\., data splits, hyperparameters, how they were chosen, type of optimizer\) necessary to understand the results?
28. Answer:\[Yes\]
29. Justification: Sec\.[4](https://arxiv.org/html/2610.02344#S4)\(splits, optimiser, schedule\), App\.[G](https://arxiv.org/html/2610.02344#A7)\(full hyperparameters\), and App\.[D](https://arxiv.org/html/2610.02344#A4)/App\.[E](https://arxiv.org/html/2610.02344#A5)\(variant\-specific settings\)\.
30. Guidelines: - •The answer\[N/A\]means that the paper does not include experiments\. - •The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them\. - •The full details can be provided either with the code, in appendix, or as supplemental material\.
31. 7\.Experiment statistical significance
32. Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
33. Answer:\[Yes\]
34. Justification: Tables report mean±\\pmstdev across seeds; the rank\-control study reports Spearmanρ\\rho\.
35. Guidelines: - •The answer\[N/A\]means that the paper does not include experiments\. - •The authors should answer\[Yes\]if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper\. - •The factors of variability that the error bars are capturing should be clearly stated \(for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions\)\. - •The method for calculating the error bars should be explained \(closed form formula, call to a library function, bootstrap, etc\.\) - •The assumptions made should be given \(e\.g\., Normally distributed errors\)\. - •It should be clear whether the error bar is the standard deviation or the standard error of the mean\. - •It is OK to report 1\-sigma error bars, but one should state it\. The authors should preferably report a 2\-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified\. - •For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range \(e\.g\., negative error rates\)\. - •If error bars are reported in tables or plots, the authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text\.
36. 8\.Experiments compute resources
37. Question: For each experiment, does the paper provide sufficient information on the computer resources \(type of compute workers, memory, time of execution\) needed to reproduce the experiments?
38. Answer:\[Yes\]
39. Justification: App\.[G](https://arxiv.org/html/2610.02344#A7)and the supplementaryREADME\.mdlist per\-experiment GPU\-hours on a single A100\.
40. Guidelines: - •The answer\[N/A\]means that the paper does not include experiments\. - •The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage\. - •The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute\. - •The paper should disclose whether the full research project required more compute than the experiments reported in the paper \(e\.g\., preliminary or failed experiments that didn’t make it into the paper\)\.
41. 9\.Code of ethics
43. Answer:\[Yes\]
44. Justification: Theoretical and methodological work on representation learning using public datasets; no human subjects, no scraped or sensitive data\.
45. Guidelines: - •The answer\[N/A\]means that the authors have not reviewed the NeurIPS Code of Ethics\. - •If the authors answer\[No\], they should explain the special circumstances that require a deviation from the Code of Ethics\. - •The authors should make sure to preserve anonymity \(e\.g\., if there is a special consideration due to laws or regulations in their jurisdiction\)\.
46. 10\.Broader impacts
47. Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
48. Answer:\[N/A\]
49. Justification: Foundational theoretical work on JEPA training stability; no direct path to a deployable application or societal impact beyond improved representation learning\.
50. Guidelines: - •The answer\[N/A\]means that there is no societal impact of the work performed\. - •If the authors answer\[N/A\]or\[No\], they should explain why their work has no societal impact or why the paper does not address societal impact\. - •Examples of negative societal impacts include potential malicious or unintended uses \(e\.g\., disinformation, generating fake profiles, surveillance\), fairness considerations \(e\.g\., deployment of technologies that could make decisions that unfairly impact specific groups\), privacy considerations, and security considerations\. - •The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments\. However, if there is a direct path to any negative applications, the authors should point it out\. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate Deepfakes for disinformation\. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster\. - •The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from \(intentional or unintentional\) misuse of the technology\. - •If there are negative societal impacts, the authors could also discuss possible mitigation strategies \(e\.g\., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML\)\.
51. 11\.Safeguards
52. Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse \(e\.g\., pre\-trained language models, image generators, or scraped datasets\)?
53. Answer:\[N/A\]
54. Justification: No high\-risk model or dataset is released; only research code and per\-run training metrics\.
55. Guidelines: - •The answer\[N/A\]means that the paper poses no such risks\. - •Released models that have a high risk for misuse or dual\-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters\. - •Datasets that have been scraped from the Internet could pose safety risks\. The authors should describe how they avoided releasing unsafe images\. - •We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort\.
56. 12\.Licenses for existing assets
57. Question: Are the creators or original owners of assets \(e\.g\., code, data, models\), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
58. Answer:\[Yes\]
59. Justification: I\-JEPA\[[Assran et al\., 2023](https://arxiv.org/html/2610.02344#bib.bib1)\]and T\-JEPA\[[Thimonier et al\., 2025](https://arxiv.org/html/2610.02344#bib.bib29)\]cited; OpenML datasets \(aloi,helena,jannis\) andtorchvisionCIFAR\-10/100, STL\-10 used under their public licenses\.
60. Guidelines: - •The answer\[N/A\]means that the paper does not use existing assets\. - •The authors should cite the original paper that produced the code package or dataset\. - •The authors should state which version of the asset is used and, if possible, include a URL\. - •The name of the license \(e\.g\., CC\-BY 4\.0\) should be included for each asset\. - •For scraped data from a particular source \(e\.g\., website\), the copyright and terms of service of that source should be provided\. - •If assets are released, the license, copyright information, and terms of use in the package should be provided\. For popular datasets,[paperswithcode\.com/datasets](https://paperswithcode.com/datasets)has curated licenses for some datasets\. Their licensing guide can help determine the license of a dataset\. - •For existing datasets that are re\-packaged, both the original license and the license of the derived asset \(if it has changed\) should be provided\. - •If this information is not available online, the authors are encouraged to reach out to the asset’s creators\.
61. 13\.New assets
62. Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
63. Answer:\[Yes\]
64. Justification: Supplementary ships an anonymised codebase withREADME\.mdand reproduction scripts\.
65. Guidelines: - •The answer\[N/A\]means that the paper does not release new assets\. - •Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates\. This includes details about training, license, limitations, etc\. - •The paper should discuss whether and how consent was obtained from people whose asset is used\. - •At submission time, remember to anonymize your assets \(if applicable\)\. You can either create an anonymized URL or include an anonymized zip file\.
66. 14\.Crowdsourcing and research with human subjects
67. Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation \(if any\)?
68. Answer:\[N/A\]
69. Justification: No crowdsourcing or human\-subjects research\.
70. Guidelines: - •The answer\[N/A\]means that the paper does not involve crowdsourcing nor research with human subjects\. - •Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper\. - •According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector\.
71. 15\.Institutional review board \(IRB\) approvals or equivalent for research with human subjects
72. Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board \(IRB\) approvals \(or an equivalent approval/review based on the requirements of your country or institution\) were obtained?
73. Answer:\[N/A\]
74. Justification: No human\-subjects research\.
75. Guidelines: - •The answer\[N/A\]means that the paper does not involve crowdsourcing nor research with human subjects\. - •Depending on the country in which research is conducted, IRB approval \(or equivalent\) may be required for any human subjects research\. If you obtained IRB approval, you should clearly state this in the paper\. - •We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution\. - •For initial submissions, do not include any information that would break anonymity \(if applicable\), such as the institution conducting the review\.
76. 16\.Declaration of LLM usage
77. Question: Does the paper describe the usage of LLMs if it is an important, original, or non\-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does*not*impact the core methodology, scientific rigor, or originality of the research, declaration is not required\.
78. Answer:\[N/A\]
79. Justification: LLMs are not part of the core methodology; any use was limited to writing and editing\.
80. Guidelines: - •The answer\[N/A\]means that the core method development in this research does not involve LLMs as any important, original, or non\-standard components\. - •Please refer to our LLM policy in the NeurIPS handbook for what should or should not be described\.