Learning the Koopman Operator using Attention Free Transformers

# Learning the Koopman Operator using Attention Free Transformers
Source: [https://arxiv.org/html/2606.23957](https://arxiv.org/html/2606.23957)
Mohammed Nagdi1, Evangelos\-Marios Nikolados1, Alexey Yermakov2, Mars Gao3, Nathan Kutz2& Filippo Menolascina1 1Institute for Bioengineering, School of Engineering and Centre for Engineering Biology, University of Edinburgh, Edinburgh, United Kingdom 2Electrical and Computer Engineering and Applied Mathematics, University of Washington, Seattle, WA, USA 3Electrical and Computer Engineering and Computer Science & Engineering, University of Washington, Seattle, WA, USA filippo\.menolascina@ed\.ac\.uk

###### Abstract

Learning Koopman operators with autoencoders enables linear prediction in a latent space, but long\-horizon rollouts often drift off the learned manifold, leading to phase and amplitude errors on systems with switching, continuous spectra, or strong transients\. We introduce two complementary components that make Koopman predictors more robust\. First, we add an*attention\-free latent memory*\(AFT\) block that aggregates a short window of past latents to produce a corrected latent before each Koopman update\. Unlike multi\-head attention, AFT operates in linear time and adds only≈30\{\\approx\}30k parameters \(3​d2\+T23d^\{2\}\{\+\}T^\{2\}, fewer than matched multi\-head attention\), yet captures the local temporal context needed to suppress error divergence\. Second, we propose*dynamic re\-encoding*: lightweight, online change\-point triggers \(EWMA, CUSUM, and sequential two\-sample tests\) that detect latent drift and project predictions back onto the autoencoder manifold\. Across three benchmark systems—Duffing oscillator, Repressilator, IRMA—our model consistently reduces error accumulation compared to a Koopman autoencoder and matched\-capacity multi\-head attention\. We also compare against GRU and Transformer autoencoders, evaluated both from initial conditions and with a 50\-step context, and find that Koopman\+AFT \(with optional re\-encoding\) attains markedly lower long\-horizon error while maintaining lower inference latency\. We report improvements over horizons up to 1000 steps, together with ablations over trigger policies\. The result is a fast, compact predictor that stays on the learned manifold over long horizons\.

## 1Introduction

The Koopman operator offers a principled way to analyze nonlinear dynamics with linear tools by lifting states to an observable space where evolution is linear\(Koopman,[1931](https://arxiv.org/html/2606.23957#bib.bib1)\)\. Neural implementations of this idea—most commonly, Koopman autoencoders \(KAE\) that learn an encoderφ\\varphi, a linear mapKK, and a decoderφ−1\\varphi^\{\-1\}—often deliver strong single\-step accuracy, but drift in long rollouts: phases slip in oscillators, amplitudes decay or explode, and trajectories peel away from attractors\(Luschet al\.,[2018](https://arxiv.org/html/2606.23957#bib.bib48)\)\. Empirically, failures are pronounced in settings with \(i\) continuous or mixed spectra \(e\.g\., undamped oscillators\), \(ii\) switching between metastable basins, and \(iii\) transient regimes where small errors compound\. This motivates mechanisms that \(a\) use short\-term temporal context to correct local errors, akin to delay\-embedding ideas in Hankel DMD / HAVOK\(Arbabi and Mezic,[2017](https://arxiv.org/html/2606.23957#bib.bib67); Bruntonet al\.,[2017](https://arxiv.org/html/2606.23957#bib.bib68)\), and \(b\) periodically project predictions back onto the learned manifold before drift becomes catastrophic\. Robustness over long horizons is therefore critical: predictors that remain near the learned manifold accumulate fewer errors and are easier to certify and use in downstream control\.

Approach overview and intuition\.We augment a standard KAE with two pieces\. \(i\) An*attention\-free latent memory*\(AFT\) block aggregates a short window of past latents and produces a corrected latent before each Koopman update, achieving linear time/memory in the context length while capturing the local correlations that drive phase and amplitude drift\(Zhaiet al\.,[2021](https://arxiv.org/html/2606.23957#bib.bib59)\)\. \(ii\)*Dynamic re\-encoding*uses lightweight streaming triggers \(EWMA, CUSUM, sequential two\-sample, and simple threshold/window tests\) to detect latent drift and apply an encode–decode–encode \(E–D–E\) projection that snaps predictions back to the autoencoder manifold\(Roberts,[2000](https://arxiv.org/html/2606.23957#bib.bib55); Moustakides,[1986](https://arxiv.org/html/2606.23957#bib.bib54); Ross and Adams,[2012](https://arxiv.org/html/2606.23957#bib.bib56)\)\. Intuitively, AFT addresses*how*we step—reducing local error before propagation byKK—while re\-encoding addresses*where*we step—bounding accumulated drift\. The mechanisms are orthogonal: one*prevents*growth, the other*bounds*it\.

##### Relation to prior work\.

Our approach builds on data\-driven Koopman learning from EDMD with fixed dictionaries to learned latent embeddings with linearly recurrent bottlenecks\(Liet al\.,[2017](https://arxiv.org/html/2606.23957#bib.bib42); Otto and Rowley,[2019](https://arxiv.org/html/2606.23957#bib.bib41); Luschet al\.,[2018](https://arxiv.org/html/2606.23957#bib.bib48)\)\. Short time\-delay context has long been used to stabilize prediction \(Hankel DMD, HAVOK\)\(Arbabi and Mezic,[2017](https://arxiv.org/html/2606.23957#bib.bib67); Bruntonet al\.,[2017](https://arxiv.org/html/2606.23957#bib.bib68)\), motivating our lightweight latent memory\. Compared to transformer\-style attention used in recent hybrids\(Luet al\.,[2024](https://arxiv.org/html/2606.23957#bib.bib63); Wanget al\.,[2022](https://arxiv.org/html/2606.23957#bib.bib65)\), our attention\-free block achieves linear cost while targeting the local correlations that drive phase/amplitude drift\. Orthogonally, projection/consistency ideas\(Nayaket al\.,[2025](https://arxiv.org/html/2606.23957#bib.bib69); Frionet al\.,[2025](https://arxiv.org/html/2606.23957#bib.bib70); Noacket al\.,[2015](https://arxiv.org/html/2606.23957#bib.bib72); Dylewskyet al\.,[2019](https://arxiv.org/html/2606.23957#bib.bib73); Guanet al\.,[2024](https://arxiv.org/html/2606.23957#bib.bib71)\)inspire our encode–decode–encode snap\-back mechanism\. For triggering, we adopt classic streaming drift detectors \(EWMA, CUSUM, sequential two\-sample\)\(Roberts,[2000](https://arxiv.org/html/2606.23957#bib.bib55); Moustakides,[1986](https://arxiv.org/html/2606.23957#bib.bib54); Ross and Adams,[2012](https://arxiv.org/html/2606.23957#bib.bib56)\)\. Broader lines on inputs and control \(KIC, Koopman MPC, safety/verification\) are discussed in Appendix[B](https://arxiv.org/html/2606.23957#A2), along with domain\-specific applications in biology and fluid mechanics\.

Benchmarks\.We target three representative systems that stress long\-horizon stability in complementary ways: \(i\) theDuffing oscillatorin the unforced, undamped regime, which exhibits closed orbits, switching between wells at higher energies, and commonly a continuous or mixed Koopman spectrum that stresses linear predictors\(Otto and Rowley,[2019](https://arxiv.org/html/2606.23957#bib.bib41); Liet al\.,[2017](https://arxiv.org/html/2606.23957#bib.bib42); Pan and Duraisamy,[2020](https://arxiv.org/html/2606.23957#bib.bib39); Alford\-Lagoet al\.,[2022](https://arxiv.org/html/2606.23957#bib.bib40); Köhneet al\.,[2025](https://arxiv.org/html/2606.23957#bib.bib38)\); \(ii\) theRepressilator, a synthetic three\-gene negative\-feedback oscillator with a canonical limit cycle\(Elowitz and Leibler,[2000](https://arxiv.org/html/2606.23957#bib.bib33)\), widely used to evaluate identification and control\(Boddupalliet al\.,[2019](https://arxiv.org/html/2606.23957#bib.bib34); Sootlaet al\.,[2018](https://arxiv.org/html/2606.23957#bib.bib35); Balakrishnanet al\.,[2022](https://arxiv.org/html/2606.23957#bib.bib36); Perez\-Carrascoet al\.,[2018](https://arxiv.org/html/2606.23957#bib.bib37)\); and \(iii\)IRMA\(*In vivo Reverse\-engineering and Modelling Assessment*\), a five\-gene yeast circuit constructed as a benchmark for modeling and control\(Cantoneet al\.,[2009](https://arxiv.org/html/2606.23957#bib.bib46); Marucciet al\.,[2009](https://arxiv.org/html/2606.23957#bib.bib45); Menolascinaet al\.,[2014](https://arxiv.org/html/2606.23957#bib.bib43); di Bernardoet al\.,[2011](https://arxiv.org/html/2606.23957#bib.bib44)\)and representative of multi\-gene regulatory dynamics where deep Koopman approaches have shown promise\(Hasnainet al\.,[2019](https://arxiv.org/html/2606.23957#bib.bib47)\)\. These three cover, respectively, mixed spectra and switching \(Duffing\), clean oscillatory behavior with phase sensitivity \(Repressilator\), and higher\-dimensional regulatory dynamics with intertwined feedback \(IRMA\)\.

Empirical summary\.We evaluate on the three primary benchmarks \(Duffing, Repressilator, IRMA\) and report both MSE and a long\-horizon*mean cumulative absolute error*\(MCAE\) that is sensitive to error accumulation\. The latent memory block outperforms matched\-capacity MHA \(4 and 10 heads\) on these systems, and coupling it with dynamic re\-encoding yields the most robust rollouts on switching and feedback\-rich systems \(Duffing, IRMA\), while plain AFT remains best on clean limit cycles \(Repressilator\)\. GRU and Transformer autoencoders, evaluated both from initial conditions and with a 50\-step context, underperform on long horizons despite their added context\.

##### Contributions\.

- •Attention\-free latent memory for Koopman prediction\.A linear\-time, low\-overhead block\(Zhaiet al\.,[2021](https://arxiv.org/html/2606.23957#bib.bib59)\)aggregates a short history of latents to produce a corrected latent before each Koopman update, reducing error accumulation on long rollouts\.
- •Dynamic re\-encoding via streaming change detection\.An encode–decode–encode projection with online triggers \(EWMA, CUSUM, sequential two\-sample, threshold/window\) detects latent drift and snaps predictions back to the learned manifold\(Roberts,[2000](https://arxiv.org/html/2606.23957#bib.bib55); Moustakides,[1986](https://arxiv.org/html/2606.23957#bib.bib54); Ross and Adams,[2012](https://arxiv.org/html/2606.23957#bib.bib56)\)\.
- •Evaluation and ablations on three representative systems\.On Duffing \(unforced, undamped\), Repressilator, and IRMA, latent memory outperforms matched MHA; latent memory \+ re\-encoding attains the lowest MSE over200/500/1000200/500/1000\-step horizons; and gains persist across Koopman operator sizes\.

![Refer to caption](https://arxiv.org/html/2606.23957v1/x1.png)Figure 1:Workflow of the Koopman autoencoder with AFT and Dynamic Re\-encoding\. \(a\)Sampled trajectories from a Duffing Oscillator serve as input\.\(b\)The coreKoopman autoencoderlearns a linear latent representation by minimizing reconstruction, linearization, and prediction losses\.\(c\)The prediction process uses aDynamic Re\-encodingmodule withAFT attentionto refine the latent state \(zt→z~tz\_\{t\}\\to\\tilde\{z\}\_\{t\}\), which is then evolved by the learnedKoopman operator K\. \(d\) The final output shows predicted trajectories matching the reference dynamics\.

## 2Methods

### 2\.1Baseline Koopman Autoencoder \(KAE\)

##### Model\.

Letxt∈ℝpx\_\{t\}\\in\\mathbb\{R\}^\{p\}denote the observed state at timettand letφ:ℝp→ℝd\\varphi:\\mathbb\{R\}^\{p\}\\\!\\to\\\!\\mathbb\{R\}^\{d\}andφ−1:ℝd→ℝp\\varphi^\{\-1\}:\\mathbb\{R\}^\{d\}\\\!\\to\\\!\\mathbb\{R\}^\{p\}be an encoder/decoder pair that maps to add–dimensional latent space\. The KAE posits a*linear*latent evolution governed by a learned Koopman matrixK∈ℝd×dK\\in\\mathbb\{R\}^\{d\\times d\}:

zt=φ​\(xt\),zt\+1=K​zt,x^t=φ−1​\(zt\),z\_\{t\}\\;=\\;\\varphi\(x\_\{t\}\),\\qquad z\_\{t\+1\}\\;=\\;K\\,z\_\{t\},\\qquad\\hat\{x\}\_\{t\}\\;=\\;\\varphi^\{\-1\}\(z\_\{t\}\),\(1\)so that anii–step rollout from an initial latentz0z\_\{0\}iszi=Ki​z0z\_\{i\}=K^\{i\}z\_\{0\}with decoded predictionx^i=φ−1​\(Ki​φ​\(x0\)\)\\hat\{x\}\_\{i\}=\\varphi^\{\-1\}\(K^\{i\}\\varphi\(x\_\{0\}\)\)\. We use the standard linearly recurrent bottleneck architecture\(Otto and Rowley,[2019](https://arxiv.org/html/2606.23957#bib.bib41); Luschet al\.,[2018](https://arxiv.org/html/2606.23957#bib.bib48)\)and learn\(φ,φ−1,K\)\(\\varphi,\\varphi^\{\-1\},K\)end\-to\-end\.

##### Training losses\.

Given an input segment\(x0,…,xT\)\(x\_\{0\},\\ldots,x\_\{T\}\), we minimize a weighted sum of \(i\) reconstruction error, \(ii\) linearity consistency in the latent space, \(iii\) decoded prediction error over the rollout, and \(iv\) a unitary regularizer onKKto discourage exploding/vanishing spectra \(cf\.Enyeart and Lin,[2024](https://arxiv.org/html/2606.23957#bib.bib60)\):

ℒ\\displaystyle\\mathcal\{L\}=α1​\(ℒrecon\+ℒpred\)\+ℒlin\+α2​ℒunitary,\\displaystyle=\\alpha\_\{1\}\(\\mathcal\{L\}\_\{\\mathrm\{recon\}\}\+\\mathcal\{L\}\_\{\\mathrm\{pred\}\}\)\+\\mathcal\{L\}\_\{\\mathrm\{lin\}\}\+\\alpha\_\{2\}\\mathcal\{L\}\_\{\\mathrm\{unitary\}\},\(2a\)ℒrecon\\displaystyle\\mathcal\{L\}\_\{\\mathrm\{recon\}\}=1T\+1​∑t=0T‖xt−φ−1​\(φ​\(xt\)\)‖22,\\displaystyle=\\frac\{1\}\{T\+1\}\\sum\_\{t=0\}^\{T\}\\\|x\_\{t\}\-\\varphi^\{\-1\}\(\\varphi\(x\_\{t\}\)\)\\\|\_\{2\}^\{2\},\(2b\)ℒlin\\displaystyle\\mathcal\{L\}\_\{\\mathrm\{lin\}\}=1T​∑i=1T‖φ​\(xi\)−Ki​φ​\(x0\)‖22,\\displaystyle=\\frac\{1\}\{T\}\\sum\_\{i=1\}^\{T\}\\\|\\varphi\(x\_\{i\}\)\-K^\{i\}\\varphi\(x\_\{0\}\)\\\|\_\{2\}^\{2\},\(2c\)ℒpred\\displaystyle\\mathcal\{L\}\_\{\\mathrm\{pred\}\}=1T​∑i=1T‖xi−φ−1​\(Ki​φ​\(x0\)\)‖22,\\displaystyle=\\frac\{1\}\{T\}\\sum\_\{i=1\}^\{T\}\\\|x\_\{i\}\-\\varphi^\{\-1\}\(K^\{i\}\\varphi\(x\_\{0\}\)\)\\\|\_\{2\}^\{2\},\(2d\)ℒunitary\\displaystyle\\mathcal\{L\}\_\{\\mathrm\{unitary\}\}=1d2​‖K​K⊤−I‖F\.\\displaystyle=\\tfrac\{1\}\{d^\{2\}\}\\\|KK^\{\\top\}\-I\\\|\_\{F\}\.\(2e\)
with weightsα1,α2\>0\\alpha\_\{1\},\\alpha\_\{2\}\\\!\>\\\!0\.ℒrecon\\mathcal\{L\}\_\{\\mathrm\{recon\}\}enforces an information\-preserving autoencoding,ℒlin\\mathcal\{L\}\_\{\\mathrm\{lin\}\}encourages consistency of the latent trajectory with powers ofKK, andℒpred\\mathcal\{L\}\_\{\\mathrm\{pred\}\}measures decoded multi\-step accuracy\. The unitary penalty mildly biasesKKtoward near\-orthogonality to improve long\-horizon stability\(Enyeart and Lin,[2024](https://arxiv.org/html/2606.23957#bib.bib60)\)\. We train by unrolling equation[1](https://arxiv.org/html/2606.23957#S2.E1)forTTsteps fromx0x\_\{0\}, computing all four losses on the same segment\. The formulation in equation[2](https://arxiv.org/html/2606.23957#S2.E2)matches common KAE practice\(Otto and Rowley,[2019](https://arxiv.org/html/2606.23957#bib.bib41); Luschet al\.,[2018](https://arxiv.org/html/2606.23957#bib.bib48)\)while making the stability prior explicit\.

### 2\.2Attention\-Free Latent Memory \(AFT\)

##### Setup\.

To mitigate local phase/amplitude drift, we augment the KAE with a lightweight latent memory that aggregates the lastTTlatents before each Koopman step \(here,TTdenotes the*AFT context length*, not the training segment length in equation[2](https://arxiv.org/html/2606.23957#S2.E2)\)\. Let the latent history at timettbeHt=\[zt−T,…,zt−1\]∈ℝT×dH\_\{t\}=\[z\_\{t\-T\},\\ldots,z\_\{t\-1\}\]\\in\\mathbb\{R\}^\{T\\times d\}\(we use causal indexing andT≪T\\\!\\ll\\\!rollout length\)\. The AFT block mapsHtH\_\{t\}to a corrected latent and updates

z~t−1=AFT​\(Ht\),zt=K​z~t−1,\\tilde\{z\}\_\{t\-1\}\\;=\\;\\mathrm\{AFT\}\(H\_\{t\}\),\\qquad z\_\{t\}\\;=\\;K\\,\\tilde\{z\}\_\{t\-1\},\(3\)so the Koopman propagator advances a*corrected*latent\.

##### Computation\.

This variant is a plug\-in replacement for Multi\-Head Attention \(MHA\) and can be considered an element\-wise linear attention mechanism\. We use the AFT\-full variant of this approach introduced byZhaiet al\.\([2021](https://arxiv.org/html/2606.23957#bib.bib59)\), where given a latent representation ofxtx\_\{t\}in the Koopman subspaceZtZ\_\{t\}\. We apply learned linear mapsWQ,WK,WV∈ℝd×dW\_\{Q\},W\_\{K\},W\_\{V\}\\in\\mathbb\{R\}^\{d\\times d\},

Qt=zt−1​WQ∈ℝd,Kt=Ht​WK/dm​o​d​e​l∈ℝT×d,\\displaystyle Q\_\{t\}\\;=\\;z\_\{t\-1\}W\_\{Q\}\\in\\mathbb\{R\}^\{d\},\\qquad K\_\{t\}\\;=\\;H\_\{t\}W\_\{K\}/\\sqrt\{d\_\{model\}\}\\in\\mathbb\{R\}^\{T\\times d\},\(4\)Vt=Ht​WV/dm​o​d​e​l∈ℝT×d,\\displaystyle V\_\{t\}\\;=\\;H\_\{t\}W\_\{V\}/\\sqrt\{d\_\{model\}\}\\in\\mathbb\{R\}^\{T\\times d\},then performs the following operation:

z~t−1=σq​\(Qt\)⊙∑t′=1Texp⁡\(Kt′\+wt,t′\)⊙Vt′∑t′=1Texp⁡\(Kt′\+wt,t′\)\\tilde\{z\}\_\{t\-1\}=\\sigma\_\{q\}\(Q\_\{t\}\)\\odot\\frac\{\\sum\_\{t^\{\\prime\}=1\}^\{T\}\\exp\(K\_\{t^\{\\prime\}\}\+w\_\{t,t^\{\\prime\}\}\)\\odot V\_\{t^\{\\prime\}\}\}\{\\sum\_\{t^\{\\prime\}=1\}^\{T\}\\exp\(K\_\{t^\{\\prime\}\}\+w\_\{t,t^\{\\prime\}\}\)\}\(5\)whereσq​\(⋅\)\\sigma\_\{q\}\(\\cdot\)represents sigmoid activation on queries,w∈ℝT×Tw\\in\\mathbb\{R\}^\{T\\times T\}denotes learnable positional biases, and⊙\\odotindicates element\-wise multiplication\. The mechanism computes attention weights from keys and positional biases, applies them to values, and then combines the result with activated queries through element\-wise operations\. This provides the benefits of attention mechanisms while maintaining linear computational complexity with respect to sequence length\. In addition to that, we also added a key/value scaling bydm​o​d​e​l\\sqrt\{d\_\{model\}\}to ensure numerical stability\.

##### Use in the predictor\.

We apply equation[3](https://arxiv.org/html/2606.23957#S2.E3)at every step with a rolling windowHtH\_\{t\}\(see Algorithm[1](https://arxiv.org/html/2606.23957#algorithm1)in Appendix[F](https://arxiv.org/html/2606.23957#A6)for the full predictor\)\. The block is*drop\-in*: it changes neither the decoder nor the Koopman loss structure in equation[2](https://arxiv.org/html/2606.23957#S2.E2)\. Empirically, the corrected latentz~t−1\\tilde\{z\}\_\{t\-1\}corrects local phase and amplitude errors before propagation byKK, reducing error accumulation over long rollouts while preserving speed and model compactness\. We train two variants: inKAE\+AFTthe block output is the corrected latent,z~t−1=AFT​\(Ht\)\\tilde\{z\}\_\{t\-1\}=\\mathrm\{AFT\}\(H\_\{t\}\); inKAE\+AFT\+Re\-enc\(§[2\.3](https://arxiv.org/html/2606.23957#S2.SS3)\) the same block is applied in residual form,z~t−1=zt−1\+AFT​\(Ht\)\\tilde\{z\}\_\{t\-1\}=z\_\{t\-1\}\+\\mathrm\{AFT\}\(H\_\{t\}\), so the original and re\-encoded branches \(Alg\.[2](https://arxiv.org/html/2606.23957#algorithm2)\) share one increment and differ only in their base latent\. Accordingly,AFT​\(⋅\)\\mathrm\{AFT\}\(\\cdot\)denotes the corrected latent in the former and the residual increment in the latter\.

### 2\.3Dynamic Re\-Encoding \(E–D–E projection and streaming triggers\)

##### Encode–Decode–Encode projection\.

Let𝒫​\(z\)≔φ​\(φ−1​\(z\)\)\\mathcal\{P\}\(z\)\\coloneqq\\varphi\\\!\\big\(\\varphi^\{\-1\}\(z\)\\big\)denote the autoencoder\-induced projection of a latentzzback onto the learned manifold \(idempotent by construction\)\. During rollout, at each step, we form a pre\-update latentz~t−1\\tilde\{z\}\_\{t\-1\}\. We compute two one\-step predictions:

ztpred=K​z~t−1,ztre−pred=K​𝒫​\(z~t−1\)\.z^\{\\mathrm\{pred\}\}\_\{t\}\\;=\\;K\\,\\tilde\{z\}\_\{t\-1\},\\qquad z^\{\\mathrm\{re\\\!\-\\\!pred\}\}\_\{t\}\\;=\\;K\\,\\mathcal\{P\}\(\\tilde\{z\}\_\{t\-1\}\)\.Their discrepancy defines a*drift proxy*

δt≜‖ztre−pred−ztpred‖22,\\delta\_\{t\}\\;\\triangleq\\;\\big\\\|\\,z^\{\\mathrm\{re\\\!\-\\\!pred\}\}\_\{t\}\-z^\{\\mathrm\{pred\}\}\_\{t\}\\,\\big\\\|\_\{2\}^\{2\},\(6\)which grows when the iterate leaves the learned manifold\. If a streaming trigger \(below\) fires at timett, we*snap*the latent back by replacingzt−1←𝒫​\(z~t−1\)z\_\{t\-1\}\\leftarrow\\mathcal\{P\}\(\\tilde\{z\}\_\{t\-1\}\)before propagating\. This keeps the Koopman update on\-manifold while leavingKKand the autoencoder unchanged\.

##### Streaming drift triggers\.

We instantiate four inexpensive, streaming tests on the scalar\{δt\}\\\{\\delta\_\{t\}\\\}:

1. 1\.Windowed Z\-score \(mean\+std\): maintainμt,σt\\mu\_\{t\},\\sigma\_\{t\}over a sliding window of sizewwand trigger ifδt\>μt\+τ​σt\\delta\_\{t\}\>\\mu\_\{t\}\+\\tau\\,\\sigma\_\{t\}\(hyperparameterτ\>0\\tau\>0\)\.
2. 2\.EWMA\(Roberts,[2000](https://arxiv.org/html/2606.23957#bib.bib55)\): updateZt=\(1−λ\)​Zt−1\+λ​δtZ\_\{t\}=\(1\-\\lambda\)Z\_\{t\-1\}\+\\lambda\\,\\delta\_\{t\}withZ0=δ1Z\_\{0\}=\\delta\_\{1\}and trigger when\|Zt−μZ\|\>L​σZ\|Z\_\{t\}\-\\mu\_\{Z\}\|\>\\\!L\\,\\sigma\_\{Z\}\(streaming estimates forμZ,σZ\\mu\_\{Z\},\\sigma\_\{Z\}; hyperparametersλ∈\(0,1\),L\>0\\lambda\\in\(0,1\),L\>0\)\.
3. 3\.CUSUM\(Moustakides,[1986](https://arxiv.org/html/2606.23957#bib.bib54)\): compute the standardized cumulative sums~t=∑i=1t\(δi−μ0\)t​σ\\tilde\{s\}\_\{t\}=\\frac\{\\sum\_\{i=1\}^\{t\}\(\\delta\_\{i\}\-\\mu\_\{0\}\)\}\{\\sqrt\{t\}\\,\\sigma\}and derive the p\-valuept=2​\[1−Φ​\(\|s~t\|\)\]p\_\{t\}=2\\left\[1\-\\Phi\\left\(\|\\tilde\{s\}\_\{t\}\|\\right\)\\right\], whereΦ\\Phiis the standard normal CDF\.
4. 4\.Sequential two\-sample\(Ross and Adams,[2012](https://arxiv.org/html/2606.23957#bib.bib56)\): compare a reference bufferRRand a current bufferCC\(disjoint, sizeww\) using a nonparametric test \(e\.g\., KS or Lepage\); re\-encode ifpt<αp\_\{t\}<\\alpha\(hyperparameterα\\alpha\)\.

All tests have low computational overhead per step: windowed Z\-score is𝒪​\(w\)\\mathcal\{O\}\(w\)for window sizeww, EWMA is𝒪​\(1\)\\mathcal\{O\}\(1\), and CUSUM is𝒪​\(1\)\\mathcal\{O\}\(1\)for time stepttwith incremental updates\. They are complementary: windowed Z\-score/EWMA react quickly to level shifts, CUSUM accumulates small persistent deviations, and two\-sample tests capture broader distributional changes\. In our experiments, we use fixed hyperparameters per system and evaluate several trigger families \(see §[4\.3](https://arxiv.org/html/2606.23957#S4.SS3)\)\. We use triggers only at*inference*; training proceeds without re\-encoding\.

## 3Experiments

### 3\.1Benchmarks and data generation

We evaluate three primary systems that emphasize long\-horizon stability in complementary ways: \(i\) theDuffing oscillator, \(ii\) theRepressilator, and \(iii\)IRMA\(introduced above\)\. Further details about these systems are provided in Appendix[A\.1](https://arxiv.org/html/2606.23957#A1.SS1)\. To assess generality without excessive tuning, we additionally report results on a nonlinear pendulum, Goodwin oscillator, Lotka–Volterra, Rössler, and a reduced\-order fluid\-flow model\(Goodwin,[1965](https://arxiv.org/html/2606.23957#bib.bib49); Fathiet al\.,[2023](https://arxiv.org/html/2606.23957#bib.bib61); Rössler,[1976](https://arxiv.org/html/2606.23957#bib.bib62); Noacket al\.,[2003](https://arxiv.org/html/2606.23957#bib.bib50)\)\. These are*sanity checks*performed with the finalized architecture to test out\-of\-the\-box behavior; unlike the core trio, we did not perform extensive ablations or per\-system tuning\. Full details and additional figures are provided in the Appendix[G\.4](https://arxiv.org/html/2606.23957#A7.SS4)\. Parameterizations, time steps, numbers of trajectories, and training/prediction horizons follow Table[9](https://arxiv.org/html/2606.23957#A8.T9)\(data splits and any deviations are detailed in the Appendix\)\. Full ODEs, solvers, parameter values, and initial\-condition ranges for all systems are provided in Appendix[H](https://arxiv.org/html/2606.23957#A8)\.

### 3\.2Protocols and metrics

Rollout protocol\.Unless stated otherwise, models are trained on fixed\-length segments and evaluated by free \(open\-loop\) rollouts from test\-set initial conditions\. We report errors at horizons\{200,500,1000\}\\\{200,500,1000\\\}steps on the three primary systems, and 200\-step errors on the additional benchmarks\.

Metrics\.We report mean\-squared error \(MSE\) at a fixed horizon and a long\-horizon*mean cumulative absolute error*\(MCAE\) that captures accumulation of deviations\. Given a rollout of lengthHH, MCAE averages, across trajectories and state dimensions, the cumulative absolute error curve:

MCAEt\\displaystyle\\text\{MCAE\}\_\{t\}=1d​∑j=1d∑k=1t\|x^k,j−xk,j\|\\displaystyle=\\tfrac\{1\}\{d\}\\sum\_\{j=1\}^\{d\}\\sum\_\{k=1\}^\{t\}\\\!\\left\|\\hat\{x\}\_\{k,j\}\-x\_\{k,j\}\\right\|\(7\)MCAEoverall\\displaystyle\\text\{MCAE\}\_\{\\text\{overall\}\}=1H​∑t=1HMCAEt\\displaystyle=\\tfrac\{1\}\{H\}\\sum\_\{t=1\}^\{H\}\\text\{MCAE\}\_\{t\}\(8\)We plot MCAE over steps to reveal error growth dynamics\.

Hyperparameters & selection\.We fix the autoencoder bottleneck dimension and AFT context \(d=100d\{=\}100,T=10T\{=\}10\) across systems unless otherwise noted, and select early stopping and trigger thresholds on the validation set\. The Koopman operator is dense by default\.

### 3\.3Baselines and ablations

We compare:

1. 1\.GRU: the baseline GRU autoencoder \(§[E\.1](https://arxiv.org/html/2606.23957#A5.SS1)\)\.
2. 2\.Transformer: the baseline transformer autoencoder \(§[E\.2](https://arxiv.org/html/2606.23957#A5.SS2)\)
3. 3\.KAE: the baseline Koopman autoencoder \(§[2\.1](https://arxiv.org/html/2606.23957#S2.SS1)\)\.
4. 4\.KAE \+ AFT: our latent\-memory augmentation \(§[2\.2](https://arxiv.org/html/2606.23957#S2.SS2)\)\.
5. 5\.KAE \+ MHA: matched\-capacity multi\-head attention with 4 or 10 heads \(same bottleneckdd, similar projection sizes\)\.
6. 6\.KAE \+ AFT \+ Re\-enc: dynamic re\-encoding with streaming triggers \(§[2\.3](https://arxiv.org/html/2606.23957#S2.SS3)\)\. We evaluate the sequential two\-sample tests as the Dynamic Re\-encoding Method using per\-system validation\-tuned thresholds, alongside periodic re\-encoding fromFathiet al\.\([2023](https://arxiv.org/html/2606.23957#bib.bib61)\)\.

Ablations vary \(i\) the Koopman operator size, \(ii\) the AFT contextTT, and \(iii\) the trigger family/thresholds\. For fairness, all baselines share the same autoencoder structure and training schedule\.

## 4Results

### 4\.1Primary comparison on three representative systems

We conducted a comprehensive testing of our three primary systems for long\-term horizon prediction\. Our evaluation encompasses the reference models mentioned in §[3\.3](https://arxiv.org/html/2606.23957#S3.SS3)\. Additionally, we assessed GRU and Transformer architectures under two experimental conditions\. Given that GRU and Transformer models require contextual information, we evaluated them first using only initial conditions, and then subsequently with a context of 50 time steps, which means in the Repressilator and IRMA escaping a large part of the transient state\. Results are shown in Table[1](https://arxiv.org/html/2606.23957#S4.T1)\.

Duffing Oscillator\.Dynamic re\-encoding is best at 200/500 steps \(MSE0\.0113/0\.09600\.0113/0\.0960\), improving on AFT \(0\.0427/0\.15360\.0427/0\.1536\) and periodic re\-encoding \(0\.0156/0\.11870\.0156/0\.1187\)\. At 1000 steps the single\-run ordering is close \(AFT0\.19470\.1947vs\. dynamic0\.20190\.2019\); across seeds, dynamic re\-encoding remains lowest \(0\.190±0\.0100\.190\{\\pm\}0\.010vs\. AFT0\.275±0\.0750\.275\{\\pm\}0\.075; Table[6](https://arxiv.org/html/2606.23957#A7.T6)\)\. The vanilla KAE drifts \(0\.1286/0\.2245/0\.24710\.1286/0\.2245/0\.2471\), and GRU/Transformer benefit from context yet remain far off Koopman variants \(e\.g\., GRU0\.08620\.0862vs\. AFT0\.04270\.0427at 200 steps\)\.*Timely snap\-backs help at switching transitions; for very long horizons, a small causal memory \(AFT\) often suffices\.*

Repressilator\.All Koopman variants handle the limit cycle, but AFT is the best Koopman variant across horizons \(0\.0001/0\.0002/0\.00050\.0001/0\.0002/0\.0005; seed means0\.0001/0\.0007/0\.00190\.0001/0\.0007/0\.0019, Table[6](https://arxiv.org/html/2606.23957#A7.T6)\)\. Dynamic/periodic re\-encoding degrade to∼\!4\!×\!10−3\\sim\!4\!\\times\!10^\{\-3\}by injecting unnecessary phase resets\. KAE is competitive at 200 steps \(0\.00020\.0002\) but worsens by 1000 \(0\.00770\.0077\)\. GRU/Transformer improve with context \(e\.g\., GRU0\.00190\.0019at 200\) yet remain1010–100×100\\timesworse than AFT\.*On smooth limit cycles, prefer AFT\-only; snap\-backs are rarely needed and can be harmful\.*

IRMA\.Dynamic re\-encoding is strongest and most stable \(0\.0001/0\.0001/0\.00030\.0001/0\.0001/0\.0003\), with periodic close behind \(0\.0002/0\.0004/0\.00080\.0002/0\.0004/0\.0008\)\. AFT is very good at short horizons \(0\.00040\.0004\) but continues to degrade by the same rate \(0\.0009/0\.00120\.0009/0\.0012\)\. The single\-run KAE diverges at 1000 \(10\.1810\.18\): without latent memory or re\-encoding the rollout can fail to settle and leaves the valid range \(visible as the diverging KAE trajectory in Fig\.[2\(b\)](https://arxiv.org/html/2606.23957#S4.F2.sf2)\)\. This divergence is intermittent across seeds \(seed mean0\.03±0\.010\.03\{\\pm\}0\.01, Table[6](https://arxiv.org/html/2606.23957#A7.T6)\), and is precisely the instability that AFT and re\-encoding remove\. GRU with context is competitive \(0\.0001/0\.0003/0\.00040\.0001/0\.0003/0\.0004\) but from initial conditions is much worse \(e\.g\.,0\.01020\.0102at 200\)\.*GRU\(\+Ctx\) benefits from being placed near the attractor; Koopman\+AFT with snap\-backs attains similar robustness without long input contexts\.*

Table 1:Prediction performance comparison \(MSE↓\\downarrow\) over different time steps across different system configurations\. Best results for each system are highlighted inbold\. Context provided for the GRU and Transformer is 50 time steps, while other results are from initial conditions, indicated as \+Ctx and Init respectively\. Per\-seed means and standard deviations for the Koopman variants are reported in Table[6](https://arxiv.org/html/2606.23957#A7.T6)\(Appendix[G\.3](https://arxiv.org/html/2606.23957#A7.SS3)\)\. Values here are single\-run; per\-seed statistics are more representative at long horizons, where single runs can include outliers \(e\.g\., IRMA KAE at 1000 steps: single\-run10\.1810\.18vs\. seed mean0\.03±0\.010\.03\{\\pm\}0\.01\) and the ranking can shift \(Duffing at 1000\)\.![Refer to caption](https://arxiv.org/html/2606.23957v1/Figures/Repressilator_multi_timesteps_prediction.png)\(a\)Repressilator — multi\-traj
![Refer to caption](https://arxiv.org/html/2606.23957v1/Figures/IRMA_multi_timesteps_prediction_0.png)\(b\)IRMA — multi\-traj
![Refer to caption](https://arxiv.org/html/2606.23957v1/Figures/duffing_multi_prediction_comparison.png)\(c\)Duffing — switching dynamics

Figure 2:Multi\-trajectory rollouts on dynamical systems\.AFT reduces phase drift across initial conditions and enables accurate detection of switching dynamics in bistable systems\.
### 4\.2Attention vs AFT comparison

We compare the latent\-memory augmentation \(AFT\) to matched\-capacity multi\-head attention \(MHA; 4 and 10 heads\) on Duffing, Repressilator, and IRMA using both MSE and long\-horizon MCAE\. Figure[3](https://arxiv.org/html/2606.23957#S4.F3)shows representative trajectories \(top\) and MCAE curves \(bottom\); summary metrics appear in Table[2](https://arxiv.org/html/2606.23957#S4.T2)\.

Duffing Oscillator\.AFT achieves the lowest error by a wide margin \(MSE0\.01240\.0124vs\.0\.09570\.0957/0\.11370\.1137for 10/4\-head MHA;∼8\\sim\\\!8–9×9\\timeslower\), and flattens error growth \(MCAE10\.9510\.95vs\.49\.0949\.09/52\.9852\.98;∼4\.5\\sim\\\!4\.5–4\.8×4\.8\\timeslower\)\. This matches the intuition that short, causal context suppresses phase/amplitude drift induced by the mixed/continuous spectrum and switching dynamics better than quadratic\-cost attention\.

Repressilator\.On the clean limit cycle, AFT again dominates \(MSE3×10−43\\\!\\times\\\!10^\{\-4\}vs\.1\.6×10−31\.6\\\!\\times\\\!10^\{\-3\}/1\.8×10−31\.8\\\!\\times\\\!10^\{\-3\};∼5\\sim\\\!5–6×6\\timeslower\)\. MCAE is likewise reduced \(1\.80 vs\. 5\.26 / 5\.66;∼3×\\sim\\\!3\\times\)\. The small, causal window corrects local misalignments before they accumulate into phase slips, yielding smoother, phase\-consistent rollouts than MHA\.

IRMA\.AFT yields the best single\-model accuracy \(MSE1×10−41\\\!\\times\\\!10^\{\-4\}vs\.1\.2×10−31\.2\\\!\\times\\\!10^\{\-3\}/1\.5×10−31\.5\\\!\\times\\\!10^\{\-3\};∼12\\sim\\\!12–15×15\\times\)\. MCAE also favors AFT \(0\.98 vs\. 4\.54 / 4\.90;∼4\.6\\sim\\\!4\.6–5\.0×5\.0\\times\), but the remaining long\-horizon drift motivates using*AFT \+ re\-encoding*\(Sec\.[4\.3](https://arxiv.org/html/2606.23957#S4.SS3)\) on this higher\-dimensional, feedback\-rich system\.

Overall, AFT consistently outperforms matched\-capacity MHA across systems and metrics while retaining*linear*cost in the context length \(cf\. Sec\.[G\.1](https://arxiv.org/html/2606.23957#A7.SS1)\), making it both more accurate and more scalable for long\-horizon prediction\.

Table 2:Performance comparison of attention mechanisms across different dynamical systems\. Lower MSE and MCAE values indicate better performance\. Best results are highlighted in bold\.95%95\\%confidence intervals over initial conditions for the per\-step MCAE are shown in Fig\.[3](https://arxiv.org/html/2606.23957#S4.F3)\.![Refer to caption](https://arxiv.org/html/2606.23957v1/Figures/IRMA_AFTvsMHA_Sample.png)\(a\)IRMA — sample trajectory
![Refer to caption](https://arxiv.org/html/2606.23957v1/Figures/Duffing_AFTvsMHA_Sample.png)\(b\)Duffing — sample trajectory
![Refer to caption](https://arxiv.org/html/2606.23957v1/Figures/Repressilator_AFTvsMHA_Sample.png)\(c\)Repressilator — sample trajectory
![Refer to caption](https://arxiv.org/html/2606.23957v1/Figures/IRMA_AFTvsMHA_MCAE.png)\(d\)IRMA — MCAE
![Refer to caption](https://arxiv.org/html/2606.23957v1/Figures/Duffing_AFTvsMHA_MCAE.png)\(e\)Duffing — MCAE
![Refer to caption](https://arxiv.org/html/2606.23957v1/Figures/Repressilator_AFTvsMHA_CMAE.png)\(f\)Repressilator — MCAE

Figure 3:AFT vs\. MHA \(4 and 10 heads\) on the three primary systems\.The top row shows sampled trajectories, and the bottom row shows the MCAE curves\. AFT reduces error growth and outperforms matched\-capacity MHA\.
### 4\.3Effect of dynamic re\-encoding \(streaming triggers\)

We study the dynamic re\-encoding with streaming triggers on the Duffing oscillator \(§[2\.3](https://arxiv.org/html/2606.23957#S2.SS3)\)\. Table[3](https://arxiv.org/html/2606.23957#S4.T3)shows that the sequential two\-sample detector attains the lowest error \(0\.0113\), followed by EWMA and CUSUM, and Fig\.[4](https://arxiv.org/html/2606.23957#S4.F4)shows the sensitivity and accuracy of the methods\. The two\-sample method relies on statistical distribution changes, which account for better detection, whereas EWMA/CUSUM uses aggregated statistics that can smooth over subtle but meaningful shifts\. Threshold\-based methods are prone to false positives/negatives, as they do not fully account for prediction memory, and drift proxies incorporate not only manifold distance but also reconstruction error\. While periodic re\-encoding can achieve good performance, it primarily targets drift frequency and does not consider when or where the change occurs, limiting its responsiveness\.

Table 3:Overall MSE per trajectory for different re\-encoding methods\.![Refer to caption](https://arxiv.org/html/2606.23957v1/x2.png)\(a\)Cumulative MAE per timestep
![Refer to caption](https://arxiv.org/html/2606.23957v1/x3.png)\(b\)State\-space comparison

Figure 4:Comparison of re\-encoding methods\. \(a\) Cumulative MAE per timestep comparison across methods\. \(b\) Comparison in latent state space for a sampled trajectory\.

## 5Discussion

What the latent memory buys\.Across Duffing, Repressilator, and IRMA, the attention\-free latent memory \(§[2\.2](https://arxiv.org/html/2606.23957#S2.SS2)\) consistently reduces phase slippage and amplitude drift over long horizons relative to the plain KAE and to a matched\-capacity multi\-head attention \(MHA\) baseline, lowering both MCAE and MSE \(see §[4\.2](https://arxiv.org/html/2606.23957#S4.SS2), Table[2](https://arxiv.org/html/2606.23957#S4.T2), Fig\.[3](https://arxiv.org/html/2606.23957#S4.F3); and §[4\.1](https://arxiv.org/html/2606.23957#S4.SS1), Table[1](https://arxiv.org/html/2606.23957#S4.T1)\)\. Empirically, a short, causal context \(T=10T\{=\}10; §[G\.5](https://arxiv.org/html/2606.23957#A7.SS5)\) is sufficient to capture the local temporal correlations that most affect error accumulation, and doing so in𝒪​\(T​d\)\\mathcal\{O\}\(Td\)time/memory per step yields stable rollouts without the quadratic attention overhead \(complexity details in App\.[G\.1](https://arxiv.org/html/2606.23957#A7.SS1); context ablation in Fig\.[9](https://arxiv.org/html/2606.23957#A7.F9), right\)\. This is a pragmatic complement to spectral\-accuracy pursuits in Koopman learning\(Korda and Mezić,[2018](https://arxiv.org/html/2606.23957#bib.bib15); Mezić,[2022](https://arxiv.org/html/2606.23957#bib.bib20); Giannakis and Valva,[2024](https://arxiv.org/html/2606.23957#bib.bib21); Colbrook and Townsend,[2024](https://arxiv.org/html/2606.23957#bib.bib14)\): even when the learnedKKis an imperfect global surrogate, local correction can improve long\-horizon behavior\. Conceptually, the AFT correction acts like a short learned delay\-embedding/HAVOK\-style forcing term\(Arbabi and Mezic,[2017](https://arxiv.org/html/2606.23957#bib.bib67); Bruntonet al\.,[2017](https://arxiv.org/html/2606.23957#bib.bib68)\), while re\-encoding is a latent\-space analogue of windowed/recursive DMD projections\(Noacket al\.,[2015](https://arxiv.org/html/2606.23957#bib.bib72); Guanet al\.,[2024](https://arxiv.org/html/2606.23957#bib.bib71)\)\.

When and why re\-encoding helps\.The dynamic re\-encoding mechanism \(§[2\.3](https://arxiv.org/html/2606.23957#S2.SS3)\) improves robustness primarily on systems with switching or stiff transients \(e\.g\., Duffing at higher energies\) or higher\-dimensional, intertwined feedback \(IRMA\), where drifting off the autoencoder manifold can be abrupt and compounding\. Quantitatively, triggers based on sequential two\-sample tests attain the lowest MSE on Duffing, followed by EWMA and CUSUM \(Table[3](https://arxiv.org/html/2606.23957#S4.T3); Fig\.[4](https://arxiv.org/html/2606.23957#S4.F4); see also §[4\.3](https://arxiv.org/html/2606.23957#S4.SS3)and Alg\.[2](https://arxiv.org/html/2606.23957#algorithm2)in App\.[F](https://arxiv.org/html/2606.23957#A6)\)\. On clean, phase\-sensitive oscillators \(e\.g\., Repressilator\), re\-encoding can occasionally*hurt*when a trigger fires near a delicate phase region: the E–D–E projection introduces a small phase shift that the Koopman update then propagates \(Table[1](https://arxiv.org/html/2606.23957#S4.T1)\)\. Practical guidance: AFT\-only for smooth limit cycles; AFT\+EWMA/CUSUM for intermittent regime changes; and two\-sample tests when residual distributions clearly separate nominal vs\. drifted behavior\.

Sensitivity and hyperparameters\.Performance is most sensitive to \(i\) the quality of the autoencoder manifold, \(ii\) the AFT contextTT, and \(iii\) trigger thresholds\. Too large aTTbrings diminishing returns and mild over\-smoothing \(Fig\.[9](https://arxiv.org/html/2606.23957#A7.F9), right\)\. Thresholds selected on validation data transfer well across test horizons in our runs, but overly aggressive settings can over\-trigger and degrade smooth oscillations\. A denseKKoffered the strongest accuracy \(consistent with prior observations\), whereas structured variants \(diagonal, banded, Jordan\) trade accuracy for interpretability; we include these ablations for completeness \(Fig\.[9](https://arxiv.org/html/2606.23957#A7.F9), left\)\.

Applications and impact\.Where long\-horizon forecasting is needed under tight computational budgets \(embedded monitoring, rapid what\-if simulation\), the𝒪​\(T​d\)\\mathcal\{O\}\(Td\)latent memory and occasional E–D–E snaps provide a practical path that keeps the standard KAE backbone intact and reproducible\. Compared to GRU and Transformer autoencoders \(architectures in App\.[E](https://arxiv.org/html/2606.23957#A5)\), Koopman\-based predictors deliver both stronger long\-horizon fidelity \(Table[1](https://arxiv.org/html/2606.23957#S4.T1)\) and substantially lower latency \(Table[5](https://arxiv.org/html/2606.23957#A7.T5)\)\. Breadth checks across additional dynamical systems indicate out\-of\-the\-box gains where appropriate \(App\.[G\.4](https://arxiv.org/html/2606.23957#A7.SS4); Table[7](https://arxiv.org/html/2606.23957#A7.T7), Fig\.[8](https://arxiv.org/html/2606.23957#A7.F8)\)\.

## 6Limitations and Future Work

Our stability claims are empirical: we do not provide convergence or spectral\-error guarantees for the learnedKKdespite relevant theory\(Korda and Mezić,[2018](https://arxiv.org/html/2606.23957#bib.bib15); Mezić,[2022](https://arxiv.org/html/2606.23957#bib.bib20); Giannakis and Valva,[2024](https://arxiv.org/html/2606.23957#bib.bib21); Colbrook and Townsend,[2024](https://arxiv.org/html/2606.23957#bib.bib14)\)\. Effectiveness depends on the autoencoder manifold; ifφ−1\\varphi^\{\-1\}is lossy, the E–D–E projection can bias latents\. Trigger policies introduce hyperparameters \(thresholds, windows\) and can degrade performance on clean limit cycles \(Table[1](https://arxiv.org/html/2606.23957#S4.T1), Repressilator\) even while helping on systems with switching or stiff transients \(Duffing, IRMA; Tables[1](https://arxiv.org/html/2606.23957#S4.T1),[3](https://arxiv.org/html/2606.23957#S4.T3), Fig\.[4](https://arxiv.org/html/2606.23957#S4.F4)\)\. Dynamic re\-encoding is used only at inference, so the model is not co\-trained with snaps\. Some configurations still degrade at very long horizons \(e\.g\., AFT on IRMA at 1000 steps in Table[1](https://arxiv.org/html/2606.23957#S4.T1)\), and vanilla KAE can collapse\. Our experiments focus on autonomous systems; inputs/control are out of scope here\. Additional benchmarks suggest “out\-of\-the\-box” generality, but we did not target per\-system SOTA\.

Future directions include bridging empirical robustness with guarantees \(resolvent/residual\-minimization objectives and stability\-biased constraints forKK; EDMD diagnostics during training\(Giannakis and Valva,[2024](https://arxiv.org/html/2606.23957#bib.bib21); Colbrook and Townsend,[2024](https://arxiv.org/html/2606.23957#bib.bib14); Mezić,[2022](https://arxiv.org/html/2606.23957#bib.bib20); Korda and Mezić,[2018](https://arxiv.org/html/2606.23957#bib.bib15)\)\), training curricula that transition from one\-step to free rollouts, uncertainty\-aware or learned triggers that retain the two\-sample sensitivity benefits on Duffing \(Table[3](https://arxiv.org/html/2606.23957#S4.T3)\) while avoiding false snaps on smooth cycles, and adaptive memory that learns/gates the AFT contextTT\(cf\. Fig\.[9](https://arxiv.org/html/2606.23957#A7.F9)\)\. Extending AFT and re\-encoding to controlled settings \(DMDc/EDMDc/KIC\) and evaluating in receding\-horizon MPC is natural, as is studying partial/noisy/hybrid systems\. Finally, exploring structuredKKfor interpretability with minimal loss, and fusing AFT with decoders for hardware\-efficient deployment, are promising for resource\-constrained use \(Table[5](https://arxiv.org/html/2606.23957#A7.T5)\)\.

## 7Reproducibility Statement

We have made every effort to ensure the reproducibility of our results\. The paper provides detailed descriptions of the model architecture, training setup, and evaluation protocols\. Hyperparameters, dataset generation, and experimental settings are included in the Appendix\. We have also provided a detailed reproducibility checklist in the Appendix[C](https://arxiv.org/html/2606.23957#A3), which outlines the entire experimental process step by step\. To further support replication, we have made the code available at[https://github\.com/MohammedNagdi/Attended\-Koopman](https://github.com/MohammedNagdi/Attended-Koopman)\.

## References

- Deep learning enhanced dynamic mode decomposition\.Chaos: An Interdisciplinary Journal of Nonlinear Science32\(3\)\.Cited by:[§A\.1\.1](https://arxiv.org/html/2606.23957#A1.SS1.SSS1.p1.2),[§B\.1](https://arxiv.org/html/2606.23957#A2.SS1.p1.1),[§1](https://arxiv.org/html/2606.23957#S1.SS0.SSS0.Px1.p2.1)\.
- H\. Arbabi and I\. Mezic \(2017\)Ergodic theory, dynamic mode decomposition, and computation of spectral properties of the koopman operator\.SIAM Journal on Applied Dynamical Systems16\(4\),pp\. 2096–2126\.Cited by:[§B\.2](https://arxiv.org/html/2606.23957#A2.SS2.p1.1),[§1](https://arxiv.org/html/2606.23957#S1.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2606.23957#S1.p1.3),[§5](https://arxiv.org/html/2606.23957#S5.p1.3)\.
- S\. Balakrishnan, A\. Hasnain, R\. Egbert, and E\. Yeung \(2022\)Data\-driven observability decomposition with koopman operators for optimization of output functions of nonlinear systems\.arXiv preprint arXiv:2210\.09343\.Cited by:[§A\.1\.2](https://arxiv.org/html/2606.23957#A1.SS1.SSS2.p1.17),[§1](https://arxiv.org/html/2606.23957#S1.SS0.SSS0.Px1.p2.1)\.
- N\. Boddupalli, A\. Hasnain, S\. P\. Nandanoori, and E\. Yeung \(2019\)Koopman operators for generalized persistence of excitation conditions for nonlinear systems\.In2019 IEEE 58th Conference on Decision and Control \(CDC\),pp\. 8106–8111\.Cited by:[§A\.1\.2](https://arxiv.org/html/2606.23957#A1.SS1.SSS2.p1.17),[§1](https://arxiv.org/html/2606.23957#S1.SS0.SSS0.Px1.p2.1)\.
- S\. L\. Brunton, B\. W\. Brunton, J\. L\. Proctor, E\. Kaiser, and J\. N\. Kutz \(2017\)Chaos as an intermittently forced linear system\.Nature communications8\(1\),pp\. 19\.Cited by:[§B\.2](https://arxiv.org/html/2606.23957#A2.SS2.p1.1),[§1](https://arxiv.org/html/2606.23957#S1.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2606.23957#S1.p1.3),[§5](https://arxiv.org/html/2606.23957#S5.p1.3)\.
- I\. Cantone, L\. Marucci, F\. Iorio, M\. A\. Ricci, V\. Belcastro, M\. Bansal, S\. Santini, M\. Di Bernardo, D\. Di Bernardo, and M\. P\. Cosma \(2009\)A yeast synthetic network for in vivo assessment of reverse\-engineering and modeling approaches\.Cell137\(1\),pp\. 172–181\.Cited by:[§A\.1\.3](https://arxiv.org/html/2606.23957#A1.SS1.SSS3.p1.1),[§B\.7](https://arxiv.org/html/2606.23957#A2.SS7.p1.1),[§1](https://arxiv.org/html/2606.23957#S1.SS0.SSS0.Px1.p2.1)\.
- K\. Cho, B\. Van Merriënboer, C\. Gulcehre, D\. Bahdanau, F\. Bougares, H\. Schwenk, and Y\. Bengio \(2014\)Learning phrase representations using rnn encoder\-decoder for statistical machine translation\.arXiv preprint arXiv:1406\.1078\.Cited by:[§B\.1](https://arxiv.org/html/2606.23957#A2.SS1.p1.1)\.
- M\. J\. Colbrook and A\. Townsend \(2024\)Rigorous data‐driven computation of spectral properties of koopman operators for dynamical systems\.Communications on Pure and Applied Mathematics77,pp\. 221–283\.External Links:[Document](https://dx.doi.org/10.1002/cpa.22125),[Link](https://doi.org/10.1002/cpa.22125),ISSN 0010\-3640Cited by:[§B\.1](https://arxiv.org/html/2606.23957#A2.SS1.p1.1),[§5](https://arxiv.org/html/2606.23957#S5.p1.3),[§6](https://arxiv.org/html/2606.23957#S6.p1.2),[§6](https://arxiv.org/html/2606.23957#S6.p2.3)\.
- D\. di Bernardo, L\. Marucci, F\. Menolascina, and V\. Siciliano \(2011\)Predicting synthetic gene networks\.InSynthetic Gene Networks: Methods and Protocols,pp\. 57–81\.Cited by:[§A\.1\.3](https://arxiv.org/html/2606.23957#A1.SS1.SSS3.p1.1),[§B\.7](https://arxiv.org/html/2606.23957#A2.SS7.p1.1),[§1](https://arxiv.org/html/2606.23957#S1.SS0.SSS0.Px1.p2.1)\.
- D\. Dylewsky, M\. Tao, and J\. N\. Kutz \(2019\)Dynamic mode decomposition for multiscale nonlinear physics\.Physical Review E99\(6\),pp\. 063311\.Cited by:[§B\.5](https://arxiv.org/html/2606.23957#A2.SS5.p1.1),[§1](https://arxiv.org/html/2606.23957#S1.SS0.SSS0.Px1.p1.1)\.
- J\. L\. Elman \(1990\)Finding structure in time\.Cognitive science14\(2\),pp\. 179–211\.Cited by:[§B\.1](https://arxiv.org/html/2606.23957#A2.SS1.p1.1)\.
- M\. B\. Elowitz and S\. Leibler \(2000\)A synthetic oscillatory network of transcriptional regulators\.Nature403\(6767\),pp\. 335–338\.Cited by:[§A\.1\.2](https://arxiv.org/html/2606.23957#A1.SS1.SSS2.p1.17),[§B\.7](https://arxiv.org/html/2606.23957#A2.SS7.p1.1),[§1](https://arxiv.org/html/2606.23957#S1.SS0.SSS0.Px1.p2.1)\.
- D\. Enyeart and G\. Lin \(2024\)Loss terms and operator forms of koopman autoencoders\.arXiv preprint arXiv:2412\.04578\.Cited by:[§B\.1](https://arxiv.org/html/2606.23957#A2.SS1.p1.1),[§2\.1](https://arxiv.org/html/2606.23957#S2.SS1.SSS0.Px2.p1.2),[§2\.1](https://arxiv.org/html/2606.23957#S2.SS1.SSS0.Px2.p2.8)\.
- M\. Fathi, C\. Gehring, J\. Pilault, D\. Kanaa, P\. Bacon, and R\. Goroshin \(2023\)Course correcting koopman representations\.arXiv preprint arXiv:2310\.15386\.Cited by:[§A\.2](https://arxiv.org/html/2606.23957#A1.SS2.SSS0.Px4.p1.4),[§B\.7](https://arxiv.org/html/2606.23957#A2.SS7.p1.1),[item 6](https://arxiv.org/html/2606.23957#S3.I1.i6.p1.1),[§3\.1](https://arxiv.org/html/2606.23957#S3.SS1.p1.1)\.
- A\. Frion, L\. Drumetz, M\. D\. Mura, G\. Tochon, and A\. Aïssa\-El\-Bey \(2025\)Augmented invertible koopman autoencoder for long\-term time series forecasting\.arXiv preprint arXiv:2503\.12930\.Cited by:[§B\.5](https://arxiv.org/html/2606.23957#A2.SS5.p1.1),[§1](https://arxiv.org/html/2606.23957#S1.SS0.SSS0.Px1.p1.1)\.
- D\. Giannakis and C\. Valva \(2024\)Consistent spectral approximation of koopman operators using resolvent compactification\.Nonlinearity37,pp\. 075021\.External Links:[Document](https://dx.doi.org/10.1088/1361-6544/ad4ade),[Link](https://doi.org/10.1088/1361-6544/ad4ade),ISSN 0951\-7715Cited by:[§B\.1](https://arxiv.org/html/2606.23957#A2.SS1.p1.1),[§5](https://arxiv.org/html/2606.23957#S5.p1.3),[§6](https://arxiv.org/html/2606.23957#S6.p1.2),[§6](https://arxiv.org/html/2606.23957#S6.p2.3)\.
- B\. C\. Goodwin \(1965\)Oscillatory behavior in enzymatic control processes\.Advances in enzyme regulation3,pp\. 425–437\.Cited by:[§A\.2](https://arxiv.org/html/2606.23957#A1.SS2.SSS0.Px3.p1.12),[§B\.7](https://arxiv.org/html/2606.23957#A2.SS7.p1.1),[§3\.1](https://arxiv.org/html/2606.23957#S3.SS1.p1.1)\.
- W\. Guan, L\. Dong, A\. Zhang, and Y\. Cai \(2024\)Output\-only modal identification with recursive dynamic mode decomposition for time\-varying systems\.Measurement224,pp\. 113852\.Cited by:[§B\.5](https://arxiv.org/html/2606.23957#A2.SS5.p1.1),[§1](https://arxiv.org/html/2606.23957#S1.SS0.SSS0.Px1.p1.1),[§5](https://arxiv.org/html/2606.23957#S5.p1.3)\.
- A\. Hasnain, S\. Sinha, Y\. Dorfan, A\. E\. Borujeni, Y\. Park, P\. Maschhoff, U\. Saxena, J\. Urrutia, N\. Gaffney, D\. Becker,et al\.\(2019\)A data\-driven method for quantifying the impact of a genetic circuit on its host\.In2019 IEEE Biomedical Circuits and Systems Conference \(BioCAS\),pp\. 1–4\.Cited by:[§B\.7](https://arxiv.org/html/2606.23957#A2.SS7.p1.1),[§1](https://arxiv.org/html/2606.23957#S1.SS0.SSS0.Px1.p2.1)\.
- S\. Hochreiter and J\. Schmidhuber \(1997\)Long short\-term memory\.Neural computation9\(8\),pp\. 1735–1780\.Cited by:[§B\.1](https://arxiv.org/html/2606.23957#A2.SS1.p1.1)\.
- E\. Kaiser, J\. N\. Kutz, and S\. L\. Brunton \(2020\)Data\-driven approximations of dynamical systems operators for control\.Lecture Notes in Control and Information Sciences,pp\. 197–234\.External Links:[Document](https://dx.doi.org/10.1007/978-3-030-35713-9%5F8),[Link](https://doi.org/10.1007/978-3-030-35713-9%5C_8),ISSN 0170\-8643Cited by:[§B\.4](https://arxiv.org/html/2606.23957#A2.SS4.p1.1)\.
- M\. Kamb, E\. Kaiser, S\. L\. Brunton, and J\. N\. Kutz \(2020\)Time\-delay observables for koopman: theory and applications\.SIAM Journal on Applied Dynamical Systems19,pp\. 886–917\.External Links:[Document](https://dx.doi.org/10.1137/18m1216572),[Link](https://doi.org/10.1137/18m1216572),ISSN 1536\-0040Cited by:[§B\.2](https://arxiv.org/html/2606.23957#A2.SS2.p1.1)\.
- F\. Köhne, F\. M\. Philipp, M\. Schaller, A\. Schiela, and K\. Worthmann \(2025\)\-Error bounds for approximations of the koopman operator by kernel extended dynamic mode decomposition\.SIAM journal on applied dynamical systems24\(1\),pp\. 501–529\.Cited by:[§A\.1\.1](https://arxiv.org/html/2606.23957#A1.SS1.SSS1.p1.2),[§B\.1](https://arxiv.org/html/2606.23957#A2.SS1.p1.1),[§1](https://arxiv.org/html/2606.23957#S1.SS0.SSS0.Px1.p2.1)\.
- B\. O\. Koopman \(1931\)Hamiltonian systems and transformation in hilbert space\.Proceedings of the National Academy of Sciences17\(5\),pp\. 315–318\.External Links:[Document](https://dx.doi.org/10.1073/pnas.17.5.315),[Link](https://doi.org/10.1073/pnas.17.5.315)Cited by:[§B\.1](https://arxiv.org/html/2606.23957#A2.SS1.p1.1),[§1](https://arxiv.org/html/2606.23957#S1.p1.3)\.
- M\. Korda and I\. Mezić \(2018\)On convergence of extended dynamic mode decomposition to the koopman operator\.Journal of Nonlinear Science28,pp\. 687–710\.External Links:[Document](https://dx.doi.org/10.1007/s00332-017-9423-0),[Link](https://doi.org/10.1007/s00332-017-9423-0),ISSN 0938\-8974Cited by:[§B\.1](https://arxiv.org/html/2606.23957#A2.SS1.p1.1),[§5](https://arxiv.org/html/2606.23957#S5.p1.3),[§6](https://arxiv.org/html/2606.23957#S6.p1.2),[§6](https://arxiv.org/html/2606.23957#S6.p2.3)\.
- M\. Korda and I\. Mezić \(2020\)Koopman model predictive control of nonlinear dynamical systems\.Unknown journal,pp\. 235–255\.External Links:[Document](https://dx.doi.org/10.1007/978-3-030-35713-9%5F9),[Link](https://doi.org/10.1007/978-3-030-35713-9%5C_9)Cited by:[§B\.4](https://arxiv.org/html/2606.23957#A2.SS4.p1.1)\.
- Q\. Li, F\. Dietrich, E\. M\. Bollt, and I\. G\. Kevrekidis \(2017\)Extended dynamic mode decomposition with dictionary learning: a data\-driven adaptive spectral decomposition of the koopman operator\.Chaos: An Interdisciplinary Journal of Nonlinear Science27\(10\)\.Cited by:[§A\.1\.1](https://arxiv.org/html/2606.23957#A1.SS1.SSS1.p1.2),[§B\.1](https://arxiv.org/html/2606.23957#A2.SS1.p1.1),[§1](https://arxiv.org/html/2606.23957#S1.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2606.23957#S1.SS0.SSS0.Px1.p2.1)\.
- F\. Lu, K\. Zlobina, S\. Osorio, H\. Yang, A\. Nava, M\. D\. Bagood, M\. Rolandi, R\. R\. Isseroff, and M\. Gomez \(2024\)DeepMapper: attention\-based autoencoder for system identification in wound healing and stage prediction\.bioRxiv,pp\. 2024–12\.Cited by:[§B\.3](https://arxiv.org/html/2606.23957#A2.SS3.p1.1),[§1](https://arxiv.org/html/2606.23957#S1.SS0.SSS0.Px1.p1.1)\.
- B\. Lusch, J\. N\. Kutz, and S\. L\. Brunton \(2018\)Deep learning for universal linear embeddings of nonlinear dynamics\.Nature communications9\(1\),pp\. 4950\.Cited by:[§A\.2](https://arxiv.org/html/2606.23957#A1.SS2.SSS0.Px2.p1.6),[§B\.1](https://arxiv.org/html/2606.23957#A2.SS1.p1.1),[§B\.7](https://arxiv.org/html/2606.23957#A2.SS7.p1.1),[§1](https://arxiv.org/html/2606.23957#S1.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2606.23957#S1.p1.3),[§2\.1](https://arxiv.org/html/2606.23957#S2.SS1.SSS0.Px1.p1.11),[§2\.1](https://arxiv.org/html/2606.23957#S2.SS1.SSS0.Px2.p2.8)\.
- L\. Marucci, D\. A\. Barton, I\. Cantone, M\. A\. Ricci, M\. P\. Cosma, S\. Santini, D\. di Bernardo, and M\. di Bernardo \(2009\)How to turn a genetic circuit into a synthetic tunable oscillator, or a bistable switch\.PloS one4\(12\),pp\. e8083\.Cited by:[§A\.1\.3](https://arxiv.org/html/2606.23957#A1.SS1.SSS3.p1.1),[§A\.1\.3](https://arxiv.org/html/2606.23957#A1.SS1.SSS3.p4.2),[§B\.7](https://arxiv.org/html/2606.23957#A2.SS7.p1.1),[§1](https://arxiv.org/html/2606.23957#S1.SS0.SSS0.Px1.p2.1)\.
- F\. Menolascina, G\. Fiore, E\. Orabona, L\. De Stefano, M\. Ferry, J\. Hasty, M\. Di Bernardo, and D\. Di Bernardo \(2014\)In\-vivo real\-time control of protein expression from endogenous and synthetic gene networks\.PLoS computational biology10\(5\),pp\. e1003625\.Cited by:[§A\.1\.3](https://arxiv.org/html/2606.23957#A1.SS1.SSS3.p1.1),[§B\.7](https://arxiv.org/html/2606.23957#A2.SS7.p1.1),[§1](https://arxiv.org/html/2606.23957#S1.SS0.SSS0.Px1.p2.1)\.
- I\. Mezić \(2022\)On numerical approximations of the koopman operator\.Mathematics10,pp\. 1180\.External Links:[Document](https://dx.doi.org/10.3390/math10071180),[Link](https://doi.org/10.3390/math10071180),ISSN 2227\-7390Cited by:[§B\.1](https://arxiv.org/html/2606.23957#A2.SS1.p1.1),[§5](https://arxiv.org/html/2606.23957#S5.p1.3),[§6](https://arxiv.org/html/2606.23957#S6.p1.2),[§6](https://arxiv.org/html/2606.23957#S6.p2.3)\.
- G\. V\. Moustakides \(1986\)Optimal stopping times for detecting changes in distributions\.the Annals of Statistics14\(4\),pp\. 1379–1387\.Cited by:[§B\.6](https://arxiv.org/html/2606.23957#A2.SS6.p1.1),[item 1](https://arxiv.org/html/2606.23957#A4.I1.i1.p1.2),[2nd item](https://arxiv.org/html/2606.23957#S1.I1.i2.p1.1),[§1](https://arxiv.org/html/2606.23957#S1.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2606.23957#S1.p2.1),[item 3](https://arxiv.org/html/2606.23957#S2.I1.i3.p1.3)\.
- I\. Nayak, A\. Chakrabarti, M\. Kumar, F\. L\. Teixeira, and D\. Goswami \(2025\)Temporally\-consistent koopman autoencoders for forecasting dynamical systems\.Scientific Reports15\(1\),pp\. 22127\.Cited by:[§B\.5](https://arxiv.org/html/2606.23957#A2.SS5.p1.1),[§1](https://arxiv.org/html/2606.23957#S1.SS0.SSS0.Px1.p1.1)\.
- E\. Nikolados, A\. Y\. Weiße, F\. Ceroni, and D\. A\. Oyarzún \(2019\)Growth defects and loss\-of\-function in synthetic gene circuits\.ACS synthetic biology8\(6\),pp\. 1231–1240\.Cited by:[§A\.1\.2](https://arxiv.org/html/2606.23957#A1.SS1.SSS2.p1.17)\.
- B\. R\. Noack, K\. Afanasiev, M\. Morzyński, G\. Tadmor, and F\. Thiele \(2003\)A hierarchy of low\-dimensional models for the transient and post\-transient cylinder wake\.Journal of Fluid Mechanics497,pp\. 335–363\.Cited by:[§A\.2](https://arxiv.org/html/2606.23957#A1.SS2.SSS0.Px6.p1.1),[§B\.7](https://arxiv.org/html/2606.23957#A2.SS7.p1.1),[§3\.1](https://arxiv.org/html/2606.23957#S3.SS1.p1.1)\.
- B\. R\. Noack, W\. Stankiewicz, M\. Morzynski, and P\. J\. Schmid \(2015\)Recursive dynamic mode decomposition of a transient cylinder wake\.arXiv preprint arXiv:1511\.06876\.Cited by:[§B\.5](https://arxiv.org/html/2606.23957#A2.SS5.p1.1),[§1](https://arxiv.org/html/2606.23957#S1.SS0.SSS0.Px1.p1.1),[§5](https://arxiv.org/html/2606.23957#S5.p1.3)\.
- S\. E\. Otto and C\. W\. Rowley \(2019\)Linearly recurrent autoencoder networks for learning dynamics\.SIAM Journal on Applied Dynamical Systems18\(1\),pp\. 558–593\.Cited by:[§A\.1\.1](https://arxiv.org/html/2606.23957#A1.SS1.SSS1.p1.2),[§B\.1](https://arxiv.org/html/2606.23957#A2.SS1.p1.1),[§1](https://arxiv.org/html/2606.23957#S1.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2606.23957#S1.SS0.SSS0.Px1.p2.1),[§2\.1](https://arxiv.org/html/2606.23957#S2.SS1.SSS0.Px1.p1.11),[§2\.1](https://arxiv.org/html/2606.23957#S2.SS1.SSS0.Px2.p2.8)\.
- S\. Pan and K\. Duraisamy \(2020\)Physics\-informed probabilistic learning of linear embeddings of nonlinear dynamics with guaranteed stability\.SIAM Journal on Applied Dynamical Systems19\(1\),pp\. 480–509\.Cited by:[§A\.1\.1](https://arxiv.org/html/2606.23957#A1.SS1.SSS1.p1.2),[§B\.1](https://arxiv.org/html/2606.23957#A2.SS1.p1.1),[§1](https://arxiv.org/html/2606.23957#S1.SS0.SSS0.Px1.p2.1)\.
- R\. Perez\-Carrasco, C\. P\. Barnes, Y\. Schaerli, M\. Isalan, J\. Briscoe, and K\. M\. Page \(2018\)Combining a toggle switch and a repressilator within the ac\-dc circuit generates distinct dynamical behaviors\.Cell systems6\(4\),pp\. 521–530\.Cited by:[§A\.1\.2](https://arxiv.org/html/2606.23957#A1.SS1.SSS2.p1.17),[§1](https://arxiv.org/html/2606.23957#S1.SS0.SSS0.Px1.p2.1)\.
- J\. L\. Proctor, S\. L\. Brunton, and J\. N\. Kutz \(2018\)Generalizing koopman theory to allow for inputs and control\.SIAM Journal on Applied Dynamical Systems17,pp\. 909–930\.External Links:[Document](https://dx.doi.org/10.1137/16m1062296),[Link](https://doi.org/10.1137/16m1062296),ISSN 1536\-0040Cited by:[§B\.4](https://arxiv.org/html/2606.23957#A2.SS4.p1.1)\.
- S\. W\. Roberts \(2000\)Control chart tests based on geometric moving averages\.Technometrics42\(1\),pp\. 97–101\.Cited by:[§B\.6](https://arxiv.org/html/2606.23957#A2.SS6.p1.1),[item 4](https://arxiv.org/html/2606.23957#A4.I1.i4.p1.6),[2nd item](https://arxiv.org/html/2606.23957#S1.I1.i2.p1.1),[§1](https://arxiv.org/html/2606.23957#S1.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2606.23957#S1.p2.1),[item 2](https://arxiv.org/html/2606.23957#S2.I1.i2.p1.5)\.
- G\. J\. Ross and N\. M\. Adams \(2012\)Two nonparametric control charts for detecting arbitrary distribution changes\.Journal of Quality Technology44\(2\),pp\. 102–116\.Cited by:[§B\.6](https://arxiv.org/html/2606.23957#A2.SS6.p1.1),[item 5](https://arxiv.org/html/2606.23957#A4.I1.i5.p1.1),[2nd item](https://arxiv.org/html/2606.23957#S1.I1.i2.p1.1),[§1](https://arxiv.org/html/2606.23957#S1.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2606.23957#S1.p2.1),[item 4](https://arxiv.org/html/2606.23957#S2.I1.i4.p1.5)\.
- O\. E\. Rössler \(1976\)An equation for continuous chaos\.Physics Letters A57\(5\),pp\. 397–398\.Cited by:[§A\.2](https://arxiv.org/html/2606.23957#A1.SS2.SSS0.Px5.p1.4),[§B\.7](https://arxiv.org/html/2606.23957#A2.SS7.p1.1),[§3\.1](https://arxiv.org/html/2606.23957#S3.SS1.p1.1)\.
- A\. Salova, J\. Emenheiser, A\. Rupe, J\. P\. Crutchfield, and R\. M\. D’Souza \(2019\)Koopman operator and its approximations for systems with symmetries\.Chaos: An Interdisciplinary Journal of Nonlinear Science29,pp\. 093128\.External Links:[Document](https://dx.doi.org/10.1063/1.5099091),[Link](https://doi.org/10.1063/1.5099091),ISSN 1054\-1500Cited by:[§B\.1](https://arxiv.org/html/2606.23957#A2.SS1.p1.1)\.
- L\. Shi, M\. Haseli, G\. Mamakoukas, D\. Bruder, I\. Abraham, T\. Murphey, J\. Cortes, and K\. Karydis \(2024\)Koopman operators in robot learning\.ArXivabs/2408\.04200\.External Links:[Document](https://dx.doi.org/10.48550/arxiv.2408.04200),[Link](https://doi.org/10.48550/arxiv.2408.04200)Cited by:[§B\.4](https://arxiv.org/html/2606.23957#A2.SS4.p1.1)\.
- A\. Sootla, A\. Mauroy, and D\. Ernst \(2018\)Optimal control formulation of pulse\-based control using koopman operator\.Automatica91,pp\. 217–224\.Cited by:[§A\.1\.2](https://arxiv.org/html/2606.23957#A1.SS1.SSS2.p1.17),[§1](https://arxiv.org/html/2606.23957#S1.SS0.SSS0.Px1.p2.1)\.
- R\. Wang, Y\. Dong, S\. Ö\. Arik, and R\. Yu \(2022\)Koopman neural forecaster for time series with temporal distribution shifts\.arXiv preprint arXiv:2210\.03675\.Cited by:[§B\.3](https://arxiv.org/html/2606.23957#A2.SS3.p1.1),[§1](https://arxiv.org/html/2606.23957#S1.SS0.SSS0.Px1.p1.1)\.
- A\. Y\. Weiße, D\. A\. Oyarzún, V\. Danos, and P\. S\. Swain \(2015\)Mechanistic links between cellular trade\-offs, gene expression, and growth\.Proceedings of the National Academy of Sciences112\(9\),pp\. E1038–E1047\.Cited by:[§A\.1\.2](https://arxiv.org/html/2606.23957#A1.SS1.SSS2.p1.17)\.
- S\. Zhai, W\. Talbott, N\. Srivastava, C\. Huang, H\. Goh, R\. Zhang, and J\. Susskind \(2021\)An attention free transformer\.arXiv preprint arXiv:2105\.14103\.Cited by:[§G\.1](https://arxiv.org/html/2606.23957#A7.SS1.p1.13),[1st item](https://arxiv.org/html/2606.23957#S1.I1.i1.p1.1),[§1](https://arxiv.org/html/2606.23957#S1.p2.1),[§2\.2](https://arxiv.org/html/2606.23957#S2.SS2.SSS0.Px2.p1.3)\.

## Appendix ASystems and Data Generation

### A\.1Dynamical Systems

#### A\.1\.1Duffing Oscillator

The Duffing oscillator represents a paradigmatic example of nonlinear dynamics described by the second\-order differential equation

x¨\+δ​x˙\+α​x\+β​x3=γ​cos⁡\(ω​t\)\.\\ddot\{x\}\+\\delta\\dot\{x\}\+\\alpha x\+\\beta x^\{3\}=\\gamma\\cos\(\\omega t\)\.This system has been studied in the context of data\-driven modelling and Koopman operator theory due to its rich dynamical behavior and analytical tractability\(Otto and Rowley,[2019](https://arxiv.org/html/2606.23957#bib.bib41); Liet al\.,[2017](https://arxiv.org/html/2606.23957#bib.bib42); Alford\-Lagoet al\.,[2022](https://arxiv.org/html/2606.23957#bib.bib40); Pan and Duraisamy,[2020](https://arxiv.org/html/2606.23957#bib.bib39); Köhneet al\.,[2025](https://arxiv.org/html/2606.23957#bib.bib38)\)\.

In this work, we focus on the unforced and undamped case governed by

x¨=x−x3,\\ddot\{x\}=x\-x^\{3\},\(9\)which captures essential features such as switching dynamics between stable states and \(mixed/\)continuous spectrum characteristics, making it ideal for showcasing our proposed methodology\. The state\-space representation is

x˙1\\displaystyle\\dot\{x\}\_\{1\}=x2,\\displaystyle=x\_\{2\},\(10\)x˙2\\displaystyle\\dot\{x\}\_\{2\}=x1−x13\.\\displaystyle=x\_\{1\}\-x\_\{1\}^\{3\}\.Herex1x\_\{1\}denotes position andx2x\_\{2\}velocity\. Initial conditions are sampled uniformly from\(x1,x2\)∈\[−2,2\]\(x\_\{1\},x\_\{2\}\)\\in\[\-2,2\]with fixed step size as in Table[9](https://arxiv.org/html/2606.23957#A8.T9)\. This configuration admits two stable centers at\(x1,x2\)=\(±1,0\)\(x\_\{1\},x\_\{2\}\)=\(\\pm 1,0\)and an unstable fixed point at the origin\(0,0\)\(0,0\)\. The system is Hamiltonian and trajectories form closed orbits in phase space: low\-energy orbits are confined to individual potential wells, while high\-energy orbits encircle both wells, exhibiting switching behavior as trajectories periodically transition between states\.

The continuous\-spectrum nature of this system poses significant challenges for traditional Koopman operator approximation methods, as there is no straightforward finite\-dimensional approximation in terms of a small number of eigenfunctions\. Additionally, the switching dynamics between potential wells create computational difficulties even for short\-term prediction\.

#### A\.1\.2Repressilator

The Repressilator, a popular synthetic gene circuit\(Elowitz and Leibler,[2000](https://arxiv.org/html/2606.23957#bib.bib33)\), has become a canonical model for studying oscillatory dynamics that emerge from negative feedback regulation\. The circuit is composed of three transcriptional repressor genes arranged in a cyclic negative feedback loop, where each gene encodes a protein that inhibits the transcription of the next gene in the cycle, creating a ring\-like structure\. Many studies extensively analysed data\-driven modelling and control of such systems in\(Boddupalliet al\.,[2019](https://arxiv.org/html/2606.23957#bib.bib34); Sootlaet al\.,[2018](https://arxiv.org/html/2606.23957#bib.bib35); Balakrishnanet al\.,[2022](https://arxiv.org/html/2606.23957#bib.bib36); Perez\-Carrascoet al\.,[2018](https://arxiv.org/html/2606.23957#bib.bib37)\)\. Here, we focus on the case where the genetic circuit is isolated from bacterial host\(Weißeet al\.,[2015](https://arxiv.org/html/2606.23957#bib.bib75); Nikoladoset al\.,[2019](https://arxiv.org/html/2606.23957#bib.bib74)\)and admits a limit cycle in the phase portrait with a single basin of attraction centered at the origin\. We define the system as:

d​m\(i\)d​t=−δm​m\(i\)\+α1\+\(p\(j\)/K\)n\+α0,d​p\(i\)d​t=−δp​p\(i\)\+β​m\(i\),\\begin\{split\}\\frac\{dm\_\{\(i\)\}\}\{dt\}&=\-\\delta\_\{m\}m\_\{\(i\)\}\+\\frac\{\\alpha\}\{1\+\(p\_\{\(j\)\}/K\)^\{n\}\}\+\\alpha\_\{0\},\\\\ \\frac\{dp\_\{\(i\)\}\}\{dt\}&=\-\\delta\_\{p\}p\_\{\(i\)\}\+\\beta m\_\{\(i\)\},\\end\{split\}\(11\)wheremim\_\{i\}andpip\_\{i\}denotes the concentration of mRNA and protein of geneii, respectively\. The indices\(i,j\)\(i,j\)cycles through the repressor pairs\{\(lacI,cI\),\(tetR,lacI\),\(cI,tetR\)\}\\\{\(\\mathrm\{lacI\},\\mathrm\{cI\}\),\\;\(\\mathrm\{tetR\},\\mathrm\{lacI\}\),\\;\(\\mathrm\{cI\},\\mathrm\{tetR\}\)\\\}\. The model parameters represent basal and maximal transcription rates\(α0,α\)\(\\alpha\_\{0\},\\alpha\), Hill repression characteristics\(K,n\)\(K,n\), degradation rates\(δm,δp\)\(\\delta\_\{m\},\\delta\_\{p\}\), and the translation rate\(β\)\(\\beta\)\. We used the parameter valuesα0=0\.03\\alpha\_\{0\}=0\.03,α=10\\alpha=10,K=40K=40,n=2n=2,δm=0\.3466\\delta\_\{m\}=0\.3466,δp=0\.0693\\delta\_\{p\}=0\.0693, andβ=10\\beta=10in dimensionless units\.

#### A\.1\.3IRMA

The IRMA \(In vivo Reverse\-engineering and Modelling Assessment\) network is a well\-characterised synthetic gene circuit inSaccharomyces cerevisiae, constructed explicitly as a benchmark for modelling and control\. IRMA consists of five yeast transcription\-factor genes \(CBF1, GAL4, SWI5, ASH1, GAL80\) with a topology containing both positive and negative feedback loops\(Marucciet al\.,[2009](https://arxiv.org/html/2606.23957#bib.bib45); di Bernardoet al\.,[2011](https://arxiv.org/html/2606.23957#bib.bib44)\)\. It was designed to be insulated from native regulation and to respond specifically when cells are cultured in galactose\. This network has been used to test system\-identification and control methods\. For example,Menolascinaet al\.\([2014](https://arxiv.org/html/2606.23957#bib.bib43)\)applied closed\-loop control to regulate IRMA’s reporter output, andCantoneet al\.\([2009](https://arxiv.org/html/2606.23957#bib.bib46)\)used IRMA time\-series data to validate reverse\-engineering algorithms\. These studies demonstrate IRMA’s predictive modelling value\.

The mathematical model of IRMA is characterised by the following system of equations:

x1˙\\displaystyle\\dot\{x\_\{1\}\}=α1\+v1⋅k1h1k1h1\+x5h1−d1​x1,\\displaystyle=\\alpha\_\{1\}\+v\_\{1\}\\cdot\\frac\{k\_\{1\}^\{h\_\{1\}\}\}\{k\_\{1\}^\{h\_\{1\}\}\+x\_\{5\}^\{h\_\{1\}\}\}\-d\_\{1\}x\_\{1\},\(12\)x2˙\\displaystyle\\dot\{x\_\{2\}\}=α2\+v2⋅x1h2k2h2\+x1h2−d2​x2,\\displaystyle=\\alpha\_\{2\}\+v\_\{2\}\\cdot\\frac\{x\_\{1\}^\{h\_\{2\}\}\}\{k\_\{2\}^\{h\_\{2\}\}\+x\_\{1\}^\{h\_\{2\}\}\}\-d\_\{2\}x\_\{2\},x3˙\\displaystyle\\dot\{x\_\{3\}\}=α3\+v3⋅x2h3k3h3\+x2h3​\(1\+x4h6γh6\)−d3​x3,\\displaystyle=\\alpha\_\{3\}\+v\_\{3\}\\cdot\\frac\{x\_\{2\}^\{h\_\{3\}\}\}\{k\_\{3\}^\{h\_\{3\}\}\+x\_\{2\}^\{h\_\{3\}\}\\left\(1\+\\tfrac\{x\_\{4\}^\{h\_\{6\}\}\}\{\\gamma^\{h\_\{6\}\}\}\\right\)\}\-d\_\{3\}x\_\{3\},x4˙\\displaystyle\\dot\{x\_\{4\}\}=α4\+v4⋅x3h4k4h4\+x3h4−d4​x4,\\displaystyle=\\alpha\_\{4\}\+v\_\{4\}\\cdot\\frac\{x\_\{3\}^\{h\_\{4\}\}\}\{k\_\{4\}^\{h\_\{4\}\}\+x\_\{3\}^\{h\_\{4\}\}\}\-d\_\{4\}x\_\{4\},x5˙\\displaystyle\\dot\{x\_\{5\}\}=α5\+v5⋅x3h5k5h5\+x3h5−d5​x5\.\\displaystyle=\\alpha\_\{5\}\+v\_\{5\}\\cdot\\frac\{x\_\{3\}^\{h\_\{5\}\}\}\{k\_\{5\}^\{h\_\{5\}\}\+x\_\{3\}^\{h\_\{5\}\}\}\-d\_\{5\}x\_\{5\}\.
wherex1,x2,x3,x4,x5x\_\{1\},x\_\{2\},x\_\{3\},x\_\{4\},x\_\{5\}represent CBF1, GAL4, SWI5, GAL80, and ASH1 respectively, parameters follow the implementation inMarucciet al\.\([2009](https://arxiv.org/html/2606.23957#bib.bib45)\)and states are sampled from a uniform distribution over the interval\[0,1\]\[0,1\]\.

The parameters of the model include the basal expression ratesαi\\alpha\_\{i\}, the maximum expression ratesviv\_\{i\}, the half\-saturation constantskik\_\{i\}, the Hill coefficientshih\_\{i\}, the degradation ratesdid\_\{i\}, and the inhibition constantγ\\gamma\. Together, these parameters govern the nonlinear gene regulatory interactions and degradation dynamics of the IRMA circuit\.

### A\.2Additional Dynamical Systems

In addition to the three main benchmark systems \(§[A\.1](https://arxiv.org/html/2606.23957#A1.SS1)\), we also consider a collection of classical dynamical systems as benchmarks\. These systems are commonly used in the literature for system identification tasks, as they display diverse and rich dynamical behaviors\. We briefly describe each of them below\.

##### Nonlinear Pendulum\.

The pendulum represents a freely swinging pole\. Unlike the linear small\-angle approximation, the full nonlinear pendulum exhibits richer dynamics\. As the system energy increases, oscillations become strongly anharmonic, leading to a continuous Koopman spectrum\. The dynamics are given by:

x1˙\\displaystyle\\dot\{x\_\{1\}\}=x2,\\displaystyle=x\_\{2\},\(13\)x2˙\\displaystyle\\dot\{x\_\{2\}\}=−sin⁡\(x1\)\.\\displaystyle=\-\\sin\(x\_\{1\}\)\.\(14\)Initial conditions with angular positionsθ0\\theta\_\{0\}from a uniform distribution over\[−π,π\]\[\-\\pi,\\pi\]radians, with angular velocities fixed atω0=0\.0\\omega\_\{0\}=0\.0\.

##### Parabolic Attractor\.

Adopted fromLuschet al\.\([2018](https://arxiv.org/html/2606.23957#bib.bib48)\), this simple dynamical system has a single fixed point and a discrete eigenvalue spectrum:

x1˙\\displaystyle\\dot\{x\_\{1\}\}=μ​x1,\\displaystyle=\\mu x\_\{1\},\(15\)x2˙\\displaystyle\\dot\{x\_\{2\}\}=λ​\(x2−x12\)\.\\displaystyle=\\lambda\\big\(x\_\{2\}\-x\_\{1\}^\{2\}\\big\)\.\(16\)The system exhibits a slow manifold for stable eigenvaluesλ<μ<0\\lambda<\\mu<0, asymptotically attracted to the parabolax2=x12x\_\{2\}=x\_\{1\}^\{2\}\. We setλ=−1\.0\\lambda=\-1\.0andμ=−0\.1\\mu=\-0\.1, with initial conditions sampled uniformly fromx1,x2∈\[−1,1\]x\_\{1\},x\_\{2\}\\in\[\-1,1\]\.

##### Goodwin Oscillator\.

The three\-state Goodwin oscillator\(Goodwin,[1965](https://arxiv.org/html/2606.23957#bib.bib49)\)is a prototypical biochemical feedback model demonstrating how delayed negative feedback generates self\-sustained oscillations\. It consists of three variables \(commonly interpreted as mRNA, protein, and inhibitor\), where the inhibitor suppresses mRNA production\. The system is governed by:

x1˙\\displaystyle\\dot\{x\_\{1\}\}=ακ\+k​x3n−β​x1,\\displaystyle=\\frac\{\\alpha\}\{\\kappa\+kx\_\{3\}^\{n\}\}\-\\beta x\_\{1\},\(17\)x2˙\\displaystyle\\dot\{x\_\{2\}\}=γ​x1−δ​x2,\\displaystyle=\\gamma x\_\{1\}\-\\delta x\_\{2\},\(18\)x3˙\\displaystyle\\dot\{x\_\{3\}\}=η​x2−θ​x3,\\displaystyle=\\eta x\_\{2\}\-\\theta x\_\{3\},\(19\)where\(x1,x2,x3\)\(x\_\{1\},x\_\{2\},x\_\{3\}\)denote the concentrations of the three states and are sampled from uniform distributions over\[−2,2\]\[\-2,2\]for each state variable and used the following parameters to generate oscillation:a1=360a\_\{1\}=360,κ1=43\\kappa\_\{1\}=43,k1=1\.0k\_\{1\}=1\.0,n=12n=12,b1=0\.6b\_\{1\}=0\.6,α1=1\.0\\alpha\_\{1\}=1\.0,β1=1\.0\\beta\_\{1\}=1\.0,γ1=1\.0\\gamma\_\{1\}=1\.0, andδ1=0\.8\\delta\_\{1\}=0\.8\.

##### Lotka–Volterra System\.

The Lotka–Volterra equations describe a classical predator\-prey model whose populations can undergo sustained oscillations:

x1˙\\displaystyle\\dot\{x\_\{1\}\}=α​x1−β​x1​x2,\\displaystyle=\\alpha x\_\{1\}\-\\beta x\_\{1\}x\_\{2\},\(20\)x2˙\\displaystyle\\dot\{x\_\{2\}\}=δ​x1​x2−γ​x2\.\\displaystyle=\\delta x\_\{1\}x\_\{2\}\-\\gamma x\_\{2\}\.\(21\)The system admits two fixed points: extinction at\(0,0\)\(0,0\)and coexistence at\(γδ,αβ\)\\big\(\\tfrac\{\\gamma\}\{\\delta\},\\tfrac\{\\alpha\}\{\\beta\}\\big\)\. We follow the setup ofFathiet al\.\([2023](https://arxiv.org/html/2606.23957#bib.bib61)\)and setα=β=γ=δ=0\.2\\alpha=\\beta=\\gamma=\\delta=0\.2, with initial conditions sampled uniformly fromx1,x2∈\[0\.02,3\.0\]x\_\{1\},x\_\{2\}\\in\[0\.02,3\.0\]\.

##### Rössler System\.

The Rössler system\(Rössler,[1976](https://arxiv.org/html/2606.23957#bib.bib62)\)is a three\-dimensional chaotic system defined by:

x1˙\\displaystyle\\dot\{x\_\{1\}\}=−x2−x3,\\displaystyle=\-x\_\{2\}\-x\_\{3\},\(22\)x2˙\\displaystyle\\dot\{x\_\{2\}\}=x1\+a​x2,\\displaystyle=x\_\{1\}\+ax\_\{2\},\(23\)x3˙\\displaystyle\\dot\{x\_\{3\}\}=b\+x3​\(x1−c\)\.\\displaystyle=b\+x\_\{3\}\(x\_\{1\}\-c\)\.\(24\)With the canonical parameter set\(a,b,c\)=\(0\.2,0\.2,5\.7\)\(a,b,c\)=\(0\.2,0\.2,5\.7\), the system yields the well\-known strange attractor characterized by oscillations in the\(x1,x2\)\(x\_\{1\},x\_\{2\}\)\-plane and intermittent growth/decay alongx3x\_\{3\}\.

##### Fluid Flow Model\.

A reduced\-order model of fluid flow past a circular cylinder at Reynolds number100100\(Noacket al\.,[2003](https://arxiv.org/html/2606.23957#bib.bib50)\)is given by:

x1˙\\displaystyle\\dot\{x\_\{1\}\}=μ​x1−ω​x2\+A​x1​x3,\\displaystyle=\\mu x\_\{1\}\-\\omega x\_\{2\}\+Ax\_\{1\}x\_\{3\},\(25\)x2˙\\displaystyle\\dot\{x\_\{2\}\}=ω​x1\+μ​x2\+A​x2​x3,\\displaystyle=\\omega x\_\{1\}\+\\mu x\_\{2\}\+Ax\_\{2\}x\_\{3\},\(26\)x3˙\\displaystyle\\dot\{x\_\{3\}\}=−λ​\(x3−x12−x22\)\.\\displaystyle=\-\\lambda\\big\(x\_\{3\}\-x\_\{1\}^\{2\}\-x\_\{2\}^\{2\}\\big\)\.\(27\)With parametersμ=0\.1\\mu=0\.1,ω=1\.0\\omega=1\.0,A=−0\.1A=\-0\.1, andλ=10\\lambda=10, this system serves as a benchmark for fluid dynamics, exhibiting self\-sustained von Kármán vortex shedding\. We consider trajectories starting both on and off the slow manifold\.

## Appendix BExtended Related Work

### B\.1Koopman operator learning

Data\-driven approximations of the Koopman operator\(Koopman,[1931](https://arxiv.org/html/2606.23957#bib.bib1)\)have matured from dictionary\-based linear models to learned latent embeddings\. Extended DMD \(EDMD\) introduces a finite dictionary of observables and performs linear regression in the lifted space\(Liet al\.,[2017](https://arxiv.org/html/2606.23957#bib.bib42)\)\. Rigorous analyses quantify when EDMD converges and how spectra are approximated\(Korda and Mezić,[2018](https://arxiv.org/html/2606.23957#bib.bib15); Mezić,[2022](https://arxiv.org/html/2606.23957#bib.bib20); Giannakis and Valva,[2024](https://arxiv.org/html/2606.23957#bib.bib21); Colbrook and Townsend,[2024](https://arxiv.org/html/2606.23957#bib.bib14)\)\. Neural formulations replace hand\-crafted dictionaries with encoders/decoders that learn Koopman\-invariant coordinates end\-to\-end, often with a linearly recurrent bottleneck\(Otto and Rowley,[2019](https://arxiv.org/html/2606.23957#bib.bib41); Luschet al\.,[2018](https://arxiv.org/html/2606.23957#bib.bib48)\)\. These approaches have been used on canonical nonlinear systems—including Duffing\-type oscillators—to demonstrate improved single\-step prediction and limited\-horizon rollout accuracy\(Otto and Rowley,[2019](https://arxiv.org/html/2606.23957#bib.bib41); Liet al\.,[2017](https://arxiv.org/html/2606.23957#bib.bib42); Alford\-Lagoet al\.,[2022](https://arxiv.org/html/2606.23957#bib.bib40); Pan and Duraisamy,[2020](https://arxiv.org/html/2606.23957#bib.bib39); Köhneet al\.,[2025](https://arxiv.org/html/2606.23957#bib.bib38)\)\. Unlike black\-box recurrent\-based neural network models such as LSTM and GRU\(Elman,[1990](https://arxiv.org/html/2606.23957#bib.bib52); Hochreiter and Schmidhuber,[1997](https://arxiv.org/html/2606.23957#bib.bib51); Choet al\.,[2014](https://arxiv.org/html/2606.23957#bib.bib53)\), Koopman\-based methods yield interpretable latent coordinates and preserve better system dynamics in extrapolation\. Regularizers that bias the learned propagator toward \(near\-\)unitary dynamics have been explored to stabilize long\-horizon rollouts\(Enyeart and Lin,[2024](https://arxiv.org/html/2606.23957#bib.bib60)\)\. Symmetry\-aware variants study how equivariances shape Koopman spectra and model structure\(Salovaet al\.,[2019](https://arxiv.org/html/2606.23957#bib.bib29)\)\.

### B\.2Delay embeddings and memory

A parallel line of work augments Markov predictors with short\-term memory via time\-delay embeddings\. Hankel DMD constructs a block\-Hankel snapshot matrix to expose linear evolution in delay coordinates\(Arbabi and Mezic,[2017](https://arxiv.org/html/2606.23957#bib.bib67)\); related theory develops universal, system\-independent time\-delay observables\(Kambet al\.,[2020](https://arxiv.org/html/2606.23957#bib.bib19)\)\. HAVOK \(Hankel Alternative View of Koopman\) further separates a low\-dimensional linear model from a data\-driven forcing term that captures intermittent or chaotic dynamics\(Bruntonet al\.,[2017](https://arxiv.org/html/2606.23957#bib.bib68)\)\. These methods show that short windows of history can substantially reduce phase slippage and amplitude drift, motivating lightweight latent\-memory mechanisms in neural Koopman models\.

### B\.3Attention and hybrid Koopman models

Recent models couple Koopman structure with attention to aggregate recent context or to adapt locally\.Luet al\.\([2024](https://arxiv.org/html/2606.23957#bib.bib63)\)employ temporal attention inside an autoencoder to attenuate noise and improve forecasting;Wanget al\.\([2022](https://arxiv.org/html/2606.23957#bib.bib65)\)pair a global \(stationary\) Koopman map with a local transformer\-based operator to handle nonstationarity and transients\. We follow the same spirit—using short temporal context for robust prediction—while replacing quadratic\-cost multi\-head attention with a linear\-cost, attention\-free aggregation in latent space\.

### B\.4Inputs, control, and MPC

Complementary work integrates inputs and control: KIC extends Koopman predictors to systems with inputs\(Proctoret al\.,[2018](https://arxiv.org/html/2606.23957#bib.bib23)\); LNCIS surveys detail operator\-learning pipelines for control\(Kaiseret al\.,[2020](https://arxiv.org/html/2606.23957#bib.bib16)\); Koopman MPC demonstrates closed\-loop planning in lifted coordinates\(Korda and Mezić,[2020](https://arxiv.org/html/2606.23957#bib.bib17)\); and recent surveys emphasize applications in robot learning\(Shiet al\.,[2024](https://arxiv.org/html/2606.23957#bib.bib18)\)\. These applications motivate robustness over long horizons, as models that remain near the learned manifold are easier to certify and use in downstream control\.

### B\.5Projection, consistency, and recursive/local modeling

To limit compounding errors, several practices periodically project or reconcile predictions with the learned manifold\. Temporal consistency regularization encourages smooth, self\-consistent multi\-step predictions\(Nayaket al\.,[2025](https://arxiv.org/html/2606.23957#bib.bib69)\); delayed\-input concatenation provides a simple memory buffer for low\-dimensional series\(Frionet al\.,[2025](https://arxiv.org/html/2606.23957#bib.bib70)\)\. In the classical setting, windowed/recursive DMD maintains local linear surrogates from sliding subsets of recent data\(Noacket al\.,[2015](https://arxiv.org/html/2606.23957#bib.bib72); Dylewskyet al\.,[2019](https://arxiv.org/html/2606.23957#bib.bib73)\), and recent variants use windowed outputs to update local linear models online\(Guanet al\.,[2024](https://arxiv.org/html/2606.23957#bib.bib71)\)\. We operationalize a complementary idea in latent space: a cheap encode–decode–encode projection that snaps predictions back to the autoencoder manifold when drift is detected\.

### B\.6Streaming drift detection

Change\-point detection from statistical process control offers streaming triggers that are inexpensive and interpretable\. CUSUM tests cumulative deviations against a nominal mean\(Moustakides,[1986](https://arxiv.org/html/2606.23957#bib.bib54)\); EWMA emphasizes recent residuals through exponential smoothing\(Roberts,[2000](https://arxiv.org/html/2606.23957#bib.bib55)\); and sequential two\-sample procedures compare reference and current windows to detect broader distributional shifts\(Ross and Adams,[2012](https://arxiv.org/html/2606.23957#bib.bib56)\)\. We instantiate all three as latent\-drift monitors to decide when to re\-encode\.

### B\.7Biological circuits and broader benchmarks

Synthetic gene networks furnish controlled, nonlinear testbeds with oscillations and feedback\. The Repressilator\(Elowitz and Leibler,[2000](https://arxiv.org/html/2606.23957#bib.bib33)\)and the IRMA network \(*In vivo Reverse\-engineering and Modelling Assessment*\)\(Marucciet al\.,[2009](https://arxiv.org/html/2606.23957#bib.bib45); di Bernardoet al\.,[2011](https://arxiv.org/html/2606.23957#bib.bib44)\)have been repeatedly used for modeling and closed\-loop control\(Menolascinaet al\.,[2014](https://arxiv.org/html/2606.23957#bib.bib43); Cantoneet al\.,[2009](https://arxiv.org/html/2606.23957#bib.bib46)\)\. Koopman\-based predictors and controllers have also been explored for genetic circuits\(Hasnainet al\.,[2019](https://arxiv.org/html/2606.23957#bib.bib47)\)\. Beyond biology, standard dynamical\-systems benchmarks probe complementary difficulties: the Goodwin oscillator\(Goodwin,[1965](https://arxiv.org/html/2606.23957#bib.bib49)\), Rössler attractor\(Rössler,[1976](https://arxiv.org/html/2606.23957#bib.bib62)\), and reduced\-order cylinder flow\(Noacket al\.,[2003](https://arxiv.org/html/2606.23957#bib.bib50)\), as well as pedagogical systems such as the parabolic attractor\(Luschet al\.,[2018](https://arxiv.org/html/2606.23957#bib.bib48)\)and Lotka–Volterra\(Fathiet al\.,[2023](https://arxiv.org/html/2606.23957#bib.bib61)\)\. In our experiments, we focus our most rigorous evaluation on three representative systems \(Duffing, Repressilator, IRMA\) and use the remaining benchmarks to sanity\-check generalization of the finalized architecture\.

## Appendix CReproducibility Checklist

Code and data\.We release code, configuration files, and scripts to \(i\) generate datasets for all systems, \(ii\) train/evaluate each model variant \(KAE, KAE\+AFT, KAE\+MHA, KAE\+AFT\+Re\-enc\), and \(iii\) reproduce all tables/figures\. Re\-encoding triggers \(EWMA, CUSUM, windowed Z\-score, two\-sample\) are provided as modular components\.

Training and evaluation protocol\.We fix optimizer, learning\-rate schedule, batch size, rollout horizons, and early\-stopping criteria as in §[H](https://arxiv.org/html/2606.23957#A8)\. Models are evaluated by free rollouts from held\-out initial conditions; we report MSE and MCAE as defined in §[3\.2](https://arxiv.org/html/2606.23957#S3.SS2)\. All reported means are over multiple random initial conditions, with95%95\\%CIs for MCAE curves\.

Hyperparameters\.Architecture and training hyperparameters are summarized in Tables[8](https://arxiv.org/html/2606.23957#A8.T8)and[9](https://arxiv.org/html/2606.23957#A8.T9)\. Unless otherwise noted, the bottleneck isd=100d\{=\}100, AFT contextT=10T\{=\}10, andKKis dense\. Any deviations are stated near the corresponding results\.

Determinism and versions\.We provide exact library versions \(PyTorch, CUDA, numpy, scipy\) and OS details\. Where relevant, we disable non\-deterministic CuDNN kernels\.

Table 4:Checklist of key reproducibility items and where they are specified\.##### Data generation\.

All ODE systems were integrated withscipy\.integrate\.odeint, a wrapper of ODEPACK’s LSODA solver that automatically detects stiffness and switches between a variable–order Adams method \(non\-stiff\) and a BDF/Gear method \(stiff\), with adaptive internal step sizes and default error control \(relative and absolute tolerances left at SciPy/LSODA defaults\) following common practice in prior Koopman and system\-identification studies \(see §[B](https://arxiv.org/html/2606.23957#A2)\)\. Solutions were returned at user–specified sample timest0,…,tTt\_\{0\},\\dots,t\_\{T\}\(uniform linspace per dataset\), so the reportedΔ​t\\Delta tin tables refers to output sampling, not the solver’s internal step\. We did not supply Jacobians or event functions; LSODA formed finite–difference Jacobians as needed\. Initial conditions were sampled from the ranges stated in Appendix[A](https://arxiv.org/html/2606.23957#A1)and, for each system, we generate separate train/validation/test sets by sampling initial conditions\.

## Appendix DDynamic Re\-encoding Methods

To implement dynamic re\-encoding, we considered a set of online change\-point detection methods that can identify shifts in the drift error and decide when re\-encoding is beneficial\. These approaches vary in complexity, from simple threshold\-based rules to more sophisticated statistical tests, but they all share the goal of adapting the model to evolving data\. Below, we provide a brief description of each method:

1. 1\.Cumulative Sum \(CUSUM\): Originating from the work ofMoustakides \([1986](https://arxiv.org/html/2606.23957#bib.bib54)\), CUSUM is a sequential analysis technique that monitors cumulative deviations of observations from a target mean\. We employ a probabilistic variant that standardizes the observed MSE difference between predictions with and without re\-encoding, computes the cumulative sum, and converts it into a standard normal statistic\. We then derive a p\-value pT=2​\[1−Φ​\(\|s~T\|\)\],p\_\{T\}=2\\left\[1\-\\Phi\\bigl\(\|\\tilde\{s\}\_\{T\}\|\\bigr\)\\right\],whereΦ\\Phidenotes the standard normal CDF\. This p\-value quantifies the improbability of the observed cumulative deviation under the no\-change hypothesis\.
2. 2\.Threshold Re\-encoding: We quantify the discrepancy between the original latent predictionYpredY\_\{\\text\{pred\}\}and the re\-encoded predictionYpred\-afterY\_\{\\text\{pred\-after\}\}using a normalized mean squared difference: Δ=‖Ypred−Ypred\-after‖2‖Ypred‖2\+ϵ\.\\Delta=\\frac\{\\\|Y\_\{\\text\{pred\}\}\-Y\_\{\\text\{pred\-after\}\}\\\|^\{2\}\}\{\\\|Y\_\{\\text\{pred\}\}\\\|^\{2\}\+\\epsilon\}\.A re\-encode is triggered whenΔ\\Deltaexceeds a predefined threshold\.
3. 3\.Window Re\-encoding: We track the MSE difference between the standard and re\-encoded predictions in a fixed\-size sliding window\. Re\-encoding is activated if the most recent MSE exceeds the window’s mean plus a configurable multiple of its standard deviation, enabling adaptive response to abnormal fluctuations while balancing stability and efficiency\.
4. 4\.Exponentially Weighted Moving Average \(EWMA\): Introduced inRoberts \([2000](https://arxiv.org/html/2606.23957#bib.bib55)\), the EWMA method computes a smoothed statistic that emphasizes recent observations\. The update rule is Zt=\(1−λ\)​Zt−1\+λ​δt,λ∈\(0,1\),Z\_\{t\}=\(1\-\\lambda\)\\,Z\_\{t\-1\}\+\\lambda\\,\\delta\_\{t\},\\quad\\lambda\\in\(0,1\),Where a new observation isδt\\delta\_\{t\}, the smoothing parameter isλ\\lambda\. The method maintains running estimates of the meanμt\\mu\_\{t\}and the standard deviationσZ\\sigma\_\{Z\}of the EWMA statistic\. A change point is declared if \|Zt−μt\|L\>σZ,\\frac\{\|Z\_\{t\}\-\\mu\_\{t\}\|\}\{L\}\>\\sigma\_\{Z\},withLLbeing a sensitivity scaling factor\.
5. 5\.Sequential Two\-Sample Test: Extending the methods ofRoss and Adams \([2012](https://arxiv.org/html/2606.23957#bib.bib56)\), this approach partitions the data stream into a “reference” and a “current” window buffer and applies nonparametric tests \(e\.g\., Kolmogorov–Smirnov, Lepage, Mann–Whitney\) to detect distributional shifts beyond mean changes—such as variance or skewness deviations\.

## Appendix EGRU and Transformer Architectures

We use GRUs and transformers in an autoencoder architecture as a baseline to compare with the Koopman autoencoders\. We chose these as baselines due to their ability to model the temporal and spatial dependence from the training data\. We use simple model architectures to allow the model to be as expressive as possible to learn from the provided data\. A description of each autoencoder is provided below\.

### E\.1GRU Autoencoder

##### Model\.

LetXt=\(xt,xt\+1,…,xt\+T−1\)∈ℝp×TX\_\{t\}=\(x\_\{t\},x\_\{t\}\+1,\\dots,x\_\{t\+T\-1\}\)\\in\\mathbb\{R\}^\{p\\times T\}denote the observed window of lengthTTof states at timettand letEg​r​u:ℝp→ℝdE\_\{gru\}:\\mathbb\{R\}^\{p\}\\\!\\to\\\!\\mathbb\{R\}^\{d\}be annn\-layer GRU encoder andDm​l​p:ℝd→ℝpD\_\{mlp\}:\\mathbb\{R\}^\{d\}\\\!\\to\\\!\\mathbb\{R\}^\{p\}be a one\-layer MLP decoder\. Together,Eg​r​uE\_\{gru\}maps the input window of lengthTTto add\-dimensional latent space andDm​l​pD\_\{mlp\}decodes back to a next\-step prediction inℝp\\mathbb\{R\}^\{p\}:

x^t\+T=Dm​l​p​\(Eg​r​u​\(Xt\)\)\\hat\{x\}\_\{t\+T\}=D\_\{mlp\}\(E\_\{gru\}\(X\_\{t\}\)\)\(28\)
so that anii–step rollout from an initial observed windowX0X\_\{0\}isiiautoregressive applications of the autoencoder, shown in algorithm[3](https://arxiv.org/html/2606.23957#algorithm3)\.

##### Training losses\.

Given an input windowX0X\_\{0\}of lengthTT, we minimize the autoregressive prediction error over a rollout ofTp​r​e​dT\_\{pred\}steps\. Letfg​r​u​\_​a​r:X0→X^Tf\_\{gru\\\_ar\}:X\_\{0\}\\rightarrow\\hat\{X\}\_\{T\}denote the algorithm described in[3](https://arxiv.org/html/2606.23957#algorithm3)for aTp​r​e​d\{T\_\{pred\}\}\-step model rollout with context lengthTT\.

ℒ\\displaystyle\\mathcal\{L\}=1Tp​r​e​d​‖XT−fg​r​u​\_​a​r​\(X0\)‖22,\\displaystyle=\\frac\{1\}\{T\_\{pred\}\}\\big\\\|X\_\{T\}\-f\_\{gru\\\_ar\}\(X\_\{0\}\)\\big\\\|\_\{2\}^\{2\},\(29a\)
We train by using this loss function over the training data\.

### E\.2Transformer Autoencoder

The transformer autoencoder is almost identical to the GRU autoencoder described in[E\.1](https://arxiv.org/html/2606.23957#A5.SS1)except that the encoder is now annn\-layer GRU followed by anmm\-layer transformer encoder\. The GRU head is used to embed the input data to a higher dimension before being passed through the transformer\.

##### Model\.

LetXt=\(xt,xt\+1,…,xt\+T−1\)∈ℝp×TX\_\{t\}=\(x\_\{t\},x\_\{t\}\+1,\\dots,x\_\{t\+T\-1\}\)\\in\\mathbb\{R\}^\{p\\times T\}denote the observed window of lengthTTof states at timettand letEt​r:ℝp→ℝdE\_\{tr\}:\\mathbb\{R\}^\{p\}\\\!\\to\\\!\\mathbb\{R\}^\{d\}be an\(n\+m\)\(n\+m\)\-layer transformer encoder with andDm​l​p:ℝd→ℝpD\_\{mlp\}:\\mathbb\{R\}^\{d\}\\\!\\to\\\!\\mathbb\{R\}^\{p\}be a one\-layer MLP decoder\. Together,Et​rE\_\{tr\}maps the input window of lengthTTto add\-dimensional latent space andDm​l​pD\_\{mlp\}decodes back to a next\-step prediction inℝp\\mathbb\{R\}^\{p\}:

x^t\+T=Dm​l​p​\(Et​r​\(Xt\)\)\\hat\{x\}\_\{t\+T\}=D\_\{mlp\}\(E\_\{tr\}\(X\_\{t\}\)\)\(30\)
so that anii–step rollout from an initial observed windowX0X\_\{0\}isiiautoregressive applications of the autoencoder, shown in algorithm[3](https://arxiv.org/html/2606.23957#algorithm3)\.

##### Training losses\.

Given an input windowX0X\_\{0\}of lengthTT, we minimize the autoregressive prediction error over a rollout ofTp​r​e​dT\_\{pred\}steps\. Letft​\_​a​r:X0→X^Tf\_\{t\\\_ar\}:X\_\{0\}\\rightarrow\\hat\{X\}\_\{T\}denote the algorithm described in[3](https://arxiv.org/html/2606.23957#algorithm3)for aTp​r​e​d\{T\_\{pred\}\}\-step model rollout with context lengthTT\.

ℒ\\displaystyle\\mathcal\{L\}=1Tp​r​e​d​‖XT−ft​\_​a​r​\(X0\)‖22,\\displaystyle=\\frac\{1\}\{T\_\{pred\}\}\\big\\\|X\_\{T\}\-f\_\{t\\\_ar\}\(X\_\{0\}\)\\big\\\|\_\{2\}^\{2\},\(31a\)
We train by using this loss function over the training data\.

## Appendix FAlgorithms

Input:initial state

x0x\_\{0\}, horizon

TpredT\_\{\\mathrm\{pred\}\}, context length

TT
Output:predicted states

\{x^t\}t=0Tpred\\\{\\hat\{x\}\_\{t\}\\\}\_\{t=0\}^\{T\_\{\\mathrm\{pred\}\}\}
Data:encoder

φ\\varphi, decoder

φ−1\\varphi^\{\-1\}, Koopman map

KK, AFT params

WQ,WK,WVW\_\{Q\},W\_\{K\},W\_\{V\}, position bias

ww, causal mask

MM
1

z0←φ​\(x0\)z\_\{0\}\\leftarrow\\varphi\(x\_\{0\}\);

x^0←x0\\hat\{x\}\_\{0\}\\leftarrow x\_\{0\};

2for*t←1t\\leftarrow 1toTpredT\_\{\\mathrm\{pred\}\}*do

//Assemble latent history \(causal, chronological, lengthTT\)

3build

Ht−1=\[zt−T,…,zt−1\]H\_\{t\-1\}=\[z\_\{t\-T\},\\ldots,z\_\{t\-1\}\]\(truncate if

t<Tt\\\!<\\\!T\);

//AFT projections

4

q←σq​\(Ht−1​WQ\)q\\leftarrow\\sigma\_\{q\}\(H\_\{t\-1\}W\_\{Q\}\);

Kt←Ht−1​WK/dm​o​d​e​lK\_\{t\}\\leftarrow H\_\{t\-1\}W\_\{K\}/\\sqrt\{d\_\{model\}\};

Vt←Ht−1​WV/dm​o​d​e​lV\_\{t\}\\leftarrow H\_\{t\-1\}W\_\{V\}/\\sqrt\{d\_\{model\}\};

//Element\-wise linear attention in latent space

5

αi,j←exp⁡\(kj\+wi,j\)⋅Mi,j\\alpha\_\{i,j\}\\leftarrow\\exp\(k\_\{j\}\+w\_\{i,j\}\)\\cdot M\_\{i,j\}for all

i,ji,j;

6

z~i←qi⊙∑jαi,j⊙vj∑jαi,j\\tilde\{z\}\_\{i\}\\leftarrow q\_\{i\}\\odot\\frac\{\\sum\_\{j\}\\alpha\_\{i,j\}\\odot v\_\{j\}\}\{\\sum\_\{j\}\\alpha\_\{i,j\}\}for all

ii;

//Koopman propagation \+ decode

7

zt←K​z~t−1z\_\{t\}\\leftarrow K\\,\\tilde\{z\}\_\{t\-1\};

x^t←φ−1​\(zt\)\\hat\{x\}\_\{t\}\\leftarrow\\varphi^\{\-1\}\(z\_\{t\}\);

8

9end for

return*\{x^t\}t=0Tpred\\\{\\hat\{x\}\_\{t\}\\\}\_\{t=0\}^\{T\_\{\\mathrm\{pred\}\}\}*

Algorithm 1AFT–Koopman rollout \(no re\-encoding\)Input:initial state

x0x\_\{0\}, horizon

TpredT\_\{\\mathrm\{pred\}\}, context length

TT, trigger config

Θ\\Theta
Output:predicted states

\{x^t\}t=0Tpred\\\{\\hat\{x\}\_\{t\}\\\}\_\{t=0\}^\{T\_\{\\mathrm\{pred\}\}\}, re\-encode steps

ℛ\\mathcal\{R\}
Data:encoder

φ\\varphi, decoder

φ−1\\varphi^\{\-1\}, Koopman map

KK, AFT params

WQ,WK,WVW\_\{Q\},W\_\{K\},W\_\{V\}, position bias

ww, causal mask

MM
1

z0←φ​\(x0\)z\_\{0\}\\leftarrow\\varphi\(x\_\{0\}\);

x^0←x0\\hat\{x\}\_\{0\}\\leftarrow x\_\{0\};

ℛ←∅\\mathcal\{R\}\\leftarrow\\varnothing;

2for*t←1t\\leftarrow 1toTpredT\_\{\\mathrm\{pred\}\}*do

//Get original and re\-encoded versions ofzt−1z\_\{t\-1\}

3

zt−1orig←zt−1z\_\{t\-1\}^\{\\mathrm\{orig\}\}\\leftarrow z\_\{t\-1\};

zt−1re−enc←φ​\(φ−1​\(zt−1\)\)z\_\{t\-1\}^\{\\mathrm\{re\\\!\-\\\!enc\}\}\\leftarrow\\varphi\(\\varphi^\{\-1\}\(z\_\{t\-1\}\)\);

//Apply AFT function to both versions

4build

Ht−1=\[zt−T,…,zt−1\]H\_\{t\-1\}=\[z\_\{t\-T\},\\ldots,z\_\{t\-1\}\]\(truncate if

t<Tt\\\!<\\\!T\);

5

Δ​zorig←AFT​\(zt−1orig,Ht−1\)\\Delta z^\{\\mathrm\{orig\}\}\\leftarrow\\mathrm\{AFT\}\(z\_\{t\-1\}^\{\\mathrm\{orig\}\},H\_\{t\-1\}\);

6

Δ​zre−enc←AFT​\(zt−1re−enc,Ht−1\)\\Delta z^\{\\mathrm\{re\\\!\-\\\!enc\}\}\\leftarrow\\mathrm\{AFT\}\(z\_\{t\-1\}^\{\\mathrm\{re\\\!\-\\\!enc\}\},H\_\{t\-1\}\);

//Update both versions with their residuals

7

zt−1orig←zt−1orig\+Δ​zorigz\_\{t\-1\}^\{\\mathrm\{orig\}\}\\leftarrow z\_\{t\-1\}^\{\\mathrm\{orig\}\}\+\\Delta z^\{\\mathrm\{orig\}\};

8

zt−1re−enc←zt−1re−enc\+Δ​zre−encz\_\{t\-1\}^\{\\mathrm\{re\\\!\-\\\!enc\}\}\\leftarrow z\_\{t\-1\}^\{\\mathrm\{re\\\!\-\\\!enc\}\}\+\\Delta z^\{\\mathrm\{re\\\!\-\\\!enc\}\};

//Apply Koopman operator to both updated versions

9

ztorig←K​zt−1origz\_\{t\}^\{\\mathrm\{orig\}\}\\leftarrow K\\,z\_\{t\-1\}^\{\\mathrm\{orig\}\};

10

ztre−enc←K​zt−1re−encz\_\{t\}^\{\\mathrm\{re\\\!\-\\\!enc\}\}\\leftarrow K\\,z\_\{t\-1\}^\{\\mathrm\{re\\\!\-\\\!enc\}\};

//Calculate difference after Koopman propagation

11

δt←‖ztre−enc−ztorig‖22\\delta\_\{t\}\\leftarrow\\\|z\_\{t\}^\{\\mathrm\{re\\\!\-\\\!enc\}\}\-z\_\{t\}^\{\\mathrm\{orig\}\}\\\|\_\{2\}^\{2\};

//Streaming triggers \(EWMA / CUSUM / window / two\-sample\)

12if*TriggerFires​\(δt;Θ\)\\mathrm\{TriggerFires\}\(\\delta\_\{t\};\\Theta\)*then

13

zt←ztre−encz\_\{t\}\\leftarrow z\_\{t\}^\{\\mathrm\{re\\\!\-\\\!enc\}\};

ℛ←ℛ∪\{t\}\\mathcal\{R\}\\leftarrow\\mathcal\{R\}\\cup\\\{t\\\};

14

15end if

16else

17

zt←ztorigz\_\{t\}\\leftarrow z\_\{t\}^\{\\mathrm\{orig\}\};

18

19end if

20

x^t←φ−1​\(zt\)\\hat\{x\}\_\{t\}\\leftarrow\\varphi^\{\-1\}\(z\_\{t\}\);

21

22end for

return*\{x^t\}t=0Tpred\\\{\\hat\{x\}\_\{t\}\\\}\_\{t=0\}^\{T\_\{\\mathrm\{pred\}\}\},ℛ\\mathcal\{R\}*

Algorithm 2AFT–Koopman rollout with dynamic re\-encoding \(inference only\)Input:initial window

X0=\(x0,x1,…,xT−1\)X\_\{0\}=\(x\_\{0\},x\_\{1\},\\dots,x\_\{T\-1\}\), horizon

TpredT\_\{\\mathrm\{pred\}\}, context length

TT
Output:predicted states

X^T=\{x^t\}t=TT\+Tpred\\hat\{X\}\_\{T\}=\\\{\\hat\{x\}\_\{t\}\\\}\_\{t=T\}^\{T\+T\_\{\\mathrm\{pred\}\}\}
Data:encoder

EE\(either

Eg​r​uE\_\{gru\}or

Et​rE\_\{tr\}\), decoder

Dm​l​pD\_\{mlp\}
1

Xt​m​p←X0X\_\{tmp\}\\leftarrow X\_\{0\};

2for*t←1t\\leftarrow 1toTpredT\_\{\\mathrm\{pred\}\}*do

//Encode the input sequence

3

Zt←E​\(Xt​m​p\)Z\_\{t\}\\leftarrow E\(X\_\{tmp\}\);

//Decode the latent space

4

x^t\+T−1=Dm​l​p​\(Zt\)\\hat\{x\}\_\{t\+T\-1\}=D\_\{mlp\}\(Z\_\{t\}\);

//Autoregressively prepare the next input

5

Xt​m​p←\(x^t,x^t\+1,…,x^t\+T−1\)X\_\{tmp\}\\leftarrow\(\\hat\{x\}\_\{t\},\\hat\{x\}\_\{t\+1\},\\dots,\\hat\{x\}\_\{t\+T\-1\}\);

6

7end for

return*X^T=\{x^t\}t=TT\+Tpred\\hat\{X\}\_\{T\}=\\\{\\hat\{x\}\_\{t\}\\\}\_\{t=T\}^\{T\+T\_\{\\mathrm\{pred\}\}\}*

Algorithm 3GRU and Transformer rollout
## Appendix GAdditional Results and Analyses

### G\.1Complexity and parameter footprint

Attention\-free latent memory vs\. MHA\.Letddbe the latent \(bottleneck\) dimension andTTthe AFT history length\. The AFT block adds three linear mapsWQ,WK,WV∈ℝd×dW\_\{Q\},W\_\{K\},W\_\{V\}\\in\\mathbb\{R\}^\{d\\times d\}and a learned position\-only biasw∈ℝT×Tw\\in\\mathbb\{R\}^\{T\\times T\}, for a total of3​d2\+T23d^\{2\}\+T^\{2\}parameters\. With the settings used in most experiments \(d=100d\{=\}100,T=10T\{=\}10\), this is≈30,100\\approx 30\{,\}100parameters\. The aggregation cost per step is linear in window length,𝒪​\(T​d\)\\mathcal\{O\}\(Td\), since we compute a weighted sum over the lastTTkey/value pairs with a fixed \(causal\) position bias\(Zhaiet al\.,[2021](https://arxiv.org/html/2606.23957#bib.bib59)\)\. By contrast, dot\-product multi\-head attention over a window of sizeTTrequires forming attention scores over all pairs, yielding𝒪​\(T2​d\)\\mathcal\{O\}\(T^\{2\}d\)time and𝒪​\(T2\)\\mathcal\{O\}\(T^\{2\}\)memory for the attention map, in addition to comparable linear projections\.

Total inference cost\.Per time step, the KAE backbone incurs oned×dd\\times dKoopman multiply and one decode; AFT adds one extrad×dd\\times dprojection and a windowed𝒪​\(T​d\)\\mathcal\{O\}\(Td\)aggregation\. The dynamic re\-encoding step introduces an additional encode\-decode\-encode \(φ−1\\varphi^\{\-1\}thenφ\\varphi\) per time step, so we incur additional\(cost​\[φ\]\+cost​\[φ−1\]\)\(\\mathrm\{cost\}\[\\varphi\]\+\\mathrm\{cost\}\[\\varphi^\{\-1\}\]\)\. All triggers operate in𝒪​\(1\)\\mathcal\{O\}\(1\)time per step with respect to rollout length \(the two\-sample test maintains fixed\-size buffers, i\.e\.,𝒪​\(w\)\\mathcal\{O\}\(w\)per update for constantww\)\.

Memory footprint\.We store the lastTTlatents \(𝒪​\(T​d\)\\mathcal\{O\}\(Td\)\) and no denseT×TT\\times Tattention maps at inference time\. This linear memory scaling enables long rollouts with a small fixed context\.

### G\.2Inference Time Evaluation

When deploying machine learning models from offline forecasting to real\-time control of dynamical systems, computational efficiency becomes as critical as prediction accuracy, since control systems operate under strict timing constraints where inference delays can destabilize the entire system\. We evaluated our models and inference methods on the IRMA dynamical system, predicting 100 time steps from 10 initial conditions and 5 trials per method to ensure statistical reliability of timing measurements\. We report four metrics: \(i\) Time \[s\], the average wall\-clock time per trial; \(ii\) Throughput \[traj/s\], the number of trajectories predicted per second; \(iii\) Latency \[ms\], the average inference time per trajectory; and \(iv\) Efficiency \[MFLOPS\], the floating\-point operations executed per second, as measured using PyTorch profiler on M3 CPU hardware\. These metrics reflect real executed operations rather than theoretical complexity estimates\.

Table 5:Table of runtime performance for different models and inference methods\.Table[5](https://arxiv.org/html/2606.23957#A7.T5)presents the inference time evaluation results for all methods on the IRMA dynamical system\. The Koopman\-based approaches demonstrate superior computational efficiency, with Koopman AE achieving the lowest average inference time of 0\.11s per trial and the highest throughput of 94\.4 trajectories per second\. The AFT variants, while slightly slower than the AE formulation, still offer good performance with throughput rates of 74\.8\-76\.5 trajectories per second and notably higher computational efficiency, achieving 6774\-6915 MFLOPS compared to 694 MFLOPS for the AE method\. This indicates that AFT methods perform more intensive computations while maintaining fast inference times\. In contrast, traditional sequence models exhibit significantly higher latency, with the Transformer and GRU requiring 491ms and 608ms per trajectory, respectively\.

### G\.3Seed Robustness, Phase\-Plane Views, and Error Accumulation for the core trio systems

Table[6](https://arxiv.org/html/2606.23957#A7.T6)reports mean±\\pmstd MSE across random seeds \(values scaled by×100\\times 100\) for the Koopman baselines and our re\-encoding variants\. OnDuffing, dynamic re\-encoding yields the lowest error at all horizons \(e\.g\.,1\.66±0\.601\.66\\\!\\pm\\\!0\.60at 200 steps\), reducing both mean and variance relative to KAE and AFT, and maintaining a gap through 1000 steps \(19\.04±0\.9519\.04\\\!\\pm\\\!0\.95vs\.25\.63±0\.9725\.63\\\!\\pm\\\!0\.97for KAE\)\. This aligns with the switching\-sensitive dynamics where timely snaps curb manifold drift \(cf\. Table[1](https://arxiv.org/html/2606.23957#S4.T1), Fig\.[2\(c\)](https://arxiv.org/html/2606.23957#S4.F2.sf3)\)\. On theRepressilator, AFT without re\-encoding is consistently best \(0\.01±0\.000\.01\\\!\\pm\\\!0\.00,0\.07±0\.050\.07\\\!\\pm\\\!0\.05,0\.19±0\.200\.19\\\!\\pm\\\!0\.20at 200/500/1000\), while snap\-backs degrade performance \(∼0\.40\\sim\\\!0\.40–0\.710\.71\), corroborating that triggers can inject phase resets on clean limit cycles \(see §[4\.1](https://arxiv.org/html/2606.23957#S4.SS1)\)\. ForIRMA, dynamic \(and periodic\) re\-encoding dominate across horizons \(e\.g\.,0\.01±0\.010\.01\\\!\\pm\\\!0\.01at 200 and0\.04±0\.010\.04\\\!\\pm\\\!0\.01at 1000\), with AFT close but consistently worse, reflecting the benefit of guarding against gradual manifold drift in higher\-dimensional feedback systems\.

Table 6:Mean Squared Error \(MSE\) over different time steps for Koopman methods running on different seed values, scaled by 100, with best values highlighted\.StepsKoopmanKoopmanAFT\+Re\-encodingAEAFTDynamicPeriodicMSE over different time stepsDuffing Oscillator20011\.24±0\.8411\.24\\pm 0\.847\.43±2\.247\.43\\pm 2\.241\.66±0\.60\\mathbf\{1\.66\\pm 0\.60\}1\.92±0\.541\.92\\pm 0\.5450023\.01±1\.8923\.01\\pm 1\.8922\.05±3\.8922\.05\\pm 3\.8911\.35±2\.64\\mathbf\{11\.35\\pm 2\.64\}11\.93±1\.4711\.93\\pm 1\.47100025\.63±0\.9725\.63\\pm 0\.9727\.54±7\.4827\.54\\pm 7\.4819\.04±0\.95\\mathbf\{19\.04\\pm 0\.95\}21\.82±0\.9721\.82\\pm 0\.97Repressilator2000\.01±0\.00\\mathbf\{0\.01\\pm 0\.00\}0\.01±0\.00\\mathbf\{0\.01\\pm 0\.00\}0\.40±0\.050\.40\\pm 0\.050\.42±0\.050\.42\\pm 0\.055000\.23±0\.050\.23\\pm 0\.050\.07±0\.05\\mathbf\{0\.07\\pm 0\.05\}0\.45±0\.030\.45\\pm 0\.030\.35±0\.010\.35\\pm 0\.0110001\.37±0\.831\.37\\pm 0\.830\.19±0\.20\\mathbf\{0\.19\\pm 0\.20\}0\.71±0\.040\.71\\pm 0\.040\.71±0\.070\.71\\pm 0\.07IRMA2001\.18±0\.201\.18\\pm 0\.200\.03±0\.000\.03\\pm 0\.000\.01±0\.01\\mathbf\{0\.01\\pm 0\.01\}0\.01±0\.01\\mathbf\{0\.01\\pm 0\.01\}5003\.13±1\.313\.13\\pm 1\.310\.08±0\.040\.08\\pm 0\.040\.02±0\.02\\mathbf\{0\.02\\pm 0\.02\}0\.02±0\.01\\mathbf\{0\.02\\pm 0\.01\}10003\.22±1\.173\.22\\pm 1\.170\.12±0\.030\.12\\pm 0\.030\.04±0\.01\\mathbf\{0\.04\\pm 0\.01\}0\.06±0\.010\.06\\pm 0\.01Figure[6](https://arxiv.org/html/2606.23957#A7.F6)complements these statistics: phase\-plane/3D rollouts illustrate that re\-encoding prevents rare\-but\-catastrophic divergence and preserves switching structure on Duffing, while remaining faithful to the attractors on Repressilator and IRMA\. The long\-horizon trajectories in Fig\.[5](https://arxiv.org/html/2606.23957#A7.F5)make this concrete: AFT remains phase\-locked to the reference over10001000steps while the plain KAE drifts\. The MCAE curves in Fig\.[7](https://arxiv.org/html/2606.23957#A7.F7)further expose error\-growth dynamics: on Duffing and IRMA, dynamic re\-encoding flattens cumulative error relative to KAE and AFT, whereas on Repressilator the AFT\-only curve remains lowest and most stable, consistent with Table[1](https://arxiv.org/html/2606.23957#S4.T1)and our guidance in §[5](https://arxiv.org/html/2606.23957#S5)\(“When and why re\-encoding helps”\)\.

![Refer to caption](https://arxiv.org/html/2606.23957v1/Figures/Repressilator_AFTvsKAE_longhorizon.png)Figure 5:AFT vs\. plain KAE on the Repressilator over a10001000\-step rollout\(two representative trajectories, first observable\)\. AFT \(red\) stays phase\-locked to the reference \(blue\) across the full horizon, whereas the plain KAE \(green\) accumulates phase and amplitude drift, illustrating the long\-horizon stability gained from the latent memory\.![Refer to caption](https://arxiv.org/html/2606.23957v1/Figures/Repressilator_3d_prediction.png)\(a\)Repressilator — 3D prediction
![Refer to caption](https://arxiv.org/html/2606.23957v1/Figures/IRMA_3d_prediction_0.png)\(b\)IRMA — 3D prediction
![Refer to caption](https://arxiv.org/html/2606.23957v1/Figures/duffing_multi_initial_conditions.png)\(c\)Duffing — multi\-traj

Figure 6:Phase Plane Visualization of the systems\.Dynamic re\-encoding prevents rare\-but\-catastrophic divergence on long rollouts and provides robust trajectory prediction across different initial conditions\.![Refer to caption](https://arxiv.org/html/2606.23957v1/Figures/mcae_duffing.png)\(a\)Duffing oscillator
![Refer to caption](https://arxiv.org/html/2606.23957v1/Figures/MCAE_IRMA.png)\(b\)IRMA system
![Refer to caption](https://arxiv.org/html/2606.23957v1/Figures/MCAE_Repressilator.png)\(c\)Repressilator system

Figure 7:Mean cumulative absolute error \(MCAE\) results for our three dynamical systems, complementing the quantitative results presented in Table[1](https://arxiv.org/html/2606.23957#S4.T1)\. The plots show prediction error accumulation over time for \(a\) the Duffing oscillator, \(b\)IRMA, and \(c\) the Repressilator\.
### G\.4Additional Dynamical Systems

To assess out\-of\-the\-box robustness, we hold architecture and training protocols fixed across tasks \(varying only loss weights\) and evaluate on a diverse suite spanning continuous spectra, limit cycles, and chaos\. The suite includes: the nonlinear pendulum \(anharmonic, continuous spectrum\), the Goodwin oscillator \(sustained biochemical oscillations; complementary 200/500\-step horizons\), the parabolic attractor \(fully linearizable by standard Koopman coordinates\), the Rössler system \(canonical 3D chaos\), Lotka–Volterra \(predator–prey oscillations\), and a reduced\-order fluid\-flow model capturing von Kármán vortex shedding\. These systems cover regimes from simple discrete spectra to chaotic attractors, providing a stringent test of generalization\. Quantitative 200\-step MSE results \(plus 500\-step for Goodwin\) appear in Table[7](https://arxiv.org/html/2606.23957#A7.T7); representative rollouts are shown in Fig\.[8](https://arxiv.org/html/2606.23957#A7.F8)\.

Table 7:Prediction performance comparison \(MSE↓\\downarrow\) over 200 prediction steps across different system configurations\. Lower values indicate better performance\. Best results for each system are highlighted in bold\.Broadly, AFT improves or matches the Koopman AE baseline, and AFT\+re\-encoding helps where drift accumulates \(pendulum, Lotka–Volterra, fluid flow, Rössler\), while offering no benefit on trivially linearizable dynamics \(parabolic\) or very clean short\-horizon oscillations \(Goodwin at 200\)\. No per\-system tuning beyond the loss weights was performed\.

![Refer to caption](https://arxiv.org/html/2606.23957v1/Figures/FluidFlow_ex.png)\(a\)FluidFlow
![Refer to caption](https://arxiv.org/html/2606.23957v1/Figures/goodwin_ex.png)\(b\)Goodwin Oscillator
![Refer to caption](https://arxiv.org/html/2606.23957v1/Figures/Rossler_ex.png)\(c\)Rössler
![Refer to caption](https://arxiv.org/html/2606.23957v1/Figures/Pendulum_ex.png)\(d\)Pendulum
![Refer to caption](https://arxiv.org/html/2606.23957v1/Figures/LotkaVolterra_ex.png)\(e\)Lotka–Volterra
![Refer to caption](https://arxiv.org/html/2606.23957v1/Figures/ParabolicAttractor_ex.png)\(f\)Parabolic Attractor

Figure 8:Additional Dynamical systems\.AFT \(and AFT\+Re\-enc where helpful\) improves or matches the baseline across diverse regimes\. We did not perform additional, extensive per\-system tuning\.
### G\.5Ablations: operator size and AFT context

During training, re\-encoding is disabled and only activated during inference \(Algorithm[1](https://arxiv.org/html/2606.23957#algorithm1)\)\. Models with larger operator sizes consistently achieve better performance than their smaller counterparts, though this performance gap narrows with the introduction of AFT\. This difference is most pronounced in the Repressilator experiments \(Fig[9](https://arxiv.org/html/2606.23957#A7.F9); left\)\. For context length, short windows \(T∈\[8,16\]T\\in\[8,16\]\) perform best \(Fig[9](https://arxiv.org/html/2606.23957#A7.F9); right\)\. This is likely because using very long attention spans introduces memory into a system that is intended to be memoryless, and we use memory primarily for detecting drift\.

![Refer to caption](https://arxiv.org/html/2606.23957v1/Figures/Repressilator_different_operator_size.png)
![Refer to caption](https://arxiv.org/html/2606.23957v1/x4.png)

Figure 9:Ablation studies on Koopman and AFT parameters\.Left: AFT robustness vs\. Koopman with different operator sizes on Repressilator\. DenseKKachieves the best accuracy; constrained forms need larger widths for parity\. Right: AFT with different context lengths\. Small context length enable learning temporal changes while longer context might lead to noise updates\.

## Appendix HAdditional Training Details and Hyperparameters

Rollout loss and supervision\.Given an input chunk\(x0,…,xT\)\(x\_\{0\},\\ldots,x\_\{T\}\), we encodez0=φ​\(x0\)z\_\{0\}=\\varphi\(x\_\{0\}\)and roll forward withKK\(and AFT when enabled\), decodingx^t=φ−1​\(zt\)\\hat\{x\}\_\{t\}=\\varphi^\{\-1\}\(z\_\{t\}\)at each step\. We minimize the composite objective in equation[2](https://arxiv.org/html/2606.23957#S2.E2):ℒrecon\\mathcal\{L\}\_\{\\mathrm\{recon\}\}enforces autoencoder fidelity,ℒlin\\mathcal\{L\}\_\{\\mathrm\{lin\}\}encourages linear evolutionzi≈Ki​z0z\_\{i\}\\approx K^\{i\}z\_\{0\}in latent space,ℒpred\\mathcal\{L\}\_\{\\mathrm\{pred\}\}supervises decoded trajectories, andℒunitary\\mathcal\{L\}\_\{\\mathrm\{unitary\}\}regularizesKK\. We use the full\-horizon weighting \(no temporal discount\) to emphasize long\-range accuracy\.

Optimization and schedules\.We train with AdamW \(initial learning rate10−310^\{\-3\}\), a step scheduler \(epochs 30/60/90, factor 0\.8\), batch size 128, and early stopping on validation MSE\. For systems with chaotic or stiff transients, we use shorter prediction horizons during training \(Table[9](https://arxiv.org/html/2606.23957#A8.T9)\) for stability; inference uses the full trajectory length, and loss weights follow Table[9](https://arxiv.org/html/2606.23957#A8.T9)\(α1,α2\\alpha\_\{1\},\\alpha\_\{2\}per system\) and we use full\-horizon weighting in equation[2](https://arxiv.org/html/2606.23957#S2.E2)without temporal discount\. The AFT block uses a causal position\-only bias with a learnedT×TT\\times Tmatrix; multi\-head attention baselines use identical bottleneckddand comparable per\-head key/value sizes\. Unless otherwise stated, the AFT context isT=10T\{=\}10andKKis dense\. We report means over multiple random initial conditions;95%95\\%CIs are shown on MCAE curves\. Re\-encoding is disabled during training and enabled only at inference \(Alg\.[2](https://arxiv.org/html/2606.23957#algorithm2)\)\.

Network Architecture\.We employ a symmetric autoencoder architecture with encoder and decoder networks each containing 2\-4 hidden layers of equal width\. We use Leaky ReLU activation functions after each hidden layer except the pre\-bottleneck layer, which uses linear activation\. The bottleneck dimension was initially determined from repressilator experiments and fixed at 100 dimensions across all subsequent models to ensure consistent comparison between the Koopman Autoencoder \(KAE\) and attention\-augmented variants\. This standardized architecture allows us to focus on comparing the prediction capabilities between the Koopman Autoencoder \(KAE\) and our attention\-augmented variant\.

We employed a consistent architectural framework across all dynamical systems, as detailed in Table[8](https://arxiv.org/html/2606.23957#A8.T8)\. Modifications to this baseline architecture were implemented only when performance proved inadequate, with adjustments confined to operator dimensionality \(bottleneck width\) or the depth of hidden layers\. The selection of 2–4 hidden layers was informed by preliminary experiments demonstrating that increased network depth yielded marginal performance gains while substantially elevating training instability for the dynamical systems we tested\. However, this architectural choice may not generalize to dynamical systems with more complex dynamics or higher\-dimensional input spaces, where deeper networks could prove beneficial\.

Table 8:Architectural parameters of the models\. Values are fixed unless otherwise specified\.Koopman Operator Forms\.We tested several variations of the Koopman operator, including dense, tridiagonal, diagonal, and Jordan forms\. The dense form consistently outperformed the alternatives\. This might be due to the additional constraints imposed by other forms, such as sparsity, block structure, or independence assumptions, which appear to limit representational capacity\. Additionally, achieving complete feature disentanglement requires a larger operator size\. The dense form provides maximum representational flexibility, which motivated its use throughout our experiments\.

Data Pipeline\.We divided the data into 80% training, 10% validation, and 10% testing\. Model inputs for training consist of either complete trajectories or trajectory chunks, where the chunk length equals the prediction horizon, as shown in Figure[10](https://arxiv.org/html/2606.23957#A8.F10)\.

![Refer to caption](https://arxiv.org/html/2606.23957v1/Figures/Trajectory_Segmentation.png)Figure 10:Trajectory segmentation for trainingFor complex dynamical systems exhibiting chaotic behavior, switching dynamics, or continuous spectra, we employ shorter prediction lengths during training, as this approach yields better performance and more stable training dynamics\. The model unrolls predictions from the initial conditionx0x\_\{0\}across the specified prediction horizon, computing both latent space predictions and their corresponding observation space reconstructions for loss evaluation\. For GRU and Transformer training, the models require contextual information to learn effectively; therefore, instead of using only the initial conditionX0X\_\{0\}to predictX1,…,XTX\_\{1\},\\ldots,X\_\{T\}, we useX0,…,XcX\_\{0\},\\ldots,X\_\{c\}to predictXc\+1,…,XTX\_\{c\+1\},\\ldots,X\_\{T\}\. System\-specific dataset and training settings—including sampling intervalΔ​t\\Delta t, number of trajectories,TpredT\_\{\\text\{pred\}\}, and total trajectory length—are summarized in Table[9](https://arxiv.org/html/2606.23957#A8.T9)\.

Table 9:System\-specific training and dataset parameters\. Learning rate is fixed at1×10−31\\times 10^\{\-3\}\.