NeurDuo-EEG: A Long-Sequence EEG Foundation Model with Persistent State and Explicit Memory

# NeurDuo-EEG: A Long-Sequence EEG Foundation Model with Persistent State and Explicit Memory
Source: [https://arxiv.org/html/2609.38587](https://arxiv.org/html/2609.38587)
Yifan Wang1Haiping Liu111footnotemark:1Yang Cui1Wenhao Cai1Shuhang Li2Xiaoyang Huang1Xianyang Liu3Jingyu Sun1Yizheng Sun1Cunhang Fan4Tianming Du5,6Jiancheng Yang5,6Zhenhong Li1Yunhao Zhang7,8Hongpeng Zhou1Jingyuan Sun11University of Manchester2ETH Zürich3Shanghai Jiao Tong University4Anhui University5ELLIS Institute Finland6Aalto University7Institute of Automation, Chinese Academy of Sciences8University of Chinese Academy of Sciences\{yifan\.wang, jingyuan\.sun\}@manchester\.ac\.uk††thanks:Equal contribution\.††thanks:Correspondence to[jingyuan\.sun@manchester\.ac\.uk](mailto:[email protected])\.

###### Abstract

Electroencephalography \(EEG\) is recorded continuously over hours, with relevant dynamics spanning timescales from milliseconds to hours\. Most EEG foundation models nevertheless process fixed windows independently, limiting their ability to capture information encoded in long\-timescale dynamics\. State\-space architectures enable persistent recurrent processing, but long\-range information remains implicitly compressed in recurrent states\. We present NeurDuo\-EEG, a causal EEG foundation model with channel\-resolved persistent memory\. NeurDuo\-EEG introduces multi\-timescale memory management with learned consolidation and selective retrieval, enabling persistent modelling of continuous EEG with fixed\-size state\. It is pre\-trained on 3,955 hours of EEG from 17 public datasets using multichannel autoregressive prediction of discrete spectral codes\. Across three short\-window and two long\-sequence downstream tasks, NeurDuo\-EEG achieves the best performance on four of five benchmarks, including all three short\-window tasks and seizure detection, where AUC\-PR improves from0\.2850\.285to0\.4710\.471over the strongest non\-NeurDuo baseline\. NeurDuo\-EEG also remains competitive on sleep staging and supports efficient streaming inference, with nearly constant per\-chunk latency as the available history grows to one hour\. Notably, the Small variant achieves this with only 4\.7M backbone parameters\. These results demonstrate the value of persistent, multi\-timescale modelling for both long\-sequence and short\-window EEG analysis\. Our code is available at[https://github\.com/YifaNNW/NeurDuo\-EEG](https://github.com/YifaNNW/NeurDuo-EEG)\.

## 1Introduction

Electroencephalography \(EEG\) enables non\-invasive monitoring of brain activity over extended periods\. In applications such as sleep assessment and epilepsy monitoring, recordings often continue for hours or even days\([Berry et al\., 2012](https://arxiv.org/html/2609.38587#bib.bib19);[Tatum et al\., 2022](https://arxiv.org/html/2609.38587#bib.bib46)\)\. Within these long recordings, however, clinically relevant dynamics unfold over very different timescales\. Epileptiform spikes may last only tens of milliseconds\([Kane et al\., 2017](https://arxiv.org/html/2609.38587#bib.bib47)\), whereas seizure evolution, sleep\-stage transitions, and broader changes in brain state develop over minutes to hours\. At the same time, long recordings are highly redundant, with informative events occurring sparsely amid extended background activity\. Moreover, EEG is inherently multichannel, with electrodes sampling activity from different scalp locations and potentially exhibiting distinct temporal dynamics\. Modelling continuous EEG therefore requiresshort\-term sensitivity,long\-term retention,selective access to relevant past information, and preservation ofchannel\-specific temporal structure\.

Most existing EEG foundation models follow a window\-based paradigm, where self\-supervised pre\-training is performed on fixed\-length segments and each window is subsequently processed independently\([Jiang et al\., 2024](https://arxiv.org/html/2609.38587#bib.bib2);[Wang et al\., 2024](https://arxiv.org/html/2609.38587#bib.bib9);[Wang et al\., 2025](https://arxiv.org/html/2609.38587#bib.bib3);[Kuruppu et al\., 2026](https://arxiv.org/html/2609.38587#bib.bib45)\)\. This prevents the backbone itself from carrying temporal state across window boundaries\. Extending the window only moves this boundary outward: more history becomes available, but without a mechanism to selectively preserve relevant information, important events may be increasingly buried in redundant context\. Moreover, longer windows increase computational and memory costs\.

Selective state\-space models \(SSMs\) support linear\-time sequence modelling\([Gu and Dao, 2023](https://arxiv.org/html/2609.38587#bib.bib4)\), and early Mamba\-based EEG models primarily exploited this efficiency within a window\-based paradigm\([Gui et al\., 2024](https://arxiv.org/html/2609.38587#bib.bib5);[Tegon et al\., 2025](https://arxiv.org/html/2609.38587#bib.bib7)\)\. However, temporal dependencies across successive windows remained unmodelled\. More recently, CaMBRAIN\([Durgam et al\., 2026](https://arxiv.org/html/2609.38587#bib.bib8)\)took a further step toward continuous EEG modelling by embedding each multichannel EEG patch into a shared token and recurrently modelling the resulting temporal stream with persistent Mamba states\. However, a single recurrent state struggles to preserve both short\-term responsiveness and long\-term retention\. Without explicit retrieval, relevant past representations become difficult to recover once compressed into the evolving state\. In addition, fusing multiple channels into a single token may obscure channel\-specific temporal dynamics\.

To address these challenges, we introduce NeurDuo\-EEG \(hereafter referred to as NeurDuo\), a causal EEG foundation model with channel\-resolved, structured persistent memory\. NeurDuo maintains fast per\-channel recurrent streams to capture short\-term dynamics while preserving channel\-specific temporal structure, with interleaved cross\-channel attention enabling interaction between electrodes\. A learned writer periodically consolidates recent fast representations into compact memory tokens, which update a slower recurrent stream for long\-term retention and populate a bounded memory bank for selective content\-based retrieval\. During pre\-training, a frozen spectral tokenizer provides discrete per\-channel targets, and NeurDuo is trained autoregressively to predict the next spectral code for all channels in parallel\. We evaluate NeurDuo against six EEG foundation models and three classical methods across three short\-window and two long\-sequence downstream tasks\. NeurDuo achieves the best performance on four of five benchmarks, including all three short\-window tasks, and improves CHB\-MIT seizure\-detection AUC\-PR from0\.2850\.285for the strongest non\-NeurDuo baseline to0\.4710\.471\. Importantly, NeurDuo maintains nearly constant per\-chunk inference latency as the available history grows from3030s to11h, enabling efficient streaming over long recordings\. Overall, NeurDuo establishes a unified framework for modelling continuous multichannel EEG through channel\-resolved causal processing and structured memory across temporal scales, while supporting efficient streaming inference over long recordings\.

## 2Related Work

##### Window\-based EEG foundation models\.

Existing EEG foundation models differ in how they tokenize the signal, what they learn during pre\-training and how they cope with heterogeneous montages\([Yang et al\., 2023](https://arxiv.org/html/2609.38587#bib.bib1);[Wang et al\., 2025](https://arxiv.org/html/2609.38587#bib.bib3);[Wang et al\., 2024](https://arxiv.org/html/2609.38587#bib.bib9);[Yang et al\., 2026](https://arxiv.org/html/2609.38587#bib.bib14);[El Ouahidi et al\., 2026](https://arxiv.org/html/2609.38587#bib.bib10);[Döner et al\., 2026](https://arxiv.org/html/2609.38587#bib.bib11);[Zhou et al\., 2026a](https://arxiv.org/html/2609.38587#bib.bib12)\), yet they share one design: each takes a bounded input segment and computes its representation from that segment alone\. LaBraM, for example, predicts masked vector\-quantized codes from context on both sides within the window\([Jiang et al\., 2024](https://arxiv.org/html/2609.38587#bib.bib2)\)\. These backbones encode bounded windows without carrying recurrent state between them\. NeurDuo instead reads the recording causally, carrying a fixed\-size state across steps, so each new sample has constant cost and context extends beyond window boundaries\.

##### State\-space models for EEG\.

State\-space models such as Mamba process sequences in linear time through an input\-dependent recurrent state, which makes them natural candidates for causal, streaming inference\([Gu and Dao, 2023](https://arxiv.org/html/2609.38587#bib.bib4)\)\. EEG models built on Mamba nonetheless still encode bounded segments, mostly with bidirectional backbones\([Gui et al\., 2024](https://arxiv.org/html/2609.38587#bib.bib5);[Tegon et al\., 2025](https://arxiv.org/html/2609.38587#bib.bib7);[Hong et al\., 2025](https://arxiv.org/html/2609.38587#bib.bib6)\)\. CaMBRAIN is the exception, carrying a causal recurrent state across successive EEG patches\([Durgam et al\., 2026](https://arxiv.org/html/2609.38587#bib.bib8)\), but it merges all channels into one representation before the recurrence and keeps long\-range information only implicitly in that state\. This limits channel\-resolved temporal modelling and explicit access to past context\. NeurDuo instead keeps a recurrent stream for each channel and assigns short\-term processing, long\-term retention and explicit retrieval to a fast state, a slow state and a content\-addressable memory, respectively\.

## 3Methods

Pre\-training has two stages: we first train and freeze a channel\-independent spectral tokenizer, then train NeurDuo to predict the next spectral code for each channel from raw multichannel EEG using a stateful multi\-timescale backbone \(Figure[1](https://arxiv.org/html/2609.38587#S3.F1)\)\.

![Refer to caption](https://arxiv.org/html/2609.38587v1/framework1.png)Figure 1:Overview of NeurDuo and its two\-stage pre\-training framework\.A frozen spectral tokenizer provides discrete per\-channel targets\. The backbone processes coordinate\-aware EEG streams through fast and slow recurrent pathways, consolidates recent states into bounded memory, and retrieves relevant past information for per\-channel next\-token prediction\.### 3\.1Spectral tokenizer

All EEG recordings are resampled to256256Hz and divided into non\-overlapping0\.50\.5\-s chunksxt,c∈ℝLx\_\{t,c\}\\in\\mathbb\{R\}^\{L\}, withL=128L=128\. For each chunk, we compute its one\-sided frequency spectrum using the fast Fourier transform \(FFT\), yieldingF=65F=65bins, and standardize its magnitude and phase separately along the frequency axis:

ϕt,c=\[std\(\|ℱxt,c\|\)∥std\(∠ℱxt,c\)\]∈ℝ2​F\.\\phi\_\{t,c\}=\\left\[\\operatorname\{std\}\\\!\\left\(\\left\|\\mathcal\{F\}x\_\{t,c\}\\right\|\\right\)\\;\\\|\\;\\operatorname\{std\}\\\!\\left\(\\angle\\mathcal\{F\}x\_\{t,c\}\\right\)\\right\]\\in\\mathbb\{R\}^\{2F\}\.\(1\)A two\-block pre\-norm residual MLP independently maps each channel’s spectral representation togt,c=E⁡\(ϕt,c\)g\_\{t,c\}=E\(\\phi\_\{t,c\}\), without cross\-channel interaction or electrode identity\. Encoder outputs and theV=2048V=2048codebook vectors areℓ2\\ell\_\{2\}\-normalized, and the discrete code is assigned by cosine similarity,yt,c=arg⁡maxv∈\{1,…,V\}​g¯t,c⊤​e¯vy\_\{t,c\}=\\arg\\max\_\{v\\in\\\{1,\\ldots,V\\\}\}\\bar\{g\}\_\{t,c\}^\{\\top\}\\bar\{e\}\_\{v\}\. std denotes frequency\-wise z\-score normalisation\.

The codebook is updated by exponential moving averages\. A lightweight decoder reconstructs the standardized magnitude and phase spectra from the selected codes\. The tokenizer is trained with

ℒtok=ℒamp\+ℒphase\+β​ℒcommit,ℒcommit=‖sg⁡\(e¯yt,c\)−g¯t,c‖22\.\\mathcal\{L\}\_\{\\mathrm\{tok\}\}=\\mathcal\{L\}\_\{\\mathrm\{amp\}\}\+\\mathcal\{L\}\_\{\\mathrm\{phase\}\}\+\\beta\\mathcal\{L\}\_\{\\mathrm\{commit\}\},\\qquad\\mathcal\{L\}\_\{\\mathrm\{commit\}\}=\\left\\\|\\operatorname\{sg\}\\\!\\left\(\\bar\{e\}\_\{y\_\{t,c\}\}\\right\)\-\\bar\{g\}\_\{t,c\}\\right\\\|\_\{2\}^\{2\}\.\(2\)ℒamp\\mathcal\{L\}\_\{\\mathrm\{amp\}\}andℒphase\\mathcal\{L\}\_\{\\mathrm\{phase\}\}are MSE reconstruction losses, andsg\\operatorname\{sg\}denotes stop\-gradient\. The trained tokenizer is then frozen to generate targetsyt,cy\_\{t,c\}for NeurDuo pre\-training\.

### 3\.2NeurDuo Architecture

NeurDuo consists of a coordinate\-aware channel encoder that preserves electrode\-specific representations, followed by a multi\-timescale stateful backbone\.

#### 3\.2\.1Coordinate\-Aware Channel Encoding

To obtain compact input tokens while preserving channel\-specific information, we encode each electrode independently within each temporal chunk and retain the channel axis throughout\.

##### Signal patch encoder\.

Each channel chunkxt,cx\_\{t,c\}is independently processed by three parallel two\-layer temporal convolutional branches with kernel widths\{3,7,15\}\\\{3,7,15\\\}, producing multi\-scale temporal features\. The branch outputs are concatenated and fused by a pointwise convolution to obtainft,c∈ℝH×Lf\_\{t,c\}\\in\\mathbb\{R\}^\{H\\times L\}, whereHHis the frontend hidden width\. These features are attention\-pooled across theLLtemporal positions using a learned scoring vector𝐰∈ℝH\\mathbf\{w\}\\in\\mathbb\{R\}^\{H\}and projected to the fast\-stream widthdfd\_\{f\}byWp∈ℝdf×HW\_\{p\}\\in\\mathbb\{R\}^\{d\_\{f\}\\times H\}:

at,c=softmax⁡\(w⊤​ft,c\),f¯t,c=ft,c​at,c,u~t,c=Wp​f¯t,c\.a\_\{t,c\}=\\operatorname\{softmax\}\\\!\\left\(w^\{\\top\}f\_\{t,c\}\\right\),\\qquad\\bar\{f\}\_\{t,c\}=f\_\{t,c\}a\_\{t,c\},\\qquad\\tilde\{u\}\_\{t,c\}=W\_\{p\}\\bar\{f\}\_\{t,c\}\.\(3\)

##### Electrode identity\.

To provide each channel\-local token with a montage\-independent electrode identity, we encode the physical electrode coordinatespc∈ℝ3p\_\{c\}\\in\\mathbb\{R\}^\{3\}together with the reference schemeρc\\rho\_\{c\}, using a learned embeddingErefE\_\{\\mathrm\{ref\}\}for the latter\. The resulting metadata representation is added to the signal token as

ec=MLP⁡\(pc\)\+Eref​\[ρc\]∈ℝdf,ut,c=LN⁡\(u~t,c\+ec\)∈ℝdf\.e\_\{c\}=\\operatorname\{MLP\}\(p\_\{c\}\)\+E\_\{\\mathrm\{ref\}\}\[\\rho\_\{c\}\]\\in\\mathbb\{R\}^\{d\_\{f\}\},\\qquad u\_\{t,c\}=\\operatorname\{LN\}\\\!\\left\(\\tilde\{u\}\_\{t,c\}\+e\_\{c\}\\right\)\\in\\mathbb\{R\}^\{d\_\{f\}\}\.\(4\)At each temporal step, the resultingCCchannel\-specific tokens\{ut,c\}c=1C\\\{u\_\{t,c\}\\\}\_\{c=1\}^\{C\}are retained as parallel streams and serve as inputs to the backbone without channel pooling\.

#### 3\.2\.2Multi\-Timescale Stateful Backbone

The backbone maintains information at complementary temporal scales\. A fast recurrent stream processes channel\-specific tokens over time, with interleaved channel attention for cross\-channel interaction\. A learned writer periodically compresses recent fast representations into memory tokens, which update a slower recurrent stream and populate a bounded bank for content\-based retrieval\.

##### Fast stream\.

The fast stream models short\-timescale dynamics at each0\.50\.5\-s step usingLfL\_\{f\}pre\-norm Mamba blocks of widthdfd\_\{f\}, applied independently along the temporal dimension of each channel stream\. Leth\(ℓ\):,ch^\{\(\\ell\)\}\_\{:,c\}denote the channel\-ccsequence after blockℓ\\ell\. The temporal update is:

h:,c\(ℓ\)=h:,c\(ℓ−1\)\+Mambaℓ\(f\)\(LN\(h:,c\(ℓ−1\)\)\),ht,c\(0\)=ut,c\.h^\{\(\\ell\)\}\_\{:,c\}=h^\{\(\\ell\-1\)\}\_\{:,c\}\+\\operatorname\{Mamba\}^\{\(f\)\}\_\{\\ell\}\\left\(\\operatorname\{LN\}\\\!\\left\(h^\{\(\\ell\-1\)\}\_\{:,c\}\\right\)\\right\),\\qquad h^\{\(0\)\}\_\{t,c\}=u\_\{t,c\}\.\(5\)
To enable cross\-channel interaction while preserving channel\-specific temporal trajectories, we apply multi\-head self\-attention across theCCchannel representations after every second Mamba block\. At each fixed temporal steptt,

ht,:\(ℓ\)←ht,:\(ℓ\)\+MHA\(LN\(ht,:\(ℓ\)\)\),ℓ≡0\(mod2\)\.h^\{\(\\ell\)\}\_\{t,:\}\\leftarrow h^\{\(\\ell\)\}\_\{t,:\}\+\\operatorname\{MHA\}\\left\(\\operatorname\{LN\}\\\!\\left\(h^\{\(\\ell\)\}\_\{t,:\}\\right\)\\right\),\\qquad\\ell\\equiv 0\\pmod\{2\}\.\(6\)
The Mamba blocks model temporal dependencies within each channel, while channel attention integrates information across electrodes at the same temporal step\. We denote the final contextualized fast representation byht,c=ht,c\(Lf\)∈ℝdfh\_\{t,c\}=h^\{\(L\_\{f\}\)\}\_\{t,c\}\\in\\mathbb\{R\}^\{d\_\{f\}\}\.

##### Memory writer\.

To bridge the fast stream with the slow stream and memory bank, the memory writer periodically consolidates recent fast representations into memory tokens\. EveryN=4N=4steps \(22s\), it summarizes the most recentw=8w=8representations \(44s\) of each channel\. Lettkt\_\{k\}denote the step of thekk\-th write\. Using a learnable scoring vectorvv, the writer attention\-pools these representations and projects the result to the slow\-stream widthdsd\_\{s\}:

αj=softmaxj⁡\(v⊤​htk−w\+j,c\),mk,c=Wm​LN⁡\(∑j=1wαj​htk−w\+j,c\)∈ℝds\.\\alpha\_\{j\}=\\operatorname\{softmax\}\_\{j\}\\left\(v^\{\\top\}h\_\{t\_\{k\}\-w\+j,c\}\\right\),\\qquad m\_\{k,c\}=W\_\{m\}\\operatorname\{LN\}\\left\(\\sum\_\{j=1\}^\{w\}\\alpha\_\{j\}h\_\{t\_\{k\}\-w\+j,c\}\\right\)\\in\\mathbb\{R\}^\{d\_\{s\}\}\.\(7\)

##### Slow stream\.

The memory tokensmk,cm\_\{k,c\}form a lower\-rate sequence for each channel, which is processed byLsL\_\{s\}pre\-norm Mamba blocks of widthdsd\_\{s\}to capture longer\-timescale temporal structure\. Lets\(ℓ\):,cs^\{\(\\ell\)\}\_\{:,c\}denote the channel\-ccsequence after slow\-stream blockℓ\\ell:

s:,c\(ℓ\)=s:,c\(ℓ−1\)\+Mambaℓ\(s\)\(LN\(s:,c\(ℓ−1\)\)\),sk,c\(0\)=mk,c\.s^\{\(\\ell\)\}\_\{:,c\}=s^\{\(\\ell\-1\)\}\_\{:,c\}\+\\operatorname\{Mamba\}^\{\(s\)\}\_\{\\ell\}\\left\(\\operatorname\{LN\}\\\!\\left\(s^\{\(\\ell\-1\)\}\_\{:,c\}\\right\)\\right\),\\qquad s^\{\(0\)\}\_\{k,c\}=m\_\{k,c\}\.\(8\)The final slow representation at writekkissk,c=sk,c\(Ls\)∈ℝdss\_\{k,c\}=s^\{\(L\_\{s\}\)\}\_\{k,c\}\\in\\mathbb\{R\}^\{d\_\{s\}\}\. Since the slow stream advances only when a new memory token is produced, it updates at1/N1/Nof the fast\-stream rate\.

##### Memory bank\.

In parallel with the slow stream, each memory tokenmk,cm\_\{k,c\}is written to a bounded channel\-specific memory bank with capacityB=8B=8\. New tokens are appended in first\-in\-first\-out order, evicting the oldest entry when full\.

##### Content\-addressable retrieval\.

To selectively access past information relevant to the current signal, the fast representationht,ch\_\{t,c\}queries the memory bank of the same channel\. Letbj,cb\_\{j,c\}denote thejj\-th available memory entry\. We obtain the query, key, and value representations asqt,c=Wq​ht,cq\_\{t,c\}=W\_\{q\}h\_\{t,c\},kj,c=Wk​bj,ck\_\{j,c\}=W\_\{k\}b\_\{j,c\}, andvj,c=Wv​bj,cv\_\{j,c\}=W\_\{v\}b\_\{j,c\}, and compute content\-based attention over the bank:

πt,c,j=softmaxj⁡\(qt,c⊤​kj,cds\),r~t,c=∑jπt,c,j​vj,c\.\\pi\_\{t,c,j\}=\\operatorname\{softmax\}\_\{j\}\\left\(\\frac\{q\_\{t,c\}^\{\\top\}k\_\{j,c\}\}\{\\sqrt\{d\_\{s\}\}\}\\right\),\\qquad\\tilde\{r\}\_\{t,c\}=\\sum\_\{j\}\\pi\_\{t,c,j\}v\_\{j,c\}\.\(9\)
A learned gate then controls how much of the retrieved content is incorporated:

rt,c=σ\(Wg\[ht,c∥r~t,c\]\)⊙r~t,c∈ℝds\.r\_\{t,c\}=\\sigma\\left\(W\_\{g\}\\left\[h\_\{t,c\}\\\|\\tilde\{r\}\_\{t,c\}\\right\]\\right\)\\odot\\tilde\{r\}\_\{t,c\}\\in\\mathbb\{R\}^\{d\_\{s\}\}\.\(10\)Attention is restricted to memory entries available at steptt, preserving causality\.

##### NeurDuo output\.

To combine recurrent long\-range context with explicitly retrieved memory, we add the retrieval representation to the current slow representation and concatenate the result with the fast representation\. We useκ⁡\(t\)\\kappa\(t\)to index the most recent memory write available at steptt, withsκ⁡\(t\),cs\_\{\\kappa\(t\),c\}denoting the corresponding slow representation\. The backbone output is

zt,c=\[ht,c∥\(sκ⁡\(t\),c\+rt,c\)\]∈ℝdz,dz=df\+ds\.z\_\{t,c\}=\\left\[h\_\{t,c\}\\;\\\|\\;\\left\(s\_\{\\kappa\(t\),c\}\+r\_\{t,c\}\\right\)\\right\]\\in\\mathbb\{R\}^\{d\_\{z\}\},\\qquad d\_\{z\}=d\_\{f\}\+d\_\{s\}\.\(11\)

### 3\.3MultiChannel Autoregressive Pre\-training

To preserve multichannel structure and train NeurDuo causally, we use an autoregressive objective that predicts the next spectral code for all channels in parallel\. The frozen spectral tokenizer maps each next chunkxt\+1,cx\_\{t\+1,c\}to its discrete codeyt\+1,c∈\{1,…,V\}y\_\{t\+1,c\}\\in\\\{1,\\ldots,V\\\}, which serves as the prediction target\. A prediction head shared across channels mapszt,cz\_\{t,c\}to a distribution over theVVspectral codes, and the model is trained with

ℒAR=−1\|𝒫\|∑\(t,c\)∈𝒫logpθ\(yt\+1,c∣zt,c\),pθ\(⋅∣zt,c\)=softmax\(MLP\(zt,c\)\),\\mathcal\{L\}\_\{\\mathrm\{AR\}\}=\-\\frac\{1\}\{\|\\mathcal\{P\}\|\}\\sum\_\{\(t,c\)\\in\\mathcal\{P\}\}\\log p\_\{\\theta\}\\left\(y\_\{t\+1,c\}\\mid z\_\{t,c\}\\right\),\\qquad p\_\{\\theta\}\(\\cdot\\mid z\_\{t,c\}\)=\\operatorname\{softmax\}\\left\(\\operatorname\{MLP\}\(z\_\{t,c\}\)\\right\),\(12\)where𝒫\\mathcal\{P\}denotes the set of valid non\-padded channel–time pairs\. The prediction head and tokenizer are used only during pre\-training and discarded downstream\.

## 4Experimental Setup

### 4\.1Pre\-training setup

##### Corpus\.

For pre\-training, we curate seventeen public EEG datasets comprising23,86323\{,\}863recordings and3,9553\{,\}955hours of EEG, spanning clinical monitoring, overnight polysomnography, and task\-based laboratory recordings, with montages ranging from six to128128electrodes\. Because NeurDuo identifies channels by electrode coordinates rather than fixed channel indices, recordings retain their native montages without interpolation to a common channel set\. Signals are band\-pass and notch filtered, resampled to256256Hz, and segmented into3232s sequences with50%50\\%overlap for backbone pre\-training\. The corpus is split by subject into22,78022\{,\}780training and1,0831\{,\}083validation recordings\. Dataset\-level composition and preprocessing details are provided in Appendix[A\.1](https://arxiv.org/html/2609.38587#A1.SS1)\.

##### Optimisation\.

The spectral tokenizer and NeurDuo are both optimized with AdamW and cosine learning\-rate decay\. NeurDuo is trained from scratch on the full corpus for 16 epochs with an effective batch size of 128 sequences using bfloat16 mixed precision on 32 GPUs\. Full optimisation hyperparameters for the tokenizer and NeurDuo are provided in Appendix[A\.1](https://arxiv.org/html/2609.38587#A1.SS1)\.

##### Model family\.

Table 1:NeurDuo model configurations\.ModelHHdfd\_\{f\}LfL\_\{f\}dsd\_\{s\}LsL\_\{s\}ParamsSmall128256619234\.70MBase19248010320524\.31MLarge25667215448767\.83MThe Small, Base, and Large variants, all pre\-trained on 32 A100 GPUs, differ only in model scale: the signal patch encoder widthHHand the fast\- and slow\-stream widths and depths\(df,Lf\)\(d\_\{f\},L\_\{f\}\)and\(ds,Ls\)\(d\_\{s\},L\_\{s\}\)increase together\.

### 4\.2Downstream Evaluation

##### Benchmarks and baselines\.

We evaluate NeurDuo on three window tasks, comprising emotion recognition \(FACED\)\([Chen et al\., 2023](https://arxiv.org/html/2609.38587#bib.bib13)\), error\-related negativity detection \(KaggleERN\)\([Mattout et al\., 2014](https://arxiv.org/html/2609.38587#bib.bib16);[Margaux et al\., 2012](https://arxiv.org/html/2609.38587#bib.bib15)\), and vigilance regression \(SEED\-VIG\)\([Zheng and Lu, 2017](https://arxiv.org/html/2609.38587#bib.bib20)\), and two sequence tasks, comprising sleep staging \(Sleep\-EDF\)\([Kemp et al\., 2000](https://arxiv.org/html/2609.38587#bib.bib17);[Goldberger et al\., 2000](https://arxiv.org/html/2609.38587#bib.bib18)\)and seizure detection \(CHB\-MIT\)\([Goldberger et al\., 2000](https://arxiv.org/html/2609.38587#bib.bib18);[Shoeb, 2009](https://arxiv.org/html/2609.38587#bib.bib44)\)\. None of these datasets appears in our pre\-training corpus\. We compare against six pre\-trained EEG models \(BIOT\([Yang et al\., 2023](https://arxiv.org/html/2609.38587#bib.bib1)\), CBraMod\([Wang et al\., 2025](https://arxiv.org/html/2609.38587#bib.bib3)\), LaBraM\-Base\([Jiang et al\., 2024](https://arxiv.org/html/2609.38587#bib.bib2)\), EEGPT\-Large\([Wang et al\., 2024](https://arxiv.org/html/2609.38587#bib.bib9)\), ST\-EEGFormer\-S\([Yang et al\., 2026](https://arxiv.org/html/2609.38587#bib.bib14)\), and REVE\-Base\([El Ouahidi et al\., 2026](https://arxiv.org/html/2609.38587#bib.bib10)\)\) and three classical pipelines \(BP\-GBDT\([Vallat and Walker, 2021](https://arxiv.org/html/2609.38587#bib.bib22);[Kastrati et al\., 2025](https://arxiv.org/html/2609.38587#bib.bib23)\), FBCov\-TS\-Lin\([Jayaram and Barachant, 2018](https://arxiv.org/html/2609.38587#bib.bib24);[Sabbagh et al\., 2019](https://arxiv.org/html/2609.38587#bib.bib25)\), and ERP\-Lin\([Yang et al\., 2026](https://arxiv.org/html/2609.38587#bib.bib14)\)\)\. Dataset and baseline details are provided in Appendices[C](https://arxiv.org/html/2609.38587#A3)and[D\.1](https://arxiv.org/html/2609.38587#A4.SS1), respectively\. CaMBRAIN\([Durgam et al\., 2026](https://arxiv.org/html/2609.38587#bib.bib8)\)is omitted because pre\-trained weights were unavailable\.

##### Evaluation protocol\.

All foundation models are fine\-tuned end\-to\-end with a common optimisation recipe and a shared readout with matched capacity\. By default, sequence samples contain twenty consecutive3030s units \(1010min\), processed by the same two\-layer causal Mamba context head\. Baseline backbones encode units independently; NeurDuo carries its fast, slow, and memory\-bank states across units\. Protocol details and the state\-reset comparison are in Table[6](https://arxiv.org/html/2609.38587#S5.T6)and Appendix[D\.2](https://arxiv.org/html/2609.38587#A4.SS2)\.

##### Metrics and reporting\.

We report the mean±\\pmstandard deviation over five seeds for FACED and SEED\-VIG, four prescribed training folds for KaggleERN, and five subject\-disjoint folds for Sleep\-EDF and five case\-level folds for CHB\-MIT, where one subject contributed two case folders \(Appendix[C\.5](https://arxiv.org/html/2609.38587#A3.SS5)\)\. Deterministic classical results on FACED and SEED\-VIG have no s\.d\.FP⋅h−1\\mathrm\{FP\}\\cdot\\mathrm\{h\}^\{\-1\}counts merged false alarms per recording hour at test\-selected thresholds targeting at least50%50\\%unit\-level sensitivity \(Appendix[C\.5](https://arxiv.org/html/2609.38587#A3.SS5)\), and is reported without s\.d\. Model size counts backbone parameters\.Boldandunderlinedscores mark the best and second\-best results per column; arrows indicate metric direction\.

## 5Experimental Results

### 5\.1Window tasks

NeurDuo leads all three short\-window benchmarks\.We first evaluate NeurDuo on three conventional short\-window EEG tasks spanning emotion recognition, error\-related negativity detection, and vigilance estimation\. NeurDuo achieves the strongest overall performance across all three tasks \(Table[2](https://arxiv.org/html/2609.38587#S5.T2)\)\. On FACED, NeurDuo\-Small already outperforms every baseline except the much larger REVE\-Base, while NeurDuo\-Base further attains the best balanced accuracy and Cohen’sκ\\kappa\. KaggleERN presents a different setting, where the classical ERP\-Lin baseline is particularly strong, achieving higher ROC\-AUC than every pre\-trained baseline; NeurDuo is the only model family to surpass it, with the Small model already achieving the best result\. On SEED\-VIG, foundation models consistently outperform the classical pipelines, yet NeurDuo again performs best, with NeurDuo\-Base improvingR2R^\{2\}from0\.2970\.297for the strongest baseline to0\.3640\.364\. These results establish NeurDuo as a strong short\-window EEG model across diverse downstream settings, with much of this capability already present in the4\.74\.7M\-parameter Small model\.

Table 2:Short\-window downstream performance\.†LaBraM was pre\-trained on KaggleERN; see Appendix[D\.1](https://arxiv.org/html/2609.38587#A4.SS1)\.MethodsModel SizeFACEDKaggleERNSEED\-VIGBal\. Acc\.↑\\uparrowCohen’sκ\\kappa↑\\uparrowROC\-AUC↑\\uparrowAUC\-PR↑\\uparrowPearson’srr↑\\uparrowR2R^\{2\}↑\\uparrowBP\-GBDT–0\.1730\.0700\.557±0\.0050\.557\\pm 0\.0050\.753±0\.0090\.753\\pm 0\.0090\.4450\.031FBCov\-TS\-Lin–0\.1850\.0840\.613±0\.0230\.613\\pm 0\.0230\.778±0\.0230\.778\\pm 0\.0230\.3440\.110ERP\-Lin–0\.2290\.1300\.623±0\.0190\.623\\pm 0\.0190\.782±0\.0120\.782\\pm 0\.0120\.020−0\.019\-0\.019BIOT3\.2M0\.364±0\.0160\.364\\pm 0\.0160\.283±0\.0170\.283\\pm 0\.0170\.496±0\.0300\.496\\pm 0\.0300\.708±0\.0230\.708\\pm 0\.0230\.535±0\.0650\.535\\pm 0\.0650\.179±0\.1360\.179\\pm 0\.136CBraMod4\.9M0\.537±0\.0080\.537\\pm 0\.0080\.476±0\.0100\.476\\pm 0\.0100\.571±0\.0150\.571\\pm 0\.0150\.755±0\.0090\.755\\pm 0\.0090\.529±0\.0380\.529\\pm 0\.0380\.196±0\.0510\.196\\pm 0\.051LaBraM†5\.8M0\.555±0\.0080\.555\\pm 0\.0080\.496±0\.0090\.496\\pm 0\.0090\.546±0\.0180\.546\\pm 0\.0180\.737±0\.0090\.737\\pm 0\.0090\.588±0\.0370\.588\\pm 0\.0370\.188±0\.1070\.188\\pm 0\.107EEGPT25\.3M0\.506±0\.0120\.506\\pm 0\.0120\.442±0\.0120\.442\\pm 0\.0120\.600±0\.0140\.600\\pm 0\.0140\.782±0\.0140\.782\\pm 0\.0140\.619±0\.0180\.619\\pm 0\.0180\.244±0\.0710\.244\\pm 0\.071ST\-EEGFormer\-S25\.4M0\.486±0\.0070\.486\\pm 0\.0070\.419±0\.0080\.419\\pm 0\.0080\.514±0\.0470\.514\\pm 0\.0470\.721±0\.0220\.721\\pm 0\.0220\.591±0\.0320\.591\\pm 0\.0320\.233±0\.0670\.233\\pm 0\.067REVE\-Base69\.2M0\.605±0\.0070\.605\\pm 0\.0070\.553±0\.0080\.553\\pm 0\.0080\.604±0\.0180\.604\\pm 0\.0180\.786±0\.0240\.786\\pm 0\.0240\.621±0\.0200\.621\\pm 0\.0200\.297±0\.0230\.297\\pm 0\.023NeurDuo \(Small\)4\.7M0\.589±0\.0050\.589\\pm 0\.0050\.534±0\.0060\.534\\pm 0\.0060\.636±0\.014\\mathbf\{0\.636\\pm 0\.014\}0\.803±0\.019\\mathbf\{0\.803\\pm 0\.019\}0\.632±0\.0080\.632\\pm 0\.0080\.333±0\.005¯\\underline\{0\.333\\pm 0\.005\}NeurDuo \(Base\)24\.3M0\.610±0\.007\\mathbf\{0\.610\\pm 0\.007\}0\.558±0\.007\\mathbf\{0\.558\\pm 0\.007\}0\.621±0\.0240\.621\\pm 0\.0240\.785±0\.0180\.785\\pm 0\.0180\.665±0\.024\\mathbf\{0\.665\\pm 0\.024\}0\.364±0\.040\\mathbf\{0\.364\\pm 0\.040\}NeurDuo \(Large\)67\.8M0\.607±0\.008¯\\underline\{0\.607\\pm 0\.008\}0\.555±0\.009¯\\underline\{0\.555\\pm 0\.009\}0\.632±0\.010¯\\underline\{0\.632\\pm 0\.010\}0\.790±0\.020¯\\underline\{0\.790\\pm 0\.020\}0\.647±0\.015¯\\underline\{0\.647\\pm 0\.015\}0\.294±0\.0140\.294\\pm 0\.014
### 5\.2Sequence tasks

Table 3:Long\-sequence downstream performance\.CHB\-MIT is evaluated using either all1515channels or only T7 and P8\.†BIOT was pre\-trained on SHHS and CHB\-MIT; see Appendix[D\.1](https://arxiv.org/html/2609.38587#A4.SS1)\.MethodsModel SizeSleep\-EDFCHB\-MIT \(15 ch\)CHB\-MIT \(2 ch\)Macro\-F1↑\\uparrowCohen’sκ\\kappa↑\\uparrowAUC\-PR↑\\uparrowFP⋅\\cdoth\-1↓\\downarrowAUC\-PR↑\\uparrowFP⋅\\cdoth\-1↓\\downarrowBP\-GBDT–0\.710±0\.0130\.710\\pm 0\.0130\.695±0\.0210\.695\\pm 0\.0210\.209±0\.1090\.209\\pm 0\.1093\.343\.340\.208±0\.0570\.208\\pm 0\.0573\.283\.28FBCov\-TS\-Lin–0\.674±0\.0180\.674\\pm 0\.0180\.666±0\.0190\.666\\pm 0\.0190\.285±0\.1360\.285\\pm 0\.1362\.142\.140\.224±0\.1240\.224\\pm 0\.1244\.124\.12ERP\-Lin–0\.306±0\.0440\.306\\pm 0\.0440\.255±0\.0310\.255\\pm 0\.0310\.006±0\.0030\.006\\pm 0\.00315\.4215\.420\.009±0\.0070\.009\\pm 0\.00712\.6512\.65BIOT†3\.2M0\.731±0\.0170\.731\\pm 0\.0170\.733±0\.0170\.733\\pm 0\.0170\.208±0\.0590\.208\\pm 0\.0594\.734\.730\.124±0\.0940\.124\\pm 0\.0946\.126\.12CBraMod4\.9M0\.759±0\.009¯\\underline\{0\.759\\pm 0\.009\}0\.749±0\.0140\.749\\pm 0\.0140\.163±0\.0850\.163\\pm 0\.0854\.484\.480\.277±0\.1500\.277\\pm 0\.1503\.403\.40LaBraM5\.8M0\.766±0\.010\\mathbf\{0\.766\\pm 0\.010\}0\.758±0\.009\\mathbf\{0\.758\\pm 0\.009\}0\.154±0\.1290\.154\\pm 0\.12912\.7712\.770\.056±0\.0680\.056\\pm 0\.06813\.4113\.41EEGPT25\.3M0\.739±0\.0140\.739\\pm 0\.0140\.729±0\.0160\.729\\pm 0\.0160\.180±0\.0790\.180\\pm 0\.0799\.279\.270\.183±0\.0940\.183\\pm 0\.0947\.737\.73ST\-EEGFormer\-S25\.4M0\.715±0\.0240\.715\\pm 0\.0240\.722±0\.0300\.722\\pm 0\.0300\.034±0\.0450\.034\\pm 0\.04519\.7119\.710\.033±0\.0250\.033\\pm 0\.02518\.1418\.14REVE\-Base69\.2M0\.741±0\.0120\.741\\pm 0\.0120\.743±0\.0170\.743\\pm 0\.0170\.163±0\.0660\.163\\pm 0\.0667\.317\.310\.131±0\.1040\.131\\pm 0\.1047\.017\.01NeurDuo \(Small\)4\.7M0\.755±0\.0080\.755\\pm 0\.0080\.749±0\.0110\.749\\pm 0\.0110\.413±0\.174¯\\underline\{0\.413\\pm 0\.174\}0\.96\\mathbf\{0\.96\}0\.370±0\.1770\.370\\pm 0\.1771\.62\\mathbf\{1\.62\}NeurDuo \(Base\)24\.3M0\.746±0\.0170\.746\\pm 0\.0170\.743±0\.0120\.743\\pm 0\.0120\.392±0\.1040\.392\\pm 0\.1041\.341\.340\.374±0\.145¯\\underline\{0\.374\\pm 0\.145\}1\.64¯\\underline\{1\.64\}NeurDuo \(Large\)67\.8M0\.753±0\.0070\.753\\pm 0\.0070\.752±0\.009¯\\underline\{0\.752\\pm 0\.009\}0\.471±0\.101\\mathbf\{0\.471\\pm 0\.101\}1\.02¯\\underline\{1\.02\}0\.429±0\.106\\mathbf\{0\.429\\pm 0\.106\}3\.953\.95NeurDuo excels at seizure detection and remains competitive on sleep staging\.We next turn to the central evaluation setting of this work: long\-sequence EEG modelling\. We benchmark NeurDuo on Sleep\-EDF and CHB\-MIT, two sequence\-labelling tasks built from ordered EEG units\. Table[3](https://arxiv.org/html/2609.38587#S5.T3)summarizes the results\. On Sleep\-EDF, LaBraM achieves the best performance, while NeurDuo remains competitive and all foundation models fall within a relatively narrow range\. One possible reason is that Sleep\-EDF provides only two EEG derivations, which leaves little cross\-channel structure for NeurDuo’s channel\-parallel design to exploit\. In contrast, CHB\-MIT shows a much clearer separation between our model and the baselines\. Using the full 15\-channel montage, all NeurDuo variants outperform every baseline on both AUC\-PR and false alarms per hour\. NeurDuo\-Small already reaches an AUC\-PR of0\.4130\.413with0\.960\.96false alarms per hour at only4\.74\.7M parameters, compared with0\.2850\.285for the strongest non\-NeurDuo baseline, while NeurDuo\-Large further improves AUC\-PR to0\.4710\.471\.

NeurDuo retains its AUC\-PR advantage with only two channels\.To examine performance with fewer channels, we repeat the CHB\-MIT evaluation using only T7 and P8, with the same splits and the montage\-specific readout budgets in Table[16](https://arxiv.org/html/2609.38587#A4.T16)\. NeurDuo retains its AUC\-PR advantage: the strongest baseline reaches0\.2770\.277, compared with0\.3700\.370for Small and0\.4290\.429for Large\. Small and Base also give the lowest false\-alarm rates in this setting,1\.621\.62and1\.641\.64per hour\. All NeurDuo variants have lower AUC\-PR than with 15 channels, consistent with a benefit from richer spatial information, although the readout budget also changes\.

Longer context helps, with task and montage dependent gains\.We evaluate NeurDuo\-Small with context lengths from3030s to6060min using the training protocol of Appendix[D\.2\.4](https://arxiv.org/html/2609.38587#A4.SS2.SSS4)\(Results in Table[4](https://arxiv.org/html/2609.38587#S5.T4)\)\. Sleep\-EDF benefits overall from longer context, reaching its highest macro\-F1 at6060min\. On 15\-channel CHB\-MIT, AUC\-PR peaks at1010min and false alarms are lowest at3030min; both worsen at6060min\. With two channels, AUC\-PR is highest at6060min, while false alarms are lowest at3030min\. The montage moves where that optimum sits: with fifteen channels AUC\-PR is already past its best by3030min, whereas with two it is still rising at6060\. Longer causal context is therefore useful, with the preferred length depending on the task, the montage and the metric\.

Table 4:Effect of causal context length on NeurDuo \(Small\)\.LLdenotes the number of consecutive3030s units; all other evaluation settings remain fixed\.†L=1\{\}^\{\\dagger\}L=1provides no cross\-unit context\.ContextLLSleep\-EDFCHB\-MIT \(15 ch\)CHB\-MIT \(2 ch\)Macro\-F1↑\\uparrowCohen’sκ\\kappa↑\\uparrowAUC\-PR↑\\uparrowFP⋅\\cdoth\-1↓\\downarrowAUC\-PR↑\\uparrowFP⋅\\cdoth\-1↓\\downarrow3030s†10\.696±0\.0100\.696\\pm 0\.0100\.695±0\.0110\.695\\pm 0\.0110\.324±0\.0950\.324\\pm 0\.0952\.022\.020\.241±0\.1880\.241\\pm 0\.1887\.967\.9611min20\.707±0\.0090\.707\\pm 0\.0090\.702±0\.0100\.702\\pm 0\.0100\.322±0\.1530\.322\\pm 0\.1534\.124\.120\.290±0\.1530\.290\\pm 0\.1534\.054\.0522min40\.725±0\.0090\.725\\pm 0\.0090\.722±0\.0100\.722\\pm 0\.0100\.404±0\.1600\.404\\pm 0\.1605\.835\.830\.358±0\.1370\.358\\pm 0\.1371\.641\.6455min100\.744±0\.0150\.744\\pm 0\.0150\.742±0\.0050\.742\\pm 0\.0050\.331±0\.1280\.331\\pm 0\.1280\.800\.800\.375±0\.1560\.375\\pm 0\.1562\.182\.181010min200\.755±0\.008¯\\underline\{0\.755\\pm 0\.008\}0\.749±0\.0110\.749\\pm 0\.0110\.413±0\.174\\mathbf\{0\.413\\pm 0\.174\}0\.960\.960\.370±0\.1770\.370\\pm 0\.1771\.621\.623030min600\.753±0\.0160\.753\\pm 0\.0160\.752±0\.011¯\\underline\{0\.752\\pm 0\.011\}0\.406±0\.065¯\\underline\{0\.406\\pm 0\.065\}0\.36\\mathbf\{0\.36\}0\.381±0\.173¯\\underline\{0\.381\\pm 0\.173\}1\.07\\mathbf\{1\.07\}6060min1200\.759±0\.010\\mathbf\{0\.759\\pm 0\.010\}0\.755±0\.011\\mathbf\{0\.755\\pm 0\.011\}0\.379±0\.1080\.379\\pm 0\.1080\.56¯\\underline\{0\.56\}0\.382±0\.180\\mathbf\{0\.382\\pm 0\.180\}1\.13¯\\underline\{1\.13\}Figure 2:Inference latency on CHB\-MIT\.
### 5\.3Streaming Cost

Per\-chunk latency remains nearly constant as history grows\.To verify that NeurDuo can condition on long histories with bounded deployment cost, we measured inference latency as history grows from 30 s to 1 h and peak GPU memory at 1,800 s on two datasets\. For each baseline, inference re\-encodes the full history window, whereas NeurDuo processes only the next 0\.5\-s chunk after the history has been streamed\. As shown in Figure[2](https://arxiv.org/html/2609.38587#S5.F2), baseline latency increases with context length, while NeurDuo remains at4\.834\.83–8\.948\.94ms per chunk and requires only0\.050\.05–0\.370\.37GB of peak memory\. These results demonstrate that NeurDuo supports long\-context, real\-time EEG processing with nearly constant per\-chunk computation and a small memory footprint\. Full measurements are provided in Appendix[F\.2](https://arxiv.org/html/2609.38587#A6.SS2)\.

### 5\.4Ablation Experiments

We evaluate the contributions of NeurDuo’s architectural components on NeurDuo \(Small\)\. Table[5](https://arxiv.org/html/2609.38587#S5.T5)reports architectural ablations and a temporal\-order control, with all architectural variants re\-pre\-trained from scratch under the same settings\. Table[6](https://arxiv.org/html/2609.38587#S5.T6)isolates the contribution of persistent backbone state by comparing state carrying across successive units with resets at every 30\-s boundary\.

Table 5:Component ablations and temporal\-order control for NeurDuo \(Small\)\.Shuffling is not applicable to FACED, whose samples are independent windows\.VariantFACEDSleep\-EDFCHB\-MIT \(15 ch\)Bal\. Acc\.↑\\uparrowκ\\kappa↑\\uparrowMacro\-F1↑\\uparrowκ\\kappa↑\\uparrowAUC\-PR↑\\uparrowFP⋅\\cdoth\-1↓\\downarrowNeurDuo \(Small\)0\.589±0\.0050\.589\\pm 0\.0050\.534±0\.0060\.534\\pm 0\.0060\.755±0\.0080\.755\\pm 0\.0080\.749±0\.0110\.749\\pm 0\.0110\.413±0\.1740\.413\\pm 0\.1740\.960\.96w/o per\-channel tokens0\.460±0\.0110\.460\\pm 0\.0110\.391±0\.0120\.391\\pm 0\.0120\.745±0\.0110\.745\\pm 0\.0110\.739±0\.0080\.739\\pm 0\.0080\.412±0\.0960\.412\\pm 0\.0961\.101\.10w/o channel attention0\.523±0\.0090\.523\\pm 0\.0090\.460±0\.0110\.460\\pm 0\.0110\.753±0\.0160\.753\\pm 0\.0160\.750±0\.0090\.750\\pm 0\.0090\.340±0\.1250\.340\\pm 0\.1254\.244\.24w/o memory retrieval0\.506±0\.0120\.506\\pm 0\.0120\.442±0\.0130\.442\\pm 0\.0130\.744±0\.0120\.744\\pm 0\.0120\.736±0\.0100\.736\\pm 0\.0100\.389±0\.1120\.389\\pm 0\.1121\.081\.08w/o slow stream & memory0\.545±0\.0150\.545\\pm 0\.0150\.485±0\.0170\.485\\pm 0\.0170\.742±0\.0170\.742\\pm 0\.0170\.738±0\.0100\.738\\pm 0\.0100\.394±0\.1790\.394\\pm 0\.1790\.470\.47Mamba\-only0\.523±0\.0070\.523\\pm 0\.0070\.461±0\.0090\.461\\pm 0\.0090\.744±0\.0130\.744\\pm 0\.0130\.737±0\.0160\.737\\pm 0\.0160\.406±0\.1680\.406\\pm 0\.1684\.604\.60Shuffled unit order——0\.735±0\.0170\.735\\pm 0\.0170\.733±0\.0130\.733\\pm 0\.0130\.371±0\.1610\.371\\pm 0\.1613\.793\.79Channel\-specific tokens and channel attention provide complementary benefits\.Removing per\-channel tokens degrades performance across all three datasets, with the largest effect on FACED, where Cohen’sκ\\kappadrops from0\.5340\.534to0\.3910\.391\. The smaller changes on Sleep\-EDF and CHB\-MIT suggest that per\-channel tokens may be more beneficial for short\-window tasks such as FACED\. Channel attention shows a different pattern: removing it has its clearest effect on CHB\-MIT, reducing AUC\-PR from0\.4130\.413to0\.3400\.340, while performance on two\-channel Sleep\-EDF changes little\. This is consistent with Section[5\.2](https://arxiv.org/html/2609.38587#S5.SS2), where richer multichannel input provides greater scope for cross\-channel interaction\.

Explicit retrieval is particularly beneficial\.Removing either memory retrieval or the slow stream with memory lowers Cohen’sκ\\kappaon FACED and Sleep\-EDF, as well as AUC\-PR on CHB\-MIT, supporting the role of both components in downstream prediction\. This suggests that explicit retrieval strengthens short\-window representations by providing access to recent summaries, which may explain its larger effect on FACED\. These benefits do not extend uniformly to every metric: removing the slow stream with memory reduces CHB\-MIT false alarms from0\.960\.96to0\.470\.47per hour, alongside a decrease in AUC\-PR from0\.4130\.413to0\.3940\.394\. Mamba\-only jointly removes channel attention and the slow stream with memory, retaining only the per\-channel fast Mamba backbone\. It performs worse across all reported metrics, showing that the full multi\-component architecture consistently outperforms the per\-channel Mamba backbone alone\. Shuffling unit order also degrades Sleep\-EDF and CHB\-MIT performance, indicating that temporal order contributes to sequence modelling\.

Table 6:Persistent state in NeurDuo \(Small\)\.Backbone state is reset every 30 s or carried across a 10 min sequence, keeping the context\-head architecture fixed\.StateSleep\-EDFCHB\-MIT \(15 ch\)Macro\-F1↑\\uparrowκ\\kappa↑\\uparrowAUC\-PR↑\\uparrowFP⋅\\cdoth\-1↓\\downarrowReset0\.745±0\.0140\.745\\pm 0\.0140\.738±0\.0130\.738\\pm 0\.0130\.307±0\.1910\.307\\pm 0\.1915\.165\.16Carry0\.755±0\.008\\mathbf\{0\.755\\pm 0\.008\}0\.749±0\.011\\mathbf\{0\.749\\pm 0\.011\}0\.413±0\.174\\mathbf\{0\.413\\pm 0\.174\}0\.96\\mathbf\{0\.96\}Persistent backbone state improves both sequence tasks for NeurDuo\-Small\.Table[6](https://arxiv.org/html/2609.38587#S5.T6)compares carrying the backbone state across the full 10\-min sequence with resetting it at every 30\-s boundary, while keeping the downstream sequence head unchanged\. Carrying state improves performance on both tasks, with a modest gain on Sleep\-EDF and a larger improvement on CHB\-MIT, where AUC\-PR increases from0\.3070\.307to0\.4130\.413\. These results show that information retained within the NeurDuo backbone across successive units contributes directly to downstream prediction\.

Pre\-training improves performance on all five tasks\.Appendix[E\.1](https://arxiv.org/html/2609.38587#A5.SS1)shows that, relative to random initialisation, it raises NeurDuo\-Small’s FACED balanced accuracy from0\.4880\.488to0\.5890\.589and 15\-channel CHB\-MIT AUC\-PR from0\.3090\.309to0\.4130\.413under the same fine\-tuning protocol\. With architecture and readout held fixed, these gains support the value of pre\-trained representations across task types\. The CHB\-MIT improvement further shows that pre\-training on3232s sequences remains beneficial when adapting to1010min downstream context\.

## 6Conclusion

This paper introduces NeurDuo, a causal EEG foundation model for continuous multichannel EEG with persistent state and structured explicit memory\. Pre\-trained on 3,955 hours of EEG, NeurDuo achieves the best performance on four of five downstream benchmarks, including all three short\-window tasks and seizure detection, while supporting efficient streaming inference over long histories\. These results show that persistent, multi\-timescale modelling extends EEG foundation models beyond isolated windows while also surpassing existing baselines on short\-window tasks\.

Several limitations remain\. NeurDuo is trained with a local prediction objective, relies on implicit recurrent state for long\-range retention beyond its short explicit memory, and is evaluated on only two continuous long\-sequence tasks\. Future work will explore longer\-horizon objectives, more adaptive memory mechanisms, and broader continuous EEG evaluation\. Nevertheless, we believe our work points toward a broader shift from isolated\-window representation learning to continuous long\-horizon modelling through persistent memory management, opening a new direction for foundation models of continuous neural signals\.

## References

- R\. B\. Berry, R\. Brooks, C\. E\. Gamaldo, S\. M\. Harding, C\. Marcus, B\. V\. Vaughn,et al\.The aasm manual for the scoring of sleep and associated events\.Rules, Terminology and Technical Specifications, Darien, Illinois, American Academy of Sleep Medicine176\(2012\),pp\. 7\.Cited by:[§1](https://arxiv.org/html/2609.38587#S1.p1.1)\.
- Blankertzet al\.\(2007\)B\. Blankertz, G\. Dornhege, M\. Krauledat, K\. Müller, and G\. CurioThe non\-invasive berlin brain–computer interface: fast acquisition of effective performance in untrained subjects\.NeuroImage37\(2\),pp\. 539–550\.Cited by:[Table 8](https://arxiv.org/html/2609.38587#A1.T8.4.16.1)\.
- Brodericket al\.\(2018\)M\. P\. Broderick, A\. J\. Anderson, G\. M\. Di Liberto, M\. J\. Crosse, and E\. C\. LalorElectrophysiological correlates of semantic dissimilarity reflect the comprehension of natural, narrative speech\.Current Biology28\(5\),pp\. 803–809\.Cited by:[Table 8](https://arxiv.org/html/2609.38587#A1.T8.4.13.1)\.
- Caoet al\.\(2019\)Z\. Cao, C\. Chuang, J\. King, and C\. LinMulti\-channel eeg recordings during a sustained\-attention driving task\.Scientific data6\(1\),pp\. 19\.Cited by:[§A\.1](https://arxiv.org/html/2609.38587#A1.SS1.SSS0.Px3.p1.1),[Table 8](https://arxiv.org/html/2609.38587#A1.T8.4.8.1)\.
- Chenet al\.\(2023\)J\. Chen, X\. Wang, C\. Huang, X\. Hu, X\. Shen, and D\. ZhangA large finer\-grained affective computing eeg dataset\.Scientific Data10\(1\),pp\. 740\.Cited by:[§C\.1](https://arxiv.org/html/2609.38587#A3.SS1.p1.1),[Table 13](https://arxiv.org/html/2609.38587#A3.T13.8.2.1.1.1),[§4\.2](https://arxiv.org/html/2609.38587#S4.SS2.SSS0.Px1.p1.1)\.
- Congedoet al\.\(2011\)M\. Congedo, M\. Goyat, N\. Tarrin, G\. Ionescu, L\. Varnet, B\. Rivet, R\. Phlypo, N\. Jrad, M\. Acquadro, and C\. Jutten” Brain invaders”: a prototype of an open\-source p300\-based video game working with the openvibe platform\.InBCI 2011\-5th International Brain\-Computer Interface Conference,pp\. 280–283\.Cited by:[Table 8](https://arxiv.org/html/2609.38587#A1.T8.4.14.1)\.
- Dettiet al\.\(2020\)P\. Detti, G\. Vatti, and G\. Zabalo Manrique de LaraEEG synchronization analysis for seizure prediction: a study on data of noninvasive recordings\.Processes8\(7\),pp\. 846\.Cited by:[Table 8](https://arxiv.org/html/2609.38587#A1.T8.4.6.1)\.
- Döneret al\.\(2026\)B\. Döner, T\. M\. Ingolfsson, L\. Benini, and Y\. LiLUNA: efficient and topology\-agnostic foundation model for eeg signal analysis\.Advances in Neural Information Processing Systems38,pp\. 70682–70708\.Cited by:[§2](https://arxiv.org/html/2609.38587#S2.SS0.SSS0.Px1.p1.1)\.
- Durgamet al\.\(2026\)A\. Durgam, N\. Siddiqui, J\. A\. Chan\-Santiago, Q\. Fu, E\. D\. Gireesh, and M\. ShahCaMBRAIN: real\-time, continuous eeg inference with causal state space models\.arXiv preprint arXiv:2605\.28792\.Cited by:[§C\.5](https://arxiv.org/html/2609.38587#A3.SS5.SSS0.Px8.p1.1),[§D\.2](https://arxiv.org/html/2609.38587#A4.SS2.p1.1),[§1](https://arxiv.org/html/2609.38587#S1.p3.1),[§2](https://arxiv.org/html/2609.38587#S2.SS0.SSS0.Px2.p1.1),[§4\.2](https://arxiv.org/html/2609.38587#S4.SS2.SSS0.Px1.p1.1)\.
- El Ouahidiet al\.\(2026\)Y\. El Ouahidi, J\. Lys, P\. Thölke, N\. Farrugia, B\. Pasdeloup, V\. Gripon, K\. Jerbi, and G\. LioiREVE: a foundation model for eeg\-adapting to any setup with large\-scale pretraining on 25,000 subjects\.Advances in Neural Information Processing Systems38,pp\. 22541–22577\.Cited by:[§2](https://arxiv.org/html/2609.38587#S2.SS0.SSS0.Px1.p1.1),[§4\.2](https://arxiv.org/html/2609.38587#S4.SS2.SSS0.Px1.p1.1)\.
- Ghassemiet al\.\(2018\)M\. M\. Ghassemi, B\. E\. Moody, L\. H\. Lehman, C\. Song, Q\. Li, H\. Sun, R\. G\. Mark, M\. B\. Westover, and G\. D\. CliffordYou snooze, you win: the physionet/computing in cardiology challenge 2018\.In2018 Computing in Cardiology Conference \(CinC\),Vol\.45,pp\. 1–4\.Cited by:[Table 8](https://arxiv.org/html/2609.38587#A1.T8.4.2.1)\.
- Goldbergeret al\.\(2000\)A\. L\. Goldberger, L\. A\. Amaral, L\. Glass, J\. M\. Hausdorff, P\. C\. Ivanov, R\. G\. Mark, J\. E\. Mietus, G\. B\. Moody, C\. Peng, and H\. E\. StanleyPhysioBank, physiotoolkit, and physionet: components of a new research resource for complex physiologic signals\.circulation101\(23\),pp\. e215–e220\.Cited by:[Table 8](https://arxiv.org/html/2609.38587#A1.T8.4.2.1),[Table 8](https://arxiv.org/html/2609.38587#A1.T8.4.9.1),[§C\.4](https://arxiv.org/html/2609.38587#A3.SS4.p1.1),[§C\.5](https://arxiv.org/html/2609.38587#A3.SS5.p1.1),[Table 13](https://arxiv.org/html/2609.38587#A3.T13.8.5.1.1.1),[Table 13](https://arxiv.org/html/2609.38587#A3.T13.8.6.1.1.1),[§4\.2](https://arxiv.org/html/2609.38587#S4.SS2.SSS0.Px1.p1.1)\.
- Gu and Dao \(2023\)A\. Gu and T\. DaoMamba: linear\-time sequence modeling with selective state spaces\.arXiv preprint arXiv:2312\.00752\.Cited by:[§1](https://arxiv.org/html/2609.38587#S1.p3.1),[§2](https://arxiv.org/html/2609.38587#S2.SS0.SSS0.Px2.p1.1)\.
- Guiet al\.\(2024\)Y\. Gui, M\. Chen, Y\. Su, G\. Luo, and Y\. YangEEGMamba: bidirectional state space model with mixture of experts for eeg multi\-task classification\.arXiv preprint arXiv:2407\.20254\.Cited by:[§1](https://arxiv.org/html/2609.38587#S1.p3.1),[§2](https://arxiv.org/html/2609.38587#S2.SS0.SSS0.Px2.p1.1)\.
- Hamidet al\.\(2020\)A\. Hamid, K\. Gagliano, S\. Rahman, N\. Tulin, V\. Tchiong, I\. Obeid, and J\. PiconeThe temple university artifact corpus: an annotated corpus of eeg artifacts\.In2020 IEEE Signal Processing in Medicine and Biology Symposium \(SPMB\),pp\. 1–4\.Cited by:[Table 8](https://arxiv.org/html/2609.38587#A1.T8.4.7.1)\.
- Honget al\.\(2025\)J\. Hong, G\. Mackellar, and S\. GhaneSAMBA: toward a long\-context eeg foundation model via spatial embedding and differential mamba\.arXiv preprint arXiv:2511\.18571\.Cited by:[§2](https://arxiv.org/html/2609.38587#S2.SS0.SSS0.Px2.p1.1)\.
- Jayaram and Barachant \(2018\)V\. Jayaram and A\. BarachantMOABB: trustworthy algorithm benchmarking for bcis\.Journal of neural engineering15\(6\),pp\. 066011\.Cited by:[§D\.1\.1](https://arxiv.org/html/2609.38587#A4.SS1.SSS1.Px2.p1.1),[§4\.2](https://arxiv.org/html/2609.38587#S4.SS2.SSS0.Px1.p1.1)\.
- Jianget al\.\(2024\)W\. Jiang, L\. Zhao, and B\. LuLarge brain model for learning generic representations with tremendous eeg data in bci\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 16405–16426\.Cited by:[§1](https://arxiv.org/html/2609.38587#S1.p2.1),[§2](https://arxiv.org/html/2609.38587#S2.SS0.SSS0.Px1.p1.1),[§4\.2](https://arxiv.org/html/2609.38587#S4.SS2.SSS0.Px1.p1.1)\.
- Kaneet al\.\(2017\)N\. Kane, J\. Acharya, S\. Beniczky, L\. Caboclo, S\. Finnigan, P\. W\. Kaplan, H\. Shibasaki, R\. Pressler, and M\. J\. Van PuttenA revised glossary of terms most commonly used by clinical electroencephalographers and updated proposal for the report format of the eeg findings\. revision 2017\.Clinical neurophysiology practice2,pp\. 170–185\.Cited by:[§1](https://arxiv.org/html/2609.38587#S1.p1.1)\.
- Kastratiet al\.\(2025\)A\. Kastrati, J\. Bürki, J\. Lauer, C\. Xuan, R\. Iaquinto, and R\. WattenhoferEeg\-bench: a benchmark for eeg foundation models in clinical applications\.arXiv preprint arXiv:2512\.08959\.Cited by:[§D\.1\.1](https://arxiv.org/html/2609.38587#A4.SS1.SSS1.Px1.p1.1),[§4\.2](https://arxiv.org/html/2609.38587#S4.SS2.SSS0.Px1.p1.1)\.
- Katsigiannis and Ramzan \(2017\)S\. Katsigiannis and N\. RamzanDREAMER: a database for emotion recognition through eeg and ecg signals from wireless low\-cost off\-the\-shelf devices\.IEEE journal of biomedical and health informatics22\(1\),pp\. 98–107\.Cited by:[Table 8](https://arxiv.org/html/2609.38587#A1.T8.4.11.1)\.
- Kempet al\.\(2000\)B\. Kemp, A\. H\. Zwinderman, B\. Tuk, H\. A\. Kamphuisen, and J\. J\. OberyeAnalysis of a sleep\-dependent neuronal feedback loop: the slow\-wave microcontinuity of the eeg\.IEEE Transactions on Biomedical Engineering47\(9\),pp\. 1185–1194\.Cited by:[§C\.4](https://arxiv.org/html/2609.38587#A3.SS4.p1.1),[Table 13](https://arxiv.org/html/2609.38587#A3.T13.8.5.1.1.1),[§4\.2](https://arxiv.org/html/2609.38587#S4.SS2.SSS0.Px1.p1.1)\.
- Khalighiet al\.\(2016\)S\. Khalighi, T\. Sousa, J\. M\. Santos, and U\. NunesISRUC\-sleep: a comprehensive public dataset for sleep researchers\.Computer methods and programs in biomedicine124,pp\. 180–192\.Cited by:[Table 8](https://arxiv.org/html/2609.38587#A1.T8.4.4.1)\.
- Koelstraet al\.\(2011\)S\. Koelstra, C\. Muhl, M\. Soleymani, J\. Lee, A\. Yazdani, T\. Ebrahimi, T\. Pun, A\. Nijholt, and I\. PatrasDeap: a database for emotion analysis; using physiological signals\.IEEE transactions on affective computing3\(1\),pp\. 18–31\.Cited by:[Table 8](https://arxiv.org/html/2609.38587#A1.T8.4.12.1)\.
- Kuruppuet al\.\(2026\)G\. Kuruppu, N\. Wagh, V\. Kremen, and Y\. VaratharajahEEG foundation models: a critical review of current progress and future directions\.Journal of neural engineering23\(2\),pp\. 021001\.Cited by:[§1](https://arxiv.org/html/2609.38587#S1.p2.1)\.
- Luciwet al\.\(2014\)M\. D\. Luciw, E\. Jarocka, and B\. B\. EdinMulti\-channel eeg recordings during 3,936 grasp and lift trials with varying weight and friction\.Scientific data1\(1\),pp\. 140047\.Cited by:[Table 8](https://arxiv.org/html/2609.38587#A1.T8.4.15.1)\.
- Margauxet al\.\(2012\)P\. Margaux, M\. Emmanuel, D\. Sébastien, B\. Olivier, and M\. JérémieObjective and subjective evaluation of online error correction during p300\-based spelling\.Advances in Human\-Computer Interaction2012\(1\),pp\. 578295\.Cited by:[§C\.2](https://arxiv.org/html/2609.38587#A3.SS2.p1.1),[Table 13](https://arxiv.org/html/2609.38587#A3.T13.8.3.1.1.1),[§4\.2](https://arxiv.org/html/2609.38587#S4.SS2.SSS0.Px1.p1.1)\.
- Mattoutet al\.\(2014\)J\. Mattout, Manu, maucle, and W\. KanBCI challenge @ ner 2015\.Note:[https://kaggle\.com/competitions/inria\-bci\-challenge](https://kaggle.com/competitions/inria-bci-challenge)KaggleCited by:[§C\.2](https://arxiv.org/html/2609.38587#A3.SS2.p1.1),[Table 13](https://arxiv.org/html/2609.38587#A3.T13.8.3.1.1.1),[§4\.2](https://arxiv.org/html/2609.38587#S4.SS2.SSS0.Px1.p1.1)\.
- Sabbaghet al\.\(2019\)D\. Sabbagh, P\. Ablin, G\. Varoquaux, A\. Gramfort, and D\. A\. EngemannManifold\-regression to predict from meg/eeg brain signals without source modeling\.Advances in neural information processing systems32\.Cited by:[§D\.1\.1](https://arxiv.org/html/2609.38587#A4.SS1.SSS1.Px2.p1.1),[§4\.2](https://arxiv.org/html/2609.38587#S4.SS2.SSS0.Px1.p1.1)\.
- Schalket al\.\(2004\)G\. Schalk, D\. J\. McFarland, T\. Hinterberger, N\. Birbaumer, and J\. R\. WolpawBCI2000: a general\-purpose brain\-computer interface \(bci\) system\.IEEE Transactions on biomedical engineering51\(6\),pp\. 1034–1043\.Cited by:[Table 8](https://arxiv.org/html/2609.38587#A1.T8.4.9.1)\.
- Schirrmeisteret al\.\(2017\)R\. T\. Schirrmeister, J\. T\. Springenberg, L\. D\. J\. Fiederer, M\. Glasstetter, K\. Eggensperger, M\. Tangermann, F\. Hutter, W\. Burgard, and T\. BallDeep learning with convolutional neural networks for eeg decoding and visualization\.Human brain mapping38\(11\),pp\. 5391–5420\.Cited by:[Table 8](https://arxiv.org/html/2609.38587#A1.T8.4.10.1)\.
- Shahet al\.\(2018\)V\. Shah, E\. Von Weltin, S\. Lopez, J\. R\. McHugh, L\. Veloso, M\. Golmohammadi, I\. Obeid, and J\. PiconeThe temple university hospital seizure detection corpus\.Frontiers in neuroinformatics12,pp\. 83\.Cited by:[Table 8](https://arxiv.org/html/2609.38587#A1.T8.4.3.1)\.
- Shoeb \(2009\)A\. H\. ShoebApplication of machine learning to epileptic seizure onset detection and treatment\.Ph\.D\. Thesis,Massachusetts Institute of Technology\.Cited by:[§C\.5](https://arxiv.org/html/2609.38587#A3.SS5.p1.1),[Table 13](https://arxiv.org/html/2609.38587#A3.T13.8.6.1.1.1),[§4\.2](https://arxiv.org/html/2609.38587#S4.SS2.SSS0.Px1.p1.1)\.
- Tangermannet al\.\(2012\)M\. Tangermann, K\. Müller, A\. Aertsen, N\. Birbaumer, C\. Braun, C\. Brunner, R\. Leeb, C\. Mehring, K\. J\. Miller, G\. R\. Müller\-Putz,et al\.Review of the bci competition iv\.Frontiers in neuroscience6,pp\. 55\.Cited by:[Table 8](https://arxiv.org/html/2609.38587#A1.T8.4.16.1)\.
- Tatumet al\.\(2022\)W\. O\. Tatum, J\. Mani, K\. Jin, J\. J\. Halford, D\. Gloss, F\. Fahoum, L\. Maillard, I\. Mothersill, and S\. BeniczkyMinimum standards for inpatient long\-term video\-eeg monitoring: a clinical practice guideline of the international league against epilepsy and international federation of clinical neurophysiology\.Clinical Neurophysiology134,pp\. 111–128\.Cited by:[§1](https://arxiv.org/html/2609.38587#S1.p1.1)\.
- Tegonet al\.\(2025\)A\. Tegon, T\. M\. Ingolfsson, X\. Wang, L\. Benini, and Y\. LiFEMBA: efficient and scalable eeg analysis with a bidirectional mamba foundation model\.In2025 47th Annual International Conference of the IEEE Engineering in Medicine and Biology Society \(EMBC\),pp\. 1–7\.Cited by:[§1](https://arxiv.org/html/2609.38587#S1.p3.1),[§2](https://arxiv.org/html/2609.38587#S2.SS0.SSS0.Px2.p1.1)\.
- Torkamani\-Azaret al\.\(2020\)M\. Torkamani\-Azar, S\. D\. Kanik, S\. Aydin, and M\. CetinPrediction of reaction time and vigilance variability from spatio\-spectral features of resting\-state eeg in a long sustained attention task\.IEEE journal of biomedical and health informatics24\(9\),pp\. 2550–2558\.Cited by:[Table 8](https://arxiv.org/html/2609.38587#A1.T8.4.18.1)\.
- Trujilloet al\.\(2017\)L\. T\. Trujillo, C\. T\. Stanfield, and R\. D\. VelaThe effect of electroencephalogram \(eeg\) reference choice on information\-theoretic measures of the complexity and integration of eeg signals\.Frontiers in neuroscience11,pp\. 425\.Cited by:[Table 8](https://arxiv.org/html/2609.38587#A1.T8.4.17.1)\.
- Vallat and Walker \(2021\)R\. Vallat and M\. P\. WalkerAn open\-source, high\-performance tool for automated sleep staging\.elife10,pp\. e70092\.Cited by:[§D\.1\.1](https://arxiv.org/html/2609.38587#A4.SS1.SSS1.Px1.p1.1),[§4\.2](https://arxiv.org/html/2609.38587#S4.SS2.SSS0.Px1.p1.1)\.
- Velosoet al\.\(2017\)L\. Veloso, J\. McHugh, E\. von Weltin, S\. Lopez, I\. Obeid, and J\. PiconeBig data resources for eegs: enabling deep learning research\.In2017 IEEE signal processing in medicine and biology symposium \(SPMB\),pp\. 1–3\.Cited by:[Table 8](https://arxiv.org/html/2609.38587#A1.T8.4.5.1)\.
- Wanget al\.\(2024\)G\. Wang, W\. Liu, Y\. He, C\. Xu, L\. Ma, and H\. LiEegpt: pretrained transformer for universal and reliable representation of eeg signals\.Advances in Neural Information Processing Systems37,pp\. 39249–39280\.Cited by:[§C\.2](https://arxiv.org/html/2609.38587#A3.SS2.p1.1),[§C\.4](https://arxiv.org/html/2609.38587#A3.SS4.p1.1),[§D\.2](https://arxiv.org/html/2609.38587#A4.SS2.p1.1),[§1](https://arxiv.org/html/2609.38587#S1.p2.1),[§2](https://arxiv.org/html/2609.38587#S2.SS0.SSS0.Px1.p1.1),[§4\.2](https://arxiv.org/html/2609.38587#S4.SS2.SSS0.Px1.p1.1)\.
- Wanget al\.\(2025\)J\. Wang, S\. Zhao, Z\. Luo, Y\. Zhou, H\. Jiang, S\. Li, T\. Li, and G\. PanCbramod: a criss\-cross brain foundation model for eeg decoding\.InInternational conference on learning representations,Vol\.2025,pp\. 75310–75346\.Cited by:[§C\.1](https://arxiv.org/html/2609.38587#A3.SS1.p1.1),[§C\.3](https://arxiv.org/html/2609.38587#A3.SS3.p1.1),[§C\.3](https://arxiv.org/html/2609.38587#A3.SS3.p2.1),[§C\.3](https://arxiv.org/html/2609.38587#A3.SS3.p3.1),[§D\.2\.4](https://arxiv.org/html/2609.38587#A4.SS2.SSS4.p1.1),[§D\.2](https://arxiv.org/html/2609.38587#A4.SS2.p1.1),[§1](https://arxiv.org/html/2609.38587#S1.p2.1),[§2](https://arxiv.org/html/2609.38587#S2.SS0.SSS0.Px1.p1.1),[§4\.2](https://arxiv.org/html/2609.38587#S4.SS2.SSS0.Px1.p1.1)\.
- Yanget al\.\(2023\)C\. Yang, M\. B\. Westover, and J\. SunBIOT: cross\-data biosignal learning in the wild\.arXiv preprint arXiv:2305\.10351\.Cited by:[§2](https://arxiv.org/html/2609.38587#S2.SS0.SSS0.Px1.p1.1),[§4\.2](https://arxiv.org/html/2609.38587#S4.SS2.SSS0.Px1.p1.1)\.
- Yanget al\.\(2026\)L\. Yang, Q\. Sun, A\. Li, and M\. M\. Van HulleAre eeg foundation models worth it? comparative evaluation with traditional decoders in diverse bci tasks\.InThe Fourteenth International Conference on Learning Representations,Cited by:[§D\.2\.4](https://arxiv.org/html/2609.38587#A4.SS2.SSS4.p1.1),[§D\.2](https://arxiv.org/html/2609.38587#A4.SS2.p1.1),[Table 14](https://arxiv.org/html/2609.38587#A4.T14.5),[§2](https://arxiv.org/html/2609.38587#S2.SS0.SSS0.Px1.p1.1),[§4\.2](https://arxiv.org/html/2609.38587#S4.SS2.SSS0.Px1.p1.1)\.
- Zheng and Lu \(2017\)W\. Zheng and B\. LuA multimodal approach to estimating vigilance using eeg and forehead eog\.Journal of Neural Engineering14\(2\),pp\. 026017\.External Links:[Link](http://stacks.iop.org/1741-2552/14/i=2/a=026017)Cited by:[§C\.3](https://arxiv.org/html/2609.38587#A3.SS3.p1.1),[Table 13](https://arxiv.org/html/2609.38587#A3.T13.8.4.1.1.1),[§4\.2](https://arxiv.org/html/2609.38587#S4.SS2.SSS0.Px1.p1.1)\.
- Zhouet al\.\(2026a\)T\. Zhou, J\. Han, L\. Shu, and Z\. ZhangMSBraM: a multi\-scale self\-supervised brain foundation model for hierarchical eeg dynamics learning\.arXiv preprint arXiv:2607\.21402\.Cited by:[§2](https://arxiv.org/html/2609.38587#S2.SS0.SSS0.Px1.p1.1)\.
- Zhouet al\.\(2026b\)Y\. Zhou, J\. Wu, Z\. Ren, Z\. Yao, W\. Lu, K\. Peng, Q\. Zheng, C\. Song, W\. Ouyang, and C\. GouCsbrain: a cross\-scale spatiotemporal brain foundation model for eeg decoding\.Advances in Neural Information Processing Systems38,pp\. 87150–87195\.Cited by:[§C\.3](https://arxiv.org/html/2609.38587#A3.SS3.p3.1)\.

## Appendix AModel and Pre\-training Details

### A\.1Pre\-training Details

##### Corpus\.

Tables[7](https://arxiv.org/html/2609.38587#A1.T7)and[8](https://arxiv.org/html/2609.38587#A1.T8)give the corpus by domain and by dataset\. The composition is uneven by necessity rather than by choice: continuous overnight and clinical recordings are the only sources that exist at scale, and they supply more than nine tenths of the hours, while the montage range is set by the task\-based corpora at the other end\. Because the metadata encoder is given electrode coordinates and never a dataset label \(Appendix[A\.2](https://arxiv.org/html/2609.38587#A1.SS2)\), the model is given no explicit corpus\-identity cue with which to exploit this imbalance\.

Table 7:Pre\-training corpus by domain\.All counts are after preprocessing\. \#Rec\. is the number of recordings and Hours their total duration; \#Ch\. is the electrodes retained once channel labels are resolved to standard positions, given as a range where a domain’s corpora use different montages; \#Windows is the number of3232s pre\-training windows the recordings are cut into, at50%50\\%overlap\. Table[8](https://arxiv.org/html/2609.38587#A1.T8)gives the per\-dataset breakdown\.DomainDatasets\#Rec\.Hours\#Ch\.\#WindowsClinicalTUSZ, TUEP, Siena, TUAR8,0071,811\.917–19395,424Sleep \(PSG\)PhysioNet\-2018, ISRUC11,3391,882\.36406,517MotorEEGMMIDB, HGD, GAL, BCI IV\-11,82195\.532–12618,415AttentionCao 201951681\.93017,652EmotionDEAP, DREAMER1,69446\.214–327,367Speech \(auditory\)Broderick38020\.61284,056ERPBrain Invaders6412\.4162,698Resting stateTrujillo, SPIS423\.864805Total17 datasets23,8633,954\.76–128852,934Table 8:Pre\-training corpus, per dataset, after preprocessing\.The per\-dataset expansion of Table[7](https://arxiv.org/html/2609.38587#A1.T7), ordered by hours\. \#Rec\. is the number of recordings and Hours their total duration; \#Ch\. is the electrodes retained once channel labels are resolved to standard positions; \#Windows is the number of3232s pre\-training windows the recordings are cut into, at50%50\\%overlap\. Where a channel count differs from the one quoted in a release’s own publication, the table reports the signal channels the released files contain\.DatasetDomain\#Rec\.Hours\#Ch\.\#WindowsPhysioNet\-2018\([Ghassemi et al\., 2018](https://arxiv.org/html/2609.38587#bib.bib26);[Goldberger et al\., 2000](https://arxiv.org/html/2609.38587#bib.bib18)\)Sleep \(PSG\)6,8301,138\.36245,880TUSZ\([Shah et al\., 2018](https://arxiv.org/html/2609.38587#bib.bib27)\)Clinical4,959911\.217–19197,388ISRUC\([Khalighi et al\., 2016](https://arxiv.org/html/2609.38587#bib.bib28)\)Sleep \(PSG\)4,509744\.06160,637TUEP\([Veloso et al\., 2017](https://arxiv.org/html/2609.38587#bib.bib29)\)Clinical2,697631\.717–19138,035Siena\([Detti et al\., 2020](https://arxiv.org/html/2609.38587#bib.bib30)\)Clinical41169\.01937,963TUAR\([Hamid et al\., 2020](https://arxiv.org/html/2609.38587#bib.bib31)\)Clinical310100\.01922,038Cao 2019\([Cao et al\., 2019](https://arxiv.org/html/2609.38587#bib.bib32)\)Attention51681\.93017,652EEGMMIDB\([Schalk et al\., 2004](https://arxiv.org/html/2609.38587#bib.bib33);[Goldberger et al\., 2000](https://arxiv.org/html/2609.38587#bib.bib18)\)Motor1,52648\.5648,282HGD\([Schirrmeister et al\., 2017](https://arxiv.org/html/2609.38587#bib.bib34)\)Motor18528\.61266,161DREAMER\([Katsigiannis and Ramzan, 2017](https://arxiv.org/html/2609.38587#bib.bib35)\)Emotion41423\.8144,807DEAP\([Koelstra et al\., 2011](https://arxiv.org/html/2609.38587#bib.bib36)\)Emotion1,28022\.4322,560Broderick\([Broderick et al\., 2018](https://arxiv.org/html/2609.38587#bib.bib37)\)Speech \(auditory\)38020\.61284,056Brain Invaders\([Congedo et al\., 2011](https://arxiv.org/html/2609.38587#bib.bib38)\)ERP6412\.4162,698GAL\([Luciw et al\., 2014](https://arxiv.org/html/2609.38587#bib.bib39)\)Motor9610\.0322,102BCI IV\-1\([Tangermann et al\., 2012](https://arxiv.org/html/2609.38587#bib.bib41);[Blankertz et al\., 2007](https://arxiv.org/html/2609.38587#bib.bib40)\)Motor148\.4491,870Trujillo\([Trujillo et al\., 2017](https://arxiv.org/html/2609.38587#bib.bib42)\)Resting state223\.064645SPIS\([Torkamani\-Azar et al\., 2020](https://arxiv.org/html/2609.38587#bib.bib43)\)Resting state200\.864160Total17 datasets23,8633,954\.76–128852,934
##### Preprocessing and windowing\.

Each recording is band\-pass filtered to0\.50\.5–7575Hz, notch filtered, and resampled to256256Hz\. Channel labels are resolved against a standard electrode dictionary and mapped to coordinates on the unit sphere; channels that cannot be resolved are dropped, which is why the polysomnographic sources retain only their EEG derivations\. Recordings are then cut into sequences of6464chunks of128128samples:3232s, with chunks non\-overlapping, taken at a stride of3232chunks for backbone pre\-training, so consecutive sequences overlap by half; the tokenizer of stage 1 is trained on the same sequences at a stride of6464, without overlap\. Each sequence is an independent training sample: the recurrent state is initialised at its start and discarded at its end, so no gradient crosses a sequence boundary during pre\-training\. The half\-sequence stride is not merely a way of drawing more samples from the same hours\. Because state is reset at every boundary, a given moment of a recording would otherwise always fall at the same offset within a sequence, and so would always be seen with the same amount of history behind it; at a stride of half a sequence it falls at two offsets3232chunks apart, and the model is required to predict from a nearly cold state and from a warm one on the same signal\. A transition shorter than the stride is also left uncut in at least one sequence\. Downstream sequences are not overlapped: there the window is fixed by the labelling unit, a scoring epoch, an annotation interval, an event\-locked trial, and the warm\-up is instead absorbed by state passing from one unit to the next within a sequence\.

Table 9:Pre\-training hyperparameters\.Stage 1 trains the spectral tokenizer and stage 2 the backbone, with the tokenizer frozen between them\. A value spanning both columns is shared by the two stages; the Tokenizer block describes the stage\-1 model itself and so is not stage\-dependent\. Every entry is identical across the three model sizes except the peak learning rate, given as S/B/L\. “–” marks a setting that does not apply to that stage\.HyperparameterStage 1 \(tokenizer\)Stage 2 \(backbone\)DataChunk128128samples \(0\.50\.5s\), non\-overlappingSequence length6464chunks \(3232s\)6464chunks \(3232s\)Sequence stride6464\(no overlap\)3232\(50%50\\%overlap\)OptimisationOptimiserAdamWAdamWPeak learning rate5×10−55\\times 10^\{\-5\}3\.0/2\.2/1\.9×10−43\.0/2\.2/1\.9\\times 10^\{\-4\}\(S/B/L\)Weight decay10−410^\{\-4\}0\.050\.05Schedulecosine to0\.05×0\.05\\timescosine to0\.05×0\.05\\timesWarm\-up500500steps1,0001\{,\}000stepsGradient clipping1\.01\.01\.01\.0Frontend LR multiplier—0\.10\.1Adam\(β1,β2\)\(\\beta\_\{1\},\\beta\_\{2\}\)\(0\.9,0\.999\)\(0\.9,0\.999\)\(0\.9,0\.999\)\(0\.9,0\.999\)Adamϵ\\epsilon10−810^\{\-8\}10−810^\{\-8\}Epochs221616Effective batch6464sequences128128sequencesUpdates12,83212\{,\}832102,096102\{,\}096Precisionfloat16 \(loss scaling\)bfloat16Seed1337133713371337TokenizerInput feature2×652\\times 65\(rFFT amplitude⊕\\oplusphase of a128128\-sample chunk\)Codebook2,048×642\{,\}048\\times 64,ℓ2\\ell\_\{2\}\-normalisedEncodertwo pre\-norm residual MLP blocks, width128128Decoder width128128Codebook updateEMA0\.990\.99, initialised from first\-batch samples, dead\-code resetLoss weightsamplitude1\.01\.0, phase1\.01\.0, commitmentβ=1\.0\\beta=1\.0Table 10:Architecture constants\.Identical in all three model sizes; the widths and depths that scale are given in Table[1](https://arxiv.org/html/2609.38587#S4.T1)\. One step is one0\.50\.5s chunk, so the durations in parentheses follow from the step counts beside them\.HyperparameterValueFrontendChunk encoder kernels\{3,7,15\}\\\{3,7,15\\\}, three parallel branchesMetadata encoder width6464Channel attentionevery22fast layers,44headsStreamsSSM state dimension1616\(fast and slow\)SSM expansion factor22\(fast and slow\)Causal convolution kernel44\(fast\),33\(slow\)MemoryWriter pooling windowww88steps \(44s\)Writer queue slots1616\(ring buffer; each write pools the most recentww\)Write intervalNN44steps \(22s\)Bank slotsBB88\(1616s of explicit memory\)ObjectivePrediction horizon11step \(0\.50\.5s\)Target vocabulary2,0482\{,\}048codesPredictor widthdfd\_\{f\}
##### Montages\.

No montage unification is applied\. Each recording enters training with the channel set its release provides, resolved to coordinates, but neither reduced to a common subset nor interpolated onto one\. The seventeen corpora contribute recordings of66,1414,1616,1717,1919,3030,3232,4949,6464,126126and128128channels, and recordings of different widths are drawn into the same batch\. Batching pads the channel axis to the widest recording in the batch and records the padding in a mask; padded channels are excluded from the loss and masked out of channel attention, so they influence no other channel\. The66–128128range in Table[8](https://arxiv.org/html/2609.38587#A1.T8)is therefore a description of the corpus rather than a supported operating range: because a channel is addressed by its coordinate and not by its position in a fixed montage \(§[3\.2\.1](https://arxiv.org/html/2609.38587#S3.SS2.SSS1)\), how many channels a recording has is not a quantity the architecture has to be told\. Where the count in the table differs from the one quoted in a release’s original publication, the table reports the signal channels the released files contain;[Cao et al\. \(2019\)](https://arxiv.org/html/2609.38587#bib.bib32), for example, is recorded with a3232\-electrode cap of which two are mastoid references, and ships3030EEG channels\.

##### Hyperparameters\.

Table[9](https://arxiv.org/html/2609.38587#A1.T9)lists both stages and Table[10](https://arxiv.org/html/2609.38587#A1.T10)the architecture constants that Table[1](https://arxiv.org/html/2609.38587#S4.T1)does not carry\. Two implementation choices are particularly relevant to training stability\. The SSM parametersAlogA\_\{\\log\}andDDare excluded from weight decay along with biases and normalisation scales:AlogA\_\{\\log\}parameterises the state decay rates, so decaying it compresses the range of timescales the recurrence can represent\. And precision must be bfloat16 rather than float16, which overflows inside the selective scan: a float16 run of an earlier configuration diverged irrecoverably after roughly three thousand updates, and gradient clipping did not prevent it\.

Figure 3:Pre\-training curves for the three sizes of Table[1](https://arxiv.org/html/2609.38587#S4.T1), on the held\-out recordings\. Small and Base reach their minima as the schedule ends, at6\.05036\.0503and6\.02726\.0272nats; Large reaches its minimum of6\.04916\.0491after51,00051\{,\}000updates and then rises to6\.07156\.0715\.
##### Training curves\.

Figure[3](https://arxiv.org/html/2609.38587#A1.F3)plots validation loss and next\-code accuracy against updates for the three sizes, which differ only in width, depth and peak learning rate; corpus, schedule, effective batch and number of updates are identical\. The loss is a2,0482\{,\}048\-way cross\-entropy taken per channel and per step, so a uniform predictor sits atln⁡2,048=7\.62\\ln 2\{,\}048=7\.62nats and the figure’s vertical range covers the last1\.61\.6nats of that\. Accuracy is the top\-1 rate on that same2,0482\{,\}048\-way choice, for which a uniform predictor scores0\.049%0\.049\\%; the three runs end at2\.932\.93,3\.053\.05and2\.97%2\.97\\%\. Predicting the exact code half a second ahead remains challenging\. The observed improvement reflects temporal structure that makes some future codes more likely than others\. The three curves separate early, and their minima then lie within 0\.03 nats of one another, which is the observation the model family rests on\. Small and Base are still descending at the end of the schedule, but at under10−310^\{\-3\}nats per thousand updates, so little is left on the table\.

### A\.2Metadata: which fields are encoded, and why

The metadata encoder of §[3\.2\.1](https://arxiv.org/html/2609.38587#S3.SS2.SSS1)builds its per\-channel biasece\_\{c\}from two fields\. It could in principle use more, so we audited every field our preprocessing pipeline carries, over the training split of the pre\-training corpus \(§[4\.1](https://arxiv.org/html/2609.38587#S4.SS1)\),22,78022\{,\}780recordings; Table[11](https://arxiv.org/html/2609.38587#A1.T11)lists the dispositions\. Two fields turn out to be constant and a third nearly so, and would only add the same offset to every sample\. One is informative but cannot be used, for the reason given below\.

Because the channel\-type field is constant, electrode coordinates are the only signal by which the model can tell one channel from another; without them the frontend carries no electrode identity: it is applied to each channel independently, so permuting the channels would simply permute its outputs, and the channel attention of §[3\.2\.2](https://arxiv.org/html/2609.38587#S3.SS2.SSS2)would receive tokens that differ by their signal alone, with nothing to say which electrode each came from\. Coordinates are also what admits a varying montage: a lookup keyed by channel index cannot be shared across corpora whose channel counts range from66to128128, whereas a position on the scalp means the same thing everywhere\.

The reference scheme is kept on different grounds: it describes how a recording was made rather than which recording it is, and unlike corpus identity it is drawn from a small shared vocabulary, so a new dataset maps onto a category the model has already seen\.

Corpus identity is excluded because it is not available where it would be used\. The pre\-training corpus comprises1717datasets, so a corpus embedding has1717trained rows, while every downstream dataset in this paper is by construction absent from that corpus, which is what makes the evaluation a transfer evaluation\. Its embedding row would therefore still be at its random initialisation, and conditioning on it would inject an untrained vector into every token of every downstream sample\.

Within pre\-training, corpus identity is genuinely informative, and that is the second objection to it: the seventeen corpora differ systematically in montage, recording device and subject population, so a model told which corpus a window came from can lower its pre\-training loss by separating corpora rather than by learning the structure they share\. A backbone\-free probe reduces cross\-entropy by about0\.150\.15nats relative to a marginal predictor using recording\-level code statistics, while extending history beyond44s adds at most0\.0170\.017nats\. Overlap between windows prevents attributing the former solely to recording identity \(Appendix[B\.2](https://arxiv.org/html/2609.38587#A2.SS2)\)\. Separately, on CHB\-MIT, an oracle told only whether a unit’s recording contains a seizure, and never given the signal, reaches AUROC0\.9050\.905, where the best published result, obtained under an easier split, is0\.9210\.921\(Appendix[C\.5](https://arxiv.org/html/2609.38587#A3.SS5)\)\. These observations concern recording\-level information rather than corpus identity\. We did not train a variant that receives corpus identity, so they motivate the decision rather than constitute an ablation of it\.

Table 11:Metadata fields available in the pipeline and their disposition\.FieldUsedReasonElectrode coordinates✓Physical position; the only cue distinguishing channels\.Reference scheme✓Eight values across the pre\-training corpus, reflecting how each recording was referenced\.Channel type—Constant: all438,323438\{,\}323channels share one value\.Sampling rate—Constant: every recording is resampled to256256Hz\.Device identifier—Near\-constant:22,74022\{,\}740of22,78022\{,\}780recordings share one value\.Corpus identifier—Informative but unavailable at transfer time\.Missing\-channel mask—Not encoded as a token, but retained to mask padded channels in the channel attention and exclude them from the codebook statistics\.

## Appendix BPre\-training Objective Analysis

### B\.1Why the prediction target is discrete

The obvious objective for a causal model is to predict its own representation of the next step\. We began there and could not make it work; because the failure survived the remedies we tried rather than yielding to tuning, and because two standard remedies do not help, we record the diagnosis here\.

##### Setup\.

These runs predate the corpus and the model family of the main text: an earlier version of the data \(7,6077\{,\}607training and400400validation recordings, against the22,78022\{,\}780and1,0831\{,\}083of §[4\.1](https://arxiv.org/html/2609.38587#S4.SS1)\) and a much smaller backbone\. The unmodified objective ran to completion,17,70317\{,\}703updates over20\.720\.7hours; the three variants are four\-hour probes stopped by their wall\-clock limit rather than by convergence, so what follows compares trajectories, not converged values\. Their configurations differ by a single field each: centring is the only change from the unmodified objective, the variance and covariance weights the only change from the centred run, and the discrete target the only change from it in turn\.

##### The objective and its collapse\.

A student model produces a state from the signal seen so far, and a predictor maps that state to the frontend tokenkksteps ahead\. The target token comes not from the student itself but from an exponential\-moving\-average copy of its frontend alone, patch encoder, metadata encoder and channel attention, which receives no gradient\. Training is contrastive with in\-batch negatives, same\-recording negatives excluded\. After twenty hours the validation loss had not moved\. A pass over the held\-out recordings, measuring the cosine similarity between target representations within and across recordings, the fraction of their norm carried by their mean, and the predictor’s alignment with the true future against a randomly drawn one, showed why\. The cosine similarity between target representations was1\.00001\.0000both within and across recordings, with99\.99%99\.99\\%of their norm carried by a component shared across all inputs\. The loss sat at8\.2535=log⁡38418\.2535=\\log 3841, the value a contrastive objective takes when its candidates are indistinguishable\. The predictor aligned with the true future at0\.12930\.1293and with a random future at0\.12900\.1290\. The student’s own frontend output collapsed in the same way, so this is not an artefact of the moving\-average copy\. It is not a failure of the data either: subtracting the shared component drops the across\-recording cosine from1\.00001\.0000to0\.0130\.013, close to orthogonal, the information was present throughout, dominated by a constant the objective had no reason to remove\.

##### Collapse in our continuous\-target configuration\.

In our setup the keys come from a momentum\-updated copy of the frontend, which receives no gradient of its own and only follows the student\. Once the target representations become indistinguishable the contrastive gradient with respect to the query vanishes, which is why the unmodified run sat atlog⁡3841\\log 3841for twenty hours\. Objectives of this shape are trained successfully elsewhere with momentum encoders, so we do not claim collapse must follow; in our configuration training did not recover from it\.

##### Two standard remedies do not fix it\.

Centring the representations before normalisation, in the manner of self\-distillation methods, repaired the training side convincingly: training loss fell from8\.258\.25to between4\.84\.8and5\.65\.6and training accuracy rose from chance to twenty or thirty percent, while validation loss rose from8\.358\.35to11\.011\.0and validation accuracy stayed flat\. Centring removes the shared component when the loss is evaluated; the representation still consists of that component plus a residual four orders of magnitude smaller\. Variance and covariance regularisation made matters worse: the regulariser took effect, its variance term falling from0\.950\.95to0\.060\.06, but validation accuracy halved and validation loss diverged past the level reached without it, and probing on validation data found the target still collapsed at an across\-recording cosine of0\.99960\.9996\.

##### A frozen discrete target removed it\.

Replacing the continuous target with the frozen tokenizer’s codes and the contrastive loss with cross\-entropy over the codebook, the only change from the centred run, reverses all three symptoms: validation loss decreases monotonically from7\.6097\.609to6\.7296\.729without diverging, validation accuracy rises monotonically instead of remaining flat, and the training–validation gap closes to within a tenth of a percentage point\. The mechanism is that the tokenizer reconstructs a fixed external quantity, the spectrum, so the target cannot drift toward a constant of its own; a constant prediction remains available to the backbone, as Appendix[D\.3](https://arxiv.org/html/2609.38587#A4.SS3)shows for the readout, but it is no longer a configuration the objective cannot move away from, as it was when the target moved with the model\. The general lesson is that collapse of this kind lives in the representation, where a regulariser applied at the loss can hide it without removing it; the informative measurement is on the representation itself, not on the loss\.

### B\.2How much history a prediction horizon requires

The temporal horizon of the pre\-training objective \(§[3\.3](https://arxiv.org/html/2609.38587#S3.SS3)\) is one step\. Because a horizon is only worth extending if the signal carries structure at that range, we measured how far back a predictor must look as a function of how far ahead it must predict\. The measurement deliberately does not involve the backbone: a model that never learned to use long history produces a flat curve whether or not the history is informative, so a backbone\-based probe cannot separate the signal has no long\-range structure from our model failed to exploit it\.

##### Probe\.

We freeze the stage\-1 tokenizer and convert recordings to code sequences at one code per0\.50\.5s\. The20482048codebook vectors are clustered intoM=32M=32coarse classes bykk\-means, so each step becomes a point in a3232\-way alphabet and each interval becomes a normalised histogram\. Given the histogramXKX\_\{K\}of the precedingKKsteps, a linearM→MM\\to Mmodel predicts the histogram of a short intervalHHsteps later; the score is held\-out soft\-label cross\-entropy \(CE\) in nats, lower being better, in the same unit as the pre\-training loss\. WithM=32M=32the predictor has about a thousand parameters and is a deliberately low\-capacity reader of19,20019\{,\}200windows\. The probe is split by recording, so no recording contributes to both fitting and evaluation\.

##### Controlling for recording identity\.

Longer history can improve this score even when the signal is stationary, because it estimates*that recording’s*own code distribution more precisely, and recording identity is exactly the kind of cue that does not transfer across subjects\. We therefore do not deform the target distribution \(an earlier version subtracted each recording’s global histogram from both sides, which pinned every cell atln⁡32=3\.466\\ln 32=3\.466and leaked the target, since that histogram was computed from the window being predicted\)\. Instead we compare two predictors: one given only a leave\-one\-out estimateRlooR\_\{\\mathrm\{loo\}\}of the recording’s code distribution, computed from that recording’s*other*windows, and one given\[Rloo,XK\]\[R\_\{\\mathrm\{loo\}\},X\_\{K\}\]\. The reported quantity is the difference,CE⁡\(identity\)−CE⁡\(identity\+K\)\\mathrm\{CE\}\(\\text\{identity\}\)\-\\mathrm\{CE\}\(\\text\{identity\}\+K\): whatKKseconds of history are worth once that estimate has been accounted for\. One limitation of this control is worth stating\. Windows are cut at half\-window stride, so a neighbouring window may overlap the interval the current one is asked to predict, andRlooR\_\{\\mathrm\{loo\}\}may therefore contain information about the target and not only about the recording\. Both predictors receive it, and we do not claim a direction for the residual bias, so we read these numbers as exploratory rather than as a leakage\-free estimate of what history contributes\. Within each row, all values ofKKuse the same estimate, which holds the conditioning statistic fixed but does not eliminate possible overlap\-induced bias in comparisons across history lengths\.

##### Implementation\.

The codebook is clustered by4040iterations of sphericalkk\-means, cosine similarity coinciding with the inner product because the codebook isℓ2\\ell\_\{2\}\-normalised\. Each window supplies256256past steps; the featureXKX\_\{K\}is the normalised histogram of the lastK∈\{8,16,32,64,128,256\}K\\in\\\{8,16,32,64,128,256\\\}of them \(44to128128s\), and the targetYHY\_\{H\}the normalised histogram of the88steps \(44s\) beginningH∈\{2,20,60,120\}H\\in\\\{2,20,60,120\\\}steps \(11,1010,3030,6060s\) after the end of the past region\. Each predictor is a single linear layer with zero\-initialised weights, trained full\-batch for300300Adam steps \(learning rate0\.050\.05, weight decay10−410^\{\-4\}\) on soft\-label cross\-entropy, with one fifth of recordings held out\. We report the best held\-out value reached during that optimisation\. These are validation\-selected scores, descriptive rather than unbiased estimates, and we read the difference between two predictors selected the same way rather than either score on its own\. Of the19,20019\{,\}200windows collected,17,02617\{,\}026come from recordings contributing at least two windows and are usable for the leave\-one\-out estimate\.

Table 12:Exploratory history\-probe results conditioned on recording\-level statistics\.Rows are prediction horizons and columns the length of history supplied\.Marginalis a predictor given no input andIdentityone given only a leave\-one\-out estimate of the recording’s own code distribution; the first two columns are their absolute held\-out cross\-entropies in nats, and the remaining six areCE⁡\(Identity\)−CE⁡\(Identity\+K\)\\mathrm\{CE\}\(\\textsc\{Identity\}\)\-\\mathrm\{CE\}\(\\textsc\{Identity\}\{\+\}K\), the reduction in cross\-entropy from addingKKseconds of history to the same recording\-level estimate\. Best denotes the history lengthKKwith the largest improvement, also marked inbold\. Measured on19,20019\{,\}200windows drawn from4,6004\{,\}600pre\-training recordings, of which17,02617\{,\}026windows support the leave\-one\-out estimate, split by recording\.Absolute CE\(↓\\downarrow\)Improvement overidentity\(↑\\uparrow\)forK=K=Predict aheadmarginalidentity𝟒\\mathbf\{4\}s𝟖\\mathbf\{8\}s𝟏𝟔\\mathbf\{16\}s𝟑𝟐\\mathbf\{32\}s𝟔𝟒\\mathbf\{64\}s𝟏𝟐𝟖\\mathbf\{128\}sBest11s3\.44353\.29110\.10480\.10900\.10970\.10700\.10380\.09821616s1010s3\.44183\.29180\.08920\.09520\.09880\.10010\.09840\.09483232s3030s3\.44333\.29220\.07610\.08350\.08910\.09150\.09200\.08956464s6060s3\.43983\.29130\.06950\.07560\.08170\.08540\.08650\.08546464s
##### Predictors compared\.

Three predictors are compared for each horizon on the same held\-out recordings\.marginalemits the mean training target without fitting, and is the highest value in every row of the table\.identityandidentity\+Kuse the linear\-softmax form and the training recipe above, and differ only in what they are given\.IdentityreceivesRlooR\_\{\\mathrm\{loo\}\}alone, whereasIdentity\+K\+Kadditionally receives the history histogramXKX\_\{K\}\. Table[12](https://arxiv.org/html/2609.38587#A2.T12)reports the first two as absolute cross\-entropies and the third as its improvement overidentity, from which every absolute value can be recovered: for the11s probe horizon and1616s of history,3\.2911−0\.1097=3\.18143\.2911\-0\.1097=3\.1814\.

##### Scope of the data\.

A window must span the128128s past region, the furthest prediction offset and the target interval, so192192s in total, which admits18,86618\{,\}866of the22,78022\{,\}780training recordings \(§[4\.1](https://arxiv.org/html/2609.38587#S4.SS1)\); the three corpora excluded outright by this filter are trial\-structured and too short to carry long\-range structure in any case\. The surviving pool is dominated by continuous sleep and clinical recordings, four corpora accounting for ninety\-two percent of it, so the table below should be read as a statement about long continuous EEG rather than about all EEG, which is also the regime that the long\-context claims of §[5\.2](https://arxiv.org/html/2609.38587#S5.SS2)concern\. Windows are drawn at random from that pool rather than from the head of the manifest\.

##### Magnitudes\.

Relative toMarginal,Identityreduces cross\-entropy by0\.1480\.148–0\.1520\.152nats across the four horizons\. The best history\-conditioned predictor at each horizon achieves a further reduction of0\.0860\.086–0\.1100\.110nats, while extending history beyond44s adds0\.0050\.005–0\.0170\.017nats\. These values describe the performance of linear probes on coarse code histograms under the stated conditioning and validation\-selection procedure\. Given the possible target overlap, they should not be interpreted as an isolated recording\-identity effect or a leakage\-free estimate of the contribution of history\.

##### Reading\.

In this exploratory probe, the best\-performing history length shifts from1616to3232,6464, and6464s as the prediction horizon increases\. Gains from additional history saturate earlier for near\-term prediction, whereas longer horizons benefit from more context\.

##### For the pre\-training objective, the window is not the binding constraint\.

At the shortest probe horizon of11s, gains from longer history largely saturate within3232s\. This provides indirect support for our pre\-training window length, since the probe predicts a44s histogram of coarse codes rather than the next per\-channel code\. Consistent with this finding, extending pre\-training sequences from3232to128128s did not improve performance on any downstream task\. A separate probe that holds the target codes fixed and varies how many past steps a trained backbone may condition on agrees independently: a model pretrained on128128s sequences fails to beat one pretrained on3232s sequences at every history length up to6464s\. We do not shorten the sequence below3232s either, for a reason internal to the architecture rather than to this table: at6464steps the memory bank is written1616times and evicts88times, whereas a1616s sequence would produce exactly88writes and never exercise eviction at all, leaving the model unexposed to memory\-bank turnover during pre\-training\.

##### For downstream tasks, context length does matter\.

The useful context length for downstream tasks is evaluated directly in §[5\.2](https://arxiv.org/html/2609.38587#S5.SS2)\. Sleep staging is such a task, and the objective we pretrain with is not: the model is trained on a local prediction problem and then asked, at evaluation time, to carry state across intervals far longer than any it saw during pre\-training\. §[5\.2](https://arxiv.org/html/2609.38587#S5.SS2)measures what that is worth, and the fact that it is worth anything is a statement about the recurrent state of §[3\.2\.2](https://arxiv.org/html/2609.38587#S3.SS2.SSS2), not about the length of the pre\-training window\.

## Appendix CDownstream Tasks and Metrics

We evaluate on five datasets in two families \(Table[13](https://arxiv.org/html/2609.38587#A3.T13)\): three*window*tasks, where one segment of a few seconds carries one label, and two*sequence*tasks, where a recording is labelled unit by unit in order: a*unit*being the interval the annotation is defined on, a3030s scored epoch for Sleep\-EDF and a3030s interval labelled from the seizure annotations for CHB\-MIT\. None appears in our pre\-training corpus\.

Table 13:The five downstream tasks\.The three above the rule are window tasks and the two below are sequence tasks\. Unit is the interval one label is defined on, \#Ch\. the electrodes used and Rate their sampling rate in Hz;LLis the number of consecutive units per sample, soL=1L=1is a single labelled segment;KKis the readout budget, fixed per dataset and identical across models \(Appendix[D\.2\.1](https://arxiv.org/html/2609.38587#A4.SS2.SSS1)\); Selection is the validation criterion a checkpoint is chosen on, and Split how the data is partitioned\.†Not a cross\-validation \(Appendix[C\.2](https://arxiv.org/html/2609.38587#A3.SS2)\)\.‡Folds are over the 24 case folders, which come from 23 subjects; see Appendix[C\.5](https://arxiv.org/html/2609.38587#A3.SS5)DatasetTaskUnit\#Ch\.Rate𝐋\\mathbf\{L\}𝐊\\mathbf\{K\}SelectionSplitFACED\([Chen et al\., 2023](https://arxiv.org/html/2609.38587#bib.bib13)\)Emotion,99\-class1010s30302502501119,20019\{,\}200κ\\kappasubject, fixedKaggleERN\([Mattout et al\., 2014](https://arxiv.org/html/2609.38587#bib.bib16);[Margaux et al\., 2012](https://arxiv.org/html/2609.38587#bib.bib15)\)ERN, binary22s1919200200113,8403\{,\}840ROC\-AUCsubj\.,44folds†SEED\-VIG\([Zheng and Lu, 2017](https://arxiv.org/html/2609.38587#bib.bib20)\)Vigilance, regression88s17172002001119,20019\{,\}200R2R^\{2\}session, fixedSleep\-EDF\([Kemp et al\., 2000](https://arxiv.org/html/2609.38587#bib.bib17);[Goldberger et al\., 2000](https://arxiv.org/html/2609.38587#bib.bib18)\)Sleep stage,55\-class3030s22100100202012,00012\{,\}000κ\\kappasubject,55\-foldCHB\-MIT\([Goldberger et al\., 2000](https://arxiv.org/html/2609.38587#bib.bib18);[Shoeb, 2009](https://arxiv.org/html/2609.38587#bib.bib44)\)Seizure, binary3030s1515256256202090,00090\{,\}000AUC\-PRcase,55\-fold‡### C\.1FACED

Nine\-class cross\-subject emotion recognition, under the protocol of CBraMod\([Wang et al\., 2025](https://arxiv.org/html/2609.38587#bib.bib3)\), on a corpus of3232\-channel recordings from123123subjects watching2828emotion\-eliciting clips\([Chen et al\., 2023](https://arxiv.org/html/2609.38587#bib.bib13)\)\. We use the officially released preprocessing, in which every trial is3030s at250250Hz, and apply no further filtering, artefact rejection or re\-referencing\. Two of the3232channels are mastoid electrodes, HEOR and HEOL, and are discarded, leaving the3030EEG channels of the reference protocol; each trial is cut into three non\-overlapping1010s segments, giving8484segments per subject and10,33210\{,\}332in total\. Subjects are sorted by identifier and partitioned subject\-independently into the first8080for training \(6,7206\{,\}720segments\), the next2020for validation \(1,6801\{,\}680\) and the last2323for test \(1,9321\{,\}932\); the split is deterministic and identical for every model and seed\. Class proportions are fixed by the stimulus design rather than by sampling and are identical across the three splits\. We report balanced accuracy and Cohen’sκ\\kappaover five seeds\. Balanced accuracy averages per\-class recall, so any input\-independent predictor scores exactly1/91/9; plain accuracy would instead depend on which degenerate strategy a collapsed model settled into\. Two cautions about the test set: with the split fixed, seeds vary only initialisation and batch order, so the reported dispersion is optimisation variance and not variance across subjects; and the1,9321\{,\}932test segments come from644644subject\-video trials,2323subjects by2828videos, so the three segments of one trial should not be treated as independent\.

### C\.2KaggleERN

Binary detection of error\-related negativity, under the protocol of EEGPT\([Wang et al\., 2024](https://arxiv.org/html/2609.38587#bib.bib9)\), on the corpus of the BCI Challenge @ NER 2015, in which each item selection in a P300 speller is followed by a visual feedback the subject may perceive as erroneous\([Mattout et al\., 2014](https://arxiv.org/html/2609.38587#bib.bib16);[Margaux et al\., 2012](https://arxiv.org/html/2609.38587#bib.bib15)\)\. The release holds5656\-channel EEG from2626subjects at200200Hz,340340feedback\-locked trials each,8,8408\{,\}840in total, with the competition’s own partition into1616labelled subjects \(5,4405\{,\}440trials\) and1010held\-out test subjects \(3,4003\{,\}400\); the classes are imbalanced at70\.8%70\.8\\%and70\.9%70\.9\\%positive respectively\. Trials are epoched from−0\.7\-0\.7to\+1\.3\+1\.3s and we keep the1919\-channel standard1010–2020subset of the reference protocol, with no band\-pass, baseline correction or re\-referencing, again following it\.

Two departures are worth stating\. The four folds are not a cross\-validation: all four score on the same fixed ten test subjects and differ only in which group of four labelled subjects is removed from training, so their dispersion measures sensitivity to the training subset and not variance across test subjects\. Our four training sets match the published fold dictionary exactly, and the seed is fixed so that the dispersion is not confounded with initialisation\. Second, the reference uses the ten test subjects as its validation set, which combined with checkpoint selection on a validation criterion would make every reported number a best\-epoch\-on\-the\-reported\-set\. We instead hold out15%15\\%of the training subjects’ trials, stratified by subject and label, giving roughly3,468/612/3,4003\{,\}468/612/3\{,\}400trials per fold, with the split keyed on the fold index so that all models see the same partition\. We report ROC\-AUC, the metric of the competition and of the reference protocol, alongside AUC\-PR\.

Ground\-truth labels for the ten test subjects are not distributed by the competition; we use the publicly circulated label file that the reference implementation instructs the user to supply, and report two consistency checks on it rather than an independent verification of its provenance\. Its positive rate is0\.70910\.7091against0\.70770\.7077in the released training labels, and scoring the competition winner’s two published submissions against it yields ROC\-AUC0\.86790\.8679and0\.84390\.8439, where a label file unrelated to the data would score near0\.50\.5\. These are consistency checks and not an independent confirmation of where the file came from\.

### C\.3SEED\-VIG

Continuous vigilance regression, under the protocol of CBraMod\([Wang et al\., 2025](https://arxiv.org/html/2609.38587#bib.bib3)\), from1717\-channel EEG at200200Hz recorded during simulated driving, with a PERCLOS value in\[0,1\]\[0,1\]derived from eye tracking once per88s window\([Zheng and Lu, 2017](https://arxiv.org/html/2609.38587#bib.bib20)\)\. The corpus holds2323sessions from2121subjects \(two subjects contributed two sessions\) of885885windows each,20,35520\{,\}355in total, and the target is a scalar per window rather than a sequence\. We follow the reference protocol, ordering sessions by numeric index and taking the first1515for training, the next44for validation and the last44for test \(13,275/3,540/3,54013\{,\}275/3\{,\}540/3\{,\}540\); every subject falls entirely within one split\. We report Pearson correlation andR2R^\{2\}over five seeds, and select on validationR2R^\{2\}\.

The session ordering is consequential, and the reference is not self\-consistent about it\. CBraMod’s text describes the split as “subject 1 to 15 for training, subject 16 to 19 for validation and subject 20 to 23 for test”\([Wang et al\., 2025](https://arxiv.org/html/2609.38587#bib.bib3)\), which can only be read as file indices in numeric order, there being2323sessions from2121subjects\. Its released preprocessing code instead sorts filenames lexicographically, and because those filenames are not zero\-padded, sessions10–19sort ahead of session2; the first fifteen files are then1,10,…,19,2,20,21,221,10,\\dots,19,2,20,21,22rather than1,…,151,\\dots,15, which moves both ends of every split\. A reproducer following the text and a reproducer running the code do not evaluate on the same data\.

We resolved this from published results rather than by choosing\. BecauseR2=1−MSE/VarR^\{2\}=1\-\\mathrm\{MSE\}/\\mathrm\{Var\}, any paper reporting bothR2R^\{2\}and RMSE pins the variance of its own test labels atVar=RMSE2/\(1−R2\)\\mathrm\{Var\}=\\mathrm\{RMSE\}^\{2\}/\(1\-R^\{2\}\)\. Four published rows, three from CBraMod’s own comparison table\([Wang et al\., 2025](https://arxiv.org/html/2609.38587#bib.bib3)\)and one from a later model\([Zhou et al\., 2026b](https://arxiv.org/html/2609.38587#bib.bib21)\), imply a test\-label standard deviation of0\.31750\.3175–0\.31760\.3176\. We measure0\.31760\.3176under numeric ordering and0\.1750\.175under lexicographic\. The published numbers therefore follow the text and not the released code, and we follow the published numbers\.

The difference is not a detail\. Under lexicographic ordering three of the four test sessions are almost flat, soR2R^\{2\}is measured against a very small denominator: predicting the training mean scores−0\.413\-0\.413there, against−0\.019\-0\.019under numeric ordering\. A reproducer who ran the released code would obtain implausibly poor numbers for every model, ours included, with nothing in the output to indicate why\.

AnR2R^\{2\}on this task is easy to misread, so we give three reference points\. A predictor that always emits the training mean scoresR2=−0\.019R^\{2\}=\-0\.019, at RMSE0\.32060\.3206\. One that cheats and emits the*test*mean scoresR2=0R^\{2\}=0by construction, at RMSE0\.31760\.3176\. The best published methods reachR2≈0\.24R^\{2\}\\approx 0\.24–0\.340\.34, at RMSE≈0\.26\\approx 0\.26\. The entire competitive range is thus an RMSE from0\.320\.32down to0\.260\.26: absolute errors move very little, and anR2R^\{2\}of a third is a strong result here rather than a weak one\.

Two further properties are worth stating\. The obvious shortcut is closed:90\.2%90\.2\\%of the test labels’ variance lies*within*a session and only9\.8%9\.8\\%between sessions, so a model cannot score by learning how drowsy each driver is on average, it has to follow the change over minutes\. And a regression head has a degenerate solution, a constant output, which yieldsR2≈0R^\{2\}\\approx 0and an undefined correlation while looking from the outside like ordinary underperformance; we checked for it, and none of the thirty runs collapsed, the lowest single\-run correlation being0\.4540\.454\.

### C\.4Sleep\-EDF

Five\-class sleep staging on the Sleep Cassette subset\([Kemp et al\., 2000](https://arxiv.org/html/2609.38587#bib.bib17);[Goldberger et al\., 2000](https://arxiv.org/html/2609.38587#bib.bib18)\), preprocessed as in EEGPT\([Wang et al\., 2024](https://arxiv.org/html/2609.38587#bib.bib9)\),153153whole\-night polysomnograms from7878subjects, of which we use only the two EEG derivations, Fpz–Cz and Pz–Oz at100100Hz\. Recordings are ambulatory and run for a median of2323hours, so most of the raw signal is daytime wakefulness; following the convention of the field we retain3030minutes of wake either side of the first and last non\-wake epoch\. Annotations are mapped to the five AASM classes with stages 3 and 4 merged, and movement and unscored epochs discarded, leaving195,469195\{,\}469epochs of3030s, which reproduces the standard7878\-subject configuration to within ten epochs and matches its per\-class counts for N1, N2, N3 and REM exactly\. Sequences are cut with strideLLso that no sequence crosses a night and every epoch is predicted exactly once; a recording whose epoch count is not a multiple ofLLyields one short final sequence, zero\-padded, with padded positions carrying an ignore label\.

The two EEG signals are bipolar derivations, whereas four backbones address channels by electrode identity and no lookup table contains an entry for a derivation pair\. We name each channel by the leading electrode of its pair, Fpz–Cz→\\toFpz and Pz–Oz→\\toPz, and verified that both names resolve in every table before training; this costs no parameters and adds no layer\. We considered and rejected the alternative used by the reference sleep pipeline, a learned1×11\\times 1convolution from two channels to the backbone’s channel count, because that places a randomly initialised layer in front of every pre\-trained backbone, precisely the configuration that drives a constant\-output solution \(Appendix[D\.3](https://arxiv.org/html/2609.38587#A4.SS3)\), and with two input channels the mixing matrix has rank at most two, so the risk is higher here rather than lower\. BIOT retains its own learned channel mixing, which is intrinsic to that model on every dataset because its pre\-trained channel tokens describe montage pairs rather than electrodes\.

Together with CHB\-MIT, this is one of the two datasets whose partition is not fixed by a reference protocol, and therefore one of the two that report cross\-subject rather than optimisation variance\. We use subject\-wise five\-fold cross\-validation: the7878subjects are permuted once with a fixed seed and split into groups of16/16/16/15/1516/16/16/15/15; foldkktests on groupkk, validates on10%10\\%of the remaining subjects drawn with a fold\-derived seed so that all models see the same partition, and trains on the rest\. Both nights of a subject always move together\. We report macro\-F1 andκ\\kappaas the mean and standard deviation of the five per\-fold test scores, and select on validationκ\\kappa, as on FACED\.

### C\.5CHB\-MIT

Binary seizure detection on980980hours of continuous paediatric scalp EEG from2323subjects across2424case folders, distributed as683683segments of roughly one hour\([Goldberger et al\., 2000](https://arxiv.org/html/2609.38587#bib.bib18);[Shoeb, 2009](https://arxiv.org/html/2609.38587#bib.bib44)\)\. Four properties of the release change the resulting benchmark and none of them raises an error, so a pipeline that gets them wrong produces a plausible dataset rather than a failure\.

##### Seizure annotations appear in two formats\.

The summary files record onsets either asSeizure Start Time:or asSeizurennStart Time:, and which form is used varies by subject rather than by how many events a file contains:chb08writesSeizure 1 Start Time:for files holding a single seizure, whilechb01andchb24never number theirs\. A parser matching only the unnumbered form recovers4040of the198198annotated seizures and reports no error\. The remaining158158are not dropped but silently relabelled interictal, so the model is trained on seizure activity presented as normal, and the positive rate falls from0\.456%0\.456\\%to below one in a thousand\.

##### Amplitudes are stored in volts\.

Read without conversion the signals are six orders of magnitude below the microvolt\-scaled data of every other corpus we use, and nothing in the pipeline would raise an error on it\.

##### Channel names collide\.

The segments carry three incompatible bipolar montages\. Backbone channel lookups are keyed by one electrode name, so keying on the first electrode of each derivation collidesFP1\-F3withFP1\-F7andFP2\-F4withFP2\-F8; we resolve each in favour of the temporal chain, leaving1515distinct electrodes\. Separately, threechb12segments holding1313seizures are recorded against a common reference rather than in the bipolar chain used elsewhere and are excluded, leaving683683of686686segments and185185of198198seizures \(93\.4%93\.4\\%\), so that a difference in signal type is not confounded with the class\. Preparation yields117,553117\{,\}553units of3030s over1515channels, of which0\.456%0\.456\\%contain seizure activity\.

##### The validation split needs four subjects, not two\.

Splits are five\-fold over case folders, so every unit is tested exactly once\. At a positive rate below one percent the usual ten\-percent validation rule is too small to select on: two validation subjects give folds holding as few as2828positive units, and on exactly those folds two different baseline architectures both fall below the random AUC\-PR baseline of0\.00460\.0046, the same folds for both models, which identifies the split rather than the models\. We instead allocate four, chosen by enumerating all four\-subject combinations and taking the one whose share of positives is closest to twenty percent, with ties broken in favour of subjects not yet used for validation; validation positives rise to7777–9191\.

##### Two case folders are one subject\.

The release contains2424case folders from2323subjects:chb21was recorded from the same person aschb01, a year and a half later\. Our folds are drawn over case folders, so that subject is treated as two, and in two of the five folds one of its folders is in the test set while the other is in training; every other subject appears on one side only\. We measured what this is worth by removing the shared subject’s test units from those two folds and recomputing the AUC\-PR of every row of Table[3](https://arxiv.org/html/2609.38587#S5.T3)\. It moves by at most0\.0180\.018on the fifteen\-channel montage and by at most0\.0090\.009on the two\-channel one: NeurDuo \(Small\) from0\.4130\.413to0\.4090\.409, Base from0\.3920\.392to0\.3840\.384, Large from0\.4710\.471to0\.4760\.476, the strongest classical pipeline from0\.2850\.285to0\.2840\.284and the strongest pre\-trained baseline from0\.2080\.208to0\.2170\.217\. Every NeurDuo variant retains its AUC\-PR advantage over every baseline on both montages\.

##### AUROC alone is not safe here\.

Segments are long and seizures cluster within them, so knowing*which recording*a window came from is by itself predictive of its label\. An oracle that never sees the signal and is told only whether a unit’s recording contains a seizure reaches AUROC0\.9050\.905averaged over our five test folds but AUC\-PR only0\.0240\.024\. Recording\-level label structure therefore settles most of the AUROC and almost none of the precision–recall area\. It is not a baseline: it uses information no model receives, and what it measures is how much of each metric is decided by which recording a unit belongs to rather than by the signal in it\. We therefore select on validation AUC\-PR and report AUC\-PR and false alarms per hour, with the random AUC\-PR baseline of0\.00460\.0046alongside\.

##### How a false alarm is counted\.

For each evaluation fold, the threshold is the median score of the positive units, taken halfway between the two central scores when their number is even, so that at least50%50\\%of positive units, not seizure events, reach it\. The units of a fold are scored as one sequence, recording after recording, and consecutive predicted\-positive units are merged into one alarm, which counts as false if it overlaps no seizure\-labelled unit; predicted\-positive units on either side of the boundary between two consecutive recordings therefore form a single alarm\. FP⋅\\cdoth\-1is computed by dividing the total false alarms by the total recording hours across all five folds\. Because thresholds are selected on the evaluated folds rather than on validation data, this metric compares models at a common sensitivity target rather than measuring deployment performance with a fixed threshold\.

##### Comparison with prior work\.

[Durgam et al\. \(2026\)](https://arxiv.org/html/2609.38587#bib.bib8)is the closest prior work, a causal state\-space model for streaming EEG with its own persistent\-state ablation on this corpus, and we take its metrics and its choice not to filter recordings\. We do not take its split, which is file\-level random and so places recordings of one patient on both sides, an arrangement that, given a shortcut worth AUROC0\.9050\.905, is considerably easier than our folds, which keep whole recordings and, with the one exception noted above, whole subjects on one side\. Its window is55s against our600600\. The published numbers above are quoted as the scale of the benchmark, not as a comparison\.

## Appendix DBaselines and Evaluation Protocol

### D\.1Baselines, and what each saw before it saw the task

Table[14](https://arxiv.org/html/2609.38587#A4.T14)lists the nine baselines of §[4\.2](https://arxiv.org/html/2609.38587#S4.SS2)\. The six pre\-trained models span the design space of this class deliberately: backbones from3\.23\.2to69\.269\.2million parameters, pre\-training corpora from a single clinical archive to9292aggregated datasets, and tokenizers dividing the signal at granularities from0\.1250\.125to11s per channel\. The three classical pipelines span the three feature families classical EEG decoding exploits and are not head\-matched, a classical pipeline having no separable backbone and readout\.

Table 14:The nine baselines\.The six above the rule are pre\-trained foundation models and the three below are decoders that were never pre\-trained\. \#Params is the pre\-trained backbone as loaded, excluding pre\-training\-only components such as decoders and projection heads; a classical pipeline has no separable backbone and readout, so the column is left blank\. Pre\-training reproduces each release’s own account of its corpus, which is why the entries are not stated on a common footing\.BaselineArchitecture / feature family\#ParamsPre\-trainingBIOTLinear\-attention transformer over per\-channel STFT patches3\.23\.2MMGH, SHHS, TUAB, TUEV, CHB\-MIT, IIICCBraModCriss\-cross transformer over the channel×\\timestime grid4\.94\.9MTUEG,\>9,000\{\>\}9\{,\}000hLaBraM\-BaseVector\-quantised tokenizer with masked\-token prediction5\.85\.8M1616corpora,\>2,500\{\>\}2\{,\}500hEEGPT\-LargeViT with dual mask\-reconstruction and alignment25\.325\.3MPhysioNet\-MI, TSUBenchmark, M3CV, SEEDST\-EEGFormer\-SViT with masked autoencoding on raw EEG25\.425\.4M\>8\{\>\}8M segments; MI, P300, SSVEPREVE\-BaseViT with 4D Fourier encoding of electrode geometry69\.269\.2M9292datasets,\>60,000\{\>\}60\{,\}000h,∼25,000\{\\sim\}25\{,\}000subjectsBP\-GBDTSpectral power: band power, differential entropy and shape statistics→\\togradient boosting—noneFBCov\-TS\-LinSpatial covariance: filter\-bank covariances→\\toRiemannian tangent space→\\tolinear—noneERP\-Lin†Evoked waveform: band\-limited, decimated, standardised windows→\\tolinear—none
†Adapted from the evoked\-response pipeline of[Yang et al\. \(2026\)](https://arxiv.org/html/2609.38587#bib.bib14)by dropping its supervised spatial filter, which requires discrete classes, so that one recipe spans our classification, regression and sequence\-labelling tasks; the band\-pass is unchanged, and so is the decimation rate except on Sleep\-EDF as described above\.

#### D\.1\.1The classical pipelines

These three exist to answer a question the shared head cannot: whether any pre\-trained model beats a decoder with no pre\-training at all\. They are deliberately not head\-matched, since a classical pipeline has no separable backbone and readout, but everything defining the task is matched\. They read the same split files by the same rules, including the fold\-derived stratified validation draw of KaggleERN, so the validation trials are literally the same ones\. Their metric functions are the samesklearncalls as the neural evaluator, key for key\. And their selection criterion is the same: where a fine\-tuned model selects an epoch, each of the two linear pipelines selects one regularisation scalar on validation from a log\-spaced ladder \(C∈\[10−8,103\]C\\in\[10^\{\-8\},10^\{3\}\]for classification,α∈\[10−2,1010\]\\alpha\\in\[10^\{\-2\},10^\{10\}\]for regression\)\.

##### BP\-GBDT \(spectral power\)\.

Welch spectra give twelve descriptors per channel: differential entropy in five bands \(11–44,44–88,88–1313,1313–3030,3030–4545Hz\), log broadband power, log standard deviation, skewness, kurtosis, Hjorth mobility and complexity, and spectral entropy scaled to\[0,1\]\[0,1\]\. Differential entropy under a Gaussian is one half of the logarithm of band power up to an additive constant, the standard feature of the emotion and vigilance literature; the statistical and entropy terms are what a sleep\-staging feature set needs\([Vallat and Walker, 2021](https://arxiv.org/html/2609.38587#bib.bib22);[Kastrati et al\., 2025](https://arxiv.org/html/2609.38587#bib.bib23)\)\. Features go to LightGBM at300300trees and learning rate0\.050\.05\. This pipeline has no grid, so it cannot benefit from validation selection at all\.

##### FBCov\-TS\-Lin \(spatial covariance\)\.

Band\-pass into the same five bands, an OAS\-shrunk covariance matrix per band, projection to the Riemannian tangent space, and a linear model\([Jayaram and Barachant, 2018](https://arxiv.org/html/2609.38587#bib.bib24);[Sabbagh et al\., 2019](https://arxiv.org/html/2609.38587#bib.bib25)\)\.

##### ERP\-Lin \(evoked waveform\)\.

Band\-pass to11–2020Hz, decimate to3232Hz, per\-channel per\-window standardisation, flatten,ℓ2\\ell\_\{2\}\-regularised linear model\. One dataset forces a deviation, recorded in the result files rather than applied silently: Sleep\-EDF decimates to1616Hz because the context rule below triples its feature vector\.

##### Sequence tasks\.

On the sequence tasks a classical pipeline has no recurrence, so we give it context explicitly: each unit’s feature vector is concatenated with the mean of the ten units before it and the mean of the ten after, within the same recording\. This gives the classical pipelines bidirectional context, an advantage over the causal context head of the neural models; units near the edges of a recording use their own features in place of a missing side\.

#### D\.1\.2Pre\-training overlap

Interpreting transfer performance requires accounting for each model’s pre\-training data\. We audited the pre\-training corpus of every model, our own included, against every downstream dataset\. Table[15](https://arxiv.org/html/2609.38587#A4.T15)summarises; the paragraphs below give the cases that are not simply clean\. We distinguish two kinds of contamination, because they are not equally serious\.*Corpus overlap*means that a downstream corpus is included in a model’s pre\-training data\. Whether that includes the labels, and which of the recordings we test on it covers, depends on the released checkpoint and on the split its authors used, so we state both per case below\.*Domain overlap*means the pre\-training data contains a different corpus of the same kind \(the same recording paradigm, montage class and clinical population\) which confers an advantage that is real but is the ordinary business of pre\-training rather than leakage\.

Table 15:Pre\-training overlap by model and task\.Every model’s pre\-training corpus, ours included, was audited against every downstream dataset\.∙\\bulletmarks corpus overlap, the downstream corpus itself appearing in that model’s pre\-training set; it occurs twice and both cases are discussed below\.∘\\circmarks domain overlap, a different corpus of the same recording paradigm and population, which is the ordinary condition of pre\-training at scale rather than leakage\. Blank means neither\. The three classical pipelines of Table[14](https://arxiv.org/html/2609.38587#A4.T14)have no row here, never having been pre\-trained\.ModelFACEDKaggleERNSEED\-VIGSleep\-EDFCHB\-MITBIOT∘\\circ∙\\bulletCBraMod∘\\circLaBraM\-Base∘\\circ∙\\bullet∘\\circEEGPT\-Large∘\\circST\-EEGFormer\-SREVE\-Base∘\\circ∘\\circNeurDuo \(ours\)∘\\circ∘\\circ##### KaggleERN and LaBraM: corpus overlap\.

LaBraM’s pre\-training set lists the Inria BCI Challenge as the fourth of its sixteen corpora: KaggleERN\. This is not domain similarity but the same corpus, and since the competition releases the raw EEG of all twenty\-six subjects while LaBraM’s pre\-training is unlabelled self\-supervision, its encoder has most likely seen the signals of our ten test subjects, though never their labels\. No overlap\-free LaBraM checkpoint exists; we mark the row and do not exclude it, since removing the model would remove a reference point the rest of the field uses\.

##### CHB\-MIT and BIOT: corpus overlap\.

We load the released ’EEG\-six\-datasets\-18\-channels’ checkpoint, which its authors describe as pre\-trained on resting and sleep corpora together with the training sets of TUAB, TUEV, IIIC and CHB\-MIT\. The CHB\-MIT part is supervised: in the released pre\-training script that corpus enters the loss through its seizure labels, and its split is by case folder, withchb01tochb20in training\. Twenty of the twenty\-four case folders we evaluate on are therefore folders this checkpoint was trained on with labels, covering88\.6%88\.6\\%of our test units and87\.1%87\.1\\%of our test positives\. We mark the row rather than drop it, and we keep the same checkpoint across every task so that the BIOT of this table is the BIOT of the other four; we do not substitute the overlap\-freeEEG\-PRESTcheckpoint for the same reason\. What the row should be read as is a baseline that was given labelled seizure supervision on most of the subjects it is scored on here\.

### D\.2Downstream protocol

Published protocols are reproduced here where they exist rather than invented, and we name them once so that the shorthand below is unambiguous\. FACED and SEED\-VIG follow CBraMod\([Wang et al\., 2025](https://arxiv.org/html/2609.38587#bib.bib3)\), and KaggleERN follows EEGPT\([Wang et al\., 2024](https://arxiv.org/html/2609.38587#bib.bib9)\); the per\-model input conventions and the warm\-up schedule follow[Yang et al\. \(2026\)](https://arxiv.org/html/2609.38587#bib.bib14)\. Sleep\-EDF takes its wake cropping from EEGPT but uses our own subject\-wise split \(Appendix[C\.4](https://arxiv.org/html/2609.38587#A3.SS4)\), and CHB\-MIT takes its choice of recordings and metrics from CaMBRAIN\([Durgam et al\., 2026](https://arxiv.org/html/2609.38587#bib.bib8)\)but uses our own split and selection \(Appendix[C\.5](https://arxiv.org/html/2609.38587#A3.SS5)\)\. Where a paragraph says*the reference*, it means whichever of these protocols governs the dataset under discussion\. Deviations from a reference are stated where they occur and are applied identically to every model\.

#### D\.2\.1The shared readout

##### Structure\.

A backbone exposestokens⁡\(x\)∈ℝB×N×D\\mathrm\{tokens\}\(x\)\\in\\mathbb\{R\}^\{B\\times N\\times D\}; everything after that point is common to all models\. The head isLN\\mathrm\{LN\}, a per\-token linear mapD→dtokD\\to d\_\{\\mathrm\{tok\}\}with no activation, a flatten of the whole grid toN​dtokNd\_\{\\mathrm\{tok\}\}, thenLinear→LeakyReLU⁡\(0\.01\)→Linear\\mathrm\{Linear\}\\to\\mathrm\{LeakyReLU\}\(0\.01\)\\to\\mathrm\{Linear\}with a hidden width of256256and dropout of0\.10\.1before each of the two linear layers\. The grid is flattened rather than pooled: pooling to one vector per window is the capacity bottleneck the shared head exists to remove, and reintroducing it would restore the confound\. For a fixed budgetKK, tasks differ only in the width of the final layer, which moves at most2,0562\{,\}056parameters\. TheLN\\mathrm\{LN\}is best read as normalisation of the backbone interface rather than as head capacity, it costs2​D2Dparameters, about0\.02%0\.02\\%of the head, and it is not optional: measured on one FACED batch the tokens arriving at the head have standard deviations spanning more than an order of magnitude across backbones, one of them uncentred at mean\+1\.5\+1\.5, so without it a single head learning rate would correspond to very different effective rates per model\. The choice of LeakyReLU over the more usual rectifiers is also forced, for a reason given in Appendix[D\.3](https://arxiv.org/html/2609.38587#A4.SS3)\.

##### HowKKis set\.

Fixingdtokd\_\{\\mathrm\{tok\}\}would make head size track token count and reward whichever tokenizer emits the most tokens; we therefore fixK=N​dtokK=N\\,d\_\{\\mathrm\{tok\}\}and letdtok=round⁡\(K/N\)d\_\{\\mathrm\{tok\}\}=\\mathrm\{round\}\(K/N\)vary\. The per\-token width then differs across models:88dimensions for ST\-EEGFormer against120120for EEGPT on FACED\. But a token is not a comparable unit across models: an ST\-EEGFormer token covers0\.1250\.125s of one channel and a CBraMod token covers11s of one channel, so those are the same capacity per channel\-second\. The constraint that actually determinesKKon every dataset is that no model’sdtokd\_\{\\mathrm\{tok\}\}may exceed its widthDDand be silently clamped, since a clamp reintroduces exactly the spread the budget removes\. Where that constraint is slack the value comes from holding readout capacity per second of EEG fixed at the1,9201\{,\}920dimensions per second set a priori on FACED\. Where it binds,K=minm⁡Nm​DmK=\\min\_\{m\}N\_\{m\}D\_\{m\}\. Table[16](https://arxiv.org/html/2609.38587#A4.T16)gives the resulting values and what they buy\. Two consequences are worth naming\. On KaggleERN the constraint is not hypothetical: at19,20019\{,\}200the22s window leavesN=32N=32–5757tokens for four of the six baselines, theirdtokd\_\{\\mathrm\{tok\}\}clamps, and head sizes spread by a factor of2\.562\.56\(1\.991\.99–5\.095\.09M\) rather than1\.0451\.045\. And on SEED\-VIG the FACED value was carried over unchanged to an88s window, so that task runs at2,4002\{,\}400dimensions per second rather than1,9201\{,\}920; the constraint is slack there and nothing clamps, but we record the inconsistency rather than present a rule the numbers do not follow\.

Table 16:Readout budget per dataset and the head sizes it produces\.K=N​dtokK=Nd\_\{\\mathrm\{tok\}\}is the product of token count and per\-token width, held fixed per dataset so that head size does not track whichever tokenizer emits the most tokens\. Determined by names the rule that sets it: either a fixed readout capacity per second of EEG, or the no\-clamp boundminm⁡Nm​Dm\\min\_\{m\}N\_\{m\}D\_\{m\}where holding that capacity would push some model’sdtokd\_\{\\mathrm\{tok\}\}above its widthDmD\_\{m\}\. Head size is the range over the nine models that receive a shared head and Spread their ratio; the Sleep\-EDF and CHB\-MIT figures include the context head, which is identical across models and therefore compresses the ratio\.Dataset𝐊\\mathbf\{K\}Determined byHead sizeSpreadFACED19,20019\{,\}2001,9201\{,\}920dim/s at1010s4\.924\.92–4\.984\.98M1\.012×1\.012\\timesKaggleERN3,8403\{,\}8401,9201\{,\}920dim/s at22s1\.001\.00–1\.051\.05M1\.045×1\.045\\timesSEED\-VIG19,20019\{,\}200carried over from FACED4\.934\.93–5\.025\.02M1\.020×1\.020\\timesSleep\-EDF12,00012\{,\}000no\-clamp boundminm⁡Nm​Dm\\min\_\{m\}N\_\{m\}D\_\{m\}4\.754\.75–4\.944\.94M1\.041×1\.041\\timesCHB\-MIT \(15 Ch\)90,00090\{,\}000no\-clamp boundminm⁡Nm​Dm\\min\_\{m\}N\_\{m\}D\_\{m\}24\.8124\.81–24\.9524\.95M1\.006×1\.006\\timesCHB\-MIT \(2 Ch\)12,00012\{,\}000no\-clamp boundminm⁡Nm​Dm\\min\_\{m\}N\_\{m\}D\_\{m\}4\.754\.75–4\.944\.94M1\.041×1\.041\\times

#### D\.2\.2The context head

A sequence sample isLLconsecutive*units*of one recording, a unit being the interval one label is defined on: a3030s scored epoch for Sleep\-EDF and a3030s interval labelled from the seizure annotations for CHB\-MIT\. We say*unit*rather than*window*because a window in this paper is either the3232s pre\-training sequence of §[4\.1](https://arxiv.org/html/2609.38587#S4.SS1)or the single labelled segment of a window task, and rather than*epoch*because a training epoch is a pass over the corpus\. Each unit is encoded using the same backbone weights\. The baselines process units independently, whereas NeurDuo carries its recurrent state across unit boundaries\. That state is not detached at the boundaries during fine\-tuning, so the backbone is trained through them and gradients reach earlier units\. A shared head then converts each unit’s token grid into a 256\-dimensional representation, using its hidden activation as the output\. Finally, a context head takes the resulting\(B,L,256\)\(B,L,256\)sequence and predicts a label for each unit\. Using the shared head’s own hidden layer rather than a pooled vector is what carries the flatten\-the\-grid and equal\-KKproperties over to the sequence tasks unchanged\.

The context head we report is two Mamba blocks \(dmodel=256d\_\{\\mathrm\{model\}\}=256,dstate=16d\_\{\\mathrm\{state\}\}=16, convolution width44, expansion44\) followed by layer normalisation and a linear map at every position, with padded positions zeroed at the input and excluded from the loss\. It has1,753,8611\{,\}753\{,\}861parameters and is byte\-identical across all models\. It is causal, and it carries no positional embedding, the recurrence being ordered by construction\. We report this setting rather than the bidirectional one because a bidirectional readout recomputes over allLLunits whenever a unit arrives, which surrenders in the readout the streaming property the backbone was built to have; a comparison that allowed it would not be measuring what the paper claims\.

To assess the sensitivity of NeurDuo to the choice of context head, we also evaluated a bidirectional alternative: two Transformer encoder layers,88heads, feed\-forward width1,0241\{,\}024, a learned positional table of lengthLL,1,586,4371\{,\}586\{,\}437parameters on our own model, holding everything else fixed\. It is parameter\-matched to within11%11\\%, deliberately: an unmatched comparison would confound the cost of causality with the cost of capacity\. On Sleep\-EDF atL=20L=20with state carried, the causal head reaches macro\-F10\.7553±0\.00810\.7553\\pm 0\.0081and the bidirectional head0\.7532±0\.01500\.7532\\pm 0\.0150over the same five folds\. Sleep staging is a task where future context is genuinely informative and the standard sequence models for it are bidirectional, so we expected a loss and did not find one\.

#### D\.2\.3Input conventions

These models were pre\-trained at different sampling rates and in different amplitude units, so feeding all of them one identical tensor would place several out of distribution and would measure that rather than the quality of the representation\. Each therefore receives the signal resampled and rescaled to its own pre\-training convention, following the reference implementation and the model’s own release otherwise; the one exception is the amplitude scaling on KaggleERN, where the reference protocol prescribes a single convention that we apply to every model alike \(Appendix[C\.2](https://arxiv.org/html/2609.38587#A3.SS2)\)\. What is held identical is everything the task consists of: the same channels, the same segments, the same labels and the same splits; only the input representation differs, and all pre\-trained weights are loaded in full for every model\.

#### D\.2\.4Optimisation

All models are fine\-tuned end to end, every parameter, under one recipe: AdamW with weight decay0\.050\.05; backbone learning rate10−410^\{\-4\}, not batch\-size scaled; head learning rate10−3​b/25610^\{\-3\}\\sqrt\{b/256\}at batchbb, the square\-root rule of CBraMod’s released fine\-tuning code\([Wang et al\., 2025](https://arxiv.org/html/2609.38587#bib.bib3)\), which is5×10−45\\times 10^\{\-4\}atb=64b=64and3\.54×10−43\.54\\times 10^\{\-4\}atb=32b=32; three linear warm\-up epochs\([Yang et al\., 2026](https://arxiv.org/html/2609.38587#bib.bib14)\)then a per\-step cosine schedule annealed to10−610^\{\-6\}; gradient\-norm clipping at1\.01\.0\. Multi\-class and sequence\-labelling tasks use cross\-entropy with label smoothing 0\.1, the binary KaggleERN task binary cross\-entropy, and regression mean squared error\. We do not transplant each model’s published recipe, since those were tuned on the datasets of their own papers\.

Two models force a deviation, and both are handled by the recipe rather than by a separate decision\. ST\-EEGFormer runs at batch3232because its2,4002\{,\}400\-token sequence does not fit at6464, and at8080rather than160160units per step on the sequence tasks; its head learning rate follows from the same rule at the smaller batch, and we halve its batch on*every*dataset, including those where the full batch would fit, so that its batch stands in the same relation to the common recipe on every dataset rather than only where memory forces the deviation\. LaBraM’s temporal embedding table has1616rows at one second per patch and therefore cannot encode a3030s epoch in one pass; rather than truncate or pool the epoch we split it into two1515s sub\-windows, encode each, and concatenate the tokens, which leaves the token count, the budget and the head size unchanged and discards no sample\. What is lost is self\-attention across that internal boundary\. LaBraM alone pays it, it is forced by the size of a pre\-trained table rather than chosen, and we name it because it is a genuine asymmetry\.

The window tasks train for5050epochs at batch6464\. The sequence tasks budget the batch in*units*rather than in sequences: the loader takesmax⁡\(1,round⁡\(160/L\)\)\\max\(1,\\mathrm\{round\}\(160/L\)\)sequences, which is160160units per optimiser step at everyLLthat divides it,180180atL=60L=60and120120atL=120L=120, the two lengths where no integer number of sequences hits the budget\. We budget units rather than sequences because fixing the number of sequences instead would let the schedule move withLL: the data seen per step would scale asLL, the optimiser steps per epoch as1/L1/L, and the head learning rate, which followsb/256\\sqrt\{b/256\}in the batchbb, asL\\sqrt\{L\}\. The context\-length results would then be confounded with all three\. The sequence tasks hold many more samples than the window tasks, Sleep\-EDF about twenty times as many as FACED, so their number of passes is set by the gradient steps it produces rather than copied across: Sleep\-EDF and CHB\-MIT train for2020epochs, which gives17,40017\{,\}400–18,24018\{,\}240and7,8407\{,\}840–10,30010\{,\}300optimiser steps per fold, against2,7502\{,\}750,5,2505\{,\}250and10,40010\{,\}400for KaggleERN, FACED and SEED\-VIG at5050epochs\. Copying the window tasks’5050epochs would instead have given Sleep\-EDF more than43,00043\{,\}000steps per fold\. The remaining difference is not removed, and we note it rather than describe the budgets as matched\. ST\-EEGFormer, at half the batch, takes twice as many steps throughout\.

### D\.3The hidden activation of the shared head

The two\-layer MLP of the shared readout uses LeakyReLU\(0\.01\)\(0\.01\), and this choice is motivated by the activation comparison below\. BIOT is the only backbone in the comparison with a randomly initialised layer \(its30→1830\\to 18channel mixing\) in*front*of pre\-trained weights\. For the first epoch that layer passes almost no signal, so the cheapest available descent direction is to stop reading the input and emit the training class prior, and the cheapest way to realise a constant output is to drive all256256hidden units negative\. Under an activation whose gradient vanishes on the negative side the run cannot then recover: no gradient reaches the channel mixing, its features never improve, and the loss sits exactly at the class\-prior value\. We measured this on CPU:99of256256units are dead at initialisation,180180after fifteen steps and210210after thirty, after which the loss holds at2\.193412\.19341\(the class\-prior loss to five decimals\) for forty\-eight consecutive epochs\.

We vary only the shared head’s hidden activation and train each setting with five seeds, with warm\-up disabled so that the activation is the only safeguard\. LeakyReLU is the only tested activation for which all five runs escape the class\-prior solution \(Table[17](https://arxiv.org/html/2609.38587#A4.T17)\)\.

Table 17:Escape from the class\-prior solution on FACED with the BIOT backbone\.Five seeds per activation, warm\-up disabled, everything else held fixed\. A run that fails to escape drives all256256hidden units of the shared head negative and emits the training class prior thereafter, at which point balanced accuracy is exactly1/91/9, the chance level for nine classes\. Rows are ordered by how much gradient the activation leaves on the negative side, evaluated atx=−8x=\-8, a value typical of the collapsed units\. The recipe of the main tables adds three warm\-up epochs, which is why the LeakyReLU row reads0\.3540\.354here against the0\.3640\.364of Table[2](https://arxiv.org/html/2609.38587#S5.T2)\.Activation𝐝/𝐝​𝒙\\mathrm\{d\}/\\mathrm\{d\}xatx=−𝟖x=\-8Seeds escapingBalanced accuracyGELU00\(and<0\{<\}0forx<−0\.75x<\-0\.75\)0/50/50\.111±0\.0000\.111\\pm 0\.000ReLU001/51/50\.165±0\.1080\.165\\pm 0\.108ELU3\.4×10−43\.4\\times 10^\{\-4\}3/53/50\.219±0\.1240\.219\\pm 0\.124LeakyReLU\(0\.01\)\(0\.01\)1\.0×10−21\.0\\times 10^\{\-2\}5/55/50\.354±0\.0130\.354\\pm 0\.013Two points are worth stating because they are not what one would guess\. GELU is the*worst*of the four rather than a safe middle ground: for strongly negative inputs it provides almost no gradient, and none of its five runs recovered\. And ELU is not sufficient at this width: an earlier head withdtok=32d\_\{\\mathrm\{tok\}\}=32survived5/55/5with ELU, but at thedtokd\_\{\\mathrm\{tok\}\}this budget gives, only3/53/5\. Uniform warm\-up independently rescues5/55/5and is applied on top, but warm\-up lowers the probability of entering the trap, whereas LeakyReLU avoided this failure in all five tested runs\. We record this because without warm\-up, replacing LeakyReLU with ReLU substantially reduced performance in this diagnostic setting, and because it is the failure mode that decides, in Appendix[C\.4](https://arxiv.org/html/2609.38587#A3.SS4), against learning a1×11\\times 1channel projection in front of every backbone\.

## Appendix EAdditional Experiments and Analysis

### E\.1The Contribution of Pre\-training

Figure 4:Effect of pre\-training\.Downstream performance of NeurDuo \(Small\) with and without pre\-training\. Arrows indicate whether higher or lower values are better\.In this experiment, we fine\-tune NeurDuo \(Small\) on every downstream task from random initialisation, keeping the architecture, shared readout, context head, data splits and optimisation recipe identical to the pre\-trained runs; only the initial backbone weights differ\. As shown in Figure[4](https://arxiv.org/html/2609.38587#A5.F4), pre\-training improves performance on all five tasks\.

### E\.2Two readings of the sleep\-staging result

Table[3](https://arxiv.org/html/2609.38587#S5.T3)reports one row per model on Sleep\-EDF and Table[6](https://arxiv.org/html/2609.38587#S5.T6)a difference of two means\. Both compress something so we give the per\-class and the per\-fold view here\.

##### Where the macro\-F1 gap sits\.

Macro\-F1 averages five per\-class scores, and on this task those five are not equally hard\. Table[18](https://arxiv.org/html/2609.38587#A5.T18)splits the results of Table[3](https://arxiv.org/html/2609.38587#S5.T3)by class\. Our deficit against LaBraM is not spread across the table: we lead on N1, the class every model finds hardest and where the whole field sits between0\.410\.41and0\.480\.48, we are best on REM, and we are within0\.0100\.010of LaBraM on W\. The entire0\.0110\.011macro\-F1 gap comes from N3, where we trail by0\.0600\.060; on the other four classes we are collectively ahead of LaBraM\. N3 is defined by absolute delta amplitude sustained over an epoch, which is the one thing in this task a single3030s window already contains\. The rows below the rule show what carrying state does to exactly that class: with the state reset the N3 gap widens to0\.0960\.096\. Carrying state raises N3 F1 from0\.7030\.703to0\.7390\.739, closing over a third of that gap, the largest per\-class movement anywhere in the table and the one that contributes most to the macro\-F1 improvement of Table[6](https://arxiv.org/html/2609.38587#S5.T6)\.

##### Why the dispersion in Table[6](https://arxiv.org/html/2609.38587#S5.T6)does not bound the effect\.

The carried setting improves macro\-F1 by0\.0100\.010while the fold\-to\-fold standard deviation reaches0\.0140\.014, so read as two independent measurements the intervals overlap and the effect looks unresolved\. They are not independent\. Both settings run on the same five subject\-disjoint folds with the same seed, so the difference can be taken within each fold, and the fold\-to\-fold variation in how separable a group of subjects happens to be cancels\. Table[19](https://arxiv.org/html/2609.38587#A5.T19)does that\. All fifteen differences are positive, and the pairedttstatistics for accuracy andκ\\kappaexceed the two\-sided5%5\\%critical value at four degrees of freedom,2\.7762\.776\. The corresponding statistic for macro\-F1 remains below this threshold\. We report the aggregate in the main text and this decomposition here rather than significance stars in the table itself, since with five folds atttest is a description of consistency more than a test\.

Table 18:Per\-class F1 on Sleep\-EDF, five\-fold means\. Classes are the five AASM stages, W for wake and N1 to N3 for non\-REM depth\.Boldmarks the best value in a column among the seven models above the rule, each in the configuration Table[3](https://arxiv.org/html/2609.38587#S5.T3)reports\. Below the rule are that same model with its state reset at every unit boundary and the difference between the two, neither of which is part of that comparison\.†Domain overlap on Sleep\-EDF, see Appendix[D\.1](https://arxiv.org/html/2609.38587#A4.SS1)\.ModelWN1N2N3REMBIOT†0\.9240\.9240\.4630\.4630\.8280\.8280\.6760\.6760\.7640\.764CBraMod0\.9140\.9140\.4580\.4580\.8450\.8450\.805\\mathbf\{0\.805\}0\.7750\.775LaBraM0\.9280\.9280\.4780\.4780\.851\\mathbf\{0\.851\}0\.7990\.7990\.7730\.773EEGPT0\.9220\.9220\.4550\.4550\.8290\.8290\.7660\.7660\.7250\.725ST\-EEGFormer\-S0\.9230\.9230\.4130\.4130\.8260\.8260\.6720\.6720\.7400\.740REVE\-Base0\.930\\mathbf\{0\.930\}0\.4780\.4780\.8350\.8350\.6800\.6800\.7840\.784NeurDuo \(Small\)0\.9180\.9180\.484\\mathbf\{0\.484\}0\.8410\.8410\.7390\.7390\.794\\mathbf\{0\.794\}NeurDuo \(Small\), state reset0\.9190\.9190\.4900\.4900\.8290\.8290\.7030\.7030\.7830\.783Δ\\Deltafrom carrying state−0\.001\-0\.001−0\.006\-0\.006\+0\.012\+0\.012\+0\.036\+0\.036\+0\.011\+0\.011Table 19:Carried minus reset on Sleep\-EDF, fold by fold\.Same folds, same seed, same readout and context head; only the backbone state differs\.ttis the paired statistic over the five differences, for which the two\-sided5%5\\%critical value at four degrees of freedom is2\.7762\.776\.MetricFold 1Fold 2Fold 3Fold 4Fold 5Mean𝐭\\mathbf\{t\}PositiveMacro\-F1\+0\.0277\+0\.0277\+0\.0159\+0\.0159\+0\.0039\+0\.0039\+0\.0002\+0\.0002\+0\.0037\+0\.0037\+0\.0103\+0\.0103\+2\.02\+2\.025/55/5Accuracy\+0\.0089\+0\.0089\+0\.0146\+0\.0146\+0\.0044\+0\.0044\+0\.0083\+0\.0083\+0\.0043\+0\.0043\+0\.0081\+0\.0081\+4\.27\+4\.275/55/5Cohen’sκ\\kappa\+0\.0141\+0\.0141\+0\.0189\+0\.0189\+0\.0051\+0\.0051\+0\.0107\+0\.0107\+0\.0040\+0\.0040\+0\.0106\+0\.0106\+3\.80\+3\.805/55/5

### E\.3Two streams in operation

The fast and slow streams of §[3\.2\.2](https://arxiv.org/html/2609.38587#S3.SS2.SSS2)are given different update rates by construction rather than by training\. Two measurements show what that produces, one read from the weights alone and one from a forward pass on real signal\. Neither can be produced for a window encoder, which has no state whose timescale could be measured\.

Figure 5:The two streams run at different rates\.*\(a\)*Nominal time constantτ=1/\(Δ​\|A\|\)\\tau=1/\(\\Delta\|A\|\), withΔ\\Deltataken from the trained bias alone, of every state dimension of the pre\-trained Small model, converted to seconds at each stream’s own step; dashed lines are medians\.*\(b\)*Relative state change per second on one ten\-minute CHB\-MIT test sequence, averaged over the1515channels; the shaded band marks its three seizure units\.##### Timescales in the trained weights\.

Figure[5](https://arxiv.org/html/2609.38587#A5.F5)\(a\) asks how fast individual state dimensions decay\. In a selective state\-space block the state decays byexp⁡\(Δ​A\)\\exp\(\\Delta A\)at each step, so a dimension has a time constant ofτ=1/\(Δ​\|A\|\)\\tau=1/\(\\Delta\|A\|\)steps;Δ\\Deltais input\-dependent, and we report the nominal valuesoftplus⁡\(bΔ\)\\mathrm\{softplus\}\(b\_\{\\Delta\}\)given by the trained bias alone, which sets the timescale before any input modulates it\. We readAAandbΔb\_\{\\Delta\}off the weights of the Small model, take every combination of block, inner dimension and state dimension, and convert to seconds using each stream’s own step,0\.50\.5s for the fast stream and22s for the slow one\. Measured that way the slow stream is the longer\-lived of the two, median24\.924\.9against7\.77\.7s, and it reaches3535minutes where the fast stream stops at1414\. The separation in seconds largely reflects the update rate rather than the decays the optimiser found: in units of each stream’s own step the two distributions largely overlap, medians15\.515\.5and12\.412\.4steps, so the3\.2×3\.2\\timesseparation in seconds follows from those medians together with theN=4N=4ratio between how often the two are advanced\. That is the mechanism of §[3\.2\.2](https://arxiv.org/html/2609.38587#S3.SS2.SSS2)behaving as specified\. It also locates what makes the slow stream slow: not mainly a decay the model had to learn, but how often it is written and what it is written with, summaries from the memory writer rather than the samples the fast stream sees\.

##### Rates at inference\.

Figure[5](https://arxiv.org/html/2609.38587#A5.F5)\(b\) asks whether the two streams also move at different rates when the model is running\. On one ten\-minute CHB\-MIT test sequence we measure how much the representation each stream emits,ht,ch\_\{t,c\}for the fast stream andsk,cs\_\{k,c\}for the slow one, changes from one of its own steps to the next, relative to its own size, and divide by the step so that both are read per second of recording rather than per update\. These are the two vectors Eq\.[11](https://arxiv.org/html/2609.38587#S3.E11)splices, not the internal SSM states of the Mamba blocks, so the figure shows the rate at which what the readout sees is rewritten\. The fast representation changes by0\.05000\.0500per second at the median and the slow one by0\.00770\.0077, a factor of six and a half, and the ordering holds at all but five of the299299points where the two can be compared\. The difference is therefore not only in the weights: while the fast stream tracks the signal, the slow one barely moves\. The fast stream takes about half a minute to settle from the zero state it starts a sequence in and the slow stream about a minute and a half, which is the cold start the half\-sequence stride of Appendix[A\.1](https://arxiv.org/html/2609.38587#A1.SS1)is designed to make the model predict from, and which state carried across a unit boundary removes at evaluation time\. The shaded band marks the sequence’s three seizure units and is drawn for orientation only: on this one sequence neither stream shows a change there that stands out from its own baseline, and a single recording could not establish one if it did\.

## Appendix FStreaming Inference and Efficiency

### F\.1Stateful Streaming Inference

Figure 6:Stateful streaming inference\.Each0\.50\.5s chunk advances the fast stream by one step; every fourth chunk writes a memory token that advances the slow stream and enters the1616s bank the fast state queries\. Nothing is reset within a recording\.Figure[6](https://arxiv.org/html/2609.38587#A6.F6)illustrates how NeurDuo runs on a continuous recording\. For each channel, the fast and slow\-stream Mamba states, the recent fast representations used by the memory writer and the memory bank are initialised once at the start of the recording and never reset\. Each incoming0\.50\.5s chunk is encoded per channel and advances the fast stream by a single recurrent step, with channel attention mixing the channels at that step\. Every four chunks \(22s\), the memory writer summarises the last eight fast representations of each channel into a memory token, which advances the slow stream by one step and is appended to the memory bank, evicting its oldest entry\. At every chunk, the current fast representation queries the memory bank of its own channel, and the retrieved content is added to the slow representation before it is concatenated with the fast representation to form the output\. No earlier chunk is ever encoded again, so the cost of each step does not depend on how much of the recording has already been processed \(Appendix[F\.2](https://arxiv.org/html/2609.38587#A6.SS2)\)\.

### F\.2The cost of long context

§[3\.2\.2](https://arxiv.org/html/2609.38587#S3.SS2.SSS2)gives NeurDuo a state that is bounded in size and advanced one chunk at a time\. This appendix measures what that is worth against the alternative, which is to widen a window encoder until it spans the same history\. All numbers are measurements on one A100 at float32\. Latency depends on the shape of the input and not on its content, so the inputs are random tensors; what we borrow from the two sequence tasks is only their electrode configuration, two channels at100100Hz and fifteen at256256Hz, so that every model is given the same job on each\.

Table 20:Cost of conditioning on longer history\.Latency \(ms\) to encode one window, with peak memory \(GB\) measured at1,8001\{,\}800s\. NeurDuo latency is measured per0\.50\.5s chunk after streaming the stated history\.ModelSleep\-EDF \(2 ch\)CHB\-MIT \(15 ch\)30 s300 s600 s1,800 s3,600 sGB30 s300 s600 s1,800 s3,600 sGBEEGPT\-Large4\.04\.027\.327\.349\.849\.8137\.6137\.6274\.0274\.01\.221\.228\.58\.566\.066\.0128\.2128\.2382\.9382\.9765\.5765\.53\.463\.46ST\-EEGFormer\-S3\.93\.940\.540\.5136\.6136\.61,2211\{,\}2215,1195\{,\}1190\.970\.9728\.928\.91,9341\{,\}9348,0618\{,\}06174,30274\{,\}302–∗5\.645\.64BIOT6\.06\.023\.323\.340\.540\.5106\.9106\.9209\.2209\.20\.890\.896\.06\.023\.323\.342\.742\.7107\.0107\.0209\.5209\.50\.980\.98LaBraM9\.99\.998\.098\.0196\.4196\.4587\.2587\.21,1741\{,\}1740\.050\.0510\.010\.098\.198\.1196\.2196\.2585\.9585\.91,1771\{,\}1770\.200\.20CBraMod5\.85\.85\.95\.96\.16\.115\.115\.138\.138\.10\.160\.165\.85\.89\.29\.219\.019\.093\.693\.6281\.4281\.40\.990\.99REVE\-Base8\.58\.512\.512\.521\.321\.390\.990\.9257\.5257\.50\.400\.409\.49\.4115\.3115\.3400\.6400\.63,6533\{,\}65315,28815\{,\}2881\.161\.16NeurDuo \(Small\)4\.89\\mathbf\{4\.89\}4\.90\\mathbf\{4\.90\}4\.84\\mathbf\{4\.84\}4\.85\\mathbf\{4\.85\}4\.89\\mathbf\{4\.89\}0\.05\\mathbf\{0\.05\}4\.90\\mathbf\{4\.90\}4\.87\\mathbf\{4\.87\}4\.85\\mathbf\{4\.85\}4\.87\\mathbf\{4\.87\}4\.83\\mathbf\{4\.83\}0\.06\\mathbf\{0\.06\}NeurDuo \(Base\)6\.28\\mathbf\{6\.28\}6\.31\\mathbf\{6\.31\}6\.29\\mathbf\{6\.29\}6\.29\\mathbf\{6\.29\}6\.29\\mathbf\{6\.29\}0\.13\\mathbf\{0\.13\}6\.43\\mathbf\{6\.43\}6\.42\\mathbf\{6\.42\}6\.41\\mathbf\{6\.41\}6\.41\\mathbf\{6\.41\}6\.42\\mathbf\{6\.42\}0\.16\\mathbf\{0\.16\}NeurDuo \(Large\)8\.72\\mathbf\{8\.72\}8\.74\\mathbf\{8\.74\}8\.74\\mathbf\{8\.74\}8\.75\\mathbf\{8\.75\}8\.73\\mathbf\{8\.73\}0\.29\\mathbf\{0\.29\}8\.94\\mathbf\{8\.94\}8\.93\\mathbf\{8\.93\}8\.94\\mathbf\{8\.94\}8\.92\\mathbf\{8\.92\}8\.93\\mathbf\{8\.93\}0\.37\\mathbf\{0\.37\}
∗Not measured: the1,8001\{,\}800\-s run already took74\.374\.3s\.

##### Widening a window against advancing a state\.

Table[20](https://arxiv.org/html/2609.38587#A6.T20)gives the measurements behind Figure[2](https://arxiv.org/html/2609.38587#S5.F2), on both montages and with peak memory\. As released, four of the six baselines stop short of an hour, but in each case the limit is a fixed\-size table or a constant fixed when the encoder is built, not the architecture, so we lift it without adding a parameter or changing a learned weight\. BIOT’s1,0001\{,\}000\-row and ST\-EEGFormer’s512512\-row position tables bound the input at500500and6464seconds respectively, but both are computed from a sinusoidal formula rather than trained, so we regenerate them at the required length and verify that the released rows are reproduced exactly\. EEGPT places patches in time by rotary encoding and attends within a time patch, so no table bounds its input and only the expected patch count has to be set to that of the window\. LaBraM’s time table is learned and has1616rows, which cannot be extended, so the window is encoded as consecutive1515s sub\-windows\. CBraMod and REVE run natively\. Every baseline therefore takes every length we report, and what the table measures is the price of the alternative: our rows give the cost of one further0\.50\.5s chunk after the stated history has been streamed, which for a window encoder is the cost of the whole window, since it has to be encoded again to take in the new chunk\. None of this affects any number in Tables[3](https://arxiv.org/html/2609.38587#S5.T3),[4](https://arxiv.org/html/2609.38587#S5.T4),[5](https://arxiv.org/html/2609.38587#S5.T5),[6](https://arxiv.org/html/2609.38587#S5.T6), where no baseline is shown more than one3030s unit and the shared context head integrates across units\.

##### What the state costs\.

Two quantities are easy to conflate and we separate them\. The first is the state itself, writtenΣt\\Sigma\_\{t\}: the convolution and SSM states of the fast blocks, the SSM states of the slow blocks, the queue of recent fast representations the writer pools over, and theBBslots of the bank\. It is what has to survive from one chunk to the next, its size follows from the architecture rather than from the recording, and Table[21](https://arxiv.org/html/2609.38587#A6.T21)breaks it down for NeurDuo \(Small\)\. At349349KB per channel it comes to0\.680\.68MB for the two channels of Sleep\-EDF and5\.125\.12MB for the fifteen of CHB\-MIT\. The second quantity is the memory a step occupies while it runs, which additionally holds the parameters and the activations of the one chunk being processed:0\.0470\.047and0\.0610\.061GB for Small on the two montages\.

Table 21:The persistent state, per channel, for NeurDuo \(Small\)\. Sizes follow from the architecture constants of Table[10](https://arxiv.org/html/2609.38587#A1.T10)and are the same at every point in a recording\. The writer queue is a ring buffer of1616slots, of which each write pools the most recentw=8w=8; an implementation that kept only those eight would save88KB per channel\.ComponentShapeValuesKB \(fp32\)Fast stream6×\(512×16​SSM\+512×4​conv\)6\\times\(512\\times 16\\ \\text\{SSM\}\+512\\times 4\\ \\text\{conv\}\)61,440240\.0Slow stream3×\(384×16​SSM\+384×3​conv\)3\\times\(384\\times 16\\ \\text\{SSM\}\+384\\times 3\\ \\text\{conv\}\)21,88885\.5Memory bank8×1928\\times 1921,5366\.0Writer queue16×25616\\times 2564,09616\.0Last output256\+192256\+1924481\.8Total per channel89,408349\.2
##### Streaming an hour\.

Table[22](https://arxiv.org/html/2609.38587#A6.T22)follows both quantities along a stream, for NeurDuo \(Small\)\. Latency is the cost of one chunk averaged over a full write cycle of four chunks, three that only advance the fast stream and one that also writes a memory token and advances the slow stream,4\.54\.5to4\.84\.8ms and5\.85\.8ms respectively, which average to the4\.94\.9ms per chunk the table reports; the cycle is timed2525times after6464warm\-up chunks, with the state restored before each repetition so that measuring does not advance the stream\. Peak memory is the high\-water mark of allocated memory since streaming began, covering parameters, state and activations but not the CUDA context, so anything accumulated across the stream would make it climb\. Over the six points the state does not change by a byte, peak memory holds to three decimal places, and latency varies by1\.5%1\.5\\%, the spread of repeated timings on an idle device\. A chunk carries0\.50\.5s of signal and takes4\.94\.9ms to absorb, so the model runs about a hundred times faster than the recording arrives, and almost as fast at fifteen channels as at two, because the channels advance as parallel rows rather than as a longer sequence; what scales with the montage is the state, linearly and by construction\.

Table 22:Streaming a recording from thirty seconds to an hour, NeurDuo \(Small\)\. Each row advances the state to the stated amount of history and then measures the next chunks, averaged over a full write cycle\. Nothing in the table depends on the history, which is the property §[3\.2\.2](https://arxiv.org/html/2609.38587#S3.SS2.SSS2)claims and this table exists to check\.Sleep\-EDF montage, two channelsCHB\-MIT montage, fifteen channelsHistoryms×\\bm\{\\times\}real timeGB peakMB statems×\\bm\{\\times\}real timeGB peakMB state3030s4\.894\.89102×102\\times0\.0470\.0470\.680\.684\.904\.90102×102\\times0\.0610\.0615\.125\.1211min4\.894\.89102×102\\times0\.0470\.0470\.680\.684\.894\.89102×102\\times0\.0610\.0615\.125\.1255min4\.904\.90102×102\\times0\.0470\.0470\.680\.684\.874\.87103×103\\times0\.0610\.0615\.125\.121010min4\.844\.84103×103\\times0\.0470\.0470\.680\.684\.854\.85103×103\\times0\.0610\.0615\.125\.123030min4\.854\.85103×103\\times0\.0470\.0470\.680\.684\.874\.87103×103\\times0\.0610\.0615\.125\.126060min4\.894\.89102×102\\times0\.0470\.0470\.680\.684\.834\.83104×104\\times0\.0610\.0615\.125\.12

## Appendix GDiscussion

##### Limitations\.

Three are worth stating plainly\. First, these results suggest that increasing model capacity alone yields limited and task\-dependent gains under the current pre\-training setup, motivating larger pre\-training corpora\. We pre\-train on3,9553\{,\}955hours, an order of magnitude less than the largest corpus among our baselines, and Figure[3](https://arxiv.org/html/2609.38587#A1.F3)shows what that costs: the Large model reaches its minimum51,00051\{,\}000updates before the schedule ends and then rises, while Small and Base are still descending when it ends, and their minima lie within0\.030\.03nats of one another\. The downstream tables say the same thing from the other side, since Large adds nothing over Base on FACED and loses to it on SEED\-VIG\. Second, the advantage is uneven across the two sequence tasks, and the pattern is informative rather than incidental\. On CHB\-MIT, which supplies fifteen electrodes, scaling from Small to Large lifts AUC\-PR from0\.4130\.413to0\.4710\.471; on Sleep\-EDF, which supplies two derivations, the three sizes are separated by0\.0090\.009macro\-F1 and we are not first, trailing LaBraM by0\.0110\.011\. Appendix[E\.2](https://arxiv.org/html/2609.38587#A5.SS2)locates that gap in a single class rather than across the five, which is consistent with a channel\-parallel frontend having little to separate when the montage is two channels wide\. Third, the benefit of context is real but bounded, and the bound is a property of the task\. Sleep staging benefits overall from longer context, with smaller gains beyond ten minutes\. Seizure detection does not: with the full montage the optimum sits between ten and thirty minutes, and the two metrics do not agree on where inside that range it falls\. Longer context is therefore not a direction that pays indefinitely, and the useful scale has to be found per task rather than assumed\.

##### Outlook\.

Three directions follow from those limits\. The first is simply more continuous signal: the architecture is indifferent to montage and to recording length, so the obstacle to a larger model is the supply of long recordings rather than anything in the design, and the curves above say that is where the next gain is\. The second is to make consolidation and retrieval adaptive\. The writer currently fires on a fixed schedule, everyN=4N=4steps, and the bank holds a fixed span of1616seconds; both are constants chosen once and held across the family, whereas what should be written and how far back should be read are plausibly functions of the signal\. A writer that fires on change rather than on a clock, and a bank whose span adapts to the timescale a task needs, are the natural next version of the mechanism this paper fixes by construction\. The third is evaluation in the setting the model was built for\. Every result here is offline, scored after the fact on complete recordings, yet the state is bounded and causal precisely so that inference can run alongside an ongoing session\.