Spectral Priors vs. Attention: Investigating the Utility of Attention Mechanisms in EEG-Based Diagnosis

# Investigating the Utility of Attention Mechanisms in EEG-Based Diagnosis
Source: [https://arxiv.org/html/2605.15433](https://arxiv.org/html/2605.15433)
11institutetext:University of Cincinnati, Cincinnati, OH, USA
11email:jawadkk@mail\.uc\.edu, \{atlurigm, ravindvm\}@ucmail\.uc\.edu###### Abstract

Electroencephalograph \(EEG\) time\-series signals are characterized by significant noise and coarse spatial resolution, which complicates the classification of neuro\-degenerative diseases\. Even the state\-of\-the\-art deep learning architectures struggle to distinguish between healthy controls and diseased subjects—or between different disease types due to high inter\-group similarity\.

In this paper, we show that a spectrally selective approach to feature construction enhances class separability\. By isolating signal strengths within the primary brainwave bands—δ,θ,α,β,γ\\delta,\\theta,\\alpha,\\beta,\\gamma— we transform high\-dimensional raw data into high\-value spectral features\.

Our results demonstrate that: a\) features derived from frequency and time\-frequency domain allow traditional machine learning models to match or exceed the performance of state\-of\-the\-art deep learning models, b\) Attention mechanism is unable to distill the stable feature signatures that characterize healthy neural activity in both resting and task EEGs, and c\) the limitations of attention\-based models in finding relevant spectral features appear to be fundamental in that providing frequency\-selective time\-domain input do not appreciably improve their performance\. We validate our methodology across three open\-source resting EEG datasets, and one task EEG dataset providing robust empirical evidence for our claims\.

## 1Introduction

Electroencephalography \(EEG\) is a non\-invasive neuroimaging data modality that measures the electrical activity in the brain\. EEG recordings are comprised of an array of electrodes placed on the surface of the skull\. EEGs measure brain activity in terms of minute voltage changes between pairs of electrodes\. EEG signals are widely used to understand, diagnose, evaluate, and monitor the brain activity and cognitive ability in neuro\-degenerative diseases\. They are also used extensively as input modalities by Brain\-Computer Interfaces \(BCIs\)\.

Time\-series classification of EEG data is of particular clinical interest for diagnosing and predicting the onset of different neurological diseases and disorders \(such as Alzheimer’s Disease, Parkinson’s Disease, Dementia, and Attention Deficit Disorders\)\. Attention\-based Transformer architectures have been proposed with the goal of levaraging temporal dependence in the signals\[[4](https://arxiv.org/html/2605.15433#bib.bib4),[5](https://arxiv.org/html/2605.15433#bib.bib5),[6](https://arxiv.org/html/2605.15433#bib.bib6)\]\. However, EEG signals are known to exhibit significant inter\-subject variability and non\-stationary temporal dynamics\. While Transformers excel at modeling long\-range dependencies in quasi\-stationary sequences \(like text data\), they lack the specific inductive bias required to disentangle the overlapping oscillatory components inherent in neuro\-physiological data\. Recent approach by Yu et al\[[28](https://arxiv.org/html/2605.15433#bib.bib28)\]have engineered a centralized token distribution strategy to align with the globally distributed discriminative signals in medical time\-series\.

In this paper, we show that traditional classification approaches, such as Quadratic Discriminant Analysis and Random Forests operating on spectral features are comparable, and sometimes outperform state\-of\-the\-art attention\-based architectures in diagnosing different neuropathologies\. We also demonstrate that providing these spectral features as input do not result in a significant improvement in performance of attention\-based architectures, suggesting that the attention mechanism suffers from a lack of temporal salience in quasi\-stationary signals, where discriminative biomarkers are distributed globally across the recording rather than localized in the discrete temporal events that self\-attention is designed to prioritize\. Methodically, a light\-weight framework extracting informative biomarkers provide stronger decision boundaries in the benchmark EEG datasets\.

## 2Methodology

Given a dataset of EEG data drawn from a population and associated labels \(ex: healthy v/s disease\), our goal is to construct relevant features with high discriminatory power\. We initiate the description of our approach by introducing relevant terminology and techniques\.

### 2\.1Feature Construction

Let an EEG session be represented by a matrixX∈ℝC×TX\\in\\mathbb\{R\}^\{C\\times T\}whereCCrepresents the number of channels andTTrepresents the number of time\-points\. After downsampling all the datasets to 256 Hz, we feed signal strengths at five frequency bands \(PSD values\) and spectral frequency measures calculated via Welch FFT to Transformers\. We band pass the original signal to keep the time\-signal representing frequency bands from 0\.5\-45 Hz\. We denote the sampling rate asfsf\_\{s\}\. From each such session, we a\) center each time series, b\) extract spectral features using the Welch method of Fast\-Fourier Transform \(FFT\) applied to get signal strengths at frequency bands and spectral frequency values in frequency domain, and c\) apply Discrete Wavelet Transform \(DWT\) to capture the non\-stationary frequency components of time\-signal and get the relative band energies in the time\-frequency domain\.

Welch\-based FFT band powers:For each channelx~c\\tilde\{x\}\_\{c\}is our centered zero\-mean time series\. For frequency resolution of time signal, we estimate the Power Spectral Density \(PSD\) via Welch’s method using a Hann window with overlapping segments\[[13](https://arxiv.org/html/2605.15433#bib.bib13)\]and denote it byS^c​\(f\)\\widehat\{S\}\_\{c\}\(f\)\. For each bandb=\[fbmin,fbmax\]b=\[f\_\{b\}^\{\\min\},f\_\{b\}^\{\\max\}\], the absolute band power isPc,babs=∫fbminfbmaxS^c​\(f\)​𝑑fP\_\{c,b\}^\{\\mathrm\{abs\}\}=\\int\_\{f\_\{b\}^\{\\min\}\}^\{f\_\{b\}^\{\\max\}\}\\widehat\{S\}\_\{c\}\(f\)\\,df\. The relative band powers over different frequency bands of interest are given asPc,brel=Pc,babs∑b′∈ℬPc,b′abs\+εP\_\{c,b\}^\{\\mathrm\{rel\}\}=\\frac\{P\_\{c,b\}^\{\\mathrm\{abs\}\}\}\{\\sum\_\{b^\{\\prime\}\\in\\mathcal\{B\}\}P\_\{c,b^\{\\prime\}\}^\{\\mathrm\{abs\}\}\+\\varepsilon\}\(withε=10−12\\varepsilon=10^\{\-12\}\)\. We compute the relative band power for the 5 canonical EEG bands \(δ\\delta:0\.50\.5–4​Hz4\\,\\mathrm\{Hz\},θ\\theta:44–8​Hz8\\,\\mathrm\{Hz\},α\\alpha:88–13​Hz13\\,\\mathrm\{Hz\},β\\beta:1313–30​Hz30\\,\\mathrm\{Hz\},γ\\gamma:3030–45​Hz45\\,\\mathrm\{Hz\}\) and aggregate across channels to form the session\-wise FFT featureFFTb=1C​∑c=1CPc,brel\\mathrm\{FFT\}\_\{b\}=\\frac\{1\}\{C\}\\sum\_\{c=1\}^\{C\}P\_\{c,b\}^\{\\mathrm\{rel\}\}\.

Using the same Welch PSDS^c​\(f\)\\widehat\{S\}\_\{c\}\(f\)\(restricted to0\.50\.5–4545Hz\), we compute per\-channel mean frequency, median frequency, and spectral entropy, and then average across channels\. Denoting the normalized spectrum bypc​\(f\)=S^c​\(f\)/∫S^c​\(u\)​𝑑up\_\{c\}\(f\)=\\widehat\{S\}\_\{c\}\(f\)/\\int\\widehat\{S\}\_\{c\}\(u\)\\,du, spectral entropy isH\(c\)=−∫pc​\(f\)​ln⁡pc​\(f\)​𝑑fH^\{\(c\)\}=\-\\int p\_\{c\}\(f\)\\ln p\_\{c\}\(f\)\\,df, with the remaining summary statistics computed analogously frompc​\(f\)p\_\{c\}\(f\)and then channel\-averaged\.

Discrete Wavelet Transform \(DWT\) band energies:For Non\-Stationary Time Signals, the temporal component of frequencies which are not found from Welch\-FFT are captured by DWT\. For each channelc∈\{1,…,C\}c\\in\\\{1,\\ldots,C\\\}, we havex~c∈ℝT\\tilde\{x\}\_\{c\}\\in\\mathbb\{R\}^\{T\}as the per\-channel DC\-centered time series andfsf\_\{s\}is our sampling rate\. We compute anLL\-level DWT \(waveletψ\\psi, defaultdb4\), yielding approximation coefficientsc​Ac,LcA\_\{c,L\}\(longer time resolution for low\-frequency up tofs/2L\+1f\_\{s\}/2^\{L\+1\}\) and detail coefficients\{c​Dc,j\}j=1L\\\{cD\_\{c,j\}\\\}\_\{j=1\}^\{L\}\(shorter time\-resolution for higher frequency components in dyadic\(power of 2\) frequency ranges\)\. We define level energies by squaredℓ2\\ell\_\{2\}norms:Ec,LA=∥c​Ac,L∥22,Ec,jD=∥c​Dc,j∥22,j=1,…,L\.E^\{A\}\_\{c,L\}=\\lVert cA\_\{c,L\}\\rVert\_\{2\}^\{2\},\\ E^\{D\}\_\{c,j\}=\\lVert cD\_\{c,j\}\\rVert\_\{2\}^\{2\},\\ \\ j=1,\\ldots,L\.

Each detail leveljjis associated with the nominal dyadic intervalℐj=\(fs/2j\+1,fs/2j\)\\mathcal\{I\}\_\{j\}=\(f\_\{s\}/2^\{j\+1\},\\,f\_\{s\}/2^\{j\}\)\. To obtain energies in the canonical EEG bandsℬ=\{δ,θ,α,β,γ\}\\mathcal\{B\}=\\\{\\delta,\\theta,\\alpha,\\beta,\\gamma\\\}, we map the DWT subband energies—those derived from the approximation coefficientsc​Ac,LcA\_\{c,L\}with supportIA=\(0,fs2L\+1\)I^\{A\}=\\left\(0,\\frac\{f\_\{s\}\}\{2^\{L\+1\}\}\\right\), and from the detail coefficients\{c​Dc,j\}j=1L\\\{cD\_\{c,j\}\\\}\_\{j=1\}^\{L\}with supportsℐj\\mathcal\{I\}\_\{j\}—to each EEG band\. We compute the fractional overlap between subband intervals and the target band limits for this mapping\. This yields a deterministic mapping from DWT subbands to canonical bands \(rather than a within\-band PSD estimate\)\. We then normalize within channel and average across channels to obtain session\-wise featuresDWTb\\mathrm\{DWT\}\_\{b\}\. We selectLLper epoch as the largest admissible decomposition depth given the epoch length and wavelet support \(PyWaveletsdwt\_max\_level\), capped at a maximum \(hereL≤8L\\leq 8\)\. For example, withfs=256f\_\{s\}=256Hz andT=256T=256\(1 s\),db4yieldsL=5L=5so that,c​Ac,LcA\_\{c,L\}=fs/2L\+1=4f\_\{s\}/2^\{L\+1\}=4Hz, aligning the approximation cutoff with the upper edge of theδ\\deltaband\.

*In all, our feature\-set is comprised of relative PSD in each 5 frequency bands, relative wavelet strengths in those frequency bands and spectral summary features \(spectral mean freq, spectral median freq and spectral entropy\)*

### 2\.2Classical Baselines and Spectral Feature Classification

We train simple classifiers that operate on spectral features \(described above\) to output predicted labels\. In our experiments, we choose Quadratic Discriminant Analysis \(QDA\) and Random Forest\. QDA minimizes the total probability of misclassification by assigning each observation to the class with the highest posterior probability\. It achieves this by calculating a quadratic decision boundary that minimizes the expected loss based on class\-specific Gaussian distributions\. Random Forest is an ensemble learning technique that works by building a many independent decision trees, each trained on a random subset of data and a random selection of features\. It produces the final prediction using a majority vote\. In the context of diagnosis with EEG data, we show that these classifiers match, and sometimes exceed many state\-of\-the\-art attention\-based deep learning models\.

## 3Results

#### 3\.0\.1Datasets

We demonstrate our results on a total of four publicly available datasets for different pathologies, including three resting state EEG datasets and one task EEG dataset\. The APAVA\[[10](https://arxiv.org/html/2605.15433#bib.bib10)\]dataset has 16 channels and 23 subjects, 12 with Alzheimers and 11 Healthy as labels\. The TDBrain\[[11](https://arxiv.org/html/2605.15433#bib.bib11)\]dataset has 33 channels and we sample 50 subjects total where 25 have Parkinsons and 25 are Healthy Controls\. ADFTD\[[12](https://arxiv.org/html/2605.15433#bib.bib12)\]has 88 subjects, 19 channels with three classes, where 23 are Dementia, 36 Alzheimers, and 29 Healthy controls\. For task\-EEG, we integrated ADHD\[[28](https://arxiv.org/html/2605.15433#bib.bib28)\]dataset having 19 channels with 61 ADHD patients and 60 Healthy Controls

### 3\.1Performance parity is observed between manually designed features on traditional classifiers and Attention\-Based methods

For each EEG session in each dataset, we compute spectral features as described in section[2\.1](https://arxiv.org/html/2605.15433#S2.SS1)\. We partition the data into train, validation and test sets using the protocol established in Medformer\[[4](https://arxiv.org/html/2605.15433#bib.bib4)\]to ensure a fair comparison\. Features undergo Z\-score normalization, where the mean and standard deviation are computed strictly on the training set and subsequently applied to the validation and test sets to prevent data leakage\.

We evaluate two traditional classifiers—Quadratic Discriminant Analysis \(QDA\) and Random Forests \(RF\)—against three state\-of\-the\-art attention\-based architectures: Medformer, Reformer\[[8](https://arxiv.org/html/2605.15433#bib.bib8)\], and EEG Conformer\[[22](https://arxiv.org/html/2605.15433#bib.bib22)\]\. For the traditional classifiers, we utilize Principal Component Analysis \(PCA\) to reduce dimensionality, selecting hyperparameters via Grid Search on the validation set to maximize macro\-F1\. In QDA, we tune the retained PCA variance and the regularization parameter to account for multi\-collinearity\. For the Random Forest, we adjusted the number of estimators, maximum depth, and minimum samples required per leaf\.

Table 1:Macro\-averaged classification performance \(%\) of traditional classifiers \(QDA and Random Forest\) and attention\-based methods \(Medformer, Reformer, and Conformer\) on four EEG datasets\.For each classifier, macro\-averaged metrics are reported per dataset\.*The highest standard deviation found for the ML models were 2\.35 \(RF in ADFTD\) and 1\.32 for \(Conformer in ADHD\)\.*We mark the best metric scores for each dataset in bold\. Our results show that across the 4 EEG datasets, QDA has outperformed all other models in 3 datasets \(RF outperforms QDA in TDBrain\) with FFT DWT components aggregated on timestamps and channels\. In general classical ML pipelines have comparable numbers with that of Transformer models in all the datasets\. Among the Transformer models, TeCh produces strong numbers in the APAVA and TDBrain, Medformer outperforms the other Transformer models in ADFTD and ADHD\. The high performance metrics obtained for the TDBrain and APAVA datasets by RF and QDA respectively can be attributed to the choice of the evaluation test set\. However, to ensure comparability across the methods, we have chosen to preserve the same train/test split as reported in the Medformer paper\[[4](https://arxiv.org/html/2605.15433#bib.bib4)\]\.The results, summarized in Table[1](https://arxiv.org/html/2605.15433#S3.T1)show that on multiple metrics, QDA achieves superior performance in ADFTD, APAVA, and ADHD\. Random Forest provides the best results on the TDBrain dataset\. We note that the core premise of the Transformer architecture is the existence of latent temporal “events” that require dynamic weighting\. In the context of resting\-state EEG \(ADFTD, APAVA, and TDBrain datasets\), however, the biomarkers of interest are primarily oscillatory power and phase\-coupling\[[21](https://arxiv.org/html/2605.15433#bib.bib21),[26](https://arxiv.org/html/2605.15433#bib.bib26)\]\. These measures are inherently stationary over the analyzed windows\. By treating the EEG as a sequence of discrete tokens, Transformers ignore the underlying continuous physics of the signal, attempting to “attend” in a temporal domain that lacks salient landmarks\. While similar considerations are relevant in task EEG \(ADHD dataset\), there are some event\-related epochs that attention mechanisms can leverage\. This explains the competitive precision, AUROC, and AUPRC by Transformer models\. Finally, we note that QDA and RF perfectly classify APAVA and TDBrain, suggesting strong within\-class grouping in these two datasets\.

![Refer to caption](https://arxiv.org/html/2605.15433v1/cm_QDA.png)\(a\)QDA
![Refer to caption](https://arxiv.org/html/2605.15433v1/cm_ADFD_Conformer.png)\(b\)Conformer

Figure 1:Holdout\-set confusion matrices on ADFTD comparing a classical pipeline vs\. a Transformer\.Rows denote ground\-truth labels \(‘A’:Alzheimers, ‘C’:Healthy Controls, ‘F’:Dementia\) and columns denote predicted labels; darker diagonal entries indicate better class\-wise performance\.Left:Quadratic Discriminant Analysis \(QDA\) trained on aggregated spectral features \(Welch\-FFT/DWT, channel\- and window\-averaged\) shows stronger separation between Healthy Controls \(C\) and Alzheimer’s disease \(A\) on the holdout subjects\.Right:Conformer trained on rs\-EEG time\-series exhibits reduced true positives for Healthy Controls and Alzheimers\. Overall, these matrices highlight that feature aggregation can provide a more stable signal\-to\-noise representation for classical models under limited rs\-EEG data\.
### 3\.2Baseline Comparison and Self\-Attention Diagnostics

In the previous section, we saw that traditional classifiers achieve parity, and sometimes outperform attention\-based methods\. To emphasize the difference in ability to learn fundamental physiological boundaries between the two approaches, we show the confusion matrix of QDA and Conformer in the ADFTD dataset in Figure[1](https://arxiv.org/html/2605.15433#S3.F1)\. This dataset comprises of three labels – Alzheimer’s \(A\), Healthy Control \(C\), and Frontotemporal Dementia \(F\)\. In the spectral domain, healthy brain activity \(Control\) constitutes a distinct, low\-variance state that is perfectly captured by QDA\. In contrast, the Conformer fails to recognize this baseline, misclassifying more than one\-third of healthy subjects\. This indicates that the attention mechanism struggles to extract the consistent feature patterns associated with healthy neural activity\. Next, we analyzed the self\-attention weights for each test predictions \(both correct and incorrect predictions\) across all four datasets\. We compute the KL divergence between between the attention maps of all pairs of correct \(TP,TN\) and incorrect \(FP, FN\) predictions\. In Table[2](https://arxiv.org/html/2605.15433#S3.T2), we see that a\) the values themselves are low, which indicates that the predictive power of the transformer models do not come from the attention mechanism, and b\)KL\(Correct\|\|Incorrect\)≈KL\(Incorrect\|\|Correct\)KL\(Correct\|\|Incorrect\)\\approx KL\(Incorrect\|\|Correct\), which reveals that the model’s internal focus is invariant of diagnostic success\.

Table 2:KL Divergence calculated for all correct vs incorrect and all incorrect vs correct predictions\.We compute query\-key attention weights for all test subjects and compute KL divergence for all pairs of correct and incorrect predictions and report the average scores for KL\(Correct \|\| Incorrect\) and KL \(Incorrect \|\| Correct\)\.The highest standard deviation found for this experiment was 5\.9e\-03 for KL\(Correct\|\|Incorrect\) on the ADHD dataset\.
### 3\.3Sensitivity analysis of input spectral features

To investigate whether the parity of attention\-based methods are due to inherent stationarity of the oscillatory power and phase\-coupling across the dataset, or the denoising effect of the spectral feature engineering, we conducted the following experiment\. Time\-series from each channel was decomposed into the 5 canonical EEG frequency bands \(α\\alpha,β\\beta,γ\\gamma,δ\\delta,θ\\theta\)\. We trained Medformer, which yielded the strongest results across all the attention\-based methods on these cleaned, deconvolved signals\.

The results, summarized in Table[3](https://arxiv.org/html/2605.15433#S3.T3)show no appreciable improvement in performance\. In fact, as Medformer was provided biologically informed, simpler signals in narrow frequency bands, we see a perfomance plateau\. We hypothesize that the self\-attention mechanism suffers from “Attention Dilution” when presented with multi\-band time\-series\. Instead of focusing on finding discriminative phase\-amplitude couplings, the model’s capacity is consumed by modeling redundant temporal correlations between highly correlated filtered versions of the same underlying signal\. Further, while sub\-band decomposition provides explicit physiological priors, it simultaneously quintuples the input dimensionality\. In the low\-data regime typical of clinical EEG \(N<100N<100\), this expansion of the hypothesis space likely exacerbates the variance of the Transformer’s attention weights, leading to the observed performance degradation\.

Table 3:Performance of Medformer with separate signals for\(δ,θ,α,β,γ\)\(\\delta,\\theta,\\alpha,\\beta,\\gamma\)bands\.For each of the three rs\-EEG datasets, we use the dataset specific configuration provided by Wang et al\[[4](https://arxiv.org/html/2605.15433#bib.bib4)\]and fine\-tune augmentations, dropout and weight decay to adjust for validation loss fluctuations\. For task\-EEG \(ADHD\) we use the authors’ validated baseline configuration\.*The highest standard deviation observed across datasets was found to be 3\.25 for ADFTD dataset*\. This table shows that the self\-attention mechanism computes redundant query\-key interactions for highly correlated signals across different frequencies\. Consequently, the subtle, disease\-relevant biomarkers are effectively “masked” within the sparse attention space, as the query\-key vectors become dominated by these redundant inter\-frequency correlations\.

## 4Related Literature

Both attention and non\-attention based methods in time\-series classification domain have employed different means to capture the recurring patterns in the signal for classification\. In TimesNet\[[19](https://arxiv.org/html/2605.15433#bib.bib19)\], Wu et al\. transform 1D time\-series tensors into 2D representations to model intra\-period and inter\-period variations using convolutional kernels\. While their application to medical time\-series classification underscores the importance of capturing recurring patterns, the method may be insufficient for specific diagnostic signals\. For instance, in EEG signals representing neuro\-degenerative disease groups, critical subject\-specific periodic variations occur within the 0\.5–45 Hz frequency bands\[[20](https://arxiv.org/html/2605.15433#bib.bib20),[21](https://arxiv.org/html/2605.15433#bib.bib21),[22](https://arxiv.org/html/2605.15433#bib.bib22)\]\. The standard TimesNet architecture does not explicitly segregate these distinct functional frequency bands, potentially overlooking nuanced pathological markers\. Existing state\-of\-the\-art methods, such as Rocket\[[18](https://arxiv.org/html/2605.15433#bib.bib18)\]and Shapelets\[[19](https://arxiv.org/html/2605.15433#bib.bib19)\], rely on applying convolutional kernels or extracting discriminative subsequences to capture relevant features for classification\. However, in the context of EEG data, standard convolutional kernels often fail to model long\-range periodicity effectively\. Furthermore, extracting and comparing shapelet subsequences over extended EEG recordings—typically sampled at rates\>100\>100Hz—becomes computationally infeasible due to the high dimensionality and length of the time\-series\.

To model long\-range periodicity in medical time\-series, state\-of\-the\-art Transformer models employ mechanisms such as Auto\-Correlation, frequency\-enhanced attention, and cross\-channel correlation\. However, the Auto\-Correlation\-based attention used in Autoformer\[[5](https://arxiv.org/html/2605.15433#bib.bib5)\]often fails to capture the specific periodic feature components necessary to discriminate between disease types or healthy controls\. Similarly, the frequency\-enhanced attention in FEDformer\[[7](https://arxiv.org/html/2605.15433#bib.bib7)\]does not explicitly segregate dominant physiological frequency bands \(α,β,γ,δ,θ\\alpha,\\beta,\\gamma,\\delta,\\theta\) from overlapping signals; consequently, it remains susceptible to noise in higher frequency ranges \(45–100 Hz\)\. More recent architectures, such as Medformer\[[8](https://arxiv.org/html/2605.15433#bib.bib8)\], incorporate inter\-granularity learning and cross\-channel correlations\. While Yu et al\.\[[28](https://arxiv.org/html/2605.15433#bib.bib28)\]address the architectural mismatch between decentralized attention and centralized medical time\-series signals, their framework primarily focuses on centralized temporal–channel interactions rather than explicitly modeling disease\-specific neuro\-pathological spectral biomarkers\. Consequently, because the learned representations are not explicitly constrained toward biomarker\-relevant spectral components, localized pathological signatures may remain diluted within the broader feature space\.

## 5Discussion

The observed performance parity between traditional classifiers with spectral features, and attention\-based architectures operating on time\-series data show that the high representational capacity of transformers does not translate to clinical utility for neurophysiological signals\. While self\-attention is optimized for sparse temporal salience, EEG biomarkers are more often globally distributed evident from table[1](https://arxiv.org/html/2605.15433#S3.T1)\. Relative Band powers captured via both time and frequency resolution enabled us to identify spatially stable signals over time\.

Furthermore, the inability of transformer\-based models to consistently discriminate Healthy Controls with Fronto\-temporal Dementia and Alzheimer’s Disease reveal that they are unable to capture globally stable spectral features that segregate different disease groups from controls\. KL divergence results in \(Table[2](https://arxiv.org/html/2605.15433#S3.T2)\) suggest that attention weights do not drive classification decisions\. This leads to sparse attention distributions that provide limited spectrally informative data for classification\. Finding proxy tokens reduces the computational overhead of inter\-token interactions in sparse\-attention models, but it still does not explicitly uncover the latent biological signatures underlying disease\-related signal patterns\. These results suggest that for current clinical data scales, principled parsimony via signal\-processing\-informed inductive biases remains more robust than domain\-agnostic models, advocating for the integration of physics\-informed kernels in future neuro\-imaging architectures\.

## References

- \[1\]Same, Mohammad Hossein, et al\. "Simplified welch algorithm for spectrum monitoring\." Applied Sciences 11\.1 \(2020\): 86\.
- \[2\]Radovanović, Miloš, Alexandros Nanopoulos, and Mirjana Ivanović\. “Nearest neighbors in high\-dimensional data: The emergence and influence of hubs\." Proceedings of the 26th Annual International Conference on Machine Learning\. 2009\.
- \[3\]Ghojogh, B\., and M\. Crowley\. “Linear and quadratic discriminant analysis: Tutorial\. arXiv 2019\." arXiv preprint arXiv:1906\.02590\.
- \[4\]Wang, Yihe, et al\. "Medformer: A multi\-granularity patching transformer for medical time\-series classification\." Advances in Neural Information Processing Systems 37 \(2024\): 36314\-36341\.
- \[5\]Wu, Haixu, et al\. “Autoformer: Decomposition transformers with auto\-correlation for long\-term series forecasting\.”*NeurIPS*34 \(2021\): 22419–22430\.
- \[6\]Cleveland, Robert B\., et al\. "STL: A seasonal\-trend decomposition\." J\. off\. Stat 6\.1 \(1990\): 3\-73\.
- \[7\]Zhou, Tian, et al\. "Fedformer: Frequency enhanced decomposed transformer for long\-term series forecasting\." International conference on machine learning\. PMLR, 2022\.
- \[8\]Kitaev, Nikita, Łukasz Kaiser, and Anselm Levskaya\. "Reformer: The efficient transformer\." arXiv preprint arXiv:2001\.04451 \(2020\)\.
- \[9\]J Escudero et al\. Analysis of electroencephalograms in alzheimer’s disease patients with multiscale entropy\. Physiological measurement, 27\(11\):1091, 2006\.
- \[10\]Hanneke van Dijk, Guido van Wingen, Damiaan Denys, Sebastian Olbrich, Rosalinde van Ruth, and Martijn Arns\. The two decades brainclinics research archive for insights in neurophysiology \(tdbrain\) database\. Scientific data, 9\(1\):333, 2022\.
- \[11\]Andreas Miltiadous, Katerina D Tzimourta et al\. A dataset of scalp eeg recordings of alzheimer’s disease, frontotemporal dementia and healthy subjects from routine eeg\. Data, 8\(6\):95, 2023\.
- \[12\]Jwo, Dah\-Jing, Wei\-Yeh Chang, and I\-Hua Wu\. "Windowing Techniques, the welch method for improvement of Power Spectrum Estimation\." Computers, materials & continua 67\.3 \(2021\)\.
- \[13\]Saha S, Baumert M\. Intra\- and Inter\-subject Variability in EEG\-Based Sensorimotor Brain Computer Interface: A Review\. Front Comput Neurosci\. 2020 Jan 21;13:87\. doi: 10\.3389/fncom\.2019\.00087\. PMID: 32038208; PMCID: PMC6985367\.
- \[14\]Xu, Yilu, Xin Huang, and Quan Lan\. "Selective cross\-subject transfer learning based on riemannian tangent space for motor imagery brain\-computer interface\." Frontiers in Neuroscience 15 \(2021\): 779231\.
- \[15\]Xu, Lichao, et al\. "Cross\-dataset variability problem in EEG decoding with deep learning\." Frontiers in human neuroscience 14 \(2020\): 103\.
- \[16\]Liang, Shuang, et al\. "Adaptive deep feature representation learning for cross\-subject EEG decoding\." BMC bioinformatics 25\.1 \(2024\): 393\.
- \[17\]Dempster, Angus, François Petitjean, and Geoffrey I\. Webb\. "ROCKET: exceptionally fast and accurate time series classification using random convolutional kernels\." Data Mining and Knowledge Discovery 34\.5 \(2020\): 1454\-1495\.
- \[18\]Wu, Haixu, et al\. "Timesnet: Temporal 2d\-variation modeling for general time series analysis\." arXiv preprint arXiv:2210\.02186 \(2022\)\.
- \[19\]Cassani, Raymundo, et al\. "Systematic review on resting‐state EEG for Alzheimer’s disease diagnosis and progression assessment\." Disease markers 2018\.1 \(2018\): 5174815\.
- \[20\]Pal, Anita, et al\. "High delta and gamma EEG power in resting state characterise dementia in Parkinson’s patients\." Biomarkers in Neuropsychiatry 3 \(2020\): 100027\.
- \[21\]Durongbhan, Pholpat, et al\. "A dementia classification framework using frequency and time\-frequency features based on EEG signals\." IEEE Transactions on Neural Systems and Rehabilitation Engineering 27\.5 \(2019\): 826\-835\.
- \[22\]Y\. Song, Q\. Zheng, B\. Liu and X\. Gao, "EEG Conformer: Convolutional Transformer for EEG Decoding and Visualization," in IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol\. 31, pp\. 710\-719, 2023, doi: 10\.1109/TNSRE\.2022\.3230250\.
- \[23\]Zeynali, Mahsa, Hadi Seyedarabi, and Reza Afrouzian\. "Classification of EEG signals using Transformer based deep learning and ensemble models\." Biomedical Signal Processing and Control 86 \(2023\): 105130\.
- \[24\]Sarker SR et al\. A Hybrid Approach to Attention Deficit Hyperactivity Disorder Detection Leveraging Transformer and XGBoost Models Using XSparseFormerNet\. Sci Rep\. 2025 Nov 20;15\(1\):41039\. doi: 10\.1038/s41598\-025\-24919\-3\. PMID: 41266583; PMCID: PMC12635201\.
- \[25\]Swann, Nicole C\., et al\. "Gamma oscillations in the hyperkinetic state detected with chronic human brain recordings in Parkinson’s disease\." Journal of Neuroscience 36\.24 \(2016\): 6445\-6458\.
- \[26\]Pal, Anita, et al\. "High delta and gamma EEG power in resting state characterise dementia in Parkinson’s patients\." Biomarkers in Neuropsychiatry 3 \(2020\): 100027\.
- \[27\]Ali Motie Nasrabadi, Armin Allahverdy, Mehdi Samavati, Mohammad Reza Mohammadi\.\(2020\)\. "EEG data for ADHD / Control children\." Web,
- \[28\]Yu, Guoqi, et al\. "Decentralized Attention Fails Centralized Signals: Rethinking Transformers for Medical Time Series\." arXiv preprint arXiv:2602\.18473 \(2026\)\.