Evaluation Protocols and Cross-Subject Generalization in EEG Emotion Recognition

# Evaluation Protocols and Cross-Subject Generalization in EEG Emotion Recognition
Source: [https://arxiv.org/html/2607.27655](https://arxiv.org/html/2607.27655)
\[1,2\]\\fnmYuwen\\surLi

1\]\\orgdivSchool of Instrument Science and Engineering,\\orgnameSoutheast University,\\orgaddress\\cityNanjing,\\postcode210096,\\countryChina 2\]\\orgdivThe State Key Laboratory of Bioelectronics,\\orgnameSoutheast University,\\orgaddress\\cityNanjing,\\postcode210096,\\countryChina

###### Abstract

Reported accuracy in electroencephalography \(EEG\) emotion recognition depends on the complete evaluation procedure, not only the classifier\. We separate the target quantity, development procedure, and reporting rule, then use one archived dynamical graph convolutional neural network \(DGCNN\) pathway on SEED and SEED\-IV as an illustrative case\. In a protocol\-matched subject\-dependent check, the SEED result was within 1\.47 percentage points of the public reference value; the 3\.40\-point SEED\-IV difference remained unresolved\. Across 30 matched SEED subject\-session trajectories, checkpoint selection based on repeated test\-set evaluation increased mean window accuracy from 0\.7855 at epoch 80 to 0\.8892\. Under five\-fold subject\-disjoint evaluation, validation\-selected checkpoints achieved training\-participant trial accuracies of 0\.9990 on SEED and 0\.9920 on SEED\-IV\. Accuracy for entirely held\-out participants was 0\.5348 \(95% conditional subject\-level bias\-corrected and accelerated \[BCa\] interval \[0\.4667, 0\.5985\]\) on SEED\. The SEED\-IV estimate was 0\.3954 \(\[0\.3343, 0\.4648\]\) and is reported only as secondary sensitivity evidence because its protocol\-matched compatibility check remained unresolved\. The observed train\-to\-held\-out\-subject gaps are inconsistent with simple optimization underfitting, but they do not isolate subject identity from implementation, preprocessing, representation, or distributional factors\. Supporting analyses further showed that participant rankings depended on representation and time scale, while a development\-selected tail\-risk ensemble did not establish a positive gain in a separate final evaluation\. Subject\-dependent, subject\-disjoint, and cross\-session results should therefore be reported as answers to different questions\.

###### keywords:

affective computing; benchmark reproducibility; test\-set reuse; model selection; distributional robustness; subject\-level inference

## 1Introduction

Electroencephalography \(EEG\) emotion recognition is usually presented as a prediction problem: a model receives an EEG\-derived representation and assigns an emotion label\. The reported accuracy, however, is produced by a longer experimental system\. That system decides which people and sessions enter training and testing, what information from the target data is available during development, which checkpoint is selected, whether predictions are scored at the window, trial, or video level, how repeated observations are aggregated, and which population the final expectation is intended to describe\. Two studies may use the same dataset and metric name while estimating different performance quantities\.

We use three task labels consistently\.Subject\-dependent trial\-disjointevaluation tests new trials from participants already represented during model development\.Target\-free subject\-disjointdenotes subject\-disjoint evaluation with no target\-data access: the evaluated participants are absent from fitting and model selection, and neither their covariates nor their labels are used during development\.Cross\-sessionevaluation compares recording sessions from the same participants\. The last setting concerns change over time, not transfer to a new person\. After this definition, we use the more direct phrasesubject\-disjoint evaluation without target\-data access\.

This distinction matters because EEG recordings are strongly clustered\. Hundreds of windows can be obtained from one participant, yet those windows do not create hundreds of independent people\. Subject\-specific signal structure, session conditions, trial duration, preprocessing, and optimization randomness can all affect a score\. Segment\-level random splitting can therefore make a model look effective when the intended use concerns an unseen participant\. Structured cross\-validation should block the independent deployment unit rather than ignore hierarchical dependence\[[25](https://arxiv.org/html/2607.27655#bib.bib25)\]\. Translational EEG studies have documented substantial optimism from leakage across related segments\[[6](https://arxiv.org/html/2607.27655#bib.bib6)\], and the broader machine\-learning literature treats split leakage and adaptive reuse of evaluation data as major threats to reproducibility\[[15](https://arxiv.org/html/2607.27655#bib.bib15)\]\. The problem is not that every subject\-dependent study is invalid\. A personalized classifier can be a legitimate object of study\. The problem is calling personalized, target\-adaptive, and unseen\-subject scores by one undifferentiated name\.

Recent work has improved the field’s experimental infrastructure\. EEGain reviewed 216 EEG emotion\-recognition studies and proposed standardized datasets, splits, metrics, and implementations\[[16](https://arxiv.org/html/2607.27655#bib.bib16)\]\. LibEER assembled a common codebase and benchmark for representative models across widely used datasets\[[21](https://arxiv.org/html/2607.27655#bib.bib21)\]\. Del Pup et al\. compared five cross\-validation settings across several EEG tasks and showed that sample\-based validation can overestimate performance relative to nested subject\-based designs\[[8](https://arxiv.org/html/2607.27655#bib.bib8)\]\. Gil and Hernández\-Sabaté distinguished window\-, state\-, and subject\-level validation within a hierarchical statistical analysis\[[12](https://arxiv.org/html/2607.27655#bib.bib12)\]\. These studies establish that partition and statistical level matter\. A remaining reporting problem is that the split name alone does not record target\-data access, checkpoint selection, the primitive scoring unit, population weighting, or the conditioning set of an uncertainty interval\.

The statistical concept of an estimand provides a useful way to state this problem\. An estimand is the quantity an analysis aims to estimate, whereas an estimate is the numerical result produced from observed data\[[22](https://arxiv.org/html/2607.27655#bib.bib22)\]\. Related descriptions have been proposed for machine\-learning evaluation\[[4](https://arxiv.org/html/2607.27655#bib.bib4)\]\. We use this distinction to specify what each EEG score represents, but the practical focus is experimental: which participants are held out, what target information is available during development, how the model is selected, and how repeated observations are summarized\.

Four research questions organize the study\.RQ1asks when two EEG emotion\-recognition scores can be treated as estimates of the same target risk\.RQ2asks how strongly reported performance changes when the model pathway and dataset label are held approximately fixed but target access, selection, and subject partition change\.RQ3asks whether subject\-level performance rankings are reproducible enough to justify persistent weighting of low\-performing participants\.RQ4asks whether a tail\-risk ensemble that succeeds in train\-only out\-of\-fold development retains its advantage in a separate subject\-dependent evaluation on held\-out trials\.

The study makes four contributions\. First, it defines a three\-part EEG protocol record that separates the target quantity, development procedure, and reporting rule\. Second, it records target\-data access and compares repeated test\-based checkpoint selection with a fixed terminal checkpoint on the same saved trajectories\. Third, it links the prediction unit and participant weighting to a clearly stated conditioning set for uncertainty\. Fourth, it integrates these elements into an auditable EEG reporting checklist and tests the checklist against one archived DGCNN pathway plus three supporting analyses\. The contribution is an EEG\-specific evaluation methodology and empirical stress test, not a new classifier, generalization algorithm, or benchmark leaderboard\.

The Results follow one sequence: a checkpoint\-selection contrast, a protocol\-matched implementation check, and subject\-disjoint evaluation without target\-data access\. The competition cohort, participant\-ranking study, and tail\-risk ensemble are supporting stress tests of population, time\-scale, and development\-boundary assumptions\.

## 2Related work

### 2\.1EEG emotion\-recognition datasets and cross\-subject models

SEED and SEED\-IV are widely used benchmarks for EEG\-based emotion recognition\. SEED introduced a three\-class emotion protocol collected over repeated sessions\[[32](https://arxiv.org/html/2607.27655#bib.bib32)\], and the differential\-entropy representation used in the released feature files was described by Duan et al\.\[[9](https://arxiv.org/html/2607.27655#bib.bib9)\]\. SEED\-IV extended the experimental line to four emotion categories in a multimodal setting\[[33](https://arxiv.org/html/2607.27655#bib.bib33)\]\. Their repeated\-subject structure supports several legitimate questions: within\-person recognition, subject\-dependent generalization to held\-out trials, transfer to an unseen person, and transfer across sessions\. A numerical result cannot be assigned to one of these questions from the dataset name alone\.

Graph neural networks are prominent in this literature because electrodes have spatial organization and functional relations\. DGCNN learns a dynamical adjacency structure while performing emotion classification\[[27](https://arxiv.org/html/2607.27655#bib.bib27)\]\. SOGNN and STGATE provide later examples of cross\-subject graph modelling\[[18](https://arxiv.org/html/2607.27655#bib.bib18),[19](https://arxiv.org/html/2607.27655#bib.bib19)\]\. A recent cross\-subject review organizes the main transfer, adaptation, graph, and generalization challenges in this area\[[20](https://arxiv.org/html/2607.27655#bib.bib20)\], while recent comparative work continues to document strong dependence on dataset and validation design\[[23](https://arxiv.org/html/2607.27655#bib.bib23)\]\. These studies motivate public benchmark comparisons, but their reported values are meaningful only with their associated partitions, preprocessing, evaluation units, and selection rules\. We therefore use an archived DGCNN implementation as a controlled case rather than claiming to reproduce every DGCNN variant or rank architectures across incompatible protocols\.

The broader EEG emotion\-recognition literature is large and application\-oriented\. Reviews describe uses in affective brain\-computer interfaces and document considerable variation in signal representation, dataset construction, and validation\[[10](https://arxiv.org/html/2607.27655#bib.bib10)\]\. Model papers often focus on representational capacity\. Our focus is complementary: we ask which population\-level prediction claim a result supports after the modelling choices have been made\.

### 2\.2Evaluation standardization, leakage, and reproducibility

EEGain and LibEER address fragmented experimental practice through standardized evaluation software and a harmonized benchmark library\[[16](https://arxiv.org/html/2607.27655#bib.bib16),[21](https://arxiv.org/html/2607.27655#bib.bib21)\]\. Del Pup et al\. provide a direct empirical analysis of subject\-based and nested cross\-validation across EEG tasks\[[8](https://arxiv.org/html/2607.27655#bib.bib8)\], while Gil and Hernández\-Sabaté model the hierarchy from windows to subjects and test across\-subject reproducibility\[[12](https://arxiv.org/html/2607.27655#bib.bib12)\]\. These contributions substantially narrow any novelty claim based on split taxonomies, benchmark harmonization, or the general observation that window\-level validation can be optimistic\. The narrower unresolved question is how to record the scientific target and the information used by the complete development procedure\. For example, two subject\-disjoint runs may still select checkpoints differently, aggregate unequal numbers of windows, or attach intervals with different conditioning sets\.

Leakage is the most visible failure mode, but it is not the only source of misinterpretation\. An EEG emotion\-classification study has directly shown that trial\-wise splitting and test\-set reuse can materially alter reported performance\[[17](https://arxiv.org/html/2607.27655#bib.bib17)\]\. Feature selection performed outside cross\-validation can bias error estimates, as shown classically in microarray analysis\[[1](https://arxiv.org/html/2607.27655#bib.bib1)\]\. Reusing a noisy validation criterion for extensive model selection also creates selection bias\[[7](https://arxiv.org/html/2607.27655#bib.bib7)\]\. Nested evaluation is a standard remedy when cross\-validation is used for tuning\[[30](https://arxiv.org/html/2607.27655#bib.bib30)\]\. These principles apply to EEG experiments even when raw samples never cross a nominal train\-test boundary: labels or performance summaries from the final target can still influence the selected checkpoint\.

Reproducibility also requires reporting variation that is not removed by a fixed split\. Neural optimization can change under small perturbations and different random seeds\[[28](https://arxiv.org/html/2607.27655#bib.bib28)\]\. Benchmark variation can arise from data splits, training stochasticity, hyperparameter search, and implementation choices\[[5](https://arxiv.org/html/2607.27655#bib.bib5)\]\. Artifact and reporting checklists improve the ability to reconstruct these choices\[[24](https://arxiv.org/html/2607.27655#bib.bib24)\]\. Our experiments therefore retain exact configuration files, recorded input hashes, per\-subject outputs, seed\-aware aggregation, and independently implemented metric reconstruction\.

### 2\.3Model selection, estimands, and distributional robustness

The estimand perspective separates the scientific target from the calculation used to estimate it\. This is especially helpful when a familiar metric such as accuracy can refer to different populations or interventions\[[22](https://arxiv.org/html/2607.27655#bib.bib22),[4](https://arxiv.org/html/2607.27655#bib.bib4)\]\. In this paper, the intervention\-like element is the evaluation protocol itself: allowing target labels during model selection changes the information set of the procedure and hence the quantity being assessed\. We do not assert that one estimand is universally correct\. A protocol is appropriate when it matches the declared use and its information requirements are feasible\.

Domain adaptation and worst\-group optimization address related but distinct problems\. Domain\-adversarial learning seeks features that are predictive for the task and less informative about domain membership\[[11](https://arxiv.org/html/2607.27655#bib.bib11)\]\. CORAL aligns second\-order feature statistics across domains\[[29](https://arxiv.org/html/2607.27655#bib.bib29)\]\. Group distributionally robust optimization targets the worst observed group loss\[[26](https://arxiv.org/html/2607.27655#bib.bib26)\]\. Such methods can be useful when domains or groups are meaningful and stable\. They do not remove the need for a development\-to\-evaluation boundary\. Persistently up\-weighting participants with low development scores assumes that their relative performance carries to the future evaluation unit\. We test that premise before defining such a group\.

Existing EEG frameworks standardize many protocol components, empirical studies show the consequences of partition choice, and statistical work explains selection bias\. This study joins those elements in one EEG protocol record\. Its independent increment is the explicit separation of target, development, and reporting layers; a same\-trajectory checkpoint\-selection contrast; and reporting rules that state the prediction unit, participant weighting, and uncertainty conditioning\. The participant\-ranking and tail\-risk studies are stress tests of that record rather than separate method claims\.

Table[1](https://arxiv.org/html/2607.27655#S2.T1)positions this contribution against the closest methodological antecedents\.

Table 1:Relationship to the closest methodological antecedents\. The final column states the specific issue retained by the present study rather than treating absent items as deficiencies\.

## 3Evaluation framework

### 3\.1Target estimand, development procedure, and reporting specification

We use three layers rather than treating every experimental choice as one estimand\. Thetarget\-estimand specificationis

ℐ=\(𝒟,𝒴,𝒮tar,𝒰,𝒢,ℓ,𝒫\),\\mathcal\{I\}=\(\\mathcal\{D\},\\mathcal\{Y\},\\mathcal\{S\}\_\{\\mathrm\{tar\}\},\\mathcal\{U\},\\mathcal\{G\},\\ell,\\mathcal\{P\}\),
where𝒟\\mathcal\{D\}identifies the dataset revision and fixed measurement/preprocessing convention;𝒴\\mathcal\{Y\}the outcome and label map;𝒮tar\\mathcal\{S\}\_\{\\mathrm\{tar\}\}the target sampling unit and subject/session scope;𝒰\\mathcal\{U\}the primitive prediction unit;𝒢\\mathcal\{G\}the aggregation operator;ℓ\\ellthe loss or metric functional; and𝒫\\mathcal\{P\}the target population and weighting\. A method\-specific feature map belongs to the learning procedure unless it changes the declared measurement target itself\.

Thedevelopment and estimation procedureis

𝒱=\(𝒮dev,𝒜,ℳ,𝒯\),𝒯=\(ℋ,Θ,Ω\),\\mathcal\{V\}=\(\\mathcal\{S\}\_\{\\mathrm\{dev\}\},\\mathcal\{A\},\\mathcal\{M\},\\mathcal\{T\}\),\\qquad\\mathcal\{T\}=\(\\mathcal\{H\},\\Theta,\\Omega\),
where𝒮dev\\mathcal\{S\}\_\{\\mathrm\{dev\}\}is the realized train/validation/test construction;𝒜\\mathcal\{A\}is the admissible target\-information set;ℳ\\mathcal\{M\}is the checkpoint, hyperparameter, feature, and rerun selection rule;ℋ\\mathcal\{H\}is the representation, model, and optimizer design;Θ\\Thetacontains fixed training and hyperparameter rules; andΩ\\Omegaspecifies stochastic elements such as initialization seeds\. Thereporting specification𝒬\\mathcal\{Q\}separately records the uncertainty unit, resampling or testing rule, treatment of repeated splits and seeds, multiplicity decisions, and the status of primary, secondary, or diagnostic endpoints\. The full auditable protocol record isΠ=\(ℐ,𝒱,𝒬\)\\Pi=\(\\mathcal\{I\},\\mathcal\{V\},\\mathcal\{Q\}\);Π\\Piis not itself described as a single estimand\. Fig\.[1](https://arxiv.org/html/2607.27655#S3.F1)summarizes these layers\.

![Refer to caption](https://arxiv.org/html/2607.27655v1/x1.png)Figure 1:Three\-layer protocol record\. The target\-estimand specification defines the point risk; the development and estimation layer records split construction, target access, selection, and the learning procedure; and the reporting layer defines the uncertainty interpretation without changing the point\-risk functional\. Procedure\-level risk additionally averages over training samples, realized partitions, selection, and training randomness\. Direct model comparisons hold the target, reporting rule, and non\-method procedure coordinates fixedFor subjectii, letDi=\{\(xi​j​u,yi​j\)\}D\_\{i\}=\\\{\(x\_\{iju\},y\_\{ij\}\)\\\}contain primitive unitsuunested in trials or videosjj\. First condition on one fitted predictorf^\\hat\{f\}and on the realized evaluation population\. Its protocol\-specific conditional risk is

Rℐcond\(f^\)=𝔼I∼𝒫\[𝔼J∣I\{ℓ\(YI​J,𝒢\(f^\(XI​J​u\):u∈J\)\)\}\]\.R^\{\\mathrm\{cond\}\}\_\{\\mathcal\{I\}\}\(\\hat\{f\}\)=\\mathbb\{E\}\_\{I\\sim\\mathcal\{P\}\}\\left\[\\mathbb\{E\}\_\{J\\mid I\}\\left\\\{\\ell\\\!\\left\(Y\_\{IJ\},\\mathcal\{G\}\\big\(\\hat\{f\}\(X\_\{IJu\}\):u\\in J\\big\)\\right\)\\right\\\}\\right\]\.
This quantity concerns the fixed fitted model at hand\. A learning procedure is random because it receives a training sampleZtrZ\_\{\\mathrm\{tr\}\}, a realized partitionS⋆S^\{\\star\}, a selected configuration or checkpointM⋆M^\{\\star\}, and stochastic training stateΩ⋆\\Omega^\{\\star\}\. Its procedure\-level risk is therefore

ℜℐ​\(𝒱\)=𝔼Ztr,S⋆,M⋆,Ω⋆​\[Rℐcond​\{f^𝒯​\(Ztr,S⋆,M⋆,Ω⋆\)\}\]\.\\mathfrak\{R\}\_\{\\mathcal\{I\}\}\(\\mathcal\{V\}\)=\\mathbb\{E\}\_\{Z\_\{\\mathrm\{tr\}\},S^\{\\star\},M^\{\\star\},\\Omega^\{\\star\}\}\\left\[R^\{\\mathrm\{cond\}\}\_\{\\mathcal\{I\}\}\\left\\\{\\hat\{f\}\_\{\\mathcal\{T\}\}\(Z\_\{\\mathrm\{tr\}\},S^\{\\star\},M^\{\\star\},\\Omega^\{\\star\}\)\\right\\\}\\right\]\.
The rules𝒮dev\\mathcal\{S\}\_\{\\mathrm\{dev\}\},ℳ\\mathcal\{M\}, andΩ\\Omegadetermine the distributions of the realized partition, selected candidate, and training randomness over which this expectation is taken\. This separation prevents a confidence interval for one fitted pipeline on one fixed subject panel from being described as uncertainty for the full development procedure\. In both risks,𝒫\\mathcal\{P\}assigns equal weight to target subjects unless another weighting is declared, preventing participants with more windows from silently dominating the result\. The reporting rule𝒬\\mathcal\{Q\}does not change either point\-risk functional\. It states which sources of variation the accompanying interval or test represents and is therefore necessary for an auditable performance claim\.

Table[2](https://arxiv.org/html/2607.27655#S3.T2)illustrates five specifications\. Subject\-dependent trial\-disjoint evaluation concerns held\-out trials from observed people, whereas subject\-disjoint evaluation without target\-data access concerns transfer to unseen people\. Target covariates may be allowed under declared unsupervised adaptation\. A checkpoint chosen by repeatedly evaluating the final test set instead describes an adaptive development process rather than an independent final test\.

Table 2:Representative three\-layer protocol records and legitimate interpretations\. Components not shown in a row must still be declared\.
### 3\.2Conditions for comparing numerical results

Two scores estimate the same target quantity when their specificationsℐ\\mathcal\{I\}define the same outcome, target population, prediction unit, aggregation, loss, and weighting\. They use comparable procedures when𝒱\\mathcal\{V\}also matches in split construction, target access, and selection, apart from the learning procedure intentionally compared\. Their reports are comparable when𝒬\\mathcal\{Q\}supports the same interpretation of uncertainty\. A direct method comparison requires all three conditions, with only the declared learning procedure𝒯\\mathcal\{T\}varying\.

Three numerical relationships follow\. Adirect performance comparisonasks which learning procedure has lower risk under a shared\(ℐ,𝒱,𝒬\)\(\\mathcal\{I\},\\mathcal\{V\},\\mathcal\{Q\}\), except for the intended method coordinate\. Aprotocol\-sensitivity comparisonchanges one declared layer, preferably while retaining the same trajectory or data, and measures the resulting change\. Aprotocol\-matched reference checkasks whether a local implementation falls within a predeclared tolerance of a public value under a partially aligned record\. Such a check is not a shared leaderboard entry and does not establish full implementation equivalence\.

An estimate can be technically reproducible while its estimand remains unclear\. Repeated runs may establish numerical repeatability, but not whether a result pertains to unseen participants, calibrated participants, or an adaptively selected test set\. Conversely, a clear estimand can be imprecisely estimated in a small cohort\. Reproducibility, identification, and precision remain separate properties\.

Comparison rule \(dataset\-and\-metric labels are insufficient\)\.If a component ofℐ\\mathcal\{I\}changes, the scores need not estimate the same target risk\. If𝒜\\mathcal\{A\},ℳ\\mathcal\{M\}, or𝒮dev\\mathcal\{S\}\_\{\\mathrm\{dev\}\}changes, the fitted random procedures are not directly comparable even when the target estimand is shared\. If𝒬\\mathcal\{Q\}changes, the point estimand can remain fixed while the interval or test acquires a different interpretation\. Equality of the dataset name and metric label establishes none of these three forms of alignment\. This rule defines a claim boundary; it does not predict the numerical direction of every protocol difference\.

### 3\.3Checkpoint selection using test\-set labels

SupposeKKcandidate checkpoints or configurations are evaluated on the same test set with known labels\. LetA^k\\widehat\{A\}\_\{k\}denote the noisy measured test accuracy of candidatekk, with finite expectationAk=𝔼​\[A^k\]A\_\{k\}=\\mathbb\{E\}\[\\widehat\{A\}\_\{k\}\]\. If the reported score is the maximum observed test accuracy, then

𝔼​\[max1≤k≤K⁡A^k\]≥max1≤k≤K⁡𝔼​\[A^k\]=maxk⁡Ak\.\\mathbb\{E\}\\\!\\left\[\\max\_\{1\\leq k\\leq K\}\\widehat\{A\}\_\{k\}\\right\]\\geq\\max\_\{1\\leq k\\leq K\}\\mathbb\{E\}\[\\widehat\{A\}\_\{k\}\]=\\max\_\{k\}A\_\{k\}\.
Selection inequality\.Becausemaxk⁡A^k≥A^j\\max\_\{k\}\\widehat\{A\}\_\{k\}\\geq\\widehat\{A\}\_\{j\}for each fixedjj, taking expectations and maximizing overjjgives the inequality above\. Thus selecting the maximum among noisy test\-set scores has non\-negative expected optimism relative to the best candidate’s expected measured accuracy\. Equality can occur when the maximizing noise is degenerate or perfectly aligned\.

The inequality supplies a direction, not a realized correction\. Candidate scores are correlated across checkpoints, the best expected candidate may be unknown, and one observed difference cannot be interpreted as a universal bias estimate\. Our empirical analysis therefore reports apaired checkpoint\-selection contrast: the selected and epoch\-80 values are read from the same saved trajectories, so the contrast describes selection sensitivity within those archived runs\.

We count each computation of a candidate performance summary from test\-set labels when that result can influence checkpoint selection, stopping, debugging, or rerunning\. This count makes the use of test\-set information explicit\. Reloading and re\-evaluating a saved candidate is counted again if the result participates in the decision process\.

### 3\.4Clustered observations and the statistical unit

Ifnnnominal observations are arranged in equal clusters of sizemmwith a common intraclass correlationρ\\rho, the standard design\-effect approximation is

D=1\+\(m−1\)​ρ,neff=nD\.D=1\+\(m\-1\)\\rho,\\qquad n\_\{\\mathrm\{eff\}\}=\\frac\{n\}\{D\}\.
The expression shows why multiplying windows does not multiply independent subjects\. Forρ\>0\\rho\>0, the effective sample size grows much more slowly than the window count\. We use this formula only as an explanatory approximation because EEG trials have unequal lengths and a nested subject\-session\-trial structure\. It is not used to construct the reported confidence intervals\.

Primary public\-benchmark accuracy is calculated in three stages\. Predictions are first aggregated to the trial level according to the stated rule\. Optimization seeds are then averaged within each subject\. Finally, the 15 subject values are weighted equally\. Subject\-cluster bootstrap resampling is applied at the participant level\. This preserves the deployment\-relevant independent unit and avoids treating repeated windows or optimization seeds as new participants\. Macro\-F1 is computed under the same declared aggregation scope\.

Cross\-validation fold dispersion is not, by itself, a universal standard error\. Fold estimates are dependent because their training samples overlap, and no universally unbiased estimator of the variance ofKK\-fold cross\-validation exists under unrestricted conditions\[[3](https://arxiv.org/html/2607.27655#bib.bib3)\]\. Cross\-validation may target a procedure averaged over training sets rather than the error of one fitted model, and the distinction matters for uncertainty interpretation\[[2](https://arxiv.org/html/2607.27655#bib.bib2)\]\. Small subject panels can also produce large prediction\-error uncertainty even when fold dispersion appears modest\[[31](https://arxiv.org/html/2607.27655#bib.bib31)\]\. We therefore report subject\-level resampling intervals from the saved aggregate results and use fold summaries descriptively\.

### 3\.5Separate final evaluation after method development

Selection\-evaluation rule\.Conditional on a fixed development procedure, a final score represents its declared target only when final\-evaluation labels and performance summaries derived from them do not influence choices made before the scoring event\.

The statement is operational\. A final\-evaluation label can influence a choice directly, through a displayed metric, or indirectly through a diagnostic derived from the same predictions\. If that happens, the scored object is no longer the pre\-specified procedure\. An apparently minor retry after seeing the final result changes the selection mechanism\.

Our final evaluation used four safeguards\. First, the development algorithm, controls, comparators, metrics, and pass/fail criteria were specified before scoring\. Second, file checksums were recorded for source artifacts and configuration documents\. Third, an independent check reconstructed coverage and comparator identity without calculating final outcomes\. Fourth, the scoring script was executed once, followed by an independent reconstruction from source artifacts to the recorded decisions\. The recognition result was reported as obtained, and the comparator and criteria were not revised afterward\. These are internal procedural controls, not third\-party preregistration or sealing\. Fig\.[2](https://arxiv.org/html/2607.27655#S3.F2)shows the increasing separation between development and final evaluation\.

![Refer to caption](https://arxiv.org/html/2607.27655v1/x2.png)Figure 2:Sequence of evaluation controls used in the study: L1 is the historical personalized result, L2 evaluates the same runs at a fixed epoch, L3 checks a protocol\-matched local implementation against public reference values, L4 holds out complete participants, and L5 evaluates development\-selected methods on held\-out trials\. The levels are not a leaderboard: they differ in target information and model\-selection purpose\. The final labels were unavailable to the development procedure until the final scoring run
### 3\.6Minimum protocol checklist

For each headline result, the record includes the following items:

1. 1\.Which data revision and preprocessing path define𝒟\\mathcal\{D\}?
2. 2\.Which outcome, class map, and label exclusions define𝒴\\mathcal\{Y\}?
3. 3\.Which subjects, sessions, trials, and folds define𝒮\\mathcal\{S\}?
4. 4\.Which target covariates, labels, or performance summaries are available through𝒜\\mathcal\{A\}?
5. 5\.How are checkpoints, features, hyperparameters, and reruns selected throughℳ\\mathcal\{M\}?
6. 6\.What primitive unit𝒰\\mathcal\{U\}is predicted, and how does𝒢\\mathcal\{G\}aggregate it?
7. 7\.Which loss or metricℓ\\ellis reported, and at what scope?
8. 8\.Which target population and weighting define𝒫\\mathcal\{P\}?
9. 9\.Which resampling unit, interval/test, repetition count, and seed define𝒬\\mathcal\{Q\}?
10. 10\.Which learning procedure𝒯\\mathcal\{T\}, comparator identity, training randomness, and number of test\-set evaluations used in selection were fixed or varied?

For a fixed final result, the record also states whether any choice changed after the final score was known\.

The checklist is a disclosure set, not a quality score\.

## 4Data, systems, and controlled protocols

### 4\.1Competition cohort

The release analyzed here was distributed for Task 4, “EEG\-data\-based emotion\-recognition algorithm,” in the Brain\-Computer Interface Track of the 11th National College Student Biomedical Engineering Innovation Design Competition; the source files and task documentation are linked from the[official Pazhou Laboratory download page](https://www.pazhoulab.com/2026/03/8165/)\. It contained 60 participants: 40 identified as healthy controls \(HC\) and 20 identified as participants with depression \(DEP\)\. Each participant contributed eight balanced videos, four neutral and four positive, recorded over 30 EEG channels\. This 60\-person release is the competition analysis subset and must not be assumed to be identical to the complete participant samples described in the associated source articles\. The analysis used the labels supplied by the competition data and did not infer symptom severity, diagnosis, or treatment response\. The dataset documentation requires citation of the associated affective BCI and adolescent\-depression studies\[[14](https://arxiv.org/html/2607.27655#bib.bib14),[13](https://arxiv.org/html/2607.27655#bib.bib13)\]\.

The practical system represented each 10\-s, 250\-Hz segment by 570 fixed features: differential entropy and relative band power for 30 channels in delta \(1–4 Hz\), theta \(4–8 Hz\), alpha \(8–13 Hz\), beta \(13–30 Hz\), and gamma \(30–45 Hz\) bands \(150 features each\); five per\-channel power ratios \(alpha/beta, theta/beta, alpha/theta, beta/gamma, and delta/theta; 150\); Hjorth mobility and complexity \(30 each\); and five\-band differential\-entropy asymmetry for 12 left–right channel pairs \(60\)\. Fourth\-order Butterworth band\-pass filters and a variance floor of10−1210^\{\-12\}were fixed in the feature extractor\.

For this pre\-existing engineering pipeline, each subject’s 40 segment rows were standardized by that subject’s own feature mean and standard deviation\. For held\-out subjects this uses the entire unlabelled target\-subject batch and is therefore a transductive covariate\-normalization rule, not target\-free deployment\. A secondStandardScalerwas fitted only on the normalized training rows\. The classifier wasSVC\(kernel="rbf", C=1\.0, gamma="scale", class\_weight=None, probability=True, random\_state=0\); no validation set or data\-driven hyperparameter selection was used in these three runs\. Each video contained five consecutive segments\. Video prediction averaged their two\-class probability vectors and selected the class with the larger mean probability\.

Three protocols are reported as secondary engineering evidence\. P1 fitted all 40 HC subjects \(1,600 segments\) and evaluated all 20 DEP subjects \(800 segments\); P2 reversed those populations\. P3 used deterministic five\-foldGroupKFoldwith subject as the group, no shuffling, and no random seed: each fold trained on 48 subjects \(1,920 segments\) and tested 12 subjects \(480 segments\), comprising eight HC and four DEP subjects\. The five fold\-level segment and video accuracies were compared by a paired two\-sided Wilcoxon signed\-rank calculation\. Because the five test folds share heavily overlapping training sets, thatpp\-value is a diagnostic summary of these folds rather than a confirmatory population\-level test\. These scores describe different training populations and are not used to infer a biological transfer mechanism\.

A later four\-cell analysis was specified before its results were examined to distinguish target\-cohort difficulty from directional transfer penalty\. For each target population, performance after training on the same population was compared with performance after training on the other population\. The primary variant used global training\-only normalization\. A subject\-batch normalization analysis was declared transductive because it used the unlabelled target batch\. The differential penalty was

Δdir=\{A​\(DEP→DEP\)−A​\(HC→DEP\)\}−\{A​\(HC→HC\)−A​\(DEP→HC\)\}\.\\Delta\_\{\\mathrm\{dir\}\}=\\\{A\(\\mathrm\{DEP\}\\\!\\rightarrow\\\!\\mathrm\{DEP\}\)\-A\(\\mathrm\{HC\}\\\!\\rightarrow\\\!\\mathrm\{DEP\}\)\\\}\-\\\{A\(\\mathrm\{HC\}\\\!\\rightarrow\\\!\\mathrm\{HC\}\)\-A\(\\mathrm\{DEP\}\\\!\\rightarrow\\\!\\mathrm\{HC\}\)\\\}\.
A positive value would support a larger DEP\-target transfer penalty\. Negative or near\-zero values do not support the originally proposed HC\-to\-DEP\-specific mechanism\.

The decomposition used 160 balanced repeated subsamples with RNG seed 20260710\. In every repeat, 15 source subjects from each population were available for the within\- and cross\-population fits, while five HC and five DEP target subjects were selected from the eligible remainder; the schedule balanced repeated target appearances across subjects\. The primary endpoint was 10\-s segment accuracy\. Subject\-level within\- and cross\-population accuracies were formed by averaging all appearances of each target subject before population contrasts were calculated\. Two\-sample percentile bootstrap intervals resampled the HC and DEP subject arrays independently with 10,000 draws\. The pre\-specified directional permutation test used 10,000 label permutations and the greater\-than alternative\. These analyses treat subjects, not repeated subsamples or segments, as the inferential units\.

### 4\.2SEED and SEED\-IV

SEED and SEED\-IV each contain repeated EEG sessions from 15 participants\. The present study used the differential\-entropy representations and archived data organization expected by the selected LibEER revision\. Dataset\-level details and the released differential\-entropy representation follow the sources required by the dataset licence\[[32](https://arxiv.org/html/2607.27655#bib.bib32),[9](https://arxiv.org/html/2607.27655#bib.bib9),[33](https://arxiv.org/html/2607.27655#bib.bib33)\]\. The experiments are separated by protocol because they use different session scopes and subject partitions\.

The historical checkpoint study used 30 subject\-session trajectories from SEED sessions 1 and 2\. It followed the archived subject\-dependent, trial\-disjoint pathway in which test\-set accuracy was examined across training checkpoints\. A separate protocol\-matched reference check followed the ordered split defined by the selected LibEER revision for DGCNN on SEED and SEED\-IV\. These are subject\-dependent evaluations of a historical procedure or local implementation, not unseen\-subject estimates\.

The strict study used session 1 and a five\-fold subject\-disjoint design\. In each fold, nine subjects formed the training set, three formed the validation set, and three formed the test set\. Each dataset therefore contributed 15 unique held\-out test subjects\. Three paired optimization seeds were run per dataset\-fold cell\. The final subject\-balanced trial endpoint averaged the three seed values within subject and then averaged equally across all 15 subjects\. This produces one prespecified estimate of cross\-subject generalization; it is not a cross\-session experiment\.

After the advisor review, a supplemental fitting diagnostic was specified before execution\. It loaded each of the 30 existing validation\-selected checkpoints in evaluation mode and predicted only that cell’s nine training subjects\. Window and mean\-logit trial metrics were computed with the original representation and aggregation code\. The script did not retrain a model, open a saved test\-prediction array, or recompute a test prediction\. Fold summaries first averaged the three optimization seeds and then described the five fold means\. Because training subjects recur across folds, no confidence interval or independent\-sample interpretation was attached to the training accuracy\.

To show the training dynamics rather than only the endpoint, a second post\-review diagnostic reran the same 30 strict dataset\-fold\-seed cells for the fixed 150\-epoch schedule\. It recorded minibatch training loss and online training\-window accuracy together with validation\-window accuracy, macro\-F1, and trial accuracy at every epoch\. Test rows were not loaded during these reruns, and the previously reported subject\-disjoint test values were used only as reference endpoints in the final plot\. A cell was accepted only if its selected epoch, selected validation macro\-F1, and checkpoint parameters reproduced the corresponding original strict run\.

Table[3](https://arxiv.org/html/2607.27655#S4.T3)summarizes the datasets, protocols, and independent evaluation units used in these analyses\.

Table 3:Datasets and independent evaluation units\.Table[4](https://arxiv.org/html/2607.27655#S4.T4)records the target, development, and reporting choices for each headline result\.

Table 4:Three\-layer protocol summary for headline evidence\. CI denotes confidence interval; inference entries describe conditional fitted\-pipeline risk unless explicitly stated otherwise\.The table records the target, realized development procedure, and reporting rule for one fixed pipeline and participant panel in every empirical row\. Subject\-resampling intervals quantify variation of the corresponding conditional fitted\-pipeline risk; none integrates over new training populations, alternative split realizations, repeated selection, and all stochastic training states required for a procedure\-level interval\.

### 4\.3Archived model and implementation controls

DGCNN was chosen because it is a recognizable graph baseline with an available archived implementation, not because the study seeks a new architecture result\. The experiments used LibEER commit39dc27e504e14138767b87ce8bce485380fd4f5a, including its historical all\-ones ChebyshevT0T\_\{0\}implementation\. Ordered train, validation, and test indices were regression\-tested against that revision\. Coverage audits checked expected subject\-session\-model cells, class support, and artifact completeness\. Separately implemented scripts reconstructed the principal aggregates directly from saved prediction artifacts\.

All six figures were generated from frozen aggregate JSON or CSV artifacts using Python and Matplotlib, then exported as vector PDF and EPS files\.

For the strict study, Pythonrandom\.Random\(2024\)created five fixed three\-subject groups\. Foldiitested groupii, validated on the next group cyclically, and trained on the remaining nine subjects\. Optimization seeds were 2024, 2025, and 2026\. DGCNN was trained for 150 epochs with AdamW, weight decay10−410^\{\-4\}, optimizer epsilon10−410^\{\-4\}, cross\-entropy, and the archival sparse\-L2 penalty 0\.01\. The selected checkpoint maximized pooled validation\-window macro\-F1 with an earliest\-epoch tie rule\. SEED used batch size 16 and learning rate 0\.001; SEED\-IV used batch size 32 and learning rate 0\.0015\. Trial predictions were the argmax after averaging logits over one\-second windows\. Exactly one test evaluation followed checkpoint loading\.

No strict test outcome was used for early stopping, hyperparameter tuning, feature choice, or rerun decisions\. Exact file hashes, environment details, and cell ledgers were retained\. Public DGCNN values were used only as reference values\. A tolerance check assessed whether the protocol\-matched local run fell within two percentage points of each reference; failing the tolerance did not authorize tuning against that value\.

### 4\.4Sequence of evaluation controls

The five levels in Fig\.[2](https://arxiv.org/html/2607.27655#S3.F2)order the experiments by the separation between development information and the final evaluation unit\. Level 1 is a historical subject\-dependent result whose checkpoint was chosen using test labels\. Level 2 evaluates the same saved trajectories at terminal epoch 80\. Level 3 checks the archived DGCNN pathway against public subject\-dependent reference values under a reconstructed ordered split\. Level 4 withholds complete participants from fitting and selection\. Level 5 selects the classical ensemble during development and evaluates the frozen choice on held\-out trials\.

These levels are not a performance ranking\. Moving between them can change the target population, data access, selection rule, scoring unit, or weighting\. Only the paired checkpoint\-selection contrast holds the saved trajectories, model family, and epoch schedule fixed\. Differences between the subject\-dependent and subject\-disjoint rows must not be interpreted as a causal leakage penalty\.

### 4\.5Supporting analysis protocols

The participant\-ranking analysis tested whether a low\-performing group was stable enough to justify persistent weighting\. Session\-local leave\-one\-subject\-out RBF SVM models produced subject accuracies for class\-balanced split halves and complete sessions\. The primarymean\_stdrepresentation and the pre\-specifiedmeansensitivity were assessed separately\. The 0\.30 split\-half and 0\.40 cross\-session thresholds were fixed engineering criteria for deciding whether method development should continue; they were not statistical\-significance cutoffs or general definitions of reliability\. Full partition, bootstrap, permutation, and hierarchy details appear in Online Resource 1, Section S1\.

The tail\-risk study fitted a simplex mixture of seven classical components from train\-only out\-of\-fold probabilities\. Mean\-risk, strongest\-single, and uniform controls were fixed before validation scoring\. The selected mixture was then evaluated once on existing held\-out trials from the same participants\. Its 555 engineered features differ from the 62\-by\-5 DE/LDS tensor used by DGCNN, so this is a comparison of complete methods rather than an ablation of the tail objective\. Online Resource 1, Sections S3–S5, reports the optimization equation, search grid, selected weights, controls, and decision outcomes\. Online Resource 2 gives the selected component order, parameters, and full\-precision mixture weights in machine\-readable form\.

Logistic regression and random forest were added after advisor review as descriptive controls\. Their configurations were fixed before execution, they used the same 555\-feature representation and final trial partitions as the classical ensemble, and they did not alter the original comparator or decision\.

### 4\.6Statistical analysis

Accuracy and macro\-F1 were reported at their declared aggregation levels\. For strict public\-benchmark endpoints, three optimization seeds were averaged within participant before participants were equally weighted; 95% subject\-level BCa intervals used 20,000 resamples and seed 20260713\. Paired final\-evaluation accuracy differences retained the method pairing within each participant and used the same conditional interpretation with 10,000 resamples and seed 240714\. Performance\-ranking stability used Spearman correlations for independent halves and sessions, summarized by the median of the three pre\-specified comparisons in each family; aligned subject\-level BCa intervals and joint one\-sided permutation tests each used 20,000 draws and seed 20240713\. Competition population contrasts used two\-sample subject\-level percentile bootstrap intervals rather than BCa intervals, with 10,000 resamples and seed 20260710 plus fixed endpoint\-specific offsets\. The 0\.30 and 0\.40 thresholds were engineering decision gates, not clinical cutoffs\.

All reported intervals resample participants from the fixed panel and are conditional on the fitted fold models, data revision, partitions, preprocessing, training settings, and stated seeds\. They do not account for dependence induced by overlapping training sets across folds and are not intervals for the full procedure\-level riskℜℐ​\(𝒱\)\\mathfrak\{R\}\_\{\\mathcal\{I\}\}\(\\mathcal\{V\}\)\. In particular, they do not integrate uncertainty from a training\-population redraw, a new split realization, repeated model selection, or stochastic retraining of the complete development procedure\.

No additional post\-hoc hypothesis tests were added during manuscript drafting\. The post\-review logistic\-regression and random\-forest controls were descriptive only, used fixed configurations, and were not allowed to replace the original comparator or change the confirmatory decision\. The competition analysis was likewise not redefined after the directional hypothesis failed\.

## 5Results

### 5\.1Checkpoint\-selection contrast and implementation check

The paired checkpoint\-selection contrast used the same 30 SEED subject\-session training trajectories\. Selecting checkpoints by repeated test\-set evaluation produced mean window accuracy 0\.8892, whereas terminal epoch 80 produced 0\.7855\. The paired difference was 0\.1036, and 28 of 30 trajectory differences were positive \(Fig\.[3](https://arxiv.org/html/2607.27655#S5.F3)a\)\. Because the trajectories, model family, and epoch schedule are shared, this is the most controlled protocol comparison in the study\. It describes selection sensitivity in these archived runs, not a universal bias correction\. Each trajectory used 80 test\-set checkpoint scores for selection and one evaluation after reload, or 81 test\-set computations per trajectory and 2,430 in total\.

![Refer to caption](https://arxiv.org/html/2607.27655v1/x3.png)Figure 3:DGCNN checkpoint and implementation checks\. \(a\) Window accuracy for 30 matched SEED subject\-session trajectories at terminal epoch 80 and after checkpoint selection based on repeated test\-set evaluation; diamonds denote means\. \(b\) Protocol\-matched local results relative to public subject\-dependent reference values and the pre\-specified \+/\-2 percentage\-point compatibility bandsThe protocol\-matched reference check then evaluated whether the archived DGCNN pathway behaved plausibly under a public subject\-dependent setting\. On SEED, subject\-equal window accuracy was 0\.8108 compared with the public reference value of 0\.8255\. The difference of−0\.0147\-0\.0147was inside the pre\-specified two\-percentage\-point tolerance\. On SEED\-IV, local accuracy was 0\.4899 compared with 0\.5239, a difference of−0\.0340\-0\.0340outside that tolerance \(Fig\.[3](https://arxiv.org/html/2607.27655#S5.F3)b\)\. Ordered split reconstruction passed for all 270 subject\-session rows\. The result supports numerical reproduction within tolerance on SEED only\.

A read\-only SEED\-IV consistency audit did not change or rerun the scientific result\. Eight verifiable audit groups passed: pinned source identity; file and four\-class label tables; raw\-trial order and first\-sample handling; preprocessing\-cache provenance; preprocessing\-function identity; trial\-split equivalence; model training and checkpoint selection; and final aggregation\. In particular, the released arrays were processed at 200 Hz with a 0\.3–50 Hz pass band, the five stated frequency bands, non\-overlapping one\-second segments, and the SEED\-IV four\-class labels0–33, rather than the three\-class SEED mapping\. These checks covered all 45 subject\-session recordings and none explained the−0\.0340\-0\.0340difference\. Exact equivalence to the historical public run could not be established because its dependency lock, processed arrays, checkpoints, and unit predictions were unavailable \(Online Resource 1, Table S2\)\. The SEED\-IV result therefore remains an unresolved protocol\-matched compatibility difference\. It is not an exact reproduction and is used only as secondary sensitivity evidence\.

### 5\.2Strict subject\-disjoint generalization

Training fit rules out optimization failure\.Across five folds, the validation\-selected checkpoints achieved mean subject\-equal training trial accuracy of 0\.9990 on SEED and 0\.9920 on SEED\-IV\. The corresponding train\-minus\-held\-out\-subject gaps were 0\.4642 and 0\.5966\. The model therefore fitted the training participants under both evaluated pipelines; simple failure to optimize the training objective cannot explain the low performance on unseen participants\. This diagnostic does not, however, isolate subject identity from preprocessing, implementation, representation, training\-distribution, or other factors\.

The complete learning histories show the same separation during optimization \(Fig\.[4](https://arxiv.org/html/2607.27655#S5.F4)a–b\)\. Mean online training\-window accuracy rose from 0\.8262 at epoch 1 to 0\.9938 at epoch 150 on SEED and from 0\.6839 to 0\.9807 on SEED\-IV\. In contrast, mean validation\-trial accuracy peaked at 0\.5719 at epoch 2 on SEED and 0\.4148 at epoch 3 on SEED\-IV, then ended at 0\.5156 and 0\.3741\. The validation\-selected epoch had median 16 \(interquartile range 3\.5–52\) for SEED and 10 \(3–82\.5\) for SEED\-IV across the 15 dataset\-fold\-seed cells\. All 30 histories reproduced their retained checkpoint states, without loading test rows or calculating test performance\. The horizontal held\-out\-participant values in Fig\.[4](https://arxiv.org/html/2607.27655#S5.F4)are the previously reported endpoints, not test curves\.

![Refer to caption](https://arxiv.org/html/2607.27655v1/x4.png)Figure 4:DGCNN learning histories under strict five\-fold subject\-disjoint evaluation\. \(a\) SEED and \(b\) SEED\-IV mean online training\-window and validation\-trial accuracy over the 15 dataset\-fold\-seed cells at each epoch; shaded bands are interquartile ranges\. Dotted vertical lines mark the median validation\-selected epoch\. Dash\-dotted horizontal lines copy the previously reported accuracies for unseen subjects after validation\-based checkpoint selection; they are reference endpoints, not test curvesAfter these diagnostics, the frozen subject\-disjoint endpoints were interpreted at their intended scope\. SEED subject\-balanced trial accuracy was 0\.5348 with a 95% conditional subject\-level BCa interval of\[0\.4667,0\.5985\]\[0\.4667,0\.5985\]; macro\-F1 was 0\.4438\. SEED\-IV accuracy was 0\.3954 with interval\[0\.3343,0\.4648\]\[0\.3343,0\.4648\]; macro\-F1 was 0\.3215\. Because the protocol\-matched SEED\-IV check remained unresolved, its strict estimate is reported only as a secondary sensitivity analysis rather than given the same evidential weight as the SEED estimate\. Each value averages the three optimization seeds within participant before weighting the 15 participants equally\. Both accuracies are numerically above nominal chance, but no chance\-level significance test was pre\-specified\. They estimate performance for unseen participants in session 1 and are not failed reproductions of subject\-dependent scores\.

Table[5](https://arxiv.org/html/2607.27655#S5.T5)brings the public\-benchmark results together while preserving their different protocol interpretations\.

Table 5:Controlled public\-benchmark evaluation\. Values from different rows are not a common leaderboard\. Intervals are conditional on the observed participants, fixed partitions, fitted models, training rules, and stated seeds\.The table is an evaluation record, not a leaderboard\. Only the 0\.1036 same\-trajectory contrast holds the saved training trajectories fixed\. The remaining differences also change subject partition, session scope, checkpoint rule, split construction, scoring unit, aggregation, or weighting\. Fig\.[3](https://arxiv.org/html/2607.27655#S5.F3)separates the comparisons into panels so that the high subject\-dependent scores establish implementation behavior without being presented as estimates of unseen\-subject performance\.

### 5\.3Competition performance and cohort composition

The competition cohort provides a supporting population stress test rather than the paper’s main benchmark\. The rich\-feature RBF\-SVM system obtained segment/video accuracies of 0\.636/0\.688 for P1, 0\.708/0\.803 for P2, and 0\.728/0\.802 for the subject\-grouped mixed protocol\. The five P3 folds gave a mean video\-minus\-segment difference of 0\.0742 withp=0\.0625p=0\.0625\. Because the folds share training participants, this is an aggregation diagnostic rather than a population\-level test\.

A four\-cell decomposition separated source and target cohorts\. Under training\-only normalization, the HC\-target within\-minus\-cross penalty was 0\.104, whereas the DEP\-target penalty was 0\.039\. Their difference was−0\.065\-0\.065, with a 95% interval of\[−0\.117,−0\.014\]\[\-0\.117,\-0\.014\], contrary to the proposed larger DEP\-target penalty\. Sensitivity analyses did not support a direction\-specific transfer mechanism\. They did retain a descriptive finding that HC within\-population accuracy exceeded DEP within\-population accuracy under the evaluated systems\.

### 5\.4Subject\-level performance dispersion and ranking stability

Strict trial accuracy varied widely across the 15 subjects in both public datasets \(Fig\.[5](https://arxiv.org/html/2607.27655#S5.F5)a\)\. Subject\-averaged accuracy ranged from approximately 0\.27 to 0\.76 on SEED and 0\.24 to 0\.60 on SEED\-IV\. This dispersion motivated a reliability check before any persistent low\-performing group was used for weighting\.

![Refer to caption](https://arxiv.org/html/2607.27655v1/x5.png)Figure 5:Subject\-level performance dispersion and ranking stability\. \(a\) Seed\-averaged trial accuracy for all 15 subjects under the strict first\-session protocol; horizontal lines are subject means\. \(b\) Primary SEED\-IV session\-1 split\-half accuracies with a dashed y=x reference line; dataset\-local subject identifiers remain available in the figure source table\. \(c\) Median split\-half and cross\-session Spearman correlations for the primary mean\-plus\-standard\-deviation representation \(circles\) and pre\-specified mean\-only sensitivity \(squares\), with subject\-level BCa intervals conditional on the fitted models; black vertical ticks mark pre\-specified engineering gates for stopping method development, not significance cutoffsOn primary SEED\-IV, themean\_stdrepresentation gave median split\-half and cross\-session correlations of 0\.1700 and 0\.0470\. Their conditional 95% BCa intervals were\[−0\.2790,0\.5291\]\[\-0\.2790,0\.5291\]and\[−0\.1925,0\.5548\]\[\-0\.1925,0\.5548\]\. The pre\-specifiedmeansensitivity changed the split\-half median to 0\.5667 but left the cross\-session median at−0\.0457\-0\.0457\(Fig\.[5](https://arxiv.org/html/2607.27655#S5.F5)b–c\)\. They=xy=xline displays split\-half agreement directly; the marks in panel c are engineering stopping criteria, not significance thresholds\. Because the primary SEED\-IV gate failed, SEED remained diagnostic\. Full pairwise values, permutation results, and SEED diagnostics are reported in Online Resource 1, Section S1 and Table S1\.

The result does not imply that participants have identical performance\. It shows that the ordering depended on representation and did not persist across sessions under the primary design\. The evidence was therefore insufficient to define a stable low\-performing group for subsequent method development\.

### 5\.5Development selection and separate subject\-dependent final evaluation

The tail\-risk mixture passed its pre\-specified development criteria\. Relative to the strongest single classical component, validation accuracy improved by 0\.0286 on SEED and 0\.0510 on SEED\-IV\. All leave\-one\-subject\-out weight refits converged, and the stability checks passed \(Fig\.[6](https://arxiv.org/html/2607.27655#S5.F6)a\)\.

![Refer to caption](https://arxiv.org/html/2607.27655v1/x6.png)Figure 6:Development and final\-evaluation evidence for the tail\-risk case study\. \(a\) Sanitized\-validation gains over the OOF\-selected strongest single classical component\. \(b\) Subject\-equal final differences relative to the fixed DGCNN comparator; accuracy bars are paired 95% subject\-level BCa intervals conditional on the fitted models \(n=15 per dataset\), and macro\-F1 points are descriptive\. \(c\) Mean\-risk minus tail\-risk final environment\-CVaR improvement\. The mechanism endpoint transferred, but the pre\-specified recognition criteria failedThe separate subject\-dependent final evaluation did not establish a recognition gain on held\-out trials\. Tail\-risk accuracy was 0\.7476 on SEED and 0\.5783 on SEED\-IV\. Relative to DGCNN, the paired accuracy differences were 0\.0193 with a conditional 95% BCa interval of\[−0\.0377,0\.0969\]\[\-0\.0377,0\.0969\]and−0\.0794\-0\.0794with interval\[−0\.1683,0\.0366\]\[\-0\.1683,0\.0366\], respectively \(Fig\.[6](https://arxiv.org/html/2607.27655#S5.F6)b\)\. These intervals do not establish positive gains and do not establish equivalence\. Because development and final partitions contain the same participants, the comparison does not estimate transfer to unseen subjects\.

Comparisons with logistic regression and random forest led to the same practical conclusion: the tail mixture did not show a consistent recognition advantage\. It exceeded both controls on SEED, but logistic regression exceeded it by 0\.0365 on SEED\-IV\. Relative to the mean\-risk mixture, the tail objective reduced outer\-environment CVaR by 0\.0497 on SEED and 0\.0001 on SEED\-IV, showing that optimization changed the intended tail\-loss criterion \(Fig\.[6](https://arxiv.org/html/2607.27655#S5.F6)c\)\. The corresponding accuracy changes within the same 555\-feature ensemble were−0\.0046\-0\.0046and\+0\.0002\+0\.0002, so the change in tail loss did not yield a consistent recognition benefit even within that representation\. Comparisons with DGCNN additionally change the representation \(555 engineered features versus a 62\-by\-5 DE/LDS tensor\) and model class, and therefore cannot isolate the tail objective\. These results support activation of the intended optimization mechanism, but not a representation\-independent recognition gain; testing transfer would require a within\-representation ablation\. Further development under this design was stopped\. Online Resource 1, Tables S3–S5, reports the full configurations, controls, and decision outcomes\.

## 6Discussion

### 6\.1Core protocol findings

RQ1: When are scores comparable?Two scores address the same target only when data revision, outcome, target population, prediction unit, metric, aggregation, and weighting agree\. A direct method comparison also requires compatible target\-data access, partition construction, selection, and uncertainty interpretation\. Subject\-dependent and subject\-disjoint results can both be valid, but they answer different deployment questions\.

RQ2: How sensitive was the archived pathway to protocol choices?Repeated test\-based checkpoint selection increased accuracy by 0\.1036 relative to terminal epoch 80 on the same saved trajectories\. This paired contrast has a narrow interpretation because most experimental components are held fixed\. By contrast, the difference between subject\-dependent and subject\-disjoint results combines changes in target population, access, selection, session scope, scoring unit, and weighting\. It is not a causal penalty attributable to any one component\.

The learning curves clarify the low subject\-disjoint scores without resolving every possible cause\. Training\-participant accuracy above 0\.99 is inconsistent with simple optimization underfitting\. Under this evaluated pathway, the large loss in accuracy appeared when prediction moved to held\-out participants rather than when the model fitted the training participants\. This statement remains conditional on the archived implementation, preprocessing, representation, partitions, and seeds\. It does not identify subject identity as the sole source of the gap\.

High subject\-dependent accuracy is therefore evidence for the protocol that produced it, not for every subject\-generalization claim\. Target covariates or calibration labels may be legitimate when they are part of the intended use\. They must still be reported\. Window\-level optimization measures, trial\-level endpoints, and participant\-level uncertainty also serve different purposes; a large number of windows does not create a large sample of independent people\.

### 6\.2Supporting stress tests

RQ3: Do participant performance rankings persist?The answer depended on representation and time scale\. The mean\-only sensitivity produced stronger within\-session split\-half ranking than the primary mean\-plus\-standard\-deviation representation, but neither established persistent cross\-session ranking on primary SEED\-IV\. A heavy error tail or one positive split\-half comparison is insufficient to define a persistent low\-performing group\.

RQ4: Did the development\-selected tail method transfer to final trials?The objective improved its intended tail\-loss measure, but its recognition advantage was not consistent across datasets or comparators\. The final intervals did not establish a positive gain\. Because the classical ensemble and DGCNN use different representations, this is a method\-level comparison rather than an ablation of the tail objective\.

The competition analysis makes the same methodological point in a different population\. It supported a descriptive difference in target\-cohort difficulty under the evaluated systems, but not the proposed direction\-specific HC\-to\-DEP transfer mechanism\. These three studies are supporting cases: each tests whether a development\-stage pattern survives a change in population, time scale, or final evaluation boundary\.

### 6\.3Relation to prior evaluation work

EEGain and LibEER already provide standardized EEG evaluation infrastructure\[[16](https://arxiv.org/html/2607.27655#bib.bib16),[21](https://arxiv.org/html/2607.27655#bib.bib21)\]\. Del Pup et al\. show how sample\-level and non\-nested validation can overestimate EEG performance\[[8](https://arxiv.org/html/2607.27655#bib.bib8)\], and Gil and Hernández\-Sabaté model the hierarchy from windows to subjects\[[12](https://arxiv.org/html/2607.27655#bib.bib12)\]\. Lei et al\. directly examine EEG trial splitting and test reuse\[[17](https://arxiv.org/html/2607.27655#bib.bib17)\], while Binette and Reiter provide the general estimand vocabulary\[[4](https://arxiv.org/html/2607.27655#bib.bib4)\]\.

These contributions leave little basis for a broad novelty claim based on split taxonomies, benchmark harmonization, or the observation that subject\-level validation is harder\. The independent increment here is narrower: target, development, and reporting are recorded separately; target\-data access and checkpoint selection are explicit; prediction unit and participant weighting are linked to the interval conditioning set; and these items are assembled into one auditable EEG checklist\. The matched checkpoint\-selection contrast and the three supporting studies test that integrated record against stored evidence\. This is an EEG\-specific evaluation methodology, not a new general theory\.

The mixed compatibility result also sets a useful boundary\. SEED reproduced the public value within tolerance, whereas SEED\-IV did not\. The failed SEED\-IV tolerance and unrecoverable historical runtime remain visible rather than being converted into a claim of exact reproduction\.

### 6\.4Reporting recommendations

Table[6](https://arxiv.org/html/2607.27655#S6.T6)translates the analysis into minimum reporting items\.

Table 6:Minimum reporting checklist for subject\-related EEG generalization studies\.Personalized, target\-adaptive, and subject\-disjoint performance without target\-data access should appear in separate table blocks or figure panels\. A public reference value should be accompanied by a statement of which protocol components match\. If important components differ, the value supplies context rather than a direct ranking\. A method developed on multiple folds should receive a final evaluation unit that did not influence its design; otherwise the claim remains developmental\.

A headline result should state the target and information boundary\. For example: “Under a first\-session, five\-fold subject\-disjoint protocol with no target\-data access, validation\-based checkpoint selection, trial scoring, and equal weighting of 15 participants, DGCNN obtained accuracy 0\.5348, with a subject\-level BCa interval of \[0\.4667, 0\.5985\] conditional on the fitted fold models\.” Full implementation details can remain in Methods and the supplement\.

A negative decision should likewise state the candidate, comparator, endpoint, interval, and consequence\. In this study, the tail\-risk mixture did not establish a positive paired accuracy gain over DGCNN on either final dataset, so further development under this design was stopped\. That statement does not imply that the algorithm cannot help under another representation or target\.

### 6\.5Limitations

The public\-benchmark evaluation examines one archived DGCNN pathway\. It is an illustrative case, not an architecture\-general test\. Other implementations, representations, and model families may respond differently\. A multi\-architecture study would be needed to estimate how widely the observed protocol effects generalize\.

SEED and SEED\-IV contain 15 participants each, and the strict analysis is limited to session 1\. The conditional subject\-level intervals expose variation within the observed panel but do not create broad population coverage\. They exclude uncertainty from redrawing the training population, generating new folds, retraining the full model set, or repeating model selection\.

The large numerical difference between historical subject\-dependent and strict subject\-disjoint results is not a controlled decomposition\. Only the paired checkpoint\-selection contrast holds the saved trajectories fixed\. The remaining comparisons change several target and procedure components at once\.

The tail\-risk comparison also changes representation and model family\. The classical mixture uses 555 engineered features, whereas DGCNN uses a 62\-channel by five\-band DE/LDS tensor\. The final result cannot assign their difference to the tail objective alone\.

The competition cohort is secondary, has unequal HC/DEP sample sizes, and includes a transductive normalization route that requires an unlabelled target batch\. Its five P3 folds share training participants, so the Wilcoxon result is diagnostic\. Diagnostic labels are used only as supplied grouping variables, and no clinical or symptom\-scale claim is made\.

Finally, recorded checksums and independent reconstruction reduce internal ambiguity but do not constitute preregistration, third\-party sealing, or independent replication\. The analytical propositions establish limited directions or approximations rather than a new statistical theory\. The manuscript should be judged on its integrated protocol record and controlled evidence, not as a state\-of\-the\-art classifier paper\.

## 7Conclusion

This study shows why an EEG accuracy should be reported together with the procedure that produced it\. On the same saved trajectories, repeated test\-based checkpoint selection increased mean accuracy by 0\.1036 relative to terminal epoch 80\. Under subject\-disjoint evaluation without target\-data access, the archived DGCNN pathway fitted its training participants almost perfectly but performed much less accurately on held\-out participants\. The loss in accuracy appeared at the held\-out\-participant boundary rather than as a failure to fit the training participants\. The evidence does not isolate a single cause for that gap\.

The supporting analyses also clarify which claims and follow\-up methods are not yet justified\. Participant rankings depended on representation and did not persist across sessions on primary SEED\-IV\. The tail\-risk objective improved its intended loss measure but did not establish a recognition gain in the final comparison\. These outcomes argue against defining a persistent low\-performing group or promoting a robustness method from one development result alone\.

Future EEG studies should state whether the intended user appears during development, record all target\-data access, separate model selection from final evaluation, and aggregate at the independent deployment unit\. Training fit should be reported when low held\-out\-participant accuracy could otherwise be mistaken for a broken optimizer\. Subject\-dependent recognition, transfer to unseen participants, and change across sessions remain useful questions, but they require different experiments and different claims\.

## Statements and Declarations

### Acknowledgements

The authors thank the organizing committee of the 11th National College Student Biomedical Engineering Innovation Design Competition and the Task 4 team of its Brain\-Computer Interface Track, including Pazhou Laboratory, for providing the competition data, documentation, and evaluation platform\.

### Funding

This work was supported in part by the National Natural Science Foundation of China under Grant 62571123; in part by the Basic Research Program of Jiangsu Province under Grant BK20252010; in part by the Fundamental Research Funds for the Central Universities \(2242026RCB0024\)\.

### Competing interests

The authors have no relevant financial or non\-financial interests to disclose\.

### Author contributions

Hanting Suo: Conceptualization, Methodology, Software, Validation, Formal analysis, Investigation, Data curation, Visualization, Writing – original draft, Writing – review and editing\. Yuwen Li: Conceptualization, Methodology, Supervision, Project administration, Resources, Funding acquisition, Writing – review and editing\. Both authors approved the final manuscript and agree to be accountable for the work\.

### Ethics approval and consent to participate

Not applicable to the present secondary analysis\. The authors did not recruit or contact participants, conduct interventions, or access directly identifying information\. Ethical oversight and consent for the original data collection remain the responsibility of the respective data providers\. The datasets were used in accordance with their applicable access and licence terms\.

### Consent for publication

Not applicable\. This article contains no identifiable participant information, images, or case details\.

### Data availability

The SEED and SEED\-IV data were obtained from the BCMI laboratory under the SJTU Emotion EEG Dataset License Agreement and are available to eligible academic researchers through the[official application portal](https://bcmi.sjtu.edu.cn/ApplicationForm/apply_form/)\. The licence permits academic research use but prohibits redistribution; raw SEED and SEED\-IV files are therefore not included with this article\. The competition release can be requested or downloaded through the organizer’s[official Task 4 page](https://www.pazhoulab.com/2026/03/8165/)subject to the current access and reuse terms\. The authors do not redistribute the competition data\.

### Code and material availability

The Supplementary Information appended to this preprint contains the supporting ranking analysis, SEED\-IV implementation audit, tail\-risk method specification, full control table, decision table, and reproducibility boundary\. The ancillary fileanc/ESM\_2\.jsonprovides the exact CF\-TRE component order, selected mean\- and tail\-risk parameters, double\-precision mixture weights, and source\-file checksum in machine\-readable form\. Analysis code, protocol implementations, audit scripts, tests, and the machine\-readable final configurations are available at[https://github\.com/hantingsuo/eeg\-generalization\-research](https://github.com/hantingsuo/eeg-generalization-research)\. Raw SEED, SEED\-IV, and competition data, together with fine\-grained participant\-level predictions governed by the source\-data terms, are not redistributed\.

## References

- \\bibcommenthead
- Ambroise and McLachlan \[2002\]Ambroise C, McLachlan GJ \(2002\) Selection bias in gene extraction on the basis of microarray gene\-expression data\. Proceedings of the National Academy of Sciences 99\(10\):6562–6566\.[10\.1073/pnas\.102102699](https://arxiv.org/doi.org/10.1073/pnas.102102699)
- Bates et al\. \[2024\]Bates S, Hastie T, Tibshirani R \(2024\) Cross\-validation: What does it estimate and how well does it do it? Journal of the American Statistical Association 119\(546\):1434–1445\.[10\.1080/01621459\.2023\.2197686](https://arxiv.org/doi.org/10.1080/01621459.2023.2197686)
- Bengio and Grandvalet \[2004\]Bengio Y, Grandvalet Y \(2004\) No unbiased estimator of the variance of k\-fold cross\-validation\. Journal of Machine Learning Research 5:1089–1105\. URL[https://www\.jmlr\.org/papers/v5/grandvalet04a\.html](https://www.jmlr.org/papers/v5/grandvalet04a.html)
- Binette and Reiter \[2024\]Binette O, Reiter JP \(2024\) Improving the validity and practical usefulness of AI/ML evaluations using an estimands framework\. URL[https://arxiv\.org/abs/2406\.10366](https://arxiv.org/abs/2406.10366),[arXiv:2406\.10366](https://arxiv.org/abs/2406.10366)
- Bouthillier et al\. \[2021\]Bouthillier X, Delaunay P, Bronzi M, et al \(2021\) Accounting for variance in machine learning benchmarks\. In: Proceedings of Machine Learning and Systems, pp 747–769, URL[https://proceedings\.mlsys\.org/paper\_files/paper/2021/hash/0184b0cd3cfb185989f858a1d9f5c1eb\-Abstract\.html](https://proceedings.mlsys.org/paper_files/paper/2021/hash/0184b0cd3cfb185989f858a1d9f5c1eb-Abstract.html)
- Brookshire et al\. \[2024\]Brookshire G, Kasper J, Blauch NM, et al \(2024\) Data leakage in deep learning studies of translational EEG\. Frontiers in Neuroscience 18:1373515\.[10\.3389/fnins\.2024\.1373515](https://arxiv.org/doi.org/10.3389/fnins.2024.1373515)
- Cawley and Talbot \[2010\]Cawley GC, Talbot NLC \(2010\) On over\-fitting in model selection and subsequent selection bias in performance evaluation\. Journal of Machine Learning Research 11:2079–2107\. URL[https://jmlr\.org/papers/v11/cawley10a\.html](https://jmlr.org/papers/v11/cawley10a.html)
- Del Pup et al\. \[2025\]Del Pup F, Zanola A, Tshimanga LF, et al \(2025\) The role of data partitioning on the performance of EEG\-based deep learning models in supervised cross\-subject analysis: A preliminary study\. Computers in Biology and Medicine 196:110608\.[10\.1016/j\.compbiomed\.2025\.110608](https://arxiv.org/doi.org/10.1016/j.compbiomed.2025.110608)
- Duan et al\. \[2013\]Duan RN, Zhu JY, Lu BL \(2013\) Differential entropy feature for EEG\-based emotion classification\. In: 2013 6th International IEEE/EMBS Conference on Neural Engineering \(NER\), pp 81–84,[10\.1109/NER\.2013\.6695876](https://arxiv.org/doi.org/10.1109/NER.2013.6695876)
- Erat et al\. \[2024\]Erat K, Sahin EB, Dogan F, et al \(2024\) Emotion recognition with EEG\-based brain–computer interfaces: A systematic literature review\. Multimedia Tools and Applications 83:79647–79694\.[10\.1007/s11042\-024\-18259\-z](https://arxiv.org/doi.org/10.1007/s11042-024-18259-z)
- Ganin et al\. \[2016\]Ganin Y, Ustinova E, Ajakan H, et al \(2016\) Domain\-adversarial training of neural networks\. Journal of Machine Learning Research 17\(59\):1–35\. URL[https://jmlr\.org/papers/v17/15\-239\.html](https://jmlr.org/papers/v17/15-239.html)
- Gil and Hernández\-Sabaté \[2026\]Gil D, Hernández\-Sabaté A \(2026\) What is the across\-subject performance of your artificial intelligence method for EEG analysis? Biomedical Signal Processing and Control 120:110020\.[10\.1016/j\.bspc\.2026\.110020](https://arxiv.org/doi.org/10.1016/j.bspc.2026.110020)
- Guan et al\. \[2025\]Guan Z, Zhang X, Huang W, et al \(2025\) A method for detecting depression in adolescence based on an affective brain–computer interface and resting\-state EEG signals\. Neuroscience Bulletin 41\(3\):434–448\.[10\.1007/s12264\-024\-01319\-7](https://arxiv.org/doi.org/10.1007/s12264-024-01319-7), published online 20 November 2024
- Huang et al\. \[2023\]Huang W, Wu W, Lucas MV, et al \(2023\) Neurofeedback training with an electroencephalogram\-based brain–computer interface enhances emotion regulation\. IEEE Transactions on Affective Computing 14\(2\):998–1011\.[10\.1109/TAFFC\.2021\.3134183](https://arxiv.org/doi.org/10.1109/TAFFC.2021.3134183)
- Kapoor and Narayanan \[2023\]Kapoor S, Narayanan A \(2023\) Leakage and the reproducibility crisis in machine\-learning\-based science\. Patterns 4:100804\.[10\.1016/j\.patter\.2023\.100804](https://arxiv.org/doi.org/10.1016/j.patter.2023.100804)
- Kukhilava et al\. \[2025\]Kukhilava N, Tsmindashvili T, Kalandadze R, et al \(2025\) Evaluation in EEG emotion recognition: State\-of\-the\-art review and unified framework\. URL[https://arxiv\.org/abs/2505\.18175](https://arxiv.org/abs/2505.18175),[arXiv:2505\.18175](https://arxiv.org/abs/2505.18175)
- Lei et al\. \[2025\]Lei P, Wu M, Yi W, et al \(2025\) Impact of trial\-wise and test data leakage on EEG\-based emotion classification\. In: Proceedings of the 1st Workshop on 4D Micro\-Expression Recognition \(4DMR 2025\), pp 69–77, URL[https://ceur\-ws\.org/Vol\-4115/paper7\.pdf](https://ceur-ws.org/Vol-4115/paper7.pdf)
- Li et al\. \[2021\]Li J, Li S, Pan J, et al \(2021\) Cross\-subject EEG emotion recognition with self\-organized graph neural network\. Frontiers in Neuroscience 15:611653\.[10\.3389/fnins\.2021\.611653](https://arxiv.org/doi.org/10.3389/fnins.2021.611653)
- Li et al\. \[2023\]Li J, Pan W, Huang H, et al \(2023\) STGATE: Spatial\-temporal graph attention network with a transformer encoder for EEG\-based emotion recognition\. Frontiers in Human Neuroscience 17:1169949\.[10\.3389/fnhum\.2023\.1169949](https://arxiv.org/doi.org/10.3389/fnhum.2023.1169949)
- Li et al\. \[2026\]Li Z, Wu X, Hao Y, et al \(2026\) Cross\-subject generalization for EEG emotion recognition: A review of methods, challenges, and future trends\. Frontiers in Computational Neuroscience 20:1865513\.[10\.3389/fncom\.2026\.1865513](https://arxiv.org/doi.org/10.3389/fncom.2026.1865513)
- Liu et al\. \[2025\]Liu H, Yang S, Zhang Y, et al \(2025\) LibEER: A comprehensive benchmark and algorithm library for EEG\-based emotion recognition\. IEEE Transactions on Affective Computing 16\(4\):3596–3613\.[10\.1109/TAFFC\.2025\.3605833](https://arxiv.org/doi.org/10.1109/TAFFC.2025.3605833)
- Lundberg et al\. \[2021\]Lundberg I, Johnson R, Stewart BM \(2021\) What is your estimand? defining the target quantity connects statistical evidence to theory\. American Sociological Review 86\(3\):532–565\.[10\.1177/00031224211004187](https://arxiv.org/doi.org/10.1177/00031224211004187)
- Margaret et al\. \[2026\]Margaret MJ, Banu NMM, Madhumithaa S, et al \(2026\) On the prediction models for brain signal\-based emotion recognition\. Scientific Reports 16:18646\.[10\.1038/s41598\-026\-47622\-3](https://arxiv.org/doi.org/10.1038/s41598-026-47622-3)
- Pineau et al\. \[2021\]Pineau J, Vincent\-Lamarre P, Sinha K, et al \(2021\) Improving reproducibility in machine learning research: A report from the NeurIPS 2019 reproducibility program\. Journal of Machine Learning Research 22\(164\):1–20\. URL[https://jmlr\.org/papers/v22/20\-303\.html](https://jmlr.org/papers/v22/20-303.html)
- Roberts et al\. \[2017\]Roberts DR, Bahn V, Ciuti S, et al \(2017\) Cross\-validation strategies for data with temporal, spatial, hierarchical, or phylogenetic structure\. Ecography 40\(8\):913–929\.[10\.1111/ecog\.02881](https://arxiv.org/doi.org/10.1111/ecog.02881)
- Sagawa et al\. \[2020\]Sagawa S, Koh PW, Hashimoto TB, et al \(2020\) Distributionally robust neural networks for group shifts: On the importance of regularization for worst\-case generalization\. In: International Conference on Learning Representations, URL[https://openreview\.net/forum?id=ryxGuJrFvS](https://openreview.net/forum?id=ryxGuJrFvS)
- Song et al\. \[2020\]Song T, Zheng W, Song P, et al \(2020\) EEG emotion recognition using dynamical graph convolutional neural networks\. IEEE Transactions on Affective Computing 11\(3\):532–541\.[10\.1109/TAFFC\.2018\.2817622](https://arxiv.org/doi.org/10.1109/TAFFC.2018.2817622)
- Summers and Dinneen \[2021\]Summers C, Dinneen MJ \(2021\) Nondeterminism and instability in neural network optimization\. In: Proceedings of the 38th International Conference on Machine Learning, pp 9913–9922, URL[https://proceedings\.mlr\.press/v139/summers21a\.html](https://proceedings.mlr.press/v139/summers21a.html)
- Sun and Saenko \[2016\]Sun B, Saenko K \(2016\) Deep CORAL: Correlation alignment for deep domain adaptation\. In: Computer Vision – ECCV 2016 Workshops, pp 443–450,[10\.1007/978\-3\-319\-49409\-8\_35](https://arxiv.org/doi.org/10.1007/978-3-319-49409-8_35)
- Varma and Simon \[2006\]Varma S, Simon R \(2006\) Bias in error estimation when using cross\-validation for model selection\. BMC Bioinformatics 7:91\.[10\.1186/1471\-2105\-7\-91](https://arxiv.org/doi.org/10.1186/1471-2105-7-91)
- Varoquaux \[2018\]Varoquaux G \(2018\) Cross\-validation failure: Small sample sizes lead to large error bars\. NeuroImage 180:68–77\.[10\.1016/j\.neuroimage\.2017\.06\.061](https://arxiv.org/doi.org/10.1016/j.neuroimage.2017.06.061)
- Zheng and Lu \[2015\]Zheng WL, Lu BL \(2015\) Investigating critical frequency bands and channels for EEG\-based emotion recognition with deep neural networks\. IEEE Transactions on Autonomous Mental Development 7\(3\):162–175\.[10\.1109/TAMD\.2015\.2431497](https://arxiv.org/doi.org/10.1109/TAMD.2015.2431497)
- Zheng et al\. \[2019\]Zheng WL, Liu W, Lu Y, et al \(2019\) EmotionMeter: A multimodal framework for recognizing human emotions\. IEEE Transactions on Cybernetics 49\(3\):1110–1122\.[10\.1109/TCYB\.2018\.2797176](https://arxiv.org/doi.org/10.1109/TCYB.2018.2797176)