在留一受试者评估下的高效神经架构搜索

# Efficient Architecture Search under Leave-One-Subject-Out Evaluation
Source: [https://arxiv.org/html/2609.21457](https://arxiv.org/html/2609.21457)
Heinke Hihn1and Friedhelm Schwenker2 1IT and Engineering Department, IU International University of Applied Sciences, Berlin, Germany 2Insitute for Neural Information Procressing, Ulm University, Ulm, Germany heinke\.hihn@iu\.org, friedhelm\.schwenker@uni\-ulm\.de††thanks:˜https://orcid\.org/0000\-0002\-3244\-3661††thanks:˜https://orcid\.org/0000\-0001\-5118\-0812

###### Abstract

Deep neural architectures are widely used for signal processing in automated pain assessment systems\. However, architecture design has remained largely a manual task despite the potential efficiency benefits of Neural Architecture Search \(NAS\)\. Embedding NAS in a Leave\-One\-Subject\-Out \(LOSO\) evaluation is computationally demanding because a fully nested implementation requiresNNindependent architecture searches and, assuming approximately linear training cost, scales as𝒪⁡\(N2\)\\mathcal\{O\}\(N^\{2\}\)\. We propose a block\-based, leakage\-controlled approach that shares NAS runs between subjects, reducing the number of searches fromNNtoBB, whereB≪NB\\ll N, dubbed PainNAS\. On the BioVid Heat Pain dataset, PainNAS yields comparable subject\-level accuracy with substantially fewer parameters and FLOPs\.

## 1INTRODUCTION

Automated pain assessment aims to build systems that classify the pain level a subject experiences\[[13](https://arxiv.org/html/2609.21457#bib.bib19)\]based on recordings of bodily reactions to a painful stimulus, such as EMG, EDA, and ECG data or video\[[13](https://arxiv.org/html/2609.21457#bib.bib19)\]\. Recently, research has shifted from hand\-crafted features to deep and hybrid approaches\[[13](https://arxiv.org/html/2609.21457#bib.bib19)\]\. However, architecture design has remained a manual task despite the potential benefits of Neural Architecture Search \(NAS\)\[[19](https://arxiv.org/html/2609.21457#bib.bib3)\]\. We argue that in part this is because of the search complexity introduced by the underlying evaluation strategy Leave\-One\-Subject\-Out \(LOSO\) Cross Validation\. LOSO is a commonly used subject\-independent evaluation protocol because nested cross\-validation separates model selection from performance evaluation and thus reduces the optimistic bias that occurs when both are conducted using the same resampling results\[[3](https://arxiv.org/html/2609.21457#bib.bib1)\]\. As such, it is widely used in setting where the model evalaution must deal with subject\-specific data, such as in Affective Computing\[[10](https://arxiv.org/html/2609.21457#bib.bib24),[9](https://arxiv.org/html/2609.21457#bib.bib25),[21](https://arxiv.org/html/2609.21457#bib.bib8),[15](https://arxiv.org/html/2609.21457#bib.bib10),[26](https://arxiv.org/html/2609.21457#bib.bib16)\]\. In a leakage\-controlled LOSO design for a dataset ofNNsubjects, a fully nested implementation would requireNNindependent architecture searches followed by anNN\-fold LOSO evaluation, thus growing in the order𝒪⁡\(N2\)\\mathcal\{O\}\(N^\{2\}\)\. Therefore, an optimisation technique in this setting must support efficient LOSO evaluation\.

\(a\) Global NASsearch on all subjects⋯\\cdotsjj⋯\\cdots𝒮\\mathcal\{S\}NASjjseenLOSO evaluationa⋆a^\{\\star\}1 NAS searchoptimistically biased\(b\) Nested LOSO NASone search per target subjecttraintest⋯\\cdotsNAS11⋯\\cdotsNAS22⋮\\vdots⋯\\cdotsNASNNN=87N=87NAS searchesleakage\-free\(c\) PainNASB=5B=5outer blocks𝒢1\\mathcal\{G\}\_\{1\}𝒢2\\mathcal\{G\}\_\{2\}𝒢b\\mathcal\{G\}\_\{b\}𝒢4\\mathcal\{G\}\_\{4\}𝒢5\\mathcal\{G\}\_\{5\}excluded𝒯b=𝒮∖𝒢b\\mathcal\{T\}\_\{b\}=\\mathcal\{S\}\\setminus\\mathcal\{G\}\_\{b\}innerK=3K=3foldsk=1k=1k=2k=2k=3k=3NASb×5\\times 5B=5B=5NAS searchesleakage\-free

Figure 1:\(a\)A single global search uses all subjects\.\(b\)Fully nested LOSO NAS excludes every target subject from its corresponding search\.\(c\)The proposed approach excludes blocks\-wise, evaluates candidates using subject\-disjoint inner folds, and shares architectures\.Deep architectures in automated pain assessment can be categorised in early fusion \(EF\) and late fusion \(LF\)\. Early fusion approaches are comparatively under\-represented \(e\.g\., the review by\[[13](https://arxiv.org/html/2609.21457#bib.bib19)\]identifies a single early\-fusion method\), and several published comparisons report stronger late\- or intermediate\-fusion results\[[21](https://arxiv.org/html/2609.21457#bib.bib8)\]\. Consequently, studies have often used modality\-specific encoders with adaptive weighting, gating, or attention\[[13](https://arxiv.org/html/2609.21457#bib.bib19)\]\. One possible reason for this is that raw concatenation requires heterogeneous physiological signals to share a common representation which is rarely the case, whereas late\-fusion models use separate processing paths before combining representations or decisions into a common learned feature space\. However, it remains unclear whether their advantage arises from the fusion principle itself or from their more specialised and extensively engineered modality\-specific encoders\. A suitably selected shared architecture may compensate for the heterogeneous temporal characteristics of the physiological modalities while retaining the efficiency advantages of early fusion\.

Based on this, we formulate the following research questions:

1. RQ1:To what extent can leakage\-free neural architecture search improve subject\-independent pain recognition compared to a manually designed architecture?
2. RQ2:What trade\-off does PainNAS provide between search cost and LOSO generalisation performance?, and
3. RQ3:Which architectural characteristics are consistently selected across subject blocks and classification settings?

To answer these questions, we introduce PainNAS: a cross\-fitted block NAS procedure that excludes every evaluated subject from the architecture search used for that subject, while reducing the number of NAS runs, thus enabling an efficient and leakage\-free NAS within a LOSO\-CV\.

## 2RELATED WORK

We divide previous work into two categories:\(i\)NAS on physiological signals, and\(ii\)efficient NAS methods\. To the best of our knowledge, there is no prior work in the literature introducing a design for subject\-wise outer evaluation which finds a leakage\-free architecture\.

### 2\.1NAS for Physiological Signal Processing

NAS has been applied to physiological signals and multimodal fusion\[[29](https://arxiv.org/html/2609.21457#bib.bib17),[5](https://arxiv.org/html/2609.21457#bib.bib18)\], but these methods do not address the repeated architecture searches required by subject\-wise outer evaluation\. In contrast, our approach reduces the number of NAS runs by allowing subjects in the same outer block to share an architecture search from which the entire block is excluded\.

### 2\.2Efficient NAS

NAS can be efficiently implemented in several ways, the most common include sharing weights among candidate architectures\[[18](https://arxiv.org/html/2609.21457#bib.bib2)\], by using zero\-cost proxies to efficiently score architectures\[[1](https://arxiv.org/html/2609.21457#bib.bib15)\], and by guiding the search to promising avenues using, e\.g\., generative models\[[17](https://arxiv.org/html/2609.21457#bib.bib14)\]\. Such approaches would reduce the complexity of the LOSO\-looped NAS but cannot find a leakage\-free architecture by design\.

## 3METHOD

Algorithm 1PainNAS: cross\-fitted block NAS with final LOSO evaluationInput:Subjects

𝒮\\mathcal\{S\}, number of outer blocks

BB, number of inner folds

KK
Output:Mean LOSO accuracy

Split

𝒮\\mathcal\{S\}into disjoint outer blocks

𝒢1,…,𝒢B\\mathcal\{G\}\_\{1\},\\ldots,\\mathcal\{G\}\_\{B\};

foreach*outer block𝒢b\\mathcal\{G\}\_\{b\}*do

Exclude

𝒢b\\mathcal\{G\}\_\{b\}and split

𝒮∖𝒢b\\mathcal\{S\}\\setminus\\mathcal\{G\}\_\{b\}into inner folds

F1,…,FKF\_\{1\},\\ldots,F\_\{K\};

foreach*candidate architecturea∈𝒜a\\in\\mathcal\{A\}*do

foreach*inner foldFiF\_\{i\}*do

Fit

aausing

⋃ℓ≠iFℓ\\bigcup\_\{\\ell\\neq i\}F\_\{\\ell\};

Evaluate

aaon

FiF\_\{i\};

end foreach

Compute the objective

Jb​\(a\)J\_\{b\}\(a\)from the inner\-fold results;

end foreach

ab⋆←arg⁡maxa∈𝒜​Jb​\(a\)a\_\{b\}^\{\\star\}\\leftarrow\\arg\\max\_\{a\\in\\mathcal\{A\}\}J\_\{b\}\(a\);

foreach*LOSO target subjectj∈𝒢bj\\in\\mathcal\{G\}\_\{b\}*do

Fit

ab⋆a\_\{b\}^\{\\star\}on

𝒮∖\{j\}\\mathcal\{S\}\\setminus\\\{j\\\};

Evaluate once on the held\-out subject

jj;

end foreach

end foreach

returnmean LOSO accuracy;

This section introduces our method PainNAS as a LOSO evaluation and its integration with a computationally feasible, leakage\-free NAS protocol\.

Table 1:Results on the BioVid dataset in the binary and MC setting\. Standard deviations are shown in parentheses where available\. Bold indicates the best reported value\.Formally, the accuracy of Leave\-One\-Subject\-Out cross\-validation is

AccLOSO=1N​∑j=1NAcc⁡\(D−j,Dj\),\\operatorname\{Acc\}\_\{\\mathrm\{LOSO\}\}=\\frac\{1\}\{N\}\\sum\_\{j=1\}^\{N\}\\operatorname\{Acc\}\\\!\\left\(D\_\{\-j\},D\_\{j\}\\right\),\(1\)whereAcc⁡\(D−j,Dj\)\\operatorname\{Acc\}\\\!\\left\(D\_\{\-j\},D\_\{j\}\\right\)denotes the classification accuracy of a model trained in datasetD−jD\_\{\-j\}, which excludes subjectjj, and evaluated on the data of that subjectDjD\_\{j\}\. GivenNNsubjects, NAS and final model fitting are performed independently for each of theNNfolds using the remainingN−1N\-1subjects\. To avoid data leakage, no samples of the held\-out subject are used during optimisation\. Thus, the total runtime is proportional to

TLOSO​\-​NAS=N\[TNAS\(N−1\)\+Tfit\(N−1\)\+Ttest\(1\)\]∈𝒪⁡\(N2\),\\displaystyle\\begin\{split\}T\_\{\\mathrm\{LOSO\\text\{\-\}NAS\}\}&=N\[T\_\{\\mathrm\{NAS\}\}\(N\-1\)\\\\ &\+T\_\{\\mathrm\{fit\}\}\(N\-1\)\+T\_\{\\mathrm\{test\}\}\(1\)\]\\\\ &\\in\\mathcal\{O\}\(N^\{2\}\),\\end\{split\}\(2\)
assumingTNAS​\(N\),Tfit​\(N\)∈𝒪⁡\(N\)T\_\{\\mathrm\{NAS\}\}\(N\),T\_\{\\mathrm\{fit\}\}\(N\)\\in\\mathcal\{O\}\(N\)\. To reduce the number of NAS runs, we divide the subjects𝒮\\mathcal\{S\}into disjointouterblocks𝒢1,…,𝒢B\\mathcal\{G\}\_\{1\},\\ldots,\\mathcal\{G\}\_\{B\}\. For each outer blockbb, NAS uses only the training set𝒯b=𝒮∖𝒢b\\mathcal\{T\}\_\{b\}=\\mathcal\{S\}\\setminus\\mathcal\{G\}\_\{b\}\. To evaluate a proxy for LOSO generalisation, we further split𝒯b\\mathcal\{T\}\_\{b\}intoKKsubject\-disjoint inner folds: For each candidate architectureaaand inner foldkk, a model is initialised independently, fitted on the otherK−1K\-1folds, and validated on foldkk\. While we can reduce the number of NAS runs toBB, the overall complexity still grows approximately quadratic in the number of subjectsNNas we still need to doNNmodel fits andNNevaluations,

TPainNAS=B​TNAS​\(N−NB\)\+N⁡\[Tfit​\(N−1\)\+Ttest​\(1\)\]∈𝒪⁡\(N2\)\.\\displaystyle\\begin\{split\}T\_\{\\mathrm\{PainNAS\}\}&=B\\,T\_\{\\mathrm\{NAS\}\}\\\!\\left\(N\-\\frac\{N\}\{B\}\\right\)\\\\ &\+N\\left\[\\,T\_\{\\mathrm\{fit\}\}\(N\-1\)\+T\_\{\\mathrm\{test\}\}\(1\)\\right\]\\in\\mathcal\{O\}\(N^\{2\}\)\.\\end\{split\}\(3\)The selection objective used the rank architectures during NAS is given as

A¯b​\(a\)\\displaystyle\\bar\{A\}\_\{b\}\(a\)=1Nb​∑i∈𝒯bAi​\(a\),\\displaystyle=\\frac\{1\}\{N\_\{b\}\}\\sum\_\{i\\in\\mathcal\{T\}\_\{b\}\}A\_\{i\}\(a\),\(4\)σ^b​\(a\)\\displaystyle\\widehat\{\\sigma\}\_\{b\}\(a\)=1Nb−1​∑i∈𝒯b\(Ai​\(a\)−A¯b​\(a\)\)2,\\displaystyle=\\sqrt\{\\frac\{1\}\{N\_\{b\}\-1\}\\sum\_\{i\\in\\mathcal\{T\}\_\{b\}\}\\left\(A\_\{i\}\(a\)\-\\bar\{A\}\_\{b\}\(a\)\\right\)^\{2\}\},\(5\)Jb​\(a\)\\displaystyle J\_\{b\}\(a\)=A¯b​\(a\)−β​σ^b​\(a\)Nb,\\displaystyle=\\bar\{A\}\_\{b\}\(a\)\-\\beta\\frac\{\\widehat\{\\sigma\}\_\{b\}\(a\)\}\{\\sqrt\{N\_\{b\}\}\},\(6\)whereA¯b​\(a\)\\bar\{A\}\_\{b\}\(a\)is the mean subject\-level validation accuracy of architectureaaover𝒯b\\mathcal\{T\}\_\{b\},Nb=\|𝒯b\|N\_\{b\}=\|\\mathcal\{T\}\_\{b\}\|is the number of subjects in𝒯b\\mathcal\{T\}\_\{b\},σ^b​\(a\)\\widehat\{\\sigma\}\_\{b\}\(a\)is the sample standard deviation of the subject\-level accuracies over𝒯b\\mathcal\{T\}\_\{b\}, andJb​\(a\)J\_\{b\}\(a\)defines a selection score\. It rewards mean subject\-level accuracy and penalises estimates that exhibit high inter\-subject variance controlled by a factorβ\\beta\. We usedβ=1\.0\\beta=1\.0\. Algorithm[1](https://arxiv.org/html/2609.21457#alg1)gives a high\-level overview\.

After the inner folds, we retain the checkpoint with the highest validation score of the architecture found by optimising Eq\.[6](https://arxiv.org/html/2609.21457#S3.E6)\. The model is then trained for additionalOPENM\)M\)epochs using the samples of all 86 non\-target subjects, including the former inner\-validation subjects and the other members of𝒢b\\mathcal\{G\}\_\{b\}\. We setMMas the median number of epochs used in the NAS search before early\-stopping\. Finally, it is evaluated once on the held\-out dataDjD\_\{j\}\. The final reported metric is the mean accuracy across all target subjects\.

## 4RESULTS

Table 2:Results of blocked NAS over early fusion integrated with LOSO evaluation for different numbers of outer blocks and inner folds\.Outer Blocks with K = 3BinaryMulti\-ClassBBAccuracyF1AccuracyF1082\.7% \(±15\.6\\pm 15\.6\)81\.3% \(±14\.0\\pm 14\.0\)35\.6% \(±9\.3\\pm 9\.3\)31\.0% \(±9\.3\\pm 9\.3\)583\.3% \(±14\.4\\pm 14\.4\)82\.3% \(±16\.4\\pm 16\.4\)35\.0% \(±9\.5\\pm 9\.5\)31\.2% \(±9\.8\\pm 9\.8\)1083\.8% \(±14\.5\\pm 14\.5\)82\.6% \(±16\.8\\pm 16\.8\)35\.8% \(±9\.6\\pm 9\.6\)31\.2% \(±10\.0\\pm 10\.0\)2083\.7% \(±14\.5\\pm 14\.5\)82\.6% \(±16\.7\\pm 16\.7\)35\.6% \(±9\.1\\pm 9\.1\)31\.0% \(±8\.8\\pm 8\.8\)8782\.3% \(±15\.5\\pm 15\.5\)81\.2% \(±17\.5\\pm 17\.5\)34\.4% \(±9\.4\\pm 9\.4\)30\.4% \(±9\.1\\pm 9\.1\)Inner Folds withBB= 5BinaryMulti\-ClassKKAccuracyF1AccuracyF1383\.3% \(±14\.4\\pm 14\.4\)82\.3% \(±16\.4\\pm 16\.4\)35\.0% \(±9\.5\\pm 9\.5\)31\.2% \(±9\.8\\pm 9\.8\)582\.3% \(±15\.4\\pm 15\.4\)82\.6% \(±17\.3\\pm 17\.3\)35\.6% \(±8\.8\\pm 8\.8\)30\.8% \(±9\.0\\pm 9\.0\)1082\.7% \(±15\.1\\pm 15\.1\)81\.4% \(±12\.5\\pm 12\.5\)34\.5% \(±9\.3\\pm 9\.3\)30\.0% \(±9\.4\\pm 9\.4\)

Figure 2:\(A\) Mean LOSO accuracy versus median inference compute for PainNAS, global\- NAS–then–LOSO runs, and fixed early\-/late\-fusion baselines \(B\) Distribution of held\-out\-subject accuracy for each setting\. \(C\) Range and median of the compute of architectures selected within each setting\.\(i,j\)\(i,j\)indicates PainNAS withiiouter blocks andjjinner folds\.In the following we present and discuss the results of the experiments we have designed to answer the research questions posed in Section[1](https://arxiv.org/html/2609.21457#S1)\.

### 4\.1Classification Results

To answer RQ1, we evaluated the BioVid heat pain dataset\[[25](https://arxiv.org/html/2609.21457#bib.bib4)\]and used the architecture proposed by\[[21](https://arxiv.org/html/2609.21457#bib.bib8)\]as a baseline\. To this end, Table[1](https://arxiv.org/html/2609.21457#S3.T1)compares with previous results in BioVid\. Within our early\-fusion protocol, PainNAS has a\+1\+1pp higher mean binary accuracy than the manually designed baseline, while the selected architectures contain fewer parameters\. In the multi\-class \(MC\) setting, our early fusion architecture outperforms previous methods by a margin of2\.82\.8pp\. In the LF setting, PainNAS yields results comparable to previous methods while drastically reducing the number of parameters \(see Section[4\.2](https://arxiv.org/html/2609.21457#S4.SS2)\)\.

### 4\.2Computational Resources and Trade\-Offs

RQ2 investigates how PainNAS compares with a fully informed \(but biased\) NAS followed by LOSO evaluation\.

Investigating the effect of a larger block count is not straightforward\. With five outer blocks and three inner folds, 17–18 subjects are excluded from search, with 10 and 20 blocks this becomes 8–9 and 4–5, respectively\. As the number of blocks increases, architecture selection uses more subjects and the transferred warm\-start checkpoint has seen more subjects\. Table[2](https://arxiv.org/html/2609.21457#S4.T2)reports the results for different block counts under binary and MC settings, indicating PainNAS yields stable results and varyingBBandKKsettings\.

Figure[2](https://arxiv.org/html/2609.21457#S4.F2)summarises the accuracy–compute trade\-off across PainNAS settings\. Binary classification consistently achieved high LOSO accuracy of approximately 80–85%, and MC performance was roughly 30–40%\. The global\-NAS–then–LOSO results should be interpreted as exploratory because their architecture search used cohort\-wide information, but nevertheless, they show a similar task\-dependent pattern\.

Using an analytical estimate of two FLOPs per convolutional or dense operation, the manually designed binary early\-fusion baseline requires134\.42​MFLOPs134\.42\\,\\mathrm\{MFLOPs\}and achieves82\.79%82\.79\\%accuracy\. Across the five PainNAS configurations, the median selected\-architecture cost decreases to7\.197\.19–24\.96​MFLOPs24\.96\\,\\mathrm\{MFLOPs\}, corresponding to5\.4×5\.4\\times–18\.7×18\.7\\timesfewer FLOPs, while subject\-averaged accuracy remains between82\.73%82\.73\\%and83\.71%83\.71\\%\. TheB=20,K=3B=20,K=3configuration has the highest observed point estimate,83\.71%83\.71\\%accuracy at8\.80​MFLOPs8\.80\\,\\mathrm\{MFLOPs\}, a15\.3×15\.3\\timesreduction relative to the baseline\. MC accuracy is likewise stable at35\.03%35\.03\\%–35\.78%35\.78\\%over median costs of13\.8513\.85–20\.41​MFLOPs20\.41\\,\\mathrm\{MFLOPs\}\.

The active run\-time on a 24 GB GPU for the fully nested binary NAS was 72\.81h, while PainNAS with five outer blocks and three inner folds took 8\.69h under the exact same settings\. In the fully nested MC run, it tool 157h compared to 19\.3 using five blocks – see Figure[3](https://arxiv.org/html/2609.21457#S4.F3)for more details\. The fully nested run is roughly 8 times slower in both task setting but the results are comparable: 83\.71% and 35\.84% for PainNAS and 82\.34% and 35\.64% accuracy for the fully nested NAS\. No statistically significant difference was observed\.

We also investigated the effect of the objective Eq\.[6](https://arxiv.org/html/2609.21457#S3.E6)\. Compared with early fusion baselines, the full variance\-penalised runs \(β=1\.0\\beta=1\.0\) selected substantially more efficient architectures\. For binary classification, median inference cost decreased from114\.21114\.21to21\.7021\.70MFLOPs \(an81\.0%81\.0\\%reduction\)\. For MC classification, the median decreased from28\.7428\.74to13\.8513\.85MFLOPs \(a51\.8%51\.8\\%reduction\)\. This pattern did not extend to late fusion \(LF\): the binary median increased from85\.1785\.17to123\.97123\.97MFLOPs \(a45\.5%45\.5\\%increase\), and the MC median increased from40\.7140\.71to93\.1393\.13MFLOPs \(a128\.7%128\.7\\%increase\)\. Thus, in these runs, the variance\-penalised objective’s compute benefit was specific to early fusion rather than a fusion\-architecture\-independent effect – see Table[1](https://arxiv.org/html/2609.21457#S3.T1)and Figure[2](https://arxiv.org/html/2609.21457#S4.F2)for results and Figure[4](https://arxiv.org/html/2609.21457#S4.F4)for the effect of the objective on architecture selection\.

Thus, our results motivate PainNAS as a resource\-aware approach to multichannel biomedical signal processing\.

Figure 3:Runtime on a 24 GB L4 GPU recorded for different outer block sizesBBdemonstrating the benefits of PainNAS\.B=87B=87represents a fully LOSO\-nested NAS\.
### 4\.3Architectural Characteristics

To answer RQ3, we examined the architectures selected by NAS across the five outer\-block winners\. Among the ten early fusion winners, separable convolutions were selected in9/109/10cases, group normalisation in7/107/10, average pooling in7/107/10, and pooling size four in9/109/10, while batch\-dependent normalisation was selected only once\. MC selected global\-average heads more often than binary models \(3/53/5versus2/52/5\), but selected four\-block architectures in only3/53/5cases \(the remaining used three blocks\), whereas all binary winners used four or five blocks\. Temporal kernels of1515were selected equally often in both settings \(2/52/5\), and width multiplier2\.02\.0was selected less often for MC than binary models \(2/52/5vs\.3/53/5\)\.

In LF, the EDA branch used a flatten head in all ten selected architectures\. Binary models selected global\-average heads more frequently for ECG \(4/54/5vs\.2/52/5for MC\) and EMG \(3/53/5vs\.2/52/5\)\. Conversely, MC late\-fusion branches more often had six or seven blocks \(13/1513/15branches, compared with10/1510/15for binary\) and used flatten heads more frequently overall \(11/1511/15vs\.8/158/15\)\. The LF reference baseline\[[23](https://arxiv.org/html/2609.21457#bib.bib28)\]used seven\-block flatten branches for all modalities\.

![Refer to caption](https://arxiv.org/html/2609.21457v1/painnas_uncertainty_objective.png)Figure 4:The relationship between the standard error used in the selection objective Eq\.[6](https://arxiv.org/html/2609.21457#S3.E6), the MFLOPs, and the mean accuracy of the selected architectures indicates that minimisingσ^b​\(a\)\\widehat\{\\sigma\}\_\{b\}\(a\)encourages finding efficient architectures\.The selections suggest that robust pain recognition may benefit from architectures that reduce temporal resolution early and avoid batch\-dependent normalisation, which can be unstable when training data are limited or distributed across subjects\. The MC preference for wider networks, longer kernels, and global\-average heads suggests that distinguishing several pain levels requires broader temporal context but benefits from a more compact final representation\.

### 4\.4Experimental Setup

We used Optuna\[[2](https://arxiv.org/html/2609.21457#bib.bib11)\]with a TPE sampler for architecture search varying the number of convolutional blocks \(3–5\), repetitions per block \(1–2\), width multiplier \(0\.50\.5,11, or22\), kernel size \(7, 11, or 15\), convolution type, normalisation and pooling types, classifier depth and width, and learning rate \(10−510^\{\-5\}–10−310^\{\-3\}, log scale\)\. In the LF setting, the architectures of the EMG and ECG branches were mirrored to reduce the search space\. Models were trained for at most 30 epochs using a batch size of 128 and early stopping\. Successive\-halving pruning removed weak trials after intermediate folds\. The final objective was given by Eq\. \([6](https://arxiv.org/html/2609.21457#S3.E6)\)\. The implementation is available athttps://github\.com/xxx/xxx\.

## 5DISCUSSION

We have proposed a method to implement a computationally feasible NAS in the LOSO setting that improved the baseline and substantially reduced the number of parameters, increasing model efficiency\. Our experimental results suggest that the method is stable under its hyper\-parameters and that the ranking objective by Eq\.[6](https://arxiv.org/html/2609.21457#S3.E6)helps finding efficient architectures\. One limitation is the high variability in the selected architectural components and their parameter counts\. Future work should investigate how this variability can be reduced, e\.g\., by combining PainNAS with meta\-learning\[[6](https://arxiv.org/html/2609.21457#bib.bib23)\]to select architectures that rapidly adapt to unseen subjects, followed by lightweight continual learning\[[7](https://arxiv.org/html/2609.21457#bib.bib27),[8](https://arxiv.org/html/2609.21457#bib.bib26)\]to handle subject\-specific physiological changes over time\. Another promising research avenue is the design of dedicated NAS algorithms for LOSO settings to capture architectural and modality interdependencies\.

## ACKNOWLEDGEMENTS

Generative AI was used to assist in writing of this manuscript \(OpenAI ChatGPT\) and in developing the code for the experiments \(OpenAI Codex\)\. All parts created by an AI have been critically examined and reviewed by the authors\.

## REFERENCES

- \[1\]\(2021\)Zero\-cost proxies for lightweight nas\.arXiv preprint arXiv:2101\.08134\.Cited by:[§2\.2](https://arxiv.org/html/2609.21457#S2.SS2.p1.1)\.
- \[2\]T\. Akiba, S\. Sano, T\. Yanase, T\. Ohta, and M\. Koyama\(2019\)Optuna: a next\-generation hyperparameter optimization framework\.InThe 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining,pp\. 2623–2631\.Cited by:[§4\.4](https://arxiv.org/html/2609.21457#S4.SS4.p1.1)\.
- \[3\]G\. C\. Cawley and N\. L\. Talbot\(2010\)On over\-fitting in model selection and subsequent selection bias in performance evaluation\.The Journal of Machine Learning Research11,pp\. 2079–2107\.Cited by:[§1](https://arxiv.org/html/2609.21457#S1.p1.1)\.
- \[4\]J\. Farmani, G\. Bargshady, S\. Gkikas, M\. Tsiknakis, and R\. F\. Rojas\(2025\)CrossMod\-transformer: multi\-modal pain detection through eda and ecg fusion\.Scientific Reports15\(1\),pp\. 29467\.External Links:[Document](https://dx.doi.org/10.1038/s41598-025-14238-y)Cited by:[Table 1](https://arxiv.org/html/2609.21457#S3.T1.1.1.1.1.1.1.3.1)\.
- \[5\]M\. I\. E\. Ghebriout, H\. Bouzidi, S\. Niar, and H\. Ouarnoughi\(2024\)Harmonic\-nas: hardware\-aware multimodal neural architecture search on resource\-constrained devices\.InAsian Conference on Machine Learning,pp\. 374–389\.Cited by:[§2\.1](https://arxiv.org/html/2609.21457#S2.SS1.p1.1)\.
- \[6\]H\. Hihn and D\. A\. Braun\(2020\)Specialization in hierarchical learning systems: a unified information\-theoretic approach for supervised, unsupervised and reinforcement learning\.Neural Processing Letters52\(3\),pp\. 2319–2352\.Cited by:[§5](https://arxiv.org/html/2609.21457#S5.p1.1)\.
- \[7\]H\. Hihn and D\. A\. Braun\(2023\)Hierarchically structured task\-agnostic continual learning\.Machine Learning112\(2\),pp\. 655–686\.Cited by:[§5](https://arxiv.org/html/2609.21457#S5.p1.1)\.
- \[8\]H\. Hihn and D\. A\. Braun\(2024\)Online continual learning through unsupervised mutual information maximization\.Neurocomputing578,pp\. 127422\.Cited by:[§5](https://arxiv.org/html/2609.21457#S5.p1.1)\.
- \[9\]H\. Hihn, S\. Meudt, and F\. Schwenker\(2016\)Inferring mental overload based on postural behavior and gestures\.InProceedings of the 2nd Workshop on Emotion Representations and Modelling for Companion Systems,ERM4CT ’16,New York, NY, USA\.External Links:ISBN 9781450345583,[Link](https://doi.org/10.1145/3009960.3009961),[Document](https://dx.doi.org/10.1145/3009960.3009961)Cited by:[§1](https://arxiv.org/html/2609.21457#S1.p1.1)\.
- \[10\]H\. Hihn, S\. Meudt, and F\. Schwenker\(2016\)On gestures and postural behavior as a modality in ensemble methods\.InArtificial Neural Networks in Pattern Recognition,F\. Schwenker, H\. M\. Abbas, N\. El Gayar, and E\. Trentin \(Eds\.\),Cham,pp\. 312–323\.External Links:ISBN 978\-3\-319\-46182\-3Cited by:[§1](https://arxiv.org/html/2609.21457#S1.p1.1)\.
- \[11\]M\. Jiang, R\. Rosio, S\. Salanterä, A\. M\. Rahmani, P\. Liljeberg, D\. S\. da Silva, V\. H\. C\. de Albuquerque, and W\. Wu\(2024\)Personalized and adaptive neural networks for pain detection from multi\-modal physiological features\.Expert Systems with Applications235,pp\. 121082\.Cited by:[Table 1](https://arxiv.org/html/2609.21457#S3.T1.1.1.1.1.1.1.7.1)\.
- \[12\]M\. Kächele, P\. Thiam, M\. Amirian, F\. Schwenker, and G\. Palm\(2016\)Methods for person\-centered continuous pain intensity assessment from bio\-physiological channels\.IEEE Journal of Selected Topics in Signal Processing10\(5\),pp\. 854–864\.Cited by:[Table 1](https://arxiv.org/html/2609.21457#S3.T1.1.1.1.1.1.1.19.1)\.
- \[13\]M\. U\. Khan, G\. Chetty, R\. Goecke, and R\. Fernandez\-Rojas\(2026\)A systematic review of multimodal signal fusion for acute pain assessment systems\.ACM Computing Surveys58\(2\),pp\. 48:1–48:35\.External Links:[Document](https://dx.doi.org/10.1145/3737281)Cited by:[§1](https://arxiv.org/html/2609.21457#S1.p1.1),[§1](https://arxiv.org/html/2609.21457#S1.p2.1)\.
- \[14\]J\. Li, J\. Luo, Y\. Wang, Y\. Jiang, X\. Chen, and Y\. Quan\(2024\)Automatic pain assessment based on physiological signals: application of multi\-scale networks and cross\-attention cross\-attention\.InProceedings of the 2024 13th International Conference on Bioinformatics and Biomedical Science,pp\. 113–122\.Cited by:[Table 1](https://arxiv.org/html/2609.21457#S3.T1.1.1.1.1.1.1.4.1)\.
- \[15\]W\. Liu, G\. Wang, Y\. He, H\. Bo, L\. Ma, and H\. Li\(2024\)A study of prototypical network techniques for cross\-subject eeg analysis\.InECAI 2024: 27th European Conference on Artificial Intelligence, 19–24 October 2024, Santiago de Compostela, Spain–Including 13th Conference on Prestigious Applications of Intelligent Systems \(PAIS 2024\),pp\. 2846–2853\.Cited by:[§1](https://arxiv.org/html/2609.21457#S1.p1.1)\.
- \[16\]Z\. Lu, B\. Ozek, and S\. Kamarthi\(2023\)Transformer encoder with multiscale deep learning for pain classification using physiological signals\.Frontiers in Physiology14,pp\. 1294577\.External Links:[Document](https://dx.doi.org/10.3389/fphys.2023.1294577)Cited by:[Table 1](https://arxiv.org/html/2609.21457#S3.T1.1.1.1.1.1.1.5.1)\.
- \[17\]J\. Lukasik, S\. Jung, and M\. Keuper\(2022\)Learning where to look–generative nas is surprisingly efficient\.InEuropean Conference on Computer Vision,pp\. 257–273\.Cited by:[§2\.2](https://arxiv.org/html/2609.21457#S2.SS2.p1.1)\.
- \[18\]H\. Pham, M\. Guan, B\. Zoph, Q\. Le, and J\. Dean\(2018\)Efficient neural architecture search via parameters sharing\.InInternational conference on machine learning,pp\. 4095–4104\.Cited by:[§2\.2](https://arxiv.org/html/2609.21457#S2.SS2.p1.1)\.
- \[19\]P\. Ren, Y\. Xiao, X\. Chang, P\. Huang, Z\. Li, X\. Chen, and X\. Wang\(2021\)A comprehensive survey of neural architecture search: challenges and solutions\.ACM Computing Surveys \(CSUR\)54\(4\),pp\. 1–34\.Cited by:[§1](https://arxiv.org/html/2609.21457#S1.p1.1)\.
- \[20\]N\.A\.K\. Steur and F\. Schwenker\(2025\)Multimodal pain recognition based on contrastive adversarial autoencoder pretraining\.Machine Learning and Knowledge Extraction7,pp\. 165\.External Links:[Document](https://dx.doi.org/10.3390/make7040165)Cited by:[Table 1](https://arxiv.org/html/2609.21457#S3.T1.1.1.1.1.1.1.10.1)\.
- \[21\]P\. Thiam, P\. Bellmann, H\. A\. Kestler, and F\. Schwenker\(2019\)Exploring deep physiological models for nociceptive pain recognition\.Sensors19\(20\),pp\. 4503\.Cited by:[§1](https://arxiv.org/html/2609.21457#S1.p1.1),[§1](https://arxiv.org/html/2609.21457#S1.p2.1),[Table 1](https://arxiv.org/html/2609.21457#S3.T1.1.1.1.1.1.1.16.1),[Table 1](https://arxiv.org/html/2609.21457#S3.T1.1.1.1.1.1.1.8.1),[§4\.1](https://arxiv.org/html/2609.21457#S4.SS1.p1.1)\.
- \[22\]P\. Thiam, H\. Hihn, D\. A\. Braun, H\. A\. Kestler, and F\. Schwenker\(2021\)Multi\-modal pain intensity assessment based on physiological signals: a deep learning perspective\.Frontiers in Physiology12,pp\. 720464\.Cited by:[Table 1](https://arxiv.org/html/2609.21457#S3.T1.1.1.1.1.1.1.9.1)\.
- \[23\]P\. Thiam, V\. Kessler, M\. Amirian, P\. Bellmann, G\. Layher, Y\. Zhang, M\. Velana, S\. Gruss, S\. Walter, H\. C\. Traue,et al\.\(2019\)Multi\-modal pain intensity recognition based on the senseemotion database\.IEEE Transactions on Affective Computing12\(3\),pp\. 743–760\.Cited by:[§4\.3](https://arxiv.org/html/2609.21457#S4.SS3.p2.1)\.
- \[24\]P\. Thiam, F\. Schwenker, and H\. A\. Kestler\(2025\)Dealing with class overlap through cluster\-based sample weighting\.Computers14\(11\),pp\. 457\.Cited by:[Table 1](https://arxiv.org/html/2609.21457#S3.T1.1.1.1.1.1.1.6.1)\.
- \[25\]S\. Walter, S\. Gruss, H\. Ehleiter, J\. Tan, H\. C\. Traue, S\. Crawcour, P\. Werner, A\. Al\-Hamadi, A\. O\. Andrade, and G\. M\. da Silva\(2013\)The biovid heat pain database: data for the advancement and systematic validation of an automated pain recognition system\.In2013 IEEE International Conference on Cybernetics \(CYBCO\),pp\. 128–131\.External Links:[Document](https://dx.doi.org/10.1109/CYBConf.2013.6617456)Cited by:[§4\.1](https://arxiv.org/html/2609.21457#S4.SS1.p1.1)\.
- \[26\]K\. Wang, Y\. Zhang, Y\. Zhang, F\. Zhang, J\. Shen, and B\. Hu\(2026\)MSFSNet: multi\-source few\-shot adaptation network for cross\-subject depression recognition from eeg signals\.IEEE Journal of Biomedical and Health Informatics\.Cited by:[§1](https://arxiv.org/html/2609.21457#S1.p1.1)\.
- \[27\]P\. Werner, A\. Al\-Hamadi, K\. Limbrecht\-Ecklundt, S\. Walter, S\. Gruss, and H\. C\. Traue\(2016\)Automatic pain assessment with facial activity descriptors\.IEEE Transactions on Affective Computing8\(3\),pp\. 286–299\.Cited by:[Table 1](https://arxiv.org/html/2609.21457#S3.T1.1.1.1.1.1.1.17.1)\.
- \[28\]P\. Werner, A\. Al\-Hamadi, R\. Niese, S\. Walter, S\. Gruss, and H\. C\. Traue\(2014\)Automatic pain recognition from video and biomedical signals\.In2014 22nd international conference on pattern recognition,pp\. 4582–4587\.Cited by:[Table 1](https://arxiv.org/html/2609.21457#S3.T1.1.1.1.1.1.1.18.1)\.
- \[29\]Y\. Wu, H\. Liu, D\. Zhang, Y\. Zhang, T\. Lou, and Q\. Zheng\(2023\)AutoEER: automatic eeg\-based emotion recognition with neural architecture search\.Journal of Neural Engineering20\(4\),pp\. 046029\.Cited by:[§2\.1](https://arxiv.org/html/2609.21457#S2.SS1.p1.1)\.