ATHENA: Knowledge-guided agentic neural architecture search for AutoFormer-based electronic health record modeling

# ATHENA: Knowledge-guided agentic neural architecture search for AutoFormer-based electronic health record modeling
Source: [https://arxiv.org/html/2608.21712](https://arxiv.org/html/2608.21712)
\[orcid=0009\-0004\-7039\-7303\]

\[orcid=0000\-0002\-8036\-2110\]

Deyi Lilideyi@ufl\.eduorganization=Department of Health Outcomes and Biomedical Informatics, College of Medicine, University of Florida, city=Gainesville, state=FL, country=USALingyao Liorganization=College of Information Science, University of Arizona, city=Tucson, state=AZ, country=USATiansheng Wangorganization=Department of Pharmaceutical Health Outcomes and Policy, College of Pharmacy, University of Houston, city=Houston, state=TX, country=USAorganization=Department of Epidemiology, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, city=Chapel Hill, state=NC, country=USAMuxuan Liangorganization=Department of Biostatistics, University of Texas MD Anderson Cancer Center, city=Houston, state=TX, country=USAMei Liumei\.liu@ufl\.edu

###### Abstract

Transformer\-based models are widely used for clinical prediction from electronic health records \(EHRs\), yet their architectures still require substantial manual tuning, and the optimal configuration may vary across tasks and hospitals\. Neural architecture search \(NAS\) automates architecture design, but conventional methods are computationally costly for Transformer\-based EHR models\. Recent large language model \(LLM\)\-guided NAS methods reduce manual search design but typically conduct each search independently, without reusing architecture knowledge across hospitals\. In this study, we propose ATHENA \(AgenticTransfer acrossHospitals forEHRNeuralArchitecture Search\), a knowledge\-guided agentic NAS framework for Transformer\-based EHR modeling\. ATHENA uses a weight\-sharing supernet that is pretrained once per hospital, allowing candidate architectures to be instantiated as inherited subnetworks and evaluated through fine\-tuning rather than independent pretraining\. It also incorporates a two\-layer cross\-hospital architecture prior\. The first layer retrieves high\-performing architecture examples from source sites based on task descriptors, while the second estimates the effects of architectural components using SHapley Additive exPlanations \(SHAP\)\-based meta\-regression\. These priors guide a multi\-agent LLM search together with validation feedback from the target hospital\. Across six clinical prediction tasks and two independent health systems, ATHENA matches or outperforms four NAS baselines in 9 of 12 hospital–task evaluations at a search budget of 30\. It also shows more consistent architecture selection across repeated searches\. ATHENA provides a practical approach for reducing manual architecture tuning in Transformer\-based EHR modeling\. Code is publicly available at[https://github\.com/GatorAIM/ATHENA](https://github.com/GatorAIM/ATHENA)\.

###### keywords

Electronic health records ,Clinical predictive modeling ,Neural architecture search ,Knowledge transfer ,Multi\-agent systems

††credit:Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Software, Visualization, Writing – original draft††credit:Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Writing – original draft††credit:Validation, Writing – original draft††credit:Validation, Writing – original draft††credit:Validation, Writing – original draft††credit:Conceptualization, Funding acquisition, Project administration, Resources, Supervision, Validation, Writing – original draft††corresponding:Corresponding author††corresponding:Corresponding author††These authors contributed equally to this work\.## 1Introduction

The increasing availability of large\-scale electronic health record \(EHR\) data has motivated the development of models that can learn predictive patterns directly from routinely collected clinical records\[[37](https://arxiv.org/html/2608.21712#bib.bib38)\]\. Deep learning has become a standard approach for these tasks, with Transformer\-based models increasingly adopted for their ability to capture dependencies among medical events across a patient’s longitudinal record\[[30](https://arxiv.org/html/2608.21712#bib.bib39)\]\. A common two\-stage paradigm consists of pretraining a Transformer backbone on large\-scale EHR data to learn generalizable clinical patterns, followed by fine\-tuning to adapt the model to task\-specific objectives\[[28](https://arxiv.org/html/2608.21712#bib.bib3),[43](https://arxiv.org/html/2608.21712#bib.bib8),[27](https://arxiv.org/html/2608.21712#bib.bib12),[19](https://arxiv.org/html/2608.21712#bib.bib16)\]\.

Despite these advances, Transformer architectures for EHR modeling are still largely determined through manual design, relying on prior experience and trial\-and\-error\. The optimal architecture can vary across prediction tasks and health systems, as differences in cohort composition, clinical workflows, and outcome characteristics may favor different architectural configurations\[[17](https://arxiv.org/html/2608.21712#bib.bib40)\]\. Consequently, a single hand\-designed architecture may not perform consistently well across tasks and sites, whereas manually redesigning the architecture for each new setting is computationally costly and fails to systematically leverage knowledge gained from previous settings\.

Neural architecture search \(NAS\) addresses this problem by treating architecture design as an optimization problem over a predefined search space\[[31](https://arxiv.org/html/2608.21712#bib.bib41),[8](https://arxiv.org/html/2608.21712#bib.bib45)\]\. Classical NAS approaches, including reinforcement learning\[[48](https://arxiv.org/html/2608.21712#bib.bib42)\], evolutionary algorithm \(EA\)\[[29](https://arxiv.org/html/2608.21712#bib.bib43)\], differentiable search\[[23](https://arxiv.org/html/2608.21712#bib.bib24)\], and Bayesian optimization\[[16](https://arxiv.org/html/2608.21712#bib.bib44)\], have been successfully applied to vision tasks\. However, evaluating each candidate architecture can require substantial training\. This is particularly problematic for Transformer\-based EHR modeling, where pretraining on large longitudinal EHR datasets can dominate the computational cost of model development\[[28](https://arxiv.org/html/2608.21712#bib.bib3)\]\. Weight\-sharing NAS reduces this cost by training a single over\-parameterized supernet whose subnetworks share parameters, allowing candidate architectures to be instantiated as inherited subnetworks and evaluated through fine\-tuning rather than independently pretrained from scratch\[[12](https://arxiv.org/html/2608.21712#bib.bib22)\]\. This makes weight\-sharing NAS particularly well suited to architecture search for Transformer\-based EHR models\.

Recent work has also explored using large language models \(LLMs\) to guide NAS\[[14](https://arxiv.org/html/2608.21712#bib.bib46),[47](https://arxiv.org/html/2608.21712#bib.bib34),[22](https://arxiv.org/html/2608.21712#bib.bib47),[34](https://arxiv.org/html/2608.21712#bib.bib31)\]\. Given descriptions of the task, search space, and computational budget, an LLM can propose new architectural configurations, providing a flexible alternative to manually designed search heuristics\. However, existing LLM\-guided approaches generally treat each search as an independent optimization problem\. When architecture search is repeated across clinical tasks or health systems, information from previous searches is not explicitly retained as a reusable prior\. Consequently, each new search must rediscover useful architectural patterns through its own evaluations, even when related tasks or sites have already been explored\.

In this study, we propose ATHENA \(AgenticTransfer acrossHospitals forEHRNeuralArchitecture Search\), a knowledge\-guided agentic NAS framework for Transformer\-based EHR modeling\. ATHENA addresses the two limitations identified above through weight\-sharing evaluation and a cross\-hospital architecture prior\. Specifically, we construct an AutoFormer\-style supernet\[[3](https://arxiv.org/html/2608.21712#bib.bib15)\]that is pretrained once for each hospital and reused throughout architecture search, allowing each candidate architecture to inherit the corresponding subnetwork weights and be evaluated through fine\-tuning rather than independent pretraining\. ATHENA then combines a multi\-agent LLM search with a two\-layer prior that transfers architectural knowledge from previously studied hospitals\. The first layer retrieves high\-performing architectures from source sites based on task descriptors, while the second estimates the effects of architectural components using SHapley Additive exPlanations \(SHAP\)\-based meta\-regression\. Together, these components allow the search to build on previously observed architectural patterns rather than treating each hospital–task pair as an independent optimization problem\.

Across six clinical prediction tasks and two independent health systems, ATHENA achieves up to a 15\.3\-fold speedup in architecture evaluation compared with conventional independent pretraining and fine\-tuning\. With a limited search budget, ATHENA matches or outperforms conventional and LLM\-guided NAS baselines while exhibiting more consistent architecture selection across repeated searches\.

## 2Related work

### 2\.1Transformer\-based EHR modeling

The longitudinal and irregular nature of EHR data makes modeling patient trajectories particularly challenging\. Transformer architectures\[[35](https://arxiv.org/html/2608.21712#bib.bib11)\]have consequently become widely used for EHR modeling because of their ability to capture dependencies across longitudinal clinical events\. BEHRT\[[21](https://arxiv.org/html/2608.21712#bib.bib2)\]represents medical codes as tokens and models longitudinal patient histories using Transformer encoders, while Med\-BERT\[[28](https://arxiv.org/html/2608.21712#bib.bib3)\]demonstrates the effectiveness of large\-scale pretraining on millions of patient records\.

Subsequent work has incorporated richer temporal and clinical information into Transformer\-based EHR models\. CEHR\-BERT\[[26](https://arxiv.org/html/2608.21712#bib.bib7)\]introduces temporal\-aware embeddings and auxiliary learning objectives, whereas Hi\-BEHRT\[[20](https://arxiv.org/html/2608.21712#bib.bib21)\]employs a hierarchical architecture to capture long\-range dependencies in extended patient histories\. More recent studies have explored Transformer architectures beyond the conventional encoder\-only design\. TransformEHR\[[43](https://arxiv.org/html/2608.21712#bib.bib8)\]adopts an encoder–decoder architecture with generative pretraining objectives for longitudinal clinical modeling, while Foresight\[[18](https://arxiv.org/html/2608.21712#bib.bib19)\]employs GPT\-style autoregressive pretraining to model patient trajectories\. CLMBR\[[39](https://arxiv.org/html/2608.21712#bib.bib20)\]learns transferable patient representations through autoregressive next\-code prediction\.

Collectively, these studies demonstrate the value of pretraining and contextualized representation learning for clinical prediction\. Despite these advances, the architectures of Transformer\-based EHR models remain largely handcrafted, with model configurations typically selected through manual tuning or limited hyperparameter search\. This limitation motivates the use of NAS to systematically explore Transformer architectures for downstream clinical prediction tasks\.

### 2\.2NAS for healthcare models

NAS automates architecture design by defining a search space, a search strategy, and a performance\-estimation scheme\[[38](https://arxiv.org/html/2608.21712#bib.bib23)\]\. Recent advances, including gradient\-based optimization\[[23](https://arxiv.org/html/2608.21712#bib.bib24)\]and weight\-sharing supernet methods\[[12](https://arxiv.org/html/2608.21712#bib.bib22)\], have substantially reduced the computational cost of architecture search\. NAS has also been extended to Transformer architectures through methods such as Evolved Transformer\[[32](https://arxiv.org/html/2608.21712#bib.bib25)\], HAT\[[36](https://arxiv.org/html/2608.21712#bib.bib26)\], NAS\-BERT\[[40](https://arxiv.org/html/2608.21712#bib.bib27)\], AutoBERT\-Zero\[[11](https://arxiv.org/html/2608.21712#bib.bib28)\], and Primer\[[33](https://arxiv.org/html/2608.21712#bib.bib29)\]\. However, these methods were primarily developed for natural\-language and vision tasks and do not explicitly account for the characteristics of longitudinal EHR data\.

Within healthcare, NAS has been applied predominantly to medical imaging\[[1](https://arxiv.org/html/2608.21712#bib.bib30),[34](https://arxiv.org/html/2608.21712#bib.bib31)\]and multimodal learning\[[42](https://arxiv.org/html/2608.21712#bib.bib32),[6](https://arxiv.org/html/2608.21712#bib.bib33)\]\. For structured EHR data, existing efforts have focused mainly on modality fusion and multi\-task prediction\. MUFASA\[[42](https://arxiv.org/html/2608.21712#bib.bib32)\]and AutoFM\[[6](https://arxiv.org/html/2608.21712#bib.bib33)\]search modality\-specific architectures and fusion strategies for integrating longitudinal EHR records with clinical notes, while recent work has also explored NAS for multi\-task disease prediction\[[6](https://arxiv.org/html/2608.21712#bib.bib33)\]\. To our knowledge, existing EHR NAS approaches primarily target fusion or task\-sharing structures rather than the internal architecture of Transformer backbones\. Consequently, architecture search for Transformer backbones tailored to different clinical prediction tasks remains largely unexplored\.

### 2\.3LLM\-driven NAS

Recent work has explored the use of LLMs for NAS, leveraging architectural knowledge encoded during pretraining to generate and refine candidate architectures\. Early studies demonstrated that LLMs can serve as architecture generators or search controllers within NAS pipelines\. GENIUS\[[47](https://arxiv.org/html/2608.21712#bib.bib34)\]employs an LLM as a black\-box architecture optimizer that iteratively proposes and refines candidate architectures through natural\-language interaction, while EvoPrompting\[[2](https://arxiv.org/html/2608.21712#bib.bib35)\]integrates LLM\-based mutation and crossover operators into an evolutionary search framework\. Subsequent work has further combined LLM reasoning with conventional search strategies\. GPT\-NAS\[[45](https://arxiv.org/html/2608.21712#bib.bib36)\]couples GPT\-guided architecture generation with evolutionary optimization, whereas LLMatic\[[25](https://arxiv.org/html/2608.21712#bib.bib37)\]integrates LLM\-based code generation with quality\-diversity search to discover diverse and robust neural architectures\.

Applications of LLM\-driven NAS to healthcare remain limited and have focused primarily on medical imaging\. For example, a recent framework for histopathology diagnosis uses an LLM to iteratively refine the search space of a one\-shot NAS pipeline, enabling the discovery of lightweight and transferable models across multiple pathology tasks\[[34](https://arxiv.org/html/2608.21712#bib.bib31)\]\. Despite these advances, LLM\-driven NAS has not been systematically explored for longitudinal EHR modeling\. Moreover, existing methods typically treat each architecture search as an independent optimization problem and do not explicitly reuse architectural knowledge from previous searches across related clinical tasks or health systems\. ATHENA extends this emerging paradigm by combining multi\-agent LLM search with retrieved high\-performing architectures and cross\-hospital architecture\-effect priors for Transformer\-based EHR model design\.

## 3Methods

### 3\.1Overview

ATHENA is a two\-stage framework for knowledge\-guided NAS of EHR Transformers \(Figure[1](https://arxiv.org/html/2608.21712#S3.F1)\)\. Let𝒮\\mathcal\{S\}denote the source hospitals,HHa held\-out target hospital, and𝒯\\mathcal\{T\}the clinical prediction tasks\. The target hospital is excluded from all metadata used to construct the cross\-hospital prior\.

InStage 1, ATHENA pretrains one hospital\-specific AutoFormer\-style supernet at eachS∈𝒮S\\in\\mathcal\{S\}using masked language modeling \(MLM\) and evaluates a fixed collection of subnet architectures on the source tasks\. Each metadata record has the form

\(S,t,a,ParamsS,t⁡\(a\),𝐦S,tsrc​\(a\)\),\\left\(S,t,a,\\operatorname\{Params\}\_\{S,t\}\(a\),\\mathbf\{m\}^\{\\mathrm\{src\}\}\_\{S,t\}\(a\)\\right\),wheret∈𝒯t\\in\\mathcal\{T\},aais an architecture,ParamsS,t⁡\(a\)\\operatorname\{Params\}\_\{S,t\}\(a\)is its parameter count, and𝐦S,tsrc​\(a\)\\mathbf\{m\}^\{\\mathrm\{src\}\}\_\{S,t\}\(a\)contains its source performance metrics\. Pooled source records yield two complementary priors:*Layer 1*retrieves concrete high\-performing architectures from a task\-matched source hospital, whereas*Layer 2*summarizes cross\-hospital associations between architectural choices and performance\.

InStage 2, a Proposal Agent generates candidates, a Critic Agent checks and refines them, and an Experiment Agent evaluates accepted subnetworks and chooses whether the next round should emphasize exploration or exploitation\. Both prior layers provide soft guidance together with feedback from earlier target\-hospital evaluations\.

![Refer to caption](https://arxiv.org/html/2608.21712v1/figs/study_workflow.jpg)Figure 1:The ATHENA framework\. Stage 1 evaluates sampled architectures across source hospitals and tasks to obtain architecture–performance metadata\. The metadata yield a retrieval prior of high\-performing source architectures and a SHAP\-based architecture\-effect prior\. Stage 2 combines these priors with target\-validation feedback to guide agentic NAS while adaptively balancing exploration and exploitation\.
### 3\.2Problem formulation

Let𝒞\\mathcal\{C\}denote the vocabulary of diagnosis, medication, laboratory, and procedure tokens\. The chronologically ordered visits of patientppare

𝒱p=\(vp,1,…,vp,Tp\),vp,τ=\{cp,τ,1,…,cp,τ,Np,τ\},\\mathcal\{V\}\_\{p\}=\(v\_\{p,1\},\\ldots,v\_\{p,T\_\{p\}\}\),\\qquad v\_\{p,\\tau\}=\\\{c\_\{p,\\tau,1\},\\ldots,c\_\{p,\\tau,N\_\{p,\\tau\}\}\\\},wherecp,τ,i∈𝒞c\_\{p,\\tau,i\}\\in\\mathcal\{C\}\. Visits are chronological, but no within\-visit order is assumed because event timestamps within an encounter may not reflect the underlying clinical sequence\[[5](https://arxiv.org/html/2608.21712#bib.bib13),[28](https://arxiv.org/html/2608.21712#bib.bib3),[46](https://arxiv.org/html/2608.21712#bib.bib14)\]\. For each prediction example, only visits available by its index time are retained\. They are flattened into a token sequence, prepended with\[CLS\], and annotated with token\-type and visit\-index embeddings\.

For target hospitalHHand tasktt, a candidate architecture is

a=\(d,L,γ,h\),a=\(d,L,\\gamma,h\),whereddis the embedding dimension,LLis the number of encoder layers,γ\\gammais the multi\-layer perceptron \(MLP\) expansion ratio, andhhis the number of attention heads\. Let𝒟\\mathcal\{D\},ℒ\\mathcal\{L\},ℛ\\mathcal\{R\}, andℋ\\mathcal\{H\}denote their respective candidate sets\. The unfiltered search space is

𝒜0=𝒟×ℒ×ℛ×ℋ,\\mathcal\{A\}\_\{0\}=\{\}\\mathcal\{D\}\\times\\mathcal\{L\}\\times\\mathcal\{R\}\\times\\mathcal\{H\},in the order\(d,L,γ,h\)\(d,L,\\gamma,h\)\. The target\-specific legal set is

𝒜H,t=\{a∈𝒜0:dmodh=0,ParamsH,t⁡\(a\)≤Pmax\},\\mathcal\{A\}\_\{H,t\}=\{\}\\left\\\{\\begin\{aligned\} a\\in\\mathcal\{A\}\_\{0\}:\\;&d\\bmod h=0,\\\\ &\\operatorname\{Params\}\_\{H,t\}\(a\)\\leq P\_\{\\max\}\\end\{aligned\}\\right\\\},wherePmaxP\_\{\\max\}is the parameter\-count limit\. The divisibility condition ensures an integral attention\-head dimension\.

During target search, each evaluated architecture produces a validation metric vector over

𝒢=\{Accuracy,F1,AUROC,AUPRC\}\.\\mathcal\{G\}=\\\{\\mathrm\{Accuracy\},F\_\{1\},\\mathrm\{AUROC\},\\mathrm\{AUPRC\}\\\}\.For binary tasks,F1F\_\{1\}is the positive\-class binaryF1F\_\{1\}, and AUROC and AUPRC use the positive\-class probability\. For multilabel tasks, all metrics are macro\-averaged over label classes\. Afterbbaccepted evaluations, let

ℰb=\(\(,,,\)\)i=1b\\mathcal\{E\}\_\{b\}=\\left\(\\left\(a\_\{i\},\\mathbf\{m\}\_\{\\mathrm\{val\},t\}\(a\_\{i\}\)\\right\)\\right\)\_\{i=1\}^\{b\}denote the ordered target\-validation history\. ATHENA ranks each metric in descending performance order, assigns average ranks to ties, and computes

Rt​\(a,ℰb\)\\displaystyle R\_\{t\}\(a;\\mathcal\{E\}\_\{b\}\)=1\|𝒢\|​∑g∈𝒢rankg,t↓⁡\(a;ℰb\),\\displaystyle=\\frac\{1\}\{\|\\mathcal\{G\}\|\}\\sum\_\{g\\in\\mathcal\{G\}\}\\operatorname\{rank\}^\{\\downarrow\}\_\{g,t\}\(a;\\mathcal\{E\}\_\{b\}\),\(1\)where lower ranks are preferred\. If composite ranks tie, the architecture evaluated first is retained, making selection deterministic with respect to the archive order\.

Letbend≤Bb\_\{\\mathrm\{end\}\}\\leq Bbe the number of architectures evaluated before budget exhaustion or early termination\. ATHENA returns

aH,t⋆=arg​mina:\(a,𝐦\)∈ℰbendRt\(a;ℰbend\)\.a^\{\\star\}\_\{H,t\}=\\argmin\_\{a:\\,\(a,\\mathbf\{m\}\)\\in\\mathcal\{E\}\_\{b\_\{\\mathrm\{end\}\}\}\}R\_\{t\}\(a;\\mathcal\{E\}\_\{b\_\{\\mathrm\{end\}\}\}\)\.HereBBis the maximum evaluation budget\.

### 3\.3Transformer subnet architecture

Given architecturea=\(d,L,γ,h\)a=\(d,L,\\gamma,h\), tokeniiis represented by the sum of its code, token\-type, and visit\-index embeddings:

𝐡i0=𝐞icode\+𝐞itype\+𝐞ivisit∈ℝd,\\mathbf\{h\}^\{0\}\_\{i\}=\\mathbf\{e\}^\{\\mathrm\{code\}\}\_\{i\}\+\\mathbf\{e\}^\{\\mathrm\{type\}\}\_\{i\}\+\\mathbf\{e\}^\{\\mathrm\{visit\}\}\_\{i\}\\in\\mathbb\{R\}^\{d\},where𝐞icode\\mathbf\{e\}^\{\\mathrm\{code\}\}\_\{i\},𝐞itype\\mathbf\{e\}^\{\\mathrm\{type\}\}\_\{i\}, and𝐞ivisit\\mathbf\{e\}^\{\\mathrm\{visit\}\}\_\{i\}denote the corresponding embedding vectors\.

The input sequence consists of a prepended\[CLS\]token followed by the flattened EHR token sequence:

𝐇0=\[𝐡\[CLS\]0,𝐡10,…,𝐡n0\]∈ℝ\(n\+1\)×d\.\\mathbf\{H\}^\{0\}=\\left\[\\mathbf\{h\}^\{0\}\_\{\\mbox\{\{\[CLS\]\}\}\},\\mathbf\{h\}^\{0\}\_\{1\},\\ldots,\\mathbf\{h\}^\{0\}\_\{n\}\\right\]\\in\\mathbb\{R\}^\{\(n\+1\)\\times d\}\.
ATHENA uses a pre\-normalization Transformer encoder\. Multi\-head attention \(MHA\), feed\-forward network \(FFN\), and layer normalization \(LN\) denote the corresponding Transformer operations below\. For layerℓ=1,…,L\\ell=1,\\ldots,L,

𝐇~ℓ\\displaystyle\\widetilde\{\\mathbf\{H\}\}^\{\\ell\}=𝐇ℓ−1\+MHAa,ℓ⁡\(LNa,ℓattn⁡\(𝐇ℓ−1\)\),\\displaystyle=\\mathbf\{H\}^\{\\ell\-1\}\+\\operatorname\{MHA\}\_\{a,\\ell\}\\\!\\left\(\\operatorname\{LN\}^\{\\mathrm\{attn\}\}\_\{a,\\ell\}\(\\mathbf\{H\}^\{\\ell\-1\}\)\\right\),𝐇ℓ\\displaystyle\\mathbf\{H\}^\{\\ell\}=𝐇~ℓ\+FFNa,ℓ⁡\(LNa,ℓffn⁡\(𝐇~ℓ\)\),\\displaystyle=\\widetilde\{\\mathbf\{H\}\}^\{\\ell\}\+\\operatorname\{FFN\}\_\{a,\\ell\}\\\!\\left\(\\operatorname\{LN\}^\{\\mathrm\{ffn\}\}\_\{a,\\ell\}\(\\widetilde\{\\mathbf\{H\}\}^\{\\ell\}\)\\right\),whereMHAa,ℓ\\operatorname\{MHA\}\_\{a,\\ell\}useshhheads andFFNa,ℓ\\operatorname\{FFN\}\_\{a,\\ell\}has hidden widthγ​d\\gamma d\. Dropout and drop\-path are applied within the residual branches\.

For downstream prediction, the hidden representation corresponding to the final\[CLS\]token is used as the patient\-level representation:

𝐳p,j,a=LNa⁡\(𝐡p,j,\[CLS\]L\),\\mathbf\{z\}\_\{p,j,a\}=\\LN\_\{a\}\\\!\\left\(\\mathbf\{h\}^\{L\}\_\{p,j,\\mathrm\{\[CLS\]\}\}\\right\),where𝐡p,j,\[CLS\]L\\mathbf\{h\}^\{L\}\_\{p,j,\\mathrm\{\[CLS\]\}\}is the final\[CLS\]hidden state for target admissionjj\.

### 3\.4AutoFormer\-style supernet training and subnet evaluation

Training every architecture independently would require repeated self\-supervised pretraining and downstream fine\-tuning\. ATHENA therefore uses an AutoFormer\-style shared\-weight supernet\[[3](https://arxiv.org/html/2608.21712#bib.bib15)\]spanning𝒜0\\mathcal\{A\}\_\{0\}\. The supernet takes the componentwise maximal configuration in the search space\. A subnet activates the firstddembedding channels, the firstLLencoder blocks, and the firstγ​d\\gamma dFFN units\. Its attention module partitions a fixed internal query/key/value representation intohhheads, while the input and output projections are sliced to widthdd\. Thus, each candidate architecture is instantiated as a weight\-inherited subnetwork of the shared supernet rather than as an independently trained model\. Figure[2](https://arxiv.org/html/2608.21712#S3.F2)illustrates how the four configurable parameters alter the resulting subnet\.

One supernet is pretrained per hospital and shared by all methods and random seeds at that hospital\. Each MLM minibatch samplesa∼Uniform⁡\(𝒜0\)a\\sim\\operatorname\{Uniform\}\(\\mathcal\{A\}\_\{0\}\), activates the corresponding parameter slices, and updates only those active weights\. During downstream fine\-tuning,aais fixed and the same inherited slices are updated at every epoch\.

Following BERT\-style masking\[[7](https://arxiv.org/html/2608.21712#bib.bib4)\], a subset of non\-special tokens is selected for MLM and corrupted\. LetΩ\\Omegadenote the set of masked positions\. The pretraining loss is

ℒMLM=−∑i∈Ωlogp𝐖a\(ci∣𝐇~0\),\\mathcal\{L\}\_\{\\mathrm\{MLM\}\}=\-\\sum\_\{i\\in\\Omega\}\\log p\_\{\\mathbf\{W\}\_\{a\}\}\\\!\\left\(c\_\{i\}\\mid\\widetilde\{\\mathbf\{H\}\}^\{0\}\\right\),where𝐇~0\\widetilde\{\\mathbf\{H\}\}^\{0\}is constructed from the corrupted tokens and𝐖a\\mathbf\{W\}\_\{a\}denotes the active subnet weights\. Selected non\-special tokens are replaced by\[MASK\], replaced by a random token of the same modality, or left unchanged\.

Each candidate inherits its encoder weights from this checkpoint, is fine\-tuned on the target training split, and is monitored on the target validation split\. The resulting validation metrics form one record in the search archive\.

![Refer to caption](https://arxiv.org/html/2608.21712v1/figs/TF_space.jpg)Figure 2:Illustration of the four configurable Transformer architecture parameters in the NAS space\.
### 3\.5Cross\-hospital prior construction

ATHENA uses source\-hospital metadata to construct a two\-layer prior\.*Layer 1*retrieves high\-performing architectures from a task\-matched source hospital\.*Layer 2*pools source hospitals to estimate task\-specific architectural preferences and interactions\. Both layers are provided to the agents as context rather than imposed as hard constraints\.

#### 3\.5\.1Task\-driven retrieval prior

For taskttat hospitalHH, the task descriptor is

𝐠H,t=\[𝕀binary​\(t\),𝕀multilabel​\(t\),C~t,ℋlabel​\(H,t\),πH,t\+,Δ~t\],\\mathbf\{g\}\_\{H,t\}=\\left\[\\mathbb\{I\}\_\{\\mathrm\{binary\}\}\(t\),\\mathbb\{I\}\_\{\\mathrm\{multilabel\}\}\(t\),\\widetilde\{C\}\_\{t\},\\mathcal\{H\}\_\{\\mathrm\{label\}\}\(H,t\),\\pi^\{\+\}\_\{H,t\},\\widetilde\{\\Delta\}\_\{t\}\\right\],where the first two entries identify the task type,C~t\\widetilde\{C\}\_\{t\}is the normalized number of output classes,ℋlabel\\mathcal\{H\}\_\{\\mathrm\{label\}\}is label entropy,π\+\\pi^\{\+\}is positive\-label prevalence, andΔ~t\\widetilde\{\\Delta\}\_\{t\}is the normalized prediction horizon\. The data\-dependent entries are computed from each hospital’s downstream training split; consequently, the target descriptor uses target\-training data only\.

When exact taskttis present at the source hospitals, ATHENA selects

St⋆=arg​maxS∈𝒮⁡𝐠H,t𝖳​𝐠S,t∥𝐠H,t∥2​∥𝐠S,t∥2\.S\_\{t\}^\{\\star\}=\\argmax\_\{S\\in\\mathcal\{S\}\}\\frac\{\\mathbf\{g\}\_\{H,t\}^\{\\mathsf\{T\}\}\\mathbf\{g\}\_\{S,t\}\}\{\\lVert\\mathbf\{g\}\_\{H,t\}\\rVert\_\{2\}\\lVert\\mathbf\{g\}\_\{S,t\}\\rVert\_\{2\}\}\.Here∥⋅∥2\\lVert\\cdot\\rVert\_\{2\}denotes the Euclidean norm\. ATHENA then ranks the legal source architectures for\(St⋆,t\)\(S\_\{t\}^\{\\star\},t\)by the composite source\-performance rank corresponding to Equation[1](https://arxiv.org/html/2608.21712#S3.E1)and retrieves the topKK\. These configurations and their source metrics form the*Layer 1*retrieval set𝒜ret\\mathcal\{A\}\_\{\\mathrm\{ret\}\}\.

If exact task metadata are unavailable, ATHENA selects the source hospital with the highest cosine similarity between standardized dataset profiles \(e\.g\., sample size, modality counts, and mean encounters per patient\)\. Within the selected hospital, ATHENA chooses the available surrogate task whose descriptor is most similar to𝐠H,t\\mathbf\{g\}\_\{H,t\}\.

#### 3\.5\.2Architecture\-effect prior

For*Layer 2*, architecture evaluations for taskttare pooled across source hospitals\. For source recordii, ATHENA defines

yt,i=−Rt​\(ai,ℰ𝒮,t\),y\_\{t,i\}=\-R\_\{t\}\\\!\\left\(a\_\{i\};\\mathcal\{E\}\_\{\\mathcal\{S\},t\}\\right\),whereℰ𝒮,t\\mathcal\{E\}\_\{\\mathcal\{S\},t\}is the pooled source metadata table for taskttand largeryt,iy\_\{t,i\}indicates better performance relative to all source records for that task\. Hospital identity is retained for the subsequent mixed\-effects analysis\. For each task, ATHENA fits the XGBoost surrogate\[[4](https://arxiv.org/html/2608.21712#bib.bib18)\]

yt,i=fXGBoost,t​\(ai\)\+εt,i\.y\_\{t,i\}=f\_\{\\mathrm\{XGBoost\},t\}\(a\_\{i\}\)\+\\varepsilon\_\{t,i\}\.The discrete\-valued architecture features are supplied to the tree model in their numeric form\. SHAP TreeExplainer\[[24](https://arxiv.org/html/2608.21712#bib.bib17)\]produces a signed contributionϕt,i\(q\)\\phi^\{\(q\)\}\_\{t,i\}for each architectural featureqqand recordii\.

To separate population\-level architectural patterns from hospital variation, ATHENA fits

ϕt,i\(q\)=μt,ℓi,q\(q\)\+ut,Si\(q\)\+ϵt,i\(q\),ut,S\(q\)∼𝒩⁡\(0,σu,q2\),\\phi^\{\(q\)\}\_\{t,i\}=\\mu^\{\(q\)\}\_\{t,\\ell\_\{i,q\}\}\+u^\{\(q\)\}\_\{t,S\_\{i\}\}\+\\epsilon^\{\(q\)\}\_\{t,i\},\\qquad u^\{\(q\)\}\_\{t,S\}\\sim\\mathcal\{N\}\(0,\\sigma\_\{u,q\}^\{2\}\),whereℓi,q\\ell\_\{i,q\}is the categorical level ofqqandut,S\(q\)u^\{\(q\)\}\_\{t,S\}is a hospital random intercept\. The reportedμt,ℓ\(q\)\\mu^\{\(q\)\}\_\{t,\\ell\}values are level\-specific mean SHAP contributions reconstructed from the reference\-coded model\.

Levels whose confidence intervals lie entirely above or below zero are labeled*preferred*or*discouraged*, respectively; the remainder are*inconclusive*\. ATHENA applies the same idea to the most influential feature pair to obtain supported interaction rules\. Feature importance, level labels, confidence intervals, and interaction rules form the*Layer 2*prior𝒫meta\\mathcal\{P\}\_\{\\mathrm\{meta\}\}\. They are presented to the agents as non\-causal, directional evidence\.

### 3\.6Agentic NAS

ATHENA coordinates a*Proposal Agent*ApA\_\{p\}, a*Critic Agent*AcA\_\{c\}, and an*Experiment Agent*\. The Experiment Agent consists of a deterministic evaluatorAeA\_\{e\}and an LLM\-based strategy moduleAsA\_\{s\}\. Proposal, critique, revision, and strategy selection are LLM calls; legality checks, subnet fine\-tuning, metric calculation, ranking, and final selection are controller operations\.

For each target hospital\-task pair\(H,t\)\(H,t\), ATHENA constructs a fixed context

𝒞H,t=\{𝐠H,t,𝒜ret,𝒫meta\},\\mathcal\{C\}\_\{H,t\}=\\\{\\mathbf\{g\}\_\{H,t\},\\mathcal\{A\}\_\{\\mathrm\{ret\}\},\\mathcal\{P\}\_\{\\mathrm\{meta\}\}\\\},containing the target task descriptor, theKKretrieved architectures, and the*Layer 2*prior\.

At roundrr, ATHENA maintains two distinct histories\. The*ordered search memory*\(or search transcript\)ℳ\(r\)\\mathcal\{M\}^\{\(r\)\}contains proposals, critiques, revisions, evaluation feedback, and strategy decisions, and is supplied as LLM context\. The*structured validation archive*

ℰ\(r\)=\(\(,,,\)\)i=1br\\mathcal\{E\}^\{\(r\)\}=\\left\(\\left\(a\_\{i\},\\mathbf\{m\}\_\{\\mathrm\{val\},t\}\(a\_\{i\}\)\\right\)\\right\)\_\{i=1\}^\{b\_\{r\}\}contains only evaluated architectures and their validation metrics\. It is used by the deterministic controller for ranking and selection\. Thus,ℳ\(r\)\\mathcal\{M\}^\{\(r\)\}carries accumulated search experience, whereasℰ\(r\)\\mathcal\{E\}^\{\(r\)\}is the authoritative record of empirical evidence\. The memory is an ordered transcript, not a separately learned or retrieval\-based long\-term\-memory system\.

The round also has remaining budgetB−brB\-b\_\{r\}and a strategy

s\(r\)=\(z\(r\),η\(r\)\),z\(r\)∈\{exploration,exploitation\},s^\{\(r\)\}=\(z^\{\(r\)\},\\eta^\{\(r\)\}\),\\qquad z^\{\(r\)\}\\in\\\{\\textit\{exploration\},\\textit\{exploitation\}\\\},whereη\(r\)\\eta^\{\(r\)\}is a natural\-language rationale\. Exploration encourages coverage of under\-sampled architectural choices, whereas exploitation focuses proposals around configurations that have performed well on target validation\.

Conditioned on\(𝒞H,t,ℳ\(r\),ℰ\(r\),B−br,s\(r\)\)\(\\mathcal\{C\}\_\{H,t\},\\mathcal\{M\}^\{\(r\)\},\\mathcal\{E\}^\{\(r\)\},B\-b\_\{r\},s^\{\(r\)\}\),ApA\_\{p\}proposes candidates with rationales\. The Critic checks legality and novelty, provides structured feedback, and allows rejected non\-duplicate candidates to be revised for at mostRRpasses\. Parameter\-limit violations and duplicates are hard rejections; disagreement with a*Layer 2*preference is only a soft concern\. The proposal–critique–revision records are appended toℳ\(r\)\\mathcal\{M\}^\{\(r\)\}even when no candidate is evaluated\. Figure[3](https://arxiv.org/html/2608.21712#S3.F3)shows the Proposal Agent prompt template\.

Proposal Agent Prompt Template Role:You are an NAS agent for Transformer models applied to longitudinal EHR data\. Generate candidate architectures for the target hospital\-task pair\. Search space and constraints:Choices forembed\_dim,depth,mlp\_ratio, andnum\_heads; requireembed\_dim % num\_heads = 0; enforce parameter\-count constraints; identify infeasible regions\. Task context:Target\-task statistics𝐠H,t\\mathbf\{g\}\_\{H,t\}\. Retrieval prior:Retrieved top\-KKsource architectures𝒜ret\\mathcal\{A\}\_\{\\mathrm\{ret\}\}and their source performance\. Architecture\-effect prior:Preferred/discouraged architectural levels, feature\-importance rankings, confidence labels, and interaction rules𝒫meta\\mathcal\{P\}\_\{\\mathrm\{meta\}\}\. Search state:Ordered transcriptℳH,t\(r\)\\mathcal\{M\}\_\{H,t\}^\{\(r\)\}, validation historyℰ\(r\)\\mathcal\{E\}^\{\(r\)\}, current best architecture, and remaining budget\. Strategy directive:s\(r\)=\(z\(r\),η\(r\)\)s^\{\(r\)\}=\(z^\{\(r\)\},\\eta^\{\(r\)\}\)\. Underexploration, prioritize diversity and coverage of under\-explored regions\. Underexploitation, refine architectures near the current best region\. Output:JSON array of candidate architectures containingembed\_dim,depth,mlp\_ratio,num\_heads, andrationale\. Return only valid JSON\. Test\-set metrics are never provided\.

Figure 3:The Proposal Agent prompt template\. The implementation populates each block with task\-specific values before invoking the LLM\.The evaluator fine\-tunes each accepted subnet and appends the resulting records to bothℰ\(r\)\\mathcal\{E\}^\{\(r\)\}andℳ\(r\)\\mathcal\{M\}^\{\(r\)\}\. The controller then recomputes the leader using Equation[1](https://arxiv.org/html/2608.21712#S3.E1)\. When budget remains,AsA\_\{s\}reads the updated transcript and validation trajectory and selects the next strategy, treating the cross\-hospital prior as secondary evidence\. The strategy decision is also appended toℳ\(r\)\\mathcal\{M\}^\{\(r\)\}\.

If a round produces no legal, non\-duplicate candidate, no evaluation budget is consumed and a consecutive\-failure counter is incremented\. The counter resets after any successful evaluation round\. Search terminates when the budgetBBis exhausted or afterFmaxF\_\{\\max\}consecutive empty rounds, and ATHENA returns the current validation\-selected leader\. A run in which no candidate is evaluated is recorded as unsuccessful\. Algorithm[1](https://arxiv.org/html/2608.21712#alg1)summarizes this search logic\.

Algorithm 1ATHENA Agentic NAS Loop1:Fixed context

𝒞H,t\\mathcal\{C\}\_\{H,t\}, legal set

𝒜H,t\\mathcal\{A\}\_\{H,t\}, budget

BB, critique\-pass limit

RR, failure limit

FmaxF\_\{\\max\}
2:Validation\-selected architecture

aH,t⋆a^\{\\star\}\_\{H,t\}, or failure if no candidate is evaluated

3:Initialize ordered search memory

ℳ←\(\)\\mathcal\{M\}\\leftarrow\(\), validation archive

ℰ←\(\)\\mathcal\{E\}\\leftarrow\(\), strategy

s←explorations\\leftarrow\\textit\{exploration\}, and failure count

f←0f\\leftarrow 0
4:while

\|ℰ\|<B\|\\mathcal\{E\}\|<Band

f<Fmaxf<F\_\{\\max\}do

5:

𝒫←Ap​\(𝒞H,t,ℳ,ℰ,B−\|ℰ\|,s\)\\mathcal\{P\}\\leftarrow A\_\{p\}\(\\mathcal\{C\}\_\{H,t\},\\mathcal\{M\},\\mathcal\{E\},B\-\|\\mathcal\{E\}\|,s\)
6:

\(𝒰,Δ​ℳ\)←CritiqueAndRevise⁡\(𝒫,𝒞H,t,ℳ,ℰ,s,R\)\(\\mathcal\{U\},\\Delta\\mathcal\{M\}\)\\leftarrow\\operatorname\{CritiqueAndRevise\}\(\\mathcal\{P\},\\mathcal\{C\}\_\{H,t\},\\mathcal\{M\},\\mathcal\{E\},s,R\)
7:

ℳ←ℳ∥Δ​ℳ\\mathcal\{M\}\\leftarrow\\mathcal\{M\}\\mathbin\{\\\|\}\\Delta\\mathcal\{M\}
8:Remove illegal and duplicate candidates from

𝒰\\mathcal\{U\}in proposal order

9:Retain at most the first

B−\|ℰ\|B\-\|\\mathcal\{E\}\|candidates in

𝒰\\mathcal\{U\}
10:if

𝒰=∅\\mathcal\{U\}=\\emptysetthen

11:

f←f\+1f\\leftarrow f\+1;continue

12:endif

13:

𝒱←Ae​\(𝒰\)\\mathcal\{V\}\\leftarrow A\_\{e\}\(\\mathcal\{U\}\)using target training and validation data

14:

ℰ←ℰ∥𝒱\\mathcal\{E\}\\leftarrow\\mathcal\{E\}\\mathbin\{\\\|\}\\mathcal\{V\};

ℳ←ℳ∥𝒱\\mathcal\{M\}\\leftarrow\\mathcal\{M\}\\mathbin\{\\\|\}\\mathcal\{V\};

f←0f\\leftarrow 0
15:

abest←arg​mina:\(a,𝐦\)∈ℰRt\(a;ℰ\)a\_\{\\mathrm\{best\}\}\\leftarrow\\argmin\_\{a:\\,\(a,\\mathbf\{m\}\)\\in\\mathcal\{E\}\}R\_\{t\}\(a;\\mathcal\{E\}\)
16:if

\|ℰ\|<B\|\\mathcal\{E\}\|<Bthen

17:

s←As​\(𝒞H,t,ℳ,ℰ,abest,B−\|ℰ\|\)s\\leftarrow A\_\{s\}\(\\mathcal\{C\}\_\{H,t\},\\mathcal\{M\},\\mathcal\{E\},a\_\{\\mathrm\{best\}\},B\-\|\\mathcal\{E\}\|\)
18:

ℳ←ℳ∥s\\mathcal\{M\}\\leftarrow\\mathcal\{M\}\\mathbin\{\\\|\}s
19:endif

20:endwhile

21:if

ℰ=∅\\mathcal\{E\}=\\emptysetthen

22:returnfailure

23:endif

24:return

aH,t⋆←abesta^\{\\star\}\_\{H,t\}\\leftarrow a\_\{\\mathrm\{best\}\}

## 4Experiment configuration

### 4\.1Data sources and cohort construction

This study uses EHR data from OneFlorida\+\[[10](https://arxiv.org/html/2608.21712#bib.bib1),[13](https://arxiv.org/html/2608.21712#bib.bib5)\]and MIMIC\-IV\[[15](https://arxiv.org/html/2608.21712#bib.bib6)\]\. OneFlorida\+ is a large clinical research network within PCORnet\[[9](https://arxiv.org/html/2608.21712#bib.bib48)\]and contains standardized EHR data across multiple health systems\. MIMIC\-IV is a public de\-identified EHR database from Beth Israel Deaconess Medical Center containing more than 524,000 hospital admissions from over 257,000 patients\. Both datasets include longitudinal diagnoses, procedures, medications, and laboratory records used to construct the clinical prediction tasks\.

For ATHENA, the processed OneFlorida\+ data comprise four prior\-source sites \(Sites A–D\) and one internal held\-out target \(Site E\); MIMIC\-IV serves as the external held\-out target\. Records from both target cohorts are excluded from the cross\-hospital prior\. The OneFlorida\+ and MIMIC\-IV cohorts follow the same EHR tokenization, task\-construction, and split conventions\.

Each hospital’s unlabeled EHR corpus is first divided into a pretraining pool of patients without downstream labels and a downstream pool containing the remaining patients\. The pretraining pool is further split 90%/10% for masked\-language\-model supernet pretraining and validation\.

We evaluate six downstream clinical prediction tasks: in\-hospital mortality \(Mortality\), Stay\>\>7d, Readmission \(3M\), Phenotype \(6M\), Phenotype \(12M\), and same\-visit Drug Recommendation\. The two phenotype tasks are multilabel predictions over 18 commonly benchmarked phenotype classes\[[41](https://arxiv.org/html/2608.21712#bib.bib9),[44](https://arxiv.org/html/2608.21712#bib.bib10)\]\. Phenotype cohorts include patients with a qualifying next admission\. Drug Recommendation uses a separate eligibility cohort consisting of patients with at least one in\-vocabulary medication\. Cohort inclusion and exclusion criteria and detailed task definitions are provided in Supplementary Method S1 and Supplementary Table S1\.

Within the downstream pool, patient\-level training, validation, and test splits are defined by task family: 20%/40%/40% for the three binary tasks \(Mortality, Stay\>\>7d, and Readmission \(3M\)\) and 40%/30%/30% for Phenotype \(6M\) and Phenotype \(12M\)\. Drug Recommendation is independently split 40%/30%/30% within its eligibility cohort\. For all tasks, the training set is used for subnet fine\-tuning, the validation set guides architecture search and model selection, and the test set is evaluated only once after the search terminates\.

### 4\.2Transformer architecture search space

The NAS search space comprises four configurable Transformer architecture parameters: embedding dimension, depth, number of attention heads, and MLP expansion ratio \(Figure[2](https://arxiv.org/html/2608.21712#S3.F2)\)\. Embedding dimension takes values in\{32,64,128,256\}\\\{32,64,128,256\\\}, while depth, number of attention heads, and MLP expansion ratio each take values in\{1,2,4,8\}\\\{1,2,4,8\\\}\. The Cartesian product of these choices yields44=2564^\{4\}=256candidate architectures\. The same search space is used across all hospitals, tasks, and NAS methods to ensure a consistent comparison\. Other implementation details are provided in Supplementary Method S2\.

### 4\.3Compared methods and ablations

We compare ATHENA against representative baselines from both classical and LLM\-based NAS:

- •Random Search\.Uniformly samples legal, non\-duplicate architectures from𝒜H,t\\mathcal\{A\}\_\{H,t\}until the evaluation budget is exhausted\.
- •EA\. Following regularized evolution for NAS\[[29](https://arxiv.org/html/2608.21712#bib.bib43)\], this baseline maintains a population of candidate architectures and iteratively selects high\-performing parents through tournament selection, generates new candidates via mutation, and removes the oldest individuals from the population\.
- •GENIUS\[[47](https://arxiv.org/html/2608.21712#bib.bib34)\]\. A single\-agent LLM\-based NAS framework that iteratively proposes candidate architectures based on search history and validation feedback, without the multi\-agent collaboration used in ATHENA\.
- •CoLLM\-NAS\[[22](https://arxiv.org/html/2608.21712#bib.bib47)\]\. A collaborative LLM\-based NAS framework that uses a stateful Navigator to iteratively refine search strategies from evaluation feedback and historical trajectories, and a stateless Generator to translate these strategies into candidate architectures\. A Coordinator manages their interaction, validates generated architectures, and maintains the search archive\. Unlike ATHENA, CoLLM\-NAS does not leverage cross\-site architecture priors\.

ATHENA incorporates a two\-layer cross\-hospital prior consisting of the*Layer 1*retrieval prior and the*Layer 2*architecture\-effect prior\. To quantify the contribution of each component, we evaluate the following ablated variants:

- •*L1\-only\.*Retains the retrieval\-based cross\-hospital prior while removing the architecture\-effect prior, quantifying the contribution of*Layer 2*\.
- •Leave\-One\-Task\-Out \(LOTO\) Retrieval\.Excludes task\-matched source records during retrieval, forcing knowledge transfer from related but non\-identical tasks and evaluating the robustness of retrieval\-based priors when exact precedents are unavailable\.
- •Cold Start\.Removes both the retrieval prior and the architecture\-effect prior, evaluating the effectiveness of the agentic NAS framework in the absence of cross\-hospital knowledge\.

## 5Results

### 5\.1Study cohorts

Table 1:Pretraining\-pool sizes and patient\-level fine\-tuning splits\. OneFlorida\+ contributor sites are de\-identified as Site A–E\.*Share*denotes the fraction of cohort patients in the pretraining pool\. Fine\-tuning counts are shown in separate training, validation, and test columns\.*Binary*denotes the common split for Mortality, Stay\>\>7d, and Readmission \(3M\)\. Phenotype and Drug Recommendation are abbreviated as Pheno\. and Drug Rec\., respectively\.†Drug Recommendation uses a separately defined drug\-eligible cohort \(≥1\\geq 1in\-vocabulary medication\) and an independent patient\-level split; its counts may therefore exceed the pretraining\-pool patient count in the*Patients*column\.PretrainingFine\-tuning \(Binary\)CohortPatientsShareTrainVal\.Test*Prior source pool*Site A22,18642\.0%6,15812,23412,253Site B29,22252\.4%5,30010,56610,647Site C37,15242\.0%10,13420,59420,647Site D37,25963\.9%4,2048,4098,410Total125,81949\.3%25,79651,80351,957*Target cohorts*Site E \(internal\)54,41064\.4%6,01312,02712,029MIMIC\-IV \(external\)42,49470\.0%3,6427,2847,286
CohortTaskTrainVal\.Test*Prior source pool*Site APheno\. 6M11,0838,1478,405Pheno\. 12M12,2359,1249,100Drug Rec\.†78,99959,24959,251Site BPheno\. 6M9,2296,8926,921Pheno\. 12M10,2857,7927,643Drug Rec\.†73,66055,24555,246Site CPheno\. 6M18,25713,92213,699Pheno\. 12M20,08214,95314,919Drug Rec\.†92,60069,45069,450Site DPheno\. 6M7,1035,3275,328Pheno\. 12M7,9885,9915,991Drug Rec\.†22,95817,21817,219TotalPheno\. 6M45,67234,28834,353Pheno\. 12M50,59037,86037,653Drug Rec\.†268,217201,162201,166*Target cohorts*Site EPheno\. 6M9,1956,8966,897Pheno\. 12M10,3827,7867,787Drug Rec\.†33,38025,03525,036MIMIC\-IVPheno\. 6M3,8662,9002,901Pheno\. 12M4,4783,3593,360Drug Rec\.†24,01718,01218,014

The cross\-hospital prior was constructed from four de\-identified OneFlorida\+ source sites \(Sites A–D\)\. Their pretraining pools comprised 125,819 patients, representing 49\.3% of patients at the four source sites \(Table[1](https://arxiv.org/html/2608.21712#S5.T1)\)\. The target cohorts were excluded from prior construction\. The pretraining pools comprised 54,410 patients at Site E \(64\.4% of the cohort\) and 42,494 patients in MIMIC\-IV \(70\.0% of the cohort\)\. The source and target cohorts differed substantially in case mix and data density \(Supplementary Table S2\)\. Across the six cohorts, Mortality ranged from 0\.4% to 4\.2%, Stay\>\>7d from 9\.6% to 34\.0%, and Readmission \(3M\) from 12\.7% to 23\.9%\. Median length of stay ranged from 2 to 5 days, and admissions per patient ranged from 1\.4 to 2\.7\. Diagnoses per admission ranged from 5\.4 to 19\.1 and medications per admission from 3\.2 to 13\.1\. Comorbidity prevalence was also heterogeneous: chronic kidney disease ranged from 2\.8% to 16\.0% of admissions and coronary atherosclerosis from 1\.1% to 23\.3%\. Together, these differences provided heterogeneous settings in which to evaluate transfer of architecture knowledge\.

### 5\.2Supernet ranking fidelity and search efficiency

Table 2:Supernet ranking fidelity on MIMIC\-IV\. For each task, 150 architectures are sampled and evaluated both as weight\-inherited subnetworks of the pretrained supernet and through independent pretraining followed by fine\-tuning\.ρ\\rhodenotes the Spearman rank correlation between the resulting architecture rankings\.Taskρ\\rho\(AUROC\)ρ\\rho\(AUPRC\)Mortality0\.5440\.567Stay\>\>7d0\.8090\.806Readmission \(3M\)0\.5700\.412Phenotype \(6M\)0\.9510\.944Phenotype \(12M\)0\.9400\.944Drug Recommendation0\.9440\.958Before using supernet\-based evaluations to guide architecture search, we first assessed whether weight\-inherited subnetworks preserved the relative performance ranking of independently trained architectures\. For each MIMIC\-IV task, we sampled 150 valid Transformer architectures \(58\.59% of the search space\) and evaluated each under two settings: fine\-tuning with weights inherited from the pretrained supernet and independent pretraining followed by fine\-tuning\. We then computed Spearman rank correlations between the resulting architecture rankings based on AUROC and AUPRC\. This analysis examined whether the computationally cheaper supernet\-based approach could serve as a reliable proxy for conventional pretrain\-then\-fine\-tune evaluation\.

The AutoFormer\-style weight\-sharing supernet preserved the relative performance of conventionally pretrained and fine\-tuned Transformer architectures \(Table[2](https://arxiv.org/html/2608.21712#S5.T2)\)\. Spearman correlations were positive for every task and metric\. Ranking fidelity was particularly high for the three multilabel tasks: correlations ranged from 0\.940 to 0\.951 for Phenotype \(6M\) and Phenotype \(12M\), and from 0\.944 to 0\.958 for Drug Recommendation\. Stay\>\>7d also showed strong agreement \(ρ=0\.809\\rho=0\.809for AUROC andρ=0\.806\\rho=0\.806for AUPRC\)\. Agreement was more moderate for Mortality and Readmission \(3M\), with the lowest correlation observed for Readmission \(3M\) AUPRC \(ρ=0\.412\\rho=0\.412\)\. Thus, although fidelity varied by task, the supernet provided an informative proxy ranking across the complete search space\.

Table 3:Search compute under weight sharing, in GPU\-minutes on one NVIDIA L4\. The shared\-weight approach pretrains the supernet*once*and fine\-tunes each ofNNcandidates on it \(Tpre\+N​TftT\_\{\\mathrm\{pre\}\}\+N\\,T\_\{\\mathrm\{ft\}\}\); conventional from\-scratch NAS incursN⁡\(Tpre\+Tft\)N\(T\_\{\\mathrm\{pre\}\}\+T\_\{\\mathrm\{ft\}\}\)\. Measured times are OneFlorida\+Tpre=30T\_\{\\mathrm\{pre\}\}\{=\}30andTft=2T\_\{\\mathrm\{ft\}\}\{=\}2, and MIMIC\-IVTpre=29T\_\{\\mathrm\{pre\}\}\{=\}29andTft=1T\_\{\\mathrm\{ft\}\}\{=\}1\. All compared NAS methods share one supernet and therefore have the same per\-candidate evaluation cost\. Values at the maximum budget of 30 evaluations are shown inbold\.OneFlorida\+ held\-outMIMIC\-IVNNSharedScratchSpeedupSharedScratchSpeedup5401604\.0×\\times341504\.4×\\times10503206\.4×\\times393007\.7×\\times20706409\.1×\\times4960012\.2×\\times309096010\.7×\\times5990015\.3×\\timesWe next measured the GPU time required to evaluate increasing numbers of candidate architectures on one NVIDIA L4\. The weight\-sharing calculation included one supernet pretraining run followed by subnet fine\-tuning for each candidate, whereas conventional search repeated pretraining and fine\-tuning independently for every candidate\. Weight sharing yielded larger reductions in search cost as more architectures were evaluated \(Table[3](https://arxiv.org/html/2608.21712#S5.T3)\)\. At a budget of five evaluations, the measured speedup over conventional pretrain\-then\-fine\-tune evaluation was 4\.0×\\timeson OneFlorida\+ and 4\.4×\\timeson MIMIC\-IV\. At 30 evaluations, the corresponding speedups increased to 10\.7×\\timesand 15\.3×\\times, reducing the estimated cost from 960 to 90 GPU\-minutes on OneFlorida\+ and from 900 to 59 GPU\-minutes on MIMIC\-IV\.

### 5\.3Task\-specific architecture\-effect prior

The cross\-hospital architecture\-effect prior captured both shared and task\-specific Transformer design preferences across the six clinical tasks \(Supplementary Figure S1\)\. The most consistent signal was model width\. Embedding dimensions of 128 and 256 were reliably preferred for Mortality, while dimensions of 64, 128, and 256 were preferred for the other five tasks\. Increasing the number of attention heads was also generally favorable: configurations with 2, 4, or 8 heads had positive effects for Mortality, Stay\>\>7d, Readmission \(3M\), Phenotype \(6M\), and Phenotype \(12M\)\. For Drug Recommendation, 4 and 8 heads had positive effects relative to the one\-head reference level, whereas the effect of 2 heads was inconclusive\.

Depth showed a more heterogeneous pattern\. Mortality and Stay\>\>7d favored depths of 2, 4, and 8 over the single\-layer reference configuration\. Readmission \(3M\) did not show the same trend: depths of 2 and 8 were discouraged, while the effect of a depth of 4 was inconclusive\. Among the multilabel tasks, Phenotype \(6M\) favored a depth of 2 but discouraged depths of 4 and 8; Phenotype \(12M\) and Drug Recommendation discouraged all depths greater than one\. The MLP expansion ratio showed a similarly conservative pattern\. Ratios greater than one were consistently discouraged for Mortality, Stay\>\>7d, Readmission \(3M\), and Phenotype \(6M\)\. For Phenotype \(12M\), ratios of 4 and 8 were discouraged and a ratio of 2 was inconclusive; for Drug Recommendation, a ratio of 4 was discouraged while ratios of 2 and 8 were inconclusive\. Thus, the prior identified wider embeddings and multi\-head attention as relatively transferable signals, while retaining task\-dependent guidance for depth and MLP expansion ratio rather than prescribing one architecture for all outcomes\.

Pairwise rules derived from each task’s two most influential features were also supplied to the search prompt \(Supplementary Table S3\)\. For five of the six tasks, the direction of the identified preference remained unchanged across all levels of the second feature\. Mortality was the exception, where the preference for an embedding dimension of 256 emerged when the number of attention heads was at least two\.

### 5\.4Predictive performance under different search budgets

Table 4:Test AUPRC across target hospitals and prediction tasks under increasing NAS budgets of 5, 20, and 30 architecture evaluations \(mean±\\pmstandard deviation over five random seeds\)\. For each row, the best mean is inboldand the second\-best mean isunderlined; tied values share the mark\.Avg\. Rankis the mean within\-row rank over the 12 hospital–task comparisons in each panel \(1 = best; ties receive the average rank\), with the number of best\-in\-row comparisons in parentheses\. All methods search the same pretrained supernet and therefore differ only in search strategy\.TargetTaskRandom SearchEAGENIUSCoLLM\-NASATHENA\(a\) Search budget = 5 evaluationsOneFlorida\+Mortality57\.78±\\pm0\.9057\.78±\\pm0\.9058\.04±\\pm0\.3457\.72±\\pm1\.3658\.23±\\pm1\.72OneFlorida\+Stay\>\>7d73\.25±\\pm0\.3373\.25±\\pm0\.3374\.00±\\pm0\.3473\.59±\\pm0\.4574\.03±\\pm0\.37OneFlorida\+Readmission \(3M\)52\.09±\\pm0\.4252\.09±\\pm0\.4252\.06±\\pm0\.5852\.16±\\pm0\.4052\.26±\\pm0\.27OneFlorida\+Phenotype \(6M\)29\.65±\\pm0\.5929\.53±\\pm0\.4029\.28±\\pm0\.8729\.29±\\pm1\.5030\.42±\\pm0\.51OneFlorida\+Phenotype \(12M\)30\.86±\\pm0\.7130\.74±\\pm0\.6930\.39±\\pm0\.5730\.23±\\pm0\.6032\.17±\\pm0\.42OneFlorida\+Drug Recommendation19\.09±\\pm0\.2419\.09±\\pm0\.2419\.11±\\pm0\.2719\.16±\\pm0\.2219\.32±\\pm0\.08MIMIC\-IVMortality79\.25±\\pm0\.6479\.25±\\pm0\.6479\.29±\\pm0\.6279\.52±\\pm0\.4379\.31±\\pm0\.40MIMIC\-IVStay\>\>7d74\.67±\\pm0\.4174\.80±\\pm0\.5975\.22±\\pm0\.6675\.02±\\pm0\.6075\.66±\\pm0\.71MIMIC\-IVReadmission \(3M\)43\.64±\\pm0\.4743\.64±\\pm0\.4744\.29±\\pm0\.2644\.55±\\pm0\.5544\.47±\\pm0\.44MIMIC\-IVPhenotype \(6M\)48\.96±\\pm0\.7648\.97±\\pm0\.7648\.19±\\pm0\.6648\.63±\\pm0\.6749\.25±\\pm0\.57MIMIC\-IVPhenotype \(12M\)48\.64±\\pm0\.4148\.63±\\pm0\.3947\.98±\\pm0\.4148\.79±\\pm0\.6449\.26±\\pm0\.09MIMIC\-IVDrug Recommendation20\.63±\\pm0\.4320\.63±\\pm0\.4320\.36±\\pm0\.3620\.24±\\pm0\.1221\.16±\\pm0\.17Avg\. Rank \(best\-in\-row\)3\.54 \(0\)3\.62 \(0\)3\.58 \(0\)3\.08 \(2\)1\.17 \(10\)\(b\) Search budget = 20 evaluationsOneFlorida\+Mortality57\.46±\\pm0\.5058\.10±\\pm0\.4858\.18±\\pm0\.6157\.88±\\pm0\.9258\.85±\\pm0\.84OneFlorida\+Stay\>\>7d73\.28±\\pm0\.5273\.42±\\pm0\.2373\.60±\\pm0\.7373\.65±\\pm0\.5173\.90±\\pm0\.28OneFlorida\+Readmission \(3M\)52\.21±\\pm0\.2252\.34±\\pm0\.1852\.40±\\pm0\.1652\.49±\\pm0\.1252\.44±\\pm0\.28OneFlorida\+Phenotype \(6M\)30\.57±\\pm0\.2030\.23±\\pm1\.0030\.02±\\pm0\.5229\.46±\\pm1\.5731\.06±\\pm0\.55OneFlorida\+Phenotype \(12M\)31\.53±\\pm0\.6632\.14±\\pm0\.8531\.37±\\pm0\.7630\.54±\\pm1\.0632\.62±\\pm0\.29OneFlorida\+Drug Recommendation19\.28±\\pm0\.1219\.27±\\pm0\.2119\.44±\\pm0\.0619\.29±\\pm0\.1919\.32±\\pm0\.04MIMIC\-IVMortality79\.16±\\pm0\.4179\.46±\\pm0\.2479\.21±\\pm0\.4579\.48±\\pm0\.6379\.75±\\pm0\.37MIMIC\-IVStay\>\>7d75\.18±\\pm0\.6475\.67±\\pm0\.2675\.39±\\pm0\.4575\.20±\\pm0\.5875\.64±\\pm0\.48MIMIC\-IVReadmission \(3M\)44\.25±\\pm0\.3544\.88±\\pm0\.9744\.61±\\pm0\.4644\.94±\\pm0\.5744\.94±\\pm0\.55MIMIC\-IVPhenotype \(6M\)49\.43±\\pm0\.5349\.22±\\pm0\.6949\.26±\\pm0\.6949\.01±\\pm0\.7349\.84±\\pm0\.35MIMIC\-IVPhenotype \(12M\)49\.10±\\pm0\.1649\.04±\\pm0\.2849\.05±\\pm0\.2448\.66±\\pm0\.5949\.42±\\pm0\.20MIMIC\-IVDrug Recommendation21\.08±\\pm0\.0920\.98±\\pm0\.3320\.75±\\pm0\.3520\.39±\\pm0\.3221\.15±\\pm0\.13Avg\. Rank \(best\-in\-row\)3\.75 \(0\)3\.25 \(1\)3\.17 \(1\)3\.54 \(2\)1\.29 \(9\)\(c\) Search budget = 30 evaluationsOneFlorida\+Mortality57\.91±\\pm0\.4058\.07±\\pm0\.4958\.18±\\pm0\.6157\.88±\\pm0\.9258\.85±\\pm0\.84OneFlorida\+Stay\>\>7d73\.52±\\pm0\.7873\.97±\\pm0\.4074\.10±\\pm0\.1373\.65±\\pm0\.5173\.90±\\pm0\.27OneFlorida\+Readmission \(3M\)52\.27±\\pm0\.2452\.41±\\pm0\.2052\.54±\\pm0\.2352\.49±\\pm0\.1252\.56±\\pm0\.24OneFlorida\+Phenotype \(6M\)30\.57±\\pm0\.2031\.01±\\pm0\.3030\.02±\\pm0\.5229\.46±\\pm1\.5731\.06±\\pm0\.55OneFlorida\+Phenotype \(12M\)31\.71±\\pm0\.4932\.34±\\pm0\.7531\.44±\\pm0\.7130\.54±\\pm1\.0632\.54±\\pm0\.40OneFlorida\+Drug Recommendation19\.32±\\pm0\.1119\.43±\\pm0\.0819\.44±\\pm0\.0619\.29±\\pm0\.1919\.34±\\pm0\.07MIMIC\-IVMortality79\.16±\\pm0\.4279\.30±\\pm0\.5479\.12±\\pm0\.5779\.48±\\pm0\.6379\.75±\\pm0\.37MIMIC\-IVStay\>\>7d75\.01±\\pm0\.5075\.69±\\pm0\.4175\.56±\\pm0\.4275\.20±\\pm0\.5875\.88±\\pm0\.43MIMIC\-IVReadmission \(3M\)44\.76±\\pm0\.1644\.78±\\pm1\.0244\.79±\\pm0\.3344\.94±\\pm0\.5745\.13±\\pm0\.53MIMIC\-IVPhenotype \(6M\)49\.61±\\pm0\.5149\.56±\\pm0\.4849\.22±\\pm0\.6849\.01±\\pm0\.7349\.84±\\pm0\.35MIMIC\-IVPhenotype \(12M\)49\.11±\\pm0\.1449\.14±\\pm0\.2949\.05±\\pm0\.2448\.66±\\pm0\.5949\.42±\\pm0\.20MIMIC\-IVDrug Recommendation21\.03±\\pm0\.1221\.15±\\pm0\.0620\.78±\\pm0\.3920\.39±\\pm0\.3221\.08±\\pm0\.08Avg\. Rank \(best\-in\-row\)3\.83 \(0\)2\.50 \(1\)3\.08 \(2\)4\.17 \(0\)1\.42 \(9\)ATHENA achieved the best overall test AUPRC ranking across all search budgets \(Table[4](https://arxiv.org/html/2608.21712#S5.T4)\)\. With only five architecture evaluations, it ranked first in 10 of 12 hospital–task comparisons and second in the remaining two, yielding an average rank of 1\.17\. It ranked first across all six OneFlorida\+ tasks and four of six MIMIC\-IV tasks, including all multilabel tasks at both targets\. The largest gains over the strongest baseline were observed for OneFlorida\+ Phenotype \(12M\), OneFlorida\+ Phenotype \(6M\), and MIMIC\-IV Drug Recommendation\.

This advantage persisted at larger budgets\. At both 20 and 30 evaluations, ATHENA ranked first in 9 of 12 comparisons, with average ranks of 1\.29 and 1\.42, respectively, compared with 3\.17 and 2\.50 for the next\-best baseline\. At budget 30, the AUROC analysis \(Supplementary Table S4\) showed a similar pattern, with ATHENA ranking first in 7 of 12 comparisons \(including one tie with EA\) and achieving the best average rank \(1\.71\)\. Overall, the advantage was consistent across both target health systems\.

Validation trajectories further showed that ATHENA’s gains generally emerged within the first few evaluations rather than only after most of the search budget had been consumed \(Figure[5](https://arxiv.org/html/2608.21712#S5.F5); Supplementary Figure S2\)\. This pattern was most evident for Phenotype \(6M\), Phenotype \(12M\), and Drug Recommendation, for which ATHENA rapidly attained strong validation AUPRC and remained leading or competitive through most of the search\. The margins were most persistent for Phenotype \(12M\) on both targets, whereas competing methods narrowed or occasionally closed the gap on the other tasks as the budget increased\. Mortality, Stay\>\>7d, and Readmission \(3M\) showed closer convergence among methods\. Some LLM\-based searches terminated before 30 evaluations when they could no longer generate new legal, non\-duplicate candidates\. Overall, ATHENA retained most of its early gains, with smaller and task\-dependent improvements from later evaluations, indicating that much of its advantage was established under a relatively small search budget\.

### 5\.5Performance–complexity trade\-offs and selection behavior

The validation Pareto analyses showed that performance was not a monotonic function of model size \(Figure[5](https://arxiv.org/html/2608.21712#S5.F5); Supplementary Figure S3\)\. In the binary tasks, relatively small architectures could be competitive with substantially larger models, while architectures with similar parameter counts often had visibly different AUPRC values\. The multilabel tasks exhibited stepwise improvements across parameter bands, but still showed considerable within\-band variation\. ATHENA evaluated candidates across these bands and repeatedly placed architectures near the empirical Pareto frontier rather than concentrating exclusively at the largest end of the search space\. This pattern was present on both the internal and external targets\.

![Refer to caption](https://arxiv.org/html/2608.21712v1/figs/figure1_search_trajectory_val_auprc_source_15.png)

Figure 4:Search trajectories on the held\-out OneFlorida\+ target for \(a\) Mortality, \(b\) Stay\>\>7d, \(c\) Readmission \(3M\), \(d\) Phenotype \(6M\), \(e\) Phenotype \(12M\), and \(f\) Drug Recommendation\. Lines and shaded bands show the mean and SD, respectively, of the best validation AUPRC attained across five random seeds as a function of the number of evaluated architectures\. Curves may end before the full budget when proposal saturation yields no new legal architecture\.![Refer to caption](https://arxiv.org/html/2608.21712v1/figs/figure2_pareto_source_15.png)

Figure 5:Validation AUPRC versus parameter count on the held\-out OneFlorida\+ target\. Markers identify the search method and the dashed line denotes the empirical Pareto frontier across evaluated architectures\.Table 5:Search behavior at a budget of 30 evaluations, averaged over the six tasks and five random seeds\.*Params*is the size of the selected architecture;*Evals*is the number of distinct architectures actually evaluated\.OneFlorida\+MIMIC\-IVMethodParams \(M\)EvalsParams \(M\)EvalsRandom Search1\.2830\.02\.3430\.0EA1\.3330\.02\.1130\.0GENIUS1\.9023\.62\.4220\.3CoLLM\-NAS1\.2810\.71\.939\.1ATHENA1\.5522\.52\.5819\.6ATHENA’s predictive performance was not explained by systematically selecting the largest models or by using more candidate evaluations \(Table[5](https://arxiv.org/html/2608.21712#S5.T5)\)\. On OneFlorida\+, the architectures selected by ATHENA averaged 1\.55 million parameters, compared with 1\.90 million for GENIUS, despite ATHENA’s substantially better average performance rank\. ATHENA evaluated an average of 22\.5 architectures on OneFlorida\+ and 19\.6 on MIMIC\-IV, below the maximum budget of 30 and slightly fewer than GENIUS on both targets\.

Table 6:Modal architecture selected by each method, reported as embed\_dim/depth/heads/mlp\_ratio, with the number of random seeds selecting that architecture \(out of five\) in parentheses\. For tasktt, letnt,an\_\{t,a\}denote the number of random seeds selecting architectureaa; the task\-level modal selection rate is100×maxa⁡\(nt,a\)/5100\\times\\max\_\{a\}\(n\_\{t,a\}\)/5\.Mean modal architecture selection \(%\)is the average of these rates across the six tasks, i\.e\.16​∑t=16100×maxa⁡\(nt,a\)/5\\frac\{1\}\{6\}\\sum\_\{t=1\}^\{6\}100\\times\\max\_\{a\}\(n\_\{t,a\}\)/5\. A dash \(—\) indicates that all five random seeds selected distinct architectures, corresponding to a task\-level rate of 20%\. Higher values indicate greater convergence toward the same architecture across random seeds and reflect selection consistency, not architecture quality\.TargetTaskRandom SearchEAGENIUSCoLLM\-NASATHENAOneFlorida\+Mortality—64/2/2/2 \(2\)256/2/8/1 \(2\)——OneFlorida\+Stay\>\>7d————128/4/4/1 \(2\)OneFlorida\+Readmission \(3M\)128/1/8/1 \(2\)——128/4/4/2 \(2\)128/2/8/2 \(2\)OneFlorida\+Phenotype \(6M\)—256/1/8/2 \(2\)256/2/4/1 \(2\)—256/1/4/1 \(2\)OneFlorida\+Phenotype \(12M\)—256/1/8/1 \(4\)—128/2/4/2 \(2\)256/1/8/1 \(4\)OneFlorida\+Drug Recommendation256/2/8/2 \(2\)256/2/4/2 \(2\)256/2/8/4 \(3\)256/4/8/2 \(3\)—Mean modal architecture selection \(%\)26\.740\.033\.333\.340\.0MIMIC\-IVMortality————256/4/8/1 \(3\)MIMIC\-IVStay\>\>7d——256/2/4/2 \(2\)—256/4/8/1 \(3\)MIMIC\-IVReadmission \(3M\)———128/4/4/4 \(2\)—MIMIC\-IVPhenotype \(6M\)—256/1/8/1 \(2\)256/2/4/2 \(2\)128/4/2/2 \(2\)256/1/8/1 \(2\)MIMIC\-IVPhenotype \(12M\)——256/2/8/4 \(2\)—256/1/4/1 \(2\)MIMIC\-IVDrug Recommendation—256/1/8/1 \(2\)256/2/8/4 \(2\)128/4/8/8 \(2\)256/1/4/2 \(2\)Mean modal architecture selection \(%\)20\.026\.733\.330\.043\.3Architecture selection was also more reproducible across repeated searches \(Table[6](https://arxiv.org/html/2608.21712#S5.T6)\)\. For each tasktt, we calculated the modal architecture selection rate as the proportion of the five random seeds selecting the most frequently chosen architecture, and averaged this rate across the six tasks:

Mean modal selection \(%\)=16​∑t=16100×maxa⁡\(nt,a\)5,\\text\{Mean modal selection \(\\%\)\}=\\frac\{1\}\{6\}\\sum\_\{t=1\}^\{6\}100\\times\\frac\{\\max\_\{a\}\(n\_\{t,a\}\)\}\{5\},wherent,an\_\{t,a\}is the number of random seeds selecting architectureaafor tasktt\. On OneFlorida\+, ATHENA and EA achieved the highest mean modal architecture selection rate of 40\.0%, compared with 26\.7% for Random Search and 33\.3% for both GENIUS and CoLLM\-NAS\. On MIMIC\-IV, ATHENA achieved the highest rate at 43\.3%, compared with 20\.0%–33\.3% for the baselines\. Higher rates indicate greater convergence toward the same architecture across repeated searches and reflect selection consistency rather than architecture quality\.

### 5\.6Robustness of the cross\-hospital prior

The ablation results separated the contributions of retrieval and architecture\-effect guidance \(Figure[6](https://arxiv.org/html/2608.21712#S5.F6); Supplementary Figure S4\)\. Removing*Layer 2*\(*L1\-only*\) generally retained competitive performance but reduced AUPRC on several tasks, particularly Phenotype \(6M\) and Phenotype \(12M\) on OneFlorida\+ and Mortality and Readmission \(3M\) on MIMIC\-IV\.

![Refer to caption](https://arxiv.org/html/2608.21712v1/figs/figure5_loto_robustness_MIMIC-IV.png)Figure 6:Cross\-hospital prior ablation on the external MIMIC\-IV target\. Bars and error bars show the mean and SD, respectively, of test AUPRC across five random seeds for exact task\-matched retrieval,L1\-onlyretrieval without theLayer 2architecture\-effect prior, LOTO Retrieval excluding exact task matches, Cold Start without either prior layer, and CoLLM\-NAS as the representative baseline\.Removing both prior layers \(Cold Start\) produced the clearest degradation for Phenotype \(6M\) and Phenotype \(12M\) on both targets\.

When exact task\-matched retrieval records were withheld, LOTO Retrieval remained comparable to exact retrieval and was occasionally higher, showing that related\-task source records could provide useful fallback guidance\.

## 6Discussion

In this study, we developed ATHENA, a knowledge\-guided agentic NAS framework that combines weight\-sharing evaluation with reusable cross\-hospital architecture knowledge for Transformer\-based EHR modeling\. Across six clinical prediction tasks and two target health systems, ATHENA achieved the strongest overall AUPRC ranking across search budgets while requiring substantially less computation than independently pretraining each candidate architecture\. ATHENA also produced more consistent architecture selections across repeated searches, and its performance gains were not explained simply by evaluating more candidates or selecting larger models\. These findings suggest that architecture\-search experience accumulated from prior hospital–task settings can be reused to make model development more efficient and reproducible when adapting EHR models across clinical prediction settings\.

Our findings also highlight the complementary roles of the two cross\-hospital priors\. The retrieval prior provides architecture\-level examples from related source settings, whereas the architecture\-effect prior summarizes task\-specific effects of individual design choices across hospitals\. The ablation results support this distinction: removing the architecture\-effect prior reduced performance on several tasks, while removing both priors produced larger declines, particularly for multilabel outcomes\. Notably, LOTO Retrieval remained competitive even when exact task matches were excluded from the source knowledge base\. This result suggests that the transferred information is not limited to reusing architectures previously optimized for the same endpoint; architectural patterns learned from related prediction problems may also provide useful guidance for a new task\. At the same time, the observed task\-specific architecture effects argue against a single universally optimal Transformer configuration for longitudinal EHR data\. Clinical prediction tasks differ in outcome structure, prediction horizon, and the information that must be integrated across a patient’s record, and these differences may favor different model capacities and configurations\. Cross\-hospital transfer of architecture knowledge should therefore complement, rather than replace, adaptation to the target task\.

ATHENA extends existing LLM\-guided NAS approaches that primarily derive search guidance from information generated during the current search process\[[47](https://arxiv.org/html/2608.21712#bib.bib34),[22](https://arxiv.org/html/2608.21712#bib.bib47)\]\. Rather than beginning each search without prior architectural evidence, ATHENA incorporates structured knowledge accumulated from previously evaluated hospital–task settings and updates this guidance using validation performance at the target site\. This distinction is particularly relevant to multi\-institutional EHR research, where models are repeatedly developed or adapted across institutions and clinical endpoints\. Differences in patient populations, coding practices, clinical care patterns, data density, and outcome prevalence can affect both model performance and the architecture best suited to a prediction problem\. Nevertheless, architecture selection is often repeated independently for each new dataset or endpoint\. Performing an extensive architecture search for every hospital–task combination can become computationally burdensome, particularly for pretrained Transformer models\. Reusing evidence from previous architecture evaluations provides a practical middle ground between applying the same architecture across all settings and repeating a computationally intensive search from scratch for every new study\.

The external evaluation on MIMIC\-IV further illustrates the potential value of this approach\. Architecture knowledge derived from the source hospitals remained informative when transferred to a target health system with different cohort characteristics and clinical data distributions\. Importantly, ATHENA did not directly transfer a source architecture as the final model\. Instead, source knowledge was used to guide candidate generation, while architecture selection continued to depend on validation performance at the target site\. This separation between transferred guidance and local evaluation is important for multi\-institutional clinical modeling because an architecture that performs well in one health system may not remain optimal in another because of differences in patient populations, clinical care patterns, and outcome distributions\. Architecture transfer may therefore be most useful when prior evidence narrows and informs the search while target\-site data retain the final role in model selection\.

Several limitations should be considered\. First, the evaluation included two target health systems and six downstream prediction tasks\. Additional institutions, clinical specialties, coding systems, patient populations, and prediction horizons will be needed to determine how broadly the observed transfer patterns generalize\. Although MIMIC\-IV provided an external target health system, it represents a single academic medical center and does not capture the heterogeneity of potential real\-world deployment environments\. Second, the search space was limited to Transformer embedding dimension, depth, number of attention heads, and MLP expansion ratio\. Other design choices, including temporal representations, attention mechanisms, prediction heads, tokenization strategies, and training hyperparameters, were held fixed and may interact with the architectural components examined here\. Third, although supernet\-based evaluation generally preserved architecture rankings, ranking fidelity was lower for some binary outcomes, particularly Readmission \(3M\), indicating that the accuracy of weight\-inherited subnetwork evaluation may vary by prediction task\. Fourth, the reported target\-search cost does not include the upfront computation required to construct the source\-site architecture knowledge base\. The efficiency advantage of ATHENA therefore depends on reuse: its potential benefit increases when accumulated architecture evidence can support multiple subsequent modeling studies rather than a single search\.

In conclusion, ATHENA demonstrates a framework for reusing architecture\-search knowledge across hospitals and clinical prediction tasks while preserving local model selection\. By combining retrieved architecture examples, architecture\-effect guidance, target\-site validation, and weight\-sharing evaluation, ATHENA identified strong Transformer configurations under limited search budgets while retaining task\- and site\-specific adaptation\. More broadly, this approach reframes architecture search as knowledge that can accumulate across EHR modeling studies rather than a computational process that must be repeated independently for each new prediction problem\. Future work should evaluate this strategy across more diverse health systems and model families, expand the range of transferable architectural and training information, and examine whether more efficient and reproducible architecture development can support robust external validation and prospective evaluation of EHR prediction models\.

## Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper\.

## Data availability

The patient\-level OneFlorida\+ data used in this study are not publicly available because of privacy, institutional, and data\-use restrictions\. Researchers may request access through the OneFlorida\+ Clinical Research Network Front Door or Coordinating Center \([https://onefl\.net/front\-door/](https://onefl.net/front-door/)\); access is contingent on eligibility, network and institutional review, applicable data\-use agreements, and ethical or institutional review board approvals\. MIMIC\-IV is a de\-identified, credentialed\-access resource available through PhysioNet \([https://physionet\.org/](https://physionet.org/)\)\. Access requires PhysioNet credentialing, completion of the required research training, and acceptance of the applicable data\-use agreement\. The study\-specific analytic datasets derived from OneFlorida\+ and MIMIC\-IV cannot be redistributed by the authors; eligible researchers must obtain the source data independently under the respective data\-governance requirements\.

## References

- \[1\]M\. Bal\-Ghaoui and M\. Tiouti\(2025\)MetaLLMix: an xai aided llm\-meta\-learning based approach for hyper\-parameters optimization\.arXiv preprint arXiv:2509\.09387\.Cited by:[§2\.2](https://arxiv.org/html/2608.21712#S2.SS2.p2.1)\.
- \[2\]A\. Chen, D\. Dohan, and D\. So\(2023\)Evoprompting: language models for code\-level neural architecture search\.Advances in neural information processing systems36,pp\. 7787–7817\.Cited by:[§2\.3](https://arxiv.org/html/2608.21712#S2.SS3.p1.1)\.
- \[3\]M\. Chen, H\. Peng, J\. Fu, and H\. Ling\(2021\)Autoformer: searching transformers for visual recognition\.InProceedings of the IEEE/CVF international conference on computer vision,pp\. 12270–12280\.Cited by:[§1](https://arxiv.org/html/2608.21712#S1.p5.1),[§3\.4](https://arxiv.org/html/2608.21712#S3.SS4.p1.1)\.
- \[4\]T\. Chen and C\. Guestrin\(2016\)Xgboost: a scalable tree boosting system\.InProceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining,pp\. 785–794\.Cited by:[§3\.5\.2](https://arxiv.org/html/2608.21712#S3.SS5.SSS2.p1.2)\.
- \[5\]E\. Choi, M\. T\. Bahadori, E\. Searles, C\. Coffey, M\. Thompson, J\. Bost, J\. Tejedor\-Sojo, and J\. Sun\(2016\)Multi\-layer representation learning for medical concepts\.InProc\. 22nd ACM SIGKDD Int\. Conf\. Knowledge Discovery and Data Mining \(KDD\),pp\. 1495–1504\.Cited by:[§3\.2](https://arxiv.org/html/2608.21712#S3.SS2.p1.2)\.
- \[6\]S\. Cui, J\. Wang, Y\. Zhong, H\. Liu, T\. Wang, and F\. Ma\(2024\)Automated fusion of multimodal electronic health records for better medical predictions\.InProceedings of the… SIAM International Conference on Data Mining\. SIAM International Conference on Data Mining,Vol\.2024,pp\. 361\.Cited by:[§2\.2](https://arxiv.org/html/2608.21712#S2.SS2.p2.1)\.
- \[7\]J\. Devlin, M\. Chang, K\. Lee, and K\. Toutanova\(2019\)Bert: pre\-training of deep bidirectional transformers for language understanding\.InProceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 \(long and short papers\),pp\. 4171–4186\.Cited by:[§3\.4](https://arxiv.org/html/2608.21712#S3.SS4.p3.1)\.
- \[8\]T\. Elsken, J\. H\. Metzen, and F\. Hutter\(2019\)Neural architecture search: a survey\.Journal of Machine Learning Research20\(55\),pp\. 1–21\.Cited by:[§1](https://arxiv.org/html/2608.21712#S1.p3.1)\.
- \[9\]R\. L\. Fleurence, L\. H\. Curtis, R\. M\. Califf, R\. Platt, J\. V\. Selby, and J\. S\. Brown\(2014\)Launching pcornet, a national patient\-centered clinical research network\.Journal of the American Medical Informatics Association21\(4\),pp\. 578–582\.Cited by:[§4\.1](https://arxiv.org/html/2608.21712#S4.SS1.p1.1)\.
- \[10\]R\. L\. Fleurence, L\. H\. Curtis, R\. M\. Califf, R\. Platt, J\. V\. Selby, and J\. S\. Brown\(2014\)Launching pcornet, a national patient\-centered clinical research network\.Journal of the American Medical Informatics Association21\(4\),pp\. 578–582\.External Links:[Document](https://dx.doi.org/10.1136/amiajnl-2014-002747),[Link](https://doi.org/10.1136/amiajnl-2014-002747)Cited by:[§4\.1](https://arxiv.org/html/2608.21712#S4.SS1.p1.1)\.
- \[11\]J\. Gao, H\. Xu, H\. Shi, X\. Ren, P\. L\. Yu, X\. Liang, X\. Jiang, and Z\. Li\(2022\)Autobert\-zero: evolving bert backbone from scratch\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.36,pp\. 10663–10671\.Cited by:[§2\.2](https://arxiv.org/html/2608.21712#S2.SS2.p1.1)\.
- \[12\]Z\. Guo, X\. Zhang, H\. Mu, W\. Heng, Z\. Liu, Y\. Wei, and J\. Sun\(2020\)Single path one\-shot neural architecture search with uniform sampling\.InEuropean conference on computer vision,pp\. 544–560\.Cited by:[§1](https://arxiv.org/html/2608.21712#S1.p3.1),[§2\.2](https://arxiv.org/html/2608.21712#S2.SS2.p1.1)\.
- \[13\]W\. R\. Hoganet al\.\(2022\)The oneflorida data trust: a centralized, translational research data infrastructure of statewide scope\.Journal of the American Medical Informatics Association29\(4\),pp\. 686–693\.External Links:[Document](https://dx.doi.org/10.1093/jamia/ocab221)Cited by:[§4\.1](https://arxiv.org/html/2608.21712#S4.SS1.p1.1)\.
- \[14\]Z\. Ji, G\. Zhu, C\. Yuan, and Y\. Huang\(2025\)RZ\-nas: enhancing llm\-guided neural architecture search via reflective zero\-cost strategy\.\.InICML,Cited by:[§1](https://arxiv.org/html/2608.21712#S1.p4.1)\.
- \[15\]A\. E\. Johnson, L\. Bulgarelli, L\. Shen, A\. Gayles, A\. Shammout, S\. Horng, T\. J\. Pollard, S\. Hao, B\. Moody, B\. Gow,et al\.\(2023\)MIMIC\-iv, a freely accessible electronic health record dataset\.Scientific data10\(1\),pp\. 1\.Cited by:[§4\.1](https://arxiv.org/html/2608.21712#S4.SS1.p1.1)\.
- \[16\]K\. Kandasamy, W\. Neiswanger, J\. Schneider, B\. Poczos, and E\. P\. Xing\(2018\)Neural architecture search with bayesian optimisation and optimal transport\.Advances in neural information processing systems31\.Cited by:[§1](https://arxiv.org/html/2608.21712#S1.p3.1)\.
- \[17\]M\. Kirchler, M\. Ferro, V\. Lorenzini, R\. P\. van de Water, F\. G\. A\. 3, C\. Lippert, and A\. Ganna\(2026\)Large language models improve transferability of electronic health record\-based predictions across countries and coding systems\.npj Digital Medicine9\(1\),pp\. 177\.Cited by:[§1](https://arxiv.org/html/2608.21712#S1.p2.1)\.
- \[18\]Z\. Kraljevic, D\. Bean, A\. Shek, R\. Bendayan, H\. Hemingway, J\. A\. Yeung, A\. Deng, A\. Balston, J\. Ross, E\. Idowu,et al\.\(2024\)Foresight—a generative pretrained transformer for modelling of patient timelines using electronic health records: a retrospective modelling study\.The Lancet Digital Health6\(4\),pp\. e281–e290\.Cited by:[§2\.1](https://arxiv.org/html/2608.21712#S2.SS1.p2.1)\.
- \[19\]D\. Li, Z\. Yao, Q\. Xu, M\. Liang, L\. Li, Z\. Xu, and M\. Liu\(2026\)DT\-behrt: disease trajectory\-aware transformer for interpretable patient representation learning\.arXiv preprint arXiv:2603\.10180\.Cited by:[§1](https://arxiv.org/html/2608.21712#S1.p1.1)\.
- \[20\]Y\. Li, M\. Mamouei, G\. Salimi\-Khorshidi, S\. Rao, A\. Hassaine, D\. Canoy, T\. Lukasiewicz, and K\. Rahimi\(2022\)Hi\-behrt: hierarchical transformer\-based model for accurate prediction of clinical events using multimodal longitudinal electronic health records\.IEEE journal of biomedical and health informatics27\(2\),pp\. 1106–1117\.Cited by:[§2\.1](https://arxiv.org/html/2608.21712#S2.SS1.p2.1)\.
- \[21\]Y\. Li, S\. Rao, J\. R\. A\. Solares, A\. Hassaine, R\. Ramakrishnan, D\. Canoy, Y\. Zhu, K\. Rahimi, and G\. Salimi\-Khorshidi\(2020\)BEHRT: transformer for electronic health records\.Scientific reports10\(1\),pp\. 7155\.Cited by:[§2\.1](https://arxiv.org/html/2608.21712#S2.SS1.p1.1)\.
- \[22\]Z\. Li, Z\. Lin, and Y\. Wang\(2026\)CoLLM\-nas: collaborative large language models for efficient knowledge\-guided neural architecture search\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 3273–3282\.Cited by:[§1](https://arxiv.org/html/2608.21712#S1.p4.1),[4th item](https://arxiv.org/html/2608.21712#S4.I1.i4.p1.1),[§6](https://arxiv.org/html/2608.21712#S6.p3.1)\.
- \[23\]H\. Liu, K\. Simonyan, and Y\. Yang\(2018\)Darts: differentiable architecture search\.arXiv preprint arXiv:1806\.09055\.Cited by:[§1](https://arxiv.org/html/2608.21712#S1.p3.1),[§2\.2](https://arxiv.org/html/2608.21712#S2.SS2.p1.1)\.
- \[24\]S\. M\. Lundberg, G\. Erion, H\. Chen, A\. DeGrave, J\. M\. Prutkin, B\. Nair, R\. Katz, J\. Himmelfarb, N\. Bansal, and S\. Lee\(2020\)From local explanations to global understanding with explainable ai for trees\.Nature Machine Intelligence2\(1\),pp\. 2522–5839\.Cited by:[§3\.5\.2](https://arxiv.org/html/2608.21712#S3.SS5.SSS2.p1.3)\.
- \[25\]M\. U\. Nasir, S\. Earle, J\. Togelius, S\. James, and C\. Cleghorn\(2024\)Llmatic: neural architecture search via large language models and quality diversity optimization\.Inproceedings of the Genetic and Evolutionary Computation Conference,pp\. 1110–1118\.Cited by:[§2\.3](https://arxiv.org/html/2608.21712#S2.SS3.p1.1)\.
- \[26\]C\. Pang, X\. Jiang, K\. S\. Kalluri, M\. Spotnitz, R\. Chen, A\. Perotte, and K\. Natarajan\(2021\)CEHR\-bert: incorporating temporal information from structured ehr data to improve prediction tasks\.InMachine Learning for Health,pp\. 239–260\.Cited by:[§2\.1](https://arxiv.org/html/2608.21712#S2.SS1.p2.1)\.
- \[27\]R\. Poulain and R\. Beheshti\(2024\)Graph transformers on ehrs: better representation improves downstream performance\.InThe Twelfth International Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2608.21712#S1.p1.1)\.
- \[28\]L\. Rasmy, Y\. Xiang, Z\. Xie, C\. Tao, and D\. Zhi\(2021\)Med\-bert: pretrained contextualized embeddings on large\-scale structured electronic health records for disease prediction\.NPJ digital medicine4\(1\),pp\. 86\.Cited by:[§1](https://arxiv.org/html/2608.21712#S1.p1.1),[§1](https://arxiv.org/html/2608.21712#S1.p3.1),[§2\.1](https://arxiv.org/html/2608.21712#S2.SS1.p1.1),[§3\.2](https://arxiv.org/html/2608.21712#S3.SS2.p1.2)\.
- \[29\]E\. Real, A\. Aggarwal, Y\. Huang, and Q\. V\. Le\(2019\)Regularized evolution for image classifier architecture search\.InProceedings of the aaai conference on artificial intelligence,Vol\.33,pp\. 4780–4789\.Cited by:[§1](https://arxiv.org/html/2608.21712#S1.p3.1),[2nd item](https://arxiv.org/html/2608.21712#S4.I1.i2.p1.1)\.
- \[30\]W\. Ren, J\. Zhu, Z\. Liu, T\. Zhao, and V\. Honavar\(2025\)A comprehensive survey of electronic health record modeling: from deep learning approaches to large language models\.arXiv preprint arXiv:2507\.12774\.Cited by:[§1](https://arxiv.org/html/2608.21712#S1.p1.1)\.
- \[31\]S\. Salmani Pour Avval, N\. D\. Eskue, R\. M\. Groves, and V\. Yaghoubi\(2025\)Systematic review on neural architecture search: ssp avval et al…\.Artificial Intelligence Review58\(3\),pp\. 73\.Cited by:[§1](https://arxiv.org/html/2608.21712#S1.p3.1)\.
- \[32\]D\. So, Q\. Le, and C\. Liang\(2019\)The evolved transformer\.InInternational conference on machine learning,pp\. 5877–5886\.Cited by:[§2\.2](https://arxiv.org/html/2608.21712#S2.SS2.p1.1)\.
- \[33\]D\. R\. So, W\. Manke, H\. Liu, Z\. Dai, N\. Shazeer, and Q\. V\. Le\(2022\)Primer: searching for efficient transformers for language modeling, 2022\.URL https://arxiv\. org/abs/2109\.08668\.Cited by:[§2\.2](https://arxiv.org/html/2608.21712#S2.SS2.p1.1)\.
- \[34\]X\. Su, Q\. Mao, Z\. Wu, X\. Lin, S\. You, Y\. Liao, and C\. Xu\(2025\)Large language models driven neural architecture search for universal and lightweight disease diagnosis on histopathology slide images\.npj Digital Medicine8\(1\),pp\. 682\.Cited by:[§1](https://arxiv.org/html/2608.21712#S1.p4.1),[§2\.2](https://arxiv.org/html/2608.21712#S2.SS2.p2.1),[§2\.3](https://arxiv.org/html/2608.21712#S2.SS3.p2.1)\.
- \[35\]A\. Vaswani, N\. Shazeer, N\. Parmar, J\. Uszkoreit, L\. Jones, A\. N\. Gomez, Ł\. Kaiser, and I\. Polosukhin\(2017\)Attention is all you need\.Advances in neural information processing systems30\.Cited by:[§2\.1](https://arxiv.org/html/2608.21712#S2.SS1.p1.1)\.
- \[36\]H\. Wang, Z\. Wu, Z\. Liu, H\. Cai, L\. Zhu, C\. Gan, and S\. Han\(2020\)Hat: hardware\-aware transformers for efficient natural language processing\.InProceedings of the 58th annual meeting of the association for computational linguistics,pp\. 7675–7688\.Cited by:[§2\.2](https://arxiv.org/html/2608.21712#S2.SS2.p1.1)\.
- \[37\]J\. Wang, J\. Luo, M\. Ye, X\. Wang, Y\. Zhong, A\. Chang, G\. Huang, Z\. Yin, C\. Xiao, J\. Sun,et al\.\(2024\)Recent advances in predictive modeling with electronic health records\.InIJCAI: proceedings of the conference,Vol\.2024,pp\. 8272\.Cited by:[§1](https://arxiv.org/html/2608.21712#S1.p1.1)\.
- \[38\]C\. White, M\. Safari, R\. Sukthanker, B\. Ru, T\. Elsken, A\. Zela, D\. Dey, and F\. Hutter\(2023\)Neural architecture search: insights from 1000 papers\.arXiv preprint arXiv:2301\.08727\.Cited by:[§2\.2](https://arxiv.org/html/2608.21712#S2.SS2.p1.1)\.
- \[39\]M\. Wornow, R\. Thapa, E\. Steinberg, J\. Fries, and N\. ShahEhrshot: an ehr benchmark for few\-shot evaluation of foundation models \(2023\)\.URL https://arxiv\. org/abs/2307\.020282307\.Cited by:[§2\.1](https://arxiv.org/html/2608.21712#S2.SS1.p2.1)\.
- \[40\]J\. Xu, X\. Tan, R\. Luo, K\. Song, J\. Li, T\. Qin, and T\. Liu\(2021\)NAS\-bert: task\-agnostic and adaptive\-size bert compression with neural architecture search\.InProceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining,pp\. 1933–1943\.Cited by:[§2\.2](https://arxiv.org/html/2608.21712#S2.SS2.p1.1)\.
- \[41\]R\. Xu, M\. K\. Ali, J\. C\. Ho, and C\. Yang\(2023\)Hypergraph transformers for ehr\-based clinical predictions\.AMIA Summits on Translational Science Proceedings2023,pp\. 582\.Cited by:[§4\.1](https://arxiv.org/html/2608.21712#S4.SS1.p4.1)\.
- \[42\]Z\. Xu, D\. R\. So, and A\. M\. Dai\(2021\)Mufasa: multimodal fusion architecture search for electronic health records\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.35,pp\. 10532–10540\.Cited by:[§2\.2](https://arxiv.org/html/2608.21712#S2.SS2.p2.1)\.
- \[43\]Z\. Yang, A\. Mitra, W\. Liu, D\. Berlowitz, and H\. Yu\(2023\)TransformEHR: transformer\-based encoder\-decoder generative model to enhance prediction of disease outcomes using electronic health records\.Nature communications14\(1\),pp\. 7857\.Cited by:[§1](https://arxiv.org/html/2608.21712#S1.p1.1),[§2\.1](https://arxiv.org/html/2608.21712#S2.SS1.p2.1)\.
- \[44\]W\. Yao, K\. Yin, W\. K\. Cheung, J\. Liu, and J\. Qin\(2024\)Drfuse: learning disentangled representation for clinical multi\-modal fusion with missing modality and modal inconsistency\.InProceedings of the AAAI conference on artificial intelligence,Vol\.38,pp\. 16416–16424\.Cited by:[§4\.1](https://arxiv.org/html/2608.21712#S4.SS1.p4.1)\.
- \[45\]C\. Yu, X\. Liu, Y\. Wang, Y\. Liu, W\. Feng, D\. Xiong, C\. Tang, and J\. Lv\(2023\)GPT\-nas: evolutionary neural architecture search with the generative pre\-trained model\.arXiv preprint arXiv:2305\.05351\.Cited by:[§2\.3](https://arxiv.org/html/2608.21712#S2.SS3.p1.1)\.
- \[46\]T\. Zhang, M\. Chen, and A\. A\. T\. Bui\(2020\)Diagnostic prediction with sequence\-of\-sets representation learning for clinical events\.InProc\. Int\. Conf\. Artificial Intelligence in Medicine \(AIME\),LNAI, Vol\.12299,pp\. 348–358\.Cited by:[§3\.2](https://arxiv.org/html/2608.21712#S3.SS2.p1.2)\.
- \[47\]M\. Zheng, X\. Su, S\. You, F\. Wang, C\. Qian, C\. Xu, and S\. Albanie\(2023\)Can gpt\-4 perform neural architecture search?\.arXiv preprint arXiv:2304\.10970\.Cited by:[§1](https://arxiv.org/html/2608.21712#S1.p4.1),[§2\.3](https://arxiv.org/html/2608.21712#S2.SS3.p1.1),[3rd item](https://arxiv.org/html/2608.21712#S4.I1.i3.p1.1),[§6](https://arxiv.org/html/2608.21712#S6.p3.1)\.
- \[48\]B\. Zoph and Q\. V\. Le\(2016\)Neural architecture search with reinforcement learning\.arXiv preprint arXiv:1611\.01578\.Cited by:[§1](https://arxiv.org/html/2608.21712#S1.p3.1)\.