DoctorAgents: an agentic framework to iteratively refine AutoML pipeline for small clinical temporal data

# DoctorAgents: Iterative Agentic Refinement for Small Clinical Temporal Data
Source: [https://arxiv.org/html/2608.05375](https://arxiv.org/html/2608.05375)
Ruilin Wang1,2, Bo\-Hong Wang1,2, Elizabeth Kourbatski1,2, Jun Bai1,2, Hegang Chen1,2, Ziyang Song1,2, Gilles Boire4,5, Marie Hudson3,\*, Yue Li1,2,\*1School of Computer Science, McGill University, Montreal, Canada 2Mila – Quebec AI Institute, Montreal, Canada 3Division of Rheumatology, Department of Medicine, McGill University, Montreal, Canada 4Centre intégré universitaire de santé et de services sociaux de l’Estrie – Centre hospitalier universitaire de Sherbrooke \(CIUSSSE\-CHUS\), Sherbrooke, Canada 5Division of Rheumatology, Faculty of Medicine and Health Sciences, University of Sherbrooke, Sherbrooke, Canada

###### Abstract\.

Clinical machine learning \(ML\) has the potential to support high\-stakes medical decision\-making, but reliable deployment is often constrained by scarce, heterogeneous, and temporal complexity\. Developing effective ML pipelines for such data remains time\-consuming and error\-prone, while existing automated machine learning \(AutoML\) systems only partially address this challenge because they largely rely on brute\-force search over predefined spaces and lack explicit reasoning and memory\. We therefore reformulate AutoML for small clinical data from exhaustive search to reasoning\-driven refinement\. We proposeDoctorAgents, an agentic AI framework that autonomously constructs and optimizes end\-to\-end ML pipelines through specialized large language model \(LLM\) agents for generation, validation, and refinement\.DoctorAgentsbackpropagates natural\-language feedback through textual gradient descent to perform targeted updates without exhaustive search\. Experiments across diverse clinical tasks show thatDoctorAgentsconsistently outperforms established AutoML baselines while producing more interpretable task\-specific representations\.

Agentic AI, automated machine learning, small clinical data, LLMs

\*\*footnotetext:Corresponding authors: yueli@cs\.mcgill\.ca and marie\.hudson@mcgill\.ca## 1\.Introduction

Existing clinical prediction tasks are often built from small cohorts, such as rare\-disease registries, single\-center studies, and narrowly defined treatment populations\(Van Smedenet al\.,[2019](https://arxiv.org/html/2608.05375#bib.bib39); Rileyet al\.,[2019](https://arxiv.org/html/2608.05375#bib.bib40)\)\. Unlike standard tabular benchmarks, these datasets are often high\-dimensional, sparse, and temporally irregular\. More specifically, temporal clinical records are not simply static tables augmented with timestamps but asynchronous and incomplete observations of an evolving patient state where patients may be observed at different frequencies and over substantially different durations, such that the available temporal resolution varies across individuals and variables\(Hornet al\.,[2020](https://arxiv.org/html/2608.05375#bib.bib58); Shukla and Marlin,[2021](https://arxiv.org/html/2608.05375#bib.bib57)\)\. The same clinical value may therefore carry different predictive meaning depending on its observation time, recency relative to the prediction time, and the trajectory preceding it\.

Moreover, the observed record reflects both the underlying disease process and the clinical observation process, making it difficult to distinguish true physiological change from variation induced by monitoring and care delivery\. Naive aggregation into static patient vectors may fail to preserve temporal ordering, rates of change, persistence, and duration\-related signals that are central to clinical progression\(Xieet al\.,[2022](https://arxiv.org/html/2608.05375#bib.bib56)\)\. Conversely, modeling the raw sequence requires careful alignment to task\-specific index times and observation windows to avoid incorporating information unavailable at prediction time\(Shukla and Marlin,[2019](https://arxiv.org/html/2608.05375#bib.bib60)\)\. Clinical records are also shaped by care processes: measurement timing, frequency, recency, and missingness may reflect both patient state and clinical attention\(Cheet al\.,[2018](https://arxiv.org/html/2608.05375#bib.bib43)\)\.

Reliable prediction therefore requires more than selecting a classifier\. Because the relevant temporal structure varies with the clinical endpoint, observation window, and prediction horizon, effective pipelines require leakage\-safe, task\-specific patient representations that capture temporal changes, variability, recency, and missingness\(Borisovet al\.,[2022](https://arxiv.org/html/2608.05375#bib.bib32)\)\. Constructing such robust end\-to\-end pipelines remains labor\-intensive and requires substantial statistical, computational, and clinical expertise\(Collinset al\.,[2024](https://arxiv.org/html/2608.05375#bib.bib46)\)\.

To alleviate this burden,*automated ML \(AutoML\)*has emerged to automate pipeline construction, including data preprocessing, model selection, and hyperparameter searching\. Early systems typically relied on predefined search spaces and black\-box optimization, including Bayesian optimization, evolutionary strategies, and bandit\-based methods\(Hutteret al\.,[2011](https://arxiv.org/html/2608.05375#bib.bib6); Feureret al\.,[2015](https://arxiv.org/html/2608.05375#bib.bib7); Thorntonet al\.,[2013](https://arxiv.org/html/2608.05375#bib.bib8); Golovinet al\.,[2017](https://arxiv.org/html/2608.05375#bib.bib9)\)\. Although empirically effective, they often require substantial compute, provide limited insight into*why*a pipeline works, and remain less suitable for small datasets, rapid prototyping, or domain\-specific tasks with subtle statistical pitfalls\.

Recent advances in*large language models \(LLMs\)*have enabled AutoML systems that synthesize executable ML pipelines from natural language specifications\(Wanget al\.,[2024b](https://arxiv.org/html/2608.05375#bib.bib27); Brownet al\.,[2020](https://arxiv.org/html/2608.05375#bib.bib21); OpenAI,[2023](https://arxiv.org/html/2608.05375#bib.bib22)\)\. Beyond static code generation, LLMs can perform data analysis, scientific reasoning, tool use, and iterative self\-revision, allowing them to act as autonomous agents\(Chenet al\.,[2021](https://arxiv.org/html/2608.05375#bib.bib10); Li and others,[2022](https://arxiv.org/html/2608.05375#bib.bib11); Wanget al\.,[2023](https://arxiv.org/html/2608.05375#bib.bib12)\)\. Recent work on*agentic AI*further shows that decomposing complex tasks into collaborative roles, such as planning, execution, critique, and optimization, improves reasoning depth and efficiency\(Wanget al\.,[2024a](https://arxiv.org/html/2608.05375#bib.bib34); Xiet al\.,[2025](https://arxiv.org/html/2608.05375#bib.bib35); Acharyaet al\.,[2025](https://arxiv.org/html/2608.05375#bib.bib36); Parket al\.,[2023](https://arxiv.org/html/2608.05375#bib.bib18); Madaanet al\.,[2023](https://arxiv.org/html/2608.05375#bib.bib19)\)\. These advances have motivated LLMs as autonomous ML engineers\. For instance, AutoML\-Agent\(Triratet al\.,[2025](https://arxiv.org/html/2608.05375#bib.bib13)\)and ERA\(Aygünet al\.,[2026](https://arxiv.org/html/2608.05375#bib.bib37)\)generate, execute, and iteratively refine ML pipelines toward user\-defined objectives\. Related works also explored tool\-augmented LLMs for data science workflows and notebook automation\(Zhouet al\.,[2023](https://arxiv.org/html/2608.05375#bib.bib14); Schicket al\.,[2023](https://arxiv.org/html/2608.05375#bib.bib15)\), including systems such as SciToolAgent\(Dinget al\.,[2025](https://arxiv.org/html/2608.05375#bib.bib24)\), which orchestrates domain\-specific tools via a curated scientific knowledge graph but focuses on tool selection rather than end\-to\-end ML pipeline optimization\.

Despite these advances, existing LLM\-based AutoML systems remain constrained by*trial\-and\-error*regime, limited persistent memory, weak reasoning over data properties such as leakage risk and evaluation validity, and reliance on wholesale regeneration rather than*targeted, incremental refinement*\. These limitations lead to unstable optimization, inefficient compute use, repetitive errors, and shallow exploration, echoing broader challenges in agentic systems with limited structured memory and feedback propagation\(Shinn and others,[2023](https://arxiv.org/html/2608.05375#bib.bib16); Yao and others,[2023](https://arxiv.org/html/2608.05375#bib.bib17)\)\. A more deliberate paradigm is therefore needed, treating pipeline development as a stateful, reasoning\-driven process rather than a sequence of independent generations\.

We introduceDoctorAgents, a multi\-agent framework for autonomous ML pipeline optimization that replaces brute\-force search with reasoning\-driven iteration\. Briefly,DoctorAgentsemploys a team of specialized LLM agents to analyze data, generate and validate executable pipelines, analyze failures, and refine solutions using structured memory of prior changes and outcomes\. As a result, the generated ML pipeline are grounded in feature semantics and rigor\. During the iterative pipeline refinement, we use adapted Textual Gradient Descent \(TGD\)\(Yuksekgonulet al\.,[2025](https://arxiv.org/html/2608.05375#bib.bib1)\)to convert natural\-language feedback into localized code updates without regenerating entire pipelines\. We further devise two sets of domain\-specialized agents that decompose optimization into dedicated data preprocessing and model development agents, enabling deeper exploration of task\-specific temporal representations while preserving explicit, editable code\. Beyond predictive performance, we assess generated features for technical validity, semantic consistency, temporal appropriateness, and clinical justification\. Across diverse clinical tasks,DoctorAgentsachieves stronger performance, stability, robustness, and feature quality than existing AutoML and agentic baselines\. In summary, our contributions include

- •a reasoning\-driven agentic AutoML framework for small clinical temporal datasets, combining dataset reasoning, execution validation, memory, and refinement,
- •a memory\-aware optimization that enables long\-term learning and prevents repeated failures,
- •a design of domain\-specialized data preprocessing for irregularly observed temporal clinical data, enabling richer task\-specific patient representations,
- •automatically composing clinically meaningful variable from temporal data, and
- •extensive empirical validation using MIMIC\-IV and in\-house clinical data, demonstrating robustness, efficiency, and interpretability beyond existing methods\.

## 2\.Related Work

##### Temporal Clinical Prediction Methods\.

Existing methods for temporal clinical prediction typically rely on standardized preprocessing pipelines or task\-specific temporal architectures\. MIMIC\-Extract\(Wanget al\.,[2020](https://arxiv.org/html/2608.05375#bib.bib49)\)supports reproducible ICU time\-series preprocessing, while ML models such as DeepCare\(Phamet al\.,[2016](https://arxiv.org/html/2608.05375#bib.bib54)\)and TrajGPT\(Songet al\.,[2026](https://arxiv.org/html/2608.05375#bib.bib53)\)encode visit importance, irregular intervals, and sparse observations through specialized LSTM or transformer\-attention layers, respectively\. Although effective, these methods usually assume a fixed representation strategy or model family, leaving limited flexibility to adapt preprocessing, feature construction, and model selection jointly across heterogeneous small clinical datasets\.

##### Foundation Models for Small Tabular Data\.

TabPFN\(Hollmannet al\.,[2025](https://arxiv.org/html/2608.05375#bib.bib4)\)is a pretrained foundation model that performs well on small tabular classification problems with minimal task\-specific tuning\. However, its monolithic prediction interface provides limited control over preprocessing, feature construction, evaluation design, and task\-specific constraints\. This limits its suitability for small clinical datasets, where leakage prevention, temporal aggregation, missingness handling, and interpretable patient\-level representations often require explicit pipeline decisions\.

##### LLM\-Based AutoML Agents\.

AutoML\-Agent\(Triratet al\.,[2025](https://arxiv.org/html/2608.05375#bib.bib13)\)generates and iteratively improves pipelines using LLMs, while ERA\(Aygünet al\.,[2026](https://arxiv.org/html/2608.05375#bib.bib37)\)combines LLM generation with tree\-based search to explore a broader solution space\. However, these systems still rely on repeated pipeline generation, with limited persistent memory and weak propagation of evaluation feedback into localized code changes\. As a result, optimization can remain unstable and sample\-inefficient under limited budgets, especially for small clinical datasets\.

## 3\.DoctorAgents

![Refer to caption](https://arxiv.org/html/2608.05375v1/Main_Figures/Overview.png)Figure 1\.Overview ofDoctorAgents\.\(a\) Initializationproduces a solid starting pipeline from user metadata and profiled data statistics using the provided tool environment\.\(b\) Executionvalidates the given code and returns performance results\.\(c\) Memoryrecords long\-term information across optimization iterations\.\(d\) Refinementiteratively improves the pipeline through evaluation\-driven updates\. The final output to the user is an optimized pipeline with the best observed performance\.DoctorAgents framework diagramWe first describe the overall multi\-agent framework ofDoctorAgentswith details on how the designated agents handle task initialization, validation, memory log, and pipeline refinement \(Fig\.[1](https://arxiv.org/html/2608.05375#S3.F1)\)\. We then describe the domain specialized \(DS\) agents in Sec\.[3\.7](https://arxiv.org/html/2608.05375#S3.SS7)\.

### 3\.1\.Agent Designs

Each agent is an LLM instructed to complete a specific task:

\\noindentparagraph

Consultant \(𝔸c\\mathbb\{A\}\_\{c\}\) acts as the high\-level analytical expert, responsible for analyzing and transforming raw inputs into a predefined structured task specification with domain knowledge to guide downstream pipeline synthesis\.

\\noindentparagraph

Generator \(𝔸g\\mathbb\{A\}\_\{g\}\) translates the Consultant’s outputs into an initial ML pipeline code, including data preprocessing, model selection, training, and evaluation\. The resulting implementation forms the starting point for the optimization workflow\.

\\noindentparagraph

Repairer \(𝔸r\\mathbb\{A\}\_\{r\}\) restores executability when the current pipeline fails and ensures minimal correction on the code for re\-validation until success before proceeding to the Logger\.

\\noindentparagraph

Logger \(𝔸l\\mathbb\{A\}\_\{l\}\) records long\-term memory for the refinement process\. The Logger records inter\-step notes that capture meaningful changes and manages memory context through summarization and cleaning at every checkpoint to maintain effective memory log\.

\\noindentparagraph

Evaluator \(𝔸e\\mathbb\{A\}\_\{e\}\) evaluates the quality of the pipeline and proposes improvement strategies\. It produces natural language evaluation, which serve as the basis for updating the pipeline in the downstream backprop\-optimization process\.

\\noindentparagraph

Optimizer \(𝔸o\\mathbb\{A\}\_\{o\}\) converts the backpropagated evaluator feedback into concrete code modifications while preserving expected pipeline output format to maintain iterative stability\. The resulting pipeline implementation is then forwarded to the next optimization loop\.

Algorithm 1Overall Workflow ofDoctorAgentsInitialization:Consultant𝔸c\\mathbb\{A\}\_\{c\}, Generator𝔸g\\mathbb\{A\}\_\{g\}, Logger𝔸l\\mathbb\{A\}\_\{l\}, Evaluator𝔸e\\mathbb\{A\}\_\{e\}, Optimizer𝔸o\\mathbb\{A\}\_\{o\}, and system stepSS

Input:Dataset metadataMMand max optimization stepTT

Output:Optimized PipelinePP

1:if

M≠∅M\\neq\\varnothingthen

2:

I←𝔸c​\(M\)I\\leftarrow\\mathbb\{A\}\_\{c\}\(M\)⊳\\trianglerightparse raw inputs

3:

P∗←𝔸g​\(I,M\)P^\{\*\}\\leftarrow\\mathbb\{A\}\_\{g\}\(I,M\)⊳\\trianglerightgenerate initial code

4:

R∗←Exec​\(P∗\)R^\{\*\}\\leftarrow\\text\{Exec\}\(P^\{\*\}\)⊳\\trianglerightvalidateP∗P^\{\*\}\(Alg\.[2](https://arxiv.org/html/2608.05375#alg2)\)

5:

L←𝔸l​\(R∗\)L\\leftarrow\\mathbb\{A\}\_\{l\}\(R^\{\*\}\)⊳\\trianglerightinitialize memory log

6:else

7:return

m​s​gmsg⊳\\trianglerighterror msg for invalid inputs

8:endif

9:while

S≤TS\\leq Tdo

10:

E←𝔸e​\(P∗,M,L\)E\\leftarrow\\mathbb\{A\}\_\{e\}\(P^\{\*\},M,L\)⊳\\trianglerightevaluateP∗P^\{\*\}\(Sec\.[3\.6](https://arxiv.org/html/2608.05375#S3.SS6)\)

11:

F←∇P∗\(E\)F\\leftarrow\\nabla\_\{P^\{\*\}\}\(E\)⊳\\trianglerightgradient computation \(Eq\.[1](https://arxiv.org/html/2608.05375#S3.E1)\)

12:

P←𝔸o​\(P∗,F\)P\\leftarrow\\mathbb\{A\}\_\{o\}\(P^\{\*\},F\)⊳\\trianglerightupdateP∗P^\{\*\}\(Eq\.[2](https://arxiv.org/html/2608.05375#S3.E2)\)

13:

R←Exec​\(P\)R\\leftarrow\\text\{Exec\}\(P\)
14:

n​o​t​e←𝔸l​\(P∗,P,R∗,R\)note\\leftarrow\\mathbb\{A\}\_\{l\}\(P^\{\*\},P,R^\{\*\},R\)⊳\\trianglerightcreate note \(Sec\.[3\.5](https://arxiv.org/html/2608.05375#S3.SS5)\)

15:

L←L\+n​o​t​eL\\leftarrow L\+note
16:if

C​P​\(L\)CP\(L\)=truethen

17:

L←SUMMA​\(𝔸l,L\)L\\leftarrow\\text\{SUMMA\}\(\\mathbb\{A\}\_\{l\},L\)⊳\\trianglerightsummarize L \(Sec\.[3\.5](https://arxiv.org/html/2608.05375#S3.SS5)\)

18:endif

19:

P∗←PP^\{\*\}\\leftarrow P
20:

R∗←RR^\{\*\}\\leftarrow R
21:

S←S\+1S\\leftarrow S\+1
22:endwhile

23:return

PP

### 3\.2\.Overall workflow

The workflow ofDoctorAgentsis depicted in Fig\.[1](https://arxiv.org/html/2608.05375#S3.F1)and Alg\.[1](https://arxiv.org/html/2608.05375#alg1), which consists of two main stages:

##### Initialization

As illustrated in Fig\.[1](https://arxiv.org/html/2608.05375#S3.F1)a,Consultant\(𝔸c\\mathbb\{A\}\_\{c\}\) is prompted to receive validated user\-provided information and metadata of the dataset \(Line 2\) and transforms them into a structured task specification with explicit requirements\. These data are then passed toGenerator\(𝔸g\\mathbb\{A\}\_\{g\}\), which is prompted to produce an initial end\-to\-end full\-pipeline implementation followed by validation \(Lines 3–4\)\. The resulting data are added to the memory log for subsequent downstream optimization \(Line 5\)\.

##### Optimization

The framework iteratively refines the pipeline to search for an improved implementation by cycling through 3 steps\.Step 1 \- code execution\(Fig\.[1](https://arxiv.org/html/2608.05375#S3.F1)b\):Repairer\(𝔸r\\mathbb\{A\}\_\{r\}\) receives error traces upon execution failure and attempts to minimally fix the code before re\-validating it \(Alg\.[2](https://arxiv.org/html/2608.05375#alg2)\)\. Successful execution returns performance results \(Lines 4, 13\)\.Step 2 \- memory composition\(Fig\.[1](https://arxiv.org/html/2608.05375#S3.F1)c\): given both the previous and updated pipelines together with the execution results,Logger\(𝔸l\\mathbb\{A\}\_\{l\}\) appends notes that capture important changes to the log \(Lines 14–15\)\. Log summarization and cleaning are periodically triggered at predefined checkpoints to minimize the in\-context length and memory storage \(Line 17\)\.Step 3 \- pipeline refinement\(Fig\.[1](https://arxiv.org/html/2608.05375#S3.F1)d\):Evaluator\(𝔸e\\mathbb\{A\}\_\{e\}\) assesses the pipeline using both the latest performance records and the history log \(Line 10\)\. Its feedback is backpropagated toOptimizer\(𝔸o\\mathbb\{A\}\_\{o\}\) through TextGrad\(Yuksekgonulet al\.,[2025](https://arxiv.org/html/2608.05375#bib.bib1)\), which updates the pipeline accordingly \(Lines 11–12\)\. The updated pipeline is then validated for the next optimization iteration \(Line 19\) and returned to the user once the stopping criterion is met\.

The following subsections provide details of the key steps including Pipeline Initialization \(Sec\.[3\.3](https://arxiv.org/html/2608.05375#S3.SS3)\), Implementation Verification \(Sec\.[3\.4](https://arxiv.org/html/2608.05375#S3.SS4)\), Memory Composition \(Sec\.[3\.5](https://arxiv.org/html/2608.05375#S3.SS5)\), Pipeline Refinement \(Sec\.[3\.6](https://arxiv.org/html/2608.05375#S3.SS6)\), and Domain\-Specific Specialization \(Sec\.[3\.7](https://arxiv.org/html/2608.05375#S3.SS7)\)\.

### 3\.3\.Pipeline Initialization

The system begins with an initialization phase to ensure a valid base pipeline \(Fig\.[1](https://arxiv.org/html/2608.05375#S3.F1)a\)\. As the performance of an autonomous system is highly sensitive to its starting point, a good initialization can achieve the optimal performance at substantially fewer iterations\.

##### Metadata

To enhance data exploration and improve downstream generation quality, we derive a set of descriptive statistics from the input datasetDD\. Specifically, a small subset of representative samples, feature statistics \(computed by LLM\-based profiling if not provided\), and user\-provided background information of the task are combined into a unified metadata representationMM, which is provided to both𝔸c\\mathbb\{A\}\_\{c\}and𝔸g\\mathbb\{A\}\_\{g\}to support robust, data\-informed pipeline initialization and task understanding\.

##### Task Specification

The Consultant𝔸c\\mathbb\{A\}\_\{c\}takes as input the metadataMMand produces a structured outputII=𝔸c\\mathbb\{A\}\_\{c\}\(MM\) as strictly predefined in the prompt \(Sec\.[A\.4\.1](https://arxiv.org/html/2608.05375#A1.SS4.SSS1)\), consisting of two parts \(Example output in Sec\.[A\.6\.1](https://arxiv.org/html/2608.05375#A1.SS6.SSS1)\):

- •Task Description: a concise and unambiguous description of the target ML task, intended to serve as a stable reference for downstream agents\.
- •Task Suggestion: task\-specific recommendations for data preprocessing and model search strategies, including candidate algorithms, normalization schemes, and missing\-value handling, which guide subsequent agent decisions\.

##### Pipeline Synthesis

Conditioned onIIandMM,𝔸g\\mathbb\{A\}\_\{g\}synthesizes the initial executable pipelineP∗P^\{\*\}\. Specifically, it translates the task description, variable statistics, and reference suggestions into a complete pipeline implementation, includingdata loading,preprocessing,model construction,training,evaluation, andresult reporting\.𝔸g\\mathbb\{A\}\_\{g\}is encouraged to follow the suggestion by𝔸c\\mathbb\{A\}\_\{c\}, but it is not constrained to a fixed template, enabling flexibility across diverse tasks and datasets\. The resultingP∗P^\{\*\}=𝔸g\\mathbb\{A\}\_\{g\}\(I,MI,M\) serves as the initial node in the downstream optimization trajectory\.

### 3\.4\.Implementation Verification

Given a pipeline codePeP\_\{e\},DoctorAgentsvalidates both the executability and efficiency\. This stage integrates program execution with an autonomous repair mechanism \(Fig\.[1](https://arxiv.org/html/2608.05375#S3.F1)b; Alg\.[2](https://arxiv.org/html/2608.05375#alg2)\), forming a closed\-loop safeguard before downstream optimization\.

Execution validation \(Exec\) attempts to execute the pipeline code through a program runnerfef\_\{e\}within a fixed time budget and produces an execution signalf​l​a​gflag, an error messagee​r​rerr, and resultsReR\_\{e\}\. Successful execution \(f​l​a​gflag= PASS\) yields concrete outputsReR\_\{e\}, involvingtraining,validation, andtest performanceof the model with itsnameandhyperparameter matrix, or metadata such asdecision\-oriented strategies and variable\-level information, which will be subsequently consumed by the downstream agents\. Program failures \(f​l​a​gflag= FAIL\) may arise from a variety of sources, including runtime errors, incompatible library usage, or resource exhaustion\. When execution fails, the error tracee​r​rerr\(either an execution error or a timeout resulting from the time budget being exceeded\), together withPeP\_\{e\}, is passed to𝔸r\\mathbb\{A\}\_\{r\}, which is prompted \(Sec\.[A\.4\.3](https://arxiv.org/html/2608.05375#A1.SS4.SSS3)\) to minimally modify the program to restore executability ofPeP\_\{e\}via𝔸r\\mathbb\{A\}\_\{r\}\(PeP\_\{e\},e​r​rerr\) while preserving the original algorithmic intent\. Typical repair actions include correcting method misuse, resolving variable scope or type errors, and mitigating runtime exceptions\. After producing a repaired version ofPeP\_\{e\}, re\-validation is performed\. This procedure repeats until either the program executes successfully or a predefined retry limit is reached \(which is rare\)\. By incorporating execution validation into optimization,DoctorAgentsresolves execution\-level issues for successful execution\.

### 3\.5\.Memory Composition

A key challenge in designing an autonomous code optimization system is the lack of a persistent state across iterations\. Without explicit memory, agents may repeatedly revisit identical or similar failures, reintroduce previously corrected errors, or overlook insights obtained from earlier iterations\. To address this issue,𝔸l\\mathbb\{A\}\_\{l\}maintains a structured, long\-term memory throughout the optimization process \(Fig\.[1](https://arxiv.org/html/2608.05375#S3.F1)c\)\.

##### Memory Log

Following pipeline initialization \(Sec\.[3\.3](https://arxiv.org/html/2608.05375#S3.SS3)\),𝔸l\\mathbb\{A\}\_\{l\}initializes the memory logLLusing the initial resultsR∗R^\{\*\}obtained from Exec\(P∗P^\{\*\}\)\. Thereafter,𝔸l\\mathbb\{A\}\_\{l\}records, organizes, and maintains inter\-iteration informationn​o​t​enotegenerated during pipeline refinement\. Given the updated pipelinePPand resultsRR= Exec\(PP\), rather than storing raw module outputs,𝔸l\\mathbb\{A\}\_\{l\}is prompted \(Sec\.[A\.4\.4](https://arxiv.org/html/2608.05375#A1.SS4.SSS4.Px1)\) to extract meaningful*structured inter\-iteration pipeline differences*\(e\.g\., changes in preprocessing logic, model configurations, or hyperparameters\) and associate them with performance outcome differences\. Hencen​o​t​enote=𝔸l\\mathbb\{A\}\_\{l\}\(P∗,P,R∗,RP^\{\*\},P,R^\{\*\},R\)\. Thesen​o​t​enotes are appended toLLand made accessible to downstream agents as historical context that informs subsequent decisions\.

##### Checkpoint \(C​PCP\) and Summarization \(SUMMA\)

As optimization progresses, unbounded accumulation of notes can lead to excessive context length and degrade reasoning quality\. To mitigate this*context explosion*problem,𝔸l\\mathbb\{A\}\_\{l\}incorporates a checkpoint\-based memory management strategy\. At the predefined checkpoints such as every 15 optimization iterations or at the 5000\-token limit \(C​PCP\(LL\) =true\),𝔸l\\mathbb\{A\}\_\{l\}triggers a cleaning procedureSUMMAthat summarizes its memory following the instruction prompts \(Sec\.[A\.4\.4](https://arxiv.org/html/2608.05375#A1.SS4.SSS4.Px2)\)\. This involves distilling notes from earlier iterations, discarding low\-priority or redundant entries, and retaining only salient insights \(e\.g\., changes in codes that significantly improved or degraded performance; an example shown in Sec\.[A\.6\.6](https://arxiv.org/html/2608.05375#A1.SS6.SSS6)\) along with a small set of most recent observations\. With this knowledge summarization strategy, the resulting logLL= SUMMA\(𝔸l\\mathbb\{A\}\_\{l\},LL\) preserves long\-term memory and maintains concise information\.

### 3\.6\.Pipeline Refinement

Once the initial pipelineP∗P^\{\*\}is validated \(Sec\.[3\.4](https://arxiv.org/html/2608.05375#S3.SS4)\) to initialize the optimizer and the firstn​o​t​enote, derived from the execution resultR∗R^\{\*\}= Exec\(P∗\)\(P^\{\*\}\), which is cached intoLL,DoctorAgentsenters the core*Refinement*phase, where pipeline quality is iteratively improved through evaluation\-driven program refinement\. This phase follows a closed\-loop structure consisting of two primary steps:EvaluationandOptimization\(Fig\.[1](https://arxiv.org/html/2608.05375#S3.F1)d\)\.

##### Evaluation

Given the current pipelineP∗P^\{\*\}, dataset metadataMM, and the memory logLL,𝔸e\\mathbb\{A\}\_\{e\}produces an assessmentEEthat typically covers three aspects \(Sec\.[A\.4\.5](https://arxiv.org/html/2608.05375#A1.SS4.SSS5)\):

- •Correctness: check ifP∗P^\{\*\}correctly implements the intended task and evaluation protocol, and suggest corrections otherwise;
- •Identified Flaws: detect weaknesses such as overfitting, inefficient preprocessing, or inappropriate model choices with respect toLLand suggest a mitigation plan;
- •Improvement Opportunities: provide suggestions to improve performance, robustness, and efficiency\.

Unlike numerical optimization in traditional ML,EE=𝔸e\\mathbb\{A\}\_\{e\}\(P∗P^\{\*\},MM,LL\) is expressed in natural language, capturing fine\-grained and context\-aware judgments about pipeline behavior and enabling more informative downstream updates\.

##### Optimization via Backpropagating TGD

The backpropagation component translatesEEinto localized instructions for code modification\. Analogous to the numerical gradient backpropagation, it propagates the high\-level evaluation signal backward through theprogram representationwith respect toP∗P^\{\*\}, identifying components that need to be improved, replaced, or discarded\(Yuksekgonulet al\.,[2025](https://arxiv.org/html/2608.05375#bib.bib1)\)\. This produces an actionable feedback signal:

\(1\)F=∇P∗\(E\),F=\\nabla\_\{P^\{\*\}\}\(E\),which𝔸o\\mathbb\{A\}\_\{o\}receives as input to update the pipeline as follows:

\(2\)P=𝔸o​\(P∗,F\),P=\\mathbb\{A\}\_\{o\}\(P^\{\*\},F\),where it incorporates changes to preprocessing plan, model configuration, hyperparameters, training procedures, and/or evaluation protocols\.𝔸o\\mathbb\{A\}\_\{o\}performs incremental, targeted modifications while preserving previously successful components, avoiding costly full regeneration\. After optimization, the revised pipelinePPis executed to obtain new resultsRRfor memory logging, and then both are forwarded back asP∗P^\{\*\}andR∗R^\{\*\}to close one optimization iteration\. This iterative process continues until the termination criterion is met \(e\.g\., predefined optimization stepsTT\)\.

![Refer to caption](https://arxiv.org/html/2608.05375v1/Main_Figures/New_Framework.png)Figure 2\.DoctorAgentsDomain\-Specific \(DS\) Design\. Both Generator and Evaluator are specialized into Data Preprocessing \(DP\) and Model Development \(MD\) to enable deeper DS exploration and targeted implementation\.DoctorAgents\-DS framework diagram\.

### 3\.7\.Domain\-Specific Specialization

Building upon theDoctorAgentsframework, we introduce a specialized variant that decomposes the pipeline into two domain\-specific subtasks—Data Preprocessing \(DP\) and Model Development \(MD\)—and assigns them to dedicated agents as Generator agents,𝔸gd​p\\mathbb\{A\}\_\{g\}^\{dp\}and𝔸gm​d\\mathbb\{A\}\_\{g\}^\{md\}, and Evaluator agents,𝔸ed​p\\mathbb\{A\}\_\{e\}^\{dp\}and𝔸em​d\\mathbb\{A\}\_\{e\}^\{md\}\(Fig\.[2](https://arxiv.org/html/2608.05375#S3.F2)\)\. This decomposition allows agents to operate within focused search domains: DP Agents focus on identifying feature engineering strategies, while MD Agents explore a richer spectrum of model families and training configurations\.

##### Initialization

𝔸gd​p\\mathbb\{A\}\_\{g\}^\{dp\}first generates a preprocessing\-only pipelinePd​p∗=𝔸gd​p​\(I,M\)P\_\{dp\}^\{\*\}=\\mathbb\{A\}\_\{g\}^\{dp\}\(I,M\)\(Sec\.[A\.4\.6](https://arxiv.org/html/2608.05375#A1.SS4.SSS6.Px1)\) given the task specificationIIand metadata of the datasetMM\. Intermediate dataRi∗R\_\{i\}^\{\*\}are produced via Exec\(Pd​p∗\)\(P\_\{dp\}^\{\*\}\), including enriched feature representations, metadata describing applied transformations, and a preprocessed dataset, thereby supporting broader exploratory behavior in the preprocessing domain\. The dataRi∗R\_\{i\}^\{\*\}are then forwarded to𝔸gm​d\\mathbb\{A\}\_\{g\}^\{md\}to synthesize the model\-development pipelinePm​d∗=𝔸gm​d​\(I,Ri∗\)P\_\{md\}^\{\*\}=\\mathbb\{A\}\_\{g\}^\{md\}\(I,R\_\{i\}^\{\*\}\)\(Sec\.[A\.4\.6](https://arxiv.org/html/2608.05375#A1.SS4.SSS6.Px3)\)\. ExecutingPm​d∗P\_\{md\}^\{\*\}produces resultsR∗R^\{\*\}which include information about top features ranked by importance \(computed with SHapley Additive exPlanations \(SHAP\)\(Lundberg and Lee,[2017](https://arxiv.org/html/2608.05375#bib.bib45)\)\) and the initial memory logL=𝔸l​\(Ri∗,R∗\)L=\\mathbb\{A\}\_\{l\}\(R\_\{i\}^\{\*\},R^\{\*\}\)\.

##### Optimization

𝔸ed​p\\mathbb\{A\}\_\{e\}^\{dp\}evaluatesPd​p∗P\_\{dp\}^\{\*\}\(Sec\.[A\.4\.6](https://arxiv.org/html/2608.05375#A1.SS4.SSS6.Px2)\) usingMMandLL\(Ed​p=𝔸ed​p​\(Pd​p∗,M,L\)E\_\{dp\}=\\mathbb\{A\}\_\{e\}^\{dp\}\(P\_\{dp\}^\{\*\},M,L\), an example shown in Sec\.[A\.6\.2](https://arxiv.org/html/2608.05375#A1.SS6.SSS2)\) to backpropagate domain\-specific feedbackFd​p=∇Pd​p∗\(Ed​p\)F\_\{dp\}=\\nabla\_\{P\_\{dp\}^\{\*\}\}\(E\_\{dp\}\)\(e\.g\., Sec\.[A\.6\.3](https://arxiv.org/html/2608.05375#A1.SS6.SSS3)\), following an evaluation scheme tailored to the data preprocessing domain\. The updated pipeline is then produced asPd​p=𝔸o​\(Pd​p∗,Fd​p\)P\_\{dp\}=\\mathbb\{A\}\_\{o\}\(P\_\{dp\}^\{\*\},F\_\{dp\}\)\. ExecutingPd​pP\_\{dp\}yields new intermediate dataRi=Exec​\(Pd​p\)R\_\{i\}=\\text\{Exec\}\(P\_\{dp\}\), which are passed to generate assessmentEm​d=𝔸em​d​\(Pm​d∗,Ri,L\)E\_\{md\}=\\mathbb\{A\}\_\{e\}^\{md\}\(P\_\{md\}^\{\*\},R\_\{i\},L\)\(prompt described in Sec\.[A\.4\.6](https://arxiv.org/html/2608.05375#A1.SS4.SSS6.Px4)\) forPm​d∗P\_\{md\}^\{\*\}\(e\.g\., Sec\.[A\.6\.4](https://arxiv.org/html/2608.05375#A1.SS6.SSS4)\)\. The resulting feedbackFm​d=∇Pm​d∗\(Em​d\)F\_\{md\}=\\nabla\_\{P\_\{md\}^\{\*\}\}\(E\_\{md\}\)\(e\.g\., Sec\.[A\.6\.5](https://arxiv.org/html/2608.05375#A1.SS6.SSS5)\) produces refinedPm​dP\_\{md\}via𝔸o​\(Pm​d∗,Fm​d\)\\mathbb\{A\}\_\{o\}\(P\_\{md\}^\{\*\},F\_\{md\}\)\. Memory log is computed between original and updated pipelines \(P∗P^\{\*\}s andPPs\) along with the execution results after optimizingPm​d∗P\_\{md\}^\{\*\}by Logger𝔸l\\mathbb\{A\}\_\{l\}, and then cached toLLfor future evaluations\. ExecutingPm​dP\_\{md\}produces updated performance resultsRRfor the current iteration\. In the end,Pd​p,Pm​d,Ri,RP\_\{dp\},P\_\{md\},R\_\{i\},Rare forwarded back for the next iteration\. Together, the DP and MD updates form one complete optimization loop\.

## 4\.Experiments

### 4\.1\.Benchmark Datasets

We evaluatedDoctorAgentson four temporal clinical datasets \(Table[A1](https://arxiv.org/html/2608.05375#A1.T1)\), including three tasks derived from the public MIMIC\-IV database\(Johnsonet al\.,[2023](https://arxiv.org/html/2608.05375#bib.bib3)\)and one rheumatoid arthritis \(RA\) dataset from the Early Undifferentiated PolyArthritis \(EUPA\) cohort\(Carrieret al\.,[2025](https://arxiv.org/html/2608.05375#bib.bib48),[2026](https://arxiv.org/html/2608.05375#bib.bib47)\)\. The MIMIC\-IV tasks are: \(1\) ICU mortality prediction, which predicts in\-ICU mortality using the first 48 hours of data; \(2\) ICU readmission prediction, which predicts readmission within seven days after the first ICU discharge using all pre\-discharge observations; and \(3\) ICU length\-of\-stay \(LOS\) prediction, which predicts whether an ICU stay exceeds 3 days using the first 24 hours of data\. These datasets include irregular temporal laboratory measurements fromlabeventsand selected vital signs fromchartevents\.

The EUPA dataset contains 1–9 years of sparsely sampled longitudinal rheumatoid arthritis \(RA\) patient records and is used to predict anti\-TNF treatment response from pre\-treatment clinical trajectories, distinguishing responders from non\-responders\.

For all datasets, temporal observations were retained with their raw irregular timestamps within each task\-specific observation window, without fixed binning or manual visit selection\. This design preserves the original temporal structure and allowsDoctorAgentsto determine suitable aggregation, recency, and change\-based representations during pipeline construction\.

We adopted task\-specific evaluation metrics according to the class distributions\. We report the area under the receiver operating characteristic curve \(AUROC\) for the approximately balanced RA treatment\-response prediction task and the area under the precision–recall curve \(AUPRC\) for the three MIMIC\-IV tasks, for which the outcome labels are imbalanced\.

### 4\.2\.Baseline Methods

We evaluated TabPFN as a traditional ML competitor for small\-sample tabular learning\(Hollmannet al\.,[2025](https://arxiv.org/html/2608.05375#bib.bib4)\)\. We also compared against three LLM\-based few\-shot baselines: a state\-of\-the\-art \(SOTA\) general\-purpose proprietary model GPT\-5\(Singhet al\.,[2025](https://arxiv.org/html/2608.05375#bib.bib29)\), and two open\-source LLMs GPT\-oss\-20b and Llama\-8b\(Agarwalet al\.,[2025](https://arxiv.org/html/2608.05375#bib.bib5); Grattafioriet al\.,[2024](https://arxiv.org/html/2608.05375#bib.bib38)\)\. To evaluate SOTA agentic approaches, we benchmarked AutoML\-Agent\(Triratet al\.,[2025](https://arxiv.org/html/2608.05375#bib.bib13)\)and ERA\(Aygünet al\.,[2026](https://arxiv.org/html/2608.05375#bib.bib37)\)\. GPT\-5 was used as the backbone LLM for all agentic frameworks including ours\. The prompt details are provided in Sec\.[A\.5](https://arxiv.org/html/2608.05375#A1.SS5)\.

### 4\.3\.Evaluation Design

To ensure rigorous evaluation, each dataset was partitioned into a meta\-train and meta\-test split under a stratified five\-fold cross\-validation design\. The final results were reported on the meta\-test split as the mean and standard deviation across the five folds\. In order to evaluate the agents’ ability to reason over and leverage dataset information during preprocessing, all datasets were only minimally processed to prevent data leakage and were otherwise kept as close to their raw form as possible\.

ForDoctorAgents, the meta\-train split was further divided into train, validation, and internal test subsets, where machine learning models were trained on the train split\. Optimization decisions were driven by performance on the validation split, while the best pipeline selection was based on the internal test split\. For a fair comparison, in each fold,DoctorAgentsand ERA were limited to a maximum of 20 optimization steps and search nodes, respectively, whereasDoctorAgents\-DS was limited to 15 optimization steps\. The retry limit for bothDoctorAgentsand AutoML\-Agent was set to 5\. Since TabPFN does not support time\-series inputs, we applied an additional compatibility\-only preprocessing step for TabPFN, representing each feature by its last observed value within the observation window\. TabPFN was then directly trained on the entire meta\-train split and evaluated on the meta\-test split, while all other baselines leveraged the meta\-train’s train\-valid\-test partitions to synthesize their ML pipelines\. All experiments were run on an NVIDIA 4x RTX PRO 6000 Blackwell Max\-Q workstation\.

![Refer to caption](https://arxiv.org/html/2608.05375v1/Main_Figures/curve_plot.png)Figure 3\.Validation and internal test performance over iteration forDoctorAgents\. Each iteration was labelled with the model name if a new model was selected\. The unlabeled iterations indicate that the best model remains the same although there could be preprocessing or hyperparameter updates\. For illustration purposes, only 10 iterations are shown althoughDoctorAgentscan reach higher validation performance for more iterations\. A table for model name explanation is provided in Table[A2](https://arxiv.org/html/2608.05375#A1.T2)\.Iterative improvements curves\.

## 5\.Results

### 5\.1\.Iterative ML Pipeline Optimization

We first validated the iterative refinement behavior ofDoctorAgents\(Fig\.[3](https://arxiv.org/html/2608.05375#S4.F3)\)\. Across all benchmarks, validation performance showed a generally improving trajectory, indicating that successive agentic iterations apply consistently beneficial refinements\. In parallel,DoctorAgentsdynamically revised its choice of learning algorithms, as reflected by the model annotations on the curve\. For mortality prediction \(Fig\.[3](https://arxiv.org/html/2608.05375#S4.F3)a\),DoctorAgentsstarted from a regularized linear model and progressively explored alternative configurations, including boosting and ensemble\-based variants, while validation AUPRC improved through subsequent preprocessing and hyperparameter refinements\. A similar pattern was observed for readmission prediction and RA anti\-TNF response prediction \(Fig\.[3](https://arxiv.org/html/2608.05375#S4.F3)b,d\), where the best\-performing models across iterations were mainly selected from regularized logistic regression and histogram\-based gradient boosting\. Notably,DoctorAgentsevaluated a broader candidate model space at each iteration, including XGBoost, Random Forest, LightGBM, and ensemble variants\. The repeated selection of linear and boosting\-based models suggests that they provide strong inductive biases for high\-dimensional, sparse, and imbalanced small clinical datasets\. In contrast, LOS prediction \(Fig\.[3](https://arxiv.org/html/2608.05375#S4.F3)c\) involved transitions among linear, boosting, and extra\-tree models, indicating that non\-linear tree ensembles may better capture interaction patterns associated with prolonged ICU stays\.

Table 1\.Performance benchmarking on the meta\-test data, reported as mean±\\pmstd across 5 cross\-validation folds\.Method / Task \(Metrics\)mortality \(AUPRC\)readmission \(AUPRC\)LOS \(AUPRC\)RA treatment response \(AUROC\)TabPFN\(Hollmannet al\.,[2025](https://arxiv.org/html/2608.05375#bib.bib4)\)0\.509±\\pm0\.0340\.163±\\pm0\.0390\.688±\\pm0\.0220\.539±\\pm0\.163GPT\-oss\(Agarwalet al\.,[2025](https://arxiv.org/html/2608.05375#bib.bib5)\)0\.443±\\pm0\.0460\.149±\\pm0\.0330\.675±\\pm0\.0230\.471±\\pm0\.139Llama\-8B\(Grattafioriet al\.,[2024](https://arxiv.org/html/2608.05375#bib.bib38)\)0\.447±\\pm0\.0350\.142±\\pm0\.0420\.664±\\pm0\.0370\.453±\\pm0\.106GPT\-5\(Singhet al\.,[2025](https://arxiv.org/html/2608.05375#bib.bib29)\)0\.456±\\pm0\.0430\.154±\\pm0\.0200\.678±\\pm0\.0260\.498±\\pm0\.161AutoML\-Agent\(Triratet al\.,[2025](https://arxiv.org/html/2608.05375#bib.bib13)\)0\.475±\\pm0\.0220\.145±\\pm0\.0100\.689±\\pm0\.0200\.520±\\pm0\.122ERA\(Aygünet al\.,[2026](https://arxiv.org/html/2608.05375#bib.bib37)\)0\.476±\\pm0\.0140\.192±\\pm0\.0440\.692±\\pm0\.0240\.564±\\pm0\.121DoctorAgents\(ours\)0\.482±\\pm0\.0510\.190±\\pm0\.0170\.696±\\pm0\.0280\.505±\\pm0\.060DoctorAgents\-DS \(ours\)0\.520±\\pm0\.0760\.212±\\pm0\.0470\.704±\\pm0\.0190\.577±\\pm0\.088

### 5\.2\.Method Comparison

The domain\-specificDoctorAgents\-DS achieves the strongest mean performance across all benchmarks, whileDoctorAgentsremains competitive against the established agentic and LLM\-based baselines \(Table[1](https://arxiv.org/html/2608.05375#S5.T1)\)\. For mortality prediction,DoctorAgents\-DS obtains the highest AUPRC, surpassing TabPFN and ERA\.DoctorAgents\-DS also outperforms all baselines by constructing richer temporal representations for readmission prediction task\. On the prolonged LOS task, bothDoctorAgentsvariants perform competitively, withDoctorAgents\-DS achieving the highest AUPRC\.DoctorAgents\-DS also attains the highest AUROC on the small longitudinal RA cohort\. These results demonstrate that the framework adapts its preprocessing and model\-selection strategies to heterogeneous prediction horizons, feature distributions, and sample sizes without relying on manually designed task\-specific pipelines or modality\-specific foundation models\.

![Refer to caption](https://arxiv.org/html/2608.05375v1/Main_Figures/ablation.png)Figure 4\.Ablation study on ICU readmission prediction as an example, comparing validation AUPRC across optimization iterations for the fullDoctorAgentsand variants without the Consultant or Memory module\.Ablation study\.
### 5\.3\.Ablation Study

We conducted an ablation study on the contribution of the Consultant and persistent Memory to the iterative optimization process \(Fig\.[4](https://arxiv.org/html/2608.05375#S5.F4)\), which are designed to improve initialization quality, optimization stability, and long\-term refinement over the essential backbone ofDoctorAgents\. The fullDoctorAgentsexhibits the most consistent overall improvement and achieves the highest validation AUPRC among ten iterations on ICU readmission prediction\.

Removing the Consultant primarily weakens pipeline initialization: the corresponding variant started from a lower validation AUPRC and remained below the full system during most early iterations, although subsequent refinement partially closed the gap\. This suggests that the Consultant provided a better\-informed initial task specification and preprocessing direction, reducing the amount of exploration required in later steps\. Without Memory, the optimization trajectory became substantially less stable, with transient improvements followed by pronounced performance degradation\. AlthoughDoctorAgentswithout Memory achieved a high validation score in the early iterations, it failed to retain and exploit the corresponding successful strategy in subsequent refinements\. This behavior indicates that persistent records of previous modifications and outcomes help prevent the optimizer from revisiting ineffective strategies or overwriting previously successful components\.

All ablated variants attain a lower maximum validation AUPRC than the complete system\. In particular, the full version reached approximately 0\.255, compared with 0\.246 without the Consultant and 0\.215 without Memory\. These results demonstrate that these modules play distinct but complementary roles in the optimization process: the Consultant supports effective initialization whereas Memory stabilizes prolonged iterative refinement\.

## 6\.Interpretability

### 6\.1\.In\-code Feature Composition

Besides selecting suitable models,DoctorAgents\-DS autonomously constructs task\-specific representations measured by mean absolute SHAP values from temporal clinical data \(Fig\.[A1](https://arxiv.org/html/2608.05375#A1.F1), Table[A\.2](https://arxiv.org/html/2608.05375#A1.SS2)\)\.

For mortality prediction, the leading features characterize early ICU physiological instability across the first 48 hours and baseline clinical vulnerability \(Fig\.[A1](https://arxiv.org/html/2608.05375#A1.F1)a, Sec\.[A\.7\.1](https://arxiv.org/html/2608.05375#A1.SS7.SSS1)\)\. They include metabolic and acid\-base dynamics \(bicarbonate\_blood\_delta\_per\_hr\), neurological status \(e\.g\.gcs\_eye\_last\), and hemodynamic condition \(mean\_arterial\_pressure\_mean\_24\_48h\)\. The inclusion ofn\_glucose\_blood\_meas\_48hfurther reflects clinical observation intensity\. Together, these features align with mortality prediction as an acute deterioration task by combining baseline vulnerability, evolving physiology, and monitoring patterns\.

For the readmission task,DoctorAgents\-DS constructs features describing residual instability and clinical attention near ICU discharge \(Fig\.[A1](https://arxiv.org/html/2608.05375#A1.F1)b, Sec\.[A\.7\.2](https://arxiv.org/html/2608.05375#A1.SS7.SSS2)\), including recent monitoring intensity through heart\-rate counts over the last 6 hours \(heart\_rate\_count\_log1p\_last6h\) and short\-window physiological trajectories through the 6\-hour respiratory\-rate slope \(respiratory\_rate\_slope\_last6h\_per\_hr\)\. These features align with readmission risk, which is tied to unresolved instability immediately before discharge rather than only to early ICU severity\.

For prolonged ICU LOS prediction, respiratory support, oxygenation, neurological function, and measurement recency are prominent \(Fig\.[A1](https://arxiv.org/html/2608.05375#A1.F1)c, Sec\.[A\.7\.3](https://arxiv.org/html/2608.05375#A1.SS7.SSS3)\)\. Features such asfio2\_mean\_last6handspo2\_fio2\_ratiosummarize oxygenation status and respiratory support requirements\.lactate\_blood\_recency\_hoursdistinguishes recent clinical evidence from stale observations, whilegcs\-related features capture neurological status and assessment intensity\. This representation matches the LOS task by encoding persistent organ\-support needs and delayed recovery signals\.

For RA treatment\-response prediction,DoctorAgents\-DS converts variable pre\-treatment visits into a patient\-level representation that includes baseline genetic and serological risk factors, recent disease activity, longitudinal variability and discrete treatment choices \(Fig\.[A1](https://arxiv.org/html/2608.05375#A1.F1)d, Sec\.[A\.7\.4](https://arxiv.org/html/2608.05375#A1.SS7.SSS4)\)\. Some of the leading features, including the presence of 2 shared epitope alleles \(SE\_2\.0\), anti\-CCP antibodies \(antiCCP\_max\_pre\) and fair functional status \(GrFonct\_2\.0\), provide baseline indicators of disease severity\. Recent disease activity is represented by patient and evaluator global assessments \(PGA\_last\_preandEGA\_last\_pre\)\. Meanwhile, longitudinal variability in fatigue \(Fatigue\_std\_365d\) and temporal change in platelet count \(FscPLQ\_delta\_365d\) reflect more distant variation in levels of function and inflammation\. Finally, the specific type of anti\-TNF drug \(AntiTNF\_or\_JAK\_First\_3, which refers to etanercept as opposed to other anti\-TNF drugs\) provides treatment level prediction information\. The importance of both static genetic and immunological markers and longitudinal pre\-treatment changes shows thatDoctorAgents\-DS preserves complementary information about patient background, disease trajectory, and current clinical status\.

Overall, these results demonstrate thatDoctorAgents\-DS composes predictive features according to each task’s temporal structure, clinical context, and non\-random missing pattern rather than relying on a fixed preprocessing template\.

### 6\.2\.Reasoning Validation

![Refer to caption](https://arxiv.org/html/2608.05375v1/Main_Figures/reasonable_features.png)Figure 5\.Distribution of high\-quality feature proportions composed byDoctorAgents\-DS, ERA, and AutoML\-Agent across all tasks\. For each method and task, the distribution comprises 10 observations from two independent runs over five folds, each denoting the proportion ofWell JustifiedorClinically Reasonablefeatures\. P\-values were obtained from one\-sided paired Wilcoxon signed\-rank tests\.High\-quality feature distribution boxplot\.To validate the features composed byDoctorAgents\-DS, we assessed the reasonability of features using GPT\-5\.4\. We used GPT\-5\.4 as an independent judge because of its consistent semantic reasoning over feature names, mathematical operations, and task context with the caveat that GPT\-5\.4 is not perfect\. Features were classified into five levels:Invalid,Weakly Justified,Plausible but Uncertain,Well Justified, andClinically Reasonable, based on clinical validity, mathematical correctness, semantic compatibility, and consistency with the task setting \(prompt details in Sec\.[A\.5\.2](https://arxiv.org/html/2608.05375#A1.SS5.SSS2.Px4)\)\.

DoctorAgents\-DS produced the highest proportion ofWell Justifiedfeatures across all four tasks \(Fig\.[A2](https://arxiv.org/html/2608.05375#A1.F2)\)\. Although its proportion ofClinically Reasonablefeatures was slightly lower than that of ERA in three tasks, this likely reflects its broader use of temporal representations, including changes across observation periods, measurement recency, and visit\-level differences\. In contrast, ERA and AutoML\-Agent relied more heavily on conventional transformations such as last values, means, and missingness indicators, which may be more readily judged asClinically Reasonableas they are widely established in prior work\.

To evaluate overall feature quality, we groupedWell JustifiedandClinically Reasonablefeatures ashigh\-quality features\. Across the paired evaluations,DoctorAgents\-DS generated a higher overall proportion of high\-quality features than both ERA and AutoML\-Agent \(Fig\.[5](https://arxiv.org/html/2608.05375#S6.F5)\), with mean improvements of7\.10%7\.10\\%and8\.33%8\.33\\%, respectively\. One\-sided paired Wilcoxon signed\-rank tests\(Wilcoxon,[1945](https://arxiv.org/html/2608.05375#bib.bib59)\)confirmed that both improvements were significant \(p=4\.61×10−8p=4\.61\\times 10^\{\-8\}and8\.00×10−118\.00\\times 10^\{\-11\}, respectively\)\.

## 7\.Discussion

We developedDoctorAgents, a reasoning\-driven agentic framework for constructing reliable ML pipelines for small clinical datasets\. Across all benchmarks,DoctorAgentsachieves competitive performance while adapting to heterogeneous prediction settings\. Ablation results show that its components jointly improve initialization and optimization stability, whileDoctorAgents\-DS produces interpretable, task\-specific representations with stronger performance and significantly more high\-quality features than competing agentic methods\. Notably, performance on RA treatment response prediction remains modest across methods, likely due to the limited availability of strong predictive biomarkers, with the small cohort further increasing fold\-level sensitivity to individual patients\. Future work will include extendingDoctorAgentsto multimodal data and improving agent specialization through instruction fine\-tuning\.

## References

- Agentic AI: autonomous intelligence for complex goals—a comprehensive survey\.IEEE Access13,pp\. 18912–18936\.External Links:[Document](https://dx.doi.org/10.1109/ACCESS.2025.3532853)Cited by:[§1](https://arxiv.org/html/2608.05375#S1.p5.1)\.
- S\. Agarwal, L\. Ahmad, J\. Ai, S\. Altman, A\. Applebaum, E\. Arbus, R\. K\. Arora, Y\. Bai, B\. Baker, H\. Bao,et al\.\(2025\)gpt\-oss\-120b & gpt\-oss\-20b model card\.External Links:2508\.10925,[Document](https://dx.doi.org/10.48550/arXiv.2508.10925)Cited by:[§4\.2](https://arxiv.org/html/2608.05375#S4.SS2.p1.1),[Table 1](https://arxiv.org/html/2608.05375#S5.T1.10.8.8.5)\.
- E\. Aygün, A\. Belyaeva, G\. Comanici, M\. Coram, H\. Cui, J\. Garrison, R\. Johnston, A\. Kast, C\. Y\. McLean, P\. Norgaard,et al\.\(2026\)An ai system to help scientists write expert\-level empirical software\.Nature654,pp\. 909–916\.External Links:[Document](https://dx.doi.org/10.1038/s41586-026-10658-6)Cited by:[§1](https://arxiv.org/html/2608.05375#S1.p5.1),[§2](https://arxiv.org/html/2608.05375#S2.SS0.SSS0.Px3.p1.1),[§4\.2](https://arxiv.org/html/2608.05375#S4.SS2.p1.1),[Table 1](https://arxiv.org/html/2608.05375#S5.T1.26.24.24.5)\.
- V\. Borisov, T\. Leemann, K\. Seßler, J\. Haug, M\. Pawelczyk, and G\. Kasneci \(2022\)Deep neural networks and tabular data: a survey\.IEEE transactions on neural networks and learning systems35\(6\),pp\. 7499–7519\.Cited by:[§1](https://arxiv.org/html/2608.05375#S1.p3.1)\.
- T\. B\. Brown, B\. Mann, N\. Ryder, M\. Subbiah, J\. D\. Kaplan, P\. Dhariwal, A\. Neelakantan, P\. Shyam, G\. Sastry, A\. Askell,et al\.\(2020\)Language models are few\-shot learners\.InAdvances in Neural Information Processing Systems,Vol\.33,Red Hook, NY, USA,pp\. 1877–1901\.Cited by:[§1](https://arxiv.org/html/2608.05375#S1.p5.1)\.
- N\. Carrier, J\. Marrugo, M\. L\. Paudel, S\. Roux, H\. Allard\-Chamard, A\. J\. de Brum\-Fernandes, P\. Liang, and G\. Boire \(2026\)Longitudinal changes in patient characteristics as drivers of clinical outcomes in the early undifferentiated polyarthritis \(eupa\) cohort\.Rheumatology \(Oxford, England\)65\(1\),pp\. keaf650\.Cited by:[§4\.1](https://arxiv.org/html/2608.05375#S4.SS1.p1.1)\.
- N\. Carrier, S\. Roux, A\. Masetto, A\. J\. de Brum\-Fernandes, P\. Liang, M\. Maoui, and G\. Boire \(2025\)Changes at presentation in patients with early rheumatoid arthritis: a 24\-year study of the early undifferentiated polyarthritis \(eupa\) cohort\.The Journal of Rheumatology52\(2\),pp\. 119–127\.Cited by:[§4\.1](https://arxiv.org/html/2608.05375#S4.SS1.p1.1)\.
- Z\. Che, S\. Purushotham, K\. Cho, D\. Sontag, and Y\. Liu \(2018\)Recurrent neural networks for multivariate time series with missing values\.Scientific reports8\(1\),pp\. 6085\.Cited by:[§1](https://arxiv.org/html/2608.05375#S1.p2.1)\.
- M\. Chen, J\. Tworek, H\. Jun, Q\. Yuan, H\. P\. d\. O\. Pinto, J\. Kaplan, H\. Edwards, Y\. Burda, N\. Joseph, G\. Brockman,et al\.\(2021\)Evaluating large language models trained on code\.External Links:2107\.03374,[Document](https://dx.doi.org/10.48550/arXiv.2107.03374)Cited by:[§1](https://arxiv.org/html/2608.05375#S1.p5.1)\.
- G\. S\. Collins, K\. G\. Moons, P\. Dhiman, R\. D\. Riley, A\. L\. Beam, B\. Van Calster, M\. Ghassemi, X\. Liu, J\. B\. Reitsma, M\. Van Smeden,et al\.\(2024\)TRIPOD\+ ai statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods\.BMJ385,pp\. e078378\.External Links:[Document](https://dx.doi.org/10.1136/bmj-2023-078378)Cited by:[§1](https://arxiv.org/html/2608.05375#S1.p3.1)\.
- K\. Ding, J\. Yu, J\. Huang, Y\. Yang, Q\. Zhang, and H\. Chen \(2025\)Scitoolagent: a knowledge\-graph\-driven scientific agent for multitool integration\.Nature Computational Science5\(10\),pp\. 962–972\.Cited by:[§1](https://arxiv.org/html/2608.05375#S1.p5.1)\.
- M\. Feurer, A\. Klein, K\. Eggensperger, J\. T\. Springenberg, M\. Blum, and F\. Hutter \(2015\)Efficient and robust automated machine learning\.InAdvances in Neural Information Processing Systems,Vol\.28,Red Hook, NY, USA,pp\. 2962–2970\.Cited by:[§1](https://arxiv.org/html/2608.05375#S1.p4.1)\.
- D\. Golovin, B\. Solnik, S\. Moitra, G\. Kochanski, J\. Karro, and D\. Sculley \(2017\)Google vizier: a service for black\-box optimization\.InProceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining,KDD ’17,New York, NY, USA,pp\. 1487–1495\.External Links:ISBN 9781450348874,[Document](https://dx.doi.org/10.1145/3097983.3098043)Cited by:[§1](https://arxiv.org/html/2608.05375#S1.p4.1)\.
- A\. Grattafiori, A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Vaughan,et al\.\(2024\)The llama 3 herd of models\.External Links:2407\.21783,[Document](https://dx.doi.org/10.48550/arXiv.2407.21783)Cited by:[§4\.2](https://arxiv.org/html/2608.05375#S4.SS2.p1.1),[Table 1](https://arxiv.org/html/2608.05375#S5.T1.14.12.12.5)\.
- N\. Hollmann, S\. Müller, L\. Purucker, A\. Krishnakumar, M\. Körfer, S\. B\. Hoo, R\. T\. Schirrmeister, and F\. Hutter \(2025\)Accurate predictions on small data with a tabular foundation model\.Nature637\(8045\),pp\. 319–326\.Cited by:[§2](https://arxiv.org/html/2608.05375#S2.SS0.SSS0.Px2.p1.1),[§4\.2](https://arxiv.org/html/2608.05375#S4.SS2.p1.1),[Table 1](https://arxiv.org/html/2608.05375#S5.T1.6.4.4.5)\.
- M\. Horn, M\. Moor, C\. Bock, B\. Rieck, and K\. Borgwardt \(2020\)Set functions for time series\.InProceedings of the 37th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.119,Virtual,pp\. 4353–4363\.Cited by:[§1](https://arxiv.org/html/2608.05375#S1.p1.1)\.
- F\. Hutter, H\. H\. Hoos, and K\. Leyton\-Brown \(2011\)Sequential model\-based optimization for general algorithm configuration\.InProceedings of the International Conference on Learning and Intelligent Optimization \(LION\),Lecture Notes in Computer Science, Vol\.6683,Berlin, Heidelberg,pp\. 507–523\.External Links:[Document](https://dx.doi.org/10.1007/978-3-642-25566-3%5F40)Cited by:[§1](https://arxiv.org/html/2608.05375#S1.p4.1)\.
- A\. E\. W\. Johnson, L\. Bulgarelli, L\. Shen, A\. Gayles, A\. Shammout, S\. Horng, T\. J\. Pollard, S\. Hao, B\. Moody, B\. Gow, L\. H\. Lehman, L\. A\. Celi, and R\. G\. Mark \(2023\)MIMIC\-IV, a freely accessible electronic health record dataset\.Sci\. Data10\(1\),pp\. 1\(en\)\.Cited by:[§4\.1](https://arxiv.org/html/2608.05375#S4.SS1.p1.1)\.
- Y\. Liet al\.\(2022\)Competition\-level code generation with alphacode\.Science378\(6624\),pp\. 1092–1097\.Cited by:[§1](https://arxiv.org/html/2608.05375#S1.p5.1)\.
- S\. M\. Lundberg and S\. Lee \(2017\)A unified approach to interpreting model predictions\.InAdvances in Neural Information Processing Systems,Vol\.30,Red Hook, NY, USA,pp\. 4765–4774\.Cited by:[§3\.7](https://arxiv.org/html/2608.05375#S3.SS7.SSS0.Px1.p1.12)\.
- A\. Madaan, N\. Tandon, P\. Gupta, S\. Hallinan, L\. Gao, S\. Wiegreffe, U\. Alon, N\. Dziri, S\. Prabhumoye, Y\. Yang,et al\.\(2023\)Self\-refine: iterative refinement with self\-feedback\.Advances in Neural Information Processing Systems36,pp\. 46534–46594\.Cited by:[§1](https://arxiv.org/html/2608.05375#S1.p5.1)\.
- OpenAI \(2023\)GPT\-4 technical report\.External Links:2303\.08774,[Document](https://dx.doi.org/10.48550/arXiv.2303.08774)Cited by:[§1](https://arxiv.org/html/2608.05375#S1.p5.1)\.
- J\. S\. Park, J\. O’Brien, C\. J\. Cai, M\. R\. Morris, P\. Liang, and M\. S\. Bernstein \(2023\)Generative agents: interactive simulacra of human behavior\.InProceedings of the 36th annual acm symposium on user interface software and technology,UIST ’23,New York, NY, USA,pp\. 1–22\.External Links:[Document](https://dx.doi.org/10.1145/3586183.3606763)Cited by:[§1](https://arxiv.org/html/2608.05375#S1.p5.1)\.
- T\. Pham, T\. Tran, D\. Phung, and S\. Venkatesh \(2016\)Deepcare: a deep dynamic memory model for predictive medicine\.InPacific\-Asia conference on knowledge discovery and data mining,Lecture Notes in Computer Science, Vol\.9652,Cham, Switzerland,pp\. 30–41\.External Links:[Document](https://dx.doi.org/10.1007/978-3-319-31750-2%5F3)Cited by:[§2](https://arxiv.org/html/2608.05375#S2.SS0.SSS0.Px1.p1.1)\.
- R\. D\. Riley, K\. I\. Snell, J\. Ensor, D\. L\. Burke, F\. E\. Harrell Jr, K\. G\. Moons, and G\. S\. Collins \(2019\)Minimum sample size for developing a multivariable prediction model: part ii\-binary and time\-to\-event outcomes\.Statistics in medicine38\(7\),pp\. 1276–1296\.Cited by:[§1](https://arxiv.org/html/2608.05375#S1.p1.1)\.
- T\. Schick, J\. Dwivedi\-Yu,et al\.\(2023\)Toolformer: language models can teach themselves to use tools\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Vol\.36,New Orleans, LA, USA,pp\. 68539–68551\.External Links:[Document](https://dx.doi.org/10.52202/075280-2997)Cited by:[§1](https://arxiv.org/html/2608.05375#S1.p5.1)\.
- N\. Shinnet al\.\(2023\)Reflexion: language agents with verbal reinforcement learning\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Vol\.36,New Orleans, LA, USA,pp\. 8634–8652\.External Links:[Document](https://dx.doi.org/10.52202/075280-0377)Cited by:[§1](https://arxiv.org/html/2608.05375#S1.p6.1)\.
- S\. N\. Shukla and B\. M\. Marlin \(2019\)Interpolation\-prediction networks for irregularly sampled time series\.InInternational Conference on Learning Representations,New Orleans, LA, USA\.External Links:1909\.07782,[Document](https://dx.doi.org/10.48550/arXiv.1909.07782)Cited by:[§1](https://arxiv.org/html/2608.05375#S1.p2.1)\.
- S\. N\. Shukla and B\. M\. Marlin \(2021\)Multi\-time attention networks for irregularly sampled time series\.InInternational Conference on Learning Representations,Virtual Conference\.External Links:2101\.10318,[Document](https://dx.doi.org/10.48550/arXiv.2101.10318)Cited by:[§1](https://arxiv.org/html/2608.05375#S1.p1.1)\.
- A\. Singh, A\. Fry, A\. Perelman, A\. Tart, A\. Ganesh, A\. El\-Kishky, A\. McLaughlin, A\. Low, A\. Ostrow, A\. Ananthram,et al\.\(2025\)OpenAI gpt\-5 system card\.Cited by:[§4\.2](https://arxiv.org/html/2608.05375#S4.SS2.p1.1),[Table 1](https://arxiv.org/html/2608.05375#S5.T1.18.16.16.5)\.
- Z\. Song, Q\. Lu, H\. Zhu, D\. Buckeridge, and Y\. Li \(2026\)TrajGPT: irregular time\-series representation learning of health trajectory\.IEEE Journal of Biomedical and Health Informatics30\(5\),pp\. 3888–3899\.External Links:[Document](https://dx.doi.org/10.1109/JBHI.2025.3620205)Cited by:[§2](https://arxiv.org/html/2608.05375#S2.SS0.SSS0.Px1.p1.1)\.
- C\. Thornton, F\. Hutter, H\. H\. Hoos, and K\. Leyton\-Brown \(2013\)Auto\-weka: combined selection and hyperparameter optimization of classification algorithms\.InProceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining,KDD ’13,New York, NY, USA,pp\. 847–855\.External Links:ISBN 9781450321747,[Document](https://dx.doi.org/10.1145/2487575.2487629)Cited by:[§1](https://arxiv.org/html/2608.05375#S1.p4.1)\.
- P\. Trirat, W\. Jeong, and S\. J\. Hwang \(2025\)AutoML\-agent: a multi\-agent LLM framework for full\-pipeline autoML\.InForty\-second International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.267,Vancouver, BC, Canada,pp\. 60099–60146\.Cited by:[§1](https://arxiv.org/html/2608.05375#S1.p5.1),[§2](https://arxiv.org/html/2608.05375#S2.SS0.SSS0.Px3.p1.1),[§4\.2](https://arxiv.org/html/2608.05375#S4.SS2.p1.1),[Table 1](https://arxiv.org/html/2608.05375#S5.T1.22.20.20.5)\.
- M\. Van Smeden, K\. G\. Moons, J\. A\. de Groot, G\. S\. Collins, D\. G\. Altman, M\. J\. Eijkemans, and J\. B\. Reitsma \(2019\)Sample size for binary logistic prediction models: beyond events per variable criteria\.Statistical methods in medical research28\(8\),pp\. 2455–2474\.Cited by:[§1](https://arxiv.org/html/2608.05375#S1.p1.1)\.
- L\. Wang, C\. Ma, X\. Feng, Z\. Zhang, H\. Yang, J\. Zhang, Z\. Chen, J\. Tang, X\. Chen, Y\. Lin,et al\.\(2024a\)A survey on large language model based autonomous agents\.Frontiers of Computer Science18\(6\),pp\. 186345\.Cited by:[§1](https://arxiv.org/html/2608.05375#S1.p5.1)\.
- S\. Wang, M\. B\. McDermott, G\. Chauhan, M\. Ghassemi, M\. C\. Hughes, and T\. Naumann \(2020\)Mimic\-extract: a data extraction, preprocessing, and representation pipeline for mimic\-iii\.InProceedings of the ACM conference on health, inference, and learning,CHIL ’20,New York, NY, USA,pp\. 222–235\.External Links:ISBN 9781450370462,[Document](https://dx.doi.org/10.1145/3368555.3384469)Cited by:[§2](https://arxiv.org/html/2608.05375#S2.SS0.SSS0.Px1.p1.1)\.
- X\. Wang, Y\. Chen, L\. Yuan, Y\. Zhang, Y\. Li, H\. Peng, and H\. Ji \(2024b\)Executable code actions elicit better llm agents\.InForty\-first International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.235,Vienna, Austria,pp\. 50208–50232\.Cited by:[§1](https://arxiv.org/html/2608.05375#S1.p5.1)\.
- X\. Wang, J\. Wei, D\. Schuurmans,et al\.\(2023\)Self\-consistency improves chain of thought reasoning in language models\.InInternational Conference on Learning Representations,Kigali, Rwanda\.External Links:2203\.11171Cited by:[§1](https://arxiv.org/html/2608.05375#S1.p5.1)\.
- F\. Wilcoxon \(1945\)Individual comparisons by ranking methods\.Biometrics bulletin1\(6\),pp\. 80–83\.Cited by:[§6\.2](https://arxiv.org/html/2608.05375#S6.SS2.p3.4)\.
- Z\. Xi, W\. Chen, X\. Guo, W\. He, Y\. Ding, B\. Hong, M\. Zhang, J\. Wang, S\. Jin, E\. Zhou,et al\.\(2025\)The rise and potential of large language model based agents: a survey\. arxiv 2023\.Science China Information Sciences68\(2\),pp\. 121101\.External Links:[Document](https://dx.doi.org/10.1007/s11432-024-4222-0)Cited by:[§1](https://arxiv.org/html/2608.05375#S1.p5.1)\.
- F\. Xie, H\. Yuan, Y\. Ning, M\. E\. H\. Ong, M\. Feng, W\. Hsu, B\. Chakraborty, and N\. Liu \(2022\)Deep learning for temporal data representation in electronic health records: a systematic review of challenges and methodologies\.Journal of biomedical informatics126,pp\. 103980\.Cited by:[§1](https://arxiv.org/html/2608.05375#S1.p2.1)\.
- S\. Yaoet al\.\(2023\)ReAct: synergizing reasoning and acting in language models\.InInternational Conference on Learning Representations,Kigali, Rwanda\.External Links:2210\.03629Cited by:[§1](https://arxiv.org/html/2608.05375#S1.p6.1)\.
- M\. Yuksekgonul, F\. Bianchi, J\. Boen, S\. Liu, P\. Lu, Z\. Huang, C\. Guestrin, and J\. Zou \(2025\)Optimizing generative ai by backpropagating language model feedback\.Nature639\(8055\),pp\. 609–616\.Cited by:[§1](https://arxiv.org/html/2608.05375#S1.p7.1),[§3\.2](https://arxiv.org/html/2608.05375#S3.SS2.SSS0.Px2.p1.4),[§3\.6](https://arxiv.org/html/2608.05375#S3.SS6.SSS0.Px2.p1.2)\.
- C\. Zhou, P\. Liu, P\. Xu, S\. Iyer, J\. Sun, Y\. Mao, X\. Ma, A\. Efrat, P\. Yu, L\. Yu, S\. Zhang, G\. Ghosh, M\. Lewis, L\. Zettlemoyer, and O\. Levy \(2023\)LIMA: less is more for alignment\.InAdvances in Neural Information Processing Systems,Vol\.36,Red Hook, NY, USA,pp\. 55006–55021\.External Links:[Document](https://dx.doi.org/10.52202/075280-2400)Cited by:[§1](https://arxiv.org/html/2608.05375#S1.p5.1)\.

## Appendix AAppendix

### A\.1\.Supplementary Figures

![Refer to caption](https://arxiv.org/html/2608.05375v1/Main_Figures/mort_beeswarm.png)![Refer to caption](https://arxiv.org/html/2608.05375v1/Main_Figures/readm_beeswarm.png)

![Refer to caption](https://arxiv.org/html/2608.05375v1/Main_Figures/los_beeswarm.png)![Refer to caption](https://arxiv.org/html/2608.05375v1/Main_Figures/ra_beeswarm.png)

Figure A1\.Beeswarm plots of top 10 features generated byDoctorAgents\-DS for each task\. A detailed explanation of feature names is provided in Table[A\.2](https://arxiv.org/html/2608.05375#A1.SS2)\.Task\-specific beeswarm plots\.![Refer to caption](https://arxiv.org/html/2608.05375v1/Main_Figures/barplot_reason_feat.png)Figure A2\.Task\-specific distributions of reasonability assigned by GPT\-5\.4 to features generated byDoctorAgents\-DS, ERA, and AutoML\-Agent, aggregated across two independent runs and five cross\-validation folds\.Feature reasoning distribution barplot\.### A\.2\.Supplementary Tables

Detailed information of the top features in the beeswarm plots generated byDoctorAgents\-DS for each task\.Derived FeatureOriginated FromDescription\\endfirstheadTable A0 – continued from previous pageDerived FeatureOriginated FromDescription\\endheadContinued on next page\\endfoot\\endlastfoot\(a\) mortalitybicarbonate\_blood\_delta\_per\_hrbicarbonate\_bloodRate of change in blood bicarbonate over the 48\-hour observation window, computed as the last observed bicarbonate value minus the first observed value divided by the elapsed hours between the first and last measurements\. This feature is only defined when at least two bicarbonate measurements are available\.anchor\_age\_staticanchor\_age\_staticStatic patient age, aggregated at the subject level from the available rows\.gcs\_eye\_lastgcs\_eyeLast observed Glasgow Coma Scale eye\-opening component within the 48\-hour observation window, reflecting the most recent eye\-response neurologic status\.mean\_arterial\_pressure\_mean\_24\_48hmean\_arterial\_pressureMean arterial pressure averaged over the 24–48 hour segment of the observation window\. This summarizes blood pressure status during the later half of the 48\-hour window\.gcs\_verbal\_stdgcs\_verbalStandard deviation of the Glasgow Coma Scale verbal component across the 48\-hour observation window\. It is defined only when at least two verbal GCS measurements are available and reflects variability in verbal neurologic assessment\.anion\_gap\_blood\_worstanion\_gap\_bloodWorst observed anion gap value within the 48\-hour observation window\. Since higher anion gap is treated as worse in this preprocessing logic, this corresponds to the maximum observed anion gap\.rass\_lastrassLast observed Richmond Agitation\-Sedation Scale score within the 48\-hour observation window, reflecting the most recent sedation or agitation status\.heart\_rate\_lastheart\_rateLast observed heart rate within the 48\-hour observation window, representing the most recent cardiovascular status\.gcs\_total\_maxgcs\_totalMaximum observed total Glasgow Coma Scale score within the 48\-hour observation window, reflecting the best recorded overall neurologic status during the window\.n\_glucose\_blood\_meas\_48hglucose\_bloodNumber of non\-missing blood glucose measurements recorded within the 48\-hour observation window, reflecting glucose\-monitoring frequency rather than glucose level itself\.\(b\) readmissionheart\_rate\_count\_log1p\_last6hheart\_rateLog\-transformed number of non\-missing heart\-rate measurements recorded within the final 6 hours before ICU discharge\.heart\_rate\_count\_capped\_last6hheart\_rateCapped number of non\-missing heart\-rate measurements recorded within the final 6 hours before ICU discharge\. For vitals in the 6\-hour window, the count is capped before being retained as a measurement\-frequency feature\.platelet\_count\_blood\_delta\_last48hplatelet\_count\_bloodChange in platelet count over the final 48 hours before ICU discharge, computed as the last observed platelet value minus the first observed platelet value within the 48\-hour window\. It is defined only when at least two platelet measurements are available\.respiratory\_rate\_slope\_last6h\_per\_hrrespiratory\_ratePer\-hour slope of respiratory rate during the final 6 hours before ICU discharge, computed from the difference between the last and first respiratory\-rate values divided by the elapsed time between those measurements\. It is defined only when at least two measurements are available\.diastolic\_bp\_last\_in\_last24hdiastolic\_bpLast observed diastolic blood pressure within the final 24 hours before ICU discharge\.mean\_arterial\_pressure\_count\_capped\_last6hmean\_arterial\_pressureCapped number of non\-missing mean arterial pressure measurements recorded within the final 6 hours before ICU discharge\.mean\_arterial\_pressure\_count\_log1p\_last6hmean\_arterial\_pressureLog\-transformed number of non\-missing mean arterial pressure measurements recorded within the final 6 hours before ICU discharge\.respiratory\_rate\_last\_in\_last6hrespiratory\_rateLast observed respiratory rate within the final 6 hours before ICU discharge\.hemoglobin\_blood\_count\_capped\_last24hhemoglobin\_bloodCapped number of non\-missing hemoglobin measurements recorded within the final 24 hours before ICU discharge\.hemoglobin\_blood\_count\_log1p\_last24hhemoglobin\_bloodLog\-transformed number of non\-missing hemoglobin measurements recorded within the final 24 hours before ICU discharge\.\(c\) LOSfio2\_mean\_last6hfio2Mean fraction of inspired oxygen \(FiO2\) during the final 6 hours before prediction, reflecting recent oxygen\-support intensity\.gcs\_verbal\_lastgcs\_verbalLast observed Glasgow Coma Scale verbal component within the 24\-hour observation window, after intubation\-related verbal assessments are treated as missing\.lactate\_blood\_recency\_hourslactate\_bloodNumber of hours between the prediction time and the most recent lactate measurement within the observation window\.po2\_blood\_recency\_hourspo2\_bloodNumber of hours between the prediction time and the most recent arterial oxygen pressure \(PaO2\) measurement within the observation window\.spo2\_fio2\_ratiospo2 & fio2SpO2/FiO2ratio, used as a non\-invasive oxygenation index\. When available, it is derived from the paired SpO2–FiO2mean ratio over the final 6 hours\.po2\_blood\_mean\_last6hpo2\_bloodMean arterial oxygen pressure \(PaO2\) during the final 6 hours before prediction\.gcs\_total\_count\_log1pgcs\_totalLog\-transformed count of observed total Glasgow Coma Scale measurements within the 24\-hour observation window, reflecting neurologic monitoring frequency\.sf\_ratio\_paired\_mean\_last6hspo2 & fio2Mean paired SpO2/FiO2ratio during the final 6 hours before prediction, where SpO2is paired with the most recent prior FiO2measurement within the allowed temporal gap\.gcs\_total\_lastgcs\_totalLast observed total Glasgow Coma Scale score within the 24\-hour observation window, reflecting latest neurologic status\.gcs\_verbal\_mean\_last6hgcs\_verbalMean Glasgow Coma Scale verbal component during the final 6 hours before prediction, summarizing recent valid verbal neurologic status\.\(d\) RA treatment responseGrFonct\_2\.0GrFonctIndicator that the patient’s baseline functional status category is encoded as 2\.0 \(range 0=good to 4=wheelchair\)\. This captures membership in one specific functional status group rather than using 2\.0 as a numeric threshold\.Fatigue\_std\_365dFatigueVariability of the patient’s fatigue scores across visits within the 365\-day pre\-treatment window\. A higher value indicates more fluctuation in fatigue before treatment initiation, while a value near zero indicates more stable fatigue scores\.SE\_2\.0SEIndicator that the patient carries two shared epitope alleles\.antiCCP\_max\_preantiCCPHighest anti\-cyclic citrullinated peptide \(CCP\) antibody level observed before treatment\. This identifies the strongest recorded serologic signal prior to treatment initiation\.PGA\_last\_prePGAPatient global assessment of disease activity at the closest available pre\-treatment visit\. This reflects the patient’s own assessment of disease activity immediately before treatment\.AntiTNF\_or\_JAK\_First\_3AntiTNF\_or\_JAK\_FirstIndicator that the patient’s first biologic is encoded as 3\. This represents etanercept as opposed to other anti\-TNF drugs namely adalimumab, infliximab, certolizumab and golimumab\.FscPLQ\_delta\_365dFscPLQChange in the platelet count over the 365\-day pre\-treatment window, computed as the last observed value minus the first observed value within that window\. A positive value indicates that the platelet count increased before treatment initiation, while a negative value indicates that it decreased\.SE\_1\.0SEIndicator that the patient carries one shared epitope allele\.EGA\_last\_preEGAEvaluator global assessment of disease activity at the closest available pre\-treatment visit\. This feature reflects the clinician/evaluator’s assessment of disease activity before treatment\.FscPMN\_last\_preFscPMNBaseline value of the absolute neutrophil count in the complete blood count, selected from the closest available pre\-treatment visit\. This feature reflects the patient’s neutrophil count immediately before treatment initiation\.
Table A1\.Characteristics and evaluation settings of the four temporal clinical prediction tasks\.Data SourcePrediction taskObservation WindowPatientsFeaturesPositive RateEvaluation MetricsMIMIC\-IVmortalityfirst 48 hours47634200\.132AUPRCMIMIC\-IVreadmissionat 1st discharge68914820\.087AUPRCMIMIC\-IVlength\-of\-stayfirst 24 hours79753990\.392AUPRCEUPAdrug responsebefore treatment105820\.504AUROC

Table A2\.Full names of the models used in each optimization step generated byDoctorAgentsin Fig\.[3](https://arxiv.org/html/2608.05375#S4.F3)\.Model nameFull settinghgbHistGradientBoostingClassifierextra\_treeExtraTreeClassifierensembleEnsemble of multiple modelslogregLogisticRegressionlogreg\_elaLogisticRegression withpenalty=elasticnetlogreg\_l1LogisticRegression withpenalty=l1logreg\_l2LogisticRegression withpenalty=l2
### A\.3\.Supplementary Pseudo\-code Algorithm

Algorithm 2Pipeline Execution Procedure ofDoctorAgentsInitialization:Repairer𝔸r\\mathbb\{A\}\_\{r\}, retry counterkk, and code execution functionfef\_\{e\}

Input:Pipeline codePeP\_\{e\}and retry limitKK

Output:ResultReR\_\{e\}

1:

f​l​a​g,e​r​r,Re←fe​\(Pe\)flag,err,R\_\{e\}\\leftarrow f\_\{e\}\(P\_\{e\}\)
2:while

f​l​a​g≠PASSflag\\neq\\text\{PASS\}and

k≤Kk\\leq Kdo

3:

Pe←𝔸r​\(Pe,e​r​r\)P\_\{e\}\\leftarrow\\mathbb\{A\}\_\{r\}\(P\_\{e\},err\)
4:

f​l​a​g,e​r​r,Re←fe​\(Pe\)flag,err,R\_\{e\}\\leftarrow f\_\{e\}\(P\_\{e\}\)
5:

k←k\+1k\\leftarrow k\+1
6:endwhile

7:return

ReR\_\{e\}

### A\.4\.Prompt Details forDoctorAgents

In this subsection, we provide details of the prompts for each agent inDoctorAgents, including Consultant \([A\.4\.1](https://arxiv.org/html/2608.05375#A1.SS4.SSS1)\), Generator \([A\.4\.2](https://arxiv.org/html/2608.05375#A1.SS4.SSS2)\), Repairer \([A\.4\.3](https://arxiv.org/html/2608.05375#A1.SS4.SSS3)\), Logger \([A\.4\.4](https://arxiv.org/html/2608.05375#A1.SS4.SSS4)\), Evaluator \([A\.4\.5](https://arxiv.org/html/2608.05375#A1.SS4.SSS5)\), and Domain\-specific Agents \([A\.4\.6](https://arxiv.org/html/2608.05375#A1.SS4.SSS6)\)\.

#### A\.4\.1\.Consultant

System Prompt for ConsultantYou are an expert data analyst specialized in ML datasets\. You will be given some information about a dataset and some sample data from the dataset\.You do not generate code\. Your job is to analyze the dataset with the given information and MUST produce a structured summary strictly in the following exact tags:1\.<TASK\_DESCRIPTION\>: Describe in one concise sentence the machine\-learning task of this dataset including the column name of groundtruth \(if available\), e\.g\. classification, regression, clustering, etc\.</TASK\_DESCRIPTION\>2\.<SUGGESTION\>: Based on your analysis, give practical recommendations for downstream ML tasks, covering:Data preprocessing \(cleaning, dropping, handling missing data, normalization, encoding, etc\.\)Feature engineering ideas \(feature selection, dimension reduction, feature creation etc\.\)Experiment settings \(train/validation splits, cross\-validation etc\.\)Suitable model families or baseline models</SUGGESTION\>Do not include any other text, explanations, or symbols outside of these tags\.

#### A\.4\.2\.Generator

System Prompt for GeneratorYou are an ML Code Generator operating within an agentic machine learning system\.You will be given a task description, task background, feature statistics and metadata, evaluation metrics, and suggestions from a consultant agent\. Your responsibility is to generate one complete, executable Python program that supports the full machine\-learning workflow: data loading, machine\-learning\-oriented data analysis, adaptive preprocessing, model search, hyperparameter exploration, training, evaluation, and artifact generation\.Important columns of the dataset: ID column: \{id\_col\}; label column: \{label\_col\}; time column: \{time\_col\}Output Constraints:\(1\)You must output only one executable Python code\.\(2\)Do not include explanations, comments outside code, markdown, or multiple codes\.In\-code Requirements:\(1\)Ignore warnings\. If you are using OneHotEncoder, Do Not use ’sparse’ as a keyword argument\.\(2\)Load raw train set from \{raw\_train\}; raw validation set from \{raw\_val\}; raw test set from \{raw\_test\}\. All data\-dependent preprocessing must be \*\*based on the train set only\*\*, then applied to the rest sets using the same preprocessing methods\. You must strictly avoid any feature engineering or preprocessing operation that uses, derives from, or is conditioned on the target label\. Do not create, modify, or select features based on the label distribution, label correlations, or label\-specific statistics\. Any form of label\-dependent transformation, leakage, or information encoding is strictly prohibited\.\(3\)At the end, MUST save \(Replace if exists\) a STRUCTURED \*\*json file\*\* to \{training\_stats\_json\_path\} which contains at least four sections: \(i\)\. train\_performance \(a single float value reporting the best model training performance\), \(ii\)\. val\_performance \(a single float value reporting the best model validation performance\), \(iii\)\. test\_performance \(a single float value reporting test performance by re\-training the best model on train \+ val set and testing on test set\), and \(iv\)\. best\_model \(ONLY hyperparameters and name of the best model, no other models should be included\)\. You can include other metrics as well\.\(4\)Avoid using any ’Random Guessing’ type of model\.\(5\)Use the specified metrics to evaluate the performance\.\(6\)Make sure your results are reproducible\.

#### A\.4\.3\.Repairer

System Prompt for RepairerYou are a code engineer that deals with bugs in codes\. Your job is to fix the bugs in a given code with its error message\.Instructions:\- ONLY fix the error, and nothing else\.\- ONLY if necessary, otherwiseDO NOTmake changes to imported packages\.\- Return ONLY the corrected Python code\.

#### A\.4\.4\.Logger

##### System Prompt for Inter\-iterationn​o​t​enoteCreation

System Prompt for LoggerYou are an expert ML engineer and code reviewer\.Your job is to compare two versions of code for the same task and dataset and describe their meaningful differences\.You should:Provide aconcise, structuredcomparison\.\- How the two versions differ indataset preprocessing\(e\.g\., preprocessing, feature engineering, data exploration etc\.\)\.\- How they differ inmodel searching\(e\.g\., algorithm selection, hyperparameter tuning, evaluation methods etc\.\)\.Do notrestate the code\.Do notdescribe general ML concepts\. Focus only on concrete differences\.Be concise, structured, and analytical\.

##### System Prompt for Log Cleaning

System Prompt for Summarization \(SUMMA\)You are an Optimization Notes Curator for an iterative ML code optimization loop\.You will be given a large set of notes describing successive code changes with metadata and performance differences across iterations\. Your only task is to reduce note explosion by selecting some important information to keep and summarizing the rest into a compact, high\-signal record\.Output Rules:1\. Output only the curated notes text\. Do not include any meta commentary, rationale, or explanations\.2\. The output must be derived strictly and exclusively from the provided notes\. Do NOT introduce, infer, assume, or add any information that is not explicitly present in the input notes\.3\. Preserve concrete facts such as numeric performance changes, important hyperparameters, step indices and etc\.Preserve information that is most useful for future optimization decisions:1\. Largest performance improvements/regressions and the changes that caused them2\. Repeated failure modes3\. High\-impact preprocessing/training strategy shifts4\. Evidence of overfitting/leakage/instability5\. Decisions that narrowed or expanded the search spaceRemove or compress:1\. Minor refactors with no measurable impact2\. Redundant observations repeated across iterations

#### A\.4\.5\.Evaluator

System Prompt for EvaluatorYou are an expert clinical ML engineer and code reviewer acting as an evaluator for an agentic machine learning system\.You will be given the task description, task background information, raw feature statistics and metadata, the current code, the specified evaluation metrics, and the history optimization notes\.You must not write or rewrite code, but you may provide pseudocode as suggestions\. Your goal is to critically evaluate the current ML pipeline and provide actionable feedback to improve downstream validation performance\. Your evaluation should be organized into three parts:PART 1, History Analysis•Evaluate the optimization history notes for actionable and meaningful insights, and identify potential reasons for performance increases, drops, stagnation, or abnormal results\.PART 2, Pipeline Assessment•Evaluate whether the preprocessing part uses clinically meaningful and task\-relevant features, applies justified and leakage\-safe feature preprocessing\.•Evaluate whether the training part fulfills the task, uses the specified metrics correctly, and applies reasonable training strategy while detecting overfitting, underfitting, metric inconsistency, or model\-feature mismatch\.PART 3, Improvement Strategy and Next\-Step Recommendation•Identify concrete opportunities for improvement, including alternative preprocessing strategies, feature engineering directions, model choices, and hyperparameter search\.•If validation performance does not show relatively obvious improvement over 5 steps, analyze likely reasons and consider changing the preprocessing or training strategy\.

#### A\.4\.6\.Domain\-specific Experts

##### Data Preprocessing \(DP\) Generator

System Prompt for DP GeneratorYou are a Data Preprocessing Code Generator designed for an agentic ML system\.You will be given a task description, task background, feature statistics, and suggestions from a consultant agent\. Your sole responsibility is to generate one complete, executable code program that performs data loading, machine\-learning\-oriented data analysis, adaptive preprocessing, and artifact generation for downstream model\-training agent\.Role Boundary:\- You should not perform any model training for performance evaluation, unless the purpose is to do feature\-related operations \(e\.g\., train a random forest to obtain feature gains to do selection\)\.Output Constraints:1\. You must output only one executable Python code\.2\. Do not include explanations, comments outside code, markdown, or multiple codes\.In\-code Requirements:1\. Do Not use ’sparse’ as a keyword argument for OneHotEncoder\.2\. \(Strict\) Load raw train set from \{raw\_train\}; raw validation set from \{raw\_val\}; raw test set from \{raw\_test\}\. The raw datasets have exactly the same feature space\. All data\-dependent preprocessing must bebased on the train set only, then applied to the validation and test sets using the same preprocessing methods\. Ensure that the final preprocessed train/validation/test sets have exactly the same feature space \(same column names and number of features\), even if some values or categories do not appear in validation or test\.3\. Label column existence: \(\{label\_col\}\)\. If exists, You must strictly avoid any feature engineering or preprocessing operation that uses, derives from, or is conditioned on the target label\. Do not create, modify, or select features based on the label distribution, label correlations, or label\-specific statistics\. Any form of label\-dependent transformation, leakage, or information encoding is strictly prohibited\.4\. \(Strict\) At the end, MUST save as\{csv\} files \(Replace if exists\) the final preprocessed train set to \{prep\_train\_path\}; the final preprocessed validation set to \{prep\_val\_path\}; the final preprocessed test set to \{prep\_test\_path\}\. Keep ID \(\{id\_col\}\)/Label \(\{label\_col\}\)/Time \(\{time\_col\}\) column names and case unchanged\. Validate your output files are non\-empty\.5\. At the end, MUST save a \(Replace if exists\) concise anddecision\-orientedmetadata JSON file to \{stats\_json\_path\}, intended for downstream LLM\-based agents\. The metadata JSON should satisfy:\- MUST contain a section have\_same\_feature\_space: \(Yes or No\) Checks whether the preprocessed train/validation/test sets have exactly the same feature space \(same column names and number of features\)\.\- Total length no longer than 100 lines\- Any feature\-level information MUST be summarized using Top\-K or grouped aggregates\- Be able to communicatekey preprocessing decisions, data risks, and modeling\-relevant signalsto downstream agents

##### DP Evaluator

System Prompt for DP EvaluatorYou are an expert in clinical ML engineering and temporal clinical data analysis acting as a code reviewer and evaluator fordata preprocessingonly\.You will be given the task description, the task background information, the feature statistics of the raw dataset, the data preprocessing code for evaluation, and the history optimization notes\.You must not write or rewrite code, but you may provide pseudocode as a suggestion\. Your goal is to critically evaluate the current preprocessing pipeline and provide actionable feedback to improve downstream validation performance\. Your evaluation should be organized into three parts:PART 1: History Analysis \* Evaluate the optimization history notes for actionable and meaningful insights, and identify potential reasons for increases or drops in performance\.PART 2: Feature Preprocessing Assessment \* Evaluate whether the selected feature set is clinically meaningful, relevant to the prediction task, and appropriate for the intended clinical objective, instead of indiscriminately using available features\.\* Evaluate any composition, creation, selection, or aggregation of features in the current data preprocessing pipeline to determine whether such operations are justified and aligned with the task intent and clinical insights\.PART 3: Quality, Generalizability, and Risk Assessment \* Evaluate whether the current data preprocessing code fulfills the intended task, and assess it comprehensively beyond correctness, including quality, robustness, generalization, interpretability, and scalability\.\* Evaluate whether the current preprocessing can generalize to new or unseen data and adapt to potential distribution shifts\.\* Detect potential data leakage and preprocessing\-induced overfitting risks\.\* Identify opportunities for improvement, alternative preprocessing strategies, and potential risks\.\*Important:If the validation performance does not show relatively obvious improvements over 5 steps, analyze the potential reasons and consider changing the preprocessing strategies\.

##### Model Development \(MD\) Generator

System Prompt for MD GeneratorYou are a Model Development Code Generator operating within an agentic machine learning system\.You will be given a task description, task background, a preprocessed dataset path, the evaluation metrics, and suggestions from a consultant agent\. You are responsible for generating one complete, executable training program that consumes a preprocessed dataset and its accompanying metadata, performs appropriate model search, hyperparameter exploration, and training, evaluates performance using strictly constrained metrics to select the best model, and writes structured training artifacts for downstream use\.Role Boundary:\- You should not perform any general preprocessing \(e\.g\., imputation, feature creation or removal\)\.\- You should not modify the dataset schema except for model\-required transformations \(e\.g\., standardization, normalization, model\-specific scaling\)\.Output Constraints:1\. You must output only one executable Python code\.2\. Do not include explanations, comments outside code, markdown, or multiple codes\.In\-code Requirements:1\. Do Not use ’sparse’ as a keyword argument for OneHotEncoder\.2\. Load train set from \{prep\_train\_path\}; validation set from \{prep\_val\_path\}; test set from \{prep\_test\_path\}\. The datasets have exactly the same feature space\. Use the train set for model training\. Use the validation set for tuning hyperparameters, evaluating performance, and selecting the best model\. After selecting the best model, refit it on the combined training \+ validation data, then perform a single final evaluation on the test set\.3\. The capitalization of ID \(\{id\_col\}\)/Label \(\{label\_col\}\)/Time \(\{time\_col\}\) column names in the preprocessed dataset could vary, identify them regardless of case\.4\. \(Strict\) At the end, MUST save \(Replace if exists\) a STRUCTUREDjson fileto \{training\_stats\_json\_path\} which contains at least four sections: \(i\) train\_performance \(a single float value reporting the best model training performance\), \(ii\) val\_performance \(a single float value reporting the best model validation performance\), \(iii\)\. test\_performance \(a single float value reporting the best model test performance\), and \(iv\) best\_model \(ONLY hyperparameters and name of the best model\)\.5\. At the end, MUST use the trained best model \(IMPORTANT: This should NOT influence the selection scheme of best model above\) to compute the SHAP values on validation set \(if validation set size≥\\geq300, then use maximum 300 stratified subsamples for computation\) for each feature \(check: If SHAP computation failed, use another trained model for computation instead\. If all models failed, output empty CSV\), and save as a csv file to shap\_csv\_path with two columns: 1\. feature\_name \(the name of each feature in the dataset\), 2\. abs\_mean\_shap \(the mean value of absolute SHAP\), the CSV file should be ordered by descending abs\_mean\_shap\. You should also compute a beeswarm plot of the top 10 \(if number of features≤\\leq10, plot all features\) important features \(ranked by abs\_mean\_shap\) and save to shap\_beeswarm\_path\.6\. Avoid using any ’Random Guessing’ type of model\.7\. Use the specified metrics to evaluate the performance\.8\. Make sure your results are reproducible\.

##### MD Evaluator

System Prompt for MD EvaluatorYou are an expert ML engineer acting as a code reviewer and evaluator formodel trainingonly\.You will be given the task description, the specified evaluation metrics, the metadata generated from data preprocessing, the model training code for evaluation, and the history optimization notes\.The given model training code will load preprocessed data\.You must not write or rewrite code, but you may provide pseudocode as a suggestion\. Your goal is to critically evaluate the current model training pipeline and provide actionable feedback to improve downstream validation performance\. Your evaluation should be organized into four parts:PART 1: History Analysis \* Evaluate the optimization history notes for actionable and meaningful insights\.\* Detect any failure modes from the history records, such as abnormally low or zero performance scores, incorrect evaluation metrics, divergence, vanishing gradients, class imbalance, improper normalization, or other training issues\.PART 2: Training Pipeline Correctness and Quality \* Evaluate whether the current model training code fulfills the intended task, and assess it comprehensively beyond correctness, including code quality, robustness, generalization, interpretability, and scalability\.\* Evaluate whether the current model training pipeline uses appropriate and correctly applied feature scaling and transformations required by the chosen model type\.PART 3: Model Selection, Hyperparameters, and Training Behavior \* Evaluate whether the current hyperparameter settings are appropriate and whether the model selection criteria are appropriate, consistent, and justified\.\* Detect potential overfitting or underfitting issues\.PART 4: Improvement Strategy and Next\-Step Recommendation \* Identify opportunities for improvement, including alternative model choices, hyperparameter search directions, and potential risks\.\*Important:If the validation performance does not show relatively obvious improvements over 5 steps, analyze the potential reasons and consider changing the choice of model family\.

### A\.5\.Experimental Details

#### A\.5\.1\.Temporal Clinical Prediction Tasks

All four clinical tasks were represented as temporal data with multiple observations per patient\. The three MIMIC\-IV cohorts included only adults and were constructed using task\-specific eligibility criteria\. For mortality prediction, we included adult patients with a first eligible ICU stay and used observations from the first 48 hours to predict in\-ICU mortality, excluding patients who died within this observation window\. For readmission prediction, we used all observations before discharge from the first eligible ICU stay to predict ICU readmission within seven days and excluded patients who died before or at discharge\. For LOS prediction, we used observations from the first 24 hours of the first eligible ICU stay to predict whether the total ICU stay exceeded three days, excluding stays that did not contain the complete observation window\. After cohort selection and restriction to the corresponding observation windows, no manual aggregation, imputation, scaling, or feature engineering was applied\.

The RA task involved 105 biologic\-naïve patients initiating anti–TNF therapy\. The in\-house dataset contained longitudinal pre\-treatment clinical assessments\. Empty rows and features that were constant across all patients were removed, resulting in 82 features; no additional preprocessing or feature engineering was performed manually\.

For all LLM\-based methods, the agents were allowed to autonomously determine how to preprocess the temporal observations and construct patient\-level representations\. TabPFN was the only exception because it does not support temporal input directly; it therefore received the last available value within the observation window for each feature for the MIMIC tasks, corresponding to the last pre\-treatment visit for the RA task\.

#### A\.5\.2\.Prompts for other LLM\-based baselines

##### Instruction prompt for GPT\-5, GPT\-oss, and Llama\-8B

For testing GPT\-5, GPT\-oss, and Llama\-8B, we simply used the same prompt and input as Generator \([A\.4\.2](https://arxiv.org/html/2608.05375#A1.SS4.SSS2)\) inDoctorAgentsfor each dataset to generate the pipeline\.

##### Instruction prompt for AutoML\-Agent

The following provides the prompt details for running AutoML\-Agent on each task\. The contents inBOLDwill change according to current task\.

AutoML\-Agent prompt detailsOutput ONE executable Python program only to fulfill the following Machine Learning Task\.TASK:Binary classification to predictLABEL\_COL\(1/0\) using tabular clinical data\.BACKGROUND:TASK\_METADATA FILES:\- Train CSV:TRAIN\_CSV\_PATH \- Val CSV:VAL\_CSV\_PATH \- Test CSV:TEST\_CSV\_PATH COLUMNS:\- ID column:ID\_COL\(must NOT be used as a feature\)\- Label column:LABEL\_COL \- Optional time column:TIME\_COL CRITICAL: Observation deduplication \(MUST EXACTLY FOLLOW\)Implement a function dedup\_visits\(df\):\- Use appropriate aggregation method and apply to allID\_COL\.\- After dedup, output must have exactly one row perID\_COL\.MODEL SELECTION & EVALUATION \(MUST\):\- Use ONLY training data for data preprocessing\.\- Select the best pipeline by validationMETRICS\.\- Refit the best pipeline on train \+ val data\.\- Evaluate ONCE on the test set\.METRICS TO REPORT ON TEST \(MUST\):accuracy, auroc, auprc, f1\_score, precision, recallOUTPUT FILE \(MUST\):Write outer\_test\_metrics\.json containing:[⬇](data:text/plain;base64,ewogICJhY2N1cmFjeSI6IGZsb2F0LAogICJhdXJvYyI6IGZsb2F0LAogICJhdXByYyI6IGZsb2F0LAogICJmMV9zY29yZSI6IGZsb2F0LAogICJwcmVjaXNpb24iOiBmbG9hdCwKICAicmVjYWxsIjogZmxvYXQsCiAgIm5fdGVzdCI6IGludAp9)\{"accuracy":float,"auroc":float,"auprc":float,"f1\_score":float,"precision":float,"recall":float,"n\_test":int\}HARD CONSTRAINTS:\- After dedup: train and test each have uniqueID\_COL\.\- Do not useLABEL\_COLfor any feature engineering, feature selection, or preprocessing decisions \(no target\-dependent transforms\)\.IMPLEMENTATION REQUIREMENTS:\- Build a robust preprocessing pipeline\.\- Model search should try multiple reasonable candidates\.\- Hyperparameter exploration should be performed\.PROGRAM OUTPUT \(MUST save toOUTPUT\_PATH\):\- Print the best model name and performance\.\- Print test metrics\.\- Save outer\_test\_metrics\.json\.Return only the Python code, no markdown, no explanation\.

##### Instruction prompt for ERA

The following provides the prompt details for running ERA on each task\. The contents inBOLDwill change according to current task\.

ERA prompt detailsTASK\_OBJECTIVE Here is a preview of the training data:DATA\_PREVIEW The following information provides metadata for the task, including dataset shape, number of unique patients, and positive rate:TASK\_METADATA The goal is to predict ’LABEL\_COL’\. The metric isMETRICS\.\(Higher / Lower\) is better\.The previous solution had a score of:SCORE Previous Solution Code:“‘pythonPARENT\_CODE “‘Meta columns in the dataset:Subject ID column: ’ID\_COL’; Label column: ’LABEL\_COL’; Time column: ’TIME\_COL’Please generate a NEW, IMPROVED Python function named ‘train\_and\_predict‘ that:1\. Accepts ‘train\_path‘ and ‘test\_path‘ as strings\.2\. Before training machine learning models, convert the data \(this dataset is sparse and provided in time\-series format, with time column = ’TIME\_COL’\) into a clinically meaningful tabular representation using appropriate feature preprocessing strategies guided by clinical insight and the task objective\.The same preprocessing strategy should not be blindly applied to all features\. Instead, evaluate which preprocessing methods are appropriate for each feature based on its clinical meaning, temporal behavior, and relevance to the prediction task\. Possible strategies include, but are not limited to:\- \*\*Feature aggregation\*\*: For time\-varying clinical features, consider aggregation methods such as mean, minimum, maximum, standard deviation, last observed value, and missingness indicators\. Select only the aggregation\(s\) that are clinically meaningful for each feature\. For example, some features may be best represented by their most recent value, while others may be better summarized by their maximum, variability, or missingness pattern\.\- \*\*Feature selection\*\*: You are allowed to manually select clinically meaningful features for the task based on domain knowledge and task intent, rather than using all available features indiscriminately\.\- \*\*Feature composition\*\*: You may create derived or composite features, such as clinically relevant biomarkers, scores, ratios, trends, or interactions, if they are meaningful for the prediction task\.You are not limited to the above strategies\. Use any clinically justified preprocessing approach that helps transform the time\-series data into an informative tabular feature set for machine learning\.Note that any of your preprocessing should be based on the train set ONLY, and then applied to the test set, otherwise it’s considered as data leakage and your result will not be meaningful\. Also make sure that after preprocessing both sets should have the exact same feature space \(same columns\)\.3\. Trains a set of classification models to select the best one\.4\. Returns the probability predictions for the test set as a numpy array or list\.5\. You can use pandas, numpy, scikit\-learn, or other packages as needed\.IMPORTANT: DO NOT use ‘xgboost‘ or ‘lightgbm‘\.Your code must look like this:[⬇](data:text/plain;base64,aW1wb3J0IHBhbmRhcyBhcyBwZAppbXBvcnQgbnVtcHkgYXMgbnAKIyAuLi4gb3RoZXIgaW1wb3J0cwoKZGVmIHRyYWluX2FuZF9wcmVkaWN0KHRyYWluX3BhdGgsIHRlc3RfcGF0aCk6CiAgICAjIExvYWQgZGF0YQogICAgdHJhaW4gPSBwZC5yZWFkX2Nzdih0cmFpbl9wYXRoKQogICAgdGVzdCA9IHBkLnJlYWRfY3N2KHRlc3RfcGF0aCkKCiAgICAjIC4uLiBGZWF0dXJlIEVuZ2luZWVyaW5nIC4uLgogICAgIyAuLi4gVHJhaW5pbmcgLi4uCgogICAgIyBQcmVkaWN0CiAgICBwcmVkaWN0aW9ucyA9IC4uLgogICAgcmV0dXJuIHByZWRpY3Rpb25z)importpandasaspdimportnumpyasnpdeftrain\_and\_predict\(train\_path,test\_path\):train=pd\.read\_csv\(train\_path\)test=pd\.read\_csv\(test\_path\)predictions=\.\.\.returnpredictionsProvide the full, runnable code including imports\.IMPORTANT CONSTRAINTS FOR SPEED:1\. DO NOT use GridSearchCV or RandomizedSearchCV\.2\. If using RandomForest or Boosting, set ‘n\_estimators‘ to maximum 50\.3\. Keep the model lightweight \(execution time limit is 60 seconds\)\.

##### Instruction prompt for classifying Feature Reasonability

The following provides the prompt details for feature reasonability judgment on each task\. The contents inBOLDwill change according to current task\.

Prompt for GPT\-5\.4 as reasonability judgeYou are evaluating the reasonableness of composed features generated by an automated data\-preprocessing agent\.Your task is to classify every supplied feature into exactly one of five levels\. Evaluate each feature internally using the dataset background, prediction task, temporal setting, and preprocessing code\.Do not assess a feature only from its name when its exact definition can be identified from the preprocessing code\.You must assess each feature independently\.DATASET BACKGROUND AND TASK SETTING———————————–DATASET\_BACKGROUND\_AND\_TASK\_SETTING All input rows have already been restricted to the valid observation window before this preprocessing code is executed\.CLASSIFICATION STANDARD———————–Level 1 — InvalidThe feature has a clear and serious defect and should not be used\.Assign this level when any of the following applies:\- It’s clinically irrational or unreasonable\.\- Its mathematical definition is incorrect\.\- Its source variables are semantically incompatible\.\- It cannot be calculated at the intended deployment time\.\- It’s produced from identifiers or other inappropriate variables\.\- Its construction contradicts the stated task setting or observation window\.Level 2 — Weakly JustifiedThe feature can technically be calculated and has no obvious leakage, but its clinical, mathematical, or task\-specific justification is weak\.Assign this level when:\- The relationship between the source variables is unclear\.\- The arithmetic operation appears arbitrary\.\- The feature lacks an interpretable clinical or statistical meaning\.\- Its units or scale are difficult to justify\.\- It appears to be an opportunistic interaction without a credible mechanism\.\- There is insufficient rationale to support its construction\.Level 3 — Plausible but UncertainThe feature has a potentially reasonable clinical or statistical interpretation, but important uncertainty remains\.Assign this level when:\- The general relationship is plausible, but the exact formula is not well supported\.\- The feature may be sensitive to missingness, outliers, irregular measurement frequency, small denominators, or unstable values\.\- It’s meaningful only for a subset of patients\.\- Its relevance to the task is plausible but requires empirical validation\.\- Its construction is defensible but not clearly established\.Level 4 — Well JustifiedThe feature has a clear clinical, physiological, statistical, or task\-specific interpretation and is appropriately constructed for the given prediction problem\.Assign this level when:\- The operation is mathematically and semantically appropriate\.\- The feature clearly represents a meaningful state, trend, change, duration, recency, burden, variability, or interaction\.\- The feature is relevant to the stated prediction task\.\- It’s not encoded from identifier features\.\- Its interpretation is clear even if it is not a formally established clinical measure\.Level 5 — Clinically ReasonableThe feature corresponds to an established or well\-recognized clinical construct, medical formula, score component, physiological relationship, or strongly supported biomarker relationship\.Assign this level when:\- The construction is clinically recognized or strongly supported\.\- Its mathematical definition follows an established medical concept\.\- Its inputs and timing are valid for the current task\.Examples may include established ratios, clinical indices, standard differences, recognized physiological relationships, or components of validated clinical scores\.FEATURES TO BE CLASSIFIED————————FEATURE\_LIST PREPROCESSING CODE——————PREPROCESSING\_CODE REQUIRED OUTPUT FORMAT———————\-Return exactly one JSON object and nothing else\.Each key must be one feature name copied exactly from the supplied feature list\. Each value must be exactly one of these level names:\[”Invalid”,”Weakly Justified”,”Plausible but Uncertain”,”Well Justified”,”Clinically Reasonable”,\]Example output format:[⬇](data:text/plain;base64,ewoiZmVhdHVyZV9hIjogIldlbGwgSnVzdGlmaWVkIiwKImZlYXR1cmVfYiI6ICJJbnZhbGlkIiwKImZlYXR1cmVfYyI6ICJQbGF1c2libGUgYnV0IFVuY2VydGFpbiIKfQ==)\{"feature\_a":"WellJustified","feature\_b":"Invalid","feature\_c":"PlausiblebutUncertain"\}

### A\.6\.Examples of intermediate output for RA dataset

This section presents representative examples of intermediate outputs produced by the agents\. These include outputs from Consultant \([A\.6\.1](https://arxiv.org/html/2608.05375#A1.SS6.SSS1)\) and the summarized note logs from Logger \([A\.6\.6](https://arxiv.org/html/2608.05375#A1.SS6.SSS6)\), which are shared by both the defaultDoctorAgentsandDoctorAgents\-DS\. In addition, we showDoctorAgents\-DS\-specific outputs, including data preprocessing \(DP\) evaluation \([A\.6\.2](https://arxiv.org/html/2608.05375#A1.SS6.SSS2)\) from DP Evaluator and its corresponding backpropagated feedback \([A\.6\.3](https://arxiv.org/html/2608.05375#A1.SS6.SSS3)\), as well as model development \(MD\) evaluation \([A\.6\.4](https://arxiv.org/html/2608.05375#A1.SS6.SSS4)\) from MD Evaluator and its corresponding backpropagated feedback \([A\.6\.5](https://arxiv.org/html/2608.05375#A1.SS6.SSS5)\)\.

#### A\.6\.1\.Consultant Output

Task DescriptionPredict whether a rheumatoid arthritis patient will respond to their first anti‑TNF treatment \(binary label column: Responder\) using pre\-treatment longitudinal clinical/lab/demographic data, with patient identifier NPatient and irregular visit time DReelle\.

SuggestionData preprocessing \(including keeping ONLY one visit per patient\):•Patient\-level unit of analysis: ensure exactly 1 row per NPatient to avoid leakage across visits and to match label definition\.–Define an “index date” for each patient as the anti\-TNF start/baseline anchoring date; if an explicit treatment start date is not available, approximate using the closest pre\-treatment visit captured by baseline\_time\_to\_DateDeb or by selecting the visit with minimum baseline\_time\_to\_DateDeb \(closest to treatment initiation\) subject to being pre\-treatment\.–Primary single\-visit strategy \(recommended baseline\): choose the last pre\-treatment visit closest to treatment initiation \(max DReelle among eligible pre\-treatment visits\), because it best reflects baseline status at decision time\.–Alternative single\-visit strategies to compare in validation:\*Earliest qualifying baseline visit within the allowed window \(to reduce “sicker\-at\-start” bias if late visits are triggered by flare\)\.\*Clinically stable window: if multiple visits occur very close to initiation, take median/mean within a short window \(e\.g\., 30–90 days\) and retain as a single baseline snapshot\.–After selecting the visit, drop DReelle from features \(or keep only derived time deltas already provided\) to prevent the model learning calendar\-time artifacts\.•Missing data / sparsity handling \(feature\-type specific\):–Separate feature groups: labs \(Fsc\*, Crp, Esr, \_1433eta\_\*\), disease activity/PROs \(DAS28CRP\_4, PGA, EGA, Fatigue, Sommeil, M\_HAQ, Raideur\), comorbidities \(many 0/1\), serologies \(FR, antiCCP\), treatments/meds \(Pred\_before\_TNF, MTX\_before\_TNF, plaquenil\_atbaseline\), demographics \(Age\_V1, Sexe, IMC, ruralurban, Scolar\), durations \(Duree\_symp\_mois, time\_first\_symptom\_to\_DateDeb, baseline\_time\_to\_DateDeb\)\.–For sparse continuous labs: avoid blanket imputation; use \(a\) missingness indicator per lab and \(b\) robust imputation \(median\) or clinically meaningful “not measured” category via indicator\. Consider dropping labs with extremely high missingness \(e\.g\., ¿80–90%\) unless clinically critical\.–For binary comorbidity flags: treat missing as “unknown” rather than 0 when appropriate; add an “unknown” indicator or impute conservatively depending on how data were recorded\.–For ordinal/categorical variables \(e\.g\., StatFum, StatMar, Emploi, SE, AntiTNF\_or\_JAK\_First\): check coding \(numeric codes vs true numeric\); encode as categorical \(one\-hot or target/ordinal encoding depending on cardinality and sample size\)\.–Outliers: winsorize/heavy\-tail robust scaling for Crp/Esr and other inflammatory markers; check implausible IMC, blood counts\.–Normalization: standardize continuous predictors \(robust scaler\) for linear models; tree\-based models can use raw scales but still benefit from outlier handling\.•Leakage checks:–Ensure all features are truly pre\-treatment; exclude any columns that could encode post\-baseline outcome timing\.–Avoid using any follow\-up DAS28 information beyond baseline \(label is derived from follow\-up DAS28\)\.Feature engineering \(clinically guided, sparse\-aware\):•Baseline\-focused features \(from the chosen single visit\):–Core RA severity set \(high priority\): DAS28CRP\_4, Crp, Esr, PGA, EGA, M\_HAQ, Raideur, Fatigue, Sharp\_total \(damage\), IMC, Age\_V1, Sexe, Duree\_symp\_mois\.–Serostatus: antiCCP \(consider positive/negative thresholding \+ titer if reliable\), FR \(often skewed; log\-transform or categorize\), plus “seropositive” composite \(antiCCP or FR positive\)\.–Inflammation composite: log\(Crp\+1\), log\(Esr\+1\), and/or a simple z\-score sum; consider interaction with serostatus\.–Medication/context: Pred\_before\_TNF, MTX\_before\_TNF, plaquenil\_atbaseline, AntiTNF\_or\_JAK\_First \(treatment class/choice\), time\_first\_symptom\_to\_DateDeb and baseline\_time\_to\_DateDeb as timing/diagnostic delay proxies\.•Missingness as signal: for key labs \(Crp, Esr, blood counts, 14\-3\-3nn\), include “measured vs not measured” indicators; in sparse EHR\-like settings this can be predictive\.•Dimensionality reduction/selection:–Prefer a curated feature subset over all 84 columns; remove near\-constant binary flags and highly collinear measures \(e\.g\., overlapping activity measures\)\.–Consider stability selection or L1\-regularized logistic regression for embedded feature selection, but keep clinically essential variables regardless\.Experiment settings \(AUROC; small N; repeated visits\):•Split at patient level only \(grouped by NPatient\) to prevent leakage\.•Because only ~105 patients, use repeated stratified GroupKFold \(e\.g\., 5 folds, repeated\) or nested CV \(inner tuning, outer evaluation\) to get stable AUROC estimates\.•Maintain class balance per fold \(positive rate ~0\.5\)\.•Report mean AUROC with confidence intervals \(bootstrap at patient level\) due to small sample\.Suitable model families / baselines:•Baselines: \(1\) regularized logistic regression \(L1/L2/elastic net\) on curated features; \(2\) gradient\-boosted trees \(XGBoost/LightGBM/CatBoost\) with careful handling of missingness; \(3\) random forest as a secondary baseline\.•If using categorical\-heavy features with missingness, CatBoost is a strong candidate; if using linear models, ensure proper encoding and scaling\.•Keep models simple to reduce overfitting; calibrate probabilities \(Platt/isotonic\) if decision thresholds matter, while still optimizing AUROC for comparison\.

#### A\.6\.2\.DP Evaluator Output \(DoctorAgents\-DS\)

DP EvaluationPART 1 — History analysis \(why performance moved\)\* Step 6→\\rightarrowStep 7 preprocessing likely caused a large signal collapse: your own QA shows 55/58 train patients have anchor\_mode=”missing” and no\_prestart=55, forcing static\-only features for almost everyone\. That explains:\* SHAP being dominated by socioeconomic/education \+ drug choice \(SE/Scolar/AntiTNF\) rather than disease activity/inflammation\.\* Validation AUROC staying mediocre \(∼\\sim0\.57\) and not improving despite more careful leakage controls\.\* The Step 7 anchoring is too strict for this dataset as currently encoded: requiring BTT within \[0,365\] and enough points basically eliminates longitudinal data for most patients, so the model can’t benefit from the clinically most relevant predictors \(baseline disease activity, CRP/ESR, trajectory\)\.\* Net: you reduced leakage risk but over\-pruned usable pre\-treatment information, causing underfitting on clinical signal and over\-reliance on demographic proxies\.PART 2 — Feature preprocessing assessment \(clinical alignment\)What is good / clinically aligned\* Chosen domains are appropriate: baseline disease activity \(DAS28/PGA/EGA\), inflammation \(CRP/ESR\), CBC, HAQ/PROs, serology \(RF/anti\-CCP\), meds \(MTX/pred\), comorbidity\. Two windows \(0–90, 91–365\) are clinically sensible\.What is currently misaligned / losing clinical value1\. Anchoring logic prevents using those clinically meaningful features for most patients, so the “intended” feature set rarely materializes\.2\. Static\-only fallback discards meds/comorbidities entirely \(Pred\_before\_TNF, MTX\_before\_TNF, etc\. set to NaN\)\. Those are not time\-series in the same sense and are often known at/around start; dropping them when anchor fails is unnecessary information loss\.3\. Binary coercion is too strict: \_coerce\_binary drops anything not exactly 0,1\. This dataset historically contains ‘unknown” encodings \(e\.g\., \-1\) \(you even saw Mpoc\_\-1 show up in Step 6 SHAP\)\. Current handling likely turns informative ‘unknown/NA\-coded” states into missing and then median\-imputes them away\.4\. Serology quantile bin feature is computed but effectively unused:\* \_add\_serology\_bins\(\) adds FR\_\_latest\_pre\_\_qbin / antiCCP\_\_latest\_pre\_\_qbin\* but \_fit\_preprocessor\(\) determines keep\_num from train\_pat before those columns exist, so they never enter feature\_cols\.\* Net: extra complexity without benefit\.Clinically important missing engineered signals \(given sparsity\)\* For sparse labs, often the most predictive features are \(a\) baseline/nearest\-to\-start value, \(b\) ‘ever measured” and \(c\) time\-to\-start of that value\. You attempted this, but anchoring failure removes it\. Also consider explicitly keeping ‘baseline vs older” indicators rather than dropping them\.PART 3 — Quality, generalizability, risk assessment \(and how to improve\)A\) Highest\-impact issue: anchoring/generalizability failure\* Your strict plausibility filter \(0≤\\leqBTT≤\\leq365\) is causing systematic missing anchors and therefore systematic missing longitudinal features\. This is the main reason you’re not getting\>\>3\-step improvements\.\* Strong suspicion: baseline\_time\_to\_DateDeb is not per\-row “days\-to\-start”, or is constant/misaligned across visits\. The preview shows implausibly large constants \(e\.g\., 1797 days\), which would trip your plausibility gate even if the patient truly has pre\-treatment visits\.Actionable fix \(conceptual/pseudocode\)\* Estimate a patient\-level start date even when BTT is\>\>365, then compute visit\-level days\_to\_start from dates:\* start\_date\_i = robust\_median\(DReelle \+ baseline\_time\_to\_DateDeb\) using all non\-missing rows \(or a trimmed median\), without restricting BTT to≤\\leq365\.\* then days\_to\_start = \(start\_date\_i \- DReelle\)\.days\* define pre\-treatment rows as days\_to\_start≥\\geq0; then windows apply as usual \(≤\\leq365 etc\.\)\.\* Add an anchor quality score \(spread of candidate start dates\) but do not zero\-out all longitudinal features unless quality is truly unusable; instead, downweight by adding a feature like anchor\_uncertainty\_days and let the model learn\.This change alone should turn QA from ‘55 static\-only” to ‘most patients have some prestart rows”, unlocking the core predictors\.B\) Leakage/overfitting risks to keep in check\* Process/measurement density features \(n\_measured, n\_visits\) can encode care patterns/site practice\. They can help but harm transportability\. Prefer keeping them but:\* restrict to a small set \(you did\), and\* consider expressing them as coarse bins \(0/1/2\+\) rather than raw counts\.\* AntiTNF\_or\_JAK\_First: if the clinical objective is ‘predict response after clinician has chosen agent”, it’s fine; if the objective is ‘predict response to first anti\-TNF irrespective of which is chosen”, this becomes a treatment\-assignment proxy and can inflate internal performance while reducing causal interpretability\.C\) Robustness/scalability issues\* Current pipeline is computationally fine for this dataset\.\* But it’s brittle to encoding quirks:\* handle \-1,0,1 explicitly for “binary” clinical flags \(unknown vs no\)\.\* avoid treating “unknown\-coded numeric” as missing and then median\-imputing to 0\.D\) Concrete improvement checklist \(preprocessing only\)1\. Rework anchoring to recover pre\-treatment rows for most patients \(patient\-level start date from DReelle \+ baseline\_time\_to\_DateDeb, then derive days\_to\_start\)\.2\. Do not drop meds/comorbidities in static\_only mode; keep them as baseline static \(with missing indicators if needed\)\.3\. Relax binary coercion to preserve “unknown” state \(e\.g\., map \-1 to a separate category or a separate indicator\)\.4\. Either remove serology \_\_qbin or include it properly in the kept numeric/categorical set \(right now it’s dead code\)\.5\. Given extreme sparsity, avoid dropping engineered features solely due to missingness when missingness is expected; instead keep a smaller number of high\-value engineered features \(baseline/last \+ measured\_any \+ days\_to\_start\_last\) for CRP/ESR/DAS28/HAQ, and drop the rest \(std/mean in tiny windows often becomes mostly noise\)\.If you implement only one change: fix anchoring so you’re not static\-only for∼\\sim95% of patients—that’s the bottleneck currently limiting validation AUROC\.

#### A\.6\.3\.DP Backpropagation Output \(DoctorAgents\-DS\)

DP Feedback1\. Fix the core failure mode: anchoring is eliminating longitudinal signal for∼\\sim95% of patientsYour own QA \(patients\_anchor\_missing=55/58, no\_prestart=55/58\) implies almost everyone becomes “static\-only”, so the pipeline can’t leverage the clinically strongest predictors \(DAS28/CRP/ESR/HAQ\)\. This is likely the dominant reason AUROC is stuck∼\\sim0\.57 and SHAP is dominated by SE/Scolar/AntiTNF\.What to change in code behavior\* Stop requiring baseline\_time\_to\_DateDeb \(BTT\) to be within \[0,365\] to estimate the anchor\. In this dataset, BTT appears often\>\>\>\>365 \(preview shows 1797\), so your plausibility gate systematically fails\.\* Compute a patient\-level start date using robust aggregation of DReelle±\\pmBTT even when BTT is large, then derive days\_to\_start = start\_date \- DReelle\. Only after you have days\_to\_start should you restrict to pre\-treatment windows \(0–365d\)\.\* Keep anchor\_quality features \(spread/IQR of candidate start dates, \#rows supporting\) rather than hard “missing anchor→\\rightarrowdrop everything”\.Why it helps the metricYou’ll move from ‘static\-only majority” to ‘most patients have at least one pre\-start row”, unlocking baseline/recency features that are plausibly predictive of response\.—2\. Remove unnecessary information loss in static\_only fallback \(currently harms AUROC\)When no pre\-start rows are found, you set meds and comorbidities to NaN:[⬇](data:text/plain;base64,aWYgc3RhdGljX29ubHkgPiAwLjA6CmZvciBjIGluIG1lZHNfYmluYXJ5ICsgY29tb3JiOiByb3dbY10gPSBucC5uYW4=)ifstatic\_only\>0\.0:forcinmeds\_binary\+comorb:row\[c\]=np\.nanBut these are not truly time\-series dependent on anchoring; they’re baseline\-known covariates \(and were useful historically\)\.Change\* Always populate meds/comorbidities from the chosen “base” row \(or any available row\), even if anchor is uncertain\.\* Add missingness indicators for them if you’re worried about contamination/availability, rather than nulling them\.WhyYou’re throwing away stable signal precisely for the hardest patients \(no reconstructed pre\-start\), increasing noise and reliance on socio\-demographic proxies\.—3\. Binary coercion is too strict; you’re silently deleting “unknown/other” states\_coerce\_binary only accepts exact 0,1; everything else becomes NaN and then median\-imputed\. If the raw data uses \-1 or other sentinel codes \(seen in prior steps\), this collapses informative ‘unknown” vs ‘no”\.Change\* For “binary\-like with unknown codes”, represent as 3\-state \(0/1/Unknown\) either:\* categorical with OHE, or\* numeric with an explicit \_\_unknown indicator\.\* At minimum, treat \-1 distinctly from missing \(don’t coerce to NaN\)\.WhyPreserving “unknown” often helps in sparse EHR\-like data; deleting it can flatten predictive structure\.—4\. Your serology quantile bins are dead code \(adds complexity, no signal\)You create FR\_\_latest\_pre\_\_qbin / antiCCP\_\_latest\_pre\_\_qbin in \_add\_serology\_bins\(\) inside \_transform\(\), after \_fit\_preprocessor\(\) decides keep\_num\. So those bins never enter feature\_cols\.Change\* Either \(a\) remove this feature engineering to reduce noise/bugs, or \(b\) generate the bins before fitting and include them as categorical \(preferred\) so the model can use them\.WhyAvoids wasted complexity and ensures any intended discretization signal is actually available to the model\.—5\. Reconsider missingness pruning for engineered longitudinal features \(currently self\-defeating under sparsity\)Dropping engineered \_\_\* features when missingness\>\>0\.80 is too aggressive given the extreme sparsity—especially when anchoring is fragile\. It causes a feedback loop: fewer anchored patients→\\rightarrowhigher missingness→\\rightarrowmore drops→\\rightarrowless clinical signal\.Change\* Switch from missing\-rate thresholds to a value\-based shortlist for longitudinal features:\* Keep for key variables \(DAS28, CRP, ESR, HAQ\): last, days\_to\_start\_last, measured\_any, maybe max \(CRP/ESR\), and one windowed mean if enough measurements\.\* Drop high\-variance low\-support features \(std in 0–90d\) unless measurement count≥\\geq2 and coverage is adequate\.\* Gate window statistics by n\_measured \(e\.g\., set std to NaN unless≥\\geq2\) to reduce noise\.WhyThis preserves the few longitudinal summaries that can generalize and avoids the “everything missing→\\rightarroweverything pruned” collapse\.—6\. Reduce dominance of treatment\-choice & socioeconomic proxies \(improves generalization, often helps val AUROC\)Current SHAP shows AntiTNF\_or\_JAK\_First, SE, Scolar dominate\. That can happen when clinical disease activity features are missing, but it can also reflect proxy learning that doesn’t validate well\.Change \(preprocessing\-level\)\* Once anchoring is fixed, ensure disease activity/inflammation features are reliably emitted \(baseline \+ last\-pre \+ recency\)\.\* Consider coarsening high\-cardinality socioeconomic variables \(merge rare levels more aggressively than MIN\_CAT\_FREQ=5, or map to ordered bins if truly ordinal\) to reduce overfitting\.WhyShifts model capacity toward clinically causal\-ish predictors \(baseline disease state\), which typically validates better\.—7\. Add diagnostics that directly optimize for “usable longitudinal coverage”To prevent repeating Step7’s failure mode, add metadata counters like:\* %patients with≥\\geq1 pre\-start row\* %patients with baseline\_gap\_days within 0–30/31–90/91–365\* distribution of inferred start\_est spread \(anchor uncertainty\)WhyThese are leading indicators for downstream AUROC in this task; you can’t improve the metric if preprocessing zeroes out the key signal again\.—If you only change two things for the biggest expected AUROC gain: \(1\) relax/replace anchoring to recover pre\-treatment rows for most patients, and \(2\) stop nulling meds/comorbidities in static\-only mode\.

#### A\.6\.4\.MD Evaluator Output \(DoctorAgents\-DS\)

MD EvaluationPART 1 — History Analysis\* Performance volatility \+ collapse events indicate data/feature instability more than model choice\.\* Step 3 “all\-zero SHAP / AUROC 0\.5” strongly suggests degenerate features \(near\-constant after preprocessing, or leakage/QA features removed leaving almost no signal\) and/or broken scaling/selection \(e\.g\., extremely small C\)\.\* Step 4 “train 1\.0 / val 0\.238” is classic high\-leakage / high\-variance behavior \(tiny n=58, many features, plus time\-alignment features that can accidentally encode outcomes/care pathway\)\.\* Anchoring strategy is the dominant failure mode\.\* Step 7 strict anchoring produced 55/58 “anchor missing / no prestart / static\-only fallback”, so the model learned mostly socioeconomic / treatment\-choice proxies \(SHAP: Scolar/Emploi/SE, AntiTNF\_or\_JAK\_First\), not clinical trajectory\.\* Step 8 swings to anchor success 100%, which is good for coverage but risky: removing plausibility gates can reintroduce post\-start contamination unless pre\-start identification is rock solid\.\* Feature engineering sparsity is biting hard\.\* Step 8 metadata shows multiple \_\_std\_0\_90d features with missing rate 1\.0 \(across all patients\)\. That’s either:\* a bug in window extraction \(no rows fall in 0–90\), or\* nearly everyone has<<2 visits in that window so std is undefined, and you’re emitting all\-NaN columns\.PART 2 — Training Pipeline Correctness and Quality\* Pipeline is “correct” mechanically, but it is likely misaligned with the preprocessing output\.\* Your preprocessing already does: pruning, median impute, clipping, RobustScaler, OHE, missing indicators\.\* Training code imputes again, removes variance again, and for LR StandardScales again\. Double transforms can:\* distort the meaning of already\-robust\-scaled variables,\* change the effective regularization strength,\* and amplify noise for tiny datasets\.\* Potential hard bug masked by current data export: if any column were truly all\-NaN, SimpleImputer\(median\) would normally error\. Since it doesn’t, it suggests the exported CSV is already fully imputed—making the training\-time imputer unnecessary at best\.\* Model selection uses validation AUROC to pick the winner after CV shortlisting\. With n=58, this can overfit to the single validation split; it’s not “wrong” for a leaderboard\-style val metric, but it will be unstable and can reward leakage\.PART 3 — Model Selection, Hyperparameters, and Training Behavior\* p\>\>\>\>n regime \(371 features vs 58 patients\): univariate SelectKBest \(k 10–30\) is doing most of the work\. This is high\-variance and very sensitive to tiny shifts in preprocessing/anchoring\.\* HGB \+ f\_classif feature selection is a questionable pairing\.\* Trees don’t need scaling; they also don’t benefit much from a linear univariate filter that can discard interaction features\.\* Overfitting risk remains high even when train AUROC isn’t 1\.0, because:\* selection is unstable \(small k, many correlated engineered features\),\* time\-alignment features can act as proxies,\* and class\_weight/sample\_weight balancing can increase variance further in tiny samples\.PART 4 — Improvement Strategy & Next\-Step RecommendationsA\) Fix anchoring/window feature reliability first \(highest ROI\)1\. Add sanity gates back, but softer than Step 7\.Keep broad coverage, but reject anchors that imply impossible timelines\.\* Pseudocode idea:\* compute anchor candidates; choose one only if it yields a reasonable fraction of visits with days\_to\_start≥\\geq0 and within a max horizon \(e\.g\., 0–730\), and if the implied start is not after the earliest visit by a large margin\.2\. Stop emitting “always missing” engineered features\.\* If n\_measured<<2 in a window, either:\* don’t create std at all, or\* set std = 0 and rely on n\_measured to indicate reliability \(preferable to all\-NaN columns that become artifacts after imputation\)\.3\. Revisit the 0–90 / 91–365 split\.If most patients have sparse visits, those windows are too granular\. Consider:\* a single 0–365 window, or\* 0–180 and 181–365, or\* adaptive windows based on available visits \(but be careful to keep it time\-safe\)\.B\) Reduce care\-pattern leakage\* In step 6/7/8 you still have measurement density signals \(n\_measured, recency, sometimes visit counts\)\. These can dominate\. Try:\* removing raw visit\-count features,\* or restricting them to a single coarse “has\_any\_measurement” reliability flag per domain\.C\) Align preprocessing with modeling \(avoid double transforms\)\* Either:\* export minimally processed features \(no scaling/imputation\) and keep all transforms in the sklearn Pipeline, or\* export fully model\-ready features and remove imputer/scaler from the training pipeline\.\* Also consider producing two feature matrices: one scaled for linear models and one unscaled for trees, to avoid hurting HGB with clipping/scaling choices optimized for LR\.D\) If val AUROC doesn’t improve in∼\\sim3 iterations: change model family/selectionGiven tiny n, high\-dim, and instability:\* Prefer a single strongly\-regularized linear model \(ridge / elastic net\) using all reasonably filtered features, and drop SelectKBest \(or set k much higher, e\.g\., 80–200\) to reduce selection variance\.\* Alternatively, try Bayesian/empirical Bayes logistic regression \(conceptually\) or at least stronger priors via smaller C with stability selection—because your main issue is variance, not bias\.Quick “next run” checklist\* Verify \(per split\) distribution of n\_visits\_prestart\_0\_90, n\_visits\_prestart\_91\_365, and the fraction with n\_measured≥\\geq2 for key PROs; if near\-zero, remove std features and/or change windows\.\* Confirm that “pre\-treatment rows” are truly pre\-treatment: audit a few patients’ inferred start vs actual visit dates\.\* Run SHAP on a model where clinical activity features are present; if SHAP is again dominated by socioeconomic/treatment\-choice, anchoring/feature availability is still failing\.

#### A\.6\.5\.MD Backpropagation Output \(DoctorAgents\-DS\)

MD FeedbackHighest\-impact issues in the current training code \(hurting AUROC\)\* You’re “preprocessing twice” \(and inconsistently\): the exported CSV is already heavily processed \(imputation/clipping/RobustScaler/OHE/missing flags per metadata\), yet the training pipeline applies median imputation \+ variance filtering \+ StandardScaler \+ SelectKBest again\. This can \(a\) distort already\-robust\-scaled features, \(b\) change the effective regularization strength, and \(c\) amplify noise in a tiny\-n regime\.\* Univariate SelectKBest\(f\_classif\) is extremely high\-variance at n=58, p=371 and is likely selecting unstable proxies \(as seen when anchors were missing, where socioeconomic/treatment\-choice dominated SHAP\)\. Even when anchors are fine, k=10–30 is so small that tiny perturbations swap the chosen set\.\* HGB \+ SelectKBest\(f\_classif\) is a mismatched combo: trees don’t need scaling, and a linear univariate filter can delete interaction/threshold signal that trees would otherwise use\. If HGB is worth trying here, it should see a broader, less pre\-filtered feature set \(or a tree\-appropriate selector\)\.\* CV robustness is brittle given class counts: fixed n\_splits=5 can still yield folds with too few positives/negatives depending on imbalance; your logic then skips folds and may invalidate otherwise good configs \(cv\_score=\-1e9\)\. This can silently bias the search toward “configs that happen not to break” rather than best AUROC\.—Concrete changes to the code that are likely to improve validation AUROC1\. Make the modeling pipeline consistent with what preprocessing exports \(choose one owner of transforms\)\.\* If the CSV is truly “model\-ready” \(already imputed/scaled\), then drop training\-time SimpleImputer and StandardScaler \(at minimum for LR\) and keep only minimal guards \(e\.g\., constant\-column removal\)\.\* If you want transforms in\-model \(preferred\), then export less processed data and let the sklearn Pipeline do impute/scale/OHE—right now you’re in an unstable middle\.2\. Add train\-time dropping of “structurally missing” engineered columns \(especially window stats\)\.\* Your metadata shows many window \_\_std\_0\_90d are missing rate 1\.0\. Even if imputation makes them constants, they waste degrees of freedom and add selector noise\.\* Add a simple filter before model search: drop columns with missing\-rate≥\\geq\(e\.g\.\) 0\.98 or with “unique after impute”≤\\leq1\. This directly reduces SelectKBest variance\.3\. Replace or de\-emphasize SelectKBest \(or at least make it much less aggressive\)\.\* Try no KBest for LR and rely on ridge/elasticnet regularization \(much more stable in p\>\>\>\>n\)\.\* If you keep KBest, push k much higher \(think 80–250\) and/or use stability selection: compute selection frequency across CV folds and keep features that recur \(this often beats picking a single brittle top\-k\)\.4\. Expand the LR regularization grid \(current grid likely misses good regimes\)\.\* Your step history already found useful solutions at C≈\\approx3 \(Step 6\), but Step 7’s search tops out at C=0\.3 for L2 and elastic\. That’s an unnecessary ceiling\.\* Also consider solver choices tailored for tiny data \(e\.g\., liblinear for L1/L2\) to reduce convergence quirks vs saga in small n\.5\. Fix HGB to be tree\-native \(if you keep it\)\.\* Let HGB consume more features: remove SelectKBest \(or use a very high k\) and remove StandardScaler \(not used now, good\)\.\* Consider using HGB’s native NaN handling by not imputing for HGB at all \(your current SimpleImputer erases informative missingness patterns that can matter in longitudinal “measured vs not measured” data\)\.\* Tune parameters that actually control overfit in small n: max\_leaf\_nodes, min\_samples\_leaf, max\_depth, learning\_rate, l2\_regularization\. Right now the HGB grid is narrow and may not find a good bias/variance point\.6\. Make CV split settings adaptive to class counts to avoid fold skipping and invalidation\.\* Set n\_splits = min\(5, min\(n\_pos, n\_neg\)\) \(or even 3 if minority class is tiny\), and/or switch to repeated stratified shuffle split for scoring robustness\.\* Also: clone the pipeline inside each fold \(sklearn\.base\.clone\) instead of reusing the same instance; this avoids subtle state carryover issues and makes results more reproducible\.7\. Use ensembling to reduce variance \(very effective at n=58\)\.\* Instead of selecting a single ‘best” spec, average predicted probabilities from the top\-N diverse specs \(e\.g\., ridge LR \+ elastic LR \+ HGB\) weighted by CV score\. This often improves AUROC in exactly this ‘small sample, unstable feature selection” regime\.8\. Guard against proxy\-dominance as a model\-selection criterion \(to improve hidden/generalization AUROC\)\.\* Add an optional ‘proxy blacklist” experiment \(education/employment/treatment\-choice columns\) and treat it as a separate candidate family; select between ‘full” vs “no\-proxy” by CV\. This helps when anchoring/window coverage is imperfect and the model otherwise latches onto non\-clinical correlates that don’t generalize\.—One “tell” to act on immediatelyIf Step 8 still has many columns with missing rate 1\.0 \(or near\-constant after impute\), your current SelectKBest\(k=10–30\) is effectively doing feature roulette\. Any change that \(a\) removes those columns and \(b\) reduces reliance on aggressive univariate selection is very likely to move AUROC more than tweaking HGB depth or learning rate\.

#### A\.6\.6\.Logger Output

Note Log ofDoctorAgents\-DS after Summarization \(SUMMA\)Step 0→\\rightarrow1: Major feature reduction; start inferred from earliest DReelle \+ median baseline\_time\_to\_DateDeb; baseline latest eligible within 365d; rare categories→\\rightarrowOther; all\-numeric clipping; train\-schema reindex; CV\-ranked/val\-shortlisted LR/HGB with weights/early stopping\. Perf: GradientBoostingClassifier train/val AUROC 1\.0/0\.69 \(Step 0\)→\\rightarrowbest LogisticRegression\_l2 train/val 1\.0/0\.571 \(Step 1\)\.Step 1→\\rightarrow2: Enforced baseline\_time\_to\_DateDeb\-based start \(median per\-row DateDeb\_i\); baseline closest pre\-start \(\+ outside\-365d flag\); separated strict 365d summaries from baseline/serology max; added timing/consistency flags \+ comorbidity burden; label\-consistency collapse \+ flag; refined clipping/scaling \(exclude binary/counts; data\-driven missing indicators\); leakage\-reduced Pipeline LR\(L2\)\+HGB with fold\-wise weighting and penalized CV selection\. Perf unchanged: train 1\.0, val 0\.571\.Step 2→\\rightarrow3: Two\-stage start inference with start\_method; removed leakage/QA flag features in favor of recency; simplified longitudinal markers; curated missing indicators; broadened LR\(ElasticNet/L2\)\+expanded HGB search with optional blending\. Features 199→\\rightarrow111 \(OHE 43→\\rightarrow19\)\. Perf collapse: best LR\_elasticnet C=1e\-5, train/val 0\.5/0\.5; SHAP all\-zero\.Step 3→\\rightarrow4: Added start\_confidence \+ anti\-contamination gate \(longitudinal features forced missing/0 when start\_confidence<<2\); baseline closest pre\-start with baseline\_stale/baseline\_gap\_bin; trajectory features \(last/delta/slope\_per\_day; CRP/ESR log1p\); binned visit intensity; moved meds/comorbidities/serology and many counts to categorical/OHE; missing indicators for numeric\>\>5% missingness; standardized prune\+median\-impute; always\-balanced weighting \(class\_weight=”balanced” \+ sample\_weight\); stricter CV penalties/degeneracy filtering; no ensembling \+ fallback guardrail\. Perf: best LR\_l2 C=1\.0, train/val 1\.0/0\.238 \(instability/overfitting signal\)\.Step 4→\\rightarrow5: Dropped confidence\-gated date reconstruction; used baseline\_time\_to\_DateDeb as sole anchor with always\-emitted baseline snapshots \+ sparse window summaries; meds/comorbidities numeric with pre\-scaling numeric pruning; k\-best feature\-selected LR with revised weighting/CV/selection and SHAP mapping\. Perf improvement: LR\_kbest\_ridge k=15 val AUROC 0\.619 \(from 0\.238\)\.Step 5→\\rightarrow6: Reconstructed robust anchor btt\_used with anchor\_mode \+ baseline\_gap\_bin; broader 365d coverage; added visit/trend/mean features and ‘time\-safe” last\-prestart fallback values; stronger category canonicalization; modeling simplified to LR/HGB ‘views” with hard no\-NaN inputs and StratifiedShuffleSplit scoring\. Perf regression: 0\.819/0\.619→\\rightarrow0\.983/0\.524 \(lr\_elastic\)\.Step 6→\\rightarrow7: Moved from flexible anchoring with single 0–365d window and manual\-view LR/HGB \(StratifiedShuffleSplit; train/val 0\.983/0\.524\) to strict pre\-treatment anchoring with static\-only fallback and two windows \(0–90/91–365d; mean/std/max/log1p; latest\-pre serology \+ optional quantile bins\), engineered\-feature drop\>\>0\.80, explicit QA \(anchor missing/no\-prestart/static\-only for 55/58\), and sklearn Pipelines \(imputation/near\-constant removal/scaling/SelectKBest\) tuned under RepeatedStratifiedKFold\. Perf: train/val 0\.893/0\.571\.Step 7→\\rightarrow8: Switched to robust\-consensus anchor without \[0,365\] gate \(dt\_plus\_btt for 58/58; anchor\_n\_support/anchor\_iqr\_days\); static covariates from latest row regardless of pre\-start; meds/comorbidities ternary categoricals even when anchoring fails; missingness drop relaxed to\>\>0\.95; categorical missing = “Missing”; serology qbins categorical one\-hot; modeling changed to direct array training after one\-time filtering with StratifiedShuffleSplit and simpler penalties\. Perf jump: val AUROC 0\.571 \(lr\_l2 C=0\.3,k=15\)→\\rightarrow0\.905 \(lr\_elastic\_saga C=0\.1,l1\_ratio=0\.9\); train/val 0\.867/0\.905\.Step 8→\\rightarrow9: Changed to earliest\-3\-visit, prestart\-only, monotonic\-consistency–scored anchor \(anchor\_dt\_btt\_corr\) removing fallback leakage; used decayed means, stricter std rules, added deltas/support gates; tighter pruning \(missingness\>\>0\.90\), higher missing\-indicator threshold; serology \_\_measured/\_\_positive\_proxy; broader CV with overfit/degeneracy filtering and no HGB early stopping\. Perf drop: LR elastic C=0\.01,l1\_ratio=0\.1, train/val 0\.588/0\.738 \(from 0\.867/0\.905\)\.Step 9→\\rightarrow10: Switched to correlation\-gated/reconstructed anchoring with validity/shifted\-late suppression of window features; dual baseline snapshots\+deltas; invariant\-any\-row categoricals and any\-row latest serology; looser/always\-keep pruning; sparse\-friendly window summaries; single validation\-split search; selected early\-stopped HGB\. Perf: train/val 0\.8/0\.595\.Step 10→\\rightarrow11: Moved to two\-candidate \(±\\pm\) median start estimator with anchor uncertainty/availability flags; broader baseline/recency indicators; 0–180/181–365d windows with NaN/window\_available\_\*; added 0–365d aggregates; tighter leakage/invariant handling and stricter pruning; CV\-penalized model screening with pipeline imputation/feature filtering\. Perf regression: val AUROC 0\.595 \(HGB\)→\\rightarrow0\.464 \(LR L1 saga C=0\.003\)\.

### A\.7\.Examples of code snippet generated byDoctorAgents\-DS

This section presents examples of generated code snippets emphasizing the feature\-engineering capabilities ofDoctorAgents\-DS\.

#### A\.7\.1\.Mortality Prediction

”bicarbonate\_blood\_delta\_per\_hr” \(L63\-66 and L353\-361\)[⬇](data:text/plain;base64,VFJBSl9MQUJTID0gWwogICAgImNyZWF0aW5pbmVfYmxvb2QiLCAidXJlYV9uaXRyb2dlbl9ibG9vZCIsICJsYWN0YXRlX2Jsb29kIiwgImJpY2FyYm9uYXRlX2Jsb29kIiwKICAgIC4uLgpdCgouLi4uLi4KCmlmIGMgaW4gVFJBSl9MQUJTOgogICAgZGVsdGEgPSBwZC50b19udW1lcmljKGxhc3RfcywgZXJyb3JzPSJjb2VyY2UiKSAtIHBkLnRvX251bWVyaWMoZmlyc3RfcywgZXJyb3JzPSJjb2VyY2UiKQogICAgZWxhcHNlZF9ociA9IChsYXN0X3QgLSBmaXJzdF90KS5kdC50b3RhbF9zZWNvbmRzKCkgLyAzNjAwLjAKICAgIGRwaCA9IHNhZmVfZGl2KGRlbHRhLCBlbGFwc2VkX2hyLnJlcGxhY2UoMC4wLCBucC5uYW4pKQogICAgb3V0W2Yie2N9X2ZpcnN0Il0gPSBmaXJzdF9zCiAgICBvdXRbZiJ7Y31fZGVsdGFfbGFzdF9taW51c19maXJzdCJdID0gZGVsdGEud2hlcmUobl9tZWFzID49IDIuMCwgbnAubmFuKS5hc3R5cGUoImZsb2F0MzIiKQogICAgb3V0W2Yie2N9X2RlbHRhX3Blcl9ociJdID0gZHBoLndoZXJlKG5fbWVhcyA+PSAyLjAsIG5wLm5hbikuYXN0eXBlKCJmbG9hdDMyIikKICAgIG91dFtmIntjfV90cmVuZF9kZWZpbmVkIl0gPSAobl9tZWFzID49IDIuMCkuYXN0eXBlKCJpbnQ4Iik=)TRAJ\_LABS=\["creatinine\_blood","urea\_nitrogen\_blood","lactate\_blood","bicarbonate\_blood",\.\.\.\]\.\.\.\.\.\.ifcinTRAJ\_LABS:delta=pd\.to\_numeric\(last\_s,errors="coerce"\)\-pd\.to\_numeric\(first\_s,errors="coerce"\)elapsed\_hr=\(last\_t\-first\_t\)\.dt\.total\_seconds\(\)/3600\.0dph=safe\_div\(delta,elapsed\_hr\.replace\(0\.0,np\.nan\)\)out\[f"\{c\}\_first"\]=first\_sout\[f"\{c\}\_delta\_last\_minus\_first"\]=delta\.where\(n\_meas\>=2\.0,np\.nan\)\.astype\("float32"\)out\[f"\{c\}\_delta\_per\_hr"\]=dph\.where\(n\_meas\>=2\.0,np\.nan\)\.astype\("float32"\)out\[f"\{c\}\_trend\_defined"\]=\(n\_meas\>=2\.0\)\.astype\("int8"\)

”gcs\_eye\_last” \(L35\-38 and L283\-294\)[⬇](data:text/plain;base64,VklUQUxTID0gWwogICAgLi4uCiAgICAiZ2NzX3RvdGFsIiwgImdjc19leWUiLCAiZ2NzX21vdG9yIiwgImdjc192ZXJiYWwiLCAicmFzcyIKXQoKLi4uLi4uCgp2YWxfbm9ubnVsbCA9IGRmdy5sb2NbZGZ3W2NdLm5vdG5hKCksIFtJRF9DT0wsIGMsIEVWRU5UX1RJTUVfQ09MXV0KaWYgbGVuKHZhbF9ub25udWxsKToKICAgIGZpcnN0X3MgPSB2YWxfbm9ubnVsbC5ncm91cGJ5KElEX0NPTCwgc29ydD1GYWxzZSlbY10uZmlyc3QoKS5yZWluZGV4KHN1YmplY3RzKQogICAgbGFzdF9zID0gdmFsX25vbm51bGwuZ3JvdXBieShJRF9DT0wsIHNvcnQ9RmFsc2UpW2NdLmxhc3QoKS5yZWluZGV4KHN1YmplY3RzKQogICAgZmlyc3RfdCA9IHZhbF9ub25udWxsLmdyb3VwYnkoSURfQ09MLCBzb3J0PUZhbHNlKVtFVkVOVF9USU1FX0NPTF0ubWluKCkucmVpbmRleChzdWJqZWN0cykKICAgIGxhc3RfdCA9IHZhbF9ub25udWxsLmdyb3VwYnkoSURfQ09MLCBzb3J0PUZhbHNlKVtFVkVOVF9USU1FX0NPTF0ubWF4KCkucmVpbmRleChzdWJqZWN0cykKLi4uCm91dFtmIntjfV9sYXN0Il0gPSBsYXN0X3M=)VITALS=\[\.\.\."gcs\_total","gcs\_eye","gcs\_motor","gcs\_verbal","rass"\]\.\.\.\.\.\.val\_nonnull=dfw\.loc\[dfw\[c\]\.notna\(\),\[ID\_COL,c,EVENT\_TIME\_COL\]\]iflen\(val\_nonnull\):first\_s=val\_nonnull\.groupby\(ID\_COL,sort=False\)\[c\]\.first\(\)\.reindex\(subjects\)last\_s=val\_nonnull\.groupby\(ID\_COL,sort=False\)\[c\]\.last\(\)\.reindex\(subjects\)first\_t=val\_nonnull\.groupby\(ID\_COL,sort=False\)\[EVENT\_TIME\_COL\]\.min\(\)\.reindex\(subjects\)last\_t=val\_nonnull\.groupby\(ID\_COL,sort=False\)\[EVENT\_TIME\_COL\]\.max\(\)\.reindex\(subjects\)\.\.\.out\[f"\{c\}\_last"\]=last\_s

”mean\_arterial\_pressure\_mean\_24\_48h” \(L75, L270\-272, and L307\-315\)[⬇](data:text/plain;base64,QklOX1ZJVEFMUyA9IFsibWVhbl9hcnRlcmlhbF9wcmVzc3VyZSIsICJoZWFydF9yYXRlIiwgInJlc3BpcmF0b3J5X3JhdGUiLCAic3BvMiIsICJmaW8yIiwgInRlbXBlcmF0dXJlIl0KCi4uLi4uLgoKZHRfZnJvbV9zdGFydCA9IChkZndbRVZFTlRfVElNRV9DT0xdIC0gZGZ3W1QwX0NPTF0pLmR0LnRvdGFsX3NlY29uZHMoKSAvIDM2MDAuMApyZWNlbnQgPSAoZHRfZnJvbV9zdGFydCA+PSAwLjApICYgKGR0X2Zyb21fc3RhcnQgPD0gMjQuMCkKZWFybHkgPSAoZHRfZnJvbV9zdGFydCA+IDI0LjApICYgKGR0X2Zyb21fc3RhcnQgPD0gNDguMCkKCi4uLi4uLgoKZm9yIGMgaW4gQklOX1ZJVEFMUzoKICAgIGlmIGMgaW4gZGZ3LmNvbHVtbnM6CiAgICAgICAgZ19yZWNlbnQgPSBkZncubG9jW3JlY2VudF0uZ3JvdXBieShJRF9DT0wsIHNvcnQ9RmFsc2UpW2NdLm1lYW4oKS5yZWluZGV4KHN1YmplY3RzKQogICAgICAgIGdfZWFybHkgPSBkZncubG9jW2Vhcmx5XS5ncm91cGJ5KElEX0NPTCwgc29ydD1GYWxzZSlbY10ubWVhbigpLnJlaW5kZXgoc3ViamVjdHMpCiAgICAgICAgLi4uCiAgICAgICAgb3V0W2Yie2N9X21lYW5fMF8yNGgiXSA9IGdfcmVjZW50CiAgICAgICAgb3V0W2Yie2N9X21lYW5fMjRfNDhoIl0gPSBnX2Vhcmx5)BIN\_VITALS=\["mean\_arterial\_pressure","heart\_rate","respiratory\_rate","spo2","fio2","temperature"\]\.\.\.\.\.\.dt\_from\_start=\(dfw\[EVENT\_TIME\_COL\]\-dfw\[T0\_COL\]\)\.dt\.total\_seconds\(\)/3600\.0recent=\(dt\_from\_start\>=0\.0\)&\(dt\_from\_start<=24\.0\)early=\(dt\_from\_start\>24\.0\)&\(dt\_from\_start<=48\.0\)\.\.\.\.\.\.forcinBIN\_VITALS:ifcindfw\.columns:g\_recent=dfw\.loc\[recent\]\.groupby\(ID\_COL,sort=False\)\[c\]\.mean\(\)\.reindex\(subjects\)g\_early=dfw\.loc\[early\]\.groupby\(ID\_COL,sort=False\)\[c\]\.mean\(\)\.reindex\(subjects\)\.\.\.out\[f"\{c\}\_mean\_0\_24h"\]=g\_recentout\[f"\{c\}\_mean\_24\_48h"\]=g\_early

”n\_glucose\_blood\_meas\_48h” \(L41\-45 and L265\-268\)[⬇](data:text/plain;base64,TEFCUyA9IFsKICAgIC4uLgogICAgInVyZWFfbml0cm9nZW5fYmxvb2QiLCAiY3JlYXRpbmluZV9ibG9vZCIsICJnbHVjb3NlX2Jsb29kIiwgImNhbGNpdW1fdG90YWxfYmxvb2QiLAogICAgLi4uCl0KLi4uLi4uCgpjb3VudF92YXJzID0gc29ydGVkKHNldChbYyBmb3IgYyBpbiAodml0YWxzX3ByZXNlbnQgKyBsYWJzX3ByZXNlbnQpIGlmIGMgaW4gKHNldChWSVRBTFMpIHwgc2V0KFRSQUpfTEFCUykgfCBzZXQoTEFCX1JFQ0VOQ1kpIHwgc2V0KFZJVEFMX1RSRU5EUykgfCBzZXQoQklOX1ZJVEFMUykgfCBzZXQoQklOX0xBQlMpIHwgeyJnY3NfdG90YWwiLCJnY3NfZXllIiwiZ2NzX21vdG9yIiwiZ2NzX3ZlcmJhbCJ9KV0pKQpmb3IgYyBpbiBjb3VudF92YXJzOgogICAgb3V0W2Yibl97Y31fbWVhc180OGgiXSA9IGRmd1tjXS5ub3RuYSgpLmdyb3VwYnkoZGZ3W0lEX0NPTF0sIHNvcnQ9RmFsc2UpLnN1bShtaW5fY291bnQ9MSkucmVpbmRleChzdWJqZWN0cykuZmlsbG5hKDAuMCkuYXN0eXBlKCJmbG9hdDMyIikKICAgIG91dFtmIntjfV9tZWFzdXJlZF9hbnlfNDhoIl0gPSAob3V0W2Yibl97Y31fbWVhc180OGgiXSA+IDAuMCkuYXN0eXBlKCJpbnQ4Iik=)LABS=\[\.\.\."urea\_nitrogen\_blood","creatinine\_blood","glucose\_blood","calcium\_total\_blood",\.\.\.\]\.\.\.\.\.\.count\_vars=sorted\(set\(\[cforcin\(vitals\_present\+labs\_present\)ifcin\(set\(VITALS\)\|set\(TRAJ\_LABS\)\|set\(LAB\_RECENCY\)\|set\(VITAL\_TRENDS\)\|set\(BIN\_VITALS\)\|set\(BIN\_LABS\)\|\{"gcs\_total","gcs\_eye","gcs\_motor","gcs\_verbal"\}\)\]\)\)forcincount\_vars:out\[f"n\_\{c\}\_meas\_48h"\]=dfw\[c\]\.notna\(\)\.groupby\(dfw\[ID\_COL\],sort=False\)\.sum\(min\_count=1\)\.reindex\(subjects\)\.fillna\(0\.0\)\.astype\("float32"\)out\[f"\{c\}\_measured\_any\_48h"\]=\(out\[f"n\_\{c\}\_meas\_48h"\]\>0\.0\)\.astype\("int8"\)

#### A\.7\.2\.Readmission Prediction

”heart\_rate\_count\_log1p\_last6h” \(L69, L529\-535, and L763\-775\)[⬇](data:text/plain;base64,Q09SRV9WSVRBTFNfNkggPSBzZXQoWyJoZWFydF9yYXRlIiwgInJlc3BpcmF0b3J5X3JhdGUiLCAic3lzdG9saWNfYnAiLCAibWVhbl9hcnRlcmlhbF9wcmVzc3VyZSIsICJzcG8yIiwgImZpbzIiLCAidGVtcGVyYXR1cmUiXSkKCi4uLi4uLgoKaWYgY29uY2VwdCBpbiBjb3JlX3ZpdGFsc182aF9rZWVwOgogICAgdzIgPSBmbG9hdCh2aXRhbF9zbWFsbF9oKQogICAgbGFzdDIsIHJlYzIsIGZpcnN0MiwgZmlyc3RfcmVjMiA9IF9maXJzdF9sYXN0X2luX3dpbmRvdyhldiwgc3JjX2NvbCwgY29uY2VwdCwgdzIpCiAgICBjbnQyID0gX2NvdW50X2luX3dpbmRvdyhldiwgc3JjX2NvbCwgY29uY2VwdCwgdzIpCiAgICBkeW5bZiJ7Y29uY2VwdH1fX2xhc3RfaW5fbGFzdHtpbnQodzIpfWgiXSA9IGxhc3QyCiAgICBkeW5bZiJ7Y29uY2VwdH1fX2xhc3RfaW5fbGFzdHtpbnQodzIpfWhfaG91cnNfYmVmb3JlX2Rpc2NoYXJnZSJdID0gcmVjMgogICAgZHluW2Yie2NvbmNlcHR9X19jb3VudF9sYXN0e2ludCh3Mil9aCJdID0gY250MgoKLi4uLi4uCgpmb3IgYyBpbiBjbnRfY29sczoKICAgIGJhc2UsIHdpbiA9IF9wYXJzZShjKQogICAgLi4uCiAgICBpZiBiYXNlIGluIGN1cmF0ZWRfY29uY2VwdHM6CiAgICAgICAgb3V0W2Yie2Jhc2V9X19tZWFzdXJlZF9hbnlfbGFzdHt3aW59aCJdID0gKHMgPiAwKS5hc3R5cGUoImludDgiKQogICAgICAgIGNhcCA9IGZsb2F0KGxhYnNfY2FwKQogICAgICAgIGlmIGJhc2UgaW4gVklUQUxfUkFOR0VTOgogICAgICAgICAgICBjYXAgPSBmbG9hdCh2aXRhbHNfY2Fwc1sxXSBpZiB3aW4gPT0gdml0YWxzX2NhcHNbMF0gZWxzZSB2aXRhbHNfY2Fwc1szXSkKICAgICAgICBvdXRbZiJ7YmFzZX1fX2NvdW50X2NhcHBlZF9sYXN0e3dpbn1oIl0gPSBucC5taW5pbXVtKHMsIGZsb2F0KGNhcCkpLmFzdHlwZSgiZmxvYXQzMiIpCiAgICAgICAgb3V0W2Yie2Jhc2V9X19jb3VudF9sb2cxcF9sYXN0e3dpbn1oIl0gPSBucC5sb2cxcChzKS5hc3R5cGUoImZsb2F0MzIiKQogICAgICAgIGtlcHQgKz0gMQ==)CORE\_VITALS\_6H=set\(\["heart\_rate","respiratory\_rate","systolic\_bp","mean\_arterial\_pressure","spo2","fio2","temperature"\]\)\.\.\.\.\.\.ifconceptincore\_vitals\_6h\_keep:w2=float\(vital\_small\_h\)last2,rec2,first2,first\_rec2=\_first\_last\_in\_window\(ev,src\_col,concept,w2\)cnt2=\_count\_in\_window\(ev,src\_col,concept,w2\)dyn\[f"\{concept\}\_\_last\_in\_last\{int\(w2\)\}h"\]=last2dyn\[f"\{concept\}\_\_last\_in\_last\{int\(w2\)\}h\_hours\_before\_discharge"\]=rec2dyn\[f"\{concept\}\_\_count\_last\{int\(w2\)\}h"\]=cnt2\.\.\.\.\.\.forcincnt\_cols:base,win=\_parse\(c\)\.\.\.ifbaseincurated\_concepts:out\[f"\{base\}\_\_measured\_any\_last\{win\}h"\]=\(s\>0\)\.astype\("int8"\)cap=float\(labs\_cap\)ifbaseinVITAL\_RANGES:cap=float\(vitals\_caps\[1\]ifwin==vitals\_caps\[0\]elsevitals\_caps\[3\]\)out\[f"\{base\}\_\_count\_capped\_last\{win\}h"\]=np\.minimum\(s,float\(cap\)\)\.astype\("float32"\)out\[f"\{base\}\_\_count\_log1p\_last\{win\}h"\]=np\.log1p\(s\)\.astype\("float32"\)kept\+=1

”respiratory\_rate\_slope\_last6h\_per\_hr” \(L69, L505, and L529\-553\)[⬇](data:text/plain;base64,Q09SRV9WSVRBTFNfNkggPSBzZXQoWyJoZWFydF9yYXRlIiwgInJlc3BpcmF0b3J5X3JhdGUiLCAic3lzdG9saWNfYnAiLCAibWVhbl9hcnRlcmlhbF9wcmVzc3VyZSIsICJzcG8yIiwgImZpbzIiLCAidGVtcGVyYXR1cmUiXSkKCi4uLi4uLgoKdml0YWxfc2xvcGVfa2VlcF82aCA9IHNldChbImhlYXJ0X3JhdGUiLCAibWVhbl9hcnRlcmlhbF9wcmVzc3VyZSIsICJzeXN0b2xpY19icCIsICJyZXNwaXJhdG9yeV9yYXRlIiwgInNwbzIiLCAiZmlvMiIsICJ0ZW1wZXJhdHVyZSJdKQoKLi4uLi4uCgppZiBjb25jZXB0IGluIGNvcmVfdml0YWxzXzZoX2tlZXA6CiAgICB3MiA9IGZsb2F0KHZpdGFsX3NtYWxsX2gpCiAgICBsYXN0MiwgcmVjMiwgZmlyc3QyLCBmaXJzdF9yZWMyID0gX2ZpcnN0X2xhc3RfaW5fd2luZG93KGV2LCBzcmNfY29sLCBjb25jZXB0LCB3MikKICAgIGNudDIgPSBfY291bnRfaW5fd2luZG93KGV2LCBzcmNfY29sLCBjb25jZXB0LCB3MikKICAgIC4uLgogICAgaWYgKGNvbmNlcHQgaW4gdml0YWxfc2xvcGVfa2VlcF82aCkgYW5kIChub3QgbGFzdDIuZW1wdHkpIGFuZCAobm90IGZpcnN0Mi5lbXB0eSk6CiAgICAgICAgZGVsdGEyID0gKGxhc3QyIC0gZmlyc3QyKS5yZXBsYWNlKFtucC5pbmYsIC1ucC5pbmZdLCBucC5uYW4pLmFzdHlwZSgiZmxvYXQzMiIpCiAgICAgICAgc3BhbjIgPSAoZmlyc3RfcmVjMiAtIHJlYzIpLnJlcGxhY2UoW25wLmluZiwgLW5wLmluZl0sIG5wLm5hbikuYXN0eXBlKCJmbG9hdDMyIikKICAgICAgICBzcGFuMiA9IHNwYW4yLndoZXJlKGNudDIgPj0gMiwgbnAubmFuKQogICAgICAgIHNsb3BlMiA9IChkZWx0YTIgLyBzcGFuMi5yZXBsYWNlKDAsIG5wLm5hbikpLnJlcGxhY2UoW25wLmluZiwgLW5wLmluZl0sIG5wLm5hbikuYXN0eXBlKCJmbG9hdDMyIikKICAgICAgICBkeW5bZiJ7Y29uY2VwdH1fX3Nsb3BlX2xhc3R7aW50KHcyKX1oX3Blcl9ociJdID0gc2xvcGUyCiAgICAgICAgZHluW2Yie2NvbmNlcHR9X19uX2dlMl9sYXN0e2ludCh3Mil9aCJdID0gKGNudDIucmVpbmRleChkeW4uaW5kZXgpLmZpbGxuYSgwKS5hc3R5cGUoImZsb2F0MzIiKSA+PSAyKS5hc3R5cGUoImludDgiKQ==)CORE\_VITALS\_6H=set\(\["heart\_rate","respiratory\_rate","systolic\_bp","mean\_arterial\_pressure","spo2","fio2","temperature"\]\)\.\.\.\.\.\.vital\_slope\_keep\_6h=set\(\["heart\_rate","mean\_arterial\_pressure","systolic\_bp","respiratory\_rate","spo2","fio2","temperature"\]\)\.\.\.\.\.\.ifconceptincore\_vitals\_6h\_keep:w2=float\(vital\_small\_h\)last2,rec2,first2,first\_rec2=\_first\_last\_in\_window\(ev,src\_col,concept,w2\)cnt2=\_count\_in\_window\(ev,src\_col,concept,w2\)\.\.\.if\(conceptinvital\_slope\_keep\_6h\)and\(notlast2\.empty\)and\(notfirst2\.empty\):delta2=\(last2\-first2\)\.replace\(\[np\.inf,\-np\.inf\],np\.nan\)\.astype\("float32"\)span2=\(first\_rec2\-rec2\)\.replace\(\[np\.inf,\-np\.inf\],np\.nan\)\.astype\("float32"\)span2=span2\.where\(cnt2\>=2,np\.nan\)slope2=\(delta2/span2\.replace\(0,np\.nan\)\)\.replace\(\[np\.inf,\-np\.inf\],np\.nan\)\.astype\("float32"\)dyn\[f"\{concept\}\_\_slope\_last\{int\(w2\)\}h\_per\_hr"\]=slope2dyn\[f"\{concept\}\_\_n\_ge2\_last\{int\(w2\)\}h"\]=\(cnt2\.reindex\(dyn\.index\)\.fillna\(0\)\.astype\("float32"\)\>=2\)\.astype\("int8"\)

#### A\.7\.3\.LOS Prediction

”fio2 / po2\_blood"\_mean\_last6h, and "lactate\_blood\_recency\_hours” \(L1120\-1124, L257\-281, and L492\-516\)[⬇](data:text/plain;base64,d29ya2Zsb3dfa2V5c19zbWFsbCA9IFtjIGZvciBjIGluIFsKICAgICJmaW8yIiwgInNwbzIiLCAiZ2NzX3RvdGFsIiwgImdjc192ZXJiYWwiLCAiZ2NzX21vdG9yIiwKICAgICJsYWN0YXRlX2Jsb29kIiwgInBoX2Jsb29kIiwgInBvMl9ibG9vZCIsICJwY28yX2Jsb29kIiwKICAgICJjaGxvcmlkZV9ibG9vZCIsICJyZHdfYmxvb2QiLCAicHR0X2Jsb29kIiwgImJhc2VfZXhjZXNzX2Jsb29kIiwgImNyZWF0aW5pbmVfYmxvb2QiCl0gaWYgYyBpbiB0cmFpbl9yYXcuY29sdW1uc10KCi4uLi4uLgoKZGVmIGZpcnN0X2xhc3Q2aF9tZWFuc19jb3VudHMoZGZ2OiBwZC5EYXRhRnJhbWUsIGNvbDogc3RyKToKICAgIC4uLgogICAgbGFzdF9zdGFydCA9IHRtcFsiX3ByZWQiXSAtIHBkLnRvX3RpbWVkZWx0YSg2LjAsIHVuaXQ9ImgiKQogICAgbGFzdF9lbmQgPSB0bXBbIl9wcmVkIl0KICAgIC4uLgogICAgc19sYXN0ID0gbGFzdF9ibGsuZ3JvdXBieShJRF9DT0wsIHNvcnQ9RmFsc2UpW2NvbF0ubWVhbigpLmFzdHlwZSgiZmxvYXQ2NCIpIGlmIG5vdCBsYXN0X2Jsay5lbXB0eSBlbHNlIHBkLlNlcmllcyhkdHlwZT0iZmxvYXQ2NCIpCiAgICAuLi4KCi4uLi4uLgoKd29ya2Zsb3dfa2V5c19zbWFsbCA9IFtjIGZvciBjIGluIHdvcmtmbG93X2tleXNfc21hbGwgaWYgYyBpbiBkZnYuY29sdW1uc10KLi4uCmZvciBjIGluIHdvcmtmbG93X2tleXNfc21hbGw6CiAgICAuLi4KICAgIHNfZmlyc3QsIHNfbGFzdCwgY19maXJzdCwgY19sYXN0ID0gZmlyc3RfbGFzdDZoX21lYW5zX2NvdW50cyhkZnYsIGMpCiAgICAuLi4KICAgIHdmID0gcGQuRGF0YUZyYW1lKHsKICAgICAgICAuLi4KICAgICAgICBmIntjfV9fbWVhbl9sYXN0NmgiOiBzX2xhc3RfciwKICAgICAgICAuLi4KICAgIH0sIGluZGV4PWNudC5pbmRleCk=)workflow\_keys\_small=\[cforcin\["fio2","spo2","gcs\_total","gcs\_verbal","gcs\_motor","lactate\_blood","ph\_blood","po2\_blood","pco2\_blood","chloride\_blood","rdw\_blood","ptt\_blood","base\_excess\_blood","creatinine\_blood"\]ifcintrain\_raw\.columns\]\.\.\.\.\.\.deffirst\_last6h\_means\_counts\(dfv:pd\.DataFrame,col:str\):\.\.\.last\_start=tmp\["\_pred"\]\-pd\.to\_timedelta\(6\.0,unit="h"\)last\_end=tmp\["\_pred"\]\.\.\.s\_last=last\_blk\.groupby\(ID\_COL,sort=False\)\[col\]\.mean\(\)\.astype\("float64"\)ifnotlast\_blk\.emptyelsepd\.Series\(dtype="float64"\)\.\.\.\.\.\.\.\.\.workflow\_keys\_small=\[cforcinworkflow\_keys\_smallifcindfv\.columns\]\.\.\.forcinworkflow\_keys\_small:\.\.\.s\_first,s\_last,c\_first,c\_last=first\_last6h\_means\_counts\(dfv,c\)\.\.\.wf=pd\.DataFrame\(\{\.\.\.f"\{c\}\_\_mean\_last6h":s\_last\_r,\.\.\.\},index=cnt\.index\)

”spo2\_fio2\_ratio” \(L332\-400, L598\-604, and L744\-753\)[⬇](data:text/plain;base64,ZGVmIHBhaXJlZF9yYXRpb19hZ2dyZWdhdGVzKGRmdjogcGQuRGF0YUZyYW1lLCBudW1lcl9jb2w6IHN0ciwgZGVub21fY29sOiBzdHIsIHByZWZpeDogc3RyLCBtYXhfZ2FwX2hvdXJzOiBmbG9hdCA9IDIuMCk6CiAgICAuLi4KICAgIHJhdGlvID0gKGdbbnVtZXJfY29sXSAvIGRlbm9tX2ZmKS53aGVyZShvaywgbnAubmFuKS5hc3R5cGUoImZsb2F0NjQiKQogICAgLi4uCiAgICBvdXRbZiJ7cHJlZml4fV9fcGFpcmVkX21lYW5fbGFzdDZoX19kZWZpbmVkIl0gPSByZXMuYXBwbHkobGFtYmRhIHg6IGZsb2F0KHhbM10pIGlmIGlzaW5zdGFuY2UoeCwgdHVwbGUpIGVsc2UgMC4wKS5hc3R5cGUoImZsb2F0NjQiKQoKLi4uLi4uCgpwcl9wYXJ0cyA9IFtdCmlmICJzcG8yIiBpbiBkZnYuY29sdW1ucyBhbmQgImZpbzIiIGluIGRmdi5jb2x1bW5zOgogICAgcHJfcGFydHMuYXBwZW5kKHBhaXJlZF9yYXRpb19hZ2dyZWdhdGVzKGRmdiwgInNwbzIiLCAiZmlvMiIsICJzZl9yYXRpbyIsIGZsb2F0KFBBSVJfTUFYX0dBUF9IT1VSUykpKQoKLi4uLi4uCgplbGlmIGhhcygic3BvMl9fbGFzdCIpIGFuZCBoYXMoImZpbzJfX2xhc3QiKToKICAgIGRlbm9tID0gZm51bShvdXRbImZpbzJfX2xhc3QiXSkucmVwbGFjZSgwLjAsIG5wLm5hbikKICAgIG91dFsic3BvMl9maW8yX3JhdGlvIl0gPSBmbnVtKG91dFsic3BvMl9fbGFzdCJdKSAvIGRlbm9tCiAgICBvdXRbInNwbzJfZmlvMl9yYXRpb19fZGVmaW5lZCJdID0gKChmbnVtKG91dC5nZXQoInNwbzJfX21lYXN1cmVkIiwgbnAubmFuKSkuZmlsbG5hKDAuMCkgPiAwKSAmIChmbnVtKG91dC5nZXQoImZpbzJfX21lYXN1cmVkIiwgbnAubmFuKSkuZmlsbG5hKDAuMCkgPiAwKSkuYXN0eXBlKCJmbG9hdDY0Iik=)defpaired\_ratio\_aggregates\(dfv:pd\.DataFrame,numer\_col:str,denom\_col:str,prefix:str,max\_gap\_hours:float=2\.0\):\.\.\.ratio=\(g\[numer\_col\]/denom\_ff\)\.where\(ok,np\.nan\)\.astype\("float64"\)\.\.\.out\[f"\{prefix\}\_\_paired\_mean\_last6h\_\_defined"\]=res\.apply\(lambdax:float\(x\[3\]\)ifisinstance\(x,tuple\)else0\.0\)\.astype\("float64"\)\.\.\.\.\.\.pr\_parts=\[\]if"spo2"indfv\.columnsand"fio2"indfv\.columns:pr\_parts\.append\(paired\_ratio\_aggregates\(dfv,"spo2","fio2","sf\_ratio",float\(PAIR\_MAX\_GAP\_HOURS\)\)\)\.\.\.\.\.\.elifhas\("spo2\_\_last"\)andhas\("fio2\_\_last"\):denom=fnum\(out\["fio2\_\_last"\]\)\.replace\(0\.0,np\.nan\)out\["spo2\_fio2\_ratio"\]=fnum\(out\["spo2\_\_last"\]\)/denomout\["spo2\_fio2\_ratio\_\_defined"\]=\(\(fnum\(out\.get\("spo2\_\_measured",np\.nan\)\)\.fillna\(0\.0\)\>0\)&\(fnum\(out\.get\("fio2\_\_measured",np\.nan\)\)\.fillna\(0\.0\)\>0\)\)\.astype\("float64"\)

#### A\.7\.4\.RA Anti\-TNF Response Prediction

"SE\_2\.0", "GrFonct\_2\.0", and "AntiTNF\_or\_JAK\_First\_3"\(L131, L260\-263, and L299\-323\)[⬇](data:text/plain;base64,Y2F0cyA9IFsiU2V4ZSIsICJTY29sYXIiLCAiU0UiLCAiRW1wbG9pIiwgIlN0YXRGdW0iLCAiU3RhdE1hciIsICJydXJhbHVyYmFuIiwgIkFudGlUTkZfb3JfSkFLX0ZpcnN0IiwgIkFDUjIwMTAiLCAiQWN0aXZpdHkiLCAiR3JGb25jdCJdCgouLi4uLi4KCmZvciBjIGluIGNhdHM6CiAgICBpZiBjIGluIGcuY29sdW1uczoKICAgICAgICB2ID0gYmFzZS5nZXQoYywgbnAubmFuKQogICAgICAgIHJvd1tjXSA9IHN0cih2KSBpZiBwZC5ub3RuYSh2KSBlbHNlICJVbmtub3duIgoKLi4uLi4uCgpjYXRfY29scyA9IFtjIGZvciBjIGluIHRyYWluX3BhdC5jb2x1bW5zIGlmIGMgbm90IGluIHByb3RlY3RlZCBhbmQgdHJhaW5fcGF0W2NdLmR0eXBlID09ICJvYmplY3QiXQouLi4KZW5jID0gT25lSG90RW5jb2RlcihoYW5kbGVfdW5rbm93bj0iaWdub3JlIiwgc3BhcnNlX291dHB1dD1GYWxzZSwgZHR5cGU9bnAuZmxvYXQzMikKZW5jLmZpdCh0cmFpbl9jYXQudmFsdWVzKQpvaGVfbmFtZXMgPSBsaXN0KGVuYy5nZXRfZmVhdHVyZV9uYW1lc19vdXQoY2F0X2NvbHMpKQ==)cats=\["Sexe","Scolar","SE","Emploi","StatFum","StatMar","ruralurban","AntiTNF\_or\_JAK\_First","ACR2010","Activity","GrFonct"\]\.\.\.\.\.\.forcincats:ifcing\.columns:v=base\.get\(c,np\.nan\)row\[c\]=str\(v\)ifpd\.notna\(v\)else"Unknown"\.\.\.\.\.\.cat\_cols=\[cforcintrain\_pat\.columnsifcnotinprotectedandtrain\_pat\[c\]\.dtype=="object"\]\.\.\.enc=OneHotEncoder\(handle\_unknown="ignore",sparse\_output=False,dtype=np\.float32\)enc\.fit\(train\_cat\.values\)ohe\_names=list\(enc\.get\_feature\_names\_out\(cat\_cols\)\)

”PGA\_last\_pre” and ”EGA\_last\_pre” \(L124\-L128 and L244\-L247\)[⬇](data:text/plain;base64,YmFzZWxpbmVfbGlrZV9udW1lcmljID0gWwogICAgIkRBUzI4Q1JQXzQiLCAiQ3JwIiwgIkVzciIsICJQR0EiLCAiRUdBIiwgIk1fSEFRIiwgIlJhaWRldXIiLCAiRmF0aWd1ZSIsICJTb21tZWlsIiwKICAgICJGc2NHQiIsICJGc2NIYiIsICJGc2NQTFEiLCAiRnNjUE1OIiwgIklNQyIsICJTaGFycF90b3RhbCIsCiAgICAiRHVyZWVfc3ltcF9tb2lzIiwgIkFnZV9WMSIsICJ0aW1lX2ZpcnN0X3N5bXB0b21fdG9fRGF0ZURlYiIKXQoKLi4uLi4uCgpmb3IgYyBpbiBiYXNlbGluZV9saWtlX251bWVyaWM6CiAgICBpZiBjIGluIGcuY29sdW1uczoKICAgICAgICB2ID0gX2FzX251bWVyaWMoYmFzZS5nZXQoYywgbnAubmFuKSkKICAgICAgICByb3dbZiJ7Y31fX2xhc3RfcHJlIl0gPSBmbG9hdCh2KSBpZiBwZC5ub3RuYSh2KSBlbHNlIG5wLm5hbg==)baseline\_like\_numeric=\["DAS28CRP\_4","Crp","Esr","PGA","EGA","M\_HAQ","Raideur","Fatigue","Sommeil","FscGB","FscHb","FscPLQ","FscPMN","IMC","Sharp\_total","Duree\_symp\_mois","Age\_V1","time\_first\_symptom\_to\_DateDeb"\]\.\.\.\.\.\.forcinbaseline\_like\_numeric:ifcing\.columns:v=\_as\_numeric\(base\.get\(c,np\.nan\)\)row\[f"\{c\}\_\_last\_pre"\]=float\(v\)ifpd\.notna\(v\)elsenp\.nan

”FscPLQ\_delta\_365d” \(L134, L71\-L117, and L265\-L266\)[⬇](data:text/plain;base64,d2luZG93X3ZhcnMgPSBbIkRBUzI4Q1JQXzQiLCAiQ3JwIiwgIkVzciIsICJQR0EiLCAiRUdBIiwgIk1fSEFRIiwgIlJhaWRldXIiLCAiRmF0aWd1ZSIsICJTb21tZWlsIiwgIkZzY1BMUSJdCgouLi4uLi4KCmRlZiBfc3VtbWFyaXplX3dpbmRvdyhlbGlnLCB0Y29sLCB2YXJzX2xpc3QpOgogICAgLi4uCiAgICBmb3IgYyBpbiB2YXJzX2xpc3Q6CiAgICAgICAgLi4uCiAgICAgICAgZmlyc3RfdiA9IHgubG9jW2ZpcnN0X2ldCiAgICAgICAgbGFzdF92ID0geC5sb2NbbGFzdF9pXQogICAgICAgIHJlc1tmIntjfV9fZGVsdGFfMzY1ZCJdID0gZmxvYXQobGFzdF92IC0gZmlyc3RfdikgaWYgKHBkLm5vdG5hKGZpcnN0X3YpIGFuZCBwZC5ub3RuYShsYXN0X3YpKSBlbHNlIG5wLm5hbgogICAgcmV0dXJuIHJlcwoKLi4uLi4uCgp3aW5fc3VtbSA9IF9zdW1tYXJpemVfd2luZG93KGVsaWdfMzY1IGlmIGhhc18zNjUgZWxzZSBlbGlnXzM2NSwgIl90Iiwgd2luZG93X3ZhcnMpCnJvdy51cGRhdGUod2luX3N1bW0p)window\_vars=\["DAS28CRP\_4","Crp","Esr","PGA","EGA","M\_HAQ","Raideur","Fatigue","Sommeil","FscPLQ"\]\.\.\.\.\.\.def\_summarize\_window\(elig,tcol,vars\_list\):\.\.\.forcinvars\_list:\.\.\.first\_v=x\.loc\[first\_i\]last\_v=x\.loc\[last\_i\]res\[f"\{c\}\_\_delta\_365d"\]=float\(last\_v\-first\_v\)if\(pd\.notna\(first\_v\)andpd\.notna\(last\_v\)\)elsenp\.nanreturnres\.\.\.\.\.\.win\_summ=\_summarize\_window\(elig\_365ifhas\_365elseelig\_365,"\_t",window\_vars\)row\.update\(win\_summ\)