ClinLens: Towards Long-Horizon Coding Agents for Longitudinal Multimodal Clinical Data Science

# ClinLens: Towards Long-Horizon Coding Agents for Longitudinal Multimodal Clinical Data Science
Source: [https://arxiv.org/html/2607.26155](https://arxiv.org/html/2607.26155)
Yuan Zhu1,∗Ethan B\. Liu1,∗,†Frank Nie1,∗Jindong Han1,† 1Shandong University, China jindong\.han@sdu\.edu\.cn

###### Abstract

Clinical data\-science agents must transform heterogeneous longitudinal records into auditable analyses, yet existing benchmarks largely isolate medical question answering, structured\-table reasoning, or generic scientific repositories\. We introduceClinLens, a benchmark of 200 executable tasks over five linked MIMIC resources spanning structured electronic health records, notes, electrocardiograms, chest radiographs, and echocardiograms\. A4×54\\times 5taxonomy crosses four patient\-time scopes with five analysis capabilities\. Program\-first reverse synthesis pairs each bounded semi\-raw package with an evaluator\-private reference workflow and checks required artifacts, cohort and temporal semantics, and the final answer\. On a fixed 126\-task suite, the strongest of 24 standardized model–scaffold configurations achieves 56\.3% scope\-macroStrictPassdespite 100%ExecSuccess\. For reference, a separately configured coding agent solves 83 of 126 tasks, while five biomedical systems adapted to GPT\-4o\-mini reach at most 2\.9% scope\-macroStrictPass\. These results expose a substantial gap between runnable submissions and correct clinical analyses\.

## 1Introduction

Clinical data science turns heterogeneous longitudinal records into cohorts, statistical analyses, and auditable evidence\. A retrospective study may require an analyst to define a population, reconcile patient and encounter identifiers, align observations to clinical time windows, fit an appropriate model, and trace the reported value back to the constructed data\(Kapoor and Narayanan,[2023](https://arxiv.org/html/2607.26155#bib.bib31); Collinset al\.,[2024](https://arxiv.org/html/2607.26155#bib.bib32)\)\. The evidence may span structured electronic health records \(EHRs\), notes, electrocardiograms \(ECGs\), chest radiographs \(CXRs\), and echocardiograms \(ECHOs\)\. Large language model \(LLM\) agents can inspect these files, generate and execute code, and revise an analysis, creating a path toward automating this multi\-step workflow\(Jianget al\.,[2025](https://arxiv.org/html/2607.26155#bib.bib25); Liuet al\.,[2026b](https://arxiv.org/html/2607.26155#bib.bib26)\)\.

Existing evaluations measure useful but narrower proxies\(Laiet al\.,[2023](https://arxiv.org/html/2607.26155#bib.bib27); Huanget al\.,[2024b](https://arxiv.org/html/2607.26155#bib.bib28);[a](https://arxiv.org/html/2607.26155#bib.bib29); Staraceet al\.,[2025](https://arxiv.org/html/2607.26155#bib.bib21)\)\. Medical question\-answering benchmarks emphasize short\-form answers\(Pampariet al\.,[2018](https://arxiv.org/html/2607.26155#bib.bib22); Leeet al\.,[2022](https://arxiv.org/html/2607.26155#bib.bib23); Baeet al\.,[2023](https://arxiv.org/html/2607.26155#bib.bib24); Tuet al\.,[2023](https://arxiv.org/html/2607.26155#bib.bib20)\), while EHRAgent evaluates executable reasoning over structured EHR tables\(Shiet al\.,[2024](https://arxiv.org/html/2607.26155#bib.bib15)\)\. Biomedical and scientific\-agent benchmarks emphasize broad tool use, computational biology, or repository\-level analysis\(Buet al\.,[2026](https://arxiv.org/html/2607.26155#bib.bib16); Mitcheneret al\.,[2025](https://arxiv.org/html/2607.26155#bib.bib17); Majumderet al\.,[2024](https://arxiv.org/html/2607.26155#bib.bib9); Guet al\.,[2024](https://arxiv.org/html/2607.26155#bib.bib11); Chenet al\.,[2024](https://arxiv.org/html/2607.26155#bib.bib10); Jianget al\.,[2025](https://arxiv.org/html/2607.26155#bib.bib25); Liuet al\.,[2026b](https://arxiv.org/html/2607.26155#bib.bib26); Keniaet al\.,[2026](https://arxiv.org/html/2607.26155#bib.bib30)\); MoSciBench further introduces multimodal scientific repositories\(Liuet al\.,[2026a](https://arxiv.org/html/2607.26155#bib.bib1)\)\. These settings do not jointly test executable analysis over clinical sources organized by patient, admission, ICU stay, study, and event time\. For example, predicting an outcome 24–72 hours after admission requires features restricted to the first 24 hours, studies linked to the correct encounter, and a patient\-level split that prevents leakage\. The missing evaluation axis is therefore not executability alone, but whether an agent preserves patient\-time semantics throughout an artifact\-traceable analysis\.

We introduceClinLens, a patient\-centered benchmark for longitudinal multimodal clinical data science\. It links five MIMIC resources while preserving source identifiers, repeated measurements, and timestamps\. Rather than serving task\-specific wide tables,ClinLensexposes bounded semi\-raw packages in which joins, temporal alignment, aggregation, modeling, and validation remain part of the task\. Its4×54\\times 5taxonomy crosses whole\-patient, admission, ICU\-stay, and event/study scopes with profiling, association analysis, event\-aligned change estimation, prediction, and phenotyping\. Program\-first reverse synthesis produces 200 tasks with evaluator\-private reference workflows and artifact\-level checks; a fixed 126\-task suite supports controlled agent comparisons\.

Across one rollout per configuration, the strongest standardized system reaches 56\.3% scope\-macroStrictPassdespite 100%ExecSuccess\. Scaffold rankings vary substantially across backbones, while five biomedical systems adapted to GPT\-4o\-mini reach at most 2\.9% scope\-macroStrictPass\. These results quantify a persistent execution–correctness gap underClinLens; they do not yet isolate a single dominant error source\. We plan to release the construction and evaluation code together with credentialed access to the derived packages\.

#### Contributions\.

We contribute \(1\) an executable clinical data\-science setting that evaluates patient\-time semantics across five linked sources; \(2\) 200 tasks produced by a deterministic reverse\-synthesis pipeline with reference workflows and artifact\-level checks; and \(3\) a standardized study of 24 model–scaffold configurations, supplemented by coding\- and biomedical\-agent evaluations, that measures the gap between runnable submissions and strict analytical correctness\.

## 2TheClinLensBenchmark

### 2\.1Task Setting

Each task is a tupleτ=\(q,𝒫,𝒜,f,ϵ\)\\tau=\(q,\\mathcal\{P\},\\mathcal\{A\},f,\\epsilon\): a natural\-language requestqq, a bounded task package𝒫\\mathcal\{P\}, required artifact specifications𝒜\\mathcal\{A\}, an evaluator\-private reference workflowff, and an answer\-matching predicateϵ\\epsilon\. The agent receives\(q,𝒫,𝒜\)\(q,\\mathcal\{P\},\\mathcal\{A\}\), but notffor the reference answer, and returns the requested artifacts withfinal\_answer\.json\. The request fixes the population, temporal constraints, target computation, and output format while leaving the implementation strategy open\.

Packages preserve the source identifier hierarchy\. Patients are indexed bysubject\_id, admissions byhadm\_id, ICU stays bystay\_id, and modality studies by their source identifiers\. Schemas and paths are standardized, but repeated observations and one\-to\-many relations remain intact\. Depending on the task, solving the request may require cross\-source linkage, temporal filtering, feature construction, aggregation, and missing\-data handling over one index patient, an index patient plus a reference cohort, or a bounded patient set\.

### 2\.2Data and Task Construction

ClinLenslinks MIMIC\-IV structured records\(Johnsonet al\.,[2024a](https://arxiv.org/html/2607.26155#bib.bib2);[2023a](https://arxiv.org/html/2607.26155#bib.bib3); Goldbergeret al\.,[2000](https://arxiv.org/html/2607.26155#bib.bib8)\), MIMIC\-IV\-Note\(Johnsonet al\.,[2023b](https://arxiv.org/html/2607.26155#bib.bib4)\), MIMIC\-IV\-ECG\(Gowet al\.,[2023](https://arxiv.org/html/2607.26155#bib.bib5)\), MIMIC\-CXR\-JPG\(Johnsonet al\.,[2019](https://arxiv.org/html/2607.26155#bib.bib34);[2024b](https://arxiv.org/html/2607.26155#bib.bib6)\), and MIMIC\-IV\-ECHO\(Gowet al\.,[2026](https://arxiv.org/html/2607.26155#bib.bib7)\)\. The linked representation keeps repeated measurements and one\-to\-many relations instead of flattening each task into a prepared table\.

We organize tasks along two axes: four patient\-time scopes—whole\-patient, admission, ICU\-stay, and event/study—and five capability families—profiling, association analysis, event\-aligned change estimation, prediction, and phenotyping\. Their Cartesian product defines 20 templates, including admission\-level landmark prediction, event\-aligned pre/post analysis, and cross\-source phenotype discovery\.

ClinLensuses program\-first reverse synthesis\. For each template, we select ten task instances satisfying its modality and size constraints\. A deterministic reference program specifies the cohort definition, temporal window, random seed, artifact contract, validation predicates, and reference answer; the package and natural\-language request are then instantiated from this specification\. Automated quality control reruns every reference workflow in a clean directory and checks asset accessibility, artifact schemas, cohort sizes, temporal bounds, required validation fields, and answer reproducibility\. All 200 tasks pass these executability and internal\-consistency checks, and all 240 sampled CXR, ECG, and ECHO assets are readable\.

### 2\.3Evaluation

A task passes only when five conditions hold: required artifacts exist, outputs are parseable, cohort row counts match the task contract, validation predicates are satisfied, and the final answer is correct\. Numerical answers use task\-specific tolerances; categorical answers use normalized exact match\. We report their conjunction asStrictPass\. We separately reportExecSuccess, which requires a completed run and a complete, parseable submission but not a correct answer\. Metrics are computed within each patient\-time scope, and Overall is their unweighted mean\.

The full benchmark contains 200 tasks: 170 numerical and 30 categorical\. The fixed 126\-task experimental suite covers all 20 templates and contains 37 whole\-patient, 31 admission, 22 ICU\-stay, and 36 event/study tasks\. Every package\-enabled system receives identical package bytes and manifests\.

## 3Experiments

#### Setup\.

We evaluate four base models—DeepSeek\-V4\-Pro, GLM\-5\.2, GPT\-5\.5, and Claude\-Opus\-4\.8—with six scaffolds\. NoDataGuess measures answer priors without package access\. ReAct alternates reasoning, execution, and revision\(Yaoet al\.,[2023](https://arxiv.org/html/2607.26155#bib.bib12)\); DataVoyager adds data profiling, planning, and critique; Reflexion adds self\-reflection\(Shinnet al\.,[2023](https://arxiv.org/html/2607.26155#bib.bib13)\); SelfDebug repairs programs from execution traces\(Chenet al\.,[2023](https://arxiv.org/html/2607.26155#bib.bib14)\); and RAG\-ReAct retrieves package\-local documentation before ReAct\(Lewiset al\.,[2020](https://arxiv.org/html/2607.26155#bib.bib33)\)\. Runs use zero\-shot prompts, temperature 0, a one\-hour code timeout, and at most three generation or repair rounds\. The 24 configurations produce 3,024 task runs\.

Table 1:Performance on the fixed 126\-task suite\. Cells reportStrictPass/ExecSuccess; Overall is the unweighted mean across four patient\-time scopes\. Bold marks the bestStrictPasswithin each backbone\.
#### Finding 1: execution is not correctness\.

The best standardized configuration, GPT\-5\.5 with SelfDebug, reaches 56\.3% scope\-macroStrictPasswith 100%ExecSuccess\. Several configurations complete nearly every task but solve fewer than half, so completion is a weak proxy for correctness\. In a separate, non\-comparable interactive harness, Codex solves 83 of 126 tasks; all 43 failures remain parseable but contain incorrect final values\.

#### Finding 2: scaffold rankings depend on the backbone\.

SelfDebug is strongest for DeepSeek\-V4\-Pro and GPT\-5\.5, ReAct narrowly leads for GLM\-5\.2, and RAG\-ReAct is strongest for Claude\-Opus\-4\.8\. RAG\-ReAct raises Claude from 43\.5% with ReAct to 53\.5%, while reducing GPT\-5\.5 from 55\.3% to 28\.6%\. Scaffold design must therefore be matched to the base model\.

#### Finding 3: adapted biomedical systems struggle onClinLens\.

Under GPT\-4o\-mini, we adapt BioMedAgent, BixBench, BioMaster, EHRAgent, and Biomni\(Buet al\.,[2026](https://arxiv.org/html/2607.26155#bib.bib16); Mitcheneret al\.,[2025](https://arxiv.org/html/2607.26155#bib.bib17); Suet al\.,[2025](https://arxiv.org/html/2607.26155#bib.bib18); Shiet al\.,[2024](https://arxiv.org/html/2607.26155#bib.bib15); Huanget al\.,[2025](https://arxiv.org/html/2607.26155#bib.bib19)\)\. BioMedAgent reaches 2\.9% scope\-macroStrictPass; EHRAgent reaches 62\.6%ExecSuccessbut only 2\.2%StrictPass\. Without a matched generic\-agent control, these results characterize the adaptations rather than biomedical specialization itself\.

## 4Discussion and Conclusion

The execution–correctness gap motivates explicit patient\-time representations, validation that recomputes cohort counts, temporal cutoffs, and statistical invariants, and traceability from each reported scalar to a specific artifact and row\. These checks target the failure surface measured byClinLenswithout requiring a larger planning stack for every task\.

ClinLensmeasures cross\-source data engineering and statistical analysis rather than raw perceptual reasoning: packages may provide structured labels alongside CXR, ECG, and ECHO assets\. The data come from one health system; quality control establishes internal consistency; and experiments use one rollout without an expert baseline\. Broader sources, expert audit, and repeated runs are the main extensions\.

In summary,ClinLensprovides 200 executable tasks across five linked clinical sources, four patient\-time scopes, and five analysis capabilities\. Current agents frequently produce runnable submissions without passing strict artifact and answer checks\.ClinLensturns this execution–correctness gap into a reproducible target for more reliable clinical data\-science agents\.

## References

- S\. Bae, D\. Kyung, J\. Ryu, E\. Cho, G\. Lee, S\. Kweon, J\. Oh, L\. Ji, E\. I\. Chang, T\. Kim, and E\. Choi \(2023\)EHRXQA: a multi\-modal question answering dataset for electronic health records with chest x\-ray images\.InAdvances in Neural Information Processing Systems,Vol\.36\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2023/hash/0c007ebef1d11fd48da6ce4f54687db6-Abstract-Datasets_and_Benchmarks.html)Cited by:[§1](https://arxiv.org/html/2607.26155#S1.p2.1)\.
- Empowering AI data scientists using a multi\-agent LLM framework with self\-evolving capabilities for autonomous, tool\-aware biomedical data analyses\.Nature Biomedical Engineering\.External Links:[Document](https://dx.doi.org/10.1038/s41551-026-01634-6)Cited by:[§1](https://arxiv.org/html/2607.26155#S1.p2.1),[§3](https://arxiv.org/html/2607.26155#S3.SS0.SSS0.Px4.p1.1)\.
- X\. Chen, M\. Lin, N\. Schärli, and D\. Zhou \(2023\)Teaching large language models to self\-debug\.arXiv preprint arXiv:2304\.05128\.Cited by:[§3](https://arxiv.org/html/2607.26155#S3.SS0.SSS0.Px1.p1.1)\.
- Z\. Chen, S\. Chen, Y\. Ning, Q\. Zhang, B\. Wang, B\. Yu, Y\. Li, Z\. Liao, C\. Wei, Z\. Lu,et al\.\(2024\)ScienceAgentBench: toward rigorous assessment of language agents for data\-driven scientific discovery\.arXiv preprint arXiv:2410\.05080\.Cited by:[§1](https://arxiv.org/html/2607.26155#S1.p2.1)\.
- G\. S\. Collins, K\. G\. M\. Moons, P\. Dhiman, R\. D\. Riley, A\. L\. Beam, B\. Van Calster, M\. Ghassemi, X\. Liu, J\. B\. Reitsma, M\. van Smeden, A\. Boulesteix, J\. C\. Camaradou, L\. A\. Celi, S\. Denaxas, A\. K\. Denniston, B\. Glocker, R\. M\. Golub, H\. Harvey, G\. Heinze, M\. M\. Hoffman, A\. P\. Kengne, E\. Lam, N\. Lee, E\. W\. Loder, L\. Maier\-Hein, B\. A\. Mateen, M\. D\. McCradden, L\. Oakden\-Rayner, J\. Ordish, R\. Parnell, S\. Rose, K\. Singh, L\. Wynants, and P\. Logullo \(2024\)TRIPOD\+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods\.BMJ385,pp\. e078378\.External Links:[Document](https://dx.doi.org/10.1136/bmj-2023-078378)Cited by:[§1](https://arxiv.org/html/2607.26155#S1.p1.1)\.
- A\. L\. Goldberger, L\. A\. N\. Amaral, L\. Glass, J\. M\. Hausdorff, P\. Ch\. Ivanov, R\. G\. Mark, J\. E\. Mietus, G\. B\. Moody, C\. Peng, and H\. E\. Stanley \(2000\)PhysioBank, PhysioToolkit, and PhysioNet: components of a new research resource for complex physiologic signals\.Circulation101\(23\),pp\. e215–e220\.External Links:[Document](https://dx.doi.org/10.1161/01.CIR.101.23.e215)Cited by:[§2\.2](https://arxiv.org/html/2607.26155#S2.SS2.p1.1)\.
- B\. Gow, T\. Pollard, N\. Greenbaum, B\. Moody, A\. Han, J\. W\. Waks, A\. Johnson, E\. Herbst, P\. Eslami, A\. Chaudhari, T\. Carbonati, S\. Berkowitz, R\. Mark, and S\. Horng \(2026\)MIMIC\-IV\-ECHO: echocardiogram matched subset\.PhysioNet\.Note:Version 1\.0External Links:[Document](https://dx.doi.org/10.13026/nrjh-5r77)Cited by:[§2\.2](https://arxiv.org/html/2607.26155#S2.SS2.p1.1)\.
- B\. Gow, T\. Pollard, L\. A\. Nathanson, A\. Johnson, B\. Moody, C\. Fernandes, N\. Greenbaum, J\. W\. Waks, P\. Eslami, T\. Carbonati, A\. Chaudhari, E\. Herbst, D\. Moukheiber, S\. Berkowitz, R\. Mark, and S\. Horng \(2023\)MIMIC\-IV\-ECG: diagnostic electrocardiogram matched subset\.PhysioNet\.Note:Version 1\.0External Links:[Document](https://dx.doi.org/10.13026/4nqg-sb35)Cited by:[§2\.2](https://arxiv.org/html/2607.26155#S2.SS2.p1.1)\.
- K\. Gu, R\. Shang, R\. Jiang, K\. Kuang, R\. Lin, D\. Lyu, Y\. Mao, Y\. Pan, T\. Wu, J\. Yu,et al\.\(2024\)BLADE: benchmarking language model agents for data\-driven science\.arXiv preprint arXiv:2408\.09667\.Cited by:[§1](https://arxiv.org/html/2607.26155#S1.p2.1)\.
- K\. Huang, S\. Zhang, H\. Wang, Y\. Qu, Y\. Lu, Y\. Roohani, R\. Li, L\. Qiu, G\. Li, J\. Zhang,et al\.\(2025\)Biomni: a general\-purpose biomedical AI agent\.bioRxiv\.External Links:[Document](https://dx.doi.org/10.1101/2025.05.30.656746)Cited by:[§3](https://arxiv.org/html/2607.26155#S3.SS0.SSS0.Px4.p1.1)\.
- Q\. Huang, J\. Vora, P\. Liang, and J\. Leskovec \(2024a\)MLAgentBench: evaluating language agents on machine learning experimentation\.InProceedings of the 41st International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.235,pp\. 20271–20309\.External Links:[Link](https://proceedings.mlr.press/v235/huang24y.html)Cited by:[§1](https://arxiv.org/html/2607.26155#S1.p2.1)\.
- Y\. Huang, J\. Luo, Y\. Yu, Y\. Zhang, F\. Lei, Y\. Wei, S\. He, L\. Huang, X\. Liu, J\. Zhao, and K\. Liu \(2024b\)DA\-code: agent data science code generation benchmark for large language models\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,pp\. 13487–13521\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.emnlp-main.748),[Link](https://aclanthology.org/2024.emnlp-main.748/)Cited by:[§1](https://arxiv.org/html/2607.26155#S1.p2.1)\.
- Y\. Jiang, K\. C\. Black, G\. Geng, D\. Park, J\. Zou, A\. Y\. Ng, and J\. H\. Chen \(2025\)MedAgentBench: a virtual EHR environment to benchmark medical LLM agents\.NEJM AI2\(9\),pp\. AIdbp2500144\.External Links:[Document](https://dx.doi.org/10.1056/AIdbp2500144)Cited by:[§1](https://arxiv.org/html/2607.26155#S1.p1.1),[§1](https://arxiv.org/html/2607.26155#S1.p2.1)\.
- A\. Johnson, L\. Bulgarelli, T\. Pollard, B\. Gow, B\. Moody, S\. Horng, L\. A\. Celi, and R\. Mark \(2024a\)MIMIC\-IV\.PhysioNet\.Note:Version 3\.1External Links:[Document](https://dx.doi.org/10.13026/kpb9-mt58)Cited by:[§2\.2](https://arxiv.org/html/2607.26155#S2.SS2.p1.1)\.
- A\. E\. W\. Johnson, L\. Bulgarelli, L\. Shen, A\. Gayles, A\. Shammout, S\. Horng, T\. J\. Pollard, S\. Hao, B\. Moody, B\. Gow, L\. H\. Lehman, L\. A\. Celi, and R\. G\. Mark \(2023a\)MIMIC\-IV, a freely accessible electronic health record dataset\.Scientific Data10\(1\),pp\. 1\.External Links:[Document](https://dx.doi.org/10.1038/s41597-022-01899-x)Cited by:[§2\.2](https://arxiv.org/html/2607.26155#S2.SS2.p1.1)\.
- A\. E\. W\. Johnson, T\. J\. Pollard, S\. J\. Berkowitz, N\. R\. Greenbaum, M\. P\. Lungren, C\. Deng, R\. G\. Mark, and S\. Horng \(2019\)MIMIC\-CXR, a de\-identified publicly available database of chest radiographs with free\-text reports\.Scientific Data6\(1\),pp\. 317\.External Links:[Document](https://dx.doi.org/10.1038/s41597-019-0322-0)Cited by:[§2\.2](https://arxiv.org/html/2607.26155#S2.SS2.p1.1)\.
- A\. Johnson, M\. Lungren, Y\. Peng, Z\. Lu, R\. Mark, S\. Berkowitz, and S\. Horng \(2024b\)MIMIC\-CXR\-JPG: chest radiographs with structured labels\.PhysioNet\.Note:Version 2\.1\.0External Links:[Document](https://dx.doi.org/10.13026/jsn5-t979)Cited by:[§2\.2](https://arxiv.org/html/2607.26155#S2.SS2.p1.1)\.
- A\. Johnson, T\. Pollard, S\. Horng, L\. A\. Celi, and R\. Mark \(2023b\)MIMIC\-IV\-Note: deidentified free\-text clinical notes\.PhysioNet\.Note:Version 2\.1External Links:[Document](https://dx.doi.org/10.13026/0p14-t007)Cited by:[§2\.2](https://arxiv.org/html/2607.26155#S2.SS2.p1.1)\.
- S\. Kapoor and A\. Narayanan \(2023\)Leakage and the reproducibility crisis in machine\-learning\-based science\.Patterns4\(9\),pp\. 100804\.External Links:[Document](https://dx.doi.org/10.1016/j.patter.2023.100804)Cited by:[§1](https://arxiv.org/html/2607.26155#S1.p1.1)\.
- R\. Kenia, X\. Zhang, and P\. Rajpurkar \(2026\)ReX\-MLE: the autonomous agent benchmark for medical imaging challenges\.InProceedings of the 9th International Conference on Medical Imaging with Deep Learning,Proceedings of Machine Learning Research, Vol\.315,pp\. 4288–4315\.External Links:[Link](https://proceedings.mlr.press/v315/kenia26a.html)Cited by:[§1](https://arxiv.org/html/2607.26155#S1.p2.1)\.
- Y\. Lai, C\. Li, Y\. Wang, T\. Zhang, R\. Zhong, L\. Zettlemoyer, W\. Yih, D\. Fried, S\. Wang, and T\. Yu \(2023\)DS\-1000: a natural and reliable benchmark for data science code generation\.InProceedings of the 40th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.202,pp\. 18319–18345\.External Links:[Link](https://proceedings.mlr.press/v202/lai23a.html)Cited by:[§1](https://arxiv.org/html/2607.26155#S1.p2.1)\.
- G\. Lee, H\. Hwang, S\. Bae, Y\. Kwon, W\. Shin, S\. Yang, M\. Seo, J\. Kim, and E\. Choi \(2022\)EHRSQL: a practical text\-to\-SQL benchmark for electronic health records\.InAdvances in Neural Information Processing Systems,Vol\.35\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2022/hash/643e347250cf9289e5a2a6c1ed5ee42e-Abstract-Datasets_and_Benchmarks.html)Cited by:[§1](https://arxiv.org/html/2607.26155#S1.p2.1)\.
- P\. Lewis, E\. Perez, A\. Piktus, F\. Petroni, V\. Karpukhin, N\. Goyal, H\. Küttler, M\. Lewis, W\. Yih, T\. Rocktäschel, S\. Riedel, and D\. Kiela \(2020\)Retrieval\-augmented generation for knowledge\-intensive NLP tasks\.InAdvances in Neural Information Processing Systems,Vol\.33,pp\. 9459–9474\.Cited by:[§3](https://arxiv.org/html/2607.26155#S3.SS0.SSS0.Px1.p1.1)\.
- F\. Liu, X\. Zeng, and H\. Liu \(2026a\)Towards multimodal data\-driven scientific discovery powered by LLM agents\.InThe Fourteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=kZHSvETWdi)Cited by:[§1](https://arxiv.org/html/2607.26155#S1.p2.1)\.
- Q\. Liu, S\. Zhang, G\. Qin, J\. M\. J\. Valanarasu, M\. Rokuss, M\. Lu, T\. Ossowski, J\. M\. Z\. Chaves, C\. Wong, P\. Argaw, Y\. Hasija, M\. Wei, W\. Yim, Q\. Liu, Z\. Jing, J\. Entenmann, N\. Usuyama, T\. Naumann, and H\. Poon \(2026b\)HealthAgentBench: a unified benchmark suite of realistic agentic healthcare environments for challenging frontier AI agents\.arXiv preprint arXiv:2606\.31179\.External Links:[Link](https://arxiv.org/abs/2606.31179)Cited by:[§1](https://arxiv.org/html/2607.26155#S1.p1.1),[§1](https://arxiv.org/html/2607.26155#S1.p2.1)\.
- B\. P\. Majumder, H\. Surana, D\. Agarwal, B\. D\. Mishra, A\. Meena, A\. Prakhar, T\. Vora, T\. Khot, A\. Sabharwal, and P\. Clark \(2024\)DiscoveryBench: towards data\-driven discovery with large language models\.arXiv preprint arXiv:2407\.01725\.Cited by:[§1](https://arxiv.org/html/2607.26155#S1.p2.1)\.
- L\. Mitchener, J\. M\. Laurent, B\. Tenmann, S\. Narayanan, G\. P\. Wellawatte, A\. White, L\. Sani, and S\. G\. Rodriques \(2025\)BixBench: a comprehensive benchmark for LLM\-based agents in computational biology\.arXiv preprint arXiv:2503\.00096\.Cited by:[§1](https://arxiv.org/html/2607.26155#S1.p2.1),[§3](https://arxiv.org/html/2607.26155#S3.SS0.SSS0.Px4.p1.1)\.
- A\. Pampari, P\. Raghavan, J\. Liang, and J\. Peng \(2018\)EmrQA: a large corpus for question answering on electronic medical records\.InProceedings of the 2018 Conference on Empirical Methods in Natural Language Processing,pp\. 2357–2368\.External Links:[Document](https://dx.doi.org/10.18653/v1/D18-1258),[Link](https://aclanthology.org/D18-1258/)Cited by:[§1](https://arxiv.org/html/2607.26155#S1.p2.1)\.
- W\. Shi, R\. Xu, Y\. Zhuang, Y\. Yu, J\. Zhang, H\. Wu, Y\. Zhu, J\. C\. Ho, C\. Yang, and M\. D\. Wang \(2024\)EHRAgent: code empowers large language models for few\-shot complex tabular reasoning on electronic health records\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,pp\. 22315–22339\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.emnlp-main.1245)Cited by:[§1](https://arxiv.org/html/2607.26155#S1.p2.1),[§3](https://arxiv.org/html/2607.26155#S3.SS0.SSS0.Px4.p1.1)\.
- N\. Shinn, F\. Cassano, A\. Gopinath, K\. Narasimhan, and S\. Yao \(2023\)Reflexion: language agents with verbal reinforcement learning\.InAdvances in Neural Information Processing Systems,Vol\.36\.Cited by:[§3](https://arxiv.org/html/2607.26155#S3.SS0.SSS0.Px1.p1.1)\.
- G\. Starace, O\. Jaffe, D\. Sherburn, J\. Aung, J\. S\. Chan, L\. Maksin, R\. Dias, E\. Mays, B\. Kinsella, W\. Thompson, J\. Heidecke, A\. Glaese, and T\. Patwardhan \(2025\)PaperBench: evaluating AI’s ability to replicate AI research\.arXiv preprint arXiv:2504\.01848\.Cited by:[§1](https://arxiv.org/html/2607.26155#S1.p2.1)\.
- H\. Su, W\. Long, and Y\. Zhang \(2025\)BioMaster: multi\-agent system for automated bioinformatics analysis workflow\.bioRxiv\.External Links:[Document](https://dx.doi.org/10.1101/2025.01.23.634608)Cited by:[§3](https://arxiv.org/html/2607.26155#S3.SS0.SSS0.Px4.p1.1)\.
- T\. Tu, S\. Azizi, D\. Driess, M\. Schaekermann, M\. Amin, P\. Chang, A\. Carroll, C\. Lau, R\. Tanno, I\. Ktena,et al\.\(2023\)Towards generalist biomedical AI\.arXiv preprint arXiv:2307\.14334\.Cited by:[§1](https://arxiv.org/html/2607.26155#S1.p2.1)\.
- S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. Narasimhan, and Y\. Cao \(2023\)ReAct: synergizing reasoning and acting in language models\.InInternational Conference on Learning Representations,Cited by:[§3](https://arxiv.org/html/2607.26155#S3.SS0.SSS0.Px1.p1.1)\.