LabAgent: Customize Any Research Hubs for Scientific Discoveries Using AI Agents

# LabAgent: Customize Any Research Hubs for Scientific Discoveries Using AI Agents
Source: [https://arxiv.org/html/2609.13437](https://arxiv.org/html/2609.13437)
\\svgsetup

inkscapelatex=false

Yikun ZhangAffiliation:Department of Computer Science, Northeastern UniversityAffiliation:EWSC Center, Broad Institute of MIT and HarvardJialin ChenAffiliation:Department of Computer Science, Yale UniversityWanjia ZhaoAffiliation:Department of Computer Science, Stanford UniversityRex YingAffiliation:Department of Computer Science, Yale UniversityWengong JinAffiliation:Department of Computer Science, Northeastern UniversityAffiliation:EWSC Center, Broad Institute of MIT and HarvardHua XuAffiliation:Interdepartmental Program in Computational Biology and Bioinformatics, Yale UniversityAffiliation:Department of Biomedical Informatics and Data Science, Yale UniversityJames ZouAffiliation:Department of Biomedical Data Science, Stanford UniversityTianyu Liu\*Affiliation:Department of Biostatistics, Yale UniversityAffiliation:EWSC Center, Broad Institute of MIT and HarvardAffiliation:Interdepartmental Program in Computational Biology and Bioinformatics, Yale UniversityAffiliation:Department of Biomedical Informatics and Data Science, Yale UniversityHongyu Zhao\*Affiliation:Department of Biostatistics, Yale UniversityAffiliation:Interdepartmental Program in Computational Biology and Bioinformatics, Yale UniversityAffiliation:Corresponding authors\. Contact emails: tianyu\.liuyale\.edu; hongyu\.zhaoyale\.edu

###### Abstract

Scientific research is a continuous process that emphasizes inheritance\. Methods developed by predecessors are often expanded upon by new researchers to explore more novel and in\-depth scientific questions\. However, the change of lab staff, such as student graduation, leads to a lack of personnel capable of replicating methods\. Methods that have been developed with significant effort and resources cannot be continued\. To address these limitations, we propose LabAgent, a reproduce and discovery harness tailored for a lab’s continuous work\. LabAgent employs two mechanisms to guarantee that all skills can be executed and verified and to record the corrective methods and experiences, allowing for direct correction or avoidance of similar errors\. We applied LabAgent to drug property prediction, biomedical problem analysis, protein variant effect prediction, and statistical genetics in life science domains\. LabAgent ranks first over commercial generalist agents in every domain, and demonstrates accurate reproduction of a published figure\. Overall, these results demonstrate that LabAgent can effectively integrate and reasonably expand laboratory knowledge\.

###### keywords

Agentic AI, AutoResearch, Agent Harness, Computaitonal Biology, Protein Design

## 1Introduction

Computational methods have become central to modern scientific discovery, especially in biology\([1](https://arxiv.org/html/2609.13437#bib.bib1),[2](https://arxiv.org/html/2609.13437#bib.bib2)\)\. Across single\-cell transcriptomic analysis\([3](https://arxiv.org/html/2609.13437#bib.bib3),[4](https://arxiv.org/html/2609.13437#bib.bib4)\), drug property prediction\([5](https://arxiv.org/html/2609.13437#bib.bib5),[6](https://arxiv.org/html/2609.13437#bib.bib6)\), protein variant effect prediction\([7](https://arxiv.org/html/2609.13437#bib.bib45)\), and genetic association mapping\([8](https://arxiv.org/html/2609.13437#bib.bib7)\), researchers have built a vast array of models, algorithms, and data\-processing pipelines and shared them openly through public code repositories, tutorials, and datasets\([9](https://arxiv.org/html/2609.13437#bib.bib8)\)\. This culture of open sharing has accelerated biological research\. A method developed in one laboratory can be reproduced, reused, and extended by peers worldwide, letting the field build cumulatively on prior work rather than starting each study from scratch\. In practice, though, much of this shared work is hard to reuse\. Codes can be public and its paper carefully written, yet reproducing the reported result often fails\([10](https://arxiv.org/html/2609.13437#bib.bib9)\), because running a method correctly depends on far more than the code that is released\. What the code leaves out is a body of implicit knowledge, such as a preprocessing step that no one wrote down, a package version that was never pinned, or a parameter chosen by hand for the dataset it was tuned on\. Because this knowledge is specific to the laboratory that produced it, it survives only tacitly, in the fine details of a repository, in scattered issue\-tracker discussions, and in the memory of individual researchers\([11](https://arxiv.org/html/2609.13437#bib.bib10)\)\. As lab members move on, it gradually erodes, taking with it the ability to build on what the laboratory actually did\. A different goal is therefore to build a system that recovers this knowledge directly from a laboratory’s public artifacts, including its papers, code, tutorials, and issue discussions\. Such a system would let others reliably run and verify the laboratory’s methods, and build on these validated methods to make new discoveries\.

Large Language Model \(LLM\)\-driven agents possibly can provide such a system\. Agents built on LLMs can read code, call tools, execute programs, and query external knowledge, and a growing number have been applied across the scientific workflow\. But each was built for a different goal, and none provides more than a fragment of what such a system would need\. Discovery agents, such as Google’s AI co\-scientist, reason from the existing literature toward new hypotheses, and can propose ideas that later hold up at the bench\([12](https://arxiv.org/html/2609.13437#bib.bib11),[13](https://arxiv.org/html/2609.13437#bib.bib12),[14](https://arxiv.org/html/2609.13437#bib.bib13),[15](https://arxiv.org/html/2609.13437#bib.bib14),[16](https://arxiv.org/html/2609.13437#bib.bib15),[17](https://arxiv.org/html/2609.13437#bib.bib16),[18](https://arxiv.org/html/2609.13437#bib.bib17),[19](https://arxiv.org/html/2609.13437#bib.bib18),[20](https://arxiv.org/html/2609.13437#bib.bib19)\)\. Engineering agents, such as SWE\-agent, understand and execute code well enough to resolve real issues in software repositories\([21](https://arxiv.org/html/2609.13437#bib.bib44),[22](https://arxiv.org/html/2609.13437#bib.bib20),[23](https://arxiv.org/html/2609.13437#bib.bib21),[24](https://arxiv.org/html/2609.13437#bib.bib22),[25](https://arxiv.org/html/2609.13437#bib.bib23),[26](https://arxiv.org/html/2609.13437#bib.bib24)\)\. Benchmarks, such as PaperBench, confirm that these abilities can be measured\([27](https://arxiv.org/html/2609.13437#bib.bib25),[28](https://arxiv.org/html/2609.13437#bib.bib26),[29](https://arxiv.org/html/2609.13437#bib.bib27)\)\. Biomedical work sets demanding benchmarks of its own\. They ask whether an agent understands an analysis and can assemble the workflow that answers it\. BiomniBench\-DA rebuilds the analyses of published papers across a range of disease areas and scores the whole trajectory against an expert’s rubric\([30](https://arxiv.org/html/2609.13437#bib.bib48)\)\. GeneBench\-Pro poses multi\-stage problems across genetics and omics and scores only the estimate at the end\([31](https://arxiv.org/html/2609.13437#bib.bib34)\)\. Each of these captures something such a system would need, yet none is organized around a specific laboratory’s validated methods and the procedures that run them\. Discovery agents draw on the general literature and public databases, not the methods a particular laboratory has developed and verified\. Engineering agents handle each task in isolation and do not turn what they learn into knowledge the laboratory can reuse, or carry it into discovery\. The most capable of them, the AI co\-scientist, runs specialized agents through a generate, debate, and evolve loop over the published literature and has produced hypotheses confirmed experimentally\([12](https://arxiv.org/html/2609.13437#bib.bib11)\)\. Yet even it neither reproduces nor verifies any laboratory’s methods, and the hypotheses it proposes must still be implemented and tested by others\. Assembling these fragments therefore takes more than a stronger agent\. It requires a way to package the knowledge behind a laboratory’s methods so that others can reuse it, rather than rebuild it for each new task\.

Skills are one of the key factors in helping agents complete reproduction tasks\. A skill packages a task’s procedural knowledge, down to the details a method usually leaves unstated, into a reusable unit that an agent can invoke on demand\([32](https://arxiv.org/html/2609.13437#bib.bib28)\)\. In this form the knowledge becomes explicit and executable, so it persists even as a laboratory’s members move on\. Building such skills by hand does not scale, because a laboratory’s know\-how is spread across far more material than anyone can package manually\. Recent work has therefore begun to build them automatically, mining skills from heterogeneous scientific resources such as code, documentation, and papers, validating and repairing them by execution, and organizing them into self\-evolving libraries that improve agent performance on scientific tasks\([33](https://arxiv.org/html/2609.13437#bib.bib29),[34](https://arxiv.org/html/2609.13437#bib.bib30),[35](https://arxiv.org/html/2609.13437#bib.bib31),[36](https://arxiv.org/html/2609.13437#bib.bib32)\)\. This line of work, however, builds a skill library for a field as a whole, drawn from shared community resources rather than from one laboratory’s own papers, code, and issue discussions\. Its skills, moreover, are validated for executability and general utility rather than against the results a specific study reports, so such systems do not reproduce a target method and confirm that the reproduction matches the original\. Nor do these libraries retain the errors and fixes from reproducing one method as experience that guides the next, so each reproduction begins again rather than building on the last\. Finally, they stop at improving task performance, without turning a laboratory’s validated methods toward proposing and testing the new hypotheses that reuse is meant to enable\.

Here we present*LabAgent*, a multi\-agent framework that instantiates a laboratory\-specific research agent directly from a laboratory’s public artifacts\. An explore agent reads the laboratory’s papers, code repositories, tutorials, and issue discussions, grounds the methods described in them against the code that implements them, and synthesizes the results into executable skills\. Each skill records not only what a method does but how it must be run, including the preprocessing steps a paper omits, the package versions a repository pins, and the failure modes its users have already reported\. The resulting library belongs to that laboratory rather than to a field as a whole, and captures the methods, conventions, and ongoing directions of the laboratory that produced it\. A reproduce agent then puts these skills to work\. Given a new reproduction target, it retrieves protocols and fixes from earlier reproductions and assembles a set of role\-specialized agents to plan, implement, run, and verify the method\. It then compares what it obtains against the metrics the original study reported\. Verification is therefore against the science, not merely against the absence of errors\. A run that completes without crashing but does not recover the published numbers is treated as a failure\. When a run does fail, the agent localizes the failure to a specific skill and to the step at which that skill broke, repairs the skill, and retries\. The execution logs, diagnoses, and fixes accumulated along the way are written back to a persistent memory, so that each new reproduction is conditioned on what earlier ones learned rather than starting from scratch\. Building on the skills that survive this verification, LabAgent turns the laboratory’s own validated methods toward proposing and testing new scientific hypotheses\. LabAgent thus moves scientific agents from task\-centric automation toward reconstructing a laboratory’s research program\.

## 2Results

Overview of LabAgent\.We present three contributions of LabAgent as a harness for reproduction and discovery\. First, we build a route from a laboratory’s public code repository to an executable skill, and we admit a skill to the library only after it has run end to end\. A library we assemble this way lets a hand\-built agent land inside the published error bar more often than either commercial harness we compare against\. Second, we carry the experience of each run into a memory that later runs read back\. A run leaves behind the command that cleared each failure it met, and a later run opens with those commands already in hand\. By contrast, a commercial harness leaves no record of that kind\. Third, we work both mechanisms through four domains of life science, and the four settings differ in what the published record withholds\. On drug property prediction the method is named and its code is public, and LabAgent rebuilds the leaderboards of the Therapeutics Data Commons from what each entry released\. On genomic and single\-cell analysis no protocol exists, and LabAgent carries an open analysis through a chain of dependent decisions and recovers a nuisance variable that the task never names\. On protein variant effect prediction no method is named, and LabAgent chooses among dozens of published predictors on its own\. On statistical genetics no figure code exists, and LabAgent rebuilds the experiment behind a published figure and leaves both of its departures from that figure locatable and named\. Based on this design, LabAgent is able not only to reproduce the selected methods but also to search for combinations of different solutions and produce discoveries\. Our comparison versus state\-of\-the\-art agentic harness also strengthens our conclusion\. We evaluate the framework on three benchmarks and two case studies\. Fig\.[1](https://arxiv.org/html/2609.13437#S2.F1)a lays out the framework, and we describe both agents and the memory in the Methods section\.

![Refer to caption](https://arxiv.org/html/2609.13437v1/fig1_pro.png)Figure 1:The LabAgent framework\.\(a\) The framework of LabAgent\. A target reaches the explore agent as a task configuration or as a paper with its repository, dataset, metric and budget\. That agent searches the literature and code and deposits an executable skill in the library\. The reproduce agent routes the target to one skill, loads its lessons, and works it through six roles from a machine probe to a failure analysis\. It compares the result with the published value and returns verified metrics with the scripts, logs and configurations behind them, and an improved skill\. The dashed path replays a verified skill\. A persistent memory holds the skills, their evidence, the logs and the diagnoses that pair a failure with its fix, and every stage reads and writes it\. \(b\) Downstream applications\. Five settings of the Results, from drug property prediction on public leaderboards to fine\-mapping with no released figure code\. \(c\) How a skill is made\. The agent reads a repository when the target names one and searches for the code when it does not\. Both routes reach one draft, and a second model call checks it against the rubrics\. Only a skill that runs end to end enters the library\.Reproducing drug property leaderboards from laboratory code\.We first took a setting in which the right answer is already on record\. The absorption, distribution, metabolism, excretion and toxicity \(ADMET\) leaderboards of the Therapeutics Data Commons rank published methods on 22 datasets, and every entry publishes both a value and the spread of that value\. Those 22 datasets follow a compound through the body\. Six of them ask how much of it crosses the gut wall and reaches the blood, three ask where it travels once it is there, six ask which cytochrome enzymes break it down, three ask how fast it leaves, and four ask what harm it does along the way\. An analysis of the oral compounds that four large pharmaceutical companies carried into development between 2000 and 2010 ties the physical properties of a compound to its failure on safety grounds\([37](https://arxiv.org/html/2609.13437#bib.bib47)\)\. A reproduction of the code these studies release lets a later group build on that work and take its own life\-science question further\. We took the top three entries of each leaderboard and reached 65 of the 66 because one repository has been deleted upstream\. Those 65 entries resolve to 11 distinct repositories, and the explore agent built one skill from each\. These entries are known to resist rebuilding, and an independent assessment of the ten distinct models that occupy the top three places found that only three could be reproduced\([38](https://arxiv.org/html/2609.13437#bib.bib33)\)\. We compared against Claude Code CLI and against Codex CLI\. Claude Code runs on Opus 4\.8 and Codex runs on GPT\-5\.5\.

We compared the three agents from two angles\. The first angle is performance, and we read it two ways\. We counted the entries that land inside the published error bar\. LabAgent landed inside the published standard deviation on 32 of the 65 entries and on more of them than either commercial harness \(Fig\.[2](https://arxiv.org/html/2609.13437#S2.F2)a–e\)\. We then ranked the four values available for every entry\. LabAgent took the best mean rank of the three at 2\.62, and it ranked first on 13 of the 65 entries against 8 for each of the others \(Fig\.[2](https://arxiv.org/html/2609.13437#S2.F2)f\)\. No agent matches the leaderboard itself at a mean rank of 1\.87\. LabAgent sits closest to it on both readings\. The second angle is cost\. LabAgent spent US$4\.56 per entry, about a third more than the closest harness \(Fig\.[2](https://arxiv.org/html/2609.13437#S2.F2)i\)\. The two angles meet on one pair of axes\. LabAgent reaches the highest count of entries inside the error bar and pays the most for that count \(Fig\.[2](https://arxiv.org/html/2609.13437#S2.F2)h\)\.

We traced one run end to end to show where the spend goes \(Fig\.[2](https://arxiv.org/html/2609.13437#S2.F2)g\)\. In that run, the agent was asked to rebuild MiniMol on a substrate dataset, and its first two attempts failed because neithergraphiumnorminimolwas importable\. It diagnosed the dependency constraint at step 14, installed the geometric extensions on a second branch, and then met a runtime error inside the transformer loader and a version conflict insidebotocore\. It pinned both packages, met a shared\-object failure intorch\_scatter, and reached an AUPRC of 0\.455 at step 67 against a published 0\.474±\\pm0\.025\.

That chain is not unusual\. Two of the tasks make the agent build third\-party research code against a live environment, and at least one turn failed in 72% and 71% of their runs\. The remaining tasks read data we stage in advance, and their rates fall to 27% and 20%\. Recovery is usually immediate\. The turn after a failing turn succeeded 116 times out of 145\. A separate 9% of runs ended early because the output of a dependency install filled the context window\. The extra spend buys that recovery, and the memory keeps the command that produced it\. We summarise the leaderboards, the split files and the scoring procedure in the Methods section\.

![Refer to caption](https://arxiv.org/html/2609.13437v1/fig2.png)Figure 2:Reproducing published ADMET leaderboard values\.\(a\)\-\(e\) One representative dataset from each ADMET category, with the three top\-ranked entries of that leaderboard\. Grey bars give the published value and its standard deviation, and the coloured bars give what LabAgent, Claude Code and Codex obtained\. \(a\) Absorption\. \(b\) Distribution\. \(c\) Metabolism\. \(d\) Excretion\. \(e\) Toxicity\. \(f\) Rank of each of the four values on all 65 entries, with the count at each rank and the mean rank above the bar\. \(g\) One reproduction trajectory over 67 steps, with the failures the agent met, the diagnoses it made and the fixes it applied\. The run ends at an AUPRC of 0\.455 against a published 0\.474±\\pm0\.025\. \(h\) Entries that fall inside the published standard deviation against cost per entry\. \(i\) Cost per entry in US dollars\.Analysing biomedical data beyond any published protocol\.We next moved from a value on record to the quality of an open analysis\. Two benchmarks carry that test between them\. BiomniBench\-DA\([30](https://arxiv.org/html/2609.13437#bib.bib48)\)curates data\-analysis tasks that each ship a public dataset, a reference trace from a domain expert and a rubric anchored at the decision points of that analysis\. Its authors score the process rather than the outcome because a correct final answer can come from memorisation or from wrong reasoning that lands on the right value\. Fifty of its tasks have been released publicly, and those fifty span 16 of its 17 task types\. GeneBench\-Pro\([31](https://arxiv.org/html/2609.13437#bib.bib34)\)scores only the number at the end\. Each of its problems gives a short context and a target estimand, and a plausible wrong turn at any fork changes every step that follows\. Its authors report that models identify local diagnostic signals and then fail to carry the implication through to the matching analysis decision\. No repository implements either set of tasks, and the explore agent built one skill for each task type\.

First, we compared the three systems on the mean rubric score\. LabAgent scored highest of the three at 74\.4 \(Fig\.[3](https://arxiv.org/html/2609.13437#S2.F3)a\)\. It leads Claude Code by 2\.1 points and Codex by 8\.7\. Codex also runs a different base model, so the wider gap measures the harness and the model together and attributes nothing to either alone\. The two Opus arms trade places task by task\. LabAgent led on 20 of the 50 tasks, trailed on 17 and drew level on 13 \(Fig\.[3](https://arxiv.org/html/2609.13437#S2.F3)c\)\. Those per\-task gaps run past 10 points in both directions, so the 2\.1 points at the mean hide a much wider spread\. The breakdown by task type puts our gains on mutation analysis and pathway enrichment and puts Claude Code ahead on clustering and cell composition \(Fig\.[3](https://arxiv.org/html/2609.13437#S2.F3)d,f\)\. Finally, we ablated the explore agent and left a reproduce agent with no skill library on this benchmark\. That arm scored 71\.6 and still placed above Codex \(Fig\.[3](https://arxiv.org/html/2609.13437#S2.F3)a\)\. Against that arm, the full system scored 2\.8 points higher with a standard error of 1\.7 \(Fig\.[3](https://arxiv.org/html/2609.13437#S2.F3)c,e\)\.

Figure 3:Open biomedical data analysis under a process\-level rubric\.\(a\) Mean rubric score across the 50 public tasks for LabAgent, Claude Code, Codex and the arm with the explore agent removed\. \(b\) Mean score against cost per task\. \(c\) Per\-task difference in score between LabAgent and each other arm, with the 50 tasks ranked by the size of the gap\. A value above zero marks a task where LabAgent leads\. \(d\) Mean score by task type for the three systems, on a scale from 0 to 1, with the number of tasks beside each type\. \(e\) The same view for LabAgent and the arm with the explore agent removed\. \(f\) The same scores as \(d\) on the rubric scale, with the value above each bar\.On GeneBench\-Pro, LabAgent passed 2 of the 10 public problems outright, Claude Code passed 1 and Codex passed none \(Fig\.[4](https://arxiv.org/html/2609.13437#S2.F4)a\)\. Partial credit averaged 31\.4 for LabAgent against 28\.0 for Claude Code and 16\.2 for Codex, at a mean US$2\.43 per problem \(Fig\.[4](https://arxiv.org/html/2609.13437#S2.F4)b,c\)\. The Codex figure is an equivalent value under a subscription and not a metered bill\. This setting stayed the hardest of the five we ran, and five of the ten problems returned zero for LabAgent\. One problem separates the three systems\. The task asks for the cis expression quantitative trait locus \(cis\-eQTL\) effect on CXCL10 in activated monocytes, and it supplies single\-cell counts that ambient RNA has contaminated\. CXCL10 is an interferon\-inducible chemokine, and monocytes produce it in quantity once they activate\. The task therefore asks for a regulatory effect for one cell state, and a measurement that pools the states averages that effect away\. The published estimate is−0\.600\-0\.600and the grader allows 0\.05\. LabAgent returned−0\.561\-0\.561, at 0\.77 of that tolerance\. Claude Code returned−0\.454\-0\.454and Codex returned−0\.382\-0\.382, at 2\.92 and 4\.36 of tolerance\.

All three systems recovered the activated population from the same 588 cells and landed close to the reference call \(Fig\.[4](https://arxiv.org/html/2609.13437#S2.F4)d–k\)\. Agreement reaches 0\.951 for LabAgent, 0\.995 for Claude Code and 0\.997 for Codex\. A hand\-built agent therefore separates monocyte activation states as well as either commercial harness does\. LabAgent agreed least of the three and missed 29 of the activated cells\. That gap does not propagate\. We swapped each system’s state call into the reference pipeline and left every other step alone, and all three landed inside the grader’s tolerance\. Any of the three state calls supports the published answer, so the difference between the submitted estimates arises at a later step\. A donor\-level variable the task never mentions accounts for it\. We measured ambient contamination for each donor and grouped the 24 by what we found\. They fall into two groups, and no donor sits between 0\.13 and 0\.24 \(Fig\.[4](https://arxiv.org/html/2609.13437#S2.F4)l\)\. Those groups follow the cis genotype\. None of the 8 donors at dosage 0 sits in the contaminated group, 4 of the 8 at dosage 1 do, and all 8 at dosage 2 do\. Ambient RNA carries CXCL10, so a donor’s contamination adds a genotype\-dependent amount to every cell we measure\. The contaminating term lines up with the genotype itself, and a pipeline that ignores it returns a biased slope and not a noisier one\. We tested that directly\. We held every other step fixed and varied the group term alone\. Each system falls outside tolerance once we withhold the group from its own pipeline, and each returns inside once we restore it \(Fig\.[4](https://arxiv.org/html/2609.13437#S2.F4)m\)\. Every pipeline was already correct in every other respect, so recovering the group is what separates a pass from a failure\.

The three systems differed in what they did about that group\. Claude Code computed contamination for each donor, found it correlated with genotype at 0\.77 and named that correlation as the task’s central difficulty\. It built its ambient profile over the five measured genes instead of the empty\-droplet total, and that deflated every donor’s contamination 4\.85\-fold\. All 24 donors fell below its threshold and the two groups disappeared, so Claude Code modelled contamination as a continuous covariate on the mean\. A linear term fits a slope where the data show a step\. Codex carried no group term anywhere\. LabAgent ran two baselines of its own before it committed, one over all cells at\+0\.27\+0\.27and one over raw counts at−0\.011\-0\.011\. The estimate changed sign between them, and LabAgent read that reversal as evidence that its correction had taken hold\. We summarise the task selection for both benchmarks, the skill\-building mode and the two scoring procedures in the Methods section\.

Figure 4:Open analysis across dependent decision points\.\(a\) Problems solved outright, of 10\. \(b\) Partial credit on each of the 10 problems\. \(c\) Mean partial credit against cost per problem\. \(d\)\-\(m\) The ambient\-state eQTL problem\. \(d\)\-\(g\) The 588 cells in the first two principal components, for the reference estimator and then for each system\. Cells the panel calls activated carry its colour and the rest stay grey\. \(h\)\-\(k\) The same cells in a UMAP embedding, in the same order\. \(l\) Ambient contamination for each of the 24 donors against its cis genotype dosage\. The shaded band holds no donor and separates the two technical groups\. \(m\) The estimated effect on CXCL10 in activated cells with its 95% confidence interval, for each system as submitted, with the technical group withheld from that system’s own pipeline, and with the group restored\. The vertical line and the band around it give the published value and the tolerance the grader allows\.Predicting protein variant effects without a named method\.We then took a setting in which we withheld the method itself\. Biologists predict the effect of a mutation to read what a protein does, and dozens of published methods now compete at the task\. A method leads the ranking on one protein and falls past rank 70 on the next\. A sequence\-only language model comes last of ten candidates on 48 of ProteinGym’s 217 assays, and an agent with no protein\-specific reasoning reaches for that model before any other\.

We handed the agent the whole list in its published state\. The explore agent turned all 24 GitHub\-hosted baselines in ProteinGym’s acknowledgements table into 26 skills\. Some of the 26 skills are current, some have not been touched in years, and some are famous and weak\. Each arm received one assay, the whole library and no method name\. A filtered library would answer half of the questions in advance and would leave every bad pick cheap\. We fixed the assay list by the same principle\. Method choice decides the score on about half of ProteinGym’s 217 substitution assays\([7](https://arxiv.org/html/2609.13437#bib.bib45)\), and on the other half every reasonable method lands in much the same place\. A study that drew only from the first half would manufacture its own result\. We stratified the benchmark by taxon and alignment depth, gave each stratum seats in proportion to its size, and took the assay closest to the median mutant count inside each one\. That draw returned 10 assays and left one of ProteinGym’s own categories empty, so we ran the same rule once more over those categories and added three\. We removed nothing from the first draw, since a sample discarded once its result is known is a sample chosen for its result\. The 13 assays sit on 13 different proteins\. Neither pass looks at which method wins, and both keep the assays where every arm ties\. We scored every reproduction at two levels\. The first is the Spearman correlation across a whole assay, and it averages over every position in the protein\. The second resolves each reproduction to single residues\. Each scanned position carries about 19 substitutions, and the agreement between the reproduction and the measurement at that position follows from those 19 alone\. We ranked every well\-covered position by that agreement\. In each of six proteins we drew the highest\-ranked positions that lie together in the fold, and a group holds from four to six of them\. Every position we drew ranks inside the top 15 of its own protein \(Fig\.[5](https://arxiv.org/html/2609.13437#S2.F5)d–i\)\.

LabAgent reached a mean Spearman correlation of 0\.443 and led all three comparison arms \(Fig\.[5](https://arxiv.org/html/2609.13437#S2.F5)a,b\)\. It won 7 of the 13 assays outright and never fell below 0\.253 on any assay, while each of the other three arms dropped below 0\.10 on at least one \(Fig\.[5](https://arxiv.org/html/2609.13437#S2.F5)b\)\. We traced the lead to a single decision\. LabAgent settled on one structure\-aware predictor for 11 of the 13 assays and committed to it for almost the whole benchmark\. Sequence\-only models sit near zero on viral proteins, and LabAgent led two of the three viral assays by a wide margin\. It reached the lead at US$7\.93 per assay against US$10\.52 for Claude Science \(Fig\.[5](https://arxiv.org/html/2609.13437#S2.F5)c\)\.

The six panels fall into two groups by where the marked positions sit\. In three of them the positions are in the core of the fold\. The SARS\-CoV\-2 spike assay scored how much protein reaches the yeast surface, and that readout reports on folding and not on receptor binding\([39](https://arxiv.org/html/2609.13437#bib.bib35)\)\. A central beta sheet flanked by helices forms the core scaffold of that domain, the receptor\-binding motif sits on that scaffold, and four disulphides hold it together\([39](https://arxiv.org/html/2609.13437#bib.bib35)\)\. C379 to C432 is the most important of the four for the core\([39](https://arxiv.org/html/2609.13437#bib.bib35)\)\. All six positions we drew are hydrophobic or aromatic, and five of the six sit below a relative solvent accessibility of 0\.15 \(Fig\.[5](https://arxiv.org/html/2609.13437#S2.F5)e\)\. They fall into F392 to Y396 and V510 to L513 with I434 on its own\. L513 sits 5\.0 angstroms from C432, so the set reaches that most important disulphide\. By contrast, the receptor\-binding motif holds 69 of the 201 scanned positions, and not one of the six falls inside it\. An assay that selects on folding marks the scaffold that folding rests on\.

The other two of that group carry the same reading at the scale of a domain\. CARD11 is an adaptor protein that carries an antigen\-receptor signal onward, and the assay scored the variants that raise that signal without a receptor\([40](https://arxiv.org/html/2609.13437#bib.bib36)\)\. All six positions lie inside the CARD, and four of the six sit below a relative solvent accessibility of 0\.15 \(Fig\.[5](https://arxiv.org/html/2609.13437#S2.F5)d\)\. The gain\-of\-function variants reported in this gene fall in the CARD, the LATCH and the coiled coil, and none of them falls in the C\-terminal domains\([41](https://arxiv.org/html/2609.13437#bib.bib37)\)\. The SRC positions are the most buried of the six, at a median relative solvent accessibility of 0\.001, and V405 lies two residues before the DFG motif \(Fig\.[5](https://arxiv.org/html/2609.13437#S2.F5)f\)\. In the other group the positions are on the surface\. The*Bacillus subtilis*lipase A positions are the most exposed of the six, at a median relative solvent accessibility of 0\.536, and they lie 14 to 25 angstroms from the catalytic triad\([42](https://arxiv.org/html/2609.13437#bib.bib38)\)\(Fig\.[5](https://arxiv.org/html/2609.13437#S2.F5)h\)\. The photosystem I PsaE positions sit on the surface of a 68\-residue fold, at a median relative solvent accessibility of 0\.481 \(Fig\.[5](https://arxiv.org/html/2609.13437#S2.F5)i\)\. The HLA\-A positions sit between the two groups at 0\.245, they form one continuous surface, and none of them faces the peptide\-binding groove \(Fig\.[5](https://arxiv.org/html/2609.13437#S2.F5)g\)\. We read each panel against the features its own protein is known for, and we drew no rule across the six\.

None of this survives a reproduction that lands on the wrong method\. On the spike assay Claude Code deployed a sequence\-only ESM variant and reached 0\.021\. Its working directory holds the environment, the scoring script, the log probabilities and the download log, and it executed what it chose without error\. Even so, the same code base ships an inverse\-folding model at 0\.465 and a structure\-aware model at 0\.521, and the repository it selected already held a better answer\. Claude Science returned the same 0\.021 here, and Codex exited with an error on all three attempts\. LabAgent reached 0\.560, within 0\.050 of what its own method reports, while the gap to Claude Code was 0\.539\. Almost all of that gap came from the choice of method and not from how well either arm ran it\. Once no method is named the choice becomes the experiment, and a wrong choice costs an order of magnitude more than any execution error we measured\. An assay measures one property of one protein, and no single predictor reads every property well\. LabAgent holds a library of published predictors and picks from it when the task names none\. It can therefore help protein variant effect prediction treat the choice of predictor as part of the experiment\. We give the library and the routing in the Methods section\.

![Refer to caption](https://arxiv.org/html/2609.13437v1/fig5.png)Figure 5:Variant effect prediction across 13 deep mutational scans\.\(a\) Spearman correlation on each assay for the four arms\. The dashed line above each group gives the best published value for that assay\. \(b\) The 13 values for each arm, with the mean as a solid line and the median as a dashed line\. \(c\) Mean Spearman correlation against cost per assay\. \(d\)\-\(i\) Six of the scanned proteins\. The cartoon carries the mean agreement between the reproduction and the measurement over each secondary\-structure element, on one scale shared by the six panels that runs from−0\.35\-0\.35to0\.700\.70, and grey marks residues the scan did not measure\. Sticks mark the positions of highest agreement that lie together in the fold, from four to six in each protein, and the inset enlarges that group\. A glycine carries no side chain and appears in no panel, so panels \(f\) and \(g\) each draw one stick fewer than the text names\. \(d\) CARD11 CARD domain\. \(e\) SARS\-CoV\-2 spike receptor\-binding domain\. \(f\) SRC kinase domain\. \(g\) HLA\-A\. \(h\)*Bacillus subtilis*lipase A\. \(i\) Photosystem I subunit PsaE\.Rebuilding a published figure in statistical genetics\.We last asked for a whole published figure instead of a single number\. A figure is where a life\-science paper puts the comparison a reader acts on, and a group that has to pick a fine\-mapping tool for its own cohort reads the panels before it reads the text\. Behind each panel sits a chain of simulation, scoring, aggregation and drawing\. Journals archive the image at the end of that chain instead of the chain itself, and what survives is an output and not the means to remake it\. Data and code are the two inputs a later group needs most, and the record falls short on both\. One survey of ecology found data for 79% of articles and code for 27%\([43](https://arxiv.org/html/2609.13437#bib.bib39)\)\. The code that does survive often fails to run\. One study collected 27,271 Jupyter notebooks from 3,467 biomedical publications, and of the notebooks it could rerun, 1,203 finished without an error and 879 of those returned what the notebook itself had recorded\([44](https://arxiv.org/html/2609.13437#bib.bib43)\)\. The SuSiE2 study\([45](https://arxiv.org/html/2609.13437#bib.bib40)\)sits on the better side of both counts\. It releases its simulation script in a public repository, and that script still ends at a table of results\. The article states the heritability grid, the number of risk genes and the causal variants each one carries\. It leaves the prior variance, the correlation threshold, the number of single effects and two properties of the eQTL layer to the script alone, and it leaves every choice that turns the table into the figure nowhere at all\. A later reader inherits the simulation and not the figure\.

We followed the experimental configuration of that study and compared five fine\-mapping methods across two causal architectures and five heritability settings\. The five settings differ in what they read beside the association signal of one trait\. SuSiE reads nothing else\([8](https://arxiv.org/html/2609.13437#bib.bib7)\)\. PAINTOR adds functional annotation\([46](https://arxiv.org/html/2609.13437#bib.bib50)\), SuSiE2 adds expression\([45](https://arxiv.org/html/2609.13437#bib.bib40)\), and mvSuSiE\([47](https://arxiv.org/html/2609.13437#bib.bib52)\)and flashfm\([48](https://arxiv.org/html/2609.13437#bib.bib53)\)add further traits\. We scored the relations the figure asserts alongside the quantities the article states\. The agent received the article, the authors’ public repository, the PAINTOR and FINEMAP\([49](https://arxiv.org/html/2609.13437#bib.bib51)\)binaries and a genotype set of 93,246 common variants on chromosome 1 for 10,000 samples\. That genotype set came from an earlier study by the same group, and an automated scan of the finished run found no reference to the outputs that study had left on the same cluster\. The agent then built its five comparators in three ways\. It ported SuSiE from the published algorithm and checked the port on a separate test locus, and it wrote SuSiE2 from the description in the text\. It drove the supplied PAINTOR binary directly\. It wrote surrogates for mvSuSiE and flashfm and never installed the published packages\. Three lines therefore carry the comparison\. The repository holds no plotting code, and the agent wrote its own to draw the six panels\. Our task specification names the 100 replicates the article ran and permits fewer\. The agent chose 50 per cell, recorded that count beside every row of its output and finished the whole grid in 2 h 40 min \(Fig\.[6](https://arxiv.org/html/2609.13437#S2.F6)\)\.

We read the three lines that carry the comparison against every ordering the published figure asserts\. Each one came back in the same direction\. We found SuSiE2 above every other method on power in all ten cells of the grid, and it raised the detection rate over single\-trait SuSiE by 11\.1% to 45\.9% against the 15% to 40% the article reports\. We then saw power rise with heritability at every step for four of the five methods while the first architecture stood above the second at every heritability for all five \(Fig\.[6](https://arxiv.org/html/2609.13437#S2.F6)a,d\)\. We next read coverage against the nominal 0\.95 at 0\.962 to 1\.000 for the two SuSiE lines but at only 0\.627 to 0\.742 for fastPAINTOR \(Fig\.[6](https://arxiv.org/html/2609.13437#S2.F6)b,e\)\. We then asked what the grid says about fine\-mapping on real human data, and we took two readings from it\. The genotypes behind it are real\. They come from 10,000 Europeans of the UK Biobank, so the linkage structure the five methods work against is the structure a real study meets\. We noticed first that the expression layer helps most where the association signal is weakest\. At the lowest heritability of the grid, SuSiE2 lifted power from 0\.333 to 0\.485, a gain of 45\.9%, and at the highest it lifted 0\.680 to 0\.770, a gain of 13\.2%\. Expression therefore matters most at the loci a single\-trait analysis handles worst\. We noticed next how much no method reached\. At the highest heritability we ran, the best of the five recovered 0\.770 of the causal variants under the first architecture and 0\.620 under the second\. Real linkage disequilibrium in a European cohort therefore still hides 23% to 38% of the causal variants from every method in the comparison, and the harder of the two architectures is the one that packs more causal variants into a single locus\.

One departure traces to a single command\-line argument\. fastPAINTOR’s power stays flat across the grid, at 0\.240 to 0\.258 under the first architecture and 0\.198 to 0\.202 under the second, against gains of more than 0\.20 for every other method \(Fig\.[6](https://arxiv.org/html/2609.13437#S2.F6)a,d\)\. For that flat line the agent called PAINTOR with its enumeration flag set to 2 and annotated that call in its own source as the fast mode\. Enumeration is the fast mode, and the 2 is a second quantity, the largest number of causal variants the model may enumerate\. Each locus carries eight causal variants under the first architecture and ten under the second, so the method could recover at most 0\.250 and 0\.200 of them\. Mean power came out at 0\.2510 and 0\.2004\. The flag is documented, and this departure is a misreading rather than a gap the article left open\. Credible set size also came back smaller than the article reports across the comparison, and the run’s README traces that to a purity and pruning rule the agent set itself \(Fig\.[6](https://arxiv.org/html/2609.13437#S2.F6)c,f\)\.

Our rubric reads the output table against the claims the article states and awards 88\.1 of 100\. That rubric sees none of the problems above\. It cannot see that two of the five lines are surrogates, and the agent chose the parameters of those two to match conclusions the figure itself asserts, so about a third of the total rests on lines that test nothing\. The run’s own record is not exact either, since the README describes the fastPAINTOR credible sets as built to 0\.80 coverage while the call site leaves the default of 0\.95 in place\. A reproduction that fails in ways a reader can name is what an audit should deliver, and a summary score on its own would have shown none of this\. A credible set is the list of variants a later experiment has to test\. LabAgent rebuilds the grid behind a published fine\-mapping figure and states the parameters that decide how long that list comes out\. A later group can therefore place a new method on the same axes as the published comparison and take that comparison to harder fine\-mapping settings\. We give the scoring rubric and the task specification in the Methods section\. Overall, LabAgent contributes to reproduction and to discovery across drug property prediction, genomic and single\-cell analysis, protein science and statistical genetics\.

Figure 6:Rebuilding a published simulation figure\.Five fine\-mapping methods across five heritability settings, in two causal architectures\. \(a\)\-\(c\) The architecture in which every causal variant is also an eQTL of a risk gene, at eight causal variants per locus\. \(d\)\-\(f\) The architecture that adds two causal variants outside the risk genes, at ten per locus\. \(a\),\(d\) Power, the share of causal variants that at least one credible set captures\. \(b\),\(e\) Coverage, the share of credible sets that hold a causal variant, with the nominal 0\.95 as a dashed line\. \(c\),\(f\) Average credible set size\. Points are means over 50 replicates per cell against the 100 the article used, error bars give the standard error across those replicates, and points sit offset horizontally within each heritability so that overlapping intervals stay visible\. The SuSiE and SuSiE2 lines are the agent’s own implementations of the published algorithms, and the fastPAINTOR line is the supplied PAINTOR binary\. The mvSuSiE and flashfm lines are surrogates the agent wrote in place of the published packages, with parameters it chose to match the reported behaviour of those methods, and we show them for completeness and not as reproductions\. PAINTOR ran with a maximum of two enumerated causal variants\. That bounds its power at 0\.250 and 0\.200\.
## 3Discussion

Reproducing and reusing published computational methods is important for any science that builds on earlier work, and it is especially important for biology\. In the past, few reproduction pipelines were both reliable and general\. A laboratory releases only part of its code, one machine differs from another, and an article leaves the parameters that matter unstated\. With the help of large language model agents, an executable skill library and a memory that persists across runs, we introduce LabAgent as a harness with a general reproduction workflow for drug property prediction, multi\-omics analysis, single\-cell eQTL estimation, protein variant effect prediction and statistical fine\-mapping\. LabAgent builds its skill library through a systematic search and validation of released laboratory code, and its reproduction workflow evolves on the experience that earlier runs accumulate\. We tested LabAgent on three benchmarks and two case studies, and it outscored three commercial harnesses on every setting that carries a comparison\.

Regarding the contributions of LabAgent to reproduction and discovery, we showed that a route from released laboratory code to a validated skill is workable, but a setting that already names the method it needs requires a larger evaluation set before the gain from that route separates\. However, most questions in biology are open ones or questions of analysis and interpretation, and they name no single best method\. We also extend the harness to open analysis settings such as BiomniBench\-DA and GeneBench\-Pro\. Although LabAgent took the highest score of the three systems on both, it will still help to ask how an agent of this kind evolves on longer and more complex problems of biological analysis, and to raise prediction and biological understanding through that evolution\. It will also be valuable to measure what the memory contributes on its own\. We also note that about one third of the entries came back above the published number, and that this rate does not depend on how large an evaluation set the entry uses\. Meanwhile, as expected, a setting that names the method leaves the three systems hard to tell apart, and a setting that withholds it opens the gap\. Our experiments cover both ends\. The three systems reproduce the ADMET leaderboards to much the same accuracy\. On ProteinGym the choice of method costs far more than any execution error does\. We record the diagnosis of each failure together with the command that resolved it, and we hand a later run a fix it can execute\. Each reproduction leaves the next one less to rebuild\. Therefore, LabAgent can help us reproduce published computational methods and explore new biological questions\.

Previous reproduction efforts score the final number and thus leave the steps in between unrecorded\. Here, we kept the whole trajectory\. It records what the agent met at each step, what it diagnosed the problem to be, and which command resolved it\. The record carries the fix itself, and thus, it is a better resource than an isolated number\. Based on our analysis, failure concentrates where third\-party research code has to build against a live environment, and recovery is usually immediate\. Moreover, the experience we accumulate still describes environments and tooling\. We lack a comparable record for the judgements an analysis makes, and thus, it will be helpful to design a reusable form for those judgements\.

Inspired by our observation that one reported number hides the steps an analysis took, we consider utilizing LabAgent to resolve the structure behind a protein variant effect at more than one scale and to expose the variables a single\-cell analysis most easily mistakes\. Based on our experiments, reading a reproduction below its reported metric can reveal which part of a protein an assay actually constrains\. Therefore, LabAgent can also be used to compare a reproduction with the biology of the assay it reproduces, and not only with the number that assay reported\. Regarding the analysis of single\-cell data, LabAgent recovered a donor\-level variable that the task never names and that decides whether the published estimate is reached\. Therefore, LabAgent can be used to expose the variables an analysis of this kind most easily mistakes\.

Despite the promising results of LabAgent, there is still a big gap between the reproduction of a number and an understanding of the biology behind that number\. Firstly, the technical variation in single\-cell data often tracks the biology under study\. A donor\-level variable can arise from the preparation of a sample, and it can also carry real biological signal\. We cannot tell in advance which one will bias an estimate\. It may be a feasible idea to build control data with known confounders and to treat the recovery of such a variable as a task that carries its own score\. Secondly, the same gap reaches published code\. A laboratory usually releases its core model, and the path from that model to the figure in the article rests on one\-off scripts, unstated parameters and manual preparation of data\. We have no way to recover that path\. LabAgent can infer a candidate path from the figure legend and the methods section, and it can then check that path against the numbers a reader takes off the figure itself\. Moreover, when a task names no method, LabAgent tends to choose a method that is strong overall, and it skips the step that matches a method to the biological question at hand\. The strength of a method on a general leaderboard is measurable\. Its fit to one class of assay has no established measure, and we lack a signal that rewards this step\. The selection pressure an assay applies and the evolutionary depth of a protein can enter the selection step as explicit inputs\. A separate evaluation of the choice itself would give this step a target of its own\. Finally, the relation between the quantity an assay measures and the part of a protein a model actually grasps remains unsettled\. A current evaluation returns one correlation coefficient for a whole assay, and it draws no distinction between residues\. We can not read off where a method holds its strength\. A score that separates positions by structure may open this question\. It would weigh the burial of each site, the domain it belongs to and its place on an interface against the selection pressure of the assay\.

Overall, LabAgent is a useful tool for reusing published computational methods from the released code of a laboratory and asking biological questions that lie beyond any published protocol\.

## 4Methods

Problem Statement\.Here we build LabAgent: a framework of two agents and a memory that persists across reproductions \(Fig\. 1\)\. It takes a laboratory’s public code repository together with the study that repository implements\. It returns a validated executable skill and a reproduction of a result the study reports\. That reproduction sometimes exceeds the value the study originally reported\. We start from a repository URL and collect the source code, the README and documentation, the tutorials and notebooks, the configuration and environment files, and the paper the repository cites\. We write this collected set as𝒜\\mathcal\{A\}, and from it an explore agentEEsynthesises a skillkk, a unit of procedural knowledge\. A skill enters the skill library𝒦\\mathcal\{K\}only if it passes an execution\-based validationVV,

k=E⁡\(𝒜\),𝒦=\{k:V⁡\(k\)=1\},k=E\(\\mathcal\{A\}\),\\qquad\\mathcal\{K\}=\\\{\\,k:V\(k\)=1\\,\\\},\(1\)so that every entry of𝒦\\mathcal\{K\}is executable rather than merely readable\. We write a reproduction target asτ=\(paper,repository,𝒟,μ,B\)\\tau=\(\\text\{paper\},\\text\{repository\},\\mathcal\{D\},\\mu,B\)\. Here𝒟\\mathcal\{D\}is the staged dataset,μ\\muthe metric the study reports, andBBthe budget for the run\. We give the components ofBBand its per\-task ceilings under Task configurations\. A reproduce agentRRthen takes a skillk∈𝒦k\\in\\mathcal\{K\}and executesτ\\tauwithinBB\. It works against a memoryℳ\\mathcal\{M\}that persists across reproductions\. It returns the valueμ^\\hat\{\\mu\}that LabAgent itself obtains for the metricμ\\mu, and an updated memory,

\(μ^,ℳ′\)=R⁡\(k,τ,ℳ,B\)\.\(\\hat\{\\mu\},\\,\\mathcal\{M\}^\{\\prime\}\)=R\\bigl\(k,\\,\\tau,\\,\\mathcal\{M\};\\,B\\bigr\)\.\(2\)We judge the run against a reference valueμ⋆\\mu^\{\\star\}and allow it a deviationδ\\deltafrom that value\. Those two form a scoring pair, and we apply it only once the run has finished\. A published study supplies the pair as the value it reports and the uncertainty it publishes alongside that value\. A task that supplies its own answer key sets the pair by its own grading procedure instead\. The reproduction counts as verified when

ver⁡\(τ\)⇔\|μ^−μ⋆\|≤δ\.\\operatorname\{ver\}\(\\tau\)\\iff\\bigl\|\\hat\{\\mu\}\-\\mu^\{\\star\}\\bigr\|\\leq\\delta\.\(3\)In the first case the source therefore sets the threshold\. A task that defines repeated splits givesμ^\\hat\{\\mu\}as the mean over them\. We require the agent to report the per\-split values and their dispersion alongside it, so that the reproduction carries an error bar of the same kind as the published value\. Neitherμ⋆\\mu^\{\\star\}norδ\\deltais therefore an argument ofRR\. Each benchmark scores the run with its own procedure once the agent has terminated\. That procedure reads only the files the agent wrote and returns nothing to it\.

### 4\.1The*Explore Agent*

Our aim here is to turn𝒜\\mathcal\{A\}into a skill that is both admissible and usable\. We do so in four steps\. We first ground the repository\. We then draft and reflect\. Next we validate in tiers\. Finally we split what passes into a form that can be retrieved a piece at a time\. Before any of this, however, the agent has to be given something to work from\. It accepts two forms of input\. Which one applies is decided by the target rather than by us\. Some targets name the implementation they are asking for\. An ADMET leaderboard entry is one such target\. For these the laboratory’s own repository is the right place to begin\. We hand that repository to the agent directly, and one repository yields one skill\. An open analysis task, by contrast, names no implementation\. Choosing a repository for it would be our judgement rather than the task’s requirement\. In this second mode we supply only the task description and let the agent find the candidates itself\. Here it rewrites the task from several expert perspectives, queries literature and code backends with each rewriting, and then deduplicates and ranks what comes back\. We issue the rewritings in place of the task itself because a single query returns only a narrow view of a literature\. Meanwhile we bound this search and guard it against re\-asking the same question\. We cap the rounds a supervisor may spend, we require a written reflection before each one, and a supervisor may declare the search finished only once every stop criterion in its own brief is met\. Which mode each task uses is stated under Task configurations\. Either way, what the agent holds at the end of this stage is a repository\.

We first ground the skill in the repository rather than in the paper because we want to capture how a method was actually run\. The need for this is practical, because a language model that writes integration code from the paper alone produces identifiers and dependencies that do not exist\. A survey of 576,000 generations from 16 models found that 19\.7% of recommended software packages do not exist\([50](https://arxiv.org/html/2609.13437#bib.bib41)\)\. The same class of fabrication has also been characterised at the level of functions, classes and import paths\([51](https://arxiv.org/html/2609.13437#bib.bib42)\)\. We therefore never draft a skill from prose\. We clone the repository first and extract an application programming interface \(API\) snapshot from it\. That snapshot parses what the code exposes\. It records the importable modules, the public functions and classes, the model constructors and the usage examples that surround them\. It also records the declared dependencies and the repository metadata\. We keep that snapshot alongside the skill as an implementation contract, and constrain the drafting model to use only identifiers that appear in it\. This excludes fabricated identifiers at the point of generation\. Downstream validation never has to discover them later\.

We have now fixed what a skill may refer to but not what it may claim\. A skill, after all, does more than name the calls a repository exposes\. It has to tell the agent what to do at every quantitative step\. Here a draft can transcribe the repository faithfully and still be wrong\. We need the agent to arrive at the right number when it reads a skill, and authoritative prose is not the same thing\. These two come apart in one specific way\. An assertion the agent cannot test overrides the agent’s own reading of the data with the skill author’s\. The agent has no way to notice when that assertion is wrong\. We accordingly treat untestable assertions as worse than silence, and we pass every draft through a second model call that a rubric governs\. That rubric demands a*check*at every quantitative step the skill directs\. A check computes a quantity, prints it, and chooses between two courses according to what it shows\. An instruction, by contrast, names that course outright and leaves nothing to check\. We cover eight situations in which that distinction decides the answer\. The first four require the skill to check direction before it derives a grouping or a composite score, to derive any threshold from the observed distribution, to report how far it extrapolates, and to declare what assumption several estimators share\. The remaining four require it to parse answer\-key semantics literally, to assert the algebraic relations between answer keys, to verify grouping indices and join counts, and to audit units and scale\. The reflection pass injects those that apply\. Meanwhile we also constrain what reflection is allowed to add\. No specific method, estimator, threshold, package or parameter setting may be named as the correct one, unless the grounded materials contain checkable evidence for it\. We can trace one case end to end to show that this works\. One rubric item states an abstract requirement only, and names no gene, task or method\. The generated skill turns it into a concrete check that groups provisionally on the most confident marker\. It then prints every other marker’s ratio between the two groups, and flags those that run the opposite way\. One marker in that task is in fact depressed more than tenfold rather than elevated, and that reversal had produced the wrong coefficient in earlier runs\. Finally, the same pass removed a rule the model had invented for itself in earlier drafts\. We had measured that rule as net\-negative against deterministic graders\.

We then require execution rather than assertion before a skill enters𝒦\\mathcal\{K\}\. A skill can read well and still not run\. Most code that a paper releases will not run as it arrives\. One survey took 15,817 Python notebooks from biomedical publications and tried to execute them\. Only 7\.61% ran without error, and 5\.56% reproduced the results those papers had reported\. The remainder raised exceptions, and these were attributable above all to incompletely recorded dependencies\([44](https://arxiv.org/html/2609.13437#bib.bib43)\)\. We accordingly put every draft through three escalating tiers before we admit it\. Tier 0 checks the skill file itself\. It asks whether the package exists in the software index, whether the code repository resolves, and whether the package identity matches the domain the skill claims\. It then checks the syntax of every code block and the consistency of its imports\. Tier 1 then resolves dependencies with a package\-manager dry run, and confirms that each call in the quick\-start section exists in the installed module\. Tier 2 finally creates a temporary environment, installs the dependencies, and executes the quick\-start end to end\. The agent rewrites only the offending section when a tier fails\. It works from the snapshot and the observed error, and then revalidates\. Only skills that clear all three tiers enter𝒦\\mathcal\{K\}\.

We now have a skill we can trust but not yet one the agent can use in one piece\. It carries a check for every quantitative step it directs and therefore describes the procedure end to end\. The agent, however, is only ever at one step of that procedure\. The whole document therefore puts most of what the agent reads out of reach of what it is doing\. The part that does apply must also compete for context with the data\. We cut the skill instead along the same joints the procedure has\. We write each skill as a root document of roughly 50 lines\. That document carries the task statement, the answer schema, the hard rules that hold throughout, and an index\. We hold the substance in four to seven*loci*\. Each locus covers one step of the procedure, and the agent retrieves it by name when it reaches that step\. We also ship a machine\-readable index alongside the directory\. We stage that whole directory into the working directory before a run, and we refuse any run whose staged loci come up empty\. A skill without that check degrades silently to its 50 lines of pointers while the run still completes normally\.

### 4\.2The*Reproduce Agent*

For reproduction, the obstacle is less often the science than the machine\. In our own runs, described below, at least one command fails in roughly seven runs out of ten wherever a method’s code has to be built against a live environment\. Where the data is already staged, fewer than three runs in ten fail at all\. The reproduce agent therefore has to make a chosen method yield a number on a machine that is not the one the skill was written on, and to do it inside a finite budget\. We break this into four steps\. The agent first selects a skill and then builds an environment for the machine in front of it\. It next works the skill’s procedure through whatever failures arise\. It finally submits what it obtains to a comparison made outside the loop\.

We take selection first because it is a harder decision than it looks\. A library may hold dozens of skills, while a single target should use exactly one of them\. A wrong choice also looks exactly like a poor execution from the outcome alone, since both end in no usable number\. We make selection a separate and deterministic decision of its own for this reason\. We show the agent a catalogue of the descriptions of every skill in𝒦\\mathcal\{K\}, and we then ask it to return one skill name in a single call at temperature zero\. We match that reply against the library exactly\. A name that matches nothing raises an error, and no default catches it\. A failed selection therefore surfaces, and the system never absorbs it silently\. We fix the call in this way, so the choice also stays repeatable\. It does not become a second source of run\-to\-run variance on top of execution\.

We select a skill that somebody else wrote on a machine that is not ours\. Its instructions carry that machine’s assumptions with them\. We therefore prescribe execution as an ordered sequence\. That order exists to force the machine to be read before anything is installed on it\. We build the loop on the default agent of mini\-swe\-agent in the SWE\-agent line\([21](https://arxiv.org/html/2609.13437#bib.bib44)\)\. The agent acts through the shell, and it issues one command per turn\. We supply a system prompt and an instance prompt around this loop, and we write two pairs of them\. The first pair serves the tasks that make the agent build third\-party research code\. Those are TDC ADMET, ProteinGym and the fine\-mapping study\. The second pair serves the two open\-analysis benchmarks\. The account below follows the first pair\. We embed the staged skill in the system prompt and lay out the six roles the loop takes in turn \(Fig\. 1\)\. We then name the dataset and the output contract in the instance prompt\. The first two roles read the machine and prepare it\. A probe reads it, and an environment and data stage then prepares what the run needs and checks the staged files against what the skill expects\. The next three produce the result\. A code stage writes whatever the skill does not already supply, a runner executes the workflow, and a verifier computes the metrics the skill documents and writes them to a results file\. The last role recovers from failure\. An analyst reads the error and proposes an alternative whenever one of the others fails\. The probe reads the compiler, the accelerator driver and toolkit, the interpreter, the package manager and the modules the cluster exposes\. The versions it finds then select the install commands for the following stage from an explicit table\. This builds the numerical stack for the machine actually in front of the agent\. Meanwhile we give each run its own virtual environment and prepend it to the path\. Packages that one run installs therefore cannot reach the next\. Runs without that isolation adapted their code to whichever version happened to be installed, and they did not pin the version the skill specified\. That drift moved one reported error from 0\.256 to 0\.273\. We did not use containers, because the compute environment does not provide them\.

We rarely reach a reproduction on the first attempt even with an environment in place\. A run therefore takes largely the shape of its failures\. Here we describe what that looks like in practice, and we draw on the 140 runs of the four tasks we report below\. A run runs as a single sequence of shell commands and their outputs, because the loop offers only one action\. The model reasons, issues a command, reads what comes back, and repeats\. We handle failure inside that sequence\. We direct the agent in the system prompt to read the error and try an alternative before it gives up\. That alternative may be a different install channel, a missing dependency or a reformatted input\. We deliver experience from earlier runs at the same point, by the mechanism we describe below\. A median of 16 to 19 turns was enough across the 140 runs\. The task where the agent chooses among skills runs longer, and its median reaches 38\. Failure, meanwhile, is routine wherever third\-party research code has to be built against a live environment\. The two tasks that require it saw at least one failing turn in 72% and 71% of runs\. The figures are 27% and 20% on the two tasks where we stage the data in advance\. Recovery, where it happens, is immediate\. A turn that does not fail follows 116 of the 145 failing turns\. The diagnose\-and\-retry loop therefore closes inside the trajectory in four cases out of five\. A run can also go wrong in more than one direction\. It can circle without progress, spend without bound, hang on a single command, or simply take too long\. A cap on any one of these would leave the others open, so we cap turns, monetary cost, the time of any single command, and the wall\-clock time of the whole run\. Whichever binds first ends the run\. None of the 65 runs on the largest task reached the 100\-turn cap, however, and one reached the cost cap\. The context window ended runs early instead, and runs exceeded it in 9% of cases because dependency installation produces very long output\. Finally, we do not take the agent’s word for whether a run delivered a result\. Its self\-report is unreliable in one direction, and no agent declared failure in any of the 140 runs\. We read the outcome from the harness’s exit status instead\.

We do not treat a run that terminates without error as a reproduction by itself\. We therefore compare againstμ⋆\\mu^\{\\star\}only after the agent has stopped, and we leave that comparison to each benchmark’s own scoring procedure\. That procedure reads only the files the agent wrote and returns nothing to it\. No part of it is reachable from inside the loop while the agent runs\.

### 4\.3The*Reproduction Memory*

Nothing an agent works out inside one run reaches the next unless we write it down, and without a store every run would begin from the same ignorance\. Our memoryℳ\\mathcal\{M\}is that store\. We keep four kinds of record in it \(Fig\. 1\): the skills themselves, the grounding evidence each skill was built from, the execution record of past runs, and the diagnoses that pair a failure with what resolved it\. We read the four at different moments\. Skills are read when one is selected, and grounding evidence when a skill is repaired\. The execution record we keep for audit rather than consult automatically\. Only the diagnoses are read back while a run is in progress, and they are what we describe here\.

We record the specific command that got the earlier run past the error because a diagnosis is only useful if the next run can act on it\. Suppose, then, that we recorded only the advice behind that command\. The next run would have to turn the advice back into a command, and that is the step where it fails\. We therefore keep the command exactly as it was typed\. We also keep a count of how often it has since worked and how often it has not\. A fix that succeeded once may have succeeded only on that machine, and later retrieval should be able to tell the two apart\. We call that record a*lesson*, and it carries five fields\. It opens with a trigger, either a literal string or a regular expression\. It then carries the text of what resolved it, the command that did so, the runs in which it has held, and its success and failure counts\. Lessons divide into two sorts by how widely they transfer\. Operational lessons concern environments and tooling, and they transfer across every domain by construction\. A wheel that will not build is the same obstacle whatever the science downstream, and so is a module that will not import\. Recipe lessons, by contrast, record the configuration that reproduced one particular result, and their scope is correspondingly narrow\.

We then deliver lessons to the agent on two channels\. The first arrives before the run begins\. We take the highest\-scoring operational lessons, and we add any recipe lesson whose trigger names the current skill or task\. We format them into a block and prepend that block to the task description\. The second arrives during the run itself\. We execute every shell command through an environment wrapper, and that wrapper matches the command’s output against every known trigger\. A match appends the corresponding lessons to the output the agent will read on its next turn\. We added this second channel because the first one arrives at the wrong time\. Context that we supply before a run must anticipate which failures will occur\. A match on command output, by contrast, delivers the record at the moment and at the site of the failure, in the channel the agent is already attending to\. Meanwhile we cap what the wrapper appends, so that it cannot crowd out the output it annotates\. We also mark it as a hint\. The agent may therefore disregard it, and the skill takes precedence where the two disagree\.

We finally grow the store through the same wrapper that delivers from it\. That wrapper recognises a fixed set of failure signatures\. We tag each of them by kind and by language\. These span environment faults, missing modules and packages, absent commands, file and schema errors, shape mismatches, accelerator faults, build failures and timeouts\. We weight that set towards dependency failures because that is where reproduction fails\. The notebook survey above found that unresolved imports and missing modules alone accounted for 41\.65% of execution errors\([44](https://arxiv.org/html/2609.13437#bib.bib43)\)\. We record the pair as a candidate lesson whenever a command matches a signature and the next command returns success\. That lesson associates the signature with the resolution\. We then hold candidates for the duration of the run and append them to the persistent store afterwards\. We tag each with whether the wider run succeeded, so that later retrieval can weight them accordingly\.

### 4\.4Benchmarks and datasets

A computational biologist needs several abilities\. Each one rests on the ones before it, and we chose one task for each\. The first ability is to run a published method on unfamiliar hardware and land on the number its authors report, and we test it on TDC ADMET\([5](https://arxiv.org/html/2609.13437#bib.bib5)\)\. The second is to carry an analysis from a dataset to a conclusion when nobody has written the procedure down, and we test it on BiomniBench\-DA\([30](https://arxiv.org/html/2609.13437#bib.bib48)\)\. The third is to see the artefact in a dataset that nobody named, and we test it on GeneBench\-Pro\([31](https://arxiv.org/html/2609.13437#bib.bib34)\)\. The fourth is to choose among many published methods the one that matches what an experiment measured, and we test it on ProteinGym\([7](https://arxiv.org/html/2609.13437#bib.bib45)\)\. The fifth is to rebuild the experiment behind a published figure, and we test it on a fine\-mapping study from our own laboratory\([45](https://arxiv.org/html/2609.13437#bib.bib40)\)\. These five abilities run from execution to design, and together they ask how far a laboratory’s own methods carry once they leave that laboratory\. The five tasks also spread across four domains of life science\. TDC ADMET sits in drug discovery\. BiomniBench\-DA and GeneBench\-Pro cover genomic and single\-cell analysis\. ProteinGym sits in protein science, and the fine\-mapping study sits in statistical genetics\. Each task carries its own ceilings on turns, on cost, on the time of any one command and on the wall\-clock time of a whole run\. They run from 40 to 150 turns and from 5 to 40 US dollars, and we raise one only after a run has died against it\.

TDC ADMET\.The ADMET leaderboards of the Therapeutics Data Commons\([5](https://arxiv.org/html/2609.13437#bib.bib5)\)rank published methods on 22 datasets\. Those datasets cover absorption, distribution, metabolism, excretion and toxicity \(ADMET\)\. These entries are known to be hard to rebuild\. Ten distinct models occupy the top three places across the 22 leaderboards\. An independent assessment found that only three of them could be reproduced\([38](https://arxiv.org/html/2609.13437#bib.bib33)\)\. The rest failed on environments that would not build, or on splits that leaked test molecules into training\. We take the top three entries of each leaderboard, which gives 66 targets\. Sixty\-five of them remain reachable, since one repository has been deleted upstream\. Those entries resolve to 11 distinct repositories\. We build one skill from each of them in repository mode, so the library holds 11 skills\. We stage each entry with a training and validation table, a test table, and five seed splits, and all of these come from the leaderboard’s own files\. The task text then names the method to reproduce, and quotes the entry’s published value together with its standard deviation\. That value comes from the leaderboard’s own public record of the entry\. Finally, the agent trains on each of the five seeds and evaluates on the corresponding test split\. It then reports the per\-seed scores with their mean and standard deviation\.

BiomniBench\-DA\.BiomniBench\-DA\([30](https://arxiv.org/html/2609.13437#bib.bib48)\)curates 100 data\-analysis tasks across 17 task types from 21 published studies\. Each task ships the study’s public dataset together with a reference analytical trace that a domain expert wrote\. It also ships a rubric of five to ten criteria\. Each criterion sits at a decision point of that analysis\. Fifty of the 100 tasks have been released publicly, and those fifty are the ones we use here\. They span 16 of the 17 task types\. We stage the data ourselves, so the task is one of analysis\. We build skills here in task\-type mode, one for each task type\. The output contract requires both a written trace of what was done and an answer file, because the rubric grades the analysis and not only its conclusion\.

GeneBench\-Pro\.GeneBench\-Pro\([31](https://arxiv.org/html/2609.13437#bib.bib34)\)poses multi\-stage analyses from genomics and quantitative biology\. It gives the agent a brief context and a target estimand, and the agent must then find its own way to the correct workflow through several dependent decision points\. The benchmark comprises 129 problems, and its authors have released ten of them publicly\. Those ten are the ones we use here, and its authors hold the rest out\. We again build skills in task\-type mode, one per problem, because no repository is named for these tasks and the candidate methods have to be found first\. We stage the data in advance, so the agent never has to build the method\. The agent writes a single answer file in a fixed JSON schema, and a deterministic grader then compares the numeric answer against a stated tolerance\.

ProteinGym Variant Effect Prediction\.ProteinGym\([7](https://arxiv.org/html/2609.13437#bib.bib45)\)assembles deep mutational scanning assays across a wide range of proteins\. Each assay measures the effect of every single amino\-acid substitution in one protein, and the benchmark publishes the score of every method it catalogues on every assay\. We configure it here so that the method is not given\. Our sampling rule has four parts\. First, we draw 10 assays by stratified sampling over taxon and alignment depth, and we allocate places to strata by largest remainder in proportion to each stratum’s share of the full set\. Second, we take the assay whose mutation count is closest to the median within each stratum, and we break ties by the assay identifier\. Third, we constrain the sample to one assay per protein and to at most two per source study\. Fourth, we apply the same within\-stratum rule once more over ProteinGym’s own assay categories and add one assay for each of expression, activity and stability\. That fourth part exists because the first draw stratified on taxon and depth alone\. It returned no assay at all from the expression category and it over\-drew organismal fitness, since the cap of two per study also capped stability, which one campaign dominates\. We added to the first draw and removed nothing from it, because a sample discarded once its result is known is a sample chosen for its result\. The three assays the fourth part adds are the SARS\-CoV\-2 spike expression scan, the Romero 2015 scan of Q59976 and the Nutschel 2020 scan of*Bacillus subtilis*esterase A\. We fixed all four parts before any method was run, and none of them consults a method’s performance\. The library itself holds 26 skills, one for each GitHub\-hosted baseline in ProteinGym’s acknowledgements table\. We deliberately do not filter it by rank, because a library in which every method is strong would answer half the question before the agent is asked it\. We stage each assay with its variant list, its multiple sequence alignment and its predicted structure\. We also report how exposed a marked position is\. We take the same predicted structure, strip the hydrogens, and compute the solvent\-accessible surface area of every residue by the Shrake\-Rupley method at a probe radius of 1\.40 angstroms and 960 sphere points\. We divide that area by the theoretical maximum for the residue type in the extended Gly\-X\-Gly state\([52](https://arxiv.org/html/2609.13437#bib.bib49)\), which gives a relative solvent accessibility\. A value below 0\.15 marks a buried position by the usual convention\. Alignment\-based, sequence\-based and structure\-based methods are therefore all available to the agent\. Meanwhile we withhold the measured effects from every system alike\. The output contract then asks for one score per variant\. The task text states the required direction identically for every system we compare\.

UK Biobank Fine\-mapping Simulation\.The UK Biobank supplies the genotypes behind the first figure of a published fine\-mapping study\([45](https://arxiv.org/html/2609.13437#bib.bib40)\)\. That study comes from our own laboratory, and we stage the genotype panel it assembled for the agent\. We take the first figure of that study as our target here\. It reports a simulation over 10,000 Europeans from that cohort\. Each region in the simulation comes at random from chromosome 1 and carries 5,000 single\-nucleotide polymorphisms \(SNPs\)\. The figure compares five fine\-mapping methods across two causal architectures and five heritability settings\. The figure reports three quantities for each combination, namely the power to detect a causal SNP, the coverage of the credible sets, and their average size\. However a reproduction of figureis the harder request\([53](https://arxiv.org/html/2609.13437#bib.bib46)\)\. It reports a pattern rather than a value, so it often gives no single number to compare against\. We give the agent the article, that genotype panel, the repository the laboratory released, and two fine\-mapping executables\. An experiment produces a figure, and a method paper’s repository releases the method\. Reproducing the figure therefore asks for what a repository of that kind is not meant to carry\. An experiment fixes a grid of conditions and a number of replicates, and it then turns what comes back into a figure\. We take that experiment as the reproduction target here\. Five of its settings appear in the repository as bare numbers\. Two of them fix the geometry of each region\. One sets the width of the window around each risk gene, and the other sets the spacing between those windows\. Two more fix how heritability divides between causal and background SNPs, once for expression quantitative trait locus heritability and once for the phenotype\. The fifth fixes how the two causal SNPs of each gene divide between their two roles\. The student who ran the simulation has since graduated, and the agent therefore has to work out what each of the five settings controls\. It then has to rebuild the experiment around a method it already has\. This task measures whether it can recover what a laboratory knew but never wrote down\.

### 4\.5Evaluations

For each task we set out the scoring procedure below, alongside what it can and cannot see, because the scoring procedure decides what that task can detect\. Every one of them runs after the agent has terminated, and reads only the files the agent wrote\. We judge each task in its own metric and its own units, and average no score across tasks, since an error and a correlation share no scale and improve in opposite directions\. We report cross\-task summaries as counts instead\. Tasks also differ in what they tell the agent aboutμ⋆\\mu^\{\\star\}\. A leaderboard entry publishes its value as part of its own public record, and the task text then quotes that value identically for every system we compare\. Other tasks, by contrast, withhold the ground truth from every system alike\.

TDC ADMET\.We do not choose a tolerance of our own for this benchmark\. We take the one the leaderboard already publishes, since every entry publishes the spread of its own result\. The source therefore supplies the threshold, and we never have to defend a number of our own\. We score each entry in the metric its own leaderboard uses\. We then count a reproduction when that score falls inside the standard deviation the leaderboard publishes beside the entry’s value\. That spread lies between 0\.5% and 2% of the value for most entries, which is considerably stricter than a conventional tolerance band\. That bar, however, is also unforgiving of small numerical differences\. We add a ratio\-scale reading as well\. A reproduction counts as close on that reading when it falls within 5% of the published value\. We report both readings for every entry we attempt\.

BiomniBench\-DA\.We have no single right number to score an open analysis against\. This benchmark therefore scores what an answer contains rather than how far it sits from a target\. Its own protocol has a language model read the agent’s trace and apply the rubric\. That model awards each criterion full, half or no credit\. We score by a second route instead, because a language model does not score deterministically\. The same trace need not receive the same verdict twice, and no part of that verdict can be traced back to the criterion it came from\. We therefore match the trace and the answer against the wording of each criterion’s fully\-correct level, and we score the overlap between them\. That procedure returns the same value for the same answer every time, and every point it awards can also be attributed to a criterion\.

GeneBench\-Pro\.The agent works through a chain of dependent decisions in these problems\. Proximity to the answer therefore carries no information at all\. An agent that takes the wrong fork early produces a number of the correct order of magnitude that is nonetheless wrong\. The benchmark grades accordingly, with a binary verdict against a recoverable target under a tolerance it calibrates for that problem\. A verdict alone, however, does not say how far a run got before it went wrong\. We accordingly record a partial score over the quantities each problem asks for\. We report that score alongside the verdict\.

ProteinGym Variant Effect Prediction\.The assay files ProteinGym distributes carry the measured effect beside each variant\. Skills that we write in the reproduce\-a\-metric idiom then end with a correlation between a prediction and exactly that column\. We would therefore make the task trivially exploitable if we staged those files unchanged\. An arm that emitted the measured effect as its own prediction would score a perfect correlation, and its transcript would look entirely compliant\. We therefore remove the measured effects from the staged data, and we hold them in a directory no arm is given\. We score predictions by the Spearman correlation between the score an arm assigns to each variant and the effect the assay measured for it\. However, we do pay a cost when we withhold the effects\. An arm can no longer check its own sign convention against the data\. We state the required direction in the task text instead, and we use identical wording for every arm\. Finally, we read two further quantities from the leaderboard for context\. We take the best score any skill in the library can reach on that assay, and we take the best score any method has published\.

UK Biobank Fine\-mapping Simulation\.A figure leaves a reader with a shape rather than a number\. It says which method leads, which way a curve moves, and which condition is the easier one\. A shape does not by itself measure anything, however, and a comparison of images would test the plotting rather than the science\. We therefore score the figure twice over, once for the relations it asserts and once for the quantities the article states beside it\. We divide a hundred points among eight claims in three groups\. Forty points go to the relations the figure asserts\. The figure says that SuSiE2 leads on power, that power rises with heritability, and that the first causal architecture is the easier of the two\. Another forty go to the quantities\. We ask how large the gain over the alternatives is, how well the credible sets cover, and how large those sets are on average, and we compare each answer against the value the article reports\. The last twenty ask whether the work exists at all\. We check that the grid of conditions is complete and that the deliverables the article names are present\. Weights within each group follow how central a claim is to the published argument, and the largest single weight, 30 of the hundred, goes to the claim that the method the article introduces leads on power\. We derive the rubric from the article itself, and we apply it only once the run has finished\. We own the target study, so its precomputed outputs sit on the same cluster and can be read there\. A separate check therefore scans the run for any reference to them\.

### 4\.6Baselines

We use Opus 4\.8 as the base model of LabAgent\. We compare it against three systems that each receive the same task text word for word, the same staged data and the same budget ceiling\. Claude Code CLI runs on Opus 4\.8, the same base model, whereas Codex CLI runs on GPT\-5\.5\. Claude Science is a workbench for scientific work\. It ships more than sixty scientific databases and toolkits, spanning genomics, single\-cell analysis, proteomics, structural biology and cheminformatics\. Within it a coordinating agent delegates to specialist sub\-agents, and a separate reviewing agent then checks citations and calculations\. We run it on ProteinGym, where a system that already arrives with a library of methods is the most informative comparison available\. A comparison means most when the systems differ in one respect and agree in every other, and that is not what these comparisons are\. Each of the three is a complete system in its own right, not LabAgent with a part removed\. LabAgent and Claude Code share a base model and differ in everything else, and LabAgent and Codex differ in the base model as well\. No pairwise difference among them therefore attributes an outcome to any single design choice, and what they give is a comparison of systems as they are built and priced\. Attribution instead needs a fourth arm, in which we run LabAgent against itself with the explore agent removed\. That arm holds the harness, the base model, the task text and the budget fixed, and varies only whether a skill library exists\. A difference between it and the full system is therefore a difference the library made\.

### 4\.7Statistics and Reproducibility

We attempt each target once\. Every value we report is therefore a point estimate at the level of the run\. A task may nonetheless call for repetition inside that run\. The ADMET tasks require five seed splits, so a value reported for one of those targets is a mean over five fits and carries a dispersion of its own\. We compare two systems with a pairedtt\-test over the tasks that both completed\. For every such comparison we give the number of pairs, the mean difference, its standard deviation, the statistic and thepp\-value\. We report these even where the difference is not significant\. We also fixed two choices before any system ran\. The stratified rule that picks the ProteinGym assays looks at no method’s performance, and the measured effects were removed from the staged assay data for every system alike\. We exclude one ADMET target throughout, an entry whose repository has been deleted upstream and which no system can attempt\. Both stages of LabAgent use the same base model\. We run skill selection at temperature zero so that the choice is repeatable, and skill drafting at temperature 0\.3, retrying up to three times when the model returns malformed output\. The runs themselves were submitted as scheduler array jobs sharded round\-robin across nodes, on the execution loop of mini\-swe\-agent 2\.4\.1\. Finally, the 65 reachable ADMET targets cost US$296\.13 in total, a median of US$2\.76 and a mean of US$4\.56 each\.

### 4\.8Data Availability

The Therapeutics Data Commons ADMET benchmark group and ProteinGym are openly available from their respective projects\. So are the public releases of BiomniBench\-DA and GeneBench\-Pro\. We read the ADMET leaderboards on 2026\-06\-07, we confirmed the ProteinGym baseline repositories on 2026\-08\-17, and we use BiomniBench\-DA and GeneBench\-Pro as their projects released them\. The genotype panel behind the fine\-mapping simulation was assembled from UK Biobank data for an earlier study from our own laboratory\([45](https://arxiv.org/html/2609.13437#bib.bib40)\), and we stage that panel unchanged\. The access code for UK Biobank data is 29900\. Individual\-level UK Biobank data cannot be redistributed, and are available to approved researchers through the UK Biobank Access Management System\. The generated skill libraries and the reproduction trajectories that support the findings of this study are provided as Supplementary Data\.

### 4\.9Code Availability

We used the resources from the Yale High Performance Center \(Yale HPC\) to conduct all of the experiments\. The codes of LabAgent can be found in[https://github\.com/fpxlei/LabAgent](https://github.com/fpxlei/LabAgent)under MIT license\.

## 5Acknowledgments

T\.L\. acknowledges the support of OpenAI Researcher Access Program\.

## 6Author contribution

L\.L\. designed this study with T\.L\. L\.L\. implemented the method and performed experiments\. L\.L\., T\.L\., Y\.Z\. performed analyses\. All authors reviewed and revised the manuscript\. T\.L\. and H\.Z\. supervised this project\.

## References

- \[1\]H\. Wang, T\. Fu, Y\. Du, W\. Gao, K\. Huang, Z\. Liu, P\. Chandak, S\. Liu, P\. Van Katwyk, A\. Deac,et al\.\(2023\)Scientific discovery in the age of artificial intelligence\.Nature620\(7972\),pp\. 47–60\.Cited by:[§1](https://arxiv.org/html/2609.13437#S1.p1.1)\.
- \[2\]J\. Wei, Y\. Yang, X\. Zhang, Y\. Chen, X\. Zhuang, Z\. Gao, D\. Zhou, G\. Wang, Z\. Gao, J\. Cao, Z\. Qiu, M\. Hu, C\. Ma, S\. Tang, J\. He, C\. Song, X\. He, Q\. Zhang, C\. You, S\. Zheng, N\. Ding, W\. Ouyang, N\. Dong, Y\. Cheng, S\. Sun, L\. Bai, and B\. Zhou\(2025\)From ai for science to agentic science: a survey on autonomous scientific discovery\.External Links:2508\.14111,[Link](https://arxiv.org/abs/2508.14111)Cited by:[§1](https://arxiv.org/html/2609.13437#S1.p1.1)\.
- \[3\]H\. Cui, C\. Wang, H\. Maan, K\. Pang, F\. Luo, N\. Duan, and B\. Wang\(2024\)ScGPT: toward building a foundation model for single\-cell multi\-omics using generative ai\.Nature Methods21,pp\. 1470 – 1480\.External Links:[Link](https://api.semanticscholar.org/CorpusID:268028472)Cited by:[§1](https://arxiv.org/html/2609.13437#S1.p1.1)\.
- \[4\]T\. Liu, T\. Huang, L\. Wang, Y\. Lin, R\. Ying, and H\. Zhao\(2025\)UNICORN: towards universal cellular expression prediction with a multi\-task learning framework\.Nature Communications16\.External Links:[Link](https://api.semanticscholar.org/CorpusID:275920313)Cited by:[§1](https://arxiv.org/html/2609.13437#S1.p1.1)\.
- \[5\]K\. Huang, T\. Fu, W\. Gao, Y\. Zhao, Y\. H\. Roohani, J\. Leskovec, C\. W\. Coley, C\. Xiao, J\. Sun, and M\. Zitnik\(2021\)Therapeutics data commons: machine learning datasets and tasks for drug discovery and development\.InNeurIPS Datasets and Benchmarks,External Links:[Link](https://api.semanticscholar.org/CorpusID:237264044)Cited by:[§1](https://arxiv.org/html/2609.13437#S1.p1.1),[§4\.4](https://arxiv.org/html/2609.13437#S4.SS4.p1.1),[§4\.4](https://arxiv.org/html/2609.13437#S4.SS4.p2.1)\.
- \[6\]T\. Liu, T\. Chu, X\. Luo, and H\. Zhao\(2025\)Building a unified model for drug synergy analysis powered by large language models\.Nature Communications16\.External Links:[Link](https://api.semanticscholar.org/CorpusID:278662786)Cited by:[§1](https://arxiv.org/html/2609.13437#S1.p1.1)\.
- \[7\]P\. Notin, A\. Kollasch, D\. Ritter, L\. Van Niekerk, S\. Paul, H\. Spinner, N\. Rollins, A\. Shaw, R\. Orenbuch, R\. Weitzman,et al\.\(2023\)Proteingym: large\-scale benchmarks for protein fitness prediction and design\.Advances in neural information processing systems36,pp\. 64331–64379\.Cited by:[§1](https://arxiv.org/html/2609.13437#S1.p1.1),[§2](https://arxiv.org/html/2609.13437#S2.p12.1),[§4\.4](https://arxiv.org/html/2609.13437#S4.SS4.p1.1),[§4\.4](https://arxiv.org/html/2609.13437#S4.SS4.p5.1)\.
- \[8\]G\. Wang, A\. K\. Sarkar, P\. Carbonetto, and M\. Stephens\(2018\)A simple new approach to variable selection in regression, with application to genetic fine mapping\.Journal of the Royal Statistical Society\. Series B, Statistical methodology82,pp\. 1273 – 1300\.External Links:[Link](https://api.semanticscholar.org/CorpusID:91375951)Cited by:[§1](https://arxiv.org/html/2609.13437#S1.p1.1),[§2](https://arxiv.org/html/2609.13437#S2.p18.1)\.
- \[9\]M\. D\. Wilkinson, M\. Dumontier, I\. J\. Aalbersberg, G\. Appleton, M\. Axton, A\. Baak, N\. Blomberg, J\. Boiten, L\. O\. B\. da Silva Santos, P\. E\. Bourne, J\. Bouwman, A\. J\. Brookes, T\. Clark, M\. Crosas, I\. Dillo, O\. Dumon, S\. C\. Edmunds, C\. T\. A\. Evelo, R\. Finkers, A\. N\. González\-Beltrán, A\. J\. G\. Gray, P\. Groth, C\. A\. Goble, J\. S\. Grethe, J\. Heringa, P\. A\. C\. ’\. Hoen, R\. W\. W\. Hooft, T\. Kuhn, R\. G\. Kok, J\. N\. Kok, S\. J\. Lusher, M\. E\. Martone, A\. Mons, A\. L\. Packer, B\. Persson, P\. Rocca\-Serra, M\. Roos, R\. C\. van Schaik, S\. Sansone, E\. A\. Schultes, T\. Sengstag, T\. Slater, G\. O\. Strawn, M\. A\. Swertz, M\. Thompson, J\. van der Lei, E\. M\. van Mulligen, J\. Velterop, A\. Waagmeester, P\. Wittenburg, K\. Wolstencroft, J\. Zhao, and B\. Mons\(2016\)The fair guiding principles for scientific data management and stewardship\.Scientific Data3\.External Links:[Link](https://api.semanticscholar.org/CorpusID:8755162)Cited by:[§1](https://arxiv.org/html/2609.13437#S1.p1.1)\.
- \[10\]M\. Baker\(2016\)1,500 scientists lift the lid on reproducibility\.Nature533,pp\. 452–454\.External Links:[Link](https://api.semanticscholar.org/CorpusID:4460617)Cited by:[§1](https://arxiv.org/html/2609.13437#S1.p1.1)\.
- \[11\]L\. Li, R\. Wang, H\. Song, Y\. Mao, T\. Zhang, Y\. Wang, J\. Fan, Y\. Zhang, J\. Ye, C\. Zhang, and Y\. Gong\(2026\)What papers don’t tell you: recovering tacit knowledge for automated paper reproduction\.External Links:2603\.01801,[Link](https://arxiv.org/abs/2603.01801)Cited by:[§1](https://arxiv.org/html/2609.13437#S1.p1.1)\.
- \[12\]J\. Gottweis, W\. Weng, A\. Daryin, T\. Tu, P\. Sirkovic, A\. Myaskovsky, G\. Glowaty, F\. Weissenberger, A\. Orlandi, D\. Popovici,et al\.\(2026\)Accelerating scientific discovery with co\-scientist\.Nature,pp\. 1–3\.Cited by:[§1](https://arxiv.org/html/2609.13437#S1.p2.1)\.
- \[13\]D\. A\. Boiko, R\. MacKnight, B\. C\. Kline, and G\. Gomes\(2023\)Autonomous chemical research with large language models\.Nature624,pp\. 570 – 578\.External Links:[Link](https://api.semanticscholar.org/CorpusID:266432059)Cited by:[§1](https://arxiv.org/html/2609.13437#S1.p2.1)\.
- \[14\]K\. Swanson, W\. Wu, N\. L\. Bulaong, J\. E\. Pak, and J\. Zou\(2025\)The virtual lab of ai agents designs new sars\-cov\-2 nanobodies\.Nature646\(8085\),pp\. 716–723\.Cited by:[§1](https://arxiv.org/html/2609.13437#S1.p2.1)\.
- \[15\]C\. Lu, C\. Lu, R\. T\. Lange, Y\. Yamada, S\. Hu, J\. Foerster, D\. Ha, and J\. Clune\(2026\)Towards end\-to\-end automation of ai research\.Nature651\(8107\),pp\. 914–919\.Cited by:[§1](https://arxiv.org/html/2609.13437#S1.p2.1)\.
- \[16\]L\. Mitchener, A\. Yiu, B\. Chang, M\. Bourdenx, T\. Nadolski, A\. Sulovari, E\. C\. Landsness, D\. L\. Barabási, S\. Narayanan, N\. Evans, S\. Reddy, M\. S\. Foiani, A\. Kamal, L\. P\. Shriver, F\. Cao, A\. T\. Wassie, J\. M\. Laurent, E\. Melville\-Green, M\. C\. Ramos, A\. Bou, K\. F\. Roberts, S\. Zagorac, T\. C\. Orr, M\. E\. Orr, K\. J\. Zwezdaryk, A\. E\. Ghareeb, L\. McCoy, B\. Gomes, E\. A\. Ashley, K\. E\. Duff, T\. Buonassisi, T\. Rainforth, R\. J\. Bateman, M\. Skarlinski, S\. G\. Rodriques, M\. M\. Hinks, and A\. D\. White\(2025\)Kosmos: an ai scientist for autonomous discovery\.ArXivabs/2511\.02824\.External Links:[Link](https://api.semanticscholar.org/CorpusID:282748827)Cited by:[§1](https://arxiv.org/html/2609.13437#S1.p2.1)\.
- \[17\]Y\. Roohani, A\. Lee, Q\. Huang, J\. Vora, Z\. Steinhart, K\. Huang, A\. Marson, P\. Liang, and J\. Leskovec\(2025\)Biodiscoveryagent: an ai agent for designing genetic perturbation experiments\.InInternational Conference on Learning Representations,Vol\.2025,pp\. 26417–26466\.Cited by:[§1](https://arxiv.org/html/2609.13437#S1.p2.1)\.
- \[18\]Z\. Wang, Q\. Jin, C\. Wei, S\. Tian, P\. Lai, Q\. Zhu, C\. Day, C\. Ross, R\. Leaman, and Z\. Lu\(2025\)GeneAgent: self\-verification language agent for gene\-set analysis using domain databases\.Nature Methods22\(8\),pp\. 1677–1685\.Cited by:[§1](https://arxiv.org/html/2609.13437#S1.p2.1)\.
- \[19\]R\. Jin, Z\. Zhang, M\. Wang, and L\. Cong\(2025\)STELLA: self\-evolving llm agent for biomedical research\.External Links:2507\.02004,[Link](https://arxiv.org/abs/2507.02004)Cited by:[§1](https://arxiv.org/html/2609.13437#S1.p2.1)\.
- \[20\]K\. Huang, S\. Zhang, H\. Wang, Y\. Qu, Y\. Lu, Y\. Roohani, R\. Li, L\. Qiu, G\. Li, J\. Zhang, D\. Yin, S\. Marwaha, J\. N\. Carter, X\. Zhou, M\. Wheeler, J\. A\. Bernstein, M\. Wang, P\. He, J\. Zhou, M\. Snyder, L\. Cong, A\. Regev, and J\. Leskovec\(2025\)Biomni: a general\-purpose biomedical ai agent\.bioRxiv\.External Links:[Document](https://dx.doi.org/10.1101/2025.05.30.656746),[Link](https://www.biorxiv.org/content/early/2025/06/02/2025.05.30.656746),https://www\.biorxiv\.org/content/early/2025/06/02/2025\.05\.30\.656746\.full\.pdfCited by:[§1](https://arxiv.org/html/2609.13437#S1.p2.1)\.
- \[21\]J\. Yang, C\. Jimenez, A\. Wettig, K\. Lieret, S\. Yao, K\. Narasimhan, and O\. Press\(2024\)Swe\-agent: agent\-computer interfaces enable automated software engineering\.Advances in Neural Information Processing Systems37,pp\. 50528–50652\.Cited by:[§1](https://arxiv.org/html/2609.13437#S1.p2.1),[§4\.2](https://arxiv.org/html/2609.13437#S4.SS2.p3.1)\.
- \[22\]X\. Wang, B\. Li, Y\. Song, F\. F\. Xu, X\. Tang, M\. Zhuge, J\. Pan, Y\. Song, B\. Li, J\. Singh,et al\.\(2025\)Openhands: an open platform for ai software developers as generalist agents\.InInternational Conference on Learning Representations,Vol\.2025,pp\. 65882–65919\.Cited by:[§1](https://arxiv.org/html/2609.13437#S1.p2.1)\.
- \[23\]X\. Zhao, Z\. Sang, Y\. Li, Q\. Shi, W\. Zhao, S\. Wang, D\. Zhang, X\. Han, Z\. Liu, and M\. Sun\(2026\)Autoreproduce: automatic ai experiment reproduction with paper lineage\.InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 21920–21942\.Cited by:[§1](https://arxiv.org/html/2609.13437#S1.p2.1)\.
- \[24\]J\. Nam, J\. Yoon, J\. Chen, J\. Shin, S\. Arik, and T\. Pfister\(2026\)Mle\-star: machine learning engineering agent via search and targeted refinement\.Advances in Neural Information Processing Systems38,pp\. 116692–116712\.Cited by:[§1](https://arxiv.org/html/2609.13437#S1.p2.1)\.
- \[25\]Z\. Jiang, D\. Schmidt, D\. Srikanth, D\. Xu, I\. Kaplan, D\. Jacenko, and Y\. Wu\(2025\)AIDE: ai\-driven exploration in the space of code\.External Links:2502\.13138,[Link](https://arxiv.org/abs/2502.13138)Cited by:[§1](https://arxiv.org/html/2609.13437#S1.p2.1)\.
- \[26\]X\. Yang, X\. Yang, S\. Fang, Y\. Zhang, J\. Wang, B\. Xian, Q\. Li, J\. Li, M\. Xu, Y\. Li, H\. Pan, Y\. Zhang, W\. Liu, Y\. Shen, W\. Chen, and J\. Bian\(2025\)R&D\-agent: an llm\-agent framework towards autonomous data science\.External Links:2505\.14738,[Link](https://arxiv.org/abs/2505.14738)Cited by:[§1](https://arxiv.org/html/2609.13437#S1.p2.1)\.
- \[27\]G\. Starace, O\. Jaffe, D\. Sherburn, J\. Aung, J\. S\. Chan, L\. Maksin, R\. Dias, E\. Mays, B\. Kinsella, W\. Thompson, J\. Heidecke, A\. Glaese, and T\. Patwardhan\(2025\)PaperBench: evaluating AI’s ability to replicate AI research\.InProceedings of the 42nd International Conference on Machine Learning,A\. Singh, M\. Fazel, D\. Hsu, S\. Lacoste\-Julien, F\. Berkenkamp, T\. Maharaj, K\. Wagstaff, and J\. Zhu \(Eds\.\),Proceedings of Machine Learning Research, Vol\.267,pp\. 56843–56873\.External Links:[Link](https://proceedings.mlr.press/v267/starace25a.html)Cited by:[§1](https://arxiv.org/html/2609.13437#S1.p2.1)\.
- \[28\]J\. S\. Chan, N\. Chowdhury, O\. Jaffe, J\. Aung, D\. Sherburn, E\. Mays, G\. Starace, K\. Liu, L\. Maksin, T\. Patwardhan, A\. Madry, and L\. Weng\(2025\)MLE\-bench: evaluating machine learning agents on machine learning engineering\.InInternational Conference on Learning Representations,Y\. Yue, A\. Garg, N\. Peng, F\. Sha, and R\. Yu \(Eds\.\),Vol\.2025,pp\. 50466–50494\.External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2025/file/7e3767db483c942b883eb4f8cfb74e31-Paper-Conference.pdf)Cited by:[§1](https://arxiv.org/html/2609.13437#S1.p2.1)\.
- \[29\]L\. Mitchener, J\. M\. Laurent, A\. Andonian, B\. Tenmann, S\. Narayanan, G\. P\. Wellawatte, A\. White, L\. Sani, and S\. G\. Rodriques\(2025\)BixBench: a comprehensive benchmark for llm\-based agents in computational biology\.External Links:2503\.00096,[Link](https://arxiv.org/abs/2503.00096)Cited by:[§1](https://arxiv.org/html/2609.13437#S1.p2.1)\.
- \[30\]Y\. Qu, Y\. Lu, X\. Tu, S\. Zhang, T\. She, A\. G\. Shaw, J\. Shih, B\. Zhao, M\. Shen, H\. Yang, J\. Yan, R\. Zhang, X\. Wu, T\. Li, L\. Cong, X\. Hu, Y\. Jiang, J\. Dong, T\. Peng, J\. Leskovec, and K\. Huang\(2026\)BiomniBench: process\-level evaluation of llm agents for real\-world biomedical research\.bioRxiv\.External Links:[Document](https://dx.doi.org/10.64898/2026.05.12.724604),[Link](https://www.biorxiv.org/content/early/2026/05/14/2026.05.12.724604),https://www\.biorxiv\.org/content/early/2026/05/14/2026\.05\.12\.724604\.full\.pdfCited by:[§1](https://arxiv.org/html/2609.13437#S1.p2.1),[§2](https://arxiv.org/html/2609.13437#S2.p6.1),[§4\.4](https://arxiv.org/html/2609.13437#S4.SS4.p1.1),[§4\.4](https://arxiv.org/html/2609.13437#S4.SS4.p3.1)\.
- \[31\]J\. Li and A\. Ho\(2026\)GeneBench\-pro: evaluating multistage statistical reasoning in genomics, quantitative biology, and translational biomedicine\.bioRxiv\.External Links:[Document](https://dx.doi.org/10.64898/2026.06.29.735386),[Link](https://www.biorxiv.org/content/early/2026/06/30/2026.06.29.735386.1),https://www\.biorxiv\.org/content/early/2026/06/30/2026\.06\.29\.735386\.1\.full\.pdfCited by:[§1](https://arxiv.org/html/2609.13437#S1.p2.1),[§2](https://arxiv.org/html/2609.13437#S2.p6.1),[§4\.4](https://arxiv.org/html/2609.13437#S4.SS4.p1.1),[§4\.4](https://arxiv.org/html/2609.13437#S4.SS4.p4.1)\.
- \[32\]AnthropicAgent skills\.Note:[https://platform\.claude\.com/docs/en/agents\-and\-tools/agent\-skills/overview](https://platform.claude.com/docs/en/agents-and-tools/agent-skills/overview)Accessed via Claude Developer Platform documentationCited by:[§1](https://arxiv.org/html/2609.13437#S1.p3.1)\.
- \[33\]S\. Shen, W\. Cheng, M\. Ma, A\. Turcan, M\. J\. Zhang, and J\. Ma\(2026\)SKILLFOUNDRY: building self\-evolving agent skill libraries from heterogeneous scientific resources\.External Links:2604\.03964,[Link](https://arxiv.org/abs/2604.03964)Cited by:[§1](https://arxiv.org/html/2609.13437#S1.p3.1)\.
- \[34\]X\. Huang, J\. Chen, Y\. Fei, Z\. Li, P\. Schwaller, and G\. Ceder\(2026\)CASCADE: cumulative agentic skill creation through autonomous development and evolution\.External Links:2512\.23880,[Link](https://arxiv.org/abs/2512.23880)Cited by:[§1](https://arxiv.org/html/2609.13437#S1.p3.1)\.
- \[35\]B\. Zheng, M\. Y\. Fatemi, X\. Jin, Z\. Z\. Wang, A\. Gandhi, Y\. Song, Y\. Gu, J\. Srinivasa, G\. Liu, G\. Neubig, and Y\. Su\(2025\)SkillWeaver: web agents can self\-improve by discovering and honing skills\.External Links:2504\.07079,[Link](https://arxiv.org/abs/2504.07079)Cited by:[§1](https://arxiv.org/html/2609.13437#S1.p3.1)\.
- \[36\]G\. Wang, Y\. Xie, Y\. Jiang, A\. Mandlekar, C\. Xiao, Y\. Zhu, L\. \(\. Fan, and A\. Anandkumar\(2023\)Voyager: an open\-ended embodied agent with large language models\.ArXivabs/2305\.16291\.External Links:[Link](https://api.semanticscholar.org/CorpusID:258887849)Cited by:[§1](https://arxiv.org/html/2609.13437#S1.p3.1)\.
- \[37\]M\. J\. Waring, J\. Arrowsmith, A\. R\. Leach, P\. D\. Leeson, S\. Mandrell, R\. M\. Owen, G\. Pairaudeau, W\. D\. Pennie, S\. D\. Pickett, J\. Wang,et al\.\(2015\)An analysis of the attrition of drug candidates from four major pharmaceutical companies\.Nature reviews Drug discovery14\(7\),pp\. 475–486\.Cited by:[§2](https://arxiv.org/html/2609.13437#S2.p2.1)\.
- \[38\]I\. Koleiev, R\. Stratiichuk, N\. Shevchuk, M\. Melnychenko, O\. Nyporko, D\. Todoryshyn, V\. Husak, S\. Starosyla, S\. Yesylevskyy, and A\. Nafiiev\(2026\)Critical assessment of ml models for admet prediction in tdc leaderboards\.bioRxiv\.External Links:[Link](https://api.semanticscholar.org/CorpusID:286229664)Cited by:[§2](https://arxiv.org/html/2609.13437#S2.p2.1),[§4\.4](https://arxiv.org/html/2609.13437#S4.SS4.p2.1)\.
- \[39\]T\. N\. Starr, A\. J\. Greaney, S\. K\. Hilton, D\. Ellis, K\. H\. Crawford, A\. S\. Dingens, M\. J\. Navarro, J\. E\. Bowen, M\. A\. Tortorici, A\. C\. Walls,et al\.\(2020\)Deep mutational scanning of sars\-cov\-2 receptor binding domain reveals constraints on folding and ace2 binding\.cell182\(5\),pp\. 1295–1310\.Cited by:[§2](https://arxiv.org/html/2609.13437#S2.p14.1)\.
- \[40\]I\. Meitlis, E\. J\. Allenspach, B\. M\. Bauman, I\. Q\. Phan, G\. Dabbah, E\. G\. Schmitt, N\. D\. Camp, T\. R\. Torgerson, D\. A\. Nickerson, M\. J\. Bamshad,et al\.\(2020\)Multiplexed functional assessment of genetic variants in card11\.The American Journal of Human Genetics107\(6\),pp\. 1029–1043\.Cited by:[§2](https://arxiv.org/html/2609.13437#S2.p15.1)\.
- \[41\]P\. Zhao, Y\. Hu, D\. Sun, Q\. Meng, L\. Zhang, X\. Zhang, L\. Tan, Y\. Zhang, Y\. Ding, and X\. He\(2022\)A novel card11 germline mutation in a chinese patient of b cell expansion with nf\-κ\\kappab and t cell anergy \(benta\) and literature review\.Frontiers in Immunology13,pp\. 943027\.Cited by:[§2](https://arxiv.org/html/2609.13437#S2.p15.1)\.
- \[42\]G\. Van Pouderoyen, T\. Eggert, K\. Jaeger, and B\. W\. Dijkstra\(2001\)The crystal structure of bacillus subtili lipase: a minimalα\\alpha/β\\betahydrolase fold enzyme\.Journal of molecular biology309\(1\),pp\. 215–226\.Cited by:[§2](https://arxiv.org/html/2609.13437#S2.p15.1)\.
- \[43\]A\. Culina, I\. Van Den Berg, S\. Evans, and A\. Sánchez\-Tójar\(2020\)Low availability of code in ecology: a call for urgent action\.PLoS Biology18\(7\),pp\. e3000763\.Cited by:[§2](https://arxiv.org/html/2609.13437#S2.p17.1)\.
- \[44\]S\. Samuel and D\. Mietchen\(2022\)Computational reproducibility of jupyter notebooks from biomedical publications\.GigaScience13\.External Links:[Link](https://api.semanticscholar.org/CorpusID:252185299)Cited by:[§2](https://arxiv.org/html/2609.13437#S2.p17.1),[§4\.1](https://arxiv.org/html/2609.13437#S4.SS1.p4.1),[§4\.3](https://arxiv.org/html/2609.13437#S4.SS3.p4.1)\.
- \[45\]X\. Zhang, W\. Jiang, and H\. Zhao\(2024\)Integration of expression qtls with fine mapping via susie\.PLoS Genetics20\(1\),pp\. e1010929\.Cited by:[Supplementary Tab\. 9](https://arxiv.org/html/2609.13437#Ax1.T9.4.6.2.1.1),[§2](https://arxiv.org/html/2609.13437#S2.p17.1),[§2](https://arxiv.org/html/2609.13437#S2.p18.1),[§4\.4](https://arxiv.org/html/2609.13437#S4.SS4.p1.1),[§4\.4](https://arxiv.org/html/2609.13437#S4.SS4.p6.1),[§4\.8](https://arxiv.org/html/2609.13437#S4.SS8.p1.1)\.
- \[46\]G\. Kichaev, W\. Yang, S\. Lindstrom, F\. Hormozdiari, E\. Eskin, A\. L\. Price, P\. Kraft, and B\. Pasaniuc\(2014\)Integrating functional data to prioritize causal variants in statistical fine\-mapping studies\.PLoS genetics10\(10\),pp\. e1004722\.Cited by:[§2](https://arxiv.org/html/2609.13437#S2.p18.1)\.
- \[47\]Y\. Zou, P\. Carbonetto, D\. Xie, G\. Wang, and M\. Stephens\(2026\)Fast and flexible joint fine\-mapping of multiple traits via the sum of single effects model\.Nature Genetics58\(2\),pp\. 454–462\.Cited by:[§2](https://arxiv.org/html/2609.13437#S2.p18.1)\.
- \[48\]N\. Hernández, J\. Soenksen, P\. Newcombe, M\. Sandhu, I\. Barroso, C\. Wallace, and J\. Asimit\(2021\)The flashfm approach for fine\-mapping multiple quantitative traits\.Nature communications12\(1\),pp\. 6147\.Cited by:[§2](https://arxiv.org/html/2609.13437#S2.p18.1)\.
- \[49\]C\. Benner, C\. C\. Spencer, A\. S\. Havulinna, V\. Salomaa, S\. Ripatti, and M\. Pirinen\(2016\)FINEMAP: efficient variable selection using summary data from genome\-wide association studies\.Bioinformatics32\(10\),pp\. 1493–1501\.Cited by:[§2](https://arxiv.org/html/2609.13437#S2.p18.1)\.
- \[50\]J\. Spracklen, R\. Wijewickrama, A\. N\. Sakib, A\. Maiti, and B\. Viswanath\(2025\)We have a package for you\! a comprehensive analysis of package hallucinations by code generating\{\\\{llms\}\\\}\.In34th USENIX Security Symposium \(USENIX Security 25\),pp\. 3687–3706\.Cited by:[§4\.1](https://arxiv.org/html/2609.13437#S4.SS1.p2.1)\.
- \[51\]F\. Liu, Y\. Liu, L\. Shi, Z\. Yang, L\. Zhang, X\. Lian, Z\. Li, and Y\. Ma\(2024\)Beyond functional correctness: exploring hallucinations in llm\-generated code\.IEEE Transactions on Software Engineering52,pp\. 1037–1055\.External Links:[Link](https://api.semanticscholar.org/CorpusID:268819908)Cited by:[§4\.1](https://arxiv.org/html/2609.13437#S4.SS1.p2.1)\.
- \[52\]M\. Z\. Tien, A\. G\. Meyer, D\. K\. Sydykova, S\. J\. Spielman, and C\. O\. Wilke\(2013\)Maximum allowed solvent accessibilites of residues in proteins\.PloS one8\(11\),pp\. e80635\.Cited by:[Supplementary Tab\. 5](https://arxiv.org/html/2609.13437#Ax1.T5),[§4\.4](https://arxiv.org/html/2609.13437#S4.SS4.p5.1)\.
- \[53\]J\. Miao, J\. R\. Davis, Y\. Zhang, J\. K\. Pritchard, and J\. Zou\(2025\)Paper2Agent: reimagining research papers as interactive and reliable ai agents\.External Links:2509\.06917,[Link](https://arxiv.org/abs/2509.06917)Cited by:[§4\.4](https://arxiv.org/html/2609.13437#S4.SS4.p6.1)\.

## Supplementary Information

### A\. Prompts and the acceptance rubric

The Methods describe what each prompt does\. We show every prompt here unchanged, with its Jinja tags intact, so that a reader can see which fields the harness fills at dispatch and which text is fixed\.

The reproduce agent works from two pairs\. The first pair serves TDC ADMET and ProteinGym, where the agent has to build third\-party research code, and its system prompt opens with a probe of the machine whose findings select the install commands that follow\. The second pair serves BiomniBench\-DA, GeneBench\-Pro and the fine\-mapping study, and it is built around skill loci the agent reads on demand, an answer file in a fixed schema and a deterministic checker\. A caller can replace the answer contract through one argument, and the fine\-mapping run did so, since its scorer reads a table of metrics rather than a JSON answer\.

YouareanautonomousexperimentreproductionagentrunningonanHPCcluster\(Linux,noDockeravailable,condamaybeavailable\)\.

Yourtaskistoreproduceanexperimentusingthe\*\*\{\{skill\.name\}\}\*\*toolbyfollowingitsdocumentationexactly\.

—

\#\#ToolDocumentation:\{\{skill\.name\}\}

\{\{skill\.description\}\}

\{\{skill\.full\_markdown\}\}

—

\#\#OperatingConstraints

\-YouhaveaccesstoONEactiontype:‘bash\_command‘\.Issueonebashcommandperturn\.

\-Workingdirectory:‘\{\{work\_dir\}\}‘

\{%ifconda\_envs\_root%\}\-Condaenvironmentsroot:‘\{\{conda\_envs\_root\}\}‘

\{%endif%\}\-DoNOTuseDockerorSingularity\.Useconda/pip/sourceinstallsasdocumented\.

\-Aftereachcommand,checktheexitcodeandoutputbeforeproceeding\.

\-Ifastepfails,readtheerrormessageandtryanalternative\(differentinstallpath,missingdependency,reformattedinput\)beforegivingup\.

\-Printinformativeprogressmessagessothetrajectoryiseasytofollow\.

\#\#Workflow\(followinthisorder\)

\#\#\#Step0–SystemProbe\(ALWAYSrunfirst\)

Runthefollowingdiagnosticsbeforetouchinganyenvironment:

“‘bash

echo"===GCC==="&&gcc–version2\>/dev/null\|head\-1\|\|echo"gccnotfound"

echo"===CUDA\(nvcc\)==="&&nvcc–version2\>/dev/null\|grep"release"\|\|echo"nvccnotfound"

echo"===CUDA\(nvidia\-smi\)==="&&nvidia\-smi2\>/dev/null\|grep\-E"CUDAVersion\|DriverVersion"\|\|echo"nvidia\-sminotavailable"

echo"===Python==="&&\(python–version2\>/dev/null\|\|python3–version\)

echo"===Conda==="&&\(conda–version2\>/dev/null\|\|echo"condanotfound"\)

echo"===HPCModules==="&&\(moduleavail2\>&1\|grep\-iE"pytorch\|cuda"\|head\-10\|\|echo"modulecommandnotavailable"\)

“‘

Recordthe\*\*CUDAversion\*\*\(e\.g\.,11\.7,12\.1\)and\*\*GCCversion\*\*\(e\.g\.,8\.5\.0,12\.2\.0\)fromtheoutput\.

ThesedeterminewhichPyTorch,torch\_geometric,andotherGPU\-dependentpackagestoinstall\.

—

\#\#\#Step1–EnvironmentSetup

Create\(orreuse\)acondaenvironment,theninstalldependenciesusingversion\-appropriatecommands

derivedfromtheStep0probe\.

\#\#\#\#1a\.PyTorch\(version\-aware\)

ChoosetheinstallcommandthatmatchesthedetectedCUDAversion:

\|DetectedCUDA\|Installcommand\|

\|—\|—\|

\|11\.7\|‘pipinstalltorch==2\.0\.1\+cu117torchvision==0\.15\.2\+cu117–index\-urlhttps://download\.pytorch\.org/whl/cu117‘\|

\|11\.8\|‘pipinstalltorch==2\.1\.2\+cu118torchvision==0\.16\.2\+cu118–index\-urlhttps://download\.pytorch\.org/whl/cu118‘\|

\|12\.1\|‘pipinstalltorch==2\.1\.2\+cu121torchvision==0\.16\.2\+cu121–index\-urlhttps://download\.pytorch\.org/whl/cu121‘\|

\|12\.2\-12\.4\|‘pipinstalltorch==2\.3\.0\+cu121torchvision==0\.18\.0\+cu121–index\-urlhttps://download\.pytorch\.org/whl/cu121‘\|

\|NoGPU/CPU\|‘pipinstalltorchtorchvision–index\-urlhttps://download\.pytorch\.org/whl/cpu‘\|

\*\*HPCshortcut:\*\*If‘moduleavail‘listedamatchingPyTorchmodule\(e\.g\.‘PyTorch/2\.1\.2\-foss\-2022b\-CUDA\-12\.1\.1‘\),

loaditwith‘moduleload<name\>‘insteadofpip\-installingtoavoidaredundantdownload\.

\#\#\#\#1b\.PyTorchGeometric\(torch\_geometric/pyg\)

AfterPyTorchisconfirmedinstalled,run:

“‘bash

\#DetecttheexacttorchversionstringandCUDAtagthatwasinstalled

TORCH=$\(python\-c"importtorch;print\(torch\.\_\_version\_\_\)"\)

CUDA=$\(python\-c"importtorch;v=torch\.version\.cuda;print\(’cu’\+v\.replace\(’\.’,”\)ifvelse’cpu’\)"2\>/dev/null\|\|echo"cpu"\)

echo"Detectedtorch=$\{TORCH\}cuda\_tag=$\{CUDA\}"

\#Installtorch\_geometriccore

pipinstalltorch\_geometric

\#InstalloptionalbutrecommendedC\+\+extensionsfromthepre\-builtwheelindex

pipinstalltorch\_scattertorch\_sparsetorch\_clustertorch\_spline\_conv\\

\-fhttps://data\.pyg\.org/whl/torch\-$\{TORCH\}\+$\{CUDA\}\.html\\

\|\|pipinstalltorch\_scattertorch\_sparsetorch\_clustertorch\_spline\_conv\\

\-fhttps://data\.pyg\.org/whl/torch\-$\{TORCH\}\.html

“‘

Ifnopre\-builtwheelisavailableforthedetectedversion,fallbacktobuildingfromsource

\(requiresGCC\>=9;ifGCC8\.xisactive,firsttry‘condainstall\-cconda\-forgegcc=12‘\):

“‘bash

pipinstalltorch\_scattertorch\_sparsetorch\_clustertorch\_spline\_conv–no\-binary:all:

“‘

\#\#\#\#1c\.scikit\-learn

Preferconda\(resolvesBLAS/LAPACKautomatically\)wheninacondaenvironment:

“‘bash

condainstall\-yscikit\-learn

“‘

Otherwiseusepip–anyversion\>=1\.0supportsPython3\.9\+:

“‘bash

pipinstall"scikit\-learn\>=1\.0"

“‘

\#\#\#\#1d\.Generaldependencyinstallationorder

Installpackagesinthissequencetominimiseconflicts:

1\.System\-leveltoolsandcompilers\(viacondaormoduleload\)

2\.PyTorch\(per1aabove\)

3\.PyTorchGeometricextensions\(per1babove\)

4\.OtherMLlibraries\(scikit\-learn,numpy,scipy–preferconda\)

5\.Domain\-specificpackageslistedintheskilldocumentation\(viapiporcondaasspecified\)

—

\#\#\#Step2–DataValidation

Confirmthedatasetexists,checkitsformat,andreformattomatchthetool’sexpectedinputifneeded\.

\#\#\#Step3–RunExperiment

Executethetool’scoreworkflowexactlyasshownintheWorkflow/QuickStartsectionabove\.

\#\#\#Step4–EvaluateResults

Inspectoutputfilesandcomputeanymetricsmentionedinthedocumentation\.

\#\#\#Step5–WriteSummary

Printthecompletionsignalbelow\.

—

\#\#CompletionSignal\(REQUIRED–printthisasyourverylastbashoutput\)

Whentheexperimentisdone,runthisasyour\*\*finalbashcommand\*\*inoneshot:

“‘bash

echo"COMPLETE\_TASK\_AND\_SUBMIT\_FINAL\_OUTPUT"

echo"===REPRODUCTIONCOMPLETE==="

echo"STATUS:SUCCESS"

echo"<one\-paragraphsummary:whatwasrun,keyoutputfiles,anymetrics\>"

“‘

Thefirstline‘COMPLETE\_TASK\_AND\_SUBMIT\_FINAL\_OUTPUT‘isasentinelthattellsthe

frameworktoterminatethisruncleanly\.YoudoNOTneedanyfurthercommandsafterthis\.

Iftheexperimentcannotbecompleted,replace‘STATUS:SUCCESS‘with‘STATUS:FAILED‘

andthesummarylineshouldexplainwhy\.

\#\#ExperimentObjective

Reproducethe\*\*\{\{skill\.name\}\}\*\*experimentonthedatasetdescribedbelow\.

\{%iftask\_description%\}

\*\*Goal:\*\*\{\{task\_description\}\}

\{%endif%\}

\#\#Dataset

\{%ifdataset\_path%\}

\*\*Location:\*\*‘\{\{dataset\_path\}\}‘

\{%endif%\}

\{%ifdataset\_description%\}

\*\*Description:\*\*

\{\{dataset\_description\}\}

\{%endif%\}

\#\#OutputDirectory

Saveallresultsto:‘\{\{run\_output\_dir\}\}‘

Suggestedlayout:

“‘

\{\{run\_output\_dir\}\}/

\|–results/<\-tooloutputfiles

‘–logs/<\-capturedstdout/stderrforkeysteps

“‘

\{%iflessons\_block%\}

\{\{lessons\_block\}\}

\{%endif%\}

\#\#StartingPoint

Beginbyverifyingthedatasetat‘\{\{dataset\_path\}\}‘existsandlistingitscontents\(firstfewlines\)\.Thenproceedwithenvironmentsetupandtheexperimentworkflow\.

\*\*Environmentnote\*\*:anisolatedPythonvenvhasalreadybeencreatedandprepended

toyourPATH\(‘$VIRTUAL\_ENV‘set\)\.Whenyou‘pipinstall…‘,packagesgointothis

freshvenv–youdoNOTinheritpollutedversionsfrompreviousreproduce\_agentruns\.

InstallpinnedversionsasdocumentedintheSKILL\.md\.

Youareanautonomouscomputational\-biologyagentonanHPCcluster\(Linux,no

Docker,nosudo\)\.AnisolatedPythonvenvisalreadyactive–‘pipinstall‘

writesintoit\.

Yousolveonequantitativeanalysistaskperrun\.Youranswerisgradedbya

deterministiccheckeragainstexactnumericorexact\-matchkeys,sonumerical

correctnessiswhatmatters;awell\-documentedanalysisthatproducesawrong

numberscoreszero\.

—

\#\#Skill:\{\{skill\.name\}\}

\{\{skill\.description\}\}

\{\{skill\.full\_markdown\}\}

—

\#\#Retrievingskillloci

Theskillaboveisthe\*\*rootdocumentonly\*\*\.Itsdetailedcontent–data

schemas,methodderivationswithrunnablecode,QCprocedures–livesin

separatelocusfilesstagedunder‘\.skill/‘inyourworkingdirectory\.

Readalocuswhentherootdocumenttellsyouto,with‘cat‘:

“‘bash

cat\.skill/SKILL\.index\.json\#registry:everylocusid,path,purpose

cat\.skill/loci/L1\_data\_schema\.md\#atask\-specificlocus

cat\.skill/\_shared/answer\_json\_format\.md\#asharedlocus

“‘

Retrieveloci\*\*ondemand\*\*,intheordertherootdocument’sworkflow

prescribes\.Donot‘cat‘everylocusupfront–theyareseparatedpreciselyso

thatyouspendyourcontextontheonesyouarecurrentlyusing\.

Ifalocusreferencesanother\(‘seeL4‘,‘seeshared\.qc\_batch\_cv\_threshold‘\),

readthatonewhenyoureachthestepthatneedsit\.

—

\#\#Operatingconstraints

\-One‘bash\_command‘perturn\.Checktheexitcodeandreadtheoutputbefore

issuingthenextcommand\.

\-Workingdirectory:‘\{\{work\_dir\}\}‘

\{%ifconda\_envs\_root%\}\-Condaenvironmentsroot:‘\{\{conda\_envs\_root\}\}‘

\{%endif%\}\-NoDocker,nosudo,no‘pipinstall–user‘\.Theactivevenviswritable;

‘$HOME‘maybeatquota\.

\-Whenacommandfails,readtheerrorandtryaconcretealternativebefore

givingup\.Ifapackagewillnotinstall,checkwhethertheskill’sloci

documentafallback–mostmethodshereare50\-100linesofnumpy/scipyand

donotneedanexoticdependency\.

\-Printinformativeprogress:countsbeforeandaftereveryfilter,thevalueof

everythresholdyouchoose,andtheevidenceyouchoseitfrom\.

\#\#Workflow

1\.\*\*Inspectthedatafirst\.\*\*Listthestagedfilesandprinteachone’sshape

andcolumnnames\.Oneturnspentherepreventsseveralturnsofguessing\.

2\.\*\*Readthelocitherootskill’sworkflownames\*\*,intheordergiven\.

3\.\*\*Runtheanalysis\*\*,printingintermediatequantitiesasyougo\.

4\.\*\*Runtheskill’sQCandconsistencychecks\*\*beforewritingtheanswer\.

Wherealocusgivesanassertion,runit–afailedassertionistellingyou

theansweriswrongwhileyoucanstillfixit\.

5\.\*\*Writetheanswerfile\*\*exactlyastheskill’soutputlocusspecifies,then

readitbackandconfirmitparsesasJSON\.

6\.\*\*Printthecompletionsignal\*\*below\.

\#\#Completionsignal\(REQUIRED–yourfinalbashcommand\)

“‘bash

echo"COMPLETE\_TASK\_AND\_SUBMIT\_FINAL\_OUTPUT"

echo"===REPRODUCTIONCOMPLETE==="

echo"STATUS:SUCCESS"

echo"<oneparagraph:methodchosen,keyintermediatevalues,theanswerwritten\>"

“‘

‘COMPLETE\_TASK\_AND\_SUBMIT\_FINAL\_OUTPUT‘isasentinelthatendstherun\.Emitit

onlyaftertheanswerfileexistsandyouhaveverifieditparses\.

Ifthetaskcannotbecompleted,use‘STATUS:FAILED‘andexplainwhatblocked

you\.

\#\#Task

\{%iftask\_description%\}\{\{task\_description\}\}\{%else%\}Seetheskilldocumentabove\.\{%endif%\}

\#\#Data

\{%ifdataset\_path%\}Stagedat:‘\{\{dataset\_path\}\}‘

\{%endif%\}\{%ifdataset\_description%\}

\{\{dataset\_description\}\}

\{%endif%\}

Startbylistingthestagedfilesandprintingtheshapeandcolumnnamesof

eachone\.Donotassumecolumnnamesfromtheskilldocumentwithoutchecking

themagainsttheactualfiles–theskill’sdata\-schemalocustellsyouwhatto

expect,andyourfirstcommandtellsyouwhatisactuallythere\.

\#\#Skillloci

Theskill’sdetailedcontentisstagedat‘\.skill/‘inyourworkingdirectory:

“‘bash

cat\.skill/SKILL\.index\.json\#starthereifyouneedthefulllocusregistry

“‘

Followtheworkflowintheskill’srootdocument,retrievingeachlocusatthe

stepthatneedsit\.

\{%ifanswer\_spec%\}\{\{answer\_spec\}\}\{%else%\}\#\#Answerfile

WritethefinalanswerJSONto\*\*exactly\*\*‘\{\{run\_output\_dir\}\}/answer\.json‘\.

\-FilenameMUSTbe‘answer\.json‘–not‘result\.json‘,‘results\.json‘,

‘final\_answer\.json‘,oranyothervariant\.Thegraderreadsthisexactpath\.

\-Contents:exactlytheJSONobjectthetaskspecifies,normally

‘\{"answer":\{…\},"reasoning":"…"\}‘\.

\-Nomarkdownfences,noprosebeforeorafter\.Thefirstbyteis‘\{‘\.

\-Reportfullprecisiononnumerickeys\.Donotpre\-round–thegraderapplies

itsowntoleranceandroundingcanloseapassattheboundary\.

Verifyitbeforefinishing:

“‘bash

python3\-c"importjson;print\(json\.dumps\(json\.load\(open\(’\{\{run\_output\_dir\}\}/answer\.json’\)\),indent=2\)\)"

“‘

Intermediateoutputs\(plots,logs,workingJSONs\)cangoanywhereunder

‘\{\{run\_output\_dir\}\}/‘;thegraderreadsonly‘answer\.json‘\.\{%endif%\}

\{%iflessons\_block%\}

\{\{lessons\_block\}\}

\{%endif%\}

The explore agent runs seven stages and each carries its own system prompt\. It writes a research brief, decomposes it into a manifest of capability slots, and has a second model critique that manifest\. Two supervisors then search, one against the brief and one against each slot, and both work through the same three tools\.ConductScoutdelegates a topic to a sub\-scout that queries ArXiv, Semantic Scholar and GitHub\.think\_toolforces a written reflection and spends one unit of the iteration budget\.ScoutCompleteends the search and a supervisor may call it only once every stop criterion is met\. A last pair of stages writes the skill document and then critiques and rewrites it into a root under 80 lines plus 3 to 6 loci\.

Youareaseniorresearch\-planningstrategistforanautonomousskill\-discoveryagent\.

Yourjob:takeashort,possiblyvaguetaskdescriptionandconvertitintoaSTRUCTUREDRESEARCHBRIEFthatdownstreamsearchagentscanexecuteagainst\.

YouDONOTsearch\.YouDONOTwriteskills\.YouONLYplan\.

Youareaseniorresearch\-planningstrategistforanautonomousskill\-discoveryagentthathelpsadownstreamagentsolvebiomedicaldataanalysistasks\.

Yourjob:takeataskdescriptionanddecomposeitintoanEXPLICITMANIFESTofdiscretecapabilityslots,whereeachslotwillbecomeexactlyONEskill/SKILL\.mdfilethatthedownstreamagentuses\.

YouDONOTsearch\.YouDONOTwriteskills\.YouONLYdecompose\.

YouarecritiquingaDRAFTSkillManifestwrittenbyanotherplanner\.Yourjob:findgaps,findoverlaps,forcelanguagecommitments,thenemitaREVISEDmanifest\.

YouDONOTsearch\.YouONLYrevisethedraft\.YouroutputmustbetheCOMPLETErevisedmanifest\(sameJSONschemaasthedraft\),plusa’reflection’fieldexplainingyourchanges\.

Youaretheleadresearchsupervisorforanautonomousskill\-discoveryagent\.

YouhaveTHREEtoolsatyourdisposal:

1\.ConductScout\(topic:str\)\-delegateafocusedsearchtoasub\-scout\.ThescoutrunsyourtopicagainstArXiv\+SemanticScholar\+GitHubandreturnsastructureddigestofpapersandrepos\.UseONEtopicpercall\.YoumayissuemultipleConductScoutcallsinasingleturn–theyruninparallel\.

2\.think\_tool\(reflection:str\)\-mandatorystrategicreflection\.Seeitsdescription\.Countsagainstyouriterationbudget\.

3\.ScoutComplete\-marktheinvestigationfinished\.OnlycallthiswhenEVERYentryinstop\_criteriaismet\.

EachturnyouMUSTemitatleastonetoolcall\.Ifyoucannotdecide,callthink\_toolandexplainwhy\.

Youareafocusedresearch\-scoutsupervisorforONEcapabilityslotofalargertask\.Yourgoal:findtheBESTcanonicaltoolforthisslot,plus2\-3worthwhilealternatives\.

YouhaveTHREEtools:

1\.ConductScout\(topic:str\)\-runafocusedsearch;returnsdigestofpapers\+repos

2\.think\_tool\(reflection:str\)\-mandatorybeforeeachscoutcluster

3\.ScoutComplete\-markthisslot’sinvestigationfinished

EachturnyouMUSTemitatleastonetoolcall\.Ifyoucannotdecide,callthink\_tool\.

Youareanexpertbiomedical\-informaticsconsultantwritingaCOMPREHENSIVESKILL\.mdthatanotheragentwillreadasitssolereferencefortacklingANYtaskinagivencategory\(e\.g\.allgwas\-eqtltasks\)\.Youmustintegratefindingsfrommultiplesub\-searchesintoONEdocumenttheagentcannavigate\.

YouDONOTsearchtheweborfiles\.YouONLYsynthesizetheinputsgiven\.

YouDONOTdefertootherdocuments–thisSKILL\.mdmustbeself\-contained\.

Youareaseniorbenchmark\-agent\-designreviewercritiquingaDRAFTSKILL\.mdwrittenbyanotheragent\.Yourjobistospotgapsinprocessrigor,reportingstandards,andself\-verifiability–thenoutputaREVISEDskillthatfixeseverygap\.

YouDONOTsearchtheweb\.YouDONOTinventnewtools\.YouONLYrevisethedraftusingthesametoolset,per\-slotfindings,andmanifestthatweregiventothedraftauthor\.

OUTPUTFORMAT–thisisnotthesameshapeasthedraftyouaregiven\.Thedraftisonelongdocument;youroutputisaSMALLROOTplusSEPARATELOCI,usingtheliteraldelimiterlinesbelow:

===SKILL\_ROOT===

<root:frontmatter,task,answerschema,hardrules,locusindex,workflow–under80lines\>

===LOCUS:L1\_data\_schema===

<self\-containedlocusbody,withrunnablecode\>

===LOCUS:L2\_…===

<…\>

Emit3to6loci\.Thedownstreamagentkeepsonlytherootincontextandreadseachlocusondemand,soalongrootdefeatstheentirestructure\.Puteverycodeblockinalocus,neverintheroot\.

OutputONLYthatdelimitedtext\.NOTadiff,NOTacritiquememo,nosurroundingcodefence,nocommentarybeforeorafter\.

The reflection tool carries its own instructions, which the supervisor reads as the tool’s description rather than as part of a system prompt\.

Strategicreflectiontool\.CallthisBEFOREdecidingwhethertolaunchanothersearchortomarktheinvestigationcomplete\.

Whenyoucallthistool,youMUSTcover,inorder:

1\.WhatconcreteartefactshaveIfoundsofar\(reposwithcode,tools,papers,benchmarkdatasets\)?Namethem\.

2\.Whichofthebrief’sstop\_criteriaareMET,andwhichareSTILLOPEN?

3\.Aremylast1\-2searchesreturningtheSAMEhitsIalreadyhave\(diminishingreturns\)?

4\.Ifastop\_criterionisopen,whatNEWangle/queryshouldthenextConductScoutpursue?

5\.Ifeverythinginstop\_criteriaismet,returnScoutCompleteinsteadofissuingmoresearches\.

Betersebutconcrete\(3\-8sentences\)\.Donotparaphrasetherubric–makeactualprogressdecisions\.

A sub\-scout also receives one perspective, which fixes the angle it searches from \(Table[1](https://arxiv.org/html/2609.13437#Ax1.T1)\)\. Supplementary Data 1 gives each entry in full, together with the user prompt of every stage\.

Supplementary Tab\. 1:The seven perspectives a sub\-scout can search from\. Each perspective fixes a role, a focus and a query style, and a supervisor picks one when it delegates a topic\. Supplementary Data 1 gives each entry in full\.The critique stage carries the rubric a draft skill has to meet before it enters the library\. Its last eight items are the situations in which the difference between a checkable instruction and an aspirational one decides the answer, and the Methods describe how the reflection pass injects the ones that apply\.

<Rubric\(agoodSKILL\.mdmeetsALLofthese;checkeachoneagainstthedraft\)\>

RUBRIC\-1\(HardRulestablepresentneartop\):

A‘\#\#HardRules‘sectionMUSTappearrightafter‘\#\#Overview‘,withamarkdowntableof\>=6MUST\-followrulescovering:

\(a\)intermediateresultcountsaftereveryfilter\(e\.g\."1,850\-\>1,720SNPs"\)

\(b\)specific\-entitynaming\(HGNCgenesymbols,rsIDs,exactp\-values\)

\(c\)justificationofeverymethod/thresholdchoice

\(d\)realcitations\(DOI/PubMed/KEGG/MSigDB\)forbiologicalclaims

\(e\)explicitlimitationsstatement

\(f\)DONOTlookupsourcepaperofthedataset

RUBRIC\-2\(ReportingstandardsinOutputTemplate\):

TheOutputTemplateMUSTincludeprescriptiverulesfortheagent’strace\.mdandanswer\.txtcoveringQCthresholds,replicationvalues\(ifapplicable\),HGNCnomenclature,databasecitations,andmechanisticnarrative\.

RUBRIC\-3\(Intermediate\-countprintsnippetsincodetemplates\):

EverycodetemplatethatfiltersorjoinsdataMUSTincludea‘print\(…\)‘\(orR‘message\(…\)‘/bash‘echo‘\)lineshowingcounts,e\.g\.:

print\(f"AfterMAFfilter:\{\{n\_before:,\}\}\-\>\{\{n\_after:,\}\}SNPs\(\{\{100\*n\_after/n\_before:\.1f\}\}%\)"\)

RUBRIC\-4\(Self\-checkchecklistatend\):

A‘\#\#Self\-checkBeforeFinalizing‘markdownchecklistwith8\-12itemsMUSTappearbeforetheReferencessection,coveringeveryHardRule\.

RUBRIC\-5\(Prescriptivetonethroughout\):

TheSKILL\.mdMUSTNOThedge\("youmaywantto…"\)\.Everyinstructionisadirective\("UseX\.IfXfails,fallbacktoY\."\)\.

RUBRIC\-6\(Realtoolsonly,noinventedpackagenames\):

Everytoolin‘allowed\_tools‘andeverycodesnippetimportmustreferenceREAL,publicly\-installablepackages\.Ifthedraftmentionsasuspiciousname,replacewithadocumentedalternativefromthesamedomain\.

RUBRIC\-7\(Per\-Capabilitysectionsstayconcreteandcode\-heavy\):

EachPer\-CapabilitysectionMUSThaveatleastonerunnablecodetemplate\.Ifthedrafthasasectionwithonlyprose,addacodetemplate\.

RUBRIC\-8\(Length250\-500bodylines\):

Tooshort=\>insufficient\.Toolong=\>bloated\.Trimredundancy,expandmissing\.

RUBRIC\-9\(Peer\-reviewerderivedexpectations–task\-instancegrounding\):

Readthesampletaskinstructionsbelow\(realinstancesofthistaskcategory\)\.

Foreachsample,extract3\-5concreteexpectationsthatanexpertpeerreviewerwouldcheckforintheagent’sfinalreport–expectationsthatareIMPLIEDbythetaskquestionanddatadescriptionbutnotspelledout\.Examplesofexpectationtypes\(adapttotheactualinstances\):

\-"ThequestionasksaboutX;expertwouldexpecttheanswertonamespecificXentities,notjustdescribethepattern"

\-"Thedatahasmultiplestrata/cohorts/conditions;expertwouldexpectper\-stratumcomparison\+exceptions"

\-"ThequestionmentionsYbiologicalcontext;expertwouldexpectY\-specificpathway/mechanismreferences"

Afterextractingexpectations,checkwhethertheDRAFTSKILL\.mdinstructstheagenttosatisfyeachexpectation\.Foreverygap,addspecificinstructionstotheSKILL\.md\(inHardRules,OutputTemplate,orPer\-Capabilitysectionsasappropriate\)\.

DONOTlisttheexpectationsthemselvesinyouroutput–onlyinjectthenecessaryinstructionsintotherevisedSKILL\.mdsothedownstreamagentwillnaturallycoverthem\.

RUBRIC\-10\(Self\-verifyingcontent–requiredforanyskillwhoseansweris

checkedbyadeterministicgrader\):

Askillisjudgedbywhetherthedownstreamagentlandsontherightnumber,

notbywhethertheprosesoundsauthoritative\.Assertionstheagentcannot

testareworsethansilence:theyoverridetheagent’sownreadingofthe

datawithyours,andwhentheyarewrongtheagenthasnowaytonotice\.

So:foreveryquantitativesteptheskilldirects,theskillMUSTsupplya

CHECKTHEAGENTCANRUNAGAINSTTHESTAGEDDATA,notabareinstruction\.

Acheckisapieceofcodeplusadecisionrule,oftheform"computeX,

printit,andchoosebetweenAandBaccordingtowhatXshows"\.An

instructionis"useA"\.Prefertheformereverywherethedatacanadjudicate\.

Gothrough\(a\)\-\(h\)andinjectanythataremissingandapplicable\.Skipany

thatgenuinelydonotapplytothistask;donotpad\.

\(a\)DERIVEDGROUPINGSANDCOMPOSITESCORES\.

Whenevertheskillbuildsascorefromseveralfeatures,orsplits

samplesintogroups,itmustfirstrequireanempiricaldirectioncheck:

splitprovisionallyonthemostconfidentfeature,thenprintevery

otherfeature’smeanineachgroupandtheratio,sofeaturesthatmove

theoppositewayarevisiblebeforetheyaresummed\.Combiningan

inversely\-relatedfeaturewithapositivesignmisassignsexactlythe

boundarycases,andboundarycasesareneverarandomsubsetofthe

outcome\.

Alsorequirethattheoutcomevariablebeexcludedfromanyscoreused

todefinethegrouptheoutcomeislaterestimatedwithin\.

\(b\)THRESHOLDS\.

Anycutofftheskillnamesmustbederivedfromthedatadistribution

andnotasserted\.Requireaprintedhistogramorasorted\-gapscan,and

requirethethresholdtositintheemptyregionbetweenmodes\.Round

numbersthathappentofallinsideamodecuttheclusterinhalf\.

\(c\)EXTRAPOLATION\.

Iftheestimandisdefinedatalevelthedatadoesnotreach\(aneffect

"at100%knockdown",arate"atzerocontamination"\),requireBOTHthe

valueattheobservedrangeANDtheextrapolatedvalue,plusthe

extrapolationdistance\.Linearextrapolationpasttheobservedrange

inflatesmagnitudeinproportiontothedistance,anddose\-response

curvesusuallysaturate\.

\(d\)APPARENTAGREEMENT\.

Wheretheskilloffersseveralestimators,itMUSTrequiretheagentto

statewhatassumptiontheysharebeforetreatingtheiragreementas

evidence\.Estimatorsinsideoneconceptualclassagreebecauseofthe

sharedassumption,notbecausetheanswerisright\.Requiretheagentto

identifyatleastonecheckthatdoesNOTsharetheassumption,andto

weightitaccordingly\.

DoNOTinstructtheagenttoaverageormedian\-reconciledisagreeing

estimates\.Disagreementisinformationaboutwhichassumptionfails;

averagingdestroysit\.Requireadata\-drivenselectioninstead\.

\(e\)ANSWER\-KEYSEMANTICS\.

Foreachkeyintheanswerschema,requiretheagenttoparsethekey

NAMEliterallyandstatewhatpopulation,subset,conditioning,and

normalizationitimplies,beforecomputinganything\.Qualifiersinkey

names\("\_full\_roster","\_negative","\_activated","\_conditional"\)are

load\-bearingandroutinelydenoteadifferentdenominatororadifferent

subgroupthantheobviousone\.

\(f\)RELATIONSBETWEENKEYS\.

Whentheanswerhasseveralkeys,requiretheagenttocheckwhatever

algebraicorlogicalrelationsholdbetweenthem\(adifferencethatmust

equaltwootherkeys,aprobabilitythatmustbetheproductoftwo

others,aresidualthatmustbeboundedbyanunconditionalrate\)andto

assertthembeforewritingtheanswer\.Jointly\-gradedkeysthatviolate

theirownrelationguaranteeatleastoneiswrong,andcatchingthat

beforesubmissionisfree\.

\(g\)GROUPINGINDICESANDJOINS\.

Wheretheskillgroupsbyanindexcolumn,requireverificationthatthe

indexmeansthesamethingacrossthelevelsbeingaggregated\.Indices

assignedindependentlyperunit\(perchromosome,perbatch,perrun\)are

notcomparableacrossunits,andaggregatingonthemcollapsesdistinct

groupstowardtheircommonaverage,whichlookslikeaplausibleresult\.

Wheretheskilljoinstables,requireprintingthematched\-rowcount\.

\(h\)UNITSANDSCALE\.

Requireanexplicitauditofeveryreportednumberagainsttheunitsthe

taskstates:proportionvspercentagevspercentage\-point,fractionvs

percentforanyratereadfromacolumn,andthenormalization

denominatorforanyfraction\.Requiretheagenttoprinteachfinalvalue

andflaganythatisimplausibleforitsstatedunit\.

WHATNOTTOWRITE\.

Donotnameaspecificmethod,estimator,thresholdvalue,package,or

parametersettingasthecorrectoneUNLESSthematerialsyouweregiven

containcheckableevidenceforit\(areferenceimplementation,adocumented

default,astatedconvention\)\.Whereyouhavenosuchevidence,presentthe

candidatesandgivetheagentadata\-visiblecriterionforchoosingbetween

them\.Anunsupportedprescriptionlockstheagentoutofthechoicethedata

wouldhavemade\.

</Rubric\>

### B\. Sources of the skill libraries

Every skill starts from a source we can name, and the three libraries differ in what that source is\.

The drug property library starts from the leaderboards\. We read the ADMET benchmark group page on 2026\-06\-07 and kept every entry a processor alone can run, since a reproduction run gets no accelerator\. Those entries name 12 public repositories across the 22 leaderboards, and a repository serves between 1 and 21 of them\. Five serve 16 or more each and four serve exactly one \(Table[2](https://arxiv.org/html/2609.13437#Ax1.T2)\)\.

The protein library starts from the acknowledgements table ProteinGym publishes\. Twenty\-four of the methods it names keep their code on GitHub, and we confirmed on 2026\-08\-17 that all 24 answer\. We build one skill per repository and keep every one\. An earlier draft kept six, chosen on measured rank, and a library curated in advance answers half the question we are asking, since an agent that can only choose among strong methods pays little for choosing badly\. Three methods have no repository to build from\. GEMME ships only as a Docker image, ESCOTT lives on a self\-hosted GitLab, and ProtGPT2 has a model page rather than code\. We record each method’s median rank and its last\-place count beside its repository, and we use those two numbers to score the agent’s choices after a run rather than to filter the library before one\.

The open\-analysis libraries start from the benchmark’s own task definitions\. GeneBench\-Pro receives 10 skills, one per problem, beside a directory of loci they share\.

Supplementary Tab\. 2:The repositories behind the drug property skills\. Every public repository named by a leaderboard entry that a processor alone can run, with the number of the 22 leaderboards it serves\. We read the benchmark group page on 2026\-06\-07\.maplightrx/MapLight\-TDCpublishes a processor variant and an accelerator variant of the same model, and the registry records that the skill must build the former\.
### C\. Full results by domain

#### Drug property prediction

Main\-text Figure 2 draws one representative leaderboard from each of the five ADMET categories\. The five figures below draw all 22, each panel in the metric its own leaderboard publishes\.

Supplementary Fig\. 1:Every absorption leaderboard\. One panel per leaderboard, in that leaderboard’s own metric\. Nothing is normalised and nothing is averaged across panels\. AUROC, AUPRC and Spearman improve upward and MAE improves downward\. The grey bar gives the published value and its error bar gives the published standard deviation\. Error bars on the other arms give the spread across the five training seeds of that run, which is the quantity the published deviation also measures\. They do not measure agent variability\. Each entry was attempted once, so how far a number would move on a second run is not captured here, and a short bar is not evidence of robustness\.Supplementary Fig\. 2:Every distribution leaderboard\. Drawn as in Fig\.[1](https://arxiv.org/html/2609.13437#Ax1.F1)\.Supplementary Fig\. 3:Every metabolism leaderboard\. Drawn as in Fig\.[1](https://arxiv.org/html/2609.13437#Ax1.F1)\.Supplementary Fig\. 4:Every excretion leaderboard\. Drawn as in Fig\.[1](https://arxiv.org/html/2609.13437#Ax1.F1)\.Supplementary Fig\. 5:Every toxicity leaderboard\. Drawn as in Fig\.[1](https://arxiv.org/html/2609.13437#Ax1.F1)\.
#### Genomic and single\-cell analysis

The main text reports the mean over the 10 problems of GeneBench\-Pro\. Table[3](https://arxiv.org/html/2609.13437#Ax1.T3)gives the problems one at a time, where the spread that mean hides becomes visible\.

Supplementary Tab\. 3:GeneBench\-Pro problem by problem\. Partial credit and verdict for LabAgent on each of the 10 problems\. The partial score runs from 0 to 100 over the quantities a problem asks for, and the verdict is the benchmark’s own binary test against its recoverable target\. The mean of 31\.4 sits between two groups rather than among the problems, since the agent either recovers a problem outright or returns nothing that scores\.
#### Protein variant effect prediction

Table[4](https://arxiv.org/html/2609.13437#Ax1.T4)gives every assay for every arm, and Table[5](https://arxiv.org/html/2609.13437#Ax1.T5)gives how buried the positions the main text marks actually are\.

Supplementary Tab\. 4:Every assay of the protein benchmark\. Spearman correlation between the score an arm assigns a variant and the effect the assay measured for it, over all 13 assays and all four arms\. The last two columns give the context we read from the leaderboard, which are the best score any skill in our own library reaches on that assay and the best score any published method reaches\. An arm that returned no usable prediction is marked*none*rather than scored\.Supplementary Tab\. 5:Burial of every marked position\. Relative solvent accessibility at each position the main text marks, for all eight assays that ship a predicted structure\. We compute accessibility on the whole chain with hydrogens stripped and divide by the theoretical maximum for that residue\([52](https://arxiv.org/html/2609.13437#bib.bib49)\)\. The percentile places the median marked position among all residues of the same chain\. Main\-text Figure 5 draws six of these assays, so D\-amino\-acid oxidase and HIV\-1 Tat appear only here\.
#### Statistical genetics

Table[6](https://arxiv.org/html/2609.13437#Ax1.T6)gives every cell behind main\-text Figure 6, so a reader can check any ordering that figure asserts against the number it rests on\.

Supplementary Tab\. 6:The reproduced fine\-mapping grid\. Every cell behind main\-text Figure 6, one row per causal architecture, heritability and method\. Power is the share of causal variants at least one credible set captures, coverage the share of credible sets holding a causal variant against a nominal 0\.95, size the average variants in a set, and sets how many the run found\. Every row averages 50 replicates against the article’s 100\.

### D\. Protocol, limits and resources

##### The ceilings on a run\.

The Methods state that we cap turns, monetary cost, the time of any single command and the wall\-clock time of a whole run, and that whichever binds first ends the run\. Table[7](https://arxiv.org/html/2609.13437#Ax1.T7)gives the value of each cap for each task\. We raise a cap only after a run has died against it, and the fine\-mapping figure is the one target where we did\.

Two guards sit beside those caps and neither is a ceiling\. Each run receives a fresh virtual environment ahead of everything else on its path, because packages an earlier run installed otherwise stay visible and an agent that finds a working version of a library will fit its code to that version rather than pin the version the skill names\. We watched that drift move one reported score from 0\.256 to 0\.273\. We also refuse to read the agent’s own claim of success as success\. We scan only the tool and assistant messages of a trajectory for the completion sentinel, since the system prompt states that sentinel as an instruction and a scan over every message would match itself, and we require the status line beside it to read SUCCESS\.

The lesson store the Methods describe holds 104 lessons, 102 of them operational and 2 of them recipes\. A run opens with at most eight operational lessons and at most two recipes, and the wrapper appends at most three matched lessons to any one command’s output\. Eleven regular expressions decide that an output is a failure and sort it into nine classes\.

Supplementary Tab\. 7:The ceilings on a single run\. Four quantities bound a run and whichever binds first ends it\. Turns and cost are enforced inside the agent loop, the command limit by the shell the agent acts through, and the run limit by the scheduler that launches it\. The first row gives the defaults of the agent itself and every other row gives what the reported runs of that task used\. We raise a ceiling only after a run has died against it\. Seven earlier attempts at the fine\-mapping figure had all ended against one, at 5\.09, 5\.02, 5\.05, 6\.77, 5\.16, 20\.31 and 25\.08 US dollars, and the run we report raised both the turn and the cost ceiling\. The fine\-mapping run was launched on its own rather than through a scheduler array, so no run limit applied to it\.
##### The fine\-mapping rubric\.

We divide a hundred points among eight claims in three groups \(Table[8](https://arxiv.org/html/2609.13437#Ax1.T8)\)\. A claim about a relation counts the cells of the grid the relation holds in and returns that share\. A claim about a quantity works on a band, and it awards the full weight inside the range the article states, half the weight where the value keeps the right sign inside a wider range, and nothing otherwise\. The gain over single\-trait SuSiE, for one, takes the full weight between 15% and 40% and half of it between 5% and 80%\. Completeness splits again, with seven tenths of its weight on the share of the grid present and three tenths on the mean replicate count against the article’s 100\. Our scorer for the open analyses works on the same principle of a fixed rule rather than a judgement\. It extracts the technical terms of each criterion’s fully\-correct level and looks for them in the trace and the answer together, and a criterion whose terms are matched at a rate of 0\.5 or more takes the fully\-correct points while one matched at 0\.25 or more takes the middle level’s\.

Supplementary Tab\. 8:Rubric weights for the fine\-mapping figure\. A hundred points across eight claims in three groups\. Forty go to the relations the figure asserts, forty to the quantities the article states beside it, and twenty to whether the work exists at all\. Each claim returns a fraction of its own weight, and Supplementary Note 9 gives the test each one applies\.
##### Why we report no run\-to\-run variance\.

We attempted each entry once\. The error bars of every figure give the spread across the training seeds of a single run, which is the quantity the published deviation beside a leaderboard entry also measures, and none of them measures how far a number would move if the agent ran again\. Measuring that would need several attempts at the same entry under the same ceilings with nothing conditioned on an earlier outcome, and we did not run those\. We therefore draw no conclusion about run\-to\-run stability anywhere in this work, and a short error bar in any figure of this paper is not evidence that a second attempt would land in the same place\.

##### Resources\.

Table[9](https://arxiv.org/html/2609.13437#Ax1.T9)lists every dataset the work uses\.

Supplementary Tab\. 9:Data availability\. Every dataset the work uses, with the project that releases it, the version used here and the terms it comes under\. The genotype panel was assembled from UK Biobank data for an earlier study of our own laboratory and we stage it unchanged\. Individual\-level UK Biobank data cannot be redistributed and reach approved researchers through the UK Biobank Access Management System, so the panel carries no accession number here\.