BixBench3: Frontier AI Agents Can Now Reproduce ~48% of Real Computational Biology Research Workflows

# 1Introduction
Source: [https://arxiv.org/html/2608.25286](https://arxiv.org/html/2608.25286)
BixBench3: Benchmarking AI agents on research\-study\-scale computational biology tasksZane Koch1, Asmamaw T\. Wassie1, Javier Valdes\-Aleman1, Jason Lee1, Michaela M\. Hinks1, Samuel G\. Rodriques1, Andrew D\. White1, Jon M\. Laurent1,\*AbstractArtificial intelligence \(AI\) promises to accelerate biological research by automating computational analyses\. Yet the ability of AI agents to carry out computational biology at the scale of complete research studies has not been systematically evaluated\. Here we introduceBixBench3, a benchmark that measures the capacity of AI agents to process raw biological data through to scientific results\. We designedBixBench3tasks to mirror the delegation of work from a scientist to an agent: the scientist chooses the research question and high\-level methods, then delegates implementation of all analyses to the agent\. In each task, an agent receives a research objective, methodological guidance, and raw data derived from a published scientific study, and must execute a sequence of analyses to achieve the research objective\. The data artifacts resulting from these analyses – such as peak call matrices or differential expression tables – are programmatically graded against the corresponding artifacts generated and reported in the original study\. Across 20BixBench3tasks encompassing the generation of 138 unique artifacts, we find that 13 frontier large language models \(LLMs\) achieve scores ranging from 0\.00 for Gemini 3\.1 Flash Lite to 0\.48 for GPT 5\.6 Sol\. Agents perform worse on tasks with larger raw datasets \(0\.36 on tasks with<100<100GB versus 0\.10 on tasks with\>100\>100GB\) and on analyses requiring more sequential steps \(0\.36 at 1–2 steps vs 0\.24 at 3\+\)\. On average, agents use 6\.8 hours, 102 million tokens, and $43 to complete each task, with the longest attempts consuming 24 hours, 1\.07 billion tokens, and $525\. Notably, the highest\-scoring agents used fewer tokens and were cheaper than less performant options\. These results reveal that LLMs vary substantially in their ability to \(1\) execute multiple sequential analysis steps coherently, \(2\) manage large quantities of raw data, and \(3\) work across scientific domains\.1Edison Scientific, Inc\., San Francisco, CA, USA \*Correspondence to Jon M\. Laurent atjon@edisonscientific\.com Preprint\. August 27, 2026\. ![[Uncaptioned image]](https://arxiv.org/html/2608.25286v1/edison.png)

## 1Introduction

LLM\-based agents are increasingly being deployed to perform scientific research\. Recent agents combine large language models with code execution, retrieval, and domain\-specific tools to carry out parts of research workflows, rather than isolated question answering\. In biology, Robin drove an iterative lab\-in\-the\-loop campaign to propose a therapeutic candidate for dry age\-related macular degeneration\([Ghareeb et al\., 2026](https://arxiv.org/html/2608.25286#bib.bib6)\)\. Kosmos produced seven discoveries spanning metabolomics, materials science, neuroscience, and statistical genetics, three of which independently reproduced unpublished or preprinted findings\([Mitchener et al\., 2025b](https://arxiv.org/html/2608.25286#bib.bib7)\)\. Scientific assistants such as Biomni and Claude Science bundle biomedical tools and databases to enable AI\-assisted research\([Huang et al\., 2026](https://arxiv.org/html/2608.25286#bib.bib2);[Anthropic, 2026](https://arxiv.org/html/2608.25286#bib.bib8)\), and a broader set of agents targets single\-cell workflows, bioinformatics pipeline design, and automated machine\-learning experimentation\([Xin et al\., 2024](https://arxiv.org/html/2608.25286#bib.bib1);[Mehandru et al\., 2025](https://arxiv.org/html/2608.25286#bib.bib3);[Martinek et al\., 2025](https://arxiv.org/html/2608.25286#bib.bib4);[Jin et al\., 2025](https://arxiv.org/html/2608.25286#bib.bib5)\)\. At the same time, recent stress tests and biomedical\-agent reviews warn that autonomous scientific agents remain brittle, with failures arising from hallucinated claims, weak uncertainty handling, and breakdowns between plans and executable analyses\([Agrawal et al\., 2026](https://arxiv.org/html/2608.25286#bib.bib11);[Gao et al\., 2024](https://arxiv.org/html/2608.25286#bib.bib12);[Zhou et al\., 2025](https://arxiv.org/html/2608.25286#bib.bib13)\)\. Despite this progress, whether AI agents can execute entire computational studies starting from raw data has not been measured\.

To address this uncertainty, we introduceBixBench3\. InBixBench3, an agent receives a high\-level research objective, methodological guidance derived from a published biology paper, and the raw data associated with that study\. The agent is tasked with achieving the research objective by analyzing the provided data\.BixBench3thus evaluates the capability of agents to construct and execute long analysis pipelines following specific instructions\. Performance is graded by comparing agent\-produced artifacts against artifacts from the original publication\. The benchmark is constructed from 20 studies spanning 17 assay types and 9 scientific domains, with 138 total graded artifacts\. In contrast to benchmarks that evaluate whether an agent can answer a question about biology or perform a single bioinformatic analysis\([Laurent et al\., 2024](https://arxiv.org/html/2608.25286#bib.bib19);[Laurent et al\., 2026](https://arxiv.org/html/2608.25286#bib.bib20);[Liu et al\., 2025](https://arxiv.org/html/2608.25286#bib.bib18);[Mitchener et al\., 2025a](https://arxiv.org/html/2608.25286#bib.bib14);[Nair et al\., 2026](https://arxiv.org/html/2608.25286#bib.bib9)\),BixBench3tests the long\-horizon capability necessary to conduct an entire computational study end\-to\-end\.

### 1\.1Related work

BixBench3sits at the intersection of two lines of previous work: long\-horizon benchmarks, which test whether AI agents can carry out extended, multi\-step tasks with tools; and biology and bioinformatics benchmarks, which test scientific and analytical competence\.BixBench3combines the long\-horizon challenge of the former with the domain\-specificity of the latter\.

Long\-horizon agent benchmarks evaluate tool use, planning, and execution over many steps\. As agentic capabilities have improved, the length of tasks that models can complete has increased from those taking humans minutes in 2024\([Zhou et al\., 2024](https://arxiv.org/html/2608.25286#bib.bib21);[Xie et al\., 2024](https://arxiv.org/html/2608.25286#bib.bib27)\)to those taking humans 16\+ hours in 2026\([METR, 2026](https://arxiv.org/html/2608.25286#bib.bib29)\)\. SWE\-bench and SWE\-bench Verified challenged agents to solve real\-world GitHub issues, with the highest\-performing model as of July 2026 \(gpt\-5\.6\-sol\) taking 3 minutes to complete each SWE\-bench Verified task on average\([Jimenez et al\., 2024](https://arxiv.org/html/2608.25286#bib.bib22);[Vals AI, 2026](https://arxiv.org/html/2608.25286#bib.bib23)\)\. On Terminal\-Bench\-2, which tests agents on terminal\-based software, data\-science, and systems tasks, the top model as of July 2026 \(Fable 5\) completed the longest tasks in approximately one hour\([Merrill et al\., 2026](https://arxiv.org/html/2608.25286#bib.bib28);[Terminal\-Bench, 2026](https://arxiv.org/html/2608.25286#bib.bib30)\)\. RE\-Bench compared human and agent ability on ML research tasks, across which agents consumed 29M input tokens on average\([Wijk et al\., 2025](https://arxiv.org/html/2608.25286#bib.bib24)\)\. PaperBench gave agents up to 36 hours to replicate machine\-learning papers, although performance mostly plateaued after the first hour\([Starace et al\., 2025](https://arxiv.org/html/2608.25286#bib.bib10)\)\. HCAST evaluated agents on 189 software, ML\-engineering, and cybersecurity tasks which each took humans up to eight hours\([Rein et al\., 2025](https://arxiv.org/html/2608.25286#bib.bib25)\)\. At the longest reported horizon, SWE\-Marathon evaluates agents on 20 project\-scale software\-engineering tasks with 2–10\-hour agent time limits and 40–400\-hour expert\-human estimates; agent attempts averaged 27\.2M tokens\([Desai et al\., 2026](https://arxiv.org/html/2608.25286#bib.bib31)\)\.

By contrast, existing biology and bioinformatics benchmarks are generally relatively short\-horizon, testing domain knowledge in a question\-and\-answer format, or agentic capability on isolated analyses\. LAB\-Bench and LABBench2 test domain knowledge and practical research skills through questions about literature retrieval, protocol troubleshooting, and sequence manipulation, while BioProBench focuses on biological protocol understanding\([Laurent et al\., 2024](https://arxiv.org/html/2608.25286#bib.bib19);[Laurent et al\., 2026](https://arxiv.org/html/2608.25286#bib.bib20);[Liu et al\., 2025](https://arxiv.org/html/2608.25286#bib.bib18)\)\. DISCOVERYWORLD required agents to make scientific discoveries in a virtual environment and took both humans and agents approximately 100–1,000 steps to complete each task\([Jansen et al\., 2024](https://arxiv.org/html/2608.25286#bib.bib26)\)\. BixBench, GenoTEX, and BioDSA\-1K require agents to perform short biological data\-analysis tasks\([Mitchener et al\., 2025a](https://arxiv.org/html/2608.25286#bib.bib14);[Liu et al\., 2024a](https://arxiv.org/html/2608.25286#bib.bib16);[Wang et al\., 2025](https://arxiv.org/html/2608.25286#bib.bib15)\)\. On GenoTEX, OpenAI o1 averaged 112,000 input tokens, 12,000 output tokens, and 192 seconds per task\([Liu et al\., 2024a](https://arxiv.org/html/2608.25286#bib.bib16)\)\. Growing in complexity, CompBioBench consists of 100 isolated computational\-biology problems, which models completed in 11–18 minutes on average\([Nair et al\., 2026](https://arxiv.org/html/2608.25286#bib.bib9)\)\. BAISBench asks agents to annotate cell types and answer discovery questions from single\-cell h5ad datasets, with evaluated systems consuming up to approximately 500,000 tokens\([Luo et al\., 2025](https://arxiv.org/html/2608.25286#bib.bib17)\)\. GeneBench\-Pro more directly targets multistage analysis: its 129 genomics and translational\-biomedicine problems give agents minimally guided, potentially errorful simulated datasets and require 3–13 dependent statistical decisions to recover a graded target estimate, with the authors estimating 10–40 hours of unaided expert work per problem\([Li and Ho, 2026](https://arxiv.org/html/2608.25286#bib.bib32)\)\. SpatialBench\-Long \(2–45 minutes per task\) and scBench\-Long \(4–65 minutes per task\) require agents to recover study\-level claims from raw or near\-raw spatial and single\-cell data, while VariantBench \(6\-hour cap\) contains 118 tasks spanning variant discovery, quality control, and interpretation\([Diks et al\., 2026a](https://arxiv.org/html/2608.25286#bib.bib57);[Diks et al\., 2026b](https://arxiv.org/html/2608.25286#bib.bib58);[Bhowmick et al\., n\.d\.](https://arxiv.org/html/2608.25286#bib.bib59);[LatchBio, 2026](https://arxiv.org/html/2608.25286#bib.bib60)\)\.

## 2Results

### 2\.1Benchmark description

BixBench3evaluates whether an AI agent can reconstruct the analysis of a published computational\-biology study from raw data, producing structured artifacts that are graded against the corresponding published results \(Figure[1](https://arxiv.org/html/2608.25286#S2.F1)A\)\. Across the 20 tasks, the highest\-scoring model reproduced 48% of the requested artifacts closely enough to preserve their principal biological meaning \(Figure[1](https://arxiv.org/html/2608.25286#S2.F1)B\)\.

![Refer to caption](https://arxiv.org/html/2608.25286v1/figures/figure1_layout_manuscript.png)Figure 1:BixBench3construction and topline performance\.\(A\) Each benchmark task is constructed by decomposing a published study into a directed acyclic graph \(DAG\) of analyses producing data artifacts\. Agent\-produced versions of those artifacts are programmatically scored against corresponding published artifacts\. \(B\) Artifact\-level pass outcomes for 13 models across 20BixBench3tasks, comprising 138 requested artifacts per model and 1,794 model–artifact evaluations\. Filled squares denote artifacts with programmatic scores of at least 0\.80, the expert\-calibrated threshold for preserving the main biological interpretation of the artifact \(Section[4\.4](https://arxiv.org/html/2608.25286#S4.SS4)\); empty squares denote artifacts below this threshold\. Bars on the right show each model’s overallBixBench3score: its mean task score across the 20 tasks, where each task score is the proportion of artifacts that pass\.BixBench3consists of 20 tasks drawn from published papers spanning a wide variety of scientific domains and analyzing a diversity of data types \(Figure[2](https://arxiv.org/html/2608.25286#S2.F2)A–B; Appendix Table[1](https://arxiv.org/html/2608.25286#A1.T1)\)\. Transcriptomics is the most common type of data \(16 tasks\), followed by epigenomics \(6 tasks\), proteomics \(2 tasks\), genomics \(1 task\), and microbiome sequencing \(1 task\); 6 tasks analyze multiple data types\. The distribution of scientific domains represented inBixBench3covers many of the same domains as recent papers posted to bioRxiv, with an over\-representation of drug\-discovery\-related research \(Figure[2](https://arxiv.org/html/2608.25286#S2.F2)B\)\.

The raw input data size averages 67 GB per task, ranging from 7 to 241 GB \(Figure[2](https://arxiv.org/html/2608.25286#S2.F2)C\)\. Each task consists of producing at least four data artifacts, with a median of five and a maximum of 14, for 138 artifacts in total across tasks \(Figure[2](https://arxiv.org/html/2608.25286#S2.F2)D\)\. Of these artifacts, 56 are direct derivations of the raw data \(depth 1; e\.g\., read count matrices\), while 44 are two analysis steps from raw \(depth 2; e\.g\., differential gene expression tables\), and 38 are three or more steps away \(depth 3\+; e\.g\., pathway enrichments, Figure[2](https://arxiv.org/html/2608.25286#S2.F2)E\)\.

![Refer to caption](https://arxiv.org/html/2608.25286v1/figures/figure1_benchmark_overview.png)Figure 2:Composition ofBixBench3\.\(A\) Number of tasks analyzing each data type\. Tasks analyzing multiple data types contribute to multiple bars\. \(B\) Shares of tasks assigned to each scientific domain inBixBench3and a random sample of 500 bioRxiv papers\. \(C\) Raw\-data size per task\. \(D\) Number of graded artifacts per task\. \(E\) Artifact analysis depth\. Depth 1 denotes outputs derived directly from raw data, depth 2 denotes outputs derived from depth\-1 artifacts, and depth 3\+ denotes all subsequent steps\.![Refer to caption](https://arxiv.org/html/2608.25286v1/figures/figure3_overall_performance.png)Figure 3:Overall model performance onBixBench3\.\(A\) Mean score for each model across 20 tasks\. \(B\) Mean binary artifact pass score by model and analysis depth, comprising 56 depth\-1, 44 depth\-2, and 38 depth\-3\+ artifacts\. \(C\) Task scores for each model, with tasks ordered by median score across models\. \(D\) Mean task score by model and scientific domain\. \(E\) Model–task score distributions stratified by raw\-data size; black points show individual model–task scores, and parenthetical values give the number of tasks in each size category\. Error bars in \(A\) and \(B\) denote the standard error\. Asterisks in \(B\) denote two\-sided Mann–Whitney U testp<0\.05p<0\.05; p\-values in \(E\) are from two\-sided Mann–Whitney U tests\.![Refer to caption](https://arxiv.org/html/2608.25286v1/figures/figure4_pareto_optimal.png)Figure 4:Accuracy and computational expense\.\(A\) Cost per task for each model across 20 tasks, shown on a log scale\. For GLM 5\.2, separate boxes show the observed OpenRouter cost and the cost after rescaling each task to the mean cache hit rate of the other models\. \(B\) Mean cost versus mean task score\. \(C, E, G\) Token usage, wall\-clock time, and model turns per task\. \(D, F, H\) Mean task score versus each corresponding measure of computational efficiency\. Error bars denote standard errors, and the dashed line marks the Pareto frontier\.![Refer to caption](https://arxiv.org/html/2608.25286v1/figures/figure5_failure_modes.png)Figure 5:Process\-judge failure modes by model and score\.\(A\) Cumulative number of failure\-mode tags assigned across all tasks for each model\. Models shown left to right in decreasing order of overall score\. \(B\) Proportion of failure\-mode tags of each type among all tags received by each model\. \(C\)log2\\log\_\{2\}enrichment of each failure mode within task\-score quantiles relative to all attempts; asterisks denote Benjamini–Hochberg\-adjustedq<0\.05q<0\.05from two\-sided Fisher’s exact tests\.
### 2\.2Model performance varies across tasks, scientific domains, and data sizes

We evaluated 13 frontier models on the 20BixBench3tasks, producing 260 completed runs and 1,794 graded artifact evaluations\. GPT 5\.6 Sol was the highest\-scoring model, reaching an average score of 0\.48 across tasks, followed by Kimi K3 at 0\.47, GLM 5\.2 at 0\.46, and Claude Opus 4\.8 at 0\.46 \(Figure[3](https://arxiv.org/html/2608.25286#S2.F3)A\)\. Claude Opus 5 scored highly on most tasks, but on tasks 5, 19, and 20 it incorrectly formatted its some artifacts, drawing its score down to 0\.41 overall \(Appendix[A\.2](https://arxiv.org/html/2608.25286#A1.SS2)\)\. The task score represents the proportion of requested artifacts which were reproduced closely enough to the corresponding published artifacts to preserve their main scientific interpretation \(Supplementary Figure[1](https://arxiv.org/html/2608.25286#Sx4.F1)\)\. This criterion was calibrated using domain\-expert review of agent\-produced artifacts \(Section[4\.4](https://arxiv.org/html/2608.25286#S4.SS4); Supplementary Figure[2](https://arxiv.org/html/2608.25286#Sx4.F2); Appendix[A\.3](https://arxiv.org/html/2608.25286#A1.SS3)\)\. GPT 5\.6 Sol’s score of 0\.48 therefore indicates that 48% of its requested artifacts had an equivalent biological meaning to the original artifacts\.

Six models exhibited a significant decrease in performance at greater analysis depths, with depth 3\+ artifacts having lower binary artifact pass scores than either depth 1 or depth 2 artifacts \(Figure[3](https://arxiv.org/html/2608.25286#S2.F3)B\)\. Averaged across models, the mean binary artifact pass score was 0\.30 for the 56 depth\-1 artifacts, 0\.44 for the 44 depth\-2 artifacts, and 0\.24 for the 38 artifacts at depth three or greater\.

Models with similar overall performance often had varied performance across individual tasks \(Figure[3](https://arxiv.org/html/2608.25286#S2.F3)C\)\. Among model pairs with defined correlations, pairwise Spearman correlations of per\-task scores ranged from−0\.27\-0\.27to0\.990\.99\(Supplementary Figure[3](https://arxiv.org/html/2608.25286#Sx4.F3)\)\. For example, although GLM 5\.2 and GPT 5\.5 differed in average score by only 0\.02 points, on Task 17 GLM 5\.2 scored 0\.50 while GPT 5\.5 scored 0\.25\. This study investigated how chromatin regulation by Polycomb/RING1affects gene activity inDrosophilacells\. To analyze this, agents had to combine chromatin immunoprecipitation sequencing \(ChIP\-seq\) profiles with Thousands of Reporters Integrated in Parallel \(TRIP\) measurements, which quantify the expression of barcoded reporter genes inserted at different genomic locations\.

The variation in scores across tasks was associated with both scientific domain and raw data size\. Models scored the highest on tasks drawn from microbiology, evolution and ecology, and immunology papers, while struggling on tasks investigating neuroscience and cell biology \(Figure[3](https://arxiv.org/html/2608.25286#S2.F3)D\)\. Additionally, for tasks with more than 100 GB of raw input data, models scored only 0\.10 on average, compared with 0\.34 for tasks with 50–100 GB and 0\.37 for tasks with less than 50 GB \(Figure[3](https://arxiv.org/html/2608.25286#S2.F3)E\)\. Performance also varied across data and assay types \(Supplementary Figure[4](https://arxiv.org/html/2608.25286#Sx4.F4)\)\.

### 2\.3Model efficiency and cost

The utility of AI agents in computational research depends not only on performance but also on cost and efficiency\. There was a 367\-fold difference in the average cost perBixBench3task across models, with a minimum cost of $0\.35 and a maximum of $129\.14 \(Figure[4](https://arxiv.org/html/2608.25286#S2.F4)A\)\. Note that because OpenRouter routed GLM 5\.2 requests across several providers, a much lower cache hit rate was achieved for this model compared to the average across other models \(40\.1% for GLM 5\.2 vs\. 96\.0% for the other models\)\. We therefore report both the observed GLM 5\.2 cost and a cache\-adjusted estimate, in which cost is recomputed assuming a deployment achieving the average cache hit rate observed across other models \(Supplementary Figure[5](https://arxiv.org/html/2608.25286#Sx4.F5)\)\. Although cost and performance were broadly associated, maximum performance was not achieved at maximum cost; instead, several models formed a Pareto frontier representing distinct cost–performance trade\-offs \(Figure[4](https://arxiv.org/html/2608.25286#S2.F4)B\)\.

On average, eachBixBench3task attempt used 102 million tokens, 6\.8 hours, and 695 model turns\. The highest performance occurred at intermediate levels of token use, run time, and model turns \(Figure[4](https://arxiv.org/html/2608.25286#S2.F4)C–H\)\. Counting each model’s membership across the four cost and efficiency frontiers, GPT 5\.6 Sol, Claude Haiku 4\.5, and Gemini 3\.1 Flash Lite were Pareto\-optimal on all four, and Kimi K3 was Pareto\-optimal on three, whereas Claude Opus 4\.8, Claude Sonnet 4\.6, GLM 5\.2, Gemini 3\.1 Pro Preview, and Gemini 3\.5 Flash were optimal on none\.

### 2\.4Models exhibit distinct modes of failure

An LLM judge was used to annotate each model–task attempt with up to 10 failure\-mode tags from a fixed set \(Section[4\.5](https://arxiv.org/html/2608.25286#S4.SS5)\)\. Models receiving more failure\-mode tags generally performed worse: across models, the total count of tags across all tasks was strongly negatively correlated with mean task score \(Spearmanρ=−0\.92\\rho=\-0\.92,p=9\.9×10−6p=9\.9\\times 10^\{\-6\}; Figure[5](https://arxiv.org/html/2608.25286#S2.F5)A\), ranging from 31–51 total tags for the three highest\-scoring models to 102–117 for the two lowest\-scoring models\. The relative frequencies of specific failure modes also differed across models \(Figure[5](https://arxiv.org/html/2608.25286#S2.F5)B\), highlighting that models fail in qualitatively distinct ways\. Premature termination and repetitive retry loops showed the strongest associations with failure, occurring approximately 2\.0\- and 2\.2\-fold more often, respectively, in the lowest task\-score quantile than across all attempts; among the 65 attempts in the highest task\-score quantile, only one terminated prematurely and none entered a repetitive retry loop \(Figure[5](https://arxiv.org/html/2608.25286#S2.F5)C\)\. Environment setup failures, incomplete data, and synthetic or placeholder outputs were also enriched among the lowest\-scoring attempts, whereas method misconfiguration was depleted, likely because a run must complete enough of the analysis for a method to be misconfigured\.

## 3Discussion

### 3\.1WhatBixBench3reveals

The scope of tasks that AI agents can complete in biology is increasing\.To our knowledge,BixBench3is the biology benchmark with the longest horizon reported to date, as measured by run time and token usage\. Our evaluation of 13 frontier models onBixBench3shows that these systems are approaching the ability to complete tasks at the scale of entire research studies\. The best models scored approximately 0\.5 onBixBench3, with performance varying sharply between tasks\. This indicates that the top\-performing models reproduced approximately half of the requested artifacts to a degree that preserved their original scientific interpretation, according to the expert\-calibrated pass threshold \(Supplementary Figure[2](https://arxiv.org/html/2608.25286#Sx4.F2); Appendix[A\.3](https://arxiv.org/html/2608.25286#A1.SS3)\)\. So while performance remains far from perfect, our results indicate that models can coherently execute many computational biology tasks in sequence, each comparable to the individual tasks that challenged models just one year ago in BixBench\([Mitchener et al\., 2025a](https://arxiv.org/html/2608.25286#bib.bib14)\)\. Notably, however, because eachBixBench3task supplies an explicit methodological plan, success here means agents can begin to execute a specified analysis pipeline – not that they can decide which questions or analyses are worth pursuing\.

Agents struggle with data scale, long analysis chains, and error recovery\.The large input datasets and long sequences of dependent analyses inBixBench3appear to challenge current AI agents\. Across tasks, agents scored lowest on those with the largest raw datasets \(Figure[3](https://arxiv.org/html/2608.25286#S2.F3)E\)\. Within tasks, performance was lowest for analyses at depth 3\+ in the dependency chain\. One possible explanation for this is that both large datasets and long analysis chains extend the horizon over which errors introduced early can accumulate\. This effect may be compounded by the difficulty LLMs have in maintaining coherence across long contexts\([Liu et al\., 2024b](https://arxiv.org/html/2608.25286#bib.bib55)\)\. Therefore, models with stronger long\-context capabilities and agent architectures designed to manage extended computations, including recursive language models\([Zhang et al\., 2025](https://arxiv.org/html/2608.25286#bib.bib56)\)and world models\([Mitchener et al\., 2025b](https://arxiv.org/html/2608.25286#bib.bib7)\), may enable increased performance onBixBench3and similar tasks\.

Maximum performance does not necessitate maximum cost\.Kimi K3 was the second\-highest\-performing model and cost approximately 58% less than other models with comparable performance\. On average, models scoring at least 0\.40 onBixBench3used 28–179M tokens and approximately 271–1,149 model turns compared with 8–285M tokens and approximately 107–1,894 turns for models scoring below 0\.40, underscoring that greater computational expenditure did not lead to better performance\.

### 3\.2Limitations and future work

BixBench3is graded programmatically to avoid the ambiguity sometimes accompanying rubric\-based approaches\. However, there are multiple limitations that follow from this\. First, each task prompt must specify the method to be used for a particular analysis\. Without this guidance, an agent could choose a scientifically valid alternative method and receive a low artifact score simply because its output differs from the published artifact in format, scale, or other respects\. Relatedly,BixBench3tasks prescribe exactly which analyses must be carried out, so that the artifacts available from the original paper may be used to grade the analyses done by the agent\. Thus,BixBench3does not test the ability of agents to decide which analyses or questions are worth pursuing, but rather their ability to execute those prescribed analyses\. Finally, the benchmark inherits any limitations that may be present in its source papers, such as flawed artifacts due to errors made by the original authors and incorrect or under\-specified methods\.

## 4Methods

### 4\.1Task prompt

For each task, the agent is given a prompt consisting of four parts: \(1\) thegeneral instructionsgive the agent an overview of the computational workspace, installed software, network policy, and reporting requirements; \(2\) theresearch objectivestates the biological question and describes the provided raw data and reference files; \(3\)method guidancenames the tools, parameters, contrasts, and filtering rules which should be applied to produce the required output artifacts, mirroring the methods of the original paper without revealing its results; and \(4\) therequired outputssection specifies the exact output path and expected file format for each analysis artifact the agent must produce\.

The complete prompt for the example task shown in Figure[6](https://arxiv.org/html/2608.25286#S4.F6)is provided in Appendix[A\.5](https://arxiv.org/html/2608.25286#A1.SS5)\.

Feedback from domain experts was used to refine the task prompts and method guidance \(Appendix[A\.3](https://arxiv.org/html/2608.25286#A1.SS3)\)\.

The artifacts in each task form a dependency graph from raw data to downstream analyses \(Figure[6](https://arxiv.org/html/2608.25286#S4.F6)A\)\. Agent outputs are compared programmatically with the corresponding published artifacts to compute an artifact score \(Section[4\.4](https://arxiv.org/html/2608.25286#S4.SS4); Figure[6](https://arxiv.org/html/2608.25286#S4.F6)B\), and binary artifact pass scores are averaged to obtain the task score\.

![Refer to caption](https://arxiv.org/html/2608.25286v1/figures/example_paper.png)Figure 6:BixBench3task structure and grading\.\(A\) An artifact dependency graph for Task 19, based on[Naim et al\. \(2026\)](https://arxiv.org/html/2608.25286#bib.bib48)\. This paper investigates how loss of theTCER\-1gene alters endogenous small\-RNA regulation and mRNA expression inC\. elegans\. First, mRNA and small\-RNA sequencing data are analyzed separately, then combined to identify coordinated changes in gene regulation\. \(B\) Artifact grading compares an agent\-produced matrix to the published ground truth by aligning rows and columns, penalizing missing or extra entries, and scoring numerical agreement; for the illustrated matrix, the artifact score is the product of row F1, column F1, and concordance correlation \(CCC\), and artifact scores greater than 0\.80 receive a binary artifact pass score of one\.
### 4\.2Runtime environment

Each agent ran on a Google Cloud n2\-standard\-32 virtual machine \(VM\) with 32 vCPUs, 128 GB of memory, a 500 GB boot disk, and no GPU\. Runs had a 24\-hour wall\-clock time limit and no fixed token limit\. The agent operated inside a Docker container which included Python and R with common scientific and data\-analysis packages; Java; package managers for PyPI, CRAN, Bioconductor, and Conda/Bioconda; and common command\-line, archive, and download utilities\. Agents could install additional method\-specific software during a run\.

The agent’s web access was restricted in two ways\. First, all HTTP and HTTPS traffic passed through a separate gateway with a standing allowlist for package registries, operating\-system mirrors, and other routine software infrastructure\. Second, the agent could request other resources through therequest\_web\_accesstool by supplying a URL and justification\. An LLM adjudicator denied resources that could leak the expected answers – such as the original paper and its processed datasets – while allowing access to bioinformatics software, documentation, and general data like reference genomes\.

### 4\.3Agent harness

The agent harness was implemented with Inspect AI\([AI Security Institute, UK, 2024](https://arxiv.org/html/2608.25286#bib.bib33)\)\. Each model ran in Inspect’s ReAct loop, alternating between model responses and tool calls while retaining the resulting messages and outputs in its context\. Agent runs were capped at 24 hours of runtime or 5,000 messages\. Inspect applied automatic context compaction when the agent hit 90% usage of its context window\. Every agent had five tools: \(1\)bash\_sessionprovided a persistent shell for running bash commands inside the Docker container; \(2\)pythonallowed execution of Python code; \(3\)text\_editorread, created, and modified text files; \(4\)request\_web\_accesssubmitted a URL and justification to the network adjudicator described above; and \(5\)submitreturned the agent’s final answer and ended the ReAct loop\.

Run costs were calculated from per\-call token usage using provider list prices\. For Gemini 3\.1 Pro and GPT 5\.5, higher long\-context rates were applied to the entire call when total prompt tokens, including cached tokens, exceeded 200,000 and 272,000, respectively\. Claude Opus 5 costs used Anthropic list prices, while Kimi K3 costs used Moonshot AI list prices because its OpenRouter runs were routed only to Moonshot AI\. GPT 5\.6 Sol input costs were bounded using rates of $5–$6\.25 per million tokens because retained input counts included unrecovered cache writes, and the reported cost is the midpoint of these bounds\. Cost estimates exclude cache\-storage costs, which Inspect does not report\.

### 4\.4Artifacts and grading

Artifacts drawn from each paper form a directed acyclic graph, with raw data at its root and the most downstream analyses at its leaves\. When an agent finishes a task, the Inspect scorer reads each agent\-produced artifact from the VM and grades it against the corresponding artifact from the original paper\. Each artifact is graded using one or more of four different metrics\. The metrics applied depend on the artifact’s format and the biological information it contains:

\(1\)Row and column recovery \(F1\)measures exact recovery of row or column identifiers using the F1 score\. For example, gene lists are scored by comparing the agent’s gene IDs with the published set, penalizing both missing and extra genes\. \(2\)Numerical agreement \(Lin’s CCC\)measures agreement between aligned numerical values using Lin’s concordance correlation coefficient\([Lin, 1989](https://arxiv.org/html/2608.25286#bib.bib34)\), with a log transform where appropriate\. CCC was used because, unlike Pearson correlation, this metric penalizes differences in scale in addition to imperfect linear association\. Thus, identical rankings with a changed slope or offset do not receive a perfect CCC\. For example, the values of expression matrices are scored by CCC, comparing the published and agent\-produced value vectors for each sample\. \(3\)Categorical\-label agreement \(macro F1\)measures agreement between categorical annotations for aligned rows by averaging F1 across label classes\. For example, cell\-metadata tables are aligned by cell barcode and then scored on the assigned cell\-type labels compared with the published labels\. \(4\)Genomic\-interval recovery \(overlap F1\)measures recovery of genomic regions under an artifact\-specific overlap rule\. For example, a set of called peaks is compared with published chromosome, start, and end intervals using a minimum reciprocal overlap\.

The metrics calculated for an artifact are multiplied to produce a single “artifact score\.” For example, an RNA\-seq count matrix may be evaluated using row F1 for gene recovery, column F1 for sample recovery, and the concordance correlation coefficient for expression\-value agreement\. Missing, empty, or unreadable files receive an artifact score of zero\.

Each artifact is considered to “pass” if it has an artifact score of≥0\.8\\geq 0\.8\. The 0\.80 threshold was chosen based on domain\-expert review of a subset of artifacts\. Experts were asked to rate 25 agent\-generated artifacts from 1 to 5 based on the degree to which the agent\-generated artifact retained the same main biological meaning as the artifact from the original paper \(Appendix[A\.3](https://arxiv.org/html/2608.25286#A1.SS3)\)\. Ratings of 4 or higher indicate that observed discrepancies did not alter the main interpretation of the artifact\. On average, artifacts rated≥4\\geq 4had a mean artifact score of 0\.80, so this was set as the binary artifact pass threshold \(Supplementary Figure[2](https://arxiv.org/html/2608.25286#Sx4.F2)\)\.

A task’s score is the proportion of its artifacts that pass\.

### 4\.5Failure mode annotation

Each completed model–task attempt was reviewed by an LLM judge \(GPT 5\.5\)\. The judge received the task prompt, the agent\-writtenMETHODS\.md, a compacted execution trace, the numeric grading summary, and previews of the generated artifacts\. The judge assigned each model–task attempt 0–10 run\-level failure\-mode tags\. The judge required concrete evidence for every tag and returned no tag when none applied\. The ten failure modes selectable by the judge and their definitions are listed in Table[3](https://arxiv.org/html/2608.25286#A1.T3)\.

### 4\.6Paper selection and benchmark construction

A paper was eligible for inclusion inBixBench3when its raw data were publicly accessible, at least four ground\-truth artifacts were available \(typically as supplementary materials\), and the analysis was judged executable within the benchmark runtime and compute budget \(32 CPUs, 500 GB of writable storage, and 24 hours\)\.

An agentic pipeline was used to screen papers and assemble tasks for human review\. The pipeline comprised six stages: \(1\)metadata screeningidentified papers with sufficient public data and supplementary materials for further review; \(2\)data\-availability reviewestablished that the raw data were accessible and the study contained both a processed matrix and a substantive scientific analysis; \(3\)artifact inventoryidentified structured outputs representing meaningful stages of the paper’s analysis; \(4\)grading\-specification constructiondefined the output contract for each artifact and selected the metrics used to grade it; \(5\)prompt and task creationproduced a research objective and methodological guidance based on each paper; and \(6\)task refinementinvolved running a subset of models on each task and iteratively identifying and correcting issues with the prompt, data, or grading\.

### 4\.7Models

Thirteen frontier models were assessed: GPT 5\.4 Nano \(openai/gpt\-5\.4\-nano\), GPT 5\.4 Mini \(openai/gpt\-5\.4\-mini\), GPT 5\.5 \(openai/gpt\-5\.5\), GPT 5\.6 Sol \(openai/gpt\-5\.6\-sol\), Claude Haiku 4\.5 \(anthropic/claude\-haiku\-4\-5\), Claude Sonnet 4\.6 \(anthropic/claude\-sonnet\-4\-6\), Claude Opus 4\.8 \(anthropic/claude\-opus\-4\-8\), Claude Opus 5 \(anthropic/claude\-opus\-5\), Gemini 3\.1 Flash Lite \(google/gemini\-3\.1\-flash\-lite\), Gemini 3\.1 Pro Preview \(google/gemini\-3\.1\-pro\-preview\), Gemini 3\.5 Flash \(google/gemini\-3\.5\-flash\), GLM 5\.2 \(openrouter/z\-ai/glm\-5\.2\), and Kimi K3 \(openrouter/moonshotai/kimi\-k3\)\. All models were accessed through their first\-party APIs, except GLM 5\.2 and Kimi K3, which were accessed through OpenRouter\. GLM 5\.2 completions were served from Fireworks, Baseten, Nebius, Together, and Chutes, while Kimi K3 used Moonshot AI\. Every model was evaluated at its respective maximum thinking effort: GPT 5\.4 Nano, GPT 5\.4 Mini, and GPT 5\.5 used extra\-high effort; GPT 5\.6 Sol used max effort; Anthropic models used max effort, except Haiku 4\.5, which was set to a 63,999\-token reasoning budget; Gemini models used high effort; and GLM 5\.2 and Kimi K3 used max effort\.

## Statement of Contributions

Z\.K\., A\.T\.W\., and J\.M\.L\. conceived and designed the overall project\. J\.M\.L\. supervised the project\. Z\.K\. designed the benchmark tasks and infrastructure\. J\.V\.\-A\., J\.L\., and Z\.K\. coordinated with scientific domain experts for evaluation\. Z\.K\. and J\.M\.L\. prepared the initial draft of the manuscript, figures, and tables\. All authors contributed to the final version of the manuscript\. M\.M\.H\., S\.G\.R\., and A\.D\.W\. supervise research at Edison Scientific, Inc\.

## Competing Interests

All authors were employed by Edison Scientific for the duration of this study\.

## Data and Code Availability

Code to runBixBench3is available on[GitHub](https://github.com/EdisonScientific/BixBench3), and the dataset itself is available on[Hugging Face](https://huggingface.co/datasets/EdisonScientific/BixBench3)\. As new models are released,BixBench3will be updated on[Edison Advances](https://advances.edisonscientific.com/benchmarks/bixbench3)\.

## References

- S\. Agrawal, H\. B\. Anadkat, K\. K\. Athimoolam, H\. Bhardwaj, T\. Chowdhury, S\. Gao, P\. K\. Kamat, V\. Makwana, M\. H\. Shariff, A\. Badkul, L\. Xie, and A\. V\. SinitskiyCan AI conduct autonomous scientific research? case studies on two real\-world tasks\.bioRxiv\.External Links:[Document](https://dx.doi.org/10.64898/2026.01.05.697809)Cited by:[§1](https://arxiv.org/html/2608.25286#S1.p1.1)\.
- AI Security Institute, UK \(2024\)AI Security Institute, UKInspect AI: framework for large language model evaluations\.Note:Open\-source softwareSoftware and documentation; accessed 2026\-07\-21External Links:[Link](https://github.com/UKGovernmentBEIS/inspect_ai)Cited by:[§4\.3](https://arxiv.org/html/2608.25286#S4.SS3.p1.1)\.
- Anthropic \(2026\)AnthropicClaude Science, an AI workbench for scientists, is now available\.Note:Accessed 2026\-07\-21External Links:[Link](https://www.anthropic.com/news/claude-science-ai-workbench)Cited by:[§1](https://arxiv.org/html/2608.25286#S1.p1.1)\.
- Batschet al\.\(2026\)M\. Batsch, A\. Puzyrko, I\. Guex, and J\. R\. van der MeerHabitat fragmentation controls bacterial community composition outcomes\.bioRxiv\.External Links:[Document](https://dx.doi.org/10.64898/2026.02.04.703711)Cited by:[Table 1](https://arxiv.org/html/2608.25286#A1.T1.4.21.3.1.1)\.
- Bhowmicket al\.\(n\.d\.\)A\. Bhowmick, N\. Rahmatpour, S\. Sharma, Q\. Xu, A\. Gupta, A\. Lagwankar, A\. Banerjee, and C\. ZouVariantBench: an agentic benchmark for genetic variant discovery and interpretation\.LatchBio\.Note:Web manuscript; accessed 2026\-07\-21External Links:[Link](https://latch.bio/variantbench)Cited by:[§1\.1](https://arxiv.org/html/2608.25286#S1.SS1.p3.1)\.
- Bolgeret al\.\(2026\)I\. Bolger, R\. Shaw, O\. H\. Tam, C\. Gouveia Roque, C\. A\. Jackson, K\. O’Neill, NYGC ALS Consortium, C\. Smith, H\. Phatnani, K\. Natarajan, and M\. Gale HammellTDP\-43 dysfunction leads to the accumulation of cryptic transposable element\-derived exons, crypTEs, in iPSC\-derived neurons and ALS/FTD patient tissues\.bioRxiv\.External Links:[Document](https://dx.doi.org/10.64898/2026.01.09.698641)Cited by:[Table 1](https://arxiv.org/html/2608.25286#A1.T1.4.16.3.1.1)\.
- Budimiret al\.\(2026\)J\. Budimir, N\. Pavlović, A\. Gelemanović, V\. Juranić\-Lisnić, S\. Sperling, M\. Ninković, M\. Radman, and K\. TrajkovićContact\-dependent regulation of UV\-B/C\-induced cell fate by neighbouring intact cells\.bioRxiv\.External Links:[Document](https://dx.doi.org/10.64898/2026.02.11.704850)Cited by:[Table 1](https://arxiv.org/html/2608.25286#A1.T1.4.3.3.1.1)\.
- Burret al\.\(2025\)S\. P\. Burr, K\. Auckland, A\. Glynos, A\. Dhawanjewar, W\. Wei, C\. Ryall, A\. Hynes\-Allen, M\. Prater, M\. Sczaniecka\-Clift, J\. Prudent, P\. F\. Chinnery, and J\. van den AmeeleMitoPerturb\-Seq identifies common and gene\-specific single\-cell responses to mitochondrial DNA depletion and heteroplasmy\.bioRxiv\.External Links:[Document](https://dx.doi.org/10.1101/2025.07.08.663208)Cited by:[Table 1](https://arxiv.org/html/2608.25286#A1.T1.4.7.3.1.1)\.
- Chappellet al\.\(2021\)G\. A\. Chappell, M\. M\. Heintz, and L\. C\. HawsTranscriptomic analyses of livers from mice exposed to 1,4\-dioxane for up to 90 days to assess potential mode\(s\) of action underlying liver tumor development\.Current Research in Toxicology2,pp\. 30–41\.External Links:[Document](https://dx.doi.org/10.1016/j.crtox.2021.01.003)Cited by:[Table 1](https://arxiv.org/html/2608.25286#A1.T1.4.12.3.1.1)\.
- Chenget al\.\(2026a\)L\. Cheng, N\. Bianchi, H\. Soldati, L\. Bersier, M\. Evrard, A\. Caillon, A\. S\. Chawla, B\. Nakken, P\. Szodoray, E\. De Bousser, N\. Festjens, T\. V\. Petrova, S\. Lemeille, L\. K\. Mackay, N\. Callewaert, M\. Swamy, S\. M\. G\. Braun, and S\. BecattiniA glycosylation\-dependent checkpoint restrains intestinal intra\-epithelial lymphocyte activation\.bioRxiv\.External Links:[Document](https://dx.doi.org/10.64898/2026.01.02.697332)Cited by:[Table 1](https://arxiv.org/html/2608.25286#A1.T1.4.19.3.1.1)\.
- Chenget al\.\(2026b\)P\.\-C\. Cheng, D\. Kamenev, P\. Kameneva, C\. Fitzpatrick, I\. Adameyko, P\. V\. Kharchenko, and K\. ZhangHigh\-resolution retrospective single cell lineage tracing with mutable homopolymers\.bioRxiv\.External Links:[Document](https://dx.doi.org/10.64898/2026.03.10.709901)Cited by:[Table 1](https://arxiv.org/html/2608.25286#A1.T1.4.14.3.1.1)\.
- Colquitt and Brainard \(2025\)B\. M\. Colquitt and M\. S\. BrainardAn interneuronal CRH and CRHBP circuit stabilizes birdsong performance\.bioRxiv\.External Links:[Document](https://dx.doi.org/10.1101/2025.08.16.670679)Cited by:[Table 1](https://arxiv.org/html/2608.25286#A1.T1.4.4.3.1.1)\.
- De Vivo \(2026\)M\. De VivoHow many are you? open data and bioinformatics reveal species misidentification and potential introgression inChordodes\(Phylum Nematomorpha\)\.bioRxiv\.External Links:[Document](https://dx.doi.org/10.64898/2026.02.03.703548)Cited by:[Table 1](https://arxiv.org/html/2608.25286#A1.T1.4.5.3.1.1)\.
- Desaiet al\.\(2026\)R\. Desai, J\. Hu, J\. Cabezas, N\. Harsola, P\. Shukla, R\. Ben Chaim, A\. El Assadi, O\. M\. Kamath, F\. Faldu, P\. Hebbar, J\. Sun, Y\. Li, P\. Srinivasan, I\. Gupta, C\. Settles, D\. Wang, D\. Chen, P\. Raja, A\. Liu, M\. Šuppa, N\. Sasikumar, L\. Kong, E\. Quintanilla, X\. Li, I\. Bercovich, and S\. DillmannSWE\-Marathon: can agents autonomously complete ultra\-long\-horizon software work?\.External Links:2606\.07682,[Document](https://dx.doi.org/10.48550/arXiv.2606.07682),[Link](https://arxiv.org/abs/2606.07682)Cited by:[§1\.1](https://arxiv.org/html/2608.25286#S1.SS1.p2.1)\.
- Dikset al\.\(2026a\)I\. Diks, H\. Muralidharan, T\. Proctor, and K\. WorkmanVerifiable benchmarking of long\-horizon spatial biology\.External Links:2605\.28065,[Document](https://dx.doi.org/10.48550/arXiv.2605.28065),[Link](https://arxiv.org/abs/2605.28065)Cited by:[§1\.1](https://arxiv.org/html/2608.25286#S1.SS1.p3.1)\.
- Dikset al\.\(2026b\)I\. Diks, Z\. Yang, A\. Banerjee, T\. Proctor, and K\. WorkmanscBench\-Long: verifiable benchmarking of long\-horizon single\-cell biology\.External Links:2606\.26563,[Document](https://dx.doi.org/10.48550/arXiv.2606.26563),[Link](https://arxiv.org/abs/2606.26563)Cited by:[§1\.1](https://arxiv.org/html/2608.25286#S1.SS1.p3.1)\.
- Fenget al\.\(2025\)L\. Feng, Y\. Hou, A\. Toghani, Z\. Wang, B\. Tang, N\. Atkinson, H\. Li, Y\. Qiao, Y\. Wang, J\. Hua, J\. Zhai, and W\. MaA conserved small RNA\-generating gene cluster undergoes sequence diversification and contributes to plant immunity\.bioRxiv\.External Links:[Document](https://dx.doi.org/10.1101/2025.07.20.665670)Cited by:[Table 1](https://arxiv.org/html/2608.25286#A1.T1.4.6.3.1.1)\.
- Gaoet al\.\(2024\)S\. Gao, A\. Fang, Y\. Huang, V\. Giunchiglia, A\. Noori, J\. R\. Schwarz, Y\. Ektefaie, J\. Kondic, and M\. ŽitnikEmpowering biomedical discovery with AI agents\.Cell187\(22\),pp\. 6125–6151\.External Links:[Document](https://dx.doi.org/10.1016/j.cell.2024.09.022)Cited by:[§1](https://arxiv.org/html/2608.25286#S1.p1.1)\.
- Ghareebet al\.\(2026\)A\. E\. Ghareeb, B\. Chang, L\. Mitchener, A\. Yiu, C\. J\. Szostkiewicz, D\. Shved, G\. J\. Gyimesi, J\. M\. Laurent, S\. M\. Wright, M\. T\. Razzak, A\. D\. White, S\. C\. Finnemann, M\. M\. Hinks, and S\. G\. RodriquesA multi\-agent system for automating scientific discovery\.Nature655\(8122\),pp\. 497–505\.External Links:[Document](https://dx.doi.org/10.1038/s41586-026-10652-y)Cited by:[§1](https://arxiv.org/html/2608.25286#S1.p1.1)\.
- Huanget al\.\(2026\)K\. Huang, S\. Zhang, H\. Wang, Y\. Qu, Y\. Lu, R\. Li, Y\. Roohani, L\. Qiu, S\. Cao, G\. Li, J\. Zhang, D\. Yin, R\. Wierenga, D\. Kavi, S\. Liu, T\. She, S\. Marwaha, J\. N\. Carter, X\. Zhou, M\. T\. Wheeler, J\. A\. Bernstein, M\. Wang, P\. He, J\. Zhou, M\. P\. Snyder, L\. Cong, A\. Regev, and J\. LeskovecAutonomous biomedical research with an artificial intelligence agent\.Science,pp\. eadz4351\.External Links:[Document](https://dx.doi.org/10.1126/science.adz4351),[Link](https://pubmed.ncbi.nlm.nih.gov/42424436/)Cited by:[§1](https://arxiv.org/html/2608.25286#S1.p1.1)\.
- Jansenet al\.\(2024\)P\. Jansen, M\. Côté, T\. Khot, E\. Bransom, B\. D\. Mishra, B\. P\. Majumder, O\. Tafjord, and P\. ClarkDiscoveryWorld: a virtual environment for developing and evaluating automated scientific discovery agents\.InAdvances in Neural Information Processing Systems,Vol\.37,pp\. 10088–10116\.Note:Datasets and Benchmarks TrackExternal Links:[Document](https://dx.doi.org/10.52202/079017-0324)Cited by:[§1\.1](https://arxiv.org/html/2608.25286#S1.SS1.p3.1)\.
- Jimenezet al\.\(2024\)C\. E\. Jimenez, J\. Yang, A\. Wettig, S\. Yao, K\. Pei, O\. Press, and K\. R\. NarasimhanSWE\-bench: can language models resolve real\-world github issues?\.InThe Twelfth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=VTF8yNQM66)Cited by:[§1\.1](https://arxiv.org/html/2608.25286#S1.SS1.p2.1)\.
- Jinet al\.\(2025\)R\. Jin, Y\. Guo, Y\. Qu, M\. Yang, C\. Shang, Q\. Yang, L\. Chao, Y\. Zhou, R\. Xu, Z\. Xu, R\. Zhou, Z\. Zhang, M\. Wang, X\. Zhang, and L\. CongBioLab: end\-to\-end autonomous life sciences research with multi\-agents system integrating biological foundation models\.bioRxiv\.External Links:[Document](https://dx.doi.org/10.1101/2025.09.03.674085)Cited by:[§1](https://arxiv.org/html/2608.25286#S1.p1.1)\.
- Kahnet al\.\(2025\)T\. G\. Kahn, A\. Garrido, A\. Yushkova, M\. Kim, A\. Glotov, S\. Sreekumar, J\. Larsson, and Y\. B\. SchwartzPolycomb repression works without Siesta\.bioRxiv\.External Links:[Document](https://dx.doi.org/10.1101/2025.07.18.664654)Cited by:[Table 1](https://arxiv.org/html/2608.25286#A1.T1.4.18.3.1.1)\.
- LatchBio \(2026\)LatchBiobenchmarks\.bio: agentic AI benchmarks on messy, real\-world biological data\.Note:Accessed 2026\-07\-21External Links:[Link](https://benchmarks.bio/)Cited by:[§1\.1](https://arxiv.org/html/2608.25286#S1.SS1.p3.1)\.
- Laurentet al\.\(2026\)J\. M\. Laurent, A\. Bou, M\. Pieler, C\. Igoe, A\. Andonian, S\. Narayanan, J\. Braza, A\. Sanchez Vassopoulos, J\. L\. Steenwyk, B\. Lash, A\. D\. White, and S\. G\. RodriquesLABBench2: an improved benchmark for ai systems performing biology research\.External Links:2604\.09554,[Document](https://dx.doi.org/10.48550/arXiv.2604.09554),[Link](https://arxiv.org/abs/2604.09554)Cited by:[§1\.1](https://arxiv.org/html/2608.25286#S1.SS1.p3.1),[§1](https://arxiv.org/html/2608.25286#S1.p2.1)\.
- Laurentet al\.\(2024\)J\. M\. Laurent, J\. D\. Janizek, M\. Ruzo, M\. M\. Hinks, M\. J\. Hammerling, S\. Narayanan, M\. Ponnapati, A\. D\. White, and S\. G\. RodriquesLAB\-Bench: measuring capabilities of language models for biology research\.External Links:2407\.10362,[Document](https://dx.doi.org/10.48550/arXiv.2407.10362),[Link](https://arxiv.org/abs/2407.10362)Cited by:[§1\.1](https://arxiv.org/html/2608.25286#S1.SS1.p3.1),[§1](https://arxiv.org/html/2608.25286#S1.p2.1)\.
- Leunget al\.\(2025\)S\. K\. Leung, E\. M\. Walker, S\. Policicchio, A\. Dahir, D\. Seiler Vellame, A\. R\. Smith, R\. Swarbrick, K\. Lunnon, E\. L\. Dempster, Z\. Ahmed, E\. Hannon, I\. Castanho, and J\. MillMethylomic signatures of tau and amyloid\-beta in transgenic mouse models of Alzheimer’s disease neuropathology\.bioRxiv\.External Links:[Document](https://dx.doi.org/10.1101/2025.07.28.666515)Cited by:[Table 1](https://arxiv.org/html/2608.25286#A1.T1.4.8.3.1.1)\.
- Li and Ho \(2026\)J\. Li and A\. HoGeneBench\-Pro: evaluating multistage statistical reasoning in genomics, quantitative biology, and translational biomedicine\.bioRxiv\.External Links:[Document](https://dx.doi.org/10.64898/2026.06.29.735386)Cited by:[§1\.1](https://arxiv.org/html/2608.25286#S1.SS1.p3.1)\.
- Lin \(1989\)L\. I\. LinA concordance correlation coefficient to evaluate reproducibility\.Biometrics45\(1\),pp\. 255–268\.External Links:[Document](https://dx.doi.org/10.2307/2532051),[Link](https://europepmc.org/article/MED/2720055)Cited by:[§4\.4](https://arxiv.org/html/2608.25286#S4.SS4.p2.1)\.
- Liuet al\.\(2024a\)H\. Liu, S\. Chen, Y\. Zhang, and H\. WangGenoTEX: an LLM agent benchmark for automated gene expression data analysis\.External Links:2406\.15341,[Document](https://dx.doi.org/10.48550/arXiv.2406.15341),[Link](https://arxiv.org/abs/2406.15341)Cited by:[§1\.1](https://arxiv.org/html/2608.25286#S1.SS1.p3.1)\.
- Liuet al\.\(2024b\)N\. F\. Liu, K\. Lin, J\. Hewitt, A\. Paranjape, M\. Bevilacqua, F\. Petroni, and P\. LiangLost in the middle: how language models use long contexts\.Transactions of the Association for Computational Linguistics12,pp\. 157–173\.External Links:[Document](https://dx.doi.org/10.1162/tacl%5Fa%5F00638),[Link](https://aclanthology.org/2024.tacl-1.9/)Cited by:[§3\.1](https://arxiv.org/html/2608.25286#S3.SS1.p2.1)\.
- Liuet al\.\(2025\)Y\. Liu, L\. Lv, X\. Zhang, J\. Wang, L\. Yuan, and Y\. TianBioProBench: comprehensive dataset and benchmark in biological protocol understanding and reasoning\.External Links:2505\.07889,[Document](https://dx.doi.org/10.48550/arXiv.2505.07889),[Link](https://arxiv.org/abs/2505.07889)Cited by:[§1\.1](https://arxiv.org/html/2608.25286#S1.SS1.p3.1),[§1](https://arxiv.org/html/2608.25286#S1.p2.1)\.
- Luoet al\.\(2025\)E\. Luo, J\. Jia, Y\. Xiong, X\. Li, X\. Guo, B\. Yu, M\. Hao, L\. Wei, and X\. ZhangBenchmarking AI scientists for omics data driven biological discovery\.External Links:2505\.08341,[Document](https://dx.doi.org/10.48550/arXiv.2505.08341),[Link](https://arxiv.org/abs/2505.08341)Cited by:[§1\.1](https://arxiv.org/html/2608.25286#S1.SS1.p3.1)\.
- Martineket al\.\(2025\)V\. Martinek, A\. Gariboldi, D\. Tzimotoudis, A\. Alberdi Escudero, E\. Blake, D\. Čechák, L\. Cassar, A\. Balestrucci, and P\. AlexiouAgentomics\-ML: autonomous machine learning experimentation agent for genomic and transcriptomic data\.External Links:2506\.05542,[Document](https://dx.doi.org/10.48550/arXiv.2506.05542),[Link](https://arxiv.org/abs/2506.05542)Cited by:[§1](https://arxiv.org/html/2608.25286#S1.p1.1)\.
- Mehandruet al\.\(2025\)N\. Mehandru, A\. K\. Hall, O\. Melnichenko, Y\. Dubinina, D\. Tsirulnikov, D\. Bamman, A\. Alaa, S\. Saponas, and V\. S\. MalladiBioAgents: bridging the gap in bioinformatics analysis with multi\-agent systems\.Scientific Reports15\(1\),pp\. 39036\.External Links:[Document](https://dx.doi.org/10.1038/s41598-025-25919-z)Cited by:[§1](https://arxiv.org/html/2608.25286#S1.p1.1)\.
- Mendeset al\.\(2025\)E\. A\. Mendes, A\. Munshi, A\. Singh, M\. D\. Evans, H\. Chou, Y\. Y\. Kim, Y\. Song, A\. Jo, D\. Lee, J\. Ciampi, A\. Chambers, S\. Weitzel, M\. Deel, R\. C\. Bentley, J\. Khan, D\. Green, and C\. M\. LinardicRUNX2 inhibition disrupts a PAX3::FOXO1\-RUNX2 feed\-forward loop and dismantles oncogenic gene programs in fusion\-positive rhabdomyosarcoma\.bioRxiv\.External Links:[Document](https://dx.doi.org/10.1101/2025.07.21.665972)Cited by:[Table 1](https://arxiv.org/html/2608.25286#A1.T1.4.15.3.1.1)\.
- Merrillet al\.\(2026\)M\. A\. Merrill A\. G\. Shawet al\.Terminal\-Bench: benchmarking agents on hard, realistic tasks in command line interfaces\.External Links:2601\.11868,[Document](https://dx.doi.org/10.48550/arXiv.2601.11868),[Link](https://arxiv.org/abs/2601.11868)Cited by:[§1\.1](https://arxiv.org/html/2608.25286#S1.SS1.p2.1)\.
- METR \(2026\)METRTask\-completion time horizons of frontier AI models\.Note:Accessed 2026\-07\-21External Links:[Link](https://metr.org/time-horizons/)Cited by:[§1\.1](https://arxiv.org/html/2608.25286#S1.SS1.p2.1)\.
- Mitcheneret al\.\(2025a\)L\. Mitchener, J\. M\. Laurent, A\. Andonian, B\. Tenmann, S\. Narayanan, G\. P\. Wellawatte, A\. White, L\. Sani, and S\. G\. RodriquesBixBench: a comprehensive benchmark for LLM\-based agents in computational biology\.External Links:2503\.00096,[Document](https://dx.doi.org/10.48550/arXiv.2503.00096),[Link](https://arxiv.org/abs/2503.00096)Cited by:[§1\.1](https://arxiv.org/html/2608.25286#S1.SS1.p3.1),[§1](https://arxiv.org/html/2608.25286#S1.p2.1),[§3\.1](https://arxiv.org/html/2608.25286#S3.SS1.p1.1)\.
- Mitcheneret al\.\(2025b\)L\. Mitchener, A\. Yiu, B\. Chang, M\. Bourdenx, T\. Nadolski, A\. Sulovari, E\. C\. Landsness, D\. L\. Barabási, S\. Narayanan, N\. Evans, S\. Reddy, M\. Foiani, A\. Kamal, L\. P\. Shriver, F\. Cao, A\. T\. Wassie, J\. M\. Laurent, E\. Melville\-Green, M\. Caldas, A\. Bou, K\. F\. Roberts, S\. Zagorac, T\. C\. Orr, M\. E\. Orr, K\. J\. Zwezdaryk, A\. E\. Ghareeb, L\. McCoy, B\. Gomes, E\. A\. Ashley, K\. E\. Duff, T\. Buonassisi, T\. Rainforth, R\. J\. Bateman, M\. Skarlinski, S\. G\. Rodriques, M\. M\. Hinks, and A\. D\. WhiteKosmos: an AI scientist for autonomous discovery\.External Links:2511\.02824,[Document](https://dx.doi.org/10.48550/arXiv.2511.02824),[Link](https://arxiv.org/abs/2511.02824)Cited by:[§1](https://arxiv.org/html/2608.25286#S1.p1.1),[§3\.1](https://arxiv.org/html/2608.25286#S3.SS1.p2.1)\.
- Naimet al\.\(2026\)N\. Naim, F\. R\. G\. Amrit, L\. L\. Bahr, B\. E\. Montgomery, S\. Kuhn, T\. A\. Montgomery, and A\. GhaziA TCER\-1\-siRNA regulatory axis suppresses antibacterial innate immunity inC\. elegans\.bioRxiv\.External Links:[Document](https://dx.doi.org/10.64898/2026.01.31.702960)Cited by:[Table 1](https://arxiv.org/html/2608.25286#A1.T1.4.20.3.1.1),[Figure 6](https://arxiv.org/html/2608.25286#S4.F6)\.
- Nairet al\.\(2026\)S\. Nair, L\. M\. Gunsalus, B\. Orcutt\-Jahns, J\. Rossen, A\. Lal, C\. De Donno, M\. H\. Çelik, K\. Fletez\-Brant, X\. Xie, H\. Corrada Bravo, and G\. EraslanAgentic systems are adept at solving well\-scoped, verifiable problems in computational biology\.bioRxiv\.External Links:[Document](https://dx.doi.org/10.64898/2026.04.06.716850)Cited by:[§1\.1](https://arxiv.org/html/2608.25286#S1.SS1.p3.1),[§1](https://arxiv.org/html/2608.25286#S1.p2.1)\.
- Paninaet al\.\(2024\)S\. B\. Panina, J\. V\. Schweer, Q\. Zhang, G\. Raina, H\. A\. Hardtke, S\. Kim, W\. Yang, D\. Siegel, and Y\. J\. ZhangTargeting of REST with rationally\-designed small molecule compounds exhibits synergetic therapeutic potential in human glioblastoma cells\.BMC Biology22,pp\. 83\.External Links:[Document](https://dx.doi.org/10.1186/s12915-024-01879-0)Cited by:[Table 1](https://arxiv.org/html/2608.25286#A1.T1.4.10.3.1.1)\.
- Perryet al\.\(2025\)E\. K\. Perry, B\. Udayasuryan, E\. K\. Zegeye, C\. M\. Rose, M\. Reichelt, and M\.\-W\. TanA microbiota\-derived bile acid modulates biofilm formation by the probiotic strainEscherichia coliNissle 1917\.bioRxiv\.External Links:[Document](https://dx.doi.org/10.1101/2025.06.02.657493)Cited by:[Table 1](https://arxiv.org/html/2608.25286#A1.T1.4.13.3.1.1)\.
- Rajopadhyeet al\.\(2025\)S\. A\. Rajopadhye, V\. Lažetić, D\. Rodriguez Crespo, E\. Troemel, P\. Meister, and C\. WickyThe chromatin remodeler LET\-418/Mi\-2 regulates the intracellular pathogen response in theC\. elegansintestine\.bioRxiv\.External Links:[Document](https://dx.doi.org/10.1101/2025.06.17.659900)Cited by:[Table 1](https://arxiv.org/html/2608.25286#A1.T1.4.2.3.1.1)\.
- Reinet al\.\(2025\)D\. Rein, J\. Becker, A\. Deng, S\. Nix, C\. Canal, D\. O’Connel, P\. Arnott, R\. Bloom, T\. Broadley, K\. Garcia, B\. Goodrich, M\. Hasin, S\. Jawhar, M\. Kinniment, T\. Kwa, A\. Lajko, N\. Rush, L\. J\. Koba Sato, S\. Von Arx, B\. West, L\. Chan, and E\. BarnesHCAST: human\-calibrated autonomy software tasks\.External Links:2503\.17354,[Document](https://dx.doi.org/10.48550/arXiv.2503.17354),[Link](https://arxiv.org/abs/2503.17354)Cited by:[§1\.1](https://arxiv.org/html/2608.25286#S1.SS1.p2.1)\.
- Ribeiroet al\.\(2025\)G\. F\. Ribeiro, W\. Danecka, L\. Tomlinson, E\. W\. J\. Wallace, and D\. S\. ChildersAntifungal exposure can enhanceCandida glabratapathogenesis\.bioRxiv\.External Links:[Document](https://dx.doi.org/10.1101/2025.07.31.667834)Cited by:[Table 1](https://arxiv.org/html/2608.25286#A1.T1.4.17.3.1.1)\.
- Staraceet al\.\(2025\)G\. Starace, O\. Jaffe, D\. Sherburn, J\. Aung, J\. S\. Chan, L\. Maksin, R\. Dias, E\. Mays, B\. Kinsella, W\. Thompson, J\. Heidecke, A\. Glaese, and T\. PatwardhanPaperBench: evaluating AI’s ability to replicate AI research\.InProceedings of the 42nd International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.267,pp\. 56843–56873\.External Links:[Link](https://proceedings.mlr.press/v267/starace25a.html)Cited by:[§1\.1](https://arxiv.org/html/2608.25286#S1.SS1.p2.1)\.
- Terminal\-Bench \(2026\)Terminal\-Benchterminal\-bench: benchmarks for AI agents in terminal environments\.Note:Accessed 2026\-07\-21External Links:[Link](https://www.tbench.ai/)Cited by:[§1\.1](https://arxiv.org/html/2608.25286#S1.SS1.p2.1)\.
- Vals AI \(2026\)Vals AISWE\-bench verified\.Note:Accessed 2026\-07\-21External Links:[Link](https://www.vals.ai/benchmarks/swebench)Cited by:[§1\.1](https://arxiv.org/html/2608.25286#S1.SS1.p2.1)\.
- Wanget al\.\(2024\)S\. Wang, S\. O\. Klein, S\. Urban, M\. Staudt, N\. P\. F\. Barthes, D\. Willmann, J\. Bacher, M\. Sum, H\. Bauer, L\. Peng, G\. A\. Rennar, C\. Gratzke, K\. M\. Schüle, L\. Zhang, O\. Einsle, H\. Greschik, C\. MacLeod, C\. G\. Thomson, M\. Jung, E\. Metzger, and R\. SchüleStructure\-guided design of a selective inhibitor of the methyltransferase KMT9 with cellular activity\.Nature Communications15,pp\. 43\.External Links:[Document](https://dx.doi.org/10.1038/s41467-023-44243-6)Cited by:[Table 1](https://arxiv.org/html/2608.25286#A1.T1.4.11.3.1.1)\.
- Wanget al\.\(2025\)Z\. Wang, B\. Danek, and J\. SunBioDSA\-1K: benchmarking data science agents for biomedical research\.External Links:2505\.16100,[Document](https://dx.doi.org/10.48550/arXiv.2505.16100),[Link](https://arxiv.org/abs/2505.16100)Cited by:[§1\.1](https://arxiv.org/html/2608.25286#S1.SS1.p3.1)\.
- Wijket al\.\(2025\)H\. Wijk, T\. R\. Lin, J\. Becker, S\. Jawhar, N\. Parikh, T\. Broadley, L\. Chan, M\. Chen, J\. M\. Clymer, J\. Dhyani, E\. Ericheva, K\. Garcia, B\. Goodrich, N\. Jurkovic, M\. Kinniment, A\. Lajko, S\. Nix, L\. J\. Koba Sato, W\. Saunders, M\. Taran, B\. West, and E\. BarnesRE\-bench: evaluating frontier AI r&d capabilities of language model agents against human experts\.InProceedings of the 42nd International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.267,pp\. 66772–66832\.External Links:[Link](https://proceedings.mlr.press/v267/wijk25a.html)Cited by:[§1\.1](https://arxiv.org/html/2608.25286#S1.SS1.p2.1)\.
- Xieet al\.\(2024\)T\. Xie, D\. Zhang, J\. Chen, X\. Li, S\. Zhao, R\. Cao, J\. H\. Toh, Z\. Cheng, D\. Shin, F\. Lei, Y\. Liu, Y\. Xu, S\. Zhou, S\. Savarese, C\. Xiong, V\. Zhong, and T\. YuOSWorld: benchmarking multimodal agents for open\-ended tasks in real computer environments\.InAdvances in Neural Information Processing Systems,Vol\.37,pp\. 52040–52094\.Note:Datasets and Benchmarks TrackExternal Links:[Document](https://dx.doi.org/10.52202/079017-1650)Cited by:[§1\.1](https://arxiv.org/html/2608.25286#S1.SS1.p2.1)\.
- Xinet al\.\(2024\)Q\. Xin, Q\. Kong, H\. Ji, Y\. Shen, Y\. Liu, Y\. Sun, Z\. Zhang, Z\. Li, X\. Xia, B\. Deng, and Y\. BaiBioInformatics Agent \(BIA\): unleashing the power of large language models to reshape bioinformatics workflow\.bioRxiv\.External Links:[Document](https://dx.doi.org/10.1101/2024.05.22.595240)Cited by:[§1](https://arxiv.org/html/2608.25286#S1.p1.1)\.
- Zagozewskiet al\.\(2022\)J\. Zagozewski, S\. Borlase, B\. J\. Guppy, L\. Coudière\-Morrison, G\. M\. Shahriary, V\. Gordon, L\. Liang, S\. Cheng, C\. J\. Porter, R\. Kelley, C\. Hawkins, J\. A\. Chan, Y\. Liang, J\. Gong, C\. Nör, O\. Saulnier, R\. J\. Wechsler\-Reya, V\. Ramaswamy, and T\. E\. Werbowetski\-OgilvieCombined MEK and JAK/STAT3 pathway inhibition effectively decreases SHH medulloblastoma tumor progression\.Communications Biology5,pp\. 697\.External Links:[Document](https://dx.doi.org/10.1038/s42003-022-03654-9)Cited by:[Table 1](https://arxiv.org/html/2608.25286#A1.T1.4.9.3.1.1)\.
- Zhanget al\.\(2025\)A\. L\. Zhang, T\. Kraska, and O\. KhattabRecursive language models\.External Links:2512\.24601,[Document](https://dx.doi.org/10.48550/arXiv.2512.24601),[Link](https://arxiv.org/abs/2512.24601)Cited by:[§3\.1](https://arxiv.org/html/2608.25286#S3.SS1.p2.1)\.
- Zhouet al\.\(2025\)J\. Zhou, J\. Jiang, Z\. Han, Z\. Wang, and X\. GaoStreamline automated biomedical discoveries with agentic bioinformatics\.Briefings in Bioinformatics26\(5\),pp\. bbaf505\.External Links:[Document](https://dx.doi.org/10.1093/bib/bbaf505),[Link](https://pubmed.ncbi.nlm.nih.gov/41016012/)Cited by:[§1](https://arxiv.org/html/2608.25286#S1.p1.1)\.
- Zhouet al\.\(2024\)S\. Zhou, F\. F\. Xu, H\. Zhu, X\. Zhou, R\. Lo, A\. Sridhar, X\. Cheng, T\. Ou, Y\. Bisk, D\. Fried, U\. Alon, and G\. NeubigWebArena: a realistic web environment for building autonomous agents\.InThe Twelfth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=oKn9c6ytLx)Cited by:[§1\.1](https://arxiv.org/html/2608.25286#S1.SS1.p2.1)\.

## Supplementary Material

![Refer to caption](https://arxiv.org/html/2608.25286v1/figures/supplementary_figure5_artifact_scores_by_task.png)Supplementary Figure 1:Artifact scores by task and model\.Each panel shows the artifacts for one of the 20BixBench3tasks\. Artifacts are numbered within each task, colored points show the programmatic artifact score for each of the 13 evaluated models, and grey boxplots summarize the artifact score distribution across models\.![Refer to caption](https://arxiv.org/html/2608.25286v1/figures/supplementary_figure3_expert_programmatic_score.png)Supplementary Figure 2:Comparison of expert ratings and artifact scores\.The figure shows programmatic artifact scores for 25 artifacts produced by GLM 5\.2 and rated by domain experts, grouped by expert reproduction rating\. Boxes show the interquartile range, center lines show medians, whiskers extend to 1\.5 times the interquartile range, and points represent individual artifacts\.![Refer to caption](https://arxiv.org/html/2608.25286v1/figures/supplementary_figure4_model_score_spearman.png)Supplementary Figure 3:Similarity of model task\-score profiles\.Pairwise Spearman correlations of per\-task scores across the 20BixBench3tasks\. Rows and columns are ordered by overall mean task score\. Asterisks indicate two\-sided significance levels:∗\*p<0\.05p<0\.05,∗⁣∗\*\*p<0\.01p<0\.01,∗∗∗\*\*\*p<0\.001p<0\.001\. Dashes indicate undefined correlations\.![Refer to caption](https://arxiv.org/html/2608.25286v1/figures/supplementary_figure1_category_heatmaps.png)Supplementary Figure 4:Task\-level performance by data and assay type\.\(A\) Mean task score by model and broad data type\. \(B\) Mean task score by model and specific assay type\. Parenthetical values give the number of tasks in each category, and tasks may contribute to multiple categories\.![Refer to caption](https://arxiv.org/html/2608.25286v1/figures/supplementary_figure2_token_usage.png)Supplementary Figure 5:Token usage\.\(A\) Mean total processed tokens, defined as the sum of full\-rate input tokens, provider\-reported cache\-write input tokens, cache\-read input tokens, and output tokens\. \(B\) Mean tokens separated by token category\. For GLM 5\.2, dark and light grey bars show the non\-cache\-adjusted composition and the cache\-adjusted composition after rescaling input tokens to the mean cache hit rate of the other models, respectively\. Both panels use logarithmic axes, and error bars denote the standard error\. Zero cache\-write values reflect provider reporting rather than the absence of cache population\.
## Appendix AAppendix

### A\.1Source papers

Table 1:Source papers for the 20BixBench3tasks\.TaskPaper titleCitation1The chromatin remodeler LET\-418/Mi\-2 regulates the intracellular pathogen response in theC\. elegansintestine\([Rajopadhye et al\., 2025](https://arxiv.org/html/2608.25286#bib.bib35)\)2Contact\-dependent regulation of UV\-B/C\-induced cell fate by neighbouring intact cells\([Budimir et al\., 2026](https://arxiv.org/html/2608.25286#bib.bib36)\)3An interneuronal CRH and CRHBP circuit stabilizes birdsong performance\([Colquitt and Brainard, 2025](https://arxiv.org/html/2608.25286#bib.bib37)\)4How many are you? Open data and bioinformatics reveal species misidentification and potential introgression inChordodes\(Phylum Nematomorpha\)\([De Vivo, 2026](https://arxiv.org/html/2608.25286#bib.bib38)\)5A conserved small RNA\-generating gene cluster undergoes sequence diversification and contributes to plant immunity\([Feng et al\., 2025](https://arxiv.org/html/2608.25286#bib.bib39)\)6MitoPerturb\-Seq identifies common and gene\-specific single\-cell responses to mitochondrial DNA depletion and heteroplasmy\([Burr et al\., 2025](https://arxiv.org/html/2608.25286#bib.bib40)\)7Methylomic signatures of tau and amyloid\-beta in transgenic mouse models of Alzheimer’s disease neuropathology\([Leung et al\., 2025](https://arxiv.org/html/2608.25286#bib.bib41)\)8Combined MEK and JAK/STAT3 pathway inhibition effectively decreases SHH medulloblastoma tumor progression\([Zagozewski et al\., 2022](https://arxiv.org/html/2608.25286#bib.bib42)\)9Targeting of REST with rationally\-designed small molecule compounds exhibits synergetic therapeutic potential in human glioblastoma cells\([Panina et al\., 2024](https://arxiv.org/html/2608.25286#bib.bib43)\)10Structure\-guided design of a selective inhibitor of the methyltransferase KMT9 with cellular activity\([Wang et al\., 2024](https://arxiv.org/html/2608.25286#bib.bib44)\)11Transcriptomic analyses of livers from mice exposed to 1,4\-dioxane for up to 90 days to assess potential mode\(s\) of action underlying liver tumor development\([Chappell et al\., 2021](https://arxiv.org/html/2608.25286#bib.bib45)\)12A microbiota\-derived bile acid modulates biofilm formation by the probiotic strainEscherichia coliNissle 1917\([Perry et al\., 2025](https://arxiv.org/html/2608.25286#bib.bib46)\)13High\-resolution retrospective single cell lineage tracing with mutable homopolymers\([Cheng et al\., 2026b](https://arxiv.org/html/2608.25286#bib.bib47)\)14RUNX2 inhibition disrupts a PAX3::FOXO1\-RUNX2 feed\-forward loop and dismantles oncogenic gene programs in fusion\-positive rhabdomyosarcoma\([Mendes et al\., 2025](https://arxiv.org/html/2608.25286#bib.bib49)\)15TDP\-43 dysfunction leads to the accumulation of cryptic transposable element\-derived exons, crypTEs, in iPSC derived neurons and ALS/FTD patient tissues\([Bolger et al\., 2026](https://arxiv.org/html/2608.25286#bib.bib50)\)16Antifungal exposure can enhanceCandida glabratapathogenesis\([Ribeiro et al\., 2025](https://arxiv.org/html/2608.25286#bib.bib51)\)17Polycomb repression works without Siesta\([Kahn et al\., 2025](https://arxiv.org/html/2608.25286#bib.bib52)\)18A glycosylation\-dependent checkpoint restrains intestinal intra\-epithelial lymphocyte activation\([Cheng et al\., 2026a](https://arxiv.org/html/2608.25286#bib.bib53)\)19A TCER\-1\-siRNA regulatory axis suppresses antibacterial innate immunity inC\. elegans\([Naim et al\., 2026](https://arxiv.org/html/2608.25286#bib.bib48)\)20Habitat fragmentation controls bacterial community composition outcomes\([Batsch et al\., 2026](https://arxiv.org/html/2608.25286#bib.bib54)\)
### A\.2Claude Opus 5 output\-format errors

Claude Opus 5 ranked seventh overall \(score 0\.406\), largely because it violated the specified artifact\-output formats on Tasks 5, 19, and 20\. For example, on Task 5, it replaced column names specified in the artifact output contract \(col0\_gene\_id,col0\_tpm, etc\.\) with alternative labels \(Col\-0,Ct\-1, etc\.\), causing the programmatic artifact grader to treat those columns as missing\. These were instruction\-following errors that most other models did not make\. When scores from those three tasks were excluded from every model’s average, Opus 5 had the second\-highest score \(0\.455\), narrowly behind GPT 5\.6 Sol \(0\.458\)\.

### A\.3Expert review of tasks

We selected 25 artifacts for review by domain experts to validate the grading and task structure \(Supplementary Figure[2](https://arxiv.org/html/2608.25286#Sx4.F2)\)\. These artifacts were drawn from 171 artifacts produced by GLM 5\.2 and selected to span the range of artifact scores\. Experts assigned each agent\-produced artifact a reproduction rating from 1 to 5 based on how faithfully it recovered the biological meaning of the corresponding published artifact \(Table[2](https://arxiv.org/html/2608.25286#A1.T2)\)\. Expert ratings were strongly associated with the continuous artifact scores \(Spearmanρ=0\.76\\rho=0\.76,p=1\.3×10−5p=1\.3\\times 10^\{\-5\}; Supplementary Figure[2](https://arxiv.org/html/2608.25286#Sx4.F2)\)\. Expert feedback was additionally used to refine the task prompts, method guidance, and grading for these tasks\.

Table 2:Expert reproduction rating definitions\.Domain experts assigned each reviewed artifact an integer rating from 1 to 5 using these criteria\.
### A\.4Process\-judge failure modes

Table 3:Definitions of the closed\-vocabulary failure modes assigned by the process judge\.Failure modeDescriptionEnvironment setup failureThe agent could not get required tools, packages, references, or paths into a usable state\. The tag is used when that unresolved setup problem prevents or materially blocks the intended analysis\.Input misinterpretationThe agent misread the staged inputs, confused sample identities, or applied an analysis intended for a different data type\. It captures an incorrect interpretation of the data supplied to the run, rather than a later choice of analysis parameters\.Wrong method substitutionThe agent used a method materially different from the one required by Method Guidance, such as a different tool family or statistical approach\. The substitution was not permitted by the prompt\.Method misconfiguredThe agent used the required method but chose parameters, contrasts, references, or filters that materially diverged from Method Guidance\. The method itself is correct, which distinguishes this failure from wrong method substitution\.Incomplete dataThe agent analyzed fewer samples, conditions, or features than the prompt specified\. The omitted data leave the analysis incomplete even if the reduced analysis ran successfully\.Output format violationThe agent produced an artifact that did not match the Required Outputs schema\. Violations include the wrong filename, location, index column, required columns, data types, or units\.Synthetic or placeholder outputThe agent wrote synthetic, dummy, empty, or formulaic values instead of deriving output from the real staged data\. The tag applies whether fabrication is explicit or used to conceal a failed or incomplete analysis\.Premature terminationThe agent stopped before writing all required outputs because it exhausted a turn budget or time limit, or because it declared completion before the work was finished\. At least one required artifact therefore remained unwritten\.Inefficient analysisThe agent failed to use available time productively\. Examples include ordering work inefficiently, missing obvious parallelization, omitting available parallel flags, or unnecessarily repeating completed steps\.Repetitive retry loopThe agent repeatedly retried substantially the same failing command or approach\. The retries lacked a meaningful diagnosis or change in strategy\.
### A\.5Example task prompt

The following is the complete prompt for the task shown in Figure[6](https://arxiv.org/html/2608.25286#S4.F6):

\#AnalysisTask

Youareacomputationalbiologistperformingananalysisendtoend\.Completetheentiretaskandproducetherequestedoutputfiles\.

\#\#WorkingDirectory

\-Inputdataandreferencefilesareprovidedunder‘\./data‘\.Youmayuseotherdatafromthewebasneeded\.

\-The‘\./data‘directoryisread\-only\.Ifyouneedtomodify,decompress,index,orotherwiserewriteaninputfile,copyitto‘\./work‘first\.

\-Writeallanalysisoutputsunder‘\./outputs‘\.

\-Youmaycreateintermediatefilesunder‘\./work‘ifuseful\.

\#\#RuntimeLimit

Youhaveamaximumwall\-clockbudgetof‘86400‘secondsforthistask\.Therunwillbeterminatedwhenthisbudgetisreached\.

\#\#AlreadyAvailableSoftware

Thesandboximagealreadyincludesthefollowinggeneral\-purposesoftware:

\-Buildtooling:‘build\-essential‘,‘gfortran‘,‘make‘,‘cmake‘,‘pkg\-config‘\.Compilersandbuildhelpersforsourcepackageinstallation\.

\-Archiveanddownloadutilities:‘wget‘,‘curl‘,‘aria2‘,‘rsync‘,‘unzip‘,‘zip‘,‘tar‘,‘pigz‘,‘jq‘\.Commoncommand\-linetoolsforfetching,unpacking,andinspectingfiles\.

\-Coredevelopmentlibraries:‘libcurl4\-openssl\-dev‘,‘libssl\-dev‘,‘libxml2\-dev‘,‘zlib1g\-dev‘,‘libbz2\-dev‘,‘liblzma\-dev‘,‘libzstd\-dev‘,‘libhdf5\-dev‘,‘libgit2\-dev‘\.HeaderlibrariesusedbycommonPython,R,andbioinformaticspackages\.

\-Plot,font,andimagelibraries:‘libpng\-dev‘,‘libjpeg\-dev‘,‘libtiff\-dev‘,‘libfreetype6\-dev‘,‘libfontconfig1\-dev‘,‘libharfbuzz\-dev‘,‘libfribidi\-dev‘,‘libcairo2\-dev‘\.Graphicslibrariesusedbyplottingandreportingpackages\.

\-Java:‘openjdk\-17\-jre\-headless‘\.JavaruntimeforJava\-basedbioinformaticstoolsandworkflowhelpers\.

\-Python:‘python3‘,‘python3\-pip‘,‘uv‘,‘numpy‘,‘pandas‘,‘scipy‘,‘scikit\-learn‘,‘statsmodels‘,‘pyarrow‘,‘h5py‘,‘matplotlib‘,‘biopython‘,‘pysam‘\.Pythonruntime,packageinstaller,scientificstack,andcorebioinformaticslibraries\.

\-R:‘r\-base‘,‘r\-base\-dev‘,‘BiocManager‘,‘data\.table‘,‘dplyr‘,‘readr‘,‘tidyr‘,‘ggplot2‘,‘Matrix‘,‘remotes‘,‘pak‘\.Rruntime,buildsupport,packageinstallers,andcommondata\-wranglingpackages\.

\-Conda\-compatiblepackagemanagement:‘micromamba‘,‘mamba‘\.Micromambaisconfiguredwithconda\-forge,bioconda,anddefaultschannels\.

\-10xGenomicssingle\-celltools:‘cellranger6\.0\.2‘,‘cellranger\-arc2\.0\.2‘\.

Method\-specificanalysispackagesarenotpreinstalledunlesslistedhere;installthetoolsandversionsrequiredbythemethodguidancewhenneeded\.

\#\#NetworkandExternalResources

Youhavenetworkaccessthroughanallow\-listproxy\.Useitfreelytoinstallpackagesandpullcanonicalreferencedataandmethodtools\.Leantowardfetchingwhatyouneedratherthanguessing\.

\#\#\#Pre\-approved\(justinstallorfetch,noapprovalcallneeded\)

\-\*\*Pythonpackages\*\*:‘pipinstall<pkg\>‘fromPyPI\(‘pypi\.org‘,‘files\.pythonhosted\.org‘\)\.

\-\*\*Condapackages\*\*:‘condainstall\-cconda\-forge\-cbioconda<pkg\>‘or‘mambainstall\.\.\.‘\(‘conda\.anaconda\.org‘,‘repo\.anaconda\.com‘,‘anaconda\.org‘\)\.

\-\*\*Rpackages\*\*:‘install\.packages\("<pkg\>"\)‘fromCRAN\(‘cran\.r\-project\.org‘,‘cloud\.r\-project\.org‘,‘packagemanager\.posit\.co‘\)and‘BiocManager::install\("<pkg\>"\)‘fromBioconductor\(‘bioconductor\.org‘\)\.

\-\*\*OSpackages\*\*:‘apt\-getinstall<pkg\>‘fromDebianandUbuntumirrors\.

\-\*\*Otherlanguageregistries\*\*:npm\(‘registry\.npmjs\.org‘\),Cargo\(‘crates\.io‘\),Gomodules\(‘proxy\.golang\.org‘\),RubyGems\.

\-\*\*Astraltools\*\*:‘uv‘,‘ruff‘,etc\.\(‘astral\.sh‘\)\.

Alwaystrythepackagemanagerfirst\.Onlyescalateto‘request\_web\_access‘ifthetoolordatayouneedisnotavailablethroughapackagingsystem\.

\#\#\#Approvedonrequestvia‘request\_web\_access\(url,reason\)‘

Whenyouneedaresourceoutsidethepre\-approvedinfrastructure,call‘request\_web\_access‘withtheexactURLandashortjustification\.ThefollowingclassesofURLsareroutinelyapproved:

\-\*\*Method\-toolsourcecodeonGitHub/GitLab/Bitbucket\*\*:arepositoryrootlike‘https://github\.com/<org\>/<repo\>‘,areleasetarball,orarawfilefromatoolyouneedtorun\.BarerepositoryURLsareclonedintoyourworkspace;specificreleaseURLsarefetchedasfiles\.

\-\*\*Canonicalpublicreferencedata\*\*fromprimaryproviders,forexample:

\-EnsemblFTP\(‘ftp\.ensembl\.org‘\)forgenomeFASTA,GTF/GFF,cDNA\.

\-GENCODE\(‘ftp\.ebi\.ac\.uk/pub/databases/gencode‘\)\.

\-UCSCgoldenPath\(‘hgdownload\.soe\.ucsc\.edu‘\)forFASTA,chromsizes,blacklists\.

\-NCBIGenome/RefSeqFTP\(‘ftp\.ncbi\.nlm\.nih\.gov‘\)\.

\-UniProt\(‘rest\.uniprot\.org‘,‘ftp\.uniprot\.org‘\)forproteinFASTAandmetadata\.

\-JASPAR,MSigDB,Pfam,InterPro,KEGGREST,OBOFoundry,andsimilarcanonicalbioinformaticsresources\.

\-\*\*Tooldocumentationpages,vignettes,manuals,READMEs,andAPIreferences\*\*whenyouneedtolookupcorrectusage\.

\-\*\*Specificraw\-readaccessionfiles\*\*fromSRA/ENAwhenthestageddatadoesnotalreadyincludethem\(onlythespecificfileURL—notasearchpage\)\.

Whenthebrokerapproves,branchontheresponse:

\-‘delivery:"allowlist"‘—thehostwasaddedtotheproxyallowlist;rerunyouroriginalfetchcommand\.

\-‘delivery:"workspace\_file"‘—thebrokerdownloadedtheresourceforyou;readitfrom‘local\_path‘\(landsunder‘/workspace/approved\_web/‘\)\.

\#\#\#Neverapproved—donotspendbudgetasking

\-PDFs,abstracts,supplements,source\-dataworkbooks,extended\-datafiles,oranyotheranswer\-bearingartifactforpublishedpapers\.

\-Processedresultmatrices,normalizedmatrices,cell\-typelabels,clusterassignments,differential\-expressiontables,peaklists,orsimilarprecomputedoutputsforpublisheddatasets—evenifhostedonGEO,ArrayExpress,Synapse,CellxGene,Zenodo,Figshare,OSF,Dryad,GoogleDrive,orDropbox\.

\-Broadsearch\-enginequeries\(Google,Bing\)containingpaper\-specifictermsoraccessions\.

\#\#Conduct

\-Usethetools,parameters,thresholds,anddatabasesnamedunder"MethodGuidance"\.

\-Whenadetailisunspecified,chooseareasonabledefaultthatisconsistentwiththenamedtoolsanddocumentitbrieflyinatop\-levelcommentoftheanalysisscriptthatusesit\.

\-Preferscripted,reproduciblestepsoverinteractiveones\.Pinsoftwareversionswhenyoucan\.

\-Containallyourworkto‘\./data‘,‘\./work‘,and‘\./outputs‘\.

\-Beefficientwithyouruseoftoolsandresources\.Donotpolllongrunningcommandsunnecessarilyfrequently\.Youcanrunlongrunningcommandsinthebackgroundorsleepbetweenchecks\.

\#\#OutputGuidance

The"RequiredOutputs"sectionbelowspecifiestheexactfilestoproduce\.Foreachfile:

\-Writetotheexactrelativepathgiven\.

\-Usetheexactformatrequested\(CSV,TSV,BED,etc\.\)\.

\-Includeaheaderrowwhentheformatistabular\.

\-Usetheexactcolumnnameslistedwhennamesaregiven\.Whenonlyadescriptionisgiven,choosestable,descriptivecolumnnamesthatmatchthedescription\.

\-Placetherow\-keycolumnasthefirstcolumn\.

\-Includeonerowperrequestedentity\.Donotaddsummary,totals,orcommentaryrows\.

\-Donotwritenotes,narrative,ormarkdownintodatafiles\.

\-UseUTF\-8textencodingandUnixnewlines\.

\#\#\#‘METHODS\.md‘\(required\)

Write‘\./outputs/METHODS\.md‘describinghoweachrequiredartifactwasproduced,thetoolsandparametersused,andthedatasourcesandintermediatefilescreated\.Writethisinthestyleofthemethodssectionofascientificpaper\.Useone‘\#\#\{artifact\_id\}‘sectionperrequiredartifact\.Foreachartifact,includethetools,versions,parameters,inputfilesunder‘\./data‘,intermediatefilesunder‘\./work‘,andoutputfilewrittenunder‘\./outputs‘\.Explicitlydiscloseanysubstitutions,skippedsteps,differenttoolversions,fallbackmethods,failedcommands,partialoutputs,orassumptions\.Thisfileshouldbesufficientlydetailedforanotherresearchertoexactlyreproducetheanalysisfromscratch\.

\#\#ResearchObjective

CharacterizehowlossofTCER\-1changesendogenoussmall\-RNAregulationandmRNAexpressionin\*Caenorhabditiselegans\*,withemphasisonWAGO\-associated22G\-RNAsandtheirtargettranscriptsduringadultgrowthonnon\-pathogenic\*E\.coli\*OP50\.

‘\./data‘containsrawsequencinginputsforthreebiologicalreplicateseachofwild\-typeN2,‘tcer\-1\(glm27\)‘,and‘tcer\-1\(tm1452\)‘gravidadultanimals\.ThemRNAlibrariesarerRNA\-depleted,directional,paired\-end150\-cycleRNA\-seqFASTQs,andthesmall\-RNAlibrariesarepolyphosphatase\-treated,16\-30ntsize\-selected,single\-end75\-cycleFASTQs\.‘\./data/metadata‘containsasamplesheetwithcondition,replicate,librarytype,andlayoutassignments,shortpipelinenotes,anexactWormBaseWS279PRJNA13758mRNA/RSEMreferencebundleunder‘\./data/metadata/reference/wormbase\_ws279\_mrna/‘,andaMontgomeryLabtinyRNAWS279referencebundleunder‘\./data/metadata/reference/montgomery\_tinyrna\_ws279/‘\.QuantifymRNAexpressionandsmall\-RNAabundance,compareeach‘tcer\-1‘allelewithN2,identifyconcordantat\-least\-twofoldchangesacrossbothalleles,andintegrateWAGO\-class22G\-RNAchangeswithpairedmRNAchangesforeachallele\.

\#\#MethodGuidance

Usethestagedsamplesheettoassignlibrariesto‘N2‘,‘tcer\-1\(glm27\)‘,and‘tcer\-1\(tm1452\)‘groups,keepingthreebiologicalreplicatespergroupandseparatemRNA\-seqandsmall\-RNA\-seqdesigns\.TreatN2asthereferenceconditionforalldifferentialanalyses\.Foreverylog2foldchange,usemutantoverN2direction:positivevaluesindicatehigherexpressionorabundanceinthemutant,andnegativevaluesindicatehigherexpressionorabundanceinN2\.

FormRNA\-seq,trimeachpairedlibrarywith‘fastp‘using‘fastp\-w16\-q30\-u70\-l30\-r\-W4\-M20‘\.Buildthe‘RSEMv1\.3\.1‘referencefromtheexactstagedWormBaseWS279PRJNA13758mRNAfiles:‘\./data/metadata/reference/wormbase\_ws279\_mrna/c\_elegans\.PRJNA13758\.WS279\.genomic\.fa\.gz‘and‘\./data/metadata/reference/wormbase\_ws279\_mrna/c\_elegans\.PRJNA13758\.WS279\.canonical\_geneset\.gtf\.gz‘,using‘rsem\-prepare\-reference\-\-gtf‘with‘\-\-star‘soWormBasegeneIDsarethegene\-levelidentifiers,andquantifywith‘STARv2\.7\.10b‘asthealigner\.Summarizetothegene\-levelrowsemittedbyRSEMexpected\-countquantification,retainingWormBasegeneidentifiersandgenenameswhereavailable\.Analyzethecountmatrixwith‘DESeq2v1\.50\.2‘,usingmedian\-of\-ratiossize\-factornormalization,acondition\-onlydesignwithN2asthereferencelevel,Wald\-statisticinference,andBenjamini\-Hochbergadjustedp\-values\.Extractthetwomutant\-versus\-N2contrasts,reportDESeq2\-normalizedcountsfortheRSEMgene\-levelrowuniverse,anduseabsolutelog2foldchangeofatleast1withadjustedp\-valuebelow0\.05whendefininggeneschangedconcordantlyacrossbothmutantalleles\.

Forsmall\-RNA\-seq,processtheninesingle\-endlibrarieswiththe‘tinyRNA‘pipelineusingdefaultsettingsagainstthestagedMontgomeryLabWS279tinyRNAbundle:‘\./data/metadata/reference/montgomery\_tinyrna\_ws279/c\_elegans\.PRJNA13758\.WS279\.genomic\.fa\.gz‘,‘\./data/metadata/reference/montgomery\_tinyrna\_ws279/sRNAs\_WS279\.gff3‘,and‘\./data/metadata/reference/montgomery\_tinyrna\_ws279/features\_cel\_v1\.5\.csv‘\.Usethelibrarydesignas16\-30nt,polyphosphatase\-treatedsmallRNA,anddefine22G\-RNAsasantisense\-strand21\-23ntreadscontaininga5\-primeG\.CarrythetinyRNAfeatureuniversethroughdownstreamtables,includingmiRNAMIMATrows,sequence\-level23H\-RNArows,andclass\-specificWBGene22Grows\.RunDESeq2within‘tinyRNA‘withthesameN2reference,mutant\-versus\-N2contrastdirection,Wald\-statisticinference,andBenjamini\-Hochbergadjustment\.UseDESeq2size\-factornormalizedsmall\-RNAcountsandthesametwofoldandadjusted\-p\-valuecriteriaforconcordantallele\-levelchanges\.Whenasmall\-RNAfeaturemapstothesamegeneinmultipleclasses,keepthefeatureclassandfeaturenameattachedsodistinctsmall\-RNAclassesremainseparate\.

Fortheintegrated22G\-RNA/mRNAanalysis,run‘RNA\-integrate‘onthecounttablesgeneratedfromthe‘tinyRNA‘and‘RSEM‘workflows,withmetadatamarkingN2asthecontrolconditionandboth‘tcer\-1\(tm1452\)‘versusN2and‘tcer\-1\(glm27\)‘versusN2ascross\-comparisons\.Use‘\./data/metadata/reference/montgomery\_tinyrna\_ws279/sRNAs\_WS279\.gff3‘astheWAGOtarget\-pairingsourceand‘\./data/metadata/reference/montgomery\_tinyrna\_ws279/features\_cel\_v1\.5\.csv‘asthetinyRNAclass\-rulesourceforthegeneclass/nameinformationrequiredby‘RNA\-integrate‘\.RestrictintegratedsummariestoannotatedWAGO22G\-RNAtargetgenes\.Foreachalleleseparately,comparethesmall\-RNAandmRNAlog2mutant\-over\-N2effects,andderiveanticorrelatedgroupswithsmallRNAdownandmRNAupmorethantwofold,aswellasthereciprocalsmallRNAupandmRNAdownmorethantwofold,usingadjustedp\-valuebelow0\.05forsignificanceclassification\.

\#\#RequiredOutputs

\#\#\#‘mrna\_normalized\_counts‘

\-Path:‘outputs/mrna\_normalized\_counts\.csv‘

\-Format:‘csv‘

\-Description:Geometricmean/DESeq2\-stylenormalizedmRNA\-seqcountsforC\.elegansgenefeaturesacrosswild\-typeN2,tcer\-1\(glm27\),andtcer\-1\(tm1452\)biologicalreplicates\.

\-Rowkey:‘Feature\_ID‘

\-Rows:Onerowpergene\-levelRSEMv1\.3\.1expected\-countentryidentifiedbyWormBasegeneIDafterquantifyingagainsttheexactstagedWormBaseWS279PRJNA13758RSEMreferencebuiltfrom‘wormbase\_ws279\_mrna/c\_elegans\.PRJNA13758\.WS279\.genomic\.fa\.gz‘and‘wormbase\_ws279\_mrna/c\_elegans\.PRJNA13758\.WS279\.canonical\_geneset\.gtf\.gz‘\.

\-Examplerowidentifiers:‘WBGene00000001‘,‘WBGene00000002‘,‘WBGene00000003‘

\-Columns:Identifierandgene\-namecolumnsfollowedbynormalizedcountcolumnsforthreeN2,threetcer\-1\(glm27\),andthreetcer\-1\(tm1452\)replicates\.

\-Requiredcolumns:‘Feature\_ID‘,‘Feature\_Name‘,‘N2\_rep\_1‘,‘N2\_rep\_2‘,‘N2\_rep\_3‘,‘tcer\-1\(glm27\)\_rep\_1‘,‘tcer\-1\(glm27\)\_rep\_2‘,‘tcer\-1\(glm27\)\_rep\_3‘,‘tcer\-1\(tm1452\)\_rep\_1‘,‘tcer\-1\(tm1452\)\_rep\_2‘,‘tcer\-1\(tm1452\)\_rep\_3‘

\#\#\#‘srna\_normalized\_counts‘

\-Path:‘outputs/srna\_normalized\_counts\.csv‘

\-Format:‘csv‘

\-Description:Normalizedsmall\-RNAabundancematrixforC\.eleganssmall\-RNAfeaturesacrosswild\-typeN2,tcer\-1\(glm27\),andtcer\-1\(tm1452\)biologicalreplicates\.UseFeature\_KeyasFeature\_ID\|Classifier\|Feature\_NamebecauseFeature\_IDaloneisnotuniqueforthissmall\-RNAtable\.

\-Rowkey:‘Feature\_Key‘

\-Rows:OnerowpertinyRNA\-testedsmall\-RNAfeatureemittedfromthestagedMontgomeryLabWS279tinyRNAbundle,keyedbytheexactconcatenationFeature\_ID\|Classifier\|Feature\_Name\.ThefeatureuniverseincludesmiRNAMIMATrows,sequence\-level23H\-RNArows,andclass\-specificWBGene22Grows\.

\-Examplerowidentifiers:‘AAAAAAGAACTGAAGAGAGTG\|23H\-RNA\|C05E7\.1,C05E7\.t3‘,‘MIMAT0000001\|miRNA\|let\-7\-5p‘,‘WBGene00001335\|WAGOClass22G\|F35A5\.8‘

\-Columns:CompositekeyandfeatureannotationcolumnsfollowedbynormalizedcountcolumnsforthreeN2,threetcer\-1\(glm27\),andthreetcer\-1\(tm1452\)replicates\.

\-Requiredcolumns:‘Feature\_Key‘,‘Feature\_ID‘,‘Classifier‘,‘Feature\_Name‘,‘N2\_rep\_1‘,‘N2\_rep\_2‘,‘N2\_rep\_3‘,‘tcer\-1\(glm27\)\_rep\_1‘,‘tcer\-1\(glm27\)\_rep\_2‘,‘tcer\-1\(glm27\)\_rep\_3‘,‘tcer\-1\(tm1452\)\_rep\_1‘,‘tcer\-1\(tm1452\)\_rep\_2‘,‘tcer\-1\(tm1452\)\_rep\_3‘

\#\#\#‘srna\_differential\_expression\_all‘

\-Path:‘outputs/srna\_differential\_expression\_all\.csv‘

\-Format:‘csv‘

\-Description:Alltestedsmall\-RNAfeaturesfromtheDESeq2\-styleanalysis\.Small\-RNAdifferentialexpressionvaluescompareeachtcer\-1mutantagainstwild\-typeN2;log2FC\_tm1452\_vs\_wtislog2\(tcer\-1\(tm1452\)/wildtype\)andlog2FC\_glm27\_vs\_wtislog2\(tcer\-1\(glm27\)/wildtype\),sopositivevaluesindicatehigherabundanceinthemutantandnegativevaluesindicatehigherabundanceinwildtype\.

\-Rowkey:‘Feature\_Key‘

\-Rows:OnerowpertinyRNA\-testedsmall\-RNAfeatureemittedfromthestagedMontgomeryLabWS279tinyRNAbundle,keyedbytheexactconcatenationFeature\_ID\|Classifier\|Feature\_Name\.ThefeatureuniverseincludesmiRNAMIMATrows,sequence\-level23H\-RNArows,andclass\-specificWBGene22Grows\.

\-Examplerowidentifiers:‘AAAAAAGAACTGAAGAGAGTG\|23H\-RNA\|C05E7\.1,C05E7\.t3‘,‘MIMAT0000001\|miRNA\|let\-7\-5p‘,‘WBGene00001335\|WAGOClass22G\|F35A5\.8‘

\-Columns:Compositerowkey,originalfeatureannotations,meanreads,adjustedpvalues,andallele\-specificlog2foldchangeswithmutant\-over\-wild\-typedirection\.

\-Requiredcolumns:‘Feature\_Key‘,‘Feature\_ID‘,‘Classifier‘,‘Feature\_Name‘,‘mean\_reads‘,‘log2FC\_tm1452\_vs\_wt‘,‘padj\_tm1452\_vs\_wt‘,‘log2FC\_glm27\_vs\_wt‘,‘padj\_glm27\_vs\_wt‘

\#\#\#‘srna\_down\_twofold\_both\_mutants‘

\-Path:‘outputs/srna\_down\_twofold\_both\_mutants\.csv‘

\-Format:‘csv‘

\-Description:Small\-RNAfeaturesreportedasdownatleasttwo\-foldinbothtcer\-1mutants\.Small\-RNAdifferentialexpressionvaluescompareeachtcer\-1mutantagainstwild\-typeN2;log2FC\_tm1452\_vs\_wtislog2\(tcer\-1\(tm1452\)/wildtype\)andlog2FC\_glm27\_vs\_wtislog2\(tcer\-1\(glm27\)/wildtype\),sopositivevaluesindicatehigherabundanceinthemutantandnegativevaluesindicatehigherabundanceinwildtype\.

\-Rowkey:‘Feature\_Key‘

\-Rows:OnerowpertinyRNA\-testedsmall\-RNAfeaturefromthestagedMontgomeryLabWS279tinyRNAbundlethatmeetsthisartifact’stwo\-allelefold\-changeandadjusted\-p\-valuerule,keyedbyFeature\_ID\|Classifier\|Feature\_Name\.

\-Examplerowidentifiers:‘AAAAAAGAACTGAAGAGAGTG\|23H\-RNA\|C05E7\.1,C05E7\.t3‘,‘MIMAT0000001\|miRNA\|let\-7\-5p‘,‘WBGene00001335\|WAGOClass22G\|F35A5\.8‘

\-Columns:Compositerowkey,originalfeatureannotations,meanreads,adjustedpvalues,andallele\-specificlog2foldchangeswithmutant\-over\-wild\-typedirection\.

\-Requiredcolumns:‘Feature\_Key‘,‘Feature\_ID‘,‘Classifier‘,‘Feature\_Name‘,‘mean\_reads‘,‘log2FC\_tm1452\_vs\_wt‘,‘padj\_tm1452\_vs\_wt‘,‘log2FC\_glm27\_vs\_wt‘,‘padj\_glm27\_vs\_wt‘

\#\#\#‘srna\_up\_twofold\_both\_mutants‘

\-Path:‘outputs/srna\_up\_twofold\_both\_mutants\.csv‘

\-Format:‘csv‘

\-Description:Small\-RNAfeaturesreportedasupatleasttwo\-foldinbothtcer\-1mutants\.Small\-RNAdifferentialexpressionvaluescompareeachtcer\-1mutantagainstwild\-typeN2;log2FC\_tm1452\_vs\_wtislog2\(tcer\-1\(tm1452\)/wildtype\)andlog2FC\_glm27\_vs\_wtislog2\(tcer\-1\(glm27\)/wildtype\),sopositivevaluesindicatehigherabundanceinthemutantandnegativevaluesindicatehigherabundanceinwildtype\.

\-Rowkey:‘Feature\_Key‘

\-Rows:OnerowpertinyRNA\-testedsmall\-RNAfeaturefromthestagedMontgomeryLabWS279tinyRNAbundlethatmeetsthisartifact’stwo\-allelefold\-changeandadjusted\-p\-valuerule,keyedbyFeature\_ID\|Classifier\|Feature\_Name\.

\-Examplerowidentifiers:‘AAAAAAGAACTGAAGAGAGTG\|23H\-RNA\|C05E7\.1,C05E7\.t3‘,‘MIMAT0000001\|miRNA\|let\-7\-5p‘,‘WBGene00001335\|WAGOClass22G\|F35A5\.8‘

\-Columns:Compositerowkey,originalfeatureannotations,meanreads,adjustedpvalues,andallele\-specificlog2foldchangeswithmutant\-over\-wild\-typedirection\.

\-Requiredcolumns:‘Feature\_Key‘,‘Feature\_ID‘,‘Classifier‘,‘Feature\_Name‘,‘mean\_reads‘,‘log2FC\_tm1452\_vs\_wt‘,‘padj\_tm1452\_vs\_wt‘,‘log2FC\_glm27\_vs\_wt‘,‘padj\_glm27\_vs\_wt‘

\#\#\#‘mrna\_differential\_expression\_all‘

\-Path:‘outputs/mrna\_differential\_expression\_all\.csv‘

\-Format:‘csv‘

\-Description:AlltestedmRNAgenefeaturesfromtheDESeq2\-styleanalysis\.mRNAdifferentialexpressionvaluescompareeachtcer\-1mutantagainstwild\-typeN2;log2FC\_tm1452\_vs\_wtislog2\(tcer\-1\(tm1452\)/wildtype\)andlog2FC\_glm27\_vs\_wtislog2\(tcer\-1\(glm27\)/wildtype\),sopositivevaluesindicatehigherexpressioninthemutantandnegativevaluesindicatehigherexpressioninwildtype\.

\-Rowkey:‘Feature\_ID‘

\-Rows:Onerowpergene\-levelRSEM/DESeq2\-testedmRNAfeatureidentifiedbyWormBasegeneIDafterquantifyingagainsttheexactstagedWormBaseWS279PRJNA13758RSEMreference\.

\-Examplerowidentifiers:‘WBGene00000001‘,‘WBGene00000002‘,‘WBGene00000003‘

\-Columns:Geneidentifiers,genenames,WAGO/CSR\-1targetannotation,normalizedreplicatecounts,adjustedpvalues,andmutant\-over\-wild\-typelog2foldchangesforeachtcer\-1allele\.

\-Requiredcolumns:‘Feature\_ID‘,‘Feature\_Name‘,‘WAGO\_CSR1\_target‘,‘WT\_rep\_1‘,‘WT\_rep\_2‘,‘WT\_rep\_3‘,‘tm1452\_rep\_1‘,‘tm1452\_rep\_2‘,‘tm1452\_rep\_3‘,‘log2FC\_tm1452\_vs\_wt‘,‘padj\_tm1452\_vs\_wt‘,‘glm27\_rep\_1‘,‘glm27\_rep\_2‘,‘glm27\_rep\_3‘,‘log2FC\_glm27\_vs\_wt‘,‘padj\_glm27\_vs\_wt‘

\#\#\#‘mrna\_up\_twofold\_both\_mutants‘

\-Path:‘outputs/mrna\_up\_twofold\_both\_mutants\.csv‘

\-Format:‘csv‘

\-Description:mRNAfeaturesreportedasupatleasttwo\-foldinbothtcer\-1mutants\.mRNAdifferentialexpressionvaluescompareeachtcer\-1mutantagainstwild\-typeN2;log2FC\_tm1452\_vs\_wtislog2\(tcer\-1\(tm1452\)/wildtype\)andlog2FC\_glm27\_vs\_wtislog2\(tcer\-1\(glm27\)/wildtype\),sopositivevaluesindicatehigherexpressioninthemutantandnegativevaluesindicatehigherexpressioninwildtype\.

\-Rowkey:‘Feature\_ID‘

\-Rows:Onerowpergene\-levelRSEM/DESeq2\-testedWormBasegeneIDwithlog2foldchangeatleast1andadjustedp\-valuebelow0\.05inbothtcer\-1mutantcontrasts\.

\-Examplerowidentifiers:‘WBGene00000001‘,‘WBGene00000002‘,‘WBGene00000003‘

\-Columns:Geneidentifiers,genenames,WAGO/CSR\-1targetannotation,normalizedreplicatecounts,adjustedpvalues,andmutant\-over\-wild\-typelog2foldchangesforeachtcer\-1allele\.

\-Requiredcolumns:‘Feature\_ID‘,‘Feature\_Name‘,‘WAGO\_CSR1\_target‘,‘WT\_rep\_1‘,‘WT\_rep\_2‘,‘WT\_rep\_3‘,‘tm1452\_rep\_1‘,‘tm1452\_rep\_2‘,‘tm1452\_rep\_3‘,‘log2FC\_tm1452\_vs\_wt‘,‘padj\_tm1452\_vs\_wt‘,‘glm27\_rep\_1‘,‘glm27\_rep\_2‘,‘glm27\_rep\_3‘,‘log2FC\_glm27\_vs\_wt‘,‘padj\_glm27\_vs\_wt‘

\#\#\#‘mrna\_down\_twofold\_both\_mutants‘

\-Path:‘outputs/mrna\_down\_twofold\_both\_mutants\.csv‘

\-Format:‘csv‘

\-Description:mRNAfeaturesreportedasdownatleasttwo\-foldinbothtcer\-1mutants\.mRNAdifferentialexpressionvaluescompareeachtcer\-1mutantagainstwild\-typeN2;log2FC\_tm1452\_vs\_wtislog2\(tcer\-1\(tm1452\)/wildtype\)andlog2FC\_glm27\_vs\_wtislog2\(tcer\-1\(glm27\)/wildtype\),sopositivevaluesindicatehigherexpressioninthemutantandnegativevaluesindicatehigherexpressioninwildtype\.

\-Rowkey:‘Feature\_ID‘

\-Rows:Onerowpergene\-levelRSEM/DESeq2\-testedWormBasegeneIDwithlog2foldchangeatmost\-1andadjustedp\-valuebelow0\.05inbothtcer\-1mutantcontrasts\.

\-Examplerowidentifiers:‘WBGene00000001‘,‘WBGene00000002‘,‘WBGene00000003‘

\-Columns:Geneidentifiers,genenames,WAGO/CSR\-1targetannotation,normalizedreplicatecounts,adjustedpvalues,andmutant\-over\-wild\-typelog2foldchangesforeachtcer\-1allele\.

\-Requiredcolumns:‘Feature\_ID‘,‘Feature\_Name‘,‘WAGO\_CSR1\_target‘,‘WT\_rep\_1‘,‘WT\_rep\_2‘,‘WT\_rep\_3‘,‘tm1452\_rep\_1‘,‘tm1452\_rep\_2‘,‘tm1452\_rep\_3‘,‘log2FC\_tm1452\_vs\_wt‘,‘padj\_tm1452\_vs\_wt‘,‘glm27\_rep\_1‘,‘glm27\_rep\_2‘,‘glm27\_rep\_3‘,‘log2FC\_glm27\_vs\_wt‘,‘padj\_glm27\_vs\_wt‘

\#\#\#‘integrative\_tm1452\_all‘

\-Path:‘outputs/integrative\_tm1452\_all\.csv‘

\-Format:‘csv‘

\-Description:AllWAGO\-targetfeaturesfromthetcer\-1\(tm1452\)integrated22G\-RNA/mRNAanalysis\.IntegratedWAGO\-target22G\-RNAandmRNAvaluescomparetcer\-1\(tm1452\)againstwild\-typeN2;thesRNAandmRNAlog2FCcolumnsarelog2\(tcer\-1\(tm1452\)/wildtype\),sopositivevaluesindicatehigherabundanceorexpressioninthemutantandnegativevaluesindicatehigherabundanceorexpressioninwildtype\.

\-Rowkey:‘Feature\_ID‘

\-Rows:OnerowperWAGO\-targetC\.elegansgenefeatureidentifiedbyWormBasegeneID,usingRNA\-integratewithWAGOtargetpairingsderivedfromthestagedWS279sRNAs\_WS279\.gff3andfeatures\_cel\_v1\.5\.csvfiles\.

\-Examplerowidentifiers:‘WBGene00000022‘,‘WBGene00000034‘,‘WBGene00000075‘

\-Columns:Geneidentifiers,genenames,meansRNAandmRNAvalues,adjustedpvalues,andmutant\-over\-wild\-typesRNAandmRNAlog2foldchangesfortcer\-1\(tm1452\)\.

\-Requiredcolumns:‘Feature\_ID‘,‘Feature\_Name‘,‘srna\_mean\_tm1452‘,‘srna\_log2FC\_tm1452\_vs\_wt‘,‘srna\_padj\_tm1452\_vs\_wt‘,‘mrna\_mean\_tm1452‘,‘mrna\_log2FC\_tm1452\_vs\_wt‘,‘mrna\_padj\_tm1452\_vs\_wt‘

\#\#\#‘integrative\_tm1452\_srna\_down\_mrna\_up‘

\-Path:‘outputs/integrative\_tm1452\_srna\_down\_mrna\_up\.csv‘

\-Format:‘csv‘

\-Description:WAGO\-targetfeaturesreportedwithsmallRNAsdownmorethantwo\-foldandmRNAupmorethantwo\-foldintcer\-1\(tm1452\)\.IntegratedWAGO\-target22G\-RNAandmRNAvaluescomparetcer\-1\(tm1452\)againstwild\-typeN2;thesRNAandmRNAlog2FCcolumnsarelog2\(tcer\-1\(tm1452\)/wildtype\),sopositivevaluesindicatehigherabundanceorexpressioninthemutantandnegativevaluesindicatehigherabundanceorexpressioninwildtype\.

\-Rowkey:‘Feature\_ID‘

\-Rows:OnerowperWAGO\-targetWormBasegeneIDfromtheRNA\-integrateresultwithsmallRNAlog2foldchangeatmost\-1,mRNAlog2foldchangeatleast1,andadjustedp\-valuebelow0\.05fortcer\-1\(tm1452\)\.

\-Examplerowidentifiers:‘WBGene00000022‘,‘WBGene00000034‘,‘WBGene00000075‘

\-Columns:Geneidentifiers,genenames,meansRNAandmRNAvalues,adjustedpvalues,andmutant\-over\-wild\-typesRNAandmRNAlog2foldchangesfortcer\-1\(tm1452\)\.

\-Requiredcolumns:‘Feature\_ID‘,‘Feature\_Name‘,‘srna\_mean\_tm1452‘,‘srna\_log2FC\_tm1452\_vs\_wt‘,‘srna\_padj\_tm1452\_vs\_wt‘,‘mrna\_mean\_tm1452‘,‘mrna\_log2FC\_tm1452\_vs\_wt‘,‘mrna\_padj\_tm1452\_vs\_wt‘

\#\#\#‘integrative\_tm1452\_srna\_up\_mrna\_down‘

\-Path:‘outputs/integrative\_tm1452\_srna\_up\_mrna\_down\.csv‘

\-Format:‘csv‘

\-Description:WAGO\-targetfeaturesreportedwithsmallRNAsupmorethantwo\-foldandmRNAdownmorethantwo\-foldintcer\-1\(tm1452\)\.IntegratedWAGO\-target22G\-RNAandmRNAvaluescomparetcer\-1\(tm1452\)againstwild\-typeN2;thesRNAandmRNAlog2FCcolumnsarelog2\(tcer\-1\(tm1452\)/wildtype\),sopositivevaluesindicatehigherabundanceorexpressioninthemutantandnegativevaluesindicatehigherabundanceorexpressioninwildtype\.

\-Rowkey:‘Feature\_ID‘

\-Rows:OnerowperWAGO\-targetWormBasegeneIDfromtheRNA\-integrateresultwithsmallRNAlog2foldchangeatleast1,mRNAlog2foldchangeatmost\-1,andadjustedp\-valuebelow0\.05fortcer\-1\(tm1452\)\.

\-Examplerowidentifiers:‘WBGene00000022‘,‘WBGene00000034‘,‘WBGene00000075‘

\-Columns:Geneidentifiers,genenames,meansRNAandmRNAvalues,adjustedpvalues,andmutant\-over\-wild\-typesRNAandmRNAlog2foldchangesfortcer\-1\(tm1452\)\.

\-Requiredcolumns:‘Feature\_ID‘,‘Feature\_Name‘,‘srna\_mean\_tm1452‘,‘srna\_log2FC\_tm1452\_vs\_wt‘,‘srna\_padj\_tm1452\_vs\_wt‘,‘mrna\_mean\_tm1452‘,‘mrna\_log2FC\_tm1452\_vs\_wt‘,‘mrna\_padj\_tm1452\_vs\_wt‘

\#\#\#‘integrative\_glm27\_all‘

\-Path:‘outputs/integrative\_glm27\_all\.csv‘

\-Format:‘csv‘

\-Description:AllWAGO\-targetfeaturesfromthetcer\-1\(glm27\)integrated22G\-RNA/mRNAanalysis\.IntegratedWAGO\-target22G\-RNAandmRNAvaluescomparetcer\-1\(glm27\)againstwild\-typeN2;thesRNAandmRNAlog2FCcolumnsarelog2\(tcer\-1\(glm27\)/wildtype\),sopositivevaluesindicatehigherabundanceorexpressioninthemutantandnegativevaluesindicatehigherabundanceorexpressioninwildtype\.

\-Rowkey:‘Feature\_ID‘

\-Rows:OnerowperWAGO\-targetC\.elegansgenefeatureidentifiedbyWormBasegeneID,usingRNA\-integratewithWAGOtargetpairingsderivedfromthestagedWS279sRNAs\_WS279\.gff3andfeatures\_cel\_v1\.5\.csvfiles\.

\-Examplerowidentifiers:‘WBGene00000022‘,‘WBGene00000034‘,‘WBGene00000075‘

\-Columns:Geneidentifiers,genenames,meansRNAandmRNAvalues,adjustedpvalues,andmutant\-over\-wild\-typesRNAandmRNAlog2foldchangesfortcer\-1\(glm27\)\.

\-Requiredcolumns:‘Feature\_ID‘,‘Feature\_Name‘,‘srna\_mean\_glm27‘,‘srna\_log2FC\_glm27\_vs\_wt‘,‘srna\_padj\_glm27\_vs\_wt‘,‘mrna\_mean\_glm27‘,‘mrna\_log2FC\_glm27\_vs\_wt‘,‘mrna\_padj\_glm27\_vs\_wt‘

\#\#\#‘integrative\_glm27\_srna\_down\_mrna\_up‘

\-Path:‘outputs/integrative\_glm27\_srna\_down\_mrna\_up\.csv‘

\-Format:‘csv‘

\-Description:WAGO\-targetfeaturesreportedwithsmallRNAsdownmorethantwo\-foldandmRNAupmorethantwo\-foldintcer\-1\(glm27\)\.IntegratedWAGO\-target22G\-RNAandmRNAvaluescomparetcer\-1\(glm27\)againstwild\-typeN2;thesRNAandmRNAlog2FCcolumnsarelog2\(tcer\-1\(glm27\)/wildtype\),sopositivevaluesindicatehigherabundanceorexpressioninthemutantandnegativevaluesindicatehigherabundanceorexpressioninwildtype\.

\-Rowkey:‘Feature\_ID‘

\-Rows:OnerowperWAGO\-targetWormBasegeneIDfromtheRNA\-integrateresultwithsmallRNAlog2foldchangeatmost\-1,mRNAlog2foldchangeatleast1,andadjustedp\-valuebelow0\.05fortcer\-1\(glm27\)\.

\-Examplerowidentifiers:‘WBGene00000022‘,‘WBGene00000034‘,‘WBGene00000075‘

\-Columns:Geneidentifiers,genenames,meansRNAandmRNAvalues,adjustedpvalues,andmutant\-over\-wild\-typesRNAandmRNAlog2foldchangesfortcer\-1\(glm27\)\.

\-Requiredcolumns:‘Feature\_ID‘,‘Feature\_Name‘,‘srna\_mean\_glm27‘,‘srna\_log2FC\_glm27\_vs\_wt‘,‘srna\_padj\_glm27\_vs\_wt‘,‘mrna\_mean\_glm27‘,‘mrna\_log2FC\_glm27\_vs\_wt‘,‘mrna\_padj\_glm27\_vs\_wt‘

\#\#\#‘integrative\_glm27\_srna\_up\_mrna\_down‘

\-Path:‘outputs/integrative\_glm27\_srna\_up\_mrna\_down\.csv‘

\-Format:‘csv‘

\-Description:WAGO\-targetfeaturesreportedwithsmallRNAsupmorethantwo\-foldandmRNAdownmorethantwo\-foldintcer\-1\(glm27\)\.IntegratedWAGO\-target22G\-RNAandmRNAvaluescomparetcer\-1\(glm27\)againstwild\-typeN2;thesRNAandmRNAlog2FCcolumnsarelog2\(tcer\-1\(glm27\)/wildtype\),sopositivevaluesindicatehigherabundanceorexpressioninthemutantandnegativevaluesindicatehigherabundanceorexpressioninwildtype\.

\-Rowkey:‘Feature\_ID‘

\-Rows:OnerowperWAGO\-targetWormBasegeneIDfromtheRNA\-integrateresultwithsmallRNAlog2foldchangeatleast1,mRNAlog2foldchangeatmost\-1,andadjustedp\-valuebelow0\.05fortcer\-1\(glm27\)\.

\-Examplerowidentifiers:‘WBGene00000022‘,‘WBGene00000034‘,‘WBGene00000075‘

\-Columns:Geneidentifiers,genenames,meansRNAandmRNAvalues,adjustedpvalues,andmutant\-over\-wild\-typesRNAandmRNAlog2foldchangesfortcer\-1\(glm27\)\.

\-Requiredcolumns:‘Feature\_ID‘,‘Feature\_Name‘,‘srna\_mean\_glm27‘,‘srna\_log2FC\_glm27\_vs\_wt‘,‘srna\_padj\_glm27\_vs\_wt‘,‘mrna\_mean\_glm27‘,‘mrna\_log2FC\_glm27\_vs\_wt‘,‘mrna\_padj\_glm27\_vs\_wt‘