BIABench: Evaluating AI agents on real-world bioimage analysis tasks

Hugging Face Daily Papers Papers
ai-agents bioimage-analysis benchmark evaluation vision-language-models long-horizon-tasks

Summary

The paper introduces BIABench, an open benchmark of 16 real-world bioimage analysis tasks reconstructed from published studies, evaluating AI agents end-to-end with outcome and process scores. Agents solved routine 2D tasks well but failed on 3D and time-lapse tasks, with neither specialization, stronger models, nor expert instructions closing the reliability gap.

Artificial-intelligence (AI) agents hold promise for automating bioimage analysis, yet no benchmark evaluates whether they can carry out real-world analyses end to end. Such analyses are hard for agents because 2D images, 3D volumes and time-lapse sequences are often too large to read as context, so an agent must choose and run an analysis through code, specialized software and rendered views. Published studies make this capability testable, because each pairs raw images with a peer-reviewed result. We introduce BIABench, a benchmark of 16 tasks reconstructed from published biological studies that retain their scientific questions, imaging data and ground truth. The tasks span eleven analysis subtasks and modalities from H&E histology to single-molecule localization microscopy. Each submission receives an outcome score, which compares the output files with the ground truth using field-standard metrics, and a process score, in which a vision-language model judges method choice and quality control against an expert-written rubric. We evaluated general-purpose and biology-specific agents across several language models, with repeated runs of every task. Routine two-dimensional tasks were solved well, but on some tasks that added a third dimension or a time axis no agent scored above 0.19. Neither biological specialization, stronger models nor detailed expert instructions closed this gap. The agents were also unreliable, with scores varying more between repeated runs of one agent than between different agents, and without ground truth a correct run could not be told from a wrong one by its process score or by the time spent. Released openly with its data and code, BIABench provides a verifiable framework for evaluating, and eventually training, agents for reliable long-horizon bioimage analysis.
Original Article
View Cached Full Text

Cached at: 10/01/26, 08:24 PM

Paper page - BIABench: Evaluating AI agents on real-world bioimage analysis tasks

Source: https://huggingface.co/papers/2609.34274

Abstract

Artificial-intelligence(AI)agentsholdpromiseforautomatingbioimageanalysis,yetnobenchmarkevaluateswhethertheycancarryoutreal-worldanalysesendtoend.Suchanalysesarehardforagentsbecause2Dimages,3Dvolumesandtime-lapsesequencesareoftentoolargetoreadascontext,soanagentmustchooseandrunananalysisthroughcode,specializedsoftwareandrenderedviews.Publishedstudiesmakethiscapabilitytestable,becauseeachpairsrawimageswithapeer-reviewedresult.WeintroduceBIABench,abenchmarkof16tasksreconstructedfrompublishedbiologicalstudiesthatretaintheirscientificquestions,imagingdataandgroundtruth.ThetasksspanelevenanalysissubtasksandmodalitiesfromH&Ehistologytosingle-moleculelocalizationmicroscopy.Eachsubmissionreceivesanoutcomescore,whichcomparestheoutputfileswiththegroundtruthusingfield-standardmetrics,andaprocessscore,inwhichavision-languagemodeljudgesmethodchoiceandqualitycontrolagainstanexpert-writtenrubric.Weevaluatedgeneral-purposeandbiology-specificagentsacrossseverallanguagemodels,withrepeatedrunsofeverytask.Routinetwo-dimensionaltasksweresolvedwell,butonsometasksthataddedathirddimensionoratimeaxisnoagentscoredabove0.19.Neitherbiologicalspecialization,strongermodelsnordetailedexpertinstructionsclosedthisgap.Theagentswerealsounreliable,withscoresvaryingmorebetweenrepeatedrunsofoneagentthanbetweendifferentagents,andwithoutgroundtruthacorrectruncouldnotbetoldfromawrongonebyitsprocessscoreorbythetimespent.Releasedopenlywithitsdataandcode,BIABenchprovidesaverifiableframeworkforevaluating,andeventuallytraining,agentsforreliablelong-horizonbioimageanalysis.

View arXiv pageView PDFProject pageGitHub6Add to collection

Get this paper in your agent:

hf papers read 2609\.34274

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2609.34274 in a model README.md to link it from this page.

Datasets citing this paper1

#### BIABench/BIABench Updated3 days ago • 871 • 2

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2609.34274 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

AutoMedBench: Towards Medical AutoResearch with Agentic AI Models

Hugging Face Daily Papers

AutoMedBench is a workflow-aware benchmark for autonomous medical-AI research, evaluating agents across five stages on diverse medical imaging tasks. Stage-level scoring reveals validation as the weakest stage, highlighting the need for reliable verification in agentic workflows.