When Do Biological Reasoning Models Use Their Biological Inputs?

Hugging Face Daily Papers Papers
biological-reasoning llm foundation-models interpretability probing protein-language-models genomics

Summary

该论文通过输入扰动、证据冲突和线性探针等方法，系统评估了六个生物推理模型在DNA、蛋白质和单细胞任务中是否真正利用了生物基础模型的表示。研究发现BioReason和BioReason-Pro几乎完全依赖文本而非生物输入，表明当前后训练策略并不能保证基础模型表示对任务性能产生实际贡献。

Biological reasoning models use post-training to connect LLMs to biological foundation model representations and biological text. Their benchmark accuracy is taken as evidence that LLMs reason over these inputs. We test this assumption in six biological reasoning models across DNA, protein, and single-cell tasks. We perturb one biological input while holding the query and other inputs fixed, construct evidence conflicts that pair the foundation model representation of one genome, protein, or cell with the text of another, fit linear probes to the representations the language model receives, and analyze reasoning traces against the biological inputs. Evo2 and ESM3 contribute little to BioReason and BioReason-Pro performance on the evaluated tasks. Shuffling the DNA sequence barely changes BioReason disease prediction accuracy, and in evidence conflicts the two models follow the text in 97.9% and 99.7% of cases. Linear probes trained on the Evo2 and ESM3 representations predict the task targets, so these foundation models encode information relevant to the task, but provide limited overall performance improvement to BioReason and BioReason-Pro. In contrast, foundation model inputs contribute to ChatNT, Prot2Text-V2, and CellWhisperer performance, and differentially expressed genes in the gene sentence contribute to Cell2Sentence-Scale performance. Across SFT and RL checkpoints of BioReason-Pro and 42 BioReason checkpoints, increases in accuracy do not imply greater performance contributions from biological inputs. BioReason traces misstate nucleotide changes, while BioReason-Pro traces describe functions omitted from final predictions under evidence conflicts. We find that current post-training strategies do not ensure that foundation model representations contribute to task performance.
Original Article
View Cached Full Text

Cached at: 10/12/26, 06:53 AM

Paper page - When Do Biological Reasoning Models Use Their Biological Inputs?

Source: https://huggingface.co/papers/2610.00898

Abstract

Biologicalreasoningmodelsusepost-trainingtoconnectLLMstobiologicalfoundationmodelrepresentationsandbiologicaltext.TheirbenchmarkaccuracyistakenasevidencethatLLMsreasonovertheseinputs.WetestthisassumptioninsixbiologicalreasoningmodelsacrossDNA,protein,andsingle-celltasks.Weperturbonebiologicalinputwhileholdingthequeryandotherinputsfixed,constructevidenceconflictsthatpairthefoundationmodelrepresentationofonegenome,protein,orcellwiththetextofanother,fitlinearprobestotherepresentationsthelanguagemodelreceives,andanalyzereasoningtracesagainstthebiologicalinputs.Evo2andESM3contributelittletoBioReasonandBioReason-Properformanceontheevaluatedtasks.ShufflingtheDNAsequencebarelychangesBioReasondiseasepredictionaccuracy,andinevidenceconflictsthetwomodelsfollowthetextin97.9%and99.7%ofcases.LinearprobestrainedontheEvo2andESM3representationspredictthetasktargets,sothesefoundationmodelsencodeinformationrelevanttothetask,butprovidelimitedoverallperformanceimprovementtoBioReasonandBioReason-Pro.Incontrast,foundationmodelinputscontributetoChatNT,Prot2Text-V2,andCellWhispererperformance,anddifferentiallyexpressedgenesinthegenesentencecontributetoCell2Sentence-Scaleperformance.AcrossSFTandRLcheckpointsofBioReason-Proand42BioReasoncheckpoints,increasesinaccuracydonotimplygreaterperformancecontributionsfrombiologicalinputs.BioReasontracesmisstatenucleotidechanges,whileBioReason-Protracesdescribefunctionsomittedfromfinalpredictionsunderevidenceconflicts.Wefindthatcurrentpost-trainingstrategiesdonotensurethatfoundationmodelrepresentationscontributetotaskperformance.

View arXiv pageView PDFAdd to collection

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2610.00898 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2610.00898 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2610.00898 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Does Learning Protein Folding Generalize to Broader Reasoning?

Hugging Face Daily Papers

论文提出 Fold2Reason，一种通过蛋白质折叠数据对语言模型进行后训练的方法，结果表明结构密集的科学数据可以将通用推理基准的宏平均准确率从45.09%提升至48.33%，并显著优于匹配的随机对照实验。

How Post-Training Shapes Biological Reasoning Models

Hugging Face Daily Papers

This paper investigates how post-training stages such as continued pre-training, supervised fine-tuning, and reinforcement learning affect generalization in biological reasoning models, finding that these stages have distinct impacts on in-domain and out-of-domain performance.

Best Performing AI with no Biological Risk Restriction

Reddit r/ArtificialInteligence

A university researcher criticizes AI companies for restricting biological research capabilities in models like GPT and Claude, and seeks high-reasoning alternatives for bioinformatics work.