递归知识构建有助于智能体核实科学共识主张吗？

# Does Recursive Knowledge Construction Help Agents Verify Consensus Claims in Science?
Source: [https://arxiv.org/html/2610.40906](https://arxiv.org/html/2610.40906)
Yu\-Heng WuJonathan SchaulLeonie KüchenhoffDenes TureiFrancesco CarliFabio PetroniAurelien Dugourd††thanks:Joint supervisionJulio Saez\-RodriguezEuropean Bioinformatics Institute \(EMBL\-EBI\), European Molecular Biology Laboratory, Hinxton, UK\. Institute for Computational Biomedicine, Heidelberg University, Faculty of Medicine, and Heidelberg University Hospital, Heidelberg, Germany\. EMBL Rome, European Molecular Biology Laboratory, Monterotondo, Italy\.\{ail,dugourd,saez\}@ebi\.ac\.uk

###### Abstract

Recent scientific agents build knowledge recursively by retrieving evidence, constructing an intermediate state, and judging whether that state contains sufficient information\. These agents are evaluated end\-to\-end, usually without ground truth, so the effect of each design choice is unknown\. Here, we examined two design axes: \(1\) what the recursion builds \(the state of evidence\) and \(2\) what controls it \(the judge that decides when to stop\)\. To measure them, we defined consensus scientific claim verification and measured these effects on this task, where a system must establish a claim’s consensus stance from the open literature\. We built a new consensus claim dataset, ProClaim\-eval, including 718 claims with SUPPORT and REFUTE gold labels from five expert\-curated databases spanning molecular biology, nuclear physics, and astronomy\. Our instrument, ProClaim, persists an evidence state outside the model’s context, so each axis can be studied with the other held fixed\. Against verification systems and general\-purpose agent harnesses, ProClaim ranks among the top three, with a significant advantage over the harnesses\. We found that the evidence state improves performance over in\-context accumulation, and it also matters what is stored in it\. In addition, a trained judge that reads the evidence state performs better than fixed\-round stopping\. A judge trained separately and never exposed to ProClaim\-eval achieves the highest agreement with the gold labels on three of five subsets, where it matches the off\-the\-shelf Claude Sonnet 4\.6 judge and outperforms Claude Haiku 4\.5\. Our results support recursively maintaining a structured evidence state outside the model’s context as an effective design for verifying claims in scientific agents\.

## 1Introduction

![Refer to caption](https://arxiv.org/html/2610.40906v1/graphic_overview-motivation.png)Figure 1:Document\-guided vs\. consensus scientific claim verification\.Scientific agents built on large language models \(LLMs\) can now retrieve, synthesise, and reason over the research literature at scale\([Bachman et al\., 2023](https://arxiv.org/html/2610.40906#bib.bib36);[Wood et al\., 2026](https://arxiv.org/html/2610.40906#bib.bib37)\)\. Many agentic systems\([Ghareeb et al\., 2025](https://arxiv.org/html/2610.40906#bib.bib45);[Skarlinski et al\., 2024](https://arxiv.org/html/2610.40906#bib.bib26);[Mitchener et al\., 2025](https://arxiv.org/html/2610.40906#bib.bib40);[Gottweis et al\., 2026](https://arxiv.org/html/2610.40906#bib.bib46)\)execute a common loop, which constructs a record of evidence and judges whether the record suffices for a given task, and if not, uses the record to direct the next retrievals\. The loop is recursive because each step’s record is the next step’s input\([Zhang et al\., 2025](https://arxiv.org/html/2610.40906#bib.bib47)\)\. However, these agents contain many components and are evaluated only on final outputs, so effectiveness cannot be attributed to any single part\. Whether*what the recursion builds*and*what controls it*actually help is therefore assumed rather than carefully examined\. We call these the two design axes of*recursive knowledge construction*\.

To measure each axis, we require a task that exercises the loop and has established consensus\. Scientific claim verification is the natural candidate\. Claims are the basic unit of literature synthesis and hypothesis generation, and building the consensus on a claim is essential to counter research misinformation\([Deng et al\., 2025](https://arxiv.org/html/2610.40906#bib.bib25);[Diggelmann et al\., 2020](https://arxiv.org/html/2610.40906#bib.bib18);[van der Linden, 2022](https://arxiv.org/html/2610.40906#bib.bib10);[West and Bergstrom, 2021](https://arxiv.org/html/2610.40906#bib.bib23)\)and to uncover gaps that may drive new discovery\([Asai et al\., 2026](https://arxiv.org/html/2610.40906#bib.bib11);[Vladika and Matthes, 2023](https://arxiv.org/html/2610.40906#bib.bib9)\)\. Existing datasets\([Diggelmann et al\., 2020](https://arxiv.org/html/2610.40906#bib.bib18);[Reisle et al\., 2025](https://arxiv.org/html/2610.40906#bib.bib6);[Saakyan et al\., 2021](https://arxiv.org/html/2610.40906#bib.bib29);[Sarrouti et al\., 2021](https://arxiv.org/html/2610.40906#bib.bib30);[Wadden et al\., 2020](https://arxiv.org/html/2610.40906#bib.bib12);[Wadden et al\., 2022a](https://arxiv.org/html/2610.40906#bib.bib5)\), however, have reduced verification to reading comprehension over a single retrieved passage\([Vladika and Matthes, 2024](https://arxiv.org/html/2610.40906#bib.bib31)\)\. This document\-guided setting \(Figure[1](https://arxiv.org/html/2610.40906#S1.F1)\) has inherent constraints, as verdicts grounded in a limited context cannot reflect the actual scientific stance of claims across the broader literature\([Deng et al\., 2025](https://arxiv.org/html/2610.40906#bib.bib25);[Kumar et al\., 2025](https://arxiv.org/html/2610.40906#bib.bib28);[Skarlinski et al\., 2024](https://arxiv.org/html/2610.40906#bib.bib26)\)\. We therefore define*consensus claim verification*as an open\-domain\([Vladika and Matthes, 2024](https://arxiv.org/html/2610.40906#bib.bib31)\), claim\-level task with labels reflecting the epistemic state of the literature\. Solving it requires retrieving, organising and reconciling sparse and conflicting evidence across many sources\([Vladika et al\., 2025](https://arxiv.org/html/2610.40906#bib.bib24);[Wang et al\., 2024b](https://arxiv.org/html/2610.40906#bib.bib14)\), which functionally demands a recursive agentic loop\.

![Refer to caption](https://arxiv.org/html/2610.40906v1/graphic_overview-recursive_knowledge_construction.png)Figure 2:Recursive knowledge construction for claim verification\. We examine two design choices in Section[4](https://arxiv.org/html/2610.40906#S4): \(1\) an evidence state that the next retrieval is conditioned on, \(2\) a judge that takes the state as input and decides either to continue or terminate the construction, after which the verdict is given from the final state\.To make this setting evaluable, we releaseProClaim\-eval, a new dataset of 718 consensus claims built from five actively curated expert databases spanning molecular biology\([Lo Surdo et al\., 2025](https://arxiv.org/html/2610.40906#bib.bib4);[Liu et al\., 2026](https://arxiv.org/html/2610.40906#bib.bib16);[Vazquez et al\., 2022](https://arxiv.org/html/2610.40906#bib.bib41)\), nuclear structure\([Tuli, 1996](https://arxiv.org/html/2610.40906#bib.bib44)\)and astronomy\([Akeson et al\., 2013](https://arxiv.org/html/2610.40906#bib.bib43);[Christiansen et al\., 2025](https://arxiv.org/html/2610.40906#bib.bib42)\)\. Prior claim verification datasets offer only passage\-level annotations because curating consensus stances demands substantial expertise\([Guo et al\., 2022](https://arxiv.org/html/2610.40906#bib.bib27);[Wadden et al\., 2020](https://arxiv.org/html/2610.40906#bib.bib12);[Wang et al\., 2024b](https://arxiv.org/html/2610.40906#bib.bib14)\)\. The chosen databases are actively maintained, so their entries track the current domain consensus rather than the stance of any one paper\. Gold labels aresupportorrefute, and a system may additionally answeruncertain, which we treat as an abstention\. To vary the two design choices in recursive knowledge construction \(Figure[2](https://arxiv.org/html/2610.40906#S1.F2)\), we builtProClaim, a minimal open\-source instrument that maintains an evolving evidence state and a replaceable judge\. Findings from our instrument only generalise if it performs on par with systems in use, so we compared ProClaim with state\-of\-the\-art verification systems\([Xie et al\., 2025](https://arxiv.org/html/2610.40906#bib.bib7);[Asai et al\., 2026](https://arxiv.org/html/2610.40906#bib.bib11);[Skarlinski et al\., 2024](https://arxiv.org/html/2610.40906#bib.bib26)\)and general\-purpose agent harnesses \(CODEX and Claude Code\)\.

Our work makes three main contributions:

- •We formulate consensus scientific claim verification and release ProClaim\-evalas a test domain for recursive knowledge construction, distinct from document\-guided verification\. ProClaim\-eval contains 718 claims from five expert\-curated databases across three domains, with binary gold labels\. On ProClaim\-eval, ProClaim ranks among the top three systems on four subsets, with a significant advantage over the Claude Code and Codex agent harnesses\.
- •Recursively built evidence state beats in\-context accumulation, and what it stores matters\.On ProClaim\-eval, ProClaim outperforms a ReAct agent that accumulates the retrieved papers only in its context, with the same LLM backbone and retrieval tools\. Using ProClaim\-retrieved papers, we show that the extracted facts in the state give better performance than abstracts or full text as input\.
- •A lightweight trained judge can transfer and replace off\-the\-shelf LLM judges\.With ProClaim as the instrument, we found that a trained judge that reads the state is better and uses fewer steps than a fixed round budget\. We trained a lightweight MLP judge, fitted only on SciFact\-Open and evaluated in transfer, as an alternative to off\-the\-shelf LLM judges, which matches the Claude Sonnet 4\.6 judge and outperforms the Claude Haiku 4\.5 judge on three subsets\.

## 2Document\-Guided Versus “Consensus” Claim Verification

Table 1:Claim diversity in the ProClaim\-eval benchmark\.We show one example claim per subset, with the subset’s claim count andsupport/refutesplit\. The five subsets instantiate five distinct single\-sentence templates over three scientific domains with a total of 718 claims\.Establishing the consensus stance of a scientific claim is essential for uncovering research gaps\([Asai et al\., 2026](https://arxiv.org/html/2610.40906#bib.bib11);[Vladika and Matthes, 2023](https://arxiv.org/html/2610.40906#bib.bib9)\)and curating reliable scientific knowledge\([Deng et al\., 2025](https://arxiv.org/html/2610.40906#bib.bib25);[Diggelmann et al\., 2020](https://arxiv.org/html/2610.40906#bib.bib18)\)\. When verifying consensus claims, the verdict emerges from reconciling heterogeneous and conflicting evidence across multiple retrieval passes\. Therefore, no single passage can be assumed authoritative for this purpose\. Existing benchmarks\([Wadden et al\., 2020](https://arxiv.org/html/2610.40906#bib.bib12);[Wadden et al\., 2022a](https://arxiv.org/html/2610.40906#bib.bib5);[Reisle et al\., 2025](https://arxiv.org/html/2610.40906#bib.bib6)\)follow this document\-guided paradigm, and are inherently limited\. Here, we distinguish these two problem settings:

- •Document\-guided verification\.A verdictv=f⁡\(𝒞,d\)v=f\(\\mathcal\{C\},d\)is associated with a claim𝒞\\mathcal\{C\}*and a single passage*d∈𝒟=\{d1,d2,…,dN\}d\\in\\mathcal\{D\}=\\\{d\_\{1\},d\_\{2\},\\ldots,d\_\{N\}\\\}, where𝒟\\mathcal\{D\}is a fixed set of documents\. The goal is to retrieve the passage given the claim and*classify the pair*\(𝒞,d\)\(\\mathcal\{C\},d\)\.
- •Consensus claim verification\.A verdictv=f⁡\(𝒞,𝒟ℒ\)v=f\(\\mathcal\{C\},\\mathcal\{D\_\{\\mathcal\{L\}\}\}\)is associated with a claim𝒞\\mathcal\{C\}*against a set of passages*𝒟ℒ⊆ℒ\\mathcal\{D\_\{\\mathcal\{L\}\}\}\\subseteq\\mathcal\{L\}, whereℒ\\mathcal\{L\}is a large document collection representing the open scientific literature\. Since the aim is to*predict the consensus verdict*, the system must determine both which documents to include in𝒟ℒ\\mathcal\{D\_\{\\mathcal\{L\}\}\}and how many passages are needed\.

#### ProClaim\-eval\.

Existing scientific claim verification datasets contain verdict annotations for each claim–passage pair\. This limits evaluation when assembling findings across multiple papers to reach scientific consensus\. To address this gap, we created ProClaim\-eval \(Appendix[B](https://arxiv.org/html/2610.40906#A2)\), 718 single\-sentence claims from five expert\-curated databases spanning three domains \(Table[1](https://arxiv.org/html/2610.40906#S2.T1)\)\. None of the databases provides claim\-level verdicts, so each claim’ssupportorrefutelabel was derived from the curators’ own record of the underlying evidence\. In molecular biology, ConnectomeDB\([Liu et al\., 2026](https://arxiv.org/html/2610.40906#bib.bib16)\)contributes 305 ligand\-receptor interactions, labelledsupportwhen added in its 2025 release with at least two supporting publications andrefutewhen rejected by its curators from the 2020 release\. SIGNOR\([Lo Surdo et al\., 2025](https://arxiv.org/html/2610.40906#bib.bib4)\)contributes 101 expert re\-curated claims about signed protein\-protein interactions, whoserefuteclaims are interactions removed in later releases or confirmed interactions with a negated sign\. ExTRI\([Vazquez et al\., 2022](https://arxiv.org/html/2610.40906#bib.bib41)\)\(part of the CollecTRI database\([Müller\-Dott et al\., 2023](https://arxiv.org/html/2610.40906#bib.bib52)\)\) contributes 125 transcription factor\-target regulations drawn from expert\-read sentences, labelledsupportwhen at least five papers agree on the direction of regulation andrefutewhen every curated sentence negates it\. In astronomy, the NASA Exoplanet Archive\([Akeson et al\., 2013](https://arxiv.org/html/2610.40906#bib.bib43);[Christiansen et al\., 2025](https://arxiv.org/html/2610.40906#bib.bib42)\)contributes 102 claims that an object is a planet orbiting a given star, labelledsupportfor confirmed planets andrefutefor published planets later refuted or found to be false positives\. In nuclear physics, the NUBASE2020 evaluation\([Kondev et al\., 2021](https://arxiv.org/html/2610.40906#bib.bib48)\)contributes 85 claims that a nuclide has an isomeric state, labelledsupportfor established states andrefutefor states reported and later shown not to exist\. In ExTRI, NUBASE, and NASA, since their most recent update took place before 2025, we further removed claims that an LLM answers correctly without retrieval, so that a verdict requires retrieving the literature rather than recalling it\.

Figure 3:Verdict prediction\-label agreement \(outlined segments\) across three verification settings on the SIGNOR and ConnectomeDB subsets of ProClaim\-eval \(406 claims\), using Claude Sonnet 4\.6\. Both document\-guided and naive retrieval verification do not produce consensus reliably and overproducerefutepredictions due to insufficient or noisy information, a limitation that ProClaim addresses\. This comparison with ProClaim highlights the benefit of recursive knowledge construction, driven by adaptive retrieval and evidence organisation before deciding the final verdict\.
#### The need for consensus claim verification\.

ProClaim\-eval assigns a label to each claim from categories \{support,refute\} \(label definitions in Appendix[B\.1](https://arxiv.org/html/2610.40906#A2.SS1)\)\. Our analysis in Figure[3](https://arxiv.org/html/2610.40906#S2.F3)shows thatdocument\-guided verification is insufficient for consensus\.With one expert reference abstract as the “silver” standard context to guide LLM claim verification, the agreement between verdict predictions and labels is only 60\.3%, which cannot reliably proxy scientific consensus\.Naive retrieval is insufficient too\.Five retrieved abstracts reduce agreement to 52\.2% and misclassify 24%supportclaims compared to using the reference abstract\. Importantly, both document\-guided verification and static retrieval overproducerefutepredictions due to failing to find explicit positive support under limited or noisy evidence, and absence of support after search is treated as REFUTE\.

## 3Recursive Knowledge Construction

Our results in Section[2](https://arxiv.org/html/2610.40906#S2)show that verifying scientific consensus cannot be reduced to reading from naively retrieved passages\. In this section, we formally define the mechanism*recursive knowledge construction*that recent scientific agents use instead\. This led us to build the minimal instrument ProClaim that allows us to vary the two design axes of recursive knowledge construction to examine their effects on consensus verification\.

### 3\.1Formal Definition

We formalise the process shared by recent scientific agents\([Skarlinski et al\., 2024](https://arxiv.org/html/2610.40906#bib.bib26);[Mitchener et al\., 2025](https://arxiv.org/html/2610.40906#bib.bib40)\)as a recursion on an explicit evidence state\. Letccbe a scientific claim andℒ\\mathcal\{L\}a corpus of documents as in the consensus claim verification setting\. At steptt, a verifier maintains an*evidence state*sts\_\{t\}as an intermediate record, which is updated via:

st\+1=U\(st,R\(c,st\)\),t=0,1,…s\_\{t\+1\}=U\\big\(s\_\{t\},\\;R\(c,s\_\{t\}\)\\big\),\\qquad t=0,1,\\ldotswhereR⁡\(c,st\)⊆ℒR\(c,s\_\{t\}\)\\subseteq\\mathcal\{L\}decides what to retrieve given what the state already holds, andUUupdates the state by recording the result\. Verification halts at the first roundTTat which a judgementJ⁡\(c,sT\)J\(c,s\_\{T\}\)holds, after which the agent produces its output verdict from the final record\. Two properties make this process recursive\. The evidence state contains the knowledge constructed by the agent, and the state in the previous round becomes the input to the next\. The state also guides both where the agent looks next and when it should terminate\.

Two design choices are key in the definition, and they are the objects of study in this paper\.

- •Evidence state\.In contrast to claim verification systems such as FIRE\([Xie et al\., 2025](https://arxiv.org/html/2610.40906#bib.bib7)\), SAFE\([Wei et al\., 2024](https://arxiv.org/html/2610.40906#bib.bib8)\)and OpenScholar\([Asai et al\., 2026](https://arxiv.org/html/2610.40906#bib.bib11)\), which keep a record of the growing prompt context, in scientific agents, for example, PaperQA2\([Skarlinski et al\., 2024](https://arxiv.org/html/2610.40906#bib.bib26)\)and Kosmos\([Mitchener et al\., 2025](https://arxiv.org/html/2610.40906#bib.bib40)\), the state is explicitly constructed as structure persisted outside the context\.
- •Judgement\.The verification process needs a stopping rule\. This could be a fixed number of rounds\([Asai et al\., 2026](https://arxiv.org/html/2610.40906#bib.bib11)\)or a function of the retrieved evidence\([Xie et al\., 2025](https://arxiv.org/html/2610.40906#bib.bib7);[Asai et al\., 2024](https://arxiv.org/html/2610.40906#bib.bib20)\)based on an LLM judgment of whether to keep searching\.

We implemented a minimal instrument agent, ProClaim, to perform recursive knowledge construction and examine the effects of those two design choices\. The experiments therefore varied one axis with the other fixed in ProClaim\.

### 3\.2The Evidence State

We designed the evidence state as a persistent JSON outside the LLM context\. The state records retrieved papers and claim\-relevant facts linked to their source paper, with a stance in\{support,refute,neutral\}\\\{\\textsc\{support\},\\textsc\{refute\},\\textsc\{neutral\}\\\}\. We followed the CodeAct\([Wang et al\., 2024a](https://arxiv.org/html/2610.40906#bib.bib21)\)approach, so at each step, the agent writes a short Python program\. This program is composed of operations to interact with the state from a small library, including fact extraction with stance labelling from abstracts or full text, subclaim decomposition, and the judge call\.

This state design is minimal and preserves agentic behaviour rather than executing a fixed pipeline\. Because the agent needs to access the state through explicit programmatic operations, its content is not compressed or paraphrased as steps accumulate, unlike evidence folded into the growing prompt that dilutes as the context grows\. The agent’s context is the execution log of the operations it has run and their printed results\. Because the query at roundt\+1t\+1is issued after reading the state, retrieval is conditioned on the accumulated evidence as we have defined in Section[3\.1](https://arxiv.org/html/2610.40906#S3.SS1)\.

### 3\.3The Judge

Section[2](https://arxiv.org/html/2610.40906#S2)shows a characteristic failure in retrieval tasks\([Xie et al\., 2025](https://arxiv.org/html/2610.40906#bib.bib7);[Jeong et al\., 2024](https://arxiv.org/html/2610.40906#bib.bib50);[Asai et al\., 2024](https://arxiv.org/html/2610.40906#bib.bib20)\), where claims require aggregating multiple documents, and an absence of support after a shallow retrieval leads to moreRefute\. Stopping too early yields a verdict from incomplete or one\-sided evidence, but more retrieval on claims can introduce noise or conflict\. Following the notion of*sufficient context*\([Joren et al\., 2025](https://arxiv.org/html/2610.40906#bib.bib22)\), we therefore wanted the judge to score a property of the evidence state itself\. We designed the judge as a scorerJ⁡\(c,st\)∈\[0,1\]J\(c,s\_\{t\}\)\\in\[0,1\]\. The agent stops whenJ⁡\(c,st\)≥δJ\(c,s\_\{t\}\)\\geq\\deltaor reaches a maximum iterationTmaxT\_\{\\max\}for practical constraints\. We implementedJJin two ways that consume the same state and the sameδ=0\.8\\delta=0\.8, with a capTmax=4T\_\{\\max\}=4\.

#### Trained judge\.

We trained a lightweight 2\-layer MLP \(hidden dimensions 64 and 32\) and exposed it to the agent as a tool, so the stopping signal is external rather than a self\-assessment\. We prepared training data using SciFact\-Open\([Wadden et al\., 2022a](https://arxiv.org/html/2610.40906#bib.bib5)\), which labels stance at the paper level for 279 claims\. Papers sharing claims were grouped into the same evidence states and used a minority ratio to annotate sufficiency labels \(Appendix[D\.3](https://arxiv.org/html/2610.40906#A4.SS3)\)\. Per\-paper features were extracted from frozen, off\-the\-shelf encoders, then pooled across all papers in the evidence state into one global representation\. We computed the cosine similarity from SBERT\([Reimers and Gurevych, 2019](https://arxiv.org/html/2610.40906#bib.bib2)\)between claim and paper text\. This distinguishes states with actually relevant evidence from states with off\-topic papers\. NLI stance probabilities \(entailment, contradiction, neutral\) come from a DeBERTa NLI model\([He et al\., 2021](https://arxiv.org/html/2610.40906#bib.bib1)\)applied to the claim against the most relevant text chunk from each paper\. This acts as a soft, per\-paper stance signal\. Together with entity coverage and bibliometric metadata, these are pooled into a 10\-dimensional global vector that feeds the MLP\. We include implementation details in Appendix[D\.4](https://arxiv.org/html/2610.40906#A4.SS4)\.

#### Off\-the\-shelf judge\.

An LLM is prompted zero\-shot with the claim and a rendering ofsts\_\{t\}\(paper titles, abstracts, the per\-paper meta\-data, and similarity scores from above\) and asked to return a sufficiency score, with the definition above stated in the prompt \(Appendix[D\.6](https://arxiv.org/html/2610.40906#A4.SS6)\)\.

## 4Experiments

Shared setting\.Unless otherwise noted, all variations were evaluated across five subsets of the ProClaim\-eval dataset\. We report*agreement*, the fraction of all claims whose predicted verdict matches the gold label, as in Section[2](https://arxiv.org/html/2610.40906#S2)\. A system may answeruncertainto abstain, and we compute the*abstention rate*as the fraction of claims answereduncertain\. Every agentic system, including ProClaim, used Claude Sonnet 4\.6 with thinking disabled and temperature 0\. Paper search was PubMed and Semantic Scholar \(S2\) on the three biology subsets and S2 alone on non\-biology subsets, except for LLM\-only baselines where models answer from parameter knowledge\. The fact extraction in ProClaim was done by Qwen 3\.5 9B\. For each finding, we compared two systems on the same claims with a paired exact McNemar test, pooled over subsets, and report Holm\-adjustedpp\-values \(Appendix[C\.2](https://arxiv.org/html/2610.40906#A3.SS2)\)\.

Table 2:Agreement \(Agr\.\) and abstention rate \(Abst\.\) per ProClaim\-eval subset\. The shaded figure beside each agreement is ProClaim’s agreement minus that row’s, on the same subset; green where ProClaim is ahead, red where it is behind\. The highest agreement per subset is underlined\.### 4\.1How Do Verifiers and Agents Perform on Consensus Claim Verification?

Settings\.*Random*draws each verdict uniformly fromsupport,refuteanduncertain, averaged over 10 draws, as a chance floor\.*LLM\-only*makes one call with no retrieved context\.*Static retrieval*retrieves the top\-5 S2 abstracts for the claim and asks the same LLM for a verdict in one shot\.*ReAct*\([Yao et al\., 2023](https://arxiv.org/html/2610.40906#bib.bib15)\)runs a thought\-action\-observation loop over the paper\-search tools\.*FIRE*\([Xie et al\., 2025](https://arxiv.org/html/2610.40906#bib.bib7)\)is a claim verifier built around web search, and we updated its prompts to adopt the paper\-search backend as described above\. FIRE stops when the LLM reports sufficient confidence\. Both ReAct and FIRE were capped at 20 steps, with a budget matching ProClaim’s average number of retrievals\.*OpenScholar*\([Asai et al\., 2026](https://arxiv.org/html/2610.40906#bib.bib11)\)retrieves from the same paper\-search sources and refines its answer with LLM feedback\. We rephrased claims as questions to use OpenScholar’s native LLM feedback pipeline\.*PaperQA2*\([Skarlinski et al\., 2024](https://arxiv.org/html/2610.40906#bib.bib26)\)keeps a ranked store of relevance\-scored evidence summaries outside the model’s context and answers from the highest\-ranked ones\. It stops when the LLM decides to answer\. We replaced its local paper index with the same paper\-search tools and open\-access full text, and used our verdict defintions, and capped it at 20 steps and 45 minutes per claim\.*Claude Code*and*Codex*were general\-purpose agent harnesses given the same paper\-search tools with web search disabled\. To bound run time, Claude Code was capped at 60 turns and Codex at 45 minutes per claim\. Every system received the same label definitions \(Appendix[B\.1](https://arxiv.org/html/2610.40906#A2.SS1)\)\.

Results\.Table[2](https://arxiv.org/html/2610.40906#S4.T2)reports agreement and abstention per subset\.First, ProClaim has the highest agreement on ConnectomeDB \(0\.78\) and NUBASE \(0\.52\), trails Codex on SIGNOR \(0\.80 vs\. 0\.86\), and trails Claude Code on ExTRI \(0\.72 vs\. 0\.82\) and FIRE on NASA \(0\.66 vs\. 0\.70\)\.Second, static retrieval scores below LLM\-only on ConnectomeDB with both LLMs and on NUBASE and NASA with Gemini 2\.5 Flash, showing that naively retrieving the top\-5 papers or snippets does not add reliable benefit over parametric knowledge\.Third, adaptive retrieval improves on static retrieval for the systems that commit to a verdict\. ReAct and FIRE exceed static retrieval on almost every subset\. OpenScholar abstains on most claims on every subset except ConnectomeDB \(from 0\.58 on SIGNOR to 0\.98 on NUBASE\) and scores below LLM\-only Sonnet 4\.6 on SIGNOR \(0\.36 vs\. 0\.63\)\.Fourth, the SOTA harnesses are strongest on the biology subsets\. Codex has the highest SIGNOR agreement \(0\.86\) and Claude Code the highest ExTRI agreement \(0\.82\), but on NUBASE both stay below the random baseline \(0\.22 and 0\.19 vs\. 0\.34\), abstaining on 0\.65 and 0\.59 of claims\.

Finding 1: Recursive construction is consistently strongRecursive knowledge construction shows strong performance on ProClaim\-eval\. Our instrument ProClaim ranks amongst the top 3 on four subsets, with a significant advantage against harnesses \(pooled agreement 0\.73 vs\. 0\.68 for Claude Code and 0\.68 for Codex,p=0\.022p=0\.022\)\.

### 4\.2What Does the Evidence State Contribute?

Without an evidence state, a verifier reads what it retrieves as raw text accumulating in its context, and it reaches the verdict from that text\. The evidence state \(Section[3\.2](https://arxiv.org/html/2610.40906#S3.SS2)\) instead stores what was read as relevant facts, each linked to its source paper and labelled with a stance towards the claim\. Stored this way, the evidence can be read in a distilled form rather than re\-read as whole papers, and it persists across rounds rather than diluting as the context grows\. We tested whether this representation is a better input to the verdict than the papers it was extracted from, holding the retrieved papers fixed\.

Table 3:Effect of the evidence state\. The same LLM \(Claude Sonnet 4\.6\) reads the different information retrieved in the evidence state of ProClaim, against no retrieval\. Every row is scored on the same randomly sampled 40 claims per subset\. We show the agreement \(Agr\.\) and abstention rate \(Abst\.\) per subset, and the highest agreement per subset is underlined\.Settings\.We varied what the recursion builds while holding the backbone and the retrieval tools fixed, in two ways\. We first took the evidence ProClaim retrieved for each claim and gave it once to the same LLM with the same prompt, packaged as retrieved abstracts, full texts where available, the stance\-labelled facts of ProClaim’s evidence state, or facts plus abstracts\.*No retrieval*is the LLM\-only baseline\. Every other settings gives the final state ProClaim built for the LLM in a single call, so these rows differ only in the content represented\.*ProClaim*is the full loop of Table[2](https://arxiv.org/html/2610.40906#S4.T2), which builds those states\. As a control with no evidence state,*ReAct \+ subclaims−\-state*gives ReAct ProClaim’s subclaims and the same retrieval tools for 20 steps, and accumulates the papers it reads in its context\.*ReAct \+ subclaims \+ facts−\-state*adds ProClaim’s fact extraction to this control, where every paper a search returns is passed, as full text where available, to the same extraction model \(Qwen 3\.5 9B\) and prompt, and ReAct sees the extracted facts with each result\. Every row was scored on randomly sampled 40 claims per subset\.

Results\.First, any retrieved evidence moves agreement well above no retrieval on every subset\.Second, with the retrieved papers held fixed, the stance\-labelled facts are a better input to the verdict than the papers they were extracted from on four of five subsets\. Pooled over the five subsets, facts give higher agreement than both abstracts and full text \(0\.77 vs\. 0\.65 and 0\.70,p=0\.024p=0\.024\), and lower abstention on five subsets, most on NUBASE \(0\.15 vs\. 0\.45 with abstracts\)\.Third, more raw text does not substitute for the facts on our dataset\. Full text is equivalent to using abstracts on SIGNOR, ConnectomeDB and NASA\. It is ahead of abstracts on ExTRI \(0\.83 vs\. 0\.75\) and NUBASE \(0\.38 vs\. 0\.28\) but stays below facts on both\.Fourth, ProClaim has higher agreement than both controls without an evidence state on four subsets\. Pooled over the five subsets, its agreement is 0\.77 against 0\.60 for ReAct \+ subclaims−\-state and 0\.63 for ReAct \+ subclaims \+ facts−\-state \(p=0\.0012p=0\.0012\)\. ReAct \+ subclaims−\-state is below both raw\-paper rows on ConnectomeDB \(0\.75\), NUBASE \(0\.20, abstaining on 0\.70 of claims\) and NASA \(0\.43\)\. ProClaim stays ahead of controls on SIGNOR, ConnectomeDB, NUBASE and NASA\.

Finding 2: The evidence state helps, but it matters what it storesThe facts in the evidence state are a better input than the abstracts or full text \(pooled agreement 0\.77 vs\. 0\.65 and 0\.70,p=0\.024p=0\.024\)\. This is consistent with the prior finding that LLMs use long contexts unevenly\([Liu et al\., 2024](https://arxiv.org/html/2610.40906#bib.bib51)\)\. An agent that keeps papers only in its context does no better than reading raw papers, and ProClaim outperforms it \(pooled 0\.77 vs\. 0\.60 and 0\.63 for the two controls,p=0\.0012p=0\.0012\)\.

### 4\.3What Does a Trained Sufficiency Judge Add over an Off\-the\-Shelf One?

Table 4:Effect of the sufficiency judge with a fixed evidence state design, on the same 40 randomly sampled claims per subset\. Agreement \(Agr\.\), abstention rate \(Abst\.\) and mean judge calls at stop \(Iter\.\) for each stopping rule, the highest agreement per subset is underlined\.Settings\.We held the evidence state, the backbone, the retrieval tools and the stop threshold \(δ=0\.8\\delta=0\.8\) fixed and replaced only the judgeJJ\(Section[3\.3](https://arxiv.org/html/2610.40906#S3.SS3)\)\. Three stopping rules ignore the state\.*Fixed rounds*prompts the agent to use up the four max rounds \(we enforced in the prompt but early termination was still up to the agent\)\.*Always continue*always returnscontinue, leaving the agent to decide when to stop\.*Random score*randomly samples stop/continue signal\. Three off\-the\-shelf judges are LLMs prompted zero\-shot on the state \(Appendix[D\.6](https://arxiv.org/html/2610.40906#A4.SS6)\): Qwen 3\.5 9B, Claude Haiku 4\.5 and Claude Sonnet 4\.6\. The trained judge was fitted once on SciFact\-Open and never saw ProClaim\-eval\.*Iterations at stop*counts the number of times the judge is called, where the max is 4\. Every judge was evaluated on the same 40 claims per subset, drawn at random\.

Results\.First, in Table[4](https://arxiv.org/html/2610.40906#S4.T4), the trained judge, fitted only on SciFact\-Open, has the highest agreement of every stopping rule on SIGNOR, ConnectomeDB and NASA, ahead of the strongest off\-the\-shelf judge, Sonnet 4\.6 \(pooled over these three subsets 0\.83 vs\. 0\.80,p=0\.50p=0\.50\), and the lowest abstention on SIGNOR and NASA \(0\.05 and 0\.05\)\. It trails Sonnet 4\.6 on ExTRI \(0\.85 vs\. 0\.90\) and Haiku 4\.5 on NUBASE \(0\.50 vs\. 0\.58\)\.Second, the judges that ignore the state are also unstable across subsets\. Always continuing and the random score are among the best rules on ExTRI \(0\.88 and 0\.88\) but the worst on NASA \(0\.50 and 0\.48\)\.Third, among off\-the\-shelf LLM judges, more capable models are often better judges\. Claude Sonnet 4\.6 has the highest agreement on four of five subsets, Claude Haiku 4\.5 leads on NUBASE \(0\.58\), and Qwen 3\.5 9B has the lowest agreement of any stopping rule on SIGNOR \(0\.63\)\. Every off\-the\-shelf judge stops after fewer judge calls than the trained judge on every subset; on SIGNOR, Qwen 3\.5 9B, Haiku 4\.5 and Sonnet 4\.6 stop after 2\.73, 2\.20 and 2\.20 calls, against 3\.20 for the trained judge\. In Appendix[D\.5](https://arxiv.org/html/2610.40906#A4.SS5), we studied this on the full SIGNOR subset, where the off\-the\-shelf judges score overconfidently and the trained judge is conservative\.

Finding 3: A lightweight trained judge can transfer and replace off\-the\-shelf LLM judgesA trained judge that reads the evidence state beats a fixed round budget using fewer steps \(pooled agreement 0\.77 vs\. 0\.65,p=0\.0021p=0\.0021\)\. A two\-layer MLP judge trained only on SciFact\-Open, and never exposed to ProClaim\-eval, gives the highest agreement of any stopping rule on SIGNOR, ConnectomeDB and NASA, where it matches Sonnet 4\.6 \(pooled 0\.83 vs\. 0\.80,p=0\.50p=0\.50\) and outperforms Haiku 4\.5 \(0\.83 vs\. 0\.74,p=0\.022p=0\.022\), hinting at miscalibrated LLM self\-evaluation\([Liu and van der Schaar, 2025](https://arxiv.org/html/2610.40906#bib.bib39)\)\.

## 5Related Work

#### Scientific claim verification datasets\.

Prior datasets\([Diggelmann et al\., 2020](https://arxiv.org/html/2610.40906#bib.bib18);[Saakyan et al\., 2021](https://arxiv.org/html/2610.40906#bib.bib29);[Sarrouti et al\., 2021](https://arxiv.org/html/2610.40906#bib.bib30);[Wadden et al\., 2020](https://arxiv.org/html/2610.40906#bib.bib12);[Wadden et al\., 2022a](https://arxiv.org/html/2610.40906#bib.bib5)\)offer verdict labels for claim\-passage pairs, which are not suitable for evaluating the consensus claim verification setting\. While CIViC\-Fact\([Reisle et al\., 2025](https://arxiv.org/html/2610.40906#bib.bib6)\)generated claims from biomedical databases, the distinction is that it also focuses on annotating claim\-passage pairs\. A separate work\([Vittor et al\., 2026](https://arxiv.org/html/2610.40906#bib.bib49)\)validated LLMs for entries in large biomedical knowledge graphs, whereas we focus on recent entries that are difficult for LLMs and cover non\-biology domains\. Matter\-of\-Fact\([Jansen et al\., 2025](https://arxiv.org/html/2610.40906#bib.bib19)\)is a materials science benchmark that offers claim\-level labels and aims at open\-retrieval verification\. However, it focuses on the feasibility of claims\. Recent work has highlighted the mismatch between abstract\-only evidence and real verification demands\. Claims may require compiling evidence from multiple sources\([Thorne et al\., 2018](https://arxiv.org/html/2610.40906#bib.bib17);[Wadden et al\., 2022a](https://arxiv.org/html/2610.40906#bib.bib5)\), and claims do not reliably appear in paper abstracts\([Reisle et al\., 2025](https://arxiv.org/html/2610.40906#bib.bib6);[Wadden et al\., 2022b](https://arxiv.org/html/2610.40906#bib.bib13)\)\. These limitations motivate open\-domain, claim\-level evaluation for consensus claim verification\.

#### Recursive knowledge construction\.

Recent scientific agents\([Skarlinski et al\., 2024](https://arxiv.org/html/2610.40906#bib.bib26);[Mitchener et al\., 2025](https://arxiv.org/html/2610.40906#bib.bib40);[Ghareeb et al\., 2025](https://arxiv.org/html/2610.40906#bib.bib45)\)and search\-augmented verifiers\([Wei et al\., 2024](https://arxiv.org/html/2610.40906#bib.bib8);[Xie et al\., 2025](https://arxiv.org/html/2610.40906#bib.bib7);[Asai et al\., 2026](https://arxiv.org/html/2610.40906#bib.bib11)\)run a loop in which a retrieval step’s output becomes the next step’s input\([Zhang et al\., 2025](https://arxiv.org/html/2610.40906#bib.bib47)\)\. The two design choices the loop rests on, what it builds and what controls it, are compared only implicitly\. Verifiers and deep\-research systems\([Wei et al\., 2024](https://arxiv.org/html/2610.40906#bib.bib8);[Xie et al\., 2025](https://arxiv.org/html/2610.40906#bib.bib7);[Asai et al\., 2026](https://arxiv.org/html/2610.40906#bib.bib11);[Huang et al\., 2026](https://arxiv.org/html/2610.40906#bib.bib34)\)keep the state in the growing prompt, where retrieved content is used unevenly as the context lengthens\([Liu et al\., 2024](https://arxiv.org/html/2610.40906#bib.bib51)\)\. Scientific agents instead persist structure outside the context, as PaperQA2 does with ranked context state and Kosmos with a world model, but none tests whether that structure provides gains, as we show in Section[4\.2](https://arxiv.org/html/2610.40906#S4.SS2)\. The criterion for deciding when to stop the recursion is either a fixed budget\([Wei et al\., 2024](https://arxiv.org/html/2610.40906#bib.bib8);[Asai et al\., 2026](https://arxiv.org/html/2610.40906#bib.bib11)\)or the LLM’s own self\-assessment\([Asai et al\., 2024](https://arxiv.org/html/2610.40906#bib.bib20);[Xie et al\., 2025](https://arxiv.org/html/2610.40906#bib.bib7)\)\. As shown by our results in Section[4\.3](https://arxiv.org/html/2610.40906#S4.SS3), a trained judge reading the state provides a clear advantage, and the LLM assessment may sometimes be miscalibrated\([Wen et al\., 2025](https://arxiv.org/html/2610.40906#bib.bib33);[Zhou et al\., 2024](https://arxiv.org/html/2610.40906#bib.bib32);[Liu and van der Schaar, 2025](https://arxiv.org/html/2610.40906#bib.bib39)\)\.

## 6Conclusion and Future Work

We formalised consensus scientific claim verification, a setting no prior benchmark addresses\. For evaluating this verification setting, we introduced a new benchmark, ProClaim\-eval, with 718 claims based on five expert\-curated databases\. Using ProClaim to examine one design axis at a time, we found that the evidence state helps mainly through what it stores\. A trained judge that reads the evidence state outperforms fixed\-round stopping on every subset, and a lightweight trained judge can transfer and replace off\-the\-shelf LLM judges\. Together, these results support recursively updating a structured evidence state outside the model’s context as an effective design for claim verification in scientific agents\. For future work, we want to apply this design to curate consensus in scientific databases such as OptimusKG\([Vittor et al\., 2026](https://arxiv.org/html/2610.40906#bib.bib49)\)or biological meta\-databases such as OmniPath\([Turei et al\., 2021](https://arxiv.org/html/2610.40906#bib.bib38)\), which could improve the robustness of downstream scientific analyses\.

## Acknowledgments

JS was supported by the Landesinstitut für Bioinformatikinfrastruktur in Baden\-Württemberg\. LK acknowledges funding by the German Federal Ministry of Research, Technology, and Space \(BMBFTR\) with the project 01EJ2501B “Curefib”\. YHW acknowledges support from the National Science and Technology Council \(NSTC\), Taiwan, under grant no\. 114\-2917\-I\-006\-014\.

The costs associated with LLM endpoint usage were supported in the form of API credits, provided by Anthropic through the Anthropic AI for Science program\.

## Conflict of Interests

JSR reports funding from GSK, Pfizer and Sanofi and fees/honoraria from Travere Therapeutics, Stadapharm, Astex, Pfizer, Vera, Grunenthal, Tempus, Moderna and Owkin\. AD reports fees from Tempus, MONTAI and Pfizer\.

## Reproducibility statement

We explain how we set up the experiments in Section[4](https://arxiv.org/html/2610.40906#S4), and provide further details in the Appendix on experimentation and implementation\. We also attach the source code for the implementation and experiments as supplementary materials\.

## AI disclosure

In this work, we used generative AI tools to design or provide feedback on research methodology or experiments, to implement methods, to clean and reformat the dataset, and to support qualitative and thematic data analysis and aid in interpreting results\. We have not used generative AI tools to generate synthetic data sets, propose or refine hypotheses, and assist with translation\. AI uses such as helping develop theoretical models or conceptual frameworks, formulating mathematical claims, providing critical ingredients for proving mathematical claims, and assisting in the writing of proofs do not apply to this work\. Additionally, we used generative AI tools to create or modify scientific figures or images, create or edit software code, create artifacts, draft parts of a research paper, brainstorm, sourcing/searching for information, edit a research paper to improve readability, and identify relevant literature\. We have reviewed all AI\-assisted work\. We take responsibility for the final content of this work, including text, claims, or artifacts produced with the aid of generative AI\.

## References

- Akesonet al\.\(2013\)R\. L\. Akeson, X\. Chen, D\. Ciardi, M\. Crane, J\. Good, M\. Harbut, E\. Jackson, S\. R\. Kane, A\. C\. Laity, S\. Leifer, M\. Lynn, D\. L\. McElroy, M\. Papin, P\. Plavchan, S\. V\. Ramírez, R\. Rey, K\. v\. Braun, M\. Wittman, M\. Abajian, B\. Ali, C\. Beichman, A\. Beekley, G\. B\. Berriman, S\. Berukoff, G\. Bryden, B\. Chan, S\. Groom, C\. Lau, A\. N\. Payne, M\. Regelson, M\. Saucedo, M\. Schmitz, J\. Stauffer, P\. Wyatt, and A\. ZhangThe NASA Exoplanet Archive: Data and Tools for Exoplanet Research\.Publications of the Astronomical Society of the Pacific125\(930\),pp\. 989\(en\)\.External Links:ISSN 1538\-3873,[Link](https://iopscience.iop.org/article/10.1086/672273),[Document](https://dx.doi.org/10.1086/672273)Cited by:[§B\.6](https://arxiv.org/html/2610.40906#A2.SS6.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2610.40906#S1.p3.1),[§2](https://arxiv.org/html/2610.40906#S2.SS0.SSS0.Px1.p1.1)\.
- Asaiet al\.\(2026\)A\. Asai, J\. He, R\. Shao, W\. Shi, A\. Singh, J\. C\. Chang, K\. Lo, L\. Soldaini, S\. Feldman, M\. D’Arcy, D\. Wadden, M\. Latzke, J\. Sparks, J\. D\. Hwang, V\. Kishore, M\. Tian, P\. Ji, S\. Liu, H\. Tong, B\. Wu, Y\. Xiong, L\. Zettlemoyer, G\. Neubig, D\. S\. Weld, D\. Downey, W\. Yih, P\. W\. Koh, and H\. HajishirziSynthesizing scientific literature with retrieval\-augmented language models\.Nature,pp\. 1–7\(en\)\.External Links:ISSN 1476\-4687,[Link](https://www.nature.com/articles/s41586-025-10072-4),[Document](https://dx.doi.org/10.1038/s41586-025-10072-4)Cited by:[§1](https://arxiv.org/html/2610.40906#S1.p2.1),[§1](https://arxiv.org/html/2610.40906#S1.p3.1),[§2](https://arxiv.org/html/2610.40906#S2.p1.1),[1st item](https://arxiv.org/html/2610.40906#S3.I1.i1.p1.1),[2nd item](https://arxiv.org/html/2610.40906#S3.I1.i2.p1.1),[§4\.1](https://arxiv.org/html/2610.40906#S4.SS1.p1.1),[§5](https://arxiv.org/html/2610.40906#S5.SS0.SSS0.Px2.p1.1)\.
- Asaiet al\.\(2024\)A\. Asai, Z\. Wu, Y\. Wang, A\. Sil, and H\. HajishirziSelf\-RAG: Learning to Retrieve, Generate, and Critique through Self\-Reflection\.InThe International Conference on Learning Representations,\(en\)\.External Links:[Link](https://openreview.net/forum?id=hSyW5go0v8)Cited by:[2nd item](https://arxiv.org/html/2610.40906#S3.I1.i2.p1.1),[§3\.3](https://arxiv.org/html/2610.40906#S3.SS3.p1.1),[§5](https://arxiv.org/html/2610.40906#S5.SS0.SSS0.Px2.p1.1)\.
- Bachmanet al\.\(2023\)J\. A\. Bachman, B\. M\. Gyori, and P\. K\. SorgerAutomated assembly of molecular mechanisms at scale from text mining and curated databases\.Molecular Systems Biology19\(5\),pp\. MSB202211325\(en\)\.External Links:ISSN 1744\-4292,[Link](https://doi.org/10.15252/msb.202211325),[Document](https://dx.doi.org/10.15252/msb.202211325)Cited by:[§1](https://arxiv.org/html/2610.40906#S1.p1.1)\.
- Christiansenet al\.\(2025\)J\. L\. Christiansen, D\. L\. McElroy, M\. Harbut, D\. R\. Ciardi, M\. Crane, J\. Good, K\. K\. Hardegree\-Ullman, A\. Y\. Kesseli, M\. B\. Lund, M\. Lynn, A\. Muthiar, R\. Nilsson, T\. Oluyide, M\. Papin, A\. Rivera, M\. Swain, N\. D\. Susemiehl, R\. Tam, J\. v\. Eyken, and C\. BeichmanThe NASA Exoplanet Archive and Exoplanet Follow\-up Observing Program: Data, Tools, and Usage\.arXiv\.External Links:[Link](http://arxiv.org/abs/2506.03299),[Document](https://dx.doi.org/10.48550/arXiv.2506.03299)Cited by:[§B\.6](https://arxiv.org/html/2610.40906#A2.SS6.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2610.40906#S1.p3.1),[§2](https://arxiv.org/html/2610.40906#S2.SS0.SSS0.Px1.p1.1)\.
- Denget al\.\(2025\)X\. Deng, X\. Wang, and M\. StevensonThe Next Phase of Scientific Fact\-Checking: Advanced Evidence Retrieval from Complex Structured Academic Papers\.InProceedings of the 2025 International ACM SIGIR Conference on Innovative Concepts and Theories in Information Retrieval \(ICTIR\),pp\. 436–448\.Note:arXiv:2506\.20844 \[cs\]External Links:[Link](http://arxiv.org/abs/2506.20844),[Document](https://dx.doi.org/10.1145/3731120.3744614)Cited by:[§1](https://arxiv.org/html/2610.40906#S1.p2.1),[§2](https://arxiv.org/html/2610.40906#S2.p1.1)\.
- Diggelmannet al\.\(2020\)T\. Diggelmann, J\. Boyd\-Graber, J\. Bulian, M\. Ciaramita, and M\. LeippoldCLIMATE\-FEVER: A Dataset for Verification of Real\-World Climate Claims\.InThe Tackling Climate Change with Machine Learning Workshop at NeurIPS,\(en\)\.External Links:[Link](https://arxiv.org/abs/2012.00614v2)Cited by:[§1](https://arxiv.org/html/2610.40906#S1.p2.1),[§2](https://arxiv.org/html/2610.40906#S2.p1.1),[§5](https://arxiv.org/html/2610.40906#S5.SS0.SSS0.Px1.p1.1)\.
- Ghareebet al\.\(2025\)A\. E\. Ghareeb, B\. Chang, L\. Mitchener, A\. Yiu, C\. J\. Szostkiewicz, J\. M\. Laurent, M\. T\. Razzak, A\. D\. White, M\. M\. Hinks, and S\. G\. RodriquesRobin: A multi\-agent system for automating scientific discovery\.\(en\)\.External Links:[Link](https://arxiv.org/abs/2505.13400v1)Cited by:[§1](https://arxiv.org/html/2610.40906#S1.p1.1),[§5](https://arxiv.org/html/2610.40906#S5.SS0.SSS0.Px2.p1.1)\.
- Gottweiset al\.\(2026\)J\. Gottweis, W\. Weng, A\. Daryin, T\. Tu, P\. Sirkovic, A\. Myaskovsky, G\. Glowaty, F\. Weissenberger, A\. Orlandi, D\. Popovici, A\. Palepu, K\. Rong, R\. Tanno, K\. Saab, F\. Zhang, J\. Blum, A\. Carroll, K\. Kulkarni, N\. Tomašev, D\. Zverinski, I\. Rendulic, E\. Vedadi, F\. Hasler, L\. Rimanic, M\. Boia, I\. Budiselic, B\. Feinstein, M\. Bellaiche, T\. Sheffer, J\. Freyberg, J\. Ratcliff, O\. Bertolli, K\. Chou, A\. Hassidim, B\. Gokturk, A\. Vahdat, Y\. Guan, V\. Dhillon, E\. D\. Vaishnav, B\. Lee, T\. R\. D\. Costa, J\. R\. Penadés, G\. Peltz, Y\. Matias, J\. Manyika, D\. Hassabis, Y\. Xu, P\. Kohli, A\. Pawlosky, A\. Karthikesalingam, and V\. NatarajanAccelerating scientific discovery with Co\-Scientist\.Nature,pp\. 1–3\(en\)\.External Links:ISSN 1476\-4687,[Link](https://www.nature.com/articles/s41586-026-10644-y),[Document](https://dx.doi.org/10.1038/s41586-026-10644-y)Cited by:[§1](https://arxiv.org/html/2610.40906#S1.p1.1)\.
- Guoet al\.\(2022\)Z\. Guo, M\. Schlichtkrull, and A\. VlachosA Survey on Automated Fact\-Checking\.Transactions of the Association for Computational Linguistics10,pp\. 178–206\.External Links:[Link](https://aclanthology.org/2022.tacl-1.11/),[Document](https://dx.doi.org/10.1162/tacl%5Fa%5F00454)Cited by:[§1](https://arxiv.org/html/2610.40906#S1.p3.1)\.
- Heet al\.\(2021\)P\. He, J\. Gao, and W\. ChenDebertav3: improving deberta using electra\-style pre\-training with gradient\-disentangled embedding sharing\.arXiv preprint arXiv:2111\.09543\.Cited by:[§C\.4](https://arxiv.org/html/2610.40906#A3.SS4.p3.1),[§3\.3](https://arxiv.org/html/2610.40906#S3.SS3.SSS0.Px1.p1.1)\.
- Houet al\.\(2020\)R\. Hou, E\. Denisenko, H\. T\. Ong, J\. A\. Ramilowski, and A\. R\. R\. ForrestPredicting cell\-to\-cell communication networks using NATMI\.Nature Communications11\(1\),pp\. 5011\(en\)\.External Links:ISSN 2041\-1723,[Link](https://www.nature.com/articles/s41467-020-18873-z),[Document](https://dx.doi.org/10.1038/s41467-020-18873-z)Cited by:[§B\.3](https://arxiv.org/html/2610.40906#A2.SS3.SSS0.Px1.p1.1)\.
- Huanget al\.\(2026\)Y\. Huang, L\. F\. R\. Ribeiro, M\. Hardalov, B\. Dhingra, M\. Dreyer, and V\. SaligramaDeepFact: Co\-Evolving Benchmarks and Agents for Deep Research Factuality\.arXiv\.Note:arXiv:2603\.05912 \[cs\] version: 1External Links:[Link](http://arxiv.org/abs/2603.05912),[Document](https://dx.doi.org/10.48550/arXiv.2603.05912)Cited by:[§5](https://arxiv.org/html/2610.40906#S5.SS0.SSS0.Px2.p1.1)\.
- Jansenet al\.\(2025\)P\. Jansen, S\. Hassan, and R\. WangMatter\-of\-Fact: A Benchmark for Verifying the Feasibility of Literature\-Supported Claims in Materials Science\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,C\. Christodoulopoulos, T\. Chakraborty, C\. Rose, and V\. Peng \(Eds\.\),Suzhou, China,pp\. 4090–4102\.External Links:ISBN 979\-8\-89176\-332\-6,[Link](https://aclanthology.org/2025.emnlp-main.203/),[Document](https://dx.doi.org/10.18653/v1/2025.emnlp-main.203)Cited by:[§5](https://arxiv.org/html/2610.40906#S5.SS0.SSS0.Px1.p1.1)\.
- Jeonget al\.\(2024\)S\. Jeong, J\. Baek, S\. Cho, S\. J\. Hwang, and J\. ParkAdaptive\-RAG: Learning to Adapt Retrieval\-Augmented Large Language Models through Question Complexity\.InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),K\. Duh, H\. Gomez, and S\. Bethard \(Eds\.\),Mexico City, Mexico\.External Links:[Link](https://aclanthology.org/2024.naacl-long.389/),[Document](https://dx.doi.org/10.18653/v1/2024.naacl-long.389)Cited by:[§3\.3](https://arxiv.org/html/2610.40906#S3.SS3.p1.1)\.
- Jorenet al\.\(2025\)H\. Joren, J\. Zhang, C\. Ferng, D\. Juan, A\. Taly, and C\. RashtchianSufficient Context: A New Lens on Retrieval Augmented Generation Systems\.InThe International Conference on Learning Representations,\(en\)\.External Links:[Link](https://openreview.net/forum?id=Jjr2Odj8DJ)Cited by:[§3\.3](https://arxiv.org/html/2610.40906#S3.SS3.p1.1)\.
- Kondevet al\.\(2021\)F\. G\. Kondev, M\. Wang, W\. J\. Huang, S\. Naimi, and G\. AudiThe NUBASE2020 evaluation of nuclear physics properties\.Chinese Physics C45\(3\),pp\. 030001\.External Links:[Document](https://dx.doi.org/10.1088/1674-1137/abddae)Cited by:[§B\.5](https://arxiv.org/html/2610.40906#A2.SS5.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2610.40906#S2.SS0.SSS0.Px1.p1.1)\.
- Kumaret al\.\(2025\)S\. Kumar, A\. Sharma, S\. H\. Khincha, G\. Shroff, S\. R\. Singh, and R\. MishraSciClaimHunt: A Large Dataset for Evidence\-based Scientific Claim Verification\.arXiv\.Note:arXiv:2502\.10003 \[cs\]External Links:[Link](http://arxiv.org/abs/2502.10003),[Document](https://dx.doi.org/10.48550/arXiv.2502.10003)Cited by:[§1](https://arxiv.org/html/2610.40906#S1.p2.1)\.
- Liuet al\.\(2024\)N\. F\. Liu, K\. Lin, J\. Hewitt, A\. Paranjape, M\. Bevilacqua, F\. Petroni, and P\. LiangLost in the Middle: How Language Models Use Long Contexts\.Transactions of the Association for Computational Linguistics12,pp\. 157–173\.External Links:ISSN 2307\-387X,[Link](https://doi.org/10.1162/tacl_a_00638),[Document](https://dx.doi.org/10.1162/tacl%5Fa%5F00638)Cited by:[§4\.2](https://arxiv.org/html/2610.40906#S4.SS2.p4.pic1.2.1.1),[§5](https://arxiv.org/html/2610.40906#S5.SS0.SSS0.Px2.p1.1)\.
- Liuet al\.\(2026\)P\. Liu, S\. E\. B\. Maezono, W\. Lin, Y\. Yeow, Y\. Liu, Y\. Hisham, R\. Hou, J\. A\. Ramilowski, and A\. R\. R\. ForrestconnectomeDB2025: a rigorously curated, multi\-species resource of experimentally supported ligand–receptor interactions\.Nucleic Acids Research54\(D1\),pp\. D546–D554\.External Links:ISSN 1362\-4962,[Link](https://doi.org/10.1093/nar/gkaf1108),[Document](https://dx.doi.org/10.1093/nar/gkaf1108)Cited by:[§B\.3](https://arxiv.org/html/2610.40906#A2.SS3.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2610.40906#S1.p3.1),[§2](https://arxiv.org/html/2610.40906#S2.SS0.SSS0.Px1.p1.1)\.
- Liu and van der Schaar \(2025\)T\. Liu and M\. van der SchaarPosition: Truly Self\-Improving Agents Require Intrinsic Metacognitive Learning\.\(en\)\.External Links:[Link](https://openreview.net/forum?id=4KhDd0Ozqe)Cited by:[§4\.3](https://arxiv.org/html/2610.40906#S4.SS3.p3.pic1.2.1.1),[§5](https://arxiv.org/html/2610.40906#S5.SS0.SSS0.Px2.p1.1)\.
- Lo Surdoet al\.\(2025\)P\. Lo Surdo, M\. Iannuccelli, K\. Karis, E\. Meo, P\. Omidi, M\. Tosoni, S\. Graziosi, S\. Panni, M\. T\. Fiorillo, L\. Licata, F\. Sacco, B\. M\. Gyori, and L\. PerfettoSIGNOR 4\.0: the 2025 update with focus on phosphorylation data\.Nucleic Acids Research\.Note:gkaf1237, Epub ahead of printExternal Links:[Document](https://dx.doi.org/10.1093/nar/gkaf1237)Cited by:[§B\.2](https://arxiv.org/html/2610.40906#A2.SS2.p1.1),[§1](https://arxiv.org/html/2610.40906#S1.p3.1),[§2](https://arxiv.org/html/2610.40906#S2.SS0.SSS0.Px1.p1.1)\.
- Mitcheneret al\.\(2025\)L\. Mitchener, A\. Yiu, B\. Chang, M\. Bourdenx, T\. Nadolski, A\. Sulovari, E\. C\. Landsness, D\. L\. Barabasi, S\. Narayanan, N\. Evans, S\. Reddy, M\. Foiani, A\. Kamal, L\. P\. Shriver, F\. Cao, A\. T\. Wassie, J\. M\. Laurent, E\. Melville\-Green, M\. Caldas, A\. Bou, K\. F\. Roberts, S\. Zagorac, T\. C\. Orr, M\. E\. Orr, K\. J\. Zwezdaryk, A\. E\. Ghareeb, L\. McCoy, B\. Gomes, E\. A\. Ashley, K\. E\. Duff, T\. Buonassisi, T\. Rainforth, R\. J\. Bateman, M\. Skarlinski, S\. G\. Rodriques, M\. M\. Hinks, and A\. D\. WhiteKosmos: An AI Scientist for Autonomous Discovery\.arXiv\.Note:arXiv:2511\.02824 \[cs\]External Links:[Link](http://arxiv.org/abs/2511.02824),[Document](https://dx.doi.org/10.48550/arXiv.2511.02824)Cited by:[§1](https://arxiv.org/html/2610.40906#S1.p1.1),[1st item](https://arxiv.org/html/2610.40906#S3.I1.i1.p1.1),[§3\.1](https://arxiv.org/html/2610.40906#S3.SS1.p1.1),[§5](https://arxiv.org/html/2610.40906#S5.SS0.SSS0.Px2.p1.1)\.
- Müller\-Dottet al\.\(2023\)S\. Müller\-Dott, E\. Tsirvouli, M\. Vazquez, R\. O\. Ramirez Flores, P\. Badia\-i\-Mompel, R\. Fallegger, D\. Türei, A\. Lægreid, and J\. Saez\-RodriguezExpanding the coverage of regulons from high\-confidence prior knowledge for accurate estimation of transcription factor activities\.Nucleic Acids Research51\(20\),pp\. 10934–10949\.External Links:ISSN 0305\-1048,[Link](https://doi.org/10.1093/nar/gkad841),[Document](https://dx.doi.org/10.1093/nar/gkad841)Cited by:[§2](https://arxiv.org/html/2610.40906#S2.SS0.SSS0.Px1.p1.1)\.
- Neumannet al\.\(2019\)M\. Neumann, D\. King, I\. Beltagy, and W\. AmmarScispaCy: fast and robust models for biomedical natural language processing\.InProceedings of the 18th BioNLP workshop and shared task,pp\. 319–327\.Cited by:[§C\.4](https://arxiv.org/html/2610.40906#A3.SS4.p3.1)\.
- Reimers and Gurevych \(2019\)N\. Reimers and I\. GurevychSentence\-bert: sentence embeddings using siamese bert\-networks\.InProceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing \(EMNLP\-IJCNLP\),pp\. 3982–3992\.Cited by:[§C\.4](https://arxiv.org/html/2610.40906#A3.SS4.p3.1),[§3\.3](https://arxiv.org/html/2610.40906#S3.SS3.SSS0.Px1.p1.1)\.
- Reisleet al\.\(2025\)C\. Reisle, C\. J\. Grisdale, K\. Krysiak, A\. M\. Danos, M\. Khanfar, E\. Pleasance, J\. Saliba, M\. Hanos, N\. V\. Patel, A\. Jain, J\. F\. McMichael, A\. C\. Venigalla, M\. Griffith, O\. L\. Griffith, and S\. J\. M\. JonesEvaluating Language Models for Biomedical Fact\-Checking: A Benchmark Dataset for Cancer Variant Interpretation Verification\.bioRxiv\(en\)\.Note:ISSN: 2692\-8205External Links:[Link](https://www.biorxiv.org/content/10.1101/2025.09.10.675443v1),[Document](https://dx.doi.org/10.1101/2025.09.10.675443)Cited by:[§D\.1](https://arxiv.org/html/2610.40906#A4.SS1.SSS0.Px3.p1.1),[§1](https://arxiv.org/html/2610.40906#S1.p2.1),[§2](https://arxiv.org/html/2610.40906#S2.p1.1),[§5](https://arxiv.org/html/2610.40906#S5.SS0.SSS0.Px1.p1.1)\.
- Saakyanet al\.\(2021\)A\. Saakyan, T\. Chakrabarty, and S\. MuresanCOVID\-Fact: Fact Extraction and Verification of Real\-World Claims on COVID\-19 Pandemic\.InProceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing \(Volume 1: Long Papers\),C\. Zong, F\. Xia, W\. Li, and R\. Navigli \(Eds\.\),Online,pp\. 2116–2129\.External Links:[Link](https://aclanthology.org/2021.acl-long.165/),[Document](https://dx.doi.org/10.18653/v1/2021.acl-long.165)Cited by:[§1](https://arxiv.org/html/2610.40906#S1.p2.1),[§5](https://arxiv.org/html/2610.40906#S5.SS0.SSS0.Px1.p1.1)\.
- Sarroutiet al\.\(2021\)M\. Sarrouti, A\. Ben Abacha, Y\. Mrabet, and D\. Demner\-FushmanEvidence\-based Fact\-Checking of Health\-related Claims\.InFindings of the Association for Computational Linguistics: EMNLP 2021,M\. Moens, X\. Huang, L\. Specia, and S\. W\. Yih \(Eds\.\),Punta Cana, Dominican Republic,pp\. 3499–3512\.External Links:[Link](https://aclanthology.org/2021.findings-emnlp.297/),[Document](https://dx.doi.org/10.18653/v1/2021.findings-emnlp.297)Cited by:[§1](https://arxiv.org/html/2610.40906#S1.p2.1),[§5](https://arxiv.org/html/2610.40906#S5.SS0.SSS0.Px1.p1.1)\.
- Skarlinskiet al\.\(2024\)M\. D\. Skarlinski, S\. Cox, J\. M\. Laurent, J\. D\. Braza, M\. Hinks, M\. J\. Hammerling, M\. Ponnapati, S\. G\. Rodriques, and A\. D\. WhiteLanguage agents achieve superhuman synthesis of scientific knowledge\.arXiv\.Note:arXiv:2409\.13740 \[cs\]External Links:[Link](http://arxiv.org/abs/2409.13740),[Document](https://dx.doi.org/10.48550/arXiv.2409.13740)Cited by:[§1](https://arxiv.org/html/2610.40906#S1.p1.1),[§1](https://arxiv.org/html/2610.40906#S1.p2.1),[§1](https://arxiv.org/html/2610.40906#S1.p3.1),[1st item](https://arxiv.org/html/2610.40906#S3.I1.i1.p1.1),[§3\.1](https://arxiv.org/html/2610.40906#S3.SS1.p1.1),[§4\.1](https://arxiv.org/html/2610.40906#S4.SS1.p1.1),[§5](https://arxiv.org/html/2610.40906#S5.SS0.SSS0.Px2.p1.1)\.
- Thorneet al\.\(2018\)J\. Thorne, A\. Vlachos, C\. Christodoulopoulos, and A\. MittalFEVER: a Large\-scale Dataset for Fact Extraction and VERification\.InProceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 \(Long Papers\),M\. Walker, H\. Ji, and A\. Stent \(Eds\.\),New Orleans, Louisiana,pp\. 809–819\.External Links:[Link](https://aclanthology.org/N18-1074/),[Document](https://dx.doi.org/10.18653/v1/N18-1074)Cited by:[§5](https://arxiv.org/html/2610.40906#S5.SS0.SSS0.Px1.p1.1)\.
- Tuli \(1996\)J\. K\. TuliEvaluated nuclear structure data file\.Nuclear Instruments and Methods in Physics Research Section A369,pp\. 506–510\.External Links:[Document](https://dx.doi.org/10.1016/S0168-9002%2896%2980040-4)Cited by:[§1](https://arxiv.org/html/2610.40906#S1.p3.1)\.
- Tureiet al\.\(2021\)D\. Turei, A\. Valdeolivas, L\. Gul, N\. Palacio\-Escat, M\. Klein, O\. Ivanova, M\. Olbei, A\. Gabor, F\. Theis, D\. Modos, T\. Korcsmaros, and J\. Saez\-RodriguezIntegrated intra\- and intercellular signaling knowledge for multicellular omics analysis\.Molecular Systems Biology\.External Links:[Document](https://dx.doi.org/10.15252/msb.20209923)Cited by:[§B\.2](https://arxiv.org/html/2610.40906#A2.SS2.SSS0.Px2.p1.1),[§6](https://arxiv.org/html/2610.40906#S6.p1.1)\.
- van der Linden \(2022\)S\. van der LindenMisinformation: susceptibility, spread, and interventions to immunize the public\.Nature Medicine28\(3\),pp\. 460–467\(en\)\.External Links:ISSN 1546\-170X,[Link](https://www.nature.com/articles/s41591-022-01713-6),[Document](https://dx.doi.org/10.1038/s41591-022-01713-6)Cited by:[§1](https://arxiv.org/html/2610.40906#S1.p2.1)\.
- Vazquezet al\.\(2022\)M\. Vazquez, M\. Krallinger, F\. Leitner, M\. Kuiper, A\. Valencia, and A\. LaegreidExTRI: Extraction of transcription regulation interactions from literature\.Biochimica et Biophysica Acta \(BBA\) \- Gene Regulatory Mechanisms1865\(1\),pp\. 194778\.External Links:ISSN 1874\-9399,[Link](https://www.sciencedirect.com/science/article/pii/S1874939921000961),[Document](https://dx.doi.org/10.1016/j.bbagrm.2021.194778)Cited by:[§B\.4](https://arxiv.org/html/2610.40906#A2.SS4.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2610.40906#S1.p3.1),[§2](https://arxiv.org/html/2610.40906#S2.SS0.SSS0.Px1.p1.1)\.
- Vittoret al\.\(2026\)L\. Vittor, A\. Noori, I\. Arango, J\. Polonuer, S\. Rodriques, A\. White, D\. A\. Clifton, and M\. ZitnikUnifying biomedical knowledge in a modern multimodal graph\.arXiv\.External Links:[Link](http://arxiv.org/abs/2604.27269),[Document](https://dx.doi.org/10.48550/arXiv.2604.27269)Cited by:[§5](https://arxiv.org/html/2610.40906#S5.SS0.SSS0.Px1.p1.1),[§6](https://arxiv.org/html/2610.40906#S6.p1.1)\.
- Vladikaet al\.\(2025\)J\. Vladika, I\. Hacajova, and F\. MatthesStep\-by\-Step Fact Verification System for Medical Claims with Explainable Reasoning\.InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 2: Short Papers\),L\. Chiruzzo, A\. Ritter, and L\. Wang \(Eds\.\),Albuquerque, New Mexico,pp\. 805–816\.External Links:ISBN 979\-8\-89176\-190\-2,[Link](https://aclanthology.org/2025.naacl-short.68/),[Document](https://dx.doi.org/10.18653/v1/2025.naacl-short.68)Cited by:[§1](https://arxiv.org/html/2610.40906#S1.p2.1)\.
- Vladika and Matthes \(2023\)J\. Vladika and F\. MatthesScientific Fact\-Checking: A Survey of Resources and Approaches\.InFindings of the Association for Computational Linguistics: ACL 2023,A\. Rogers, J\. Boyd\-Graber, and N\. Okazaki \(Eds\.\),Toronto, Canada,pp\. 6215–6230\.External Links:[Link](https://aclanthology.org/2023.findings-acl.387/),[Document](https://dx.doi.org/10.18653/v1/2023.findings-acl.387)Cited by:[§1](https://arxiv.org/html/2610.40906#S1.p2.1),[§2](https://arxiv.org/html/2610.40906#S2.p1.1)\.
- Vladika and Matthes \(2024\)J\. Vladika and F\. MatthesComparing Knowledge Sources for Open\-Domain Scientific Claim Verification\.InProceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics \(Volume 1: Long Papers\),Y\. Graham and M\. Purver \(Eds\.\),St\. Julian’s, Malta,pp\. 2103–2114\.External Links:[Link](https://aclanthology.org/2024.eacl-long.128/),[Document](https://dx.doi.org/10.18653/v1/2024.eacl-long.128)Cited by:[§1](https://arxiv.org/html/2610.40906#S1.p2.1)\.
- Waddenet al\.\(2020\)D\. Wadden, S\. Lin, K\. Lo, L\. L\. Wang, M\. van Zuylen, A\. Cohan, and H\. HajishirziFact or Fiction: Verifying Scientific Claims\.InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),B\. Webber, T\. Cohn, Y\. He, and Y\. Liu \(Eds\.\),Online,pp\. 7534–7550\.External Links:[Link](https://aclanthology.org/2020.emnlp-main.609/),[Document](https://dx.doi.org/10.18653/v1/2020.emnlp-main.609)Cited by:[§1](https://arxiv.org/html/2610.40906#S1.p2.1),[§1](https://arxiv.org/html/2610.40906#S1.p3.1),[§2](https://arxiv.org/html/2610.40906#S2.p1.1),[§5](https://arxiv.org/html/2610.40906#S5.SS0.SSS0.Px1.p1.1)\.
- Waddenet al\.\(2022a\)D\. Wadden, K\. Lo, B\. Kuehl, A\. Cohan, I\. Beltagy, L\. L\. Wang, and H\. HajishirziSciFact\-Open: Towards open\-domain scientific claim verification\.InFindings of the Association for Computational Linguistics: EMNLP 2022,Y\. Goldberg, Z\. Kozareva, and Y\. Zhang \(Eds\.\),Abu Dhabi, United Arab Emirates,pp\. 4719–4734\.External Links:[Link](https://aclanthology.org/2022.findings-emnlp.347/),[Document](https://dx.doi.org/10.18653/v1/2022.findings-emnlp.347)Cited by:[§1](https://arxiv.org/html/2610.40906#S1.p2.1),[§2](https://arxiv.org/html/2610.40906#S2.p1.1),[§3\.3](https://arxiv.org/html/2610.40906#S3.SS3.SSS0.Px1.p1.1),[§5](https://arxiv.org/html/2610.40906#S5.SS0.SSS0.Px1.p1.1)\.
- Waddenet al\.\(2022b\)D\. Wadden, K\. Lo, L\. L\. Wang, A\. Cohan, I\. Beltagy, and H\. HajishirziMultiVerS: Improving scientific claim verification with weak supervision and full\-document context\.InFindings of the Association for Computational Linguistics: NAACL 2022,M\. Carpuat, M\. de Marneffe, and I\. V\. Meza Ruiz \(Eds\.\),Seattle, United States,pp\. 61–76\.External Links:[Link](https://aclanthology.org/2022.findings-naacl.6/),[Document](https://dx.doi.org/10.18653/v1/2022.findings-naacl.6)Cited by:[§D\.1](https://arxiv.org/html/2610.40906#A4.SS1.SSS0.Px3.p1.1),[§5](https://arxiv.org/html/2610.40906#S5.SS0.SSS0.Px1.p1.1)\.
- Wanget al\.\(2024a\)X\. Wang, Y\. Chen, L\. Yuan, Y\. Zhang, Y\. Li, H\. Peng, and H\. JiExecutable code actions elicit better LLM agents\.InProceedings of the International Conference on Machine Learning,ICML’24, Vol\.235,Vienna, Austria,pp\. 50208–50232\.Cited by:[§3\.2](https://arxiv.org/html/2610.40906#S3.SS2.p1.1)\.
- Wanget al\.\(2024b\)Y\. Wang, R\. Gangi Reddy, Z\. M\. Mujahid, A\. Arora, A\. Rubashevskii, J\. Geng, O\. Mohammed Afzal, L\. Pan, N\. Borenstein, A\. Pillai, I\. Augenstein, I\. Gurevych, and P\. NakovFactcheck\-Bench: Fine\-Grained Evaluation Benchmark for Automatic Fact\-checkers\.InFindings of the Association for Computational Linguistics: EMNLP 2024,Y\. Al\-Onaizan, M\. Bansal, and Y\. Chen \(Eds\.\),Miami, Florida, USA,pp\. 14199–14230\.External Links:[Link](https://aclanthology.org/2024.findings-emnlp.830/),[Document](https://dx.doi.org/10.18653/v1/2024.findings-emnlp.830)Cited by:[§1](https://arxiv.org/html/2610.40906#S1.p2.1),[§1](https://arxiv.org/html/2610.40906#S1.p3.1)\.
- Weiet al\.\(2024\)J\. Wei, C\. Yang, X\. Song, Y\. Lu, N\. Z\. Hu, J\. Huang, D\. Tran, D\. Peng, R\. Liu, D\. Huang, C\. Du, and Q\. V\. LeLong\-form factuality in large language models\.InThe Thirty\-eighth Annual Conference on Neural Information Processing Systems,\(en\)\.External Links:[Link](https://openreview.net/forum?id=4M9f8VMt2C)Cited by:[1st item](https://arxiv.org/html/2610.40906#S3.I1.i1.p1.1),[§5](https://arxiv.org/html/2610.40906#S5.SS0.SSS0.Px2.p1.1)\.
- Wenet al\.\(2025\)B\. Wen, J\. Yao, S\. Feng, C\. Xu, Y\. Tsvetkov, B\. Howe, and L\. L\. WangKnow Your Limits: A Survey of Abstention in Large Language Models\.Transactions of the Association for Computational Linguistics13,pp\. 529–556\.External Links:[Link](https://aclanthology.org/2025.tacl-1.26/),[Document](https://dx.doi.org/10.1162/tacl%5Fa%5F00754)Cited by:[§D\.5](https://arxiv.org/html/2610.40906#A4.SS5.p1.1),[§5](https://arxiv.org/html/2610.40906#S5.SS0.SSS0.Px2.p1.1)\.
- West and Bergstrom \(2021\)J\. D\. West and C\. T\. BergstromMisinformation in and about science\.Proceedings of the National Academy of Sciences118\(15\),pp\. e1912444117\.External Links:[Link](https://www.pnas.org/doi/10.1073/pnas.1912444117),[Document](https://dx.doi.org/10.1073/pnas.1912444117)Cited by:[§1](https://arxiv.org/html/2610.40906#S1.p2.1)\.
- Woodet al\.\(2026\)V\. Wood, M\. Jeffryes, A\. F\. Green, M\. Blum, S\. Orchard, S\. Panni, F\. Quaglia, R\. Rodriguez\-Esteban, J\. Seager, S\. C\. E\. Tosatto, U\. Wittig, and M\. HarrisonEmpowering biological knowledgebases: advances in human\-in\-the\-loop AI\-driven literature curation\.Bioinformatics Advances6\(1\),pp\. vbag028\.External Links:ISSN 2635\-0041,[Link](https://pmc.ncbi.nlm.nih.gov/articles/PMC12904773/),[Document](https://dx.doi.org/10.1093/bioadv/vbag028)Cited by:[§1](https://arxiv.org/html/2610.40906#S1.p1.1)\.
- Xieet al\.\(2025\)Z\. Xie, R\. Xing, Y\. Wang, J\. Geng, H\. Iqbal, D\. Sahnan, I\. Gurevych, and P\. NakovFIRE: Fact\-checking with Iterative Retrieval and Verification\.InFindings of the Association for Computational Linguistics: NAACL 2025,L\. Chiruzzo, A\. Ritter, and L\. Wang \(Eds\.\),Albuquerque, New Mexico,pp\. 2901–2914\.External Links:ISBN 979\-8\-89176\-195\-7,[Link](https://aclanthology.org/2025.findings-naacl.158/),[Document](https://dx.doi.org/10.18653/v1/2025.findings-naacl.158)Cited by:[§1](https://arxiv.org/html/2610.40906#S1.p3.1),[1st item](https://arxiv.org/html/2610.40906#S3.I1.i1.p1.1),[2nd item](https://arxiv.org/html/2610.40906#S3.I1.i2.p1.1),[§3\.3](https://arxiv.org/html/2610.40906#S3.SS3.p1.1),[§4\.1](https://arxiv.org/html/2610.40906#S4.SS1.p1.1),[§5](https://arxiv.org/html/2610.40906#S5.SS0.SSS0.Px2.p1.1)\.
- Yaoet al\.\(2023\)S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. R\. Narasimhan, and Y\. CaoReAct: Synergizing Reasoning and Acting in Language Models\.InProceedings of the International Conference on Learning Representations,\(en\)\.External Links:[Link](https://openreview.net/forum?id=WE_vluYUL-X)Cited by:[§4\.1](https://arxiv.org/html/2610.40906#S4.SS1.p1.1)\.
- Zhanget al\.\(2025\)A\. L\. Zhang, T\. Kraska, and O\. KhattabRecursive Language Models\.arXiv\.External Links:[Link](http://arxiv.org/abs/2512.24601),[Document](https://dx.doi.org/10.48550/arXiv.2512.24601)Cited by:[§1](https://arxiv.org/html/2610.40906#S1.p1.1),[§5](https://arxiv.org/html/2610.40906#S5.SS0.SSS0.Px2.p1.1)\.
- Zhouet al\.\(2024\)L\. Zhou, W\. Schellaert, F\. Martínez\-Plumed, Y\. Moros\-Daval, C\. Ferri, and J\. Hernández\-OralloLarger and more instructable language models become less reliable\.Nature634\(8032\),pp\. 61–68\(en\)\.External Links:ISSN 1476\-4687,[Link](https://www.nature.com/articles/s41586-024-07930-y),[Document](https://dx.doi.org/10.1038/s41586-024-07930-y)Cited by:[§D\.5](https://arxiv.org/html/2610.40906#A4.SS5.p1.1),[§5](https://arxiv.org/html/2610.40906#S5.SS0.SSS0.Px2.p1.1)\.

## Table of Contents

## Appendix ALimitations\.

Results are based on a single frontier model, Claude Sonnet 4\.6, at the time when we started our investigation\. Owing to the resource constraints, we only ran extensive analyses on a subset of ProClaim\-eval in Section[4\.2](https://arxiv.org/html/2610.40906#S4.SS2)and[4\.3](https://arxiv.org/html/2610.40906#S4.SS3)\. ProClaim\-eval covers factual claim sentences from three domains instead of free\-form claims\. While we selected the best suitable databases, the gold labels can inherit the coverage and update lag of the curated databases they come from\. Our comparisons focus on one design axis at a time within ProClaim, but ProClaim still differs in prompt and uses several times more compute, and the judge comparison uses 40 claims per subset, so small differences should be read as indicative rather than conclusive\. Finally, the trained judge relies on features and training data from biomedicine \(SciFact\-Open\) mainly, and its transfer to other fields is shown only on the NASA and NUBASE subsets\.

## Appendix BProClaim\-eval Details\.

### B\.1Verdict Label Definitions\.

Scientific literature, especially in biological and biomedical domains, does not constitute ground truth\. Individual papers report findings under specific experimental conditions, and the same phenomenon may be reported differently across studies depending on methodology, statistical power, or the state of knowledge at the time of publication\. As a result, verification should not be reduced to a binary decision\. Systems restricted tosupport/refutelabels risk systematically misclassifying claims when evidence is inconclusive, internally conflicting, or incomplete\.

We adopted a three\-way verdict taxonomy motivated by the epistemology of scientific hypothesis testing\. In this framework, a claim carries the burden of proof, where the initial position of a claim is non\-acceptance, and the evidence must affirmatively establish the claim’s validity for asupportverdict to be warranted\. This is analogous to the logic of null hypothesis significance testing, where the null is maintained until sufficient evidence is gathered to reject it\. In particular, we group active contradiction and substantive absence under therefuteverdict, which concerns evidential warrant for accepting the claim, not a metaphysical assertion that the claim is false\. Theuncertainverdict is distinct fromsupportandrefutein that relevant evidence exists and engages with the claim, but fails to resolve it\.

Verdict definitionWe used the same verdict definition \(below\) in our experimentation for baselines and ProClaim\. Minor modifications were made to the LLM\-only setting when no retrieval is performed; there would be no external evidence\.•SUPPORT:The retrieved evidence contains statements that directly corroborate the claim\. The evidence, taken at face value, is sufficient to conclude that the claim is true or highly likely true\.•REFUTE:Either \(a\) the retrieved evidence contains statements that directly contradict the claim, or \(b\) given the scope of the retrieved corpus, a thorough search yields no evidence that substantiates the claim\. In both cases, the evidence base does not support accepting the claim as true\.•UNCERTAIN \(abstention\):The retrieved evidence is relevant to the claim but is ambiguous, incomplete, or internally conflicting such that neither a clear supportive nor a clear refutatory conclusion can be drawn\. This includes cases where evidence partially supports the claim but with meaningful caveats, or where sources of comparable credibility disagree\.

### B\.2Data Generation Process: SIGNOR

We constructed one subset of ProClaim\-eval from the SIGNOR database\([Lo Surdo et al\., 2025](https://arxiv.org/html/2610.40906#bib.bib4)\), a manually curated resource of experimentally supported causal interactions between biological entities\. Each claim is expressed as a natural\-language statement of the form “\{Protein A\}directly activates/inhibits\{Protein B\}” This subset comprises two complementary edge sets: a set of ground\-truth incorrect edges that should be removed, and a set of ground\-truth correct edges that serve as positive controls\.

#### Refuteexamples\.

We established the set of ground\-truth incorrect edges by systematically comparing SIGNOR releases spanning 2018 to 2025\. Candidate removals were identified by tracking yearly changes in both SIGNOR IDs and entity names to filter out administrative identifier reassignments\. To ensure these removals reflect deliberate curation decisions rather than artifacts of entity deprecation or database restructuring, we retained only those edges whose constituent entities persisted in the October 2025 release\.

#### Supportexamples\.

To construct a matched set of positive edges, we filtered the October 2025 SIGNOR release \(26,127 interactions\)\. We prioritized 3,887 interactions involving proteins from the incorrect edge set, assuming that these entities had undergone more rigorous expert scrutiny\. These candidates were ranked by their reference counts in the OmniPath database\([Turei et al\., 2021](https://arxiv.org/html/2610.40906#bib.bib38)\)and passed through a deduplication pipeline that resolved bidirectional redundancies and selected the most\-referenced edges for each source and target node\. To ensure the benchmark’s challenge, we excluded ‘easy’ cases, where the edges are correctly classified by a small language model \(gpt\-oss\-20b\) across five independent trials\. Following final human expert adjudication, 65 edges were confirmed as the expert\-curated base set: 35Supportand 30Refute\.

#### Claim augmentation via effect flipping\.

To increase the number ofRefuteexamples beyond those derived from historically removed edges, we augmented the dataset by flipping the effect sign of 36 base edges \(changing “activates” to “inhibits” or vice versa\)\. For 33 of the 35Supportedges, the original edge is confirmed correct, so the sign\-flipped claim is labelledRefute\. For 3Refuteedges, the adjudication found the curated effect sign to be wrong, so the sign\-flipped claim is labelledSupport\. This subset therefore comprises 38Supportand 63Refute, for a total of 101 claims\.

#### Verdict curation protocol\.

We recruited three experts with molecular biology or computational biology background for curation\. To ensure the curation process had the same level of standard, we provided human curators with a structured guideline for validating signed directed protein–protein interactions \(PPIs\) sourced from existing databases\.

1. 1\.Claim parsing: extract source protein, target protein, sign \(activation/inhibition\), and mechanism\.
2. 2\.Protein characterisation: identify functional class, known aliases, and gene family membership using authoritative databases \(e\.g\., UniProt, NCBI Gene\)\.
3. 3\.Confusable name check: flag lexically similar protein names \(e\.g\., CDK2 vs\. CK2\) that are a major source of annotation error in text\-mined databases\.
4. 4\.Biological plausibility assessment: evaluate whether the source protein’s functional class is consistent with the claimed mechanism and cellular context\.
5. 5\.Literature evidence search: conduct systematic searches using gene symbols, aliases, and identified confusable names, examining retrieved papers for both supporting evidence and potential mis\-annotation sources \(e\.g\., shared regulators, pathway co\-membership\)\.
6. 6\.Evidence quality evaluation: assess experimental methodology, directness, specificity, and sign/direction consistency\.
7. 7\.Verdict assignment: classify each interaction with justifications as true, false\.

We decided the final verdict based on majority voting over independent expert annotations, where each annotator assigned a stance label to an edge\. The majority label was taken as the provisional decision, and cases without a clear majority or with substantive disagreement were escalated to adjudication\. During adjudication, experts reviewed the underlying evidence and resolved disagreements to produce a single consensus label\.

### B\.3Data Generation Process: ConnectomeDB

#### Claim creation\.

We included ligand\-receptor interactions involved in cell\-cell communication based on the ConnectomeDB database\. The most recent two releases were in 2020\([Hou et al\., 2020](https://arxiv.org/html/2610.40906#bib.bib35)\)and 2025 \(latest\)\([Liu et al\., 2026](https://arxiv.org/html/2610.40906#bib.bib16)\)\. We constructed this subset by converting curated interaction records from ConnectomeDB into a uniform natural\-language claim format “\{Ligand\}as ligand directly interacts extracellularly with\{Receptor\}as receptor” and for each claim, we extracted the primary ligand and receptor gene symbols\. Then, we attached one verdict per claim as described below\.

#### Supportexamples\.

Supportexamples were created from the newly added interaction set in the 2025 ConnectomeDB release with direct literature support from at least two publications\.

#### Refuteexamples\.

Refuteexamples were derived from data in the 2020 release that were removed during the 2025 curation process\. We retained only curator\-rejected data with specific mechanistic grounds for rejection and excluded entries without primary evidence due to the ambiguity of which evidence was considered\. The remaining rejected pairs were labelled deterministically from the curator’s comments\. Cases such as intracellular interactions, in\-cis interactions, wrong directionality, retracted papers, or non\-protein interactions were mapped to REFUTE, while cases labelled “old paper without gene symbol” were dropped\. Importantly, all examples use the same affirmative claim template\.

#### Dataset\.

The final dataset contains 305 claims: 184support, 121refute\. It combines 184 positive ligand\-receptor entries from the ConnectomeDB 2025 release with 121 expert\-curation rejected entries from the 2020 release\. Each instance includes the claim, label, expert annotator comment, ligand, and receptor symbols and identifiers\. For positive examples, provenance is taken from the Perplexity AI summary of the 2025 release curation, which embeds the supporting PubMed identifiers\. For rejected examples, provenance includes the curator comment and the PMID of the originally cited publication\. The positive subset is supported by literature spanning multiple species, with 145 human, 36 mouse, and 3 rat interactions, while the rejected subset is human\-curated and removed in the ConnectomeDB 2025 release\.

### B\.4Data Generation Process: ExTRI

#### Claim creation\.

We built this subset from the curated sentence corpus behind ExTRI\([Vazquez et al\., 2022](https://arxiv.org/html/2610.40906#bib.bib41)\), 47,373 sentences on transcription factor \(TF\) to target\-gene regulation, each labelled independently by at least two domain experts with disagreements resolved to consensus\. We did not use the text\-mined ExTRI resource, whose edges are machine output\. We kept sentences marked valid that passed quality control, excluded the complex identifiers NFKB and AP1, and grouped the remaining 24,256 sentences into 11,354 TF–target pairs\. Each claim uses the template “\{TF\}directly activates/represses transcription of\{target\}”, with the direction taken from the curated regulation sign, so the wording does not encode the label\.

#### Supportexamples\.

A pair issupportwhen all of its curated sentences agree on one regulation sign and none negates the regulation\. We further required at least five independent supporting papers, so that asupportclaim reflects accumulated evidence rather than a single report\.

#### Refuteexamples\.

A pair isrefutewhen every curated sentence for it states that the regulation does not occur \(a curated negation\), with nothing affirming it\. The claim states the curated direction affirmatively, and the label follows deterministically from the negation\. Curated negations are almost always single\-paper, sorefuteclaims are exempt from the five\-paper requirement\.

#### Filtering\.

To keep claims independent, we selected them so that no paper is shared between two claims \(1,504 candidates\)\. We removed ‘easy’ claims that a small language model \(gpt\-oss\-20b\) answered correctly in all of five trials, removed a claim whose curated sentences are all non\-mammalian, and kept at most two claims per TF for each label\.

#### Dataset\.

The final subset contains 125 claims: 49supportand 76refute\. Each instance includes the claim, label, the curated sign or negation, the supporting PubMed identifiers, and the TF and target symbols\.

### B\.5Data Generation Process: NUBASE

#### Claim creation\.

We built this subset from NUBASE2020\([Kondev et al\., 2021](https://arxiv.org/html/2610.40906#bib.bib48)\), the evaluated table of nuclear ground\- and isomeric\-state properties \(5,868 records\)\. A claim asserts that a nuclear isomeric state exists, named by its reported half\-life: “\{Element\}\-\{A\}has an isomeric state with a half\-life of about\{value\}\{unit\}”\. Tworefutestates carry only an upper limit and are phrased “…with a half\-life shorter than \{value\} \{unit\}”\. Nuclides are written with the element name \(e\.g\. Cobalt\-52\), and units are spelled out\.

#### Supportexamples\.

Supportexamples are measured isomeric states discovered in 2015 or later\. Of 84 candidates, we removed 7 that gpt\-oss\-20b answered correctly in all of five trials\.

#### Refuteexamples\.

Refuteexamples are isomeric states that NUBASE2020 flags as reported but non\-existent \(states claimed in the literature and later disproved\)\. The label follows from this flag\. States flagged as of uncertain existence were excluded\.

#### Dataset\.

The final subset contains 85 claims: 77supportand 8refute\. Each instance includes the claim, label, the NUBASE2020 status, the nuclide and the state, with the NUBASE2020 publication as evidence\.

### B\.6Data Generation Process: NASA Exoplanet Archive

#### Claim creation\.

We built this subset from the dispositions of the NASA Exoplanet Archive\([Akeson et al\., 2013](https://arxiv.org/html/2610.40906#bib.bib43);[Christiansen et al\., 2025](https://arxiv.org/html/2610.40906#bib.bib42)\), where every candidate planet carries a curated status\. All claims use the template “\{object\}is a planet orbiting the star\{star\}”, and the label follows deterministically from the archive’s disposition\.

#### Supportexamples\.

Supportexamples are planets confirmed in 2024 or later, not flagged as controversial, and backed by at least two distinct publications \(excluding the archive’s link to its own follow\-up service\)\. As for the other subsets, we removed claims that gpt\-oss\-20b answered correctly in all of five trials\.

#### Refuteexamples\.

Refuteexamples come from K2 mission objects in two groups\. All 8 planets that were published and later refuted, with their refuting papers attached, and 39 of the 242 K2 false positives\. We excluded pending candidates, false positives without stated grounds, and false\-positive flags from automated vetting pipelines, whose adjudication is not in the citable literature\.

#### Dataset\.

The final subset contains 102 claims: 55supportand 47refute\. Each instance includes the claim, label, the archive disposition, the object and its host star, and the supporting references\.

### B\.7Data Release

To preserve the dataset’s value as an evaluation benchmark, we release the full ProClaim\-eval in a serialized binary format rather than as plain text\. This prevents the dataset from being ingested during web\-crawl\-based LLM pretraining, which would risk data contamination and undermine its utility for future evaluation\. Researchers can load the dataset programmatically using our provided loading scripts\. We believe this release strategy balances open access for reproducibility with long\-term benchmark integrity\.

## Appendix CExperimental details

### C\.1Experimental Results in Section[4\.1](https://arxiv.org/html/2610.40906#S4.SS1)

Table[5](https://arxiv.org/html/2610.40906#A3.T5)reports macro\-F1 and balanced accuracy for the same runs as Table[2](https://arxiv.org/html/2610.40906#S4.T2)\.

Table 5:Macro\-F1 \(F1\) and balanced accuracy \(BAcc\) per ProClaim\-eval subset, for the same run behind each cell of Table[2](https://arxiv.org/html/2610.40906#S4.T2)\.*Random*is the mean over its 10 draws\. The shaded figure beside each macro\-F1 is ProClaim’s macro\-F1 minus that row’s, on the same subset; green where ProClaim is ahead, red where it is behind\. The highest value per column is underlined\.
### C\.2Significance Tests for the Findings

Each finding statement was tested with a paired comparison of two systems, A and B, on the same claims\. The five statements are: \(1\) the trained judge beats the Haiku 4\.5 judge and matches the Sonnet 4\.6 judge on SIGNOR, ConnectomeDB and NASA, and \(2\) the trained judge beats fixed rounds \(Finding 3\); \(3\) facts beat abstracts and full text and \(4\) ProClaim beats in\-context accumulation \(Finding 2\); and \(5\) ProClaim beats the SOTA harnesses Claude Code and Codex \(Finding 1\)\. The unit is a claim scored by both systems\. A verdict is correct \(1\) if it matches the gold label and incorrect \(0\) otherwise\.uncertain, missing and errored answers count as 0\. Pooled over the statement’s subsets,bbcounts the claims A gets right and B gets wrong, andccthe claims B gets right and A gets wrong\. We applied an exact two\-sided binomial McNemar test tobbagainstcc, and a Holm correction across the five statements\. A statement that compares A with two systems holds only if A beats both, so itspp\-value is the larger of its two tests \(an intersection–union test\)\. Every comparison uses the runs shown in the corresponding table\. Statements \(2\)–\(5\) pool all five subsets\. Statement \(1\) pools the 40\-claim samples of SIGNOR, ConnectomeDB and NASA\. Its comparison with Sonnet 4\.6 is reported with its two\-sidedpp\-value and is not part of the Holm correction\. Table[6](https://arxiv.org/html/2610.40906#A3.T6)reports, for each comparison, the number of claimsnn, the discordant countsbbandcc, the differenceΔ\\Deltabetween A’s and B’s pooled agreement, the exactpp\-value, and the statement’s Holm\-adjustedpp\-value\.

Table 6:Paired exact McNemar tests of the finding statements, pooled over subsets\.nn: claims;bb: claims A gets right and B gets wrong;cc: claims B gets right and A gets wrong;Δ\\Delta: A’s pooled agreement minus B’s;pp: exact two\-sided McNemarpp\-value;pHolmp\_\{\\text\{Holm\}\}: the statement’spp\-value \(the larger of its tests\), Holm\-adjusted across the five statements\.Statement: A vs\. BSubsetsnnbbccΔ\\DeltapppHolmp\_\{\\text\{Holm\}\}\(1\) Trained judge vs\. Haiku 4\.5 judge3×\\times40120132\+0\.0920\.00740\.022Trained judge vs\. Sonnet 4\.6 judge120128\+0\.0330\.50∗\(2\) Trained judge vs\. Fixed rounds5×\\times402003511\+0\.120<<0\.0010\.0021\(3\) Facts vs\. Abstracts5×\\times40200307\+0\.115<<0\.0010\.024Facts vs\. Full text2002410\+0\.0700\.024\(4\) ProClaim vs\. ReAct \+ subclaims−\-state5×\\times402004712\+0\.175<<0\.0010\.0012ProClaim vs\. ReAct \+ subclaims \+ facts−\-state2004214\+0\.140<<0\.001\(5\) ProClaim vs\. Claude Code5, full7189965\+0\.0470\.00980\.022ProClaim vs\. Codex71810468\+0\.0500\.0074
### C\.3LLM Thinking Mode and Temperature

All methods, including ProClaim and each baseline, were evaluated with thinking/reasoning mode disabled and a fixed LLM temperature of 0\. Disabling the thinking mode ensures that the models operate under comparable inference budgets and that any performance differences could inform the architectural and methodological choices rather than variations in internal chain\-of\-thought reasoning afforded by extended thinking tokens\. Setting the temperature to 0 maximizes the reproducibility of model outputs and comparisons\.

### C\.4Computational Resources and Cost

Table 7:Mean input \(In\) and output \(Out\) tokens per claim, in thousands, for the run behind each cell of Table[2](https://arxiv.org/html/2610.40906#S4.T2)\. Input tokens include cached input\. ProClaim counts its main agent only\.All experiments were run on an internal HPC cluster using 8 NVIDIA A100 \(80 GB\) GPUs, 8 CPU cores, and 64 GB RAM\. We served a local Qwen 3\.5 9B instance for fact extraction and called the Anthropic API \(Claude Sonnet 4\.6\) as the main agent\. For claims from ConnectomeDB \(305 cases\), total GPU time was≈\\approx40 A100 GPU\-hours\. For claims from SIGNOR \(101 cases\), the total GPU time was≈\\approx12 A100 GPU\-hours\. For claims from ExTRI \(125 cases\), NUBASE \(85 cases\) and NASA \(102 cases\), the total GPU time was≈\\approx13,≈\\approx20 and≈\\approx17 A100 GPU\-hours, respectively\. Total GPU time for the main evaluations \(Section[4\.1](https://arxiv.org/html/2610.40906#S4.SS1)\) was≈\\approx102 A100 GPU\-hours\.

Our architecture follows a modular design where computationally intensive but routine subtasks are handled by a locally hosted model at zero marginal API cost, while the main agent leverages a frontier model for high\-level planning\. Since the subagent accounts for the majority of LLM calls per claim, hosting it locally on institutional hardware substantially reduces the per\-claim cost\. The main agent could alternatively use a locally hosted model to reduce API costs\.

The primary temporal bottleneck is the per\-paper NLP feature extraction required by the sufficiency judge \(Section[D\.4](https://arxiv.org/html/2610.40906#A4.SS4)\)\. For each newly retrieved paper, the pipeline computes named entity coverage \(viaen\_core\_sci\_sm\([Neumann et al\., 2019](https://arxiv.org/html/2610.40906#bib.bib3)\),≈\\approx15 MB\), semantic similarity \(via SBERTall\-MiniLM\-L6\-v2\([Reimers and Gurevych, 2019](https://arxiv.org/html/2610.40906#bib.bib2)\),≈\\approx80 MB\), and natural language inference scores \(viacross\-encoder/nli\-deberta\-v3\-large\([He et al\., 2021](https://arxiv.org/html/2610.40906#bib.bib1)\),≈\\approx1\.5 GB\), which together account for approximately 40% of total wall time per claim\. The main agent \(Claude Sonnet 4\.6\) contributes roughly 25% of wall time, with an average latency of approximately 30 seconds per agent step\. Fact extraction via the local Qwen 3\.5 9B subagent accounts for≈\\approx10% of wall time\.

Table[7](https://arxiv.org/html/2610.40906#A3.T7)reports the mean input and output tokens per claim for every run in Table[2](https://arxiv.org/html/2610.40906#S4.T2)\. Since ProClaim’s main agent re\-sends its full execution log \(accumulated tool calls and outputs\), the input context grows substantially over a claim’s trajectory\. With prompt caching, most of this repeated prefix is billed at the cached\-input rate rather than the full input rate\.

### C\.5Prompts for Compared Methods in Section[4\.1](https://arxiv.org/html/2610.40906#S4.SS1)

We have included updated prompts for all compared methods in Appendix[E](https://arxiv.org/html/2610.40906#A5)\. When adapting SAFE, FIRE and OpenScholar, we made minimal revisions to the prompts via Claude Sonnet 4\.6 for ProClaim\-eval’s label definitions\.

## Appendix DProClaim Details

### D\.1Implementation

We implemented ProClaim in a persistent notebook\-style execution environment in which the agent alternates between narrative reasoning and executable evidence updates\. The concrete runtime uses domain\-specific Python tools for literature search, fact extraction, and stance labelling, while preserving the abstract loop defined in Section[3](https://arxiv.org/html/2610.40906#S3)\.

#### State representation\.

The evidence state is implemented as a structured data model with JSON persistence\. In addition to retrieved papers, it stores extracted facts with source identifiers and text spans, detected conflicts between papers, optional subclaim decompositions, and the history of sufficiency scores\. Because the state persists after each update, the agent can accumulate evidence across many steps without depending on the context window to remember previously processed papers\.

#### Execution layer\.

The primitive operations are realised as Python functions executed in separate kernel sessions\. In our current implementation, this includes literature search over PubMed and Semantic Scholar, batched fact extraction and stance assignment, and evidence filtering utilities that reduce neutral or irrelevant papers before sufficiency assessment\. Search operations return paper metadata and abstracts\. Extraction operations create fact\-stance pairs from abstracts or full text, if available\. The execution log transcript accumulated so far \(markdown reasoning, executed code cells, and their outputs\) is sent as user message to the LLM at each step, so the agent can condition its decisions on the full history of evidence operations and their results\.

#### Programmatic control\.

Running inside an executable kernel allows the agent to compose evidence operations with loops, conditionals, and targeted subroutines\. This is important when verification requires addressing unresolved subclaims or launching gap\-driven retrieval conditioned on the current state\. We implemented full paper text access since abstracts may not always be sufficient to resolve a claim\([Reisle et al\., 2025](https://arxiv.org/html/2610.40906#bib.bib6);[Wadden et al\., 2022b](https://arxiv.org/html/2610.40906#bib.bib13)\)\. Our implementation is restricted to accessing full texts through licensed APIs \(Unpaywall and Elsevier\) and does not redistribute available full\-text content\.

#### Audit trail\.

The notebook also functions as a reproducible trace of the verification process\. Executed code cells record evidence\-gathering actions and printed outputs, while interleaved markdown cells capture the agent’s intermediate reasoning\. The saved notebook, therefore, exposes both the final evidence state and the sequence of decisions that produced it\.

#### Caching\.

The execution log’s length directly determines the size of the input context the model must process\. The log is append\-only, where each iteration’s reasoning, executed code cells, and outputs are appended without modifying or removing earlier entries, so the prompt sent to the agent at stepttis always an exact prefix of the prompt at stept\+1t\+1\. This stability lets us mark the system prompt and the unchanged prefix of the execution log with cache breakpoints\. For Claude Sonnet 4\.6, this allows repeated turns to reuse cached prefixes at a 0\.1×\\timesread price \(with a 1\.25×\\timeswrite surcharge on first use\)\. This significantly reduces per\-claim costs without affecting the agent’s behaviour or its verdicts\.

#### Hyperparameters\.

The hyperparameterδ\\deltagives a threshold of evidence sufficiency for ProClaim to emit a verdict\. For all of our experiments,δ\\deltawas set to 0\.8\. The maximum number of iterationsTmaxT\_\{\\max\}was set to 4 for all experiments\. The hyperparameterτ\\taucontrols the conflict ratio threshold for sufficiency judge training labels \(Section[D\.4](https://arxiv.org/html/2610.40906#A4.SS4)\), and it was set to 0\.50 for the main experiment\.

### D\.2ProClaim Tools and Operations

At the outer loop, ProClaim is given only two generic tools:bashandread\_file\. We refer to all functionality exposed to ProClaim as the evidence API, and the Python function schema is injected into the system prompt\. This design encourages ProClaim to solve the task through lightweight code generation over a stable, structured interface rather than through a large collection of specialized tools\. Theread\_filetool complements this setup by allowing ProClaim to inspect persisted artifacts such asevidence\_state\.json,verdict\.json, and large intermediate outputs when standard output is insufficient\.

Below, we describe the main functions available to ProClaim\. Each subprocess begins withsetup\_workspace\(\), which initializes the verification session, loads the evidence state, and constructs handles for the LLM subagents\. This makes every subprocess idempotent, so ProClaim can resume from disk on every step without depending on hidden conversational state\. To retrieve papers, ProClaim uses retrieval functions such assearch\_pubmed\_llm\(\)andsearch\_semantic\_scholar\(\), which return identifiers for potentially relevant papers\. The functionextract\_and\_add\_facts\(\)then retrieves available full text and converts it into stance\-labelled evidence records stored in the evidence state\. Each fact remains linked to its source paper and is stored in a structured format that can be reused across iterations\.

Later iterations can target missing evidence directly throughsearch\_for\_gap\(\), where ProClaim writes the query\. The direct setting also exposes broader corpus expansion functions such assearch\_semantic\_scholar\_recommendations\(\), which let ProClaim move beyond PubMed keyword search via Semantic Scholar Recommendations seeded with papers already judged relevant\. Seeds are derived automatically from papers that produced at least onesupportfact\. Papers with onlyrefutefacts are used as negative seeds to steer results away from irrelevant directions\. Once candidate papers are identified,get\_paper\_text\(\)orget\_full\_text\_article\(\)retrieves the available paper text for downstream extraction\.

Before deciding whether it has enough evidence, ProClaim callspopulate\_paper\_features\(\)\. This function computes the natural language and metadata\-derived feature representation consumed by the sufficiency judge\. The sufficiency decision itself is made bycheck\_sufficiency\(\)to return a confidence score\. In addition, this automatically invokes LLM\-based gap findings from the current evidence state\. When the accumulated evidence is sufficient, ProClaim finalizes the run withemit\_verdict\(\), which writes a structured verdict object containing the final label, reasoning, and supporting evidence references\. If the evidence remains insufficient, ProClaim continues the loop by reading the latest sufficiency output or sufficiency history, issuing new gap\-targeted searches, and repeating the extraction and feature\-population steps\.

### D\.3Dataset for training the sufficiency judge

We intended to label a statesufficientwhen the evidence is very clear and unanimous, while any conflicting support or refute findings or added noise makes itinsufficient\. That meanssufficienthere does not mean there is strong evidence, but that the current evidence state is a clean, internally consistent stopping state\. So the model was trained to withhold a positive label unless the whole pool looks settled\.

We trained the MLP on the SciFact\-Open dataset, which contains 279 claims with labels assigned at the claim\-paper pair level\. Of these, 206 claims have at least one evidence paper labelled as eithersupportorrefute, and 73 claims have no supporting evidence\. To quantify the degree of conflict between supporting and refuting evidence, we define the minority ratio asrminority=min⁡\(Nsupport,Nrefute\)/\(Nsupport\+Nrefute\)r\_\{\\text\{minority\}\}=\\min\(N\_\{\\text\{support\}\},N\_\{\\text\{refute\}\}\)/\(N\_\{\\text\{support\}\}\+N\_\{\\text\{refute\}\}\)when at least one paper provides a stance \(i\.e\.,Nsupport\+Nrefute\>0N\_\{\\text\{support\}\}\+N\_\{\\text\{refute\}\}\>0\)\. Ifrminority≥τr\_\{\\text\{minority\}\}\\geq\\tau, the state is labelled insufficient \(y=0y=0\) and contains conflicting support/contradict evidence or irrelevant noise papers sampled for claims without evidence annotations\. For the 73 NEI claims, we performed data augmentation by randomly appending 1 to 3 irrelevant papers and labelling the resulting states as insufficient to train the model’s ability to recognise irrelevant evidence\. Otherwise, the state is categorized as sufficient \(y=1y=1\)\. In Section[3\.3](https://arxiv.org/html/2610.40906#S3.SS3), we usedτ=0\.5\\tau=0\.5as a simple criterion to decide conflicting states as they have equal amount of supporting and refuting evidence\.

For each claim\-paper pair, we extracted metadata features from PubMed/OpenAlex, including publication year, journal impact proxy, normalized citation count, and maximum authorhh\-index, together with NLP features computed from the paper full text when available, otherwise from the abstract\. The NLP features are claim–evidence semantic similarity, biomedical entity coverage, and NLI entailment/contradiction/neutral scores, computed over overlapping text chunks for long documents\.

### D\.4Sufficiency judge Implementation

For each paperPi∈ℰP\_\{i\}\\in\\mathcal\{E\}, we extracted a feature vector𝐯i=ψ⁡\(𝒞,Pi\)=\[𝐯imeta,𝐯inlp\]∈ℝk\\mathbf\{v\}\_\{i\}=\\psi\(\\mathcal\{C\},P\_\{i\}\)=\[\\mathbf\{v\}\_\{i\}^\{\\text\{meta\}\},\\mathbf\{v\}\_\{i\}^\{\\text\{nlp\}\}\]\\in\\mathbb\{R\}^\{k\}combining publication quality metadata with NLP\-based signals\. These local vectors are aggregated into a global representation𝐯global=g⁡\(𝐯1,…,𝐯N\)∈ℝd\\mathbf\{v\}\_\{\\text\{global\}\}=g\(\\mathbf\{v\}\_\{1\},\\ldots,\\mathbf\{v\}\_\{N\}\)\\in\\mathbb\{R\}^\{d\}via pooling and cross\-weighting statistics that summarize corpus\-level consensus and controversy\. These paper\-level signals are aggregated into a 10\-dimensional state representation\.

We trained a two\-hidden\-layer MLP with hidden dimensions6464and3232using class\-weighted binary cross\-entropy to predict the sufficiency label,

y^=fϕ​\(𝒞,ℰ\)=σ⁡\(MLP⁡\(𝐯global\)\),\\hat\{y\}=f\_\{\\phi\}\(\\mathcal\{C\},\\mathcal\{E\}\)=\\sigma\\\!\\left\(\\mathrm\{MLP\}\(\\mathbf\{v\}\_\{\\mathrm\{global\}\}\)\\right\),where𝐯global\\mathbf\{v\}\_\{\\mathrm\{global\}\}denotes the aggregated feature representation of the current evidence state for claim𝒞\\mathcal\{C\}\. We used a stratified70/15/1570/15/15train/validation/test split\. At inference time, the model outputs a sufficiency score for the full evidence state, which serves as the stopping criterion in the recursive retrieval loop\.

The sufficiency MLP classifier was trained to reward clean, consensus\-like states\. Its input is a 10\-dimensional vector pooled over the papers in the state: \(1\) mean claim–paper similarity, \(2\) mean entity coverage, \(3\) contradiction ratio \(the fraction of papers whose most likely NLI class is contradiction\), \(4\) controversy index \(the entropy of the papers’ NLI stance votes\), \(5\) recency\-weighted entailment \(∑ie−0\.1​ai​pient\\sum\_\{i\}e^\{\-0\.1\\,a\_\{i\}\}\\,p^\{\\text\{ent\}\}\_\{i\}, withaia\_\{i\}the paper’s age in years\), \(6\) mean normalised citation count, \(7\) mean log impact factor, \(8\) year span, \(9\) number of papers, and \(10\) number of full texts\. To ensure reliability of the sufficiency signal, a sufficient label is forced back to insufficient when the state holds fewer than 3 papers, except in the last round\. This fallback applies to every judge that can return sufficient, including the off\-the\-shelf LLM judges and the random score in Section[4\.3](https://arxiv.org/html/2610.40906#S4.SS3)\. This design aims at making the whole system keep searching unless the evidence state is both strong and structurally clean\. From our experiments, we found that the sufficiency labels are not monotonic\. Adding papers can lower consensus features or increase controversy, so a run can become sufficient and then fall back to insufficient\.

### D\.5Judge Ablation: How Does the Choice of Judge Affect ProClaim’s Performance?

We compared the trained judgeJ⁡\(c,st\)J\(c,s\_\{t\}\)with two off\-the\-shelf LLM judges, Qwen 3\.5 9B and Claude Haiku 4\.5, prompted zero\-shot on the same evidence state \(Appendix[D\.6](https://arxiv.org/html/2610.40906#A4.SS6)\), on the full SIGNOR subset\. All other settings follow Section[4\.3](https://arxiv.org/html/2610.40906#S4.SS3)\. Agreement and abstention were computed as in the main text, and precision \(P\), recall \(R\) and F1 were computed for the two gold labels\. Table[8](https://arxiv.org/html/2610.40906#A4.T8)shows that the trained judge has the highest agreement \(0\.80 vs\. 0\.71 for Qwen 3\.5 9B and 0\.76 for Haiku 4\.5\) and macro F1 \(0\.85 vs\. 0\.78 and 0\.81\), with the highestrefuterecall \(0\.78 vs\. 0\.65 and 0\.67\), while Haiku 4\.5 has the highestsupportrecall \(0\.92\)\. The trained judge also keeps the recursion running longest \(3\.21 judge calls at stop vs\. 2\.63 and 2\.48\)\. Both LLM judges assign higher sufficiency scores than the trained judge on average \(Figure[4](https://arxiv.org/html/2610.40906#A4.F4)a\)\. Haiku 4\.5’s scores cluster near 1\.0, the systematic overconfidence discussed in prior work\([Wen et al\., 2025](https://arxiv.org/html/2610.40906#bib.bib33);[Zhou et al\., 2024](https://arxiv.org/html/2610.40906#bib.bib32)\), and Qwen 3\.5 9B’s are bimodal, near one or near zero for most claims\. The trained judge is fitted on SciFact\-Open, which covers general biomedical topics, and is used on SIGNOR without domain\-specific fine\-tuning\.

Table 8:Judge ablation on SIGNOR \(101 claims\), with the evidence state, backbone, retrieval tools and stop threshold held fixed\. Agreement \(Agr\., abstention counted as a disagreement\), abstention rate \(Abst\.\), macro F1 over the two gold labels, mean judge calls at stop \(Iter\.\), and precision \(P\), recall \(R\) and F1 per gold label\.Figure 4:Effect of the judge on sufficiency scoring and ProClaim termination\.\(a\)The trained judge is conservative, assigning low scores for most claims and driving the agent to collect more evidence across multiple rounds\. Qwen 3\.5 9B exhibits a bimodal distribution, with a dominant mode near 1\.0 and a secondary mode near 0\.0\. Haiku assigns uniformly high scores with almost no mass near zero\.\(b\)The trained judge’s conservative scoring terminates ProClaim at three and four iterations, while Qwen 3\.5 9B’s bimodal pattern produces peaks at both one and four iterations, and Haiku distributes more evenly\.
### D\.6LLM Judge Prompt

To evaluate the impact of replacing our MLP sufficiency judge with an LLM, we designed an ablation study using an LLM\-based sufficiency judge\. The LLM is prompted to act as a scientific evaluator\. It assesses whether the provided reference context contains sufficient evidence to support or refute the claim\. The complete prompt template used for this LLM\-based evaluator is presented below\.

Prompt Template: LLM sufficiency judgeYou are a scientific evaluator\. Given a CLAIM and a REFERENCE \(paper titles, abstracts, and NLP features\), assess whether the REFERENCE contains sufficient evidence to support or refute the CLAIM\.Assign a sufficiency score between 0\.0 and 1\.0:•1\.0: The REFERENCE contains decisive evidence — either clearly supporting OR clearly refuting the CLAIM\. Both directions are equally "sufficient"\. Papers that unanimously refute the claim deserve the same high score as papers that unanimously support it\.•0\.0: The REFERENCE is entirely irrelevant \(entities, topic, and semantics are unrelated to the claim\), or no papers were found\.•Values in between reflect partial relevance, inconclusive evidence, or genuinely conflicting evidence\. Conflicting evidence means the papers are relevant and credible, but split in direction — after weighing their content, methodology, and credibility, no clear verdict can be reached\. Do NOT judge conflict by the ratio of papers alone; a single high\-quality paper can outweigh several weaker ones\.\#\#\# FEATURE DEFINITIONS Each paper in the REFERENCE includes pre\-computed Metadata and NLP fields:Metadata:•publication\_year: Year the paper was published\.•log\_impact\_factor: log\(1 \+ journal impact factor\)\. Higher = more prestigious venue\.•normalized\_citation\_count: Citations / paper age\. Measures community attention\.•author\_h\_index\_max: Highest H\-index among all authors\. Proxy for author credibility\.NLP \(computed relative to the CLAIM\):•claim\_entity\_coverage: Fraction of claim biomedical entities also in the paper \(0–1\)\.•semantic\_similarity: SBERT cosine similarity between claim and abstract \(0–1\)\.•nli\_entailment: DeBERTa NLI probability that thenli\_best\_chunk\_textENTAILS the claim \(0–1\)\.•nli\_contradiction: Probability thenli\_best\_chunk\_textCONTRADICTS \(refutes\) the claim \(0–1\)\.•nli\_neutral: Probability thenli\_best\_chunk\_textis NEUTRAL \(0–1\)\. Sum≈\\approx1\.•nli\_best\_chunk\_text: The most relevant text passage used to compute NLI scores\.\#\#\# CLAIM \{claim\}\#\#\# REFERENCE \{reference\}First, output a brief\#\#\# EXPLANATION\(plain text reasoning\)\. Then output\#\#\# EVALUATIONas a JSON object with exactly these keys:\{ "sufficiency\_score": <float 0\.0\-\-1\.0\>, "reasoning": "<one\-sentence summary\>" \}Output ONLY the EXPLANATION block followed by the EVALUATION JSON block\. No other text\.

### D\.7ProClaim Prompts

ProClaim System PromptYou are an evidence\-programming agent that verifies scientific claims and produces verdicts \[\{verdict\_names\}\]\.\{verdict\_definitions\}You work by writing Python code executed via the \`bash\` tool\. The \`command\`argument you pass to \`bash\` must be the literal shell command\`python3 \-c ’\.\.\.’\` with your Python code inside the quotes — NOT the barePython code by itself \(bare Python code passed directly as the shell commandwill fail with \`command not found\` / syntax errors, since bash does not knowPython syntax\)\. All evidence API functions are available as Python imports\.\#\# First step — set upCall \`bash\` with the setup code to bootstrap the workspace\. The \`command\`argument must look exactly like this \(a single shell string, \`python3 \-c\`followed by your code in quotes\):\`\`\`bashpython3 \-c ’from proclaim\.verification\.evidence\_api import setup\_workspacestate, llm, workspace = setup\_workspace\(claim="\{claim\}",workspace\_path="\{workspace\}",\)print\("Ready"\)’\`\`\`After this, every subsequent bash call must re\-load state from disk\(there is no persistent kernel\)\. Use setup\_workspace which is idempotent\(loads existing state if present\)\. Again, wrap it in \`python3 \-c ’\.\.\.’\`:\`\`\`bashpython3 \-c ’from proclaim\.verification\.evidence\_api import setup\_workspacestate, llm, workspace = setup\_workspace\(claim="\{claim\}", workspace\_path="\{workspace\}"\)\# \.\.\. your evidence API calls here \.\.\.’\`\`\`\{function\_docs\}\{schemas\}\#\# Workflow1\. Call bash with the setup code above\.2\. Decompose the claim into 1–5 atomic subclaims, each a single independentlyverifiable assertion\. Use 1 \(the original claim\) if the claim is already simpleenough to search directly\. Include alternative names or aliases for key entities\.Set subclaims before searching: \`state\.subclaims = \[\.\.\.\]\` then \`state\.\_auto\_save\(\)\`\.\{subclaim\_examples\}3\. Search \(iteration 0\):a\. call search\_pubmed\_llm\(state\.claim, state, llm\) — runs two LLM\-generated PubMed queries \(claim\-only and subclaim\-enriched\), deduplicatedb\. call search\_semantic\_scholar\_dual\(state\.claim, state, llm\) — runs two S2 queries \(claim\-only and subclaim\-enriched\), covers bioRxiv preprints and non\-MEDLINE journals4\. Extract facts: call extract\_and\_add\_facts\(llm, pmids, state, max\_workers=8\)to process all newly retrieved papers in parallel\. Do NOT loop over paper IDs\.5\. Call populate\_paper\_features\(state\) — REQUIRED before check\_sufficiency\(\)\.6\. Call filter\_papers\_by\_stance\(state\) — removes papers with only neutral facts\.7\. Check sufficiency: result = check\_sufficiency\(state, llm\); print\(result\)8\. Call get\_sufficiency\_history\(state\) to monitor the confidence trend\.9\. If insufficient: use gap\-targeted retrieval:a\. search\_for\_gap\(gap\_description, state\)b\. search\_semantic\_scholar\_recommendations\(state\) — S2 graph expansionc\. formulate\_gap\_queries\(llm, state\) — LLM\-generated gap queries10\. Repeat until confidence \>= \{sufficiency\_threshold\} or \{max\_iterations\} iterations\.\{sufficiency\_mode\_note\}11\. Call emit\_verdict\(\.\.\.\) to produce the final verdict\.\#\# Rules\- Use bash for ALL evidence API calls — write Python code\.\- Print results to stdout so you can see them\.\- ALL state mutations are auto\-saved to evidence\_state\.json\.\- \`state\` does NOT persist across bash calls — reload it each time\(or call setup\_workspace again\)\.\- When emit\_verdict is called, the verification is complete\.\- Do NOT attempt to debug, patch, or work around evidence API functions that return 0 facts\. A 0\-fact result means the paper lacks relevant evidence or accessible full text — not a tool bug\. Move on to other papers or emit a verdict\.\#\# CRITICAL: Grounded Evidence Only\- NEVER fabricate facts\. Every fact must come from a retrieved paper\.\- Use extract\_and\_add\_facts\(llm, pmids, state\) — pass the full paper ID list\.\- If extraction returns 0 for multiple paper IDs, use refine\_search\_for\_failed\_papers:failed\_ids = \[paper\_id for paper\_id, count in results\.items\(\) if count == 0\]new\_ids = refine\_search\_for\_failed\_papers\(failed\_ids, state, llm, max\_new\_papers=5\)results2 = extract\_and\_add\_facts\(llm, new\_ids, state\)\- NEVER call add\_facts\_from\_dicts with manually written text\.\- If no papers contain relevant evidence, say so in the verdict\.\- add\_extraction\_context\_note is for SYNONYM/ALIAS MAPPINGS ONLY\. Never writesearch goals, task descriptions, or paper\-specific findings into it\.\#\# Feature Computation for MLP ClassifierYou MUST call populate\_paper\_features\(state\) after extracting facts andbefore check\_sufficiency\(\)\. Without it, the MLP classifier receivesall\-zero features and predictions will be inaccurate\.Required sequence in EVERY iteration:1\. search \(PubMed \+ Semantic Scholar\)2\. extract\_and\_add\_facts\(llm, pmids, state\)3\. populate\_paper\_features\(state\) ← REQUIRED4\. filter\_papers\_by\_stance\(state\) ← REQUIRED \(removes neutral/irrelevant papers\)5\. check\_sufficiency\(state, llm\)\#\# Important\- Workspace path: \{workspace\}\- Max iterations: \{max\_iterations\}\- Sufficiency threshold: \{sufficiency\_threshold\}\#\# Common Python Mistakes to Avoid\- When building the \`reasoning\` string for emit\_verdict, compute all valuesBEFORE constructing the string\. Use f\-strings \(prefix with \`f\`\) or\`\.format\(\)\`, never bare curly braces in a regular string\.BAD: reasoning = "Found \{\{len\(facts\)\}\} facts" — literal braces, not evaluatedGOOD: n = len\(facts\); reasoning = f"Found \{\{n\}\} facts" — f\-string evaluates n

ProClaim Fact ExtractionYou are a scientific fact extraction specialist\.Given a paper and a claim with subclaims, extract every atomic fact that addresses the claim, accounting for varying terminology, synonyms, or aliases used in the text\.For each fact provide a JSON object with:\- "text": factual statement \(one sentence, self\-contained\)\- "stance": one of \{stance\_options\}\- "source\_pmid": "\{source\_pmid\}"\- "confidence": 0\.0\-1\.0, how clearly the paper states thisExample: \[\{\{"text": "Gs alpha directly stimulates adenylyl cyclase activity\.", "stance": "SUPPORT", "source\_pmid": "\{source\_pmid\}", "confidence": 0\.9\}\}\]Stance definitions:\{stance\_block\}Rules:\- Base facts strictly on the provided text\. Recognize equivalent terms, but do not hallucinate logical leaps not present in the paper\.\- If the paper has no relevance to the claim or subclaims, return \[\]\.\- Each fact must be independently verifiable from the source paper\.Now extract facts for the following:Claim: \{claim\}Subclaims:\{subclaims\_str\}\{context\_block\}Paper \(paper ID: \{source\_pmid\}\):\{paper\_text\}Output ONLY a JSON array of fact objects\. No other text\.

ProClaim PubMed Query GenerationYou are a biomedical search expert\. Generate a comprehensive PubMed query for the following scientific claim:Claim: \{claim\}Requirements:\- Include key entities \(genes, mutations, diseases, drugs, proteins\)\- Add synonyms and common variants using OR operators where appropriate\- Use appropriate field tags when beneficial \(e\.g\., \[Title/Abstract\], \[MeSH\]\)\- Keep query length reasonable \(5\-20 words\)\- Use proper PubMed syntax with Boolean operators \(AND, OR, NOT\)\- Use parentheses to group related termsExamples:\- Diagnosis claim: "JAK2 V617F is not associated with lymphoid leukemia"→ \(JAK2 AND V617F\) AND \(lymphoid leukemia OR B\-ALL OR T\-ALL OR CLL\)\- Drug resistance claim: "NT5C2 K359Q mutation does not confer resistance to Nelarabine"→ \(NT5C2 AND K359Q\) AND \(Nelarabine OR Arabinosylguanine\) AND \(resistance OR sensitivity\)\- PPI claim: "MAPK1 activates H3\-3A through phosphorylation"→ \(MAPK1 OR ERK2\) AND \(H3\-3A OR HIST1H3A\) AND \(phosphorylation OR activation\)Output ONLY the query string without any explanation or markdown formatting\.

ProClaim Semantic Scholar Query GenerationYou are a biomedical search expert\. Generate a Semantic Scholar query for the following scientific claim:Claim: \{claim\}\{subclaims\_section\}Requirements:\- Use plain keyword phrases \(Semantic Scholar does not support PubMed field tags\)\- Include alternative names, gene symbols, aliases, and common synonyms for key entities\- Keep query concise \(3\-15 words\)\- Prefer terms that appear in titles and abstracts of relevant papersExamples:\- PPI claim: "MAPK1 activates H3\-3A through phosphorylation"→ MAPK1 ERK2 H3\.3 phosphorylation activation\- Drug resistance: "NT5C2 K359Q mutation does not confer resistance to Nelarabine"→ NT5C2 K359Q nelarabine arabinosylguanine resistanceOutput ONLY the query string without any explanation or markdown formatting\.

ProClaim Gap\-Targeted Query GenerationGenerate a focused PubMed query to find evidence for this gap:Claim: \{claim\}Gap: \{gap\_description\}Generate a query that specifically targets this missing evidence\. Keep it concise \(3\-10 words\)\.Output ONLY the query string without explanation\.

## Appendix EPrompts for Baselines and Compared Methods

Static retrieval searches S2 with the claim text\. On NUBASE it searches the nuclide followed by “isomer half\-life” \(e\.g\. “Cobalt\-52 isomer half\-life”\), because the full claim sentence returns no paper for 83 of the 85 claims\.

Retrieval Final Verdict System PromptYou are a scientific claim verification expert\.Given a claim and retrieved evidence passages, assign one of the following verdicts:\- SUPPORT — The retrieved evidence contains statements that directly corroborate the claim\. The evidence, taken at face value, is sufficient to conclude that the claim is true or highly likely true\.\- REFUTE — Either \(a\) the retrieved evidence contains statements that directly contradict the claim, or \(b\) given the scope of the retrieved corpus, a thorough search yields no evidence that substantiates the claim\. In both cases, the evidence base does not support accepting the claim as true\.\- UNCERTAIN — The retrieved evidence is relevant to the claim but is ambiguous, incomplete, or internally conflicting such that neither a clear supportive nor a clear refutatory conclusion can be drawn\. This includes cases where evidence partially supports the claim but with meaningful caveats, or where sources of comparable credibility disagree\.Respond with valid JSON only — no markdown fences, no extra keys:\{"label": "SUPPORT" \| "REFUTE" \| "UNCERTAIN","reasoning": "One or two sentences citing specific evidence","evidence": \["PMID1", "PMID2"\]\}

Retrieval Final Verdict User TemplateClaim: \{claim\}Retrieved Evidence:\{evidence\}Based on the above evidence, classify the claim\. Output JSON\.

LLM\-Only Final Verdict System PromptYou are a scientific claim verification expert\.Given a claim, assess whether it is supported by established scientific knowledge:\- SUPPORT — Your knowledge of the scientific literature supports the claim as true or highly likely true based on established experimental evidence\.\- REFUTE — Your knowledge of the scientific literature contradicts the claim, or the claim is not substantiated by any known experimental evidence\.\- UNCERTAIN — The scientific literature is ambiguous, incomplete, or conflicting on this claim, or your knowledge is insufficient to make a reliable determination\.Respond with valid JSON only — no markdown fences, no extra keys:\{"label": "SUPPORT" \| "REFUTE" \| "UNCERTAIN","reasoning": "One or two sentences explaining your verdict"\}

LLM\-Only Final Verdict User TemplateClaim: \{claim\}Based on your scientific knowledge, classify the claim\. Output JSON\.

ReAct System Prompt \(Semantic Scholar\)You are a scientific claim verification agent using the ReAct framework\.Your task: determine whether a scientific claim is "SUPPORT" or "REFUTE" or "UNCERTAIN" based on evidence you retrieve from academic literature\.Verdict definitions:\- "SUPPORT" — The retrieved evidence contains statements that directly corroborate the claim\. The evidence, taken at face value, is sufficient to conclude that the claim is true or highly likely true\.\- "REFUTE" — Either \(a\) the retrieved evidence contains statements that directly contradict the claim, or \(b\) given the scope of the retrieved corpus, a thorough search yields no evidence that substantiates the claim\. In both cases, the evidence base does not support accepting the claim as true\.\- "UNCERTAIN" — The retrieved evidence is relevant to the claim but is ambiguous, incomplete, or internally conflicting such that neither a clear supportive nor a clear refutatory conclusion can be drawn\. This includes cases where evidence partially supports the claim but with meaningful caveats, or where sources of comparable credibility disagree\.Available tools:\- search\_papers\(query\): Search Semantic Scholar for academic papers\. Returns titles, abstracts, and metadata\. Use specific queries targeting the entities and relationships in the claim\.Strategy:1\. Think step\-by\-step about what evidence you need\.2\. Issue targeted search queries to find that evidence\.3\. After each search result, assess whether you have enough evidence\.4\. When confident, respond with your final verdict in the EXACT format:VERDICT: <label\>REASONING: <one or two sentences citing key evidence\>

ReAct User PromptVerify this scientific claim:\{claim\}

FIRE System PromptYou are a fact\-checking agent responsible for verifying the accuracy of claims\.

FIRE Final Answer Or Next Search PromptInstructions:1\. You are provided with a STATEMENT and relevant KNOWLEDGE points\.2\. Based on the KNOWLEDGE, assess the factual accuracy of the STATEMENT\.3\. Before presenting your conclusion, think through the process step\-by\-step\.Include a summary of the key points from the KNOWLEDGE as part of your reasoning\.4\. If the KNOWLEDGE allows you to confidently make a decision, output the finalanswer as a JSON object in the following format:\{"final\_answer": "SUPPORT" or "REFUTE" or "UNCERTAIN"\}\- "SUPPORT" \-\- The retrieved evidence contains statements that directly corroborate the claim\. The evidence, taken at face value, is sufficient to conclude that the claim is true or highly likely true\.\- "REFUTE" \-\- Either \(a\) the retrieved evidence contains statements that directly contradict the claim, or \(b\) given the scope of the retrieved corpus, a thorough search yields no evidence that substantiates the claim\. In both cases, the evidence base does not support accepting the claim as true\.\- "UNCERTAIN" \-\- The retrieved evidence is relevant to the claim but is ambiguous, incomplete, or internally conflicting such that neither a clear supportive nor a clear refutatory conclusion can be drawn\. This includes cases where evidence partially supports the claim but with meaningful caveats, or where sources of comparable credibility disagree\.5\. If the KNOWLEDGE is insufficient to make a judgment, issue ONE PubMed and Semantic Scholar searchquery that could provide additional evidence\. Output the search query in JSONformat, as follows:\{"search\_query": "Your PubMed and Semantic Scholar search query here"\}6\. The query should aim to obtain new information not already present in theKNOWLEDGE, specifically helpful for verifying the STATEMENT’s accuracy\.KNOWLEDGE:\[KNOWLEDGE\]STATEMENT:\[STATEMENT\]

OpenScholar Claim FramingIs the following scientific claim supported or refuted by the literature: \{claim\}? Answer uncertain otherwise\.

OpenScholar Claim Verdict QuestionGiven a scientific question about whether a claim is supported or refuted and a set of retrieved reference passages, output a verdict as valid JSON with exactly these keys:\{"label": "SUPPORT" \| "REFUTE" \| "UNCERTAIN","reasoning": "<one or two sentences citing specific reference numbers\>","evidence": \["\[0\]", "\[1\]"\]\}Verdict definitions:\- SUPPORT: The retrieved passages contain statements that directly corroborate the claim\. The evidence, taken at face value, is sufficient to conclude that the claim is true or highly likely true\.\- REFUTE: Either \(a\) the retrieved passages contain statements that directly contradict the claim, or \(b\) given the scope of the retrieved corpus, a thorough reading yields no evidence that substantiates the claim\. In both cases the evidence base does not support accepting the claim as true\.\- UNCERTAIN: The retrieved passages are relevant to the claim but are ambiguous, incomplete, or internally conflicting such that neither a clear supportive nor a clear refutatory conclusion can be drawn\. This includes cases where evidence partially supports the claim but with meaningful caveats, or where sources of comparable credibility disagree\.Rules:\- Base your verdict ONLY on the provided references\. Do not use outside knowledge\.\- If a passage explicitly states the opposite of the claim, that is strong evidence for REFUTE\.\- If no passage mentions the entities or relationship in the claim, prefer REFUTE over UNCERTAIN\.\- Output valid JSON only\. No markdown fences, no extra keys, no prose outside the JSON object\.Question: Is the following scientific claim supported or refuted by the literature: \{claim\}? Answer uncertain otherwise\.