SciToolAgent-Evo: An Ontology-Aware Self-Evolving Agent for Open-World Scientific Tool Acquisition

# SciToolAgent-Evo: An Ontology-Aware Self-Evolving Agent for Open-World Scientific Tool Acquisition
Source: [https://arxiv.org/html/2607.28692](https://arxiv.org/html/2607.28692)
Yuqi Tang1,2, Chenyi Zhou1, Libin Wang3, Keyan Ding2, Qiang Zhang2,3\\corresponding, Huajun Chen1,2\\corresponding

###### Abstract

Large language model \(LLM\) agents have been increasingly adopted in scientific research for organizing and invoking specialized computational tools\. However, their reliance on predefined tool spaces with static semantics limits their applicability to open\-world scientific workflows, where tool requirements, capabilities, and boundaries evolve dynamically\. To this end, we propose SciToolAgent\-Evo, an ontology\-aware self\-evolving agent for open\-world scientific tool acquisition\. Driven by an evolving memory of skills, experiences, and an ontologized tool graph, it distills generalizable knowledge from contrastive trajectories during accumulation, whereas during inference, it formulates active requests and utilizes a LinUCB\-based bandit gate to dynamically balance exploration and exploitation\. Once a novel tool is acquired, its scientific ontology is completed online for seamless integration into the known graph\. Moreover, we introduce OpenSciToolBench, a benchmark containing 900 realistic tasks across four difficulty levels\. Extensive evaluations show that SciToolAgent\-Evo achieves state\-of\-the\-art performance, validating its robustness and generalization\.

## 1Introduction

Large language model \(LLM\) agents have demonstrated remarkable intelligence in complex interactive tasks\(Minet al\.[2023](https://arxiv.org/html/2607.28692#bib.bib13); Vempralaet al\.[2024](https://arxiv.org/html/2607.28692#bib.bib16); Liuet al\.[2024](https://arxiv.org/html/2607.28692#bib.bib30); Tanget al\.[2025](https://arxiv.org/html/2607.28692#bib.bib15)\), driving their growing adoption in scientific research to organize and invoke specialized computational tools\(Wanget al\.[2024](https://arxiv.org/html/2607.28692#bib.bib45); Wuet al\.[2024](https://arxiv.org/html/2607.28692#bib.bib46); Chaudhariet al\.[2026](https://arxiv.org/html/2607.28692#bib.bib47); Ghareebet al\.[2026](https://arxiv.org/html/2607.28692#bib.bib48)\)\. However, real scientific workflows rarely begin with a complete and perfectly specified toolbox\(Mekalaet al\.[2024](https://arxiv.org/html/2607.28692#bib.bib23)\)\. For example, a scientific agent may be asked to identify a molecule satisfying multiple constraints, but the required workflow may involve a descriptor calculator or conversion tool absent from its known tool space\. In such cases, closed\-world agents are confined to existing tools and often fail when the required capabilities are missing\.

This observation suggests a shift from closed\-world tool selection to open\-world scientific tool acquisition\. Advanced scientific tool\-use frameworks, such as ChemCrow\(M\. Branet al\.[2024](https://arxiv.org/html/2607.28692#bib.bib39)\), GeneGPT\(Jinet al\.[2024](https://arxiv.org/html/2607.28692#bib.bib40)\), ChatMOF\(Kang and Kim[2024](https://arxiv.org/html/2607.28692#bib.bib41)\), and SciToolAgent\(Dinget al\.[2025](https://arxiv.org/html/2607.28692#bib.bib1)\)can retrieve, select, and invoke tools from large\-scale libraries\(Tanget al\.[2023](https://arxiv.org/html/2607.28692#bib.bib36)\)\. However, their reliance on predefined tool spaces with static semantics hinders capability\-gap recognition, leading them to passively over\-exploit the known\-tool space even when it is insufficient\. Moreover, without a self\-evolving consolidation mechanism, they struggle to reliably ground and reuse newly acquired tool knowledge in future tasks\.

Meanwhile, existing scientific tool\-use benchmarks\(Maet al\.[2024a](https://arxiv.org/html/2607.28692#bib.bib12); Chenet al\.[2025](https://arxiv.org/html/2607.28692#bib.bib22); Dinget al\.[2025](https://arxiv.org/html/2607.28692#bib.bib1)\)typically involve short tool chains, with task descriptions that contain explicit references to tool names\. Such designs make them insufficient for evaluating the core capabilities required for open\-world scientific tool acquisition: recognizing capability gaps, balancing known\-tool exploitation with unknown\-tool exploration, and continually consolidating tool knowledge\.

To address these challenges, we propose SciToolAgent\-Evo, an ontology\-aware self\-evolving agent for open\-world scientific tool acquisition\. Specifically, SciToolAgent\-Evo maintains an evolving memory consisting of a skill library, an experience pool, and an ontologized tool graph for continual knowledge consolidation and reuse\. During accumulation, the agent distills compact tool\-use knowledge from contrastive trajectories\. At inference time, it formulates active requests and employs a LinUCB\-based bandit gate to balance exploration and exploitation\. Once an unknown tool is acquired, its ontology is completed online and integrated into the known graph\. Additionally, we construct OpenSciToolBench, a hierarchical evaluation benchmark for open\-world scientific tool use\. Extensive experiments across scientific tool\-use benchmarks demonstrate that SciToolAgent\-Evo exhibits strong robustness and generalization\. Our main contributions are as follows:

- •We proposeSciToolAgent\-Evo, an ontology\-aware self\-evolving agent for open\-world scientific tool acquisition\. It integrates an evolving memory, active tool requests, and an exploration–exploitation bandit gate to enable autonomous adaptation in dynamic scientific environments\.
- •We developOpenSciToolBench, a hierarchical benchmark for open\-world scientific tool use, containing 900 realistic tasks across four difficulty levels\.
- •We conduct extensive evaluations to demonstrate the superiority of SciToolAgent\-Evo over competitive baselines\. Detailed analyses further validate its core mechanisms, including evolving\-memory consolidation, active tool requests, and the bandit gate\.

## 2Related Work

### 2\.1LLM Agents

LLM\-based agents have shown strong abilities in complex tool\-use tasks\(Shenet al\.[2023](https://arxiv.org/html/2607.28692#bib.bib28); Paranjapeet al\.[2023](https://arxiv.org/html/2607.28692#bib.bib29); Wanget al\.[2025a](https://arxiv.org/html/2607.28692#bib.bib27); Qianet al\.[2026](https://arxiv.org/html/2607.28692#bib.bib26)\)\. Frameworks such as ReAct\(Yaoet al\.[2023](https://arxiv.org/html/2607.28692#bib.bib4)\), Reflexion\(Shinnet al\.[2023](https://arxiv.org/html/2607.28692#bib.bib5)\), Toolformer\(Schicket al\.[2023](https://arxiv.org/html/2607.28692#bib.bib10)\), and ToolLLM\(Qinet al\.[2024](https://arxiv.org/html/2607.28692#bib.bib11)\)improve tool planning and API matching through in\-context learning, feedback\-driven refinement, or targeted fine\-tuning\. Recent work further extends agents to scientific domains, with systems such as SciToolAgent\(Dinget al\.[2025](https://arxiv.org/html/2607.28692#bib.bib1)\)and SciAgent\(Maet al\.[2024a](https://arxiv.org/html/2607.28692#bib.bib12)\)automating specialized computation and disciplinary reasoning\. However, these methods largely act as passive selectors within predefined scientific tool spaces, leaving capability\-gap recognition, unknown\-tool exploration, and continual knowledge consolidation unresolved\. We aim to fill this gap by introducing SciToolAgent\-Evo, an ontology\-aware self\-evolving agent for open\-world scientific tool acquisition\.

### 2\.2Tool\-Use Benchmarks

In recent years, the development of comprehensive tool\-use benchmarks has gained increasing attention\(Mekalaet al\.[2024](https://arxiv.org/html/2607.28692#bib.bib23); Luet al\.[2025](https://arxiv.org/html/2607.28692#bib.bib24); Chenet al\.[2025](https://arxiv.org/html/2607.28692#bib.bib22)\)\. Early benchmarks, such as APIBank\(Liet al\.[2023](https://arxiv.org/html/2607.28692#bib.bib17)\)and RESTBench\(Songet al\.[2023](https://arxiv.org/html/2607.28692#bib.bib18)\), mainly evaluate static tool selection and invocation under explicit specifications\. Subsequent benchmarks improve realism by introducing larger tool spaces and unseen\-tool generalization; representative examples include Gorilla\(Patilet al\.[2024](https://arxiv.org/html/2607.28692#bib.bib19)\), ToolGen\(Wanget al\.[2025b](https://arxiv.org/html/2607.28692#bib.bib20)\), and AppWorld\(Trivediet al\.[2024](https://arxiv.org/html/2607.28692#bib.bib21)\)\. In the scientific domain, SciToolEval\(Dinget al\.[2025](https://arxiv.org/html/2607.28692#bib.bib1)\)evaluates agents’ multi\-step API planning and execution capabilities, while SciToolBench\(Maet al\.[2024a](https://arxiv.org/html/2607.28692#bib.bib12)\)assesses agents’ ability to retrieve and reason with composable scientific functions\. Despite their effectiveness, these benchmarks typically involve short tool chains and task descriptions with explicit tool\-name cues, limiting their ability to evaluate the core capabilities required for open\-world scientific tool acquisition\. To this end, we construct OpenSciToolBench, a hierarchical benchmark for open\-world scientific tool use\.

## 3OpenSciToolBench

This section presents the dataset construction process of OpenSciToolBench, including its benchmark taxonomy, question generation process, and quality control protocol\.

![Refer to caption](https://arxiv.org/html/2607.28692v1/x1.png)Figure 1:Overview of OpenSciToolBench\. The benchmark contains 900 scientific tool\-use questions constructed from SciToolKG, spanning three scientific domains, four question formats, and four task difficulty levels\. The lower panel illustrates the data generation pipeline, including question generation and quality control to construct the final dataset\.### 3\.1Benchmark Taxonomy

Inspired by the Cognitive Rigor Matrix\(Hesset al\.[2009](https://arxiv.org/html/2607.28692#bib.bib42)\), we formulate four difficulty levels to characterize agents’ capability boundaries in open\-world scientific tool\-use scenarios:

- •Level\-1 \(1–2 tools\): Assessing explicit tool invocation under directly specified tool requirements\.
- •Level\-2 \(2–4 tools\): Evaluating property analysis through direct retrieval, computation, or comparison\.
- •Level\-3 \(3–6 tools\): Evaluating constraint\-aware reasoning that integrates multi\-dimensional properties under multiple scientific constraints\.
- •Level\-4 \(5–10 tools\): Challenging open\-ended research strategy formulation under realistic research goals\.

These four levels form the foundation of OpenSciToolBench and provide a systematic framework for evaluating open\-world scientific tool use\.

### 3\.2Dataset Generation

Building on SciToolKG\(Dinget al\.[2025](https://arxiv.org/html/2607.28692#bib.bib1)\), which encompasses approximately 500 scientific tools across biology, chemistry, and materials science, we first randomly sample small tool subsets, ensuring that tools within each subset belong to the same domain and are connected through input\-output compatibility\. We organize their node information \(e\.g\., descriptions, input/output types, sources\) into task specifications\. We then craft level\-specific prompts and use GPT\-5\.1\(OpenAI[2025](https://arxiv.org/html/2607.28692#bib.bib2)\)to select reasonable tool paths from each subset and generate corresponding questions, ensuring that the generated tasks are scientifically meaningful and aligned with the required competencies\.

Detailed prompts are provided in Appendix[C](https://arxiv.org/html/2607.28692#A3)\. The resulting questions span four formats: question & answer, multiple choice, content completion, and true/false, enabling a comprehensive evaluation of open\-world scientific tool use\.

### 3\.3Quality Control

To ensure scientific rigor and tool\-chain coherence, we adopt a two\-stage verification process:

- •LLM\-as\-a\-Judge\.We used advanced LLM \(e\.g\., Gemini\-3\.1\-Pro\) as the judge to automatically evaluate question clarity, task–tool alignment, and the validity of input–output dependencies within each reference tool chain\. Only instances receiving a “Yes” judgment were retained\.
- •Human Validation\.Student reviewers and domain experts then assessed the filtered samples using four predefined criteria: \(1\) Level Alignment, \(2\) Semantic Clarity, \(3\) Tool\-Chain Coherence, and \(4\) Answerability\. Only samples approved by a majority of the reviewers were accepted\.

Based on the process described above, we construct the final OpenSciToolBench, which contains 900 questions across four difficulty levels, multiple scientific domains, and four question formats\. More details about the validation protocol and dataset statistics can be found in Appendix[D](https://arxiv.org/html/2607.28692#A4)\.

## 4Methods

In this section, we introduce SciToolAgent\-Evo, an ontology\-aware self\-evolving agent for open\-world scientific tool acquisition\. As illustrated in Figure[2](https://arxiv.org/html/2607.28692#S4.F2), SciToolAgent\-Evo consists of two phases: \(1\)Knowledge Accumulation\. Given a set of training tasks𝒟t​r​a​i​n\\mathcal\{D\}\_\{train\}, the agent performs deterministic baseline sampling and temperature\-based exploratory sampling, and distills reusable knowledge from contrastive trajectories\. \(2\)Ontology\-Aware Inference\. For a test taskqtestq\_\{\\mathrm\{test\}\}, the agent retrieves memory, formulates active tool requests, and uses a LinUCB\-based bandit gate to acquire tools for executable tool\-chain construction\.

![Refer to caption](https://arxiv.org/html/2607.28692v1/x2.png)Figure 2:Overview of the SciToolAgent\-Evo framework\.Phase I\(left\) performs knowledge accumulation over training tasks through contrastive rollout generation, trajectory critique, and knowledge distillation\.Phase II\(right\) performs ontology\-aware inference through skill and task routing, active request formulation, ontology\-guided retrieval, LinUCB\-based gate control, tool\-chain execution, and knowledge update\. Together, the two phases continuously enrich a shared evolving memory, which consists of a skill library, an experience pool, and a tool graph\.### 4\.1Problem Formulation

Given a scientific tool\-use taskqq, the agent aims to construct an executable tool chain

τ=\(t1,t2,…,tm\)\.\\tau=\(t\_\{1\},t\_\{2\},\\ldots,t\_\{m\}\)\.\(1\)It then derives the final answer via sequential execution, denoted as

y^=Exec​\(q,τ\)\.\\hat\{y\}=\\mathrm\{Exec\}\(q,\\tau\)\.\(2\)Let𝒯\\mathcal\{T\}denote the complete tool space, which is partitioned into a known tool set𝒯K\\mathcal\{T\}\_\{K\}and an unknown tool set𝒯U\\mathcal\{T\}\_\{U\}\. Tools within𝒯K\\mathcal\{T\}\_\{K\}are organized as a known tool graphGK=\(VK,EK\)G\_\{K\}=\(V\_\{K\},E\_\{K\}\), whereas unknown tools remain outside this graph structure:

𝒯=𝒯K∪𝒯U,𝒯K∩𝒯U=∅\.\\mathcal\{T\}=\\mathcal\{T\}\_\{K\}\\cup\\mathcal\{T\}\_\{U\},\\quad\\mathcal\{T\}\_\{K\}\\cap\\mathcal\{T\}\_\{U\}=\\emptyset\.\(3\)During each planning step, the agent dynamically decides whether to exploit known tools fromGKG\_\{K\}or explore unknown tools within𝒯U\\mathcal\{T\}\_\{U\}\. To formalize this open\-world scientific tool acquisition process, we introduce the following four core components\.

#### Definition 1 \(Ontology\)\.

An ontologyo∈𝒪o\\in\\mathcal\{O\}is a structured representation that characterizes the semantics, invocation constraints, and scientific applicability of a tool in the known tool graph\. Formally,ot=\(bt,ht\)o\_\{t\}=\(b\_\{t\},h\_\{t\}\), wherebtb\_\{t\}denotes basic attributes, including functionality, input and output types, and category;hth\_\{t\}denotes advanced scientific attributes, including applicable objects and scientific scenarios\. To simulate realistic open\-world settings, each unknown toolu∈𝒯Uu\\in\\mathcal\{T\}\_\{U\}is initially under\-specified, with only a minimal set of basic attributes exposed\.

#### Definition 2 \(Skill\)\.

A skillki∈𝒦k\_\{i\}\\in\\mathcal\{K\}is a task\-level tool\-chain guidance unit that provides a generalizable workflow for a specific class of tasks\. Each skill is defined aski=\(Ni,Zi,Ei\)k\_\{i\}=\(N\_\{i\},Z\_\{i\},E\_\{i\}\), whereNiN\_\{i\}denotes the skill name,ZiZ\_\{i\}is a concise description of the skill, andEi=\(t1,t2,…,tm\)E\_\{i\}=\(t\_\{1\},t\_\{2\},\\ldots,t\_\{m\}\)represents reusable tool chains\(Jianget al\.[2026](https://arxiv.org/html/2607.28692#bib.bib8)\)\.

#### Definition 3 \(Experience\)\.

An experiencee∈ℰe\\in\\mathcal\{E\}is a non\-parametric prompt that provides fine\-grained guidance for tool selection\. Each experience is represented ase=\(de,se,ae,he\)e=\(d\_\{e\},s\_\{e\},a\_\{e\},h\_\{e\}\), whereded\_\{e\}denotes the associated domain,ses\_\{e\}specifies the applicable scientific situation,aea\_\{e\}gives the corresponding tool\-selection guidance, andhe∈ℝdhh\_\{e\}\\in\\mathbb\{R\}^\{d\_\{h\}\}denotes its retrieval embedding\.

#### Definition 4 \(Bandit Gate\)\.

A bandit gate is a LinUCB\-based\(Liet al\.[2010](https://arxiv.org/html/2607.28692#bib.bib9)\)decision module that balances the inherent open\-world exploration–exploitation trade\-off\. For each tool requestrjr\_\{j\}, the gate constructs an ontology\-guided, four\-dimensional context vector𝐱j=\[fj,mj,sj,pj\]⊤\\mathbf\{x\}\_\{j\}=\[f\_\{j\},m\_\{j\},s\_\{j\},p\_\{j\}\]^\{\\top\}, wherefjf\_\{j\}denotes the top\-1 known\-tool feasibility,mjm\_\{j\}denotes the confidence margin between the top\-2 candidates,sjs\_\{j\}denotes the ontology alignment score, andpjp\_\{j\}denotes the historical success prior of unknown\-tool exploration\.

With these definitions, we model each open\-world scientific tool acquisition task as a Partially Observable Markov Decision Process \(POMDP\)\. Letqqdenote a task instance\. The agent is equipped with a known tool graphGK=\(VK,EK\)G\_\{K\}=\(V\_\{K\},E\_\{K\}\), which contains structured tool attributes and tool\-level relations\. At each time steptt, the agent receives an observationoto\_\{t\}of tools retrieved from the graph, and generates an actionata\_\{t\}based on the current statests\_\{t\}\. The action corresponds to either known\-tool exploitation or unknown\-tool exploration\. This process yields a trajectoryξ=\[\(s0,a0,o0\),…,\(sT,aT,oT\)\]\\xi=\\big\[\(s\_\{0\},a\_\{0\},o\_\{0\}\),\\ldots,\(s\_\{T\},a\_\{T\},o\_\{T\}\)\\big\], which finally concludes with execution and summarization to produce the final answery^\\hat\{y\}\. The objective is to construct an evolving memoryℳ=\(GK,𝒦,ℰ\)\\mathcal\{M\}=\(G\_\{K\},\\mathcal\{K\},\\mathcal\{E\}\)that, when combined with the agent, executor, and summarizer, maximizes the probability of generating the correct answer:maxℳ⁡P​\(y^=y∗∣q,ℳ\)\\max\_\{\\mathcal\{M\}\}P\(\\hat\{y\}=y^\{\*\}\\mid q,\\mathcal\{M\}\)\.

### 4\.2Knowledge Accumulation

In long\-horizon tool\-use tasks, a single failed trajectory is too verbose and noisy to determine whether an error arises from a policy flaw or stochastic execution noise\. Therefore, without updating the backbone LLM parameters, SciToolAgent\-Evo distills tool\-use knowledge from contrastive rollouts\.

#### Contrastive Rollout Generation\.

For each training taskqi∈𝒟t​r​a​i​nq\_\{i\}\\in\\mathcal\{D\}\_\{train\}, the agent first generates the trajectoryξib​a​s​e\\xi\_\{i\}^\{base\}at temperatureTbaseT\_\{\\mathrm\{base\}\}\. Ifξib​a​s​e\\xi\_\{i\}^\{base\}succeeds, the system extracts a skillkik\_\{i\}abstracted from the trajectory and stores it in𝒦\\mathcal\{K\}\. Otherwise, the agent rolls outN−1N\-1additional trajectories\{ξi\}i=1N−1\\\{\\xi\_\{i\}\\\}\_\{i=1\}^\{N\-1\}at different temperaturesTexp=\{γn\}n=1N−1T\_\{\\mathrm\{exp\}\}=\\\{\\gamma\_\{n\}\\\}\_\{n=1\}^\{N\-1\}\. If any exploratory trajectory succeeds, the system samples one successful trajectoryξi\+\\xi\_\{i\}^\{\+\}and constructs the contrastive pair\(ξib​a​s​e,ξi\+\)\(\\xi\_\{i\}^\{base\},\\xi\_\{i\}^\{\+\}\)\.

#### Rollout Critique and Knowledge Distillation\.

Given the contrastive pair\(ξib​a​s​e,ξi\+\)\(\\xi\_\{i\}^\{base\},\\xi\_\{i\}^\{\+\}\), the agent first identifies the critical divergence and produces a diagnostic textCiC\_\{i\}to explain the corresponding failure reason\. Based onCiC\_\{i\}, the agent further distills two types of tool\-selection experiences, namely domain\-specific experienceseidome\_\{i\}^\{\\mathrm\{dom\}\}and general experienceseigene\_\{i\}^\{\\mathrm\{gen\}\}\. Before adding them to the experience pool, the system checks whether any existing entry exhibits a cosine similarity aboveθsim\\theta\_\{\\mathrm\{sim\}\}with the candidate experienceee\. If so, the candidate experience is discarded; otherwise,eeis added directly\. Detailed prompts for skill extraction and experience distillation are provided in Appendix[I](https://arxiv.org/html/2607.28692#A9)\.

MethodLevel\-1Level\-2Level\-3Level\-4OverallToolAns\.ToolAns\.ToolAns\.ToolAns\.ToolAns\.\\rowcolormodelrowA![[Uncaptioned image]](https://arxiv.org/html/2607.28692v1/x3.png)GPT\-5\.4\-miniDirect52\.66±1\.6951\.43±1\.3446\.31±1\.4943\.68±0\.7424\.71±1\.6621\.18±3\.1223\.03±2\.7921\.05±2\.2738\.11±0\.8635\.87±0\.42CoT75\.24±0\.0073\.33±0\.9566\.32±2\.9864\.56±1\.6138\.83±1\.6636\.08±2\.0428\.29±4\.6525\.43±3\.7254\.44±0\.5952\.17±0\.89RAG64\.76±0\.9562\.54±1\.4657\.89±3\.1655\.44±3\.2231\.76±2\.0428\.23±3\.1119\.74±2\.6318\.42±2\.6345\.70±0\.4843\.31±1\.37ReAct74\.06±0\.6771\.31±0\.6758\.95±2\.9854\.21±3\.7230\.58±1\.1725\.88±0\.8323\.68±3\.7219\.74±1\.3249\.45±1\.5645\.26±0\.64Reflexion78\.30±1\.3476\.19±1\.3464\.39±4\.8760\.34±3\.3840\.10±3\.0235\.29±2\.3626\.32±3\.7220\.62±2\.0154\.70±2\.1650\.69±1\.39SciToolAgent53\.33±2\.6951\.91±1\.9848\.42±1\.1647\.37±3\.0133\.94±2\.7730\.58±2\.7717\.11±1\.8614\.91±3\.0439\.85±0\.7037\.90±1\.73SciToolAgent\-Evo85\.71±0\.0083\.49±0\.5576\.84±3\.0175\.26±2\.2358\.82±1\.6556\.32±1\.1736\.84±4\.1332\.89±3\.4866\.76±0\.7464\.28±0\.85\\rowcolormodelrowB![[Uncaptioned image]](https://arxiv.org/html/2607.28692v1/x4.png)Gemini\-3\-flashDirect43\.34±0\.6740\.64±0\.6728\.42±1\.4827\.37±1\.0517\.06±0\.8316\.08±0\.686\.58±0\.006\.14±0\.7625\.48±0\.3924\.10±1\.05CoT57\.14±1\.3456\.51±1\.6143\.16±3\.0842\.45±3\.0830\.59±1\.6629\.80±0\.6826\.32±1\.8624\.12±1\.8640\.72±0\.5539\.70±0\.64RAG65\.71±0\.9662\.86±0\.0047\.37±1\.0545\.26±2\.1126\.27±2\.9623\.92±1\.8017\.11±0\.0015\.35±0\.7641\.37±0\.6439\.06±0\.83ReAct75\.24±1\.9172\.07±1\.4658\.78±2\.7353\.68±2\.2341\.51±3\.6936\.47±3\.1227\.63±0\.7623\.68±0\.0052\.94±2\.2748\.66±1\.12Reflexion84\.43±0\.9580\.31±2\.4067\.17±2\.8763\.51±4\.2654\.44±3\.6048\.24±1\.3634\.21±3\.9529\.82±1\.5262\.25±1\.3757\.71±1\.21SciToolAgent63\.81±2\.4362\.54±1\.4653\.58±3\.2853\.16±2\.2343\.53±0\.4741\.76±2\.4922\.76±4\.9321\.05±3\.9547\.65±2\.4546\.44±2\.10SciToolAgent\-Evo83\.81±0\.9582\.22±1\.3282\.11±2\.1079\.65±2\.1068\.24±5\.4365\.09±3\.7835\.53±2\.7933\.55±0\.9369\.53±2\.7767\.26±0\.96

Table 1:Evaluation results onOpenSciToolBenchwithGPT\-5\.4\-miniandGemini\-3\-flashbackbones\. Results are averaged over 3 independent runs and reported as mean±\\pmstandard deviation\. The best result isbolded, and the second best isunderlined\.ToolandAns\.indicate tool\-use and final answer accuracy\.### 4\.3Ontology\-Aware Inference

#### Skill and Task Routing\.

Given a test taskqq, the agent first checks if a relevant skill can be retrieved from𝒦\\mathcal\{K\}\. If a matched skill is found, it executes its tool chain and summarizes the final answer\. Otherwise, the agent estimates difficulty from expected tool and entity counts\. Simple tasks are solved with lightweight CoT planning, while complex tasks are routed to active tool acquisition\.

#### Active Tool Acquisition\.

The agent first retrieves relevant experiences to augment its context\. While general experiencesℰgen\\mathcal\{E\}\_\{\\mathrm\{gen\}\}provide default guidance, domain\-specific experiences are retrieved via semantic similarity:

ℰq=ℰgen∪TopK​\(ℰdq,q,K\)\.\\mathcal\{E\}\_\{q\}=\\mathcal\{E\}\_\{\\mathrm\{gen\}\}\\cup\\mathrm\{TopK\}\(\\mathcal\{E\}\_\{d\_\{q\}\},q,K\)\.\(4\)Here,dqd\_\{q\}denotes the task domain, andKKspecifies the number of retrieved experiences\.

Rather than passively selecting tools based on the raw query, the agent actively formulates tool requests via ontology\-guided reasoning\. Givenqqandℰq\\mathcal\{E\}\_\{q\}, it generates a set of active tool requestsRq=\{r1,r2,…,rl\}R\_\{q\}=\\\{r\_\{1\},r\_\{2\},\\ldots,r\_\{l\}\\\}, where eachrjr\_\{j\}is a concise requirement description for one required tool\. We justify this design in Appendix[F](https://arxiv.org/html/2607.28692#A6)\.

For each requestrjr\_\{j\}, SciToolAgent\-Evo identifies candidate tools within the known tool graphGKG\_\{K\}\. Lett∈VKt\\in V\_\{K\}denote a tool node andS​\(rj,t\)S\(r\_\{j\},t\)represent its semantic similarity to the request\. The initial retrieval set is formulated as:

Cj0=TopK​\{t∣S​\(rj,t\),∀t∈VK\}\.C\_\{j\}^\{0\}=\\mathrm\{TopK\}\\\{t\\mid S\(r\_\{j\},t\),\\forall t\\in V\_\{K\}\\\}\.\(5\)After that, graph exploration and combination reranking refineCj0C\_\{j\}^\{0\}into the final candidate setCjC\_\{j\}\.

The sub\-agent then utilizesCjC\_\{j\}to produce the 4D context vector𝐱j=\[fj,mj,sj,pj\]⊤∈ℝ4\\mathbf\{x\}\_\{j\}=\[f\_\{j\},m\_\{j\},s\_\{j\},p\_\{j\}\]^\{\\top\}\\in\\mathbb\{R\}^\{4\}for the bandit gate\. The gate selects between known\-tool exploitation and unknown\-tool exploration using a LinUCB policy:

aj=arg⁡maxa∈\{a1,a2\}⁡\(𝜽^a⊤​𝐱j\+α​𝐱j⊤​Aa−1​𝐱j\),\\begin\{aligned\} a\_\{j\}=\\arg\\max\_\{a\\in\\\{\\texttt\{a1\},\\texttt\{a2\}\\\}\}\\left\(\\hat\{\\boldsymbol\{\\theta\}\}\_\{a\}^\{\\top\}\\mathbf\{x\}\_\{j\}\+\\alpha\\sqrt\{\\mathbf\{x\}\_\{j\}^\{\\top\}A\_\{a\}^\{\-1\}\\mathbf\{x\}\_\{j\}\}\\right\),\\end\{aligned\}

\(6\)
wherea1anda2denote known\-tool exploitation and unknown\-tool exploration, respectively\. Here,𝜽^a=Aa−1​𝐛a\\hat\{\\boldsymbol\{\\theta\}\}\_\{a\}=A\_\{a\}^\{\-1\}\\mathbf\{b\}\_\{a\}estimates the reward of actionaa, andα\\alphacontrols the exploration strength\. For exploitation, the agent selects the top\-ranked candidate fromCjC\_\{j\}; for exploration, it infers tool utility from sparse attributes and retrieves the most compatible candidate from𝒯U\\mathcal\{T\}\_\{U\}\. The action is applied iteratively until a complete tool chainτ^\\hat\{\\tau\}is constructed\. Finally, the Executor runsτ^\\hat\{\\tau\}sequentially, and the Summarizer producesy^\\hat\{y\}\.

#### Knowledge Update\.

Inspired by human learning, successful exploration is consolidated into reusable knowledge\. When a rollout successfully utilizes an unknown tool, the agent completes its ontologyoto\_\{t\}from the interaction history and migrates it into the known tool graphGKG\_\{K\}\. The same execution signal provides a binary rewardr∈\{0,1\}r\\in\\\{0,1\\\}for updating the parameters of the selected actionaa:Aa←Aa\+𝐱j​𝐱j⊤,𝐛a←𝐛a\+r​𝐱jA\_\{a\}\\leftarrow A\_\{a\}\+\\mathbf\{x\}\_\{j\}\\mathbf\{x\}\_\{j\}^\{\\top\},\\mathbf\{b\}\_\{a\}\\leftarrow\\mathbf\{b\}\_\{a\}\+r\\mathbf\{x\}\_\{j\}\. In addition, the resulting trajectory is abstracted into a skillknewk\_\{\\mathrm\{new\}\}and stored in𝒦\\mathcal\{K\}\. Together, these updates establish a closed self\-evolving loop, enabling the agent to adapt continuously in open\-world environments\.

MethodLevel\-1Level\-2OverallToolAns\.ToolAns\.ToolAns\.\\rowcolormodelrowA![[Uncaptioned image]](https://arxiv.org/html/2607.28692v1/x5.png)GPT\-5\.4\-miniDirect53\.29±0\.9351\.75±1\.2446\.43±1\.6845\.24±1\.9849\.01±0\.7047\.69±0\.81CoT86\.84±1\.8685\.53±1\.3272\.62±2\.8171\.43±2\.3877\.97±2\.4576\.73±1\.73RAG76\.32±1\.3274\.12±0\.7648\.94±1\.9948\.41±2\.1059\.24±1\.5158\.09±1\.51ReAct83\.56±3\.4882\.45±2\.7469\.05±3\.4668\.25±3\.4674\.51±2\.7073\.59±2\.23Reflexion87\.28±2\.2886\.39±2\.0170\.63±1\.6868\.78±1\.9976\.89±1\.2475\.41±2\.20SciToolAgent62\.50±2\.7961\.18±2\.7946\.43±1\.6845\.24±1\.1252\.48±2\.1051\.24±1\.75SciToolAgent\-Evo90\.13±0\.9388\.60±0\.7680\.16±3\.3779\.37±3\.3783\.91±1\.7582\.84±1\.87\\rowcolormodelrowB![[Uncaptioned image]](https://arxiv.org/html/2607.28692v1/x6.png)Gemini\-3\-flashDirect55\.26±1\.8653\.95±1\.8625\.00±2\.8025\.00±2\.8036\.37±2\.4535\.89±2\.45CoT87\.50±0\.9386\.84±0\.7651\.98±2\.8150\.79±2\.2565\.34±2\.1064\.35±1\.40RAG78\.07±3\.3176\.75±3\.3153\.44±1\.2152\.38±1\.5962\.71±1\.5961\.55±1\.87ReAct85\.53±2\.7885\.09±1\.5269\.84±1\.1269\.04±0\.5775\.74±1\.7575\.08±0\.76Reflexion88\.82±1\.3288\.16±1\.4265\.48±3\.9664\.68±2\.7174\.26±1\.5173\.51±1\.03SciToolAgent71\.06±3\.7270\.39±2\.7955\.17±2\.7654\.81±2\.1861\.14±1\.4060\.67±0\.33SciToolAgent\-Evo87\.72±0\.8886\.84±1\.8773\.41±1\.6872\.62±1\.3278\.79±1\.4077\.97±1\.75

Table 2:Evaluation results onSciToolEvalwithGPT\-5\.4\-miniandGemini\-3\-flashbackbones\. Results are averaged over 3 independent runs and reported as mean±\\pmstandard deviation\. The best result isbolded, and the second best isunderlined\.ToolandAns\.indicate tool\-use and final answer accuracy\.## 5Experiments

### 5\.1Evaluation Benchmarks and Metrics

We conduct a comprehensive evaluation of SciToolAgent\-Evo on two scientific tool\-use benchmarks:

- •OpenSciToolBench: Contains 900 scientific tool\-use questions to assess capabilities ranging from explicit tool invocation to open\-ended research strategy formulation\.
- •SciToolEval\(Dinget al\.[2025](https://arxiv.org/html/2607.28692#bib.bib1)\): Features 531 questions comprising 152 single\-tool and 379 multi\-tool tasks, covering molecular property prediction, protein analysis, and materials retrieval\.

For both benchmarks, we applyTool Accuracy\(measuring tool\-chain alignment with the reference\) andAnswer Accuracy\(assessing final\-answer correctness\) as the final metrics\.

### 5\.2Baselines

We evaluate SciToolAgent\-Evo under two proprietary LLM backbones, GPT\-5\.4\-mini\(OpenAI[2025](https://arxiv.org/html/2607.28692#bib.bib2)\)and Gemini\-3\-flash\(Teamet al\.[2023](https://arxiv.org/html/2607.28692#bib.bib25)\), and compare it against various representative methods:Directgenerates the final answer directly without producing intermediate reasoning;CoT\(Weiet al\.[2022](https://arxiv.org/html/2607.28692#bib.bib6)\)produces an intermediate reasoning trajectory before answering;RAG\(Lewiset al\.[2020](https://arxiv.org/html/2607.28692#bib.bib43)\)retrieves relevant external knowledge to augment answer generation;ReAct\(Yaoet al\.[2023](https://arxiv.org/html/2607.28692#bib.bib4)\)andReflexion\(Shinnet al\.[2023](https://arxiv.org/html/2607.28692#bib.bib5)\)serve as interactive tool\-use methods, relying on the Thought\-Action\-Observation loop and trajectory\-level self\-feedback for planning respectively; andSciToolAgent\(Dinget al\.[2025](https://arxiv.org/html/2607.28692#bib.bib1)\)leverages a tool knowledge graph for tool retrieval, tool\-chain planning, and execution\. More details of these baselines are presented in Appendix[H](https://arxiv.org/html/2607.28692#A8)\.

### 5\.3Implementation Details

We employ locally deployed Qwen3\-Embedding\-0\.6B and Qwen3\-1\.7B\(Yanget al\.[2025](https://arxiv.org/html/2607.28692#bib.bib7)\)as our retrieval model and reranking model, respectively\. For all baselines, we use the same open\-world split, with 50% of the toolset designated as known and the remaining 50% as unknown\. During accumulation, we sampleK=4K=4trajectories for each task, including one deterministic baseline trajectory withTbase=0T\_\{\\mathrm\{base\}\}=0and three exploratory trajectories withTexp∈\{0\.2,0\.5,0\.8\}T\_\{\\mathrm\{exp\}\}\\in\\\{0\.2,0\.5,0\.8\\\}\. At test time, we use a LinUCB\-based bandit gate withα0=0\.8\\alpha\_\{0\}=0\.8, decay rate0\.010\.01,αmin=0\.2\\alpha\_\{\\min\}=0\.2, and a 20\-step failure window to scaleα\\alphawithin\[0\.7,1\.3\]\[0\.7,1\.3\]\. More implementation details are provided in Appendix[G](https://arxiv.org/html/2607.28692#A7)\.

### 5\.4Main Results

Tables[4\.2](https://arxiv.org/html/2607.28692#S4.SS2.SSS0.Px2)and[4\.3](https://arxiv.org/html/2607.28692#S4.SS3.SSS0.Px3)show the results across two benchmarks and two backbone models\. Our method demonstrates substantial improvements over all baselines\. Compared with the runner\-up results, SciToolAgent\-Evo improves overall answer accuracy on OpenSciToolBench by 12\.11 and 9\.55 points with GPT\-5\.4\-mini and Gemini\-3\-flash, respectively\. On SciToolEval, the corresponding gains are 6\.11 and 2\.89 points\. Notably, SciToolAgent\-Evo exceeds SciToolAgent by a striking 17% to 31% margin across all benchmarks\. This gain reflects a key distinction: SciToolAgent remains constrained by a fixed known\-tool graph, whereas SciToolAgent\-Evo dynamically balances exploration and exploitation and consolidates acquired knowledge to enable reliable acquisition of unknown tools, yielding stronger open\-world adaptability\.

From a fine\-grained analysis across difficulty levels, we observe that while the performance gap is marginal on simpler tasks \(Level\-1\), SciToolAgent\-Evo demonstrates a significant advantage in solving complex multi\-tool tasks\. For instance, on OpenSciToolBench with GPT\-5\.4\-mini, SciToolAgent\-Evo achieves 56\.32% answer accuracy on Level\-3, surpassing Reflexion and ReAct by more than 20 and 30 points\. Empirically, we observe that in\-context learning methods, such as CoT, ReAct, and Reflexion, rely heavily on local step\-wise reasoning and lack a stable global strategy for long tool\-chain planning\. In open tool spaces with vast and initially unobserved candidates, this deficiency inevitably incurs high exploration costs and degrades tool\-planning accuracy\. SciToolAgent\-Evo instead maintains an evolving memory, reducing exploration costs and guaranteeing robust adaptation in open\-world environments\.

MethodLevel\-1Level\-2Level\-3Level\-4Overallw/o Knowledge Accumulation80\.00↓\\downarrow5\.7157\.89↓\\downarrow18\.9545\.88↓\\downarrow14\.9314\.47↓\\downarrow22\.3752\.35↓\\downarrow14\.87w/o Tool Migration78\.10↓\\downarrow7\.6162\.11↓\\downarrow14\.7341\.17↓\\downarrow19\.6413\.16↓\\downarrow23\.6851\.53↓\\downarrow15\.69w/o Inference Migration81\.90↓\\downarrow3\.8172\.63↓\\downarrow4\.2156\.47↓\\downarrow4\.3435\.53↓\\downarrow1\.3163\.71↓\\downarrow3\.51w/o Skill83\.81↓\\downarrow1\.9073\.68↓\\downarrow3\.1654\.12↓\\downarrow6\.6935\.53↓\\downarrow1\.3163\.99↓\\downarrow3\.23w/o Experience85\.71↓\\downarrow0\.0075\.79↓\\downarrow1\.0557\.65↓\\downarrow3\.1634\.21↓\\downarrow2\.6365\.65↓\\downarrow1\.57\\rowcolormodelgrayOurs \- Full Pipeline85\.7176\.8460\.8136\.8467\.22

Table 3:Ablation study onOpenSciToolBench\. All variants use GPT\-5\.4\-mini as the backbone\. We remove key components to assess their contributions, and↓\\downarrowdenotes the absolute performance drop from the full pipeline\.![Refer to caption](https://arxiv.org/html/2607.28692v1/x7.png)Figure 3:Performance sensitivity to known/unknown tool splits onOpenSciToolBenchwith GPT\-5\.4\-mini\.
### 5\.5Ablation Study

#### Component Ablation\.

We conduct a systematic ablation study on OpenSciToolBench with GPT\-5\.4\-mini to analyze the contribution of each component\. As shown in Table[3](https://arxiv.org/html/2607.28692#S5.T3), we obtain the following results: \(1\) Removing knowledge accumulation causes a 14\.87\-point drop, showing its necessity for robust adaptation in open\-world scientific tool settings\. \(2\) Disabling unknown tool migration entirely leads to the largest degradation\. Conversely, disabling it only during inference causes a much smaller drop, underscoring the necessity of continual tool migration\. \(3\) Excluding skills and experiences degrades performance by 3\.23 and 1\.57 points, respectively, demonstrating that both knowledge forms are critical\. These results indicate that each component addresses a distinct challenge in open\-world scientific tool use, and their integration enables continual refinement of tool memory and acquisition policy\. Further fine\-grained ablation results for the inference pipeline are reported in Appendix[B1](https://arxiv.org/html/2607.28692#A2.SS1)\.

#### Known/Unknown Split Sensitivity\.

To further validate the robustness of our method, we evaluate SciToolAgent\-Evo on OpenSciToolBench across four known/unknown tool splits using GPT\-5\.4\-mini\. As shown in Figure[3](https://arxiv.org/html/2607.28692#S5.F3), overall accuracy generally improves as the known\-tool ratio grows, rising from 59\.01% under the 30/70 split to 66\.63% under the 60/40 split\. This suggests that a richer known\-tool graph reduces open\-world exploration difficulty and stabilizes tool\-chain planning\. Crucially, even when 70% of tools are initially unknown, SciToolAgent\-Evo still achieves a competitive 59\.01% overall accuracy\. This result demonstrates the framework’s robust tool\-acquisition capability in challenging open\-world settings\.

![Refer to caption](https://arxiv.org/html/2607.28692v1/x8.png)

![Refer to caption](https://arxiv.org/html/2607.28692v1/x9.png)

Figure 4:Performance comparison across knowledge accumulation stages onOpenSciToolBenchwith GPT\-5\.4\-mini and Gemini\-3\-flash, measured by Plan Acc\. \(planning accuracy\) and Unk\. Tool Hit \(hit rate on initially unknown tools\)\.

### 5\.6Knowledge Accumulation Dynamics

To investigate how knowledge accumulation affects performance, we evaluate SciToolAgent\-Evo at different accumulation stages on OpenSciToolBench with GPT\-5\.4\-mini and Gemini\-3\-flash\. As shown in Figure[4](https://arxiv.org/html/2607.28692#S5.F4), both Planning Accuracy and Unknown Tool Hit Rate improve consistently as the process advances\. These gains stem from the key mechanisms of the accumulation process, including contrastive experience extraction, reusable skill distillation, and dynamic tool migration\. Notably, both metrics grow rapidly from 0% to 25% progress and then improve more smoothly, suggesting that a limited amount of data is sufficient for the agent to establish robust priors for open\-world adaptation\. Detailed statistics are provided in Appendix[B2](https://arxiv.org/html/2607.28692#A2.SS2)\.

## 6Conclusion

In this work, we propose an ontology\-aware self\-evolving agent for open\-world scientific tool acquisition\. SciToolAgent\-Evo follows a two\-stage design: during accumulation, it distills reusable knowledge from contrastive trajectories; during inference, it uses active requests and a LinUCB\-based bandit gate to balance tool exploration and exploitation\. By leveraging an adaptive memory of skills, experiences, and an ontologized tool graph, SciToolAgent\-Evo enables agents to continually expand their capabilities in open\-world scientific environments\. Moreover, we introduce OpenSciToolBench, a scientific tool\-use benchmark with 900 tasks across four difficulty levels\. Extensive experiments demonstrate that SciToolAgent\-Evo outperforms strong baselines, indicating its robustness and generalization\.

## References

- Modular large language model agents for multi\-task computational materials science\.Communications Materials7\(1\),pp\. 131\.Cited by:[§1](https://arxiv.org/html/2607.28692#S1.p1.1)\.
- Z\. Chen, S\. Chen, Y\. Ning, Q\. Zhang, B\. Wang, B\. Yu, Y\. Li, Z\. Liao, C\. Wei, Z\. Lu,et al\.\(2025\)Scienceagentbench: toward rigorous assessment of language agents for data\-driven scientific discovery\.InInternational Conference on Learning Representations,Vol\.2025,pp\. 96934–96990\.Cited by:[§1](https://arxiv.org/html/2607.28692#S1.p3.1),[§2\.2](https://arxiv.org/html/2607.28692#S2.SS2.p1.1)\.
- K\. Ding, J\. Yu, J\. Huang, Y\. Yang, Q\. Zhang, and H\. Chen \(2025\)SciToolAgent: a knowledge\-graph\-driven scientific agent for multitool integration\.Nature Computational Science5\(10\),pp\. 962–972\.Cited by:[6th item](https://arxiv.org/html/2607.28692#A8.I1.i6.p1.1),[§1](https://arxiv.org/html/2607.28692#S1.p2.1),[§1](https://arxiv.org/html/2607.28692#S1.p3.1),[§2\.1](https://arxiv.org/html/2607.28692#S2.SS1.p1.1),[§2\.2](https://arxiv.org/html/2607.28692#S2.SS2.p1.1),[§3\.2](https://arxiv.org/html/2607.28692#S3.SS2.p1.1),[2nd item](https://arxiv.org/html/2607.28692#S5.I1.i2.p1.1),[§5\.2](https://arxiv.org/html/2607.28692#S5.SS2.p1.1)\.
- A\. E\. Ghareeb, B\. Chang, L\. Mitchener, A\. Yiu, C\. J\. Szostkiewicz, D\. Shved, G\. J\. Gyimesi, J\. M\. Laurent, S\. M\. Wright, M\. T\. Razzak,et al\.\(2026\)A multi\-agent system for automating scientific discovery\.Nature,pp\. 1–3\.Cited by:[§1](https://arxiv.org/html/2607.28692#S1.p1.1)\.
- K\. Hess, B\. Jones, D\. Carlock, and J\. R\. Walkup \(2009\)Cognitive rigor: blending the strengths of bloom’s taxonomy and webb’s depth of knowledge to enhance classroom\-level processes\.Cited by:[Appendix C](https://arxiv.org/html/2607.28692#A3.p1.1),[§3\.1](https://arxiv.org/html/2607.28692#S3.SS1.p1.1)\.
- G\. Jiang, Z\. Su, X\. Qu, and Y\. R\. Fung \(2026\)Xskill: continual learning from experience and skills in multimodal agents\.arXiv:2603\.12056\.Cited by:[§4\.1](https://arxiv.org/html/2607.28692#S4.SS1.SSS0.Px2.p1.5)\.
- Q\. Jin, Y\. Yang, Q\. Chen, and Z\. Lu \(2024\)GeneGPT: augmenting large language models with domain tools for improved access to biomedical information\.Bioinformatics40\(2\),pp\. btae075\.Cited by:[§1](https://arxiv.org/html/2607.28692#S1.p2.1)\.
- Y\. Kang and J\. Kim \(2024\)ChatMOF: an artificial intelligence system for predicting and generating metal\-organic frameworks using large language models\.Nature communications15\(1\),pp\. 4705\.Cited by:[§1](https://arxiv.org/html/2607.28692#S1.p2.1)\.
- P\. Lewis, E\. Perez, A\. Piktus, F\. Petroni, V\. Karpukhin, N\. Goyal, H\. Küttler, M\. Lewis, W\. Yih, T\. Rocktäschel,et al\.\(2020\)Retrieval\-augmented generation for knowledge\-intensive nlp tasks\.Advances in neural information processing systems33,pp\. 9459–9474\.Cited by:[3rd item](https://arxiv.org/html/2607.28692#A8.I1.i3.p1.1),[§5\.2](https://arxiv.org/html/2607.28692#S5.SS2.p1.1)\.
- L\. Li, W\. Chu, J\. Langford, and R\. E\. Schapire \(2010\)A contextual\-bandit approach to personalized news article recommendation\.InProceedings of the 19th international conference on World wide web,pp\. 661–670\.Cited by:[§4\.1](https://arxiv.org/html/2607.28692#S4.SS1.SSS0.Px4.p1.6)\.
- M\. Li, Y\. Zhao, B\. Yu, F\. Song, H\. Li, H\. Yu, Z\. Li, F\. Huang, and Y\. Li \(2023\)Api\-Bank: a comprehensive benchmark for tool\-augmented llms\.InProceedings of the 2023 conference on empirical methods in natural language processing,pp\. 3102–3116\.Cited by:[§2\.2](https://arxiv.org/html/2607.28692#S2.SS2.p1.1)\.
- X\. Liu, H\. Yu, H\. Zhang, Y\. Xu, X\. Lei, H\. Lai, Y\. Gu, H\. Ding, K\. Men, K\. Yang,et al\.\(2024\)Agentbench: evaluating llms as agents\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 52989–53046\.Cited by:[§1](https://arxiv.org/html/2607.28692#S1.p1.1)\.
- J\. Lu, T\. Holleis, Y\. Zhang, B\. Aumayer, F\. Nan, H\. Bai, S\. Ma, S\. Ma, M\. Li, G\. Yin,et al\.\(2025\)Toolsandbox: a stateful, conversational, interactive evaluation benchmark for llm tool use capabilities\.InFindings of the Association for Computational Linguistics: NAACL 2025,pp\. 1160–1183\.Cited by:[§2\.2](https://arxiv.org/html/2607.28692#S2.SS2.p1.1)\.
- A\. M\. Bran, S\. Cox, O\. Schilter, C\. Baldassari, A\. D\. White, and P\. Schwaller \(2024\)Augmenting large language models with chemistry tools\.Nature machine intelligence6\(5\),pp\. 525–535\.Cited by:[§1](https://arxiv.org/html/2607.28692#S1.p2.1)\.
- Y\. Ma, Z\. Gou, J\. Hao, R\. Xu, S\. Wang, L\. Pan, Y\. Yang, Y\. Cao, and A\. Sun \(2024a\)Sciagent: tool\-augmented language models for scientific reasoning\.InProceedings of the 2024 conference on empirical methods in natural language processing,pp\. 15701–15736\.Cited by:[§1](https://arxiv.org/html/2607.28692#S1.p3.1),[§2\.1](https://arxiv.org/html/2607.28692#S2.SS1.p1.1),[§2\.2](https://arxiv.org/html/2607.28692#S2.SS2.p1.1)\.
- Z\. Ma, W\. Huang, J\. Zhang, T\. Gupta, and R\. Krishna \(2024b\)M & m’s: a benchmark to evaluate tool\-use for m ulti\-step m ulti\-modal tasks\.InEuropean Conference on Computer Vision,pp\. 18–34\.Cited by:[§B3](https://arxiv.org/html/2607.28692#A2.SS3.p1.1)\.
- D\. Mekala, J\. E\. Weston, J\. Lanchantin, R\. Raileanu, M\. Lomeli, J\. Shang, and J\. Dwivedi\-Yu \(2024\)Toolverifier: generalization to new tools via self\-verification\.InFindings of the Association for Computational Linguistics: EMNLP 2024,pp\. 5026–5041\.Cited by:[§1](https://arxiv.org/html/2607.28692#S1.p1.1),[§2\.2](https://arxiv.org/html/2607.28692#S2.SS2.p1.1)\.
- B\. Min, H\. Ross, E\. Sulem, A\. P\. B\. Veyseh, T\. H\. Nguyen, O\. Sainz, E\. Agirre, I\. Heintz, and D\. Roth \(2023\)Recent advances in natural language processing via large pre\-trained language models: a survey\.ACM Computing Surveys56\(2\),pp\. 1–40\.Cited by:[§1](https://arxiv.org/html/2607.28692#S1.p1.1)\.
- OpenAI \(2025\)GPT\-5 system card\.Technical reportOpenAI\.Cited by:[§3\.2](https://arxiv.org/html/2607.28692#S3.SS2.p1.1),[§5\.2](https://arxiv.org/html/2607.28692#S5.SS2.p1.1)\.
- B\. Paranjape, S\. Lundberg, S\. Singh, H\. Hajishirzi, L\. Zettlemoyer, and M\. T\. Ribeiro \(2023\)Art: automatic multi\-step reasoning and tool\-use for large language models\.arXiv:2303\.09014\.Cited by:[§2\.1](https://arxiv.org/html/2607.28692#S2.SS1.p1.1)\.
- S\. G\. Patil, T\. Zhang, X\. Wang, and J\. E\. Gonzalez \(2024\)Gorilla: large language model connected with massive apis\.Advances in Neural Information Processing Systems37,pp\. 126544–126565\.Cited by:[§2\.2](https://arxiv.org/html/2607.28692#S2.SS2.p1.1)\.
- C\. Qian, E\. C\. Acikgoz, Q\. He, H\. Wang, X\. Chen, D\. Hakkani\-Tur, G\. Tur, and H\. Ji \(2026\)Toolrl: reward is all tool learning needs\.Advances in Neural Information Processing Systems38,pp\. 105523–105553\.Cited by:[§2\.1](https://arxiv.org/html/2607.28692#S2.SS1.p1.1)\.
- Y\. Qin, S\. Liang, Y\. Ye, K\. Zhu, L\. Yan, Y\. Lu, Y\. Lin, X\. Cong, X\. Tang, B\. Qian,et al\.\(2024\)ToolLLM: facilitating large language models to master 16000\+ real\-world apis\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 9695–9717\.Cited by:[§2\.1](https://arxiv.org/html/2607.28692#S2.SS1.p1.1)\.
- T\. Schick, J\. Dwivedi\-Yu, R\. Dessì, R\. Raileanu, M\. Lomeli, E\. Hambro, L\. Zettlemoyer, N\. Cancedda, and T\. Scialom \(2023\)Toolformer: language models can teach themselves to use tools\.Advances in neural information processing systems36,pp\. 68539–68551\.Cited by:[§2\.1](https://arxiv.org/html/2607.28692#S2.SS1.p1.1)\.
- Y\. Shen, K\. Song, X\. Tan, D\. Li, W\. Lu, and Y\. Zhuang \(2023\)Hugginggpt: solving ai tasks with chatgpt and its friends in hugging face\.Advances in Neural Information Processing Systems36,pp\. 38154–38180\.Cited by:[§2\.1](https://arxiv.org/html/2607.28692#S2.SS1.p1.1)\.
- N\. Shinn, F\. Cassano, A\. Gopinath, K\. Narasimhan, and S\. Yao \(2023\)Reflexion: language agents with verbal reinforcement learning\.Advances in neural information processing systems36,pp\. 8634–8652\.Cited by:[5th item](https://arxiv.org/html/2607.28692#A8.I1.i5.p1.1),[§2\.1](https://arxiv.org/html/2607.28692#S2.SS1.p1.1),[§5\.2](https://arxiv.org/html/2607.28692#S5.SS2.p1.1)\.
- Y\. Song, W\. Xiong, D\. Zhu, W\. Wu, H\. Qian, M\. Song, H\. Huang, C\. Li, K\. Wang, R\. Yao,et al\.\(2023\)RestGPT: connecting large language models with real\-world restful apis\.arXiv:2306\.06624\.Cited by:[§B3](https://arxiv.org/html/2607.28692#A2.SS3.p1.1),[§2\.2](https://arxiv.org/html/2607.28692#S2.SS2.p1.1)\.
- Q\. Tang, Z\. Deng, H\. Lin, X\. Han, Q\. Liang, B\. Cao, and L\. Sun \(2023\)Toolalpaca: generalized tool learning for language models with 3000 simulated cases\.arXiv:2306\.05301\.Cited by:[§1](https://arxiv.org/html/2607.28692#S1.p2.1)\.
- Y\. Tang, K\. Feng, Y\. Wang, Z\. Chen, C\. Lv, G\. Yu, Q\. Zhang, K\. Ding, and H\. Chen \(2025\)Learning an efficient multi\-turn dialogue evaluator from multiple llm judges\.arXiv:2508\.00454\.Cited by:[§1](https://arxiv.org/html/2607.28692#S1.p1.1)\.
- G\. Team, R\. Anil, S\. Borgeaud, J\. Alayrac, J\. Yu, R\. Soricut, J\. Schalkwyk, A\. M\. Dai, A\. Hauth, K\. Millican,et al\.\(2023\)Gemini: a family of highly capable multimodal models\.arXiv:2312\.11805\.Cited by:[§5\.2](https://arxiv.org/html/2607.28692#S5.SS2.p1.1)\.
- H\. Trivedi, T\. Khot, M\. Hartmann, R\. Manku, V\. Dong, E\. Li, S\. Gupta, A\. Sabharwal, and N\. Balasubramanian \(2024\)Appworld: a controllable world of apps and people for benchmarking interactive coding agents\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 16022–16076\.Cited by:[§2\.2](https://arxiv.org/html/2607.28692#S2.SS2.p1.1)\.
- S\. H\. Vemprala, R\. Bonatti, A\. Bucker, and A\. Kapoor \(2024\)ChatGPT for robotics: design principles and model abilities\.Ieee Access12,pp\. 55682–55696\.Cited by:[§1](https://arxiv.org/html/2607.28692#S1.p1.1)\.
- H\. Wang, C\. Qian, W\. Zhong, X\. Chen, J\. Qiu, S\. Huang, B\. Jin, M\. Wang, K\. Wong, and H\. Ji \(2025a\)OTC: optimal tool calls via reinforcement learning\.arXiv–2504\.Cited by:[§2\.1](https://arxiv.org/html/2607.28692#S2.SS1.p1.1)\.
- L\. Wang, C\. Ma, X\. Feng, Z\. Zhang, H\. Yang, J\. Zhang, Z\. Chen, J\. Tang, X\. Chen, Y\. Lin,et al\.\(2024\)A survey on large language model based autonomous agents\.Frontiers of Computer Science18\(6\),pp\. 186345\.Cited by:[§1](https://arxiv.org/html/2607.28692#S1.p1.1)\.
- R\. Wang, X\. Han, L\. Ji, S\. Wang, T\. Baldwin, and H\. Li \(2025b\)Toolgen: unified tool retrieval and calling via generation\.InInternational Conference on Learning Representations,Vol\.2025,pp\. 73473–73498\.Cited by:[§2\.2](https://arxiv.org/html/2607.28692#S2.SS2.p1.1)\.
- J\. Wei, X\. Wang, D\. Schuurmans, M\. Bosma, F\. Xia, E\. Chi, Q\. V\. Le, D\. Zhou,et al\.\(2022\)Chain\-of\-thought prompting elicits reasoning in large language models\.Advances in neural information processing systems35,pp\. 24824–24837\.Cited by:[2nd item](https://arxiv.org/html/2607.28692#A8.I1.i2.p1.1),[§5\.2](https://arxiv.org/html/2607.28692#S5.SS2.p1.1)\.
- H\. Wu, Z\. He, X\. Zhang, X\. Yao, S\. Zheng, H\. Zheng, and B\. Yu \(2024\)Chateda: a large language model powered autonomous agent for eda\.IEEE Transactions on Computer\-Aided Design of Integrated Circuits and Systems43\(10\),pp\. 3184–3197\.Cited by:[§1](https://arxiv.org/html/2607.28692#S1.p1.1)\.
- A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv,et al\.\(2025\)Qwen3 technical report\.arXiv:2505\.09388\.Cited by:[§5\.3](https://arxiv.org/html/2607.28692#S5.SS3.p1.8)\.
- S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. R\. Narasimhan, and Y\. Cao \(2023\)ReAct: synergizing reasoning and acting in language models\.InThe Eleventh International Conference on Learning Representations,Cited by:[4th item](https://arxiv.org/html/2607.28692#A8.I1.i4.p1.1),[§2\.1](https://arxiv.org/html/2607.28692#S2.SS1.p1.1),[§5\.2](https://arxiv.org/html/2607.28692#S5.SS2.p1.1)\.
- J\. Yu, Y\. Tang, K\. Feng, L\. Liang, Q\. Zhang, K\. Ding, and H\. Chen \(2026\)SciCuEval: A comprehensive dataset for evaluating scientific context understanding in large language models\.Scientific Data\.Cited by:[Appendix D](https://arxiv.org/html/2607.28692#A4.p1.1)\.

## Appendix ALimitations

Despite the efficacy of SciToolAgent\-Evo in open\-world scientific tool acquisition, several limitations remain\. First, our evaluation is conducted in a purely text\-based setting, which abstracts away critical multi\-modal aspects of real scientific workflows and leaves multi\-modal tool adaptation underexplored\. For future work, we will evaluate our framework on multi\-modal scientific tasks and explore dynamic memory updating and active unlearning strategies for more durable open\-world tool adaptation\.

## Appendix BAdditional Results

In this section, we demonstrate the effectiveness of SciToolAgent\-Evo through additional experiments\. We first analyze fine\-grained ablations \(Appendix[B1](https://arxiv.org/html/2607.28692#A2.SS1)\) and knowledge accumulation dynamics \(Appendix[B2](https://arxiv.org/html/2607.28692#A2.SS2)\)\. We then evaluate out\-of\-domain generalization \(Appendix[B3](https://arxiv.org/html/2607.28692#A2.SS3)\)\. Finally, we investigate inference efficiency \(Appendix[B4](https://arxiv.org/html/2607.28692#A2.SS4)\)\.

### B1Fine\-Grained Ablation Study

In Section[5\.5](https://arxiv.org/html/2607.28692#S5.SS5), we reported the overall component\-level ablation results \(Table[3](https://arxiv.org/html/2607.28692#S5.T3)\)\. In this section, we conduct a fine\-grained ablation of the ontology\-aware inference pipeline using GPT\-5\.4\-mini as the backbone to assess the contribution of each module\. Table[A1](https://arxiv.org/html/2607.28692#A2.T1)yields the following observations: \(1\) Disabling the Task Router reduces Level\-1 and Level\-2 performance by 8\.57 and 10\.52 points, respectively, suggesting that it avoids unnecessary reasoning on simpler tasks\. \(2\) Removing Active Request or replacing it with Retrieval Rewriting substantially degrades performance, indicating that requirement\-conditioned retrieval better captures task\-specific tool needs than raw\-query matching\. \(3\) Excluding Reranking degrades overall performance by 4\.06 points, confirming the value of candidate refinement\. \(4\) LinUCB outperforms both the Fixed Threshold Gate and the LLM\-based Gate, indicating a more effective balance between known\-tool exploitation and unknown\-tool exploration\. Overall, these results confirm that each component is necessary for an effective ontology\-aware inference pipeline\.

MethodLevel\-1Level\-2Level\-3Level\-4Overallw/o Task Router77\.14↓\\downarrow8\.5766\.32↓\\downarrow10\.5261\.18↑\\uparrow0\.3736\.840\.000\.0062\.05↓\\downarrow5\.17w/o Active Request81\.90↓\\downarrow3\.8165\.26↓\\downarrow11\.5843\.53↓\\downarrow17\.2823\.52↓\\downarrow13\.3256\.20↓\\downarrow11\.02Retrieval Rewriting82\.86↓\\downarrow2\.8566\.32↓\\downarrow10\.5249\.41↓\\downarrow11\.4026\.32↓\\downarrow10\.5258\.73↓\\downarrow8\.49Query Decomposition82\.86↓\\downarrow2\.8576\.840\.000\.0057\.65↓\\downarrow3\.1635\.53↓\\downarrow1\.3165\.38↓\\downarrow1\.84w/o Reranking84\.76↓\\downarrow0\.9569\.47↓\\downarrow7\.3754\.12↓\\downarrow6\.6935\.53↓\\downarrow1\.3163\.16↓\\downarrow4\.06Fixed Threshold Gate85\.710\.000\.0074\.74↓\\downarrow2\.1058\.82↓\\downarrow1\.9930\.26↓\\downarrow6\.5864\.82↓\\downarrow2\.40LLM\-based Gate86\.67↑\\uparrow0\.9675\.79↓\\downarrow1\.0557\.65↓\\downarrow3\.1632\.89↓\\downarrow3\.9565\.65↓\\downarrow1\.57\\rowcolormodelgrayOurs \- Full Pipeline85\.7176\.8460\.8136\.8467\.22

Table A1:Fine\-grained ablation study of ontology\-aware inference modules onOpenSciToolBench\. All variants use GPT\-5\.4\-mini as the backbone\. We remove or replace key components to assess their contributions\.↓\\downarrowand↑\\uparrowdenote the absolute performance drop and improvement relative to the full pipeline, respectively, while gray values indicate no change\.
### B2Detailed Knowledge Accumulation Statistics

Table[A2](https://arxiv.org/html/2607.28692#A2.T2)presents detailed statistics of the knowledge accumulation process on OpenSciToolBench with GPT\-5\.4\-mini\. We report results at five checkpoints, tracking the number of migrated unknown tools, the number of accumulated skills, the skill\-based success rate, planning accuracy, and the hit rate on initially unknown tools\. As accumulation progresses, the numbers of acquired tools and accumulated skills increase steadily, while planning accuracy and the unknown\-tool hit rate improve by 26\.37 and 32\.94 points, respectively\. These results further demonstrate the importance of maintaining an evolving memory throughout knowledge accumulation\.

Progress\#Acquired Tools\#SkillsSkill HitPlan Acc\.Unk\. Tool Hit0%001\.2047\.2555\.3025%39804\.4067\.0383\.0950%621477\.6968\.1385\.2975%692146\.5971\.4386\.76100%7626812\.0973\.6288\.24Table A2:Performance evolution across knowledge accumulation stages onOpenSciToolBenchwith GPT\-5\.4\-mini\. Metrics include the number of migrated tools \(\#Acquired Tools\), accumulated skills \(\#Skills\), Skill\-based success rate \(Skill Hit\), planning accuracy \(Plan Acc\.\), and hit rate on initially unknown tools \(Unk\. Tool Hit\)\.
### B3Out\-of\-Domain Generalization

While our primary focus remains on enhancing agent performance in open\-world scientific tool acquisition, we also acknowledge the potential of SciToolAgent\-Evo in out\-of\-domain settings\. To validate the generalizability of our approach, we conducted additional experiments on m&m’s\(Maet al\.[2024b](https://arxiv.org/html/2607.28692#bib.bib44)\), a multi\-step multimodal tool\-use benchmark, and RestBench\(Songet al\.[2023](https://arxiv.org/html/2607.28692#bib.bib18)\), a benchmark for planning and executing RESTful API calls\. Following the official evaluation protocols and metrics of each benchmark, we used GPT\-5\.4\-nano as the backbone model for all methods and compared against the same baselines introduced in Section[5\.2](https://arxiv.org/html/2607.28692#S5.SS2)to ensure a fair comparison\.

As shown in Table[A3](https://arxiv.org/html/2607.28692#A2.T3), SciToolAgent\-Evo achieves the highest tool\-F1 \(80\.82\) and argname\-F1 \(79\.04\) on m&m’s, outperforming prompting\-based \(Direct and CoT\), retrieval\-based \(RAG and SciToolAgent\), and interactive \(ReAct and Reflexion\) baselines\. On RestBench, SciToolAgent\-Evo achieves the best overall performance on both TMDB and Spotify, particularly in success rate and completion percentage\. Overall, these results demonstrate that SciToolAgent\-Evo generalizes effectively to broader tool\-use domains\.

Methodm&m’sRestBenchtool\-F1↑\\uparrowargname\-F1↑\\uparrowTMDBSpotifySuccess%↑\\uparrowCP%↑\\uparrowΔ\\DeltaSol\. Len\.↓\\downarrowCP%↑\\uparrowΔ\\DeltaSol\. Len\.↓\\downarrowDirect61\.7763\.2330\.034\.00\.4234\.480\.76CoT71\.5871\.4644\.048\.0−0\.06\-0\.0644\.83−0\.07\-0\.07RAG71\.6272\.2641\.044\.0−0\.10\-0\.1046\.56−0\.36\\mathbf\{\-0\.36\}ReAct69\.6570\.3142\.046\.00\.5648\.280\.48Reflexion70\.1770\.8140\.046\.01\.2851\.720\.83SciToolAgent65\.4867\.9240\.042\.0−0\.24\\mathbf\{\-0\.24\}48\.28−0\.14\-0\.14SciToolAgent\-Evo80\.8279\.0454\.062\.0−0\.13\-0\.1353\.45−0\.36\\mathbf\{\-0\.36\}

Table A3:Comparison of model performance onm&m’sandRestBenchusing GPT\-5\.4\-nano as the backbone\. The best and second\-best results in each column are bolded and underlined, respectively\.
### B4Inference Efficiency

We further compare the inference efficiency of SciToolAgent\-Evo with representative baselines on OpenSciToolBench using GPT\-5\.4\-mini\. As shown in Table[A4](https://arxiv.org/html/2607.28692#A2.T4), although SciToolAgent\-Evo introduces additional overhead over SciToolAgent due to modules such as evolving\-memory retrieval and active request formulation, it uses substantially fewer total tokens and has lower API cost than ReAct and Reflexion\. It also achieves the lowest output\-token usage and relatively low inference latency\. Overall, SciToolAgent\-Evo maintains strong inference efficiency while enhancing open\-world tool acquisition\.

MethodLLM Calls↓\\downarrowInput Tokens↓\\downarrowOutput Tokens↓\\downarrowTotal Tokens↓\\downarrowLatency \(s\)↓\\downarrowAPI Cost / Task \($\)↓\\downarrowReAct8\.272,115\.85427\.8572,543\.727\.90\.056Reflexion16\.1149,435\.95931\.15150,367\.144\.70\.116SciToolAgent4\.76,859\.45441\.057,300\.517\.20\.007SciToolAgent\-Evo11\.321,711\.35387\.822,099\.1526\.40\.018

Table A4:Inference cost comparison onOpenSciToolBenchwith GPT\-5\.4\-mini\. All reported results are averaged across tasks, with the best result bolded and the second\-best underlined\.

## Appendix CTask Levels and Prompts

In this section, we provide detailed definitions of the four levels in OpenSciToolBench and present the prompt templates below\. The distinctions among these levels are inspired by the Cognitive Rigor Matrix\(Hesset al\.[2009](https://arxiv.org/html/2607.28692#bib.bib42)\), with task difficulty characterized jointly by knowledge mapping, reasoning depth, constraint integration, and decision complexity\.

Level\-1 \(1–2 tools\)consists of explicit invocation tasks, where the question explicitly provides the tool name or clear hints to the required tool\. These tasks typically involve a single tool call or a short tool chain and focus on basic data retrieval for a single scientific entity\.

Prompt for Generating Level\-1 QuestionsSystem Prompt:You are a science expert\.User Prompt:Please generate one scientific tool\-use question based on the given candidate tools\. The question type should be one of Question & Answer, Multiple\-Choice, Content Completion, and True/False\.Candidate Tools:\{tools\_str\}Task Objective:Generate a basic instruction\-level task\. The question should focus on a single substance and use standard scientific terminology to clearly indicate the capability corresponding to a single tool or a very short tool chain\.\[RULE\]:1\. The question must evaluate explicit instruction following and clearly require a specific scientific operation\.2\. tool\_path should contain a single tool or a very short tool chain arranged in the correct execution order\.3\. The question may not describe the solving steps or reveal any tool outputs, numerical results, or answer hints\.4\. The Parameter field should contain only the substance name or ID selected in the question\.5\. Ensure natural, diverse phrasing and avoid template\-like wording\.Strictly output the response invalid JSONformat only\. The requiredJSONformat is as follows:\{"tool\_path": \["tool\_selected", "xxx"\],"question": "question text","Parameter": "substance name"\}

Level\-2 \(2–4 tools\)consists of property analysis tasks, where the question focuses on a single or a small number of target properties\. These tasks typically involve short tool chains for property retrieval, direct computation, or comparison, requiring the model to select appropriate tools and make direct judgments based on their outputs\.

Prompt for Generating Level\-2 QuestionsSystem Prompt:You are a science expert\.User Prompt:Please generate one scientific tool\-use question based on the given candidate tools\. The question type should be one of Question & Answer, Multiple\-Choice, Content Completion, and True/False\.Candidate Tools:\{tools\_str\}Task Objective:Generate a terminology\-level task\. The question should use standard scientific terminology to imply the target property to be evaluated\. It typically involves a short tool chain from entity representation conversion to property computation, comparison, or filtering over a few substances, requiring the model to infer the necessary tool capabilities from the property requirements\.\[RULE\]:1\. The question must evaluate implicit grounding from scientific terminology to a tool\-capability chain\.2\. tool\_path should contain a short tool chain arranged in the correct execution order, supporting entity representation conversion, property computation, or property comparison\.3\. The question must not directly mention any tool name, but should use standard scientific terminology to clearly imply the target property to be evaluated; it must not describe the solving steps or reveal any tool outputs, numerical results, or answer hints\.4\. The Parameter field should contain only the referenced substance names or IDs, with a consistent format\.5\. The question must not use comparison criteria that are unnatural in real scientific queries, such as median; instead, it should use more realistic scientific constraints such as maximum/minimum values, threshold\-based judgments, or structural inclusion\.6\. Ensure natural, diverse phrasing and avoid template\-like wording\.Strictly output the response invalid JSONformat only\. The requiredJSONformat is as follows:\{"tool\_path": \["tool\_selected", "xxx"\],"question": "question text","Parameter": \["substance name 1", "substance name 2", "xxx"\]\}

Level\-3 \(3–6 tools\)consists of constraint reasoning tasks, where the question specifies multiple candidate entities, multi\-dimensional interrelated properties, and explicit scientific constraints\. These tasks require integrating information from multiple sources and performing constraint\-aware reasoning to infer latent relationships and identify the most appropriate candidate\.

Prompt for Generating Level\-3 QuestionsSystem Prompt:You are a science expert\.User Prompt:Please generate one scientific tool\-use question based on the given candidate tools\. The question type should be one of Question & Answer, Multiple\-Choice, Content Completion, and True/False\.Candidate Tools:\{tools\_str\}Task Objective:Generate an expert decision\-level task\. The question should provide multiple candidate objects and multiple scientific constraints\. It typically requires cross\-tool and cross\-attribute integration over multiple substances, requiring the model to perform candidate selection and execute a relatively long tool chain under multiple constraints\.\[RULE\]:1\. The question must evaluate deep scientific reasoning, including property derivation, comprehensive threshold\-based judgment, candidate filtering, or selection of the optimal object satisfying all conditions\.2\. tool\_path should contain a multi\-tool chain arranged in the correct execution order, supporting cross\-attribute analysis and constraint\-based reasoning over multiple candidate objects\.3\. The question must not directly mention any tool name, but should use standard scientific terminology to clearly describe the candidate objects, target properties, and scientific constraints; it must not describe the solving steps or reveal any tool outputs, numerical results, or answer hints\.4\. The Parameter field should contain only the candidate substance names or IDs involved in the question, with a consistent representation format\.5\. The question must not use comparison criteria that are unnatural in real scientific queries, such as median; instead, it should use scientific constraints such as maximum/minimum values, threshold\-based judgments, or structural inclusion\.6\. Ensure natural, diverse phrasing and avoid template\-like wording\.Strictly output the response invalid JSONformat only\. The requiredJSONformat is as follows:\{"tool\_path": \["tool\_selected", "xxx"\],"question": "question text","Parameter": \["candidate substance 1", "candidate substance 2", "xxx"\]\}

Level\-4 \(5–10 tools\)consists of research strategy tasks, where the question simulates realistic scientific scenarios with open\-ended research goals and multi\-dimensional evaluation criteria\. These tasks typically involve long tool chains and multi\-objective trade\-offs, requiring the model to autonomously plan a complete analysis workflow and formulate interpretable research decisions\.

Prompt for Generating Level\-4 QuestionsSystem Prompt:You are a science expert\.User Prompt:Please generate one scientific tool\-use question based on the given candidate tools\. The question type should be one of Question & Answer, Multiple\-Choice, Content Completion, and True/False\.Candidate Tools:\{tools\_str\}Task Objective:Generate a research strategy\-level task\. The question should simulate a realistic scientific scenario, specify an open\-ended research objective and multidimensional evaluation criteria, and typically involve a long tool chain and multi\-objective trade\-offs, requiring the model to autonomously plan the analysis workflow and produce a rigorous scientific decision\.\[RULE\]:1\. The question must present a complete and realistic scientific motivation, avoiding a rote list of tool calls; it may focus on drug screening, materials selection, property prediction, mechanism investigation, safety assessment, or de novo design validation\.2\. The question should use 1\-2 sentences to establish the research background, followed by precise and non\-redundant scientific statements specifying the analysis objective and evaluation criteria\.3\. tool\_path should contain a relatively long tool chain arranged in a reasonable execution order, supporting multidimensional analysis, evidence integration, and scientific decision\-making\.4\. The question must not directly mention any tool name, but should use standard scientific terminology to describe the research background, analysis objective, and evaluation criteria; it must not describe the specific solving steps or reveal any tool outputs, numerical results, or answer hints\.5\. The Parameter field should contain the specific substances, sequences, or other scientific objects selected in the question\.6\. Ensure natural, coherent, and situated in a realistic research context\.Strictly output the response invalid JSONformat only\. The requiredJSONformat is as follows:\{"tool\_path": \["tool\_selected", "xxx"\],"question": "question text","Parameter": \["scientific object 1", "scientific object 2", "xxx"\]\}

Level\#Questions\#Train\#TestQ&AMCQCCT/FAvg\.PathMax\.PathAvg\.\#Param\.Avg\.\#Cons\.Avg\.\#Attr\.Avg\.Depth\#ReuseLevel\-12621571052530451\.5721\.000\.001\.141\.0054Level\-223814395637931652\.2643\.231\.841\.452\.1831Level\-321312885397063413\.6864\.113\.252\.613\.5226Level\-418711176893528355\.21101\.614\.764\.124\.8817Total9005393614441841261463\.03102\.452\.242\.192\.71128Table A5:Statistics of theOpenSciToolBenchdataset across four difficulty levels, covering dataset scale, question formats, tool\-path structure, and task complexity\.Avg\. PathandMax\. Pathdenote the average and maximum tool\-path length;Avg\. \#Param\.,Avg\. \#Cons\., andAvg\. \#Attr\.denote the average number of input parameters, constraints, and tool attributes per question;Avg\. Depthdenotes the average reasoning depth, and\#Reusedenotes the number of reusable skills\.
## Appendix DBenchmark Validation

This section details the benchmark validation protocol of OpenSciToolBench, which consists of two stages, namely LLM\-as\-a\-Judge verification and human validation\(Yuet al\.[2026](https://arxiv.org/html/2607.28692#bib.bib3)\)\.

LLM\-as\-a\-Judge\.We used Gemini\-3\.1\-Pro as the judge to assess each generated sample along three dimensions: \(1\) whether the question is clearly stated and free of obvious ambiguity; \(2\) whether the reference tool chain fully covers the scientific operations required by the question; and \(3\) whether the input\-output dependencies between adjacent tools in the chain are valid\. Samples that failed this verification process were removed\.

Prompt for Sample Quality EvaluationSystem Prompt:You are a scientific evaluation expert\.User Prompt:Below are a question, a reference tool chain, and the corresponding parameter\. Please use the following rule to determine whether the sample satisfies the quality standard\.\[RULE\]:1\. Check whether the question is clearly stated and free of obvious ambiguity\.2\. Check whether tool\_path covers the scientific operations required by the question\.3\. Check whether the input\-output dependencies between adjacent tools in tool\_path are valid and can form an executable chain\.4\. Judge as “No” if any condition fails; otherwise, “Yes”\.Question:\{question\}Reference tool chain:\{tool\_path\}Parameter:\{parameter\}Strictly output the response invalid JSONformat only\. The requiredJSONformat is as follows:\{"decision": "Yes / No","reason": "brief explanation"\}

Human Validation\.To further ensure the quality and rigor of our dataset, we conducted human validation on the samples that passed the initial LLM\-based verification\. For Levels 1–3, we invited five students with backgrounds in biology, chemistry, and materials science to perform the review\. For Level\-4, we invited three domain experts, whose expertise was better suited for evaluating the open\-ended research scenarios\. Each sample was independently reviewed by three annotators according to the following four criteria:

- •Level Alignment: whether the question effectively tests the design objective of the assigned level and accurately reflects the core competency intended to be evaluated at that level\.
- •Semantic Clarity: whether the question is clearly stated, logically coherent, and free of obvious ambiguity, such that the scientific goal and constraints can be accurately understood\.
- •Tool\-Chain Coherence: whether the reference tool chain can fully support the question, and whether the input\-output dependencies between consecutive tools are valid, avoiding mismatched tool capabilities or broken data flows\.
- •Answerability: whether the involved scientific entities exist, and whether the answer can be obtained through executing the given tool chain and applying the necessary model reasoning\.

Student reviewers were compensated at a rate of $10 for every 20 cases, while expert reviewers received $15 for every 10 cases\. The entire validation process took five days and cost $1,900 in total\. Each reviewer independently provided a “Yes” or “No” judgment for every assigned sample\. We then applied a majority\-vote rule to aggregate the three reviewers’ assessments and obtain the final consensus decision\. Among the samples entering human validation, 85\.7% satisfied the above criteria, resulting in a final benchmark of 900 questions across four difficulty levels, three scientific domains, and four question formats\. Detailed dataset statistics are reported in Tables[A5](https://arxiv.org/html/2607.28692#A3.T5)\.

## Appendix EOntology\-Aware Inference Algorithm

Algorithm[1](https://arxiv.org/html/2607.28692#alg1)presents the details of ontology\-aware inference, including skill and task routing, active tool acquisition, and knowledge update\.

Algorithm 1Ontology\-Aware Inference0:Test task

qq, task domain

dqd\_\{q\}, memory

ℳ=\(GK,𝒦,ℰ\)\\mathcal\{M\}=\(G\_\{K\},\\mathcal\{K\},\\mathcal\{E\}\), unknown tools

TUT\_\{U\}, bandit gate

ℬ\\mathcal\{B\}
0:Answer

y^\\hat\{y\}, updated

ℳ\\mathcal\{M\},

TUT\_\{U\}and

ℬ\\mathcal\{B\}
1:

τ^←\[\]\\hat\{\\tau\}\\leftarrow\[\\,\],

𝒜←\[\]\\mathcal\{A\}\\leftarrow\[\\,\]
2:Step 1: Skill and task routing

3:

k⋆←SkillMatch​\(q,𝒦\)k^\{\\star\}\\leftarrow\\textsc\{SkillMatch\}\(q,\\mathcal\{K\}\)
4:if

k⋆≠∅k^\{\\star\}\\neq\\emptysetthen

5:

τ^←\\hat\{\\tau\}\\leftarrowTool chain stored in

k⋆k^\{\\star\}
6:

y^←Execute​\(q,τ^\)\\hat\{y\}\\leftarrow\\textsc\{Execute\}\(q,\\hat\{\\tau\}\)
7:return

y^,ℳ,TU,ℬ\\hat\{y\},\\mathcal\{M\},T\_\{U\},\\mathcal\{B\}
8:endif

9:

cq←AssessDifficulty​\(q\)c\_\{q\}\\leftarrow\\textsc\{AssessDifficulty\}\(q\)
10:if

cq=Simplec\_\{q\}=\\textsc\{Simple\}then

11:

τ^←\\hat\{\\tau\}\\leftarrowCoT planning over

GK∪TUG\_\{K\}\\cup T\_\{U\}
12:

y^←Execute​\(q,τ^\)\\hat\{y\}\\leftarrow\\textsc\{Execute\}\(q,\\hat\{\\tau\}\)
13:return

y^,ℳ,TU,ℬ\\hat\{y\},\\mathcal\{M\},T\_\{U\},\\mathcal\{B\}
14:endif

15:Step 2: Active Tool acquisition

16:

ℰq←ℰg​e​n∪TopK​\(ℰdq,q\)\\mathcal\{E\}\_\{q\}\\leftarrow\\mathcal\{E\}\_\{gen\}\\cup\\textsc\{TopK\}\(\\mathcal\{E\}\_\{d\_\{q\}\},q\)
17:

Rq=\{r1,…,rl\}←ActiveRequest​\(q,ℰq\)R\_\{q\}=\\\{r\_\{1\},\\ldots,r\_\{l\}\\\}\\leftarrow\\textsc\{ActiveRequest\}\(q,\\mathcal\{E\}\_\{q\}\)
18:foreach request

rj∈Rqr\_\{j\}\\in R\_\{q\}do

19:

Cj0←TopKt∈VK​S​\(rj,t\)C\_\{j\}^\{0\}\\leftarrow\\textsc\{TopK\}\_\{t\\in V\_\{K\}\}S\(r\_\{j\},t\)
20:

Cj←SubgraphExpand​\(Cj0,GK\)C\_\{j\}\\leftarrow\\textsc\{SubgraphExpand\}\(C\_\{j\}^\{0\},G\_\{K\}\)
21:

xj=\[fj,mj,sj,pj\]⊤←GateCtx​\(rj,Cj\)x\_\{j\}=\[f\_\{j\},m\_\{j\},s\_\{j\},p\_\{j\}\]^\{\\top\}\\leftarrow\\textsc\{GateCtx\}\(r\_\{j\},C\_\{j\}\)
22:

aj←ℬ​\(xj\)a\_\{j\}\\leftarrow\\mathcal\{B\}\(x\_\{j\}\), where

aj∈\{Known,Unknown\}a\_\{j\}\\in\\\{\\textsc\{Known\},\\textsc\{Unknown\}\\\}
23:if

aj=Knowna\_\{j\}=\\textsc\{Known\}then

24:

tj←Top​\(Cj\)t\_\{j\}\\leftarrow\\textsc\{Top\}\(C\_\{j\}\)
25:else

26:

tj←SearchUnknown​\(rj,TU\)t\_\{j\}\\leftarrow\\textsc\{SearchUnknown\}\(r\_\{j\},T\_\{U\}\)
27:endif

28:

τ^←τ^⊕tj\\hat\{\\tau\}\\leftarrow\\hat\{\\tau\}\\oplus t\_\{j\},

𝒜←𝒜⊕\(xj,aj\)\\mathcal\{A\}\\leftarrow\\mathcal\{A\}\\oplus\(x\_\{j\},a\_\{j\}\)
29:endfor

30:Step 3: Knowledge update

31:

\(y^,H,r\)←Execute​\(q,τ^\)\(\\hat\{y\},H,r\)\\leftarrow\\textsc\{Execute\}\(q,\\hat\{\\tau\}\), where

r∈\{0,1\}r\\in\\\{0,1\\\}
32:if

r=1r=1then

33:foreach unknown tool

u∈τ^u\\in\\hat\{\\tau\}do

34:

ou←CompleteOntology​\(u,H\)o\_\{u\}\\leftarrow\\textsc\{CompleteOntology\}\(u,H\)
35:

GK←Migrate​\(GK,u,ou\)G\_\{K\}\\leftarrow\\textsc\{Migrate\}\(G\_\{K\},u,o\_\{u\}\)
36:

TU←TU∖\{u\}T\_\{U\}\\leftarrow T\_\{U\}\\setminus\\\{u\\\}
37:endfor

38:

𝒦←𝒦∪SkillDistill​\(q,τ^,H\)\\mathcal\{K\}\\leftarrow\\mathcal\{K\}\\cup\\textsc\{SkillDistill\}\(q,\\hat\{\\tau\},H\)
39:endif

40:

ℳ←\(GK,𝒦,ℰ\)\\mathcal\{M\}\\leftarrow\(G\_\{K\},\\mathcal\{K\},\\mathcal\{E\}\)
41:return

y^,ℳ,TU,ℬ\\hat\{y\},\\mathcal\{M\},T\_\{U\},\\mathcal\{B\}

## Appendix FActive Tool Request Analysis

In this section, we provide a brief analysis of why SciToolAgent\-Evo adopts active tool requests instead of passive tool selection\.

Let𝒯¯=𝒯K∪𝒯~U\\bar\{\\mathcal\{T\}\}=\\mathcal\{T\}\_\{K\}\\cup\\tilde\{\\mathcal\{T\}\}\_\{U\}denote the conditionally observable tool space, where𝒯K\\mathcal\{T\}\_\{K\}is the known tool graph and𝒯~U\\tilde\{\\mathcal\{T\}\}\_\{U\}contains unknown\-tool attributes accessed only whenGKG\_\{K\}is insufficient\. LetΠ​\(𝒯¯\)\\Pi\(\\bar\{\\mathcal\{T\}\}\)denote the candidate tool\-chain space andτ∗\\tau^\{\*\}the reference tool chain for taskqq\.

A passive strategy directly matches the raw task against candidate tool chains:

Ppassive​\(τ∗∣q,𝒯¯\)=P​\(q∣τ∗,𝒯¯\)​P​\(τ∗∣𝒯¯\)∑τ∈Π​\(𝒯¯\)P​\(q∣τ,𝒯¯\)​P​\(τ∣𝒯¯\)\.\\begin\{aligned\} P\_\{\\mathrm\{passive\}\}\(\\tau^\{\*\}\\mid q,\\bar\{\\mathcal\{T\}\}\)=\\frac\{P\(q\\mid\\tau^\{\*\},\\bar\{\\mathcal\{T\}\}\)P\(\\tau^\{\*\}\\mid\\bar\{\\mathcal\{T\}\}\)\}\{\\sum\_\{\\tau\\in\\Pi\(\\bar\{\\mathcal\{T\}\}\)\}P\(q\\mid\\tau,\\bar\{\\mathcal\{T\}\}\)P\(\\tau\\mid\\bar\{\\mathcal\{T\}\}\)\}\.\\end\{aligned\}

\(7\)This direct query\-based matching is inefficient in large heterogeneous tool spaces and unreliable for weakly described unknown tools\.

In contrast, givenqqand retrieved experiencesℰq\\mathcal\{E\}\_\{q\}, SciToolAgent\-Evo actively generates a request setRq=\{r1,r2,…,rl\}R\_\{q\}=\\\{r\_\{1\},r\_\{2\},\\ldots,r\_\{l\}\\\}, where eachrjr\_\{j\}is a concise description of one tool need\. Tool\-chain acquisition is then conditioned on these requests:

Pactive​\(τ∗∣q,ℰq,𝒯¯\)=∑RqP​\(τ∗∣Rq,𝒯¯\)​P​\(Rq∣q,ℰq\)\.\\begin\{aligned\} P\_\{\\mathrm\{active\}\}\(\\tau^\{\*\}\\mid q,\\mathcal\{E\}\_\{q\},\\bar\{\\mathcal\{T\}\}\)=\\sum\_\{R\_\{q\}\}P\(\\tau^\{\*\}\\mid R\_\{q\},\\bar\{\\mathcal\{T\}\}\)P\(R\_\{q\}\\mid q,\\mathcal\{E\}\_\{q\}\)\.\\end\{aligned\}

\(8\)Here,P​\(Rq∣q,ℰq\)P\(R\_\{q\}\\mid q,\\mathcal\{E\}\_\{q\}\)captures the agent’s ability to express task\-specific tool needs under ontology\-guided reasoning\.

From an information\-acquisition view, an effective request set should reduce uncertainty over the required tool chain:

Rq∗=arg⁡maxRq⁡I​\(τ∗;Rq∣q,ℰq\)=arg⁡maxRq⁡\[H​\(τ∗∣q,ℰq\)−H​\(τ∗∣Rq,q,ℰq\)\]\.\\begin\{aligned\} R\_\{q\}^\{\*\}&=\\arg\\max\_\{R\_\{q\}\}I\(\\tau^\{\*\};R\_\{q\}\\mid q,\\mathcal\{E\}\_\{q\}\)\\\\ &=\\arg\\max\_\{R\_\{q\}\}\\left\[H\(\\tau^\{\*\}\\mid q,\\mathcal\{E\}\_\{q\}\)\-H\(\\tau^\{\*\}\\mid R\_\{q\},q,\\mathcal\{E\}\_\{q\}\)\\right\]\.\\end\{aligned\}

\(9\)Moreover, functionality\-level requests are better aligned with both known\-tool ontology and unknown\-tool functionalities than raw task queries, since they describe the required capability in the same semantic form as tool descriptions\. Thus, active tool requests transform scientific tool acquisition from costly raw\-query matching into requirement\-conditioned retrieval\.

## Appendix GExperimental Settings

During knowledge accumulation, we first sample a deterministic baseline trajectory \(Tbase=0T\_\{\\mathrm\{base\}\}=0\)\. If this trajectory fails, we further sample three exploratory trajectories in parallel withTexp∈\{0\.2,0\.5,0\.8\}T\_\{\\mathrm\{exp\}\}\\in\\\{0\.2,0\.5,0\.8\\\}\. After accumulation, we deduplicate the experience pool by removing experience rules whose pairwise semantic similarity exceedsθsim\\theta\_\{\\mathrm\{sim\}\}= 0\.85\.

At inference time, the agent utilizes all global experience rules and retrieves the top\-5 domain\-specific entries\. The decoding temperature of the Main Agent is set to 0\.3, with max retries = 2, and the final answer is evaluated by GPT\-5\-mini\. We split the toolset into two equal partitions, designating 50% as known and the remaining 50% as unknown\. The difficulty routing module uses two branch thresholds, namely an estimated tool count of 2 and a scientific entity count of 4\. During this stage, locally deployed Qwen3\-Embedding\-0\.6B is used as the retrieval model \(m​a​x​\_​t​o​o​l​s=5max\\\_tools=5\), while Qwen3\-1\.7B is used as the reranking model\. We implement a LinUCB\-based bandit gate with a 4D context vector\. The exploration coefficientα\\alphainitializes at0\.80\.8and decays by0\.010\.01per step to a minimum of0\.20\.2\. Furthermore, a 20\-task sliding window monitors the known\-tool failure rate, dynamically rescalingα\\alphawithin\[0\.7,1\.3\]\[0\.7,1\.3\]to adaptively balance exploration and exploitation\.

Specifically, we adopt a unified and standardized downstream pipeline for tool execution, summarization, and evaluation\. This setup ensures a fair comparison by confining methodological variations strictly to the tool\-chain planning strategies\.

## Appendix HIntroduction of Baselines

We have selected the following methods as our baselines:

- •Direct: Generates the final result directly without intermediate reasoning trajectories\. In our setting, the model first identifies suitable tools from the known set, retrieves candidates from the unknown set if necessary, and directly outputs the planned tool chain for execution and summarization\.
- •CoT\(Weiet al\.[2022](https://arxiv.org/html/2607.28692#bib.bib6)\): Produces an explicit reasoning process prior to generating the result\. Specifically, it reasons over the known and unknown tool sets sequentially to identify required tools, subsequently merging them into a coherent chain for execution\.
- •RAG\(Lewiset al\.[2020](https://arxiv.org/html/2607.28692#bib.bib43)\): Augments tool\-chain planning with retrieved tool candidates\. It retrieves relevant tools from the known graph and, when capability gaps are identified, retrieves additional candidates from the unknown set before constructing the tool chain for execution\.
- •ReAct\(Yaoet al\.[2023](https://arxiv.org/html/2607.28692#bib.bib4)\): Integrates reasoning with action execution through an iterative Thought\-Action\-Observation loop\. The model generates a reasoning trajectory to select and execute actions, dynamically updating its plan and exploring unknown tools based on real\-time observations until completion\.
- •Reflexion\(Shinnet al\.[2023](https://arxiv.org/html/2607.28692#bib.bib5)\): Improves agent planning through verbal self\-reflection\. When an attempt fails, the model analyzes the failure mode to generate feedback, which then informs and refines subsequent planning attempts\.
- •SciToolAgent\(Dinget al\.[2025](https://arxiv.org/html/2607.28692#bib.bib1)\): Leverages a scientific tool knowledge graph to facilitate multi\-tool orchestration\. It retrieves relevant tools from the graph and searches the unknown set when gaps are identified, following which it synthesizes and executes the tool chain\.

## Appendix IDetailed Prompts for Agent Workflow

To ensure the robustness and reproducibility of SciToolAgent\-Evo, we present the core prompt templates used in our workflow, covering active tool requirement description, candidate tool reranking and scoring, experience rule generation, and reusable skill extraction\.

Prompt for Active Tool DescriptionSystem Prompt:You are a science expert\.User Prompt:You need to describe the tool capabilities required for a scientific task\.Experience:\{experience\_section\}Task:\{question\}Previous Error \(if any\):\{error\_text\}\[IMPORTANT RULE\]:1\. Each tool capability must be an atomic computational primitive, used only to derive one property or perform one transformation\. It must not merge multiple properties or operations, nor include decision logic such as comparison, filtering, or aggregation\.2\. Use the fewest and most direct steps for task analysis\. If a single tool capability can directly produce the target result, do not introduce redundant preprocessing, intermediate transformations, or repeated computations\. Each type of property should be computed only once\.3\. If the task involves state or representation transformation, tool capabilities must follow data\-form evolution, ensuring causal coherence between steps without skipping intermediate forms\.4\. Tool descriptions should be generic capability descriptions and must not include specific substance names, numerical values, candidate answers, task conclusions, or instantiated information\.5\. If the task involves multiple substances, sequences, or scientific objects, describe only the tool capabilities required for processing one object; do not introduce cross\-object comparison or selection logic\.6\. Use goal\-oriented backward reasoning by first determining the final output internally, then inferring the required inputs and tool capabilities, and finally outputting only the tool capability descriptions\.7\. Each tool description must be highly concise, with no more than 15 English words\.Now analyze the given task and list all required tool capabilities\.The required output format is as follows:\(1\) tool description 1 \(2\) tool description 2 …Do not output any additional explanation\.

Prompt for Rerank and ScoreSystem Prompt:You are a science expert\.User Prompt:Complete the following two\-step task\.Task Requirement:\{tool\_request\}Task Constraints:Category=\{target\_category\}\{expected\_input\}Candidate Tools:\{candidates\_text\}Step 1:Analyze each candidate tool based on its Function description, Input type, Output type, and Category\. Select the best\-matching tool as top\-1 and the second\-best tool as top\-2\.Step 2:Score the selected top\-1 and top\-2 tools according to the following criteria:1\. known\_top1\_feasibility\(0\.0\(0\.0–1\.0\)1\.0\): the overall feasibility of top\-1\.•1\.01\.0: Function fully matches, with aligned I/O and Category\.•0\.750\.75: Function strongly matches, with minor I/O or Category mismatch\.•0\.50\.5: Function partially matches, with moderate I/O or Category mismatch\.•0\.250\.25: Function is weakly relevant, with clear I/O or Category mismatch\.•0\.00\.0: Function is completely mismatched\.2\. known\_margin: the rationale for prioritizing top\-1 over top\-2\.•1\.01\.0: top\-1 is clearly better than top\-2\.•0\.750\.75: top\-1 is slightly better than top\-2\.•0\.50\.5: top\-1 and top\-2 are comparable\.•0\.250\.25: top\-2 is slightly better than top\-1\.•0\.00\.0: top\-2 is clearly better than top\-1\.3\. ontology\_alignment\_score\(0\.0\(0\.0–1\.0\)1\.0\): the overall ontology\-level alignment between top\-1 and the task\.•1\.01\.0: Category and I/O types are fully aligned\.•0\.750\.75: Category matches, with minor I/O type mismatch\.•0\.50\.5: Category matches but I/O types differ, or I/O matches but Category differs\.•0\.250\.25: Category and I/O both differ but remain partially relevant\.•0\.00\.0: Category or I/O types are completely mismatched\.Strictly output the response invalid JSONformat only\. The requiredJSONformat is as follows:\{"top1": "ToolName1","top2": "ToolName2","known\_top1\_feasibility": F,"known\_margin": M,"ontology\_alignment\_score": O\}

Prompt for Generating Experience RulesSystem Prompt:You are a science expert\.User Prompt:Based on the diagnosis and the rules, generate highly reusable experience rules\.Question:\{question\}Diagnosis:\{diagnosis\}Please generate the following fields:1\. general\_prompt:One general principle applicable to scientific tool planning\.2\. domain:Strictly choose one from the following: biology, chemistry, materials\.3\. domain\_prompt:A robust tool mapping rule targeting the diagnosed error\.The core expression must follow:“When requiring \[specific scientific context/constraint\], select a tool that provides \[specific functional capability\]\.”\[RULE\]:1\. All rules must provide clear and actionable operational guidance\.2\. Limit each rule to 25 words\.Strictly output the response invalid JSONformat only\. The requiredJSONformat is as follows:\{"general\_prompt": "1 universal actionable rule","domain": "biology/chemistry/materials","domain\_prompt": "domain\-specific rule"\}

Prompt for Generating Reusable SkillsSystem Prompt:You are a science expert\.User Prompt:Given a specific task and its successfully executed tool chain, extract one standardized and reusable skill\.Task:\{question\}Tool Chain:\{tool\_path\}Core Extraction Criteria:1\. Avoid over\-abstraction: preserve specific scientific properties\.2\. Remove concrete parameters: do not include numerical values, thresholds, filtering conditions, or specific substance names\.3\. Skill naming: the key Ni should use short snake\_case and reflect only the property/function combination\.4\. Skill description: the key Zi must be a coherent sequence of atomic computational primitives\. Use connectives to form one fluent description within 20 words\.Strictly output the response invalid JSONformat only\. The requiredJSONformat is as follows:\{"Ni": "property\_combination\_name","Zi": "A cohesive English sentence describing the property extraction pipeline\."\}

## Appendix JDataset Examples

In this section, we provide representative questions from OpenSciToolBench, illustrating one example for each of the four difficulty levels\.

Level\-1Question:For the molecule CCCCCCCCCCCCCCCCCC/C=C\\CCCCCCCC\(=O\)O, what is its molecular weight, and what is its Kappa1 topological descriptor?Parameters:CCCCCCCCCCCCCCCCCC/C=C\\CCCCCCCC\(=O\)OStandard\_Toolpath:\["SMILESToWeight", "GetKappa1"\]

Level\-2Question:First convert the molecule names: Propionamide, Methane, Ethane, and Butane, to their respective SMILES representations\. True or False: among these four molecules, the compound that contains at least one amide bond also has a larger number of heavy atoms than any of the other three\.Parameters:Propionamide, Methane, Ethane, ButaneStandard\_Toolpath:\["NameToSMILES", "GetAmideBondsNum", "GetHeavyAtomsNum"\]

Level\-3Question:First convert the following SLN strings to SMILES: B\-H\-B3, CH3CH2C\[\*\]HCH2Br, O\[1\]CH2CH\(@1\)CH3, C\[1\]\[H2\]CH2CH2CH2CH2@1, CH3CH3, and C\[1\]:C:CH:CH:CH:CH:@1\. Among the converted molecules, identify the one that simultaneously satisfies all of these conditions: \(1\) it is achiral, meaning its stereo code indicates no chirality; \(2\) it contains no ring system; \(3\) its Chi3v value is nonzero, indicating an acyclic branching pattern; and \(4\) it has the highest asphericity among those meeting conditions \(2\)–\(4\), indicating it is the least spherical\.Options:A\. B\-H\-B3B\. CH3CH2C\[\*\]HCH2BrC\. O\[1\]CH2CH\(@1\)CH3D\. C\[1\]\[H2\]CH2CH2CH2CH2@1E\. CH3CH3F\. C\[1\]:C:CH:CH:CH:CH:@1Parameters:B\-H\-B3, CH3CH2C\[\*\]HCH2Br, O\[1\]CH2CH\(@1\)CH3, C\[1\]\[H2\]CH2CH2CH2CH2@1, CH3CH3, C\[1\]:C:CH:CH:CH:CH:@1Standard\_Toolpath:\["SlnToSmiles", "GetStereoCodeFrom\-Smiles", "GetRingsNum", "GetChi3v", "GetAsphericity"\]

Level\-4Question:A synthetic biology team is developing a compact DNA chassis for stable mammalian expression and wants to understand whether a newly generated sequence is a viable regulatory element rather than a problematic genomic insert\. The sequence should be examined from sequence composition through structural plausibility and expression\-related properties so that the team can decide whether it is worth advancing to experimental validation\.Generate one random DNA sequence\. Predict whether it contains CpG islands that could trigger epigenetic silencing, and identify restriction enzyme recognition sites that could compromise cloning stability\. Calculate its molecular weight in both double\-stranded linear and single\-stranded circular forms for construct planning\. Identify the longest open reading frame, translate it into an amino\-acid sequence, convert the resulting protein into a 3D structure, compute its hydrophobicity profile, and predict its overall solubility\. Based on this integrated characterization, is the generated sequence more suitable as a benign engineered element or as a candidate likely to cause manufacturing or expression problems, and which features most strongly support that conclusion?Parameters:NoneStandard\_Toolpath:\["RandomDNAGeneration", "CpGIslandPrediction", "SummaryEnzymeCleavageSites", "CalculateDoubleLinearDNAMolecularWeight", "CalculateSingleCircleDNAMolecularWeight", "FindORF", "TranslateDNAtoAminoAcidSequence", "SequenceToPdb", "ComputeProtScale", "ProteinSolubilityPredictor"\]