SCI-PRM: A Tool Aware Process Reward Model for Scientific Reasoning Verification

# Sci-PRM: A Tool Aware Process Reward Model for Scientific Reasoning Verification
Source: [https://arxiv.org/html/2606.04579](https://arxiv.org/html/2606.04579)
Xiangyu Zhao1,2, Hengyuan Zhao3, Yiheng Wang2,4, Wanghan Xu2,4, Yuhao Zhou2,5, Qinglong Cao2,4, Zhiwang Zhou2,6, Lei Bai2, Wenlong Zhang2 ✉, Xiao\-Ming Wu1 ✉ 1The Hong Kong Polytechnic University2Shanghai AI Lab3National University of Singapore 4Shanghai Jiao Tong University5Sichuan University6Tongji University xiang\-yu\.zhao@connect\.polyu\.hk, zhaohengyuan99@gmail\.com zhangwenlong@pjlab\.org\.cn, xiao\-ming\.wu@polyu\.edu\.hk

###### Abstract

While Process Reward Models \(PRMs\) have achieved remarkable success in mathematical reasoning, their application in complex scientific domains—such as biology, chemistry, and physics remains largely unexplored\. Scientific problems demand not only logical rigor but also factual consistency and the precise usage of domain\-specific tools, areas where current models often suffer from hallucinations and lack of verification\. In this paper, we first construct SCIPRM70K, a large\-scale dataset featuring “Chain\-of\-Tool” trajectories that explicitly interleave reasoning with the execution of scientific tools\. Building upon this, we train an efficient reward model called Sci\-PRM to provide fine\-grained supervision on tool selection, execution accuracy, and result interpretation at each step in one inference\. Experiments demonstrate that Sci\-PRM significantly enhances foundation models in two key aspects: \(1\) it enables effective test\-time scaling via Best\-of\-N selection; and \(2\) when integrated into Reinforcement Learning, it serves as a dense reward signal that mitigates the critical issue of advantage disappearance, allowing the model to break through existing performance ceilings\. Sci\-PRM is publicly available to foster further research and innovation:[https://github\.com/InternScience/Sci\-PRM](https://github.com/InternScience/Sci-PRM)\.

![Refer to caption](https://arxiv.org/html/2606.04579v1/x1.png)Figure 1:Step\-level F1 scores of different judge models used as process rewards on four scientific benchmarks, separated by whether they contain tool\-calling in current steps\. The SoTA proprietary model GPT\-5\-Mini shows only minor gaps on non–tool\-calling steps but degrades substantially on tool\-calling steps\. In contrast, Sci\-PRM \(Ours\), trained on our curated data, maintains high accuracy on both types of steps, highlighting its advantages and specialization in scientific domains\.## 1Introduction

Large language models \(LLMs\)\(Singhet al\.,[2025](https://arxiv.org/html/2606.04579#bib.bib1); Google,[2025](https://arxiv.org/html/2606.04579#bib.bib2); Yanget al\.,[2025](https://arxiv.org/html/2606.04579#bib.bib3)\)are increasingly used as evaluators to provide training and test\-time feedback, a paradigm often referred to as*LLM\-as\-a\-Judge*\. Within this line of work, reward modeling approaches typically fall into two categories:*outcome\-supervised reward models*\(ORMs\), which assess only the final answer, and*process\-supervised reward models*\(PRMs\), which provide supervision over intermediate reasoning steps\. ORMs are appealing for their simplicity and scalability, but they can assign high reward to outputs that are correct for the wrong reasons\(Wenet al\.,[2025](https://arxiv.org/html/2606.04579#bib.bib6); Weng,[2024](https://arxiv.org/html/2606.04579#bib.bib7)\), an incidence of “false positives”\(Yuanet al\.,[2025](https://arxiv.org/html/2606.04579#bib.bib5)\)\(e\.g\., reward hacking, miracle steps\), thereby reinforcing unreliable responses\.

![Refer to caption](https://arxiv.org/html/2606.04579v1/x2.png)Figure 2:Sci\-PRM \(Ours\) vs\. Standard Verifiers\. Sci\-PRM efficiently detects code logic flaws without execution overhead \(top\) and accurately identifies citation hallucinations in literature search \(bottom\)\.Despite these advances, PRM has not been systematically developed for*scientific reasoning*\. Scientific reasoning in biology\(Liuet al\.,[2021](https://arxiv.org/html/2606.04579#bib.bib29)\), chemistry\(Zhanget al\.,[2024](https://arxiv.org/html/2606.04579#bib.bib30)\), physics\(Fenget al\.,[2025b](https://arxiv.org/html/2606.04579#bib.bib32)\), and earth science\(Zhaoet al\.,[2025b](https://arxiv.org/html/2606.04579#bib.bib31); Zhouet al\.,[2025](https://arxiv.org/html/2606.04579#bib.bib65); Fenget al\.,[2025c](https://arxiv.org/html/2606.04579#bib.bib68)\)differs fundamentally from math\-only reasoning\(Patelet al\.,[2024](https://arxiv.org/html/2606.04579#bib.bib9); Shaoet al\.,[2024](https://arxiv.org/html/2606.04579#bib.bib8); Yanget al\.,[2024](https://arxiv.org/html/2606.04579#bib.bib13)\)\. Correctness often depends not merely on symbolic results, but on evidence\-grounded reasoning that must remain consistent with domain knowledge, physical constraints, experimental measurements, and rapidly evolving factual databases\. Consequently, LLMs frequently exhibitdomain\-specific hallucinations: they may produce plausible mechanisms, parameters, or citations that areinconsistentwith known science\. Importantly, many scientific questions are naturally*tool\-verifiable*: a model can access molecular property databases, gene/protein sequence alignment tools, equation solvers, unit converters, or literature retrieval systems to validate intermediate claims\. As shown in Figure[1](https://arxiv.org/html/2606.04579#S0.F1), general\-purpose judges such as GPT\-5\-Mini maintain reasonable accuracy on ordinary reasoning steps but*degrade obviously*when evaluating trajectories involving scientific tool\-calling steps\. This gap highlights the need for domain\-specialized process judges\. However, existing PRM\(Zouet al\.,[2025](https://arxiv.org/html/2606.04579#bib.bib10); Wanget al\.,[2025a](https://arxiv.org/html/2606.04579#bib.bib44); Zhanget al\.,[2025c](https://arxiv.org/html/2606.04579#bib.bib14); Sheet al\.,[2025](https://arxiv.org/html/2606.04579#bib.bib15)\)pipelines rarely incorporate such tools, and most tool\-oriented reward modeling works\(Qianet al\.,[2025](https://arxiv.org/html/2606.04579#bib.bib21); Liet al\.,[2025b](https://arxiv.org/html/2606.04579#bib.bib17); Xiet al\.,[2025](https://arxiv.org/html/2606.04579#bib.bib18); Wuet al\.,[2025](https://arxiv.org/html/2606.04579#bib.bib19)\)focus primarily on general domains rather than scientific reasoning\.

At the same time, tool\-augmented LLMsSinghet al\.\([2025](https://arxiv.org/html/2606.04579#bib.bib1)\); Google \([2025](https://arxiv.org/html/2606.04579#bib.bib2)\); Zhaoet al\.\([2026](https://arxiv.org/html/2606.04579#bib.bib60)\); Linget al\.\([2026](https://arxiv.org/html/2606.04579#bib.bib61)\); Xuet al\.\([2025](https://arxiv.org/html/2606.04579#bib.bib66)\)have emerged as a promising solution for improving reliability\. By delegating computation and fact verification to external tools, LLMs can reduce hallucinations and obtain plausible answers\. Recent reward modeling studies have begun to explore tool use\(Qianet al\.,[2025](https://arxiv.org/html/2606.04579#bib.bib21); Liet al\.,[2025b](https://arxiv.org/html/2606.04579#bib.bib17)\), demonstrating that specialized reward models can better judge function\-calling trajectories than frontier general\-purpose LLM judges\(Zhenget al\.,[2023](https://arxiv.org/html/2606.04579#bib.bib22)\)\. Nonetheless, these efforts largely target generic tool environments\(Donget al\.,[2025](https://arxiv.org/html/2606.04579#bib.bib20); Gouet al\.,[2024](https://arxiv.org/html/2606.04579#bib.bib23); Agarwalet al\.,[2026](https://arxiv.org/html/2606.04579#bib.bib24)\)\(e\.g\., calculators, web search, simple APIs\) and emphasize response\-level preferences\. Scientific reasoning presentsadditional challenges:\(1\) choice of toolmust match the scientific subproblem;\(2\) callingmust be correct, including units, formats, and domain\-specific parameters; and\(3\) interpretation and utilizationof tool outputs must preserve constraints and uncertainty\. A reward model that cannot distinguish these failure modes provides limited signal for training or test\-time selection\. As illustrated in Figure[2](https://arxiv.org/html/2606.04579#S1.F2), our analysis compares Sci\-PRM against general judge models regarding their evaluation of common scientific tools, such as web search and code execution\. The comparison highlights that scientific domains demand a significantly higher degree of rigor in tool usage\. For instance, LLMs utilizing integrated search tools frequently suffer from“citation hallucinations”, generating plausible\-sounding but incorrect mappings between paper titles and DOIs—errors that generic verifiers often fail to localize accurately\. On the other hand, using code\-based tools is time\-consuming\. Calling scientific APIs \(e\.g\., BLAST for protein alignment or RDKit for molecular processing\) often takes 1–5 minutes per run, making them impractical for test\-time scaling or as efficient reward signals in reinforcement learning \(RL\)\. This underscores the need for a dedicated process reward model like Sci\-PRM, which can provide fast, accurate verification to advance scientific LLMs\.

We proposeSci\-PRM, a*tool\-aware process reward model*designed to judge step\-by\-step scientific reasoning through domain tools\. Sci\-PRM is trained on a new dataset SCIPRM70K of multi\-disciplinary scientific problems paired with reasoning trajectories that contain both a*chain\-of\-thought*and a*chain\-of\-tool*\(i\.e\., explicit scientific tool calls, tool outputs, and grounded interpretations\)\. To enable scalable supervision, we introduce an automated step\-level labeling pipeline that assigns fine\-grained quality labels to each reasoning and tool\-use step\. In particular, for tool\-related steps we label three dimensions:\(1\) tool selection correctness\(is the chosen tool appropriate?\),\(2\) tool calling accuracy\(are the arguments and constraints correct?\), and\(3\) result validity and utilization\(is the returned evidence correctly interpreted and used downstream?\)\. These labels allow Sci\-PRM to produce reliable scalar rewards, capturing where and how scientific reasoning fails\. In the experimental stage, we use Sci\-PRM in both the test\-time scaling and RL training to validate the effectiveness\. During test\-time scaling, given a new problem, a base model generates multiple candidate solution trajectories \(e\.g\., Best\-of\-NNsampling\)\. Sci\-PRM scores each step and aggregates trajectory\-level rewards to select the most reliable path, reducing “false positives and evidence\-inconsistent reasoning”\. We test Sci\-PRM across representative scientific reasoning benchmarks spanning several disciplines, showing that tool\-aware process supervision yields consistent gains over outcome\-only judging and over generic tool\-use critics, especially on problems requiring multi\-step verification\. In summary, our contributions are:

- •A tool\-augmented scientific dataset SCIPRM70K\. We construct a multi\-domain dataset of challenging science questions with trajectories that explicitly interleave reasoning steps and structured tool calls \(*chain\-of\-tool*\), enabling supervised learning of evidence\-grounded reasoning\.
- •Fine\-grained step\-level supervision for scientific tool use\.We propose an automated labeling framework that distinguishes tool selection, calling, and result utilization errors, providing scalable process supervision beyond math\.
- •Sci\-PRM: a tool\-aware process reward model\.We train a reward model that scores partial scientific reasoning trajectories and supports test\-time selection and self\-correction, improving reliability and reducing hallucinations\.
- •Empirical gains on scientific reasoning\.Across diverse benchmarks, Sci\-PRM improves accuracy and factual consistency compared to ORM\-style judges and generic tool\-use reward models, demonstrating the value of tool\-aware process supervision in science\.

## 2Related Works

### 2\.1Process\-Supervised Reward Modeling

Reward modeling serves as a cornerstone for aligning LLMs with human values and logical consistency\(Ouyanget al\.,[2022](https://arxiv.org/html/2606.04579#bib.bib4)\)\. While early approaches primarily relied on Outcome\-Supervised Reward Models \(ORMs\) that evaluate only the final response, recent research has shifted toward Process\-Supervised Reward Models \(PRMs\)\(Zouet al\.,[2025](https://arxiv.org/html/2606.04579#bib.bib10); Uesatoet al\.,[2022](https://arxiv.org/html/2606.04579#bib.bib12); Zhanget al\.,[2025c](https://arxiv.org/html/2606.04579#bib.bib14); Sheet al\.,[2025](https://arxiv.org/html/2606.04579#bib.bib15); Wanget al\.,[2025a](https://arxiv.org/html/2606.04579#bib.bib44)\)\. PRMs provide fine\-grained feedback on intermediate reasoning steps, which has been shown to significantly reduce logical errors and hallucinations compared to ORMs\(Lightmanet al\.,[2024](https://arxiv.org/html/2606.04579#bib.bib11); Wanget al\.,[2024](https://arxiv.org/html/2606.04579#bib.bib25)\)\. This granular supervision has proven particularly effective in domains with structured logic, such as mathematics and code generation\. However, existing PRM research\(Qianet al\.,[2025](https://arxiv.org/html/2606.04579#bib.bib21); Liet al\.,[2025b](https://arxiv.org/html/2606.04579#bib.bib17); Xiet al\.,[2025](https://arxiv.org/html/2606.04579#bib.bib18); Wuet al\.,[2025](https://arxiv.org/html/2606.04579#bib.bib19)\)focuses predominantly on general domains’ reasoning\. There is a notable scarcity of research investigating process supervision in specialized scientific domains, such as biomedicine, chemistry, and earth sciences, where reasoning requires not only logical coherence but also rigorous adherence to physical laws and experimental facts\.

### 2\.2Tool\-Augmented Reasoning and Evaluation

To overcome the limitations of parametric knowledgeWanget al\.\([2025b](https://arxiv.org/html/2606.04579#bib.bib64)\); Liuet al\.\([2025b](https://arxiv.org/html/2606.04579#bib.bib67)\), researchers have integrated external tools into LLMs\(Schicket al\.,[2023](https://arxiv.org/html/2606.04579#bib.bib34); Qinet al\.,[2024](https://arxiv.org/html/2606.04579#bib.bib35)\), enabling capabilities such as web browsing and code execution\. In the context of evaluation, recent works have explored “Tool\-assisted Judges” to enhance the reliability of reward models\. For instance, approaches like Math\-Shepherd\(Wanget al\.,[2024](https://arxiv.org/html/2606.04579#bib.bib25)\)\. While effective for general computational tasks, these approaches typically rely on generic tools \(e\.g\., calculators or search engines\) and often employ prompting strategies rather than training the reward model to intrinsically verify tool usage\. Recently, autonomous tool construction methods\(Liuet al\.,[2024](https://arxiv.org/html/2606.04579#bib.bib52)\)have been proposed\. Crucially, existing tool\-augmented evaluators lack the specialized capability to validate scientific workflows, such as verifying molecular properties via RDKit or aligning sequences via Biopython\. Consequently, they often fail to detect domain\-specific errors in parameter selection or the misinterpretation of complex scientific data\.

### 2\.3Reinforcement Learning for Scientific Reasoning

Reinforcement learning \(RL\) has recently been employed to enhance scientific reasoningZouet al\.\([2026](https://arxiv.org/html/2606.04579#bib.bib59)\); Raoet al\.\([2026](https://arxiv.org/html/2606.04579#bib.bib62)\); Fenget al\.\([2026](https://arxiv.org/html/2606.04579#bib.bib63)\); Tanget al\.\([2025](https://arxiv.org/html/2606.04579#bib.bib69)\); Baiet al\.\([2025a](https://arxiv.org/html/2606.04579#bib.bib70)\)and tool\-integrated reasoning\. Strategies that train models to interleave reasoning with execution have shown promise in improving performance on benchmarks like GSM8K\(Cobbeet al\.,[2021](https://arxiv.org/html/2606.04579#bib.bib26)\)\. However, most existing methods rely on an “execution\-in\-the\-loop” paradigm, where the model must execute the tool and receive feedback during the RL training process \(e\.g\., via PPO\(Schulmanet al\.,[2017](https://arxiv.org/html/2606.04579#bib.bib27)\)or GRPO\(Shaoet al\.,[2024](https://arxiv.org/html/2606.04579#bib.bib8)\)\)\. While feasible for lightweight tasks like arithmetic, this approach is computationally prohibitive for scientific discovery\. Calling specialized scientific packages \(e\.g\., protein folding simulations or complex chemical reaction analysis\) entails significant latency, making it impractical to execute these tools for every step during large\-scale RL training\. Our work addresses this critical bottleneck by proposingSci\-PRM\. By serving as a high\-quality proxy that scores the reasoning and tool\-usage process without requiring live execution during every update, Sci\-PRM enables efficient and scalable RL training in complex scientific domains\.

## 3Creation of the SCIPRM70K Dataset

![Refer to caption](https://arxiv.org/html/2606.04579v1/x3.png)Figure 3:Data curation process for Sci\-PRM\. The two parts on the left represent data preprocessing, while the two parts on the right encompass the automated generation of VQA and expert\-AI collaborative filtering\.### 3\.1Trajectory Preparation

#### 3\.1\.1Task Sourcing and Normalization

Most scientific benchmarks evaluate final answers but do not explicitly require tool use\(Zhenget al\.,[2025](https://arxiv.org/html/2606.04579#bib.bib41); Zhanget al\.,[2025a](https://arxiv.org/html/2606.04579#bib.bib42)\)\. To construct tool\-augmented process data, we collect diverse scientific QA tasks spanning biology, chemistry, physics, earth science, and general scientific understanding from public sources, including: general science benchmarks such asSGI\-DeepSearch\(Xuet al\.,[2025](https://arxiv.org/html/2606.04579#bib.bib66)\),SCI\-VerifyBench\(Zhenget al\.,[2025](https://arxiv.org/html/2606.04579#bib.bib41)\), and domain\-specific benchmarks such asMol\-Instruction\(Fanget al\.,[2023](https://arxiv.org/html/2606.04579#bib.bib39)\),MSEarth\(Zhaoet al\.,[2025b](https://arxiv.org/html/2606.04579#bib.bib31)\),ChemBench4K\(Zhanget al\.,[2024](https://arxiv.org/html/2606.04579#bib.bib30)\), andPHYSICS\(Fenget al\.,[2025b](https://arxiv.org/html/2606.04579#bib.bib32)\)\. We standardize all tasks into a unified record

x=\(u,q,a,m\),x=\(u,q,a,m\),\(1\)whereuuis a unique id,qqis the question,aais the gold answer, andmmdenotes optional metadata \(e\.g\., domain, dataset source, and provenance signals such as paper titles or reference identifiers\)\.

#### 3\.1\.2Tool Suite Construction

We support two main tool families:

- •Search toolsfor evidence retrieval and citation grounding\.
- •Code execution toolswith scientific libraries/APIs for quantitative computation and domain\-specific processing \(e\.g\.,rdkit,numpy,sympy,pyscf,biopython, etc\.\)\.

For Code execution tools, we run code in an executable sandbox and record standard outputs \(stdout/return values/exceptions\)\. For search tools, we return top 5 results with source information \(titles/urls/snippets when available\)\. See Table[6](https://arxiv.org/html/2606.04579#S7.T6)for details\.

### 3\.2Tool\-Augmented Trajectory Generation

We construct step\-labeled trajectories using agenerate–execute–judgeparadigm to obtain reliable step\-level supervision\.

#### 3\.2\.1Step Generation via Interleaved Execution

Given each normalized sample\(q,a,m\)\(q,a,m\), we prompt strong tool\-integrated LLMs \(e\.g\.,Gemini\-series\(Google,[2025](https://arxiv.org/html/2606.04579#bib.bib2)\)and Doubao\-series\(Guoet al\.,[2025](https://arxiv.org/html/2606.04579#bib.bib36)\)\) to generate a structured tool\-augmented reasoning trajectory\. Unlike standard Chain\-of\-Thought reasoning, our generation process isinteractive\. When the model decides to invoke a tool, the generation stream is temporarilypaused\. The system intercepts the tool call, executes it in the real environment, and appends the result \(observation\) to the history\. The model thenresumesreasoning based on this actual feedback\. This ensures that subsequent reasoning steps are grounded in valid intermediate results rather than hallucinated expectations\.

Each step in the trajectory is recorded as:

zt=\(\\displaystyle z\_\{t\}=\(step\_id,tool\_used,tool\_type,tool\_details,\\displaystyle\\texttt\{step\\\_id\},\\ \\texttt\{tool\\\_used\},\\ \\texttt\{tool\\\_type\},\\ \\texttt\{tool\\\_details\},\\tool\_output,reasoning\_process\)\.\\displaystyle\\texttt\{tool\\\_output\},\\texttt\{reasoning\\\_process\}\)\.\(2\)A final fieldfinal\_resultis stored for outcome checking\.

#### 3\.2\.2Tool Execution Environment

Validating tool steps requires a robust execution backend\. During the paused state described above:

- •Web Search:We issue the query to the search engine and return the top\-5 results \(snippets/titles/urls\) as the observation\.
- •Code Execution:We execute the generated code snippet in a controlled sandbox environment\. We capturestdout, return values, and tracebacks \(in case of exceptions\) as the tool output\.

This execution feedback closes the loop, enabling the model to self\-correct or refine its scientific reasoning based on the results\.

### 3\.3Trajectory Labeling and Quality Control

To train a robust Process Reward Model, we construct a dataset containing both positive reasoning paths and diverse negative samples\. We implement a streamlined two\-stage pipeline combining execution\-based technical validation with MCTS\-based logical verification—to assign reliable binary labelsst∈\{1,0\}s\_\{t\}\\in\\\{1,0\\\}to each step\.

Stage 1: Execution\-Based Technical Verification\. This stage assesses the technical feasibility of tool calls and provides grounded observations for subsequent reasoning\. For theweb\_searchtool, the interface returns evidence items containing titles, URLs, and summaries\. We assign a negative label \(st=0s\_\{t\}=0\) to steps resulting in unparsable queries or empty retrieval lists\. Similarly, for Python\-based tools, we execute generated code within a secure sandbox enforced by a strict 300\-second timeout\. Execution failures—including syntax errors, runtime exceptions, or timeouts—are automatically labeled as negative\. Conversely, valid execution outputs \(stdout, return values, or search summaries\) are captured and appended to the context\. These validated outputs serve as the factual basis for the next stage of logical verification\.

Stage 2: Tool\-Aware Step Labeling via MCTS Consistency\. Steps that pass technical validation may still be logically flawed or irrelevant\. Following previous automated function\-calling benchmark works\(Liuet al\.,[2024](https://arxiv.org/html/2606.04579#bib.bib52)\), we leverage LLM as the semantic checker\. To distinguish strategically sound steps from subtle reasoning errors, following previous works\(Wanget al\.,[2025a](https://arxiv.org/html/2606.04579#bib.bib44); Kuanget al\.,[2025](https://arxiv.org/html/2606.04579#bib.bib50)\)we employ Monte Carlo Tree Search \(MCTS\) consistency checks\. We performKKindependent rollouts from the current stepztz\_\{t\}to the final answer\. A step is labeled positive \(st=1s\_\{t\}=1\) only if it consistently leads to the correct gold answeraaexceeding a confidence thresholdγ\\gamma:

P​\(success\|z1:t\)=1K​∑k=1K𝕀​\(eval​\(ak,a\)=True\)≥γ\.P\(\\text\{success\}\|z\_\{1:t\}\)=\\frac\{1\}\{K\}\\sum\_\{k=1\}^\{K\}\\mathbb\{I\}\(\\text\{eval\}\(a\_\{k\},a\)=\\text\{True\}\)\\geq\\gamma\.\(3\)During these rollouts, an LLM judge acts as a critic, evaluating tool selection appropriateness, argument accuracy, and the correct utilization of the tool outputs captured in Stage 1\. Steps failing this consistency check are labeled as negative \(st=0s\_\{t\}=0\)\.

#### 3\.3\.1Unified Dataset Construction

We aggregate the validated trajectories into a final datasetD=\{\(q,a,m,\{zt\}t=1T,\{st\}t=1T\)\}D=\\\{\(q,a,m,\\\{z\_\{t\}\\\}\_\{t=1\}^\{T\},\\\{s\_\{t\}\\\}\_\{t=1\}^\{T\}\)\\\}\. Each stepztz\_\{t\}follows a unified schema:

zt=\(\\displaystyle z\_\{t\}=\(id,tool\_used,tool\_type,tool\_details,tool\_output,\\displaystyle\\texttt\{id\},\\ \\texttt\{tool\\\_used\},\\ \\texttt\{tool\\\_type\},\\ \\texttt\{tool\\\_details\},\\ \\texttt\{tool\\\_output\},reasoning\_process,step\_correctness\)\.\\displaystyle\\texttt\{reasoning\\\_process\},\\ \\texttt\{step\\\_correctness\}\)\.\(4\)This rigorous labeling process ensures the PRM is trained on grounded scientific inquiry, effectively penalizing both hallucinated tool usage and logical inconsistencies\.

### 3\.4Data Statistics

Utilizing this pipeline, we compile a multi\-disciplinary scientific reasoning dataset comprising17,818tool\-integrated trajectories and86,314annotated reasoning steps\. The data is aggregated from seven high\-quality scientific benchmarks covering biology, chemistry, physics, earth science, and general scientific inquiry\. Each trajectory contains, on average∼4\.8\\sim 4\.8steps, with domains such as earth science exhibiting longer reasoning chains due to multi\-hop verification\.

The dataset spans two major tool categories:Web Searchfor factual grounding andPython Interpretersfor computational analysis \(e\.g\., RDKit molecular computation, physics simulations\)\. We reserve1\.2ktrajectories as the evaluation set \(SCIPRM\-Bench\), while the remaining data is used for subsequent model training\. Figure[4](https://arxiv.org/html/2606.04579#S4.F4)and Table[6](https://arxiv.org/html/2606.04579#S7.T6)provide an overview\. See details in Appendix[8](https://arxiv.org/html/2606.04579#S8)\.

## 4Science Process Reward Modeling

### 4\.1Definition

We presentSci\-PRM, a*tool\-aware process reward model*for scientific reasoning\. Given a question and a partially generated reasoning trajectory with tool interactions, Sci\-PRM assigns a scalar quality score to the current step conditioned on the full history so far\. This enables \(i\)test\-time scalingvia Best\-of\-NNtrajectory selection and \(ii\)on\-policy RL trainingusing Sci\-PRM as a reward signal to train tool\-using agents with reinforcement learning\.

Formally, for each problem instance we represent a reasoning trajectory as a sequence of steps

τ=\(z1,z2,…,zT\),\\tau=\(z\_\{1\},z\_\{2\},\\ldots,z\_\{T\}\),\(5\)where each stepztz\_\{t\}is a structured record containing: whether a tool is used, the tool type, tool\-call details, tool outputs, and the natural\-language reasoning for that step\. Sci\-PRM predicts a score

s^t=fϕ​\(q,z≤t\)∈\[0,1\],\\hat\{s\}\_\{t\}=f\_\{\\phi\}\(q,z\_\{\\leq t\}\)\\in\[0,1\],\(6\)wherez≤tz\_\{\\leq t\}denotes the trajectory prefix up to steptt\. Supervisionsts\_\{t\}is obtained from automatic step\-level judging \(True/False; mapped to1/01/0\)\. We train Sci\-PRM with regression directly on these step labels, without constructing preference pairs\.

Overview\.Our pipeline consists of three stages: \(1\) constructing a scientific tool\-augmented reasoning dataset with explicit Chain\-of\-Thought and Chain\-of\-Tool; \(2\) automatically annotating each reasoning step with tool\-aware verification to obtain step\-level labels; and \(3\) training Sci\-PRM and applying it to improve reasoning via test\-time scaling and RL\.

![Refer to caption](https://arxiv.org/html/2606.04579v1/x4.png)Figure 4:Data statistics of SCIPRM70K\.
### 4\.2Dual\-Stage Training

We initialize the Sci\-PRM using the Qwen3\-VL\-8B backbone, leveraging its strong multimodal and scientific understanding capabilities\. Training follows a two\-stage curriculum: domain\-specific Supervised Fine\-Tuning \(SFT\) followed by Reinforcement Learning \(RL\) to enhance verification consistency and generalization\.

#### 4\.2\.1Supervised Fine\-Tuning \(SFT\)

The first stage focuses on injecting the model with the ability to generate structured reasoning and precise step\-level labels\. We utilize the curated datasetDDdescribed in Sec\.[3\.3](https://arxiv.org/html/2606.04579#S3.SS3), which contains high\-quality trajectories and their corresponding MCTS\-verified labels\. Unlike standard SFT that emphasizes output formatting, our verification\-centric SFT focuses on transferring domain\-specific grounding knowledge\. We filter the training traces to retain only concise and valuable reasoning processes, discarding overly verbose or unstructured responses\. For each sample, the model is trained to predict the reasoning process and the step\-correctness labelsts\_\{t\}simultaneously\. The training objective is defined as:

ℒSFT​\(θ\)=−𝔼\(q,z,s\)∼D​\[log⁡πθ​\(zt,st∣q,z<t\)\],\\mathcal\{L\}\_\{\\text\{SFT\}\}\(\\theta\)=\-\\mathbb\{E\}\_\{\(q,z,s\)\\sim D\}\\left\[\\log\\pi\_\{\\theta\}\(z\_\{t\},s\_\{t\}\\mid q,z\_\{<t\}\)\\right\],\(7\)whereπθ\\pi\_\{\\theta\}represents the model policy\. This stage ensures that the model acquires a fundamental understanding of the "stop\-and\-go" tool\-use protocol and the criteria for scientific verification\.

#### 4\.2\.2Reinforcement Learning via DAPO\-GRPO

To improve the model’s robustness and mitigate overfitting on the SFT distribution, we employDynamic Advantage Policy Optimization \(DAPO\)\(Yuet al\.,[2025](https://arxiv.org/html/2606.04579#bib.bib46)\), a refined version of Group Relative Policy Optimization \(GRPO\)\(Shaoet al\.,[2024](https://arxiv.org/html/2606.04579#bib.bib8)\)\. This framework allows the model to explore alternative reasoning paths and self\-correct based on relative performance within a group of samples\.

##### Dynamic Group Sampling and Filtering\.

For each questionqq, we sample a group ofGGreasoning trajectories\{oi\}i=1G\\\{o\_\{i\}\\\}\_\{i=1\}^\{G\}from the current policyπθ\\pi\_\{\\theta\}\. We implement dynamic sample filtering to stabilize training: trajectories that are trivially easy \(where all samples in the group succeed\) or excessively difficult \(where all fail\) are assigned lower weights or filtered out to ensure a meaningful learning signal\. To encourage efficiency in scientific inquiry, we incorporate a length penalty into the reward functionRiR\_\{i\}, penalizing redundant tool calls or circular reasoning\.

##### Optimization Objective\.

The advantage functionA^i,t\\hat\{A\}\_\{i,t\}is calculated by normalizing the final rewardRiR\_\{i\}relative to the group mean:

A^i,t=Ri−mean​\(\{Rj\}j=1G\)std​\(\{Rj\}j=1G\)\+ϵ\.\\hat\{A\}\_\{i,t\}=\\frac\{R\_\{i\}\-\\text\{mean\}\(\\\{R\_\{j\}\\\}\_\{j=1\}^\{G\}\)\}\{\\text\{std\}\(\\\{R\_\{j\}\\\}\_\{j=1\}^\{G\}\)\+\\epsilon\}\.\(8\)The overall objective is to maximize the clipped surrogate reward, ensuring the policy does not deviate too drastically from the old policyπθold\\pi\_\{\\theta\_\{\\text\{old\}\}\}:

𝒥DAPO\(θ\)=𝔼\[1G∑i=1G∑t=1\|oi\|min\(ri,t​\(θ\)​A^i,t,clip\(ri,t\(θ\),1−ϵ,1\+ϵ\)A^i,t\)\],\\begin\{split\}\\mathcal\{J\}\_\{\\text\{DAPO\}\}\(\\theta\)=\\mathbb\{E\}\\Bigg\[\\frac\{1\}\{G\}\\sum\_\{i=1\}^\{G\}\\sum\_\{t=1\}^\{\|o\_\{i\}\|\}\\min\\Big\(&r\_\{i,t\}\(\\theta\)\\hat\{A\}\_\{i,t\},\\\\ &\\text\{clip\}\\big\(r\_\{i,t\}\(\\theta\),1\-\\epsilon,1\+\\epsilon\\big\)\\hat\{A\}\_\{i,t\}\\Big\)\\Bigg\],\\end\{split\}\(9\)whereri,t​\(θ\)=πθ​\(oi,t\|q,oi,<t\)πθold​\(oi,t\|q,oi,<t\)r\_\{i,t\}\(\\theta\)=\\frac\{\\pi\_\{\\theta\}\(o\_\{i,t\}\|q,o\_\{i,<t\}\)\}\{\\pi\_\{\\theta\_\{\\text\{old\}\}\}\(o\_\{i,t\}\|q,o\_\{i,<t\}\)\}is the importance sampling ratio\. This RL stage refines the Sci\-PRM’s ability to distinguish nuanced differences in tool usage and evidence interpretation, yielding a more reliable process\-based reward signal for scientific tasks\.

### 4\.3Sci\-PRM for Inference and Training

#### 4\.3\.1Test\-Time Scaling via Best\-of\-NNTrajectory Selection

At inference time, we prompt a base model to generateNNcandidate tool\-augmented trajectories for a new questionqq\(Snellet al\.,[2025](https://arxiv.org/html/2606.04579#bib.bib43)\)\. Sci\-PRM scores each step conditioned on the prefix; we aggregate step scores into a trajectory score:

S​\(τ\)=∑t=1Tfϕ​\(q,z≤t\)−λ⋅T,S\(\\tau\)=\\sum\_\{t=1\}^\{T\}f\_\{\\phi\}\(q,z\_\{\\leq t\}\)\-\\lambda\\cdot T,\(10\)whereλ\\lambdapenalizes unnecessarily long trajectories\. We select the best trajectory

τ∗=arg⁡maxτi⁡S​\(τi\),\\tau^\{\\ast\}=\\arg\\max\_\{\\tau\_\{i\}\}S\(\\tau\_\{i\}\),\(11\)and output its final answer\. The base model is prompted with a tool\-use template \(reason–tool–answer\) to ensure the trajectory is executable and easy to score\.

#### 4\.3\.2Sci\-PRM as a Reward Model for Tool\-Using RL

Sci\-PRM can further serve as a learned reward signal to train tool\-using LLMs/MLLMs with reinforcement learning\. Compared to outcome\-only rewards, Sci\-PRM provides process\-sensitive feedback\. In practice, Sci\-PRM can be combined with auxiliary rewards such as answer correctness, format adherence, and tool execution success to stabilize training in interactive environments\.

## 5Experiments

### 5\.1Datasets

#### 5\.1\.1Dataset Setup for Training Sci\-PRM

We train Sci\-PRM using the SCIPRM70K corpus described in Section[3](https://arxiv.org/html/2606.04579#S3)\. We adopt a three\-way split to support both SFT and RL training\.

Training Set\.We allocate8\.5k trajectoriesfor SFT, covering diverse scientific domains and tool\-use patterns\. An additional8\.1k trajectoriesare split into27\.3K stepsfor RL training\.

Evaluation Set\.We hold out1\.5ktrajectories asSCIPRM\-Bench, our process\-level scientific reasoning benchmark\. The evaluation set spans all disciplines and includes both web\-search and Python\-based tasks, ensuring robust measurement of factual grounding, multi\-hop reasoning, and tool\-integrated correctness\.

#### 5\.1\.2Evaluation Benchmarks for TTS and RL

We evaluate Sci\-PRM from both Test\-Time Scaling and RL on 4 representative and challenging scientific benchmarks\. Training details can be found in Appendix[10](https://arxiv.org/html/2606.04579#S10)\. These benchmarks are separate from training data for fair comparison:

BioProBench\(Liuet al\.,[2025c](https://arxiv.org/html/2606.04579#bib.bib37)\):Focuses on rigorous procedural question answering regarding biological experiments and laboratory protocols\.

ChemBench\(Mirzaet al\.,[2024](https://arxiv.org/html/2606.04579#bib.bib38)\):Tests advanced chemical reasoning, including reaction prediction and property calculation\.

Mol\-Instructions \(Test Split\)\(Fanget al\.,[2023](https://arxiv.org/html/2606.04579#bib.bib39)\):Assesses the ability to analyze protein structures and biomolecular interactions\.

MSEarth \(MCQ\)\(Zhaoet al\.,[2025a](https://arxiv.org/html/2606.04579#bib.bib40)\):Evaluates earth science reasoning based on observational data and phenomena analysis\.

### 5\.2Baselines

We compare ourSci\-PRMagainst various baselines from two distinct perspectives: \(1\) the judging capability as a reward model\(Zhanget al\.,[2025b](https://arxiv.org/html/2606.04579#bib.bib53); Liet al\.,[2025a](https://arxiv.org/html/2606.04579#bib.bib54)\), and \(2\) the downstream reasoning performance when the reward model is used for inference scaling \(Best\-of\-NN\)\(Snellet al\.,[2025](https://arxiv.org/html/2606.04579#bib.bib43)\)and RL training\(Yuet al\.,[2025](https://arxiv.org/html/2606.04579#bib.bib46)\)\.

Multimodal Large Language Models:We select both strong proprietary and open\-source models: GPT\-5\-Mini\(Singhet al\.,[2025](https://arxiv.org/html/2606.04579#bib.bib1)\), Gemini\-3\-Flash\(Google,[2025](https://arxiv.org/html/2606.04579#bib.bib2)\), Llama\-3\.2\-11B\-Vision\-Instruct\(LLaMA team,[2024](https://arxiv.org/html/2606.04579#bib.bib48)\), GLM\-4\.6V\(Zenget al\.,[2025](https://arxiv.org/html/2606.04579#bib.bib51)\), Qwen3\-VL\-8B and Qwen3\-VL\-32B\(Baiet al\.,[2025b](https://arxiv.org/html/2606.04579#bib.bib45)\)\.

Test\-time Scaling Approches:We apply different decoding strategies to a fixed base policy model to demonstrate the gains from test\-time compute:

1. 1\.Majority Voting: SamplesNNindependent paths and selects the final answer that appears most frequently\. This relies on consensus rather than explicit verification\.
2. 2\.ORM\-Guided Best\-of\-NN: SamplesNNpaths and selects the one with the highest score according to the Outcome Reward Model described above\.
3. 3\.Sci\-PRM Best\-of\-NN\(Ours\): SamplesNNpaths and selects the trajectory with the highest aggregated process score, utilizing our fine\-grained verification of tool selection, calling, and result interpretation\.

Reward Model Baselines\.We compare Sci\-PRM against a standard outcome\-based approach\.This model is trained on the same dataset but supervises only the final result\. It treats the entire trajectory as a single unit, assigning a binary label based on the correctness of the final answer \(exact matching\), while disregarding intermediate tool usage errors or reasoning flaws\.

Table 1:Main results onSCIPRM\-Bench\. We report F1 scores for tool\-calling steps across three domains and summarize each model by Global Tool F1, Global No\-Tool F1, Overall F1, and the Tool Gap\. The Tool Gap is defined asGlobal No\-Tool F1−\-Global Tool F1, measuring how much a judge degrades when evaluating tool\-calling steps\. The full Acc/Prec/Rec/F1 table is provided in Appendix\.ModelTypeTool\-Calling F1↑\\uparrowGlobal F1↑\\uparrowTool Gap↓\\downarrowChemBenchMol\-InstructPhysicsGlobalToolNo\-ToolOverallBaselineRandomBaseline0\.38450\.33430\.32420\.35230\.35230\.60830\.53270\.2560Closed\-Source ModelsDoubao\-Seed\-1\.6Closed0\.43900\.50920\.80000\.51290\.51290\.76210\.68510\.2492GLM\-4\.6VClosed0\.47110\.42750\.59750\.46890\.46890\.85400\.73070\.3851Gemini\-3\-FlashClosed0\.47710\.59760\.81440\.55870\.55870\.78600\.72110\.2273GPT\-5\-MiniClosed0\.46300\.49870\.81560\.53380\.53380\.82940\.75920\.2956Open\-Source ModelsQwen3\-VL\-8BOpen0\.45950\.47920\.72090\.49590\.49590\.76850\.68180\.2726LLaMA\-3\.2\-11B\-VOpen0\.18680\.36440\.46240\.32130\.32130\.57540\.52810\.2541Qwen3\-VL\-32BOpen0\.42730\.43440\.58590\.45690\.45690\.84690\.75390\.3900Sci\-PRM \(Ours\)Open0\.48580\.53620\.84570\.56190\.56190\.83330\.76910\.2714

Table 2:Search Information Verification\. It evaluates the models’ ability to distinguish between authentic and hallucinated scientific papers retrieved via theweb\_searchtool\.HDR: Hallucination Detection Rate \(True Negative Rate\)\.ModelAcc\.Prec\.Rec\.F1HDRLlama\-3\.2\-11B\-V0\.30630\.32640\.68420\.44190\.0529GLM\-4\.6V0\.42140\.37780\.72030\.49560\.2265Doubao\-Search0\.43330\.37440\.63870\.47200\.2983Qwen3\-VL\-32B0\.57000\.43420\.27730\.33850\.7624Qwen3\-VL\-8B0\.62080\.52580\.43220\.47440\.7444Gemini\-3\-Flash0\.56670\.77530\.38550\.51490\.8347GPT\-5\-Mini0\.59320\.47730\.17800\.25930\.8701Sci\-PRM\(Ours\)0\.74580\.75000\.53850\.62690\.8820
### 5\.3Evaluation Results on Sci\-PRMBench

#### 5\.3\.1Execution Tools Verification

Table[2](https://arxiv.org/html/2606.04579#S5.T2)presents the comprehensive evaluation results on the Sci\-PRM Bench, more detailed results can be found in table[7](https://arxiv.org/html/2606.04579#S8.T7)in appendix\. A striking observation across all models is the significant performance gap betweenReasoning Capability\(Tool Calling=False\) andTool Utilization\(Tool Calling=True\)\. While advanced models like GPT\-5\-Mini and Gemini\-3\-Flash achieve high F1 scores in reasoning tasks \(e\.g\., 0\.8294 and 0\.7860 globally\), their performance drops precipitously when tasked with verifying scientific code usage \(dropping to 0\.5338 and 0\.5587, respectively\)\. Our model achieves the highest Global F1 score \(0\.7691\), surpassing GPT\-5\-Mini \(0\.7592\) and significantly outperforming open\-source baselines like Qwen3\-VL\-32B \(0\.6818\)\.

Tool Utilization Competence\.In the criticalTool Calling=Truesplit, our model attains the best global F1 score of 0\.5619\. Notably, in the Physics domain, our model achieves an F1 of 0\.8457, demonstrating superior understanding of physical simulation codes compared to Qwen3\-VL\-32B \(0\.7209\)\.

Reasoning Capability\.Our model maintains competitive reasoning capabilities \(F1=0\.8333 globally\), effectively matching the state\-of\-the\-art GPT\-5\-Mini \(0\.8294\)\. This indicates that our training strategy enhances the model’s sensitivity to code correctness without compromising its general scientific reasoning ability, making it a more reliable agent for time\-sensitive scientific exploration\.

#### 5\.3\.2Search Tool Verification

Beyond tool utilization, the capability to verify the authenticity of information retrieved from the web is essential for scientific AI agents\. Table[2](https://arxiv.org/html/2606.04579#S5.T2)summarizes the performance of various models indetecting hallucinationswithin retrieved academic citations\. One can observe that while some baseline models like LLaMA\-3\.2 and GLM\-4\.6V exhibit relatively high recall \(reaching 0\.6842 and 0\.7203, respectively\), their Hallucination Detection Rate \(HDR\) is significantly low \(e\.g\., 0\.0529 for LLaMA\-3\.2\)\. This indicates a “blind trust” bias, where these models tend to classify most retrieved information as authentic, even when the provided DOIs or titles are fabricated\. Conversely, models such as GPT\-5\-Mini and Gemini\-3\-Flash demonstrate a more conservative behavior with higher HDR \(over 0\.83\), but suffer from a substantial drop in recall, often misjudging real papers as hallucinations\. Our model achieves the best balance between precision and safety\. Specifically, Ours outperforms all baseline models in Accuracy \(0\.7458\), F1\-score \(0\.6269\), and HDR \(0\.8820\)\. The high HDR suggests that our model is particularly robust against “DOI hijacking” and fabricated metadata, which are common failure modes in automated scientific search\. This superior performance is attributed to the integration of rigorous bibliographic verification capabilities, enabling the model to conduct a stringent cross\-check between tool outputs and the scientific context\. Our experimental results highlight two critical advantages of integrating process\-level supervision into scientific MLLMs: the superiority of dense reward signals during training and the reliability of verification during inference\.

![Refer to caption](https://arxiv.org/html/2606.04579v1/x5.png)Figure 5:Efficacy of Sci\-PRM in Inference and Training\.\(A\) Inference: Sci\-PRM effectively closes the gap to the Best\-of\-N upper bound on Qwen3\-VL\-8B\. \(B\) Training: Sci\-PRM demonstrates superior efficiency over ORM, notably gaining \+14\.4 points on Mol\-Instruct\.Table 3:RL Training Results\. Comparison of our method against the Outcome Reward Model \(ORM\) and other representative reward models\.ModelBioProBench\-ERRChemBenchMol\-Instructions\-testReTool\-55\.1335\.62Qwen3\-VL\-8B \+ ORM68\.7465\.1840\.23Qwen3\-VL\-8B \+ Skywork66\.3262\.9639\.46Qwen3\-VL\-8B \+ Sci\-PRM69\.6368\.9054\.62Table 4:Test\-Time Scaling Results\. Evaluation of different verification and scaling strategies\.ModelBioProBench\-ERRMol\-Instructions\-testChemBenchQwen3\-VL\-8B61\.0437\.5360\.13Qwen3\-VL\-8B \+ Majority Vote @864\.2142\.1464\.65Qwen3\-VL\-8B \+ Skywork64\.9341\.7564\.36Qwen3\-VL\-8B \+ Sci\-PRM66\.0444\.2266\.17

### 5\.4Ablation Studies

#### 5\.4\.1Test\-Time Scaling

As detailed in Table[4](https://arxiv.org/html/2606.04579#S5.T4)and supported by Figure[5](https://arxiv.org/html/2606.04579#S5.F5)A, Sci\-PRM serves as a robust verifier\. We evaluate our approach against standard decoding,Majority Vote\(@8\), and a state\-of\-the\-art general\-purpose PRM, Skywork\-Reward\-Qwen3\-8BLiuet al\.\([2025a](https://arxiv.org/html/2606.04579#bib.bib55)\)\. WhileMajority Voterelies on the statistical consensus of redundant samples, it struggles when the base model exhibits systematic hallucinations \(i\.e\., when the majority is wrong\)\. Furthermore, although generic PRMs like Skywork provide token\-level verification and improve upon the base model, they lack specialized training on scientific domain reasoning and complex tool\-use operations, restricting their effectiveness in scientific scenarios\. Sci\-PRM, by meticulously evaluating the validity of the scientific reasoning trajectory, effectively filters out plausible\-sounding but factually incorrect responses\. As shown in the quantitative results, our method surpasses both Majority Vote \(@8\) and the generic PRM across all domains, and significantly closes the gap towards the empiricalBest\-of\-Nupper bound, proving that domain\-aware verifiable reasoning is a more compute\-efficient path to reliability than blind over\-generation\.

#### 5\.4\.2RL Training

In Table[3](https://arxiv.org/html/2606.04579#S5.T3)and Figure[5](https://arxiv.org/html/2606.04579#S5.F5)B, we validate the effectiveness of Sci\-PRM as a reward signal for Reinforcement Learning\. We compare Sci\-PRM against an Outcome Reward Model \(ORM\) baseline, as well as two representative reward models: the generic Skywork PRMLiuet al\.\([2025a](https://arxiv.org/html/2606.04579#bib.bib55)\)and ReToolFenget al\.\([2025a](https://arxiv.org/html/2606.04579#bib.bib58)\), a tool\-aware reward model\. Training with Sci\-PRM consistently outperforms all baselines\. The advantage of Sci\-PRM is particularly pronounced in complex reasoning tasks\. Notably, on theMol\-Instructionsbenchmark, which requires multi\-step protein function reasoning, Sci\-PRM achieves a substantial gain of\+14\.39%over the ORM\. We attribute this to sparse outcome supervision: ORM only penalizes the final answer, failing to correct intermediate logical fallacies\. Moreover, existing generic PRMs and tool\-aware RMs struggle to generalize to our setting\. Specifically, ReTool is exclusively designed for code sandbox tool interaction and lacks the capability to handle search\-based reasoning, rendering it entirely incompatible with the BioProBench evaluation\. In contrast, Sci\-PRM provides dense, step\-aware supervision that comprehends diverse tool invocations \(both search and code\), guiding the model to follow rigorous scientific principles rather than merely fitting the final answer distribution\.

#### 5\.4\.3Efficiency Analysis

To evaluate the practical deployment feasibility of scientific reasoning models, we analyzed the inference latency and accuracy trade\-offs on themol\-instructsubset of Sci\-PRMBench \(specifically instances requiring Tool Calling\)\. Table[5](https://arxiv.org/html/2606.04579#S5.T5)presents the comparison between the base model \(Qwen2\.5\-VL\-7B\), the base model augmented with a Python sandbox for code execution, and our proposedSci\-PRM\.

Table 5:Efficiency and performance on the SCIPRM\-Benchmol\-instructtest set \(Tool Calling=True\)\. Execution time includes external API latency \(e\.g\., NCBI BLAST\)\.ModelAvg\. Time \(s\)AccuracyQwen3\-VL\-8B0\.720\.51Qwen3\-VL\-8B \(w/ Execution\)0\.85 \(282\.73\*\)0\.86Sci\-PRM \(Ours\)0\.760\.76As shown in Table[5](https://arxiv.org/html/2606.04579#S5.T5), while augmenting the base model with an execution environment significantly improves accuracy \(from 0\.51 to 0\.81\), it introduces a prohibitive latency overhead\. The average time skyrockets to 282\.73 seconds per query\. This delay is primarily driven by external dependencies, such as remote server calls \(e\.g\., BLAST searches\) and complex chemical property calculations, rather than the model’s inference time itself \(which remains low at 0\.85s\)\. This magnitude of latency renders the tool\-augmented approach impractical for real\-time scientific applications\. In contrast,Sci\-PRMdemonstrates a superior balance between efficiency and accuracy\. By internalizing scientific reasoning capabilities, our model achieves an accuracy of 0\.76—closely approaching the sandbox\-augmented upper bound \(0\.86\)—while maintaining an inference speed of 0\.76 seconds\. This deficiency in tool utilization is critical in real\-world scientific workflows\. Scientific tools \(e\.g\., NCBI BLAST, molecular docking simulations\) often require substantial computational time—ranging from minutes to hours—to execute\. If an agent cannot proactively identify erroneous tool\-use code \(e\.g\., incorrect API parameters or logic errors\) before execution, it leads to severe inefficiencies and resource wastage\. The low precision in theTool Calling=Truesetting \(e\.g\., Llama\-3\.2\-11b at 0\.4954 globally\) suggests that current multimodal LLM/MLLMs struggle to distinguish between executable and buggy scientific code, often hallucinating that incorrect code is valid\.

## 6Conclusion

We present Sci\-PRM, a tool\-aware process reward model designed to enforce rigor in scientific reasoning\. By supervising step\-by\-step tool usage—spanning selection, calling, and interpretation—Sci\-PRM effectively mitigates hallucinations where general judges and outcome\-based models fail\. Our empirical results demonstrate that Sci\-PRM not only identifies citation fabrications and code logic errors without expensive execution overhead but also significantly boosts downstream performance via inference\-time verification\. This work underscores the necessity of fine\-grained process supervision for building trustworthy and verifiable scientific AI agents\.

## 7Acknowledgment

This work is supported by Shanghai Artificial Intelligence Laboratory\.

## References

- ToolRM: outcome reward models for tool\-calling large language models\.External Links:2509\.11963,[Link](https://arxiv.org/abs/2509.11963)Cited by:[§1](https://arxiv.org/html/2606.04579#S1.p3.1)\.
- L\. Bai, Z\. Cai, Y\. Cao, M\. Cao, W\. Cao, C\. Chen, H\. Chen, K\. Chen, P\. Chen, Y\. Chen,et al\.\(2025a\)Intern\-s1: a scientific multimodal foundation model\.arXiv preprint arXiv:2508\.15763\.Cited by:[§2\.3](https://arxiv.org/html/2606.04579#S2.SS3.p1.1)\.
- S\. Bai, Y\. Cai, R\. Chen, K\. Chen, X\. Chen, Z\. Cheng, L\. Deng, W\. Ding, C\. Gao, C\. Ge, W\. Ge, Z\. Guo, Q\. Huang, J\. Huang, F\. Huang, B\. Hui, S\. Jiang, Z\. Li, M\. Li, M\. Li, K\. Li, Z\. Lin, J\. Lin, X\. Liu, J\. Liu, C\. Liu, Y\. Liu, D\. Liu, S\. Liu, D\. Lu, R\. Luo, C\. Lv, R\. Men, L\. Meng, X\. Ren, X\. Ren, S\. Song, Y\. Sun, J\. Tang, J\. Tu, J\. Wan, P\. Wang, P\. Wang, Q\. Wang, Y\. Wang, T\. Xie, Y\. Xu, H\. Xu, J\. Xu, Z\. Yang, M\. Yang, J\. Yang, A\. Yang, B\. Yu, F\. Zhang, H\. Zhang, X\. Zhang, B\. Zheng, H\. Zhong, J\. Zhou, F\. Zhou, J\. Zhou, Y\. Zhu, and K\. Zhu \(2025b\)Qwen3\-vl technical report\.External Links:2511\.21631,[Link](https://arxiv.org/abs/2511.21631)Cited by:[§5\.2](https://arxiv.org/html/2606.04579#S5.SS2.p2.1)\.
- K\. Cobbe, V\. Kosaraju, M\. Bavarian, M\. Chen, H\. Jun, L\. Kaiser, M\. Plappert, J\. Tworek, J\. Hilton, R\. Nakano, C\. Hesse, and J\. Schulman \(2021\)Training verifiers to solve math word problems\.External Links:2110\.14168,[Link](https://arxiv.org/abs/2110.14168)Cited by:[§2\.3](https://arxiv.org/html/2606.04579#S2.SS3.p1.1)\.
- G\. Dong, Y\. Chen, X\. Li, J\. Jin, H\. Qian, Y\. Zhu, H\. Mao, G\. Zhou, Z\. Dou, and J\. Wen \(2025\)Tool\-star: empowering llm\-brained multi\-tool reasoner via reinforcement learning\.External Links:2505\.16410,[Link](https://arxiv.org/abs/2505.16410)Cited by:[§1](https://arxiv.org/html/2606.04579#S1.p3.1)\.
- Y\. Fang, X\. Liang, N\. Zhang, K\. Liu, R\. Huang, Z\. Chen, X\. Fan, and H\. Chen \(2023\)Mol\-instructions: a large\-scale biomolecular instruction dataset for large language models\.arXiv preprint arXiv:2306\.08018\.Cited by:[§3\.1\.1](https://arxiv.org/html/2606.04579#S3.SS1.SSS1.p1.5),[§5\.1\.2](https://arxiv.org/html/2606.04579#S5.SS1.SSS2.p4.1.1)\.
- J\. Feng, S\. Huang, X\. Qu, G\. Zhang, Y\. Qin, B\. Zhong, C\. Jiang, J\. Chi, and W\. Zhong \(2025a\)Retool: reinforcement learning for strategic tool use in llms\.arXiv preprint arXiv:2504\.11536\.Cited by:[§5\.4\.2](https://arxiv.org/html/2606.04579#S5.SS4.SSS2.p1.1)\.
- K\. Feng, Y\. Zhao, Y\. Liu, T\. Yang, C\. Zhao, J\. Sous, and A\. Cohan \(2025b\)Physics: benchmarking foundation models on university\-level physics problem solving\.InFindings of the Association for Computational Linguistics: ACL 2025,W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),Vienna, Austria,pp\. 11717–11743\.External Links:[Link](https://aclanthology.org/2025.findings-acl.610/),[Document](https://dx.doi.org/10.18653/v1/2025.findings-acl.610),ISBN 979\-8\-89176\-256\-5Cited by:[§1](https://arxiv.org/html/2606.04579#S1.p2.1),[§3\.1\.1](https://arxiv.org/html/2606.04579#S3.SS1.SSS1.p1.5)\.
- P\. Feng, Z\. Lv, J\. Ye, X\. Wang, X\. Huo, J\. Yu, W\. Xu, W\. Zhang, L\. Bai, C\. He,et al\.\(2025c\)Earth\-agent: unlocking the full landscape of earth observation with agents\.arXiv preprint arXiv:2509\.23141\.Cited by:[§1](https://arxiv.org/html/2606.04579#S1.p2.1)\.
- S\. Feng, R\. Ma, X\. Yan, Y\. Fan, Y\. Hu, S\. Huang, S\. Zhang, Z\. Cao, T\. Peng, J\. Yuan,et al\.\(2026\)Internagent\-1\.5: a unified agentic framework for long\-horizon autonomous scientific discovery\.arXiv preprint arXiv:2602\.08990\.Cited by:[§2\.3](https://arxiv.org/html/2606.04579#S2.SS3.p1.1)\.
- Google \(2025\)Gemini 3Google DeepMind\.External Links:[Link](https://aistudio.google.com/models/gemini-3)Cited by:[§1](https://arxiv.org/html/2606.04579#S1.p1.1),[§1](https://arxiv.org/html/2606.04579#S1.p3.1),[§3\.2\.1](https://arxiv.org/html/2606.04579#S3.SS2.SSS1.p1.1),[§5\.2](https://arxiv.org/html/2606.04579#S5.SS2.p2.1)\.
- Z\. Gou, Z\. Shao, Y\. Gong, yelong shen, Y\. Yang, N\. Duan, and W\. Chen \(2024\)CRITIC: large language models can self\-correct with tool\-interactive critiquing\.InThe Twelfth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=Sx038qxjek)Cited by:[§1](https://arxiv.org/html/2606.04579#S1.p3.1)\.
- D\. Guo, F\. Wu, F\. Zhu, F\. Leng, G\. Shi, H\. Chen, H\. Fan, J\. Wang, J\. Jiang, J\. Wang,et al\.\(2025\)Seed1\. 5\-vl technical report\.arXiv preprint arXiv:2505\.07062\.Cited by:[§3\.2\.1](https://arxiv.org/html/2606.04579#S3.SS2.SSS1.p1.1)\.
- P\. Kuang, X\. Wang, W\. Liu, J\. Dong, and K\. Xu \(2025\)TIM\-prm: verifying multimodal reasoning with tool\-integrated prm\.arXiv preprint arXiv:2511\.22998\.Cited by:[§3\.3](https://arxiv.org/html/2606.04579#S3.SS3.p3.5)\.
- L\. Li, Y\. Wei, Z\. Xie, X\. Yang, Y\. Song, P\. Wang, C\. An, T\. Liu, S\. Li, B\. Y\. Lin, L\. Kong, and Q\. Liu \(2025a\)VL\-rewardbench: a challenging benchmark for vision\-language generative reward models\.InCVPR,Cited by:[§5\.2](https://arxiv.org/html/2606.04579#S5.SS2.p1.1)\.
- Q\. Li, X\. Dai, X\. Li, W\. Zhang, Y\. Wang, R\. Tang, and Y\. Yu \(2025b\)CodePRM: execution feedback\-enhanced process reward model for code generation\.InFindings of the Association for Computational Linguistics: ACL 2025,W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),Vienna, Austria,pp\. 8169–8182\.External Links:[Link](https://aclanthology.org/2025.findings-acl.428/),[Document](https://dx.doi.org/10.18653/v1/2025.findings-acl.428),ISBN 979\-8\-89176\-256\-5Cited by:[§1](https://arxiv.org/html/2606.04579#S1.p2.1),[§1](https://arxiv.org/html/2606.04579#S1.p3.1),[§2\.1](https://arxiv.org/html/2606.04579#S2.SS1.p1.1)\.
- H\. Lightman, V\. Kosaraju, Y\. Burda, H\. Edwards, B\. Baker, T\. Lee, J\. Leike, J\. Schulman, I\. Sutskever, and K\. Cobbe \(2024\)Let’s verify step by step\.InThe Twelfth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=v8L0pN6EOi)Cited by:[§2\.1](https://arxiv.org/html/2606.04579#S2.SS1.p1.1)\.
- F\. Ling, Z\. Guo, J\. Wang, W\. Wei, X\. Yue, Z\. Jiang, W\. Xu, J\. Luo, L\. Cheng, Y\. Ham,et al\.\(2026\)A self\-evolving ai agent system accelerating the understanding of climate change and variability\.Cited by:[§1](https://arxiv.org/html/2606.04579#S1.p3.1)\.
- B\. Liu, L\. Zhan, L\. Xu, L\. Ma, Y\. Yang, and X\. Wu \(2021\)SLAKE: a semantically\-labeled knowledge\-enhanced dataset for medical visual question answering\.External Links:2102\.09542,[Link](https://arxiv.org/abs/2102.09542)Cited by:[§1](https://arxiv.org/html/2606.04579#S1.p2.1)\.
- C\. Y\. Liu, L\. Zeng, Y\. Xiao, J\. He, J\. Liu, C\. Wang, R\. Yan, W\. Shen, F\. Zhang, J\. Xu,et al\.\(2025a\)Skywork\-reward\-v2: scaling preference data curation via human\-ai synergy\.arXiv preprint arXiv:2507\.01352\.Cited by:[§5\.4\.1](https://arxiv.org/html/2606.04579#S5.SS4.SSS1.p1.1),[§5\.4\.2](https://arxiv.org/html/2606.04579#S5.SS4.SSS2.p1.1)\.
- H\. Liu, J\. Liu, S\. Liu, H\. Duan, Y\. Li, M\. Su, X\. Liu, G\. Zhai, X\. Fang, Q\. Ma,et al\.\(2025b\)ATLAS: a high\-difficulty, multidisciplinary benchmark for frontier scientific reasoning\.arXiv preprint arXiv:2511\.14366\.Cited by:[§2\.2](https://arxiv.org/html/2606.04579#S2.SS2.p1.1)\.
- Y\. Liu, L\. Lv, X\. Zhang, L\. Yuan, and Y\. Tian \(2025c\)BioProBench: comprehensive dataset and benchmark in biological protocol understanding and reasoning\.arXiv preprint arXiv:2505\.07889\.Cited by:[§5\.1\.2](https://arxiv.org/html/2606.04579#S5.SS1.SSS2.p2.1.1)\.
- Z\. Liu, T\. Hoang, J\. Zhang, M\. Zhu, T\. Lan, J\. Tan, W\. Yao, Z\. Liu, Y\. Feng, R\. RN,et al\.\(2024\)Apigen: automated pipeline for generating verifiable and diverse function\-calling datasets\.Advances in Neural Information Processing Systems37,pp\. 54463–54482\.Cited by:[§2\.2](https://arxiv.org/html/2606.04579#S2.SS2.p1.1),[§3\.3](https://arxiv.org/html/2606.04579#S3.SS3.p3.5)\.
- LLaMA team \(2024\)Llama 3\.2: revolutionizing edge ai and vision with open, customizable modelsLLaMA team\.External Links:[Link](https://ai.meta.com/blog/llama-3-2-connect-2024-vision-edge-mobile-devices)Cited by:[§5\.2](https://arxiv.org/html/2606.04579#S5.SS2.p2.1)\.
- A\. Mirza, N\. Alampara, S\. Kunchapu, M\. Ríos\-García, B\. Emoekabu, A\. Krishnan, T\. Gupta, M\. Schilling\-Wilhelmi, M\. Okereke, A\. Aneesh,et al\.\(2024\)Are large language models superhuman chemists?\.arXiv preprint arXiv:2404\.01475\.Cited by:[§5\.1\.2](https://arxiv.org/html/2606.04579#S5.SS1.SSS2.p3.1.1)\.
- L\. Ouyang, J\. Wu, X\. Jiang, D\. Almeida, C\. L\. Wainwright, P\. Mishkin, C\. Zhang, S\. Agarwal, K\. Slama, A\. Ray, J\. Schulman, J\. Hilton, F\. Kelton, L\. Miller, M\. Simens, A\. Askell, P\. Welinder, P\. Christiano, J\. Leike, and R\. Lowe \(2022\)Training language models to follow instructions with human feedback\.External Links:2203\.02155,[Link](https://arxiv.org/abs/2203.02155)Cited by:[§2\.1](https://arxiv.org/html/2606.04579#S2.SS1.p1.1)\.
- B\. Patel, S\. Chakraborty, W\. A\. Suttle, M\. Wang, A\. S\. Bedi, and D\. Manocha \(2024\)AIME: ai system optimization via multiple llm evaluators\.External Links:2410\.03131,[Link](https://arxiv.org/abs/2410.03131)Cited by:[§1](https://arxiv.org/html/2606.04579#S1.p2.1)\.
- C\. Qian, E\. C\. Acikgoz, Q\. He, H\. WANG, X\. Chen, D\. Hakkani\-Tür, G\. Tur, and H\. Ji \(2025\)ToolRL: reward is all tool learning needs\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=eOLdGbXT6t)Cited by:[§1](https://arxiv.org/html/2606.04579#S1.p2.1),[§1](https://arxiv.org/html/2606.04579#S1.p3.1),[§2\.1](https://arxiv.org/html/2606.04579#S2.SS1.p1.1)\.
- Y\. Qin, S\. Liang, Y\. Ye, K\. Zhu, L\. Yan, Y\. Lu, Y\. Lin, X\. Cong, X\. Tang, B\. Qian, S\. Zhao, L\. Hong, R\. Tian, R\. Xie, J\. Zhou, M\. Gerstein, dahai li, Z\. Liu, and M\. Sun \(2024\)ToolLLM: facilitating large language models to master 16000\+ real\-world APIs\.InThe Twelfth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=dHng2O0Jjr)Cited by:[§2\.2](https://arxiv.org/html/2606.04579#S2.SS2.p1.1)\.
- J\. Rao, Y\. Qiu, J\. Zhang, J\. Deng, S\. Sun, F\. Ling, H\. Chen, N\. Dong, Z\. Gao, S\. Sun,et al\.\(2026\)SciDataCopilot: an agentic data preparation framework for agi\-driven scientific discovery\.arXiv preprint arXiv:2602\.09132\.Cited by:[§2\.3](https://arxiv.org/html/2606.04579#S2.SS3.p1.1)\.
- T\. Schick, J\. Dwivedi\-Yu, R\. Dessi, R\. Raileanu, M\. Lomeli, E\. Hambro, L\. Zettlemoyer, N\. Cancedda, and T\. Scialom \(2023\)Toolformer: language models can teach themselves to use tools\.InThirty\-seventh Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=Yacmpz84TH)Cited by:[§2\.2](https://arxiv.org/html/2606.04579#S2.SS2.p1.1)\.
- J\. Schulman, F\. Wolski, P\. Dhariwal, A\. Radford, and O\. Klimov \(2017\)Proximal policy optimization algorithms\.External Links:1707\.06347,[Link](https://arxiv.org/abs/1707.06347)Cited by:[§2\.3](https://arxiv.org/html/2606.04579#S2.SS3.p1.1)\.
- Z\. Shao, P\. Wang, Q\. Zhu, R\. Xu, J\. Song, X\. Bi, H\. Zhang, M\. Zhang, Y\. K\. Li, Y\. Wu, and D\. Guo \(2024\)DeepSeekMath: pushing the limits of mathematical reasoning in open language models\.External Links:2402\.03300,[Link](https://arxiv.org/abs/2402.03300)Cited by:[§1](https://arxiv.org/html/2606.04579#S1.p2.1),[§2\.3](https://arxiv.org/html/2606.04579#S2.SS3.p1.1),[§4\.2\.2](https://arxiv.org/html/2606.04579#S4.SS2.SSS2.p1.1)\.
- S\. She, J\. Liu, Y\. Liu, J\. Chen, X\. Huang, and S\. Huang \(2025\)R\-PRM: reasoning\-driven process reward modeling\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,C\. Christodoulopoulos, T\. Chakraborty, C\. Rose, and V\. Peng \(Eds\.\),Suzhou, China,pp\. 13438–13451\.External Links:[Link](https://aclanthology.org/2025.emnlp-main.679/),[Document](https://dx.doi.org/10.18653/v1/2025.emnlp-main.679),ISBN 979\-8\-89176\-332\-6Cited by:[§1](https://arxiv.org/html/2606.04579#S1.p2.1),[§2\.1](https://arxiv.org/html/2606.04579#S2.SS1.p1.1)\.
- A\. Singh, A\. Fry, A\. Perelman, A\. Tart, A\. Ganesh, A\. El\-Kishky, A\. McLaughlin, A\. Low, A\. Ostrow, A\. Ananthram,et al\.\(2025\)Openai gpt\-5 system card\.arXiv preprint arXiv:2601\.03267\.Cited by:[§1](https://arxiv.org/html/2606.04579#S1.p1.1),[§1](https://arxiv.org/html/2606.04579#S1.p3.1),[§5\.2](https://arxiv.org/html/2606.04579#S5.SS2.p2.1)\.
- C\. V\. Snell, J\. Lee, K\. Xu, and A\. Kumar \(2025\)Scaling LLM test\-time compute optimally can be more effective than scaling parameters for reasoning\.InThe Thirteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=4FWAwZtd2n)Cited by:[§4\.3\.1](https://arxiv.org/html/2606.04579#S4.SS3.SSS1.p1.2),[§5\.2](https://arxiv.org/html/2606.04579#S5.SS2.p1.1)\.
- X\. Tang, W\. Xu, Y\. Wang, Z\. Guo, D\. Shao, J\. Chen, C\. Zhang, Z\. Wang, L\. Zhang, G\. Wan,et al\.\(2025\)Eigen\-1: adaptive multi\-agent refinement with monitor\-based rag for scientific reasoning\.arXiv preprint arXiv:2509\.21193\.Cited by:[§2\.3](https://arxiv.org/html/2606.04579#S2.SS3.p1.1)\.
- J\. Uesato, N\. Kushman, R\. Kumar, F\. Song, N\. Siegel, L\. Wang, A\. Creswell, G\. Irving, and I\. Higgins \(2022\)Solving math word problems with process\- and outcome\-based feedback\.External Links:2211\.14275,[Link](https://arxiv.org/abs/2211.14275)Cited by:[§2\.1](https://arxiv.org/html/2606.04579#S2.SS1.p1.1)\.
- P\. Wang, L\. Li, Z\. Shao, R\. Xu, D\. Dai, Y\. Li, D\. Chen, Y\. Wu, and Z\. Sui \(2024\)Math\-shepherd: verify and reinforce LLMs step\-by\-step without human annotations\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),L\. Ku, A\. Martins, and V\. Srikumar \(Eds\.\),Bangkok, Thailand,pp\. 9426–9439\.External Links:[Link](https://aclanthology.org/2024.acl-long.510/),[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.510)Cited by:[§2\.1](https://arxiv.org/html/2606.04579#S2.SS1.p1.1),[§2\.2](https://arxiv.org/html/2606.04579#S2.SS2.p1.1)\.
- W\. Wang, Z\. Gao, L\. Chen, Z\. Chen, J\. Zhu, X\. Zhao, Y\. Liu, Y\. Cao, S\. Ye, X\. Zhu,et al\.\(2025a\)VisualPRM: an effective process reward model for multimodal reasoning\.arXiv preprint arXiv:2503\.10291\.Cited by:[§1](https://arxiv.org/html/2606.04579#S1.p2.1),[§2\.1](https://arxiv.org/html/2606.04579#S2.SS1.p1.1),[§3\.3](https://arxiv.org/html/2606.04579#S3.SS3.p3.5)\.
- Y\. Wang, Y\. Chen, S\. Li, Y\. Zhou, B\. Liu, H\. Gao, J\. Yuan, J\. Bu, W\. Xu, Y\. Zhou,et al\.\(2025b\)SciEvalKit: an open\-source evaluation toolkit for scientific general intelligence\.arXiv preprint arXiv:2512\.22334\.Cited by:[§2\.2](https://arxiv.org/html/2606.04579#S2.SS2.p1.1)\.
- J\. Wen, R\. Zhong, A\. Khan, E\. Perez, J\. Steinhardt, M\. Huang, S\. R\. Bowman, H\. He, and S\. Feng \(2025\)Language models learn to mislead humans via RLHF\.InThe Thirteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=xJljiPE6dg)Cited by:[§1](https://arxiv.org/html/2606.04579#S1.p1.1)\.
- L\. Weng \(2024\)Reward hacking in reinforcement learning\.\.lilianweng\.github\.io\.External Links:[Link](https://lilianweng.github.io/posts/2024-11-28-reward-hacking/)Cited by:[§1](https://arxiv.org/html/2606.04579#S1.p1.1)\.
- F\. Wu, W\. Zhu, Y\. Zhang, S\. Chatterjee, J\. Zhu, F\. Mo, R\. Luo, and J\. Gao \(2025\)PORTool: tool\-use llm training with rewarded tree\.External Links:2510\.26020,[Link](https://arxiv.org/abs/2510.26020)Cited by:[§1](https://arxiv.org/html/2606.04579#S1.p2.1),[§2\.1](https://arxiv.org/html/2606.04579#S2.SS1.p1.1)\.
- Z\. Xi, C\. Liao, G\. Li, Y\. Yang, W\. Chen, Z\. Zhang, B\. Wang, S\. Jin, Y\. Zhou, J\. Guan, W\. Wu, T\. Ji, T\. Gui, Q\. Zhang, and X\. Huang \(2025\)AgentPRM: process reward models for llm agents via step\-wise promise and progress\.External Links:2511\.08325,[Link](https://arxiv.org/abs/2511.08325)Cited by:[§1](https://arxiv.org/html/2606.04579#S1.p2.1),[§2\.1](https://arxiv.org/html/2606.04579#S2.SS1.p1.1)\.
- W\. Xu, Y\. Zhou, Y\. Zhou, Q\. Cao, S\. Li, J\. Bu, B\. Liu, Y\. Chen, X\. He, X\. Zhao,et al\.\(2025\)Probing scientific general intelligence of llms with scientist\-aligned workflows\.arXiv preprint arXiv:2512\.16969\.Cited by:[§1](https://arxiv.org/html/2606.04579#S1.p3.1),[§3\.1\.1](https://arxiv.org/html/2606.04579#S3.SS1.SSS1.p1.5)\.
- A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv, C\. Zheng, D\. Liu, F\. Zhou, F\. Huang, F\. Hu, H\. Ge, H\. Wei, H\. Lin, J\. Tang, J\. Yang, J\. Tu, J\. Zhang, J\. Yang, J\. Yang, J\. Zhou, J\. Zhou, J\. Lin, K\. Dang, K\. Bao, K\. Yang, L\. Yu, L\. Deng, M\. Li, M\. Xue, M\. Li, P\. Zhang, P\. Wang, Q\. Zhu, R\. Men, R\. Gao, S\. Liu, S\. Luo, T\. Li, T\. Tang, W\. Yin, X\. Ren, X\. Wang, X\. Zhang, X\. Ren, Y\. Fan, Y\. Su, Y\. Zhang, Y\. Zhang, Y\. Wan, Y\. Liu, Z\. Wang, Z\. Cui, Z\. Zhang, Z\. Zhou, and Z\. Qiu \(2025\)Qwen3 technical report\.External Links:2505\.09388,[Link](https://arxiv.org/abs/2505.09388)Cited by:[§1](https://arxiv.org/html/2606.04579#S1.p1.1)\.
- A\. Yang, B\. Zhang, B\. Hui, B\. Gao, B\. Yu, C\. Li, D\. Liu, J\. Tu, J\. Zhou, J\. Lin, K\. Lu, M\. Xue, R\. Lin, T\. Liu, X\. Ren, and Z\. Zhang \(2024\)Qwen2\.5\-math technical report: toward mathematical expert model via self\-improvement\.External Links:2409\.12122,[Link](https://arxiv.org/abs/2409.12122)Cited by:[§1](https://arxiv.org/html/2606.04579#S1.p2.1)\.
- Q\. Yu, Z\. Zhang, R\. Zhu, Y\. Yuan, X\. Zuo, Y\. Yue, W\. Dai, T\. Fan, G\. Liu, L\. Liu, X\. Liu, H\. Lin, Z\. Lin, B\. Ma, G\. Sheng, Y\. Tong, C\. Zhang, M\. Zhang, W\. Zhang, H\. Zhu, J\. Zhu, J\. Chen, J\. Chen, C\. Wang, H\. Yu, Y\. Song, X\. Wei, H\. Zhou, J\. Liu, W\. Ma, Y\. Zhang, L\. Yan, M\. Qiao, Y\. Wu, and M\. Wang \(2025\)DAPO: an open\-source llm reinforcement learning system at scale\.External Links:2503\.14476,[Link](https://arxiv.org/abs/2503.14476)Cited by:[§4\.2\.2](https://arxiv.org/html/2606.04579#S4.SS2.SSS2.p1.1),[§5\.2](https://arxiv.org/html/2606.04579#S5.SS2.p1.1)\.
- Y\. Yuan, Q\. Mang, J\. Chen, H\. Wan, X\. Liu, J\. Xu, J\. Huang, W\. Wang, W\. Jiao, and P\. He \(2025\)Curing miracle steps in llm mathematical reasoning with rubric rewards\.External Links:2510\.07774,[Link](https://arxiv.org/abs/2510.07774)Cited by:[§1](https://arxiv.org/html/2606.04579#S1.p1.1)\.
- A\. Zeng, X\. Lv, Q\. Zheng, Z\. Hou, B\. Chen, C\. Xie, C\. Wang, D\. Yin, H\. Zeng, J\. Zhang,et al\.\(2025\)Glm\-4\.5: agentic, reasoning, and coding \(arc\) foundation models\.arXiv preprint arXiv:2508\.06471\.Cited by:[§5\.2](https://arxiv.org/html/2606.04579#S5.SS2.p2.1)\.
- D\. Zhang, W\. Liu, Q\. Tan, J\. Chen, H\. Yan, Y\. Yan, J\. Li, W\. Huang, X\. Yue, D\. Zhou, S\. Zhang, M\. Su, H\. Zhong, Y\. Li, and W\. Ouyang \(2024\)ChemLLM: a chemical large language model\.External Links:2402\.06852Cited by:[§1](https://arxiv.org/html/2606.04579#S1.p2.1),[§3\.1\.1](https://arxiv.org/html/2606.04579#S3.SS1.SSS1.p1.5)\.
- T\. Zhang, M\. Cao, A\. Lam, S\. Zhang, and K\. Chen \(2025a\)Compassjudger\-2: towards generalist judge model via verifiable rewards\.arXiv preprint arXiv:2507\.09104\.Cited by:[§3\.1\.1](https://arxiv.org/html/2606.04579#S3.SS1.SSS1.p1.5)\.
- Y\. Zhang, H\. Yang, H\. Zhang, Y\. Shi, Z\. Chen, H\. Tian, C\. Fu, H\. Wang, K\. Wu, B\. Cui, X\. Wang, J\. Pan, H\. Wang, Z\. Zhang, and L\. Wang \(2025b\)BaseReward: a strong baseline for multimodal reward model\.External Links:2509\.16127,[Link](https://arxiv.org/abs/2509.16127)Cited by:[§5\.2](https://arxiv.org/html/2606.04579#S5.SS2.p1.1)\.
- Z\. Zhang, C\. Zheng, Y\. Wu, B\. Zhang, R\. Lin, B\. Yu, D\. Liu, J\. Zhou, and J\. Lin \(2025c\)The lessons of developing process reward models in mathematical reasoning\.InFindings of the Association for Computational Linguistics: ACL 2025,W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),Vienna, Austria,pp\. 10495–10516\.External Links:[Link](https://aclanthology.org/2025.findings-acl.547/),[Document](https://dx.doi.org/10.18653/v1/2025.findings-acl.547),ISBN 979\-8\-89176\-256\-5Cited by:[§1](https://arxiv.org/html/2606.04579#S1.p2.1),[§2\.1](https://arxiv.org/html/2606.04579#S2.SS1.p1.1)\.
- S\. Zhao, F\. Liu, X\. Zhang, H\. Chen, X\. Gu, Z\. Jiang, F\. Ling, B\. Fei, W\. Zhang, J\. Wang,et al\.\(2026\)Openearth\-agent: from tool calling to tool creation for open\-environment earth observation\.arXiv preprint arXiv:2603\.22148\.Cited by:[§1](https://arxiv.org/html/2606.04579#S1.p3.1)\.
- X\. Zhao, W\. Xu, B\. Liu, Y\. Zhou, F\. Ling, B\. Fei, X\. Yue, L\. Bai, W\. Zhang, and X\. Wu \(2025a\)MSEarth: a benchmark for multimodal scientific comprehension of earth science\.arXiv preprint arXiv:2505\.20740\.Cited by:[§5\.1\.2](https://arxiv.org/html/2606.04579#S5.SS1.SSS2.p5.1.1)\.
- X\. Zhao, W\. Xu, B\. Liu, Y\. Zhou, F\. Ling, B\. Fei, X\. Yue, L\. Bai, W\. Zhang, and X\. Wu \(2025b\)MSEarth: a multimodal scientific dataset and benchmark for phenomena uncovering in earth science\.External Links:2505\.20740,[Link](https://arxiv.org/abs/2505.20740)Cited by:[§1](https://arxiv.org/html/2606.04579#S1.p2.1),[§3\.1\.1](https://arxiv.org/html/2606.04579#S3.SS1.SSS1.p1.5)\.
- L\. Zheng, W\. Chiang, Y\. Sheng, S\. Zhuang, Z\. Wu, Y\. Zhuang, Z\. Lin, Z\. Li, D\. Li, E\. Xing, H\. Zhang, J\. E\. Gonzalez, and I\. Stoica \(2023\)Judging LLM\-as\-a\-judge with MT\-bench and chatbot arena\.InThirty\-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track,External Links:[Link](https://openreview.net/forum?id=uccHPGDlao)Cited by:[§1](https://arxiv.org/html/2606.04579#S1.p3.1)\.
- S\. Zheng, C\. Huang, F\. Yu, J\. Yao, J\. Ye, T\. Chen, Y\. Luo, N\. Ding, L\. Bai, G\. Cui,et al\.\(2025\)Sci\-verifier: scientific verifier with thinking\.arXiv preprint arXiv:2509\.24285\.Cited by:[§3\.1\.1](https://arxiv.org/html/2606.04579#S3.SS1.SSS1.p1.5)\.
- Z\. Zhou, Y\. Pu, X\. He, Y\. Liu, Y\. Chen, J\. Gong, X\. Zhuang, W\. Xu, Q\. Cao, S\. Tang,et al\.\(2025\)Omni\-weather: unified multimodal foundation model for weather generation and understanding\.arXiv preprint arXiv:2512\.21643\.Cited by:[§1](https://arxiv.org/html/2606.04579#S1.p2.1)\.
- J\. Zou, L\. Yang, J\. Gu, J\. Qiu, K\. Shen, J\. He, and M\. Wang \(2025\)ReasonFlux\-PRM: trajectory\-aware PRMs for long chain\-of\-thought reasoning in LLMs\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=f3sZjkQbv2)Cited by:[§1](https://arxiv.org/html/2606.04579#S1.p2.1),[§2\.1](https://arxiv.org/html/2606.04579#S2.SS1.p1.1)\.
- Y\. Zou, D\. Zhu, L\. Zhu, T\. Zhu, Y\. Zhou, P\. Zhou, X\. Zhou, D\. Zhou, Z\. Zhou, Y\. Zhou,et al\.\(2026\)Intern\-s1\-pro: scientific multimodal foundation model at trillion scale\.arXiv preprint arXiv:2603\.25040\.Cited by:[§2\.3](https://arxiv.org/html/2606.04579#S2.SS3.p1.1)\.

Table 6:Statistics of theSci\-PRMtraining dataset after filtering\. The dataset covers diverse scientific domains\.Stepsdenotes total reasoning steps, whileTool Usagecounts specific tool callings\. The top\-5 most frequently used packages are listed for Python\-based tasks\.DisciplineSource BenchmarkTool TypeQA SamplesStepsTool UsageTop\-5 Packages \(Freq\)General ScienceSGI\-DeepResearchWeb Search3181,350650search\_engine\(650\)SCI\-VerifyBenchSearch / Python2,50012,4223,309sympy\(411\),math\(227\)numpy\(200\),rdkit\(193\)itertools\(118\)Earth ScienceMSEarthWeb Search5,00016,2695,117search\_engine\(5,117\)BiologyMol\-InstructionsPython5,00018,9758,620requests\(1,848\),time\(1,732\)Bio\(1,623\),json\(365\)bioservices\(211\)ChemistryChemBench4KPython4,00014,1704,860rdkit\(4,046\),requests\(186\)pandas\(111\),numpy\(106\)bs4\(72\)PhysicsPHYSICS \(en\)Python1,0008,9582,292sympy\(1,387\),math\(780\)numpy\(501\),symbols\(164\)Step\(99\)TotalAll SourcesMixed17,81872,14424,848\-

## 8Dataset

### 8\.1Dataset Details

We constructed a comprehensive dataset designed to enhance the model’s capabilities in scientific reasoning and domain\-specific tool utilization\. As presented in Table[6](https://arxiv.org/html/2606.04579#S7.T6), the curated training dataset encompasses five major scientific disciplines:General Science,Earth Science,Biology,Chemistry, andPhysics\. The final training corpus consists of 17,818 quality\-filtered QA samples, accumulating over 72K steps and nearly 25K specific tool invocations\.

The dataset is distinguished by its rich integration of external tools, categorized into Web Search and Python\-based code execution\.

- •ForGeneral ScienceandEarth Science, the model interacts primarily with search engines \(e\.g\., sources from SGI\-DeepResearch and MSEarth\) to retrieve up\-to\-date scientific knowledge\.
- •For more specialized domains likeBiology,Chemistry, andPhysics, the data involves complex Python interactions\. As shown in the statistics, the dataset captures high\-frequency usage of domain\-specific libraries, such asrdkitfor chemical structure analysis,Bio\(Biopython\) for genomic data processing, andsympyfor physical equation solving\.

##### SCIPRM\-Bench Construction\.

To rigorously evaluate the model’s performance on unseen data, we constructed a dedicated held\-out test set namedSCIPRM\-Bench\. This benchmark was created by randomly selecting a subset of data from four key source benchmarks prior to the training set filtering process\. Specifically, the test set composition is as follows: \(1\)400 samplesfrom Mol\-Instructions \(Biology\); \(2\)300 samplesfrom MSEarth \(Earth Science\); \(3\)400 samplesfrom ChemBench4K \(Chemistry\); and \(4\)200 samplesfrom PHYSICS \(Physics\)\. The remaining data from these sources, along with the General Science samples, constitute the training set detailed in Table[6](https://arxiv.org/html/2606.04579#S7.T6)\.

### 8\.2Trajectory Construction

To construct reasoning paths that leverage the web\_search function for solving scientific problems, we employ Doubao\-seed\-1\.6 \(web\_search\) as the path generation model\. This model is capable of returning the top\-5 search results for a given query, providing the specific title, URL, and summary information for each retrieval\.

Conversely, to construct reasoning paths that utilize Python scientific packages or API functions, we adoptGemini\-3\-Flashas the generative model\. To ensure the robustness of code execution, we established a sandbox environment pre\-installed with a comprehensive suite of scientific packages likely to be utilized during the reasoning process\. This isolated environment guarantees that the generated code runs smoothly without interruptions caused by missing dependencies or installation errors\.

Table 7:Results onModelBench\. To highlight the advantages in reasoning, rows forTool Calling=Trueare highlighted with a background color\. Models are categorized into Closed\-Source and Open\-Source\.ModelSplitChemBenchMol\-InstructPhysicsGlobalAccPrecRecF1AccPrecRecF1AccPrecRecF1AccPrecRecF1BaselineRandomTool Calling=True0\.50000\.31240\.50000\.38450\.50000\.25110\.50000\.33430\.50000\.23990\.50000\.32420\.50000\.27190\.50000\.3523Tool Calling=False0\.50000\.84520\.50000\.62830\.50000\.94290\.50000\.65350\.50000\.52010\.50000\.50980\.50000\.77650\.50000\.6083Overall0\.50000\.65900\.50000\.56860\.50000\.55540\.50000\.52620\.50000\.42960\.50000\.46210\.50000\.57000\.50000\.5327Closed\-Source ModelsDoubao\-Seed\-1\.6Tool Calling=True0\.37840\.30570\.77840\.43900\.58630\.36260\.85470\.50920\.88470\.68520\.96100\.80000\.56620\.36910\.84000\.5129Tool Calling=False0\.70850\.90710\.72980\.80880\.44820\.96200\.43180\.59610\.79490\.75360\.90000\.82030\.67180\.87160\.67710\.7621Overall0\.59310\.67500\.73780\.70500\.52550\.57810\.53890\.55780\.82390\.73950\.91100\.81640\.62860\.66290\.70890\.6851GLM\-4\.6VTool Calling=True0\.36710\.31880\.90210\.47110\.36890\.27680\.93850\.42750\.69780\.43900\.93510\.59750\.43200\.31440\.92220\.4689Tool Calling=False0\.75610\.87730\.82700\.85140\.86790\.96330\.89390\.92730\.71320\.65920\.92860\.77110\.77020\.84300\.86520\.8540Overall0\.62010\.66870\.83950\.74440\.58840\.58340\.90520\.70950\.70820\.60430\.92970\.73250\.63180\.62660\.87640\.7307Gemini\-3\-FlashTool Calling=True0\.39290\.32640\.88660\.47710\.76580\.52540\.69270\.59760\.90340\.75560\.88310\.81440\.65260\.42670\.80890\.5587Tool Calling=False0\.83820\.92200\.88330\.90220\.23930\.95540\.20270\.33440\.87370\.86910\.89140\.88010\.70780\.91120\.69110\.7860Overall0\.68260\.70750\.88390\.78590\.53420\.66380\.32670\.43790\.88330\.84630\.88990\.86760\.68520\.72830\.71410\.7211GPT\-5\-MiniTool Calling=True0\.54430\.36640\.62890\.46300\.73770\.47940\.51960\.49870\.89720\.71570\.94810\.81560\.69610\.45790\.64000\.5338Tool Calling=False0\.70500\.92400\.70930\.80250\.79290\.95580\.81820\.88160\.80090\.76470\.89140\.82320\.75260\.89250\.77470\.8294Overall0\.64880\.75250\.69600\.72320\.76200\.81270\.74260\.77610\.83200\.75490\.90160\.82180\.72950\.77040\.74840\.7592Open\-Source ModelsQwen3\-VL\-8BTool Calling=True0\.34460\.30950\.89180\.45950\.50910\.32660\.89940\.47920\.85050\.65260\.80520\.72090\.51360\.34520\.88000\.4959Tool Calling=False0\.70760\.87280\.76560\.81570\.47860\.94360\.47540\.63220\.75780\.70920\.90570\.79550\.66810\.83820\.70940\.7685Overall0\.58080\.65040\.78650\.71200\.49570\.54280\.58270\.56210\.78770\.69930\.88760\.78220\.60480\.63010\.74270\.6818LLaMA\-3\.2\-11B\-VTool Calling=True0\.66340\.38100\.12370\.18680\.78960\.75440\.24020\.36440\.71030\.41670\.51950\.46240\.72690\.49540\.23780\.3213Tool Calling=False0\.46710\.84250\.45450\.59040\.39820\.94010\.38640\.54770\.57800\.60510\.54290\.57230\.48220\.79210\.45180\.5754Overall0\.53570\.79320\.39970\.53150\.61740\.90150\.34940\.50360\.62070\.56100\.53860\.54960\.58230\.74180\.41000\.5281Qwen3\-VL\-32BTool Calling=True0\.49920\.33240\.59790\.42730\.66760\.37920\.50840\.43440\.74450\.47930\.75320\.58590\.61930\.37320\.58890\.4569Tool Calling=False0\.70330\.85070\.78710\.81770\.89460\.96810\.91860\.94270\.75480\.70880\.89710\.79190\.76270\.84850\.84530\.8469Overall0\.63200\.70630\.75580\.73020\.76750\.77730\.81470\.79560\.75150\.65960\.87120\.75080\.70400\.71660\.79520\.7539Model \(Ours\)Tool Calling=True0\.56360\.38440\.65980\.48580\.75740\.51550\.55870\.53620\.91590\.75510\.96100\.84570\.71540\.48320\.67110\.5619Tool Calling=False0\.71020\.92460\.71550\.80670\.79820\.95600\.82390\.88500\.80530\.76900\.89430\.82690\.75760\.89430\.78010\.8333Overall0\.65900\.75940\.70620\.73190\.77530\.82430\.75670\.78910\.84100\.76630\.90630\.83050\.74040\.77980\.75880\.7691

## 9Implementation Details

All the experiments are conducted on two NVIDIA H200 GPUs\. We use Qwen3\-VL\-8B\-Instruct as the base model and employ the ms\-swift framework for training\. We adopt Low\-Rank Adaptation \(LoRA\) for parameter\-efficient fine\-tuning across both stages\. First, we perform Supervised Fine\-Tuning \(SFT\) for 1 epoch with a learning rate of1×10−41\\times 10^\{\-4\}and a warmup ratio of 0\.05\. Subsequently, we align the model using DAPO with a learning rate of5×10−75\\times 10^\{\-7\}, a group size of 8 generations, and a KL coefficientβ\\betaof 0\.001\. As for the fine\-grained beam search with Sci\-PRM, we set the temperature as 0\.8, and select the number of beamsNNand the beam widthMMfrom\{1,2,4,8\}\\\{1,2,4,8\\\}\. The detailed hyperparameters for SFT and RL phases are listed in Table[8](https://arxiv.org/html/2606.04579#S9.T8)and Table[9](https://arxiv.org/html/2606.04579#S9.T9), respectively\.

Table 8:SFT Configurations for SCI\-Verifier\.ParameterValueBase ModelQwen3\-VL\-8B\-InstructPrecisionBF16Training Frameworkms\-swiftTraining TypeLoRALoRA Rank / Alpha8 / 32Target Modulesall\-linearOptimization StrategyDeepSpeed Zero2Learning Rate1×10−41\\times 10^\{\-4\}Warmup Ratio0\.05Max Sequence Length16384Number of Training Epochs1Per Device Train Batch Size2Gradient Accumulation Steps2GPUs Per Node1Table 9:RL \(DAPO\) Configurations for SCI\-Verifier\.ParameterValueRL AlgorithmDAPOPrecisionBF16Training TypeLoRA \(Rank 8, Alpha 32\)Optimization StrategyDeepSpeed Zero2Learning Rate5×10−75\\times 10^\{\-7\}Warmup Ratio0\.01KL Coefficient \(β\\beta\)0\.001Temperature1\.0Number of Generations8Max Prompt Length4096Max Completion Length4096Number of Training Epochs1Number of Iterations1Per Device Train Batch Size2Gradient Accumulation Steps16Reward Functionverdicts\_acc, soft\_overlong
## 10PRM & ORM Training Details

To validate the effectiveness of Sci\-PRM as a reward model within the Reinforcement Learning framework, we conduct comparative experiments based on the Qwen3\-VL\-8B model\. We employ Group Relative Policy Optimization \(GRPO\) as the training algorithm\. To assess the impact of dense process supervision, we compare two distinct reward configurations:

- •ORM\-based GRPO:The model is trained using an Outcome Reward Model \(ORM\) that provides sparse feedback based solely on the correctness of the final answer\.
- •Sci\-PRM\-based GRPO:The model is trained using our proposed Sci\-PRM, which provides fine\-grained, step\-wise reward signals throughout the reasoning process\.

The training dataset is constructed to cover diverse scientific tasks, sourced from the MSEarth training dataset, Mol\-Instruct training dataset, BioProtein training dataset and ChemBench4K\. Detailed hyperparameters for the GRPO training phase are provided in Table[10](https://arxiv.org/html/2606.04579#S10.T10)\. The quantitative results are summarized in Table[12](https://arxiv.org/html/2606.04579#S10.T12)\. As illustrated, Reinforcement Learning significantly enhances the model’s performance compared to the supervised fine\-tuning \(SFT\) baseline \(Qwen3\-VL\-8B\)\. While the ORM\-based GRPO yields consistent improvements across all domains \(e\.g\., improving from 60\.13 to 65\.18 on ChemBench and 61\.04 to 68\.74 on BioProBench\), the Sci\-PRM\-based approach demonstrates superior performance on all benchmarks\. Notably, the advantage of dense process supervision is most pronounced in complex reasoning tasks\. On theMol\-Instructionsbenchmark, which requires intricate biomolecular reasoning, the ORM method only achieves a modest gain \(40\.23 vs\. 37\.53\)\. In contrast, our Sci\-PRM achieves a remarkable score of54\.62, outperforming the ORM baseline by over 14 points\. This substantial margin indicates that for tasks involving long chains of thought, sparse outcome signals are insufficient, whereas fine\-grained step\-wise rewards provided by Sci\-PRM successfully guide the model toward correct reasoning paths\.

Table 10:RL \(GRPO\) Configurations\.ParameterValueRL AlgorithmGRPOPrecisionBF16Training TypeLoRA \(Rank 8, Alpha 32\)Optimization StrategyDeepSpeed Zero2Learning Rate5×10−75\\times 10^\{\-7\}Warmup Ratio0\.01KL Coefficient \(β\\beta\)0\.001Temperature1\.0Number of Generations8Max Prompt Length4096Max Completion Length4096Number of Training Epochs1Number of Iterations1Per Device Train Batch Size2Gradient Accumulation Steps16Reward Functionverdicts\_acc, soft\_overlongTable 11:Performance comparison on scientific benchmarks\.BenchmarkQwen3\-VLMV@8\+Sci\-PRMBest\-of\-NMsearth49\.2149\.5751\.6655\.68BioProBench61\.0464\.2166\.0474\.56Mol\-Instructions37\.5342\.1444\.2256\.68ChemBench60\.1364\.6566\.1772\.79
Note:MV@8: Majority Vote \(8 samples\)\. The proprietary model \(Mol\-Instructions\) achieves a Rouge\-L score of 52\.00\. Bold values denote the best performance\.

Table 12:Performance comparison of RL training methods\. We compare the base model \(SFT\) against RL training with ORM and our Sci\-PRM\.BenchmarkBase ModelRL Training\(Qwen3\-VL\)\+ORM\+Sci\-PRM \(Ours\)Msearth49\.2152\.1253\.72BioProBench\-ERR61\.0468\.7469\.63ChemBench60\.1365\.1868\.90Mol\-Instructions37\.5340\.2354\.62
Note:All models are initialized from Qwen3\-VL\-8B\. The\+ORMbaseline uses a standard Outcome Reward Model, while\+Sci\-PRMutilizes our fine\-grained process reward model\. Bold values indicate the best performance\.

## 11Model

We evaluate a diverse set of large language models \(LLMs\) covering both general\-purpose and tool\-aware reasoning systems:

- •Doubao\-Seed\-1\.6: A general\-purpose instruction\-following LLM deployed via an API service\. We use it as a strong industrial baseline for structured\-output generation and tool\-aware reasoning, since it generally supports stable chat\-style prompting and deterministic decoding settings\.
- •Gemini\-3\-Flash: A latency\- and throughput\-optimized model designed for fast response\. We include it to examine how an efficiency\-oriented model behaves on verification tasks, especially for enforcing strict output formats and maintaining consistent reasoning across multi\-step prompts\.
- •Qwen3\-VL\-8B: A compact model from the Qwen “VL” family \(8B scale\)\. It serves as a smaller\-capacity baseline to characterize the effect of limited model scale on \(i\) compliance with step schemas, \(ii\) detection of flawed reasoning/tool misuse, and \(iii\) overall verification accuracy\.
- •LLaMA\-3\.2\-11B\-Vision\-Instruct: An instruction\-tuned model from the Llama family with vision\-language capabilities\. Although our evaluation focuses on text\-based verification prompts, this model is included for breadth and to assess how a general instruction\-following model \(with multimodal pretraining\) performs on structured, tool\-aware reasoning\-chain verification\.
- •Qwen3\-VL\-32B: A larger\-capacity model in the Qwen “VL” family \(32B scale\)\. We use it as a higher\-capacity open\-model baseline to quantify scaling effects relative toQwen3\-VL\-8B, and to test whether increased capacity improves the detection of subtle logical errors and inconsistent tool usage\.
- •gpt\-5\-mini: A compact GPT\-family model used as a high\-quality commercial baseline\. We include it to benchmark verification performance under strong instruction\-following behavior, particularly on strict schema compliance, tool\-specification correctness \(query/code/API usage\), and robustness across different splits\.

## 12SCIPRM Reasoning\-Chain Dataset and Prompts

### 12\.1Dataset Construction Details

Biomolecular \(Protein & Molecular\) DataYou are a helpful assistant and a professional bioinformatics researcher\. You are in an English\-speaking environment\. Please answer strictly in English\.Question:\{question\}Input Protein Sequence:\{sequence\}Answer this question\. You can use tools; if you use tools, you must provide specific code\. I will run the code and give you the output for your next step of reasoning\. The code needs to be directly runnable and concise\. Do not require manual API key insertion if possible\.OUTPUT FORMAT \(STRICT\):Return STRICT JSON only \(no extra text\)\. The root MUST be a JSON ARRAY; each element is one reasoning step\. Each step MUST include: \- step\_id: number \- tool\_used: boolean \- tool\_type: string \("python\_code" / "scientific\_api" / "none"\) \- tool\_details: string \(runnable code / exact query; empty string if tool\_used is false\) \- reasoning: string \(what the step checks and how it supports discriminating among options; do not invent tool results\)

Physical Sciences DataYou are a rigorous physics and scientific researcher\. Given a physics problem, produce an evidence\-supported reasoning chain to solve it\. You may propose tool steps \(especially runnable Python code for calculation\), but you cannot execute tools yourself\. If you need computation, consolidate everything into fewer Python code blocks\. All other steps must have tool\_type ’none’\.Question: \{question\}I need an evidence\-supported reasoning chain to solve this problem\. You may use tools such as Python packages to support the calculation and you need to give me the final results\. If you use tools, you must provide specific code\. The code needs to be directly runnable and concise\. Do not require manual API key insertion if possible\.TOOL RULES \(STRICT\): \- If a tool is needed, put ONLY directly runnable code \(or an exact query string\) in tool\_details\. \- Do NOT include any explanation, comments, markdown, or extra text inside tool\_details\.PYTHON\_CODE RULES: \- If tool\_type is "python\_code", tool\_details MUST be a single runnable Python snippet\. \- The code MUST end with at least one print\(\.\.\.\) that prints the final computed evidence \(e\.g\., a number or dict\)\. \- Do not rely on implicit/interactive display of variables\.OUTPUT FORMAT \(STRICT\): Return STRICT JSON only \(no extra text\)\. The root MUST be a JSON ARRAY; each element is one reasoning step\. Each step MUST include: \- step\_id: number \- tool\_used: boolean \- tool\_type: string \("python\_code" / "scientific\_api" / "none"\) \- tool\_details: string \(runnable code / exact query; empty string if tool\_used is false\) \- reasoning: string \(what the step checks/calculates and how it contributes to the solution\)

Chemical Sciences DataSYSTEM PROMPT:You are a rigorous chemistry/cheminformatics researcher\. Given a multiple\-choice question asking which compounds are commonly used to synthesize a target SMILES, you must produce an evidence\-supported reasoning chain\. You are allowed to propose tool steps \(especially runnable Python/RDKit code\), but you cannot execute tools yourself\.USER PROMPT TEMPLATE:Question: \{question\}Options: A: \{A\} B: \{B\} C: \{C\} D: \{D\}I need an evidence\-supported reasoning chain\. You may use tools such as Python packages \(e\.g\., RDKit\) to support the reasoning\. If you use tools, you must provide specific code\. The code needs to be directly runnable and concise\. Do not require manual API key insertion if possible\. I will run the code and provide the outputs to you later; therefore, do NOT fabricate tool outputs\.IMPORTANT tool rule: \- If a tool is needed, put ONLY directly runnable code \(or an exact query string\) in tool\_details\. \- Do NOT include any explanation, comments, markdown, or extra text inside tool\_details\.Output requirement: Output the answer in STRICT JSON format\. The ROOT must be a JSON ARRAY, where each element is one reasoning step\. Each step MUST include the following fields: \- tool\_used: boolean \(true/false\) \- tool\_type: string \("python\_code" / "scientific\_api" / "none"\) \- tool\_details: string \(runnable code; or empty string if tool\_used is false\) \- reasoning: string \(explain what this step is verifying\)Additional constraints: \- Prefer python\_code steps that verify structural compatibility \(substructure matching, key functional\-group motifs, molecular\-weight sanity checks\)\. \- If you can reasonably decide without tools, you may set tool\_used=false, but you should still provide evidence\-based chemical logic\. \- Do NOT output the final choice separately; the chain should make the decision clear in the reasoning steps\.Return STRICT JSON only \(no extra text\)\.

Earth & Environmental Sciences DataSYSTEM PROMPT:You are a careful clinical/scientific reasoning assistant\. You must produce STRICT JSON only \(no extra text\)\. If you use any tool, you must describe exact usage \(query string / runnable python code / API endpoint\)\.USER PROMPT \(ANSWER TEMPLATE\):Question: \{question\}You can use tools to help you answer \(web\_search / python\_code / scientific\_api / none\)\. Return the answer step by step in English\.Output STRICT JSON with this schema: \{ "steps": \[ \{ "step\_id": 1, "tool\_used": true/false, "tool\_type": "web\_search"\|"python\_code"\|"scientific\_api"\|"none", "tool\_details": "exact query string OR full runnable python code OR exact API usage OR ’none’", "tool\_output": "retrieved info / code execution result / api response summary / ’none’", "reasoning\_process": "explain the reasoning for this step" \} \], "final\_result": "final answer to the question" \}Constraints: \- If you choose web\_search, include the exact query string\. \- If you choose python\_code, include complete runnable code \(no placeholders\)\. \- If you do not use tools in a step, set tool\_type="none", tool\_details="none", tool\_output="none"\. Return STRICT JSON only\.Constraints: \- Tool usage requirements are the same: exact query / runnable python / explicit API usage\. \- step\_correctness should assess the logic/content of that step \(not tool availability\)\. Return STRICT JSON only\.

### 12\.2Evaluation Prompts

Evaluation Prompts \(Tool\-use Verifiers\)A\. Evaluation: Questions Requiring Code Execution or API ToolsSYSTEM PROMPT:You are an expert process verifier\. Your task is to evaluate the proposed reasoning step and tool\-usage code BEFORE execution\. Check if the logic is sound and the code is correct for the given problem\. The execution result of the previous step \(if any\) is provided as context\. Response format: "Valid" or "Invalid" only\.USER PROMPT TEMPLATE \(per step\):Question: \{question\}Observation \(Output from previous step\) \[optional\]: \{previous\_tool\_output\}Current Step \{step\_id\}: Reasoning: \{reasoning\}Tool Type: \{tool\_type\} \[only if tool\_used=true\] Proposed Code: \[only if tool\_used=true\] \{tool\_details\}Is this step valid and correct?ASSISTANT OUTPUT FORMAT \(STRICT\):Return exactly one token: "Valid" or "Invalid"\. No extra text\.B\. Evaluation: Prompts Requiring Retrieval / Search ToolsSYSTEM PROMPT:You are an expert scientific fact\-checker and researcher\. Your task is to verify the authenticity and relevance of a specific academic paper/citation retrieved by an AI agent during a problem\-solving process\. Strictly output your response in JSON format\.USER PROMPT TEMPLATE \(per step to verify\):Problem ContextContext Paper Title: \{context\_title\} Question: \{question\} Image Caption: \{caption\}Reasoning History\{history\} \(or "No previous steps\."\)Step to Verify \(Web Search / Paper Retrieval\)Tool Used: \{tool\_type\} Tool Input \(Query\): \{tool\_input\} Tool Output \(Retrieved Paper/Info\): \{tool\_output\}Reasoning: \{reasoning\}Your Verification Tasks:1\.Authenticity Check \(Crucial\):\- Does the paper mentioned in the Tool Output actually exist? \- Do the Title, Author \(if any\), Year, and DOI \(if have\) match a real publication? \- Important: If the DOI is fake, or if the title does not belong to the DOI, mark it as Hallucinated\.2\.Relevance Check:\- If the paper exists, is it helpful for answering the specific Question above?OUTPUT FORMAT \(STRICT JSON\):\{ "status": "Authentic" or "Hallucinated", "analysis": "Step\-by\-step verification logic\. First state if the DOI/Title exists\. Then state if it is relevant\." \}NOTES:\- Return "Authentic" ONLY if the paper is real AND the metadata \(DOI/Year if have\) is consistent\. \- Return "Hallucinated" if the paper does not exist, the DOI is fake, or the title/DOI mismatch\.

## 13Case Study

### 13\.1Case 1: Code Execution Verification

This case demonstrates the verification of a Python script interacting with the InterProScan API\. The proposed code contains logic errors \(incorrect header usage and return parsing\)\.

Input: Protein Analysis RequestQuestion:Analyze the protein corresponding to the amino acid sequence:MSDSEQT\.\.\.\(truncated\)\.Proposed Step:Submit sequence to InterProScan via EBI REST API to identify domains\.Tool Type:python\_codeProposed Code:[⬇](data:text/plain;base64,aW1wb3J0IHJlcXVlc3RzCmRlZiBzdWJtaXRfaW50ZXJwcm9zY2FuKHNlcXVlbmNlLCBlbWFpbCk6CiB1cmwgPSAiaHR0cHM6Ly93d3cuZWJpLmFjLnVrL1Rvb2xzL3NlcnZpY2VzL3Jlc3QvaW50ZXJwcm9zY2FuL3J1biIKIGhlYWRlcnMgPSB7CiAgIkNvbnRlbnQtVHlwZSI6ICJhcHBsaWNhdGlvbi94LXd3dy1mb3JtLXVybGVuY29kZWQiLAogICJBY2NlcHQiOiAiYXBwbGljYXRpb24vanNvbiIsCiAgIlgtRU1BSUwiOiBlbWFpbCAjIEluY29ycmVjdCBwbGFjZW1lbnQKIH0KIGRhdGEgPSB7CiAgInNlcXVlbmNlIjogc2VxdWVuY2UsCiAgInR5cGUiOiAicHJvdGVpbiIsCiAgInRvb2wiOiAiaW50ZXJwcm9zY2FuIiwKICAic3R5cGUiOiAic2VxdWVuY2UiCiB9CiByZXNwb25zZSA9IHJlcXVlc3RzLnBvc3QodXJsLCBoZWFkZXJzPWhlYWRlcnMsIGRhdGE9ZGF0YSkKIGlmIHJlc3BvbnNlLnN0YXR1c19jb2RlID09IDIwMDoKICByZXR1cm4gcmVzcG9uc2UuanNvbigpICMgSW5jb3JyZWN0IHJlc3BvbnNlIHBhcnNpbmcKIGVsc2U6CiAgcmV0dXJuIGYiRXJyb3I6IHtyZXNwb25zZS5zdGF0dXNfY29kZX0iCgpzZXEgPSAiTVNEU0VRVC4uLiIKcmVzdWx0ID0gc3VibWl0X2ludGVycHJvc2NhbihzZXEsICJ1c2VyQGV4YW1wbGUuY29tIikKcHJpbnQoIkpvYiBJRDoiLCByZXN1bHQuZ2V0KCdqb2JJZCcpKQ==)importrequestsdefsubmit\_interproscan\(sequence,email\):url="https://www\.ebi\.ac\.uk/Tools/services/rest/interproscan/run"headers=\{"Content\-Type":"application/x\-www\-form\-urlencoded","Accept":"application/json","X\-EMAIL":email\}data=\{"sequence":sequence,"type":"protein","tool":"interproscan","stype":"sequence"\}response=requests\.post\(url,headers=headers,data=data\)ifresponse\.status\_code==200:returnresponse\.json\(\)else:returnf"Error:␣\{response\.status\_code\}"seq="MSDSEQT\.\.\."result=submit\_interproscan\(seq,"user@example\.com"\)print\("Job␣ID:",result\.get\(’jobId’\)\)Actual Execution Result \(Stderr\):AttributeError: ’str’ object has no attribute ’get’

Model EvaluationsOurs:\[Invalid\]Reason:The code incorrectly passes ’email’ in headers instead of body, uses invalid ’stype’, and attempts to parse plain\-text Job ID as JSON\.Qwen3\-VL\-8B:\[Valid\]Reason:The endpoint URL and headers are logically aligned… no runtime errors are evident in the logic\.GPT\-5\-Mini:\[Invalid\]Reason:The EBI Tools endpoint expects form fields like "email" and typically returns a plain\-text job identifier… relying onresponse\.json\(\)will fail\.

### 13\.2Case 2: Search Result Verification

This example illustrates the verification of a retrieved citation regarding the "Southern Annular Mode\."

Input: Search VerificationQuestion:Which meteorological phenomenon is linked to stronger zonal winds in this figure? \(Figure context provided\)\.Retrieved Item:•Title:The Southern Annular Mode and its influence on Southern Hemisphere climate•DOI:10\.1029/1999JD901133•Year:2000DOI Ground Truth:•DOI:10\.1029/1999JD901133→\\rightarrowStatus:VALID DOI•Real Title:Simulations of water vapor in the lower stratosphere and upper troposphere•Real Authors:Andrew Gettelman, James R\. Holton, Anne R\. Douglass•Real Year:2000•Journal/Source:Journal of Geophysical Research: Atmospheres

Model EvaluationsOurs:\[Hallucinated\]Analysis:The DOI 10\.1029/1999JD901133 is valid and corresponds to a real publication, but the title and authors do not match the paper cited in the ’Tool Output’\. The actual paper is titled ’Simulations of water vapor…’ by Andrew Gettelman et al\., which is unrelated to the Southern Annular Mode\. The ’Tool Output’ incorrectly attributes a paper on SAM to this DOI\.Qwen3\-VL\-32B:\[Authentic\]Analysis:The DOI resolves to a real publication… The title and year match exactly\. The paper is well\-known\.Gemini\-Flash:\[Authentic\]Analysis:The paper by Thompson and Wallace \(2000\) is a real and influential publication\. The DOI correctly points to this title\.