MedCalc-Pro: Solving Complex Medical Calculations with LLM Agents

# MedCalc-Pro: Solving Complex Medical Calculations with LLM Agents
Source: [https://arxiv.org/html/2607.02879](https://arxiv.org/html/2607.02879)
Siran Zhao♢\\diamondsuit,Ruihui Hou♢\\diamondsuit11footnotemark:1,Ziyue Huai♢\\diamondsuit,Chennuo Zhang♢\\diamondsuit, Tong Ruan♢\\diamondsuit ♢\\diamondsuitEast China University of Science and Technology, Shanghai, China,

###### Abstract

Current benchmarks for evaluating large language models \(LLMs\) in medical calculation are largely based on simplified settings, where each patient case corresponds to a single calculator and the required tool is explicitly specified in the query\. However, real clinical scenarios often require multiple calculators for joint evaluation, nested\-scale calculation, and fuzzy queries that do not directly specify the target calculator\. To this end, we propose a new medical calculation benchmark, MedCalc\-Pro, which covers three progressively challenging task settings: single\-calculator, multi\-calculator, and nested\-calculator calculation settings\. MedCalc\-Pro contains 2,268 real\-world clinical cases, covering 77 medical calculators across 14 clinical departments\. Meanwhile, to address the limited performance of existing frameworks and methods in complex clinical scenarios, we further propose a more generalizable agent framework that supports multi\-tool selection and nested\-tool calling, while suppressing parameter error propagation through structured validation and evidence review\. We conduct systematic comparisons across open\-source, closed\-source, and medical\-specialized LLMs, and the results show that our framework achieves the best performance across all three task settings\. This work provides a new benchmark and method for evaluating and applying LLMs in challenging medical calculation scenarios\.

MedCalc\-Pro: Solving Complex Medical Calculations with LLM Agents

Siran Zhao♢\\diamondsuit††thanks:Equal Contribution\., Ruihui Hou♢\\diamondsuit11footnotemark:1, Ziyue Huai♢\\diamondsuit, Chennuo Zhang♢\\diamondsuit,Tong Ruan♢\\diamondsuit††thanks:Corresponding authors♢\\diamondsuitEast China University of Science and Technology, Shanghai, China,

## 1Introduction

![Refer to caption](https://arxiv.org/html/2607.02879v1/x1.png)Figure 1:Illustrative examples of three medical calculation scenarios: single\-calculator, multi\-calculator, and nested\-calculator scenarios\.Large language models \(LLMs\) have recently achieved remarkable progress in information understanding\(Minaeeet al\.,[2024](https://arxiv.org/html/2607.02879#bib.bib29)\)and complex reasoning\(Haoet al\.,[2024](https://arxiv.org/html/2607.02879#bib.bib28)\)\. However, they still exhibit clear limitations on certain tasks that precise execution by external tools, such as numerical computation and rule\-based clinical score calculation\(Huanget al\.,[2024](https://arxiv.org/html/2607.02879#bib.bib27); Chen,[2020](https://arxiv.org/html/2607.02879#bib.bib31)\)\. This issue is particularly salient in the medical domain\. In clinical practice, medical calculators are widely used for risk assessment and prognosis prediction\(Moonset al\.,[2015](https://arxiv.org/html/2607.02879#bib.bib9)\)\. Consequently, there has been growing interest in integrating LLMs with medical calculators\.

However, existing medical calculation benchmarks, including MedCalc\-Bench\(Khandekaret al\.,[2024](https://arxiv.org/html/2607.02879#bib.bib26)\), CMedCalc\-Bench\(Zhanget al\.,[2025](https://arxiv.org/html/2607.02879#bib.bib35)\), OpenMedCalc\(Goodellet al\.,[2023](https://arxiv.org/html/2607.02879#bib.bib25)\), CalcQA\(Zhuet al\.,[2025](https://arxiv.org/html/2607.02879#bib.bib32)\), and AgentMD\(Jinet al\.,[2025](https://arxiv.org/html/2607.02879#bib.bib2)\), remain largely restricted to simplified settings\. These benchmarks typically pair an explicit target\-calculator query with a clinical case description, with each case corresponding to only one calculator\. Such settings are insufficient for capturing more realistic medical calculation scenarios\. In practice, multiple calculators often need to be used jointly, and some complex calculators further depend on sub\-calculator outputs, forming nested tool\-calling chains\.

Table 1:Comparison of medical calculator benchmarks\. “Fuzzy Query” indicates whether the benchmark includes goal\-driven queries that do not explicitly specify the target calculator\. “Proc\. Eval\.” denotes whether process\-level evaluation is provided\. “Multi\-calculator” indicates whether cases may require multiple calculators, and “Nested\-calculator” indicates whether cases involve dependencies between calculators\.To address these limitations, we propose a new medical calculation benchmark,MedCalc\-Pro, for systematically evaluating LLMs in complex medical calculation scenarios\. The benchmark covers three progressively challenging task settings: single\-calculator, multi\-calculator, and nested\-calculator calculation\. Specifically, single\-calculator task corresponds to the basic setting where a single patient case requires one calculator; multi\-calculator task corresponds to tasks where a single case requires multiple calculators for joint evaluation; and nested\-calculator task further requires the model to identify dependencies among calculators\. Figure[1](https://arxiv.org/html/2607.02879#S1.F1)illustrates these three task settings\. MedCalc\-Pro contains 2,268 real\-world clinical cases, covering 77 medical calculators across 14 clinical departments\. Compared with existing benchmarks, MedCalc\-Pro not only covers task settings that are closer to real clinical usage but also places greater emphasis on clinically goal\-driven queries, thereby avoiding direct exposure of the tool name to the model and imposing stricter requirements on query understanding, tool selection, parameter extraction, and multi\-step execution\. Table[1](https://arxiv.org/html/2607.02879#S1.T1)summarizes the key differences between existing medical calculator benchmarks and MedCalc\-Pro\.

Meanwhile, to address the limited generalization of existing methods and frameworks in complex settings, we propose a more generalizable medical calculator agent framework\. The framework follows realistic medical calculation workflows and consists of four stages: Query Rewriting, Retrieval and Reranking, Tool Selection, and Tool Execution\. Query Rewriting enhances clinical\-intent representation through multi\-dimensional rewriting of the original query\. Retrieval and Reranking identifies relevant candidate calculators from the tool repository\. Tool Selection enables the model to choose the appropriate calculators for the clinical task\. Finally, Tool Execution performs dependency\-aware nested\-calculator invocation, while structured validation and evidence auditing are introduced to suppress error propagation and improve the robustness of the final calculation\.

Our main contributions are as follows:

- •We construct a new medical calculation benchmark, MedCalc\-Pro, to enable more realistic evaluation of LLMs across single\-calculator, multi\-calculator, and nested\-calculator scenarios\.
- •We propose a generalizable agent framework for complex medical calculation, supporting multi\-calculator selection and nested\-calculator execution through query rewriting, retrieval and reranking, tool selection, and dependency\-aware tool execution with structured validation and evidence review\.
- •We compare our method with representative frameworks across open\-source, closed\-source, and medical\-specialized models\. Results show that it achieves the best performance across all three task settings and demonstrates strong robustness and generalization in complex scenarios\.

## 2Related Works

### 2\.1Medical Calculator Benchmarks

In recent years, as LLMs have been applied to an increasingly wide range of real\-world tasks\(Morriset al\.,[2023a](https://arxiv.org/html/2607.02879#bib.bib5); Noriet al\.,[2023](https://arxiv.org/html/2607.02879#bib.bib33)\), their performance in domain\-specific applications has attracted growing attention\(Singhalet al\.,[2023](https://arxiv.org/html/2607.02879#bib.bib8)\)\. In the medical domain, many early datasets, such as MedQA\(Jinet al\.,[2021](https://arxiv.org/html/2607.02879#bib.bib3)\), PubMedQA\(Jinet al\.,[2019](https://arxiv.org/html/2607.02879#bib.bib24)\), MedMCQA\(Palet al\.,[2022](https://arxiv.org/html/2607.02879#bib.bib22)\), and MMLU medical subjects\(Hendryckset al\.,[2021](https://arxiv.org/html/2607.02879#bib.bib21)\), primarily adopt multiple\-choice and open\-ended formats, which assess whether LLMs can recall and reason over medical knowledge\. Furthermore, several recent studies have proposed benchmarks specifically targeting medical calculation tasks such as CMedCalc\-Bench, CalcQA, and MedCalc\-Bench\. Despite these efforts, existing benchmarks typically assume a single\-calculator setting, and the task description often explicitly specifies which calculator should be used\. In real\-world clinical scenarios, however, a single patient case often requires evaluation using multiple medical calculators, and clinicians may not explicitly indicate which calculators should be applied\. To better reflect realistic clinical workflows, we constructed the MedCalc\-Pro that supports three types of settings: single\-calculator, multi\-calculator, and nested\-calculator calculation\.

### 2\.2LLM\-based Methods for Medical Calculator Execution

Relying solely on the model for the entire computation process poses a significant limitation\. The model’s numerical reasoning and calculation performance is often subpar\(Morriset al\.,[2023b](https://arxiv.org/html/2607.02879#bib.bib7)\), reducing transparency and making intermediate steps difficult to verify, which is particularly critical in medical scenarios\. To improve reliability and transparency, more recent studies have explored integrating LLMs with external tools\(Schicket al\.,[2023b](https://arxiv.org/html/2607.02879#bib.bib6)\), such as code interpreters\(Gaoet al\.,[2023](https://arxiv.org/html/2607.02879#bib.bib16); Chenet al\.,[2023](https://arxiv.org/html/2607.02879#bib.bib17)\)and external APIs\(Schicket al\.,[2023a](https://arxiv.org/html/2607.02879#bib.bib15)\)\. This approach has also been widely adopted in medical calculation tasks, such as MeNTi\(Zhuet al\.,[2025](https://arxiv.org/html/2607.02879#bib.bib32)\), and AgentMD enable LLMs to interact with structured medical calculators, allowing deterministic execution of clinical scoring programs\. However, these frameworks also typically assume that each patient case corresponds to a single\-calculator calculation, limiting their applicability in realistic clinical scenarios\. These challenges motivate our framework design, which supports flexible multi\-calculator and nested\-calculator execution through a structured agent workflow\.

## 3Methodology

### 3\.1Task Definition

Given a patient’s electronic medical record \(EMR\) and a queryQQ, the task aims to select appropriate medical calculators from a set of candidates and compute the corresponding answer based on the patient’s clinical information\. As illustrated in Figure[1](https://arxiv.org/html/2607.02879#S1.F1), when the query is “Which tools should be used to assess the patient’s stroke risk and bleeding risk to guide anticoagulation therapy?”, the model first selects the relevant calculators, CHA2DS2\-VASc Score and HAS\-BLED Score\. It then applies the scoring rules of these calculators using the patient’s clinical information and finally produces the corresponding scores of 6 and 3\.

![Refer to caption](https://arxiv.org/html/2607.02879v1/x2.png)Figure 2:Overview of our framework\. This agent contains four stages: Query Rewriting, Retrieval and Reranking, Tool Selection, and Tool Execution\.
### 3\.2Benchmark Creation

To evaluate the capability of LLMs in medical calculation tasks, we construct MedCalc\-Pro, a benchmark covering multiple evaluation scenarios\. The benchmark includes three types of calculation tasks: single\-calculator, multi\-calculator, and nested\-calculator\.

For thesingle\-calculatortask, we use MedRaC as the raw dataset\. It contains 1,048 patient cases covering 55 medical calculators, including 20 rule\-based calculators and 35 numerical calculators\.

For themulti\-calculatortask, we first construct cases from MedCalc\-Bench by grouping entries with the same patient case and merging only their ground\-truth calculator annotations, yielding 1,066 multi\-calculator candidates\. However, this dataset is highly imbalanced, with rule\-based calculators appearing much less frequently than numerical calculators\. To improve rule\-based calculator coverage, we further construct cases from PMC\-Patients\(Zhaoet al\.,[2022](https://arxiv.org/html/2607.02879#bib.bib30)\)\. Specifically, we use regular expression matching to identify clinical records that explicitly mention multiple calculator names and scores from 180,142 cases, obtaining 3,517 candidate descriptions\. After filtering for cases with sufficient clinical information to compute all required calculators, we obtain 92 additional multi\-calculator cases covering 33 rule\-based calculators\. For all multi\-calculator cases, we discard the original queries and use GPT\-5\.1 to generate fuzzy, goal\-oriented queries from the corresponding clinical texts\. This design better simulates realistic medical calculation requests, where physicians typically describe clinical goals rather than explicitly naming all target calculators\. Detailed prompts are provided in Appendix[B](https://arxiv.org/html/2607.02879#A2)\.

For thenested\-calculatortask, we also construct data from PMC\-Patients using regular expression matching to identify clinical records that mention calculators with dependency relationships\. In addition to the parent calculator names, we require the presence of sub\-calculator names together with their corresponding scores and supporting clinical descriptions, ensuring that both parent and sub\-calculators can be derived from the clinical text\. After filtering for cases with sufficient information to support the full dependency chain, we obtain 62 nested\-calculator cases\.

Quality Control\.We apply a three\-stage quality\-control process to ensure the reliability and realism of the constructed benchmark\. First, to prevent answer leakage, we remove explicit mentions of calculator names and final scores from the clinical texts when such information directly reveals the target calculators or expected outputs\. Second, three trained graduate students review all cases to verify consistency among clinical descriptions, required parameter values, and expected outputs, while ensuring that the queries are faithful to the corresponding clinical texts and clinically plausible\. Third, a clinician audits 20% of the cases for clinical correctness and query realism\. Unqualified cases are returned to the student reviewers with explanations for revision, and the review process is repeated iteratively until overall annotation accuracy exceeds 95%\.

Toolkit Construction\.We build the toolkit in two stages\. First, we reuse the validated computational logic of 46 calculators from MedCalc\-Bench and reorganize them under a unified configuration schema, standardizing parameter definitions and supplementing descriptions from authoritative clinical references such as the MSD Manuals111[https://www\.msdmanuals\.com/](https://www.msdmanuals.com/)\. Second, we implement 31 additional calculators to expand coverage in underrepresented areas, including neurological assessment \(e\.g\., NIHSS\), functional evaluation \(e\.g\., ECOG\), and psychiatric screening \(e\.g\., PHQ\-9\)\. All calculators serve as callable tools within the agent framework\.

### 3\.3Agent Method

To address the limitations of existing approaches in complex clinical scenarios, we propose an agent framework composed of four core modules: query rewriting, retrieval and reranking, tool selection, and tool execution\. Detailed prompts are provided in Appendix[B](https://arxiv.org/html/2607.02879#A2)and a complete execution trace is provided in Appendix[C](https://arxiv.org/html/2607.02879#A3)\.

Query Rewriting\.In real clinical practice, physicians’ queries are often expressed in terms of task goals, such as“assessing stroke risk”or“evaluating overall disease severity”\. Directly matching such queries with calculator descriptions often leads to inaccurate retrieval results\. To better capture the underlying clinical intent, this module rewrites the original query along three medically meaningful dimensions: clinical context, key parameters, and core functionality\. This multi\-dimensional rewriting helps the system better align the query with relevant medical calculators and more accurately recover the physician’s intended task\.

Retrieval and Reranking\.For each rewritten query, we perform dense retrieval\(Zhuanget al\.,[2023](https://arxiv.org/html/2607.02879#bib.bib18); Zhouet al\.,[2023](https://arxiv.org/html/2607.02879#bib.bib19)\)against the corresponding calculator description fields in the toolkit\. Each query independently retrieves top\-KKcandidate calculators\. Candidates retrieved from multiple query routes are then combined and reranked using Reciprocal Rank Fusion \(RRF\)\(Cormacket al\.,[2009](https://arxiv.org/html/2607.02879#bib.bib20)\), yielding a final candidate set ofNNcalculators\. The calculation formula is:

RRF​\(d\)=∑i=141k\+ri​\(d\)\\mathrm\{RRF\}\(d\)=\\sum\_\{i=1\}^\{4\}\\frac\{1\}\{k\+r\_\{i\}\(d\)\}\(1\)
Tool Selection\.In multi\-calculator scenarios, the number of applicable calculators for a given clinical case is unknown\. Therefore, directly selecting the top\-1 candidate based on similarity is insufficient\. At this stage, the model is provided with the detailed descriptions and parameter requirements of all candidate calculators, enabling more fine\-grained reasoning to determine which calculators should be executed\.

Tool Execution\.Even after appropriate calculators are selected, successful execution is not guaranteed\. Some complex calculators depend on sub\-calculator outputs for computation\. Therefore, the system first identifies dependencies, determining whether the target calculator requires inputs that cannot be directly extracted from the patient case but must be obtained by executing other calculators\. If dependencies exist, relevant sub\-calculators are executed first, and their outputs are used as inputs for the parent calculator\.

After determining the execution order, the extracted parameters are validated through a pipeline: Completeness Check→\\rightarrowType Check→\\rightarrowEnum Check→\\rightarrowUnit Check\. The system then performs an evidence audit to ensure consistency with the original clinical text and medical logic\.

- •Completeness Check: whether all parameters required for the calculation are present in the returned results;
- •Type Check: whether the extracted values match the expected parameter types;
- •Enum Check: whether the values of enumerated variables belong to the allowed options;
- •Unit Check: whether the units of numerical values are consistent with calculator requirements\.

Overall, if an error is detected at any stage, the system returns the error information and backtracks to the parameter extraction stage for correction\. Only after all checks are satisfied will the corresponding calculators be executed to produce the final result\. This mechanism helps prevent error propagation caused by incorrect parameters during nested calculator execution\.

## 4Experiment

In this section, we first conduct extensive experiments to evaluate the effectiveness of our framework\. Next, we provide a detailed analysis to offer deeper insights into our framework\.

### 4\.1Evaluation Setup

Baselines\.To evaluate the effectiveness of our framework, we compare it with three representative agent methods: \(1\) MeNTi\(Zhuet al\.,[2025](https://arxiv.org/html/2607.02879#bib.bib32)\), which uses a meta\-tool mechanism for tool selection and invocation; \(2\) MedRaC\(Wanget al\.,[2025](https://arxiv.org/html/2607.02879#bib.bib23)\), which integrates retrieval\-augmented generation with LLM\-based code generation for medical calculation; and \(3\) ReAct\(Yaoet al\.,[2022](https://arxiv.org/html/2607.02879#bib.bib14)\), a reasoning\-and\-acting framework that performs iterative step\-by\-step tool use\. For each method, we evaluate six LLMs to examine the interaction between model capability and framework design, including four open\-source general LLMs \(Qwen3\-235B\-A22B\(Yanget al\.,[2025](https://arxiv.org/html/2607.02879#bib.bib11)\), gpt\-oss\-120b\(Agarwalet al\.,[2025](https://arxiv.org/html/2607.02879#bib.bib4)\), Llama\-3\.3\-70B\-Instruct\(Grattafioriet al\.,[2024](https://arxiv.org/html/2607.02879#bib.bib34)\), and DeepSeek\-V3\.1\(Liuet al\.,[2024](https://arxiv.org/html/2607.02879#bib.bib13)\)\), one closed\-source LLM \(GPT\-5\-mini\), and one medical\-specialized LLM \(MedResearcher\-R1\-32B\(Yuet al\.,[2025](https://arxiv.org/html/2607.02879#bib.bib12)\)\)\.

Metrics\.We evaluate systems across three aspects: tool selection, parameter extraction, and final score computation\. We use R\-F1 to measure the model’s ability to select correct calculators, Extraction Accuracy to evaluate whether required parameters are correctly extracted, and Score Accuracy to assess whether each selected calculator’s final score is correct\. For nested\-calculator tasks, we further introduce two dependency\-related metrics: Dependency Detection Accuracy, which measures whether the model correctly identifies dependencies between the target calculator and its sub\-calculators, and Sub\-calculator Score Accuracy, which evaluates whether intermediate results from dependent sub\-calculator tools are correct\.

Implementations\.Our system is deployed through an OpenAI\-compatible API interface\. For closed\-source GPT model, we use the default settings\. For all other open\-source models, we set the temperature to 0\.01 to encourage stable and deterministic outputs\. In the semantic retrieval stage, we use m3e\-base\(Chenet al\.,[2024](https://arxiv.org/html/2607.02879#bib.bib10)\)as the embedding model and perform vector retrieval over the calculator toolkit using normalized cosine similarity on GPU\. For each retrieval route, we retrievekroute=16k\_\{\\text\{route\}\}=16candidate tools\. After reranking, the final number of candidates provided to the LLM selector is set toKcand=32K\_\{\\text\{cand\}\}=32\. During tool selection, the LLM is free to determine how many tools to select from the candidate list, and we do not impose an additional upper bound on the number of selected tools\. All experiments are conducted on a cluster with four NVIDIA A800 80GB GPUs\.

Table 2:Main results on the single\-calculator and multi\-calculator tasks \(%\)\. The best and second best results are highlighted inboldandunderline\.Table 3:Main results on the nested\-calculator tasks\. “Dep Detect” denotes Dependency Detection Accuracy, “Overall” denotes Parent Calculator Score Accuracy, and “Sub Overall” denotes Sub\-calculator Score Accuracy\.
### 4\.2Main Results

To evaluate the effectiveness of the proposed model, we conduct systematic comparisons with all baseline methods\. The experimental results are presented in Table[2](https://arxiv.org/html/2607.02879#S4.T2)and Table[3](https://arxiv.org/html/2607.02879#S4.T3)\.

From the table, we conclude that: 1\)Single\-Calculator task performance\. On this task, although the query explicitly specifies the target calculator, traditional methods still struggle to reliably complete tool invocation and computation\. Our method improves Tool Selection R\-F1 by 13\.05% and Score Accuracy by 4\.77% over the strongest baseline, MedRaC\. These results indicate that existing agent frameworks lack stability even in relatively simple medical calculation scenarios\. 2\)Multi\-Calculator task performance\. In this task, the performance gap further increases as task complexity grows\. This setting requires selecting multiple relevant calculators for a single\-patient case\. Compared with the strongest baseline, ReAct, our method improves parameter extraction accuracy by 19\.07% and Score Accuracy by 21\.96%, suggesting that in multi\-calculator scenarios, the primary bottleneck shifts from tool selection to parameter extraction and execution stability\. 3\)Nested\-Calculator task performance\. For the nested\-calculator task, method differences become more pronounced\. This setting requires identifying calculator dependencies and propagating intermediate results correctly\. Our method improves Sub Overall by 32\.27% and Parent Overall by 14\.51% over the strongest baseline, MeNTi\. The smaller parent\-level gain suggests sub\-calculator execution gains only partly translate into final parent\-level outputs, as residual intermediate\-result errors can propagate through the dependency chain and interact with parent\-calculator execution\. 4\)Failure analysis of MedRaC\. The performance collapse of MedRaC on the multi\-calculator and nested\-calculator tasks stems from a common limitation\. Unlike the single setting, these tasks formulate queries in terms of clinical goals rather than explicit calculator names\. Under this condition, the RAG\-based retrieval mechanism fails to recover the corresponding formulas, preventing the execution pipeline from being established\. This issue is particularly evident in the nested setting, where MedRaC fails to retrieve any valid tools, causing all metrics to drop to 0\.00%\.

### 4\.3Ablation Study

To analyze the contribution of each core component, we conduct ablation experiments by removing three key modules: \(1\) Query Rewriting, where multi\-dimensional rewriting is disabled and retrieval is performed using the original query; \(2\) Retrieval and Reranking, where candidate retrieval is removed and the model selects tools directly from the full calculator set; and \(3\) Variant\-1 in the tool execution module, where the structural validation and evidence review stages after parameter extraction are removed\. We use gpt\-oss\-120b as the base model in this experiment and all subsequent analysis experiments\. The results are presented in Table[4](https://arxiv.org/html/2607.02879#S4.T4)and Table[5](https://arxiv.org/html/2607.02879#S4.T5)\.

Table 4:Ablation results on the single\-calculator and multi\-calculator tasks\.Table 5:Ablation results on the nested\-calculator task\.The results show that: 1\) Retrieval is critical for tool selection\.Removing retrieval causes the largest drop in the single\-calculator task: R\-F1, Extract, and Score decrease by 19\.96%, 17\.61%, and 12\.40%, respectively\. This indicates that even with the target calculator explicitly specified, reliable candidate filtering remains essential for stable tool selection in a large tool space\. 2\) Variant\-1 is crucial for multi\-step execution\. Removing Variant\-1 most severely affects nested\-calculator tasks, where Extract decreases by 19\.70%, Parent Overall by 11\.3%, and Sub Overall by 6\.5%\. This suggests that Variant\-1 and auditing are essential to prevent parameter errors from propagating through dependency chains and affecting final results\. 3\) Query rewriting improves downstream reasoning\. Removing query rewriting mainly affects parameter extraction and final scoring rather than tool recovery\. In the multi\-calculator setting, R\-F1 drops by only 2\.95%, while Extract and Score decrease by 7\.20% and 9\.72%, indicating rewriting helps reconstruct task semantics from the original query and supports more reliable parameter identification and execution\.

### 4\.4Hyperparameter Analysis

The retrieval stage involves two key hyperparameters:KrouteK\_\{\\text\{route\}\}andKcandK\_\{\\text\{cand\}\}\.KrouteK\_\{\\text\{route\}\}denotes the retrieval depth of each route, namely the number of candidates recalled by each retrieval branch, whileKcandK\_\{\\text\{cand\}\}denotes the maximum size of the final candidate pool passed to the downstream Tool Selection stage\. We perform three independent grid searches on the multi\-calculator tasks withKcand∈\{8,16,32,64\}K\_\{\\text\{cand\}\}\\in\\\{8,16,32,64\\\}andKroute∈\{8,16,32,77\}K\_\{\\text\{route\}\}\\in\\\{8,16,32,77\\\}\.

Firstly, we determine the optimalKcandK\_\{\\text\{cand\}\}by comparing the average best F1 across runs \(Table[6](https://arxiv.org/html/2607.02879#S4.T6)\)\. Although F1 increases monotonically with larger candidate budgets, the marginal gain quickly diminishes: whenKcandK\_\{\\text\{cand\}\}increases from 32 to 64, the average F1 only improves from 64\.57% to 66\.73% \(\+1\.16%\)\. Meanwhile, the prompt length in the tool selection stage grows almost proportionally with the number of candidate tool descriptions, making the token consumption withKcand=64K\_\{\\text\{cand\}\}=64nearly twice that ofKcand=32K\_\{\\text\{cand\}\}=32\. Considering the minimal performance gain and substantially higher inference cost, we adoptKcand=32K\_\{\\text\{cand\}\}=32\.

WithKcandK\_\{\\text\{cand\}\}fixed, we further analyze the effect of the retrieval depthKrouteK\_\{\\text\{route\}\}\(Table[7](https://arxiv.org/html/2607.02879#S4.T7)\)\. The best performance occurs atKroute=16K\_\{\\text\{route\}\}=16, while larger values bring no additional benefit and slightly degrade performance\. Overall,Kcand=32K\_\{\\text\{cand\}\}=32andKroute=16K\_\{\\text\{route\}\}=16provide a good balance between candidate coverage and quality\.

Table 6:Validation F1 under differentKcandK\_\{\\text\{cand\}\}values\.Δ\\Deltaavg denotes the difference from theKcand=64K\_\{\\text\{cand\}\}=64setting\.†Near\-unconstrained candidate retention setting\.Table 7:Effect ofKrouteK\_\{\\text\{route\}\}whenKcand=32K\_\{\\text\{cand\}\}=32\.

## 5Conclusion

In this work, we introduce MedCalc\-Pro, a benchmark consisting of 2,268 cases and 77 calculators, incorporating three task types: single\-calculator, multi\-calculator, and nested\-calculator calculation, which better reflect real\-world clinical scenarios\. To address the challenges posed by these tasks, we propose a more generalized framework that integrates dependency identification and multi\-tool nested invocation\. Extensive experiments demonstrate that our framework outperforms existing approaches across all task types, achieving the best performance in each case\. We hope that our framework will help facilitate the application of LLMs in the medical field, advancing clinical decision support systems\.

## Limitations and Future Work

Although the proposed framework substantially improves LLM performance on medical calculator tasks through dependency identification, nested tool execution, and structured validation, several limitations still remain\. First, the data for nested\-calculator tasks is still relatively limited\. Although we intentionally expanded the coverage of rule\-based calculators and specifically constructed nested\-calculator cases, the number of nested cases remains much smaller than that of the other two settings\.

Second, the current benchmark is still mainly based on relatively clean clinical text and does not sufficiently capture the noise commonly encountered in real medical settings\. Examples include OCR recognition errors, non\-standard abbreviations, spelling mistakes, formatting irregularities, and cross\-source document concatenation, all of which are common in real electronic medical records and scanned documents\.

Third, the current benchmark mainly adopts a single\-turn query–response setting\. In real clinical scenarios, however, the input is often incomplete, and physicians or systems may need to actively ask follow\-up questions, conduct multi\-turn clarification, and confirm missing parameters before the calculation can be completed\. As a result, the present benchmark does not assess the model’s ability to proactively interact under incomplete information, nor does it capture performance in collaborative multi\-turn clinical calculation\.

Overall, these limitations suggest that there is still a gap between the current benchmark setting and realistic deployment conditions\. Future work may therefore focus on expanding nested cases, incorporating noisier clinical text, and moving from single\-turn evaluation toward more interactive multi\-turn clinical calculation settings\.

## Ethics Consideration

Because the medical domain is inherently high\-stakes, any errors in tool selection, parameter extraction, or calculator execution may lead to incorrect clinical assessments if such systems are used without appropriate oversight\. Therefore, our proposed benchmark and agent framework are intended for research purposes only and should not be used as a standalone basis for real\-world diagnosis, prognosis, or treatment decisions\.

Our benchmark is constructed from real\-world clinical cases collected from existing benchmark resources and PMC\-patient open\-access dataset\. The benchmark is intended for methodological evaluation rather than patient\-level decision support\. Although we do not intentionally include personally identifiable information, even de\-identified or publicly available clinical text may still raise privacy and data governance concerns\. We therefore emphasize that future extensions of this work should continue to follow applicable data protection standards and institutional review requirements\.

Another ethical concern is that improved medical calculator agents may be over\-interpreted as reliable clinical assistants\. Although our framework improves performance in complex settings, the results also show that substantial errors remain, especially in parameter extraction and nested execution scenarios\. Accordingly, these systems should be viewed as assistive research prototypes rather than clinically deployable tools, and human oversight by qualified medical professionals remains essential\.

## References

- S\. Agarwal, L\. Ahmad, J\. Ai, S\. Altman, A\. Applebaum, E\. Arbus, R\. K\. Arora, Y\. Bai, B\. Baker, H\. Bao,et al\.\(2025\)Gpt\-oss\-120b & gpt\-oss\-20b model card\.arXiv preprint arXiv:2508\.10925\.Cited by:[§4\.1](https://arxiv.org/html/2607.02879#S4.SS1.p1.1)\.
- J\. Chen, S\. Xiao, P\. Zhang, K\. Luo, D\. Lian, and Z\. Liu \(2024\)M3\-embedding: multi\-linguality, multi\-functionality, multi\-granularity text embeddings through self\-knowledge distillation\.InFindings of the association for computational linguistics: ACL 2024,pp\. 2318–2335\.Cited by:[§4\.1](https://arxiv.org/html/2607.02879#S4.SS1.p3.2)\.
- L\. Chen \(2020\)Overview of clinical prediction models\.Annals of translational medicine8\(4\),pp\. 71\.Cited by:[§1](https://arxiv.org/html/2607.02879#S1.p1.1)\.
- W\. Chen, X\. Ma, X\. Wang, and W\. W\. Cohen \(2023\)Program of thoughts prompting: disentangling computation from reasoning for numerical reasoning tasks\.Transactions on Machine Learning Research\.Note:External Links:ISSN 2835\-8856,[Link](https://openreview.net/forum?id=YfZ4ZPt8zd)Cited by:[§2\.2](https://arxiv.org/html/2607.02879#S2.SS2.p1.1)\.
- G\. V\. Cormack, C\. L\. A\. Clarke, and S\. Buettcher \(2009\)Reciprocal rank fusion outperforms condorcet and individual rank learning methods\.InProceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval,SIGIR ’09,New York, NY, USA,pp\. 758–759\.External Links:ISBN 9781605584836,[Link](https://doi.org/10.1145/1571941.1572114),[Document](https://dx.doi.org/10.1145/1571941.1572114)Cited by:[§3\.3](https://arxiv.org/html/2607.02879#S3.SS3.p3.2)\.
- L\. Gao, A\. Madaan, S\. Zhou, U\. Alon, P\. Liu, Y\. Yang, J\. Callan, and G\. Neubig \(2023\)Pal: program\-aided language models\.InInternational conference on machine learning,pp\. 10764–10799\.Cited by:[§2\.2](https://arxiv.org/html/2607.02879#S2.SS2.p1.1)\.
- A\. J\. Goodell, S\. N\. Chu, D\. Rouholiman, and L\. F\. Chu \(2023\)Augmentation of chatgpt with clinician\-informed tools improves performance on medical calculation tasks\.MedRxiv,pp\. 2023–12\.Cited by:[Table 1](https://arxiv.org/html/2607.02879#S1.T1.1.1.5.4.1),[§1](https://arxiv.org/html/2607.02879#S1.p2.1)\.
- A\. Grattafiori, A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Vaughan,et al\.\(2024\)The llama 3 herd of models\.arXiv preprint arXiv:2407\.21783\.Cited by:[§4\.1](https://arxiv.org/html/2607.02879#S4.SS1.p1.1)\.
- S\. Hao, S\. Sukhbaatar, D\. Su, X\. Li, Z\. Hu, J\. Weston, and Y\. Tian \(2024\)Training large language models to reason in a continuous latent space\.arXiv preprint arXiv:2412\.06769\.Cited by:[§1](https://arxiv.org/html/2607.02879#S1.p1.1)\.
- D\. Hendrycks, C\. Burns, S\. Kadavath, A\. Arora, S\. Basart, E\. Tang, D\. Song, and J\. Steinhardt \(2021\)Measuring mathematical problem solving with the MATH dataset\.InThirty\-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track \(Round 2\),External Links:[Link](https://openreview.net/forum?id=7Bywt2mQsCe)Cited by:[§2\.1](https://arxiv.org/html/2607.02879#S2.SS1.p1.1)\.
- Y\. Huang, J\. Shi, Y\. Li, C\. Fan, S\. Wu, Q\. Zhang, Y\. Liu, P\. Zhou, Y\. Wan, N\. Z\. Gong, and L\. Sun \(2024\)MetaTool benchmark for large language models: deciding whether to use tools and which to use\.InThe Twelfth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=R0c2qtalgG)Cited by:[§1](https://arxiv.org/html/2607.02879#S1.p1.1)\.
- D\. Jin, E\. Pan, N\. Oufattole, W\. Weng, H\. Fang, and P\. Szolovits \(2021\)What disease does this patient have? a large\-scale open domain question answering dataset from medical exams\.Applied Sciences11\(14\),pp\. 6421\.Cited by:[§2\.1](https://arxiv.org/html/2607.02879#S2.SS1.p1.1)\.
- Q\. Jin, B\. Dhingra, Z\. Liu, W\. Cohen, and X\. Lu \(2019\)PubMedQA: a dataset for biomedical research question answering\.InProceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing \(EMNLP\-IJCNLP\),K\. Inui, J\. Jiang, V\. Ng, and X\. Wan \(Eds\.\),Hong Kong, China,pp\. 2567–2577\.External Links:[Link](https://aclanthology.org/D19-1259/),[Document](https://dx.doi.org/10.18653/v1/D19-1259)Cited by:[§2\.1](https://arxiv.org/html/2607.02879#S2.SS1.p1.1)\.
- Q\. Jin, Z\. Wang, Y\. Yang, Q\. Zhu, D\. Wright, T\. Huang, N\. Khandekar, N\. Wan, X\. Ai, W\. J\. Wilbur,et al\.\(2025\)AgentMD: empowering language agents for risk prediction with large\-scale clinical tool learning\.Nature Communications16\(1\),pp\. 9377\.Cited by:[Table 1](https://arxiv.org/html/2607.02879#S1.T1.1.1.2.1.1),[§1](https://arxiv.org/html/2607.02879#S1.p2.1)\.
- N\. Khandekar, Q\. Jin, G\. Xiong, S\. Dunn, S\. Applebaum, Z\. Anwar, M\. Sarfo\-Gyamfi, C\. Safranek, A\. Anwar, A\. Zhang,et al\.\(2024\)Medcalc\-bench: evaluating large language models for medical calculations\.Advances in Neural Information Processing Systems37,pp\. 84730–84745\.Cited by:[Table 1](https://arxiv.org/html/2607.02879#S1.T1.1.1.6.5.1),[§1](https://arxiv.org/html/2607.02879#S1.p2.1)\.
- A\. Liu, B\. Feng, B\. Xue, B\. Wang, B\. Wu, C\. Lu, C\. Zhao, C\. Deng, C\. Zhang, C\. Ruan,et al\.\(2024\)Deepseek\-v3 technical report\.arXiv preprint arXiv:2412\.19437\.Cited by:[§4\.1](https://arxiv.org/html/2607.02879#S4.SS1.p1.1)\.
- S\. Minaee, T\. Mikolov, N\. Nikzad, M\. Chenaghlu, R\. Socher, X\. Amatriain, and J\. Gao \(2024\)Large language models: a survey\.arXiv preprint arXiv:2402\.06196\.Cited by:[§1](https://arxiv.org/html/2607.02879#S1.p1.1)\.
- K\. G\. Moons, D\. G\. Altman, J\. B\. Reitsma, J\. P\. Ioannidis, P\. Macaskill, E\. W\. Steyerberg, A\. J\. Vickers, D\. F\. Ransohoff, and G\. S\. Collins \(2015\)Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis \(tripod\): explanation and elaboration\.Annals of internal medicine162\(1\),pp\. W1–W73\.Cited by:[§1](https://arxiv.org/html/2607.02879#S1.p1.1)\.
- M\. R\. Morris, J\. Sohl\-Dickstein, N\. Fiedel, T\. Warkentin, A\. Dafoe, A\. Faust, C\. Farabet, and S\. Legg \(2023a\)Levels of agi for operationalizing progress on the path to agi\.arXiv preprint arXiv:2311\.02462\.Cited by:[§2\.1](https://arxiv.org/html/2607.02879#S2.SS1.p1.1)\.
- M\. R\. Morris, J\. Sohl\-Dickstein, N\. Fiedel, T\. Warkentin, A\. Dafoe, A\. Faust, C\. Farabet, and S\. Legg \(2023b\)Levels of agi for operationalizing progress on the path to agi\.arXiv preprint arXiv:2311\.02462\.Cited by:[§2\.2](https://arxiv.org/html/2607.02879#S2.SS2.p1.1)\.
- H\. Nori, N\. King, S\. M\. McKinney, D\. Carignan, and E\. Horvitz \(2023\)Capabilities of gpt\-4 on medical challenge problems\.arXiv preprint arXiv:2303\.13375\.Cited by:[§2\.1](https://arxiv.org/html/2607.02879#S2.SS1.p1.1)\.
- A\. Pal, L\. K\. Umapathi, and M\. Sankarasubbu \(2022\)Medmcqa: a large\-scale multi\-subject multi\-choice dataset for medical domain question answering\.InConference on health, inference, and learning,pp\. 248–260\.Cited by:[§2\.1](https://arxiv.org/html/2607.02879#S2.SS1.p1.1)\.
- T\. Schick, J\. Dwivedi\-Yu, R\. Dessì, R\. Raileanu, M\. Lomeli, E\. Hambro, L\. Zettlemoyer, N\. Cancedda, and T\. Scialom \(2023a\)Toolformer: language models can teach themselves to use tools\.Advances in neural information processing systems36,pp\. 68539–68551\.Cited by:[§2\.2](https://arxiv.org/html/2607.02879#S2.SS2.p1.1)\.
- T\. Schick, J\. Dwivedi\-Yu, R\. Dessì, R\. Raileanu, M\. Lomeli, E\. Hambro, L\. Zettlemoyer, N\. Cancedda, and T\. Scialom \(2023b\)Toolformer: language models can teach themselves to use tools\.Advances in neural information processing systems36,pp\. 68539–68551\.Cited by:[§2\.2](https://arxiv.org/html/2607.02879#S2.SS2.p1.1)\.
- K\. Singhal, S\. Azizi, T\. Tu, S\. S\. Mahdavi, J\. Wei, H\. W\. Chung, N\. Scales, A\. Tanwani, H\. Cole\-Lewis, S\. Pfohl,et al\.\(2023\)Large language models encode clinical knowledge\.Nature620\(7972\),pp\. 172–180\.Cited by:[§2\.1](https://arxiv.org/html/2607.02879#S2.SS1.p1.1)\.
- B\. Wang, I\. Xia, Y\. Zhang, J\. Wang, F\. Ouyang, S\. Han, A\. Cohan, H\. Yu, and Z\. Yao \(2025\)From scores to steps: diagnosing and improving LLM performance in evidence\-based medical calculations\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,C\. Christodoulopoulos, T\. Chakraborty, C\. Rose, and V\. Peng \(Eds\.\),Suzhou, China,pp\. 10809–10833\.External Links:[Link](https://aclanthology.org/2025.emnlp-main.548/),[Document](https://dx.doi.org/10.18653/v1/2025.emnlp-main.548),ISBN 979\-8\-89176\-332\-6Cited by:[Table 1](https://arxiv.org/html/2607.02879#S1.T1.1.1.3.2.1),[§4\.1](https://arxiv.org/html/2607.02879#S4.SS1.p1.1),[Table 2](https://arxiv.org/html/2607.02879#S4.T2.1.1.9.7.1.1),[Table 3](https://arxiv.org/html/2607.02879#S4.T3.1.1.8.7.1.1)\.
- A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv,et al\.\(2025\)Qwen3 technical report\.arXiv preprint arXiv:2505\.09388\.Cited by:[§4\.1](https://arxiv.org/html/2607.02879#S4.SS1.p1.1)\.
- S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. R\. Narasimhan, and Y\. Cao \(2022\)React: synergizing reasoning and acting in language models\.InThe eleventh international conference on learning representations,Cited by:[§4\.1](https://arxiv.org/html/2607.02879#S4.SS1.p1.1),[Table 2](https://arxiv.org/html/2607.02879#S4.T2.1.1.15.13.1.1),[Table 3](https://arxiv.org/html/2607.02879#S4.T3.1.1.14.13.1.1)\.
- A\. Yu, L\. Yao, J\. Liu, Z\. Chen, J\. Yin, Y\. Wang, X\. Liao, Z\. Ye, J\. Li, Y\. Yue,et al\.\(2025\)Medresearcher\-r1: expert\-level medical deep researcher via a knowledge\-informed trajectory synthesis framework\.arXiv preprint arXiv:2508\.14880\.Cited by:[§4\.1](https://arxiv.org/html/2607.02879#S4.SS1.p1.1)\.
- Y\. Zhang, Z\. Zhu, and X\. Wu \(2025\)CMedCalc\-bench: a fine\-grained benchmark for Chinese medical calculations in LLM\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,C\. Christodoulopoulos, T\. Chakraborty, C\. Rose, and V\. Peng \(Eds\.\),Suzhou, China,pp\. 25650–25659\.External Links:[Link](https://aclanthology.org/2025.emnlp-main.1302/),[Document](https://dx.doi.org/10.18653/v1/2025.emnlp-main.1302),ISBN 979\-8\-89176\-332\-6Cited by:[Table 1](https://arxiv.org/html/2607.02879#S1.T1.1.1.4.3.1),[§1](https://arxiv.org/html/2607.02879#S1.p2.1)\.
- Z\. Zhao, Q\. Jin, F\. Chen, T\. Peng, and S\. Yu \(2022\)PMC\-patients: a large\-scale dataset of patient summaries and relations for benchmarking retrieval\-based clinical decision support systems\.arXiv preprint arXiv:2202\.13876\.Cited by:[§3\.2](https://arxiv.org/html/2607.02879#S3.SS2.p3.1)\.
- S\. Zhou, U\. Alon, F\. F\. Xu, Z\. Jiang, and G\. Neubig \(2023\)DocPrompting: generating code by retrieving the docs\.InThe Eleventh International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=ZTCxT2t2Ru)Cited by:[§3\.3](https://arxiv.org/html/2607.02879#S3.SS3.p3.2)\.
- Y\. Zhu, S\. Wei, X\. Wang, K\. Xue, S\. Zhang, and X\. Zhang \(2025\)MeNTi: bridging medical calculator and LLM agent with nested tool calling\.InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),L\. Chiruzzo, A\. Ritter, and L\. Wang \(Eds\.\),Albuquerque, New Mexico,pp\. 5097–5116\.External Links:[Link](https://aclanthology.org/2025.naacl-long.263/),[Document](https://dx.doi.org/10.18653/v1/2025.naacl-long.263),ISBN 979\-8\-89176\-189\-6Cited by:[Table 1](https://arxiv.org/html/2607.02879#S1.T1.1.1.7.6.1),[§1](https://arxiv.org/html/2607.02879#S1.p2.1),[§2\.2](https://arxiv.org/html/2607.02879#S2.SS2.p1.1),[§4\.1](https://arxiv.org/html/2607.02879#S4.SS1.p1.1),[Table 2](https://arxiv.org/html/2607.02879#S4.T2.1.1.3.1.1.1),[Table 3](https://arxiv.org/html/2607.02879#S4.T3.1.1.2.1.1.1)\.
- Y\. Zhuang, Y\. Yu, K\. Wang, H\. Sun, and C\. Zhang \(2023\)ToolQA: a dataset for LLM question answering with external tools\.InThirty\-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track,External Links:[Link](https://openreview.net/forum?id=pV1xV2RK6I)Cited by:[§3\.3](https://arxiv.org/html/2607.02879#S3.SS3.p3.2)\.

## Appendix AAppendix

### A\.1Error Type Experiments

To further analyze the remaining failure sources of the full framework, we manually examined 50 error cases and categorized the errors into six types\. Table[8](https://arxiv.org/html/2607.02879#A1.T8)presents the distribution of the specific error types\. Two clear phenomena emerge from the analysis\. \(1\) Parameter\-related errors constitute the majority of the remaining failures\. In particular, Parameter Value Error and Enum Mapping Error account for a large proportion of the residual cases, indicating that parameter normalization remains a critical step in the pipeline\. This issue becomes particularly influential in nested scenarios, where incorrect parameters produced at the sub\-calculator stage may propagate along dependency chains and ultimately affect the final parent\-calculator calculation\. \(2\) Retrieval and selection errors mainly arise in multi\-tool scenarios, where a single clinical case may involve multiple relevant calculators\. Errors such as Retrieval Miss and Selection Error indicate that identifying the complete set of applicable tools from a single clinical case remains challenging due to the complexity of multi\-tool reasoning\.

Table 8:Error type distribution across three datasets\.Table 9:Tool\-level performance by difficulty\. Difficulty levels are defined by the number of required parameters: Easy \(≤\\leq3\), Medium \(4–7\), and Hard \(\>\>7\)\.
### A\.2Tool\-Level Analysis by Difficulty\.

To better understand how calculator complexity affects system performance, we conduct a tool\-level complexity analysis by grouping calculators according to the number of required input parameters\. The corresponding tool\-level results are reported in Table[9](https://arxiv.org/html/2607.02879#A1.T9)\. The results show a clear performance decline as tool complexity increases\. For Easy tools, Extract and Score reach 65\.22% and 68\.91%, respectively\. When the difficulty increases to Medium, these metrics drop by 17\.70% and 14\.93%, and further decrease by 28\.90% and 26\.10% for Hard tools relative to the Easy group\. This trend indicates that as the number of required parameters grows, the model faces increasing difficulty in accurate parameter extraction, and extraction errors are more likely to propagate through downstream computation, leading to lower final score accuracy\. These findings highlight tool complexity as a key factor affecting system performance and further emphasize the importance of structured validation and auditing mechanisms for stabilizing execution in complex tools\.

### A\.3Cost\-Efficiency Analysis

To further analyze the computational cost of different methods, we conduct a component\-level efficiency analysis\. Specifically, we randomly sample 100 cases from the single\-calculator dataset and 100 cases from the multi\-calculator dataset, and include all 62 cases from the nested\-calculator dataset\. For each case, we record the end\-to\-end running time as well as the total number of input and output tokens consumed during inference\. The averaged results are reported in Table[10](https://arxiv.org/html/2607.02879#A1.T10)\.

Table 10:Comparison of performance and inference cost across methods\. Avg Time is measured in seconds, Score Acc\. is reported in percentage, and Avg Input/Output Tokens denote the average number of input/output tokens\.

## Appendix BPrompt Design for Key Components

This section presents the detailed prompts used in several key components of our framework, including query rewriting, tool selection, dependency identification, and fuzzy query generation during data construction\. These prompts are provided to enhance transparency and facilitate reproducibility of our system\.

B\.1 Prompt: Query RewritingYou are assisting a retrieval system for medical scales and clinical calculators\.Task:Based on the \[Physician Query\] and \[Patient Case\], generate exactly 3 search phrases for semantic retrieval, each corresponding to one of three index dimensions:\- Core Function\- Clinical Context\- Key Input Variables\.Goal: The phrases must enable retrieval of the correct scale/calculator\.Input:Physician Query:\{INSERT\_QUERY\_HERE\}Patient Case:\{INSERT\_CASE\_HERE\}Index Dimensions:Core clinical function\(what clinical state, risk, severity, or outcome is assessed\)Target population and clinical context\(which patient group, care setting, or stage\)Key input variables\(core variables, risk factors, symptom items, or formula inputs typically required by this class of tool\)Mandatory Rules\[Master Principle: Intent Lock \(Focus Lock\)\]A\) First extract the single assessment target from the physician query \(e\.g\., pneumonia severity / AF stroke risk / VTE risk stratification / GFR estimation / upper GI bleeding triage / depression severity / pre\-operative cardiac risk / MAP calculation\)\.B\) All three phrases must center on this target\. Other diagnoses, tests, or scales in the case that do not change "which tool should be used" mustNOTbe included\.\[Forbidden\]1\) No scale, score, or rule names in any of the three phrases \(e\.g\., APACHE, Caprini, Wells, PERC, NIHSS, GCS, PHQ are all forbidden\)\.2\) Phrase 3 \(Key Variables\) mustNOTsimply list prominent variables from the case; list only variables that are strongly tied to the assessment target and typically used by this class of tool\.\[Task Type Templates — for guiding Phrase 3 content\]•T1 Risk Prediction/Stratification: Phrase 3: risk factors, comorbidities, exposures, surgical/activity status\.•T2 Severity/Prognosis Scoring: Phrase 3: vital signs \+ key lab values \+ mental status / organ function\.•T3 Symptom Scales: Phrase 3: symptom items, frequency, duration, functional impact\.•T4 Physiological/Lab Calculators: Phrase 3: formula input variables \(e\.g\., SBP/DBP, serum glucose/sodium, creatinine/age/sex, last menstrual period\)\.•T5 Diagnostic Probability/Exclusion Rules: Phrase 3: key discriminating features \+ clinical signs/lab results\.\[Phrase Generation Requirements\]OutputONLYa JSON array of exactly 3 strings, in the following fixed order:•\(1\) Core Function Query:
Describe only the assessment target \(risk/severity/probability/calculated quantity\)\.
Include common abbreviations/synonyms \(e\.g\., CAP/pneumonia; GFR/glomerular filtration rate; VTE/DVT/PE\), butNO scale names\.
Do not include patient background or specific variables\.•\(2\) Clinical Context Query:
Describe the population and care stage/setting \(e\.g\., emergency / ICU / pre\-operative / post\-operative / inpatient / outpatient; adult / pediatric / pregnant\)\.
No specific variables, no score values, no scale names\.
Include only context strongly relevant to tool selection\.•\(3\) Key Variables Query:
List 5\-10 variable types/risk factors/symptom items/formula inputs, comma\-separated\.
Prioritize variables that appear in the case AND are strongly relevant to the assessment target; standard tool variables may also be included\.
DoNOTlist variables or imaging findings irrelevant to the assessment target\.Output format:A single JSON array only, in the following format:```
["Core Function Query here", "Clinical Context Query here", "Key Variables Query here"]
```

B\.2 Prompt : Tool SelectionYou are a rigorous Clinical Decision Support System \(CDSS\) tool selector\.Task:Based on the physician’s query intent and patient record, select the most appropriate medical scale\(s\)/calculator\(s\) from the \[Candidate Tool List\]\.\[Candidate Tool List\]\(selection must come exclusively from this list; copy strings verbatim\)INSERT\_TOOL\_LIST\_HERE\[Physician Query\]INSERT\_QUERY\_HERE\[Patient Case\]INSERT\_CASE\_HEREINSERT\_MAX\_TOOLS\_CONSTRAINT\_HERESelection Rules \(mandatory\):1\.\[Highest Priority — Exact Name Match\]If the physician query explicitly names a specific tool or scale, you MUST identify the tool whose name exactly corresponds in the candidate list\. Do NOT substitute a similarly named but functionally different tool\.2\.\[Computability Check\]For each candidate tool, check whether its Required Inputs can be reliably supported by the patient case text \(including summary, lab results, physical exam, and explicitly negated findings\)\.3\.\[Tool Selection\]Output only one tool for single\-tool queries\. For multi\-tool queries, only select tools that are necessary and applicable\.4\.\[Nested Scale Detection\]If a selected tool depends on another tool’s output \(child tool\), declare the dependency in the "dependencies" field\.Output Requirements \(strict JSON; no additional text\):\- Output a single JSON object\. No Markdown, no explanatory prefix or suffix\.\- "selected\_tools" must be a string array \(may be empty\); contains only parent tools\.\- "dependencies" must be an object: \{ "parent\_tool\_id": \{ "param": "child\_tool\_id" \} \}; use \{\} if no dependencies\.\- "analysis" should be brief \(less than 80 words\); do not enumerate every candidate\.Output JSON structure:```
{
    "analysis": "brief rationale here...",
    "selected_tools": [
        "Tool ID A",
        "Tool ID B"
    ],
    "dependencies": {
        "Tool ID A": {
        "param_name_in_A":
        "sub_tool_id"
        }
    }
}
```

B\.3 Prompt: Dependency IdentificationYou are a strict clinical tool dependency checker\.Task:For each selected parent tool, examine its Required Inputs parameter list\. Determine whether any input parameter represents a composite score that must be COMPUTED by another medical calculator \(rather than directly extracted from the patient record\)\.\[Doctor Query\]INSERT\_QUERY\_HERE\[Patient Case\]INSERT\_CASE\_HERE\[Selected Parent Tools \(with their Required Inputs\)\]INSERT\_SELECTED\_TOOLS\_HERE\[Candidate Tool List\]INSERT\_TOOL\_LIST\_HERE\[Existing Dependencies \(may be empty\)\]INSERT\_EXISTING\_DEPENDENCIES\_HERERules:1\.Only output dependencies where parent tool is in Selected Parent Tools\.2\.Child tool does NOT need to be from Candidate Tools — it can be any medical scale/calculator that exists\.3\.Do not add child tools into selected tools; only express them in dependencies\.4\.For each selected parent tool, examine its Required Inputs\. If any input parameter represents a composite score or scale total , that parameter has a dependency on the corresponding sub\-tool\.5\.A parameter is a dependency if its value must be COMPUTED via another scoring system, not directly read from the patient record\. Raw vitals \(temperature, blood pressure, heart rate\) are NOT dependencies; scale totals \(GCS score, NIHSS score\) ARE dependencies\.6\.If no dependency is needed, return \{\}\.7\.Keep exact tool IDs and exact parameter names\.Output format \(strict JSON only\):```
{
  "analysis": "short reason",
  "dependencies": {
    "Parent Tool ID": {
      "parameter_name_in_parent": "child_tool_id"
    }
  }
}
```

B\.4 Prompt: Fuzzy Query GenerationYou are a medical benchmark dataset curator\.Task:Given a patient case and the ground\-truth medical scale\(s\) that should be applied, write one physician query that naturally leads a clinician to select the specified scale\(s\)\. DoNOTmention the scale name\(s\) or acronym\(s\) explicitly\.\[Patient Case\]\{INSERT\_CASE\_HERE\}\[Ground\-Truth Scale\(s\)\]\{INSERT\_TOOLS\_HERE\}\[Scale Description\(s\)\]\{INSERT\_TOOL\_DESC\_HERE\}\[Key Parameters the Scale Requires\]\{INSERT\_MEASUREMENTS\_HERE\}Requirements:1\.Clinical authenticity— The query must sound like a real clinical communication \(e\.g\., attending instruction, referral question, or bedside decision prompt\); avoid academic or test\-like phrasing\.2\.No scale name leakage— DoNOTmention the scale/tool name, acronym, or any sub\-score name\.3\.Answerability— The query must be answerable using the patient case and the ground\-truth scale\(s\); do not introduce information needs absent from the case\.4\.Language— Output in English\.Output format:A single plain string only, with no JSON wrapping and no extra text\.```
<query>Your physician query here.</query>
```

## Appendix CInference Example

This section presents a representative clinical case to illustrate the end\-to\-end execution process of the proposed framework\. The example demonstrates how the system handles a multi\-calculator scenario, including query rewriting, multi\-route retrieval, tool selection, and dependency\-aware execution, providing a detailed view of each stage in the pipeline\.

C\.Full Execution Trace of a multi\-calculator Case\#\# Doctor Query:For this patient with a history of atrial fibrillation, which tools should be used to assess stroke risk, the degree of neurological deficits, and functional recovery?\#\# 1\. Query Rewriting:The system rewrote the original query into three auxiliary retrieval queries:\(1\)stroke risk stratification, neurological deficit severity, and functional recovery assessment\(2\)adult patients with atrial fibrillation presenting with acute neurological symptoms in emergency or inpatient setting\(3\)age, hypertension, diabetes mellitus, prior stroke/TIA, heart failure, sex, systolic blood pressure, level of consciousness, motor strength, speech impairmentTogether with the original query, these formed four retrieval routes\.\#\# 2\. Multi\-Route Retrieval:For each route, the system retrieved a ranked list of candidate tools\.Route 1 top\-5: NIHSS, mRS Score, ECOG Performance Status, GCS, Trauma IndexRoute 2 top\-5: PHQ\-9, CURB\-65, NIHSS, ABCD2, MEWSRoute 3 top\-5: NYHA, Framingham Risk, CURB\-65, APACHE II Score, PHQ\-9Route 4 top\-5: NIHSS, ABCD2, GCS, mRS Score, Hunt–Hess ScaleAfter fusion and reranking, the final candidate set contained 32 tools, including NIHSS, mRS Score, CHA2DS2\-VASc, ABCD2, GCS, and Framingham Risk\.\#\# 3\. Tool Selection:The system finally selected the following tools:\- CHA2DS2\-VASc\- NIHSS\- mRS ScoreSelection Rationale:The query asks for stroke risk, neurological deficit severity, and functional recovery\. These correspond naturally to CHA2DS2\-VASc, NIHSS, and mRS, respectively\.\#\# 4\. Parameter Extraction:NIHSS \(initial extraction\):The first NIHSS extraction produced a complete parameter set, including normal consciousness, normal gaze, no visual loss, no motor drift, no sensory deficit, no aphasia, anddysarthria = severe\.mRS Score:The system extracted:\-mrs\_grade = 0Supporting evidence:During the outpatient neurology evaluation, the patient continued to be free of dysarthria, dysphagia, and the stigmata of TIA or CVA\.CHA2DS2\-VASc:The extracted parameters were:\- congestive heart failure = no\- hypertension = yes\- age = 73\- diabetes = no\- stroke/TIA history = no\- vascular disease = no\- sex category = female\#\# 5\. Final Calculator Execution:NIHSS:\- Final score: 1\- Label:Minor StrokemRS Score:\- Final score: 0\- Label:mRS Grade 0CHA2DS2\-VASc:\- Final score: 3\- Label:High Risk\- Recommendation:Oral anticoagulation \(OAC\) is recommended