Getting the Parameters Right: A Difficulty-Graded Benchmark and Probe-Guided Training for LLM Tool Calls

# Getting the Parameters Right: A Difficulty-Graded Benchmark and Probe-Guided Training for LLM Tool Calls
Source: [https://arxiv.org/html/2608.03071](https://arxiv.org/html/2608.03071)
Guoyao Yu, Xiaoqing Sun, Ziqi Huang, Shaojing Fan, Zhongyi Zhang, Xiaomeng Hu, Xiaobo Xue, Yangyang Shi, Xiong Xiao, Yang Song, Biao Lyu, Rong Wen, Xing Li, Qinming He, Shunming Zhu, Zhenguang Liu\\corresponding

###### Abstract

Large language model agents derive much of their capability from tool use\. Existing research on tool use has largely focused on selecting the right tool and orchestrating the order of calls\. However, correctly filling the parameters of a tool call is equally critical for successful execution and has received far less attention\. In domains such as cloud networking, even frontier models correctly complete fewer than half of tool calls\. Inspired by recent analyses showing that LLM hidden states encode rich information about model predictions, we discover that while the model generates a parameter value, its hidden state contains a strong correctness signal: a simple linear probe can accurately predict whether the value will be correct\. Based on this observation, we propose a unified probe\-guided framework with two complementary approaches: probe\-filtered bootstrapped training \(PBT\), which uses the probe to filter reliable self\-generated calls for fine\-tuning, and probe\-guided reranking \(PGR\), which uses the probe to select better candidates during inference\. To support systematic evaluation, we releaseParamBench, a benchmark built from real cloud\-network APIs that categorizes every instance into five difficulty levels according to parameter nesting depth, cross\-parameter dependencies, and the reasoning required to derive values from earlier calls\. Extensive experiments across 5 open models onParamBenchand 6 external benchmarks demonstrate that our method substantially improves parameter generation, raising the average exact match from 19\.7% to 59\.6%\.

## 1Introduction

Tool use is now the core capability of large language model \(LLM\) agents\. It is what connects an agent to the world outside its context window: the agent perceives its environment by reading what its API calls return, and acts by issuing further calls\. Making tool use work well is not simple\. Industry has built invocation standards such as the Model Context Protocol\(Anthropic[2024](https://arxiv.org/html/2608.03071#bib.bib1)\)and function calling\(OpenAI[2023](https://arxiv.org/html/2608.03071#bib.bib23)\); research has studied when to call a tool, which APIs to call, and in what order to call them\(Qin et al\.[2024](https://arxiv.org/html/2608.03071#bib.bib29); Patil et al\.[2024](https://arxiv.org/html/2608.03071#bib.bib26); Li et al\.[2023b](https://arxiv.org/html/2608.03071#bib.bib18); Qin et al\.[2025](https://arxiv.org/html/2608.03071#bib.bib28)\)\. However, in many real\-world scenarios that involve complex tool calls, getting the parameters of a call right is itself a hard problem, and it has received far less systematic study\.

![Refer to caption](https://arxiv.org/html/2608.03071v1/figs/teaser_v6.png)Figure 1:Tool Use Overview and Parameter GenerationFormally, we define this step as*parameter generation*: given an instruction, the API schemas, and the results of earlier calls, the model must fill every parameter of a tool call with a correct value\. There are three structural properties that make it hard:\(i\) deep nesting: parameters can be objects nested several layers deep;\(ii\) conditional dependency: whether a field is required or valid can depend on other fields;\(iii\) cross\-call derivation: some values must be derived from the results of earlier calls\. Figure[1](https://arxiv.org/html/2608.03071#S1.F1)illustrates the task and the three difficulties\. In our tests, Claude Opus 4\.7 fills only 33\.4% of the calls correctly on NESTFUL in the 0\-shot setting\.

Inspired by prior work showing that a model’s hidden states carry signals such as whether its own output is true\(Azaria and Mitchell[2023](https://arxiv.org/html/2608.03071#bib.bib2); Marks and Tegmark[2024](https://arxiv.org/html/2608.03071#bib.bib22); Du, Xiao, and Li[2024](https://arxiv.org/html/2608.03071#bib.bib6)\), we examine a similar signal at the parameter level and find that a linear probe on the hidden state predicts whether a parameter value will be correct, with an in\-domain AUC of 0\.986\. Building on this observation, we use the probe in two places: probe\-filtered bootstrapped training \(PBT\) on the training side and probe\-guided reranking \(PGR\) on the inference side\. PBT targets the common case where labeled answers are scarce and unlabeled instructions are plentiful\. A seed model is first fine\-tuned on the small labeled set, and then generates answers for the unlabeled instructions; the answers that pass the probe are added to the labeled set, and the base model is fine\-tuned on the enlarged set to produce the final model\. In PGR, the model samples several candidate calls at inference time, the probe scores them, and a selection strategy \(candidate\-level, field\-level, or field\-set\) picks the final call\. The two sides pay different costs: PBT changes the model once at training time, while PGR leaves the model unchanged and spends extra samples at inference time; either can be used alone, and they can also be combined\.

For evaluating parameter generation, existing benchmarks match the generated call against a reference call\(Patil et al\.[2025](https://arxiv.org/html/2608.03071#bib.bib25); Li et al\.[2023b](https://arxiv.org/html/2608.03071#bib.bib18); Wu et al\.[2024](https://arxiv.org/html/2608.03071#bib.bib37); Basu et al\.[2025](https://arxiv.org/html/2608.03071#bib.bib3)\)\. They do not classify where parameter values come from, and they do not grade how hard an instance is\. We therefore presentParamBench, built from real agent tool\-call execution traces in cloud networking\. Based on nesting depth, cross\-parameter dependency, and the reasoning required to derive values from earlier calls, we design a fixed rule that grades the parameter generation difficulty of every instance into five levels, L1 to L5\. This gives a fine\-grained view of how well a model fills tool\-call parameters\.

We evaluate PBT and PGR onParamBenchand 6 external tool\-use benchmarks, including BFCL and API\-Bank, over 5 open models such as Qwen3\-8B and Gemma\-4\-12B\. Averaged over models and datasets, plain supervised fine\-tuning raises exact match from 19\.7% to 51\.6%, and PBT raises it to 59\.6%\. On the two deepest\-nesting benchmarks, PGR adds a further 4\.6 points at inference time\. Moreover, compared with frontier models such as Claude Opus 4\.7 and GPT\-5\.4, a Qwen3\-8B model with PBT and PGR reaches the frontier level on all 7 datasets and is the best on 3 of them\.

Our main contributions are as follows\.

- C1\(New focus\)\.We put parameter generation at the center of tool\-use research, define it formally, and analyze its three difficulties: deep nesting, conditional dependency, and cross\-call derivation\.
- C2\(Internal\-signal supervision\)\.We demonstrate that hidden\-state correctness signals can provide effective supervision for structured tool\-call parameter generation\. Building on this observation, we propose probe\-filtered bootstrapped training \(PBT\) for selecting reliable pseudo\-labels during self\-training, and probe\-guided reranking \(PGR\) for selecting the best candidate at inference\.
- C3\(ParamBench Dataset\)\.For fine\-grained evaluation of parameter generation, we releaseParamBench, a benchmark built from real cloud\-network agent traces, and grade each tool\-call instance into five difficulty levels by the structural properties of its parameters\.
- C4\(Extensive evaluation\)\.We run large\-scale comparisons and ablations over 7 benchmarks and 5 base models, compare 4 selection signals and 4 leading tool\-use models, and show that PBT and PGR are the most effective strategies\.

## 2Related Work

#### Tool learning and function calling\.

Equipping language models with external tools has grown into a mature research area, commonly termed*tool learning*\(Qin et al\.[2025](https://arxiv.org/html/2608.03071#bib.bib28); Qu et al\.[2025](https://arxiv.org/html/2608.03071#bib.bib30)\), supported by widely adopted standards such as the Model Context Protocol\(Anthropic[2024](https://arxiv.org/html/2608.03071#bib.bib1)\)and OpenAI’s function\-calling interface\(OpenAI[2023](https://arxiv.org/html/2608.03071#bib.bib23)\)\. Research in this area has centered on three questions\. The first is*when*to call a tool: Toolformer learns where an API call should be inserted in the text\(Schick et al\.[2023](https://arxiv.org/html/2608.03071#bib.bib31)\), and MetaTool benchmarks the decision of whether to invoke a tool\(Huang et al\.[2024](https://arxiv.org/html/2608.03071#bib.bib12)\)\. The second is*which*tool to select: ToolLLM retrieves relevant APIs from a pool of more than 16,000 candidates\(Qin et al\.[2024](https://arxiv.org/html/2608.03071#bib.bib29)\), Gorilla couples retrieval with instruction tuning\(Patil et al\.[2024](https://arxiv.org/html/2608.03071#bib.bib26)\), and Re\-Invoke rewrites the query for zero\-shot retrieval\(Chen et al\.[2024](https://arxiv.org/html/2608.03071#bib.bib4)\)\. The third is in*what order*to compose several calls: ReAct alternates reasoning steps and tool actions\(Yao et al\.[2023](https://arxiv.org/html/2608.03071#bib.bib39)\), and LLMCompiler plans a graph of calls that can run in parallel\(Kim et al\.[2024](https://arxiv.org/html/2608.03071#bib.bib15)\)\. These works all treat one accurate call as a single unit, and parameter generation inside the call has not been studied as a separate problem\.

#### Function\-calling methods\.

In all current research, the parameter values that fill each call are left to free\-form generation\. Constrained decoding guarantees that the output parses, but it cannot tell which legal value is the right one\(Willard and Louf[2023](https://arxiv.org/html/2608.03071#bib.bib36); Dong et al\.[2025](https://arxiv.org/html/2608.03071#bib.bib5)\)\. Tool\-call SFT, on benchmark or large synthetic call data, covers the hard cases only where the training data happens to contain them\(Liu et al\.[2025](https://arxiv.org/html/2608.03071#bib.bib20); Qin et al\.[2024](https://arxiv.org/html/2608.03071#bib.bib29); Patil et al\.[2024](https://arxiv.org/html/2608.03071#bib.bib26); Zhang et al\.[2025](https://arxiv.org/html/2608.03071#bib.bib40)\)\. Few\-shot prompting covers only what the examples show\. ReAct\-style reasoning gets the value right only when the reasoning happens to reach it\(Yao et al\.[2023](https://arxiv.org/html/2608.03071#bib.bib39)\)\. To date, a method that specifically addresses the difficulties of parameter generation is still missing\.

#### Tool\-use benchmarks\.

On the evaluation side, existing benchmarks center on tool selection and multi\-step orchestration\(Li et al\.[2023b](https://arxiv.org/html/2608.03071#bib.bib18); Basu et al\.[2025](https://arxiv.org/html/2608.03071#bib.bib3)\)\. Where they do inspect arguments, the measure is coarse: BFCL folds argument correctness into a single call\-success score\(Patil et al\.[2025](https://arxiv.org/html/2608.03071#bib.bib25)\),τ\\tau\-bench reveals wrong parameters only through the final task outcome\(Yao et al\.[2025](https://arxiv.org/html/2608.03071#bib.bib38)\), and Seal\-Tools reports only an overall parameter match rate\(Wu et al\.[2024](https://arxiv.org/html/2608.03071#bib.bib37)\)\. NesTools is the closest prior effort to score nested\-parameter filling as a separate axis, and current models struggle on it\(Han et al\.[2025](https://arxiv.org/html/2608.03071#bib.bib7)\)\. Even so, no existing dataset scores parameter generation at the field level and grades every instance by difficulty\.

#### Internal signals of model correctness\.

A line of work shows that a model’s hidden states encode whether its own output is correct, often more reliably than its stated confidence or its token probabilities\. Early probing studies established this on factual statements: a simple probe on hidden activations tells whether a statement is true, and truth even appears as a linear direction in the representation space\(Azaria and Mitchell[2023](https://arxiv.org/html/2608.03071#bib.bib2); Marks and Tegmark[2024](https://arxiv.org/html/2608.03071#bib.bib22)\)\. Hidden states, embeddings, and gradients of a completed answer likewise detect hallucinated text\(Hu et al\.[2024](https://arxiv.org/html/2608.03071#bib.bib11); Orgad et al\.[2025](https://arxiv.org/html/2608.03071#bib.bib24)\), and follow\-up work reads the signal before or during generation, so an error can be predicted before the answer is complete\(Ji et al\.[2024](https://arxiv.org/html/2608.03071#bib.bib13); Kossen et al\.[2024](https://arxiv.org/html/2608.03071#bib.bib16); Snyder, Moisescu, and Zafar[2024](https://arxiv.org/html/2608.03071#bib.bib33)\)\. The signal can also be acted on: likely errors can be flagged\(Du, Xiao, and Li[2024](https://arxiv.org/html/2608.03071#bib.bib6)\), or generation can be guided toward a more truthful answer\(Li et al\.[2023a](https://arxiv.org/html/2608.03071#bib.bib17)\)\. In tool use, however, this signal is nearly untouched: the only attempt detects a bad call after it is fully generated, with one yes\-or\-no label for the whole call\(Healy et al\.[2026](https://arxiv.org/html/2608.03071#bib.bib9)\)\.

## 3Problem Formalization

In production tool\-use systems, selecting the appropriate API can often be addressed through retrieval or routing, whereas correctly instantiating structured API parameters remains a major challenge\. This section formalizes tool\-call parameter generation as a value\-filling task and identifies its three structural sources of difficulty: deep nesting, inter\-field conditional dependencies, and cross\-call value derivation\.

Given a target API schema𝒮\\mathcal\{S\}\(each parameter’s name, type, required flag, nesting structure, and field description\), a natural\-language instructionℐ\\mathcal\{I\}, and a data\-flow context𝒟=\{\(ti,oi\)\}\\mathcal\{D\}=\\\{\(t\_\{i\},o\_\{i\}\)\\\}of upstream tool calls and their outputs, the task is to produce a parameter instance𝐩\\mathbf\{p\}that is format\-compliant under𝒮\\mathcal\{S\}and value\-correct against a reference𝐩∗\\mathbf\{p\}^\{\\ast\}\. In practice, only value correctness is hard: a preliminary audit of 7 frontier models shows that only 2\.1% of their failed calls violate the schema, while the others are schema\-valid but value\-wrong\. The three challenges below are the main reasons the values go wrong\.

#### CH\-1: Deep Nesting\.

Enterprise API parameters are often not flat key\-value maps but nested objects and object lists\. A typical example is theFiltersparameter ofListTransitRouterRouteEntries:

```
Filters = [ {Key: DestinationCidrBlock,
             Value: [10.0.0.0/16]},
            {Key: Status, Value: [Active]},
            ... ]
```

a list of filter objects whoseValuefield is itself a list: an array inside an object inside an array\. A constrained decoder can keep this shape well\-formed, but the right literal must still be placed at the right depth\.

#### CH\-2: Inter\-Field Conditional Dependencies\.

Some parameters are required, or take constrained values, only when a sibling parameter takes a specific value:NextHopIdis required whenNextHopType="RouterInterface"; apeerInfosub\-object is required only whentype="VBR"\. The same schema therefore admits several valid shapes, selected by one controlling field\. In the public schemas of production APIs these rules are rarely written in machine\-readable form; they live in human\-readable field descriptions, and the model must infer which shape the current situation calls for\.

#### CH\-3: Cross\-Call Value Derivation\.

Many correct values are not in the instruction but in an earlier tool’s output: theVpcIdreturned byDescribeVpcsis an input toDescribeVSwitches\. The model must find the right field in an upstream response that is often dozens of fields wide and several layers deep, extract the value, and place it into the right downstream slot\. Picking the wrong field, or the wrong element of a list, is a frequent failure\.

## 4Probe\-Guided Training and Reranking

Inspired by a line of work that reads correctness signals from a model’s hidden states\(Azaria and Mitchell[2023](https://arxiv.org/html/2608.03071#bib.bib2); Marks and Tegmark[2024](https://arxiv.org/html/2608.03071#bib.bib22); Du, Xiao, and Li[2024](https://arxiv.org/html/2608.03071#bib.bib6)\), we build a linear probe: it reads the hidden state just before the model writes a parameter value, and predicts whether that value will be correct\. We then use this signal in two places\. On the training side, probe\-filtered bootstrapped training \(PBT\) uses it to produce more useful supervised fine\-tuning \(SFT\) data for the model\. On the inference side, probe\-guided reranking \(PGR\) uses it to rerank sampled parameter candidates\. Figure[2](https://arxiv.org/html/2608.03071#S4.F2)gives an overview\.

![Refer to caption](https://arxiv.org/html/2608.03071v1/figs/method_overview_v5.png)Figure 2:Overview of our framework\. Left: the base model is fine\-tuned with LoRA, which adapts only the attention layers\. Middle: a linear probe is trained on labeled data, using hidden states from roughly two\-thirds of the seed model’s depth\. Top right: the trained probe defines a candidate filter that converts unlabeled data into pseudo\-labeled data, enabling Probe\-Filtered Bootstrapped Training \(PBT\)\. Bottom right: at inference time, the same probe reranks sampled candidates to select the final call, yielding Probe\-Guided Reranking \(PGR\)\.### 4\.1The Probe: Correctness Before Emission

To obtain a reliable probe signal, we keep the model frozen throughout\. At the position just before the model writes a parameter value, we take the hidden state and feed it to a logistic regression classifier, which outputs the probability that the value will be correct\. We train one probe per layer and keep the layer that performs best, which sits at roughly two\-thirds of the network depth \(Figure[6](https://arxiv.org/html/2608.03071#A2.F6)in the appendix\)\.

Formally, lettjt\_\{j\}be the decision point of thejj\-th parameter, which is the position just before the model writes its value, and lethj\(ℓ\)h\_\{j\}^\{\(\\ell\)\}be the hidden state that layerℓ\\ellproduces attjt\_\{j\}\. The probe is a logistic regression over this vector:

rj=σ​\(w⊤​hj\(ℓ∗\)\+b\),r\_\{j\}\\;=\\;\\sigma\\\!\\left\(w^\{\\top\}h\_\{j\}^\{\(\\ell^\{\*\}\)\}\+b\\right\),\(1\)wherewwandbbare the learned weight vector and bias,σ\\sigmais the sigmoid function that turns the score into a probability,ℓ∗\\ell^\{\*\}is the best layer chosen as above, andrjr\_\{j\}is the predicted probability that parameterjjwill be correct\. To trainwwandbb, we take training\-side instructions that have gold answers, let the model sample candidate values, and label each parameteryj=1y\_\{j\}=1if its value matches the gold answer andyj=0y\_\{j\}=0otherwise;wwandbbare then fit by minimizing the standard binary cross\-entropy loss betweenrjr\_\{j\}andyjy\_\{j\}\.

A probe built this way reaches an in\-domain AUC of 0\.986, well above the baseline of token log\-probability \(0\.914\), a common signal for the confidence of model outputs\(Kadavath et al\.[2022](https://arxiv.org/html/2608.03071#bib.bib14)\)\. The signal is also robust: the probe signal survives the move to the next training checkpoint \(AUC 0\.982\), and across model sizes, before and after fine\-tuning, the AUC stays between 0\.93 and 0\.99, always above log\-probability\. One property to note is that the probe is specialized: it is trained for a given dataset, model, and sampling temperature, and its accuracy degrades outside that setting\. In our design, each domain therefore trains its own probe on its own training set, which keeps the probe in the setting it handles best\.

### 4\.2Probe\-Filtered Bootstrapped Training

In real tool\-use applications, instructions and gold answers come at very different costs\. Instructions are nearly free, since user requests accumulate in production logs on their own\. Gold answers are expensive, because each one must be written and checked, field by field, by an expert who knows the API\. Most domains therefore end up with a small labeled set, denotedℒ\\mathcal\{L\}, and a large pool of unlabeled instructions, denoted𝒰\\mathcal\{U\}\.

The natural way to use𝒰\\mathcal\{U\}is self\-training: let the model answer the unlabeled instructions and train on its own answers\. The difficulty lies in the fact that many of these answers are wrong, and they are hard to catch: there is no gold answer to compare against, and schema checks pass nearly all wrong calls\. The probe fills this gap: it predicts correctness without seeing the gold answer\.*Probe\-filtered bootstrapped training*\(PBT\) builds the judge into the self\-training loop and runs in the following five steps\.

1. 1\.Fine\-tune the model onℒ\\mathcal\{L\}with LoRA\(Hu et al\.[2022](https://arxiv.org/html/2608.03071#bib.bib10)\)\. The result is the*seed model*\.
2. 2\.Build the probe of Equation \([1](https://arxiv.org/html/2608.03071#S4.E1)\) for the seed model, onℒ\\mathcal\{L\}only: the seed model samples candidate values, each labeled against the gold answer, andwwandbbare fit\.
3. 3\.Run the seed model over𝒰\\mathcal\{U\}and collect several candidate calls for each instruction\.
4. 4\.Score every candidate with the probe\. The score of a candidate callccis the mean probe score over themmparameters it fills: s​\(c\)=1m​∑j=1mrj\.s\(c\)\\;=\\;\\frac\{1\}\{m\}\\sum\_\{j=1\}^\{m\}r\_\{j\}\.\(2\)For each instruction, keep the highest\-scoring candidate only if its score passes a thresholdτ\\tau\. Merge the kept instruction–call pairs withℒ\\mathcal\{L\}; call the resultℒ\+\\mathcal\{L\}^\{\+\}\.
5. 5\.Fine\-tune the base model onℒ\+\\mathcal\{L\}^\{\+\}with LoRA\. The result is the final model\.

The thresholdτ\\tauis set per domain on the training set, by sweeping a small grid and keeping the value that yields the best filtered training set; typical values are 0\.9 and 0\.95\.

### 4\.3Probe\-Guided Reranking

At inference time, the final model generates a pool of candidate calls for each instruction: one greedy decode and several sampled ones\. The probe scores every candidate, the greedy one included, with the same scores​\(c\)s\(c\)as in Equation \([2](https://arxiv.org/html/2608.03071#S4.E2)\)\.

A decision strategy turns these scores into a single call to submit\. Our strategies form 3 families that differ in the unit over which they decide\.*Candidate\-level*strategies return one whole candidate and differ only in the ranking score: the probe scores​\(c\)s\(c\), the candidate log\-probability, or a weighted sum of the two\.*Field\-level*strategies take the set of fields from one candidate and then choose each value on its own, from whichever candidate scores best on that field\.*Field\-set*strategies do the reverse: they keep the greedy values and decide only which fields survive, preferring the smallest field set that the probe still rates highly\. A field is dropped when its own probe score is low, when few candidates emit it, or when a candidate that scores nearly as well omits it\.

Each family also has a version that stays close to the greedy decode and changes it only on clear evidence\. We do not fix one strategy in advance\. For each domain the training set is split into 5 parts, the strategies are compared on 4 of them, and the winner is applied to the fifth\. The cost of PGR is small: the probe is shared with PBT, and scoring one candidate adds one forward pass\.

## 5ParamBench

In this section, we presentParamBench, a benchmark for tool\-call parameter generation\. As discussed in Section[2](https://arxiv.org/html/2608.03071#S2), existing benchmarks check a tool call only as a whole and do not grade how hard its parameters are to fill\. In contrast,ParamBenchshifts the focus to the parameter values:once the right tool is chosen, can the model correctly fill the parameters that the call requires?The task follows the value\-filling setup of Section[3](https://arxiv.org/html/2608.03071#S3)\.ParamBenchdraws a large pool of tool\-call instances from real agent traces and the schemas of 81 cloud\-network APIs, filters them with structural checks, and grades every instance into five difficulty levels by the structural features of its parameters, giving 1,022 instances in total\. Each instance keeps the standard JSON\-Schema call format and can be directly loaded and used by existing function\-calling evaluation tools\.

### 5\.1Deterministic Five\-Level Difficulty Scale

EachParamBenchinstance pairs a target API schema with a natural\-language instruction, an upstream context with the outputs of earlier calls, and a gold answer\. From these parts, four complexity scores are computed automatically:

- •nesting\_depth\(d\): the deepest JSON nesting among the required input fields;
- •num\_conditional\_dependencies\(c\): how many fields depend on the value of a sibling field;
- •num\_upstream\_transfers\(t\): how many inputs come from a prior tool’s output;
- •upstream\_extract\_depth\(e\): how deep the transferred value sits in that output\.

A fixed rule \(Table[1](https://arxiv.org/html/2608.03071#S5.T1), left\) turns the four scores into a level: it checks the levels in order and assigns the first one that fits, so the level rises as nesting gets deeper, conditional dependencies appear, and more values must be derived from upstream outputs\. Because the rule is a fixed function, every instance gets its level the same way, with no human judgment\.

### 5\.2Dual\-Source Construction

ParamBenchinstances come from two sources: human\-verified multi\-step diagnosis traces from an industrial cloud\-network agent system, and synthesis directly from the 81 frozen API schemas\.

#### Trace extraction\.

Each trace is a sequence of verified tool calls, in which every step records the tool it called, the parameters it used, and the output it returned\. A converter walks each trace and turns every executable step with a non\-trivial parameter into one instance, which consists of an instruction, an upstream context, and a gold answer\. The instruction is assembled from the trace’s task description and the step’s context; the upstream context collects the outputs of the preceding steps, trimmed to the fields the call actually reads; and the gold answer is the step’s verified parameters\. Difficulty rises naturally with a step’s position, because a first step needs no upstream value while a later step must copy several from earlier outputs\. Before release, every instance is checked to confirm that the gold answer validates against the frozen schema, and resource ids are replaced by anonymized ids in the same format\.

#### Schema\-driven synthesis\.

The hard instances are synthesized directly from the 81 frozen API schemas\. Because each schema’s complexity profile sets the difficulty level that its API can host, a synthesis plan assigns every API a number of instances to generate at its level, so that the APIs able to host L4–L5 receive the largest shares\. For each planned instance, the generation pipeline produces an instruction, an upstream\-tool context, and the target parameters\. The result is kept only if it passes a structural check: the required fields are present, the gold parameters validate against the JSON Schema, and the level label is legal and consistent with the actual nesting depth\.

grade\(d, c, t, e\):if d=0 and c=0 and t=0:L1elif d≤\\leq1 and c=0and t≤\\leq1 and e=0:L2elif d≤\\leq2 and c=0:L3elif d≤\\leq3 and c≥\\geq1:L4else:L5LevelInst\.ShareL110210\.0%L219819\.4%L320820\.4%L425625\.0%L525825\.2%Total1,022100%
Table 1:The difficulty grading rule \(left\) and the resulting instance distribution \(right\)\.

## 6Experiments

Table 2:Exact match \(EM\) and field\-level F1 of the 6 methods onParamBench\(PB\) and 6 external tool\-use benchmarks \(CFB = ComplexFuncBench\) with all models in their base variants\. Bold marks the best value in each column of a model block\.In this section, we evaluate PBT and PGR from multiple perspectives\. The study is organized around four research questions \(RQs\)\.

- RQ1Does generating SFT data with the probe signal, as in PBT, lead to better parameter generation?
- RQ2How much do PBT and PGR improve parameter generation over other leading tool\-use methods?
- RQ3How should the PGR strategy be chosen, and how much can it improve performance at inference time?
- RQ4How do PBT and PGR perform across the five difficulty levels, L1 to L5?

### 6\.1Experiment Settings

#### Datasets\.

ParamBenchis the in\-domain benchmark\. We add 6 external tool\-use benchmarks: NESTFUL\(Basu et al\.[2025](https://arxiv.org/html/2608.03071#bib.bib3)\), Seal\-Tools\(Wu et al\.[2024](https://arxiv.org/html/2608.03071#bib.bib37)\), xLAM\(Liu et al\.[2024](https://arxiv.org/html/2608.03071#bib.bib21)\), BFCL\(Patil et al\.[2025](https://arxiv.org/html/2608.03071#bib.bib25)\), API\-Bank\(Li et al\.[2023b](https://arxiv.org/html/2608.03071#bib.bib18)\), and ComplexFuncBench\(Zhong et al\.[2025](https://arxiv.org/html/2608.03071#bib.bib41)\)\.ParamBenchis split by API: 57 APIs \(729 instances\) form the training set, and 24 unseen APIs \(293 instances\) form the test set\. Each external benchmark is split into a training set and a test set, using the official split where available\.

#### Adaptation\.

The external benchmarks were built to test which tool to pick and how to chain calls\. We convert every instance into a per\-call record: the input gives the instruction, the spec of the target tool, and the outputs of the earlier calls in the chain; the model must output only the parameter object of the current call\. Tool selection is taken out of the task, so what remains is parameter generation\. When a benchmark accepts several values for a field, matching any of them counts \(BFCL\)\. After adaptation, each dataset is treated as its own domain: the probe and the hyperparameters come from its own training set only\.

Table 3:Parameter generation against 4 open tool\-use models, under the per\-call protocol; the open models use 3\-shot prompting and their native function\-calling format\. Bold marks the best value in each column\.![Refer to caption](https://arxiv.org/html/2608.03071v1/x1.png)Figure 3:Exact match of Qwen3\-8B climbing from 0\-shot to PBT\+PGR, against Claude Opus 4\.7\.
![Refer to caption](https://arxiv.org/html/2608.03071v1/x2.png)Figure 4:PGR with and without PBT; dashed lines give the group mean before reranking, solid lines after\.
![Refer to caption](https://arxiv.org/html/2608.03071v1/x3.png)Figure 5:Exact match of Qwen3\-8B by difficulty level\. Labels give the gain of PBT\+PGR over SeedSFT at each level\.

#### Models\.

We use 5 open models from 4 families: Qwen3 \(8B and 14B\), Gemma\-4\-12B, Ministral\-3\-8B, and Llama\-3\.1\-8B; the main analysis is on their base variants\. For comparison, we also measure 4 frontier models, Claude Opus 4\.7, GPT\-5\.4, DeepSeek\-V4\-Pro, and Qwen\-3\.6\-Plus, under 0\-shot and 3\-shot prompting\.

#### Metrics\.

We use two metrics\. Exact match \(EM\) counts a call as correct only if the generated parameter object equals the gold one exactly, so a single wrong field makes the whole call wrong\. Field\-level F1 gives partial credit for the fields that are right: the predicted and the gold parameter objects are flattened into<<path, value\>\>pairs with every field weighted equally, and F1 is the harmonic mean of precision and recall over the two sets\.

### 6\.2Results

#### RQ1: Probe\-filtered SFT\.

Table[2](https://arxiv.org/html/2608.03071#S6.T2)compares PBT with 5 baselines over 5 base models and 7 datasets\. 0\-shot prompts the base model, and SeedSFT fine\-tunes it on a seed set of gold\-labeled instances\. Each remaining method retrains the base model on the seed set plus self\-generated answers, and they differ only in the selection signal: SelfTrain keeps every answer\(Scudder[1965](https://arxiv.org/html/2608.03071#bib.bib32); He et al\.[2020](https://arxiv.org/html/2608.03071#bib.bib8)\); LogprobTrain and ConsistTrain keep the same number of answers as PBT, selected by mean log\-probability or by majority vote\(Wang et al\.[2023](https://arxiv.org/html/2608.03071#bib.bib34)\); and PBT selects by the probe signal of Section[4](https://arxiv.org/html/2608.03071#S4)\.

0\-shot shows why fine\-tuning is necessary: a base model often cannot even emit a well\-formed parameter object, and it averages only 19\.7 EM and 30\.5 F1\. SeedSFT fixes most of the format errors and lifts the averages to 51\.6 EM and 67\.5 F1\. The three self\-training baselines select data by unreliable signals, so the added data is noisy and the outcome is unstable: on average LogprobTrain reaches only 44\.2 EM, ConsistTrain 48\.9, and SelfTrain 53\.9\. PBT is the only method that improves steadily: it is above SeedSFT in all 35 model–dataset pairs, and it raises the averages to 59\.6 EM and 75\.1 F1, a gain of 8\.0 EM points over SeedSFT\.

#### RQ2: PBT and PGR versus other tool\-use methods\.

Table[3](https://arxiv.org/html/2608.03071#S6.T3)compares our model, Qwen3\-8B with PBT and PGR, against 4 open tool\-use models at the same scale: Llama\-xLAM\-2\-8b\-fc\-r\(Prabhakar et al\.[2025](https://arxiv.org/html/2608.03071#bib.bib27)\), Hammer2\.1\-7b\(Lin et al\.[2025](https://arxiv.org/html/2608.03071#bib.bib19)\), ToolACE\-2\.5\-Llama\-3\.1\-8B\(Liu et al\.[2025](https://arxiv.org/html/2608.03071#bib.bib20)\), and watt\-tool\-8B\(Watt AI[2024](https://arxiv.org/html/2608.03071#bib.bib35)\)\. Our model is the best in every column, and it averages 62\.5 EM over the 7 datasets, 21\.8 points above the strongest of the other 4 models \(Hammer2\.1\-7b, 40\.8\)\.

We further compare our model with the 4 frontier models, Claude Opus 4\.7, GPT\-5\.4, DeepSeek\-V4\-Pro, and Qwen\-3\.6\-Plus, under 3\-shot prompting: our model reaches the frontier level on all 7 datasets and surpasses all 4 frontier models on 3 datasets\. Figure[5](https://arxiv.org/html/2608.03071#S6.F5)shows this climb on BFCL and Seal\-Tools: Qwen3\-8B starts far below Claude Opus 4\.7, gains at every stage from prompting to SeedSFT and PBT, and ends above Opus 4\.7 on both datasets\. The 0\-shot setting and the full frontier panel are in the appendix\.

#### RQ3: PGR strategy choice and inference\-time gain\.

PGR samples a pool of candidate calls for each instruction and picks the final call from this pool\. Section[4\.3](https://arxiv.org/html/2608.03071#S4.SS3)gives 3 families of decision strategies, and each fits a different failure mode\.*Candidate\-level*strategies fit when the pool usually holds one fully correct call;*field\-level*strategies fit when no candidate is right as a whole but every field is right in some candidate;*field\-set*strategies fit when the greedy values are right but the call adds fields it should not\. A base model gets many values wrong, so there is much to rebuild: the field\-level strategy, which takes each field from its best candidate, gains 7\.5 EM over greedy on the base Qwen3\-14B\. After PBT the greedy call is already right most of the time, so the conservative strategy that stays close to it works best, liftingParamBenchEM from 35\.8 to 44\.4\. In addition, as shown in Figure[5](https://arxiv.org/html/2608.03071#S6.F5), PGR does not depend on PBT and works on its own: applied directly to models that were never fine\-tuned, as in the left half of the figure, it lifts exact match by 4\.2 points on average, and applied after PBT, as in the right half, it adds 4\.6 points\.

#### RQ4: Performance across difficulty levels\.

Figure[5](https://arxiv.org/html/2608.03071#S6.F5)splits the exact match of Qwen3\-8B onParamBenchand API\-Bank by the five difficulty levels, comparing SeedSFT with PBT\+PGR\. At the easy end the two methods are close: L1 is already solved well by SeedSFT alone, and L2 gains a little\. The gains concentrate at the hard levels\. OnParamBench, PBT\+PGR adds 11 points at L3, 5 points at L4, and 12 points at L5, where SeedSFT is close to zero; on API\-Bank, the first four levels already sit above 80 EM, and the gain appears at L5, which rises by 13 points\. The hard levels are exactly where deep nesting, conditional dependencies, and cross\-call derivation appear, so the improvement there shows that PBT and PGR learn part of what makes parameter generation difficult, instead of only polishing what the seed model already handles\.

## 7Conclusion

This paper puts tool\-call parameter generation at the center of tool\-use research, tracing its difficulty to deep nesting, conditional dependency, and cross\-call derivation\. We show that a linear probe on the hidden state, read before a parameter object is written, predicts whether the value will be correct; PBT uses the signal to filter self\-generated training data, and PGR to rerank sampled candidates\. For evaluation, we releaseParamBench, a benchmark built from real cloud\-network agent traces and graded into five difficulty levels\. Extensive experiments across open models and external benchmarks validate the effectiveness of both PBT and PGR\.

#### Limitations\.

First,ParamBenchcovers only one scenario: its APIs and traces all come from cloud networking, although the difficulty scale carries over to other domains\. Second, the probe does not transfer: tied to one dataset, model, and sampling temperature, it degrades outside that setting, so a new probe must be fit for every new domain\. Third, PBT and PGR depend on a small labeled set and a pool of unlabeled instructions; more general methods for strengthening parameter generation remain to be explored\.

## References

- Anthropic \(2024\)Anthropic\. 2024\.Introducing the Model Context Protocol\.https://www\.anthropic\.com/news/model\-context\-protocol\.
- Azaria and Mitchell \(2023\)Azaria, A\.; and Mitchell, T\. 2023\.The Internal State of an LLM Knows When It’s Lying\.In*Findings of the Association for Computational Linguistics: EMNLP 2023*, 967–976\. Association for Computational Linguistics\.
- Basu et al\. \(2025\)Basu, K\.; Abdelaziz, I\.; Kate, K\.; Agarwal, M\.; Crouse, M\.; Rizk, Y\.; Bradford, K\.; Munawar, A\.; Kumaravel, S\.; Goyal, S\.; Wang, X\.; Lastras, L\. A\.; and Kapanipathi, P\. 2025\.NESTFUL: A Benchmark for Evaluating LLMs on Nested Sequences of API Calls\.In*Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing \(EMNLP\)*, 33538–33547\. Association for Computational Linguistics\.
- Chen et al\. \(2024\)Chen, Y\.; Yoon, J\.; Sachan, D\. S\.; Wang, Q\.; Cohen\-Addad, V\.; Bateni, M\.; Lee, C\.\-Y\.; and Pfister, T\. 2024\.Re\-Invoke: Tool Invocation Rewriting for Zero\-Shot Tool Retrieval\.In*Findings of the Association for Computational Linguistics: EMNLP 2024*, 4705–4726\. Association for Computational Linguistics\.
- Dong et al\. \(2025\)Dong, Y\.; Ruan, C\. F\.; Cai, Y\.; Xu, Z\.; Zhao, Y\.; Lai, R\.; and Chen, T\. 2025\.XGrammar: Flexible and Efficient Structured Generation Engine for Large Language Models\.In*Proceedings of the Eighth Conference on Machine Learning and Systems \(MLSys 2025\)*\. mlsys\.org\.
- Du, Xiao, and Li \(2024\)Du, X\.; Xiao, C\.; and Li, S\. 2024\.HaloScope: Harnessing Unlabeled LLM Generations for Hallucination Detection\.In*Advances in Neural Information Processing Systems 37 \(NeurIPS 2024\)*\.
- Han et al\. \(2025\)Han, H\.; Zhu, T\.; Zhang, X\.; Wu, M\.; Hao, X\.; and Chen, W\. 2025\.NesTools: A Dataset for Evaluating Nested Tool Learning Abilities of Large Language Models\.In*Proceedings of the 31st International Conference on Computational Linguistics \(COLING 2025\)*, 9824–9844\. Association for Computational Linguistics\.
- He et al\. \(2020\)He, J\.; Gu, J\.; Shen, J\.; and Ranzato, M\. 2020\.Revisiting Self\-Training for Neural Sequence Generation\.In*The Eighth International Conference on Learning Representations \(ICLR 2020\)*\. OpenReview\.net\.
- Healy et al\. \(2026\)Healy, K\.; Srinivasan, B\.; Madathil, V\.; and Wu, J\. 2026\.Internal Representations as Indicators of Hallucinations in Agent Tool Selection\.arXiv preprint arXiv:2601\.05214\.
- Hu et al\. \(2022\)Hu, E\. J\.; Shen, Y\.; Wallis, P\.; Allen\-Zhu, Z\.; Li, Y\.; Wang, S\.; Wang, L\.; and Chen, W\. 2022\.LoRA: Low\-Rank Adaptation of Large Language Models\.In*The Tenth International Conference on Learning Representations \(ICLR 2022\)*\. OpenReview\.net\.
- Hu et al\. \(2024\)Hu, X\.; Zhang, Y\.; Peng, R\.; Zhang, H\.; Wu, C\.; Chen, G\.; and Zhao, J\. 2024\.Embedding and Gradient Say Wrong: A White\-Box Method for Hallucination Detection\.In*Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing \(EMNLP\)*, 1950–1959\. Association for Computational Linguistics\.
- Huang et al\. \(2024\)Huang, Y\.; Shi, J\.; Li, Y\.; Fan, C\.; Wu, S\.; Zhang, Q\.; Liu, Y\.; Zhou, P\.; Wan, Y\.; Gong, N\. Z\.; and Sun, L\. 2024\.MetaTool Benchmark for Large Language Models: Deciding Whether to Use Tools and Which to Use\.In*The Twelfth International Conference on Learning Representations \(ICLR 2024\)*\. OpenReview\.net\.
- Ji et al\. \(2024\)Ji, Z\.; Chen, D\.; Ishii, E\.; Cahyawijaya, S\.; Bang, Y\.; Wilie, B\.; and Fung, P\. 2024\.LLM Internal States Reveal Hallucination Risk Faced With a Query\.In*Proceedings of the 7th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP \(BlackboxNLP 2024\)*, 88–104\. Association for Computational Linguistics\.
- Kadavath et al\. \(2022\)Kadavath, S\.; Conerly, T\.; Askell, A\.; Henighan, T\.; Drain, D\.; Perez, E\.; Schiefer, N\.; Hatfield\-Dodds, Z\.; DasSarma, N\.; Tran\-Johnson, E\.; Johnston, S\.; Showk, S\. E\.; Jones, A\.; Elhage, N\.; Hume, T\.; Chen, A\.; Bai, Y\.; Bowman, S\.; Fort, S\.; Ganguli, D\.; Hernandez, D\.; Jacobson, J\.; Kernion, J\.; Kravec, S\.; Lovitt, L\.; Ndousse, K\.; Olsson, C\.; Ringer, S\.; Amodei, D\.; Brown, T\.; Clark, J\.; Joseph, N\.; Mann, B\.; McCandlish, S\.; Olah, C\.; and Kaplan, J\. 2022\.Language Models \(Mostly\) Know What They Know\.arXiv preprint arXiv:2207\.05221\.
- Kim et al\. \(2024\)Kim, S\.; Moon, S\.; Tabrizi, R\.; Lee, N\.; Mahoney, M\. W\.; Keutzer, K\.; and Gholami, A\. 2024\.An LLM Compiler for Parallel Function Calling\.In*Proceedings of the 41st International Conference on Machine Learning \(ICML 2024\)*, volume 235 of*Proceedings of Machine Learning Research*, 24370–24391\. PMLR\.
- Kossen et al\. \(2024\)Kossen, J\.; Han, J\.; Razzak, M\.; Schut, L\.; Malik, S\. A\.; and Gal, Y\. 2024\.Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMs\.arXiv preprint arXiv:2406\.15927\.
- Li et al\. \(2023a\)Li, K\.; Patel, O\.; Viégas, F\. B\.; Pfister, H\.; and Wattenberg, M\. 2023a\.Inference\-Time Intervention: Eliciting Truthful Answers from a Language Model\.In*Advances in Neural Information Processing Systems 36 \(NeurIPS 2023\)*\.
- Li et al\. \(2023b\)Li, M\.; Zhao, Y\.; Yu, B\.; Song, F\.; Li, H\.; Yu, H\.; Li, Z\.; Huang, F\.; and Li, Y\. 2023b\.API\-Bank: A Comprehensive Benchmark for Tool\-Augmented LLMs\.In*Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing \(EMNLP\)*, 3102–3116\. Association for Computational Linguistics\.
- Lin et al\. \(2025\)Lin, Q\.; Wen, M\.; Peng, Q\.; Nie, G\.; Liao, J\.; Wang, J\.; Mo, X\.; Zhou, J\.; Cheng, C\.; Zhao, Y\.; Wang, J\.; and Zhang, W\. 2025\.Robust Function\-Calling for On\-Device Language Model via Function Masking\.In*The Thirteenth International Conference on Learning Representations \(ICLR 2025\)*\. OpenReview\.net\.
- Liu et al\. \(2025\)Liu, W\.; Huang, X\.; Zeng, X\.; Hao, X\.; Yu, S\.; Li, D\.; Wang, S\.; Gan, W\.; Liu, Z\.; Yu, Y\.; Wang, Z\.; Wang, Y\.; Ning, W\.; Hou, Y\.; Wang, B\.; Wu, C\.; Wang, X\.; Liu, Y\.; Wang, Y\.; Tang, D\.; Tu, D\.; Shang, L\.; Jiang, X\.; Tang, R\.; Lian, D\.; Liu, Q\.; and Chen, E\. 2025\.ToolACE: Winning the Points of LLM Function Calling\.In*The Thirteenth International Conference on Learning Representations \(ICLR 2025\)*\. OpenReview\.net\.
- Liu et al\. \(2024\)Liu, Z\.; Hoang, T\.; Zhang, J\.; Zhu, M\.; Lan, T\.; Kokane, S\.; Tan, J\.; Yao, W\.; Liu, Z\.; Feng, Y\.; Murthy, R\.; Yang, L\.; Savarese, S\.; Niebles, J\. C\.; Wang, H\.; Heinecke, S\.; and Xiong, C\. 2024\.APIGen: Automated PIpeline for Generating Verifiable and Diverse Function\-Calling Datasets\.In*Advances in Neural Information Processing Systems 37 \(NeurIPS 2024\) Datasets and Benchmarks Track*\.
- Marks and Tegmark \(2024\)Marks, S\.; and Tegmark, M\. 2024\.The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets\.In*First Conference on Language Modeling \(COLM 2024\)*\. OpenReview\.net\.
- OpenAI \(2023\)OpenAI\. 2023\.Function Calling and Other API Updates\.https://openai\.com/index/function\-calling\-and\-other\-api\-updates/\.
- Orgad et al\. \(2025\)Orgad, H\.; Toker, M\.; Gekhman, Z\.; Reichart, R\.; Szpektor, I\.; Kotek, H\.; and Belinkov, Y\. 2025\.LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinations\.In*The Thirteenth International Conference on Learning Representations \(ICLR 2025\)*\. OpenReview\.net\.
- Patil et al\. \(2025\)Patil, S\. G\.; Mao, H\.; Yan, F\.; Ji, C\. C\.\-J\.; Suresh, V\.; Stoica, I\.; and Gonzalez, J\. E\. 2025\.The Berkeley Function Calling Leaderboard \(BFCL\): From Tool Use to Agentic Evaluation of Large Language Models\.In*Proceedings of the 42nd International Conference on Machine Learning \(ICML 2025\)*, volume 267 of*Proceedings of Machine Learning Research*, 48371–48392\. PMLR\.
- Patil et al\. \(2024\)Patil, S\. G\.; Zhang, T\.; Wang, X\.; and Gonzalez, J\. E\. 2024\.Gorilla: Large Language Model Connected with Massive APIs\.In*Advances in Neural Information Processing Systems 37 \(NeurIPS 2024\)*\.
- Prabhakar et al\. \(2025\)Prabhakar, A\.; Liu, Z\.; Zhu, M\.; Zhang, J\.; Awalgaonkar, T\. M\.; Wang, S\.; Liu, Z\.; Chen, H\.; Hoang, T\.; Niebles, J\. C\.; Heinecke, S\.; Yao, W\.; Wang, H\.; Savarese, S\.; and Xiong, C\. 2025\.APIGen\-MT: Agentic Pipeline for Multi\-Turn Data Generation via Simulated Agent\-Human Interplay\.In*Advances in Neural Information Processing Systems 38 \(NeurIPS 2025\) Datasets and Benchmarks Track*\.
- Qin et al\. \(2025\)Qin, Y\.; Hu, S\.; Lin, Y\.; Chen, W\.; Ding, N\.; Cui, G\.; Zeng, Z\.; Zhou, X\.; Huang, Y\.; Xiao, C\.; Han, C\.; Fung, Y\. R\.; Su, Y\.; Wang, H\.; Qian, C\.; Tian, R\.; Zhu, K\.; Liang, S\.; Shen, X\.; Xu, B\.; Zhang, Z\.; Ye, Y\.; Li, B\.; Tang, Z\.; Yi, J\.; Zhu, Y\.; Dai, Z\.; Yan, L\.; Cong, X\.; Lu, Y\.; Zhao, W\.; Huang, Y\.; Yan, J\.; Han, X\.; Sun, X\.; Li, D\.; Phang, J\.; Yang, C\.; Wu, T\.; Ji, H\.; Li, G\.; Liu, Z\.; and Sun, M\. 2025\.Tool Learning with Foundation Models\.*ACM Comput\. Surv\.*, 57\(4\): 101:1–101:40\.
- Qin et al\. \(2024\)Qin, Y\.; Liang, S\.; Ye, Y\.; Zhu, K\.; Yan, L\.; Lu, Y\.; Lin, Y\.; Cong, X\.; Tang, X\.; Qian, B\.; Zhao, S\.; Hong, L\.; Tian, R\.; Xie, R\.; Zhou, J\.; Gerstein, M\.; Li, D\.; Liu, Z\.; and Sun, M\. 2024\.ToolLLM: Facilitating Large Language Models to Master 16000\+ Real\-world APIs\.In*The Twelfth International Conference on Learning Representations \(ICLR 2024\)*\. OpenReview\.net\.
- Qu et al\. \(2025\)Qu, C\.; Dai, S\.; Wei, X\.; Cai, H\.; Wang, S\.; Yin, D\.; Xu, J\.; and Wen, J\. 2025\.Tool learning with large language models: a survey\.*Frontiers Comput\. Sci\.*, 19\(8\): 198343\.
- Schick et al\. \(2023\)Schick, T\.; Dwivedi\-Yu, J\.; Dessì, R\.; Raileanu, R\.; Lomeli, M\.; Hambro, E\.; Zettlemoyer, L\.; Cancedda, N\.; and Scialom, T\. 2023\.Toolformer: Language Models Can Teach Themselves to Use Tools\.In*Advances in Neural Information Processing Systems 36 \(NeurIPS 2023\)*\.
- Scudder \(1965\)Scudder, H\. J\., III\. 1965\.Probability of Error of Some Adaptive Pattern\-Recognition Machines\.*IEEE Transactions on Information Theory*, 11\(3\): 363–371\.
- Snyder, Moisescu, and Zafar \(2024\)Snyder, B\.; Moisescu, M\.; and Zafar, M\. B\. 2024\.On Early Detection of Hallucinations in Factual Question Answering\.In*Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining \(KDD 2024\)*, 2721–2732\. ACM\.
- Wang et al\. \(2023\)Wang, X\.; Wei, J\.; Schuurmans, D\.; Le, Q\. V\.; Chi, E\. H\.; Narang, S\.; Chowdhery, A\.; and Zhou, D\. 2023\.Self\-Consistency Improves Chain of Thought Reasoning in Language Models\.In*The Eleventh International Conference on Learning Representations \(ICLR 2023\)*\. OpenReview\.net\.
- Watt AI \(2024\)Watt AI\. 2024\.watt\-tool\-8B\.https://huggingface\.co/watt\-ai/watt\-tool\-8B\.
- Willard and Louf \(2023\)Willard, B\. T\.; and Louf, R\. 2023\.Efficient Guided Generation for Large Language Models\.arXiv preprint arXiv:2307\.09702\.
- Wu et al\. \(2024\)Wu, M\.; Zhu, T\.; Han, H\.; Tan, C\.; Zhang, X\.; and Chen, W\. 2024\.Seal\-Tools: Self\-instruct Tool Learning Dataset for Agent Tuning and Detailed Benchmark\.In*Natural Language Processing and Chinese Computing: 13th National CCF Conference, Part II \(NLPCC 2024\)*, volume 15360 of*Lecture Notes in Computer Science*, 372–384\. Springer\.
- Yao et al\. \(2025\)Yao, S\.; Shinn, N\.; Razavi, P\.; and Narasimhan, K\. R\. 2025\.τ\\tau\-bench: A Benchmark for Tool\-Agent\-User Interaction in Real\-World Domains\.In*The Thirteenth International Conference on Learning Representations \(ICLR 2025\)*\. OpenReview\.net\.
- Yao et al\. \(2023\)Yao, S\.; Zhao, J\.; Yu, D\.; Du, N\.; Shafran, I\.; Narasimhan, K\. R\.; and Cao, Y\. 2023\.ReAct: Synergizing Reasoning and Acting in Language Models\.In*The Eleventh International Conference on Learning Representations \(ICLR 2023\)*\. OpenReview\.net\.
- Zhang et al\. \(2025\)Zhang, J\.; Lan, T\.; Zhu, M\.; Liu, Z\.; Hoang, T\.; Kokane, S\.; Yao, W\.; Tan, J\.; Liu, Z\.; Feng, Y\.; Niebles, J\. C\.; Heinecke, S\.; Wang, H\.; Savarese, S\.; and Xiong, C\. 2025\.xLAM: A Family of Large Action Models to Empower AI Agent Systems\.In*Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\) \(NAACL\)*, 11583–11597\. Association for Computational Linguistics\.
- Zhong et al\. \(2025\)Zhong, L\.; Du, Z\.; Zhang, X\.; Hu, H\.; and Tang, J\. 2025\.ComplexFuncBench: Exploring Multi\-Step and Constrained Function Calling under Long\-Context Scenario\.arXiv preprint arXiv:2501\.10132\.

## Appendix AParamBench Details

### A\.1Positioning Against Existing Benchmarks

Table[4](https://arxiv.org/html/2608.03071#A1.T4)contrastsParamBenchwith the representative tool\-use benchmarks discussed in Section[2](https://arxiv.org/html/2608.03071#S2)\.

Table 4:Positioning ofParamBenchagainst representative tool\-use benchmarks: ToolBench\(Qin et al\.[2024](https://arxiv.org/html/2608.03071#bib.bib29)\), API\-Bank\(Li et al\.[2023b](https://arxiv.org/html/2608.03071#bib.bib18)\), NESTFUL\(Basu et al\.[2025](https://arxiv.org/html/2608.03071#bib.bib3)\), and Seal\-Tools\(Wu et al\.[2024](https://arxiv.org/html/2608.03071#bib.bib37)\)\. “Param\. nesting” = nesting depth of a single call’s parameter object; NESTFUL and Seal\-Tools nest at the call level \(one call’s output feeds a later call\) while their per\-call parameters stay largely flat\. “Cond\. deps” = inter\-field conditional dependencies; “Cross\-call” = a downstream parameter explicitly derived from an upstream output\. NESTFUL and Seal\-Tools also serve as the external evaluation domains in Section[6](https://arxiv.org/html/2608.03071#S6)\.
### A\.2Construction and Validation

Trace\-extracted instances are produced by the converter described in Section[5](https://arxiv.org/html/2608.03071#S5)\. The converter walks each verified diagnosis trace and turns every executable step into one instance: the upstream context keeps only the fields that the call actually reads, and the step’s verified parameters become the gold answer\. Before release, every instance passed two checks\. First, the gold answer must validate against the frozen schema of its tool\. Second, all organization\-specific values are replaced with anonymized aliases under one consistent mapping covering resource ids \(prefix and length kept\), region ids and display names, account ids, domains, IP addresses, instance specs, metric namespaces and keys, runbook and scenario codes, and uniformly shifted timestamps\. A value maps to the same alias in the instruction, the context, and the gold answer, preserving cross\-call transfers and exact\-match scoring\.

Synthesized instances follow a per\-schema plan\. Because the complexity profile of a schema determines which difficulty levels its API can host, the plan assigns the largest shares to the APIs that can host L4 and L5\. A synthesized instance is kept only when three conditions hold: its required fields are present, its gold parameters validate against the schema, and its level label matches the structure that was actually generated\.

Each instance names its target tool, and every name resolves against the frozen pool of 81 cloud\-network API schemas\.

### A\.3A Complete Example Instance

The listing below shows an L5 instance in full\. The target call isDescribeDBInstancePerformance\. Three parts of the gold answer cannot be copied from the instruction\. TheDBInstanceIdmust be taken from the output of an earlierDescribeDBInstancescall\. TheKeyfield must compose 3 metric names from the reported symptoms\. The time window must be converted from Beijing time in the instruction to UTC, and its start is anchored by an event detail returned by an earlierDescribeEventscall\.

```
{"sample_id": "PB-eval2-gen-L5-0004",
 "tool": "DescribeDBInstancePerformance",
 "instruction": "[translated from Chinese]
   An RDS MySQL 8.0 instance reports
   intermittent slow-query alerts. Pull the
   QPS/TPS, slow-log count, and memory/CPU
   curves for 03:00-09:00 Beijing time
   today.",
 "golden_params": {
   "DBInstanceId": "rm-aaaaaaaaaaaaaaaxe",
   "Key": "DB_QPSTPS,DB_SlowLogs,
           DB_MemCpuUsage",
   "StartTime": "2026-06-03T19:00Z",
   "EndTime": "2026-06-04T01:00Z"},
 "complexity_labels": {"level": "L5",
   "num_upstream_transfers": 2},
 "data_flow_context": {"upstream": [
   {"tool": "DescribeDBInstances",
    "output": {"DBInstanceId":
      "rm-aaaaaaaaaaaaaaaxe",
      "Engine": "MySQL", ...}},
   {"tool": "DescribeEvents",
    "output": {"EventName":
      "SlowQueryThresholdExceeded",
      "EventTime": "2026-06-04T00:17Z",
      "Detail": "[translated] window
        started at 2026-06-03T19:02Z",
      ...}}]}}
```

An L5ParamBenchinstance, abridged\. The released record renders these blocks inside chat messages; the listing regroups them, names the API behind each upstream output, translates Chinese free text, and truncates long outputs\. Identifier and value fields match the released record\.

## Appendix BExperimental Setup Details

### B\.1External Benchmark Adaptation

Every external instance is converted into the per\-call record described in Section[6](https://arxiv.org/html/2608.03071#S6)\. A record contains the user query, the name and JSON spec of the one tool to call, the outputs of previously executed steps \(each truncated to 280 characters\), and the gold arguments\. The conversion rules differ from source to source\. BFCL parallel\-call categories are skipped, because a parallel call has no single current tool\. xLAM calls whose tool list contains duplicate names are dropped\. NESTFUL is split at the sequence level with an 80/20 ratio, and its test records are drawn only from the held\-out sequences\. Seal\-Tools uses its official split\. For xLAM, the training pool excludes every sample whose source id appears in the test file, and a split audit confirms that the train and test source ids do not overlap\.

Table[5](https://arxiv.org/html/2608.03071#A2.T5)lists the frozen test sets\. For the 3 benchmarks used only for evaluation \(BFCL, API\-Bank, ComplexFuncBench\), the few\-shot examples \(30 per dataset, fixed seed\) are carved out of the pool first, the 150 labeled and 250 unlabeled training records are drawn next, and the remainder is the frozen evaluation set\.

The frontier panel of Table[9](https://arxiv.org/html/2608.03071#A4.T9)was run on a different record subset for these 3 benchmarks \(500/456/500 records against our 770/226/770\)\. Both panels are therefore rescored on the intersection of the two subsets \(n=318/226/318n=318/226/318\) from per\-record outputs\. On API\-Bank the intersection equals our full set, and the rescored PBT row reproduces Table[2](https://arxiv.org/html/2608.03071#S6.T2)exactly\.

Table 5:Frozen test sets after per\-call conversion\.
### B\.2Prompt Templates

Two templates are used\. TheParamBenchrunner uses the system prompt of Figure[8](https://arxiv.org/html/2608.03071#A2.F8)\(English translation; the released records embed the Chinese original\)\. The user message gives the instruction, the target API schema, and the upstream context as titled blocks\. The prompt requests a fixed four\-key reasoning object whose finalparamskey carries the parameter object scored against the gold\. In the 3\-shot setting, the examples come from difficulty levels other than the target’s\.

![Refer to caption](https://arxiv.org/html/2608.03071v1/figs/probe_layer_auc_draft.png)Figure 6:Per\-layer AUC of the correctness probe\. The best layer sits at about two\-thirds of network depth\.```
You are a Platform-X OpenAPI expert. Given
a natural-language instruction, the target
API schema, and the upstream tool-output
context, reason in three steps (schema
structure analysis -> parameter source
judgment -> layer-by-layer construction of
the nested parameters) and produce the
parameter JSON for a direct call.

Output strictly the following structure
(all four top-level keys required):
{
  "step1_schema_analysis": "...",
  "step2_source_judgment": [
    {"param_path": "...",
     "source_kind": "literal|from_context
                     |from_step|derive",
     "source_detail": "..."}
  ],
  "step3_param_construction": "...",
  "params": { ... final JSON ... }
}

Do not return markdown fences. Do not add
explanatory text.
```

Figure 7:System prompt of theParamBenchrunner, in English translation\. The released records embed the Chinese original\.```
You are an API function-calling expert.
Given a user query, the JSON spec of ONE
tool to call, and the outputs of previously
executed steps, produce ONLY the arguments
for the current tool call as a JSON object.
When an argument value must come from a
previous step’s output, use the same
reference convention shown in the
examples/context (e.g. a step label or the
literal value). Output strictly a JSON
object, no prose.
```

Figure 8:System prompt of the external per\-call template\.The 6 external benchmarks use the per\-call template of Figure[8](https://arxiv.org/html/2608.03071#A2.F8); the user message gives the user query, the spec of the tool to call now, and the outputs of previously executed steps\.

### B\.3Hyperparameters

Table[6](https://arxiv.org/html/2608.03071#A2.T6)lists the final settings behind the main tables\. The learning rate and the number of epochs were chosen per model family from\{5×10−4,1×10−4\}×\{2,4\}\\\{5\\times 10^\{\-4\},1\\times 10^\{\-4\}\\\}\\times\\\{2,4\\\}on a held\-out 20% split of the seed set: Gemma\-4 uses1×10−41\\times 10^\{\-4\}; Llama\-3\.1 uses1×10−41\\times 10^\{\-4\}and Ministral\-3 uses 2 epochs on NESTFUL; all other cells use5×10−45\\times 10^\{\-4\}and 4 epochs\. The PBT thresholdτ\\tauwas swept over\{0\.8,0\.9,0\.95\}\\\{0\.8,0\.9,0\.95\\\}on the training side\. All remaining values were fixed in advance and not searched\.

Table 6:Final hyperparameters behind the main tables\. Per\-family exceptions are listed in the text\.
### B\.4Compute, Seeds, and Statistical Tests

#### Compute infrastructure\.

Training and evaluation ran on cloud GPU nodes, each with 8 NVIDIA H20\-3e GPUs \(140 GB memory per GPU\), two 48\-core Intel Xeon Platinum 8575C CPUs, and 2 TiB RAM, running Linux with CUDA 12\.8\. The result campaigns used up to 4 nodes \(32 GPUs\)\. All jobs ran in Docker containers with PyTorch 2\.8, Transformers 4\.57\.1, and PEFT 0\.19\.1; candidate pools were generated with Hugging Facegenerateor vLLM under the same sampling settings\. Some of the Ministral and Gemma arms were trained on a university server with 8 NVIDIA A100\-SXM4 GPUs \(40 GB\)\. The 4 frontier models were accessed through their public inference APIs\.

#### Runs and seeds\.

Unless stated otherwise, every number in the main tables comes from a single run with training seed 42\. The seed 43/47/53 rows of Table[11](https://arxiv.org/html/2608.03071#A4.T11)report seed variants separately\. Candidate sampling does not fix a generator seed, so sampled pools vary across runs; the repeatedParamBenchand ComplexFuncBench runs behind the cells of Table[10](https://arxiv.org/html/2608.03071#A4.T10)show the size of this variation\.

#### Statistical significance\.

Two tests support the main comparisons\. Over the 35 model–dataset pairs of Table[2](https://arxiv.org/html/2608.03071#S6.T2), PBT is above SeedSFT in every pair, and a two\-sided Wilcoxon signed\-rank test on the paired EM values givesp=5\.8×10−11p=5\.8\\times 10^\{\-11\}\. At the instance level, paired exact McNemar tests on shared test instances confirm individual contrasts: for example, the Qwen3\-14BParamBenchgain from SeedSFT \(26\.6\) to PBT \(35\.8\) over the 293 test instances hasp=1\.4×10−5p=1\.4\\times 10^\{\-5\}\. All fine\-tuning runs use a single seed, so all tests are paired at the instance level\.

## Appendix CProbe Analysis

### C\.1Per\-Layer AUC

Figure[6](https://arxiv.org/html/2608.03071#A2.F6)gives the per\-layer AUC of the correctness probe of Section[4](https://arxiv.org/html/2608.03071#S4)\. One probe is trained per layer of the grid in Table[6](https://arxiv.org/html/2608.03071#A2.T6)on the hidden state at the decision point of each parameter, and the layer that performs best sits at roughly two\-thirds of the network depth\.

### C\.2Robustness Across Scales, Checkpoints, and Datasets

Table[7](https://arxiv.org/html/2608.03071#A3.T7)reports the decision\-point probe AUC onParamBenchacross 4 Qwen3 scales, before and after fine\-tuning\. All 8 cells fall between 0\.944 and 0\.993, consistent to rounding with the 0\.93–0\.99 range stated in Section[4](https://arxiv.org/html/2608.03071#S4)\. The before and after columns use different prompting protocols and some before\-cells rest on a few hundred parameters each, so the cells are indicative rather than strictly comparable\.

The signal is also stable in three further directions\. Refitting with 10 random seeds gives AUC0\.983±0\.0030\.983\\pm 0\.003\. A probe trained on one checkpoint keeps 0\.982 on the next \(0\.986 in its own setting\), and a 3\-checkpoint transfer matrix stays within 0\.948–0\.986\. Nor is it Qwen\-specific: a Llama\-3\.1\-8B probe reaches 0\.963 onParamBench\. What it does not survive is a change of dataset: NESTFUL AUC drops to 0\.770 \(Qwen3\-8B\) and 0\.711 \(Llama\-3\.1\-8B\), which is why each domain trains its own probe, as stated in Section[4](https://arxiv.org/html/2608.03071#S4)\.

Table 7:Decision\-point probe AUC onParamBenchacross model scales, before and after fine\-tuning\.

## Appendix DFull Results

### D\.1Open Tool\-Use Models

Table[8](https://arxiv.org/html/2608.03071#A4.T8)reports the full open tool\-use panel behind Table[3](https://arxiv.org/html/2608.03071#S6.T3): the 4 open tool\-use models of the main text plus Qwen3\-8B prompted in its native function\-calling format, at the same 7 to 8B scale, 0\-shot and 3\-shot, against our Qwen3\-8B under both PBT and PBT\+PGR\. The main text keeps the 3\-shot rows of the 4 tuned models and the PBT\+PGR row\.

Three patterns stand out in the full panel\. First, few\-shot prompting moves the open models unevenly: 3 shots lift API\-Bank by 11 to 13 points for xLAM\-2\-8B\-fc\-r, Hammer2\.1\-7B, and watt\-tool\-8B, but leave theirParamBenchand NESTFUL columns almost where they were\. Second, NESTFUL is the weakest column for every open model: none of the 5 exceeds 10\.8 EM, against 38\.8 for our model, and the distance comes from the cross\-call derivations that the per\-call record makes explicit\. Third, the native function\-calling variant of Qwen3\-8B is the strongest open entry on BFCL, at 53\.9 EM 0\-shot, yet it still trails our PBT model by about 20 points on that dataset, so the gap is not a matter of output format alone\.

Table 8:Open tool\-use models onParamBenchand the 6 external benchmarks under the identical per\-call protocol of Section[6](https://arxiv.org/html/2608.03071#S6), 0\-shot and 3\-shot, against Qwen3\-8B trained with PBT and reranked with PGR\. All rows share identical record sets on all 7 datasets, so every column is row\-comparable\. Bold marks the best value in each column across the 12 rows\.Table 9:Frontier models onParamBenchand the 6 external benchmarks under the identical per\-call protocol of Section[6](https://arxiv.org/html/2608.03071#S6), 0\-shot and 3\-shot, against Qwen3\-8B trained with PBT and reranked with PGR\. GPT\-5\.4 fails to emit parseable calls 0\-shot on BFCL and recovers with 3 shots\. The frontier panel was run on a different record subset for BFCL, API\-Bank and ComplexFuncBench, so on those 3 datasets both panels are scored here on the intersection of the two subsets \(n=318/226/318n=318/226/318\), recomputed from per\-record outputs\. On BFCL the Ours rows use the 13\-candidate pool of Table[6](https://arxiv.org/html/2608.03071#A2.T6), where reranking does not help \(76\.4 to 75\.5 EM\); Figure[5](https://arxiv.org/html/2608.03071#S6.F5)draws a 49\-candidate BFCL pool, about 4 times that budget, where PGR gains 1\.6 EM \(76\.1 to 77\.7\) on the same intersection\. Every column is therefore row\-comparable and bold marks the best value in each column across all 10 rows\.
### D\.2Frontier Models

Table[9](https://arxiv.org/html/2608.03071#A4.T9)reports the full frontier panel referenced in Section[6](https://arxiv.org/html/2608.03071#S6): 4 frontier models measured on all 7 datasets under the same adapted per\-call protocol as the local models, 0\-shot and 3\-shot, against our Qwen3\-8B under both PBT and PBT\+PGR\.

For the frontier models, the value of 3 shots is concentrated where the output format is the obstacle\. GPT\-5\.4 fails to emit parseable calls on BFCL 0\-shot \(5\.7 EM\) and recovers to 61\.6 with examples, and the other 3 models gain 9 to 13 points on BFCL from the same treatment\. On the remaining datasets the panel moves by a few points at most and clusters tightly: at 3 shots the 4 models sit within about 2 points of one another on Seal\-Tools, xLAM, and API\-Bank\. The margin of our model over the panel is largest exactly there: \+12\.6 EM on BFCL over the best 3\-shot frontier score and \+7\.9 on API\-Bank, both reached by PBT alone, before any reranking\.

### D\.3Probe\-Guided Reranking, Per\-Arm Results

Tables[10](https://arxiv.org/html/2608.03071#A4.T10)and[11](https://arxiv.org/html/2608.03071#A4.T11)give the per\-arm reranking measurements behind Figure[5](https://arxiv.org/html/2608.03071#S6.F5), both recomputed from the archived candidate pools under the same 5\-fold protocol\. Table[10](https://arxiv.org/html/2608.03071#A4.T10)covers instruction\-tuned models without fine\-tuning; its 9 cells average\+4\.2\+4\.2EM, the left half of the figure\. The base\-variant comparison quoted in the main text comes from an earlier archived pool: on the base Qwen3\-14BParamBenchpool \(greedy 28\.3\), field\-level splice adds\+7\.5\+7\.5EM and candidate\-level probe argmax\+6\.1\+6\.1, while ranking by log\-probability loses points\. Table[11](https://arxiv.org/html/2608.03071#A4.T11)covers the same 2 datasets after PBT; the 13 arms drawn in the figure average\+4\.6\+4\.6EM, its right half\. For the Qwen3\-14B\-BaseParamBenchcell both archived pools are listed: the main text quotes the n12 pool \(35\.8 to 44\.4\), the figure draws the RS pool\.

The two sides of the figure behave differently\. Without fine\-tuning, the gain tracks the weakness of the greedy decode: onParamBenchthe cells gain \+5\.2 to \+9\.9 EM, largest on Ministral\-8B \(from a greedy baseline of 16\.7\), and for Llama\-3\.1\-8B the F1 gain reaches \+16\.8, mostly by repairing structure; on ComplexFuncBench, where greedy already sits near 29 to 37, gains stay between \+0\.8 and \+1\.7\. After PBT all 13 arms improve, by \+0\.8 to \+10\.2 EM, and the spread again concentrates onParamBench: the seed variants of the same Qwen3\-8B cell span \+3\.4 to \+10\.2, the sampling and seed variance of Appendix[B\.4](https://arxiv.org/html/2608.03071#A2.SS4)\.

Table 10:Probe\-guided reranking applied to instruction\-tuned models that received no fine\-tuning, on the 2 datasets of Figure[5](https://arxiv.org/html/2608.03071#S6.F5)\. Every number follows the 5\-fold selection protocol of Section[4\.3](https://arxiv.org/html/2608.03071#S4.SS3); repeated runs of a cell share one greedy decode and are averaged\. Gemma\-4\-12B is excluded by a prompt\-template defect; base variants are excluded because their greedy decodes in this campaign’s pools are frequently unparseable, turning reranking into format repair rather than selection\.Table 11:Probe\-guided reranking applied on top of PBT, on the 2 datasets of Figure[5](https://arxiv.org/html/2608.03071#S6.F5)and under the protocol of Table[10](https://arxiv.org/html/2608.03071#A4.T10)\. Arms differ in training seed, sampling temperature, pool size, or archived pool; a dash marks the default arm\. Both archived pools of the Qwen3\-14B\-BaseParamBenchcell are listed; a dash in the last column marks an arm whose pool was not retained, so the winning family could not be recomputed\. Greedy values are recomputed from the archived pools\.
### D\.4Results by Difficulty Level

The per\-level results of Figure[5](https://arxiv.org/html/2608.03071#S6.F5)rest on uneven sample sizes: theParamBenchtest split has 20/72/64/85/52 instances at L1 to L5, and the API\-Bank evaluation set has 40/73/54/7/52\. The two datasets place their headroom differently\. OnParamBenchthe seed model already solves L1 at 85 EM and fails L5 completely, so the gains of PBT\+PGR concentrate on the hard levels, as the \+11, \+5, and \+12 labels of Figure[5](https://arxiv.org/html/2608.03071#S6.F5)show for L3, L4, and L5\. On API\-Bank the first four levels are close to saturated, between 81 and 88 EM, and the remaining headroom sits at L5, where the gain is \+13\. Two reading notes apply\. The API\-Bank L4 cell rests on only 7 instances and is best read together with its neighbors, and the L1 cells of both datasets are small as well, so the flat L1 bars mean saturation rather than failure of the methods\. The middle levels also split: L3 gains 11 points while L2 gains only 3, although both start far from the ceiling\. By the grading rule of Section[5](https://arxiv.org/html/2608.03071#S5), an L2 call is at most one level deep and carries no conditional dependencies, so the structural patterns that the filtered training data teaches have less room to help there\. The overall picture matches the main text: self\-training with a probe filter does not manufacture ability on levels the seed model never solves alone, but where a level is within reach, the filtered data and the reranker convert unstable successes into stable ones\.

#### Released artifact\.

The package contains the material behind this appendix: the benchmark instances, the 81 frozen API schemas, the probe, PBT, and PGR implementations of Section[4](https://arxiv.org/html/2608.03071#S4), the per\-call adaptation and evaluation pipeline, and the scripts that regenerate the figures and appendix tables\. The aggregate scores behind Figures 3 to 5 ship with it, so those figures regenerate without a GPU, and an offline replay backend runs the evaluation without network access\. A configuration example and the exact dependency list are included; all entry points are Python modules, with unit tests covering split construction, adaptation, and training arms\.