Towards Robust Tool Use in Agents via Experience-Driven Adaptive Guidance

# Towards Robust Tool Use in Agents via Experience-Driven Adaptive Guidance
Source: [https://arxiv.org/html/2608.03403](https://arxiv.org/html/2608.03403)
Can Wang1,2, Haoran Chen2, Li Yu2, Ding Hao3, Bohai Zhao1, Zhaoyang Liu2, Zhiying Tu1 1Shandong Key Laboratory of Digital Service Computing Technology and Systems 2Alibaba Token Hub 3Department of Electrical and Electronic Engineering, The Hong Kong Polytechnic University

###### Abstract

The performance bottleneck of agents is increasingly shifting from model capability to the robustness of their execution processes\. Tools play a central role as the primary interface through which agents interact with external environments, yet existing methods rarely focus on ensuring robust tool use across diverse runtime conditions\. To address this problem, we proposeExpG, a mechanism that builds and refines adaptive guidance capturing each tool’s capability boundaries and best practices, thereby enabling agents to use tools more robustly and effectively\.ExpGconsists of three phases: \(1\) experience acquisition, which analyzes tool invocation quality from historical execution trajectories, producing structured learnable experiences through multi\-aspect attribution; \(2\) experience distillation, which keeps the experience pool effective by filtering unhelpful experiences, selecting representative ones with an equivalence\-class\-based method, and summarizing them into generalizable guidance; and \(3\) experience reuse, which applies the guidance adaptively during future task solving\. Extensive experiments show thatExpGbrings consistent improvements across the tool selection, tool calling, and response generation tasks, enabling smaller agents to outperform larger ones that do not useExpG\. Moreover,ExpGachieves particularly strong gains in challenging settings, suggesting a promising path toward more robust tool use\. Our code, experiments, and results are available at[https://github\.com/WangCan1178/ExpG](https://github.com/WangCan1178/ExpG)\.

## 1Introduction

The performance bottleneck of agents is shifting from the capabilities of large language models \(LLMs\) to the robustness of their execution processes\. Agent harnessesZhouet al\.\([2026](https://arxiv.org/html/2608.03403#bib.bib6)\); Panet al\.\([2026](https://arxiv.org/html/2608.03403#bib.bib7)\); Leeet al\.\([2026](https://arxiv.org/html/2608.03403#bib.bib1)\)have therefore been proposed to provide runtime control that enables behavioral governance and fault tolerance, allowing agents to remain stable when performing complex tasks in dynamic environments\. Within such harnesses, tools serve as the primary interface through which agents interact with external environmentsZhanget al\.\([2024](https://arxiv.org/html/2608.03403#bib.bib8)\); Wuet al\.\([2025](https://arxiv.org/html/2608.03403#bib.bib14)\), and prior studiesMenget al\.\([2026](https://arxiv.org/html/2608.03403#bib.bib34)\)show that even minor improvements in tool\-use robustness can yield performance gains ranging from 6\.7% to 68\.3%\.

However, existing tool\-use methodsQuet al\.\([2025b](https://arxiv.org/html/2608.03403#bib.bib16)\)rarely consider robustness across diverse runtime environments, as they typically developed under conditions where tool\-use behavior remains predictable\. Concretely, it implies that \(1\) tools the agent has already mastered can always be invoked correctly and return expected responses, and \(2\) even when tools are invoked incorrectly, their failures are accompanied by informative signals, enabling the agent to continuously self\-reflect and eventually learn to use the tools correctly\. Unfortunately, real\-world conditions break these assumptions\. On the one hand, the quality of tool invocations is strongly affected by environmental dynamics\(Chenet al\.,[2025](https://arxiv.org/html/2608.03403#bib.bib18); Chenget al\.,[2025](https://arxiv.org/html/2608.03403#bib.bib17)\)\. Even for the same tool, the agent’s tool\-use behavior can change with the task context, and may also drift over time or across different usage environments\. This is particularly evident when there is a gap between development and production: many tools that work well in the original setting become unreliable or even unusable because their previous usage patterns no longer apply\(Guoet al\.,[2024](https://arxiv.org/html/2608.03403#bib.bib20)\)\. On the other hand, the responses from tool invocations are often noisy or coarse\-grainedSunet al\.\([2024](https://arxiv.org/html/2608.03403#bib.bib19)\)\. For instance, a buggy calculator may return an incorrect numerical result while still reporting the invocation as “successful”\. Figure[1](https://arxiv.org/html/2608.03403#S1.F1)\(a\) illustrates examples of these two challenges\. Without a clear understanding of a tool’s capability boundaries and how to use it robustly under varying conditions, agents struggle to generalize tool use reliably across diverse environments and often fall back on trial and error\. Once a mistake occurs, the agent may receive only ambiguous signals, which can lead to an increasingly complicated and often incorrect inference path\.

![Refer to caption](https://arxiv.org/html/2608.03403v1/x1.png)Figure 1:\(a\) Two real\-world challenges: the agent uses the same tool but gets different results due to the different context; there are different reasons for the error, but the tools respond with the same coarse\-grained signal \(both return just “error”\)\. \(b\) Guidance learned from past tool‑use experience\. \(c\) Results across different stages of tool usage demonstrate thatExpGimproves agent tool\-use performance from multiple aspects\.To address these challenges, we proposeExpG, a mechanism that producesExperience\-Driven adaptiveGuidance to help agents use tools effectively in the presence of environmental dynamics and noisy feedback\. In contrast to existing approaches that rely on static assumptions and treat tool invocations as black\-box, isolated, and repeatable events, our work views tool invocations as analyzable, interdependent, and learnable experiences\. By continuously acquiring, distilling, and reusing these experiences,ExpGmaintains an evolving experience pool inspired by agent memoryHuet al\.\([2025](https://arxiv.org/html/2608.03403#bib.bib32)\), which enables it to adaptively build and refine guidance for tool use\. Specifically, in the first phase,ExpGacquires experiences by analyzing agents’ execution trajectories and attributing each tool invocation’s success or failure to contributing factors\. Experiences related to the same tool are categorized into different invocation patterns through equivalence\-class partitioning, resulting in an initial structured experience pool\. A filtering method is then applied to maintain the quality of the experience pool by removing unhelpful experiences and selecting helpful ones, thereby retaining representative experiences covering as many invocation patterns as possible while preserving the original distribution\. With these filtered experiences,ExpGperforms qualitative behavior analysis alongside quantitative performance analysis to construct guidance for each tool, organizing fragmented invocation\-level experiences into generalizable tool\-level guidance\. The guidance characterizes the capability boundaries and best practices of tools, and can be adaptively refined as the experience pool evolves\. Finally, the guidance is reused in subsequent tasks whenever the agent invokes the same tool, either as dynamic contextual prompts or as stable schema constraints, thereby improving task performance in complex and changing real\-world applications through more robust tool use\.

In summary, our contributions are as follows:

- •To overcome unpredictable tool\-use behaviors caused by environmental dynamics and noisy feedback, we break out of the static view of tool invocation by introducing a structured characterization of tool invocations, thereby acquiring learnable experiences through equivalence‑class partitioning that encapsulate comprehensive information about the environment, the agent, and the tool itself\.
- •We propose a mechanismExpGto maintain an evolving experience pool at the tool level\. By acquiring, distilling, and reusing past experiences,ExpGbuilds capability boundaries and best practices for tools, which serve as practical guidance and can be continually refined as the agent becomes more effective in tool usage\.
- •Extensive experiments across different stages of tool use validate the effectiveness of our approach across a range of models, showing that small models equipped withExpGcan outperform larger ones without it\. Notably,ExpGremains robust under challenging settings, such as noisy environments with distractor tools or imperfect information\.

## 2Related Works

Tool\-augmented LLMs\.Tool\-augmented LLMs substantially enhance domain expertiseZhanget al\.\([2024](https://arxiv.org/html/2608.03403#bib.bib8)\); Liaoet al\.\([2025](https://arxiv.org/html/2608.03403#bib.bib11)\), knowledge acquisitionWuet al\.\([2025](https://arxiv.org/html/2608.03403#bib.bib14)\); Songet al\.\([2025](https://arxiv.org/html/2608.03403#bib.bib15)\), and interaction qualityZhanget al\.\([2025](https://arxiv.org/html/2608.03403#bib.bib12)\); Wanget al\.\([2024](https://arxiv.org/html/2608.03403#bib.bib13)\), and are seen as a key step in the evolution from pure language models toward intelligent agents\. Existing methods for improving tool use can be divided into two categories based on whether model parameters are explicitly updatedQuet al\.\([2025b](https://arxiv.org/html/2608.03403#bib.bib16)\)\. Tuning\-based methods update model parameters to internalize tool\-use policies\. They typically rely on supervised learningQinet al\.\([2024](https://arxiv.org/html/2608.03403#bib.bib21)\); Patilet al\.\([2024](https://arxiv.org/html/2608.03403#bib.bib22)\)or reinforcement learningZhouet al\.\([2025](https://arxiv.org/html/2608.03403#bib.bib23)\); Qianet al\.\([2025](https://arxiv.org/html/2608.03403#bib.bib24)\), using pre\-collected data and tool\-execution feedback to train LLMs\. In contrast, tuning\-free methods keep model parameters fixed and instead improve tool use by guiding the LLM during inference, for example through carefully designed promptsYuanet al\.\([2025](https://arxiv.org/html/2608.03403#bib.bib25)\), chain\-of\-thought promptingChenet al\.\([2023](https://arxiv.org/html/2608.03403#bib.bib26)\), and the ReAct paradigmYaoet al\.\([2023](https://arxiv.org/html/2608.03403#bib.bib27)\)\. Recent studies have highlighted robustness challenges of tool\-augmented LLMs in real\-world settingsGuoet al\.\([2024](https://arxiv.org/html/2608.03403#bib.bib20)\); Sunet al\.\([2024](https://arxiv.org/html/2608.03403#bib.bib19)\); Chenet al\.\([2025](https://arxiv.org/html/2608.03403#bib.bib18)\); Chenget al\.\([2025](https://arxiv.org/html/2608.03403#bib.bib17)\)\. Building on this observation, we construct guidance that can be continually refined from structured experiences, enabling tool\-augmented agents to cope better with unpredictable environments\.

Harness Engineering\.As LLM become increasingly capable, research attention has gradually shifted from improving model weights to engineering the surrounding agent infrastructure that enables reliable deploymentXuet al\.\([2024](https://arxiv.org/html/2608.03403#bib.bib28)\); Huanget al\.\([2024a](https://arxiv.org/html/2608.03403#bib.bib29)\)\. In particular, harness engineeringZhouet al\.\([2026](https://arxiv.org/html/2608.03403#bib.bib6)\); Panet al\.\([2026](https://arxiv.org/html/2608.03403#bib.bib7)\)has emerged as a key direction for improving the robustness of LLM agents\. The execution harnessMenget al\.\([2026](https://arxiv.org/html/2608.03403#bib.bib34)\); Leeet al\.\([2026](https://arxiv.org/html/2608.03403#bib.bib1)\)governs how an agent interacts with its environment by managing the execution loop, tool access, context construction, state persistence, and runtime monitoring\. Harness optimization can be achieved through multiple external components such as memory storesHuet al\.\([2025](https://arxiv.org/html/2608.03403#bib.bib32)\); Chhikaraet al\.\([2025](https://arxiv.org/html/2608.03403#bib.bib41)\), reusable skillsLinget al\.\([2026](https://arxiv.org/html/2608.03403#bib.bib2)\); Chenet al\.\([2026](https://arxiv.org/html/2608.03403#bib.bib3)\), and context engineeringZhanget al\.\([2026](https://arxiv.org/html/2608.03403#bib.bib4)\); Yeet al\.\([2026](https://arxiv.org/html/2608.03403#bib.bib5)\), which respectively improve information persistence, procedural reuse, and structured task execution during runtime\. Most existing work improves robustness by externalizing capabilities at the task or workflow level within the harness\. In contrast,ExpGoperates at the fine\-grained granularity of the tool, externalizing the limits per tool capability and best practices that are complementary to methods based on higher level memory or skills\.

![Refer to caption](https://arxiv.org/html/2608.03403v1/x2.png)Figure 2:Overview ofExpG\. It consists of three phases: acquiring tool invocation experiences from past trajectories, distilling them into guidance, and reusing them to improve future task performance\. These phases form a mechanism that builds capability boundaries and best practices for each tool\.
## 3Methodology

### 3\.1Overview ofExpG

Our work,ExpG, as illustrated in Figure[2](https://arxiv.org/html/2608.03403#S2.F2), operates through three sequential phases: experience acquisition, distillation, and reuse\. In theexperience acquisitionphase, an evaluator attributes the success or failure of each tool invocation from historical trajectories, acquiring structured and learnable experiences\. In theexperience distillationphase, experiences are first filtered to ensure they are helpful, then summarized into guidance that captures the tool’s capability boundaries and best practices\. Finally, theexperience reusephase injects these distilled guidance into the agent’s inference process, either as part of prompts or tool schemas, to enable more robust and effective tool invocation\. These phases work together to maintain an evolving experience pool for the tool, which can be continuously updated and refined through ongoing experiences, driving sustained improvements in the agent’s performance\.

### 3\.2Experience Acquisition

We begin with formalizing a tool invocation experience asE=⟨h,m,c⟩E=\\langle h,m,c\\rangle, wherehhis the hash\-based unique identifier of the experience\. The metadatammcaptures tool invocation details, including tool information \(name and schema\), usage information \(agent context and tool input\), and execution information \(tool response, success flag, time cost, and token cost\)\. The experience contentccis produced by an agentLLMevaluator\\mathrm\{LLM\}\_\{\\mathrm\{evaluator\}\}that takesmmas input and consists of: 1\)s​c​o​r​e​s=\[s1,…,sd\]scores=\[s\_\{1\},\\ldots,s\_\{d\}\], add\-dimensional binary vector withsi∈\{0,1\}s\_\{i\}\\in\\\{0,1\\\}that characterizes different aspects of invocation quality, and 2\)e​x​p​l​a​n​a​t​i​o​nexplanation, a short natural language explanation of thes​c​o​r​e​sscoresthat explains why the tool invocation succeeded or failed\.

To acquire as many valuable experiences as possible for improving agents’ tool use capability,ExpGfirst collects trajectories in which an agent solves diverse tasks using different tools\. For each agent\-tool interaction in these trajectories, metadatammis extracted, with a hash valuehhcomputed to distinguish different invocations of the same tool under different tool context, input, or response\. An agentLLMevaluator\\mathrm\{LLM\}\_\{\\mathrm\{evaluator\}\}then evaluates each tool invocation to produce the experience contentcc\. To ensure accurate and reliable evaluation, our LLM\-as\-a\-Judge\(Zhenget al\.,[2023](https://arxiv.org/html/2608.03403#bib.bib43)\)method guidesLLMevaluator\\mathrm\{LLM\}\_\{\\mathrm\{evaluator\}\}to answer a series of simple yes\-or\-no questions, such as “Are the input parameters in the correct format for their respective fields?”\. The evaluation can be detailed intoddaspects, and the answers to theddbinary questions are encoded as add\-dimensional vectors​c​o​r​e​sscores\. By providing a natural\-language explanation for each dimension, denoted ase​x​p​l​a​n​a​t​i​o​nexplanation,LLMevaluator\\mathrm\{LLM\}\_\{\\mathrm\{evaluator\}\}can attribute the reasons for a tool invocation’s success or failure\. A generic evaluation we designed for tool invocation covers 10 aspects \(the prompt template is shown in Appendix[A](https://arxiv.org/html/2608.03403#A1)\), and the corresponding explanation provides comprehensive information by grouping these aspects from three perspectives: \(1\)tool\-use environment, which assesses whether the tool is selected appropriately and invoked at the right time given the context; \(2\)agent behavior, which assesses whether the agent invokes the tool with an input that is correct both syntactically and semantically; and \(3\)tool reliability, which assesses whether the tool executes stably and returns a response as expected\. All experiences acquired are indexed by tool name and stored into a vector database, forming an initial structured experience pool for the tool\. With the notion of equivalence classesDevlin \([2017](https://arxiv.org/html/2608.03403#bib.bib10)\),ExpGnaturally utilizes the output of the evaluation and partitions the experience poolℰτ=\{Ei\}i=1Nτ\\mathcal\{E\}\_\{\\tau\}=\\\{E\_\{i\}\\\}\_\{i=1\}^\{N\_\{\\tau\}\}of the same toolτ\\tau, intoKKequivalence classes based on theirs​c​o​r​e​sscores, as shown in Eq\.[1](https://arxiv.org/html/2608.03403#S3.E1):

κ​\(E\)\\displaystyle\\kappa\(E\)≜s​c​o​r​e​s​\(E\)∈\{0,1\}d,\\displaystyle\\triangleq scores\(E\)\\in\\\{0,1\\\}^\{d\},\(1\)\{𝒞k\}k=1K\\displaystyle\\\{\\mathcal\{C\}\_\{k\}\\\}\_\{k=1\}^\{K\}←\{κ−1​\(v\):v∈κ​\(ℰτ\)\}\.\\displaystyle\\leftarrow\\\{\\kappa^\{\-1\}\(v\):v\\in\\kappa\(\\mathcal\{E\}\_\{\\tau\}\)\\\}\.Each equivalence class corresponds to a distinct pattern of tool invocation\. For example, the perfect invocation pattern can be denoted as class𝒞perfect≜κ−1​\(\[1,1,…,1\]\)\\mathcal\{C\}\_\{\\mathrm\{perfect\}\}\\triangleq\\kappa^\{\-1\}\\\!\\left\(\[1,1,\\ldots,1\]\\right\), which indicates that the tool is invoked in an appropriate context with correct inputs and returns the expected response\.

Input:Equivalence classes\{𝒞k\}k=1K\\\{\\mathcal\{C\}\_\{k\}\\\}\_\{k=1\}^\{K\}of toolτ\\tauderived fromℰτ\\mathcal\{E\}\_\{\\tau\}; Target selection quotaQQ

Output:Selected subset

𝒮τ\\mathcal\{S\}\_\{\\tau\}
// Select one from each of theQQlargest classes when the number of classes exceeds the quota

if*K\>QK\>Q*then

\(𝒞\(1\),…,𝒞\(K\)\)←Sortdesc\(\{𝒞k\}k=1K,\|⋅\|\)\(\\mathcal\{C\}\_\{\(1\)\},\\ldots,\\mathcal\{C\}\_\{\(K\)\}\)\\leftarrow\\textsc\{Sort\}\_\{\\mathrm\{desc\}\}\(\\\{\\mathcal\{C\}\_\{k\}\\\}\_\{k=1\}^\{K\},\\,\|\\cdot\|\)
𝒮τ←∅\\mathcal\{S\}\_\{\\tau\}\\leftarrow\\emptyset
for*k←1k\\leftarrow 1toQQ*do

E←Sample​\(𝒞\(k\),1\)E\\leftarrow\\textsc\{Sample\}\(\\mathcal\{C\}\_\{\(k\)\},1\),

𝒮τ←𝒮τ∪\{E\}\\mathcal\{S\}\_\{\\tau\}\\leftarrow\\mathcal\{S\}\_\{\\tau\}\\cup\\\{E\\\}
else

// Coarse\-grained allocation

for*k←1k\\leftarrow 1toKK*do

ak←1a\_\{k\}\\leftarrow 1,

nk←\|𝒞k\|n\_\{k\}\\leftarrow\|\\mathcal\{C\}\_\{k\}\|,

pk←nk/∑j=1Knjp\_\{k\}\\leftarrow n\_\{k\}/\\sum\_\{j=1\}^\{K\}n\_\{j\}
ak←ak\+⌊\(Q−K\)⋅pk⌋a\_\{k\}\\leftarrow a\_\{k\}\+\\lfloor\(Q\-K\)\\cdot p\_\{k\}\\rfloor
if*ak\>nka\_\{k\}\>n\_\{k\}*then

ak←nka\_\{k\}\\leftarrow n\_\{k\}
// Fine\-grained adjustment

σ←Argsortdesc​\(\{pk\}k=1K\)\\sigma\\leftarrow\\textsc\{Argsort\}\_\{\\mathrm\{desc\}\}\(\\\{p\_\{k\}\\\}\_\{k=1\}^\{K\}\),

L←Q−∑k=1KakL\\leftarrow Q\-\\sum\_\{k=1\}^\{K\}a\_\{k\}
while*L\>0L\>0and∃k:ak<nk\\exists k:a\_\{k\}<n\_\{k\}*do

for*t←1t\\leftarrow 1toKK*do

k←σ​\(t\)k\\leftarrow\\sigma\(t\)
if*ak<nka\_\{k\}<n\_\{k\}andL\>0L\>0*then

ak←ak\+1a\_\{k\}\\leftarrow a\_\{k\}\+1,

L←L−1L\\leftarrow L\-1
// In\-class sampling

𝒮τ←∅\\mathcal\{S\}\_\{\\tau\}\\leftarrow\\emptyset
for*k←1k\\leftarrow 1toKK*do

𝒮τ←𝒮τ∪Sample​\(𝒞k,ak\)\\mathcal\{S\}\_\{\\tau\}\\leftarrow\\mathcal\{S\}\_\{\\tau\}\\cup\\textsc\{Sample\}\(\\mathcal\{C\}\_\{k\},a\_\{k\}\)
return

𝒮τ\\mathcal\{S\}\_\{\\tau\}

Algorithm 1Equivalence\-Class\-Based Experience Selection Algorithm for Many\-shot Tool
### 3\.3Experience Distillation

Equipped with rich experiences, the goal of the second phase inExpGis to distill various invocation\-level experiences into unified tool\-level guidance, which can be flexibly generalized to diverse future tasks and environments\. Concretely, this phase consists of two steps:experience filterandexperience summary\. The former keeps the experience pool up\-to\-date and high\-quality by removing unhelpful experiences and selecting helpful ones, while the latter analyzes and summarizes the retained experiences, deriving insightful and practical guidance that characterizes both qualitative tool\-use behavior and quantitative tool\-use performance\.

In the experience filtering step,ExpGremoves redundant tool invocation experiences for each tool based on their hash values\. It then sorts the experiences by their creation time \(which is correlated with the actual invocation time\) and discards outdated ones to ensure the timeliness of the experience pool\. Then, we perform an effectiveness check with a sliding time window to remove bad experiences that may introduce negative effects\. For a given timestamp, if the averages​c​o​r​e​sscoresof the next three consecutive timestamps are consistently lower than that of the current timestamp, all invocation experiences associated with the current timestamp that are used for distilling guidance are removed\. After removing unhelpful experiences,ExpGimplements different selection strategies by categorizing tools into three groups according to the number of valid experiences: 1\)one\-shot tool: to avoid biased guidance summarized from only a single invocation, no experience is selected and the subsequent summary step is skipped for the tool; 2\)few\-shot tool: all retained experiences are used to summarize comprehensive guidance for the tool; and 3\)many\-shot tool: a designed equivalence\-class\-based selection method is applied to select at mostQQrepresentative experiences, whereQQis computed according to the available context length\. The procedure is described in Algorithm[1](https://arxiv.org/html/2608.03403#algorithm1)and analyzed in depth in Appendix[B](https://arxiv.org/html/2608.03403#A2)\. The method promotes diversity while preserving the original score distribution, thereby ensuring that the selected experiences are representative by covering as many invocation patterns as possible while emphasizing the most frequently observed ones\.

After experience filtering is completed,ExpGconducts qualitative behavior analysis alongside quantitative performance analysis over the retained experiences, producing summarized guidance\. The qualitative analysis reflects how the tool should be used\. Prompted with chain\-of\-thought reasoning \(see Appendix[A](https://arxiv.org/html/2608.03403#A1)\), an agentLLMsummarizer\\mathrm\{LLM\}\_\{\\mathrm\{summarizer\}\}can summarize the usage behavior for the tool from four aspects: core function, success patterns, common issues, and best practices\. The quantitative analysis reflects the tool’s performance and is based on statistics over historical experiences, including the invocations’ average success rate, average score, average time cost, and average token cost\.

### 3\.4Experience Reuse

The final phase ofExpGis experience reuse, which aims to apply the guidance to new task scenarios, enabling the agent to shift from blind trial\-and\-error tool use to informed guided tool use\. Instead of treating all distilled guidance equally,ExpGcategorizes it into dynamic and stable types based on the consistency of invocation patterns, and applies different usage strategies accordingly\.

SinceExpGis an adaptive mechanism and the experience pool can be updated over time and across environments, dynamic guidance is by default used as lightweight contextual prompts to guide the agent to invoke tools appropriately\. In contrast, stable guidance corresponds to reliable experiences that have been extensively validated\. In this case, the distilled guidance is combined into the tool schema, constraining agent behavior more directly and strictly\. The categorization criterion is given in Eq\.[2](https://arxiv.org/html/2608.03403#S3.E2), whereC∗C^\{\*\}denotes the largest equivalence class among all classes except the perfect invocation class𝒞perfect\\mathcal\{C\}\_\{\\mathrm\{perfect\}\}\(perfect tool use does not require stricter constraints\), andα\\alphais a threshold that controls the strictness of the criterion\. Specifically, when the number of experiences reaches half of the selection quota \(Q/2\{Q\}/\{2\}\) and most experiences belong to a single non\-perfect class, the criterion functionffis triggered and the guidance is classified as stable\. Andα\\alphais recommended to be set asK/QK/Q, which comes from the mixing coefficient in the experience selection strategies as analyzed in the Appendix[B](https://arxiv.org/html/2608.03403#A2), capturing the trade\-off between diversity coverage and distribution preservation\. Using this threshold ensures that stable guidance is triggered only when a high\-frequency and consistent equivalence class dominates\. The stable guidance covers situations in which a tool exhibits the consistent failure pattern across different invocations\. For example, a typical stable guidance may reflect tool usability issues under our generic evaluation\. That is, even when the tool is invoked in an appropriate context with correct inputs, its response can still deviate substantially from expectations, causing the equivalence class𝒞≜κ−1​\(\[1,1,1,1,1,1,0,0,0,0\]\)\\mathcal\{C\}\\triangleq\\kappa^\{\-1\}\\\!\\left\(\[1,1,1,1,1,1,0,0,0,0\]\\right\)to dominate\.

Type​\(g​u​i​d​a​n​c​eτ\)=\{stable,if​f​\(ℰτ\)dynamic,otherwisef​\(ℰτ\)≜\(\|ℰτ\|≥Q2\)∧\(\|𝒞∗\|≥α​\|ℰτ\|\)\.\\begin\{gathered\}\\textsc\{Type\}\(guidance\_\{\\tau\}\)=\\begin\{cases\}\\text\{stable\},&\\text\{if \}f\(\\mathcal\{E\}\_\{\\tau\}\)\\\\ \\text\{dynamic\},&\\text\{otherwise\}\\end\{cases\}\\\\\[4\.0pt\] f\(\\mathcal\{E\}\_\{\\tau\}\)\\triangleq\\Big\(\|\\mathcal\{E\}\_\{\\tau\}\|\\geq\\frac\{Q\}\{2\}\\Big\)\\ \\wedge\\ \(\|\\mathcal\{C\}^\{\*\}\|\\geq\\alpha\\,\|\\mathcal\{E\}\_\{\\tau\}\|\)\.\\end\{gathered\}\(2\)
Overall, experience reuse leverages distilled guidance to steer tool invocation in new scenarios\. The different usage strategy allowsExpGto choose an appropriate way to apply guidance based on its reliability and generalizability\. It keeps dynamic guidance flexible to adapt to changing environments, while ensuring the validated stable guidance provides consistent constraints across tasks\.

## 4Experiments

### 4\.1Experimental Settings

##### Baselines\.

To evaluate the effectiveness ofExpG, we compare it with: \(1\) No Method, and three different types of baseline methods improve the agent’s tool use by learning from prior experience \(2\) Few\-shot learningSonget al\.\([2023](https://arxiv.org/html/2608.03403#bib.bib31)\), a prompting method that provides a small number of tool\-use examples in the prompt to let the model learn from them, \(3\) DRAFTQuet al\.\([2025a](https://arxiv.org/html/2608.03403#bib.bib42)\), a method that dynamically improves the quality of tool documents to enhance agents’ understanding and use of tools, and \(4\) Mem0\(Chhikaraet al\.,[2025](https://arxiv.org/html/2608.03403#bib.bib41)\), a method which builds conversational memory for an agent by extracting key information from its interactions with the external world\. These three methods provide guidance for tool\-use from different aspects, with further details provided in Appendix[C](https://arxiv.org/html/2608.03403#A3)\.

##### Datasets and Metrics\.

The task\-solving process of tool\-augmented LLMs is often decomposed into four stages: task planning, tool selection, tool calling, and response generationQuet al\.\([2025b](https://arxiv.org/html/2608.03403#bib.bib16)\)\. To thoroughly evaluate the effectiveness ofExpG, we conduct experiments on three benchmark datasets, each targeting different stages of tool usage: MetaToolHuanget al\.\([2024b](https://arxiv.org/html/2608.03403#bib.bib39)\), API\-BankLiet al\.\([2023](https://arxiv.org/html/2608.03403#bib.bib36)\), BFCL\-V3\(Patilet al\.,[2025](https://arxiv.org/html/2608.03403#bib.bib44)\)\. For MetaTool, we select the 100 most frequently used tools and sample up to 10 tasks for each tool under each sub\-task setting, resulting in 1,669 tasks in total\. For API\-Bank, we use the full lv1 dataset, comprising 399 tasks\. Both datasets are evaluated using leave\-one\-out cross\-validation\. For BFCL\-V3, since some tasks involve tools that are never reused in other tasks, we filter out these cases and retain 461 tasks for the test set and 1542 tasks for the training set\. We use both Avg@3 and Pass@3 as evaluation metrics\. Avg@3 denotes the average task success rate over three independent runs, while Pass@3 measures the probability that at least one of the three runs is successful\. As different datasets target different stages of tool usage, we follow their original evaluation criteria when defining “successful” to maintain consistency\. See Appendix[D](https://arxiv.org/html/2608.03403#A4)for more dataset and metric details\.

##### Implementation Details\.

We conduct experiments on GPT\-5 nanoOpenAI \([2025](https://arxiv.org/html/2608.03403#bib.bib48)\), Deepseek\-V3DeepSeek\-AI \([2025](https://arxiv.org/html/2608.03403#bib.bib47)\), and the Qwen3 series models\(Qwen\-Team,[2025](https://arxiv.org/html/2608.03403#bib.bib45)\)to demonstrate the effectiveness ofExpGacross different model architectures and scales\. We use prompt\-based tool invocation method and sett​e​m​p​e​r​a​t​u​r​e=0\.5temperature=0\.5for trajectory sampling\. During experience acquisition and experience distillation, we adopt Qwen3\-max as theLLMevaluator\\mathrm\{LLM\}\_\{\\mathrm\{evaluator\}\}andLLMsummarizer\\mathrm\{LLM\}\_\{\\mathrm\{summarizer\}\}to produce reliable tool invocation experiences and distilled guidance\. During experience reuse, we setα=0\.8\\alpha=0\.8to control the criterion in our different usage strategy of guidance\. In addition, to support our design of removing unhelpful experiences over time, we execute tasks sequentially\. For fair comparison, we keep these settings fixed unless otherwise specified in ablation studies\. More implementation details can be found in the Appendix[E](https://arxiv.org/html/2608.03403#A5)\.

Table 1:Performance comparison \(%\) across MetaTool, API\-Bank, and BFCL\-V3\.Boldindicates the best results within each model\.

### 4\.2Main Results

Table[1](https://arxiv.org/html/2608.03403#S4.T1)reports the main results ofExpGacross models on three datasets that target different phases of tool invocation\. Overall,ExpGachieves the best Avg@3 and Pass@3 on every dataset, consistently outperforming all competing methods\. For Qwen3\-8B,ExpGimproves the total metric by7\.41↑\\uparrowin Avg@3 and6\.92↑\\uparrowin Pass@3 over the “No Method” setting\. We also report Pass@1, which highlights clearer distinctions between methods under lower inference budgets\. These gains persist as model size increases, demonstrating the generality of our approach\. Importantly,ExpGallows smaller models to match or even surpass larger models in the “No Method” setting, indicating that our approach helps mitigate the tool\-use capability gap between weaker and stronger models\.

Beyond the overall performance, we further conduct a detailed analysis ofExpGon each dataset\. Figure[4](https://arxiv.org/html/2608.03403#S4.F4)presents the results on MetaTool and BFCL\-V3 across different sub\-task settings\. In tool selection \(MetaTool\) tasks,ExpGis particularly effective \(16\.34↑\\uparrow\) when faced with distractor tools that are very similar\. In tool calling \(API\-Bank\) tasks, we observe that different methods achieve comparable scores\. This is likely because the dataset is relatively simplistic, allowing all approaches to bring model performance close to the upper limit of their inherent capabilities\. In response generation tasks \(BFCL\-V3\),ExpGdemonstrates strong performance \(13\.09↑\\uparrowin single\-turn tasks and10\.33↑\\uparrowin multi\-turn tasks\) on noisy settings, such as missing parameters, undefined tools or irrelevant tool choices\. This indicates thatExpGhelps the agent generalize tool use effectively in unpredictable or even problematic environments\. Besides, we observe an interesting phenomenon that our method reduces the average inference steps in normal multi\-turn tasks from 10\.14 to 9\.02, but increases the step count by 2\.4 in problematic multi\-turn tasks\. This indicates thatExpGenables the agent to achieve higher success rates with fewer trials under normal settings, reaching correct solutions with fewer unnecessary interactions\. At the same time, it encourages more deliberate behaviors in challenging scenarios, prompting the agent to interact with the environment to ensure more reliable tool use\. Such a shift toward a more flexible tool\-use strategy is desirable, as it suggests that the agent becomes more robust across diverse situations under our guidance\.

![Refer to caption](https://arxiv.org/html/2608.03403v1/latex/figures/output.png)Figure 3:Avg@3 performance \(%\) on MetaTool and BFCL\-V3 across sub\-task settings\.
![Refer to caption](https://arxiv.org/html/2608.03403v1/latex/figures/ablation.png)Figure 4:Ablation results \(Avg@3, %\) of different phases inExpG\.

### 4\.3Ablation Studies

As shown in Figure[4](https://arxiv.org/html/2608.03403#S4.F4), we conduct an ablation study of the key components ofExpGusing Qwen3\-8B as the backbone model\. Compared with the baseline without any method, progressively adding each phase yields consistent improvements\. First, only usingexperience acquisitionas guidance during inference achieves better performance than few\-shot learning, indicating that structured experiences acquired from task interactions provide more effective signals\. Second, addingexperience distillationresults in a substantial additional gain, highlighting the importance of maintaining a high\-quality experience pool and aggregating multiple experiences\. Finally, although the additional improvement after addingexperience reuseis milder than that of the first two phases, it is crucial for driving long\-term evolution of the agent and tool quality in real applications\.

Table 2:Effect of the number of acquired experiences on MetaTool performance \(Avg@3, %\) inExpG\.Δ\\Deltadenotes the absolute change compared to the previous setting\.Additionally, to study how the number of experiences affectsExpG, we conduct experiments on MetaTool, selecting tools with sufficiently diverse invocations\. Table[2](https://arxiv.org/html/2608.03403#S4.T2)shows thatExpGimproves quickly once the number of experiences for a tool meets our few\-shot tool criterion\. As more experiences are acquired, performance peaks at around 9 experiences, after which adding additional experiences introduces fluctuations\. With further increases in experience,ExpGstabilizes and maintains strong performance, as the experience filtering phase continuously removes unhelpful experiences while keeping the most helpful ones\.

### 4\.4More Analysis

Table 3:Impact of agent size on BFCL\-V3 performance \(%\) inExpG\.△\\triangleindicates the change relative to the original Qwen3\-max setting\.
Table 4:Self\-evolvingExpGwith Qwen3\-8B \(%\)\.△\\triangleindicates the change relative to No Method\.

![Refer to caption](https://arxiv.org/html/2608.03403v1/latex/figures/sampling_comparison_3d_d_q.png)Figure 5:Distribution alignment and diversity under different selection methods\.##### Agent Capabilities\.

SinceExpGrelies on the LLM\-as\-judge method to acquire experiences and summarize guidance effectively, we examine the impact of agent size for both roles,LLMevaluator\{\\text\{LLM\}\}\_\{\\text\{evaluator\}\}andLLMsummarizer\{\\text\{LLM\}\}\_\{\\text\{summarizer\}\}\. The experiments are conducted on BFCL\-V3, and the results are shown in Table[4](https://arxiv.org/html/2608.03403#S4.T4)\. Replacing either agent with a smaller model degrades the final task performance, with a more pronounced drop observed forLLMevaluator\{\\text\{LLM\}\}\_\{\\text\{evaluator\}\}\. Specifically, substituting theLLMevaluator\{\\text\{LLM\}\}\_\{\\text\{evaluator\}\}with the weaker Qwen3\-8B yields a larger performance decline, likely because processing complex trajectories and making accurate attributions is challenging for weaker agents\. In contrast, using a moderately sized model such as Qwen3\-32B leads to a relatively smaller degradation\. To further check whether the gains mainly come from a high\-tier model, we replace both the evaluator and the summarizer with Qwen3\-8B\. Table[4](https://arxiv.org/html/2608.03403#S4.T4)shows that even without a high\-tier model,ExpGstill yields large and stable gains over No Method on all three datasets, and these gains remain comparable to those of Qwen3\-Max\. This indicates that the gains mainly come from structured distillation and reuse of tool experiences, and that agents can self\-evolve throughExpGwithout a stronger external model\.

##### Equivalence\-Class\-Based Selection Method\.

We compare our equivalence\-class\-based experience selection method with two typical selection methods: instance\-balanced, which selects experiences uniformly, and class\-balanced, which selects similar numbers from each class\. We use negative Kullback\-Leibler divergenceKullback and Leibler \([1951](https://arxiv.org/html/2608.03403#bib.bib38)\)to measure distribution alignment with the original experience distribution, and Shannon entropyShannon \([1948](https://arxiv.org/html/2608.03403#bib.bib37)\)to measure the diversity of the selected set\. Figure[5](https://arxiv.org/html/2608.03403#S4.F5)shows the results under different dimensionsddof thes​c​o​r​e​sscoresand selection ratios\. In terms of distribution alignment, our method performs similarly to the instance\-balanced method, maintaining strong consistency with the original distribution\. It can even outperform the instance\-balanced method through fine\-grained adjustment of the selection quotas at higher selection ratios\. From the perspective of diversity, the class\-balanced method usually achieves the highest entropy by equalizing class proportions, whereas the instance\-balanced method has lower entropy, indicating weaker class coverage\. Our method achieves much higher entropy than the instance\-balanced method and slightly lower entropy than the class\-balanced method in most settings\. It ensures that the chosen experiences both cover as many invocation patterns as possible and remain representative of the original distribution\.

##### Case Analysis\.

We perform case analysis to better understand the strengths and limitations ofExpG\. These cases show thatExpGmainly helps the agent: 1\) improve tool\-use awareness, 2\) distinguish similar tools, 3\) invoke tools correctly, 4\) fix tool\-use errors, 5\) identify tool dependencies, and 6\) understand tool reliability\. We also study cases whereExpGfails, such as unseen situations, hard\-to\-analyze trajectories, incorrect attributions, and over\-reaction\. Detailed analyses and a case\-coverage comparison with other methods can be found in Appendix[F](https://arxiv.org/html/2608.03403#A6)\.

## 5Conclusion

This paper addresses the robustness challenges of tool use within the agent harness during runtime execution\. We propose a mechanismExpG, which forms adaptive guidance for each tool through three phases: experience acquisition, distillation, and reuse\. Extensive experiments validate the effectiveness of our method across various benchmarks, demonstrating that our guidance helps agents shift to a more informed tool\-use strategy\. We believeExpGrepresents an early step toward trainable tools, paving the way for building more robust and adaptive agent systems\.

## References

- Learning evolving tools for large language models\.InThe Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24\-28,Cited by:[§1](https://arxiv.org/html/2608.03403#S1.p2.1),[§2](https://arxiv.org/html/2608.03403#S2.p1.1)\.
- S\. Chen, J\. Gai, R\. Zhou, J\. Zhang, T\. Zhu, J\. Li, K\. Wang, Z\. Wang, Z\. Chen, K\. Kaleb, N\. Miao, S\. Gao, C\. Lu, M\. Li, J\. He, and Y\. W\. Teh \(2026\)SkillCraft: can llm agents learn to use tools skillfully?\.Cited by:[§2](https://arxiv.org/html/2608.03403#S2.p2.1)\.
- Z\. Chen, K\. Zhou, B\. Zhang, Z\. Gong, X\. Zhao, and J\. Wen \(2023\)ChatCoT: tool\-augmented chain\-of\-thought reasoning on chat\-based large language models\.InFindings of the Association for Computational Linguistics: EMNLP 2023, Singapore, December 6\-10,pp\. 14777–14790\.Cited by:[§2](https://arxiv.org/html/2608.03403#S2.p1.1)\.
- Z\. Cheng, H\. Wang, Z\. Liu, Y\. Guo, Y\. Guo, Y\. Wang, and H\. Wang \(2025\)ToolSpectrum: towards personalized tool utilization for large language models\.InFindings of the Association for Computational Linguistics: ACL 2025,Vienna, Austria,pp\. 20679–20699\.Cited by:[§1](https://arxiv.org/html/2608.03403#S1.p2.1),[§2](https://arxiv.org/html/2608.03403#S2.p1.1)\.
- P\. Chhikara, D\. Khant, S\. Aryan, T\. Singh, and D\. Yadav \(2025\)Mem0: building production\-ready ai agents with scalable long\-term memory\.arXiv preprint arXiv:2504\.19413\.Cited by:[Appendix C](https://arxiv.org/html/2608.03403#A3.SS0.SSS0.Px3.p1.1),[§2](https://arxiv.org/html/2608.03403#S2.p2.1),[§4\.1](https://arxiv.org/html/2608.03403#S4.SS1.SSS0.Px1.p1.1)\.
- DeepSeek\-AI \(2025\)DeepSeek\-v3 technical report\.arXiv preprint arXiv:2412\.19437\.Cited by:[§4\.1](https://arxiv.org/html/2608.03403#S4.SS1.SSS0.Px3.p1.4)\.
- F\. M\. Dekking, C\. Kraaikamp, H\. P\. Lopuhaä, and L\. E\. Meester \(2005\)A modern introduction to probability and statistics\.1 edition,Springer London,London\.Cited by:[§E\.1](https://arxiv.org/html/2608.03403#A5.SS1.p1.4)\.
- K\. J\. Devlin \(2017\)Sets, functions, and logic: an introduction to abstract mathematics, third edition\.3 edition,Chapman and Hall/CRC Press\.Cited by:[§3\.2](https://arxiv.org/html/2608.03403#S3.SS2.p2.15)\.
- Z\. Guo, S\. Cheng, H\. Wang, S\. Liang, Y\. Qin, P\. Li, Z\. Liu, M\. Sun, and Y\. Liu \(2024\)StableToolBench: towards stable large\-scale benchmarking on tool learning of large language models\.InFindings of the Association for Computational Linguistics: ACL 2024,Bangkok, Thailand,pp\. 11143–11156\.Cited by:[§1](https://arxiv.org/html/2608.03403#S1.p2.1),[§2](https://arxiv.org/html/2608.03403#S2.p1.1)\.
- Y\. Hu, S\. Liu, Y\. Yue, G\. Zhang, B\. Liu, F\. Zhu, J\. Lin, H\. Guo, S\. Dou, Z\. Xi, S\. Jin, J\. Tan, Y\. Yin, J\. Liu, Z\. Zhang, Z\. Sun, Y\. Zhu, H\. Sun, B\. Peng, Z\. Cheng, X\. Fan, J\. Guo, X\. Yu, Z\. Zhou, Z\. Hu, J\. Huo, J\. Wang, Y\. Niu, Y\. Wang, Z\. Yin, X\. Hu, Y\. Liao, Q\. Li, K\. Wang, W\. Zhou, Y\. Liu, D\. Cheng, Q\. Zhang, T\. Gui, S\. Pan, Y\. Zhang, P\. Torr, Z\. Dou, J\. Wen, X\. Huang, Y\. Jiang, and S\. Yan \(2025\)Memory in the age of ai agents\.arXiv preprint arXiv:2512\.13564\.Cited by:[§1](https://arxiv.org/html/2608.03403#S1.p3.1),[§2](https://arxiv.org/html/2608.03403#S2.p2.1)\.
- X\. Huang, S\. Cheng, S\. Huang, J\. Shen, Y\. Xu, C\. Zhang, and Y\. Qu \(2024a\)QueryAgent: a reliable and efficient reasoning framework with environmental feedback based self\-correction\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics,Bangkok, Thailand,pp\. 5014–5035\.Cited by:[§2](https://arxiv.org/html/2608.03403#S2.p2.1)\.
- Y\. Huang, J\. Shi, Y\. Li, C\. Fan, S\. Wu, Q\. Zhang, Y\. Liu, P\. Zhou, Y\. Wan, N\. Z\. Gong, and L\. Sun \(2024b\)MetaTool benchmark for large language models: deciding whether to use tools and which to use\.InThe Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7\-11,Cited by:[Appendix D](https://arxiv.org/html/2608.03403#A4.SS0.SSS0.Px1.p1.4),[§4\.1](https://arxiv.org/html/2608.03403#S4.SS1.SSS0.Px2.p1.1)\.
- S\. Kullback and R\. A\. Leibler \(1951\)On information and sufficiency\.The Annals of Mathematical Statistics22\(1\),pp\. 79–86\.Cited by:[§B\.2\.1](https://arxiv.org/html/2608.03403#A2.SS2.SSS1.p1.1),[§4\.4](https://arxiv.org/html/2608.03403#S4.SS4.SSS0.Px2.p1.2)\.
- Y\. Lee, R\. Nair, Q\. Zhang, K\. Lee, O\. Khattab, and C\. Finn \(2026\)Meta\-harness: end\-to\-end optimization of model harnesses\.Cited by:[§1](https://arxiv.org/html/2608.03403#S1.p1.1),[§2](https://arxiv.org/html/2608.03403#S2.p2.1)\.
- F\. Lei, Y\. Yang, W\. Sun, and D\. Lin \(2025\)MCPVerse: an expansive, real\-world benchmark for agentic tool use\.arXiv preprint arXiv:2508\.16260\.Cited by:[§E\.4](https://arxiv.org/html/2608.03403#A5.SS4.p1.3)\.
- M\. Li, F\. Song, Y\. Bowen, H\. Yu, Z\. Li, F\. Huang, and Y\. Li \(2023\)API\-bank: a comprehensive benchmark for tool\-augmented llms\.InConference on Empirical Methods in Natural Language Processing,Cited by:[Appendix D](https://arxiv.org/html/2608.03403#A4.SS0.SSS0.Px2.p1.1),[§4\.1](https://arxiv.org/html/2608.03403#S4.SS1.SSS0.Px2.p1.1)\.
- Y\. Liao, S\. Jiang, Y\. Wang, and Y\. Wang \(2025\)ReflecTool: towards reflection\-aware tool\-augmented clinical agents\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics,Vienna, Austria,pp\. 13507–13531\.Cited by:[§2](https://arxiv.org/html/2608.03403#S2.p1.1)\.
- G\. Ling, S\. Zhong, and R\. Huang \(2026\)Agent skills: a data\-driven analysis of claude skills for extending large language model functionality\.arXiv preprint arXiv:2602\.08004\.Cited by:[§2](https://arxiv.org/html/2608.03403#S2.p2.1)\.
- Q\. Meng, Y\. Wang, L\. Chen, Q\. Wang, C\. Lu, W\. Wu, Y\. Gao, Y\. Wu, and Y\. Hu \(2026\)Agent harness for large language model agents: a survey\.Preprints\.External Links:[Document](https://dx.doi.org/10.20944/preprints202604.0428.v2)Cited by:[§1](https://arxiv.org/html/2608.03403#S1.p1.1),[§2](https://arxiv.org/html/2608.03403#S2.p2.1)\.
- OpenAI \(2025\)GPT\-5 system card\.Technical ReportOpenAI\.External Links:[Link](https://cdn.openai.com/gpt-5-system-card.pdf)Cited by:[§4\.1](https://arxiv.org/html/2608.03403#S4.SS1.SSS0.Px3.p1.4)\.
- W\. Pan, S\. Liu, X\. Zhou, S\. Zhang, W\. Shi, M\. Xu, and X\. Jia \(2026\)M⋆: every task deserves its own memory harness\.Cited by:[§1](https://arxiv.org/html/2608.03403#S1.p1.1),[§2](https://arxiv.org/html/2608.03403#S2.p2.1)\.
- S\. G\. Patil, H\. Mao, F\. Yan, C\. C\. Ji, V\. Suresh, I\. Stoica, and J\. E\. Gonzalez \(2025\)The berkeley function calling leaderboard \(bfcl\): from tool use to agentic evaluation of large language models\.InForty\-second International Conference on Machine Learning, ICML 2025, Vancouver, BC, Canada, July 13\-19, 2025,Cited by:[Appendix D](https://arxiv.org/html/2608.03403#A4.SS0.SSS0.Px3.p1.4),[§4\.1](https://arxiv.org/html/2608.03403#S4.SS1.SSS0.Px2.p1.1)\.
- S\. G\. Patil, T\. Zhang, X\. Wang, and J\. E\. Gonzalez \(2024\)Gorilla: large language model connected with massive apis\.InAdvances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 \- 15, 2024,Cited by:[§2](https://arxiv.org/html/2608.03403#S2.p1.1)\.
- C\. Qian, E\. C\. Acikgoz, Q\. He, H\. WANG, X\. Chen, D\. Hakkani\-Tür, G\. Tur, and H\. Ji \(2025\)ToolRL: reward is all tool learning needs\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems,Cited by:[§2](https://arxiv.org/html/2608.03403#S2.p1.1)\.
- Y\. Qin, S\. Liang, Y\. Ye, K\. Zhu, L\. Yan, Y\. Lu, Y\. Lin, X\. Cong, X\. Tang, B\. Qian, S\. Zhao, L\. Hong, R\. Tian, R\. Xie, J\. Zhou, M\. Gerstein, D\. Li, Z\. Liu, and M\. Sun \(2024\)ToolLLM: facilitating large language models to master 16000\+ real\-world apis\.InThe Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7\-11, 2024,Cited by:[§E\.4](https://arxiv.org/html/2608.03403#A5.SS4.p1.3),[§2](https://arxiv.org/html/2608.03403#S2.p1.1)\.
- C\. Qu, S\. Dai, X\. Wei, H\. Cai, S\. Wang, D\. Yin, J\. Xu, and J\. Wen \(2025a\)From exploration to mastery: enabling llms to master tools via self\-driven interactions\.InThe Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24\-28,Cited by:[Appendix C](https://arxiv.org/html/2608.03403#A3.SS0.SSS0.Px2.p1.1),[§4\.1](https://arxiv.org/html/2608.03403#S4.SS1.SSS0.Px1.p1.1)\.
- C\. Qu, S\. Dai, X\. Wei, H\. Cai, S\. Wang, D\. Yin, J\. Xu, and J\. Wen \(2025b\)Tool learning with large language models: a survey\.Front\. Comput\. Sci\.19\(8\)\.Cited by:[§1](https://arxiv.org/html/2608.03403#S1.p2.1),[§2](https://arxiv.org/html/2608.03403#S2.p1.1),[§4\.1](https://arxiv.org/html/2608.03403#S4.SS1.SSS0.Px2.p1.1)\.
- Qwen\-Team \(2025\)Qwen3 technical report\.arXiv preprint arXiv:2505\.09388\.Cited by:[§4\.1](https://arxiv.org/html/2608.03403#S4.SS1.SSS0.Px3.p1.4)\.
- C\. E\. Shannon \(1948\)A mathematical theory of communication\.The Bell System Technical Journal27\(3\),pp\. 379–423\.Cited by:[§B\.2\.2](https://arxiv.org/html/2608.03403#A2.SS2.SSS2.p1.1),[§4\.4](https://arxiv.org/html/2608.03403#S4.SS4.SSS0.Px2.p1.2)\.
- Y\. Song, T\. Wang, P\. Cai, S\. K\. Mondal, and J\. P\. Sahoo \(2023\)A comprehensive survey of few\-shot learning: evolution, applications, challenges, and opportunities\.ACM Comput\. Surv\.55\(13s\)\.Cited by:[Appendix C](https://arxiv.org/html/2608.03403#A3.SS0.SSS0.Px1.p1.1),[§4\.1](https://arxiv.org/html/2608.03403#S4.SS1.SSS0.Px1.p1.1)\.
- Y\. Song, F\. F\. Xu, S\. Zhou, and G\. Neubig \(2025\)Beyond browsing: API\-based web agents\.InFindings of the Association for Computational Linguistics: ACL 2025,Vienna, Austria,pp\. 11066–11085\.Cited by:[§2](https://arxiv.org/html/2608.03403#S2.p1.1)\.
- J\. Sun, S\. Y\. Min, Y\. Chang, and Y\. Bisk \(2024\)Tools fail: detecting silent errors in faulty tools\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,Miami, Florida, USA,pp\. 14272–14289\.Cited by:[§1](https://arxiv.org/html/2608.03403#S1.p2.1),[§2](https://arxiv.org/html/2608.03403#S2.p1.1)\.
- Y\. Wang, Z\. Jiang, Z\. Chen, F\. Yang, Y\. Zhou, E\. Cho, X\. Fan, Y\. Lu, X\. Huang, and Y\. Yang \(2024\)RecMind: large language model powered agent for recommendation\.InFindings of the Association for Computational Linguistics: NAACL 2024,Mexico City, Mexico,pp\. 4351–4364\.Cited by:[§2](https://arxiv.org/html/2608.03403#S2.p1.1)\.
- J\. Wu, B\. Li, R\. Fang, W\. Yin, L\. Zhang, Z\. Wang, Z\. Tao, D\. Zhang, Z\. Xi, X\. Tang, Y\. Jiang, P\. Xie, F\. Huang, and J\. Zhou \(2025\)WebDancer: towards autonomous information seeking agency\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems,Cited by:[§1](https://arxiv.org/html/2608.03403#S1.p1.1),[§2](https://arxiv.org/html/2608.03403#S2.p1.1)\.
- Q\. Xu, Y\. Li, H\. Xia, and W\. Li \(2024\)Enhancing tool retrieval with iterative feedback from large language models\.InFindings of the Association for Computational Linguistics: EMNLP 2024,Miami, Florida, USA,pp\. 9609–9619\.Cited by:[§2](https://arxiv.org/html/2608.03403#S2.p2.1)\.
- S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. R\. Narasimhan, and Y\. Cao \(2023\)ReAct: synergizing reasoning and acting in language models\.InThe Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1\-5,Cited by:[§2](https://arxiv.org/html/2608.03403#S2.p1.1)\.
- H\. Ye, X\. He, V\. Arak, H\. Dong, and G\. Song \(2026\)Meta context engineering via agentic skill evolution\.Cited by:[§2](https://arxiv.org/html/2608.03403#S2.p2.1)\.
- S\. Yuan, K\. Song, J\. Chen, X\. Tan, Y\. Shen, K\. Ren, D\. Li, and D\. Yang \(2025\)EASYTOOL: enhancing LLM\-based agents with concise tool instruction\.InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies,Albuquerque, New Mexico,pp\. 951–972\.Cited by:[§2](https://arxiv.org/html/2608.03403#S2.p1.1)\.
- G\. Zhang, L\. Niu, J\. Fang, K\. Wang, L\. Bai, and X\. Wang \(2025\)Multi\-agent architecture search via agentic supernet\.InProceedings of the 42nd International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.267,pp\. 75834–75852\.Cited by:[§2](https://arxiv.org/html/2608.03403#S2.p1.1)\.
- K\. Zhang, J\. Li, G\. Li, X\. Shi, and Z\. Jin \(2024\)CodeAgent: enhancing code generation with tool\-integrated agent systems for real\-world repo\-level coding challenges\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics,Bangkok, Thailand,pp\. 13643–13658\.Cited by:[§1](https://arxiv.org/html/2608.03403#S1.p1.1),[§2](https://arxiv.org/html/2608.03403#S2.p1.1)\.
- Q\. Zhang, C\. Hu, S\. Upasani, B\. Ma, F\. Hong, V\. Kamanuru, J\. Rainton, C\. Wu, M\. Ji, H\. Li, U\. Thakker, J\. Zou, and K\. Olukotun \(2026\)Agentic context engineering: evolving contexts for self\-improving language models\.Cited by:[§2](https://arxiv.org/html/2608.03403#S2.p2.1)\.
- L\. Zheng, W\. Chiang, Y\. Sheng, S\. Zhuang, Z\. Wu, Y\. Zhuang, Z\. Lin, Z\. Li, D\. Li, E\. Xing,et al\.\(2023\)Judging llm\-as\-a\-judge with mt\-bench and chatbot arena\.Advances in neural information processing systems36,pp\. 46595–46623\.Cited by:[§3\.2](https://arxiv.org/html/2608.03403#S3.SS2.p2.15)\.
- C\. Zhou, H\. Chai, W\. Chen, Z\. Guo, R\. Shan, Y\. Song, T\. Xu, Y\. Yang, A\. Yu, W\. Zhang, C\. Zheng, J\. Zhu, Z\. Zheng, Z\. Zhang, X\. Lou, C\. Zhang, Z\. Fu, J\. Wang, W\. Liu, J\. Lin, and W\. Zhang \(2026\)Externalization in llm agents: a unified review of memory, skills, protocols and harness engineering\.Cited by:[§1](https://arxiv.org/html/2608.03403#S1.p1.1),[§2](https://arxiv.org/html/2608.03403#S2.p2.1)\.
- Y\. Zhou, S\. Levine, J\. E\. Weston, X\. Li, and S\. Sukhbaatar \(2025\)Self\-challenging language model agents\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems,Cited by:[§2](https://arxiv.org/html/2608.03403#S2.p1.1)\.

## Appendix ATemplate Prompt for LLM\-as\-judge Agent

Prompt for agentLLMevaluator\{\\text\{LLM\}\}\_\{\\text\{evaluator\}\}You are an expert in evaluating tool invocation process\. The tool is invoked by an AI agent\.Tool invocation Information: ∙\\bulletTool Name: \{tool\_name\}∙\\bulletSuccess Flag: \{success\_flag\}∙\\bulletTime Cost: \{time\_cost\}s∙\\bulletToken Cost: \{token\_cost\} tokens∙\\bulletAgent Context: \{context\}∙\\bulletInput Parameters: \{input\_params\}∙\\bulletTool Response: \{response\}∙\\bulletTool Schema: \{schema\}Evaluation Method:Start from a default score list of scores = \[0, 0, 0, 0, 0, 0, 0, 0, 0, 0\]\. For each item below that is satisfied, assign 1 point to the corresponding index\. The final scores should be a list of 10 integers, each being either 0 or 1\.1\. Use Quality \(total 2 points\. If context is provided, use it as an aid when evaluating\): ∙\\bulletIndex 1: Should the tool be invoked now? Consider whether the all necessary information for the tool’s invocation is ready, and whether the tool execution environment is correct\. If it is a multi\-round conversation, also consider the dependency relationships of the tool chain\.∙\\bulletIndex 2: If should, is the chosen tool appropriate?2\. Input Quality \(total 4 points\. When evaluating, consider both the context and the tool schema\): ∙\\bulletIndex 3: Are all required parameters provided?∙\\bulletIndex 4: Are the input parameters valid and supported by the tool?∙\\bulletIndex 5: Are the input parameters in the correct format for their respective fields?∙\\bulletIndex 6: Does the value\(content\) of input parameter correctly reflect and match the given context?3\. Response Quality \(total 4 points\): ∙\\bulletIndex 7: Does the response provide meaningful and useful information? or are there any error messages or information can be used as guidance for agent invoking tool better?∙\\bulletIndex 8: Does the response match the tool’s intended purpose/function?∙\\bulletIndex 9: Does the response value correct \(content appropriate\) given the input parameters?∙\\bulletIndex 10: Does the response help accomplish the task within the given context?Important: 1\. Sometimes there is not enough information in the context or schema to make a complete evaluation\. In such cases, make your best judgment based on the available information\.2\. Some tools \(commonly system tools such as mkdir, touch, echo, etc\.\) modify the external environment\. Since these results cannot be obtained, they return “None” as the response\. At this point, all the scores in the quality of the response should be obtained and should not be seen as a problem for the tool\.3\. Evaluation independently from the success flag\. Thesuccess\_flagindicates whether the tool executed without technical errors\. Theevaluationshould evaluate the quality of the tool invocation\. A tool can execute successfully \(Success Flag=1\) but still produce low\-quality or irrelevant responses, leading to a low evaluation score\.4\. Sometimes an agent will execute multiple steps and invoke multiple tools to complete a task, but you only need to evaluate the use of one tool for one of the steps, not whether the final task is completed or not\.Answer Format: Please provide your answer in the following JSON format:`‘‘‘`json \{ “scores”: A list of 10 integers \(0 or 1\) based on the evaluation criteria above\. “explanation”: A brief evaluation \(2\-3 sentences\) explaining the quality of the tool invocation, based on your evaluation\. Low\-quality aspects need to be reified, especially the causes of tool invocation errors\. \} `‘‘‘`

Prompt for agentLLMsummarizer\{\\text\{LLM\}\}\_\{\\text\{summarizer\}\}You are an expert in analyzing tool usage patterns and generating practical usage guidance for agents\.Tool Information: ∙\\bulletTool Name: \{tool\_name\}∙\\bulletTool Schema: \{tool\_schema\}Recent Tool Invocation Experiences: \{experiences\}Important: 1\. Assume the tool \(tool schema\) can’t be changed, your task is to guide agent to use it better\.2\. Your answer must be based on the information given, don’t make it up\. If not enough data, state “Not enough data to determine Core Function/Success Patterns/Common Issues/Best Practices\.”3\. Your answer will be used to guide the use of the tool in the future, so do not include content related to recent tool invocation experience such as “case \#3” or “Call \#2”, but some values can be used as examples\.4\. Pay attention to information not mentioned in the tool schema, such as the response upon successful tool invocation\. It’s also welcome to uncover insights, such as how tools can be used more effectively, and possible dependencies between tools\. But if they aren’t, don’t make them up\.5\. Finally, to avoid deriving incorrect guidance from individual invocation, check whether, if the agent follows the proposed guidance, it can perform better on all recent invocation histories\. If not, revise the guidance until it can\. Specifically:∙\\bulletdon’t write guidance in an absolute tone without a very deterministic message \(meaning that all invocation histories are satisfied, otherwise it will result in failure\)\.∙\\bulletsometimes there may be inconsistencies\. Consider whether this is due to the context in which the tool is being used\.Your Task: Based on the tool invocation history, generate a concise and logical tool usage guidance following this structure:1\.Core Function: What this tool does and when to use it\.2\.Success Patterns: Parameter patterns and usage scenarios that work well\.3\.Common Issues: Main pitfalls to avoid and why they fail\.4\.Best Practices: 2\-3 actionable recommendations\.Answer Format: Provide a structured, concise guidance \(max 200 words\)\. Focus on actionable insights derived from actual usage data\. Avoid generic advice and think step by step\.`‘‘‘`txt Your concise, data\-driven tool usage guidance `‘‘‘`

## Appendix BAnalysis of Equivalence\-Class\-Based Selection Method

### B\.1Problem Restatement and Notation

We consider a budgeted experience selection problem for a fixed toolτ\\tau\. Letℰτ=\{Ei\}i=1Nτ\\mathcal\{E\}\_\{\\tau\}=\\\{E\_\{i\}\\\}\_\{i=1\}^\{N\_\{\\tau\}\}be the experience pool, whereNτ≜\|ℰτ\|N\_\{\\tau\}\\triangleq\|\\mathcal\{E\}\_\{\\tau\}\|\. We selectQQexperiences under a context\-length budget\. In our structured definition, each experienceEEis associated with a binary score vector

κ​\(E\)≜s​c​o​r​e​s​\(E\)∈\{0,1\}d,\\kappa\(E\)\\triangleq scores\(E\)\\in\\\{0,1\\\}^\{d\},\(3\)whereddis the score dimension\. Experiences are grouped into equivalence classes according to identical score vectors\. Let\{𝒞k\}k=1K\\\{\\mathcal\{C\}\_\{k\}\\\}\_\{k=1\}^\{K\}denote the induced partition ofℰτ\\mathcal\{E\}\_\{\\tau\}, and letKKbe the number of observed equivalence classes\. The class distribution on the experience poolℰτ\\mathcal\{E\}\_\{\\tau\}ispk≜\|𝒞k\|/Nτp\_\{k\}\\triangleq\{\|\\mathcal\{C\}\_\{k\}\|\}/\{N\_\{\\tau\}\}fork=1,…,Kk=1,\\dots,K, andpk\>0p\_\{k\}\>0\. After selection, we obtain a distributionq=\(q1,…,qK\)q=\(q\_\{1\},\\dots,q\_\{K\}\)in the selected set over the sameKKclasses, whereqkq\_\{k\}denotes the fraction of selected experiences belonging to class𝒞k\\mathcal\{C\}\_\{k\}and∑k=1Kqk=1\\sum\_\{k=1\}^\{K\}q\_\{k\}=1\.

Our selection method can be viewed at the class\-distribution level as forming a convex combination between the uniform distribution \(encouraging coverage\) and the original distribution \(preserving representativeness\), denoted as

q=α​u\+\(1−α\)​p,α≜KQ,q=\\alpha u\+\(1\-\\alpha\)p,\\qquad\\alpha\\triangleq\\frac\{K\}\{Q\},\(4\)whereu∈ΔK−1u\\in\\Delta^\{K\-1\}denotes the uniform distribution over theKKobserved equivalence classes, anduk≜1/Ku\_\{k\}\\triangleq 1/K, and we focus only on the common regimeQ≥KQ\\geq Kwhere full class coverage is feasible\. Table[5](https://arxiv.org/html/2608.03403#A2.T5)summarizes the notations\.

Table 5:Notations\.
### B\.2Metrics and Bounds

The equivalence\-class\-based selection method aims to induce a distributionqqthat \(1\) remains representative of the original experience pool while \(2\) covering as many distinct invocation patterns as possible given quotaQQ\. We quantify these two objectives using \(i\) distribution alignment and \(ii\) diversity, and derive lower bounds to show the effectiveness of ours\.

#### B\.2\.1Distribution Alignment

Distribution alignment can be measured by the negative KL divergence\[[13](https://arxiv.org/html/2608.03403#bib.bib38)\]

Align​\(q,p\)≜−DKL​\(q∥p\)=−∑k=1Kqk​log⁡qkpk,\\text\{Align\}\(q,p\)\\triangleq\-D\_\{\\mathrm\{KL\}\}\(q\\\|p\)=\-\\sum\_\{k=1\}^\{K\}q\_\{k\}\\log\\frac\{q\_\{k\}\}\{p\_\{k\}\},\(5\)where larger values indicate better alignment\.

Under Eq\. \([4](https://arxiv.org/html/2608.03403#A2.E4)\), we lower boundAlign​\(q,p\)\\text\{Align\}\(q,p\)by upper boundingDKL​\(q∥p\)D\_\{\\mathrm\{KL\}\}\(q\\\|p\)\. Using the convexity ofDKL\(⋅∥p\)D\_\{\\mathrm\{KL\}\}\(\\cdot\\\|p\)in its first argument, we have

DKL​\(q∥p\)\\displaystyle D\_\{\\mathrm\{KL\}\}\(q\\\|p\)=DKL\(αu\+\(1−α\)p∥p\)\\displaystyle=D\_\{\\mathrm\{KL\}\}\\\!\\left\(\\alpha u\+\(1\-\\alpha\)p\\,\\middle\\\|\\,p\\right\)≤α​DKL​\(u∥p\)\+\(1−α\)​DKL​\(p∥p\)\\displaystyle\\leq\\alpha D\_\{\\mathrm\{KL\}\}\(u\\\|p\)\+\(1\-\\alpha\)D\_\{\\mathrm\{KL\}\}\(p\\\|p\)=α​DKL​\(u∥p\)\.\\displaystyle=\\alpha D\_\{\\mathrm\{KL\}\}\(u\\\|p\)\.\(6\)Equivalently,

Align​\(q,p\)≥−α​DKL​\(u∥p\)\.\\text\{Align\}\(q,p\)\\geq\-\\alpha D\_\{\\mathrm\{KL\}\}\(u\\\|p\)\.\(7\)
As the quotaQQincreases,α=K/Q\\alpha=K/Qdecreases, the bound in Eq\. \([6](https://arxiv.org/html/2608.03403#A2.E6)\) tightens, andqqbecomes progressively closer to the original distributionpp\. In addition, our fine\-grained adjustment reallocates any leftover quota to other classes that still have available instances\. This avoids wasting quota due to over\-allocation to small classes and helps keep the induced distributionqqcloser to the original distributionpp, which further improves theAlign​\(q,p\)\\text\{Align\}\(q,p\)\. The distribution alignmentAlign​\(q,p\)\\text\{Align\}\(q,p\)achieves its upper bound of0when the selected distributionqqexactly matches the original distributionpp\. This design preserves representativeness of the selected experiences by allocating more quota to larger equivalence classes, which correspond to more frequent invocation patterns in the original experience pool of toolτ\\tau\.

#### B\.2\.2Diversity

Diversity can be measured using Shannon entropy\[[29](https://arxiv.org/html/2608.03403#bib.bib37)\]

Div​\(q\)≜H​\(q\)=−∑k=1Kqk​log⁡qk,\\text\{Div\}\(q\)\\triangleq H\(q\)=\-\\sum\_\{k=1\}^\{K\}q\_\{k\}\\log q\_\{k\},\(8\)where larger values correspond to more uniform coverage over invocation patterns\.

Under Eq\. \([4](https://arxiv.org/html/2608.03403#A2.E4)\), we can guarantee a non\-trivial level of diversity by lower boundingH​\(q\)H\(q\)\. Using the concavity of entropy,

H​\(q\)\\displaystyle H\(q\)=H​\(α​u\+\(1−α\)​p\)\\displaystyle=H\\\!\\left\(\\alpha u\+\(1\-\\alpha\)p\\right\)≥α​H​\(u\)\+\(1−α\)​H​\(p\)\\displaystyle\\geq\\alpha H\(u\)\+\(1\-\\alpha\)H\(p\)=α​log⁡K\+\(1−α\)​H​\(p\)\.\\displaystyle=\\alpha\\log K\+\(1\-\\alpha\)H\(p\)\.\(9\)Therefore, our method guaranteesDiv​\(q\)\\text\{Div\}\(q\)is at least a convex combination of maximal diversitylog⁡K\\log Kand the original diversityH​\(p\)H\(p\), controlled byα=K/Q\\alpha=K/Q, andDiv​\(q\)\\text\{Div\}\(q\)approaches its maximum valuelog⁡K\\log Kwhen the selection quotaQQis close to the number of equivalence classesKK\. This ensures that the selected experiences maintain broad coverage over invocation patterns even under a limited quota, which is particularly beneficial when original distributionqqis long\-tailed\. In such cases, it mitigates the dominance of a few high\-frequency patterns and preserves coverage of rare yet informative ones\.

### B\.3Discussion

As a robust and inherently scalable architecture, equivalence class partitioning redefines how quality signals are aggregated\. It systematically maps the outputs of LLM\-as\-a\-Judge into a rigorous taxonomy of pattern categories\. This foundational partitioning empowers a superior selection mechanism that masters the critical equilibrium between representativeness and diversity\. Beyond mere subset selection, this framework serves as a universal engine for any task requiring precise distributional control, ranging from strategic data mixture to the fundamental optimization of loss functions\.

## Appendix CBaseline Details

##### Few\-shot Learning\.

A prompting method\[[30](https://arxiv.org/html/2608.03403#bib.bib31)\]that provides a small number of tool\-use examples in the prompt to let the model learn from them\. To ensure a fair comparison, we used the same invocations in the train set as inExpGas examples, and the same truncation for each tool according to the context length\.

##### DRAFT\.

Dynamically Refining tool documentation through the Analysis of Feedback and Trials \(DRAFT\)\[[26](https://arxiv.org/html/2608.03403#bib.bib42)\]is a framework designed to bridge the comprehension gap between LLMs and external tools through an iterative and self\-improving methodology\. Unlike our approach, DRAFT decouples the process from the agent’s inference procedure and improves tool documentation quality through three steps: experience gathering, learning from experience, and documentation rewriting\. Since it requires the tool to be actually executed in order to function, on the tools that cannot be actually executed in the MetaTool dataset and parts of the BFCL\-v3 dataset, we use the tool invocation information we extracted inExpGinstead of the real execution\. We use Qwen3\-max as the agent model, consistent with our setup\.

##### Mem0\.

Mem0\[[5](https://arxiv.org/html/2608.03403#bib.bib41)\]is a memory layer designed to provide agents with a persistent knowledge base\. In our experiments, we specifically utilize itsprocedural\_memorymodule \(invoked via theaddmethod withmemory\_type="procedural\_memory"\)\. As its native design cannot reliably capture tool\-level signals \(it mainly stores user preferences or coarse\-grained agent behaviors\), we explicitly provide the agent–tool interactions from the training set\. When retrieve the memory during inference, we apply the same truncation for each tool according to the context length\. We use Qwen3\-max as the agent model, consistent with our setup\.

## Appendix DDataset and Metric

##### MetaTool

MetaTool\[[12](https://arxiv.org/html/2608.03403#bib.bib39)\]is a benchmark designed to evaluate whether LLMs have tool usage awareness and can correctly choose tools\. We use four sub\-task settings that test tool selection to construct a dataset centered on the 100 most frequently used tools, with a maximum of 10 tasks per sub\-task involving the same tool, forming total 1669 tasks\. Specifically,Similarsub\-task setting denotes tool selection tasks with similar tool candidates \(500 tasks\);Scenariosub\-task setting denotes tool selection tasks in specific scenarios \(530 tasks\);Reliabilitysub\-task setting denotes tool selection tasks with possible reliability issues, where the ground truth tool is excluded from the candidate list \(500 tasks\); andMulti\-toolsub\-task setting denotes multi\-tool selection tasks \(139 tasks\)\. For evaluation, we follow the benchmark’s matching method to assess whether the set of tools selected by the agent exactly matches the ground truth answers\.

##### API\-Bank

API\-Bank\[[16](https://arxiv.org/html/2608.03403#bib.bib36)\]is a benchmark designed to evaluate the end\-to\-end performance of tool\-augmented LLMs, specifically in planning, retrieving, and invoking tools within dialogue systems\. It simulates a comprehensive tool\-use environment containing 53 diverse tools and includes 399 tasks in its Level\-1 subset, which specifically targets the correctness of tool invocations\. For evaluation, we follow the dataset’s designed matching method, focusing on precision by checking whether the predicted tool name and parameter values exactly match the ground truth annotations\.

##### BFCL\-V3

The Berkeley Function Calling Leaderboard V3 \(BFCL\-V3\)\[[22](https://arxiv.org/html/2608.03403#bib.bib44)\]is a comprehensive benchmark assessing tool invocation capabilities across diverse programming languages and interaction patterns\. It comprises varied categories, ranging from simple function invocations to complex scenarios involving multiple or parallel invocations\. Because some of these tools have only been invoked one time, we aggregate the tasks according to the use of the tools in the ground truth\. We then filter the data to keep only tasks in which every tool used also appears in at least two other related tasks, and we randomly sample up to 50 tasks per category, excluding the “Long Context” category to avoid confounding effects from different context\-length limits\. The remaining 461 tasks constitute the test set\. To build the training set, we gather all corresponding related tasks and, for each tool, randomly sample 10 tasks when it is associated with more than 10, resulting in 1,542 training tasks\. In our analysis, we divide the tasks into four sub\-tasks according to \(1\) the number of inference turns required and \(2\) whether the task involves problematic settings, such as testing tool\-use relevance \(irrelevance\), missing tools, or missing required information for tool invocation\. Specifically,Normalsingle\\textit\{Normal\}\\mathrm\{\}\_\{\\mathrm\{single\}\}sub\-task setting denotes single\-turn baseline tasks under normal settings \(255 tasks\);Normalmulti\\textit\{Normal\}\\mathrm\{\}\_\{\\mathrm\{multi\}\}sub\-task setting denotes multi\-turn baseline tasks under normal settings \(50 tasks\);Problemsingle\\textit\{Problem\}\\mathrm\{\}\_\{\\mathrm\{single\}\}sub\-task setting denotes single\-turn tasks with problematic settings \(56 tasks\); andProblemmulti\\textit\{Problem\}\\mathrm\{\}\_\{\\mathrm\{multi\}\}sub\-task setting denotes multi\-turn tasks with problematic settings \(100 tasks\)\. For evaluation, the benchmark employs a rigorous dual\-evaluation framework: Abstract Syntax Tree matching to verify syntactic correctness and executable testing to ensure the functional outcome matches the ground truth\. A task is considered successful only if all the rounds and steps of inference are judged to have passed by both\.

## Appendix EImplementation details

### E\.1Settings about Experience

In the main experiments, we use our designed generic tool invocation evaluation to collect experience with the 10\-dimensional vectors​c​o​r​e​sscores\. For the MetaTool dataset, the answer has only the tool name and no real execution of the tool\. Thus, we include a simple prompt “This is a special tool selection task, you only need to evaluate the Use Quality part, the Input Quality and the Response Quality should be assumed to be perfect and get all the points by default\.” that guides the agent to focus only on tool selection\. For tools in the dataset that can be executed, we use their actual execution results and returned outputs as the tool response field in metadatamm\. This returned content may be vague or even empty, which constitutes one of the challenges thatExpGis designed to handle\. For tools that cannot be executed, we instead use the ground truth as the tool response\. When validating the effectiveness of our equivalence\-class\-based selection method, since the experiences we collected all haved=10d=10, we compute the expected probability of 1 for a single dimension, and use a Bernoulli distribution\[[7](https://arxiv.org/html/2608.03403#bib.bib9)\]to simulate cases whereddtakes other values\.

### E\.2Hyperparameter QuotaQQ

Given a toolτ\\tau, we aim to select a safe maximum quota under a fixed context\-length budgetBB\(e\.g\., the maximum tokens allowed forLLMsummarizer\{\\text\{LLM\}\}\_\{\\text\{summarizer\}\}beyond the prompt\)\. Letℰτ=\{Ei\}i=1Nτ\\mathcal\{E\}\_\{\\tau\}=\\\{E\_\{i\}\\\}\_\{i=1\}^\{N\_\{\\tau\}\}be the experience pool forτ\\tau\. We define the lengthℓ​\(Ei\)\\ell\(E\_\{i\}\)of an experience as the token length of its serialized form used at inference time\. We first sort experiences by length in descending order\. Letπ\\pibe a permutation such that

ℓ​\(Eπ​\(1\)\)≥ℓ​\(Eπ​\(2\)\)≥⋯≥ℓ​\(Eπ​\(Nτ\)\)\.\\small\\ell\\\!\\left\(E\_\{\\pi\(1\)\}\\right\)\\geq\\ell\\\!\\left\(E\_\{\\pi\(2\)\}\\right\)\\geq\\cdots\\geq\\ell\\\!\\left\(E\_\{\\pi\(N\_\{\\tau\}\)\}\\right\)\.\(10\)We then compute the target selection quotaQτQ\_\{\\tau\}via a greedy procedure that iterates from the longest experience to the shortest and stops once the budget would be exceeded, as denoted:

Qτ≜max⁡\{q∈\{0,…,Nτ\}\|∑j=1qℓ​\(Eπ​\(j\)\)≤Bτ\}\.\\small Q\_\{\\tau\}\\triangleq\\max\\left\\\{q\\in\\\{0,\\ldots,N\_\{\\tau\}\\\}\\;\\middle\|\\;\\sum\_\{j=1\}^\{q\}\\ell\\\!\\left\(E\_\{\\pi\(j\)\}\\right\)\\leq B\_\{\\tau\}\\right\\\}\.\(11\)
As a result,QτQ\_\{\\tau\}is the safe maximum number of experiences that can be accommodated within budgetBB, which is used for subsequent experience distillation and experience reuse phases\.

### E\.3Use of Guidance

After obtaining guidance fromExpG, we use it in different strategies depending on its category\. For dynamic guidance, we append it to the end of the tool description using the following template: “This is optional guidance on how to better use the tool\. You may refer to it selectively: \{dynamic guidance\}\.” For stable guidance, we store it in a file and load it alongside the tool schema\. It is then inserted into the schema using the prompt: “This is guidance on using the tool learned from previous experience\. You must follow it: \{stable guidance\}\.”

### E\.4Things We Tried

During the design and evaluation ofExpG, we explored many alternative variants in the early stage\. Some did not work or did not align with our experimental setting, and here are something we can remember:Guidance insertion position\.We append the generated guidance to the end of the system prompt following the order of the tool list\. This slightly degrades performance\. We conjecture that it introduces an extra step for the model to look up the corresponding guidance for each tool from the system prompt based on the tool list\.Adding ground truth as positive trajectories intoExpG\.Acquiring experiences from ground truth yields a small improvement \(less than 1%\)\. This suggests thatExpGcan already summarize tool’s capability boundaries and best practices well without ground truth or fine\-grained feedback\.Compare standard deviation\.We compute the standard deviation over the three runs in our main experiment\. Our method achieves the smallest standard deviation compared to all other baselines in most settings, providing additional evidence thatExpGimproves the robustness of the agent’s tool use\.More epochs\.We test more epoch inference on our experimental datasets\. Each epoch allowsExpGto acquire experiences from different epoch for the same task\. We observe almost no further improvement after the second epoch\. This indicates thatExpGconverges quickly, and tasks that remain unsolved reflect the upper bound of the backbone agent and our method\.Comparison with fine\-tuning\-based methods\.We also compare our method with two available fine\-tuning\-based baselines\. They are ToolLLM\[[25](https://arxiv.org/html/2608.03403#bib.bib21)\]and a Qwen3\-8B\-sft111[https://huggingface\.co/1nstaller/qwen3\_8b\_sft\_tool\_use](https://huggingface.co/1nstaller/qwen3_8b_sft_tool_use)model released on HuggingFace\. ToolLLM achieves very low scores, nearly half of the “No Method” baseline, perhaps because its backbone LLM is outdated and does not match current tasks\. The Qwen3\-8B\-sft shows some improvement over the base model \(about 3%\), but is removed because its training details are not disclosed\.Reliability of agent judgments\.We manually inspect the guidance generation process for a subset of tools\. Although theLLMevaluator\{\\text\{LLM\}\}\_\{\\text\{evaluator\}\}can occasionally misjudge certain aspects and assign inaccurates​c​o​r​e​sscoresduring the evaluation, most of the acquired experiences \(over 95%\) are reliable and informative\. Moreover, imperfect experiences are likely to be corrected during experience distillation, sinceLLMsummarizer\{\\text\{LLM\}\}\_\{\\text\{summarizer\}\}can synthesize them from a higher\-level perspective\. Overall, 98\.7% of the resulting guidance are deemed reliable by human reviewers\. Guidance considered unreliable can also be partially removed by the experience filtering step in subsequent epochs\.More challenging tasks\.We also verify the effectiveness of our method on other benchmarks\. Under the “Oracle” setting of MCPVerse\[[15](https://arxiv.org/html/2608.03403#bib.bib49)\]using Qwen3\-32B as the backbone model, our method improves the average accuracy from 23\.00% to 26\.10%\. However, for smaller models, the gains are substantially constrained by agent capability and context limitations\. To more clearly show our method’s gains in tool use, we do not choose these challenging tasks as our main experimental benchmarks\.Alternative design variants\.We also explore several alternative variants across different phases ofExpGto better understand the impact of each design choice\. Each row replaces exactly one component of the original design without combining modifications\. The results are summarized in Table[6](https://arxiv.org/html/2608.03403#A5.T6)\.

Table 6:Ablation study of alternative variants across different phases ofExpG\. All numbers are Pass@1 on BFCL\-V3 with Qwen3\-8B\. Each row replaces one component of the original design\.PhaseVariantAccuracyReduction vs OriginalOriginalFull method67\.460\.00Experience AcquisitionDirectly use full trajectories \(no experiences\)60\.08\-7\.38Directly prompt the LLM to extract experiences63\.34\-4\.12Experience DistillationUse only the most recent 1 experience62\.26\-5\.20Keep only the most recent 3 experiences64\.64\-2\.82Remove the experience filter step65\.94\-1\.52Remove the experience summary step63\.56\-3\.90Experience ReuseOnly dynamic strategy66\.37\-1\.09Only stable strategy63\.56\-3\.90

## Appendix FCase Analysis

### F\.1ExpGImproves the Agent’s Tool\-Use Ability from Multiple Aspects

We summarize representative cases of tool\-use improvements brought by our method into six aspects\. For each aspect, we provide an illustrative example with a brief explanation\.

##### \(1\) Improve Tool\-Use Awareness\.

When the user implicitly asks about something related to a historical period, the agent tends to answer directly from its internal knowledge in a conversational manner instead of invoking thetimeporttool\.ExpGlearns from past experience that the tool should be invoked in such cases\.

![Refer to caption](https://arxiv.org/html/2608.03403v1/latex/figures/case1.png)Figure 6:Successful case of improved tool use: improve tool\-use awareness\.
##### \(2\) Distinguish Similar Tools\.

The agent usesRestaurantBookingToolto make restaurant reservations\. When the user’s request is ambiguous, the agent often confuses it with the semantically similarlocaltool, sincelocalcan be used to search for restaurants\.ExpGlearns the capability boundary from past experience, i\.e\., it should invoke the booking tool only when the user explicitly asks to reserve a table\.

![Refer to caption](https://arxiv.org/html/2608.03403v1/latex/figures/case2.png)Figure 7:Successful case of improved tool use: distinguish similar tools\.
##### \(3\) Invoke Tools Correctly\.

The agent uses theinventory\_managementtool to complete inventory management tasks\.ExpGlearns from past experience the constraints on the allowed parameter content and required formats for this tool\.

![Refer to caption](https://arxiv.org/html/2608.03403v1/latex/figures/case3.png)Figure 8:Successful case of improved tool use: invoke tools correctly\.
##### \(4\) Fix Tool\-Use Errors\.

The agent usesEmergencyKnowledgeto retrieve possible diseases, but in the first invocation is failed as the agent provides an overly descriptive input\. AfterExpGrecognizes this issue, it fixes the input formulation in subsequent invocations\.

![Refer to caption](https://arxiv.org/html/2608.03403v1/latex/figures/case4.png)Figure 9:Successful case of improved tool use: fix tool\-use errors\.
##### \(5\) Identify Tool Dependencies\.

The agent uses the file\-system toolwcto count the number of lines in a file\.ExpGlearns from past experience that the error “file does not exist” occurs frequently\. The guidance encourages the agent to runlsorfindbeforehand, which can effectively avoid such failures\.

![Refer to caption](https://arxiv.org/html/2608.03403v1/latex/figures/case5.png)Figure 10:Successful case of improved tool use: identify tool dependencies\.
##### \(6\) Understand Tool Reliability\.

From prior experiences where the agent usesQueryRegistrationto check patient appointment registrations,ExpGobserves frequent failures caused by tool availability issues and identifies them as tool\-side problems\. It therefore reminds the agent to check tool availability before invocation or to seek alternative tools\.

![Refer to caption](https://arxiv.org/html/2608.03403v1/latex/figures/case6.png)Figure 11:Successful case of improved tool use: understand tool reliability\.

### F\.2Cases WhereExpGFails

##### \(1\) Unseen Situations\.

If the agent uses an incorrect tool, or fails to use a tool when it should,ExpGcannot collect trajectories of the correct tool invocation and therefore cannot learn from it to improve\. As a result, there is no experience or guidance for that tool\. For previously seen tools,ExpGmay still fail when experience is limited: some invocation patterns or error modes may not have appeared in the past trajectories, leaving the agent without relevant guidance to prevent them\.

##### \(2\) Hard\-to\-analyze Trajectories\.

The agent invokesedit\_ticketto handle the user request: “Please edit ticket No\. 3 to contain shipping information\.” However, the agent interprets “shipping information” as including full details such as all item names, quantities, and price breakdowns, which differs from what the user expected information formatted according to a template\. As a result, even though the tool invocation is successful, the user’s request is still not satisfied\. This error is caused by the agent’s misinterpretation of the user’s intent rather than a tool invocation failure, and it falls outside the scope of issues thatExpGcan address\.

![Refer to caption](https://arxiv.org/html/2608.03403v1/latex/figures/badcase2.png)Figure 12:Case of failed tool use: hard\-to\-analyze trajectories\.
##### \(3\) Incorrect Attributions\.

The agent adjusts the device state usingCancelTimedSwitch\. Because the actual behavior of the tool is not consistent with its schema \(the schema specifies that the device id should be passed as the content of parameter “device\_name”\), the agent will never successfully invoke it\. However, the tool always returns a “device not present” response because it can’t find the device\.ExpGincorrectly attributes the failure to inappropriate behavior of the agent, as the agent did not pass in the correct device name, rather than a problem with the consistency of the tool itself\.

![Refer to caption](https://arxiv.org/html/2608.03403v1/latex/figures/badcase3.png)Figure 13:Case of failed tool use: incorrect attributions\.
##### \(4\) Over\-Reaction\.

When doctors’ names always included a prefix, the agent was able to useModifyRegistrationsuccessfully in past experience\. Consequently,ExpGinferred a piece of guidance: always use the name with a title\. However, those earlier successes occurred only because the doctor names stored in the system already contained the prefix\. When a doctor is entered with a full name rather than a titled name, the agent, following the guidance, still forcibly adds the “Dr\.” prefix, which leads to a failure\.

![Refer to caption](https://arxiv.org/html/2608.03403v1/latex/figures/badcase4.png)Figure 14:Case of failed tool use: over\-reaction\.

### F\.3Case Coverage Comparison

![Refer to caption](https://arxiv.org/html/2608.03403v1/latex/figures/overlap.png)Figure 15:Case coverage comparison with baseline methods\.Using API\-Bank as the dataset, we select one run as a representative result and compare case coverage betweenExpGand other methods\. Figure[15](https://arxiv.org/html/2608.03403#A6.F15)shows the numbers of cases that each method gets right or wrong compared to the “No Method” baseline\. In the subset where methods outperform the baseline,ExpGcovers most of the cases solved by Few\-shot and DRAFT, while exhibiting complementary gains to Mem0, possibly because Mem0 enhances the agent from a process perspective rather than focusing on tool invocation\. In the subset where methods underperform the baseline,ExpGintroduces substantially fewer error cases than the other methods\. We attribute this robustness to the experience distillation phase, whereExpGperforms an effectiveness check and filters out unhelpful experiences\.

Table 7:Overhead ofExpGon the three main benchmarks \(Qwen3\-8B, parallelism=4=4\)\.

## Appendix GCost Analysis

##### Full reproduction\.

Full reproduction requires valid API keys for the commercial LLM endpoints referenced in the Experiments section\. All backbone, evaluator, and summarizer inference is obtained through these APIs\. A CPU\-only machine is sufficient for orchestration and local preprocessing\. In our setup, the driver process, dataset input and output, logging, and a lightweight vector index for storing experiences typically remained below approximately 16 GB of RAM at peak usage\. With reduced caching or smaller concurrent batches, 8 GB of RAM is often sufficient\. However, the risk of out\-of\-memory errors increases if many trajectories are materialized simultaneously\. Wall\-clock time is primarily determined by provider latency, API rate limits, and the specific models or ablations that are enabled\. In our environment, running the full set of main experiments required approximately 12 days from start to completion\. This duration may vary substantially across different environments and API service tiers\. Reproducing all tables and baselines reported in the paper is expected to require more than8×1058\\times 10^\{5\}remote API or model calls in total\.

##### Online overhead ofExpG\.

In online use,ExpGdoes not rely on embeddings or similarity matching, and experience\-pool maintenance is almost only file storage and I/O\. On average, each experience takes about 5\.7 KB and each tool about 44 KB, while a retrieve and an update take about 47 ms and 170 ms, respectively\. The extra LLM overhead mainly comes from two calls by the evaluator and the summarizer, with an average extra prompt length of 691 words\. Table[7](https://arxiv.org/html/2608.03403#A6.T7)reports the measured overhead: each tool invocation incurs only an extra $0\.0018 and about 2\.5 seconds\. In resource\-constrained settings, we recommend usingExpGin a “generate once, reuse many times” manner: because the guidance is synthesized from multiple past experiences, it tends to be broadly applicable, making it possible to avoid these extra costs while maintaining comparable performance\.