Toollery: Scaling LLM Agents to Thousands of Skills and Tools

# Toollery: Scaling LLM Agents to Thousands of Skills and Tools
Source: [https://arxiv.org/html/2609.22218](https://arxiv.org/html/2609.22218)
###### Abstract

As LLM agents are exposed to hundreds to tens of thousands of skills, tools, and API functions, full\-library prompting becomes costly, slow, and less reliable: each added candidate increases prompt tokens and latency, while longer candidate lists introduce more distractors for LLM selection\. We presentToollery, a training\-free candidate\-compression framework for scalable LLM skill/tool selection\. Following established document\-side query expansion, Toollery generates user\-intent queries from each skill/tool specification and builds a retrieval index that maps real user requests to compact candidate sets before final LLM decision\-making\. By treating high\-level skills and atomic tools as selectable capabilities, Toollery can be applied to both skill libraries and tool registries\. We evaluate Toollery on the roughly 79K\-capability SkillRouter benchmark, BFCL\-V4 with over 440 atomic tools, and 3,396 proprietary smart\-cockpit requests over 220 tools\. Across these settings, Toollery keeps online selection bounded to a compact top\-kkcandidate set and improves recall over ordinary specification retrieval\. At a fixed top\-10 budget, Toollery improves end\-to\-end selection on the cockpit dataset, and maintains comparable AST Accuracy on BFCL\-V4\. These results support Toollery as a practical candidate\-compression framework for large and evolving agent capability libraries, while showing that quality and cost gains depend on workload coverage and provider caching\.

![Refer to caption](https://arxiv.org/html/2609.22218v1/figs/figure1_new.jpg)Figure 1:Overview of Toollery\.Toollery constructs a manual by generating user\-intent queries from each capability specification\. At inference time, it retrieves a compact candidate set using the manual and passes only the retrieved candidate specifications to the final LLM for decision making\.## 1Introduction

LLM agents increasingly rely on external resources to complete user requests: a resource may be a high\-level procedural skill, an MCP service, an atomic tool, or an API function\([Schick et al\., 2023](https://arxiv.org/html/2609.22218#bib.bib4);[Qin et al\., 2024](https://arxiv.org/html/2609.22218#bib.bib8);[Patil et al\., 2024](https://arxiv.org/html/2609.22218#bib.bib5)\)\. Although these resources differ in granularity and execution interface, they play the same role before execution: the agent must decide whether each one should be exposed to the final LLM for the current user request\. We therefore treat them as selectable*capabilities*throughout this paper\. As capability libraries grow, skill/tool selection becomes a production bottleneck\. Industrial reports and our own agent deployments show that loading skill/tool specifications upfront can consume tens of thousands of tokens before any user work begins, while selection accuracy degrades as the library grows\([Anthropic, 2026](https://arxiv.org/html/2609.22218#bib.bib17);[Anthropic, 2025](https://arxiv.org/html/2609.22218#bib.bib18)\)\. In our deployment, after loading more than 200 skills and tools, even a trivial greeting can consume more than 50K input tokens and over 5 seconds of response latency\. Scalable LLM skill/tool selection is therefore necessary for controlling token cost, latency, and accuracy as public ecosystems aggregate increasingly large capability libraries\([OpenClaw, 2026](https://arxiv.org/html/2609.22218#bib.bib16);[Lin et al\., 2025](https://arxiv.org/html/2609.22218#bib.bib9);[Mo et al\., 2025](https://arxiv.org/html/2609.22218#bib.bib13);[Xu et al\., 2025](https://arxiv.org/html/2609.22218#bib.bib14)\)\.

The common practice is full\-library prompting: place every skill/tool specification in the prompt and let the LLM choose\. This preserves the full decision context, but makes the agent pay the cost of reviewing every available capability on every request, even when the request needs no tool\. As a result, easy or irrelevant requests still incur high prompt cost, higher latency, and more distractors for final LLM decision making\. Practical agents therefore need a candidate\-compression layer that narrows the action space before LLM inference\.

Retrieval is a natural way to implement candidate compression, but ordinary specification retrieval is limited\. Existing retrieval\-based methods commonly retrieve candidates directly from skill/tool specifications, including names, descriptions, and schemas\([Lin et al\., 2025](https://arxiv.org/html/2609.22218#bib.bib9);[Zheng et al\., 2026](https://arxiv.org/html/2609.22218#bib.bib10)\)\. This assumes that user requests resemble specifications\. In practice, users describe goals, constraints, or partial tasks, while specifications describe capabilities, parameters, and developer\-facing usage details\. This intent–specification mismatch is especially problematic for heterogeneous libraries whose capabilities are written at different levels of abstraction\.

We presentToollery111Code:[https://github\.com/XiangxiTian/toollery](https://github.com/XiangxiTian/toollery), a training\-free framework that frames scalable LLM skill/tool selection asintent\-level retrieval\. Rather than retrieving over specifications directly, Toollery asks what kinds of user requests should select each capability\. It treats high\-level skills, MCP services, atomic tools, and API functions as selectable capabilities under the same abstraction\. Toollery generates user\-intent queries from each capability specification, retrieves a compact top\-kkcandidate set using these intent\-augmented representations, and forwards only the selected candidates to the final LLM\. This separates library\-scale matching from schema\-aware final LLM decision making\.

Our experiments show that Toollery turns the growing skill/tool\-selection bottleneck into a bounded retrieval problem\. On the SkillRouter benchmark, BFCL\-V4, and a proprietary smart cockpit dataset, candidate pools range from hundreds of tools to roughly 79K capabilities, yet Toollery keeps online LLM selection bounded to a compact top\-kkcandidate set\. This reduces the number of specifications exposed to the final LLM, keeping token cost and latency controllable as the library grows\. At the same time, the retrieved top\-kksets achieve higher recall than ordinary specification retrieval; under the same top\-10 candidate budget, Toollery improves final tool\-set selection on the cockpit dataset while maintaining comparable BFCL AST Accuracy\.

This paper makes the following contributions:

- •We treat high\-level skills, MCP services, atomic tools, and API functions as selectable capabilities under a unified abstraction, and study their selection through intent\-level candidate compression\.
- •We present Toollery, a training\-free framework that generates user\-intent queries offline and retrieves compact top\-kkcandidate sets for final LLM decision making\.
- •We evaluate Toollery across large\-scale skill routing, public atomic\-tool benchmarks, and proprietary vehicle\-control data, showing higher retrieval recall, reduced prompt context, and improved fixed\-budget cockpit selection\.

## 2Related Work

Prior work improves LLM tool use by training or evaluating models that invoke external tools once relevant tools are available in context or training data\([Yao et al\., 2022](https://arxiv.org/html/2609.22218#bib.bib3);[Schick et al\., 2023](https://arxiv.org/html/2609.22218#bib.bib4);[Patil et al\., 2024](https://arxiv.org/html/2609.22218#bib.bib5);[Qin et al\., 2024](https://arxiv.org/html/2609.22218#bib.bib8);[Li et al\., 2023](https://arxiv.org/html/2609.22218#bib.bib6);[Patil et al\., 2025](https://arxiv.org/html/2609.22218#bib.bib7)\)\. Planning\-oriented systems such as PLUTO and ToolGen address related orchestration problems\([Huang et al\., 2024](https://arxiv.org/html/2609.22218#bib.bib20);[Wang et al\., 2025](https://arxiv.org/html/2609.22218#bib.bib22)\); Toollery instead addresses selection of a compact candidate set before final LLM decision\-making\.

Recent retrieval and routing methods reduce the action space before tool selection\([Lin et al\., 2025](https://arxiv.org/html/2609.22218#bib.bib9);[Zheng et al\., 2026](https://arxiv.org/html/2609.22218#bib.bib10);[Paramanayakam et al\., 2025](https://arxiv.org/html/2609.22218#bib.bib12)\)\. Re\-Invoke and related query\-generation work generate tool\-use examples for retrieval\([Chen et al\., 2024](https://arxiv.org/html/2609.22218#bib.bib19);[Kachuee et al\., 2025](https://arxiv.org/html/2609.22218#bib.bib21)\); MassTool and SkillRouter learn or combine retrieval signals at scale; and RAG\-MCP\-style systems retrieve tool specifications\([Gan and Sun, 2025](https://arxiv.org/html/2609.22218#bib.bib11)\)\. Dynamic ReAct and dynamic system instructions expose tools selectively during reasoning\([Gaurav et al\., 2025](https://arxiv.org/html/2609.22218#bib.bib23);[Franko, 2026](https://arxiv.org/html/2609.22218#bib.bib24)\)\. MCP Compressor instead controls runtime schema disclosure and is complementary to candidate retrieval; we do not claim superiority without a matched end\-to\-end comparison\([Atlassian Labs, 2026](https://arxiv.org/html/2609.22218#bib.bib25)\)\.

Query and document expansion have been shown to reduce vocabulary mismatch in retrieval\([Gao et al\., 2023](https://arxiv.org/html/2609.22218#bib.bib2);[Gospodinov et al\., 2023](https://arxiv.org/html/2609.22218#bib.bib1)\)\. Toollery follows a similar approach but differs in a more agentic direction \- we operationalize expansion as a capability\-conditioned manual construction layer rather than as generic document augmentation\. Toollery does not propose a new retriever; its contribution is a deployment\-oriented candidate\-compression pipeline that uses generated intent manuals to bridge user requests and heterogeneous skill/tool specifications before final LLM selection\.

Retrieval SettingRetrieverOnline ms/queryHit@1Recall@10Recall@50MRR@10nDCG@10FullCoverage@10Raw specificationBM25309\.40\.5530\.5710\.7260\.6250\.5120\.413Raw specificationRAGAnything38210\.60\.6000\.5680\.7120\.6630\.5000\.393Raw specificationSkillRouter†495\.80\.7400\.7040\.8140\.7910\.6700\.520Toollery \(manual only\)BM25218\.90\.8400\.8800\.9510\.8810\.8460\.760Toollery \(manual only\)RAGAnything21505\.90\.9000\.9220\.9530\.9250\.8820\.827Toollery \(manual \+ specification\)BM25623\.00\.9200\.9420\.9680\.9420\.8800\.853Toollery \(manual \+ specification\)RAGAnything41070\.50\.9330\.9000\.9500\.9430\.8430\.827

Table 1:SkillRouter benchmark retrieval over roughly 79K skills, averaged over the easy and hard splits\. Raw specification retrieves from original skill specifications only\. Full\-library prompting at this scale would require roughly 43\.1M input tokens per request; Toollery top\-10 prompting requires roughly 27\.7K tokens, over three orders of magnitude smaller\.†denotes the SkillRouter reference using SR\-Emb\-0\.6B×\\timesSR\-Rank\-0\.6B\.
## 3The Toollery Design

Toollery separates full\-library search from final LLM decision\-making\. Figure[1](https://arxiv.org/html/2609.22218#S0.F1)shows the overall pipeline: Toollery first generates user\-intent queries for each selectable capability, then uses a standard retriever at inference time to retrieve a compact candidate set for the final LLM\. Throughout the paper, we use*capability*as the general unit of selection: a capability may be a high\-level procedural skill, an MCP service, an atomic tool, or an API function\. Toollery can retrieve from either generated user\-intent queries alone or skill/tool specifications augmented with those generated queries; the latter preserves both user\-facing intent signals and developer\-facing specification details\.

### 3\.1Problem Formulation

Let𝒜=\{a1,…,aM\}\\mathcal\{A\}=\\\{a\_\{1\},\\ldots,a\_\{M\}\\\}denote the full library of available capabilities, where each capability may be a high\-level skill, an MCP service, an atomic tool, or an API function\. Each capabilityaia\_\{i\}has a specificationsis\_\{i\}, which may include its name, description, schema, instruction, usage constraints, examples, or documentation\. Given a user requestxx, the goal is to construct a compact candidate set

𝒞k​\(x\)⊂𝒜,\|𝒞k​\(x\)\|=k≪M,\\mathcal\{C\}\_\{k\}\(x\)\\subset\\mathcal\{A\},\\qquad\|\\mathcal\{C\}\_\{k\}\(x\)\|=k\\ll M,\(1\)that contains the capability or capabilities needed to satisfy the request\. The final LLM then receivesxxand the specifications of only the candidates in𝒞k​\(x\)\\mathcal\{C\}\_\{k\}\(x\), and produces the final selection and any required arguments\.

The main difficulty is the mismatch between user requests and specifications\. Users describe goals, constraints, or partial tasks, while specifications describe capabilities, parameters, and usage details\. Toollery addresses this mismatch by constructing a manual for the full library: a set of generated user\-intent queries that describe when each capability should be selected\.

### 3\.2Manual Construction

For each capabilityaia\_\{i\}, a generator modelGGproducesNNuser\-intent queries from its specification:

Qi=G⁡\(si,N,π\),Q\_\{i\}=G\(s\_\{i\};N,\\pi\),\(2\)whereπ\\piprompts the generator to write realistic user requests that would requireaia\_\{i\}\. The manual for the full library is the union of the generated user\-intent queries for all capabilities:

ℳ=\{\(q,ai\):q∈Qi,ai∈𝒜\}\.\\mathcal\{M\}=\\\{\(q,a\_\{i\}\):q\\in Q\_\{i\},\\ a\_\{i\}\\in\\mathcal\{A\}\\\}\.\(3\)Each manual entry stores a generated user\-intent query and the identifier of the capability whose specification produced it\. We keep specifications and generated queries as separate provenance\-linked fields; the final LLM receives only the authoritative specifications\. Unless otherwise stated, we generateN=3N=3queries per capability\. Generation uses only capability specifications and metadata, finishes before test retrieval, and receives no evaluation requests, demonstrations, or labels\.

Toollery optionally applies an offline source\-capability recovery filter to improve manual quality\. For each generated user\-intent query, a verifier is asked to identify the source capability from a pool containing that capability and eight randomly sampled distractors\. Toollery keeps the query only when the verifier recovers the source capability, filtering out intents that are too broad or non\-discriminative\. This step uses no evaluation requests or gold labels, so it acts as an offline quality\-control step for generated intents rather than a benchmark\-specific reranker\. Appendix[A](https://arxiv.org/html/2609.22218#A1)provides implementation details, Appendix[A\.1](https://arxiv.org/html/2609.22218#A1.SS1)gives the prompt template, and Appendix[A\.3](https://arxiv.org/html/2609.22218#A1.SS3)shows a concrete walkthrough\.

### 3\.3Retrieval over the Manual

At inference time, Toollery retrieves candidates using the user requestxxand the manualℳ\\mathcal\{M\}\. The retriever is not tied to a specific implementation: it can be instantiated with sparse lexical retrieval such as BM25, dense retrieval, or a RAG\-style retriever\. In this work, we provide two Toollery retrieval settings\. The first setting scores generated user\-intent queries in the manual directly, and the second setting scores specifications augmented with the corresponding user\-intent queries from the manual\.

When retrieval is performed over manual entries, multiple retrieved user\-intent queries may point to the same capability\. Toollery therefore aggregates query\-level evidence into capability\-level scores\. Letf⁡\(x,q\)f\(x,q\)be the retrieval score between user requestxxand a generated user\-intent queryqq\. Toollery ranks each capability by aggregating the strongest matching queries associated with that capability:

S⁡\(x,ai\)=Agg⁡\{f⁡\(x,q\):\(q,ai\)∈ℳ\}\.S\(x,a\_\{i\}\)=\\operatorname\{Agg\}\\\{f\(x,q\):\(q,a\_\{i\}\)\\in\\mathcal\{M\}\\\}\.\(4\)The candidate set is then

𝒞k​\(x\)=TopKai∈𝒜⁡S⁡\(x,ai\)\.\\mathcal\{C\}\_\{k\}\(x\)=\\operatorname\{TopK\}\_\{a\_\{i\}\\in\\mathcal\{A\}\}S\(x,a\_\{i\}\)\.\(5\)In our implementation,Agg\\operatorname\{Agg\}averages the strongest retrieved query scores for each capability and adds a small evidence\-count bonus; exact aggregation parameters are listed in Appendix[A](https://arxiv.org/html/2609.22218#A1)\.

### 3\.4Candidate Compression and Final Selection

For each selected capability, Toollery keeps only the specification fields required for the final LLM decision\. The final LLM receives the original user request and the compact specifications for𝒞k​\(x\)\\mathcal\{C\}\_\{k\}\(x\), rather than the full library or the manual; retrieval narrows the action space, while the LLM resolves residual ambiguity, selects the final capability, and fills arguments\.

This design changes the online cost from full\-library prompting to subset prompting\. LetMMbe the number of capabilities andLsL\_\{s\}the average compact\-specification length\. Full\-library prompting sends approximatelyO⁡\(M⋅Ls\)O\(M\\cdot L\_\{s\}\)specification tokens, so prompt cost grows linearly with the library\. Toollery instead sends only the retrieved top\-kkspecifications, approximatelyO⁡\(k⋅Ls\)O\(k\\cdot L\_\{s\}\)tokens, wherek≪Mk\\ll Mis a deployment\-time candidate budget\. Becausekkcan remain fixed asMMgrows, the final LLM sees a bounded candidate set while the manual and retrieval index absorb library growth offline\.

Retrieval SettingRetrieverOnline ms/queryHit@1Recall@10Recall@50MRR@10nDCG@10FullCoverage@10Raw specificationBM253\.60\.6430\.9310\.9760\.7440\.7900\.931Raw specificationRAGAnything152\.50\.7370\.9760\.9950\.8310\.8670\.976Toollery \(manual only\)BM250\.30\.4680\.8220\.9340\.5840\.6420\.822Toollery \(manual only\)RAGAnything81\.80\.5430\.8550\.9290\.6530\.7020\.855Toollery \(manual \+ specification\)BM255\.00\.7310\.9480\.9860\.8110\.8450\.948Toollery \(manual \+ specification\)RAGAnything139\.70\.7760\.9800\.9970\.8560\.8870\.980

Table 2:BFCL\-V4 live multiple retrieval over the complete 457\-tool pool\.Retrieval SettingRetrieverOnline ms/queryHit@1Recall@10Recall@50MRR@10nDCG@10FullCoverage@10Raw specificationBM2514\.40\.7800\.9700\.9950\.8540\.8830\.970Raw specificationRAGAnything524\.60\.8251\.0001\.0000\.8950\.9221\.000Toollery \(manual only\)BM250\.20\.5550\.8850\.9900\.6680\.7210\.885Toollery \(manual only\)RAGAnything77\.90\.6050\.9250\.9850\.7220\.7720\.925Toollery \(manual \+ specification\)BM2525\.90\.8151\.0001\.0000\.8870\.9151\.000Toollery \(manual \+ specification\)RAGAnything657\.30\.8451\.0001\.0000\.9040\.9281\.000

Table 3:BFCL\-V4 non\-live multiple retrieval over the complete 443\-tool pool\.DatasetSelection Setting\# CandidatesInput TokensLatencyAccuracyBFCL liveFull\-library prompt45784\.1K11\.8s\.770BFCL liveRaw specification top\-10102\.7K2\.8s\.834BFCL liveToollery top\-10102\.1K1\.6s\.837BFCL non\-liveFull\-library prompt44357\.0K8\.4s\.703BFCL non\-liveRaw specification top\-10101\.9K2\.2s\.825BFCL non\-liveToollery top\-10101\.3K1\.5s\.830CockpitRaw specification top\-10102\.2K3\.5s\.163CockpitToollery top\-10101\.9K2\.8s\.268

Table 4:End\-to\-end selection under full\-library prompting and fixed top\-10 retrieval\. The BFCL full\-library rows retain the DeepSeek V3\.2 results as scaling references, whereas all fixed top\-10 rows use DeepSeek\-V4\-Pro\. BFCL reports official AST Accuracy and Cockpit reports exact\-set accuracy\. Input tokens and latency are per\-call means\.

## 4Experiments

We evaluate Toollery as a candidate compression layer for scalable capability selection\. The experiments test whether Toollery preserves the needed capabilities in compact top\-kkcandidate sets and whether this compression improves end\-to\-end LLM selection at a matched candidate budget\.

### 4\.1Experimental Setup

#### Benchmarks\.

We evaluate Toollery on two public benchmarks and one proprietary industrial dataset\. The SkillRouter benchmark\([Li et al\., 2026](https://arxiv.org/html/2609.22218#bib.bib15);[Zheng et al\., 2026](https://arxiv.org/html/2609.22218#bib.bib10)\)contains 75 evaluation queries: 24 single\-skill queries and 51 multi\-skill queries, each evaluated against a roughly 79K\-capability pool\. BFCL\-V4\([Patil et al\., 2025](https://arxiv.org/html/2609.22218#bib.bib7)\)provides 200 non\-live multiple requests over 443 tools and 1,053 live multiple requests over 457 tools\. Finally, we evaluate on 3,396 real proprietary smart\-cockpit requests over a 220\-tool production pool; half contain two gold tools and half contain three\. This last setting measures unordered multi\-intent set coverage, not dependent multi\-step execution\.

#### Settings and baselines\.

We compare retrieval settings that differ in what information is available to the retriever\.*Raw specification*retrieves only from original specifications\. Toollery evaluates*manual only*and*manual \+ specification*representations using BM25 or RAGAnything\. For the direct document\-expansion comparison, we independently reimplement Re\-Invoke’s expansion and retrieval path with ten queries per capability and disable its online intent LLM\([Chen et al\., 2024](https://arxiv.org/html/2609.22218#bib.bib19)\)\. This is an expansion\-only baseline under the same no\-online\-generation, top\-10 serving contract, not a reproduction of the complete Re\-Invoke system\.

#### Metrics and cost\.

For retrieval, we report Hit@1, Recall@10/50, MRR@10, nDCG@10, FullCoverage@10, and Avg ms/query for warmed online retrieval latency\. Recall is the fraction of gold capabilities retrieved; FullCoverage equals one only when every gold capability appears in the top 10\. The reported values are averages over all requests\. Cockpit end\-to\-end accuracy requires the unordered predicted tool set to equal the gold set\. BFCL AST Accuracy uses the official category\-specific checker for call count, names, and arguments\. Retrieval misses and invalid outputs remain in every denominator\.

Offline manual construction cost is reported in Appendix[A](https://arxiv.org/html/2609.22218#A1)\. Using official DeepSeek\-V4\-Pro prices as of June 9, 2026,222[https://api\-docs\.deepseek\.com/quick\_start/pricing](https://api-docs.deepseek.com/quick_start/pricing), accessed June 9, 2026\.construction costs approximately $0\.22 per 100 skills on average, including generated\-query construction and source\-capability recovery filtering\. This offline cost is excluded from the online retrieval latency numbers in the main tables\.

DataRaw specificationRe\-Invoke†Toollery\(N=0N=0\)\(N=10N=10\)\(N=3N=3\)Cockpit\.545 / \.224\.593 / \.240\.684 / \.381BFCL non\-live1\.000 / 1\.0001\.000 / 1\.0001\.000 / 1\.000BFCL live\.976 / \.976\.976 / \.976\.980 / \.980

Table 5:Retrieval coverage under the same top\-10 candidate budget\. Each cell reports Recall@10 / FullCoverage@10, andNNis the number of generated queries per capability\. Cockpit uses BM25, whereas BFCL uses Qwen3\-Embedding\-0\.6B dense retrieval\.†Expansion\-only Re\-Invoke reimplementation without its online intent model\.Retrieval SettingRetrieverOnline ms/queryHit@1Recall@10Recall@20Recall@50MRR@10nDCG@10FullCoverage@10Raw specificationBM256\.40\.4300\.5450\.6960\.8670\.5760\.4460\.224Raw specificationRAGAnything144\.50\.4080\.5820\.7180\.8710\.5880\.4640\.229Toollery \(manual only\)BM250\.70\.4810\.6700\.7840\.9180\.6480\.5440\.358Toollery \(manual only\)RAGAnything221\.60\.4530\.6660\.7810\.9020\.6330\.5380\.357Toollery \(manual \+ specification\)BM2569\.70\.4840\.6840\.8030\.9260\.6540\.5530\.381Toollery \(manual \+ specification\)RAGAnything253\.40\.4030\.6300\.7580\.8930\.5910\.5000\.316

Table 6:Proprietary smart cockpit retrieval over the complete 220\-tool pool\. Results are averaged over 3,396 real company\-internal requests with two or three gold tools\. The proprietary dataset is used only for deployment\-facing evaluation and is not released\.

### 4\.2Skill Retrieval on the SkillRouter Benchmark

The SkillRouter benchmark tests Toollery at a scale where full\-library prompting is infeasible\. Prompting over the full ~79K\-capability library would require roughly 43M input tokens per request, while Toollery top\-10 prompting requires about 27\.7K tokens\. The question is therefore whether retrieval can compress this full library into a small candidate set without losing the needed capabilities\.

Table[1](https://arxiv.org/html/2609.22218#S2.T1)shows that Toollery substantially improves large\-library retrieval\. With BM25, Toollery \(manual \+ specification\) improves Hit@1 from 0\.553 to 0\.920, Recall@10 from 0\.571 to 0\.942, and FullCoverage@10 from 0\.413 to 0\.853 over raw specification retrieval\. The paired FullCoverage improvement is 0\.440 \(95% CI \[0\.340, 0\.547\]\) across 75 requests\. The table’s latency column reports retrieval time only; with BM25, Toollery performs this ~79K\-capability compression in 623\.0 ms/query\.

### 4\.3Atomic\-Tool Retrieval and End\-to\-End Selection on BFCL\-V4

BFCL\-V4 tests whether the same manual\-based design transfers from high\-level skills to atomic tools and API functions\. Unlike SkillRouter, BFCL\-V4 is small enough to use full\-library prompting as a scaling reference, but our primary quality comparison holds the retrieved candidate budget at ten\. Tables[2](https://arxiv.org/html/2609.22218#S3.T2)and[3](https://arxiv.org/html/2609.22218#S3.T3)evaluate retrieval over the complete BFCL\-live 457\-tool pool and BFCL non\-live 443\-tool pool\.

Toollery remains effective in this atomic\-tool setting\. With RAGAnything, Toollery \(manual \+ specification\) reaches 0\.776 Hit@1 and 0\.980 Recall@10 on BFCL\-live, and 0\.845 Hit@1 and 1\.000 Recall@10 on BFCL non\-live\. The manual\-only setting slightly underperforms the raw specification retrieval because the atomic tool specs are already short and schema\-structured, but the manual still improves top\-kkrecall by adding user\-intent signals; adding specifications back recovers and exceeds the baseline\.

Table[5](https://arxiv.org/html/2609.22218#S4.T5)places these results under a matched serving contract\. Raw dense retrieval, Re\-Invoke expansion, and Toollery all reach 1\.000 Recall and FullCoverage on non\-live; on live, Toollery obtains 0\.980 versus 0\.976 for both baselines\. Table[4](https://arxiv.org/html/2609.22218#S3.T4)retains the DeepSeek V3\.2 full\-library results as scaling references and compares raw\-specification retrieval with Toollery while holding the candidate budget, final selector, prompt, decoding, and evaluation population fixed\. BFCL AST Accuracy changes from 0\.825 to 0\.830 on multiple and from 0\.834 to 0\.837 on live multiple; neither difference is statistically significant\. Toollery therefore provides little quality gain once raw top\-10 retrieval is saturated\. Additional end\-to\-end scaling results appear in Appendix[C](https://arxiv.org/html/2609.22218#A3)\.

### 4\.4Atomic\-Tool Retrieval on Proprietary Dataset

The proprietary smart cockpit dataset evaluates Toollery in a production\-style multi\-intent setting, where each real user request maps to two or three gold tools from a 220\-tool pool\. Full\-library prompting over this pool would require roughly 35K input tokens per request; Toollery reduces this to about 1\.9K tokens with top\-10 candidates and about 9\.1K tokens with top\-50 candidates\.

Table[6](https://arxiv.org/html/2609.22218#S4.T6)reports the complete cockpit retrieval results, while Table[5](https://arxiv.org/html/2609.22218#S4.T5)compares raw specification retrieval, expansion\-only Re\-Invoke, and Toollery under the same top\-10 candidate budget\. Compared with raw BM25 specification retrieval, Toollery improves Recall@10 from 0\.545 to 0\.684 and FullCoverage@10 from 0\.224 to 0\.381\. Against expansion\-only Re\-Invoke, the gains are 0\.091 Recall \(95% CI \[0\.080, 0\.102\]\) and 0\.141 FullCoverage \(\[0\.123, 0\.158\]\), despite using three rather than ten generated queries\. Fixed\-budget exact tool\-set selection rises from 0\.163 to 0\.268, a 0\.105 gain \(\[0\.092, 0\.119\]\)\. The end\-to\-end gain accompanies an increase in FullCoverage@10 from 0\.224 to 0\.381\.

## 5Discussion and Conclusion

The experiments show that full\-library prompting quickly becomes a poor serving strategy as capability libraries grow\. At SkillRouter scale, it is infeasible: prompting over the full ~79K\-capability library would require roughly 43\.1M input tokens per request\. Even at smaller scales, the cost remains substantial: BFCL\-V4 requires 57\.0K–84\.1K input tokens and 8\.4–11\.8 seconds of latency over 443–457 tools, while the 220\-tool smart cockpit pool still requires roughly 35\.2K input tokens\. The deployment pressure is therefore structural: the final LLM pays for every available capability on every request, including trivial or irrelevant ones\.

Toollery addresses this bottleneck by moving semantic matching out of the final prompt and into offline manual construction plus online retrieval\. Across all three settings, Toollery keeps final selection bounded to a compact top\-kkcandidate set: SkillRouter drops from roughly 43\.1M full\-library tokens to 27\.7K top\-10 tokens; BFCL\-V4 drops from 57\.0K–84\.1K to 1\.3K–2\.1K tokens; and the proprietary dataset drops from 35\.2K to 1\.9K top\-10 tokens\. On BFCL\-V4, these reductions also translate into latency drops from 8\.4–11\.8s to 1\.5–1\.6s\.

The accuracy results show that Toollery does not buy efficiency at the expense of selection quality\. Manual\-based retrieval improves or matches raw specification retrieval across SkillRouter, BFCL\-V4, and the proprietary multi\-intent dataset, while substantially improving candidate\-set coverage for multi\-capability requests\. On BFCL\-V4, where raw top\-10 dense retrieval is already near saturation, Toollery maintains comparable fixed\-budget AST Accuracy while substantially reducing prompt context and latency\. Together, these results show that candidate compression can improve coverage where retrieval is limiting without sacrificing end\-to\-end selection quality in saturated settings\.

Taken together, these results suggest that scalable capability selection should be treated as candidate compression rather than prompting over an ever\-growing full library\. Toollery instantiates this principle by constructing a provenance\-linked manual offline, retrieving a compact candidate set online, and allowing the final LLM to decide only over authoritative retrieved specifications\. Overall, Toollery provides a practical retrieval\-and\-selection layer when paired with workload\-specific calibration and fallback behavior\.

## 6Limitations

Toollery depends on the coverage and precision of the manual\. If the generated user\-intent queries fail to cover realistic ways users ask for a capability, retrieval recall is bounded regardless of the retrieval backend\. Conversely, overly broad user\-intent queries can introduce noisy matches and increase distractor exposure in the retrieved candidate set\. Source\-capability recovery filtering reduces this risk by removing queries that do not recover their source capability under distractor competition, but it does not provide a formal correctness guarantee\. As shown in Appendix[D](https://arxiv.org/html/2609.22218#A4), remaining failures often involve functionally similar capabilities or mismatches between broad retrieved capabilities and finer\-grained benchmark gold labels; exact recall may therefore under\-credit plausible alternatives while still exposing real disambiguation challenges\.

Toollery also introduces an offline maintenance cost\. When capabilities are added, removed, or modified, the corresponding manual entries and retrieval index must be updated\. This cost is amortized across future requests and separated from online retrieval latency, but it may matter for rapidly changing capability libraries\. Our experiments cover large public skill/tool pools and one proprietary smart cockpit deployment setting; broader validation across more domains, dynamic capability updates, and interactive multi\-step agent workflows remains future work\.

## References

- Anthropic \(2025\)AnthropicIntroducing Advanced Tool Use on the Claude Developer Platform\.Note:[https://www\.anthropic\.com/engineering/advanced\-tool\-use](https://www.anthropic.com/engineering/advanced-tool-use)Published: 2025\-11\-24; accessed: 2026\-06\-09Cited by:[§1](https://arxiv.org/html/2609.22218#S1.p1.1)\.
- Anthropic \(2026\)AnthropicTool Search Tool\.Note:[https://platform\.claude\.com/docs/en/agents\-and\-tools/tool\-use/tool\-search\-tool](https://platform.claude.com/docs/en/agents-and-tools/tool-use/tool-search-tool)Accessed: 2026\-06\-09Cited by:[§1](https://arxiv.org/html/2609.22218#S1.p1.1)\.
- Atlassian Labs \(2026\)Atlassian Labsmcp\-compressor: an mcp server wrapper for reducing tokens consumed by mcp tools\.Note:[https://github\.com/atlassian\-labs/mcp\-compressor](https://github.com/atlassian-labs/mcp-compressor)Accessed: 2026\-08\-24Cited by:[§2](https://arxiv.org/html/2609.22218#S2.p2.1)\.
- Chenet al\.\(2024\)Y\. Chen, J\. Yoon, D\. S\. Sachan, Q\. Wang, V\. Cohen\-Addad, M\. Bateni, C\. Lee, and T\. PfisterRe\-invoke: tool invocation rewriting for zero\-shot tool retrieval\.InFindings of the Association for Computational Linguistics: EMNLP 2024,Miami, Florida, USA,pp\. 4705–4726\.External Links:[Link](https://aclanthology.org/2024.findings-emnlp.270/),[Document](https://dx.doi.org/10.18653/v1/2024.findings-emnlp.270)Cited by:[§2](https://arxiv.org/html/2609.22218#S2.p2.1),[§4\.1](https://arxiv.org/html/2609.22218#S4.SS1.SSS0.Px2.p1.1)\.
- Franko \(2026\)U\. FrankoDynamic system instructions and tool exposure for efficient agentic llms\.arXiv preprint arXiv:2602\.17046\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2602.17046),[Link](https://arxiv.org/abs/2602.17046)Cited by:[§2](https://arxiv.org/html/2609.22218#S2.p2.1)\.
- Gan and Sun \(2025\)T\. Gan and Q\. SunRag\-mcp: mitigating prompt bloat in llm tool selection via retrieval\-augmented generation\.arXiv preprint arXiv:2505\.03275\.Cited by:[§2](https://arxiv.org/html/2609.22218#S2.p2.1)\.
- Gaoet al\.\(2023\)L\. Gao, X\. Ma, J\. Lin, and J\. CallanPrecise zero\-shot dense retrieval without relevance labels\.InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 1762–1777\.Cited by:[§2](https://arxiv.org/html/2609.22218#S2.p3.1)\.
- Gauravet al\.\(2025\)N\. Gaurav, A\. Akarsh, A\. Ranjan, and M\. BajajDynamic react: scalable tool selection for large\-scale mcp environments\.arXiv preprint arXiv:2509\.20386\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2509.20386),[Link](https://arxiv.org/abs/2509.20386)Cited by:[§2](https://arxiv.org/html/2609.22218#S2.p2.1)\.
- Gospodinovet al\.\(2023\)M\. Gospodinov, S\. MacAvaney, and C\. MacdonaldDoc2Query–: when less is more\.InEuropean Conference on Information Retrieval,pp\. 414–422\.Cited by:[§2](https://arxiv.org/html/2609.22218#S2.p3.1)\.
- Huanget al\.\(2024\)T\. Huang, D\. Jung, V\. Kumar, M\. Kachuee, X\. Li, P\. Xu, and M\. ChenPlanning and editing what you retrieve for enhanced tool learning\.InFindings of the Association for Computational Linguistics: NAACL 2024,Mexico City, Mexico,pp\. 975–988\.External Links:[Link](https://aclanthology.org/2024.findings-naacl.61/),[Document](https://dx.doi.org/10.18653/v1/2024.findings-naacl.61)Cited by:[§2](https://arxiv.org/html/2609.22218#S2.p1.1)\.
- Kachueeet al\.\(2025\)M\. Kachuee, S\. Ahuja, V\. Kumar, P\. Xu, and X\. LiuImproving tool retrieval by leveraging large language models for query generation\.InProceedings of the 31st International Conference on Computational Linguistics: Industry Track,Abu Dhabi, UAE,pp\. 29–38\.External Links:[Link](https://aclanthology.org/2025.coling-industry.3/)Cited by:[§2](https://arxiv.org/html/2609.22218#S2.p2.1)\.
- Liet al\.\(2023\)M\. Li, Y\. Zhao, B\. Yu, F\. Song, H\. Li, H\. Yu, Z\. Li, F\. Huang, and Y\. LiAPI\-bank: a comprehensive benchmark for tool\-augmented LLMs\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,H\. Bouamor, J\. Pino, and K\. Bali \(Eds\.\),Singapore,pp\. 3102–3116\.External Links:[Link](https://aclanthology.org/2023.emnlp-main.187/),[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.187)Cited by:[§2](https://arxiv.org/html/2609.22218#S2.p1.1)\.
- Liet al\.\(2026\)X\. Li, Y\. Liu, W\. Chen, B\. You, Z\. Di, Y\. He, S\. Zheng, K\. W\. Choe, J\. Sun, S\. Wang, C\. Tao, B\. Li, X\. Zhao, H\. Geng, X\. Wu, J\. Zhou, X\. Chen, H\. Xing, Y\. Li, Q\. Zeng, D\. Wang, Y\. Wang, R\. Ben Chaim, P\. Jiang, H\. Shen, L\. Kong, X\. Liu, R\. Wang, X\. Liu, J\. Li, X\. Lan, Y\. Lin, W\. Ye, J\. He, S\. Li, Y\. Zhang, Y\. Gao, Y\. Li, Z\. Ma, L\. Jing, T\. Wang, K\. Li, Y\. Xue, H\. Lyu, Y\. He, Y\. Tian, S\. Wu, B\. Wang, Y\. Gao, B\. Chen, L\. Liu, S\. Cheng, J\. Bao, S\. Tong, S\. Xu, T\. Y\. Zhuo, T\. Ye, Q\. Qi, M\. Li, L\. Liao, Z\. Tan, C\. Shi, X\. Tang, S\. Tankasala, B\. Yuan, Y\. Qian, J\. Tu, C\. Wang, Y\. Sun, W\. Wang, A\. Taylor, Z\. Yang, C\. Guan, Z\. Dong, X\. Zhang, S\. Dillmann, H\. Lee, and D\. SongSkillsBench: benchmarking how well agent skills work across diverse tasks\.arXiv preprint arXiv:2602\.12670\.External Links:[Link](https://arxiv.org/abs/2602.12670)Cited by:[§4\.1](https://arxiv.org/html/2609.22218#S4.SS1.SSS0.Px1.p1.1)\.
- Linet al\.\(2025\)J\. Lin, X\. Wang, X\. Dai, M\. Zhu, B\. Chen, R\. Tang, Y\. Yu, and W\. ZhangMassTool: a multi\-task search\-based tool retrieval framework for large language models\.arXiv preprint arXiv:2507\.00487\.Cited by:[§1](https://arxiv.org/html/2609.22218#S1.p1.1),[§1](https://arxiv.org/html/2609.22218#S1.p3.1),[§2](https://arxiv.org/html/2609.22218#S2.p2.1)\.
- Moet al\.\(2025\)G\. Mo, W\. Zhong, J\. Chen, Q\. Yuan, X\. Chen, Y\. Lu, H\. Lin, B\. He, X\. Han, and L\. SunLiveMCPBench: can agents navigate an ocean of MCP tools?\.arXiv preprint arXiv:2508\.01780\.Cited by:[§1](https://arxiv.org/html/2609.22218#S1.p1.1)\.
- OpenClaw \(2026\)OpenClawClawHub: Skill and Plugin Registry for OpenClaw\.Note:[https://github\.com/openclaw/clawhub](https://github.com/openclaw/clawhub)Accessed: 2026\-06\-09Cited by:[§1](https://arxiv.org/html/2609.22218#S1.p1.1)\.
- Paramanayakamet al\.\(2025\)V\. Paramanayakam, A\. Karatzas, I\. Anagnostopoulos, and D\. StamoulisLess is more: optimizing function calling for llm execution on edge devices\.In2025 Design, Automation & Test in Europe Conference \(DATE\),pp\. 1–7\.Cited by:[§2](https://arxiv.org/html/2609.22218#S2.p2.1)\.
- Patilet al\.\(2025\)S\. G\. Patil, H\. Mao, F\. Yan, C\. C\. Ji, V\. Suresh, I\. Stoica, and J\. E\. GonzalezThe berkeley function calling leaderboard \(bfcl\): from tool use to agentic evaluation of large language models\.InForty\-second International Conference on Machine Learning,Cited by:[§2](https://arxiv.org/html/2609.22218#S2.p1.1),[§4\.1](https://arxiv.org/html/2609.22218#S4.SS1.SSS0.Px1.p1.1)\.
- Patilet al\.\(2024\)S\. G\. Patil, T\. Zhang, X\. Wang, and J\. E\. GonzalezGorilla: large language model connected with massive apis\.Advances in Neural Information Processing Systems37,pp\. 126544–126565\.Cited by:[§1](https://arxiv.org/html/2609.22218#S1.p1.1),[§2](https://arxiv.org/html/2609.22218#S2.p1.1)\.
- Qinet al\.\(2024\)Y\. Qin, S\. Liang, Y\. Ye, K\. Zhu, L\. Yan, Y\. Lu, Y\. Lin, X\. Cong, X\. Tang, B\. Qian, S\. Zhao, L\. Hong, R\. Tian, R\. Xie, J\. Zhou, M\. Gerstein, D\. Li, Z\. Liu, and M\. SunToolLLM: facilitating large language models to master 16000\+ real\-world APIs\.InThe Twelfth International Conference on Learning Representations,External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2024/hash/28e50ee5b72e90b50e7196fde8ea260e-Abstract-Conference.html)Cited by:[§1](https://arxiv.org/html/2609.22218#S1.p1.1),[§2](https://arxiv.org/html/2609.22218#S2.p1.1)\.
- Schicket al\.\(2023\)T\. Schick, J\. Dwivedi\-Yu, R\. Dessì, R\. Raileanu, M\. Lomeli, E\. Hambro, L\. Zettlemoyer, N\. Cancedda, and T\. ScialomToolformer: language models can teach themselves to use tools\.Advances in Neural Information Processing Systems36,pp\. 68539–68551\.Cited by:[§1](https://arxiv.org/html/2609.22218#S1.p1.1),[§2](https://arxiv.org/html/2609.22218#S2.p1.1)\.
- Wanget al\.\(2025\)R\. Wang, X\. Han, L\. Ji, S\. Wang, T\. Baldwin, and H\. LiToolGen: unified tool retrieval and calling via generation\.InThe Thirteenth International Conference on Learning Representations,External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2025/hash/b646bdebeb87dfafe2c6f77a63b564e1-Abstract-Conference.html)Cited by:[§2](https://arxiv.org/html/2609.22218#S2.p1.1)\.
- Xuet al\.\(2025\)H\. Xu, X\. Huang, Y\. Liu, K\. Yu, and Z\. DengTPS\-Bench: evaluating AI agents’ tool planning & scheduling abilities in compounding tasks\.arXiv preprint arXiv:2511\.01527\.Cited by:[§1](https://arxiv.org/html/2609.22218#S1.p1.1)\.
- Yaoet al\.\(2022\)S\. Yao, H\. Chen, J\. Yang, and K\. NarasimhanWebshop: towards scalable real\-world web interaction with grounded language agents\.Advances in Neural Information Processing Systems35,pp\. 20744–20757\.Cited by:[§2](https://arxiv.org/html/2609.22218#S2.p1.1)\.
- Zhenget al\.\(2026\)Y\. Zheng, Z\. Zhang, C\. Ma, Y\. Yu, J\. Zhu, Y\. Wu, T\. Xu, B\. Dong, H\. Zhu, R\. Huang, and G\. YuSkillRouter: skill routing for LLM agents at scale\.arXiv preprint arXiv:2603\.22455\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2603.22455),[Link](https://arxiv.org/abs/2603.22455)Cited by:[§1](https://arxiv.org/html/2609.22218#S1.p3.1),[§2](https://arxiv.org/html/2609.22218#S2.p2.1),[§4\.1](https://arxiv.org/html/2609.22218#S4.SS1.SSS0.Px1.p1.1)\.

## Appendix AReproducibility and Deployment Details

Table[7](https://arxiv.org/html/2609.22218#A1.T7)summarizes the main configuration choices used in the reported runs\. We separate offline manual construction from online retrieval and final selection: manual construction includes user\-intent query generation and optional source\-capability recovery filtering, while online latency reports only serving\-time retrieval or final\-selection latency with warmed indexes and caches\. We also plan to provide plugin adapters for popular agent runtimes, including OpenClaw, Hermes Agent, Claude Code, and Codex, so that Toollery can be used as a candidate\-compression layer in existing agent systems\.

ItemValue used in reported runsSkillRouter pools78,361 easy capabilities; 79,141 hard capabilities; 75 requests per poolBFCL\-live end\-to\-end pool457 tools; 1,053 live multiple requests; final column uses the complete poolBFCL\-non\-live end\-to\-end pool443 tools; 200 multiple requests; final column uses the complete poolManual constructionDeepSeek\-V4\-Pro through an OpenAI\-compatible API; 3 user\-intent queries per capability; seed 31; 4 workers; batch size 4Source\-capability recovery verifierasks DeepSeek\-V4\-Pro to choose the matching capability id from the source capability plus distractors, or NONE if no candidate fitsDistractor constructionverifier pool uses the source capability plus 8 randomly sampled non\-gold distractors; scaled BFCL candidate pools sample the gold tool plus non\-gold tools without replacementBM25 configurationlocal tokenizer intoollery\.text; Okapi BM25 withk1=1\.5,b=0\.75k\_\{1\}=1\.5,b=0\.75; Toollery \(manual \+ specification\) concatenates name, category, description, schema, and user\-intent queries from the manualRAGAnything configurationtext\-only RAGAnything/LightRAG cache with one compact chunk per capability; retrieval queries LightRAG chunk vectors and recovers capability ids from returned context orderRetrieval embedding modelQwen/Qwen3\-Embedding\-0\.6B for neural retrieval and latency replayAggregation parameterstop\-c=3c=3averaged manual\-query scores with evidence bonusb=5,λ=0\.05b=5,\\lambda=0\.05Candidate budgetstop\-50 for retrieval\-quality tables; top\-10 for the main BFCL\-V4 end\-to\-end cost comparison; appendix end\-to\-end table columns denote the candidate\-pool size before Toollery compressionLatency protocolwarmed indexes/caches; reports online retrieval or final\-selection latency only; excludes offline manual construction, verification, embedding, and index constructionOffline costSkillRouter easy and hard pools; DeepSeek\-V4\-Pro generator and verifier; 3 user\-intent queries per capability; source capability plus 8 random distractors for verification; cost computed with cache\-miss input pricingRelease planUpon acceptance, we will release the Toollery code, prompts, normalized manual artifacts, run configurations, metric scripts, and plugin adapters publicly\.Table 7:Reproducibility details for the main retrieval and efficiency results\. Secrets and local API endpoints are omitted; paths in the released run configurations identify the exact output artifacts\.### A\.1Manual Construction Prompt

The following prompt template is used to generate user\-intent queries for manual construction in the SkillRouter and BFCL\-V4 runs\. Each capability is serialized by its specification, including available fields such as name, description, category, schema, and usage instructions\. The generatedqueryfields are stored as manual entries associated with the source capability\.

Generate realistic user\-intent queries for multiple skills or tools\.For each skill or tool, write things real users might ask when they need that exact skill or tool\.Use practical task context, constraints, and natural wording\.Avoid copying skill text verbatim\.Return only one valid JSON object\. The keys must be the exact skill names\.Each value must be a JSON array of objects\. Each object must contain query, scenario\_type, generation\_notes\.Return exactly \{N\} objects per skill\.

The returnedqueryfields become manual entries associated with the source capability, whilescenario\_typeandgeneration\_notesare retained for auditability but are not used by the retriever\.

![Refer to caption](https://arxiv.org/html/2609.22218v1/figs/figure2.jpg)Figure 2:Toollery pipeline walkthrough\.Toollery converts capability specifications into user\-intent queries during offline manual construction\. During inference, the user request retrieves matching manual entries and corresponding capability specifications, which are aggregated into a compact candidate set for final LLM decision making\.
### A\.2Source\-Capability Recovery Filtering

For each generated user\-intent queryq∈Qiq\\in Q\_\{i\}, Toollery optionally constructs a verification pool𝒫i,q=\{ai\}∪𝒟i,q\\mathcal\{P\}\_\{i,q\}=\\\{a\_\{i\}\\\}\\cup\\mathcal\{D\}\_\{i,q\}, whereaia\_\{i\}is the source capability whose specification producedqqand𝒟i,q\\mathcal\{D\}\_\{i,q\}contains eight randomly sampled distractor capabilities from the same full library\. A verifier LLM receivesqqand the specifications of all capabilities in𝒫i,q\\mathcal\{P\}\_\{i,q\}, then selects the best matching capability or abstains\. Toollery keepsqqonly when the verifier recovers the source capabilityaia\_\{i\}\.

This filter removes user\-intent queries that are too broad or non\-discriminative before indexing\. Because filtering is performed during offline manual construction, its cost is amortized across future online requests together with query generation\.

### A\.3Pipeline Walkthrough Example

Figure[2](https://arxiv.org/html/2609.22218#A1.F2)illustrates one concrete Toollery workflow\. Starting from a capability specification, Toollery first generates user\-intent queries that describe realistic requests for which the capability should be selected\. These generated queries are stored in the manual and remain linked to the source capability\. At inference time, a real user request is matched against the manual and, optionally, the original capability specifications\. Retrieved manual entries are aggregated back to capability\-level scores, producing a compact top\-kkcandidate set\. The final LLM then receives only the user request and the retrieved candidate specifications, rather than the full library, and performs the final schema\-aware decision\. This example highlights the main serving\-time change: the final LLM no longer pays the token, latency, and distractor cost of the full library, but only reasons over a bounded set of retrieved candidate specifications\.

### A\.4Offline Manual Construction Cost

We report offline manual construction cost separately from online latency because this cost is paid before deployment and amortized across future requests\. On SkillRouter, we use DeepSeek\-V4\-Pro as both the generator and the source\-capability recovery verifier\. Each capability receives three generated user\-intent queries, and each non\-empty query is verified against its source capability plus eight randomly sampled distractors\.

Table[8](https://arxiv.org/html/2609.22218#A1.T8)reports the measured construction cost for the SkillRouter easy and hard pools\. Under official DeepSeek\-V4\-Pro cache\-miss input and output pricing,333[https://api\-docs\.deepseek\.com/quick\_start/pricing/](https://api-docs.deepseek.com/quick_start/pricing/), accessed June 9, 2026\.offline manual construction costs approximately $0\.22 per 100 capabilities on average\. These values describe offline construction only; all online latency results in the main tables use warmed indexes and exclude generation, verification, embedding, and index construction\.

SettingSkillsCand\.Ret\.Tok\.CostSkillRouter easy78,361235K228K361\.8M$168\.70SkillRouter hard79,141237K230K365\.4M$170\.38Table 8:Offline construction cost for generated\-query construction and verifier filtering\. The measured rows use the SkillRouter easy and hard pools with three generated queries per capability\. Cost uses official DeepSeek\-V4\-Pro pricing: $0\.435/M cache\-miss input tokens and $0\.87/M output tokens; cache\-hit discounts are not applied\.

## Appendix BAdditional Analyses and Deployment Stress Tests

Table[9](https://arxiv.org/html/2609.22218#A2.T9)isolates the interaction between indexed representation and retriever at fixedN=3N=3,k=10k=10, and random\-negative recovery\. Generated queries only is a retrieval ablation: Toollery still loads authoritative specifications before final selection\. Dense retrieval uses Qwen3\-Embedding\-0\.6B, and hybrid retrieval uses equal\-weight reciprocal\-rank fusion with RRFk=60k=60\.

DatasetIndexed representationBM25Dense \(Qwen3\)Hybrid RRFR@10FC@10R@10FC@10R@10FC@10BFCL non\-liveSpecification0\.9700\.9701\.0001\.0000\.9900\.990Generated queries0\.8850\.8850\.9250\.9250\.9750\.975Specification \+ generated queries1\.0001\.0001\.0001\.0001\.0001\.000BFCL liveSpecification0\.9310\.9310\.9760\.9760\.9760\.976Generated queries0\.8220\.8220\.8550\.8550\.9160\.916Specification \+ generated queries0\.9480\.9480\.9800\.9800\.9800\.980CockpitSpecification0\.5450\.2240\.5820\.2290\.6870\.371Generated queries0\.6700\.3580\.6660\.3570\.6310\.310Specification \+ generated queries0\.6840\.3810\.6300\.3160\.7090\.405SkillRouterSpecification0\.5710\.4130\.5680\.3930\.6210\.467Generated queries0\.8800\.7600\.9220\.8270\.9410\.840Specification \+ generated queries0\.9420\.8530\.9000\.8270\.9450\.867

Table 9:Retrieval ablation across indexed representations and retrievers\. Results report Recall@10 \(R@10\) and FullCoverage@10 \(FC@10\); bold marks the best result in each dataset–retriever–metric group, including ties\. SkillRouter averages the easy and hard pools, and Hybrid RRF fuses BM25 and dense rankings\.No representation or retriever is uniformly best\. Generated queries alone are weaker than specifications on BFCL, whereas the combined representation reaches 1\.000/1\.000 with BM25 and dense retrieval on BFCL non\-live\. On SkillRouter, generated queries improve substantially over specifications and hybrid retrieval is strongest at 0\.945/0\.867\. On cockpit, dense retrieval improves the specification\-only representation, BM25 is stronger for the two query\-augmented representations, and hybrid retrieval is strongest for the combined representation at 0\.709/0\.405\. Dense and hybrid retrieval add roughly 10 ms over BM25 in our local runs, so the representation and retriever should be selected on held\-out in\-domain traffic rather than treated as universally optimal\.

DatasetNNR@10FC@10Query\-texttokens \(est\.\)BFCL non\-live10\.9800\.98010,83530\.9950\.99533,58250\.9950\.99556,012100\.9950\.995111,042BFCL live10\.9320\.93211,82930\.9420\.94236,07850\.9490\.94960,189100\.9500\.950121,314Cockpit10\.5850\.26370730\.6800\.3652,45850\.6910\.3714,310100\.7080\.3918,921Table 10:Generated\-query budget sweep atk=10k=10\. Bold marks the defaultN=3N=3configuration\. Rows use nested prefixes from a separately generatedNmax=10N\_\{\\max\}=10batch; thisN=3N=3arm is therefore distinct from the main\-run artifact\.BFCL non\-live saturates atN=3N=3\. On BFCL live, FullCoverage increases from 0\.932 atN=1N=1to 0\.942 atN=3N=3and 0\.950 atN=10N=10\. Cockpit gains most from the first three queries \(0\.263 to 0\.365\); increasing toN=10N=10reaches 0\.391, an additional 0\.026 while using 3\.63×\\timesthe query\-text index size\. We therefore useN=3N=3as a compact construction\-budget choice, not a universal optimum\. Across the nested prefixes, normalized exact duplicates are zero; this check does not detect fuzzy or semantic near\-duplicates\.

FilterBFCL\-NLBFCL\-LCockpitNo filter0\.995 / 0\.9950\.941 / 0\.9410\.623 / 0\.310Random neg\.0\.995 / 0\.9950\.942 / 0\.9420\.680 / 0\.365BM25 hard neg\.0\.995 / 0\.9950\.941 / 0\.9410\.602 / 0\.288Table 11:Recovery\-filter ablation atN=3,k=10N=3,k=10with eight verifier negatives\. Cells report Recall@10 / FullCoverage@10; bold marks the best result per dataset, including ties\. BFCL\-NL and BFCL\-L denote non\-live and live\. Hard\-negative verification is diagnostic only\.Within the separately generatedNmax=10N\_\{\\max\}=10batch, random\-negative recovery is neutral on BFCL non\-live and produces the strongestN=3N=3result on BFCL live and cockpit\. The largest practical gain is on cockpit, where Recall increases from 0\.623 to 0\.680 and FullCoverage from 0\.310 to 0\.365\. Hard\-negative recovery does not improve retrieval and reduces cockpit coverage, so we treat it as a diagnostic stress test rather than a validated improvement\.

kkRecall / FCCandidate tokensms/query10\.7002540\.1530\.8646640\.1550\.9111,0630\.15100\.9422,0390\.15200\.9623,9650\.15500\.9799,7340\.17Table 12:Candidate\-budget sweep on BFCL live using combined BM25 retrieval\. Bold marks the defaultk=10k=10configuration\. Each request has one gold function, so Recall equals FullCoverage\.The nested generated\-query batches contain no normalized exact duplicates\. This check does not establish semantic diversity, so paraphrases and near\-duplicates may remain\.

Cockpit route \(k=10k=10\)R@10FC@10Held\-out hot only0\.0800\.004Raw BM250\.5450\.224Toollery0\.6840\.3813 hot \+ Toollery0\.6280\.3025 hot \+ Toollery0\.5630\.230Table 13:Five\-fold held\-out evaluation of static popularity routing\. Hot rankings are computed on four folds and evaluated on the unseen fold; bold marks the best route\.Table[13](https://arxiv.org/html/2609.22218#A2.T13)shows that, atk=10k=10, Toollery exceeds the held\-out hot\-only route by 0\.604 Recall and 0\.377 FullCoverage\. Reserving three or five slots for globally popular tools reduces FullCoverage by 0\.079 and 0\.151, respectively, because those slots displace request\-specific candidates\. A static popularity set is therefore not an effective substitute for request\-conditioned retrieval on this relatively flat, multi\-intent workload\.

### B\.1Test Isolation

Manual generation finishes before test retrieval and uses only capability specifications and metadata; no evaluation request, demonstration, or label is supplied\. Table[14](https://arxiv.org/html/2609.22218#A2.T14)reports a post\-hoc overlap audit between evaluation requests and generated queries\.

SplitRequestsCapabilitiesExactMax 5\-gram JaccardUnitSkillRouter easy7578,36100\.063tokenSkillRouter hard7579,14100\.060tokenBFCL non\-live20044300\.455tokenBFCL live1,05345700\.229tokenCockpit3,39622000\.417character

Table 14:Evaluation\-to\-generated\-query overlap audit\. Exact denotes normalized exact matches; English splits use token 5\-grams and cockpit uses character 5\-grams\.All splits have zero normalized exact matches\. On cockpit, the maximum Jaccard pair matches one capability mentioned within a multi\-intent request rather than reproducing the full request\. The data\-flow boundary is the primary isolation evidence; the overlap audit provides evidence against exact copying but does not establish semantic independence or rule out benchmark exposure during generator pretraining\.

### B\.2Cache\-Aware Replay

Cache regimeMethodCallsCandidatesASTLatencyBilled input costInput tokensHit rateNatural replayFull library600443–4570\.7873\.76 s$0\.00074987,0740\.989Toollery600100\.8402\.33 s$0\.0008192,1960\.144Cache\-suppressedFull library300443–4570\.7979\.03 s$0\.03787987,0780\.000Toollery300100\.8172\.15 s$0\.0009512,1870\.000Forced\-warm exactFull library300443–4570\.8073\.43 s$0\.00032887,0771\.000Toollery300100\.8532\.14 s$0\.0000362,1880\.970

Table 15:Controlled cache\-aware BFCL replay\. Latency, billed input cost, input tokens, and hit rate are per\-request means\. Forced\-warm values exclude the paid priming call and represent a boundary condition rather than a deployable cost forecast\.Table[15](https://arxiv.org/html/2609.22218#A2.T15)shows that caching changes the realized cost benefit of candidate compression\. In the natural replay, Toollery reduces model\-visible input by 97\.5% and mean latency by 1\.43 s, but the billed\-input\-cost difference is inconclusive: Toollery minus full library is $0\.000070 per request \(95% CI \[USD−0\.000524\-0\.000524, USD0\.0004620\.000462\]\)\. We therefore find no billed\-input\-cost advantage in this regime\. With caching suppressed, Toollery reduces billed input cost by 97\.5% and latency by 6\.88 s\. Forced\-warm exact prompts also favor Toollery, but exclude the paid priming call\. Cost savings therefore depend on cache state and traffic reuse; visible\-token reduction alone does not imply proportional monetary savings\.

Table 16:BFCL\-V4 live multiple resultsModel≤\\leq1020304050100457Qwen3\-4B \(Prompt\)85\.5086\.0088\.0082\.5078\.0069\.5024\.88Qwen3\-4B \(Prompt\) \+ Toollery87\.00⇑\\Uparrow84\.5087\.0086\.50⇑\\Uparrow82\.50⇑\\Uparrow81\.00⇑\\Uparrow83\.75⇑\\UparrowQwen3\-8B \(Prompt\)97\.0093\.5092\.0084\.0083\.0084\.5038\.50Qwen3\-8B \(Prompt\) \+ Toollery95\.5098\.50⇑\\Uparrow98\.50⇑\\Uparrow92\.50⇑\\Uparrow94\.5086\.50⇑\\Uparrow79\.38⇑\\UparrowDeepSeek V3\.2 \(Prompt\)94\.5080\.0084\.0089\.5079\.5063\.0077\.00DeepSeek V3\.2 \(Prompt\) \+ Toollery98\.00⇑\\Uparrow78\.5094\.50⇑\\Uparrow99\.50⇑\\Uparrow89\.50⇑\\Uparrow89\.00⇑\\Uparrow83\.13⇑\\UparrowA major proprietary LLM \(Prompt\)83\.0093\.0090\.0087\.0088\.0088\.0079\.75A major proprietary LLM \(Prompt\) \+ Toollery83\.50⇑\\Uparrow94\.00⇑\\Uparrow89\.5095\.00⇑\\Uparrow89\.50⇑\\Uparrow89\.00⇑\\Uparrow84\.50⇑\\UparrowA major proprietary LLM \(FC\)67\.0062\.0063\.5062\.0061\.0063\.5057\.00A major proprietary LLM \(FC\) \+ Toollery74\.50⇑\\Uparrow67\.50⇑\\Uparrow69\.00⇑\\Uparrow69\.50⇑\\Uparrow67\.00⇑\\Uparrow67\.50⇑\\Uparrow66\.00⇑\\UparrowQwen3\-8B \(FC\)93\.5094\.5092\.5083\.0083\.5083\.0060\.00Qwen3\-8B \(FC\) \+ Toollery93\.5098\.00⇑\\Uparrow90\.5099\.00⇑\\Uparrow91\.50⇑\\Uparrow88\.50⇑\\Uparrow85\.88⇑\\UparrowQwen3\-235B \(FC\)98\.0093\.0087\.5089\.5085\.5080\.0081\.13Qwen3\-235B \(FC\) \+ Toollery98\.50⇑\\Uparrow92\.5089\.00⇑\\Uparrow89\.5090\.0091\.50⇑\\Uparrow88\.63⇑\\Uparrow

Table 17:BFCL\-V4 non\-live multiple resultsModel≤\\leq1020304050100443Qwen3\-4B \(Prompt\)95\.5099\.5097\.5096\.5095\.5090\.0024\.25Qwen3\-4B \(Prompt\) \+ Toollery100\.00⇑\\Uparrow97\.5098\.00⇑\\Uparrow95\.0098\.00⇑\\Uparrow91\.50⇑\\Uparrow83\.63⇑\\UparrowQwen3\-8B \(Prompt\)97\.5098\.5099\.5097\.5097\.0089\.5037\.63Qwen3\-8B \(Prompt\) \+ Toollery99\.50⇑\\Uparrow100\.00⇑\\Uparrow97\.5096\.0098\.00⇑\\Uparrow91\.00⇑\\Uparrow80\.63⇑\\UparrowDeepSeek V3\.2 \(Prompt\)86\.0099\.5097\.0094\.5096\.5085\.5070\.25DeepSeek V3\.2 \(Prompt\) \+ Toollery97\.50⇑\\Uparrow97\.5095\.0098\.00⇑\\Uparrow98\.50⇑\\Uparrow89\.50⇑\\Uparrow81\.38⇑\\UparrowA major proprietary LLM \(Prompt\)88\.0087\.5092\.5085\.0088\.5088\.0080\.00A major proprietary LLM \(Prompt\) \+ Toollery88\.0089\.50⇑\\Uparrow91\.5086\.50⇑\\Uparrow95\.00⇑\\Uparrow90\.00⇑\\Uparrow82\.00⇑\\UparrowA major proprietary LLM \(FC\)79\.5069\.5065\.5072\.0070\.5063\.5058\.63A major proprietary LLM \(FC\) \+ Toollery81\.50⇑\\Uparrow77\.00⇑\\Uparrow75\.50⇑\\Uparrow73\.00⇑\\Uparrow76\.00⇑\\Uparrow69\.50⇑\\Uparrow68\.63⇑\\UparrowQwen3\-8B \(FC\)97\.5097\.5097\.5098\.5096\.0085\.0060\.75Qwen3\-8B \(FC\) \+ Toollery99\.50⇑\\Uparrow96\.5097\.5096\.5097\.00⇑\\Uparrow90\.50⇑\\Uparrow86\.75⇑\\UparrowQwen3\-235B \(FC\)95\.5099\.5096\.5096\.0097\.5080\.0079\.13Qwen3\-235B \(FC\) \+ Toollery96\.00⇑\\Uparrow99\.0098\.50⇑\\Uparrow98\.00⇑\\Uparrow98\.00⇑\\Uparrow93\.50⇑\\Uparrow94\.00⇑\\Uparrow

## Appendix CAdditional End\-to\-End Results

Tables[16](https://arxiv.org/html/2609.22218#A2.T16)and[17](https://arxiv.org/html/2609.22218#A2.T17)provide additional BFCL\-V4 end\-to\-end selection results beyond the main DeepSeek V3\.2 comparison in Table[4](https://arxiv.org/html/2609.22218#S3.T4)\. We vary the final selector, the decision interface, and the candidate\-pool size before Toollery compression\. The BFCL\-V4 live multiple split contains 1,053 requests and 457 unique tools, while the non\-live multiple split contains 200 requests and 443 unique tools; therefore, the final columns correspond to the complete candidate pools for the two splits\.

These results test whether candidate compression remains useful beyond a single final LLM\. Across prompt\-based and function\-calling settings, Toollery is most helpful when the candidate pool is large, where full\-library prompting exposes the final LLM to more tool specifications and distractors\. The trend supports the main result: Toollery improves the serving tradeoff by retrieving a compact candidate set before final LLM decision\-making, rather than asking the final LLM to select from the full library\.

DatasetQuery / taskGold capabilityTop\-5 retrievalSkillRouterYou need to fix build errors in a Java codebase\. The repository is located in …gt/maven\-build\-lifecyclegt/maven\-dependency\-managementgt/maven\-plugin\-configurationgt/analyze\-cigt/uv\-package\-managerdevelopment/ci\-debugother/creating\-commitother/github\-pr\-fixSkillRouterBuild an itinerary for the user according to the following requirements: We require a 7\-day travel itinerary for two leaving from Minneapolis …gt/search\-accommodationsgt/search\-attractionsgt/search\-citiesgt/search\-driving\-distancegt/search\-flightsgt/search\-restaurantsother/travel\-planner\-ai\-claude\-skillsother/tour\-operationsgt/search\-accommodationsdevelopment/travel\-plannerother/travel\-requirements\-expertSkillRouterIn this task, you are given a clip of screen recording of a player playing the game Super Mario\. Your goal is to analyze the key frames in this video and count how many coins / enemies / turtles ever showing up in each frame and write the results into a CSV file\.gt/ffmpeggt/image\_editinggt/object\_countergt/object\_countergt/ffmpeggt/videoframeextractionother/videoprocessingdata/videoframereaderBFCL\-liveI’m having trouble understanding how to use the list\_files functionality\. Help me\.dartfx\_helplist\_fileslist\_directory\_contentshelplist\_projectsadriel\_list\_projectsBFCL\-livesearch flight prices from NYC to NEw Delhi use tools, finally respond in jsonsearch\_web\_toolget\_tripFlights\_4\_SearchRoundtripFlightsFlights\_4\_SearchOnewayFlightrender\_ui\_finishTrains\_1\_GetTrainTicketsBFCL\-liveFind a recent news article about corona virussearch\_web\_toolsearch\_engine\.queryget\_news\_reportHNA\_WQA\.searchHNA\_WQA\.queryHNA\_NEWS\.searchTable 18:Representative Toollery failure cases\. Skill\-routing failures often miss implementation\-specific or complementary capabilities, while BFCL\-live failures often arise from near\-duplicate tool interfaces or underspecified requests\.
## Appendix DFailure Case Analysis

Table[18](https://arxiv.org/html/2609.22218#A3.T18)shows representative retrieval failures from SkillRouter and BFCL\-live\. These cases suggest that many Toollery failures are near misses rather than unrelated retrieval errors\. In SkillRouter, failures often reflect two intertwined issues: incomplete coverage of multi\-capability workflows and mismatch between the benchmark’s exact gold capability set and functionally similar retrieved capabilities\. In BFCL\-live, failures more often arise from fine\-grained ambiguity among atomic tool interfaces, where the retrieved tools belong to the right tool family but do not match the exact gold interface\.

#### Functionally similar capabilities can fail exact\-gold matching\.

Several failures in Table[18](https://arxiv.org/html/2609.22218#A3.T18)are near misses under exact\-label evaluation\. In the SkillRouter build\-repair case, the benchmark gold capabilities are Maven\-specific, includinggt/maven\-build\-lifecycle,gt/maven\-dependency\-management, andgt/maven\-plugin\-configuration\. Toollery instead retrieves generic CI and debugging capabilities such asgt/analyze\-ci,gt/uv\-package\-manager, anddevelopment/ci\-debug, which are plausible for repairing build failures but do not match the exact gold labels\. BFCL\-live shows the same issue at the atomic\-tool level: for a request asking for help withlist\_files, Toollery retrieveslist\_files,list\_directory\_contents, and generic help tools, but misses the annotated gold tooldartfx\_help\. For flight\-price and news\-search requests, Toollery retrieves flight\-specific, news\-specific, or search\-like tools, but misses the goldsearch\_web\_tool\. These cases suggest that some retrieval failures reflect exact\-interface or exact\-label mismatch rather than completely unrelated retrieval\.

#### Broad capabilities can substitute for decomposed gold labels\.

The travel\-planning case illustrates a capability\-granularity mismatch\. The user request asks for an end\-to\-end itinerary, and Toollery retrieves broad travel\-planning capabilities such as travel\-planner\-style agents, along with one exact gold capability,gt/search\-accommodations\. The benchmark gold set, however, decomposes the task into several fine\-grained capabilities, including attraction search, driving\-distance search, flight search, and restaurant search\. Under exact recall, the broad travel\-planner capability receives no credit for covering these subfunctions, even if it may be able to implement a complete itinerary workflow\. A similar pattern appears in the Super Mario video\-analysis case: Toollery retrieves broad video\-processing or task\-level capabilities together withgt/object\_counterandgt/ffmpeg, but misses the exact goldgt/image\_editing\. These cases show that some failures reflect mismatch between the granularity of retrieved capabilities and the granularity of benchmark annotations\.

Overall, the failure cases show two main boundary conditions for Toollery\. First, in large capability libraries, multiple capabilities may be functionally overlapping, so exact\-label evaluation can count feasible alternatives as failures\. Second, for compositional tasks, a broad retrieved capability may cover an end\-to\-end workflow while the benchmark gold set decomposes the task into finer\-grained capabilities\. Future improvements should therefore consider interface\-aware reranking for near\-duplicate tools and staged retrieval for multi\-capability workflows\.