Decoupling Search from Reasoning: A Vendor-Agnostic Grounding Architecture for LLM Agents

# Decoupling Search from Reasoning: A Vendor-Agnostic Grounding Architecture for LLM Agents
Source: [https://arxiv.org/html/2606.18947](https://arxiv.org/html/2606.18947)
Emmanuel Aboah Boateng, Kyle MacDonald, Amardeep Kumar, Siddharth Kodwani, Sudeep Das DoorDash, Inc\. \{e\.aboahboateng, kyle\.macdonald, amardeep\.kumar, siddharth\.kodwani, sudeep\.das2\}@doordash\.com

###### Abstract

Production LLM agents increasingly depend on real\-time search, yet native search grounding bundles retrieval policy, provider choice, evidence injection, cost, latency, and generation behavior behind a single model\-provider boundary\. This coupling makes grounding hard to inspect, tune, reuse, or port, and can trigger Search\-Induced Verbosity that breaks strict output contracts\. We present Decoupled Search Grounding \(DSG\), a vendor\-agnostic boundary that moves grounding outside the reasoning model through an MCP\-compatible gateway, exposing provider routing, source\-aware context rendering, configured fallback, retrieval\-depth control, and exact plus semantic caching as first\-class controls\. Across five frontier models on SimpleQA, FreshQA, and HotpotQA, native search leads on recency\-sensitive FreshQA, but DSG exposes a stronger frontier when control matters: on SimpleQA it nearly matches native accuracy \(86\.1% vs\. 87\.7%\) at 91% lower search cost, preserves concise answer contracts, and reaches a 99\.4% warm\-cache hit rate with 68% lower latency\. Deployed as a shared production grounding layer for large\-scale agentic workloads with interchangeable models, DSG matches or slightly exceeds native\-search accuracy on an e\-commerce query\-understanding \(QIU\) workload while cutting search cost by over 98%\. Real\-time grounding is best treated as an optimizable interface boundary, not a fixed model feature\.

Decoupling Search from Reasoning: A Vendor\-Agnostic Grounding Architecture for LLM Agents

Emmanuel Aboah Boateng, Kyle MacDonald, Amardeep Kumar,Siddharth Kodwani, Sudeep DasDoorDash, Inc\.\{e\.aboahboateng, kyle\.macdonald, amardeep\.kumar,siddharth\.kodwani, sudeep\.das2\}@doordash\.com

## 1Introduction

![Refer to caption](https://arxiv.org/html/2606.18947v1/figures/decoupled_search_comparison.png)Figure 1:Decoupled Search Grounding \(DSG\) separates retrieval from model\-native generation, making provider choice, caching, and output control explicit\.The deployment of Large Language Models \(LLMs\) in production increasingly centers on agentic tasks: multi\-step workflows in which a model reasons over intermediate state, decides when to call external tools, and often uses search to ground actions in fresh evidence\. These systems must satisfy strict cost, latency, reliability, and output\-contract constraints\(Schicket al\.,[2023](https://arxiv.org/html/2606.18947#bib.bib1); Mialonet al\.,[2023](https://arxiv.org/html/2606.18947#bib.bib4); Yaoet al\.,[2023](https://arxiv.org/html/2606.18947#bib.bib5); Liet al\.,[2023](https://arxiv.org/html/2606.18947#bib.bib6); Qinet al\.,[2024](https://arxiv.org/html/2606.18947#bib.bib7)\)\. As static Retrieval\-Augmented Generation \(RAG\) pipelines evolve into dynamic, tool\-using workflows\(Lewiset al\.,[2020](https://arxiv.org/html/2606.18947#bib.bib9); Karpukhinet al\.,[2020](https://arxiv.org/html/2606.18947#bib.bib10); Izacard and Grave,[2021](https://arxiv.org/html/2606.18947#bib.bib11); Jianget al\.,[2023](https://arxiv.org/html/2606.18947#bib.bib12); Asaiet al\.,[2024](https://arxiv.org/html/2606.18947#bib.bib14)\), the interface between the reasoning model and real\-time search becomes a first\-order systems decision\. Deployed NLP systems already show that retrieval design, evaluation, latency, and serving cost can dominate model choice in practice\(Murtazaet al\.,[2025](https://arxiv.org/html/2606.18947#bib.bib30); Chenet al\.,[2025](https://arxiv.org/html/2606.18947#bib.bib32); González Juclàet al\.,[2026](https://arxiv.org/html/2606.18947#bib.bib33); Liet al\.,[2025](https://arxiv.org/html/2606.18947#bib.bib31)\)\.

A common integration path isnative search grounding: a model\-provider feature in which the model API invokes a provider\-managed search stack and injects retrieved evidence into generation\. Native integrations are convenient and often strong, especially on recency\-sensitive questions\. Teams can instead integrate external providers directly, but their APIs, pricing, result formats, ranking, and failure surfaces differ substantially, which pushes engineers toward the default native path even when production systems need explicit control over provider choice and retrieval behavior\. This coupling hides decisions that production teams commonly need to control: which provider is queried, how much context is retrieved, how costs are bounded, whether repeated queries can be cached, and how tool outputs interact with downstream schema constraints\.

We study this interface boundary directly\. Prior work links verbose LLM responses to uncertainty and studies over\-searching in search\-augmented systems\(Zhanget al\.,[2025](https://arxiv.org/html/2606.18947#bib.bib25); Xieet al\.,[2026](https://arxiv.org/html/2606.18947#bib.bib26)\); here we isolate an interface\-specific failure mode,Search\-Induced Verbosity, in which native search changes response style despite strict output instructions\. Strict prompts such as “Provide only the final answer entity” can yield explanatory paragraphs, which matters for production systems where LLMs frequently act as intermediate nodes whose outputs are parsed, classified, cached, or routed by downstream services\.

We propose a vendor\-agnostic Decoupled Search Grounding \(DSG\) architecture implemented through an MCP\-compatible gateway\. Instead of treating search as an opaque model feature, DSG exposes search as a structured tool layer that can normalize provider outputs, route requests, cache repeated queries, and preserve the separation between retrieval and reasoning\. The key contribution is the controllable boundary itself: DSG externalizes provider policy, cache reuse, fallback, and source\-aware context rendering while keeping the reasoning model interchangeable\.

We evaluate DSG against native search across five frontier models \(GPT\-4o, GPT\-4o\-mini, Gemini 2\.5 Flash, Gemini 2\.5 Pro, and Claude Sonnet 4\), public QA benchmarks \(SimpleQA, FreshQA, HotpotQA\), and proprietary e\-commerce Query Intent Understanding \(QIU\) datasets\. Our contributions are:

1. 1\.A systems boundary for grounding:We formalize search grounding as a decoupled, provider\-agnostic layer that keeps the reasoning model interchangeable while exposing retrieval policy, caching, and provider choice as explicit system controls\.
2. 2\.A search\-intelligence and standardization framework:We implement DSG as an MCP\-compatible gateway that gives agents a stable search interface, normalizes heterogeneous provider outputs into source\-aware context, and supports cache\-gated retrieval with configured provider fallback across search backends\.
3. 3\.Evidence on behavior, quality, and operations:We identify Search\-Induced Verbosity as a prompt\-compliance risk in native search grounding and show how DSG’s structured tool boundary helps mitigate it, while quantifying accuracy, cost, latency, retrieval\-depth, and cache\-reuse trade\-offs across public QA benchmarks and a production workload\.

DSG makes grounding an interface decision: provider choice, cost policy, cache reuse, context rendering, fallback, and output\-contract behavior become first\-class controls while the reasoning model stays interchangeable\. Native and decoupled search expose different trade\-offs; the value of DSG is exposing that choice as an explicit, controllable decision\.

## 2Related Work

Toolformer\(Schicket al\.,[2023](https://arxiv.org/html/2606.18947#bib.bib1)\), ReAct\(Yaoet al\.,[2023](https://arxiv.org/html/2606.18947#bib.bib5)\), API\-Bank\(Liet al\.,[2023](https://arxiv.org/html/2606.18947#bib.bib6)\), ToolLLM\(Qinet al\.,[2024](https://arxiv.org/html/2606.18947#bib.bib7)\), and WebGPT\(Nakanoet al\.,[2021](https://arxiv.org/html/2606.18947#bib.bib8)\)establish that LLMs can use external tools and APIs\. These works primarily study agent capability and benchmark performance; we focus instead on the production consequences of where the search interface sits in the model stack\.

RAG has become a standard mechanism for improving factuality\(Lewiset al\.,[2020](https://arxiv.org/html/2606.18947#bib.bib9); Karpukhinet al\.,[2020](https://arxiv.org/html/2606.18947#bib.bib10); Izacard and Grave,[2021](https://arxiv.org/html/2606.18947#bib.bib11)\), with later work exploring dynamic retrieval and retrieval\-reasoning interaction, including FLARE\(Jianget al\.,[2023](https://arxiv.org/html/2606.18947#bib.bib12)\), RARR\(Gaoet al\.,[2023](https://arxiv.org/html/2606.18947#bib.bib13)\), Self\-RAG\(Asaiet al\.,[2024](https://arxiv.org/html/2606.18947#bib.bib14)\), Adaptive\-RAG\(Jeonget al\.,[2024](https://arxiv.org/html/2606.18947#bib.bib15)\), and IRCoT\(Trivediet al\.,[2023](https://arxiv.org/html/2606.18947#bib.bib16)\)\. In deployed workflows, these retrieval choices interact with context construction, prompt optimization\(Zhouet al\.,[2023](https://arxiv.org/html/2606.18947#bib.bib38); Boatenget al\.,[2026a](https://arxiv.org/html/2606.18947#bib.bib35)\), and strong\-to\-weak adaptation of smaller reasoning models\(Hsiehet al\.,[2023](https://arxiv.org/html/2606.18947#bib.bib39); Aboah Boatenget al\.,[2025](https://arxiv.org/html/2606.18947#bib.bib36); Boatenget al\.,[2025](https://arxiv.org/html/2606.18947#bib.bib37)\); DSG is complementary because it fixes the grounding interface while keeping the reasoning model interchangeable\. Industry applications such as multi\-category e\-commerce intent understanding further motivate multi\-source grounding\(Boatenget al\.,[2026b](https://arxiv.org/html/2606.18947#bib.bib27)\)\. Our contribution is orthogonal: we study whether search is embedded as opaque provider behavior or exposed as an explicit, controllable subsystem\.

Our evaluation combines short\-form factuality \(SimpleQA;Weiet al\.,[2024](https://arxiv.org/html/2606.18947#bib.bib17)\), multi\-hop stress testing \(HotpotQA;Yanget al\.,[2018](https://arxiv.org/html/2606.18947#bib.bib19)\), and recency\-sensitive grounding \(FreshQA/FreshLLMs;Vuet al\.,[2024](https://arxiv.org/html/2606.18947#bib.bib18)\), aligning with calls for dynamic evaluation such as Dynabench\(Kielaet al\.,[2021](https://arxiv.org/html/2606.18947#bib.bib24)\)\. For scalable assessment, we draw on LLM\-as\-judge and RAG evaluation work including G\-Eval\(Liuet al\.,[2023](https://arxiv.org/html/2606.18947#bib.bib22)\), MT\-Bench / Chatbot Arena analysis\(Zhenget al\.,[2023](https://arxiv.org/html/2606.18947#bib.bib23)\), ARES\(Saad\-Falconet al\.,[2024](https://arxiv.org/html/2606.18947#bib.bib20)\), RAGAs\(Eset al\.,[2024](https://arxiv.org/html/2606.18947#bib.bib21)\), and agentic task\-completion evaluation\(Bhonsleet al\.,[2025](https://arxiv.org/html/2606.18947#bib.bib34)\)\.

Industry\-track work increasingly documents deployed retrieval trade\-offs in enterprise support, dynamic vector stores, and latency\-constrained search\(Murtazaet al\.,[2025](https://arxiv.org/html/2606.18947#bib.bib30); González Juclàet al\.,[2026](https://arxiv.org/html/2606.18947#bib.bib33); Chenet al\.,[2025](https://arxiv.org/html/2606.18947#bib.bib32); Liet al\.,[2025](https://arxiv.org/html/2606.18947#bib.bib31)\)\. MCP has emerged as a practical standard for connecting models to tools and data; early work studies tool\-description quality and production design patterns\(Hasanet al\.,[2026](https://arxiv.org/html/2606.18947#bib.bib28); Srinivasan,[2026](https://arxiv.org/html/2606.18947#bib.bib29)\)\. We use MCP as the connection substrate for DSG, and evaluate decoupled search grounding against proprietary native search\. Distinct from these lines of work, we treat real\-time search grounding itself as a controllable, vendor\-agnostic interface and measure the cost, latency, caching, and output\-contract trade\-offs it exposes against proprietary native search across multiple frontier models and a production workload\.

## 3The Challenge of Native Search

Native search is provider\-managed grounding: the model API invokes search and injects evidence through provider\-specific context handling\. This one\-call abstraction is attractive for agents, but it moves retrieval policy inside the provider boundary\. Developers lose control over provider selection, result normalization, caching, tool\-output formatting, and migration paths when a better reasoning model or search provider becomes available\.

When the model call is an intermediate step in an automated pipeline, this coupling creates four deployment challenges: model\-provider lock\-in, opaque latency, fixed cost structures, and instruction\-following degradation that we termSearch\-Induced Verbosity\.

![Refer to caption](https://arxiv.org/html/2606.18947v1/x1.png)Figure 2:Prompt\-compliance diagnostic on HotpotQA\. Native search can shift concise answer extraction into explanatory output, while DSG preserves the answer\-only interface required by downstream parsers\.### 3\.1Search\-Induced Verbosity

Production pipelines often use LLMs as intermediate reasoning nodes whose outputs must parse as JSON, boolean flags, or short entities\. Prompts therefore specify strict output contracts, such as “Provide ONLY the final answer entity\.”

We observe that native search can shift the model from concise extraction to explanatory summarization, even when the final\-answer instruction is explicit\. The answer may be present, but extra prose can break exact\-match evaluation or downstream parsers\.

Example: Prompt\-Compliance FailureInstruction:“Provide ONLY the final answer entity\.” Question:“Is the building at 200 West Street taller than 888 7th Avenue?”Native search output:“Based on the search results, I can compare the heights of both buildings: 200 West Street is 749 feet tall… while 888 7th Avenue is 628 feet tall…”245 charsDSG output:“Yes”3 chars

We treat this as a deployment\-risk diagnostic, not as a claim that native search always changes output style\. In the evaluated cases, native search sometimes shifted models from concise extraction to explanatory summarization despite strict final\-answer instructions\. In our HotpotQA diagnostics, an affected Claude Sonnet 4 native\-search run produced answers beginning with “Based on the search results…” in 78\.1% of predictions; among exact\-match failures, 62\.3% still contained the correct answer as a substring\. Table[8](https://arxiv.org/html/2606.18947#A7.T8)reports the diagnostic summary\.

This matters when a semantically correct answer is unusable because it violates an exact\-entity, JSON, boolean, or categorical output contract\. Headline accuracy uses task\-level judge/classifier scoring, while format compliance is reported separately as a diagnostic\. DSG mitigates this by keeping retrieval as a structured tool response with a stable output boundary; future work with provider\-side telemetry could further isolate the internal triggers\.

![Refer to caption](https://arxiv.org/html/2606.18947v1/figures/search_grounding_architecture.png)Figure 3:Decoupled Search Grounding architecture\. DSG separates agentic applications and interchangeable reasoning models from external search providers through a stable tool interface, cache\-gated search intelligence, provider abstraction, operational controls, and normalized model\-facing grounding context\.

## 4Decoupled Search Grounding for LLM Agents

We implement Decoupled Search Grounding \(DSG\) as an MCP\-compatible gateway between agentic applications, interchangeable reasoning models, and provider\-agnostic search backends \(Figure[3](https://arxiv.org/html/2606.18947#S3.F3)\)\.

### 4\.1Source\-Aware Tool Formatting

Unlike native integrations that hide retrieval inside a provider\-specific API, our architecture exposes search through explicit, configurable tool schemas\. Engineers specify the query interface and provider options once at the gateway layer, so each agent inherits them without hard\-coding\. When the LLM emits a search tool call, DSG executes the query through its MCP\-compatible interface and normalizes provider responses into a structured internal result object\. The rendered tool context is consistent across providers and pairs snippets with source URLs, giving the model explicit provenance cues for which evidence to use\.

These URLs also give engineers an inspectable artifact for logging, debugging, source filtering, and fallback\. Returning results as a discrete tool response preserves the retrieval\-generation boundary, which a provider\-managed context stream collapses\.

### 4\.2Provider Abstraction and Fallback

The gateway acts as a grounding control plane where provider choice, fallback order, and cost metadata are configurable policy applied across agents\. TheProvider Registrynormalizes agnostic search providers \(e\.g\., Serper, BrightData, Firecrawl, Exa\)\. Standard HTTP providers can be onboarded through a YAML adapter that specifies endpoint, method, headers, request templates, response\-field mappings, capabilities, and cost metadata; richer providers use dedicated adapters behind the same interface\. This lets researchers and engineers tune grounding for different applications, including freshness, cost, latency, or source coverage, without changing the reasoning model or application prompt\. A provider timeout or error response advances the request along the configured fallback chain, so transient provider failures do not interrupt the grounding context the model receives\.

### 4\.3Search Intelligence Layer

A central advantage of decoupling is that search policy can be optimized across requests, beyond the scope of a single model call\. DSG introduces a search intelligence layer that applies a tiered decision policy: exact cache lookup, semantic cache reuse, and configured provider fallback for novel queries\. Letqqdenote a query,q¯\\bar\{q\}its normalized form,𝒞=\{\(qi,Ri\)\}\\mathcal\{C\}=\\\{\(q\_\{i\},R\_\{i\}\)\\\}a provider\-isolated semantic cache,e​\(⋅\)e\(\\cdot\)an embedding function, andp1,…,pkp\_\{1\},\\ldots,p\_\{k\}a configured provider fallback chain\. The gateway decision is:

j∗\\displaystyle j^\{\*\}=arg⁡maxj⁡cos⁡\(e​\(q\),e​\(qj\)\),\\displaystyle=\\arg\\max\_\{j\}\\cos\(e\(q\),e\(q\_\{j\}\)\),\(1\)D​\(q\)\\displaystyle D\(q\)=\{Re,E​\(q¯\)Rj∗,sj∗≥τS​\(𝐩,q\),else\\displaystyle=
wheresj∗=cos⁡\(e​\(q\),e​\(qj∗\)\)s\_\{j^\{\*\}\}=\\cos\(e\(q\),e\(q\_\{j^\{\*\}\}\)\),EEdenotes an exact\-cache hit,SSexecutes the configured fallback chain𝐩=\(p1,…,pk\)\\mathbf\{p\}=\(p\_\{1\},\\ldots,p\_\{k\}\), andτ\\tauis a configurable semantic\-match threshold\. The DSG cache architecture stores exact and semantic entries with provider\-scoped keys, so cached results from one provider do not silently replace another provider’s output\. Cache entries also carry provider and domain specific time\-to\-live bounds, so recency\-sensitive queries expire quickly while static factoid results persist, keeping high cache\-hit rates from compromising grounding freshness\. This keeps provider policy explicit: repeated queries can bypass external APIs, semantically similar queries can reuse compatible evidence, and novel queries fall through to the configured provider chain\. Algorithm[1](https://arxiv.org/html/2606.18947#algorithm1)summarizes the gateway execution path\.

Input:Query

qq, threshold

τ\\tau, fallback chain

p1,…,pkp\_\{1\},\\ldots,p\_\{k\}
Output:Rendered tool context

TT
q¯←Normalize​\(q\)\\bar\{q\}\\leftarrow\\text\{Normalize\}\(q\);

R←ExactCache\.Get​\(q¯\)R\\leftarrow\\text\{ExactCache\.Get\}\(\\bar\{q\}\);

if*R≠∅R\\neq\\emptyset*then

return

RenderContext​\(R\)\\text\{RenderContext\}\(R\);

end if

z←Embed​\(q\)z\\leftarrow\\text\{Embed\}\(q\);

\(qj,Rj,sj\)←NearestSemanticCache​\(z\)\(q\_\{j\},R\_\{j\},s\_\{j\}\)\\leftarrow\\text\{NearestSemanticCache\}\(z\);

if*sj≥τs\_\{j\}\\geq\\tau*then

return

RenderContext​\(Rj\)\\text\{RenderContext\}\(R\_\{j\}\);

end if

for*p∈\(p1,…,pk\)p\\in\(p\_\{1\},\\ldots,p\_\{k\}\)*do

if*ppis unavailable*then

continue;

end if

Rraw←ExecuteSearch​\(p,q\)R\_\{\\mathrm\{raw\}\}\\leftarrow\\text\{ExecuteSearch\}\(p,q\);

if*RrawR\_\{\\mathrm\{raw\}\}has results*then

R←NormalizeProviderResult​\(Rraw\)R\\leftarrow\\text\{NormalizeProviderResult\}\(R\_\{\\mathrm\{raw\}\}\);

ExactCache\.Set​\(q¯,R\)\\text\{ExactCache\.Set\}\(\\bar\{q\},R\);

SemanticCache\.Set​\(q,z,R\)\\text\{SemanticCache\.Set\}\(q,z,R\);

return

RenderContext​\(R\)\\text\{RenderContext\}\(R\);

end if

end for

return

RenderContext​\(∅\)\\text\{RenderContext\}\(\\emptyset\);

Algorithm 1Cache\-Gated Provider ExecutionThe result is reusable search intelligence: cache policy, retrieval depth, provider selection, and fallback become measurable controls the system can observe and tune\. Because grounding is intercepted at a single boundary, each request emits structured telemetry, including the selected provider, retrieval depth, cache outcome, end\-to\-end latency, and provider\-side cost, so researchers and engineers can attribute grounding cost and latency per query and per application and detect regressions when a provider degrades\. Expressing these controls as configuration rather than code lets a single grounding layer be governed consistently across many agents and reasoning models without retraining or reprompting\. Because the cache sits behind a shared gateway, evidence retrieved for one application can be reused by another, so warm\-cache hit rates compound as more workloads route through DSG\. In Section[5](https://arxiv.org/html/2606.18947#S5), a repeated\-query replay reaches a 99\.4% cache hit rate, reducing benchmark latency by 68% and driving marginal search cost near zero\. Appendix[I](https://arxiv.org/html/2606.18947#A9)summarizes the resulting deployment control surfaces, including provider policy, repeated\-query reuse, source inspectability, and output\-contract reliability\.

## 5Empirical Evaluation

We evaluate DSG as a systems choice: accuracy retained, cost paid, and controls exposed when search moves outside the model\-provider boundary\. Benchmarks span static factuality \(SimpleQA;Weiet al\.,[2024](https://arxiv.org/html/2606.18947#bib.bib17)\), recency\-sensitive knowledge \(FreshQA;Vuet al\.,[2024](https://arxiv.org/html/2606.18947#bib.bib18)\), multi\-hop QA \(HotpotQA;Yanget al\.,[2018](https://arxiv.org/html/2606.18947#bib.bib19)\), and production\-style e\-commerce grounding\. Public QA uses fixed samples, fixed prompts, and task\-level scoring\. SimpleQA and FreshQA use GPT\-4\.1 judging, with a 3\-judge SimpleQA validation ablation showing 97\.78% mean unanimous agreement \(Appendix[G\.3](https://arxiv.org/html/2606.18947#A7.SS3)\); HotpotQA uses deterministic EM/F1\. Prompt\-compliance diagnostics are reported separately from headline accuracy\. For QIU, we follow the multi\-category marketplace intent\-routing protocol ofBoatenget al\.\([2026b](https://arxiv.org/html/2606.18947#bib.bib27)\):Retail\(N=7,988N=7\{,\}988\) covers general non\-food retail queries, andTail \(Synthetic\)\(N=2,335N=2\{,\}335\) covers rare long\-tail queries\. Costs are provider\-side search costs per 1K queries, excluding model inference cost; latencies are benchmark latencies from evaluation outputs\.

![Refer to caption](https://arxiv.org/html/2606.18947v1/x2.png)Figure 4:Operational controls enabled by DSG: retrieval\-depth tuning and semantic caching reduce warm\-cache latency from 4,570ms to 1,465ms and nearly eliminate marginal search cost\.Table 1:Main results: accuracy / search cost per 1K queries\. Academic rows average five models; QIU uses Gemini Flash\.### 5\.1Cost\-Accuracy Pareto Optimality

Native integrations package retrieval quality with fixed provider pricing\. Table[1](https://arxiv.org/html/2606.18947#S5.T1)and Figure[5](https://arxiv.org/html/2606.18947#S5.F5)show that DSG routing creates a broader operating frontier\. On SimpleQA, DSG\+BrightData reaches 86\.1% mean accuracy, close to native search at 87\.7%, while reducing average search\-provider cost from $20\.00 to $1\.80 per 1K queries\. DSG\+Serper provides a lower\-cost operating point at 83\.3% accuracy and $0\.67 per 1K queries\. Bootstrap confidence intervals are reported in Appendix[G\.1](https://arxiv.org/html/2606.18947#A7.SS1)\.

![Refer to caption](https://arxiv.org/html/2606.18947v1/x3.png)Figure 5:SimpleQA cost\-accuracy frontier, averaged across five models\. DSG\+BrightData nearly matches native accuracy at roughly 11x lower search cost, while DSG\+Serper provides the lowest\-cost high\-accuracy operating point\.The trade\-off is task\-dependent\. On FreshQA, native search leads on recency\-sensitive questions, consistent with tightly integrated proprietary freshness or indexing advantages\. This reinforces the case for decoupling: choosing the grounding backend becomes a routing decision the team controls, with native search available as one provider within the same interface when freshness dominates and general\-purpose providers when cost, inspection, portability, or caching matter more\.

### 5\.2Real\-World Impact: E\-commerce QIU

QIU classifies ambiguous marketplace queries using catalog and web evidence, with accuracy measured on the final resolved intent\(Boatenget al\.,[2026b](https://arxiv.org/html/2606.18947#bib.bib27)\); DSG serves as the shared production grounding layer for multiple applications, including the QIU model evaluated here\. The QIU rows in Table[1](https://arxiv.org/html/2606.18947#S5.T1)show the production\-relevant cost\-quality story: on Retail, DSG\+Serper reaches 93\.90% versus 93\.40% for native search while reducing search cost from $7\.90 to $0\.110 per 1K queries; on Tail \(Synthetic\), it reaches 87\.79% versus 87\.62% for native search at $0\.146 per 1K queries, over 98% below native search\. This is striking because our native baseline, Gemini Flash, natively supports Google Search grounding\(Gemini Team,[2025](https://arxiv.org/html/2606.18947#bib.bib2); Google AI for Developers,[2026](https://arxiv.org/html/2606.18947#bib.bib3)\); matching or exceeding it with general\-purpose providers suggests externalized control can offset tightly coupled native search\. Full provider tables are in Appendix[F](https://arxiv.org/html/2606.18947#A6)\.

### 5\.3Operational Controls: Caching and Tuning

Because DSG intercepts structured tool calls, it exposes controls unavailable in black\-box search\. A SimpleQAmax\_resultssweep shows quality rising sharply from 2 to 4–6 results before saturating around 6–10 \(Figure[4](https://arxiv.org/html/2606.18947#S5.F4)A\), turning grounding into a tunable budget knob\. In a repeated\-query cache experiment with GPT\-4o and DSG\+BrightData, the warm pass reaches a 99\.4% hit rate, reduces average latency from 4,570ms to 1,465ms, and nearly eliminates marginal search\-provider cost \(Figure[4](https://arxiv.org/html/2606.18947#S5.F4)B–C\)\.

## 6Conclusion

We argue that real\-time grounding for LLM agents should be an optimizable systems boundary that teams control directly\. By separating retrieval from reasoning, DSG keeps the reasoning model interchangeable while exposing provider policy, source\-aware rendering, retrieval\-depth tuning, fallback, and warm\-cache reuse as explicit controls\. Across five frontier models and a production workload, it approaches native\-search accuracy on public QA, matches or slightly exceeds it in deployment at far lower cost, and holds strict output contracts where native search drifts into prose\. The most consequential open problem we surface is Search\-Induced Verbosity, where a correct answer wrapped in prose still breaks downstream parsers; future work should measure it across models with provider\-side telemetry, add contract\-aware validators at the boundary, and learn adaptive routing, retrieval depth, and multi\-hop strategies over the same interface\.

## Limitations

While our DSG architecture demonstrates significant advantages in cost, latency, and prompt compliance, we acknowledge several limitations in our study and the proposed system\.

#### Reliance on Third\-Party APIs

The decoupled architecture fundamentally relies on the stability, latency, and data quality of third\-party search vendors \(e\.g\., Serper, BrightData, Firecrawl\)\. Changes to these external APIs, including pricing adjustments, rate limits, or indexing behavior, could impact the cost\-accuracy Pareto frontier presented in this work\. Reported provider costs reflect pricing observed during our evaluation period and are expected to change over time; our contribution is the controllable grounding interface and the operating frontiers it exposes, not any specific price point\.

#### Multi\-Hop Reasoning Constraints

Our evaluation on the HotpotQA benchmark \(detailed in Appendix[E](https://arxiv.org/html/2606.18947#A5)\) revealed that search augmentation, whether native or DSG\-based, provides only modest gains for complex, multi\-hop reasoning tasks compared to single\-hop factoid retrieval \(SimpleQA\)\. This suggests that while our architecture effectively grounds models for direct queries, solving deep, multi\-step reasoning problems likely requires more advanced retrieval\-reasoning coordination, such as iterative retrieval, planning, or self\-reflection\(Trivediet al\.,[2023](https://arxiv.org/html/2606.18947#bib.bib16); Asaiet al\.,[2024](https://arxiv.org/html/2606.18947#bib.bib14); Jeonget al\.,[2024](https://arxiv.org/html/2606.18947#bib.bib15)\), beyond simple search decoupling\. We hypothesize that DSG’s tool boundary is most effective for controllable retrieval, while complex multi\-hop tasks may require tighter iterative loops with planning, reflection, and repeated evidence acquisition\.

#### Evaluation Methodology

For academic benchmarks \(SimpleQA, FreshQA\), we utilized an LLM\-as\-a\-judge framework \(GPT\-4\.1\)\. While this methodology is highly scalable and increasingly well\-supported in the literature\(Liuet al\.,[2023](https://arxiv.org/html/2606.18947#bib.bib22); Zhenget al\.,[2023](https://arxiv.org/html/2606.18947#bib.bib23); Saad\-Falconet al\.,[2024](https://arxiv.org/html/2606.18947#bib.bib20); Eset al\.,[2024](https://arxiv.org/html/2606.18947#bib.bib21)\), automated evaluators can exhibit biases, particularly regarding formatting sensitivities\. We mitigated this by utilizing a multi\-judge majority voting ablation \(Appendix[G\.3](https://arxiv.org/html/2606.18947#A7.SS3)\), but human evaluation remains the gold standard for nuanced qualitative assessment\.

## Ethical Considerations

For semantic caching, the gateway is designed to cache only provider\-permitted and ethically allowable context under applicable provider terms and data\-use agreements, rather than indiscriminately storing retrieved content\. The planned public artifact excludes proprietary QIU data, provider\-restricted retrieved snippets, and internal deployment details\. Upon acceptance, we intend to release non\-proprietary prompts, plotting scripts, aggregate public\-benchmark outputs, and DSG configuration templates\.

## Acknowledgments

We thank Elyse Winer, Drishya Giri, Kaiwen Bian, Jamie Gu, Prabhjot Saini, Stephanie Poon, Sebastian Connelly, Tanvi Priya, Johny Rufus, and Tom Tang for their contributions, feedback, and support throughout this work\. An early version of this work received the Engineering Excellence award at the 2026 DoorDash AI Hackathon\.

## References

- Using large generative models to improve the performance of weak language models in performing complex tasks\.Note:U\.S\. Patent Application US\-20250348745\-A1Cited by:[§2](https://arxiv.org/html/2606.18947#S2.p2.1)\.
- A\. Asai, Z\. Wu, Y\. Wang, A\. Sil, and H\. Hajishirzi \(2024\)Self\-RAG: learning to retrieve, generate, and critique through self\-reflection\.InThe Twelfth International Conference on Learning Representations,Note:OralExternal Links:[Link](https://openreview.net/forum?id=hSyW5go0v8)Cited by:[§1](https://arxiv.org/html/2606.18947#S1.p1.1),[§2](https://arxiv.org/html/2606.18947#S2.p2.1),[Multi\-Hop Reasoning Constraints](https://arxiv.org/html/2606.18947#Sx1.SS0.SSS0.Px2.p1.1)\.
- R\. Bhonsle, R\. Dutta, S\. Vavilapalli, H\. Seth, A\. Jaye, Y\. Chang, M\. Rungta,et al\.\(2025\)Auto\-eval judge: towards a general agentic framework for task completion evaluation\.arXiv preprint arXiv:2508\.05508\.External Links:[Link](https://arxiv.org/abs/2508.05508)Cited by:[§2](https://arxiv.org/html/2606.18947#S2.p3.1)\.
- E\. A\. Boateng, C\. O\. Becker, N\. Asghar, K\. Walia, A\. Srinivasan, E\. Nosakhare,et al\.\(2025\)Concept distillation from strong to weak models via hypotheses\-to\-theories prompting\.InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies,Cited by:[§2](https://arxiv.org/html/2606.18947#S2.p2.1)\.
- E\. A\. Boateng, Z\. Johnson, T\. Xia, S\. Zhang, A\. Jay, J\. Feng, A\. Mate,et al\.\(2026a\)SAPO: secure automated prompt optimization via multi\-agent collaboration\.InProceedings of the ACM Conference on AI and Agentic Systems,pp\. 773–784\.Cited by:[§2](https://arxiv.org/html/2606.18947#S2.p2.1)\.
- E\. A\. Boateng, K\. MacDonald, A\. Viswanathan, and S\. Das \(2026b\)Agentic multi\-source grounding for enhanced query intent understanding: a DoorDash case study\.arXiv preprint arXiv:2603\.01486\.External Links:[Link](https://arxiv.org/abs/2603.01486)Cited by:[§2](https://arxiv.org/html/2606.18947#S2.p2.1),[§5\.2](https://arxiv.org/html/2606.18947#S5.SS2.p1.1),[§5](https://arxiv.org/html/2606.18947#S5.p1.2)\.
- Q\. Chen, W\. Tao, Z\. Zhu, M\. Xi, L\. Guo, Y\. Wang, W\. Wang, and Y\. Lan \(2025\)ComRAG: retrieval\-augmented generation with dynamic vector stores for real\-time community question answering in industry\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 6: Industry Track\),pp\. 749–763\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.acl-industry.53),[Link](https://aclanthology.org/2025.acl-industry.53/)Cited by:[§1](https://arxiv.org/html/2606.18947#S1.p1.1),[§2](https://arxiv.org/html/2606.18947#S2.p4.1)\.
- S\. Es, J\. James, L\. Espinosa\-Anke, and S\. Schockaert \(2024\)RAGAs: automated evaluation of retrieval augmented generation\.InProceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations,pp\. 150–158\.External Links:[Link](https://aclanthology.org/2024.eacl-demo.16/)Cited by:[§2](https://arxiv.org/html/2606.18947#S2.p3.1),[Evaluation Methodology](https://arxiv.org/html/2606.18947#Sx1.SS0.SSS0.Px3.p1.1)\.
- L\. Gao, Z\. Dai, P\. Pasupat, A\. Chen, A\. T\. Chaganty, Y\. Fan, V\. Zhao, N\. Lao, H\. Lee, D\. Juan, and K\. Guu \(2023\)RARR: researching and revising what language models say, using language models\.InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 16477–16508\.External Links:[Document](https://dx.doi.org/10.18653/v1/2023.acl-long.910),[Link](https://aclanthology.org/2023.acl-long.910/)Cited by:[§2](https://arxiv.org/html/2606.18947#S2.p2.1)\.
- Gemini Team \(2025\)Gemini 2\.5: pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities\.arXiv preprint arXiv:2507\.06261\.External Links:[Link](https://arxiv.org/abs/2507.06261)Cited by:[§5\.2](https://arxiv.org/html/2606.18947#S5.SS2.p1.1)\.
- D\. González Juclà, M\. Tuteja, M\. Esteve Casademunt, K\. Unnikrishnan, Y\. Usmani, and A\. Roshaan \(2026\)Retrieval enhancements for RAG: insights from a deployed customer support chatbot\.InProceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics \(Volume 5: Industry Track\),pp\. 169–180\.External Links:[Document](https://dx.doi.org/10.18653/v1/2026.eacl-industry.13),[Link](https://aclanthology.org/2026.eacl-industry.13/)Cited by:[§1](https://arxiv.org/html/2606.18947#S1.p1.1),[§2](https://arxiv.org/html/2606.18947#S2.p4.1)\.
- Google AI for Developers \(2026\)Grounding with google search\.Note:[https://ai\.google\.dev/gemini\-api/docs/google\-search](https://ai.google.dev/gemini-api/docs/google-search)Accessed 2026\-06\-15Cited by:[§5\.2](https://arxiv.org/html/2606.18947#S5.SS2.p1.1)\.
- M\. M\. Hasan, H\. Li, G\. K\. Rajbahadur, B\. Adams, and A\. E\. Hassan \(2026\)Model context protocol \(MCP\) tool descriptions are smelly\! towards improving AI agent efficiency with augmented MCP tool descriptions\.arXiv preprint arXiv:2602\.14878\.External Links:[Link](https://arxiv.org/abs/2602.14878)Cited by:[§2](https://arxiv.org/html/2606.18947#S2.p4.1)\.
- C\. Hsieh, C\. Li, C\. Yeh, H\. Nakhost, Y\. Fujii, A\. Ratner, R\. Krishna, C\. Lee, and T\. Pfister \(2023\)Distilling step\-by\-step\! outperforming larger language models with less training data and smaller model sizes\.InFindings of the Association for Computational Linguistics: ACL 2023,pp\. 8003–8017\.External Links:[Link](https://arxiv.org/abs/2305.02301)Cited by:[§2](https://arxiv.org/html/2606.18947#S2.p2.1)\.
- G\. Izacard and E\. Grave \(2021\)Leveraging passage retrieval with generative models for open domain question answering\.InProceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume,pp\. 874–880\.External Links:[Document](https://dx.doi.org/10.18653/v1/2021.eacl-main.74),[Link](https://aclanthology.org/2021.eacl-main.74/)Cited by:[§1](https://arxiv.org/html/2606.18947#S1.p1.1),[§2](https://arxiv.org/html/2606.18947#S2.p2.1)\.
- S\. Jeong, J\. Baek, S\. Cho, S\. J\. Hwang, and J\. C\. Park \(2024\)Adaptive\-RAG: learning to adapt retrieval\-augmented large language models through question complexity\.InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),pp\. 7036–7050\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.naacl-long.389),[Link](https://aclanthology.org/2024.naacl-long.389/)Cited by:[§2](https://arxiv.org/html/2606.18947#S2.p2.1),[Multi\-Hop Reasoning Constraints](https://arxiv.org/html/2606.18947#Sx1.SS0.SSS0.Px2.p1.1)\.
- Z\. Jiang, F\. F\. Xu, L\. Gao, Z\. Sun, Q\. Liu, J\. Dwivedi\-Yu, Y\. Yang, J\. Callan, and G\. Neubig \(2023\)Active retrieval augmented generation\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,pp\. 7969–7992\.External Links:[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.495),[Link](https://aclanthology.org/2023.emnlp-main.495/)Cited by:[§1](https://arxiv.org/html/2606.18947#S1.p1.1),[§2](https://arxiv.org/html/2606.18947#S2.p2.1)\.
- V\. Karpukhin, B\. Oguz, S\. Min, P\. Lewis, L\. Wu, S\. Edunov, D\. Chen, and W\. Yih \(2020\)Dense passage retrieval for open\-domain question answering\.InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),pp\. 6769–6781\.External Links:[Document](https://dx.doi.org/10.18653/v1/2020.emnlp-main.550),[Link](https://aclanthology.org/2020.emnlp-main.550/)Cited by:[§1](https://arxiv.org/html/2606.18947#S1.p1.1),[§2](https://arxiv.org/html/2606.18947#S2.p2.1)\.
- D\. Kiela, M\. Bartolo, Y\. Nie, D\. Kaushik, A\. Geiger, Z\. Wu, B\. Vidgen, G\. Prasad, A\. Singh, P\. Ringshia, Z\. Ma, T\. Thrush, S\. Riedel, Z\. Waseem, P\. Stenetorp, R\. Jia, M\. Bansal, C\. Potts, and A\. Williams \(2021\)Dynabench: rethinking benchmarking in NLP\.InProceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,pp\. 4110–4124\.External Links:[Document](https://dx.doi.org/10.18653/v1/2021.naacl-main.324),[Link](https://aclanthology.org/2021.naacl-main.324/)Cited by:[§2](https://arxiv.org/html/2606.18947#S2.p3.1)\.
- P\. Lewis, E\. Perez, A\. Piktus, F\. Petroni, V\. Karpukhin, N\. Goyal, H\. Küttler, M\. Lewis, W\. Yih, T\. Rocktäschel,et al\.\(2020\)Retrieval\-augmented generation for knowledge\-intensive NLP tasks\.Advances in Neural Information Processing Systems33,pp\. 9459–9474\.Cited by:[§1](https://arxiv.org/html/2606.18947#S1.p1.1),[§2](https://arxiv.org/html/2606.18947#S2.p2.1)\.
- M\. Li, Y\. Zhao, B\. Yu, F\. Song, H\. Li, H\. Yu, Z\. Li, F\. Huang, and Y\. Li \(2023\)API\-Bank: a comprehensive benchmark for tool\-augmented LLMs\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,pp\. 3102–3116\.External Links:[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.187),[Link](https://aclanthology.org/2023.emnlp-main.187/)Cited by:[§1](https://arxiv.org/html/2606.18947#S1.p1.1),[§2](https://arxiv.org/html/2606.18947#S2.p1.1)\.
- X\. Li, X\. Li, L\. Gao, Y\. Liu, X\. Wang, S\. Wang, J\. Wang, and D\. Yin \(2025\)Proactive guidance of multi\-turn conversation in industrial search\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 6: Industry Track\),pp\. 706–717\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.acl-industry.50),[Link](https://aclanthology.org/2025.acl-industry.50/)Cited by:[§1](https://arxiv.org/html/2606.18947#S1.p1.1),[§2](https://arxiv.org/html/2606.18947#S2.p4.1)\.
- Y\. Liu, D\. Iter, Y\. Xu, S\. Wang, R\. Xu, and C\. Zhu \(2023\)G\-Eval: NLG evaluation using GPT\-4 with better human alignment\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,pp\. 2511–2522\.External Links:[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.153),[Link](https://aclanthology.org/2023.emnlp-main.153/)Cited by:[§2](https://arxiv.org/html/2606.18947#S2.p3.1),[Evaluation Methodology](https://arxiv.org/html/2606.18947#Sx1.SS0.SSS0.Px3.p1.1)\.
- G\. Mialon, R\. Dessì, M\. Lomeli, C\. Nalmpantis, R\. Pasunuru, R\. Raileanu, B\. Rozière, T\. Schick, J\. Dwivedi\-Yu, A\. Celikyilmaz,et al\.\(2023\)Augmented language models: a survey\.arXiv preprint arXiv:2302\.07842\.External Links:[Link](https://arxiv.org/abs/2302.07842)Cited by:[§1](https://arxiv.org/html/2606.18947#S1.p1.1)\.
- S\. S\. Murtaza, Y\. Nie, E\. Avan, U\. Soni, W\. Liao, A\. Carnegie, C\. J\. Mathias, J\. Jiang, and E\. Wen \(2025\)Implementing retrieval augmented generation technique on unstructured and structured data sources in a call center of a large financial institution\.InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 3: Industry Track\),pp\. 598–606\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.naacl-industry.48),[Link](https://aclanthology.org/2025.naacl-industry.48/)Cited by:[§1](https://arxiv.org/html/2606.18947#S1.p1.1),[§2](https://arxiv.org/html/2606.18947#S2.p4.1)\.
- R\. Nakano, J\. Hilton, S\. Balaji, J\. Wu, L\. Ouyang, C\. Kim, C\. Hesse, S\. Jain, V\. Kosaraju, W\. Saunders, X\. Jiang, K\. Cobbe, T\. Eloundou, G\. Krueger, K\. Button, M\. Knight, B\. Chess, and J\. Schulman \(2021\)WebGPT: browser\-assisted question\-answering with human feedback\.arXiv preprint arXiv:2112\.09332\.External Links:[Link](https://arxiv.org/abs/2112.09332)Cited by:[§2](https://arxiv.org/html/2606.18947#S2.p1.1)\.
- Y\. Qin, S\. Liang, Y\. Ye, K\. Zhu, L\. Yan, Y\. Lu, Y\. Lin, X\. Cong, X\. Tang, B\. Qian, S\. Zhao, L\. Hong, R\. Tian, R\. Xie, J\. Zhou, M\. Gerstein, D\. Li, Z\. Liu, and M\. Sun \(2024\)ToolLLM: facilitating large language models to master 16000\+ real\-world APIs\.InThe Twelfth International Conference on Learning Representations,Note:SpotlightExternal Links:[Link](https://openreview.net/forum?id=dHng2O0Jjr)Cited by:[§1](https://arxiv.org/html/2606.18947#S1.p1.1),[§2](https://arxiv.org/html/2606.18947#S2.p1.1)\.
- J\. Saad\-Falcon, O\. Khattab, C\. Potts, and M\. Zaharia \(2024\)ARES: an automated evaluation framework for retrieval\-augmented generation systems\.InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),pp\. 338–354\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.naacl-long.20),[Link](https://aclanthology.org/2024.naacl-long.20/)Cited by:[§2](https://arxiv.org/html/2606.18947#S2.p3.1),[Evaluation Methodology](https://arxiv.org/html/2606.18947#Sx1.SS0.SSS0.Px3.p1.1)\.
- T\. Schick, J\. Dwivedi\-Yu, R\. Dessì, R\. Raileanu, M\. Lomeli, L\. Zettlemoyer, N\. Cancedda, and T\. Scialom \(2023\)Toolformer: language models can teach themselves to use tools\.InAdvances in Neural Information Processing Systems,Vol\.36\.Cited by:[§1](https://arxiv.org/html/2606.18947#S1.p1.1),[§2](https://arxiv.org/html/2606.18947#S2.p1.1)\.
- V\. Srinivasan \(2026\)Bridging protocol and production: design patterns for deploying AI agents with model context protocol\.arXiv preprint arXiv:2603\.13417\.External Links:[Link](https://arxiv.org/abs/2603.13417)Cited by:[§2](https://arxiv.org/html/2606.18947#S2.p4.1)\.
- H\. Trivedi, N\. Balasubramanian, T\. Khot, and A\. Sabharwal \(2023\)Interleaving retrieval with chain\-of\-thought reasoning for knowledge\-intensive multi\-step questions\.InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 10014–10037\.External Links:[Document](https://dx.doi.org/10.18653/v1/2023.acl-long.557),[Link](https://aclanthology.org/2023.acl-long.557/)Cited by:[§2](https://arxiv.org/html/2606.18947#S2.p2.1),[Multi\-Hop Reasoning Constraints](https://arxiv.org/html/2606.18947#Sx1.SS0.SSS0.Px2.p1.1)\.
- T\. Vu, M\. Iyyer, X\. Wang, N\. Constant, J\. Wei, J\. Wei, C\. Tar, Y\. Sung, D\. Zhou, Q\. Le, and T\. Luong \(2024\)FreshLLMs: refreshing large language models with search engine augmentation\.InFindings of the Association for Computational Linguistics: ACL 2024,pp\. 13697–13720\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.findings-acl.813),[Link](https://aclanthology.org/2024.findings-acl.813/)Cited by:[§2](https://arxiv.org/html/2606.18947#S2.p3.1),[§5](https://arxiv.org/html/2606.18947#S5.p1.2)\.
- J\. Wei, K\. Nguyen, H\. W\. Chung, Y\. J\. Jiao, S\. Papay, A\. Glaese, J\. Schulman, and W\. Fedus \(2024\)Measuring short\-form factuality in large language models\.arXiv preprint arXiv:2411\.04368\.External Links:[Link](https://arxiv.org/abs/2411.04368)Cited by:[§2](https://arxiv.org/html/2606.18947#S2.p3.1),[§5](https://arxiv.org/html/2606.18947#S5.p1.2)\.
- R\. Xie, D\. Gopinath, D\. Qiu, D\. Lin, H\. Sun, S\. Potdar, and B\. Dhingra \(2026\)Over\-searching in search\-augmented large language models\.arXiv preprint arXiv:2601\.05503\.External Links:[Link](https://arxiv.org/abs/2601.05503)Cited by:[§1](https://arxiv.org/html/2606.18947#S1.p3.1)\.
- Z\. Yang, P\. Qi, S\. Zhang, Y\. Bengio, W\. Cohen, R\. Salakhutdinov, and C\. D\. Manning \(2018\)HotpotQA: a dataset for diverse, explainable multi\-hop question answering\.InProceedings of the 2018 Conference on Empirical Methods in Natural Language Processing,pp\. 2369–2380\.External Links:[Document](https://dx.doi.org/10.18653/v1/D18-1259),[Link](https://aclanthology.org/D18-1259/)Cited by:[§2](https://arxiv.org/html/2606.18947#S2.p3.1),[§5](https://arxiv.org/html/2606.18947#S5.p1.2)\.
- S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. Narasimhan, and Y\. Cao \(2023\)ReAct: synergizing reasoning and acting in language models\.InThe Eleventh International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=WE_vluYUL-X)Cited by:[§1](https://arxiv.org/html/2606.18947#S1.p1.1),[§2](https://arxiv.org/html/2606.18947#S2.p1.1)\.
- Y\. Zhang, S\. S\. S\. Das, and R\. Zhang \(2025\)Demystify verbosity compensation behavior of large language models\.InProceedings of the 2nd Workshop on Uncertainty\-Aware NLP \(UncertaiNLP 2025\),pp\. 160–178\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.uncertainlp-main.14),[Link](https://aclanthology.org/2025.uncertainlp-main.14/)Cited by:[§1](https://arxiv.org/html/2606.18947#S1.p3.1)\.
- L\. Zheng, W\. Chiang, Y\. Sheng, S\. Zhuang, Z\. Wu, Y\. Zhuang, Z\. Lin, Z\. Li, D\. Li, E\. P\. Xing, H\. Zhang, J\. E\. Gonzalez, and I\. Stoica \(2023\)Judging LLM\-as\-a\-judge with MT\-Bench and Chatbot Arena\.InAdvances in Neural Information Processing Systems,External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2023/hash/91f18a1287b398d378ef22505bf41832-Abstract-Datasets_and_Benchmarks.html)Cited by:[§2](https://arxiv.org/html/2606.18947#S2.p3.1),[Evaluation Methodology](https://arxiv.org/html/2606.18947#Sx1.SS0.SSS0.Px3.p1.1)\.
- Y\. Zhou, A\. I\. Muresanu, Z\. Han, K\. Paster, S\. Pitis, H\. Chan, and J\. Ba \(2023\)Large language models are human\-level prompt engineers\.InThe Eleventh International Conference on Learning Representations,External Links:[Link](https://arxiv.org/abs/2211.01910)Cited by:[§2](https://arxiv.org/html/2606.18947#S2.p2.1)\.

## Appendix ADataset Details

To ensure a comprehensive evaluation, we utilized a mix of public academic benchmarks and proprietary industry datasets\.

#### SimpleQA

A benchmark designed to evaluate the factuality of LLMs on short, single\-hop factoid questions\. It is highly sensitive to hallucinations and requires precise entity extraction\. We utilized a random subsample ofN=1,000N=1,000questions\.

#### FreshQA

A dynamic QA benchmark \(N=500N=500\) that tests a model’s ability to retrieve highly recency\-sensitive knowledge \(e\.g\., “Who won the most recent Super Bowl?”\)\. This dataset specifically tests the latency and freshness of the underlying search indices\.

#### HotpotQA

A multi\-hop reasoning dataset \(N=1,000N=1,000\) where answering the question requires retrieving and synthesizing information from multiple distinct sources\.

#### Proprietary QIU \(Retail & Tail \(Synthetic\)\)

Query Intent Understanding \(QIU\) datasets sourced from a large\-scale e\-commerce platform\. TheRetaildataset \(N=7,988N=7,988\) covers general non\-food retail queries, while theTail \(Synthetic\)dataset \(N=2,335N=2,335\) covers rare long\-tail queries\. These datasets represent the complex, noisy inputs typical of real\-world marketplace search\.

## Appendix BVisual Summary of Full Results

Figures[6](https://arxiv.org/html/2606.18947#A2.F6)–[8](https://arxiv.org/html/2606.18947#A2.F8)provide compact visual summaries of the comprehensive tables below\. The plots are intended for fast pattern inspection; the exact values and caveats remain in the corresponding tables\.

![Refer to caption](https://arxiv.org/html/2606.18947v1/x4.png)Figure 6:SimpleQA accuracy heatmap across all evaluated models and search providers\. Outlined cells mark the best configuration for each row; the mean row summarizes the overall provider pattern\.![Refer to caption](https://arxiv.org/html/2606.18947v1/x5.png)Figure 7:FreshQA accuracy heatmap\. Outlined cells mark the best configuration for each row\. Native search leads across all evaluated models on this recency\-sensitive benchmark; Exa was not run for FreshQA\.![Refer to caption](https://arxiv.org/html/2606.18947v1/x6.png)Figure 8:QIU cost\-accuracy trade\-offs for Retail and Tail \(Synthetic\)\. Exact provider results are reported in Table[5](https://arxiv.org/html/2606.18947#A6.T5)\.
## Appendix CComprehensive Results: SimpleQA

Table[2](https://arxiv.org/html/2606.18947#A3.T2)presents the comprehensive results on the SimpleQA benchmark \(N=1,000N=1,000\) across all evaluated models and search providers\. This expands upon the summary presented in Section[5](https://arxiv.org/html/2606.18947#S5)\.

Table 2:Comprehensive SimpleQA results detailing accuracy, cost, and latency across all tested configurations\.
## Appendix DComprehensive Results: FreshQA

Table[3](https://arxiv.org/html/2606.18947#A4.T3)presents the comprehensive results on the FreshQA benchmark \(N=500N=500\), which tests recency\-sensitive knowledge\. Native search is the accuracy leader across all five models; Exa was not run for FreshQA and is therefore omitted\.

Table 3:Comprehensive FreshQA results\. Evaluated using GPT\-4\.1 judge in RELAXED mode\.
## Appendix EComprehensive Results: HotpotQA

Table[4](https://arxiv.org/html/2606.18947#A5.T4)presents the comprehensive results on the HotpotQA benchmark \(N=1,000N=1,000\)\. Unlike SimpleQA, which tests single\-hop factoid retrieval, HotpotQA requires multi\-hop reasoning\. Search augmentation provides modest and model\-dependent gains, but DSG\+BrightData is the strongest overall DSG configuration and is competitive with or better than native search for most models\. All rows are reported with task\-level EM/F1 scoring\.

Table 4:Comprehensive HotpotQA results\. Exact Match \(EM\) and F1 scores are deterministic\. Results are more mixed than SimpleQA, reflecting the additional difficulty of multi\-hop retrieval and exact\-answer formatting\.
## Appendix FQIU Provider Tables

The QIU datasets are proprietary e\-commerce intent\-understanding evaluations\. Table[5](https://arxiv.org/html/2606.18947#A6.T5)reports the final provider comparison used in the main analysis\.

Tail \(Synthetic\) \(N=2,335N=2\{,\}335\)

Retail \(N=7,988N=7\{,\}988\)

Table 5:QIU provider comparisons for Tail \(Synthetic\) and Retail\.
## Appendix GAblations and Diagnostics

### G\.1SimpleQA Bootstrap Uncertainty

Table[6](https://arxiv.org/html/2606.18947#A7.T6)reports bootstrap confidence intervals for the SimpleQA frontier summarized in Figure[5](https://arxiv.org/html/2606.18947#S5.F5)\. We resample queries with replacement within each model/configuration, compute model\-level accuracy, and then average across the five evaluated models for each bootstrap replicate\.

Table 6:Bootstrap uncertainty for SimpleQA model\-mean accuracy\. Intervals use 10,000 bootstrap replicates over query\-level correctness from the primary single\-judge results\.

### G\.2Retrieval Tuning and Cache Efficiency

Table[7](https://arxiv.org/html/2606.18947#A7.T7)reports the two operational ablations summarized in Figure[4](https://arxiv.org/html/2606.18947#S5.F4): retrieval\-depth tuning and repeated\-query caching\.

max\_resultsGPT\-4oGPT\-4o\-miniClaude S4276\.8%76\.8%82\.4%482\.2%81\.8%86\.4%682\.4%84\.8%87\.4%1084\.8%85\.8%85\.8%1583\.2%86\.6%86\.8%
Table 7:Operational ablations\. Top: effect ofmax\_resultson SimpleQA accuracy with DSG\+BrightData\. Bottom: semantic cache replay on SimpleQA \(N=500N=500\), where the warm pass nearly eliminates marginal search cost and reduces average benchmark latency by 68%\.

### G\.3Multi\-Judge Validation

To ensure the reliability of our single\-judge LLM evaluation \(GPT\-4\.1\) on SimpleQA, we conducted a multi\-judge validation ablation using a 3\-judge majority verdict\. The mean 3\-judge unanimous agreement was 97\.78% \(range 95\.9%–98\.6%\)\. The 3\-judge majority provided a slight uplift \(\+1\.15pp\) over the single judge, confirming that our primary evaluation metric is highly stable and slightly conservative\.

### G\.4Prompt\-Compliance Diagnostics

Table[8](https://arxiv.org/html/2606.18947#A7.T8)summarizes the Claude Sonnet 4 HotpotQA diagnostic shown in Figure[2](https://arxiv.org/html/2606.18947#S3.F2)\. The diagnostic measures the specific format\-drift pattern discussed in Section[3](https://arxiv.org/html/2606.18947#S3); it is separate from the final HotpotQA benchmark table in Appendix[E](https://arxiv.org/html/2606.18947#A5), which uses task\-level EM/F1 scoring\.

Table 8:Prompt\-compliance diagnostic for Claude Sonnet 4 on HotpotQA\. Length is measured in output characters\. “Starts Based…” is the fraction of predictions beginning with explanatory search\-result framing; “Extractable Failures” is the fraction of exact\-match failures that still contained the target answer as a substring\.
We treat Search\-Induced Verbosity as a deployment\-risk diagnostic rather than a headline accuracy metric\. Headline accuracy uses task\-level scoring, while this diagnostic isolates whether the final response obeys the requested output contract\. The diagnostic instead isolates a production\-interface risk: in workflows that require exact entities, JSON, or categorical flags, an answer can be present but wrapped in prose that breaks downstream parsers\. DSG mitigates this risk by keeping retrieval as a structured tool response with a stable output boundary\.

#### Additional Gemini example\.

The same pattern appears in Gemini 2\.5 Flash with native grounding \(Table[9](https://arxiv.org/html/2606.18947#A7.T9)\)\. The model has the correct answer, but activating native search shifts the final response into explanatory prose that fails the exact\-output contract\.

Table 9:Additional Search\-Induced Verbosity example for Gemini 2\.5 Flash\.

## Appendix HSystem Prompts and Reproducibility

A central observation of this paper is that, in the cases we evaluated, native search integrations can weaken adherence to strict output\-format instructions\. To ensure reproducibility and transparency, we provide the exact system prompt utilized across all models during the factoid retrieval \(SimpleQA\) and multi\-hop \(HotpotQA\) evaluations\.

Standardized System PromptYou are a highly precise answering agent\. Your task is to answer the user’s question as concisely as possible\.CRITICAL INSTRUCTIONS: 1\. Provide ONLY the final answer entity or short phrase\. 2\. DO NOT include any conversational filler \(e\.g\., "The answer is\.\.\."\)\. 3\. DO NOT explain your reasoning\. 4\. If you must use a search tool to find the answer, do so, but your final output to the user must still be ONLY the exact entity\.

Despite these explicit, capitalized instructions, in our runs some models using native search \(e\.g\., Claude Sonnet 4 withweb\_search\) often returned paragraphs of text beginning with"Based on the search results…"\. In contrast, DSG processes search results as a distinct tool response rendered from structured provider\-normalized results, allowing the LLM to maintain its persona and adhere strictly to the formatting constraints\.

### H\.1Source\-Aware Context Rendering

DSG normalizes heterogeneous search responses into a common evidence record with fields such astitle,url, andcontent\. The model\-facing rendering preserves this structure as numbered snippets with explicit source URLs, rather than merging retrieved text into an opaque context block\. The public benchmark outputs report predictions and aggregate metrics, but omit raw retrieved snippets\.

Table 10:Illustrative source\-aware DSG tool context\.Qualitative inspection suggests that preserving titles and URLs helps the model distinguish primary\-source evidence from generic web snippets and prioritize evidence by source provenance\. This source\-aware rendering gives the reasoning model consistent cues about where each piece of evidence came from\.

## Appendix IDeployment Criteria Comparison

Table 11:Expanded deployment\-criteria comparison between native search and DSG\. Accuracy is not assumed to favor one interface universally: native search is strongest on freshness\-sensitive QA, while DSG matches or exceeds native on the evaluated QIU production workload and exposes deployment controls unavailable in native integrations\.