Which RAG Paradigm Wins at Scale? A Scaling Study of Retrieval-Augmented Generation Paradigms

# Which RAG Paradigm Wins at Scale? A Scaling Study of Retrieval-Augmented Generation Paradigms
Source: [https://arxiv.org/html/2607.26497](https://arxiv.org/html/2607.26497)
\\providecommand\\paperoptions

ustc,normalcite1\]University of Science and Technology of China, Hefei, China 2\]Metastone Technology, Beijing, China 3\]Information Technology Research Center, Beijing Academy of Agriculture and Forestry Sciences, Beijing, China\\contribution\[†\]Project Lead

Benfeng XuShaohan WangXin ZengHuarui WuLei ZhangLicheng Zhang\[\[\[[zlczlc@mail\.ustc\.edu\.cn](https://arxiv.org/html/2607.26497v1/mailto:[email protected])

\(July 28, 2026\)

###### Abstract

Retrieval\-augmented generation \(RAG\) methods range from lexical and dense retrieval to graph\-based indexing and agentic search\. They are usually evaluated on different benchmarks at one corpus size, leaving their accuracy–cost scaling unclear\. To bridge this gap, we present a controlled corpus\-scaling study of these four paradigms\. A ladder of 28 strictly nested tiers grows from roughly 1,000 to 512,000 documents while questions and a fixed bedrock of relevant and adversarial documents remain unchanged\. Under one reader and judging protocol, we measure official accuracy, construction/query tokens, and latency\. Our experimental results show that BM25 scales best in this controlled setting: it defines the low\-cost end of the Pareto frontier at every measured tier and leads accuracy from mid\-scale onward, without LLM\-based construction\. The File\-System Agent matches or slightly exceeds BM25 at the smallest tiers but uses 39 times more query tokens per answer at the bedrock and falls nearly 20 points behind at full scale\. A matched retrieval swap reverses this failure: Agent\+BM25 scores 69\.4 at full scale, versus 36\.9 for raw\-file agency and 54\.8 for native BM25 on the same 150 questions\. Graph\-based RAG hits a construction wall: its heaviest builders use up to 24\.6 generative\-LLM tokens per indexed corpus token yet stop within the first 2% of the full corpus, while scalable variants remain less accurate than BM25 at shared tiers\.

\\correspondence

Licheng Zhang at

## 1Introduction

Retrieval\-augmented generation grounds the outputs of large language models in external corpora, mitigating hallucination\(Lewis et al\.,[2021](https://arxiv.org/html/2607.26497#bib.bib15)\)\. Its methods have diverged into paradigms whose costs arise at different stages and in different forms\. Lexical retrieval\(Robertson and Zaragoza,[2009](https://arxiv.org/html/2607.26497#bib.bib23)\)and dense retrieval\(Karpukhin et al\.,[2020](https://arxiv.org/html/2607.26497#bib.bib12)\)require little preparation: indexing takes at most an embedding pass over the corpus\. Graph\-based RAG, including MS\-GraphRAG\(Edge et al\.,[2025](https://arxiv.org/html/2607.26497#bib.bib3)\), LightRAG\(Guo et al\.,[2025](https://arxiv.org/html/2607.26497#bib.bib5)\), and HippoRAG 2\(Gutiérrez et al\.,[2025a](https://arxiv.org/html/2607.26497#bib.bib6),[b](https://arxiv.org/html/2607.26497#bib.bib7)\), invests heavily at indexing time: it runs an LLM over every chunk to extract entities and relations, so that the resulting structure can be exploited at query time; LinearRAG\(Zhuang et al\.,[2025](https://arxiv.org/html/2607.26497#bib.bib34)\)builds the same kind of graph with a lightweight named\-entity recognizer and embeddings instead\.

File\-System Agent spends its cost at query time, using an LLM to search the corpus through iterative file\-system tool calls\(Yao et al\.,[2023](https://arxiv.org/html/2607.26497#bib.bib32)\)\. Each call is conditioned on earlier results, making the tool loop a sequential retrieval policy\. The same file\-and\-command interface is established in coding\-agent systems\(Jimenez et al\.,[2024](https://arxiv.org/html/2607.26497#bib.bib10); Yang et al\.,[2024a](https://arxiv.org/html/2607.26497#bib.bib30); Wang et al\.,[2025](https://arxiv.org/html/2607.26497#bib.bib29)\)and is used by leading industrial agents such as Claude Code and Codex\. We instantiate this interface over the corpus’ raw per\-source file tree, requiring no retrieval index\. Agents operating over graph indexes are covered by our access\-layer experiments\.

However, far less is known about how these paradigms scale\. Each is typically evaluated on its own benchmark at a single corpus size, whereas deployed corpora such as enterprise knowledge bases hold hundreds of thousands of documents and keep growing\. By scaling we mean how a paradigm behaves as the corpus grows while the workload stays fixed\.

![[Uncaptioned image]](https://arxiv.org/html/2607.26497v1/x1.png)

Figure 1:Coverage\-adjusted accuracy versus amortized token cost over six shared tiers\. Unsupported graph tiers count as zero only in this teaser; costs are averaged over completed tiers\.
Question\-relevant and adversarial evidence is also fixed from the smallest tier onward\. We measure it on three axes: answer accuracy, offline cost in construction tokens \(generative and embedding\), and online cost in query tokens and latency\. Measuring it accurately is hard, because accuracy differences are easily confounded by the reader model, the judge, the question set, and corpus difficulty\. Existing comparisons typically fix the corpus at a single size and vary these factors freely, so the scaling question has remained open\.

To bridge this gap, we conduct a controlled corpus\-scaling study on EnterpriseRAG\-Bench\(Sun et al\.,[2026](https://arxiv.org/html/2607.26497#bib.bib27)\), an enterprise corpus of 511,959 documents with 500 questions and per\-question adversarial distractors\. Scale is varied along a ladder of 28 strictly nested tiers, growing 1\.25 times per rung from 1,144 to 511,959 documents \(1\.7M to 601M tokens\), that all contain the same question\-relevant documents and distractors\. Additional documents follow one fixed, source\-and\-noise\-stratified order\. Every paradigm answers the same questions through the same reader with token\-level cost metering, and all predictions are scored under the benchmark’s official protocol, with robustness checked using an independent judge and a binary protocol\. The findings, previewed in Figure[1](https://arxiv.org/html/2607.26497#S1.F1), are consistent: BM25 defines the low\-cost end of the cost–accuracy frontier at every measured tier and leads official combined score from mid scales onward\. The File\-System Agent, the strongest alternative, matches or slightly exceeds BM25 at the smallest tiers, but spends 39 times more query tokens per answer at the bedrock and falls nearly 20 points behind at the full corpus\. Graph\-based RAG faces a dilemma: its heaviest builders reach early construction ceilings, and scalable variants remain less accurate than BM25 at every shared tier they reach\.

Our contributions are as follows\.

- •A reusable nested ladder of 28 corpus tiers that grows roughly 450\-fold in size while questions, gold documents, and distractors stay fixed\.
- •A unified scaling evaluation of seven native pipelines across four RAG paradigms, with token\-level cost metering, cross\-checked judging, and matched retrieval, harness, and substrate controls\.
- •A mechanism analysis tying BM25’s advantage to global candidate discovery: Agent\+BM25 raises the same harness from 36\.9 to 69\.4 on matched full\-corpus questions\.
- •An agentic access layer exposing each graph index through typed tools, separating structure from agency\.

## 2Related Work

Retrieval\-augmented generation\.RAG grounds generation in retrieved text through a learned or frozen reader\(Lewis et al\.,[2021](https://arxiv.org/html/2607.26497#bib.bib15); Guu et al\.,[2020](https://arxiv.org/html/2607.26497#bib.bib8); Ram et al\.,[2023](https://arxiv.org/html/2607.26497#bib.bib22)\); adaptive variants further decide when to retrieve or critique evidence\(Asai et al\.,[2023](https://arxiv.org/html/2607.26497#bib.bib1); Jiang et al\.,[2023](https://arxiv.org/html/2607.26497#bib.bib9)\)\. Retrieval ranges from lexical BM25\(Robertson and Zaragoza,[2009](https://arxiv.org/html/2607.26497#bib.bib23); Lin et al\.,[2021](https://arxiv.org/html/2607.26497#bib.bib16)\)to learned sparse, dense, and late\-interaction models\(Formal et al\.,[2021](https://arxiv.org/html/2607.26497#bib.bib4); Karpukhin et al\.,[2020](https://arxiv.org/html/2607.26497#bib.bib12); Khattab and Zaharia,[2020](https://arxiv.org/html/2607.26497#bib.bib13)\)\. We use BM25 and compact chunk embeddings as representative lexical and dense interfaces, while holding the reader fixed\.

Graph\-based RAG\.These methods construct explicit structure before answering\. MS\-GraphRAG builds hierarchical entity communities and reports\(Edge et al\.,[2025](https://arxiv.org/html/2607.26497#bib.bib3)\); LightRAG indexes entities and relations\(Guo et al\.,[2025](https://arxiv.org/html/2607.26497#bib.bib5)\); and HippoRAG 2 retrieves through query\-linked facts and Personalized PageRank\(Gutiérrez et al\.,[2025a](https://arxiv.org/html/2607.26497#bib.bib6),[b](https://arxiv.org/html/2607.26497#bib.bib7)\)\. LinearRAG instead builds an entity co\-occurrence graph with lightweight NER and embeddings, without generative build calls\(Zhuang et al\.,[2025](https://arxiv.org/html/2607.26497#bib.bib34)\)\. This family also includes tree\-, curated\-KG\-, and GNN\-based indexes\(Sarthi et al\.,[2024](https://arxiv.org/html/2607.26497#bib.bib24); Chen et al\.,[2024](https://arxiv.org/html/2607.26497#bib.bib2); Mavromatis and Karypis,[2024](https://arxiv.org/html/2607.26497#bib.bib17); Peng et al\.,[2024](https://arxiv.org/html/2607.26497#bib.bib19)\), but published evaluations generally remain at a fixed, relatively small corpus size\.

Agentic retrieval\.A third direction replaces the one\-shot pipeline with an LLM agent that interleaves reasoning and tool calls\(Yao et al\.,[2023](https://arxiv.org/html/2607.26497#bib.bib32); Schick et al\.,[2023](https://arxiv.org/html/2607.26497#bib.bib25); Qin et al\.,[2023](https://arxiv.org/html/2607.26497#bib.bib21)\), searches the open web\(Nakano et al\.,[2022](https://arxiv.org/html/2607.26497#bib.bib18)\), retrieves adaptively during multi\-step reasoning\(Trivedi et al\.,[2023](https://arxiv.org/html/2607.26497#bib.bib28); Press et al\.,[2023](https://arxiv.org/html/2607.26497#bib.bib20)\), reflects on failures\(Shinn et al\.,[2023](https://arxiv.org/html/2607.26497#bib.bib26)\), or learns a search policy with reinforcement learning\(Jin et al\.,[2025](https://arxiv.org/html/2607.26497#bib.bib11)\)\. These systems vary both the controller and the retrieval substrate, whereas our scaling question requires the controller to remain fixed\. We therefore instantiate the paradigm in the form established by coding agents: an identical tool loop navigating a corpus through plain file\-system operations\(Jimenez et al\.,[2024](https://arxiv.org/html/2607.26497#bib.bib10); Yang et al\.,[2024a](https://arxiv.org/html/2607.26497#bib.bib30); Wang et al\.,[2025](https://arxiv.org/html/2607.26497#bib.bib29)\)\. Our File\-System Agent applies this interface to the raw enterprise corpus; the substrate experiment then holds its policy model, prompt, tool loop, and budget fixed while replacing raw files with graph indexes\.

![Refer to caption](https://arxiv.org/html/2607.26497v1/x2.png)Figure 2:Evaluation methodology\. Four paradigms access the same nested corpus ladder \(T0⊂⋯⊂T27T\_\{0\}\\subset\\cdots\\subset T\_\{27\}\), which grows by 1\.25 per rung while retaining a fixed 1,144\-document bedrock\. The 500 questions split into 470 source\-grounded, 10 scaffold\-supported high\-level, and 20 not\-found questions\. Each paradigm uses the same reader and is metered on build/query tokens, latency, and official accuracy\.Benchmarks and judging\.Existing RAG benchmarks evaluate answer quality at a fixed corpus size and rarely meter offline cost\(Yang et al\.,[2024b](https://arxiv.org/html/2607.26497#bib.bib31)\)\. We extend EnterpriseRAG\-Bench’s multi\-source corpus and official evaluation\(Sun et al\.,[2026](https://arxiv.org/html/2607.26497#bib.bib27)\)with nested scaling, unified cost accounting, and cross\-paradigm comparison\. We bound judge dependence\(Zheng et al\.,[2023](https://arxiv.org/html/2607.26497#bib.bib33)\)through dual protocols and measured cross\-judge agreement\.

## 3Benchmark and Scaling Methodology

This section describes the corpus, the nested scaling ladder, and the unified answering and metering harness\. Figure[2](https://arxiv.org/html/2607.26497#S2.F2)summarizes the full pipeline\.

Corpus and questions\.EnterpriseRAG\-Bench models a fictional company that serves LLM inference\. The corpus contains 511,957 documents totaling 600\.8M tokens, drawn from nine sources that include wiki pages, chat threads, tickets, e\-mail, meeting transcripts, CRM records, and code reviews\. Together with the benchmark’s two organizational overview pages, which we include as scaffolds, the full evaluation tier holds 511,959 documents\. The benchmark provides 500 questions in ten types, ranging from basic lookups to completeness, conflicting\-information, and high\-level questions\. Each question is annotated with gold documents, a gold answer, and a list of atomic answer facts, with 722 gold documents in total\. The corpus natively marks 7\.7% of documents as noise, misfiled under a wrong source or path or near\-duplicated with outdated facts, which serves as realistic filing noise\. Depending on type, questions are generated from source documents or by an agent exploring the corpus\. The released gold set was subsequently refined by pooling candidate evidence from BM25, dense retrieval, and file\-agent search; label construction is therefore not tied to BM25 alone\.

The bedrock\.The smallest tier is constructed deterministically as the union of the 722 documents the benchmark annotates as gold for any of the 500 questions, the 326 mined traps, the 99 lures, and the corpus’ own two organizational pages, a company overview and an initiative index, which serve as scaffolds\. Removing five cross\-category duplicates yields 1,144\. Traps are mined through method\-blind filtering: for each target question, BM25 contributes its full\-corpus top\-10, while dense retrieval reranks a BM25 top\-200 pool \(top\-1,000 for not\-found questions\) and contributes ten candidates\. An LLM filter retains every candidate that concerns the same entity or topic as a gold document while reporting the wrong version, date, or decision\. This procedure yields 326 traps under a single topicality\-and\-wrong\-fact criterion\. Lures serve the info\-not\-found questions: the five most similar candidates per question that the filter verifies cannot answer it, so that unanswerable questions cannot be solved by the mere absence of retrieved text\. BM25 top\-10 and dense reranking of a BM25 top\-200 pool contribute ten candidates each; no evaluated\-system score or answer judgment enters selection\. A direct full\-corpus DenseRAG audit independently tests sensitivity to this proposal pool\. Because every adversarial document resides in the bedrock rather than being introduced gradually, question\-specific evidence and the adversarial set remain fixed as the corpus grows through added non\-bedrock documents\.

Nested tiers\.Letπ\\pidenote a single seeded, source\-and\-noise\-stratified order of the non\-bedrock corpus\. Tierttconsists of the bedrock plus the firstnt−1,144n\_\{t\}\-1\{,\}144documents ofπ\\pi\. The source and noise distribution of each added prefix approximates the global background corpus\. Sizes follownt\+1≈1\.25​ntn\_\{t\+1\}\\approx 1\.25\\,n\_\{t\}, which yields 28 tiers\. Prefix construction guaranteesT1⊂T2⊂⋯T\_\{1\}\\subset T\_\{2\}\\subset\\cdotsexactly, verified through manifest checksums, and permits incremental builders to extend an index from one tier to the next, which is also how marginal build cost is metered\. We report scale in corpus tokens rather than document counts because document lengths differ across sources\.

## 4Unified Evaluation Protocol

Paradigms\.Our main scaling ladder evaluates seven native pipelines\. BM25 uses an inverted index built without LLM involvement\. DenseRAG performs dense retrieval over chunk embeddings\. HippoRAG 2 builds an open\-vocabulary triple graph and answers queries by linking them to facts and running Personalized PageRank\. MS\-GraphRAG builds an entity graph with hierarchical community reports and answers through its local search mode\. LightRAG maintains dual\-level entity and relation indexes and answers through its hybrid mode\. LinearRAG constructs an entity co\-occurrence graph using a lightweight named\-entity recognizer and the shared embedding model, without generative\-LLM calls\. The File\-System Agent operates without any index: an agent explores the raw per\-source file tree using read\-only listing, search, and reading tools under a budget of 80 LLM calls per question\. Every method that exposes a retrieval depth uses the top\-5 chunks\. BM25, DenseRAG, and HippoRAG 2 operate on the same chunk segmentation, so their retrieval quality is directly comparable\. MS\-GraphRAG and LightRAG chunk internally, and their retrieved evidence is matched back to the shared segmentation for recall computation\.

Reader and metering\.All paradigms use the same reader, Qwen3\.6\-27B at temperature zero, served by vLLM\(Kwon et al\.,[2023](https://arxiv.org/html/2607.26497#bib.bib14)\)\. The File\-System Agent uses the same model as its policy, and all main\-pipeline embedding calls go through a shared Qwen3\-Embedding\-0\.6B model\. A shared metering layer intercepts every LLM call and attributes prompt and completion tokens to the build phase or the query phase, per paradigm and per question\. Embedding calls are accounted separately from generative calls: because embedding requests carry no prompt template, their token cost is counted from tokenizer inputs\. DenseRAG counts are exact; LinearRAG includes a documented 3% estimate for its unavailable entity\-name stream\. Latency is measured end to end under single\-stream conditions on an idle server\.

Judging\.Every prediction of every family is scored with the benchmark’s official protocol: holistic alignment of the candidate against the gold answer, the fraction of atomic answer facts entailed by the candidate, a combined score that gates completeness by correctness, and document\-level recall of the retrieved evidence\. All main\-ladder cells were produced in one scoring session with one judge model\. Each matched control is likewise scored in a single shared session with identical prompts and judge model; control estimates are not mixed into the main ladder\. Each system is run once per question; uncertainty resamples questions rather than stochastic reruns\. To establish that the results do not depend on this choice, we re\-scored the predictions with an independent judge and with a simpler binary protocol\. The binary protocol preserves rankings at all nine shared scales, while the independent judge agrees on 96\.2% of pooled alignment verdicts; details follow below\.

## 5Results

Table 1:Official main results\. Build/query tokens, completeness, and document recall are bedrock values\. Combined score gates completeness by correctness\. Dashes denote unbuilt or unevaluated tiers; Figure[3](https://arxiv.org/html/2607.26497#S5.F3)shows every measured tier\.‡denotes non\-generative construction: embeddings for DenseRAG, and local NER plus embeddings for LinearRAG\.§marks LightRAG runs that did not log retrieved\-evidence identifiers\. LinearRAG tok/q averages its fixed, stratified 100\-question metered run\.### 5\.1Accuracy versus Corpus Scale

Table[1](https://arxiv.org/html/2607.26497#S5.T1)and Figure[3](https://arxiv.org/html/2607.26497#S5.F3)show that at the bedrock, File\-System Agent and BM25 lead with 77\.4 and 74\.7; their 95% intervals \(73\.9–80\.8 and 71\.4–77\.9\) overlap\. At full scale, BM25 retains 50\.5, versus 30\.7 for File\-System Agent and 29\.9 for DenseRAG\. Under the fixed 80\-call budget used here, iterative raw\-file search resists corpus growth less well than global lexical ranking despite many more model calls, while dense retrieval remains below both\.

HippoRAG 2 and LinearRAG stop at 131,876 documents, MS\-GraphRAG at 8,750, and LightRAG at 2,254\. Thus large\-scale graph results are about construction feasibility, not extrapolated accuracy: unsupported tiers remain missing in the main curves and count as zero only in the coverage\-adjusted teaser\.

### 5\.2Build Cost and Scaling Walls

Figure[4](https://arxiv.org/html/2607.26497#S5.F4)and Table[2](https://arxiv.org/html/2607.26497#S5.T2)identify construction as the graph families’ scaling wall\. HippoRAG 2 is approximately linear \(b=1\.01b=1\.01\), extrapolating to 2\.9B generative tokens and roughly three single\-instance days at full scale\. Yet at 155M corpus tokens its 724M\-token build scores 41\.0, about fifteen points below BM25\. MS\-GraphRAG also fits near\-linear growth and completes through 8,750 documents; its full\-scale estimate is 7\.9B tokens and about 50 instance\-days\.

![[Uncaptioned image]](https://arxiv.org/html/2607.26497v1/x3.png)

Figure 3:Official combined score with 95% confidence bands\. File\-System Agent ties BM25 at the smallest tiers; BM25 leads from mid\-scale onward\. Graph curves end at their last completed build\.

Table 2:Build\-cost fitsC​\(x\)=a​xbC\(x\)=ax^\{b\}against corpus tokensxx, extrapolated to the 600\.8M\-token full corpus\.LightRAG is super\-linear \(b=1\.36b=1\.36\), does not complete at 2,826 documents within our resource envelope, and extrapolates to 102B tokens or four instance\-years\. Even an optimisticb=1\.2b=1\.2leaves its estimate near 42B tokens\.

Token\-based fits isolate construction work from hardware throughput\. LightRAG’s repeated entity\-merge rewrites yield super\-linear growth; even at three times the measured throughput, its projected full\-scale build exceeds one instance\-year\. Embedding\-only construction is far cheaper: DenseRAG completes the full corpus with 659\.4M embedding tokens\. LinearRAG reports zero generative construction tokens, but still incurs local NER and embedding work\.

![[Uncaptioned image]](https://arxiv.org/html/2607.26497v1/x4.png)

Figure 4:Construction tokens and fitted power laws\. Hollow markers report embedding tokens for the two non\-generative builders; BM25 and the File\-System Agent require no index\.
Parallelism can reduce calendar time but not total construction work or the resulting accuracy\.

### 5\.3Query Cost and Latency

Figure[5](https://arxiv.org/html/2607.26497#S5.F5)reports end\-to\-end latency\. Query cost is nearly scale\-invariant for the one\-shot pipelines: BM25, DenseRAG, and HippoRAG 2 use 5\.8K, 4\.9K, and 6\.5K tokens per question, dominated by the shared reader prompt\. File\-System Agent instead grows from 226K at the bedrock to 343K atN=21,614N=21\{,\}614, respectively 39 and 60 times BM25 as exploration deepens\. Its median LLM calls rise from 5 to 8 byN=42,587N=42\{,\}587; budget exhaustion is below 7% through that tier, but reaches 15% atN=131,876N=131\{,\}876and 31% at full scale\. Accuracy also falls among within\-budget questions, so truncation alone does not explain the collapse\.

Amortized over 500 questions, BM25 obtains 74\.7 at 2\.9M tokens, versus 77\.4 at 112\.8M for File\-System Agent\. HippoRAG 2 reaches 66\.2 at 10\.7M, while MS\-GraphRAG and LightRAG remain dominated near 40M tokens\. The BM25 point remains on the frontier for amortization horizons from 10 to 10,000\.

![[Uncaptioned image]](https://arxiv.org/html/2607.26497v1/x5.png)

Figure 5:Single\-stream query latency\. Lines report medians; the File\-System band spans P10–P90\.

Table 3:Official score of the same agent harness over each retrieval substrate, with native\-pipeline scores on identical questions in parentheses\. Bedrock uses all 500 questions and later tiers a fixed 150\-question subset for both members of each pair\. Files has no native pipeline; dashes denote unavailable matched evaluations\. These scores come from a separate matched scoring session and are not mixed with Table[1](https://arxiv.org/html/2607.26497#S5.T1)\.
### 5\.4Behavior by Question Type

Figure[6](https://arxiv.org/html/2607.26497#S5.F6)shows that atN=42,587N=42\{,\}587, File\-System Agent leads BM25 on intra\-document, project\-related, completeness, and conflicting\-information questions, including 56 versus 27 on completeness\.

![Refer to caption](https://arxiv.org/html/2607.26497v1/x6.png)Figure 6:Question\-type score atN=42,587N=42\{,\}587\(counts in labels\); MS\-GraphRAG and LightRAG are unavailable\.BM25 is best or tied on the other five types\. The saturated not\-found scores mainly measure abstention: one\-shot readers decline without supporting evidence, whereas iterative exploration can commit to an unsupported answer\. A graph system leads only on miscellaneous\.

### 5\.5Cross\-Protocol Robustness

Re\-scoring with the binary protocol preserves every family ranking at nine shared scales\. Official correctness is 2\.60 points lower on average, with a cell range of−4\.60\-4\.60to\+0\.69\+0\.69\. An independent official judge agrees on 96\.2% of pooled alignment verdicts \(94\.4–98\.0% per cell\), with combined\-score changes from−3\.56\-3\.56to\+1\.18\+1\.18\. Close\-pair rankings can vary, while the broad separation across retrieval families remains\.

At the bedrock, BM25 retrieves any annotated gold chunk for 94\.7% of the 470 answerable questions, all annotated gold chunks for 56\.4%, and averages 75\.1% gold\-chunk recall at five\. Its document recall is 90\.3%, placing many residual errors in within\-document passage selection or evidence synthesis\.

## 6Separating Agency from Structure

### 6\.1Retrieval Before Agency

The full\-scale BM25–File\-System gap is a candidate\-discovery failure, not weak evidence synthesis\. We isolate the retrieval primitive on the same fixed, stratified 150 questions at the bedrock and full\-corpus tiers—300 question–scale pairs, not 300 unique questions\. Agent\+BM25 uses the File\-System Agent’s model, prompt, harness, and 80\-call budget, but replaces raw\-tree search with ranked lexical search and chunk reading\. Its first search is forced to use the original question; an audit confirms that its ordered top\-5 exactly matches Native BM25 on every pair\.

Table 4:Retrieval\-primitive control on the same 150 questions at each scale\. Combined score and full\-scale document recall \(DocR\) come from one shared rejudge\. Full\-scale calls and tokens/question include failed attempts that were retried\. Native BM25 tokens/question is the descriptive full\-500 average\.Table 5:Same\-session harness control atN=N\{=\}1,144 on the same 150\-question subset\. Policy model, raw\-file substrate, budget, and judge are fixed; scores are not compared across sessions\.Table[4](https://arxiv.org/html/2607.26497#S6.T4)shows that at the bedrock, Native BM25 and the File\-System Agent are statistically tied: BM25 trails by 5\.73 points, with a paired bootstrap 95% CI of\[−11\.57,0\.15\]\[\-11\.57,0\.15\]\. At the full corpus, BM25 instead leads by 17\.97 points\[9\.68,26\.14\]\[9\.68,26\.14\]\. The mechanism is discovery\. Among the queries on which each method finds at least one gold document—a descriptive, selection\-conditioned slice—the File\-System Agent scores 85\.9 versus BM25’s 73\.8, but its any\-gold hit rate collapses to 39\.0%, against 71\.6% for BM25\.

Changing only the retrieval primitive reverses the collapse\. At full scale Agent\+BM25 scores 69\.4, beating the raw\-file agent by 32\.52 points\[24\.07,40\.90\]\[24\.07,40\.90\]and Native BM25 by 14\.56 points\[9\.22,20\.12\]\[9\.22,20\.12\]\. Its any\-gold hit rate reaches 78\.0%, while its 101K tokens/question are roughly one ninth of raw\-file exploration\. Agency therefore helps after globally ranked candidate discovery; repeated local search is not a substitute for that global ranking\. We treat this two\-scale intervention as a mechanism control, not as an eighth system on the 28\-tier native\-pipeline ladder\.

Table[5](https://arxiv.org/html/2607.26497#S6.T5)shows that within this matched resweep, harness choice matters, and ours achieves the highest score of the three file agents\. Agent\+BM25 therefore evaluates the retrieval swap with the strongest observed raw\-file implementation\.

### 6\.2Agency over Graph Substrates

The File\-System Agent result leaves a confound: its strength may derive from agency, that is, iterative and self\-correcting retrieval, or from its substrate, the raw files\. To separate the two we built a paradigm\-neutral agentic access layer\. Each paradigm’s index is served by a resident process that exposes typed, read\-only tools: semantic search over entities, facts, relations, and community reports; neighborhood expansion that includes the predicate text discarded by the original pipeline; Personalized PageRank from seed entities; and chunk reading\. The paradigm’s native one\-shot ranker is also exposed as a single tool, so the agent always has the pipeline available as a lower bound\. The tools are offered to the same tool\-calling harness that drives the File\-System Agent, with the model, budget, and judge held fixed, so the substrate is the only variable\. A command\-line form of the same tools is included in the artifact\. Agents on graph substrates have no access to raw files, so any gain is attributable to the index contents\.

Table[3](https://arxiv.org/html/2607.26497#S5.T3)reports the fully matched comparison\. For LightRAG, agentic access changes the native score by 22\.2 to 29\.9 points across 4 matched tiers\. For HippoRAG2, agentic access changes the native score by \-0\.6 to 14\.5 points across 7 matched tiers\. For MS\-GraphRAG, agentic access changes the native score by \-6\.7 to 0\.4 points across 5 matched tiers\. The result separates substrate quality from retrieval policy: the same stored index can support materially different outcomes when exposed to an agent rather than its native one\-shot ranker\.

## 7Discussion

The lexical advantage\.Enterprise questions contain precise lexical anchors, while traps are semantically similar but factually wrong; exact matching is therefore advantaged\. Table[6](https://arxiv.org/html/2607.26497#S7.T6)evaluates the ordering’s sensitivity to question wording and retrieval depth\.

Table 6:Wording and depth controls: official combined score \(%\); dashes denote unmeasured pairs\. Scores usen=148n=148–150\.These controls leave BM25 above dense and graph retrieval under comparable retrieval depth\. In a 90\-question proposal\-sensitivity audit, direct full\-corpus DenseRAG top\-10 retrieval overlaps the historical BM25\-prefiltered dense candidates by 1\.2 documents on average, recovers 57 of 431 confirmed items, and identifies 115 traps plus 100 not\-found lures\. Together, these controls show that BM25’s advantage persists under altered wording and matched retrieval depth, while adversarial candidates are also recoverable through a direct dense proposal path\.

Graph failure modes\.The shortfall decomposes into three parts\. Construction introduces extraction noise, as the 32K entities extracted from the 1,144\-document bedrock include malformed fragments, and incurs prohibitive cost at scale\. Retrieval over the graph ranks semantically related but factually wrong neighborhoods highly, which is exactly what the traps punish\. Construction also loses information: the HippoRAG 2 graph stores only co\-occurrence weights and discards the predicate text that its own extraction produced, so the reader never sees relation semantics\. That LinearRAG, whose construction involves no generative LLM, is within 1\.8 points of MS\-GraphRAG and LightRAG at the bedrock suggests that LLM\-extracted structure adds cost faster than it adds signal on this workload\.

Scaling mechanism\.Across paradigms, the decisive factor is how each system obtains a corpus\-wide candidate set\. BM25 and DenseRAG amortize global ranking in an index, keeping query cost nearly constant; on this corpus, lexical matching also rejects semantically similar but factually incorrect traps more effectively\. The File\-System Agent instead discovers candidates sequentially through local tree navigation, so each question pays for exploration and relevant branches become harder to reach as the corpus grows\. Graph methods recover global structure only after corpus\-wide extraction, where build cost, entity noise, and representation loss become the bottleneck\. Agent\+BM25 isolates the mechanism: at full scale, changing only retrieval reduces calls from 36\.12 to 5\.79 and tokens from 895K to 101K per question, while document recall rises from 36\.8 to 72\.4 and combined score from 36\.9 to 69\.4\. Iteration therefore helps most after globally ranked discovery, rather than serving as its substitute\.

Evaluation implication\.A single corpus size can conceal scaling behavior\. At the bedrock, raw\-file agency matches or exceeds lexical retrieval, but their ordering separates as distractors accumulate; graph systems can be compared only where their indexes complete\. Cross\-paradigm evaluations should therefore report accuracy over nested scales, construction and query cost, and index coverage\. Measured cells capture answer quality conditional on completed indexes; coverage\-adjusted summaries additionally capture deployability\. An unavailable tier is not an observed answer failure: assigning it zero measures deployment coverage, whereas omitting it measures quality conditional on successful construction\. Build and query costs should likewise remain separate before workload\-specific amortization, because graph indexes pay primarily before deployment while raw\-file agents pay repeatedly online\. We report both—tier\-level tables and the coverage\-adjusted Figure[1](https://arxiv.org/html/2607.26497#S1.F1)—so answer quality and scalability are not conflated\.

Practical guidance\.For enterprise\-style corpora, BM25 is the appropriate default\. Agent\+BM25 is the practical hybrid for aggregation\-heavy questions: use lexical ranking for global discovery, then spend agentic calls on narrowed candidates\. LLM\-built graph indexes are difficult to justify at10510^\{5\}–10610^\{6\}documents unless construction is near\-linear and relational questions dominate the workload\.

## 8Conclusion

We presented a controlled scaling study of lexical, dense, graph\-based, and agentic RAG on an enterprise corpus\. Our 28\-tier nested ladder spans 1,144 to 511,959 documents while holding questions, relevant evidence, and adversarial distractors fixed; a common reader and judge, token\-level construction and query metering, and matched access\-layer and retrieval\-swap controls isolate corpus growth and separate retrieval substrate from agentic policy\. Under this protocol, BM25 remains Pareto\-optimal at every measured scale and leads official combined score from mid scale onward without LLM\-based construction\. Raw\-file agency loses global candidate discovery as the corpus grows, whereas replacing raw search with BM25 raises the full\-scale agent from 36\.9 to 69\.4 using roughly one ninth of its query tokens\. Graph\-based RAG encounters construction walls, and its scalable variants remain below BM25 at shared tiers\. Overall, global lexical ranking is a strong default for enterprise\-scale RAG, with agentic reasoning best applied after candidate ranking\.

## References

- Asai et al\. \(2023\)Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi\.Self\-RAG: Learning to retrieve, generate, and critique through self\-reflection, 2023\.[https://arxiv\.org/abs/2310\.11511](https://arxiv.org/abs/2310.11511)\.
- Chen et al\. \(2024\)Liyi Chen, Panrong Tong, Zhongming Jin, Ying Sun, Jieping Ye, and Hui Xiong\.Plan\-on\-Graph: Self\-correcting adaptive planning of large language model on knowledge graphs, 2024\.[https://arxiv\.org/abs/2410\.23875](https://arxiv.org/abs/2410.23875)\.
- Edge et al\. \(2025\)Darren Edge, Ha Trinh, Newman Cheng, Joshua Bradley, Alex Chao, Apurva Mody, Steven Truitt, Dasha Metropolitansky, Robert Osazuwa Ness, and Jonathan Larson\.From local to global: A Graph RAG approach to query\-focused summarization, 2025\.[https://arxiv\.org/abs/2404\.16130](https://arxiv.org/abs/2404.16130)\.
- Formal et al\. \(2021\)Thibault Formal, Benjamin Piwowarski, and Stéphane Clinchant\.SPLADE: Sparse lexical and expansion model for first stage ranking, 2021\.[https://arxiv\.org/abs/2107\.05720](https://arxiv.org/abs/2107.05720)\.
- Guo et al\. \(2025\)Zirui Guo, Lianghao Xia, Yanhua Yu, Tu Ao, and Chao Huang\.LightRAG: Simple and fast retrieval\-augmented generation, 2025\.[https://arxiv\.org/abs/2410\.05779](https://arxiv.org/abs/2410.05779)\.
- Gutiérrez et al\. \(2025a\)Bernal Jiménez Gutiérrez, Yiheng Shu, Yu Gu, Michihiro Yasunaga, and Yu Su\.HippoRAG: Neurobiologically inspired long\-term memory for large language models, 2025a\.[https://arxiv\.org/abs/2405\.14831](https://arxiv.org/abs/2405.14831)\.
- Gutiérrez et al\. \(2025b\)Bernal Jiménez Gutiérrez, Yiheng Shu, Weijian Qi, Sizhe Zhou, and Yu Su\.From RAG to memory: Non\-parametric continual learning for large language models, 2025b\.[https://arxiv\.org/abs/2502\.14802](https://arxiv.org/abs/2502.14802)\.
- Guu et al\. \(2020\)Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming\-Wei Chang\.REALM: Retrieval\-augmented language model pre\-training, 2020\.[https://arxiv\.org/abs/2002\.08909](https://arxiv.org/abs/2002.08909)\.
- Jiang et al\. \(2023\)Zhengbao Jiang, Frank F Xu, Luyu Gao, Zhiqing Sun, Qian Liu, Jane Dwivedi\-Yu, Yiming Yang, Jamie Callan, and Graham Neubig\.Active retrieval augmented generation, 2023\.[https://arxiv\.org/abs/2305\.06983](https://arxiv.org/abs/2305.06983)\.
- Jimenez et al\. \(2024\)Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan\.SWE\-bench: Can language models resolve real\-world GitHub issues?, 2024\.[https://arxiv\.org/abs/2310\.06770](https://arxiv.org/abs/2310.06770)\.
- Jin et al\. \(2025\)Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Jiawei Han\.Search\-R1: Training LLMs to reason and leverage search engines with reinforcement learning, 2025\.[https://arxiv\.org/abs/2503\.09516](https://arxiv.org/abs/2503.09516)\.
- Karpukhin et al\. \(2020\)Vladimir Karpukhin, Barlas Oğuz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen\-tau Yih\.Dense passage retrieval for open\-domain question answering, 2020\.[https://arxiv\.org/abs/2004\.04906](https://arxiv.org/abs/2004.04906)\.
- Khattab and Zaharia \(2020\)Omar Khattab and Matei Zaharia\.ColBERT: Efficient and effective passage search via contextualized late interaction over BERT, 2020\.[https://arxiv\.org/abs/2004\.12832](https://arxiv.org/abs/2004.12832)\.
- Kwon et al\. \(2023\)Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E\. Gonzalez, Hao Zhang, and Ion Stoica\.Efficient memory management for large language model serving with PagedAttention, 2023\.[https://arxiv\.org/abs/2309\.06180](https://arxiv.org/abs/2309.06180)\.
- Lewis et al\. \(2021\)Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen\-tau Yih, Tim Rocktäschel, et al\.Retrieval\-augmented generation for knowledge\-intensive NLP tasks, 2021\.[https://arxiv\.org/abs/2005\.11401](https://arxiv.org/abs/2005.11401)\.
- Lin et al\. \(2021\)Jimmy Lin, Xueguang Ma, Sheng\-Chieh Lin, Jheng\-Hong Yang, Ronak Pradeep, and Rodrigo Nogueira\.Pyserini: An easy\-to\-use python toolkit to support replicable IR research with sparse and dense representations, 2021\.[https://arxiv\.org/abs/2102\.10073](https://arxiv.org/abs/2102.10073)\.
- Mavromatis and Karypis \(2024\)Costas Mavromatis and George Karypis\.GNN\-RAG: Graph neural retrieval for large language model reasoning, 2024\.[https://arxiv\.org/abs/2405\.20139](https://arxiv.org/abs/2405.20139)\.
- Nakano et al\. \(2022\)Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, et al\.WebGPT: Browser\-assisted question\-answering with human feedback, 2022\.[https://arxiv\.org/abs/2112\.09332](https://arxiv.org/abs/2112.09332)\.
- Peng et al\. \(2024\)Boci Peng, Yun Zhu, Yongchao Liu, Xiaohe Bo, Haizhou Shi, Chuntao Hong, Yan Zhang, and Siliang Tang\.Graph retrieval\-augmented generation: A survey, 2024\.[https://arxiv\.org/abs/2408\.08921](https://arxiv.org/abs/2408.08921)\.
- Press et al\. \(2023\)Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah A Smith, and Mike Lewis\.Measuring and narrowing the compositionality gap in language models, 2023\.[https://arxiv\.org/abs/2210\.03350](https://arxiv.org/abs/2210.03350)\.
- Qin et al\. \(2023\)Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, et al\.ToolLLM: Facilitating large language models to master 16000\+ real\-world APIs, 2023\.[https://arxiv\.org/abs/2307\.16789](https://arxiv.org/abs/2307.16789)\.
- Ram et al\. \(2023\)Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton\-Brown, and Yoav Shoham\.In\-context retrieval\-augmented language models, 2023\.[https://arxiv\.org/abs/2302\.00083](https://arxiv.org/abs/2302.00083)\.
- Robertson and Zaragoza \(2009\)Stephen Robertson and Hugo Zaragoza\.The probabilistic relevance framework: BM25 and beyond\.*Foundations and Trends in Information Retrieval*, 3\(4\):333–389, 2009\.[10\.1561/1500000019](https://arxiv.org/doi.org/10.1561/1500000019)\.
- Sarthi et al\. \(2024\)Parth Sarthi, Salman Abdullah, Aditi Tuli, Shubh Khanna, Anna Goldie, and Christopher D\. Manning\.RAPTOR: Recursive abstractive processing for tree\-organized retrieval, 2024\.[https://arxiv\.org/abs/2401\.18059](https://arxiv.org/abs/2401.18059)\.
- Schick et al\. \(2023\)Timo Schick, Jane Dwivedi\-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom\.Toolformer: Language models can teach themselves to use tools, 2023\.[https://arxiv\.org/abs/2302\.04761](https://arxiv.org/abs/2302.04761)\.
- Shinn et al\. \(2023\)Noah Shinn, Federico Cassano, Edward Berman, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao\.Reflexion: Language agents with verbal reinforcement learning, 2023\.[https://arxiv\.org/abs/2303\.11366](https://arxiv.org/abs/2303.11366)\.
- Sun et al\. \(2026\)Yuhong Sun, Joachim Rahmfeld, Chris Weaver, Weijia Chen, Roshan Desai, Wenxi Huang, and Mark H\. Butler\.EnterpriseRAG\-Bench: A RAG benchmark for company internal knowledge, 2026\.[https://arxiv\.org/abs/2605\.05253](https://arxiv.org/abs/2605.05253)\.
- Trivedi et al\. \(2023\)Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal\.Interleaving retrieval with chain\-of\-thought reasoning for knowledge\-intensive multi\-step questions, 2023\.[https://arxiv\.org/abs/2212\.10509](https://arxiv.org/abs/2212.10509)\.
- Wang et al\. \(2025\)Xingyao Wang, Boxuan Li, Yufan Song, Frank F\. Xu, Xiangru Tang, Mingchen Zhuge, Jiayi Pan, Yueqi Song, et al\.OpenHands: An open platform for AI software developers as generalist agents, 2025\.[https://arxiv\.org/abs/2407\.16741](https://arxiv.org/abs/2407.16741)\.
- Yang et al\. \(2024a\)John Yang, Carlos E\. Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press\.SWE\-agent: Agent\-computer interfaces enable automated software engineering, 2024a\.[https://arxiv\.org/abs/2405\.15793](https://arxiv.org/abs/2405.15793)\.
- Yang et al\. \(2024b\)Xiao Yang, Kai Sun, Hao Xin, Yushi Sun, Nikita Bhalla, Xiangsen Chen, Sajal Choudhary, Rongze Daniel Gui, et al\.CRAG – comprehensive RAG benchmark, 2024b\.[https://arxiv\.org/abs/2406\.04744](https://arxiv.org/abs/2406.04744)\.
- Yao et al\. \(2023\)Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao\.ReAct: Synergizing reasoning and acting in language models, 2023\.[https://arxiv\.org/abs/2210\.03629](https://arxiv.org/abs/2210.03629)\.
- Zheng et al\. \(2023\)Lianmin Zheng, Wei\-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, et al\.Judging LLM\-as\-a\-Judge with MT\-Bench and chatbot arena, 2023\.[https://arxiv\.org/abs/2306\.05685](https://arxiv.org/abs/2306.05685)\.
- Zhuang et al\. \(2025\)Luyao Zhuang, Shengyuan Chen, Yilin Xiao, Huachi Zhou, Yujing Zhang, Hao Chen, Qinggang Zhang, and Xiao Huang\.LinearRAG: Linear graph retrieval augmented generation on large\-scale corpora, 2025\.[https://arxiv\.org/abs/2510\.10114](https://arxiv.org/abs/2510.10114)\.

## Appendix AScope

This supplement provides additional experimental detail, result tables, and reproducibility notes for the main paper\. The main paper is self\-contained; this document expands the protocol and gives additional audit material\. All identifiers are anonymized, and all local machine paths are omitted\.

## Appendix BCorpus Ladder

The study uses EnterpriseRAG\-Bench, a public synthetic enterprise benchmark with slightly more than 500K documents and 500 questions\. The scaling ladder is built by first fixing a bedrock tier that contains all question\-relevant documents, hard negatives, and not\-found lures\. Every larger tier appends a seeded, source\- and noise\-stratified prefix of the remaining corpus\. Thus the question set, relevant evidence, and adversarial documents are fixed while only background corpus size grows\. Manifest checks verify exact nesting\.

Table 7:The 28 strictly nested corpus tiers\. Corpus tokens are measured with the shared tokenizer; chunks are produced by the shared 1,200\-token chunker with 100\-token overlap for systems that use the shared chunks\.
## Appendix CMethod Configuration

All native pipelines are evaluated under one reader and judging protocol\. The shared reader is Qwen3\.6\-27B served with temperature zero and thinking disabled\. The shared embedding model is Qwen3\-Embedding\-0\.6B\. BM25, DenseRAG, and HippoRAG 2 use the same shared chunks\. MS\-GraphRAG and LightRAG use their native internal chunking, and retrieved evidence is mapped back to the shared chunk identifiers when document recall is available\.

Table 8:Shared settings used across native pipelines and matched controls\.### C\.1Reader Prompt

All non\-agentic pipelines use the same reader prompt\. The system message is:

Reader system promptYou are a retrieval\-based QA assistant\. Answer the QUESTION using ONLY the information in the CONTEXT\. Do not use outside knowledge or invent facts\. If the CONTEXT does not contain enough information to answer, say so explicitly\. Answer in the same language as the QUESTION; be concise\.

The user message template is:

Reader user\-message templateCONTEXT: \{context\}QUESTION: \{question\} ANSWER:

### C\.2File\-System Agent Prompt and Tools

The File\-System Agent uses the same Qwen3\.6\-27B model as its policy model\. The prompt gives only read\-only access to the corpus tree:

File\-System Agent system promptYou are a research assistant over an enterprise document corpus, organized as files under source folders \(slack, gmail, jira, confluence, google\_drive, linear, github, fireflies, hubspot\)\. Use list\_dir to orient, grep to locate relevant documents by keyword, and read\_doc to read them\. Answer ONLY from the documents\. Be concise and factual, and cite the relative file paths you used\.

Table 9:Read\-only tools exposed to the raw File\-System Agent\. Path resolution rejects traversal outside the corpus root\.
### C\.3Agent\+BM25 Prompt and Control Guarantee

Agent\+BM25 uses the same agent harness and budget, but replaces raw tree search with a ranked BM25 search tool\. Its system message is:

Agent\+BM25 system promptYou are a research assistant over an enterprise document corpus\. Use bm25\_search with the user’s original question first\. Inspect the returned top\-5 chunks carefully\. If they are sufficient, answer immediately; otherwise reformulate the search query or use read\_doc on a returned source path to inspect more context\. Answer ONLY from retrieved documents\. Be concise and factual, and cite source paths or chunk IDs\. Do not use outside knowledge\.

The firstbm25\_searchcall is programmatically forced to use the original question, regardless of the tool argument\. This makes the first returned top\-5 exactly the native BM25 top\-5\. Later calls may use agent reformulations and are the agentic part of the intervention\.

### C\.4Official Judge Prompts

The official combined score has two LLM\-judged components and one non\-LLM document\-recall component\. First, a holistic answer\-alignment prompt receives the query, gold answer, and candidate answer and asks for a JSON object with a one\-sentence reason and analignedfield whose value isyesorno\. The prompt instructs the judge to accept stylistic differences and extra relevant context, but to reject contradictions, wrong quantities, or missing core parts of the query\. Second, each atomic answer fact is validated with a yes/no prompt asking whether the candidate answer contains and is consistent with the statement\. The combined score for a question is:

combined​\(q\)=\{completeness​\(q\),aligned​\(q\)=1,0,aligned​\(q\)=0\.\\mathrm\{combined\}\(q\)=\\begin\{cases\}\\mathrm\{completeness\}\(q\),&\\mathrm\{aligned\}\(q\)=1,\\\\ 0,&\\mathrm\{aligned\}\(q\)=0\.\\end\{cases\}Reported cell scores are the mean combined score over the evaluated questions\. Document recall is computed by exact set overlap between retrieved document identifiers and gold document identifiers for answerable questions\.

## Appendix DFigure and Table Metrics

The main ladder reports native\-pipeline scores only for tiers actually built and evaluated\. Dashes in the tables therefore denote missing native results rather than extrapolated failures\.

Figure 1 is a compact teaser over six fixed tiers \(N=1,144,2,254,6,980,42,587,131,876,511,959N=1\{,\}144,2\{,\}254,6\{,\}980,42\{,\}587,131\{,\}876,511\{,\}959\) and one fixed 150\-question sample\. Its vertical coordinate is coverage\-adjusted: unsupported graph tiers are assigned zero only for this summary figure\. Its horizontal coordinate is the mean over completed tiers of construction cost plus a normalized 500\-query workload\. Embedding tokens are parameter\-weighted by0\.6/270\.6/27when combined with 27B reader or generative tokens\. The point ledger used by the plot is included asfigures/teaser\_crossscale\_points\.csvin the code/data supplement\.

Table 1 in the main paper reports bedrock build/query tokens and bedrock completeness/recall alongside selected ladder scores\. Table 2 fits build\-token scaling laws of the formy=a​xby=ax^\{b\}on measured build tokens against corpus tokens and projects to the 600\.8M\-token full corpus\. These projections are not used as hidden accuracy estimates; the accuracy curves show only measured cells\.

## Appendix EToken Metering

The metering layer records prompt, completion, and total tokens for each LLM call and assigns the call to construction or query phases\. For systems that expose OpenAI\-compatible responses, usage fields are read directly from the response\. For HippoRAG 2, build tokens are recovered from its LLM cache metadata\. For MS\-GraphRAG, build tokens are recovered from its indexing logs\. Embedding calls are counted from tokenizer inputs and reported separately from generative LLM calls\. LinearRAG’s zero generative\-build entry therefore means zero generative LLM construction tokens, not zero CPU work, storage, local NER, or embedding work\.

Table 10:Token\-accounting sources by family\. “Zero” denotes zero model\-token construction cost, not zero CPU, storage, or indexing work\.
## Appendix FFull Native\-Ladder Scores

Table[12](https://arxiv.org/html/2607.26497#A6.T12)and Table[12](https://arxiv.org/html/2607.26497#A6.T12)expand the main result matrix beyond the subset displayed in the main paper\. Dashes denote tiers that were not built or not evaluated for that native pipeline\.

Table 11:Official combined score \(%\) for smaller native\-ladder tiers\.Table 12:Official combined score \(%\) for larger native\-ladder tiers\.
## Appendix GMatched Controls

The retrieval\-primitive control replaces raw file\-tree exploration with BM25 search while holding the agent model, prompt, tools, and 80\-call budget fixed\. The first search is forced to use the original question, and an audit confirms that the returned top\-5 order equals the native BM25 top\-5 on each matched pair before any further agentic reasoning\. This isolates global candidate discovery from the policy loop\.

The graph\-substrate control exposes each graph index through typed, read\-only tools: entity/fact/relation search, neighborhood expansion, Personalized PageRank where supported, and chunk reading\. Agents over graph substrates cannot read raw files, so gains over native one\-shot graph rankers come from policy access to the stored index rather than from access to extra corpus text\.

Table 13:Matched controls used to separate retrieval substrate, agentic policy, question wording, and retrieval\-depth effects\.Table 14:Wording and retrieval\-depth controls\. Graph entries are HippoRAG 2 and, where available, MS\-GraphRAG\.

## Appendix HRobustness Checks

The official judge uses correctness\-gated completeness as the combined score\. A simpler binary protocol preserves all broad family rankings at the nine shared native\-pipeline scales\. An independent official judge agrees with the primary judge on 96\.2% of pooled alignment decisions, with per\-cell agreement ranging from 94\.4% to 98\.0%\. Close pairs can move by a few points, but the main separation between BM25, dense retrieval, raw\-file agency at scale, and graph construction families is stable\.

The lexical\-overlap controls use paraphrased questions and top\-10 retrieval budgets at the two smallest tiers where enough methods are available\. BM25 remains above dense and graph retrieval under comparable retrieval depth\. A proposal\-sensitivity audit also performs direct full\-corpus DenseRAG top\-10 retrieval for 90 questions, confirming that trap proposals are not exclusively reachable through the BM25\-prefiltered dense pool used during benchmark construction\. Exact matched\-cell values are reported in Table[14](https://arxiv.org/html/2607.26497#A7.T14)\.

## Appendix IFailure and Stopping Criteria

For LLM\-built graph methods, a tier is counted as completed only when the construction artifact is usable for all scheduled questions and its token ledger is available\. If a build exceeds the resource envelope, fails during entity/relation extraction or merge, or produces an incomplete index that cannot answer the scheduled cell, the tier is reported as unavailable rather than assigned an inferred accuracy in the main ladder\. The teaser is the only place where unavailable graph tiers are converted to zero, and its caption labels the result as coverage\-adjusted\.

File\-System Agent failures are handled at question level\. Failed or retried attempts are included in the full\-scale call and token totals\. Budget exhaustion is recorded separately from answer correctness\. The main text reports that accuracy also falls among within\-budget questions, so the full\-scale loss is not only a truncation artifact\.

## Appendix JData Schemas

Native prediction files use one JSON object per question with fields:

Native prediction recordid, question, predicted\_answer, retrieved\_chunk\_ids, retrieved\_doc\_ids, latency\_sec, metadata

Official judgment files bind a judgment to the answer and retrieved evidence using a SHA256 hash and record:

Official judgment recordid, aligned, reason, completeness\_pct, n\_facts, fact\_results, doc\_recall, prediction\_sha256

Token ledgers are phase separated:

Phase\-separated token ledgerbaseline, dataset, source, build\{prompt,completion,total,calls\}, qa\{prompt,completion,total,calls\}, build\_qa\_total\_tokens

## Appendix KBootstrap Intervals

Confidence intervals in the native ladder resample questions with replacement within each method–tier cell\. Each interval uses 10,000 bootstrap resamples and reports the 2\.5th and 97\.5th percentiles\. Matched controls use paired resampling over the same question IDs so that the confidence interval is over the paired score difference\.

## Appendix LArtifact\-to\-Claim Map

The major empirical claims map to the following code/data artifacts\.

- •Native\-ladder accuracy and confidence bands:results/judge\_official\_canonical\_20260724/\.
- •Build scaling:results/\*scaling\_ledger\.csvandresults/build\_cost\_authoritative\.csv\.
- •Query tokens:results/token\_usage/\.
- •Agent\+BM25:results/judge\_agent\_bm25\_control/andresults/agent\_bm25\_control\_report\.md\.
- •Graph\-substrate agents:results/judge\_official\_table3\_20260724/\.
- •Lexical controls:results/judge\_paraphrase\_\*andresults/judge\_topk10\_\*\.
- •Figure 1 summary:figures/teaser\_crossscale\_points\.csv\.

## Appendix MArtifact Contents

The code/data supplement is organized as follows\.

- •scripts/: sanitized scripts for constructing tiers, running native pipelines, metering tokens, judging outputs, and producing paper figures\.
- •results/: aggregate CSV/JSON ledgers used for the paper tables, figures, and mechanism controls\.
- •figures/: figure\-generation inputs and final PDFs;README\.md: environment variables and reproduction order\.