ScalableRAG: High-Quality RAG at Zero Ingestion Cost

# ScalableRAG: High-Quality RAG at Zero Ingestion Cost
Source: [https://arxiv.org/html/2607.25135](https://arxiv.org/html/2607.25135)
Hilaf Hasson1, Aditya Chakravarty1, Jayant Thomas1, Krishna Gogineni1 1Cohesity 1\{hilaf\.hasson,aditya\.chakravarty,jayant\.thomas,krishna\.gogineni\}@cohesity\.com

###### Abstract

Recent advances in RAG aim to optimize for performance by paying high ingestion costs for knowledge ingestion: building knowledge graphs or extracting SQL tables\. In this work we show that the operations that such knowledge bases allow can be replicated with zero ingestion costs \(not even a vector database\); in fact our solution, Zero\-Ingestion ScalableRAG, handily out\-performs all baselines \(including knowledge graph approaches\) in three out of the six corpora considered here, and only marginally missing maximum performance on the other three, with average accuracy across all six datasets 7\.36% above the next most competitive baseline\. It achieves this by keeping a workspace of document sets and values sets that it can write into and read from, allowing for on\-the\-fly aggregative reasoning in all situations where grouping is required on a primary key that is in one to one correspondence with a subset of the total document set\.

Capping the number of LLM calls by a constant independent of the corpus size, we also introduce Limited\-Ingestion ScalableRAG, which does use a minimal vector database as well as an automated pattern discovery from a sample of documents, to further improve accuracy at scale\. Our code is available at[https://github\.com/cohesity/ScalableRAG](https://github.com/cohesity/ScalableRAG)\.

ScalableRAG: High\-Quality RAG at Zero Ingestion Cost

## 1Introduction

Retrieval\-augmented generation\(Karpukhinet al\.,[2020](https://arxiv.org/html/2607.25135#bib.bib2); Lewiset al\.,[2020](https://arxiv.org/html/2607.25135#bib.bib1); Douzeet al\.,[2025](https://arxiv.org/html/2607.25135#bib.bib5)\)was first introduced as creating a vector database from chunks of a corpus of documents, and then using nearest neighbors between the embedding of the question and the embeddings of the corpus chunks\. Since then there have been two main advances in RAG: The first is to create a knowledge data structure by having an LLM read the entire corpus and indexing in either a knowledge graph\(Edgeet al\.,[2024](https://arxiv.org/html/2607.25135#bib.bib18); Baiet al\.,[2025](https://arxiv.org/html/2607.25135#bib.bib10); Gutiérrezet al\.,[2024](https://arxiv.org/html/2607.25135#bib.bib14),[2025](https://arxiv.org/html/2607.25135#bib.bib15)\)or a schema\(Koshoreket al\.,[2025](https://arxiv.org/html/2607.25135#bib.bib8)\)\. Such methods have the benefit that they make it possible for the retrieval agent to group by any key, and create sets on which some aggregative operation \(count, average, etc\.\) produces the correct answer\. A second nascent approach is to keep ingestion to be vector embeddings only, and to add onto it an agentic retrieval\(Duet al\.,[2026](https://arxiv.org/html/2607.25135#bib.bib16); Huiet al\.,[2025](https://arxiv.org/html/2607.25135#bib.bib17)\)\.

In this paper we aim to replicate the success of creating a knowledge data structure, but with minimal, even zero, ingestion costs\. The main insight we use is that frequently the key on which we want the system to group by is in one to one correspondence with a subset of the set of documents in the corpus, and thus can efficiently be generated at inference\.

Zero\-Ingestion ScalableRAGrequires no ingestion costs whatsoever, including no vector database creation\. Using regex tools, it can create and persist subsets of filenames, sets numbers, sets of dates, and sets of lists\. Each such set gets named, and can be used in future steps using set operations, filtering, and aggregative tools\. Each time a set is created, the previous sets are used in order to give the agent context about the differences between the sets, so that it can make better decisions on whether the filtering worked as anticipated\. Already this base case, with zero ingestion, outperforms most ingestion\-heavy methods in most of our experiments\.

In order to further improve performance while keep ingestion costs minimal, we build on this solution to introduceLimited\-Ingestion ScalableRAGby adding two more capabilities: Vector embedding \(with large chunks\), allowing for cosine similarity, as well as inference\-time classifier training and inference on chunks; and a thorough pattern discovery and validation preprocessing step that uses only a sample of the documents\. The latter decomposes as: 1\. Deterministic discovery for regex patterns, 2\. LLM\-generated regex discovery, and 3\. Discovery of extraction hints for values that cannot easily be extracted through regex\. The three patterns get thoroughly vetted at ingestion, and then exposed strategically to the retrieval agent\.

To summarize, our contributions are:

1. 1\.IntroducingZero\-Ingestion ScalableRAG\(Section[3\.1](https://arxiv.org/html/2607.25135#S3.SS1), a RAG system that does not even require vector embeddings, and uses inference\-time set persistence to achieve state of the art performance compared to even heavy\-ingestion RAG systems\. This solution handily out\-performs all baselines \(including knowledge graph approaches\) in three out of the six corpora considered here, and only marginally missing maximum performance on the other three, with average accuracy across all six datasets about 7% above the next most competitive baseline\. \(See Section[4](https://arxiv.org/html/2607.25135#S4)\.\)
2. 2\.IntroducingLimited\-Ingestion ScalableRAG\(Section[3\.2](https://arxiv.org/html/2607.25135#S3.SS2)\), which builds on Zero\-Ingestion ScalableRAG but adds vector embeddings, and a thorough pattern and extraction discovery and validation pre\-processing so as to improve performance with a*constant*number of LLM calls\. This leads to moderate accuracy improvements in those datasets where extraction is more challenging\.

## 2Related Work

##### Baseline RAG\.

RAG based on dense retrieval\(Karpukhinet al\.,[2020](https://arxiv.org/html/2607.25135#bib.bib2); Lewiset al\.,[2020](https://arxiv.org/html/2607.25135#bib.bib1); Douzeet al\.,[2025](https://arxiv.org/html/2607.25135#bib.bib5)\)remains the standard reference for dense passage retrieval\. There have been some improvements that retain dense retrieval as the core primitive\. To name a few: HyDE\(Gaoet al\.,[2023](https://arxiv.org/html/2607.25135#bib.bib3)\)augments documents with synthetic queries at ingestion to improve performance; HyQE\(Zhouet al\.,[2024](https://arxiv.org/html/2607.25135#bib.bib4)\)augments questions with synthetic documents at inference to improve performance; FLARE\(Jianget al\.,[2023](https://arxiv.org/html/2607.25135#bib.bib6)\)retrieves at generation when the LLM is “uncertain”; Iter\-RetGen\(Shaoet al\.,[2023](https://arxiv.org/html/2607.25135#bib.bib7)\)iterates retrieval and generation; and IRCoT\(Trivediet al\.,[2023](https://arxiv.org/html/2607.25135#bib.bib11)\)interleaves retrieval with chain\-of\-thought\.

##### Ingestion\-Heavy Knowledge Representation

GraphRAG\(Edgeet al\.,[2024](https://arxiv.org/html/2607.25135#bib.bib18)\)popularized the idea of indexing a corpus as an entity\-relation graph and retrieving community\-level summaries\. HippoRAG\(Gutiérrezet al\.,[2024](https://arxiv.org/html/2607.25135#bib.bib14)\)and HippoRAG2\(Gutiérrezet al\.,[2025](https://arxiv.org/html/2607.25135#bib.bib15)\)extend this with personalized PageRank over an openIE\-derived graph, motivated by hippocampal indexing theory\. AutoSchemaKG\(Baiet al\.,[2025](https://arxiv.org/html/2607.25135#bib.bib10)\)pushes the scale further by automatically inducing schemas \(conceptualizing entities into abstract types\) and building a triple index across 50M\+ documents\. For a good survery on knowledge graph solutions for RAG seePenget al\.\([2025](https://arxiv.org/html/2607.25135#bib.bib12)\)\.

SRAG\(Koshoreket al\.,[2025](https://arxiv.org/html/2607.25135#bib.bib8)\)takes a different approach by ingesting the knowledge into forming a SQL table with one row per document, populated via LLM extraction\. We remark that while SRAG follows a completely different algorithm to ours, by allowing only one row per document \(and having a single table rather than multiple tables joined by foreign keys\) it also makes the bet we are making: that most questions group by a primary key that is in one\-to\-one correspondence with a subset of the documents\.

All of these methods require heavy ingestion: typically at least one LLM call per document, often much more\.

##### Agentic RAG\.

A\-RAG\(Duet al\.,[2026](https://arxiv.org/html/2607.25135#bib.bib16)\)is an agent equipped with three retrieval tools:keyword\_searchfor exact lexical matching that returns snippet\-level evidence,semantic\_searchfor dense sentence\-level retrieval \(using a vector database\) grouped by chunks, andchunk\_readfor reading the full text of selected chunks\. Interact\-RAG\(Huiet al\.,[2025](https://arxiv.org/html/2607.25135#bib.bib17)\), in contrast, exposes the retrieval process itself as an interactive environment, enabling an agent to explicitly control retrieval strategies \(semantic vs\. exact search and their fusion\), enforce entity\-anchored matching, and dynamically shape context through inclusion, exclusion, and scale adjustment actions\. We remark that neither solution creates persistent artifacts for future steps\.

## 3Proposed Method

We describe ScalableRAG in two stages\. In Section[3\.1](https://arxiv.org/html/2607.25135#S3.SS1)we present the*zero\-ingestion*system: a stateful agent whose sole input is a directory of plain\-text files\. In Section[3\.2](https://arxiv.org/html/2607.25135#S3.SS2)we layer on the two optional ingestion modules that together constitute*limited\-ingestion*ScalableRAG: vector embeddings, automatic pattern discovery\.

### 3\.1Zero\-Ingestion ScalableRAG

The zero\-ingestion system requires no preprocessing: not only no knowledge graph or schema extraction, but also no vector embeddings\. The agent is given only the corpus and a question\.

#### 3\.1\.1Agent Architecture

Let𝒞=\{d1,…,dN\}\\mathcal\{C\}=\\\{d\_\{1\},\\dots,d\_\{N\}\\\}be the corpus ofNNdocuments\.

ScalableRAG is at its base a ReAct\-style agent\(Yaoet al\.,[2022](https://arxiv.org/html/2607.25135#bib.bib9)\)that interacts with the corpus through tool calls\. Given a questionqq, the agent produces a sequence of\(at,ot\)\(a\_\{t\},o\_\{t\}\)pairs, whereata\_\{t\}is a JSON\-formatted tool invocation andoto\_\{t\}is the tool observation, until it emits a final answer\.

However, unlike a ReAct\-style agent, at turntt, the agent maintains a*set registry*𝒮\(t\)=\{\(si,Di\)\}i\\mathcal\{S\}^\{\(t\)\}=\\\{\(s\_\{i\},D\_\{i\}\)\\\}\_\{i\}, where for eachiiDiD\_\{i\}is a subset of𝒞\\mathcal\{C\}, and its name issis\_\{i\}\. The registry is initialized to𝒮\(0\)=\{\(all,𝒞\)\}\\mathcal\{S\}^\{\(0\)\}=\\\{\(\\texttt\{all\},\\mathcal\{C\}\)\\\}and is updated in\-place after every tool invocation\. In addition, the agent maintains a set of*value sets*𝒱\(t\)=\{\(vj,fj\)\}j\\mathcal\{V\}^\{\(t\)\}=\\\{\(v\_\{j\},f\_\{j\}\)\\\}\_\{j\}, where thevjv\_\{j\}is the name of the functionfj:D→ℝf\_\{j\}:D\\to\\mathbb\{R\}\(scalar\),fj:D→ℝkf\_\{j\}:D\\to\\mathbb\{R\}^\{k\}\(date\), orfj:D→2Σ∗f\_\{j\}:D\\to 2^\{\\Sigma^\{\*\}\}\(list of strings\), whereDDis one of the sets inS\(t\)S^\{\(t\)\}\.

At each step the agent selects a toolτt\\tau\_\{t\}and argumentsαt\\alpha\_\{t\}; the tool reads from and writes to the shared state:

\(𝒮\(t\),𝒱\(t\),ot\)=τt​\(αt;𝒮\(t−1\),𝒱\(t−1\),𝒞\)\(\\mathcal\{S\}^\{\(t\)\},\\mathcal\{V\}^\{\(t\)\},o\_\{t\}\)=\\tau\_\{t\}\(\\alpha\_\{t\};\\mathcal\{S\}^\{\(t\-1\)\},\\mathcal\{V\}^\{\(t\-1\)\},\\mathcal\{C\}\)\(1\)
This is inherently different from all prior agentic RAG solutions, and allows the agent to gain significant helpful knowledge at each turn by automatizing computations that refer to existing sets; see Section[3\.1\.2](https://arxiv.org/html/2607.25135#S3.SS1.SSS2)for details\.

The full system prompt is given verbatim in Appendix[A](https://arxiv.org/html/2607.25135#A1); and context management details \(per\-tool output budgets, observation redaction after consumption\) are described in Appendix[B](https://arxiv.org/html/2607.25135#A2)\.

##### Lexical pre\-analysis \(used only for guidance\)\.

Before the first LLM turn, ScalableRAG performs a zero\-cost scan to give the agent a coarse map of the search space\. Concretely, it extracts up to five distinct*question keywords*\(maximal alphanumeric spans of length≥4\\geq 4\) and, for each keyword, counts how many documents contain it by case\-insensitive substring inclusion\. It also reports filename hit counts and up to three pairwise intersections among the keywords that matches at least one but fewer than all of the documents\. These statistics are not retrieval: they are a cheap prior that tells the agent whether it should start from filenames, from a single broad text filter, or from a narrower combination\. The question keywords above get sorted based on a lightweight logic \(see Appendix[B](https://arxiv.org/html/2607.25135#A2)\), and are henceforth called “anchor tokens”\.

Algorithm[1](https://arxiv.org/html/2607.25135#alg1)formalizes the agent loop\.

Algorithm 1ScalableRAG Agent Loop0:Corpus

𝒞\\mathcal\{C\}, question

qq, LLM

ℳ\\mathcal\{M\}, max steps

TT, context budget

BB
1:

𝒮\(0\)←\{\(all,𝒞\)\}\\mathcal\{S\}^\{\(0\)\}\\leftarrow\\\{\(\\texttt\{all\},\\mathcal\{C\}\)\\\};

𝒱\(0\)←∅\\mathcal\{V\}^\{\(0\)\}\\leftarrow\\emptyset
2:

ℓ←QuestionLandscape​\(q,𝒞\)\\ell\\leftarrow\\textsc\{QuestionLandscape\}\(q,\\mathcal\{C\}\)\{keyword statistics\}

3:

msgs←\[SysPrompt,q⊕ℓ\]\\text\{msgs\}\\leftarrow\[\\text\{SysPrompt\},\\;q\\oplus\\ell\]
4:for

t=1,…,Tt=1,\\ldots,Tdo

5:

msgs←RedactAndCompress​\(msgs,B\)\\text\{msgs\}\\leftarrow\\textsc\{RedactAndCompress\}\(\\text\{msgs\},B\)\{context management\}

6:

response←ℳ​\(msgs\)\\text\{response\}\\leftarrow\\mathcal\{M\}\(\\text\{msgs\}\)
7:ifresponse containsanswerthen

8:returnresponse\.answer

9:endif

10:Parse

\(τt,αt\)←\(\\tau\_\{t\},\\alpha\_\{t\}\)\\leftarrowresponse\.tool, response\.args

11:

\(𝒮\(t\),𝒱\(t\),ot\)←τt​\(αt;𝒮\(t−1\),𝒱\(t−1\),𝒞\)\(\\mathcal\{S\}^\{\(t\)\},\\mathcal\{V\}^\{\(t\)\},o\_\{t\}\)\\leftarrow\\tau\_\{t\}\(\\alpha\_\{t\};\\mathcal\{S\}^\{\(t\-1\)\},\\mathcal\{V\}^\{\(t\-1\)\},\\mathcal\{C\}\)
12:Append assistant response and

oto\_\{t\}to msgs

13:endfor

14:

15:return

ℳ​\(msgs⊕“give your best answer”\)\\mathcal\{M\}\(\\text\{msgs\}\\oplus\\text\{\`\`give your best answer''\}\)

##### Producing document sets\.

Every filtering operation takes an existing named set as its*target*\. This means filtering is always*relative*: the agent refines incrementally, and all observations are computed with respect to the target set, not the full corpus\.

- •apply\_filter\(ρ,S\)→\(S\+,S−\)\(\\rho,S\)\\to\(S^\{\+\},S^\{\-\}\): applies regexρ\\rhoto every document in the named target setSS, partitioning it into two new named sets: a positive setS\+⊆SS^\{\+\}\\subseteq S\(documents matchingρ\\rho\) and a negative setS−=S∖S\+S^\{\-\}=S\\setminus S^\{\+\}\. Both are registered in the workspace\. Because the partition is relative toSS, not to the full corpus, the agent sees snippets from both sides of*its current working set*: positive snippets show why a document matched, negative snippets are centered on a question\-derived anchor token \(see Appendix[B](https://arxiv.org/html/2607.25135#A2)\) so the agent can inspect format variants that the regex missed within the same scope\.
- •search\_filenames\(p,S\)→D\(p,S\)\\to D: filters by filename pattern, optionally restricted to a target set\.
- •set\_operation\(A,B,op\)→D\(A,B,\\text\{op\}\)\\to D: explicit∩\\cap,∪\\cup,∖\\setminus\. The registry also resolves*lazy compound names*: the agent may writeA\_and\_Bin any tool argument and the system computesA∩BA\\cap Bon the fly without a separate call\.
- •filter\_values\(v,op,θ\)→D\(v,\\operatorname\{op\},\\theta\)\\to D: promotes a value set back into a document set by thresholding extracted values\. Operationally, the workspace stores a value setvjv\_\{j\}as a partial mapping from document IDs to typed values \(numbers or dates\) for the subset of documents where extraction succeeded\. The tool returnsD=\{d∈dom​\(vj\):vj​\(d\)​op⁡θ\}D=\\\{d\\in\\mathrm\{dom\}\(v\_\{j\}\):v\_\{j\}\(d\)\\ \\operatorname\{op\}\\ \\theta\\\}\(e\.g\., date≥\\geq2020\-01\-01, revenue\>\>1B\) and registers it as a new named set; documents with missing or non\-numeric values are simply absent fromdom​\(vj\)\\mathrm\{dom\}\(v\_\{j\}\)\.

##### Producing value sets\.

Value sets lift a document set into structured per\-document data\. These tools perform*regex extraction*: the regex is used to capture a value \(via a capture group\) and populate a per\-document mapping, not to decide whether the document belongs in the set\.

- •extract\_field\(ρ,S\)→v\(\\rho,S\)\\to v: runs a regex with a capture group over every document inSS; stores the first match per doc as a scalar \(number, text, or date\)\.
- •extract\_list\(ρ,S\)→v\(\\rho,S\)\\to v: stores*all*matches per doc as a list, for repeating items \(participants, line items, references\)\.
- •extract\_from\_filename\(ρ,S\)→v\(\\rho,S\)\\to v: captures metadata from filenames \(e\.g\., ticker, quarter, category\) via a capture group\.

##### Reducing to answers\.

aggregate\(v,op\)\(v,\\text\{op\}\)reduces a value set:sum\\mathrm\{sum\},avg\\mathrm\{avg\},min\\mathrm\{min\},max\\mathrm\{max\},count\\mathrm\{count\}for scalars;unique​\_​values\\mathrm\{unique\\\_values\},count​\_​unique\\mathrm\{count\\\_unique\},value​\_​counts\\mathrm\{value\\\_counts\},group​\_​by​\_​value\\mathrm\{group\\\_by\\\_value\},per​\_​doc​\_​count\\mathrm\{per\\\_doc\\\_count\}for lists\. Notably,min\\mathrm\{min\}/max\\mathrm\{max\}can additionally produce a new document set containing the extremal documents, letting the agent read them\.count\_set\(S\)\(S\)returns the exact cardinality\|S\|\|S\|\.calculate\(expr\)\(\\text\{expr\}\)evaluates arithmetic safely\.

##### Observation and planning\.

read\_docsreads up tokkdocuments from a target set, optionally sampling from the end viaoffset=−k=\-kto expose format variants\. For long documents, it supports*contiguous paging*\(chunk\_idx\) and precise slices \(start\_char,max\_chars\) so the agent can zoom into specific regions without dumping entire filings/transcripts\.find\_in\_docis a deterministic in\-document navigator that returns character offsets for exact match \(substring or regex\) and a query\-aware ranked\-window mode; used withread\_docs\(start\_char=\.\.\., max\_chars=\.\.\.\), it enables reliable “jump\-to\-section” behavior in huge files\.table\_lookuplocates a row in aligned/table\-like text and returns the row plus nearby header lines \(often containing years/columns\) and parsed numbers\.explore\_patternsevaluates a small batch of candidate regexes against a target set and returns match counts and samples without creating new sets, enabling cheap comparison before committing\.create\_regexregisters a regex for reuse \(or asks the LLM to propose one given a small sample set\)\.list\_setsexposes the current workspace state \(set names, sizes, and provenance\), which the agent uses as a planning view\.

##### Branch\-and\-Select for deterministic tools\.

Many tool calls are deterministic given their JSON arguments \(e\.g\., lexical search, in\-document navigation, and table lookup\)\. ScalableRAG can*speculatively preview*a small set of alternative tool invocations, generated automatically from the agent’s proposed action, and then ask the LLM to select which branch to execute\. Only the selected action is committed to the workspace; previews are discarded\.

#### 3\.1\.2Diagnostic Feedback: Using Set Persistence to Compute Intermediate Validation Hints

A key reason ScalableRAG outperforms stateless retrieval agents is that every tool observation is enriched with*automatically computed diagnostics*that guide the agent’s next action\. We enumerate these for the primary tools\.

##### apply\_filterdiagnostics\.

When the agent partitions a target setSSby regexρ\\rho, the observation contains far more than just the resulting set names and sizes\. All statistics below are computed*relative toSS*, so when the agent filters ammdocument subset, it sees token hit counts out ofmm, not out of the full corpus\. This relative scoping is what makes iterative refinement informative:

- •*Token decomposition*: for each token \(≥\\geq4 chars\) extracted from the*regex string*\(via`\[ˆ\\W\_\]\{4,\}`\), the number of documents in the target set whose normalized text matches that token as a standalone, case\-insensitive regex probe\. This reveals which token is the bottleneck \(too strict\) or which is vacuous \(matches everything\)\.
- •*Keyword intersection*: the number of documents containing*both*the top two keywords extracted fromρ\\rhosimultaneously, revealing how much the combined pattern over\-constrains\.
- •*Selectivity gap*: when the keyword intersection is larger than the regex match count, the system flags: “kkdocs likely use a different format\. Read negatives\.” This prompts the agent to investigate format variants it is missing\.
- •*Anchor\-token coverage*\(entity\_coveragein logs\): for the question’s primary anchor token \(see Appendix[B](https://arxiv.org/html/2607.25135#A2)\), how many documents containing that token fall inside vs\. outside the current set\.
- •*Negative anchor\-token check*: how many negative \(excluded\) documents still contain the top question\-derived anchor token, alerting the agent to potential false negatives\.
- •*Scope check*: when all docs in a narrow subset match, the system checks how many docs*outside*that subset also match the same regex, preventing premature narrowing\.
- •*Positive and negative snippets*: text excerpts from both sides of the partition, centered on the regex match \(positive\) or the top anchor token \(negative\), so the agent can see*why*a document was included or excluded\.

##### count\_setdiagnostics\.

Beyond the count itself:

- •*Provenance warning*: if the set was never refined by a condition\-specific regex, the system warns “unrefined set—count may include false positives\.”
- •*Anchor\-token coverage*: how many docs containing the top anchor token are outside the counted set\.
- •*Filename range*: the lexicographically first and last filenames, showing the scope of the set at a glance\.

##### extract\_field/extract\_listdiagnostics\.

The observation reports: how many documents were scanned, how many yielded a value, and a sample of extracted values, letting the agent immediately see whether its regex captures the intended structure or misses most documents\.

Example: multi\-hop question over MuSiQue\.“What county is Erik Hort’s birthplace a part of?”Step 1: apply\_filter\("Erik", all\)→\\tohas\_Erik\_pos\(30 docs\)\.Step 2: apply\_filter\("Hort", has\_Erik\_pos\)→\\to2 docs, including*Erik\_Hort\.txt*\. Observation snippets reveal birthplace = Montebello, NY\.Step 3: apply\_filter\("Montebello", all\)→\\to3 docs\. Observation includes*Montebello\_New\_York\.txt*: “incorporated village in… Rockland County\.”Answer:Rockland County\. \(3 tool calls, correct\.\)

Figure 1:Set\-algebraic multi\-hop reasoning\. Each hop narrows the set, and the agent uses the intermediate set as both evidence and anchor for the next hop\.

### 3\.2Limited\-Ingestion ScalableRAG

The zero\-ingestion system described above already achieves strong performance, see Section[4](https://arxiv.org/html/2607.25135#S4)\. Two optional modules, each requiring only modest, one\-time preprocessing, extend the workspace with additional tools for further performance boost improvement\.

#### 3\.2\.1Module 1: Vector Embeddings

Pre\-computing document embeddings addscosine\_search\(q,k\)→D\(q,k\)\\to Dto the workspace: dense retrieval producing a named document set from the documents that contain the topkkchunks\.*In our experiments we assume that the budget for vector embedding is minimal, and our chunks are as large as the encoder allows*\.

Additionally, the embeddings enable an optionallabel\_docs→\\totrain\_classifierpipeline: the agent can label a small sample of documents, train a lightweight classifier over their embeddings \(XGBoost by default; logistic regression and random forest are also supported\), and apply it corpus\-wide\. In practice, the agent rarely invokes this pipeline, but it provides a principled fallback for corpora where relevant documents share no lexical signal\.

#### 3\.2\.2Module 2: Automatic Pattern Discovery

##### Overview\.

We distinguish three preprocessing outputs surfaced to the agent:

##### Stage 1: Pattern Discovery\.

*Deterministic regexes\.*A scanner identifies recurring “Label: Value” lines at the start of a line \(label up to 80 characters; value up to 300\)\. We normalize the label key \(whitespace collapsing; case\-folding\) and count document frequency\. A label becomes a pattern if it appears in at least 3 documents with at least 2 distinct values\. Each discovered field yields a regex template anchored to the label and a capture group\(?P<value\>…\)\.

*LLM\-generated regexes\.*Independently, an LLM reads a small, diverse sample \(default 20 documents\) and proposes multi\-line patterns: named sections that span multiple lines and cannot be captured as a single “Label: Value” field\. Each pattern includes a name, description, parameter list, and a parameterized regex template with named capture groups\.

*Semantic patterns*In addition to regex patterns, the preprocessing step discovers*semantic patterns*: prose\-embedded concepts not captured by any regex\. Each semantic pattern includes a short description, an extraction hint, and a small list of search keywords; we estimate its coverage by keyword matching over the corpus\.

Each of the three types of patterns also gets associated keywords, and get exposed to the agent if these keywords are in the user question\. \(For deterministic patterns it is simply the label\.\)

##### Stage 2: Validation and refinement \(LLM patterns only\)\.

Each LLM\-generated regex is executed over the corpus in a sandbox with a 2\-second per\-document timeout \(guarding against catastrophic backtracking\)\. We estimate precision from an LLM\-judged sample of matches, and recall from an LLM\-judged sample of non\-matches plus partial matches \(regex fired but capture groups empty\)\. If either is below threshold \(defaults 0\.90/0\.85\), the LLM is asked to refine the regex using false\-negative evidence, iterating up to 8 rounds and keeping the best\-scoring template\.

A curation step assigns each pattern \(regexes from the deterministic field scanner, LLM\-generated regexes, and the semantic patterns\) a 0–10 utility score of “question\-answering utility” via LLM judgment, keeping only patterns scoring≥5\\geq 5\.

##### Stage 3: Gap clusters\.

All patterns above a minimum validation score \(0\.5 by default\) are indexed by executing their regex over each document \(using a 12,000\-character head\+tail window\)\. We then compute*gap clusters*as follows: for each keyword token, find documents where the token appears in the scannable text but*not*inside any captured group; extract the*line*containing the token \(truncated\), lowercase it, replace the token by\{\}, then cluster identical templates and count them\.

Algorithm 2Automatic pattern discovery and extraction indexing0:Corpus

𝒞\\mathcal\{C\}, LLM

ℳ\\mathcal\{M\}, sample size

mm\(default 20\), max refinement rounds

RR\(default 8\)

1:

Pfield←DeterministicRegexes​\(𝒞\)P\_\{\\mathrm\{field\}\}\\leftarrow\\textsc\{DeterministicRegexes\}\(\\mathcal\{C\}\)\{deterministic label patterns\}

2:

Pllm←ℳ​\(LLMGeneratedRegexes​\(Sample​\(𝒞,m\)\)\)P\_\{\\mathrm\{llm\}\}\\leftarrow\\mathcal\{M\}\(\\textsc\{LLMGeneratedRegexes\}\(\\textsc\{Sample\}\(\\mathcal\{C\},m\)\)\)\{multi\-line regex templates\}

3:foreach

p∈Pllmp\\in P\_\{\\mathrm\{llm\}\}do

4:

Validate​\(p,𝒞\)→\(prec^,rec^\)\\textsc\{Validate\}\(p,\\mathcal\{C\}\)\\to\(\\widehat\{\\mathrm\{prec\}\},\\widehat\{\\mathrm\{rec\}\}\)
5:for

r=1r=1to

RRwhile below thresholdsdo

6:

p←ℳ​\(RefinePattern​\(p,FalseNegatives\)\)p\\leftarrow\\mathcal\{M\}\(\\textsc\{RefinePattern\}\(p,\\textsc\{FalseNegatives\}\)\)
7:

Validate​\(p,𝒞\)\\textsc\{Validate\}\(p,\\mathcal\{C\}\)
8:endfor

9:endfor

10:

P←\{p∈\(Pfield∪Pllm\):LLMScore​\(p\)≥0\.5\}P\\leftarrow\\\{p\\in\(P\_\{\\mathrm\{field\}\}\\cup P\_\{\\mathrm\{llm\}\}\):\\mathrm\{LLMScore\}\(p\)\\geq 0\.5\\\}
11:

I←BuildIndex​\(P,𝒞\)I\\leftarrow\\textsc\{BuildIndex\}\(P,\\mathcal\{C\}\)\{matches, values, value keywords\}

12:

ComputeGapClusters​\(I,𝒞\)\\textsc\{ComputeGapClusters\}\(I,\\mathcal\{C\}\)\{surrounding\-text templates\}

13:

Psem←ℳ​\(SemanticPatterns​\(P,Sample​\(𝒞,m\)\)\)P\_\{\\mathrm\{sem\}\}\\leftarrow\\mathcal\{M\}\(\\textsc\{SemanticPatterns\}\(P,\\textsc\{Sample\}\(\\mathcal\{C\},m\)\)\)\{prose\-embedded concepts \+ extraction hints\}

14:

Psem←EstimateCoverage​\(Psem,𝒞\)P\_\{\\mathrm\{sem\}\}\\leftarrow\\textsc\{EstimateCoverage\}\(P\_\{\\mathrm\{sem\}\},\\mathcal\{C\}\)\{keyword\-based coverage estimates\}

15:return

I⊕PsemI\\oplus P\_\{\\mathrm\{sem\}\}

\#\(1\)Deterministiclabeled\-fieldpattern\(Transcripts;fieldscanner\)

name:field\_present

regex\_template:\(?:^\|\[\\n\]\)\[\\t\]\*Present\\s\*:\\s\*\(?P<value\>\[^\\n\]\{1,300\}\)

\#\(2\)LLM\-generatedstructuralsection\(Hotels;multi\-lineregextemplate\)

name:facilities\_amenities\_section

parameters:\["facilities\_amenities\_block"\]

regex\_template:\(?:^\|\\n\)\#\#\\s\*Facilities\\s\*&\\s\*Amenities\\n\(?P<facilities\_amenities\_block\>\(?:\.\{0,200\}\\n\)\+?\)\(?=\\n\#\#\\s\*Pricing\|\\n\#\|\\Z\)

\#\(2b\)Patternusefulnessinpractice:answerinonetoolcall\(Hotels\)

question:"Howmanyhotelpageshaveaswimmingpoolfacility?"

tool:pattern\_search

args:\{"pattern":"facilities\_amenities\_section","params":\{"facilities\_amenities\_block":"swimming"\},"create\_set":"has\_swimming\_pool"\}

confirmed\_matches:24

capture\_sample:"SwimmingPool\\n\\nFitnessCenter\\n\\nBar\\n\\nSpa\\n\\nSauna\\n\\nConcierge\.\.\."

\#\(3\)Gapclusterreturnedatquerytime:keywordoccursoutsidecapture\(FinanceBench\)

question:"Whichdebtsecuritiesareregisteredtotradeonanationalsecuritiesexchangeunder3M’snameasofQ2of2023?"

tool:pattern\_search

args:\{"pattern":"ExhibitList/Table","params":\{"content":"registered"\},"create\_set":"registered\_exhibits"\}

confirmed\_matches:6

coverage\.keyword\_in\_pattern\_not\_captured:71

coverage\.gap\_clusters\[0\]:\{template:"securities\{\}pursuanttosection12\(b\)oftheact:",count:54\}

coverage\.gap\_clusters\[1\]:\{template:"securities\{\}pursuanttosection12\(b\)ofthesecuritiesexchangeactof1934:",count:2\}

Listing 1:Examples for extracted patterns and gap clusters\.
##### Agent Exposure to Patterns\.

A compact*corpus profile*is prepended to the agent’s input: a bulleted list of each indexed pattern with its coverage percentage and sample values\. The profile also includes a compact summary of frequently occurring labeled fields and semantic patterns \(as extraction hints\)\. Additionally, a*pattern expert section*in the question landscape matches the question’s keywords against the index’s value\-keyword catalog, suggesting specific patterns\. This “pattern routing” happens*before the first LLM call*and costs zero tokens of generation\.

A new tool calledpattern\_searchis exposed for using regex\-based patterns\. LLM\-generated structural regex patterns are listed fully; deterministic labeled\-field regexes, of which there can be hundreds, only show a few based on prevalence, and the rest based on whether the label appeared in the question; LLM\-generated non\-regex semantic patterns are shown only as extraction hints with keyword\-based coverage estimates, but not aspattern\_searchtargets\. We also pre\-register named document setspat\_\{name\}for the curated regex patterns, so the agent can immediately intersect structural sets with other constraints\.

## 4Experiments

### 4\.1Experimental Setup

##### Datasets\.

We evaluate on six corpora: MuSiQueTrivediet al\.\([2022](https://arxiv.org/html/2607.25135#bib.bib19)\), with 11,656 docs and 1,000 questions; 2WikiMultiHopQAHoet al\.\([2020](https://arxiv.org/html/2607.25135#bib.bib20)\), with 6,118 docs and 1,000 questions; Transcripts \(a new dataset we construct from public U\.S\. Government Publishing Office \(GPO\) records of the 117th Congress;111Source documents from[https://www\.govinfo\.gov/app/collection/chrg/117](https://www.govinfo.gov/app/collection/chrg/117)\.see Appendix[D](https://arxiv.org/html/2607.25135#A4)for construction details\), with 75 U\.S\. Congressional hearing transcripts and 100 questions; FinanceBenchIslamet al\.\([2023](https://arxiv.org/html/2607.25135#bib.bib23)\), with 84 financial filing documents and 150 questions; ComplexTRTanet al\.\([2024](https://arxiv.org/html/2607.25135#bib.bib22)\), with 872 documents and 200 temporal multi\-hop questions; and HotelsKoshoreket al\.\([2025](https://arxiv.org/html/2607.25135#bib.bib8)\)\(only the train split\), with 50 hotel page documents and 138 questions\.

### 4\.2Results

Table 1:LLM\-as\-judge accuracy \(top, %\) and SQuAD\-style exact match / token\-overlap F1 \(bottom, %; EM/F1\)\. All runs used GPT4\.1 and run on a n1\-standard\-8 instance on GCP\. NaN values represent runs that either ran out of memory or took more than 24 hours\. For each question, before the LLM\-as\-a\-judge and EM and F1 computation, there is first an LLM call \(GPT4\.1\) to ensure that all reasoning is removed from the answer\. The LLM\-as\-a\-judge then assigns a numeric score based on a list of rules for judging different types of questions; we report the mean over questions\. See Appendix[E](https://arxiv.org/html/2607.25135#A5)for a full account\.Table 2:Steps and tokens per question for ScalableRAG and A\-RAGZero\-Ingestion ScalableRAG already beats all other baselines, including the high cost ingestion baselines \(HippoRAG2, SRAG, GraphRAG, and AutoSchemaKG\) in 3 out of the 6 datasets: MuSiQue, Transcripts, and Hotels\. We remark that Hotels was first introduced in the SRAG paper\(Koshoreket al\.,[2025](https://arxiv.org/html/2607.25135#bib.bib8)\)as a dataset that is challenging for requiring aggregative reasoning, yet we beat SRAG by 36\.23% with no ingestion whatsoever\. On the 3 remaining datasets, 2Wiki, FinanceBench, and ComplexTR; it had lost by an average of only 1\.63% compared to the best performing baseline for each dataset\. Only A\-RAG wins across all 3 of these datasets compared Zero\-Ingestion ScalableRAG\. We remark that the reason that A\-RAG loses significantly in Hotels and Transcripts specifically, despite having a vector database at its disposal, is exactly because it does not write and read persistent sets as ScalableRAG does, and therefore cannot answer aggregative questions\. Note also in Table[2](https://arxiv.org/html/2607.25135#S4.T2)that the number of steps in ScalableRAG is similar to A\-RAG, but that A\-RAG uses significantly more tokens at inference\.

The lift of Limited\-Ingestion ScalableRAG compared to Zero\-Ingesetion ScalableRAG is in exactly those datasets that have more interesting structure: Hotels, and Transcripts\. We remark that in Limited\-Ingestion ScalableRAG we used most coarse chunking possible following the philosophy of limited ingestion \(using`text\-embedding\-3\-small`\); perhaps finer granularity such as in A\-RAG would improve results\.

Vanilla RAG is using`all\-MiniLM\-L6\-v2`with 900 character windows and 50% overlap\.

## 5Conclusion

We presented*ScalableRAG*, a stateful agentic RAG framework that replicates the aggregative reasoning of ingestion\-heavy knowledge\-base methods at a fraction of their preprocessing cost\. Its two variants,*Zero\-Ingestion*\(no preprocessing\) and*Limited\-Ingestion*\(vector embeddings plus a one\-time, sample\-based pattern discovery whose cost is constant in the corpus size\), share the same set\-algebraic workspace: every tool reads from and writes back to a typed registry of named document sets and value sets, and every observation is enriched with diagnostics that reference the workspace state\. On six diverse corpora both ScalableRAG variants beat all ingestion\-heavy and agentic baselines on average, with Limited\-Ingestion ScalableRAG providing an additional lift on corpora with rich structure amenable to pattern discovery\.

## 6Limitations

A core design choice of our algorithm is that every set in the workspace is in one\-to\-one correspondence with a subset of the set of documents\. This allows for easy aggregations so long as the primary key one aggregates by is itself in one\-to\-one correspondence with a subset of the set of documents\. We found in practice that questions that violate this hypothesis are rare in datasets for question\-answering on a corpus of documents; but we expect knowledge graph approaches to perform better on such questions\.

## 7Ethical Considerations

All corpora used in our experiments are publicly available under permissive licenses, with the exception of the Transcripts dataset\. The Transcripts dataset is constructed from U\.S\. Government Publishing Office releases \(govinfo\.gov\), which place the underlying hearing transcripts in the public domain\. Our experiments do not involve human subjects, and we are not publishing ScalableRAG as a pre\-trained model, and it therefore makes no use of personally identifying information\. As with all LLM\-based systems, the outputs depend on the underlying language model; downstream users should treat answers as derived evidence rather than authoritative claims\. Any artifacts generated by AI \(Cursor/ChatGPT/Claude\) as part of this paper’s code generation, ideation, or paper editing had been under manual supervision and verification by the authors\.

## References

- J\. Bai, W\. Fan, Q\. Hu, Q\. Zong, C\. Li, H\. T\. Tsang, H\. Luo, Y\. Yim, H\. Huang, X\. Zhou,et al\.\(2025\)Autoschemakg: autonomous knowledge graph construction through dynamic schema induction from web\-scale corpora\.arXiv preprint arXiv:2505\.23628\.Cited by:[§1](https://arxiv.org/html/2607.25135#S1.p1.1),[§2](https://arxiv.org/html/2607.25135#S2.SS0.SSS0.Px2.p1.1)\.
- M\. Douze, A\. Guzhva, C\. Deng, J\. Johnson, G\. Szilvasy, P\. Mazaré, M\. Lomeli, L\. Hosseini, and H\. Jégou \(2025\)The faiss library\.IEEE Transactions on Big Data\.Cited by:[§1](https://arxiv.org/html/2607.25135#S1.p1.1),[§2](https://arxiv.org/html/2607.25135#S2.SS0.SSS0.Px1.p1.1)\.
- A\-rag: scaling agentic retrieval\-augmented generation via hierarchical retrieval interfaces\.arXiv preprint arXiv:2602\.03442\.Cited by:[§1](https://arxiv.org/html/2607.25135#S1.p1.1),[§2](https://arxiv.org/html/2607.25135#S2.SS0.SSS0.Px3.p1.1)\.
- D\. Edge, H\. Trinh, N\. Cheng, J\. Bradley, A\. Chao, A\. Mody, S\. Truitt, D\. Metropolitansky, R\. O\. Ness, and J\. Larson \(2024\)From local to global: a graph rag approach to query\-focused summarization\.arXiv preprint arXiv:2404\.16130\.Cited by:[§1](https://arxiv.org/html/2607.25135#S1.p1.1),[§2](https://arxiv.org/html/2607.25135#S2.SS0.SSS0.Px2.p1.1)\.
- L\. Gao, X\. Ma, J\. Lin, and J\. Callan \(2023\)Precise zero\-shot dense retrieval without relevance labels\.InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 1762–1777\.Cited by:[§2](https://arxiv.org/html/2607.25135#S2.SS0.SSS0.Px1.p1.1)\.
- B\. J\. Gutiérrez, Y\. Shu, Y\. Gu, M\. Yasunaga, and Y\. Su \(2024\)Hipporag: neurobiologically inspired long\-term memory for large language models\.Advances in neural information processing systems37,pp\. 59532–59569\.Cited by:[§1](https://arxiv.org/html/2607.25135#S1.p1.1),[§2](https://arxiv.org/html/2607.25135#S2.SS0.SSS0.Px2.p1.1)\.
- B\. J\. Gutiérrez, Y\. Shu, W\. Qi, S\. Zhou, and Y\. Su \(2025\)From rag to memory: non\-parametric continual learning for large language models\.arXiv preprint arXiv:2502\.14802\.Cited by:[§1](https://arxiv.org/html/2607.25135#S1.p1.1),[§2](https://arxiv.org/html/2607.25135#S2.SS0.SSS0.Px2.p1.1)\.
- X\. Ho, A\. D\. Nguyen, S\. Sugawara, and A\. Aizawa \(2020\)Constructing a multi\-hop qa dataset for comprehensive evaluation of reasoning steps\.InProceedings of the 28th International Conference on Computational Linguistics,pp\. 6609–6625\.Cited by:[§4\.1](https://arxiv.org/html/2607.25135#S4.SS1.SSS0.Px1.p1.1)\.
- Y\. Hui, C\. Chen, Z\. Fu, Y\. Liu, J\. Ye, and H\. Zhang \(2025\)Interact\-rag: reason and interact with the corpus, beyond black\-box retrieval\.arXiv preprint arXiv:2510\.27566\.Cited by:[§1](https://arxiv.org/html/2607.25135#S1.p1.1),[§2](https://arxiv.org/html/2607.25135#S2.SS0.SSS0.Px3.p1.1)\.
- P\. Islam, A\. Kannappan, D\. Kiela, R\. Qian, N\. Scherrer, and B\. Vidgen \(2023\)Financebench: a new benchmark for financial question answering\.arXiv preprint arXiv:2311\.11944\.Cited by:[§4\.1](https://arxiv.org/html/2607.25135#S4.SS1.SSS0.Px1.p1.1)\.
- Z\. Jiang, F\. F\. Xu, L\. Gao, Z\. Sun, Q\. Liu, J\. Dwivedi\-Yu, Y\. Yang, J\. Callan, and G\. Neubig \(2023\)Active retrieval augmented generation\.InProceedings of the 2023 conference on empirical methods in natural language processing,pp\. 7969–7992\.Cited by:[§2](https://arxiv.org/html/2607.25135#S2.SS0.SSS0.Px1.p1.1)\.
- V\. Karpukhin, B\. Oguz, S\. Min, P\. Lewis, L\. Wu, S\. Edunov, D\. Chen, and W\. Yih \(2020\)Dense passage retrieval for open\-domain question answering\.InProceedings of the 2020 conference on empirical methods in natural language processing \(EMNLP\),pp\. 6769–6781\.Cited by:[§1](https://arxiv.org/html/2607.25135#S1.p1.1),[§2](https://arxiv.org/html/2607.25135#S2.SS0.SSS0.Px1.p1.1)\.
- O\. Koshorek, N\. Granot, A\. Alloni, S\. Admati, R\. Hendel, I\. Weiss, A\. Arazi, S\. Cohen, and Y\. Belinkov \(2025\)Structured rag for answering aggregative questions\.arXiv preprint arXiv:2511\.08505\.Cited by:[§1](https://arxiv.org/html/2607.25135#S1.p1.1),[§2](https://arxiv.org/html/2607.25135#S2.SS0.SSS0.Px2.p2.1),[§4\.1](https://arxiv.org/html/2607.25135#S4.SS1.SSS0.Px1.p1.1),[§4\.2](https://arxiv.org/html/2607.25135#S4.SS2.p1.1)\.
- P\. Lewis, E\. Perez, A\. Piktus, F\. Petroni, V\. Karpukhin, N\. Goyal, H\. Küttler, M\. Lewis, W\. Yih, T\. Rocktäschel,et al\.\(2020\)Retrieval\-augmented generation for knowledge\-intensive nlp tasks\.Advances in neural information processing systems33,pp\. 9459–9474\.Cited by:[§1](https://arxiv.org/html/2607.25135#S1.p1.1),[§2](https://arxiv.org/html/2607.25135#S2.SS0.SSS0.Px1.p1.1)\.
- B\. Peng, Y\. Zhu, Y\. Liu, X\. Bo, H\. Shi, C\. Hong, Y\. Zhang, and S\. Tang \(2025\)Graph retrieval\-augmented generation: a survey\.ACM Transactions on Information Systems44\(2\),pp\. 1–52\.Cited by:[§2](https://arxiv.org/html/2607.25135#S2.SS0.SSS0.Px2.p1.1)\.
- Z\. Shao, Y\. Gong, Y\. Shen, M\. Huang, N\. Duan, and W\. Chen \(2023\)Enhancing retrieval\-augmented large language models with iterative retrieval\-generation synergy\.InFindings of the Association for Computational Linguistics: EMNLP 2023,pp\. 9248–9274\.Cited by:[§2](https://arxiv.org/html/2607.25135#S2.SS0.SSS0.Px1.p1.1)\.
- Q\. Tan, H\. T\. Ng, and L\. Bing \(2024\)Towards robust temporal reasoning of large language models via a multi\-hop qa dataset and pseudo\-instruction tuning\.InFindings of the Association for Computational Linguistics: ACL 2024,pp\. 6272–6286\.Cited by:[§4\.1](https://arxiv.org/html/2607.25135#S4.SS1.SSS0.Px1.p1.1)\.
- H\. Trivedi, N\. Balasubramanian, T\. Khot, and A\. Sabharwal \(2022\)MuSiQue: multihop questions via single\-hop question composition\.Transactions of the Association for Computational Linguistics10,pp\. 539–554\.Cited by:[§4\.1](https://arxiv.org/html/2607.25135#S4.SS1.SSS0.Px1.p1.1)\.
- H\. Trivedi, N\. Balasubramanian, T\. Khot, and A\. Sabharwal \(2023\)Interleaving retrieval with chain\-of\-thought reasoning for knowledge\-intensive multi\-step questions\.InProceedings of the 61st annual meeting of the association for computational linguistics \(volume 1: long papers\),pp\. 10014–10037\.Cited by:[§2](https://arxiv.org/html/2607.25135#S2.SS0.SSS0.Px1.p1.1)\.
- S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. Narasimhan, and Y\. Cao \(2022\)React: synergizing reasoning and acting in language models\.arXiv preprint arXiv:2210\.03629\.Cited by:[§3\.1\.1](https://arxiv.org/html/2607.25135#S3.SS1.SSS1.p2.4)\.
- W\. Zhou, J\. Zhang, H\. Hasson, A\. Singh, and W\. Li \(2024\)Hyqe: ranking contexts with hypothetical query embeddings\.InFindings of the Association for Computational Linguistics: EMNLP 2024,pp\. 13014–13032\.Cited by:[§2](https://arxiv.org/html/2607.25135#S2.SS0.SSS0.Px1.p1.1)\.

## Appendix ASystem Prompt

The following is the complete system prompt provided to the ScalableRAG agent, copied verbatim from the source code\. Sections between<<IF:X\>\>and<<ENDIF:X\>\>are conditionally included depending on which modules are enabled \(embeddings, pattern\_search, agg\_tools\)\. Template variables \(\{n\_docs\},\{total\_chars\}, etc\.\) are filled at runtime\.

Youareadocumentretrievalagent\.Youhaveacorpusof\{n\_docs\}documents\(\{total\_chars:,\}characters\)\.AnswerthequestionusingONLYevidencefromthedocuments\.Neverguess\.

Contextbudget:~\{context\_budget\}tokens\.Previousobservationsmayberedactedtosavespace\-\-\-yourthoughtispreserved,soalwaysrecordkeyfindingsinyourthought\.

=======================================================================

STEP0\-\-\-DECOMPOSETHEQUESTION

=======================================================================

BeforedoingANYTHING,analyzethequestionforcomplexity:

∙\\bulletDoesthequestionreferenceentitiesINDIRECTLY\("theXwho\.\.\.","theYwhere\.\.\."\)?EachindirectreferenceisaHOP\-\-\-afactyoumustresolvebeforeyoucananswer\.

∙\\bulletCountthehops\.IfthereareNhops,decomposethequestionintoN\+1numberedsub\-questionsinyourFIRSTthought\.

∙\\bulletAfterresolvingeachsub\-question,state:"Sub\-QKresolved:\[result\]\.Remaining:Sub\-Q\[K\+1,\.\.\.\]\."

∙\\bulletBudgetatleast2toolcallsperhop\.

\*\*\*YourfinalanswermustaddresstheOUTERMOSTquestion,notanintermediatesub\-question\.IfyouresolvedanintermediatefactbutthequestionasksaboutaPROPERTYofthatfact,youareNOTdone\-\-\-keepsearching\.\*\*\*

=======================================================================

APPROACH\-\-\-foreachsub\-question\(orthewholequestionifsingle\-hop\)

=======================================================================

1\.STARTBROAD\-\-\-onekeywordatatime\.Yourfirstapply\_filterMUSTuseaSINGLEentitykeyword,neveracompoundpatternwithmultipleterms\.Forstructuralconstraints\(dates,categories,documenttypes\),prefersearch\_filenames\-\-\-filenamesencodemetadatareliablywhiletextmentionscanbeincidental\.Combinesetswithset\_operation\(intersect\)\.Falsepositivesarefine\-\-\-falsenegativesarehardtorecover\.

2\.EXPLOREfromBOTHENDS\.read\_docsfromthebeginningANDendofyourset\(useoffset=\-3\)\.Documentsaresortedbyfilename,sobeginning/endoftenshowdifferentdocumenttypes\.NoteALLformatvariantsyouobserve\-\-\-ageneralcategorymayappearinseveraldistinctformats\.BuildyourrefinementtocoverALLofthem\.

3\.REFINE\(mandatoryforcounting/aggregation\)\."Containskeyword"≠\\neq"Satisfiesthecondition\."Writeamorespecificregextargetingtheactualformatyouobserved\-\-\-anchortostructuralmarkers\(sectionheadings,fieldlabels\),notbarekeywords\.ApplyWITHINyourworkingset\.Ifthenegativesethasdocsinadifferentformat,buildaseparatefilterandUNION\.

<<IF:agg\_tools\>\>

4\.NUMERICALAGGREGATION\.Forquestionsaskingforaverage,sum,min,max,ortotalofavalue:firstnarrowtotheexactrelevantsubsetofdocs\(not"all"unlessthequestionisaboutalldocs\)\.Thenreadafewdocstolearntheexactformatofthevalue\.Thenuseextract\_fieldwitharegexcapturegrouptoscanEVERYdocinthatsubset\-\-\-nevercomputeextremafromasampleorfromsearchresultsalone\.Checkthe"extracted"countmatchesyourexpectation\.Finally,aggregatewiththerightoperation\.Formin/maxquestionsthataskaboutaPROPERTYoftheextremaldoc,usecreate\_settogetthedoc,thenread\_docs\.Fordates,useas\_type="date"somin/maxgiveearliest/latest\.

4b\.LISTEXTRACTION\.Whenaquestionasks"howmanyXperdoc","whoaretheX","listallX",orrequirescollectingrepeatingitems\(names,lineitems,participants,references\)fromeachdocument,useextract\_list\-\-\-NOTextract\_field\.extract\_fieldreturnsONEvalueperdoc;extract\_listreturnsALLmatchesasalistperdoc\.Readafewdocsfirsttolearntheexactlineformat,writearegexwithacapturegroupthatmatcheseachitem,thenrunextract\_listonthetargetset\.Followwithaggregateusingalistoperation:unique\_values\(deduplicatedlist\),count\_unique\(numberofdistinctitems\),value\_counts\(frequencytable\),orgroup\_by\_value\(whichdocsshareeachvalue\)\.Usetheseforcross\-documentquestionslike"whichXappearinbothAandB"or"howmanyuniqueXacrossalldocs"\.

<<ENDIF:agg\_tools\>\>

5\.VERIFY&ANSWER\.Forcounts:read2\-3docsfromyourNEGATIVE\(excluded\)settoverifyyourrefinementregexdidnotmissvalidvariants\.Ifyoufindadifferentformat,buildasecondfilterandUNION\.Onlythencount\_setontherefinedset\(notthebroadset\-\-\-abroadcountisanupperbound,nevertheanswer\)\.Forfacts:quoteexactwordingfromthesource\.IncludeALLrelevantstructuredfields\.Usethedocument’sownwords\-\-\-donotparaphrase\.

=======================================================================

TOOLS

=======================================================================

apply\_filter\-\-\-Applyregextoaset\.Creates\_posand\_negsets\.Textissection\-normalized:lineswithinaparagrapharejoined\(use\.\*tospanwithinasection,\\nforsectionboundaries\)\.Theresultincludesakeyword\_decompositionshowinghowmanydocsmatcheachindividualkeyword\-\-\-usethistogaugeselectivityandfindabetterstrategywhenyourcombinedpatternistoostrict\.

\{\{"filter\_type":"regex","pattern":"term","target\_set":"all","output\_prefix":"has\_term"\}\}

\{\{"filter\_type":"regex","pattern":"precise","target\_set":"broad\_pos","output\_prefix":"refined"\}\}

<<IF:embeddings\>\>

\{\{"filter\_type":"classifier","filter\_id":"clf\_1","target\_set":"working","output\_prefix":"refined"\}\}

<<ENDIF:embeddings\>\>

explore\_patterns\-\-\-Dry\-run2\-5regexpatternsWITHOUTcreatingsets\.Returnsmatchcountsandsamples\.Compareideascheaply\.

\{\{"patterns":\["pat\_a","pat\_b"\],"target\_set":"all"\}\}

set\_operation\-\-\-Combinetwonamedsets\(intersect,union,difference\)\.

\{\{"operation":"intersect","set\_a":"A","set\_b":"B","result\_name":"A\_and\_B"\}\}

read\_docs\-\-\-ReaddocumentsbysetorIDs\.Useoffsettosampledifferentpositions\.Setfull\_text=trueforthecompletedocument\(noexcerpting\)\-\-\-free,zeroLLMcost\.Usechunk\_idxtopagethroughalargedocumentincontiguouschunks\.

\{\{"set\_name":"my\_set","max\_docs":5,"query":"keywords"\}\}

\{\{"set\_name":"my\_set","max\_docs":3,"offset":\-3\}\}

\{\{"doc\_ids":\["id"\],"full\_text":true\}\}

\{\{"doc\_ids":\["id"\],"chunk\_idx":0\}\}

\#Forhugedocs,pagethroughacontiguousslice:

\{\{"doc\_ids":\["id"\],"start\_char":0,"max\_chars":50000\}\}

find\_in\_doc\-\-\-NavigateinsideONEdocumentbyreturningcharoffsets\.Twomodes:\(1\)exactmatch:providepattern\(\+use\_regex\);\(2\)lexicalpassageranking:providequerytosurfacehigh\-overlapwindows\.Inquerymode,comparemultiplematchesusingtheirpreviewsandhit\_tokens\-\-\-don’tblindlytakeonlythetop\-scoredwindowifitdoesn’tcontaintherightsection/table\.Usewithread\_docs\(start\_char=\.\.\.,max\_chars=\.\.\.\)tojumpdirectlyinhugedocs\.

\{\{"doc\_id":"id","pattern":"BalanceSheets","max\_matches":10\}\}

\{\{"doc\_id":"id","pattern":"cashandcashequivalents","case\_sensitive":false\}\}

\{\{"doc\_id":"id","pattern":"Item\\s\+7\\\.?\\s\+Management","use\_regex":true\}\}

\{\{"doc\_id":"id","query":"businesssegmentsnetincome2022Q2","max\_matches":5\}\}

table\_lookup\-\-\-Findaspecificrowinaligned/table\-liketextandreturntherow\+nearbyheaderlines\(oftencontainingyears/columns\),plusparsednumbers\.Usethistoavoidmis\-readingthewrongyear/columninstatements\.

\{\{"doc\_id":"id","row":"Totalcurrentliabilities","max\_matches":5\}\}

\{\{"doc\_id":"id","row":"Netproperty,plant,andequipment","use\_regex":true\}\}

count\_set\-\-\-Countdocsinaset\.Showsverificationsnippets\.

\{\{"set\_name":"my\_set"\}\}

search\_filenames\-\-\-Finddocsbyfilenamepattern\.Supportstarget\_settorestricttoanexistingset\.Combinetwofilenamesearcheswithset\_operation\(intersect\)forprecisecounts\.

\{\{"pattern":"value","create\_set":"matched"\}\}

\{\{"pattern":"value","target\_set":"existing\_set","create\_set":"refined"\}\}

create\_regex\-\-\-RegisteraregexorhavetheLLMcreateonefromsampledocuments\.

\{\{"pattern":"your\_regex"\}\}

\{\{"condition":"whattodetect","sample\_set\_name":"working"\}\}

<<IF:embeddings\>\>

label\_docs\-\-\-LLMreadseachdocandlabelsit0/1forconditions\.

\{\{"set\_name":"working","labels":\["condition"\],"max\_docs":20\}\}

train\_classifier\-\-\-Trainonembeddings\+labelsfromlabel\_docs\.

\{\{"label\_batch\_id":"label\_0","label\_key":"condition"\}\}

cosine\_search\-\-\-Semanticsearch\.Createsanamedset\(default"cosine\_top\_k",orspecifycreate\_set\)\.Usewhenkeywordsearchreturns0forakeyentity\.Readfromthecreatedset,learncorpusstructure,thenbuildregexfiltersforthefinalanswer\.

\{\{"query":"description","top\_k":10,"create\_set":"my\_cosine"\}\}

<<ENDIF:embeddings\>\>

<<IF:pattern\_search\>\>

pattern\_search\-\-\-Queryapre\-computedextractionindex\.Usesimplesingle\-keywordparams\(e\.g\."Smith"not"John\.\*Smith"\)forbestrecall\.Scopewithtarget\_setFIRSTwhenthequestionconstrainsbyyear/category\.Theresultincludes:\(1\)confirmed\_matches\-\-\-keywordinsidethecapturedstructuralregion;\(2\)gap\_clusters\-\-\-pre\-computedstructuraltemplatesshowingwherethekeywordappearsOUTSIDEthecapturedregion,eachwithatemplate\(keywordreplacedby\{\{\}\}\)andacount\.Compareeachclustertemplatetothecapture\_sampleandtoyourquestion\-\-\-readafewgapdocsfromtheexplore\_setwhenaclustermayberelevanttotheanswer\.

\{\{"pattern":"name","params":\{\{"key":"value"\}\},"target\_set":"some\_set","create\_set":"result"\}\}

<<ENDIF:pattern\_search\>\>

<<IF:agg\_tools\>\>

extract\_field\-\-\-ExtractONEvaluefromEACHdocusingaregexcapturegroup\.Storesresultsasascalarvalueset\(doc→\\rightarrowvalue\)\.Usegroup=1forthefirstcapturegrouporgroup="name"foranamedgroup\(?P<name\>\.\.\.\)\.as\_type:"number"\(default,parseasfloat\),"text"\(rawstring\),or"date"\(parsedate→\\rightarrowenablesmin/maxforearliest/latest\)\.Firstread\_docstolearntheexactvalueformat,thenwritearegexthatcapturesitreliably\.

\{\{"pattern":"your\_regex\_with\_\(capture\)","target\_set":"filtered\_set","as\_type":"number","save\_as":"my\_values"\}\}

\{\{"pattern":"\(?:established\|founded\)\.\*?\(\\w\+\\d\{\{1,2\}\},?\\d\{\{4\}\}\)","target\_set":"all","as\_type":"date","save\_as":"est\_dates"\}\}

extract\_list\-\-\-ExtractALLmatchesofaregexfromEACHdocasalist\.Unlikeextract\_field\(onevalueperdoc\),thisreturnseverymatchperdoc\.Useforrepeatingitems:names,lineitems,references,participants,etc\.Runsonrawdocumenttext\(preservesnewlines\)\.Followwithaggregateusinglistoperations\(unique\_values,count\_unique,value\_counts,group\_by\_value\)\.

\{\{"pattern":"^\(\[A\-Z\]\[a\-z\]\+\[A\-Z\]\[a\-z\]\+\)","target\_set":"filtered\_set","save\_as":"names"\}\}

\{\{"pattern":"Item\\s\+\(\\d\+\)","target\_set":"all","group":1,"save\_as":"item\_numbers"\}\}

aggregate\-\-\-Computestatisticsoveranamedvalueset\.Forscalarsets\(fromextract\_field\):sum,avg,min,max,count\.Formin/max,returnsthematchingdoc\(s\)andoptionallycreatesadocsetforfurtherinspection\.Forlistsets\(fromextract\_list\):unique\_values\(deduplicatedlist\),count\_unique\(numberofdistinctitems\),value\_counts\(frequencytable\),flatten\(allitems\),group\_by\_value\(whichdocsshareeachvalue\-\-\-usefulforcross\-docoverlapquestions\),per\_doc\_count\(totalitemsperdoc→\\rightarrowscalarvalueset\),per\_doc\_count\_unique\(uniqueitemsperdoc→\\rightarrowscalarvalueset\)\.per\_doc\_count\_uniqueisessentialfor"averageuniqueXperdoc"questions:firstextract\_list,thenaggregate\(per\_doc\_count\_unique,create\_set="counts"\),thenaggregate\(value\_set="counts",operation="avg"\)\.

\{\{"value\_set":"my\_values","operation":"avg"\}\}

\{\{"value\_set":"my\_values","operation":"min","create\_set":"lowest\_docs"\}\}

\{\{"value\_set":"names","operation":"unique\_values"\}\}

\{\{"value\_set":"names","operation":"per\_doc\_count\_unique","create\_set":"name\_counts"\}\}

extract\_from\_filename\-\-\-Extractmetadatafromeachdocument’sFILENAMEusingaregexcapturegroup\.Unlikeextract\_field\(whichrunsondocumenttext\),thisrunsonthefilenamestring\.Useformetadataencodedinfilenames:companytickers,dates,quarters,categories\.Storesresultsasascalarvalueset\(doc→\\rightarrowcapturedstring\)\.

\{\{"pattern":"^\(\[A\-Z\]\+\)\_","target\_set":"all","save\_as":"company\_ticker"\}\}

\{\{"pattern":"\(\\d\{\{4\}\}\-Q\\d\)","target\_set":"all","save\_as":"quarter"\}\}

filter\_values\-\-\-Filterdocsbycomparingtheirextractedvaluetoathreshold\.Takesavaluesetfromextract\_field,anoperator\(<,<=,\>,\>=,==,\!=\),andathreshold\.Createsanewdocsetcontainingonlythedocsthatpass\.Fordates,thethresholdcanbeadatestring\(e\.g\."2020\-01\-01"\)\.

\{\{"value\_set":"ratings","operator":"<","threshold":8\.23,"create\_set":"low\_rated"\}\}

\{\{"value\_set":"est\_dates","operator":"\>=","threshold":"2020\-01\-01","create\_set":"recent"\}\}

<<ENDIF:agg\_tools\>\>

calculate\-\-\-Evaluatemathexpressionssafely\.Supports\+,\-,\*,/,//,%,\*\*\(includingfractionalexponentslikex\*\*0\.5\),parentheses,andfunctions:sqrt,cbrt,log,log2,log10,ln,exp,abs,round,ceil,floor,pow,min,max,sum,factorial,gcd\.Constants:pi,e\.UseforANYarithmetic\-\-\-neverdomentalmath\.Formulti\-stepcalculations,passalistofnamedexpressions;eachstepcanreferenceearlierstepnamesasvariables\.

\{\{"expr":"1234\.5/67\.8"\}\}

\{\{"expr":"\(revenue\-cost\)/revenue\*100","variables":\{\{"revenue":5400,"cost":3200\}\}\}\}

\{\{"expressions":\[\{\{"name":"margin","expr":"revenue\-cost"\}\},\{\{"name":"pct","expr":"margin/revenue\*100"\}\}\],"variables":\{\{"revenue":5400,"cost":3200\}\}\}\}

list\_sets<<IF:embeddings\>\>/list\_classifiers<<ENDIF:embeddings\>\>\-\-\-Inspectcurrentstate\.

=======================================================================

GUARDRAILS

=======================================================================

∙\\bullet"ContainsX"≠\\neq"IsaboutX\."Alwaysrefinebeforeanswering\.

∙\\bulletValidatefromdifferentpositions\-\-\-beginning,middle,andend\.

∙\\bulletWatchalternation:"A\|B:\.\*X"means"A"OR"B:\.\*X"\.Use"\(?:A\|B:\)\.\*X"forwhatyouprobablyintend\.

∙\\bulletDifferentformat≠\\neqdifferentanswer\.UNIONformatvariants\.

∙\\bulletRecordfindingsinyourthought\-\-\-observationsgetredacted\.

∙\\bulletSCOPEFIDELITY\.Donotnarrowbeyondwhatthequestionasks\.Ifthequestionusesageneralterm,youranswermustcoverthefullcategory,notasinglesub\-type\.

∙\\bulletBEVERYSKEPTICALOFZERORESULTS\.0matchesusuallymeansyourpatterniswrong,notthedata\.Try:shortersubstrings,differentspellings,broaderregex<<IF:embeddings\>\>,cosine\_searchwithanaturallanguagedescription<<ENDIF:embeddings\>\>\.Exhaustatleast3differentstrategiesbeforeconcludinginformationisabsent\.

∙\\bulletNEVERSTOPATANINTERMEDIATEFINDING\.Ifyouresolved"AisassociatedwithB"butthequestionasksaboutapropertyOFB,youmustkeepsearching\.Re\-readyoursub\-questiondecomposition\.

∙\\bulletINSUFFICIENT\_EVIDENCEisaclaimthatrequiresevidence\.Youmayonlyconclude"insufficientevidence"ifyousearchedthoroughlyandcanstatewhatyousearchedforandwhynothingmatched\.Ifyouhavepartialfindings,giveyourbestanswerandstatewhatisuncertain\.

∙\\bulletCOUNTING:count\_setgivesanEXACTdoccount\.Answer"howmany"withasingleintegerfromcount\_set\.NEVERsay"atleastN"or"approximatelyN"\-\-\-hedgingisalwayswrong\.Ifuncertainaboutfilterquality,refinethesetandre\-countinsteadofhedging\.

<<IF:pattern\_search\>\>

∙\\bulletCONFIRMEDvs\.GAP:pattern\_searchreportsconfirmed\_matches\(keywordinsidethecapturedstructuralregion\)andgap\_clusters\(keywordOUTSIDEit,inadifferentstructuralposition\)\.Thecapturedregiondefineswhatthepatternwasdesignedtoextract\.AgapclusterwithaDIFFERENTsurrounding\-textpatternthanthecapture\_samplemeansthekeywordservesadifferentfunctionthere\-\-\-doNOTcountthosedocs\.Onlyreadgapdocs\(fromtheexplore\_set\)whenagapclustertemplateisgenuinelyambiguousandyoucannotdeterminefromthetemplatealonewhetheritsatisfiesthequestion\.Theconfirmedcountishigh\-confidence;gapadditionsneedclearevidenceofequivalence\.

<<ENDIF:pattern\_search\>\>

∙\\bulletCORPUSONLY\.Yourpre\-trainedknowledgemayconflictwiththiscorpus\.Anabbreviation,name,ortermmayhaveacorpus\-specificmeaningthatdiffersfromtherealworld\.AlwaysderivemeaningFROMthedocuments,neverfrommemory\.

∙\\bulletread\_docsFOCUS\.Whenyouneedaspecificsectionofalargedocument\(e\.g\.,aclause,adate,aspecificterm\),passquery="thespecificterms"tofocusexcerptsonwhatyouneed\.Ifexcerptsstillmissthesection,pagecontiguouslywithchunk\_idx\(coarse\)orstart\_char/max\_chars\(precise\)\.Usefind\_in\_doctogetoffsetsforstart\_char:usepattern/use\_regexwhenyouknowtheexactphrase/sectiontitle;ifyouonlyhaveanatural\-languagedescription,usefind\_in\_doc\(query=\.\.\.\)tosurfacelikelywindows\(comparepreviews\+hit\_tokens\),thenpagecontiguouslywithstart\_char/max\_chars\.Withoutquery,excerptsshowthefilter\-matchcontext\.

∙\\bullet\{max\_steps\}toolcalls\.Typical:5\-15steps\.

=======================================================================

SELF\-CHECKBEFOREANSWERING

=======================================================================

□\\squareDidIresolveALLsub\-questions,oramIansweringanintermediatehop?

□\\squareIsmyanswerbasedontextIactuallyread\(read\_docs\),notjustkeywordcountsorsnippetpreviews?

□\\squareForcounting:ismycountasingleintegerfromcount\_set\-\-\-nothedgedwith"atleast"or"approximately"?

<<IF:agg\_tools\>\>

□\\squareForaggregation\(avg/sum/min/max\):didIuseextract\_field\+aggregate\-\-\-notmanualadditionorestimation?

□\\squareForlist/enumerationquestions:didIuseextract\_list\+aggregate\(unique\_values/count\_unique\)\-\-\-notmanualcounting?

<<ENDIF:agg\_tools\>\>

□\\squareForanyarithmetic\(division,percentages,ratios,differences\):didIusecalculate\-\-\-notmentalmath?

□\\squarePercentvsratio:ONLYmultiplyby100oradda%signifthequestionexplicitlyasksforapercentage/percent/%\.Otherwise,reportratiosasdecimals\(e\.g\.,0\.83\),notpercent\(83%\)\.

□\\squareDoesmyanswersatisfyEVERYconstraintintheoriginalquestion?

□\\squareAmIusingtheCORPUSmeaningofterms,notmyownworldknowledge?

=======================================================================

RESPONSEFORMAT

=======================================================================

Eachturn:exactlyONEJSONobject,nothingelse\.

Toolcall:

\{\{"thought":"whatIplantodoandwhy","tool":"name","args":\{\{\.\.\.\}\}\}\}

Finalanswer:

\{\{"thought":"summaryofevidence","answer":"preciseanswer"\}\}

## Appendix BContext Management Details

##### Question landscape\.

Before the agent’s first turn, we extract all maximal alphanumeric tokens of length≥4\\geq 4from the question \("keywords"\)\. For the first five keywords \(in order of appearance\), the system scans the full corpus via case\-insensitive substring match and records: \(i\) the number of documents whose body text contains the keyword, and \(ii\) the number of filenames containing it\. For four keywords that match at least11but less than all of the documents \(sorted by ascending match count\), we additionally compute and report pairwise intersection sizes, i\.e\., the number of documents containing both keywords simultaneously\. A 600\-character snippet from the top keyword’s first match is appended, showing the agent one formatted example\. The entire landscape is prepended to the user message\.

##### Tokenization and matching rules \(for disambiguation\)\.

The code uses similar\-looking but distinct lexical operations in different places; we list them explicitly to avoid ambiguity\.

- •Question landscape tokens \(“keywords”\)\.Tokens are extracted from the question via`\[ˆ\\W\_\]\{4,\}`\. A token is counted as present in a document if it matches by case\-insensitive substring inclusion \(tok\.lower\(\) in text\.lower\(\)\)\. These counts provide a cheap prior only\.
- •Anchor tokens \(“entities”\)\.The system sorts a subset the keywords with0<count​\(t​o​k\)<N0<\\mathrm\{count\}\(tok\)<N, preferring tokens that appear capitalized in the original question and then by increasing document frequency\. These anchor tokens are used only for entity\-aware diagnostics \(scope checks; negative\-entity checks\), not as a learned NER component\.
- •apply\_filterdecomposition tokens\.For diagnostics, tokens are extracted from the*regex string*with the same tokenizer`\[ˆ\\W\_\]\{4,\}`\. Each token is then probed as a single\-token case\-insensitive regex over normalized text to estimate per\-token selectivity and token intersections\.
- •Extraction\-index value tokens \(gap clusters\)\.During indexing, tokens are extracted from captured values via`\[ˆ\\W\\d\_\]\{3,\}`\(letter\-only, length≥3\\geq 3\), lowercased, and used to pre\-computekeyword\_gaps: cases where a value token appears in a document but outside the captured region\.pattern\_searchsurfaces these as gap clusters \(surrounding\-text templates with the token replaced by\{\}\)\.

##### Context management overview\.

The agent’s context window must accommodate the system prompt, the question, the question landscape, and a growing sequence of tool observations and agent responses\. Because individual documents can be hundreds of thousands of characters and trajectories can span 25 steps, two mechanisms prevent overflow while preserving the set\-algebraic state that the agent needs for planning\.

##### Per\-tool output budgets\.

Every tool observation is hard\-capped atBtoolB\_\{\\mathrm\{tool\}\}characters \(set to 200,000 for models with≥\\geq500K context, proportionally less for smaller models\)\. Forread\_docsspecifically, the budget is further divided evenly among the requested documents: when readingkkdocuments, each gets⌊Bcall/k⌋\\lfloor B\_\{\\mathrm\{call\}\}/k\\rfloorcharacters, whereBcall=min⁡\(Cmax/6,Btool\)B\_\{\\mathrm\{call\}\}=\\min\(C\_\{\\max\}/6,B\_\{\\mathrm\{tool\}\}\)\. Documents exceeding their per\-doc budget are presented via*keyword\-guided excerpting*: the system identifies text windows around query\-relevant terms and concatenates only those passages\. For long\-document navigation without relying on excerpting, the agent can also page contiguously \(chunked reads\) or request explicitstart\_char/max\_charsslices\.

##### Observation redaction\.

After the agent responds to an observation \(i\.e\., the observation is “consumed"\), the full text is replaced by a structural summary\. The summary retains a whitelisted set of JSON keys:

- •*Set identity*:positive\_set,negative\_set,result\_set,created\_set\.
- •*Cardinalities*:positive\_count,negative\_count,total\_filtered,count\.
- •*Diagnostics*:keyword\_decomposition\(per\-term match counts\),entity\_coverage,selectivity\_gap,zero\_result\_hint,scope\_check,negative\_entity\_check\.
- •*Extraction state*:value\_set,extracted,scanned,n\_values,matching\_docs\.

Everything else \(raw document text, full match listings\) is discarded\. The agent’s ownthoughtfield \(which the system prompt instructs it to use as a scratchpad\) preserves factual findings across turns\.

##### Last\-resort compression\.

If the total conversation still exceeds the budgetB=0\.6×CmaxB=0\.6\\times C\_\{\\max\}characters after redaction, messages in the middle of the conversation \(excluding the system prompt, the initial question, and the four most recent messages\) are further truncated to their first 150 characters\. If still over budget, middle messages are dropped entirely\.

The net effect is that the*workspace state*\(the names, sizes, provenance, and diagnostic metadata of all sets\) is always visible to the agent, while raw document content is available only for the most recent observation and must be captured in the agent’s thought before it becomes redacted\.

## Appendix CTrace Examples

The following are representative agent traces copied verbatim from logs\. They illustrate how diagnostic feedback drives the agent’s reasoning\.

### C\.1Aggregation: Transcripts Corpus

This trace illustrates ScalableRAG answering a corpus\-wide aggregation using two filename filters and a difference computation\.

Question:Howmanymorehearingsdidonechamberholdthantheotherinthiscorpus?

Goldenanswer:5\.

Agentanswer:TheHouseheld5morehearingsthantheSenateinthiscorpus\.

Step0:search\_filenames\(\{

"pattern":"hhrg",

"create\_set":"house\_hearings"

\}\)

Agentthought:Toanswerhowmanymorehearingsonechamberheldthantheother,Ineedto\(1\)identifywhichhearingswereheldbyeachchamber\(HousevsSenate\),\(2\)countthenumberofhearingsforeach,and\(3\)computethedifference\.Thebestwaytodistinguishchambersisbyfilename\(e\.g\.,’hhrg’forHouse,’shrg’forSenate\),asthisisastandardconventionincongressionalhearingtranscripts\.Iwillfirstsearchfilenamesfor’hhrg’\(House\)and’shrg’\(Senate\),counteach,andthencalculatethedifference\.

Observation:\{

"count":39,

"searched":75,

"matches":\[

\{

"doc\_id":"CHRG\-117hhrg44411\_\_396ab7bdb162",

"filename":"CHRG\-117hhrg44411\.txt",

"chars":87030

\},

\{

"doc\_id":"CHRG\-117hhrg47493\_\_dadb2f96711a",

"filename":"CHRG\-117hhrg47493\.txt",

"chars":180244

\},

\{

"doc\_id":"CHRG\-117hhrg47530\_\_281e009dab4d",

"filename":"CHRG\-117hhrg47530\.txt",

"chars":85073

\},

\{

"doc\_id":"CHRG\-117hhrg47569\_\_dcfab7dcae2c",

"filename":"CHRG\-117hhrg47569\.txt",

"chars":445619

\},

\{

"doc\_id":"CHRG\-117hhrg47691\_\_58319a4c1cfd",

"filename":"CHRG\-117hhrg47691\.txt",

"chars":168012

\},

\{

"doc\_id":"CHRG\-117hhrg47737\_\_9026617797e9",

"filename":"CHRG\-117hhrg47737\.txt",

"chars":65819

\},

\{

"doc\_id":"CHRG\-117hhrg47840\_\_2ee4adedbbb6",

"filename":"CHRG\-117hhrg47840\.txt",

"chars":96050

\},

\{

"doc\_id":"CHRG\-117hhrg47882\_\_e93645a43e27",

"filename":"CHRG\-117hhrg47882\.txt",

"chars":84991

\},

\{

"doc\_id":"CHRG\-117hhrg47929\_\_d773214aa378",

"filename":"CHRG\-117hhrg47929\.txt",

"chars":114632

\},

\{

"doc\_id":"CHRG\-117hhrg48013\_\_a4aeb68a5f5f",

"filename":"CHRG\-117hhrg48013\.txt",

"chars":78597

\},

\{

"note":"\.\.\.\(24more\)\.\.\."

\},

\{

"doc\_id":"CHRG\-117hhrg59789\_\_4c57831bcc67",

"filename":"CHRG\-117hhrg59789\.txt",

"chars":150905

\},

\{

"doc\_id":"CHRG\-117hhrg60497\_\_4d86153ef05b",

"filename":"CHRG\-117hhrg60497\.txt",

"chars":96107

\},

\{

"doc\_id":"CHRG\-117hhrg61455\_\_fa67ee74f3ce",

"filename":"CHRG\-117hhrg61455\.txt",

"chars":158615

\},

\{

"doc\_id":"CHRG\-117hhrg61990\_\_8cbf8eef8f8b",

"filename":"CHRG\-117hhrg61990\.txt",

"chars":159716

\},

\{

"doc\_id":"CHRG\-117hhrg62185\_\_7d975bb7125d",

"filename":"CHRG\-117hhrg62185\.txt",

"chars":159174

\}

\],

"set\_created":"house\_hearings",

"not\_matching":36,

"filename\_range":"CHRG\-117hhrg44411\.txt\.\.\.CHRG\-117hhrg62185\.txt"

\}

\[Auto\-sampledsnippets\(house\_hearings\)\-\-\-3of39docs,explorefurtherifneeded\]

\[CHRG\-117hhrg44411\.txt\]\-\[H\.A\.S\.C\.No\.117\-12\]DEPARTMENTOFDEFENSEELECTROMAGNETICSPECTRUMOPERATIONS:CHALLENGESANDOPPORTUNITIESINTHEINVISIBLEBATTLESPACE

\[HouseHearing,117Congress\]

\[FromtheU\.S\.GovernmentPublishingOffice\]

\.

\[H\.A\.S\.C\.No\.117\-12\]

DEPARTMENTOFDEFENSE

ELECTROMAGNETICSPECTRUM

OPERATIONS:CHALLENGESAND

OPPORTUNITIESINTHE

INVISIBLEBATTLESPACE

\_\_\_\_\_\_\_\_\_\_

HEARING

BEFORETHE

SUBCOMMITTEEONCYBER,INNOVATIVE

TECHNOLOGIES,ANDINFORMATIONSYSTEMS

OFTHE

COMMITTEEONARMEDSERVICES

HOUSEOFREPRESENTATIVES

ONEHUNDREDSEVENTEENTHCONGRESS

FIRSTSESSION

\_\_\_\_\_\_\_\_\_\_

HEARINGHELD

MARCH19,2021

\[GRAPHICNOTAVAILABLEINTIFFFORMAT\]

\_\_\_\_\_\_\_\_\_\_

U\.S\.GOVERNMENTPUBLISHINGOFFICE

44\-411WASHINGTON:2021

\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\.\.\.

\[CHRG\-117hhrg48949\.txt\]\.\.\.rmerandthefounderofCrackOpentheDoorandGetClemencyNow\.Ihavecontributedtonearlyadozenindividualsbeingreleasedearlyfromprison,nineofwhichwereservinglifewithoutparole\.IamanauthorwhohaswrittenabookcalledClemencyNow,GetClemencyNow,whichteachespeopleincarceratedandtheirfamilieswhotoputtogetherpowerfulclemencypetitions\.Iamthe2017BlackChamberofCommerceTrailblazerandCommunityCivicLeaderoftheYear\.Iamthe2018McKinneyVolunteeroftheYear\.Iamthe2021LeadershipMcKinneyAlumnusoftheYear\.I’malsothe2022NACDL’sChampionofJusticeoftheYear\.IamalsothefounderandExecutiveDirectorofAtLast,whichisaleadershipprogramforLatinohighschoolstudentswhichteachesthemhowtobecomeleadersintheirsc\.\.\.

\[CHRG\-117hhrg62185\.txt\]\-NOTIMETOWASTE:SOLUTIONSFORAMERICA’SBROKENRECYCLINGSYSTEM

\[HouseHearing,117Congress\]

\[FromtheU\.S\.GovernmentPublishingOffice\]

NOTIMETOWASTE:SOLUTIONSFORAMERICA’S

BROKENRECYCLINGSYSTEM

=====================================

HYBIRDHEARING

BEFORETHE

SUBCOMMITTEEONENVIRONMENTANDCLIMATE

CHANGE

OFTHE

COMMITTEEONENERGYANDCOMMERCE

HOUSEOFREPRESENTATIVES

ONEHUNDREDSEVENTEENTHCONGRESS

SECONDSESSION

\_\_\_\_\_\_\_\_\_\_

JUNE30,2022

\_\_\_\_\_\_\_\_\_\_

SerialNo\.117\-92

\[GRAPHICNOTAVAILABLEINTIFFFORMAT\]

PublishedfortheuseoftheCommitteeonEnergyandCommerce

govinfo\.gov/committee/house\-energy

energycommerce\.house\.gov

\_\_\_\_\_\_\_\_\_\_

U\.S\.GOVERNMENTPUBLISHINGOFFICE

62\-185PDFWASHINGTON:2026

\-\.\.\.

Step1:search\_filenames\(\{

"pattern":"shrg",

"create\_set":"senate\_hearings"

\}\)

Agentthought:Ihaveidentified39Househearingsbysearchingfor’hhrg’infilenames\.Now,Iwillsearchfor’shrg’infilenamestoidentifySenatehearings,countthem,andthencalculatethedifferencebetweenthetwochambers\.

Observation:\{

"count":34,

"searched":75,

"matches":\[

\{

"doc\_id":"CHRG\-117shrg19104915\_\_a98345dc64a0",

"filename":"CHRG\-117shrg19104915\.txt",

"chars":824153

\},

\{

"doc\_id":"CHRG\-117shrg39104013\_\_e0f425d86cc4",

"filename":"CHRG\-117shrg39104013\.txt",

"chars":256863

\},

\{

"doc\_id":"CHRG\-117shrg39104020\_\_c38508a0247b",

"filename":"CHRG\-117shrg39104020\.txt",

"chars":161017

\},

\{

"doc\_id":"CHRG\-117shrg39104988\_\_ee0e35c33612",

"filename":"CHRG\-117shrg39104988\.txt",

"chars":149038

\},

\{

"doc\_id":"CHRG\-117shrg39104991\_\_6a10d5809ff3",

"filename":"CHRG\-117shrg39104991\.txt",

"chars":174014

\},

\{

"doc\_id":"CHRG\-117shrg47818\_\_daec5e8ab066",

"filename":"CHRG\-117shrg47818\.txt",

"chars":117084

\},

\{

"doc\_id":"CHRG\-117shrg47983\_\_c74f8b4a35f8",

"filename":"CHRG\-117shrg47983\.txt",

"chars":110745

\},

\{

"doc\_id":"CHRG\-117shrg48137\_\_6073c99ce3d1",

"filename":"CHRG\-117shrg48137\.txt",

"chars":586346

\},

\{

"doc\_id":"CHRG\-117shrg48287\_\_522b12a1d919",

"filename":"CHRG\-117shrg48287\.txt",

"chars":126257

\},

\{

"doc\_id":"CHRG\-117shrg48907\_\_23b1cd53b6a3",

"filename":"CHRG\-117shrg48907\.txt",

"chars":183568

\},

\{

"note":"\.\.\.\(19more\)\.\.\."

\},

\{

"doc\_id":"CHRG\-117shrg54763\_\_e3aec0d0e287",

"filename":"CHRG\-117shrg54763\.txt",

"chars":309122

\},

\{

"doc\_id":"CHRG\-117shrg55777\_\_06ae1ae00374",

"filename":"CHRG\-117shrg55777\.txt",

"chars":154286

\},

\{

"doc\_id":"CHRG\-117shrg55818\_\_63675610a12d",

"filename":"CHRG\-117shrg55818\.txt",

"chars":130758

\},

\{

"doc\_id":"CHRG\-117shrg59763\_\_055014dfc194",

"filename":"CHRG\-117shrg59763\.txt",

"chars":169810

\},

\{

"doc\_id":"CHRG\-117shrg59770\_\_0237ff3bf0d9",

"filename":"CHRG\-117shrg59770\.txt",

"chars":158404

\}

\],

"set\_created":"senate\_hearings",

"not\_matching":41,

"filename\_range":"CHRG\-117shrg19104915\.txt\.\.\.CHRG\-117shrg59770\.txt"

\}

\[Auto\-sampledsnippets\(senate\_hearings\)\-\-\-3of34docs,explorefurtherifneeded\]

\[CHRG\-117shrg19104915\.txt\]\-NONDEPARTMENTALWITNESSES

\[SenateHearing117\-\]

\[FromtheU\.S\.GovernmentPublishingOffice\]

STATE,FOREIGNOPERATIONS,ANDRELATEDPROGRAMSAPPROPRIATIONSFOR

FISCALYEAR2022

\-\-\-\-\-\-\-\-\-\-

U\.S\.Senate,

SubcommitteeoftheCommitteeonAppropriations,

Washington,DC\.

NONDEPARTMENTALWITNESSES

\[Clerk’snote\.\-\-Thesubcommitteewasunabletohold

hearingsonnondepartmentalwitnesses\.Thestatementsand

lettersofthosesubmittingwrittentestimonyareasfollows:\]

PreparedStatementoftheAccountabilityCounsel

DearChairmanCoons,RankingMemberGraham,andmembersofthe

subcommittee:

OnbehalfofAccountabilityCounsel,thankyouforthisopportunity

toprovideinputontheFY2022State,ForeignOperations,andRelated

Programs\(SFOPs\)appropriationsprocess\.I\.\.\.

\[CHRG\-117shrg51394\.txt\]\-FARMBILL2023:RESEARCHPROGRAMS

\[SenateHearing117\-613\]

\[FromtheU\.S\.GovernmentPublishingOffice\]

S\.Hrg\.117\-613

FARMBILL2023:RESEARCHPROGRAMS

=====================================

HEARING

BEFORETHE

COMMITTEEONAGRICULTURE,

NUTRITION,ANDFORESTRY

UNITEDSTATESSENATE

ONEHUNDREDSEVENTEENTHCONGRESS

SECONDSESSION

\_\_\_\_\_\_\_\_\_\_

December6,2022

\_\_\_\_\_\_\_\_\_\_

Printedfortheuseofthe

CommitteeonAgriculture,Nutrition,andForestry

\[GRAPHICNOTAVAILABLEINTIFFFORMAT\]

Availableonhttp://www\.govinfo\.gov/

\_\_\_\_\_\_\_\_\_\_

U\.S\.GOVERNMENTPUBLISHINGOFFICE

51\-394PDFWASHINGTON:2024

\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-

COMMITTEEONAGRICULTURE,NUTRITIO\.\.\.

\[CHRG\-117shrg59770\.txt\]\-WORLDWIDETHREATS

\[SenateHearing117\-960\]

\[FromtheU\.S\.GovernmentPublishingOffice\]

S\.Hrg\.117\-960

WORLDWIDETHREATS

=====================================

HEARING

BEFORETHE

COMMITTEEONARMEDSERVICES

UNITEDSTATESSENATE

ONEHUNDREDSEVENTEENTHCONGRESS

SECONDSESSION

\_\_\_\_\_\_\_\_\_\_

May10,2022

\_\_\_\_\_\_\_\_\_\_

PrintedfortheuseoftheCommitteeonArmedServices

\[GRAPHCNOTAVAILABLEINTIFFFORMAT\]

Availablevia:http://www\.govinfo\.gov

\_\_\_\_\_\_\_\_\_\_

U\.S\.GOVERNMENTPUBLISHINGOFFICE

59\-770PDFWASHINGTON:2025

\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-

COMMITTEEONARMEDSERVICES

JACKREED,RhodeIsland,ChairmanJAMESM\.INHOFE,Oklahoma

JEANNESHAHEEN,NewHampshire\.\.\.

===Automaticallycomputed===

Setcompositions\-\-\-house\_hearings\(39\)×\\timessenate\_hearings\(34\):

intersect→\\rightarrow0

union→\\rightarrow73

house\_hearings\\senate\_hearings→\\rightarrow39

senate\_hearings\\house\_hearings→\\rightarrow34

Step2:calculate\(\{

"expr":"39\-34"

\}\)

Agentthought:Ihaveidentified39Househearingsand34Senatehearingsbyfilenamepattern\.Thedifferenceis39\-34=5\.Iwillnowcalculatethisdifferencetoanswerhowmanymorehearingsonechamberheldthantheother\.

Observation:\{

"name":"result",

"expr":"39\-34",

"value":5,

"display":"5"

\}

### C\.2Multi\-Hop with Diagnostic Recovery: 2WikiMultiHopQA

This trace illustrates a two\-hop question solved by constructing and comparing named document sets\. The agent starts with a broad lexical filter, then refines by intersecting constraints \(song title, composer mention\), and uses diagnostics such asscope\_checkandnegative\_entity\_checkplus auto\-sampled snippets to confirm the correct entity before extracting the birthplace\.

Question:WherewasthecomposerofsongBackInTheU\.S\.A\.born?

Goldenanswer:St\.Louis,Missouri

Agentanswer:St\.Louis,Missouri

Step0:apply\_filter\(\{

"filter\_type":"regex",

"pattern":"composer",

"target\_set":"all",

"output\_prefix":"has\_composer"

\}\)

Agentthought:Thequestionis:Wherewasthecomposerofthesong’BackInTheU\.S\.A\.’born?Thisisatwo\-hopquestion:\(1\)Whoisthecomposerof’BackInTheU\.S\.A\.’?\(2\)Wherewasthatcomposerborn?Iwillfirstsearchfordocumentsmentioningboth’composer’and’Back’toidentifythecomposer,thensearchforthecomposer’sbirthplace\.

Observation:\{

"positive\_set":"has\_composer\_pos",

"positive\_count":116,

"negative\_set":"has\_composer\_neg",

"negative\_count":6002,

"total\_filtered":6118,

"sample\_positive\_docs":\[

\{

"doc\_id":"A\_Time\_of\_Destiny\_\_1b00ec88a396",

"filename":"A\_Time\_of\_Destiny\.txt"

\},

\{

"doc\_id":"Abd\\u00fclaziz\_\_6cfec66ec8bc",

"filename":"Abd\\u00fclaziz\.txt"

\},

\{

"doc\_id":"Keith\_Emerson\_\_f2d70a357c06",

"filename":"Keith\_Emerson\.txt"

\},

\{

"doc\_id":"Kurt\_Schr\\u00f6der\_\_4830d105778e",

"filename":"Kurt\_Schr\\u00f6der\.txt"

\},

\{

"doc\_id":"Youve\_Got\_Possibilities\_\_e9a4993a16da",

"filename":"Youve\_Got\_Possibilities\.txt"

\},

\{

"doc\_id":"\\u00c1ngel\_Recasens\_\_6497d74babaa",

"filename":"\\u00c1ngel\_Recasens\.txt"

\}

\],

"sample\_negative\_docs":\[

\{

"doc\_id":"11\_Harrowhouse\_\_08cc6c95c2f9",

"filename":"11\_Harrowhouse\.txt"

\},

\{

"doc\_id":"1971\_Copa\_Libertadores\_\_8369d1b96cbe",

"filename":"1971\_Copa\_Libertadores\.txt"

\},

\{

"doc\_id":"Khalid\_Abdel\_Nasser\_\_b9f61f804f4b",

"filename":"Khalid\_Abdel\_Nasser\.txt"

\},

\{

"doc\_id":"Khalid\_al\-Habib\_\_ca209876aaad",

"filename":"Khalid\_al\-Habib\.txt"

\},

\{

"doc\_id":"\\u021aerova\_\_10c94b229856",

"filename":"\\u021aerova\.txt"

\},

\{

"doc\_id":"\\u02bbElisiva\_Fusipala\_Tauki\\u02bbonetuku\_\_3e99f95742e5",

"filename":"\\u02bbElisiva\_Fusipala\_Tauki\\u02bbonetuku\.txt"

\}

\],

"scope\_check":"apply\_filtersearchesDOCUMENTTEXT\.116docsmatchedintext,but4filenamesmatchthesamepattern\.Ifyouintendedtofilterbyfilename/date,usesearch\_filenamesinstead\.",

"negative\_entity\_check":"139/6002negativescontain’Back’\(keywordonly\)\."

\}

\[Auto\-sampledsnippets\(positive\)\-\-\-3of116docs,explorefurtherifneeded\]

\[A\_Time\_of\_Destiny\.txt\]ATimeofDestiny

ATimeofDestinyisa1988AmericandramafilmdirectedbyGregoryNavaandwrittenbyNavaandAnnaThomas\.Thestoryisbasedontheopera"Laforzadeldestino"byGiuseppeVerdi\.ThemotionpicturewasexecutiveproducedbyShepGordonandCarolynPfeiffer\.ItfeaturesoriginalmusicbyveterancomposerEnnioMorricone\.SetduringWorldWarIIinItalyandSanDiego,thefilmtellsoftwofriendswhobecomeenemiesduringthewar\.

\[Kurt\_Schröder\.txt\]KurtSchröder

KurtSchröder\(1888\-\-1962\)wasaGermancomposer\.Schrödercomposedanumberoffilmscores\.Duringthe1930sheworkedinBritainforAlexanderKorda’sLondonFilmProductions,andscoredthecompany’sbreakthroughhit"ThePrivateLifeofHenryVIII"in1933\.

\[Ángel\_Recasens\.txt\]ÁngelRecasens

ÁngelRecasens\(4March1938inCambrils\-\-2August2007\)wasaCatalanorganist,teacher,composerandmusicologistbestknownasachoralconductor\.From1975to1986,hewasdirectoroftheCorodeSantEsteveofVila\-seca\.There,heperformedmusicfromtheromanticsSchumannandMendelssohn,tothecontemporarymusicofLigetiandSchnittke\.HealsodirectedtheCoralVergedelCamíinhishometownCambrils\.Inhislateryears,hededicatedhimselftorevivinglostSpanishchoralworksfromthe16thto18thcenturies;firstwithlaCapillaPríncipedeViana,thenwiththeexpandedLaGrandeChapelleandhisownlabelLaudaRecords\.BoththeensembleandthelabelwerecontinuedbyhissonAlbertRecasens\.

\[Auto\-sampledsnippets\(negative\)\-\-\-3of6002docs,explorefurtherifneeded\]

\[11\_Harrowhouse\.txt\]11Harrowhouse

11Harrowhouseisa1974BritishDeLuxeColorfilmdirectedbyAramAvakianandstarringCharlesGrodin,CandiceBergen,JamesMason,TrevorHowardandJohnGielgudinPanavision\.ItwasadaptedbyCharlesGrodinbaseduponthenovelbyGeraldA\.BrownewiththescreenplaybyJeffreyBloom\.

\[Khalid\_al\-Habib\.txt\]Khalidal\-Habib

Khalidal\-HabibalsobelievedtousethenomdeguerreKhalidalHarbiisanAl\-QaedaleaderwhohasoverseenAl\-QaedaoperationsinAfghanistan\.In2005itwasreportedthatal\-HabibdiedinaUSmissilestrikeonthePakistanivillageofDamadolathoughitisunclearwhetherthisisthecaseasPakistaniofficialshaveclaimedthatnoAl\-Qaedaleadersdiedintheattack\.al\-Habib’snationalityisnotclearwithsourcesclaiminghemaybeEgyptianorMoroccan\.

\[’Elisiva\_Fusipala\_Tauki’onetuku\.txt\]’ElisivaFusipalaTauki’onetuku

’ElisivaFusipalaTauki’onetuku\(18May1850\-\-September1889\)wasthemotherofKingGeorgeTupouII\.

===Automaticallycomputed===

Filenamesearchcounts:

search\_filenames\("composer"\)→\\rightarrow4docs

Step1:apply\_filter\(\{

"filter\_type":"regex",

"pattern":"Back",

"target\_set":"has\_composer\_pos",

"output\_prefix":"composer\_and\_Back"

\}\)

Agentthought:NowIneedtonarrowtodocsthatmentionboth’composer’and’Back’tofindthecomposerof’BackInTheU\.S\.A\.’\.Thiswillhelpidentifythecomposerbeforesearchingfortheirbirthplace\.

Observation:\{

"positive\_set":"composer\_and\_Back\_pos",

"positive\_count":7,

"negative\_set":"composer\_and\_Back\_neg",

"negative\_count":109,

"total\_filtered":116,

"sample\_positive\_docs":\[

\{

"doc\_id":"Deva\_composer\_\_371eabd0fe49",

"filename":"Deva\_composer\.txt"

\},

\{

"doc\_id":"Johann\_Sebastian\_Bach\_\_d44cb4c567c7",

"filename":"Johann\_Sebastian\_Bach\.txt"

\},

\{

"doc\_id":"Mario\_Bonnard\_\_be124b2927fc",

"filename":"Mario\_Bonnard\.txt"

\},

\{

"doc\_id":"Michael\_Corcoran\_musician\_\_f988e03f8c59",

"filename":"Michael\_Corcoran\_musician\.txt"

\},

\{

"doc\_id":"Raffy\_Shart\_\_4b2c2178b7ca",

"filename":"Raffy\_Shart\.txt"

\},

\{

"doc\_id":"Youve\_Got\_Possibilities\_\_e9a4993a16da",

"filename":"Youve\_Got\_Possibilities\.txt"

\}

\],

"sample\_negative\_docs":\[

\{

"doc\_id":"A\_Time\_of\_Destiny\_\_1b00ec88a396",

"filename":"A\_Time\_of\_Destiny\.txt"

\},

\{

"doc\_id":"Abd\\u00fclaziz\_\_6cfec66ec8bc",

"filename":"Abd\\u00fclaziz\.txt"

\},

\{

"doc\_id":"Karel\_Zich\_\_efc220f3e985",

"filename":"Karel\_Zich\.txt"

\},

\{

"doc\_id":"Karl\_Wilhelm\_conductor\_\_fdea54ab7eab",

"filename":"Karl\_Wilhelm\_conductor\.txt"

\},

\{

"doc\_id":"William\_Axt\_\_f2e5b44b3b94",

"filename":"William\_Axt\.txt"

\},

\{

"doc\_id":"\\u00c1ngel\_Recasens\_\_6497d74babaa",

"filename":"\\u00c1ngel\_Recasens\.txt"

\}

\],

"scope\_check":"apply\_filtersearchesDOCUMENTTEXT\.7docsmatchedintext,but0filenamesmatchthesamepattern\.Ifyouintendedtofilterbyfilename/date,usesearch\_filenamesinstead\."

\}

\[Auto\-sampledsnippets\(positive\)\-\-\-3of7docs,explorefurtherifneeded\]

\[Deva\_composer\.txt\]Deva\(composer\)

DevanesanChokkalingam,popularlyknownasDeva,isanIndianfilmcomposerandsinger\.HehascomposedsongsandprovidedbackgroundmusicforTamil,Telugu,MalayalamandKannadafilmsinacareerspanningabout20years\.Manyknowhisgaanasongs,writtenmostlyusingMadrasTamil\.Heisknownasthe"FatherofGaanaGenre"intheTamilfilmindustry\.Devahascomposedmusicformanyfilms\.Hedebutedasafilmmusicdirectorinthefilm"ManasukkeththaMaharaasa"in1989\.Intheinterveningyearshehascomposedmusicforatotalofmorethan400films\.

\[Michael\_Corcoran\_musician\.txt\]MichaelCorcoran\(musician\)

MichaelThomasCorcoran\(bornDecember9,1972\),knownprofessionallyasBackhouseMikeorKenLofkoll,isanAmericanmusician,singer\-songwriter,composer,sounddesigner,musicalarranger,orchestrator,andrecordproducer\.

\[Youve\_Got\_Possibilities\.txt\]\.\.\.epryyoufromyourshell\.\.\.You’vegotpossibilities\.\.\.youdon’tevenknowyou’vegot"\)\."It’saBird\.\.\.It’saPlane\.\.\.It’sSuperman"wasnotabighit,but"You’veGotPossibilities",generallyconsideredtheshow’smostmemorabletune,becamesomethingofacabaretstandard\.PeggyLeerecordedthesongonher1966album"Big$pender"andreleaseditastheB\-sideofthesingle"ComeBackToMe"\.JoanieSommersreleased"You’veGotPossibilities"astheB\-sideofhersingle"NeverThrowYourDreamsAway",alsoin1966,whileCarolVenturareleaseditalsoin1966asanA\-sidesingle\.LindaLavinincludedthesongonher2011album"Possibilities"\(shehadearliersungitonthe1966originalcastalbumfor"It’saBird\.\.\.It’saPlane\.\.\.It’sSuperman"\)\.MattMonro’sversion\.\.\.

\[Auto\-sampledsnippets\(negative\)\-\-\-3of109docs,explorefurtherifneeded\]

\[A\_Time\_of\_Destiny\.txt\]ATimeofDestiny

ATimeofDestinyisa1988AmericandramafilmdirectedbyGregoryNavaandwrittenbyNavaandAnnaThomas\.Thestoryisbasedontheopera"Laforzadeldestino"byGiuseppeVerdi\.ThemotionpicturewasexecutiveproducedbyShepGordonandCarolynPfeiffer\.ItfeaturesoriginalmusicbyveterancomposerEnnioMorricone\.SetduringWorldWarIIinItalyandSanDiego,thefilmtellsoftwofriendswhobecomeenemiesduringthewar\.

\[Karl\_Wilhelm\_conductor\.txt\]KarlWilhelm\(conductor\)

KarlWilhelm,alsoCarlWilhelm\(5September1815,Schmalkalden\-\-26August1873,Schmalkalden\)wasaGermanchoraldirector\.Heisbestknownasthecomposerofthemusicofthesong‘‘DieWachtamRhein\.’’

\[Ángel\_Recasens\.txt\]ÁngelRecasens

ÁngelRecasens\(4March1938inCambrils\-\-2August2007\)wasaCatalanorganist,teacher,composerandmusicologistbestknownasachoralconductor\.From1975to1986,hewasdirectoroftheCorodeSantEsteveofVila\-seca\.There,heperformedmusicfromtheromanticsSchumannandMendelssohn,tothecontemporarymusicofLigetiandSchnittke\.HealsodirectedtheCoralVergedelCamíinhishometownCambrils\.Inhislateryears,hededicatedhimselftorevivinglostSpanishchoralworksfromthe16thto18thcenturies;firstwithlaCapillaPríncipedeViana,thenwiththeexpandedLaGrandeChapelleandhisownlabelLaudaRecords\.BoththeensembleandthelabelwerecontinuedbyhissonAlbertRecasens\.

\[Contextaudit\-\-\-’Back’in5of7positivedocs:\]

\.\.\.ger\.HehascomposedsongsandprovidedbackgroundmusicforTamil,Telugu,Malayala\.\.\.

\.\.\.manyinfilmsdirectedbyLuisTrenker\.BackinItalyin1932,hebecameaprolific\.\.\.

\.\.\.ember9,1972\),knownprofessionallyasBackhouseMikeorKenLofkoll,isanAmerica\.\.\.

\.\.\.kingattheÜberbrettlcabaret\.HewentbacktoViennaandbeganwritingoperettas,\.\.\.

\.\.\.editastheB\-sideofthesingle"ComeBackToMe"\.JoanieSommersreleased"You’ve\.\.\.

===Automaticallycomputed===

Filenamesearchcounts:

search\_filenames\("Back"\)→\\rightarrow11docs

Setcompositions\-\-\-has\_composer\_pos\(116\)×\\timescomposer\_and\_Back\_pos\(7\):

intersect→\\rightarrow7

union→\\rightarrow116

has\_composer\_pos\\composer\_and\_Back\_pos→\\rightarrow109

composer\_and\_Back\_pos\\has\_composer\_pos→\\rightarrow0

Step2:apply\_filter\(\{

"filter\_type":"regex",

"pattern":"BackInTheU\\\\\.S\\\\\.A\\\\\.",

"target\_set":"all",

"output\_prefix":"has\_BackInUSA"

\}\)

Agentthought:Noneofthe7docsincomposer\_and\_Back\_posappeartobeaboutthesong’BackInTheU\.S\.A\.’\-\-\-thematchesarefor’BackhouseMike’,’ComeBackToMe’,andotherunrelatedusesof’Back’\.Ineedtotryabroadersearchforthesongtitle’BackInTheU\.S\.A\.’toidentifythecomposerdirectly\.Iwillsearchfor’BackInTheU\.S\.A\.’inalldocs\.

Observation:\{

"positive\_set":"has\_BackInUSA\_pos",

"positive\_count":1,

"negative\_set":"has\_BackInUSA\_neg",

"negative\_count":6117,

"total\_filtered":6118,

"sample\_positive\_docs":\[

\{

"doc\_id":"Back\_in\_the\_USA\_\_a505370ce874",

"filename":"Back\_in\_the\_USA\.txt"

\}

\],

"sample\_negative\_docs":\[

\{

"doc\_id":"11\_Harrowhouse\_\_08cc6c95c2f9",

"filename":"11\_Harrowhouse\.txt"

\},

\{

"doc\_id":"1971\_Copa\_Libertadores\_\_8369d1b96cbe",

"filename":"1971\_Copa\_Libertadores\.txt"

\},

\{

"doc\_id":"Khalid\_Abdel\_Nasser\_\_b9f61f804f4b",

"filename":"Khalid\_Abdel\_Nasser\.txt"

\},

\{

"doc\_id":"Khalid\_al\-Habib\_\_ca209876aaad",

"filename":"Khalid\_al\-Habib\.txt"

\},

\{

"doc\_id":"\\u021aerova\_\_10c94b229856",

"filename":"\\u021aerova\.txt"

\},

\{

"doc\_id":"\\u02bbElisiva\_Fusipala\_Tauki\\u02bbonetuku\_\_3e99f95742e5",

"filename":"\\u02bbElisiva\_Fusipala\_Tauki\\u02bbonetuku\.txt"

\}

\],

"scope\_check":"apply\_filtersearchesDOCUMENTTEXT\.1docsmatchedintext,but0filenamesmatchthesamepattern\.Ifyouintendedtofilterbyfilename/date,usesearch\_filenamesinstead\.",

"negative\_entity\_check":"145/6117negativescontain’Back’\(keywordonly\)\."

\}

\[Auto\-sampledsnippets\(positive\)\-\-\-3of1docs,explorefurtherifneeded\]

\[Back\_in\_the\_USA\.txt\]BackintheU\.S\.A\.

"BackintheU\.S\.A\."isasongwrittenbyChuckBerrythatwasreleasedin1959andwasatop40hit\.Acoverversionin1978byLindaRonstadtwasalsoahit\.

\[Auto\-sampledsnippets\(negative\)\-\-\-3of6117docs,explorefurtherifneeded\]

\[11\_Harrowhouse\.txt\]11Harrowhouse

11Harrowhouseisa1974BritishDeLuxeColorfilmdirectedbyAramAvakianandstarringCharlesGrodin,CandiceBergen,JamesMason,TrevorHowardandJohnGielgudinPanavision\.ItwasadaptedbyCharlesGrodinbaseduponthenovelbyGeraldA\.BrownewiththescreenplaybyJeffreyBloom\.

\[Khalid\_al\-Habib\.txt\]Khalidal\-Habib

Khalidal\-HabibalsobelievedtousethenomdeguerreKhalidalHarbiisanAl\-QaedaleaderwhohasoverseenAl\-QaedaoperationsinAfghanistan\.In2005itwasreportedthatal\-HabibdiedinaUSmissilestrikeonthePakistanivillageofDamadolathoughitisunclearwhetherthisisthecaseasPakistaniofficialshaveclaimedthatnoAl\-Qaedaleadersdiedintheattack\.al\-Habib’snationalityisnotclearwithsourcesclaiminghemaybeEgyptianorMoroccan\.

\[’Elisiva\_Fusipala\_Tauki’onetuku\.txt\]’ElisivaFusipalaTauki’onetuku

’ElisivaFusipalaTauki’onetuku\(18May1850\-\-September1889\)wasthemotherofKingGeorgeTupouII\.

===Automaticallycomputed===

Filenamesearchcounts:

search\_filenames\("Back"\)→\\rightarrow11docs

Setcompositions\-\-\-composer\_and\_Back\_pos\(7\)×\\timeshas\_BackInUSA\_pos\(1\):

intersect→\\rightarrow0

union→\\rightarrow8

composer\_and\_Back\_pos\\has\_BackInUSA\_pos→\\rightarrow7

has\_BackInUSA\_pos\\composer\_and\_Back\_pos→\\rightarrow1

Step3:apply\_filter\(\{

"filter\_type":"regex",

"pattern":"ChuckBerry",

"target\_set":"all",

"output\_prefix":"has\_ChuckBerry"

\}\)

Agentthought:Thedocument’Back\_in\_the\_USA\.txt’states:’BackintheU\.S\.A\.’isasongwrittenbyChuckBerry\.Thisanswerssub\-question1:thecomposerisChuckBerry\.Now,IneedtofindwhereChuckBerrywasborn\.Iwillsearchfor’ChuckBerry’and’born’tofindhisbirthplace\.

Observation:\{

"positive\_set":"has\_ChuckBerry\_pos",

"positive\_count":3,

"negative\_set":"has\_ChuckBerry\_neg",

"negative\_count":6115,

"total\_filtered":6118,

"sample\_positive\_docs":\[

\{

"doc\_id":"Alan\_Reeves\_composer\_\_89c0d8e6240e",

"filename":"Alan\_Reeves\_composer\.txt"

\},

\{

"doc\_id":"Back\_in\_the\_USA\_\_a505370ce874",

"filename":"Back\_in\_the\_USA\.txt"

\},

\{

"doc\_id":"Chuck\_Berry\_\_ff7ec8ef49a2",

"filename":"Chuck\_Berry\.txt"

\}

\],

"sample\_negative\_docs":\[

\{

"doc\_id":"11\_Harrowhouse\_\_08cc6c95c2f9",

"filename":"11\_Harrowhouse\.txt"

\},

\{

"doc\_id":"1971\_Copa\_Libertadores\_\_8369d1b96cbe",

"filename":"1971\_Copa\_Libertadores\.txt"

\},

\{

"doc\_id":"Khalid\_al\-Habib\_\_ca209876aaad",

"filename":"Khalid\_al\-Habib\.txt"

\},

\{

"doc\_id":"Khalid\_bin\_Abdullah\_bin\_Abdulaziz\_Al\_Saud\_\_

b63dd78ee252",

"filename":"Khalid\_bin\_Abdullah\_bin\_Abdulaziz\_Al\_Saud\.txt"

\},

\{

"doc\_id":"\\u021aerova\_\_10c94b229856",

"filename":"\\u021aerova\.txt"

\},

\{

"doc\_id":"\\u02bbElisiva\_Fusipala\_Tauki\\u02bbonetuku\_\_3e99f95742e5",

"filename":"\\u02bbElisiva\_Fusipala\_Tauki\\u02bbonetuku\.txt"

\}

\],

"scope\_check":"apply\_filtersearchesDOCUMENTTEXT\.3docsmatchedintext,but0filenamesmatchthesamepattern\.Ifyouintendedtofilterbyfilename/date,usesearch\_filenamesinstead\.",

"keyword\_decomposition":\[

\{

"keyword":"Chuck",

"matches":10,

"pct":0\.2

\},

\{

"keyword":"Berry",

"matches":21,

"pct":0\.3

\}

\],

"selectivity\_note":"Combinedpatternmatched3;individualkeywordsmatch10\\u201321each\.Considerstartingwithonekeyword\.",

"keyword\_intersection":\{

"keywords":\[

"Chuck",

"Berry"

\],

"both\_match":3

\},

"negative\_entity\_check":"144/6115negativescontain’Back’\(keywordonly\)\."

\}

\[Auto\-sampledsnippets\(positive\)\-\-\-3of3docs,explorefurtherifneeded\]

\[Alan\_Reeves\_composer\.txt\]AlanReeves\(composer\)

AlanDavidReevesisaBritishfilmcomposer,musicproducer,andHammondB3virtuoso\.Inthecourseofhiscareerhehasreceived35internationalawards,includingaGoldeneSchallplatte\.HebecameknownforhisworkwiththebandsTheShowtimersandClinicaswellasforthemusicforthefilmsToWalkwithLions,,andKillBillVol\.2\.Hehasappearedplayedorrecordedwith/foramongothers,theRollingStones,JimiHendrix,ChuckBerryandDavidBowieandDavidGilmore

\[Back\_in\_the\_USA\.txt\]BackintheU\.S\.A\.

"BackintheU\.S\.A\."isasongwrittenbyChuckBerrythatwasreleasedin1959andwasatop40hit\.Acoverversionin1978byLindaRonstadtwasalsoahit\.

\[Chuck\_Berry\.txt\]ChuckBerry

CharlesEdwardAndersonBerry\(October18,1926\-\-March18,2017\)wasanAmericansingerandsongwriter,andoneofthepioneersofrockandrollmusic\.Nicknamedthe"FatherofRockandRoll",Berryrefinedanddevelopedrhythmandbluesintothemajorelementsthatmaderockandrolldistinctivewithsongssuchas"Maybellene"\(1955\),"RollOverBeethoven"\(1956\),"RockandRollMusic"\(1957\)and"JohnnyB\.Goode"\(1958\)\.Writinglyricsthatfocusedonteenlifeandconsumerism,anddevelopingamusicstylethatincludedguitarsolosandshowmanship,Berrywasamajorinfluenceonsubsequentrockmusic\.Bornintoamiddle\-classAfrican\-AmericanfamilyinSt\.Louis,Missouri,BerryhadaninterestinmusicfromanearlyageandgavehisfirstpublicperformanceatSumnerHig\.\.\.

\[Auto\-sampledsnippets\(negative\)\-\-\-3of6115docs,explorefurtherifneeded\]

\[11\_Harrowhouse\.txt\]11Harrowhouse

11Harrowhouseisa1974BritishDeLuxeColorfilmdirectedbyAramAvakianandstarringCharlesGrodin,CandiceBergen,JamesMason,TrevorHowardandJohnGielgudinPanavision\.ItwasadaptedbyCharlesGrodinbaseduponthenovelbyGeraldA\.BrownewiththescreenplaybyJeffreyBloom\.

\[Khalid\_bin\_Abdullah\_bin\_Abdulaziz\_Al\_Saud\.txt\]KhalidbinAbdullahbinAbdulazizAlSaud

KhalidbinAbdullah\(born1950,Riyadh\)istheeldestchildofKingAbdullahofSaudiArabia\.HeisamemberoftheAllegianceCouncil\.

\[’Elisiva\_Fusipala\_Tauki’onetuku\.txt\]’ElisivaFusipalaTauki’onetuku

’ElisivaFusipalaTauki’onetuku\(18May1850\-\-September1889\)wasthemotherofKingGeorgeTupouII\.

\[Contextaudit\-\-\-’Back’in2of3positivedocs:\]

BackintheU\.S\.A\."BackintheU\.S\.A\."is\.\.\.

\.\.\.ormer,playinghispasthitswithlocalbackupbandsofvariablequality\.However,i\.\.\.

===Automaticallycomputed===

Filenamesearchcounts:

search\_filenames\("Chuck"\)→\\rightarrow2docs

search\_filenames\("Berry"\)→\\rightarrow3docs

Setcompositions\-\-\-has\_BackInUSA\_pos\(1\)×\\timeshas\_ChuckBerry\_pos\(3\):

intersect→\\rightarrow1

union→\\rightarrow3

has\_BackInUSA\_pos\\has\_ChuckBerry\_pos→\\rightarrow0

has\_ChuckBerry\_pos\\has\_BackInUSA\_pos→\\rightarrow2

### C\.3Zero\-Result Diagnostics and Recovery: 2WikiMultiHopQA

Here the agent’s initial detention\-related regexes yield zero results within its already\-narrowed working set\. Thezero\_result\_hintdiagnostic prompts it to try broader lexical variants \(e\.g\.,detain→\\rightarrowarrest→\\rightarrowprison\|jail\), and the agent recovers without widening the search back to the full corpus\.

Question:WherewastheperformerofsongBBoy\(Song\)detained?

Goldenanswer:StateCorrectionalInstitution\-\-Chester

Agentanswer:StateCorrectionalInstitution\-\-ChesterinChester,Pennsylvania\.

Step0:apply\_filter\(\{

"filter\_type":"regex",

"pattern":"BBoy",

"target\_set":"all",

"output\_prefix":"has\_b\_boy"

\}\)

Agentthought:Thisisamulti\-hopquestion:\(1\)Identifytheperformerof’BBoy\(Song\)’;\(2\)Determinewherethatperformerwasdetained\.Iwillstartbysearchingfor’BBoy’toidentifytheperformer,thensearchfordetentioninformationaboutthatperson\.

Observation:\{

"positive\_set":"has\_b\_boy\_pos",

"positive\_count":1,

"negative\_set":"has\_b\_boy\_neg",

"negative\_count":6117,

"total\_filtered":6118,

"sample\_positive\_docs":\[

\{

"doc\_id":"B\_Boy\_song\_\_638a840ec141",

"filename":"B\_Boy\_song\.txt"

\}

\],

"sample\_negative\_docs":\[

\{

"doc\_id":"11\_Harrowhouse\_\_08cc6c95c2f9",

"filename":"11\_Harrowhouse\.txt"

\},

\{

"doc\_id":"1971\_Copa\_Libertadores\_\_8369d1b96cbe",

"filename":"1971\_Copa\_Libertadores\.txt"

\},

\{

"doc\_id":"Khalid\_Abdel\_Nasser\_\_b9f61f804f4b",

"filename":"Khalid\_Abdel\_Nasser\.txt"

\},

\{

"doc\_id":"Khalid\_al\-Habib\_\_ca209876aaad",

"filename":"Khalid\_al\-Habib\.txt"

\},

\{

"doc\_id":"\\u021aerova\_\_10c94b229856",

"filename":"\\u021aerova\.txt"

\},

\{

"doc\_id":"\\u02bbElisiva\_Fusipala\_Tauki\\u02bbonetuku\_\_3e99f95742e5",

"filename":"\\u02bbElisiva\_Fusipala\_Tauki\\u02bbonetuku\.txt"

\}

\],

"scope\_check":"apply\_filtersearchesDOCUMENTTEXT\.1docsmatchedintext,but0filenamesmatchthesamepattern\.Ifyouintendedtofilterbyfilename/date,usesearch\_filenamesinstead\.",

"negative\_entity\_check":"279/6117negativescontain’Where’\(keywordonly\)\."

\}

\[Auto\-sampledsnippets\(positive\)\-\-\-3of1docs,explorefurtherifneeded\]

\[B\_Boy\_song\.txt\]BBoy\(song\)

"BBoy"isasongbyAmericanhiphoprecordingartistMeekMill\.Itwasreleasedasathirdsinglefrom"DreamsWorthMoreThanMoney"onJanuary3,2015,byMaybachMusicGroupandAtlanticRecords\.Thesong,whichwasproducedbySap,featuresguestappearancesfromBigSeanandASAPFerg\.

\[Auto\-sampledsnippets\(negative\)\-\-\-3of6117docs,explorefurtherifneeded\]

\[11\_Harrowhouse\.txt\]11Harrowhouse

11Harrowhouseisa1974BritishDeLuxeColorfilmdirectedbyAramAvakianandstarringCharlesGrodin,CandiceBergen,JamesMason,TrevorHowardandJohnGielgudinPanavision\.ItwasadaptedbyCharlesGrodinbaseduponthenovelbyGeraldA\.BrownewiththescreenplaybyJeffreyBloom\.

\[Khalid\_al\-Habib\.txt\]Khalidal\-Habib

Khalidal\-HabibalsobelievedtousethenomdeguerreKhalidalHarbiisanAl\-QaedaleaderwhohasoverseenAl\-QaedaoperationsinAfghanistan\.In2005itwasreportedthatal\-HabibdiedinaUSmissilestrikeonthePakistanivillageofDamadolathoughitisunclearwhetherthisisthecaseasPakistaniofficialshaveclaimedthatnoAl\-Qaedaleadersdiedintheattack\.al\-Habib’snationalityisnotclearwithsourcesclaiminghemaybeEgyptianorMoroccan\.

\[’Elisiva\_Fusipala\_Tauki’onetuku\.txt\]’ElisivaFusipalaTauki’onetuku

’ElisivaFusipalaTauki’onetuku\(18May1850\-\-September1889\)wasthemotherofKingGeorgeTupouII\.

Step1:apply\_filter\(\{

"filter\_type":"regex",

"pattern":"MeekMill",

"target\_set":"all",

"output\_prefix":"has\_meek\_mill"

\}\)

Agentthought:Theperformerof’BBoy\(song\)’isMeekMill,withguestappearancesfromBigSeanandASAPFerg\.Thequestionaskswheretheperformerwasdetained\.IwillnowsearchforinformationaboutMeekMillbeingdetained,focusingonlocation\.Iwillstartbysearchingfor’MeekMill’inthecorpus\.

Observation:\{

"positive\_set":"has\_meek\_mill\_pos",

"positive\_count":2,

"negative\_set":"has\_meek\_mill\_neg",

"negative\_count":6116,

"total\_filtered":6118,

"sample\_positive\_docs":\[

\{

"doc\_id":"B\_Boy\_song\_\_638a840ec141",

"filename":"B\_Boy\_song\.txt"

\},

\{

"doc\_id":"Meek\_Mill\_\_52e01b469e2a",

"filename":"Meek\_Mill\.txt"

\}

\],

"sample\_negative\_docs":\[

\{

"doc\_id":"11\_Harrowhouse\_\_08cc6c95c2f9",

"filename":"11\_Harrowhouse\.txt"

\},

\{

"doc\_id":"1971\_Copa\_Libertadores\_\_8369d1b96cbe",

"filename":"1971\_Copa\_Libertadores\.txt"

\},

\{

"doc\_id":"Khalid\_Abdel\_Nasser\_\_b9f61f804f4b",

"filename":"Khalid\_Abdel\_Nasser\.txt"

\},

\{

"doc\_id":"Khalid\_al\-Habib\_\_ca209876aaad",

"filename":"Khalid\_al\-Habib\.txt"

\},

\{

"doc\_id":"\\u021aerova\_\_10c94b229856",

"filename":"\\u021aerova\.txt"

\},

\{

"doc\_id":"\\u02bbElisiva\_Fusipala\_Tauki\\u02bbonetuku\_\_3e99f95742e5",

"filename":"\\u02bbElisiva\_Fusipala\_Tauki\\u02bbonetuku\.txt"

\}

\],

"scope\_check":"apply\_filtersearchesDOCUMENTTEXT\.2docsmatchedintext,but0filenamesmatchthesamepattern\.Ifyouintendedtofilterbyfilename/date,usesearch\_filenamesinstead\.",

"keyword\_decomposition":\[

\{

"keyword":"Meek",

"matches":3,

"pct":0\.0

\},

\{

"keyword":"Mill",

"matches":157,

"pct":2\.6

\}

\],

"selectivity\_note":"Combinedpatternmatched2;individualkeywordsmatch3\\u2013157each\.Considerstartingwithonekeyword\.",

"keyword\_intersection":\{

"keywords":\[

"Meek",

"Mill"

\],

"both\_match":2

\},

"negative\_entity\_check":"279/6116negativescontain’Where’\(keywordonly\)\."

\}

\[Auto\-sampledsnippets\(positive\)\-\-\-3of2docs,explorefurtherifneeded\]

\[B\_Boy\_song\.txt\]BBoy\(song\)

"BBoy"isasongbyAmericanhiphoprecordingartistMeekMill\.Itwasreleasedasathirdsinglefrom"DreamsWorthMoreThanMoney"onJanuary3,2015,byMaybachMusicGroupandAtlanticRecords\.Thesong,whichwasproducedbySap,featuresguestappearancesfromBigSeanandASAPFerg\.

\[Meek\_Mill\.txt\]MeekMill

RobertRihmeekWilliams\(bornMay6,1987\),knownprofessionallyasMeekMill,isanAmericanrapper,songwriter,andactivist\.BornandraisedinPhiladelphia,heembarkedonhismusiccareerasabattlerapper,andlaterformedashort\-livedrapgroup,TheBloodhoundz\.In2008,Atlanta\-basedrapperT\.I\.signedMeekMilltohisfirstrecorddeal\.InFebruary2011,afterleavingGrandHustleRecords,MillsignedwithMiami\-basedrapperRickRoss’sMaybachMusicGroup\(MMG\)\.Mill’sdebutalbum,"DreamsandNightmares",wasreleasedin2012underMMGandWarnerBros\.Records\.Thealbum,precededbytheleadsingle"Amen"\(featuringDrake\),debutedatnumbertwoontheU\.S\."Billboard"200\.InOctober2012,Millannouncedthelaunchofhisownlabelimprint,DreamChasersRecords,\.\.\.

\[Auto\-sampledsnippets\(negative\)\-\-\-3of6116docs,explorefurtherifneeded\]

\[11\_Harrowhouse\.txt\]11Harrowhouse

11Harrowhouseisa1974BritishDeLuxeColorfilmdirectedbyAramAvakianandstarringCharlesGrodin,CandiceBergen,JamesMason,TrevorHowardandJohnGielgudinPanavision\.ItwasadaptedbyCharlesGrodinbaseduponthenovelbyGeraldA\.BrownewiththescreenplaybyJeffreyBloom\.

\[Khalid\_al\-Habib\.txt\]Khalidal\-Habib

Khalidal\-HabibalsobelievedtousethenomdeguerreKhalidalHarbiisanAl\-QaedaleaderwhohasoverseenAl\-QaedaoperationsinAfghanistan\.In2005itwasreportedthatal\-HabibdiedinaUSmissilestrikeonthePakistanivillageofDamadolathoughitisunclearwhetherthisisthecaseasPakistaniofficialshaveclaimedthatnoAl\-Qaedaleadersdiedintheattack\.al\-Habib’snationalityisnotclearwithsourcesclaiminghemaybeEgyptianorMoroccan\.

\[’Elisiva\_Fusipala\_Tauki’onetuku\.txt\]’ElisivaFusipalaTauki’onetuku

’ElisivaFusipalaTauki’onetuku\(18May1850\-\-September1889\)wasthemotherofKingGeorgeTupouII\.

===Automaticallycomputed===

Filenamesearchcounts:

search\_filenames\("Meek"\)→\\rightarrow1docs

search\_filenames\("Mill"\)→\\rightarrow12docs

Setcompositions\-\-\-has\_b\_boy\_pos\(1\)×\\timeshas\_meek\_mill\_pos\(2\):

intersect→\\rightarrow1

union→\\rightarrow2

has\_b\_boy\_pos\\has\_meek\_mill\_pos→\\rightarrow0

has\_meek\_mill\_pos\\has\_b\_boy\_pos→\\rightarrow1

Step2:apply\_filter\(\{

"filter\_type":"regex",

"pattern":"detain",

"target\_set":"has\_meek\_mill\_pos",

"output\_prefix":"meek\_mill\_detain"

\}\)

Agentthought:Theperformerof’BBoy\(song\)’isMeekMill,asconfirmedintheB\_Boy\_song\.txtandMeek\_Mill\.txtdocuments\.Now,IneedtodeterminewhereMeekMillwasdetained\.Iwillsearchfortheterm’detain’\(tocover’detained’,’detainment’,etc\.\)intheMeek\_Mill\.txtdocumenttofindanymentionofhisdetentionanditslocation\.

Observation:\{

"positive\_set":"meek\_mill\_detain\_pos",

"positive\_count":0,

"negative\_set":"meek\_mill\_detain\_neg",

"negative\_count":2,

"total\_filtered":2,

"sample\_positive\_docs":\[\],

"sample\_negative\_docs":\[

\{

"doc\_id":"B\_Boy\_song\_\_638a840ec141",

"filename":"B\_Boy\_song\.txt"

\},

\{

"doc\_id":"Meek\_Mill\_\_52e01b469e2a",

"filename":"Meek\_Mill\.txt"

\}

\],

"zero\_result\_hint":"Zeromatcheson2docs\.Try:\(1\)asimpler/shorterregex,\(2\)search\_filenamesiftheconditionisaboutdocumentnames\."

\}

===Automaticallycomputed===

Setcompositions\-\-\-has\_meek\_mill\_pos\(2\)×\\timesmeek\_mill\_detain\_pos\(0\):

intersect→\\rightarrow0

union→\\rightarrow2

has\_meek\_mill\_pos\\meek\_mill\_detain\_pos→\\rightarrow2

meek\_mill\_detain\_pos\\has\_meek\_mill\_pos→\\rightarrow0

Step3:apply\_filter\(\{

"filter\_type":"regex",

"pattern":"arrest",

"target\_set":"has\_meek\_mill\_pos",

"output\_prefix":"meek\_mill\_arrest"

\}\)

Agentthought:Nomatchesfor’detain’oritsvariantsintheMeekMill\-relateddocuments\.Toensurecoverage,Iwilltrysearchingfor’arrest’\(acommonsynonymfor’detain’\)inthesameset,asthecorpusmayusedifferentterminology\.

Observation:\{

"positive\_set":"meek\_mill\_arrest\_pos",

"positive\_count":0,

"negative\_set":"meek\_mill\_arrest\_neg",

"negative\_count":2,

"total\_filtered":2,

"sample\_positive\_docs":\[\],

"sample\_negative\_docs":\[

\{

"doc\_id":"B\_Boy\_song\_\_638a840ec141",

"filename":"B\_Boy\_song\.txt"

\},

\{

"doc\_id":"Meek\_Mill\_\_52e01b469e2a",

"filename":"Meek\_Mill\.txt"

\}

\],

"zero\_result\_hint":"Zeromatcheson2docs\.Try:\(1\)asimpler/shorterregex,\(2\)search\_filenamesiftheconditionisaboutdocumentnames\."

\}

Step4:apply\_filter\(\{

"filter\_type":"regex",

"pattern":"prison\|jail",

"target\_set":"has\_meek\_mill\_pos",

"output\_prefix":"meek\_mill\_prison\_jail"

\}\)

Agentthought:Nomatchesfor’arrest’or’detain’intheMeekMilldocuments\.ToensureIamnotmissingalternateterminology,Iwillsearchfor’prison’and’jail’intheMeekMill\-relateddocuments,asthesearecommontermsusedtodescribewheresomeonewasdetained\.

Observation:\{

"positive\_set":"meek\_mill\_prison\_jail\_pos",

"positive\_count":1,

"negative\_set":"meek\_mill\_prison\_jail\_neg",

"negative\_count":1,

"total\_filtered":2,

"sample\_positive\_docs":\[

\{

"doc\_id":"Meek\_Mill\_\_52e01b469e2a",

"filename":"Meek\_Mill\.txt"

\}

\],

"sample\_negative\_docs":\[

\{

"doc\_id":"B\_Boy\_song\_\_638a840ec141",

"filename":"B\_Boy\_song\.txt"

\}

\],

"scope\_check":"apply\_filtersearchesDOCUMENTTEXT\.1docsmatchedintext,but0filenamesmatchthesamepattern\.Ifyouintendedtofilterbyfilename/date,usesearch\_filenamesinstead\.",

"keyword\_decomposition":\[

\{

"keyword":"prison",

"matches":1,

"pct":50\.0

\},

\{

"keyword":"jail",

"matches":0,

"pct":0\.0

\}

\]

\}

\[Auto\-sampledsnippets\(positive\)\-\-\-3of1docs,explorefurtherifneeded\]

\[Meek\_Mill\.txt\]\.\.\.and"ImaBoss",beingincludedonvolumeone\.Hereleasedhissecondalbum,"DreamsWorthMoreThanMoney",in2015andhisthirdalbum,"Wins&Losses",in2016\.Hisfourthstudioalbum,"Championships",wasreleasedinNovember2018anddebutedatnumberoneonthe"Billboard"200\.Itsleadsingle,"GoingBad"\(featuringDrake\),peakedatnumber6onthe"Billboard"Hot100\.InNovember2017,hewassentencedtotwotofouryearsinprisonforviolatingparoleandservedfivemonthsattheStateCorrectionalInstitution\-\-ChesterinChester,Pennsylvania,beforebeingreleasedwhilehistrialcontinues\.InAugust2019,adocumentaryseriesabouthisbattlewiththecriminaljusticesystem,"FreeMeek",wasreleasedonAmazon’sPrimeVideo\.Millservedasexecutiveproducerontheseries\.

\[Auto\-sampledsnippets\(negative\)\-\-\-3of1docs,explorefurtherifneeded\]

\[B\_Boy\_song\.txt\]BBoy\(song\)

"BBoy"isasongbyAmericanhiphoprecordingartistMeekMill\.Itwasreleasedasathirdsinglefrom"DreamsWorthMoreThanMoney"onJanuary3,2015,byMaybachMusicGroupandAtlanticRecords\.Thesong,whichwasproducedbySap,featuresguestappearancesfromBigSeanandASAPFerg\.

===Automaticallycomputed===

Filenamesearchcounts:

search\_filenames\("prison"\)→\\rightarrow3docs

Setcompositions\-\-\-meek\_mill\_arrest\_pos\(0\)×\\timesmeek\_mill\_prison\_jail\_pos\(1\):

intersect→\\rightarrow0

union→\\rightarrow1

meek\_mill\_arrest\_pos\\meek\_mill\_prison\_jail\_pos→\\rightarrow0

meek\_mill\_prison\_jail\_pos\\meek\_mill\_arrest\_pos→\\rightarrow1

## Appendix DTranscripts Dataset Construction

This appendix describes the construction of the Transcripts corpus and question set used in our experiments \(Section[4](https://arxiv.org/html/2607.25135#S4)\)\. The goal is to produce questions where \(i\) all gold answers are*derivable from the corpus text*, \(ii\) a substantial fraction of questions require*corpus\-wide aggregation*that a top\-kkretrieval system structurally cannot answer, and \(iii\) the questions are phrased as a domain analyst would naturally phrase them, with no leading references to formatting features\.

##### Source data\.

The corpus is drawn from the U\.S\. Government Publishing Office \(GPO\) collection of 117th\-Congress hearing transcripts, distributed publicly viagovinfo\.gov\. We sample 75 transcripts, balanced across the House \(39\), the Senate \(34\), and joint House–Senate sessions \(2\), and spanning calendar 2021–2022\. Each transcript is converted to UTF\-8 plain text from the GPO HTML release; document filenames follow GPO’s package\-id convention:CHRG\-117hhrg…\.txtfor House hearings,CHRG\-117shrg…\.txtfor Senate hearings, andCHRG\-117jhrg…\.txtfor joint House\-Senate sessions \(chamber is encoded by thehhrg/shrg/jhrgsubstring\)\.

##### Structural features used as gold\.

CHRG transcripts have a strongly conventional structure that makes deterministic gold construction tractable\. We rely on two families of features:

1. 1\.Cover\-page metadata\.The first∼\\sim160 lines of each transcript contain the chamber identifier \(e\.g\.,U\.S\. HOUSE OF REPRESENTATIVES,UNITED STATES SENATE, or aJOINT HEARINGmarker\), the main committee heading \(COMMITTEE ON …\), an optional subcommittee heading \(SUBCOMMITTEE ON …\), the hearing date in the formWeekday, Month Day, Year, and the hearing title\. A parser extracts these four fields per document\.
2. 2\.Labeled\-field markers in the body\.CHRG transcripts use a small set of conventional line\-anchored labels that recur across hearings:Present:andMembers present:\(formal attendance roll\);Staff Present:;Also present:\(visiting members\);Available via the World Wide Web:\(publication URL\); the closing parliamentary notation\[Whereupon, at HH:MM …\]; andResponses to written questions of …\(post\-hearing QFR section header\)\. Each marker is detected by a line\-anchored regex over the full transcript text; coverage counts \(e\.g\., 50/75 transcripts contain an attendance roll underPresent:∪\\cupMembers present:; 26/75 adjourned during the 11 AM hour\) are taken directly from these regex hits\.

##### Question types\.

All questions and gold answers are produced deterministically from the parsed corpus by a generation script \(we release the script and the independent validator\)\. No hand\-editing is applied\. The resulting evaluation file has 100 questions across four groups:

1. 1\.Single\-document lookups\(69/100\): per\-hearing questions about the committee, the chamber, the hearing date, and the number of witnesses listed in the Contents block\.
2. 2\.Chamber\-level aggregations\(6/100\): corpus\-wide counts and comparisons across chambers, e\.g\., the total number of House versus Senate hearings, the size of the House–Senate difference, and which chamber predominates corpus\-wide or within a year\.
3. 3\.Chamber×\\timesyear intersections and year totals\(3/100\): e\.g\., how many House hearings occurred in 2022; the total number of hearings in a given year\.
4. 4\.Labeled\-field 3\-way concept\-majority questions\(22/100\): given three corpus\-level features \(e\.g\., an attendance roll, a published online transcript link, and an 11 AM adjournment notation\), the question asks which of the three appears in the most hearings; chamber\-restricted variants ask the same within House\- or Senate\-only subsets\.

##### Natural\-language phrasing without leading hints\.

Both the question text and the gold answers are phrased in natural language, without quoted regex markers or formatting hints\. The two semantically equivalent attendance markers \(Present:andMembers present:\) are merged into a single concept so that a natural reading of the question is unambiguous\.

## Appendix ELLM\-as\-a\-Judge Details

We evaluate all systems using the same LLM\-as\-a\-judge pipeline to avoid system\-specific evaluators\. For each question, we run two judge\-model calls:

1. 1\.Answer\-only extraction:transform a possibly verbose model output into a single “final answer” string \(orINSUFFICIENT\_EVIDENCE\)\.
2. 2\.Judging:compare the extracted final answer against the gold answer and return a JSON verdict and score\.

Prompt for answer\-only extraction \(reasoning removal\):

YouareaQAassistant\.

Youwillbegiven:

\-thequestion

\-amodelanswerthatmaycontainreasoning

Yourtask:

\-OutputONLYthefinalanswertothequestion\.

\-DoNOTincludeanyreasoning,explanation,preamble,orextratext\.

\-DoNOTusemarkdowncodefences\.

Iftheanswercannotbedeterminedfromtheprovidedmodelanswer,outputexactly:

INSUFFICIENT\_EVIDENCE

SQuAD\-style exact\-match and token\-overlap F1 are computed on the output of the above normalization\.

Prompt for judging rules and scoring:

YouareanaccurateevaluatorforQA\.

Youwillbegiven:

\-thequestion

\-thegoldanswer

\-themodelanswer\(finalansweronly;noreasoning\)

Decideifthemodelanswermatchesthegoldanswer\.

Bestrictaboutfactualcorrectness,butallowparaphrasesandequivalentnumericformats\.

Ifthegoldanswerisempty,markasincorrectunlessthemodelexplicitlysaysitcannotbedetermined\(e\.g\.,INSUFFICIENT\_EVIDENCE\)\.

INSUFFICIENT\_EVIDENCErule:

\-Ifthemodelanswerisexactlythestring"INSUFFICIENT\_EVIDENCE"ANDthegoldanswerisnon\-empty,verdictMUSTbe"incorrect"andscoreMUSTbe0\.

Specialscoringrules:

1\)Countquestions:

\-Ifthequestionisaskingforacount/number,theanswerisONLYcorrectifthepredictednumberexactlymatchesthegoldnumber\.

\-Ifitisoffbyeven1,scoreMUSTbe0andverdictMUSTbe"incorrect"\.

2\)List/setquestions:

\-Treatthegoldanswerandmodelanswerassetsofitems\.

\-LetLbethenumberofdistinctgolditems\.

\-Startwithscore=1\.0\.

\-Applyapenaltyof\(1/L\)foreachmissinggolditemANDforeachextrapredicteditem\.

\-Treatpossiblealiasesascorrect\.Donotgivepartialscoreperelement\.

\-Scoreisflooredat0\.

\-Ifthemodelincludesadditionalexplanatorytext,ignoreitandfocusontheitemset\.

Whentheanswerisapropername/namedentity:

\-Treatclearaliasesoralternativespellingsascorrect\(score=1\)\.

\-Ifthemodelanswerisambiguous\(couldrefertomultipleentities\)butnotclearlywrong,allowpartialcreditwithashortrationale\.

\-Givescore=0onlyifitisclearlythewrongentity\.

ReturnaJSONobjectwith:

\-verdict:oneof\["correct","incorrect","partial"\]

\-score:numberin\[0,1\]wherecorrect=1,incorrect=0,partialisbetween

\-rationale:shortexplanation\(1\-3sentences\)

\-abs\_diff:OPTIONAL\.IncludeONLYforcount/numberquestions\.Itmustbeanon\-negativeintegerequalto\|gold\_number\-predicted\_number\|ifyoucanextractbothnumbers;otherwisenull\.

Rules:

\-OutputmustbevalidJSON\(nomarkdownfences,noextratext\)\.