Beyond Retrieval Failure: Understanding Why LLMs Miss Evidence in Multi-Hop RAG

# Beyond Retrieval Failure: Understanding Why LLMs Miss Evidence in Multi-Hop RAG
Source: [https://arxiv.org/html/2610.21184](https://arxiv.org/html/2610.21184)
Conference:The Eighth International Conference on Distributed Artificial Intelligence; November 29–December 2, 2026; Hong Kong, ChinaThe Eighth International Conference on Distributed Artificial Intelligence \(DAI ’26\), November 29–December 2, 2026, Hong Kong, ChinaCCS:Information systems Retrieval models and rankingCCS:Information systems Question answeringCCS:Computing methodologies Natural language processingJiaqi WeiNote:Both authors contributed equally to this research\.Affiliation:Zhejiang University,Hangzhou,ChinaAffiliation:Shanghai AI Laboratory,Shanghai,Chinaemail:[jiaqi\.wei@zju\.edu\.cn](mailto:[email protected])Wenxuan HuangAffiliation:Fudan University,Shanghai,Chinaemail:[winshion0819@gmail\.com](mailto:[email protected]),Chengdong XuAffiliation:Sun Yat\-sen University,Guangzhou,Chinaemail:[codepoet2077@gmail\.com](mailto:[email protected]),Xiang ZhangAffiliation:University of British Columbia,Vancouver,Canadaemail:[xzhang23@ualberta\.ca](mailto:[email protected])andWanli OuyangAffiliation:The Chinese University of Hong Kong,Hong Kong SAR,Chinaemail:[wlouyang@ie\.cuhk\.edu\.hk](mailto:[email protected])

© ,

###### Abstract\.

Multi\-hop retrieval\-augmented generation \(RAG\) requires an agent to retrieve evidence for a sequence of dependent sub\-questions\. When a wrong answer lacks a required passage, the passage may be inaccessible to the retriever, or the agent may have failed to issue a query that would retrieve it\. Standard retrieval evaluations conflate these two causes\. We study how much apparent retrieval failure is instead caused by*search drift*: the search no longer tracks an unanswered sub\-question\. Holding the retriever, corpus, and generation model fixed, we introduce ExecutionRAG, a training\-free requirement register that records open sub\-questions and their supporting evidence\. The register changes search guidance but not evidence access, so each gold passage it recovers certifies that the passage was reachable by the original retriever\. On 1,000 MuSiQue questions, the number of wrong answers with unretrieved required evidence falls from 382 to 234 with all\-MiniLM\-L6\-v2 and from 383 to 274 with Qwen3\-Embedding\-0\.6B\. These counts correspond to 69% to 44% and 66% to 48% of each system’s wrong answers, respectively\. Crucially, evidence recovery does not produce a proportional reduction in total error: premature resolutions and abstentions increase as missing\-evidence failures decrease\. The central finding is therefore a shift in the remaining bottleneck, from guiding search toward required evidence to verifying that the retrieved evidence is sufficient\.

###### Keywords:

Retrieval\-augmented generation, multi\-hop question answering, agentic retrieval, search drift, failure analysis

## 1\.Introduction

Retrieval\-augmented generation \(RAG\)\([Lewis et al\., 2020](https://arxiv.org/html/2610.21184#bib.bib21);[Gao et al\., 2023](https://arxiv.org/html/2610.21184#bib.bib8)\)grounds language\-model answers in retrieved evidence\. This is especially important for multi\-hop questions, whose answers depend on several documents\. Consider the question*“Who married the publisher of the abolitionist newspaper The North Star?”*Answering it requires two linked facts: Frederick Douglass published*The North Star*, and Helen Pitts Douglass was his wife\. A retrieval\-augmented agent may search for “The North Star, publisher, married,” anchor on William Cooper Nell, who wrote for rather than published the paper, and spend the remaining budget on Nell’s marriage\. Both required facts remain in the corpus\. The failure is not evidence access; it is the loss of the intermediate entity that should guide the next query\.

Most work on complex RAG focuses on evidence access\. Methods organize a corpus into graphs or hierarchies to make passages easier to retrieve\([Edge et al\., 2024](https://arxiv.org/html/2610.21184#bib.bib7);[Sarthi et al\., 2024a](https://arxiv.org/html/2610.21184#bib.bib32);[Guo et al\., 2024](https://arxiv.org/html/2610.21184#bib.bib9)\), while agentic methods determine what to retrieve next\. These approaches do not address the example above when the retriever can already return the needed passage but the search no longer represents the intermediate target\. On MuSiQue, maintaining the unresolved sub\-questions across steps, without changing the retriever or corpus, reduces the share of wrong answers that lack a required passage from 382 to 234 cases under one retriever and from 383 to 274 under another\. This result motivates a different question: how much apparent retrieval failure can be reduced by improving search guidance without changing retriever coverage?

We call this failure*search drift*\. During a multi\-hop search, the agent may discard an unanswered sub\-question, anchor on the wrong intermediate entity, or keep querying for facts it has already found\([Joren et al\., 2024](https://arxiv.org/html/2610.21184#bib.bib19);[Ru et al\., 2024](https://arxiv.org/html/2610.21184#bib.bib31);[Kortukov et al\., 2024](https://arxiv.org/html/2610.21184#bib.bib20)\)\. Search drift appears as a retrieval symptom because a required passage is absent from the trace, even though it is present in the corpus\. It is also the dominant failure category for a strong agentic baseline on MuSiQue: 66–69% of wrong answers omit at least one required passage, and this share increases from 62% at two hops to 81% at four hops\.

ExecutionRAG implements this measurement with an explicit requirement register\. The register is a list of the question’s unresolved sub\-questions, initialized from the question and seed evidence rather than from the question alone\. Each entry records its state and the passages that support it\. Open entries guide subsequent search; an entry is retired only when later evidence contradicts it; and the agent answers only after no entry remains open\. The language model still decides whether evidence resolves an entry\. Two deterministic rules preserve the record by preventing open entries from silently disappearing and by retaining all previously recorded evidence\.

The main result is not simply that the register improves accuracy, but that it reveals what happens after more evidence is recovered\. Missing\-evidence failures decrease because the agent continues to search for unresolved entries, yet premature resolutions and abstentions increase, so total error falls by a much smaller amount\. Because each resolution is explicit, we can compare it with gold supporting evidence and identify cases in which the agent answers before the evidence is sufficient\. The register therefore exposes a transition from a search guidance problem to an evidence verification problem\.

In summary, this paper makes the following contributions:

- •We distinguish two causes of missing required evidence: evidence that the fixed retriever cannot reach and evidence that the search never requests\. This distinction turns evidence recovery under fixed retrieval into a diagnostic of search drift\.
- •We show a substantial controlled reduction in missing\-evidence failures\. On MuSiQue, the count decreases from 382 to 234 with all\-MiniLM\-L6\-v2 and from 383 to 274 with Qwen3\-Embedding\-0\.6B\. However, premature resolutions and abstentions increase, making evidence verification the principal remaining bottleneck\.
- •We introduce ExecutionRAG, a training\-free requirement register that provides the controlled intervention\. It tracks unresolved sub\-questions and their supporting passages while leaving the retriever, corpus, and generation\-model parameters unchanged\.

![Three-stage schematic of ExecutionRAG: a seed retrieval builds a register of the question's sub-questions; each step selects an open item, searches for it, and marks it answered or retired; the agent answers once no item is still open.](https://arxiv.org/html/2610.21184v1/figure/fig_method.png)Figure 1\.Overview of ExecutionRAG\.The agent keeps a requirement register of the question’s open sub\-questions and works through it in three stages:\(1\)build the register from the question and an initial \(seed\) retrieval, so its items are grounded in evidence the corpus can actually return;\(2\)at each step, select an open item, retrieve evidence, and update the register\. The agent marks an item answered when a passage supports it and records that passage, or drops the item if later evidence contradicts it; and\(3\)answer only once no item is still open, or return a bounded partial answer when the search budget runs out\.Three\-stage schematic of ExecutionRAG: a seed retrieval builds a register of the question's sub\-questions; each step selects an open item, searches for it, and marks it answered or retired; the agent answers once no item is still open\.
## 2\.Problem Setup: What “Retrieval Failure” Hides

Given a queryqqand a corpus𝒦\\mathcal\{K\}, conventional RAG retrieves context𝒞\\mathcal\{C\}and generates an answer in one shot,𝒞=R⁡\(q,𝒦\)\\mathcal\{C\}=R\(q;\\mathcal\{K\}\)anda=G⁡\(q,𝒞\)a=G\(q,\\mathcal\{C\}\), treating the retrieved passages as a single context\. For a multi\-hop question, the challenge is not only whether the right passages are retrieved, but also whether the agent retains which sub\-questions are resolved and which remain open as the search proceeds\.

### 2\.1\.Two Causes Inside One Symptom

Multi\-hop benchmarks let us say precisely what went wrong\. For a questionqq, letP⋆​\(q\)P^\{\\star\}\(q\)be its*gold supporting passages*: the minimal evidence a correct answer needs\. We call a wrong answer an*apparent retrieval failure*when somep∈P⋆​\(q\)p\\in P^\{\\star\}\(q\)was never retrieved\. Read literally, the name blames the retriever\. But withRRand𝒦\\mathcal\{K\}held fixed, the event splits into two*disjoint*causes:unreachable, no query the agent could have issued would have madeRRreturnpp; andunasked, some query would have returnedpp, but the agent never issued it\. Only the first is a retrieval problem; the second is*search drift*, where the evidence was within reach the whole time and the search stopped asking for it\. The two are indistinguishable in a trace, because both end with the same observable, namely a missing passage\. The category is therefore routinely attributed to the retriever\.

### 2\.2\.What a Fixed Retriever Identifies

The unasked share is not directly observable: deciding that*no*query would have worked means quantifying over a query space we cannot enumerate\. One direction, however, is decidable\. If an intervention leavesRRand𝒦\\mathcal\{K\}untouched and the needed passage is nonetheless retrieved, then a query returning it existed, so that failure was unasked\. Recovering evidence*certifies reachability*; failing to recover it proves nothing\. Holding the retriever fixed is therefore necessary for the inference\.

This argument identifies reachability at the level of an individual recovered passage\. At the aggregate level, however, a reduction in the frequency of missing\-evidence failures does not by itself identify which baseline failures were converted, because the two systems can fail on different questions\. We therefore use the controlled reduction as evidence of a recoverable component of apparent retrieval failure, not as a numerical lower bound on the total prevalence of search drift\. The minimal intervention keeps this interpretation specific to search guidance without claiming that every remaining missing passage is unreachable\.

### 2\.3\.The Probe: An Explicit Requirement Register

We therefore need an intervention that keeps the open sub\-questions in view while changing nothing about retrieval: an explicit*requirement register*, in effect a checklist the agent must clear before it answers\. For a queryqq, let𝒟⁡\(q\)=\{d1,…,dm\}\\mathcal\{D\}\(q\)=\\\{d\_\{1\},\\dots,d\_\{m\}\\\}be the question’s answer requirements, namely the sub\-questions the answer must satisfy \(e\.g\., “who is the publisher” and “who did that person marry”\)\. At stepttthe register records, for each item, its statuszi\(t\)z\_\{i\}^\{\(t\)\}, a short evidence noteei\(t\)e\_\{i\}^\{\(t\)\}, and the passagesPi\(t\)P\_\{i\}^\{\(t\)\}that support it:

\(1\)𝒮t=\{\(di,zi\(t\),ei\(t\),Pi\(t\)\)\}i=1m\.\\mathcal\{S\}\_\{t\}=\\\{\(d\_\{i\},z\_\{i\}^\{\(t\)\},e\_\{i\}^\{\(t\)\},P\_\{i\}^\{\(t\)\}\)\\\}\_\{i=1\}^\{m\}\.It says, at a glance, what is answered and what is still open, and it tells the agent what to search for next: an item that is still open, rather than repeatedly searching fromqqalone\. BecausePi\(t\)P\_\{i\}^\{\(t\)\}names the passage the agent*believed*settleddid\_\{i\}, it can be checked againstP⋆​\(q\)P^\{\\star\}\(q\)afterwards, which is what turns the register from a control structure into an instrument \(Section[4\.3](https://arxiv.org/html/2610.21184#S4.SS3)\)\.

## 3\.Method

ExecutionRAG is deliberately the smallest intervention that satisfies Section[2\.2](https://arxiv.org/html/2610.21184#S2.SS2): it changes what the agent searches*for*, and nothing about how retrieval works\. The retriever, the corpus, the tool interface, and the generator are untouched, so any evidence it recovers was evidence the same retriever could already reach\.

### 3\.1\.What the Register Records

ExecutionRAG maintains one entry per item, holding its status, a short evidence note, and the passages that support it\. Each item’s status takes one of four values:

\(2\)zi\(t\)∈\{pending,active,resolved,retired\}\.z\_\{i\}^\{\(t\)\}\\in\\\{\\texttt\{pending\},\\texttt\{active\},\\texttt\{resolved\},\\texttt\{retired\}\\\}\.pendingmeans the item has been identified but is not the current search target;activeis the item currently guiding the search;resolvedmeans enough evidence has been found to answer it; andretiredmeans it has been dropped, and only with an explicit reason\.

We keep the status set deliberately small\. Partial progress lives in the evidence noteei\(t\)e\_\{i\}^\{\(t\)\}and the supporting passagesPi\(t\)P\_\{i\}^\{\(t\)\}, not in extra labels, so an item stays open until its evidence is actually enough\. The status says*whether*an item is answered; the note and passages say*how*\. The four values are also what make the agent’s judgment legible: the step fromactivetoresolvedis the moment it declares an item settled\. Recording this decision as an event, rather than leaving it implicit in free\-form reasoning, allows us to compare it with gold evidence and determine whether it was premature\.

### 3\.2\.Building the Register from Seed Evidence

The register should not be written from the question alone\. Splitting a question in the abstract tends to produce items that are vague, redundant, or ask for things the corpus cannot answer\. So ExecutionRAG first runs one*seed*retrieval and builds the register from what comes back:

\(3\)𝒞\(0\)=Rseed​\(q,𝒦\)\.\\mathcal\{C\}^\{\(0\)\}=R\_\{\\text\{seed\}\}\(q;\\mathcal\{K\}\)\.The model reads\(q,𝒞\(0\)\)\(q,\\mathcal\{C\}^\{\(0\)\}\)and writes the initial register𝒮0\\mathcal\{S\}\_\{0\}, so its items are shaped by the question but grounded in evidence the corpus can actually return\. One item is setactiveas the first search target and the rest startpending, with empty notes:

\(4\)zb0\(0\)=active,zi\(0\)=pending\(i≠b0\),ei\(0\)=∅,Pi\(0\)=∅\.\\begin\{gathered\}z\_\{b\_\{0\}\}^\{\(0\)\}=\\texttt\{active\},\\qquad z\_\{i\}^\{\(0\)\}=\\texttt\{pending\}\\ \(i\\neq b\_\{0\}\),\\\\ e\_\{i\}^\{\(0\)\}=\\varnothing,\\qquad P\_\{i\}^\{\(0\)\}=\\varnothing\.\\end\{gathered\}This gives the agent one concrete thing to search for while keeping the other sub\-questions on the list for later\.

### 3\.3\.Updating the Register and Steering the Search

Starting from𝒮0\\mathcal\{S\}\_\{0\}, ExecutionRAG repeats a simple loop: pick an open item, search for it, and update that item\.

#### Searching for one open item

At steptt, ExecutionRAG picks an open itembtb\_\{t\}from𝒮t−1\\mathcal\{S\}\_\{t\-1\}and builds a query for it:

\(5\)ut=Compose​\(q,dbt,ebt\(t−1\)\)\.u\_\{t\}=\\textsc\{Compose\}\(q,d\_\{b\_\{t\}\},e\_\{b\_\{t\}\}^\{\(t\-1\)\}\)\.The original questionqqkeeps the overall goal in view,dbtd\_\{b\_\{t\}\}says which item is being chased, and the noteebt\(t−1\)e\_\{b\_\{t\}\}^\{\(t\-1\)\}points the search at what is still missing for that item\. The retriever returns new evidenceΔ​𝒞t\\Delta\\mathcal\{C\}\_\{t\}; repeated identical queries are cached so the agent does not loop\.

#### Updating only the items it touches

GivenΔ​𝒞t\\Delta\\mathcal\{C\}\_\{t\}, the register is updated as

\(6\)𝒮t=U⁡\(𝒮t−1,Δ​𝒞t\)\.\\mathcal\{S\}\_\{t\}=U\(\\mathcal\{S\}\_\{t\-1\},\\Delta\\mathcal\{C\}\_\{t\}\)\.The update rewrites the notes for the items the new evidence bears on, adds the supporting passages, and moves an item toresolvedonly when there is enough support, or toretiredonly with an explicit reason\. Items the new evidence does not touch are left unchanged\. In effect, each retrieval is turned into progress on one open item, rather than piling up more unsorted context\.

### 3\.4\.When to Answer, and Two Bookkeeping Rules

#### Answering only when the register is clear

After each update, ExecutionRAG checks whether any item is still open\. If none is, it writes the final answer from the register and its supporting passages; otherwise it keeps searching for the items that remain open\. If the search budget runs out first, it returns a bounded partial answer that says plainly which items are still unsupported\.

#### Showing only what changed

To keep the context short, ExecutionRAG does not replay the whole register after every step\. It shows the agent only a*progress delta*: the items whose status or note changed from𝒮t−1\\mathcal\{S\}\_\{t\-1\}to𝒮t\\mathcal\{S\}\_\{t\}\. This keeps the signal the agent needs for its next move without re\-reading the full list each time\.

#### Two rules that run outside the model

Because the register is updated by multiple LLM calls, ExecutionRAG enforces two deterministic rules to preserve its state\. First, an open item can leave the list only through an explicit transition toretired:

\(7\)Req⁡\(𝒮t\)⊇Req⁡\(𝒮t−1\)∖\{dk∣zk\(t\)=retired\}\.\\mathrm\{Req\}\(\\mathcal\{S\}\_\{t\}\)\\supseteq\\mathrm\{Req\}\(\\mathcal\{S\}\_\{t\-1\}\)\\setminus\\\{d\_\{k\}\\mid z\_\{k\}^\{\(t\)\}=\\texttt\{retired\}\\\}\.Second, supporting evidence is appended rather than overwritten:

\(8\)Pi\(t\)=Pi\(t−1\)∪Δ​Pi\(t\)\.P\_\{i\}^\{\(t\)\}=P\_\{i\}^\{\(t\-1\)\}\\cup\\Delta P\_\{i\}^\{\(t\)\}\.A later update may revise an item’s note, but it cannot erase the record of passages that previously supported it\. These rules preserve the requirements and their evidence trail; they do not guarantee that the language model resolves an item correctly\. Keeping this decision explicit allows us to evaluate it against gold evidence \(Section[4\.3](https://arxiv.org/html/2610.21184#S4.SS3)\)\.

## 4\.Experiments

### 4\.1\.Experimental Questions

Section[2\.2](https://arxiv.org/html/2610.21184#S2.SS2)defines the controlled comparison: holding the retriever and corpus fixed, we measure how much the frequency of missing\-evidence failures changes when only search guidance is modified\. The experiments first quantify this reduction and then examine whether recovering evidence reduces total error or reveals a different bottleneck\. The framing also makes predictions that could fail\. If the recovered evidence were an artifact of the register writing longer answers that favor the judge, it would not improve recall of gold evidence or the objective substring metric\. If the register keeps search on target without making the generator a better verifier, the failures left behind should move from “never retrieved the evidence” toward answering too early or abstaining\. And if the effect were really about reading more text, it should vanish where the register reads no more than the baseline\.

Accordingly, we ask:

- •RQ1: Under a fixed retriever, how much does search guidance reduce failures caused by missing evidence?
- •RQ2: How does the failure distribution change after more required evidence is recovered?
- •RQ3: Does the reduction persist with a different embedding model?
- •RQ4: Does the recovered evidence improve answer accuracy, particularly when sub\-questions must be combined across documents?
- •RQ5: Is the gain explained by retrieving more context?
- •RQ6: Do seed grounding and iterative updates both contribute to the result?

### 4\.2\.Experimental Setup

#### Datasets

We evaluate on three standard multi\-hop QA benchmarks: HotpotQA\([Yang et al\., 2018](https://arxiv.org/html/2610.21184#bib.bib48)\), MuSiQue\([Trivedi et al\., 2022](https://arxiv.org/html/2610.21184#bib.bib35)\), and 2WikiMultiHopQA \(2Wiki\)\([Ho et al\., 2020](https://arxiv.org/html/2610.21184#bib.bib15)\)\. Following HippoRAG, we use the same retrieval corpora and sample 1,000 questions from each validation set as test queries\.

#### Baselines

We compare against three groups of baselines: \(1\)*Direct inference*: GPT\-4o\-mini without retrieval; \(2\)*Vanilla RAG*: top\-kkretrieval withk∈\{3,5,10\}k\\in\\\{3,5,10\\\}, CoT\-augmented retrieval, and IRCoT\([Trivedi et al\., 2023](https://arxiv.org/html/2610.21184#bib.bib36)\); \(3\)*Structured and agentic RAG*: KGP\([Wang et al\., 2024](https://arxiv.org/html/2610.21184#bib.bib37)\), G\-Retriever\([He et al\., 2024](https://arxiv.org/html/2610.21184#bib.bib14)\), RAPTOR\([Sarthi et al\., 2024b](https://arxiv.org/html/2610.21184#bib.bib33)\), LightRAG\([Guo et al\., 2025](https://arxiv.org/html/2610.21184#bib.bib10)\), HippoRAG and HippoRAG 2\([Gutiérrez et al\., 2024](https://arxiv.org/html/2610.21184#bib.bib11);[Gutiérrez et al\., 2025a](https://arxiv.org/html/2610.21184#bib.bib12)\), GFM\-RAG\([Luo et al\., 2025](https://arxiv.org/html/2610.21184#bib.bib28)\), LinearRAG\([Zhuang et al\., 2025](https://arxiv.org/html/2610.21184#bib.bib55)\), LAG\([Xiao et al\., 2025](https://arxiv.org/html/2610.21184#bib.bib44)\), and A\-RAG\([Du et al\., 2026](https://arxiv.org/html/2610.21184#bib.bib6)\)\. This comparison spans direct answering, flat retrieval, corpus structuring, and iterative retrieval control\.

#### Metrics

Following LinearRAG\([Zhuang et al\., 2025](https://arxiv.org/html/2610.21184#bib.bib55)\), we report*Contain\-Match Accuracy*, which checks whether the gold answer appears as a substring of the prediction, and*LLM\-Evaluation Accuracy*, which uses an LLM judge to assess semantic equivalence\.

#### Implementation details

Unless otherwise specified, all methods use GPT\-4o\-mini as the generation model\. For the main comparison, we use all\-MiniLM\-L6\-v2 with top\-k=5k=5retrieval, and additionally evaluate Qwen3\-Embedding\-0\.6B as an alternative embedding backbone\. All main results use the full 1,000\-question evaluation sets\. The ablation study is conducted on the full 1,000\-question MuSiQue evaluation set\. Additional implementation details are provided in Appendix[B](https://arxiv.org/html/2610.21184#A2)\.

### 4\.3\.RQ1: Reduction in Failures Caused by Missing Evidence

We measure the controlled reduction described in Section[2\.2](https://arxiv.org/html/2610.21184#S2.SS2)by assigning each wrong answer on MuSiQue a single cause using the gold supporting paragraphs\. The full counts and percentages for both retrievers appear in Appendix[E](https://arxiv.org/html/2610.21184#A5)\.

#### How we label a failure

Each wrong answer receives exactly one cause, assigned by a fixed priority order so the categories cannot overlap:*judge error*\(the substring metric counts the answer as right but the judge does not\),*gave up*\(the answer explicitly abstains\),*reasoning error*\(every supporting passage was retrieved and the answer is still wrong, or, for ExecutionRAG, an item was resolved while supporting evidence was still missing\), and*missing evidence*\(a supporting passage was never retrieved and no premature resolution occurred\)\. What makes this labelling possible at all is that MuSiQue ships, for each question, the supporting paragraphs a correct answer minimally needs: we match each supporting paragraph to its corpus chunk by a 40\-character fingerprint \(87\.9% of paragraphs match\) and ask whether the search ever retrieved it\. Because the two methods fail on different numbers of questions, every share below is a percentage of that method’s own failures\.

The largest change is a reduction in failures caused by missing evidence: wrong answers for which a required passage was never retrieved\. With all\-MiniLM\-L6\-v2, the count decreases from 382 for A\-RAG to 234 for ExecutionRAG; these values represent 69\.3% of 551 baseline failures and 43\.9% of 533 ExecutionRAG failures\. With Qwen3\-Embedding\-0\.6B, the count decreases from 383 to 274, or from 66\.4% of 577 failures to 47\.9% of 572 failures\. The same retriever is used within each comparison, so the reductions show that search guidance can recover a substantial portion of evidence that the baseline search did not retrieve\. They do not imply that every residual missing passage is unreachable \(Figure[2](https://arxiv.org/html/2610.21184#S4.F2)\)\.

Recovering this evidence does not reduce total error proportionally\. Reasoning errors after retrieval increase from 9% to 26% with MiniLM and from 8% to 21% with Qwen; these errors include cases in which the register is resolved before its evidence is sufficient\. Abstentions also increase, from 7% to 13% and from 12% to 17%, respectively\. Against A\-RAG, LLM\-Evaluation Accuracy on MuSiQue rises from 44\.9 to 50\.2, a smaller change than the reduction in missing\-evidence failures\. This redistribution is the central result: once search guidance recovers more required evidence, evidence verification and answer commitment become the dominant sources of the remaining error\.

### 4\.4\.RQ2: Evidence Recovery Shifts the Remaining Failures

#### Why reasoning errors become more visible

Part of this increase reflects observability rather than a change in the model’s reasoning alone\. A premature resolution is an explicit register event and can therefore be assigned to reasoning error\. A\-RAG has no analogous event; when it answers from incomplete evidence, the error is generally assigned to the missing passage instead\. The comparison may thus undercount premature answers by the baseline\. The register exposes this decision for evaluation; it does not establish that the generation model reasons worse\.

The register guides search but does not provide a reliable evidence verifier: the language model still decides whether a requirement has been resolved, and this decision can be incorrect\. It also cannot make genuinely difficult passages retrievable\. Residual failures caused by missing evidence for ExecutionRAG remain highest on four\-hop questions, which contain longer and more weakly connected evidence chains\. In this setting, the remaining bottlenecks are verifying retrieved evidence and retrieving evidence for long chains\.

Figure 2\.Failure distribution on MuSiQue, by failure type and hop count\. The share of failures that never retrieve a required passage decreases after adding the register, while reasoning errors after retrieval and abstentions increase\. For ExecutionRAG, failures caused by missing evidence remain most common at four hops\.Stacked bars of wrong\-answer causes on MuSiQue for A\-RAG and ExecutionRAG, overall and split by hop count\. The share that never retrieves a required passage decreases under ExecutionRAG while reasoning errors after retrieval and abstentions increase\.

### 4\.5\.RQ3: The Reduction Persists Across Embedding Models

Figure 3\.Performance under Qwen3\-Embedding\-0\.6B\.Left: per\-benchmark Contain\-Match and LLM\-Evaluation Accuracy\. Right: average accuracy across benchmarks\.Bar charts of Contain\-Match and LLM\-Evaluation accuracy per benchmark, plus the cross\-benchmark average, under the Qwen3\-Embedding\-0\.6B retriever\. ExecutionRAG stays above A\-RAG on all three benchmarks\.Figure[3](https://arxiv.org/html/2610.21184#S4.F3)tests whether the performance pattern depends on the embedding model\. After replacing all\-MiniLM\-L6\-v2 with Qwen3\-Embedding\-0\.6B, ExecutionRAG outperforms A\-RAG on all three benchmarks: 80\.0, 68\.3, and 42\.8 LLM\-Evaluation Accuracy on HotpotQA, 2Wiki, and MuSiQue, respectively, compared with 76\.4, 55\.2, and 42\.3 for A\-RAG\. The largest gain remains on 2Wiki \(\+13\.1\+13\.1\), while MuSiQue shows a smaller positive gain \(\+0\.5\+0\.5\)\. The positive difference across all three benchmarks shows that the effect is not specific to one embedding model\. The larger gain on 2Wiki is also consistent with the proposed mechanism because its questions require dependent sub\-questions to be maintained across multiple retrieval steps\.

Table 1\.Main results on three multi\-hop QA benchmarks\.Bold marks the best*Contain\-Match*result per benchmark; the grey columns report*LLM\-Evaluation Accuracy*for comparability with prior work\. These answer\-level metrics complement the primary failure analysis by showing whether recovered evidence improves the final prediction\. The lower block is the matched comparison our attribution rests on: A\-RAG shares ExecutionRAG’s generator, tools, and corpora, so it is ExecutionRAG*without*the register\. Gains concentrate where sub\-questions must be combined across documents; HotpotQA is a tie\.
### 4\.6\.RQ4: Answer Accuracy

Table[1](https://arxiv.org/html/2610.21184#S4.T1)reports the main comparison\. We first consider Contain\-Match Accuracy because it does not rely on an LLM judge\. On MuSiQue, ExecutionRAG reaches 49\.1, a 6\.3\-point improvement over LAG \(42\.8\)\. On 2Wiki, it reaches 73\.0, a 1\.1\-point improvement over LAG \(71\.9\)\. On HotpotQA, it reaches 77\.0, which is essentially tied with A\-RAG \(77\.3\)\. LLM\-Evaluation Accuracy follows the same pattern and is highest for ExecutionRAG on all three benchmarks: 79\.8, 74\.3, and 50\.2 on HotpotQA, 2Wiki, and MuSiQue, respectively\.

The differences are consistent with the proposed mechanism\. The register has little effect on HotpotQA, whose questions are mostly two\-hop and are often answerable from a single comparison\. It improves performance on 2Wiki and MuSiQue, where the agent must maintain several dependent sub\-questions across documents\.

A\-RAG is the matched baseline for the failure analysis\. It uses the same generation model, retrieval tools, and corpora as ExecutionRAG; the requirement register is the added component\. The difference between A\-RAG and ExecutionRAG in Sections[4\.3](https://arxiv.org/html/2610.21184#S4.SS3)–[4\.5](https://arxiv.org/html/2610.21184#S4.SS5)therefore cannot be attributed to a different retriever or generation backbone\. The register adds bookkeeping calls, but these calls alone do not explain the gain: on 2Wiki, the largest improvement occurs with comparable or fewer retrieved tokens \(Section[4\.7](https://arxiv.org/html/2610.21184#S4.SS7)\)\.

#### Where the gains concentrate

Figure[5](https://arxiv.org/html/2610.21184#A3.F5)in Appendix[C](https://arxiv.org/html/2610.21184#A3)reports category\-level results\. Under both embedding models, questions with longer or less direct evidence chains remain more difficult\. The register helps most when required evidence is retrievable but must be tracked across several steps\. On MuSiQue, accuracy decreases with hop count\. On HotpotQA, Comparison questions are easier than Bridge questions; on 2Wiki, Inference questions are hardest and Comparison questions are easiest\. The register improves overall accuracy but does not remove the difficulty of long evidence chains\.

Figure 4\.Accuracy–efficiency trade\-off\. Pareto frontiers of LLM\-Evaluation Accuracy vs\. retrieved tokens per question for ExecutionRAG and A\-RAG under two embedding backbones\. Bottom row: ablation variants; bars show LLM\-Acc and the line shows average retrieved tokens per question\.Pareto scatter of LLM\-Evaluation accuracy against retrieved tokens per question for ExecutionRAG and A\-RAG under two embedding backbones, with ablation variants below\.

### 4\.7\.RQ5: Retrieved Context and Accuracy

Figure[4](https://arxiv.org/html/2610.21184#S4.F4)and Table[2](https://arxiv.org/html/2610.21184#S4.T2)test whether the gains can be explained by retrieving more context\. On MuSiQue and HotpotQA, ExecutionRAG reads more tokens than A\-RAG because it builds and updates the register\. On 2Wiki, however, it uses comparable or fewer retrieved tokens while obtaining its largest accuracy improvement\. Within ExecutionRAG, removing register components reduces both token use and accuracy\. The result is consistent with the register improving the targeting of retrieval rather than simply increasing the amount of retrieved context\.

#### What the register costs

The register increases the cost of some searches\. On MuSiQue, it reads 7,161 retrieved tokens per question, compared with 5,015 for A\-RAG; on HotpotQA, it reads 3,373 compared with 2,860\. On 2Wiki, token use is comparable at 3,966 and 3,992, respectively\. The corresponding API costs are $0\.0064 versus $0\.0034 on MuSiQue and $0\.0039 versus $0\.0027 on 2Wiki\. Search is limited to fifteen retrieval loops and a cumulative context of 128,000 tokens\. The agent answers when no requirement remains open, so the register shortens some trajectories and lengthens others; the case studies in Appendix[F](https://arxiv.org/html/2610.21184#A6)complete in two and six loops, whereas A\-RAG uses five and thirteen\. Appendix[D](https://arxiv.org/html/2610.21184#A4)provides the complete breakdown\.

Table 2\.Ablation study on MuSiQue\.LLM\-Evaluation and Contain\-Match Accuracy, under the default setup\.

### 4\.8\.RQ6: Effects of Seed Grounding and Iterative Updates

Table[2](https://arxiv.org/html/2610.21184#S4.T2)isolates the contributions of the register’s components\. Building the register from the question alone rather than from seed evidence lowers LLM\-Acc from 50\.2 to 48\.7, indicating that grounding requirements in retrievable evidence is beneficial\. Removing iterative updates and constructing the register only once lowers LLM\-Acc further to 48\.3\. Both seed grounding and iterative updates contribute to the observed gain\.

## 5\.Related Work

#### Corpus structure for RAG

A major line of work improves multi\-hop QA by structuring corpus knowledge to improve evidence access\. GraphRAG\([Edge et al\., 2024](https://arxiv.org/html/2610.21184#bib.bib7)\)builds entity\-relation graphs with community summaries; RAPTOR\([Sarthi et al\., 2024a](https://arxiv.org/html/2610.21184#bib.bib32)\)organizes documents into hierarchical retrieval trees; LightRAG\([Guo et al\., 2024](https://arxiv.org/html/2610.21184#bib.bib9)\)combines graph\-augmented indexing with efficient retrieval; HippoRAG and HippoRAG 2\([Jimenez Gutierrez et al\., 2024](https://arxiv.org/html/2610.21184#bib.bib18);[Gutiérrez et al\., 2025b](https://arxiv.org/html/2610.21184#bib.bib13)\)use graph\-based retrieval with Personalized PageRank; and LinearRAG\([Zhuang et al\., 2025](https://arxiv.org/html/2610.21184#bib.bib55)\), GFM\-RAG\([Luo et al\., 2025](https://arxiv.org/html/2610.21184#bib.bib28)\), and StructRAG\([Li et al\., 2025](https://arxiv.org/html/2610.21184#bib.bib27)\)further explore graph construction, graph foundation models, and hybrid structurization\. These methods improve how dispersed evidence is represented and retrieved before answering\. ExecutionRAG is complementary: it does not restructure the corpus, but maintains an explicit register of unresolved sub\-questions during search\.

#### Iterative and agentic RAG

A related line of work improves retrieval control during inference\. IRCoT\([Trivedi et al\., 2023](https://arxiv.org/html/2610.21184#bib.bib36)\)and FLARE\([Jiang et al\., 2023](https://arxiv.org/html/2610.21184#bib.bib16)\)interleave reasoning with retrieval; Self\-RAG\([Asai et al\., 2023](https://arxiv.org/html/2610.21184#bib.bib2)\)uses reflection tokens to critique retrieved passages; and recent agentic methods expose richer retrieval policies through tool use, query reformulation, and multi\-step search\. A\-RAG\([Du et al\., 2026](https://arxiv.org/html/2610.21184#bib.bib6)\)replaces fixed pipelines with model\-driven hierarchical retrieval interfaces, while LAG\([Xiao et al\., 2025](https://arxiv.org/html/2610.21184#bib.bib44)\)and LogicRAG\([Chen et al\., 2026](https://arxiv.org/html/2610.21184#bib.bib5)\)decompose questions into subproblems, infer dependencies, and sequence retrieval accordingly\. This view is also related to work that formulates language\-model inference as search\([Wei et al\., 2026c](https://arxiv.org/html/2610.21184#bib.bib41)\), analyzes how chain\-of\-thought prompting interacts with the answer space\([Zhang et al\., 2025b](https://arxiv.org/html/2610.21184#bib.bib52)\), learns when a model should disclose its reasoning\([Wei et al\., 2026a](https://arxiv.org/html/2610.21184#bib.bib38)\), learns execution\-time workflows for multi\-agent systems\([Xu et al\., 2026a](https://arxiv.org/html/2610.21184#bib.bib45)\), or scales multi\-agent processing over many documents at test time\([Cao et al\., 2025](https://arxiv.org/html/2610.21184#bib.bib4)\)\. These methods mainly optimize retrieval control: what to retrieve next and in what order\. ExecutionRAG does not introduce a new retrieval policy\. Instead, it maintains an explicit register of unresolved sub\-questions that guides subsequent search and determines when the agent can answer\. Even with adaptive retrieval, an agent can discard an unresolved sub\-question, overwrite earlier progress, or answer from partial evidence\. The register preserves these requirements throughout the search\. It is also narrower than agent\-memory work that retrieves over a persistent episodic record of an agent’s past decisions\([Xu et al\., 2026b](https://arxiv.org/html/2610.21184#bib.bib46)\): the register tracks the open requirements of the*current*question only, and is discarded once the answer is given\.

#### Long executions outside question answering

The same pressure to keep intermediate state explicit appears in domains where an agent must execute a long chain before it can commit to an answer\. Multi\-agent drafting protocols compress intermediate reasoning\([Li et al\., 2026a](https://arxiv.org/html/2610.21184#bib.bib23)\), and retrieval\-augmented multi\-agent pipelines are built for tasks whose verdict depends on evidence gathered in several steps\([Li et al\., 2026b](https://arxiv.org/html/2610.21184#bib.bib24)\)\. Software\-engineering agents are trained against execution feedback and process\-level reward precisely because their trajectories are too long to supervise end to end\([Li et al\., 2026d](https://arxiv.org/html/2610.21184#bib.bib26);[Li et al\., 2026c](https://arxiv.org/html/2610.21184#bib.bib25);[Tan et al\., 2026](https://arxiv.org/html/2610.21184#bib.bib34)\)\. There too the difficulty is less the first attempt than checking what the agent has actually done\. The register is the minimal instance of that bookkeeping for retrieval: it records the requirements a trajectory still owes, and nothing else\.

#### Sufficiency signals, reading notes, and failure analysis

Several methods track information gaps or judge evidence sufficiency during RAG\. Self\-RAG\([Asai et al\., 2023](https://arxiv.org/html/2610.21184#bib.bib2)\)critiques passages with reflection tokens\([Zhang et al\., 2025a](https://arxiv.org/html/2610.21184#bib.bib51)\)\. Related work adds corrective retrieval \(CRAG\([Yan et al\., 2024](https://arxiv.org/html/2610.21184#bib.bib47)\)\), sequential reading notes \(Chain\-of\-Note\([Yu et al\., 2024](https://arxiv.org/html/2610.21184#bib.bib50)\)\), and structured missing\-information items with sufficiency checks \(S2G\-RAG\([Li et al\., 2026e](https://arxiv.org/html/2610.21184#bib.bib22)\), SURE\-RAG\([Qiu et al\., 2026](https://arxiv.org/html/2610.21184#bib.bib30)\)\)\. Test\-time critique and optimization provide another mechanism for improving retrieved evidence and generated answers\([Wei et al\., 2025b](https://arxiv.org/html/2610.21184#bib.bib42)\)\. Other methods diagnose or repair agent trajectories after the fact, such as Doctor\-RAG\([Jiao et al\., 2026](https://arxiv.org/html/2610.21184#bib.bib17)\)\. The requirement register is an explicit list of sub\-questions with states and evidence provenance\. Our contribution is empirical: by holding the retriever and generator fixed, we measure how such a register changes the multi\-hop failure distribution\. It reduces failures caused by missing evidence under a fixed retriever and shifts residual errors toward reasoning after retrieval, a distinction that prior end\-to\-end evaluations do not isolate\.

#### Naming the two causes behind one symptom

Skill\-RAG\([Wei et al\., 2026b](https://arxiv.org/html/2610.21184#bib.bib43)\)makes the closest claim to Section[2\.1](https://arxiv.org/html/2610.21184#S2.SS1)we know of: retrieval failures “stem not from the absence of relevant evidence but from an alignment gap between the query and the evidence space,” with an exit route for cases it calls truly irreducible\. This distinguishes unreachable and unasked causes conceptually and routes learned repair skills accordingly\. It does not define the failure event using gold evidence or quantify either share\. We instead treat the distinction as the measurement target\. Outside retrieval, Binding Drift\([Babu and Indukuri, 2026](https://arxiv.org/html/2610.21184#bib.bib3)\)separates drift from error propagation using disjoint evaluation sets\. We apply the same separation to a missing passage by asking whether the search could have retrieved it\.

#### Implications

The results distinguish two directions for future work\. Because the recovered evidence was reachable by the same retriever, increasing index coverage, improving embeddings, or expanding queries does not address the observed failures associated with inadequate search guidance\. Those failures require a mechanism that preserves unresolved sub\-questions\. Conversely, the increase in premature resolution identifies evidence verification as a separate target\. The register makes the resolution decision explicit, which enables a verifier to be trained or evaluated against it\. Residual failures caused by missing evidence concentrate on long, weakly connected chains, where improving retrieval coverage remains necessary\. These predictions are testable: a verifier should improve answer accuracy without changing retrieved evidence, whereas an intervention that improves only coverage should not eliminate failures caused by unasked queries\.

## 6\.Conclusion

We study how much apparent retrieval failure in multi\-hop RAG can be reduced by search guidance while the retriever remains fixed\. On MuSiQue, the requirement register reduces wrong answers with unretrieved required evidence from 382 to 234 cases under one retriever and from 383 to 274 under another\. This recovery does not translate proportionally into answer accuracy: premature resolutions and abstentions increase as missing\-evidence failures decrease\. The result separates two bottlenecks that aggregate accuracy obscures\. Maintaining unresolved requirements helps the agent retrieve reachable evidence, but converting that evidence into correct answers requires more reliable verification after retrieval\.

###### Acknowledgements\.

Supported by Shanghai Artificial Intelligence Laboratory\.

## References

- Asai et al\.\(2023\)Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi\. 2023\.Self\-RAG: Learning to Retrieve, Generate, and Critique through Self\-Reflection\.arXiv:2310\.11511 \[cs\.CL\][https://arxiv\.org/abs/2310\.11511](https://arxiv.org/abs/2310.11511)
- Babu and Indukuri \(2026\)Rahul Suresh Babu and Shashank Indukuri\. 2026\.Binding Drift in Multi\-Step Tool\-Augmented Agents\.arXiv:2607\.18316 \[cs\.CL\][https://arxiv\.org/abs/2607\.18316](https://arxiv.org/abs/2607.18316)
- Cao et al\.\(2025\)Juntai Cao, Xiang Zhang, Raymond Li, Jiaqi Wei, Chuyuan Li, Shafiq Joty, and Giuseppe Carenini\. 2025\.Multi2: Multi\-agent test\-time scalable framework for multi\-document processing\. In*Proceedings of The 5th New Frontiers in Summarization Workshop*\. 135–156\.
- Chen et al\.\(2026\)Shengyuan Chen, Chuang Zhou, Zheng Yuan, Qinggang Zhang, Zeyang Cui, Hao Chen, Yilin Xiao, Jiannong Cao, and Xiao Huang\. 2026\.You don’t need pre\-built graphs for rag: Retrieval augmented generation with adaptive reasoning structures\. In*Proceedings of the AAAI Conference on Artificial Intelligence*, Vol\. 40\. 30270–30278\.
- Du et al\.\(2026\)Mingxuan Du, Benfeng Xu, Chiwei Zhu, Shaohan Wang, Pengyu Wang, Xiaorui Wang, and Zhendong Mao\. 2026\.A\-RAG: Scaling Agentic Retrieval\-Augmented Generation via Hierarchical Retrieval Interfaces\.*arXiv preprint arXiv:2602\.03442*\(2026\)\.
- Edge et al\.\(2024\)Darren Edge, Ha Trinh, Newman Cheng, Joshua Bradley, Alex Chao, Apurva Mody, Steven Truitt, and Jonathan Larson\. 2024\.From local to global: A graph rag approach to query\-focused summarization\.*arXiv preprint arXiv:2404\.16130*\(2024\)\.
- Gao et al\.\(2023\)Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, and Haofen Wang\. 2023\.Retrieval\-augmented generation for large language models: A survey\.*arXiv preprint arXiv:2312\.10997*\(2023\)\.
- Guo et al\.\(2024\)Zirui Guo, Lianghao Xia, Yanhua Yu, Tu Ao, and Chao Huang\. 2024\.LightRAG: Simple and Fast Retrieval\-Augmented Generation\.*arXiv preprint arXiv:2410\.05779*\(2024\)\.
- Guo et al\.\(2025\)Zirui Guo, Lianghao Xia, Yanhua Yu, Tu Ao, and Chao Huang\. 2025\.LightRAG: Simple and Fast Retrieval\-Augmented Generation\.arXiv:2410\.05779 \[cs\.IR\][https://arxiv\.org/abs/2410\.05779](https://arxiv.org/abs/2410.05779)
- Gutiérrez et al\.\(2024\)Bernal Jiménez Gutiérrez, Yiheng Shu, Yu Gu, Michihiro Yasunaga, and Yu Su\. 2024\.Hipporag: Neurobiologically inspired long\-term memory for large language models\. In*The Thirty\-eighth Annual Conference on Neural Information Processing Systems*\.
- Gutiérrez et al\.\(2025a\)Bernal Jiménez Gutiérrez, Yiheng Shu, Weijian Qi, Sizhe Zhou, and Yu Su\. 2025a\.From RAG to Memory: Non\-Parametric Continual Learning for Large Language Models\. In*Forty\-second International Conference on Machine Learning*\.[https://openreview\.net/forum?id=LWH8yn4HS2](https://openreview.net/forum?id=LWH8yn4HS2)
- Gutiérrez et al\.\(2025b\)Bernal Jiménez Gutiérrez, Yiheng Shu, Weijian Qi, Sizhe Zhou, and Yu Su\. 2025b\.From RAG to Memory: Non\-Parametric Continual Learning for Large Language Models\. In*Forty\-second International Conference on Machine Learning*\.[https://openreview\.net/forum?id=LWH8yn4HS2](https://openreview.net/forum?id=LWH8yn4HS2)
- He et al\.\(2024\)Xiaoxin He, Yijun Tian, Yifei Sun, Nitesh Chawla, Thomas Laurent, Yann LeCun, Xavier Bresson, and Bryan Hooi\. 2024\.G\-retriever: Retrieval\-augmented generation for textual graph understanding and question answering\.*Advances in Neural Information Processing Systems*37 \(2024\), 132876–132907\.
- Ho et al\.\(2020\)Xanh Ho, Anh\-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa\. 2020\.Constructing A Multi\-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps\. In*Proceedings of the 28th International Conference on Computational Linguistics*, Donia Scott, Nuria Bel, and Chengqing Zong \(Eds\.\)\. International Committee on Computational Linguistics, Barcelona, Spain \(Online\), 6609–6625\.[doi:10\.18653/v1/2020\.coling\-main\.580](https://doi.org/10.18653/v1/2020.coling-main.580)
- Jiang et al\.\(2023\)Zhengbao Jiang, Frank F Xu, Luyu Gao, Zhiqing Sun, Qian Liu, Jane Dwivedi\-Yu, Yiming Yang, Jamie Callan, and Graham Neubig\. 2023\.Active retrieval augmented generation\. In*Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing*\.
- Jiao et al\.\(2026\)Shuguang Jiao, Chengkai Huang, Shuhan Qi, Xuan Wang, Yifan Li, Quanchi Weng, Lingchuan Liu, Xunliang Cai, and Lina Yao\. 2026\.Doctor\-RAG: A Failure\-Aware Repair Framework for Agentic Retrieval\-Augmented Generation\.arXiv:2604\.00865 \[cs\.IR\][https://arxiv\.org/abs/2604\.00865](https://arxiv.org/abs/2604.00865)
- Jimenez Gutierrez et al\.\(2024\)Bernal Jimenez Gutierrez, Yiheng Shu, Yu Gu, Michihiro Yasunaga, and Yu Su\. 2024\.Hipporag: Neurobiologically inspired long\-term memory for large language models\.*Advances in Neural Information Processing Systems*37 \(2024\), 59532–59569\.
- Joren et al\.\(2024\)Hailey Joren, Jianyi Zhang, Chun\-Sung Ferng, Da\-Cheng Juan, Ankur Taly, and Cyrus Rashtchian\. 2024\.Sufficient context: A new lens on retrieval augmented generation systems\.*arXiv preprint arXiv:2411\.06037*\(2024\)\.
- Kortukov et al\.\(2024\)Evgenii Kortukov, Alexander Rubinstein, Elisa Nguyen, and Seong Joon Oh\. 2024\.Studying large language model behaviors under context\-memory conflicts with real documents\.*arXiv preprint arXiv:2404\.16032*\(2024\)\.
- Lewis et al\.\(2020\)Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen\-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela\. 2020\.Retrieval\-augmented generation for knowledge\-intensive NLP tasks\. In*Proceedings of the 34th International Conference on Neural Information Processing Systems*\(Vancouver, BC, Canada\)*\(NIPS ’20\)*\. Curran Associates Inc\., Red Hook, NY, USA, Article 793, 16 pages\.
- Li et al\.\(2026e\)Minghan Li, Junjie Zou, Xinxuan Lv, Chao Zhang, and Guodong Zhou\. 2026e\.S2G\-RAG: Structured Sufficiency and Gap Judging for Iterative Retrieval\-Augmented QA\. In*Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, Maria Liakata, Viviane P\. Moreira, Jiajun Zhang, and David Jurgens \(Eds\.\)\. Association for Computational Linguistics, San Diego, California, United States, 25846–25862\.[doi:10\.18653/v1/2026\.acl\-long\.1185](https://doi.org/10.18653/v1/2026.acl-long.1185)
- Li et al\.\(2026a\)Yuanhao Li, Mingshan Liu, Hongbo Wang, Yiding Zhang, Yifei Ma, and Wei Tan\. 2026a\.DRAFT\-RL: Multi\-Agent Chain\-of\-Draft Reasoning for Reinforcement Learning\-Enhanced LLMs\.*Proceedings of the AAAI Conference on Artificial Intelligence*40, 35 \(2026\), 29530–29537\.[doi:10\.1609/aaai\.v40i35\.40195](https://doi.org/10.1609/aaai.v40i35.40195)
- Li et al\.\(2026b\)Yuanhao Li, Hongbo Wang, Xuhong Chen, and Yiming Cao\. 2026b\.Retrieval\-Augmented Multi\-Agent Multimodal Framework for Fake News Detection\. In*ICASSP 2026*\.[https://cmsworkshops\.com/ICASSP2026/view\_paper\.php?PaperNum=9709](https://cmsworkshops.com/ICASSP2026/view_paper.php?PaperNum=9709)
- Li et al\.\(2026c\)Yuanhao Li, Hongbo Wang, Xuhong Chen, Yiming Cao, and Xunzhu Tang\. 2026c\.Counterfactual Rollout Replay: Forkable Environments as Free Process Rewards for Software Engineering Agents\.arXiv:2609\.33875 \[cs\.SE\][https://arxiv\.org/abs/2609\.33875](https://arxiv.org/abs/2609.33875)Accepted at NeurIPS 2026\.
- Li et al\.\(2026d\)Yuanhao Li, Hongbo Wang, Xiaotang Shang, Xunzhu Tang, Yiming Cao, and Xuhong Chen\. 2026d\.BoostAPR: Boosting Automated Program Repair via Execution\-Grounded Reinforcement Learning with Dual Reward Models\.arXiv:2605\.09134 \[cs\.AI\][https://arxiv\.org/abs/2605\.09134](https://arxiv.org/abs/2605.09134)Accepted at ICML 2026\.
- Li et al\.\(2025\)Zhuoqun Li, Xuanang Chen, Haiyang Yu, Hongyu Lin, Yaojie Lu, Qiaoyu Tang, Fei Huang, Xianpei Han, Le Sun, and Yongbin Li\. 2025\.StructRAG: Boosting Knowledge Intensive Reasoning of LLMs via Inference\-time Hybrid Information Structurization\. In*The Thirteenth International Conference on Learning Representations*\.[https://openreview\.net/forum?id=GhexuBLxbO](https://openreview.net/forum?id=GhexuBLxbO)
- Luo et al\.\(2025\)Linhao Luo, Zicheng Zhao, Gholamreza Haffari, Dinh Phung, Chen Gong, and Shirui Pan\. 2025\.GFM\-RAG: Graph Foundation Model for Retrieval Augmented Generation\.*arXiv preprint arXiv:2502\.01113*\(2025\)\.
- OpenAI \(2024\)OpenAI\. 2024\.GPT\-4o System Card\.arXiv:2410\.21276 \[cs\.CL\][https://arxiv\.org/abs/2410\.21276](https://arxiv.org/abs/2410.21276)
- Qiu et al\.\(2026\)Jingxi Qiu, Zeyu Han, and Cheng Huang\. 2026\.SURE\-RAG: Sufficiency and Uncertainty\-Aware Evidence Verification for Selective Retrieval\-Augmented Generation\.arXiv:2605\.03534 \[cs\.CL\][https://arxiv\.org/abs/2605\.03534](https://arxiv.org/abs/2605.03534)
- Ru et al\.\(2024\)Dongyu Ru, Lin Qiu, Xiangkun Hu, Tianhang Zhang, Peng Shi, Shuaichen Chang, Cheng Jiayang, Cunxiang Wang, Shichao Sun, Huanyu Li, et al\.2024\.Ragchecker: A fine\-grained framework for diagnosing retrieval\-augmented generation\.*Advances in Neural Information Processing Systems*37 \(2024\), 21999–22027\.
- Sarthi et al\.\(2024a\)Parth Sarthi, Salman Abdullah, Aditi Tuli, Shubh Khanna, Anna Goldie, and Christopher D\. Manning\. 2024a\.RAPTOR: Recursive Abstractive Processing for Tree\-Organized Retrieval\. In*International Conference on Learning Representations \(ICLR\)*\.
- Sarthi et al\.\(2024b\)Parth Sarthi, Salman Abdullah, Aditi Tuli, Shubh Khanna, Anna Goldie, and Christopher D Manning\. 2024b\.Raptor: Recursive abstractive processing for tree\-organized retrieval\. In*The Twelfth International Conference on Learning Representations*\.
- Tan et al\.\(2026\)Wei Tan, Yuanhao Li, and Wenkai Liang\. 2026\.Evo\-CuRL: Curriculum\-Aware Reinforcement Learning over Code Lineage Graphs for Software Engineering Reasoning\. In*ICMR 2026*\. 1327–1335\.[doi:10\.1145/3805622\.3810701](https://doi.org/10.1145/3805622.3810701)
- Trivedi et al\.\(2022\)Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal\. 2022\.MuSiQue: Multihop Questions via Single\-hop Question Composition\.*Trans\. Assoc\. Comput\. Linguistics*10 \(2022\), 539–554\.[doi:10\.1162/TACL\_A\_00475](https://doi.org/10.1162/TACL_A_00475)
- Trivedi et al\.\(2023\)Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal\. 2023\.Interleaving Retrieval with Chain\-of\-Thought Reasoning for Knowledge\-Intensive Multi\-Step Questions\. In*The 61st Annual Meeting Of The Association For Computational Linguistics*\.
- Wang et al\.\(2024\)Yu Wang, Nedim Lipka, Ryan A Rossi, Alexa Siu, Ruiyi Zhang, and Tyler Derr\. 2024\.Knowledge graph prompting for multi\-document question answering\. In*Proceedings of the AAAI Conference on Artificial Intelligence*, Vol\. 38\. 19206–19214\.
- Wei et al\.\(2026a\)Jiaqi Wei, Xuehang Guo, Pengfei Yu, Xiang Zhang, Wanli Ouyang, Siqi Sun, Qingyun Wang, and Chenyu You\. 2026a\.When to Think, When to Speak: Learning Disclosure Policies for LLM Reasoning\.*arXiv preprint arXiv:2605\.03314*\(2026\)\.
- Wei et al\.\(2022\)Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed Chi, Quoc V Le, and Denny Zhou\. 2022\.Chain\-of\-Thought Prompting Elicits Reasoning in Large Language Models\. In*Advances in Neural Information Processing Systems*, S\. Koyejo, S\. Mohamed, A\. Agarwal, D\. Belgrave, K\. Cho, and A\. Oh \(Eds\.\), Vol\. 35\. Curran Associates, Inc\., 24824–24837\.[https://proceedings\.neurips\.cc/paper\_files/paper/2022/file/9d5609613524ecf4f15af0f7b31abca4\-Paper\-Conference\.pdf](https://proceedings.neurips.cc/paper_files/paper/2022/file/9d5609613524ecf4f15af0f7b31abca4-Paper-Conference.pdf)
- Wei et al\.\(2025a\)Jiaqi Wei, Yuejin Yang, Xiang Zhang, Yuhan Chen, Xiang Zhuang, Zhangyang Gao, Dongzhan Zhou, Guangshuai Wang, Zhiqiang Gao, Juntai Cao, et al\.2025a\.From ai for science to agentic science: A survey on autonomous scientific discovery\.*arXiv preprint arXiv:2508\.14111*\(2025\)\.
- Wei et al\.\(2026c\)Jiaqi Wei, Xiang Zhang, Yuejin Yang, Wenxuan Huang, Juntai Cao, Sheng Xu, Xiang Zhuang, Zhangyang Gao, Muhammad Abdul\-Mageed, Laks VS Lakshmanan, et al\.2026c\.When LLM Meets Tree Search: A Systematic View of Inference as Search in Large Language Models\.*arXiv preprint arXiv:2608\.30395*\(2026\)\.
- Wei et al\.\(2025b\)Jiaqi Wei, Hao Zhou, Xiang Zhang, Di Zhang, Zijie Qiu, Noah Wei, Jinzhe Li, Wanli Ouyang, and Siqi Sun\. 2025b\.Retrieval is Not Enough: Enhancing RAG through Test\-Time Critique and Optimization\. In*The Thirty\-ninth Annual Conference on Neural Information Processing Systems*\.
- Wei et al\.\(2026b\)Kai Wei, Raymond Li, Xi Zhu, Zhaoqian Xue, Jiaojiao Han, Jingcheng Niu, and Fan Yang\. 2026b\.Skill\-RAG: Failure\-State\-Aware Retrieval Augmentation via Hidden\-State Probing and Skill Routing\.arXiv:2604\.15771 \[cs\.CL\][https://arxiv\.org/abs/2604\.15771](https://arxiv.org/abs/2604.15771)
- Xiao et al\.\(2025\)Yilin Xiao, Chuang Zhou, Yujing Zhang, Qinggang Zhang, Su Dong, Shengyuan Chen, Chang Yang, and Xiao Huang\. 2025\.Lag: Logic\-augmented generation from a cartesian perspective\.*arXiv preprint arXiv:2508\.05509*\(2025\)\.
- Xu et al\.\(2026a\)Chengdong Xu, Kaiqiang Ke, Ziheng Liu, Jiaqi Wei, Zibo Shao, Weile Guo, and Chao Yu\. 2026a\.EvoMAS: Learning Execution\-Time Workflows for Multi\-Agent Systems\.*arXiv preprint arXiv:2605\.08769*\(2026\)\.
- Xu et al\.\(2026b\)Nuoyue Xu, Jiang Liu, Wenxuan Huang, Xiang Zhang, Juntai Cao, and Jiaqi Wei\. 2026b\.Agent Memory with Episodic Retrieval for Financial Decision\-Making\.*arXiv preprint arXiv:2609\.28771*\(2026\)\.
- Yan et al\.\(2024\)Shi\-Qi Yan, Jia\-Chen Gu, Yun Zhu, and Zhen\-Hua Ling\. 2024\.Corrective Retrieval Augmented Generation\.[https://openreview\.net/forum?id=JnWJbrnaUE](https://openreview.net/forum?id=JnWJbrnaUE)
- Yang et al\.\(2018\)Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D\. Manning\. 2018\.HotpotQA: A Dataset for Diverse, Explainable Multi\-hop Question Answering\. In*Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing*, Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun’ichi Tsujii \(Eds\.\)\. Association for Computational Linguistics, Brussels, Belgium, 2369–2380\.[doi:10\.18653/v1/D18\-1259](https://doi.org/10.18653/v1/D18-1259)
- Yang et al\.\(2026\)Zhihan Yang, Jiaqi Wei, Xiang Zhang, Haoyu Dong, Yiwen Wang, Xiaoke Guo, Pengkun Zhang, Yiwei Xu, and Chenyu You\. 2026\.FORESTLLM: Large Language Models Make Random Forest Great on Few\-shot Tabular Learning\.*arXiv preprint arXiv:2601\.11311*\(2026\)\.
- Yu et al\.\(2024\)Wenhao Yu, Hongming Zhang, Xiaoman Pan, Peixin Cao, Kaixin Ma, Jian Li, Hongwei Wang, and Dong Yu\. 2024\.Chain\-of\-Note: Enhancing Robustness in Retrieval\-Augmented Language Models\. In*Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing*, Yaser Al\-Onaizan, Mohit Bansal, and Yun\-Nung Chen \(Eds\.\)\. Association for Computational Linguistics, Miami, Florida, USA, 14672–14685\.[doi:10\.18653/v1/2024\.emnlp\-main\.813](https://doi.org/10.18653/v1/2024.emnlp-main.813)
- Zhang et al\.\(2025a\)Xiang Zhang, Juntai Cao, Jiaqi Wei, Yiwei Xu, and Chenyu You\. 2025a\.Tokenization constraints in llms: A study of symbolic and arithmetic reasoning limits\.*arXiv preprint arXiv:2505\.14178*\(2025\)\.
- Zhang et al\.\(2025b\)Xiang Zhang, Juntai Cao, Jiaqi Wei, Chenyu You, and Dujian Ding\. 2025b\.Why does your cot prompt \(not\) work? theoretical analysis of prompt space complexity, its interaction with answer space during cot reasoning with llms: A recurrent perspective\.*arXiv preprint arXiv:2503\.10084*\(2025\)\.
- Zhang et al\.\(2025c\)Zhilin Zhang, Xiang Zhang, Jiaqi Wei, Yiwei Xu, and Chenyu You\. 2025c\.Postergen: Aesthetic\-aware paper\-to\-poster generation via multi\-agent llms\.*arXiv e\-prints*\(2025\), arXiv–2508\.
- Zhou et al\.\(2025\)Yuhao Zhou, Yiheng Wang, Xuming He, Ao Shen, Ruoyao Xiao, Zhiwei Li, Qiantai Feng, Zijie Guo, Yuejin Yang, Hao Wu, et al\.2025\.Scientists’ First Exam: Probing Cognitive Abilities of MLLM via Perception, Understanding, and Reasoning\.*arXiv preprint arXiv:2506\.10521*\(2025\)\.
- Zhuang et al\.\(2025\)Luyao Zhuang, Shengyuan Chen, Yilin Xiao, Huachi Zhou, Yujing Zhang, Hao Chen, Qinggang Zhang, and Xiao Huang\. 2025\.LinearRAG: Linear Graph Retrieval Augmented Generation on Large\-scale Corpora\.arXiv:2510\.10114 \[cs\.CL\][https://arxiv\.org/abs/2510\.10114](https://arxiv.org/abs/2510.10114)

## Appendix Overview

This appendix is organized as follows:

- •[Appendix A: Algorithm\.](https://arxiv.org/html/2610.21184#A1)Full inference procedure of ExecutionRAG, including register construction, register updates, progress\-delta injection, and budget\-aware control\.
- •[Appendix B: Extended Implementation Details\.](https://arxiv.org/html/2610.21184#A2)Additional details on model configuration, retrieval tools, budgets, deduplication, progress\-delta injection, and other engineering choices\.
- •[Appendix C: Per\-Category Breakdown\.](https://arxiv.org/html/2610.21184#A3)Category\-level results for HotpotQA, 2WikiMultiHopQA, and MuSiQue under both embedding backbones\.
- •[Appendix D: Retrieval Efficiency Analysis\.](https://arxiv.org/html/2610.21184#A4)Retrieved tokens and API cost across methods, ablations, and embedding backbones\.
- •[Appendix E: Failure Case Analysis\.](https://arxiv.org/html/2610.21184#A5)Error breakdown by type, hop count, and embedding model\.
- •[Appendix F: Case Study: How the Register Guides the Search\.](https://arxiv.org/html/2610.21184#A6)Qualitative examples showing how the register keeps the search on target and supports self\-correction\.
- •[Appendix G: Prompt Templates\.](https://arxiv.org/html/2610.21184#A7)Key prompts used in ExecutionRAG, including initialization, update, budget warning, and force\-answer prompts\.
- •[Appendix H: Broader Impacts\.](https://arxiv.org/html/2610.21184#A8)Discussion of potential applications, transparency benefits, and responsible usage considerations\.

## Appendix AAlgorithm

Algorithm 1ExecutionRAG Inference1:Query

qq, corpus

𝒦\\mathcal\{K\}, retrieval budget

BB
2:Final answer

aa
3:

𝒞\(0\)←Rseed​\(q,𝒦\)\\mathcal\{C\}^\{\(0\)\}\\leftarrow R\_\{\\text\{seed\}\}\(q;\\mathcal\{K\}\)
4:

𝒮0←InitializeState​\(q,𝒞\(0\)\)\\mathcal\{S\}\_\{0\}\\leftarrow\\textsc\{InitializeState\}\(q,\\mathcal\{C\}^\{\(0\)\}\)
5:Inject the initial register summary of

𝒮0\\mathcal\{S\}\_\{0\}into the agent context

6:for

t=1t=1to

BBdo

7:if

ReadyToAnswer​\(q,𝒮t−1\)\\textsc\{ReadyToAnswer\}\(q,\\mathcal\{S\}\_\{t\-1\}\)then

8:return

G⁡\(q,𝒮t−1\)G\(q,\\mathcal\{S\}\_\{t\-1\}\)
9:endif

10:

bt←SelectActiveRequirement​\(𝒮t−1\)b\_\{t\}\\leftarrow\\textsc\{SelectActiveRequirement\}\(\\mathcal\{S\}\_\{t\-1\}\)
11:

ut←Compose​\(q,dbt,ebt\(t−1\)\)u\_\{t\}\\leftarrow\\textsc\{Compose\}\(q,d\_\{b\_\{t\}\},e\_\{b\_\{t\}\}^\{\(t\-1\)\}\)
12:

Δ​𝒞t←R⁡\(ut,𝒦\)\\Delta\\mathcal\{C\}\_\{t\}\\leftarrow R\(u\_\{t\};\\mathcal\{K\}\)
13:if

Δ​𝒞t=∅\\Delta\\mathcal\{C\}\_\{t\}=\\varnothingthen

14:

𝒮t←𝒮t−1\\mathcal\{S\}\_\{t\}\\leftarrow\\mathcal\{S\}\_\{t\-1\}
15:else

16:

𝒮~t←U⁡\(𝒮t−1,Δ​𝒞t\)\\tilde\{\\mathcal\{S\}\}\_\{t\}\\leftarrow U\(\\mathcal\{S\}\_\{t\-1\},\\Delta\\mathcal\{C\}\_\{t\}\)
17:

𝒮t←ApplyInvariants​\(𝒮~t,𝒮t−1\)\\mathcal\{S\}\_\{t\}\\leftarrow\\textsc\{ApplyInvariants\}\(\\tilde\{\\mathcal\{S\}\}\_\{t\},\\mathcal\{S\}\_\{t\-1\}\)
18:endif

19:ifHasProgressDelta\(

𝒮t−1,𝒮t\\mathcal\{S\}\_\{t\-1\},\\mathcal\{S\}\_\{t\}\)then

20:Inject progress delta into the agent context

21:endif

22:if

t≥0\.8​Bt\\geq 0\.8Bthen

23:Inject budget\-aware nudge

24:endif

25:endfor

26:return

G⁡\(q,𝒮B\)G\(q,\\mathcal\{S\}\_\{B\}\)

## Appendix BExtended Implementation Details

#### Language model

All experiments use GPT\-4o\-mini as the default LLM\. Structured JSON calls for seed analysis and register updates usetemperature = 0\.0to reduce output variance, while the tool\-using agent loop uses the model’s default sampling settings\. The maximum output length for each LLM call is 16,384 tokens\.

#### Retrieval tools

The agent has access to three tools followed by A\-RAG\([Du et al\., 2026](https://arxiv.org/html/2610.21184#bib.bib6)\):semantic\_searchfor dense cosine retrieval with the active embedding model \(default top\-k=5k=5\),keyword\_searchfor BM25\-style lexical retrieval \(default top\-k=5k=5\), andread\_chunkfor direct chunk lookup by identifier\. Seed retrieval usessemantic\_searchwith top\-k=5k=5\. Retrieved chunks are deduplicated within each loop, and previously read chunks are not re\-exposed to the register update prompt\.

#### Embedding models

The primary embedding model isall\-MiniLM\-L6\-v2, a 384\-dimensional bi\-encoder optimized for semantic similarity\. The alternative backbone isQwen3\-Embedding\-0\.6B, a 1024\-dimensional embedding model\. Embeddings are pre\-computed and stored as flat similarity indices; no re\-indexing is performed between runs\.

#### Evaluation corpus

We follow the HippoRAG evaluation protocol, using the same retrieval corpora and the same 1,000\-question splits sampled from the official validation sets of HotpotQA, 2WikiMultiHopQA, and MuSiQue\. All baselines are evaluated on identical question sets\.

#### Retrieval budget

The agent is allowed at most 15 retrieval loops per question\. We also enforce a cumulative context budget of 128,000 tokens\. If this budget is exceeded, a force\-answer prompt is injected and further tool calls are disabled, yielding a budget\-exhausted answer, categorized as “Gave Up” in the error analysis\.

#### Search deduplication

Each retrieval action is cached by a normalized key: sorted keywords forkeyword\_searchand the lowercased query string forsemantic\_search\. If the agent issues an identical query, the system returns a deduplication notice with the previously retrieved chunk IDs, prompting the agent to read existing chunks or reformulate the query\. This reduces cyclic search and unnecessary token usage\.

#### Progress delta injection

Rather than replaying the full register after every step, the system injects a compact\[Progress\]delta only when at least one item changes materially\. The delta records the changed items with abbreviated evidence notes and a one\-line summary of the items still open\. On 4\-hop questions, this reduces register\-injection overhead by approximately 40% relative to replaying the full list\.

#### Budget\-aware nudge

When the agent reaches 80% of its loop budget, a soft warning is injected to encourage synthesis if the current evidence is sufficient, or to make the remaining retrieval steps more targeted\. This reduces redundant last\-mile searches near budget exhaustion\.

## Appendix CPer\-Category Breakdown

Tables[3](https://arxiv.org/html/2610.21184#A3.T3)–[8](https://arxiv.org/html/2610.21184#A3.T8)report per\-category Contain\-Match Accuracy and LLM\-Evaluation Accuracy \(%\) for ExecutionRAG with GPT\-4o\-mini on the full 1,000\-question evaluation sets, under two embedding backbones: all\-MiniLM\-L6\-v2 and Qwen3\-Embedding\-0\.6B\.

Table 3\.Per\-category accuracy onMuSiQueby hop count \(GPT\-4o\-mini \+ all\-MiniLM\-L6\-v2,n=1,000n\{=\}1\{,\}000\)\.Table 4\.Per\-category accuracy onHotpotQAby question type \(GPT\-4o\-mini \+ all\-MiniLM\-L6\-v2,n=1,000n\{=\}1\{,\}000\)\.Table 5\.Per\-category accuracy on2WikiMultiHopQAby question type \(GPT\-4o\-mini \+ all\-MiniLM\-L6\-v2,n=1,000n\{=\}1\{,\}000\)\.Table 6\.Per\-category accuracy onMuSiQueby hop count \(GPT\-4o\-mini \+ Qwen3\-Embedding\-0\.6B,n=1,000n\{=\}1\{,\}000\)\.Table 7\.Per\-category accuracy onHotpotQAby question type \(GPT\-4o\-mini \+ Qwen3\-Embedding\-0\.6B,n=1,000n\{=\}1\{,\}000\)\.Table 8\.Per\-category accuracy on2WikiMultiHopQAby question type \(GPT\-4o\-mini \+ Qwen3\-Embedding\-0\.6B,n=1,000n\{=\}1\{,\}000\)\.Figure 5\.Per\-category performance of ExecutionRAG under both embedding backbones\.We report Contain\-Match Accuracy and LLM\-Evaluation Accuracy on MuSiQue \(by hop count\), HotpotQA \(by question type\), and 2WikiMultiHopQA \(by question type\)\.Per\-category accuracy of ExecutionRAG under both embedding backbones: MuSiQue by hop count, HotpotQA by question type, and 2WikiMultiHopQA by question type\.
## Appendix DRetrieval Efficiency Analysis

We analyze the retrieval efficiency of ExecutionRAG relative to the strongest agentic baseline A\-RAG and to our ablation variants\. Unless otherwise specified, all main comparisons use GPT\-4o\-mini with all\-MiniLM\-L6\-v2 on the full 1,000\-question evaluation sets\.

Table 9\.Retrieval efficiency comparison on three benchmarks \(n=1,000n\{=\}1\{,\}000, GPT\-4o\-mini \+ all\-MiniLM\-L6\-v2\)\.Tokens: average retrieved tokens per question;Cost: average API cost per question \(USD\)\.Table 10\.Retrieval efficiency under ablation on MuSiQue \(n=1,000n\{=\}1\{,\}000, GPT\-4o\-mini \+ all\-MiniLM\-L6\-v2\)\. We report token consumption and cost alongside the accuracy numbers from[Table2](https://arxiv.org/html/2610.21184#S4.T2)\.Table 11\.Efficiency comparison across embedding models \(n=1,000n\{=\}1\{,\}000, GPT\-4o\-mini\)\. ExecutionRAG maintains its accuracy advantage with both embedding backbones, with comparable or fewer retrieved tokens on 2WikiMultiHopQA\.Figure 6\.Accuracy–efficiency trade\-off across ablation variantson MuSiQue, the same variants as Table[2](https://arxiv.org/html/2610.21184#S4.T2)\. Removing a register component lowers retrieved tokens and accuracy together\.Accuracy against retrieved tokens per question for the ablation variants on MuSiQue; removing register components lowers tokens and accuracy together\.
## Appendix EFailure Case Analysis

To better understand the remaining failure modes of ExecutionRAG, we conduct a systematic error analysis on MuSiQue \(n=1,000n\{=\}1\{,\}000\) and compare against the strongest agentic baseline, A\-RAG\. We report results under both embedding backbones to assess whether the observed patterns are robust to retriever choice\.

#### Annotation method

We leverage MuSiQue’s ground\-truth annotations, which provide for each question a set of*supporting paragraphs*corresponding to the minimal evidence needed for a correct answer\. We map each supporting paragraph to its corresponding corpus chunk using 40\-character fingerprint matching \(paragraph\-level recall: 87\.9%\), and compute*hop\-level evidence recall*as the fraction of supporting paragraphs for which at least one matching chunk was retrieved\. Each failure \(LLM\-Acc=0\\text\{LLM\-Acc\}\{=\}0\) is assigned to exactly one category using the following priority order:

1. \(1\)Judge Error\.Contain\-Acc=1\\text\{Contain\-Acc\}\{=\}1butLLM\-Acc=0\\text\{LLM\-Acc\}\{=\}0, indicating an evaluation mismatch\.
2. \(2\)Gave Up\.The predicted answer contains explicit abstention phrases \(e\.g\., “unable to find”, “not found”\)\.
3. \(3\)Reasoning Error\.Either \(a\) all supporting chunks were retrieved but the answer is still wrong, or \(b\) \[ExecutionRAG only\] the register marks the question sufficiently answered despite remaining evidence gaps \(*false resolution*\)\.
4. \(4\)Missing Evidence\.Some supporting evidence was never retrieved and no false resolution occurred\. This observable category includes both unreachable and unasked evidence\.

#### Reading the counts

All comparisons below are reported as within\-method percentages of each method’s own total failures\. The totals themselves come from this failure\-analysis pass, and for ExecutionRAG under all\-MiniLM\-L6\-v2 the total is slightly larger than the failure count implied by that method’s main\-table accuracy, so the reduction reported for that pairing is if anything conservative\.

#### Results \(all\-MiniLM\-L6\-v2\)

Table 12\.Failure case analysis on MuSiQue \(n=1,000n\{=\}1\{,\}000, GPT\-4o\-mini \+all\-MiniLM\-L6\-v2\)\. Percentages are relative to total failures for each method\.Table 13\.Failure distribution by hop count \(%, MiniLM\)\. ExecutionRAG reduces Missing Evidence at every hop level \(−25\.4\-25\.4pp overall\)\.
#### Results \(Qwen3\-Embedding\-0\.6B\)

Table 14\.Failure case analysis on MuSiQue \(n=1,000n\{=\}1\{,\}000, GPT\-4o\-mini \+Qwen3\-Embedding\-0\.6B\)\. Percentages are relative to total failures for each method\.Table 15\.Failure distribution by hop count \(%, Qwen\)\. ExecutionRAG reduces Missing Evidence at every hop level \(−18\.5\-18\.5pp overall\)\.
#### Key findings

First, Missing Evidence is reduced under both embedding backbones\.With all\-MiniLM\-L6\-v2, ExecutionRAG reduces Missing Evidence from 382 to 234 cases \(69\.3%→\\to43\.9% of each method’s failures\)\. With Qwen3\-Embedding\-0\.6B, it reduces the count from 383 to 274 \(66\.4%→\\to47\.9%\)\. This suggests that keeping the open sub\-questions on an explicit register improves evidence acquisition robustly across retrievers\.

Second, evidence recovery exposes a different remaining bottleneck\.As Missing Evidence decreases, Reasoning Error becomes more prominent for ExecutionRAG \(MiniLM: 9\.4%→\\to25\.9%; Qwen: 7\.8%→\\to21\.2%\)\. These cases include false resolution, where the register marks an item answered too optimistically, as well as cases where retrieved evidence is present but incorrectly synthesized\.

Third, abstention increases as the register makes control more explicit\.TheGave Uprate rises from 6\.5% to 13\.1% under MiniLM and from 12\.0% to 17\.3% under Qwen, indicating that ExecutionRAG is more willing to stop when the tracked evidence remains insufficient, rather than forcing a weakly supported answer\.

Finally, long\-hop retrieval remains the dominant bottleneck\.For ExecutionRAG, Missing Evidence still increases with hop count, reaching 55\.3% on 4\-hop MuSiQue under MiniLM\. This suggests that although the register substantially improves the search, acquiring long and weakly connected evidence chains remains the main frontier for further progress\.

## Appendix FCase Study: How the Register Guides the Search

To give qualitative intuition for how the register keeps the search on target and heads off common failures, we show three representative examples from MuSiQue \(GPT\-4o\-mini \+ all\-MiniLM\-L6\-v2\)\. The\[Progress\]annotations in the ExecutionRAG column are the delta messages the agent actually sees: only register changes that materially affect the search are surfaced\. The three cases illustrate three effects: anchoring on the right entity \(Case 1\), completing a long chain via progress deltas \(Case 2\), and self\-correction by dropping a wrong item \(Case 3\)\.

### Case 1: Preventing Entity Confusion \(2\-hop, Success\)

Question:*Who married the publisher of abolitionist newspaper The North Star?* Gold answer:Helen Pitts Douglass

Table 16\.Case 1: ExecutionRAG’s seed\-grounded register anchors the correct entity, Frederick Douglass, before iterative retrieval begins\. A\-RAG instead latches onto William Cooper Nell, a contributor rather than the publisher, and continues retrieval along the wrong branch\.Analysis\.This example illustrates the value of*anchoring on the right entity*at the start\. The seed\-grounded register records Frederick Douglass as the entity the answer chain must stay attached to, so the later search stays on the correct branch\. A\-RAG has no explicit record of which entity the question has already grounded; once it latches onto Nell, later retrieval follows that mistaken branch and never recovers\.

### Case 2: Completing a Long Dependency Chain \(4\-hop, Success\)

Question:*What day is the feast held, in the city that holds the headquarters of the only music company that is larger than the record label of Vilaiyaadu Mankatha?* Gold answer:May 4

Table 17\.Case 2: ExecutionRAG resolves a long dependency chain by turning register updates into targeted follow\-up retrieval\. Once the register records “Universal Music Group” and “Santa Monica,” the next search becomes sharply focused\. A\-RAG lacks an explicit record of this intermediate progress and keeps issuing weakly constrained searches\.Analysis\.This case shows how register updates turn a long multi\-hop problem into a sequence of targeted retrieval steps\. The key advantage is not decomposition alone, but the fact that newly answered items are surfaced as*progress deltas*, which immediately sharpen the next search\. Once “Santa Monica” is recorded on the register, the next retrieval step directly targets the remaining gap\. Without such a register, A\-RAG repeatedly searches in a much flatter space and accumulates substantial irrelevant context before the decisive entity is stabilized\.

### Case 3: Self\-Correction via Requirement Retirement \(4\-hop, Success\)

Question:*When did the birthplace of Ah Via Musicom’s performer become the capital of the state Knowles is from?* Gold answer:1839

Table 18\.Case 3: ExecutionRAG uses theretiredtransition to explicitly discard an invalid requirement when later evidence contradicts its premise\. A\-RAG reads overlapping evidence but lacks an explicit mechanism for requirement revision, and therefore closes on a plausible but incorrect year\.Analysis\.This example highlights*self\-correction by dropping a wrong item*\. Both systems initially move toward the wrong city, but ExecutionRAG can explicitly retire the invalid item once later evidence contradicts it\. Dropping it is recorded as an explicit event rather than silently overwritten, which forces the search to reopen the chain with a corrected target\. A\-RAG has no explicit way to invalidate an earlier intermediate commitment, so its trajectory closes on a plausible but unsupported date\.

#### Summary

These three cases highlight the main qualitative effects of the register:

- •Anchoring on the right entity\(Case 1\): building the register from seed evidence keeps retrieval on the correct entity and prevents early drift onto a wrong branch\.
- •Delta\-guided chain completion\(Case 2\): answered items are surfaced as progress deltas, enabling sharply targeted follow\-up retrieval instead of repeated weakly constrained search\.
- •Self\-correction by dropping a wrong item\(Case 3\): contradictory evidence can explicitly invalidate an earlier item, letting the agent revise the chain instead of closing on a stale assumption\.

## Appendix GPrompt Templates

This section lists the key prompts used in ExecutionRAG\. All prompts are invoked with GPT\-4o\-mini unless otherwise stated\.

### A\.1 Default QA Agent System Prompt

Used to initialize the tool\-using retrieval agent\. The agent iteratively callskeyword\_search,semantic\_search, andread\_chunkwhile using the register as guidance\.

### A\.2 Seed\-Grounded Requirement Initialization Prompt

Run once before iterative retrieval begins\. Given the question and seed evidence, this prompt builds the initial register \(stored as therequirement\_register\)\.

### A\.3 Requirement\-State Injection

Appended to the agent context after warm\-up\. It exposes the current register as guidance without overriding the agent’s autonomy\.

### A\.4 Requirement Register Update Prompt

Called after each retrieval loop to update the register\. The LLM decides whether new evidence advances, resolves, or invalidates each tracked item\.

### A\.5 Force Final Answer Prompt

Appended when the agent reaches the maximum loop or token budget\. It disables further tool calls and forces the agent to answer from the current evidence state\.

Force Final Answer PromptYou have reached the budget limit\. You MUST now provide a final answer based on the information gathered so far\. Do NOT call any more tools\. If the evidence is sufficient, answer directly\. If the evidence is still incomplete, provide the best bounded answer you can and clearly state the remaining uncertainty\.

## Appendix HBroader Impacts

Our work improves multi\-hop QA by keeping an explicit register of open sub\-questions during multi\-step retrieval\. Related language\-model and agent systems have been studied for autonomous scientific discovery\([Wei et al\., 2025a](https://arxiv.org/html/2610.21184#bib.bib40)\), few\-shot tabular learning\([Yang et al\., 2026](https://arxiv.org/html/2610.21184#bib.bib49)\), paper\-to\-poster generation\([Zhang et al\., 2025c](https://arxiv.org/html/2610.21184#bib.bib53)\), and benchmarks that probe the cognitive abilities such systems require\([Zhou et al\., 2025](https://arxiv.org/html/2610.21184#bib.bib54)\)\. These settings illustrate applications in which an agent must maintain intermediate requirements while integrating information from multiple sources\. By making evidence support and provenance explicit, ExecutionRAG can improve transparency and auditability in such workflows\.

However, the method still depends on the underlying retriever and LLM, so biases or errors in source documents may propagate\. The framework could be misused in automated decision\-making systems without human oversight, producing overconfident answers based on incomplete evidence\. We recommend deployment in settings where accountability and verification are feasible\.