Verification Without Sufficiency: Per-Chunk Filtering Fails on Multi-Hop RAG, and Decomposition Repairs It

# Per-Chunk Filtering Fails on Multi-Hop RAG, and Decomposition Repairs It
Source: [https://arxiv.org/html/2608.00585](https://arxiv.org/html/2608.00585)
## Verification Without Sufficiency: Per\-Chunk Filtering Fails on Multi\-Hop RAG, and Decomposition Repairs It

###### Abstract

Verification for retrieval\-augmented generation usually scores each retrieved chunk and drops the ones that fail\. We show this cannot work for multi\-hop questions, and show what does\. Per\-chunk scoring assumes one chunk is a sufficient premise for the answer\. Multi\-hop questions are built so that none is, and the paragraph carrying the answer is the one the question does not name\. Entailment scoring reaches 0\.643, 0\.523 and 0\.560 AUC on HotpotQA, 2WikiMultihopQA and MuSiQue, against 0\.951 on single\-hop SQuAD\. Seven controls rule out model capacity, premise length, hypothesis template, decision threshold, retriever, answer\-matching criterion and prompt\. End to end across three datasets, three generator sizes and two prompts, per\-chunk gating is significantly worse than not filtering at all in every cell, and its penalty grows with generator capability\. The repair is to condition verification on the decomposed sub\-question rather than the original query\. Using MuSiQue’s gold decomposition, entailment on a later hop rises from 0\.546, which is chance, to 0\.840, a paired lift of\+0\.355\+0\.355with a bootstrap interval of\[0\.331,0\.382\]\[0\.331,0\.382\]\. An off\-the\-shelf Qwen2\.5\-7B decomposer, given the question and the top retrieved paragraph, reaches 0\.637 and captures 31% of that ceiling; decomposing without retrieval reaches 0\.533, below the original question\. Iterative retrieval systems already produce such decompositions and discard them before verifying\.

## IIntroduction

Retrieval\-augmented generation grounds a language model in a corpus by retrieving passages and conditioning generation on them\[[1](https://arxiv.org/html/2608.00585#bib.bib1),[2](https://arxiv.org/html/2608.00585#bib.bib2)\]\. Retrievers make mistakes, and a generator handed a topically adjacent but factually unrelated passage will use it anyway\. The standard response is a verification step: score each retrieved chunk, keep the good ones, drop the rest\. CRAG\[[3](https://arxiv.org/html/2608.00585#bib.bib3)\]trains an evaluator that labels chunks correct, incorrect or ambiguous\. Self\-RAG\[[4](https://arxiv.org/html/2608.00585#bib.bib4)\]trains the generator to emit reflection tokens\. Faithfulness metrics such as RAGAS\[[5](https://arxiv.org/html/2608.00585#bib.bib5)\]score entailment between answer and context\.

We built such a verifier with three signals, embedding cosine similarity, natural language inference entailment, and a structural role\-filler score based on Holographic Reduced Representations\[[17](https://arxiv.org/html/2608.00585#bib.bib17)\]\. Before evaluating end to end we measured the thing a verifier has to do: separate gold evidence from distractors\. On HotpotQA the answer was 0\.887 for embedding similarity, 0\.643 for entailment and 0\.620 for the structural score\. Since embedding similarity is what the retriever already computes, the two signals we added contributed nothing\.

This paper explains why, and the explanation turns out not to be about our implementation\. Per\-chunk verification treats each chunk as a sufficient premise for the answer\. Multi\-hop questions are constructed so that no single retrieved paragraph is sufficient, and the paragraph that carries the answer is the one the question does not name\. A verifier conditioned on the question is therefore strongest on the evidence already in hand and weakest on the evidence being sought\.

We establish this with seven measurements, each closing an alternative account:

1. 1\.It is not one dataset\.Entailment reaches 0\.643, 0\.523 and 0\.560 AUC on HotpotQA, 2WikiMultihopQA and MuSiQue \(Section[IV](https://arxiv.org/html/2608.00585#S4)\)\.
2. 2\.The failure has a direction\.All three signals prefer the paragraph named in the question\. On comparison questions, where both entities are named, the deficit disappears \(Section[V](https://arxiv.org/html/2608.00585#S5)\)\.
3. 3\.It deepens with hop count\.On MuSiQue, AUC falls monotonically from 2\-hop to 4\-hop questions \(Section[VI](https://arxiv.org/html/2608.00585#S6)\)\.
4. 4\.It is sufficiency, not length\.Both gold paragraphs together reach 0\.881 AUC; a gold paired with a distractor scores 0\.127 against 0\.540, at greater length \(Section[VII](https://arxiv.org/html/2608.00585#S7)\)\.
5. 5\.It is not the task\.On single\-hop SQuAD the same pipeline reaches 0\.951 \(Section[VIII](https://arxiv.org/html/2608.00585#S8)\)\.
6. 6\.No threshold rescues it\.At the most permissive threshold tested, 83% of gold paragraphs are already rejected \(Section[IX](https://arxiv.org/html/2608.00585#S9)\)\.
7. 7\.It is not the embedder\.The deficit holds for three embedding models \(Section[IX](https://arxiv.org/html/2608.00585#S9)\)\.

Section[XI](https://arxiv.org/html/2608.00585#S11)measures what this costs\. Across three datasets \(500, 500 and 259 evaluated questions\), three generator sizes and two prompts, per\-chunk gating is the worst of seven selectors in every cell, and the penalty grows with generator capability\.

Section[XII](https://arxiv.org/html/2608.00585#S12)shows what repairs it\. Conditioning the hypothesis on the decomposed sub\-question rather than the original query lifts entailment on a later hop from 0\.546 to 0\.840, and hop count stops mattering\. This is the positive result of the paper, and it says where verification for multi\-hop retrieval should be built: on the same decomposition that iterative retrieval already produces\.

## IIRelated Work

#### Corrective and self\-critiquing RAG

CRAG\[[3](https://arxiv.org/html/2608.00585#bib.bib3)\]adds a trained retrieval evaluator that branches generation on a per\-chunk verdict\. Self\-RAG\[[4](https://arxiv.org/html/2608.00585#bib.bib4)\]trains the model to critique its own retrieval during decoding\. Both operate on individual passages and are evaluated on tasks that are largely single\-hop\. Our results suggest the per\-chunk granularity is the part that does not transfer\.

#### NLI for grounding

Entailment models\[[6](https://arxiv.org/html/2608.00585#bib.bib6),[7](https://arxiv.org/html/2608.00585#bib.bib7),[8](https://arxiv.org/html/2608.00585#bib.bib8)\]are widely used to check whether generated text is supported by retrieved context\[[5](https://arxiv.org/html/2608.00585#bib.bib5)\]\. That use is post\-hoc and set\-level: the premise is the whole context the generator saw\. Moving the check earlier, to filter chunks, changes the premise to a single passage\. It is that change our measurements isolate\.

#### Multi\-hop retrieval

HotpotQA\[[9](https://arxiv.org/html/2608.00585#bib.bib9)\], 2WikiMultihopQA\[[10](https://arxiv.org/html/2608.00585#bib.bib10)\]and MuSiQue\[[11](https://arxiv.org/html/2608.00585#bib.bib11)\]supply gold supporting paragraphs alongside hard distractors, and MuSiQue is constructed specifically to resist single\-hop shortcuts\. Iterative methods such as Self\-Ask\[[13](https://arxiv.org/html/2608.00585#bib.bib13)\]and IRCoT\[[14](https://arxiv.org/html/2608.00585#bib.bib14)\]rewrite the query between hops because the second hop is not reachable from the original question\. Our findings say the same thing about verification that those methods say about retrieval, and we quantify it\.

#### Holographic representations

HRRs encode role\-filler structure in fixed\-width vectors via circular convolution\[[17](https://arxiv.org/html/2608.00585#bib.bib17),[18](https://arxiv.org/html/2608.00585#bib.bib18)\], a construction shared with the wider vector\-symbolic literature\[[19](https://arxiv.org/html/2608.00585#bib.bib19),[20](https://arxiv.org/html/2608.00585#bib.bib20)\]\. We are not aware of prior work using HRR as a chunk verification signal, and Section[X](https://arxiv.org/html/2608.00585#S10)reports why it does not work\.

## IIISetup

### III\-AThree verification schemes

Figure[1](https://arxiv.org/html/2608.00585#S3.F1)contrasts the three ways of scoring evidence that this paper compares\. All of them use the same models; they differ only in what constitutes a premise\.

\(a\) per chunkpremise = one paragraphc1c\_\{1\}hop 1c2c\_\{2\}hop 2c3c\_\{3\}c4c\_\{4\}sent​\(q,ci\)s\_\{\\mathrm\{ent\}\}\(q,c\_\{i\}\)c2c\_\{2\}alone does notentail the answerkeep / drop\(b\) set levelpremise = every pair,\(k2\)\\binom\{k\}\{2\}callsc1c\_\{1\}c2c\_\{2\}c3c\_\{3\}c4c\_\{4\}sent​\(q,ci⊕cj\)s\_\{\\mathrm\{ent\}\}\(q,c\_\{i\}\\\!\\oplus\\\!c\_\{j\}\)argmax over 10 pairslands correctly 44% of the timebest pair\(c\) conditional \(ours\)anchor on hop 1,k−1k\-1callsc1c\_\{1\}anchorc2c\_\{2\}c3c\_\{3\}c4c\_\{4\}arg⁡max⁡semb\\arg\\max s\_\{\\mathrm\{emb\}\}92% goldsent​\(q,c1⊕cj\)s\_\{\\mathrm\{ent\}\}\(q,c\_\{1\}\\\!\\oplus\\\!c\_\{j\}\)recall0\.7550\.755vs0\.6830\.683at 4 calls not 10completion

Figure 1:Three ways to score retrieved evidence\. Only the premise changes\. \(a\) Per\-chunk scoring asks whether one paragraph entails the answer, which for the second hop of a multi\-hop question it cannot\. \(b\) Set\-level scoring restores sufficiency but searches blindly\. \(c\) Conditional scoring anchors on the paragraph the embedding signal identifies reliably, then asks which paragraph completes it\. Numbers are measured on HotpotQA, 500 questions\.
### III\-BThe three signals

Given a questionqqand a chunkcc:

#### Embedding

Cosine similarity between bi\-encoder embeddings \(BAAI/bge\-small\-en\-v1\.5, 33 M\), rescaled to\[0,1\]\[0,1\]:semb​\(q,c\)=12​\(1\+cos⁡\(E​\(q\),E​\(c\)\)\)s\_\{\\mathrm\{emb\}\}\(q,c\)=\\tfrac\{1\}\{2\}\\big\(1\+\\operatorname\{cos\}\(E\(q\),E\(c\)\)\\big\)\.

#### Entailment

A rule\-based operatorℋ\\mathcal\{H\}converts the interrogative into a declarative proposition and a cross\-encoder scores entailment with the chunk as premise,sent​\(q,c\)=P​\(entail∣c,ℋ​\(q\)\)s\_\{\\mathrm\{ent\}\}\(q,c\)=P\(\\text\{entail\}\\mid c,\\mathcal\{H\}\(q\)\)\. Gettingℋ\\mathcal\{H\}right matters more than it appears\. A hypothesis phrased as a statement about the passage, of the form “this text contains information aboutqq”, is out of distribution for a model trained on SNLI or MNLI, where premise and hypothesis are both object\-level claims, and it collapses into a topicality detector\. We generate object\-level propositions with the answer slot underspecified: “Who directed Inception?” becomes “a person directed Inception\.” We reportslotas above andoracle, which fills the slot with the gold answer and therefore upper\-bounds any deployable version of the signal\.

#### Structure

Role\-filler pairs over\{subj,obj,root,ent\}\\\{\\textsc\{subj\},\\textsc\{obj\},\\textsc\{root\},\\textsc\{ent\}\\\}are bound with circular convolution into a trace and recovered by unbinding:

T​\(c\)\\displaystyle T\(c\)=∑r∈ℛc𝐯r⊛𝐯ϕc​\(r\),\\displaystyle=\\textstyle\\sum\_\{r\\in\\mathcal\{R\}\_\{c\}\}\\mathbf\{v\}\_\{r\}\\circledast\\mathbf\{v\}\_\{\\phi\_\{c\}\(r\)\},\(1\)sstr​\(q,c\)\\displaystyle s\_\{\\mathrm\{str\}\}\(q,c\)=1\|ℛq\|​∑r∈ℛqmax⁡\(0,cos⁡\(𝐯r†⊛T​\(c\),𝐯ϕq​\(r\)\)\)\.\\displaystyle=\\tfrac\{1\}\{\|\\mathcal\{R\}\_\{q\}\|\}\\textstyle\\sum\_\{r\\in\\mathcal\{R\}\_\{q\}\}\\max\\\!\\big\(0,\\operatorname\{cos\}\(\\mathbf\{v\}\_\{r\}^\{\\dagger\}\\circledast T\(c\),\\mathbf\{v\}\_\{\\phi\_\{q\}\(r\)\}\)\\big\)\.\(2\)Fillers are whitened before binding, since raw sentence embeddings are anisotropic and violate the near\-orthogonality binding assumes\. Whitening reduces mean pairwise cosine from0\.4880\.488to−0\.0001\-0\.0001\.

### III\-CData

Three multi\-hop datasets, 500 questions each, sampled with a fixed seed: HotpotQA\[[9](https://arxiv.org/html/2608.00585#bib.bib9)\]distractor \(10 paragraphs per question, 2\.0 gold\), 2WikiMultihopQA\[[10](https://arxiv.org/html/2608.00585#bib.bib10)\]\(10 paragraphs, 2\.5 gold\) and MuSiQue\[[11](https://arxiv.org/html/2608.00585#bib.bib11)\]\(20 paragraphs, 2\.6 gold, and 2, 3 or 4 hops\)\. Retrieval is per\-question over the supplied candidates, so no global index is involved\. All 10 or 20 paragraphs are scored, giving 20 000 question\-paragraph pairs\.

For the single\-hop control we use SQuAD v1\.1\[[12](https://arxiv.org/html/2608.00585#bib.bib12)\], with distractors drawn from other paragraphs of the same Wikipedia article so they remain hard negatives\. 300 questions, nine distractors each\.

### III\-DModels and protocol

Entailment usesnli\-deberta\-v3\-base\(184 M\) unless noted; we also testxsmall\(44 M\) and a 435 M DeBERTa\-v3\-large\. Before use we verified each model’s entailment logit index against two unambiguous NLI pairs; all three assign the correct label with margins above0\.950\.95, so no result below is a label\-mapping artefact\. Generation uses Qwen2\.5\-1\.5B\-Instruct\[[16](https://arxiv.org/html/2608.00585#bib.bib16)\]with greedy decoding and seed 1337\.

We report AUC of separating gold paragraphs from distractors\. AUC is threshold\-free, and a verifier that cannot rank gold above distractor cannot be rescued by tuning a cutoff\. Section[IX](https://arxiv.org/html/2608.00585#S9)confirms that directly\.

2233440\.50\.50\.60\.60\.70\.70\.80\.8chance\.819\.780\.677\.645\.606\.592\.590\.550\.542hops required to answerAUC, gold vs distractorsembs\_\{\\mathrm\{emb\}\}embeddingsents\_\{\\mathrm\{ent\}\}oraclesents\_\{\\mathrm\{ent\}\}slot\(deployable\)Figure 2:MuSiQue, 500 questions split by hop count \(259 / 163 / 78\), with question\-stratified bootstrap intervals on the embedding curve\. Embedding similarity degrades significantly with hop count, from 0\.819 \[0\.802, 0\.837\] at two hops to 0\.677 \[0\.646, 0\.709\] at four\. The entailment curves also fall but their intervals overlap, so we report the trend without claiming it\. The deployable entailment variant is within 0\.04 of chance at four hops\.

## IVThe signals are weak on every multi\-hop dataset

TABLE I:Separating gold paragraphs from distractors\. 500 questions per dataset\.*deficit*is AUC on bridge\-only gold minus AUC on answer\-bearing gold; positive means the signal prefers the paragraph without the answer\.DatasetSignalAUCanswer\-bearingbridgeonlyHotpotQAsembs\_\{\\mathrm\{emb\}\}0\.8870\.8490\.941sents\_\{\\mathrm\{ent\}\}oracle0\.6690\.6810\.652sents\_\{\\mathrm\{ent\}\}slot0\.6430\.6350\.6552Wikisembs\_\{\\mathrm\{emb\}\}0\.8070\.8400\.788sents\_\{\\mathrm\{ent\}\}oracle0\.6140\.6560\.589sents\_\{\\mathrm\{ent\}\}slot0\.5230\.5480\.507MuSiQuesembs\_\{\\mathrm\{emb\}\}0\.7620\.7020\.798sents\_\{\\mathrm\{ent\}\}oracle0\.6320\.6880\.597sents\_\{\\mathrm\{ent\}\}slot0\.5600\.5900\.541HotpotQA,sstrs\_\{\\mathrm\{str\}\}chunk whitened0\.6200\.5600\.696HotpotQA,sstrs\_\{\\mathrm\{str\}\}sentence whitened0\.5950\.5360\.677Table[I](https://arxiv.org/html/2608.00585#S4.T1)is the result that redirected this work\. Embedding similarity separates gold from distractor at 0\.76 to 0\.89\. Neither signal we added reaches 0\.67 on any dataset, and the best entailment variant is the one handed the gold answer\.

That last point deserves emphasis\.oracleis not deployable; it exists to bound the signal\. If entailment cannot exceed 0\.669 even when the hypothesis contains the answer verbatim, no amount of work on hypothesis generation will help, because hypothesis generation is trying to approximate exactly that\.

Scaling the entailment model does not rescue it either\. On a 200\-pair calibration subset the 44 M model reaches 0\.590oracleAUC, the 184 M model 0\.668, and the 435 M model 0\.543\. We do not read much into the ordering at that sample size, but a tenfold parameter increase producing no improvement is itself informative\.

## VThe failure has a direction

TABLE II:AUC by question type\. 2Wiki labels four types, HotpotQA two\. Comparison questions name both entities; compositional and inference questions hide the second hop behind the first\.Type \(nn\)Signalanswer\-bearingbridgeonlydeficit2Wiki comparison \(138\)sembs\_\{\\mathrm\{emb\}\}0\.9740\.976\+0\.002\+0\.002sents\_\{\\mathrm\{ent\}\}0\.6950\.493−0\.203\-0\.2032Wiki compositional \(189\)sembs\_\{\\mathrm\{emb\}\}0\.7080\.979\+0\.271\\mathbf\{\+0\.271\}sents\_\{\\mathrm\{ent\}\}0\.4690\.658\+0\.189\+0\.1892Wiki inference \(54\)sembs\_\{\\mathrm\{emb\}\}0\.6830\.972\+0\.289\\mathbf\{\+0\.289\}sents\_\{\\mathrm\{ent\}\}0\.4650\.594\+0\.128\+0\.128HotpotQA bridge \(543\)sembs\_\{\\mathrm\{emb\}\}0\.7960\.927\+0\.131\+0\.131sents\_\{\\mathrm\{ent\}\}0\.6050\.695\+0\.090\+0\.090HotpotQA comparison \(157\)sembs\_\{\\mathrm\{emb\}\}0\.9720\.969−0\.003\-0\.003sents\_\{\\mathrm\{ent\}\}0\.6160\.513−0\.103\-0\.103Multi\-hop gold evidence comes in two kinds\. In a bridge question such as*“What government position was held by the woman who portrayed Corliss Archer in Kiss and Tell?”*, one paragraph is about*Kiss and Tell*and supplies the bridge entity, and the other is about Shirley Temple and supplies the answer\. The first is named in the question\. The second is not, and is reachable only through the first\.

We label a gold paragraph*answer\-bearing*if the normalised gold answer occurs in it, and*bridge\-only*otherwise\. On HotpotQA, 58% of gold paragraphs are answer\-bearing and 502 of 700 questions have exactly one\.

#### The entailment result is the surprising one

Theoraclehypothesis contains the gold answer string, and the answer\-bearing paragraph contains that same string\. Lexical overlap alone should push entailment higher there\. It does not: 0\.681 \[0\.659, 0\.704\] on answer\-bearing gold against 0\.652 \[0\.625, 0\.679\] on bridge\-only gold for HotpotQA, and on 2Wiki and MuSiQue the deployable variant is barely above chance on either\. An entailment model handed the answer cannot reliably tell which paragraph contains it, because the proposition it is checking requires both hops\.

#### The embedding result is the expected baseline

Thatsembs\_\{\\mathrm\{emb\}\}prefers the bridge paragraph is closer to a restatement of what embedding similarity measures: the question names the bridge entity, so the question embedding sits near the paragraph about it\. We report it because it calibrates the entailment numbers, not as a discovery\. Its magnitude is nonetheless large,\+0\.271\+0\.271on 2Wiki compositional questions, where the bridge paragraph separates at 0\.979 and the answer\-bearing paragraph at 0\.708\.

#### Comparison questions are the control

On 2Wiki comparison questions, which name both entities, the embedding deficit collapses to\+0\.002\+0\.002and absolute AUC rises to 0\.974\. HotpotQA reproduces the contrast,\+0\.131\+0\.131against−0\.003\-0\.003\. Nothing about the models, the corpus or the retriever changes between those rows\. Only the relationship between the question and the evidence changes\.

#### A note on aggregation

The 2Wiki row of Table[I](https://arxiv.org/html/2608.00585#S4.T1)shows a*negative*aggregate deficit,−0\.052\-0\.052, while Table[II](https://arxiv.org/html/2608.00585#S5.T2)shows\+0\.271\+0\.271and\+0\.289\+0\.289on its compositional and inference subsets\. There is no inconsistency: 2Wiki contains 138 comparison questions whose deficit is strongly negative, and they pull the pooled figure below zero\. The aggregate is a mixture over question types with opposite signs and should not be read on its own\. HotpotQA, with a smaller comparison share, does not invert\.

#### A worked example

*“What government position was held by the woman who portrayed Corliss Archer in Kiss and Tell?”*The bridge paragraph is titled*Kiss and Tell*; its title appears verbatim in the question\. The answer\-bearing paragraph is titled*Shirley Temple*, a name the question never uses\. Averaged over the 543 HotpotQA bridge questions, the bridge paragraph is separated from distractors at 0\.927 by embedding and 0\.695 by entailment; the answer\-bearing paragraph at 0\.796 and 0\.605\. The gap runs the same way for all three signals\.

## VIIt deepens with hop count

MuSiQue labels each question with the number of hops its answer requires, which turns the mechanism into a dose\-response test\. If the problem is that query\-conditioned signals cannot see past the first hop, then adding hops should make it worse\.

Figure[2](https://arxiv.org/html/2608.00585#S3.F2)shows it does for the embedding signal, where the intervals separate cleanly: 0\.819 \[0\.802, 0\.837\] at two hops against 0\.677 \[0\.646, 0\.709\] at four\. The entailment signals fall in the same direction, from 0\.590 to 0\.542 for the deployable variant and 0\.645 to 0\.592 for the oracle, but their intervals overlap and we do not claim a decline for them\. The reason they cannot fall far is that they start close to chance: at two hops the deployable variant is already at 0\.590, so there is little room left\. The clean version of the dose\-response test appears in Section[XII](https://arxiv.org/html/2608.00585#S12), where removing the mechanism lifts every hop count to the same value\.

## VIISufficiency, not length

gold\+goldgold\+distr\.distr\.\+distr\.00\.20\.20\.40\.40\.60\.60\.5400\.1270\.025meanP​\(entail\)P\(\\text\{entail\}\)120120140140160160180180200200220220premise length \(words\)premise lengthFigure 3:HotpotQA, 200 questions, all premises two paragraphs\. Entailment falls as premise length rises, so the set\-level effect is not a length artefact\. What matters is whether both hops are present\.If per\-chunk scoring fails because a chunk is not a sufficient premise, then supplying both hops should repair it\. It does\. Holding the model, the questions and theoraclehypothesis fixed and changing only the premise, scoring single chunks gives 0\.664 AUC and scoring the concatenation of both gold paragraphs against other pairs gives0\.881, a lift of\+0\.217\+0\.217\. A gold chunk alone receives 0\.103 mean entailment probability; a gold pair receives 0\.560\.

The obvious alternative is that longer premises simply score higher\. Figure[3](https://arxiv.org/html/2608.00585#S7.F3)rules it out in the strongest available direction: the relationship runs backwards\. Gold pairs are the*shortest*premises at 148 words and score 0\.540\. Gold with a distractor averages 172 words and scores 0\.127\. Two distractors average 198 words and score 0\.025\. More text, less entailment\.

The gold\-plus\-distractor row matters most\. It contains a gold paragraph, it is longer than the gold pair, and it still scores four times lower\. Half the evidence is not most of the way to the answer\.

The deployableslothypothesis behaves the same way at set level, reaching 0\.811 AUC over pairs drawn from the top five retrieved chunks against 0\.620 per chunk\. The gap to its 0\.848 oracle is0\.0370\.037, so the set\-level effect does not depend on seeing the answer\.

## VIIIThe single\-hop control

TABLE III:Single\-hop control\. SQuAD v1\.1, 300 questions, nine same\-article distractors each, identical models and hypothesis construction\.SignalSQuADHotpotQAsembs\_\{\\mathrm\{emb\}\}embedding cosine0\.9330\.887sents\_\{\\mathrm\{ent\}\}slot\(deployable\)0\.9270\.643sents\_\{\\mathrm\{ent\}\}oracle0\.9510\.669meanP​\(entail\)P\(\\text\{entail\}\), gold0\.6930\.103meanP​\(entail\)P\(\\text\{entail\}\), distractor0\.015n/aEverything so far is consistent with a competing account we have not yet excluded: that entailment scoring of retrieved passages simply does not work at this model scale, and multi\-hop structure is incidental\.

Table[III](https://arxiv.org/html/2608.00585#S8.T3)excludes it\. On SQuAD, where the gold paragraph contains the answer and distractors come from the same article, the identical pipeline reaches 0\.951 AUC withoracleand 0\.927 with the deployableslothypothesis\. Gold paragraphs receive 0\.693 mean entailment probability, distractors 0\.015\.

Two things follow\. Entailment verification works, and works well, when the sufficiency assumption holds\. And the deployable variant sits 0\.024 behind the oracle on SQuAD and 0\.026 behind on HotpotQA, so the hypothesis template is close to its ceiling in both cases; the ceilings differ by 0\.282\.

## IXNot the threshold, not the embedder

5⋅10−25\\cdot 10^\{\-2\}0\.10\.10\.150\.150\.20\.20\.250\.250\.30\.30\.350\.350\.40\.40\.450\.450\.50\.50\.550\.550\.60\.60\.650\.650\.70\.70\.750\.750\.80\.800\.20\.20\.40\.40\.60\.60\.80\.811per\-chunk thresholdτ\\tauratedistractors rejectedbridge\-only gold keptanswer\-bearing gold keptFigure 4:HotpotQA\. Sweeping the per\-chunk threshold moves nothing\. Even atτ=0\.10\\tau=0\.10, 84% of answer\-bearing gold paragraphs are already rejected\. The entailment score distribution is collapsed near zero, so there is no operating point to find\.#### No threshold works

A natural objection is that our per\-chunk gate simply used a bad cutoff\. Figure[4](https://arxiv.org/html/2608.00585#S9.F4)sweeps it\. Raisingτ\\taufrom 0\.10 to 0\.75 gains 2\.7 points of distractor rejection and costs 4\.0 points of gold recall, and the curves are nearly flat throughout\. More decisively, at the most permissive threshold tested, only 16\.1% of answer\-bearing gold paragraphs pass at all\. The score distribution is compressed against zero, so thresholding is close to inoperative\. There is no operating point being missed\.

#### Not the embedder

TABLE IV:The directional deficit across three embedding models\. HotpotQA, 200 questions\.Embedderanswer\-bearingbridgeonlydeficitBAAI/bge\-small\-en\-v1\.50\.8210\.937\+0\.116\+0\.116intfloat/e5\-base\-v20\.8340\.941\+0\.107\+0\.107thenlper/gte\-base0\.7980\.948\+0\.150\+0\.150Table[IV](https://arxiv.org/html/2608.00585#S9.T4)repeats the split with three widely used embedding models spanning two architectures and three training recipes\. Every one prefers the bridge paragraph by 0\.107 to 0\.150\. The directional bias is a property of conditioning on the question, not of any particular encoder\.

## XThe structural signal

The HRR signal fails for the same reason as entailment and for a second reason on top of it, and we treat it briefly because it is not central\. It has the largest directional deficit of any signal,\+0\.136\+0\.136, so it inherits the same blindness\. Separately, it faces an extraction dilemma\. Our extractor takes the first filler for each of four roles, pinningkk, the bindings per trace, at 3\.98 against 41\.9 candidate pairs available per paragraph, discarding 90\.5% of the structure\. Binding all of it does not help: for a trace ofkkbindings with near\-orthogonal roles, unbinding returns the target filler plusk−1k\-1crosstalk terms of comparable norm, so𝔼​\[cos⁡\(𝐯r†⊛T​\(c\),𝐯ϕc​\(r\)\)\]≈k−1/2\\mathbb\{E\}\[\\operatorname\{cos\}\(\\mathbf\{v\}\_\{r\}^\{\\dagger\}\\circledast T\(c\),\\mathbf\{v\}\_\{\\phi\_\{c\}\(r\)\}\)\]\\approx k^\{\-1/2\}, which is 0\.154 atk=42k=42and below any usable threshold\. Paragraph\-level HRR is squeezed from both sides\. An earlier version of this work blamed a capacity threshold in the generator; that was wrong, since the generator never sees the structural score in a form it could exploit\.

## XIWhat it costs, and what conditioning recovers

Algorithm 1Conditional evidence selection1:question

qq, candidates

𝒞\\mathcal\{C\}, budget

kk
2:

𝒯←\\mathcal\{T\}\\leftarrowtop\-

kkof

𝒞\\mathcal\{C\}by

semb​\(q,⋅\)s\_\{\\mathrm\{emb\}\}\(q,\\cdot\)
3:

c1←arg⁡maxc∈𝒯⁡semb​\(q,c\)c\_\{1\}\\leftarrow\\arg\\max\_\{c\\in\\mathcal\{T\}\}s\_\{\\mathrm\{emb\}\}\(q,c\)⊳\\trianglerighthop 1; gold 92% of the time

4:

h←ℋ​\(q\)h\\leftarrow\\mathcal\{H\}\(q\)⊳\\trianglerightobject\-level proposition, no gold answer

5:for all

cj∈𝒯∖\{c1\}c\_\{j\}\\in\\mathcal\{T\}\\setminus\\\{c\_\{1\}\\\}do

6:

uj←P​\(entail∣c1⊕cj,h\)u\_\{j\}\\leftarrow P\\big\(\\text\{entail\}\\mid c\_\{1\}\\oplus c\_\{j\},\\ h\\big\)⊳\\trianglerightc1c\_\{1\}makes the premise sufficient

7:endfor

8:

c2←arg⁡maxj⁡ujc\_\{2\}\\leftarrow\\arg\\max\_\{j\}u\_\{j\}
9:return

\{c1,c2\}\\\{c\_\{1\},c\_\{2\}\\\}⊳\\trianglerightk−1k\-1calls, against\(k2\)\\binom\{k\}\{2\}for blind pair search

TABLE V:Paired exact McNemar against B1, unfiltered retrieval\.bbandccare discordant counts;ppis Holm\-adjusted within each dataset\. Qwen2\.5\-1\.5B, standard prompt\.p∗<\.05\{\}^\{\*\}p<\.05,p∗∗<\.01\{\}^\{\*\*\}p<\.01,p∗⁣∗∗<\.001\{\}^\{\*\*\*\}p<\.001\.DatasetSelectorΔ\\DeltaEMbbccHolmppHotpotQAB5\+8\.0\+8\.06828<\.001∗⁣∗∗<\.001^\{\*\*\*\}B4\+0\.8\+0\.835311\.000COND−1\.2\-1\.237431\.000SET−2\.6\-2\.632450\.513B2−6\.2\-6\.220510\.001∗∗0\.001^\{\*\*\}PC−13\.4\-13\.43198<\.001∗⁣∗∗<\.001^\{\*\*\*\}2WikiB5\+8\.2\+8\.276350\.001∗⁣∗∗0\.001^\{\*\*\*\}B2\+0\.6\+0\.649461\.000B4\+0\.2\+0\.248471\.000COND−1\.4\-1\.443501\.000SET−1\.6\-1\.647551\.000PC−6\.2\-6\.242730\.025∗0\.025^\{\*\}MuSiQueB5\+20\.1\+20\.1619<\.001∗⁣∗∗<\.001^\{\*\*\*\}B4−3\.1\-3\.115230\.308B2−3\.9\-3\.915250\.308COND−6\.6\-6\.610270\.023∗0\.023^\{\*\}SET−7\.3\-7\.39280\.010∗0\.010^\{\*\}PC−11\.2\-11\.2938<\.001∗⁣∗∗<\.001^\{\*\*\*\}TABLE VI:End to end, Qwen2\.5\-1\.5B, greedy decoding\.*rec*is the fraction of available gold paragraphs reaching the generator\. Brackets are 95% Wilson intervals\. MuSiQue is restricted to its 2\-hop questions, since pair selectors cannot reach a 3\- or 4\-hop answer by construction\.DataSelectorEM95% CIrecHotpotQAn=500n=500B5 oracle gold pair55\.0\[50\.6, 59\.3\]1\.00B4 reranker top\-247\.8\[43\.5, 52\.2\]0\.86B1 all five, no filter47\.0\[42\.7, 51\.4\]0\.92COND\(ours\)45\.8\[41\.5, 50\.2\]0\.76SET blind pair search44\.4\[40\.1, 48\.8\]0\.68B2 top\-2 by retriever40\.8\[36\.6, 45\.2\]0\.77PC per\-chunk gate33\.6\[29\.6, 37\.9\]0\.462Wikin=500n=500B5 oracle gold pair42\.2\[38\.0, 46\.6\]1\.00B2 top\-2 by retriever34\.6\[30\.6, 38\.9\]0\.70B4 reranker top\-234\.2\[30\.2, 38\.5\]0\.74B1 all five, no filter34\.0\[30\.0, 38\.3\]0\.87COND\(ours\)32\.6\[28\.6, 36\.8\]0\.61SET blind pair search32\.4\[28\.5, 36\.6\]0\.46PC per\-chunk gate27\.8\[24\.1, 31\.9\]0\.42MuSiQuen=259n=259B5 oracle gold pair40\.9\[35\.1, 47\.0\]1\.00B1 all five, no filter20\.9\[16\.4, 26\.2\]0\.74B4 reranker top\-217\.8\[13\.6, 22\.9\]0\.60B2 top\-2 by retriever17\.0\[12\.9, 22\.0\]0\.54COND\(ours\)14\.3\[10\.6, 19\.1\]0\.51SET blind pair search13\.5\[ 9\.9, 18\.2\]0\.41PC per\-chunk gate9\.7\[ 6\.6, 13\.9\]0\.35An AUC gap is not an answer\-quality gap, so we measured both\. Seven selectors feed the same generator: no filtering, truncation to top two by retriever score, a cross\-encoder reranker\[[15](https://arxiv.org/html/2608.00585#bib.bib15)\], the per\-chunk gate this paper has been analysing, blind set\-level search over all\(52\)\\binom\{5\}\{2\}pairs, the conditional selector of Algorithm[1](https://arxiv.org/html/2608.00585#alg1), and an oracle supplying the gold paragraphs\.

Table[V](https://arxiv.org/html/2608.00585#S11.T5)gives paired exact McNemar tests against B1, unfiltered retrieval, Holm\-corrected within each dataset\. B1 is the right reference because the question is whether filtering helps at all\.

#### Per\-chunk gating is significantly worse than not filtering

On all three datasets:−13\.4\-13\.4Exact Match on HotpotQA \(p<\.001p<\.001\),−6\.2\-6\.2on 2Wiki \(p=\.025p=\.025\),−11\.2\-11\.2on MuSiQue \(p<\.001p<\.001\)\. The mechanism is in the recall column: the gate keeps 1\.2 chunks on average and lets only 35% to 46% of gold paragraphs through\. This is the strongest form our claim takes\.

Conditioning recovers evidence at lower cost, though not accuracy\. Algorithm[1](https://arxiv.org/html/2608.00585#alg1)beats blind pair search on gold recall by\+0\.072\+0\.072,\+0\.149\+0\.149and\+0\.078\+0\.078, usingk−1=4k\-1=4entailment calls against\(52\)=10\\binom\{5\}\{2\}=10, a 60% reduction\. Its anchor lands on a gold paragraph 92\.0% of the time on HotpotQA and 96\.8% on 2Wiki, but only 77\.8% on MuSiQue, whose twenty candidates make the first hop harder to fix; that is why it helps least there\.

Against B1 the accuracy picture is worse than we first read it\. On HotpotQA and 2Wiki the conditional selector is indistinguishable from no filtering \(−1\.2\-1\.2,p=1\.0p=1\.0;−1\.4\-1\.4,p=1\.0p=1\.0\)\. On MuSiQue it is*significantly worse*\(−6\.6\-6\.6,p=\.023p=\.023\), as is blind pair search \(−7\.3\-7\.3,p=\.010p=\.010\)\. Our own proposal does not survive its own test on the hardest dataset, and we report that rather than resting on the recall gain\.

No selector is significantly better than B1 anywhere\. The reranker comes closest and never separates \(\+0\.8\+0\.8,\+0\.2\+0\.2,−3\.1\-3\.1; allp\>\.3p\>\.3\)\. Truncation to two chunks is significantly worse on HotpotQA \(−6\.2\-6\.2,p=\.001p=\.001\)\. Only the oracle beats B1, by\+8\.0\+8\.0,\+8\.2\+8\.2and\+20\.1\+20\.1, allp<\.001p<\.001\.

The headroom is real and unclaimed\. The oracle sits 7\.2 points above the best deployable selector on HotpotQA, 7\.6 on 2Wiki and20\.1on MuSiQue, where it nearly doubles the best deployable score\. None of the verifiers we tested collect any of it\.

#### Why good AUC did not produce good answers

Set\-level pair AUC is 0\.811, well above 0\.620 per chunk\. But a deployed selector is an argmax, not a ranking, and the argmax lands on the correct gold pair only 44\.4% of the time, though it contains at least one gold paragraph 92\.2% of the time\. That is why set\-level scoring beats the per\-chunk gate and still trails no filtering\. AUC is necessary for a verifier and it is not sufficient; reporting it without argmax accuracy overstates what a signal will do in place\.

## XIIWhat repairs it

2233440\.60\.60\.80\.8chance\.848\.849\.804\.710\.556\.560\.570\.503\.558hops required to answerAUC on the supporting paragraphgold sub\-questionQwen2\.5\-7B, anchoredoriginal questionFigure 5:MuSiQue, 9997 question\-paragraph pairs\. Asking whether a paragraph supports a later hop is near chance when the hypothesis comes from the original question and becomes tractable when it comes from the sub\-question\. The gold curve is flat within its intervals\. An off\-the\-shelf decomposer matches most of the gain at two hops and falls back to the original\-question level beyond that; the drop from 0\.710 \[0\.676, 0\.743\] to 0\.556 \[0\.513, 0\.594\] lies outside the intervals\.Everything so far says per\-chunk verification fails because the hypothesis is built from a question whose answer no single chunk supports\. That predicts a repair: build the hypothesis from the sub\-question instead\.

MuSiQue lets us test this without building a decomposer\. It shipsquestion\_decomposition, giving each hop’s sub\-question and the index of the paragraph that supports it\. We take a later hop, resolve its placeholders against earlier answers, and ask a single question of the entailment model: which paragraph supports*this*hop? We compare three hypotheses, all scored against the same 9997 paragraph candidates\.

TABLE VII:Identifying the paragraph that supports a later hop\. MuSiQue, 500 questions, 9997 candidate paragraphs, identical NLI model throughout\. The decomposer is Qwen2\.5\-7B\-Instruct with no fine\-tuning and no gold annotation at inference\. Percentages are of the distance from the original question to the gold sub\-question\.Hypothesis built fromAUC\[95% CI\]the original question0\.546\[0\.523, 0\.569\]the decomposed sub\-question0\.840\[0\.824, 0\.856\]the sub\-question with its gold answer0\.936\[0\.924, 0\.947\]paired lift:\+0\.355\\mathbf\{\+0\.355\}\[0\.331, 0\.382\]Table[VII](https://arxiv.org/html/2608.00585#S12.T7)and Figure[5](https://arxiv.org/html/2608.00585#S12.F5)give the result\.

The original question gives 0\.546 AUC for identifying the paragraph a later hop depends on, which is chance\. That is the sharpest form of this paper’s negative claim: a verifier conditioned on the question as asked cannot tell which paragraph carries the second hop\.

The gold sub\-question gives 0\.840, a paired lift of\+0\.355\+0\.355with an interval of\[0\.331,0\.382\]\[0\.331,0\.382\]\. The same NLI model, the same paragraphs and the same task become tractable purely by changing what the hypothesis asks about\.

Hop count stops mattering under decomposition\. With the original question the AUC is 0\.570, 0\.503 and 0\.558 at two, three and four hops, with overlapping intervals and no usable value at any depth\. With the gold sub\-question it is 0\.848 \[0\.826, 0\.869\], 0\.849 \[0\.823, 0\.873\] and 0\.804 \[0\.750, 0\.850\]: flat within its intervals, and every interval excluding the whole original\-question range\. A Depth stops predicting difficulty once the verifier is pointed at the hop in question\.

The lift is a paired bootstrap on the difference, so it is not an artefact of comparing two separately estimated quantities\.

#### An off\-the\-shelf decomposer captures a third of it

The gold annotation makes this a ceiling\. To see how much of it is reachable, we had Qwen2\.5\-7B\-Instruct produce the decomposition with no fine\-tuning and no gold annotation, in two settings\. Given only the question, it reaches 0\.533 \[0\.510, 0\.557\], which is*below*the original\-question baseline\. Given the question and the top retrieved paragraph, in the shape Self\-Ask uses, it reaches 0\.637 \[0\.611, 0\.662\], a paired lift of\+0\.091\+0\.091\[0\.066, 0\.115\] and 31% of the ceiling\.

The blind setting hurts and the anchored setting helps, which fits the rest of the results\. A decomposer conditioned only on the original question has the same blind spot the verifier does; it needs retrieved text to work from\.

Two things bound the deployable version\. Its gain is gated by whether the retrieval anchor is correct: 0\.661 on the 389 questions where the top chunk is gold and 0\.546 on the 111 where it is not, the latter being exactly the original\-question baseline\. And it works at two hops, 0\.710 \[0\.676, 0\.743\], then falls to 0\.556 \[0\.513, 0\.594\] and 0\.560 \[0\.499, 0\.614\] at three and four, a drop outside the intervals\. Our prompt asks for one follow\-up question, which is well posed when a single hop remains and ambiguous when three do\. Iterative decomposition, one hop at a time, would address that\. We have not run it\.

The effect comes from restructuring the question and not from having more text available\. We also tested the naive intervention of appending the top retrieved chunk to the query at three granularities, title, first sentence and full paragraph, and re\-scoring\. Mean lift on answer\-bearing gold was\+0\.034\+0\.034across datasets, and on HotpotQA the full\-paragraph variant*hurt*by0\.0280\.028\. A deployable approximation that conditions the hypothesis on the anchor title recovered\+0\.059\+0\.059,\+0\.042\+0\.042and−0\.034\-0\.034on the three datasets, a small fraction of the oracle’s\+0\.355\+0\.355\.

Producing decompositions is not an open problem: Self\-Ask and IRCoT generate them already, for retrieval\. What we report is that the same artefact repairs verification, that nobody appears to be using it there, and that a general\-purpose model reaches a third of the available gain without being asked to do anything special\. The remaining 0\.20 AUC is what a decomposer built for this would be worth\.

111A measurement artefact in our own tooling is worth recording, since it is the effect this paper documents\. An earlier version of this experiment forced every gold sub\-question into a relational template, producing hypotheses of the form “The answer of*\[question\]*is something\.” Most MuSiQue sub\-questions are natural language rather than relational, so that template turned them into meta\-statements about an answer, and the measured AUC fell from 0\.840 to 0\.779\. That is the out\-of\-distribution effect described in Section[III](https://arxiv.org/html/2608.00585#S3), showing up in our own measurement code\.

## XIIIRobustness

TABLE VIII:The end\-to\-end ordering across three generator sizes\. Rank 1 is best\. Per\-chunk gating is last in every cell but one, and its penalty grows with capability\.Dataset0\.5B1\.5B3B*PC minus best deployable selector, Exact Match*HotpotQA−4\.6\-4\.6−14\.2\-14\.2−19\.4\\mathbf\{\-19\.4\}2Wiki0\.0\\ \\ 0\.0−6\.8\-6\.8−13\.6\\mathbf\{\-13\.6\}MuSiQue−1\.9\-1\.9−11\.2\-11\.2−8\.1\-8\.1*oracle minus best deployable selector, Exact Match*HotpotQA−0\.6\-0\.6\+7\.2\+7\.2\+10\.6\\mathbf\{\+10\.6\}2Wiki−1\.0\-1\.0\+7\.6\+7\.6\+7\.2\+7\.2MuSiQue\+5\.0\+5\.0\+20\.1\+20\.1\+23\.6\\mathbf\{\+23\.6\}#### Generator scale

Table[VIII](https://arxiv.org/html/2608.00585#S13.T8)repeats the seven\-selector grid at 0\.5B, 1\.5B and 3B\. Two patterns hold across all three datasets\. First, the cost of per\-chunk gating*grows*with generator capability, from 4\.6 Exact Match at 0\.5B to 19\.4 at 3B on HotpotQA\. A stronger model is hurt more by deleted evidence, because it could have used it\. Second, the oracle gap grows the same way, reaching 23\.6 points on MuSiQue at 3B\. Better selection becomes more valuable, not less, as generators improve\.

At 0\.5B nothing is distinguishable\. All seven selectors land within a few points with fully overlapping intervals, and on two datasets the oracle is not even best\. A 0\.5B generator handed both gold paragraphs answers 10\.8% of HotpotQA and 7\.7% of MuSiQue\. Selection cannot matter when the generator cannot use the selection, and any verification result reported at that scale on these tasks is measuring noise\.

#### Prompt

Repeating the comparison with a differently framed generation prompt moves every absolute number down by 8 to 10 Exact Match and leaves the ordering intact: oracle, then reranker, then the conditional selector and unfiltered retrieval within noise of each other, then per\-chunk gating last, on all three datasets\.

#### Answer\-matching criterion

The answer\-bearing split underpins Sections[V](https://arxiv.org/html/2608.00585#S5)and[VII](https://arxiv.org/html/2608.00585#S7), so we recomputed it three ways: exact normalised string match, match against MuSiQue’s answer aliases, and token overlap at 0\.8\. The directional deficit moves by at most 0\.007 on any dataset\. The split is not an artefact of how answer\-bearing is operationalised\.

## XIVDiscussion

The recommendation we hold with most confidence is negative: do not gate multi\-hop retrieval per chunk\. Entailment filtering is the worst of seven selectors on three datasets and it fails by removing the evidence the question needs\. A system that adds such a gate will appear to be doing careful work while deleting the second hop\.

If some selection is required, anchor before searching\. The embedding signal is reliable on the first hop, at 0\.92 to 0\.97 AUC on question\-named evidence, and unreliable on the second, so Algorithm[1](https://arxiv.org/html/2608.00585#alg1)uses each signal where it works: embedding to fix the anchor, entailment to score completions given that anchor\. It recovers 7 to 15 points of gold recall over blind search at 40% of the verification cost\. We report the design because the evidence points at it, not because this instance of it beats doing nothing\. It does not\.

Self\-Ask\[[13](https://arxiv.org/html/2608.00585#bib.bib13)\]and IRCoT\[[14](https://arxiv.org/html/2608.00585#bib.bib14)\]decompose the query between hops because the second hop is not reachable from the original question\. They use the decomposition for retrieval and discard it before verification\. Section[XII](https://arxiv.org/html/2608.00585#S12)indicates that discarding it is what breaks the verifier: the same artefact lifts entailment on a later hop from chance to 0\.840 and flattens the hop\-count effect\.

The integration is cheap\. A pipeline already running Self\-Ask or IRCoT has the sub\-questions in hand, and passing them to the verifier instead of the original query costs one string substitution\. The ceiling is\+0\.355\+0\.355AUC and an off\-the\-shelf 7B decomposer reaches\+0\.091\+0\.091of it without any adaptation, so about two thirds of the distance is still open\. Two specific things would close part of it: decomposing iteratively rather than in one shot, since the gain survives only at two hops, and improving first\-hop retrieval, since the gain vanishes entirely when the anchor is wrong\.

On method: the AUC measurements cost about three dollars of cloud compute and showed the per\-chunk signal was unusable before a single answer was generated\. Separation alone would have misled us in the other direction, though\. A pair AUC of 0\.811 looked like a fix and became a selector that still loses to no filtering, because a deployed selector is an argmax and the argmax is right 44\.4% of the time\. Both measurements are cheap and neither substitutes for the other\.

Both the cost of bad selection and the value of good selection grow with generator scale \(Table[VIII](https://arxiv.org/html/2608.00585#S13.T8)\)\. Per\-chunk gating costs 4\.6 Exact Match at 0\.5B and 19\.4 at 3B, and the oracle gap widens from−0\.6\-0\.6to\+10\.6\+10\.6on HotpotQA, reaching 23\.6 on MuSiQue\. Evidence selection for multi\-hop RAG becomes more valuable as generators improve\.

We would encourage anyone running a comparison like this to include the oracle row\. Without it we would have read the reranker’s 47\.8 as near\-ceiling and the separation gain as the main thing left to chase\. It costs one extra configuration and it repriced every other row in the table\.

#### Single\-hop deployments are a different case

The deployableslotvariant reaches 0\.927 AUC on SQuAD, close to its 0\.951 oracle, at 184 M parameters\. Where one passage carries the whole answer, entailment gating rests on an assumption that holds\. Everything negative here is specific to multi\-hop\.

## XVLimitations

The decomposition ceiling uses MuSiQue’s gold annotations, and while we measure what an off\-the\-shelf decomposer reaches, we did not build one, did not run the decomposed verifier end to end, and did not test transfer to HotpotQA or 2Wiki, which ship no decompositions\. The deployable measurement also inherits the retrieval anchor: on the 111 questions where the top chunk is not gold it provides nothing\. The gaps among the top deployable selectors are inside their intervals, so we order them only weakly\. The MuSiQue end\-to\-end rows cover its 2\-hop subset, because pair selectors cannot reach a 3\- or 4\-hop answer; the AUC analysis uses every hop count, and selection over larger sets remains untested\. Our conditional selector is one design among many: beam search over larger sets, or rewriting the hypothesis using the anchor rather than concatenating it, might do better and we have not tried either\. The single\-hop control uses SQuAD, where the gold paragraph is guaranteed to contain the answer, which is easier than open\-domain single\-hop retrieval; the true single\-hop number likely lies between 0\.664 and 0\.951\. The role extractor is one implementation, though thek−1/2k^\{\-1/2\}crosstalk bound in Section[X](https://arxiv.org/html/2608.00585#S10)limits how much a better one could help\. All entailment numbers come from one model family\.

## XVIConclusion

We built a three\-signal chunk verifier for retrieval\-augmented generation and measured, before evaluating it end to end, how well each signal separates gold evidence from distractors\. On three multi\-hop datasets none of them worked: embedding similarity reached 0\.76 to 0\.89, entailment never exceeded 0\.67 even when handed the gold answer, and structural matching reached 0\.62\.

The cause is an assumption rather than a defect\. Per\-chunk verification treats each chunk as a sufficient premise, and in multi\-hop retrieval that is false for the chunk carrying the second hop\. Seven measurements pin it down: the signals prefer the paragraph named in the question, the preference vanishes on comparison questions, it deepens monotonically with hop count, supplying both hops lifts entailment from 0\.664 to 0\.881, premise length runs the opposite way, single\-hop SQuAD reaches 0\.951, and no threshold or embedder changes any of it\.

End to end this costs what the analysis predicts, across three datasets, three generator sizes and two prompts\. Per\-chunk gating is worst in every cell, and its penalty grows with generator capability, from 4\.6 Exact Match at 0\.5B to 19\.4 at 3B\. An oracle selector sits up to 23\.6 points above the best deployable method, and that gap widens with scale too\.

What repairs it is changing the question the verifier is asked\. Conditioning the hypothesis on the decomposed sub\-question rather than the original query lifts entailment on a later hop from 0\.546, which is chance, to 0\.840, a paired lift of\+0\.355\+0\.355\[0\.331, 0\.382\]\. Hop count then stops mattering: 0\.848, 0\.849 and 0\.804 at two, three and four hops\. An off\-the\-shelf 7B decomposer with no adaptation reaches 0\.637, a third of that ceiling, and only at two hops\. Appending retrieved text to the query does not work at all, so the effect comes from restructuring the question rather than from added context\. Iterative retrieval systems already build these decompositions and discard them before verifying\.

Verification is not broken\. It is being asked to judge sufficiency one chunk at a time, against a question that no single chunk can answer\.

## Reproducibility

Code, the exact question identifiers for every split, per\-question signal traces, and the scripts that generate every table and figure here are at[https://github\.com/iamhero2709/verification\-without\-sufficiency](https://github.com/iamhero2709/verification-without-sufficiency)\. All runs use greedy decoding and seed 1337\. Every experiment in this paper cost under fifteen dollars of cloud compute and about five hours of wall clock\. Confidence intervals are question\-stratified bootstraps with 1000 resamples and the tables reporting them are emitted by script, not typed\. Every number in every table is produced by a script reading the released traces\.

## References

- \[1\]P\. Lewis, E\. Perez, A\. Piktus*et al\.*, “Retrieval\-augmented generation for knowledge\-intensive NLP tasks,” in*Adv\. Neural Inf\. Process\. Syst\.*, vol\. 33, 2020, pp\. 9459–9474\.
- \[2\]Y\. Gao, Y\. Xiong, X\. Gao*et al\.*, “Retrieval\-augmented generation for large language models: A survey,”*arXiv:2312\.10997*, 2023\.
- \[3\]S\.\-Q\. Yan, J\.\-C\. Gu, Y\. Zhu, and Z\.\-H\. Ling, “Corrective retrieval augmented generation,”*arXiv:2401\.15884*, 2024\.
- \[4\]A\. Asai, Z\. Wu, Y\. Wang, A\. Sil, and H\. Hajishirzi, “Self\-RAG: Learning to retrieve, generate, and critique through self\-reflection,” in*Proc\. ICLR*, 2024\.
- \[5\]S\. Es, J\. James, L\. Espinosa\-Anke, and S\. Schockaert, “RAGAS: Automated evaluation of retrieval augmented generation,” in*Proc\. EACL \(System Demonstrations\)*, 2024, pp\. 150–158\.
- \[6\]S\. R\. Bowman, G\. Angeli, C\. Potts, and C\. D\. Manning, “A large annotated corpus for learning natural language inference,” in*Proc\. EMNLP*, 2015, pp\. 632–642\.
- \[7\]A\. Williams, N\. Nangia, and S\. R\. Bowman, “A broad\-coverage challenge corpus for sentence understanding through inference,” in*Proc\. NAACL\-HLT*, 2018, pp\. 1112–1122\.
- \[8\]P\. He, J\. Gao, and W\. Chen, “DeBERTaV3: Improving DeBERTa using ELECTRA\-style pre\-training with gradient\-disentangled embedding sharing,” in*Proc\. ICLR*, 2023\.
- \[9\]Z\. Yang, P\. Qi, S\. Zhang*et al\.*, “HotpotQA: A dataset for diverse, explainable multi\-hop question answering,” in*Proc\. EMNLP*, 2018, pp\. 2369–2380\.
- \[10\]X\. Ho, A\.\-K\. D\. Nguyen, S\. Sugawara, and A\. Aizawa, “Constructing a multi\-hop QA dataset for comprehensive evaluation of reasoning steps,” in*Proc\. COLING*, 2020, pp\. 6609–6625\.
- \[11\]H\. Trivedi, N\. Balasubramanian, T\. Khot, and A\. Sabharwal, “MuSiQue: Multihop questions via single\-hop question composition,”*Trans\. Assoc\. Comput\. Linguist\.*, vol\. 10, pp\. 539–554, 2022\.
- \[12\]P\. Rajpurkar, J\. Zhang, K\. Lopyrev, and P\. Liang, “SQuAD: 100,000\+ questions for machine comprehension of text,” in*Proc\. EMNLP*, 2016, pp\. 2383–2392\.
- \[13\]O\. Press, M\. Zhang, S\. Min*et al\.*, “Measuring and narrowing the compositionality gap in language models,” in*Findings of EMNLP*, 2023, pp\. 5687–5711\.
- \[14\]H\. Trivedi, N\. Balasubramanian, T\. Khot, and A\. Sabharwal, “Interleaving retrieval with chain\-of\-thought reasoning for knowledge\-intensive multi\-step questions,” in*Proc\. ACL*, 2023, pp\. 10014–10037\.
- \[15\]S\. Xiao, Z\. Liu, P\. Zhang, N\. Muennighoff, D\. Lian, and J\.\-Y\. Nie, “C\-Pack: Packed resources for general Chinese embeddings,” in*Proc\. SIGIR*, 2024, pp\. 641–649\.
- \[16\]A\. Yang, B\. Yang, B\. Hui*et al\.*, “Qwen2\.5 technical report,”*arXiv:2412\.15115*, 2024\.
- \[17\]T\. A\. Plate, “Holographic reduced representations,”*IEEE Trans\. Neural Netw\.*, vol\. 6, no\. 3, pp\. 623–641, 1995\.
- \[18\]T\. A\. Plate,*Holographic Reduced Representation: Distributed Representation for Cognitive Structures*\. Stanford, CA: CSLI Publications, 2003\.
- \[19\]P\. Kanerva, “Hyperdimensional computing: An introduction to computing in distributed representation with high\-dimensional random vectors,”*Cognitive Computation*, vol\. 1, no\. 2, pp\. 139–159, 2009\.
- \[20\]M\. Nickel, L\. Rosasco, and T\. Poggio, “Holographic embeddings of knowledge graphs,” in*Proc\. AAAI*, 2016, pp\. 1955–1961\.