Probe, Don't Prompt: A Hidden-State Probe for Metadata Filtering in Multi-Meta-RAG

# A Hidden-State Probe for Metadata Filtering in Multi-Meta-RAG
Source: [https://arxiv.org/html/2607.03929](https://arxiv.org/html/2607.03929)
###### Abstract

Multi\-Meta\-RAG improves retrieval for multi\-hop question answering by filtering a vector store on metadata \(the news*source*\) that it extracts from each query by promptinggpt\-3\.5\-turbo\. We show this proprietary, free\-form extractor can be replaced by a local, deterministic probe trained on the hidden states of a*small*open\-source language model\. On all 2556 MultiHop\-RAG queries the probe reaches90\.9%set\-exact accuracy against 88\.0% for a model\-free substring baseline and 80\.9% for GPT\-3\.5, a margin that comes entirely from null queries, on which GPT\-3\.5 never abstains; on non\-null queries all three stay within about a point\. Because the probe’s output space is exactly the fixed 49\-source vocabulary, it cannot drift outside the allow\-list as the prompted model does\. Three design choices make it work: selecting a*shallow*layer,*mean*pooling, and*class\-imbalance\-aware*multi\-label training over the long tail of sources\. A 135M\-parameter model lands within∼\\sim1\.5 points of a 1\.5B one, so the filter is cheap to output: a partial forward pass through the first few layers plus one linear head, with no API\. The code is available at[https://github\.com/mxpoliakov/Multi\-Meta\-RAG](https://github.com/mxpoliakov/Multi-Meta-RAG)\.

## IIntroduction

Retrieval\-augmented generation \(RAG\)\[[9](https://arxiv.org/html/2607.03929#bib.bib1)\]grounds a language model on documents fetched from an external store, but struggles on*multi\-hop*queries that must assemble evidence from several documents at once\. The MultiHop\-RAG benchmark\[[16](https://arxiv.org/html/2607.03929#bib.bib2)\]makes this failure mode concrete: 2556 news queries whose answers require combining two to four articles\. Multi\-Meta\-RAG\[[12](https://arxiv.org/html/2607.03929#bib.bib3)\]improves multi\-hop retrieval on this benchmark by filtering the vector store on metadata before similarity search, specifically the news*source*\(e\.g\.*The Verge*,*TechCrunch*\) named in the query\. The filter is obtained by promptinggpt\-3\.5\-turboto read the source\(s\) out of the query text\.

This extractor has two costs\. First, it is a proprietary API call on every query, adding token cost and network latency to the retrieval path\. Second, it*drifts*: despite a prompt that fixes a 49\-source allow\-list, GPT\-3\.5 emits 128 distinct source strings across the dataset, 79 of them off\-list, so many of its filters reference labels the vector store never indexed\.

Our starting observation is that the source named in a query is almost always present in the query*surface form*: the source name appears verbatim in the query in 95\.4% of gold query–source pairs\. Extraction therefore need not be a generative act\. The information required is already linearly available in the model’s own internal representation\. We therefore train a lightweight*probe*\[[1](https://arxiv.org/html/2607.03929#bib.bib5)\]on the hidden states of a*small*open\-source model, motivated by evidence that intermediate representations carry task\-relevant signal for retrieval\[[10](https://arxiv.org/html/2607.03929#bib.bib4)\]\. The probe is a fixed\-vocabulary multi\-label classifier: its output space is exactly the 49 sources, so it is structurally incapable of the allow\-list drift the prompted baseline exhibits\.

Our main contributions are as follows\.

1. 1\.A drop\-in, fixed\-vocabulary probe that replaces GPT\-3\.5 source extraction in Multi\-Meta\-RAG, with no API cost, no allow\-list drift, and deterministic output\.
2. 2\.A set\-exact head\-to\-head on all 2556 MultiHop\-RAG queries against both GPT\-3\.5 and a model\-free substring baseline: the probe scores90\.9%overall versus 88\.0% \(substring\) and 80\.9% \(GPT\-3\.5\), with its margin coming entirely from null\-query abstention; on non\-null queries it stays within about a point of both\.
3. 3\.Three findings that make it work:*shallow*layers win \(not the middle or last\),*mean*pooling beats last\-token pooling, and*class\-imbalance\-aware*multi\-label training matters for the long tail of rare sources\.
4. 4\.A size study from 135M to 1\.5B parameters showing that a 135M model stays within∼\\sim1\.5 points of a 1\.5B one, so the filter is cheap to output\.

The rest of the paper is organized as follows\. We first review related work\. We then describe the probe and its three design choices\. Next, we report the head\-to\-head against GPT\-3\.5 and the layer, pooling, and size analyses\. Finally, we discuss limitations and future work\. The code is available at[https://github\.com/mxpoliakov/Multi\-Meta\-RAG](https://github.com/mxpoliakov/Multi-Meta-RAG)\.

## IIRelated Work

### II\-AProbing Classifiers

Shallow classifiers trained on frozen representations read out what those representations encode\[[1](https://arxiv.org/html/2607.03929#bib.bib5),[5](https://arxiv.org/html/2607.03929#bib.bib14)\]; see\[[3](https://arxiv.org/html/2607.03929#bib.bib6)\]for a survey\. Rigorous probing controls for what is genuinely*learned*versus recoverable from surface form\[[6](https://arxiv.org/html/2607.03929#bib.bib15)\]\. Our model\-free string\-match baseline plays exactly that control role here\. Layer\-wise probing shows that information is localized by depth\[[17](https://arxiv.org/html/2607.03929#bib.bib7)\]\. We use probing not for interpretability but as a*deployable component*of a retrieval pipeline\.

### II\-BWhich Layer Carries the Signal

Intermediate layers often beat the last layer for downstream use\[[14](https://arxiv.org/html/2607.03929#bib.bib8)\], including specifically for multi\-hop retrieval\[[10](https://arxiv.org/html/2607.03929#bib.bib4)\]\. We test this directly and report a*contrast*: for near\-lexical source identity the*shallowest*layers win, not the middle ones, because the target attribute is present in the query surface form rather than in deep semantics\.

### II\-CRAG and Metadata Filtering

RAG\[[9](https://arxiv.org/html/2607.03929#bib.bib1)\]over dense retrieval\[[8](https://arxiv.org/html/2607.03929#bib.bib13)\]underperforms on multi\-hop queries\[[16](https://arxiv.org/html/2607.03929#bib.bib2)\]; metadata\-filtered retrieval with LLM\-extracted filters addresses this\[[12](https://arxiv.org/html/2607.03929#bib.bib3)\]\. Rather than*distilling*\[[7](https://arxiv.org/html/2607.03929#bib.bib17)\]the prompted extractor by training on its outputs, we replace it with a small local model supervised directly on gold evidence sources\. Capable small open\-source models\[[13](https://arxiv.org/html/2607.03929#bib.bib9),[4](https://arxiv.org/html/2607.03929#bib.bib10)\]and the finding that decoder hidden states are strong text features\[[2](https://arxiv.org/html/2607.03929#bib.bib16)\]make this practical\.

## IIIMethod

### III\-ATask and Data

We use all 2556 MultiHop\-RAG queries: 856 comparison, 816 inference, 583 temporal, and 301 null queries\. The label space is exactly the49sources of the Multi\-Meta\-RAG prompt allow\-list\. The task is multi\-label: a query references one to four sources \(1→\\to598, 2→\\to1340, 3→\\to300, 4→\\to17 queries\)\. Let𝒮=\{s1,…,sK\}\\mathcal\{S\}=\\\{s\_\{1\},\\dots,s\_\{K\}\\\}be the fixed vocabulary ofK=49K=49sources\. We define the gold source set of each queryqqautomatically from its evidence,

Sq⋆=set​\(evidence sources of​q\),Sq⋆=∅​for null queries,S\_\{q\}^\{\\star\}=\\mathrm\{set\}\\big\(\\text\{evidence sources of \}q\\big\),\\qquad S\_\{q\}^\{\\star\}=\\varnothing\\ \\text\{ for null queries,\}\(1\)mapping the 301null queries to the empty setsince they carry no evidence, and encode it as a multi\-hot vectoryq∈\{0,1\}Ky\_\{q\}\\in\\\{0,1\\\}^\{K\}withyq​c=𝟏​\[sc∈Sq⋆\]y\_\{qc\}=\\mathbf\{1\}\[s\_\{c\}\\in S\_\{q\}^\{\\star\}\]\. The evaluation metric is theset\-exact hitaccuracy over theN=2556N=2556queries,

Acc=1N​∑q=1N𝟏​\[S^q=Sq⋆\],\\mathrm\{Acc\}=\\frac\{1\}\{N\}\\sum\_\{q=1\}^\{N\}\\mathbf\{1\}\\\!\\left\[\\hat\{S\}\_\{q\}=S\_\{q\}^\{\\star\}\\right\],\(2\)the fraction whose predicted setS^q\\hat\{S\}\_\{q\}equals the gold set exactly\. Date operators are ignored, as the baseline filter file uses only$​i​n\\mathdollar inonsource\.

### III\-BPipeline

Figure[1](https://arxiv.org/html/2607.03929#S3.F1)shows the pipeline\. For a query tokenised intoTqT\_\{q\}tokens we run one forward pass through a small open\-source model withoutput\_hidden\_states=True, giving hidden stateshℓ\(t\)∈ℝdh\_\{\\ell\}^\{\(t\)\}\\in\\mathbb\{R\}^\{d\}at layerℓ\\ellfor tokentt\. We cache, for every layer, two pooled representations,

hℓmean=1Tq​∑t=1Tqhℓ\(t\),hℓlast=hℓ\(Tq\),h\_\{\\ell\}^\{\\mathrm\{mean\}\}=\\frac\{1\}\{T\_\{q\}\}\\sum\_\{t=1\}^\{T\_\{q\}\}h\_\{\\ell\}^\{\(t\)\},\\qquad h\_\{\\ell\}^\{\\mathrm\{last\}\}=h\_\{\\ell\}^\{\(T\_\{q\}\)\},\(3\)as a\[N,nℓ,d\]\[N,n\_\{\\ell\},d\]tensor, so the layer/pooling sweep is cheap to run offline\. On the pooled statehℓh\_\{\\ell\}of a*single*chosen layer, a 49\-way multi\-label head scores each sourceccwith an independent logistic unit and one global decision thresholdτ\\tau,

pq​c=σ​\(wc⊤​hℓ\+bc\),S^q=\{sc:pq​c≥τ\},p\_\{qc\}=\\sigma\\\!\\big\(w\_\{c\}^\{\\top\}h\_\{\\ell\}\+b\_\{c\}\\big\),\\qquad\\hat\{S\}\_\{q\}=\\\{\\,s\_\{c\}:p\_\{qc\}\\geq\\tau\\,\\\},\(4\)whereσ\\sigmais the logistic sigmoid and features are standardised per dimension\. At inference the predicted setS^q\\hat\{S\}\_\{q\}is written back as asource\.$infilter, a drop\-in for the field the prompted extractor produced\.

Query“…according to*The Verge*and*Wired*?”Small open\-source LLMpartial forward passto layerℓ\\ellMean\-poolover tokens→hℓ∈ℝd\\to h\_\{\\ell\}\\in\\mathbb\{R\}^\{d\}49\-way multi\-labelhead \(sigmoid\)\+\+thresholdτ\\tausource\.$infilter\{The Verge, Wired\}only first few layersfixed 49\-source vocabularyFigure 1:The probe pipeline\. A single partial forward pass through the first few layers of a small open\-source model produces a pooled hidden state; a 49\-way multi\-label head reads the source set out of it and writes asource\.$infilter\. The output space is exactly the 49 allow\-list sources, so it cannot drift\.
### III\-CWhat the Probe Depends On

Three factors determine the probe’s accuracy: which layer it reads, how the token states are pooled, and how class imbalance is handled\. We fix each by a cross\-validated sweep or by standard multi\-label practice rather than by hand\. A configuration that gets all three wrong \(an arbitrary middle layer, last\-token pooling, and unweighted training\) does not beat the baseline\.

Layer\.We sweep every hidden\-state layer under both poolings with iterative\-stratified 5\-fold cross\-validation and select the layer with the best out\-of\-fold micro F1\. The intermediate\-layer literature\[[14](https://arxiv.org/html/2607.03929#bib.bib8),[10](https://arxiv.org/html/2607.03929#bib.bib4)\]would predict a middle\-layer peak, but the sweep selects a shallow layer \(index 1–4\) for all four models \(Table[I](https://arxiv.org/html/2607.03929#S4.T1)\), consistent with source identity being a near\-lexical attribute\.

Pooling\.We compare mean against last\-token pooling on the same swept layers\. Mean pooling gives the higher out\-of\-fold F1 for all four models and at nearly every depth \(Fig\.[2](https://arxiv.org/html/2607.03929#S4.F2)\), consistent with the source name appearing anywhere in the query rather than only at its end\.

Class imbalance\.The source distribution is long\-tailed:*TechCrunch*appears 1190 times, while 26 of the 49 sources appear fewer than 20 times and the rarest appears twice\. We therefore train each logistic unit by minimising a class\-balanced binary cross\-entropy,

ℒc=−∑q=1N\[αc\+​yq​c​log⁡pq​c\+αc−​\(1−yq​c\)​log⁡\(1−pq​c\)\],\\mathcal\{L\}\_\{c\}=\-\\sum\_\{q=1\}^\{N\}\\Big\[\\,\\alpha\_\{c\}^\{\+\}\\,y\_\{qc\}\\log p\_\{qc\}\+\\alpha\_\{c\}^\{\-\}\\,\(1\-y\_\{qc\}\)\\log\(1\-p\_\{qc\}\)\\Big\],\(5\)with weights inversely proportional to class frequency,αc\+=N/\(2​nc\)\\alpha\_\{c\}^\{\+\}=N/\(2n\_\{c\}\)andαc−=N/\(2​\(N−nc\)\)\\alpha\_\{c\}^\{\-\}=N/\\big\(2\(N\-n\_\{c\}\)\\big\), wherenc=∑qyq​cn\_\{c\}=\\sum\_\{q\}y\_\{qc\}is the support of sourcecc, so the long tail is upweighted\. We use iterative\-stratified multi\-label splits\[[15](https://arxiv.org/html/2607.03929#bib.bib12)\]so that rare sources occur in every fold, and report both micro and macro F1,

F1micro=2​∑cTPc2​∑cTPc\+∑cFPc\+∑cFNc,F1macro=1K​∑c=1KF1c,\\mathrm\{F1\}\_\{\\mathrm\{micro\}\}=\\frac\{2\\sum\_\{c\}\\mathrm\{TP\}\_\{c\}\}\{2\\sum\_\{c\}\\mathrm\{TP\}\_\{c\}\+\\sum\_\{c\}\\mathrm\{FP\}\_\{c\}\+\\sum\_\{c\}\\mathrm\{FN\}\_\{c\}\},\\quad\\mathrm\{F1\}\_\{\\mathrm\{macro\}\}=\\frac\{1\}\{K\}\\sum\_\{c=1\}^\{K\}\\mathrm\{F1\}\_\{c\},\(6\)since macro F1 weights every source equally and so keeps the rare\-source tail visible\.

### III\-DFixed\-Vocabulary Property and Baselines

Because the output space is exactly the 49 labels, the probe*structurally*cannot drift outside the allow\-list, unlike the generative baseline that emits∼\\sim128 distinct strings\. We tune one global thresholdτ\\tauon out\-of\-fold probabilities and produce all predictions out\-of\-fold, so no query is ever scored by a model that trained on it; the layer and threshold are themselves selected on the same out\-of\-fold scores, so the reported numbers carry the mild optimism of that selection\. We also fit a 1\-hidden\-layer MLP as a capacity check\. We compare against two baselines: \(a\) the GPT\-3\.5 filter file shipped with Multi\-Meta\-RAG, and \(b\) a model\-freesubstring matchover the 49 source names \(case\-insensitive containment\), a strong reference at micro 0\.957 / macro 0\.942 that doubles as the probing control task\.

## IVResults

We report four findings: the probe beats GPT\-3\.5 overall but only matches a substring baseline on non\-null queries, the overall gain is concentrated in null queries, the three design choices are what carry it, and model size barely matters\. All numbers are taken verbatim from our experiment logs\.

TABLE I:Selected probe configuration and cross\-validated F1 per model\.TABLE II:Set\-exact hit accuracy by question type and overall, probe \(out\-of\-fold\) vs\. a model\-free substring match and GPT\-3\.5\.![Refer to caption](https://arxiv.org/html/2607.03929v1/figures/accuracy_vs_depth.png)Figure 2:Probe micro \(left\) and macro \(right\) F1 versus layer for Qwen2\.5\-1\.5B, under both poolings\. Mean pooling \(orange\) dominates last\-token \(blue\) almost everywhere; last\-token rises to a shallow local peak, sags through the middle, and recovers only at the final layer, while mean stays high and nearly flat, so neither favours the middle layer the deep\-semantics regime would predict\. Layer0is the embedding layer\.### IV\-AProbe Configuration \(Table[I](https://arxiv.org/html/2607.03929#S4.T1)\)

Table[I](https://arxiv.org/html/2607.03929#S4.T1)reports the selected configuration per model, chosen by out\-of\-fold micro F1 over the class\-weighted logistic\-regression sweep; the Layer column gives the hidden\-state index \(0=0=embeddings\) over the model’s transformer\-layer count\.*Mean pooling and a very shallow layer win every time*: the best hidden\-state index is 1–4 out of 24–32 transformer layers\. The logistic\-regression sweep reaches micro F1 0\.963–0\.970 and approaches the strong string\-match reference \(0\.957 micro\), and the MLP does not improve on it, so capacity is not the bottleneck\. Macro F1 \(0\.881–0\.892\) trails micro, locating the residual difficulty in the rarest sources\.

### IV\-BHead\-to\-Head \(Table[II](https://arxiv.org/html/2607.03929#S4.T2)\)

The main comparison spans all 2556 queries against two baselines, GPT\-3\.5 and a model\-free substring match\.*Overall with null queries, the probe scores 90\.9%, against the substring baseline’s 88\.0% and GPT\-3\.5’s 80\.9%*\.*Overall without null queries the three are within about a point: 90\.6% \(probe\), 90\.8% \(substring\), 91\.7% \(GPT\-3\.5\)\.*The probe’s overall margin is the*null query*row: it predicts the empty set 93\.7% of the time, whereas GPT\-3\.5 always extracts the sources it sees named in the text and so scores 0/301, and even the substring baseline over\-fires on the third of null queries that name a source \(66\.8%\)\. Null abstention is thus the probe’s one real edge; on non\-null queries it matches but does not beat either baseline\.

### IV\-CAnalysis

The overall margin comes from the null queries\. On non\-null queries all three methods are close \(Table[II](https://arxiv.org/html/2607.03929#S4.T2)\): GPT\-3\.5 is higher on comparison \(98\.2% vs\. 94\.4%\), the probe is higher on temporal \(84\.2% vs\. 81\.8%\), and the substring baseline \(90\.8%\) is on par with the probe \(90\.6%\)\. The probe’s advantage is therefore not more accurate source reading but that it can predict the empty set on null queries, which neither a generator prompted to read sources from the query text \(0/301\) nor a substring matcher \(66\.8%\) can do\.

The selected layer is shallow in every case, index 1–4 of the 24–32 transformer layers \(Table[I](https://arxiv.org/html/2607.03929#S4.T1), Fig\.[2](https://arxiv.org/html/2607.03929#S4.F2)\), rather than the middle layer reported as strongest for multi\-hop retrieval\[[10](https://arxiv.org/html/2607.03929#bib.bib4)\]and for general downstream use\[[14](https://arxiv.org/html/2607.03929#bib.bib8)\]\. A likely reason is that the source name appears verbatim in the query in 95\.4% of gold query–source pairs, so source identity is a near\-lexical attribute that a partial forward pass already exposes; the tasks in that prior work depend on deeper semantics\.

Model size has little effect on this task\. SmolLM2\-360M reaches 90\.5% against 90\.9% for the4×4\\timeslarger Qwen2\.5\-1\.5B, and the 135M model stays within∼\\sim1\.5 points at 89\.4% \(Table[II](https://arxiv.org/html/2607.03929#S4.T2)\)\.

The probe runs locally as a partial forward pass through the first few layers \(∼\\sim3–13% of the layers, depending on the model\) followed by one linear head; the selected depths are 2/28, 4/32, 2/24, and 1/30\. We state this cost architecturally—the head reads a layer the truncated pass has already computed—rather than as measured latency; the experiments themselves cached all layers from full forward passes\. The baseline instead issues one paid GPT\-3\.5 call per query and shows the allow\-list drift noted earlier\.

## VConclusions

A shallow, mean\-pooled, imbalance\-aware probe on a small open\-source model is a drop\-in replacement for the prompted GPT\-3\.5 metadata extractor in Multi\-Meta\-RAG\[[12](https://arxiv.org/html/2607.03929#bib.bib3)\]:\+10\+10points set\-exact overall, with the margin coming from null\-query abstention \(it matches but does not beat a substring baseline on non\-null queries\), no API cost, and no allow\-list drift\. The broader lesson for probing\-in\-the\-loop systems is to*read the cheapest layer that suffices*\. For near\-lexical attributes that is a shallow layer, a counterpoint to the middle\-layer literature\.

Limitations\.The study is a single dataset and domain \(news\); the source attribute is near\-lexical, so a string\-match baseline is already strong; and macro F1 trails micro on the rarest sources\.

Future work\.\(1\) Learned date$gt/$ltoperators, which are absent from the baseline filters today; \(2\) a lexical\-∪\\cup\-probe hybrid with a null gate to push non\-null accuracy past string match; \(3\) focal loss\[[11](https://arxiv.org/html/2607.03929#bib.bib11)\]or per\-class thresholds for the rare\-source macro\-F1 tail; and \(4\) measuring the end\-to\-end retrieval impact \(MRR@10, Hits@kk\) of swapping the extractor, not just extraction accuracy\.

## References

- \[1\]G\. Alain and Y\. Bengio\(2016\)Understanding intermediate layers using linear classifier probes\.arXiv preprint arXiv:1610\.01644\.External Links:[Link](https://arxiv.org/abs/1610.01644)Cited by:[§I](https://arxiv.org/html/2607.03929#S1.p3.1),[§II\-A](https://arxiv.org/html/2607.03929#S2.SS1.p1.1)\.
- \[2\]P\. BehnamGhader, V\. Adlakha, M\. Mosbach, D\. Bahdanau, N\. Chapados, and S\. Reddy\(2024\)LLM2Vec: large language models are secretly powerful text encoders\.InProceedings of the First Conference on Language Modeling \(COLM\),Note:arXiv:2404\.05961External Links:[Link](https://arxiv.org/abs/2404.05961)Cited by:[§II\-C](https://arxiv.org/html/2607.03929#S2.SS3.p1.1)\.
- \[3\]Y\. Belinkov\(2022\)Probing classifiers: promises, shortcomings, and advances\.Computational Linguistics48\(1\),pp\. 207–219\.External Links:[Link](https://doi.org/10.1162/coli_a_00422)Cited by:[§II\-A](https://arxiv.org/html/2607.03929#S2.SS1.p1.1)\.
- \[4\]L\. Ben Allal, A\. Lozhkov, E\. Bakouch, G\. M\. Blázquez, G\. Penedo, L\. Tunstall, A\. Marafioti, H\. Kydlíček,et al\.\(2025\)SmolLM2: when smol goes big – data\-centric training of a small language model\.arXiv preprint arXiv:2502\.02737\.External Links:[Link](https://arxiv.org/abs/2502.02737)Cited by:[§II\-C](https://arxiv.org/html/2607.03929#S2.SS3.p1.1)\.
- \[5\]A\. Conneau, G\. Kruszewski, G\. Lample, L\. Barrault, and M\. Baroni\(2018\)What you can cram into a single $&\!\#\* vector: probing sentence embeddings for linguistic properties\.InProceedings of the 56th Annual Meeting of the Association for Computational Linguistics \(ACL\),pp\. 2126–2136\.External Links:[Link](https://aclanthology.org/P18-1198/)Cited by:[§II\-A](https://arxiv.org/html/2607.03929#S2.SS1.p1.1)\.
- \[6\]J\. Hewitt and P\. Liang\(2019\)Designing and interpreting probes with control tasks\.InProceedings of the 2019 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),pp\. 2733–2743\.External Links:[Link](https://aclanthology.org/D19-1275/)Cited by:[§II\-A](https://arxiv.org/html/2607.03929#S2.SS1.p1.1)\.
- \[7\]G\. Hinton, O\. Vinyals, and J\. Dean\(2015\)Distilling the knowledge in a neural network\.arXiv preprint arXiv:1503\.02531\.External Links:[Link](https://arxiv.org/abs/1503.02531)Cited by:[§II\-C](https://arxiv.org/html/2607.03929#S2.SS3.p1.1)\.
- \[8\]V\. Karpukhin, B\. Oğuz, S\. Min, P\. Lewis, L\. Wu, S\. Edunov, D\. Chen, and W\. Yih\(2020\)Dense passage retrieval for open\-domain question answering\.InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),pp\. 6769–6781\.External Links:[Link](https://aclanthology.org/2020.emnlp-main.550/)Cited by:[§II\-C](https://arxiv.org/html/2607.03929#S2.SS3.p1.1)\.
- \[9\]P\. Lewis, E\. Perez, A\. Piktus, F\. Petroni, V\. Karpukhin, N\. Goyal, H\. Küttler, M\. Lewis, W\. Yih, T\. Rocktäschel, S\. Riedel, and D\. Kiela\(2020\)Retrieval\-augmented generation for knowledge\-intensive NLP tasks\.Advances in Neural Information Processing Systems33,pp\. 9459–9474\.External Links:[Link](https://dl.acm.org/doi/abs/10.5555/3495724.3496517)Cited by:[§I](https://arxiv.org/html/2607.03929#S1.p1.1),[§II\-C](https://arxiv.org/html/2607.03929#S2.SS3.p1.1)\.
- \[10\]J\. Lin, J\. Liu, and Y\. Liu\(2025\)Optimizing multi\-hop document retrieval through intermediate representations\.arXiv preprint arXiv:2503\.04796\.External Links:[Link](https://arxiv.org/abs/2503.04796)Cited by:[§I](https://arxiv.org/html/2607.03929#S1.p3.1),[§II\-B](https://arxiv.org/html/2607.03929#S2.SS2.p1.1),[§III\-C](https://arxiv.org/html/2607.03929#S3.SS3.p2.1),[§IV\-C](https://arxiv.org/html/2607.03929#S4.SS3.p2.1)\.
- \[11\]T\. Lin, P\. Goyal, R\. Girshick, K\. He, and P\. Dollár\(2017\)Focal loss for dense object detection\.InProceedings of the IEEE International Conference on Computer Vision \(ICCV\),pp\. 2980–2988\.External Links:[Link](https://doi.org/10.1109/ICCV.2017.324)Cited by:[§V](https://arxiv.org/html/2607.03929#S5.p3.2)\.
- \[12\]M\. Poliakov and N\. Shvai\(2024\)Multi\-Meta\-RAG: improving RAG for multi\-hop queries using database filtering with LLM\-extracted metadata\.InInformation and Communication Technologies in Education, Research, and Industrial Applications \(ICTERI 2024\),Communications in Computer and Information Science, Vol\.2359,pp\. 334–342\.Note:arXiv:2406\.13213External Links:[Document](https://dx.doi.org/10.1007/978-3-031-81372-6%5F25),[Link](https://doi.org/10.1007/978-3-031-81372-6_25)Cited by:[§I](https://arxiv.org/html/2607.03929#S1.p1.1),[§II\-C](https://arxiv.org/html/2607.03929#S2.SS3.p1.1),[§V](https://arxiv.org/html/2607.03929#S5.p1.1)\.
- \[13\]Qwen Team\(2024\)Qwen2\.5 technical report\.arXiv preprint arXiv:2412\.15115\.External Links:[Link](https://arxiv.org/abs/2412.15115)Cited by:[§II\-C](https://arxiv.org/html/2607.03929#S2.SS3.p1.1)\.
- \[14\]O\. Skean, M\. R\. Arefin, D\. Zhao, N\. Patel, J\. Naghiyev, Y\. LeCun, and R\. Shwartz\-Ziv\(2025\)Layer by layer: uncovering hidden representations in language models\.InProceedings of the 42nd International Conference on Machine Learning \(ICML\),Proceedings of Machine Learning Research, Vol\.267\.Note:arXiv:2502\.02013External Links:[Link](https://proceedings.mlr.press/v267/skean25a.html)Cited by:[§II\-B](https://arxiv.org/html/2607.03929#S2.SS2.p1.1),[§III\-C](https://arxiv.org/html/2607.03929#S3.SS3.p2.1),[§IV\-C](https://arxiv.org/html/2607.03929#S4.SS3.p2.1)\.
- \[15\]P\. Szymański and T\. Kajdanowicz\(2017\)A network perspective on stratification of multi\-label data\.InProceedings of the First International Workshop on Learning with Imbalanced Domains: Theory and Applications \(PMLR\),pp\. 22–35\.External Links:[Link](https://proceedings.mlr.press/v74/szyma%C5%84ski17a.html)Cited by:[§III\-C](https://arxiv.org/html/2607.03929#S3.SS3.p4.4)\.
- \[16\]Y\. Tang and Y\. Yang\(2024\)MultiHop\-RAG: benchmarking retrieval\-augmented generation for multi\-hop queries\.InProceedings of the First Conference on Language Modeling \(COLM\),Note:arXiv:2401\.15391External Links:[Link](https://arxiv.org/abs/2401.15391)Cited by:[§I](https://arxiv.org/html/2607.03929#S1.p1.1),[§II\-C](https://arxiv.org/html/2607.03929#S2.SS3.p1.1)\.
- \[17\]I\. Tenney, D\. Das, and E\. Pavlick\(2019\)BERT rediscovers the classical NLP pipeline\.InProceedings of the 57th Annual Meeting of the Association for Computational Linguistics \(ACL\),pp\. 4593–4601\.External Links:[Link](https://aclanthology.org/P19-1452/)Cited by:[§II\-A](https://arxiv.org/html/2607.03929#S2.SS1.p1.1)\.