STAIR (STructure Aware Information Retriever): A novel dataset and LLM based retriever for document structure augmentation

# STAIR (STructure Aware Information Retriever): A novel dataset and LLM based retriever for document structure augmentation
Source: [https://arxiv.org/html/2609.03874](https://arxiv.org/html/2609.03874)
Vineet Kumar111Work done while at IBM\. Currently at Amazon Books Science\.Meghanadh Pulivarthi Vishwajeet Kumar Jaydeep SenAffiliation:Riyaz Ahmad Bhat Sachindra JoshiAffiliation:IBMAffiliation:vineet\.mundhra@gmail\.com vishk024@in\.ibm\.comAffiliation:jaydesen@in\.ibm\.com Riyaz\.Bhat@ibm\.com jsachind@in\.ibm\.com

###### Abstract

Retrieval Augmented Generation \(RAG\) is a key component for generating accurate and hallucination free answers using Large Language Models \(LLMs\)\. LLMs are improving at handling long context, but still suffer from “lost in the middle” problem\. Thus, precise and accurate retrieval is important\. Current retrievers chunk long context into length\-based manageable chunks – in the process throwing away rich and informative semantic global structure in the corpus\. We introduce a novel retrieval systemSTAIRthat empowers an LLM to exploit global structure in a corpus such as a Table of Contents \(ToC\) to efficiently store and retrieve information from its model parameters\. Our thorough and careful ablation studies with a finetuned Differentiable Search Index \(DSI\) system show thatToChelps build a low hallucination \(less than 0\.05%\) generative Information Retrieval \(IR\) system and can generalize to examples where very few training samples are available\. To further research in this novel direction ofToCbased retrieval we releaseSearchTome– a diverse benchmark created from 18 books across 6 diverse domains to further research in this novel direction\.STAIRachieves a high Recall@1 score of 82\.6% onSearchTomeas compared to DSI \(76\.9%\), where the difference is found to be statistically significant\.STAIReasily beats other strong baselines such as BM25 \(59\.5%\), DPR \(68\.7%\) and out\-of\-the\-box Mistral \(13\.8%\)\. The benchmark data and code used for trainingSTAIRis available at[https://anonymous\.4open\.science/r/s\_331/README\.md](https://anonymous.4open.science/r/s_331/README.md)\.

## 1Introduction

The burgeoning interest in Retrieval Augmented Generation \(RAG\) has led to a significant surge in the development of advanced Information Retrieval \(IR\) systems\. Large Language Models \(LLMs\) in turn can now handle large contexts[Chen et al\. \(2023\)](https://arxiv.org/html/2609.03874#bib.bib3);[Liu et al\. \(2024a\)](https://arxiv.org/html/2609.03874#bib.bib2), though they suffer from a “lost in the middle” problem[Liu et al\. \(2024c\)](https://arxiv.org/html/2609.03874#bib.bib42);[Bai et al\. \(2024a\)](https://arxiv.org/html/2609.03874#bib.bib7);[Bai et al\. \(2024b\)](https://arxiv.org/html/2609.03874#bib.bib6);[Li et al\. \(2024\)](https://arxiv.org/html/2609.03874#bib.bib4)\. Therefore, retrieving precise information[Pipitone and Alami \(2024\)](https://arxiv.org/html/2609.03874#bib.bib5)is extremely important to curb hallucinations[Laban et al\. \(2024\)](https://arxiv.org/html/2609.03874#bib.bib43)and generate accurate responses\. Current retrievers address this by creating length based chunks[Setty et al\. \(2024\)](https://arxiv.org/html/2609.03874#bib.bib44)and throwing away rich and informative semantic global structure in the corpus\. This leads to a sub\-optimal retrieval quality – length\-based chunks compete with each other due to a lack of semantic coherence and boundaries\.

In this work, we address this key limitation by augmenting the retriever with a structured global view of the corpus\. Global structured view over a long context helps knowledge ingestion[Liu et al\. \(2024b\)](https://arxiv.org/html/2609.03874#bib.bib40)\. Further, LLMs are capable of storing the entire corpus in its model parameters to directly generate a document identifier for a user query[Tay et al\. \(2022\)](https://arxiv.org/html/2609.03874#bib.bib24)\. We posit that by empowering an LLM with a global structure of the search corpus, it can store and retrieve information more accurately from its model parameters\. Such a global structure already exists for a Wikipedia[Wikipedia \(2024\)](https://arxiv.org/html/2609.03874#bib.bib29)page, textbooks, and enterprise help and product feature webpages[SAP \(2024\)](https://arxiv.org/html/2609.03874#bib.bib30)and technical reports[SEC \(2024\)](https://arxiv.org/html/2609.03874#bib.bib31)\.

![Refer to caption](https://arxiv.org/html/2609.03874v1/images/f1_latex.png)Figure 1:A human can answer “What is the plurality voting system?” by looking at the table of contents and picking Section 4\.2\.3 without even reading the book[Introduction to Comparative Government and Politics](https://open.umn.edu/opentextbooks/textbooks/1435)\.Figure[1](https://arxiv.org/html/2609.03874#S1.F1)demonstrates that for a question such as “What is the plurality voting system?”, it is natural for a human to consult theToCand first narrow down that it could be answered by the Chapter “Democracies and Democratization”\. Further subsections reveal that perhaps the Section “Institutions within Democracy” and the subsection “Electoral Systems & Political Parties” could contain the answer\. Content for this subsection indeed contains the definition for the plural voting system\. Text content associated withToCentries is semantically coherent organized around a topic as captured by the title of theToCentry\. Further, aToCentry has a clear semantic boundary with otherToCentries\.ToCas a unit of retrieval is a novel yet intuitive way of organizing long context for retrieval\.

In this work, we presentSTAIR\(STructureAwareInformationRetriever\) forToCbased retrieval, inspired by how a human searches for information usingToC, and the fact that LLMs can store the entire corpus in the model’s parameters\. Our careful and thorough ablation studies reveal that empowering an LLM withToChelps in two critical dimensions \- curbing hallucinations and generalizing for content with a low number of training examples\.STAIRachieves a high Recall@1 score of 82\.6, which is is \(\+7\.4%\+7\.4\\%\) better than its closest baseline Differentiable Search Index \(DSI\)[Tay et al\. \(2022\)](https://arxiv.org/html/2609.03874#bib.bib24)\. DSI needs to discover the book structure \(indirectly\) through the completion part of training examples, whileSTAIRcan use exploit the structure provided as part of the input to accurately store and retrieve information from its model’s parameters\.

We develop and release a new multi\-domain benchmarkSearchTome, made up of1818books of varying sizes across66domains, parsed and cleaned to extract theirToCmapped to the text of the pertinent sections\. We also release a set of train, dev and test queries for each book as part of the benchmark, with goldToCentries labeled for each query serving as the retrieval output\. We believe this will be a first step towards promoting further research onToCbased retrieval, withSearchTomeserving as a standard evaluation benchmark\. We train and evaluateSTAIRusingSearchTomeand compare it with strong baseline retrievers like BM25 and a fully fine\-tuned Sequence\-to\-Sequence model, such as DSI[Tay et al\. \(2022\)](https://arxiv.org/html/2609.03874#bib.bib24), which does not utilizeToC\. Our experimental results show thatSTAIRsignificantly benefits from incorporatingToCin its training and easily outperforms the traditional DSI\. Our main contributions are as follows:

- •We propose a novel direction leveraging the Table of Contents \(ToC\) of a lengthy corpus, like a book, as a valuable element for efficiently indexing corpus knowledge and utilizingToCentries as effective retrieval units\.
- •We build and release a multi\-domain and diverse benchmarkSearchTome, consisting of1818books from66diverse domains for the task ofToCretrieval
- •We develop an LLM basedSTAIRand evaluate it onSearchTomeand compare it with strong competitive baselines

## 2Related Work

Efficient and precise retrieval is a critical research problem in natural language processing, with a wide range of applications such as semantic search[Guha et al\. \(2003\)](https://arxiv.org/html/2609.03874#bib.bib17), open domain question answering[Chen and Yih \(2020\)](https://arxiv.org/html/2609.03874#bib.bib16);[Zhu et al\. \(2021\)](https://arxiv.org/html/2609.03874#bib.bib15)and Retrieval Augmented Generation\(RAG\)[Lewis et al\. \(2020\)](https://arxiv.org/html/2609.03874#bib.bib14)to name a few\. With the rise of RAG applications, precise retrieval has become increasingly important, particularly in curbing hallucination in generative models\. We can broadly classify all retrieval techniques into three main categories:

Dense Retrieval[Cai et al\. \(2021\)](https://arxiv.org/html/2609.03874#bib.bib25);[Karpukhin et al\. \(2020\)](https://arxiv.org/html/2609.03874#bib.bib27)represent a query and document using a dense vector and compute the similarity based on the distance between their vectors\. DPR[Karpukhin et al\. \(2020\)](https://arxiv.org/html/2609.03874#bib.bib27)uses dual encoder, ColBERT[Khattab and Zaharia \(2020\)](https://arxiv.org/html/2609.03874#bib.bib28)represents every token in the query and document using dense vector and performs late interaction to score query and document pairs\. Recently,[Sarthi et al\. \(2024\)](https://arxiv.org/html/2609.03874#bib.bib8)introduced RAPTOR, a method for building a hierarchical tree by recursively embedding, clustering, and summarizing content from lengthy documents to retrieve relevant text at various levels of abstraction\. Our approach differs in two key aspects: while RAPTOR uses dense retrievers for indexing and retrieval, we adopt model\-based indexing\. Additionally, we enhance model\-based indexing by leveraging table\-of\-contents \(ToC\) structures, unlike RAPTOR, which relies solely on dense retrievers to access information at multiple levels of abstraction\.

Learned Sparse Retrievalsystems use sparse vector representation for a query and document\. Unlike traditional sparse retrieval methods like BM25[Robertson and Zaragoza \(2009\)](https://arxiv.org/html/2609.03874#bib.bib13), which rely on exact token matches, these models learn to represent queries and documents in a sparse high\-dimensional space, allowing for efficient lexical matching\. SPLADE[Hai et al\. \(2023\)](https://arxiv.org/html/2609.03874#bib.bib26)\(Sparse Lexical and Expansion Model for Information Retrieval\) is a prominent example of such systems\. Several other models have been developed to enhance sparse retrieval, including SPLADEv2[Formal et al\. \(2021\)](https://arxiv.org/html/2609.03874#bib.bib21), DeepCT[Dai and Callan \(2019\)](https://arxiv.org/html/2609.03874#bib.bib20), uniCOIL[Lin and Ma \(2021\)](https://arxiv.org/html/2609.03874#bib.bib19)and DeepImpact[Basnet et al\. \(2024\)](https://arxiv.org/html/2609.03874#bib.bib18)\.

Model\-based Indexing[Metzler et al\. \(2021\)](https://arxiv.org/html/2609.03874#bib.bib10)such as Differential Search Index \(DSI\)[Tay et al\. \(2022\)](https://arxiv.org/html/2609.03874#bib.bib24)embed knowledge of the entire corpus directly into its model parameters – greatly simplifying the retrieval process\.STAIRtakes this paradigm one step further by learning to exploit a rich hierarchal and semantic global structure within the corpus\.

## 3SearchTome: A new benchmark for Table of Contents based retrieval

Table 1:SearchTomeBook links & Statistics### 3\.1Breaking the Mold: Why a New Benchmark Is Needed

There are multiple benchmarks proposed around long context applications such as ContractNLI\([Koreeda and Manning, 2021](https://arxiv.org/html/2609.03874#bib.bib38)\)focused on NLI orScrolls\([Shaham et al\., 2022](https://arxiv.org/html/2609.03874#bib.bib39)\)with seven challenging tasks\. However, there doesn’t exist any benchmark for long context retrieval with any form of structured view, which we hypothesize as the key for precise retrieval\. Benchmarks such as GovReport\([Huang et al\., 2021](https://arxiv.org/html/2609.03874#bib.bib32)\), SummScreen\([Chen et al\., 2022](https://arxiv.org/html/2609.03874#bib.bib33)\), and QM\-Sum[Zhong et al\. \(2021\)](https://arxiv.org/html/2609.03874#bib.bib34)focus on generating summaries of a length document, whereas, Qasper\([Dasigi et al\., 2021](https://arxiv.org/html/2609.03874#bib.bib35)\), QuALITY\([Pang et al\., 2022](https://arxiv.org/html/2609.03874#bib.bib37)\)and NarrativeQA\([Kočiský et al\., 2018](https://arxiv.org/html/2609.03874#bib.bib36)\)focus on the generation aspect withonly answersfor a user query anddo not contain the gold passage for retrieval\. The closest benchmark isLocoV1\([Saad\-Falcon et al\., 2024](https://arxiv.org/html/2609.03874#bib.bib41)\)which does include gold passages for each query\. However, due tolack of ToC or a global structure for the input document, we cannot use this benchmark for our task\. To the best of our knowledge,SearchTomeis the first benchmark which provides a clean structured view with Table of Contents for evaluating long context retrieval with structures \.

### 3\.2SearchTome: A diverse and novel benchmark forToCRetrieval

Our main motivation for a new benchmark is to evaluate how well a global semantic structure such as aToCcan help in better retrieval for long context applications which is typical for technical reports or voluminous books\. Thus to create a clean and effective benchmark we turn to[opentextbooks](https://open.umn.edu/opentextbooks/), which is perhaps the largest collection of such textbooks\. To make the benchmark a comprehensive one across domains, we picked six diverse domains namely – Education, Finance, Law, Medicine, Natural Sciences and Social Sciences and selected three books from each domain \(Table[1](https://arxiv.org/html/2609.03874#S3.T1)\)\. We parsed the PDF for each book and extracted Table of Contents using pymupdf111[https://pymupdf\.readthedocs\.io/en/latest/](https://pymupdf.readthedocs.io/en/latest/)\. A sample of the final cleaned content for one of the books can be viewed[here](https://anonymous.4open.science/r/s_331/data/education/book1/v1/content.jsonl)\. For each paragraph we ask a powerful LLM Mixtral 8x7b model\([Jiang et al\., 2024](https://arxiv.org/html/2609.03874#bib.bib11)\)to generate multiple questions covering all important topics in the paragraph[Zhang et al\. \(2024\)](https://arxiv.org/html/2609.03874#bib.bib1)\. Following the same technique as was proposed in DSI[Tay et al\. \(2022\)](https://arxiv.org/html/2609.03874#bib.bib24)to train the parametric index, we use a portion of these questions for training DSI[Tay et al\. \(2022\)](https://arxiv.org/html/2609.03874#bib.bib24)andSTAIR, a small portion as development set which is to help us pick the best checkpoint\. Majority of the generated questions were picked as test questions – to thoroughly test coverage of content by an IR system\. We believe our benchmarkSearchTomewill further research in building IR systems which can leverage structure in a corpus and retrieve the most relevant chunks for a user query\.

## 4STAIR: Table of Contents Searcher

We now formally define the problem statement and the technique for retrieving a section title given a book and its Table of Contents \(ToC\)

### 4\.1Notations

We are given a long documentDDand its Table of ContentsToCD=\{T1,T2,…\.,Tn\}\{\{\\textbf\{\{ToC\}\}\}\}\_\{D\}=\\\{T\_\{1\},T\_\{2\},\.\.\.\.,T\_\{n\}\\\}\. An edgee:Tp→Tce:T\_\{p\}\\rightarrow T\_\{c\}between two nodesTpT\_\{p\},TcT\_\{c\}∈ToCD\\in\{\{\\textbf\{\{ToC\}\}\}\}\_\{D\}is defined asTpT\_\{p\}= parent\(TcT\_\{c\}\), if the section represented by titleTpT\_\{p\}is further divided into multiple sub\-sections includingTcT\_\{c\}\. The set of leaf nodes can thus be defined asL​NDLN\_\{D\}=\{Tl∈ToCD\|∃\!Tc∈ToCD,Tl=parent\_of\(Tc\)\}⊆ToCD\\\{T\_\{l\}\\in\{\\textbf\{\{ToC\}\}\}\_\{D\}\|\\exists\!T\_\{c\}\\in\{\\textbf\{\{ToC\}\}\}\_\{D\},T\_\{l\}=parent\\\_of\(T\_\{c\}\)\\\}\\subseteq\{\{\\textbf\{\{ToC\}\}\}\}\_\{D\}\. Our goal is to retrieve the correct leaf nodeTl∈L​NDT\_\{l\}\\in LN\_\{D\}whose content can answer a user queryqq\.

### 4\.2TrainingSTAIR

Figure 2:Overview of our pipeline\. Stage 1 prepares synthetic QA data from the input book\. Stage 2 fine\-tunes a language model to map a ToC\-augmented query to the correct leaf node\. Stage 3 runs inference with constrained generation, restricting the output vocabulary to valid ToC leaf nodes\.Figure[2](https://arxiv.org/html/2609.03874#S4.F2)depicts the training pipeline forSTAIRwhich learns two correlated butdifferenttasks: Corpus knowledge ingestion:STAIRneeds to learn the information troveDDby mapping a book’s text content to the corresponding section titlesTi∈ToCDT\_\{i\}\\in\{\\textbf\{\{ToC\}\}\}\_\{D\}\. This is facilitated by the training data, which consists of a series of queries that are linked to their corresponding section titles\. Consequently, this phase of training integrates the book’s knowledge into the LLM’s parameters, similar to model\-based indexing systems like DSI, but with a key difference\. Unlike other model\-based indexing systems, the parameters inSTAIRremember the corpus knowledge not by a document identifier, but by using fine\-grained section title entries in theToC, which are likely to have a much stronger semantic connection with the content being ingested into parametric knowledge\.

Learning to generate from ToC:STAIRlearns to pick the most relevant leaf node from the complete global structure \(ToC\) of a document\.

We use the training split fromSearchTometo trainSTAIRthrough supervised fine\-tuning\. During training, we provide the queryqqalong with the completeToCD\{\\textbf\{\{ToC\}\}\}\_\{D\}as input\. Consequently, the complete input during training consists ofprompt\+\(q,ToCD\)\(q,\{\\textbf\{\{ToC\}\}\}\_\{D\}\)\. Note thatToCD\{\\textbf\{\{ToC\}\}\}\_\{D\}, remains consistent across queries from the same book, while the prompt remains unchanged throughout the training process\. We fine\-tune the model to generate the correct leaf nodeTl∈L​NDT\_\{l\}\\in LN\_\{D\}that corresponds to the queryqq\.

Table 2:Sample training examplePrompt\#\#\# Book: Teaching in a Digital AgePick the best section from the table of contents belowwhich can answer the user query below\. Only generatethe section name, do not generate any explanation\!\#\#\# Table of Contents1 Chapter 1: Fundamental Change in Education⋯\\cdotsleaf 12\.7 Step five:master the technologyleaf 12\.8 Step six:set appropriate learning goalsleaf 12\.9 Step seven:design course structure and learning⋯\\cdotsleaf 13\.7 Building the future\#\#\# Query: What is the suggested time allocationfor students studying a course or program?\#\#\# Section: leafCompletion12\.9 Step seven: design course structure and learning

## 5Experiments

We report Recall@1 \(R@1\), Recall@3 \(R@3\) and Normalized Discounted Cumulative Gain \(nDCG@3\) metrics on the test set for each book\. BeIR[Thakur et al\. \(2021\)](https://arxiv.org/html/2609.03874#bib.bib22)is used for computing the metrics\.

### 5\.1Baselines

Mistral\(Out of the Box LLM\): Beam search is used to generate the Top\-KKpredictions for Mistral Instruct v0\.2 BM25[Robertson and Zaragoza \(2009\)](https://arxiv.org/html/2609.03874#bib.bib13): We index content using Elastic Search v 8\.11\.2\. Leaf node acts as a unique document identifier for a book DPR: We use NV\-Embed\-v2[Lee et al\. \(2024\)](https://arxiv.org/html/2609.03874#bib.bib9)out of the box with text lengths set to 512 for passages and 256 for queries\. NV\-Embed\-v2 is the leading text\-only embedding model with open weights on the MTEB leaderboard222As per Jan 23’2025 MTEB Leaderboard Snapshot\.\. The model is based on Mistral\-7B\-v0\.1333https://huggingface\.co/mistralai/Mistral\-7B\-v0\.1, making it a suitable baseline forSTAIRas both are finetuned over Mistral\. DSI\([Tay et al\., 2022](https://arxiv.org/html/2609.03874#bib.bib24)\)infuses the entire knowledge of a corpus in the parameters of an LLM and directly generates a document identifier\. We fine\-tune DSI using the train split for each book\.

### 5\.2Fine\-tuningSTAIRand DSI

We fine\-tune Mistral Instruct v0\.2444https://huggingface\.co/mistralai/Mistral\-7B\-Instruct\-v0\.2[Jiang et al\. \(2023\)](https://arxiv.org/html/2609.03874#bib.bib12)for a maximum of 200 epochs using a LoRA adapter[Hu et al\. \(2021\)](https://arxiv.org/html/2609.03874#bib.bib23)with ar=16r=16andα=32\\alpha=32\. We early stop with a patience of 20 epochs, by computing Recall@1 on the dev set\. We set the maximum input length to 14k tokens forSTAIRand512512tokens for DSI and a maximum output length of 64 tokens\. See Table[2](https://arxiv.org/html/2609.03874#S4.T2)for a sample training input and output forSTAIR\.

## 6Results and Analysis

Table 3:Examples of model predictions\. Gold leaf is highlighted inboldin Table of ContentsExample 1:Book:[The Whole Child: Development in the Early Years](https://open.umn.edu/opentextbooks/textbooks/1621)Query: How do preschoolers react when caregivers and teachers belittle their autonomous actions?Table of ContentsContent⋯\\cdots2 Chapter Two: Theorists and Theories of DevelopmentInitiative vs\. Guilt \(Preschool Years\)leaf 2\.1 Theories of DevelopmentThe development of courage and independence …2\.2 Psychosocial Theory…leaf 2\.2\.1 Trust vs\. Mistrust \(Infancy\)…leaf 2\.2\.2 Autonomy vs\. Shame/Doubt \(Toddlerhood\)…leaf 2\.2\.3 Initiative vs\. Guilt \(Preschool Years\)leaf 2\.2\.4 Industry vs\. Inferiority \(Elementary Years\)If caregivers and preschool teachers encourage and supportleaf 2\.2\.5 Identity vs\. Role\-Confusion \(Adolescence\)children’s efforts while also helping them make realisticleaf 2\.2\.6 Intimacy vs\. Isolation \(Early Adulthood\)and appropriate choices, children develop a healthy sense of initiative inleaf 2\.2\.7 Generativity vs\. Stagnation \(Middle Adulthood\)planning and undertaking activities If, instead, adults discourage theleaf 2\.2\.8 Integrity vs\. Despair \(Older Adulthood\)pursuit of independent activities or dismiss them as silly and⋯\\cdotsbothersome, children develop guilt about their needs and desires\.3 Chapter Three: Domains in DevelopmentMistral: 8\.3\.3 Moral Development, leaf 7\.3\.2 Hitting/Scratching and Temper Tantrums,leaf 2\.2\.3 Initiative vs\. Guilt \(Preschool Years\)BM25: 8\.3\.2 Moral DevelopmentDPR:2\.2\.2 Autonomy vs\. Shame/Doubt \(Toddlerhood\)DSI: 2\.2\.2 Autonomy vs\. Shame/Doubt \(Preschool Years\)STAIR:2\.2\.3 Initiative vs\. Guilt \(Preschool Years\)Example 2:Book:[Nursing Fundamentals](https://open.umn.edu/opentextbooks/textbooks/1013)Query: In what way does a chronic illness affect an elderly person’s ability to perform daily activities?Table of ContentsContent⋯\\cdotsleaf 30\.5 Spiritual Care of SelfApplying the Nursing Processleaf 30\.6 Putting It All Together…31 Care of the Older AdultIt is also important to consider the impact of chronic diseaseleaf 31\.1 Care of the Older Adult Introductionon their ability to function and complete Activities of Daily Livingleaf 31\.2 Basic Concepts\(ADLs\)\. older adults who are able to perform ADLs without assistanceleaf31\.3 Applying the Nursing Processconsider themselves healthy\.Mistral:31\.3 Applying the Nursing ProcessBM25: 11\.2 Sensory Impairments Basic ConceptsDPR:31\.2 Basic ConceptsDSI: 31\.2 Basic ConceptsSTAIR:31\.3 Applying the Nursing ProcessExample 3:Book:[Introduction to Comparative Government and Politics](https://open.umn.edu/opentextbooks/textbooks/1435)Query: What are irregular armed organizations and how are they used by states?Table of ContentsContent9 Chapter 5: Non\-Democracies and Democratic Backsliding⋯\\cdotsAnother powerful instrument of repression are paramilitaries\.9\.2 Strategies for staying in powerThese refer to groups with access to military\-grade weapons and trainingleaf 9\.2\.1Institutional channelsyet they are not part of the national military\.leaf 9\.2\.2 Cultural and ideological controlsThey are “irregular armed organizations that carry out acts of violence9\.3 Varieties of non\-democracyagainst civilians on behalf of a state,”⋯\\cdotsMistral: 9\.3\.5 Illiberal and hybrid regimesBM25: 15\.3\.1 Insurgencies/Civil WarsDPR: 15\.2\.2 External State\-Sponsored Political Violence \(State\-Sponsored Terrorism\)DSI: 15\.2\.1 Internal State\-Sponsored Political Violence \(Government Terrorism\)STAIR:9\.2\.1 Institutional channels

Table 4:Recall@1, Recall@3 and nDCG@3 forSearchTome; Haiku is Claude haiku 4\.5Table[4](https://arxiv.org/html/2609.03874#S6.T4)comparesSTAIRwith all the baselines listed in Section[5\.1](https://arxiv.org/html/2609.03874#S5.SS1)and addresses the following two research questions:

- •RQ1: DoesToCbased training makesSTAIRmore accurate as a retriever?
- •RQ2: Can we finetune an LLM to learn the new task of generating leaf nodes fromToC?

Our key findings are as follows:

1. 1\.STAIRoutperforms strong baselines such as BM25, DPR and fine\-tuned DSI
2. 2\.R@1 for Mistral is much lower than BM25 – this suggests the LLM not only needs to learn the new task of picking the best section but also needs to ingest knowledge from the corpus
3. 3\.DPR performs much better than BM25 but worse than DSI\. This result is expected, as NV\-Embed\-v2 is not fine\-tuned forSearchTome
4. 4\.Tuning withToCas input helpsSTAIR: The only difference in the input between DSI andSTAIRis thatSTAIRleveragesToC\.STAIRoutperforms DSI by7\.4%7\.4\\%, as it is able to access the entire structure of the book and better align it with the queries during fine\-tuning\. DSI on the other hand must learn the semantic alignment between queries and sections by looking at all the training data\. We can thus conclude thatRQ1andRQ2are answered in affirmative\.

Statistical significance testingwas done for the Recall@1 performance difference between DSI andSTAIR\. We follow the randomization test tailored to retrieval systems as described in[Smucker et al\. \(2007\)](https://arxiv.org/html/2609.03874#bib.bib45)\. The null hypothesis for the significance testing is that outputs from SystemA \(DSI\) and SystemB \(STAIR\) may belong to the same underlying distribution and the difference of performance between SystemA \(DSI\) and SystemB \(STAIR\) is because of the sampling variance\. We test the null hypothesis with the traditionalppvalue of 0\.05 at the domain level i\.e\. for each of the66domains separately \(by combining all the books from a single domain\)\. The significance test results conclude that the difference in DSI andSTAIRis indeed statistically significant for all the66domains where the null hypothesis is successfully invalidated\. More specifically, through our sample runs for randomization test, we find that the probability of seeing this scale of Recall@1 performance difference as observed in Table[4](https://arxiv.org/html/2609.03874#S6.T4)when being randomly sampled from the same distribution falls belowp=0\.05p=0\.05\. As with any statistical significance testing, the key takeaway from these experiments is a confirmation that the performance improvement seen inSTAIRis statistically significant and therefore, it is a property of theSTAIRdesign and not because of the data set size or data distribution inSearchTome\.

### 6\.1Ablations

We conduct ablation studies to study the impact of usingToCas an input\. Two key conclusions are as follows:

ToC reduces hallucinations: We define hallucination as the generation of a non\-leaf node \(invalid document identifier\)\. Figure[3](https://arxiv.org/html/2609.03874#S6.F3)illustrates the number of hallucinations per leaf node\. We observed that the hallucination rate ofSTAIRremains nearly constant at close to zero, regardless of the number of training examples\. This indicates thatSTAIRhas effectively learned to generate outputs solely from the leaf nodes, making it inherently less prone to hallucination\. In contrast, DSI, which does not useToC, exhibits a higher hallucination rate thanSTAIR, particularly for leaf nodes with fewer training examples\. Remembering the structure is hard without ToCFigure[4](https://arxiv.org/html/2609.03874#S6.F4)shows that for leaf nodes with less number of training examples, the Recall@1 difference between DSI andSTAIRis much higher\. This supports our hypothesis, that withoutToC, DSI needs to discover the book structure and also remember it within its parameters\. DSI attempts to learn this via the training data on query and associatedToCleaf node pairs\. Thus, leaf nodes which have low representations in training data are prone to be missed by DSI\. In contrast,STAIRdoes not need to “remember” the leaf nodes\. As the number of training examples for a leaf node increase, Recall@1 gap between DSI andSTAIRdecreases – althoughSTAIRconsistently achieves higher Recall@1 numbers\.

### 6\.2Error Analysis with Anecdotal Examples

We observe that out\-of\-the\-box Mistral’s error rate is 86\.20%, and 26\.81% of its mistakes are as it predicts a non\-leaf node, while 23\.86% are hallucinations\. This suggests that Mistral out\-of\-box needs knowledge ingestion and task finetuning\. For DSI, the error rate drops to 24\.31%, with 3\.25% of its predictions being non\-leaf nodes \(suggesting knowledge infusion helps\)\.STAIRhas the lowest error rate of 18\.67%, with only 0\.05% of its predictions being non\-leaf nodes\.

Figure 3:As number of training examples decrease, hallucination rate for DSI increases![Refer to caption](https://arxiv.org/html/2609.03874v1/images/hrate.png)Figure 4:Recall@1 for each leaf acrossSearchTome\.STAIRshows high accuracy even with a low number of training examples![Refer to caption](https://arxiv.org/html/2609.03874v1/images/acc.png)Table[3](https://arxiv.org/html/2609.03874#S6.T3)lists a few qualitative examples\. We discuss next the first example “How do preschoolers react when caregivers and teachers belittle their autonomous actions” from the book “The Whole Child: Development in the Early Years”:

Mistralcould correctly guess that the query can be answered by “2\.2\.3 Initiative vs\. Guilt \(Preschool Years\)” using its world knowledge\. It doesn’t understand that it needs to pick only one option – due to lack of the task training BM25suffers from the known bias of keyword matching where and picks the section “8\.3\.2\. Moral Development” purely because of highest lexical overlap with query tokens, which is the incorrect answer DPRis used out of the box and does a word sensed matching where it predicts “2\.2\.2 Autonomy vs\. Shame/Doubt \(Toddlerhood\)” being most relevant to the query tokens mentioning “belittling of autonomous actions”\. However, without the corpus specific training it falls short of identifying the correct section\. DSIis finetuned for the corpus but faulters to find the precise section\. It errs just like DPR in this example but might have chosen a more sensible match “2\.2\.2 Autonomy vs\. Shame/Doubt \(Preschool years\)” which takes into account the importance of keyword “preschool” for getting a better topic match\. However, careful consideration shows this is a hallucinated header not present in the inputToC\. This correlates with our ablation in figure[3](https://arxiv.org/html/2609.03874#S6.F3)on DSI hallucination\. STAIR: Our proposed systemSTAIRcorrectly identifies the gold leaf node as “Initiative vs\. Guilt \(Preschool Years\)”\.

The comparison between DPR, DSI andSTAIRis interesting as they all are built on top odf the same base LLM – Mistral\. The difference between DSI andSTAIRlargely points towards the importance of havingToCas an input during training, which makes it easier for the LLM to align much better to the corpus\. Other anecdotal examples in Table[3](https://arxiv.org/html/2609.03874#S6.T3)also enumerate the same partial ordering seen among the baselines and provide more qualitative analysis to support the claim that the design ofSTAIRhelps it do a much better retrieval\.

## 7Conclusion

We introduceSTAIR, a novel LLM based IR system that leveragesToCto store and retrieve information from its parameters\. Given a query,STAIRgenerates the most probably leaf section header from aToCwhich could answer it\. We use Mistral Instruct v0\.2 to demonstrate that it is possible to instruction finetune the modern day LLMs to efficiently use theToCstructure to generate the correct leaf node while reducing the hallucinations to almost zero\.STAIRoutperforms all strong baselines such as BM25, finetuned DSI and DPR and achieves a Recall@1 score of 82\.6% which is around7\.4%7\.4\\%gain over the next best system \(DSI\)\. We release a new comprehensive benchmarkSearchTomeacross66domains with1818books comprising of train, dev and test splits to further research in this novel direction\.

As a future work, we want to explore directions where aToClike structure is created dynamically over an unseen search corpus, as has been proposed by some previous works\. We envision that having aToC\-based retrieval paradigm will gain more traction in future for agentic frameworks needing multi\-hop retrieval and reasoning over retrieved context\. To that end, we want to developSTAIRto work in a complete zero shot setup where it iteratively retrievesToCleaf nodes and makes intelligent decisions by reasoning on the content of the leaf node to do precise information retrieval\.

## 8Limitations

Our current evaluation is limited to corpora where a global structure exists\. While this setting is suitable for initial validation, it may not fully represent the diversity of real\-world use cases\. In future work, we plan to extend our evaluation to standard benchmarks by artificially inducing a Table of Contents structure\. Additionally, we aim to test our model on enterprise datasets where such a structure already exists at a very large scale \(possibly millions of URLs in the corpus\)\.

## References

- Y\. Bai, X\. Lv, J\. Zhang, H\. Lyu, J\. Tang, Z\. Huang, Z\. Du, X\. Liu, A\. Zeng, L\. Hou, Y\. Dong, J\. Tang, and J\. LiLongBench: a bilingual, multitask benchmark for long context understanding\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),Bangkok, Thailand,pp\. 3119–3137\.External Links:[Link](https://aclanthology.org/2024.acl-long.172),[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.172)Cited by:[§1](https://arxiv.org/html/2609.03874#S1.p1.1)\.
- Baiet al\.\(2024b\)Y\. Bai, S\. Tu, J\. Zhang, H\. Peng, X\. Wang, X\. Lv, S\. Cao, J\. Xu, L\. Hou, Y\. Dong, J\. Tang, and J\. LiLongBench v2: towards deeper understanding and reasoning on realistic long\-context multitasks\.arXiv preprint arXiv:2412\.15204\.Cited by:[§1](https://arxiv.org/html/2609.03874#S1.p1.1)\.
- Basnetet al\.\(2024\)S\. Basnet, J\. Gou, A\. Mallia, and T\. SuelDeeperImpact: optimizing sparse learned index structures\.arXiv preprint arXiv:2405\.17093\.Cited by:[§2](https://arxiv.org/html/2609.03874#S2.p3.1)\.
- Caiet al\.\(2021\)Y\. Cai, Y\. Fan, J\. Guo, F\. Sun, R\. Zhang, and X\. ChengSemantic models for the first\-stage retrieval: a comprehensive review\.ACM Transactions on Information Systems \(TOIS\)40,pp\. 1 – 42\.External Links:[Link](https://api.semanticscholar.org/CorpusID:232147859)Cited by:[§2](https://arxiv.org/html/2609.03874#S2.p2.1)\.
- Chen and Yih \(2020\)D\. Chen and W\. YihOpen\-domain question answering\.InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics: Tutorial Abstracts,A\. Savary and Y\. Zhang \(Eds\.\),Online,pp\. 34–37\.External Links:[Link](https://aclanthology.org/2020.acl-tutorials.8),[Document](https://dx.doi.org/10.18653/v1/2020.acl-tutorials.8)Cited by:[§2](https://arxiv.org/html/2609.03874#S2.p1.1)\.
- Chenet al\.\(2022\)M\. Chen, Z\. Chu, S\. Wiseman, and K\. GimpelSummScreen: a dataset for abstractive screenplay summarization\.InProceedings of the 60th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),Dublin, Ireland,pp\. 8602–8615\.External Links:[Link](https://aclanthology.org/2022.acl-long.589),[Document](https://dx.doi.org/10.18653/v1/2022.acl-long.589)Cited by:[§3\.1](https://arxiv.org/html/2609.03874#S3.SS1.p1.1)\.
- Chenet al\.\(2023\)S\. Chen, S\. Wong, L\. Chen, and Y\. TianExtending context window of large language models via positional interpolation\.arXiv preprint arXiv:2306\.15595\.Cited by:[§1](https://arxiv.org/html/2609.03874#S1.p1.1)\.
- Dai and Callan \(2019\)Z\. Dai and J\. CallanContext\-aware sentence/passage term importance estimation for first stage retrieval\.arXiv preprint arXiv:1910\.10687\.Cited by:[§2](https://arxiv.org/html/2609.03874#S2.p3.1)\.
- Dasigiet al\.\(2021\)P\. Dasigi, K\. Lo, I\. Beltagy, A\. Cohan, N\. A\. Smith, and M\. GardnerA dataset of information\-seeking questions and answers anchored in research papers\.InProceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,Online,pp\. 4599–4610\.External Links:[Link](https://aclanthology.org/2021.naacl-main.365),[Document](https://dx.doi.org/10.18653/v1/2021.naacl-main.365)Cited by:[§3\.1](https://arxiv.org/html/2609.03874#S3.SS1.p1.1)\.
- Formalet al\.\(2021\)T\. Formal, C\. Lassance, B\. Piwowarski, and S\. ClinchantSPLADE v2: sparse lexical and expansion model for information retrieval\.arXiv preprint arXiv:2109\.10086\.Cited by:[§2](https://arxiv.org/html/2609.03874#S2.p3.1)\.
- Guhaet al\.\(2003\)R\. Guha, R\. McCool, and E\. MillerSemantic search\.InProceedings of the 12th International Conference on World Wide Web,WWW ’03,New York, NY, USA,pp\. 700–709\.External Links:ISBN 1581136803,[Link](https://doi.org/10.1145/775152.775250),[Document](https://dx.doi.org/10.1145/775152.775250)Cited by:[§2](https://arxiv.org/html/2609.03874#S2.p1.1)\.
- Haiet al\.\(2023\)N\. L\. Hai, T\. Gerald, T\. Formal, J\. Nie, B\. Piwowarski, and L\. SoulierCoSPLADE: contextualizing splade for conversational information retrieval\.ArXivabs/2301\.04413\.External Links:[Link](https://api.semanticscholar.org/CorpusID:255595720)Cited by:[§2](https://arxiv.org/html/2609.03874#S2.p3.1)\.
- Huet al\.\(2021\)J\. E\. Hu, Y\. Shen, P\. Wallis, Z\. Allen\-Zhu, Y\. Li, S\. Wang, and W\. ChenLoRA: low\-rank adaptation of large language models\.ArXivabs/2106\.09685\.External Links:[Link](https://api.semanticscholar.org/CorpusID:235458009)Cited by:[§5\.2](https://arxiv.org/html/2609.03874#S5.SS2.p1.1)\.
- Huanget al\.\(2021\)L\. Huang, S\. Cao, N\. Parulian, H\. Ji, and L\. WangEfficient attentions for long document summarization\.InProceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,Online,pp\. 1419–1436\.External Links:[Link](https://aclanthology.org/2021.naacl-main.112),[Document](https://dx.doi.org/10.18653/v1/2021.naacl-main.112)Cited by:[§3\.1](https://arxiv.org/html/2609.03874#S3.SS1.p1.1)\.
- Jianget al\.\(2024\)A\. Q\. Jiang, A\. Sablayrolles, A\. Roux, A\. Mensch, B\. Savary, C\. Bamford, D\. S\. Chaplot, D\. d\. l\. Casas, E\. B\. Hanna, F\. Bressand,et al\.Mixtral of experts\.arXiv preprint arXiv:2401\.04088\.Cited by:[§3\.2](https://arxiv.org/html/2609.03874#S3.SS2.p1.1)\.
- Jianget al\.\(2023\)A\. Q\. Jiang, A\. Sablayrolles, A\. Mensch, C\. Bamford, D\. S\. Chaplot, D\. de Las Casas, F\. Bressand, G\. Lengyel, G\. Lample, L\. Saulnier, L\. R\. Lavaud, M\. Lachaux, P\. Stock, T\. L\. Scao, T\. Lavril, T\. Wang, T\. Lacroix, and W\. E\. SayedMistral 7b\.ArXivabs/2310\.06825\.External Links:[Link](https://api.semanticscholar.org/CorpusID:263830494)Cited by:[§5\.2](https://arxiv.org/html/2609.03874#S5.SS2.p1.1)\.
- Karpukhinet al\.\(2020\)V\. Karpukhin, B\. Oğuz, S\. Min, P\. Lewis, L\. Y\. Wu, S\. Edunov, D\. Chen, and W\. YihDense passage retrieval for open\-domain question answering\.ArXivabs/2004\.04906\.External Links:[Link](https://api.semanticscholar.org/CorpusID:215737187)Cited by:[§2](https://arxiv.org/html/2609.03874#S2.p2.1)\.
- Khattab and Zaharia \(2020\)O\. Khattab and M\. A\. ZahariaColBERT: efficient and effective passage search via contextualized late interaction over bert\.Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval\.External Links:[Link](https://api.semanticscholar.org/CorpusID:216553223)Cited by:[§2](https://arxiv.org/html/2609.03874#S2.p2.1)\.
- Kočiskýet al\.\(2018\)T\. Kočiský, J\. Schwarz, P\. Blunsom, C\. Dyer, K\. M\. Hermann, G\. Melis, and E\. GrefenstetteThe NarrativeQA reading comprehension challenge\.Transactions of the Association for Computational Linguistics6,pp\. 317–328\.External Links:[Link](https://aclanthology.org/Q18-1023),[Document](https://dx.doi.org/10.1162/tacl%5Fa%5F00023)Cited by:[§3\.1](https://arxiv.org/html/2609.03874#S3.SS1.p1.1)\.
- Koreeda and Manning \(2021\)Y\. Koreeda and C\. ManningContractNLI: a dataset for document\-level natural language inference for contracts\.InFindings of the Association for Computational Linguistics: EMNLP 2021,Punta Cana, Dominican Republic,pp\. 1907–1919\.External Links:[Link](https://aclanthology.org/2021.findings-emnlp.164),[Document](https://dx.doi.org/10.18653/v1/2021.findings-emnlp.164)Cited by:[§3\.1](https://arxiv.org/html/2609.03874#S3.SS1.p1.1)\.
- Labanet al\.\(2024\)P\. Laban, A\. R\. Fabbri, C\. Xiong, and C\. WuSummary of a haystack: a challenge to long\-context llms and rag systems\.InConference on Empirical Methods in Natural Language Processing,External Links:[Link](https://api.semanticscholar.org/CorpusID:270869972)Cited by:[§1](https://arxiv.org/html/2609.03874#S1.p1.1)\.
- Leeet al\.\(2024\)C\. Lee, R\. Roy, M\. Xu, J\. Raiman, M\. Shoeybi, B\. Catanzaro, and W\. PingNV\-embed: improved techniques for training llms as generalist embedding models\.arXiv preprint arXiv:2405\.17428\.Cited by:[§5\.1](https://arxiv.org/html/2609.03874#S5.SS1.p1.1)\.
- Lewiset al\.\(2020\)P\. Lewis, E\. Perez, A\. Piktus, F\. Petroni, V\. Karpukhin, N\. Goyal, H\. Küttler, M\. Lewis, W\. Yih, T\. Rocktäschel,et al\.Retrieval\-augmented generation for knowledge\-intensive nlp tasks\.Advances in Neural Information Processing Systems33,pp\. 9459–9474\.Cited by:[§2](https://arxiv.org/html/2609.03874#S2.p1.1)\.
- Liet al\.\(2024\)T\. Li, G\. Zhang, Q\. D\. Do, X\. Yue, and W\. ChenLong\-context llms struggle with long in\-context learning\.ArXivabs/2404\.02060\.External Links:[Link](https://api.semanticscholar.org/CorpusID:268857023)Cited by:[§1](https://arxiv.org/html/2609.03874#S1.p1.1)\.
- Lin and Ma \(2021\)J\. Lin and X\. MaA few brief notes on deepimpact, coil, and a conceptual framework for information retrieval techniques\.arXiv preprint arXiv:2106\.14807\.Cited by:[§2](https://arxiv.org/html/2609.03874#S2.p3.1)\.
- Liuet al\.\(2024a\)H\. Liu, W\. Yan, M\. Zaharia, and P\. AbbeelWorld model on million\-length video and language with blockwise ringattention\.arXiv preprint arXiv:2402\.08268\.Cited by:[§1](https://arxiv.org/html/2609.03874#S1.p1.1)\.
- Liuet al\.\(2024b\)K\. Liu, Z\. Chen, Z\. Fu, R\. Jiang, F\. Zhou, Y\. Chen, Y\. Wu, and J\. YeStructure\-aware domain knowledge injection for large language models\.External Links:[Link](https://openreview.net/forum?id=Sc382pFw86)Cited by:[§1](https://arxiv.org/html/2609.03874#S1.p2.1)\.
- Liuet al\.\(2024c\)N\. F\. Liu, K\. Lin, J\. Hewitt, A\. Paranjape, M\. Bevilacqua, F\. Petroni, and P\. LiangLost in the middle: how language models use long contexts\.Transactions of the Association for Computational Linguistics12,pp\. 157–173\.External Links:[Link](https://aclanthology.org/2024.tacl-1.9/),[Document](https://dx.doi.org/10.1162/tacl%5Fa%5F00638)Cited by:[§1](https://arxiv.org/html/2609.03874#S1.p1.1)\.
- Metzleret al\.\(2021\)D\. Metzler, Y\. Tay, D\. Bahri, and M\. NajorkRethinking search: making experts out of dilettantes\.ArXivabs/2105\.02274\.External Links:[Link](https://api.semanticscholar.org/CorpusID:233864793)Cited by:[§2](https://arxiv.org/html/2609.03874#S2.p4.1)\.
- Panget al\.\(2022\)R\. Y\. Pang, A\. Parrish, N\. Joshi, N\. Nangia, J\. Phang, A\. Chen, V\. Padmakumar, J\. Ma, J\. Thompson, H\. He, and S\. BowmanQuALITY: question answering with long input texts, yes\!\.InProceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,Seattle, United States,pp\. 5336–5358\.External Links:[Link](https://aclanthology.org/2022.naacl-main.391),[Document](https://dx.doi.org/10.18653/v1/2022.naacl-main.391)Cited by:[§3\.1](https://arxiv.org/html/2609.03874#S3.SS1.p1.1)\.
- Pipitone and Alami \(2024\)N\. Pipitone and G\. H\. AlamiLegalBench\-rag: a benchmark for retrieval\-augmented generation in the legal domain\.ArXivabs/2408\.10343\.External Links:[Link](https://api.semanticscholar.org/CorpusID:271909426)Cited by:[§1](https://arxiv.org/html/2609.03874#S1.p1.1)\.
- Robertson and Zaragoza \(2009\)S\. Robertson and H\. ZaragozaThe probabilistic relevance framework: bm25 and beyond\.ACM Transactions on Information Systems27\(3\),pp\. 1–27\.Cited by:[§2](https://arxiv.org/html/2609.03874#S2.p3.1),[§5\.1](https://arxiv.org/html/2609.03874#S5.SS1.p1.1)\.
- Saad\-Falconet al\.\(2024\)J\. Saad\-Falcon, D\. Y\. Fu, S\. Arora, N\. Guha, and C\. R’eBenchmarking and building long\-context retrieval models with loco and m2\-bert\.ArXivabs/2402\.07440\.External Links:[Link](https://api.semanticscholar.org/CorpusID:267626843)Cited by:[§3\.1](https://arxiv.org/html/2609.03874#S3.SS1.p1.1.4)\.
- SAP \(2024\)SAPSAP\-product\-documentation\.Note:[https://help\.sap\.com/doc/saphelp\_snc70/7\.0/en\-US/ef/c08d5377a0ec23e10000000a174cb4/frameset\.htm](https://help.sap.com/doc/saphelp_snc70/7.0/en-US/ef/c08d5377a0ec23e10000000a174cb4/frameset.htm)Cited by:[§1](https://arxiv.org/html/2609.03874#S1.p2.1)\.
- Sarthiet al\.\(2024\)P\. Sarthi, S\. Abdullah, A\. Tuli, S\. Khanna, A\. Goldie, and C\. D\. ManningRaptor: recursive abstractive processing for tree\-organized retrieval\.arXiv preprint arXiv:2401\.18059\.Cited by:[§2](https://arxiv.org/html/2609.03874#S2.p2.1)\.
- SEC \(2024\)SECSEC 10k structure\.Note:[https://www\.sec\.gov/files/reada10k\.pdf](https://www.sec.gov/files/reada10k.pdf)Cited by:[§1](https://arxiv.org/html/2609.03874#S1.p2.1)\.
- Settyet al\.\(2024\)S\. Setty, K\. Jijo, E\. Chung, and N\. VidraImproving retrieval for rag based question answering models on financial documents\.ArXivabs/2404\.07221\.External Links:[Link](https://api.semanticscholar.org/CorpusID:269042793)Cited by:[§1](https://arxiv.org/html/2609.03874#S1.p1.1)\.
- Shahamet al\.\(2022\)U\. Shaham, E\. Segal, M\. Ivgi, A\. Efrat, O\. Yoran, A\. Haviv, A\. Gupta, W\. Xiong, M\. Geva, J\. Berant, and O\. LevySCROLLS: standardized CompaRison over long language sequences\.InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing,Abu Dhabi, United Arab Emirates,pp\. 12007–12021\.External Links:[Link](https://aclanthology.org/2022.emnlp-main.823)Cited by:[§3\.1](https://arxiv.org/html/2609.03874#S3.SS1.p1.1.1)\.
- Smuckeret al\.\(2007\)M\. Smucker, J\. Allan, and B\. CarteretteA comparison of statistical significance tests for information retrieval evaluation\.pp\. 623–632\.External Links:[Document](https://dx.doi.org/10.1145/1321440.1321528)Cited by:[§6](https://arxiv.org/html/2609.03874#S6.p3.1)\.
- Tayet al\.\(2022\)Y\. Tay, V\. Q\. Tran, M\. Dehghani, J\. Ni, D\. Bahri, H\. Mehta, Z\. Qin, K\. Hui, Z\. Zhao, J\. Gupta, T\. Schuster, W\. W\. Cohen, and D\. MetzlerTransformer memory as a differentiable search index\.ArXivabs/2202\.06991\.External Links:[Link](https://api.semanticscholar.org/CorpusID:246863488)Cited by:[§1](https://arxiv.org/html/2609.03874#S1.p2.1),[§1](https://arxiv.org/html/2609.03874#S1.p4.1),[§1](https://arxiv.org/html/2609.03874#S1.p5.1),[§2](https://arxiv.org/html/2609.03874#S2.p4.1),[§3\.2](https://arxiv.org/html/2609.03874#S3.SS2.p1.1),[§5\.1](https://arxiv.org/html/2609.03874#S5.SS1.p1.1.5)\.
- Thakuret al\.\(2021\)N\. Thakur, N\. Reimers, A\. Rücklé, A\. Srivastava, and I\. GurevychBEIR: a heterogeneous benchmark for zero\-shot evaluation of information retrieval models\.InThirty\-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track \(Round 2\),External Links:[Link](https://openreview.net/forum?id=wCu6T5xFjeJ)Cited by:[§5](https://arxiv.org/html/2609.03874#S5.p1.1)\.
- Wikipedia \(2024\)WikipediaWikipedia sample layout\.Note:[https://en\.wikipedia\.org/wiki/Wikipedia:Manual\_of\_Style/Layout](https://en.wikipedia.org/wiki/Wikipedia:Manual_of_Style/Layout)Cited by:[§1](https://arxiv.org/html/2609.03874#S1.p2.1)\.
- Zhanget al\.\(2024\)J\. Zhang, W\. Cui, Y\. Huang, K\. Das, and K\. SricharanSynthetic knowledge ingestion: towards knowledge refinement and injection for enhancing large language models\.InConference on Empirical Methods in Natural Language Processing,External Links:[Link](https://api.semanticscholar.org/CorpusID:273346804)Cited by:[§3\.2](https://arxiv.org/html/2609.03874#S3.SS2.p1.1)\.
- Zhonget al\.\(2021\)M\. Zhong, D\. Yin, T\. Yu, A\. Zaidi, M\. Mutuma, R\. Jha, A\. H\. Awadallah, A\. Celikyilmaz, Y\. Liu, X\. Qiu, and D\. RadevQMSum: a new benchmark for query\-based multi\-domain meeting summarization\.InProceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,Online,pp\. 5905–5921\.External Links:[Link](https://aclanthology.org/2021.naacl-main.472),[Document](https://dx.doi.org/10.18653/v1/2021.naacl-main.472)Cited by:[§3\.1](https://arxiv.org/html/2609.03874#S3.SS1.p1.1)\.
- Zhuet al\.\(2021\)F\. Zhu, W\. Lei, C\. Wang, J\. Zheng, S\. Poria, and T\. ChuaRetrieving and reading: a comprehensive survey on open\-domain question answering\.arXiv e\-prints,pp\. arXiv–2101\.Cited by:[§2](https://arxiv.org/html/2609.03874#S2.p1.1)\.