GuidedRAG: Semantic Steering of Retrieval-Augmented Generation

# GuidedRAG: Semantic Steering of Retrieval-Augmented Generation
Source: [https://arxiv.org/html/2607.26071](https://arxiv.org/html/2607.26071)
\(2026\)

###### Abstract\.

In this work, we proposeGuidedRAG, a novel extension to traditional Retrieval\-Augmented Generation \(RAG\) that introduces a dedicated selection stage and semantic steering during retrieval\. In contrast to current state\-of\-the\-art RAG approaches, which depend on increasingly complex retrieval and knowledge structures,GuidedRAGconstrains the knowledge base using semantics before retrieval, aligning the retrieval space with user intent while substantially reducing the search space\. Our evaluation shows thatGuidedRAGimproves retrieval relevance by 14\.0–15\.8%, mitigates a 19\.7–27\.4% loss in retrieval precision, and reduces retrieval overhead by orders of magnitude\. Moreover, relevant chunks are consistently retrieved earlier in the ranking process, while alignment with user intent improves by 31\.8–36\.8%\. We further show thatGuidedRAGachieves full coverage across 15 diverse RAG variants, demonstrating generalizability across the literature\. Together, these findings establish semantic steering and selections as a powerful and generalizable paradigm for improving the current state\-of\-the\-art in RAG\.

information retrieval, retrieval augmented generation, large language model, artificial intelligence, semantics

††copyright:acmcopyright††journalyear:2026††conference:45thTwente Student Conference on IT; July 3, 2026; Enschede, The Netherlands††ccs:Information systems Information retrieval![Refer to caption](https://arxiv.org/html/2607.26071v1/x1.png)Figure 1\.InGuidedRAGwe investigate five stages, namely: TheConfiguration,Selection,Retrieval,Augmentation, andGenerationstage\. TheConfigurationstage specifies the knowledge base \(KB\) and underlying semantics, and configures the degree of user control\. In the Retrieval\-Augmented Generation \(RAG\) process, a user specifiessemanticsand aprompt, which are used in subsequent steps\. In theSelectionstage, a subset of the KB is created based on the provided semantics\. This subset is then used to apply ranking in theRetrievalstage to find relevant chunks\. The user\-specified prompt, semantics and retrieved chunks are then augmented into a new prompt in theAugmentationstage, after which a Large Language Model \(LLM\) generates a response in theGenerationstage\.Pipeline describing GuidedRAG## 1\.Introduction

The rapid advancement ofLarge Language Models \(LLMs\)has contributed significantly to natural language understanding and generation across a wide range of applications, includingInformation Retrieval \(IR\) Systems\(Zhai,[2024](https://arxiv.org/html/2607.26071#bib.bib1); Joho and Jose,[2025](https://arxiv.org/html/2607.26071#bib.bib20); Zhuet al\.,[2025](https://arxiv.org/html/2607.26071#bib.bib21)\)\. However, despite their impressive capabilities, LLMs remain limited by their static training data and susceptibility to generating factually incorrect or outdated information\(Lewiset al\.,[2020](https://arxiv.org/html/2607.26071#bib.bib3); Brownet al\.,[2020](https://arxiv.org/html/2607.26071#bib.bib22)\)\.Retrieval\-Augmented Generation \(RAG\)seeks to mitigate these issues by augmenting prompts with external knowledge retrieved from a knowledge base at inference time, thereby improving factual accuracy and adherence to domain\-specific knowledge\(Lewiset al\.,[2020](https://arxiv.org/html/2607.26071#bib.bib3)\)\.

While RAG has demonstrated substantial benefits, its effectiveness is fundamentally dependent on the quality and relevance of the retrieval process\(Huang and Huang,[2026](https://arxiv.org/html/2607.26071#bib.bib2); Lewiset al\.,[2020](https://arxiv.org/html/2607.26071#bib.bib3)\)\. Traditional RAG\(Lewiset al\.,[2020](https://arxiv.org/html/2607.26071#bib.bib3)\)systems often rely on broad semantic similarity searches over large document collections, which can introduce irrelevant context, increase computational costs, and reduce retrieval precision\(Amirazet al\.,[2025](https://arxiv.org/html/2607.26071#bib.bib18); Zhaoet al\.,[2024](https://arxiv.org/html/2607.26071#bib.bib19)\)\. As knowledge bases continue to grow in size and complexity, these limitations become increasingly pronounced, negatively impacting both efficiency and response quality\. Furthermore, approaches such asGraphRAG\(Edgeet al\.,[2025](https://arxiv.org/html/2607.26071#bib.bib4)\)rely heavily on generated entity annotations\. As a result, inaccuracies or omissions in these annotations can introduce a semantic gap between the user’s intent and the knowledge retrieved\(Volkanovska,[2025](https://arxiv.org/html/2607.26071#bib.bib23)\)\.

To address these challenges, we proposeGuidedRAGwhich extends traditional RAG approaches by introducing a\(i\)selection stage and\(ii\)semantic steering during the canonical RAG process\. In the selection stage, a knowledge base \(KB\) is scoped to form a selection space using semantic information, such as categories or entities\. This reduces retrieval overhead by narrowing the search scope, thereby making RAG systems more scalable\. Furthermore, it improves retrieval relevance by grounding the search space in high\-level semantics\. As a result, retrieval focuses on semantically relevant portions of the corpus, reducing the inclusion of irrelevant context\. Moreover, while standard approaches rely solely on a user prompt to steer retrieval,GuidedRAGimproves user control by allowing pre\-specified semantics to steer retrieval\. This results in closer alignment between user intent and machine interpretation\. In this work, we demonstrate howGuidedRAGcan improve efficiency, accuracy, and precision versus traditional RAG approaches\.

Main RQ:How doesGuidedRAGcompare to state\-of\-the\-art retrieval\-augmented generation approaches in terms of\(i\)retrieval relevance,\(ii\)retrieval efficiency, and\(iii\)retrieval precision?

1. \(i\)Retrieval Relevance:How does the relevance and accuracy of retrieved context compare againstRAGandGraphRAG?
2. \(ii\)Retrieval Efficiency:How much do semantic selections improve retrieval efficiency and ranking of relevant context?
3. \(iii\)Retrieval Precision:To what extent is the inclusion of irrelevant context reduced compared toRAGandGraphRAG?

## 2\.Background

LLMs are neural networks, specifically transformers\(Vaswaniet al\.,[2017](https://arxiv.org/html/2607.26071#bib.bib8)\), trained on large collections of text to perform natural language understanding and generation tasks\. During training, knowledge is encoded in the model’s parameters, enabling it to generate responses based on patterns learned from the training corpus\. While this approach has proven highly effective, it also introduces several limitations\. Since knowledge is stored in pre\-trained parameters, LLMs cannot directly access information unavailable during training\(Lewiset al\.,[2020](https://arxiv.org/html/2607.26071#bib.bib3); Robertset al\.,[2020](https://arxiv.org/html/2607.26071#bib.bib24)\)\. Furthermore, domain\-specific knowledge may be underrepresented, resulting in inaccurate or incomplete responses\(Lewiset al\.,[2020](https://arxiv.org/html/2607.26071#bib.bib3)\)\. This can additionally result in hallucinations, where LLMs make up facts or claims\(Anh\-Hoanget al\.,[2025](https://arxiv.org/html/2607.26071#bib.bib9)\)\. These limitations motivate the use of external knowledge sources during inference, thereby grounding knowledge and further augmenting the context window and understanding of LLMs\(Lewiset al\.,[2020](https://arxiv.org/html/2607.26071#bib.bib3)\)\.

Retrieval\-Augmented Generation \(RAG\)\(Lewiset al\.,[2020](https://arxiv.org/html/2607.26071#bib.bib3)\)addresses the limitations of static knowledge by retrieving information from an external KB and incorporating it alongside a user\-generated query provided to an LLM\. As a result, generated responses can be grounded in information beyond the model’s pre\-trained parameters\(Lewiset al\.,[2020](https://arxiv.org/html/2607.26071#bib.bib3)\)\. Traditional RAG systems rely on embedding models and dense retrieval\. Documents, i\.e\., chunks of text, are transformed into embeddings and stored in a \(vector\) database, while user queries are converted into embeddings using the same embedding model according to the vector space model\(Saltonet al\.,[1975](https://arxiv.org/html/2607.26071#bib.bib10)\)\. Relevant chunks are then retrieved by comparing embedding similarity\(Lewiset al\.,[2020](https://arxiv.org/html/2607.26071#bib.bib3)\)\. In recent years, the definition of what constitutes a RAG system has grown to include alternative approaches such asGraphRAG\(Edgeet al\.,[2025](https://arxiv.org/html/2607.26071#bib.bib4)\)\. These approaches generally change one or multiple components in the canonical RAG process of retrieval, augmentation, and generation\. In GraphRAG, a knowledge graph \(KG\)\(Hoganet al\.,[2021](https://arxiv.org/html/2607.26071#bib.bib6)\)is used to provide relevant semantic context during retrieval to incorporate relational context during the generation stage\. In this work, we will evaluateGuidedRAGagainst traditional RAG and GraphRAG, as these are two popular yet distinct approaches that are commonly cited in the literature and used in industry\(Huang and Huang,[2026](https://arxiv.org/html/2607.26071#bib.bib2); Hanet al\.,[2025](https://arxiv.org/html/2607.26071#bib.bib7)\)\. Although RAG certainly improves access to external knowledge, retrieval quality often declines as the knowledge base grows\(Izacardet al\.,[2023](https://arxiv.org/html/2607.26071#bib.bib25)\)\. Larger search spaces increase computational costs and can introduce irrelevant and misleading context into the retrieval process\(Izacardet al\.,[2023](https://arxiv.org/html/2607.26071#bib.bib25)\)\. Furthermore, retrieval noise can cascade into the generation stage, leading to hallucinations\(Anh\-Hoanget al\.,[2025](https://arxiv.org/html/2607.26071#bib.bib9)\), incorrect or flawed responses, and increased token usage, all of which persist even with large context windows\(Duet al\.,[2025](https://arxiv.org/html/2607.26071#bib.bib26); Lewiset al\.,[2020](https://arxiv.org/html/2607.26071#bib.bib3)\)\.

The effectiveness of a RAG\-based system is largely determined by the quality of its retrieval process\(Lewiset al\.,[2020](https://arxiv.org/html/2607.26071#bib.bib3); Izacardet al\.,[2023](https://arxiv.org/html/2607.26071#bib.bib25)\)\. Since generated responses depend on the retrieved context as input, retrieval should provide information that is both relevant and efficient to obtain\(Izacardet al\.,[2023](https://arxiv.org/html/2607.26071#bib.bib25); Lewiset al\.,[2020](https://arxiv.org/html/2607.26071#bib.bib3)\)\. Retrieving too much information can cause information to be diluted or the context window to overflow, while retrieving too little might cause relevant context to be lost during inference\(Duet al\.,[2025](https://arxiv.org/html/2607.26071#bib.bib26); Vaswaniet al\.,[2017](https://arxiv.org/html/2607.26071#bib.bib8)\)\. Therefore, it is important to evaluate strategies that achieve higher retrieval relevance and that score relevant chunks more favorably\. This will allow for the inclusion of fewer chunks during augmentation, reducing token usage and generation delays\. Furthermore, retrieval should remain resilient in the face of ever\-growing knowledge bases while retrieving context that closely aligns with user intent\(Izacardet al\.,[2023](https://arxiv.org/html/2607.26071#bib.bib25)\)\. Improving accuracy alone is not sufficient; precision is also important to avoid hallucinations when information is absent from the KB\(Lewiset al\.,[2020](https://arxiv.org/html/2607.26071#bib.bib3); Izacardet al\.,[2023](https://arxiv.org/html/2607.26071#bib.bib25)\)\. These requirements motivate the use of a selection stage inGuidedRAG; this stage narrows the KB to include semantically relevant regions during retrieval\. This reduces retrieval overhead, as embedding similarity is compared over a smaller corpus, and improves response quality by considering only semantically relevant chunks during retrieval\. As semantics are grounded in the KB, it would be possible to determine whether relevant chunks exist, improving precision\. Furthermore, by leveraging semantics, the canonical RAG process can be steered to employ retrieval or augmentation techniques tailored to the scenario, such as vector similarity\(Saltonet al\.,[1975](https://arxiv.org/html/2607.26071#bib.bib10)\)or maximal marginal relevance\(Carbonell and Goldstein,[1998](https://arxiv.org/html/2607.26071#bib.bib11)\)\. The use of semantic steering and selections remains underexplored in existing RAG literature and the current state\-of\-the\-art approaches\.

## 3\.Framework

We presentGuidedRAGas illustrated in Figure[1](https://arxiv.org/html/2607.26071#S0.F1)\. Adding aConfigurationandSelectionstage that facilitates semantic steering in the canonicalRetrieval,Augmentation, andGenerationstages\.

### 3\.1\.Configuration

TheConfigurationstage defines the KB and user interactions within a RAG system\. KBs can take different forms, such as vector databases that use the vector space model\(Saltonet al\.,[1975](https://arxiv.org/html/2607.26071#bib.bib10)\), or knowledge graphs\(Hoganet al\.,[2021](https://arxiv.org/html/2607.26071#bib.bib6)\)that contain entities, chunks, and their relations\. Moreover, this stage defines the prompt and semantics that are used in subsequent stages, resulting in a selection over the KB or adjustments in retrieval and augmentation approaches\. In general, the prompt and semantics are generated by a user\. However, depending on system requirements, semantics can be automatically deduced from a prompt or alternatively inferred from system interactions\. Therefore, prompt creation and semantic inference are reliant on a balance between user expertise and control\. The semantics can serve multiple roles within the framework, including defining retrieval scopes through KB metadata, guiding the retrieval stage toward relevant information, and enriching the augmented prompt with supplementary context\.

Example:A user specifies a prompt containing a factoid question and two semantics:Entity: Rome,Question Type: Factoid\.

### 3\.2\.Selection

In theSelectionstage, semantics are used to create a selection over the entire KB\. This selection then creates a subset of the KB used in subsequent stages\. Depending on the selection, the corpus is narrowed, improving efficiency and relevance during retrieval\. A user or system may also specify no selection, in which case the entire KB is used\. As a consequence, the system uses the underlying RAG approach as is\. Semantics used for selection can be explicitly specified by a user or determined through user interactions within a system\. In the latter case, a system may guide a user to create suitable selections\. A selection may consist of one or multiple selection methods; these methods have been defined in Figure[2](https://arxiv.org/html/2607.26071#S3.F2), grounded in IR literature\(Baloget al\.,[2011](https://arxiv.org/html/2607.26071#bib.bib14); Garigliotti and Balog,[2017](https://arxiv.org/html/2607.26071#bib.bib13)\), and can be used to create a selection space\. Similar to query languages, such as SQL, different selections form sets that can be manipulated usingmathematical set theoryoperations, such as: union\(∪\)\(\\cup\), intersection\(∩\)\(\\cap\), symmetric difference\(△\)\(\\triangle\), etc\.

Example:The selection stage finds all relevant chunks related toEntity: Rome, drastically reducing the search space\.

![Refer to caption](https://arxiv.org/html/2607.26071v1/x2.png)Figure 2\.Selection methods use different semantics \(e\.g\., entities or relations\) to create subsets over a KB, allowing for retrieval from semantically relevant regions of the KB\. For example, selecting chunks from2002\-\-2026 AD\.
### 3\.3\.Retrieval

For theRetrievalstage, various retrieval methods can be used, depending on the underlying RAG approach\. If a selection was specified, a given retrieval technique will rank relevant chunks within the selection space, rather than across the entire KB\. This makes retrieval far more efficient, depending on the specificity of the selection, as ranking metrics are computed over a smaller corpus\. In traditional RAG methods, ranking techniques are statically configured and rely exclusively on a user prompt as input\. In contrast, inGuidedRAG, semantics can be used to switch ranking techniques\. For example, an open question may benefit from a diverse ranking method such as Maximum Marginal Relevance \(MMR\)\(Carbonell and Goldstein,[1998](https://arxiv.org/html/2607.26071#bib.bib11)\), whereas a factoid question may be better served by a similarity\-based ranking method\(Saltonet al\.,[1975](https://arxiv.org/html/2607.26071#bib.bib10)\)\. Additionally, the semantics can make ranking more diverse by enabling the use of new ranking techniques\. This results in potential improvements in retrieval performance\. A powerful consequence of incorporating semantics into the retrieval process is the ability to determine whether information is actually present within the KB\. When the specified semantics cannot be matched with equivalent semantics in the KB, the retrieval stage can infer that the relevant information is likely absent\. This improves precision and reduces the number of false positives that would have otherwise been retrieved\. In traditional RAG approaches, retrieval mechanisms typically return the most similar chunks based on similarity scores, even when no truly relevant information exists\(Kamathet al\.,[2020](https://arxiv.org/html/2607.26071#bib.bib12); Izacardet al\.,[2023](https://arxiv.org/html/2607.26071#bib.bib25)\)\.

Example:Based on the specifiedQuestion Type: Factoid, the retrieval stage usescosine similarityto retrieve relevant chunks\.

### 3\.4\.Augmentation

In theAugmentationstage, chunks from the retrieval stage are augmented, along with a user query, into a system\-specified prompt format\. The chunks are used in the later generation stage to ground responses in the provided context, improving faithfulness\(Tamberet al\.,[2025](https://arxiv.org/html/2607.26071#bib.bib27)\)\. InGuidedRAG, semantics can additionally be added to the augmented prompt to provide further context during the generation stage\. This approach is already used in RAG frameworks such as GraphRAG\(Edgeet al\.,[2025](https://arxiv.org/html/2607.26071#bib.bib4)\), where relational triplets are augmented into a prompt\. In contrast,GuidedRAGallows for any specification of semantics, generalizing this approach\. Furthermore, the semantics can also be used to employ different prompting strategies\. This results in different prompt formats, or LLMs, enabling better task descriptions and the use of more suitable models for specific use\-cases or requirements\.

Example:The augmentation stage combines the prompt, chunks, and semantics, providing relevant additional context\.

### 3\.5\.Generation

Finally, in theGenerationstage, a response is generated using the augmented prompt and returned to a user\. In traditional RAG methods, a response may indicate what chunks were utilized to improve transparency\. Using the semantics inGuidedRAG, it would be further possible to provide additional context, such as sources or metadata from the retrieved chunks\. Furthermore, as the reduction in the search space removes irrelevant context, chunks are retrieved earlier in rankings\. This results in potentially fewer chunks in the augmented prompt, reducing response time and token usage\.

Example:The generation stage responds to the user, using the augmented prompt\. As the selection and retrieval resulted in fewer, but more relevant chunks, the response was generated more quickly and accurately, and consumed fewer tokens\.

## 4\.Methodology

### 4\.1\.Dataset

In order to validateGuidedRAGagainst base RAG\(Lewiset al\.,[2020](https://arxiv.org/html/2607.26071#bib.bib3)\)and GraphRAG\(Edgeet al\.,[2025](https://arxiv.org/html/2607.26071#bib.bib4)\), it is essential to use a knowledge base that supports both graph\- and vector\-based retrieval\. Furthermore, the knowledge base must contain textual chunks, entities, and relations, as well as an extensive set of questions paired with corresponding ground truths \(i\.e\., answers\)\. For the evaluation ofGuidedRAG, these questions must also be accompanied by semantic annotations to enable appropriate use of the selection stage and semantic steering\. The current state\-of\-the\-art dataset, given the constraints, isAll Relations Lead to Rome\(ARLtR\), created by Jansen op de Haar et al\.\(op de Haaret al\.,[2026](https://arxiv.org/html/2607.26071#bib.bib42)\)\.ARLtRis a knowledge graph stored in Neo4j111https://neo4j\.com/containing chunks, entities, and relations that have been annotated by an LLM\. The dataset contains data from 300 Wikipedia articles about theRoman Empire\.

ARLtRincludes 6,000 answerable question\-answer pairs, denoted withQaQ\_\{a\}\. Each question has a set of attributes that can be used in the evaluation, as illustrated in Table[1](https://arxiv.org/html/2607.26071#S4.T1)\. These attributes serve as semantics to define a selection space during retrieval\. Questions are categorized as either open\-ended or factoids\. In the latter case, a unique ground\-truth answer exists, enabling relevance assessment of retrieved chunks\. Furthermore, the answers are grounded in the KB; so each question is linked to a specific entity, such asRomeorHannibal\. Thecomplexityattribute defines the number of entities \(i\.e\., one \(C1C\_\{1\}\) or two \(C2C\_\{2\}\)\) in a question, while therelationalattribute indicates whether it concerns an entity or its relation\. The latter is particularly important when evaluating GraphRAG\(Edgeet al\.,[2025](https://arxiv.org/html/2607.26071#bib.bib4)\), as it generally performs better on multi\-hop and relation\-centric questions\(Hanet al\.,[2025](https://arxiv.org/html/2607.26071#bib.bib7); Edgeet al\.,[2025](https://arxiv.org/html/2607.26071#bib.bib4)\)\. Moreover, eachbase questionhas three formulations that correspond to levels of user expertise\. This distinction is commonly used in the field ofHuman\-Computer Interaction \(HCI\)\(Oulasvirtaet al\.,[2011](https://arxiv.org/html/2607.26071#bib.bib15)\)and provides a better representation of user expertise levels\.ARLtRincludes an additional 2,400 unanswerable question\-answer pairs about semantics that are not present in the KB, denoted withQnQ\_\{n\}\. This facilitates precision evaluation, as RAG approaches may incorrectly mark chunks as relevant, leading to false positives\.

Table 1\.ARLtRQuestion Attributes
### 4\.2\.RAG Methods

To compare traditional RAG approaches againstGuidedRAG, we implement base RAG\(Lewiset al\.,[2020](https://arxiv.org/html/2607.26071#bib.bib3)\)and GraphRAG\(Edgeet al\.,[2025](https://arxiv.org/html/2607.26071#bib.bib4)\), two widely used approaches in the literature, along with their correspondingGuidedRAGcounterparts\. The implementation for each approach, corresponding to theGuidedRAGstages, is described in Figure[3](https://arxiv.org/html/2607.26071#S4.F3)\. To improve reproducibility, each approach uses cosine similarity\(Saltonet al\.,[1975](https://arxiv.org/html/2607.26071#bib.bib10)\)for retrieval to reduce variance caused by the choice of retrieval method rather than the influence of semantic steering\. Traditional RAG\(Lewiset al\.,[2020](https://arxiv.org/html/2607.26071#bib.bib3)\)makes use of a vector\-based KB and cosine similarity to find thet​o​p−ktop\-krelevant chunks, wherekkdefines the number of chunks retrieved\. ItsGuidedRAGcounterpart uses a pre\-specified entity \(EE\) to narrow the corpus before applying cosine similarity\. Meanwhile, for GraphRAG\(Edgeet al\.,[2025](https://arxiv.org/html/2607.26071#bib.bib4)\), an entity is tagged using an LLM or Embedding Model \(EM\), after which the tagged entity is matched to one in the KB\. This entity is then used to applyk−h​o​p​sk\-hops; expanding the selection space via connected entities to include relevant additional context, wherekkis the number of hops taken\. In contrast, theGuidedRAGcounterpart to GraphRAG uses predefined entities and relations, allowing relation hops that expand the selection space over specific relation types rather than all relations\. These relation hops are often infeasible in traditional GraphRAG systems because inaccuracies in entity taggers can introduce irrelevant context into the selection space\. In the implementation and evaluation, we use an LLM \(i\.e\.,minimax 2\.7222https://www\.minimax\.io/models/text/m27\) and an EM \(i\.e\.,gemini\-embedding\-2333https://deepmind\.google/models/gemini/embedding/\)\.

![Refer to caption](https://arxiv.org/html/2607.26071v1/x3.png)Figure 3\.RAG Methods overGuidedRAGFramework Stages
### 4\.3\.Selection and Retrieval

The evaluation will use a best\-effort strategy, depending on the RAG approach, to showcase performance across the various question configurations inARLtRas set out in Table[1](https://arxiv.org/html/2607.26071#S4.T1)\. Furthermore, as performance in the augmentation and generation stages largely depends on the choice of LLM, we will primarily evaluate the selection and retrieval stages\. These stages will best illustrate the impact of introducingGuidedRAG, in particular on retrieval performance\.

In addition,ARLtRprovides entities and relations that define the question configuration\. Using this information, we use thecomplexityandrelationalattributes to determine which operations are used to construct a selection space\. Each configuration, and its subsequent selection space, is shown in Table[2](https://arxiv.org/html/2607.26071#S4.T2)\. The selection spaces are defined usingmathematical set theory\. In base RAG, the entire corpus is selected, which is denoted byUU\. TheGuidedRAGcounterpart makes use of pre\-specified entities to scope the corpus, denoted byEnE\_\{n\}, wherenndenotes thent​hn\_\{th\}entity\. In the event of a Double \(C2C\_\{2\}\) Entity configuration, theGuidedRAGcounterpart uses the intersection operation\(∩\)\(\\cap\), as said chunks will contain contextual information about both entities\. Furthermore, when using GraphRAG, a similar selection is used for the Entity questions\. However, we use a more conservative strategy, employing the union operation\(∪\)\(\\cup\), for theC2C\_\{2\}Entity configuration\. This is because incorrect entity tagging may obstruct relevant context\. For relational questions, thek−h​o​pk\-hopstrategy is used, in which the selection space is expanded from a tagged entity to include neighbors linked by relations; these neighbors are denoted by the setAA\. In theGuidedRAGcounterpart, rather than usingk−h​o​p​sk\-hops, relational hops can be used to expand the selection space according to specific relation types rather than all relations\.

Table 2\.Retrieval Configurations and Corresponding SelectionsConfigurationRAGRAG\*GraphRAGGraphRAG\*C1C\_\{1\}EntityUUE1E\_\{1\}E1E\_\{1\}E1E\_\{1\}C2C\_\{2\}EntityUUE1∩E2E\_\{1\}\\cap E\_\{2\}E1∪E2E\_\{1\}\\cup E\_\{2\}E1∩E2E\_\{1\}\\cap E\_\{2\}C1C\_\{1\}RelationUUE1E\_\{1\}E1∪Ah​o​pE\_\{1\}\\cup A\_\{hop\}E1∩Ar​e​lE\_\{1\}\\cap A\_\{rel\}C2C\_\{2\}RelationUUUUE1∪A2∗h​o​pE\_\{1\}\\cup A\_\{2\*hop\}Ar​e​lA\_\{rel\}\*GuidedRAGvariants\.

## 5\.Evaluation

### 5\.1\.Entity Tagging

In order to compare the different RAG approaches, we first examine the impact of using automated entity taggers versus pre\-specified semantics\. The former are widely used in approaches such as GraphRAG\(Edgeet al\.,[2025](https://arxiv.org/html/2607.26071#bib.bib4)\)\. In our comparison, we investigate three approaches: an\(i\)LLM\-based tagger,\(ii\)cosine similarity, and a\(iii\)hybrid approachin which LLM\-generated tags are matched using an EM \(i\.e, with 3,072 dimensions\)\. The evaluation will include 300 open questions and 300 factoid questions, with an even split over thecomplexityattribute\. This results in a total of 900 entities to be tagged, sinceC2C\_\{2\}questions are paired with two entities\.

The results are shown in Table[3](https://arxiv.org/html/2607.26071#S5.T3)\. We observe that a naivecosine similaritytagging strategy severely underperforms compared to the other two methods\. Meanwhile, theLLM\-based taggingstrategies perform better, with slightly over half of the tags correctly identified\. The combined method achieves the highest performance because the standard LLM tagger may use incorrect entity formulations, which are corrected during the cosine similarity step\. Nevertheless, when compared against the ground truth, entities are often misaligned\. This is a direct result of question formulations, as entities might not always be explicitly mentioned or, in rare cases, might be formulated incorrectly, such as with the novice user type\. It is important to note that users may also choose suboptimal semantics, leading to suboptimal selections\. Nevertheless, in automated tagging methods, a discrepancy between user intent and a written prompt can arise, a problem thatGuidedRAGavoids because semantics are pre\-specified\.

![Refer to caption](https://arxiv.org/html/2607.26071v1/x4.png)Figure 4\.Percentage of Correct Tags per Tagging MethodIn addition, automated tagging strategies are computationally expensive because they require LLMs and EMs\. This increases computational complexity, as shown in Table[3](https://arxiv.org/html/2607.26071#S5.T3), resulting in greater token usage and overhead that would otherwise be avoided with pre\-specified semantics\. Furthermore, in Figure[4](https://arxiv.org/html/2607.26071#S5.F4), we observe that as the complexity attribute \(CC\) increases, the automated tagging strategies perform sub\-optimally\. This is particularly true forcosine similarity, which otherwise has a respectable performance forC1C\_\{1\}type questions\. The poor performance is due to usingcosine similarityon a single question to determine multiple entities; often shifting focus on a single entity rather than identifying multiple entities\.

Table 3\.Complexity and Accuracy over Tagging Methods
### 5\.2\.Retrieval Relevance

To evaluate retrieval relevance, we examine the performance of the four RAG approaches, as set out in Figure[3](https://arxiv.org/html/2607.26071#S4.F3), on a subset of 600 factoid questions fromQaQ\_\{a\}\. Each factoid has a correspondingfact\(i\.e\., answer\), grounded in the KB\. Utilizing thisfact, we can determine whether a chunk is relevant to answer a question in the generation stage\. In comparing the RAG approaches, we evaluate whether each approach can correctly retrieve thefactfrom at least one of five chunks per question \(i\.e\.,t​o​p−ktop\-kchunks, wherek=5k=5\)\. This results in 12,000 total chunks; each having its relevance annotated as a Boolean value by an LLM at a temperature of zero\. These annotations are kept consistent across each RAG approach for a givenbase question\.

Furthermore, entity semantics are taken directly fromARLtRrather than being automatically tagged prior to retrieval\. This distinction mitigates the influence of entity\-tagging variations, which would otherwise confound comparison\. The annotations were averaged over three separate iterations to reduce variance in the tagging process, as the inference process is non\-deterministic\. In addition, to validate the annotation procedure, a representative subset of the LLM annotations, comprising 20% of the corpus, was independently annotated by the author, yielding aCohen’sκ\\kappaof 92\.8%\(Cohen,[1960](https://arxiv.org/html/2607.26071#bib.bib28)\)\.

Table 4\.Top\-K Hit StatisticsApproachAvg\. Hit \(k\)Median Hit \(k\)Relevant \(%\)RAG3\.523\.059\.7%RAG\*3\.082\.068\.2%GraphRAG3\.322\.566\.1%GraphRAG\*2\.711\.076\.6%\*GuidedRAGvariants\.In Figure[5](https://arxiv.org/html/2607.26071#S5.F5), the percentage of answerable questions \(i\.e\., using afact\) per configuration is shown\. We observe thatGuidedRAGapproaches perform considerably better than their traditional counterparts and even outperform state\-of\-the\-art approaches such as GraphRAG\(Edgeet al\.,[2025](https://arxiv.org/html/2607.26071#bib.bib4)\)\. Interestingly, theGuidedRAGvariant of base RAG performs better than GraphRAG, despite the latter being highly resource\-intensive and requiring a knowledge graph\. The addition of a selection space yields substantial improvements forentitytype questions, particularly those of high complexity, where multiple entities may mislead thecosine similarityranking\. Interestingly, only theGuidedRAGvariant of GraphRAG does well on therelationaltype questions, while regular GraphRAG underperforms on this metric\. This is due to thek−h​o​pk\-hopstrategy, which expands the search scope over all relation types, barely narrowing the search space\. However, it should be noted that much of the performance in GraphRAG is a result of the augmentation stage, where relations are added into a prompt\(Edgeet al\.,[2025](https://arxiv.org/html/2607.26071#bib.bib4); Hanet al\.,[2025](https://arxiv.org/html/2607.26071#bib.bib7)\)\. Therefore, even if the retrieval stage does not retrieve thefact, the relations augmented into the prompt may still yield a correct response during generation\. On the contrary, it is important to note that the evaluation uses ground\-truth entity annotations, which favor GraphRAG\. Overall,GuidedRAGperforms much better and retrieves more relevant context\. In particular, theGuidedRAGvariant of GraphRAG performs extremely well on the dataset by fully leveraging the provided semantics\.

![Refer to caption](https://arxiv.org/html/2607.26071v1/x5.png)Figure 5\.Percentage of Questions with at Least One Correct ChunkTable[4](https://arxiv.org/html/2607.26071#S5.T4)illustrates the average and mediankt​hk\_\{th\}position in thet​o​p−ktop\-kranking where a givenfactwas retrieved \(i\.e\.,hit\)\. Each metric accounts for questions where nofactwas identified, which are weighted as having afactatk=6k=6, favoring larger search spaces and thereby traditional RAG approaches\. We observe that theGuidedRAGapproaches encounter afactmuch higher in thet​o​p−ktop\-kranking when compared to their traditional counterparts\.

InARLtRallfactsare grounded in the KB, which means that relevant chunks will eventually appear as thekkparameter int​o​p−ktop\-kranking increases to include the entire KB \(UU\)\. Hence, systems often improve recall by increasingkkand retrieving more chunks, which provide additional context\. However, this comes at the cost of token usage and computational complexity, as the generation stage will require an LLM to process a larger input prompt\. Moreover, despite large context windows, this can lead to inaccurate responses\(Duet al\.,[2025](https://arxiv.org/html/2607.26071#bib.bib26)\), or thelost in the middleeffect, in which LLMs neglect the middle section of a given prompt\(Liuet al\.,[2024](https://arxiv.org/html/2607.26071#bib.bib16)\)\.GuidedRAGmitigates these issues, as relevant chunks are, on average, retrieved higher in the similarity ranking\. This allows systems to handle a lowerkkparameter in thet​o​p−ktop\-kretrieval\. This results in reduced computational overhead, improved accuracy, and decreased token usage during generation\.

### 5\.3\.Retrieval Efficiency

To assess retrieval efficiency, we investigate the reduction in search space\(S\)\(S\)during retrieval resulting from the introduction of a selection stage\. The minimum, average, and maximum search space, for each selection in Table[2](https://arxiv.org/html/2607.26071#S4.T2), are depicted in Table[5](https://arxiv.org/html/2607.26071#S5.T5)\. The total number of chunks inARLtRis 16,069, which represents the search space when using base RAG; ascosine similarityis applied across the entire corpus\(U\)\(U\)\. The results indicate a severe reduction in the search space across different selections, decreasing the computational overhead by orders of magnitude\. The use ofGuidedRAGyields very specific subsets of the KB, improving recall and making retrieval substantially faster, as embedding distances are calculated over a fraction of the corpus\. In addition, unlike traditional RAG approaches, embedding computations are dependent on the average number of chunks per selection space \(e\.g\., per entity\), rather than the corpus size, thereby makingGuidedRAGscalable even with much larger and complex corpora\. Furthermore, when usingk−h​o​p​sk\-hops, we observe that with the increase ofkk, the scope of the search space rapidly increases\. This is an emergent property in graphs caused by the low degree of separation between any two entities, also known as thesmall world problem\(Watts and Strogatz,[1998](https://arxiv.org/html/2607.26071#bib.bib17)\)\. Consequently, this reduces the impact of automated tagging misalignment\. However, it results in a sizable search space and additional overhead forGraphRAGduring retrieval, which likely explains its moderate retrieval relevance\. It is important to note thatARLtRis a dense knowledge graph; therefore, the average search space is higher than in sparser graphs\.

Table 5\.Search Spaces per SelectionIn Figure[6](https://arxiv.org/html/2607.26071#S5.F6), we report retrieval turbulence\(T\)\(T\)per RAG approach across three paraphrased formulations\(P\)\(P\)of 200 distinctbase questionsfromQaQ\_\{a\}, which denote question intent\. Each formulation corresponds to a given level of user experience, as shown in Table[1](https://arxiv.org/html/2607.26071#S4.T1)\. Each formulation is provided inARLtRand hast​o​p−ktop\-kretrieved chunks, wherek=5k=5\. Turbulence is computed as a function of the number of distinct retrieved chunks across these formulations, denotedDcD\_\{c\}\.

\(1\)T=Dc−k\(P⋅k\)−ko​rT=Dc−510T=\\frac\{D\_\{c\}\-k\}\{\(P\\cdot k\)\-k\}\\quad\\quad or\\quad\\quad T=\\frac\{D\_\{c\}\-5\}\{10\}
Interestingly, theGuidedRAGvariants have substantially reduced turbulence across different formulation levels compared to their traditional counterparts\. This suggests that differences in user expertise have a smaller effect on retrieval, and further supports our claim thatGuidedRAGmore closely aligns with user intent\. This is most certainly a consequence of the reduced search space, as small differences in similarity scoring are less likely to change the ranking results\. Nevertheless, this shows great promise in facilitating usage across different levels of user expertise and intent alignment, as retrieval is less dependent on prompt formulation and user expertise\.

![Refer to caption](https://arxiv.org/html/2607.26071v1/x6.png)Figure 6\.Turbulence of Chunks in Retrieval \(0 = low, 1 = high\)
### 5\.4\.Retrieval Precision

Retrieval precision is evaluated using a representative sample of 540 unanswerable questions on non\-existent semantics sampled fromQnQ\_\{n\}\. In traditional RAG systems, chunk relevance is determined using a cosine similarity score that decides thet​o​p−ktop\-kranking\. Additionally, a threshold score is used to determine whether a question is answerable: if thet​o​p−ktop\-kranking contains chunks below this threshold, the system assumes no answer exists\. In the comparison, we evaluate similarity scores using unanswerable questions, which are then compared with scores from acontrol setof answerable questions taken fromQaQ\_\{a\}\. The former indicates a loss of precision, as chunks are retrieved for questions with no answers\. We evaluate RAG and GraphRAG on both question sets\.GuidedRAGis omitted because it scopes the KB using predefined semantics\. This makes the search space equivalent to the empty set \(Ø\), resulting in no loss of precision\. In Figure[7](https://arxiv.org/html/2607.26071#S5.F7), a cumulative percentage is shown, indicating the relative amount of chunks retrieved at a given threshold\. A high threshold will improve precision but comes at a proportional cost to recall, as relevant chunks with lower similarity are not retrieved\.

![Refer to caption](https://arxiv.org/html/2607.26071v1/x7.png)Figure 7\.Cumulative Chunks Retained at Similarity Threshold \(%\)In Figure[7](https://arxiv.org/html/2607.26071#S5.F7), we observe that, at a threshold of 0\.83, the RAG approaches retrieve the majority of chunks in the control set, while retrieving 19%–28% from the unanswerable set\. Therefore, 0\.83 is a sensible threshold that balances precision and recall\. Nevertheless, retrieval precision remains low: on average, a quarter of the total corpus is retrieved as false positives\. Interestingly, GraphRAG achieves higher precision because its semantics are used to scope the KB, resulting in fewer high\-similarity hits\. This suggests a trend in which more precise semantics yield higher precision, allowing for a lower threshold, which in turn improves recall\. This is a striking result, as it underscores the main reason forGuidedRAG’s high overall performance\. Traditional approaches cannot retrieve additional context beyond a given threshold because retrieval is constrained by thet​o​p−ktop\-kranking\. In contrast, by using semantics,GuidedRAGcan retrieve relevant chunks that fall far below this threshold, thereby improving recall without compromising on precision as semantically unrelated chunks are excluded\. This suggests thatGuidedRAGmitigates the precision\-recall dilemma inherent to traditional RAG approaches and IR\(Raghavanet al\.,[1989](https://arxiv.org/html/2607.26071#bib.bib29)\)\. Moreover, as non\-existent entities return the empty set \(Ø\),GuidedRAGcan indicate prior to retrieval whether a givenfactdoes not exist in the KB\. This can mitigate unnecessary computations, which would otherwise introduce overhead, waste tokens, and lead to false responses and hallucinations\(Anh\-Hoanget al\.,[2025](https://arxiv.org/html/2607.26071#bib.bib9); Zhuet al\.,[2025](https://arxiv.org/html/2607.26071#bib.bib21); Huang and Huang,[2026](https://arxiv.org/html/2607.26071#bib.bib2); Izacardet al\.,[2023](https://arxiv.org/html/2607.26071#bib.bib25)\)\.

## 6\.Discussion

### 6\.1\.General Implications

We observed thatGuidedRAGimproves\(i\) retrieval relevanceby considering semantically relevant chunks, and increasing recall by ranking these chunks more highly in thet​o​p−ktop\-kranking\. Consequently, RAG systems can generate more accurate responses across different domains and KBs\. Furthermore, we have shown thatGuidedRAGoutperforms state\-of\-the\-art approaches, such as GraphRAG \(\+\+15\.8%\), even when only extending base RAG \(\+\+2\.1%\)\. These gains enable more scalable systems without sacrificing accuracy or recall, as state\-of\-the\-art approaches typically incur additional time and space complexity to achieve comparable or lower improvements\.

The results indicate a substantial improvement in\(ii\) retrieval efficiencyforGuidedRAG, as embedding distances are calculated over smaller search spaces\. This means that retrieval is faster in RAG systems and, consequently, that corpus size can scale substantially without slowing retrieval\. The latter is explained by retrieval speed scaling with the average number of chunks per search space \(e\.g\., per entity\) rather than with the corpus size\. This drastically improves scalability, facilitating larger systems that can answer a wider range of questions\. This effectively gives LLMs more domain\-specific knowledge, enabling them to provide better responses to users\. Furthermore, as token usage is reduced through the smallert​o​p−ktop\-kranking or avoided as unanswerable questions can be mitigated prior to retrieval, this can make the operation of RAG systems considerably more affordable\. The smallert​o​p−ktop\-kranking can also encourage the use of small open\-source LLMs with reduced context windows, which is particularly useful for RAG systems that handle privacy\-sensitive data, such as in the medical or financial domains\.

The evaluation suggests thatGuidedRAGavoids the loss of\(iii\) retrieval precisionobserved in traditional RAG approaches \(19%–28%\) when using unanswerable questions\. A striking result is that the similarity threshold underlines a precision\-recall dilemma, inherent to traditional RAG approaches\. This dilemma is circumvented through semantic steering by identifying and excluding non\-existent semantics prior to retrieval, facilitating lower threshold values, and improving recall\. This may lead to new use cases for RAG systems, as designers are less constrained by the precision\-recall dilemma\. In addition, through semantic steering, it would be possible to use alternative retrieval, augmentation, or generation methods to further improve performance\. This would, for example, allow the use of MMR instead of cosine similarity for open questions\.

Another benefit ofGuidedRAGis implicit in its design\. Traditional RAG systems rely solely on user\-generated prompts to steer IR\. This means that steering in IR relies solely on variations in prompt formulation\. Meanwhile, inGuidedRAG, a user can additionally steer IR by specifying semantics\. This directly improves user control by giving users two steering methods\. Furthermore, by reducing retrieval turbulence caused by different user formulations,GuidedRAGmore closely aligns with user intent than traditional RAG approaches\. This accommodates a wider range of user expertise levels\. Nevertheless, there is a contrast: introducing semantic steering may require experience, since users need know\-how to choose relevant semantics\. Despite this gap, systems can guide users to select appropriate semantics through user interactions\. As an example, a user may select a location on a map, which the system then uses as a semantic to applyGuidedRAG\. In addition, while we evaluated entity selection, alternative selection methods \(i\.e\., see Figure[2](https://arxiv.org/html/2607.26071#S3.F2)\) can further increaseGuidedRAG’s flexibility, making it easier to adapt to diverse applications while still using the same underlying KB\.

### 6\.2\.Generalizability

In our evaluation, we focus exclusively on the comparison with traditional RAG and GraphRAG, despite the existence of many alternative approaches\. To assess the generalizability ofGuidedRAG, we therefore map15distinct RAG approaches onto our framework and examine whether they can be fully expressed within its structure\. This is achieved by explicitly defining, for each stage inGuidedRAG, the method or mechanism used in the corresponding approach, as shown in Table[6](https://arxiv.org/html/2607.26071#A1.T6)\. Using this procedure, we obtainedfull coverageacross all evaluated RAG variants\. Importantly,GuidedRAGdoes not aim to replace existing RAG approaches, but rather to unify and extend them\. This follows directly from its design: if the selection stage includes the full corpus \(UU\), the search space\(S\)\(S\)reduces to the underlying RAG approach\. As a result,GuidedRAGcan be applied to any RAG approach that adheres to the canonical retrieval, augmentation, and generation scheme, making it broadly applicable\.

### 6\.3\.Limitations and Future Work

GuidedRAGrelies on semantics to enable semantic steering, making its effectiveness dependent on the presence and quality of these semantics in the KB\. InARLtR, semantics are introduced through LLM\-based chunk annotations, enabling entity\-level specifications, at the cost of computational overhead and potential noise\(op de Haaret al\.,[2026](https://arxiv.org/html/2607.26071#bib.bib42)\); fine\-grained entity semantics may not always be available or practical, as they require chunk\-level annotations\. Nevertheless,GuidedRAGcan also operate on higher\-level semantic information, such as document sources, dates, or categories, which are far less resource\-intensive to obtain as they rely on document\-level annotation while still substantially reducing the search space\. In future work, it is important to investigate alternative semantic selection methods, which remain underexplored in our evaluation and present further opportunities \(see Figure[2](https://arxiv.org/html/2607.26071#S3.F2)\)\. In particular, the use ofmanual selectionsmay shed light on the relation between user control and experience\.

The evaluation was conducted onARLtR, as no other benchmark currently exists for semantic steering in RAG\. Consequently, the reported results may not directly generalize to other knowledge bases or application domains\. In addition,ARLtRuses LLM\-generated semantics to construct the selection space\. AsGuidedRAGrelies on the quality of these semantics, inaccuracies in the automated annotations may reduce its effectiveness\. Nevertheless, LLM\-generated semantics are widely used in practice, makingARLtRrepresentative of many existing knowledge bases used for RAG\(op de Haaret al\.,[2026](https://arxiv.org/html/2607.26071#bib.bib42)\)\. Future work should evaluateGuidedRAGacross additional benchmarks as they become available, as well as investigate how different annotation strategies influence the performance of semantic steering\.

In the evaluation, we usedcosine similarity, a widely used retrieval method in RAG approaches\. However, other retrieval methods exist that, for example, employ re\-ranking to improve chunk relevance\. The use of semantic steering is applicable across a wide range of retrieval methods, since the selection space forms a new corpus over which retrieval is applied as is\. Nevertheless, the performance ofGuidedRAGmay vary across different retrieval methods, and the improvements shown in the evaluation will vary across different pipelines\. EvaluatingGuidedRAGin combination with alternative retrieval methods may provide further insight into how semantic steering interacts with these approaches\. In particular, it may prove to be a more resource\-efficient alternative or complement to more computationally expensive retrieval techniques, while providing comparable or higher improvements in retrieval performance\.

The current evaluation ofGuidedRAGfocuses primarily on the retrieval stage\. However,GuidedRAGincludes several other stages, such as the augmentation and generation stage, that could be explored in future work\. These stages may highlight additional benefits, as semantic steering can be used to employ alternative augmentation or generation strategies that best fit user questions\. For example, GraphRAG augments its prompts with relations, which are then used to answer relational questions during the generation stage\(Edgeet al\.,[2025](https://arxiv.org/html/2607.26071#bib.bib4)\)\. Therefore, semantic steering could lead to holistic pipeline changes by combining traditional IR paradigms with RAG\.

While this work evaluatesGuidedRAGat the level of RAG approaches, some existing RAG systems implicitly implement forms of semantic steering through user interactions that influence the canonical RAG stages\(Tianet al\.,[2026](https://arxiv.org/html/2607.26071#bib.bib30)\)\. These mechanisms are often not explicitly modeled as semantic steering, but effectively induce similar behaviour at the system level\.GuidedRAGcan serve as a unifying abstraction for these interactions, enabling a more systematic description, comparison, and design of such systems\. Future work should therefore explore howGuidedRAGcan be applied at the system level to unify and analyse different RAG systems\.

## 7\.Conclusion

In this paper, we investigated the impact of introducing semantic steering and adding selections to traditional RAG approaches throughGuidedRAG\. Traditional approaches rely on overhead in space and time complexity to improve performance in theRetrieval,Augmentation, andGenerationstages\. Furthermore, said approaches perform poorly on precision and recall, as chunks with high similarity scores are naively added to the context window, even when they lack semantic relevance to the underlying question, thereby bloating retrieval\.GuidedRAGmitigates these effects drastically by introducing aSelectionstage that makes retrieval and generation far more efficient, precise, and accurate\. In our evaluation, we determined that computational overhead across the entire canonical RAG process can be reduced by leveraging semantics, avoiding automated entity tagging, and reducing the search space by orders of magnitude\. Moreover, compared to state\-of\-the\-art RAG approaches, a noticeable improvement in retrieval relevance is observed due to the removal of semantically irrelevant chunks from the search space\. In addition,GuidedRAGprovides a solution to the precision\-recall dilemma by simultaneously avoiding loss of precision while improving recall\. Overall, these benefits allow for more scalable systems\. Besides performance metrics,GuidedRAGoffers a way to increase user control, allowing users to steer the inference process using semantics while reducing turbulence\. This makesGuidedRAGa powerful tool for aligning user intent with machine interpretation\.

## References

- C\. Amiraz, F\. Cuconasu, S\. Filice,et al\.\(2025\)The distracting effect: understanding irrelevant passages in rag\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 18228–18258\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.acl-long.892),[Link](http://dx.doi.org/10.18653/v1/2025.acl-long.892)Cited by:[§1](https://arxiv.org/html/2607.26071#S1.p2.1)\.
- D\. Anh\-Hoang, V\. Tran, and L\. Nguyen \(2025\)Survey and analysis of hallucinations in large language models: attribution to prompting strategies or model behavior\.Frontiers in Artificial IntelligenceVolume 8 \- 2025\.External Links:[Document](https://dx.doi.org/10.3389/frai.2025.1622292),[Link](https://www.frontiersin.org/journals/artificial-intelligence/articles/10.3389/frai.2025.1622292)Cited by:[§2](https://arxiv.org/html/2607.26071#S2.p1.1),[§2](https://arxiv.org/html/2607.26071#S2.p2.1),[§5\.4](https://arxiv.org/html/2607.26071#S5.SS4.p2.1)\.
- A\. Asai, Z\. Wu, Y\. Wang,et al\.\(2024\)Self\-RAG: learning to retrieve, generate, and critique through self\-reflection\.InThe Twelfth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=hSyW5go0v8)Cited by:[Table 6](https://arxiv.org/html/2607.26071#A1.T6.3.8.5.2)\.
- K\. Balog, M\. Bron, and M\. De Rijke \(2011\)Query modeling for entity search based on terms, categories, and examples\.ACM Trans\. Inf\. Syst\.29\(4\),pp\. 1–31\.External Links:[Document](https://dx.doi.org/10.1145/2037661.2037667),[Link](https://doi.org/10.1145/2037661.2037667)Cited by:[§3\.2](https://arxiv.org/html/2607.26071#S3.SS2.p1.3)\.
- S\. Borgeaud, A\. Mensch, J\. Hoffmann,et al\.\(2022\)Improving language models by retrieving from trillions of tokens\.External Links:[Link](https://arxiv.org/abs/2112.04426)Cited by:[Table 6](https://arxiv.org/html/2607.26071#A1.T6.3.13.10.2)\.
- T\. B\. Brown, B\. Mann, N\. Ryder,et al\.\(2020\)Language models are few\-shot learners\.InProceedings of the 34th International Conference on Neural Information Processing Systems,pp\. 1877–1901\.Cited by:[§1](https://arxiv.org/html/2607.26071#S1.p1.1)\.
- J\. Carbonell and J\. Goldstein \(1998\)The use of mmr, diversity\-based reranking for reordering documents and producing summaries\.InProceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval,pp\. 335–336\.External Links:[Document](https://dx.doi.org/10.1145/290941.291025),[Link](https://doi.org/10.1145/290941.291025)Cited by:[§2](https://arxiv.org/html/2607.26071#S2.p3.1),[§3\.3](https://arxiv.org/html/2607.26071#S3.SS3.p1.1)\.
- J\. Cohen \(1960\)A coefficient of agreement for nominal scales\.Educational and Psychological Measurement20\(1\),pp\. 37–46\.External Links:[Document](https://dx.doi.org/10.1177/001316446002000104),[Link](https://doi.org/10.1177/001316446002000104)Cited by:[§5\.2](https://arxiv.org/html/2607.26071#S5.SS2.p2.1)\.
- Y\. Du, M\. Tian, S\. Ronanki,et al\.\(2025\)Context length alone hurts LLM performance despite perfect retrieval\.InFindings of the Association for Computational Linguistics: EMNLP 2025,pp\. 23281–23298\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.findings-emnlp.1264),[Link](https://aclanthology.org/2025.findings-emnlp.1264/)Cited by:[§2](https://arxiv.org/html/2607.26071#S2.p2.1),[§2](https://arxiv.org/html/2607.26071#S2.p3.1),[§5\.2](https://arxiv.org/html/2607.26071#S5.SS2.p5.6)\.
- D\. Edge, H\. Trinh, N\. Cheng,et al\.\(2025\)From local to global: a graph rag approach to query\-focused summarization\.External Links:[Link](https://arxiv.org/abs/2404.16130)Cited by:[Table 6](https://arxiv.org/html/2607.26071#A1.T6),[Table 6](https://arxiv.org/html/2607.26071#A1.T6.3.5.2.2),[§1](https://arxiv.org/html/2607.26071#S1.p2.1),[§2](https://arxiv.org/html/2607.26071#S2.p2.1),[§3\.4](https://arxiv.org/html/2607.26071#S3.SS4.p1.1),[§4\.1](https://arxiv.org/html/2607.26071#S4.SS1.p1.1),[§4\.1](https://arxiv.org/html/2607.26071#S4.SS1.p2.4),[§4\.2](https://arxiv.org/html/2607.26071#S4.SS2.p1.5),[§5\.1](https://arxiv.org/html/2607.26071#S5.SS1.p1.1),[§5\.2](https://arxiv.org/html/2607.26071#S5.SS2.p3.1),[§6\.3](https://arxiv.org/html/2607.26071#S6.SS3.p4.1)\.
- D\. Garigliotti and K\. Balog \(2017\)On type\-aware entity retrieval\.InProceedings of the ACM SIGIR International Conference on Theory of Information Retrieval,pp\. 27–34\.External Links:[Document](https://dx.doi.org/10.1145/3121050.3121054),[Link](https://doi.org/10.1145/3121050.3121054)Cited by:[§3\.2](https://arxiv.org/html/2607.26071#S3.SS2.p1.3)\.
- Z\. Guo, L\. Xia, Y\. Yu,et al\.\(2025\)LightRAG: simple and fast retrieval\-augmented generation\.InFindings of the Association for Computational Linguistics: EMNLP 2025,pp\. 10746–10761\.External Links:[Link](https://aclanthology.org/2025.findings-emnlp.568/),[Document](https://dx.doi.org/10.18653/v1/2025.findings-emnlp.568)Cited by:[Table 6](https://arxiv.org/html/2607.26071#A1.T6.3.6.3.2)\.
- B\. J\. Gutiérrez, Y\. Shu, Y\. Gu,et al\.\(2024\)HippoRAG: neurobiologically inspired long\-term memory for large language models\.InProceedings of the 38th International Conference on Neural Information Processing Systems,pp\. 59532–59569\.Cited by:[Table 6](https://arxiv.org/html/2607.26071#A1.T6.3.11.8.2)\.
- K\. Guu, K\. Lee, Z\. Tung,et al\.\(2020\)REALM: retrieval\-augmented language model pre\-training\.InProceedings of the 37th International Conference on Machine Learning,pp\. 3929–3938\.Cited by:[Table 6](https://arxiv.org/html/2607.26071#A1.T6.3.9.6.2)\.
- H\. Han, L\. Ma, Y\. Wang,et al\.\(2025\)Rag vs\. graphrag: a systematic evaluation and key insights\.arXiv preprint arXiv:2502\.11371\.Cited by:[§2](https://arxiv.org/html/2607.26071#S2.p2.1),[§4\.1](https://arxiv.org/html/2607.26071#S4.SS1.p2.4),[§5\.2](https://arxiv.org/html/2607.26071#S5.SS2.p3.1)\.
- S\. Hofstätter, J\. Chen, K\. Raman,et al\.\(2023\)FiD\-light: efficient and effective retrieval\-augmented text generation\.InProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval,pp\. 1437–1447\.External Links:[Link](https://doi.org/10.1145/3539618.3591687),[Document](https://dx.doi.org/10.1145/3539618.3591687)Cited by:[Table 6](https://arxiv.org/html/2607.26071#A1.T6.3.15.12.2),[Table 6](https://arxiv.org/html/2607.26071#A1.T6.3.7.4.2)\.
- A\. Hogan, E\. Blomqvist, M\. Cochez,et al\.\(2021\)Knowledge graphs\.ACM Computing Surveys \(Csur\)54\(4\),pp\. 1–37\.Cited by:[§2](https://arxiv.org/html/2607.26071#S2.p2.1),[§3\.1](https://arxiv.org/html/2607.26071#S3.SS1.p1.1)\.
- Y\. Huang and J\. X\. Huang \(2026\)A survey on retrieval\-augmented text generation for large language models\.ACM Comput\. Surv\.58\(12\),pp\. 1–38\.External Links:[Document](https://dx.doi.org/10.1145/3805774),[Link](https://doi.org/10.1145/3805774)Cited by:[Table 6](https://arxiv.org/html/2607.26071#A1.T6),[§1](https://arxiv.org/html/2607.26071#S1.p2.1),[§2](https://arxiv.org/html/2607.26071#S2.p2.1),[§5\.4](https://arxiv.org/html/2607.26071#S5.SS4.p2.1)\.
- G\. Izacard, P\. Lewis, M\. Lomeli,et al\.\(2023\)Atlas: few\-shot learning with retrieval augmented language models\.J\. Mach\. Learn\. Res\.24\(1\),pp\. 11912 – 11954\.Cited by:[Table 6](https://arxiv.org/html/2607.26071#A1.T6.3.14.11.2),[§2](https://arxiv.org/html/2607.26071#S2.p2.1),[§2](https://arxiv.org/html/2607.26071#S2.p3.1),[§3\.3](https://arxiv.org/html/2607.26071#S3.SS3.p1.1),[§5\.4](https://arxiv.org/html/2607.26071#S5.SS4.p2.1)\.
- S\. Jeong, J\. Baek, S\. Cho,et al\.\(2024\)Adaptive\-rag: learning to adapt retrieval\-augmented large language models through question complexity\.External Links:[Link](https://arxiv.org/abs/2403.14403)Cited by:[Table 6](https://arxiv.org/html/2607.26071#A1.T6.3.17.14.2)\.
- Z\. Jiang, F\. Xu, L\. Gao,et al\.\(2023\)Active retrieval augmented generation\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,pp\. 7969–7992\.External Links:[Link](https://aclanthology.org/2023.emnlp-main.495/),[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.495)Cited by:[Table 6](https://arxiv.org/html/2607.26071#A1.T6.3.12.9.2)\.
- H\. Joho and J\. M\. Jose \(2025\)An instruction\-response perspective on large language models in information retrieval tasks\.InProceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval,pp\. 3843–3852\.External Links:[Document](https://dx.doi.org/10.1145/3726302.3730346),[Link](https://doi.org/10.1145/3726302.3730346)Cited by:[§1](https://arxiv.org/html/2607.26071#S1.p1.1)\.
- A\. Kamath, R\. Jia, and P\. Liang \(2020\)Selective question answering under domain shift\.InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics,pp\. 5684–5696\.External Links:[Document](https://dx.doi.org/10.18653/v1/2020.acl-main.503),[Link](https://aclanthology.org/2020.acl-main.503/)Cited by:[§3\.3](https://arxiv.org/html/2607.26071#S3.SS3.p1.1)\.
- V\. Karpukhin, B\. Oguz, S\. Min,et al\.\(2020\)Dense passage retrieval for open\-domain question answering\.InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),pp\. 6769–6781\.External Links:[Link](https://aclanthology.org/2020.emnlp-main.550/),[Document](https://dx.doi.org/10.18653/v1/2020.emnlp-main.550)Cited by:[Table 6](https://arxiv.org/html/2607.26071#A1.T6.3.3.3)\.
- P\. Lewis, E\. Perez, A\. Piktus,et al\.\(2020\)Retrieval\-augmented generation for knowledge\-intensive nlp tasks\.InProceedings of the 34th International Conference on Neural Information Processing Systems,pp\. 9459–9474\.Cited by:[Table 6](https://arxiv.org/html/2607.26071#A1.T6),[Table 6](https://arxiv.org/html/2607.26071#A1.T6.3.4.1.2),[§1](https://arxiv.org/html/2607.26071#S1.p1.1),[§1](https://arxiv.org/html/2607.26071#S1.p2.1),[§2](https://arxiv.org/html/2607.26071#S2.p1.1),[§2](https://arxiv.org/html/2607.26071#S2.p2.1),[§2](https://arxiv.org/html/2607.26071#S2.p3.1),[§4\.1](https://arxiv.org/html/2607.26071#S4.SS1.p1.1),[§4\.2](https://arxiv.org/html/2607.26071#S4.SS2.p1.5)\.
- N\. F\. Liu, K\. Lin, J\. Hewitt,et al\.\(2024\)Lost in the middle: how language models use long contexts\.Transactions of the Association for Computational Linguistics12,pp\. 157–173\.External Links:[Document](https://dx.doi.org/10.1162/tacl%5Fa%5F00638),[Link](https://aclanthology.org/2024.tacl-1.9/)Cited by:[§5\.2](https://arxiv.org/html/2607.26071#S5.SS2.p5.6)\.
- M\. J\. op de Haar, T\. Stähle, and L\. Gatti \(2026\)All relations lead to rome: automated knowledge graph creation and question generation\.External Links:2606\.22645,[Link](https://arxiv.org/abs/2606.22645)Cited by:[§4\.1](https://arxiv.org/html/2607.26071#S4.SS1.p1.1),[§6\.3](https://arxiv.org/html/2607.26071#S6.SS3.p1.1),[§6\.3](https://arxiv.org/html/2607.26071#S6.SS3.p2.1)\.
- A\. Oulasvirta, M\. Wahlström, and K\. Anders Ericsson \(2011\)What does it mean to be good at using a mobile device? an investigation of three levels of experience and skill\.International Journal of Human\-Computer Studies69\(3\),pp\. 155–169\.External Links:[Document](https://dx.doi.org/10.1016/j.ijhcs.2010.11.003),[Link](https://www.sciencedirect.com/science/article/pii/S107158191000145X)Cited by:[§4\.1](https://arxiv.org/html/2607.26071#S4.SS1.p2.4)\.
- V\. Raghavan, P\. Bollmann, and G\. S\. Jung \(1989\)A critical investigation of recall and precision as measures of retrieval system performance\.ACM Trans\. Inf\. Syst\.7\(3\),pp\. 205–229\.External Links:[Document](https://dx.doi.org/10.1145/65943.65945),[Link](https://doi.org/10.1145/65943.65945)Cited by:[§5\.4](https://arxiv.org/html/2607.26071#S5.SS4.p2.1)\.
- A\. Roberts, C\. Raffel, and N\. Shazeer \(2020\)How much knowledge can you pack into the parameters of a language model?\.InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),pp\. 5418–5426\.External Links:[Document](https://dx.doi.org/10.18653/v1/2020.emnlp-main.437),[Link](https://aclanthology.org/2020.emnlp-main.437/)Cited by:[§2](https://arxiv.org/html/2607.26071#S2.p1.1)\.
- G\. Salton, A\. Wong, and C\. S\. Yang \(1975\)A vector space model for automatic indexing\.Commun\. ACM18\(11\),pp\. 613–620\.External Links:[Document](https://dx.doi.org/10.1145/361219.361220),[Link](https://doi.org/10.1145/361219.361220)Cited by:[§2](https://arxiv.org/html/2607.26071#S2.p2.1),[§2](https://arxiv.org/html/2607.26071#S2.p3.1),[§3\.1](https://arxiv.org/html/2607.26071#S3.SS1.p1.1),[§3\.3](https://arxiv.org/html/2607.26071#S3.SS3.p1.1),[§4\.2](https://arxiv.org/html/2607.26071#S4.SS2.p1.5)\.
- M\. S\. Tamber, F\. S\. Bao, C\. Xu,et al\.\(2025\)Benchmarking LLM faithfulness in RAG with evolving leaderboards\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: Industry Track,pp\. 799–811\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.emnlp-industry.54),[Link](https://aclanthology.org/2025.emnlp-industry.54/)Cited by:[§3\.4](https://arxiv.org/html/2607.26071#S3.SS4.p1.1)\.
- H\. Tian, Y\. Feng, Z\. Wen,et al\.\(2026\)RAGExplorer: a visual analytics system for the comparative diagnosis of rag systems\.External Links:[Link](https://arxiv.org/abs/2601.12991)Cited by:[§6\.3](https://arxiv.org/html/2607.26071#S6.SS3.p5.1)\.
- A\. Vaswani, N\. Shazeer, N\. Parmar,et al\.\(2017\)Attention is all you need\.InProceedings of the 31st International Conference on Neural Information Processing Systems,pp\. 6000–6010\.Cited by:[§2](https://arxiv.org/html/2607.26071#S2.p1.1),[§2](https://arxiv.org/html/2607.26071#S2.p3.1)\.
- E\. Volkanovska \(2025\)A study of errors in the output of large language models for domain\-specific few\-shot named entity recognition\.Journal for Language Technology and Computational Linguistics38\(2\),pp\. 31–42\.External Links:[Document](https://dx.doi.org/10.21248/jlcl.38.2025.281),[Link](https://jlcl.org/article/view/281)Cited by:[§1](https://arxiv.org/html/2607.26071#S1.p2.1)\.
- L\. Wang, N\. Yang, and F\. Wei \(2023\)Query2doc: query expansion with large language models\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,pp\. 9414–9423\.External Links:[Link](https://aclanthology.org/2023.emnlp-main.585/),[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.585)Cited by:[Table 6](https://arxiv.org/html/2607.26071#A1.T6.3.16.13.2)\.
- D\. J\. Watts and S\. H\. Strogatz \(1998\)Collective dynamics of ‘small\-world’ networks\.Nature393\(6684\),pp\. 440–442\.External Links:[Document](https://dx.doi.org/10.1038/30918)Cited by:[§5\.3](https://arxiv.org/html/2607.26071#S5.SS3.p1.4)\.
- C\. Zhai \(2024\)Large language models and future of information retrieval: opportunities and challenges\.InProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval,pp\. 481–490\.External Links:[Document](https://dx.doi.org/10.1145/3626772.3657848),[Link](https://doi.org/10.1145/3626772.3657848)Cited by:[§1](https://arxiv.org/html/2607.26071#S1.p1.1)\.
- T\. Zhang, S\. G\. Patil, N\. Jain,et al\.\(2024\)RAFT: adapting language model to domain specific RAG\.InFirst Conference on Language Modeling,External Links:[Link](https://openreview.net/forum?id=rzQGHXNReU)Cited by:[Table 6](https://arxiv.org/html/2607.26071#A1.T6.3.10.7.2)\.
- P\. Zhao, H\. Zhang, Q\. Yu,et al\.\(2024\)Retrieval\-augmented generation for ai\-generated content: A survey\.CoRRabs/2402\.19473,pp\. 1–29\.External Links:[Document](https://dx.doi.org/10.48550/arxiv.2402.19473),[Link](https://doi.org/10.48550/arXiv.2402.19473)Cited by:[§1](https://arxiv.org/html/2607.26071#S1.p2.1)\.
- Y\. Zhu, H\. Yuan, S\. Wang,et al\.\(2025\)Large language models for information retrieval: a survey\.ACM Transactions on Information Systems44\(1\),pp\. 1–54\.External Links:[Document](https://dx.doi.org/10.1145/3748304),[Link](http://dx.doi.org/10.1145/3748304)Cited by:[Table 6](https://arxiv.org/html/2607.26071#A1.T6),[§1](https://arxiv.org/html/2607.26071#S1.p1.1),[§5\.4](https://arxiv.org/html/2607.26071#S5.SS4.p2.1)\.

## Appendix AComparison of RAG approaches

Table 6\.This table presents a mapping of 15 distinct RAG approaches that fit inside theGuidedRAGframework\. Each approach is mapped across theGuidedRAGstages, with each stage corresponding to a method in the given RAG approach\. The mapping covers each approach exhaustively, resulting in full coverage of the corpus\. Therefore,GuidedRAGcan extend any of the listed RAG approaches, and generally extends to any other RAG approach that makes use of the canonicalRetrieval,Augmentation, andGenerationstages in their implementation\. In addition,GuidedRAGsupports any knowledge base in theConfigurationstage, as long as it contains semantics and chunks\. The latter being a prerequisite for any RAG framework\(Lewiset al\.,[2020](https://arxiv.org/html/2607.26071#bib.bib3); Edgeet al\.,[2025](https://arxiv.org/html/2607.26071#bib.bib4); Huang and Huang,[2026](https://arxiv.org/html/2607.26071#bib.bib2); Zhuet al\.,[2025](https://arxiv.org/html/2607.26071#bib.bib21)\)\.- †\\daggerChunks are omitted, as these are always present in the augmentation stage
- ‡\\ddaggerWe include approaches that do not rely on LLMs for generation
- §\\SNo explicit generation model

## Appendix BGuidedRAG WORKFLOW EXAMPLE

![[Uncaptioned image]](https://arxiv.org/html/2607.26071v1/x8.png)

Figure 8\.In this example, we illustrate an instance of usingGuidedRAG\.\(i\)During theConfigurationstage, a user specifies a prompt and a semantic\. In this case, the question concerns theRoman Empire\.\(ii\)Using this semantic, the knowledge base is scoped in theSelectionstage to include only chunks related to theRoman Empire\.\(iii\)This selection space is then used to rank the chunks according to their similarity to the user prompt, using cosine similarity as theRetrievalmethod\.\(iv\)The user prompt, retrieved chunks, and semantics are then combined with a system prompt in theAugmentationstage, resulting in an augmented prompt\.\(v\)Finally, the augmented prompt is used in theGenerationstage and sent to an LLM, which generates a response that is sent to the user\.

## Appendix CGuidedRAG Extended Examples

In this section, we provide some additional examples ofGuidedRAG, namely: theConfiguration,Selection,Retrieval,Augmentation, andGenerationstages contextualized in concrete examples of systems in whichGuidedRAGcan be used; to best illustrate where semantic steering and selections can come into play\.

### C\.1\.Map Example

In this example, we discussGooMaps, afictionalnavigation and map\-based system that contains various locations, restaurants, and stores \(i\.e\., entities\)\. The application can be used to explore these entities and to summarize reviews and blog posts associated with them\.

Configuration:InGooMaps, information is stored about individual locations, restaurants, and stores, including reviews and blog posts\. This information is represented using a vector database, where chunks are associated with specific entities \(e\.g\., a restaurant\) and additional categories \(e\.g\., cuisine\)\. A user interacts withGooMapsby selecting an entity on the map, such as a restaurant\. Upon selection, the system can generate a summary of the restaurant usingGuidedRAG\. Additionally, the user may ask specific questions, such as details about menu items\.

Selection:After a restaurant has been selected, the selection stage defines a selection space over the relevant chunks\. InGooMaps, this consists of chunks associated only with the selected restaurant, such as its reviews, descriptions, and related blog content\. By restricting the selection space to a single entity, the system reduces the search scope while maintaining relevance, without requiring additional knowledge\-base filtering\.

Retrieval:During retrieval, the scoped selection space is used to support both summarization and question answering\. For summarization, maximal marginal relevance is applied to retrieve a diverse set of chunks that capture different aspects of the restaurant \(e\.g\., food quality, service, and atmosphere\)\. For question answering, cosine similarity is used to retrieve the most relevant chunks with respect to the user query, such as a question about vegetarian options or pricing\.

Augmentation:After retrieving the relevant chunks, the augmentation layer combines them with the user query to construct a system prompt\. InGooMaps, this prompt explicitly includes the restaurant name and a base description, ensuring that the generation stage remains grounded in the selected entity\. This additional context helps the model distinguish between similar entities and improves response specificity\.

Generation:Finally, the response is generated using an LLM\. For summarization, the model produces a coherent overview of the restaurant by integrating information from multiple chunks\. For question answering, the model generates a focused response based on the retrieved evidence\. In both cases, restricting the output to entity\-specific chunks ensures the generated output remains accurate and relevant to the selected restaurant\.

### C\.2\.Legal Example

In this example, we discussAceWright, afictionallegal information system that contains various laws, regulations, and legal cases \(i\.e\., entities\)\. The application can be used to explore these entities, summarize legal documents, and answer legal questions\.

Configuration:The configuration stage describes the knowledge base and user\-system interactions\.\(i\)InAceWright, information is stored about legal entities such as statutes, case law, and regulations, including their descriptions, interpretations, and references\. This information is represented using a combination of a knowledge graph and a vector database, where chunks are associated with specific entities \(e\.g\., a law or case\) and relations \(e\.g\.,cites,amends, orapplies to\)\.\(ii\)A user interacts withAceWrightby selecting a legal entity or submitting a legal query, such as a specific law or legal topic\. Upon selection, the system can generate a summary of the selected law usingGuidedRAG\. Additionally, the user may ask targeted questions, such as how a law applies in a specific scenario or which cases reference it\.

Selection:After a legal entity has been selected, the selection stage defines a selection space over the relevant chunks\. InAceWright, this consists of chunks associated with the selected law or case, as well as closely related entities connected through the knowledge graph \(e\.g\., referenced cases or amended statutes\)\. By restricting the selection space to the selected entity and its immediate relations, the system reduces the search scope while preserving legally relevant context\.

Retrieval:During retrieval, the scoped selection space is used to support both summarization and question answering\. For summarization, maximal marginal relevance is applied to retrieve a diverse set of chunks capturing different aspects of the legal entity \(e\.g\., definitions, applications, and interpretations\)\. For question answering, cosine similarity is used to retrieve the most relevant chunks with respect to the user query, such as a question about the applicability of a law in a specific case\.

Augmentation:After retrieving the relevant chunks, the augmentation layer combines them with the user query to construct a system prompt\. InAceWright, this prompt explicitly includes the name of the legal entity \(e\.g\., a statute\) and a base description, ensuring that the generation stage remains grounded in the correct legal context\. Additionally, references to related entities may be included to provide supporting context for interpretation\.

Generation:Finally, the response is generated using an LLM\. For summarization, the model produces a coherent overview of the legal entity by integrating information from multiple chunks, such as its purpose and scope\. For question answering, the model generates a focused response based on the retrieved evidence, such as explaining how a law applies to a given scenario\. In both cases, restricting the output to entity\- and relation\-specific chunks ensures that the generated response remains accurate, grounded, and legally consistent\.

## Appendix DPrompt

This section details the prompt used to mark a given chunk as relevant during the evaluation\. Annotations were averaged over three separate iterations, due to the inherent non\-determinism of large language models, even when the temperature is set to zero\.

Relevance Annotation:You are the best LLM\-based annotator\. You are evaluating whether a chunk retrieved through RAG contains a fact; if it does, then it is deemed relevant\. Return ONLY valid JSON that contains a boolean indicating whether a chunk was relevant \(true\) or not \(false\)\.Make sure that the chunk explicitly contains the fact, as otherwise you should return false\.FACT:\{ fact \}QUESTION:\{ question \}CHUNK:\{ chunk \}\.OUTPUT FORMAT:\{ ”relevant”: boolean \}