GRACE-RAG: Governed Retrieval Architecture for Canonical Evidence Synthesis, Enabling Lightweight Deployment in Closed-Domain Institutional Settings

# Governed Retrieval Architecture for Canonical Evidence Synthesis, Enabling Lightweight Deployment in Closed-Domain Institutional Settings
Source: [https://arxiv.org/html/2607.00013](https://arxiv.org/html/2607.00013)
Aman Kumar National Payments Corporation of India Prashant Devadiga National Payments Corporation of India

###### Abstract

Retrieval\-Augmented Generation \(RAG\) systems are widely used in institutional question answering settings where responses must be grounded in authoritative documentation\(Gaoet al\.,[2023](https://arxiv.org/html/2607.00013#bib.bib1)\)\. In entity\-dense domains where relevant information is distributed across heterogeneous documents, vector\-only retrieval often produces fragmented evidence and increases dependence on inference\-time reasoning\(Zhaoet al\.,[2024](https://arxiv.org/html/2607.00013#bib.bib2)\)\. This paper introducesGRACE\-RAG, a retrieval\-governed, graph\-augmented RAG architecture that externalizes structural reasoning from the generative stage to a structured retrieval layer, resolving structural ambiguity offline, enabling deployment on self\-hosted lightweight models calibrated to closed\-domain institutional vocabulary\. Experiments across three model capacities: Mistral 24B, GPT OSS 120B, and Gemini 2\.5 Flash show consistent improvements in completeness, depth, and anticipatory coverage, with overall quality gains of up to 20% under mid\-scale models, indicating that retrieval architecture governs structural quality over model scale, reducing computational and latency footprint without dependence on proprietary systems\.111Preprint

## 1Introduction

Institutional question answering systems operate under constraints that differ substantially from open\-domain conversational assistants\(Penget al\.,[2024](https://arxiv.org/html/2607.00013#bib.bib4); Lund,[2025](https://arxiv.org/html/2607.00013#bib.bib19)\)\. Queries in such environments frequently reference domain\-specific entities, operational limits, eligibility rules, or conditional workflows whose relevant information is distributed across heterogeneous documents\(Xu and others,[2024](https://arxiv.org/html/2607.00013#bib.bib13)\)\.

Retrieval\-Augmented Generation \(RAG\) grounds language model outputs in external knowledge sources\(Gaoet al\.,[2023](https://arxiv.org/html/2607.00013#bib.bib1)\), but in entity\-dense institutional corpora, semantic proximity alone is insufficient as queries may span multiple documents through implicit relational dependencies\(Lewiset al\.,[2020](https://arxiv.org/html/2607.00013#bib.bib6)\)\.

To address these limitations, many systems introduce prompt\-level orchestration or agent\-based control flow\(Gupta and others,[2024](https://arxiv.org/html/2607.00013#bib.bib12)\), increasing latency and computational cost through reliance on proprietary models that prioritize cross\-domain generalization at the expense of domain\-specific terminology precision\(Arslan,[2024](https://arxiv.org/html/2607.00013#bib.bib7)\)\.

This work adopts an alternative perspective: structural ambiguity should be resolved prior to generation, and the language model should be restricted to synthesizing evidence rather than performing implicit structural reasoning\(Chenget al\.,[2025](https://arxiv.org/html/2607.00013#bib.bib8)\)\. We therefore introduceGRACE\-RAG, a retrieval\-governed RAG architecture that externalizes entity normalization, relationship modeling, and semantic boundary alignment into an offline structure\-manufacturing pipeline\. During online inference, hybrid retrieval operates over dual embedding surfaces, content chunks and relationship summaries, allowing relational hypotheses to be ranked and validated before generation\(Gupta and others,[2024](https://arxiv.org/html/2607.00013#bib.bib12); Wan and others,[2025](https://arxiv.org/html/2607.00013#bib.bib10)\)\.

The contributions of this paper are threefold:

- •We introduceGRACE\-RAG, a retrieval\-governed RAG architecture that decouples structural reasoning from generation through offline knowledge construction and bounded hybrid retrieval\.
- •We present a dual\-surface retrieval mechanism in which relationship summaries are embedded and indexed independently, enabling relational validation and graph\-guided expansion without uncontrolled traversal\.
- •We empirically demonstrate that governed retrieval with canonical evidence structuring delivers consistent structural quality gains, enabling a practical shift to self\-hosted lightweight models with measurably reduced computational cost and operational footprint\.

Domain DocumentsSemantic ChunkingEntity & Relationship ExtractionEntity CanonicalizationGraph Construction\+ Dual EmbeddingsOffline Pipeline: Structure ManufacturingUser QueryQuery Processing\(Typing \+ Decomposition\)Hybrid RetrievalGlobal RerankingLLM Generation\(Evidence Synthesis Only\)Online Pipeline: Retrieval\-Governed InferenceRetrieval Artifacts Boundary

Figure 1:Offline–Online architectural separation\. Structural reasoning is resolved during offline structure manufacturing, while online inference performs bounded hybrid retrieval prior to generation\.
## 2Related Work

### 2\.1Vector\-Based Retrieval\-Augmented Generation

Vector\-based Retrieval\-Augmented Generation \(RAG\) has become a dominant paradigm for grounding language model outputs in external knowledge sources\(Gaoet al\.,[2023](https://arxiv.org/html/2607.00013#bib.bib1)\)\. This approach is effective when relevant information is localized and semantically aligned with user queries\.

However, in entity\-dense institutional corpora, semantic proximity alone may be insufficient to capture relevance\(Xu and others,[2024](https://arxiv.org/html/2607.00013#bib.bib13)\)\. Information necessary to answer a single query can be distributed across multiple documents, linked implicitly through relational dependencies, or expressed using heterogeneous surface forms\. Under these conditions, retrieval often yields fragmented context, increasing reliance on inference\-time reasoning within the language model\(Gupta and others,[2024](https://arxiv.org/html/2607.00013#bib.bib12)\)\.

### 2\.2Graph\-Based Retrieval\-Augmented Generation

Graph\-based RAG systems introduce explicit relational structure through knowledge graphs constructed from extracted entities and relationships\(Penget al\.,[2024](https://arxiv.org/html/2607.00013#bib.bib4)\)\. Graph augmentation can improve multi\-hop reasoning and enhance contextual completeness in domains where relational structure is central\(Mavromatis and others,[2025](https://arxiv.org/html/2607.00013#bib.bib20)\)\.

Despite these advantages, practical implementations often face challenges related to entity fragmentation, noisy extraction, and shallow traversal depth\(Knollmeyer and others,[2025](https://arxiv.org/html/2607.00013#bib.bib16)\)\. When entity surface forms are not normalized, structurally equivalent concepts may be represented as distinct nodes, reducing graph connectivity and recall\. Additionally, graph\-derived context is frequently appended to vector\-retrieved text rather than integrated within a unified ranking framework\(Zhuet al\.,[2025](https://arxiv.org/html/2607.00013#bib.bib5)\)\. As a result, structural signals may remain underutilized during retrieval\-time decision making\.

### 2\.3Hybrid Retrieval Architectures

Hybrid retrieval architectures combine vector similarity search with graph\-based expansion in an effort to balance semantic relevance and relational awareness\(Wan and others,[2025](https://arxiv.org/html/2607.00013#bib.bib10)\)\. In many existing systems, contexts obtained from vector search and graph traversal are merged or concatenated prior to generation\(Linders and others,[2025](https://arxiv.org/html/2607.00013#bib.bib17)\)\. While this approach can increase coverage, integration typically occurs at the context level rather than at the retrieval\-ranking level\. The language model is therefore responsible for reconciling heterogeneous evidence during generation\(Chenget al\.,[2025](https://arxiv.org/html/2607.00013#bib.bib8)\)\.

Moreover, hybrid designs often provide limited mechanisms for entity canonicalization, controlled traversal, or bounded inference complexity\(Zhuet al\.,[2025](https://arxiv.org/html/2607.00013#bib.bib5)\)\. Without explicit structural constraints, improvements in answer quality may depend heavily on prompt design or model capacity rather than retrieval architecture itself\.

### 2\.4Model\-Centric vs Architecture\-Centric Paradigms

Recent advances in large language models have led to a model\-centric view of RAG effectiveness, where performance gains are frequently attributed to increased model scale, extended context windows, or complex prompt orchestration\(Gaoet al\.,[2023](https://arxiv.org/html/2607.00013#bib.bib1)\)\. Multi\-stage routing and agent\-based pipelines further expand inference\-time complexity in pursuit of improved coverage\(Gupta and others,[2024](https://arxiv.org/html/2607.00013#bib.bib12)\)\. While such approaches can yield empirical gains, they also increase latency, operational cost, and dependence on proprietary systems\(Arslan,[2024](https://arxiv.org/html/2607.00013#bib.bib7)\)\.

## 3Methodology

### 3\.1Governing Principles

The GRACE\-RAG architecture is designed around three guiding principles:

1. 1\.Structural ambiguity is resolved offline:Entity equivalence, relational dependencies, and semantic boundaries are externalized into retrieval artifacts constructed prior to inference, rather than implicitly resolved at generation time\(Knollmeyer and others,[2025](https://arxiv.org/html/2607.00013#bib.bib16)\)\.
2. 2\.Retrieval governs and bounds generation:Relevance decisions are determined through structured retrieval with a fixed sequence of operations\(Wan and others,[2025](https://arxiv.org/html/2607.00013#bib.bib10)\), restricting the language model to evidence synthesis over curated context, minimizing latent reasoning and reducing sensitivity to model scale\.
3. 3\.Closed\-domain precision is a retrieval property, not a model property:Institutional terminology fidelity is encoded within retrieval artifacts rather than delegated to generative model capacity, enabling deployment on lightweight self\-hosted models with reduced computational and operational footprint\(Arslan,[2024](https://arxiv.org/html/2607.00013#bib.bib7); Wan and others,[2025](https://arxiv.org/html/2607.00013#bib.bib10)\)\.

### 3\.2Offline Structure Manufacturing

The offline pipeline, shown in Fig\.[1](https://arxiv.org/html/2607.00013#S1.F1), transforms raw institutional documents into structured retrieval artifacts\. All computationally intensive and domain\-specific reasoning is resolved during this stage to minimize online complexity\.

#### 3\.2\.1Corpus Characteristics

The experimental evaluation was conducted on a large institutional document corpus exhibiting high entity density, with frequent references to operational parameters, actors, and conditional constraints\. The corpus configuration is summarized in Table[1](https://arxiv.org/html/2607.00013#S3.T1)\.

Table 1:Experimental corpus characteristics
#### 3\.2\.2LLM\-Assisted Semantic Chunking

Document segmentation is treated as a semantic alignment problem rather than a fixed\-window tokenization task\(Zhaoet al\.,[2024](https://arxiv.org/html/2607.00013#bib.bib2)\)\. An LLM\-assisted preprocessing stage restructures documents into context\-preserving segments aligned with conceptual boundaries such as definitions, constraints, procedural steps, and eligibility conditions\(Zhaoet al\.,[2024](https://arxiv.org/html/2607.00013#bib.bib2)\)\. This reduces semantic fragmentation and improves retrieval precision without inflating context size\(Chenget al\.,[2025](https://arxiv.org/html/2607.00013#bib.bib8)\)\.

#### 3\.2\.3Relationship Extraction and Typing

Each chunk is processed to extract entities and typed relationships using an LLM\-based information extraction stage\(Han and others,[2024](https://arxiv.org/html/2607.00013#bib.bib15)\)\. Relationships capture dependencies such as eligibility constraints, procedural sequencing, transactional limits, and policy applicability\.

Extracted relationships are categorized into coarse semantic buckets \(e\.g\., transactional, compliance\-related, policy\-driven\)\. These categories act as retrieval priors rather than rigid ontological constraints, enabling intent\-aligned filtering during online inference\.

For instance, the sentence“UPI Lite transactions are capped at Rs\.500 per transaction”yields the triple: \(UPI Lite,per\_txn\_limit\_500,Transaction Limit\) under thelimitscategory\.

#### 3\.2\.4Construction of Dual Retrieval Indices

The offline pipeline constructs two independent embedding indices: one over semantically aligned document chunks and another over extracted relationship summaries\(Han and others,[2024](https://arxiv.org/html/2607.00013#bib.bib15)\)\. Each identified relationship is condensed into a single\-sentence semantic representation and embedded separately from its originating chunk\(Wan and others,[2025](https://arxiv.org/html/2607.00013#bib.bib10)\)\. This design enables relational hypotheses to be scored and ranked directly through vector similarity without requiring implicit structural reasoning at generation time\(Linders and others,[2025](https://arxiv.org/html/2607.00013#bib.bib17)\)\.

Chunk EmbeddingsVector SearchRelationship SummariesRelational Similarity SearchGlobal RerankingFigure 2:Dual retrieval surfaces\. Chunk embeddings and relationship\-summary embeddings are indexed independently and unified through global reranking prior to generation\.
#### 3\.2\.5Entity Canonicalization and Graph Densification

Raw entity extraction produces surface\-form variation and structural fragmentation\. To address this, entities are grouped via embedding similarity and validated through LLM\-assisted confirmation to determine canonical equivalence\(Zhuet al\.,[2025](https://arxiv.org/html/2607.00013#bib.bib5)\)\.

For example, security credential variants such as “UPI PIN”, “PIN”, “UPI Security PIN”, and “4\-digit PIN” are consolidated into a single canonical entity representation \(“UPI PIN”\)\. Similarly, organizational references such as “Board of Directors”, “Director’s Meeting”, and “Board Decision” are normalized to a unified structural entity \(“Board”\)\. These transformations reduce redundant nodes while preserving relational semantics\. Fig\.[3](https://arxiv.org/html/2607.00013#S3.F3)illustrates this structural transformation, contrasting the fragmented entity space with the canonicalized graph\.

This procedure reduces entity fragmentation and increases average node degree, improving relational recall under bounded traversal\. The objective is not merely normalization but topological densification of the knowledge graph\.

Table[2](https://arxiv.org/html/2607.00013#S3.T2)summarizes the qualitative impact of this process\.

Table 2:Entity canonicalization impactUPI PINPINUPI LiteSecurityMeasure4\-digitAuth\.TxnLimitFragmented Entity SpaceUPI PIN\(PIN\)UPI LiteSecurityMeasure4\-digitAuth\.TxnLimitCanonicalized Graph

Figure 3:Entity fragmentation versus canonical consolidation\. Canonicalization merges semantically equivalent entities, increasing node degree and structural connectivity\.
#### 3\.2\.6Graph Community Detection and Structural Signals

Beyond entity canonicalization, structural properties of the knowledge graph are analyzed using community detection algorithms, including Louvain and Leiden modularity optimization\(Penget al\.,[2024](https://arxiv.org/html/2607.00013#bib.bib4)\)\. These algorithms identify densely connected subgraphs representing semantically cohesive clusters within the institutional domain\.

For instance, in the evaluated corpus, the canonical entity*UPI PIN*was assigned to a high\-density community comprising approximately 120 related entities \(e\.g\., authentication workflows, transaction limits, security controls\)\. The node exhibited a degree of approximately 18, compared to a post\-deduplication graph average of∼\\sim6\.0\. Such centrality indicates that the entity participates in multiple relational contexts across documents\(Zhuet al\.,[2025](https://arxiv.org/html/2607.00013#bib.bib5); Mavromatis and others,[2025](https://arxiv.org/html/2607.00013#bib.bib20)\)\. Community membership and degree centrality therefore act as structural signals during retrieval, enabling prioritization of topologically influential nodes when forming relational hypotheses\.

### 3\.3Online Retrieval\-Governed Inference

#### 3\.3\.1Query Typing and Intent Conditioning

The online pipeline \(Fig\.[1](https://arxiv.org/html/2607.00013#S1.F1)\) processes incoming queries exclusively over artifacts generated offline\. Incoming queries are first classified into a small set of intent categories \(e\.g\., factual, procedural, constraint\-oriented\)\. This classification does not alter control flow but conditions retrieval priorities, enabling intent\-aligned filtering of relational signals during hybrid retrieval\.

#### 3\.3\.2Subquery Decomposition and Parallel Processing

Complex queries are decomposed into atomic subqueries to improve retrieval precision\. Entity extraction and canonical mapping are applied to each subquery independently\. Subqueries are processed in parallel, ensuring bounded latency and avoiding iterative agent\-style loops\.

#### 3\.3\.3Hybrid Retrieval Per Subquery

For each subquery, retrieval proceeds through two coordinated channels: semantic similarity search over chunk embeddings and graph\-guided expansion over canonical entities\(Han and others,[2024](https://arxiv.org/html/2607.00013#bib.bib15)\)\. Graph traversal produces candidate relational hypotheses, which are validated using relationship\-summary embeddings prior to chunk selection\(Wan and others,[2025](https://arxiv.org/html/2607.00013#bib.bib10)\)\. Retrieved evidence from both channels is merged and globally reranked before generation\. Figure[4](https://arxiv.org/html/2607.00013#S3.F4)illustrates this bounded hybrid retrieval process\.

User QueryQuery ProcessingVector SearchGraph TraversalEvidence MergeGlobal RerankingLLM Generation

Figure 4:Per\-subquery hybrid retrieval\. Vector similarity search and graph\-guided expansion operate in parallel before evidence merging and global reranking\.
#### 3\.3\.4Bounded Inference Complexity

The online pipeline executes a fixed sequence of retrieval and ranking operations per query\(Gupta and others,[2024](https://arxiv.org/html/2607.00013#bib.bib12)\)\. No iterative reasoning loops or adaptive routing strategies are employed\. By constraining inference complexity, the architecture ensures predictable latency and reduces dependence on model\-scale reasoning\.

## 4Model capacity independence

Model capacity independence refers to the extent to which response quality remains stable across variations in parameter scale when retrieval structure is controlled\. The GRACE\-RAG architecture externalizes structural reasoning prior to generation, ensuring the generator operates over curated and structurally coherent evidence rather than heterogeneous, weakly aligned context\.

To evaluate whether this restructuring reduces sensitivity to parameter scale, we analyze performance across ten evaluation dimensions grouped into three categories\. Concise definitions of all ten metrics are provided in Appendix[B](https://arxiv.org/html/2607.00013#A2)\.

Structural Metrics:Correctness, Completeness, Depth, Anticipatory Coverage, Search Engine Quality, and Structure\.

Presentation Metrics:Clarity and Formatting\.

Hybrid Metrics:Relevance and Informativeness\.

Based on this grouping, two hypotheses are formulated:

- •H1 \(Architecture Dominance\):Architectural restructuring produces larger improvements than parameter scaling for structurally governed metrics\.
- •H2 \(Scale Dominance in Presentation\):Parameter scaling produces larger improvements than architectural restructuring for presentation\-oriented metrics\.

Under this framework, model capacity independence does not imply that scale is irrelevant\. Rather, it asserts that once structural ambiguity is resolved at retrieval time, improvements in relational completeness and dependency integration become less sensitive to parameter magnitude\.

## 5Evaluation

Table 3:Comparative performance across architectures and model capacities### 5\.1Experimental Setup

Three model capacities were evaluated: Mistral 24B222Mistral\-Small\-3\.1\-24B\-Instruct\-2503, GPT OSS 120B333gpt\-oss\-120b: an open\-weight Mixture\-of\-Experts model with 120B total parameters and approximately 5B active parameters per forward pass, released by OpenAI, and Gemini 2\.5 Flash, each under two retrieval regimes: aBaseline Architectureusing vector\-based retrieval with standard chunk embeddings, and theGRACE\-RAG Architectureusing a retrieval\-governed hybrid pipeline with entity canonicalization, graph traversal, and dual embedding surfaces\. This results in six total evaluation configurations under controlled conditions\(Pipitone and Houir Alami,[2024](https://arxiv.org/html/2607.00013#bib.bib11)\)\.

### 5\.2Evaluation Criteria

Response quality was assessed using the ten\-parameter framework introduced in Section[4](https://arxiv.org/html/2607.00013#S4)and summarized in Appendix[B](https://arxiv.org/html/2607.00013#A2), employing an LLM\-as\-a\-judge evaluation protocol\(Gaoet al\.,[2023](https://arxiv.org/html/2607.00013#bib.bib1)\)\. Each response was evaluated across all ten dimensions on a standardized 0–10 scale\.

The evaluation protocol focuses on identifying two distinct sources of performance variation: improvements attributable to retrieval architecture and improvements attributable to model capacity\(Mansurova and others,[2025](https://arxiv.org/html/2607.00013#bib.bib18)\)\. By maintaining identical prompts, document collections, and preprocessing pipelines across all configurations, the evaluation isolates the effect of architectural restructuring from parameter scaling\. LLM\-as\-a\-judge evaluation is adopted as the primary protocol given the absence of ground\-truth answer annotations in closed\-domain institutional settings, consistent with established practice in open\-ended RAG evaluation\(Zhenget al\.,[2024](https://arxiv.org/html/2607.00013#bib.bib21)\)\. Judge scores were spot\-checked against human assessments on a representative sample to validate scoring consistency, as detailed in the Reproducibility Statement\.

### 5\.3Architecture Effect \(Vertical Comparison\)

Under Mistral 24B, structurally governed metrics exhibit substantial gains\. Completeness improves from 6\.88 to 8\.66, representing a25\.9%increase\. Depth increases from 5\.96 to 8\.07 \(\+35\.4%\), while anticipatory coverage rises from 5\.11 to 7\.77 \(\+52\.1%\)\. The overall average score improves from 6\.99 to 8\.39, corresponding to a20\.0%increase\.

Similar patterns are observed for GPT OSS 120B, where the overall score increases by18\.7%\(7\.15 to 8\.49\)\. Even under Gemini 2\.5 Flash, which already exhibits strong baseline performance, the GRACE\-RAG architecture produces a 6\.0% increase in overall score \(8\.40 to 8\.90\)\.

These results indicate that architectural restructuring yields consistent structural gains across model capacities\.

### 5\.4Model Scaling Effect \(Horizontal Comparison\)

Within each architectural regime in Table[3](https://arxiv.org/html/2607.00013#S5.T3), scaling from Mistral 24B to GPT OSS and Gemini primarily improves presentation\-oriented metrics\. For example, clarity and formatting demonstrate the most pronounced gains under larger models, while structural dimensions such as completeness and depth exhibit comparatively smaller incremental improvements once retrieval structure is fixed\(Renet al\.,[2023](https://arxiv.org/html/2607.00013#bib.bib3)\)\.

This pattern suggests that parameter scaling predominantly enhances linguistic refinement, whereas relational completeness and constraint integration are more strongly influenced by retrieval organization\(Gaoet al\.,[2023](https://arxiv.org/html/2607.00013#bib.bib1); Xu and others,[2024](https://arxiv.org/html/2607.00013#bib.bib13)\)\.

### 5\.5Interaction Effect

Cross\-architecture comparison reveals a compression of performance gaps across model capacities\. Notably, the GRACE\-RAG architecture operating with Mistral 24B \(overall score 8\.39\) approaches the baseline performance of Gemini 2\.5 Flash \(8\.40\)\. In structurally governed metrics such as depth and completeness, the proposed mid\-scale configuration matches or exceeds larger baseline models\.

This interaction effect indicates that disciplined retrieval design reduces sensitivity to parameter magnitude in entity\-dense institutional settings\. Fig\.[5](https://arxiv.org/html/2607.00013#S6.F5)illustrates the compression of performance gaps across model sizes under the two retrieval architectures\.

## 6Discussion

### 6\.1Architecture as a Structural Multiplier

The empirical results indicate that retrieval structuring functions as a structural multiplier rather than a marginal optimization\. By resolving entity equivalence, encoding typed relationships, and integrating relational validation prior to generation, the GRACE\-RAG architecture increases the density and coherence of evidence presented to the language model\. This suggests that relational signal quality, rather than parameter magnitude alone, governs performance in entity\-dense institutional domains\(Xu and others,[2024](https://arxiv.org/html/2607.00013#bib.bib13)\)\. A qualitative response comparison illustrating these structural differences is provided in Appendix[A](https://arxiv.org/html/2607.00013#A1)\.

Model Size \(B params\)Response Quality5678924Mistral120GPT OSS⋯\\cdotsGemini Flash6\.997\.158\.408\.398\.498\.90Baseline\(dashed\)GRACE\-RAG\(solid\)Figure 5:Scaling behavior of response quality across model sizes\. Architectural restructuring shifts the quality curve upward and reduces sensitivity to parameter magnitude\.
### 6\.2Parameter Scale versus Retrieval Discipline

The comparative analysis separates two largely orthogonal effects\. Parameter scaling primarily enhances linguistic fluency and surface\-level organization, as reflected in clarity and formatting metrics\(Gaoet al\.,[2023](https://arxiv.org/html/2607.00013#bib.bib1)\)\. Retrieval discipline, in contrast, governs relational integration, constraint coverage, and evidence alignment\(Chenget al\.,[2025](https://arxiv.org/html/2607.00013#bib.bib8)\)\. Once structural ambiguity is reduced at retrieval time, incremental gains from additional parameters in structurally governed dimensions diminish\.

### 6\.3Performance Gap Compression Across Model Capacities

The GRACE\-RAG architecture operating with a mid\-scale model approaches or exceeds the structural performance of larger models under baseline retrieval\. This compression of performance gaps demonstrates that disciplined retrieval reduces sensitivity to parameter count in structurally governed tasks\. Consequently, improvements traditionally attributed to scale may instead reflect unresolved structural ambiguity in weaker retrieval pipelines\(Huanget al\.,[2025](https://arxiv.org/html/2607.00013#bib.bib9); Gupta and others,[2024](https://arxiv.org/html/2607.00013#bib.bib12)\)\. As shown in Fig\.[5](https://arxiv.org/html/2607.00013#S6.F5), the GRACE\-RAG quality curve shifts upward uniformly while inter\-model variance narrows\.

### 6\.4Implications for Retrieval\-Augmented System Design

These findings support an architecture\-centric perspective on RAG system development\. In practical terms, this reduces dependence on frontier\-scale proprietary systems for structurally coherent responses\(Wan and others,[2025](https://arxiv.org/html/2607.00013#bib.bib10)\)\. Architectural refinement therefore provides a controllable and reproducible mechanism for improving relational quality without proportionally increasing model capacity\(Chenget al\.,[2025](https://arxiv.org/html/2607.00013#bib.bib8)\)\.

## 7Conclusion

This work introducedGRACE\-RAG, a retrieval\-governed, graph\-augmented RAG architecture designed to externalize structural reasoning from the generative stage to the retrieval layer\. Through offline structural reasoning and bounded hybrid retrieval, GRACE\-RAG restructures how evidence is aggregated prior to generation\.

Empirical evaluation across three model capacities: Mistral 24B, GPT OSS 120B \(MoE\), and Gemini 2\.5 Flash, demonstrates consistent improvements in structurally governed response metrics\. As demonstrated in Table[3](https://arxiv.org/html/2607.00013#S5.T3), completeness, depth, and anticipatory coverage increase substantially under the GRACE\-RAG pipeline, with overall response quality improving by up to 20% under mid\-scale models\. These improvements persist across model capacities, indicating that relational coherence and constraint coverage are primarily determined by retrieval organization rather than by parameter scale alone\(Renet al\.,[2023](https://arxiv.org/html/2607.00013#bib.bib3)\)\. In knowledge\-intensive domains, retrieval discipline enables self\-hosted lightweight models to match or exceed proprietary systems on structural quality, preserving domain jargon precision while reducing computational and latency footprint\(Gupta and others,[2024](https://arxiv.org/html/2607.00013#bib.bib12)\)\.

## Ethics Statement

All experiments were conducted on anonymized, internally curated document corpora\. No personally identifiable information \(PII\) was used at any stage of this work\. No confidential or proprietary information is disclosed through the results, examples, or findings presented in this paper\.

## References

- M\. Arslan \(2024\)A survey on RAG with LLMs\.Procedia Computer Science204,pp\. 157–168\.Cited by:[§1](https://arxiv.org/html/2607.00013#S1.p3.1),[§2\.4](https://arxiv.org/html/2607.00013#S2.SS4.p1.1),[item 3](https://arxiv.org/html/2607.00013#S3.I1.i3.p1.1)\.
- M\. Cheng, Y\. Luo, J\. Ouyang, Q\. Liu, H\. Liu, L\. Li, S\. Yu,et al\.\(2025\)A survey on knowledge\-oriented retrieval\-augmented generation\.arXiv preprint arXiv:2503\.10677\.Cited by:[§1](https://arxiv.org/html/2607.00013#S1.p4.1),[§2\.3](https://arxiv.org/html/2607.00013#S2.SS3.p1.1),[§3\.2\.2](https://arxiv.org/html/2607.00013#S3.SS2.SSS2.p1.1),[§6\.2](https://arxiv.org/html/2607.00013#S6.SS2.p1.1),[§6\.4](https://arxiv.org/html/2607.00013#S6.SS4.p1.1)\.
- Y\. Gao, Y\. Xiong, X\. Gao, K\. Jia, J\. Pan, Y\. Bi, Y\. Dai, J\. Sun, M\. Wang,et al\.\(2023\)Retrieval\-augmented generation for large language models: a survey\.arXiv preprint arXiv:2312\.10997\.Cited by:[§1](https://arxiv.org/html/2607.00013#S1.p2.1),[§2\.1](https://arxiv.org/html/2607.00013#S2.SS1.p1.1),[§2\.4](https://arxiv.org/html/2607.00013#S2.SS4.p1.1),[§5\.2](https://arxiv.org/html/2607.00013#S5.SS2.p1.1),[§5\.4](https://arxiv.org/html/2607.00013#S5.SS4.p2.1),[§6\.2](https://arxiv.org/html/2607.00013#S6.SS2.p1.1)\.
- S\. Guptaet al\.\(2024\)A comprehensive survey of retrieval\-augmented generation: evolution, current landscape and future directions\.arXiv preprint arXiv:2410\.12837\.Cited by:[§1](https://arxiv.org/html/2607.00013#S1.p3.1),[§1](https://arxiv.org/html/2607.00013#S1.p4.1),[§2\.1](https://arxiv.org/html/2607.00013#S2.SS1.p2.1),[§2\.4](https://arxiv.org/html/2607.00013#S2.SS4.p1.1),[§3\.3\.4](https://arxiv.org/html/2607.00013#S3.SS3.SSS4.p1.1),[§6\.3](https://arxiv.org/html/2607.00013#S6.SS3.p1.1),[§7](https://arxiv.org/html/2607.00013#S7.p2.1)\.
- Y\. Hanet al\.\(2024\)GraphRAG: retrieval\-augmented generation with graph\-structured knowledge\.arXiv preprint arXiv:2501\.00309\.Cited by:[§3\.2\.3](https://arxiv.org/html/2607.00013#S3.SS2.SSS3.p1.1),[§3\.2\.4](https://arxiv.org/html/2607.00013#S3.SS2.SSS4.p1.1),[§3\.3\.3](https://arxiv.org/html/2607.00013#S3.SS3.SSS3.p1.1)\.
- L\. Huang, W\. Yu,et al\.\(2025\)A survey on hallucination in large language models: principles, taxonomy, challenges, and open questions\.ACM Transactions on Information Systems43\(2\),pp\. 1–38\.Cited by:[§6\.3](https://arxiv.org/html/2607.00013#S6.SS3.p1.1)\.
- S\. Knollmeyeret al\.\(2025\)Document GraphRAG: knowledge graph enhanced retrieval\-augmented generation\.Electronics14\.Cited by:[§2\.2](https://arxiv.org/html/2607.00013#S2.SS2.p2.1),[item 1](https://arxiv.org/html/2607.00013#S3.I1.i1.p1.1)\.
- P\. Lewis, B\. Oguz, R\. Rinott, S\. Edunov, T\. Kocisky, and L\. Zettlemoyer \(2020\)Retrieval\-augmented generation for knowledge\-intensive NLP tasks\.InAdvances in Neural Information Processing Systems,Vol\.33,pp\. 9459–9474\.Cited by:[§1](https://arxiv.org/html/2607.00013#S1.p2.1)\.
- J\. Linderset al\.\(2025\)Knowledge graph\-extended retrieval\-augmented generation with chain\-of\-thought\.Applied Intelligence\.Cited by:[§2\.3](https://arxiv.org/html/2607.00013#S2.SS3.p1.1),[§3\.2\.4](https://arxiv.org/html/2607.00013#S3.SS2.SSS4.p1.1)\.
- B\. Lund \(2025\)Prospects and challenges of retrieval\-augmented generation in institutional contexts\.arXiv preprint arXiv:2501\.06796\.Cited by:[§1](https://arxiv.org/html/2607.00013#S1.p1.1)\.
- A\. Mansurovaet al\.\(2025\)A systematic evaluation of large language models with retrieval\-augmented generation for question answering\.Information16\.Cited by:[§5\.2](https://arxiv.org/html/2607.00013#S5.SS2.p2.1)\.
- C\. Mavromatiset al\.\(2025\)GNN\-RAG: graph neural retrieval for large language models\.InFindings of the Association for Computational Linguistics: ACL 2025,Cited by:[§2\.2](https://arxiv.org/html/2607.00013#S2.SS2.p1.1),[§3\.2\.6](https://arxiv.org/html/2607.00013#S3.SS2.SSS6.p2.1)\.
- B\. Peng, Y\. Zhu, Y\. Liu, X\. Bo, H\. Shi, C\. Hong, Y\. Zhang, and S\. Tang \(2024\)Graph retrieval\-augmented generation: a survey\.arXiv preprint arXiv:2408\.08921\.Cited by:[§1](https://arxiv.org/html/2607.00013#S1.p1.1),[§2\.2](https://arxiv.org/html/2607.00013#S2.SS2.p1.1),[§3\.2\.6](https://arxiv.org/html/2607.00013#S3.SS2.SSS6.p1.1)\.
- N\. Pipitone and G\. Houir Alami \(2024\)LegalBench\-RAG: a benchmark for retrieval\-augmented generation in the legal domain\.arXiv preprint arXiv:2408\.10343\.Cited by:[§5\.1](https://arxiv.org/html/2607.00013#S5.SS1.p1.1)\.
- R\. Ren, Y\. Wang, Y\. Qu,et al\.\(2023\)Investigating the factual knowledge boundary of large language models with retrieval augmentation\.arXiv preprint arXiv:2307\.11019\.Cited by:[§5\.4](https://arxiv.org/html/2607.00013#S5.SS4.p1.1),[§7](https://arxiv.org/html/2607.00013#S7.p2.1)\.
- Y\. Wanet al\.\(2025\)Empowering LLMs by hybrid retrieval\-augmented generation\.Advanced Engineering Informatics\.Cited by:[§1](https://arxiv.org/html/2607.00013#S1.p4.1),[§2\.3](https://arxiv.org/html/2607.00013#S2.SS3.p1.1),[item 2](https://arxiv.org/html/2607.00013#S3.I1.i2.p1.1),[item 3](https://arxiv.org/html/2607.00013#S3.I1.i3.p1.1),[§3\.2\.4](https://arxiv.org/html/2607.00013#S3.SS2.SSS4.p1.1),[§3\.3\.3](https://arxiv.org/html/2607.00013#S3.SS3.SSS3.p1.1),[§6\.4](https://arxiv.org/html/2607.00013#S6.SS4.p1.1)\.
- Z\. Xuet al\.\(2024\)Retrieval\-augmented generation with knowledge graphs for customer service QA\.InProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval,Cited by:[§1](https://arxiv.org/html/2607.00013#S1.p1.1),[§2\.1](https://arxiv.org/html/2607.00013#S2.SS1.p2.1),[§5\.4](https://arxiv.org/html/2607.00013#S5.SS4.p2.1),[§6\.1](https://arxiv.org/html/2607.00013#S6.SS1.p1.1)\.
- P\. Zhao, H\. Zhang, Q\. Yu, Z\. Wang, Y\. Geng, F\. Fu, and B\. Cui \(2024\)Retrieval\-augmented generation for AI\-generated content: a survey\.arXiv preprint arXiv:2402\.19473\.Cited by:[§3\.2\.2](https://arxiv.org/html/2607.00013#S3.SS2.SSS2.p1.1)\.
- L\. Zheng, W\. Chiang, Y\. Sheng,et al\.\(2024\)Judging LLM\-as\-a\-judge with MT\-bench and chatbot arena\.InAdvances in Neural Information Processing Systems,Vol\.36\.Cited by:[§5\.2](https://arxiv.org/html/2607.00013#S5.SS2.p2.1)\.
- Z\. Zhu, T\. Huang, K\. Wang, J\. Ye, X\. Chen, and S\. Luo \(2025\)Graph\-based approaches and functionalities in retrieval\-augmented generation: a comprehensive survey\.arXiv preprint arXiv:2504\.10499\.Cited by:[§2\.2](https://arxiv.org/html/2607.00013#S2.SS2.p2.1),[§2\.3](https://arxiv.org/html/2607.00013#S2.SS3.p2.1),[§3\.2\.5](https://arxiv.org/html/2607.00013#S3.SS2.SSS5.p1.1),[§3\.2\.6](https://arxiv.org/html/2607.00013#S3.SS2.SSS6.p2.1)\.

## Reproducibility Statement

The offline and online pipeline components of GRACE\-RAG are described in sufficient procedural detail in Sections 3\.2 and 3\.3 to permit independent reimplementation\. Exact model identifiers are provided as footnotes in Section 5\.1\. Embedding models, reranking strategy, graph community detection libraries, evaluation protocol, and judge\-consistency checks are documented in Appendix[C](https://arxiv.org/html/2607.00013#A3)\. The experimental corpus comprises approximately 300 institutional documents and will be released upon acceptance\. Partial code release covering the offline pipeline is planned upon acceptance\.

## Appendix AIllustrative response comparison

The following examples illustrate qualitative differences between baseline retrieval and the GRACE\-RAG architecture across query types\.

##### Example 1

User Query:Hello, I am considering linking my RuPay Credit Card to UPI\. What are the advantages?

##### Baseline Response \(Vector\-Based Retrieval\)

> Linking your RuPay Credit Card to UPI offers several advantages: 1. 1\.Seamless Payments:Make credit\-based merchant payments using UPI QR codes without needing a physical card\. 2. 2\.Secure Transactions:Transactions are authenticated via UPI PIN, ensuring security\. 3. 3\.AutoPay Setup:You can set up AutoPay for bill payments, making recurring payments convenient\. 4. 4\.Dispute Resolution:Access dispute resolution via UPIHelp for any issues that arise\. 5. 5\.No Physical Card Needed:Perform transactions directly through your UPI app \(like BHIM, PhonePe, Google Pay, or Paytm\)\.

##### GRACE\-RAG Response

> Linking a RuPay Credit Card to UPI allows customers to make secure and seamless credit\-based merchant payments using UPI QR codes, without needing a physical card\. This feature enables users to link their RuPay Credit Card to UPI apps \(like BHIM, PhonePe, Google Pay, etc\.\) using their registered mobile number\. Once linked, users can make payments by scanning merchant QR codes and authenticating with a UPI PIN\. The primary advantage is the convenience of making contactless payments at UPI\-enabled merchants\. Users can also set up AutoPay for recurring bill payments, ensuring timely payments without manual intervention\. Notable benefits: - •Seamless linking via UPI app using the registered mobile number - •QR\-based merchant payments authenticated via UPI PIN - •AutoPay setup for recurring bill payments - •Access to dispute resolution via UPIHelp

## Appendix BEvaluation metric definitions

The ten evaluation parameters used throughout the study are defined as follows:

Table 4:Evaluation metric definitions
## Appendix CReproducibility Details

Models and Infrastructure Experiments were conducted using Mistral\-Small\-3\.1\-24B\-Instruct\-2503, gpt\-oss\-120b, and Gemini 2\.5 Flash, with exact model identifiers provided as footnotes in Section 5\.1\. Chunk and relationship summary indices were constructed usingsentence\-transformersembeddings\. Global reranking was performed using a cross\-encoder reranker\. Graph community detection was implemented using the Louvain and Leiden algorithms viaNetworkX\.

Chunking and Retrieval Hyperparameters

Table 5:Retrieval and generation hyperparameters used across all experiments\.Evaluation Protocol Response quality was assessed using an LLM\-as\-a\-judge protocol with Mistral\-Small\-3\.1\-24B\-Instruct\-2503 as the judge model\. All ten evaluation metrics were scored simultaneously within a single structured prompt on a standardized 0–10 scale\. The evaluation set comprised more than 200 queries spanning factual, procedural, and constraint\-oriented intent categories\. Judge scores were spot\-checked against human assessments on a representative 30\-query sample; mean absolute deviation between judge and human scores was within 0\.4 points across all metrics, validating scoring consistency\.

Corpus and Code The experimental corpus comprises approximately 300 institutional documents and will be fully released upon acceptance\. Partial code release covering the offline structure manufacturing pipeline and dual\-index construction is planned upon acceptance\. Components dependent on proprietary infrastructure will be documented with sufficient specification to permit reimplementation\.