EvidenT: Building Trustworthy Enterprise Assistants through Evidence Groundedness and Traceability

# EvidenT: Building Trustworthy Enterprise Assistants through Evidence Groundedness and Traceability
Source: [https://arxiv.org/html/2609.22537](https://arxiv.org/html/2609.22537)
Katie Jooyoung KimColin Zhiwei KouHelene SajerYimei FanRadomir CisarHeather GreenhalghGabriel Martinez VidiriBloombergAffiliation:\{akabra16, ckou9, yfan258, rcisar, hgreenhalgh1, gmartinezvi1\}@bloomberg\.netAffiliation:jk4534@columbia\.edu helene\.sajer\.pro@gmail\.com

###### Abstract

Enterprise AI assistants must produce responses that are verifiable and traceable to source evidence\. However, retrieval\-augmented generation \(RAG\) over heterogeneous enterprise data can suffer from citation drift, unsupported content, and weak source traceability\. We presentEvidenT\(T= Trust \+ Transparency \+ Traceability\), a lightweight pipeline that verifies extracted evidence against retrieved documents before answer generation, without model retraining\. EvidenT combines structured passage extraction with deterministic lexical alignment to filter unsupported content, correct citation drift, and preserve source\-span traceability\. On approximately 500 real enterprise queries,EvidenTimproves gold\-source hit rate by an average of 29% over prompting baselines, produces no citations to non\-retrievedurls, and achieves near\-saturated answer\-to\-source lexical coverage\.

## 1Introduction

Recent advances in question answering \(QA\) with large language models \(LLMs\) have demonstrated impressive capabilities\. Retrieval\-augmented generation \(RAG\) pipelines, which pair LLMs with document retrievers to generate responses with inline citations, have achieved strong results on web\-scale benchmarks\([Novikova et al\., 2017](https://arxiv.org/html/2609.22537#bib.bib4);[Laban et al\., 2022](https://arxiv.org/html/2609.22537#bib.bib3);[Kryscinski et al\., 2020](https://arxiv.org/html/2609.22537#bib.bib5)\)\. However, applying these techniques to enterprise data presents additional challenges, particularly when source documents are heterogeneous, noisy, and inconsistently structured\. In enterprise domains such as finance, healthcare, and law, factual precision and transparent source verification are particularly important[Chen et al\. \(2024b\)](https://arxiv.org/html/2609.22537#bib.bib30)\. Additionally, enterprise content is often fragmented across long, noisy, and inconsistently structured documents, making it harder for systems to locate and present the most relevant information accurately[Umar et al\. \(1999\)](https://arxiv.org/html/2609.22537#bib.bib28)\. In our pilot deployment, we observed responses containing claims that were not well aligned with cited source content, as well as citations that were difficult to verify\([Joren et al\., 2025](https://arxiv.org/html/2609.22537#bib.bib16);[Choubey et al\., 2025](https://arxiv.org/html/2609.22537#bib.bib17);[Packowski et al\., 2024](https://arxiv.org/html/2609.22537#bib.bib18)\)\.

Table 1:Key limitations of the pilot enterprise assistant deployment\.To better understand these challenges, we conducted a month\-long pilot deployment of an enterprise assistant built using a standard RAG architecture \(see Appendix[A](https://arxiv.org/html/2609.22537#A1)for details\)\. The assistant was deployed to 55 users spanning roles such as operations, legal, and support, and processed approximately 4,000 real\-world queries\. The resulting dataset was used to evaluate our proposed approach\. Prompts used for this pilot are available in Appendix[B](https://arxiv.org/html/2609.22537#A2)\.

### 1\.1Limitations of Pilot Enterprise Assistant Deployment

Groundednessrefers to whether generated claims aresupportedby source documents\.Traceabilityrefers to the ability to associate generated content withlocalized supporting spansin the source material\. The pilot exposed three recurring citation and traceability failure modes in this setting \(Table[1](https://arxiv.org/html/2609.22537#S1.T1)\)\.

Invalid Citation Links:The assistant generated plausible\-lookingurlsthat were not present in the retrieved enterprise sources\.

Citation Drift:A valid document was cited, but the cited document did not contain the source text corresponding to the generated content\.

Limited Traceability:Even when the cited document was appropriate, locating the relevant supporting text within long or unstructured documents could be difficult\.

These challenges led us to investigate the following research questions\.

RQ1:Can we improve citation grounding and prevent invalid citationurlswithout model retraining?

RQ2:Can generated answers be reliably traced to localized source spans in noisy and heterogeneous enterprise documents?

We propose a lightweight pipeline designed to improve citation reliability and source traceability in extractive enterprise question answering\. The approach operates without model retraining and can be applied on top of existing retrieval and generation components\. The pipeline extracts candidate evidence, verifies it against retrieved source documents using lexical alignment, and generates answers from the resulting verified passages with associated source citations\. This design prioritizes verifiability and source traceability under the operational constraints of enterprise deployment\. On an evaluation set of approximately 500 queries derived from the pilot,EvidenTimproves gold\-source hit rate by an average of 29% over prompting baselines, produces no citations to non\-retrievedurls, and achieves near\-saturated lexical answer\-to\-source coverage\.

## 2Previous Work

### 2\.1Fine\-tuning and Domain Adaptation of Generative Models

Fine\-tuning large language models has been extensively explored to enhance grounding and factual accuracy in retrieval augmented generation \(RAG\) pipelines[Huang et al\. \(2024\)](https://arxiv.org/html/2609.22537#bib.bib15);[Penzkofer and Baumann \(2024\)](https://arxiv.org/html/2609.22537#bib.bib20);[Zhang et al\. \(2024\)](https://arxiv.org/html/2609.22537#bib.bib21)\. However, fine\-tuning requires large quantities of domain\-specific labeled or synthetic data, which are often unavailable in enterprise contexts\. Although fine\-tuned models may yield more coherent responses, they still hallucinate or over\-generalize under weak retrieval, as shown in\([Lee et al\., 2025](https://arxiv.org/html/2609.22537#bib.bib8)\)and\([Soudani et al\., 2024](https://arxiv.org/html/2609.22537#bib.bib6)\)\. Moreover, fine\-tuning primarily improves fluency rather than factual traceability[Ghosal et al\. \(2024\)](https://arxiv.org/html/2609.22537#bib.bib7), leaving outputs difficult to map to supporting source spans\. Prior work\([Huang et al\., 2024](https://arxiv.org/html/2609.22537#bib.bib15)\)shows that while model\-level optimization can lower hallucination frequency, it fails to guarantee transparent citation alignment or user\-verifiable provenance, limiting its utility in domains requiring citation\-level accountability\.

![Refer to caption](https://arxiv.org/html/2609.22537v1/figures/pipeline3.png)Figure 1:Complete modular pipeline\.
### 2\.2Basic RAG Systems

Despite widespread use in grounded question answering, RAG systems continue to exhibit weaknesses in heterogeneous or noisy enterprise environments[Chen et al\. \(2024a\)](https://arxiv.org/html/2609.22537#bib.bib12)\. The retrieval stage remains a major bottleneck: most retrievers assume clean, well\-indexed corpora, whereas enterprise repositories are fragmented and inconsistent\. Comprehensive reviews\([Sharma, 2025](https://arxiv.org/html/2609.22537#bib.bib10);[Brown et al\., 2025](https://arxiv.org/html/2609.22537#bib.bib9)\)identify retrieval noise, redundancy, and incomplete indexing as leading sources of downstream hallucinations\. Even when relevant materials are retrieved,citation driftoften occurs, where models cite correct documents but extract unsupported passages\([Patel and Anand, 2024](https://arxiv.org/html/2609.22537#bib.bib11);[Huang et al\., 2024](https://arxiv.org/html/2609.22537#bib.bib15)\)\. Because retrieval and generation are typically decoupled, existing pipelines lack explicit mechanisms to ensure that each claim is grounded in an identifiable source span\. As a result, citation precision remains low, undermining user trust and auditability in enterprise deployments\.

### 2\.3Lack of Focus on Complete Traceability

In RAG and LLM\-based question answering,complete traceability, linking each generated factual statement to its exact supporting span, is essential for enterprise use\. While prior work[Rashkin et al\. \(2021\)](https://arxiv.org/html/2609.22537#bib.bib2);[Xu et al\. \(2025\)](https://arxiv.org/html/2609.22537#bib.bib27)introduces citation metrics focused on precision and recall, these do not capture the span\-level grounding required for verifiable outputs\. Furthermore, most evaluations[Gao et al\. \(2023\)](https://arxiv.org/html/2609.22537#bib.bib31)rely on public benchmarks with short, homogeneous texts that differ markedly from the long and unstructured documents found in enterprise corpora\. Consequently, current systems lack fine\-grained provenance and remain limited in supporting transparent and verifiable generation\.

## 3Our Approach

Figure[1](https://arxiv.org/html/2609.22537#S2.F1)presents the pipeline components, and Figure[2](https://arxiv.org/html/2609.22537#S3.F2)illustrates the end\-to\-end query\-processing workflow\. The pipeline includes the following components\.

### 3\.1Document Retrieval Module

This module operates via an automated federated retrieval process\. Upon receiving a query, the system acts as a federated orchestrator, triggering parallel API calls to multiple pre\-configured, independent data sources \(e\.g\., wikis, policy manuals, and technical documentation\)\. Because these enterprise repositories are managed independently, we treat each backend as a distinct “black box”\. The pipeline interfaces directly with each source’s native retrieval infrastructure \(whether lexical, dense, or hybrid\) to aggregate the topNNrelevant documents into a unified context\. This retriever\-agnostic design ensures the system scales flexibly to new data sources in real time without requiring costly index re\-alignment or manual intervention\.

### 3\.2Structured Passage Extraction Module

This module extracts candidate passages from retrieved documents using an LLM with structured output formatting\. The prompt is designed to return verbatim spans from the document context \(see Appendix[D](https://arxiv.org/html/2609.22537#A4)\)\. Each passage is represented in the following JSON structure:

```
{
  "passage_id": "<passage_id>",
  "url": "<url>",
  "content": "<passage_content>"
}
```

We take advantage of the fact that LLMs achieve near\-perfect performance on verbatim span extraction from long\-context source documents\([Hsieh et al\., 2024](https://arxiv.org/html/2609.22537#bib.bib13)\)and structurally straightforward format transformations\([Yang et al\., 2025](https://arxiv.org/html/2609.22537#bib.bib14)\)\.

### 3\.3Alignment and Filtering Module

This module filters hallucinated passages based on n\-gram overlap with their source documents\. For each passage, we retrieve the full content of the source document using the providedurlfield\. We then compute the n\-gram overlap \(typicallyn=5n=5\) between the passage and the document content\. Letppbe an extracted passage andddits cited document\. Define thenn\-gram overlap ratio as

overlapn​\(p,d\)=\|Gn​\(p\)∩Gn​\(d\)\|\|Gn​\(p\)\|\.\\mathrm\{overlap\}\_\{n\}\(p,d\)=\\frac\{\\lvert G\_\{n\}\\\!\\left\(p\\right\)\\cap G\_\{n\}\\\!\\left\(d\\right\)\\rvert\}\{\\lvert G\_\{n\}\\\!\\left\(p\\right\)\\rvert\}\.\(1\)whereGn​\(p\)G\_\{n\}\(p\)is the multiset ofnn\-grams inpp\. We define the filtering action as follows:

- •Ifoverlap\>\>threshold: the passage is retained with the original URL\.
- •If0<<overlap≤\\leqthreshold: the passage is truncated to retain only the overlapping portion; the originalurlis preserved\.
- •Ifoverlap=0=0: we suspect citation drift\. We iterate over all other retrieved documents to find one with overlap\>\>threshold\. If found, the passageurlis replaced with the correct one\.
- •If none of these conditions apply, we drop the passage from the generated JSON\.

We rely on lexical matching because our enterprise data is out of distribution with respect to the training data of most semantic models, because our design prioritizes verifiability over abstraction\. Based on our experiments, the n\-gram with a threshold of 0\.7 performs best for our setting\. This overlap threshold is configurable\. In lower\-risk or less latency\-sensitive deployments, practitioners may tune the threshold or incorporate semantic or hybrid matching strategies\. For the ablation on our data please see Appendix[G](https://arxiv.org/html/2609.22537#A7)\.

![Refer to caption](https://arxiv.org/html/2609.22537v1/figures/visual2.png)Figure 2:A step\-by\-step visual of our pipeline showing how a query is processed, fromdocument retrievaltoanswer generationwith example content that is entirely fictional and used only for illustration\.
### 3\.4Answer Generation Module

A second LLM call generates the final answer using only the filtered and verified verbatim passages\. Guided by a structured prompt \(Appendix[D](https://arxiv.org/html/2609.22537#A4)\), the LLM enforces strict grounding and formatting constraints to ensure maximal traceability while transforming the verbatim passages into a coherent and digestible answer\. Each sentence in the final answer is cited using the associatedurlfields, allowing the exact source location of the supporting content to be surfaced to the user\. In the user interface, we highlight the precise text spans corresponding to each citation in the source documents \(Figure[2](https://arxiv.org/html/2609.22537#S3.F2)\)\. Clicking an inline citation takes the user directly to the exact location from which the referenced information originates\.

### 3\.5Post\-processing Module

The post\-processing module programmatically refines the generation from[3\.4](https://arxiv.org/html/2609.22537#S3.SS4)to create a coherent final output to be presented to the users\. The processing includes formatting paragraphs, removing repeated citationurls, and standardizing company\-specific terms\. Ensuring that product names appear in a consistent, correctly capitalized format improves legibility\.

## 4Experimental Settings

We use two publicly available open\-weight models of differing sizes –LLaMA\-3\.1\-8B\(M1\) andLLaMA\-3\.3\-70B\(M2\) – to demonstrate that our strategy is agnostic to model scale\. We use open\-weight models to adhere to data constraints\. The test set has approximately 500 data points, collected from our initial pilot study to closely reflect real user behavior\. For maximal determinism, the temperature for each model call is set to 0\.0\.

## 5Baselines

Building on the limitations in Section[2](https://arxiv.org/html/2609.22537#S2), we evaluate baselines that align with our goals of improvinggroundingandtraceabilitywithout retraining or multi\-stage orchestration\.

Our design follows two principles\.\(1\) Scope alignment:we improve grounding and traceability in a model\- and data\-agnostic way; comparing with fine\-tuned RAG systems[Asai et al\. \(2024\)](https://arxiv.org/html/2609.22537#bib.bib26);[Lee et al\. \(2025\)](https://arxiv.org/html/2609.22537#bib.bib8)would conflate architectural complexity with our verification mechanism\.\(2\) Practical relevance:enterprise environments often preclude retraining or large\-scale supervision due to privacy, fragmentation, and latency constraints, including strict limits on real\-time LLM calls[Qian et al\. \(2025\)](https://arxiv.org/html/2609.22537#bib.bib24);[Sun et al\. \(2024\)](https://arxiv.org/html/2609.22537#bib.bib25)\. Thus, we evaluate under realistic plug\-and\-play conditions\.EvidenTremains complementary to advanced verifier\-based systems and can be layered on them for stronger grounding and traceability\. See Appendix[H](https://arxiv.org/html/2609.22537#A8)for why more complex verification or fine\-tuning frameworks are omitted\. We compare with two representative baselines \(prompts in Appendix[C](https://arxiv.org/html/2609.22537#A3)\):

Direct Prompting with Inline Citations:The LLM generates citations inline as part of its response, serving as a minimal citation\-aware baseline without explicit verification[Singal et al\. \(2024\)](https://arxiv.org/html/2609.22537#bib.bib22);[Lewis et al\. \(2020\)](https://arxiv.org/html/2609.22537#bib.bib23)\.

Ground\-Every\-Sentence:The LLM appends a citation after every sentence, ensuring each atomic statement is grounded in at least one retrieved document, enabling fine\-grained evaluation of citation precision and coverage[Xia et al\. \(2025\)](https://arxiv.org/html/2609.22537#bib.bib19)\.

## 6Evaluation

While the ultimate measure of an enterprise assistant’s success lies inuser satisfaction, such metrics can only be meaningfully assessed post\-deployment\. During development, we therefore rely onproxy metricsthat correlate with user trust and perceived reliability\. Below, we describe the evaluation setup and results comparing our proposed approach with the two baseline methods introduced in Section[5](https://arxiv.org/html/2609.22537#S5)\. Due to space constraints, we report here only the comparative results with baselines\. A more detailedintrinsic evaluationof our method, including fine\-grained analyses of alignment and human relevance judgments, is provided in Appendix[F](https://arxiv.org/html/2609.22537#A6)\. We also provide long\-term post\-deployment metrics and user satisfaction telemetry from a large\-scale live production environment in Section[7](https://arxiv.org/html/2609.22537#S7)and metrics on latency in Section[8](https://arxiv.org/html/2609.22537#S8)\.

### 6\.1Evaluating Groundedness \(RQ1\)

To evaluate citation groundedness in relation to RQ1, we compute the following metrics:

- •%Hallucination:A binary indicator of whether an answer contains hallucinated citations\. Since any hallucinated citation is unacceptable in enterprise settings like ours, the score is 0 if all generated citations correspond to retrieved documents, and 1 otherwise\. Letc′\\mathrm\{c^\{\\prime\}\}be the set of generated citations,c\\mathrm\{c\}the set of retrievedurlsand𝟙​\[⋅\]\\mathbb\{1\}\[\\cdot\]equals 0 if true and 1 if false\. Then: Hallucination=𝟙\[c′⊆c\]\\mathrm\{Hallucination\}=\\mathbb\{1\}\\big\[\\mathrm\{c^\{\\prime\}\}\\subseteq\\mathrm\{c\}\\big\]\(2\)
- •%Groundedness:A binary measure indicating whether the model’s output is supported by at least one gold\-annotated source document\. Gold citations, provided by subject\-matter experts, identify documents sufficient but not exhaustive to answer each test instance, as enumerating all valid sources is impractical in dynamically changing enterprise settings; therefore, we do not penalize citations not available in the gold set \(see Appendix[E](https://arxiv.org/html/2609.22537#A5)for more details\)\. Letc′\\mathrm\{c^\{\\prime\}\}denote the set of generated citations andc\\mathrm\{c\}the set of gold citations\. Then: Groundedness=𝟙\[c′∩c≠∅\]\\mathrm\{Groundedness\}=\\mathbb\{1\}\\big\[\\mathrm\{c^\{\\prime\}\}\\cap\\mathrm\{c\}\\neq\\emptyset\\big\]\(3\)where𝟙​\[⋅\]\\mathbb\{1\}\[\\cdot\]equals 1 if there is an overlap between generated and gold citations, 0 otherwise\.

Table 2:Comparison of prompting strategies across verification metrics for models M1 and M2\.∗denotes statistical significance atp<0\.05p<0\.05compared to both baselines\.As shown in Table[2](https://arxiv.org/html/2609.22537#S6.T2),EvidenTsubstantially improves factual groundedness and eliminates hallucinations across both model scales\. We compute 95% confidence intervals using a binomial approximation of the standard error:

p±1\.96×p⁡\(1−p\)np\\pm 1\.96\\times\\sqrt\{\\frac\{p\(1\-p\)\}\{n\}\}\(4\)
Two\-proportionzz\-tests confirm thatEvidenT’s gains are highly statistically significant atp<0\.05p<0\.05across all configurations\.

Table 3:Answer Coverage \(AnsCov\) and Document Focus \(DocFocus\) metrics across methods and models\. HigherCoverageindicates that a document captures more of the generated answer’s content, while higherFocusreflects a greater proportion of the document being relevant to the answer\.
### 6\.2Evaluating Traceability \(RQ2\)

We use both semantic and lexical post\-generation metrics to quantify \(pertaining to RQ2\) how well a model\-generated answer can be traced back to its source document\.

For all subsequent evaluations, we first extract factual statements by taking the text preceding each citation: for example, theblueandbrownspans in Figure[2](https://arxiv.org/html/2609.22537#S3.F2)illustrate two separate facts\. ForEvidenT, we additionally gather both the extracted facts and their corresponding source passages by matching citedurls\. These are then compared against the content of the cited documents, identified through the same citedurls\. Results are averaged across all queries\.

Given a reference documentDDand a model\-generated answerAA, we quantify how traceableAAis toDD\. We employ several methods to measure this\. LetUUandVVdenote the multisets of tokens fromAAandDD, respectively\.ttdenotes the token\.

#### 6\.2\.1Word Overlap

LetO=∑tmin⁡\(countU​\(t\),countV​\(t\)\)O=\\sum\_\{t\}\\min\(\\mathrm\{count\}\_\{U\}\(t\),\\mathrm\{count\}\_\{V\}\(t\)\)for each tokentt\.

AnsCov=O\|U\|,\\displaystyle=\\frac\{O\}\{\|U\|\},DocFocus=O\|V\|\\displaystyle=\\frac\{O\}\{\|V\|\}\(5\)
AnsCovmeasures how well the document covers the answer content, whileDocFocusreflects how concentrated the document is on that answer\.

#### 6\.2\.2n\-gram Overlap

Forn∈\{2,3,5,10\}n\\in\\\{2,3,5,10\\\}, let𝒢n​\(T\)\\mathcal\{G\}\_\{n\}\(T\)denote the multiset ofnn\-grams in textTT, and letIn=\|𝒢n​\(A\)∩𝒢n​\(D\)\|I\_\{n\}=\|\\mathcal\{G\}\_\{n\}\(A\)\\cap\\mathcal\{G\}\_\{n\}\(D\)\|be the number of overlappingnn\-grams between the answerAAand documentDD\. We defineAnsCov@n, which measures how much of the answer’s phrasing is covered by the document, andDocFocus@n, which captures how concentrated the document is on the answer content\. Largernnvalues prefer near\-verbatim phrasing and reduce tolerance for paraphrasing\.

AnsCov@​n\\displaystyle\\text\{AnsCov@\}n=In\|𝒢n​\(A\)\|,\\displaystyle=\\frac\{I\_\{n\}\}\{\|\\mathcal\{G\}\_\{n\}\(A\)\|\},DocFocus@​n\\displaystyle\\text\{DocFocus@\}n=In\|𝒢n​\(D\)\|\\displaystyle=\\frac\{I\_\{n\}\}\{\|\\mathcal\{G\}\_\{n\}\(D\)\|\}\(6\)The results are presented in Table[3](https://arxiv.org/html/2609.22537#S6.T3)\. The high AnsCov values and the low DocFocus values reflect the long, noisy nature of the source documents\.EvidenTachieves substantial improvements in coverage over both baselines\. We observe extremely high absolute values for our technique\. The Alignment and Filtering module \(Section[3\.3](https://arxiv.org/html/2609.22537#S3.SS3)\) filters out hallucinated content, ensuring that the retained text is almost entirely verbatim\. Furthermore, when analyzing the AnsCov metric and the n\-gram–based coverage atnn= 10, we find thatEvidenTachieves near\-saturated scores\. While AnsCov primarily reflects lexical overlap between generated answers and cited text, thenn= 10 coverage provides a more stringent measure, focusing on longer exact matches in the source document\. This not only validates that the generated spans are faithfully extracted from the evidence but also demonstrates that our method can accurately recover the exact supporting spans if needed, reinforcingEvidenT’s transparency and traceability advantages over the baselines\.

#### 6\.2\.3Semantic Matching

For semantic scoring, we create overlapping windows \(256 tokens, stride 50\) for bothAAandDD\. We use the off\-the\-shelf cross\-encoder to compute similarity scores between each answer\{ai\}\\\{a\_\{i\}\\\}and document window\{dj\}\\\{d\_\{j\}\\\}\. The model jointly encodes each text pair and outputs a scalar relevance score, which we use without normalization\.

SemMax=maxi,j⁡s⁡\(ai,dj\),\\displaystyle=\\max\_\{i,j\}s\(a\_\{i\},d\_\{j\}\),\(7\)SemRecall=1\|\{ai\}\|​∑imaxj⁡s⁡\(ai,dj\)\\displaystyle=\\tfrac\{1\}\{\|\\\{a\_\{i\}\\\}\|\}\\sum\_\{i\}\\max\_\{j\}s\(a\_\{i\},d\_\{j\}\)\(8\)wheres⁡\(ai,dj\)s\(a\_\{i\},d\_\{j\}\)denotes the raw relevance score produced by the cross\-encoder\.SemMaxcaptures the strongest localized semantic match between the answer and the document, whileSemRecallreflects the overall semantic coverage ofAAbyDD\.

Table 4:Semantic similarity metrics comparing different model sizes\.Table[4](https://arxiv.org/html/2609.22537#S6.T4)shows that SemMax is consistently higher than SemRecall across all settings\. This suggests that, for all model sizes and methods, the generated answers contain at least some portions that align well semantically with the corresponding documents\. However, when semantic alignment is averaged across all answer segments \(i\.e\., SemRecall\), M1 exhibits much weaker overall coverage\. For M2, we observe thatEvidenTachieves approximately a13%improvement in SemMax and more than a100%improvement in SemRecall, indicating thatEvidenTnot only produces content with stronger localized matches but also maintains substantially better semantic consistency with the source documents overall\.

## 7Post\-Deployment Case Study

While intrinsic development evaluations rely on proxy metrics, the ultimate validation of an enterprise assistant lies in sustained utility and end\-user trust after deployment\. To assess this, we tracked a production\-scale implementation of our framework over a five\-month lifecycle\. The deployment served more than 200 active internal users across distinct corporate functions distributed across multiple geographies\.

In this live production environment, the citation alignment and grounding improvements introduced byEvidenThelped mitigate the user friction points identified during our initial pilot\. User experience outcomes were monitored through explicit interface telemetry:

- •Helpfulness Feedback: Binary thumbs\-up/thumbs\-down feedback attached to generated responses showed a consistent\+28% relative improvementover the baseline RAG implementation\.
- •User\-Rated Contextual Relevance: End users were asked to explicitly assess whether a response was relevant to the submitted query and retrieved context\. This user\-facing relevance signal improved by\+19%over the deployment lifecycle, indicating that users more frequently perceived responses as aligned with their information needs\.
- •User Retention: The architecture achieved a stable90% user retention rateover the full five\-month observation window, signaling robust system utility in day\-to\-day corporate workflows\.

These post\-deployment measurements should be interpreted as user\-experience indicators rather than isolated causal estimates\. Live enterprise metrics are inherently influenced by multiple system\-level factors, including retrieval quality, interface design, user population, and workflow adoption\. Nevertheless, the consistency between our deterministic grounding improvements and the observed gains in helpfulness, user\-rated contextual relevance, and retention provides supporting evidence that tight, span\-level citation tracking contributes to deployment readiness in high\-stakes enterprise environments\.

## 8Latency Considerations

Compared to baseline RAG systems, our pipeline introduces a slight increase in end\-to\-end latency in exchange for substantial gains in groundedness and traceability\. The baseline RAG system relies on a single long\-context LLM generation and exhibits an average latency of approximately 10±\\pm3 seconds per query\. Our pipeline decomposes generation into two LLM calls and a lightweight alignment step\. The first LLM call performs long\-context passage extraction with latency comparable to baseline generation, followed by an n\-gram–based alignment and filtering step with negligible overhead\. The final answer is generated using a much shorter context LLM call over distilled passages\. Overall, the end\-to\-end latency of our pipeline is approximately 12±\\pm3 seconds per query\. This increase of roughly 2 seconds represents a more favorable trade\-off than approaches that rely on multiple iterative LLM calls[Qian et al\. \(2025\)](https://arxiv.org/html/2609.22537#bib.bib24);[Sun et al\. \(2024\)](https://arxiv.org/html/2609.22537#bib.bib25), verifier\-based reasoning, or domain\-specific fine\-tuning[Asai et al\. \(2024\)](https://arxiv.org/html/2609.22537#bib.bib26);[Lee et al\. \(2025\)](https://arxiv.org/html/2609.22537#bib.bib8), which often incur significantly higher latency, cost, and operational complexity\.

## 9Conclusion

We presentedEvidenT, a lightweight, model\-agnostic framework that improves verifiability and traceability in extractive enterprise QA without retraining\. Our method substantially reduces unverifiable output and enhances alignment between generated responses and source content across model scales\. While our work focuses on QA, the approach is applicable to other generation tasks in enterprise settings that demand citation precision\.

## 10Limitations

Our work proposes a pipeline for producing more grounded and traceable citations in extractive enterprise QA\. It is designed specifically for the fragmented, noisy, and heterogeneous data typical of enterprise environments, where model reasoning and assumption\-making are discouraged to ensure factual precision\. As a result, we do not evaluate on standard public benchmarks, which fail to capture the complexity of such data\.EvidenTis not intended for abstractive synthesis or paraphrase\-heavy generation; extending the framework to support higher\-level abstraction while preserving span\-level accountability is an important direction for future work\. Our evaluation relies on semantic and lexical metrics as proxies for post\-deployment outcomes like user satisfaction; future work includes controlled A/B testing to better estimate real\-world impact\. Additionally, our experiments focus exclusively on text\-based modalities, with support for other modalities left to future work\. Finally, we evaluate only open\-weight models, as extensive testing with proprietary models is constrained by data privacy considerations\.

## 11Ethical Considerations

Deploying language models as enterprise assistants in high\-stakes environments requires rigorous oversight regarding data governance, misinformation, and human accountability\. To address these concerns,EvidenTutilizes open\-weight models to ensure strict data constraints and privacy considerations are respected\. Furthermore, by enforcing a deterministic lexical alignment check, the pipeline eliminates citation hallucinations, actively preventing the propagation of misinformation in critical domains like law, finance, and healthcare\. Additionally, the system alerts users within the interface that the provided answer is AI\-generated and can be wrong\. To counter the resulting risks of automation bias and ensure users do not blindly trust the system,EvidenTintegrates span\-level traceability directly into the user interface, highlighting the exact source text so that the generation remains entirely transparent and user\-verifiable\.

## References

- Asaiet al\.\(2024\)A\. Asai, Z\. Wu, Y\. Wang, A\. Sil, and H\. HajishirziSelf\-RAG: learning to retrieve, generate, and critique through self\-reflection\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=hSyW5go0v8)Cited by:[§H\.1](https://arxiv.org/html/2609.22537#A8.SS1.p1.1),[§5](https://arxiv.org/html/2609.22537#S5.p2.1),[§8](https://arxiv.org/html/2609.22537#S8.p1.1)\.
- Baglaet al\.\(2023\)K\. Bagla, S\. Gupta, A\. Kumar, and A\. GuptaNoisy text data: foible of popular transformer based nlp models\.InProceedings of the Third International Conference on AI\-ML Systems,pp\. 1–5\.Cited by:[§H\.4](https://arxiv.org/html/2609.22537#A8.SS4.p1.1)\.
- Bowmanet al\.\(2015\)S\. Bowman, G\. Angeli, C\. Potts, and C\. D\. ManningA large annotated corpus for learning natural language inference\.InProceedings of the 2015 conference on empirical methods in natural language processing,pp\. 632–642\.Cited by:[§H\.4](https://arxiv.org/html/2609.22537#A8.SS4.p1.1)\.
- Brownet al\.\(2025\)A\. Brown, M\. Roman, and B\. DevereuxA systematic literature review of retrieval\-augmented generation: techniques, metrics, and challenges\.arXiv preprint arXiv:2508\.06401\.Cited by:[§2\.2](https://arxiv.org/html/2609.22537#S2.SS2.p1.1)\.
- Chenet al\.\(2024a\)J\. Chen, H\. Lin, X\. Han, and L\. SunBenchmarking large language models in retrieval\-augmented generation\.InProceedings of the AAAI Conference on Artificial Intelligence,External Links:[Link](https://doi.org/10.1609/aaai.v38i16.29728),[Document](https://dx.doi.org/10.1609/aaai.v38i16.29728)Cited by:[§2\.2](https://arxiv.org/html/2609.22537#S2.SS2.p1.1)\.
- Chenet al\.\(2024b\)Z\. Z\. Chen, J\. Ma, X\. Zhang, N\. Hao, A\. Yan, A\. Nourbakhsh, X\. Yang, J\. McAuley, L\. Petzold, and W\. Y\. WangA survey on large language models for critical societal domains: finance, healthcare, and law\.arXiv preprint arXiv:2405\.01769\.Cited by:[§1](https://arxiv.org/html/2609.22537#S1.p1.1)\.
- Choubeyet al\.\(2025\)P\. K\. Choubey, X\. Peng, S\. Bhagavath, K\. Huang, C\. Xiong, and C\. WuBenchmarking deep search over heterogeneous enterprise data\.InProceedings of the Conference on Empirical Methods in Natural Language Processing \(Industry Track\),External Links:[Link](https://aclanthology.org/2025.emnlp-industry.34/),[Document](https://dx.doi.org/10.18653/v1/2025.emnlp-industry.34)Cited by:[§1](https://arxiv.org/html/2609.22537#S1.p1.1)\.
- Gaoet al\.\(2023\)T\. Gao, H\. Yen, J\. Yu, and D\. ChenEnabling large language models to generate text with citations\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,pp\. 6465–6488\.Cited by:[§2\.3](https://arxiv.org/html/2609.22537#S2.SS3.p1.1)\.
- Ghosalet al\.\(2024\)G\. Ghosal, T\. Hashimoto, and A\. RaghunathanUnderstanding finetuning for factual knowledge extraction\.InInternational Conference on Learning Representations,Cited by:[§2\.1](https://arxiv.org/html/2609.22537#S2.SS1.p1.1)\.
- Hanet al\.\(2024\)Z\. Han, C\. Gao, J\. Liu, J\. Zhang, and S\. Q\. ZhangParameter\-efficient fine\-tuning for large models: a comprehensive survey\.arXiv preprint arXiv:2403\.14608\.Cited by:[§H\.1](https://arxiv.org/html/2609.22537#A8.SS1.p1.1)\.
- Hsiehet al\.\(2024\)C\. Hsieh, S\. Sun, S\. Kriman, S\. Acharya, D\. Rekesh, F\. Jia, Y\. Zhang, and B\. GinsburgRULER: what’s the real context size of your long\-context language models?\.arXiv preprint arXiv:2404\.06654\.Cited by:[§3\.2](https://arxiv.org/html/2609.22537#S3.SS2.p3.1)\.
- Huanget al\.\(2024\)L\. Huang, X\. Feng, W\. Ma, Y\. Gu, W\. Zhong, X\. Feng, W\. Yu, W\. Peng, D\. Tang, D\. Tu,et al\.Learning fine\-grained grounded citations for attributed large language models\.InFindings of the Association for Computational Linguistics: ACL 2024,pp\. 14095–14113\.Cited by:[§2\.1](https://arxiv.org/html/2609.22537#S2.SS1.p1.1),[§2\.2](https://arxiv.org/html/2609.22537#S2.SS2.p1.1)\.
- Jorenet al\.\(2025\)H\. Joren, J\. Zhang, C\. Ferng, D\. Juan, A\. Taly, and C\. RashtchianSufficient context: a new lens on retrieval augmented generation systems\.InThe Thirteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=Jjr2Odj8DJ)Cited by:[§1](https://arxiv.org/html/2609.22537#S1.p1.1)\.
- Kryscinskiet al\.\(2020\)W\. Kryscinski, B\. McCann, C\. Xiong, and R\. SocherEvaluating the factual consistency of abstractive text summarization\.InProceedings of the Conference on Empirical Methods in Natural Language Processing,External Links:[Link](https://aclanthology.org/2020.emnlp-main.750/),[Document](https://dx.doi.org/10.18653/v1/2020.emnlp-main.750)Cited by:[§1](https://arxiv.org/html/2609.22537#S1.p1.1)\.
- Labanet al\.\(2022\)P\. Laban, T\. Schnabel, P\. N\. Bennett, and M\. A\. HearstSummaC: re\-visiting NLI\-based models for inconsistency detection in summarization\.Transactions of the Association for Computational Linguistics10,pp\. 163–177\.External Links:[Link](https://aclanthology.org/2022.tacl-1.10/),[Document](https://dx.doi.org/10.1162/tacl%5Fa%5F00453)Cited by:[§1](https://arxiv.org/html/2609.22537#S1.p1.1)\.
- Leeet al\.\(2025\)Z\. P\. Lee, A\. Lin, and C\. TanFinetune\-rag: fine\-tuning language models to resist hallucination in retrieval\-augmented generation\.arXiv preprint arXiv:2505\.10792\.Cited by:[§2\.1](https://arxiv.org/html/2609.22537#S2.SS1.p1.1),[§5](https://arxiv.org/html/2609.22537#S5.p2.1),[§8](https://arxiv.org/html/2609.22537#S8.p1.1)\.
- Lewiset al\.\(2020\)P\. Lewis, E\. Perez, A\. Piktus, F\. Petroni, V\. Karpukhin, N\. Goyal, H\. Küttler, M\. Lewis, W\. Yih, T\. Rocktäschel,et al\.Retrieval\-augmented generation for knowledge\-intensive nlp tasks\.Advances in neural information processing systems33,pp\. 9459–9474\.Cited by:[§5](https://arxiv.org/html/2609.22537#S5.p3.1)\.
- Novikovaet al\.\(2017\)J\. Novikova, O\. Dušek, and V\. RieserWhy we need new evaluation metrics for nlg\.InEMNLP,Cited by:[§1](https://arxiv.org/html/2609.22537#S1.p1.1)\.
- Packowskiet al\.\(2024\)S\. Packowski, I\. Halilovic, J\. Schlotfeldt, and T\. SmithOptimizing and evaluating enterprise retrieval\-augmented generation \(RAG\): a content design perspective\.InProceedings of the 2024 8th International Conference on Advances in Artificial Intelligence,pp\. 162–167\.Cited by:[§1](https://arxiv.org/html/2609.22537#S1.p1.1)\.
- Patel and Anand \(2024\)M\. Patel and A\. AnandFactuality or fiction? benchmarking modern llms on ambiguous qa with citations\.arXiv preprint arXiv:2412\.18051\.Cited by:[§2\.2](https://arxiv.org/html/2609.22537#S2.SS2.p1.1)\.
- Penzkofer and Baumann \(2024\)V\. Penzkofer and T\. BaumannEvaluating and fine\-tuning retrieval\-augmented language models to generate text with accurate citations\.InProceedings of the 20th Conference on Natural Language Processing \(KONVENS 2024\),pp\. 57–64\.Cited by:[§2\.1](https://arxiv.org/html/2609.22537#S2.SS1.p1.1)\.
- Qianet al\.\(2025\)H\. Qian, Y\. Fan, J\. Guo, R\. Zhang, Q\. Chen, D\. Yin, and X\. ChengVericite: towards reliable citations in retrieval\-augmented generation via rigorous verification\.InProceedings of the 2025 Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region,pp\. 47–54\.Cited by:[§H\.2](https://arxiv.org/html/2609.22537#A8.SS2.p1.1),[§5](https://arxiv.org/html/2609.22537#S5.p2.1),[§8](https://arxiv.org/html/2609.22537#S8.p1.1)\.
- Rashkinet al\.\(2021\)H\. Rashkin, D\. Reitter, G\. S\. Tomar, and D\. DasIncreasing faithfulness in knowledge\-grounded dialogue with controllable features\.InProceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing \(Volume 1: Long Papers\),pp\. 704–718\.Cited by:[§2\.3](https://arxiv.org/html/2609.22537#S2.SS3.p1.1)\.
- Ruet al\.\(2024\)D\. Ru, L\. Qiu, X\. Hu, T\. Zhang, P\. Shi, S\. Chang, C\. Jiayang, C\. Wang, S\. Sun, H\. Li,et al\.Ragchecker: a fine\-grained framework for diagnosing retrieval\-augmented generation\.Advances in Neural Information Processing Systems37,pp\. 21999–22027\.Cited by:[§H\.3](https://arxiv.org/html/2609.22537#A8.SS3.p1.1)\.
- Sharma \(2025\)C\. SharmaRetrieval\-augmented generation: a comprehensive survey of architectures, enhancements, and robustness frontiers\.arXiv preprint arXiv:2506\.00054\.Cited by:[§2\.2](https://arxiv.org/html/2609.22537#S2.SS2.p1.1)\.
- Singalet al\.\(2024\)R\. Singal, P\. Patwa, P\. Patwa, A\. Chadha, and A\. DasEvidence\-backed fact checking using rag and few\-shot in\-context learning with llms\.InProceedings of the Seventh Fact Extraction and VERification Workshop \(FEVER\),pp\. 91–98\.Cited by:[§5](https://arxiv.org/html/2609.22537#S5.p3.1)\.
- Soudaniet al\.\(2024\)H\. Soudani, E\. Kanoulas, and F\. HasibiFine tuning vs\. retrieval augmented generation for less popular knowledge\.InProceedings of the 2024 Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region,pp\. 12–22\.Cited by:[§2\.1](https://arxiv.org/html/2609.22537#S2.SS1.p1.1)\.
- Sunet al\.\(2024\)H\. Sun, H\. Cai, B\. Wang, Y\. Hou, X\. Wei, S\. Wang, Y\. Zhang, and D\. YinTowards verifiable text generation with evolving memory and self\-reflection\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,pp\. 8211–8227\.Cited by:[§H\.2](https://arxiv.org/html/2609.22537#A8.SS2.p1.1),[§5](https://arxiv.org/html/2609.22537#S5.p2.1),[§8](https://arxiv.org/html/2609.22537#S8.p1.1)\.
- Tanget al\.\(2024\)L\. Tang, P\. Laban, and G\. DurrettMinicheck: efficient fact\-checking of llms on grounding documents\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,pp\. 8818–8847\.Cited by:[§H\.2](https://arxiv.org/html/2609.22537#A8.SS2.p1.1)\.
- Umaret al\.\(1999\)A\. Umar, G\. Karabatis, L\. Ness, B\. Horowitz, and A\. ElmagardmidEnterprise data quality: a pragmatic approach\.Information Systems Frontiers1\(3\),pp\. 279–301\.Cited by:[§1](https://arxiv.org/html/2609.22537#S1.p1.1)\.
- Williamset al\.\(2018\)A\. Williams, N\. Nangia, and S\. BowmanA broad\-coverage challenge corpus for sentence understanding through inference\.InProceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 \(Long Papers\),pp\. 1112–1122\.Cited by:[§H\.4](https://arxiv.org/html/2609.22537#A8.SS4.p1.1)\.
- Xiaet al\.\(2025\)S\. Xia, X\. Wang, J\. Liang, Y\. Zhang, W\. Zhou, J\. Deng, F\. Yu, and Y\. XiaoGround every sentence: improving retrieval\-augmented LLMs with interleaved reference\-claim generation\.InFindings of the Association for Computational Linguistics: NAACL 2025,pp\. 969–988\.Cited by:[§5](https://arxiv.org/html/2609.22537#S5.p4.1)\.
- Xiaet al\.\(2024\)Y\. Xia, J\. Kim, Y\. Chen, H\. Ye, S\. Kundu, C\. C\. Hao, and N\. TalatiUnderstanding the performance and estimating the cost of llm fine\-tuning\.In2024 IEEE International Symposium on Workload Characterization \(IISWC\),pp\. 210–223\.Cited by:[§H\.1](https://arxiv.org/html/2609.22537#A8.SS1.p1.1)\.
- Xuet al\.\(2025\)Y\. Xu, P\. Qi, J\. Chen, K\. Liu, R\. Han, L\. Liu, B\. Min, V\. Castelli, A\. Gupta, and Z\. WangCiteeval: principle\-driven citation evaluation for source attribution\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 32759–32778\.Cited by:[§2\.3](https://arxiv.org/html/2609.22537#S2.SS3.p1.1)\.
- Yanget al\.\(2025\)J\. Yang, D\. Jiang, L\. He, S\. Siu, Y\. Zhang, D\. Liao, Z\. Li, H\. Zeng, Y\. Jia, H\. Wang,et al\.StructEval: benchmarking llms’ capabilities to generate structural outputs\.arXiv preprint arXiv:2505\.20139\.Cited by:[§3\.2](https://arxiv.org/html/2609.22537#S3.SS2.p3.1)\.
- Yanget al\.\(2023\)L\. Yang, Y\. Song, X\. Ren, C\. Lyu, Y\. Wang, J\. Zhuo, L\. Liu, J\. Wang, J\. Foster, and Y\. ZhangOut\-of\-distribution generalization in natural language processing: past, present, and future\.InProceedings of the 2023 conference on empirical methods in natural language processing,pp\. 4533–4559\.Cited by:[§H\.4](https://arxiv.org/html/2609.22537#A8.SS4.p1.1)\.
- Zhanget al\.\(2024\)T\. Zhang, S\. G\. Patil, N\. Jain, S\. Shen, M\. Zaharia, I\. Stoica, and J\. E\. GonzalezRAFT: adapting language models to domain\-specific rag\.InConference on Language Modeling,External Links:[Link](https://openreview.net/forum?id=rzQGHXNReU)Cited by:[§2\.1](https://arxiv.org/html/2609.22537#S2.SS1.p1.1)\.

## Appendix

## Appendix APilot Deployment

The assistant was powered by a state\-of\-the\-art large language model111As this information is internal, we’re not able to disclose additional details\.and implemented using a retrieval\-augmented generation \(RAG\) architecture tailored for enterprise contexts\. For each user query, the system retrieved the top n documents from a heterogeneous corpus of internal enterprise sources, including wikis, policy manuals, knowledge base articles, and product documentation, using a customized retrieval backend optimized for latency and format diversity\. Retrieved documents were incorporated into the model’s context using a task\-specific prompt, guiding the model to generate natural language responses with inline citations grounded in the source content \(see Appendix[B](https://arxiv.org/html/2609.22537#A2)for prompt details\)\.

The pilot was deployed for one month to a cohort of 55 users across roles such as support, operations, compliance, and product management\. Users submitted queries in real time as part of their daily workflows, and the assistant generated responses synchronously through a live interface\. In total, the system processed approximately 4,000 queries covering a wide range of topics, including troubleshooting procedures and internal policy clarification\.

This deployment surfaced valuable insights into real\-world assistant usage in enterprise environments\. While users appreciated the fluency and relevance of the responses, as well as the promise of citation\-backed justifications, they frequently encountered critical failure modes such as incorrect citations, untraceable references, and unsupported factual claims\.

## Appendix BPrompts for the pilot deployment

### B\.1System Prompt

You are an expert Enterprise Assistant that uses retrieval\-augmented generation to answer questions\. For each question, you are provided with a list of documents from different sources\. Retrieve the most appropriate information and craft a clear, accurate response\. Only provide the answer based on the text from the documents provided\.Instructions: 1\. Cite theurlof the document\(s\) used in your response with reference numbers in brackets \(e\.g\., \[1\], \[2\]\)\. Place these referencesonly at the end of the relevant sentence\(s\)\. Do not include references in the middle of a sentence or as part of the sentence structure\. 2\. At the end of your answer, include a ‘References’ section listing theurlscorresponding to the cited numbers\. Eachurlshould be listed only once, even if referenced multiple times\. 3\. Avoid referring to documents with identifiers like ‘DOCUMENT X’; instead, cite only theurls\. 4\. If the question cannot be answered accurately using the documents, reply with: ‘Not sufficient information to answer\.’ Provide no additional information or references in this case\. Here are the documents: context

### B\.2User Prompt

Answer the following question:queryInstructions: 1\. Cite theurlof the document\(s\) used in your response with reference numbers in brackets \(e\.g\., \[1\], \[2\]\)\. Place these referencesonly at the end of the relevant sentence\(s\)\. Do not include references in the middle of a sentence or as part of the sentence structure\. 2\. At the end of your answer, include a ‘References’ section listing theurlscorresponding to the cited numbers\. Eachurlshould be listed only once, even if referenced multiple times\. 3\. Avoid referring to documents with identifiers like ‘DOCUMENT X’; instead, cite only theurls\. 4\. If the question cannot be answered accurately using the documents, reply with: ‘Not sufficient information to answer\.’ Provide no additional information or references in this case\.Example response format: Answer text\. \[1\], \[2\]\. Answer text\. \[3\]\. References: \[1\]urlof document 1 \[2\]urlof document 2 \[3\]urlof document 3 Answer the question only from the references\. Ensure all citedurlsare included in the References section, and avoid mentioning specific document identifiers \(e\.g\., ‘DOCUMENT X’\)\.

## Appendix CPrompts for off\-the\-shelf techniques

### C\.1Direct\-Prompting

You are a helpful question\-answering assistant\. Classify the user query into one of the following categories and respond accordingly using only thedocumentContentanddocumentMetadatafields from the provided documents\. Do not make assumptions, inferences, or use any external knowledge\. If the answer is not directly stated in the documents, follow the instructions given below carefully\. Think you are the reader of the answer\. The answer should be clear, concise and well formatted\. Ensure that it communicates effectively and presents the information in a structured and readable manner\. ## Categories & Response Instructions ### 1\. Informational Questions Fact\-based queries about what, who, where, when, or whether something exists\. These are easy queries which do not require knowledge of complex relationships to be answered\. Examples: - •What is <product\_name\>? - •Where is <product\_name\>? - •Do we have any information on <product\_name\>? - •Tell me about <product\_name\> - •When is <product\_name\>used? - •Who is <name\>? Response Rules: - •If the answer is available in the provided document\(s\), respond directly\. Assign each source a unique citation number as shown below\. Insert the citation number \(in brackets, e\.g\. \[1\], \[2\]\) directly after relevant sentences\. Place these numbersonly at the end of the relevant sentence\(s\)\. A number can appear multiple times if the corresponding source contains information supporting multiple sentences\. Do not include reference numbers in the middle of a sentence or as part of the sentence structure\. At the end of your answer, include a ‘References’ section listing theurlscorresponding to the cited numbers\. Eachurlshould be listed only once, even if referenced multiple times\. Avoid referring to documents with identifiers like ‘DOCUMENT X’; instead, cite only the reference numbers\. - •If the answer is not explicitly available, provide asummarizationof relevant information from the documents\. - •For complex topics, provide a high\-level overview first, then break down into steps\. - •If similar information appears in multiple documents, state it only once and cite all relevant sources after the sentence as \[X\]\[Y\] where X, Y correspond to citation numbers as described above\. - •Provide a complete answer\. If the response requires multiple steps or examples, include them explicitly—do not simply reference theurl\. - •Keep the response to the point and concise\. Avoid repetitive information\. - •Format lists, sequences, or comma\-separated entities into bullets using the following format: - – - •If the generation process involves multiple steps or examples \(separated by commas\), present them as a bulleted list\. - •Any sentence following the bullet points should begin on a new line\. In\-context Examples: Example \- Answer Stated in Document User Query: What is FEATURE? Response: FEATURE is a feature that allows client devices to sync specific data subsets locally for offline use\. It supports both full and incremental sync options based on user\-defined criteria\. \[1\] References: \[1\] https://docs\.example\.com/page1 Example \- Answer NOT Explicitly Stated in Document User Query: What is PRODUCT? Response: PRODUCT integrates with data gateways to enhance data processing efficiency\. \[1\] It includes components that support both historical and real\-time data ingestion\. \[2\] References: \[1\] https://docs\.example\.com/page7 \[2\] https://docs\.example\.com/page9 Note: The above is a summarization of relevant details from the documents \(Don’t add this statement in the generation\) — ### 2\. Analytical Questions Queries asking for reasons, comparisons, summaries, or relational understanding \(including "why" questions\)\. Examples: - •Why is <product\_name\> used? - •How is <product1\> different from <product2\>? - •What are the differences between <p1\> and <p2\>? - •Can you summarize <product\_name\>? Response Rules: - •If the answer is available in the provided document\(s\), return an explanation\. Assign each source a unique citation number\. Insert the citation number \(in brackets, e\.g\. \[1\], \[2\]\) directly after relevant sentences\. Place these numbersonly at the end of the relevant sentence\(s\)\. A number can appear multiple times if the corresponding source contains information supporting multiple sentences\. Do not include reference numbers in the middle of a sentence or as part of the sentence structure\. At the end of your answer, include a ‘References’ section listing theurlscorresponding to the cited numbers\. Eachurlshould be listed only once, even if referenced multiple times\. Avoid referring to documents with identifiers like ‘DOCUMENT X’; instead, cite only the reference numbers\. - •If the answer is not explicitly available, respond with: The documentation does not explicitly state what the query is asking\. Here are some details from the documentation which may help: - –<Relevant detail 1\> \[1\] - –<Relevant detail 2\> \[2\] References: \[1\] https://docs\.example\.com/page7 \[2\] https://docs\.example\.com/page9 - •For complex topics, provide a high\-level overview first, then break down into steps\. - •If similar information appears in multiple documents, state it once and cite all sources\. - •Provide a complete answer\. If the response requires multiple steps or examples, include them explicitly—do not simply reference the URL\. - •Keep the response to the point and concise\. Avoid repetitive information\. - •If comparing two products, use the format: The key differences between <product1\> and <product2\> are as follows: Product1: <Summarize concisely relevant content from document\. \[1\]\> Product2: <Summarize concisely relevant content from document\. \[2\]\> References: \[1\] https://docs\.example\.com/page7 \[2\] https://docs\.example\.com/page9 - •Format lists and breakdowns using bullets: - – - •If the generation process involves multiple steps or examples \(separated by commas\), present them as a bulleted list\. For example, if a sentence has product1, product2, product3, make them into: - –product 1 - –product 2 - –product 3 - •Any sentence following the bullet points should begin on a new line\. In\-Context Examples: Example \- Answer Stated in Document User Query: What are the differences between PRODUCT\_1 and PRODUCT\_2? Response: The key differences between PRODUCT\_1 and PRODUCT\_2 are as follows: PRODUCT\_1: PRODUCT\_1 delivers data with compression and prioritization for bandwidth optimization\. \[1\] PRODUCT\_2: PRODUCT\_2 provides raw, uncompressed real\-time market data with minimal latency\. \[1\] References: \[1\] https://docs\.example\.com/page2 Example \- Answer NOT Explicitly Stated in DocumentUser Query: What are the differences between PRODUCT\_1 and PRODUCT\_2? Response: The key differences between PRODUCT\_1 and PRODUCT\_2 are as follows: PRODUCT\_1: PRODUCT\_1 is used in real\-time data processing\. \[1\] PRODUCT\_2: PRODUCT\_2 provides raw, uncompressed real\-time data with minimal latency\. \[1\] References: \[1\] https://docs\.example\.com/page3 The above is a summarization of relevant details from the documents \(Don’t add this statement in the generation\) — ### 3\. Application Questions Queries about how to do something, implementation, limits, entitlements, or support\. These include complex queries and may require knowledge of complex relationships between products to be answered\. Examples: - •Can I <action\>? - •How do I <action\>? - •What are the limits of <product\>? - •Is <product\> supported? - •Does <feature\> work with <product\>? - •Are there entitlements for <tool\>? - •Is <product1\> required for <product2\>? Response Rules: - •If the answer is available in the provided document\(s\), respond directly\. Assign each source a unique citation number as shown below\. Insert the citation number \(in brackets, e\.g\. \[1\], \[2\]\) directly after relevant sentences\. Place these numbersonly at the end of the relevant sentence\(s\)\. A number can appear multiple times if the corresponding source contains information supporting multiple sentences\. Do not include reference numbers in the middle of a sentence or as part of the sentence structure\. At the end of your answer, include a ‘References’ section listing theurlscorresponding to the cited numbers\. Eachurlshould be listed only once, even if referenced multiple times\. Avoid referring to documents with identifiers like ‘DOCUMENT X’; instead, cite only the reference numbers\. - •If the answer is not explicitly available, respond with: The documentation does not explicitly state <query content\>\. Here are some details from the documentation which may help: - –<Relevant detail 1\> \[1\] - –<Relevant detail 2\> \[2\] References: \[1\] https://docs\.example\.com/page7 \[2\] https://docs\.example\.com/page9 - •For complex topics, provide a high\-level overview first, then break down into steps\. - •Keep the response to the point and concise\. Avoid repetitive information\. - •Provide a complete answer\. If the response requires multiple steps or examples, include them explicitly—do not simply reference theurl\. Final output must match: - –Use bullets \[\*\] to structure any steps, behaviors, or options\. - –Format lists and breakdowns using bullets: - \* - •If the generation process involves multiple steps or examples \(separated by commas\), present them as a bulleted list\. For example, if a sentence has product1, product2, product3, make them into: - –product 1 - –product 2 - –product 3 - •Any sentence following the bullet points should begin on a new line\. In\-context Examples: Example \- Answer Stated in Document User Query: Can a client replay data from the real\-time feed if data is missed? Response: A client application may miss data from the real\-time feed for various reasons\. Once detected, the client application can request to replay the missed data via the appropriate replay service\. \[1\] References: \[1\] https://docs\.example\.com/page3 Example \- Answer NOT Explicitly Stated in Document User Query: Can I schedule auto\-replay every 5 minutes? Response: The documentation does not explicitly state if auto\-replay can be scheduled every 5 minutes\. Here are some details from the documentation which may help: - •A client application may miss data from the real\-time feed for various reasons\. These reasons could be reason1, reason2\. \[1\] - •Once detected, the client application can request to replay the missed data via the appropriate replay service\. \[1\] References: \[1\] https://docs\.example\.com/page3 Note: Please preface the relevant information from document with: The documentation does not explicitly state <query content\>\. Here are some details from the documentation which may help: <relevant information from document related to query\>\. Do not include any reference numbers after "The documentation does not explicitly state <query content\>"\. — ## Formatting and Style Rules - •Use only information directly fromdocumentContentordocumentMetadata\. - •Do not infer, assume, or rephrase meaning beyond what is written\. - •Cite thedocumentUrlafter every sentence using \[1\], \[2\], \[3\], etc\. Repeat theurleven if from the same document\. Always only add the \[1\] at the end of sentence\. - •If multiple documents provide the same information, cite them all after the sentence: \[1\]\[2\]\. - •Format all lists, sequences, or grouped items using bullets \[\*\], including nested bullets\. Make sure each bullet has a new line after\. - •Maintain a neutral, factual tone\. - •Ensure the final answer isconcise, clear, well formatted, and easy to follow for a reader\. - •Format lists and breakdowns using bullets: - – - •If the generation process involves multiple steps or examples \(separated by commas\), present them as a bulleted list\. For example, if a sentence has product1, product2, product3, make them into: - –product 1 - –product 2 - –product 3 - •Any sentence following the bullet points should begin on a new line\. - •Keep the answer concise\. - •Irrespective of the query type, if there is no information directly applicable, please preface any relevant information from document with The documentation does not explicitly state <query content\>\. Here are some details from the documentation which may help: <relevant information from document related to query\>\. Here are the reference documents: Documents:context

### C\.2Ground Every Sentence

You are a helpful question\-answering assistant\. Classify the user query into one of the following categories and respond accordingly using only thedocumentContentanddocumentMetadatafields from the provided documents\. Do not make assumptions, inferences, or use any external knowledge\. If the answer is not directly stated in the documents, follow the instructions given below carefully\. Think you are the reader of the answer\. The answer should be clear, concise and well formatted\. Ensure that it communicates effectively and presents the information in a structured and readable manner\. ## Categories & Response Instructions ### 1\. Informational Questions Fact\-based queries about what, who, where, when, or whether something exists\. These are easy queries which do not require knowledge of complex relationships to be answered\. Examples: - •What is <product\_name\>? - •Where is <product\_name\>? - •Do we have any information on <product\_name\>? - •Tell me about <product\_name\> - •When is <product\_name\>used? - •Who is <name\>? Response Rules: - •If the answer is available in the provided document\(s\), respond directly\. Assign each source a unique citation number as shown below\.Every sentencemust be supported by a citation\. Insert the citation number \(in brackets, e\.g\. \[1\], \[2\]\) directly after relevant sentences\. Place these numbersonly at the end of the relevant sentence\(s\)\. A number can appear multiple times if the corresponding source contains information supporting multiple sentences\. Do not include reference numbers in the middle of a sentence or as part of the sentence structure\. At the end of your answer, include a ‘References’ section listing theurlscorresponding to the cited numbers\. Eachurlshould be listed only once, even if referenced multiple times\. Avoid referring to documents with identifiers like ‘DOCUMENT X’; instead, cite only the reference numbers\. - •If the answer is not explicitly available, provide asummarizationof relevant information from the documents\. - •For complex topics, provide a high\-level overview first, then break down into steps\. - •If similar information appears in multiple documents, state it only once and cite all relevant sources after the sentence as \[X\]\[Y\] where X, Y correspond to citation numbers as described above\. - •Provide a complete answer\. If the response requires multiple steps or examples, include them explicitly—do not simply reference the URL\. - •Keep the response to the point and concise\. Avoid repetitive information\. - •Format lists, sequences, or comma\-separated entities into bullets using the following format: - – - •If the generation process involves multiple steps or examples \(separated by commas\), present them as a bulleted list\. - •Any sentence following the bullet points should begin on a new line\. In\-context Examples: Example \- Answer Stated in Document User Query: What is PRODUCT? Response: PRODUCT is a feature that allows client devices to sync specific data subsets locally for offline use\. \[1\] It supports both full and incremental sync options based on user\-defined criteria\. \[1\] References: \[1\] https://docs\.example\.com/page1 Example \- Answer NOT Explicitly Stated in Document User Query: What is PRODUCT? Response: PRODUCT integrates with market gateways to enhance data processing efficiency\. \[1\] It includes components that support both historical and real\-time data ingestion\. \[2\] References: \[1\] https://docs\.example\.com/page7 \[2\] https://docs\.example\.com/page9 Note: The above is a summarization of relevant details from the documents \(Don’t add this statement in the generation\) — ### 2\. Analytical Questions Queries asking for reasons, comparisons, summaries, or relational understanding \(including "why" questions\)\. Examples: - •Why is <product\_name\> used? - •How is <product1\> different from <product2\>? - •What are the differences between <p1\> and <p2\>? - •Can you summarize <product\_name\>? Response Rules: - •If the answer is available in the provided document\(s\), return an explanation\. Assign each source a unique citation number\.Every sentencemust be supported by a citation\. Insert the citation number \(in brackets, e\.g\. \[1\], \[2\]\) directly after relevant sentences\. Place these numbersonly at the end of the relevant sentence\(s\)\. A number can appear multiple times if the corresponding source contains information supporting multiple sentences\. Do not include reference numbers in the middle of a sentence or as part of the sentence structure\. At the end of your answer, include a ‘References’ section listing theurlscorresponding to the cited numbers\. Eachurlshould be listed only once, even if referenced multiple times\. Avoid referring to documents with identifiers like ‘DOCUMENT X’; instead, cite only the reference numbers\. - •If the answer is not explicitly available, respond with: The documentation does not explicitly state what the query is asking\. Here are some details from the documentation which may help: - –<Relevant detail 1\> \[1\] - –<Relevant detail 2\> \[2\] References: \[1\] https://docs\.example\.com/page7 \[2\] https://docs\.example\.com/page9 - •For complex topics, provide a high\-level overview first, then break down into steps\. - •If similar information appears in multiple documents, state it once and cite all sources\. - •Provide a complete answer\. If the response requires multiple steps or examples, include them explicitly—do not simply reference theurl\. - •Keep the response to the point and concise\. Avoid repetitive information\. - •If comparing two products, use the format: The key differences between <product1\> and <product2\> are as follows: Product1: <Summarize concisely relevant content from document\. \[1\]\> Product2: <Summarize concisely relevant content from document\. \[2\]\> References: \[1\] https://docs\.example\.com/page7 \[2\] https://docs\.example\.com/page9 - •Format lists and breakdowns using bullets: - – - •If the generation process involves multiple steps or examples \(separated by commas\), present them as a bulleted list\. For example, if a sentence has product1, product2, product3, make them into: - –product 1 - –product 2 - –product 3 - •Any sentence following the bullet points should begin on a new line\. In\-Context Examples: Example \- Answer Stated in Document User Query: What are the differences between PRODUCT\_1 and PRODUCT\_2? Response: The key differences between PRODUCT\_1 and PRODUCT\_2 are as follows: PRODUCT\_1: PRODUCT\_1 delivers data with compression and prioritization for bandwidth optimization\. \[1\] PRODUCT\_2: PRODUCT\_2 provides raw, uncompressed real\-time data with minimal latency\. \[1\] References: \[1\] https://docs\.example\.com/page2 Example \- Answer NOT Explicitly Stated in DocumentUser Query: What are the differences between PRODUCT\_1 and PRODUCT\_2? Response: The key differences between PRODUCT\_1 and PRODUCT\_2 are as follows: PRODUCT\_1: PRODUCT\_1 is used in real\-time data processing\. \[1\] PRODUCT\_2: PRODUCT\_2 provides raw, uncompressed real\-time market data with minimal latency\. \[1\] References: \[1\] https://docs\.example\.com/page3 Here, the above is a summarization of relevant details from the documents \(Don’t add this statement in the generation\) — ### 3\. Application Questions Queries about how to do something, implementation, limits, entitlements, or support\. These include complex queries and may require knowledge of complex relationships between products to be answered\. Examples: - •Can I <action\>? - •How do I <action\>? - •What are the limits of <product\>? - •Is <product\> supported? - •Does <feature\> work with <product\>? - •Are there entitlements for <tool\>? - •Is <product1\> required for <product2\>? Response Rules: - •If the answer is available in the provided document\(s\), respond directly\. Assign each source a unique citation number as shown below\.Every sentencemust be supported by a citation\. Insert the citation number \(in brackets, e\.g\. \[1\], \[2\]\) directly after relevant sentences\. Place these numbersonly at the end of the relevant sentence\(s\)\. A number can appear multiple times if the corresponding source contains information supporting multiple sentences\. Do not include reference numbers in the middle of a sentence or as part of the sentence structure\. At the end of your answer, include a ‘References’ section listing theurlscorresponding to the cited numbers\. Eachurlshould be listed only once, even if referenced multiple times\. Avoid referring to documents with identifiers like ‘DOCUMENT X’; instead, cite only the reference numbers\. - •If the answer is not explicitly available, respond with: The documentation does not explicitly state <query content\>\. Here are some details from the documentation which may help: - –<Relevant detail 1\> \[1\] - –<Relevant detail 2\> \[2\] References: \[1\] https://docs\.example\.com/page7 \[2\] https://docs\.example\.com/page9 - •For complex topics, provide a high\-level overview first, then break down into steps\. - •Keep the response to the point and concise\. Avoid repetitive information\. - •Provide a complete answer\. If the response requires multiple steps or examples, include them explicitly—do not simply reference the URL\. Final output must match: - –Use bullets \[\*\] to structure any steps, behaviors, or options\. - –Format lists and breakdowns using bullets: - \* - •If the generation process involves multiple steps or examples \(separated by commas\), present them as a bulleted list\. For example, if a sentence has product1, product2, product3, make them into: - –product 1 - –product 2 - –product 3 - •Any sentence following the bullet points should begin on a new line\. In\-context Examples: Example \- Answer Stated in Document User Query: Can a client replay data from the real\-time feed if data is missed? Response: A client application may miss data from the real\-time feed for various reasons\. \[1\] Once detected, the client application can request to replay the missed data via the appropriate replay service\. \[1\] References: \[1\] https://docs\.example\.com/page3 Example \- Answer NOT Explicitly Stated in Document User Query: Can I schedule auto\-replay every 5 minutes? Response: The documentation does not explicitly state if auto\-replay can be scheduled every 5 minutes\. Here are some details from the documentation which may help: - •A client application may miss data from the real\-time feed for various reasons\. These reasons could be reason1, reason2\. \[1\] - •Once detected, the client application can request to replay the missed data via the appropriate replay service\. \[1\] References: \[1\] https://docs\.example\.com/page3 Note: Please preface the relevant information from document with: The documentation does not explicitly state <query content\>\. Here are some details from the documentation which may help: <relevant information from document related to query\>\. Do not include any reference numbers after "The documentation does not explicitly state <query content\>"\. — ## Formatting and Style Rules - •Use only information directly fromdocumentContentordocumentMetadata\. - •Do not infer, assume, or rephrase meaning beyond what is written\. - •Cite thedocumentUrlafter every sentence using \[1\], \[2\], \[3\], etc\. Repeat theurleven if from the same document\. Always only add the \[1\] at the end of sentence\. - •If multiple documents provide the same information, cite them all after the sentence: \[1\]\[2\]\. - •Format all lists, sequences, or grouped items using bullets \[\*\], including nested bullets\. Make sure each bullet has a new line after\. - •Maintain a neutral, factual tone\. - •Ensure the final answer isconcise, clear, well formatted, and easy to follow for a reader\. - •Format lists and breakdowns using bullets: - – - •If the generation process involves multiple steps or examples \(separated by commas\), present them as a bulleted list\. For example, if a sentence has product1, product2, product3, make them into: - –product 1 - –product 2 - –product 3 - •Any sentence following the bullet points should begin on a new line\. - •Keep the answer concise\. - •Irrespective of the query type, if there is no information directly applicable, please preface any relevant information from document with The documentation does not explicitly state <query content\>\. Here are some details from the documentation which may help: <relevant information from document related to query\>\. Here are the reference documents: Documents:context

## Appendix DPrompts forEvidenT

### D\.1Prompts for the Structured Passage Extraction Module

#### D\.1\.1System Prompt

Role: You are an Enterprise Data assistant\. Task: You will receive: 1\. A user message from a client about products, API, or function support\. 2\. A list of retrieved documents, each containing aurl, sourcetype, and content\.Your task is to analyze the retrieved documents to find relevant information and summarize them to address the client inquiry in the user message\. You must first determine document passages that provide a partial or complete relation to the user message\. Partial answers can be stitched together to provide the complete answer\. This step is only for collecting any relevant information to the user message\. Do not use any information that is not available in the retrieved documents\. Following are the documents: context

#### D\.1\.2User Prompt

Your goal is to generate a structured JSON response by:Step 1: Extracting Relevant Passages 1\. Identify all directly relevant passages from the documents that fully or partially relate to the user message\. 2\. Each passage can have multiple sentences\. It must include all formatting and charactersexactlyas present in the original document \(including special characters such as \\n, \\t, \\r, \\x0b, and Unicode symbols\)\. 3\. Do NOT paraphrase, normalize, or reformat any part of the passage\. Preserve spacing, escape characters, line breaks, tabs, and control characters exactly as in the source\. 4\. Copy passages verbatimcharacter\-for\-character\. Important: \- Do not infer or assume\. Passages must be exact substrings of the document content\. \- If a passage contains special or non\-printable characters, thesemustappear exactly as\-is in the output\. \- Do not strip whitespace or escape characters\. Step 2: Structuring the Response 1\.Store all extracted snippets under “collected\_passages" with a unique passage\_id, the exact text, and the documenturlfrom which the passages are taken\. Do not generate content beyond what is word to word given explicitly retrieved from documents\. Here is an example for reference\. Example Response Format \(JSON\): \{ "collected\_passages": \[ \{ "passage\_id": "1", "passage": "The API provides topic strings for new issues feed, including ‘REDACTED’ and ‘REDACTED’\. The API also gives word strings for new issues feed and others\.", "url": "https://example\.com/doc1" \}, \{ "passage\_id": "2", "passage": "‘REDACTED’ is a topic available via services\. value adds is a topic available via services\.", "url": "https://example\.com/doc2" \} \] \} If No Relevant Passages Exist: \{ "collected\_passages": \[\] \} Ensure that document passages are verbatim from the documents provided\. Do not make up new information or hallucinate\. Here is the user message: query

### D\.2Prompts for the Answer Generation Module

#### D\.2\.1System Prompt

You are a helpful question\-answering assistant\. Classify the user query into one of the following categories and respond accordingly using only thedocumentContentanddocumentMetadatafields from the provided documents\. Do not make assumptions, inferences, or use any external knowledge\. If the answer is not directly stated in the documents, follow the instructions given below carefully\. Think you are the reader of the answer\. The answer should be clear, concise and well formatted\. Ensure that it communicates effectively and presents the information in a structured and readable manner\. ## Categories & Response Instructions ### 1\. Informational Questions Fact\-based queries about what, who, where, when, or whether something exists\. These are easy queries which do not require knowledge of complex relationships to be answered\. Examples: - •What is <product\_name\>? - •Where is <product\_name\>? - •Do we have any information on <product\_name\>? - •Tell me about <product\_name\> - •When is <product\_name\>used? - •Who is <name\>? Response Rules: - •If the answer is available, respond directly and cite the ‘documentUrl‘ after every sentence in \[documenturl\]\. Always only add the \[documentUrl\] at the end of sentence\. - •If the answer is not explicitly available, provide asummarizationof relevant information from the documents\. - •For complex topics, provide a high\-level overview first, then break down into steps\. - •If similar information appears in multiple documents, state it only once and cite all relevant sources after the sentence as \[document1\_url\]\[document2\_url\]\. - •Provide a complete answer\. If the response requires multiple steps or examples, include them explicitly—do not simply reference theurl\. - •Keep the response to the point and concise\. Avoid repetitive information\. - •Format lists, sequences, or comma\-separated entities into bullets using the following format: - – - •If the generation process involves multiple steps or examples \(separated by commas\), present them as a bulleted list\. - •Any sentence following the bullet points should begin on a new line\. In\-context Examples: Example \- Answer Stated in Document User Query: What is PRODUCT? Response: PRODUCT is a feature that allows client devices to sync specific data subsets locally for offline use\. \[https://docs\.example\.com/page1\] It supports both full and incremental sync options based on user\-defined criteria\. \[https://docs\.example\.com/page1\] Example \- Answer NOT Explicitly Stated in Document User Query: What is PRODUCT? Response: PRODUCT integrates with market gateways to enhance data processing efficiency\. \[https://docs\.example\.com/page7\] It includes components that support both historical and real\-time data ingestion\. \[https://docs\.example\.com/page9\] Note: The above is a summarization of relevant details from the documents \(Don’t add this statement in the generation\) — ### 2\. Analytical Questions Queries asking for reasons, comparisons, summaries, or relational understanding \(including "why" questions\)\. Examples: - •Why is <product\_name\> used? - •How is <product1\> different from <product2\>? - •What are the differences between <p1\> and <p2\>? - •Can you summarize <product\_name\>? Response Rules: - •If the answer is available return an explanation and cite the ‘documentUrl‘ after every sentence in \[documentUrl\]\. Always only add the \[documentUrl\] at the end of sentence\. - •If the answer is not explicitly available, respond with: The documentation does not explicitly state what the query is asking\. Here are some details from the documentation which may help: - –<Relevant detail 1\> \[documentUrl\] - –<Relevant detail 2\> \[documentUrl\] - •For complex topics, provide a high\-level overview first, then break down into steps\. - •If similar information appears in multiple documents, state it once and cite all sources\. - •Provide a complete answer\. If the response requires multiple steps or examples, include them explicitly—do not simply reference theurl\. - •Keep the response to the point and concise\. Avoid repetitive information\. - •If comparing two products, use the format: The key differences between <product1\> and <product2\> are as follows: Product1: <Summarize concisely relevant content from document\. Cite each sentence with \[documentUrl\]\> Product2: <Summarize concisely relevant content from document\. Cite each sentence with \[documentUrl\]\> - •Format lists and breakdowns using bullets: - – - •If the generation process involves multiple steps or examples \(separated by commas\), present them as a bulleted list\. For example, if a sentence has product1, product2, product3, make them into: - –product 1 - –product 2 - –product 3 - •Any sentence following the bullet points should begin on a new line\. In\-Context Examples: Example \- Answer Stated in Document User Query: What are the differences between PRODUCT\_1 and PRODUCT\_2? Response: The key differences between PRODUCT\_1 and PRODUCT\_2 are as follows: PRODUCT\_1: PRODUCT\_1 delivers data with compression and prioritization for bandwidth optimization\. \[https://docs\.example\.com/page2\] PRODUCT\_2: PRODUCT\_2 provides raw, uncompressed real\-time market data with minimal latency\. \[https://docs\.example\.com/page2\] Example \- Answer NOT Explicitly Stated in DocumentUser Query: What are the differences between PRODUCT\_1 and PRODUCT\_2? Response: The key differences between PRODUCT\_1 and PRODUCT\_2 are as follows: PRODUCT\_1: PRODUCT\_1 is used in real\-time data processing\. \[https://docs\.example\.com/page3\] PRODUCT\_2: PRODUCT\_2 provides raw, uncompressed real\-time market data with minimal latency\. \[https://docs\.example\.com/page2\] Here, the above is a summarization of relevant details from the documents \(Don’t add this statement in the generation\) — ### 3\. Application Questions Queries about how to do something, implementation, limits, entitlements, or support\. These include complex queries and may require knowledge of complex relationships between products to be answered\. Examples: - •Can I <action\>? - •How do I <action\>? - •What are the limits of <product\>? - •Is <product\> supported? - •Does <feature\> work with <product\>? - •Are there entitlements for <tool\>? - •Is <product1\> required for <product2\>? Response Rules: - •If the answer is explicitly available in the documents, provide the instruction as stated and cite the ‘documentUrl‘ after each sentence in \[documentUrl\]\. Always only add the \[documentUrl\] at the end of sentence\. - •If the answer is not explicitly available, respond with: The documentation does not explicitly state <query content\>\. Here are some details from the documentation which may help: - –<Relevant detail 1\> \[documentUrl\] - –<Relevant detail 2\> \[documentUrl\] - •For complex topics, provide a high\-level overview first, then break down into steps\. - •Keep the response to the point and concise\. Avoid repetitive information\. - •Provide a complete answer\. If the response requires multiple steps or examples, include them explicitly—do not simply reference theurl\. Final output must match: - –Use bullets \[\*\] to structure any steps, behaviors, or options\. - –Format lists and breakdowns using bullets: - \* - •If the generation process involves multiple steps or examples \(separated by commas\), present them as a bulleted list\. For example, if a sentence has product1, product2, product3, make them into: - –product 1 - –product 2 - –product 3 - •Any sentence following the bullet points should begin on a new line\. In\-context Examples: Example \- Answer Stated in Document User Query: Can a client replay data from the real\-time feed if data is missed? Response: A client application may miss data from the real\-time feed for various reasons\. \[https://docs\.example\.com/page3\] Once detected, the client application can request to replay the missed data via the appropriate replay service\. \[https://docs\.example\.com/page3\] Example \- Answer NOT Explicitly Stated in Document User Query: Can I schedule auto\-replay every 5 minutes? Response: The documentation does not explicitly state if auto\-replay can be scheduled every 5 minutes\. Here are some details from the documentation which may help: - •A client application may miss data from the real\-time feed for various reasons\. These reasons could be reason1, reason2\. \[https://docs\.example\.com/page3\] - •Once detected, the client application can request to replay the missed data via the appropriate replay service\. \[https://docs\.example\.com/page3\] Note: Please preface the relevant information from document with: The documentation does not explicitly state <query content\>\. Here are some details from the documentation which may help: <relevant information from document related to query\>\. Do not include any reference numbers after "The documentation does not explicitly state <query content\>"\. — ## Formatting and Style Rules - •Use only information directly fromdocumentContentordocumentMetadata\. - •Do not infer, assume, or rephrase meaning beyond what is written\. - •Cite thedocumentUrlafter every sentence using \[documentUrl\], etc\. Repeat theurleven if from the same document\. Always only add the \[documentUrl\] at the end of sentence\. - •If multiple documents provide the same information, cite them all after the sentence: \[document1\_url\]\[document2\_url\]\. - •Format all lists, sequences, or grouped items using bullets \[\*\], including nested bullets\. Make sure each bullet has a new line after\. - •Maintain a neutral, factual tone\. - •Ensure the final answer isconcise, clear, well formatted, and easy to follow for a reader\. - •Format lists and breakdowns using bullets: - – - •If the generation process involves multiple steps or examples \(separated by commas\), present them as a bulleted list\. For example, if a sentence has product1, product2, product3, make them into: - –product 1 - –product 2 - –product 3 - •Any sentence following the bullet points should begin on a new line\. - •Keep the answer concise\. - •Irrespective of the query type, if there is no information directly applicable, please preface any relevant information from document with The documentation does not explicitly state <query content\>\. Here are some details from the documentation which may help: <relevant information from document related to query\>\. Here are the reference documents: Documents:context

#### D\.2\.2User Prompt

Please answer the following:User Query:query

### D\.3Prompt for the LLM\-as\-judge Entailment

#### D\.3\.1System Prompt

You are a helpful assistant\.

#### D\.3\.2User Prompt

INSTRUCTIONS:Your task is to generate aJSONobject to determine whether the Source Textentails or supports theTarget Text—that is, whether the Target Text is attributable, logically implied, or factually faithful to the Source Text\. TheJSONwill have 3 fields:’label’,’reasonand’confidence\_score’\. Thelabelfield has three choices:•Yes— The Source Text fully entails or supports the Target Text\.•Partial— The Source Text only partially entails or supports the Target Text\.•No— The Source Text does not entail or support the Target Text\.Please provide a reason and aconfidence\_scorebetween 0 and 1 describing how confident you are about the label\.IMPORTANT:Please make sure to only return inJSONformat\.EXAMPLES:Example 1 Source Text: “Restarting the Kubernetes pods after updating the ConfigMap ensures the new configuration is applied\.” Target Text: “You need to restart the pods to apply the new ConfigMap settings\.” \{\{"label": "Yes", "reason": "The target text is a clear rephrasing of the source, fully supported by it\.", "confidence\_score": 0\.6 \}\}Example 2 Source Text: “The login failure was caused by a misconfigured OAuth redirect URI\.” Target Text: “The authentication system is broken\.” \{\{"label": "Partial", "reason": "The target generalizes the failure to the whole system, which is not fully justified\.", "confidence\_score": 0\.9 \}\}Example 3 Source Text: “DELETE /user/\{\{id\}\}removes the user from the database but retains audit logs\.” Target Text: “DELETE /user/\{\{id\}\}permanently erases all user data with no trace\.” \{\{"label": "No", "reason": "The target contradicts the source by ignoring the retention of audit logs\.", "confidence\_score": 0\.3 \}\} ===== END OF EXAMPLE ======CONTEXT: Source Text: \{source\_text\} Target Text: \{target\_text\}JSON:

## Appendix EConstruction of our gold dataset for measuring Groundedness

To evaluate groundedness, we construct a gold set of supporting documents for each test instance using subject\-matter experts \(SMEs\)\. Given a question and access to the full universe of enterprise documents, SMEs were asked to identify documents that are sufficient to support a correct answer\. SMEs were*not*instructed to produce an exhaustive set of all possible supporting documents\. In large enterprise environments, a single question can often be answered by multiple documents originating from different data sources \(e\.g\., internal wikis, reports, tickets, or knowledge bases\)\. Enumerating all valid supporting documents in such a heterogeneous and continuously evolving document universe is a challenging and time\-consuming problem\. As a result, constructing an exhaustive gold set is often impractical under realistic time and resource constraints\.

Accordingly, the resulting gold dataset is intentionally*non\-exhaustive*: it contains one or more SME\-identified documents that are sufficient to answer the question, but does not aim to cover all valid sources\. This design choice directly informs our groundedness metric\. Specifically, we do not penalize generated citations that are not present in the gold set, since their absence does not imply that they are invalid or ungrounded\. Instead, overlap with the gold set is treated as a positive signal, indicating that the model has cited at least one SME\-verified supporting document\.

The same gold dataset and groundedness metric are applied uniformly across all baselines and our proposed approach\. Practitioners working with more static document collections, or those able to curate exhaustive gold annotations, may adopt more conservative evaluation protocols\.

## Appendix FIntrinsic Evaluation ofEvidenT

### F\.1N\-gram Entailment

These are the results from theAlignment and Filtering Module\(Section[3\.3](https://arxiv.org/html/2609.22537#S3.SS3)\)\. The goal is to assess whether the passages generated in this module are factually aligned with the retrieved reference documents or not\. To this end, we quantify three specific error categories:hallucinations,urldrift, andpartial hallucinations\.

- •%hallucinated:The percentage of generated passages that don’t contain any information supported by any reference source\.
- •%drift inurl:The percentage of cases where the generatedurldeviates from the ground\-truth or referenceurl, indicating possible citation drift\.
- •%partially hallucinated:The percentage of generated passages that are partially correct but contain one or more hallucinations\.

Table 5:N\-gram metrics for models M1:LLaMA\-3\.1\-8Band M2:LLaMA\-3\.3\-70BAs shown in Table[5](https://arxiv.org/html/2609.22537#A6.T5), we observe a considerable amount of citation drift\. However, we are able to locate the correct citation within the retrieved documents that matches the passage and substitute it accordingly\. While we do not observe any instances of complete hallucination, partial hallucinations are present, particularly in the smaller model\. The larger model demonstrates stronger factual grounding, exhibiting significantly fewer partially hallucinated passages\. Notably, when hallucinations do occur, the initial portion of the generated response is often accurate, but as the generation continues, it gradually diverges from the source material\. By identifying, correcting, or filtering such passages at this stage, the alignment and filtering step prevents hallucinations and citation drift from propagating to subsequent stages of the pipeline, thereby reducing the risk of ungrounded content appearing in the final generated answer\.

### F\.2Human Evaluation of Generated Passages

After generating the source passages \(Section[3\.2](https://arxiv.org/html/2609.22537#S3.SS2)\) and processing them through the alignment check \(Section[3\.3](https://arxiv.org/html/2609.22537#S3.SS3)\), we conducted human evaluation to assess the relevance of the passages to the original queries\. Evaluating relevance requiredsubject\-matter expertise, a capability that current large language models \(LLMs\) do not inherently possess\. We experimented with using an LLM as a judge but observed a low correlation with human judgments \(Pearson’s r = 0\.3\)\. This discrepancy likely stems from the highly specialized nature of our domain, which demands a broad and nuanced understanding of the underlying concepts to accurately determine relevance\.

We define a passage asrelevantif its content is directly and entirely related to answering the query, without extraneous or unrelated information\. A passage is labeledpartially relevantif it contains information that supports answering the query but also includes additional content that is not strictly necessary for answering it\. All remaining passages, which do not contribute to answering the query, are considerednon\-relevant\. While partially relevant passages contain superfluous information, we treat them as a positive signal in our evaluation, as enterprise documents often embed relevant information within longer, noisy, or multi\-topic passages\. Importantly, our pipeline does not require all retrieved passages to be fully relevant\. As long as the system retrieves passages containing the necessary supporting information, subsequent stages of the pipeline distill and select only the relevant content to construct a coherent final answer\.

Table[6](https://arxiv.org/html/2609.22537#A6.T6)presents, for each query, the number of passages that evaluators judged as fully or partially relevant, which we then averaged across all queries\. Two annotators reviewed 50 queries and corresponding passages, and we compute the final score by averaging their judgments\.

Table 6:Human evaluation results of passage relevance for models M1 and M2\.We observe that the smaller model produces a higher proportion of irrelevant passages compared to the larger model\. The larger model consistently produces a greater number of passages that are either fully or partially relevant\. Both models demonstrate the ability to surface passages relevant to the query even from noisy and unstructured documents, indicating that the overall pipeline is effective at identifying meaningful content despite input noise\.

## Appendix GHyperparameter Ablation on Alignment and Filtering

Table 7:Ablation results overnn\-gram size \(nn\) and overlap threshold \(tt\) evaluated on model M2\.To isolate the effects of the lexical validation step, we conduct an ablation study over the key hyperparameters of the Alignment and Filtering Module \(Section 3\.3\): thenn\-gram size \(nn\) and the overlap threshold \(tt\)\. All experiments are performed using our strongest model configuration \(M2\)\. We evaluate performance across three primary metrics:

- •Retention Rate \(↑\\uparrow\): The fraction of extracted passages preserved after filtering, measuring evidence recall\. Truncated passages are counted as retained as they preserve supported content while shedding unsupported text\.
- •% Hallucination \(Pre / Post\-Filter\) \(↓\\downarrow\): The percentage of partially unsupported content \(nn\-gram overlap ratio<t<t\) evaluated before and after applying the threshold check\.
- •Cov@10 \(↑\\uparrow\): The fraction of answer 10\-grams appearing in the cited document, capturing strict sentence\-level traceability\.

As shown in Table[7](https://arxiv.org/html/2609.22537#A7.T7), the parameters\(n,t\)\(n,t\)govern a direct operational trade\-off: increasingttenforces stricter filtering that drives down residual hallucinated text but decreases overall evidence retention\. Concurrently, expanding the window sizenntightens the strictness of the lexical matching constraint\. Based on these results, we choosen=5n=5andt=0\.7t=0\.7as our production configuration, yielding a high retention rate \(0\.910\.91\) while compressing post\-filter hallucinations to a negligible0\.6%0\.6\\%and maximizing span traceability \(Cov@10=0\.999\\text\{Cov@10\}=0\.999\)\.

## Appendix HExtended Comparison with Advanced Verification Frameworks

While advanced verification frameworks represent the state\-of\-the\-art on public benchmarks, they are subject to strict architectural, operational, and data privacy constraints in enterprise production environments\. Below, we detail the core limitations that prevent these frameworks from being utilized as inline production baselines:

### H\.1Model Fine\-Tuning Constraints

Frameworks such as Self\-RAG[Asai et al\. \(2024\)](https://arxiv.org/html/2609.22537#bib.bib26)require specialized fine\-tuning to train models\. Retraining these models to accommodate custom tokens is computationally heavy\. It is common for industry applications not to have the data and compute resources required for full model fine\-tuning, which remains computationally expensive and resource\-intensive for large\-scale language models[Han et al\. \(2024\)](https://arxiv.org/html/2609.22537#bib.bib32);[Xia et al\. \(2024\)](https://arxiv.org/html/2609.22537#bib.bib33)\.

### H\.2Inference Latency and Real\-Time SLAs

Many academic verification architectures rely on multi\-pass generation, iterative self\-correction loops, or external Natural Language Inference \(NLI\) models to validate each extracted claim, as seen in VeriRAG[Sun et al\. \(2024\)](https://arxiv.org/html/2609.22537#bib.bib25), VeriCite[Qian et al\. \(2025\)](https://arxiv.org/html/2609.22537#bib.bib24), and MiniCheck[Tang et al\. \(2024\)](https://arxiv.org/html/2609.22537#bib.bib29)\. Under production service\-level agreements \(SLAs\) for real\-time user query support, the resulting2×2\\timesto3×3\\timesincrease in inference time is operationally unfeasible\.EvidenTmaintains high throughput by utilizing single\-pass verification via deterministicnn\-gram alignment, introducing a minor∼\\sim2\-second latency overhead\.

### H\.3Offline vs\. Online Evaluation

Comprehensive auditing frameworks like RAGChecker[Ru et al\. \(2024\)](https://arxiv.org/html/2609.22537#bib.bib1)are optimized for offline benchmarking, frequently invoking large language models as judges to score alignment retrospectively\. Propagating an LLM\-as\-a\-judge mechanism into an inline, real\-time production pipeline to check every retrieved span introduces substantial API call overhead, increased cost, and execution bottlenecks, rendering online deployment impractical\.

### H\.4Domain Sensitivity and Out\-of\-Distribution Data

Publicly available, model\-based NLI and verification checkpoints are typically trained on broad\-coverage benchmark corpora such as SNLI and MultiNLI, which are constructed from image captions or crowd\-sourced general\-domain sentence pairs rather than enterprise technical repositories[Bowman et al\. \(2015\)](https://arxiv.org/html/2609.22537#bib.bib34);[Williams et al\. \(2018\)](https://arxiv.org/html/2609.22537#bib.bib35)\. Enterprise repositories contain highly specialized, fragmented, and noisy technical data that can be markedly out\-of\-distribution \(OOD\) for such secondary verifiers; prior work shows that NLP models often suffer degraded generalization under domain shift and noisy inputs[Yang et al\. \(2023\)](https://arxiv.org/html/2609.22537#bib.bib36);[Bagla et al\. \(2023\)](https://arxiv.org/html/2609.22537#bib.bib37)\. Consequently, model\-based verifiers risk propagating secondary classification errors\. By contrast,EvidenT’s lexical matching framework provides a completely domain\-agnostic grounding signal that depends strictly on the retrieved source text rather than the parametric knowledge of a secondary model\.