TRACE: Trustworthy Retrieval-Augmented Conversational Engine

# TRACE: Trustworthy Retrieval-Augmented Conversational Engine
Source: [https://arxiv.org/html/2608.10176](https://arxiv.org/html/2608.10176)
,Laila CureWichita State UniversityWichitaKSUSAandSouvika SarkarWichita State UniversityWichitaKSUSA

###### Abstract\.

Public service chatbots are expected to deliver recommendations from an underlying public service directory, while also making sure that the recommendations respect explicit user constraints\. In practice, public service directories are noisy and inconsistent, and general\-purpose large language model \(LLM\) or AI\-based chatbots frequently generate unreliable recommendations, citing unverified sources from the web\. We investigate the impact of retrieval quality on constraint\-aware recommendation in public service conversational systems built over noisy and heterogeneous service directories\. We proposeTRACE\(TrustworthyRetrieval\-AugmentedConversationalEngine\), a retrieval\-based, constraint\-aware framework that parses input user queries into structural and semantic constraints for downstream retrieval, with the help of a dual data representation schema\. Using a curated statewide pantry directory and a synthetic query benchmark, we evaluate multiple knowledge\-representation variants with and without knowledge graphs \(KGs\)\. We experiment with several open\-source LLMs and a proprietary model, showing that strengthening retrieval substantially improves user constraint satisfaction while reducing hallucinated recommendations\. Performance differences across LLMs narrowed in our experiments as retrieval quality improved, making results less sensitive to model size\. These findings suggest that the quality of retrieval is key for robust public service conversational systems\.

Information retrieval, Large language models, Knowledge graphs, Retrieval\-augmented generation, Public service recommendation\.

## 1\.Introduction

Public service conversational systems are typically directory\-grounded systems, i\.e\., they answer user requests by retrieving candidate providers from an underlying public service directory\(Dowet al\.,[2018](https://arxiv.org/html/2608.10176#bib.bib10)\)\. These chatbots are expected to deliver reliable recommendations \(e\.g\., where to obtain food assistance\) from the directory while respecting explicit user constraints such as location, operating hours, and eligibility requirements\(Larsen and Følstad,[2024](https://arxiv.org/html/2608.10176#bib.bib14); Van Noordt and Misuraca,[2019](https://arxiv.org/html/2608.10176#bib.bib9)\)\. In practice, these constraints are difficult to satisfy because public service directories are often noisy and inconsistently formatted\(Dowet al\.,[2018](https://arxiv.org/html/2608.10176#bib.bib10); Cherry,[2002](https://arxiv.org/html/2608.10176#bib.bib11)\)\. As a result, general\-purpose large language model \(LLM\) or AI\-powered chatbots may generate unreliable recommendations by citing unverified sources or implicitly violating user constraints\(Dreylinget al\.,[2024](https://arxiv.org/html/2608.10176#bib.bib12)\)\. In preliminary trials with marketplace LLM chatbots \(e\.g\., ChatGPT\), out\-of\-directory recommendations and occasional reliance on unverified web sources were observed, particularly under explicit constraints\(Kleiman and Barbosa,[2025](https://arxiv.org/html/2608.10176#bib.bib13)\)\. These failures are especially costly in community\-help settings, where users rely on the system for reliable time\-sensitive guidance\.

These failures point to a fundamental gap: without reliable retrieval over an authoritative directory, even capable LLMs cannot deliver trustworthy recommendations in public service settings\. Our premise is that*generation quality in this domain is largely determined by retrieval quality*: if the system can reliably retrieve a small set of candidates that satisfy the user’s constraints, the LLM’s role is primarily to synthesize a response from the evidence than to “invent” new recommendations\. To operationalize this idea, we introduceTRACE\(TrustworthyRetrieval\-AugmentedConversationalEngine\), a constraint\-aware framework that relies on robust retrieval prior to LLM response generation\. We evaluate our framework on a real\-world food pantry dataset built from a curated directory\. We further develop a synthetic benchmark of constraint\-driven user queries with ground\-truth answers\. Our evaluation compares multiple knowledge representation variants \(with and without knowledge graphs\) and multiple LLMs under an identical retrieval setup, measuring constraint satisfaction and out\-of\-dataset hallucinations\. Results support the view that retrieval quality is the primary lever for dependable public service conversational access\. Through TRACE, our key contributions are as follows:

- •Dual\-representation, constraint\-aware retrieval framework\.A retrieval\-first pipeline that explicitly parses user constraints into: \(i\) structural constraints via knowledge graphs and \(ii\) semantic constraints via text\. This dual representation grounds LLM response generation in directory records and reduces out\-of\-directory recommendations\.
- •Knowledge graph \(KG\) structure ablation for conversational retrieval\.We systematically compare multiple knowledge graph representations with a no\-KG baseline and report retrieval and end\-to\-end reliability metrics\.
- •Retrieval\-centric robustness across LLMs\.Under a fixed retrieval pipeline, we evaluate 15 open\-source LLMs and one proprietary model, showing that stronger retrieval substantially narrows performance differences across LLMs \(regardless of size\) in the public service directory setting\.

## 2\.Related Work

![Refer to caption](https://arxiv.org/html/2608.10176v1/x1.png)

Figure 1\.Overview of the TRACE \(Trustworthy Retrieval\-Augmented Conversational Engine\) framework\.Constraint\-aware conversational systems\.Task\-oriented dialogue treats user requests as structured slots \(e\.g\., location, time, eligibility\) that must be filled to complete an information need\(Cohen,[2019](https://arxiv.org/html/2608.10176#bib.bib22); Louvan and Magnini,[2020](https://arxiv.org/html/2608.10176#bib.bib15)\)\. Recently, the conversational information retrieval \(IR\) community has emphasized that systems must \(i\) interpret underspecified queries\(Malaviyaet al\.,[2025](https://arxiv.org/html/2608.10176#bib.bib19)\), \(ii\) ask clarification questions when key constraints are missing\(Zamaniet al\.,[2020](https://arxiv.org/html/2608.10176#bib.bib20); Majumderet al\.,[2021](https://arxiv.org/html/2608.10176#bib.bib21)\), and \(iii\) maintain constraint consistency across the dialogue\(Gaoet al\.,[2020](https://arxiv.org/html/2608.10176#bib.bib16)\)\. Prior work on clarification question generation and conversational search demonstrates the value of explicitly identifying missing facets and eliciting them, rather than guessing or returning loosely related results\(Tavakoli,[2020](https://arxiv.org/html/2608.10176#bib.bib17); Sekulićet al\.,[2021](https://arxiv.org/html/2608.10176#bib.bib18)\)\.

Reliable public\-service question answering \(QA\)\.Public\-service assistants face a distinctive reliability requirement: answers should be grounded in a verified directory rather than generated from open\-web sources\(Hasanet al\.,[2026](https://arxiv.org/html/2608.10176#bib.bib40); Arends and Mawela,[2024](https://arxiv.org/html/2608.10176#bib.bib23); Stamatiset al\.,[2020](https://arxiv.org/html/2608.10176#bib.bib24)\)\. Document\-grounded dialogue benchmarks such asdoc2dial\(Fenget al\.,[2020](https://arxiv.org/html/2608.10176#bib.bib25)\)andMultiDoc2Dial\(Fenget al\.,[2021](https://arxiv.org/html/2608.10176#bib.bib26)\)formalize this setting by requiring systems to retrieve supporting evidence from documents to produce answers\. In parallel, recent work has shown that general\-purpose LLMs produce fluent but incorrect statements and confidently present unsupported claims, reinforcing the need for explicit grounding and verification\(Tripathiet al\.,[2025](https://arxiv.org/html/2608.10176#bib.bib28)\)\. These findings are especially relevant in community\-help scenarios, where unverified recommendations directly reduce trust and utility\(Aoki,[2020](https://arxiv.org/html/2608.10176#bib.bib27); Zhouet al\.,[2025](https://arxiv.org/html/2608.10176#bib.bib29)\)\.

Knowledge graphs \(KGs\) for retrieval\.KGs have been widely studied to organize entities and relations for search and QA\(Huanget al\.,[2019](https://arxiv.org/html/2608.10176#bib.bib30); Khan,[2023](https://arxiv.org/html/2608.10176#bib.bib31)\)\. In IR, entity\-centric representations and graph structures support precise filtering and reduce wrong\-entity retrieval\(Reinandaet al\.,[2020](https://arxiv.org/html/2608.10176#bib.bib32); Dietzet al\.,[2018](https://arxiv.org/html/2608.10176#bib.bib33)\)\. In QA, hybrid approaches combine graph traversal with text to handle both relational constraints and free\-text details\(Agarwalet al\.,[2025](https://arxiv.org/html/2608.10176#bib.bib34); Polignanoet al\.,[2021](https://arxiv.org/html/2608.10176#bib.bib35)\), and recent “graph\-enhanced” retrieval frameworks use graph structures to guide candidate selection before ranking evidence\(Liet al\.,[2025](https://arxiv.org/html/2608.10176#bib.bib36); Yuet al\.,[2026](https://arxiv.org/html/2608.10176#bib.bib37); Zhuet al\.,[2025](https://arxiv.org/html/2608.10176#bib.bib38); Xuet al\.,[2024](https://arxiv.org/html/2608.10176#bib.bib39)\)\.

Uniqueness of our work\.While prior work has studied \(i\) conversational systems that elicit missing slots and constraints, \(ii\) document\-grounded QA to reduce unsupported answers, and \(iii\) KG representations for retrieval and hybrid QA; they are rarely evaluated together in a public\-service directory setting where users pose explicit constraints and the cost of wrong recommendations is high\. Our work is distinct in three ways\.First, we frame public\-service assistance as constraint\-aware conversational retrieval and operationalize a dual\-constraint pipeline that enforces evidence retrieval and candidate checking\.Second, instead of assuming a fixed KG schema, we perform a KG\-structure ablation tailored to public\-service data, quantifying how representation choices affect retrieval quality\.Third, we study the hypothesis that retrieval dominates generation in this setting by evaluating multiple LLMs\.

## 3\.Proposed Framework

Illustrated in Figure[1](https://arxiv.org/html/2608.10176#S2.F1), we proposeTRACE\(TrustworthyRetrieval\-AugmentedConversationalEngine\), for public\-service conversational search\. Given a user query, TRACE \(a\) extracts structural constraint\(s\) and, if none are present, asks a clarification question; \(b\) retrieves a top\-kkcandidate set satisfying the constraints; \(c\) applies semantic filtering over the retrieved candidates, and \(d\) if no candidate satisfies the semantic constraints, fetches the next top\-kkcandidates\. Once semantic constraints are satisfied, TRACE generates the final response based on the retrieved candidate set\.

### 3\.1\.Parsing user constraints

The framework is designed around two types of constraints: \(i\) structural and \(ii\) semantic\. Given an input user queryqq, we extract:

- •Structural constraintsCsC\_\{s\}: attributes to be enforced exactly \(e\.g\., city/county/ZIP\)\. These constraints define the candidate space and are evaluated with strict AND semantics\. When no structural constraints are present inqq, the system asks a targeted clarification question \(e\.g\., “What is your nearby ZIP, city, or county?”\) to obtain the structural constraint\(s\)\.
- •Semantic constraintsCmC\_\{m\}: expressed in natural language that typically appear in unstructured text fields \(e\.g\., eligibility requirements\)\. These constraints are checked using retrieved evidence from text fields within the candidate set\.

Example\.For the queryqq: “Show me pantries in Sedgwick County that need ID,” we extract structural constraintsCs=\{county=Sedgwick County\}C\_\{s\}=\\\{\\texttt\{county\}=\\text\{Sedgwick County\}\\\}and semantic constraintsCm=\{need ID\}C\_\{m\}=\\\{\\text\{need ID\}\\\}\. We first retrieve candidates by querying the KG usingCsC\_\{s\}, then verifyCmC\_\{m\}using textual evidence from the candidates’eligibilityfields\.

### 3\.2\.Dual knowledge representation

We construct a dual representation of the public service directory:

1. \(1\)Knowledge graphs \(structural view\):Each service provider \(e\.g\., pantry\) is linked to a structured entity such as location \(city, county, ZIP\)\. We evaluate multiple knowledge graph \(KG\) variants as ablations against a no\-KG baseline\.
2. \(2\)Vector embeddings \(semantic view\):For each provider, we retain unstructured fields such as eligibility requirements\. These fields are embedded to support semantic retrieval\.

### 3\.3\.Retrieval and constraint checking

Given a user queryqq, the system proceeds in two stages:

#### Stage 1: Candidate generation \(structural filtering\)\.

UsingCsC\_\{s\}, we query the graph to retrieve a top\-kkset of candidates:

𝒫k=TopK​\(\{p∈𝒫∣p⊧Cs\}\),\\mathcal\{P\}\_\{k\}=\\mathrm\{TopK\}\\big\(\\\{p\\in\\mathcal\{P\}\\mid p\\models C\_\{s\}\\\}\\big\),wherep⊧Csp\\models C\_\{s\}denotes thatppsatisfies*all*structural constraints\.

Stage 2: Evidence\-based semantic filtering \(within retrieved candidates\)\.We filter retrieved candidates𝒫k\\mathcal\{P\}\_\{k\}with semantic constraints:

𝒫ksem=\{p∈𝒫k\|check​\(p,Cm\)=1\}\.\\mathcal\{P\}^\{\\text\{sem\}\}\_\{k\}\\;=\\;\\\{\\,p\\in\\mathcal\{P\}\_\{k\}\\;\|\\;\\mathrm\{check\}\(p,C\_\{m\}\)=1\\,\\\}\.Here,check​\(p,Cm\)\\mathrm\{check\}\(p,C\_\{m\}\)is a semantic constraint checker computed from evidence retrieved from the candidate’s text fields\. If𝒫ksem=∅\\mathcal\{P\}^\{\\text\{sem\}\}\_\{k\}=\\emptyset, we fetch the next top\-kkcandidates and repeat*Stage 2*\.

## 4\.Experimental Setup

### 4\.1\.Dataset curation

We evaluate on a curated food pantry directory containing∼\\sim800 pantries with structured fields \(name, address, city, county, ZIP code, phone, hours\) and unstructured fields \(eligibility\)\. We use a publicly available Kansas directory111Kansas Food Source\. Find Food in the Sunflower State\.[https://kansasfoodsource\.org/](https://kansasfoodsource.org/)because it provides statewide coverage and access to verified pantry listings\. This pipeline applies to any public\-service directory and can be extended to other domains \(e\.g\., clinics, shelters, transportation services\)\. For evaluation, this directory is assumed to be the only desired source of information for the purpose of our study, and any suggestions not included in this dataset are counted as out\-of\-dataset hallucinations\.

### 4\.2\.Knowledge graph variants

To study how knowledge representation via knowledge graphs \(KGs\) affects retrieval, we compare the following:

- •KG\-0 \(No\-KG Baseline\):text\-only indexing over pantries\.
- •KG\-1 \(Location KG\):a location\-modeled graph linkingPantrynodes toCity/County/Zipcode\.
- •KG\-2 \(Hours KG\):a graph linkingPantrytoHours\.
- •KG\-3 \(Location \+ Hours KG\):a combined graph including both location and hours relations \(KG\-1 \+ KG\-2\)\.

### 4\.3\.Query benchmark and ground truth

We construct a synthetic benchmark of 1,000 user queries to reflect common public\-service information needs\. Queries are grouped into five families \(see Table[1](https://arxiv.org/html/2608.10176#S4.T1)\) covering: \(i\) location\-only requests \(city/county/ZIP\), \(ii\) eligibility questions \(ID\), \(iii\) open\-hours queries \(day/time\), \(iv\) recall\-style queries that mention a specific pantry name, and \(v\) queries combining location with eligibility and/or hours constraints\. For evaluation, we derive directory\-grounded “gold” answers by \(1\) applying strict structural filtering over the pantry directory for the structural constraints in the query and \(2\) returning the top candidates as the reference context\.

Table 1\.Distribution of benchmarked queries across families\.FamilyLocationEligibilityHoursRecallCombinedQueries5002002001090
### 4\.4\.Models evaluated

We evaluate 15 open\-source instruction\-tuned LLMs across diverse model families like Qwen\(Yanget al\.,[2025](https://arxiv.org/html/2608.10176#bib.bib41)\), Llama\(Grattafioriet al\.,[2024](https://arxiv.org/html/2608.10176#bib.bib42)\), Gemma\(Teamet al\.,[2024](https://arxiv.org/html/2608.10176#bib.bib43)\), Phi\(Abdinet al\.,[2024](https://arxiv.org/html/2608.10176#bib.bib44)\), DeepSeek R1 Distill\(Guoet al\.,[2025](https://arxiv.org/html/2608.10176#bib.bib45)\), spanning small \(≤\\leq3B\), medium \(4–8B\), and large \(\>\>8B\) parameter size groups\. We also include one stronger proprietary baseline, GPT 5\.5\(Singhet al\.,[2025](https://arxiv.org/html/2608.10176#bib.bib46)\)\. All models are tested with the same queries mentioned in Table[1](https://arxiv.org/html/2608.10176#S4.T1)\.

### 4\.5\.Evaluation metrics

Retrieval quality\.We report Precision@k and Recall@k \(withk∈\{3,5\}k\\in\\\{3,5\\\}\) against the gold candidate set for each query\. We also report F1@k as the harmonic mean of Precision@k and Recall@k\(Manning,[2008](https://arxiv.org/html/2608.10176#bib.bib47)\)\.

Constraint satisfaction\.Fraction of queries where all explicitly stated constraints are satisfied by the returned recommendations\.

Hallucination rate\.Fraction of queries where the response includes at least one recommended pantry not present in the directory\.

Semantic similarity\.We measure cosine similarity between fixed sentence embeddings of the model\-generated responses and the directory\-grounded gold standard responses\(Hliaoutakiset al\.,[2009](https://arxiv.org/html/2608.10176#bib.bib48)\)\. Higher values indicate closer semantic alignment\.

## 5\.Results and Discussion

Model GroupModel NameConstraint Satisfaction \(%\)Hallucination Rate \(%\)Semantic SimilarityKG\-0KG\-1KG\-2KG\-3KG\-0KG\-1KG\-2KG\-3KG\-0KG\-1KG\-2KG\-3ProprietaryGPT\-5\.572\.1892\.0693\.2197\.945\.711\.631\.120\.610\.750\.840\.850\.87LargeLlama 3\.3 70B Instruct73\.5890\.0898\.1197\.635\.411\.920\.920\.970\.740\.830\.860\.85DeepSeek R1 Distill Qwen \(32B\)72\.3389\.0490\.1397\.025\.961\.881\.210\.960\.730\.810\.820\.84Gemma 4 31B Instruct71\.0995\.4494\.6795\.186\.100\.991\.061\.010\.720\.830\.820\.83DeepSeek R1 Distill Qwen \(14B\)70\.1286\.0387\.4494\.196\.712\.241\.581\.010\.710\.780\.790\.81MediumLlama 3\.1 8B Instruct66\.0883\.5285\.1191\.339\.012\.922\.111\.020\.670\.750\.760\.78DeepSeek R1 Distill Llama \(8B\)66\.5784\.0192\.0891\.228\.632\.830\.981\.050\.680\.760\.790\.78Qwen 3 8B65\.4282\.1090\.0688\.839\.433\.031\.041\.180\.670\.740\.770\.76Qwen 2\.5 7B Instruct64\.8981\.1482\.0589\.279\.883\.392\.941\.910\.660\.730\.740\.76Qwen 3 4B61\.7778\.0380\.1186\.4811\.524\.263\.572\.060\.650\.720\.730\.75Phi 3\.5 Mini Instruct \(4B\)62\.2186\.0284\.9185\.4010\.832\.172\.352\.280\.650\.750\.740\.74SmallQwen 2\.5 3B Instruct58\.6674\.0876\.7783\.2914\.315\.434\.613\.220\.620\.690\.700\.72Llama 3\.2 3B Instruct57\.8475\.4684\.0782\.6214\.895\.112\.883\.020\.610\.700\.730\.72Gemma 3 1B Instruct56\.0380\.4478\.1279\.3815\.673\.914\.104\.020\.600\.710\.700\.70Llama 3\.2 1B Instruct55\.2771\.3179\.0677\.8216\.446\.524\.024\.280\.600\.660\.700\.69Qwen 2\.5 0\.5B Instruct52\.4169\.2272\.0878\.1118\.097\.716\.885\.390\.580\.650\.660\.69Average acrossall models64\.1582\.3785\.5088\.4810\.543\.502\.592\.120\.670\.750\.760\.77Table 2\.End\-to\-end results across LLMs and KG variants\. Bold indicates the best\-performing KG per metric for each model\.KG\-0KG\-1KG\-2KG\-360608080100100Constraint Satisfaction \(%\)\(a\)KG\-0KG\-1KG\-2KG\-3055101015152020Hallucination Rate \(%\)\(b\)KG\-0KG\-1KG\-2KG\-30\.60\.60\.70\.70\.80\.80\.90\.9Semantic Similarity\(c\)Figure 2\.Performance trends of constraint satisfaction, hallucination rate, and semantic similarity across KG variants\. Darker lines indicate larger models\.### 5\.1\.Retrieval performance across KG variants

Table[3](https://arxiv.org/html/2608.10176#S5.T3)reports retrieval quality for each KG variant atk∈\{3,5\}k\\in\\\{3,5\\\}using Precision@k, Recall@k, and F1@k\. These metrics evaluate only the retrieval, i\.e\., the quality of the top\-kkcandidate set returned before any LLM response generation\. Overall, KG\-based representations substantially improve retrieval compared to the no\-KG baseline\. Atk=3k=3, KG\-3 achieves the strongest retrieval across all reported metrics, reaching P@3=0\.87, R@3=0\.91, and F1@3=0\.86 respectively\. A similar pattern holds atk=5k=5, where KG\-3 remains best, while KG\-0 remains the weakest baseline\. These results suggest that combining location and hours in KG\-3 yields the most accurate retrieved candidate sets prior to downstream constraint checking and response generation via LLM\.

Table 3\.Retrieval quality for each KG variant atk=3,5k\{=\}3,5, reported via Precision@k \(P@k\), Recall@k \(R@k\), and F1@k\. Best values per column are marked in bold\.KG VariantP@3R@3F1@3P@5R@5F1@5KG\-0 \(No\-KG\)0\.380\.400\.390\.300\.440\.36KG\-1 \(Location\)0\.700\.720\.710\.620\.750\.68KG\-2 \(Hours\)0\.660\.630\.640\.560\.700\.62KG\-3 \(Location \+ Hours\)0\.870\.910\.860\.770\.930\.84
### 5\.2\.Reliability improves with stronger retrieval

Table[2](https://arxiv.org/html/2608.10176#S5.T2)and Figure[2](https://arxiv.org/html/2608.10176#S5.F2)\(a,b,c\) show a consistent reliability trend as the KG representation becomes more structured\. Averaged across all models, constraint satisfaction increases from 64\.15% \(KG\-0\) to 88\.48% \(KG\-3\) while semantic similarity also improves \(0\.67 to 0\.77\)\. The upward trajectories in Figure[2](https://arxiv.org/html/2608.10176#S5.F2)\(a,c\) indicate that retrieval\-centric improvements dominate the end\-to\-end behavior across model families, supporting the premise that candidate quality is a primary bottleneck in directory\-based public\-service QA\.

### 5\.3\.Reducing out\-of\-dataset hallucinations

A key benefit of stronger retrieval is the reduction in out\-of\-dataset recommendations\. Table[2](https://arxiv.org/html/2608.10176#S5.T2)shows that the hallucination rate drops from 10\.54% under KG\-0 to 2\.12% under KG\-3 on average, and Figure[2](https://arxiv.org/html/2608.10176#S5.F2)\(b\) confirms a sharp decline from KG\-0 to KG\-1 that continues through KG\-3\. This supports the interpretation that when the LLM is restricted to a verified candidate set retrieved under explicit constraints, it becomes less likely to suggest outside the directory\.

### 5\.4\.Retrieval reduces sensitivity to model choice

Figure[2](https://arxiv.org/html/2608.10176#S5.F2)\(a,b,c\) highlights that model\-to\-model variance is the largest under KG\-0 and shrinks under stronger KG variants\. In particular, as retrieval improves \(KG\-1 to KG\-3\), performance curves across models become more tightly clustered, suggesting that the LLM’s role shifts toward shallow reasoning over retrieved evidence\. This pattern aligns with our central claim: in public\-service directory settings, stronger retrieval reduces sensitivity to model choice and enables more robust performance even with smaller models\.

## 6\.Conclusion

This paper investigates whether retrieval quality determines recommendation reliability in public service conversational agents\. To answer this, we propose TRACE, a constraint\-aware retrieval framework grounded in knowledge graph representations of public service directories\. Our experiments demonstrate that improving retrieval consistently improves end\-to\-end recommendation reliability, and that stronger retrieval stabilizes performance across diverse LLMs\. Together, these findings establish retrieval as the critical bottleneck in public service conversational agents\.

## References

- M\. Abdin, J\. Aneja, H\. Behl, S\. Bubeck, R\. Eldan, S\. Gunasekar, M\. Harrison, R\. J\. Hewett, M\. Javaheripi, P\. Kauffmann,et al\.\(2024\)Phi\-4 technical report\.arXiv preprint arXiv:2412\.08905\.Cited by:[§4\.4](https://arxiv.org/html/2608.10176#S4.SS4.p1.2)\.
- A\. Agarwal, C\. Devaguptapu,et al\.\(2025\)Hybrid graphs for table\-and\-text based question answering using llms\.InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),pp\. 858–875\.Cited by:[§2](https://arxiv.org/html/2608.10176#S2.p3.1)\.
- N\. Aoki \(2020\)An experimental study of public trust in ai chatbots in the public sector\.Government information quarterly37\(4\),pp\. 101490\.Cited by:[§2](https://arxiv.org/html/2608.10176#S2.p2.1)\.
- M\. Arends and T\. Mawela \(2024\)Chatbot adoption in public service delivery\.In2024 International Conference on Computer and Applications \(ICCA\),pp\. 1–7\.Cited by:[§2](https://arxiv.org/html/2608.10176#S2.p2.1)\.
- R\. Cherry \(2002\)Who uses service directories? extending the behavioral model to information use by older people\.Research On Aging24\(5\),pp\. 548–574\.Cited by:[§1](https://arxiv.org/html/2608.10176#S1.p1.1)\.
- P\. R\. Cohen \(2019\)Foundations of collaborative task\-oriented dialogue: what’s in a slot?\.InProceedings of the 20th Annual SIGdial Meeting on Discourse and Dialogue,pp\. 198–209\.Cited by:[§2](https://arxiv.org/html/2608.10176#S2.p1.1)\.
- L\. Dietz, A\. Kotov, and E\. Meij \(2018\)Utilizing knowledge graphs for text\-centric information retrieval\.InThe 41st international ACM SIGIR conference on research & development in information retrieval,pp\. 1387–1390\.Cited by:[§2](https://arxiv.org/html/2608.10176#S2.p3.1)\.
- A\. Dow, R\. Comber, and J\. Vines \(2018\)Between grassroots and the hierarchy: lessons learned from the design of a public services directory\.InProceedings of the 2018 CHI Conference on Human Factors in Computing Systems,pp\. 1–13\.Cited by:[§1](https://arxiv.org/html/2608.10176#S1.p1.1)\.
- R\. Dreyling, T\. Koppel, T\. Tammet, and I\. Pappel \(2024\)Challenges of generative ai chatbots in public services\-an integrative review\.Available at SSRN 4850714\.Cited by:[§1](https://arxiv.org/html/2608.10176#S1.p1.1)\.
- S\. Feng, S\. S\. Patel, H\. Wan, and S\. Joshi \(2021\)MultiDoc2Dial: modeling dialogues grounded in multiple documents\.InProceedings of the 2021 conference on empirical methods in natural language processing,pp\. 6162–6176\.Cited by:[§2](https://arxiv.org/html/2608.10176#S2.p2.1)\.
- S\. Feng, H\. Wan, C\. Gunasekara, S\. Patel, S\. Joshi, and L\. Lastras \(2020\)Doc2dial: a goal\-oriented document\-grounded dialogue dataset\.InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),pp\. 8118–8128\.Cited by:[§2](https://arxiv.org/html/2608.10176#S2.p2.1)\.
- J\. Gao, C\. Xiong, and P\. Bennett \(2020\)Recent advances in conversational information retrieval\.InProceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval,pp\. 2421–2424\.Cited by:[§2](https://arxiv.org/html/2608.10176#S2.p1.1)\.
- A\. Grattafiori, A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Vaughan,et al\.\(2024\)The llama 3 herd of models\.arXiv preprint arXiv:2407\.21783\.Cited by:[§4\.4](https://arxiv.org/html/2608.10176#S4.SS4.p1.2)\.
- D\. Guo, D\. Yang, H\. Zhang, J\. Song, P\. Wang, Q\. Zhu, R\. Xu, R\. Zhang, S\. Ma, X\. Bi,et al\.\(2025\)Deepseek\-r1: incentivizing reasoning capability in llms via reinforcement learning\.arXiv preprint arXiv:2501\.12948\.Cited by:[§4\.4](https://arxiv.org/html/2608.10176#S4.SS4.p1.2)\.
- T\. Hasan, L\. Cure, and S\. Sarkar \(2026\)Retrieval challenges in low\-resource public service information: a case study on food pantry access\.arXiv preprint arXiv:2602\.21598\.Cited by:[§2](https://arxiv.org/html/2608.10176#S2.p2.1)\.
- A\. Hliaoutakis, G\. Varelas, E\. Voutsakis, E\. G\. Petrakis, and E\. Milios \(2009\)Information retrieval by semantic similarity\.InMedical Informatics: Concepts, Methodologies, Tools, and Applications,pp\. 647–665\.Cited by:[§4\.5](https://arxiv.org/html/2608.10176#S4.SS5.p4.1)\.
- X\. Huang, J\. Zhang, D\. Li, and P\. Li \(2019\)Knowledge graph embedding based question answering\.InProceedings of the twelfth ACM international conference on web search and data mining,pp\. 105–113\.Cited by:[§2](https://arxiv.org/html/2608.10176#S2.p3.1)\.
- A\. Khan \(2023\)Knowledge graphs querying\.ACM SIGMOD Record52\(2\),pp\. 18–29\.Cited by:[§2](https://arxiv.org/html/2608.10176#S2.p3.1)\.
- F\. Kleiman and M\. M\. Barbosa \(2025\)Management and performance program chatbot: a use case of large language model in the federal public sector in brazil\.Digital Government: Research and Practice6\(2\),pp\. 1–11\.Cited by:[§1](https://arxiv.org/html/2608.10176#S1.p1.1)\.
- A\. G\. Larsen and A\. Følstad \(2024\)The impact of chatbots on public service provision: a qualitative interview study with citizens and public service providers\.Government Information Quarterly41\(2\),pp\. 101927\.Cited by:[§1](https://arxiv.org/html/2608.10176#S1.p1.1)\.
- M\. Li, S\. Miao, and P\. Li \(2025\)Simple is effective: the roles of graphs and large language models in knowledge\-graph\-based retrieval\-augmented generation\.InInternational Conference on Learning Representations,Vol\.2025,pp\. 6061–6089\.Cited by:[§2](https://arxiv.org/html/2608.10176#S2.p3.1)\.
- S\. Louvan and B\. Magnini \(2020\)Recent neural methods on slot filling and intent classification for task\-oriented dialogue systems: a survey\.InProceedings of the 28th international conference on computational linguistics,pp\. 480–496\.Cited by:[§2](https://arxiv.org/html/2608.10176#S2.p1.1)\.
- B\. P\. Majumder, S\. Rao, M\. Galley, and J\. McAuley \(2021\)Ask what’s missing and what’s useful: improving clarification question generation using global knowledge\.InProceedings of the 2021 conference of the north american chapter of the association for computational linguistics: human language technologies,pp\. 4300–4312\.Cited by:[§2](https://arxiv.org/html/2608.10176#S2.p1.1)\.
- C\. Malaviya, J\. C\. Chang, D\. Roth, M\. Iyyer, M\. Yatskar, and K\. Lo \(2025\)Contextualized evaluations: judging language model responses to underspecified queries\.Transactions of the Association for Computational Linguistics13,pp\. 878–900\.Cited by:[§2](https://arxiv.org/html/2608.10176#S2.p1.1)\.
- C\. D\. Manning \(2008\)Introduction to information retrieval\.Syngress Publishing,\.Cited by:[§4\.5](https://arxiv.org/html/2608.10176#S4.SS5.p1.1)\.
- M\. Polignano, C\. Musto, M\. De Gemmis, P\. Lops, and G\. Semeraro \(2021\)Together is better: hybrid recommendations combining graph embeddings and contextualized word representations\.InProceedings of the 15th ACM conference on recommender systems,pp\. 187–198\.Cited by:[§2](https://arxiv.org/html/2608.10176#S2.p3.1)\.
- R\. Reinanda, E\. Meij, and M\. d\. Rijke \(2020\)Knowledge graphs: an information retrieval perspective\.Foundations and Trends® in Information Retrieval14\(4\),pp\. 289–444\.Cited by:[§2](https://arxiv.org/html/2608.10176#S2.p3.1)\.
- I\. Sekulić, M\. Aliannejadi, and F\. Crestani \(2021\)Towards facet\-driven generation of clarifying questions for conversational search\.InProceedings of the 2021 ACM SIGIR international conference on theory of information retrieval,pp\. 167–175\.Cited by:[§2](https://arxiv.org/html/2608.10176#S2.p1.1)\.
- A\. Singh, A\. Fry, A\. Perelman, A\. Tart, A\. Ganesh, A\. El\-Kishky, A\. McLaughlin, A\. Low, A\. Ostrow, A\. Ananthram,et al\.\(2025\)Openai gpt\-5 system card\.arXiv preprint arXiv:2601\.03267\.Cited by:[§4\.4](https://arxiv.org/html/2608.10176#S4.SS4.p1.2)\.
- A\. Stamatis, A\. Gerontas, A\. Dasyras, and E\. Tambouris \(2020\)Using chatbots and life events to provide public service information\.InProceedings of the 13th International Conference on Theory and Practice of Electronic Governance,pp\. 54–61\.Cited by:[§2](https://arxiv.org/html/2608.10176#S2.p2.1)\.
- L\. Tavakoli \(2020\)Generating clarifying questions in conversational search systems\.InProceedings of the 29th ACM International Conference on Information & Knowledge Management,pp\. 3253–3256\.Cited by:[§2](https://arxiv.org/html/2608.10176#S2.p1.1)\.
- G\. Team, T\. Mesnard, C\. Hardin, R\. Dadashi, S\. Bhupatiraju, S\. Pathak, L\. Sifre, M\. Rivière, M\. S\. Kale, J\. Love,et al\.\(2024\)Gemma: open models based on gemini research and technology\.arXiv preprint arXiv:2403\.08295\.Cited by:[§4\.4](https://arxiv.org/html/2608.10176#S4.SS4.p1.2)\.
- S\. Tripathi, M\. T\. Nafis, I\. Hussain, and J\. Gao \(2025\)The confidence paradox: can llm know when it’s wrong?\.InProceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia\-Pacific Chapter of the Association for Computational Linguistics,pp\. 2078–2087\.Cited by:[§2](https://arxiv.org/html/2608.10176#S2.p2.1)\.
- C\. Van Noordt and G\. Misuraca \(2019\)New wine in old bottles: chatbots in government: exploring the transformative impact of chatbots in public service delivery\.InInternational Conference on Electronic Participation,pp\. 49–59\.Cited by:[§1](https://arxiv.org/html/2608.10176#S1.p1.1)\.
- Z\. Xu, M\. J\. Cruz, M\. Guevara, T\. Wang, M\. Deshpande, X\. Wang, and Z\. Li \(2024\)Retrieval\-augmented generation with knowledge graphs for customer service question answering\.InProceedings of the 47th international ACM SIGIR conference on research and development in information retrieval,pp\. 2905–2909\.Cited by:[§2](https://arxiv.org/html/2608.10176#S2.p3.1)\.
- A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv,et al\.\(2025\)Qwen3 technical report\.arXiv preprint arXiv:2505\.09388\.Cited by:[§4\.4](https://arxiv.org/html/2608.10176#S4.SS4.p1.2)\.
- J\. Yu, Y\. Liu, J\. Gu, P\. Torr, and D\. Zhou \(2026\)Can knowledge\-graph\-based retrieval augmented generation really retrieve what you need?\.Advances in Neural Information Processing Systems38,pp\. 95653–95682\.Cited by:[§2](https://arxiv.org/html/2608.10176#S2.p3.1)\.
- H\. Zamani, S\. Dumais, N\. Craswell, P\. Bennett, and G\. Lueck \(2020\)Generating clarifying questions for information retrieval\.InProceedings of the web conference 2020,pp\. 418–428\.Cited by:[§2](https://arxiv.org/html/2608.10176#S2.p1.1)\.
- J\. Zhou, R\. Shen, Y\. You, C\. DiSalvo, L\. Dombrowski, and C\. J\. MacLellan \(2025\)Improving public service chatbot design and civic impact: investigation of citizens’ perceptions of a metro city 311 chatbot\.InProceedings of the 2025 ACM Designing Interactive Systems Conference,pp\. 2143–2155\.Cited by:[§2](https://arxiv.org/html/2608.10176#S2.p2.1)\.
- X\. Zhu, Y\. Xie, Y\. Liu, Y\. Li, and W\. Hu \(2025\)Knowledge graph\-guided retrieval augmented generation\.InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),pp\. 8912–8924\.Cited by:[§2](https://arxiv.org/html/2608.10176#S2.p3.1)\.