SimXAI：一个简单的即插即用层可提升对话式可解释人工智能中的交互质量与理解程度

# SimXAI: A Simple Drop-in Layer Increases Interaction Quality and Understanding in Conversational XAI
Source: [https://arxiv.org/html/2610.41856](https://arxiv.org/html/2610.41856)
Qianli WangAffiliation:Technische Universität BerlinAffiliation:German Research Center for Artificial Intelligence \(DFKI\)Nils FeldhusAffiliation:University of GroningenAffiliation:German Research Center for Artificial Intelligence \(DFKI\)Jörg SchlöttererAffiliation:University of MarburgChristin SeifertAffiliation:University of Marburg

###### Abstract

Conversational explainable AI \(ConvXAI\) agents often use GUS\-style architectures with separate modules for intent recognition, dialogue management, and template\-based response generation\. These pipelines are hard to adapt and often struggle with context\-dependent follow\-up questions\. We presentSimXAI, a lightweight LLM\-centric conversational layer that replaces these modules with a single tool\-calling LLM that bridges access to existing XAI resources\. Rather than a standalone agent,SimXAIserves as a modular interface that enables conversational access to external XAI resources\. We further introduce an evaluation suite for single\- and multi\-turn parsing, response\-to\-tool faithfulness, and contextualization\. Across existing and new datasets,SimXAIachieves strong function\-selection accuracy and high response\-to\-tool faithfulness while better handling context\-dependent requests\. In a user study replacing only the conversational layer ofTalkToModel, a representative GUS\-based agent, users completed tasks more accurately and in fewer turns\. Overall,SimXAIimproves conversational interaction while leaving the underlying XAI resources unchanged111The source code \(including evaluation suite\) is available at:[https://github\.com/bach\-mr/SimXAI](https://github.com/bach-mr/SimXAI)\.

22footnotetext:Work done at TU Berlin\.## 1Introduction

![Refer to caption](https://arxiv.org/html/2610.41856v1/agent_fig_1_gray.png)Figure 1:Comparison ofTalkToModel’s traditional GUS architecture withSimXAI’s LLM\-centric conversational component\. Traditional systems rely on multiple components that require rules, templates, or fine\-tuning\. In contrast,SimXAIrequires only function specifications and a single LLM\. Both architectures share the same XAI resources\.The XAI community has reached a consensus that explanations should take a conversational form[Miller \(2019\)](https://arxiv.org/html/2610.41856#bib.bib19);[Lakkaraju et al\. \(2022\)](https://arxiv.org/html/2610.41856#bib.bib3);[Zhang et al\. \(2025\)](https://arxiv.org/html/2610.41856#bib.bib13)\. Conversational explainable AI \(ConvXAI\) agents therefore move beyond one\-off explanations and embrace conversational approaches to provide explanations\. These conversational explanations have been demonstrated to be more effective for users’ model understanding[Shen et al\. \(2023\)](https://arxiv.org/html/2610.41856#bib.bib8);[Mindlin et al\. \(2024\)](https://arxiv.org/html/2610.41856#bib.bib5);[He et al\. \(2025\)](https://arxiv.org/html/2610.41856#bib.bib2)\. Conceptually, a ConvXAI agent can be decomposed into two parts: \(i\) a conversational layer \(ConvXAI layer\), responsible for managing the interaction with the user, and \(ii\) the underlyingXAI resources, which store or generate the explanatory information \(e\.g\., model/data cards[Mitchell et al\. \(2019\)](https://arxiv.org/html/2610.41856#bib.bib21);[Pushkarna et al\. \(2022\)](https://arxiv.org/html/2610.41856#bib.bib22), or feature\-attribution methods like SHAP[Lundberg and Lee \(2017\)](https://arxiv.org/html/2610.41856#bib.bib33)or counterfactual explanations[Mothilal et al\. \(2020\)](https://arxiv.org/html/2610.41856#bib.bib39);[Ross et al\. \(2021\)](https://arxiv.org/html/2610.41856#bib.bib40)\)\. Most current ConvXAI agents \(i\.a\.,[Nguyen et al\. 2023](https://arxiv.org/html/2610.41856#bib.bib15);[Slack et al\. 2023](https://arxiv.org/html/2610.41856#bib.bib9);[Feldhus et al\. 2023](https://arxiv.org/html/2610.41856#bib.bib1);[Wang et al\. 2024a](https://arxiv.org/html/2610.41856#bib.bib10)\) adopt the traditional GUS \(Genial Understanding System\) architecture[Bobrow et al\. \(1977\)](https://arxiv.org/html/2610.41856#bib.bib20)for the ConvXAI layer, which comprises three distinct components:natural language understanding \(NLU\)for intent recognition,dialogue managementfor context maintenance and rule application, andnatural language generation \(NLG\)for response production\. Although such GUS\-based ConvXAI layers present explanations through interactive dialogue, they rely onmultiple models separately fine\-tunedonhandcrafted datafor intent recognition in NLU,rule\-baseddialogue management, andtemplate\-basedresponse generation in NLG \(Figure[1](https://arxiv.org/html/2610.41856#S1.F1)\)\. As a result, they remain limited in contextualization, scalability, and generalization[Rastogi et al\. \(2020\)](https://arxiv.org/html/2610.41856#bib.bib23)\. These limitations highlight the need for lightweight, adaptive, and flexible ConvXAI layers\. Recent developments suggest that LLMs can fulfill this role and replace traditional ConvXAI layers in such systems[Li et al\. \(2024\)](https://arxiv.org/html/2610.41856#bib.bib41)\.

SystemContextualizationScalabilityGeneralizationResponsefaithfulnessTalkToModel[Slack et al\. \(2023\)](https://arxiv.org/html/2610.41856#bib.bib9)MediumLow \(FT, D, T, R\)LowHighInterroLang[Feldhus et al\. \(2023\)](https://arxiv.org/html/2610.41856#bib.bib1)WeakLow \(FT, D, T, R\)LowHighXAgent[Nguyen et al\. \(2023\)](https://arxiv.org/html/2610.41856#bib.bib15)WeakMedium \(FT, D, T, R\)LowHighSimXAIStrongHighHighHigh

Table 1:Design\-level comparison ofSimXAIwith the conversational layers of prior GUS\-based ConvXAI agents in terms of contextualization, scalability, generalization, and response\-to\-resource faithfulness\. The table summarizes architectural properties from the respective system designs\. Scalability constraints include: Finetuning \(FT\), Handcrafted Data \(D\), Templates \(T\), and Rules \(R\)\.We introduceSimXAI, a simple, lightweight, LLM\-centric framework that acts as a ConvXAI layer between users and existing XAI resources \(see[Figure1](https://arxiv.org/html/2610.41856#S1.F1)\)\. Unlike rule\- or template\-based systems,SimXAIuses an LLM to support natural, context\-aware dialogue without requiring handcrafted intent datasets or conversational fine\-tuning\. The layer isdata\- and model\-agnostic\. Integrating it into an agent requires only function implementations that provide access to the agent’s existing XAI resources\.SimXAIcan therefore replace the conversational components of GUS\-based agents \(e\.g\.,[Slack et al\. 2023](https://arxiv.org/html/2610.41856#bib.bib9);[Nguyen et al\. 2023](https://arxiv.org/html/2610.41856#bib.bib15)\) while preserving the underlying explanations\.SimXAIis not a standalone agentand does not itself select or validate XAI methods\. Instead, it provides a modular interface to resources supplied by the host system\.

Using LLMs for tool calling is not technically novel\. Our contribution is a ConvXAI\-specific design that separates a replaceable conversational layer from XAI resources\. The design also provides a function interface based on XAI question types and an evaluation framework that separately measures function parsing, response\-to\-tool faithfulness, and contextualization\.

Prior work on evaluating ConvXAI has mainly focused on isolated turns[Malandri et al\. \(2023\)](https://arxiv.org/html/2610.41856#bib.bib4);[Wang et al\. \(2024a\)](https://arxiv.org/html/2610.41856#bib.bib10)and has often overlooked dialogue context\. To address this gap, we evaluateSimXAIon parsing accuracy in single\- and multi\-turn dialogues, contextualization, and response\-to\-tool faithfulness\. For each aspect, we provide a dataset with manually annotated ground truth, corresponding metrics, and an automated evaluation pipeline\. The faithfulness evaluation measures whether the conversational layer preserves the returned XAI output\. It does not measure whether the underlying explanation method is faithful to the predictive model\.

In a user study with 30 participants, replacing the ConvXAI layer ofTalkToModelwithSimXAIwhile keeping the underlying XAI resources unchanged, significantly improved both, perceived interaction quality and task performance\.

In summary, our contributions are threefold\.First, based on the analysis of limitations of prior work \(§[2](https://arxiv.org/html/2610.41856#S2)\), we introduceSimXAI, an LLM\-centric framework that addresses these limitations and offers a practical alternative to conventional ConvXAI layers \(§[3](https://arxiv.org/html/2610.41856#S3)\)\.Second, we present a multi\-aspect evaluation suite for ConvXAI layers, including automated assessment of parsing, response\-to\-tool faithfulness, and contextualization \(§[4](https://arxiv.org/html/2610.41856#S4)\)\.Third, we show through quantitative evaluation and a user study that replacing a conventional conversational layer withSimXAIimproves significantly conversational quality while preserving faithfulness to underlying XAI outputs \(§[5](https://arxiv.org/html/2610.41856#S5), §[6](https://arxiv.org/html/2610.41856#S6)\)\.

## 2Limitations of Conversational Layers in Prior ConvXAI Agents

GUS\-based ConvXAI layers \(such as[Nguyen et al\. 2023](https://arxiv.org/html/2610.41856#bib.bib15);[Slack et al\. 2023](https://arxiv.org/html/2610.41856#bib.bib9);[Feldhus et al\. 2023](https://arxiv.org/html/2610.41856#bib.bib1)\) support follow\-up questions, but their conversational behavior is typically implemented through task\-specific grammars, intent schemas, templates, or dialogue rules\. As summarized in Table[1](https://arxiv.org/html/2610.41856#S1.T1), this design leads to a recurring trade\-off betweenfaithfulnessand three other dimensions:contextualization,scalability, andgeneralization\.

First, prior systems provide onlyweak\-to\-moderate contextualization, handling context primarily through pre\-defined operational structures rather than flexible discourse\-level reasoning\.TalkToModelresolves follow\-up questions through grammar operators such asprevious\_filterandprevious\_operation[Slack et al\. \(2023\)](https://arxiv.org/html/2610.41856#bib.bib9)\.InterroLangextends this design with intent recognition, slot filling, reference resolution, and clarification strategies[Feldhus et al\. \(2023\)](https://arxiv.org/html/2610.41856#bib.bib1)\.XAgentsupports multi\-turn interaction through a fixed dialogue manager and a curated phrase bank[Nguyen et al\. \(2023\)](https://arxiv.org/html/2610.41856#bib.bib15)\. Such mechanisms can handle follow\-up questions only to a limited extent and often struggle when users rely on ellipsis, anaphora, or long\-range dependencies \(see[Figure1](https://arxiv.org/html/2610.41856#S1.F1)for an example of a failure case\)\.

Second, this design leads tolow\-to\-moderate scalabilityandlimited generalizationacross domains and tasks\.TalkToModelrequires dataset\-specific grammars and synthetic utterance–parse pairs[Slack et al\. \(2023\)](https://arxiv.org/html/2610.41856#bib.bib9),InterroLangdepends on annotated intents, slots, and task\-specific operations[Feldhus et al\. \(2023\)](https://arxiv.org/html/2610.41856#bib.bib1), andXAgentrelies on manually curated phrases and response templates[Nguyen et al\. \(2023\)](https://arxiv.org/html/2610.41856#bib.bib15)\. As a result, extending such systems to new settings usually requires additional annotation or rule engineering rather than simply integrating new XAI resources\. This also limits generalization: prior systems are typically demonstrated in narrow settings, such as tabular data or NLP\-specific pipelines, and their conversational control logic does not transfer easily beyond those conditions\.

At the same time, these constrained designs maximally preservefaithfulnessin response generation\. As they generate outputs through rules or templates, the conversational layer is less likely to introduce unsupported claims or misrepresent the underlying XAI method results\. The problem with prior systems is not faithfulness itself, but the rigid control logic they use to maintain it\. That logic limits how well prior systems handle context and makes these systems hard to adapt across domains\.

In contrast,SimXAIaddresses these problems by replacing handcrafted conversational control logic with a tool\-calling LLM that bridges access to existing XAI resources, without requiring finetuning, handcrafted data, or templates\. This design enablesstrong contextualization,high scalabilityandhigh generalization, as it does not rely on domain\-specific implementations\.

## 3SimXAI

SimXAI adds a conversational interface to existing XAI resources using only two components: \(1\) an off\-the\-shelf LLM with tool\-calling support and \(2\) a set of general function specifications that map user requests onto available XAI resources\. The novelty lies in this ConvXAI\-specific separation and interface, rather than in the general mechanism of LLM tool calling\. Unlike prior systems,SimXAIdoes not rely on task\-specific grammars, intent schemas, dialogue managers, or response templates\. Instead,SimXAIinterprets the current user query together with the dialogue history, selects the appropriate function, fills its arguments from context when applicable, and verbalizes the returned result\. The backend remains responsible for computing the prediction, explanation, or data/model information returned by that function\.

Formally, given a user queryqq, dialogue historyHH, and a set of available functions𝒯\\mathcal\{T\}, the system either answers directly or invokes a functiont∈𝒯t\\in\\mathcal\{T\}\. When a function call is required, the system extracts the necessary arguments fromqqandHH, asks for clarification if required information cannot be recovered from the dialogue context, executes the function, and generates a final response grounded in the returned output\. This design replaces handcrafted conversational control logic with a single tool\-calling LLM interface that bridges access to existing XAI resources\. Algorithm[1](https://arxiv.org/html/2610.41856#alg1)summarizes this interaction loop\.

Algorithm 1Response Generation1:User query

qq, dialogue history

HH, tool set

𝒯\\mathcal\{T\}
2:Final response

rr
3:

u←DecideToolUsage​\(q,H,𝒯\)u\\leftarrow\\textsc\{DecideToolUsage\}\(q,H,\\mathcal\{T\}\)
4:if

u=Noneu=\\text\{None\}then

5:

r←InternalKnowledge​\(q,H\)r\\leftarrow\\textsc\{InternalKnowledge\}\(q,H\)
6:return

rr
7:endif

8:

t←SelectTool​\(q,H,𝒯\)t\\leftarrow\\textsc\{SelectTool\}\(q,H,\\mathcal\{T\}\)
9:

a←ExtractArguments​\(t,q,H\)a\\leftarrow\\textsc\{ExtractArguments\}\(t,q,H\)
10:while

¬Valid​\(a,t\)\\neg\\textsc\{Valid\}\(a,t\)do

11:

m←FindMissingArguments​\(a,t\)m\\leftarrow\\textsc\{FindMissingArguments\}\(a,t\)
12:

a′←SearchInDialogue​\(m,H\)a^\{\\prime\}\\leftarrow\\textsc\{SearchInDialogue\}\(m,H\)
13:if

a′≠∅a^\{\\prime\}\\neq\\emptysetthen

14:

a←UpdateArguments​\(a,a′\)a\\leftarrow\\textsc\{UpdateArguments\}\(a,a^\{\\prime\}\)
15:else

16:

c←AskUserForClarification​\(m\)c\\leftarrow\\textsc\{AskUserForClarification\}\(m\)
17:

H←H∪\{c\}H\\leftarrow H\\cup\\\{c\\\}
18:

a←UpdateArguments​\(a,c\)a\\leftarrow\\textsc\{UpdateArguments\}\(a,c\)
19:endif

20:endwhile

21:

o←CallTool​\(t,a\)o\\leftarrow\\textsc\{CallTool\}\(t,a\)
22:

r←GenerateResponse​\(q,H,o\)r\\leftarrow\\textsc\{GenerateResponse\}\(q,H,o\)
23:return

rr

### 3\.1Tool\-Calling LLM

We use ToolACE\-2\.5\-Llama\-3\.1\-8B222[https://huggingface\.co/Team\-ACE/ToolACE\-2\.5\-Llama\-3\.1\-8B](https://huggingface.co/Team-ACE/ToolACE-2.5-Llama-3.1-8B), an off\-the\-shelf LLM trained for tool\-use reasoning[Liu et al\. \(2025\)](https://arxiv.org/html/2610.41856#bib.bib14);[Grattafiori et al\. \(2024\)](https://arxiv.org/html/2610.41856#bib.bib16)\. InSimXAI, the LLM is responsible for four decisions: \(i\) whether tool use is needed, \(ii\) if so, which function to call, \(iii\) how to ground its arguments in the current turn and dialogue history, and \(iv\) how to generate a final natural\-language response from the function output\. No additional parser, dialogue manager, or template bank is required\.

FunctionDescriptionget\_prediction\(instance\)Return the model prediction for an instance\.get\_global\_explanation\(\)Provide a global explanation of model behavior\.get\_local\_explanation\(instance, prediction\_label\)Explain why the model produced a specific prediction\.explain\_why\_not\(instance, expected\_label\)Explain why an expected label was not predicted\.explain\_how\_to\_be\_that\(instance, target\_label\)Suggest how the input could be changed to obtain a target label\.explain\_how\_to\_still\_be\_this\(instance\)Identify changes that would preserve the current prediction\.explain\_what\_if\(instance\)Show how predictions change under input modifications\.get\_model\_performance\(metric\)Report a model performance metric\.get\_data\_information\(\)Provide information about the training and evaluation data\.get\_model\_information\(\)Describe the model, training setup, strengths, and limitations\.get\_output\_information\(\)Explain the meaning of the model outputs\.Table 2:Core functions used bySimXAI\.
### 3\.2Function Set

Our set of functions is based on the question bank of[Liao and Varshney \(2022\)](https://arxiv.org/html/2610.41856#bib.bib17);[Liao et al\. \(2020\)](https://arxiv.org/html/2610.41856#bib.bib18)\. This question bank was created from practitioner interviews and organized around model\-, data\-, and context\-related questions about AI systems\. From this bank, we implement 11 functions: one retrieves the model prediction for a given instance and the other ten address explanatory questions, such as*why*the model made a particular prediction or*how*to modify an instance to change the prediction\. For each function, we specify its purpose, inputs, arguments, and outputs\.[Table2](https://arxiv.org/html/2610.41856#S3.T2)summarizes the full set\. The set is extensible: if no predefined function matches a user’s intent, a new function can be added with its own specification \([Figure2](https://arxiv.org/html/2610.41856#S3.F2)\)\.

##### Function Specification\.

Function specification is critical for ensuring that the LLM calls the correct function\. For each function, we adopt a standard specification template \([Figure2](https://arxiv.org/html/2610.41856#S3.F2)\) that defines the function name, argument list and types, documentation, and expected input and output types\.These specifications are not tied to any particular dataset or model\. Instead, we use general specifications that apply across datasets and models, enabling broad generalization \(cf\.[Figure5](https://arxiv.org/html/2610.41856#A1.F5)in the appendix for a concrete example of a counterfactual function specification\)\.

function\_name\(argument:type\)\-\>return\_type

\#summaryofwhatthefunctiondoes\.

Args:

argument\(type\):Descriptionoftheinput\(format,constraints\)\.

Returns:

return\_type:Descriptionoftheoutput\(labels,scorerange,etc\.\)\.

Figure 2:Template for Function Specification\.

## 4Evaluation

We evaluate ConvXAI layers along three aspects:parsing,response\-to\-tool faithfulness, andcontextualization\. Parsing measures whether the system selects the correct function and arguments for a user request, response\-to\-tool faithfulness measures whether the final response is supported by the returned XAI output, and contextualization measures whether the system uses dialogue history appropriately when interpreting the current turn\. Table[4](https://arxiv.org/html/2610.41856#S4.T4)summarizes the evaluation suite\. This scope evaluates the conversational layer only: it neither measures nor guarantees the intrinsic correctness or model\-faithfulness of the backend XAI method that produced the tool output\. We use this suite for two purposes\. First, we assessSimXAIdirectly across settings of increasing conversational complexity\. Second, we qualitatively analyze gains from replacing the conversational components of prior GUS\-based ConvXAI systems withSimXAIwhile keeping their underlying XAI resources unchanged\.

### 4\.1Parsing

Parsing evaluates whether the system maps a user request to the correct function and arguments\. We report exact\-match accuracy over the selected function\. Unlike prior work, which mainly evaluates parsing in single\-turn settings[Slack et al\. \(2023\)](https://arxiv.org/html/2610.41856#bib.bib9);[Feldhus et al\. \(2023\)](https://arxiv.org/html/2610.41856#bib.bib1);[Nguyen et al\. \(2023\)](https://arxiv.org/html/2610.41856#bib.bib15), we consider three levels of conversational complexity\.

##### Single\-turn parsing\.

We evaluate standard single\-turn function selection on two datasets in ConvXAI settings:CoXQL[Wang et al\. \(2024b\)](https://arxiv.org/html/2610.41856#bib.bib11)for explanation request parsing \(Figure[4](https://arxiv.org/html/2610.41856#A1.F4)\) andCompass[Wang et al\. \(2025\)](https://arxiv.org/html/2610.41856#bib.bib12)for user input extraction \(Figure[6](https://arxiv.org/html/2610.41856#A1.F6)\)\. In this setting, the current query can be interpreted without dialogue history\.

##### Multi\-turn parsing\.

To evaluate short\-range contextual parsing, we constructCompass\-MultiTurnfromCompassby merging related single\-turn interactions into short conversations and removing redundant user turns \(details in Appendix[A](https://arxiv.org/html/2610.41856#A1)\)\. In this setting, the use of recent dialogue history is required to parse the users’ intent correctly\.

##### Dialogue\-level parsing\.

To evaluate long\-range dependencies, we generate synthetic dialogue\-level datasets in two domains: sentiment analysisSYN\-Sentimentand heart rate predictionSYN\-HeartRate\. Each of the two datasets combines dialogues generated under two settings:Constrained, in which we provide our 11 function specifications in the prompt and ask GPT\-5\.1 to generate realistic conversations covering all operations, andUnconstrained, in which GPT\-5\.1 is prompted to generate realistic conversations without any predefined constraints\. Each user turn defines one parsing instance, with the full preceding dialogue provided as context\. Human validation assessed these synthetic dialogues as mostly realistic \(details in Appendix[B](https://arxiv.org/html/2610.41856#A2)\)\.

Together, these datasets assess whether ConvXAI systems can select the correct function corresponding to user’s intent\. They range from isolated questions up to full multi\-turn conversations with long range dependencies\. Table[3](https://arxiv.org/html/2610.41856#S4.T3)summarizes the parsing datasets\.

Dataset\#Dialogues\#Turns\#QueriesCoXQL\+Compass–1462Compass\-MultiTurn104–626SYN\-Sentiment412–2440SYN\-HeartRate412–2431

Table 3:Statistics for parsing datasets with increasing conversational complexity\.

### 4\.2Response\-to\-Tool Faithfulness

As LLMs are prone to hallucination[Huang et al\. \(2025\)](https://arxiv.org/html/2610.41856#bib.bib24), we evaluate the faithfulness of the agent’s responses with respect to the provided XAI resources\. Following prior work[Adlakha et al\. \(2024\)](https://arxiv.org/html/2610.41856#bib.bib27), we define*response\-to\-tool faithfulness*as the extent to which the conversational layer preserves the information contained in a returned XAI output without introducing unsupported claims\. This is distinct from the intrinsic faithfulness of the XAI method to the predictive model, which is outside our scope\.

We construct two faithfulness datasets,SYN\-QA\-SentimentandSYN\-QA\-HeartRate, based onSYN\-SentimentandSYN\-HeartRate\. The datasets contain 71 question–answer pairs covering prediction lookup, explanation outputs, model descriptions, and dataset information\. For each question, we manually construct a gold standard function output and evaluate whether the system response faithfully preserves this information using a binary judgment and accuracy as aggregated metric\.

### 4\.3Contextualization

Contextualization evaluates whether the system uses dialogue history appropriately to answer the current query\. We first assess*context relevance*by manually determining whether a query can be answered independently or requires the preceding dialogue\. For queries that depend on prior context, we then assess whether the response incorporates the necessary dialogue history to resolve the current turn coherently\. This*contextual answer coherence*is a binary judgment, aggregated as accuracy \(the fraction of contextually coherent answers\)\.

AspectSubtaskWhat is evaluatedDatasetSimXAIParsingSingle\-turnCorrect function selection from a single user queryCoXQL,Compass86%Multi\-turnCorrect function selection from short dialogue historyCompass\-MultiTurn88%Dialogue\-levelCorrect function selection from full conversations with long\-range dependenciesSYN\-Sentiment92\.5%SYN\-HeartRate96%Faithfulness–Whether the response is fully supported by the underlying XAI outputSYN\-QA\-Sentiment93\.6%–SYN\-QA\-HeartRate91\.5%ContextualizationContextual relevanceWhether the current query requires prior dialogue contextSYN\-SentimentN/ASYN\-HeartRateN/AContextual coherenceWhether responses to context\-dependent queries use prior context coherentlySYN\-Sentiment90\.9%SYN\-HeartRate94\.74%

Table 4:Overview of the evaluation suite for the ConvXAI layer and the corresponding accuracy results ofSimXAI\.

## 5Results

To assessSimXAI, we report performance on parsing, response\-to\-tool faithfulness, and contextual coherence\. Following recent work on LLM\-based evaluation[Xu et al\. \(2025\)](https://arxiv.org/html/2610.41856#bib.bib26);[Tamber et al\. \(2025\)](https://arxiv.org/html/2610.41856#bib.bib25), we additionally examine opportunities for automated evaluation by measuring the agreement between human rating and LLM\-as\-a\-judge\.333We use[https://huggingface\.co/meta\-llama/Llama\-3\.1\-8B\-Instruct](https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct)Specifically, we report how often the LLM judge assigns the same label as the human annotator for response\-to\-tool faithfulness and contextualization, includingcontext relevanceandcontextual coherence\.

### 5\.1SimXAI Performance

[Table4](https://arxiv.org/html/2610.41856#S4.T4)reports results across the three aspects in the evaluation suite: parsing, response\-to\-tool faithfulness, and contextualization\.

For*parsing*,SimXAIperforms strongly in all settings, achieving above 86% accuracy for single\-turn, multi\-turn, and dialogue\-level evaluation\. Interestingly, accuracy is higher on the multi\-turn and dialogue\-level datasets, suggesting that dialogue context helps the model to resolve user intents better\.

In most cases \(over 91%\),SimXAIproduces responses that arefaithful to the tool output, indicating that the conversational layer preserves the returned XAI information rather than introducing unsupported claims\. This is particularly important becauseSimXAIreplaces rule\- and template\-based conversational components with an LLM\-based interface, which may increase the risk of hallucinations\. These results do not evaluate the intrinsic faithfulness of the backend XAI methods\.

SimXAIfurther shows highcontextual answer coherenceacross domains: 90\.91% for sentiment analysis and 94\.74% for heart rate, indicating that prior dialogue context is generally incorporated correctly in context\-dependent queries\.

Overall,SimXAIperforms well across all three aspects of the evaluation suite\. Parsing accuracy remains high from single\-turn to dialogue\-level evaluation, responses are usually supported by the returned XAI outputs, and most context\-dependent queries are answered coherently\.

### 5\.2LLM\-as\-a\-judge Performance

[Figure3](https://arxiv.org/html/2610.41856#S5.F3)shows the agreement between LLM\-as\-a\-judge and human annotation\. The agreement onresponse\-to\-tool faithfulnessis high \(around 90%\), suggesting that LLM\-based evaluation is a promising proxy for this aspect on our datasets\. While the agreement on*contextual relevance*in sentiment analysis is equally high \(around 90%\), it drops to 76\.60% in heart rate prediction\. This indicates that automatically determining whether a query depends on prior context remains challenging in some domains\. For*contextual answer coherence*, the LLM judge agrees with human annotations in more than 80% of cases across both domains, suggesting that it can serve as a useful proxy for coherence evaluation\.

In summary, evaluation with LLM\-as\-a\-judge shows potential for automated assessment of response\-to\-tool faithfulness and contextualization\. Despite lower agreement on context relevance in heart rate prediction, it may still serve as an efficient approach for initial comparative evaluation\.

Figure 3:Agreement between LLM\-as\-a\-judge and human annotation\.

## 6GUS\-based system comparison

SimXAIis designed to be integrated into existing conversational XAI agents by replacing their original conversational components\. We qualitatively examine howSimXAIcan improve existing systems, by replacing the conversational layer of the ConvXAI agentsTalkToModel[Slack et al\. \(2023\)](https://arxiv.org/html/2610.41856#bib.bib9),InterroLang[Feldhus et al\. \(2023\)](https://arxiv.org/html/2610.41856#bib.bib1), andXAgent[Nguyen et al\. \(2023\)](https://arxiv.org/html/2610.41856#bib.bib15)withSimXAIand re\-assess cases, in which the original system failed \(e\.g\., due to missing context handling abilities\)\.

For each baseline, we keep the original function set and underlying XAI outputs fixed, and replace only the module that maps user requests and dialogue history onto function selection and response generation\. This isolates the contribution ofSimXAIacting solely as conversational layer and not as a modified ConvXAI backend\.

We further conduct a user study on the integration ofSimXAIintoTalkToModel, a representative grammar\-based system\. We directly compare the original model with itsSimXAI\-integrated counterpart in terms of both, objective and subjective metrics\.

### 6\.1Qualitative results

[Figure1](https://arxiv.org/html/2610.41856#S1.F1)shows an example failure case of incorrect context handling ofTalkToModel’s native conversational layers\.TalkToModelfails to maintain contextual understanding because the agent misinterprets the referent of “it” \(anaphora\), which leads to an incorrect final decision\. ReplacingTalkToModel’s conversational layers bySimXAIfixes this issue and correctly resolves the context in the same conversation dialogue\. We observe similar patterns of incorrect context handling in the other agents under comparison, which are partially unable to provide an answer at all\. Replacing the conversational layers of these agents bySimXAIcorrects such issues \(examples in Appendix[C](https://arxiv.org/html/2610.41856#A3)\)\.

### 6\.2User Study

We conduct a controlled within\-subjects study to evaluate whether integratingSimXAIintoTalkToModel[Slack et al\. \(2023\)](https://arxiv.org/html/2610.41856#bib.bib9)1\) improves the perceived quality of the conversation and 2\) leads to a better understanding of the machine learning model\. To isolate the effect of the conversational component, we keep all other components \(datasets, classifiers, and XAI resources\) unchanged\. We compare two agents: \(i\) the originalTalkToModelwith its fine\-tuned ConvXAI layers and \(ii\)TalkToModelwithSimXAIreplacing the ConvXAI layers\. Both agents follow the originalTalkToModelconfiguration described by[Slack et al\. \(2023\)](https://arxiv.org/html/2610.41856#bib.bib9)\.

#### 6\.2\.1Setup

We recruited 30 lay participants via Prolific\.444[https://www\.prolific\.com/](https://www.prolific.com/)Participants receive a compensation of $8/hourThis sample size aligns with prior HCI and XAI studies evaluating interactive explanation systems[Kaur et al\. \(2020\)](https://arxiv.org/html/2610.41856#bib.bib28);[Slack et al\. \(2023\)](https://arxiv.org/html/2610.41856#bib.bib9)\. Participants interact with both agents using the Diabetes dataset[Kahn \(1994\)](https://arxiv.org/html/2610.41856#bib.bib32), as in the originalTalkToModelstudy\.

For each agent, participants complete three tasks\. These tasks cover multiple categories from an established XAI question taxonomy[Liao et al\. \(2020\)](https://arxiv.org/html/2610.41856#bib.bib18), including data understanding, local explanations, and counterfactual inspection\. Each task contains at least two questions, for a total of nine questions across all tasks\. The questions are sampled from a pool of predefined multiple\-choice questions for each task \(details in Appendix[SectionD\.1](https://arxiv.org/html/2610.41856#A4.SS1)\)\. They are designed to be answerable using the ground\-truth information provided by the agent’s explanations\. Some questions by design require multi\-turn interactions to arrive at the correct answer, in order to assess conversational coherence and context handling\. We randomize the order of agents, and although the question types remain consistent across conditions, specific values \(e\.g\., instance identifiers\) are varied to mitigate learning effects\.

We collect both, objective and subjective measures\.Objective metricsare \(1\) answer accuracy, \(2\) conversational efficiency measured as the number of turns per correctly answered question, which is commonly used in dialogue agent evaluation[Su et al\. \(2016\)](https://arxiv.org/html/2610.41856#bib.bib30);[Peng et al\. \(2017\)](https://arxiv.org/html/2610.41856#bib.bib29), and \(3\) indicators of conversational breakdowns, such as repeated or rephrased questions and agent failures, where the agent does not provide meaningful answers\.Subjective measuresconsist of two parts\. First, we use the 5\-point Likert BUS\-11 usability scale[Borsci and Schmettow \(2024\)](https://arxiv.org/html/2610.41856#bib.bib31), which includes 11 items across five subscales, focusing on perceived conversation and information quality and perceived chatbot functionality\. Second, we include apairwise comparisonquestionnaire between the two agents to assess user preference and explanation effectiveness\.

#### 6\.2\.2Results

This section summarizes the key objective and subjective results\. Details on the significance tests for the reported metrics, the full BUS\-11 scale, and the complete pairwise comparison are provided in Appendix[SectionD\.3](https://arxiv.org/html/2610.41856#A4.SS3)\.

MeasureTalkToModelSimXAIAccuracy↑\\uparrow59\.092\.1Chat turns↓\\downarrow12\.410\.0Chat time \(min\)↓\\downarrow7\.45\.8Turns per correct answer↓\\downarrow2\.701\.32

Table 5:Objective results from the user study\.BUS\-11 item \(shortened\)TalkToModelSimXAIInteraction Clarity3\.334\.60Context Awareness3\.134\.37Response Comprehensibility3\.604\.57Goal Facilitation3\.104\.37Information Sufficiency3\.234\.37Information Relevance3\.134\.13Response Accuracy3\.634\.47Response Time4\.374\.03Cronbach’sα\\alpha0\.9250\.856

Table 6:Mean BUS\-11 ratings from the user study \(1 = strongly disagree, 5 = strongly agree\)\. We report eight relevant items in shortened form, full results are in[Table11](https://arxiv.org/html/2610.41856#A4.T11)\. Cronbach’sα\>0\.8\\alpha\>0\.8indicates high internal consistency\.##### Objective results\.

ReplacingTalkToModel’s conversational component withSimXAIleads to clear improvements in both, task performance and perceived interaction quality\.[Table5](https://arxiv.org/html/2610.41856#S6.T5)shows that participants achieve substantially higher accuracy withSimXAIthan with the originalTalkToModel\(92\.1% vs\. 59\.0%,p<\.001p<\.001\)\.SimXAIalso reduces interaction costs: users complete the task in fewer turns on average \(10\.0 vs\. 12\.4 turns\) and in less time \(5\.8 vs\. 7\.4 minutes\)\. Across all sessions, this translates to roughly half as many turns per correct answer withSimXAI\(1\.32 vs\. 2\.70\)\.

##### Subjective results\.

Table[6](https://arxiv.org/html/2610.41856#S6.T6)reports the BUS\-11 usability ratings\. We observe the largest improvements in clarity of communication, context tracking, helpfulness in achieving the user’s goal, and providing an appropriate amount of information \(allp<\.001p<\.001\)\. Perceived response time is the only item whereSimXAIscores lower, suggesting a possible trade\-off between conversational quality and latency\. We attribute this to the models deployed for parsing inTalkToModel, which are comparably smaller \(ranging from 60M to 6B\) than to the tool\-augmented LLM we use inSimXAI\(§[3\.1](https://arxiv.org/html/2610.41856#S3.SS1)\)\. However, the difference is not statistically significant \(p\>0\.1\)\.

##### Pairwise comparison\.

80\.0% of the participants preferSimXAIoverall, while 16\.7% preferTalkToModeland 3\.3% indicate no preference \(cf\.[Table9](https://arxiv.org/html/2610.41856#A4.T9)in the Appendix\)\. We observe the same proportions for perceived task effectiveness\.

Overall, the user study shows that replacing the conversational component ofTalkToModelwithSimXAIimproves both, perceived interaction quality and task performance, even when the underlying XAI resources remain unchanged\.

## 7Related Work

Besides the systems analyzed in §[2](https://arxiv.org/html/2610.41856#S2), that rely on fine\-tuned models, off\-the\-shelf LLMs are increasingly adopted in ConvXAI\.[Nguyen et al\. \(2024\)](https://arxiv.org/html/2610.41856#bib.bib34)extendXAgentby Llama\-2[Touvron et al\. \(2023\)](https://arxiv.org/html/2610.41856#bib.bib37)for postprocessing NLG and answering general factual questions \(e\.g\., about machine learning terminology\)\.KnowThySelf[Prasai et al\. \(2025\)](https://arxiv.org/html/2610.41856#bib.bib6)is a multi\-agent framework, which at its core still resembles the GUS architecture\. It employs Gemma3\-27B[Team et al\. \(2025\)](https://arxiv.org/html/2610.41856#bib.bib36)for most parts of NLU and NLG, while user intents are matched to specialized agents by embedding\-based similarity, using nomic\-embed\-text[Nussbaum et al\. \(2025\)](https://arxiv.org/html/2610.41856#bib.bib35)\. These specialized agents encapsulate XAI resources\. Closest to our work isEcho[Vanbrabant et al\. \(2025\)](https://arxiv.org/html/2610.41856#bib.bib7), another agentic framework with a tool\-calling LLM at its core\. However,Echotrades faithfulness for human validation “… might not always result in a helpful or correct answer \[…\] This further highlights the importance of human\-in\-the\-loop workflows, in which the LLM can try again until the answer is satisfactory”\. This undermines the purpose of ConvXAI to explain a model’s true reasoning, as any plausible answer is likely satisfactory\. Yet, plausibility and correctness are different evaluation criteria that should not be conflated[Nauta et al\. \(2023\)](https://arxiv.org/html/2610.41856#bib.bib38)\(a plausible answer might not be correct, while a correct one might not be plausible, e\.g\., if the model’s reasoning is flawed\)\.

All these systems are demos without quantitative or qualitative evaluation\. Unlike prior GUS\-based systems or existing tool\-calling ConvXAI agents, our contribution is not the generic tool\-calling mechanism itself\.SimXAIintroduces a simple, lightweight, open\-source intermediate layer with a ConvXAI\-specific separation between conversational control and underlying XAI resources, an extensible XAI function design, and a dedicated evaluation suite\. The measured faithfulness in this suite is response\-to\-tool faithfulness and should not be interpreted as validation of the backend explanation method\.

## 8Conclusion

SimXAIenables natural conversational interactions with existing XAI resources through general callable functions\. Across evaluations of parsing, response\-to\-tool faithfulness, and contextualization, its lightweight LLM\-centric design supports conversational explanations while usually preserving the information returned by the XAI resources\. Our user study shows that replacing the conversational layers of a traditional GUS\-based ConvXAI agent bySimXAIsignificantly improves task performance and perceived interaction quality\. This suggests that LLM\-centric architectures provide tangible benefits to users in understanding AI model behavior\. In future work, we plan to explore and enhance fully automated evaluation for LLM\-based conversational XAI layers\. Based on that, we plan to study how LLM choice affects tool calling and to develop further improvements to ConvXAI\.

## Limitations

Like other LLM\-based systems,SimXAIintroduces additional latency and remains susceptible to hallucinations\.

SimXAIimproves only the conversational component: it assumes that appropriate XAI resources are already available and does not assess the intrinsic quality, correctness, or model\-faithfulness of the underlying explanation methods themselves\. Consequently, a response can be faithful to a tool output even when that output is incomplete or misleading\.

We do not study the impact of the LLM choice for tool calling or LLM\-as\-a\-judge\. Our experiments use a lightweight 8B LLaMA 3\.1 model, and results may vary with other models\.

## References

- Adlakhaet al\.\(2024\)V\. Adlakha, P\. BehnamGhader, X\. H\. Lu, N\. Meade, and S\. ReddyEvaluating correctness and faithfulness of instruction\-following models for question answering\.Transactions of the Association for Computational Linguistics12,pp\. 681–699\.External Links:[Link](https://aclanthology.org/2024.tacl-1.38/),[Document](https://dx.doi.org/10.1162/tacl%5Fa%5F00667)Cited by:[§4\.2](https://arxiv.org/html/2610.41856#S4.SS2.p1.1)\.
- Bobrowet al\.\(1977\)D\. G\. Bobrow, R\. M\. Kaplan, M\. Kay, D\. A\. Norman, H\. Thompson, and T\. WinogradGUS, a frame\-driven dialog system\.Artificial Intelligence8\.External Links:ISSN 0004\-3702,[Document](https://dx.doi.org/10.1016/0004-3702%2877%2990018-2),[Link](https://www.sciencedirect.com/science/article/pii/0004370277900182)Cited by:[§1](https://arxiv.org/html/2610.41856#S1.p1.1)\.
- Borsci and Schmettow \(2024\)S\. Borsci and M\. SchmettowRe\-examining the chatbot usability scale \(bus\-11\) to assess user experience with customer relationship management chatbots\.Personal and Ubiquitous Computing28\(6\),pp\. 1033–1044\.Cited by:[§6\.2\.1](https://arxiv.org/html/2610.41856#S6.SS2.SSS1.p3.1)\.
- Feldhuset al\.\(2023\)N\. Feldhus, Q\. Wang, T\. Anikina, S\. Chopra, C\. Oguz, and S\. MöllerInterroLang: exploring NLP models and datasets through dialogue\-based explanations\.InFindings of the Association for Computational Linguistics: EMNLP 2023,H\. Bouamor, J\. Pino, and K\. Bali \(Eds\.\),Singapore,pp\. 5399–5421\.External Links:[Link](https://aclanthology.org/2023.findings-emnlp.359),[Document](https://dx.doi.org/10.18653/v1/2023.findings-emnlp.359)Cited by:[Table 1](https://arxiv.org/html/2610.41856#S1.T1.2.1.3.1),[§1](https://arxiv.org/html/2610.41856#S1.p1.1),[§2](https://arxiv.org/html/2610.41856#S2.p1.1),[§2](https://arxiv.org/html/2610.41856#S2.p2.1),[§2](https://arxiv.org/html/2610.41856#S2.p3.1),[§4\.1](https://arxiv.org/html/2610.41856#S4.SS1.p1.1),[§6](https://arxiv.org/html/2610.41856#S6.p1.1)\.
- Grattafioriet al\.\(2024\)A\. Grattafiori, A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Vaughan, A\. Yang, A\. Fan, A\. Goyal, A\. Hartshorn, A\. Yang, A\. Mitra, A\. Sravankumar, A\. Korenev, A\. Hinsvark, A\. Rao, A\. Zhang, A\. Rodriguez, A\. Gregerson, A\. Spataru, B\. Roziere, B\. Biron, B\. Tang, B\. Chern, C\. Caucheteux, C\. Nayak, C\. Bi, C\. Marra, C\. McConnell, C\. Keller, C\. Touret, C\. Wu, C\. Wong, C\. C\. Ferrer, C\. Nikolaidis, D\. Allonsius, D\. Song, D\. Pintz, D\. Livshits, D\. Wyatt, D\. Esiobu, D\. Choudhary, D\. Mahajan, D\. Garcia\-Olano, D\. Perino, D\. Hupkes, E\. Lakomkin, E\. AlBadawy, E\. Lobanova, E\. Dinan, E\. M\. Smith, F\. Radenovic, F\. Guzmán, F\. Zhang, G\. Synnaeve, G\. Lee, G\. L\. Anderson, G\. Thattai, G\. Nail, G\. Mialon, G\. Pang, G\. Cucurell, H\. Nguyen, H\. Korevaar, H\. Xu, H\. Touvron, I\. Zarov, I\. A\. Ibarra, I\. Kloumann, I\. Misra, I\. Evtimov, J\. Zhang, J\. Copet, J\. Lee, J\. Geffert, J\. Vranes, J\. Park, J\. Mahadeokar, J\. Shah, J\. van der Linde, J\. Billock, J\. Hong, J\. Lee, J\. Fu, J\. Chi, J\. Huang, J\. Liu, J\. Wang, J\. Yu, J\. Bitton, J\. Spisak, J\. Park, J\. Rocca, J\. Johnstun, J\. Saxe, J\. Jia, K\. V\. Alwala, K\. Prasad, K\. Upasani, K\. Plawiak, K\. Li, K\. Heafield, K\. Stone, K\. El\-Arini, K\. Iyer, K\. Malik, K\. Chiu, K\. Bhalla, K\. Lakhotia, L\. Rantala\-Yeary, L\. van der Maaten, L\. Chen, L\. Tan, L\. Jenkins, L\. Martin, L\. Madaan, L\. Malo, L\. Blecher, L\. Landzaat, L\. de Oliveira, M\. Muzzi, M\. Pasupuleti, M\. Singh, M\. Paluri, M\. Kardas, M\. Tsimpoukelli, M\. Oldham, M\. Rita, M\. Pavlova, M\. Kambadur, M\. Lewis, M\. Si, M\. K\. Singh, M\. Hassan, N\. Goyal, N\. Torabi, N\. Bashlykov, N\. Bogoychev, N\. Chatterji, N\. Zhang, O\. Duchenne, O\. Çelebi, P\. Alrassy, P\. Zhang, P\. Li, P\. Vasic, P\. Weng, P\. Bhargava, P\. Dubal, P\. Krishnan, P\. S\. Koura, P\. Xu, Q\. He, Q\. Dong, R\. Srinivasan, R\. Ganapathy, R\. Calderer, R\. S\. Cabral, R\. Stojnic, R\. Raileanu, R\. Maheswari, R\. Girdhar, R\. Patel, R\. Sauvestre, R\. Polidoro, R\. Sumbaly, R\. Taylor, R\. Silva, R\. Hou, R\. Wang, S\. Hosseini, S\. Chennabasappa, S\. Singh, S\. Bell, S\. S\. Kim, S\. Edunov, S\. Nie, S\. Narang, S\. Raparthy, S\. Shen, S\. Wan, S\. Bhosale, S\. Zhang, S\. Vandenhende, S\. Batra, S\. Whitman, S\. Sootla, S\. Collot, S\. Gururangan, S\. Borodinsky, T\. Herman, T\. Fowler, T\. Sheasha, T\. Georgiou, T\. Scialom, T\. Speckbacher, T\. Mihaylov, T\. Xiao, U\. Karn, V\. Goswami, V\. Gupta, V\. Ramanathan, V\. Kerkez, V\. Gonguet, V\. Do, V\. Vogeti, V\. Albiero, V\. Petrovic, W\. Chu, W\. Xiong, W\. Fu, W\. Meers, X\. Martinet, X\. Wang, X\. Wang, X\. E\. Tan, X\. Xia, X\. Xie, X\. Jia, X\. Wang, Y\. Goldschlag, Y\. Gaur, Y\. Babaei, Y\. Wen, Y\. Song, Y\. Zhang, Y\. Li, Y\. Mao, Z\. D\. Coudert, Z\. Yan, Z\. Chen, Z\. Papakipos, A\. Singh, A\. Srivastava, A\. Jain, A\. Kelsey, A\. Shajnfeld, A\. Gangidi, A\. Victoria, A\. Goldstand, A\. Menon, A\. Sharma, A\. Boesenberg, A\. Baevski, A\. Feinstein, A\. Kallet, A\. Sangani, A\. Teo, A\. Yunus, A\. Lupu, A\. Alvarado, A\. Caples, A\. Gu, A\. Ho, A\. Poulton, A\. Ryan, A\. Ramchandani, A\. Dong, A\. Franco, A\. Goyal, A\. Saraf, A\. Chowdhury, A\. Gabriel, A\. Bharambe, A\. Eisenman, A\. Yazdan, B\. James, B\. Maurer, B\. Leonhardi, B\. Huang, B\. Loyd, B\. D\. Paola, B\. Paranjape, B\. Liu, B\. Wu, B\. Ni, B\. Hancock, B\. Wasti, B\. Spence, B\. Stojkovic, B\. Gamido, B\. Montalvo, C\. Parker, C\. Burton, C\. Mejia, C\. Liu, C\. Wang, C\. Kim, C\. Zhou, C\. Hu, C\. Chu, C\. Cai, C\. Tindal, C\. Feichtenhofer, C\. Gao, D\. Civin, D\. Beaty, D\. Kreymer, D\. Li, D\. Adkins, D\. Xu, D\. Testuggine, D\. David, D\. Parikh, D\. Liskovich, D\. Foss, D\. Wang, D\. Le, D\. Holland, E\. Dowling, E\. Jamil, E\. Montgomery, E\. Presani, E\. Hahn, E\. Wood, E\. Le, E\. Brinkman, E\. Arcaute, E\. Dunbar, E\. Smothers, F\. Sun, F\. Kreuk, F\. Tian, F\. Kokkinos, F\. Ozgenel, F\. Caggioni, F\. Kanayet, F\. Seide, G\. M\. Florez, G\. Schwarz, G\. Badeer, G\. Swee, G\. Halpern, G\. Herman, G\. Sizov, Guangyi, Zhang, G\. Lakshminarayanan, H\. Inan, H\. Shojanazeri, H\. Zou, H\. Wang, H\. Zha, H\. Habeeb, H\. Rudolph, H\. Suk, H\. Aspegren, H\. Goldman, H\. Zhan, I\. Damlaj, I\. Molybog, I\. Tufanov, I\. Leontiadis, I\. Veliche, I\. Gat, J\. Weissman, J\. Geboski, J\. Kohli, J\. Lam, J\. Asher, J\. Gaya, J\. Marcus, J\. Tang, J\. Chan, J\. Zhen, J\. Reizenstein, J\. Teboul, J\. Zhong, J\. Jin, J\. Yang, J\. Cummings, J\. Carvill, J\. Shepard, J\. McPhie, J\. Torres, J\. Ginsburg, J\. Wang, K\. Wu, K\. H\. U, K\. Saxena, K\. Khandelwal, K\. Zand, K\. Matosich, K\. Veeraraghavan, K\. Michelena, K\. Li, K\. Jagadeesh, K\. Huang, K\. Chawla, K\. Huang, L\. Chen, L\. Garg, L\. A, L\. Silva, L\. Bell, L\. Zhang, L\. Guo, L\. Yu, L\. Moshkovich, L\. Wehrstedt, M\. Khabsa, M\. Avalani, M\. Bhatt, M\. Mankus, M\. Hasson, M\. Lennie, M\. Reso, M\. Groshev, M\. Naumov, M\. Lathi, M\. Keneally, M\. Liu, M\. L\. Seltzer, M\. Valko, M\. Restrepo, M\. Patel, M\. Vyatskov, M\. Samvelyan, M\. Clark, M\. Macey, M\. Wang, M\. J\. Hermoso, M\. Metanat, M\. Rastegari, M\. Bansal, N\. Santhanam, N\. Parks, N\. White, N\. Bawa, N\. Singhal, N\. Egebo, N\. Usunier, N\. Mehta, N\. P\. Laptev, N\. Dong, N\. Cheng, O\. Chernoguz, O\. Hart, O\. Salpekar, O\. Kalinli, P\. Kent, P\. Parekh, P\. Saab, P\. Balaji, P\. Rittner, P\. Bontrager, P\. Roux, P\. Dollar, P\. Zvyagina, P\. Ratanchandani, P\. Yuvraj, Q\. Liang, R\. Alao, R\. Rodriguez, R\. Ayub, R\. Murthy, R\. Nayani, R\. Mitra, R\. Parthasarathy, R\. Li, R\. Hogan, R\. Battey, R\. Wang, R\. Howes, R\. Rinott, S\. Mehta, S\. Siby, S\. J\. Bondu, S\. Datta, S\. Chugh, S\. Hunt, S\. Dhillon, S\. Sidorov, S\. Pan, S\. Mahajan, S\. Verma, S\. Yamamoto, S\. Ramaswamy, S\. Lindsay, S\. Lindsay, S\. Feng, S\. Lin, S\. C\. Zha, S\. Patil, S\. Shankar, S\. Zhang, S\. Zhang, S\. Wang, S\. Agarwal, S\. Sajuyigbe, S\. Chintala, S\. Max, S\. Chen, S\. Kehoe, S\. Satterfield, S\. Govindaprasad, S\. Gupta, S\. Deng, S\. Cho, S\. Virk, S\. Subramanian, S\. Choudhury, S\. Goldman, T\. Remez, T\. Glaser, T\. Best, T\. Koehler, T\. Robinson, T\. Li, T\. Zhang, T\. Matthews, T\. Chou, T\. Shaked, V\. Vontimitta, V\. Ajayi, V\. Montanez, V\. Mohan, V\. S\. Kumar, V\. Mangla, V\. Ionescu, V\. Poenaru, V\. T\. Mihailescu, V\. Ivanov, W\. Li, W\. Wang, W\. Jiang, W\. Bouaziz, W\. Constable, X\. Tang, X\. Wu, X\. Wang, X\. Wu, X\. Gao, Y\. Kleinman, Y\. Chen, Y\. Hu, Y\. Jia, Y\. Qi, Y\. Li, Y\. Zhang, Y\. Zhang, Y\. Adi, Y\. Nam, Yu, Wang, Y\. Zhao, Y\. Hao, Y\. Qian, Y\. Li, Y\. He, Z\. Rait, Z\. DeVito, Z\. Rosnbrick, Z\. Wen, Z\. Yang, Z\. Zhao, and Z\. MaThe llama 3 herd of models\.External Links:2407\.21783,[Link](https://arxiv.org/abs/2407.21783)Cited by:[§3\.1](https://arxiv.org/html/2610.41856#S3.SS1.p1.1)\.
- Heet al\.\(2025\)G\. He, N\. Aishwarya, and U\. GadirajuIs conversational xai all you need? human\-ai decision making with a conversational xai assistant\.InProceedings of the 30th International Conference on Intelligent User Interfaces,IUI ’25,New York, NY, USA,pp\. 907–924\.External Links:ISBN 9798400713064,[Link](https://doi.org/10.1145/3708359.3712133),[Document](https://dx.doi.org/10.1145/3708359.3712133)Cited by:[§1](https://arxiv.org/html/2610.41856#S1.p1.1)\.
- Huanget al\.\(2025\)L\. Huang, W\. Yu, W\. Ma, W\. Zhong, Z\. Feng, H\. Wang, Q\. Chen, W\. Peng, X\. Feng, B\. Qin,et al\.A survey on hallucination in large language models: principles, taxonomy, challenges, and open questions\.ACM Transactions on Information Systems43\(2\),pp\. 1–55\.Cited by:[§4\.2](https://arxiv.org/html/2610.41856#S4.SS2.p1.1)\.
- Kahn \(1994\)M\. KahnDiabetes\.Note:UCI Machine Learning RepositoryDOI: https://doi\.org/10\.24432/C5T59GCited by:[§6\.2\.1](https://arxiv.org/html/2610.41856#S6.SS2.SSS1.p1.1)\.
- Kauret al\.\(2020\)H\. Kaur, H\. Nori, S\. Jenkins, R\. Caruana, H\. Wallach, and J\. Wortman VaughanInterpreting interpretability: understanding data scientists’ use of interpretability tools for machine learning\.InProceedings of the 2020 CHI Conference on Human Factors in Computing Systems,CHI ’20,New York, NY, USA,pp\. 1–14\.External Links:ISBN 9781450367080,[Link](https://doi.org/10.1145/3313831.3376219),[Document](https://dx.doi.org/10.1145/3313831.3376219)Cited by:[§6\.2\.1](https://arxiv.org/html/2610.41856#S6.SS2.SSS1.p1.1)\.
- Lakkarajuet al\.\(2022\)H\. Lakkaraju, D\. Slack, Y\. Chen, C\. Tan, and S\. SinghRethinking explainability as a dialogue: a practitioner’s perspective\.External Links:2202\.01875,[Link](https://arxiv.org/abs/2202.01875)Cited by:[§1](https://arxiv.org/html/2610.41856#S1.p1.1)\.
- Liet al\.\(2024\)Z\. Li, Z\. Z\. Chen, M\. Ross, P\. Huber, S\. Moon, Z\. Lin, L\. Dong, A\. Sagar, X\. Yan, and P\. A\. CrookLarge language models as zero\-shot dialogue state tracker through function calling\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),L\. Ku, A\. Martins, and V\. Srikumar \(Eds\.\),Bangkok, Thailand,pp\. 8688–8704\.External Links:[Link](https://aclanthology.org/2024.acl-long.471/),[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.471)Cited by:[§1](https://arxiv.org/html/2610.41856#S1.p1.1)\.
- Liaoet al\.\(2020\)Q\. V\. Liao, D\. Gruen, and S\. MillerQuestioning the ai: informing design practices for explainable ai user experiences\.InProceedings of the 2020 CHI Conference on Human Factors in Computing Systems,CHI ’20,New York, NY, USA,pp\. 1–15\.External Links:ISBN 9781450367080,[Link](https://doi.org/10.1145/3313831.3376590),[Document](https://dx.doi.org/10.1145/3313831.3376590)Cited by:[§D\.1](https://arxiv.org/html/2610.41856#A4.SS1.p1.1),[§3\.2](https://arxiv.org/html/2610.41856#S3.SS2.p1.1),[§6\.2\.1](https://arxiv.org/html/2610.41856#S6.SS2.SSS1.p2.1)\.
- Liao and Varshney \(2022\)Q\. V\. Liao and K\. R\. VarshneyHuman\-centered explainable ai \(xai\): from algorithms to user experiences\.External Links:2110\.10790,[Link](https://arxiv.org/abs/2110.10790)Cited by:[§3\.2](https://arxiv.org/html/2610.41856#S3.SS2.p1.1)\.
- Liuet al\.\(2025\)W\. Liu, X\. Huang, X\. Zeng, x\. hao, S\. Yu, D\. Li, S\. Wang, W\. Gan, Z\. Liu, Y\. Yu, Z\. WANG, Y\. Wang, W\. Ning, Y\. Hou, B\. Wang, C\. Wu, W\. Xinzhi, Y\. Liu, Y\. Wang, D\. Tang, D\. Tu, L\. Shang, X\. Jiang, R\. Tang, D\. Lian, Q\. Liu, and E\. ChenToolACE: winning the points of llm function calling\.InInternational Conference on Representation Learning,Y\. Yue, A\. Garg, N\. Peng, F\. Sha, and R\. Yu \(Eds\.\),Vol\.2025,pp\. 41359–41381\.External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2025/file/663865ea167425c6c562cb0b6bcf76c7-Paper-Conference.pdf)Cited by:[§3\.1](https://arxiv.org/html/2610.41856#S3.SS1.p1.1)\.
- Lundberg and Lee \(2017\)S\. M\. Lundberg and S\. LeeA unified approach to interpreting model predictions\.Advances in neural information processing systems30\.Cited by:[§1](https://arxiv.org/html/2610.41856#S1.p1.1)\.
- Malandriet al\.\(2023\)L\. Malandri, F\. Mercorio, M\. Mezzanzanica, and N\. NobaniConvxai: a system for multimodal interaction with any black\-box explainer\.Cognitive Computation15\(2\),pp\. 613–644\.Cited by:[§1](https://arxiv.org/html/2610.41856#S1.p4.1)\.
- Miller \(2019\)T\. MillerExplanation in artificial intelligence: Insights from the social sciences\.Artificial Intelligence267,pp\. 1–38\(en\)\.External Links:ISSN 0004\-3702,[Link](https://www.sciencedirect.com/science/article/pii/S0004370218305988),[Document](https://dx.doi.org/10.1016/j.artint.2018.07.007)Cited by:[§1](https://arxiv.org/html/2610.41856#S1.p1.1)\.
- Mindlinet al\.\(2024\)D\. Mindlin, A\. S\. Robrecht, M\. Morasch, and P\. CimianoMeasuring user understanding in dialogue\-based xai systems\.InECAI 2024,pp\. 1148–1155\.Cited by:[§1](https://arxiv.org/html/2610.41856#S1.p1.1)\.
- Mitchellet al\.\(2019\)M\. Mitchell, S\. Wu, A\. Zaldivar, P\. Barnes, L\. Vasserman, B\. Hutchinson, E\. Spitzer, I\. D\. Raji, and T\. GebruModel cards for model reporting\.InProceedings of the conference on fairness, accountability, and transparency,pp\. 220–229\.Cited by:[§1](https://arxiv.org/html/2610.41856#S1.p1.1)\.
- Mothilalet al\.\(2020\)R\. K\. Mothilal, A\. Sharma, and C\. TanExplaining machine learning classifiers through diverse counterfactual explanations\.InProceedings of the 2020 conference on fairness, accountability, and transparency,pp\. 607–617\.Cited by:[§1](https://arxiv.org/html/2610.41856#S1.p1.1)\.
- Nautaet al\.\(2023\)M\. Nauta, J\. Trienes, S\. Pathak, E\. Nguyen, M\. Peters, Y\. Schmitt, J\. Schlötterer, M\. van Keulen, and C\. SeifertFrom anecdotal evidence to quantitative evaluation methods: a systematic review on evaluating explainable ai\.ACM Comput\. Surv\.55\(13s\)\.External Links:ISSN 0360\-0300,[Link](https://doi.org/10.1145/3583558),[Document](https://dx.doi.org/10.1145/3583558)Cited by:[§7](https://arxiv.org/html/2610.41856#S7.p1.1)\.
- Nguyenet al\.\(2023\)V\. B\. Nguyen, J\. Schlötterer, and C\. SeifertFrom black boxes to conversations: incorporating xai in a conversational agent\.InWorld Conference on Explainable Artificial Intelligence,pp\. 71–96\.Cited by:[Table 1](https://arxiv.org/html/2610.41856#S1.T1.2.1.4.1),[§1](https://arxiv.org/html/2610.41856#S1.p1.1),[§1](https://arxiv.org/html/2610.41856#S1.p2.1),[§2](https://arxiv.org/html/2610.41856#S2.p1.1),[§2](https://arxiv.org/html/2610.41856#S2.p2.1),[§2](https://arxiv.org/html/2610.41856#S2.p3.1),[§4\.1](https://arxiv.org/html/2610.41856#S4.SS1.p1.1),[§6](https://arxiv.org/html/2610.41856#S6.p1.1)\.
- Nguyenet al\.\(2024\)V\. B\. Nguyen, J\. Schlötterer, and C\. SeifertXagent: a conversational xai agent harnessing the power of large language models\.InJoint Proceedings of the xAI 2024 Late\-breaking Work, Demos and Doctoral Consortium co\-located with the 2nd World Conference on eXplainable Artificial Intelligence \(xAI\-2024\), Valletta, Malta, July 17,pp\. 273–280\.Cited by:[§7](https://arxiv.org/html/2610.41856#S7.p1.1)\.
- Nussbaumet al\.\(2025\)Z\. Nussbaum, J\. X\. Morris, A\. Mulyar, and B\. DuderstadtNomic embed: training a reproducible long context text embedder\.Transactions on Machine Learning Research\.Note:Reproducibility CertificationExternal Links:ISSN 2835\-8856,[Link](https://openreview.net/forum?id=IPmzyQSiQE)Cited by:[§7](https://arxiv.org/html/2610.41856#S7.p1.1)\.
- Penget al\.\(2017\)B\. Peng, X\. Li, L\. Li, J\. Gao, A\. Celikyilmaz, S\. Lee, and K\. WongComposite task\-completion dialogue policy learning via hierarchical deep reinforcement learning\.InProceedings of the 2017 Conference on Empirical Methods in Natural Language Processing,M\. Palmer, R\. Hwa, and S\. Riedel \(Eds\.\),Copenhagen, Denmark,pp\. 2231–2240\.External Links:[Link](https://aclanthology.org/D17-1237/),[Document](https://dx.doi.org/10.18653/v1/D17-1237)Cited by:[§6\.2\.1](https://arxiv.org/html/2610.41856#S6.SS2.SSS1.p3.1)\.
- Prasaiet al\.\(2025\)S\. Prasai, M\. Du, Y\. Zhang, and F\. YangKnowThyself: an agentic assistant for llm interpretability\.External Links:2511\.03878,[Link](https://arxiv.org/abs/2511.03878)Cited by:[§7](https://arxiv.org/html/2610.41856#S7.p1.1)\.
- Pushkarnaet al\.\(2022\)M\. Pushkarna, A\. Zaldivar, and O\. KjartanssonData cards: purposeful and transparent dataset documentation for responsible ai\.InProceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency,pp\. 1776–1826\.Cited by:[§1](https://arxiv.org/html/2610.41856#S1.p1.1)\.
- Rastogiet al\.\(2020\)A\. Rastogi, X\. Zang, S\. Sunkara, R\. Gupta, and P\. KhaitanTowards scalable multi\-domain conversational agents: the schema\-guided dialogue dataset\.Proceedings of the AAAI Conference on Artificial Intelligence34\(05\),pp\. 8689–8696\.External Links:[Link](https://ojs.aaai.org/index.php/AAAI/article/view/6394),[Document](https://dx.doi.org/10.1609/aaai.v34i05.6394)Cited by:[§1](https://arxiv.org/html/2610.41856#S1.p1.1)\.
- Rosset al\.\(2021\)A\. Ross, A\. Marasović, and M\. E\. PetersExplaining nlp models via minimal contrastive editing \(mice\)\.InFindings of the Association for Computational Linguistics: ACL\-IJCNLP 2021,pp\. 3840–3852\.Cited by:[§1](https://arxiv.org/html/2610.41856#S1.p1.1)\.
- Shenet al\.\(2023\)H\. Shen, C\. Huang, T\. Wu, and T\. K\. HuangConvXAI: delivering heterogeneous AI explanations via conversations to support human\-AI scientific writing\.InComputer Supported Cooperative Work and Social Computing,CSCW ’23 Companion,New York, NY, USA,pp\. 384–387\.External Links:ISBN 9798400701290,[Link](https://doi.org/10.1145/3584931.3607492),[Document](https://dx.doi.org/10.1145/3584931.3607492)Cited by:[§1](https://arxiv.org/html/2610.41856#S1.p1.1)\.
- Slacket al\.\(2023\)D\. Slack, S\. Krishna, H\. Lakkaraju, and S\. SinghExplaining machine learning models with interactive natural language conversations using TalkToModel\.Nature Machine Intelligence\.External Links:ISSN 2522\-5839,[Document](https://dx.doi.org/10.1038/s42256-023-00692-8),[Link](https://doi.org/10.1038/s42256-023-00692-8)Cited by:[Table 1](https://arxiv.org/html/2610.41856#S1.T1.2.1.2.1),[§1](https://arxiv.org/html/2610.41856#S1.p1.1),[§1](https://arxiv.org/html/2610.41856#S1.p2.1),[§2](https://arxiv.org/html/2610.41856#S2.p1.1),[§2](https://arxiv.org/html/2610.41856#S2.p2.1),[§2](https://arxiv.org/html/2610.41856#S2.p3.1),[§4\.1](https://arxiv.org/html/2610.41856#S4.SS1.p1.1),[§6\.2\.1](https://arxiv.org/html/2610.41856#S6.SS2.SSS1.p1.1),[§6\.2](https://arxiv.org/html/2610.41856#S6.SS2.p1.1),[§6](https://arxiv.org/html/2610.41856#S6.p1.1)\.
- Suet al\.\(2016\)P\. Su, M\. Gašić, N\. Mrkšić, L\. M\. Rojas\-Barahona, S\. Ultes, D\. Vandyke, T\. Wen, and S\. YoungOn\-line active reward learning for policy optimisation in spoken dialogue systems\.InProceedings of the 54th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),K\. Erk and N\. A\. Smith \(Eds\.\),Berlin, Germany,pp\. 2431–2441\.External Links:[Link](https://aclanthology.org/P16-1230/),[Document](https://dx.doi.org/10.18653/v1/P16-1230)Cited by:[§6\.2\.1](https://arxiv.org/html/2610.41856#S6.SS2.SSS1.p3.1)\.
- Tamberet al\.\(2025\)M\. S\. Tamber, F\. S\. Bao, C\. Xu, G\. Luo, S\. Kazi, M\. Bae, M\. Li, O\. Mendelevitch, R\. Qu, and J\. LinBenchmarking LLM faithfulness in RAG with evolving leaderboards\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: Industry Track,S\. Potdar, L\. Rojas\-Barahona, and S\. Montella \(Eds\.\),Suzhou \(China\),pp\. 799–811\.External Links:[Link](https://aclanthology.org/2025.emnlp-industry.54/),[Document](https://dx.doi.org/10.18653/v1/2025.emnlp-industry.54),ISBN 979\-8\-89176\-333\-3Cited by:[§5](https://arxiv.org/html/2610.41856#S5.p1.1)\.
- Teamet al\.\(2025\)G\. Team, A\. Kamath, J\. Ferret, S\. Pathak, N\. Vieillard, R\. Merhej, S\. Perrin, T\. Matejovicova, A\. Ramé, M\. Rivière, L\. Rouillard, T\. Mesnard, G\. Cideron, J\. Grill, S\. Ramos, E\. Yvinec, M\. Casbon, E\. Pot, I\. Penchev, G\. Liu, F\. Visin, K\. Kenealy, L\. Beyer, X\. Zhai, A\. Tsitsulin, R\. Busa\-Fekete, A\. Feng, N\. Sachdeva, B\. Coleman, Y\. Gao, B\. Mustafa, I\. Barr, E\. Parisotto, D\. Tian, M\. Eyal, C\. Cherry, J\. Peter, D\. Sinopalnikov, S\. Bhupatiraju, R\. Agarwal, M\. Kazemi, D\. Malkin, R\. Kumar, D\. Vilar, I\. Brusilovsky, J\. Luo, A\. Steiner, A\. Friesen, A\. Sharma, A\. Sharma, A\. M\. Gilady, A\. Goedeckemeyer, A\. Saade, A\. Feng, A\. Kolesnikov, A\. Bendebury, A\. Abdagic, A\. Vadi, A\. György, A\. S\. Pinto, A\. Das, A\. Bapna, A\. Miech, A\. Yang, A\. Paterson, A\. Shenoy, A\. Chakrabarti, B\. Piot, B\. Wu, B\. Shahriari, B\. Petrini, C\. Chen, C\. L\. Lan, C\. A\. Choquette\-Choo, C\. Carey, C\. Brick, D\. Deutsch, D\. Eisenbud, D\. Cattle, D\. Cheng, D\. Paparas, D\. S\. Sreepathihalli, D\. Reid, D\. Tran, D\. Zelle, E\. Noland, E\. Huizenga, E\. Kharitonov, F\. Liu, G\. Amirkhanyan, G\. Cameron, H\. Hashemi, H\. Klimczak\-Plucińska, H\. Singh, H\. Mehta, H\. T\. Lehri, H\. Hazimeh, I\. Ballantyne, I\. Szpektor, I\. Nardini, J\. Pouget\-Abadie, J\. Chan, J\. Stanton, J\. Wieting, J\. Lai, J\. Orbay, J\. Fernandez, J\. Newlan, J\. Ji, J\. Singh, K\. Black, K\. Yu, K\. Hui, K\. Vodrahalli, K\. Greff, L\. Qiu, M\. Valentine, M\. Coelho, M\. Ritter, M\. Hoffman, M\. Watson, M\. Chaturvedi, M\. Moynihan, M\. Ma, N\. Babar, N\. Noy, N\. Byrd, N\. Roy, N\. Momchev, N\. Chauhan, N\. Sachdeva, O\. Bunyan, P\. Botarda, P\. Caron, P\. K\. Rubenstein, P\. Culliton, P\. Schmid, P\. G\. Sessa, P\. Xu, P\. Stanczyk, P\. Tafti, R\. Shivanna, R\. Wu, R\. Pan, R\. Rokni, R\. Willoughby, R\. Vallu, R\. Mullins, S\. Jerome, S\. Smoot, S\. Girgin, S\. Iqbal, S\. Reddy, S\. Sheth, S\. Põder, S\. Bhatnagar, S\. R\. Panyam, S\. Eiger, S\. Zhang, T\. Liu, T\. Yacovone, T\. Liechty, U\. Kalra, U\. Evci, V\. Misra, V\. Roseberry, V\. Feinberg, V\. Kolesnikov, W\. Han, W\. Kwon, X\. Chen, Y\. Chow, Y\. Zhu, Z\. Wei, Z\. Egyed, V\. Cotruta, M\. Giang, P\. Kirk, A\. Rao, K\. Black, N\. Babar, J\. Lo, E\. Moreira, L\. G\. Martins, O\. Sanseviero, L\. Gonzalez, Z\. Gleicher, T\. Warkentin, V\. Mirrokni, E\. Senter, E\. Collins, J\. Barral, Z\. Ghahramani, R\. Hadsell, Y\. Matias, D\. Sculley, S\. Petrov, N\. Fiedel, N\. Shazeer, O\. Vinyals, J\. Dean, D\. Hassabis, K\. Kavukcuoglu, C\. Farabet, E\. Buchatskaya, J\. Alayrac, R\. Anil, Dmitry, Lepikhin, S\. Borgeaud, O\. Bachem, A\. Joulin, A\. Andreev, C\. Hardin, R\. Dadashi, and L\. HussenotGemma 3 technical report\.External Links:2503\.19786,[Link](https://arxiv.org/abs/2503.19786)Cited by:[§7](https://arxiv.org/html/2610.41856#S7.p1.1)\.
- Touvronet al\.\(2023\)H\. Touvron, L\. Martin, K\. Stone, P\. Albert, A\. Almahairi, Y\. Babaei, N\. Bashlykov, S\. Batra, P\. Bhargava, S\. Bhosale, D\. Bikel, L\. Blecher, C\. C\. Ferrer, M\. Chen, G\. Cucurull, D\. Esiobu, J\. Fernandes, J\. Fu, W\. Fu, B\. Fuller, C\. Gao, V\. Goswami, N\. Goyal, A\. Hartshorn, S\. Hosseini, R\. Hou, H\. Inan, M\. Kardas, V\. Kerkez, M\. Khabsa, I\. Kloumann, A\. Korenev, P\. S\. Koura, M\. Lachaux, T\. Lavril, J\. Lee, D\. Liskovich, Y\. Lu, Y\. Mao, X\. Martinet, T\. Mihaylov, P\. Mishra, I\. Molybog, Y\. Nie, A\. Poulton, J\. Reizenstein, R\. Rungta, K\. Saladi, A\. Schelten, R\. Silva, E\. M\. Smith, R\. Subramanian, X\. E\. Tan, B\. Tang, R\. Taylor, A\. Williams, J\. X\. Kuan, P\. Xu, Z\. Yan, I\. Zarov, Y\. Zhang, A\. Fan, M\. Kambadur, S\. Narang, A\. Rodriguez, R\. Stojnic, S\. Edunov, and T\. ScialomLlama 2: open foundation and fine\-tuned chat models\.External Links:2307\.09288,[Link](https://arxiv.org/abs/2307.09288)Cited by:[§7](https://arxiv.org/html/2610.41856#S7.p1.1)\.
- Vanbrabantet al\.\(2025\)S\. Vanbrabant, G\. Eerlings, G\. A\. Rovelo Ruiz, and D\. VanackenECHO: enhancing conversational explainable ai through tool\-augmented language models\.Proc\. ACM Hum\.\-Comput\. Interact\.9\(4\)\.External Links:[Link](https://doi.org/10.1145/3734191),[Document](https://dx.doi.org/10.1145/3734191)Cited by:[§7](https://arxiv.org/html/2610.41856#S7.p1.1)\.
- Wanget al\.\(2024a\)Q\. Wang, T\. Anikina, N\. Feldhus, J\. Genabith, L\. Hennig, and S\. MöllerLLMCheckup: conversational examination of large language models via interpretability tools and self\-explanations\.InProceedings of the Third Workshop on Bridging Human–Computer Interaction and Natural Language Processing,S\. L\. Blodgett, A\. C\. Curry, S\. Dey, M\. Madaio, A\. Nenkova, D\. Yang, and Z\. Xiao \(Eds\.\),Mexico City, Mexico,pp\. 89–104\.External Links:[Link](https://aclanthology.org/2024.hcinlp-1.9),[Document](https://dx.doi.org/10.18653/v1/2024.hcinlp-1.9)Cited by:[§1](https://arxiv.org/html/2610.41856#S1.p1.1),[§1](https://arxiv.org/html/2610.41856#S1.p4.1)\.
- Wanget al\.\(2024b\)Q\. Wang, T\. Anikina, N\. Feldhus, S\. Ostermann, and S\. MöllerCoXQL: a dataset for parsing explanation requests in conversational XAI systems\.InFindings of the Association for Computational Linguistics: EMNLP 2024,Y\. Al\-Onaizan, M\. Bansal, and Y\. Chen \(Eds\.\),Miami, Florida, USA,pp\. 1410–1422\.External Links:[Link](https://aclanthology.org/2024.findings-emnlp.76/),[Document](https://dx.doi.org/10.18653/v1/2024.findings-emnlp.76)Cited by:[§4\.1](https://arxiv.org/html/2610.41856#S4.SS1.SSS0.Px1.p1.1)\.
- Wanget al\.\(2025\)Q\. Wang, T\. Anikina, N\. Feldhus, S\. Ostermann, F\. Splitt, J\. Li, Y\. Tsoneva, S\. Möller, and V\. SchmittMultilingual datasets for custom input extraction and explanation requests parsing in conversational XAI systems\.InFindings of the Association for Computational Linguistics: EMNLP 2025,C\. Christodoulopoulos, T\. Chakraborty, C\. Rose, and V\. Peng \(Eds\.\),Suzhou, China,pp\. 534–555\.External Links:[Link](https://aclanthology.org/2025.findings-emnlp.29/),ISBN 979\-8\-89176\-335\-7Cited by:[§4\.1](https://arxiv.org/html/2610.41856#S4.SS1.SSS0.Px1.p1.1)\.
- Xuet al\.\(2025\)A\. Xu, S\. Bansal, Y\. Ming, S\. Yavuz, and S\. JotyDoes context matter? ContextualJudgeBench for evaluating LLM\-based judges in contextual settings\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),Vienna, Austria,pp\. 9541–9564\.External Links:[Link](https://aclanthology.org/2025.acl-long.470/),[Document](https://dx.doi.org/10.18653/v1/2025.acl-long.470),ISBN 979\-8\-89176\-251\-0Cited by:[§5](https://arxiv.org/html/2610.41856#S5.p1.1)\.
- Zhanget al\.\(2025\)T\. Zhang, M\. Zhang, W\. Y\. Low, X\. J\. Yang, and B\. A\. LiConversational explanations: discussing explainable ai with non\-ai experts\.InProceedings of the 30th International Conference on Intelligent User Interfaces,IUI ’25,New York, NY, USA,pp\. 409–424\.External Links:ISBN 9798400713064,[Link](https://doi.org/10.1145/3708359.3712143),[Document](https://dx.doi.org/10.1145/3708359.3712143)Cited by:[§1](https://arxiv.org/html/2610.41856#S1.p1.1)\.

## Appendix AParsing

##### Failure in single\-turn parsing\.

Figure[6](https://arxiv.org/html/2610.41856#A1.F6)shows the comparison between single\-turn and multi\-turn parsing\. For multi\-turn parsing, LLMs should be able to utilize the context from previous turns\. As illustrated in the third example in Figure[6](https://arxiv.org/html/2610.41856#A1.F6), users can simply use pronouns like “it” to reference previous questions without repeating the actual question from the last turn\. Such cases would fail in single\-turn parsing, as there is no conversational context to resolve what “it” refers to\.

##### From single\-turn to multi\-turn\.

Table[3](https://arxiv.org/html/2610.41856#S4.T3)To build the multi\-turn version \(Figure[6](https://arxiv.org/html/2610.41856#A1.F6), right side\), related consecutive questions that operated on the same input text were grouped together into multi\-turn conversation sets\. Specifically, questions that originally repeated the full instance text had those explicit instance references replaced with pronouns like “it” turning them into natural follow\-up turns\. This way, the conversation flows naturally: the first turn establishes the target text, and subsequent turns refer back to it implicitly — just as a human would in real dialogue\. In total, we created 10 multi\-turn dialogues with averaging 4–6 turns each\.

Figure 4:CoXQLexamples\.defexplain\_how\_to\_be\_that\(instance:str,target\_label:str=None,number\_of\_iterations:int=1\)\-\>str:

"""

Callthiswheneveryouneedtoexplainhowtomodifytheinstanceminimallytoachievethetargetprediction\(counterfactual\)\.Forexample,userasked"HowcanIchangethisinstancetogetpredictionX?"

Thetoolsuggestschangesneededtoflipthecurrentpredictiontothedesiredone\.

Args:

instance:Theinputinstance\.

target\_label:\(Optional\)Thedesiredpredictionorclass\.IfNone,suggestschangestoflipthecurrentprediction\.

number\_of\_iterations:Numberofcounterfactualsuggestionstogenerate\.

Output:

Counterfactualchanges\(e\.g\.,"increasefeatureXby0\.2"\)\.

"""

Figure 5:Counterfactual Function Specification Example \(Python\)![Refer to caption](https://arxiv.org/html/2610.41856v1/figures/parsing.png)

Figure 6:Comparison between single\-turn and multi\-turn parsing\.

## Appendix BRealism Validation

Because the conversations in our dataset were generated using GPT\-5\.1, we additionally evaluated their realism from a human perspective to verify that they resemble plausible explanatory dialogues\.

Specifically, we asked five members of our research group with a background in machine learning to assess the realism of user utterances in the synthetically generated explanatory conversations\. Participants were asked to imagine themselves as users seeking to understand how a classification model makes its decisions\. They were then presented with generated dialogues between aUserand anExplainer\. Their task was to judge whether the User’s messages resembled authentic user questions, reflected a consistent level of knowledge, and remained on topic throughout the conversation\.

For each conversation, participants provided a rating on a 5\-point Likert scale ranging from 1 \(completely unrealistic\) to 5 \(completely realistic\)\. In addition to the numerical rating, respondents were required to provide a justification for their score, allowing us to collect both quantitative assessments and qualitative rationales regarding perceived conversational realism\.

The average score across all conversations was 3\.75, suggesting that the generated dialogues achieved a reasonable level of perceived realism\.

## Appendix CQualitative examples

In this section, we present additional qualitative results illustrating cases where the original conversational XAI agents fail to capture dialogue context and howSimXAIimproves their responses\.[Table7](https://arxiv.org/html/2610.41856#A3.T7)provides examples for each existing agent\. Each agent operates on different tasks and exhibits different types of contextualization failures\.SimXAIimproves these responses across all cases without requiring task\-specific or architecture\-specific adaptations\.

AgentSpeakerOriginalSimXAITalkToModelUserWhat if the glucose was 50?Agentwith the glucose level set to 50, it is still likely to have no diabeteswith the glucose level set to 50, it is still likely to have no diabetes\.UserWhat if it was 105?Agent… glucose is set to 50\.0 and bmi is set to 105\.0 is predicted likely to have diabetes\.with the glucose level set to 105, it is still likely to have no diabetes\.InterrolangUserWhat is the prediction for instance ID 81?AgentThe instance with id equal to 81 is predicted True\.The instance with id 81 is predicted True\.UserHow to change the instance to get a different prediction?AgentThis is likely out of my expertise, can I help you with something else?To get a different prediction, here are the list of words …XAgentUserWhat is the prediction for this instanceAgentThe prediction for this instance is You will not get the loan…\.The prediction for this instance is you will not get the loanUserWhy?AgentSorry, I am not exactly sure whether I understood your question correctly\. Did you mean any of the below?…The model utilized all features, … The following graph depicts key features that influence this prediction…

Table 7:Qualitative results
## Appendix DUser Study Details

### D\.1Question Pool

[Table8](https://arxiv.org/html/2610.41856#A4.T8)presents the three tasks and the corresponding question templates that participants were asked to answer\. The questions cover several categories of the XAI question taxonomy[Liao et al\. \(2020\)](https://arxiv.org/html/2610.41856#bib.bib18), including data understanding, local explanations, and counterfactual inspection\.

Some question templates contain placeholders that are replaced with specific values during the study\. For these questions, the feature and value are randomly selected when generating the question\. The study uses two IDs, which are randomly assigned to the conversational agents\.

Each task includes multiple possible questions, and all three tasks are presented to each participant\. For each task, questions are randomly selected from the pool without replacement, ensuring that within a single session participants receive as many different questions as possible\.

Several selection rules are applied\.Bolded questionsare always asked, and all types are always included\. In addition,italicized questionsrepresent follow\-up questions that are always selected when their preceding question appears\.

TaskTypeQuestion TemplateExplore the dataset and answer the following question\(s\)\.Data Exploration \(global\)•Number of features in the dataset is:•Is the feature<feature\>included in the dataset?•The maximum value of<feature\>recorded in the dataset is:•Find the minimum value of<feature\>recorded in the dataset and choose the correct value:Explorepatient with ID<patient\_id\>and answer the following question\(s\)\.Data Exploration \(local\)•Find the value of<feature\_A\>for this patient and choose the correct value\.•What is the value of<feature\_A\>for this patient:•How about the<feature\_B\>value of this patient?•How about this patient’s<feature\_B\>value?Find the prediction of the model forpatient with ID<patient\_id\>and answer the following question\(s\)\.Local Explanation•What does the model predict for this patient?•The most important feature for the prediction is:•The least important feature for the prediction is:•Is<feature\>used for the prediction for this patient?•Top 3 most important features for the prediction are:Counterfactual•What would be the prediction if the value of<feature\>is increased to<value\>?•What would be the prediction if the value of<feature\>is decreased to<value\>?•What would be the prediction if it is decreased/increased to<value\>?

Table 8:Tasks and corresponding questions used in the user study\.
### D\.2Instructions and User Interface

[Figure7](https://arxiv.org/html/2610.41856#A4.F7)shows screenshots of general instructions and informed consent,[Figure8](https://arxiv.org/html/2610.41856#A4.F8)screenshots of information on the overall study procedure and how to conduct the tasks in the study and[Figure9](https://arxiv.org/html/2610.41856#A4.F9)shows the interface, participants interact with\.

### D\.3Results

Across 30 completed sessions, integratingSimXAIintoTalkToModelimproved both objective task performance and subjective user experience\. Unless otherwise noted, we report two\-tailed pairedtt\-tests over participant\-level scores\. Table[10](https://arxiv.org/html/2610.41856#A4.T10)summarizes the objective outcomes, Table[11](https://arxiv.org/html/2610.41856#A4.T11)reports the BUS\-11 item means, Table[12](https://arxiv.org/html/2610.41856#A4.T12)presents the statistical comparisons, and Table[9](https://arxiv.org/html/2610.41856#A4.T9)summarizes the pairwise preferences\.

##### Objective performance\.

As shown in Table[10](https://arxiv.org/html/2610.41856#A4.T10),SimXAIsubstantially improves users’ ability to answer explanation\-related questions correctly\. If we count all cases when users indicated that the question is not answerable \(“unable to answer”\) as incorrect, accuracy increases from 53\.3±\\pm19\.6 withTalkToModelto 84\.1±\\pm17\.7 withSimXAI\. The same pattern holds when excludingunable to answerresponses from the denominator, with accuracy increasing from 59\.0±\\pm21\.3 to 92\.1±\\pm11\.6\. The corresponding paired tests in Table[12](https://arxiv.org/html/2610.41856#A4.T12)confirm that both improvements are highly significant \(p<\.001p<\.001in both cases\)\.

The aggregate counts in Table[10](https://arxiv.org/html/2610.41856#A4.T10)show the same trend: participants produced many more correct answers withSimXAI\(227\) than withTalkToModel\(138\)\. Importantly, both systems generated the same number of explicitunable to answerresponses \(24 each\), suggesting that the gain came primarily from better answers rather than fewer abstentions\.

SimXAI also made interactions more efficient\. Compared withTalkToModel, it reduced average chat time \(7\.4±\\pm3\.3 minutes vs\. 5\.8±\\pm3\.0 minutes\) and average conversation length \(12\.4±\\pm4\.1 turns vs\. 10\.0±\\pm1\.8 turns\)\. This pattern is also reflected in the turns\-per\-correct\-answer measure reported in Table[10](https://arxiv.org/html/2610.41856#A4.T10), which was notably lower forSimXAI\(1\.32\) than forTalkToModel\(2\.70\)\. Overall, users reached correct answers more quickly and with fewer conversational steps\.

##### Subjective evaluation\.

The subjective ratings mirror the objective results\. Table[11](https://arxiv.org/html/2610.41856#A4.T11)shows thatSimXAIreceived higher mean ratings on 10 of the 11 BUS\-11 items, and Table[12](https://arxiv.org/html/2610.41856#A4.T12)shows that 9 of these differences are statistically significant\. The strongest improvements are observed on dimensions central to multi\-turn explanatory dialogue, especially clarity, context retention, helpfulness, and informativeness\. For example, users rateSimXAIhigher for clear communication \(4\.60 vs\. 3\.33\), keeping track of conversational context \(4\.37 vs\. 3\.13\), and helping them achieve their goal \(4\.37 vs\. 3\.10\)\.

The same overall trend holds across the remaining items:SimXAIis rated as easier to understand, more accurate, and easier to use for finding information\. Only two BUS\-11 items do not differ significantly\. First, privacy communication is rated relatively low in both conditions, indicating that neither system communicated privacy\-related issues particularly well\. Second, perceived waiting/response time slightly favorsTalkToModelnumerically, but the difference is not significant\. Taken together, the BUS\-11 results show thatSimXAImainly improves the quality and usefulness of the interaction\. These aspects are central to ConvXAI\.

##### Pairwise comparison\.

The direct comparison results in Table[9](https://arxiv.org/html/2610.41856#A4.T9)further support this pattern\. A large majority of participants preferSimXAIoverall \(80\.0%\), while only 16\.7% preferTalkToModeland 3\.3% report no preference\. The same distribution appears for perceived task effectiveness: 80\.0% judgeSimXAImore helpful, 16\.7% preferTalkToModel, and 3\.3% consider the two systems equally helpful\. These preferences are consistent with both the objective performance results and the BUS\-11 ratings\.

##### Summary\.

Overall, the results across Tables[10](https://arxiv.org/html/2610.41856#A4.T10)–[9](https://arxiv.org/html/2610.41856#A4.T9)show a consistent pattern: replacingTalkToModel’s conversational component withSimXAIimproves answer quality, increases task success, reduces interaction cost, and leads to more favorable user judgments\. The clearest gains appear on aspects that matter most for explanation\-oriented multi\-turn interaction, namely clarity, contextual coherence, helpfulness, and efficiency\.

Comparison itemTalkToModelSimXAIOverall preference5 \(16\.7%\)24 \(80\.0%\)No preference1 \(3\.3%\)Task effectiveness5 \(16\.7%\)24 \(80\.0%\)Equally helpful1 \(3\.3%\)Table 9:Pairwise comparison results\. Most participants preferredSimXAIand judged it more effective\.MetricTalkToModelSimXAIParticipants3030Accuracy \(unable = wrong\)53\.3±\\pm19\.684\.1±\\pm17\.7Accuracy \(excluding unable\)59\.0±\\pm21\.392\.1±\\pm11\.6Total correct / answered \(incl\. unable\)138 / 262227 / 270Total correct / answered \(excl\. unable\)138 / 238227 / 246Unable\-to\-answer count2424Questions responded265270Chat time \(min\)7\.4±\\pm3\.35\.8±\\pm3\.0Chat turns12\.4±\\pm4\.110\.0±\\pm1\.8Turns per correct answer2\.701\.32Table 10:Objective user\-study results\.SimXAIimproves accuracy and reduces interaction cost\. Means are reported with standard deviations\.BUS\-11 itemShort FormTalkToModelSimXAIQ1 Chatbot function was easily detectableInteraction Clarity3\.80±\\pm0\.764\.33±\\pm0\.76Q2 Easy to find information via the chatbotInformation Sufficiency4\.13±\\pm0\.784\.50±\\pm0\.57Q3 Communicating with the chatbot was clearInteraction Clarity3\.33±\\pm1\.274\.60±\\pm0\.56Q4 Chatbot kept track of conversation contextContext Awareness3\.13±\\pm1\.314\.37±\\pm0\.93Q5 Responses were easy to understandResponse Comprehensibility3\.60±\\pm1\.164\.57±\\pm0\.68Q6 Chatbot helped achieve my goalGoal Facilitation3\.10±\\pm1\.214\.37±\\pm0\.67Q7 Appropriate amount of information providedInformation Sufficiency3\.23±\\pm1\.174\.37±\\pm0\.72Q8 Only gave needed informationInformation Relevance3\.13±\\pm1\.314\.13±\\pm0\.94Q9 Responses were accurateResponse Accuracy3\.63±\\pm0\.964\.47±\\pm0\.57Q10 Informs about privacy issuesPrivacy Transparency2\.30±\\pm1\.022\.57±\\pm1\.30Q11 Short waiting / response timeResponse Efficiency4\.37±\\pm0\.614\.03±\\pm1\.13

Table 11:Post\-interaction BUS\-11 ratings \(1–5 Likert\)\.SimXAIis rated higher on 10 of 11 items\.MetrictpAccuracy incl\. unable\-9\.646<\.001∗∗∗<\.001^\{\*\*\*\}Accuracy excl\. unable\-8\.293<\.001∗∗∗<\.001^\{\*\*\*\}Q1 Chatbot function was easily detectable\-2\.898\.0071∗∗Q2 Easy to find information via the chatbot\-2\.483\.0190∗Q3 Communicating with the chatbot was clear\-5\.091<\.001∗∗∗<\.001^\{\*\*\*\}Q4 Chatbot kept track of conversation context\-4\.570<\.001∗∗∗<\.001^\{\*\*\*\}Q5 Responses were easy to understand\-3\.778<\.001∗∗∗<\.001^\{\*\*\*\}Q6 Chatbot helped achieve my goal\-4\.911<\.001∗∗∗<\.001^\{\*\*\*\}Q7 Appropriate amount of information provided\-4\.335<\.001∗∗∗<\.001^\{\*\*\*\}Q8 Only gave needed information\-3\.257\.0029∗∗Q9 Responses were accurate\-3\.878<\.001∗∗∗<\.001^\{\*\*\*\}Q10 Informs about privacy issues\-1\.765\.0881Q11 Short waiting / response time\+1\.439\.1608Table 12:Paired two\-tailedtt\-tests\.![Refer to caption](https://arxiv.org/html/2610.41856v1/figures/user-study/general-instructions.png)\(a\)General Instructions\.
![Refer to caption](https://arxiv.org/html/2610.41856v1/figures/user-study/consent.png)\(b\)Consent form\.

Figure 7:General instructions and consent form \(identifying information removed for blind submission\)\.![Refer to caption](https://arxiv.org/html/2610.41856v1/figures/user-study/study-overview.png)\(a\)Study overview\.
![Refer to caption](https://arxiv.org/html/2610.41856v1/figures/user-study/your-task.png)\(b\)Task instructions\.

Figure 8:Information on the study procedure and how to conduct tasks\.![Refer to caption](https://arxiv.org/html/2610.41856v1/figures/user-study/interface.png)Figure 9:User interface for conversational XAI agent interaction and question answering\.