MEA: A Reward-Driven Multi-Agent System for Faithful Model Explanations

# Mea: A Reward-Driven Multi-Agent System for Faithful Model Explanations
Source: [https://arxiv.org/html/2610.02480](https://arxiv.org/html/2610.02480)
Raghav Kaushik RaviSrivarshinee SridharSriparna SahaAkash GhoshChirag Agarwal

###### Abstract

Recent years have seen the employment of a plethora of machine learning \(ML\) models in high\-stakes domains, but they remain largely opaque to the practitioners who act on their predictions\. While post\-hoc explanation methods offer a lens into this model behavior, wielding them effectively demands expertise most domain experts lack: navigating high\-dimensional outputs, selecting the best explanations, and synthesizing evidence across disparate tools\. To this end, we presentMea, a multi\-agent framework that removes the explanation knowledge barrier entirely: aProposeragent selects and configures explanation tools based on the question and modality, while anActoragent is optimized end\-to\-end against faithfulness, transforming the outputs into natural language explanations grounded in model behavior across tabular, text, and vision modalities\. Further, we introduce diverse question types spanning feature attribution, counterfactual reasoning, and spurious feature detection, each paired with a perturbation\-based faithfulness metric\. We find that frontier LLMs systematically produce unfaithful explanations\. By optimizing against faithfulness rewards augmented with a modality\-adaptive penalty,Meaconsistently outperforms post hoc explainers, agentic, and closed\-source baselines across six datasets, with reward\-driven optimization yielding faithfulness gains of\+28%\+28\\%\(tabular\),\+21%\+21\\%\(text\), and\+34%\+34\\%\(vision\) over the untrained backbone\. More broadly, our findings suggest that AI agents themselves can serve as a scalable, adaptable interface to ML explainability, opening a path toward natural\-language explainability that generalizes beyond the fixed, single\-purpose tools that have long defined the field\.

1University of Virginia2Vellore Institute of Technology3Indian Institute of Technology, Patna

## 1Introduction

While machine learning \(ML\) models are increasingly deployed in high\-stakes domains, such as healthcare, finance, and legal decision\-making\[[1](https://arxiv.org/html/2610.02480#bib.bib8),[2](https://arxiv.org/html/2610.02480#bib.bib9),[3](https://arxiv.org/html/2610.02480#bib.bib10),[4](https://arxiv.org/html/2610.02480#bib.bib12)\], their opacity has become a critical concern for practitioners, regulators, and end users, hindering their trustworthiness\. This gap between predictive performance and human interpretability is a central challenge that explainability research seeks to address\. More broadly, foundation models can generate hallucinated or unsupported outputs, making reliability a particularly important concern in high\-stakes applications\[[5](https://arxiv.org/html/2610.02480#bib.bib1),[6](https://arxiv.org/html/2610.02480#bib.bib5)\]\.

Prior works have introduced a plethora of post\-hoc explanation methods\[[7](https://arxiv.org/html/2610.02480#bib.bib30),[8](https://arxiv.org/html/2610.02480#bib.bib39),[9](https://arxiv.org/html/2610.02480#bib.bib28),[10](https://arxiv.org/html/2610.02480#bib.bib31),[11](https://arxiv.org/html/2610.02480#bib.bib33),[12](https://arxiv.org/html/2610.02480#bib.bib32),[13](https://arxiv.org/html/2610.02480#bib.bib29)\]that probe trained models to generate explanations\. However, each method implements a fixed procedure yielding a single output type,i\.e\.,an attribution vector, a heatmap, or a set of super\-pixels, independent of what the user actually wants to know\. Further, deploying them correctly requiresselecting among competing methods, interpreting high\-dimensional outputs, and synthesizing results across tools, imposing a prohibitive knowledge barrier for domain practitioners without ML expertise\.

![Refer to caption](https://arxiv.org/html/2610.02480v1/xai-agent.png)Figure 1:Overview ofMea\.Given a user question about model behavior, the Proposer agent reasons over the model prediction and input modality to select an explanation strategy, choosing from a multimodal \(tabular, text, and vision\) toolkit and agent reasoning tasks \(grounding, reasoning, and comparison\), the Actor agent then executes the strategy, processes raw explanation outputs, and produces a structured natural\-language explanation\. The explanation is evaluated via input perturbation against faithfulness metrics\. In training, both agents are jointly optimized end\-to\-end via GRPO\[[14](https://arxiv.org/html/2610.02480#bib.bib36)\], using a reward combining faithfulness and tool penalties\.Conversational explanation frameworks \(CEF\) address this barrier by allowing users to interrogate models in natural language\[[15](https://arxiv.org/html/2610.02480#bib.bib13),[16](https://arxiv.org/html/2610.02480#bib.bib34),[17](https://arxiv.org/html/2610.02480#bib.bib14),[18](https://arxiv.org/html/2610.02480#bib.bib21)\], mapping free\-form queries to structured explanation operations without requiring users to select or configure individual methods\. However, these systems share three limitations that remain unaddressed\. First, CEF are largelytraining\-free: they rely on hand\-crafted parsing rules or prompt engineering to route user queries to explanation tools,without any learned component that adapts to explanation quality or user intent, resulting in unfaithful explanations\. Second, CEF have not been evaluated across large\-scale datasets across different modalities\.[Kroeger et al\. \[19\]](https://arxiv.org/html/2610.02480#bib.bib23)found that LLM\-generated explanations fall behind post\-hoc explainers and are susceptible to alphabetical bias, questioning whether LLMs can genuinely outperform existing explainers\. Third, existing CEF lack a principled, quantitative evaluation framework: they either assess quality through single explanation types\[[20](https://arxiv.org/html/2610.02480#bib.bib35),[21](https://arxiv.org/html/2610.02480#bib.bib27)\]or focus on simple linear models, where generated explanations are treated as final outputs with no verification step against model behavior\.

Present work\.We address the above gaps by introducingMea, a novel multi\-agent framework \(see Fig\.[1](https://arxiv.org/html/2610.02480#S1.F1)\) for*prediction\-level explainability across diverse modalities*\.Meacomprises of a Proposer agent that selects and configures an appropriate combination of tools given a question and data modality, and an Actor agent that executes the tools and synthesizes a natural\-language explanation\. To scale across diverse explanation difficulty and data modalities,Meaoptimizes a multi\-modal agent backbone end\-to\-end via reinforcement learning, with faithfulness scores to directly incentivize explanations that are verifiably grounded in model behavior rather than merely plausible\. Since explanation complexity and scale differ fundamentally across modalities, we further introduce a*modality\-adaptive reward*that pairs a universal tool\-count penalty with modality\-specific constraints, preventing reward hacking while encouraging concise, targeted explanations uniformly across all modalities\. Further, we introduce the first comprehensive, model\- and modality\-agnostic questiontaxonomy, comprising diverse question types that spans feature attribution, counterfactual reasoning, and spurious feature detection with their respective faithfulness metrics to quantify alignment between the agent’s explanation and the model’s actual behavior via perturbation\-based testing\. To enable large\-scale training and evaluation, we constructMea\-Bench, a dataset coveringthree data modalities \(tabular, text, and vision\), six model architectures, and six datasets, comprising 13K training and 3\.5K test instances\.

Our main contributions are as follows: 1\) we presentMea, a multi\-agent framework that selects, executes, and synthesizes explanation tools into faithful natural\-language explanations across tabular, text, and vision modalities; 2\) we introduceMea\-Bench, comprising diverse faithfulness\-verified question types,12dataset\-model combinations, and 13k training and 3\.5k test instances across three modalities; 3\) we demonstrate that frontier agents systematically hallucinate on explainability tasks and address this by optimizingMeaend\-to\-end via reinforcement learning, using faithfulness as the reward augmented by a modality\-adaptive penalty that prevents reward hacking across modalities; and 4\) our trainedMeaoutperforms post\-hoc explainers, closed\-source frontier models, and advanced agentic frameworks, yielding average faithfulness gains of\+28%\(tabular\),\+21%\(text\), and\+34%\(vision\) over the untrained backbone\.

## 2Related Works

Post\-hoc Explainability\.Post\-hoc explainers attribute predictions to input features without modifying the model\. Representative approaches include LIME\[[7](https://arxiv.org/html/2610.02480#bib.bib30)\], which fits a locally linear surrogate around the instance, and SHAP\[[10](https://arxiv.org/html/2610.02480#bib.bib31)\], which provides Shapley\-value\-based attributions with consistency guarantees\. For vision, Grad\-CAM\[[9](https://arxiv.org/html/2610.02480#bib.bib28)\]and gradient techniques\[[8](https://arxiv.org/html/2610.02480#bib.bib39),[12](https://arxiv.org/html/2610.02480#bib.bib32)\]supply pixel\-level attribution signals\.

Research Gap\.Existing faithfulness evaluation\[[20](https://arxiv.org/html/2610.02480#bib.bib35),[22](https://arxiv.org/html/2610.02480#bib.bib15)\]is conducted*offline*as a post\-hoc assessment; no prior work adapts it into an*online training signal*to optimize an agentic framework across a comprehensive, multi\-question, multi\-modal setup\.

Conversational and Agentic XAI\.Early interactive works framed explanations as dialogues: TalkToModel\[[15](https://arxiv.org/html/2610.02480#bib.bib13)\]enables natural\-language exploration of model behavior, andEcho\[[16](https://arxiv.org/html/2610.02480#bib.bib34)\]grounds answers in computed attributions rather than parametric knowledge\.Fax\[[23](https://arxiv.org/html/2610.02480#bib.bib16)\]integrates agentic tool use with faithfulness evaluation, though it targets open\-ended questions and relies on an LLM\-as\-a\-judge pipeline\. Beyond explainability, recent multimodal agentic systems have explored actor–critic architectures with tool grounding and iterative feedback for complex multi\-step tasks\[[24](https://arxiv.org/html/2610.02480#bib.bib4),[25](https://arxiv.org/html/2610.02480#bib.bib6)\]\. A parallel line applies multimodal agents to*mechanistic*interpretability of internal model representations, includingMaia\[[26](https://arxiv.org/html/2610.02480#bib.bib18)\],OpenMaia\[[27](https://arxiv.org/html/2610.02480#bib.bib19)\], andNeuronScope\[[28](https://arxiv.org/html/2610.02480#bib.bib20)\]; however, these arenotdesigned for the prediction\-level questions domain practitioners ask\.

Research Gap\.No existing work jointly optimizes a multi\-agent pipeline end\-to\-end with RL across diverse modalities and question types\. Our framework addresses this via a Proposer–Actor structure trained with GRPO, using faithfulness as the reward signal\.

## 3Our Framework:Mea

Problem Formulation\.We study natural\-language explanation generation for black\-box classifiers\. Given a trained classifierf:𝒳→𝒴f:\\mathcal\{X\}\\rightarrow\\mathcal\{Y\}, a questionqqfrom a predefined taxonomy𝒬\\mathcal\{Q\}, and input instance\(s\)x∈𝒳x\\in\\mathcal\{X\},Meagenerates a structured explanationeethat answersqqwhile remaining faithful toff’s decision process\.Measupports three modalities \(vision, text, and tabular data\), wherexxis an image, token sequence, or feature vector respectively, and localizes evidence accordingly: a bounding box, token spans, or feature keys, identifying the subset ofxxthat accounts forff’s behavior with respect toqq\.

XAI Question Taxonomy\.Drawing on prior work\[[16](https://arxiv.org/html/2610.02480#bib.bib34),[20](https://arxiv.org/html/2610.02480#bib.bib35)\], we define ten question types in three semantic categories \(Appendix Table[3](https://arxiv.org/html/2610.02480#A1.T3)\): 1\)Feature Attributionasks which input parts drove or suppressed a prediction; 2\)Counterfactualprobes prediction sensitivity to hypothetical modifications; and 3\)Spurious Featureexamines whether the model relies on irrelevant cues\. We exclude Q8 and Q9 from training and use them as*out\-of\-distribution*\(OOD\) evaluation questions, since these are only defined over*misclassified*instances \(y^≠y∗\\hat\{y\}\\neq y^\{\*\}\), which limits eligible training samples and destabilizes gradient estimation; we treat them as a held\-out probe of generalization to unseen question types without task\-specific supervision\.

### 3\.1MeaOverview

Meaoperates as a two\-stage framework on a shared multimodal LLM backboneθ\\theta\. First, theProposer AgentπθProp\\pi^\{\\mathrm\{Prop\}\}\_\{\\theta\}receives questionqq, input𝐱\\mathbf\{x\}, and predictiony^=f⁡\(𝐱\)\\hat\{y\}=f\(\\mathbf\{x\}\), and produces a structured strategySSspecifying which XAI tools and autonomous reasoning tasks to invoke \(Appendix[A\.2](https://arxiv.org/html/2610.02480#A1.SS2)\)\. Then, theActor AgentπθAct\\pi^\{\\mathrm\{Act\}\}\_\{\\theta\}executesSS, processes the raw tool outputs, and synthesizes an explanationee:

S\\displaystyle S∼πθProp\(S\|q,𝐱,y^,𝒯m\),\\displaystyle\\;\\sim\\;\\pi^\{\\mathrm\{Prop\}\}\_\{\\theta\}\\\!\\left\(S\\;\\middle\|\\;q,\\,\\mathbf\{x\},\\,\\hat\{y\},\\,\\mathcal\{T\}\_\{m\}\\right\),\(1\)e\\displaystyle e∼πθAct\(e\|q,𝐱,y^,S,\{ot\}t∈𝒯sel\),\\displaystyle\\;\\sim\\;\\pi^\{\\mathrm\{Act\}\}\_\{\\theta\}\\\!\\left\(e\\;\\middle\|\\;q,\\,\\mathbf\{x\},\\,\\hat\{y\},\\,S,\\,\\\{o\_\{t\}\\\}\_\{t\\in\\mathcal\{T\}\_\{\\mathrm\{sel\}\}\}\\right\),\(2\)where𝒯m\\mathcal\{T\}\_\{m\}is the tool set for modalitymm,𝒯sel⊆𝒯m\\mathcal\{T\}\_\{\\mathrm\{sel\}\}\\subseteq\\mathcal\{T\}\_\{m\}is the Proposer’s selected subset, andoto\_\{t\}is tooltt’s summarized output\.

### 3\.2Proposer Agent: Intent\-Aware Strategy Formation

Prior conversational systems rely on fixed pipelines or hand\-crafted routing rules with no learned component adapting to question semantics\. To address this limitation, we introduce the trainable Proposer AgentπθProp\\pi^\{\\mathrm\{Prop\}\}\_\{\\theta\}, which performs strategy formation before any tool is executed\. Given question typeQiQ\_\{i\}, modalitymm, and predictiony^=f⁡\(𝐱\)\\hat\{y\}=f\(\\mathbf\{x\}\), it generatesS=\(stype,𝒯sel,𝒜auto\)S=\(s\_\{\\mathrm\{type\}\},\\,\\mathcal\{T\}\_\{\\mathrm\{sel\}\},\\,\\mathcal\{A\}\_\{\\mathrm\{auto\}\}\):

S∼πθProp\(S\|q,𝐱,y^,𝒯m\),S\\;\\sim\\;\\pi^\{\\mathrm\{Prop\}\}\_\{\\theta\}\\\!\\left\(S\\;\\middle\|\\;q,\\,\\mathbf\{x\},\\,\\hat\{y\},\\,\\mathcal\{T\}\_\{m\}\\right\),\(3\)wherestype∈\{autonomous,tools,hybrid\}s\_\{\\mathrm\{type\}\}\\in\\\{\\texttt\{autonomous\},\\,\\texttt\{tools\},\\,\\texttt\{hybrid\}\\\}selects the reasoning mode,𝒯sel⊆𝒯m\\mathcal\{T\}\_\{\\mathrm\{sel\}\}\\subseteq\\mathcal\{T\}\_\{m\}specifies which tools to invoke, and𝒜auto\\mathcal\{A\}\_\{\\mathrm\{auto\}\}enumerates grounding/reasoning subtasks for the Actor\. An ablation on the Proposer’s strategy modes \(Appendix[A\.10](https://arxiv.org/html/2610.02480#A1.SS10)\) confirms both tool use and autonomous reasoning are independently load\-bearing, and thatMealearns to combine them more effectively than the base agent\.

### 3\.3Actor Agent: Tool\-Grounded Explanation Synthesis

Even when a well\-formed strategy is available, raw tool outputs are not directly consumable as explanations\.[Kroeger et al\. \[19\]](https://arxiv.org/html/2610.02480#bib.bib23)show that LLMs systematically bypass tool evidence in favor of parametric knowledge, producing plausible but unfaithful conclusions\. To address this, the Actor AgentπθAct\\pi^\{\\mathrm\{Act\}\}\_\{\\theta\}operates in two steps enforcing a structural dependency on attribution evidence\.

Step 1: Tool Execution and Summarization\.For each toolt∈𝒯selt\\in\\mathcal\{T\}\_\{\\mathrm\{sel\}\}, the Actor invokesttagainstffand𝐱\\mathbf\{x\}, obtaining heatmaps, per\-token, or feature importance scores, which are converted into structured text summaries \(execution status, natural\-language description, key statistics\) rather than passed as raw tensors\. To standardize granularity, the Actor returns the top 25% of features/tokens for tabular/text inputs, and aggregates top\-1% attribution coordinates into a bounding box for vision\.

Step 2: Evidence\-Grounded Explanation Generation\.The per\-tool summaries are assembled into a structured prompt over which the Actor reasons to identify the most relevant features:

e∼πθAct\(e\|q,𝐱,y^,S,\{ot\}t∈𝒯sel\),e∈ℰm,e\\;\\sim\\;\\pi^\{\\mathrm\{Act\}\}\_\{\\theta\}\\\!\\left\(e\\;\\middle\|\\;q,\\,\\mathbf\{x\},\\,\\hat\{y\},\\,S,\\,\\\{o\_\{t\}\\\}\_\{t\\in\\mathcal\{T\}\_\{\\mathrm\{sel\}\}\}\\right\),\\quad e\\in\\mathcal\{E\}\_\{m\},\(4\)whereℰm\\mathcal\{E\}\_\{m\}is the modality\-appropriate output space: a bounding box, token span, or feature list\.

### 3\.4Perturbation\-based Faithfulness Evaluation

Evaluating explanation correctness without ground\-truth labels is difficult\. Prior work relies on LLM\-as\-a\-judge pipelines susceptible to the same hallucinations they assess\[[19](https://arxiv.org/html/2610.02480#bib.bib23),[21](https://arxiv.org/html/2610.02480#bib.bib27)\], or evaluates a single explanation type in isolation\[[20](https://arxiv.org/html/2610.02480#bib.bib35),[23](https://arxiv.org/html/2610.02480#bib.bib16)\]; none adapts faithfulness evaluation into an online training signal across a multi\-question, multi\-modal setup\. We address this with a unified perturbation\-based protocol: explanationeeidentifying regionrris faithful iff maskingrrproduces the model behavior changeqqpredicts\. For eachQiQ\_\{i\}, metricmim\_\{i\}is computed by maskingrrin𝐱\\mathbf\{x\}and re\-queryingff, where𝐱−r\\mathbf\{x\}^\{\-r\}is the masked input \(zero\-filled for tabular, grey\-filled for vision, span\-deleted for text\) andP⁡\(y∣𝐱\)P\(y\\mid\\mathbf\{x\}\)the predicted class probability\.

Feature Attribution \(Q1\-Q4\)\.Q1\-Q2 form a complementary pair: masking the most responsible region should drop confidence; masking the least responsible should not:

m1\\displaystyle m\_\{1\}=max⁡\(0,P⁡\(y^∣𝐱\)−P⁡\(y^∣𝐱−r\)\),\\displaystyle=\\max\\\!\\left\(0,\\;P\(\\hat\{y\}\\mid\\mathbf\{x\}\)\-P\(\\hat\{y\}\\mid\\mathbf\{x\}^\{\-r\}\)\\right\),m2\\displaystyle m\_\{2\}=1−\|P⁡\(y^∣𝐱\)−P⁡\(y^∣𝐱−r\)\|\.\\displaystyle=1\-\\left\|P\(\\hat\{y\}\\mid\\mathbf\{x\}\)\-P\(\\hat\{y\}\\mid\\mathbf\{x\}^\{\-r\}\)\\right\|\.\(5\)
Q3 checks whether masking the distinctive region collapses the top\-1/top\-2 margin; Q4 measures cross\-instance contrastiveness viasim⁡\(FA,FB\)\\mathrm\{sim\}\(F\_\{A\},F\_\{B\}\), requiring different attributions for differently\-predicted instances\.

Counterfactual \(Q5–Q7\)\.Q5 verifies the predicted direction of change agrees with the observed probability shift; Q6\-Q7 measure whether the proposed modification raises the target\-class probability and whether the predicted new class becomes likely:

m6\\displaystyle m\_\{6\}=max⁡\(0,P⁡\(y^∣𝐱\+Δ\)−P⁡\(y^∣𝐱\)\),\\displaystyle=\\max\\\!\\left\(0,\\;P\(\\hat\{y\}\\mid\\mathbf\{x\}^\{\+\\Delta\}\)\-P\(\\hat\{y\}\\mid\\mathbf\{x\}\)\\right\),m7\\displaystyle m\_\{7\}=P⁡\(y^′∣𝐱−r\),\\displaystyle=P\(\\hat\{y\}^\{\\prime\}\\mid\\mathbf\{x\}^\{\-r\}\),\(6\)where𝐱\+Δ\\mathbf\{x\}^\{\+\\Delta\}is the agent\-proposed modification andy^′\\hat\{y\}^\{\\prime\}the predicted class\.

Spurious Features\.Q8 measures the normalized gain in correct\-class probability after masking:

m8\\displaystyle m\_\{8\}=max⁡\(0,min⁡\(1,D⁡\(𝐱,r\)\)\),\\displaystyle=\\max\\\!\\left\(0,\\;\\min\\\!\\left\(1,\\;D\(\\mathbf\{x\},r\)\\right\)\\right\),D⁡\(𝐱,r\)\\displaystyle D\(\\mathbf\{x\},r\)=P⁡\(y∗∣𝐱−r\)−P⁡\(y∗∣𝐱\)1−P⁡\(y∗∣𝐱\),\\displaystyle=\\frac\{P\(y^\{\*\}\\mid\\mathbf\{x\}^\{\-r\}\)\-P\(y^\{\*\}\\mid\\mathbf\{x\}\)\}\{1\-P\(y^\{\*\}\\mid\\mathbf\{x\}\)\},\(7\)normalized by remaining headroom for fair cross\-instance comparison\. Q9 averagesm8m\_\{8\}overNNmisclassified instances; Q10 mirrors Q4’s contrastive intuition on misclassified instances\. Full definitions are in Appendix[A\.3](https://arxiv.org/html/2610.02480#A1.SS3)\.

### 3\.5End\-to\-End Training

Reward\-based optimization for trustworthy generation has increasingly relied on carefully designed task\-specific objectives\[[29](https://arxiv.org/html/2610.02480#bib.bib2)\]\. However relying exclusively on raw faithfulness scores frequently precipitates reward hacking, typically via two degenerate strategies: trivial probability\-drop maximization \(e\.g\.,masking an entire image\) and indiscriminate tool invocation, which inflates faithfulness without isolating informative features\[[14](https://arxiv.org/html/2610.02480#bib.bib36)\]\. Since naive objectives converge on these unfaithful traces, we employ GRPO\[[30](https://arxiv.org/html/2610.02480#bib.bib11)\]to jointly optimize both agents end\-to\-end , building on recent applications of GRPO for reasoning optimization\[[31](https://arxiv.org/html/2610.02480#bib.bib3)\]\. Each episode captures two transitions: strategy generation by the Proposer and explanation synthesis by the Actor\. We formalize this as an MDP with deterministic tool\-execution transitions and a terminal reward in Appendix[A\.4](https://arxiv.org/html/2610.02480#A1.SS4)\. The faithfulness rewardmim\_\{i\}is evaluated strictly after the Actor’s final output, and the trajectory\-level group\-normalized advantage

A^k=Rk−mean⁡\(\{Rj\}j=1G\)std⁡\(\{Rj\}j=1G\),\\hat\{A\}\_\{k\}=\\frac\{R\_\{k\}\-\\mathrm\{mean\}\\left\(\\\{R\_\{j\}\\\}\_\{j=1\}^\{G\}\\right\)\}\{\\mathrm\{std\}\\left\(\\\{R\_\{j\}\\\}\_\{j=1\}^\{G\}\\right\)\},\(8\)is applied uniformly to both transitions, treating the Proposer\-Actor as jointly responsible for the outcome: a misaligned Proposer strategy cannot be recovered by the Actor, so the optimization signal must flow to both agents equally\.

Size Penalty \(Vision\)\.To foreclose the large\-mask shortcut, we penalize large bounding boxes:

mivision=max⁡\(0,mi−λsize⋅ρ\),m\_\{i\}^\{\\mathrm\{vision\}\}=\\max\\\!\\left\(0,\\;m\_\{i\}\-\\lambda\_\{\\mathrm\{size\}\}\\cdot\\rho\\right\),\(9\)whereρ∈\[0,1\]\\rho\\in\[0,1\]is the bounding\-box\-to\-image area ratio andλsize=0\.3\\lambda\_\{\\mathrm\{size\}\}=0\.3\. For tabular/text, the fixed top\-25% output constraint \(Section[3\.3](https://arxiv.org/html/2610.02480#S3.SS3)\) already governs scope, making an explicit size penalty unnecessary\.

Tool Penalty\.We apply anℓ1\\ell\_\{1\}tool\-count penalty across all modalities:r=max⁡\(0,min⁡\(1,mi−λtool⋅ntoolsnmax\)\)r\{=\}\\max\\\!\\left\(0,\\;\\min\\\!\\left\(1,\\;m\_\{i\}\{\-\}\\lambda\_\{\\mathrm\{tool\}\}\\cdot\\frac\{n\_\{\\mathrm\{tools\}\}\}\{n\_\{\\mathrm\{max\}\}\}\\right\)\\right\), wherentoolsn\_\{\\mathrm\{tools\}\}is the number of invoked tools/tasks,nmax=9n\_\{\\mathrm\{max\}\}\{=\}9,λtool=0\.05\\lambda\_\{\\mathrm\{tool\}\}\{=\}0\.05–kept small to avoid suppressing gradients on question types with low pre\-training scores \(e\.g\.,tabular Q1/Q8: 0\.01/0\.04 at initialization\)\. These penalties let the Proposer learn informative tool selection while the Actor learns faithful grounding, each reinforcing the other via shared gradients\.

## 4Experiments

We address the following key research questions: RQ1\) DoesMeagenerates faithful explanation across all three modalities? RQ2\) DoesMeaoutperform widely used post hoc explainers? RQ3\) Is the prompting style and agentic architecture necessary forMea’s performance? and RQ4\) DoesMeageneralize to OOD explanation questions, unseen datasets, and novel model architectures?

### 4\.1Datasets and Experimental Setup

We first describe datasets designed to study the faithfulness of explanations generated byMeaand then outline experimental setup\. Figure[2](https://arxiv.org/html/2610.02480#S4.F2)illustratesMea’s end\-to\-end pipeline on representative samples from each modality, showing how the Proposer selects appropriate tools and the Actor synthesizes faithful explanations across diverse question types\.

![Refer to caption](https://arxiv.org/html/2610.02480v1/mea_case_study.png)Figure 2:Case study ofMeaacross three modalities\.Each column shows a complete pipeline execution forMea: the input, Proposer’s strategy and tool, Actor’s execution output, and final explanation with faithfulness evaluation\.Dataset\.We construct a multi\-modal benchmark for analyzing agent behavior across tabular, text, and vision modalities \(Appendix[A\.5](https://arxiv.org/html/2610.02480#A1.SS5)\), generated using predictions from ML models trained on standard supervised learning tasks\. For each dataset\-model pair, we generate structured questions \(Table[3](https://arxiv.org/html/2610.02480#A1.T3)\) covering feature attribution, counterfactual analysis, and spurious feature identification\.Tabular: Adult Census Income\[[32](https://arxiv.org/html/2610.02480#bib.bib45)\]and Breast Cancer\[[33](https://arxiv.org/html/2610.02480#bib.bib46)\], evaluated with a two\-layer network and TabNet\.Text: SNLI\[[34](https://arxiv.org/html/2610.02480#bib.bib40)\]and IMDb\[[35](https://arxiv.org/html/2610.02480#bib.bib47)\], evaluated with a lightweight neural network and a CNN\.Vision: CUB\-200\-2011\[[36](https://arxiv.org/html/2610.02480#bib.bib41)\]and STL\-10\[[37](https://arxiv.org/html/2610.02480#bib.bib48)\], evaluated with ResNet\-50\[[38](https://arxiv.org/html/2610.02480#bib.bib49)\]and DenseNet\-201\[[39](https://arxiv.org/html/2610.02480#bib.bib50)\]\. Each instance contains the original input, ground\-truth label, model prediction, and a natural\-language analytical prompt\.

Training Setup\.We fine\-tune Qwen3\.6\-35B\-A3B\[[40](https://arxiv.org/html/2610.02480#bib.bib37)\]using LoRA \(r=32r\{=\}32\)\[[41](https://arxiv.org/html/2610.02480#bib.bib38)\]for 3 epochs, following[Shao et al\. \[14\]](https://arxiv.org/html/2610.02480#bib.bib36)for GRPO hyperparameters \(learning rate1×10−51\\times 10^\{\-5\}, batch size 32, KL coefficient 0\)\. We generateK=4K\{=\}4rollouts per question atT=1\.0T\{=\}1\.0\(K=8K\{=\}8for harder types\), with advantages group\-normalized over each question’sKKrollouts; evaluation usesT=0\.0T\{=\}0\.0\. For training across modalities, we compareSequential\(tabular→\\totext→\\tovision, each stage initialized from the previous checkpoint\) againstHybrid\(all modalities interleaved per batch\)\. Sequential training suffers from catastrophic forgetting\. Performance on earlier modalities degrades and behavioral diversity collapses on the final modality \(Appendix[A\.6](https://arxiv.org/html/2610.02480#A1.SS6)\), so we adopt Hybrid training for all main results\. We also ablate on a 4B\-parameter backbone \(Appendix[A\.11](https://arxiv.org/html/2610.02480#A1.SS11)\), confirming faithfulness gains transfer to smaller models; computational cost details are in Appendix[A\.14](https://arxiv.org/html/2610.02480#A1.SS14)\.

Baselines\.To isolate the impact of collaborative, iterative refinement, we benchmarkMeaagainst four prompting paradigms sharing identical model infrastructure, execution environments, and XAI toolsets:*Zero\-Shot*\(control\),*Chain\-of\-Thought \(CoT\)*\[[42](https://arxiv.org/html/2610.02480#bib.bib25)\],*ReAct*\[[43](https://arxiv.org/html/2610.02480#bib.bib24)\], and*Tree of Thoughts \(ToT\)*\[[44](https://arxiv.org/html/2610.02480#bib.bib26)\]\. We further compare against standard post\-hoc explainers\[[7](https://arxiv.org/html/2610.02480#bib.bib30),[10](https://arxiv.org/html/2610.02480#bib.bib31),[12](https://arxiv.org/html/2610.02480#bib.bib32),[9](https://arxiv.org/html/2610.02480#bib.bib28),[13](https://arxiv.org/html/2610.02480#bib.bib29),[11](https://arxiv.org/html/2610.02480#bib.bib33),[8](https://arxiv.org/html/2610.02480#bib.bib39)\], and against specialized counterfactual generators, DiCE\[[45](https://arxiv.org/html/2610.02480#bib.bib51)\]and Growing Spheres\[[46](https://arxiv.org/html/2610.02480#bib.bib52)\]\.

![Refer to caption](https://arxiv.org/html/2610.02480v1/faithfulness_combined_all_modalities.png)Figure 3:Mea’s performance\.Faithfulness scores ofMeavs\. baselines across three modalities:\(A\)tabular,\(B\)text, and\(C\)vision\. Higher mean faithfulness scores indicate more faithful explanations\. Results show thatMeaoutperform all baselines, including closed\-source models and techniques like CoT, ToT, and ReACT\.Evaluation\.Following[Agarwal et al\. \[47\]](https://arxiv.org/html/2610.02480#bib.bib7), we evaluate explanation quality through*faithfulness*: whether the agent’s claimed explanation is verified by the target model’s behavior under perturbation\. All metrics \(Sec\.[3\.4](https://arxiv.org/html/2610.02480#S3.SS4)\) follow a common*perturb\-and\-measure*protocol: given an explanation identifying a regionrr\(feature keys for tabular, text spans for text, a bounding box for vision\), we perturbrrto obtainx′x^\{\\prime\}, re\-run the prediction, and score it against the corresponding metric\. Perturbations are modality\-specific: vision regions are gray\-filled \(or inpainted with Stable Diffusion\[[48](https://arxiv.org/html/2610.02480#bib.bib17)\]for counterfactuals\), text spans are deleted or replaced with agent\-provided counterfactual text, and tabular features are mean\-filled or set to agent\-specified values\.

## 5Results

Here, we discuss experimental results that answer key questions \(RQ1\-RQ4\) highlighted before\.

RQ1\)Meaimproves explanation faithfulness across all modalities\.A key challenge in XAI is that different explanation needs \(feature attribution, counterfactual reasoning, and spurious feature detection\) traditionally require separate tools\.Meaaddresses this by serving as a unified framework handling diverse question types across all three modalities without modality\-specific engineering\. Fig\.[3](https://arxiv.org/html/2610.02480#S4.F3)showsMeaachieves the highest faithfulness scores across all three modalities\. On tabular data,Meascores0\.8550\.855on counterfactual questions, outperforming the best baseline \(Base Agent,0\.6700\.670\) by28%, and0\.9970\.997on spurious features vs\. Gemini\-3\.1’s0\.8790\.879\. On text,Meascores0\.7080\.708on counterfactual questions vs\. Gemini\-3\.1’s0\.5740\.574\(\+23%\)\. The gap is largest on vision counterfactual questions, whereMeaachieves0\.5980\.598vs\. GPT\-5\.4’s0\.3280\.328\(\+82%\)\. Across all three modalities,Meaattains a perfect or near\-perfect spurious feature detection score \(0\.9970\.997/1\.0001\.000/1\.0001\.000\)\. To verify these similarities reflect genuine statistical equivalence rather than noise, we ran two\-sided Welch’stt\-tests\[[49](https://arxiv.org/html/2610.02480#bib.bib42)\]on all targeted comparisons: on spurious\-feature questions \(Q10\),Meaachieves ceiling performance \(1\.0001\.000\) on text and vision, and the strongest baselines \(Gemini\-3\.1:0\.9960\.996text, Base Agent:0\.9810\.981vision\) differ by at most0\.020\.02–significant only due toMea’s zero variance at ceiling \(Appendix[A\.13](https://arxiv.org/html/2610.02480#A1.SS13)\)\. Appendix[A\.12](https://arxiv.org/html/2610.02480#A1.SS12)further shows this faithfulness tracks the target model’s decision logic, not the LLM’s own priors\.

These results confirm thatMeaacts as aone\-stop\-shopdelivering strong, reliable faithfulness across all ten question types and all three modalities\.Meaalso outperforms all closed\-source models and alternative agentic frameworks \(CoT, ToT, ReAct\) on all question types across all modalities, confirming its role as a unified solution for diverse explainability needs\.

![Refer to caption](https://arxiv.org/html/2610.02480v1/faithfulness_tool_only_combined.png)Figure 4:Meavs\. Explainers\.Mean faithfulness scores of identifying most salient input features across\(A\)tabular,\(B\)text, and\(C\)vision modalities\.Meaoutperforms all baselines on tabular and text inputs, and matches the best\-performing method on vision inputs\.RQ2\)Meaoutperforms post\-hoc explainers\.Standard post\-hoc explainers \(LIME, SHAP, GradCAM, etc\.\) are architecturally single\-purpose, so a like\-for\-like comparison is well\-defined only for Q1:Which parts of the instance are most responsible for the model’s prediction?This restriction itself illustrates one ofMea’s motivations: no single tool can address the diverse explanation needs spanned byMea’s ten\-question taxonomy\. We therefore compareMeaagainst two*disjoint*families of specialized baselines, each matched to the question type it targets: general\-purpose attribution explainers on Q1, and specialized counterfactual generators on Q6\.

Q1 \(Feature Attribution\)\.Following the same output standardization used inMea\(top\-25% of features for tabular/text, top\-1% pixels as a bounding box for vision\), Fig\.[4](https://arxiv.org/html/2610.02480#S5.F4)showsMeaoutperforms the best post\-hoc explainer on tabular \(0\.1950\.195vs\.0\.1500\.150;\+30%\) and text \(0\.2650\.265vs\.0\.2370\.237;\+12%\), and matches the best single\-tool baseline on vision \(0\.4990\.499vs\.0\.4970\.497;p=0\.944p=0\.944, Appendix[A\.13](https://arxiv.org/html/2610.02480#A1.SS13)\)\.

Q6 \(Flip Prediction\)\.We further compareMeaagainst two specialized counterfactual generators, DiCE\[[45](https://arxiv.org/html/2610.02480#bib.bib51)\]and Growing Spheres\[[46](https://arxiv.org/html/2610.02480#bib.bib52)\]\. Neither has a meaningful notion of searching over text spans or image regions, so this comparison is restricted to*tabular*data, using the same test instances, preprocessor, and metrics asMea’s evaluation \(Table[1](https://arxiv.org/html/2610.02480#S5.T1)\)\.Meaachieves comparable flip\-success rates while achieving substantially higher faithfulness throughout \(avg\. target\-class probability0\.8860\.886–0\.8940\.894vs\.0\.4990\.499–0\.6310\.631\)\.

Table 1:Meavs\. specialized counterfactual explainers on Q6 \(flip prediction\), aggregated across model architectures per dataset \(n=63n=63for Adult Census,n=44n=44for Breast Cancer\)\. Best per row group inbold\.Together, these results show thatMea, despite operating as a singlegeneral\-purpose agentacross all ten question types, is competitive with or superior to specialized single\-purpose explainers on both feature attribution and counterfactual generation\.

RQ3\)Meais robust to different prompt styles and outperforms critic\-augmented agents\.Here, we first test sensitivity to question phrasing by evaluating three paraphrased variants of the question taxonomy \(see[A\.8](https://arxiv.org/html/2610.02480#A1.SS8)\) and compare mean faithfulness against the original\. Next, we investigate whether augmenting a base agent with a Critic\[[50](https://arxiv.org/html/2610.02480#bib.bib22)\], which receives the original strategy and explanation, critiques them, and regenerates improved outputs, can close the gap withMea\. Results in Appendix[A\.8](https://arxiv.org/html/2610.02480#A1.SS8)Fig\.[7](https://arxiv.org/html/2610.02480#A1.F7)show that paraphrase variants score within<𝟏%\\mathbf\{<1\\%\}of the original across all modalities and question groups \(e\.g\.,tabular counterfactual: paraphrase0\.9340\.934vs\.Mea0\.9390\.939; vision feature attribution:0\.6860\.686vs\.0\.6920\.692\), confirming thatMeais robust to prompt rephrasing\.Meaalso consistently outperforms a base agent augmented with an inference\-time Critic \(Agent\+Reflection\) across all three modalities, with the largest gains on counterfactual questions:\+36%on text \(0\.8760\.876vs\.0\.6440\.644\) and\+30%on vision \(0\.5690\.569vs\.0\.4390\.439\), and Agent\+Reflection even*degrading*performance relative to the untrained base agent on tabular spurious features \(Appendix[A\.9](https://arxiv.org/html/2610.02480#A1.SS9), Fig\.[8](https://arxiv.org/html/2610.02480#A1.F8)\)\. Appendix[A\.13](https://arxiv.org/html/2610.02480#A1.SS13)shows paraphrase robustness tests results, which demonstrate no significant performance degradation across 8 of 9 question\-type×\\timesmodality conditions \(p\>0\.05p\>0\.05in all but vision counterfactual\)\. These results demonstrate that RL\-based end\-to\-end training provides substantially stronger and more reliable gains than inference\-time critique\.

RQ4\)Meageneralizes to OOD datasets and model architectures\.We evaluateMeaon unseen datasets and model architectures across all modalities \(Yelp Review Polarity dataset\[[51](https://arxiv.org/html/2610.02480#bib.bib56)\]with a BERT\-based classifier\[[52](https://arxiv.org/html/2610.02480#bib.bib55)\]for text; German Credit dataset\[[53](https://arxiv.org/html/2610.02480#bib.bib58)\]with a three\-layer neural network for tabular; CIFAR\-10\[[54](https://arxiv.org/html/2610.02480#bib.bib57)\]with ResNet\-18\[[38](https://arxiv.org/html/2610.02480#bib.bib49)\]for vision\)\. Results in Table[2](https://arxiv.org/html/2610.02480#S5.T2)show thatMeaachieves the highest overall faithfulness on all three modalities \(0\.5810\.581vs\.0\.3790\.379on tabular,0\.6890\.689vs\.0\.6250\.625on text,0\.5760\.576vs\.0\.4480\.448on vision\)\. The largest gains appear on tabular inputs \(\+53%\+\\textbf\{53\\%\}\), with feature attribution alone rising from0\.4470\.447to0\.7830\.783\(\+75%\)\. On vision, counterfactual faithfulness improves from0\.2460\.246to0\.5210\.521\(\+112%\)\.Meaonly underperforms the base agent on text counterfactual questions \(0\.7700\.770vs\.0\.8600\.860\) and vision feature attribution \(0\.5830\.583vs\.0\.6210\.621\), which we attribute to further hyperparameter tuning needed for better modality\-specific trade\-offs\.Meaalso generalizes to concept\-level explanation, a question type outside the trained taxonomy evaluated via representation\-space CAV ablation, outperforming the base agent by∼\\sim18% in mean faithfulness\. Overall, these results confirm thatMeageneralizesto unseen datasets and model architectures, and to explanation mechanisms beyond input perturbation \(see Appendix Table[6](https://arxiv.org/html/2610.02480#A1.T6)for detailed results, Table[7](https://arxiv.org/html/2610.02480#A1.T7)for concept\-level results, and Appendix[A\.7](https://arxiv.org/html/2610.02480#A1.SS7)for an ablation study of OOD question types\)\.

Table 2:OOD faithfulness on datasets/architectures unseen during training, averaged across question types per group\.Base: Proposer–Actor without GRPO fine\-tuning;Mea: full GRPO\-tuned system\. Best per row inbold\.
## 6Conclusion

We presentedMea, a multi\-agent framework for faithful, prediction\-level explanation across tabular, text, and vision modalities, andMea\-Bench, a model\- and modality\-agnostic benchmark of 16k instances each paired with a faithfulness metric\. OptimizingMeaend\-to\-end via RL with a modality\-adaptive reward yields an agent that consistently outperforms post\-hoc explainers, frontier models, and advanced agentic frameworks by\+28%\+28\\%\(tabular\),\+21%\+21\\%\(text\), and\+34%\+34\\%\(vision\) faithfulness, showinglearning to explainis more principled than prompting to explain\.Meais currently scoped to classification tasks and a fixed tool library, with a perturbation\-based protocol sensitive to distributional shift like others in its class; extending to other tasks, tool libraries, and faithfulness signals is promising future work\.

## Acknowledgements

We would like to thank all members of[Aikyam Lab](https://chirag-agarwall.github.io/)for their valuable feedback\. C\.A\. is supported, in part, by grants from Capital One, LaCross Institute for Ethical AI in Business, the UVA Environmental Institute, OpenAI Researcher Program, Thinking Machine’s Tinker Research Grant, and Cohere\. The views expressed are those of the authors and do not reflect the official policy or the position of the funding agencies\.

## References

- \[1\]P\. Rajpurkar, E\. Chen, O\. Banerjee, and E\. J\. Topol\(2022\)AI in health and medicine\.Nature medicine28\(1\),pp\. 31–38\.Cited by:[§1](https://arxiv.org/html/2610.02480#S1.p1.1)\.
- \[2\]M\. P\. Cary Jr, A\. Zink, S\. Wei, A\. Olson, M\. Yan, R\. Senior, S\. Bessias, K\. Gadhoumi, G\. Jean\-Pierre, D\. Wang,et al\.\(2023\)Mitigating racial and ethnic bias and advancing health equity in clinical algorithms: a scoping review: scoping review examines racial and ethnic bias in clinical algorithms\.Health Affairs42\(10\),pp\. 1359–1368\.Cited by:[§1](https://arxiv.org/html/2610.02480#S1.p1.1)\.
- \[3\]K\. Cao and H\. You\(2024\)Fundamental analysis via machine learning\.Financial Analysts Journal80\(2\),pp\. 74–98\.Cited by:[§1](https://arxiv.org/html/2610.02480#S1.p1.1)\.
- \[4\]A\. W\. Flores, K\. Bechtel, and C\. T\. Lowenkamp\(2016\)False positives, false negatives, and false analyses: a rejoinder to machine bias: there’s software used across the country to predict future criminals\. and it’s biased against blacks\.Fed\. Probation80,pp\. 38\.Cited by:[§1](https://arxiv.org/html/2610.02480#S1.p1.1)\.
- \[5\]P\. Sahoo, P\. Meharia, A\. Ghosh, S\. Saha, V\. Jain, and A\. Chadha\(2024\)A comprehensive survey of hallucination in large language, image, video and audio foundation models\.Findings of the association for computational linguistics: EMNLP 2024,pp\. 11709–11724\.Cited by:[§1](https://arxiv.org/html/2610.02480#S1.p1.1)\.
- \[6\]A\. Ghosh, S\. Sridhar, R\. K\. Ravi, M\. Muhsin, S\. Saha, and C\. Agarwal\(2025\)Clinic: evaluating multilingual trustworthiness in language models for healthcare\.arXiv preprint arXiv:2512\.11437\.Cited by:[§1](https://arxiv.org/html/2610.02480#S1.p1.1)\.
- \[7\]M\. T\. Ribeiro, S\. Singh, and C\. Guestrin\(2016\)"Why should i trust you?": explaining the predictions of any classifier\.External Links:1602\.04938,[Link](https://arxiv.org/abs/1602.04938)Cited by:[§A\.2](https://arxiv.org/html/2610.02480#A1.SS2.p3.1),[§1](https://arxiv.org/html/2610.02480#S1.p2.1),[§2](https://arxiv.org/html/2610.02480#S2.p1.1),[§4\.1](https://arxiv.org/html/2610.02480#S4.SS1.p4.1)\.
- \[8\]M\. Sundararajan, A\. Taly, and Q\. Yan\(2017\)Axiomatic attribution for deep networks\.External Links:1703\.01365,[Link](https://arxiv.org/abs/1703.01365)Cited by:[§A\.2](https://arxiv.org/html/2610.02480#A1.SS2.p2.1),[§1](https://arxiv.org/html/2610.02480#S1.p2.1),[§2](https://arxiv.org/html/2610.02480#S2.p1.1),[§4\.1](https://arxiv.org/html/2610.02480#S4.SS1.p4.1)\.
- \[9\]R\. R\. Selvaraju, M\. Cogswell, A\. Das, R\. Vedantam, D\. Parikh, and D\. Batra\(2019\)Grad\-cam: visual explanations from deep networks via gradient\-based localization\.International Journal of Computer Vision128\(2\),pp\. 336–359\.External Links:ISSN 1573\-1405,[Link](http://dx.doi.org/10.1007/s11263-019-01228-7),[Document](https://dx.doi.org/10.1007/s11263-019-01228-7)Cited by:[§A\.2](https://arxiv.org/html/2610.02480#A1.SS2.p2.1),[§1](https://arxiv.org/html/2610.02480#S1.p2.1),[§2](https://arxiv.org/html/2610.02480#S2.p1.1),[§4\.1](https://arxiv.org/html/2610.02480#S4.SS1.p4.1)\.
- \[10\]S\. Lundberg and S\. Lee\(2017\)A unified approach to interpreting model predictions\.External Links:1705\.07874,[Link](https://arxiv.org/abs/1705.07874)Cited by:[§A\.2](https://arxiv.org/html/2610.02480#A1.SS2.p3.1),[§1](https://arxiv.org/html/2610.02480#S1.p2.1),[§2](https://arxiv.org/html/2610.02480#S2.p1.1),[§4\.1](https://arxiv.org/html/2610.02480#S4.SS1.p4.1)\.
- \[11\]C\. Pichery\(2014\)Sensitivity analysis\.Encyclopedia of Toxicology,pp\.\.External Links:[Document](https://dx.doi.org/10.1016/B978-0-12-386454-3.00431-0)Cited by:[§A\.2](https://arxiv.org/html/2610.02480#A1.SS2.p2.1),[§1](https://arxiv.org/html/2610.02480#S1.p2.1),[§4\.1](https://arxiv.org/html/2610.02480#S4.SS1.p4.1)\.
- \[12\]D\. Smilkov, N\. Thorat, B\. Kim, F\. Viégas, and M\. Wattenberg\(2017\)SmoothGrad: removing noise by adding noise\.External Links:1706\.03825,[Link](https://arxiv.org/abs/1706.03825)Cited by:[§A\.2](https://arxiv.org/html/2610.02480#A1.SS2.p2.1),[§1](https://arxiv.org/html/2610.02480#S1.p2.1),[§2](https://arxiv.org/html/2610.02480#S2.p1.1),[§4\.1](https://arxiv.org/html/2610.02480#S4.SS1.p4.1)\.
- \[13\]J\. T\. Springenberg, A\. Dosovitskiy, T\. Brox, and M\. Riedmiller\(2015\)Striving for simplicity: the all convolutional net\.External Links:1412\.6806,[Link](https://arxiv.org/abs/1412.6806)Cited by:[§A\.2](https://arxiv.org/html/2610.02480#A1.SS2.p2.1),[§1](https://arxiv.org/html/2610.02480#S1.p2.1),[§4\.1](https://arxiv.org/html/2610.02480#S4.SS1.p4.1)\.
- \[14\]Z\. Shao, P\. Wang, Q\. Zhu, R\. Xu, J\. Song, X\. Bi, H\. Zhang, M\. Zhang, Y\. K\. Li, Y\. Wu, and D\. Guo\(2024\)DeepSeekMath: pushing the limits of mathematical reasoning in open language models\.External Links:2402\.03300,[Link](https://arxiv.org/abs/2402.03300)Cited by:[Figure 1](https://arxiv.org/html/2610.02480#S1.F1),[§3\.5](https://arxiv.org/html/2610.02480#S3.SS5.p1.1),[§4\.1](https://arxiv.org/html/2610.02480#S4.SS1.p3.1)\.
- \[15\]D\. Slack, S\. Krishna, H\. Lakkaraju, and S\. Singh\(2023\)TalkToModel: explaining machine learning models with interactive natural language conversations\.External Links:2207\.04154,[Link](https://arxiv.org/abs/2207.04154)Cited by:[§1](https://arxiv.org/html/2610.02480#S1.p3.1),[§2](https://arxiv.org/html/2610.02480#S2.p3.1)\.
- \[16\]S\. Vanbrabant, G\. Eerlings, G\. A\. Rovelo Ruiz, and D\. Vanacken\(2025\)ECHO: enhancing conversational explainable ai through tool\-augmented language models\.Proc\. ACM Hum\.\-Comput\. Interact\.9\(4\)\.External Links:[Link](https://doi.org/10.1145/3734191),[Document](https://dx.doi.org/10.1145/3734191)Cited by:[§1](https://arxiv.org/html/2610.02480#S1.p3.1),[§2](https://arxiv.org/html/2610.02480#S2.p3.1),[§3](https://arxiv.org/html/2610.02480#S3.p2.1)\.
- \[17\]H\. Kim, H\. Chen, C\. Li, and J\. M\. Lee\(2025\)TalkToAgent: a human\-centric explanation of reinforcement learning agents with large language models\.External Links:2509\.04809,[Link](https://arxiv.org/abs/2509.04809)Cited by:[§1](https://arxiv.org/html/2610.02480#S1.p3.1)\.
- \[18\]Y\. He and D\. Martens\(2026\)An agentic approach to generating xai\-narratives\.External Links:2603\.20003,[Link](https://arxiv.org/abs/2603.20003)Cited by:[§1](https://arxiv.org/html/2610.02480#S1.p3.1)\.
- \[19\]N\. Kroeger, D\. Ley, S\. Krishna, C\. Agarwal, and H\. Lakkaraju\(2024\)In\-context explainers: harnessing llms for explaining black box models\.External Links:2310\.05797,[Link](https://arxiv.org/abs/2310.05797)Cited by:[§1](https://arxiv.org/html/2610.02480#S1.p3.1),[§3\.3](https://arxiv.org/html/2610.02480#S3.SS3.p1.1),[§3\.4](https://arxiv.org/html/2610.02480#S3.SS4.p1.1)\.
- \[20\]X\. Li, M\. Du, J\. Chen, Y\. Chai, H\. Lakkaraju, and H\. Xiong\(2023\)$\\mathcal\{m\}^4$: a unified XAI benchmark for faithfulness evaluation of feature attribution methods across metrics, modalities and models\.InThirty\-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track,External Links:[Link](https://openreview.net/forum?id=6zcfrSz98y)Cited by:[§1](https://arxiv.org/html/2610.02480#S1.p3.1),[§2](https://arxiv.org/html/2610.02480#S2.p2.1),[§3\.4](https://arxiv.org/html/2610.02480#S3.SS4.p1.1),[§3](https://arxiv.org/html/2610.02480#S3.p2.1)\.
- \[21\]K\. Rawal, Z\. Fu, E\. Delaney, and C\. Russell\(2025\)Evaluating model explanations without ground truth\.InProceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency,FAccT ’25,pp\. 3400–3411\.External Links:[Link](http://dx.doi.org/10.1145/3715275.3732219),[Document](https://dx.doi.org/10.1145/3715275.3732219)Cited by:[§1](https://arxiv.org/html/2610.02480#S1.p3.1),[§3\.4](https://arxiv.org/html/2610.02480#S3.SS4.p1.1)\.
- \[22\]S\. Zhang, T\. Han, U\. Bhalla, and H\. Lakkaraju\(2025\)Towards unified attribution in explainable ai, data\-centric ai, and mechanistic interpretability\.External Links:2501\.18887,[Link](https://arxiv.org/abs/2501.18887)Cited by:[§2](https://arxiv.org/html/2610.02480#S2.p2.1)\.
- \[23\]J\. Kim, S\. Mun, S\. Lee, J\. Cho, and J\. Ok\(2025\)Towards faithful agentic XAI\.External Links:[Link](https://openreview.net/forum?id=uGPbHJ0MSe)Cited by:[§2](https://arxiv.org/html/2610.02480#S2.p3.1),[§3\.4](https://arxiv.org/html/2610.02480#S3.SS4.p1.1)\.
- \[24\]A\. Ghosh, T\. Ashraf, R\. K\. Singh, N\. Saeed, S\. Saha, X\. Chen, and S\. Khan\(2026\)Carepilot: a multi\-agent framework for long\-horizon computer task automation in healthcare\.arXiv preprint arXiv:2603\.24157\.Cited by:[§2](https://arxiv.org/html/2610.02480#S2.p3.1)\.
- \[25\]T\. K\. Halder, A\. Ghosh, S\. Baidya, A\. Roy, and S\. Saha\(2026\)ArogyaSutra: a multi\-agent framework for multimodal medical reasoning in indic languages\.arXiv preprint arXiv:2606\.13572\.Cited by:[§2](https://arxiv.org/html/2610.02480#S2.p3.1)\.
- \[26\]T\. R\. Shaham, S\. Schwettmann, F\. Wang, A\. Rajaram, E\. Hernandez, J\. Andreas, and A\. Torralba\(2025\)A multimodal automated interpretability agent\.External Links:2404\.14394,[Link](https://arxiv.org/abs/2404.14394)Cited by:[§2](https://arxiv.org/html/2610.02480#S2.p3.1)\.
- \[27\]J\. L\. Camuñas, C\. Li, T\. R\. Shaham, A\. Torralba, and A\. Lapedriza\(2025\)OpenMAIA: a multimodal automated interpretability agent based on open\-source models\.InMechanistic Interpretability Workshop at NeurIPS 2025,External Links:[Link](https://openreview.net/forum?id=KitDRi76It)Cited by:[§2](https://arxiv.org/html/2610.02480#S2.p3.1)\.
- \[28\]W\. Liu, Y\. Miao, H\. Zhao, Y\. Liu, and M\. Du\(2026\)NeuronScope: a multi\-agent framework for explaining polysemantic neurons in language models\.External Links:2601\.03671,[Link](https://arxiv.org/abs/2601.03671)Cited by:[§2](https://arxiv.org/html/2610.02480#S2.p3.1)\.
- \[29\]A\. Ghosh, N\. Kumar, N\. Patnaik, A\. Prakash, R\. Raj, and S\. Saha\(2026\)RADO: trustworthy radiology impression generation using safety and faithfulness\-based preference optimization\.ACM Transactions on Computing for Healthcare7\(3\),pp\. 1–18\.Cited by:[§3\.5](https://arxiv.org/html/2610.02480#S3.SS5.p1.1)\.
- \[30\]D\. Guo, D\. Yang, H\. Zhang, J\. Song, P\. Wang, Q\. Zhu, R\. Xu, R\. Zhang, S\. Ma, X\. Bi,et al\.\(2025\)Deepseek\-r1: incentivizing reasoning capability in llms via reinforcement learning\.arXiv preprint arXiv:2501\.12948\.Cited by:[§3\.5](https://arxiv.org/html/2610.02480#S3.SS5.p1.1)\.
- \[31\]E\. Onyame, A\. Ghosh, S\. Baidya, S\. Saha, X\. Chen, and C\. Agarwal\(2026\)Cure\-med: curriculum\-informed reinforcement learning for multilingual medical reasoning\.InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 30682–30703\.Cited by:[§3\.5](https://arxiv.org/html/2610.02480#S3.SS5.p1.1)\.
- \[32\]B\. Becker and R\. Kohavi\(1996\)Adult\.Note:UCI Machine Learning RepositoryDOI: https://doi\.org/10\.24432/C5XW20Cited by:[§4\.1](https://arxiv.org/html/2610.02480#S4.SS1.p2.1)\.
- \[33\]M\. Zwitter and M\. Soklic\(1988\)Breast Cancer\.Note:UCI Machine Learning RepositoryDOI: https://doi\.org/10\.24432/C51P4MCited by:[§4\.1](https://arxiv.org/html/2610.02480#S4.SS1.p2.1)\.
- \[34\]S\. R\. Bowman, G\. Angeli, C\. Potts, and C\. D\. Manning\(2015\)A large annotated corpus for learning natural language inference\.External Links:1508\.05326,[Link](https://arxiv.org/abs/1508.05326)Cited by:[§4\.1](https://arxiv.org/html/2610.02480#S4.SS1.p2.1)\.
- \[35\]A\. L\. Maas, R\. E\. Daly, P\. T\. Pham, D\. Huang, A\. Y\. Ng, and C\. Potts\(2011\)Learning word vectors for sentiment analysis\.InProceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies,Portland, Oregon, USA,pp\. 142–150\.External Links:[Link](http://www.aclweb.org/anthology/P11-1015)Cited by:[§4\.1](https://arxiv.org/html/2610.02480#S4.SS1.p2.1)\.
- \[36\]P\. Welinder, S\. Branson, T\. Mita, C\. Wah, F\. Schroff, S\. Belongie, and P\. Perona\(2010\)Caltech\-UCSD birds 200\.Technical reportTechnical ReportCNS\-TR\-2010\-001,California Institute of Technology\.Cited by:[§4\.1](https://arxiv.org/html/2610.02480#S4.SS1.p2.1)\.
- \[37\]A\. Coates, A\. Ng, and H\. Lee\(2011\)An analysis of single\-layer networks in unsupervised feature learning\.InProceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics,G\. Gordon, D\. Dunson, and M\. Dudík \(Eds\.\),Proceedings of Machine Learning Research, Vol\.15,Fort Lauderdale, FL, USA,pp\. 215–223\.External Links:[Link](https://proceedings.mlr.press/v15/coates11a.html)Cited by:[§4\.1](https://arxiv.org/html/2610.02480#S4.SS1.p2.1)\.
- \[38\]K\. He, X\. Zhang, S\. Ren, and J\. Sun\(2015\)Deep residual learning for image recognition\.External Links:1512\.03385,[Link](https://arxiv.org/abs/1512.03385)Cited by:[§4\.1](https://arxiv.org/html/2610.02480#S4.SS1.p2.1),[§5](https://arxiv.org/html/2610.02480#S5.p9.1)\.
- \[39\]G\. Huang, Z\. Liu, L\. van der Maaten, and K\. Q\. Weinberger\(2018\)Densely connected convolutional networks\.External Links:1608\.06993,[Link](https://arxiv.org/abs/1608.06993)Cited by:[§4\.1](https://arxiv.org/html/2610.02480#S4.SS1.p2.1)\.
- \[40\]Qwen Team\(2026\)Qwen3\.6\-35B\-A3B: agentic coding power, now open to all\.External Links:[Link](https://qwen.ai/blog?id=qwen3.6-35b-a3b)Cited by:[§A\.14](https://arxiv.org/html/2610.02480#A1.SS14.p1.1),[§4\.1](https://arxiv.org/html/2610.02480#S4.SS1.p3.1)\.
- \[41\]E\. J\. Hu, Y\. Shen, P\. Wallis, Z\. Allen\-Zhu, Y\. Li, S\. Wang, L\. Wang, and W\. Chen\(2021\)LoRA: low\-rank adaptation of large language models\.External Links:2106\.09685,[Link](https://arxiv.org/abs/2106.09685)Cited by:[§4\.1](https://arxiv.org/html/2610.02480#S4.SS1.p3.1)\.
- \[42\]J\. Wei, X\. Wang, D\. Schuurmans, M\. Bosma, B\. Ichter, F\. Xia, E\. Chi, Q\. Le, and D\. Zhou\(2023\)Chain\-of\-thought prompting elicits reasoning in large language models\.External Links:2201\.11903,[Link](https://arxiv.org/abs/2201.11903)Cited by:[§4\.1](https://arxiv.org/html/2610.02480#S4.SS1.p4.1)\.
- \[43\]S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. Narasimhan, and Y\. Cao\(2023\)ReAct: synergizing reasoning and acting in language models\.External Links:2210\.03629,[Link](https://arxiv.org/abs/2210.03629)Cited by:[§4\.1](https://arxiv.org/html/2610.02480#S4.SS1.p4.1)\.
- \[44\]S\. Yao, D\. Yu, J\. Zhao, I\. Shafran, T\. L\. Griffiths, Y\. Cao, and K\. Narasimhan\(2023\)Tree of thoughts: deliberate problem solving with large language models\.External Links:2305\.10601,[Link](https://arxiv.org/abs/2305.10601)Cited by:[§4\.1](https://arxiv.org/html/2610.02480#S4.SS1.p4.1)\.
- \[45\]R\. K\. Mothilal, A\. Sharma, and C\. Tan\(2020\)Explaining machine learning classifiers through diverse counterfactual explanations\.InProceedings of the 2020 Conference on Fairness, Accountability, and Transparency,pp\. 607–617\.Cited by:[§4\.1](https://arxiv.org/html/2610.02480#S4.SS1.p4.1),[§5](https://arxiv.org/html/2610.02480#S5.p6.1)\.
- \[46\]T\. Laugel, M\. Lesot, C\. Marsala, X\. Renard, and M\. Detyniecki\(2017\)Inverse classification for comparison\-based interpretability in machine learning\.arXiv preprint arXiv:1712\.08443\.Cited by:[§4\.1](https://arxiv.org/html/2610.02480#S4.SS1.p4.1),[§5](https://arxiv.org/html/2610.02480#S5.p6.1)\.
- \[47\]C\. Agarwal, S\. Krishna, E\. Saxena, M\. Pawelczyk, N\. Johnson, I\. Puri, M\. Zitnik, and H\. Lakkaraju\(2022\)Openxai: towards a transparent evaluation of model explanations\.Advances in neural information processing systems35,pp\. 15784–15799\.Cited by:[§4\.1](https://arxiv.org/html/2610.02480#S4.SS1.p5.1)\.
- \[48\]R\. Rombach, A\. Blattmann, D\. Lorenz, P\. Esser, and B\. Ommer\(2022\)High\-resolution image synthesis with latent diffusion models\.External Links:2112\.10752,[Link](https://arxiv.org/abs/2112.10752)Cited by:[§4\.1](https://arxiv.org/html/2610.02480#S4.SS1.p5.1)\.
- \[49\]B\. L\. WELCH\(1947\)THE generalization of ‘student’s’ problem when several different population varlances are involved\.Biometrika34\(1\-2\),pp\. 28–35\.External Links:ISSN 0006\-3444,[Document](https://dx.doi.org/10.1093/biomet/34.1-2.28),[Link](https://doi.org/10.1093/biomet/34.1-2.28),https://academic\.oup\.com/biomet/article\-pdf/34/1\-2/28/553093/34\-1\-2\-28\.pdfCited by:[§A\.13](https://arxiv.org/html/2610.02480#A1.SS13.p1.1),[§5](https://arxiv.org/html/2610.02480#S5.p2.1)\.
- \[50\]N\. Shinn, F\. Cassano, E\. Berman, A\. Gopinath, K\. Narasimhan, and S\. Yao\(2023\)Reflexion: language agents with verbal reinforcement learning\.External Links:2303\.11366,[Link](https://arxiv.org/abs/2303.11366)Cited by:[§5](https://arxiv.org/html/2610.02480#S5.p8.1)\.
- \[51\]X\. Zhang, J\. Zhao, and Y\. LeCun\(2016\)Character\-level convolutional networks for text classification\.External Links:1509\.01626,[Link](https://arxiv.org/abs/1509.01626)Cited by:[§5](https://arxiv.org/html/2610.02480#S5.p9.1)\.
- \[52\]J\. Devlin, M\. Chang, K\. Lee, and K\. Toutanova\(2019\)BERT: pre\-training of deep bidirectional transformers for language understanding\.External Links:1810\.04805,[Link](https://arxiv.org/abs/1810.04805)Cited by:[§5](https://arxiv.org/html/2610.02480#S5.p9.1)\.
- \[53\]D\. Dua and C\. Graff\(2019\)UCI machine learning repository\.Note:[http://archive\.ics\.uci\.edu/ml](http://archive.ics.uci.edu/ml)Cited by:[§5](https://arxiv.org/html/2610.02480#S5.p9.1)\.
- \[54\]A\. Krizhevsky G\. Hintonet al\.\(2009\)Learning multiple layers of features from tiny images\.\(2009\)\.Cited by:[§5](https://arxiv.org/html/2610.02480#S5.p9.1)\.
- \[55\]J\. Bai, S\. Bai, S\. Yang, S\. Wang, S\. Tan, P\. Wang, J\. Lin, C\. Zhou, and J\. Zhou\(2023\)Qwen\-vl: a versatile vision\-language model for understanding, localization, text reading, and beyond\.arXiv preprint arXiv:2308\.12966\.Cited by:[§A\.6](https://arxiv.org/html/2610.02480#A1.SS6.p1.1)\.
- \[56\]B\. Kim, M\. Wattenberg, J\. Gilmer, C\. Cai, J\. Wexler, F\. Viegas, and R\. Sayres\(2018\)Interpretability beyond feature attribution: quantitative testing with concept activation vectors \(tcav\)\.InInternational Conference on Machine Learning,pp\. 2668–2677\.Cited by:[§A\.7](https://arxiv.org/html/2610.02480#A1.SS7.p5.1)\.
- \[57\]Qwen Team\(2026\)Qwen3\.5: towards native multimodal agents\.External Links:[Link](https://qwen.ai/blog?id=qwen3.5)Cited by:[§A\.11](https://arxiv.org/html/2610.02480#A1.SS11.p1.1)\.
- \[58\]T\. M\. Lab\(2025\)Tinker\.External Links:[Link](https://thinkingmachines.ai/tinker/)Cited by:[§A\.14](https://arxiv.org/html/2610.02480#A1.SS14.SSS0.Px1.p1.1)\.

## Appendix AAppendix

The Appendix provides supplementary material supporting the main paper, including the complete question taxonomy \([A\.1](https://arxiv.org/html/2610.02480#A1.SS1)\), details of the XAI tools employed \([A\.2](https://arxiv.org/html/2610.02480#A1.SS2)\), faithfulness evaluation metrics \([A\.3](https://arxiv.org/html/2610.02480#A1.SS3)\), a formal RL problem definition \([A\.4](https://arxiv.org/html/2610.02480#A1.SS4)\), dataset statistics \([A\.5](https://arxiv.org/html/2610.02480#A1.SS5)\), descriptions of the sequential and hybrid training procedures \([A\.6](https://arxiv.org/html/2610.02480#A1.SS6)\), ablation studies on out\-of\-distribution \(OOD\) question types, unseen datasets/architectures, and concept\-level explanation generalization \([A\.7](https://arxiv.org/html/2610.02480#A1.SS7)\), prompt robustness \([A\.8](https://arxiv.org/html/2610.02480#A1.SS8)\), the critic agent \([A\.9](https://arxiv.org/html/2610.02480#A1.SS9)\), the Proposer’s strategy modes \([A\.10](https://arxiv.org/html/2610.02480#A1.SS10)\), backbone scale transfer \([A\.11](https://arxiv.org/html/2610.02480#A1.SS11)\), a sanity check on a classifier with a synthetic shortcut \([A\.12](https://arxiv.org/html/2610.02480#A1.SS12)\), statistical significance analyses \([A\.13](https://arxiv.org/html/2610.02480#A1.SS13)\), computational cost details \([A\.14](https://arxiv.org/html/2610.02480#A1.SS14)\) and prompt templates for Actor and Proposer Agents \([A\.15](https://arxiv.org/html/2610.02480#A1.SS15)\)\.

### A\.1Question Taxonomy

The detailed question definition and category is showed in Table[3](https://arxiv.org/html/2610.02480#A1.T3)\.

Table 3:Overview of the ten explanation questions organised by category\.IDCategoryQuestionFeature Attribution \- which parts of the input drove or suppressed the prediction?Q1Most ResponsibleWhich parts of the instance are most responsible for the model’s prediction?Q2Least ResponsibleWhich parts of the instance are least responsible for the model’s prediction?Q3DistinctiveWhich specific parts of the instance distinguish its prediction from the next\-best alternative?Q4ContrastiveWhy are instancesAAandBBgiven different predictions?Counterfactual \- how sensitive is the prediction to hypothetical modifications?Q5Mask PredictionIf we mask a certain part of this instance, would the prediction change?Q6Flip PredictionHow should the instance change to flip the model into a different prediction?Q7Change PredictionIf we remove or change one important part of the instance, how would the prediction change?Spurious Feature \- does the model rely on irrelevant cues?Q8Irrelevant PartsIs there any irrelevant part in this instance that causes the model’s wrong prediction?Q9Shared FeatureWhat shared feature makes misclassified instancesAA,BB,CC, …difficult for the model?Q10Similar\-DifferentWhy are instancesAAandBBassigned different predictions despite apparent similarity?
### A\.2XAI Tools

To support explanation generation, our framework provides a library of established attribution methods spanning all three modalities:

Gradient\-based\.GradCAM\[[9](https://arxiv.org/html/2610.02480#bib.bib28)\]\(layer\-wise gradient attribution, vision only\), Integrated Gradients\[[8](https://arxiv.org/html/2610.02480#bib.bib39)\], Guided Backpropagation\[[13](https://arxiv.org/html/2610.02480#bib.bib29)\], SmoothGrad\[[12](https://arxiv.org/html/2610.02480#bib.bib32)\]\(noise\-averaged gradient estimation\), and Sensitivity Analysis\[[11](https://arxiv.org/html/2610.02480#bib.bib33)\]\(input\-gradient saliency, text and tabular only\)\.

Perturbation\-based\.LIME\[[7](https://arxiv.org/html/2610.02480#bib.bib30)\]\(Local Interpretable Model\-agnostic Explanations with superpixel or token\-level segmentation for vision, and token\-level perturbation for text\) and SHAP\[[10](https://arxiv.org/html/2610.02480#bib.bib31)\]\(Shapley Additive exPlanations via marginal contribution estimation\)\.

### A\.3Faithfulness Metrics

Table 4:Summary of evaluation metrics corresponding to different explanation faithfulness questions\.Heregaporig,gapmod\\text\{gap\}\_\{\\text\{orig\}\},\\text\{gap\}\_\{\\text\{mod\}\}are the original gap between top\-1 and top\-2 predicted classes and the gap after perturbation;y∗y^\{\*\}is the ground\-truth class;y^\\hat\{y\}is the predicted class after perturbation;δ\\deltais the prediction probability drop after perturbation; andFcorrect,FwrongF\_\{\\text\{correct\}\},F\_\{\\text\{wrong\}\}are attribution maps for the correctly and incorrectly classified instances respectively\.

A higher metric score indicates a more faithful explanation\. For binary metrics \(Q3, Q5, Q6, Q7\), the score is 1 if the condition holds and 0 otherwise\. For Q6, tabular and text inputs are evaluated against a specific target classyty\_\{t\}, whereas vision inputs are evaluated by whether the prediction changes to any alternative class, reflecting the substantially larger label space in vision tasks\.

The detailed metric calculation method and intuition is showed in Table[4](https://arxiv.org/html/2610.02480#A1.T4)\.

### A\.4Formal RL Problem Definition

We formalizeMea’s training as a two\-step Markov Decision Process \(MDP\) over a single episode, with the Proposer and Actor as two sequential decision points sharing backboneθ\\theta\.

State \(Proposer step\)\.sProp=\(q,𝐱,y^,𝒯m\)s^\{\\mathrm\{Prop\}\}=\(q,\\,\\mathbf\{x\},\\,\\hat\{y\},\\,\\mathcal\{T\}\_\{m\}\): the question, input instance, frozen\-classifier prediction, and the modality\-compatible tool set \(Sec\.[3\.3](https://arxiv.org/html/2610.02480#S3.SS3), Eq\.[3](https://arxiv.org/html/2610.02480#S3.E3)\)\.

Action \(Proposer\)\.The structured strategyS=\(stype,𝒯sel,𝒜auto\)S=\(s\_\{\\mathrm\{type\}\},\\,\\mathcal\{T\}\_\{\\mathrm\{sel\}\},\\,\\mathcal\{A\}\_\{\\mathrm\{auto\}\}\)\(Sec\.[3\.3](https://arxiv.org/html/2610.02480#S3.SS3)\)\.𝒯sel\\mathcal\{T\}\_\{\\mathrm\{sel\}\}and𝒜auto\\mathcal\{A\}\_\{\\mathrm\{auto\}\}are combinatorial subsets rather than a single discrete choice, so this is regarded as a structured action space\.

Transition\.Deterministic: each selected toolt∈𝒯selt\\in\\mathcal\{T\}\_\{\\mathrm\{sel\}\}is executed against the frozen classifierffand input𝐱\\mathbf\{x\}, producing summarized outputs\{ot\}t∈𝒯sel\\\{o\_\{t\}\\\}\_\{t\\in\\mathcal\{T\}\_\{\\mathrm\{sel\}\}\}\(Sec\.[3\.3](https://arxiv.org/html/2610.02480#S3.SS3), Step 1\)\. No stochasticity is introduced by the environment; the only randomness in the episode comes from the two agents’ own sampling\.

State \(Actor step\)\.sAct=\(q,𝐱,y^,S,\{ot\}t∈𝒯sel\)s^\{\\mathrm\{Act\}\}=\(q,\\,\\mathbf\{x\},\\,\\hat\{y\},\\,S,\\,\\\{o\_\{t\}\\\}\_\{t\\in\\mathcal\{T\}\_\{\\mathrm\{sel\}\}\}\): the Proposer’s emitted strategy together with the summarized tool outputs produced by the transition above\.

Action \(Actor\)\.The explanatione∈ℰme\\in\\mathcal\{E\}\_\{m\}\(Eq\.[4](https://arxiv.org/html/2610.02480#S3.E4)\)\.

Reward\.Terminal only:R=mi​\(e\)R=m\_\{i\}\(e\), the faithfulness metric for question typeQiQ\_\{i\}\(Sec\.[3\.4](https://arxiv.org/html/2610.02480#S3.SS4)\), augmented by the modality\-adaptive size/tool penalties \(Eq\.[9](https://arxiv.org/html/2610.02480#S3.E9)and the tool penalty in Sec\.[3\.3](https://arxiv.org/html/2610.02480#S3.SS3)\), evaluated strictly after the Actor’s final output\. No intermediate reward is given at the Proposer’s transition\.

Shared parameterization\.BothπθProp\\pi^\{\\mathrm\{Prop\}\}\_\{\\theta\}andπθAct\\pi^\{\\mathrm\{Act\}\}\_\{\\theta\}route through the*same*LoRA\-adapted backboneθ\\theta; the two roles are differentiated entirely by distinct system prompts \(Appendix[A\.15](https://arxiv.org/html/2610.02480#A1.SS15), Figs\.[12](https://arxiv.org/html/2610.02480#A1.F12),[11](https://arxiv.org/html/2610.02480#A1.F11)\) rather than separate adapters or a mode flag\. BecauseRRis terminal and both transitions belong to the same episode, the group\-normalized advantageA^k\\hat\{A\}\_\{k\}\(Eq\.[8](https://arxiv.org/html/2610.02480#S3.E8)\) is applied identically to every token in both the Proposer’s and the Actor’s transitions, implementing the joint credit assignment described in Sec\.[3\.3](https://arxiv.org/html/2610.02480#S3.SS3)\.

### A\.5Dataset Statistics

![Refer to caption](https://arxiv.org/html/2610.02480v1/images/dataset_stats.jpeg)Figure 5:Dsitribution of data across modalities\.Due to the heterogeneous prediction behavior exhibited across models and datasets, the number of generated instances varies across categories \(Q1\-Q10\)\. Different question types require specific prediction configurations, such as correct predictions, informative misclassifications, contrastive prediction pairs, or shared behavioral patterns across models\. Consequently, the final instance count for each category is determined by the availability of meaningful and semantically valid examples satisfying these construction criteria\. This naturally results in varying sample counts across question types, while ensuring that each generated instance reflects authentic model behavior and preserves the fidelity of the underlying prediction distributions\.

### A\.6Sequential and Hybrid Training

We compare two multi\-modal GRPO training schedules, both using Qwen3\-VL\-30B\-A3B\-Instruct\[[55](https://arxiv.org/html/2610.02480#bib.bib44)\]\.Sequentialtraining fine\-tunes one modality at a time, carrying each checkpoint forward to the next phase\.Hybridtraining interleaves all three modalities within every batch, grouped by question type via Q\-type stratification\.

### Sequential Training

#### Phase 1 \- Tabular\.

Fine\-tuning on tabular data across 3 epochs drives the mean batch reward from 0\.3 to 0\.6\. The resulting checkpoint achieves a tabular faithfulness score of 0\.632\.

#### Phase 2 \- Text\.

Continuing from the tabular checkpoint drives the batch reward from 0\.4 to 0\.6 on text examples\. However, re\-evaluating the new checkpoint on the*tabular*test set reveals measurable catastrophic forgetting\. The degradation happens across question types: Q3 drops from 0\.326 to 0\.264 \(−\-0\.06\) and Q6 from 0\.780 to 0\.728 \(−\-0\.05\)\. Both Q\-types demand accurate numerical reasoning over feature rankings or feature value intervals – skills reinforced during tabular training and partially overwritten as the gradient signal shifts to text\-span operations\.

#### Phase 3 \- Vision\.

Vision training for 200 steps shows a markedly weaker learning signal: the batch reward rises only from 0\.512 to 0\.578, compared to the steep trajectories in Phases 1 and 2 \(Figure[6](https://arxiv.org/html/2610.02480#A1.F6), left\)\. We attribute the limited learning to two compounding effects\. First, by Phase 3 the model’s policy is already tightly peaked around tabular/text output formats, leaving insufficient entropy to explore the vision\-specific bounding\-box\. Second, the flat reward curve indicates that rollout diversity has collapsed: all sampled trajectories produce similar rewards, so the group\-normalized advantageA^k≈0\\hat\{A\}\_\{k\}\\approx 0for most rollouts, suppressing the gradient signal\.

### Hybrid Training

Hybrid training on all three modalities simultaneously runs for 902 steps across 3 epochs, with the batch reward rising from 0\.313 to 0\.594 \(Figure[6](https://arxiv.org/html/2610.02480#A1.F6), right\)\. The slower convergence relative to any single\-modality phase reflects the wider diversity of the joint training distribution: the model must maintain a flexible policy across tabular, text, and vision inputs simultaneously\.

Figure 6:Training reward curves\.\(Left\)Sequential training:Vertical dashed lines mark phase boundaries \(Tab→\\toText→\\toVision\); the vision reward barely rises, indicating collapsed rollout diversity\.\(Right\)Hybrid training \(all modalities interleaved\):The recurring spikes are an artifact of the sampling seed, which concentrates relatively easy questions towards the end of each epoch\.
### A\.7Ablation Study: OOD Generalization

We evaluateMea’s out\-of\-distribution generalization along three axes: held\-out question types within the proposed taxonomy, unseen datasets and model architectures, and a held\-out question type entirely outside the taxonomy \(concept\-level explanation\)\.

Held\-out question types\.We use spurious feature questions as OOD question types, asMeais never trained on them \(see Sec\.[3](https://arxiv.org/html/2610.02480#S3)\)\. Results in Table[5](https://arxiv.org/html/2610.02480#A1.T5)show thatMeaachieves higher overall faithfulness on tabular and text modalities, with particularly notable gains on Q8 for text\. On vision inputs, the base agent slightly outperformsMea, suggesting that generalization to spurious feature reasoning remains challenging for vision modality without explicit training signal\.

Table 5:OOD faithfulness scores on held\-out question types Q8 and Q9\. Best per row inbold\.Unseen datasets and model architectures\.\.Table[6](https://arxiv.org/html/2610.02480#A1.T6)reports faithfulness on entirely unseen dataset\-model combinations for each modality\.Meaachieves the highest overall faithfulness across all three modalities, though it underperforms the base agent on text counterfactual and vision feature attribution, which we attribute to further hyperparameter tuning needed for better modality\-specific trade\-offs\.

Table 6:OOD faithfulness scores on different datasets and model architectures\. Best performance isbolded\.Concept\-level explanation generalization\.Mea’s faithfulness protocol \(Sec\.[3\.4](https://arxiv.org/html/2610.02480#S3.SS4)\) is fundamentally tied to*local input\-region perturbation*: masking a feature subset, token span, or bounding box\. To test generalization beyond input\-perturbation and beyond the trained question taxonomy entirely, we introduce a held\-out question type,Q11\(“Which concept was most responsible for the model’s prediction?”\), on CUB\-200\-2011, which asks the agent to name a semantic concept \(e\.g\.,“bill shape: hooked seabird”\) rather than a region\.

Faithfulness is scored by training aConcept Activation Vector\(CAV\)\[[56](https://arxiv.org/html/2610.02480#bib.bib54)\], which is a linear probe on the model’s penultimate\-layer representation for each concept, and orthogonally projecting that direction out of the activation \(h′=h−\(h⋅v^\)​v^h^\{\\prime\}=h\-\(h\\cdot\\hat\{v\}\)\\hat\{v\}\) before re\-running only the classifier head; the probability dropmax⁡\(0,Poriginal−Pmodified\)\\max\(0,P\_\{\\mathrm\{original\}\}\-P\_\{\\mathrm\{modified\}\}\)is the faithfulness score, and the entire measurement happens in representation space\. The Proposer/Actor pipeline still usesMea’s existing tool library to decide which concept to name \(Appendix[A\.2](https://arxiv.org/html/2610.02480#A1.SS2)\); only the evaluation mechanism differs from Q1–Q10\.

Table[7](https://arxiv.org/html/2610.02480#A1.T7)shows that, across 120 questions \(cub\_resnet\+cub\_densenet\),Meascores∼\\sim18% higher mean concept faithfulness than the base agent despite never being trained on this question type\. Together, these results indicateMea’s thought\-action\-observation loop generalizes across ablation targets \(input region*or*representation direction\)\.

Table 7:Base agent vs\.Meaon Q11 \(concept attribution\), 120 questions\.
### A\.8Ablation Study: Prompt Robustness

To assess prompt robustness, we generate three semantically\-equivalent paraphrases of every question in the taxonomy\. Dynamic values \(feature names, row IDs, class labels, masked tokens\) are extracted from the original question via regular expressions and re\-injected verbatim into each paraphrase template, preserving semantic equivalence\. Q1\-Q4 receive three lexically distinct variants; Q5\-Q7 and Q10 share a single uniform paraphrase across all three runs \(variant\-independent templates\)\. Table[8](https://arxiv.org/html/2610.02480#A1.T8)lists the question\-field phrasings for each type, illustrated with tabular\-modality examples\. Due to computational constraints, paraphrase evaluation is conducted on one dataset per modality\.

![Refer to caption](https://arxiv.org/html/2610.02480v1/ablation_paraphrase_vs_trained.png)Figure 7:Ablation on prompt robustness\.Mean faithfulness scores ofMeaevaluated on three paraphrased variants of the question taxonomy across\(A\)tabular,\(B\)text, and\(C\)vision modalities\. Minimum variance indicates thatMeais robust to changes in question phrasing\.Table 8:Three paraphrase variants of each question type \(Q\-field only; tabular modality shown\)\. Dynamic values such as feature names, row IDs, and class labels \(denoted \{placeholder\}\) are extracted from the original question via regex and re\-injected verbatim at generation time\.
### A\.9Ablation Study: Critic Agent

The Critic Agent augments the base agent pipeline with an inference\-time reflection step\. After the Actor Agent produces a strategy and explanation and receives a faithfulness score, the Critic receives a structured prompt and returns two critiques: a*Proposer Reflection*\(feedback on tool selection\) and an*Actor Reflection*\(feedback on explanation quality\)\. The actor agent then regenerates its strategy and explanation conditioned on this feedback\. Fig\.[8](https://arxiv.org/html/2610.02480#A1.F8)shows thatMeaconsistently outperforms Agent\+Reflection despite the added inference\-time step: on text counterfactual questions, Agent\+Reflection scores0\.6440\.644whileMeareaches0\.8760\.876\(\+36%\); on vision counterfactual questions,0\.4390\.439vs\.0\.5690\.569\(\+30%\)\. Strikingly, on tabular spurious features the critic step even*degrades*performance \(0\.5140\.514\) relative to the base agent \(0\.6770\.677\), whileMeaachieves0\.9970\.997\. These results demonstrate that RL\-based end\-to\-end training provides substantially stronger and more reliable gains than inference\-time critique\.

![Refer to caption](https://arxiv.org/html/2610.02480v1/ablation_three_way_comparison.png)Figure 8:Ablation on critic agent\.Mean faithfulness scores across three que1on categories for the base agent, base agent augmented with a critic agent, andMea, evaluated on\(A\)tabular,\(B\)text, and\(C\)vision modalities\.Meaconsistently outperforms both baselines, demonstrating that RL\-based training yields stronger improvements than inference\-time critique\.Prompt\.Figure[10](https://arxiv.org/html/2610.02480#A1.F10)shows the complete prompt structure passed to the Critic Agent\. Each section is populated with runtime values extracted from the pipeline outputs\.

Output Schema\.The Critc returns two feedbacks, separated by the markersPROPOSER\_REFLECTIONandACTOR\_REFLECTION\. Table[9](https://arxiv.org/html/2610.02480#A1.T9)describes each field\. The Proposer Reflection is consumed by the Proposer Agent to revise its tool selection; the Actor Reflection is consumed by the Actor Agent to revise its explanation\.

Table 9:Output JSON schema of the critic agent\.ObjectFieldDescriptionPROPOSERanalysisOverall assessment of tool selectiontool\_analysisPer\-tool rating \(high/medium/low\) and recommendation \(keep/remove/replace\)tools\_to\_keepTools to retain in the revised strategytools\_to\_removeTools to droptools\_to\_addNew tools to includestrategy\_suggestionsFree\-text suggestions to improve tool selectionACTORanalysisOverall assessment of explanation qualityidentified\_issuesSpecific errors or weaknesses in the explanationregion\_feedbackFeedback on the accuracy of the identified region or featureexplanation\_feedbackFeedback on clarity and correctnessimprovement\_suggestionsFree\-text suggestions to improve the explanationCoT Actor Preamble\#\# Reasoning Mode: Chain\-of\-Thought Before generating your explanation, reason through the evidence step\-by\-step: 1\.Understand the question: What exactly is being asked?2\.Analyze the XAI tool results: What do the attribution maps, importance scores, and statistics tell us?3\.Identify the key features/regions: Based on the tool outputs, which specific parts of the input are most relevant?4\.Synthesize your answer: Combine the evidence into a clear, specific, and well\-supported explanation\.Think step by step, then provide your final answer as valid JSON\.

ReAct Actor Preamble\#\# Reasoning Mode: ReAct \(Reasoning \+ Acting\) Interleave your reasoning with the evidence to build your explanation: For each piece of evidence from the XAI tools: Thought: What does this result tell us about the model’s decision? Observation: Note the key values, features, or regions highlighted\. Reflection: How does this connect to answering the question? After processing all evidence: Final Thought: Synthesize all observations into a coherent answer\. Then provide your final answer as valid JSON\.

ToT Actor Preamble\#\# Reasoning Mode: Tree of Thought Consider multiple possible interpretations of the evidence before settling on your answer: Interpretation 1: What does the evidence suggest if you weight the highest\-attribution features most heavily? Interpretation 2: What alternative explanation could account for the same tool results? Interpretation 3: Which interpretation is most consistent with ALL the evidence, not just the strongest signal? Evaluate each interpretation against the evidence quality, specificity, and completeness\. Select the strongest interpretation and provide your final answer as valid JSON\.

Figure 9:Generic actor preambles prepended to the actor prompt for each reasoning baseline across all question types\.
### A\.10Ablation Study: Proposer Strategy Mode

The Proposer’s strategy schemaS=\(stype,𝒯sel,𝒜auto\)S=\(s\_\{\\mathrm\{type\}\},\\mathcal\{T\}\_\{\\mathrm\{sel\}\},\\mathcal\{A\}\_\{\\mathrm\{auto\}\}\)\(Sec\.[3\.3](https://arxiv.org/html/2610.02480#S3.SS3)\) allowsstype∈\{autonomous,tools,hybrid\}s\_\{\\mathrm\{type\}\}\\in\\\{\\texttt\{autonomous\},\\,\\texttt\{tools\},\\,\\texttt\{hybrid\}\\\}, but the main results always let the Proposer choose freely among the three\. To isolate how much autonomous reasoning contributes beyond tool use alone, we force the Proposer into each single\-mechanism regime –tool\_only\(external XAI tools alone, autonomous reasoning disabled\) andautonomous\_only\(self\-reasoning alone, zero tool calls\) – for both the base agent andMea, over a fixed 500\-question sample stratified by \(dataset×\\timesquestion type\) and allocated across modalities in proportion to test\-set size \(tabular 150 / text 195 / vision 155; Q1–Q7, Q10\)\. Isolation is enforced at two levels: the disallowed branch’s schema field is omitted from the Proposer prompt entirely, and a post\-hoc check strips any leaked fields and fails the sample if the required branch is empty – verified directly against execution logs \(zero leaked autonomous tasks intool\_onlyruns, zero tool invocations inautonomous\_onlyruns\)\.

Table 10:Mean faithfulness by model and Proposer strategy mode\.Tool use is the stronger single mechanism for both models, but autonomous reasoning with*zero*external tool calls already recovers 87\.3% \(base\) / 91\.3% \(Mea\) of tool\-only faithfulness\. Training improves the autonomous\-only arm*more*than the tool\-only arm \(\+0\.153\+0\.153vs\.\+0\.141\+0\.141mean faithfulness\), narrowing the tool/autonomous gap after training\. Most notably, forMealetting the Proposer choose freely \(0\.7480\.748\) outperforms*both*forced single\-mechanism arms in every one of the three modalities \(tabular\+0\.006\+0\.006, text\+0\.018\+0\.018, vision\+0\.026\+0\.026over the better single mode\), whereas for the base agent, free choice \(0\.5160\.516\) is statistically indistinguishable from autonomous\-only and clearly below tools\-only \(0\.5890\.589\) – i\.e\., productively combining the two mechanisms is a capability training creates, not a free byproduct of offering both options\. On Q6 \(flip prediction\) specifically,Mea’s free\-choice arm \(0\.8220\.822\) exceeds both tool\_only \(0\.7660\.766\) and autonomous\_only \(0\.7820\.782\) taken individually on the identical instances, a direct in\-model demonstration that the two mechanisms are complementary rather than redundant\.

### A\.11Ablation Study: Backbone Scale Transfer

To test whetherMea’s training benefit transfers to a substantially smaller policy backbone, we repeat the Base\-Agent\-vs\-Meacomparison onQwen3\.5\-4B\[[57](https://arxiv.org/html/2610.02480#bib.bib53)\]\(roughly an order of magnitude smaller in active parameters\), in addition to training\-reward curves on a second backbone Qwen3\-VL\-30B\-A3B\-Instruct in Appendix[A\.6](https://arxiv.org/html/2610.02480#A1.SS6)\. We train Qwen3\.5\-4B for a single epoch \(300 LoRA steps, rank 32\) with the identical reward pipeline, under the same evaluation protocol used throughout the paper\.

Table[11](https://arxiv.org/html/2610.02480#A1.T11)shows that, on this smaller 4B backbone,Meashows similar improvement as reported for the 30B & 35B backbones, suggesting that training improves faithfulness in every modality overall \(tabular0\.419→0\.6710\.419\\to 0\.671\(\+60%\+60\\%\), text0\.403→0\.5750\.403\\to 0\.575\(\+43%\+43\\%\), vision0\.142→0\.5080\.142\\to 0\.508\(\+258%\+258\\%\)\), demonstrating that faithfulness gain comes from the training procedure rather than being an artifact specific to a single, large backbone\.

Table 11:Faithfulness by question category, 4B backbone\. Best per row inbold\.
### A\.12Sanity Check: Synthetic Shortcut on a Biased Classifier

A concern with any LLM\-based explanation agent is that it may be faithful to the target model only when the model’s decisions already align with the LLM’s own semantic priors, rather than to the target model’s actual \(possibly idiosyncratic\) decision process\. To test this directly, we trained aTwoLayerNNon Adult Census with an injected synthetic shortcut column,verification\_flag,∼\\sim90% correlated with the true label \(noise rate0\.100\.10, seed4242\) – a feature whose dominance a domain expert or LLM prior would not anticipate for an income\-prediction task\. We independently verified the injected bias before any agent evaluation: overall test accuracy0\.9390\.939; accuracy on rows where the flag agrees with the true label \(89\.7%89\.7\\%of rows\) is0\.9810\.981, dropping to0\.5750\.575on the10\.3%10\.3\\%of rows where they disagree, and on those disagreement rows the model follows the flag \(not the truth\)42\.5%42\.5\\%of the time; an ablation flip\-rate test confirmsverification\_flagis∼\\sim20×\\timesmore influential on the model’s output than a comparable real feature \(education\-num flip rate0\.8%0\.8\\%vs\.16\.1%16\.1\\%\)\.

Table 12:Faithfulness score by question category on the biased classifier, pairedtt\-test by \(q\_type, row\_no\)\.Meawins or ties on every one of the 8 question types tested \(Q1–Q7, Q10\), with 5 of 8 significant atp<0\.05p<0\.05\(Table[12](https://arxiv.org/html/2610.02480#A1.T12)\)\. Beyond the aggregate score, we checked whether each agent actually*names*the shortcut when it matters: on the 13/20 Q1 rows whereverification\_flagis verifiably the top driver by occlusion importance,Meanames it correctly in 12/13 vs\. 9/13 for the base agent; on Q2 \(“least responsible feature”\), the base agent incorrectly callsverification\_flagleast responsible on 10/20 rows where it is actually dominant – a genuine faithfulness failure – whileMeadoes this 0/20 times\. On a classifier whose decision logic is engineered to deviate from human/LLM\-intuitive reasoning,Mea’s explanations remain grounded in the model’s actual \(biased\) decision process rather than reverting to prior\-knowledge\-consistent reasoning\. We scope this finding to the single bias\-injection design and tabular modality tested here, and note it does not directly test vision\-specific shortcuts \(e\.g\.,watermarks\) or amplified real sensitive attributes\.

### A\.13Statistical Significance Tests

We report two\-sided Welch’stt\-tests \(unequal variance, unpaired\)\[[49](https://arxiv.org/html/2610.02480#bib.bib42)\]for comparisons against independent baselines, and pairedtt\-tests for the paraphrase ablation \(where scores are matched by sample ID across conditions\)\. The significance level isα=0\.05\\alpha=0\.05; “n\.s\.” denotesp\>0\.05p\>0\.05\. All values are mean±\\pmstandard error of the mean \(SEM\)\.

Vision Feature\-Attribution Tool\-Only Baselines\.

Table[13](https://arxiv.org/html/2610.02480#A1.T13)comparesMea\(Q1, vision;n=96n=96, mean=0\.499±0\.029=0\.499\\pm 0\.029\) against six tool\-only XAI baselines evaluated on the same vision feature\-attribution task \(Sensitivity had no evaluation data and is excluded\)\.Meais statistically equivalent to three gradient\-based methods \(IG, Guided Backprop, SmoothGrad\) and significantly outperforms three others \(SHAP, LIME, GradCAM\)\.

Table 13:Welch’stt\-test:Meavs\. tool\-only XAI baselines \(vision, feature attribution Q1\)\.Meamean=0\.499±0\.029=0\.499\\pm 0\.029\(n=96n=96\)\.Spurious\-Feature Faithfulness \(Q10\)\.Table[14](https://arxiv.org/html/2610.02480#A1.T14)reports Welch’stt\-test results for the spurious\-feature question type \(Q10\) across text and vision modalities\.Meaachievesceiling performance\(1\.000±0\.0001\.000\\pm 0\.000,n=160n=160\) on both modalities–every sample is scored correctly\. All Welch’stt\-tests returnp<0\.05p<0\.05; however, this should be interpreted in light of the ceiling effect: when one group has zero variance, any non\-zero difference in means will be flagged as significant regardless of its practical magnitude\. For the strongest baselines the actual performance gap is negligible–Gemini\-3\.1 reaches0\.9960\.996on text \(gap:\+0\.004\+0\.004\) and Base Agent reaches0\.9810\.981on vision \(gap:\+0\.019\+0\.019\)–indicating that top\-performing models already saturate Q10 andMea’s advantage is marginal\. The significantpp\-values for weaker baselines \(e\.g\. Claude\-Haiku\-4\.5:0\.6650\.665text,0\.8550\.855vision\) do reflect a meaningful improvement\. CoT, ToT, and ReAct baselines had no Q10 evaluation data available\.

Table 14:Welch’stt\-test:Mea\(1\.000±0\.0001\.000\\pm 0\.000,n=160n=160\) vs\. baselines on spurious\-feature questions \(Q10\)\.Paraphrase Robustness Ablation\.Table[15](https://arxiv.org/html/2610.02480#A1.T15)presents pairedtt\-tests comparing Paraphrase \(mean of three paraphrase runs, matched on common sample IDs\) againstMea\. In 8 of 9 question\-type×\\timesmodality conditions, there is no statistically significant difference, confirming thatMea’s performance is robust to prompt paraphrasing\. The single significant difference \(vision counterfactual,p=0\.026p=0\.026\) shows Paraphrase outperformingMeaby0\.0750\.075, suggesting that this question type may benefit from additional training diversity\.

Table 15:Pairedtt\-test: Paraphrase vs\.Mea\(matched common samples\)\.Δ=\\Delta=Paraphrase−\-Mea\.Note:“–” denotes that both conditions achieve identical ceiling performance \(all scores=1\.0=1\.0\), makingtt\-test inapplicable\.

Critic Agent Prompt TemplateSystem Role:You are a critical analyst evaluating an XAI agent’s strategy and explanation\.User Input:Original Question⟨\\langlequestion text / example context⟩\\rangleQuestion ContextQuestion Type: Q⟨\\langle1\-10⟩\\rangleModality:⟨\\langletabular\|\|text\|\|vision⟩\\rangleStrategy UsedStrategy Type:⟨\\langletool\-based⟩\\rangleSelected Tools:⟨\\langlelist of tool names⟩\\rangleStrategy Reasoning:⟨\\langleProposer’s reasoning for tool selection⟩\\rangleGenerated ExplanationOutput:⟨\\langleActor’s structured output dict⟩\\rangleExplanation:⟨\\langlenatural language explanation⟩\\rangleConfidence:⟨\\langleActor’s self\-reported confidence⟩\\rangleFaithfulness EvaluationScore:⟨\\langlefaithfulness score⟩\\rangleMetric:⟨\\langlemetric name and formula⟩\\ranglePoriginalP\_\{\\text\{original\}\}:⟨\\langlevalue⟩\\ranglePmodifiedP\_\{\\text\{modified\}\}:⟨\\langlevalue⟩\\rangle⋮\\vdotsTask Instruction:Analyze the strategy and results, then providetwo separate JSON outputs:1\.PROPOSER\_REFLECTION: feedback on tool selection strategy\.2\.ACTOR\_REFLECTION: feedback on explanation generation\.Consider what went wrong, which tools were most or least helpful, and how both the strategy and explanation could be improved\.Figure 10:Complete prompt structure of the Critic Agent\.
### A\.14Computational Cost

Model architecture and parameters\.MeausesQwen3\.6\-35B\-A3B\[[40](https://arxiv.org/html/2610.02480#bib.bib37)\]as its backbone, a Mixture\-of\-Experts multi\-modal language model with35 billion total parametersand approximately3\.5 billion parameters activated per forward pass\(A3B\)\. During inference for evaluation, we serve the model via LoRA adapters with rank 32, adding<1%<1\\%additional parameters relative to the frozen base\. Baseline comparisons using the un\-finetuned backbone employedQwen3\.6\-35B\-A3B, while closed\-source frontier baselines \(Gemini\-3\.1\-flash, GPT\-5\.4, Claude\-Haiku\-4\.5\) were accessed via their respective public APIs\.

#### GRPO training\.

All reinforcement learning training was conducted throughTinker\[[58](https://arxiv.org/html/2610.02480#bib.bib43)\], a cloud\-based training service that manages GPU allocation and distributed rollout execution internally; per\-GPU\-hour accounting is therefore not directly available from our training logs\. The main hybrid training run ran for∼\\sim93 GRPO steps with a group size of 128\-256 rollouts per step processed per step, covering all three modalities jointly\. Each step produced a mean episode length of≈3\.6\\approx 3\.6turns and≈393\\approx 393action tokens\.

Post\-hoc explainers\.All post\-hoc explainers and faithfulness metric computation, and ablations were run on a NVIDIA A100 GPUs \(1 GPU per job, 16 CPU cores, 30 GB system RAM\)\. Jobs were parallelized across datasets and question types \(up to 4 concurrent jobs per modality\)\. The full test\-set evaluation for a single modality \(3\.5K instances across 4 dataset\-model combinations and 10 question types\) required approximately 8\-12 A100 GPU\-hours depending on modality complexity, with vision being the most expensive due to image preprocessing and SmoothGrad/GradCAM computation\. Summing across the three modalities, primary test evaluations consumed an estimated 30\-40 A100 GPU\-hours\. Ablation runs \(prompt robustness, critic agent, OOD datasets\) added approximately an additional 20\-30 A100 GPU\-hours\. Total estimated evaluation compute is therefore on the order of 50\-70 A100 GPU\-hours\.

### A\.15Prompt Templates for Proposer and Actor Agent

Refer to Fig\.[11](https://arxiv.org/html/2610.02480#A1.F11)for the Actor Agent’s prompt template and Fig\.[12](https://arxiv.org/html/2610.02480#A1.F12)for the Proposer Agent’s prompt template\.

Actor Agent Prompt TemplateSystem Role:You are an XAI expert\. Based on the provided analysis,⟨\\langleexecute task\-specific directive for Q1\-Q10⟩\\rangle\.User Input:Context & PredictionQuestion \(⟨\\langleQ1\-10⟩\\rangle\):⟨\\langlespecific question text⟩\\rangleModel Prediction: Class:⟨\\langlepredicted class⟩\\rangleConf:⟨\\langlescore⟩\\rangleInput Data⟨\\langleraw input text / tabular row / image data⟩\\rangleXAI AnalysisTool Results Summary:⟨\\langlestatus and most influential features per tool⟩\\rangleDetailed Statistics:•Integrated Gradients:⟨\\langletop positive/negative attributions⟩\\rangle•SHAP:⟨\\langletop positive/negative shap values⟩\\rangle•Sensitivity:⟨\\langlemost sensitive prob\_drops⟩\\rangleAutonomous Reasoning Results:GROUNDING:⟨\\langleextracted features matching query⟩\\rangleREASONING:⟨\\langlesemantic explanation for features⟩\\rangleCOMPARISON:⟨\\langlealignment with model prediction⟩\\rangleYour Task⟨\\langleSpecific directive corresponding to Question Q1\-10⟩\\rangleRequired Output Format \(JSON only\)\{
"output": \{
"⟨\\langlemodality\_key⟩\\rangle":⟨\\langleexpected data structure⟩\\rangle
\},
"explanation": "2\-3 sentences explaining your reasoning\."
\}Critical Requirements•For vision:bounding\_boxas⟨\\langle\[x\_min, y\_min, x\_max, y\_max\]⟩\\rangle•For text:spansas a list of⟨\\langle\{start\_index, end\_index\}⟩\\rangle•For tabular:feature\_keysas a list of column names•OUTPUT SIZE CONSTRAINT:⟨\\langlesize limit dictated by question type⟩\\rangle•Base your decision on the attribution analysis\.Final Instruction:Respond with ONLY JSON\.Figure 11:Prompt structure for the Actor Agent\. Task directives, constraints, and JSON schema keys are populated dynamically at runtime based on the specific question type \(Q1\-Q10\) and data modality\.Proposer Agent Prompt TemplateSystem Role:You are an AI explainability expert designing a strategy to answer a user’s question about a machine learning model’s prediction on⟨\\langleMODALITY⟩\\rangle\.User Input:Question & Data ContextQuestion:⟨\\langleuser question text⟩\\rangleInput Content:⟨\\langletruncated input text, image, or tabular row⟩\\rangleModel InformationModel:⟨\\langlemodel name⟩\\rangleArch:⟨\\langlearchitecture⟩\\ranglePrediction:⟨\\langleclass index & name⟩\\rangle\(Conf:⟨\\langlescore⟩\\rangle\)Top\-K Predictions:⟨\\langlelist of dicts containing top probabilities⟩\\rangleAvailable Methods1\. Autonomous Analysis:⟨\\langleinstructions on using internal reasoning to ground/extract features⟩\\rangle2\. External XAI Tools:•lime:⟨\\langleexecutes local surrogate explanation⟩\\rangle•integrated\_gradients:⟨\\langlecomputes path attribution⟩\\rangle•shap:⟨\\langlecomputes shapley values⟩\\rangle•sensitivity\_analysis:⟨\\langleleave\-one\-out ablation⟩\\rangle•guided\_backprop:⟨\\langlemodified ReLU gradients⟩\\rangle•smoothgrad:⟨\\langleaveraged noisy saliency⟩\\rangleYour Task⟨\\langleSpecific strategy design directive corresponding to Question Q1\-10⟩\\rangleRequired Output Format \(JSON only\)\{
"strategy\_type": "autonomous" \| "tools" \| "hybrid",
"reasoning": "Explain strategy choice \(2\-3 sentences\)",
"autonomous\_tasks": \[
\{
"task\_type": "grounding" \| "reasoning" \| "comparison",
"query": "Specific query for autonomous analysis",
"expected\_output": "What should be extracted"
\}
\],
"tool\_selection": \{
"selected\_tools":⟨\\langlesubset of available tools⟩\\rangle,
"reasoning": "Why these tools for this modality"
\}
\}Final Instruction:Provide your strategy as a JSON object:Figure 12:Prompt structure for the Proposer Agent\. Task directives are populated dynamically based on the active question type \(Q1\-Q10\), ensuring the strategy generated aligns with the specific objective\.