缺乏锚点的忠实性：关于可解释人工智能中特征重要性忠实性指标结构性错配的立场论文

# A Position Paper on the Structural Misalignments of Explainable AI Faithfulness Metrics for Feature Importance
Source: [https://arxiv.org/html/2610.12493](https://arxiv.org/html/2610.12493)
## Faithfulness Without Anchors: A Position Paper on the Structural Misalignments of Explainable AI Faithfulness Metrics for Feature ImportanceThanks:Presented at the IJCAI\-ECAI 2026 Workshop on Explainable Artificial Intelligence \(XAI\)

Emily SchillerIvan GentileDavid DembinskyAdriano LucieriKilian GöllerSteffen SeitzAffiliation:TU Delft, XITASO GmbH, University College Cork, IFAB Foundation, RPTU University Kaiserslautern\-Landau, DFKI, TU Dresden, SECAI

###### Abstract

Explainable Artificial Intelligence \(XAI\) tools have often been presented as methods to increase trust in machine learning models by making aspects of their algorithms more human\-understandable\. However, evidence accumulated over the last decade points to an overall failure of this objective, instead promoting overreliance on incorrect predictions, which can subsequently lead to algorithmic aversion\. Rather, XAI appears to function primarily as a debugging tool, useful for obtaining anecdotal insights into, for example, features learned by the model\. One of the main issues underlying this limitation is that XAI methods often oversimplify the inner workings of the model, resulting in misalignment between the explanation and the model itself—a behavior referred to as*unfaithfulness*or*infidelity*\.*Faithfulness*can be interpreted as the degree of approximation between an explanation and the model\. Computing faithfulness in a rigorous manner would thus make it possible to quantify one of the various elements of trustworthiness of an explanation\. However, existing metrics present significant limitations, making this quantification either infeasible or computationally expensive\. In this position paper, we outline five major shortcomings of current faithfulness metrics and propose four possible avenues for research to bolster their reliability\.

AIartificial intelligenceANNartificial neural networkAOPCarea over the perturbation curveAUPCarea under the perturbation curveECEexpected calibration errorFIfeature importanceLeRFleast relevant firstMoRFmost relevant firstOoDout\-of\-distributionROARremove\-and\-retrainUEuncertainty estimationXAIexplainable AI
## 1Introduction

With the increasing power of[AI](https://arxiv.org/html/2610.12493#id1)\([AI](https://arxiv.org/html/2610.12493#id1)\) models, their internal algorithmic processes have become more difficult to interpret, as exemplified in the notorious interpretability\-accuracy trade\-off[James et al\. \(2021\)](https://arxiv.org/html/2610.12493#bib.bib3)\. This opacity conflicts with transparency requirements such as the GDPR “Right to Explanation”[Goodman and Flaxman \(2017\)](https://arxiv.org/html/2610.12493#bib.bib4)\.[XAI](https://arxiv.org/html/2610.12493#id12)\([XAI](https://arxiv.org/html/2610.12493#id12)\) has emerged as a field to enable transparency of*black\-box*models by providing*explanations*that simplify*black\-box*models for human understanding\. Haufeet al\.[Haufe et al\. \(2026\)](https://arxiv.org/html/2610.12493#bib.bib2)indicate ideal goals of[XAI](https://arxiv.org/html/2610.12493#id12), including the recognition of model and data diagnostics \(i\.e\., discovering incorrect model behaviors, such as “Clever Hans effects”[Lapuschkin et al\. \(2019\)](https://arxiv.org/html/2610.12493#bib.bib5), or bias in datasets\), the discovery of unknown relationships between variables, and the identification of intervention targets for modifying said relationships\. However, evidence from roughly ten years of[XAI](https://arxiv.org/html/2610.12493#id12)research shows these tools often fall short on these promises, as recently discussed by Afrooghet al\.[Afroogh et al\. \(2026\)](https://arxiv.org/html/2610.12493#bib.bib1)\. Beyond highlighting misalignment in explanation recipients, they report that[XAI](https://arxiv.org/html/2610.12493#id12)can foster user overreliance on incorrect[AI](https://arxiv.org/html/2610.12493#id1)predictions that would otherwise be questioned\. In safety\-critical applications, this may cause serious consequences and potential algorithmic aversion\. Both Haufeet al\.[Haufe et al\. \(2026\)](https://arxiv.org/html/2610.12493#bib.bib2)and Afrooghet al\.[Afroogh et al\. \(2026\)](https://arxiv.org/html/2610.12493#bib.bib1)indicate*infidelity*\(or*unfaithfulness*\) of[XAI](https://arxiv.org/html/2610.12493#id12)outputs to the original model as a source of said downfalls\.

However, these works do not examine the technical issues of faithfulness metrics, nor explore faithfulness beyond[FI](https://arxiv.org/html/2610.12493#id6)\([FI](https://arxiv.org/html/2610.12493#id6)\)\. In their survey on the evaluation of[XAI](https://arxiv.org/html/2610.12493#id12), Dembinskyet al\.[Dembinsky et al\. \(2026\)](https://arxiv.org/html/2610.12493#bib.bib6)cite 19 faithfulness metrics, applicable to different types of[XAI](https://arxiv.org/html/2610.12493#id12):[FI](https://arxiv.org/html/2610.12493#id6), concepts, example explanations, white\-box surrogates, and natural language explanations\. Faithfulness for[FI](https://arxiv.org/html/2610.12493#id6)is by far the type with the most approaches, driven also by the fact that[FI](https://arxiv.org/html/2610.12493#id6)is currently the most common[XAI](https://arxiv.org/html/2610.12493#id12)task[Saarela and Podgorelec \(2024\)](https://arxiv.org/html/2610.12493#bib.bib50)\. Nevertheless, techniques for the computation of faithfulness, which mostly rely on \(iterative\) feature removal, are problematic at best: they are unreliable—e\.g\., due to issues with[OOD](https://arxiv.org/html/2610.12493#id9)\([OOD](https://arxiv.org/html/2610.12493#id9)\) data, require arbitrary baselines, or require re\-training phases which alter the nature of the original model\. Some other fidelity metrics do not even measure proper faithfulness, or conflate it with other[XAI](https://arxiv.org/html/2610.12493#id12)properties\. This unreliability raises the question whether[FI](https://arxiv.org/html/2610.12493#id6)can be truly considered*unfaithful*, or whether faithfulness metrics themselves are dubious\.

In this position paper, we analyze recent studies on faithfulness metric unreliability, arguing that modern[XAI](https://arxiv.org/html/2610.12493#id12)should prioritize more trustworthy approaches\. The field should move beyond established but unreliable metrics like pixel flipping[Bach et al\. \(2015\)](https://arxiv.org/html/2610.12493#bib.bib7)and[ROAR](https://arxiv.org/html/2610.12493#id10)\([ROAR](https://arxiv.org/html/2610.12493#id10)\)[Hooker et al\. \(2019\)](https://arxiv.org/html/2610.12493#bib.bib8)toward techniques considering\(i\)what is the real goal behind faithfulness,\(ii\)the model’s[UE](https://arxiv.org/html/2610.12493#id11)\([UE](https://arxiv.org/html/2610.12493#id11)\) capabilities, and\(iii\)expected properties of[FI](https://arxiv.org/html/2610.12493#id6)coefficients with respect to model predictions\.We conclude with research avenues for advancing faithfulness evaluation in[FI](https://arxiv.org/html/2610.12493#id6), restricting our claims to this domain given the substantial available research\.

## 2Background

### 2\.1Feature Importance

[FI](https://arxiv.org/html/2610.12493#id6)is one of the foundational types of[XAI](https://arxiv.org/html/2610.12493#id12)[Dembinsky et al\. \(2026\)](https://arxiv.org/html/2610.12493#bib.bib6)\. Let us consider a modelf:ℝd→𝒴f:\\mathbb\{R\}^\{d\}\\rightarrow\\mathcal\{Y\}, where𝒴\\mathcal\{Y\}is the output space\. We can define a[FI](https://arxiv.org/html/2610.12493#id6)technique as a mappinge:ℝd→ℝde:\\mathbb\{R\}^\{d\}\\rightarrow\\mathbb\{R\}^\{d\}whose output we term*explanation*\. The[FI](https://arxiv.org/html/2610.12493#id6)explanation provides a value \(which we can term importance*coefficient*or*score*\) for each feature111For simplicity, we consider only[FI](https://arxiv.org/html/2610.12493#id6)s that generate one importance value per raw feature\. This is not necessarily true, e\.g\., when considering feature interaction[Fumagalli et al\. \(2023\)](https://arxiv.org/html/2610.12493#bib.bib15), supersets of features[Ribeiro et al\. \(2016\)](https://arxiv.org/html/2610.12493#bib.bib44), or explanations in different domains than the input’s[Mishra et al\. \(2020\)](https://arxiv.org/html/2610.12493#bib.bib39)\., each value indicating the*importance*of the feature in the model’s predictive dynamics\.[FI](https://arxiv.org/html/2610.12493#id6)can be*global*\(thus, the importance coefficients are valid for the model as a whole\) or*local*\(i\.e\., the importance scores are valid only for one specific point of the input space\)\. Importance scores with a high magnitude indicate that the model relies greatly on the corresponding features for outputting the prediction; the sign instead indicates whether the feature contributes in increasing the model prediction \(if positive\) or in decreasing it \(if negative\)\. Some techniques might only present positive coefficients: for instance Permutation Feature Importance[Fisher et al\. \(2019\)](https://arxiv.org/html/2610.12493#bib.bib10)only outputs importance in term of magnitude, while other techniques, such as Grad\-CAM[Selvaraju et al\. \(2020\)](https://arxiv.org/html/2610.12493#bib.bib11)suppresses negative importance scores for simplifying the interpretation of the explanation\. A faithful[FI](https://arxiv.org/html/2610.12493#id6)should correctly indicate how important individual features are in the model’s predictive dynamics\. In the case of some[FI](https://arxiv.org/html/2610.12493#id6)techniques, such as SHAP[Lundberg and Lee \(2017\)](https://arxiv.org/html/2610.12493#bib.bib12), this is often defined in terms of*feature removal*: the deletion of an important feature \(leaving the others intact\) should cause a change in the model prediction, and that this change should be proportional to the importance coefficient\. However, other tools, such as gradient\-based[FI](https://arxiv.org/html/2610.12493#id6)for neural networks just report \(aggregations of\) gradients as scores, without referring to feature removal\.[FI](https://arxiv.org/html/2610.12493#id6)can also be called*feature attribution*; when working with image data, these explanations can also be termed*\(importance\) heatmaps*or*saliency maps*\.

### 2\.2Evaluation of Feature Importance

According to Nautaet al\.[Nauta et al\. \(2023\)](https://arxiv.org/html/2610.12493#bib.bib14), explanations can be evaluated from the perspectives of their content, their presentations, or their users\. The assessment of the content can be further grouped in different categories—alignment between model and explanation \(which the authors call*correctness*and*completeness*and that overall makes up the faithfulness[Dembinsky et al\. \(2026\)](https://arxiv.org/html/2610.12493#bib.bib6)\), continuity \(robustness\) of the explanation to perturbations \(to input, output, or model\), and compactness, among others\. The presentation point of view assesses the mean through which the explanation is presented, regardless of its content\. The user viewpoint describes instead the conditions \(context, previous knowledge, etc\.\) in which the user interacts with the explanation\. A subgroup of the user category worth mentioning is*coherence*, which other works also call*plausibility*: the alignment between an explanation and human prior knowledge\.

#### 2\.2\.1Faithfulness in Feature Importance

Faithfulness \(also called*fidelity*or*correctness*in the literature\) describes how truthful the explanation is to the original modelff[Nauta et al\. \(2023\)](https://arxiv.org/html/2610.12493#bib.bib14)\. For[FI](https://arxiv.org/html/2610.12493#id6), this requires alignment between features the model truly considers important and those the explanation identifies as important\. In their survey, Dembinskyet al\.[Dembinsky et al\. \(2026\)](https://arxiv.org/html/2610.12493#bib.bib6)identify input perturbation as the primary methodology for evaluating[FI](https://arxiv.org/html/2610.12493#id6)faithfulness\. The rationale follows from the feature removal considerations presented in[Section2\.1](https://arxiv.org/html/2610.12493#S2.SS1): perturbing \(i\.e\.,*removing*\) important features should substantially change model predictions, with magnitude proportional to feature importance\. The sign should be the opposite of the sign of the[FI](https://arxiv.org/html/2610.12493#id6)output\. The authors distinguish between*unguided*and*guided*perturbation approaches\. Unguided approaches operate a perturbation without considering the hierarchy dictated by the[FI](https://arxiv.org/html/2610.12493#id6)output\. Guided approaches, conversely, perturb a data point sequentially targeting features by importance, either most\-to\-least in[MORF](https://arxiv.org/html/2610.12493#id8)\([MORF](https://arxiv.org/html/2610.12493#id8)\) or least\-to\-most in[LERF](https://arxiv.org/html/2610.12493#id7)\([LERF](https://arxiv.org/html/2610.12493#id7)\)\. Nautaet al\.[Nauta et al\. \(2023\)](https://arxiv.org/html/2610.12493#bib.bib14)additionally distinguish between*incremental deletion*or*incremental addition*approaches: in the former, the features are sequentially perturbed starting from an unperturbed data point; in the latter the initial data point is completely perturbed, and the original features are*revealed*in[MORF](https://arxiv.org/html/2610.12493#id8)or[LERF](https://arxiv.org/html/2610.12493#id7)order\. Plotting model output versus perturbation iteration yields a*perturbation curve*\. In the case of classification, the model*confidence*\(probability value associated to the predicted class\) is often used as output in the plot\. We showcase a possible example for classification in[Figure1](https://arxiv.org/html/2610.12493#S2.F1)—[MORF](https://arxiv.org/html/2610.12493#id8)removal causes a sharp decrease in confidence at first, while removing less important features has a small effect on the output; with[LERF](https://arxiv.org/html/2610.12493#id7)removal the confidence drops at an increasing rate, proportional to the importance of features removed\. We point out that the behavior presented in[Figure1](https://arxiv.org/html/2610.12493#S2.F1)is ideal—often, for reasons explored in the next sections, the situation can be very noisy\. Other factors, such as*gradient starvation*[Pezeshki et al\. \(2021\)](https://arxiv.org/html/2610.12493#bib.bib41), can also modify the behavior of the curve: if multiple features are associated with a given prediction, models may concentrate originally only on a small subset of these\. After perturbation of important features, the models may shift the focus on the other features, while still outputting the same prediction, modifying the*ideal*scenario in[Figure1](https://arxiv.org/html/2610.12493#S2.F1)\. To quantify faithfulness for an\(f,e\)\(f,e\)pair, the technique is applied overnndata points and aggregated via:

1. i\)Correlation\-based metrics: compute the Pearson’s or Spearman’s correlation, or Kendall’sτ\\tau, between the importance of the perturbed features and the opposite of the change in model prediction\. A positive correlation should be observed\.
2. ii\)Area\-based metrics: plot the model prediction over the importance of perturbed features and measure faithfulness as the[AUPC](https://arxiv.org/html/2610.12493#id4)\([AUPC](https://arxiv.org/html/2610.12493#id4)\) or[AOPC](https://arxiv.org/html/2610.12493#id3)\([AOPC](https://arxiv.org/html/2610.12493#id3)\) \(or a combination of both[Knoll et al\. \(2025\)](https://arxiv.org/html/2610.12493#bib.bib43)\)\. The metric can be reported either absolute or normalized\. Depending on the approach adopted—incremental deletion versus addition,[MORF](https://arxiv.org/html/2610.12493#id8)versus[LERF](https://arxiv.org/html/2610.12493#id7)—a highly faithful explanation should lead to a normalized area value close to 1 or 0 \(if the metric is normalized\)\.

y^0\\hat\{y\}\_\{0\}y^1\\hat\{y\}\_\{1\}1100[MORF](https://arxiv.org/html/2610.12493#id8)[LERF](https://arxiv.org/html/2610.12493#id7)Proportion of features removedModel confidence

Figure 1:Example of possible perturbation curves obtained in a classification setting for an incremental deletion faithfulness evaluation\. The model starts by predicting a confidence valuey^0\\hat\{y\}\_\{0\}for the predicted class on the unperturbed data\. As more features are removed, the model confidence drops until reaching a minimumy^1\\hat\{y\}\_\{1\}when the data point is completely perturbed\. The curve depicted in blue shows an optimal[MORF](https://arxiv.org/html/2610.12493#id8)confidence trend, while the red curve is expected to be attained for a[LERF](https://arxiv.org/html/2610.12493#id7)perturbation\.

## 3Issues with existing faithfulness metrics

### 3\.1Faithfulness computation is almost exclusively operated on classification

Most of the literature on faithfulness computation is limited to classification settings \(e\.g\.,[Bhatt et al\. \(2021\)](https://arxiv.org/html/2610.12493#bib.bib37);[Zheng et al\. \(2024\)](https://arxiv.org/html/2610.12493#bib.bib22)\), while regression tasks remain insufficiently addressed\. This limitation is not merely a gap in coverage but reflects incompatibilities between metric design assumptions and regression\. Most metrics assume that removing \(or inserting\) important features should induce monotonic degradation \(or improvement\) in model confidence, specifically measured through classification scores[Samek et al\. \(2016\)](https://arxiv.org/html/2610.12493#bib.bib20)such as softmax probabilities or logits\. This assumption is well\-grounded in classification, where the relevant quantity, class probability, is naturally bounded between 0 and 1, and where degradation has a clear interpretation of decreased confidence\. In regression, the target quantity itself is ambiguous: should faithfulness measure changes in the predicted value or in deviation from the ground truth[Demetriou et al\. \(2025\)](https://arxiv.org/html/2610.12493#bib.bib38)? These probe fundamentally different qualities\. Because both quantities are typically unbounded, normalization becomes necessary but problematic: faithfulness scores become incomparable across datasets and even across instances within the same dataset, undermining their utility as evaluation metrics\. Additionally, perturbations may cause arbitrary shifts in predictions without clear monotonic relationships, making the “degradation” criterion ill\-defined\. Focusing on prediction error conflates model accuracy with explanation quality, measuring whether the model fails rather than whether the explanation faithfully represents the model’s reasoning\. Altogether, the literature reflects a deep bias of faithfulness metrics for[FI](https://arxiv.org/html/2610.12493#id6)toward classification tasks that cannot be trivially adapted to regression contexts\.

### 3\.2Some faithfulness metrics conflate fidelity with other properties of explanations

The very definition of faithfulness which we presented in[Section2\.2\.1](https://arxiv.org/html/2610.12493#S2.SS2.SSS1)is often misrepresented in some mainstream approaches for faithfulness computation presented in the literature\. We first note that sometimes the term*correctness*is confused for*plausibility*, as in the cases of Rony and Hassan[Rony and Hassan \(2025\)](https://arxiv.org/html/2610.12493#bib.bib17), and Maathuiset al\.[Maathuis et al\. \(2026\)](https://arxiv.org/html/2610.12493#bib.bib18)\.

On a more foundational level, we notice that, additionally, some metrics for faithfulness conflate faithfulness itself with the notion of plausibility\. Since faithfulness is aimed at measuring the alignment between the explanation and the model, its evaluation should not depend on external notions such as human\-generated labels or ground\-truth annotations\. In line with the observations on fidelity computation from Jacovi and Goldberg[Jacovi and Goldberg \(2020\)](https://arxiv.org/html/2610.12493#bib.bib9), we posit that faithfulness should be assessed only on the basis of the model internals\. Similarly, Dembinsky et al\.[Dembinsky et al\. \(2026\)](https://arxiv.org/html/2610.12493#bib.bib6)argue that ground\-truth dataset evaluation assesses faithfulness only when a uniquely defined rationale can be assumed beyond reasonable doubt; otherwise, it merely evaluates plausibility\. However, some highly\-adopted approaches for faithfulness computation target changes in performance metrics of the model instead of prediction values\. This is the case, e\.g\., for ROAR[Hooker et al\. \(2019\)](https://arxiv.org/html/2610.12493#bib.bib8)and ROAD[Rong et al\. \(2022\)](https://arxiv.org/html/2610.12493#bib.bib16)\. ROAR and ROAD measure faithfulness in terms of performance difference after perturbation \(and re\-training for the former\)\. Measuring faithfulness, e\.g\., in terms of accuracy drops is probing the alignment between explanation and ground truth, not between explanation and model\.

Additionally, since ROAR requires a re\-training of the model on perturbed input, we posit that this metric is not even assessing the quality of the explanations according to the original model, but according to a new model, which can be completely distinct in its internals from the original—an issue which Dembinskyet al\.[Dembinsky et al\. \(2026\)](https://arxiv.org/html/2610.12493#bib.bib6)call*different contextuality*\. For these reasons, we will avoid including ROAR in future discussions, even though it is often cited as being a state\-of\-the\-art faithfulness metric[Song et al\. \(2023\)](https://arxiv.org/html/2610.12493#bib.bib33);[Klein et al\. \(2024\)](https://arxiv.org/html/2610.12493#bib.bib31)\.

### 3\.3Perturbation as a stand\-in for feature removal requires arbitrary baselines

As stated in[Section2\.2\.1](https://arxiv.org/html/2610.12493#S2.SS2.SSS1), perturbations are involved in the faithfulness computation procedure as a stand\-in for feature removal, which is often impossible to reproduce with most machine learning models unless a retraining procedure is operated\. The main problem with this methodology is that there does not exist a single best solution for reproducing feature removal—the baseline for this procedure is thus arbitrary and it is thus impossible to define what perturbation would encode this*missingness of information*\. The problem is equivalent to the one faced by[FI](https://arxiv.org/html/2610.12493#id6)tools that work by feature removal, the most important one being SHAP\. Strumfelset al\.[Sturmfels et al\. \(2020\)](https://arxiv.org/html/2610.12493#bib.bib19)already ran an extensive study on the effect of the baseline selection \(constant baseline, blurring, random uniform and gaussian\) for image\-based[FI](https://arxiv.org/html/2610.12493#id6)generation, which led to the conclusion of there not being a “best” solution\. For what concerns faithfulness computation, different approaches are presented in the literature and largely depend on the data type: in image data, the most common approach is*fixed imputation*, where a patch of a uniform color \(e\.g\., white, gray, black, or the dataset mean\) is applied to the image to*occlude*important features[Samek et al\. \(2016\)](https://arxiv.org/html/2610.12493#bib.bib20);[Sundararajan et al\. \(2017\)](https://arxiv.org/html/2610.12493#bib.bib13); other approaches use blurring or noising[Zheng et al\. \(2024\)](https://arxiv.org/html/2610.12493#bib.bib22)or use generative models for image inpainting[Agarwal and Nguyen \(2020\)](https://arxiv.org/html/2610.12493#bib.bib21)\. For text data, hard token deletion can be applied[DeYoung et al\. \(2020\)](https://arxiv.org/html/2610.12493#bib.bib23), or tokens can be replaced with specific mask tokens[Atanasova et al\. \(2020\)](https://arxiv.org/html/2610.12493#bib.bib24), fixed embeddings \(e\.g\., zero\-embedding[Li et al\. \(2016\)](https://arxiv.org/html/2610.12493#bib.bib25)\) or with tokens generated by another language model[Kim et al\. \(2020\)](https://arxiv.org/html/2610.12493#bib.bib27);[Wu et al\. \(2021\)](https://arxiv.org/html/2610.12493#bib.bib26)\. In tabular data, common approaches include fixed imputation with a constant or marginal baseline or noising\.

### 3\.4Many perturbation\-based approaches cause[OOD](https://arxiv.org/html/2610.12493#id9)issues with models

Besides the difficulty with identifying what information missingness means for a specific model or type of data, the search for*good*perturbation baselines is also driven by another noticeable issue with[ANN](https://arxiv.org/html/2610.12493#id2): their overconfidence on[OOD](https://arxiv.org/html/2610.12493#id9)data\. Indeed, these models have been largely known to output random, overconfident predictions when presented with data which are too*distant*from their original training data distribution[Nguyen et al\. \(2015\)](https://arxiv.org/html/2610.12493#bib.bib28)\. All the perturbation strategies insofar presented are somewhat sensitive to the[OOD](https://arxiv.org/html/2610.12493#id9)issue[Hase et al\. \(2021\)](https://arxiv.org/html/2610.12493#bib.bib40), and some[Rong et al\. \(2022\)](https://arxiv.org/html/2610.12493#bib.bib16);[Zheng et al\. \(2024\)](https://arxiv.org/html/2610.12493#bib.bib22)have specifically been developed to tackle this problem\. Another connected issue is that classification[ANN](https://arxiv.org/html/2610.12493#id2), unless explicitly trained on data with missing information, or correctly calibrated on the uncertainty profile[Guo et al\. \(2017\)](https://arxiv.org/html/2610.12493#bib.bib29), these models do not possess the ability to say “*I do not know*” \(expressed as a flat output distribution\)\. The way the[OOD](https://arxiv.org/html/2610.12493#id9)issue reflects on the perturbation curve is the fact that the model response becomes erratic, with highly\-perturbed data often having a higher confidence than the previous iterations\. We display a real\-life example in[Figure4](https://arxiv.org/html/2610.12493#S3.F4)\. However, the faithfulness metrics introduced in[Section2\.2\.1](https://arxiv.org/html/2610.12493#S2.SS2.SSS1)all work under the assumption that the perturbation curve is monotonically decreasing, which clearly conflicts from the “clean” examples from[Figure1](https://arxiv.org/html/2610.12493#S2.F1)\.

The main approaches for combating the[OOD](https://arxiv.org/html/2610.12493#id9)issue in faithfulness are:\(i\)augmentative training[Hase et al\. \(2021\)](https://arxiv.org/html/2610.12493#bib.bib40)\(inlcuding the F\-fidelity framework[Zheng et al\. \(2024\)](https://arxiv.org/html/2610.12493#bib.bib22)\), where the training data is augmented by reproducing the perturbation patterns used in the faithfulness assessment, and\(ii\)generative inpainting[Agarwal and Nguyen \(2020\)](https://arxiv.org/html/2610.12493#bib.bib21), where the features are removed using an inpainting model that should still produce feasible data\.However, these two approaches do not solve the problem: augmentative training only works until a reasonable level of perturbation\. High levels of perturbation will likely cause a complete erasing of the original features, which, in the case of supervised learning, will potentially force the model to learn an incorrect relationship between data and response\. This potentially causes confident predictions even with a large portion of important features removed\. Generative inpainting causes absence of information in the image, which a non\-calibrated model does not know how to interpret—a behavior that leads to overconfident[OOD](https://arxiv.org/html/2610.12493#id9)predictions, thus impairing an approach which in principle should be valid\. Notice also that these approaches are mainly aimed at image data, and may not be easily translated to other data modalities\. Recent advances in[XAI](https://arxiv.org/html/2610.12493#id12)\-aware model calibration[Sridhar et al\. \(2026\)](https://arxiv.org/html/2610.12493#bib.bib34);[Cai et al\. \(2026\)](https://arxiv.org/html/2610.12493#bib.bib35)seem to go in the right direction: address the model calibration, without proposing other workarounds for[OOD](https://arxiv.org/html/2610.12493#id9)detection\.

### 3\.5The Formal Divergence of Correlation and Area\-Based Metrics from Explanatory Faithfulness

y^0\\hat\{y\}\_\{0\}y^1\\hat\{y\}\_\{1\}1100Proportion of features removedModel prediction

Figure 2:Example of perturbation curves whose Spearman’s correlation coefficient is always \-1\. The dotted line represents a case in which Pearson’s correlation coefficient is \-1\. In either case \([MORF](https://arxiv.org/html/2610.12493#id8)or[LERF](https://arxiv.org/html/2610.12493#id7)\), this would lead to a perfect faithfulness, although these configurations clearly represent different cases\. In the[MORF](https://arxiv.org/html/2610.12493#id8)case, the lower curves would represent an ideal scenario, whereby the most important features have a higher impact on the model prediction; the dotted line and the two upper curves represent instead an unsound scenario for faithfulness, with highly\-important feature having close to no effect on the prediciton, while features with a lower importance contribute more\. This shows the fault in correlation coefficients as a faithfulness metric\.y^0\\hat\{y\}\_\{0\}y^1\\hat\{y\}\_\{1\}1100Proportion of features removedModel prediction

Figure 3:Example perturbation curves in a[MORF](https://arxiv.org/html/2610.12493#id8), incremental deletion setting\. Despite the dashedbluecurve representing an desirable case of faithfulness,[AOPC](https://arxiv.org/html/2610.12493#id3)rewards theredcurve with a perfect score \(normalized AOPC of 1\)\. However, the latter is a degenerate situation, whereby only the feature with the highest importance affects the model’s prediction, while lower\-importance features have no effect\. On the other hand[AOPC](https://arxiv.org/html/2610.12493#id3)should reward the blue case, abiding to the aforementioned assumption of proportionality\.This point represents the fundamental criticism we raise to current[FI](https://arxiv.org/html/2610.12493#id6)faithfulness metrics—the existing correlation\-based and area\-based metrics do not really measure faithfulness correctly\. Miró\-Nicolauet al\.[Miró\-Nicolau et al\. \(2025\)](https://arxiv.org/html/2610.12493#bib.bib30)already showed, on simple experiments, that existing faithfulness metrics seem to be misaligned to the actual goal of measuring faithfulness\. In this section, we expand on their experiments by providing additional criticism on the current approaches\. For simplicity, we will work with an example of classification model, on which we evaluate explanations using guided perturbation fidelity obtained via[MORF](https://arxiv.org/html/2610.12493#id8), incremental deletion feature perturbation\. We additionally assume that the explanation only presents positive importance scores\. We expect the behavior of the curve to follow a trend similar to the one depicted in[Figure1](https://arxiv.org/html/2610.12493#S2.F1): there should be a proportionality between the importance of features removed and the drop in confidence observed\. We can observe that, in this case, neither correlation\-based, nor area\-based metrics are aligned with measuring faithfulness since they break this assumption:

- ▶\\blacktrianglerightPearson’s correlation assumes a linear relationship between the two axes\. Thus, a perfectly faithful explanation \(see[Figure2](https://arxiv.org/html/2610.12493#S3.F2)\) would require a uniform drop in confidence for each iteration\. This would mean that the contribution of each feature is the same, regardless of its importance\.
- ▶\\blacktrianglerightNonlinear correlation metrics \(e\.g\., Spearman’sρ\\rho\) are just measuring the monotonicity of the trend, they do not consider its convexity\. Thus, the proportionality assumption could be violated, since any monotonic trend would lead to a maximum faithfulness score, as depicted in[Figure2](https://arxiv.org/html/2610.12493#S3.F2)\. High nonlinear correlation is hence more of a*necessary*property of a faithful explanation than a*sufficient*one\.
- ▶\\blacktriangleright[AOPC](https://arxiv.org/html/2610.12493#id3)would require a perfectly faithful explanation to follow the trend highlighted in[Figure3](https://arxiv.org/html/2610.12493#S3.F3)\. This conflicts with the expected trend from[Figure1](https://arxiv.org/html/2610.12493#S2.F1)and violates the proportionality assumption: only the most important feature\(s\) would lead to a drop in confidence, and all the relevant features with lower importance would not have any effect on the model prediction\. The only situation in which[AOPC](https://arxiv.org/html/2610.12493#id3)would be a valid faithfulness metric is when the explanation is binarized, thus possibly leading to the behavior in[Figure3](https://arxiv.org/html/2610.12493#S3.F3)\.

In addition, there is one more point to be made: the assumption of confidence drop in classification can only be reliably met when the faithfulness metrics are run on the predicted class\. In case the explanation is run on a non\-predicted class, it is likely that the confidence would be very low, possibly close to 0\. In the ideal case of a perfectly calibrated model, removing important features would possibly cause an*increase*in confidence prediction \(towards1/K1/K, withKKbeing the number of classes\)\. In the uncalibrated case, this confidence increase could be even higher, due to the existence of a so called*catch\-all category*[Miró\-Nicolau et al\. \(2025\)](https://arxiv.org/html/2610.12493#bib.bib30)that the model designates as a default class for[OOD](https://arxiv.org/html/2610.12493#id9)examples\. This makes the current approaches infeasible for measuring the faithfulness of non\-predicted classes\.[Figure4](https://arxiv.org/html/2610.12493#S3.F4)shows a real\-life example of perturbation curves in a model with poor[OOD](https://arxiv.org/html/2610.12493#id9)capabilities, where one class is designated as a high\-confident*catch\-all*\.

\(a\)\(b\)\(c\)
Figure 4:Example of perturbation curves obtained on a ResNet\-18[ANN](https://arxiv.org/html/2610.12493#id2)[He et al\. \(2016\)](https://arxiv.org/html/2610.12493#bib.bib49)trained on the HAM10000 dataset[Codella et al\. \(2019\)](https://arxiv.org/html/2610.12493#bib.bib48)\. The dataset contains images from seven classes of skin cancer \(akiec=Actinic keratoses,bcc=basal cell carcinoma,bkl=benign keratosis\-like lesions,df=dermatofibroma,mel=melanoma,nv=melanocytic nevi,vasc=vascular lesions\)\. The plots are obtained by incrementally deleting features in[MORF](https://arxiv.org/html/2610.12493#id8)order with black patches\. In all cases, the model designs thenvclass as a*catch\-all category*\. \(a\) and \(b\) show images from ground truth classes different thannv, while \(c\) shows an example from thenvclass, where the model confidence remains extremely high when the image is completely black\. This demonstrates that faithfulness computation is flawed for models with poor[OOD](https://arxiv.org/html/2610.12493#id9)capabilities\. The1/K1/Kline represents the optimal case for a well\-calibrated model, which should output a flat distribution for completely black images\. None of the three figures showcase anything close to this behavior\.

## 4Our proposal for the direction of[FI](https://arxiv.org/html/2610.12493#id6)faithfulness research

Having outlined key issues with current faithfulness computation, we propose research directions towards more solid and trustworthy faithfulness evaluation\. This aims to restore confidence in[XAI](https://arxiv.org/html/2610.12493#id12)methods, which, per[Section1](https://arxiv.org/html/2610.12493#S1), has declined due to unfulfilled promises regarding model interpretability\.

Proposal I – faithfulness should be evaluated on well\-calibrated models robust to[OOD](https://arxiv.org/html/2610.12493#id9)data:faithfulness relies on the assumption that models should reduce the confidence in their predictions as the data gets progressively perturbed, as explored in[Section2\.2\.1](https://arxiv.org/html/2610.12493#S2.SS2.SSS1)\. This collides with the fact that most[ANN](https://arxiv.org/html/2610.12493#id2)are largely overconfident in their predictions, especially for what concerns[OOD](https://arxiv.org/html/2610.12493#id9)data, as seen in[Section3\.4](https://arxiv.org/html/2610.12493#S3.SS4)\. We hence posit that faithfulness should be computed on well\-calibrated,[OOD](https://arxiv.org/html/2610.12493#id9)\-robust models\. In that regard, we believe the recent research on[XAI](https://arxiv.org/html/2610.12493#id12)\-aware model calibration[Sridhar et al\. \(2026\)](https://arxiv.org/html/2610.12493#bib.bib34);[Cai et al\. \(2026\)](https://arxiv.org/html/2610.12493#bib.bib35)to be a valid direction, favoring a more reliable faithfulness computation\. Another underexplored line of research that needs to be expanded upon is[XAI](https://arxiv.org/html/2610.12493#id12)for Bayesian neural networks[Valdenegro\-Toro and Mulye \(2024\)](https://arxiv.org/html/2610.12493#bib.bib42): these models tend to display a more natural robustness to[OOD](https://arxiv.org/html/2610.12493#id9)data\. In addition to these benefits, calibration has other positive effects on the model trustworthiness: a model whose uncertainty estimates are well calibrated enhances human oversight, allowing stakeholders to immediately get an indication on the possible \(in\)correctness of a prediction—outputs with low confidence are more likely to be inaccurate than highly\-confident ones[Guo et al\. \(2017\)](https://arxiv.org/html/2610.12493#bib.bib29)\.

Proposal II – faithfulness metrics for classification needs to consider the cases of non\-predicted classes:as outlined in[Section3\.5](https://arxiv.org/html/2610.12493#S3.SS5), faithfulness currently lacks an evaluation approach for explanations generated on non\-predicted classes\. The generation of explanations on classes other than the predicted ones could be necessary in some cases, e\.g\., for contrastive explanations[Sarti et al\. \(2024\)](https://arxiv.org/html/2610.12493#bib.bib46);[Zhang et al\. \(2025\)](https://arxiv.org/html/2610.12493#bib.bib47)\. Thus, we posit that approaches for faithfulness computation for classification tasks should generalize to all classes, not only the predicted one\. Until such approaches are formulated, we posit that practitioners should concentrate only on generating explanations on the predicted category\.

Proposal III – faithfulness for regression needs more research efforts:as discussed in[Section3\.1](https://arxiv.org/html/2610.12493#S3.SS1), current faithfulness metrics are predominantly tailored to classification tasks and cannot be trivially adapted to regression settings without introducing conceptual problems\. Unlike classification, regression requires metrics that assess whether[FI](https://arxiv.org/html/2610.12493#id6)explanations correctly predict both the direction and magnitude of changes in predicted values\. We argue that addressing this gap requires dedicated research efforts that explicitly account for the unique characteristics of regression tasks\. Specifically, the community should develop formalizations of faithfulness metrics designed specifically for regression, rather than adapting classification\-based approaches that rely on incompatible assumptions\. New metrics such as TIMING[Jang et al\. \(2025\)](https://arxiv.org/html/2610.12493#bib.bib45)represent a solid avenue, although the reliance on area\-based metrics may still be problematic, due to the issues raised in the present paper\. We additionally highlight that this proposal may pose issues in conjunction with Proposal I, since calibration for regression models less well\-established than for classification[Levi et al\. \(2022\)](https://arxiv.org/html/2610.12493#bib.bib52)\.

Proposal IV – faithfulness metrics should be more aligned to their goal:with regards to what has already been outlined in[Section3\.5](https://arxiv.org/html/2610.12493#S3.SS5), we believe the current approaches based on correlation and area\-based metrics \(and related work\-arounds\) should be scraped, favoring instead approaches that consider what are the*expectations*from a faithful explanation, and that reliable metrics be developed abiding to these expectations\. We consider the following points:

1. i\.The x\-axis of the perturbation curve should represent the importance of the features removed, while many approaches use the iteration number instead\. The latter does not connect to a direct value of[FI](https://arxiv.org/html/2610.12493#id6), which does not allow to get a sense of the prediction change proportional to said importance\.
2. ii\.If an optimal shape of the prediction curve is established \(e\.g\., similar to the one in[Figure1](https://arxiv.org/html/2610.12493#S2.F1)\), then an inspiration for the faithfulness computation could be coming from the field of uncertainty estimation\. Specifically, the[ECE](https://arxiv.org/html/2610.12493#id5)\([ECE](https://arxiv.org/html/2610.12493#id5)\) already offers an example of a metric considering a difference with an expected curve shape—see[Figure5](https://arxiv.org/html/2610.12493#S4.F5)\. We stress out that currently there does not exist an agreed\-upon definition of*optimality*for a perturbation curve, and we posit more research should be devoted to eliciting this\. The choice of perturbation and phenomena such as gradient starvation \(see[Section3\.5](https://arxiv.org/html/2610.12493#S3.SS5)\) may pose a big hurdle in determining an optimal shape of the curve\.
3. iii\.Since faithfulness metrics should foster model trustworthiness, these should necessarily be relative in order to allow for direct interpretation of the faithfulness value\. Absolute metrics calculated, e\.g\., on the output logits, such as in the case of IROF[Rieger and Hansen \(2020\)](https://arxiv.org/html/2610.12493#bib.bib36)do not offer a relative value \(e\.g\., between 0 and 1\) and logits can easily differ between models, making comparisons impossible and faithfulness scores uninterpretable\.

000\.20\.20\.40\.40\.60\.60\.80\.811000\.20\.20\.40\.40\.60\.60\.80\.811OverconfidentUnderconfidentPredicted confidenceObserved accuracyPerfect calibrationModel \(overconfident\)

Figure 5:Example of calibration plot for a toy classification model\. The model is evaluated on a validation dataset\. The data are then split into bins depending on the confidence and the average accuracy per bin is computed and plotted \(solid black line\)\. The[ECE](https://arxiv.org/html/2610.12493#id5)is calculated as the magnitude of differences \(dotted blue lines\) between these points and the diagonal \(gray dashed line\), which represents perfect calibration where confidence equals accuracy\.
## 5Contextualizing Faithfulness Assessment

Issues with faithfulness metrics are well\-known in the[XAI](https://arxiv.org/html/2610.12493#id12)literature\. Ever since the still\-used pixel/feature flipping experiment[Bach et al\. \(2015\)](https://arxiv.org/html/2610.12493#bib.bib7), subsequent methods have tried to address its shortcomings, mainly on[OOD](https://arxiv.org/html/2610.12493#id9)data created in the perturbation process: we have cited the examples of ROAR[Hooker et al\. \(2019\)](https://arxiv.org/html/2610.12493#bib.bib8)—which proposes a retraining process—and the derivative ROAD[Rong et al\. \(2022\)](https://arxiv.org/html/2610.12493#bib.bib16)—which instead proposes blurring to avoid the expensive retraining\. More recently, works like F\-fidelity[Zheng et al\. \(2024\)](https://arxiv.org/html/2610.12493#bib.bib22)have proposed the usage of augmentative finetuning to curb the[OOD](https://arxiv.org/html/2610.12493#id9)issues\. However, there are papers which have targeted a more comprehensive critical approach to faithfulness evaluation: the previously cited work by Miró\-Nicolauet al\.[Miró\-Nicolau et al\. \(2025\)](https://arxiv.org/html/2610.12493#bib.bib30)indicates how the main approaches seem to be misaligned with the goal of measuring faithfulness in models trained on simple tasks, where the important features are known in advance\. The authors frame their findings on a critical level, not listing concrete proposals for future directions\. Additionally, Chanet al\.[Chan et al\. \(2022\)](https://arxiv.org/html/2610.12493#bib.bib32), in their comparative study, notice inconsistencies between different faithfulness metrics on natural language processing tasks\. They argue in favor of correlation\-based metrics; however, they miss the issues behind these approaches which we instead outline in[Section3\.5](https://arxiv.org/html/2610.12493#S3.SS5)\. Seth and Sankarapu[Seth and Sankarapu \(2025\)](https://arxiv.org/html/2610.12493#bib.bib51)highlight the importance of a sound evaluation of[XAI](https://arxiv.org/html/2610.12493#id12)and criticize the current field because of\(i\)fragmentation—a lack of standard in metrics—and\(ii\)manipulability—metrics are not robust and can be manipulated easily\.However, their work is designed at a governance level and does not address the technical details behind these shortcomings\. Finally, the previously mentioned Haufeet al\.[Haufe et al\. \(2026\)](https://arxiv.org/html/2610.12493#bib.bib2)also address[XAI](https://arxiv.org/html/2610.12493#id12)shortcomings\. Their perspective encompasses several points, including the lack of formalization in[XAI](https://arxiv.org/html/2610.12493#id12), the fragility of the current metrics for[XAI](https://arxiv.org/html/2610.12493#id12)evaluation, the lack of verification that explanations abide by the formal requirements, and the scarcity of real\-world data for performing empirical evaluations\. The authors also mention that current[XAI](https://arxiv.org/html/2610.12493#id12)tools and faithfulness metrics do not take into consideration the data generating distribution; however, they do not propose how to integrate this into existing methods\. Our proposals go in the direction highlighted by the authors; specifically, we also call for higher formality in[XAI](https://arxiv.org/html/2610.12493#id12)evaluation, e\.g\., by soliciting research to define a desired form of perturbation curve \(Proposal IV\) and by posing more stringent requirements on the model calibration \(Proposal I\)\.

## 6Discussion and conclusion

In the present paper, we have summarized several issues connected with the current metrics for the computation of faithfulness of[FI](https://arxiv.org/html/2610.12493#id6)\. Faithfulness is defined as the level of alignment between the[FI](https://arxiv.org/html/2610.12493#id6)explanation and the internals of the model on which said explanation is generated\. We highlighted how the main metrics approach the computation from the process of*data intervention*[Dembinsky et al\. \(2026\)](https://arxiv.org/html/2610.12493#bib.bib6): perturbing specific features of a given input should yield variations in the model output which are proportional to the importance of the features\. This is often done in an iterative fashion, by incrementally*removing*\(or*revealing*\) features by means of perturbations operated in a[MORF](https://arxiv.org/html/2610.12493#id8)or[LERF](https://arxiv.org/html/2610.12493#id7)fashion\. The output of the model is tracked for each perturbed input and then plotted against the fraction of features removed, as shown in[Figure1](https://arxiv.org/html/2610.12493#S2.F1)\. Faithfulness is then summarized with a scalar by either computing the correlation of the perturbation curve or its area under \(or over\) the curve\.

We posit that these approaches present several flaws\. Some of these criticisms are novel, while some others are already well known in the literature\. We present five main shortcomings:\(1\)disproportionate focus on classification for faithfulness research,\(2\)conflation of faithfulness with other[XAI](https://arxiv.org/html/2610.12493#id12)evaluation facets,\(3\)necessity of an arbitrary baseline for simulating feature removal,\(4\)emergence of[OOD](https://arxiv.org/html/2610.12493#id9)data \(which most[ANN](https://arxiv.org/html/2610.12493#id2)handle poorly\) as effect of the perturbation, and\(5\)misalignment between the main metrics and the proper computation of faithfulness\.These drawbacks severely limit the assessment of faithfulness—a metric whose importance is vital in[XAI](https://arxiv.org/html/2610.12493#id12)evaluation, since it essentially quantifies the*level of approximation*of an explanation with regard to the model, and is hence fundamental for fostering reliability of[FI](https://arxiv.org/html/2610.12493#id6)as the main[XAI](https://arxiv.org/html/2610.12493#id12)task\. Increasing the trustworthiness of[XAI](https://arxiv.org/html/2610.12493#id12)is hence necessary, in a time in which, after roughly a decade of research, the field is being severely criticized for its shortcomings in fulfilling its initial commitments in improving model transparency for its users[Afroogh et al\. \(2026\)](https://arxiv.org/html/2610.12493#bib.bib1)\. We conclude our paper by proposing four avenues for faithfulness research, namely:\(I\)contemplate model calibration when[FI](https://arxiv.org/html/2610.12493#id6)needs formal evaluation,\(II\)for classification, consider that explanations can be generated either on predicted or non\-predicted classes \(while current methods concentrate solely on predicted classes\),\(III\)direct more effort to faithfulness for regression, and\(IV\)align metrics to the actual goal of computing faithfulness\.We hope our manuscript can provide a clear and comprehensive summary of the current problems with the computation of faithfulness, at the same time giving directions to practitioners and researchers on how the research in the field can be improved\. We close our paper by shifting back to[Section1](https://arxiv.org/html/2610.12493#S1)—we posit Afrooghet al\.’s[Afroogh et al\. \(2026\)](https://arxiv.org/html/2610.12493#bib.bib1)point on infidelity to be inaccurate: it is not that[XAI](https://arxiv.org/html/2610.12493#id12)is unfaithful, but we are currently lacking a principled way for assessing faithfulness\. Even though a more formal approach to faithfulness will not solve all of the shortcomings of[XAI](https://arxiv.org/html/2610.12493#id12), as outlined by Haufeet al\.[Haufe et al\. \(2026\)](https://arxiv.org/html/2610.12493#bib.bib2), this will still help with bolstering its credibility\.

Limitations: despite providing a comprehensive critical outline of the current metrics for[FI](https://arxiv.org/html/2610.12493#id6)faithfulness, our paper does not consider other popular[XAI](https://arxiv.org/html/2610.12493#id12)tasks, such as counterfactual examples or concepts, that all require faithfulness computation[Dembinsky et al\. \(2026\)](https://arxiv.org/html/2610.12493#bib.bib6)\. Concepts, for instance, present similar challenges to[FI](https://arxiv.org/html/2610.12493#id6)in this regard\. Additionally, if we exclude the example from[Figure4](https://arxiv.org/html/2610.12493#S3.F4), the present paper lacks experiments that summarize the issues we highlight in[Section3](https://arxiv.org/html/2610.12493#S3), although many of these problems are already well\-known in the literature\. Finally, some of our directions from[Section4](https://arxiv.org/html/2610.12493#S4)\(e\.g\., Proposals II and III\) act more as recommendations for research and do not contain any concrete idea on how the limitations should be tackled\. Similarly, the limitation on the arbitrary nature of perturbation baselines from[Section3\.3](https://arxiv.org/html/2610.12493#S3.SS3)is not tackled by our proposals and remains currently unresolved\. Notwithstanding these limitations, we believe our position paper still effectively summarizes shortcomings of current approaches and provides valuable insights for directing future research on faithfulness\.

### Acknowledgment

This work is partly supported by BMFTR \(Federal Ministry of Research, Technology and Space\) in DAAD project 57616814 \(SECAI, School of Embedded Composite AI, https://secai\.org/\)\.

## References

- Afrooghet al\.\(2026\)S\. Afroogh, S\. I\. Ahmed, P\. Ahrweiler, D\. Alvarez\-Melis, M\. M\. Arief, E\. Barakova, F\. J\. Bargagli\-Stoffi, E\. Biyik, H\. Chen, X\. Chen,et al\.Beyond Explainable AI \(XAI\): An Overdue Paradigm Shift and Post\-XAI Research Directions\.arXiv preprint arXiv:2602\.24176\.Cited by:[§1](https://arxiv.org/html/2610.12493#S1.p1.1),[§6](https://arxiv.org/html/2610.12493#S6.p2.1)\.
- Agarwal and Nguyen \(2020\)C\. Agarwal and A\. NguyenExplaining image classifiers by removing input features using generative models\.InProceedings of the Asian Conference on Computer Vision,Cited by:[item \(ii\)](https://arxiv.org/html/2610.12493#S3.I1.i2.2),[§3\.3](https://arxiv.org/html/2610.12493#S3.SS3.p1.1)\.
- Atanasovaet al\.\(2020\)P\. Atanasova, J\. G\. Simonsen, C\. Lioma, and I\. AugensteinA diagnostic study of explainability techniques for text classification\.InProceedings of the 2020 conference on empirical methods in natural language processing \(EMNLP\),pp\. 3256–3274\.Cited by:[§3\.3](https://arxiv.org/html/2610.12493#S3.SS3.p1.1)\.
- Bachet al\.\(2015\)S\. Bach, A\. Binder, G\. Montavon, F\. Klauschen, K\. Müller, and W\. SamekOn pixel\-wise explanations for non\-linear classifier decisions by layer\-wise relevance propagation\.PloS one10\(7\),pp\. e0130140\.Cited by:[§1](https://arxiv.org/html/2610.12493#S1.p3.1),[§5](https://arxiv.org/html/2610.12493#S5.p1.1)\.
- Bhattet al\.\(2021\)U\. Bhatt, A\. Weller, and J\. M\. F\. MouraEvaluating and aggregating feature\-based model explanations\.InProceedings of the Twenty\-Ninth International Joint Conference on Artificial Intelligence,IJCAI’20\.External Links:ISBN 9780999241165Cited by:[§3\.1](https://arxiv.org/html/2610.12493#S3.SS1.p1.1)\.
- Caiet al\.\(2026\)Y\. Cai, T\. Ardoin, M\. Gulati, and G\. WunderRethinking Explanation Evaluation Under the Retraining Scheme\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.40,pp\. 19826–19834\.Cited by:[§3\.4](https://arxiv.org/html/2610.12493#S3.SS4.p2.1),[§4](https://arxiv.org/html/2610.12493#S4.p2.1)\.
- Chanet al\.\(2022\)C\. S\. Chan, H\. Kong, and L\. GuanqingA comparative study of faithfulness metrics for model interpretability methods\.InProceedings of the 60th annual meeting of the association for computational linguistics \(volume 1: Long papers\),pp\. 5029–5038\.Cited by:[§5](https://arxiv.org/html/2610.12493#S5.p1.1)\.
- Codellaet al\.\(2019\)N\. Codella, V\. Rotemberg, P\. Tschandl, M\. E\. Celebi, S\. Dusza, D\. Gutman, B\. Helba, A\. Kalloo, K\. Liopyris, and M\. MarchettiSkin lesion analysis toward melanoma detection 2018: A challenge hosted by the international skin imaging collaboration \(ISIC\)\.arXiv preprint arXiv:1902\.03368\.Cited by:[Figure 4](https://arxiv.org/html/2610.12493#S3.F4),[Figure 4](https://arxiv.org/html/2610.12493#S3.F4.15)\.
- Dembinskyet al\.\(2026\)D\. Dembinsky, A\. Lucieri, S\. Frolov, H\. Najjar, K\. Watanabe, and A\. DengelUnifying VXAI: A Systematic Review and Framework for the Evaluation of Explainable AI\.Transactions on Machine Learning Research\.Cited by:[§1](https://arxiv.org/html/2610.12493#S1.p2.1),[§2\.1](https://arxiv.org/html/2610.12493#S2.SS1.p1.1),[§2\.2\.1](https://arxiv.org/html/2610.12493#S2.SS2.SSS1.p1.1),[§2\.2](https://arxiv.org/html/2610.12493#S2.SS2.p1.1),[§3\.2](https://arxiv.org/html/2610.12493#S3.SS2.p2.1),[§3\.2](https://arxiv.org/html/2610.12493#S3.SS2.p3.1),[§6](https://arxiv.org/html/2610.12493#S6.p1.1),[§6](https://arxiv.org/html/2610.12493#S6.p3.1)\.
- Demetriouet al\.\(2025\)X\. Demetriou, S\. Sananikone, V\. T\. Westmoreland, M\. Sabatelli, and M\. ZullichA Study on the Faithfulness of Feature Attribution Explanations in Pruned Vision\-Based Multi\-Task Learning\.InJoint of the xAI 2025 Late\-Breaking Work, Demos and Doctoral Consortium, LB/D/DC@ xAI 2025,pp\. 121–128\.Cited by:[§3\.1](https://arxiv.org/html/2610.12493#S3.SS1.p1.1)\.
- DeYounget al\.\(2020\)J\. DeYoung, S\. Jain, N\. F\. Rajani, E\. Lehman, C\. Xiong, R\. Socher, and B\. C\. WallaceERASER: A benchmark to evaluate rationalized NLP models\.InProceedings of the 58th annual meeting of the association for computational linguistics,pp\. 4443–4458\.Cited by:[§3\.3](https://arxiv.org/html/2610.12493#S3.SS3.p1.1)\.
- Fisheret al\.\(2019\)A\. Fisher, C\. Rudin, and F\. DominiciAll models are wrong, but many are useful: Learning a variable’s importance by studying an entire class of prediction models simultaneously\.Journal of Machine Learning Research20\(177\),pp\. 1–81\.Cited by:[§2\.1](https://arxiv.org/html/2610.12493#S2.SS1.p1.1)\.
- Fumagalliet al\.\(2023\)F\. Fumagalli, M\. Muschalik, P\. Kolpaczki, E\. Hüllermeier, and B\. HammerSHAP\-IQ: Unified approximation of any\-order shapley interactions\.Advances in Neural Information Processing Systems36,pp\. 11515–11551\.Cited by:[footnote 1](https://arxiv.org/html/2610.12493#footnote1)\.
- Goodman and Flaxman \(2017\)B\. Goodman and S\. FlaxmanEuropean Union regulations on algorithmic decision\-making and a “right to explanation”\.AI magazine38\(3\),pp\. 50–57\.Cited by:[§1](https://arxiv.org/html/2610.12493#S1.p1.1)\.
- Guoet al\.\(2017\)C\. Guo, G\. Pleiss, Y\. Sun, and K\. Q\. WeinbergerOn calibration of modern neural networks\.InInternational conference on machine learning,pp\. 1321–1330\.Cited by:[§3\.4](https://arxiv.org/html/2610.12493#S3.SS4.p1.1),[§4](https://arxiv.org/html/2610.12493#S4.p2.1)\.
- Haseet al\.\(2021\)P\. Hase, H\. Xie, and M\. BansalThe out\-of\-distribution problem in explainability and search methods for feature importance explanations\.Advances in neural information processing systems34,pp\. 3650–3666\.Cited by:[item \(i\)](https://arxiv.org/html/2610.12493#S3.I1.i1.2),[§3\.4](https://arxiv.org/html/2610.12493#S3.SS4.p1.1)\.
- Haufeet al\.\(2026\)S\. Haufe, R\. Wilming, B\. Clark, R\. Zhumagambetov, A\. Boubekki, J\. Martin, and D\. PankninExplainable AI needs formalization\.npj Artificial Intelligence2\(1\),pp\. 42\.Cited by:[§1](https://arxiv.org/html/2610.12493#S1.p1.1),[§5](https://arxiv.org/html/2610.12493#S5.p1.1),[§6](https://arxiv.org/html/2610.12493#S6.p2.1)\.
- Heet al\.\(2016\)K\. He, X\. Zhang, S\. Ren, and J\. SunDeep residual learning for image recognition\.InProceedings of the IEEE conference on computer vision and pattern recognition,pp\. 770–778\.Cited by:[Figure 4](https://arxiv.org/html/2610.12493#S3.F4),[Figure 4](https://arxiv.org/html/2610.12493#S3.F4.15)\.
- Hookeret al\.\(2019\)S\. Hooker, D\. Erhan, P\. Kindermans, and B\. KimA benchmark for interpretability methods in deep neural networks\.Advances in neural information processing systems32\.Cited by:[§1](https://arxiv.org/html/2610.12493#S1.p3.1),[§3\.2](https://arxiv.org/html/2610.12493#S3.SS2.p2.1),[§5](https://arxiv.org/html/2610.12493#S5.p1.1)\.
- Jacovi and Goldberg \(2020\)A\. Jacovi and Y\. GoldbergTowards Faithfully Interpretable NLP Systems: How Should We Define and Evaluate Faithfulness?\.InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics,pp\. 4198–4205\.Cited by:[§3\.2](https://arxiv.org/html/2610.12493#S3.SS2.p2.1)\.
- Jameset al\.\(2021\)G\. James, D\. Witten, T\. Hastie, and R\. TibshiraniAn Introduction to Statistical Learning: with Applications in R\.Springer Texts in Statistics\.External Links:ISBN 978\-1\-07\-161417\-4 978\-1\-07\-161418\-1,[Document](https://dx.doi.org/10.1007/978-1-0716-1418-1)Cited by:[§1](https://arxiv.org/html/2610.12493#S1.p1.1)\.
- Janget al\.\(2025\)H\. Jang, C\. Kim, and E\. YangTIMING: Temporality\-Aware Integrated Gradients for Time Series Explanation\.InForty\-second International Conference on Machine Learning,Cited by:[§4](https://arxiv.org/html/2610.12493#S4.p4.1)\.
- Kimet al\.\(2020\)S\. Kim, J\. Yi, E\. Kim, and S\. YoonInterpretation of NLP models through input marginalization\.InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),pp\. 3154–3167\.Cited by:[§3\.3](https://arxiv.org/html/2610.12493#S3.SS3.p1.1)\.
- Kleinet al\.\(2024\)L\. Klein, C\. Lüth, U\. Schlegel, T\. Bungert, M\. El\-Assady, and P\. JägerNavigating the maze of explainable ai: A systematic approach to evaluating methods and metrics\.Advances in Neural Information Processing Systems37,pp\. 67106–67146\.Cited by:[§3\.2](https://arxiv.org/html/2610.12493#S3.SS2.p3.1)\.
- Knollet al\.\(2025\)C\. Knoll, J\. Ullrich, T\. Manjooran, K\. Göller, H\. Amjad, R\. Tetzlaff, and S\. SeitzXAIEV–a framework for the evaluation of xai\-algorithms for image classification\.InWorld Conference on Explainable Artificial Intelligence,pp\. 250–263\.Cited by:[item ii\)](https://arxiv.org/html/2610.12493#S2.I1.i2.p1.1)\.
- Lapuschkinet al\.\(2019\)S\. Lapuschkin, S\. Wäldchen, A\. Binder, G\. Montavon, W\. Samek, and K\. MüllerUnmasking Clever Hans predictors and assessing what machines really learn\.Nature communications10\(1\),pp\. 1096\.Cited by:[§1](https://arxiv.org/html/2610.12493#S1.p1.1)\.
- Leviet al\.\(2022\)D\. Levi, L\. Gispan, N\. Giladi, and E\. FetayaEvaluating and calibrating uncertainty prediction in regression tasks\.Sensors22\(15\),pp\. 5540\.Cited by:[§4](https://arxiv.org/html/2610.12493#S4.p4.1)\.
- Liet al\.\(2016\)J\. Li, X\. Chen, E\. Hovy, and D\. JurafskyVisualizing and understanding neural models in NLP\.InProceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,pp\. 681–691\.Cited by:[§3\.3](https://arxiv.org/html/2610.12493#S3.SS3.p1.1)\.
- Lundberg and Lee \(2017\)S\. M\. Lundberg and S\. LeeA unified approach to interpreting model predictions\.Advances in neural information processing systems30\.Cited by:[§2\.1](https://arxiv.org/html/2610.12493#S2.SS1.p1.1)\.
- Maathuiset al\.\(2026\)H\. Maathuis, M\. Stalenhoef, S\. Van Otterloo, R\. Zwaal, K\. Van Montfort, and D\. SentDesigning effective explainable AI: a human\-centered evaluation of explanation formats in financial decision\-making\.Frontiers in Artificial Intelligence9,pp\. 1668029\.Cited by:[§3\.2](https://arxiv.org/html/2610.12493#S3.SS2.p1.1)\.
- Miró\-Nicolauet al\.\(2025\)M\. Miró\-Nicolau, A\. Jaume\-i\-Capó, and G\. Moyà\-AlcoverA comprehensive study on fidelity metrics for XAI\.Information Processing & Management62\(1\),pp\. 103900\.Cited by:[§3\.5](https://arxiv.org/html/2610.12493#S3.SS5.p1.1),[§3\.5](https://arxiv.org/html/2610.12493#S3.SS5.p2.1),[§5](https://arxiv.org/html/2610.12493#S5.p1.1)\.
- Mishraet al\.\(2020\)S\. Mishra, E\. Benetos, B\. L\. Sturm, and S\. DixonReliable local explanations for machine listening\.In2020 International Joint Conference on Neural Networks \(IJCNN\),pp\. 1–8\.Cited by:[footnote 1](https://arxiv.org/html/2610.12493#footnote1)\.
- Nautaet al\.\(2023\)M\. Nauta, J\. Trienes, S\. Pathak, E\. Nguyen, M\. Peters, Y\. Schmitt, J\. Schlötterer, M\. van Keulen, and C\. SeifertFrom anecdotal evidence to quantitative evaluation methods: A systematic review on evaluating explainable ai\.ACM Computing Surveys55\(13s\),pp\. 1–42\.Cited by:[§2\.2\.1](https://arxiv.org/html/2610.12493#S2.SS2.SSS1.p1.1),[§2\.2](https://arxiv.org/html/2610.12493#S2.SS2.p1.1)\.
- Nguyenet al\.\(2015\)A\. Nguyen, J\. Yosinski, and J\. CluneDeep neural networks are easily fooled: High confidence predictions for unrecognizable images\.InProceedings of the IEEE conference on computer vision and pattern recognition,pp\. 427–436\.Cited by:[§3\.4](https://arxiv.org/html/2610.12493#S3.SS4.p1.1)\.
- Pezeshkiet al\.\(2021\)M\. Pezeshki, O\. Kaba, Y\. Bengio, A\. C\. Courville, D\. Precup, and G\. LajoieGradient starvation: a learning proclivity in neural networks\.Advances in Neural Information Processing Systems34,pp\. 1256–1272\.Cited by:[§2\.2\.1](https://arxiv.org/html/2610.12493#S2.SS2.SSS1.p1.1)\.
- Ribeiroet al\.\(2016\)M\. T\. Ribeiro, S\. Singh, and C\. Guestrin” Why should i trust you?” explaining the predictions of any classifier\.InProceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining,pp\. 1135–1144\.Cited by:[footnote 1](https://arxiv.org/html/2610.12493#footnote1)\.
- Rieger and Hansen \(2020\)L\. Rieger and L\. K\. HansenIrof: a low resource evaluation metric for explanation methods\.arXiv preprint arXiv:2003\.08747\.Cited by:[item iii\.](https://arxiv.org/html/2610.12493#S4.I1.i3.p1.1)\.
- Ronget al\.\(2022\)Y\. Rong, T\. Leemann, V\. Borisov, G\. Kasneci, and E\. KasneciA consistent and efficient evaluation strategy for attribution methods\.arXiv preprint arXiv:2202\.00449\.Cited by:[§3\.2](https://arxiv.org/html/2610.12493#S3.SS2.p2.1),[§3\.4](https://arxiv.org/html/2610.12493#S3.SS4.p1.1),[§5](https://arxiv.org/html/2610.12493#S5.p1.1)\.
- Rony and Hassan \(2025\)M\. M\. U\. Rony and N\. HassanBetween Consensus and Ambiguity: Expert Evaluation of LLM Explanations for Misleading Headlines\.InProceedings of the Computation \+ Journalism Symposium \(C\+J ’25\),Miami, FL\.Cited by:[§3\.2](https://arxiv.org/html/2610.12493#S3.SS2.p1.1)\.
- Saarela and Podgorelec \(2024\)M\. Saarela and V\. PodgorelecRecent applications of explainable AI \(XAI\): A systematic literature review\.Applied Sciences14\(19\),pp\. 8884\.Cited by:[§1](https://arxiv.org/html/2610.12493#S1.p2.1)\.
- Sameket al\.\(2016\)W\. Samek, A\. Binder, G\. Montavon, S\. Lapuschkin, and K\. MüllerEvaluating the visualization of what a deep neural network has learned\.IEEE transactions on neural networks and learning systems28\(11\),pp\. 2660–2673\.Cited by:[§3\.1](https://arxiv.org/html/2610.12493#S3.SS1.p1.1),[§3\.3](https://arxiv.org/html/2610.12493#S3.SS3.p1.1)\.
- Sartiet al\.\(2024\)G\. Sarti, G\. Chrupała, M\. Nissim, and A\. BisazzaQuantifying the plausibility of context reliance in neural machine translation\.InThe Twelfth International Conference on Learning Representations \(ICLR 2024\),Cited by:[§4](https://arxiv.org/html/2610.12493#S4.p3.1)\.
- Selvarajuet al\.\(2020\)R\. R\. Selvaraju, M\. Cogswell, D\. Abhishek, V\. Ramakrishna, P\. Devi, and B\. DhruvGrad\-CAM: Visual Explanations from Deep Networks via Gradient\-Based Localization\.International Journal of Computer Vision128\(2\),pp\. 336–359\.Cited by:[§2\.1](https://arxiv.org/html/2610.12493#S2.SS1.p1.1)\.
- Seth and Sankarapu \(2025\)P\. Seth and V\. K\. SankarapuBridging the Gap in XAI\-Why Reliable Metrics Matter for Explainability and Compliance\.InEurIPS 2025 Workshop on Private AI Governance,Cited by:[§5](https://arxiv.org/html/2610.12493#S5.p1.1)\.
- Songet al\.\(2023\)J\. Song, K\. Cha, and J\. SeoOn Pitfalls of RemOve\-And\-Retrain: Data Processing Inequality Perspective\.arXiv preprint arXiv:2304\.13836\.Cited by:[§3\.2](https://arxiv.org/html/2610.12493#S3.SS2.p3.1)\.
- Sridharet al\.\(2026\)S\. Sridhar, A\. Xue, and E\. WongMissingness Bias Calibration in Feature Attribution Explanations\.InThe Fourteenth International Conference on Learning Representations,Cited by:[§3\.4](https://arxiv.org/html/2610.12493#S3.SS4.p2.1),[§4](https://arxiv.org/html/2610.12493#S4.p2.1)\.
- Sturmfelset al\.\(2020\)P\. Sturmfels, S\. Lundberg, and S\. LeeVisualizing the Impact of Feature Attribution Baselines\.Distill\.Note:https://distill\.pub/2020/attribution\-baselinesExternal Links:[Document](https://dx.doi.org/10.23915/distill.00022)Cited by:[§3\.3](https://arxiv.org/html/2610.12493#S3.SS3.p1.1)\.
- Sundararajanet al\.\(2017\)M\. Sundararajan, A\. Taly, and Q\. YanAxiomatic attribution for deep networks\.InInternational conference on machine learning,pp\. 3319–3328\.Cited by:[§3\.3](https://arxiv.org/html/2610.12493#S3.SS3.p1.1)\.
- Valdenegro\-Toro and Mulye \(2024\)M\. Valdenegro\-Toro and M\. MulyeSanity checks for explanation uncertainty\.InEuropean Conference on Computer Vision,pp\. 255–270\.Cited by:[§4](https://arxiv.org/html/2610.12493#S4.p2.1)\.
- Wuet al\.\(2021\)T\. Wu, M\. T\. Ribeiro, J\. Heer, and D\. S\. WeldPolyjuice: Generating counterfactuals for explaining, evaluating, and improving models\.InProceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing \(Volume 1: Long Papers\),pp\. 6707–6723\.Cited by:[§3\.3](https://arxiv.org/html/2610.12493#S3.SS3.p1.1)\.
- Zhanget al\.\(2025\)Z\. Zhang, J\. Gu, A\. Chowdhury, Z\. Mai, D\. Carlyn, T\. Berger\-Wolf, Y\. Su, and W\. ChaoFiner\-cam: Spotting the difference reveals finer details for visual explanation\.InProceedings of the Computer Vision and Pattern Recognition Conference,pp\. 9611–9620\.Cited by:[§4](https://arxiv.org/html/2610.12493#S4.p3.1)\.
- Zhenget al\.\(2024\)X\. Zheng, F\. Shirani, Z\. Chen, C\. Lin, W\. Cheng, W\. Guo, and D\. LuoF\-fidelity: A robust framework for faithfulness evaluation of explainable ai\.arXiv preprint arXiv:2410\.02970\.Cited by:[item \(i\)](https://arxiv.org/html/2610.12493#S3.I1.i1.2),[§3\.1](https://arxiv.org/html/2610.12493#S3.SS1.p1.1),[§3\.3](https://arxiv.org/html/2610.12493#S3.SS3.p1.1),[§3\.4](https://arxiv.org/html/2610.12493#S3.SS4.p1.1),[§5](https://arxiv.org/html/2610.12493#S5.p1.1)\.