DocXEval：多模态文档理解中可解释性的系统评估

# DocXEval: A Systematic Evaluation of Explainability in Multimodal Document Understanding
Source: [https://arxiv.org/html/2610.36929](https://arxiv.org/html/2610.36929)
Saifullah SaifullahEmail:[firstname\.lastname@dfki\.de](mailto:[email protected])Affiliation:Smart Data and Knowledge Services \(SDS\), German Research Center for Artificial Intelligence \(DFKI\), Trippstadter Str\. 122, Kaiserslautern, 67663, GermanyAffiliation:Department of Computer Science, University of Kaiserslautern\-Landau \(RPTU\), Erwin\-Schrödinger\-Str\. 52, Kaiserslautern, 67663, GermanyDavid DembinskyAffiliation:Smart Data and Knowledge Services \(SDS\), German Research Center for Artificial Intelligence \(DFKI\), Trippstadter Str\. 122, Kaiserslautern, 67663, GermanyAffiliation:Department of Computer Science, University of Kaiserslautern\-Landau \(RPTU\), Erwin\-Schrödinger\-Str\. 52, Kaiserslautern, 67663, GermanyStefan AgneAffiliation:Smart Data and Knowledge Services \(SDS\), German Research Center for Artificial Intelligence \(DFKI\), Trippstadter Str\. 122, Kaiserslautern, 67663, GermanyAffiliation:DeepReader GmbH, Kaiserslautern, 67663, GermanyAndreas DengelAffiliation:Smart Data and Knowledge Services \(SDS\), German Research Center for Artificial Intelligence \(DFKI\), Trippstadter Str\. 122, Kaiserslautern, 67663, GermanyAffiliation:Department of Computer Science, University of Kaiserslautern\-Landau \(RPTU\), Erwin\-Schrödinger\-Str\. 52, Kaiserslautern, 67663, GermanySheraz AhmedAffiliation:Smart Data and Knowledge Services \(SDS\), German Research Center for Artificial Intelligence \(DFKI\), Trippstadter Str\. 122, Kaiserslautern, 67663, GermanyAffiliation:DeepReader GmbH, Kaiserslautern, 67663, Germany

###### Abstract

Multimodal transformer models have driven unprecedented performance breakthroughs in modern document understanding\. However, the inherent black\-box nature of these models remains a major obstacle to their safe deployment in high\-stakes sectors\. Despite significant progress in explainability for vision and language tasks, explainable AI \(XAI\) research tailored to multimodal document understanding has received little attention\. To address this gap, we presentDocXEval, the first comprehensive explainability framework that adapts existing XAI methods and evaluation metrics to the multimodal \(text, layout, and image\) document understanding context, while introducing efficient multi\-target adaptations of both methods and metrics for domain\-specific tasks such as semantic entity recognition\. UsingDocXEval, we systematically evaluate 13 XAI methods across 3 document understanding tasks, 8 datasets, 4 models, and 17 evaluation metrics, and find that only a few methods perform consistently across all evaluation criteria, with Occlusion emerging as the most faithful and interpretable method overall\. Through a dedicated meta\-evaluation, we observe moderate to strong agreement among several faithfulness and complexity metrics while identifying certain ones that produce unreliable results\. Finally, through multiple dataset\-wide diagnostic analyses, we uncover key insights into model behavior, including the contribution of individual modalities across tasks and labels, spatial attribution patterns around target entities, and potential dataset biases\. Overall, this work serves as both a benchmark and a practical guide for future XAI research in multimodal document understanding\. To support reproducibility, we publicly release our code at[https://github\.com/saifullah3396/docxeval/](https://github.com/saifullah3396/docxeval/), along with thetorchxailibrary implementing all multimodal and multi\-target method and metric adaptations at[https://atriaml\.github\.io/torchxai/](https://atriaml.github.io/torchxai/)\.

### 1Introduction

Deep learning has driven state\-of\-the\-art performance in document understanding across a wide range of tasks, including document classification\([1](https://arxiv.org/html/2610.36929#bib.bib1),[2](https://arxiv.org/html/2610.36929#bib.bib2)\), information extraction\([3](https://arxiv.org/html/2610.36929#bib.bib3),[4](https://arxiv.org/html/2610.36929#bib.bib4)\), document visual question answering\([5](https://arxiv.org/html/2610.36929#bib.bib5)\), and more\([6](https://arxiv.org/html/2610.36929#bib.bib6),[7](https://arxiv.org/html/2610.36929#bib.bib7)\)\. As multimodal document understanding models become increasingly integrated into automated decision\-making pipelines\([8](https://arxiv.org/html/2610.36929#bib.bib8),[2](https://arxiv.org/html/2610.36929#bib.bib2),[9](https://arxiv.org/html/2610.36929#bib.bib9),[10](https://arxiv.org/html/2610.36929#bib.bib10),[11](https://arxiv.org/html/2610.36929#bib.bib11)\), ensuring the transparency of their decisions has become critical, especially for deployment in high\-stakes sectors where inadvertent errors can lead to devastating legal, financial, or societal consequences\([12](https://arxiv.org/html/2610.36929#bib.bib12),[13](https://arxiv.org/html/2610.36929#bib.bib13)\)\. However, despite a vast array of explainable AI \(XAI\) methods\([14](https://arxiv.org/html/2610.36929#bib.bib14),[15](https://arxiv.org/html/2610.36929#bib.bib15)\)and an equally diverse set of XAI evaluation metrics\([16](https://arxiv.org/html/2610.36929#bib.bib16),[17](https://arxiv.org/html/2610.36929#bib.bib17),[18](https://arxiv.org/html/2610.36929#bib.bib18)\)proposed for vision and language models\([19](https://arxiv.org/html/2610.36929#bib.bib19),[20](https://arxiv.org/html/2610.36929#bib.bib20),[21](https://arxiv.org/html/2610.36929#bib.bib21),[22](https://arxiv.org/html/2610.36929#bib.bib22)\), there is little guidance on extending these methods and metrics to the multimodal document understanding context, where models typically combine multiple heterogeneous input modalities, such as text, layout, and visual/image features, in an end\-to\-end fashion\([4](https://arxiv.org/html/2610.36929#bib.bib4),[2](https://arxiv.org/html/2610.36929#bib.bib2),[10](https://arxiv.org/html/2610.36929#bib.bib10),[23](https://arxiv.org/html/2610.36929#bib.bib23)\)\.

Existing XAI methods and evaluation metrics are typically sensitive to numerous configurable parameters, including the choice of baselines, feature grouping strategies, and attribution aggregation schemes\([16](https://arxiv.org/html/2610.36929#bib.bib16),[14](https://arxiv.org/html/2610.36929#bib.bib14),[15](https://arxiv.org/html/2610.36929#bib.bib15)\)\. In multimodal document understanding, where features originate from multiple modalities, these design choices become even more complex, as decisions must be made separately for each modality\. For example, a meaningful baseline \(i\.e\., one representing the absence of information\) for text tokens differs fundamentally from one for continuous layout coordinates or visual patches\. Analogous decisions arise for feature grouping and aggregation\. For instance, should text attributions be aggregated at the token, word, or line level, and should visual attributions be aggregated at the pixel or patch level? Each such decision critically affects both the resulting explanations and the evaluation metrics used to assess them\([17](https://arxiv.org/html/2610.36929#bib.bib17),[18](https://arxiv.org/html/2610.36929#bib.bib18)\), yet no principled guidelines exist for these choices in the existing XAI literature on document understanding\.

A separate challenge arises from the nature of the tasks themselves\. Many core document understanding tasks, such as semantic entity recognition and question answering, involve multiple predictions per input sequence\. For example, in semantic entity recognition, each token is independently classified, resulting in numerous explanation targets per sample\. In contrast, existing XAI methods\([15](https://arxiv.org/html/2610.36929#bib.bib15),[24](https://arxiv.org/html/2610.36929#bib.bib24),[16](https://arxiv.org/html/2610.36929#bib.bib16)\)and evaluation metrics\([18](https://arxiv.org/html/2610.36929#bib.bib18),[25](https://arxiv.org/html/2610.36929#bib.bib25)\)are typically designed around single\-target scenarios such as image classification\. This makes them either computationally infeasible or conceptually ill\-suited for direct application to multi\-target document tasks\.

In the absence of a principled framework for reasoning about these design choices in the multimodal document understanding context, several fundamental research questions \(RQs\) remain unanswered\.RQ1:How should existing XAI methods and evaluation metrics be adapted to handle multiple heterogeneous input modalities and multi\-target prediction tasks in document understanding?RQ2:Which XAI methods yield the most faithful and interpretable explanations in multimodal document understanding?RQ3:Given the widespread metric disagreement documented in prior work\([25](https://arxiv.org/html/2610.36929#bib.bib25),[26](https://arxiv.org/html/2610.36929#bib.bib26)\), what agreement and disagreement patterns emerge among evaluation metrics in this domain?RQ4:How can the resulting explanations be systematically analyzed to gain insights into model behavior across modalities? To address these RQs, we make the following contributions:

1. 1\.We introduceDocXEval, the first comprehensive explainability framework for multimodal document understanding that adapts1313XAI methods and1717evaluation metrics to generate and evaluate multimodal explanations, and additionally introduces efficient adaptations of both methods and metrics for multi\-target tasks such as semantic entity recognition\.
2. 2\.UsingDocXEval, we systematically benchmark these methods across33tasks,88datasets, and44models, and find that only a few methods perform consistently across all evaluation criteria, with Occlusion \(OCC\)\[[27](https://arxiv.org/html/2610.36929#bib.bib27)\]emerging as the most faithful and interpretable overall\.
3. 3\.We conduct a dedicated meta\-evaluation of metrics and find moderate to strong agreement among several faithfulness and complexity metrics, while identifying Monotonicity \(mon\)\([28](https://arxiv.org/html/2610.36929#bib.bib28)\)and Infidelity \(infid\)\([18](https://arxiv.org/html/2610.36929#bib.bib18)\)111Note that Infidelity in this work refers to the noisy baseline formulation proposed in the original work\[[18](https://arxiv.org/html/2610.36929#bib.bib18)\]\.as unreliable\.
4. 4\.Through multiple dataset\-wide diagnostic analyses, we uncover key insights into model behavior, such as variable modality reliance across tasks and labels, potential reliance on task\-irrelevant features, and spatial attribution patterns around target entities\.

### 2Related Work

Multimodal Document Understanding\.While the broader document understanding landscape is increasingly shifting towards generative approaches\([6](https://arxiv.org/html/2610.36929#bib.bib6),[29](https://arxiv.org/html/2610.36929#bib.bib29),[30](https://arxiv.org/html/2610.36929#bib.bib30)\), discriminative multimodal document foundation models \(predominantly transformer\-based\([31](https://arxiv.org/html/2610.36929#bib.bib31)\)\) remain a prominent choice for structured document understanding tasks\([2](https://arxiv.org/html/2610.36929#bib.bib2),[8](https://arxiv.org/html/2610.36929#bib.bib8),[7](https://arxiv.org/html/2610.36929#bib.bib7),[11](https://arxiv.org/html/2610.36929#bib.bib11)\)\. This is primarily due to their ease of adaptation \(such as fine\-tuning on downstream tasks\), reduced data privacy risks \(since most models are small enough to be deployed in\-house\), and lower computational costs\. Multimodal document understanding models typically employ early or late fusion techniques to integrate multiple document modalities\. For example, the LayoutLM family\([3](https://arxiv.org/html/2610.36929#bib.bib3),[4](https://arxiv.org/html/2610.36929#bib.bib4),[2](https://arxiv.org/html/2610.36929#bib.bib2)\)fuses text, position, and layout modalities directly through the early addition of token\-level embeddings, and fuses the image modality through self\-attention between image and text tokens\. LiLT\([10](https://arxiv.org/html/2610.36929#bib.bib10)\), on the other hand, omits the image modality entirely and takes text and layout as separate input tokens, which are fused through bi\-directional cross\-attention\. In contrast, DocFormer\([23](https://arxiv.org/html/2610.36929#bib.bib23)\)proposes a unique multimodal attention layer to fuse the image, text, and layout features, and additionally uses residual connections to inject visual and layout information into each attention block\. Several similar approaches have been proposed in the literature, differing in their modality fusion and training strategies\([8](https://arxiv.org/html/2610.36929#bib.bib8),[9](https://arxiv.org/html/2610.36929#bib.bib9),[32](https://arxiv.org/html/2610.36929#bib.bib32)\); however, few, if any, of these works explicitly consider model explainability\.

Explainable AI\.The field of XAI has produced a rich body of literature over the years\[[20](https://arxiv.org/html/2610.36929#bib.bib20)\], with numerous methods\[[33](https://arxiv.org/html/2610.36929#bib.bib33),[34](https://arxiv.org/html/2610.36929#bib.bib34),[15](https://arxiv.org/html/2610.36929#bib.bib15),[24](https://arxiv.org/html/2610.36929#bib.bib24),[35](https://arxiv.org/html/2610.36929#bib.bib35),[27](https://arxiv.org/html/2610.36929#bib.bib27)\]proposed to explain the decisions of black\-box AI models\. These methods can be largely categorized into two families: \(1\)post\-hocmethods, which generate explanations post\-training via an external explainer, and \(2\)intrinsicmethods, which build interpretability directly into the model architecture\. Among post\-hoc methods, feature\-attribution \(FA\) methods are the most prominent\. These methods assign importance to input features or feature groups by estimating their relevance to the model’s decision, typically resulting in importance maps that highlight both positively and negatively influencing features\[[33](https://arxiv.org/html/2610.36929#bib.bib33),[24](https://arxiv.org/html/2610.36929#bib.bib24),[35](https://arxiv.org/html/2610.36929#bib.bib35),[36](https://arxiv.org/html/2610.36929#bib.bib36)\]\. FA methods can be further divided into two main categories: backpropagation\-based methods\[[33](https://arxiv.org/html/2610.36929#bib.bib33),[35](https://arxiv.org/html/2610.36929#bib.bib35)\]and perturbation\-based methods\[[15](https://arxiv.org/html/2610.36929#bib.bib15),[24](https://arxiv.org/html/2610.36929#bib.bib24),[27](https://arxiv.org/html/2610.36929#bib.bib27)\]\.

Backpropagation\-based methods typically depend on propagating feature relevance backward through the model from the output to the input, utilizing either gradients\[[33](https://arxiv.org/html/2610.36929#bib.bib33),[35](https://arxiv.org/html/2610.36929#bib.bib35)\], modified gradient rules\[[27](https://arxiv.org/html/2610.36929#bib.bib27),[34](https://arxiv.org/html/2610.36929#bib.bib34)\], or activation differences\[[36](https://arxiv.org/html/2610.36929#bib.bib36)\]\. Popular examples include Saliency \(SAL\)\[[33](https://arxiv.org/html/2610.36929#bib.bib33)\], Input×\\timesGradient \(IxG\)\[[36](https://arxiv.org/html/2610.36929#bib.bib36)\], Integrated Gradients \(IG\)\[[35](https://arxiv.org/html/2610.36929#bib.bib35)\], DeepLIFT \(DL\)\[[36](https://arxiv.org/html/2610.36929#bib.bib36)\], and DeepSHAP \(DS\)\[[24](https://arxiv.org/html/2610.36929#bib.bib24)\], among others\.

Perturbation\-based methods generate importance maps by perturbing the input features and evaluating the impact of the perturbation on the model’s output\[[20](https://arxiv.org/html/2610.36929#bib.bib20)\]\. These methods are especially valued for their model\-agnostic nature, making them applicable to any model without requiring access to its internal architecture\. However, this advantage is generally accompanied by increased computational overhead\. Prominent perturbation\-based methods include Feature Ablation \(FA\)\[[27](https://arxiv.org/html/2610.36929#bib.bib27)\], Occlusion \(OCC\)\[[27](https://arxiv.org/html/2610.36929#bib.bib27)\], LIME\[[15](https://arxiv.org/html/2610.36929#bib.bib15)\], and KernelSHAP \(KS\)\[[24](https://arxiv.org/html/2610.36929#bib.bib24)\]\.

Recent work has also adapted feature attribution methods to document understanding tasks\[[37](https://arxiv.org/html/2610.36929#bib.bib37),[38](https://arxiv.org/html/2610.36929#bib.bib38),[39](https://arxiv.org/html/2610.36929#bib.bib39),[40](https://arxiv.org/html/2610.36929#bib.bib40),[41](https://arxiv.org/html/2610.36929#bib.bib41)\]\. Saifullahet al\.\[[42](https://arxiv.org/html/2610.36929#bib.bib42)\]adapted existing feature\-attribution methods to explain document image classification models and later proposed DocXplain\[[43](https://arxiv.org/html/2610.36929#bib.bib43)\], a perturbation\-based explainability framework for document image classification\. Briniet al\.\[[40](https://arxiv.org/html/2610.36929#bib.bib40)\]adapted gradient\-based methods to explain document layout analysis models\. Pouliquenet al\.\[[39](https://arxiv.org/html/2610.36929#bib.bib39)\]used gradient\-based feature\-attribution for hologram verification in documents\. In a different direction, Saifullahet al\.\[[38](https://arxiv.org/html/2610.36929#bib.bib38)\]explored intrinsic explainability for document image classification by augmenting existing models with an attention\-based explainability head\. Despite these advances, existing work remains focused on unimodal explanations and single\-target scenarios\.

![Refer to caption](https://arxiv.org/html/2610.36929v1/images/methodology/docxeval.png)Figure 1:Overview of theDocXEvalframework\. Given a dataset and a trained model, the framework \(1\) extracts modality\-specific inputs from each document, \(2\) defines feature segmentation masks, reference baselines per modality, and targets, \(3\) computes modality\-wise attribution scores via XAI methods, \(4\) evaluates explanation quality across four XAI evaluation criteria, and \(5\) conducts a systematic qualitative and quantitative analysis of results across datasets, models, and tasks\.Multimodal Explainable AI\.Explainable AI for multimodal models has also gained significant attention in recent years\. However, the field has primarily focused on vision\-language tasks such as visual question answering\([44](https://arxiv.org/html/2610.36929#bib.bib44),[45](https://arxiv.org/html/2610.36929#bib.bib45),[46](https://arxiv.org/html/2610.36929#bib.bib46)\)and image captioning\([47](https://arxiv.org/html/2610.36929#bib.bib47)\)\. For instance, VL\-InterpreT\([46](https://arxiv.org/html/2610.36929#bib.bib46)\)visualizes attention patterns via heatmaps for vision\-language transformers\. M2Lens\([48](https://arxiv.org/html/2610.36929#bib.bib48)\)provides an interactive analytics system integrating feature\-attribution methods\([15](https://arxiv.org/html/2610.36929#bib.bib15),[24](https://arxiv.org/html/2610.36929#bib.bib24),[35](https://arxiv.org/html/2610.36929#bib.bib35)\)for multimodal sentiment analysis\. Other works\([49](https://arxiv.org/html/2610.36929#bib.bib49),[45](https://arxiv.org/html/2610.36929#bib.bib45)\)decompose multimodal predictions into unimodal and cross\-modal components for vision\-language tasks to quantify interaction effects\. In the document domain, Souibguiet al\.\([41](https://arxiv.org/html/2610.36929#bib.bib41)\)proposed DocVXQA for explaining generative DocVQA models; however, it operates on visual inputs only and is not applicable to discriminative multimodal models\. While these works have advanced multimodal explainability considerably, they remain scoped to vision\-language settings\. Furthermore, most rely primarily on qualitative visual analysis\[[46](https://arxiv.org/html/2610.36929#bib.bib46),[45](https://arxiv.org/html/2610.36929#bib.bib45),[48](https://arxiv.org/html/2610.36929#bib.bib48)\], and those that do employ quantitative evaluation typically use only a small subset of metrics\[[41](https://arxiv.org/html/2610.36929#bib.bib41)\]\. To the best of our knowledge, no existing work systematically adapts and evaluates XAI methods and metrics for multimodal document understanding, the central research gap that we aim to address in this work\.

### 3DocXEval: Our Framework

Problem Formulation\.We consider a general document understanding setup, where a modelfθf\_\{\\theta\}is trained on a dataset𝒟=\{\(xi,yi\)\}i=1N\\mathcal\{D\}=\\\{\(x\_\{i\},y\_\{i\}\)\\\}\_\{i=1\}^\{N\}of document\-label pairs for a downstream taskTT\. Each documentx∈𝒟x\\in\\mathcal\{D\}is represented by up to four modalities222While we consider four modalities in this work, the framework is not limited to them and can directly accommodate additional modalities depending on the task\.: textual content𝒯\\mathcal\{T\}, 1D sequence positions𝒮\\mathcal\{S\}, 2D layout informationℒ\\mathcal\{L\}, and visual features𝒱\\mathcal\{V\}, which the model jointly processes to produce task\-level predictions\. We assume a preprocessing functionϕ:x↦\(𝒯,𝒮,ℒ,𝒱\)\\phi:x\\mapsto\(\\mathcal\{T\},\\mathcal\{S\},\\mathcal\{L\},\\mathcal\{V\}\)that extracts these modalities from each document, e\.g\., via standard optical character recognition \(OCR\) pipelines commonly used in multimodal document understanding\([4](https://arxiv.org/html/2610.36929#bib.bib4),[8](https://arxiv.org/html/2610.36929#bib.bib8),[10](https://arxiv.org/html/2610.36929#bib.bib10)\)\. Given this setting, we seek to \(i\) explain model predictions through modality\-wise feature attributions𝐀ℳ\\mathbf\{A\}\_\{\\mathcal\{M\}\}, and \(ii\) systematically evaluate them in terms of axiomatic soundness, faithfulness, complexity, and robustness\. Here,𝐀ℳ\\mathbf\{A\}\_\{\\mathcal\{M\}\}denotes the attribution scores assigned to modalityℳ∈\{𝒯,𝒮,ℒ,𝒱\}\\mathcal\{M\}\\in\\\{\\mathcal\{T\},\\mathcal\{S\},\\mathcal\{L\},\\mathcal\{V\}\\\}for a given prediction target\.

Framework Overview\.As illustrated in Fig\.[1](https://arxiv.org/html/2610.36929#S2.F1),DocXEvaloperates in five steps: \(1\) modality\-specific inputs are extracted from each document and projected to input embeddings; \(2\) feature segmentation masks and reference baselines are defined per modality, and task\-specific attribution targets are generated; \(3\) adapted XAI methods are applied to produce modality\-wise explanations with respect to each target; \(4\) the resulting explanations are evaluated through standardized metrics to generate evaluation scores for each sample\-target pair; and \(5\) evaluation scores are aggregated per dataset and systematically analyzed across datasets, models, and tasks\. We describe the main components of the framework in the following sections\.

#### 3\.1Input and Preprocessing

The first stage of the framework prepares a uniform attribution space across models and modalities\. Multimodal document understanding models\[[3](https://arxiv.org/html/2610.36929#bib.bib3),[23](https://arxiv.org/html/2610.36929#bib.bib23),[8](https://arxiv.org/html/2610.36929#bib.bib8)\]generally differ in how they represent inputs\. For example, text may be tokenized using different vocabularies, spatial positioning may be encoded as a bounding\-box or polygon coordinates, and visual features may be derived from CNN\-based embeddings\([23](https://arxiv.org/html/2610.36929#bib.bib23),[3](https://arxiv.org/html/2610.36929#bib.bib3)\)or patch\-based projections\([2](https://arxiv.org/html/2610.36929#bib.bib2)\)\. Irrespective of these differences, however, the models typically map their discrete token\-level inputs to continuous embedding representations via modality\-specific embedding functions\. We denote these asπℳ:ℳ→ℝd,ℳ∈\{𝒯,𝒮,ℒ\}\\pi\_\{\\mathcal\{M\}\}:\\mathcal\{M\}\\rightarrow\\mathbb\{R\}^\{d\},\\quad\\mathcal\{M\}\\in\\\{\\mathcal\{T\},\\mathcal\{S\},\\mathcal\{L\}\\\}\. For token\-level inputs, our framework operates at this embedding level, meaning the attribution scores are computed over the full embedding space of each token\. This provides three main advantages: \(1\) a consistent attributable unit across models regardless of tokenization scheme or input representation; \(2\) a common dimensionality across all token\-level modalities, ensuring each modality has the same number of attributable features per token; and \(3\) a uniform input representation across both gradient\-based and perturbation\-based methods333Gradient\-based methods require continuous representations for computing attribution scores, and while perturbation\-based methods do not strictly require them, a uniform input representation across both ensures a consistent comparison and evaluation\.\. For visual inputs𝒱\\mathcal\{V\}, our framework operates directly on raw image pixels rather than model\-specific visual embeddings \(e\.g\., patch projections\), as raw images are inherently continuous and can be directly segmented into a set of attributable features\.

#### 3\.2Feature Segmentation and Aggregation

In the second stage, we define a feature segmentation mask for each modality to group features into a smaller set of semantically meaningful units\. This is necessary for three reasons:

1. 1\.Since our framework operates on token\-level embeddings \(each of dimensionℝd\\mathbb\{R\}^\{d\}\) and raw image space \(typically of dimensionsℝC×H×W\\mathbb\{R\}^\{C\\times H\\times W\}\), the resulting attributable feature space becomes very large\. For instance, an input sequence of lengthL=512L=512with each token of dimensionℝd\\mathbb\{R\}^\{d\}, combined with visual features of dimensionℝ224×224×3\\mathbb\{R\}^\{224\\times 224\\times 3\}, yields hundreds of thousands of features across modalities, making explanations expensive to compute and evaluate\.
2. 2\.Perturbation\-based methods inherently operate over segmented features, producing coarser explanations with lower complexity scores, whereas gradient\-based methods produce fine\-grained per\-feature attributions\. Therefore, aggregating explanations over the same set of features is necessary for a fair cross\-method evaluation\.
3. 3\.Aggregating explanations into human\-understandable units improves interpretability; for instance, word\-level attributions are more meaningful than subword\-level scores, and image patch attributions are more interpretable than pixel\-level maps\.

Segmentation Strategy\.Formally, we define feature segmentation through a mask that assigns each input feature to a semantically coherent group\. For discrete modalitiesℳ∈\{𝒯,𝒮,ℒ\}\\mathcal\{M\}\\in\\\{\\mathcal\{T\},\\mathcal\{S\},\\mathcal\{L\}\\\}withLLtokens and embedding dimensiondd, we define a feature\-level maskmℳ∈ℕL×dm\_\{\\mathcal\{M\}\}\\in\\mathbb\{N\}^\{L\\times d\}where allddfeatures of tokenjjshare the same group assignmentkk\. The feature set for groupkkin modalityℳ\\mathcal\{M\}is then:

ℱℳ,k=\{πℳ​\(tj,l\)∣mℳ​\(j,l\)=k\}\\mathcal\{F\}\_\{\\mathcal\{M\},k\}=\\\{\\pi\_\{\\mathcal\{M\}\}\(t\_\{j\},l\)\\mid m\_\{\\mathcal\{M\}\}\(j,l\)=k\\\}\(1\)whereπℳ​\(tj,l\)\\pi\_\{\\mathcal\{M\}\}\(t\_\{j\},l\)denotes thell\-th element of the embedding of tokenjj\. Similarly, for visual modality𝒱\\mathcal\{V\}, we define a feature\-level maskm𝒱∈ℕH×W×Cm\_\{\\mathcal\{V\}\}\\in\\mathbb\{N\}^\{H\\times W\\times C\}where allCCchannels of pixel\(x,y\)\(x,y\)share the same group assignmentpp\. The feature set for visual groupppis:

ℱ𝒱,p=\{𝒱⁡\(x,y,c\)∣m𝒱​\(x,y,c\)=p\}\\mathcal\{F\}\_\{\\mathcal\{V\},p\}=\\\{\\mathcal\{V\}\(x,y,c\)\\mid m\_\{\\mathcal\{V\}\}\(x,y,c\)=p\\\}\(2\)In this work, we employ word\-level grouping for discrete modalities, where each groupkkin Eq\. \([1](https://arxiv.org/html/2610.36929#S3.E1)\) corresponds to a complete word, and patch\-level grouping for visual features, where each groupppin Eq\. \([2](https://arxiv.org/html/2610.36929#S3.E2)\) corresponds to a spatial image region\.

Feature Aggregation\.Given fine\-grained attribution scores computed at the embedding element or pixel level, we aggregate them to group\-level scores for downstream evaluation and analysis\. For a modalityℳ\\mathcal\{M\}with scalar attribution scoresaℳ,i∈ℝa\_\{\\mathcal\{M\},i\}\\in\\mathbb\{R\}assigned to each individual featureii, we first sum all attribution scores within a group and then normalize by a scaling factorwℳ,kw\_\{\\mathcal\{M\},k\}:

aℳ,k=wℳ,k​∑i∈𝒢ℳ,kaℳ,i,wℳ,k=minℳ′,k′⁡\|𝒢ℳ′,k′\|\|𝒢ℳ,k\|\\begin\{split\}a\_\{\\mathcal\{M\},k\}&=w\_\{\\mathcal\{M\},k\}\\sum\_\{i\\in\\mathcal\{G\}\_\{\\mathcal\{M\},k\}\}a\_\{\\mathcal\{M\},i\},\\\\ w\_\{\\mathcal\{M\},k\}&=\\frac\{\\min\_\{\\mathcal\{M\}^\{\\prime\},k^\{\\prime\}\}\|\\mathcal\{G\}\_\{\\mathcal\{M\}^\{\\prime\},k^\{\\prime\}\}\|\}\{\|\\mathcal\{G\}\_\{\\mathcal\{M\},k\}\|\}\\end\{split\}\(3\)where𝒢ℳ,k=\{i∣mℳ,i=k\}\\mathcal\{G\}\_\{\\mathcal\{M\},k\}=\\\{i\\mid m\_\{\\mathcal\{M\},i\}=k\\\}denotes the set of scalar feature indices in groupkk\. The scaling factorwℳ,kw\_\{\\mathcal\{M\},k\}is defined as the ratio of the smallest group size across all modalities to the size of the current group\. This ensures that attribution scores for the smallest feature group remain unscaled, while those for larger groups are scaled down proportionally to prevent their larger feature counts from inflating their importance\. For instance, in our setting, a single\-token word with embedding dimensiond=768d=768and an image patch of dimension16×16×3=76816\\times 16\\times 3=768both receive a weight of1\.01\.0, retaining their original scores, whereas a word spanningnntokens has its attributions downscaled by a factor ofnn\.

#### 3\.3Baseline Selection

![Refer to caption](https://arxiv.org/html/2610.36929v1/images/methodology/pert_rvlcdip.png)

![Refer to caption](https://arxiv.org/html/2610.36929v1/images/methodology/pert_rvlcdip.png)\(a\)RVL\-CDIP\[[50](https://arxiv.org/html/2610.36929#bib.bib50)\]\(classification\)
![Refer to caption](https://arxiv.org/html/2610.36929v1/images/methodology/pert_funsd.png)\(b\)FUNSD\[[51](https://arxiv.org/html/2610.36929#bib.bib51)\]\(SER\)

Figure 2:Modality\-wise model sensitivity to different baselines, shown for LayoutLMv3\[[2](https://arxiv.org/html/2610.36929#bib.bib2)\]on RVL\-CDIP\[[50](https://arxiv.org/html/2610.36929#bib.bib50)\]and FUNSD\[[51](https://arxiv.org/html/2610.36929#bib.bib51)\]\. The model is generally less sensitive to theMeanbaseline for the image modality on both datasets, whereas for spatial position, it is less sensitive to theZero Embeddingbaseline compared to the\[PAD\]baseline\. For the text modality, the model is less sensitive to the\[MASK\]token compared to other baselines\.Many feature attribution methods\([35](https://arxiv.org/html/2610.36929#bib.bib35),[15](https://arxiv.org/html/2610.36929#bib.bib15),[24](https://arxiv.org/html/2610.36929#bib.bib24)\)and evaluation metrics\([25](https://arxiv.org/html/2610.36929#bib.bib25)\)require a reference baseline in their computation\. For example, perturbation\-based XAI methods and evaluation metrics assess the impact of input features on model output by replacing their values with the baseline\. However, selecting a suitable baseline is not straightforward as it is possible that replacing a feature with a baseline produces an out\-of\-distribution input \(unseen during training\), and therefore the resulting impact on the model output may not reflect actual information removal but rather a distribution shift\([52](https://arxiv.org/html/2610.36929#bib.bib52)\)\. In a multimodal setup, this becomes further complicated, as each modality requires its own baseline, and the optimal combination of modality\-specific baselines for minimizing the out\-of\-distribution effect is not obvious\.

To address this, we propose a perturbation\-based baseline selection strategy which extends theReplacestrategy proposed by Hase\[[52](https://arxiv.org/html/2610.36929#bib.bib52)\]to the multimodal context\. In particular, for each modalityℳ\\mathcal\{M\}, we define a set of baseline candidatesℬℳ=\{b1ℳ,…,bnℳ\}\\mathcal\{B\}\_\{\\mathcal\{M\}\}=\\\{b\_\{1\}^\{\\mathcal\{M\}\},\\ldots,b\_\{n\}^\{\\mathcal\{M\}\}\\\}representing different mechanisms for removing information from that modality, and construct the Cartesian product𝒞=∏ℳℬℳ\\mathcal\{C\}=\\prod\_\{\\mathcal\{M\}\}\\mathcal\{B\}\_\{\\mathcal\{M\}\}over all modality baselines\. Then, for each combination𝐜∈𝒞\\mathbf\{c\}\\in\\mathcal\{C\}and varying perturbation levelsα∈\{0\.1,0\.25,0\.4,0\.8\}\\alpha\\in\\\{0\.1,0\.25,0\.4,0\.8\\\}, we compute the performance of the pretrained model on the perturbed test set, where for each sample we randomly replace a fractionα\\alphaof feature groups \(as defined in Section[3\.2](https://arxiv.org/html/2610.36929#S3.SS2)\) with their corresponding baseline values\. We repeat this process overR=3R=3random seeds, and compute the mean performance:

f¯​\(𝐜,α\)=1R​∑r=1Rf⁡\(𝐜,α,r\)\\bar\{f\}\(\\mathbf\{c\},\\alpha\)=\\frac\{1\}\{R\}\\sum\_\{r=1\}^\{R\}f\(\\mathbf\{c\},\\alpha,r\)\(4\)wheref⁡\(𝐜,α,r\)f\(\\mathbf\{c\},\\alpha,r\)refers to a task\-specific metric such as accuracy or F1\-score that determines the model performance\. To isolate the effect of a specific baselinebiℳb\_\{i\}^\{\\mathcal\{M\}\}for modalityℳ\\mathcal\{M\}on model performance, we take the average over all combinations containing that baseline, effectively marginalizing out the influence of other modalities’ baselines:

f¯​\(biℳ,α\)=1\|𝒞iℳ\|​∑𝐜∈𝒞iℳf¯​\(𝐜,α\)\\bar\{f\}\(b\_\{i\}^\{\\mathcal\{M\}\},\\alpha\)=\\frac\{1\}\{\|\\mathcal\{C\}\_\{i\}^\{\\mathcal\{M\}\}\|\}\\sum\_\{\\mathbf\{c\}\\in\\mathcal\{C\}\_\{i\}^\{\\mathcal\{M\}\}\}\\bar\{f\}\(\\mathbf\{c\},\\alpha\)\(5\)where𝒞iℳ=\{𝐜∈𝒞:𝐜ℳ=biℳ\}\\mathcal\{C\}\_\{i\}^\{\\mathcal\{M\}\}=\\\{\\mathbf\{c\}\\in\\mathcal\{C\}:\\mathbf\{c\}\_\{\\mathcal\{M\}\}=b\_\{i\}^\{\\mathcal\{M\}\}\\\}\. Finally, we compute the model sensitivity to baselinebiℳb\_\{i\}^\{\\mathcal\{M\}\}asf¯​\(biℳ,α\)−f¯0\\bar\{f\}\(b\_\{i\}^\{\\mathcal\{M\}\},\\alpha\)\-\\bar\{f\}\_\{0\}, wheref¯0\\bar\{f\}\_\{0\}denotes the unperturbed model performance\.

Fig\.[2](https://arxiv.org/html/2610.36929#S3.F2)shows the resulting modality\-wise sensitivity of LayoutLMv3\[[2](https://arxiv.org/html/2610.36929#bib.bib2)\]to different baseline candidates:\[PAD\],\[MASK\],Zero Embedding,Meanimage, andRandomimage\. As shown, sensitivity varies across modalities and strategies, with differences becoming more pronounced at higher perturbation levels\. To select the baselines for downstream explainability evaluation, for each modality, we compute the average sensitivity of each strategy across perturbation levels and choose the strategy that results in the lowest overall model sensitivity\. Intuitively, since all baselines inherently remove information, greater model sensitivity is more likely attributable to out\-of\-distribution inputs than to genuine information removal\.

#### 3\.4Attribution Targets

Our framework is designed to work across three types of document understanding tasks: \(1\) classification \(CLS\), where the model outputs a distributionp⁡\(y∣x\)p\(y\\mid x\)overCCdocument labels; \(2\) semantic entity recognition \(SER\), where the model outputs distributionsp⁡\(yi∣x\)p\(y\_\{i\}\\mid x\)overCCentity types for each token; and \(3\) question answering \(QA\), where given a questionqq, the model independently outputs distributionsp⁡\(s∣x,q\)p\(s\\mid x,q\)andp⁡\(e∣x,q\)p\(e\\mid x,q\)over start and end answer span positions\. For each task, we define an attribution target with respect to which the explanations are computed\. For document classification, the target is defined as the probability of the predicted classarg⁡max​p​\(y∣x\)\\arg\\max p\(y\\mid x\)\. For SER, a target is defined for each word as the probability of the predicted entity labelarg⁡max​p​\(yi∣x\)\\arg\\max p\(y\_\{i\}\\mid x\)of its first subword token\. This results inKKattribution targets per sample, whereKKis the total number of words in the sample\. ForQA, the targets are defined as the probabilities of the predicted start and end span positionsarg⁡max​p​\(s∣x,q\)\\arg\\max p\(s\\mid x,q\)andarg⁡max​p​\(e∣x,q\)\\arg\\max p\(e\\mid x,q\), respectively\.

#### 3\.5XAI Method Adaptations

Algorithm 1Batched Multi\-Target Backprop1:Forward function

ff, input

xx, targets

\{c1,…,cK\}\\\{c\_\{1\},\\ldots,c\_\{K\}\\\}, batch size

bb
2:Gradients

\{∇xfck​\(x\)\}k=1K\\\{\\nabla\_\{x\}f\_\{c\_\{k\}\}\(x\)\\\}\_\{k=1\}^\{K\}
3:

𝐲←f⁡\(x\)\\mathbf\{y\}\\leftarrow f\(x\)⊳\\trianglerightSingle forward pass

4:

𝐄←eye​\(K\)\\mathbf\{E\}\\leftarrow\\texttt\{eye\}\(K\)⊳\\trianglerightUnit vectors for each target

5:

𝐆←zeros​\(K,\|x\|\)\\mathbf\{G\}\\leftarrow\\texttt\{zeros\}\(K,\|x\|\)⊳\\trianglerightInitialize gradient storage

6:for

j=0j=0to

K−1K\-1step

bbdo

7:

𝐕←𝐄\[j:j\+b\]\\mathbf\{V\}\\leftarrow\\mathbf\{E\}\[j:j\+b\]⊳\\trianglerightBatch of unit vectors

8:

𝐆\[j:j\+b\]←autograd\.grad\(𝐲,x,𝐕\)\\mathbf\{G\}\[j:j\+b\]\\leftarrow\\texttt\{autograd\.grad\}\(\\mathbf\{y\},x,\\mathbf\{V\}\)⊳\\trianglerightAccumulate gradients with graph retention

9:endfor

10:return

𝐆\\mathbf\{G\}

This section describes how we adapt the existing XAI methods in our framework to work with multimodal document inputs and multi\-target tasks\.

Backpropagation\-based methods\.In single\-target setups such as classification, applying backpropagation\-based methods is straightforward\. Simple backpropagation\-based methods such as Saliency \(SAL\)\[[33](https://arxiv.org/html/2610.36929#bib.bib33)\], Guided Backpropagation \(GBP\)\[[34](https://arxiv.org/html/2610.36929#bib.bib34)\], and Input×\\timesGradient \(IxG\)\[[36](https://arxiv.org/html/2610.36929#bib.bib36)\]compute the backward pass with respect to each input modality, while methods that require reference baselines, such as Integrated Gradients \(IG\)\[[35](https://arxiv.org/html/2610.36929#bib.bib35)\]and DeepLIFT \(DL\)\[[36](https://arxiv.org/html/2610.36929#bib.bib36)\], use the predetermined modality\-wise baselines to compute attribution scores\.

In multi\-target tasks such as SER, each document containsKKattribution targets \(e\.g\., corresponding toKKpredicted word entities\), requiringKKattributions\{𝐀ℳ\(c1\),…,𝐀ℳ\(cK\)\}\\\{\\mathbf\{A\}\_\{\\mathcal\{M\}\}^\{\(c\_\{1\}\)\},\\ldots,\\mathbf\{A\}\_\{\\mathcal\{M\}\}^\{\(c\_\{K\}\)\}\\\}to be computed independently for each target\. A naive approach would requireKKseparate forward\-backward passes per sample\. To avoid this, after a single forward pass, we compute the Jacobian𝐉ℳ\\mathbf\{J\}\_\{\\mathcal\{M\}\}over allKKtargets simultaneously\. For instance, Saliency \(SAL\)\[[33](https://arxiv.org/html/2610.36929#bib.bib33)\]scores overKKtargets of a single sample can be computed as follows:

𝐉ℳ=\[∇ℳfc1​\(x\)T∇ℳfcK​\(x\)T\]∈ℝK×\|ℳ\|\\mathbf\{J\}\_\{\\mathcal\{M\}\}=\\begin\{bmatrix\}\\nabla\_\{\\mathcal\{M\}\}f\_\{c\_\{1\}\}\(x\)^\{T\}\\\\ \\vdots\\\\ \\nabla\_\{\\mathcal\{M\}\}f\_\{c\_\{K\}\}\(x\)^\{T\}\\end\{bmatrix\}\\in\\mathbb\{R\}^\{K\\times\|\\mathcal\{M\}\|\}\(6\)where each rowkkcontains the attribution for targetckc\_\{k\}with respect to modalityℳ\\mathcal\{M\}\. In practice, we compute the above Jacobian withK/bK/bbatched backward passes using PyTorchautograd\(see Algorithm[1](https://arxiv.org/html/2610.36929#alg1)\), wherebbis the batch size for the gradient computation\. We integrate all backpropagation\-based methods with this modified backward pass to accommodate multiple targets in each computation run\.

Perturbation\-based methods\.For single\-target scenarios, applying perturbation\-based methods is also straightforward with predetermined modality\-wise feature segmentation masks and reference baselines\. For instance, Feature Ablation \(FA\)\[[27](https://arxiv.org/html/2610.36929#bib.bib27)\]directly perturbs grouped features by replacing them with their corresponding reference baselines\. Occlusion \(OCC\)\[[27](https://arxiv.org/html/2610.36929#bib.bib27)\]works similarly, but instead of using a perturbation mask, uses a sliding window and strides over each modality to perturb the data\. LIME\[[15](https://arxiv.org/html/2610.36929#bib.bib15)\]and KernelSHAP \(KS\)\[[24](https://arxiv.org/html/2610.36929#bib.bib24)\]generate perturbed neighborhood samples around the input using the modality\-wise feature masks and reference baselines, and then fit local surrogate models to approximate feature contributions\.

For multi\-target tasks, since each perturbed forward pass already produces predictions for allKKtargets, we extend these methods to simultaneously accumulate the attribution scores over allKKtargets at each perturbation step, resulting inKKattribution maps in a single computation run\. For surrogate\-based methods, such as LIME\[[15](https://arxiv.org/html/2610.36929#bib.bib15)\]and KernelSHAP \(KS\)\[[24](https://arxiv.org/html/2610.36929#bib.bib24)\], we create the neighborhood dataset once for all targets and then trainKKsurrogate models independently\.

Attention\-based methods\.Attention\-based methods cannot directly produce attribution maps over the embedding space of all modalities, as the accessible modalities depend on the model architecture; for instance, LayoutLMv3\[[2](https://arxiv.org/html/2610.36929#bib.bib2)\]produces attention scores over text and visual tokens, whereas LiLT\[[10](https://arxiv.org/html/2610.36929#bib.bib10)\]produces attention scores over both text and layout tokens\. We therefore restrict attention\-based attributions to the modalities that are explicitly represented as separate tokens in each model\. Moreover, since attention methods only provide token\-level attribution scores, we distribute each score uniformly across the embedding dimensionddso that the resulting output lies in the same space as the outputs of other XAI methods\. This ensures a consistent downstream evaluation across all methods\.

For all single\-target scenarios, we use the method implementations available incaptum444[https://captum\.ai/](https://captum.ai/)and extend them with multi\-target support in thetorchxai555[https://atriaml\.github\.io/torchxai/](https://atriaml.github.io/torchxai/)library\.

#### 3\.6XAI Metric Adaptations

Table 1:Overview of the 17 evaluation metrics adapted in this work\.Perturbation Typeindicates whether the metric requires ordered \(features removed in attribution\-ranked order\), unordered \(random subset removal\), or no perturbation of the input\. FS refers to whether a metric supports computation over modality\-wise segmented features\.MTindicates whether the metric supports efficient multi\-target computation\. Chunking indicates whether a metric supports perturbation over feature chunks to improve computational efficiency\.TypeMetricPerturbation TypeRequires ModelRequires BaselineFS supportMT SupportChunkingCompleteness \(compl\)\([35](https://arxiv.org/html/2610.36929#bib.bib35)\)\(↓\\downarrow\)N/A✓✓N/A✓×\\timesAxiomaticNon\-Sensitivity \(non\-sens\)\([53](https://arxiv.org/html/2610.36929#bib.bib53)\)\(↓\\downarrow\)Unordered✓✓✓✓✓Area Over Perturbation Curve \(aopc\)\([54](https://arxiv.org/html/2610.36929#bib.bib54)\)\(↑\\uparrow\)Ordered✓✓✓×\\times✓Area Between Perturbation Curves \(abpc\)\([54](https://arxiv.org/html/2610.36929#bib.bib54)\)\(↑\\uparrow\)Ordered✓✓✓×\\times✓Sufficiency \(suf\)\([55](https://arxiv.org/html/2610.36929#bib.bib55)\)\(↓\\downarrow\)Ordered✓✓✓×\\times✓Sensitivity\-N \(sens\-n\)\([56](https://arxiv.org/html/2610.36929#bib.bib56)\)\(↓\\downarrow\)Unordered✓✓✓✓✓Infidelity \(infid\)\([18](https://arxiv.org/html/2610.36929#bib.bib18)\)\(↓\\downarrow\)Unordered✓×\\timesN/A✓N/AFaithfulness Estimation \(faith\-est\)\([57](https://arxiv.org/html/2610.36929#bib.bib57)\)\(↑\\uparrow\)Ordered✓✓✓×\\times✓Faithfulness Correlation \(faith\-corr\)\([58](https://arxiv.org/html/2610.36929#bib.bib58)\)\(↑\\uparrow\)Unordered✓✓✓✓✓Monotonicity \(mon\)\([28](https://arxiv.org/html/2610.36929#bib.bib28)\)\(↑\\uparrow\)Ordered✓✓✓×\\times✓FaithfulnessMonotonicity Correlation \(mon\-corr\)\([53](https://arxiv.org/html/2610.36929#bib.bib53)\)\(↑\\uparrow\)Unordered✓✓✓✓✓Effective Complexity \(eff\-comp\)\([53](https://arxiv.org/html/2610.36929#bib.bib53)\)\(↓\\downarrow\)Ordered✓✓✓×\\times✓Entropy\-based Complexity \(ent\-comp\)\([58](https://arxiv.org/html/2610.36929#bib.bib58)\)\(↓\\downarrow\)N/A×\\times×\\times✓N/AN/ASundararajan Complexity \(s\-comp\)\([35](https://arxiv.org/html/2610.36929#bib.bib35)\)\(↓\\downarrow\)N/A×\\times×\\times✓N/AN/AComplexitySparseness \(spar\)\([59](https://arxiv.org/html/2610.36929#bib.bib59)\)\(↑\\uparrow\)N/A×\\times×\\times✓N/AN/AMax Sensitivity \(max\-sens\)\([18](https://arxiv.org/html/2610.36929#bib.bib18)\)\(↓\\downarrow\)Unordered✓×\\timesN/A✓N/ARobustnessAvg Sensitivity \(avg\-sens\)\([18](https://arxiv.org/html/2610.36929#bib.bib18)\)\(↓\\downarrow\)Unordered✓×\\timesN/A✓N/A

This section describes how we adapt the existing XAI evaluation metrics to multimodal and multi\-target settings\.

Multimodal adaptation\.Since each XAI method produces separate attribution maps per modality𝐀ℳ,ℳ∈\{𝒯,𝒮,ℒ,𝒱\}\\mathbf\{A\}\_\{\\mathcal\{M\}\},\\mathcal\{M\}\\in\\\{\\mathcal\{T\},\\mathcal\{S\},\\mathcal\{L\},\\mathcal\{V\}\\\}, we first perform modality\-wise feature aggregation as described in Section[3\.2](https://arxiv.org/html/2610.36929#S3.SS2), so that each feature group carries a single score\. All metrics are then computed over these aggregated scores, treating each feature group independently of its originating modality\. For perturbation\-based metrics, the perturbation order is determined by ranking all feature groups jointly across modalities, and the corresponding features are then perturbed using their modality\-specific segmentation masks and reference baselines defined in previous sections\. There are two exceptions to this general pipeline: Completeness \(compl\)\([35](https://arxiv.org/html/2610.36929#bib.bib35)\)is computed on the raw attribution scores rather than group\-aggregated scores, as the axiom requires the exact sum over all individual attributions, and Infidelity \(infid\)\([18](https://arxiv.org/html/2610.36929#bib.bib18)\)adds noise directly to each modality input independently, rather than operating on the aggregated features, to keep the implementation close to the original formulation\.

Table 2:Overview of the eight datasets used in this work\. For each dataset, we provide the number of classes or entity types, dataset splits, the size of the subset used for XAI evaluation, and the task\-specific performance metric\.TaskDatasetClassesTrainValidationTestXAI Eval\. SetMetricTobacco3482\([60](https://arxiv.org/html/2610.36929#bib.bib60)\)102504278700700AccuracyRVL\-CDIP\[[50](https://arxiv.org/html/2610.36929#bib.bib50)\]1632000040000400001000AccuracyCLSDocLayNet\([61](https://arxiv.org/html/2610.36929#bib.bib61)\)1169103648049991000AccuracyFUNSD\[[51](https://arxiv.org/html/2610.36929#bib.bib51)\]4149—5050F1CORD\([62](https://arxiv.org/html/2610.36929#bib.bib62)\)30800100100100F1SROIE\([63](https://arxiv.org/html/2610.36929#bib.bib63)\)5626—347100F1SERWildReceipts\([64](https://arxiv.org/html/2610.36929#bib.bib64)\)131267—472100F1QADocVQA\([65](https://arxiv.org/html/2610.36929#bib.bib65)\)—10194128612871000ANLSMulti\-target adaptation\.As with the explanation methods, adapting the metrics for multi\-target settings is necessary to reduce computational costs\. Metrics that use unordered or random perturbations \(such asnon\-sens,mon\-corr,sens\-n,infid, andfaith\-corr\) can be efficiently adapted, as the effect of each perturbation can be accumulated over allKKtargets in a single forward pass\. We implement multi\-target support for all of these metrics\. However, metrics that require ordered perturbations \(such asaopc,suf,mon,faith\-est, andeff\-comp\) cannot be adapted for multi\-target computation, since the perturbation order depends on each target’s unique attribution scores\. Metrics that require recomputing explanations \(such asmax\-sensandavg\-sens\) are adapted by using the underlying multi\-target explanation mode\. Metrics that do not require model forward passes \(ent\-comp,s\-comp, andspar\) are computed independently for each target’s attribution map\.

All metrics are implemented inPyTorchwith GPU support, batched computation, and compatibility withcaptum\. We reuse the implementations of Max Sensitivity \(max\-sens\)\([18](https://arxiv.org/html/2610.36929#bib.bib18)\)and Infidelity \(infid\)\([18](https://arxiv.org/html/2610.36929#bib.bib18)\)fromcaptumand extend them for the multi\-target case\. The remaining metrics, along with their multi\-target adaptations, are implemented in thetorchxailibrary\. Table[1](https://arxiv.org/html/2610.36929#S3.T1)summarizes the supported metrics; details on individual metrics and their adaptations are provided in Appendix[8](https://arxiv.org/html/2610.36929#S8)\.

### 4Experimental Setup

Datasets\.We evaluate our framework on 8 benchmark datasets across three tasks:CLS,SER, andQA\. For classification, we use the RVL\-CDIP\[[50](https://arxiv.org/html/2610.36929#bib.bib50)\], Tobacco3482\([60](https://arxiv.org/html/2610.36929#bib.bib60)\), and DocLayNet\([61](https://arxiv.org/html/2610.36929#bib.bib61)\)datasets\. ForSER, we use the FUNSD\[[51](https://arxiv.org/html/2610.36929#bib.bib51)\], CORD\([62](https://arxiv.org/html/2610.36929#bib.bib62)\), SROIE\([63](https://arxiv.org/html/2610.36929#bib.bib63)\), and WildReceipts\([64](https://arxiv.org/html/2610.36929#bib.bib64)\)datasets\. ForQA, we use the DocVQA\([65](https://arxiv.org/html/2610.36929#bib.bib65)\)dataset\. Table[2](https://arxiv.org/html/2610.36929#S3.T2)presents an overview of the datasets\.

Models\.We select four models representing different modality fusion strategies: BERT\[[66](https://arxiv.org/html/2610.36929#bib.bib66)\]and RoBERTa\[[67](https://arxiv.org/html/2610.36929#bib.bib67)\]as text\-only baselines \(𝒯\\mathcal\{T\},𝒮\\mathcal\{S\}\), LiLT\[[10](https://arxiv.org/html/2610.36929#bib.bib10)\]as a multimodal model processing text, positional, and layout modalities \(𝒯\\mathcal\{T\},𝒮\\mathcal\{S\},ℒ\\mathcal\{L\}\), and LayoutLMv3\[[2](https://arxiv.org/html/2610.36929#bib.bib2)\]as a fully multimodal model processing all four modalities \(𝒯\\mathcal\{T\},𝒮\\mathcal\{S\},ℒ\\mathcal\{L\},𝒱\\mathcal\{V\}\)\. This selection enables analysis of how explainability methods behave across models with varying numbers of modalities and fusion strategies\.

Table 3:Model performance on individual datasets\. Best results are highlighted inbold\. Training hyperparameters are reported in Appendix[4](https://arxiv.org/html/2610.36929#S9.T4)\.Training details\.We initialize all models from publicly available pretrained checkpoints on Hugging Face666[https://huggingface\.co](https://huggingface.co/)and fine\-tune them using theAdamorAdam\-Woptimizers\. We use 40 training epochs for the classification task, 50 for the DocVQA task, and 100 for the SER task, with early stopping based on validation performance \(patience of 10\)\. Table[3](https://arxiv.org/html/2610.36929#S4.T3)reports the performance achieved by each model on the individual datasets\. Additional training hyperparameter details are provided in Appendix[9](https://arxiv.org/html/2610.36929#S9)\.

XAI Methods\.We adapt and investigate 13 XAI methods in this work, spanning three families: seven backpropagation\-based methods, namely Saliency \(SAL\)\[[33](https://arxiv.org/html/2610.36929#bib.bib33)\], Input×\\timesGradient \(IxG\)\[[36](https://arxiv.org/html/2610.36929#bib.bib36)\], Guided Backpropagation \(GBP\)\[[34](https://arxiv.org/html/2610.36929#bib.bib34)\], Integrated Gradients \(IG\)\[[35](https://arxiv.org/html/2610.36929#bib.bib35)\], GradientSHAP \(GS\)\[[24](https://arxiv.org/html/2610.36929#bib.bib24)\], DeepLIFT \(DL\)\[[36](https://arxiv.org/html/2610.36929#bib.bib36)\], and DeepSHAP \(DS\)\[[24](https://arxiv.org/html/2610.36929#bib.bib24)\]; four perturbation\-based methods, namely Feature Ablation \(FA\)\[[27](https://arxiv.org/html/2610.36929#bib.bib27)\], Occlusion \(OCC\)\[[27](https://arxiv.org/html/2610.36929#bib.bib27)\], LIME\[[15](https://arxiv.org/html/2610.36929#bib.bib15)\], and KernelSHAP \(KS\)\[[24](https://arxiv.org/html/2610.36929#bib.bib24)\]; and two attention\-based methods, namely Raw Attention \(RA\)\[[66](https://arxiv.org/html/2610.36929#bib.bib66)\]and Attention Rollout \(AR\)\[[68](https://arxiv.org/html/2610.36929#bib.bib68)\]\. Within attention\-based methods, we also considered Attention Flow \(AF\)\[[68](https://arxiv.org/html/2610.36929#bib.bib68)\]but excluded it due to its high computational cost when applied to large sequences\.

We configure each explainability method to achieve a suitable trade\-off between quality and computational efficiency\. For methods that require sampling or interpolation steps, such as Integrated Gradients \(IG\)\[[35](https://arxiv.org/html/2610.36929#bib.bib35)\], GradientSHAP \(GS\)\[[24](https://arxiv.org/html/2610.36929#bib.bib24)\], LIME\[[15](https://arxiv.org/html/2610.36929#bib.bib15)\], and KernelSHAP \(KS\)\[[24](https://arxiv.org/html/2610.36929#bib.bib24)\], we use 200 samples or steps\. For DeepSHAP \(DS\)\[[24](https://arxiv.org/html/2610.36929#bib.bib24)\], a reference baseline distribution ofN=100N=100samples is precomputed from the training set of each dataset\. For Occlusion \(OCC\)\[[27](https://arxiv.org/html/2610.36929#bib.bib27)\], we define modality\-specific sliding windows and strides: a window of 8 tokens with a stride of 4 for text, position, and layout modalities, and a window of3×16×163\\times 16\\times 16with a stride of3×8×83\\times 8\\times 8for the image modality\.

XAI Evaluation Details\.We evaluate the explanations using the 17 metrics listed in Table[1](https://arxiv.org/html/2610.36929#S3.T1), spanning four criteria: axiomatic soundness, faithfulness, complexity, and robustness\. For Sensitivity\-N \(sens\-n\)\([56](https://arxiv.org/html/2610.36929#bib.bib56)\)and Faithfulness Correlation \(faith\-corr\)\([58](https://arxiv.org/html/2610.36929#bib.bib58)\), we evaluate at perturbation levels of 20%, 40%, and 60%\. For perturbation\-based metrics, we use a chunk size of 1% where applicable, meaning 1% of feature groups are removed at each perturbation step\. Moreover, all metrics use the same modality\-wise feature segmentation masks and baselines as those used for computing the XAI attributions\. All experiments are conducted on subsets of the test sets as listed in Table[2](https://arxiv.org/html/2610.36929#S3.T2)\(XAI Eval\. Set\), using NVIDIA H200 and RTX A6000 GPUs\. Full hyperparameter configurations for explainability methods and evaluation metrics are provided in Appendix[9](https://arxiv.org/html/2610.36929#S9)\.

### 5Experimental Results

#### 5\.1Rank Analysis of XAI Methods

To determine which XAI methods perform best under each evaluation criterion, we conduct a rank\-based analysis\. For each metric, we first summarize the results per dataset via mean aggregation over the sample\-level scores; for multi\-target scenarios, each sample\-target pair is treated as an individual data point in this computation\. We then rank all XAI methods per dataset\-model setting and average the ranks across all settings\. We adopt a rank\-based approach because metric values can vary in scale across different setups, and therefore, direct aggregation may disproportionately favor certain setups\. However, while ranking solves the scaling issue, the rank difference between two methods may not be statistically significant\. To address this, we first apply the Friedman test over aggregated metric values to check the null hypothesis that all methods perform equally\. If the null hypothesis is rejected, the Holm\-adjusted pairwise Wilcoxon signed\-rank test is then applied \(as suggested in\([69](https://arxiv.org/html/2610.36929#bib.bib69)\)\) to determine pairwise indistinguishable groups\. For all statistical analyses, we assume anα\\alphaof 0\.05\. Both the mean ranks and statistical significance results are jointly visualized using critical difference \(CD\) diagrams\([70](https://arxiv.org/html/2610.36929#bib.bib70)\), where methods with statistically insignificant differences are connected by a horizontal bar\. Besides metric\-level ranking, we also perform mean aggregation across metrics within each category to obtain a global method ranking per criterion\. Fig\.[3](https://arxiv.org/html/2610.36929#S5.F3)shows the critical difference diagram for the XAI method rankings across all tasks, and Fig\.[4](https://arxiv.org/html/2610.36929#S5.F4)shows the mean ranks per evaluation criterion\. Per\-task results are additionally provided in Appendix[10\.1](https://arxiv.org/html/2610.36929#S10.SS1)\. For theaopcranking, we use the difference betweenaopcMoRFandaopcRANDat100%100\\%features removed as a single summary score\. For completeness,aopccurves over multiple feature removal steps are additionally provided in Appendix[10\.3](https://arxiv.org/html/2610.36929#S10.SS3)\.

Method Rankings and Statistical Significance\.From Fig\.[3](https://arxiv.org/html/2610.36929#S5.F3), we observe that although methods vary considerably in their raw rankings across the metrics, many of these differences are not statistically significant \(as highlighted by the horizontal bar\)\. For instance, oncompl, KS is ranked higher than IG, but both are statistically indistinguishable\. Likewise, on Non\-Sensitivity \(non\-sens\)\([53](https://arxiv.org/html/2610.36929#bib.bib53)\)and Sparseness \(spar\)\([59](https://arxiv.org/html/2610.36929#bib.bib59)\), the vast majority of methods show no statistically significant differences despite a wide separation in their raw ranks\. This effect is most extreme on Monotonicity \(mon\)\([28](https://arxiv.org/html/2610.36929#bib.bib28)\)and Infidelity \(infid\)\([18](https://arxiv.org/html/2610.36929#bib.bib18)\), where all or nearly all methods form a single indistinguishable group\. Upon investigation, we found that Monotonicity \(mon\)\([28](https://arxiv.org/html/2610.36929#bib.bib28)\), being a binary metric \(0 or 1\), yielded a value of00the majority of the time, as even minor noise in an otherwise monotonic perturbation curve is sufficient to trigger non\-monotonicity\. Similarly, Infidelity \(infid\)\([18](https://arxiv.org/html/2610.36929#bib.bib18)\)produced near\-identical raw scores across all methods, with the exception of RA and AR, for which the scores diverged from the rest but remained comparable to each other\. Based on these observations, we consider both metrics unsuitable for evaluation in this setting\. Overall, we observe similar patterns in task\-level method rankings, with slightly more pronounced statistical indistinguishability on some metrics, as detailed in Appendix[10\.1](https://arxiv.org/html/2610.36929#S10.SS1)\.

Figure 3:Critical difference diagram showing per\-metric XAI method rankings across all tasks\. Methods connected by a horizontal bar are statistically indistinguishable\. Lower ranks indicate better performance\. Task\-level critical difference diagrams are additionally provided in Appendix[10\.1](https://arxiv.org/html/2610.36929#S10.SS1)\.Ranking Patterns Across Metrics\.When examining rankings across individual metrics, some important observations arise\. On bothfaith\-corrandsens\-n, methods achieve very similar rankings across the 20%, 40%, and 60% perturbation levels, indicating that a single configuration may suffice for these metrics\. We also observe comparable orderings across some groups of faithfulness metrics; for instance, onaopc,abpc,faith\-est,faith\-corr, andsuf, methods produce similar ranking patterns\. This observation also holds for the robustness metricsmax\-sensandavg\-sens\.

Best\-Performing Methods per Criterion\.Despite the widespread statistical indistinguishability, certain method groups consistently rank higher or lower across metrics\. Basic backpropagation\-based methods, including SAL, GBP, IxG, and DL, occupy the lowest ranks across the majority of metrics\. Oncompl, IG and KS perform the best, whereas LIME and OCC lead onnon\-sens\. These observations are also reflected in the mean ranking across axiomatic properties \(Fig\.[4](https://arxiv.org/html/2610.36929#S5.F4)\), where LIME, IG, RA, and OCC lead overall\. On individual complexity metrics, KS and AR perform the worst, whereas LIME, RA, OCC, and FA consistently rank higher than other methods\. The average ranking on the complexity criterion \(see Fig\.[4](https://arxiv.org/html/2610.36929#S5.F4)\) confirms these findings, with LIME, RA, OCC, and FA ranking the highest, while KS ranks considerably lower than all other methods\. On faithfulness metrics, OCC, FA, and RA show the best rankings overall\. For instance, onfaith\-estandmon\-corr, OCC and FA considerably outperform other methods\. Attention\-based methods, RA and AR, show variable results on faithfulness; however, RA generally ranks as one of the best alongside OCC\. The average ranking on faithfulness supports these findings \(see Fig\.[4](https://arxiv.org/html/2610.36929#S5.F4)\), with OCC, RA, FA, and IG being the top\-4 methods\. On robustness, attention\-based methods considerably outperform all other method families; the average ranking on robustness confirms this, with AR ranking first, followed by RA, IG, and OCC\. Taken together, the methods that most consistently stand out as best\-performing across all evaluation criteria are OCC, RA, and IG\.

Figure 4:Aggregated mean ranks of XAI methods per evaluation criterion across all tasks, datasets, and models\. Lower ranks indicate better performance\. Per\-task breakdowns are provided in Appendix[10\.1](https://arxiv.org/html/2610.36929#S10.SS1)\.![Refer to caption](https://arxiv.org/html/2610.36929v1/images/results/all_datasets/ranked_correlation_heatmaps_faithfulness.png)Figure 5:Pairwise Spearman correlation between faithfulness metric rankings across all datasets, models, and explainer settings\. Per\-task correlation matrices are provided in Appendix[10\.2](https://arxiv.org/html/2610.36929#S10.SS2)\.![Refer to caption](https://arxiv.org/html/2610.36929v1/images/results/all_datasets/ranked_correlation_heatmaps_complexity.png)Figure 6:Pairwise Spearman correlation between complexity metric rankings across all datasets, models, and explainer settings\. Per\-task correlation matrices are provided in Appendix[10\.2](https://arxiv.org/html/2610.36929#S10.SS2)\.
#### 5\.2Metric Agreement Analysis

The metric disagreement problem has been well\-noted in prior work\([25](https://arxiv.org/html/2610.36929#bib.bib25),[26](https://arxiv.org/html/2610.36929#bib.bib26)\), and therefore, it is important to investigate whether multiple evaluation metrics agree or disagree on the relative ranking of explanation methods\. We assess metric agreement for the faithfulness and complexity categories by computing the Spearman correlation between metric rankings across all datasets, models, and explainer settings\. Such analysis can not only improve our confidence in the evaluation, but also help determine whether some metrics are redundant, e\.g\., when two metrics are highly correlated\. Fig\.[5](https://arxiv.org/html/2610.36929#S5.F5)and Fig\.[6](https://arxiv.org/html/2610.36929#S5.F6)present the correlation heatmaps for faithfulness and complexity metrics, respectively; per\-task results are additionally provided in Appendix[10\.2](https://arxiv.org/html/2610.36929#S10.SS2)\.

Faithfulness Metrics\.On the faithfulness criterion, bothfaith\-corrandsens\-nshow strong correlation across their respective perturbation levels \(20%, 40%, 60%\), aligning with the observations made via the critical difference diagrams\. Strong correlation is observed betweenaopc,faith\-est,abpc,faith\-corr, andsuf\.sens\-nalso shows a moderate correlation with other faithfulness metrics \(around 0\.5–0\.65\), whereasmon\-corris only weakly correlated withfaith\-est\. Notably, neitherinfidnormonshow any correlation with other faithfulness metrics\. This lack of correlation coincides with the statistical indistinguishability and unreliability observed for these metrics in the critical difference diagrams\.

Complexity Metrics\.On the complexity criterion,s\-comp,spar, andent\-compshow a moderate correlation with each other \(around 0\.54–0\.66\)\.eff\-comp, however, shows only a very weak correlation withent\-compandspar, and no correlation withs\-comp\.

For both criteria, the observations remained largely consistent across individual task\-level results \(see Appendix[10\.2](https://arxiv.org/html/2610.36929#S10.SS2)\)\.

#### 5\.3Computational Efficiency of Multi\-Target Adaptations

In this section, we examine the computational efficiency of our multi\-target adaptations for both XAI methods and evaluation metrics\.

Figure 7:Execution time comparison between iterative \(batched\) and multi\-target attribution computation on CORD \(K=26K=26\) and FUNSD \(K=179K=179\) datasets across two GPUs\. Numbers above bars indicate the speedup factor of multi\-target over iterative computation\. Note the log scale on the y\-axis\.XAI Methods\.Fig\.[7](https://arxiv.org/html/2610.36929#S5.F7)compares the execution time of iterative versus multi\-target attribution computation for five XAI methods \(OCC, LIME, SAL, DL, and IG\)\. We report the average execution time over a fixed number of samples on the CORD and FUNSD datasets across two GPUs \(H200 and RTX A6000\), along with the speedup factor for each method\. Note that for the iterative scenario, we use batched computation to keep a fair comparison\. Perturbation\-based methods such as OCC achieve speedups roughly proportional to the number of targets per sample, reaching approximately 25×\\timeson CORD \(26 targets/sample\) and up to 189×\\timeson FUNSD \(179 targets/sample\), with minor variations between GPUs\. For surrogate\-based methods such as LIME, speedups are lower and more GPU\-dependent, ranging from 7×\\timesto 17×\\timesdepending on the dataset and hardware\. Backpropagation\-based methods show comparatively modest but still notable gains, consistently achieving 1\.5×\\times–1\.7×\\timesspeedups regardless of the number of targets\.

![Refer to caption](https://arxiv.org/html/2610.36929v1/images/results/seq_cls/runtime_summary_plot.png)\(a\)Classification \(single\-target\)
![Refer to caption](https://arxiv.org/html/2610.36929v1/images/results/token_cls/runtime_summary_plot.png)\(b\)Semantic Entity Recognition \(multi\-target\)

Figure 8:Mean per\-sample\-target computation time of each evaluation metric onCLS\(single\-target\) andSER\(multi\-target\) tasks\. All results are computed on H100/H200 GPUs\. Unordered perturbation metrics benefit significantly from multi\-target adaptations, while ordered metrics remain expensive as they cannot be adapted for multi\-target computation\.Evaluation Metrics\.Fig\.[8](https://arxiv.org/html/2610.36929#S5.F8)compares the per\-sample\-target execution time of all evaluation metrics acrossCLS\(single\-target\) andSER\(multi\-target\) settings\. All results are computed on H100/H200 GPUs, with execution times averaged over all datasets and XAI methods\. Within the axiomatic metrics,complbenefits significantly from the multi\-target support, with execution times reaching below11ms\. Complexity metricss\-comp,spar, andent\-compshow similar computational times on both tasks even without multi\-target adaptations\. For robustness metrics, although a direct comparison is not possible as the two tasks are computed on different datasets, computation on theSERtask is roughly3\.8×3\.8\\timesfaster than on theCLStask, which can be attributed to the underlying use of multi\-target adaptations of the explainers when computing these metrics\. The advantage of multi\-target adaptations becomes most apparent when comparing metrics that use ordered perturbations \(e\.g\.,aopc,abpc,suf\) against those that use unordered perturbations\. On the single\-targetCLStask, unordered metrics take approximately193193–521521ms while ordered metrics take approximately3,0693\{,\}069ms, making the former roughly66–16×16\\timesfaster\. On theSERtask, multi\-target adaptations widen this gap significantly: execution times for unordered metrics drop to approximately1010–1313ms per sample\-target, while those for ordered metrics remain at approximately2,2872\{,\}287ms as they cannot be adapted, yielding speedups of176176–229×229\\times\.

#### 5\.4Qualitative Analysis

To complement the quantitative evaluation, we qualitatively analyze modality\-wise explanations produced by the three best\-performing methods: IG, OCC, and RA\. Fig\.[9](https://arxiv.org/html/2610.36929#S5.F9), Fig\.[10](https://arxiv.org/html/2610.36929#S5.F10), and Fig\.[11](https://arxiv.org/html/2610.36929#S5.F11)show randomly selected samples across all three tasks:CLS\(RVL\-CDIP\[[50](https://arxiv.org/html/2610.36929#bib.bib50)\]and DocLayNet\([61](https://arxiv.org/html/2610.36929#bib.bib61)\)\),SER\(CORD\([62](https://arxiv.org/html/2610.36929#bib.bib62)\)and FUNSD\[[51](https://arxiv.org/html/2610.36929#bib.bib51)\]\), andQA, respectively\. For each sample, we visualize the text, position, layout, aggregated text, and image attribution maps\. Note that for RA, only aggregated text and image attributions are visualized, as it does not generate position\-level and layout\-level explanations\. Additional samples are provided in Appendix[10\.5](https://arxiv.org/html/2610.36929#S10.SS5)\.

IG \(RVL\-CDIP\)

![Refer to caption](https://arxiv.org/html/2610.36929v1/figs/main/seq_cls/images/img_0001.png)
Occ\. \(RVL\-CDIP\)

![Refer to caption](https://arxiv.org/html/2610.36929v1/figs/main/seq_cls/images/img_0002.png)
RA \(RVL\-CDIP\)

![Refer to caption](https://arxiv.org/html/2610.36929v1/figs/main/seq_cls/images/img_0003.png)
IG \(DocLayNet\)

![Refer to caption](https://arxiv.org/html/2610.36929v1/figs/main/seq_cls/images/img_0004.png)
Occ\. \(DocLayNet\)

![Refer to caption](https://arxiv.org/html/2610.36929v1/figs/main/seq_cls/images/img_0005.png)
RA \(DocLayNet\)

![Refer to caption](https://arxiv.org/html/2610.36929v1/figs/main/seq_cls/images/img_0006.png)

Figure 9:Modality\-wise explanations of the LayoutLMv3\[[2](https://arxiv.org/html/2610.36929#bib.bib2)\]model predictions for two randomly selected samples from RVL\-CDIP\[[50](https://arxiv.org/html/2610.36929#bib.bib50)\]and DocLayNet\([61](https://arxiv.org/html/2610.36929#bib.bib61)\)\(CLStask\)\. Results are shown for three methods: IG, OCC, and RA\.IG \(CORD\)

![Refer to caption](https://arxiv.org/html/2610.36929v1/figs/main/token_cls/images/img_0001.png)
Occ\. \(CORD\)

![Refer to caption](https://arxiv.org/html/2610.36929v1/figs/main/token_cls/images/img_0002.png)
RA \(CORD\)

![Refer to caption](https://arxiv.org/html/2610.36929v1/figs/main/token_cls/images/img_0003.png)
IG \(FUNSD\)

![Refer to caption](https://arxiv.org/html/2610.36929v1/figs/main/token_cls/images/img_0004.png)
Occ\. \(FUNSD\)

![Refer to caption](https://arxiv.org/html/2610.36929v1/figs/main/token_cls/images/img_0005.png)
RA \(FUNSD\)

![Refer to caption](https://arxiv.org/html/2610.36929v1/figs/main/token_cls/images/img_0006.png)

Figure 10:Modality\-wise explanations of the LayoutLMv3\[[2](https://arxiv.org/html/2610.36929#bib.bib2)\]model predictions for two randomly selected samples from CORD\([62](https://arxiv.org/html/2610.36929#bib.bib62)\)and FUNSD\[[51](https://arxiv.org/html/2610.36929#bib.bib51)\]\(SERtask\)\. Results are shown for three methods: IG, OCC, and RA, and for each sample, the target entity to be labeled is highlighted in a green bounding box\.IG \(DocVQA\)

![Refer to caption](https://arxiv.org/html/2610.36929v1/figs/main/qa/images/img_0001.png)
Occ\. \(DocVQA\)

![Refer to caption](https://arxiv.org/html/2610.36929v1/figs/main/qa/images/img_0002.png)
RA \(DocVQA\)

![Refer to caption](https://arxiv.org/html/2610.36929v1/figs/main/qa/images/img_0003.png)
IG \(DocVQA\)

![Refer to caption](https://arxiv.org/html/2610.36929v1/figs/main/qa/images/img_0004.png)
Occ\. \(DocVQA\)

![Refer to caption](https://arxiv.org/html/2610.36929v1/figs/main/qa/images/img_0005.png)
RA \(DocVQA\)

![Refer to caption](https://arxiv.org/html/2610.36929v1/figs/main/qa/images/img_0006.png)

Figure 11:Modality\-wise explanations of the LayoutLMv3\[[2](https://arxiv.org/html/2610.36929#bib.bib2)\]model predictions for two randomly selected samples from DocVQA\([65](https://arxiv.org/html/2610.36929#bib.bib65)\)\(QAtask\)\. Results are shown for three methods: IG, OCC, and RA\.Classification\.Fig\.[9](https://arxiv.org/html/2610.36929#S5.F9)presents modality\-wise explanations for LayoutLMv3\[[2](https://arxiv.org/html/2610.36929#bib.bib2)\]on two classification samples from RVL\-CDIP\[[50](https://arxiv.org/html/2610.36929#bib.bib50)\]and DocLayNet\([61](https://arxiv.org/html/2610.36929#bib.bib61)\)\. On the RVL\-CDIP\[[50](https://arxiv.org/html/2610.36929#bib.bib50)\]sample \(budget class\), IG suggests the model relies on both text and image modalities for its classification decision\. One notable observation is that it assigns strong positive importance to the text modality of certain tokens \(e\.g\., the document identity number or document header\), but this is accompanied by negative importance from the layout modality of the same tokens\. Interestingly, from the aggregated text\-level attributions \(Agg\. Text\.\), it can be observed that for some tokens, these counteracting attributions cancel each other out, resulting in negligible aggregated attribution, but many words still emerge as more prominent in the aggregated view\. For instance, the document identity number emerges with a strongly negative influence in the aggregated view\. OCC, in contrast to IG, suggests the model relies heavily on the visual layout of the image as well as the spatial positioning of text, including table headers\. RA similarly indicates strong reliance on the image modality, while for the text modality, it suggests a strong focus on table header values\. Notably, all three methods assign noticeable importance to the wordESTIMATE, which intuitively makes sense for the budget class\. On the DocLayNet\([61](https://arxiv.org/html/2610.36929#bib.bib61)\)sample \(scientific article\), all three methods indicate that the model relies predominantly on the image modality for its classification decision, with no significant focus on textual modalities\. Here, the focus appears to be mainly on the visual formatting of the document, particularly paragraph indentation and white space around the main text\.

Semantic Entity Recognition\.Fig\.[10](https://arxiv.org/html/2610.36929#S5.F10)presents the modality\-wise explanations for the predictions of the LayoutLMv3\[[2](https://arxiv.org/html/2610.36929#bib.bib2)\]model on two randomly selected SER samples from CORD\([62](https://arxiv.org/html/2610.36929#bib.bib62)\)and FUNSD\[[51](https://arxiv.org/html/2610.36929#bib.bib51)\]\. For each sample, the target entity is highlighted in a green bounding box\. On the CORD\([62](https://arxiv.org/html/2610.36929#bib.bib62)\)sample, where the target entity isI\-MENU\.NM, IG and OCC show no focus on the image modality, while RA only very weakly focuses on some image regions far from the actual target\. All three methods show strong focus on text in close vicinity to the target entity\. OCC in particular shows that the model strongly focuses on the 1D position modality of tokens preceding the target entity, with slight focus on the 2D positions of the same tokens\. RA shows a similar pattern but also highlights some text that is not in the immediate vicinity of the target\. In contrast to RA and OCC, IG assigns particularly high importance to tokens far from the target token, a pattern that is difficult to interpret\. Additionally, it again shows counteracting positive and negative attributions for some tokens that zero out in the aggregated text attributions\. Similar patterns are observed on the FUNSD sample \(target entityI\-QUESTION\), where OCC and RA provide concise explanations focused on text tokens in the local neighborhood of the target token\. IG, on the other hand, spreads importance across the entire document, significantly increasing the explanation complexity\.

Document Visual Question Answering\.Fig\.[11](https://arxiv.org/html/2610.36929#S5.F11)presents the modality\-wise explanations for the predictions of the LayoutLMv3\[[2](https://arxiv.org/html/2610.36929#bib.bib2)\]model on two randomly selected DocVQA\([65](https://arxiv.org/html/2610.36929#bib.bib65)\)samples\. In each case, the explanation is computed with respect to the predicted start or end answer token\. On the first sample, IG indicates a strong focus on the question tokens along with some focus on the tokens relevant to the answer token\. OCC indicates strong focus on the text and position modalities of the answer token$23\.7and the tokens in its close vicinity\. RA only assigns high importance to the answer token itself and the wordInvestment, which is relevant to the prediction\. Interestingly, it also indicates some focus on the top region of the image\. Similar patterns are observed on the second sample, where again RA assigns high importance to the answer token and top image patches\. OCC assigns high importance to the tokens in the close vicinity of the answer token, and especially to its layout and position modalities\. IG assigns high importance to the question text and to tokens vertically and horizontally aligned with the answer token\.

![Refer to caption](https://arxiv.org/html/2610.36929v1/images/results/seq_cls/modality_fraction_bar_RVL-CDIP_LayoutLMv3.png)
![Refer to caption](https://arxiv.org/html/2610.36929v1/images/results/seq_cls/modality_fraction_bar_DocLayNet_LayoutLMv3.png)
![Refer to caption](https://arxiv.org/html/2610.36929v1/images/results/token_cls/modality_fraction_bar_CORD_LayoutLMv3.png)
![Refer to caption](https://arxiv.org/html/2610.36929v1/images/results/token_cls/modality_fraction_bar_FUNSD_LayoutLMv3.png)
![Refer to caption](https://arxiv.org/html/2610.36929v1/images/results/qa/modality_fraction_bar_DocVQA_LayoutLMv3.png)

Figure 12:Modality contributions computed over explanations generated by IG, OCC, and RA for the LayoutLMv3\[[2](https://arxiv.org/html/2610.36929#bib.bib2)\]model\. Results are reported as a dataset\-wide mean \(left\) and as a per\-label mean for four randomly selected labels \(right\)\. For DocVQA\([65](https://arxiv.org/html/2610.36929#bib.bib65)\), contributions are reported separately for the start and end tokens of the predicted answer\.
#### 5\.5Global Attribution Analysis

While qualitative analysis of individual samples provides useful insights into local model behavior, inspecting explanations one sample at a time is neither scalable nor sufficient for drawing broader conclusions\. To obtain global insights, we conduct three dataset\-wide analyses: a modality contribution analysis that examines which modalities the model relies on \(Section[5\.5\.1](https://arxiv.org/html/2610.36929#S5.SS5.SSS1)\), a spatial attribution locality analysis that measures how concentrated attributions are around target entities \(Section[5\.5\.2](https://arxiv.org/html/2610.36929#S5.SS5.SSS2)\), and a top\-kkword frequency analysis that identifies which words are most frequently attributed across classes \(Section[5\.5\.3](https://arxiv.org/html/2610.36929#S5.SS5.SSS3)\)\.

##### 5\.5\.1Modality Contribution Analysis

To quantify the contribution of each modality across the dataset, we select the top\-kkmost important features in each sample’s explanation and compute the fraction of features belonging to each modality\. The results are then aggregated over the dataset or per class\. In practice, we compute modality contribution on the top 10% of features\. As a representative example, Fig\.[12](https://arxiv.org/html/2610.36929#S5.F12)presents these fractions for the LayoutLMv3\[[2](https://arxiv.org/html/2610.36929#bib.bib2)\]model on five datasets \(RVL\-CDIP\[[50](https://arxiv.org/html/2610.36929#bib.bib50)\], DocLayNet\([61](https://arxiv.org/html/2610.36929#bib.bib61)\), CORD\([62](https://arxiv.org/html/2610.36929#bib.bib62)\), FUNSD\[[51](https://arxiv.org/html/2610.36929#bib.bib51)\], and DocVQA\([65](https://arxiv.org/html/2610.36929#bib.bib65)\)\), reported both as a dataset\-wide mean and per label for four randomly selected labels per dataset\. For DocVQA\([65](https://arxiv.org/html/2610.36929#bib.bib65)\), fractions are reported separately for the start and end tokens of the predicted answer\. Additional results are provided in Appendix[10\.6](https://arxiv.org/html/2610.36929#S10.SS6)\.

Focusing on RVL\-CDIP\[[50](https://arxiv.org/html/2610.36929#bib.bib50)\], RA indicates that the model relies heavily on the image modality \(up to 80%\), with considerably lower importance on text\. Occlusion \(OCC\)\[[27](https://arxiv.org/html/2610.36929#bib.bib27)\]suggests a similar pattern but with moderate reliance on text and layout modalities as well\. IG, in contrast, suggests the opposite pattern, with high importance on the text modality and moderate importance on layout\. This is consistent with our observations in the qualitative analysis, where IG often distributed positive and negative contributions across text modalities\. Overall, we observe that the class\-wise importance scores align well with the dataset\-wide mean scores\.

On DocLayNet\([61](https://arxiv.org/html/2610.36929#bib.bib61)\), similar patterns are observed\. Compared to the RVL\-CDIP\[[50](https://arxiv.org/html/2610.36929#bib.bib50)\]results, IG indicates relatively larger importance on layout and image modalities\. RA suggests the model relies almost exclusively on the image modality\. OCC indicates reliance on both image and text, with variation among classes; for instance, forgovernment\_tenders, it suggests a relatively higher focus on the text modality compared tofinancial\_reportswhere the image modality dominates\.

On the CORD\([62](https://arxiv.org/html/2610.36929#bib.bib62)\)and FUNSD\[[51](https://arxiv.org/html/2610.36929#bib.bib51)\]datasets, OCC and IG demonstrate a similar pattern, both indicating negligible importance on the image modality and comparable importance on the text, position, and layout modalities\. On CORD\([62](https://arxiv.org/html/2610.36929#bib.bib62)\), RA also indicates a moderate reliance on the image modality, with notably high importance assigned to the image modality for theMENU\.ITEMSUBTOTALandMENU\.UNITPRICEentity types\. On DocVQA\([65](https://arxiv.org/html/2610.36929#bib.bib65)\), OCC and IG show a similar importance distribution for both start and end tokens of the predicted answer\. In contrast, RA indicates a particularly high reliance on the image modality for the start token\.

![Refer to caption](https://arxiv.org/html/2610.36929v1/images/results/token_cls/diagnostic_scalars_FUNSD_LayoutLMv3.png)
![Refer to caption](https://arxiv.org/html/2610.36929v1/images/results/token_cls/diagnostic_scalars_SROIE_LayoutLMv3.png)
![Refer to caption](https://arxiv.org/html/2610.36929v1/images/results/token_cls/diagnostic_scalars_CORD_LayoutLMv3.png)
![Refer to caption](https://arxiv.org/html/2610.36929v1/images/results/token_cls/diagnostic_scalars_Wild_Receipts_LayoutLMv3.png)

Figure 13:Spatial attribution locality scores computed over explanations generated by IG, OCC, and RA for the LayoutLMv3\[[2](https://arxiv.org/html/2610.36929#bib.bib2)\]model\. For each dataset, vertical locality, horizontal locality, and total spread are reported as a dataset\-wide mean \(left\) and as a per\-entity\-type mean for four randomly selected entity types \(right\)\.
##### 5\.5\.2Spatial Attribution Locality Analysis

In theSERtask, the large number of target entities per sample makes it impractical to analyze each individual sample\-target pair qualitatively\. Here, spatial attribution locality analysis provides a broader view of how the model distributes importance around each target entity type; for instance, whether the model focuses entirely on the entities themselves, their vertical or horizontal surrounding tokens, or whether its focus is spread out to irrelevant entities across the document\. In particular, we analyze the spatial locality of attributions by computing three quantities\.*Vertical locality*, the total attribution assigned to tokens in the same column as the target \(i\.e\., vertically aligned\):

Lv=∑i:\|xi−xt\|≤τxaiL\_\{v\}=\\sum\_\{i:\\,\|x\_\{i\}\-x\_\{t\}\|\\leq\\tau\_\{x\}\}a\_\{i\}\(7\)*Horizontal locality*, the total attribution assigned to tokens on the same row as the target \(i\.e\., horizontally aligned\):

Lh=∑i:\|yi−yt\|≤τyaiL\_\{h\}=\\sum\_\{i:\\,\|y\_\{i\}\-y\_\{t\}\|\\leq\\tau\_\{y\}\}a\_\{i\}\(8\)*Total spread*, the attribution\-weighted root mean squared distance of all tokens from the target:

S=∑iai​\(\(xi−xt\)2\+\(yi−yt\)2\)S=\\sqrt\{\\sum\_\{i\}a\_\{i\}\\left\(\(x\_\{i\}\-x\_\{t\}\)^\{2\}\+\(y\_\{i\}\-y\_\{t\}\)^\{2\}\\right\)\}\(9\)For all three quantities, attributions are first aggregated to the text level, clamped to non\-negative values, and normalized to sum to one\. Higher vertical and horizontal locality values indicate that the method focuses on tokens along the respective axes, while lower spread values indicate that attributions are spatially compact overall\. Both thresholdsτx\\tau\_\{x\}andτy\\tau\_\{y\}are set to a fixed ratio of0\.0250\.025relative to the total image width and height, respectively\.

Fig\.[13](https://arxiv.org/html/2610.36929#S5.F13)presents the spatial attribution locality scores for the LayoutLMv3\[[2](https://arxiv.org/html/2610.36929#bib.bib2)\]model, computed over explanations generated by OCC, IG, and RA\. For each of the fourSERdatasets, we report vertical locality, horizontal locality, and total spread as a dataset\-wide mean across all samples, as well as per\-entity\-type means for four randomly selected entity types\.

A number of interesting observations can be made from these results\. On both SROIE\([63](https://arxiv.org/html/2610.36929#bib.bib63)\)and CORD\([62](https://arxiv.org/html/2610.36929#bib.bib62)\), OCC and RA show a similar pattern, with notably larger horizontal locality than vertical locality\. This suggests that, when predicting entity types on these datasets, the model pays relatively greater attention to the target entity itself and to other words in its horizontal neighborhood\. In contrast, IG shows a considerably higher spread on all datasets, with no particular preference for horizontal or vertical locality\.

The label\-wise results further reveal how the attribution locality distribution varies across entity types\. For instance, on SROIE\([63](https://arxiv.org/html/2610.36929#bib.bib63)\), OCC shows high horizontal locality scores for theADDRESS,COMPANY, andDATEentity types, meaning that importance is primarily assigned to tokens in the same row as the target token\. On the other hand, on WildReceipts\([64](https://arxiv.org/html/2610.36929#bib.bib64)\), RA shows both high vertical and horizontal locality with negligible overall spread for all listed entity types\. This indicates that the RA attributions for these entity types focus almost exclusively on the target words themselves, with only minimal attention to their surrounding neighborhood\.

![Refer to caption](https://arxiv.org/html/2610.36929v1/images/results/seq_cls/RVL-CDIP_word_freq.png)\(a\)RVL\-CDIP\[[50](https://arxiv.org/html/2610.36929#bib.bib50)\]
![Refer to caption](https://arxiv.org/html/2610.36929v1/images/results/seq_cls/DocLayNet_word_freq.png)\(b\)DocLayNet\([61](https://arxiv.org/html/2610.36929#bib.bib61)\)
![Refer to caption](https://arxiv.org/html/2610.36929v1/images/results/token_cls/CORD_word_freq.png)\(c\)CORD\([62](https://arxiv.org/html/2610.36929#bib.bib62)\)
![Refer to caption](https://arxiv.org/html/2610.36929v1/images/results/token_cls/FUNSD_word_freq.png)\(d\)FUNSD\[[51](https://arxiv.org/html/2610.36929#bib.bib51)\]
![Refer to caption](https://arxiv.org/html/2610.36929v1/images/results/qa/DocVQA_word_freq.png)\(e\)DocVQA\([65](https://arxiv.org/html/2610.36929#bib.bib65)\)

Figure 14:Top\-kkword frequency analysis for the LayoutLMv3\[[2](https://arxiv.org/html/2610.36929#bib.bib2)\]model across five datasets\. For each dataset, the most frequently attributed words \(top 1% by attribution magnitude\) are shown for up to six randomly selected labels\. \(a–b\)CLStask, \(c–d\)SERtask, \(e\)QAtask\.
##### 5\.5\.3Top\-k Word Frequency Analysis

To analyze whether the model relies on specific textual patterns for its predictions, we analyze the most frequently attributed words across each dataset\. For each sample, we aggregate the attribution scores across all modalities at the word level and select the top 1% of word tokens by attribution magnitude\. We then compute the frequency of each word appearing in this top 1% across all samples in the dataset and report the most frequent words per class\. ForCLS, frequencies are reported per document class; forSER, per entity type; and forQA, separately for the start and end tokens of the predicted answer\. Fig\.[14](https://arxiv.org/html/2610.36929#S5.F14)presents the results for the LayoutLMv3\[[2](https://arxiv.org/html/2610.36929#bib.bib2)\]model on five datasets, showing the most frequently attributed words for up to six randomly selected labels per dataset\.

It is noticeable that across all datasets and tasks, stop words \(labeled asSTOP\) dominate the attributions\. While in some cases stop words may carry genuine structural or contextual cues relevant to the prediction \(e\.g\., to serve as contextual separators between entities\), their widespread and uniform dominance across all classes and entity types suggests that this may largely be driven by their high frequency in the data\. Interestingly, some task\-relevant words do emerge across the datasets\. For instance, on RVL\-CDIP\[[50](https://arxiv.org/html/2610.36929#bib.bib50)\], words such asMr\.,Dear, andDr\.appear frequently for the letter class, whileUniversity,Department, andVol\.are among the most attributed words for scientific publications\. Notably, words such asTobacco,Cigarette, andHealthyappear consistently across multiple classes, despite having no apparent relevance to the class labels themselves\. While their prevalence in the dataset is unsurprising given its tobacco industry origins, their high attribution across unrelated classes suggests the model may be relying on dataset\-specific artifacts rather than task\-relevant features\.

On DocLayNet\([61](https://arxiv.org/html/2610.36929#bib.bib61)\), various forms of the wordContractappear consistently for thegovernment\_tendersclass, which aligns well with the class semantics\. Forfinancial\_reports, the wordsDecemberand31\.appear with high frequency despite having no direct relevance to the class, alongside more plausible words such asCompanyandFinancial\. For the manuals class, specific abbreviations such asPDF,IVM, andAFPemerge as high\-frequency top\-kkwords, though whether these reflect genuine class\-relevant patterns or spurious artifacts requires closer inspection of the dataset\.

On FUNSD\[[51](https://arxiv.org/html/2610.36929#bib.bib51)\], words such asFAX,DATE,Account, andFacsimileappear consistently across all entity types\. On CORD\([62](https://arxiv.org/html/2610.36929#bib.bib62)\), words such asPCS,TOTAL, andSUBTOTALdominate the top\-kkattributions, all of which appear relevant to the receipt domain\. However, specific numbers and percentages such as50,000or10%also appear with high frequency for some entity types, which may reflect a bias in the model’s decision\-making driven by their prevalence in the training data\. On DocVQA\([65](https://arxiv.org/html/2610.36929#bib.bib65)\), the attributions for both start and end answer tokens are largely dominated by stop words\. However, the wordssourceandnumberalso appear frequently among the top\-kkattributed words\.

### 6Discussion

Explainable AI has received considerable attention in deep learning over the past decade; however, its extension to the multimodal document understanding domain has remained largely unexplored\. By providing a systematic evaluation framework and a rigorous benchmark across multiple models, methods, datasets, and evaluation metrics, this study takes a significant step forward in improving the explainability of multimodal document understanding models and highlighting the challenges associated with existing methods\. In the following, we discuss the key findings of this work, its limitations, broader impact, and future directions\.

#### 6\.1Key Findings

Top\-performing XAI methods\.Our results showed that the majority of XAI methods, particularly basic backpropagation methods, consistently underperformed across all tasks and evaluation criteria\. Among the 13 methods evaluated, only three from different families emerged as the strongest: OCC \(perturbation\-based\), RA \(attention\-based\), and IG \(gradient\-based\)\. However, when considering both quantitative and qualitative evaluation jointly, only OCC stood out as the most reliable overall\. Unlike IG, which tended to spread importance across the entire document and produce high complexity attribution maps, OCC produced concise and interpretable modality\-wise contributions\. Furthermore, RA can only attribute importance to the modalities explicitly represented as separate tokens in the model, a limitation that does not apply to OCC due to its model\-agnostic nature\.

Computational trade\-offs of methods\.Despite its strengths, OCC remains one of the most expensive methods to compute, since it requires perturbations over sliding windows, and therefore its computational cost scales proportionally to the size of the input\. In multi\-target scenarios, however, this overhead can be effectively reduced due to shared perturbations over targets\. IG, on the other hand, is expensive in both single\- and multi\-target scenarios, as even with our multi\-target adaptation, it only gained modest speedups of up to1\.51\.5–1\.7×1\.7\\times\. In this aspect, RA can be considered a particularly strong choice, as, despite its limitations, it requires no additional computation other than a forward pass for both single\- and multi\-target scenarios\.

Completeness of methods\.Another important consideration is the Completeness Axiom, as it allows us to evaluate the exact contribution of each feature towards the output\. However, neither OCC nor RA satisfy completeness\. IG does satisfy the Completeness Axiom, but as we observed in our experiments, its high complexity made the results difficult to interpret\. Another method that satisfies Completeness is KS; however, it showed considerably poorer performance on all other evaluation criteria in our experiments\. This is expected, since KS approximates Shapley values by fitting a surrogate model over a sampled subset of feature coalitions from a space of2n2^\{n\}possible coalitions\. In our setting,nncan easily grow large \(up to hundreds or thousands of features\) when accounting for separate text, position, layout, and image features, making reliable estimation of the Shapley values infeasible with a limited number of perturbation samples\. Increasing the number of perturbations may help, but it incurs impractical computational costs\. Note that these limitations apply equally to LIME\.

Metric reliability and agreement\.Our analysis of the evaluation metrics revealed two important findings\. First, for many XAI evaluation metrics, different methods showed statistically indistinguishable results when observed across datasets, models, and tasks\. Therefore, practitioners should exercise caution when ranking one method over another, and we recommend evaluating methods across multiple datasets and models before drawing conclusions about their suitability\. Second, we observed moderate to strong agreement between several faithfulness metrics, includingaopc,faith\-est,abpc,faith\-corr,suf, andsens\-n\. While each of these metrics measures a different aspect, their mutual agreement can help reinforce confidence in their results\. Conversely, neither Monotonicity \(mon\)\([28](https://arxiv.org/html/2610.36929#bib.bib28)\)nor Infidelity \(infid\)\([18](https://arxiv.org/html/2610.36929#bib.bib18)\)showed a positive correlation with any other metric, and upon deeper inspection, we found the results of both of these metrics to be unreliable for evaluating the methods\. The complexity metrics also showed moderate mutual correlation amongent\-comp,spar, ands\-comp, whileeff\-compshowed only negligible correlation with the other three\. This is expected, sinceeff\-compis computed over perturbed inputs, unlike the other complexity metrics which are computed directly on the generated explanations\.

Modality reliance across tasks and labels\.Our modality contribution analysis of the LayoutLMv3\[[2](https://arxiv.org/html/2610.36929#bib.bib2)\]model revealed variable reliance on individual modalities across tasks, datasets, and individual labels\. For instance, on classification tasks, OCC indicated a stronger reliance on the image modality, whereas on SER tasks, it indicated that text and layout were the primary contributors\. Within datasets, label\-level variation was also observed \(for example, higher contribution of text modality forgovernment\_tendersvs\. image forfinancial\_reports\)\. Such insights can help optimize, debug, and validate model behavior\. For example, our results indicate that the image modality contributes negligibly to the SER task\. Therefore, the image modality could potentially be removed as an additional input for this task to optimize inference costs\. On the other hand, if strong reliance on the image modality were expected for a particular task, modality contribution analysis could serve as a means of verifying whether the model is indeed utilizing it\.

Spatial attribution locality patterns\.Our spatial locality analysis of the LayoutLMv3\[[2](https://arxiv.org/html/2610.36929#bib.bib2)\]model also revealed several insights into which spatial regions the model attends to for different entity types\. For example, on the SROIE\([63](https://arxiv.org/html/2610.36929#bib.bib63)\)dataset, OCC indicated greater focus on the horizontal neighborhood of target entities, especially for entity typesADDRESS,DATE, andCOMPANY\. This aligns with our expectation, as these entities are typically horizontally spread out in a document, and therefore, the model’s focus on horizontally aligned tokens seems reasonable\. On the other hand, for many entity types, RA indicated focus only on the target token itself\. Such entity\-level insights can be particularly useful for detecting unexpected model behavior\. For instance, if a model were found to focus on spatially distant and irrelevant tokens for a given entity type, this could signal a potential spurious correlation that warrants further investigation\.

Word frequency and dataset biases\.Our top\-kkword frequency analysis revealed that stop words dominated the attributions across all tasks and datasets\. While in theSERandQAtasks, this is plausible as stop words frequently appear in the neighborhood of target entities and may carry contextual relevance, the dominance of stop words in theCLStask may reflect a frequency bias instead, as stop words should hold no class\-specific relevance\. Besides stop words, we also noted a mix of class\-relevant and potentially spurious words emerging across the datasets \(e\.g\., tobacco\-related terms across multiple classes in RVL\-CDIP\[[50](https://arxiv.org/html/2610.36929#bib.bib50)\], or specific dates forfinancial\_reportsin DocLayNet\([61](https://arxiv.org/html/2610.36929#bib.bib61)\)\), suggesting potential dataset biases that warrant deeper inspection\. Note that the word frequency analysis can be further strengthened when combined with the modality contribution analysis\. For instance, if the model relies heavily on text modality for a given class, the word frequency analysis can pinpoint which specific words drive the predictions\. Conversely, if the text modality receives negligible importance, the presence of spurious words among the top attributions is less concerning, as the model’s predictions are not primarily driven by text\.

Limited consensus across methods\.Despite the utility of the above analyses, it is important to note that the three top\-ranking methods showed limited consensus in their attributions, leading to different interpretations depending on the method used\. Therefore, to draw reliable insights from these analyses, one should ideally rely on a method with established faithfulness\. In our evaluation, OCC consistently ranked among the top methods on this criterion and can therefore serve as a reasonable choice, though this also highlights the need for developing more faithful domain\-specific XAI methods for multimodal document understanding\.

#### 6\.2Limitations

While our study presents a comprehensive evaluation of existing XAI methods in the context of multimodal document understanding, there are some limitations that warrant discussion\. First, we evaluate all methods and metrics only on four models: BERT\[[66](https://arxiv.org/html/2610.36929#bib.bib66)\], RoBERTa\[[67](https://arxiv.org/html/2610.36929#bib.bib67)\], LiLT\[[10](https://arxiv.org/html/2610.36929#bib.bib10)\], and LayoutLMv3\[[2](https://arxiv.org/html/2610.36929#bib.bib2)\], of which only two are document\-specific multimodal models and only one processes all four modalities\. This limitation arises primarily from the lack of open\-source model architectures and pretrained weights\. Several existing works that employ significantly different fusion strategies, such as DocFormer and TiLT, do not release their model weights, and reproducing them is non\-trivial, as each requires complex pretraining routines that are also closed source\. Secondly, all evaluated models are Transformer\-based, and our findings may not generalize to other model families\. That said, our framework is directly extensible to such architectures, as it treats the model as a black box, independent of the specific modalities over which attributions are computed and how they are aggregated for evaluation\.

#### 6\.3Broader Impact

Despite the limitations, our work makes several contributions that extend beyond the immediate scope of this study\. As part of this work, we release code for all XAI method and evaluation metric adaptations presented in this study, along with a unified evaluation pipeline that ensures consistent metric computation across criteria\. We believe this will lower the barrier for researchers and practitioners to apply and evaluate explainability methods at scale in both single\- and multi\-target settings\. Furthermore, while this work focuses on multimodal document understanding, the multimodal and multi\-target adaptations and evaluation suite are task\-agnostic and can be directly applied to other domains\. Overall, as multimodal models are increasingly deployed in high\-stakes sectors such as finance, legal, and healthcare, where transparency is often legally required, our framework can serve as both a benchmark for future research and a practical guide for evaluating the explainability of such models\.

#### 6\.4Future Work

While our results provide a first comprehensive picture, several directions remain open for future work\. First, while Occlusion emerged as the most faithful and interpretable method in our analysis, it does not satisfy completeness\. KernelSHAP, which does satisfy completeness, could serve as an alternative; however, it performed considerably worse in our setting due to the large multimodal feature space\. To address this, one direction worth exploring is hierarchical decomposition of Shapley values over nested coalition structures\[[71](https://arxiv.org/html/2610.36929#bib.bib71)\]\. Rather than computing Shapley values over the full set of individual features, such an approach would decompose the attribution process into multiple stages \(e\.g\., attributing importance to blocks, then lines, and finally individual words and word modalities\), reducingnnto a tractable number at each stage\. This could preserve completeness guarantees while simultaneously producing interpretable explanations at each level of the hierarchy\. Second, while our framework considers modality\-wise attributions, capturing cross\-modal interactions, as investigated in DIME\[[45](https://arxiv.org/html/2610.36929#bib.bib45)\], remains an open problem in the document domain\. Third, while we explored several attribution methods in this work, a fundamental limitation remains in that they revealwhichfeatures the model relies on but notwhy\. In this regard, exploring \(generative\) counterfactual explanations\[[72](https://arxiv.org/html/2610.36929#bib.bib72)\]for multimodal document understanding may be a promising direction for uncovering the underlying causal factors\. Finally, while we proposed multiple dataset\-wide analyses to obtain global insights, developing additional domain\-specific diagnostic approaches and dedicated visualization interfaces, especially for multi\-target tasks, would further extend the framework’s utility\.

### 7Conclusion

In this work, we presentedDocXEval, the first comprehensive framework for generating and evaluating explanations in multimodal document understanding\. Using our framework, we systematically evaluated 13 XAI methods across 3 tasks, 8 datasets, 4 models, and 17 evaluation metrics, and found that only a few methods perform consistently across all criteria, with Occlusion \(OCC\)\[[27](https://arxiv.org/html/2610.36929#bib.bib27)\]emerging as the most faithful and interpretable overall\. Our meta\-evaluation of metrics showed moderate to strong agreement among several faithfulness and complexity metrics, while identifying Monotonicity and Infidelity as unreliable\. Furthermore, through multiple dataset\-wide analyses, we uncovered several insights into model behavior, including variable modality reliance across tasks and labels, spatial attribution patterns around target entities, and potential dataset biases\. Overall, we believe our findings, together with the publicly released framework, will serve as a valuable reference for future research on XAI evaluation, especially in the multimodal document understanding context\.

##### Acknowledgements

We thank Adriano Lucieri for helpful discussions and feedback on this work\.

## References

- \(1\)Ferrando, J\.*et al\.*Krzhizhanovskaya, V\. V\.*et al\.*\(eds\)*Improving Accuracy and Speeding Up Document Image Classification Through Parallel Systems*\.\(eds Krzhizhanovskaya, V\. V\.*et al\.*\)*Computational Science – ICCS 2020*, 387–400 \(Springer International Publishing, Cham, 2020\)\.
- \(2\)Huang, Y\., Lv, T\., Cui, L\., Lu, Y\. & Wei, F\.*LayoutLMv3: Pre\-Training for Document AI with Unified Text and Image Masking*, MM ’22, 4083–4091 \(Association for Computing Machinery, New York, NY, USA, 2022\)\.URL[https://doi\.org/10\.1145/3503161\.3548112](https://doi.org/10.1145/3503161.3548112)\.Event\-place: Lisboa, Portugal\.
- \(3\)Xu, Y\.*et al\.*Gupta, R\., Liu, Y\., Tang, J\. & Prakash, B\. A\. \(eds\)*LayoutLM: pre\-training of text and layout for document image understanding*\.\(eds Gupta, R\., Liu, Y\., Tang, J\. & Prakash, B\. A\.\)*Proc\. 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining*, 1192–1200 \(virtual, 2020\)\.
- \(4\)Xu, Y\.*et al\.**LayoutLMv2: Multi\-modal Pre\-training for Visually\-rich Document Understanding*, 2579–2591 \(Association for Computational Linguistics \(ACL\), 2021\)\.\_eprint: 2012\.14740\.
- \(5\)Borchmann, L\.*et al\.*Vanschoren, J\. & Yeung, S\.\-K\. \(eds\)*DUE: end\-to\-end document understanding benchmark*\.\(eds Vanschoren, J\. & Yeung, S\.\-K\.\)*NeurIPS Datasets and Benchmarks*, Vol\. 1 \(virtual, 2021\)\.
- \(6\)Wang, W\.*et al\.*Document intelligence in the era of large language models: A survey \(2025\)\.URL[https://arxiv\.org/abs/2510\.13366](https://arxiv.org/abs/2510.13366)\.[arXiv:2510\.13366](https://arxiv.org/abs/2510.13366)\.
- \(7\)Shen, Z\.*et al\.*Lladós, J\., Lopresti, D\. & Uchida, S\. \(eds\)*LayoutParser: A Unified Toolkit for Deep Learning Based Document Image Analysis*\.\(eds Lladós, J\., Lopresti, D\. & Uchida, S\.\)*Document Analysis and Recognition – ICDAR 2021*, 131–146 \(Springer International Publishing, Cham, 2021\)\.
- \(8\)Powalski, R\.*et al\.*Lladós, J\., Lopresti, D\. & Uchida, S\. \(eds\)*Going Full\-TILT Boogie on Document Understanding with Text\-Image\-Layout Transformer*\.\(eds Lladós, J\., Lopresti, D\. & Uchida, S\.\)*Document Analysis and Recognition – ICDAR 2021*, 732–747 \(Springer International Publishing, Cham, 2021\)\.
- \(9\)Garncarek, L\.*et al\.*inLAMBERT: layout\-aware language modeling for information extraction\(eds Lladós, J\., Lopresti, D\. & Uchida, S\.\)*Document analysis and recognition \- ICDAR 2021*, Vol\. 12821 of*LNCS*532–547 \(Springer, Cham, 2021\)\.
- \(10\)Wang, J\., Jin, L\. & Ding, K\.Muresan, S\., Nakov, P\. & Villavicencio, A\. \(eds\)*LiLT: a simple yet effective language\-independent layout transformer for structured document understanding*\.\(eds Muresan, S\., Nakov, P\. & Villavicencio, A\.\)*ACL*, 7747–7757 \(Dublin, Ireland, 2022\)\.
- \(11\)Lamott, M\., Shakir, M\. A\., Ulges, A\., Weweler, Y\.\-N\. & Shafait, F\.SlimDoc: lightweight distillation of document transformer models\.*Ijdar*28, 457–473 \(2025\)\.
- \(12\)Bibal, A\., Lognoul, M\., De Streel, A\. & Frénay, B\.Legal requirements on explainability in machine learning\.*Artificial Intelligence and Law*29, 149–169 \(2021\)\.URL[https://www\.scopus\.com/pages/publications/85088816026](https://www.scopus.com/pages/publications/85088816026)\.
- \(13\)Perc, M\., Ozer, M\. & Hojnik, J\.Social and juristic challenges of artificial intelligence\.*Palgrave Communications*5, 61 \(2019\)\.URL[https://www\.nature\.com/articles/s41599\-019\-0278\-x](https://www.nature.com/articles/s41599-019-0278-x)\.Publisher: Palgrave\.
- \(14\)Lundberg, S\. M\., Erion, G\. G\. & Lee, S\.\-I\.Consistent Individualized Feature Attribution for Tree Ensembles \(2019\)\.\_eprint: 1802\.03888\.
- \(15\)Ribeiro, M\., Singh, S\. & Guestrin, C\.DeNero, J\., Finlayson, M\. & Reddy, S\. \(eds\)*“Why Should I Trust You?”: Explaining the Predictions of Any Classifier*\.\(eds DeNero, J\., Finlayson, M\. & Reddy, S\.\)*Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations*, 97–101 \(Association for Computational Linguistics, San Diego, California, 2016\)\.URL[https://aclanthology\.org/N16\-3020](https://aclanthology.org/N16-3020)\.ArXiv:1602\.04938 \[cs\]\.
- \(16\)Samek, W\., Wiegand, T\. & Müller, K\.\-R\.Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models \(2017\)\.URL[http://arxiv\.org/abs/1708\.08296](http://arxiv.org/abs/1708.08296)\.ArXiv:1708\.08296 \[cs\]\.
- \(17\)Adebayo, J\.*et al\.*Bengio, S\.*et al\.*\(eds\)*Sanity Checks for Saliency Maps*\.\(eds Bengio, S\.*et al\.*\)*Advances in Neural Information Processing Systems*, Vol\. 31 \(Curran Associates, Inc\., 2018\)\.URL[https://proceedings\.neurips\.cc/paper\_files/paper/2018/file/294a8ed24b1ad22ec2e7efea049b8737\-Paper\.pdf](https://proceedings.neurips.cc/paper_files/paper/2018/file/294a8ed24b1ad22ec2e7efea049b8737-Paper.pdf)\.
- \(18\)Yeh, C\.\-K\., Hsieh, C\.\-Y\., Suggala, A\., Inouye, D\. I\. & Ravikumar, P\. K\.Wallach, H\.*et al\.*\(eds\)*On the \(In\)fidelity and Sensitivity of Explanations*\.\(eds Wallach, H\.*et al\.*\)*Advances in Neural Information Processing Systems*, Vol\. 32 \(Curran Associates, Inc\., 2019\)\.URL[https://proceedings\.neurips\.cc/paper\_files/paper/2019/file/a7471fdc77b3435276507cc8f2dc2569\-Paper\.pdf](https://proceedings.neurips.cc/paper_files/paper/2019/file/a7471fdc77b3435276507cc8f2dc2569-Paper.pdf)\.
- \(19\)Vilone, G\. & Longo, L\.Explainable artificial intelligence: a systematic review\.*arXiv preprint arXiv:2006\.00093*\(2020\)\.URL[http://arxiv\.org/abs/2006\.00093](http://arxiv.org/abs/2006.00093)\.ArXiv:2006\.00093 \[cs\]\.
- \(20\)Ras, G\., Xie, N\., van Gerven, M\. & Doran, D\.Explainable Deep Learning: A Field Guide for the Uninitiated\.*J\. Artif\. Int\. Res\.*73, 329–397 \(2022\)\.URL[https://doi\.org/10\.1613/jair\.1\.13200](https://doi.org/10.1613/jair.1.13200)\.Place: El Segundo, CA, USA Publisher: AI Access Foundation\.
- \(21\)Danilevsky, M\.*et al\.**A survey of the state of explainable ai for natural language processing*, 447–459 \(Association for Computational Linguistics, 2020\)\.URL[http://dx\.doi\.org/10\.18653/v1/2020\.aacl\-main\.46](http://dx.doi.org/10.18653/v1/2020.aacl-main.46)\.ArXiv:2010\.00711 \[cs\]\.
- \(22\)Dembinsky, D\.*et al\.*Unifying vxai: a systematic review and framework for the evaluation of explainable ai\.*arXiv preprint arXiv:2506\.15408*\(2025\)\.
- \(23\)Appalaraju, S\., Jasani, B\., Kota, B\. U\., Xie, Y\. & Manmatha, R\.*DocFormer: end\-to\-end transformer for document understanding*, 993–1003 \(virtual, 2021\)\.
- \(24\)Lundberg, S\. M\. & Lee, S\.\-I\.*A Unified Approach to Interpreting Model Predictions*, NIPS’17, 4768–4777 \(Curran Associates Inc\., Red Hook, NY, USA, 2017\)\.Event\-place: Long Beach, California, USA\.
- \(25\)Klein, L\.*et al\.**Navigating the Maze of Explainable AI: A Systematic Approach to Evaluating Methods and Metrics*, Vol\. 37 \(Curran Associates, Inc\., 2024\)\.
- \(26\)Tomsett, R\., Harborne, D\., Chakraborty, S\., Gurram, P\. & Preece, A\.Sanity Checks for Saliency Metrics\.*Proceedings of the AAAI Conference on Artificial Intelligence*34, 6021–6029 \(2020\)\.
- \(27\)Zeiler, M\. D\. & Fergus, R\.Fleet, D\., Pajdla, T\., Schiele, B\. & Tuytelaars, T\. \(eds\)*Visualizing and Understanding Convolutional Networks*\.\(eds Fleet, D\., Pajdla, T\., Schiele, B\. & Tuytelaars, T\.\)*Computer Vision – ECCV 2014*, 818–833 \(Springer International Publishing, Cham, 2014\)\.
- \(28\)Arya, V\.*et al\.*One explanation does not fit all: A toolkit and taxonomy of ai explainability techniques \(2019\)\.URL[https://arxiv\.org/abs/1909\.03012](https://arxiv.org/abs/1909.03012)\.[arXiv:1909\.03012](https://arxiv.org/abs/1909.03012)\.
- \(29\)Wang, D\.*et al\.*Ku, L\.\-W\., Martins, A\. & Srikumar, V\. \(eds\)*DocLLM: A layout\-aware generative language model for multimodal document understanding*\.\(eds Ku, L\.\-W\., Martins, A\. & Srikumar, V\.\)*Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, 8529–8548 \(Association for Computational Linguistics, Bangkok, Thailand, 2024\)\.URL[https://aclanthology\.org/2024\.acl\-long\.463/](https://aclanthology.org/2024.acl-long.463/)\.
- \(30\)Fujitake, M\.Calzolari, N\.*et al\.*\(eds\)*LayoutLLM: Large language model instruction tuning for visually rich document understanding*\.\(eds Calzolari, N\.*et al\.*\)*Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation \(LREC\-COLING 2024\)*, 10219–10224 \(ELRA and ICCL, Torino, Italia, 2024\)\.URL[https://aclanthology\.org/2024\.lrec\-main\.892/](https://aclanthology.org/2024.lrec-main.892/)\.
- \(31\)Vaswani, A\.*et al\.*Guyon, I\.*et al\.*\(eds\)*Attention is All you Need*\.\(eds Guyon, I\.*et al\.*\)*Advances in Neural Information Processing Systems*, Vol\. 30 \(Curran Associates, Inc\., 2017\)\.URL[https://proceedings\.neurips\.cc/paper\_files/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa\-Paper\.pdf](https://proceedings.neurips.cc/paper_files/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf)\.
- \(32\)Luo, C\., Cheng, C\., Zheng, Q\. & Yao, C\.Geolayoutlm: Geometric pre\-training for visual information extraction \(2023\)\.URL[https://arxiv\.org/abs/2304\.10759](https://arxiv.org/abs/2304.10759)\.[arXiv:2304\.10759](https://arxiv.org/abs/2304.10759)\.
- \(33\)Simonyan, K\., Vedaldi, A\. & Zisserman, A\.Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps\.*CoRR*abs/1312\.6034\(2013\)\.URL[https://api\.semanticscholar\.org/CorpusID:1450294](https://api.semanticscholar.org/CorpusID:1450294)\.
- \(34\)Springenberg, J\. T\., Dosovitskiy, A\., Brox, T\. & Riedmiller, M\.*Striving for Simplicity: The All Convolutional Net*\(2015\)\.URL[http://lmb\.informatik\.uni\-freiburg\.de/Publications/2015/DB15a](http://lmb.informatik.uni-freiburg.de/Publications/2015/DB15a)\.
- \(35\)Sundararajan, M\., Taly, A\. & Yan, Q\.*Axiomatic attribution for deep networks*, 3319–3328 \(PMLR, 2017\)\.
- \(36\)Shrikumar, A\., Greenside, P\. & Kundaje, A\.*Learning Important Features through Propagating Activation Differences*, ICML’17, 3145–3153 \(JMLR\.org, Sydney, NSW, Australia, 2017\)\.
- \(37\)Saifullah, S\., Agne, S\., Dengel, A\. & Ahmed, S\.DocVCE: diffusion\-based visual counterfactual explanations for document image classification \(2025\)\.URL[http://arxiv\.org/abs/2508\.04233](http://arxiv.org/abs/2508.04233)\.ArXiv:2508\.04233 \[cs\]\.
- \(38\)Saifullah, S\., Agne, S\., Dengel, A\. & Ahmed, S\.DocXclassifier: towards a robust and interpretable deep neural network for document image classification\.*International Journal on Document Analysis and Recognition \(IJDAR\)*27, 447–473 \(2024\)\.URL[https://link\.springer\.com/10\.1007/s10032\-024\-00483\-w](https://link.springer.com/10.1007/s10032-024-00483-w)\.
- \(39\)Pouliquen, G\., Chiron, G\., Chazalon, J\., Géraud, T\. & Awal, A\. M\.Weakly supervised training for hologram verification in identity documents \(2024\)\.URL[https://arxiv\.org/abs/2404\.17253](https://arxiv.org/abs/2404.17253)\.[arXiv:2404\.17253](https://arxiv.org/abs/2404.17253)\.
- \(40\)Brini, I\., Mehri, M\., Ingold, R\. & Essoukri Ben Amara, N\.Nguyen, N\. T\., Manolopoulos, Y\., Chbeir, R\., Kozierkiewicz, A\. & Trawiński, B\. \(eds\)*An End\-to\-End Framework for Evaluating Explainable D<b\></b\>eep Models: Application to Historical Document Image Segmentation*\.\(eds Nguyen, N\. T\., Manolopoulos, Y\., Chbeir, R\., Kozierkiewicz, A\. & Trawiński, B\.\)*Computational Collective Intelligence*, 106–119 \(Springer International Publishing, Cham, 2022\)\.
- \(41\)Souibgui, M\. A\.*et al\.**DocVXQA: context\-aware visual explanations for document question answering*\(2025\)\.URL[https://openreview\.net/forum?id=wex0vL4c2Y](https://openreview.net/forum?id=wex0vL4c2Y)\.
- \(42\)Saifullah, Agne, S\., Dengel, A\. & Ahmed, S\.The reality of high performing deep learning models: a case study on document image classification\.*IEEE Access*12, 103537–103564 \(2024\)\.URL[https://ieeexplore\.ieee\.org/abstract/document/10591981](https://ieeexplore.ieee.org/abstract/document/10591981)\.
- \(43\)Saifullah, S\., Agne, S\., Dengel, A\. & Ahmed, S\.Barney Smith, E\. H\., Liwicki, M\. & Peng, L\. \(eds\)*DocXplain: a novel model\-agnostic explainability method for document image classification*\.\(eds Barney Smith, E\. H\., Liwicki, M\. & Peng, L\.\)*Document Analysis and Recognition \- ICDAR 2024*, 103–123 \(Springer Nature Switzerland, Cham, 2024\)\.
- \(44\)Goyal, Y\., Mohapatra, A\., Parikh, D\. & Batra, D\.Towards Transparent AI Systems: Interpreting Visual Question Answering Models \(2016\)\.URL[http://arxiv\.org/abs/1608\.08974](http://arxiv.org/abs/1608.08974)\.ArXiv:1608\.08974 \[cs\]\.
- \(45\)Lyu, Y\., Liang, P\. P\., Deng, Z\., Salakhutdinov, R\. & Morency, L\.\-P\.DIME: fine\-grained interpretations of multimodal models via disentangled local explanations \(2022\)\.URL[http://arxiv\.org/abs/2203\.02013](http://arxiv.org/abs/2203.02013)\.ArXiv:2203\.02013 \[cs\]\.
- \(46\)Aflalo, E\.*et al\.*VL\-InterpreT: An Interactive Visualization Tool for Interpreting Vision\-Language Transformers \(2022\)\.URL[http://arxiv\.org/abs/2203\.17247](http://arxiv.org/abs/2203.17247)\.ArXiv:2203\.17247 \[cs\]\.
- \(47\)Parcalabescu, L\. & Frank, A\.*MM\-SHAP: A Performance\-agnostic Metric for Measuring Multimodal Contributions in Vision and Language Models & Tasks*, 4032–4059 \(2023\)\.URL[http://arxiv\.org/abs/2212\.08158](http://arxiv.org/abs/2212.08158)\.ArXiv:2212\.08158 \[cs\]\.
- \(48\)Wang, X\.*et al\.*M2Lens: Visualizing and Explaining Multimodal Models for Sentiment Analysis\.*IEEE Transactions on Visualization and Computer Graphics*28, 802–812 \(2022\)\.URL[http://arxiv\.org/abs/2107\.08264](http://arxiv.org/abs/2107.08264)\.ArXiv:2107\.08264 \[cs\]\.
- \(49\)Hessel, J\. & Lee, L\.Webber, B\., Cohn, T\., He, Y\. & Liu, Y\. \(eds\)*Does my multimodal model learn cross\-modal interactions? It’s harder to tell than you might think\!*\(eds Webber, B\., Cohn, T\., He, Y\. & Liu, Y\.\)*Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing \(EMNLP\)*, 861–877 \(Association for Computational Linguistics, Online, 2020\)\.URL[https://aclanthology\.org/2020\.emnlp\-main\.62/](https://aclanthology.org/2020.emnlp-main.62/)\.
- \(50\)Harley, A\. W\., Ufkes, A\. & Derpanis, K\. G\.*Evaluation of deep convolutional nets for document image classification and retrieval*, 991–995 \(2015\)\.
- \(51\)Jaume, G\., Kemal Ekenel, H\. & Thiran, J\.\-P\.*Funsd: A dataset for form understanding in noisy scanned documents*, Vol\. 2, 1–6 \(2019\)\.
- \(52\)Hase, P\., Xie, H\. & Bansal, M\.*The Out\-of\-Distribution Problem in Explainability and Search Methods for Feature Importance Explanations*, Vol\. 34, 3650–3666 \(Curran Associates, Inc\., 2021\)\.URL[https://papers\.neurips\.cc/paper\_files/paper/2021/hash/1def1713ebf17722cbe300cfc1c88558\-Abstract\.html](https://papers.neurips.cc/paper_files/paper/2021/hash/1def1713ebf17722cbe300cfc1c88558-Abstract.html)\.
- \(53\)phi Nguyen, A\. & Martínez, M\. R\.On quantitative aspects of model interpretability \(2020\)\.URL[https://arxiv\.org/abs/2007\.07584](https://arxiv.org/abs/2007.07584)\.[arXiv:2007\.07584](https://arxiv.org/abs/2007.07584)\.
- \(54\)Samek, W\., Binder, A\., Montavon, G\., Lapuschkin, S\. & Müller, K\.\-R\.Evaluating the Visualization of What a Deep Neural Network Has Learned\.*IEEE Transactions on Neural Networks and Learning Systems*28, 2660–2673 \(2017\)\.
- \(55\)DeYoung, J\.*et al\.*Jurafsky, D\., Chai, J\., Schluter, N\. & Tetreault, J\. \(eds\)*ERASER: A benchmark to evaluate rationalized NLP models*\.\(eds Jurafsky, D\., Chai, J\., Schluter, N\. & Tetreault, J\.\)*Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics*, 4443–4458 \(Association for Computational Linguistics, Online, 2020\)\.URL[https://aclanthology\.org/2020\.acl\-main\.408/](https://aclanthology.org/2020.acl-main.408/)\.
- \(56\)Ancona, M\., Ceolini, E\., Öztireli, C\. & Gross, M\.Towards better understanding of gradient\-based attribution methods for deep neural networks \(2018\)\.URL[https://arxiv\.org/abs/1711\.06104](https://arxiv.org/abs/1711.06104)\.[arXiv:1711\.06104](https://arxiv.org/abs/1711.06104)\.
- \(57\)Alvarez Melis, D\. & Jaakkola, T\.*Towards Robust Interpretability with Self\-Explaining Neural Networks*, Vol\. 31 \(Curran Associates, Inc\., 2018\)\.URL[https://proceedings\.neurips\.cc/paper\_files/paper/2018/hash/3e9f0fc9b2f89e043bc6233994dfcf76\-Abstract\.html](https://proceedings.neurips.cc/paper_files/paper/2018/hash/3e9f0fc9b2f89e043bc6233994dfcf76-Abstract.html)\.
- \(58\)Bhatt, U\., Weller, A\. & Moura, J\. M\. F\.Evaluating and aggregating feature\-based model explanations \(2020\)\.URL[https://arxiv\.org/abs/2005\.00631](https://arxiv.org/abs/2005.00631)\.[arXiv:2005\.00631](https://arxiv.org/abs/2005.00631)\.
- \(59\)Chalasani, P\., Chen, J\., Chowdhury, A\. R\., Jha, S\. & Wu, X\.Concise explanations of neural networks using adversarial training \(2020\)\.URL[https://arxiv\.org/abs/1810\.06583](https://arxiv.org/abs/1810.06583)\.[arXiv:1810\.06583](https://arxiv.org/abs/1810.06583)\.
- \(60\)Afzal, M\. Z\.*et al\.**Deepdocclassifier: Document classification with deep Convolutional Neural Network*, 1111–1115 \(2015\)\.
- \(61\)Pfitzmann, B\., Auer, C\., Dolfi, M\., Nassar, A\. S\. & Staar, P\.*DocLayNet: A Large Human\-Annotated Dataset for Document\-Layout Segmentation*, KDD ’22 \(ACM, 2022\)\.URL[http://dx\.doi\.org/10\.1145/3534678\.3539043](http://dx.doi.org/10.1145/3534678.3539043)\.
- \(62\)Park, S\.*et al\.**CORD: a consolidated receipt dataset for post\-OCR parsing*\(Vancouver, Canada, 2019\)\.
- \(63\)Huang, Z\.*et al\.**ICDAR2019 competition on scanned receipt OCR and information extraction*, 1516–1520 \(Sydney, Australia, 2019\)\.
- \(64\)Sun, H\., Kuang, Z\., Yue, X\., Lin, C\. & Zhang, W\.Spatial dual\-modality graph reasoning for key information extraction\.*ArXiv*abs/2103\.14470\(2021\)\.
- \(65\)Mathew, M\., Karatzas, D\. & Jawahar, C\. V\.*DocVQA: a dataset for VQA on document images*\(Waikoloa, HI, 2021\)\.
- \(66\)Devlin, J\., Chang, M\.\-W\., Lee, K\. & Toutanova, K\.Burstein, J\., Doran, C\. & Solorio, T\. \(eds\)*BERT: Pre\-training of Deep Bidirectional Transformers for Language Understanding*\.\(eds Burstein, J\., Doran, C\. & Solorio, T\.\)*Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 \(Long and Short Papers\)*, 4171–4186 \(Association for Computational Linguistics, Minneapolis, Minnesota, 2019\)\.URL[https://aclanthology\.org/N19\-1423](https://aclanthology.org/N19-1423)\.
- \(67\)Liu, Y\.*et al\.*Roberta: a robustly optimized bert pretraining approach\.*arXiv Preprint arXiv:1907\.11692*\(2019\)\.
- \(68\)Abnar, S\. & Zuidema, W\.Jurafsky, D\., Chai, J\., Schluter, N\. & Tetreault, J\. \(eds\)*Quantifying Attention Flow in Transformers*\.\(eds Jurafsky, D\., Chai, J\., Schluter, N\. & Tetreault, J\.\)*Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics*, 4190–4197 \(Association for Computational Linguistics, Online, 2020\)\.URL[https://aclanthology\.org/2020\.acl\-main\.385](https://aclanthology.org/2020.acl-main.385)\.
- \(69\)Benavoli, A\., Corani, G\. & Mangili, F\.Should we really use post\-hoc tests based on mean\-ranks?*Journal of Machine Learning Research*17, 1–10 \(2016\)\.URL[http://jmlr\.org/papers/v17/benavoli16a\.html](http://jmlr.org/papers/v17/benavoli16a.html)\.
- \(70\)Demšar, J\.Statistical comparisons of classifiers over multiple data sets\.*Journal of Machine Learning Research*7, 1–30 \(2006\)\.URL[http://jmlr\.org/papers/v7/demsar06a\.html](http://jmlr.org/papers/v7/demsar06a.html)\.
- \(71\)Owen, G\.*Values of Games with a Priori Unions*, 76–88 \(Springer Berlin Heidelberg, 1977\)\.URL[http://dx\.doi\.org/10\.1007/978\-3\-642\-45494\-3\_7](http://dx.doi.org/10.1007/978-3-642-45494-3_7)\.
- \(72\)Verma, S\., Dickerson, J\. P\. & Hines, K\. E\.Counterfactual Explanations for Machine Learning: A Review\.*ArXiv*abs/2010\.10596\(2020\)\.URL[https://api\.semanticscholar\.org/CorpusID:224818450](https://api.semanticscholar.org/CorpusID:224818450)\.

### 8Metric Descriptions and Adaptations

In this section, we provide detailed descriptions of the evaluation metrics used in this work\. Unless otherwise stated, all metrics are computed over group\-aggregated attribution scores, treating each feature group independently of its originating modality\. For perturbation\-based metrics, the perturbation order is determined by ranking all feature groups jointly across modalities, and the corresponding features are then perturbed using their modality\-specific segmentation masks and reference baselines\. Note that for all perturbation\-based metrics, feature groups associated with special tokens such as \[CLS\], \[PAD\], or \[SEP\] are excluded from perturbation\.

#### 8\.1Axiomatic Metrics

Completeness\.Completeness \(compl\)\([35](https://arxiv.org/html/2610.36929#bib.bib35)\)is an axiomatic property of an explanation method that measures whether the attributions add up to the difference between the model output at the input and at the baseline:

compl​\(x\)=\|∑ℳ∑i𝐚iℳ−\(f⁡\(x\)−f⁡\(x′\)\)\|\\textsc\{compl\}\(x\)=\\left\|\\sum\_\{\\mathcal\{M\}\}\\sum\_\{i\}\\mathbf\{a\}\_\{i\}^\{\\mathcal\{M\}\}\-\\left\(f\(x\)\-f\(x^\{\\prime\}\)\\right\)\\right\|\(10\)where𝐚iℳ\\mathbf\{a\}\_\{i\}^\{\\mathcal\{M\}\}are the raw attribution scores for featureiiin modalityℳ\\mathcal\{M\},f⁡\(x\)f\(x\)is the model output at the input, andf⁡\(x′\)f\(x^\{\\prime\}\)is the model output at the baseline\. A score of zero indicates perfect completeness, and some methods, like Integrated Gradients \(IG\)\[[35](https://arxiv.org/html/2610.36929#bib.bib35)\], satisfy completeness by construction\. Unlike other metrics, Completeness \(compl\)\([35](https://arxiv.org/html/2610.36929#bib.bib35)\)is computed on the raw attribution scores instead of group\-aggregated scores, as the axiom requires the exact sum over all individual attributions\. For multi\-target tasks, we adapt the metric to efficiently compute the difference between the model output at the input and at the baseline over allTTtargets in a single step\.

Non\-Sensitivity\.Non\-Sensitivity \(non\-sens\)\([53](https://arxiv.org/html/2610.36929#bib.bib53)\)is another axiomatic property that determines whether features with zero attribution have a negligible influence on the model output\. Formally, letA0⊂\{1,…,N\}A\_\{0\}\\subset\\\{1,\\ldots,N\\\}denote the subset of feature indices with zero attribution, i\.e\.,ai=0a\_\{i\}=0\. Further, letX0=\{i∈\{1,…,N\}∣𝔼⁡\(ℓ⁡\(y∗,fi\)\)=0\}X\_\{0\}=\\\{i\\in\\\{1,\\ldots,N\\\}\\mid\\mathbb\{E\}\(\\ell\(y^\{\*\},f\_\{i\}\)\)=0\\\}denote the subset of features that the modelffis not functionally dependent on\. The non\-sensitivity score is then defined as\|A0​△​X0\|\|A\_\{0\}\\triangle X\_\{0\}\|, where\|⋅\|\|\\cdot\|is the cardinality of a set and△\\triangledenotes the symmetric difference\. In our adaptation, we computenon\-sensover the unified feature space using group\-aggregated attribution scores\{ak\}k=1K\\\{a\_\{k\}\\\}\_\{k=1\}^\{K\}and instantiate the otherwise abstract loss functionℓ\\ellas the relative variance of the model output under perturbation of each feature groupkk:

vk=1N​∑n=1N\(f⁡\(xk′\(n\)\)−f⁡\(x\)\)2f​\(x\)2v\_\{k\}=\\frac\{\\frac\{1\}\{N\}\\sum\_\{n=1\}^\{N\}\\left\(f\(x\_\{k\}^\{\\prime\(n\)\}\)\-f\(x\)\\right\)^\{2\}\}\{f\(x\)^\{2\}\}\(11\)wheref⁡\(xk′\(n\)\)f\(x\_\{k\}^\{\\prime\(n\)\}\)is the model output with thekk\-th feature group perturbed andf⁡\(x\)f\(x\)is the model output on the unperturbed input\.NNdenotes the number of perturbations per feature group\. When using a random baseline function,N\>1N\>1is necessary to obtain a stable variance estimate; however, with a fixed reference baseline, as in our case, each perturbation is deterministic, and therefore we useN=1N=1\. We can then computeA0=\{k:\|ak\|<ϵa\}A\_\{0\}=\\\{k:\|a\_\{k\}\|<\\epsilon\_\{a\}\\\}andX0=\{k:vk<ϵv\}X\_\{0\}=\\\{k:v\_\{k\}<\\epsilon\_\{v\}\\\}, whereϵa\\epsilon\_\{a\}andϵv\\epsilon\_\{v\}are thresholds that define near\-zero values\. Furthermore, as the number of features can vary drastically across multimodal sequential inputs, we normalize the non\-sensitivity score by the number of features to ensure comparability across setups:

non\-sens​\(x\)=\|A0​△​X0\|K\\textsc\{non\-sens\}\(x\)=\\frac\{\|A\_\{0\}\\triangle X\_\{0\}\|\}\{K\}\(12\)whereKKis the total number of feature groups\. Lowernon\-sensvalues indicate better alignment between zero\-attribution features and features that do not affect the model output\. Note thatnon\-sensis computed jointly withmon\-corr\(described in Section[8\.3](https://arxiv.org/html/2610.36929#S8.SS3)\), as both metrics share the same per\-feature perturbation results\. Furthermore, to reduce computational cost, we apply chunking, where rather than perturbing each feature individually, a fixed percentage of feature groups is perturbed together in each step\. For multi\-target tasks, we adapt the metric to efficiently compute the relative variancesvkv\_\{k\}over allTTtargets of a single sample simultaneously\.

#### 8\.2Complexity Metrics

Complexity metrics assess how easily a user can interpret the explanations, with simpler explanations that concentrate importance on fewer features being generally preferred\. We adapt all complexity metrics to be computed over the group\-aggregated attribution scores\. For complexity metrics that do not require a model forward pass, multi\-target adaptations are not necessary; instead, these metrics are computed independently for each sample\-target pair\.

Effective Complexity\.Effective Complexity \(eff\-comp\)\([53](https://arxiv.org/html/2610.36929#bib.bib53)\)measures the minimum number of top\-attributed features sufficient to influence the model output\. Formally, leta\(i\)a^\{\(i\)\}be the attributions ordered increasingly by their absolute value, andx\(i\)x^\{\(i\)\}the corresponding features\. LetMk=\{x\(N−k\),…,x\(N\)\}M\_\{k\}=\\\{x^\{\(N\-k\)\},\\ldots,x^\{\(N\)\}\\\}be the set of topkkfeatures\. The effective complexity is defined as:

k∗=argmink∈\{1,…,N\}\|Mk\|s\.t\.𝔼⁡\(ℓ⁡\(y∗,f−Mk\)\|𝐱Mk∗\)<ϵk^\{\*\}=\\operatorname\*\{argmin\}\_\{k\\in\\\{1,\\ldots,N\\\}\}\|M\_\{k\}\|\\quad\\text\{s\.t\.\}\\quad\\mathbb\{E\}\(\\ell\(y^\{\*\},f\_\{\-M\_\{k\}\}\)\|\\mathbf\{x\}^\{\*\}\_\{M\_\{k\}\}\)<\\epsilonwheref−Mkf\_\{\-M\_\{k\}\}is the restriction of the modelffto the non\-important features, given fixed values for the important features inMkM\_\{k\}\. In our adaptation,eff\-compfollows a similar perturbation strategy tonon\-sensand instantiates loss as the relative variance of the model output under perturbation of each feature groupkk\(see Eq\.[11](https://arxiv.org/html/2610.36929#S8.E11)\)\. To findMkM\_\{k\}, we remove feature groups cumulatively in ascending order of their absolute attribution scores\{ak\}k=1K\\\{a\_\{k\}\\\}\_\{k=1\}^\{K\}and find the indexk∗k^\{\*\}at which the relative variance exceeds a thresholdϵv\\epsilon\_\{v\}\. The normalized effective complexity is the ratio of top\-k features to the total number of features:

eff\-comp​\(x\)=K−k∗K\\textsc\{eff\-comp\}\(x\)=\\frac\{K\-k^\{\*\}\}\{K\}\(13\)Lowereff\-compvalues indicate that the explanation concentrates importance on fewer important features\. To reduce the computational costs, we also apply chunking to computeeff\-comp, where a fixed percentage of feature groups are perturbed together in each step\. Furthermore, sinceeff\-comprequires ordered perturbation, multi\-target adaptation is not possible; the metric is therefore computed independently for each sample\-target pair\.

Entropy\-based Complexity\.Entropy\-based Complexity \(ent\-comp\)\([58](https://arxiv.org/html/2610.36929#bib.bib58)\)measures the total entropy of the attribution distribution over feature groups\. In our adaptation, we first compute the normalized absolute group\-aggregated attribution scores:

pk=\|ak\|∑k\|ak\|p\_\{k\}=\\frac\{\|a\_\{k\}\|\}\{\\sum\_\{k\}\|a\_\{k\}\|\}We then compute normalized entropy over feature groups to keep the entropy scores comparable across samples of varying feature sets:

ent\-comp​\(x\)=−∑kpklogpklog⁡K\\textsc\{ent\-comp\}\(x\)=\\frac\{\-\\sum\_\{k\}p\_\{k\}\\log p\_\{k\}\}\{\\log K\}whereKKis the total number of feature groups\. A value near 0 indicates that the attribution mass is concentrated on very few feature groups, while a value near 1 indicates a near\-uniform spread\.

Sundararajan Complexity\.Sundararajan Complexity \(s\-comp\)\([35](https://arxiv.org/html/2610.36929#bib.bib35)\)measures the fraction of features with non\-negligible attribution\. In our adaptation, we first compute the normalized absolute group\-aggregated attribution scores:

a^k=\|ak\|∑k′\|ak′\|\\hat\{a\}\_\{k\}=\\frac\{\|a\_\{k\}\|\}\{\\sum\_\{k^\{\\prime\}\}\|a\_\{k^\{\\prime\}\}\|\}\(14\)The complexity score is then defined as the fraction of feature groups whose normalized attribution exceeds a thresholdϵ\\epsilon:

s\-comp\(x\)=\|\{k:a^k\>ϵ\}\|K\\textsc\{s\-comp\}\(x\)=\\frac\{\|\\\{k:\\hat\{a\}\_\{k\}\>\\epsilon\\\}\|\}\{K\}\(15\)whereKKis the total number of feature groups\. Lower values indicate sparser, more concise explanations\.

Sparseness\.Sparseness \(spar\)\([59](https://arxiv.org/html/2610.36929#bib.bib59)\)measures the concentration of attribution scores using the Gini index\. In our adaptation, we compute the Gini index over the absolute group\-aggregated attribution scores:

spar​\(x\)=G⁡\(\{\|a1\|,…,\|aK\|\}\)\\textsc\{spar\}\(x\)=G\(\\\{\|a\_\{1\}\|,\\ldots,\|a\_\{K\}\|\\\}\)\(16\)whereKKis the total number of feature groups\. Higher values indicate sparser attributions, meaning the explanation concentrates importance on fewer features\.

#### 8\.3Faithfulness Metrics

Faithfulness metrics assess whether the attributions accurately reflect the model’s decision\-making process\.

Area Over the Perturbation Curve\.Area Over Perturbation Curve \(aopc\)\([54](https://arxiv.org/html/2610.36929#bib.bib54)\)measures the average change in model output as features are progressively removed in order of their attributed importance\. Given an inputxxand an ordered sequence of feature removals, the MoRF \(Most Relevant First\) perturbation curve is defined recursively asx\(0\)=xx^\{\(0\)\}=xandx\(k\)=g⁡\(x\(k−1\),rk\),∀1≤k≤Lx^\{\(k\)\}=g\(x^\{\(k\-1\)\},r\_\{k\}\),\\forall\\;1\\leq k\\leq L, whereggindicates removal of the feature at locationrkr\_\{k\}\. The AOPC metric is then computed as:

aopc=1L\+1​⟨∑k=0Lf⁡\(x\(0\)\)−f⁡\(x\(k\)\)⟩p⁡\(x\)\\textsc\{aopc\}=\\frac\{1\}\{L\+1\}\\left\\langle\\sum\_\{k=0\}^\{L\}f\(x^\{\(0\)\}\)\-f\(x^\{\(k\)\}\)\\right\\rangle\_\{p\(x\)\}\(17\)whereLLis the total number of removal steps and⟨⋅⟩p⁡\(x\)\\langle\\cdot\\rangle\_\{p\(x\)\}denotes the mean over the dataset\. A higheraopcMoRF\\textsc\{aopc\}\_\{\\text\{MoRF\}\}indicates that the most important features, as ranked by the attribution, have a larger impact on the model output, suggesting more faithful explanations\. In our adaptation, feature groups are removed cumulatively based on the perturbation order determined in the aggregated feature space, and the removal is performed by replacing the corresponding features with the modality\-wise reference baselines\. Furthermore, to ensure comparability across samples with varying numbers of feature groups, we remove features in fixed\-size chunks, where each chunk contains an equal percentage of features, resulting in the same number of removal stepsLLregardless of the feature count\. In practice, theaopccurve is computed in three orderings: descending \(MoRF\), ascending \(LeRF\), and random \(averaged overnnrandom permutations\)\. Sinceaopcrequires ordered cumulative perturbation and the perturbation order differs for each target, efficient multi\-target computation is not possible\.

Area Between the Perturbation Curves\.Area Between Perturbation Curves \(abpc\)\([54](https://arxiv.org/html/2610.36929#bib.bib54)\)measures the area between the descending \(MoRF\) and ascending \(LeRF\) perturbation curves:

abpc=1L\+1​⟨∑k=0Lf⁡\(xLeRF\(k\)\)−f⁡\(xMoRF\(k\)\)⟩p⁡\(x\)\\textsc\{abpc\}=\\frac\{1\}\{L\+1\}\\left\\langle\\sum\_\{k=0\}^\{L\}f\(x\_\{\\text\{LeRF\}\}^\{\(k\)\}\)\-f\(x\_\{\\text\{MoRF\}\}^\{\(k\)\}\)\\right\\rangle\_\{p\(x\)\}\(18\)A higherabpcindicates that removing high\-importance features degrades model performance quickly, while removing low\-importance features has comparatively little effect\. In our implementation, we derive theabpcscore directly from theaopccomputation and report scores at full feature removalLL\.

Sufficiency\.Sufficiency \(suf\)\([55](https://arxiv.org/html/2610.36929#bib.bib55)\)measures whether the top\-attributed features alone are sufficient to retain the model’s prediction\. In the original formulation\([55](https://arxiv.org/html/2610.36929#bib.bib55)\), sufficiency is computed by retaining only the top\-ranked features and measuring the change in model output\. In our adaptation, we compute sufficiency similarly toaopcbut with an inverse perturbation direction: starting from the reference baseline, features are progressively added back in descending order of their attribution scores, and the area under the resulting perturbation curve is computed\. Lowersufscores indicate better faithfulness\.

Infidelity\.Infidelity \(infid\)\([18](https://arxiv.org/html/2610.36929#bib.bib18)\)measures the discrepancy between the importance assigned to features and their observed impact on the model output under perturbation\. Given an explanation functionalΦ\\Phi, a modelff, and a random perturbation variable𝐈∈ℝd\\mathbf\{I\}\\in\\mathbb\{R\}^\{d\}, infidelity is defined as:

INFD​\(Φ,f,x\)=𝔼𝐈∼μI​\[\(𝐈⊤​Φ​\(f,x\)−\(f⁡\(x\)−f⁡\(x−𝐈\)\)\)2\]\\text\{INFD\}\(\\Phi,f,x\)=\\mathbb\{E\}\_\{\\mathbf\{I\}\\sim\\mu\_\{I\}\}\\left\[\\left\(\\mathbf\{I\}^\{\\top\}\\Phi\(f,x\)\-\(f\(x\)\-f\(x\-\\mathbf\{I\}\)\)\\right\)^\{2\}\\right\]\(19\)where𝐈\\mathbf\{I\}represents meaningful perturbations of interest and can be instantiated in various ways\. In our implementation, we use the noisy baseline variant where𝐈=x−z0\\mathbf\{I\}=x\-z\_\{0\}withz0=x0\+ϵz\_\{0\}=x\_\{0\}\+\\epsilonandϵ∼𝒩⁡\(0,σ2\)\\epsilon\\sim\\mathcal\{N\}\(0,\\sigma^\{2\}\)\. Unlike other faithfulness metrics,infidapplies random Gaussian noise directly to each modality input, rather than operating on the aggregated feature space, to keep the implementation close to the original formulation\. Lowerinfidvalues indicate that the attribution scores more faithfully reflect the impact of features on the model output\. For multi\-target tasks, we adapt theinfidmetric such that for each input perturbation, the differences between the model outputs on the unperturbed and perturbed inputs are computed for all targetsTTsimultaneously\.

Sensitivity\-N\.Sensitivity\-N \(sens\-n\)\([56](https://arxiv.org/html/2610.36929#bib.bib56)\)is a specific instantiation of infidelity where the perturbation corresponds to the subset\-of\-difference\-to\-baseline variant\[[18](https://arxiv.org/html/2610.36929#bib.bib18)\]\. An attribution method satisfies Sensitivity\-N when, for any random subset of featuresSk⊆\[d\]S\_\{k\}\\subseteq\[d\]of cardinalityNN, the sum of the attributions equals the change in model output caused by removing those features:

sens\-n\(Φ,f,x\)=𝔼Sk\[\(∑i∈SkΦ\(f,x\)i−\(f\(x\)−f\(x\[xSk=x0\]\)\)\)2\]\\textsc\{sens\-n\}\(\\Phi,f,x\)=\\mathbb\{E\}\_\{S\_\{k\}\}\[\(\\sum\_\{i\\in S\_\{k\}\}\\Phi\(f,x\)\_\{i\}\-\(f\(x\)\-f\(x\_\{\[x\_\{S\_\{k\}\}=x\_\{0\}\]\}\)\)\)^\{2\}\]\(20\)where the expectation is estimated overPPrandomly sampled subsetsSkS\_\{k\}ofNNfeature groups, andx\[xSk=x0\]x\_\{\[x\_\{S\_\{k\}\}=x\_\{0\}\]\}denotes the input with features inSkS\_\{k\}replaced by the reference baseline\. Unlikeinfid, we computesens\-nover the segmented feature space with group\-aggregated attribution scores\. The parameterNNcan be specified either as an absolute count or as a percentage of the total number of feature groups\. We evaluatesens\-nat perturbation levels ofN=\{20%,40%,60%\}N=\\\{20\\%,40\\%,60\\%\\\}\. For multi\-target tasks, we apply the same adaptation as forinfid, that is, the differences between the model outputs on the unperturbed and perturbed inputs are computed for all targetsTTsimultaneously\.

Monotonicity Correlation\.Monotonicity Correlation \(mon\-corr\)\([53](https://arxiv.org/html/2610.36929#bib.bib53)\)measures whether features with higher attribution scores have a proportionally larger effect on the model output when perturbed\. In our adaptation ofmon\-corr, for each feature groupkk, the relative variancevkv\_\{k\}of the output under perturbation is computed as described fornon\-sens\(see Eq\.[11](https://arxiv.org/html/2610.36929#S8.E11)\)\. The monotonicity correlation is then defined as the Spearman rank correlation between the absolute group\-aggregated attribution scores and the corresponding output variances:

mon\-corr​\(x\)=ρs​\(\{\|ak\|\}k=1K,\{vk\}k=1K\)\\textsc\{mon\-corr\}\(x\)=\\rho\_\{s\}\\left\(\\\{\|a\_\{k\}\|\\\}\_\{k=1\}^\{K\},\\\{v\_\{k\}\\\}\_\{k=1\}^\{K\}\\right\)\(21\)A value close to 1 indicates that the attribution ranking is well\-aligned with the actual feature importance as measured by the model’s sensitivity to perturbations\. As noted in Section[8\.1](https://arxiv.org/html/2610.36929#S8.SS1),mon\-corris computed jointly withnon\-sens, sharing the same per\-feature perturbation results\. For multi\-target tasks, perturbations are applied once and the output changes are accumulated over allTTtargets simultaneously\.

Monotonicity\.Monotonicity \(mon\)\([28](https://arxiv.org/html/2610.36929#bib.bib28)\)evaluates whether the model output increases monotonically as features are progressively added to the input from least important to most important\. We computemonby starting from a reference baseline input and introducing feature groups in ascending order of their attribution scores, recording the model output at each step\. If the output increases monotonically across all steps, the score is11; otherwise, it is00\. Since this metric uses ordered perturbations, multi\-target adaptation is not possible and the metric is computed independently for each sample\-target pair\.

Faithfulness Estimation\.Faithfulness Estimation \(faith\-est\)\([57](https://arxiv.org/html/2610.36929#bib.bib57)\)assesses faithfulness by measuring the correlation between the individual feature attributions and the observed effect of removing a feature on the model output\. In our adaptation, each feature groupkkis independently removed by replacing it with the corresponding modality\-wise reference baseline, and the resulting change in model output is recorded\. The faithfulness estimation score is the Pearson correlation between the attribution scores and the corresponding output changes:

faith\-est​\(x\)=r⁡\(\{ak\}k=1K,\{f⁡\(x\)−f⁡\(x∖k\)\}k=1K\)\\textsc\{faith\-est\}\(x\)=r\\left\(\\\{a\_\{k\}\\\}\_\{k=1\}^\{K\},\\\{f\(x\)\-f\(x\_\{\\setminus k\}\)\\\}\_\{k=1\}^\{K\}\\right\)\(22\)whereaka\_\{k\}is the group\-aggregated attribution score for feature groupkkandx∖kx\_\{\\setminus k\}denotes the input with feature groupkkreplaced by the reference baseline\. A value close to 1 indicates that the attribution scores faithfully predict the impact of removing individual features\. This metric uses ordered perturbations, and therefore multi\-target adaptation is not possible and the metric is computed independently for each sample\-target pair\.

Faithfulness Correlation\.Faithfulness Correlation \(faith\-corr\)\([58](https://arxiv.org/html/2610.36929#bib.bib58)\)works in a similar fashion tosens\-n, but instead of computing the mean squared error, it measures the Pearson correlation between the sum of attributions over a randomly selected subset of features and the corresponding change in model output\. In our adaptation, for each sample, we generateRRrandomly sampled subsetsSrS\_\{r\}containingNNfeature groups:

faith\-corr\(x\)=r\(\{∑i∈Srai\}r=1R,\{f\(x\)−f\(x\[xSr=x0\]\)\}r=1R\)\\textsc\{faith\-corr\}\(x\)=r\\left\(\\left\\\{\\sum\_\{i\\in S\_\{r\}\}a\_\{i\}\\right\\\}\_\{r=1\}^\{R\},\\left\\\{f\(x\)\-f\(x\_\{\[x\_\{S\_\{r\}\}=x\_\{0\}\]\}\)\\right\\\}\_\{r=1\}^\{R\}\\right\)\(23\)whereaia\_\{i\}is the group\-aggregated attribution score for feature groupii,x\[xSr=x0\]x\_\{\[x\_\{S\_\{r\}\}=x\_\{0\}\]\}denotes the input with features inSrS\_\{r\}replaced by the reference baseline, andRRis the number of random perturbation samples\. A value close to 1 indicates that the attribution scores faithfully predict the effect of removing feature subsets\. We evaluatefaith\-corrat perturbation levels ofN=\{20%,40%,60%\}N=\\\{20\\%,40\\%,60\\%\\\}of the total number of feature groups\. For multi\-target tasks, perturbations are applied once and the output changes are accumulated over allTTtargets simultaneously\.

#### 8\.4Robustness Metrics

Robustness metrics measure the stability of explanations under small input perturbations\.

Max Sensitivity\.Max Sensitivity \(max\-sens\)\([18](https://arxiv.org/html/2610.36929#bib.bib18)\)measures the maximum change in the explanation within a local neighborhood of the input\. Given an explanation functionalΦ\\Phi, a modelff, and an input neighborhood radiusrr, max sensitivity is defined as:

max\-sens​\(Φ,f,x,r\)=max‖y−x‖≤r⁡‖Φ⁡\(f,y\)−Φ⁡\(f,x\)‖\\textsc\{max\-sens\}\(\\Phi,f,x,r\)=\\max\_\{\\\|y\-x\\\|\\leq r\}\\\|\\Phi\(f,y\)\-\\Phi\(f,x\)\\\|\(24\)In practice, we approximate this by samplingNNrandom perturbations, applied independently to each input modality by adding uniform random noise in\[−r,r\]\[\-r,r\]\. Lower Max Sensitivity \(max\-sens\)\([18](https://arxiv.org/html/2610.36929#bib.bib18)\)values indicate more robust explanations\. Since this metric requires recomputing explanations for each perturbation, the underlying multi\-target explanation mode is used for multi\-target tasks\.

Avg\. Sensitivity\.Avg Sensitivity \(avg\-sens\)\([18](https://arxiv.org/html/2610.36929#bib.bib18)\)is defined identically to Max Sensitivity \(max\-sens\)\([18](https://arxiv.org/html/2610.36929#bib.bib18)\)but replaces the maximum with the mean over the sampled perturbations\. Lower values indicate more robust explanations\. The same multi\-target adaptation asmax\-sensapplies\.

### 9Training and Evaluation Hyperparameters

#### 9\.1Training Hyperparameters

Table[4](https://arxiv.org/html/2610.36929#S9.T4)lists the training hyperparameters for all model\-task combinations used in this work\. We useAdamorAdam\-Wwith linear warmup for optimization and apply early stopping based on validation performance on task\-specific metrics with a patience of1010\. All models are trained with a dropout rate of0\.10\.1\. For LayoutLMv3\[[2](https://arxiv.org/html/2610.36929#bib.bib2)\], input images are resized to224×224224\\times 224following the original work\.

Table 4:Training hyperparameters for all model\-task combinations used in this work\. Modality columns indicate which input modalities each model processes:𝒯\\mathcal\{T\}\(text\),𝒮\\mathcal\{S\}\(position\),ℒ\\mathcal\{L\}\(layout\),𝒱\\mathcal\{V\}\(image\)\.
#### 9\.2Explainer Hyperparameters

Table[5](https://arxiv.org/html/2610.36929#S9.T5)lists the hyperparameters used for the XAI methods\. We use the same set of hyperparameters across all tasks\. For methods that require sampling or interpolation steps, such as Integrated Gradients \(IG\)\[[35](https://arxiv.org/html/2610.36929#bib.bib35)\], GradientSHAP \(GS\)\[[24](https://arxiv.org/html/2610.36929#bib.bib24)\], LIME\[[15](https://arxiv.org/html/2610.36929#bib.bib15)\], and KernelSHAP \(KS\)\[[24](https://arxiv.org/html/2610.36929#bib.bib24)\], we use 200 samples or steps\. For DeepSHAP \(DS\)\[[24](https://arxiv.org/html/2610.36929#bib.bib24)\], a reference baseline distribution of 100 samples is precomputed from the training set of each dataset\.

Table 5:Hyperparameters for the evaluation of XAI methods used in this work\. Methods that are not listed have no configurable parameters\.Note that increasing the number of samples in the baseline may improve the performance of DeepSHAP \(DS\)\[[24](https://arxiv.org/html/2610.36929#bib.bib24)\], but this comes at extreme computational cost, as the method requiresNNbackward passes for each input sample \(or input sample\-target pair when using the naive multi\-target implementation\), whereNNis the size of the baseline distribution\. For Occlusion \(OCC\)\[[27](https://arxiv.org/html/2610.36929#bib.bib27)\], we define modality\-specific sliding windows and strides\. Thealphaparameter in LIME\[[15](https://arxiv.org/html/2610.36929#bib.bib15)\]denotes theL1L\_\{1\}penalty coefficient of the LASSO regression used to fit the surrogate model\. Note that for methods that require interpolation or sampling, an additional parameterinternal\_batch\_sizeis used that determines the internal computation batch size\. Similarly, for backpropagation\-based methods in multi\-target scenarios, an additional parametergrad\_batch\_sizeis used that determines the batch of targets for which backpropagation is performed simultaneously\. Both parameters are configured based on the available GPU memory\. For all single\-target scenarios, we use the method implementations available incaptum777[https://captum\.ai/](https://captum.ai/), with the exception of DeepSHAP \(DS\)\[[24](https://arxiv.org/html/2610.36929#bib.bib24)\]and GradientSHAP \(GS\)\[[24](https://arxiv.org/html/2610.36929#bib.bib24)\], both of which we extend for batched computation\. For multi\-target scenarios, we extend allcaptumimplementations to support the multi\-target setup\. These extended implementations are made publicly available in thetorchxai888[https://atriaml\.github\.io/torchxai/](https://atriaml.github.io/torchxai/)library\.

#### 9\.3Metric Hyperparameters

Table[6](https://arxiv.org/html/2610.36929#S9.T6)lists the hyperparameters for all XAI metrics used in the evaluation\. Perturbation\-based metrics that support chunking use a chunk size of 1%\. The parametermax\_features\_per\_batchdetermines the batch size for metrics that require a model forward pass\. For feature grouping and removal, all metrics use the same modality\-wise feature segmentation masks and baselines as those used in the computation of the XAI attributions\.

Table 6:Hyperparameters for all XAI metrics used in the evaluation\. Metrics that are not listed \(e\.g\.,compl,ent\-comp,spar\) have no configurable parameters\.non\-sensandmon\-corrshare the same hyperparameters as they are computed jointly\. Similarly,abpcandsufshare hyperparameters withaopc\.TypeMetricHyperparameterValueAxiomaticn\_perturbations\_per\_feature1chunk\_size1%ϵa\\epsilon\_\{a\}1\.00e\-3ϵv\\epsilon\_\{v\}1\.00e\-1Non\-Sensitivity \(ns\) \(↓\\downarrow\)max\_features\_per\_batch100Faithfulnesstotal\_feature\_bins200n\_random\_perms3AOPC \(aopc\)\(↑\\uparrow\)max\_features\_per\_batch100total\_feature\_bins200n\_random\_perms3ABPC \(abpc\) \(↑\\uparrow\)max\_features\_per\_batch100total\_feature\_bins200n\_random\_perms3Sufficiency \(suf\) \(↓\\downarrow\)max\_features\_per\_batch100n\_perturb\_samples200Sensitivity\-N \(sn\)\(↓\\downarrow\)max\_examples\_per\_batch100n\_perturb\_samples200noise\_scale0\.1Infidelity \(inf\) \(↓\\downarrow\)max\_examples\_per\_batch100chunk\_size1Faithfulness Estimation \(fe\)\(↑\\uparrow\)max\_features\_per\_batch100n\_perturb\_samples200Faithfulness Correlation \(fc\) \(↑\\uparrow\)max\_examples\_per\_batch100chunk\_size1%Monotonicity \(m\) \(↑\\uparrow\)max\_features\_per\_batch100n\_perturbations\_per\_feature1chunk\_size1%ϵa\\epsilon\_\{a\}1\.00e\-3ϵv\\epsilon\_\{v\}1\.00e\-1Monotonicity Correlation \(mc\) \(↑\\uparrow\)max\_features\_per\_batch100Complexityn\_perturbations\_per\_feature1chunk\_size1%ϵv\\epsilon\_\{v\}1\.00e\-1Effective Complexity \(ec\) \(↓\\downarrow\)max\_features\_per\_batch100Sundararajan Complexity \(sc\)\(↓\\downarrow\)ϵ\\epsilon1\.00e\-3Robustnessperturb\_radius0\.02n\_perturb\_samples10normFrobeniusMax Sensitivity \(ms\) \(↓\\downarrow\)max\_examples\_per\_batch1perturb\_radius0\.02n\_perturb\_samples10normFrobeniusAvg Sensitivity \(as\) \(↓\\downarrow\)max\_examples\_per\_batch1

We use the implementations of Max Sensitivity \(max\-sens\)\([18](https://arxiv.org/html/2610.36929#bib.bib18)\)and Infidelity \(infid\)\([18](https://arxiv.org/html/2610.36929#bib.bib18)\)fromcaptum, and extend them for multi\-target scenarios\. All remaining metric implementations, along with the multi\-target adaptations, are made publicly available in thetorchxailibrary\.

### 10Additional Results

#### 10\.1Per\-Task XAI Method Rankings

Fig\.[15](https://arxiv.org/html/2610.36929#S10.F15)presents the critical difference diagrams for each task individually\. For theQAtask \(Fig\.[15\(c\)](https://arxiv.org/html/2610.36929#S10.F15.sf3)\), only rankings are reported, as we were unable to perform pairwise significance testing due to the limited number of observations\. The overall patterns largely mirror the aggregated results over all tasks reported in Fig\.[3](https://arxiv.org/html/2610.36929#S5.F3); however, the statistical indistinguishability among methods for some metrics was more pronounced on individual tasks\. For instance, on theSERtask, bothmonandinfidshow large statistical indistinguishability across all methods, which suggests that these metrics do not provide any meaningful results for evaluation\. Similar results can be seen on theCLStask formonandsparmetrics\.

Fig\.[16](https://arxiv.org/html/2610.36929#S10.F16)shows the aggregated mean ranks per evaluation criterion for each task\. The per\-task patterns are broadly consistent with the overall aggregated ranking \(Fig\.[4](https://arxiv.org/html/2610.36929#S5.F4)\) with minor task\-specific variations\.

\(a\)Classification\(b\)Semantic Entity Recognition\(c\)Document VQA
Figure 15:Per\-metric critical difference diagrams for each task\. Methods connected by a horizontal bar are not statistically significantly different \(Wilcoxon signed\-rank test with Holm correction,α=0\.05\\alpha=0\.05\)\. Lower ranks indicate better performance\.\(a\)Classification\(b\)Semantic Entity Recognition\(c\)Document VQA
Figure 16:Aggregated mean ranks of XAI methods per evaluation criterion for each task\. Lower ranks indicate better performance\.![Refer to caption](https://arxiv.org/html/2610.36929v1/images/results/seq_cls/ranked_correlation_heatmaps_faithfulness.png)\(a\)CLS
![Refer to caption](https://arxiv.org/html/2610.36929v1/images/results/token_cls/ranked_correlation_heatmaps_faithfulness.png)\(b\)SER
![Refer to caption](https://arxiv.org/html/2610.36929v1/images/results/qa/ranked_correlation_heatmaps_faithfulness.png)\(c\)QA

Figure 17:Per\-task pairwise Spearman correlation between faithfulness metric rankings across respective datasets, models, and explainer settings\. On theQAtask, Monotonicity \(mon\)\([28](https://arxiv.org/html/2610.36929#bib.bib28)\)produced undefined values and is therefore excluded\.![Refer to caption](https://arxiv.org/html/2610.36929v1/images/results/seq_cls/ranked_correlation_heatmaps_complexity.png)\(a\)CLS
![Refer to caption](https://arxiv.org/html/2610.36929v1/images/results/token_cls/ranked_correlation_heatmaps_complexity.png)\(b\)SER
![Refer to caption](https://arxiv.org/html/2610.36929v1/images/results/qa/ranked_correlation_heatmaps_complexity.png)\(c\)QA

Figure 18:Per\-task pairwise Spearman correlation between complexity metric rankings across respective datasets, models, and explainer settings\.![Refer to caption](https://arxiv.org/html/2610.36929v1/images/results/seq_cls/aopc_curves.png)Figure 19:aopccurves for all XAI methods across all dataset\-model combinations on theCLStask\. MoRF \(solid\) and LeRF \(dotted\) curves are shown relative to the random baseline\.![Refer to caption](https://arxiv.org/html/2610.36929v1/images/results/token_cls/aopc_curves.png)Figure 20:aopccurves for all XAI methods across all dataset\-model combinations on theSERtask\. MoRF \(solid\) and LeRF \(dotted\) curves are shown relative to the random baseline\.![Refer to caption](https://arxiv.org/html/2610.36929v1/images/results/qa/aopc_curves.png)Figure 21:aopccurves for all XAI methods across all dataset\-model combinations on theQAtask\. MoRF \(solid\) and LeRF \(dotted\) curves are shown relative to the random baseline\.IG \(RVL\-CDIP\)

![Refer to caption](https://arxiv.org/html/2610.36929v1/figs/main/seq_cls/images/img_0001.png)
Occ\. \(RVL\-CDIP\)

![Refer to caption](https://arxiv.org/html/2610.36929v1/figs/main/seq_cls/images/img_0002.png)
RA \(RVL\-CDIP\)

![Refer to caption](https://arxiv.org/html/2610.36929v1/figs/main/seq_cls/images/img_0003.png)
IG \(DocLayNet\)

![Refer to caption](https://arxiv.org/html/2610.36929v1/app/figs/layoutlm/seq_cls/images/img_0004.png)
Occ\. \(DocLayNet\)

![Refer to caption](https://arxiv.org/html/2610.36929v1/app/figs/layoutlm/seq_cls/images/img_0005.png)
RA \(DocLayNet\)

![Refer to caption](https://arxiv.org/html/2610.36929v1/app/figs/layoutlm/seq_cls/images/img_0006.png)

Figure 22:Modality\-wise explanations of the LayoutLMv3\[[2](https://arxiv.org/html/2610.36929#bib.bib2)\]model predictions for two randomly selected samples from RVL\-CDIP\[[50](https://arxiv.org/html/2610.36929#bib.bib50)\]and DocLayNet\([61](https://arxiv.org/html/2610.36929#bib.bib61)\)\(CLStask\)\. Results are shown for three methods: IG, OCC, and RA\.IG \(CORD\)

![Refer to caption](https://arxiv.org/html/2610.36929v1/app/figs/layoutlm/token_cls/images/img_0001.png)
Occ\. \(CORD\)

![Refer to caption](https://arxiv.org/html/2610.36929v1/app/figs/layoutlm/token_cls/images/img_0002.png)
RA \(CORD\)

![Refer to caption](https://arxiv.org/html/2610.36929v1/app/figs/layoutlm/token_cls/images/img_0003.png)
IG \(FUNSD\)

![Refer to caption](https://arxiv.org/html/2610.36929v1/app/figs/layoutlm/token_cls/images/img_0004.png)
Occ\. \(FUNSD\)

![Refer to caption](https://arxiv.org/html/2610.36929v1/app/figs/layoutlm/token_cls/images/img_0005.png)
RA \(FUNSD\)

![Refer to caption](https://arxiv.org/html/2610.36929v1/app/figs/layoutlm/token_cls/images/img_0006.png)

Figure 23:Modality\-wise explanations of the LayoutLMv3\[[2](https://arxiv.org/html/2610.36929#bib.bib2)\]model predictions for two randomly selected samples from CORD\([62](https://arxiv.org/html/2610.36929#bib.bib62)\)and FUNSD\[[51](https://arxiv.org/html/2610.36929#bib.bib51)\]\(SERtask\)\. Results are shown for three methods: IG, OCC, and RA\.IG \(DocVQA\)

![Refer to caption](https://arxiv.org/html/2610.36929v1/app/figs/layoutlm/qa/images/img_0001.png)
Occ\. \(DocVQA\)

![Refer to caption](https://arxiv.org/html/2610.36929v1/app/figs/layoutlm/qa/images/img_0002.png)
RA \(DocVQA\)

![Refer to caption](https://arxiv.org/html/2610.36929v1/app/figs/layoutlm/qa/images/img_0003.png)
IG \(DocVQA\)

![Refer to caption](https://arxiv.org/html/2610.36929v1/figs/main/qa/images/img_0004.png)
Occ\. \(DocVQA\)

![Refer to caption](https://arxiv.org/html/2610.36929v1/figs/main/qa/images/img_0005.png)
RA \(DocVQA\)

![Refer to caption](https://arxiv.org/html/2610.36929v1/figs/main/qa/images/img_0006.png)

Figure 24:Modality\-wise explanations of the LayoutLMv3\[[2](https://arxiv.org/html/2610.36929#bib.bib2)\]model predictions for two randomly selected samples from DocVQA\([65](https://arxiv.org/html/2610.36929#bib.bib65)\)\(QAtask\)\. Results are shown for three methods: IG, OCC, and RA\.IG \(RVL\-CDIP\)

![Refer to caption](https://arxiv.org/html/2610.36929v1/app/figs/lilt/seq_cls/images/img_0001.png)
Occ\. \(RVL\-CDIP\)

![Refer to caption](https://arxiv.org/html/2610.36929v1/app/figs/lilt/seq_cls/images/img_0002.png)
RA \(RVL\-CDIP\)

![Refer to caption](https://arxiv.org/html/2610.36929v1/app/figs/lilt/seq_cls/images/img_0003.png)
IG \(DocLayNet\)

![Refer to caption](https://arxiv.org/html/2610.36929v1/app/figs/lilt/seq_cls/images/img_0004.png)
Occ\. \(DocLayNet\)

![Refer to caption](https://arxiv.org/html/2610.36929v1/app/figs/lilt/seq_cls/images/img_0005.png)
RA \(DocLayNet\)

![Refer to caption](https://arxiv.org/html/2610.36929v1/app/figs/lilt/seq_cls/images/img_0006.png)

Figure 25:Modality\-wise explanations of the LiLT\[[10](https://arxiv.org/html/2610.36929#bib.bib10)\]model predictions for two randomly selected samples from RVL\-CDIP\[[50](https://arxiv.org/html/2610.36929#bib.bib50)\]and DocLayNet\([61](https://arxiv.org/html/2610.36929#bib.bib61)\)\(CLStask\)\. Results are shown for three methods: IG, OCC, and RA\.IG \(CORD\)

![Refer to caption](https://arxiv.org/html/2610.36929v1/app/figs/lilt/token_cls/images/img_0001.png)
Occ\. \(CORD\)

![Refer to caption](https://arxiv.org/html/2610.36929v1/app/figs/lilt/token_cls/images/img_0002.png)
RA \(CORD\)

![Refer to caption](https://arxiv.org/html/2610.36929v1/app/figs/lilt/token_cls/images/img_0003.png)
IG \(FUNSD\)

![Refer to caption](https://arxiv.org/html/2610.36929v1/app/figs/lilt/token_cls/images/img_0004.png)
Occ\. \(FUNSD\)

![Refer to caption](https://arxiv.org/html/2610.36929v1/app/figs/lilt/token_cls/images/img_0005.png)
RA \(FUNSD\)

![Refer to caption](https://arxiv.org/html/2610.36929v1/app/figs/lilt/token_cls/images/img_0006.png)

Figure 26:Modality\-wise explanations of the LiLT\[[10](https://arxiv.org/html/2610.36929#bib.bib10)\]model predictions for two randomly selected samples from CORD\([62](https://arxiv.org/html/2610.36929#bib.bib62)\)and FUNSD\[[51](https://arxiv.org/html/2610.36929#bib.bib51)\]\(SERtask\)\. Results are shown for three methods: IG, OCC, and RA\.IG \(DocVQA\)

![Refer to caption](https://arxiv.org/html/2610.36929v1/app/figs/lilt/qa/images/img_0001.png)
Occ\. \(DocVQA\)

![Refer to caption](https://arxiv.org/html/2610.36929v1/app/figs/lilt/qa/images/img_0002.png)
RA \(DocVQA\)

![Refer to caption](https://arxiv.org/html/2610.36929v1/app/figs/lilt/qa/images/img_0003.png)
IG \(DocVQA\)

![Refer to caption](https://arxiv.org/html/2610.36929v1/app/figs/lilt/qa/images/img_0004.png)
Occ\. \(DocVQA\)

![Refer to caption](https://arxiv.org/html/2610.36929v1/app/figs/lilt/qa/images/img_0005.png)
RA \(DocVQA\)

![Refer to caption](https://arxiv.org/html/2610.36929v1/app/figs/lilt/qa/images/img_0006.png)

Figure 27:Modality\-wise explanations of the LiLT\[[10](https://arxiv.org/html/2610.36929#bib.bib10)\]model predictions for two randomly selected samples from DocVQA\([65](https://arxiv.org/html/2610.36929#bib.bib65)\)\(QAtask\)\. Results are shown for three methods: IG, OCC, and RA\.Table 7:Modality contribution fractions computed for each model across allCLSdatasets\.Table 8:Modality contribution fractions computed for each model across allSERdatasets\.Table 9:Modality contribution fractions computed for each model on the DocVQA\([65](https://arxiv.org/html/2610.36929#bib.bib65)\)dataset\.
#### 10\.2Per\-Task Metric Agreement

Fig\.[17](https://arxiv.org/html/2610.36929#S10.F17)and Fig\.[18](https://arxiv.org/html/2610.36929#S10.F18)present the task\-wise pairwise Spearman correlations for the faithfulness and complexity metrics\. Overall, the task\-wise faithfulness and complexity correlation patterns were largely consistent with the aggregated results reported in Fig\.[5](https://arxiv.org/html/2610.36929#S5.F5)\.

#### 10\.3AOPC Curves

Figs\.[19](https://arxiv.org/html/2610.36929#S10.F19),[20](https://arxiv.org/html/2610.36929#S10.F20), and[21](https://arxiv.org/html/2610.36929#S10.F21)present theaopccurves for all XAI methods across all dataset\-model combinations for theCLS,SER, andQAtasks, respectively\. We visualize both MoRF \(solid\) and LeRF \(dotted\) curves relative to the random baseline\. Higher MoRF and lower LeRF curves indicate better faithfulness\.

#### 10\.4Full Metric Results

The complete evaluation results across all 17 metrics, 13 XAI methods, 8 datasets, and 4 models are provided at[https://docs\.google\.com/spreadsheets/d/12oEqjm\-a1fHae8TKfcVGBTjq0mFAMtpsEI\-\_CFQJeeU/edit?gid=0\#gid=0](https://docs.google.com/spreadsheets/d/12oEqjm-a1fHae8TKfcVGBTjq0mFAMtpsEI-_CFQJeeU/edit?gid=0#gid=0)\. These results include the per\-dataset mean scores for each metric\-method\-model combination, which were used to compute the rankings and correlation analyses presented in Section[5\.1](https://arxiv.org/html/2610.36929#S5.SS1)and Section[5\.2](https://arxiv.org/html/2610.36929#S5.SS2)\.

#### 10\.5Additional Qualitative Results

Figs\.[22](https://arxiv.org/html/2610.36929#S10.F22),[23](https://arxiv.org/html/2610.36929#S10.F23), and[24](https://arxiv.org/html/2610.36929#S10.F24)present additional qualitative results for the LayoutLMv3\[[2](https://arxiv.org/html/2610.36929#bib.bib2)\]model, and Figs\.[25](https://arxiv.org/html/2610.36929#S10.F25),[26](https://arxiv.org/html/2610.36929#S10.F26), and[27](https://arxiv.org/html/2610.36929#S10.F27)for the LiLT\[[10](https://arxiv.org/html/2610.36929#bib.bib10)\]model\. All explanations are generated using OCC, IG, and RA\.

#### 10\.6Dataset\-wise Modality Contributions

Table[7](https://arxiv.org/html/2610.36929#S10.T7), Table[8](https://arxiv.org/html/2610.36929#S10.T8), and Table[9](https://arxiv.org/html/2610.36929#S10.T9)respectively present the overall modality contribution for LayoutLMv3\[[2](https://arxiv.org/html/2610.36929#bib.bib2)\]and LiLT\[[10](https://arxiv.org/html/2610.36929#bib.bib10)\]across theCLS,SER, andQAtasks, computed using Occlusion \(OCC\)\[[27](https://arxiv.org/html/2610.36929#bib.bib27)\], Integrated Gradients \(IG\)\[[35](https://arxiv.org/html/2610.36929#bib.bib35)\], and Raw Attention \(RA\)\[[66](https://arxiv.org/html/2610.36929#bib.bib66)\]\.