Explainable Artificial Intelligence (XAI) in Computational Pathology: Definitions, Taxonomy, and Recommendations

# Explainable Artificial Intelligence (XAI) in Computational Pathology: Definitions, Taxonomy, and Recommendations
Source: [https://arxiv.org/html/2608.28820](https://arxiv.org/html/2608.28820)
Shubham InnaniAddress:Division of Computational Pathology, Department of Pathology and Laboratory Medicine, Indiana University School of Medicine, Indianapolis, IN, USAAddress:Indiana University Melvin and Bren Simon Comprehensive Cancer Center, Indianapolis, IN, USASuhang YouAddress:Division of Computational Pathology, Department of Pathology and Laboratory Medicine, Indiana University School of Medicine, Indianapolis, IN, USAAddress:Indiana University Melvin and Bren Simon Comprehensive Cancer Center, Indianapolis, IN, USAAdam ShephardAddress:Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UKFrancesco CiompiAddress:Department of Pathology, Radboud University Medical Center, Nijmegen, The NetherlandsJoe YeongAddress:Singapore General Hospital, SingaporeNasir RajpootAddress:Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UKMichael FeldmanAddress:Division of Computational Pathology, Department of Pathology and Laboratory Medicine, Indiana University School of Medicine, Indianapolis, IN, USAAddress:Indiana University Melvin and Bren Simon Comprehensive Cancer Center, Indianapolis, IN, USASolene Florence Kammerer\-JacquetAddress:Department of Pathology, Radboud University Medical Center, Nijmegen, The NetherlandsDimitrios MakrisAddress:Department of Computer Science, School of Computer Science and Mathematics, Kingston University London, London, UKGeert LitjensAddress:Department of Pathology, Radboud University Medical Center, Nijmegen, The NetherlandsAnne L\. MartelAddress:Department of Medical Biophysics, University of Toronto, Toronto, ON, CanadaAddress:Physical Sciences Platform, Sunnybrook Research Institute, Toronto, ON, CanadaAddress:Vector Institute, Toronto, ON, CanadaJana LipkovaAddress:University of California, Irvine, School of Medicine, CA, USAApril KhademiAddress:Toronto Metropolitan University, Toronto, ON, CanadaAddress:Vector Institute, Toronto, ON, CanadaSpyridon BakasAddress:Division of Computational Pathology, Department of Pathology and Laboratory Medicine, Indiana University School of Medicine, Indianapolis, IN, USAAddress:Indiana University Melvin and Bren Simon Comprehensive Cancer Center, Indianapolis, IN, USAAddress:Department of Computer Science, School of Computer Science and Mathematics, Kingston University London, London, UKAddress:Departments of Biostatistics and Health Data Science; Radiology and Imaging Sciences; Neurological Surgery, Indiana University School of Medicine, Indianapolis, IN, USAAddress:Department of Computer Science, Luddy School of Informatics, Computing, and Engineering, Indiana University, Indianapolis, IN, USACorresponding author:Corresponding author\.for MICCAI SIG\-CompPathEmail:[spbakas@iu\.edu](mailto:[email protected])

###### Abstract

Computational pathology \(CompPath\) is transforming medicine by leveraging artificial intelligence \(AI\) algorithms to support diagnosis, prognosis, and treatment prediction from gigapixel whole\-slide images\. Clinical adoption is progressing, but is constrained by concerns about safety, accountability, and regulatory oversight in high\-stakes clinical environments\. Explainable AI \(XAI\) systems hold promise for building trust and enabling verification, yet the literature remains fragmented due to inconsistent terminology, overlapping methodological families, ad hoc validation, and current reviews\. This review aims to formalize XAI methods in CompPath through the: i\) introduction of a pathology\-centric vocabulary comprising seven core terms; ii\) development of a taxonomy across methodological families and three orthogonal axes \(stage, type, scope\); and iii\) establishment of a task\-driven framework that maps five clinical questions to recommended methods, method evaluation, and deployment context\. Five key gaps between current XAI capabilities and clinical deployment are identified, and actionable steps are proposed to advance XAI for CompPath\.

###### Keywords:

XAI , computational pathology , explainability , taxonomy , artificial intelligence , interpretability

## Introduction

Histopathology has been the cornerstone of disease diagnosis, traditionally based on the visual inspection of tissue samples under a microscope\[[1](https://arxiv.org/html/2608.28820#bib.bib1)\]\. Recent advances in digital scanners and high\-throughput infrastructure have transformed this field into digital pathology, in which glass slides of tissue sections are digitized as whole\-slide images \(WSIs\)\. This transition has enabled large\-scale, data\-driven computational analysis for diagnosis, prognosis, treatment response prediction, and biomarker discovery\[[2](https://arxiv.org/html/2608.28820#bib.bib2),[3](https://arxiv.org/html/2608.28820#bib.bib3),[67](https://arxiv.org/html/2608.28820#biba.bib67),[113](https://arxiv.org/html/2608.28820#biba.bib113),[6](https://arxiv.org/html/2608.28820#bib.bib6),[7](https://arxiv.org/html/2608.28820#bib.bib7),[112](https://arxiv.org/html/2608.28820#biba.bib112),[22](https://arxiv.org/html/2608.28820#biba.bib22),[10](https://arxiv.org/html/2608.28820#bib.bib10),[11](https://arxiv.org/html/2608.28820#bib.bib11),[12](https://arxiv.org/html/2608.28820#bib.bib12),[114](https://arxiv.org/html/2608.28820#biba.bib114),[14](https://arxiv.org/html/2608.28820#bib.bib14),[15](https://arxiv.org/html/2608.28820#bib.bib15),[16](https://arxiv.org/html/2608.28820#bib.bib16),[17](https://arxiv.org/html/2608.28820#bib.bib17),[18](https://arxiv.org/html/2608.28820#bib.bib18)\], defining the domain of Computational Pathology \(CompPath\)\. These advances can reshape diagnostic workflows, improving efficiency, accuracy, and patient care\[[19](https://arxiv.org/html/2608.28820#bib.bib19),[20](https://arxiv.org/html/2608.28820#bib.bib20),[21](https://arxiv.org/html/2608.28820#bib.bib21),[22](https://arxiv.org/html/2608.28820#bib.bib22),[23](https://arxiv.org/html/2608.28820#bib.bib23)\]\.

![Refer to caption](https://arxiv.org/html/2608.28820v1/figs/block_diagram_raw_updated.png)Figure 1:CompPath pipelines comprise data acquisition, model training, and outcome generation\. Inputs span WSIs and tissue microarrays, as well as pathology reports, structured clinical and laboratory data, molecular profiles, and prior radiology findings\. Outcomes include classification, segmentation and biomarker scores to text outputs generated by language and vision\-language models, including structured\-report drafts, concept\-grounded retrieval results, and natural\-language rationales\. XAI can be integrated at any part of such pipeline to assess whether predictions rely on clinically relevant evidence\.In recent years, clinical\-grade AI tools for pathology images, such as Paige Prostate\[[24](https://arxiv.org/html/2608.28820#bib.bib24)\]and Ibex Galen\[[25](https://arxiv.org/html/2608.28820#bib.bib25)\], have obtained regulatory approval for cancer detection\. These tools signal that clinical adoption is possible, but they also illustrate a central challenge: unlike long\-established medical technologies, current AI models are new, with limited real\-world deployment and longitudinal validation across diverse patient populations\. Established medical devices build trust through years of validation, well\-characterized limitations, and integration into medical training\. Most current AI models in CompPath lack that evidence\[[19](https://arxiv.org/html/2608.28820#bib.bib19),[20](https://arxiv.org/html/2608.28820#bib.bib20),[21](https://arxiv.org/html/2608.28820#bib.bib21),[26](https://arxiv.org/html/2608.28820#bib.bib26)\], and their failure modes, robustness, and generalizability beyond curated datasets remain under\-studied\[[27](https://arxiv.org/html/2608.28820#bib.bib27),[12](https://arxiv.org/html/2608.28820#biba.bib12)\]\.

A major reason is that CompPath pipelines, from data acquisition to model training and outcome generation, rely on complex deep learning \(DL\) models whose decision processes are not directly interpretable or explainable\[[102](https://arxiv.org/html/2608.28820#biba.bib102)\]\. The opacity of these systems means that model failures and lack of robustness cannot be directly understood, and pathologists cannot always trace why a particular decision was made\. Whether interpretability is a strict prerequisite for clinical deployment is a matter of active debate\. A recent Delphi consensus of 37 experts convened by ESMO classified ground truth, performance, and generalizability as essential requirements for AI\-based biomarkers while treating explainability as recommended rather than mandatory, reflecting a view that reproducible tools validated in clinical trials can justify adoption without a mechanistic account of every prediction\[[30](https://arxiv.org/html/2608.28820#bib.bib30)\]\. Ghassemi and colleagues have argued similarly that current XAI methods often provide false hope at the patient\-decision level\[[31](https://arxiv.org/html/2608.28820#bib.bib31)\]\. We take a middle position: explainability is not always strictly required, but is a strong contributor to trust, especially in early adoption, high\-stakes clinical decisions, and regulatory review\. To address this challenge, explainable AI \(XAI\) techniques have been proposed to provide insights into the decision\-making process of these tools\. Figure[1](https://arxiv.org/html/2608.28820#S1.F1)demonstrates the final recommendations and guidelines given in this review that highlights how XAI can illuminate the “black\-box” by revealing the underlying model reasoning\.

XAI methods for CompPath can support model interrogation, failure analysis, and assessment of whether predictions align with established pathology knowledge\. Their value is application\-dependent: high\-risk tasks such as treatment\-response prediction, diagnostic decision support, or companion diagnostics may require stronger explainability evidence, whereas lower\-risk or intrinsically inspectable outputs, such as segmentation maps, may require different forms of validation\. The key question is therefore not whether XAI is universally required, but when, why, and for whom explainability matters in CompPath\.

A persistent challenge in XAI literature is inconsistent terminology\[[32](https://arxiv.org/html/2608.28820#bib.bib32),[33](https://arxiv.org/html/2608.28820#bib.bib33),[34](https://arxiv.org/html/2608.28820#bib.bib34),[35](https://arxiv.org/html/2608.28820#bib.bib35),[2](https://arxiv.org/html/2608.28820#biba.bib2),[1](https://arxiv.org/html/2608.28820#biba.bib1)\]and lack of categorization and taxonomical definitions\. Terms such asinterpretability,transparency, andexplainabilityare often used interchangeably across technical and clinical audiences, creating confusion about what different XAI methods actually provide, and which method families are prevalent\. This ambiguity interferes with efforts to define what constitutes sufficient explainability evidence for regulatory approval\. It necessitates to establish what the underlying terms mean, and inconsistent usage across research, regulatory, and clinical communities creates delays\. To address this, we build upon prior foundational work on XAI and its applications in biomedical imaging\[[32](https://arxiv.org/html/2608.28820#biba.bib32),[39](https://arxiv.org/html/2608.28820#bib.bib39),[33](https://arxiv.org/html/2608.28820#biba.bib33),[28](https://arxiv.org/html/2608.28820#biba.bib28),[42](https://arxiv.org/html/2608.28820#bib.bib42),[43](https://arxiv.org/html/2608.28820#bib.bib43),[44](https://arxiv.org/html/2608.28820#bib.bib44),[102](https://arxiv.org/html/2608.28820#biba.bib102),[45](https://arxiv.org/html/2608.28820#bib.bib45)\]and adapt them specifically for CompPath\. This review makes the following contributions:

1. 1\.TerminologyProvide definitions for seven XAI concepts \(interpretability, explainability, transparency, causal validity, causability, justifiability, and contestability\) for CompPath \(Section[2](https://arxiv.org/html/2608.28820#S2), Table[1](https://arxiv.org/html/2608.28820#S2.T1)\)\.
2. 2\.Three\-axis taxonomy:Position XAI methods along three orthogonal dimensions: stage \(intrinsic vs post hoc\), type \(model\-specific vs model\-agnostic\), and scope \(local vs regional/slide vs global\) \(Section[3\.2](https://arxiv.org/html/2608.28820#S3.SS2), Figure[2\(a\)](https://arxiv.org/html/2608.28820#S3.F2.sf1)\)\.
3. 3\.Method Families:Organise the field into five method families \(backpropagation\-, perturbation\-, feature\-, concept\-, and example\-based\) \(Figure[2\(b\)](https://arxiv.org/html/2608.28820#S3.F2.sf2)\), with strengths, limitations, and validation practices summarised in Supplementary Table[3](https://arxiv.org/html/2608.28820#S3.T3)–[7](https://arxiv.org/html/2608.28820#S3.T7)\. Over 100 CompPath studies are surveyed \(Section[3\.2](https://arxiv.org/html/2608.28820#S3.SS2)\)\.
4. 4\.Recommendations:Propose recommendations for task\-driven application of XAI methods based on five clinical questions reflecting explainability versus a universal “best” method, with conditional guidance on model architecture, deployment context, and validation requirements \(Section[3](https://arxiv.org/html/2608.28820#S4.F3)\)\.

## Definitions

This article assesses seven interconnected terms, defined here and consolidated in Table[1](https://arxiv.org/html/2608.28820#S2.T1)\. Detailed pathology\-grounded definitions with examples are in Suppl\. Section[S1](https://arxiv.org/html/2608.28820#S1a)\. Interpretability, explainability, and transparency are used inconsistently and these three terms form a closely related family with the common goal of making model behavior comprehensible to humans\. The distinctions matter when: i\) validation evidence must be documented, ii\) methods are compared across studies, and iii\) a clinical or regulatory reviewer asks what specific claims about a model actually means\. A recent multi\-disciplinary consensus convened by the Pathology Innovation Collaborative Community \(PICC\) illustrates the same pattern for the related term validation, which carries substantially different meanings across communication science, AI/ML, clinical and laboratory practice, regulatory science, and business contexts, with documented consequences for clinical translation\[[30](https://arxiv.org/html/2608.28820#biba.bib30)\]; the same fragmentation applies XAI\.

Table 1:CompPath XAI terminology, treated as distinct, complementary concepts with different scopes and evaluation criteria\.Our proposed framework builds on prior XAI literature and adapts it for CompPath\. Distinction between interpretability and explainability follows Lipton\[[1](https://arxiv.org/html/2608.28820#biba.bib1)\]and subsequent surveys\[[2](https://arxiv.org/html/2608.28820#biba.bib2),[33](https://arxiv.org/html/2608.28820#bib.bib33)\]\. Our treatment of transparency as a system\- and governance\-level property follows Lipton\[[1](https://arxiv.org/html/2608.28820#biba.bib1)\]and emerging regulatory frameworks \(EU AI Act, FDA guidance on AI/ML\-based Software as a Medical Device\)\[[14](https://arxiv.org/html/2608.28820#biba.bib14),[13](https://arxiv.org/html/2608.28820#biba.bib13)\]\. The notion of*causability*, the quality of an explanation as judged by a domain expert, is taken from Holzinger et al\.\[[24](https://arxiv.org/html/2608.28820#biba.bib24)\]\.*Causal validity*as a term originates with Campbell, who proposed it to denote demonstrated causal relationships in a specific study context\[[51](https://arxiv.org/html/2608.28820#bib.bib51),[52](https://arxiv.org/html/2608.28820#bib.bib52)\]; in XAI evaluation it has been adopted to describe if perturbing a model’s input actually changes the prediction, distinguishing genuine input\-output causation from mere correlation\[[18](https://arxiv.org/html/2608.28820#biba.bib18),[19](https://arxiv.org/html/2608.28820#biba.bib19)\]\. The terms*justifiability*and*contestability*are drawn from clinical AI ethics and governance literature\[[15](https://arxiv.org/html/2608.28820#biba.bib15),[27](https://arxiv.org/html/2608.28820#biba.bib27),[28](https://arxiv.org/html/2608.28820#biba.bib28),[29](https://arxiv.org/html/2608.28820#biba.bib29)\], where they capture the ability to defend, audit, and challenge model outputs\. We define the seven terms used throughout this review as summarized in Table[1](https://arxiv.org/html/2608.28820#S2.T1), which consolidates each term’s core question, a pathology\-specific example, and the criterion against which it is evaluated \(detailed examples are in Supplementary Section[S1](https://arxiv.org/html/2608.28820#S1a)\)\.

Three features distinguish our framework from generic XAI taxonomies\. First, we keep*causal validity*and*causability*as technically separate concepts: causal validity is internal to the model \(does perturbing a candidate region change the prediction?\), while causability is external \(does the resulting explanation align with established histopathological knowledge?\)\. A model can satisfy one without the other, and clinical trust requires both\. Second, every term is defined for CompPath such as nuclear pleomorphism, gland density, mitotic count, and tumor\-infiltrating lymphocyte density, and we describe each term in relation to WSIs, TMAs, region\-of\-interest \(ROI\) analyses, and multiple instance learning \(MIL\) pipelines\. Third, we treat the seven terms as a coherent system for clinical translation, rather than as competing synonyms: interpretability and explainability describe what is readable from the model and the evidence accompanying its output; transparency describes what the deployment system documents; causal validity and causability describe what an explanation must satisfy technically and clinically; justifiability and contestability describe the governance scaffolding that allows pathologists and regulators to act on the explanation\.

## Taxonomy

![Refer to caption](https://arxiv.org/html/2608.28820v1/figs/XAI_stage_type_scope.png)\(a\)Stage\-Type\-Scope Taxonomy of XAI Methods in CompPath\.
\(b\)XAI Methodological Families in CompPath\.

Figure 2:Taxonomies of XAI methods in CompPath\.\(a\)the Stage\-Type\-Scope taxonomy describes 3 orthogonal dimensions for each XAI method\. Stage \(intrinsic vs\. post hoc\) addresses when the explanation is produced\. Type \(model\-specific vs\. model\-agnostic\) addresses how dependent the explanation is on the model’s architecture\. Scope \(local vs\. regional/slide vs\. global\) addresses the level at which the explanation operates\.\(b\)XAI Methodological Families and their positioning within the Stage\-Type\-Scope taxonomy\.### Three Orthogonal Dimensions

We position XAI methods along three orthogonal dimensions \(Stage, Type, and Scope \- Figure[2\(a\)](https://arxiv.org/html/2608.28820#S3.F2.sf1)\), providing a structured basis for comparing approaches and clarifying common terminology in the literature\[[31](https://arxiv.org/html/2608.28820#biba.bib31),[32](https://arxiv.org/html/2608.28820#biba.bib32),[33](https://arxiv.org/html/2608.28820#biba.bib33)\]\.

- •Stagedistinguishes*intrinsic*methods, in which the explanation is part of the model’s own computation \(e\.g\., concept\-bottleneck models, attention weights\), from*post hoc*methods, in which a separate procedure is applied to an already\-trained model \(e\.g\., Grad\-CAM, SHAP\)\.
- •Typedistinguishes*model\-specific*methods, which exploit gradients, activations, or architectural internals \(and therefore depend on the model class\), from*model\-agnostic*methods, which use only input–output behavior through perturbation, sampling, or surrogate fitting\.
- •Scopedistinguishes*local*explanations of an individual case \(a single WSI, patch, or ROI\) from*global*explanations that characterize the model’s behavior across a cohort or population\.

These axes are independent: a method’s position on one does not fix its position on the others, although in CompPath most methods are clustered as post hoc, model\-specific, and local\. Detailed definitions and examples for each axis are provided in Supplementary Section[S2](https://arxiv.org/html/2608.28820#S2a), and Supplementary Table[2](https://arxiv.org/html/2608.28820#S2.T2)categorizes each method\.

### Methodological Families

XAI methods in CompPath can be categorized into five methodological families: \(i\) backpropagation\-based, \(ii\) perturbation\-based, \(iii\) feature\-based, \(iv\) concept\-based, and \(v\) example\-based\. Each family is characterized along five attributes relevant to clinical translation: a\) operating principle, b\) interpretability granularity \(patch\-, region\-, or slide\-level\), c\) computational cost, d\) validation requirements, and e\) readiness for clinical or regulatory integration\. Figure[2\(b\)](https://arxiv.org/html/2608.28820#S3.F2.sf2)situates these families within the Stage–Type–Scope taxonomy, and Table[3](https://arxiv.org/html/2608.28820#S3.T3)–[7](https://arxiv.org/html/2608.28820#S3.T7)summarizes the strengths, limitations, and recommended validation practices for each method across the five families\. A survey of over 100 CompPath studies across anatomies are provided in Supplementary Sections[S3](https://arxiv.org/html/2608.28820#S3a)and[S5](https://arxiv.org/html/2608.28820#S5a)\.

#### Backpropagation\-based methods

Backpropagation\-based methods are typically post hoc, model\-specific, and local in scope\. They compute gradients of a model’s output with respect to its inputs \(or to intermediate activations\) to identify the pixels, patches, or features that most influence a prediction\. In CompPath, this family produces the familiar heatmap\-style explanations of WSI classifiers and MIL models, ranging from coarse class\-level maps \(CAM, Grad\-CAM\) to fine\-grained, axiomatic attributions \(integrated gradients, LRP, DeepLIFT\)\. Attention\-based methods \(attention weights in MIL, self\-attention in transformers\) are often discussed alongside backpropagation\-based methods because both operate on internal model components, but they differ in an important way: attention is computed during the forward pass as part of the model’s native output, whereas gradient\-based methods are applied post hoc\. We treat attention within this family for organisational convenience but discuss its distinctive properties separately \(Supplementary Section[S3\.1](https://arxiv.org/html/2608.28820#S3.SS1a)\)\. Methods in this family share a common strength, namely, tight coupling to the trained model, and a common weakness, namely, that the gradient signal is correlational rather than causal and is sensitive to scanner, stain, and institutional variation\[[8](https://arxiv.org/html/2608.28820#biba.bib8),[9](https://arxiv.org/html/2608.28820#biba.bib9),[12](https://arxiv.org/html/2608.28820#biba.bib12)\]\. Examples are provided in Supplementary Section[S3\.1](https://arxiv.org/html/2608.28820#S3.SS1a)and Supplementary Table[3](https://arxiv.org/html/2608.28820#S3.T3)\.

#### Perturbation\-based methods

Perturbation\-based methods are typically post hoc and model\-agnostic\. They explain predictions by systematically modifying the input, occluding regions \(occlusion sensitivity\), sampling random masks \(RISE\), or fitting local surrogate models \(LIME\), and observing the resulting change in the model’s output\. Because perturbations are applied at inference time and do not require access to gradients or weights, this family generalizes across architectures, including black\-box vendor models\. The main practical constraints in CompPath are computational, namely thousands of forward passes per WSI for RISE\-style estimators, and the biological realism of the perturbation itself, since zero\-masking or solid\-color occlusion of a WSI patch produces non\-tissue artifacts that distort the resulting attribution\. Supplementary Section[S3\.2](https://arxiv.org/html/2608.28820#S3.SS2a)and Supplementary Table[4](https://arxiv.org/html/2608.28820#S3.T4)describes CompPath\-specific examples for perturbation\-based approaches\.

#### Feature\-based methods

Feature\-based methods attribute predictions to interpretable input features rather than to raw pixels\. They span post hoc and intrinsic designs, model\-specific and model\-agnostic implementations, and local\-to\-global scope\. Two representatives are emphasized in this review: SHAP, which uses Shapley values from cooperative game theory to produce additive attributions over cells, glands, or tissue regions\[[40](https://arxiv.org/html/2608.28820#biba.bib40)\], and activation\-layer visualization \(ALV\), which inspects intermediate feature maps to characterize what a CNN has learned at successive depths\[[56](https://arxiv.org/html/2608.28820#biba.bib56)\]\. Feature\-based methods are particularly useful when models operate on biologically meaningful features \(e\.g\., nuclei, glands, regions\), but their faithfulness depends on how those features are defined and on how the background distribution is sampled\. See Supplementary Section[S3\.3](https://arxiv.org/html/2608.28820#S3.SS3)and Supplementary Table[5](https://arxiv.org/html/2608.28820#S3.T5)for feature\-based methods for CompPath\.

#### Concept\-based methods

Concept\-based methods explain model predictions in terms of human\-interpretable concepts rather than low\-level features\. In CompPath, the relevant concepts are pathologist\-recognizable structures and patterns: necrosis, cribriform glands, lymphocytic infiltrate, tumor border, and mitotic figures\. This family includes post hoc methods that probe an existing model for sensitivity to predefined concepts \(TCAV\[[47](https://arxiv.org/html/2608.28820#biba.bib47)\]\), intrinsic architectures that predict concepts as an intermediate output before the final clinical label \(concept\-bottleneck models\[[37](https://arxiv.org/html/2608.28820#biba.bib37)\]\), and language\-grounded methods that align image regions with diagnostic terminology \(image captioning, vision\-language models such as PLIP and BioViL\[[58](https://arxiv.org/html/2608.28820#biba.bib58),[101](https://arxiv.org/html/2608.28820#biba.bib101)\]\)\. Concept\-based methods trade annotation cost for interpretability gain, namely, concept labels need to be curated, but the resulting explanations are directly auditable by clinicians\. Concept\-based approaches for CompPath are provided in Supplementary Section[S3\.4](https://arxiv.org/html/2608.28820#S3.SS4)and Supplementary Table[6](https://arxiv.org/html/2608.28820#S3.T6)\.

#### Example\-based methods

Example\-based methods explain a prediction by surfacing real cases: representative examples that resemble the input \(prototypes\), hypothetical edits that would change the prediction \(counterfactuals\), or neighborhood\-based reliability estimates \(trust scores\)\. They mirror routine clinical reasoning, in which a pathologist contextualizes an ambiguous case by comparison with familiar reference cases\. Strengths include intuitive case\-based reasoning and natural support for uncertainty triage; limitations include sensitivity to the quality of the underlying embedding space and the difficulty of generating realistic counterfactual histology without violating biological constraints\. Supplementary Section[S3\.5](https://arxiv.org/html/2608.28820#S3.SS5)and Supplementary Table[7](https://arxiv.org/html/2608.28820#S3.T7)describes examples for CompPath\.

## Recommendations for Task\-Driven Application of XAI in CompPath

![Refer to caption](https://arxiv.org/html/2608.28820v1/figs/recommendations_updated_grayscale.png)Figure 3:Task\-driven recommendation framework for selecting XAI methods in CompPath according to “Explanation questions"\. Five clinical questions \(spatial localization, feature attribution, causal validation, concept\-/example\-based reasoning, and decision\-boundary assessment\) map to recommended methods, validation evidence, and deployment use\. All the rows are complementary rather than sequential: spatial localization and feature attribution provide associational evidence, causal validation tests whether highlighted regions influence the model output, concept\- and example\-based methods translate model behaviour into pathologist\-aligned reasoning, and decision\-boundary methods identify unstable, ambiguous, or out\-of\-distribution cases that warrant review\. Validation evidence and deployment requirements for each method family are summarised in Table[3](https://arxiv.org/html/2608.28820#S3.T3)–[7](https://arxiv.org/html/2608.28820#S3.T7)and discussed in Supplementary Section[S4](https://arxiv.org/html/2608.28820#S4a)\.This section focuses providing practical guidance to researchers, clinicians, and regulators, based on the premise that XAI in CompPath is not one\-size\-fits\-all\. The choice of XAI method depends on three factors: the XAI question being asked, the model architecture being explained, and the deployment context \(research exploration, regulatory submission, or clinical integration\)\. Rather than prescribing universal “best” methods, we provide task\-driven recommendations organized around five questions essential for XAI\. Practical, regulatory, and deployment considerations are discussed in Supplementary Section[S4](https://arxiv.org/html/2608.28820#S4a)\. Figure[3](https://arxiv.org/html/2608.28820#S4.F3)provides an overview for selecting XAI methods in CompPath\.

### Spatial localization: Where did the model focus?

Spatial localization verifies the model attends to histologically relevant regions \(tumor epithelium, mitotic figures, immune infiltrates\) rather than artifacts \(pen marks, tissue folds, scanner signatures\)\. Recommended methods include CAM, Grad\-CAM for gradient\-based localization, attention overlays for MIL and transformer models, and LIME or RISE for cases where the model’s internal architecture, weights, or gradients are not accessible \(for example, closed\-source vendor models or third\-party APIs\)\. These approaches provide rapid, intuitive visualization of model focus at the patch level, making them well\-suited for preliminary confirmation that model activations correspond to biologically plausible regions\.

Attention\-weighted aggregation in MIL or transformer models can generate slide\-level maps, while preserving contextual relationships between patches\. Patch\-level heatmaps from Grad\-CAM may be stitched into continuous WSI maps for high\-resolution inspection\. However, these methods primarily reveal correlation rather than causation and may overlook fine\-grained histologic features\. Spatial maps should be validated against expert\-annotated ROIs and assessed for spatial consistency across scanners and institutions\.

### Feature Attribution: How much did each feature contribute to a model prediction?

Quantifying feature importance enables systematic validation and provides numerical evidence for regulatory submissions, unlike qualitative visualization alone\. Recommended methods include Layer\-wise Relevance Propagation \(LRP\), Integrated Gradients \(IG\), and DeepLIFT for high\-fidelity attribution with full model access, xMIL\-LRP for MIL\-based models with tile\-level attribution needs\. SHAP is best reserved for pipelines where the input features are already biologically interpretable \(cell densities, gland counts, morphometric summaries\)\. For end\-to\-end MIL models operating on raw patch embeddings, attention weights, LRP, or perturbation\-based methods are typically more appropriate than SHAP, which lack interpretable feature semantics\. These approaches provide mathematically grounded attribution scores that quantify the relative importance of pixels, patches, or features driving the prediction, enabling systematic assessment of whether the model relies on diagnostically meaningful regions\.

For WSIs, relevance propagation methods such as xMIL\-LRP extend attribution through MIL aggregation layers, while feature\-based summarization approaches \(e\.g\., SHAP\) can aggregate cell\-level or region\-level metrics into global slide\-level importance distributions\. Attribution maps depend on rule selection \(for LRP\) and baseline choice \(for IG and DeepLIFT\), and they are computationally intensive at the WSI scale\. Implementation details, including propagation rules, baseline definitions, and integration steps should be reported, and faithfulness evaluation using insertion/deletion curves or RemOve And Retrain \(ROAR\)\[[19](https://arxiv.org/html/2608.28820#biba.bib19)\]\.

### Causal Validation: Does this region truly influence the decision?

Most localization and attribution methods, including attention maps, Grad\-CAM, saliency maps, LRP, Integrated Gradients, DeepLIFT, and SHAP, identify regions or features associated with a prediction, but do not by themselves establish causal influence\. Interventional or perturbation\-based validation is therefore required before claiming that a highlighted region directly drives the model output\. A region may receive high attribution scores because it correlates with the true causal feature, and not because it directly drives the prediction\. Perturbation methods provide interventional evidence that a region influences the model output under a specified perturbation design\. Recommended methods include Occlusion Sensitivity for conceptually straightforward causal validation and RISE for more stable results through randomized masking aggregation\. When applying perturbation strategies, use biologically plausible perturbations such as Gaussian blurring \(to simulate resolution loss\), inpainting with neighboring tissue \(to maintain texture continuity\), or color jittering \(to simulate stain variation\), while avoiding zero\-masking or solid colors, which can create unrealistic artifacts\.

These gradient\-free methods perturb input regions and directly measure changes in model predictions, providing architecture\-agnostic validation of causal influence\. For WSI, perturbation\-based aggregation can validate tile\-level relevance at slide scale, with efficient batching and stratified sampling improving scalability\. However, the approach is computationally intensive, and naive masking can produce unrealistic artifacts that mislead the model\. Always quantify faithfulness using metrics such as insertion curves, deletion curves, or ROAR to verify that removing high\-importance regions degrades performance more than removing low\-importance regions\.

### Concept\-/Example\-based Reasoning: Why did the model decide this way?

Pixel\-level or patch\-level attribution maps answer “where” and “how much”, but not “why” in clinical terms\. Pathologists reason about concepts \(glandular differentiation, immune infiltration, nuclear pleomorphism\) rather than pixel gradients\. Recommended methods include SHAP for feature\-level attribution, Testing with Concept Activation Vectors \(TCAV\) for quantifying concept influence, Vision\-Language Models \(VLMs\) for zero\-shot concept retrieval, Prototypes for case\-based reasoning, and Counterfactuals for generating hypothetical scenarios\. These methods connect model reasoning to human\-understandable concepts, clinical terminology, or representative examples\. Concept\-based methods link internal representations to quantitative morphometrics, while example\-based approaches provide case\-based evidence aligned with clinical reasoning\.

For WSI, concept overlays from TCAV or VLMs can project semantic sensitivity \(e\.g\., necrosis, gland formation\) spatially across slides, helping pathologists understand which clinical features the model is responding to\. Prototype selection and counterfactual visualization can prioritize diagnostically informative regions for review\. However, these methods require well\-defined feature sets or concept sets\. Poorly defined examples or semantic drift, for example, when a model’s learned concept \(e\.g\., “stroma”\) diverges from the clinical definition, can produce biased or misleading explanations\. Always validate concept grounding by comparing model\-identified concept regions with expert annotations and by testing for consistency across institutions\.

### Decision Boundary: What flips the prediction?

For safe clinical deployment, cases that warrant pathologist review or have ambiguous morphology, out\-of\-distribution artifacts, or borderline diagnostic features must be identified\. Example\-based XAI methods address this by locating a test slide relative to the model’s decision boundary and its training manifold\. Three failure modes are diagnostically distinct\. A case is*unstable*when it sits close to the decision boundary, so small input changes flip the prediction;*counterfactuals*are the relevant probe, since they identify the minimal change that crosses the boundary\. A case is*ambiguous*when it lies in a region of poorly learned manifold with inconsistent local neighbors;*prototypes*are the relevant probe, since the labels of the nearest training cases reveal whether the model has learned a coherent local concept\. A case is effectively*out\-of\-distribution*when no training example is similar;*trust scores*\[[61](https://arxiv.org/html/2608.28820#biba.bib61)\]are the relevant probe, since they quantify distance to neighboring labeled examples in embedding space\.

For WSI applications, approximate nearest\-neighbor indexing enables scalable deployment of prototype retrieval and trust scores, though similarity thresholds must be calibrated separately for each institution and scanner domain to account for distributional shift\[[12](https://arxiv.org/html/2608.28820#biba.bib12)\]\. Example\-based evidence does not by itself explain why a prediction may be unreliable; it should be interpreted alongside attribution or concept\-based evidence \(e\.g\., “which region is the model focusing on?”\)\. Where the deployment context also requires calibrated probabilities of error or formal uncertainty estimates, these are the province of model\-calibration and uncertainty\-quantification methods, which are distinct from XAI and beyond the scope of this review; we refer readers to dedicated CompPath surveys\[[68](https://arxiv.org/html/2608.28820#bib.bib68),[69](https://arxiv.org/html/2608.28820#bib.bib69)\]\.

## Discussion

In this article, we introduced a taxonomy of XAI Methods and offer practical, task\-driven recommendations for researchers, clinicians, and regulators, based on a comprehensive literature review of over 100 XAI CompPath studies \(Table[8](https://arxiv.org/html/2608.28820#S5.T8)\)\. The presented taxonomy offers a two\-fold categorization, by introducing a\) a methodological organization across scale, type, and scope as orthogonal dimensions, and b\) familial grouping based on core methodological principles\. To facilitate these, we also provided clear definitions of the terms explainability, interpretability, justifiability, causality, and transparency, which have been traditionally used interchangeably in the literature\.

The primary use of XAI in CompPath research has been visualization, enabled by its integration with CNN and MIL architectures, producing heatmaps, attention overlays, or prototype examples that make model behavior intuitively accessible\. This is valuable for exploratory analysis and model debugging, but insufficient for clinical deployment or regulatory approval\. visualization answers “where did the model look at?”, but not “is the model reasoning correctly?”\. A heatmap cannot distinguish between spurious correlations and causal reasoning\. Verification requires three components that visualization alone cannot provide:

1. 1\.Causal attribution: Does occluding the highlighted region actually change the prediction? Perturbation\-based methods \(Occlusion Sensitivity, RISE\) provide this, but are computationally expensive for WSI\.
2. 2\.Concept\-level validation: Does the model respond to clinically defined features \(e\.g\., cribriform glands, necrosis, immune infiltration\) or to uninterpretable activation patterns? Concept\-based methods \(TCAV, CBMs\) provide this, but require expert concept annotations that are expensive to curate and prone to inter\-observer variability\.
3. 3\.Cross\-site robustness: Do explanations remain consistent across institutions, scanners, and staining protocols? Current XAI studies rarely test this\. The validation for most is performed on held\-out test sets from the same institution as training data\.

Only a few current CompPath XAI studies jointly evaluate model\-level causal attribution, concept\-level validity, and cross\-site robustness\. Achieving verification requires combining multiple methods such as spatial localization \(Grad\-CAM, attention\), quantitative attribution \(LRP, IG\), perturbation testing \(Occlusion, RISE\), and concept validation \(TCAV, CBMs\) into integrated pipelines\. Section[3](https://arxiv.org/html/2608.28820#S4.F3)provides recommendations for such pipelines\. The shift from visualization to verification is not just a technical challenge, but a cultural one\. Researchers trained to evaluate XAI methods by visual plausibility \(“does the heatmap look reasonable?”\) must adopt quantitative validation \(faithfulness metrics, cross\-site testing, concept grounding\)\. Regulators accustomed to evaluating device performance through sensitivity/specificity must develop frameworks for evaluating explanation quality\. For Pathologists to trust AI outputs they must learn to interrogate not just the predictions, but also the reasoning behind them\. None of these have occurred at scale for clinical verification yet since:

1. 1\.XAI methods are currently evaluated qualitatively through visual inspection of heatmaps, with little quantitative validation\. There is no CompPath equivalent of ImageNet for XAI, no shared dataset, ground\-truth explanations, or standardized evaluation protocol\. What is needed is a curated benchmark dataset, governed as a public utility\[[70](https://arxiv.org/html/2608.28820#bib.bib70)\], spanning multiple anatomies, institutions, scanners, and staining protocols, with expert\-annotated ROIs, multi\-pathologist annotations capturing inter\-observer variability, reference\-standard concept labels, and standardized metrics for faithfulness, stability, and cross\-site robustness\.
2. 2\.Most XAI methods produce pixel\- or patch\-level attributions, but pathologists reason about concepts such as gland architecture, mitotic activity, and immune infiltration\. Validating whether a model has learned clinically relevant concepts requires reference\-standard concept annotations, which are expensive to obtain and prone to inter\-observer variability\[[71](https://arxiv.org/html/2608.28820#bib.bib71)\], and concept\-based XAI methods \(TCAV, CBMs\) remain underutilized as a result\. Addressing this gap requires hierarchical concept taxonomies for common diagnostic tasks, multi\-pathologist annotations with agreement metrics, automated concept discovery, and protocols to detect concept drift across institutions and populations\.
3. 3\.High\-fidelity XAI methods \(LRP, IG, RISE\) are computationally expensive, often requiring hours per slide at a gigapixel scale\. This forces most research onto fast but low\-fidelity alternatives \(Grad\-CAM, attention overlays\) that provide directional insights without quantitative rigor\. Recent advances in foundation models for pathology show that gigapixel\-scale computation is tractable when architectures and algorithms are co\-optimized; applying similar engineering to XAI, together with feature\-space rather than pixel\-space attribution, is achievable\.
4. 4\.Regulatory frameworks \(FDA, CE Mark\) do not currently specify what constitutes sufficient explainability evidence, so developers do not know what to provide, and reviewers do not know what to require\. Closing this gap requires explicit guidance stratified by risk level: qualitative visualization may suffice for low\-risk applications \(e\.g\., slide pre\-screening\), moderate\-risk applications should require quantitative attribution with faithfulness metrics, and high\-risk applications should mandate multi\-method validation with cross\-site robustness testing\. Standardized reporting templates analogous to TRIPOD\[[72](https://arxiv.org/html/2608.28820#bib.bib72),[73](https://arxiv.org/html/2608.28820#bib.bib73)\]and CLEAR\[[74](https://arxiv.org/html/2608.28820#bib.bib74)\], and trained regulatory reviewers, are needed to establish consistent standards\.
5. 5\.XAI outputs are typically presented as standalone visualizations disconnected from clinical workflows\. Pathologists must context\-switch between the diagnostic viewer and the XAI interface, creating friction that limits adoption, and most methods produce per\-slide explanations, although diagnostic decisions integrate multiple slides, clinical history, and ancillary tests\. Practical deployment requires XAI integrated directly into digital pathology viewers, multi\-slide aggregation, unified dashboards combining attribution, concepts, uncertainty, and prototypes, and user studies of how pathologists actually use XAI outputs in diagnostic workflows\.

Two recent shifts deserve specific mention\. First, the rise of pathology foundation models \(FMs\) raises new XAI challenges, rather than solving them\. Tile\-level FMs provide reusable embeddings but do not intrinsically add interpretability, whereas slide\-level FMs \(GigaPath, TITAN\) aggregate over the WSI in ways that make even the modest interpretability of attention\-based MIL less accessible\. Vision\-language models offer promising directions for concept\-grounded retrieval and natural\-language rationales, but they primarily describe what is in an image rather than which image features drive a specific downstream prediction \(e\.g\., survival, treatment response\)\. Concept\-bottleneck approaches point toward closing this gap but strong CompPath examples remain few\. XAI in the FM era therefore requires new methods that operate on high\-dimensional learned embeddings, rather than assuming that model scale itself confers interpretability\. Second, different XAI methods routinely highlight different regions of the same slide\. Disagreement is informative rather than a failure: convergence across methods \(e\.g\., Grad\-CAM, LRP, and perturbation testing all pointing to the same region\) strengthens the evidence, while divergence flags either method\-specific artifacts or genuinely ambiguous reasoning that warrants pathologist review\. We recommend reporting at least one gradient\-based and one perturbation\-based method together, with disagreement explicitly characterized\.

The transition from visualization to verification is the defining challenge for XAI in CompPath\. Heatmaps confirm where a model considered, whereas verification confirms whether it reasoned correctly\. The methods reviewed here provide the technical building blocks, but establishing clinical utility requires cultural shifts among researchers, regulators, and pathologists, cross\-institutional benchmarks, and clear regulatory standards, outlined above\. If these take place collectively and systematically, XAI can improve the adoption of AI in pathology, along with earning the trust required to scale it\.

#### Contributors

SI and SB conceived the review\. SI led the literature analysis\. SY provided substantial writing support across multiple sections of the original manuscript and contributed detailed constructive feedback during revision\. SB supervised the work and oversaw the development of the final taxonomy\. All other authors reviewed, provided constructive feedback, edited, and approved the content and structure final version of the manuscript\.

#### Declaration of interests

We declare no competing interests\.

#### Data sharing

This review does not generate new primary data\. All studies discussed are publicly available through their original publications\.

## References

- \[1\]Gurcan MN, Boucheron LE, Can A, Madabhushi A, Rajpoot NM, Yener B\.Histopathological image analysis: A review\.IEEE reviews in biomedical engineering\. 2009;2:147\-71\.
- \[2\]Innani S, Nasrallah MP, Bell WR, Baheti B, Bakas S\.Multi\-scale Whole Slide Image Assessment Improves Deep Learning based WHO 2021 Glioma Classification\.In: Proceedings of the MICCAI Workshop on Computational Pathology\. vol\. 254 of Proceedings of Machine Learning Research\. PMLR; 2024\. p\. 142\-53\.
- \[3\]Collins K, Innani S, Ebare K, Saad M, Siegmund SE, Williamson SR, et al\.Artificial Intelligence–Based Classification of Renal Oncocytic Neoplasms: Advancing From a 2\-Class Model of Renal Oncocytoma and Low\-Grade Oncocytic Tumor to a 3\-Class Model Including Chromophobe Renal Cell Carcinoma\.Archives of Pathology & Laboratory Medicine\. 2025\.
- \[4\]Innani S, Bell WR, Nasrallah MP, Baheti B, Bakas S\.AI\-driven WHO 2021 classification of gliomas based only on H&E\-stained slides\.Neuro\-Oncology\. 2025:noaf189\.
- \[5\]Innani S, Bell WR, Nasrallah M, Baheti B, Bakas S\.Artificial intelligence predicts 2021 WHO glioma subtypes from whole slide images\.Cancer Research\. 2025;85\(8\_Supplement\_1\):6247\-7\.
- \[6\]Baheti B, Innani S, Mehdiratta G, Nasrallah MP, Bakas S\.P13\.13\.B INTERPRETABLE WHOLE SLIDE IMAGE PROGNOSTIC STRATIFICATION OF GLIOBLASTOMA PATIENTS FURTHERING DISEASE UNDERSTANDING\.Neuro\-Oncology\. 2023 09;25\(Supplement\_2\):ii103\-4\.
- \[7\]Baheti B, Innani S, Nasrallah MP, Bakas S\.OS03\.6\.A UNSUPERVISED CLUSTERING OF MORPHOLOGY PATTERNS ON WHOLE SLIDE IMAGES GUIDE PROGNOSTIC STRATIFICATION OF GLIOBLASTOMA PATIENTS\.Neuro\-Oncology\. 2023 09;25\(Supplement\_2\):ii15\-5\.
- \[8\]Baheti B, Rai S, Innani S, Mehdiratta G, Bell WR, Guntuku SC, et al\.Multimodal Explainable Artificial Intelligence for Prognostic Stratification of Glioblastoma Patients\.Modern Pathology\. 2025:100797\.
- \[9\]Baheti B, Innani S, Nasrallah M, Bakas S\.Prognostic stratification of glioblastoma patients by unsupervised clustering of morphology patterns on whole slide images furthering our disease understanding\.Frontiers in Neuroscience\. 2024;18:1304191\.
- \[10\]Innani S, Baheti B, Nasrallah MP, Bakas S\.Weakly Supervised IDH\-Status Glioma Classification from H&E\-Stained Whole Slide Images\.In: 2024 IEEE International Symposium on Biomedical Imaging \(ISBI\)\. IEEE; 2024\. p\. 1\-5\.
- \[11\]Innani S, Baheti B, Nasrallah MP, Bakas S\.PATH\-39\. INTERPRETABLE IDH CLASSIFICATION FROM H&E\-STAINED HISTOLOGY SLIDES\.Neuro\-Oncology\. 2023 11;25\(Supplement\_5\):v177\-7\.
- \[12\]Innani S, Baheti B, Nasrallah MP, Bell WR, Bakas S\.PATH\-39\. AI\-BASED IDENTIFICATION OF GLIOMA IDH MUTATIONAL STATUS FROM H&;E\-STAINED WHOLE SLIDE IMAGES\.Neuro\-Oncology\. 2024 11;26\(Supplement\_8\):viii187\-7\.
- \[13\]Innani S, Bell WR, Harmsen H, Nasrallah MP, Baheti B, Bakas S\.Interpretable artificial intelligence based determination of glioma IDH mutation status directly from histology slides\.Neuro\-Oncology Advances\. 2025;7\(1\):vdaf140\.
- \[14\]Innani S, Pitarch\-Abaigar C, Marwan MM, Harmsen H, Bell WR, Makris D, et al\.PATH\-69\. AI\-based prognostic risk stratification of adult\-type diffuse gliomas using H&E\-stained slides\.Neuro\-Oncology\. 2025;27\(Supplement\_5\):v257\-7\.
- \[15\]Graham S, Minhas F, Bilal M, Ali M, Tsang YW, Eastwood M, et al\.Screening of normal endoscopic large bowel biopsies with interpretable graph learning: a retrospective study\.Gut\. 2023;72\(9\):1709\-21\.
- \[16\]Innani S, Bell WR, Nasrallah MP, Baheti B, Bakas S\.PATH\-67\. AI\-based WHO 2021 classification of adult\-type diffuse glioma solely from H&E\-stained slides\.Neuro\-Oncology\. 2025;27\(Supplement\_5\):v256\-6\.
- \[17\]Thakur S, Malec S, Pitarc C, Linardos A, Innani S, Adap S, et al\.IMG\-121\. BraTS\-Pathology 2024: Insights and Future Directions Informed by the AI\-RANO & RANO\-RGP Effort to Assess Glioblastoma Heterogeneity\.Neuro\-Oncology\. 2025;27\(Supplement\_5\):v303\-4\.
- \[18\]Thakur S, Faghani S, Moassefi M, Baid U, Chung V, Pati S, et al\.TMIC\-60\. BRATS\-PATH: Assessing Heterogeneous Histopathologic Regions in Glioblastoma\.Neuro\-Oncology\. 2024;26\(Supplement\_8\):viii312\-2\.
- \[19\]Matthews GA, McGenity C, Bansal D, Treanor D\.Public evidence on AI products for digital pathology\.NPJ Digital Medicine\. 2024;7\(1\):300\.
- \[20\]Liu Y, Yu W, Dillon T\.Regulatory responses and approval status of artificial intelligence medical devices with a focus on China\.NPJ Digital Medicine\. 2024;7\(1\):255\.
- \[21\]Schmidt J, Schutte NM, Buttigieg S, Novillo\-Ortiz D, Sutherland E, Anderson M, et al\.Mapping the regulatory landscape for artificial intelligence in health within the European Union\.npj Digital Medicine\. 2024;7\(1\):229\.
- \[22\]Poalelungi DG, Neagu AI, Fulga A, Neagu M, Tutunaru D, Nechita A, et al\.Revolutionizing pathology with artificial intelligence: Innovations in immunohistochemistry\.Journal of Personalized Medicine\. 2024;14\(7\):693\.
- \[23\]McGenity C, Clarke EL, Jennings C, Matthews G, Cartlidge C, Stocken DD, et al\.Artificial intelligence in digital pathology: a systematic review and meta\-analysis of diagnostic test accuracy\.npj Digital Medicine\. 2024;7\(1\):114\.
- \[24\]Scans CH\.The Paige Prostate Suite: Assistive Artificial Intelligence for Prostate Cancer Diagnosis: Emerging Health Technologies\.Ottawa \(ON\): Canadian Agency for Drugs and Technologies in Health\. 2024\.
- \[25\]Pantanowitz L, Quiroga\-Garza GM, Bien L, Heled R, Laifenfeld D, Linhart C, et al\.An artificial intelligence algorithm for prostate cancer diagnosis in whole slide images of core needle biopsies: a blinded clinical validation and deployment study\.The Lancet Digital Health\. 2020;2\(8\):e407\-16\.
- \[26\]Van der Laak J, Litjens G, Ciompi F\.Deep learning in histopathology: the path to the clinic\.Nature medicine\. 2021;27\(5\):775\-84\.
- \[27\]Health I\. Hallucinating AI Perfection in Healthcare: Navigating the Challenge of Hallucinations\. Medium; 2024\.Available from:[https://inflecthealth\.medium\.com/hallucinating\-ai\-perfection\-in\-healthcare\-navigating\-the\-challenge\-of\-hallucinations\-4e052a4492e5](https://inflecthealth.medium.com/hallucinating-ai-perfection-in-healthcare-navigating-the-challenge-of-hallucinations-4e052a4492e5)\.
- \[28\]Jahanifar M, Raza M, Xu K, Vuong TTL, Jewsbury R, Shephard A, et al\.Domain generalization in computational pathology: survey and guidelines\.ACM Computing Surveys\. 2025;57\(11\):1\-37\.
- \[29\]Evans T, Retzlaff CO, Geißler C, Kargl M, Plass M, Müller H, et al\.The explainability paradox: Challenges for xAI in digital pathology\.Future Generation Computer Systems\. 2022;133:281\-96\.
- \[30\]Aldea M, Salto\-Tellez M, Marra A, Umeton R, Stenzinger A, Koopman M, et al\.ESMO basic requirements for AI\-based biomarkers in oncology \(EBAI\)\.Annals of Oncology\. 2025\.
- \[31\]Ghassemi M, Oakden\-Rayner L, Beam AL\.The false hope of current approaches to explainable artificial intelligence in health care\.The lancet digital health\. 2021;3\(11\):e745\-50\.
- \[32\]Dwivedi R, Dave D, Naik H, Singhal S, Omer R, Patel P, et al\.Explainable AI \(XAI\): Core ideas, techniques, and solutions\.ACM Computing Surveys\. 2023;55\(9\):1\-33\.
- \[33\]Doshi\-Velez F, Kim B\.Towards a rigorous science of interpretable machine learning\.arXiv preprint arXiv:170208608\. 2017\.
- \[34\]Holzinger A, Biemann C, Pattichis CS, Kell DB\.What do we need to build explainable AI systems for the medical domain?arXiv preprint arXiv:171209923\. 2017\.
- \[35\]Hassija V, Chamola V, Mahapatra A, Singal A, Goel D, Huang K, et al\.Interpreting black\-box models: a review on explainable artificial intelligence\.Cognitive Computation\. 2024;16\(1\):45\-74\.
- \[36\]Arrieta AB, Díaz\-Rodríguez N, Del Ser J, Bennetot A, Tabik S, Barbado A, et al\.Explainable Artificial Intelligence \(XAI\): Concepts, taxonomies, opportunities and challenges toward responsible AI\.Information fusion\. 2020;58:82\-115\.
- \[37\]Lipton ZC\.The mythos of model interpretability: In machine learning, the concept of interpretability is both important and slippery\.Queue\. 2018;16\(3\):31\-57\.
- \[38\]Adadi A, Berrada M\.Peeking inside the black\-box: a survey on explainable artificial intelligence \(XAI\)\.IEEE access\. 2018;6:52138\-60\.
- \[39\]Mueller ST, Hoffman RR, Clancey W, Emrey A, Klein G\.Explanation in human\-AI systems: A literature meta\-review, synopsis of key ideas and publications, and bibliography for explainable AI\.arXiv preprint arXiv:190201876\. 2019\.
- \[40\]Murdoch WJ, Singh C, Kumbier K, Abbasi\-Asl R, Yu B\.Definitions, methods, and applications in interpretable machine learning\.Proceedings of the National Academy of Sciences\. 2019;116\(44\):22071\-80\.
- \[41\]Ibrahim R, Shafiq MO\.Explainable convolutional neural networks: a taxonomy, review, and future directions\.ACM Computing Surveys\. 2023;55\(10\):1\-37\.
- \[42\]Das A, Rad P\.Opportunities and challenges in explainable artificial intelligence \(xai\): A survey\.arXiv preprint arXiv:200611371\. 2020\.
- \[43\]Bhati D, Neha F, Amiruzzaman M\.A survey on explainable artificial intelligence \(xai\) techniques for visualizing deep learning models in medical imaging\.Journal of Imaging\. 2024;10\(10\):239\.
- \[44\]Van der Velden BH, Kuijf HJ, Gilhuijs KG, Viergever MA\.Explainable artificial intelligence \(XAI\) in deep learning\-based medical image analysis\.Medical Image Analysis\. 2022;79:102470\.
- \[45\]Pocevičiūtė M, Eilertsen G, Lundström C\.Survey of XAI in digital pathology\.Artificial intelligence and machine learning for digital pathology: state\-of\-the\-art and future challenges\. 2020:56\-88\.
- \[46\]Dy A, Buetow SM, Bredemeyer AJ, Saini ML, Lucas F, Bennett S, et al\.Clarifying validation terminologies in healthcare\.NPJ Digital Medicine\. 2026;9\(1\):318\.
- \[47\]Hedström A, Weber L, Krakowczyk D, Bareeva D, Motzkus F, Samek W, et al\.Quantus: an explainable AI toolkit for responsible evaluation of neural network explanations and beyond\.Journal of Machine Learning Research\. 2023;24\(34\):1\-11\.
- \[48\]European Parliament and Council\. Regulation \(EU\) 2024/1689 of the European Parliament and of the Council of 13 June 2024 laying down harmonised rules on artificial intelligence; 2024\.Official Journal of the European Union\.Available from:[https://eur\-lex\.europa\.eu/eli/reg/2024/1689/oj](https://eur-lex.europa.eu/eli/reg/2024/1689/oj)\.
- \[49\]Food U, gov\) DA\.Artificial Intelligence and Machine Learning in Software as a Medical Device\. 2021\.Accessed on 22 Sept 2021\.Available from:[https://www\.fda\.gov/files/medical%20devices/published/US\-FDA\-Artificial\-Intelligence\-and\-Machine\-Learning\-Discussion\-Paper\.pdf](https://www.fda.gov/files/medical%20devices/published/US-FDA-Artificial-Intelligence-and-Machine-Learning-Discussion-Paper.pdf)\.
- \[50\]Holzinger A, Langs G, Denk H, Zatloukal K, Müller H\.Causability and explainability of artificial intelligence in medicine\.Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery\. 2019;9\(4\):e1312\.
- \[51\]Campbell DT\.Relabeling internal and external validity for applied social scientists\.New Directions for Program Evaluation\. 1986;1986\(31\):67\-77\.
- \[52\]West SG, Thoemmes F\.Campbell’s and Rubin’s perspectives on causal inference\.In: Causality in the Sciences; 2010\. Spencer Foundation conference paper; available at https://jenni\.uchicago\.edu/Spencer\_Conference/\.
- \[53\]Adebayo J, Gilmer J, Muelly M, Goodfellow I, Hardt M, Kim B\.Sanity checks for saliency maps\.Advances in neural information processing systems\. 2018;31\.
- \[54\]Hooker S, Erhan D, Kindermans PJ, Kim B\.A benchmark for interpretability methods in deep neural networks\.Advances in neural information processing systems\. 2019;32\.
- \[55\]Mittelstadt B\.Principles alone cannot guarantee ethical AI\.Nature machine intelligence\. 2019;1\(11\):501\-7\.
- \[56\]Graziani M, Andrearczyk V, Marchand\-Maillet S, Müller H\.Concept attribution: Explaining CNN decisions to physicians\.Computers in biology and medicine\. 2020;123:103865\.
- \[57\]Rueda J, Rodríguez JD, Jounou IP, Hortal\-Carmona J, Ausín T, Rodríguez\-Arias D\.“Just” accuracy? Procedural fairness demands explainability in AI\-based medical resource allocations\.AI & society\. 2024;39\(3\):1411\-22\.
- \[58\]Kamath U, Liu J\.Explainable artificial intelligence: an introduction to interpretable machine learning\. vol\. 2\.Springer; 2021\.
- \[59\]Jain S, Wallace BC\.Attention is not explanation\.arXiv preprint arXiv:190210186\. 2019\.
- \[60\]Wiegreffe S, Pinter Y\.Attention is not not explanation\.arXiv preprint arXiv:190804626\. 2019\.
- \[61\]Lundberg SM, Lee SI\.A unified approach to interpreting model predictions\.Advances in neural information processing systems\. 2017;30\.
- \[62\]Yosinski J, Clune J, Nguyen A, Fuchs T, Lipson H\.Understanding neural networks through deep visualization\.arXiv preprint arXiv:150606579\. 2015\.
- \[63\]Kim B, Wattenberg M, Gilmer J, Cai C, Wexler J, Viegas F, et al\.Interpretability beyond feature attribution: Quantitative testing with concept activation vectors \(tcav\)\.In: International conference on machine learning\. PMLR; 2018\. p\. 2668\-77\.
- \[64\]Koh PW, Nguyen T, Tang YS, Mussmann S, Pierson E, Kim B, et al\.Concept bottleneck models\.In: International conference on machine learning\. PMLR; 2020\. p\. 5338\-48\.
- \[65\]Zuo J, Hong J, Zhang F, Yu C, Zhou H, Gao C, et al\.Plip: Language\-image pre\-training for person representation learning\.Advances in Neural Information Processing Systems\. 2024;37:45666\-702\.
- \[66\]Bannur S, Hyland S, Liu Q, Perez\-Garcia F, Ilse M, Castro DC, et al\.Learning to exploit temporal structure for biomedical vision\-language processing\.In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition; 2023\. p\. 15016\-27\.
- \[67\]Jiang H, Kim B, Guan M, Gupta M\.To trust or not to trust a classifier\.Advances in neural information processing systems\. 2018;31\.
- \[68\]Abdar M, Pourpanah F, Hussain S, Rezazadegan D, Liu L, Ghavamzadeh M, et al\.A review of uncertainty quantification in deep learning: Techniques, applications and challenges\.Information fusion\. 2021;76:243\-97\.
- \[69\]He W, Jiang Z, Xiao T, Xu Z, Li Y\.A survey on uncertainty quantification methods for deep learning\.ACM Computing Surveys\. 2026;58\(7\):1\-35\.
- \[70\]Haendel MA, Ahern R, Bailey KB, Bakas S, Barth\-Jones DC, Bohl A, et al\.Governing real\-world health data as a public utility\.Science\. 2026;391\(6789\):993\-6\.
- \[71\]Bakas S, Thakur SP, Faghani S, Moassefi M, Baid U, Chung V, et al\.Brats\-path challenge: Assessing heterogeneous histopathologic brain tumor sub\-regions\.arXiv preprint arXiv:240510871\. 2024\.
- \[72\]Moons KG, Altman DG, Reitsma JB, Ioannidis JP, Macaskill P, Steyerberg EW, et al\.Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis \(TRIPOD\): explanation and elaboration\.Annals of internal medicine\. 2015;162\(1\):W1\-W73\.
- \[73\]Collins GS, Reitsma JB, Altman DG, Moons KG\.Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis \(TRIPOD\): the TRIPOD statement\.Journal of British Surgery\. 2015;102\(3\):148\-58\.
- \[74\]Kocak B, Baessler B, Bakas S, Cuocolo R, Fedorov A, Maier\-Hein L, et al\.CheckList for EvaluAtion of Radiomics research \(CLEAR\): a step\-by\-step reporting guideline for authors and reviewers endorsed by ESR and EuSoMII\.Insights into imaging\. 2023;14\(1\):75\.

## Supplementary Material

This supplementary material provides expanded detail on the four components of the main paper that require depth beyond: \(i\) the seven pathology\-centric XAI term definitions \(Section[S1](https://arxiv.org/html/2608.28820#S1a)\), \(ii\) the three\-axis taxonomy sub\-dimensions and stage/type/scope categorisation table \(Section[S2](https://arxiv.org/html/2608.28820#S2a)\), \(iii\) per\-method narrative descriptions across the five method families \(Section[S3](https://arxiv.org/html/2608.28820#S3a)\), and \(iv\) a survey of over 100 CompPath studies organised by anatomy and XAI method \(Section[S5](https://arxiv.org/html/2608.28820#S5a)\)\. Section[S4](https://arxiv.org/html/2608.28820#S4a)provides extended deployment guidance not included in the main recommendations section\.

## Pathology\-Centric Definitions of Seven XAI Terms

- Interpretabilityin deep learning tools for CompPath refers to the degree to which a human user can understand and reason about a model’s input\-output behavior directly from the model’s structure, learned components, or native outputs, without requiring any additional explanation to be generated\[[1](https://arxiv.org/html/2608.28820#biba.bib1),[2](https://arxiv.org/html/2608.28820#biba.bib2),[3](https://arxiv.org/html/2608.28820#biba.bib3)\]\. Interpretability is a non\-binary property of the model that depends on the model class, feature representation, task complexity, scale, and user expertise\. Sparse linear models, decision trees, generalized additive models, concept bottleneck models, prototype\-based models, and models using pathologist\-defined semantic features may be interpretable when their components are inspectable, and their behavior can be traced\. For example, a logistic regression predicting tumor grade from nuclear area, mitotic count, and gland density is interpretable because each feature’s contribution can be examined directly\. Models that output pathologist\-recognizable quantities such as a mitotic count, a Gleason score, a nuclear grade, a TIL density, or a segmented gland map are interpretable at the level of their outputs, because the prediction itself is a quantity a pathologist already reasons about\[[4](https://arxiv.org/html/2608.28820#biba.bib4),[5](https://arxiv.org/html/2608.28820#biba.bib5),[6](https://arxiv.org/html/2608.28820#biba.bib6)\]\. In CompPath applications, however, WSIs are gigapixel inputs and models routinely have hundreds of millions to billions of parameters\[[7](https://arxiv.org/html/2608.28820#biba.bib7)\], so most deep architectures used for tasks such as whole\-slide classification, biomarker prediction, or survival analysis are not directly interpretable\. Attention\-based MIL and transformer attention maps are partially interpretable: attention weights indicate which patches of a WSI were associated with the prediction and can be inspected without an additional method, but they do not by themselves provide faithful causal attribution\[[8](https://arxiv.org/html/2608.28820#biba.bib8),[9](https://arxiv.org/html/2608.28820#biba.bib9)\]\. Interpretability is evaluated by inspecting the model itself: whether a pathologist can trace, from the model’s structure, learned components, or native outputs, how the input maps to the prediction\. Unlike the criteria that follow, no additional explanation artifact is generated or evaluated; the model itself is the object of evaluation\.
- Explainabilityrefers to the capacity to provide faithful, context\-appropriate evidence for a model output in a form understandable to the intended user\[[2](https://arxiv.org/html/2608.28820#biba.bib2),[10](https://arxiv.org/html/2608.28820#biba.bib10)\]\. The key distinction from interpretability is what is being evaluated: interpretability is a property of the model itself, whereas explainability is a property of an additional artifact \(a heatmap, attribution score, segmentation, concept score, or natural\-language rationale\) that is delivered alongside the prediction\. The artifact may be produced by the model during inference or by a separate method applied after training; we defer that distinction to the Stage axis of the taxonomy[S2](https://arxiv.org/html/2608.28820#S2a)and use “explainability” here for the artifact itself, regardless of how it is generated\. For a WSI\-level Gleason 4 prediction, an explanation artifact might be a Grad\-CAM heatmap that localizes the regions most associated with the prediction, a calibrated probability that quantifies confidence, a segmentation map of glandular structures, or a set of concept\-level scores indicating which pathologist\-recognizable patterns \(e\.g\., cribriform architecture, glandular fusion\) the model relied on\. Several such artifacts can accompany the same prediction, each answering a different clinical question\. Because an explanation is an artifact added on top of the prediction rather than the prediction itself, it can be evaluated independently of the model\. Two methods applied to the same model can highlight different regions of the same WSI, and explanation artifacts have been shown to be sensitive to scanner, staining, and institutional variation in pathology data\[[11](https://arxiv.org/html/2608.28820#biba.bib11),[12](https://arxiv.org/html/2608.28820#biba.bib12)\]\. Evaluation criteria specific to explanation artifacts includefidelity, namely whether the artifact actually reflects the model’s behavior \(e\.g\., whether removing highlighted regions degrades the prediction\);stability, namely whether the same model yields consistent artifacts across stains, scanners, and institutions; andclinical plausibility, namely whether the artifact aligns with pathologist judgment about what the model should be attending to\. These criteria apply to the explanation artifact as an object of evaluation; interpretability, causal validity, and causability are evaluated against different objects \(the model, the prediction\-feature link, and the user’s clinical understanding respectively\)\.
- Transparencyin CompPath refers to the accessibility of information needed to scrutinize, reproduce, monitor, and govern an AI system across its lifecycle\[[1](https://arxiv.org/html/2608.28820#biba.bib1),[13](https://arxiv.org/html/2608.28820#biba.bib13),[14](https://arxiv.org/html/2608.28820#biba.bib14)\]\. Unlike interpretability and explainability, which are technical properties of the model or its accompanying evidence, transparency is a property of the system and is delivered through documentation rather than through any computation on a specific case\. It encompasses model architecture, training data composition \(the number and diversity of WSIs, scanners, staining protocols, and patient demographics\), preprocessing pipelines \(tissue segmentation, stain normalization, patch extraction\), reference standards used for ground truth labeling, intended use, subgroup performance, known failure modes, and procedures for post\-deployment monitoring\. A pathology AI system can be highly explainable yet not transparent: a vendor product may produce attention\-based heatmaps over each WSI while withholding the architecture, training cohort composition, and validation cohort demographics, making it impossible to assess whether the model’s training population matches the deploying institution’s case mix\. Conversely, a system can be transparent yet not interpretable: a publicly released foundation model with billions of parameters\[[7](https://arxiv.org/html/2608.28820#biba.bib7)\]may have its architecture, training data, and weights fully disclosed and yet remain opaque to direct human inspection\. Transparency requirements operate at two levels\. At the technical level, they govern what a developer or independent researcher can reconstruct about a model’s design, data, and behaviour from publicly available documentation\. At the regulatory and legal level, they govern what a manufacturer must disclose to authorities and deploying institutions; the U\.S\. Food and Drug Administration’s Predetermined Change Control Plan framework\[[13](https://arxiv.org/html/2608.28820#biba.bib13)\]and the EU AI Act’s high\-risk device requirements\[[14](https://arxiv.org/html/2608.28820#biba.bib14)\]both mandate disclosure of training data, intended use, subgroup performance characteristics, and change\-management procedures\. The two levels overlap but are not identical: a system can meet regulatory disclosure requirements while remaining technically opaque to external researchers, and a system can be technically transparent \(e\.g\., open weights and training code\) without yet meeting the structured disclosure expectations of a specific regulatory framework\. Evaluation of transparency is based on the sufficiency, accessibility, and verifiability of the disclosure relative to the system’s intended use and risk level\[[15](https://arxiv.org/html/2608.28820#biba.bib15),[16](https://arxiv.org/html/2608.28820#biba.bib16),[17](https://arxiv.org/html/2608.28820#biba.bib17)\], rather than on any single property of the model’s output on a specific case\.
- Causal validity\.Causal validity in computational pathology refers to whether the features or regions highlighted by an explanation actually influence the model’s prediction, as opposed to merely correlating with it\[[18](https://arxiv.org/html/2608.28820#biba.bib18),[19](https://arxiv.org/html/2608.28820#biba.bib19)\]\. Causal validity is a property of the*explanation*, supported by interventional evidence, and is distinct from both technical fidelity and clinical plausibility: a method can produce a visually plausible heatmap over a tumor region that has no causal effect on the prediction\[[18](https://arxiv.org/html/2608.28820#biba.bib18),[8](https://arxiv.org/html/2608.28820#biba.bib8)\]\. Establishing causal validity requires perturbation\- or counterfactual\-based testing on the actual WSI or TMA input: occluding a region highlighted by Grad\-CAM and verifying that the prediction degrades, performing insertion or deletion curves over patches ranked by attribution score, or generating counterfactual histology images in which a candidate feature \(e\.g\., mitotic figures, cribriform glands\) is removed or synthetically modified and observing whether the prediction changes\[[20](https://arxiv.org/html/2608.28820#biba.bib20),[21](https://arxiv.org/html/2608.28820#biba.bib21)\]\. In CompPath, causal validity is particularly important because models trained on WSIs can latch onto non\-causal cues such as staining batch, scanner color profile, slide background, or pen marks\[[22](https://arxiv.org/html/2608.28820#biba.bib22)\]\. An explanation that highlights cribriform architecture but is unchanged when that architecture is occluded indicates a causal failure: the model is using something else, and the explanation is misleading\. Evaluation of causal validity uses interventional metrics such as Area Over the Perturbation Curve \(AOPC\), insertion/deletion AUC, RemOve And Retrain \(ROAR\), and counterfactual fidelity\[[23](https://arxiv.org/html/2608.28820#biba.bib23),[19](https://arxiv.org/html/2608.28820#biba.bib19)\]\. Causal validity is internal to the model: it answers whether the explanation reflects the model’s own decision process, not whether the highlighted feature is biologically causal for disease\.
- Causability\.Causability in computational pathology refers to the extent to which an explanation enables a pathologist to form a useful, clinically meaningful causal understanding of the model’s output\[[24](https://arxiv.org/html/2608.28820#biba.bib24)\]\. Whereas causal validity is internal to the model \(does the explanation reflect the model’s actual decision process?\), causability is a property of the*explanation*as evaluated by a human expert: does the explanation map to recognized histopathological features and biological mechanisms in a way that the pathologist can verify against domain knowledge? An explanation with high causal validity can still have low causability if it highlights regions or features with no established pathological meaning\. For example, a survival model applied to glioblastoma WSIs may faithfully attribute its prediction to a region of dense pseudopalisading necrosis \(high causal validity\), and the pathologist recognizes pseudopalisading necrosis as an established adverse prognostic feature in high\-grade gliomas \(high causability\)\. In contrast, the same model attributing its prediction to slide edges, tissue\-folding artifacts, scanner\-specific color casts, or out\-of\-tissue background regions has zero causability regardless of the technical fidelity of the attribution\[[25](https://arxiv.org/html/2608.28820#biba.bib25)\]\. Causability is therefore the bridge between technical explanation and clinical trust: it depends on the explanation aligning with pathologist\-recognized structures such as nuclear atypia, gland formation, mitotic figures, lymphocytic infiltrate, or specific architectural patterns \(cribriform, papillary, solid\)\. Evaluation of causability requires pathologist input: reader studies in which pathologists rate the agreement between explanations and known histopathological features, comparison of model\-highlighted regions against expert\-annotated regions of interest, and structured assessment of whether explanations align with established grading criteria such as the Gleason, Nottingham, or WHO systems\[[26](https://arxiv.org/html/2608.28820#biba.bib26)\]\.
- Justifiability\.Justifiability in computational pathology refers to whether the deployment of an AI system and the use of its outputs can be reasonably defended within established clinical and regulatory standards\[[15](https://arxiv.org/html/2608.28820#biba.bib15),[27](https://arxiv.org/html/2608.28820#biba.bib27),[28](https://arxiv.org/html/2608.28820#biba.bib28),[29](https://arxiv.org/html/2608.28820#biba.bib29)\]\. Justifiability is a property of the*governance*surrounding deployment, not of the model or its explanations in isolation\. It depends on the strength of validation evidence \(analytical validation on WSI cohorts, clinical validation against pathologist diagnoses, and documented performance across scanners, stains, and patient populations\[[12](https://arxiv.org/html/2608.28820#biba.bib12),[30](https://arxiv.org/html/2608.28820#biba.bib30)\]\); fairness across demographic groups; compliance with regulatory frameworks \(FDA clearance for the intended use, CE marking under the EU IVDR/MDR, or laboratory accreditation for laboratory\-developed tests under CLIA\); and alignment with the standard of care for the diagnostic task at hand\. A heatmap may support a clinical justification, but it cannot by itself justify clinical deployment or patient\-level action\. Evaluation of justifiability relies on the completeness and quality of the validation, documentation, and regulatory dossier rather than on the model’s output in any given case\.
- Contestability\.Contestability in computational pathology refers to the ability of pathologists, patients, and other stakeholders to inspect, challenge, override, or appeal the outputs of an AI system\[[15](https://arxiv.org/html/2608.28820#biba.bib15),[14](https://arxiv.org/html/2608.28820#biba.bib14)\]\. Contestability is a property of the*workflow*into which the system is deployed, encompassing user\-facing mechanisms \(e\.g\., a one\-click override of an AI\-suggested Gleason grade, mitotic count, or molecular subtype prediction, with the disagreement recorded in the diagnostic report\), patient\- and clinician\-facing mechanisms \(the ability to request human re\-review of an AI\-augmented diagnosis\), and audit trails that permit regulators or quality teams to investigate specific cases retrospectively\. Contestability requires that explanations be available, that overrides be possible without workflow penalty, and that disagreements feed back into model monitoring\. The EU AI Act explicitly requires contestability for high\-risk AI systems\[[14](https://arxiv.org/html/2608.28820#biba.bib14)\]\. Evaluation of contestability assesses the availability and usability of these mechanisms within the clinical workflow\.

## XAI Taxonomy for CompPath: Three Orthogonal XAI Dimensions

We organize XAI methods along three orthogonal dimensions: stage, type, and scope \(Figure[2\(a\)](https://arxiv.org/html/2608.28820#S3.F2.sf1)\. This framework provides a structured basis for comparing XAI approaches and clarifying common terminology in the literature\[[31](https://arxiv.org/html/2608.28820#biba.bib31),[32](https://arxiv.org/html/2608.28820#biba.bib32),[33](https://arxiv.org/html/2608.28820#biba.bib33)\]\.

### Stage of XAI

The Stage axis distinguishes methods that are built into the model’s own computation \(intrinsic\) from methods that are applied afterwards to an already\-trained model \(post hoc\)\.

#### Intrinsic

Intrinsic XAI is achieved by models whose decision\-making processes can be understood directly from their structure\. These methods explicitly link inputs to outputs, making them transparent without additional explanation\. Examples include linear models, decision trees, and generalized additive models\[[33](https://arxiv.org/html/2608.28820#biba.bib33)\]\. In CompPath, intrinsic XAI has been explored using feature\-specific models that predict outcomes based on predefined histological attributes\. For instance, methods that quantify tumor\-infiltrating lymphocytes \(TILs\)\[[34](https://arxiv.org/html/2608.28820#biba.bib34)\]or compute mitotic counts\[[35](https://arxiv.org/html/2608.28820#biba.bib35)\]both of which return quantities a pathologist already reasons about\. Recent neural\-network designs have introduced forms of intrinsic interpretability through self\-explaining networks\[[36](https://arxiv.org/html/2608.28820#biba.bib36)\], concept\-bottleneck and concept\-aware architectures\[[37](https://arxiv.org/html/2608.28820#biba.bib37)\], and hybrid rule\-based designs\[[38](https://arxiv.org/html/2608.28820#biba.bib38)\]\.

Attention\-based multiple instance learning \(MIL\) sits in an intermediate position\[[39](https://arxiv.org/html/2608.28820#biba.bib39)\]\. Attention weights are computed during the model’s forward pass and are available without an additional explanation procedure, which makes their generation intrinsic in the procedural sense\. However, attention weights have been shown to not necessarily reflect feature importance in a faithful or causal way\[[8](https://arxiv.org/html/2608.28820#biba.bib8),[9](https://arxiv.org/html/2608.28820#biba.bib9)\], so their use as an interpretability claim requires the same kind of faithfulness, stability, and clinical plausibility evaluation that post hoc explanations require\. In this review, we therefore describe attention\-based MIL as a widely used mechanism for visualizing model focus, but not as a sufficient substitute for explicit explainability evaluation\.

#### Post hoc

Post hoc methods generate explanations by applying a separate procedure to an already\-trained model\. They are particularly relevant for deep architectures where the model’s internal computation is not directly inspectable\. Common families include feature attribution methods such as SHAP\[[40](https://arxiv.org/html/2608.28820#biba.bib40)\], gradient\-based visualization methods such as Grad\-CAM\[[41](https://arxiv.org/html/2608.28820#biba.bib41)\]that highlight image regions contributing to a prediction, and example\-based\[[42](https://arxiv.org/html/2608.28820#biba.bib42)\]and counterfactual\[[43](https://arxiv.org/html/2608.28820#biba.bib43)\]explanations that illustrate how small input changes would affect the prediction\. Post hoc methods can be applied to any trained model without altering its architecture, but the explanation is an output of the explanation method, not of the model, and may not faithfully reflect the model’s internal reasoning\.

### Type of XAI

XAI techniques are classified by their type based on their architectural dependency\. Model\-specific approaches are tailored to particular architectures and utilize internal model parameters to generate rationales, whereas model\-agnostic approaches are universally applicable across heterogeneous pipelines because they infer importance from perturbations rather than internal mechanics\.

#### Model\-specific

Model\-specific techniques are tailored to specific architectures and use internal components, such as gradients, activations, or attention weights, to generate explanations\. In CNNs, for example, saliency maps or class activation mappings highlight spatial regions that influenced predictions\[[33](https://arxiv.org/html/2608.28820#biba.bib33)\]\. In pathology, such methods have been used to highlight nuclear features, stromal patterns, or glandular structures associated with cancer grade\. Their advantage is that they leverage model weights and activations directly, but their limitation is narrow applicability; techniques designed for CNNs may not translate to transformers or MIL models\.

#### Model\-agnostic

Model\-agnostic approaches operate independently of the model’s architecture, relying only on inputs and outputs\. They can therefore be applied universally, making them attractive for heterogeneous pipelines\. Examples include SHAP\[[40](https://arxiv.org/html/2608.28820#biba.bib40)\], which provides mathematically grounded feature attributions, and perturbation\-based methods such as LIME\[[44](https://arxiv.org/html/2608.28820#biba.bib44)\]or occlusion sensitivity\[[45](https://arxiv.org/html/2608.28820#biba.bib45)\], which systematically alter input regions to infer importance\. In pathology, these methods have been applied to WSIs by perturbing patches or altering color channels \(e\.g\., simulating stain variability\)\. Their generality is valuable, but they can be computationally expensive and sensitive to noise\.

### Scope of XAI

The scope of explanation distinguishes whether an XAI method provides a local rationale for an individual case or a global overview of the model’s systemic behavior\. Local methods identify the specific features responsible for a single prediction, whereas global methods capture the model’s underlying logic across an entire dataset or patient population\.

#### Global

Global methods capture the model’s overall behavior across a dataset\. For example, feature importance rankings can reveal which histological features are consistently influential in tumor classification\[[46](https://arxiv.org/html/2608.28820#biba.bib46)\]\. Concept\-based methods such as Testing with Concept Activation Vectors \(TCAV\)\[[47](https://arxiv.org/html/2608.28820#biba.bib47)\]extend this idea, quantifying the influence of higher\-level concepts \(e\.g\., presence of necrosis, gland morphology\) on predictions\. Global explanations are particularly useful in pathology when the goal is to validate whether a model captures clinically relevant features across populations\.

#### Local

Local methods focus on individual predictions, providing case\-specific rationales\. These are particularly relevant for clinical settings where pathologists must trust or challenge individual model outputs\. Techniques such as SHAP and LIME highlight the image regions or cellular features most responsible for a specific prediction\. For example, a local explanation may show that a risk prediction for a single patient was driven by mitotic activity in a tumor hotspot\. Local explanations thus align closely with clinical workflows, where decisions are made case by case\.

Table 2:XAI methods in CompPath categorized by families and the Stage\-Type\-Scope taxonomy\.Legend:Type: Agnostic∘\\circ, Specific∙\\bullet; Scope: Local□\\square, Global■\\blacksquare; Stage: Post hoc△\\triangle, Intrinsic▲\\blacktriangle

## Per\-Method Descriptions of XAI Methods in CompPath

This section provides per\-method narrative descriptions for each XAI technique summarised in Table 2 of the main paper\. We conducted a narrative literature review of XAI methods in CompPath: studies were included if they applied, developed, or evaluated XAI methods for histopathology or whole\-slide image analysis, and excluded if they addressed general medical imaging without pathology\-specific data or did not provide an explanation\-related method or evaluation\. A formal systematic review under PRISMA\[[62](https://arxiv.org/html/2608.28820#biba.bib62)\]was considered but deemed inappropriate for the scope of this work for three reasons\. First, the field of XAI in CompPath is methodologically heterogeneous: studies vary widely in task \(classification, segmentation, prognosis\), tissue type, model architecture, and XAI evaluation criteria, which precludes the comparable outcome measures that systematic reviews and meta\-analyses require\. Second, our objective is conceptual synthesis, namely organizing methods into a unified taxonomy, terminology, and task\-driven framework, rather than estimating a pooled effect or comparing reported performance\. Third, the literature is evolving rapidly, with foundation models, vision\-language models, and concept\-bottleneck designs introduced after most systematic\-search windows would have closed\. We therefore adopted a narrative review approach following established guidance for non\-systematic literature reviews in fast\-moving fields\[[63](https://arxiv.org/html/2608.28820#biba.bib63),[64](https://arxiv.org/html/2608.28820#biba.bib64)\]\. The tables consequently present representative rather than exhaustive coverage, prioritizing methodological diversity, anatomical breadth, and clinical relevance over completeness\. Each description follows a uniform structure: operating principle, strengths, limitations, computational and practical considerations, current adoption in CompPath, and recommended validation practices\.

Table 3:Summary of backpropagation\-based XAI methods in CompPath: strengths, limitations, and recommended practices for validation and reporting\.MethodStrengthsLimitationsCautions & best practicesAttention
\(MIL / Transformer\)Highlights decision\-driving patches; interpretable at the instance level; integrates naturally with weakly supervised slide\-level learning\.Attention weights capture correlation, not causation; sensitive to site\-specific bias, stain, and scanner artefacts; diffuse across heads/layers in transformers\.Validate attention maps against expert ROIs and morphometric features; test stability across scanners, stains, and institutions\[[12](https://arxiv.org/html/2608.28820#biba.bib12)\]\.CAMSimple, fast, intuitive; identifies broad discriminative regions with minimal overhead\.Requires a GAP layer; coarse spatial resolution misses subtle nuclear or sub\-cellular features\.Use for rapid qualitative visualisation; confirm findings with higher\-resolution methods or expert annotations\.Grad\-CAMArchitecture\-agnostic; widely supported; produces visually intuitive, class\-specific localisations on standard CNNs\.Coarse heatmaps; poor at localising multiple instances of the same class; gradients can be attenuated or distorted under MIL pooling; no signed relevance\.Report the target convolutional layer; run sanity checks\[[18](https://arxiv.org/html/2608.28820#biba.bib18)\]; verify faithfulness via insertion/deletion\[[20](https://arxiv.org/html/2608.28820#biba.bib20)\]and cross\-site stability\[[12](https://arxiv.org/html/2608.28820#biba.bib12)\]\.Grad\-CAM\+\+Better multi\-instance localisation than Grad\-CAM through higher\-order gradient weighting; handles multiple objects in a single image\.Same correlational limitation as Grad\-CAM; sensitive to gradient noise in very deep networks\.Use when multiple instances of the target class are expected; benchmark against Grad\-CAM on the same model; verify cross\-site stability\[[12](https://arxiv.org/html/2608.28820#biba.bib12)\]\.Saliency mapsPixel\-level sensitivity; single\-pass computation per tile; useful as a baseline against which newer methods are evaluated\.Noisy and unstable; highly sensitive to preprocessing, stain normalisation, and scanner colour variation; often emphasises edges over morphology\.Smooth or combine with IG; assess stability under stain/rotation perturbations; confirm overlap with expert annotations\.SmoothGradReduces high\-frequency noise in saliency by averaging over noise\-perturbed inputs; can wrap around any gradient\-based method; produces spatially coherent maps\.NNextra forward and backward passes per attribution; sensitive to noise levelσ\\sigmaand sample countNN; can over\-smooth and lose fine morphological detail\.Reportσ\\sigmaandNN; benchmark against vanilla saliency; verify cross\-site stability\[[12](https://arxiv.org/html/2608.28820#biba.bib12)\]\.FullGradAggregates per\-layer gradient and bias contributions; satisfies a completeness axiom; higher spatial resolution than Grad\-CAM with class\-discriminative signal\.Higher compute and memory cost than Grad\-CAM; sensitive to post\-processing choices; requires architectures with explicit bias terms\.Report post\-processing operations; benchmark against Grad\-CAM and IG on the same model; assess cross\-site stability\[[12](https://arxiv.org/html/2608.28820#biba.bib12)\]\.LRPConserves relevance through layers; produces stable, fine\-grained attributions; adaptable to MIL and transformer architectures\.Computationally heavy; results depend on the chosen propagation rule and architecture\-specific implementation\.Report propagation rules and parameter choices; benchmark against Grad\-CAM and perturbation\-based faithfulness metrics; evaluate consistency across cohorts and scanners\[[12](https://arxiv.org/html/2608.28820#biba.bib12)\]\.DeepLIFTCaptures signed contributions; handles feature dependencies often missed by simple gradients\.Results vary substantially with the choice of baseline or reference input; implementation details affect outputs\.Test multiple baseline choices; verify the sum\-to\-delta property; confirm consistency with IG and occlusion analyses\.DeconvolutionReveals what individual filters and layers have learned; useful for debugging and understanding learned representations\.Produces generic, prototypical textures rather than case\-specific attribution; layer\-dependent reconstructions\.Use for representation analysis and model debugging; do not present as the sole basis for case\-level clinical justification\.Integrated GradientsAxiomatic guarantees; reduces gradient noise; widely used as a faithfulness reference\.Sensitive to baseline definition and number of integration steps; computationally heavy when applied tile\-by\-tile across WSIs\.Report baseline and integration step count; evaluate faithfulness via insertion/deletion and region perturbation; test robustness across staining and scanner conditions\[[12](https://arxiv.org/html/2608.28820#biba.bib12)\]\.\-

### Backpropagation\-based methods

Backpropagation\-based methods are typically of post hoc stage, model\-specific type, and primarily local scope\. They compute gradients of model outputs with respect to input features, identifying pixels or regions guiding predictions\. In CompPath, backpropagation\-based approaches that compute gradients are often used to identify histological “hotspots”, which can be linked to tissue morphology\.

#### Trainable attention\[[48](https://arxiv.org/html/2608.28820#biba.bib48)\]

has been a widely applied method in CompPath, particularly with weakly supervised MIL models\[[65](https://arxiv.org/html/2608.28820#biba.bib65),[66](https://arxiv.org/html/2608.28820#biba.bib66)\]\. In such models, WSIs are partitioned in a set of image patches \(instances\), and attention weights are learned to indicate the relative contribution of each patch to slide\-level prediction\. The attention mechanism thus enables both classification and localization of patches that contributed most in the model’s decision\. A widely used example of such models is ABMIL\[[39](https://arxiv.org/html/2608.28820#biba.bib39)\], which generates heatmaps based on attention pooling to highlight morphological subregions that drive model decisions without the need for pixel\-level supervision\.

Strengths of the ‘trainable attention’ include the i\) ability to identify influential patches using only slide\-level labels, ii\) suitability for MIL settings, and iii\) generally greater stability and data efficiency compared to gradient\-based explanation methods\. However, several limitations and pitfalls remain, as i\) their high attention weights do not necessarily imply causal importance, ii\) the learned weights can be influenced by dataset specific biases \(e\.g\., stain or scanner variations\), and iii\) attention maps often lack cellular\-level precision where high attention scores in positive prediction are more interpretable rather than high attention in negative cases\. Furthermore, prior studies have shown that attention visualizations may only be correlations rather than the true reasoning process of the model\[[8](https://arxiv.org/html/2608.28820#biba.bib8),[9](https://arxiv.org/html/2608.28820#biba.bib9)\]\. From a computational and practical standpoint, trainable attention is efficient at inference, as attention weights are obtained during the forward pass and can be exported for visualization with minimal additional computational cost\.

In terms of adoption and readiness, trainable attention mechanisms are widely used in research\-oriented MIL\-based pathology models\[[65](https://arxiv.org/html/2608.28820#biba.bib65),[66](https://arxiv.org/html/2608.28820#biba.bib66)\]\. Demonstrated applications include predicting clinically relevant outcomes, such as the identification of conclusive diagnoses in glioma\[[67](https://arxiv.org/html/2608.28820#biba.bib67)\]\. Despite this growing use, standardized evaluation protocols and regulatory\-grade validation remain limited, hindering direct clinical deployment\. Recommended validation practices include assessing overlap between attention\-highlighted regions and expert\-annotated areas or quantitative morphometric features \(e\.g\., nuclear density or glandular organization\)\. Robustness should be evaluated across cohorts, scanners, and staining variations, and quantitative sanity checks, such as attention perturbation or ablation studies, are encouraged to assess the stability, robustness, and reliability of the explanations\.

#### Transformer\-based Attention\[[49](https://arxiv.org/html/2608.28820#biba.bib49)\]

provides an intrinsic explanation through the self\-attention weight computed during the forward pass\. These weights encode contextual dependencies among image patches and captures relationships between spatially distinct regions using multi\-head self\-attention, enabling models to reason over local and global tissue context\.

Strengths of ‘transformer\-based attention’ include its inherent ability to i\) provide hierarchical explanations, ii\) model both cellular detail and higher\-order tissue organization, and iii\) naturally integrate contextual reasoning, e\.g\., tumor\-immune spatial patterns\. However, it is not without limitations and pitfalls as i\) attention weights indicate correlation rather than causation, ii\) multiple heads and layers produce complex diffuse attention maps that are difficult to interpret\. In addition, transformers incur high computational and practical costs, particularly during training, due to the large token counts involved in processing WSIs\. Hierarchical designs \(e\.g\., HIPT\[[68](https://arxiv.org/html/2608.28820#biba.bib68)\]\) mitigate cost by structuring aggregation across scales, though visualization still requires resource\-intensive layer aggregation\.

Regarding adoption and readiness, these mechanisms are increasingly used in pathology research \(e\.g\., HIPT\[[68](https://arxiv.org/html/2608.28820#biba.bib68)\]\)\. Despite promising performance, attention\-based visualization remains research\-focused and lacks standardized clinical validation\. Recommended validation strategies include i\) inspecting individual attention heads and layers independently before aggregation, ii\) correlating attention hotspots with expert annotations and quantitative features \(e\.g\., cell density, immune infiltration\), and iii\) using gradient\-based or prototype\-based cross\-validation \(e\.g\., Chefer explainability\[[49](https://arxiv.org/html/2608.28820#biba.bib49)\]\) to ensure faithfulness\.

#### Class Activation Mapping \(CAM\)\[[50](https://arxiv.org/html/2608.28820#biba.bib50)\]

is one of the earliest and widely used visualization techniques for explaining CNN predictions\. In CompPath, CAM identifies image regions that contribute to a model’s decision by linearly weighting feature maps from the final convolutional layer using class\-specific activation weights\. The method relies on Global Average Pooling \(GAP\) layer, which aggregates spatial information before classification\. The resulting weighted activation maps are projected back onto the input space to create coarse heatmaps that highlight discriminative tissue regions such as tumor epithelium or necrotic foci\.

Strengths of CAM include its conceptual simplicity, minimal architectural requirements, and the production of intuitive/easy\-to\-interpret heatmaps, making it useful for quick verification that the model’s focus aligns with pathological structures\. However, notable limitations/pitfalls include i\) the requirement for a GAP layer, ii\) low spatial resolution maps, which may miss small or subtle histological structures, and iii\) performance sensitivity to the design of the final convolutional layer and network depth\. From a computational and practical perspective, CAM i\) incurs very low cost, ii\) does not require any architectural modifications, beyond GAP, and iii\) can be applied directly from the trained model\.

In terms of adoption and readiness, CAM has been widely used in research \(including for visualizing molecular subtype localization in bladder cancer\[[69](https://arxiv.org/html/2608.28820#biba.bib69)\]\), primarily as a qualitative interpretability tool rather than a quantitative or regulatory standard\. Recommended validation practices include employing CAM for rapid, high\-level localization in GAP\-based CNNs and confirming highlighted areas with higher\-resolution methods \(e\.g\., Grad\-CAM or LRP\) or pathologist annotations, when available\.

#### Gradient\-weights Class Activation Mapping \(Grad\-CAM\)\[[41](https://arxiv.org/html/2608.28820#biba.bib41)\]

is a widely used post hoc method that generates class\-specific heatmaps by leveraging the gradients of a target output \(e\.g\., class score\) with respect to the feature map in CNNs\.

In terms of strengths, compared with CAM, Grad\-CAM removes the dependence on GAP layers \(i\.e\., is architecture\-agnostic\)\. By weighting feature maps according to the gradient intensity associated with a target class, Grad\-CAM highlights regions of an image that most strongly influence a model’s decision, offering a coarse but visually intuitive spatial explanation\. Grad\-CAM’s primary advantage in the pathology workflow is practical, as it i\) does not require any architectural modifications and is compatible with existing CNN\-based pipelines, ii\) adds low computational overhead, and iii\) produces heatmaps that pathologists can inspect visually\. Limitations and pitfalls are particularly relevant in CompPath and MIL settings, as gradient signals can be attenuated or distorted by instance\-level pooling, resulting in misleading explanations\. Grad\-CAM may also highlight background regions or imaging artifacts, and its faithfulness can vary across datasets and tasks\. From a computational and practical standpoint, Grad\-CAM i\) incurs moderate inference cost, ii\) requires gradient computation and spatial upsampling, and iii\) needs careful selection of the target convolutional layer to obtain localization of clinically relevant regions\.

Regarding adoption and readiness, Grad\-CAM is commonly used in research, e\.g\., metastasis localization\[[70](https://arxiv.org/html/2608.28820#biba.bib70)\]\. Adaptations \(e\.g\., MILPLE\[[71](https://arxiv.org/html/2608.28820#biba.bib71)\]\) extend it to instance\-level MIL, despite reported inconsistencies in spatial visual explanations due to disrupted gradient flow\. Recommended validation includes i\) explicitly reporting the chosen target layer, ii\) performing sanity checks to verify dependence on trained model parameters, by randomizing weights/labels and ensuring the maps change accordingly\[[18](https://arxiv.org/html/2608.28820#biba.bib18)\], iii\) evaluating pixel\-flipping tests and insertion/deletion metrics, to assess whether removing or adding Grad\-CAM\-highlighted regions meaningfully affects model confidence\[[20](https://arxiv.org/html/2608.28820#biba.bib20)\], and iv\) cross\-validate against expert annotations\.

#### Grad\-CAM\+\+\[[72](https://arxiv.org/html/2608.28820#biba.bib72)\]

Grad\-CAM\+\+ extends Grad\-CAM to better localise multiple instances of the same object class in an image, which is a known weakness of the original Grad\-CAM in CompPath applications where a WSI can contain many tumour foci, mitotic figures, or immune infiltrates simultaneously\. Grad\-CAM\+\+ uses higher\-order gradient information \(weighted combinations of positive partial derivatives\) to compute pixel\-wise weights, producing sharper localisations and better multi\-instance coverage than Grad\-CAM\. It has been applied in CompPath to lung histology classification\[[73](https://arxiv.org/html/2608.28820#biba.bib73)\]\. Limitations follow Grad\-CAM’s \(coarse resolution, correlation not causation\), with additional sensitivity to gradient noise in deep networks\.

#### Saliency Maps\[[74](https://arxiv.org/html/2608.28820#biba.bib74)\]

are among the earliest gradient\-based visualization methods for model explanation\. This method estimates how sensitive a model’s output is to each input pixel by computing the gradient of the prediction score with respect to the input image\. The resulting heatmap highlights pixels where small perturbations would induce the largest change in the output, thereby indicating regions most influential to the model’s decision\.

Strengths of saliency maps include their extremely fast computation, while able to produce fine\-grained, high\-resolution maps, and their use as a reference baseline for evaluating newer XAI methods\. However, significant limitations and pitfalls limit their utility in CompPath, as they are very sensitive to small image variations \(e\.g\., such as slight color changes, scanner noise\) and preprocessing steps \(e\.g\., normalization or stain variation\), which often result in noisy visualizations emphasizing edges or imaging artifacts rather than meaningful histological features\. From a computational and practical perspective, saliency maps are fast to generate, and their results are often post\-processed using smoothing or filtering to reduce visual noise to improve visualization\.

With respect to adoption and readiness, saliency maps are commonly used in research for patch\-level interpretability in pathology \(e\.g\., highlighting nuclei or tumor regions\[[75](https://arxiv.org/html/2608.28820#biba.bib75)\]\), but are generally insufficient on their own for quality assurance or clinical reporting due to instability and a lack of quantitative validation\. Recommended validation practices include i\) assessing stability under small perturbations of the same patch \(e\.g\., stain variation, rotation\), ii\) comparing results with more robust attribution methods, such as Integrated Gradients or occlusion\-based methods, and iii\) visually confirming overlap with expert annotations\.

Two extensions of vanilla saliency address its main weaknesses and are widely used in CompPath\. SmoothGrad\[[76](https://arxiv.org/html/2608.28820#biba.bib76)\]mitigates the high\-frequency noise in input gradients by averaging saliency maps over multiple noise\-perturbed copies of the input \(x\+𝒩⁡\(0,σ2\)x\+\\mathcal\{N\}\(0,\\sigma^\{2\}\)\), producing visually cleaner and more spatially coherent attributions at the cost of a forward and backward pass per noise sample\. It has been used in CompPath to generate denoised saliency maps for prostate cancer Gleason\-grading\[[77](https://arxiv.org/html/2608.28820#biba.bib77)\]and is also commonly applied as a wrapper around Grad\-CAM and Integrated Gradients rather than vanilla saliency\. Full\-Gradient representation \(FullGrad\)\[[78](https://arxiv.org/html/2608.28820#biba.bib78)\]addresses the complementary problem of spatial coarseness by decomposing the model’s output into per\-layer contributions from the input gradient and the gradients of all intermediate bias terms; the resulting attribution map satisfies a completeness property \(the sum of per\-pixel attributions equals the model’s scalar output\) and produces visually sharper, more class\-discriminative maps than vanilla saliency at higher compute cost than Grad\-CAM but lower than Integrated Gradients\. FullGrad has been applied in CompPath to breast cancer histology classification\[[79](https://arxiv.org/html/2608.28820#biba.bib79)\]and to interpretable prediction of tumor microenvironment phenotypes from H&E whole\-slide images\[[80](https://arxiv.org/html/2608.28820#biba.bib80)\]\. The cautions that apply to vanilla saliency apply to both extensions: report post\-processing operations \(for SmoothGrad: noise levelσ\\sigmaand number of samplesNN; for FullGrad: absolute value, channel aggregation, and normalization\), benchmark against Grad\-CAM and Integrated Gradients on the same model, and verify cross\-site stability of the resulting maps, since fine spatial detail can otherwise reflect scanner\- or stain\-specific texture rather than tissue biology\[[12](https://arxiv.org/html/2608.28820#biba.bib12)\]\.

#### Layer\-wise Relevance Propagation \(LRP\)\[[81](https://arxiv.org/html/2608.28820#biba.bib81)\]

is a backpropagation\-based explainability method that redistributes a model’s output prediction score backward through the network to assign relevance values to individual input features\. Relevance is propagated proportionally from each neuron to its inputs, based on predefined conservation rules, ensuring that total relevance is preserved across layers\. This principle of “relevance conservation” distinguishes LRP from standard gradient\-based methods and enables more stable attribution maps that explain why a model made a particular decision, rather than merely indicating regions with large gradients\.

Strengths of LRP include its explicit conservation property, which yields stable and fine\-grained explanations, and its ability to attribute and propagate relevance across layers, even in complex MIL or transformer architectures\. LRP has also been shown to provide faithful and visually interpretable explanations, and in some settings to even outperform Grad\-CAM for MIL\-based interpretability\[[82](https://arxiv.org/html/2608.28820#biba.bib82)\]\. Nevertheless, LRP is sensitive to the choice of propagation rule \(e\.g\.,ϵ\\epsilon\-rule,α​β\\alpha\\beta\-rule\) and implementation requires network\-specific adaptation, especially for MIL or transformer blocks\. From a computational and practical perspective, LRP is computationally more demanding than simple gradient\-based methods mainly due to relevance propagation through all layers, and requires comprehensive documentation of parameter settings \(e\.g\., rule choice, normalization\) towards facilitating reproducibility\.

In terms of adoption and readiness, LRP has successfully demonstrated slide\-level interpretability in recent work, such as xMIL\-LRP\[[82](https://arxiv.org/html/2608.28820#biba.bib82)\]for breast and lung cancer WSIs, but remains limited to research use\. Recommended validation practices include i\) explicitly reporting the propagation rules and parameter choices, ii\) comparing relevance maps with complementary methods, such as Grad\-CAM, Integrated Gradients, and perturbation\-based faithfulness metrics \(insertion/deletion, ROAR\), and iii\) evaluating overlap with expert ROIs and consistency across architectures and datasets\.

#### Deep Learning Important FeaTures \(DeepLIFT\)\[[53](https://arxiv.org/html/2608.28820#biba.bib53)\]

explain model predictions by decomposing output differences relative to a predefined reference \(baseline\) input\. Instead of relying on local gradients, DeepLIFT attributes importance by comparing each neuron’s activation to its reference activation and propagating contribution scores backward through the network\. This approach enables signed attributions that indicate which input features increase or decrease the model’s confidence and supports both pixel\-level and patch\-level visualizations\.

Strengths of DeepLIFT include its i\) ability to distinguish positive and negative contributions to a prediction, ii\) capacity to capture feature dependencies that are often missed by simple gradient methods, and iii\) improved stability compared with plain saliency maps in some CNN architectures\. However, notable limitations and pitfalls arise from its strong dependence on the choice of baseline or reference image \(for example, a blank patch\), as well as on implementation details such as scaling and normalization, which affect the results\. From a computational and practical perspective, DeepLIFT incurs a moderate computational cost, comparable to a single backward pass per patch once implemented, but careful verification and reporting of the selected baseline and the “sum\-to\-delta” property \(ensuring all contributions add up to the prediction difference\) are essential for reproducibility\.

Regarding adoption and readiness, DeepLIFT has been applied in research settings for patch\-level analysis of colon cancer WSIs\[[83](https://arxiv.org/html/2608.28820#biba.bib83)\], but it has not yet been standardized for WSI or clinical workflows\. Recommended validation practices include i\) evaluating sensitivity across multiple baseline choices, ii\) confirming consistency with related attribution methods such as integrated gradient or occlusion analyses, and iii\) visually assessing whether highlighted structures correspond to known histologic features\.

#### Deconvolution\[[54](https://arxiv.org/html/2608.28820#biba.bib54)\]

visualizes internal representations of CNN by approximately reversing the sequence of convolution, pooling, and activation operations to reconstruct the input pattern that strongly activates specific filters or neurons\. This approach produces patch\-level visualizations of intermediate feature activations, highlighting textures, edges, or structural patterns learned by the network rather than directly explaining individual predictions\.

Strengths of deconvolution include its ability to provide insight into what different filters or layers of a CNN have learned, making it useful for debugging network behavior and understanding how histological features, such as nuclei or tissue textures, are internally represented\. However, limitations and pitfalls include limited interpretability in decision\-level settings\. More fundamentally, deconvolution addresses a different question than attribution\-based methods: it reveals what filter has learned to detect, not why a specific prediction was made, i\.e\., they often show generic or “prototypical” textures learned by the model rather than evidence specific to a particular case\. Moreover, visualizations may vary depending on the chosen layer or reconstruction method\. From a computational and practical perspective, deconvolution incurs moderate computational cost, requires access to intermediate activations, and can generate large visualization outputs for deep architectures\.

In terms of adoption and readiness, deconvolution has been used in research for qualitative inspection in lung cancer\[[84](https://arxiv.org/html/2608.28820#biba.bib84)\]\. However, it is best suited for exploratory analysis and model debugging rather than clinical justification\. Recommended validation involves using deconvolution to interpret learned features and complementing it with attribution\-based or perturbation\-based methods, when explaining individual predictions or supporting clinical conclusions\.

#### Integrated Gradient\(IG\)\[[55](https://arxiv.org/html/2608.28820#biba.bib55)\]

attributes model predictions by integrating the gradients of the output with respect to the model input along with a continuous path from a predefined baseline to the actual input\. This path\-based integration accumulates attribution scores across interpolation steps, yielding smoother and more stable explanations than single\-step gradient methods\.

Strengths of IG include its i\) strong theoretical foundation, through the axioms of sensitivity and implementation invariance, and ii\) ability to produce quantitatively interpretable attributions that reduce gradient noise\. As a result, IG is widely recognized as a benchmark for evaluating faithfulness in neural explanations\. Nevertheless, IG has several limitations and pitfalls that require careful selection of both the baseline and the number of integration steps: an inappropriate baseline choice can distort attributions, and too few steps can yield incomplete integration and leave results sensitive to staining variability and scanner differences\. From a computational and practical standpoint, IG incurs substantially higher computational cost than single\-step gradient approaches \(e\.g\., saliency maps, Grad\-CAM\), as it is typically applied per tile or patch and then aggregated into a slide\-level map\. Moreover, its reproducibility depends on consistent baseline selection and integration step settings\.

In terms of adoption and readiness, IG has been used in research to highlight metastasis\-relevant areas in lymph node WSIs\[[85](https://arxiv.org/html/2608.28820#biba.bib85)\]\. It is often treated as a reliable reference for benchmarking explainability methods, but it is not yet standardized for prospective or regulatory workflows due to its scalability and interpretive complexity\. Recommended validation practices include i\) explicit reporting of baseline choices \(e\.g\., integration steps and implementation details\), ii\) evaluating faithfulness using insertion/deletion and region perturbation metrics, and iii\) comparing attributions against expert\-defined regions of interest across scanners and staining conditions\.

### Perturbation\-based methods

Table 4:Summary of perturbation\-based XAI methods in CompPath: strengths, limitations, and recommended practices for validation and reporting\.Perturbation\-based methods are typically post hoc and model\-agnostic\. They usually provide local explanations, but their results can be aggregated across patches, slides, or cohorts to support global analysis\. It explains model behavior by systematically modifying input data and observing the impact on predictions\. These approaches do not rely on gradients, making them model\-agnostic\. By perturbing input data, these methods help identify which features or components of the input most influence the model’s predictions\. Perturbation methods can also help uncover biases in the model by revealing how specific alterations to the input data affect the output\. Below, we describe various perturbation\-based techniques\.

#### Local Interpretable Model\-agnostic Explanations \(LIME\)\[[44](https://arxiv.org/html/2608.28820#biba.bib44)\]

explains individual predictions by locally approximating the decision surface of a black\-box model with an interpretable surrogate, most commonly a sparse linear regression\. For a given instance, LIME generates perturbed samples by modifying subsets of input features, queries the original model for corresponding predictions, and fits the surrogate model using weights based on similarity to the original instance\. The resulting surrogate coefficients quantify the local contribution to the prediction\. LIME is typically applied at the patch or region level, and explanations from multiple tiles can be aggregated to derive slide\-level insights\.

Strengths of LIME include its model\-agnostic nature and its ability to provide intuitive interpretability via simple local surrogate models without requiring access to gradients\. However, its use in CompPath is limited because it is highly sensitive to segmentation choices, kernel width, and perturbation design, and its explanation fidelity strongly depends on the sampling strategy and surrogate regularization\. Furthermore, standard implementations often produce unstable explanations across runs and non\-realistic tissue perturbations that limit biological interpretability\[[36](https://arxiv.org/html/2608.28820#biba.bib36)\]\. From a computational and practical perspective, LIME incurs moderate computational cost and requires hundreds of forward passes per instance, which limits scalability to gigapixel WSIs and necessitates careful tiling strategies\.

With respect to adoption and readiness, LIME has been used in research for patch\-level interpretability in tasks, such as breast cancer classification and lymph node metastasis detection\[[86](https://arxiv.org/html/2608.28820#biba.bib86),[87](https://arxiv.org/html/2608.28820#biba.bib87)\]\. Although stabilized variants such as SLICE\[[88](https://arxiv.org/html/2608.28820#biba.bib88)\]and BayLIME\[[89](https://arxiv.org/html/2608.28820#biba.bib89)\]have demonstrated improved reproducibility and faithfulness at the expense of computational overhead, they are currently limited to experimental and research applications\. Recommended validation practices include explicitly reporting kernel width, segmentation strategy, and sampling parameters\. Other recommendations comprise i\) assessing stability across repeated runs using rank correlation, ii\) evaluating explanation fidelity with insertion/deletion or Area Over the Perturbation Curve \(AOPC\) curves, and iii\) verifying overlap with expert\-defined ROIs while ensuring that perturbations remain histologically plausible\.

#### Occlusion Sensitivity \(OS\)\[[45](https://arxiv.org/html/2608.28820#biba.bib45)\]

evaluates the importance of image regions by systematically occluding different parts of the input and measuring the resulting change in model predictions\. By directly observing how model confidence varies when specific regions are hidden, OS identifies areas of an image that most strongly influence the decision, offering an intuitive and interventional form of model\-level sensitivity analysis\.

Strengths of OS include its direct assessment of feature importance through controlled perturbations and its independence from model architectures or gradient information\. However, OS is sensitive to the design and size of occlusion masks, since larger masks may remove multiple features, whereas smaller ones can amplify noise and produce unstable estimates\. In addition, the use of non\-biological occlusions \(e\.g\., zeroing, solid colors\) produces unrealistic tissue textures, leading to misleading results\. From a computational and practical perspective, OS incurs a high computational cost that scales with the number and granularity of occluded regions, making it feasible only for patch subsets or representative tiles\. Realistic perturbations \(e\.g\., blur, inpainting\) and efficient patch batching can improve OS’s scalability\.

Regarding adoption and readiness, OS is commonly used in research to assess region importance in prostate cancer and other histopathology tasks\[[90](https://arxiv.org/html/2608.28820#biba.bib90)\]\. It is currently a validation tool rather than a clinically validated method, although extended frameworks like HIPPO\[[91](https://arxiv.org/html/2608.28820#biba.bib91)\]aim to provide clinically meaningful causal insights\. Recommended validation practices include i\) employing biologically plausible perturbations such as blurring or inpainting instead of zero masking, ii\) testing multiple mask sizes to assess stability, iii\) quantifying faithfulness using insertion/deletion metrics, and iv\) confirming consistency of highlighted regions across institutions, scanners, and staining protocols\.

#### Randomized Input Sampling for Explanation \(RISE\)\[[20](https://arxiv.org/html/2608.28820#biba.bib20)\]

estimates pixel\-level importance by generating random binary masks that stochastically occlude a subset of the input image regions and observing resulting changes in the model’s predictions\. For a given image, RISE creates thousands of random masks, each hiding a different random subset of pixels\. The model is evaluated on each masked image, and the final importance map is computed as a weighted sum of the masks, with each mask’s contribution proportional to the model’s predicted confidence for that masked input\. Extensions such as Transformer Input Sampling\[[92](https://arxiv.org/html/2608.28820#biba.bib92)\]adapt RISE to vision transformers \(ViTs\) by sampling a subset of attention tokens rather than pixels, thereby improving patch\-level localization\.

Strengths of RISE include its model\-agnostic formulation, which does not rely on gradients or model internals; its unbiased Monte Carlo\-based estimation of feature importance; and its inherent noise reduction through aggregation over randomly perturbed masks\. However, RISE typically requires thousands of random masks to achieve convergence, resulting in substantial computational overhead\. Moreover, random masking patterns may not correspond to realistic tissue regions, and the effective spatial resolution is limited by the mask size and stride\. From a computational and practical standpoint, RISE is computationally intensive, as each masked input requires a forward pass\. Efficiency can be improved through batching, GPU parallelization, or stratified sampling\. Coarse\-grained RISE variants can balance fidelity and runtime for WSI\-scale models\.

In terms of adoption and readiness, RISE has been used primarily to compare explainability methods, including for lymph node metastasis classification\[[85](https://arxiv.org/html/2608.28820#biba.bib85)\], and remains a research\-level approach due to its high computational demand\. Transformer variants \(e\.g\., ViT\-RISE\) show promise for scalability in multi\-head attention networks but still require systematic validation\. Recommended validation practices include i\) clearly reporting the number, size, and stride of masks, ii\) comparing explanations with complementary occlusion\-based and gradient\-based methods \(e\.g\., OS and IG\), iii\) evaluating faithfulness using insertion/deletion or ROAR metrics, and iv\) verifying overlap between generated attribution maps and expert\-defined regions\.

### Feature\-based methods

Table 5:Summary of feature\-based XAI methods in CompPath: strengths, limitations, and recommended practices for validation and reporting\.Feature attribution and representation analysis methods are typically at a post hoc stage, can be model\-specific or model\-agnostic, and can provide a local\-to\-global scope\. Feature attribution and representation analysis explain model predictions by analyzing the contribution or importance of individual features in the input data\. These methods quantify the influence of a particular feature on a model’s output, providing insights into which input elements drive specific predictions\. Unlike perturbation or backpropagation\-based techniques, feature\-based methods often rely on statistical analyses, feature attribution scores, or model\-inherent properties to determine feature relevance\.

#### Activation Layer Visualization \(ALV\)\[[56](https://arxiv.org/html/2608.28820#biba.bib56)\]

is one of the earliest model\-specific explainability methods designed to examine internal feature representations\. ALV inspects the activation \(feature maps\) produced at different layers of the convolutional neural network \(CNN\) when an input image is propagated forward, revealing how information is progressively transformed across the network\.

Strengths of ALV include its ability to expose the hierarchical construction of representations, from simple textures and edges in early layers to more complex morphological patterns in deeper layers, making it useful for understanding what types of features a model has learned\. However, ALV provides qualitative rather than quantitative attributions, and interpretation can be subjective and dependent on the researcher’s domain knowledge\. Furthermore, individual feature maps can be selectively interpreted, potentially biasing conclusions\. From a computational and practical standpoint, ALV is lightweight, requiring only a forward pass to extract activations, but storing large numbers of feature maps from WSI\-scale data can be memory\-intensive\.

In terms of adoption and readiness, ALV has primarily been used in research settings as an audit tool rather than as a clinical interpretability method\. Example studies include visualizing learned feature hierarchies in breast cancer histology\[[93](https://arxiv.org/html/2608.28820#biba.bib93)\]\. Recommended validation practices include correlating activated channels with morphometric features \(nuclear density, gland\) and testing consistency across staining protocols and scanner domains to assess robustness\.

#### SHapley Additive exPlanations \(SHAP\)

\[[40](https://arxiv.org/html/2608.28820#biba.bib40)\]is a unified method for attributing the contribution of individual features to a model’s prediction using Shapley values from cooperative game theory\. It quantifies how much each “feature player” contributes to shifting the model’s output away from a baseline expectation by averaging its marginal contribution across all possible feature combinations\. In practice, these values are approximated through either model\-agnostic \(Kernel SHAP\) or model\-specific implementations such as Tree SHAP and Deep SHAP\.

Strengths of SHAP include its strong theoretical guarantees, its ability to produce additive attributions that satisfy local accuracy and missingness axioms, and its ability to bridge local \(instance\-level\) and global \(cohort\-level\) explanations\. On the other hand, kernel SHAP is computationally expensive, and accuracy depends on meaningful feature definitions and background datasets\. When applied directly to raw pixels, SHAP attributions can appear noisy or biologically implausible, and naive aggregation of feature attributions without proper normalization may distort clinical interpretation\. From a computational and practical perspective, SHAP incurs moderate\-to\-high computational cost\. It is tractable for tabular or region\-based pipelines, but becomes prohibitively expensive for pixel\-level image explanations\. Consistent baseline data selection and feature scaling are therefore essential for reproducibility\.

In terms of adoption and readiness, SHAP has been increasingly used in CompPath to assess feature importance in breast cancer and related pipelines\[[94](https://arxiv.org/html/2608.28820#biba.bib94),[86](https://arxiv.org/html/2608.28820#biba.bib86),[95](https://arxiv.org/html/2608.28820#biba.bib95)\]\. More recently, PATH\-X framework\[[96](https://arxiv.org/html/2608.28820#biba.bib96)\]integrated SHAP with ViT and autoencoders to explain patient\-level risk stratification\. While SHAP is relatively mature for feature\- and region\-based workflows, its application to end\-to\-end image attribution remains experimental\. Recommended validation practices include i\) defining features at biologically interpretable scales \(cells, glands, tissue regions\), ii\) reporting background dataset and sampling strategy, iii\) comparing global SHAP rankings with known pathology markers and ablation or occlusion analyses, and iv\) visually inspecting local SHAP overlays for spatial and biological plausibility\.

### Concept\-based methods

Table 6:Summary of concept\-based XAI methods in CompPath: strengths, limitations, and recommended practices for validation and reporting\.Concept\-based explainability methods are typically post hoc, model\-agnostic, and primarily local in scope\. It focuses on identifying meaningful, human\-interpretable representations within deep learning models\. Rather than analyzing individual features or perturbing inputs in backpropagation\-based or perturbation\-based methods, these concept\-based methods aim to capture high\-level, abstract concepts that align with human understanding\. This makes them particularly valuable in CompPath, where clinically relevant patterns such as tumor morphology, tissue characteristics, or histological features can provide more intuitive insights into model decision\-making\.

#### Testing with Concept Activation Vectors \(TCAV\)\[[47](https://arxiv.org/html/2608.28820#biba.bib47)\]

is a concept\-based model\-access dependent method that quantifies how strongly human\-interpretable concepts influence model predictions\. Rather than attributing importance to individual pixels or low\-level features, TCAV evaluates the directional sensitivity of a model’s internal representations to predefined, semantically meaningful concepts\. These concepts are encoded as concept activation vectors \(CAVs\), linear directions in latent feature space that capture high\-level properties such as “necrosis” or “glandular pattern”\. For each concept, a CAV is learned by training a simple linear classifier \(e\.g\., SVM\) to separate activations of concept examples from randomly sampled counterexamples at a selected intermediate layer\. The directional derivative of the model output along the CAV measures how increasing the presence of the concept would affect the model’s confidence for a target class\. This formulation enables hypothesis\-driven interpretation, allowing targeted questions such as “Does the model rely on lymphocyte infiltration to predict tumor grade?”\.

Strengths of TCAV include its i\) explicit linkage between human\-understandable concepts and deep model representations, ii\) support for hypothesis\-driven model auditing and causal probing, and iii\) flexibility in incorporating both expert\-curated and automatically discovered concepts\. However, several limitations and pitfalls must be considered as TCAV assumes linear separability of concepts in latent space, and its reliability is highly sensitive to the quality, quantity, and diversity of concept exemplars\. Furthermore, the choice of network layer affects explainability, and spurious \(as well as overlapping\) concepts can distort attribution scores\. From a computational and practical perspective, TCAV incurs a moderate computational cost, requiring one linear \(SVM\) classifier per concept and layer\. The primary overhead lies in curating representative concept examples or in clustering patches and extracting corresponding activations\.

In terms of adoption and readiness, TCAV is emerging in CompPath research\[[97](https://arxiv.org/html/2608.28820#biba.bib97)\]\. Pathology\-specific adaptations have been proposed, such as the work by Zitvnansky\[[98](https://arxiv.org/html/2608.28820#biba.bib98)\], which combines automatic concept discovery with TCAV\-style to generate tissue\-level concepts in prostate biopsy images\[[98](https://arxiv.org/html/2608.28820#biba.bib98)\]\. While these studies demonstrate promise, TCAV remains largely research\-focused\. Recommended validation practices include evaluating multiple concept sets across different network layers to assess robustness, using held\-out concept examples from diverse sites or scanners, and incorporating negative\-control concepts to detect spurious correlations or overfitting\.

#### Concept\-Bottleneck Models \(CBMs\)

\[[37](https://arxiv.org/html/2608.28820#biba.bib37)\]provide intrinsic interpretability by embedding human\-defined or pathologist\-recognizable concepts directly into a model’s architecture as intermediate representations\. Rather than learning opaque latent features, CBMs constrain part of the network to predict interpretable concepts \(e\.g\., cribriform glands, fused glands, poorly formed glands\), which are subsequently combined to yield the final clinical prediction \(e\.g\., Gleason grade group\)\. This structure design closely mirrors expert diagnostic reasoning and enables direct inspection of how each histologic concept quantitatively contributes to the model’s decision\.

Strengths of CBMs include their i\) built\-in transparency through explicitly modeling pathologist\-defined tissue features, ii\) support for quantitative validation of concept\-level predictions, and iii\) enhanced auditability for regulatory and clinical review\. By exposing intermediate concept predictions, CBMs also facilitate uncertainty estimation when implemented with stochastic variants\. However, limitations and pitfalls remain\. CBMs require detailed, concept\-specific expert annotations, which limit scalability across large, multi\-institutional datasets\. Fixed concept definitions may fail to capture subtle or composite morphologies, and the imposed bottlenecks can constrain learning if diagnostically relevant but unannotated cues are excluded\. From a computational and practical standpoint, CBMs have computational requirements comparable to segmentation\-style models, with training costs that scale with concept dimensionality and annotation density\. Inference, however, is typically efficient, enabling practical integration into digital pathology pipelines\.

In terms of adoption and readiness, CBMs have been demonstrated in pathology\-specific applications, such as Gleason grading for prostate cancer\[[99](https://arxiv.org/html/2608.28820#biba.bib99)\], representing one of the first intrinsic XAI designs tailored to CompPath\. Their concept\-level accountability makes them promising, but they are still limited by challenges in annotation and generalization\. Recommended validation practices include i\) evaluating concept predictions against expert annotations and inter\-observer variability, ii\) assessing cross\-site and cross\-scanner consistency of learned concepts, iii\) monitoring potential concept leakage using ablation or counterfactual analyses, and iv\) reporting per\-concept uncertainty along with its impact on diagnostic downstream performance\.

#### Image Captioning \(IC\)\[[57](https://arxiv.org/html/2608.28820#biba.bib57)\]

is a concept\-based explainability method that links visual patterns in WSI to human\-readable textual descriptions, enabling AI models to communicate their reasoning in natural language\. In CompPath, IC models often implemented using vision\-language architectures are trained to generate descriptive text for image tiles or regions, which can then be aggregated into slide\-level summaries or structured, report\-like outputs\.

Strengths of image captioning include its ability to translate visual evidence into natural language, facilitate multi\-modal reasoning by aligning image features with medical terminology, and support automated generation of descriptive pathology reports\. In terms of limitations, IC requires large, well\-curated paired image\-text datasets, is prone to generating inaccurate or “hallucinated” descriptions, which may not be directly supported by the visual input, and hence its objective evaluation of factual accuracy remains challenging\. Output quality is also highly dependent on the underlying language model, training data distribution, and terminology alignment\. From a computational and practical perspective, IC models are computationally expensive to train and require substantial effort to curate pathology\-specific vocabularies, although inference costs are typically moderate once trained\.

In terms of adoption and readiness, IC remains at an early research stage in CompPath\[[7](https://arxiv.org/html/2608.28820#biba.bib7)\], but promising for generating interpretable clinical\-style outputs\. Recommended validation practices include i\) developing standardized terminology in collaboration with expert pathologists, ii\) verifying factual correctness of generated captions against ground\-truth reports, and iii\) testing model generalization across institutions and scanner variations\.

#### Vision\-Language Interpretability\[[100](https://arxiv.org/html/2608.28820#biba.bib100)\]

extends concept\-based methods beyond caption generation by aligning VLMs in a bidirectional and interactive manner\. Rather than solely producing text from images, VLMs such as CLIP, BioViL, and PLIP learn shared embedding spaces that align visual regions with pathology\-relevant language\. This alignment enables zero\-shot classification, matching images to textual diagnostic terms without explicit training labels, as well as semantic retrieval, such as identifying image regions corresponding to concepts like “necrosis”, tumor border”, or querying which textual concepts best describe a given region\.

Strengths of VLI include that it i\) establishes a direct link between visual and text representations, ii\) supports zero\-shot concept matching and semantic search without explicit annotations, iii\) allows clinically intuitive queries such as “what visual evidence supports this term?” or “which pathology term describes this region?”, and iv\) enhances transparency in multimodal computational workflows\. However, VLI requires large\-scale, high\-quality paired datasets of WSIs and text reports for robustness, and interpretability depends on the precision of terminology and the quality of cross\-modal alignment\. Models may suffer from semantic drift, in which visual features are weakly or incorrectly associated with textual terms, leading to semantically plausible but visually ungrounded outputs\. From a computational and practical standpoint, training VLMs is computationally intensive due to large paired datasets and high\-dimensional embeddings, although inference is efficient and scalable for retrieval and similarity\-based analyses\. Notably, rigorous evaluation of grounding often requires additional tools for visual\-text correspondence\.

Regarding adoption and readiness, VLI has seen early adoption in CompPath, with models such as CLIP, BioViL, and PLIP used to align pathology images with diagnostic reports and structured concepts\[[58](https://arxiv.org/html/2608.28820#biba.bib58),[101](https://arxiv.org/html/2608.28820#biba.bib101)\]\. Nevertheless, VLI is promising for multimodal learning and transparent clinical reporting, but remains in the early research phases\. Recommended validation practices include evaluating both visual grounding \(whether retrieved image regions truly match the described concept\) and semantic grounding \(whether generated or matched terms carry the correct pathological meaning\), conducting expert review of retrieved examples, testing across institutions and scanners, and comparing with oncology\-based term mappings to ensure factual and semantic consistency\.

### Example\-based methods

Table 7:Summary of example\-based XAI methods in CompPath: strengths, limitations, and recommended practices for validation and reporting\.Example\-based methods are typically post hoc in stage, agnostic in type, and primarily local in scope\. Example\-based methods in explainable AI aim to enhance model explainability by leveraging real data examples to support predictions\. These methods provide tangible references that help users understand the rationale behind a model’s decisions\. Evans et al\.\[[102](https://arxiv.org/html/2608.28820#biba.bib102)\]comprehensively reviewed these approaches in the context of CompPath\. Below, we describe example\-based techniques used in WSI analysis, highlighting their role in improving explainability and trust in deep learning models for clinical decision support\.

#### Prototypes\[[103](https://arxiv.org/html/2608.28820#biba.bib103)\]

are example\-based explainability methods that provide insights into model behavior by identifying and visualizing instances representative of a particular class, feature, or prediction outcome\. A prototype may be selected using clustering techniques \(e\.g\., k\-means\), where cluster centroids or nearest examples serve as representative cases, or learned directly during model training via a prototype\-based network that optimizes similarity between inputs and class\-specific prototypes\.

Strengths of prototype\-based explanations lie in their intuitive, case\-based reasoning, which closely aligns with routine clinical practice by presenting concrete visual examples\. Prototypes help identify class boundaries and detect out\-of\-distribution or anomalous patches, enhancing transparency in audit studies\. However, several limitations and pitfalls should be noted, as the representativeness of prototypes depends strongly on the diversity and quality of the training data, as well as on the clustering choices and the embedding space\. Poor embedding or a biased dataset may result in prototypes that over\-represent common artifacts rather than biologically meaningful features\. From a computational and practical standpoint, prototype extraction incurs a moderate cost, typically involving clustering or nearest\-neighbor searches in the embedding space\.

In terms of adoption and readiness, prototype\-based explanations have been reviewed and demonstrated in CompPath\[[102](https://arxiv.org/html/2608.28820#biba.bib102)\], including applications that identify representative Ki\-67\-positive and Ki\-67\-negative annotations within a region\. Recommended validation practices include i\) curating prototype banks across institutions, ii\) assessing pathologist agreement on prototype relevance, and iii\) analyzing class boundaries via counter\-prototypes that illustrate contrasting cases\.

#### Counterfactuals\[[60](https://arxiv.org/html/2608.28820#biba.bib60)\]

provide hypothetical scenarios illustrating how a slight, targeted change to an input could lead to different model predictions\.

Strengths of counterfactual explanations include their ability to support causal reasoning by testing how specific, interpretable changes affect predictions, making them useful for auditing model behavior and identifying key diagnostic cues\. However, counterfactuals also come with their limitations as i\) generating realistic counterfactual tissue images is challenging, ii\) synthetic counterfactuals may introduce artifacts or unrealistic textures, iii\) edits can violate biological constraints \(such as nuclear shape or tissue organization\) if not properly constrained, and iv\) interpretation requires expert review\. From a practical perspective, counterfactual generation can be computationally intensive when implemented using generative models \(such as diffusion models or GANs\), although latent space counterfactuals can be comparatively lightweight if suitable representations are available\. In addition, counterfactuals require well\-defined priors and morphological constraints to ensure biological realism\.

Regarding adoption and readiness, counterfactual explanations remain an emerging research area in CompPath, with recent work exploring the generation of plausible counterfactual patches for WSI classification tasks\[[104](https://arxiv.org/html/2608.28820#biba.bib104)\]\. Their use is currently research\-focused and not yet suitable for clinical deployment\. Recommended validation strategies include i\) conducting expert “Turing tests” to assess visual plausibility, ii\) quantitatively verifying faithfulness by confirming that small, realistic edits produce consistent induced class changes, and iii\) enforcing morphological and staining constraints to maintain histologic credibility\.

#### Trust Scores\[[105](https://arxiv.org/html/2608.28820#biba.bib105)\]

provide a quantitative measure of how reliable a model’s prediction is by comparing it to neighboring samples in the learned feature space\. Trust scores are not explanations in the strict sense\. They provide reliable evidence by estimating whether a test case lies near training examples with consistent labels\. In XAI workflows, they are best used as uncertainty or triage signals alongside attribution, concept, or prototype\-based explanations\.

Strengths of trust scores include their i\) intuitive, data\-driven proxy for prediction confidence by comparing each prediction to its nearest neighbors in feature space, ii\) ability to identify unreliable predictions, outliers, or out\-of\-distribution predictions, and to complement visual attribution methods by quantifying uncertainty\. However, trust scores depend heavily on the quality and calibration of the embedding space, domain shifts, scanner variability, or poorly learned representations can produce misleading confidence estimates\. Moreover, trust scores do not explain underlying reasoning and may still assign high confidence to predictions driven/dominated by artifacts\. From a computational and practical standpoint, trust scores are relatively lightweight, relying on k\-nearest\-neighbor searches in the embedding space that can be efficiently scaled using approximate nearest neighbor indexing for large datasets\.

In terms of adoption and readiness, trust scores have been applied in CompPath research as a quality assurance and triage tool for reviewing model outputs\[[102](https://arxiv.org/html/2608.28820#biba.bib102)\]\. Recommended validation practices include i\) calibrating trust thresholds separately for each site or scanner domain, ii\) correlating trust scores with pathologist disagreement rates and known error cases, iii\) jointly inspecting low\-trust predictions alongside attribution maps \(e\.g\., Grad\-CAM or LRP\) to identify and investigate potential model failures\.

## Extended Deployment Guidance

### Other Considerations

#### Practical and Regulatory Readiness

XAI methods vary widely in computational cost, making deployment context essential to method selection\. For computationally constrained environments \(e\.g\., limited GPU memory, real\-time inference requirements, or edge devices\), lightweight visualization methods such as CAM, Grad\-CAM, or attention export provide quick qualitative verification with minimal computational overhead \(typically less than 10% increase in inference time\)\. These methods are suitable for exploratory analysis and preliminary model validation but may lack the quantitative rigor required for high\-stakes applications\.

For prospective validation studies or regulatory submissions, a multi\-method approach is recommended\. Combine intrinsic interpretability \(e\.g\., attention mechanisms, prototype\-based models\) with post hoc attribution methods \(Integrated Gradients, LRP, DeepLIFT\) and gradient\-free validation \(Occlusion Sensitivity, RISE\) to generate measurable trustworthiness metrics\. This layered validation provides both qualitative evidence \(visualizations that pathologists can inspect\) and quantitative evidence \(faithfulness metrics that regulators can audit\)\. The computational cost is substantial; high\-fidelity methods such as LRP or RISE may increase inference time by 10\-100×, but this investment is necessary to produce evidence suitable for regulatory documentation, internal validation, or risk management\.

To enable independent validation and regulatory review, XAI studies must document implementation details comprehensively\. Report i\) model architecture specifics, including which layer was analyzed for gradient\-based methods, baseline definitions for Integrated Gradients and DeepLIFT, and propagation rules for LRP; ii\) quantitative evaluation metrics such as faithfulness scores \(insertion/deletion curves, ROAR\), stability metrics \(rank correlation across repeated runs for stochastic methods like LIME\), and cross\-site robustness \(correlation of attribution maps across institutions and scanners\); and iii\) validation against expert knowledge, including overlap with pathologist\-annotated regions of interest, inter\-observer agreement metrics, and definitions of features or concepts used\. Failure to document these details is a leading cause of irreproducibility in XAI literature\.

A practical development strategy balances validation rigor with deployment efficiency\. During model development and validation, apply computationally expensive methods \(perturbation\-based validation, faithfulness metrics, concept grounding\) to establish that the model reasons correctly and to generate evidence for regulatory submissions\. Archive this validation evidence as supporting documentation\. For production deployment, use only fast methods \(Grad\-CAM, attention overlays\) that provide real\-time feedback to clinicians without degrading workflow efficiency\. This two\-stage approach ensures both rigorous validation and practical usability\.

#### Deployment Guidance by Stage of Adoption

Explainability requirements evolve as AI models progress from research prototypes to regulatory approval to clinical integration\. At each stage, different XAI methods and validation standards are appropriate\.

During early\-stage validation and clinical pilot studies, the primary goal is to verify that the model attends to histologically plausible regions and to identify systematic failure modes\. Use fast visualization methods, such as Grad\-CAM or attention overlays, to confirm that the model’s focus aligns with expected histologic regions \(e\.g\., tumor nuclei for grade prediction, immune infiltrates for immunotherapy response\)\. Document cases where the model fails \(e\.g\., misclassifications, ambiguous predictions, predictions driven by artifacts\), and characterize failure patterns \(e\.g\., model misclassified cases with extensive necrosis\)\. This exploratory analysis informs model refinement and helps set realistic expectations for clinical performance\. Quantitative faithfulness metrics are not required at this stage\.

For regulatory submissions or conformity assessment, including De Novo, 510\(k\), PMA, CE marking, or equivalent pathways, XAI evidence may support documentation of intended use, model limitations, human factors, failure modes, and risk controls\. However, explanation outputs should be presented as supporting evidence, not as substitutes for analytical validation, clinical validation, usability assessment, calibration, robustness testing, and post\-market monitoring\. Combine spatial localization methods \(Grad\-CAM or attention\) with quantitative attribution methods \(LRP or Integrated Gradients with faithfulness metrics such as insertion/deletion curves\) and perturbation\-based causal validation \(Occlusion Sensitivity or RISE\)\. Document cross\-site robustness by applying XAI methods to slides from multiple institutions, scanners, and staining batches, and report whether explanations remain consistent across these variations\. Characterize failure modes systematically, identify which types of cases produce ambiguous or misleading explanations and explain why\.

For clinical deployment and routine diagnostic use, prioritize methods that communicate reasoning in pathologist\-aligned language\. Concept\-based methods \(TCAV, Concept Bottleneck Models, Vision\-Language Models\) and example\-based methods \(Prototypes, Counterfactuals\) are more useful in clinical workflows than pixel\-level heatmaps\. Pathologists need actionable insights for, e\.g\., “this case is high\-grade because of cribriform architecture and necrosis”, rather than abstract gradient visualizations\. Combine explanations with uncertainty estimates \(trust scores, prediction confidence\) to flag ambiguous cases that warrant human review or second opinions\. Integrate XAI outputs directly into digital pathology viewers to minimize workflow disruption; standalone explanation interfaces that require context\-switching reduce adoption\.

Several XAI directions active outside CompPath have yet to see meaningful uptake, and warrant attention as the field matures\. Mechanistic interpretability techniques originating in language\-model research\[[106](https://arxiv.org/html/2608.28820#biba.bib106),[107](https://arxiv.org/html/2608.28820#biba.bib107)\], notably sparse autoencoders that decompose activations into monosemantic features, could in principle recover pathologist\-recognisable structures from FM activations without upfront concept annotations\. Causal graph neural networks and causal representation learning\[[108](https://arxiv.org/html/2608.28820#biba.bib108),[109](https://arxiv.org/html/2608.28820#biba.bib109)\]offer a principled route to distinguishing causal from confounded relationships in cell\-graph and tissue\-graph pipelines\. Migrating these methods into gigapixel\-scale CompPath, and validating them against pathologist\-defined concepts, is a productive frontier for the field\.

## Literature

Table 8:Studies reviewed in this article with corresponding XAI method in specific anatomies\.AnatomyMethodBladder,BrainCAM\[[69](https://arxiv.org/html/2608.28820#biba.bib69)\]Grad\-CAM\[[110](https://arxiv.org/html/2608.28820#biba.bib110),[111](https://arxiv.org/html/2608.28820#biba.bib111)\]Attention\[[112](https://arxiv.org/html/2608.28820#biba.bib112),[113](https://arxiv.org/html/2608.28820#biba.bib113),[67](https://arxiv.org/html/2608.28820#biba.bib67),[114](https://arxiv.org/html/2608.28820#biba.bib114)\]Head & NeckGrad\-CAM\[[115](https://arxiv.org/html/2608.28820#biba.bib115)\]BreastActivation Layer Visualization\[[116](https://arxiv.org/html/2608.28820#biba.bib116)\]Grad\-CAM\[[117](https://arxiv.org/html/2608.28820#biba.bib117),[118](https://arxiv.org/html/2608.28820#biba.bib118)\]CAM\[[119](https://arxiv.org/html/2608.28820#biba.bib119)\]Saliency Maps\[[120](https://arxiv.org/html/2608.28820#biba.bib120)\]Attention\[[121](https://arxiv.org/html/2608.28820#biba.bib121),[122](https://arxiv.org/html/2608.28820#biba.bib122),[123](https://arxiv.org/html/2608.28820#biba.bib123)\]SHAP\[[94](https://arxiv.org/html/2608.28820#biba.bib94),[86](https://arxiv.org/html/2608.28820#biba.bib86),[95](https://arxiv.org/html/2608.28820#biba.bib95)\]LIME\[[87](https://arxiv.org/html/2608.28820#biba.bib87),[86](https://arxiv.org/html/2608.28820#biba.bib86)\]LRP\[[124](https://arxiv.org/html/2608.28820#biba.bib124),[125](https://arxiv.org/html/2608.28820#biba.bib125)\]Feature Attribution\[[27](https://arxiv.org/html/2608.28820#biba.bib27),[126](https://arxiv.org/html/2608.28820#biba.bib126)\]Occlusion\[[127](https://arxiv.org/html/2608.28820#biba.bib127),[128](https://arxiv.org/html/2608.28820#biba.bib128)\]Deconvolution\[[116](https://arxiv.org/html/2608.28820#biba.bib116)\]TCAV\[[97](https://arxiv.org/html/2608.28820#biba.bib97)\]Integrated Gradient\(IG\)\[[127](https://arxiv.org/html/2608.28820#biba.bib127)\]ChestGrad\-CAM\[[129](https://arxiv.org/html/2608.28820#biba.bib129),[130](https://arxiv.org/html/2608.28820#biba.bib130),[131](https://arxiv.org/html/2608.28820#biba.bib131)\]Saliency Maps\[[132](https://arxiv.org/html/2608.28820#biba.bib132),[84](https://arxiv.org/html/2608.28820#biba.bib84)\]Attention\[[133](https://arxiv.org/html/2608.28820#biba.bib133),[65](https://arxiv.org/html/2608.28820#biba.bib65)\]CervicalGrad\-CAM\[[134](https://arxiv.org/html/2608.28820#biba.bib134),[135](https://arxiv.org/html/2608.28820#biba.bib135)\]Attention\[[136](https://arxiv.org/html/2608.28820#biba.bib136)\]GastrointestinalCAM\[[137](https://arxiv.org/html/2608.28820#biba.bib137),[138](https://arxiv.org/html/2608.28820#biba.bib138),[139](https://arxiv.org/html/2608.28820#biba.bib139),[140](https://arxiv.org/html/2608.28820#biba.bib140)\]Grad\-CAM\[[141](https://arxiv.org/html/2608.28820#biba.bib141),[142](https://arxiv.org/html/2608.28820#biba.bib142),[143](https://arxiv.org/html/2608.28820#biba.bib143),[144](https://arxiv.org/html/2608.28820#biba.bib144),[145](https://arxiv.org/html/2608.28820#biba.bib145)\]Saliency Maps\[[146](https://arxiv.org/html/2608.28820#biba.bib146),[147](https://arxiv.org/html/2608.28820#biba.bib147),[148](https://arxiv.org/html/2608.28820#biba.bib148)\]Attention\[[149](https://arxiv.org/html/2608.28820#biba.bib149),[150](https://arxiv.org/html/2608.28820#biba.bib150),[151](https://arxiv.org/html/2608.28820#biba.bib151),[152](https://arxiv.org/html/2608.28820#biba.bib152),[153](https://arxiv.org/html/2608.28820#biba.bib153),[144](https://arxiv.org/html/2608.28820#biba.bib144),[154](https://arxiv.org/html/2608.28820#biba.bib154)\]LIME\[[83](https://arxiv.org/html/2608.28820#biba.bib83)\]DeepLIFT\[[53](https://arxiv.org/html/2608.28820#biba.bib53),[83](https://arxiv.org/html/2608.28820#biba.bib83)\]Counterfactual\[[104](https://arxiv.org/html/2608.28820#biba.bib104)\]Lymph nodesGradient\[[70](https://arxiv.org/html/2608.28820#biba.bib70),[85](https://arxiv.org/html/2608.28820#biba.bib85)\]Saliency Maps\[[155](https://arxiv.org/html/2608.28820#biba.bib155),[75](https://arxiv.org/html/2608.28820#biba.bib75)\]Attention\[[156](https://arxiv.org/html/2608.28820#biba.bib156),[157](https://arxiv.org/html/2608.28820#biba.bib157),[65](https://arxiv.org/html/2608.28820#biba.bib65),[85](https://arxiv.org/html/2608.28820#biba.bib85)\]SHAP\[[85](https://arxiv.org/html/2608.28820#biba.bib85)\]RISE\[[85](https://arxiv.org/html/2608.28820#biba.bib85)\]Integrated Gradient\[[85](https://arxiv.org/html/2608.28820#biba.bib85)\]LungGrad\-CAM\[[158](https://arxiv.org/html/2608.28820#biba.bib158)\]Grad\-CAM\+\+\[[73](https://arxiv.org/html/2608.28820#biba.bib73)\]OvarianIG\[[159](https://arxiv.org/html/2608.28820#biba.bib159)\]Saliency mapS\[[159](https://arxiv.org/html/2608.28820#biba.bib159)\]Grad\-CAM\[[159](https://arxiv.org/html/2608.28820#biba.bib159)\]DeepLift\[[159](https://arxiv.org/html/2608.28820#biba.bib159)\]ProstateGrad\-CAM\[[160](https://arxiv.org/html/2608.28820#biba.bib160),[161](https://arxiv.org/html/2608.28820#biba.bib161)\]Saliency Maps\[[162](https://arxiv.org/html/2608.28820#biba.bib162),[163](https://arxiv.org/html/2608.28820#biba.bib163)\]Attention\[[164](https://arxiv.org/html/2608.28820#biba.bib164),[165](https://arxiv.org/html/2608.28820#biba.bib165),[166](https://arxiv.org/html/2608.28820#biba.bib166)\]Occlusion based\[[90](https://arxiv.org/html/2608.28820#biba.bib90)\]LIME, SHAP\[[167](https://arxiv.org/html/2608.28820#biba.bib167)\]Concept bottleneck models\[[99](https://arxiv.org/html/2608.28820#biba.bib99)\]Activation Layer Visualization\[[167](https://arxiv.org/html/2608.28820#biba.bib167)\]SkinSaliency Maps\[[168](https://arxiv.org/html/2608.28820#biba.bib168),[169](https://arxiv.org/html/2608.28820#biba.bib169)\]Attention\[[170](https://arxiv.org/html/2608.28820#biba.bib170),[65](https://arxiv.org/html/2608.28820#biba.bib65),[171](https://arxiv.org/html/2608.28820#biba.bib171)\]MultipleFeature Visualization\[[172](https://arxiv.org/html/2608.28820#biba.bib172)\]Grad\-CAM\[[173](https://arxiv.org/html/2608.28820#biba.bib173),[173](https://arxiv.org/html/2608.28820#biba.bib173)\]CAM\[[174](https://arxiv.org/html/2608.28820#biba.bib174)\]LRP\[[175](https://arxiv.org/html/2608.28820#biba.bib175),[82](https://arxiv.org/html/2608.28820#biba.bib82),[176](https://arxiv.org/html/2608.28820#biba.bib176)\]Attention\[[177](https://arxiv.org/html/2608.28820#biba.bib177),[178](https://arxiv.org/html/2608.28820#biba.bib178),[179](https://arxiv.org/html/2608.28820#biba.bib179),[180](https://arxiv.org/html/2608.28820#biba.bib180),[181](https://arxiv.org/html/2608.28820#biba.bib181),[182](https://arxiv.org/html/2608.28820#biba.bib182)\]Feature Attribution\[[183](https://arxiv.org/html/2608.28820#biba.bib183)\]
## Supplementary References

- \[1\]Lipton ZC\.The mythos of model interpretability: In machine learning, the concept of interpretability is both important and slippery\.Queue\. 2018;16\(3\):31\-57\.
- \[2\]Arrieta AB, Díaz\-Rodríguez N, Del Ser J, Bennetot A, Tabik S, Barbado A, et al\.Explainable Artificial Intelligence \(XAI\): Concepts, taxonomies, opportunities and challenges toward responsible AI\.Information fusion\. 2020;58:82\-115\.
- \[3\]Li X, Xiong H, Li X, Wu X, Zhang X, Liu J, et al\.Interpretable deep learning: Interpretation, interpretability, trustworthiness, and beyond\.Knowledge and Information Systems\. 2022;64\(12\):3197\-234\.
- \[4\]Shaban M, Khurram SA, Fraz MM, Alsubaie N, Masood I, Mushtaq S, et al\.A novel digital score for abundance of tumour infiltrating lymphocytes predicts disease free survival in oral squamous cell carcinoma\.Scientific reports\. 2019;9\(1\):13341\.
- \[5\]Graham S, Vu QD, Raza SEA, Azam A, Tsang YW, Kwak JT, et al\.Hover\-net: Simultaneous segmentation and classification of nuclei in multi\-tissue histology images\.Medical image analysis\. 2019;58:101563\.
- \[6\]Shephard AJ, Bashir RMS, Mahmood H, Jahanifar M, Minhas F, Raza SEA, et al\.A fully automated and explainable algorithm for predicting malignant transformation in oral epithelial dysplasia\.npj Precision Oncology\. 2024;8\(1\):137\.
- \[7\]Lu MY, Chen B, Williamson DF, Chen RJ, Liang I, Ding T, et al\.A visual\-language foundation model for computational pathology\.Nature Medicine\. 2024;30\(3\):863\-74\.
- \[8\]Jain S, Wallace BC\.Attention is not explanation\.arXiv preprint arXiv:190210186\. 2019\.
- \[9\]Wiegreffe S, Pinter Y\.Attention is not not explanation\.arXiv preprint arXiv:190804626\. 2019\.
- \[10\]Guidotti R, Monreale A, Ruggieri S, Turini F, Giannotti F, Pedreschi D\.A survey of methods for explaining black box models\.ACM Computing Surveys\. 2018;51\(5\):1\-42\.
- \[11\]Arun N, Gaw N, Singh P, Chang K, Aggarwal M, Chen B, et al\.Assessing the trustworthiness of saliency maps for localizing abnormalities in medical imaging\.Radiology: Artificial Intelligence\. 2021;3\(6\):e200267\.
- \[12\]Jahanifar M, Raza M, Xu K, Vuong TTL, Jewsbury R, Shephard A, et al\.Domain generalization in computational pathology: survey and guidelines\.ACM Computing Surveys\. 2025;57\(11\):1\-37\.
- \[13\]Food U, gov\) DA\.Artificial Intelligence and Machine Learning in Software as a Medical Device\. 2021\.Accessed on 22 Sept 2021\.Available from:[https://www\.fda\.gov/files/medical%20devices/published/US\-FDA\-Artificial\-Intelligence\-and\-Machine\-Learning\-Discussion\-Paper\.pdf](https://www.fda.gov/files/medical%20devices/published/US-FDA-Artificial-Intelligence-and-Machine-Learning-Discussion-Paper.pdf)\.
- \[14\]European Parliament and Council\. Regulation \(EU\) 2024/1689 of the European Parliament and of the Council of 13 June 2024 laying down harmonised rules on artificial intelligence; 2024\.Official Journal of the European Union\.Available from:[https://eur\-lex\.europa\.eu/eli/reg/2024/1689/oj](https://eur-lex.europa.eu/eli/reg/2024/1689/oj)\.
- \[15\]Mittelstadt B\.Principles alone cannot guarantee ethical AI\.Nature machine intelligence\. 2019;1\(11\):501\-7\.
- \[16\]Gebru T, Morgenstern J, Vecchione B, Vaughan JW, Wallach H, Iii HD, et al\.Datasheets for datasets\.Communications of the ACM\. 2021;64\(12\):86\-92\.
- \[17\]Mitchell M, Wu S, Zaldivar A, Barnes P, Vasserman L, Hutchinson B, et al\.Model cards for model reporting\.In: Proceedings of the Conference on Fairness, Accountability, and Transparency; 2019\. p\. 220\-9\.
- \[18\]Adebayo J, Gilmer J, Muelly M, Goodfellow I, Hardt M, Kim B\.Sanity checks for saliency maps\.Advances in neural information processing systems\. 2018;31\.
- \[19\]Hooker S, Erhan D, Kindermans PJ, Kim B\.A benchmark for interpretability methods in deep neural networks\.Advances in neural information processing systems\. 2019;32\.
- \[20\]Petsiuk V\.Rise: Randomized Input Sampling for Explanation of black\-box models\.arXiv preprint arXiv:180607421\. 2018\.
- \[21\]Wachter S, Mittelstadt B, Russell C\.Counterfactual explanations without opening the black box: Automated decisions and the GDPR\.Harv JL & Tech\. 2017;31:841\.
- \[22\]Baheti B, Innani S, Nasrallah M, Bakas S\.Prognostic stratification of glioblastoma patients by unsupervised clustering of morphology patterns on whole slide images furthering our disease understanding\.Frontiers in Neuroscience\. 2024;18:1304191\.
- \[23\]Samek W, Binder A, Montavon G, Lapuschkin S, Müller KR\.Evaluating the visualization of what a deep neural network has learned\.IEEE transactions on neural networks and learning systems\. 2016;28\(11\):2660\-73\.
- \[24\]Holzinger A, Langs G, Denk H, Zatloukal K, Müller H\.Causability and explainability of artificial intelligence in medicine\.Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery\. 2019;9\(4\):e1312\.
- \[25\]Schmitt M, Maron RC, Hekler A, Stenzinger A, Hauschild A, Weichenthal M, et al\.Hidden variables in deep learning digital pathology and their potential to cause batch effects: Prediction model study\.Journal of medical Internet research\. 2021;23\(2\):e23436\.
- \[26\]Holzinger A, Carrington A, Müller H\.Measuring the quality of explanations: the system causability scale \(SCS\) comparing human and machine explanations\.KI\-Künstliche Intelligenz\. 2020;34\(2\):193\-8\.
- \[27\]Graziani M, Andrearczyk V, Marchand\-Maillet S, Müller H\.Concept attribution: Explaining CNN decisions to physicians\.Computers in biology and medicine\. 2020;123:103865\.
- \[28\]Ibrahim R, Shafiq MO\.Explainable convolutional neural networks: a taxonomy, review, and future directions\.ACM Computing Surveys\. 2023;55\(10\):1\-37\.
- \[29\]Rueda J, Rodríguez JD, Jounou IP, Hortal\-Carmona J, Ausín T, Rodríguez\-Arias D\.“Just” accuracy? Procedural fairness demands explainability in AI\-based medical resource allocations\.AI & society\. 2024;39\(3\):1411\-22\.
- \[30\]Dy A, Buetow SM, Bredemeyer AJ, Saini ML, Lucas F, Bennett S, et al\.Clarifying validation terminologies in healthcare\.NPJ Digital Medicine\. 2026;9\(1\):318\.
- \[31\]Kamath U, Liu J\.Explainable artificial intelligence: an introduction to interpretable machine learning\. vol\. 2\.Springer; 2021\.
- \[32\]Adadi A, Berrada M\.Peeking inside the black\-box: a survey on explainable artificial intelligence \(XAI\)\.IEEE access\. 2018;6:52138\-60\.
- \[33\]Murdoch WJ, Singh C, Kumbier K, Abbasi\-Asl R, Yu B\.Definitions, methods, and applications in interpretable machine learning\.Proceedings of the National Academy of Sciences\. 2019;116\(44\):22071\-80\.
- \[34\]Shephard AJ, Jahanifar M, Wang R, Dawood M, Graham S, Sidlauskas K, et al\.An automated pipeline for tumour\-infiltrating lymphocyte scoring in breast cancer\.In: 2024 IEEE International Symposium on Biomedical Imaging \(ISBI\)\. IEEE; 2024\. p\. 1\-5\.
- \[35\]Aubreville M, Stathonikos N, Bertram CA, Klopfleisch R, Ter Hoeve N, Ciompi F, et al\.Mitosis domain generalization in histopathology images—the MIDOG challenge\.Medical Image Analysis\. 2023;84:102699\.
- \[36\]Alvarez Melis D, Jaakkola T\.Towards robust interpretability with self\-explaining neural networks\.Advances in neural information processing systems\. 2018;31\.
- \[37\]Koh PW, Nguyen T, Tang YS, Mussmann S, Pierson E, Kim B, et al\.Concept bottleneck models\.In: International conference on machine learning\. PMLR; 2020\. p\. 5338\-48\.
- \[38\]Kierner S, Kucharski J, Kierner Z\.Taxonomy of hybrid architectures involving rule\-based reasoning and machine learning in clinical decision systems: A scoping review\.Journal of biomedical informatics\. 2023;144:104428\.
- \[39\]Ilse M, Tomczak J, Welling M\.Attention\-based deep multiple instance learning\.In: International conference on machine learning\. PMLR; 2018\. p\. 2127\-36\.
- \[40\]Lundberg SM, Lee SI\.A unified approach to interpreting model predictions\.Advances in neural information processing systems\. 2017;30\.
- \[41\]Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D\.Grad\-CAM: Visual Explanations From Deep Networks via Gradient\-Based Localization\.In: Proceedings of the IEEE International Conference on Computer Vision \(ICCV\); 2017\. \.
- \[42\]Poché A, Hervier L, Bakkay MC\.Natural example\-based explainability: a survey\.In: World Conference on eXplainable Artificial Intelligence\. Springer; 2023\. p\. 24\-47\.
- \[43\]Verma S, Dickerson J, Hines K\.Counterfactual explanations for machine learning: A review\.arXiv preprint arXiv:201010596\. 2020;2\(1\):1\.
- \[44\]Ribeiro MT, Singh S, Guestrin C\." Why should i trust you?" Explaining the predictions of any classifier\.In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining; 2016\. p\. 1135\-44\.
- \[45\]Fong RC, Vedaldi A\.Interpretable explanations of black boxes by meaningful perturbation\.In: Proceedings of the IEEE international conference on computer vision; 2017\. p\. 3429\-37\.
- \[46\]Olden JD, Joy MK, Death RG\.An accurate comparison of methods for quantifying variable importance in artificial neural networks using simulated data\.Ecological modelling\. 2004;178\(3\-4\):389\-97\.
- \[47\]Kim B, Wattenberg M, Gilmer J, Cai C, Wexler J, Viegas F, et al\.Interpretability beyond feature attribution: Quantitative testing with concept activation vectors \(tcav\)\.In: International conference on machine learning\. PMLR; 2018\. p\. 2668\-77\.
- \[48\]Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, et al\.Attention is all you need\.Advances in neural information processing systems\. 2017;30\.
- \[49\]Chefer H, Gur S, Wolf L\.Transformer interpretability beyond attention visualization\.In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition; 2021\. p\. 782\-91\.
- \[50\]Zhou B, Khosla A, Lapedriza A, Oliva A, Torralba A\.Learning deep features for discriminative localization\.In: Proceedings of the IEEE conference on computer vision and pattern recognition; 2016\. p\. 2921\-9\.
- \[51\]Mundhenk TN, Chen BY, Friedland G\.Efficient saliency maps for explainable AI\.arXiv preprint arXiv:191111293\. 2019\.
- \[52\]Montavon G, Binder A, Lapuschkin S, Samek W, Müller KR\.Layer\-wise relevance propagation: an overview\.Explainable AI: interpreting, explaining and visualizing deep learning\. 2019:193\-209\.
- \[53\]Shrikumar A, Greenside P, Kundaje A\.Learning important features through propagating activation differences\.In: International conference on machine learning\. PMlR; 2017\. p\. 3145\-53\.
- \[54\]Zeiler M\.Visualizing and Understanding Convolutional Networks\.In: European conference on computer vision/arXiv\. vol\. 1311; 2014\. \.
- \[55\]Qi Z, Khorram S, Li F\.Visualizing Deep Networks by Optimizing with Integrated Gradients\.In: CVPR workshops\. vol\. 2; 2019\. p\. 1\-4\.
- \[56\]Yosinski J, Clune J, Nguyen A, Fuchs T, Lipson H\.Understanding neural networks through deep visualization\.arXiv preprint arXiv:150606579\. 2015\.
- \[57\]Vinyals O, Toshev A, Bengio S, Erhan D\.Show and tell: A neural image caption generator\.In: Proceedings of the IEEE conference on computer vision and pattern recognition; 2015\. p\. 3156\-64\.
- \[58\]Zuo J, Hong J, Zhang F, Yu C, Zhou H, Gao C, et al\.Plip: Language\-image pre\-training for person representation learning\.Advances in Neural Information Processing Systems\. 2024;37:45666\-702\.
- \[59\]Chen C, Li O, Tao D, Barnett A, Rudin C, Su JK\.This looks like that: deep learning for interpretable image recognition\.Advances in neural information processing systems\. 2019;32\.
- \[60\]Stepin I, Alonso JM, Catala A, Pereira\-Fariña M\.A survey of contrastive and counterfactual explanation generation methods for explainable artificial intelligence\.IEEE Access\. 2021;9:11974\-2001\.
- \[61\]Jiang H, Kim B, Guan M, Gupta M\.To trust or not to trust a classifier\.Advances in neural information processing systems\. 2018;31\.
- \[62\]Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al\.The PRISMA 2020 statement: an updated guideline for reporting systematic reviews\.bmj\. 2021;372\.
- \[63\]Greenhalgh T, Thorne S, Malterud K\.Time to challenge the spurious hierarchy of systematic over narrative reviews?European journal of clinical investigation\. 2018;48\(6\):e12931\.
- \[64\]Sukhera J\.Narrative reviews: flexible, rigorous, and practical\.Journal of graduate medical education\. 2022;14\(4\):414\-7\.
- \[65\]Lu MY, Williamson DF, Chen TY, Chen RJ, Barbieri M, Mahmood F\.Data\-efficient and weakly supervised computational pathology on whole\-slide images\.Nature biomedical engineering\. 2021;5\(6\):555\-70\.
- \[66\]Campanella G, Hanna MG, Geneslaw L, Miraflor A, Werneck Krauss Silva V, Busam KJ, et al\.Clinical\-grade computational pathology using weakly supervised deep learning on whole slide images\.Nature medicine\. 2019;25\(8\):1301\-9\.
- \[67\]Innani S, Bell WR, Nasrallah MP, Baheti B, Bakas S\.AI\-driven WHO 2021 classification of gliomas based only on H&E\-stained slides\.Neuro\-Oncology\. 2025:noaf189\.
- \[68\]Chen RJ, Chen C, Li Y, Chen TY, Trister AD, Krishnan RG, et al\.Scaling vision transformers to gigapixel images via hierarchical self\-supervised learning\.In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition; 2022\. p\. 16144\-55\.
- \[69\]Woerl AC, Eckstein M, Geiger J, Wagner DC, Daher T, Stenzel P, et al\.Deep learning predicts molecular subtype of muscle\-invasive bladder cancer from conventional histopathological slides\.European urology\. 2020;78\(2\):256\-64\.
- \[70\]Ji J\.Gradient\-based interpretation on convolutional neural network for classification of pathological images\.In: 2019 International Conference on Information Technology and Computer Application \(ITCA\)\. IEEE; 2019\. p\. 83\-6\.
- \[71\]Sadafi A, Adonkina O, Khakzar A, Lienemann P, Hehr RM, Rueckert D, et al\.Pixel\-level explanation of multiple instance learning models in biomedical single cell images\.In: International Conference on Information Processing in Medical Imaging\. Springer; 2023\. p\. 170\-82\.
- \[72\]Chattopadhay A, Sarkar A, Howlader P, Balasubramanian VN\.Grad\-cam\+\+: Generalized gradient\-based visual explanations for deep convolutional networks\.In: 2018 IEEE winter conference on applications of computer vision \(WACV\)\. IEEE; 2018\. p\. 839\-47\.
- \[73\]Han C, Lin J, Mai J, Wang Y, Zhang Q, Zhao B, et al\.Multi\-layer pseudo\-supervision for histopathology tissue semantic segmentation using patch\-level classification labels\.Medical Image Analysis\. 2022;80:102487\.
- \[74\]Simonyan K, Vedaldi A, Zisserman A\.Deep inside convolutional networks: Visualising image classification models and saliency maps\.arXiv preprint arXiv:13126034\. 2013\.
- \[75\]Bejnordi BE, Veta M, Van Diest PJ, Van Ginneken B, Karssemeijer N, Litjens G, et al\.Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer\.Jama\. 2017;318\(22\):2199\-210\.
- \[76\]Smilkov D, Thorat N, Kim B, Viégas F, Wattenberg M\.Smoothgrad: removing noise by adding noise\.arXiv preprint arXiv:170603825\. 2017\.
- \[77\]Dolezal JM, Wolk R, Hieromnimon HM, Howard FM, Srisuwananukorn A, Karpeyev D, et al\.Deep learning generates synthetic cancer histology for explainability and education\.NPJ precision oncology\. 2023;7\(1\):49\.
- \[78\]Srinivas S, Fleuret F\.Full\-gradient representation for neural network visualization\.Advances in neural information processing systems\. 2019;32\.
- \[79\]Mondol RK, Millar EK, Graham PH, Browne L, Sowmya A, Meijering E\.GRAPHITE: Graph\-based interpretable tissue examination for enhanced explainability in breast cancer histopathology\.Computers in Biology and Medicine\. 2025;197:111106\.
- \[80\]Chen Q, Wang Z, Lin X, Shi Y, Xu B, Chai J, et al\.Interpretable multitask model for clinical pathology image prediction and interpretation\.npj Systems Biology and Applications\. 2026\.
- \[81\]Bach S, Binder A, Montavon G, Klauschen F, Müller KR, Samek W\.On pixel\-wise explanations for non\-linear classifier decisions by layer\-wise relevance propagation\.PloS one\. 2015;10\(7\):e0130140\.
- \[82\]Hense J, Jamshidi Idaji M, Eberle O, Schnake T, Dippel J, Ciernik L, et al\.xMIL: Insightful Explanations for Multiple Instance Learning in Histopathology\.arXiv e\-prints\. 2024:arXiv\-2406\.
- \[83\]Hossain M, Haque SS, Ahmed H, Mahdi HA, Aich A\. Early stage detection and classification of colon cancer using deep learning and explainable AI on histopathological images\. Brac University; 2022\.
- \[84\]Coudray N, Ocampo PS, Sakellaropoulos T, Narula N, Snuderl M, Fenyö D, et al\.Classification and mutation prediction from non–small cell lung cancer histopathology images using deep learning\.Nature medicine\. 2018;24\(10\):1559\-67\.
- \[85\]Rahnfeld J, Naouar M, Kalweit G, Boedecker J, Dubruc E, Kalweit M\.A Comparative Study of Explainability Methods for Whole Slide Classification of Lymph Node Metastases using Vision Transformers\.medRxiv\. 2024:2024\-05\.
- \[86\]Peta J, Koppu S\.Explainable Soft Attentive EfficientNet for breast cancer classification in histopathological images\.Biomedical Signal Processing and Control\. 2024;90:105828\.
- \[87\]Palatnik de Sousa I, Maria Bernardes Rebuzzi Vellasco M, Costa da Silva E\.Local interpretable model\-agnostic explanations for classification of lymph node metastases\.Sensors\. 2019;19\(13\):2969\.
- \[88\]Bora RP, Terhörst P, Veldhuis R, Ramachandra R, Raja K\.Slice: Stabilized lime for consistent explanations for image classification\.In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition; 2024\. p\. 10988\-96\.
- \[89\]Zhao X, Huang W, Huang X, Robu V, Flynn D\.Baylime: Bayesian local interpretable model\-agnostic explanations\.In: Uncertainty in artificial intelligence\. PMLR; 2021\. p\. 887\-96\.
- \[90\]Gallo M, Krajňanskỳ V, Nenutil R, Holub P, Brázdil T\.Shedding light on the black box of a neural network used to detect prostate cancer in whole slide images by occlusion\-based explainability\.New Biotechnology\. 2023;78:52\-67\.
- \[91\]Kaczmarzyk JR, Saltz JH, Koo PK\.Explainable AI for computational pathology identifies model limitations and tissue biomarkers\.ArXiv\. 2024:arXiv\-2409\.
- \[92\]Englebert A, Stassin S, Nanfack G, Mahmoudi SA, Siebert X, Cornu O, et al\.Explaining through transformer input sampling\.In: Proceedings of the IEEE/CVF International Conference on Computer Vision; 2023\. p\. 806\-15\.
- \[93\]Sitaula C, Aryal S\.Fusion of whole and part features for the classification of histopathological image of breast tissue\.Health Information Science and Systems\. 2020;8:1\-12\.
- \[94\]Altini N, Puro E, Taccogna MG, Marino F, De Summa S, Saponaro C, et al\.Tumor Cellularity Assessment of Breast Histopathological Slides via Instance Segmentation and Pathomic Features Explainability\.Bioengineering\. 2023;10\(4\)\.Available from:[https://www\.mdpi\.com/2306\-5354/10/4/396](https://www.mdpi.com/2306-5354/10/4/396)\.
- \[95\]Wang J, Mao Y, Guan N, Xue CJ\.SHAP\-CAT: A interpretable multi\-modal framework enhancing WSI classification via virtual staining and shapley\-value\-based multimodal fusion\.arXiv preprint arXiv:241001408\. 2024\.
- \[96\]Hussein A, Prasad M, Anaissi A, Braytee A\.Vision Transformers with Autoencoders and Explainable AI for Cancer Patient Risk Stratification Using Whole Slide Imaging\.arXiv preprint arXiv:250404749\. 2025\.
- \[97\]Gamble P, Jaroensri R, Wang H, Tan F, Moran M, Brown T, et al\.Determining breast cancer biomarker status and associated morphological features using deep learning\.Communications medicine\. 2021;1\(1\):14\.
- \[98\]ŽITŇANSKÝ BA\.Concept learning in digital pathology\.Academic Thesis\. 2024\.
- \[99\]Mittmann G, Laiouar\-Pedari S, Mehrtens HA, Haggenmüller S, Bucher TC, Chanda T, et al\.Pathologist\-like explainable AI for interpretable Gleason grading in prostate cancer\.Nature Communications\. 2025 Oct;16\(1\):8959\.
- \[100\]Kazmierczak R, Berthier E, Frehse G, Franchi G\.Explainability and vision foundation models: A survey\.Information Fusion\. 2025;122:103184\.
- \[101\]Bannur S, Hyland S, Liu Q, Perez\-Garcia F, Ilse M, Castro DC, et al\.Learning to exploit temporal structure for biomedical vision\-language processing\.In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition; 2023\. p\. 15016\-27\.
- \[102\]Evans T, Retzlaff CO, Geißler C, Kargl M, Plass M, Müller H, et al\.The explainability paradox: Challenges for xAI in digital pathology\.Future Generation Computer Systems\. 2022;133:281\-96\.
- \[103\]Li O, Liu H, Chen C, Rudin C\.Deep learning for case\-based reasoning through prototypes: A neural network that explains its predictions\.In: Proceedings of the AAAI conference on artificial intelligence\. vol\. 32; 2018\. \.
- \[104\]Žigutyte L, Lenz T, Han T, Hewitt KJ, Reitsam NG, Foersch S, et al\.Counterfactual Diffusion Models for Mechanistic Explainability of Artificial Intelligence Models in Pathology\.bioRxiv\. 2024\.Available from:[https://www\.biorxiv\.org/content/early/2024/11/03/2024\.10\.29\.620913](https://www.biorxiv.org/content/early/2024/11/03/2024.10.29.620913)\.
- \[105\]Markus AF, Kors JA, Rijnbeek PR\.The role of explainability in creating trustworthy artificial intelligence for health care: a comprehensive survey of the terminology, design choices, and evaluation strategies\.Journal of biomedical informatics\. 2021;113:103655\.
- \[106\]Templeton A, Conerly T, Marcus J, Lindsey J, Bricken T, Chen B, et al\.Scaling monosemanticity: extracting interpretable features from Claude 3 Sonnet\.Transformer Circuits Thread\. 2024\.
- \[107\]Cunningham H, Ewart A, Riggs L, Huben R, Sharkey L\.Sparse autoencoders find highly interpretable features in language models\.arXiv preprint arXiv:230908600\. 2023\.
- \[108\]Schölkopf B, Locatello F, Bauer S, Ke NR, Kalchbrenner N, Goyal A, et al\.Toward causal representation learning\.Proceedings of the IEEE\. 2021;109\(5\):612\-34\.
- \[109\]Zečević M, Dhami DS, Veličković P, Kersting K\.Relating graph neural networks to structural causal models\.In: arXiv preprint arXiv:2109\.04173; 2021\. \.
- \[110\]Tang Z, Chuang KV, DeCarli C, Jin LW, Beckett L, Keiser MJ, et al\.Interpretable classification of Alzheimer’s disease pathologies with a convolutional neural network pipeline\.Nature communications\. 2019;10\(1\):2173\.
- \[111\]Kubach J, Muhlebner\-Fahrngruber A, Soylemezoglu F, Miyata H, Niehusmann P, Honavar M, et al\.Same same but different: A Web\-based deep learning application revealed classifying features for the histopathologic distinction of cortical malformations\.Epilepsia\. 2020;61\(3\):421\-32\.
- \[112\]Baheti B, Rai S, Innani S, Mehdiratta G, Bell WR, Guntuku SC, et al\.Multimodal Explainable Artificial Intelligence for Prognostic Stratification of Glioblastoma Patients\.Modern Pathology\. 2025:100797\.
- \[113\]Innani S, Bell WR, Nasrallah M, Baheti B, Bakas S\.Artificial intelligence predicts 2021 WHO glioma subtypes from whole slide images\.Cancer Research\. 2025;85\(8\_Supplement\_1\):6247\-7\.
- \[114\]Innani S, Bell WR, Harmsen H, Nasrallah MP, Baheti B, Bakas S\.Interpretable artificial intelligence based determination of glioma IDH mutation status directly from histology slides\.Neuro\-Oncology Advances\. 2025;7\(1\):vdaf140\.
- \[115\]Dörrich M, Hecht M, Fietkau R, Hartmann A, Iro H, Gostian AO, et al\.Explainable convolutional neural networks for assessing head and neck cancer histopathology\.Diagnostic Pathology\. 2023;18\(1\):121\.
- \[116\]Gecer B, Aksoy S, Mercan E, Shapiro LG, Weaver DL, Elmore JG\.Detection and classification of cancer in whole slide breast histopathology images using deep convolutional networks\.Pattern recognition\. 2018;84:345\-56\.
- \[117\]Obikane S, Aoki Y\.Weakly supervised domain adaptation with point supervision in histopathological image segmentation\.In: Pattern Recognition: ACPR 2019 Workshops, Auckland, New Zealand, November 26, 2019, Proceedings 5\. Springer; 2020\. p\. 127\-40\.
- \[118\]Pirovano A, Heuberger H, Berlemont S, Ladjal S, Bloch I\.Improving interpretability for computer\-aided diagnosis tools on whole slide imaging with multiple instance learning and gradient\-based explanations\.In: Interpretable and Annotation\-Efficient Learning for Medical Image Computing: Third International Workshop, iMIMIC 2020, Second International Workshop, MIL3ID 2020, and 5th International Workshop, LABELS 2020, Held in Conjunction with MICCAI 2020, Lima, Peru, October 4–8, 2020, Proceedings 3\. Springer; 2020\. p\. 43\-53\.
- \[119\]Yang H, Kim JY, Kim H, Adhikari SP\.Guided soft attention network for classification of breast cancer histopathology images\.IEEE transactions on medical imaging\. 2019;39\(5\):1306\-15\.
- \[120\]Zeiser FA, da Costa CA, de Oliveira Ramos G, Bohn HC, Santos I, Roehe AV\.DeepBatch: A hybrid deep learning model for interpretable diagnosis of breast cancer in whole\-slide images\.Expert Systems with Applications\. 2021;185:115586\.
- \[121\]Liang M, Chen Q, Li B, Wang L, Wang Y, Zhang Y, et al\.Interpretable classification of pathology whole\-slide images using attention based context\-aware graph convolutional neural network\.Computer Methods and Programs in Biomedicine\. 2023;229:107268\.
- \[122\]Dooper S, Pinckaers H, Aswolinskiy W, Hebeda K, Jarkman S, van der Laak J, et al\.Gigapixel end\-to\-end training using streaming and attention\.Medical Image Analysis\. 2023:102881\.
- \[123\]Sharma Y, Shrivastava A, Ehsan L, Moskaluk CA, Syed S, Brown D\.Cluster\-to\-Conquer: A Framework for End\-to\-End Multi\-Instance Learning for Whole Slide Image Classification\.In: Heinrich M, Dou Q, de Bruijne M, Lellmann J, Schläfer A, Ernst F, editors\. Proceedings of the Fourth Conference on Medical Imaging with Deep Learning\. vol\. 143 of Proceedings of Machine Learning Research; 2021\. p\. 682\-98\.
- \[124\]Binder A, Bockmayr M, Hägele M, Wienert S, Heim D, Hellweg K, et al\.Morphological and molecular breast cancer profiling through explainable machine learning\.Nature Machine Intelligence\. 2021;3\(4\):355\-66\.
- \[125\]Seegerer P, Binder A, Saitenmacher R, Bockmayr M, Alber M, Jurmeister P, et al\.Interpretable deep neural network to predict estrogen receptor status from haematoxylin\-eosin images\.Artificial Intelligence and Machine Learning for Digital Pathology: State\-of\-the\-Art and Future Challenges\. 2020:16\-37\.
- \[126\]Graziani M, Andrearczyk V, Muller H\.Visualizing and interpreting feature reuse of pretrained CNNs for histopathology\.Technological University Dublin\. 2019\.
- \[127\]Francis A U Imouokhome OGE, Chete FO\.Diagnosis and Interpretation of Breast Cancer Using Explainable Artificial Intelligence\.NIPES \- Journal of Science and Technology Research\. 2023 Jun;5\(2\)\.Available from:[https://journals\.nipes\.org/index\.php/njstr/article/view/603](https://journals.nipes.org/index.php/njstr/article/view/603)\.
- \[128\]Afshar P, Hashembeiki S, Khani P, Fatemizadeh E, Rohban MH\.Ibo: Inpainting\-based occlusion to enhance explainable artificial intelligence evaluation in histopathology\.arXiv preprint arXiv:240816395\. 2024\.
- \[129\]He J, Shang L, Ji H, Zhang X\.Deep learning features for lung adenocarcinoma classification with tissue pathology images\.In: Neural Information Processing: 24th International Conference, ICONIP 2017, Guangzhou, China, November 14–18, 2017, Proceedings, Part IV 24\. Springer; 2017\. p\. 742\-51\.
- \[130\]Teramoto A, Yamada A, Kiriyama Y, Tsukamoto T, Yan K, Zhang L, et al\.Automated classification of benign and malignant cells from lung cytological images using deep convolutional neural network\.Informatics in Medicine Unlocked\. 2019;16:100205\.
- \[131\]Zhang H, Meng Y, Zhao Y, Qiao Y, Yang X, Coupland SE, et al\.DTFD\-MIL: Double\-tier feature distillation multiple instance learning for histopathology whole slide image classification\.In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition; 2022\. p\. 18802\-12\.
- \[132\]Wang X, Chen H, Gan C, Lin H, Dou Q, Tsougenis E, et al\.Weakly supervised deep learning for whole slide lung cancer image analysis\.IEEE transactions on cybernetics\. 2019;50\(9\):3950\-62\.
- \[133\]Zhao L, Xu X, Hou R, Zhao W, Zhong H, Teng H, et al\.Lung cancer subtype classification using histopathological images based on weakly supervised multi\-instance learning\.Physics in Medicine & Biology\. 2021;66\(23\):235013\.
- \[134\]Gupta M, Das C, Roy A, Gupta P, Pillai GR, Patole K\.Region of interest identification for cervical cancer images\.In: 2020 IEEE 17th International Symposium on Biomedical Imaging \(ISBI\)\. IEEE; 2020\. p\. 1293\-6\.
- \[135\]Kiran GV K, Meghana Reddy G\.Automatic classification of whole slide pap smear images using CNN with PCA based feature interpretation\.In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops; 2019\. p\. 0\-0\.
- \[136\]Fremond S, Andani S, Wolf JB, Dijkstra J, Melsbach S, Jobsen JJ, et al\.Interpretable deep learning model to predict the molecular classification of endometrial cancer from haematoxylin and eosin\-stained whole\-slide images: a combined analysis of the PORTEC randomised trials and clinical cohorts\.The Lancet Digital Health\. 2023;5\(2\):e71\-82\.
- \[137\]Heinemann F, Birk G, Stierstorfer B\.Deep learning enables pathologist\-like scoring of NASH models\.Scientific reports\. 2019;9\(1\):18454\.
- \[138\]Kiani A, Uyumazturk B, Rajpurkar P, Wang A, Gao R, Jones E, et al\.Impact of a deep learning assistant on the histopathologic classification of liver cancer\.NPJ digital medicine\. 2020;3\(1\):23\.
- \[139\]Yan C, Xu J, Xie J, Cai C, Lu H\.Prior\-aware CNN with multi\-task learning for colon images analysis\.In: 2020 IEEE 17th International Symposium on Biomedical Imaging \(ISBI\)\. IEEE; 2020\. p\. 254\-7\.
- \[140\]Kather JN, Krisam J, Charoentong P, Luedde T, Herpel E, Weis CA, et al\.Predicting survival from colorectal cancer histology slides using deep learning: A retrospective multicenter study\.PLoS medicine\. 2019;16\(1\):e1002730\.
- \[141\]Korbar B, Olofson AM, Miraflor AP, Nicka CM, Suriawinata MA, Torresani L, et al\.Looking under the hood: Deep neural network visualization to interpret whole\-slide image analysis outcomes for colorectal polyps\.In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops; 2017\. p\. 69\-75\.
- \[142\]Kowsari K, Sali R, Ehsan L, Adorno W, Ali A, Moore S, et al\.Hmic: Hierarchical medical image classification, a deep learning approach\.Information\. 2020;11\(6\):318\.
- \[143\]Niehues JM, Quirke P, West NP, Grabsch HI, van Treeck M, Schirris Y, et al\.Generalizable biomarker prediction from cancer pathology slides with self\-supervised deep learning: A retrospective multi\-centric study\.Cell Reports Medicine\. 2023;4\(4\)\.
- \[144\]Li J, Li W, Sisk A, Ye H, Wallace WD, Speier W, et al\.A multi\-resolution model for histopathology image classification and localization with multiple instance learning\.Computers in biology and medicine\. 2021;131:104253\.
- \[145\]Kanavati F, Ichihara S, Rambeau M, Iizuka O, Arihiro K, Tsuneki M\.Deep learning models for gastric signet ring cell carcinoma classification in whole slide images\.Technology in Cancer Research & Treatment\. 2021;20:15330338211027901\.
- \[146\]Laleh NG, Echle A, Muti HS, Hewitt KJ, Schulz V, Kather JN\.Deep Learning for interpretable end\-to\-end survival prediction in gastrointestinal cancer histopathology\.In: COMPAY 2021: The third MICCAI workshop on Computational Pathology; 2021\. \.
- \[147\]Kuntz S, Krieghoff\-Henning E, Kather JN, Jutzi T, Höhn J, Kiehl L, et al\.Gastrointestinal cancer classification and prognostication from histology using deep learning: Systematic review\.European Journal of Cancer\. 2021;155:200\-15\.
- \[148\]Neto PC, Oliveira SP, Montezuma D, Fraga J, Monteiro A, Ribeiro L, et al\.iMIL4PATH: A semi\-supervised interpretable approach for colorectal whole\-slide images\.Cancers\. 2022;14\(10\):2489\.
- \[149\]Zhu Z, Ding X, Zhang D, Wang L\.Weakly\-supervised balanced attention network for gastric pathology image localization and classification\.In: 2020 IEEE 17th International Symposium on Biomedical Imaging \(ISBI\)\. IEEE; 2020\. p\. 1\-4\.
- \[150\]Calderaro J, Ghaffari Laleh N, Zeng Q, Maille P, Favre L, Pujals A, et al\.Deep learning\-based phenotyping reclassifies combined hepatocellular\-cholangiocarcinoma\.Nature Communications\. 2023;14\(1\):8290\.
- \[151\]Zeng Q, Klein C, Caruso S, Maille P, Allende DS, Mínguez B, et al\.Artificial intelligence\-based pathology as a biomarker of sensitivity to atezolizumab–bevacizumab in patients with hepatocellular carcinoma: a multicentre retrospective study\.The Lancet Oncology\. 2023;24\(12\):1411\-22\.
- \[152\]Jiang X, Hoffmeister M, Brenner H, Muti HS, Yuan T, Foersch S, et al\.End\-to\-end prognostication in colorectal cancer by deep learning: a retrospective, multicentre study\.The Lancet Digital Health\. 2024;6\(1\):e33\-43\.
- \[153\]Zhang Z, Chen P, McGough M, Xing F, Wang C, Bui M, et al\.Pathologist\-level interpretable whole\-slide cancer diagnosis with deep learning\.Nature Machine Intelligence\. 2019;1\(5\):236\-45\.
- \[154\]Chakraborty S, Gupta R, Ma K, Govind D, Sarder P, Choi WT, et al\.Predicting the Visual Attention of Pathologists Evaluating Whole Slide Images of Cancer\.In: International Workshop on Medical Optical Imaging and Virtual Microscopy Image Analysis\. Springer; 2022\. p\. 11\-21\.
- \[155\]Wang X, Chen Y, Gao Y, Zhang H, Guan Z, Dong Z, et al\.Predicting gastric cancer outcome from resected lymph node histopathology images using deep learning\.Nature communications\. 2021;12\(1\):1637\.
- \[156\]Wu S, Hong G, Xu A, Zeng H, Chen X, Wang Y, et al\.Artificial intelligence\-based model for lymph node metastases detection on whole slide images in bladder cancer: a retrospective, multicentre, diagnostic study\.The Lancet Oncology\. 2023;24\(4\):360\-70\.
- \[157\]Yu JG, Wu Z, Ming Y, Deng S, Li Y, Ou C, et al\.Prototypical multiple instance learning for predicting lymph node metastasis of breast cancer from whole\-slide pathological images\.Medical Image Analysis\. 2023;85:102748\.
- \[158\]Civit\-Masot J, Bañuls\-Beaterio A, Domínguez\-Morales M, Rivas\-Pérez M, Muñoz\-Saavedra L, Corral JMR\.Non\-small cell lung cancer diagnosis aid with histopathological images using Explainable Deep Learning techniques\.Computer Methods and Programs in Biomedicine\. 2022;226:107108\.
- \[159\]Radhakrishnan M, Sampathila N, Muralikrishna H, Swathi KS\.Advancing Ovarian Cancer Diagnosis Through Deep Learning and eXplainable AI: A Multiclassification Approach\.IEEE Access\. 2024;12:116968\-86\.
- \[160\]Pinckaers H, Bulten W, van der Laak J, Litjens G\.Detection of prostate cancer in whole\-slide images through end\-to\-end training with image\-level labels\.IEEE Transactions on Medical Imaging\. 2021;40\(7\):1817\-26\.
- \[161\]Ferrero A, Ghelichkhan E, Manoochehri H, Ho MM, Albertson DJ, Brintz BJ, et al\.HistoEM: A Pathologist\-Guided and Explainable Workflow Using Histogram Embedding for Gland Classification\.Modern Pathology\. 2024;37\(4\):100447\.
- \[162\]Silva\-Rodríguez J, Colomer A, Sales MA, Molina R, Naranjo V\.Going deeper through the Gleason scoring scale: An automatic end\-to\-end system for histology prostate grading and cribriform pattern detection\.Computer methods and programs in biomedicine\. 2020;195:105637\.
- \[163\]Singhal N, Soni S, Bonthu S, Chattopadhyay N, Samanta P, Joshi U, et al\.A deep learning system for prostate cancer diagnosis and grading in whole slide images of core needle biopsies\.Scientific reports\. 2022;12\(1\):3383\.
- \[164\]Tavolara TE, Su Z, Gurcan MN, Niazi MKK\.One label is all you need: Interpretable AI\-enhanced histopathology for oncology\.In: Seminars in Cancer Biology\. Elsevier; 2023\. \.
- \[165\]Xiang J, Wang X, Wang X, Zhang J, Yang S, Yang W, et al\.Automatic diagnosis and grading of Prostate Cancer with weakly supervised learning on whole slide images\.Computers in Biology and Medicine\. 2023;152:106340\.
- \[166\]Grisi C, Litjens G, van der Laak J\.Hierarchical Vision Transformers for Context\-Aware Prostate Cancer Grading in Whole Slide Images\.arXiv preprint arXiv:231212619\. 2023\.
- \[167\]Bhattacharjee S, Hwang YB, Ikromjanov K, Sumon RI, Kim HC, Choi HK\.An explainable computer vision in histopathology: techniques for interpreting black box model\.In: 2022 International Conference on Artificial Intelligence in Information and Communication \(ICAIIC\)\. IEEE; 2022\. p\. 392\-8\.
- \[168\]Thomas SM, Lefevre JG, Baxter G, Hamilton NA\.Interpretable deep learning systems for multi\-class segmentation and classification of non\-melanoma skin cancer\.Medical Image Analysis\. 2021;68:101915\.
- \[169\]Xie P, Zuo K, Zhang Y, Li F, Yin M, Lu K\.Interpretable classification from skin cancer histology slides using deep learning: A retrospective multicenter study\.arXiv preprint arXiv:190406156\. 2019\.
- \[170\]del Amor R, Launet L, Colomer A, Moscardó A, Mosquera\-Zamudio A, Monteagudo C, et al\.An attention\-based weakly supervised framework for spitzoid melanocytic lesion diagnosis in WSI\.arXiv preprint arXiv:210409878\. 2021\.
- \[171\]Geijs D, Dooper S, Aswolinskiy W, Hillen LM, Amir AL, Litjens G\.Detection and subtyping of basal cell carcinoma in whole\-slide histopathology using weakly\-supervised learning\.Medical Image Analysis\. 2023:103063\.
- \[172\]Xu Y, Jia Z, Wang LB, Ai Y, Zhang F, Lai M, et al\.Large scale tissue histopathology image classification, segmentation, and visualization via deep convolutional activation features\.BMC bioinformatics\. 2017;18:1\-17\.
- \[173\]Chan L, Hosseini MS, Rowsell C, Plataniotis KN, Damaskinos S\.Histosegnet: Semantic segmentation of histological tissue type in whole slide images\.In: Proceedings of the IEEE/CVF International Conference on Computer Vision; 2019\. p\. 10662\-71\.
- \[174\]Huang Y, Chung AC\.Evidence localization for pathology images using weakly supervised learning\.In: Medical Image Computing and Computer Assisted Intervention–MICCAI 2019: 22nd International Conference, Shenzhen, China, October 13–17, 2019, Proceedings, Part I 22\. Springer; 2019\. p\. 613\-21\.
- \[175\]Hägele M, Seegerer P, Lapuschkin S, Bockmayr M, Samek W, Klauschen F, et al\.Resolving challenges in deep learning\-based analyses of histopathological images using explanation methods\.Scientific reports\. 2020;10\(1\):6423\.
- \[176\]Keyl J, Keyl P, Montavon G, Hosch R, Brehmer A, Mochmann L, et al\.Decoding pan\-cancer treatment outcomes using multimodal real\-world data and explainable artificial intelligence\.medRxiv\. 2023:2023\-10\.
- \[177\]Saldanha OL, Loeffler CM, Niehues JM, van Treeck M, Seraphin TP, Hewitt KJ, et al\.Self\-supervised attention\-based deep learning for pan\-cancer mutation prediction from histopathology\.NPJ Precision Oncology\. 2023;7\(1\):35\.
- \[178\]Lee Y, Park JH, Oh S, Shin K, Sun J, Jung M, et al\.Derivation of prognostic contextual histopathological features from whole\-slide images of tumours via graph deep learning\.Nature Biomedical Engineering\. 2022:1\-15\.
- \[179\]Wang Z, Gao Q, Yi X, Zhang X, Zhang Y, Zhang D, et al\.Surformer: An interpretable pattern\-perceptive survival transformer for cancer survival prediction from histopathology whole slide images\.Computer Methods and Programs in Biomedicine\. 2023;241:107733\.
- \[180\]Shao Z, Bian H, Chen Y, Wang Y, Zhang J, Ji X, et al\.Transmil: Transformer based correlated multiple instance learning for whole slide image classification\.Advances in neural information processing systems\. 2021;34:2136\-47\.
- \[181\]Wang X, Yang S, Zhang J, Wang M, Zhang J, Yang W, et al\.Transformer\-based unsupervised contrastive learning for histopathological image classification\.Medical image analysis\. 2022;81:102559\.
- \[182\]Kapse S, Pati P, Das S, Zhang J, Chen C, Vakalopoulou M, et al\.SI\-MIL: Taming Deep MIL for Self\-Interpretability in Gigapixel Histopathology\.In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition; 2024\. p\. 11226\-37\.
- \[183\]Diao JA, Wang JK, Chui WF, Mountain V, Gullapally SC, Srinivasan R, et al\.Human\-interpretable image features derived from densely mapped cancer pathology slides predict diverse molecular phenotypes\.Nature communications\. 2021;12\(1\):1613\.