qZACH-ViT: Quantization-Aware Intrinsic Explanations with Recursive Attribution-Stabilized Optimization

# qZACH-ViT: Quantization-Aware Intrinsic Explanations with Recursive Attribution-Stabilized Optimization
Source: [https://arxiv.org/html/2607.15421](https://arxiv.org/html/2607.15421)
Athanasios Angelakis1,2 1BioML Lab, Research Institute CODE, UniBw, Munich, Germany 2Epidemiology and Data Science, Amsterdam UMC, Amsterdam, Netherlands athanasios\.angelakis@unibw\.de

###### Abstract

Compact medical\-image classifiers need efficiency and interpretable evidence, yet these goals are often addressed separately\. We introduce qZACH\-ViT, a quantization\-aware extension of the zero\-token \(CLS\-token\-free\), position\-free ZACH\-ViT backbone with recursive intrinsic patch\-level class evidence\. We also introduce Recursive Attribution\-Stabilized Optimization \(RASO\), which norm\-matches classification and attribution gradients and removes attribution components that conflict with classification\. We evaluate four controlled conditions on seven MedMNIST datasets using 50 training images per class and ten fixed seeds, completing 280 runs\. All 210 qZACH\-ViT checkpoints are converted to executable mixed\-precision ONNX INT8 graphs containing 16 signed INT8MatMulIntegerprojections with INT32 accumulation\. Deployed mixed\-precision INT8 qZACH\-ViT with Adam improves the FP32 ZACH\-ViT baseline mean on all seven datasets, with a mean paired gain of 0\.0313 in the dataset\-specific primary metric; qZACH\-ViT with RASO yields a mean gain of 0\.0368\. Across 964,920 source\-to\-INT8 test comparisons, prediction agreement is 99\.9751%, with a mean absolute primary\-metric change of 0\.000133 and a maximum of 0\.004386\. Across 3,600 matched intrinsic maps, mean cosine similarity is 0\.999955, mean rank correlation is 0\.9944, and mean top\-10% overlap is 0\.9692\. ONNX artifacts are 70\.0% smaller than source checkpoints and provide1\.41×1\.41\\timesand2\.39×2\.39\\timesend\-to\-end CPU speedups with one and four threads\. RASO significantly reduces sufficiency error and improves input\-noise stability over Adam with the same attribution loss, but does not dominate every predictive or explainable artificial intelligence \(XAI\) metric\. These results establish qZACH\-ViT as a deployable compact intrinsically explainable model and RASO as a targeted stability\-oriented optimization procedure\.

## 1Introduction

Vision Transformers can be accurate but expensive, and their predictions are commonly explained only after training\. These issues become tightly coupled under compression: a quantized model may preserve a class label while changing the internal evidence that supports it\. In medical imaging, prediction retention alone is therefore insufficient\. Deployment validation should ask whether the converted model remains accurate, whether its evidence remains stable, and whether the claimed integer operations are actually present and executed\.

The starting point of this study is the ZACH\-ViT line\. The architecture was first introduced for low\-data lung\-ultrasound classification\(Angelakis et al\.,[2025](https://arxiv.org/html/2607.15421#bib.bib1)\), then formalized and evaluated systematically across seven MedMNIST tasks\(Angelakis,[2026](https://arxiv.org/html/2607.15421#bib.bib2)\), and subsequently stress\-tested under common corruptions and adversarial perturbations\(Angelakis and Gomez\-Barrero,[2026](https://arxiv.org/html/2607.15421#bib.bib3)\)\. The present work addresses a different question: whether the same compact zero\-token backbone can provide intrinsic patch evidence, support explanation\-stabilized optimization, and survive conversion to an executable INT8 deployment graph\. ZACH\-ViT uses neither positional embeddings nor a classification token, aggregates patch representations by global average pooling, and contains adaptive residual projections\. This design yields approximately 0\.25 million parameters and makes direct patch\-wise evidence accounting possible without attaching a separate post\-hoc explainer\.

We develop this opportunity in two steps\. The first contribution, qZACH\-ViT, is an architectural and deployment extension\. It adds W8A8 quantization\-aware training and two patch\-level class\-evidence heads\. The raw evidence exactly reconstructs the class logit through spatial averaging and recursive stage fusion, while normalized maps are available during training and inference\. After training, the selected checkpoints are converted to mixed\-precision ONNX graphs in which all 16 learned projections use signed INT8MatMulIntegeroperators with INT32 accumulation\. The second contribution, RASO, is an optimization procedure for the coupled classification and attribution objectives\. It norm\-matches the attribution gradient to the classification gradient and, when the two conflict, removes the conflicting attribution component before the Adam update\. Classification is treated as the protected objective, while explanation consistency is optimized when compatible\.

The distinction between these contributions determines the controlled experiment\. ZACH\-ViT with Adam is the full\-precision baseline\. qZACH\-ViT with Adam isolates quantization\-aware intrinsic explanation\. qZACH\-ViT with Adam plus the same attribution loss tests whether ordinary scalar loss addition is sufficient\. qZACH\-ViT with RASO tests the additional value of norm matching and conflict\-aware projection\. The three qZACH\-ViT conditions share identical initial states within every dataset\-seed pair\. Predictive results for the three qZACH\-ViT conditions are reported from the actual converted ONNX INT8 models, not from the fake\-quantized training path\.

Our contributions are:

- •We introduce qZACH\-ViT, a compact quantization\-aware extension of ZACH\-ViT with recursive intrinsic patch\-level class evidence and exact raw\-evidence logit completeness\.
- •We introduce RASO, an asymmetric gradient\-combination procedure that targets recursive and quantization\-consistent explanations while protecting the classification direction\.
- •We validate actual deployment for 210 converted checkpoints\. Every graph contains 16 signed INT8MatMulIntegerprojections, and every ONNX Runtime profile records integer execution\.
- •We evaluate 964,920 source\-to\-INT8 test predictions and 3,600 matched intrinsic maps, together with 280 controlled training runs, 21,600 source\-model XAI evaluations, and 168 parameter\-randomization checks\.
- •We report positive and negative findings\. Actual INT8 qZACH\-ViT improves the baseline mean on all seven datasets and retains predictions, primary metrics, and intrinsic maps with high fidelity\. RASO strengthens selected stability and sufficiency properties, but it is not universally best in prediction or faithfulness, and several post\-hoc methods remain stronger on individual XAI metrics\.

The deployment is mixed precision rather than fully integer\-only\. LayerNorm, residual additions, attention score and context products, softmax, pooling, dequantization, bias addition, and attribution normalization remain in FP32\. The study also contains no localization annotations and makes no clinical\-localization claim\.

## 2Related Work

#### Compact Vision Transformers and the ZACH\-ViT lineage\.

The original Vision Transformer represents an image as a sequence of patch tokens and generally benefits from large\-scale pretraining\(Dosovitskiy et al\.,[2021](https://arxiv.org/html/2607.15421#bib.bib5)\); DeiT showed that careful distillation and augmentation can improve data efficiency\(Touvron et al\.,[2021](https://arxiv.org/html/2607.15421#bib.bib6)\)\. ZACH\-ViT follows a different compact\-design route\. It was first introduced for lung\-ultrasound classification\(Angelakis et al\.,[2025](https://arxiv.org/html/2607.15421#bib.bib1)\), then formalized as a position\-free, zero\-token architecture and evaluated under a strict low\-data MedMNIST protocol\(Angelakis,[2026](https://arxiv.org/html/2607.15421#bib.bib2)\)\. A later study examined its corruption and adversarial robustness\(Angelakis and Gomez\-Barrero,[2026](https://arxiv.org/html/2607.15421#bib.bib3)\)\. The current work is not a robustness extension\. It introduces a quantization\-aware intrinsically explainable branch, qZACH\-ViT, and a dedicated optimization method, RASO\.

#### Quantization of Vision Transformers\.

Quantization\-aware training uses simulated low\-precision operations during optimization so that a model can adapt to clipping and rounding\(Jacob et al\.,[2018](https://arxiv.org/html/2607.15421#bib.bib7)\); broader treatments distinguish fake\-quantized training from converted integer execution and hardware realization\(Gholami et al\.,[2022](https://arxiv.org/html/2607.15421#bib.bib8)\)\. Vision Transformers create additional challenges because LayerNorm, GELU, softmax, attention maps, and activation outliers are not uniformly quantization\-friendly\. Prior work includes fully differentiable learned\-bit quantization\(Li et al\.,[2022a](https://arxiv.org/html/2607.15421#bib.bib9)\), post\-training twin\-uniform quantization\(Yuan et al\.,[2022](https://arxiv.org/html/2607.15421#bib.bib10)\), fully quantized LayerNorm and softmax approximations\(Lin et al\.,[2022](https://arxiv.org/html/2607.15421#bib.bib11)\), integer\-only ViT inference\(Li and Gu,[2023](https://arxiv.org/html/2607.15421#bib.bib12)\), low\-bit attention rectification and distillation\(Li et al\.,[2022b](https://arxiv.org/html/2607.15421#bib.bib13)\), and scale reparameterization for post\-training quantization\(Li et al\.,[2023](https://arxiv.org/html/2607.15421#bib.bib14)\)\. Our goal differs from maximizing ImageNet accuracy at the lowest possible bit width\. We use a fixed signed W8A8 scheme to study whether decisions and intrinsic patch evidence can be retained together in a compact model\. Quantization\-aware training is followed by explicit ONNX conversion and execution auditing: all learned projections use signed INT8 matrix multiplication with INT32 accumulation, while non\-projection operations remain in FP32\.

#### Post\-hoc, intrinsic, and explanation\-by\-design models\.

Integrated Gradients is derived from sensitivity and implementation\-invariance axioms\(Sundararajan et al\.,[2017](https://arxiv.org/html/2607.15421#bib.bib15)\)\. Grad\-CAM uses class gradients to weight feature maps\(Selvaraju et al\.,[2017](https://arxiv.org/html/2607.15421#bib.bib16)\)\. Attention rollout estimates information flow across transformer layers\(Abnar and Zuidema,[2020](https://arxiv.org/html/2607.15421#bib.bib17)\), while transformer\-specific relevance propagation extends explanation beyond raw attention\(Chefer et al\.,[2021](https://arxiv.org/html/2607.15421#bib.bib18)\)\. RISE estimates salience through randomized masking and black\-box prediction changes\(Petsiuk et al\.,[2018](https://arxiv.org/html/2607.15421#bib.bib19)\)\. These methods are useful, but attention alone need not be explanatory\(Jain and Wallace,[2019](https://arxiv.org/html/2607.15421#bib.bib20)\), explanations may violate input invariance\(Kindermans et al\.,[2019](https://arxiv.org/html/2607.15421#bib.bib21)\), and visually plausible maps can persist after model randomization\(Adebayo et al\.,[2018](https://arxiv.org/html/2607.15421#bib.bib22)\)\.

Intrinsic approaches make interpretability part of the predictive computation\. Self\-explaining neural networks learn locally interpretable basis concepts\(Alvarez\-Melis and Jaakkola,[2018](https://arxiv.org/html/2607.15421#bib.bib23)\); prototype networks explain decisions through similarity to learned exemplars\(Chen et al\.,[2019](https://arxiv.org/html/2607.15421#bib.bib24)\); concept bottleneck models predict human\-specified concepts before the target\(Koh et al\.,[2020](https://arxiv.org/html/2607.15421#bib.bib25)\); and B\-cos models promote input\-weight alignment for inherently interpretable vision models\(Böhle et al\.,[2023a](https://arxiv.org/html/2607.15421#bib.bib26),[b](https://arxiv.org/html/2607.15421#bib.bib27)\)\. qZACH\-ViT follows another route: the class logit is explicitly decomposed into patch\-level evidence at two recursive depths\. The raw evidence is complete by construction, whereas the normalized map is used for visualization and regularization\.

#### Faithfulness, robustness, and evaluation of explanations\.

Deletion and insertion tests measure score changes when regions are removed or restored in attribution order\(Petsiuk et al\.,[2018](https://arxiv.org/html/2607.15421#bib.bib19)\)\. Infidelity and sensitivity formalize complementary desirable properties\(Yeh et al\.,[2019](https://arxiv.org/html/2607.15421#bib.bib28)\), while remove\-and\-retrain evaluation tests whether identified features carry predictive information under retraining\(Hooker et al\.,[2019](https://arxiv.org/html/2607.15421#bib.bib29)\)\. Quantus organizes families of faithfulness, robustness, localization, complexity, and randomization metrics\(Hedström et al\.,[2023](https://arxiv.org/html/2607.15421#bib.bib30)\)\. Explanations can nevertheless be fragile under visually negligible perturbations\(Ghorbani et al\.,[2019](https://arxiv.org/html/2607.15421#bib.bib31); Dombrowski et al\.,[2019](https://arxiv.org/html/2607.15421#bib.bib32)\), and deletion\-style metrics can be affected by out\-of\-distribution perturbations and rank\-only evaluation\(Gomez et al\.,[2022](https://arxiv.org/html/2607.15421#bib.bib33)\)\. SaCo was proposed to better discriminate faithful transformer explanations from random attribution\(Wu et al\.,[2024](https://arxiv.org/html/2607.15421#bib.bib34)\)\. We therefore combine perturbation, sufficiency, stability, runtime, and randomization measures rather than treating any single score as decisive\. The seven MedMNIST archives contain no masks or boxes, so we do not report localization scores without ground truth\.

#### Gradient conflict and multi\-objective optimization\.

Multi\-gradient descent seeks Pareto\-stationary solutions for multi\-task learning\(Sener and Koltun,[2018](https://arxiv.org/html/2607.15421#bib.bib35)\)\. GradNorm balances loss gradients by their magnitudes\(Chen et al\.,[2018](https://arxiv.org/html/2607.15421#bib.bib36)\)\. PCGrad removes conflicting components between task gradients\(Yu et al\.,[2020](https://arxiv.org/html/2607.15421#bib.bib37)\), GradVac adapts gradient similarity targets\(Wang et al\.,[2020](https://arxiv.org/html/2607.15421#bib.bib38)\), CAGrad seeks a conflict\-averse direction with multi\-objective guarantees\(Liu et al\.,[2021](https://arxiv.org/html/2607.15421#bib.bib39)\), and Nash\-MTL formulates gradient combination as a bargaining problem\(Navon et al\.,[2022](https://arxiv.org/html/2607.15421#bib.bib40)\)\. RASO is related to this literature and is not presented as the first gradient\-projection method\. Its contribution is the specific asymmetric construction for a protected classification objective and a recursive quantization\-consistency objective: it combines norm matching with projection of only the attribution gradient, then applies Adam\. The loss\-only control determines whether this procedure adds value beyond ordinary scalar regularization\.

## 3Method

### 3\.1Backbone

An RGB imagex∈ℝ224×224×3x\\in\\mathbb\{R\}^\{224\\times 224\\times 3\}is divided into non\-overlapping16×1616\\times 16patches, producingN=196N=196flattened patch vectors\. A learned projection maps each patch to 128 dimensions, followed by ReLU\. Two transformer blocks use 8\-head self\-attention, pre\-normalization, dropout 0\.1, ReLU feed\-forward layers, and adaptive residual projection when the feature dimension changes\. The block widths are 128 and 64\. The original ZACH\-ViT baseline then applies global average pooling, a 128\-to\-64 MLP head, and the classifier\. No positional embeddings and no classification token are used\(Angelakis,[2026](https://arxiv.org/html/2607.15421#bib.bib2)\)\.

### 3\.2qZACH\-ViT

qZACH\-ViT preserves the zero\-token backbone but changes the prediction head so that class evidence is available at each patch\. Lethi\(1\)h\_\{i\}^\{\(1\)\}be the representation of patchiiafter the first transformer block\. A linear evidence head produces

ei\(1\)=W1​hi\(1\)\+b1\.e\_\{i\}^\{\(1\)\}=W\_\{1\}h\_\{i\}^\{\(1\)\}\+b\_\{1\}\.\(1\)After the second transformer block, a token\-wise MLP with widths 128 and 64 producesri\(2\)r\_\{i\}^\{\(2\)\}, followed by a second evidence head

ei\(2\)=W2​ri\(2\)\+b2\.e\_\{i\}^\{\(2\)\}=W\_\{2\}r\_\{i\}^\{\(2\)\}\+b\_\{2\}\.\(2\)For multiclass tasks,ei\(r\)∈ℝCe\_\{i\}^\{\(r\)\}\\in\\mathbb\{R\}^\{C\}\. Binary tasks use one logit\. Stage logits and the final logit are

z\(r\)=1N​∑i=1Nei\(r\),z=ρ​z\(1\)\+\(1−ρ\)​z\(2\),z^\{\(r\)\}=\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}e\_\{i\}^\{\(r\)\},\\qquad z=\\rho z^\{\(1\)\}\+\(1\-\\rho\)z^\{\(2\)\},\(3\)withρ=0\.25\\rho=0\.25\. Equation[3](https://arxiv.org/html/2607.15421#S3.E3)is an exact decomposition: the implementation audit obtains zero reconstruction error at floating\-point precision for all seven output dimensions tested\. qZACH\-ViT contains 248,386 to 250,326 parameters depending on class count, only 129 to 1,419 more than the corresponding baseline\.

### 3\.3Recursive intrinsic attribution

For target classcc, raw patch evidence is converted into a normalized map by

Ai\(r\)​\(c\)=exp⁡\(ei,c\(r\)/τ\)∑j=1Nexp⁡\(ej,c\(r\)/τ\),τ=1\.A\_\{i\}^\{\(r\)\}\(c\)=\\frac\{\\exp\(e\_\{i,c\}^\{\(r\)\}/\\tau\)\}\{\\sum\_\{j=1\}^\{N\}\\exp\(e\_\{j,c\}^\{\(r\)\}/\\tau\)\},\\qquad\\tau=1\.\(4\)For binary classification, evidence is signed according to the target label before normalization\. The recursive attribution is

Arec​\(c\)=ρ​A\(1\)​\(c\)\+\(1−ρ\)​A\(2\)​\(c\)\.A^\{\\mathrm\{rec\}\}\(c\)=\\rho A^\{\(1\)\}\(c\)\+\(1\-\\rho\)A^\{\(2\)\}\(c\)\.\(5\)The exact completeness statement applies to the raw evidence in Equation[3](https://arxiv.org/html/2607.15421#S3.E3)\. The normalized maps in Equation[5](https://arxiv.org/html/2607.15421#S3.E5)preserve patch order and support comparison and visualization, but they are not themselves additive logit decompositions\.

### 3\.4W8A8 quantization\-aware training

Weights and activations use signed symmetric fake quantization during training with integer range\[−127,127\]\[\-127,127\]and a straight\-through estimator\. For tensoruuand scaless,

Q​\(u;s\)=s​clip⁡\(round⁡\(u/s\),−127,127\)\.Q\(u;s\)=s\\,\\operatorname\{clip\}\\left\(\\operatorname\{round\}\(u/s\),\-127,127\\right\)\.\(6\)Weight scales use the current absolute maximum per output channel\. Activation scales use a per\-tensor exponential moving average of the absolute maximum with momentum 0\.95\. Activation observers are updated during the first three epochs and then frozen\. Quantization\-aware operations cover the patch projection, query, key, value and attention\-output projections, feed\-forward layers, adaptive residual projections, token MLP, and class\-evidence heads\. Biases, LayerNorm, attention softmax, residual additions, recursive fusion, loss computation, Adam states, and matrix accumulation remain in FP32 during training\.

Each quantized training forward pass is paired with a float\-shadow pass using the same weights, target class, dropout state, and random\-number state\. This matched path supports the attribution objective below; it is not used as the final deployment result\.

### 3\.5Exact mixed\-precision ONNX INT8 deployment

After checkpoint selection, the frozen QAT scales and trained weights are converted directly into an ONNX graph\. For every learned projection, activations are clipped to the trained signed range and quantized per tensor to INT8\. Weights are quantized per output channel to signed INT8\. Each projection is evaluated as

y=FP32⁡\(MatMulInteger⁡\(x8,W8⊤\)\)⊙\(sx​sW\)\+b,y=\\operatorname\{FP32\}\\\!\\left\(\\operatorname\{MatMulInteger\}\(x\_\{8\},W\_\{8\}^\{\\top\}\)\\right\)\\odot\(s\_\{x\}s\_\{W\}\)\+b,\(7\)where the matrix product accumulates in INT32,sxs\_\{x\}is the frozen activation scale,sWs\_\{W\}is the vector of per\-output\-channel weight scales, andbbis added in FP32\. Output quantization is applied only where it was present during QAT\. The two evidence\-head outputs remain FP32 after integer accumulation, matching the trained model rather than introducing an untrained output bottleneck\.

Every converted model contains 16MatMulIntegerprojection nodes and 16 signed INT8 weight initializers\. LayerNorm, residual additions, ReLU, attention score and context products, softmax, pooling, dequantization, bias addition, and attribution normalization remain FP32\. The deployment is therefore an actual mixed\-precision ONNX INT8 implementation, not a fully integer\-only network\.

### 3\.6Attribution objective

LetAq\(1\)A\_\{q\}^\{\(1\)\},Aq\(2\)A\_\{q\}^\{\(2\)\}, andAqrecA\_\{q\}^\{\\mathrm\{rec\}\}denote quantized\-training\-path maps, and letAf​precA\_\{fp\}^\{\\mathrm\{rec\}\}denote the matched float\-shadow map\. The attribution objective is

ℒattr=12​JS⁡\(Aq\(1\),Aq\(2\)\)\+12​JS⁡\(Aqrec,Af​prec\)\.\\mathcal\{L\}\_\{\\mathrm\{attr\}\}=\\frac\{1\}\{2\}\\operatorname\{JS\}\\left\(A\_\{q\}^\{\(1\)\},A\_\{q\}^\{\(2\)\}\\right\)\+\\frac\{1\}\{2\}\\operatorname\{JS\}\\left\(A\_\{q\}^\{\\mathrm\{rec\}\},A\_\{fp\}^\{\\mathrm\{rec\}\}\\right\)\.\(8\)The first term encourages agreement between recursive depths\. The second encourages retention under the training\-time quantization model\. The loss\-only control minimizesℒcls\+0\.1​ℒattr\\mathcal\{L\}\_\{\\mathrm\{cls\}\}\+0\.1\\mathcal\{L\}\_\{\\mathrm\{attr\}\}with ordinary Adam\.

### 3\.7Recursive Attribution\-Stabilized Optimization

RASO computes separate gradients

gc=∇θℒcls,ga=∇θℒattr\.g\_\{c\}=\\nabla\_\{\\theta\}\\mathcal\{L\}\_\{\\mathrm\{cls\}\},\\qquad g\_\{a\}=\\nabla\_\{\\theta\}\\mathcal\{L\}\_\{\\mathrm\{attr\}\}\.\(9\)The attribution gradient is first norm\-matched:

u=∥gc∥2∥ga∥2\+ϵ​ga\.u=\\frac\{\\lVert g\_\{c\}\\rVert\_\{2\}\}\{\\lVert g\_\{a\}\\rVert\_\{2\}\+\\epsilon\}g\_\{a\}\.\(10\)When⟨gc,u⟩<0\\langle g\_\{c\},u\\rangle<0, its conflicting component is removed:

u~=u−⟨u,gc⟩∥gc∥22\+ϵ​gc\.\\tilde\{u\}=u\-\\frac\{\\langle u,g\_\{c\}\\rangle\}\{\\lVert g\_\{c\}\\rVert\_\{2\}^\{2\}\+\\epsilon\}g\_\{c\}\.\(11\)Otherwise,u~=u\\tilde\{u\}=u\. The gradient supplied to Adam is

gR​A​S​O=gc\+λa​u~,λa=0\.1\.g\_\{RASO\}=g\_\{c\}\+\\lambda\_\{a\}\\tilde\{u\},\\qquad\\lambda\_\{a\}=0\.1\.\(12\)This asymmetric choice protects the classification direction: the attribution objective can contribute an orthogonal or aligned component, but it cannot introduce a first\-order component directly opposed togcg\_\{c\}\.

Algorithm 1One RASO optimization step1:Batch

\(x,y\)\(x,y\), model parameters

θ\\theta, coefficient

λa\\lambda\_\{a\}
2:Compute quantized and matched float\-shadow forward passes

3:Compute

ℒcls\\mathcal\{L\}\_\{\\mathrm\{cls\}\}and

ℒattr\\mathcal\{L\}\_\{\\mathrm\{attr\}\}from Equation[8](https://arxiv.org/html/2607.15421#S3.E8)

4:

gc←∇θℒclsg\_\{c\}\\leftarrow\\nabla\_\{\\theta\}\\mathcal\{L\}\_\{\\mathrm\{cls\}\},

ga←∇θℒattrg\_\{a\}\\leftarrow\\nabla\_\{\\theta\}\\mathcal\{L\}\_\{\\mathrm\{attr\}\}
5:

u←∥gc∥2​ga/\(∥ga∥2\+ϵ\)u\\leftarrow\\lVert g\_\{c\}\\rVert\_\{2\}g\_\{a\}/\(\\lVert g\_\{a\}\\rVert\_\{2\}\+\\epsilon\)
6:if

⟨gc,u⟩<0\\langle g\_\{c\},u\\rangle<0then

7:Compute

u~\\tilde\{u\}using Equation[11](https://arxiv.org/html/2607.15421#S3.E11)

8:else

9:

u~←u\\tilde\{u\}\\leftarrow u
10:endif

11:Supply

gc\+λa​u~g\_\{c\}\+\\lambda\_\{a\}\\tilde\{u\}to the Adam moment update

## 4Experimental Design

### 4\.1Datasets and low\-data protocol

We use seven 2D MedMNIST datasets\(Yang et al\.,[2023](https://arxiv.org/html/2607.15421#bib.bib4)\): BloodMNIST, PathMNIST, BreastMNIST, PneumoniaMNIST, DermaMNIST, OCTMNIST, and OrganAMNIST\. They span blood\-cell microscopy, colorectal histopathology, breast ultrasound, chest radiography, dermoscopy, retinal optical coherence tomography, and abdominal CT\. BreastMNIST and PneumoniaMNIST are binary; the remaining tasks contain 4 to 11 classes\.

For every dataset and seed, exactly 50 images per class are sampled without replacement from the official training split\. Official validation and test sets remain unchanged\. Images are converted to RGB and resized to224×224224\\times 224\. Training uses batch size 16, 23 epochs, Adam with learning rate10−410^\{\-4\},ϵ=10−8\\epsilon=10^\{\-8\}, no weight decay, and no scheduler\. Models are trained from scratch\. The seeds are

\{3,5,7,11,13,17,19,23,29,31\}\.\\\{3,5,7,11,13,17,19,23,29,31\\\}\.\(13\)The original ZACH\-ViT study used the first five seeds\. We extend the controlled protocol to ten seeds\. The PyTorch baseline is parameter\-matched to the published architecture, but it is treated as the common experimental baseline rather than as an assertion of numerical replication of the earlier TensorFlow implementation\.

### 4\.2Controlled training conditions

We train four conditions:

1. 1\.ZACH\-ViT \+ Adam: full\-precision baseline without intrinsic attribution\.
2. 2\.qZACH\-ViT \+ Adam: W8A8 quantization\-aware training and intrinsic evidence, classification loss only\.
3. 3\.qZACH\-ViT \+ Adam \+ℒattr\\mathcal\{L\}\_\{\\mathrm\{attr\}\}: the same architecture and attribution objective as the full method, optimized by scalar loss addition\.
4. 4\.qZACH\-ViT \+ RASO: the full architecture and the gradient procedure in Equations[11](https://arxiv.org/html/2607.15421#S3.E11)and[12](https://arxiv.org/html/2607.15421#S3.E12)\.

For each dataset\-seed pair, the three qZACH\-ViT conditions start from the same saved parameter state\. Sampling manifests, initial\-state hashes, checkpoints, and histories are retained\. The best validation checkpoint according to the inherited primary metric is selected\. All 280 expected training runs completed with no failure file\.

### 4\.3Actual INT8 conversion and deployment audit

Each of the 210 selected qZACH\-ViT checkpoints is converted independently using Equation[7](https://arxiv.org/html/2607.15421#S3.E7)\. Conversion covers seven datasets, ten seeds, and three training conditions\. The official test split is then evaluated in full, yielding 964,920 matched source\-to\-INT8 predictions\. Intrinsic deployment retention is evaluated on the original 3,600 selected XAI examples\.

For every graph, we countMatMulInteger,QuantizeLinear,DequantizeLinear, and signed INT8 weight initializers\. ONNX Runtime profiling must record integer matrix\-multiplication execution\. We store source and INT8 logits, probabilities, predictions, intrinsic maps, intermediate\-layer parity, serialized size, and checksums\. Prediction retention is summarized by exact agreement and change counts\. Primary\-metric retention uses Macro\-F1 for multiclass datasets and AUC@0\.5 for binary datasets\. We additionally inspect whether a changed prediction has a source decision margin larger than twice its corresponding maximum logit perturbation; this is reported only as a numerical audit, not as a standard robustness metric\.

Batch\-size\-1 CPU latency is measured after ten warm\-up iterations over 50 timed iterations, with one and four threads\. The source path is PyTorch 2\.3\.1 eager fake\-QAT execution; the deployed path is ONNX Runtime 1\.23\.2 withCPUExecutionProvider\. Consequently, the latency ratio is an end\-to\-end deployment comparison that includes both quantization and runtime\-stack differences, not a pure isolated\-kernel speedup\. The run used Python 3\.10\.16, ONNX 1\.22\.0, MedMNIST 3\.0\.2, and a 32\-logical\-CPU x86\_64 Linux host\. The exact processor model was not recorded by the audit script\.

### 4\.4Predictive metrics and statistical analysis

Macro\-F1 is primary for multiclass datasets\. For continuity with the predecessor protocol, binary primary performance is the ROC\-AUC of predictions thresholded at 0\.5, denoted AUC@0\.5\. Because this is not conventional probability AUROC, probability AUROC, accuracy, and Macro\-F1 are also retained and reported in Appendix[B\.1](https://arxiv.org/html/2607.15421#A2.SS1)\. The main qZACH\-ViT predictive table and all paired contrasts use the actual ONNX INT8 outputs\. The ZACH\-ViT baseline remains FP32\.

Results are mean±\\pmstandard deviation across ten seeds\. Dataset\-level mean ranks are compared with a Friedman test across the seven datasets\. Within each dataset, paired two\-sided Wilcoxon signed\-rank tests compare conditions over ten seeds, with Holm correction across seven datasets for each planned contrast\. The 70 dataset\-seed differences are also reported descriptively through their mean and number of positive pairs\.

### 4\.5XAI baselines and compatibility audit

The full same\-model comparison explains the source\-model predictions of each trained qZACH\-ViT \+ RASO checkpoint with:

- •the intrinsic recursive map;
- •Integrated Gradients with 32 integration steps;
- •token\-adapted Grad\-CAM;
- •Attention Rollout\-ZT;
- •Gradient Attention Rollout\-ZT;
- •RISE with 128 masks;
- •deterministic patch occlusion;
- •random attribution as a negative control\.

The gradient\-based post\-hoc comparison is performed on the differentiable PyTorch source model\. The actual ONNX INT8 deployment is evaluated separately for intrinsic\-map retention because the integer graph is not used as a differentiable post\-hoc explanation backend\. ZACH\-ViT has no classification token\. The two rollout methods therefore average final\-query contribution in a manner consistent with global average pooling rather than extracting a nonexistent classification\-token row\. Exact transformer LRP is excluded because no relevance\-conservation rule was derived for the custom quantized attention, adaptive residual projections, recursive evidence fusion, and zero\-token aggregation\.

The benchmark selects two examples per class where possible, with 16 to 32 examples per dataset\-seed checkpoint\. This yields 1,200 matched explanations per method for the qZACH\-ViT \+ RASO same\-model comparison, 1,200 intrinsic explanations for each of the three qZACH\-ViT conditions, and 1,200 explanations per post\-hoc method on the ZACH\-ViT baseline\. In total, 21,600 source\-model XAI rows are evaluated, plus 168 parameter\-randomization rows and 3,600 source\-to\-INT8 intrinsic\-map pairs\.

### 4\.6Faithfulness, stability, and sanity metrics

Faithfulness is assessed by deletion AUC, insertion AUC, comprehensiveness after removing the top 20% patches, sufficiency error when retaining those patches, SaCo with ten salience groups, and rank correlation with deterministic patch occlusion\. Stability under small input noise uses Gaussian noise with standard deviation 0\.01 and reports cosine similarity, rank correlation, top\-10% overlap, and JS divergence\. Runtime per image, approximate forward passes, and approximate backward passes are recorded\.

The official MedMNIST NPZ archives were inspected directly\. They contain split images and classification labels but no masks, bounding boxes, landmarks, or localization targets\. We therefore do not compute Dice, intersection\-over\-union, or pointing\-game metrics\. Parameter\-randomization checks compare original and fully randomized models on four examples per dataset\.

For intrinsic\-XAI comparisons, each metric is first averaged over matched examples within a dataset\-seed checkpoint, yielding 70 paired units per condition\. RASO is compared with the attribution\-loss control by two\-sided Wilcoxon tests, with Holm correction across ten XAI metrics\. We emphasize effect direction and multiplicity\-corrected results rather than isolated uncorrectedpp\-values\.

## 5Results

### 5\.1Actual INT8 predictive performance

Table[1](https://arxiv.org/html/2607.15421#S5.T1)reports the FP32 ZACH\-ViT baseline and the actual converted ONNX INT8 outputs for all three qZACH\-ViT conditions\. Every deployed qZACH\-ViT condition exceeds the ZACH\-ViT mean on every dataset\. Relative to the baseline across 70 dataset\-seed pairs, actual INT8 qZACH\-ViT \+ Adam has a mean gain of 0\.0313 and improves 53 pairs\. The attribution\-loss control has a mean gain of 0\.0309 and improves 52 pairs\. Actual INT8 qZACH\-ViT \+ RASO has the largest mean gain, 0\.0368, and improves 57 pairs\.

Table 1:Test performance under the 50\-images\-per\-class protocol\. The ZACH\-ViT baseline is FP32; all qZACH\-ViT values are from the actual converted ONNX INT8 graphs\. Values are mean±\\pmstandard deviation over ten seeds\. Macro\-F1 is used for multiclass datasets and AUC@0\.5 for binary datasets\. The best mean in each row is bold\.DatasetMetricZACH\-ViT \+ AdamqZACH\-ViT \+ AdamqZACH\-ViT \+ Adam \+ℒattr\\mathcal\{L\}\_\{\\mathrm\{attr\}\}qZACH\-ViT \+ RASOBloodMNISTMacro\-F10\.675±\\pm0\.0390\.723±\\pm0\.0260\.722±\\pm0\.0180\.733±\\pm0\.024PathMNISTMacro\-F10\.504±\\pm0\.0590\.571±\\pm0\.0260\.577±\\pm0\.0290\.564±\\pm0\.031BreastMNISTAUC@0\.50\.584±\\pm0\.0450\.619±\\pm0\.0510\.613±\\pm0\.0530\.620±\\pm0\.045PneumoniaMNISTAUC@0\.50\.738±\\pm0\.0130\.746±\\pm0\.0160\.748±\\pm0\.0180\.745±\\pm0\.021DermaMNISTMacro\-F10\.340±\\pm0\.0210\.354±\\pm0\.0210\.348±\\pm0\.0230\.345±\\pm0\.015OCTMNISTMacro\-F10\.238±\\pm0\.0250\.262±\\pm0\.0340\.263±\\pm0\.0350\.274±\\pm0\.026OrganAMNISTMacro\-F10\.463±\\pm0\.0220\.487±\\pm0\.0350\.490±\\pm0\.0310\.520±\\pm0\.030

The best deployed qZACH\-ViT condition remains task dependent\. RASO has the best mean on BloodMNIST, BreastMNIST, OCTMNIST, and OrganAMNIST; the attribution\-loss control is best on PathMNIST and PneumoniaMNIST; and Adam is best on DermaMNIST\. Mean ranks are 1\.86 for RASO, 2\.00 for the attribution\-loss control, 2\.14 for Adam, and 4\.00 for ZACH\-ViT\. The Friedman test rejects equal ranks \(χF2=12\.77\\chi^\{2\}\_\{F\}=12\.77,p=0\.0052p=0\.0052\), but the small and heterogeneous differences among the three deployed qZACH\-ViT conditions do not support a universal optimizer ranking\.

Across the seven dataset means, each deployed qZACH\-ViT condition is higher than ZACH\-ViT on all seven tasks, giving a two\-sided signed\-rank value ofp=0\.0156p=0\.0156for each comparison\. After per\-contrast Holm correction across datasets, actual INT8 qZACH\-ViT \+ Adam is significant on BloodMNIST and PathMNIST\. Actual INT8 qZACH\-ViT \+ RASO is significant on BloodMNIST, PathMNIST, OCTMNIST, and OrganAMNIST\. Full paired results are given in Appendix[B\.2](https://arxiv.org/html/2607.15421#A2.SS2)\.

![Refer to caption](https://arxiv.org/html/2607.15421v1/x1.png)Figure 1:Mean paired change of the actual ONNX INT8 qZACH\-ViT models relative to the FP32 ZACH\-ViT baseline\. Error bars are 95% confidence intervals across the ten seed\-wise differences within each dataset\.The architectural result therefore survives real conversion: quantization\-aware intrinsic patch evidence does not impose an observed predictive penalty in this protocol, and every deployed qZACH\-ViT dataset mean remains above the FP32 baseline\. RASO adds an average 0\.0059 over the deployed loss\-only control across 70 pairs and is positive in 42 pairs, but this additional predictive effect is heterogeneous\.

### 5\.2Exact INT8 deployment fidelity

All 210 selected qZACH\-ViT checkpoints were converted successfully\. Every ONNX graph contains 16 signed INT8MatMulIntegerprojections and 16 signed INT8 weight initializers, and every ONNX Runtime profile records integer matrix\-multiplication execution\. Across 964,920 matched test predictions, 240 labels change after conversion, corresponding to 99\.9751% exact agreement\. The mean absolute change in the inherited primary metric is 0\.000133 across checkpoints, the maximum is 0\.004386, and 126 of 210 checkpoints have an exactly unchanged inherited primary metric\.

Table 2:Actual ONNX INT8 retention by training condition\. Prediction counts aggregate all official test images\. Primary\-metric changes are absolute checkpoint\-level changes relative to the corresponding source QAT model\. Intrinsic\-XAI values aggregate the matched deployment maps\.ConditionChanged / totalAgreementMean\|Δ\|\|\\Delta\|Max\|Δ\|\|\\Delta\|Map cosineRankρ\\rhoTop\-10% overlapqZACH\-ViT \+ Adam95/321,64099\.9705%0\.0001240\.0043860\.9999480\.9938510\.9691qZACH\-ViT \+ Adam \+ℒattr\\mathcal\{L\}\_\{\\mathrm\{attr\}\}89/321,64099\.9723%0\.0001660\.0043860\.9999480\.9946550\.9701qZACH\-ViT \+ RASO56/321,64099\.9826%0\.0001100\.0012820\.9999680\.9945860\.9684

The smallest number of changed predictions occurs for RASO: 56 of 321,640, compared with 95 for Adam and 89 for the loss\-only control\. No changed prediction has a source decision margin larger than twice its corresponding maximum logit perturbation\. This margin diagnostic is not a general robustness guarantee, but it shows that all 240 label changes occur near a source decision boundary relative to the observed conversion error\. Across all samples, the median per\-sample maximum logit error is 0\.000203, the 99th percentile is 0\.0145, and the maximum is 0\.1585; the isolated maximum does not translate into a large primary\-metric change\.

The deployment also preserves intrinsic evidence\. Across 3,600 source\-to\-INT8 map pairs, mean recursive cosine is 0\.999955, mean JS divergence is1\.13×10−51\.13\\times 10^\{\-5\}, mean Spearman patch\-rank correlation is 0\.9944, and mean top\-10% overlap is 0\.9692\. The worst individual recursive cosine is 0\.997916\. RASO has the highest mean recursive cosine, 0\.999969, and the lowest mean recursive JS divergence among the three conditions, but it does not have the highest top\-patch overlap\. These results support strong deployment retention without claiming bitwise identity\.

![Refer to caption](https://arxiv.org/html/2607.15421v1/x2.png)Figure 2:Changed predictions per 10,000 source\-to\-INT8 comparisons\. Values use all official test images for every dataset, condition, and seed\.
### 5\.3Serialized size and CPU latency

The mean serialized source checkpoint is 0\.988 MiB and the mean exact INT8 ONNX artifact is 0\.296 MiB, a 70\.03% reduction\. This is measured file size, including graph and container overhead, rather than theoretical parameter accounting\. Dataset\-specific values are reported in Appendix[A\.2](https://arxiv.org/html/2607.15421#A1.SS2)\.

Table 3:Batch\-size\-1 CPU latency\. Values are the mean±\\pmstandard deviation of checkpoint\-level median latency over 210 models\. Speedup is computed per checkpoint and then averaged\.ThreadsSource PyTorch QAT \(ms\)Actual INT8 ONNX \(ms\)Mean speedupCheckpoint range16\.853±\\pm0\.0744\.876±\\pm0\.0381\.41×\\times1\.38 to 1\.50×\\times45\.870±\\pm0\.1312\.456±\\pm0\.0832\.39×\\times2\.05 to 2\.57×\\timesAt one thread, median latency averaged across checkpoints decreases from 6\.853 ms to 4\.876 ms, a mean 1\.41 times speedup\. At four threads, it decreases from 5\.870 ms to 2\.456 ms, a mean 2\.39 times speedup\. These are end\-to\-end comparisons between PyTorch eager fake\-QAT execution and ONNX Runtime exact INT8 execution\. They should not be interpreted as an isolated quantization\-kernel ablation\.

![Refer to caption](https://arxiv.org/html/2607.15421v1/x3.png)Figure 3:Serialized size and CPU latency normalized independently to the corresponding source value\. The actual INT8 ONNX artifact is 70\.0% smaller, and end\-to\-end inference is 1\.41 times and 2\.39 times faster with one and four threads, respectively\.
### 5\.4What RASO changes in intrinsic XAI

Table[4](https://arxiv.org/html/2607.15421#S5.T4)compares the intrinsic explanation across the three differentiable source qZACH\-ViT conditions\. RASO has the lowest deletion AUC and sufficiency error, and the highest noise cosine\. It also has the lowest noise JS divergence\. It is not best on insertion AUC, comprehensiveness, SaCo, occlusion correlation, or every top\-patch stability measure\.

Table 4:Source\-model intrinsic\-XAI comparison\. Each value averages 1,200 matched explanations\. Del\. is deletion AUC, Ins\. is insertion AUC, Comp\. is comprehensiveness, Suff\. is sufficiency error, Occ\.ρ\\rhois correlation with patch occlusion, and Noise JS is divergence under input noise\.ConditionDel\.↓\\downarrowIns\.↑\\uparrowComp\.↑\\uparrowSuff\.↓\\downarrowSaCo↑\\uparrowOcc\.ρ\\rho↑\\uparrowNoise cos\.↑\\uparrowNoise JS↓\\downarrowqZACH\-ViT \+ Adam0\.3520\.3750\.1900\.3390\.2050\.1160\.9910\.003qZACH\-ViT \+ Adam \+ℒattr\\mathcal\{L\}\_\{\\mathrm\{attr\}\}0\.3440\.3670\.1860\.3350\.2170\.0880\.9930\.002qZACH\-ViT \+ RASO0\.3390\.3640\.1750\.3040\.2070\.1080\.9950\.002

After aggregating examples within each dataset\-seed checkpoint and applying Holm correction across ten metrics, RASO improves three measures significantly relative to Adam with the same attribution loss: sufficiency error decreases by 0\.0300 \(pHolm=0\.0111p\_\{\\mathrm\{Holm\}\}=0\.0111\), noise cosine increases by 0\.00177 \(pHolm<0\.0001p\_\{\\mathrm\{Holm\}\}<0\.0001\), and noise JS decreases by 0\.00055 \(pHolm<0\.0001p\_\{\\mathrm\{Holm\}\}<0\.0001\)\. The other seven metrics do not survive correction\. This supports a targeted stability and sufficiency claim, not broad superiority across all notions of faithfulness\.

![Refer to caption](https://arxiv.org/html/2607.15421v1/x4.png)Figure 4:Selected source\-model intrinsic\-XAI metrics normalized within each metric for visualization\. Raw values and statistical tests are reported in Table[4](https://arxiv.org/html/2607.15421#S5.T4)and Appendix[C\.2](https://arxiv.org/html/2607.15421#A3.SS2)\.The projection mechanism is active rather than dormant\. Classification and attribution gradients conflict in 45\.9% of training batches on average\. Dataset means range from 37\.7% on OCTMNIST to 53\.4% on OrganAMNIST\.

![Refer to caption](https://arxiv.org/html/2607.15421v1/x5.png)Figure 5:Mean fraction of RASO training batches with a negative classification\-attribution gradient inner product\. Error bars show variation across epochs and seeds\.
### 5\.5Same\-model comparison with post\-hoc methods

Table[5](https://arxiv.org/html/2607.15421#S5.T5)compares explanations for exactly the same source qZACH\-ViT \+ RASO predictions\. Gradient Attention Rollout\-ZT is the strongest non\-oracle method on deletion, insertion, SaCo, and correlation with patch occlusion\. The intrinsic method is therefore not presented as uniformly state of the art in post\-hoc faithfulness\. It is competitive with Token Grad\-CAM on deletion and occlusion correlation, exceeds RISE on deletion and SaCo, and exceeds random attribution on deletion, comprehensiveness, sufficiency, SaCo, and occlusion correlation\. Random attribution is slightly higher on insertion AUC, illustrating why no single perturbation metric is sufficient\. The actual INT8 retention results above show that the intrinsic maps evaluated here are preserved after deployment conversion\.

Table 5:Same\-model source\-XAI comparison on qZACH\-ViT \+ RASO\. Each row summarizes 1,200 matched explanations\. Patch occlusion is both an explainer and the reference for Occ\.ρ\\rho, so its value of 1 is tautological\.MethodDel\.↓\\downarrowIns\.↑\\uparrowSaCo↑\\uparrowOcc\.ρ\\rho↑\\uparrowNoise cos\.↑\\uparrowms/image↓\\downarrowqZACH intrinsic0\.3390\.3640\.2070\.1080\.9959\.9Integrated Gradients0\.3100\.3880\.2200\.0360\.990623\.4Token Grad\-CAM0\.3410\.3640\.1880\.1090\.98921\.1Attention Rollout\-ZT0\.3020\.3840\.3040\.0470\.99411\.3Gradient Attention Rollout\-ZT0\.2790\.4230\.4700\.1961\.00022\.4RISE0\.3560\.3850\.0890\.0111\.00045\.6Patch Occlusion0\.3020\.4090\.6021\.0000\.90881\.4Random Attribution0\.3760\.3760\.0050\.0011\.0000\.4The intrinsic map requires one forward pass and no backward pass, averaging 9\.9 ms per image in the matched source\-model XAI implementation\. Integrated Gradients averages 623\.4 ms, Token Grad\-CAM 21\.1 ms, Gradient Attention Rollout\-ZT 22\.4 ms, RISE 45\.6 ms, and patch occlusion 81\.4 ms\. Attention Rollout\-ZT is the closest at 11\.3 ms\. These XAI runtimes are separate from the logits\-only deployment latency in Table[3](https://arxiv.org/html/2607.15421#S5.T3)\.

![Refer to caption](https://arxiv.org/html/2607.15421v1/x6.png)Figure 6:SaCo versus explanation runtime for the same source qZACH\-ViT \+ RASO predictions\. The intrinsic explanation lies on the fast end of the tradeoff but is not the most faithful by SaCo\.
### 5\.6Parameter\-randomization sanity checks

A faithful explainer should depend on learned parameters\. Integrated Gradients changes most strongly after full parameter randomization, with cosine 0\.628 and JS 0\.089\. The intrinsic map retains a high raw cosine of 0\.963, but its patch\-rank correlation becomes negative and its top\-10% overlap falls to 0\.109, close to the 0\.10 chance level\. This combination shows why cosine alone is unreliable for smooth nonnegative maps\. Gradient Attention Rollout\-ZT and RISE have near\-unit cosine and extremely small JS after randomization, although their rank and top\-patch overlap also deteriorate\. We therefore interpret their strong perturbation metrics cautiously\. Full values are in Appendix[C\.3](https://arxiv.org/html/2607.15421#A3.SS3)\.

### 5\.7Qualitative behavior

Figure[7](https://arxiv.org/html/2607.15421#S5.F7)shows a matched OCTMNIST example\. The explainers differ markedly despite sharing the same prediction\. Gradient and occlusion methods emphasize a narrow lower region, while the intrinsic and token Grad\-CAM maps are more diffuse\. Figure[8](https://arxiv.org/html/2607.15421#S5.F8)compares the three intrinsic conditions\. RASO produces a smoother recursive allocation in this example, but the prediction is incorrect and no localization mask exists\. The figure must therefore be read as a visualization of model evidence, not proof of medical correctness\.

![Refer to caption](https://arxiv.org/html/2607.15421v1/figures/qualitative_oct_matched.png)Figure 7:Matched XAI methods for one OCTMNIST test image and one source qZACH\-ViT \+ RASO checkpoint\. The original and overlay are shown for each method\. The model predicts class 3 while the true class is 2\.![Refer to caption](https://arxiv.org/html/2607.15421v1/figures/qualitative_oct_three_q.png)Figure 8:Intrinsic maps from the three source qZACH\-ViT conditions for the same OCTMNIST image\. The visual difference is descriptive; no anatomical localization target is available\.

## 6Discussion

The results support three separate conclusions\.

First, qZACH\-ViT is a viable architectural extension of the compact backbone, and this conclusion survives actual deployment\. The classification\-only qZACH\-ViT condition improves 53 of 70 paired runs over the FP32 baseline before an attribution objective is introduced\. After conversion, every qZACH\-ViT dataset mean remains above the ZACH\-ViT mean, and the mean absolute checkpoint\-level primary\-metric change from source QAT to actual INT8 is only 0\.000133\. Intrinsic explanation is therefore not purchased at the cost of an observed predictive collapse in this protocol\.

Second, the deployment evidence is stronger than a fake\-quantization result\. All 210 ONNX models contain and execute 16 signed INT8 projection operators\. Across nearly one million predictions, exact agreement is 99\.9751%, while 3,600 intrinsic maps retain mean cosine 0\.999955, mean rank correlation 0\.9944, and mean top\-10% overlap 0\.9692\. The actual artifacts are 70\.0% smaller than the serialized source checkpoints and faster in the measured end\-to\-end CPU setting\. These findings show that both decisions and intrinsic evidence survive conversion\. They do not establish fully integer\-only execution because several non\-projection operations remain FP32, and the latency comparison includes a change of runtime stack\.

Third, RASO changes explanation behavior in a narrower and more defensible way than a universal optimizer claim would suggest\. It obtains the largest mean predictive gain over the baseline, the fewest source\-to\-INT8 changed predictions, and the highest mean deployment\-map cosine\. In the source\-model XAI benchmark, it significantly improves sufficiency error and input\-noise stability over Adam with the identical attribution loss\. However, it is not the best condition on every dataset, does not lead every faithfulness metric, and has slightly lower mean top\-10% deployment overlap than the other two qZACH\-ViT conditions\. Its contribution is best described as stabilizing selected properties of recursive intrinsic evidence while protecting classification\.

The comparison with post\-hoc methods further clarifies the tradeoff\. The intrinsic map is available without a backward pass or repeated perturbation, making it substantially faster than Integrated Gradients, RISE, and occlusion in the matched source implementation\. Gradient Attention Rollout\-ZT nevertheless yields stronger average deletion, insertion, SaCo, and occlusion\-correlation values\. Intrinsic availability, deployability, and computational efficiency therefore do not automatically imply maximal perturbation faithfulness\.

The architecture also exposes a conceptual advantage\. Raw patch evidence is not an auxiliary saliency head trained only to imitate a post\-hoc map\. It is the quantity whose spatial mean produces the stage logit\. This exact completeness is stronger than claiming that an attention matrix explains a prediction\. The normalized heatmap remains a transformation of that evidence, and its clinical relevance is unverified, but the evidence path is part of the predictor and remains numerically stable after INT8 conversion\.

RASO should be understood in relation to gradient\-surgery methods\. Its projection resembles the normal\-plane operation used in PCGrad, while norm matching resembles the motivation of GradNorm\. The novelty is the asymmetric combination and its use for a recursive quantization\-consistency objective with a classification\-protected direction\. Direct comparisons with PCGrad, CAGrad, GradNorm, and MGDA remain important for a later conference version\. The present loss\-only control establishes that the gradient procedure can add value beyond scalar regularization, but it is not a complete optimizer benchmark\.

Finally, this study demonstrates the value of multiple XAI criteria and multiple quantization checks\. Smooth maps can retain high cosine after model randomization, and a single maximum numerical error can look large without changing decisions or task metrics\. The analysis therefore combines graph inspection, runtime profiling, full\-test prediction agreement, primary\-metric retention, rank\-based map retention, perturbation tests, stability, and randomization rather than relying on one favorable scalar\.

## 7Limitations

1. 1\.Mixed precision rather than fully integer\-only inference\.All 16 learned projections use signed INT8MatMulIntegerwith INT32 accumulation, but LayerNorm, residual additions, ReLU, attention score and context products, softmax, pooling, dequantization, bias addition, and attribution normalization remain FP32\.
2. 2\.Runtime\-stack confounding\.The latency comparison is between PyTorch eager fake\-QAT execution and ONNX Runtime actual INT8 execution\. It is a valid end\-to\-end deployment comparison for the tested software stack, but it does not isolate quantization from graph optimization and runtime implementation\.
3. 3\.Incomplete hardware metadata\.The audit records an x86\_64 Linux host with 32 logical CPUs and all software versions, but the exact processor model was not captured\. Latency values should therefore not be treated as universal hardware benchmarks\.
4. 4\.No peak\-memory claim\.Serialized artifact size is measured directly, but reliable process\-level peak\-memory reduction was not established and is not claimed\.
5. 5\.No ground\-truth localization\.The seven MedMNIST archives contain classification labels only\. Faithfulness is evaluated through perturbation, sufficiency, stability, deployment retention, and randomization rather than anatomical masks or expert regions\.
6. 6\.Single benchmark family\.The seven tasks cover several modalities, but all are distributed through MedMNIST and share standardized preprocessing\. External blood microscopy, histopathology, dermoscopy, and XAI\-specific datasets are needed to assess transfer\.
7. 7\.Coarse patch resolution\.A14×1414\\times 14grid limits spatial precision\. Upsampled heatmaps may appear more anatomically precise than the underlying patch evidence\.
8. 8\.RASO baselines\.The optimizer\-specific control is Adam with the identical attribution loss\. Direct comparisons with PCGrad, CAGrad, GradNorm, and other multi\-objective methods are deferred\.
9. 9\.Framework port\.The full\-precision baseline is a parameter\-matched PyTorch implementation of the published architecture\. It is not claimed to reproduce every earlier TensorFlow result numerically\. Internal comparisons remain controlled because all four conditions share the same framework, samples, and training code\.
10. 10\.Metric dependence\.Deletion and insertion can introduce out\-of\-distribution inputs\. SaCo, sufficiency, randomization, and the margin\-based conversion audit also have assumptions\. No single score is treated as ground truth\.
11. 11\.No clinical claim\.The heatmaps indicate model evidence, not verified pathology, diagnosis, or causal anatomy\. Expert review and task\-specific localization data are required before clinical interpretation\.

## 8Conclusion

We introduced qZACH\-ViT and RASO as two distinct contributions and validated them beyond quantization simulation\. qZACH\-ViT combines a compact zero\-token Vision Transformer with W8A8 quantization\-aware training and recursive patch\-level evidence that exactly reconstructs the logits before normalization\. The selected models were converted into 210 executable mixed\-precision ONNX INT8 graphs in which all 16 learned projections use signed INT8MatMulIntegerwith INT32 accumulation\. Across seven low\-data MedMNIST tasks and ten seeds, the actual deployed qZACH\-ViT means exceed the FP32 ZACH\-ViT baseline on every dataset\. Across 964,920 source\-to\-INT8 test comparisons, prediction agreement is 99\.9751%, and across 3,600 intrinsic\-map pairs, mean cosine is 0\.999955\. The actual ONNX artifacts are 70\.0% smaller than the serialized source checkpoints and provide 1\.41 times and 2\.39 times end\-to\-end CPU speedups with one and four threads in the measured environment\. RASO yields the largest overall predictive gain, the fewest changed deployment predictions, and targeted improvements in sufficiency and input\-noise stability, but it is not universally best across datasets or XAI metrics\. Strong post\-hoc methods remain superior on several faithfulness criteria\. These balanced results establish qZACH\-ViT as a genuinely deployable compact intrinsically explainable model and RASO as a stability\-oriented optimization procedure\. External datasets, localization\-grounded evaluation, fully integer non\-projection operators, complete hardware profiling, and broader optimizer comparisons remain the principal next steps\.

## Reproducibility Statement

The training archive contains the configuration, method lock, 280 checkpoints, 70 sampling manifests, locked initial states for the three qZACH\-ViT conditions, complete training histories, seed\-level and summary tables, the XAI compatibility audit, 21,600 source\-model XAI rows, 168 parameter\-randomization rows, qualitative panels, and a completion audit\. The deployment archive contains 210 ONNX models, 210 complete prediction archives, 210 intrinsic\-XAI retention files, 210 graph audits, 210 ONNX Runtime execution audits, 1,260 intermediate\-layer parity rows, 840 latency measurements, checksums, and zero failure files\. The manuscript source additionally includes recomputed deployment summaries and a corrected audit derived from the saved prediction and XAI files\. Exact seeds, hyperparameters, quantization rules, operator coverage, software versions, and evaluation settings are stated in the manuscript\.

## Appendix ADetailed Architecture and Hardware\-Deployed INT8

### A\.1Architecture and actual serialized deployment size

All models use 196 patch tokens\. The architecture audit verifies that the recursive raw patch evidence reconstructs the logits with zero numerical error at floating\-point precision for every dataset\-specific output dimension\. Quantized weights include the patch projection, all attention projections, feed\-forward projections, adaptive residual projections, token MLP projections, and evidence\-head weights\.

Table[6](https://arxiv.org/html/2607.15421#A1.T6)reports measured serialized file sizes rather than theoretical parameter accounting\. Source size is the mean selected PyTorch QAT checkpoint size across the three qZACH\-ViT conditions and ten seeds\. ONNX size is the mean actual converted artifact size\. Each ONNX graph contains 16MatMulIntegerprojection nodes\.

Table 6:Architecture and actual serialized deployment size\. Sizes are MiB\. Reduction is relative to the corresponding serialized source checkpoint\.DatasetqZACH paramsINT8 weightsSource checkpointINT8 ONNXReductionInteger projectionsBloodMNIST249,744247,2960\.9900\.29770\.04%16PathMNIST249,938247,4880\.9900\.29770\.04%16BreastMNIST248,386245,9520\.9850\.29570\.02%16PneumoniaMNIST248,386245,9520\.9850\.29570\.03%16DermaMNIST249,550247,1040\.9890\.29670\.03%16OCTMNIST248,968246,5280\.9870\.29670\.02%16OrganAMNIST250,326247,8720\.9920\.29770\.05%16The mean source checkpoint is 0\.988 MiB and the mean ONNX artifact is 0\.296 MiB\. The 70\.03% reduction includes graph and container overhead\. It should not be interpreted as peak resident\-memory reduction\.

### A\.2Detailed deployment fidelity

Table[7](https://arxiv.org/html/2607.15421#A1.T7)gives the actual INT8 retention by dataset and training condition\. Prediction agreement is computed over all official test images and all ten seeds\. The primary\-metric delta is the mean absolute checkpoint\-level change in Macro\-F1 or AUC@0\.5\. XAI values use the matched intrinsic\-map deployment set\.

Table 7:Detailed actual ONNX INT8 retention\. Lower is better for changed predictions and mean\|Δ\|\|\\Delta\|; higher is better for agreement, cosine, rank correlation, and top\-patch overlap\.DatasetTraining conditionChanged / totalAgreementMean\|Δ\|\|\\Delta\|Map cosineRankρ\\rhoTop\-10%BloodMNISTAdam4/34,21099\.9883%0\.0000740\.9999600\.9947510\.9688BloodMNISTAdam \+ℒattr\\mathcal\{L\}\_\{\\mathrm\{attr\}\}4/34,21099\.9883%0\.0000820\.9999570\.9942900\.9675BloodMNISTRASO3/34,21099\.9912%0\.0001010\.9999650\.9946160\.9591PathMNISTAdam14/71,80099\.9805%0\.0000510\.9999490\.9933450\.9744PathMNISTAdam \+ℒattr\\mathcal\{L\}\_\{\\mathrm\{attr\}\}12/71,80099\.9833%0\.0001140\.9999570\.9937660\.9692PathMNISTRASO9/71,80099\.9875%0\.0000520\.9999730\.9951870\.9708BreastMNISTAdam1/1,56099\.9359%0\.0004390\.9999700\.9985580\.9844BreastMNISTAdam \+ℒattr\\mathcal\{L\}\_\{\\mathrm\{attr\}\}1/1,56099\.9359%0\.0004390\.9999750\.9983560\.9844BreastMNISTRASO0/1,560100\.0000%0\.0000000\.9999790\.9993090\.9866PneumoniaMNISTAdam0/6,240100\.0000%0\.0000000\.9999490\.9883540\.9597PneumoniaMNISTAdam \+ℒattr\\mathcal\{L\}\_\{\\mathrm\{attr\}\}2/6,24099\.9679%0\.0000850\.9999580\.9888930\.9591PneumoniaMNISTRASO2/6,24099\.9679%0\.0002560\.9999740\.9888190\.9600DermaMNISTAdam6/20,05099\.9701%0\.0001610\.9999470\.9876020\.9566DermaMNISTAdam \+ℒattr\\mathcal\{L\}\_\{\\mathrm\{attr\}\}5/20,05099\.9751%0\.0000190\.9999620\.9909390\.9594DermaMNISTRASO3/20,05099\.9850%0\.0000880\.9999670\.9877130\.9494OCTMNISTAdam2/10,00099\.9800%0\.0000080\.9999600\.9971430\.9756OCTMNISTAdam \+ℒattr\\mathcal\{L\}\_\{\\mathrm\{attr\}\}8/10,00099\.9200%0\.0002760\.9999610\.9974740\.9797OCTMNISTRASO2/10,00099\.9800%0\.0001800\.9999750\.9970720\.9781OrganAMNISTAdam68/177,78099\.9618%0\.0001350\.9999120\.9963350\.9650OrganAMNISTAdam \+ℒattr\\mathcal\{L\}\_\{\\mathrm\{attr\}\}57/177,78099\.9679%0\.0001420\.9998890\.9978000\.9711OrganAMNISTRASO37/177,78099\.9792%0\.0000930\.9999470\.9980240\.9727

### A\.3Numerical and runtime audits

Across all 964,920 source\-to\-INT8 comparisons, the median per\-sample maximum logit error is 0\.000203, the 95th percentile is 0\.004532, the 99th percentile is 0\.014512, the 99\.9th percentile is 0\.033276, and the maximum is 0\.158520\. The maximum is an extreme value rather than a typical error; no changed prediction has a source margin exceeding twice its corresponding per\-sample maximum logit error\.

Every graph has 16MatMulInteger, 32QuantizeLinear, 32DequantizeLinear, and four FP32 attentionMatMulnodes\. All 210 runtime profiles record integer matrix\-multiplication execution\. The deployment remains mixed precision because the operations enumerated in Section 3 remain FP32\.

## Appendix BAdditional Predictive Results and Statistical Tests

### B\.1Binary secondary metrics

Table[8](https://arxiv.org/html/2607.15421#A2.T8)reports conventional probability AUROC in addition to the inherited thresholded metric\. The ZACH\-ViT rows are FP32; the three qZACH\-ViT rows are actual ONNX INT8 results\.

Table 8:Secondary binary test metrics, mean±\\pmstandard deviation over ten seeds\.DatasetConditionProbability AUROCAUC@0\.5AccuracyBreastMNISTZACH\-ViT \+ Adam0\.637±\\pm0\.0480\.584±\\pm0\.0450\.535±\\pm0\.128BreastMNISTqZACH\-ViT \+ Adam0\.673±\\pm0\.0430\.619±\\pm0\.0510\.549±\\pm0\.082BreastMNISTqZACH\-ViT \+ Adam \+ℒattr\\mathcal\{L\}\_\{\\mathrm\{attr\}\}0\.671±\\pm0\.0440\.613±\\pm0\.0530\.597±\\pm0\.104BreastMNISTqZACH\-ViT \+ RASO0\.674±\\pm0\.0380\.620±\\pm0\.0450\.572±\\pm0\.054PneumoniaMNISTZACH\-ViT \+ Adam0\.837±\\pm0\.0170\.738±\\pm0\.0130\.737±\\pm0\.021PneumoniaMNISTqZACH\-ViT \+ Adam0\.836±\\pm0\.0130\.746±\\pm0\.0160\.749±\\pm0\.027PneumoniaMNISTqZACH\-ViT \+ Adam \+ℒattr\\mathcal\{L\}\_\{\\mathrm\{attr\}\}0\.841±\\pm0\.0170\.748±\\pm0\.0180\.743±\\pm0\.024PneumoniaMNISTqZACH\-ViT \+ RASO0\.835±\\pm0\.0200\.745±\\pm0\.0210\.743±\\pm0\.033
### B\.2Paired predictive tests

Table[9](https://arxiv.org/html/2607.15421#A2.T9)gives the planned contrasts between the actual ONNX INT8 qZACH\-ViT outputs and the FP32 ZACH\-ViT baseline\. Holm correction is performed separately for the seven qZACH\-ViT \+ Adam tests and the seven qZACH\-ViT \+ RASO tests\.

Table 9:Paired Wilcoxon tests across ten seeds\. Positive pairs count seed\-wise improvements of the deployed INT8 model over the FP32 baseline\.DatasetContrastMean deltaPositive pairsRawppHolmppBloodMNISTqZACH \+ Adam\+0\.047810/100\.00200\.0137BloodMNISTqZACH \+ RASO\+0\.057710/100\.00200\.0137PathMNISTqZACH \+ Adam\+0\.067810/100\.00200\.0137PathMNISTqZACH \+ RASO\+0\.06029/100\.00390\.0195BreastMNISTqZACH \+ Adam\+0\.03576/100\.19340\.3926BreastMNISTqZACH \+ RASO\+0\.03608/100\.13090\.3926PneumoniaMNISTqZACH \+ Adam\+0\.00757/100\.04880\.2441PneumoniaMNISTqZACH \+ RASO\+0\.00677/100\.37500\.7500DermaMNISTqZACH \+ Adam\+0\.01386/100\.32230\.3926DermaMNISTqZACH \+ RASO\+0\.00424/100\.84570\.8457OCTMNISTqZACH \+ Adam\+0\.02347/100\.08400\.3359OCTMNISTqZACH \+ RASO\+0\.035610/100\.00200\.0137OrganAMNISTqZACH \+ Adam\+0\.02337/100\.13090\.3926OrganAMNISTqZACH \+ RASO\+0\.05719/100\.00390\.0195The global descriptive seed\-level summaries are: actual INT8 qZACH\-ViT \+ Adam,\+0\.03132\+0\.03132with 53 of 70 positive pairs; actual INT8 qZACH\-ViT \+ Adam \+ℒattr\\mathcal\{L\}\_\{\\mathrm\{attr\}\},\+0\.03089\+0\.03089with 52 positive pairs; and actual INT8 qZACH\-ViT \+ RASO,\+0\.03677\+0\.03677with 57 positive pairs\. These 70 values combine different datasets and are not used as the sole inferential basis\.

## Appendix CXAI Compatibility, Metrics, and Sanity Checks

### C\.1Compatibility audit

Every implemented method returned a finite normalized 196\-patch map on a trained qZACH\-ViT \+ RASO checkpoint\. Token Grad\-CAM treats the final token grid as the feature map\. Attention Rollout\-ZT and Gradient Attention Rollout\-ZT average final\-query relevance because the architecture uses global average pooling and no classification token\. Exact transformer LRP is excluded rather than approximated because no relevance\-conservation derivation was implemented for the custom quantization\-aware training modules and recursive evidence fusion\.

### C\.2Intrinsic\-XAI paired tests

Table[10](https://arxiv.org/html/2607.15421#A3.T10)compares RASO with Adam plus the same attribution loss after averaging examples within each of 70 dataset\-seed checkpoints\. Holm correction covers ten metrics\.

Table 10:Paired intrinsic\-XAI comparison: qZACH\-ViT \+ RASO minus qZACH\-ViT \+ Adam \+ℒattr\\mathcal\{L\}\_\{\\mathrm\{attr\}\}\.MetricBetter directionMean deltaRawppHolmppDeletion AUClower\-0\.004320\.124490\.65528Insertion AUChigher\-0\.003240\.359741\.00000Comprehensivenesshigher\-0\.011810\.093610\.65528Sufficiency errorlower\-0\.029980\.001380\.01106SaCohigher\-0\.007900\.817191\.00000Occlusion rank correlationhigher\+0\.021720\.927721\.00000Noise cosinehigher\+0\.001770\.000000\.00002Noise rank correlationhigher\+0\.008680\.094760\.65528Noise top\-10% overlaphigher\-0\.003060\.514071\.00000Noise JSlower\-0\.000550\.000000\.00000
### C\.3Parameter\-randomization checks

The expected top\-10% overlap for independent rankings is approximately 0\.10\. Low rank correlation and chance\-level overlap therefore provide evidence of parameter dependence even when cosine is inflated by smooth maps\.

Table 11:Similarity between explanations before and after full model\-parameter randomization\. Lower cosine, rank, and overlap and higher JS indicate stronger sensitivity to learned parameters\.MethodnnCosineRankρ\\rhoTop\-10% overlapJSqZACH intrinsic280\.963\-0\.1630\.1090\.0102Integrated Gradients280\.6280\.0860\.2250\.0890Token Grad\-CAM280\.9190\.0280\.1890\.0373Attention Rollout\-ZT280\.8610\.1580\.2590\.0210Gradient Attention Rollout\-ZT281\.000\-0\.1300\.1410\.0000RISE280\.999\-0\.0760\.0960\.0002The high cosine values of several methods should not be interpreted in isolation\. For qZACH\-ViT intrinsic attribution, cosine is 0\.963, while rank correlation is−0\.163\-0\.163and top\-10% overlap is 0\.109\. Gradient Attention Rollout\-ZT and RISE have near\-unit cosine and tiny JS but rank and top\-patch changes closer to chance\. Integrated Gradients is the most clearly parameter\-sensitive under all four measures in this audit\.

## Appendix DAdditional Qualitative Comparisons

The following panels show one fixed seed\-3 example per remaining dataset\. They are not selected as best cases and include errors\. Each panel compares the same qZACH\-ViT \+ RASO prediction across intrinsic and post\-hoc methods\.

![Refer to caption](https://arxiv.org/html/2607.15421v1/figures/qualitative_blood_matched.png)Figure 9:BloodMNIST matched XAI methods\.
![Refer to caption](https://arxiv.org/html/2607.15421v1/figures/qualitative_path_matched.png)Figure 10:PathMNIST matched XAI methods\.
![Refer to caption](https://arxiv.org/html/2607.15421v1/figures/qualitative_breast_matched.png)Figure 11:BreastMNIST matched XAI methods\.
![Refer to caption](https://arxiv.org/html/2607.15421v1/figures/qualitative_pneumonia_matched.png)Figure 12:PneumoniaMNIST matched XAI methods\.
![Refer to caption](https://arxiv.org/html/2607.15421v1/figures/qualitative_derma_matched.png)Figure 13:DermaMNIST matched XAI methods\.
![Refer to caption](https://arxiv.org/html/2607.15421v1/figures/qualitative_organ_matched.png)Figure 14:OrganAMNIST matched XAI methods\.

## References

- Angelakis et al\. \(2025\)A\. Angelakis, A\. Mousa, M\. L\. A\. Heldeweg, L\. A\. Biesheuvel, M\. A\. Haaksma, J\. M\. Smit, P\. R\. Tuinman, and P\. W\. G\. Elbers\.ZACH\-ViT: A Zero\-Token Vision Transformer with ShuffleStrides Data Augmentation for Robust Lung Ultrasound Classification\.*arXiv preprint arXiv:2510\.17650*, 2025\.
- Angelakis \(2026\)A\. Angelakis\.ZACH\-ViT: Regime\-Dependent Inductive Bias in Compact Vision Transformers for Medical Imaging\.*arXiv preprint arXiv:2602\.17929*, 2026\.
- Angelakis and Gomez\-Barrero \(2026\)A\. Angelakis and M\. Gomez\-Barrero\.Extending ZACH\-ViT to Robust Medical Imaging: Corruption and Adversarial Stress Testing in Low\-Data Regimes\.*arXiv preprint arXiv:2604\.06099*, 2026\.
- Yang et al\. \(2023\)J\. Yang, R\. Shi, D\. Wei, Z\. Liu, L\. Zhao, B\. Ke, H\. Pfister, and B\. Ni\.MedMNIST v2: A Large\-Scale Lightweight Benchmark for 2D and 3D Biomedical Image Classification\.*Scientific Data*, 10:41, 2023\.
- Dosovitskiy et al\. \(2021\)A\. Dosovitskiy, L\. Beyer, A\. Kolesnikov, D\. Weissenborn, X\. Zhai, T\. Unterthiner, M\. Dehghani, M\. Minderer, G\. Heigold, S\. Gelly, J\. Uszkoreit, and N\. Houlsby\.An Image Is Worth 16x16 Words: Transformers for Image Recognition at Scale\.In*International Conference on Learning Representations*, 2021\.
- Touvron et al\. \(2021\)H\. Touvron, M\. Cord, M\. Douze, F\. Massa, A\. Sablayrolles, and H\. Jégou\.Training Data\-Efficient Image Transformers and Distillation through Attention\.In*Proceedings of the 38th International Conference on Machine Learning*, pages 10347–10357, 2021\.
- Jacob et al\. \(2018\)B\. Jacob, S\. Kligys, B\. Chen, M\. Zhu, M\. Tang, A\. Howard, H\. Adam, and D\. Kalenichenko\.Quantization and Training of Neural Networks for Efficient Integer\-Arithmetic\-Only Inference\.In*Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition*, pages 2704–2713, 2018\.
- Gholami et al\. \(2022\)A\. Gholami, S\. Kim, Z\. Dong, Z\. Yao, M\. W\. Mahoney, and K\. Keutzer\.A Survey of Quantization Methods for Efficient Neural Network Inference\.In*Low\-Power Computer Vision: Improve the Efficiency of Artificial Intelligence*, pages 291–326\. Chapman and Hall/CRC, 2022\.
- Li et al\. \(2022a\)Z\. Li, T\. Yang, P\. Wang, and J\. Cheng\.Q\-ViT: Fully Differentiable Quantization for Vision Transformer\.*arXiv preprint arXiv:2201\.07703*, 2022a\.
- Yuan et al\. \(2022\)Z\. Yuan, C\. Xue, Y\. Chen, Q\. Wu, and G\. Sun\.PTQ4ViT: Post\-Training Quantization for Vision Transformers with Twin Uniform Quantization\.In*European Conference on Computer Vision*, pages 191–207, 2022\.
- Lin et al\. \(2022\)Y\. Lin, T\. Zhang, P\. Sun, Z\. Li, and S\. Zhou\.FQ\-ViT: Post\-Training Quantization for Fully Quantized Vision Transformer\.In*Proceedings of the Thirty\-First International Joint Conference on Artificial Intelligence*, pages 1173–1179, 2022\.
- Li and Gu \(2023\)Z\. Li and Q\. Gu\.I\-ViT: Integer\-Only Quantization for Efficient Vision Transformer Inference\.In*Proceedings of the IEEE/CVF International Conference on Computer Vision*, pages 17065–17075, 2023\.
- Li et al\. \(2022b\)Y\. Li, S\. Xu, B\. Zhang, X\. Cao, P\. Gao, and G\. Guo\.Q\-ViT: Accurate and Fully Quantized Low\-Bit Vision Transformer\.In*Advances in Neural Information Processing Systems*, volume 35, pages 34451–34463, 2022b\.
- Li et al\. \(2023\)Z\. Li, J\. Xiao, L\. Yang, and Q\. Gu\.RepQ\-ViT: Scale Reparameterization for Post\-Training Quantization of Vision Transformers\.In*Proceedings of the IEEE/CVF International Conference on Computer Vision*, pages 17227–17236, 2023\.
- Sundararajan et al\. \(2017\)M\. Sundararajan, A\. Taly, and Q\. Yan\.Axiomatic Attribution for Deep Networks\.In*Proceedings of the 34th International Conference on Machine Learning*, pages 3319–3328, 2017\.
- Selvaraju et al\. \(2017\)R\. R\. Selvaraju, M\. Cogswell, A\. Das, R\. Vedantam, D\. Parikh, and D\. Batra\.Grad\-CAM: Visual Explanations from Deep Networks via Gradient\-Based Localization\.In*Proceedings of the IEEE International Conference on Computer Vision*, pages 618–626, 2017\.
- Abnar and Zuidema \(2020\)S\. Abnar and W\. Zuidema\.Quantifying Attention Flow in Transformers\.In*Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics*, pages 4190–4197, 2020\.
- Chefer et al\. \(2021\)H\. Chefer, S\. Gur, and L\. Wolf\.Transformer Interpretability Beyond Attention Visualization\.In*Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 782–791, 2021\.
- Petsiuk et al\. \(2018\)V\. Petsiuk, A\. Das, and K\. Saenko\.RISE: Randomized Input Sampling for Explanation of Black\-Box Models\.In*Proceedings of the British Machine Vision Conference*, 2018\.
- Jain and Wallace \(2019\)S\. Jain and B\. C\. Wallace\.Attention Is Not Explanation\.In*Proceedings of NAACL\-HLT*, pages 3543–3556, 2019\.
- Kindermans et al\. \(2019\)P\.\-J\. Kindermans, S\. Hooker, J\. Adebayo, M\. Alber, K\. T\. Schütt, S\. Dähne, D\. Erhan, and B\. Kim\.The \(Un\)reliability of Saliency Methods\.In*Explainable AI: Interpreting, Explaining and Visualizing Deep Learning*, pages 267–280\. Springer, 2019\.
- Adebayo et al\. \(2018\)J\. Adebayo, J\. Gilmer, M\. Muelly, I\. Goodfellow, M\. Hardt, and B\. Kim\.Sanity Checks for Saliency Maps\.In*Advances in Neural Information Processing Systems*, volume 31, 2018\.
- Alvarez\-Melis and Jaakkola \(2018\)D\. Alvarez\-Melis and T\. S\. Jaakkola\.Towards Robust Interpretability with Self\-Explaining Neural Networks\.In*Advances in Neural Information Processing Systems*, volume 31, 2018\.
- Chen et al\. \(2019\)C\. Chen, O\. Li, C\. Tao, A\. J\. Barnett, J\. Su, and C\. Rudin\.This Looks Like That: Deep Learning for Interpretable Image Recognition\.In*Advances in Neural Information Processing Systems*, volume 32, 2019\.
- Koh et al\. \(2020\)P\. W\. Koh, T\. Nguyen, Y\. S\. Tang, S\. Mussmann, E\. Pierson, B\. Kim, and P\. Liang\.Concept Bottleneck Models\.In*Proceedings of the 37th International Conference on Machine Learning*, pages 5338–5348, 2020\.
- Böhle et al\. \(2023a\)M\. Böhle, N\. Singh, M\. Fritz, and B\. Schiele\.B\-cos Alignment for Inherently Interpretable CNNs and Vision Transformers\.*arXiv preprint arXiv:2306\.10898*, 2023a\.
- Böhle et al\. \(2023b\)M\. Böhle, M\. Fritz, and B\. Schiele\.Holistically Explainable Vision Transformers\.*arXiv preprint arXiv:2301\.08669*, 2023b\.
- Yeh et al\. \(2019\)C\.\-K\. Yeh, C\.\-Y\. Hsieh, A\. Suggala, D\. I\. Inouye, and P\. K\. Ravikumar\.On the \(In\)fidelity and Sensitivity of Explanations\.In*Advances in Neural Information Processing Systems*, volume 32, 2019\.
- Hooker et al\. \(2019\)S\. Hooker, D\. Erhan, P\.\-J\. Kindermans, and B\. Kim\.A Benchmark for Interpretability Methods in Deep Neural Networks\.In*Advances in Neural Information Processing Systems*, volume 32, 2019\.
- Hedström et al\. \(2023\)A\. Hedström, L\. Weber, D\. Bareeva, D\. Krakowczyk, F\. Motzkus, W\. Samek, S\. Lapuschkin, and M\. M\.\-C\. Höhne\.Quantus: An Explainable AI Toolkit for Responsible Evaluation of Neural Network Explanations and Beyond\.*Journal of Machine Learning Research*, 24\(34\):1–11, 2023\.
- Ghorbani et al\. \(2019\)A\. Ghorbani, A\. Abid, and J\. Zou\.Interpretation of Neural Networks Is Fragile\.In*Proceedings of the AAAI Conference on Artificial Intelligence*, 33:3681–3688, 2019\.
- Dombrowski et al\. \(2019\)A\.\-K\. Dombrowski, M\. Alber, C\. Anders, M\. Ackermann, K\.\-R\. Müller, and P\. Kessel\.Explanations Can Be Manipulated and Geometry Is to Blame\.In*Advances in Neural Information Processing Systems*, volume 32, 2019\.
- Gomez et al\. \(2022\)T\. Gomez, T\. Fréour, and H\. Mouchère\.Metrics for Saliency Map Evaluation of Deep Learning Explanation Methods\.*arXiv preprint arXiv:2201\.13291*, 2022\.
- Wu et al\. \(2024\)J\. Wu, W\. Kang, H\. Tang, Y\. Hong, and Y\. Yan\.On the Faithfulness of Vision Transformer Explanations\.*arXiv preprint arXiv:2404\.01415*, 2024\.
- Sener and Koltun \(2018\)O\. Sener and V\. Koltun\.Multi\-Task Learning as Multi\-Objective Optimization\.In*Advances in Neural Information Processing Systems*, volume 31, 2018\.
- Chen et al\. \(2018\)Z\. Chen, V\. Badrinarayanan, C\.\-Y\. Lee, and A\. Rabinovich\.GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks\.In*Proceedings of the 35th International Conference on Machine Learning*, pages 794–803, 2018\.
- Yu et al\. \(2020\)T\. Yu, S\. Kumar, A\. Gupta, S\. Levine, K\. Hausman, and C\. Finn\.Gradient Surgery for Multi\-Task Learning\.In*Advances in Neural Information Processing Systems*, volume 33, pages 5824–5836, 2020\.
- Wang et al\. \(2020\)Z\. Wang, Y\. Tsvetkov, S\. Ruder, and G\. Neubig\.Gradient Vaccine: Investigating and Improving Multi\-Task Optimization in Massively Multilingual Models\.In*International Conference on Learning Representations*, 2021\.
- Liu et al\. \(2021\)B\. Liu, X\. Liu, X\. Jin, P\. Stone, and Q\. Liu\.Conflict\-Averse Gradient Descent for Multi\-Task Learning\.In*Advances in Neural Information Processing Systems*, volume 34, pages 18878–18890, 2021\.
- Navon et al\. \(2022\)A\. Navon, A\. Shamsian, I\. Achituve, H\. Maron, K\. Kawaguchi, G\. Chechik, and E\. Fetaya\.Multi\-Task Learning as a Bargaining Game\.In*Proceedings of the 39th International Conference on Machine Learning*, pages 16428–16446, 2022\.