Mitigating Class-Tail Undercoverage in Medical Vision-Language Models under Clinical Shift

# Mitigating Class-Tail Undercoverage in Medical Vision-Language Models under Clinical Shift
Source: [https://arxiv.org/html/2607.28696](https://arxiv.org/html/2607.28696)
Mushir Akhtar M\. Tanveer Department of Mathematics, Indian Institute of Technology Indore, India mushirakhtar\.ml@gmail\.commtanveer@iiti\.ac\.in

###### Abstract

Medical vision\-language models \(VLMs\) can retain high observed marginal coverage after clinical shift while substantially under\-covering an individual disease class\. The affected class varies with acquisition protocol and backbone geometry, so source prevalence does not reliably reveal the failure\. Existing localized and tail\-aware conformal methods respectively adapt to test neighborhoods and source\-frequency tails, leaving held\-out class\-wise coverage failure unmodeled\. We introduce Class\-Tail Adaptive Localized Conformal Deferral \(CALCoDe\), a post\-hoc reliability layer for frozen medical VLMs\. Cross\-fitted validation predictions identify classes at risk of undercoverage, and a disjoint calibration split estimates their class\-conditional tail thresholds\. CALCoDe combines each protected threshold with a localized conformal threshold using a one\-sided maximum\. The resulting set contains every label admitted by the localized rule, with additional protection confined to validation\-identified classes\. An independently calibrated support audit defers cases with insufficient inlier support\. Under exchangeability among accepted examples within each protected class, CALCoDe provides finite\-sample coverage at the prespecified guard level and contains the corresponding localized conformal sets; coverage on shifted external cohorts is evaluated empirically\. Among standard conformal baselines and recent VLM\-specific conformal methods evaluated across two dermatology shifts \(HAM10000→\\rightarrowISIC 2019 and HAM10000→\\rightarrowPAD\-UFES\-20\) and four frozen VLM backbones \(BiomedCLIP, OpenAI CLIP ViT\-B/32, PubMedCLIP ViT\-B/32, and MedSigLIP\-448\), CALCoDe is the only approach whose observed marginal and worst\-class accepted coverage both reach 0\.95 in all eight settings\. On HAM10000→\\rightarrowISIC 2019, its average worst\-class accepted coverage is 0\.970, compared with 0\.926 for sTACP and 0\.864 for LCP\-VLM\.

## 1Introduction and Motivation

Medical image classification increasingly relies on foundation models that align images with clinical text\. Medical vision\-language models \(VLMs\) transfer across label spaces through class names and prompt descriptions, either zero\-shot or through parameter\-efficient adaptation\(Radford and others,[2021](https://arxiv.org/html/2607.28696#bib.bib1); Wanget al\.,[2022](https://arxiv.org/html/2607.28696#bib.bib2); Zhang and others,[2023](https://arxiv.org/html/2607.28696#bib.bib4); Xie and others,[2025](https://arxiv.org/html/2607.28696#bib.bib7); Koleilatet al\.,[2025](https://arxiv.org/html/2607.28696#bib.bib9)\)\. Prompt learning, medical knowledge mining, calibration\-aware adaptation, and dynamic biomedical prompting have strengthened this interface\(Zhouet al\.,[2022](https://arxiv.org/html/2607.28696#bib.bib3); Liu and others,[2025](https://arxiv.org/html/2607.28696#bib.bib8); Basuet al\.,[2025](https://arxiv.org/html/2607.28696#bib.bib11); Koleilatet al\.,[2025](https://arxiv.org/html/2607.28696#bib.bib9); Miao and others,[2026](https://arxiv.org/html/2607.28696#bib.bib22); Shao and others,[2026](https://arxiv.org/html/2607.28696#bib.bib23); Cui and others,[2026](https://arxiv.org/html/2607.28696#bib.bib24)\)\. The resulting classifier depends jointly on visual geometry, textual class representations, prompt wording, and the cohort used for calibration\.

Clinical shift tests every part of that dependency\. Hospitals differ in acquisition protocol, hardware, patient population, disease prevalence, and annotation practice\. Confidence learned on one cohort can remain high after the supporting visual geometry has changed\(Ovadia and others,[2019](https://arxiv.org/html/2607.28696#bib.bib33); Xia and others,[2024](https://arxiv.org/html/2607.28696#bib.bib5); Gutbrodet al\.,[2025](https://arxiv.org/html/2607.28696#bib.bib10); Cheng and others,[2025](https://arxiv.org/html/2607.28696#bib.bib17)\)\. Average accuracy and marginal coverage offer incomplete summaries\. A conformal predictor may cover nearly all patients in aggregate while repeatedly excluding the true label for one rare, visually unstable, or poorly supported disease class\. Such failures are consequential precisely because the cohort\-level metric appears satisfactory\.

Several recent conformal directions bear directly on this problem\. Localized conformal prediction for VLMs \(LCP\-VLM\) adapts calibration to the neighborhood of a test image\(Guan,[2023](https://arxiv.org/html/2607.28696#bib.bib36); Hore and Barber,[2025](https://arxiv.org/html/2607.28696#bib.bib40); Fuchset al\.,[2025](https://arxiv.org/html/2607.28696#bib.bib19)\)\. Conformal prediction with optimal transport \(Conf\-OT\) operates over the calibration and query pools, while Laplacian\-Assisted Transductive Adaptation \(LATA\) refines medical VLM scores on an image graph\(Silva\-Rodriguezet al\.,[2025](https://arxiv.org/html/2607.28696#bib.bib14); Bozorgtabar and others,[2026](https://arxiv.org/html/2607.28696#bib.bib18)\)\. Empirical\-Bayes conformal prediction withrr\-values \(CPr​\-value\\mathrm\{CP\}\_\{r\\text\{\-value\}\}\) uses repeated VLM evidence to model score instability\(Zenget al\.,[2026](https://arxiv.org/html/2607.28696#bib.bib21)\)\. Tail\-Aware Conformal Prediction \(TACP\) and soft Tail\-Aware Conformal Prediction \(sTACP\) rebalance coverage using source\-label frequency\(Liuet al\.,[2026](https://arxiv.org/html/2607.28696#bib.bib20)\)\. These methods address local heterogeneity, set efficiency, prompt variability, or long\-tailed labels\. External clinical shift introduces a different class\-tail problem: the least reliable target class need not be a source\-frequency tail, and its identity can change with the backbone geometry\. A fixed class partition or a marginal local threshold can therefore leave the target\-limiting class exposed\.

To address this failure, we introduce Class\-Tail Adaptive Localized Conformal Deferral \(CALCoDe\), which separates class\-tail discovery from independent protection\. Out\-of\-fold validation predictions identify classes whose localized conformal behavior signals undercoverage risk\. After this set of fragile classes is fixed, a disjoint calibration split supplies their class\-conditional quantiles\. A one\-sided maximum combines each class\-tail threshold with the localized threshold, adding protection for a fragile class while preserving every label admitted by the localized rule\. Trimmed prompt evidence defines an APS nonconformity score, embedding\-space localization adapts its threshold, and a separately calibrated support audit governs deferral when the representation lacks inlier support\.

Figure[1](https://arxiv.org/html/2607.28696#S1.F1)makes the class\-tail failure concrete\. For an accepted ISIC 2019 case with histopathology\-confirmed actinic keratosis, BiomedCLIP assigns its largest probability to melanocytic nevus\. LCP\-VLM returns a two\-label set that excludes the true diagnosis\. The cohort\-level statistic appears satisfactory–accepted coverage is 0\.964–yet coverage for actinic keratosis is only 0\.770\. CALCoDe includes the protected diagnosis in this case and raises its class coverage to 0\.970\. Across HAM10000→\\rightarrowISIC 2019 and HAM10000→\\rightarrowPAD\-UFES\-20, its observed marginal and worst\-class accepted coverage exceed 0\.95 in all eight backbone–dataset settings\. These external\-site values are empirical; the finite\-sample result concerns protected classes exchangeable with the accepted calibration data\.

![Refer to caption](https://arxiv.org/html/2607.28696v1/x1.png)\(a\)Accepted case with a missing true label\.
![Refer to caption](https://arxiv.org/html/2607.28696v1/x2.png)\(b\)Marginal coverage conceals class\-tail undercoverage\.

Figure 1:Marginal accepted coverage masks class\-tail undercoverage for BiomedCLIP on HAM10000→\\rightarrowISIC 2019\. \(a\) LCP\-VLM excludes the histopathology\-confirmed AK label for ISIC\_0057499; CALCoDe includes it\. \(b\) LCP\-VLM reaches 0\.964 marginal accepted coverage but only 0\.770 accepted coverage for AK; CALCoDe raises AK coverage to 0\.970\. Abbreviations: AK \(actinic keratosis\); BKL \(benign keratosis\-like lesion\); MEL \(melanoma\); NV \(melanocytic nevus\)\. Image source: ISIC Archive, ISIC\_0057499, Hospital Clínic de Barcelona; CC BY\-NC\.
## 2Contributions

Our work makes three contributions to the reliability of medical VLM classification under clinical shift\. First, we turn worst\-class undercoverage from a post\-hoc diagnostic into an adaptive calibration target: vulnerable classes are identified from out\-of\-fold coverage behavior rather than source prevalence and protected using independently estimated class\-conditional thresholds\. The resulting one\-sided construction contains the corresponding localized conformal set and yields finite\-sample coverage for protected classes under accepted\-example exchangeability\. Second, CALCoDe defines a leakage\-controlled selective inference protocol with separate data roles for representation\-support estimation, design selection, audit calibration, and conformal calibration\. The protocol combines robust image–text evidence with a calibrated three\-action interface while keeping the medical VLM unchanged\. Third, we evaluate class\-tail reliability across two external dermatology shifts and four frozen VLM backbones against standard and recent conformal methods\. Class\-wise coverage, paired bootstrap intervals, full\-set frequency, and autonomous informative rate distinguish class\-tail protection from coverage achieved through nearly complete prediction sets, thereby identifying regimes in which frozen representation geometry limits clinically useful automation\.

## 3Problem Statement

Let𝒳\\mathcal\{X\}denote the space of medical images and let𝒴=\{1,…,K\}\\mathcal\{Y\}=\\\{1,\\ldots,K\\\}denote a fixed set of diagnostic classes for a target classification task\. A medical vision\-language classifier consists of an image encoderfθ:𝒳→𝒵f\_\{\\theta\}:\\mathcal\{X\}\\rightarrow\\mathcal\{Z\}, a set of text representations for class descriptions or prompts, and a scoring function that assigns class evidence to each label\. For an input imagexx, the classifier produces a vector of class scores or probabilitiesp​\(y∣x\)p\(y\\mid x\)over𝒴\\mathcal\{Y\}\.

The conventional classification problem returns a single label

y^​\(x\)=arg⁡maxy∈𝒴⁡p​\(y∣x\)\.\\hat\{y\}\(x\)=\\arg\\max\_\{y\\in\\mathcal\{Y\}\}p\(y\\mid x\)\.This forced\-decision setting is poorly matched to medical deployment\. If the image is ambiguous, affected by acquisition artifacts, drawn from a shifted domain, or visually consistent with conditions outside the label space, a single top\-1 prediction can be misleading even when the classifier assigns high confidence\. Set\-valued classification with abstention has a natural statistical interpretation in this setting because an empty set can indicate that no class is sufficiently supported by the reference distribution\(Guan and Tibshirani,[2022](https://arxiv.org/html/2607.28696#bib.bib35)\)\. The research problem is to construct a decision function

g​\(x\)∈𝒴∪\{C:C⊆𝒴,\|C\|\>1\}∪\{defer\},g\(x\)\\in\\mathcal\{Y\}\\cup\\\{C:C\\subseteq\\mathcal\{Y\},\|C\|\>1\\\}\\cup\\\{\\mathrm\{defer\}\\\},\(1\)whereg​\(x\)g\(x\)may return a single label, a non\-singleton set of plausible labels, or a deferral decision\. Empty prediction sets are treated as deferrals because they indicate that no candidate label satisfies the calibrated reliability criterion\.

The desired behavior is constrained by three requirements\. First, when the input is consistent with the calibration distribution and the classifier has strong evidence, the system should return a compact decision, ideally a single label\. Second, when multiple labels remain plausible, the system should return a prediction set whose empirical coverage is controlled at a user\-specified risk level\. Third, when the input is sufficiently shifted from the reliable operating regime, the system should defer in place of a falsely precise diagnosis\. The main design challenge is to maintain safety without making the system clinically unusable: excessive deferral reduces utility, while overly small prediction sets can conceal uncertainty\.

We formalize this tradeoff through a selective decision risk\. For non\-deferred examples, letCg​\(x\)⊆𝒴C\_\{g\}\(x\)\\subseteq\\mathcal\{Y\}denote the returned singleton or prediction set; for deferred examples, defineCg​\(x\)=∅C\_\{g\}\(x\)=\\emptyset\. Let the nonnegative weightscerrc\_\{\\mathrm\{err\}\},csetc\_\{\\mathrm\{set\}\}, andcdefc\_\{\\mathrm\{def\}\}encode the deployment costs of an uncovered true label, an overly broad diagnostic set, and a deferral, respectively\. A cost\-sensitive selective\-risk objective is

R​\(g\)=E​\[cerr​Lerr​\(X,Y\)\+cset​Lset​\(X\)\+cdef​Ldef​\(X\)\],R\(g\)=\\mathrm\{E\}\\big\[c\_\{\\mathrm\{err\}\}L\_\{\\mathrm\{err\}\}\(X,Y\)\+c\_\{\\mathrm\{set\}\}L\_\{\\mathrm\{set\}\}\(X\)\+c\_\{\\mathrm\{def\}\}L\_\{\\mathrm\{def\}\}\(X\)\\big\],\(2\)whereLerr=𝟏​\{Y∉Cg​\(X\),g​\(X\)≠defer\}L\_\{\\mathrm\{err\}\}=\\mathbf\{1\}\\\{Y\\notin C\_\{g\}\(X\),g\(X\)\\neq\\mathrm\{defer\}\\\},Lset=\|Cg​\(X\)\|L\_\{\\mathrm\{set\}\}=\|C\_\{g\}\(X\)\|, andLdef=𝟏​\{g​\(X\)=defer\}L\_\{\\mathrm\{def\}\}=\\mathbf\{1\}\\\{g\(X\)=\\mathrm\{defer\}\\\}, subject to a target coverage constraint on accepted in\-regime examples,

Pr⁡\{Y∈Cg​\(X\)∣g​\(X\)≠defer,X∈ℛ\}≥1−δ,\\Pr\\\{Y\\in C\_\{g\}\(X\)\\mid g\(X\)\\neq\\mathrm\{defer\},X\\in\\mathcal\{R\}\\\}\\geq 1\-\\delta,\(3\)whereℛ\\mathcal\{R\}denotes the intended operating region represented by the calibration distribution\. The costs express clinical preference and are specified outside the algorithm\. For example, a screening tool may tolerate larger sets to avoid missed findings, whereas a triage tool may prefer deferral when evidence is weak\.

The difficulty is that medical distribution shift affects both uncertainty estimation and coverage\. Standard confidence scores are often miscalibrated under shift\(Guoet al\.,[2017](https://arxiv.org/html/2607.28696#bib.bib28); Ovadia and others,[2019](https://arxiv.org/html/2607.28696#bib.bib33)\)\. Out\-of\-distribution \(OOD\) scores can detect some distributional deviations but may confuse challenging in\-distribution examples with OOD inputs\(Gutbrodet al\.,[2025](https://arxiv.org/html/2607.28696#bib.bib10)\)\. Conformal prediction provides finite\-sample coverage statements under exchangeability, yet those statements fail to extend automatically to shifted test distributions\(Tibshiraniet al\.,[2019](https://arxiv.org/html/2607.28696#bib.bib34)\)\. Medical VLMs add another source of instability: predictions depend jointly on the image representation, textual prompts, class descriptions, and image\-text alignment margins\(Zhouet al\.,[2022](https://arxiv.org/html/2607.28696#bib.bib3); Koleilatet al\.,[2025](https://arxiv.org/html/2607.28696#bib.bib9); Mahapatraet al\.,[2026](https://arxiv.org/html/2607.28696#bib.bib25)\)\.

We consider a post\-hoc deployment setting with a pretrained or adapted medical VLM and labeled data from its intended operating distribution\. Reference data support embedding\-space comparisons, validation data govern score design, a gate\-calibration split calibrates inlier support, and a conformal\-calibration split constructs prediction sets\. The objective is a decision rule that reduces high\-confidence errors and weak\-class undercoverage under clinically plausible shifts without backbone retraining, hidden training data, test labels, or complete metadata for every source of shift\. Cross\-fitting can implement these roles when labeled data are scarce, provided that score tuning, gate calibration, and prediction\-set calibration retain separate label information\.

## 4Proposed Method

### 4\.1Overview

Class\-Tail Adaptive Localized Conformal Deferral \(CALCoDe\) wraps a frozen medical VLM with a selective set\-valued decision rule\. Its core operation is adaptive class\-tail protection: a localized conformal set remains the default, while classes that fail an out\-of\-fold validation coverage criterion receive independently calibrated class\-conditional thresholds\. Trimmed prompt probabilities supply an Adaptive Prediction Sets \(APS\) nonconformity score, and embedding\-space neighborhoods localize its calibration\. A separate support audit determines whether an input enters prediction\-set construction\.

CALCoDe assigns five distinct roles to the available data\. A reference set𝒟ref\\mathcal\{D\}\_\{\\mathrm\{ref\}\}supplies embeddings for the support audit\. A validation set𝒟val\\mathcal\{D\}\_\{\\mathrm\{val\}\}selects the audit diagnostic and supplies out\-of\-fold class\-tail discovery\. A gate\-calibration set𝒟gate\\mathcal\{D\}\_\{\\mathrm\{gate\}\}, drawn from the intended inlier distribution, calibrates the deferral score after diagnostic selection\. A conformal\-calibration set𝒟cal\\mathcal\{D\}\_\{\\mathrm\{cal\}\}is accessed only after the audit and protected class set are frozen; it calibrates the localized and class\-conditional thresholds\. The test cohort𝒟test\\mathcal\{D\}\_\{\\mathrm\{test\}\}is reserved for evaluation\. This ordering prevents final\-calibration scores from selecting the classes they subsequently protect\.

For an inputxx, CALCoDe first aggregates prompt\-level class evidence and converts it to a label\-wise APS score\. The audit either defers the image or passes it to embedding\-localized calibration\. The class\-tail guard is then applied only to validation\-identified fragile classes, after which the resulting set is returned as a singleton or a non\-singleton prediction set\.

### 4\.2Robust Prompt Evidence

The first stage stabilizes image–text class evidence against prompt wording\. For each classyy, let𝒯y=\{ty,1,…,ty,M\}\\mathcal\{T\}\_\{y\}=\\\{t\_\{y,1\},\\ldots,t\_\{y,M\}\\\}denote clinically meaningful prompt variants\. Letz=fθ​\(x\)z=f\_\{\\theta\}\(x\)be the image embedding andey,me\_\{y,m\}be the text embedding of promptty,mt\_\{y,m\}\. We usesim⁡\(⋅,⋅\)\\operatorname\{sim\}\(\\cdot,\\cdot\)for cosine similarity between normalized image and text embeddings, andT\>0T\>0for the temperature parameter that scales the similarity logits\. For prompt indexmm, the VLM induces

ℓm​\(x,y\)\\displaystyle\\ell\_\{m\}\(x,y\)=sim⁡\(z,ey,m\)T,\\displaystyle=\\frac\{\\operatorname\{sim\}\(z,e\_\{y,m\}\)\}\{T\},\(4\)pm​\(y∣x\)\\displaystyle p\_\{m\}\(y\\mid x\)=exp⁡\(ℓm​\(x,y\)\)∑k=1Kexp⁡\(ℓm​\(x,k\)\)\.\\displaystyle=\\frac\{\\exp\(\\ell\_\{m\}\(x,y\)\)\}\{\\sum\_\{k=1\}^\{K\}\\exp\(\\ell\_\{m\}\(x,k\)\)\}\.Prompt ensembling is useful but fragile: an isolated prompt can distort class evidence\. CALCoDe uses trimmed prompt aggregation\. Letℳκ​\(x,y\)\\mathcal\{M\}\_\{\\kappa\}\(x,y\)be the prompt indices remaining after removing theκ\\kappalargest andκ\\kappasmallest values of\{pm​\(y∣x\)\}m=1M\\\{p\_\{m\}\(y\\mid x\)\\\}\_\{m=1\}^\{M\}\. Define

ay​\(x\)\\displaystyle a\_\{y\}\(x\)=1\|ℳκ​\(x,y\)\|​∑m∈ℳκ​\(x,y\)pm​\(y∣x\),\\displaystyle=\\frac\{1\}\{\|\\mathcal\{M\}\_\{\\kappa\}\(x,y\)\|\}\\sum\_\{m\\in\\mathcal\{M\}\_\{\\kappa\}\(x,y\)\}p\_\{m\}\(y\\mid x\),\(5\)pyrob​\(x\)\\displaystyle p\_\{y\}^\{\\mathrm\{rob\}\}\(x\)=ay​\(x\)∑k=1Kak​\(x\)\+ε\.\\displaystyle=\\frac\{a\_\{y\}\(x\)\}\{\\sum\_\{k=1\}^\{K\}a\_\{k\}\(x\)\+\\varepsilon\}\.Trimmed aggregation preserves the post\-hoc interface and reduces sensitivity to isolated prompt failures\.

### 4\.3APS Nonconformity Score

The trimmed probabilities in Eq\. \([5](https://arxiv.org/html/2607.28696#S4.E5)\) are converted to a rank\-sensitive APS score\(Angelopouloset al\.,[2021](https://arxiv.org/html/2607.28696#bib.bib30)\)\. Letπx​\(1\),…,πx​\(K\)\\pi\_\{x\}\(1\),\\ldots,\\pi\_\{x\}\(K\)order the labels by decreasingpyrob​\(x\)p\_\{y\}^\{\\mathrm\{rob\}\}\(x\), and letrx​\(y\)r\_\{x\}\(y\)be the rank of candidate labelyy\. CALCoDe uses

sAPS​\(x,y\)=∑j=1rx​\(y\)pπx​\(j\)rob​\(x\)\.s\_\{\\mathrm\{APS\}\}\(x,y\)=\\sum\_\{j=1\}^\{r\_\{x\}\(y\)\}p\_\{\\pi\_\{x\}\(j\)\}^\{\\mathrm\{rob\}\}\(x\)\.\(6\)The score is small for labels near the head of the robust prompt distribution and increases with the cumulative mass preceding a candidate label\. Eq\. \([6](https://arxiv.org/html/2607.28696#S4.E6)\) supplies the nonconformity values for localized calibration, out\-of\-fold tail discovery, and the final class\-tail guard\.

### 4\.4Audited Conformal Deferral

The label\-wise score in Eq\. \([6](https://arxiv.org/html/2607.28696#S4.E6)\) governs set membership, whereas deferral requires a sample\-level assessment\. CALCoDe forms candidate unreliability scoresBa​\(x\)B\_\{a\}\(x\), with larger values indicating weaker support\. The base candidates are robust global distance, negative energy, negative maximum robust prompt probability, and prompt\-distribution disagreement\. The global support score is

Dglobrob​\(x\)=median⁡\{d\(1\)​\(z\),…,d\(k\)​\(z\)\},D\_\{\\mathrm\{glob\}\}^\{\\mathrm\{rob\}\}\(x\)=\\operatorname\{median\}\\\{d\_\{\(1\)\}\(z\),\\ldots,d\_\{\(k\)\}\(z\)\\\},whered\(j\)​\(z\)d\_\{\(j\)\}\(z\)is thejthj^\{\\mathrm\{th\}\}smallest distance fromzzto the pooled inlier reference embeddings\. Prompt disagreement is measured by the average divergence between prompt\-specific class distributions and the trimmed aggregate distribution:

Bprompt\(x\)=1M∑m=1MKL\(pm\(⋅∣x\)∥prob\(⋅∣x\)\)\.B\_\{\\mathrm\{prompt\}\}\(x\)=\\frac\{1\}\{M\}\\sum\_\{m=1\}^\{M\}\\mathrm\{KL\}\\\!\\left\(p\_\{m\}\(\\cdot\\mid x\)\\,\\\|\\,p^\{\\mathrm\{rob\}\}\(\\cdot\\mid x\)\\right\)\.HereKL​\(u∥v\)\\mathrm\{KL\}\(u\\,\\\|\\,v\)denotes Kullback–Leibler divergence from distributionuuto distributionvv\.

For each base diagnostic, its empirical gate rank yields a support valuep~a​\(x\)\\widetilde\{p\}\_\{a\}\(x\)\. A fused\-rank candidate,

Bfuse​\(x\)=1−mina⁡p~a​\(x\),B\_\{\\mathrm\{fuse\}\}\(x\)=1\-\\min\_\{a\}\\widetilde\{p\}\_\{a\}\(x\),\(7\)responds when any base diagnostic assigns weak inlier support\. The candidate set for audit selection contains the four base diagnostics andBfuseB\_\{\\mathrm\{fuse\}\}\.

The validation split determines the diagnostic\. Lety^​\(x\)=arg⁡maxy⁡pyrob​\(x\)\\hat\{y\}\(x\)=\\arg\\max\_\{y\}p\_\{y\}^\{\\mathrm\{rob\}\}\(x\)and defineqi=𝟏​\{y^​\(xi\)≠yi\}q\_\{i\}=\\mathbf\{1\}\\\{\\hat\{y\}\(x\_\{i\}\)\\neq y\_\{i\}\\\}for validation examples\. CALCoDe selects

a⋆=arg⁡maxa⁡AUROC​\(Ba​\(xi\),qi\),a^\{\\star\}=\\arg\\max\_\{a\}\\mathrm\{AUROC\}\\big\(B\_\{a\}\(x\_\{i\}\),q\_\{i\}\\big\),\(8\)where AUROC denotes area under the receiver operating characteristic curve\. CALCoDe then fixesBa⋆B\_\{a^\{\\star\}\}before accessing the gate\-calibration split\. For a test imagexx, the audited inlier p\-value is

paudit​\(x\)=1\+∑j=1ng𝟏​\{Ba⋆​\(xj\)≥Ba⋆​\(x\)\}ng\+1,p\_\{\\mathrm\{audit\}\}\(x\)=\\frac\{1\+\\sum\_\{j=1\}^\{n\_\{g\}\}\\mathbf\{1\}\\\{B\_\{a^\{\\star\}\}\(x\_\{j\}\)\\geq B\_\{a^\{\\star\}\}\(x\)\\\}\}\{n\_\{g\}\+1\},\(9\)wherexj∈𝒟gatex\_\{j\}\\in\\mathcal\{D\}\_\{\\mathrm\{gate\}\}\. The acceptance indicator isA​\(x\)=𝟏​\{paudit​\(x\)≥αdef\}A\(x\)=\\mathbf\{1\}\\\{p\_\{\\mathrm\{audit\}\}\(x\)\\geq\\alpha\_\{\\mathrm\{def\}\}\\\}\. For a selected base diagnostic fixed before gate calibration, Supplementary Section S1\.1 provides the usual conformal rank statement under inlier exchangeability\. The fused\-rank candidate in Eq\. \([7](https://arxiv.org/html/2607.28696#S4.E7)\) uses empirical gate ranks and is treated as a calibrated relative\-support diagnostic rather than as a separate distribution\-free test\. Severe localized covariate shift also violates gate–target exchangeability and can loosen false\-alarm control\. Target\-cohort deferral is therefore reported empirically and carries no distribution\-free OOD guarantee\.

### 4\.5Localized Conformal Calibration

Inputs accepted by Eq\. \([9](https://arxiv.org/html/2607.28696#S4.E9)\) proceed to localized prediction\-set calibration\. The accepted calibration subset is

𝒟calA=\{\(xi,yi\)∈𝒟cal:paudit​\(xi\)≥αdef\}\.\\mathcal\{D\}\_\{\\mathrm\{cal\}\}^\{A\}=\\\{\(x\_\{i\},y\_\{i\}\)\\in\\mathcal\{D\}\_\{\\mathrm\{cal\}\}:p\_\{\\mathrm\{audit\}\}\(x\_\{i\}\)\\geq\\alpha\_\{\\mathrm\{def\}\}\\\}\.\(10\)For each accepted calibration example, computeri=sAPS​\(xi,yi\)r\_\{i\}=s\_\{\\mathrm\{APS\}\}\(x\_\{i\},y\_\{i\}\)using Eq\. \([6](https://arxiv.org/html/2607.28696#S4.E6)\)\. A global split\-conformal threshold weights all accepted calibration scores equally, although acquisition and phenotype can change the relevance of those examples to a particular image\. We therefore localize calibration in the VLM embedding space\(Guan,[2023](https://arxiv.org/html/2607.28696#bib.bib36); Hore and Barber,[2025](https://arxiv.org/html/2607.28696#bib.bib40)\)\. For an accepted test imagexx, define

Hh​\(x,xi\)\\displaystyle H\_\{h\}\(x,x\_\{i\}\)=exp⁡\{−d​\(z,zi\)2/h2\},\\displaystyle=\\exp\\\{\-d\(z,z\_\{i\}\)^\{2\}/h^\{2\}\\\},\(11\)wi​\(x\)\\displaystyle w\_\{i\}\(x\)=Hh​\(x,xi\)1\+∑jHh​\(x,xj\)\.\\displaystyle=\\frac\{H\_\{h\}\(x,x\_\{i\}\)\}\{1\+\\sum\_\{j\}H\_\{h\}\(x,x\_\{j\}\)\}\.The remaining mass1/\(1\+∑jHh​\(x,xj\)\)1/\(1\+\\sum\_\{j\}H\_\{h\}\(x,x\_\{j\}\)\)is placed at∞\\infty, yielding the conservative weighted empirical distribution used in localized conformal prediction\. The local quantile levelηδ,h\\eta\_\{\\delta,h\}is chosen by leave\-one\-out calibration on𝒟calA\\mathcal\{D\}\_\{\\mathrm\{cal\}\}^\{A\}so that the accepted calibration scores are covered at target level1−δ1\-\\delta\. The resulting thresholdτδ,h​\(x\)\\tau\_\{\\delta,h\}\(x\)defines

Cδ,hloc​\(x\)=\{y∈𝒴:sAPS​\(x,y\)≤τδ,h​\(x\)\}\.C\_\{\\delta,h\}^\{\\mathrm\{loc\}\}\(x\)=\\\{y\\in\\mathcal\{Y\}:s\_\{\\mathrm\{APS\}\}\(x,y\)\\leq\\tau\_\{\\delta,h\}\(x\)\\\}\.\(12\)When localization is disabled, the construction reduces to ordinary split conformal prediction on accepted examples\. Supplementary Section S1\.2 states the corresponding accepted\-set exchangeability condition\.

### 4\.6Validation\-Discovered Class\-Tail Protection

Localized calibration adapts to the neighborhood of a test image, yet Eq\. \([12](https://arxiv.org/html/2607.28696#S4.E12)\) can cover most accepted examples while leaving a rare or unstable class under\-covered\. CALCoDe identifies such classes before the final calibration split is accessed\. Let

VyA=\{i:\(xi,yi\)∈𝒟val,yi=y,paudit​\(xi\)≥αdef\}\.V\_\{y\}^\{A\}=\\\{i:\(x\_\{i\},y\_\{i\}\)\\in\\mathcal\{D\}\_\{\\mathrm\{val\}\},\\,y\_\{i\}=y,\\,p\_\{\\mathrm\{audit\}\}\(x\_\{i\}\)\\geq\\alpha\_\{\\mathrm\{def\}\}\\\}\.Partition the accepted validation examples intoFFstratified folds\. For foldff, fit a pilot localized ruleCδ,h\(−f\)C\_\{\\delta,h\}^\{\(\-f\)\}using the remainingF−1F\-1folds and evaluate only the held\-out examples\. CALCoDe estimates class\-wise accepted coverage from these out\-of\-fold decisions,

Cov^yval=1\|VyA\|∨1​∑i∈VyA𝟏​\{yi∈Cδ,h\(−f​\(i\)\)​\(xi\)\}\.\\widehat\{\\mathrm\{Cov\}\}\_\{y\}^\{\\mathrm\{val\}\}=\\frac\{1\}\{\|V\_\{y\}^\{A\}\|\\vee 1\}\\sum\_\{i\\in V\_\{y\}^\{A\}\}\\mathbf\{1\}\\\{y\_\{i\}\\in C\_\{\\delta,h\}^\{\(\-f\(i\)\)\}\(x\_\{i\}\)\\\}\.\(13\)Classes whose out\-of\-fold coverage falls below a margin\-adjusted target are marked as tail risks\. Classes with fewer thannminn\_\{\\min\}accepted validation examples are protected by default because the available evidence cannot establish stability:

𝒴tail=\\displaystyle\\mathcal\{Y\}\_\{\\mathrm\{tail\}\}=\{\}\{y:nyval<nmin\}\\displaystyle\\\{y:n\_\{y\}^\{\\mathrm\{val\}\}<n\_\{\\min\}\\\}\(14\)∪\{y:nyval≥nmin,Cov^yval<1−δ−γ\}\.\\displaystyle\\cup\\\{y:n\_\{y\}^\{\\mathrm\{val\}\}\\geq n\_\{\\min\},\\;\\widehat\{\\mathrm\{Cov\}\}\_\{y\}^\{\\mathrm\{val\}\}<1\-\\delta\-\\gamma\\\}\.The set𝒴tail\\mathcal\{Y\}\_\{\\mathrm\{tail\}\}is frozen before𝒟cal\\mathcal\{D\}\_\{\\mathrm\{cal\}\}is used\. For eachy∈𝒴taily\\in\\mathcal\{Y\}\_\{\\mathrm\{tail\}\}, CALCoDe computes the finite\-sample\-corrected class\-global quantile at guard level1−δy1\-\\delta\_\{y\}from accepted calibration examples with labelyy\. A class\-local quantile can enlarge this value when enough class\-yyexamples are available\. Their maximum definesτytail\\tau\_\{y\}^\{\\mathrm\{tail\}\}; an unavailable finite\-sample quantile is represented by\+∞\+\\infty\. The final class\-specific threshold is

τCALCoDe​\(x,y\)=\{max⁡\{τδ,h​\(x\),τytail\},y∈𝒴tail,τδ,h​\(x\),y∉𝒴tail\.\\tau\_\{\\mathrm\{CALCoDe\}\}\(x,y\)=\\begin\{cases\}\\max\\\{\\tau\_\{\\delta,h\}\(x\),\\tau\_\{y\}^\{\\mathrm\{tail\}\}\\\},&y\\in\\mathcal\{Y\}\_\{\\mathrm\{tail\}\},\\\\ \\tau\_\{\\delta,h\}\(x\),&y\\notin\\mathcal\{Y\}\_\{\\mathrm\{tail\}\}\.\\end\{cases\}\(15\)The CALCoDe prediction set is

CCALCoDe​\(x\)=\{y∈𝒴:sAPS​\(x,y\)≤τCALCoDe​\(x,y\)\}\.C\_\{\\mathrm\{CALCoDe\}\}\(x\)=\\\{y\\in\\mathcal\{Y\}:s\_\{\\mathrm\{APS\}\}\(x,y\)\\leq\\tau\_\{\\mathrm\{CALCoDe\}\}\(x,y\)\\\}\.\(16\)The maximum in Eq\. \([15](https://arxiv.org/html/2607.28696#S4.E15)\) is the defining structural choice\. It allocates additional calibration slack to a validation\-identified fragile class while preserving every inclusion made by the localized rule\.

### 4\.7Theoretical Properties

###### Proposition 1\(Set monotonicity\)

For anyxxand protected classy∈𝒴taily\\in\\mathcal\{Y\}\_\{\\mathrm\{tail\}\},

y∈Cδ,hloc​\(x\)⟹y∈CCALCoDe​\(x\)\.y\\in C\_\{\\delta,h\}^\{\\mathrm\{loc\}\}\(x\)\\quad\\Longrightarrow\\quad y\\in C\_\{\\mathrm\{CALCoDe\}\}\(x\)\.\(17\)

The result follows directly fromτCALCoDe​\(x,y\)≥τδ,h​\(x\)\\tau\_\{\\mathrm\{CALCoDe\}\}\(x,y\)\\geq\\tau\_\{\\delta,h\}\(x\)\.

###### Proposition 2\(Protected\-class coverage\)

Fixy∈𝒴taily\\in\\mathcal\{Y\}\_\{\\mathrm\{tail\}\}before accessing𝒟cal\\mathcal\{D\}\_\{\\mathrm\{cal\}\}\. Suppose that the accepted class\-yycalibration scores and an accepted class\-yytest score are exchangeable, and thatτytail\\tau\_\{y\}^\{\\mathrm\{tail\}\}contains the corrected class\-global quantile at level1−δy1\-\\delta\_\{y\}\. Then

Pr⁡\{Y∈CCALCoDe​\(X\)∣Y=y,A​\(X\)=1\}≥1−δy\.\\Pr\\\!\\left\\\{Y\\in C\_\{\\mathrm\{CALCoDe\}\}\(X\)\\mid Y=y,\\,A\(X\)=1\\right\\\}\\geq 1\-\\delta\_\{y\}\.\(18\)

Cross\-fitted discovery and the disjoint final calibration split keep the class\-selection event independent of the order statistic used in Eq\. \([18](https://arxiv.org/html/2607.28696#S4.E18)\)\. Supplementary Sections S1\.3 and S1\.4 provide the discovery argument, both proofs, and the finite\-sample correction\. The experiments set1−δy=0\.951\-\\delta\_\{y\}=0\.95, matching the pooled target; the validation discovery boundary is 0\.90\. Observed target\-site coverage remains empirical because target exchangeability is not assumed\.

### 4\.8Three\-Action Decision Rule

The calibrated audit p\-value and guarded prediction set jointly determine the clinical action\. LetC∗​\(x\)=CCALCoDe​\(x\)C\_\{\*\}\(x\)=C\_\{\\mathrm\{CALCoDe\}\}\(x\)\. The resulting three\-action decision rule is

g​\(x\)=\{defer,paudit​\(x\)<αdef,defer,\|C∗​\(x\)\|=0,y,C∗​\(x\)=\{y\},C∗​\(x\),\|C∗​\(x\)\|\>1\.g\(x\)=\\begin\{cases\}\\mathrm\{defer\},&p\_\{\\mathrm\{audit\}\}\(x\)<\\alpha\_\{\\mathrm\{def\}\},\\\\ \\mathrm\{defer\},&\|C\_\{\*\}\(x\)\|=0,\\\\ y,&C\_\{\*\}\(x\)=\\\{y\\\},\\\\ C\_\{\*\}\(x\),&\|C\_\{\*\}\(x\)\|\>1\.\\end\{cases\}\(19\)The three outputs correspond to distinct clinical actions: automated narrow prediction, bounded diagnostic ambiguity, and escalation outside the automated classifier\. Empty sets are escalated because no label satisfies the calibrated criterion\. For multiclass tasks, non\-singleton sets can still narrow the plausible differential diagnosis, and set size is reported as a utility cost\.

For cost\-aware deployment, validation data can select an operating point that minimizes the selective risk in Eq\. \([2](https://arxiv.org/html/2607.28696#S3.E2)\), subject to accepted\-coverage and informative\-decision constraints\. The selected policy is fixed before held\-out reporting and adds no distributional guarantee\.

Supplementary Section S2 positions CALCoDe relative to recent medical VLM adaptation, conformal calibration, class\-tail protection, and OOD detection approaches\.

## 5Experiments

### 5\.1Evaluation Setup

The primary evaluation uses two external\-shift dermatology settings\. HAM10000 supplies the reference, validation, gate\-calibration, and conformal\-calibration splits\. ISIC 2019 and PAD\-UFES\-20 serve as external test cohorts and differ from HAM10000 in acquisition protocol, population, prevalence, and image source\. Four frozen image–text backbones span general and medical pretraining: BiomedCLIP\(Zhang and others,[2023](https://arxiv.org/html/2607.28696#bib.bib4)\), OpenAI CLIP ViT\-B/32\(Radford and others,[2021](https://arxiv.org/html/2607.28696#bib.bib1)\), PubMedCLIP ViT\-B/32\(Eslamiet al\.,[2023](https://arxiv.org/html/2607.28696#bib.bib15)\), and MedSigLIP\-448\(Google Health AI Developer Foundations,[2025](https://arxiv.org/html/2607.28696#bib.bib16)\)\. All methods receive the same cached model outputs, class names, prompt templates, and data partitions for a given backbone\. Supplementary Section S3\.1 provides the complete construction and hyperparameter values\.

The main comparison includes adaptive prediction sets \(APS\)\(Angelopouloset al\.,[2021](https://arxiv.org/html/2607.28696#bib.bib30)\), Mondrian conformal prediction \(Mondrian CP\)\(Vovket al\.,[2005](https://arxiv.org/html/2607.28696#bib.bib29)\), localized conformal prediction \(local CP\)\(Guan,[2023](https://arxiv.org/html/2607.28696#bib.bib36)\), Conf\-OT\(Silva\-Rodriguezet al\.,[2025](https://arxiv.org/html/2607.28696#bib.bib14)\), LCP\-VLM\(Fuchset al\.,[2025](https://arxiv.org/html/2607.28696#bib.bib19)\), matched\-promptCPr​\-value\\mathrm\{CP\}\_\{r\\text\{\-value\}\}\(Zenget al\.,[2026](https://arxiv.org/html/2607.28696#bib.bib21)\), the APS variants of TACP and sTACP\(Liuet al\.,[2026](https://arxiv.org/html/2607.28696#bib.bib20)\), and LATA\(Bozorgtabar and others,[2026](https://arxiv.org/html/2607.28696#bib.bib18)\)\. The complete LATA system includes a separately trained ViLU head\. We therefore evaluate its graph\-refinement component and label the comparison explicitly as “LATA graph refinement \(no ViLU\)\.” Supplementary Section S3\.5 records each adaptation and reports the broader catalog of singleton, calibration, deferral, split\-conformal, and regularized adaptive prediction sets \(RAPS\)\.

The fixed comparison point uses target accepted coverage1−δ=0\.951\-\\delta=0\.95, class\-tail guard level1−δy=0\.951\-\\delta\_\{y\}=0\.95, validation discovery boundary1−δ−γ=0\.901\-\\delta\-\\gamma=0\.90, andαdef=0\.05\\alpha\_\{\\mathrm\{def\}\}=0\.05\. Every diagnostic, protected class set, threshold, and operating point is fixed before test evaluation\. Coverage, worst\-class coverage, and set size are computed on accepted examples; worst\-class coverage is the minimum accepted coverage across disease classes\. Deferral is measured over the full test cohort\. Supplementary Section S3\.3 defines the class\-stratified paired bootstrap\. Supplementary Section S4 reports the binary histopathology stress test, while Supplementary Section S5 reports full\-set frequency, autonomous informative rate, and cost\-sensitive policies\.

The experiments address three questions\.Q1asks whether CALCoDe protects the least\-covered class under external shift;Q2examines how frozen\-backbone geometry changes the support audit; andQ3identifies which components drive class\-tail reliability\.

### 5\.2Q1: Does CALCoDe Protect the Least\-Covered Class?

Table[1](https://arxiv.org/html/2607.28696#S5.T1)reports accepted coverage, worst\-class accepted coverage, average accepted set size, and deferral together\. The first two quantities assess reliability; the latter two show how much utility is spent to obtain it\. Reporting all four is necessary because a nearly complete label set can achieve excellent coverage while contributing limited diagnostic specificity\.

\(a\) HAM10000→\\rightarrowISIC 2019

\(b\) HAM10000→\\rightarrowPAD\-UFES\-20

Table 1:Class\-tail reliability across two external dermatology shifts and four frozen VLM backbones\. Each cell reports marginal accepted coverage / worst\-class accepted coverage / mean accepted\-set size / deferral rate\. The final column averages each metric across backbones; “–” denotes methods without an explicit deferral mechanism\.On ISIC 2019, CALCoDe reaches 0\.989 accepted coverage and 0\.970 worst\-class coverage averaged over four backbones\. The corresponding values are 0\.970 and 0\.926 for sTACP, and 0\.973 and 0\.864 for LCP\-VLM\. CALCoDe’s average accepted set contains 6\.71 of seven labels, improving slightly upon sTACP’s 6\.74 while remaining close to LCP\-VLM’s 6\.45; its deferral rate is 0\.132\. OpenAI CLIP clarifies why coverage cannot be read alone: APS and TACP reach 1\.000 by returning essentially the full seven\-class set\. Supplementary Section S5 reports full\-set frequency and autonomous informative rate, separating conservative saturation from useful specificity\.

The paired class\-stratified intervals in Supplementary Figure S1 and Table S2 show that CALCoDe improves accepted and worst\-class coverage over sTACP on both shifts and over LCP\-VLM on ISIC 2019; the PAD\-UFES\-20 differences from LCP\-VLM include zero\.

PAD\-UFES\-20 marks the utility boundary\. CALCoDe averages 0\.999 accepted coverage and 0\.989 worst\-class coverage, while LCP\-VLM obtains 0\.998 and 0\.989\. Their paired differences include zero\. Both methods return approximately 4\.99 labels from a five\-class space; CALCoDe additionally defers 0\.321 of cases on average\. Its reliability improvement over sTACP is significant, but the resulting outputs remain largely non\-specific\. We treat this cohort as evidence that post\-hoc uncertainty control cannot recover class separation absent from the frozen representation\.

Across all eight dataset–backbone settings, CALCoDe is the only evaluated method with observed marginal and worst\-class accepted coverage both at or above 0\.95 in every setting\. LCP\-VLM meets the marginal target in 8/8 settings and the worst\-class target in 5/8; sTACP meets them in 7/8 and 1/8\. These counts summarize empirical external\-shift behavior rather than a target\-domain guarantee\. Class\-wise sample sizes and Wilson intervals in Supplementary Section S3\.3 qualify the point estimates for small limiting classes\.

### 5\.3Q2: How Does Backbone Geometry Change the Audit?

Supplementary Section S3\.2 reports the full PAD\-UFES\-20 geometry audit\. BiomedCLIP has the largest class\-local shift ratio,5\.805\.80, the lowest median audited p\-value,0\.0040\.004, and lower effective covariance rank than OpenAI CLIP,38\.5338\.53versus47\.9947\.99\. The lower effective rank quantitatively supports the representation\-anisotropy and class\-local over\-clustering hypothesis under the shift from dermoscopy to clinical photography\. OpenAI CLIP and MedSigLIP defer less often, but their accepted sets remain nearly full and their singleton rate is zero; deferral and diagnostic specificity must therefore be interpreted jointly\.

### 5\.4Q3: Which Components Drive Class\-Tail Reliability?

Table[2](https://arxiv.org/html/2607.28696#S5.T2)changes one component at a time within the verified APS implementation and averages results over both dermatology shifts and four backbones\. Removing the class\-tail guard lowers worst\-class accepted coverage from 0\.979 to 0\.917, and removing localization lowers it to 0\.900\. Prompt trimming provides a smaller but consequential gain: without it, worst\-class coverage is 0\.970 and reaches 0\.95 in only six of the eight settings\. Guarding every class raises worst\-class coverage to 0\.997 but increases mean set size by 0\.13 labels, showing that validation\-based discovery concentrates the expansion on classes with observed coverage failure\. Removing the support audit leaves accepted coverage nearly unchanged while reducing deferral to zero\. The guard and localization therefore drive class\-tail coverage, whereas the audit controls which cases enter the automated prediction\-set pathway\. Supplementary Section S3\.4 reports per\-setting ablations and target\-attainment counts\.

Increasingαdef\\alpha\_\{\\mathrm\{def\}\}from 0\.01 to 0\.10 raises average deferral and selective accuracy, as expected for a stricter gate\. We useαdef=0\.05\\alpha\_\{\\mathrm\{def\}\}=0\.05as the fixed comparison point rather than selecting it on the test cohorts\. Supplementary Section S5 instantiates the risk in Eq\. \([2](https://arxiv.org/html/2607.28696#S3.E2)\) under four cost regimes and reports validation\-selected operating points\. When deferral is expensive, validation selection lowers mean held\-out deferral by 0\.150 across the eight dermatology settings for the non\-tail\-guarded policy\. The near\-full PAD\-UFES\-20 sets persist, confirming that operating\-point selection cannot recover diagnostic specificity from an incompatible frozen representation\.

Table 2:Contributions of CALCoDe components to class\-tail reliability\.

## 6Discussion

### 6\.1Class\-Tail Failure and Representation Failure

Clinical shift exposes reliability structure that marginal coverage can conceal\. LCP\-VLM preserves marginal coverage while leaving three ISIC 2019 backbones below the worst\-class target, and frequency\-based TACP variants transfer unevenly when source prevalence does not identify the limiting target class\. CALCoDe uses held\-out coverage behavior to identify fragile classes and reserves an independent split for their calibration\. This data separation matters: a class enters the guard because its out\-of\-fold predictions reveal undercoverage, not because its test performance is known or its source count is small\. The one\-sided maximum then increases the threshold only where the validation evidence calls for protection, while retaining every label admitted by the localized conformal rule\.

The experiments distinguish a correctable calibration failure from a representation\-limited failure\. On ISIC 2019, the frozen embeddings preserve enough class\-relevant structure for targeted threshold expansion to raise the least\-covered class without forcing every prediction to contain all seven labels\. On PAD\-UFES\-20, the shift from dermoscopy to clinical photography weakens diagnostic separation, and both CALCoDe and LCP\-VLM approach the complete five\-class set\. Coverage remains high in this regime, but the resulting sets carry limited diagnostic specificity\. A post\-hoc calibration layer can redistribute uncertainty already represented by the backbone; it cannot create class separation absent from the embedding space\.

The geometry audit helps locate this boundary\. BiomedCLIP exhibits a class\-local shift ratio of 5\.80, a median audited p\-value of 0\.004, and a lower effective covariance rank than OpenAI CLIP \(38\.53 versus 47\.99\) on PAD\-UFES\-20\. The lower rank quantitatively supports the representation\-anisotropy and class\-local over\-clustering explanation: a narrower source geometry leaves target clinical photographs with little local support\. OpenAI CLIP and MedSigLIP trigger fewer deferrals, yet their accepted sets remain nearly full\. Deferral frequency therefore reflects the interaction between the audit and the backbone geometry; it is not, by itself, a measure of diagnostic usefulness\.

### 6\.2Interpreting Prediction Sets and Deferral

CALCoDe exposes three outcomes that have different operational meanings\. A singleton supports autonomous classification at the selected operating point\. A non\-singleton set preserves plausible alternatives for downstream review\. Deferral indicates insufficient calibrated support for either form of automated output\. Their clinical value depends on the label space and workflow\. A two\-label set on a binary histopathology task communicates no diagnostic preference, whereas a two\-label set in a seven\-class dermatology task can still narrow the differential diagnosis\. For this reason, coverage, worst\-class coverage, set size, full\-set frequency, singleton rate, and deferral should be reported together\.

This joint view also prevents favorable interpretations based on one metric alone\. Low deferral may coexist with nearly complete prediction sets, as observed for several PAD\-UFES\-20 backbones\. Conversely, a higher deferral rate may be appropriate when an error on a fragile class carries a greater clinical cost than review\. The cost analysis in Supplementary Section S5 makes this dependence explicit: the preferred operating point changes with the relative costs of error, set ambiguity, and deferral\. CALCoDe supplies calibrated statistics and a three\-action interface; the deployment institution must specify the cost regime and available review capacity\.

### 6\.3Statistical Scope

The protected\-class statement rests on two design conditions\. The vulnerable classes are fixed using cross\-fitted validation predictions before the calibration split is accessed, and the accepted calibration and test scores are exchangeable within each protected class\. Under these conditions, the corrected class\-conditional quantile yields finite\-sample coverage at the guard level, while set monotonicity follows deterministically from the maximum operator\. The latter property holds for every input and does not depend on exchangeability\.

External clinical shift requires a narrower interpretation\. Test examples from a new acquisition protocol need not be exchangeable with accepted calibration examples, so target\-site coverage is an empirical result rather than a distribution\-free guarantee\. The gate p\-value has its usual conformal type\-I interpretation only for inlier examples exchangeable with the gate split\. On a shifted cohort, it serves as a calibrated relative\-support index; severe localized shift can make its false\-alarm rate loose\. The experiments consequently report external\-site coverage, class\-wise coverage, and audit behavior without extending the finite\-sample statement to arbitrary covariate shift\.

### 6\.4Limitations and Research Directions

The study centers on two external dermatology shifts, with Camelyon17\-WILDS used as a histopathology stress test\. Broader evidence is needed across radiology, ophthalmology, multimodal clinical records, and prospective multi\-site workflows\. The experiments also use frozen backbones and fixed label spaces\. Representation adaptation may recover specificity in the saturation regime, although its training data and calibration data must remain separated to preserve the stated inference protocol\. Hierarchical label spaces offer another path: a system could retain coarse diagnostic informativeness when fine\-grained classes are not separable\.

CALCoDe requires labeled reference, validation, gate, and calibration partitions\. This sample\-splitting discipline limits leakage but can reduce the number of examples available for rare\-class calibration\. Multi\-site calibration, principled partial pooling across related classes, and sequential recalibration as reviewed labels accumulate are promising extensions\. Such procedures would need explicit control of selection and temporal dependence\. Prospective evaluation should further measure review time, downstream diagnostic decisions, and subgroup\-specific consequences rather than treating deferral as an abstract scalar cost\.

## 7Conclusion

Marginal coverage can hide the disease class most exposed by clinical shift\. CALCoDe makes that failure an explicit calibration target through cross\-fitted class\-tail discovery, independent class\-conditional calibration, and a one\-sided localized conformal guard\. The construction contains the corresponding localized conformal set and provides finite\-sample protected\-class coverage under accepted\-example exchangeability\. Across eight external\-shift settings, CALCoDe is the only evaluated method whose observed marginal and worst\-class accepted coverage both reach the 0\.95 target in every setting\.

The results also identify the boundary of post\-hoc uncertainty correction\. When the frozen representation retains class\-relevant support, targeted calibration can repair class\-specific undercoverage while preserving the localized baseline\. When representations collapse under a severe acquisition shift, prediction sets expand toward the full label space and cease to provide useful diagnostic discrimination\. That behavior should be treated as evidence that the representation, label space, or clinical evidence requires revision, rather than as a successful deployment outcome based on coverage alone\.

Class\-tail coverage, prediction\-set specificity, and deferral together provide a more informative reliability profile for medical VLMs than marginal coverage in isolation\. CALCoDe offers a statistically disciplined way to construct that profile without retraining the underlying model\. Its broader implication is methodological: reliability layers for clinical foundation models should identify which classes fail, state the assumptions under which protection holds, and reveal when calibrated uncertainty has reached the limit imposed by the representation\.

Supplementary Material Mitigating Class\-Tail Undercoverage in Medical Vision\-Language Models under Clinical Shift

## Appendix S1Finite\-Sample Statements

### S1\.1Audited Inlier\-Support Rank Validity

The audited gate uses validation data to choose a scalar reliability diagnostic and an independent gate split to calibrate that diagnostic\. Sample splitting yields the following finite\-sample statement\. Suppose the selected diagnosticBa⋆B\_\{a^\{\\star\}\}is fixed without using𝒟gate\\mathcal\{D\}\_\{\\mathrm\{gate\}\}, and letXXbe an inlier test example exchangeable with the gate examples\{Xj\}j=1ng\\\{X\_\{j\}\\\}\_\{j=1\}^\{n\_\{g\}\}\. Define

paudit​\(X\)=1\+∑j=1ng𝟏​\{Ba⋆​\(Xj\)≥Ba⋆​\(X\)\}ng\+1\.p\_\{\\mathrm\{audit\}\}\(X\)=\\frac\{1\+\\sum\_\{j=1\}^\{n\_\{g\}\}\\mathbf\{1\}\\\{B\_\{a^\{\\star\}\}\(X\_\{j\}\)\\geq B\_\{a^\{\\star\}\}\(X\)\\\}\}\{n\_\{g\}\+1\}\.\(20\)Then, for anyα∈\[0,1\]\\alpha\\in\[0,1\],

Pr⁡\{paudit​\(X\)≤α\}≤α\.\\Pr\\\{p\_\{\\mathrm\{audit\}\}\(X\)\\leq\\alpha\\\}\\leq\\alpha\.\(21\)The proof is the standard conformal rank argument: conditional on the multiset of gate and test scores, the rank of the exchangeable test score among theng\+1n\_\{g\}\+1scores is uniform up to ties, and the conservative numerator with the added one makes the resulting p\-value super\-uniform\. This statement applies when the selected scalar diagnostic is fixed independently of the gate split\. CALCoDe also considers a fused\-rank diagnostic constructed from empirical gate ranks; that candidate is interpreted as a calibrated relative\-support score and is not covered by the fixed\-score rank statement\. For a deployment cohort that is not exchangeable with𝒟gate\\mathcal\{D\}\_\{\\mathrm\{gate\}\}, either audit score remains an empirical inlier\-support diagnostic without distribution\-free false\-alarm control\.

### S1\.2Accepted\-Set Exchangeability

After the gate is fixed, accepted calibration examples are

𝒟calA=\{\(Xi,Yi\):paudit​\(Xi\)≥αdef\}\.\\mathcal\{D\}\_\{\\mathrm\{cal\}\}^\{A\}=\\\{\(X\_\{i\},Y\_\{i\}\):p\_\{\\mathrm\{audit\}\}\(X\_\{i\}\)\\geq\\alpha\_\{\\mathrm\{def\}\}\\\}\.\(22\)If calibration and test pairs are exchangeable from the intended inlier distribution, and the gate is a fixed measurable function of the image and frozen\-model outputs, then accepted calibration pairs and an accepted inlier test pair are exchangeable conditional on acceptance\. Consequently, when localization is disabled, ordinary split conformal prediction applied to𝒟calA\\mathcal\{D\}\_\{\\mathrm\{cal\}\}^\{A\}provides the usual finite\-sample coverage statement for accepted inlier examples, conditional on there being at least one accepted calibration example\. With localized weights, CALCoDe uses the construction in Eq\. \(12\) of the manuscript and inherits the assumptions stated in Section*Localized Conformal Calibration*\. The reported marginal, class\-wise, subgroup, and bootstrap diagnostics quantify empirical behavior beyond the exchangeable inlier setting\.

### S1\.3Cross\-Fitted Class\-Tail Discovery

The accepted validation set is partitioned intoFFstratified folds\. Each validation example is evaluated by a pilot localized conformal rule fitted on the otherF−1F\-1folds\. The resulting out\-of\-fold coverage indicators determine𝒴tail\\mathcal\{Y\}\_\{\\mathrm\{tail\}\}\. Hence𝒴tail\\mathcal\{Y\}\_\{\\mathrm\{tail\}\}is measurable with respect to𝒟ref\\mathcal\{D\}\_\{\\mathrm\{ref\}\},𝒟val\\mathcal\{D\}\_\{\\mathrm\{val\}\}, and𝒟gate\\mathcal\{D\}\_\{\\mathrm\{gate\}\}, and is fixed before𝒟cal\\mathcal\{D\}\_\{\\mathrm\{cal\}\}is accessed\. Classes with fewer thannminn\_\{\\min\}accepted validation examples are included conservatively rather than declared reliable from insufficient evidence\.

### S1\.4Protected\-Class Coverage and Structural Monotonicity

Fix a protected classy∈𝒴taily\\in\\mathcal\{Y\}\_\{\\mathrm\{tail\}\}\. Letnyn\_\{y\}be the number of accepted class\-yyexamples in𝒟cal\\mathcal\{D\}\_\{\\mathrm\{cal\}\}, letr1\(y\),…,rny\(y\)r\_\{1\}^\{\(y\)\},\\ldots,r\_\{n\_\{y\}\}^\{\(y\)\}be their true\-label nonconformity scores, and set

ky=⌈\(ny\+1\)​\(1−δy\)⌉,qy=r\(ky\)\(y\),k\_\{y\}=\\left\\lceil\(n\_\{y\}\+1\)\(1\-\\delta\_\{y\}\)\\right\\rceil,\\qquad q\_\{y\}=r\_\{\(k\_\{y\}\)\}^\{\(y\)\},\(23\)where⌈t⌉\\lceil t\\rceildenotes the smallest integer greater than or equal tott, and an additional value\+∞\+\\inftyis appended before taking the order statistic\. Suppose the frozen gate and score are fixed, and an accepted class\-yytest example is exchangeable with the accepted class\-yycalibration examples\. Conditional on the preceding selection stages,

Pr⁡\{sAPS​\(X,y\)≤qy∣Y=y,A​\(X\)=1,y∈𝒴tail\}≥1−δy\.\\Pr\\\{s\_\{\\mathrm\{APS\}\}\(X,y\)\\leq q\_\{y\}\\mid Y=y,\\,A\(X\)=1,\\,y\\in\\mathcal\{Y\}\_\{\\mathrm\{tail\}\}\\\}\\geq 1\-\\delta\_\{y\}\.\(24\)This follows from the exchangeable rank of the test score among theny\+1n\_\{y\}\+1scores\. Cross\-fitted discovery is essential here: the eventy∈𝒴taily\\in\\mathcal\{Y\}\_\{\\mathrm\{tail\}\}does not reuse the final class\-yycalibration scores that defineqyq\_\{y\}\.

CALCoDe takesτytail\\tau\_\{y\}^\{\\mathrm\{tail\}\}to be at leastqyq\_\{y\}, optionally enlarged by a class\-local threshold\. For any protected classyy, the final threshold is

τCALCoDe​\(x,y\)=max⁡\{τδ,h​\(x\),τytail\},\\tau\_\{\\mathrm\{CALCoDe\}\}\(x,y\)=\\max\\\{\\tau\_\{\\delta,h\}\(x\),\\tau\_\{y\}^\{\\mathrm\{tail\}\}\\\},\(25\)so class membership is monotone relative to localized conformal prediction:

y∈Cδ,hloc​\(x\)⇒y∈CCALCoDe​\(x\)\.y\\in C\_\{\\delta,h\}^\{\\mathrm\{loc\}\}\(x\)\\quad\\Rightarrow\\quad y\\in C\_\{\\mathrm\{CALCoDe\}\}\(x\)\.\(26\)The guard therefore cannot remove a protected true class that the localized set already included, and Eq\. \([24](https://arxiv.org/html/2607.28696#A1.E24)\) is inherited from the class\-global component\. The guarantee is at the prespecified guard level1−δy1\-\\delta\_\{y\}, which may differ from the pooled localized target\. Neither statement supplies distribution\-free protection for a target cohort that violates the accepted\-example exchangeability condition\.

## Appendix S2Relation to Recent Approaches

Medical VLM adaptation usually optimizes the text–image interface while retaining a top\-1 output\(Liu and others,[2025](https://arxiv.org/html/2607.28696#bib.bib8); Koleilatet al\.,[2025](https://arxiv.org/html/2607.28696#bib.bib9); Miao and others,[2026](https://arxiv.org/html/2607.28696#bib.bib22); Shao and others,[2026](https://arxiv.org/html/2607.28696#bib.bib23); Cui and others,[2026](https://arxiv.org/html/2607.28696#bib.bib24)\)\. CALCoDe acts after that interface: it converts frozen class evidence into a selective prediction set and targets the least reliable class rather than average accuracy\.

Calibration\-aware prompt learning improves confidence during adaptation\(Basuet al\.,[2025](https://arxiv.org/html/2607.28696#bib.bib11)\)\. Conf\-OT, LCP\-VLM, and LATA pursue transfer or local set efficiency\(Silva\-Rodriguezet al\.,[2025](https://arxiv.org/html/2607.28696#bib.bib14); Fuchset al\.,[2025](https://arxiv.org/html/2607.28696#bib.bib19); Bozorgtabar and others,[2026](https://arxiv.org/html/2607.28696#bib.bib18)\);CPr​\-value\\mathrm\{CP\}\_\{r\\text\{\-value\}\}models repeated\-score instability\(Zenget al\.,[2026](https://arxiv.org/html/2607.28696#bib.bib21)\)\. CALCoDe uses prompt variability only in its robust evidence and audit layers\. Its class\-tail guard is driven by held\-out class\-wise coverage failure and calibrated after that failure is discovered\.

TACP and sTACP are the closest class\-tail baselines\(Liuet al\.,[2026](https://arxiv.org/html/2607.28696#bib.bib20)\)\. Their penalties are derived from source\-label frequency, which is appropriate when the long\-tailed label distribution determines the vulnerable classes\. CALCoDe defines fragility operationally through out\-of\-fold coverage\. This distinction matters under external site shift, where the target\-limiting class can be common in the source cohort or can change across frozen backbones\.

CALCoDe is also distinct from Mondrian conformal prediction\(Vovket al\.,[2005](https://arxiv.org/html/2607.28696#bib.bib29)\)\. Mondrian calibration stratifies by class and is a natural way to seek class\-conditional reliability, but it applies separate calibration rigidly across all classes, including classes that show no validation evidence of tail risk\. Under clinical shift this can inflate set sizes broadly or still fail when the limiting issue is local support rather than class frequency alone\. CALCoDe uses a one\-sided adaptive guard: localized calibration remains the default, and additional class\-conditional slack is introduced only for validation\-detected tail\-risk classes\. The guard therefore targets weak\-class undercoverage without converting every class into an independent worst\-case calibration problem\.

Relative to standard OOD detection and simple distance gates, CALCoDe couples rejection with calibrated set construction\. OOD scores usually produce a binary accept–reject decision or anomaly ranking, and recent medical OOD benchmarks show that natural\-image conclusions transfer poorly to medical imaging\(Gutbrodet al\.,[2025](https://arxiv.org/html/2607.28696#bib.bib10)\)\. CALCoDe uses a validation\-audited support score for deferral and embedding\-localized conformal calibration for accepted cases\. Its assumptions remain tied to the stated exchangeability conditions\. Empirically, the method combines robust prompt\-derived APS nonconformity, local calibration, validation\-discovered class\-tail protection, and calibrated abstention for medical VLM classification\.

## Appendix S3Experimental Setup

### S3\.1Implementation and Data Splits

The primary external\-shift dermatology experiments evaluate four frozen image\-text backbones: Microsoft BiomedCLIP, OpenAI CLIP ViT\-B/32, PubMedCLIP ViT\-B/32, and Google MedSigLIP\-448\. The exact model identifiers, preprocessing choices, and resolved configuration files are saved with each run\. No backbone is fine\-tuned\. Each dermatology class is represented using five prompt templates:

“a dermoscopic image showing \{label\}\.”“a clinical dermatology image consistent with \{label\}\.”“a skin lesion diagnosed as \{label\}\.”“a medical image of \{label\}\.”“a pigmented lesion image showing \{label\}\.”\\begin\{gathered\}\\text\{\`\`a dermoscopic image showing \\\{label\\\}\.''\}\\\\ \\text\{\`\`a clinical dermatology image consistent with \\\{label\\\}\.''\}\\\\ \\text\{\`\`a skin lesion diagnosed as \\\{label\\\}\.''\}\\\\ \\text\{\`\`a medical image of \\\{label\\\}\.''\}\\\\ \\text\{\`\`a pigmented lesion image showing \\\{label\\\}\.''\}\\end\{gathered\}Experiments use two compute environments\. The Windows 10 workstation contains two Intel Xeon Gold 6226R processors \(32 physical cores in total\) and two NVIDIA RTX A4500 GPUs with 20 GB memory each; it uses NVIDIA driver 553\.35, CUDA 12\.4, Python 3\.10\.20, and PyTorch 2\.5\.1 with CUDA 12\.1 support\. The Linux DGX A100 node contains eight NVIDIA A100\-SXM4 GPUs with 40 GB memory each; each Slurm job reserves one GPU and uses NVIDIA driver 535\.54\.03, CUDA 12\.2, Python 3\.10\.12, and PyTorch 2\.5\.1 with CUDA 12\.1 support\. Feature extraction uses deterministic evaluation mode\. The mini\-batch size is 32 for ViT\-B/32\-scale backbones on the Windows workstation and 16 for MedSigLIP\-448 because it uses 448\-resolution inputs\. Camelyon17 feature extraction for BiomedCLIP and OpenAI CLIP uses one A100 GPU and batch size 128\. Batch size changes throughput only because all backbones remain frozen and evaluation is deterministic\. The reported CALCoDe runs use APS nonconformity from prompt probabilities trimmed withκ=1\\kappa=1,k=10k=10neighbors for the global audit distance, an automatically selected localization bandwidth, localη\\etasearch grid size 1001, five stratified folds for validation\-only class\-tail discovery, discovery threshold0\.900\.90, andnmin=10n\_\{\\min\}=10\. Classes belownminn\_\{\\min\}are protected by default\. The final class\-tail guard level is0\.950\.95, fixed before test evaluation\. Unless otherwise stated, results use pooled target coverage0\.950\.95and deferral levelαdef=0\.05\\alpha\_\{\\mathrm\{def\}\}=0\.05\. The utility\-policy diagnostic searches target coverages\{0\.90,0\.925,0\.95\}\\\{0\.90,0\.925,0\.95\\\}and deferral levels\{0\.01,0\.02,0\.05,0\.10\}\\\{0\.01,0\.02,0\.05,0\.10\\\}on the validation split, fixes the selected operating point, and then reports held\-out test metrics from the cached predictions\.

#### HAM10000 in\-distribution evaluation\.

The 10,015\-image HAM10000 collection, released as the training data for ISIC 2018 Task 3 \(lesion diagnosis\), is split by lesion group into reference, validation, gate, conformal\-calibration, and test roles\. The resulting split sizes are: reference 2040, validation 1935, gate 1986, conformal calibration 2050, and test 2004\.

#### ISIC 2019 external\-shift evaluation\.

For the external\-shift setting, HAM10000 provides the reference, validation, gate, and conformal\-calibration data\. ISIC 2019 test images form the evaluation cohort\. The combined experiment contains 16249 images, with split sizes: reference 2040, validation 1935, gate 1986, conformal calibration 2050, and test 8238\. Among the ISIC 2019 test images, 6026 are inlier examples mapped to the seven HAM10000 labels, while 2212 are OOD examples from classes outside the HAM10000 label space\.

#### PAD\-UFES\-20 external\-shift evaluation\.

For the second external\-shift setting, HAM10000 supplies the reference, validation, gate, and conformal\-calibration data, while PAD\-UFES\-20 supplies the evaluation cohort\. The processed PAD\-UFES\-20 evaluation contains 2106 images mapped to the dermatology label set used in the conformal experiments\. This cohort serves as an external\-domain stress test; OOD detection lies outside the associated claim\.

### S3\.2Backbone Geometry Audit

The PAD\-UFES\-20 shift reveals strong coupling between the audited gate and the frozen embedding geometry\. An audit from cached features quantifies that coupling\. Letdk​\(x,ℛ\)d\_\{k\}\(x,\\mathcal\{R\}\)be the median cosine distance fromxxto itsk=10k=10nearest neighbors in the HAM10000 reference setℛ\\mathcal\{R\}\. The global shift ratio is the median ofdkd\_\{k\}on PAD\-UFES\-20 divided by the corresponding median on held\-out HAM10000 source examples\. The class\-local shift ratio uses only reference neighbors with the same mapped class label\. The audit also reports a linear source\-versus\-target probe balanced accuracy, effective rank of the HAM10000 source embedding covariance, and the top principal\-component variance share\. All diagnostics are computed after feature normalization and play no role in method training or selection\.

Table S1:Detailed PAD\-UFES\-20 embedding\-geometry audit\. BiomedCLIP places PAD\-UFES\-20 farthest from the HAM10000 reference manifold and has the smallest audited p\-values\. OpenAI CLIP and MedSigLIP defer less, with accepted set sizes remaining nearly full; low deferral is therefore insufficient evidence of diagnostic specificity\.At the fixed comparison operating point, the median audited inlier\-support p\-value on PAD\-UFES\-20 is 0\.004 for BiomedCLIP, 0\.388 for OpenAI CLIP ViT\-B/32, 0\.082 for PubMedCLIP, and 0\.778 for MedSigLIP\-448\. OpenAI CLIP and MedSigLIP defer less often, yet their accepted prediction sets remain close to the full five\-class label space and their accepted singleton rate is zero\. Deferral, set size, and singleton rate therefore require joint interpretation under this shift\.

#### Camelyon17\-WILDS stress test\.

Camelyon17\-WILDS is used as a supplementary hospital\-shift stress test in histopathology\. The source hospitals provide reference, validation, gate, and calibration splits, while hospital 2 is held out as the target test domain\. The binary label space contains non\-tumor lymph node tissue and metastatic tumor tissue\. To keep the experiment computationally controlled, the held\-out test set is balanced to 5000 examples per class\.

### S3\.3Metrics, Confidence Intervals, and Audit Stability

#### Metrics\.

Coverage is the fraction of accepted inlier examples for which the true class appears in the prediction set\. Set size is the average number of labels returned on accepted examples\. Deferral is the fraction of examples rejected by the deferral gate\. In\-def\. and OOD\-def\. are deferral rates for inlier and OOD examples, respectively\. Selective accuracy is top\-1 accuracy on accepted inlier examples\. ECE is expected calibration error on accepted examples\. Worst\-class coverage is the minimum accepted coverage across inlier disease classes\.

#### Bootstrap confidence intervals\.

Per\-setting confidence intervals use 500 bootstrap resamples over the saved per\-example prediction files\. The recent\-baseline comparisons use 1000 paired resamples, stratified by true class within each backbone; each replicate recomputes the metric difference and then averages it over the four backbones\. Both procedures are applied after all model outputs, gates, and prediction sets have been fixed\. They measure sampling variation in the empirical metrics without changing any model decision\. Figure[S1](https://arxiv.org/html/2607.28696#A3.F1)visualizes the paired differences, and Table[S2](https://arxiv.org/html/2607.28696#A3.T2)provides their numerical intervals\. The released diagnostic package includes:

recent\_baselines\_summary\.csv recent\_baselines\_dataset\_means\.csv recent\_baselines\_classwise\.csv recent\_baselines\_deltas\.csv recent\_baselines\_paired\_bootstrap\.csv calcode\_matched\_ablation\_summary\.csv calcode\_matched\_ablation\_means\.csv calcode\_matched\_ablation\_deltas\.csv calcode\_matched\_ablation\_reference\_check\.csv audit\_score\_stability\.csv geometry\_summary\.csv distance\_quantiles\.csv pca\_domain\_projection\.csv gamma\_sensitivity\_average\.csv utility\_calcode\_comparison\.csv utility\_calcode\_dataset\_means\.csv validation\_selected\_risk\_delta\_summary\.csv conservative\_xmodal\_budget\_summary\.csv camelyon\_binary\_informative\_utility\_cov0\.95\_def0\.05\.csv ann\_knn\_scaling\.csv

Table S2:Paired macro\-average differences, CALCoDe minus comparator, with 95% class\-stratified bootstrap intervals\. Positive values favor CALCoDe\.![Refer to caption](https://arxiv.org/html/2607.28696v1/x3.png)\(a\)Accepted\-coverage difference\.
![Refer to caption](https://arxiv.org/html/2607.28696v1/x4.png)\(b\)Worst\-class accepted\-coverage difference\.

Figure S1:Paired CALCoDe\-minus\-comparator differences after predictions are fixed, macro\-averaged over four backbones\. Bars show 95% class\-stratified bootstrap intervals\. CALCoDe improves both metrics over sTACP on both shifts and over LCP\-VLM on ISIC 2019; differences from LCP\-VLM on PAD\-UFES\-20 include zero\.
#### Audit\-score stability\.

Since the gate diagnostic is selected on validation data, we also bootstrap the validation split and re\-run the score\-selection step for the core candidate diagnostics: distance, energy, maximum\-softmax confidence, prompt disagreement, and their fused\-rank score\. Table[S3](https://arxiv.org/html/2607.28696#A3.T3)reports the diagnostic selected by the full validation split, its validation AUROC for top\-1 failure detection, the AUROC margin over the second\-best diagnostic, and the fraction of 1000 bootstrap resamples that select the same diagnostic\. Six of the eight settings are stable above 0\.95\. The two unstable cases are both OpenAI CLIP settings, where all candidate diagnostics have weak failure\-detection signal and the AUROC margin is below 0\.01\. The audit is therefore interpreted as a data\-dependent gate selector, with no universal score\-ranking theorem implied\.

Table S3:Audit\-score stability over 1000 validation bootstrap resamples\. MSP denotes negative maximum softmax probability, so larger values indicate weaker confidence\. Bootstrap frequency is the probability that the resampled validation split selects the same diagnostic as the full validation split\.

### S3\.4Component Ablations and Sensitivity

#### Ablation summary\.

Table[S4](https://arxiv.org/html/2607.28696#A3.T4)reports the component ablation associated with Section*Q3: Which Components Drive Class\-Tail Reliability?*of the manuscript\. The values are averaged over HAM10000→\\rightarrowISIC 2019 and HAM10000→\\rightarrowPAD\-UFES\-20 across the four frozen backbones\.

Table S4:Matched CALCoDe ablations averaged over the two dermatology external\-shift datasets and four backbones\. Each variant changes one component of the verified APS implementation\. Shading identifies the complete method\.Removing the class\-tail guard or localization lowers worst\-class coverage to 0\.917 and 0\.900, respectively\. Prompt trimming has a smaller average effect but prevents additional setting\-level failures: the untrimmed variant reaches the 0\.95 worst\-class target in six of eight settings, compared with eight of eight for CALCoDe\. Guarding all classes reaches 0\.997 worst\-class coverage while adding 0\.13 labels to the mean set, which identifies adaptive discovery as an efficiency mechanism\. The no\-audit variant retains similar accepted\-example coverage and sets deferral to zero; the audit governs escalation rather than generating the class\-tail coverage gain\. The released matched\-ablation summary also records each setting and verifies that the complete\-method rows reproduce the primary results exactly\.

#### Tail\-risk margin sensitivity\.

CALCoDe marks a class as tail\-risk when its validation coverage falls below1−δ−γ1\-\\delta\-\\gamma\. Table[S5](https://arxiv.org/html/2607.28696#A3.T5)variesγ\\gammawith the class\-tail guard held at 0\.90, keeping the frozen backbones, calibration splits, audited gate, and pooled target fixed\. The selected valueγ=0\.05\\gamma=0\.05corresponds to a validation discovery boundary of0\.900\.90at pooled target coverage0\.950\.95\. The broad plateau establishes that the discovered class set is not sensitive to small changes in the margin\. A separate confirmatory run in Table[S6](https://arxiv.org/html/2607.28696#A3.T6)sets the final guard to 0\.95, as used by the reported CALCoDe method\.

Table S5:Sensitivity to the validation tail\-risk marginγ\\gammaat class\-tail guard level 0\.90, averaged over the eight dermatology settings\. Largerγ\\gammamarks fewer validation classes as tail\-risk classes; bold marks the fixed discovery margin\.
#### Guard\-level confirmation\.

The final method uses a 0\.95 class\-tail guard, matching the pooled target\. Table[S6](https://arxiv.org/html/2607.28696#A3.T6)compares this choice with the earlier 0\.90 guard while holding the validation\-discovered class sets, audit, localization, and all data splits fixed\. Raising the guard improves ISIC 2019 worst\-class coverage by 0\.0035 and increases its average set size by 0\.028 labels\. PAD\-UFES\-20 coverage and deferral are unchanged to the displayed precision\.

Table S6:Confirmatory comparison of class\-tail guard levels, macro\-averaged over four backbones\. The reported experiments use the 0\.95 row\.

### S3\.5Recent Baseline Implementation

All recent baselines are evaluated from the same cached prompt\-level probabilities, frozen embeddings, class names, and split assignments used by CALCoDe\. The matched\-promptCPr​\-value\\mathrm\{CP\}\_\{r\\text\{\-value\}\}baseline follows the released nonparametric greedy majority ordering: the five prompt variants form repeated score realizations for both calibration true labels and test candidates\. TACP and sTACP use HAM10000 source\-class priors and APS scores\. Their regularization weight and rank parameter are selected by five\-fold source validation, subject to a 0\.01 marginal\-coverage tolerance, before the disjoint calibration split supplies the final threshold\. Neither method accesses target labels\.

The LATA comparison applies its Laplacian graph\-refinement component to the pooled source and unlabeled target embeddings, usingk=15k=15, graph weight 0\.35, and eight refinement iterations\. The separately trained ViLU head is unavailable in this frozen\-backbone setting, so every table names the method “LATA graph refinement \(no ViLU\)\.” These results characterize the graph component and do not support a claim about the complete LATA system\. LCP\-VLM, Conf\-OT, Mondrian CP, local CP, and APS use the same calibration split and target level as CALCoDe\. Empty sets are counted as non\-informative outputs; methods without an explicit rejection mechanism show “–” for deferral in Table 1 of the manuscript\.

## Appendix S4Camelyon17\-WILDS Hospital\-Shift Stress Test

Table[S7](https://arxiv.org/html/2607.28696#A4.T7)reports the Camelyon17\-WILDS hospital\-shift stress test, which is separate from the primary dermatology benchmarks and evaluates CALCoDe without class\-tail guarding\. For weaker zero\-shot backbones, the prediction sets approach the full binary label set, yielding conservative behavior with limited diagnostic specificity\. BiomedCLIP achieves the most informative Camelyon17 tradeoff, with accepted coverage 0\.981, average set size 1\.746, and worst\-class coverage 0\.962\.

Table S7:Camelyon17\-WILDS hospital\-shift stress test\. Metrics are computed on accepted examples from held\-out hospital 2\. In the binary setting, set size near 2 indicates conservative full\-set behavior\.
## Appendix S5Utility and Deployment Diagnostics

#### Reliability–utility profile\.

Table[S8](https://arxiv.org/html/2607.28696#A5.T8)complements Table 1 of the manuscript with two cohort\-level utility measures\. Accepted full\-set rate is the fraction of accepted examples assigned every available label\. Autonomous informative rate is the fraction of the full cohort that is accepted with a set smaller than the complete label space\. The target\-hit column counts the eight dataset–backbone settings with observed accepted coverage and worst\-class coverage at least 0\.95\. CALCoDe has the most consistent class\-tail reliability, while its autonomous informative rate is low on PAD\-UFES\-20\. Conf\-OT andCPr​\-value\\mathrm\{CP\}\_\{r\\text\{\-value\}\}return more informative sets but miss the reliability target in most settings\. LCP\-VLM and CALCoDe both approach complete five\-class sets on PAD\-UFES\-20\.

Table S8:Reliability and utility macro\-averages over four backbones\. Full set is the accepted full\-label\-set rate; Auto\. info\. is the full\-cohort autonomous informative rate\. Target hits count settings with empirical marginal/worst\-class accepted coverage at least 0\.95\. Higher target hits and Auto\. info\. are preferable; lower Full set is preferable\.
#### Cost\-sensitive risk\.

Eq\. \(2\) of the manuscript defines selective decision risk\. Its clinical costs depend on the deployment setting\. For the empirical summary in Table[S9](https://arxiv.org/html/2607.28696#A5.T9), letd^\\hat\{d\}denote the deferral rate,C^\\widehat\{C\}accepted coverage,C^w\\widehat\{C\}\_\{\\mathrm\{w\}\}worst\-class accepted coverage,S^\\widehat\{S\}accepted set size, andKKthe number of classes\. We evaluate

R^=\(1−d^\)​\[cerr​\(1−C^\)\+cerr2​\(1−C^w\)\+cset​S^−1K−1\]\+cdef​d^\.\\widehat\{R\}=\(1\-\\hat\{d\}\)\\left\[c\_\{\\mathrm\{err\}\}\(1\-\\widehat\{C\}\)\+\\frac\{c\_\{\\mathrm\{err\}\}\}\{2\}\(1\-\\widehat\{C\}\_\{\\mathrm\{w\}\}\)\+c\_\{\\mathrm\{set\}\}\\frac\{\\widehat\{S\}\-1\}\{K\-1\}\\right\]\+c\_\{\\mathrm\{def\}\}\\hat\{d\}\.\(27\)The values are averaged over the eight dermatology external\-shift settings at the fixed operating point reported in Section*Evaluation Setup*of the manuscript\. The resulting risks identify the operating regimes suited to each method\. CALCoDe is favored when weak\-class protection has substantial cost; LCP\-VLM is favored when deferral is assigned a very high cost\.

Table S9:Cost\-sensitive risk averaged over HAM10000→\\rightarrowISIC 2019 and HAM10000→\\rightarrowPAD\-UFES\-20 across four backbones\. Lower is better\. CALCoDe is strongest in balanced, safety\-heavy, and efficiency\-heavy regimes; LCP\-VLM is preferred when any deferral is much more costly than class\-tail undercoverage\.
#### Validation\-selected utility policy\.

The fixed comparison operating point supports method comparison, while a deployment site may prefer a different point on the calibrated grid after specifying its clinical costs\. Table[S10](https://arxiv.org/html/2607.28696#A5.T10)reports the average held\-out change obtained by selecting the audited base\-layer operating point on the validation split and then evaluating the selected policy on the external\-shift test cohort\. The selector minimizes the risk above over the saved grid, subject to an accepted\-coverage floor of 0\.95, a minimum of 50 accepted validation examples when feasible, and a minimum informative\-decision rate of 0\.02\. The feasibility constraint is relaxed when no grid point satisfies it, which occurs in three of eight dataset–backbone settings because the validation split itself exposes limited utility\. The deployment policy chooses among calibrated operating points after clinical costs are specified and adds no statistical guarantee\.

Table S10:Validation\-selected utility policy for the audited base layer, averaged over the eight dermatology external\-shift settings\. Deltas are held\-out test differences relative to the fixed comparison pointαdef=0\.05\\alpha\_\{\\mathrm\{def\}\}=0\.05\. Negative risk and deferral deltas are improvements; positive coverage and worst\-class\-coverage deltas are improvements\.
#### Cross\-modal disagreement diagnostic\.

We also evaluated cross\-modal disagreement as a candidate audit score\. The score compares a confident text\-induced class prediction with weak class\-local image support\. A conservative selector allowed this score, or a fused rank containing it, to replace the baseline audit only when it improved validation failure\-detection AUROC by at least 0\.02 and did not exceed a validation deferral budget\. Table[S11](https://arxiv.org/html/2607.28696#A5.T11)shows the target\-domain switches that passed this validation filter\. Two switches are benign or helpful, while two substantially increase target deferral\. The observed target\-domain instability supports reporting cross\-modal disagreement as a diagnostic analysis\. CALCoDe uses the remaining audited reliability candidates in its decision rule\.

Table S11:Accepted switches under conservative cross\-modal disagreement selection\. Deltas are relative to the baseline audited gate at the same operating point\. The validation\-budget rule permits harmful target\-domain deferral increases, so the disagreement analysis remains exploratory and outside the proposed method\.
#### Binary full\-set utility\.

In a binary task, returning the full set𝒴\\mathcal\{Y\}is mathematically covered yet diagnostically uninformative\. Table[S12](https://arxiv.org/html/2607.28696#A5.T12)reports both the original binary utility diagnostic and the stricter deployment rule in Eq\. \(19\) of the manuscript, which converts empty or full binary sets into deferrals\. The stress test exposes a real limitation: except for BiomedCLIP, most frozen zero\-shot backbones produce few informative binary decisions under the reliability constraint\. In those cases, the trust layer identifies the backbone as unsuitable for autonomous hospital\-shift deployment\.

Table S12:Camelyon17 binary utility diagnostic for the audited base layer\. Orig\. informative is the all\-example singleton rate before converting full binary sets into deferrals\. Utility def\. is the final deferral rate after empty or full binary sets are escalated\. Singleton cov\. is coverage among the remaining autonomous singleton decisions\.
#### Compute and storage overhead\.

CALCoDe stores frozen image embeddings for the reference, calibration, and test splits\. In the completed runs, the saved feature caches range from 9\.4 MB to 41\.2 MB per dataset\-backbone pair\. The exact reference sets contain 1992–2040 dermatology images and 1937 Camelyon17 images; embedding dimensions are 512 for ViT\-B/32\-scale CLIP/BiomedCLIP/PubMedCLIP models and 1152 for MedSigLIP\-448\. The global audit distance uses exactkk\-nearest\-neighbor \(kk\-NN\) search withk=10k=10at this scale, and localized calibration uses exact cosine distances to accepted calibration embeddings\. Table[S13](https://arxiv.org/html/2607.28696#A5.T13)evaluates a Hierarchical Navigable Small World \(HNSW\) approximate\-nearest\-neighbor index on cached dermatology embeddings\. The approximate index yields lower per\-image neighbor\-query latency and near\-identical neighbor support distances\. The HNSW experiment measures support\-index scalability; all reported coverage results use exact neighbor calculations\.

Table S13:Approximate\-nearest\-neighbor support\-index diagnostic using HNSW withk=10k=10,M=32M=32, and ef\-search 200\. P95 err\. is the 95th percentile absolute error in the median cosine\-distance support statistic relative to exactkk\-NN\.

## Appendix S6Supporting BiomedCLIP Results

Table S14:HAM10000 in\-distribution results using frozen BiomedCLIP\. Coverage, set size, selective accuracy, ECE, and worst\-class coverage are computed on accepted examples\.Table S15:Class\-wise accepted coverage on the ISIC 2019 external\-shift evaluation using frozen BiomedCLIP\. Class counts are AKIEC: 374, BCC: 975, BKL: 660, DF: 91, NV: 2495, MEL: 1327, and VASC: 104\.Table S16:Class\-wise accepted coverage on the HAM10000 in\-distribution evaluation using frozen BiomedCLIP\. Class counts are AKIEC: 65, BCC: 101, BKL: 223, DF: 20, NV: 1344, MEL: 227, and VASC: 24\.
## References

- Uncertainty sets for image classifiers using conformal prediction\.InInternational Conference on Learning Representations,Cited by:[§4\.3](https://arxiv.org/html/2607.28696#S4.SS3.p1.4),[§5\.1](https://arxiv.org/html/2607.28696#S5.SS1.p2.1)\.
- A\. Basu, F\. Shamshad, A\. Sharifdeen, K\. Nandakumar, and M\. H\. Khan \(2025\)Calibration\-aware prompt learning for medical vision\-language models\.In36th British Machine Vision Conference,External Links:[Link](https://bmva-archive.org.uk/bmvc/2025/assets/papers/Paper_1062/paper.pdf)Cited by:[Appendix S2](https://arxiv.org/html/2607.28696#A2.p2.1),[§1](https://arxiv.org/html/2607.28696#S1.p1.1)\.
- B\. Bozorgtabaret al\.\(2026\)LATA: laplacian\-assisted transductive adaptation for conformal uncertainty in medical VLMs\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 36311–36320\.Cited by:[Appendix S2](https://arxiv.org/html/2607.28696#A2.p2.1),[§1](https://arxiv.org/html/2607.28696#S1.p3.2),[§5\.1](https://arxiv.org/html/2607.28696#S5.SS1.p2.1)\.
- Z\. Chenget al\.\(2025\)Understanding the robustness of vision\-language models to medical image artefacts\.npj Digital Medicine8,pp\. 727\.External Links:[Document](https://dx.doi.org/10.1038/s41746-025-02108-w)Cited by:[§1](https://arxiv.org/html/2607.28696#S1.p2.1)\.
- X\. Cuiet al\.\(2026\)BiomedCCPL: causal conditional prompt learning for biomedical vision\-language models\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 40812–40821\.Cited by:[Appendix S2](https://arxiv.org/html/2607.28696#A2.p1.1),[§1](https://arxiv.org/html/2607.28696#S1.p1.1)\.
- S\. Eslami, C\. Meinel, and G\. de Melo \(2023\)PubMedCLIP: how much does CLIP benefit visual question answering in the medical domain?\.InFindings of the Association for Computational Linguistics: EACL 2023,pp\. 1181–1193\.External Links:[Document](https://dx.doi.org/10.18653/v1/2023.findings-eacl.88)Cited by:[§5\.1](https://arxiv.org/html/2607.28696#S5.SS1.p1.1)\.
- C\. Fuchs, T\. Bary, and B\. Macq \(2025\)Localized conformal prediction for image classification with vision\-language models\.In2025 13th European Workshop on Visual Information Processing,pp\. 1–6\.External Links:[Document](https://dx.doi.org/10.1109/EUVIP66349.2025.11238757)Cited by:[Appendix S2](https://arxiv.org/html/2607.28696#A2.p2.1),[§1](https://arxiv.org/html/2607.28696#S1.p3.2),[§5\.1](https://arxiv.org/html/2607.28696#S5.SS1.p2.1)\.
- Google Health AI Developer Foundations \(2025\)MedSigLIP: medical image\-text encoder for zero\-shot classification and retrieval\.Note:[https://developers\.google\.com/health\-ai\-developer\-foundations/medsiglip](https://developers.google.com/health-ai-developer-foundations/medsiglip)Model cardCited by:[§5\.1](https://arxiv.org/html/2607.28696#S5.SS1.p1.1)\.
- L\. Guan and R\. Tibshirani \(2022\)Prediction and outlier detection in classification problems\.Journal of the Royal Statistical Society Series B: Statistical Methodology84\(2\),pp\. 524–546\.External Links:[Document](https://dx.doi.org/10.1111/rssb.12443)Cited by:[§3](https://arxiv.org/html/2607.28696#S3.p2.3)\.
- L\. Guan \(2023\)Localized conformal prediction: a generalized inference framework for conformal prediction\.Biometrika110\(1\),pp\. 33–50\.External Links:[Document](https://dx.doi.org/10.1093/biomet/asac040)Cited by:[§1](https://arxiv.org/html/2607.28696#S1.p3.2),[§4\.5](https://arxiv.org/html/2607.28696#S4.SS5.p1.2),[§5\.1](https://arxiv.org/html/2607.28696#S5.SS1.p2.1)\.
- C\. Guo, G\. Pleiss, Y\. Sun, and K\. Q\. Weinberger \(2017\)On calibration of modern neural networks\.InProceedings of the 34th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.70,pp\. 1321–1330\.External Links:[Link](https://proceedings.mlr.press/v70/guo17a.html)Cited by:[§3](https://arxiv.org/html/2607.28696#S3.p5.1)\.
- M\. Gutbrod, D\. Rauber, D\. W\. Nunes, and C\. Palm \(2025\)OpenMIBOOD: open medical imaging benchmarks for out\-of\-distribution detection\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 25874–25886\.Cited by:[Appendix S2](https://arxiv.org/html/2607.28696#A2.p5.1),[§1](https://arxiv.org/html/2607.28696#S1.p2.1),[§3](https://arxiv.org/html/2607.28696#S3.p5.1)\.
- R\. Hore and R\. F\. Barber \(2025\)Conformal prediction with local weights: randomization enables robust guarantees\.Journal of the Royal Statistical Society Series B: Statistical Methodology87\(2\),pp\. 549–578\.External Links:[Document](https://dx.doi.org/10.1093/jrsssb/qkae103)Cited by:[§1](https://arxiv.org/html/2607.28696#S1.p3.2),[§4\.5](https://arxiv.org/html/2607.28696#S4.SS5.p1.2)\.
- T\. Koleilat, H\. Asgariandehkordi, H\. Rivaz, and Y\. Xiao \(2025\)BiomedCoOp: learning to prompt for biomedical vision\-language models\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 14766–14776\.Cited by:[Appendix S2](https://arxiv.org/html/2607.28696#A2.p1.1),[§1](https://arxiv.org/html/2607.28696#S1.p1.1),[§3](https://arxiv.org/html/2607.28696#S3.p5.1)\.
- J\. Liuet al\.\(2025\)KPL: training\-free medical knowledge mining of vision\-language models\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.39,pp\. 18852–18860\.External Links:[Document](https://dx.doi.org/10.1609/aaai.v39i18.34075)Cited by:[Appendix S2](https://arxiv.org/html/2607.28696#A2.p1.1),[§1](https://arxiv.org/html/2607.28696#S1.p1.1)\.
- S\. Liu, J\. Huang, and L\. Ong \(2026\)Conformal prediction meets long\-tail classification\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.40,pp\. 23828–23836\.External Links:[Document](https://dx.doi.org/10.1609/aaai.v40i28.39558)Cited by:[Appendix S2](https://arxiv.org/html/2607.28696#A2.p3.1),[§1](https://arxiv.org/html/2607.28696#S1.p3.2),[§5\.1](https://arxiv.org/html/2607.28696#S5.SS1.p2.1)\.
- D\. Mahapatra, B\. Bozorgtabar, S\. Roy, I\. Razzak, and M\. Reyes \(2026\)VALIANT: prompt instability for active learning in black\-box medical imaging\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.40,pp\. 7901–7909\.External Links:[Document](https://dx.doi.org/10.1609/aaai.v40i10.37734)Cited by:[§3](https://arxiv.org/html/2607.28696#S3.p5.1)\.
- P\. Miaoet al\.\(2026\)BioDPP: dynamic prompt policy learning for biomedical vision\-language models\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.40,pp\. 8052–8060\.External Links:[Document](https://dx.doi.org/10.1609/aaai.v40i10.37751)Cited by:[Appendix S2](https://arxiv.org/html/2607.28696#A2.p1.1),[§1](https://arxiv.org/html/2607.28696#S1.p1.1)\.
- Y\. Ovadiaet al\.\(2019\)Can you trust your model’s uncertainty? evaluating predictive uncertainty under dataset shift\.InAdvances in Neural Information Processing Systems,Vol\.32\.Cited by:[§1](https://arxiv.org/html/2607.28696#S1.p2.1),[§3](https://arxiv.org/html/2607.28696#S3.p5.1)\.
- A\. Radfordet al\.\(2021\)Learning transferable visual models from natural language supervision\.InProceedings of the 38th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.139,pp\. 8748–8763\.External Links:[Link](https://proceedings.mlr.press/v139/radford21a.html)Cited by:[§1](https://arxiv.org/html/2607.28696#S1.p1.1),[§5\.1](https://arxiv.org/html/2607.28696#S5.SS1.p1.1)\.
- M\. Shaoet al\.\(2026\)vMFCoOp: towards equilibrium on a unified hyperspherical manifold for prompting biomedical VLMs\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.40,pp\. 8851–8859\.External Links:[Document](https://dx.doi.org/10.1609/aaai.v40i11.37839)Cited by:[Appendix S2](https://arxiv.org/html/2607.28696#A2.p1.1),[§1](https://arxiv.org/html/2607.28696#S1.p1.1)\.
- J\. Silva\-Rodriguez, I\. Ben Ayed, and J\. Dolz \(2025\)Conformal prediction for zero\-shot models\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 19931–19941\.Cited by:[Appendix S2](https://arxiv.org/html/2607.28696#A2.p2.1),[§1](https://arxiv.org/html/2607.28696#S1.p3.2),[§5\.1](https://arxiv.org/html/2607.28696#S5.SS1.p2.1)\.
- R\. J\. Tibshirani, R\. F\. Barber, E\. J\. Candes, and A\. Ramdas \(2019\)Conformal prediction under covariate shift\.InAdvances in Neural Information Processing Systems,Vol\.32,pp\. 2526–2536\.Cited by:[§3](https://arxiv.org/html/2607.28696#S3.p5.1)\.
- V\. Vovk, A\. Gammerman, and G\. Shafer \(2005\)Algorithmic learning in a random world\.Springer\.Cited by:[Appendix S2](https://arxiv.org/html/2607.28696#A2.p4.1),[§5\.1](https://arxiv.org/html/2607.28696#S5.SS1.p2.1)\.
- Z\. Wang, Z\. Wu, D\. Agarwal, and J\. Sun \(2022\)MedCLIP: contrastive learning from unpaired medical images and text\.InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing,pp\. 3876–3887\.External Links:[Document](https://dx.doi.org/10.18653/v1/2022.emnlp-main.256)Cited by:[§1](https://arxiv.org/html/2607.28696#S1.p1.1)\.
- P\. Xiaet al\.\(2024\)CARES: a comprehensive benchmark of trustworthiness in medical vision language models\.InAdvances in Neural Information Processing Systems,Vol\.37,pp\. 140334–140365\.External Links:[Document](https://dx.doi.org/10.52202/079017-4455)Cited by:[§1](https://arxiv.org/html/2607.28696#S1.p2.1)\.
- Y\. Xieet al\.\(2025\)MedTrinity\-25M: a large\-scale multimodal dataset with multigranular annotations for medicine\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=IwgmgidYPS)Cited by:[§1](https://arxiv.org/html/2607.28696#S1.p1.1)\.
- J\. Zeng, Y\. Prabhu, Z\. Zeng, M\. A\. Newton, and V\. Singh \(2026\)Empirical bayes conformal prediction for vision and language models\.arXiv preprint arXiv:2605\.23189\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2605.23189)Cited by:[Appendix S2](https://arxiv.org/html/2607.28696#A2.p2.1),[§1](https://arxiv.org/html/2607.28696#S1.p3.2),[§5\.1](https://arxiv.org/html/2607.28696#S5.SS1.p2.1)\.
- S\. Zhanget al\.\(2023\)BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image\-text pairs\.arXiv preprint arXiv:2303\.00915\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2303.00915)Cited by:[§1](https://arxiv.org/html/2607.28696#S1.p1.1),[§5\.1](https://arxiv.org/html/2607.28696#S5.SS1.p1.1)\.
- K\. Zhou, J\. Yang, C\. C\. Loy, and Z\. Liu \(2022\)Learning to prompt for vision\-language models\.International Journal of Computer Vision130\(9\),pp\. 2337–2348\.External Links:[Document](https://dx.doi.org/10.1007/s11263-022-01653-1)Cited by:[§1](https://arxiv.org/html/2607.28696#S1.p1.1),[§3](https://arxiv.org/html/2607.28696#S3.p5.1)\.