OmniMed-Jev：通过 System One 校准 LVLM 置信度以实现可信的医学多模态决策

# Calibrating LVLM Confidencefor Trustworthy Medical Multimodal Decisions via System One
Source: [https://arxiv.org/html/2610.00381](https://arxiv.org/html/2610.00381)
OmniMed\-Jev

## OmniMed\-Jev: Calibrating LVLM Confidence for Trustworthy Medical Multimodal Decisions via System One

Luyao TangEmail:[lytang1999@gmail\.com](mailto:)Affiliation:Cheng ChenEmail:[cchen@eee\.hku\.hk](mailto:)Affiliation:Department of Electrical and Computer EngineeringAffiliation:The University of Hong Kong, Hong Kong SAR, China

###### Abstract

Medical models are judged not only on whether they answer correctly, but on whether the confidence they report corresponds to how often they are right\. Generalist multimodal medical models have widened what a single model can perceive and solve, yet they still express bounded decisions such as a diagnosis, a set of findings or a cell count as generated text, so the probability the model reports is a probability about the next token rather than about the decision\. Motivated by decision\-native interfaces such as Jev, we introduceOmniMed\-Jev, which represents each medical decision as aChoice,NoulorScoredecision over a candidate set supplied at runtime and returns a full distribution over that set: mutually exclusive classes, binary presence of a finding, or a bounded ordered value\. The design is omni in three respects: the same framework accepts different imaging modalities, covers different prediction tasks, and expresses them through one candidate\-conditioned probability model, so heterogeneous outputs become comparable probabilities rather than task\-specific strings\. Under an interface\-controlled comparison against a generative baseline trained on the same backbone, data and schedule, OmniMed\-Jev’s reported probabilities track observed correctness far more closely, cutting calibration error by up to an order of magnitude and reliability error by up to two, while point\-prediction performance remains comparable and counting is the one family in which the generative baseline stays ahead: making the decision distribution the model’s output is not a change of format but what turns the numbers a medical model reports into probabilities that mean what they say\. These results support explicit decision modeling as a way to make reported confidence meaningful within the tasks evaluated here, and they are not evidence of clinical readiness: the comparison cannot separate the interface from the training differences that come with it, and we state those limits with the results\. Code is available at[github\.com/lytang63/OmniMed\-Jev](https://github.com/lytang63/OmniMed-Jev)\.

Keywords:medical multimodal large language models, decision modeling, calibration, selective prediction, uncertainty

## 1\. Introduction

Consider two models that make the same diagnosis on the same image and are wrong equally often\. One reports high confidence on the cases it gets wrong; the other lowers its confidence when it is about to be wrong\. Their accuracy is identical, yet only the second is useful for deciding which cases deserve re\-examination, and only the second can be audited against outcomes\. Medical AI is therefore judged twice, on whether its answer is correct and on whether the confidence it reports corresponds to how often it is right, and this paper is about the second question\. In group terms, predictions made at roughly80%80\\%confidence should be correct roughly80%80\\%of the time, and lower confidence should identify a larger share of errors; an individual prediction is either right or wrong, so the requirement is a property of the reported probabilities rather than of a single answer\. The mismatch runs in both directions:

1. 1\.Overconfidence: the model reports more certainty than its accuracy supports, so errors are accepted with too little scrutiny\.
2. 2\.Underconfidence: it reports less certainty than its accuracy supports, so cases that could be handled automatically are escalated instead\.

Two capabilities are at stake: being accurate, and reporting confidence that can be checked against how often the model is actually right\. Generalist medical multimodal large language models have become genuinely generalist in what they perceive and in what they solve\. One model reads histopathology slides, chest radiographs, computed tomography, fundus photographs, ultrasound and blood\-smear microscopy, and answers questions from naming a tumour subtype to reporting a pleural effusion\([Singhal et al\., 2023](https://arxiv.org/html/2610.00381#bib.bib3);[Singhal et al\., 2025](https://arxiv.org/html/2610.00381#bib.bib4);[Li et al\., 2023](https://arxiv.org/html/2610.00381#bib.bib5);[Moor et al\., 2023](https://arxiv.org/html/2610.00381#bib.bib6);[Zhang et al\., 2024](https://arxiv.org/html/2610.00381#bib.bib7);[Sellergren et al\., 2025](https://arxiv.org/html/2610.00381#bib.bib1);[Google, 2026](https://arxiv.org/html/2610.00381#bib.bib2)\)\. This diversity stops at the output\. A diagnosis, a set of findings, a cell count and a continuous measurement are bounded decisions with different structures, and all of them are serialized into text, so the computation behind the answer is the same conditional token distribution the model uses to write prose\. The probability a token\-level model assigns to a string is not directly the probability of the decision that string encodes, and recovering one from the other is an additional inference step rather than a property of the model\. Alternatives exist, including constrained decoding, explicit candidate scoring and post\-hoc calibration, but they all operate on a quantity the model was not trained to produce, and the medical literature has measured the consequence repeatedly: reported confidence overestimates accuracy, calibration varies across specialties, and deferral policies behave unpredictably because the confidence they consume was never the probability of the decision\([Bentegeac et al\., 2025](https://arxiv.org/html/2610.00381#bib.bib51);[Savage et al\., 2025](https://arxiv.org/html/2610.00381#bib.bib49);[Naderi et al\., 2026](https://arxiv.org/html/2610.00381#bib.bib53);[Testoni and Calixto, 2026](https://arxiv.org/html/2610.00381#bib.bib63);[Byun et al\., 2026](https://arxiv.org/html/2610.00381#bib.bib65);[Machcha et al\., 2026](https://arxiv.org/html/2610.00381#bib.bib70);[Jabbour et al\., 2025](https://arxiv.org/html/2610.00381#bib.bib69)\)\.

Typed decision interfaces point at a different arrangement\.*Jev motivates the decision interface; medical multimodality and task heterogeneity motivate its extension\.*The interface makes the candidate set explicit and returns a distribution over it, and three kinds of decision map naturally onto medical prediction:Choicefor mutually exclusive selection such as a differential diagnosis,Noulfor binary presence such as a finding on a radiograph, andScorefor a bounded ordered value such as a cell count or an ejection fraction\. Medicine then adds a requirement of its own\. A generalist medical model has to satisfy three requirements at once:

1. 1\.Omni\-modality: it accepts what medicine shows it, different imaging types from histology to fundus photography\.
2. 2\.Omni\-task: it solves what medicine asks of it, from naming a class to counting cells\.
3. 3\.Omni\-decision: it decides in one comparable form, so that a class, a finding and a grid value are probabilities of the same kind rather than one output format per task\.

The third requirement is the methodological claim here, and it is worth separating from the first two: coverage across modalities and tasks is a property of the evaluation suite, whereas a shared candidate\-conditioned probability model is what makes heterogeneous decisions expressible, comparable and auditable in the same way\. OmniMed\-Jev is therefore our medical implementation and test of the decision\-interface idea rather than a medical copy of a general decision model: it keeps a medical multimodal backbone and adds a shared candidate scoring mechanism\.

This raises an empirical question rather than a claim: if the decision distribution is the model’s output object, do the reported probabilities correspond better to observed correctness, and does the model keep its predictive performance? Which families benefit, and which do not? We answer with a controlled comparison: a generative baseline trained on the same backbone, images, adaptation recipe and optimization schedule and evaluated on the same held\-out states, so that the comparison isolates how a decision is represented\. We hypothesise that a substantial part of the mismatch lies in the output interface; the comparison cannot separate the interface from the denser supervision and the additional decision head that come with it, and we mark that boundary wherever it matters\.

Our contributions are:

- •We formulateconfidence–correctness alignment, the correspondence between reported probabilities and observed correctness, as a research question for generalist medical models alongside accuracy\.
- •We introduceOmniMed\-Jev, a decision model that is omni in modality, task and decision form:Choice,NoulandScoredecisions over candidate sets specified at runtime, with many parallel decisions per state and one shared training objective\.
- •We evaluateit under an interface\-controlled comparison across four task families and show that the interface buys confidence rather than accuracy: reliability error falls by 9×\\timesto 126×\\timesin three of the four families while point predictions stay comparable, and the one family that favours the generative baseline is a documented property of the ordered\-grid representation rather than a limit of the approach\.

\(a\) Generalist Medical MLLMs:Omni Tasks, One Generative InterfaceMedical StatesPathology / CTRadiology / FundusUltrasound / Hematology⋯\\cdotsMedical MLLMMedGemma\-1\.5\-4BClassification"melanoma"Multi\-label"effusion; \.\.\."Counting"27"Regression"54\.3"Many tasks→\\rightarrowone generative interfaceDifferent medical semantics areserialized as text\.Omnito\(b\) OmniMed\-Jev:System One Medical DecisionsMedical StateImage\(s\) \+ QuestionShared BackboneMedGemma\-1\.5\-4BDecision SchemaType \+ Runtime CandidatesChoiceP⁡\(c1\),…,P⁡\(cK\)P\(c\_\{1\}\),\\ldots,P\(c\_\{K\}\)diagnosis / classNoulP⁡\(No\),P⁡\(Yes\)P\(\\mathrm\{No\}\),P\(\\mathrm\{Yes\}\)medical findingsScoreP⁡\(v1\),…,P⁡\(vK\)P\(v\_\{1\}\),\\ldots,P\(v\_\{K\}\)count / regressionOne state→\\rightarrowmany parallel decisionsCandidate semantics are inputs;probabilities are native outputs\.3/4task families improved∙\\bullet30/36probability metrics improved∙\\bullet126×\\timeslower multi\-label reliability error

Figure 1:OmniMed\-Jev rethinks medical MLLMs as System One decision models\.\(a\) Generalist medical MLLMs already span heterogeneous modalities and tasks, yet bounded predictions with fundamentally different semantics are serialized through the same autoregressive text interface\. Classification, multi\-label recognition, counting, and regression therefore share the same generative output mechanism despite requiring distinct decision semantics\. \(b\) OmniMed\-Jev replaces unnecessary generation with typed probabilistic decisions over runtime\-specified candidate spaces\. The shared medical backbone directly produces Choice, Noul, and Score distributions, allowing one medical state to support multiple parallel decisions\. When a medical output is inherently a decision, OmniMed\-Jev decides directly rather than first generating that decision as text\.
## 2\. Related Work

### 2\.1\. Generalist medical multimodal models

The generalist turn in medical AI has widened perception and task coverage without changing the output interface\. Clinical language models first showed that medical knowledge can be elicited to expert\-level accuracy on multiple\-choice questions\([Singhal et al\., 2023](https://arxiv.org/html/2610.00381#bib.bib3);[Singhal et al\., 2025](https://arxiv.org/html/2610.00381#bib.bib4)\), and multimodal instruction tuning carried the recipe to images: medical visual assistants\([Li et al\., 2023](https://arxiv.org/html/2610.00381#bib.bib5);[Moor et al\., 2023](https://arxiv.org/html/2610.00381#bib.bib6)\), unified biomedical pretraining\([Zhang et al\., 2024](https://arxiv.org/html/2610.00381#bib.bib7)\), chest\-radiograph and radiology\-report systems\([Chen et al\., 2024](https://arxiv.org/html/2610.00381#bib.bib8);[Bannur et al\., 2024](https://arxiv.org/html/2610.00381#bib.bib9);[Zambrano Chaves et al\., 2025](https://arxiv.org/html/2610.00381#bib.bib10)\), and open medical backbones evaluated across modalities with shared weights\([Sellergren et al\., 2025](https://arxiv.org/html/2610.00381#bib.bib1);[Google, 2026](https://arxiv.org/html/2610.00381#bib.bib2)\)\. Benchmarks grew in parallel, from curated VQA collections to broad examinations and clinician\-written rubrics\([Liu et al\., 2021](https://arxiv.org/html/2610.00381#bib.bib13);[Lau et al\., 2018](https://arxiv.org/html/2610.00381#bib.bib14);[Yang et al\., 2023](https://arxiv.org/html/2610.00381#bib.bib11);[Liu et al\., 2020](https://arxiv.org/html/2610.00381#bib.bib12);[Pal et al\., 2022](https://arxiv.org/html/2610.00381#bib.bib15);[Jin et al\., 2021](https://arxiv.org/html/2610.00381#bib.bib16);[Arora et al\., 2025](https://arxiv.org/html/2610.00381#bib.bib17)\)\. What this literature does not unify is how an answer is represented: classification, findings, counts and measurements are all serialized into strings and compared against a reference, so neither the objective nor the evaluation asks the model to distribute probability mass over a finite, auditable set of options\.

### 2\.2\. Uncertainty and calibration in medicine

Measurements of medical model confidence converge on one result: what a model reports about its own certainty carries little information about whether it is right, whether verbalized or read from token probabilities\([Bentegeac et al\., 2025](https://arxiv.org/html/2610.00381#bib.bib51);[Savage et al\., 2025](https://arxiv.org/html/2610.00381#bib.bib49);[Naderi et al\., 2026](https://arxiv.org/html/2610.00381#bib.bib53);[Testoni and Calixto, 2026](https://arxiv.org/html/2610.00381#bib.bib63)\), and multimodal medical models inherit the deficit\([Byun et al\., 2026](https://arxiv.org/html/2610.00381#bib.bib65);[Du et al\., 2025](https://arxiv.org/html/2610.00381#bib.bib64);[Sharifdeen et al\., 2026](https://arxiv.org/html/2610.00381#bib.bib66)\)\. The cause is structural: instruction tuning and preference optimization raise accuracy while inflating confidence\([Xiong et al\., 2024](https://arxiv.org/html/2610.00381#bib.bib27);[Zhou et al\., 2024](https://arxiv.org/html/2610.00381#bib.bib40)\), so the step that makes a model useful damages its probability\. Both standard responses keep the interface intact: post\-hoc calibration rescales the model’s output\([Guo et al\., 2017](https://arxiv.org/html/2610.00381#bib.bib18);[Zhao et al\., 2021](https://arxiv.org/html/2610.00381#bib.bib28);[Xie et al\., 2024](https://arxiv.org/html/2610.00381#bib.bib34)\); calibration\-aware objectives train it to emit a calibrated confidence\([Damani et al\., 2026](https://arxiv.org/html/2610.00381#bib.bib41);[Bani\-Harouni et al\., 2026](https://arxiv.org/html/2610.00381#bib.bib42);[Li et al\., 2025](https://arxiv.org/html/2610.00381#bib.bib45)\), which is then verbalized as text\. The deferral, human\-factors and regulatory literature inherits that object \(Appendix[H](https://arxiv.org/html/2610.00381#A8)\)\.

### 2\.3\. From language likelihood to decision probability

The machinery that consumes a probability is old and well developed: the reject option, selective classification and conformal prediction take the probability as given and optimize what to do with it\([Chow, 1970](https://arxiv.org/html/2610.00381#bib.bib22);[Geifman and El\-Yaniv, 2017](https://arxiv.org/html/2610.00381#bib.bib21);[Angelopoulos and Bates, 2023](https://arxiv.org/html/2610.00381#bib.bib23)\)\. In medicine the required calibrated probability is often unavailable: abstention is close to orthogonal to accuracy\([Machcha et al\., 2026](https://arxiv.org/html/2610.00381#bib.bib70);[Cocchieri et al\., 2026](https://arxiv.org/html/2610.00381#bib.bib71)\), deferral relocates error rather than removing it\([Jabbour et al\., 2025](https://arxiv.org/html/2610.00381#bib.bib69)\), and cascading on the same signal buys little\([Fanconi and van der Schaar, 2025](https://arxiv.org/html/2610.00381#bib.bib72)\)\. A recent wave of decision\-native models removes the generation step entirely, making the candidate set part of the input and returning a distribution over it without emitting text\([TypeSafe AI, 2026](https://arxiv.org/html/2610.00381#bib.bib92);[Li et al\., 2026c](https://arxiv.org/html/2610.00381#bib.bib93);[Ibrahim and Zaki, 2026](https://arxiv.org/html/2610.00381#bib.bib95);[Rafe and Das, 2026](https://arxiv.org/html/2610.00381#bib.bib99);[Jiang et al\., 2026](https://arxiv.org/html/2610.00381#bib.bib98);[Ren et al\., 2026](https://arxiv.org/html/2610.00381#bib.bib100);[Wu and Lim, 2026](https://arxiv.org/html/2610.00381#bib.bib96);[Cheng et al\., 2026](https://arxiv.org/html/2610.00381#bib.bib102);[Yu and Yao, 2026](https://arxiv.org/html/2610.00381#bib.bib103)\); the framing is old\([Kahneman, 2011](https://arxiv.org/html/2610.00381#bib.bib101)\)\. The one controlled study of how such a model binds answers to options shows that swapping which rubric is attached to which option name changes a substantial fraction of its answers while the type\-error rate stays at zero\([Sun and Xu, 2026](https://arxiv.org/html/2610.00381#bib.bib94)\), and the single medical evaluation to date judges report factuality without measuring calibration\([Huang et al\., 2026](https://arxiv.org/html/2610.00381#bib.bib97)\)\.

The field therefore diagnoses the problem and proposes an interface, but nothing connects the two under control: confidence studies repair generative confidence post hoc, calibration\-aware objectives still verbalize the probability, and decision\-native models report cost and accuracy rather than calibration\. This paper supplies that controlled comparison\.

## 3\. OmniMed\-Jev

### 3\.1\. From generation to decisions

A generative medical MLLM factorizes its answer as a product of token probabilities,pLM​\(y∣S,Q\)=∏tp⁡\(yt∣y<t,S,Q\)p\_\{\\mathrm\{LM\}\}\(y\\mid S,Q\)=\\prod\_\{t\}p\(y\_\{t\}\\mid y\_\{<t\},S,Q\), whereSSis a multimodal medical state andQQa question\. Under that interface a diagnosis, a set of findings, a count and a continuous measurement are all sequences of tokens, and any notion of a decision probability has to be recovered from the factorization after the fact\. OmniMed\-Jev replaces the answer string with a decision,Dj=\(Qj,Tj,Cj\)D\_\{j\}=\(Q\_\{j\},T\_\{j\},C\_\{j\}\)over a candidate spaceCj=\{cj​1,…,cj​Kj\}C\_\{j\}=\\\{c\_\{j1\},\\dots,c\_\{jK\_\{j\}\}\\\}, whereTjT\_\{j\}is the decision type, and trains the model to return a distribution over that space,

pθ​\(c∣S,Dj\),c∈Cj,∑c∈Cjpθ​\(c∣S,Dj\)=1\.p\_\{\\theta\}\\\!\\left\(c\\mid S,D\_\{j\}\\right\),\\qquad c\\in C\_\{j\},\\qquad\\sum\_\{c\\in C\_\{j\}\}p\_\{\\theta\}\(c\\mid S,D\_\{j\}\)=1\.\(1\)The output is thus a distribution over an enumerable set rather than a string; a point prediction, when one is needed, is a deterministic function of that distribution, which we keep as the primary object because it is what a downstream decision rule consumes\. Medical decisions differ in structure, and the interface reflects this with three types, each defined by what a decision rule needs from it\.Choiceis a mutually exclusive selection: the candidates are the classes, as in a differential diagnosis, and what the rule needs is the probability of each class rather than only the name of the winner\.Noulis binary presence: one decision per finding per state, as in whether a pleural effusion is present on a radiograph, and each finding needs its own probability because a rule that escalates findings acts on them one by one\.Scoreis a bounded ordered value: the candidates are the admissible values in order, as in a cell count or an ejection fraction, and a distribution over that grid is what makes an expectation and an interval available; because position defines the order, Score candidate sets are never permuted\. Free\-form report generation and bounding\-box detection are outside the scope of this paper: span decisions for boxes were removed on both models because the coordinate representation did not localize findings in our implementation \(Appendix[A](https://arxiv.org/html/2610.00381#A1)\), and unbounded text has no candidate set to score\. Neither exclusion is a claim about what the interface could express\.

### 3\.2\. Candidate\-conditioned shared decision scoring

Omni\-modeling is the methodological contribution, and the commitment behind it is that candidates are*input semantics*rather than indices of a fixed classifier\. All three decision types are scored by one mechanism, so a class, a finding and a grid value are represented alike and their probabilities are comparable\. A decision block is rendered into the context: the question followed by the candidate options, each terminated by a marker token, with a second marker closing the decision\. The model reads this sequence with the image interleaved, and a shared scoring head consumes the hidden state at the candidate marker together with the state representation,

zj​k=fϕ​\(hstate,hcj​k\),pj​k=exp⁡\(zj​k\)∑lexp⁡\(zj​l\)\.z\_\{jk\}=f\_\{\\phi\}\\\!\\left\(h\_\{\\mathrm\{state\}\},\\,h\_\{c\_\{jk\}\}\\right\),\\qquad p\_\{jk\}=\\frac\{\\exp\(z\_\{jk\}\)\}\{\\sum\_\{l\}\\exp\(z\_\{jl\}\)\}\.\(2\)Because the head reads hidden states at marked positions, no projection to the vocabulary is required, and because the candidate text is in the context, the model must read the option it is scoring instead of learning a positional label map\. Training reshuffles Choice and Noul candidates every epoch, while Score candidates stay in place because their position encodes their value; the generative baseline receives the equivalent augmentation under the same schedule, since comparing an interface that ignores option order against one that exploits it would confound the comparison\.

### 3\.3\. One state, many decisions

A medical state usually carries several decisions at once: a chest radiograph is annotated with a set of findings, a blood smear with a total count and cell\-type counts\. The interface encodes the state once and returns all of its decisions in one forward pass,

S⟶\{D1,…,DM\},\{pθ\(⋅∣S,D1\),…,pθ\(⋅∣S,DM\)\}\.S\\;\\longrightarrow\\;\\\{D\_\{1\},\\dots,D\_\{M\}\\\},\\qquad\\left\\\{p\_\{\\theta\}\(\\cdot\\mid S,D\_\{1\}\),\\dots,p\_\{\\theta\}\(\\cdot\\mid S,D\_\{M\}\)\\right\\\}\.\(3\)Under the generative interface the same information is serialized into one string or elicited with separate prompts\. For training, the annotations are factorized according to their decision semantics rather than collapsed into one answer per image, so the supervision is denser even though no new labels are collected\.

### 3\.4\. Unified decision learning

All three decision types are trained with one objective, the cross\-entropy between the model distribution and the target distribution induced by the annotation for each decision,

ℒ\(θ\)=−1M∑j=1M∑k=1Kjqj​klogpj​k,\\mathcal\{L\}\(\\theta\)=\-\\frac\{1\}\{M\}\\sum\_\{j=1\}^\{M\}\\sum\_\{k=1\}^\{K\_\{j\}\}q\_\{jk\}\\log p\_\{jk\},\(4\)whereqj​kq\_\{jk\}is the target distribution overCjC\_\{j\}for decisionjjandMMthe number of decisions in the batch\. Adaptation proceeds in two stages on top of the frozen backbone: the decision head is trained first, and then the head is trained together with low\-rank adapters\. The generative baseline uses the same backbone, adapter configuration and number of optimization steps with the corresponding next\-token objective\. Hyperparameters, parameter counts and compute are reported in Section[4\.2](https://arxiv.org/html/2610.00381#S4.SS2)and Appendix[E](https://arxiv.org/html/2610.00381#A5)\.

Table 1:Point\-prediction performance of the two models on the held\-out suite \(n=200n\{=\}200for classification, multi\-label and regression;n=126n\{=\}126for counting\)\. Task metrics are computed by the task evaluator on folded predictions; the two models share the backbone, training images and main training budget, and their output objectives differ \(Section[4\.2](https://arxiv.org/html/2610.00381#S4.SS2)\)\. Multi\-label F1 is a macro average; under micro averaging the models exchange places\. Paired bootstrap intervals and McNemar and sign tests are in Appendix[E](https://arxiv.org/html/2610.00381#A5)\.

## 4\. Experiments

### 4\.1\. An omni\-task, omni\-modality suite

Table[3](https://arxiv.org/html/2610.00381#A1.T3)in Appendix[A](https://arxiv.org/html/2610.00381#A1)summarizes the evaluation suite\. Four task families instantiate the three decision types and are drawn from fifteen source datasets covering histology, pathology, computed tomography, haematology, dermatology, ophthalmology, chest radiography, ultrasound and fundus photography\. Classification pools ten MedMNIST collections into a single choice problem over their organ and tissue classes; multi\-label recognition asks one presence decision per chest\-radiograph finding; counting predicts cell counts in blood\-smear microscopy; and regression predicts left\-ventricular ejection fraction from cardiac ultrasound and a retinal score from fundus photographs\. The suite is deliberately heterogeneous: it contains families with overlapping label semantics across modalities, families with many parallel decisions per state, and families whose candidate space is a large ordered grid\. The tasks were selected and the splits constructed by us rather than adopted from a published benchmark, so that neither model can profit from a generalist model having been tuned against a widely used evaluation; what the comparison reflects is the two output interfaces, not two levels of familiarity with a known test set\. Splits and candidate grids are in Appendix[A](https://arxiv.org/html/2610.00381#A1)\.

### 4\.2\. Generative baseline and matched settings

The comparison is interface\-controlled: the two models share the instruction\-tuned medical backbone, the training images, the preprocessing, the low\-rank adapter configuration and the main training budget, and differ in their output objective and in the training implementation it requires\. The generative baseline answers by producing the answer text under the corresponding next\-token objective\. OmniMed\-Jev adapts in two stages: the decision head is trained first, and then the head is trained together with the adapters\. The head carries1,057,4101\{,\}057\{,\}410parameters; the adapted stage trains39,555,20239\{,\}555\{,\}202of the backbone’s2,529,778,1622\{,\}529\{,\}778\{,\}162parameters,1\.564%1\.564\\%\. Both models are evaluated at step 2424, OmniMed\-Jev’s best\-validation checkpoint; full hyperparameters and hardware are in Appendix[E](https://arxiv.org/html/2610.00381#A5)\.

Two further differences follow from the interface rather than from tuning: because annotations are factorized into one target per decision, OmniMed\-Jev sees more supervision per image than a baseline that emits a single answer, and its decision head is an additional trained component\. Appendix[I](https://arxiv.org/html/2610.00381#A9)lists them with the other respects in which the two models are not like\-for\-like\.

### 4\.3\. Metrics

Four families of metrics are reported, each measuring a property a medical decision rule consumes: calibration, the separation of reliability from resolution, selective behaviour, and decision\-space coverage\. Their definitions are in Appendix[C](https://arxiv.org/html/2610.00381#A3)and, for the quantities of Table[2](https://arxiv.org/html/2610.00381#S5.T2), in Section[5\.2](https://arxiv.org/html/2610.00381#S5.SS2)\.

### 4\.4\. Recovering probabilities from the generative baseline

A generative model does not emit a distribution over candidates, so the comparison requires a reconstruction\. We render the same decision block for the baseline, score each candidate by the summed log\-probability of its continuation under the language model, and normalize over the candidate set,

p~​\(cj​k\)=exp∑tlogpLM\(y\(j​k\)t∣y\(j​k\)<t,S,Dj\)∑lexp∑tlogpLM\(y\(j​l\)t∣y\(j​l\)<t,S,Dj\),\\tilde\{p\}\(c\_\{jk\}\)=\\frac\{\\exp\\sum\_\{t\}\\log p\_\{\\mathrm\{LM\}\}\\\!\\left\(y^\{\(jk\)\}\_\{t\}\\mid y^\{\(jk\)\}\_\{<t\},S,D\_\{j\}\\right\)\}\{\\sum\_\{l\}\\exp\\sum\_\{t\}\\log p\_\{\\mathrm\{LM\}\}\\\!\\left\(y^\{\(jl\)\}\_\{t\}\\mid y^\{\(jl\)\}\_\{<t\},S,D\_\{j\}\\right\)\},\(5\)whereyt\(j​k\)y^\{\(jk\)\}\_\{t\}denotes the tokens of candidatecj​kc\_\{jk\}in the same decision block, so a candidate is scored as a whole continuation rather than by its opening token\. The reconstruction is generous, since the baseline receives the same candidate set and prompt as OmniMed\-Jev, and it remains a reconstruction: the continuation need not be the text the model would have produced\. Appendix[F](https://arxiv.org/html/2610.00381#A6)gives the estimator, the alternatives we tried and its failure modes\.

## 5\. Results

### 5\.1\. Accuracy is not where the interface changes the model

Table[1](https://arxiv.org/html/2610.00381#S3.T1)reports the task metrics\. The two models land within a few points of each other on every family, and the differences that appear to favour decision modeling do not exclude a null difference: the classification, multi\-label and regression gaps are not distinguishable from zero at95%95\\%, the multi\-label ordering reverses under pooled micro averaging, and the tally is checkpoint\-dependent\. The one difference distinguishable from zero is the counting MAE, and it favours the generative baseline\. The point\-prediction comparison therefore establishes no advantage for either representation, which is exactly the result the rest of the paper needs: the question is not whether the decision interface predicts better, but what the probabilities it reports are worth\.

Figure 2:Learning dynamics across four task families\.All ten matched test checkpoints are shown; the dashed line and enlarged markers mark the comparison step\. Counting and regression use logarithmic MAE axes to retain early outliers without hiding later differences\. The interfaces differ in trajectory as well as in endpoint: the generative baseline leads early in classification and is overtaken after step 1616, while in regression it passes through a large mid\-training excursion that OmniMed\-Jev does not exhibit\.
### 5\.2\. Calibration and discrimination

The reported probability in this section is the probability that the model’s top candidate is correct: of the class it selects for a Choice decision, of the finding it considers present for a Noul decision, and of the annotated value for a Score decision\. It is a probability about the decision, not the probability that a disease is present or that a measurement is clinically acceptable; those are different quantities, defined by a downstream rule, and the metrics below measure the one the model reports\. Making that distribution a native output makes calibration directly trainable and measurable rather than a post\-hoc repair, and the answer to whether it is in fact calibrated is the central empirical claim of this paper\.

Calibration\.Expected calibration error bins decisions by reported confidence and averages the gap between confidence and accuracy,

ECE=∑b=1Bnbn​\|acc⁡\(b\)−conf⁡\(b\)\|,\\mathrm\{ECE\}=\\sum\_\{b=1\}^\{B\}\\frac\{n\_\{b\}\}\{n\}\\,\\bigl\|\\mathrm\{acc\}\(b\)\-\\mathrm\{conf\}\(b\)\\bigr\|,\(6\)where a decision’s confidence is the probability of its top candidate\. The quantity answers the question a clinician asks of a reported probability: when the model says it is right with probabilitypp, is it right aboutppof the time? The generative baseline is badly miscalibrated everywhere: its stated confidence is off by0\.260\.26on classification,0\.300\.30on multi\-label recognition,0\.420\.42on counting and0\.130\.13on regression, so a threshold placed on that number is not a probability that the decision is correct\. OmniMed\-Jev cuts the error to0\.080\.08,0\.010\.01,0\.040\.04and0\.090\.09in the same four families, a reduction of3×3\\timesto26×26\\times, and its reported probabilities sit close to the rate at which its decisions are actually correct\.This is the paper’s central empirical claim: the same backbone, trained to output the decision distribution instead of the answer string, reports numbers that mean what they say\.

Reliability versus resolution\.The Brier score decomposes into a calibration term, a discrimination term and an irreducible constant,

BS=∑bnbn​\(confb−accb\)2⏟reliability−∑bnbn​\(accb−y¯\)2⏟resolution\+y¯​\(1−y¯\)⏟uncertainty,\\mathrm\{BS\}=\\underbrace\{\\sum\_\{b\}\\tfrac\{n\_\{b\}\}\{n\}\\bigl\(\\mathrm\{conf\}\_\{b\}\-\\mathrm\{acc\}\_\{b\}\\bigr\)^\{2\}\}\_\{\\text\{reliability\}\}\-\\underbrace\{\\sum\_\{b\}\\tfrac\{n\_\{b\}\}\{n\}\\bigl\(\\mathrm\{acc\}\_\{b\}\-\\bar\{y\}\\bigr\)^\{2\}\}\_\{\\text\{resolution\}\}\+\\underbrace\{\\bar\{y\}\(1\-\\bar\{y\}\)\}\_\{\\text\{uncertainty\}\},\(7\)withy¯\\bar\{y\}the base correctness rate\. Reliability error asks whether the reported numbers are probabilities; resolution asks whether the ranking separates correct from incorrect decisions\. The two move differently, and both directions have to be reported\. The generative baseline passes the second test more often than the first: its reliability error exceeds its resolution in classification, multi\-label recognition and counting \(Figure[3](https://arxiv.org/html/2610.00381#S5.F3)\), so it orders cases usefully while its reported numbers overstate how often it is right\. OmniMed\-Jev inverts the pattern decisively, cutting reliability error by9×9\\timeson classification,44×44\\timeson counting and126×126\\timeson multi\-label recognition while resolution stays comparable in classification and regression: the same ordering is expressed in a quantity a downstream rule can consume\.

Table 2:Decision quality of the two models\. Each cell gives Gen\-SFT / OmniMed\-Jev; lower is better for ECE, reliability and AURC, higher for resolution\. Reliability and resolution are the Murphy decomposition of the top\-label decision\. The generative baseline’s distribution is the reconstruction of Eq\. \([5](https://arxiv.org/html/2610.00381#S4.E5)\) rather than a trained output, and OmniMed\-Jev’s values are from the same step\. Counting is not a like\-for\-like sample size; Section[5\.5](https://arxiv.org/html/2610.00381#S5.SS5)reports the matched comparison\.Figure 3:Break\-even boundary for top\-label probability skill\.The dashed boundary isresolution=reliability\\text\{resolution\}=\\text\{reliability\}in the Murphy decomposition; green marksresolution\>reliability\\text\{resolution\}\>\\text\{reliability\}and pale rust the opposite\. All four OmniMed\-Jev families are on the green side; the generative baseline crosses to the rust side in classification, multi\-label recognition and counting\. The boundary is relative to each model’s own top\-label correctness rate, not a clinical decision threshold\.The multi\-label family is where the interface pays off most, because there the decisions are many and independent: reliability error falls from0\.12940\.1294to0\.00100\.0010, a factor of126126, and the area under the selective risk curve from0\.24740\.2474to0\.01260\.0126, a factor of2020\. A radiograph carries fourteen presence decisions, and OmniMed\-Jev’s per\-finding probabilities are close to the rate at which each finding is correct, so a rule that escalates a finding above a fixed probability now behaves the way its specification says\. Its ranking of findings is flatter than the baseline’s \(0\.07760\.0776against0\.01020\.0102\), which costs nothing operationally here: the baseline’s ordering is expressed in numbers that overstate correctness by a quarter, and its selective risk is twenty times higher as a result\. Regression is the family where the interface adds least, improving calibration \(0\.12670\.1267to0\.09130\.0913\) without changing selective behaviour\.

### 5\.3\. Does confidence identify errors?

Selective prediction tests whether the reported confidence can find the errors: ordering decisions by confidence and answering only the most confident fraction gives a risk\-coverage curve, whose error at coverageccand area are

risk⁡\(c\)=error among the​c​most confident decisions,AURC=∫01risk⁡\(c\)​𝑑c,\\mathrm\{risk\}\(c\)=\\text\{error among the \}c\\text\{ most confident decisions\},\\qquad\\mathrm\{AURC\}=\\int\_\{0\}^\{1\}\\mathrm\{risk\}\(c\)\\,dc,\(8\)and the quantity a deferral policy consumes is whether low confidence marks the decisions worth handing to a clinician at the stated error level\.

Ranking by OmniMed\-Jev’s probabilities and retaining the most confident half leaves a much lower error rate than ranking by the baseline’s reconstructed confidence at the same coverage, and the gap widens as coverage increases \(Figure[4](https://arxiv.org/html/2610.00381#S5.F4)\); on multi\-label recognition the area under the curve falls from0\.24740\.2474to0\.01260\.0126, a twenty\-fold reduction in the error a deferral policy carries at every coverage level\.

Counting is the case in which the curve measures the reconstruction rather than the interface\. The baseline’s reconstructed distribution places no count on the annotated value in any of the126126counting decisions, so its ordering is uninformative and its curve is flat at an error of1\.01\.0, which the AURC value follows from\. We read this as a failure of the reconstruction when the answer is a value on a large ordered grid, not as evidence that a generative model cannot order counting decisions at all\.

Figure 4:Selective prediction at reported coverage levels\.Points show top\-label error when the model answers the most confident 50%, 80%, 90%, 95% and 100% of decisions; lines connect only those reported aggregates, not a reconstructed full curve\. Panel AURC pairs are OmniMed\-Jev / Gen\-SFT\. For counting the baseline’s reconstruction orders no decision correctly, so its curve is flat at an error of1\.01\.0, which reflects the reconstruction of Section[4\.4](https://arxiv.org/html/2610.00381#S4.SS4)rather than generative confidence in general\.The same distributions support shortlists\.DDx​@​0\.9\\mathrm\{DDx\}@0\.9is the number of top\-ranked candidates needed to carry0\.90\.9of the probability mass, and it shrinks in every family \(Appendix[D](https://arxiv.org/html/2610.00381#A4)\), which says that the distribution is more concentrated; whether the annotated candidate lies inside a shortlist of a given size is a separate quantity, and the two should not be conflated\. For a differential diagnosis the difference is between a bounded shortlist whose length is part of the model’s output and a ranked list whose length the clinician has to choose\.

### 5\.4\. Manipulating the decision space at inference time

Because candidates are input semantics, the decision space can be manipulated at inference time without retraining, and the effect depends on the family \(Table[6](https://arxiv.org/html/2610.00381#A4.T6)\): permuting the candidate order leaves classification answers almost unchanged \(top\-1 agreement0\.900\.90\) while counting and regression answers change substantially \(0\.420\.42and0\.670\.67\), consistent with a model sensitive to the semantics of the options it reads rather than to their positions\. Restricting the candidate set to a short sub\-list leaves accuracy far above chance \(0\.870\.87for a two\-option list against a chance level of0\.500\.50\), and removing the annotated candidate makes the model select a plausible runner\-up \(0\.880\.88\) rather than collapse\. None of these operations has a counterpart under the generative interface without adding a separate scoring mechanism \(Appendix[D](https://arxiv.org/html/2610.00381#A4)\)\.

### 5\.5\. The counting boundary

Counting is the family where the generative baseline is better, and the reason is specific to the Score design\. A count is a bounded measurement whose annotation is a single integer; OmniMed\-Jev represents it as a distribution over an ordered grid and folds it to the expectation, while the baseline emits the integer directly, and its point estimates are closer on average, the only task\-level difference distinguishable from zero at95%95\\%\(Table[1](https://arxiv.org/html/2610.00381#S3.T1)\)\. The likely mechanism is the folding: an expectation over a wide grid shrinks extreme values towards its middle, so a model uncertain across the upper range reports a value closer to the centre\. We state this as a hypothesis about the Score representation as implemented here; it is the only family in the suite where the probability object costs accuracy, and the mechanism is specific enough to be fixed rather than fundamental\.

## 6\. Discussion

What the decision interface contributes\.The three decision types let heterogeneous medical outputs be expressed as explicit decisions: a class, a finding and a grid value are all candidate sets with a distribution over them, so the diagnosis, the findings and the counts in one report are probabilities of the same kind\. The experiments show that these probabilities correspond far more closely to observed correctness, by an order of magnitude or more in three of the four families, and that they support selective behaviour the reconstruction cannot; no advantage in prediction is claimed, and none is needed\.OmniMed\-Jev does not primarily make the model more accurate; it makes the model faithful about how likely its decisions are to be correct\.The interface adds no information; it changes the object the model is trained to report, and that object is the one a decision rule consumes\.

What medicine adds to the idea\.Applying a decision interface in medicine is not a matter of porting one: a generalist model has to accept different modalities, cover different tasks, and express them in one comparable probability object, which is what we mean by omni\. Two design commitments follow: several decisions can share one state, and candidate semantics belong in the input rather than in a fixed output layer\. The contribution is therefore medical rather than architectural: the interface had to be extended to cover modalities and tasks, and medicine is where that extension is load\-bearing\.

The scope of the claim\.The comparison cannot separate the interface from the denser supervision and the additional decision head that come with it, so what we measure is decision modeling as implemented rather than the interface alone\. The probabilities are evaluated retrospectively on held\-out data from the same sources as training, and per\-class, per\-source and out\-of\-distribution behaviour is not measured\. Post\-hoc calibration remains complementary: it repairs a number the model already emits, whereas the interface makes the decision an input\.

## 7\. Conclusion

We asked whether a generalist medical multimodal model can report confidence that corresponds to how often it is right, and whether making the decision its output object is the way to achieve it\. Under an interface\-controlled comparison the answer is yes on both counts: the reported probabilities track observed correctness far more closely, by up to two orders of magnitude in the reliability term, while point\-prediction performance stays comparable and counting remains the boundary case\. Generalist medical models should therefore be omni not only in what they perceive and solve, but also in how faithfully they quantify their own decisions\.

## References

- A\. N\. Angelopoulos and S\. BatesConformal Prediction: A Gentle Introduction\.Foundations and Trends in Machine Learning16\(4\),pp\. 494–591\.External Links:[Document](https://dx.doi.org/10.1561/2200000101)Cited by:[§2\.3](https://arxiv.org/html/2610.00381#S2.SS3.p1.1)\.
- Aroraet al\.\(2025\)R\. K\. Arora, J\. Wei, R\. Soskin Hicks, P\. Bowman, J\. Quiñonero\-Candela, F\. Tsimpourlas, M\. Sharman, M\. Shah, A\. Vallone, A\. Beutel, J\. Heidecke, and K\. SinghalHealthBench: Evaluating Large Language Models Towards Improved Human Health\.External Links:2505\.08775,[Link](https://arxiv.org/abs/2505.08775)Cited by:[§2\.1](https://arxiv.org/html/2610.00381#S2.SS1.p1.1)\.
- Banerjiet al\.\(2023\)C\. R\. S\. Banerji, T\. Chakraborti, C\. Harbron, and B\. D\. MacArthurClinical AI tools must convey predictive uncertainty for each individual patient\.Nature Medicine29\(12\),pp\. 2996–2998\.External Links:[Document](https://dx.doi.org/10.1038/s41591-023-02562-7)Cited by:[§H\.1](https://arxiv.org/html/2610.00381#A8.SS1.p1.1)\.
- Bani\-Harouniet al\.\(2026\)D\. Bani\-Harouni, C\. Pellegrini, P\. Stangel, E\. Özsoy, K\. Zaripova, N\. Navab, and M\. KeicherRewarding Doubt: A Reinforcement Learning Approach to Calibrated Confidence Expression of Large Language Models\.InInternational Conference on Learning Representations \(ICLR\),External Links:2503\.02623,[Link](https://openreview.net/forum?id=yResLmrVO1)Cited by:[§H\.3](https://arxiv.org/html/2610.00381#A8.SS3.p1.1),[§2\.2](https://arxiv.org/html/2610.00381#S2.SS2.p1.1)\.
- Bannuret al\.\(2024\)S\. Bannur, K\. Bouzid, D\. C\. Castro, A\. Schwaighofer, A\. Thieme, S\. Bond\-Taylor,et al\.MAIRA\-2: Grounded Radiology Report Generation\.External Links:2406\.04449,[Link](https://arxiv.org/abs/2406.04449)Cited by:[§2\.1](https://arxiv.org/html/2610.00381#S2.SS1.p1.1)\.
- Bediet al\.\(2025\)S\. Bedi, Y\. Liu, L\. Orr\-Ewing, D\. Dash, S\. Koyejo, A\. Callahan, J\. A\. Fries, M\. Wornow, A\. Swaminathan, L\. S\. Lehmann, H\. J\. Hong, M\. Kashyap, A\. R\. Chaurasia, N\. R\. Shah, K\. Singh, T\. Tazbaz, A\. Milstein, M\. A\. Pfeffer, and N\. H\. ShahTesting and Evaluation of Health Care Applications of Large Language Models\.JAMA333\(4\),pp\. 319–328\.External Links:[Document](https://dx.doi.org/10.1001/jama.2024.21700)Cited by:[§H\.4](https://arxiv.org/html/2610.00381#A8.SS4.p2.1)\.
- Bentegeacet al\.\(2025\)R\. Bentegeac, B\. Le Guellec, G\. Kuchcinski, P\. Amouyel, and A\. HamrounToken Probabilities to Mitigate Large Language Models Overconfidence in Answering Medical Questions: Quantitative Study\.Journal of Medical Internet Research27,pp\. e64348–e64348\.External Links:[Document](https://dx.doi.org/10.2196/64348)Cited by:[§1](https://arxiv.org/html/2610.00381#S1.p2.1),[§2\.2](https://arxiv.org/html/2610.00381#S2.SS2.p1.1)\.
- Boieet al\.\(2026\)S\. D\. Boie, F\. Reis, N\. Frey, E\. Grünewald, and F\. BalzerCalibration of Self\-Reported Confidence and Accuracy of Large Language Models in Medical Question Answering\.Journal of Medical Systems50\(1\),pp\. 103\.External Links:[Document](https://dx.doi.org/10.1007/s10916-026-02430-0)Cited by:[§H\.1](https://arxiv.org/html/2610.00381#A8.SS1.p1.1)\.
- BRIER \(1950\)G\. W\. BRIERVERIFICATION OF FORECASTS EXPRESSED IN TERMS OF PROBABILITY\.Monthly Weather Review78\(1\),pp\. 1–3\.External Links:[Document](https://dx.doi.org/10.1175/1520-0493%281950%29078%3C0001%3Avofeit%3E2.0.co%3B2)Cited by:[§H\.1](https://arxiv.org/html/2610.00381#A8.SS1.p1.1)\.
- Byunet al\.\(2026\)J\. Y\. Byun, Y\. Park, J\. Corbeil, and A\. Ben AbachaOverconfidence and Calibration in Medical VQA: Empirical Findings and Hallucination\-Aware Mitigation\.External Links:2604\.02543,[Link](https://arxiv.org/abs/2604.02543)Cited by:[§1](https://arxiv.org/html/2610.00381#S1.p2.1),[§2\.2](https://arxiv.org/html/2610.00381#S2.SS2.p1.1)\.
- Chenet al\.\(2024\)Z\. Chen, M\. Varma, J\. Xu, M\. Paschali, D\. V\. Veen, A\. Johnston,et al\.A Vision\-Language Foundation Model to Enhance Efficiency of Chest X\-ray Interpretation\.External Links:2401\.12208,[Link](https://arxiv.org/abs/2401.12208)Cited by:[§2\.1](https://arxiv.org/html/2610.00381#S2.SS1.p1.1)\.
- Chenget al\.\(2026\)Z\. Cheng, W\. Dai, and J\. Sunthis\-that\-model\-1\.0: A typed decision model that decides in 30 ms, for a millionth of a cent\.External Links:2609\.23886,[Link](https://arxiv.org/abs/2609.23886)Cited by:[§2\.3](https://arxiv.org/html/2610.00381#S2.SS3.p1.1)\.
- Chow \(1970\)C\. ChowOn optimum recognition error and reject tradeoff\.IEEE Transactions on Information Theory16\(1\),pp\. 41–46\.External Links:[Document](https://dx.doi.org/10.1109/tit.1970.1054406)Cited by:[§2\.3](https://arxiv.org/html/2610.00381#S2.SS3.p1.1)\.
- Cocchieriet al\.\(2026\)A\. Cocchieri, L\. Ragazzi, G\. Tagliavini, and G\. MoroLLMs \(Almost\) Never Abstain Under Medical Uncertainty\.InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics \(ACL\),pp\. 29573–29613\.External Links:[Document](https://dx.doi.org/10.18653/v1/2026.acl-long.1365)Cited by:[§H\.4](https://arxiv.org/html/2610.00381#A8.SS4.p1.1),[§2\.3](https://arxiv.org/html/2610.00381#S2.SS3.p1.1)\.
- Collinset al\.\(2024\)G\. S\. Collins, K\. G\. M\. Moons, P\. Dhiman, R\. D\. Riley, A\. L\. Beam, B\. Van Calster, M\. Ghassemi, X\. Liu, J\. B\. Reitsma, M\. van Smeden, A\. Boulesteix, J\. C\. Camaradou, L\. A\. Celi, S\. Denaxas, A\. K\. Denniston, B\. Glocker, R\. M\. Golub, H\. Harvey, G\. Heinze, M\. M\. Hoffman, A\. P\. Kengne, E\. Lam, N\. Lee, E\. W\. Loder, L\. Maier\-Hein, B\. A\. Mateen, M\. D\. McCradden, L\. Oakden\-Rayner, J\. Ordish, R\. Parnell,et al\.TRIPOD\+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods\.BMJ385,pp\. e078378\.External Links:[Document](https://dx.doi.org/10.1136/bmj-2023-078378)Cited by:[§H\.4](https://arxiv.org/html/2610.00381#A8.SS4.p2.1)\.
- Dağlı \(2026\)V\. DağlıUniversal Fractal Natural Language Decision Map: Real\-Time Edge Triage Across Heterogeneous Domains\.External Links:2609\.25498,[Link](https://arxiv.org/abs/2609.25498)Cited by:[Appendix H](https://arxiv.org/html/2610.00381#A8.p1.1)\.
- Damaniet al\.\(2026\)M\. Damani, I\. Puri, S\. Slocum, I\. Shenfeld, L\. Choshen, Y\. Kim, and J\. AndreasBeyond Binary Rewards: Training LMs to Reason About Their Uncertainty\.InInternational Conference on Learning Representations \(ICLR\),External Links:2507\.16806,[Link](https://openreview.net/forum?id=ASQ649zdHm)Cited by:[§2\.2](https://arxiv.org/html/2610.00381#S2.SS2.p1.1)\.
- Denget al\.\(2026\)B\. Deng, S\. Fan, H\. Zhang, and X\. XieJev for Scientific Decisions: Evaluating Semantic Choices and Their Consequences\.External Links:2609\.24965,[Link](https://arxiv.org/abs/2609.24965)Cited by:[Appendix H](https://arxiv.org/html/2610.00381#A8.p1.1)\.
- Dratschet al\.\(2023\)T\. Dratsch, X\. Chen, M\. Rezazade Mehrizi, R\. Kloeckner, A\. Mähringer\-Kunz, M\. Püsken, B\. Baeßler, S\. Sauer, D\. Maintz, and D\. Pinto dos SantosAutomation Bias in Mammography: The Impact of Artificial Intelligence BI\-RADS Suggestions on Reader Performance\.Radiology307\(4\),pp\. e222176\.External Links:[Document](https://dx.doi.org/10.1148/radiol.222176)Cited by:[§H\.4](https://arxiv.org/html/2610.00381#A8.SS4.p1.1),[Appendix I](https://arxiv.org/html/2610.00381#A9.SS0.SSS0.Px3.p1.1)\.
- Duet al\.\(2025\)Y\. Du, Y\. Wang, M\. Kong, T\. Liang, Q\. Long, B\. Chen, and Q\. ZhuConfidence Calibration for Multimodal LLMs: An Empirical Study Through Medical VQA\.InMedical Image Computing and Computer Assisted Intervention – MICCAI 2025,pp\. 89–99\.External Links:[Document](https://dx.doi.org/10.1007/978-3-032-04978-0%5F9)Cited by:[§2\.2](https://arxiv.org/html/2610.00381#S2.SS2.p1.1)\.
- ECRI \(2026\)ECRITop 10 Health Technology Hazards for 2026\.External Links:[Link](https://home.ecri.org/blogs/ecri-thought-leadership-resources/top-10-health-technology-hazards-for-2026-executive-brief)Cited by:[§H\.4](https://arxiv.org/html/2610.00381#A8.SS4.p2.1)\.
- European Parliament and Council of the European Union \(2024\)European Parliament and Council of the European UnionRegulation \(EU\) 2024/1689 of the European Parliament and of the Council of 13 June 2024 Laying Down Harmonised Rules on Artificial Intelligence \(Artificial Intelligence Act\), Article 14\.External Links:[Link](https://artificialintelligenceact.eu/article/14/)Cited by:[§H\.4](https://arxiv.org/html/2610.00381#A8.SS4.p2.1)\.
- Fanconi and van der Schaar \(2025\)C\. Fanconi and M\. van der SchaarCascaded Language Models for Cost\-effective Human\-AI Decision\-Making\.External Links:2506\.11887,[Link](https://arxiv.org/abs/2506.11887)Cited by:[§H\.4](https://arxiv.org/html/2610.00381#A8.SS4.p1.1),[§2\.3](https://arxiv.org/html/2610.00381#S2.SS3.p1.1)\.
- Gallifantet al\.\(2025\)J\. Gallifant, M\. Afshar, S\. Ameen, Y\. Aphinyanaphongs, S\. Chen, G\. Cacciamani, D\. Demner\-Fushman, D\. Dligach, R\. Daneshjou, C\. Fernandes, L\. H\. Hansen, A\. Landman, L\. Lehmann, L\. G\. McCoy, T\. Miller, A\. Moreno, N\. Munch, D\. Restrepo, G\. Savova, R\. Umeton, J\. W\. Gichoya, G\. S\. Collins, K\. G\. M\. Moons, L\. A\. Celi, and D\. S\. BittermanThe TRIPOD\-LLM reporting guideline for studies using large language models\.Nature Medicine31\(1\),pp\. 60–69\.External Links:[Document](https://dx.doi.org/10.1038/s41591-024-03425-5)Cited by:[§H\.4](https://arxiv.org/html/2610.00381#A8.SS4.p2.1)\.
- Geifman and El\-Yaniv \(2017\)Y\. Geifman and R\. El\-YanivSelective Classification for Deep Neural Networks\.InAdvances in Neural Information Processing Systems 30,External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2017/hash/4a8423d5e91fda00bb7e46540e2b0cf1-Abstract.html)Cited by:[§H\.4](https://arxiv.org/html/2610.00381#A8.SS4.p1.1),[§2\.3](https://arxiv.org/html/2610.00381#S2.SS3.p1.1)\.
- Google \(2026\)GoogleMedGemma 1\.5 model card\.Note:Hugging Face model cardExternal Links:[Link](https://huggingface.co/google/medgemma-1.5-4b-it)Cited by:[§1](https://arxiv.org/html/2610.00381#S1.p2.1),[§2\.1](https://arxiv.org/html/2610.00381#S2.SS1.p1.1)\.
- Griotet al\.\(2025\)M\. Griot, C\. Hemptinne, J\. Vanderdonckt, and D\. YukselLarge Language Models lack essential metacognition for reliable medical reasoning\.Nature Communications16\(1\),pp\. 642\.External Links:[Document](https://dx.doi.org/10.1038/s41467-024-55628-6)Cited by:[§H\.1](https://arxiv.org/html/2610.00381#A8.SS1.p1.1)\.
- Guoet al\.\(2017\)C\. Guo, G\. Pleiss, Y\. Sun, and K\. Q\. WeinbergerOn Calibration of Modern Neural Networks\.InProceedings of the 34th International Conference on Machine Learning,pp\. 1321–1330\.External Links:[Link](https://proceedings.mlr.press/v70/guo17a.html)Cited by:[§2\.2](https://arxiv.org/html/2610.00381#S2.SS2.p1.1)\.
- Guoet al\.\(2026\)K\. H\. Guo, C\. Yan, A\. Baidya, K\. Brown, X\. Gao, J\. Xiong, Z\. Yin, and B\. A\. MalinCLEAR: Revealing How Noise and Ambiguity Degrade Reliability in LLMs for Medicine\.External Links:2605\.01011,[Link](https://arxiv.org/abs/2605.01011)Cited by:[§H\.1](https://arxiv.org/html/2610.00381#A8.SS1.p1.1)\.
- Huet al\.\(2026\)T\. Hu, B\. Minixhofer, and N\. CollierNavigating the Alignment\-Calibration Trade\-off: A Pareto\-Superior Frontier via Model Merging\.InFindings of the Association for Computational Linguistics: ACL 2026,pp\. 42405–42422\.External Links:[Document](https://dx.doi.org/10.18653/v1/2026.findings-acl.2104)Cited by:[§H\.2](https://arxiv.org/html/2610.00381#A8.SS2.p1.1)\.
- Huanget al\.\(2026\)J\. Huang, H\. Yang, X\. Ma, X\. Liang, K\. Cai, J\. Ma, S\. Chen, Y\. Sun, and T\. TanCan Jev Judge Radiology Reports? Evaluating a System One Model for Clinical Factuality\.External Links:2609\.27607,[Link](https://arxiv.org/abs/2609.27607)Cited by:[§2\.3](https://arxiv.org/html/2610.00381#S2.SS3.p1.1)\.
- Ibrahim and Zaki \(2026\)H\. Ibrahim and Y\. ZakiEvaluating Decision Models for Text Annotation in Computational Social Science\.External Links:2609\.24574,[Link](https://arxiv.org/abs/2609.24574)Cited by:[§2\.3](https://arxiv.org/html/2610.00381#S2.SS3.p1.1)\.
- Jabbouret al\.\(2025\)S\. Jabbour, D\. Fouhey, N\. Banovic, S\. D\. Shepard, E\. Kazerooni, M\. W\. Sjoding, and J\. WiensSelective Prediction Reduces the Negative Effects of Automation Bias Overall but Increases False Negatives\.External Links:2508\.07617,[Link](https://arxiv.org/abs/2508.07617)Cited by:[§H\.4](https://arxiv.org/html/2610.00381#A8.SS4.p1.1),[§1](https://arxiv.org/html/2610.00381#S1.p2.1),[§2\.3](https://arxiv.org/html/2610.00381#S2.SS3.p1.1)\.
- Jabbouret al\.\(2023\)S\. Jabbour, D\. Fouhey, S\. Shepard, T\. S\. Valley, E\. A\. Kazerooni, N\. Banovic, J\. Wiens, and M\. W\. SjodingMeasuring the Impact of AI in the Diagnosis of Hospitalized Patients\.JAMA330\(23\),pp\. 2275–2284\.External Links:[Document](https://dx.doi.org/10.1001/jama.2023.22295)Cited by:[§H\.4](https://arxiv.org/html/2610.00381#A8.SS4.p1.1),[Appendix I](https://arxiv.org/html/2610.00381#A9.SS0.SSS0.Px3.p1.1)\.
- Jianget al\.\(2026\)D\. Jiang, Y\. Li, and B\. LiJev\-Mem: System\-One\-Controlled Agentic Memory for Efficient AI Agents\.External Links:2609\.23986,[Link](https://arxiv.org/abs/2609.23986)Cited by:[§2\.3](https://arxiv.org/html/2610.00381#S2.SS3.p1.1)\.
- Jinet al\.\(2021\)D\. Jin, E\. Pan, N\. Oufattole, W\. Weng, H\. Fang, and P\. SzolovitsWhat Disease Does This Patient Have? A Large\-Scale Open Domain Question Answering Dataset from Medical Exams\.Applied Sciences11\(14\),pp\. 6421\.External Links:[Document](https://dx.doi.org/10.3390/app11146421)Cited by:[§2\.1](https://arxiv.org/html/2610.00381#S2.SS1.p1.1)\.
- Kadavathet al\.\(2022\)S\. Kadavath, T\. Conerly, A\. Askell, T\. Henighan, D\. Drain, E\. Perez,et al\.Language Models \(Mostly\) Know What They Know\.External Links:2207\.05221,[Link](https://arxiv.org/abs/2207.05221)Cited by:[§H\.3](https://arxiv.org/html/2610.00381#A8.SS3.p1.1)\.
- Kahneman \(2011\)D\. KahnemanThinking, Fast and Slow\.Farrar, Straus and Giroux,New York\.Cited by:[§2\.3](https://arxiv.org/html/2610.00381#S2.SS3.p1.1)\.
- Kapooret al\.\(2024\)S\. Kapoor, N\. Gruver, M\. Roberts, K\. Collins, A\. Pal, U\. Bhatt, A\. Weller, S\. Dooley, M\. Goldblum, and A\. G\. WilsonLarge Language Models Must Be Taught to Know What They Don’t Know\.InAdvances in Neural Information Processing Systems 37,External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2024/hash/9c20f16b05f5e5e70fa07e2a4364b80e-Abstract-Conference.html)Cited by:[§H\.4](https://arxiv.org/html/2610.00381#A8.SS4.p2.1)\.
- Kimet al\.\(2024\)S\. S\. Y\. Kim, Q\. V\. Liao, M\. Vorvoreanu, S\. Ballard, and J\. W\. Vaughan"I’m Not Sure, But…": Examining the Impact of Large Language Models’ Uncertainty Expression on User Reliance and Trust\.InProceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency \(FAccT\),pp\. 822–835\.External Links:[Document](https://dx.doi.org/10.1145/3630106.3658941)Cited by:[§H\.4](https://arxiv.org/html/2610.00381#A8.SS4.p1.1),[Appendix I](https://arxiv.org/html/2610.00381#A9.SS0.SSS0.Px3.p1.1)\.
- Kimet al\.\(2026\)Y\. Kim, H\. Kim, M\. Bahl, M\. H\. Lev, R\. G\. González, M\. S\. Gee, and S\. DoDefining operational safety in clinical artificial intelligence systems\.npj Digital Medicine9\(1\),pp\. 281\.External Links:[Document](https://dx.doi.org/10.1038/s41746-026-02450-7)Cited by:[§H\.4](https://arxiv.org/html/2610.00381#A8.SS4.p2.1)\.
- Kimet al\.\(2025\)Y\. Kim, H\. Jeong, S\. Chen, S\. S\. Li, C\. Park, M\. Lu,et al\.Medical Hallucinations in Foundation Models and Their Impact on Healthcare\.External Links:2503\.05777,[Link](https://arxiv.org/abs/2503.05777)Cited by:[§H\.1](https://arxiv.org/html/2610.00381#A8.SS1.p1.1)\.
- Kuhnet al\.\(2023\)L\. Kuhn, Y\. Gal, and S\. FarquharSemantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation\.InInternational Conference on Learning Representations \(ICLR\),External Links:2302\.09664,[Link](https://arxiv.org/abs/2302.09664)Cited by:[§H\.3](https://arxiv.org/html/2610.00381#A8.SS3.p1.1)\.
- Lauet al\.\(2018\)J\. J\. Lau, S\. Gayen, A\. Ben Abacha, and D\. Demner\-FushmanA dataset of clinically generated visual questions and answers about radiology images\.Scientific Data5\(1\),pp\. 180251\.External Links:[Document](https://dx.doi.org/10.1038/sdata.2018.251)Cited by:[§2\.1](https://arxiv.org/html/2610.00381#S2.SS1.p1.1)\.
- Lechuga Lopezet al\.\(2026\)L\. J\. Lechuga Lopez, F\. E\. Shamout, and T\. G\. J\. RudnerAn Empirical Analysis of Calibration and Selective Prediction in Multimodal Clinical Condition Classification\.InProceedings of the 7th Conference on Health, Inference, and Learning,pp\. 794–833\.External Links:[Link](https://proceedings.mlr.press/v333/lopez26a.html)Cited by:[§H\.4](https://arxiv.org/html/2610.00381#A8.SS4.p1.1)\.
- Lenget al\.\(2025\)J\. Leng, C\. Huang, B\. Zhu, and J\. HuangTaming Overconfidence in LLMs: Reward Calibration in RLHF\.InInternational Conference on Learning Representations \(ICLR\),External Links:2410\.09724,[Link](https://openreview.net/forum?id=l0tg0jzsdL)Cited by:[§H\.3](https://arxiv.org/html/2610.00381#A8.SS3.p1.1)\.
- Liet al\.\(2023\)C\. Li, C\. Wong, S\. Zhang, N\. Usuyama, H\. Liu, J\. Yang, T\. Naumann, H\. Poon, and J\. GaoLLaVA\-Med: Training a Large Language\-and\-Vision Assistant for Biomedicine in One Day\.InAdvances in Neural Information Processing Systems 36 \(Datasets and Benchmarks Track\),External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2023/hash/5abcdf8ecdcacba028c6662789194572-Abstract-Datasets_and_Benchmarks.html)Cited by:[§1](https://arxiv.org/html/2610.00381#S1.p2.1),[§2\.1](https://arxiv.org/html/2610.00381#S2.SS1.p1.1)\.
- Liet al\.\(2026a\)D\. Li, X\. Wang, H\. Gong, R\. Lang, and G\. YuFast Intent\-Driven Service Orchestration with Jev for 6G Edge Networks\.External Links:2609\.23136,[Link](https://arxiv.org/abs/2609.23136)Cited by:[Appendix H](https://arxiv.org/html/2610.00381#A8.p1.1)\.
- Liet al\.\(2026b\)D\. Li, X\. Wang, H\. Gong, R\. Lang, and G\. YuReplacing Large Language Models with Jev Decision Models for Low\-Latency Edge Service Orchestration\.External Links:2609\.22753,[Link](https://arxiv.org/abs/2609.22753)Cited by:[Appendix H](https://arxiv.org/html/2610.00381#A8.p1.1)\.
- Li \(2026\)H\. LiKITE: Scaling Jev Population Experiments with Sparse Flagship Calibration\.External Links:2609\.27535,[Link](https://arxiv.org/abs/2609.27535)Cited by:[Appendix H](https://arxiv.org/html/2610.00381#A8.p1.1)\.
- Liet al\.\(2025\)Y\. Li, M\. Xiong, J\. Wu, and B\. HooiConfTuner: Training Large Language Models to Express Their Confidence Verbally\.InAdvances in Neural Information Processing Systems 38,pp\. 59517–59546\.External Links:[Document](https://dx.doi.org/10.52202/085713-1783)Cited by:[§H\.3](https://arxiv.org/html/2610.00381#A8.SS3.p1.1),[§2\.2](https://arxiv.org/html/2610.00381#S2.SS2.p1.1)\.
- Liet al\.\(2026c\)Y\. Li, Y\. Miao, R\. Krishnan, and R\. PadmanJEV\-as\-a\-Judge: Accept When Confident, Escalate When Unsure\.External Links:2609\.26550,[Link](https://arxiv.org/abs/2609.26550)Cited by:[§2\.3](https://arxiv.org/html/2610.00381#S2.SS3.p1.1)\.
- Linet al\.\(2022\)S\. Lin, J\. Hilton, and O\. EvansTeaching Models to Express Their Uncertainty in Words\.Transactions on Machine Learning Research\.External Links:[Link](https://openreview.net/forum?id=8s8K2UZGTZ)Cited by:[§H\.3](https://arxiv.org/html/2610.00381#A8.SS3.p1.1)\.
- Liuet al\.\(2021\)B\. Liu, L\. Zhan, L\. Xu, L\. Ma, Y\. Yang, and X\. WuSLAKE: A Semantically\-Labeled Knowledge\-Enhanced Dataset for Medical Visual Question Answering\.In2021 IEEE 18th International Symposium on Biomedical Imaging \(ISBI\),pp\. 1650–1654\.External Links:[Document](https://dx.doi.org/10.1109/ISBI48211.2021.9434010)Cited by:[§2\.1](https://arxiv.org/html/2610.00381#S2.SS1.p1.1)\.
- Liuet al\.\(2020\)J\. Liu, J\. Lian, and Y\. YuChestX\-Det10: Chest X\-ray Dataset on Detection of Thoracic Abnormalities\.External Links:2006\.10550,[Link](https://arxiv.org/abs/2006.10550)Cited by:[§2\.1](https://arxiv.org/html/2610.00381#S2.SS1.p1.1)\.
- Maet al\.\(2026a\)W\. Ma, L\. Zhao, Y\. Zeng, and J\. ZhaoJEV\-Star: Fast, Low\-Cost StarCraft II Control with Language\-Model Planning\.External Links:2609\.27331,[Link](https://arxiv.org/abs/2609.27331)Cited by:[Appendix H](https://arxiv.org/html/2610.00381#A8.p1.1)\.
- Maet al\.\(2026b\)Z\. Ma, X\. Wen, B\. Cao, Y\. Lu, H\. Lin, J\. Yang, M\. He, X\. Han, and L\. SunDecoupling Reasoning and Confidence: Resurrecting Calibration in Reinforcement Learning from Verifiable Rewards\.InProceedings of the 43rd International Conference on Machine Learning,External Links:2603\.09117,[Link](https://arxiv.org/abs/2603.09117)Cited by:[§H\.3](https://arxiv.org/html/2610.00381#A8.SS3.p1.1)\.
- Machchaet al\.\(2026\)S\. Machcha, S\. Yerra, S\. Gupta, A\. Sahoo, S\. Sultana, H\. Yu, and Z\. YaoKnowing When to Abstain: Medical LLMs Under Clinical Uncertainty\.InProceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics \(EACL\),pp\. 6153–6182\.External Links:[Document](https://dx.doi.org/10.18653/v1/2026.eacl-long.291)Cited by:[§H\.4](https://arxiv.org/html/2610.00381#A8.SS4.p1.1),[§1](https://arxiv.org/html/2610.00381#S1.p2.1),[§2\.3](https://arxiv.org/html/2610.00381#S2.SS3.p1.1)\.
- Malinin and Gales \(2021\)A\. Malinin and M\. GalesUncertainty Estimation in Autoregressive Structured Prediction\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://openreview.net/forum?id=jN5y-zb5Q7m)Cited by:[§H\.3](https://arxiv.org/html/2610.00381#A8.SS3.p1.1)\.
- Mehrtenset al\.\(2023\)H\. Mehrtens, T\. Bucher, and T\. J\. BrinkerPitfalls of Conformal Predictions for Medical Image Classification\.InUncertainty for Safe Utilization of Machine Learning in Medical Imaging \(UNSURE\), MICCAI Workshop,pp\. 198–207\.External Links:2506\.18162,[Document](https://dx.doi.org/10.1007/978-3-031-44336-7%5F20)Cited by:[§H\.4](https://arxiv.org/html/2610.00381#A8.SS4.p1.1)\.
- Mooret al\.\(2023\)M\. Moor, Q\. Huang, S\. Wu, M\. Yasunaga, Y\. Dalmia, J\. Leskovec, C\. Zakka, E\. P\. Reis, and P\. RajpurkarMed\-Flamingo: a Multimodal Medical Few\-shot Learner\.InProceedings of the 3rd Machine Learning for Health Symposium \(ML4H\),pp\. 353–367\.External Links:[Link](https://proceedings.mlr.press/v225/moor23a.html)Cited by:[§1](https://arxiv.org/html/2610.00381#S1.p2.1),[§2\.1](https://arxiv.org/html/2610.00381#S2.SS1.p1.1)\.
- Mozannar and Sontag \(2020\)H\. Mozannar and D\. SontagConsistent Estimators for Learning to Defer to an Expert\.InProceedings of the 37th International Conference on Machine Learning,pp\. 7076–7087\.External Links:[Link](https://proceedings.mlr.press/v119/mozannar20b.html)Cited by:[§H\.2](https://arxiv.org/html/2610.00381#A8.SS2.p1.1)\.
- Murphy \(1973\)A\. H\. MurphyA New Vector Partition of the Probability Score\.Journal of Applied Meteorology12\(4\),pp\. 595–600\.External Links:[Document](https://dx.doi.org/10.1175/1520-0450%281973%29012%3C0595%3Aanvpot%3E2.0.co%3B2)Cited by:[§H\.1](https://arxiv.org/html/2610.00381#A8.SS1.p1.1)\.
- Naderiet al\.\(2026\)N\. Naderi, S\. A\. A\. Safavi\-Naini, T\. Savage, M\. A\. Khalafi, P\. R\. Lewis, Z\. Atf, G\. Nadkarni, and A\. SoroushAcross generations, sizes, and types, large language models poorly report self\-confidence in gastroenterology clinical reasoning tasks\.npj Gut and Liver3\(1\),pp\. 6\.External Links:[Document](https://dx.doi.org/10.1038/s44355-026-00053-3)Cited by:[§1](https://arxiv.org/html/2610.00381#S1.p2.1),[§2\.2](https://arxiv.org/html/2610.00381#S2.SS2.p1.1)\.
- National Institute of Standards and Technology \(2024\)National Institute of Standards and TechnologyArtificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile\.Note:NIST AI 600\-1External Links:[Document](https://dx.doi.org/10.6028/NIST.AI.600-1),[Link](https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf)Cited by:[§H\.4](https://arxiv.org/html/2610.00381#A8.SS4.p2.1)\.
- Nicolsonet al\.\(2025\)A\. Nicolson, E\. Bradburn, Y\. Gal, A\. T\. Papageorghiou, and J\. A\. NobleThe human factor in explainable artificial intelligence: clinician variability in trust, reliance, and performance\.npj Digital Medicine8\(1\),pp\. 658\.External Links:[Document](https://dx.doi.org/10.1038/s41746-025-02023-0)Cited by:[§H\.4](https://arxiv.org/html/2610.00381#A8.SS4.p1.1),[Appendix I](https://arxiv.org/html/2610.00381#A9.SS0.SSS0.Px3.p1.1)\.
- Omaret al\.\(2025\)M\. Omar, R\. Agbareia, B\. S\. Glicksberg, G\. N\. Nadkarni, and E\. KlangBenchmarking the Confidence of Large Language Models in Answering Clinical Questions: Cross\-Sectional Evaluation Study\.JMIR Medical Informatics13,pp\. e66917–e66917\.External Links:[Document](https://dx.doi.org/10.2196/66917)Cited by:[§H\.1](https://arxiv.org/html/2610.00381#A8.SS1.p1.1)\.
- Palet al\.\(2022\)A\. Pal, L\. K\. Umapathi, and M\. SankarasubbuMedMCQA: A Large\-scale Multi\-Subject Multi\-Choice Dataset for Medical Domain Question Answering\.InProceedings of the Conference on Health, Inference, and Learning \(CHIL\),pp\. 248–260\.External Links:[Link](https://proceedings.mlr.press/v174/pal22a.html)Cited by:[§2\.1](https://arxiv.org/html/2610.00381#S2.SS1.p1.1)\.
- Qaziet al\.\(2026\)I\. A\. Qazi, A\. Ali, A\. U\. Khawaja, M\. J\. Akhtar, A\. Z\. Sheikh, and M\. H\. AlizaiAutomation Bias in Large Language Model–Assisted Diagnostic Reasoning among Physicians Trained in AI Literacy — A Randomized Clinical Trial\.NEJM AI3\(5\)\.External Links:[Document](https://dx.doi.org/10.1056/aioa2501001)Cited by:[§H\.4](https://arxiv.org/html/2610.00381#A8.SS4.p1.1),[Appendix I](https://arxiv.org/html/2610.00381#A9.SS0.SSS0.Px3.p1.1)\.
- Rafe and Das \(2026\)A\. Rafe and S\. DasCalibrated Decisions at Scale: Converting Police Crash Narratives into Probabilistic Crash Variables with a System One Model \(Jev\)\.External Links:2609\.24052,[Link](https://arxiv.org/abs/2609.24052)Cited by:[§2\.3](https://arxiv.org/html/2610.00381#S2.SS3.p1.1)\.
- Renet al\.\(2026\)S\. Ren, K\. Zewde, X\. Shen, Y\. Zhou, D\. Ng, A\. Raj, T\. Duong, Y\. Zhang, and N\. TiangratanakulOpen\-Jev Judgments on CallScreenBench: Calibrated One\-Pass Scam Screening with a Small Language Model\.External Links:2609\.23959,[Link](https://arxiv.org/abs/2609.23959)Cited by:[§2\.3](https://arxiv.org/html/2610.00381#S2.SS3.p1.1)\.
- Robitzaet al\.\(2026\)W\. Robitzaet al\.JEVQA — Video Quality from Metadata, Bitstream, and Pixel Features with a General\-Purpose Decision Model\.External Links:2609\.24395,[Link](https://arxiv.org/abs/2609.24395)Cited by:[Appendix H](https://arxiv.org/html/2610.00381#A8.p1.1)\.
- Sadanandan and Behzadan \(2026\)B\. Sadanandan and V\. BehzadanConsistent but Dangerous: Per\-Sample Safety Classification Reveals False Reliability in Medical Vision\-Language Models\.External Links:2603\.20985,[Link](https://arxiv.org/abs/2603.20985)Cited by:[§H\.1](https://arxiv.org/html/2610.00381#A8.SS1.p1.1)\.
- Sanz\-Guerreroet al\.\(2026\)M\. Sanz\-Guerrero, M\. Mager, and K\. von der WenseLarge Language Models Are Overconfident in Their Own Responses\.InFindings of the Association for Computational Linguistics: ACL 2026,pp\. 31406–31418\.External Links:[Document](https://dx.doi.org/10.18653/v1/2026.findings-acl.1570)Cited by:[§H\.2](https://arxiv.org/html/2610.00381#A8.SS2.p1.1)\.
- Savageet al\.\(2025\)T\. Savage, J\. Wang, R\. Gallo, A\. Boukil, V\. Patel, S\. A\. A\. Safavi\-Naini, A\. Soroush, and J\. H\. ChenLarge language model uncertainty proxies: discrimination and calibration for medical diagnosis and treatment\.Journal of the American Medical Informatics Association32\(1\),pp\. 139–149\.External Links:[Document](https://dx.doi.org/10.1093/jamia/ocae254)Cited by:[§1](https://arxiv.org/html/2610.00381#S1.p2.1),[§2\.2](https://arxiv.org/html/2610.00381#S2.SS2.p1.1)\.
- Sellergrenet al\.\(2025\)A\. Sellergren, S\. Kazemzadeh, T\. Jaroensri, A\. Kiraly, M\. Traverse, T\. Kohlberger, S\. Xu, F\. Jamil,et al\.MedGemma Technical Report\.External Links:2507\.05201,[Link](https://arxiv.org/abs/2507.05201)Cited by:[§1](https://arxiv.org/html/2610.00381#S1.p2.1),[§2\.1](https://arxiv.org/html/2610.00381#S2.SS1.p1.1)\.
- Sharifdeenet al\.\(2026\)A\. Sharifdeen, S\. A\. Ahamed, U\. Khan, M\. A\. Munir, S\. Ibrahim, M\. R\. M\. Ahamed, Y\. Xie, I\. Razzak, and M\. H\. KhanMVC\-Bench: Benchmarking Calibration of Medical Vision\-Language Models\.External Links:2608\.27004,[Link](https://arxiv.org/abs/2608.27004)Cited by:[§2\.2](https://arxiv.org/html/2610.00381#S2.SS2.p1.1)\.
- Shekaret al\.\(2025\)S\. Shekar, P\. Pataranutaporn, C\. Sarabu, G\. A\. Cecchi, and P\. MaesPeople Overtrust AI\-Generated Medical Advice despite Low Accuracy\.NEJM AI2\(6\)\.External Links:[Document](https://dx.doi.org/10.1056/aioa2300015)Cited by:[§H\.4](https://arxiv.org/html/2610.00381#A8.SS4.p1.1),[Appendix I](https://arxiv.org/html/2610.00381#A9.SS0.SSS0.Px3.p1.1)\.
- Sikoraet al\.\(2026\)A\. Sikora, L\. A\. Celi, and R\. E\. AbdulnourCan AI Say “I Don’t Know”?\.New England Journal of Medicine394\(19\),pp\. 1873–1875\.External Links:[Document](https://dx.doi.org/10.1056/nejmp2517624)Cited by:[§H\.4](https://arxiv.org/html/2610.00381#A8.SS4.p1.1),[Appendix I](https://arxiv.org/html/2610.00381#A9.SS0.SSS0.Px3.p1.1)\.
- Singhet al\.\(2026\)S\. Singh, A\. Reddy, and M\. ChopraVerifiable Rewards for Calibrated Probabilistic Forecasting\.External Links:2607\.00164,[Link](https://arxiv.org/abs/2607.00164)Cited by:[§H\.3](https://arxiv.org/html/2610.00381#A8.SS3.p1.1)\.
- Singhalet al\.\(2023\)K\. Singhal, S\. Azizi, T\. Tu, S\. S\. Mahdavi, J\. Wei, H\. W\. Chung, N\. Scales, A\. Tanwani, H\. Cole\-Lewis, S\. Pfohl, P\. Payne, M\. Seneviratne, P\. Gamble, C\. Kelly, A\. Babiker, N\. Schärli, A\. Chowdhery, P\. Mansfield, D\. Demner\-Fushman, B\. Agüera y Arcas, D\. Webster, G\. S\. Corrado, Y\. Matias, K\. Chou, J\. Gottweis, N\. Tomasev, Y\. Liu, A\. Rajkomar, J\. Barral, C\. Semturs,et al\.Large language models encode clinical knowledge\.Nature620\(7972\),pp\. 172–180\.External Links:[Document](https://dx.doi.org/10.1038/s41586-023-06291-2)Cited by:[§1](https://arxiv.org/html/2610.00381#S1.p2.1),[§2\.1](https://arxiv.org/html/2610.00381#S2.SS1.p1.1)\.
- Singhalet al\.\(2025\)K\. Singhal, T\. Tu, J\. Gottweis, R\. Sayres, E\. Wulczyn, M\. Amin, L\. Hou, K\. Clark, S\. R\. Pfohl, H\. Cole\-Lewis, D\. Neal, Q\. M\. Rashid, M\. Schaekermann, A\. Wang, D\. Dash, J\. H\. Chen, N\. H\. Shah, S\. Lachgar, P\. A\. Mansfield, S\. Prakash, B\. Green, E\. Dominowska, B\. Agüera y Arcas, N\. Tomašev, Y\. Liu, R\. Wong, C\. Semturs, S\. S\. Mahdavi, J\. K\. Barral, D\. R\. Webster,et al\.Toward expert\-level medical question answering with large language models\.Nature Medicine31\(3\),pp\. 943–950\.External Links:[Document](https://dx.doi.org/10.1038/s41591-024-03423-7)Cited by:[§1](https://arxiv.org/html/2610.00381#S1.p2.1),[§2\.1](https://arxiv.org/html/2610.00381#S2.SS1.p1.1)\.
- Steyverset al\.\(2025\)M\. Steyvers, H\. Tejeda, A\. Kumar, C\. Belem, S\. Karny, X\. Hu, L\. W\. Mayer, and P\. SmythWhat large language models know and what people think they know\.Nature Machine Intelligence7\(2\),pp\. 221–231\.External Links:[Document](https://dx.doi.org/10.1038/s42256-024-00976-7)Cited by:[§H\.1](https://arxiv.org/html/2610.00381#A8.SS1.p1.1)\.
- Stronget al\.\(2025\)J\. Strong, Q\. Men, and J\. A\. NobleTrustworthy and Practical AI for Healthcare: A Guided Deferral System with Large Language Models\.InProceedings of the AAAI Conference on Artificial Intelligence,pp\. 28413–28421\.External Links:[Document](https://dx.doi.org/10.1609/aaai.v39i27.35063)Cited by:[§H\.3](https://arxiv.org/html/2610.00381#A8.SS3.p1.1),[§H\.4](https://arxiv.org/html/2610.00381#A8.SS4.p1.1)\.
- Sunet al\.\(2026\)E\. Sun, J\. Strong, and A\. NobleLearning to Defer with Guidance on Real World Medical Data\.External Links:2609\.26384,[Link](https://arxiv.org/abs/2609.26384)Cited by:[§H\.4](https://arxiv.org/html/2610.00381#A8.SS4.p1.1)\.
- Sun and Xu \(2026\)Y\. Sun and J\. XuType\-Safe Is Not Error\-Free: A Constrained Decision Head Follows the Option Name, Not the Rubric Bound to It\.External Links:2609\.26758,[Link](https://arxiv.org/abs/2609.26758)Cited by:[§2\.3](https://arxiv.org/html/2610.00381#S2.SS3.p1.1)\.
- Tanet al\.\(2026\)C\. H\. Tan, Z\. Lin, M\. Motani, and W\. S\. LeeOn the effectiveness of reward functions in reinforcement learning for confidence calibration of large language models\.External Links:2607\.04332,[Link](https://arxiv.org/abs/2607.04332)Cited by:[§H\.3](https://arxiv.org/html/2610.00381#A8.SS3.p1.1)\.
- Testoni and Calixto \(2026\)A\. Testoni and I\. CalixtoMind the Gap: Benchmarking LLM Uncertainty and Calibration with Specialty\-Aware Clinical QA and Reasoning\-Based Behavioural Features\.InProceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics \(EACL\),pp\. 2364–2382\.External Links:[Document](https://dx.doi.org/10.18653/v1/2026.eacl-long.106)Cited by:[§1](https://arxiv.org/html/2610.00381#S1.p2.1),[§2\.2](https://arxiv.org/html/2610.00381#S2.SS2.p1.1)\.
- Tianet al\.\(2023\)K\. Tian, E\. Mitchell, A\. Zhou, A\. Sharma, R\. Rafailov, H\. Yao, C\. Finn, and C\. D\. ManningJust Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine\-Tuned with Human Feedback\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),pp\. 5433–5442\.External Links:[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.330)Cited by:[§H\.3](https://arxiv.org/html/2610.00381#A8.SS3.p1.1)\.
- TypeSafe AI \(2026\)TypeSafe AIIntroducing System One Models and Jev\.Note:TypeSafe AI technical blogExternal Links:[Link](https://typesafe.ai/)Cited by:[§2\.3](https://arxiv.org/html/2610.00381#S2.SS3.p1.1)\.
- U\.S\. Food and Drug Administration, Center for Devices and Radiological Health, Digital Health Center of Excellence \(2026\)U\.S\. Food and Drug Administration, Center for Devices and Radiological Health, Digital Health Center of ExcellenceConsiderations for the Regulation of Generative AI\-Enabled Medical Devices: Discussion Paper and Request for Feedback\.Note:Discussion paper, docket FDA\-2026\-N\-7874External Links:[Link](https://www.fda.gov/medical-devices/digital-health-center-excellence/considerations-regulation-generative-ai-enabled-medical-devices-discussion-paper-and-request)Cited by:[§H\.4](https://arxiv.org/html/2610.00381#A8.SS4.p2.1)\.
- Van Calsteret al\.\(2025\)B\. Van Calster, G\. S\. Collins, A\. J\. Vickers, L\. Wynants, K\. F\. Kerr, L\. Barreñada, G\. Varoquaux, K\. Singh, K\. G\. Moons, T\. Hernandez\-Boussard, D\. Timmerman, D\. J\. McLernon, M\. van Smeden, and E\. W\. SteyerbergEvaluation of performance measures in predictive artificial intelligence models to support medical decisions: overview and guidance\.The Lancet Digital Health7\(12\),pp\. 100916\.External Links:[Document](https://dx.doi.org/10.1016/j.landig.2025.100916)Cited by:[§H\.1](https://arxiv.org/html/2610.00381#A8.SS1.p1.1)\.
- Vazhentsevet al\.\(2026\)A\. Vazhentsev, I\. Sviridov, A\. Barseghyan, G\. Kuzmin, A\. Panchenko, A\. Nesterov, A\. Shelmanov, and M\. PanovUncertainty\-aware abstention in medical diagnosis based on medical texts\.npj Artificial Intelligence\.External Links:[Document](https://dx.doi.org/10.1038/s44387-026-00097-z)Cited by:[§H\.4](https://arxiv.org/html/2610.00381#A8.SS4.p1.1)\.
- Vermaet al\.\(2023\)R\. Verma, D\. Barrejon, and E\. NalisnickLearning to Defer to Multiple Experts: Consistent Surrogate Losses, Confidence Calibration, and Conformal Ensembles\.InProceedings of The 26th International Conference on Artificial Intelligence and Statistics,pp\. 11415–11434\.External Links:[Link](https://proceedings.mlr.press/v206/verma23a.html)Cited by:[§H\.3](https://arxiv.org/html/2610.00381#A8.SS3.p1.1)\.
- Wanget al\.\(2025\)Z\. Wang, Z\. Shi, H\. Zhou, S\. Gao, Q\. Sun, and J\. LiTowards Objective Fine\-tuning: How LLMs’ Prior Knowledge Causes Potential Poor Calibration?\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(ACL\),pp\. 14830–14853\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.acl-long.722)Cited by:[§H\.2](https://arxiv.org/html/2610.00381#A8.SS2.p1.1)\.
- Wanget al\.\(2026\)Z\. Wang, X\. Lou, M\. Wu, Z\. Wen, and J\. ZhangCalibration\-Aware Policy Optimization for Reasoning LLMs\.InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics \(ACL\),pp\. 18375–18390\.External Links:[Document](https://dx.doi.org/10.18653/v1/2026.acl-long.836)Cited by:[§H\.3](https://arxiv.org/html/2610.00381#A8.SS3.p1.1)\.
- World Health Organization \(2024\)World Health OrganizationEthics and Governance of Artificial Intelligence for Health: Guidance on Large Multi\-Modal Models\.Note:WHO guideline, ISBN 978\-92\-4\-008475\-9External Links:[Link](https://www.who.int/publications/i/item/9789240084759)Cited by:[§H\.4](https://arxiv.org/html/2610.00381#A8.SS4.p2.1)\.
- Wu and Lim \(2026\)T\. Wu and W\. Y\. B\. LimREFLEX with Jev for Efficient Selective Control in LLM Agents\.External Links:2609\.26532,[Link](https://arxiv.org/abs/2609.26532)Cited by:[§2\.3](https://arxiv.org/html/2610.00381#S2.SS3.p1.1)\.
- Xiaoet al\.\(2025\)J\. Xiao, B\. Hou, Z\. Wang, R\. Jin, Q\. Long, W\. J\. Su, and L\. ShenRestoring Calibration for Aligned Large Language Models: A Calibration\-Aware Fine\-Tuning Approach\.InProceedings of the 42nd International Conference on Machine Learning,pp\. 68364–68390\.External Links:[Link](https://proceedings.mlr.press/v267/xiao25b.html)Cited by:[§H\.2](https://arxiv.org/html/2610.00381#A8.SS2.p1.1)\.
- Xieet al\.\(2024\)J\. Xie, A\. S\. Chen, Y\. Lee, and E\. MitchellCalibrating Language Models with Adaptive Temperature Scaling\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),pp\. 18128–18138\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.emnlp-main.1007)Cited by:[§2\.2](https://arxiv.org/html/2610.00381#S2.SS2.p1.1)\.
- Xionget al\.\(2024\)M\. Xiong, Z\. Hu, X\. Lu, Y\. Li, J\. Fu, J\. He, and B\. HooiCan LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://openreview.net/forum?id=gjeQKFxFpZ)Cited by:[§H\.2](https://arxiv.org/html/2610.00381#A8.SS2.p1.1),[§2\.2](https://arxiv.org/html/2610.00381#S2.SS2.p1.1)\.
- Xuet al\.\(2026\)X\. ’\. Xu, H\. Hu, H\. Zhang, W\. K\. Wang, R\. Wang, L\. R\. Soenksen, O\. Badri, S\. Jafry, E\. Burger, L\. Nwandu, A\. Mehta, E\. P\. Duhaime, A\. Qasim, H\. Lin, J\. K\. Pereira, J\. Hershon, P\. Mui, A\. A\. Gru, N\. Elhadad, L\. Mamykina, M\. Groh, P\. Tschandl, R\. Daneshjou, and M\. GhassemiDivergent impacts of explainable AI for dermatological diagnosis on clinicians versus lay people\.Nature Medicine32\(8\),pp\. 3000–3009\.External Links:[Document](https://dx.doi.org/10.1038/s41591-026-04553-w)Cited by:[§H\.4](https://arxiv.org/html/2610.00381#A8.SS4.p1.1),[Appendix I](https://arxiv.org/html/2610.00381#A9.SS0.SSS0.Px3.p1.1)\.
- Yaldizet al\.\(2026\)D\. N\. Yaldiz, E\. Spiliopoulou, Z\. Qi, S\. Varia, S\. Doss, and N\. PappasBalancing Classification and Calibration Performance in Decision\-Making LLMs via Calibration Aware Reinforcement Learning\.InFindings of the Association for Computational Linguistics: ACL 2026,pp\. 12537–12553\.External Links:[Document](https://dx.doi.org/10.18653/v1/2026.findings-acl.610)Cited by:[§H\.3](https://arxiv.org/html/2610.00381#A8.SS3.p1.1)\.
- Yanget al\.\(2023\)J\. Yang, R\. Shi, D\. Wei, Z\. Liu, L\. Zhao, B\. Ke, H\. Pfister, and B\. NiMedMNIST v2 \- A large\-scale lightweight benchmark for 2D and 3D biomedical image classification\.Scientific Data10\(1\),pp\. 41\.External Links:[Document](https://dx.doi.org/10.1038/s41597-022-01721-8)Cited by:[§2\.1](https://arxiv.org/html/2610.00381#S2.SS1.p1.1)\.
- Yanget al\.\(2024\)K\. Yang, D\. Klein, A\. Celikyilmaz, N\. Peng, and Y\. TianRLCD: Reinforcement Learning from Contrastive Distillation for Language Model Alignment\.InInternational Conference on Learning Representations \(ICLR\),External Links:2307\.12950,[Link](https://arxiv.org/abs/2307.12950)Cited by:[§H\.3](https://arxiv.org/html/2610.00381#A8.SS3.p1.1)\.
- Yu and Yao \(2026\)G\. Yu and Y\. YaoVisual Jev: Accurate and Efficient Decisions from Shared Visual Context\.External Links:2609\.25845,[Link](https://arxiv.org/abs/2609.25845)Cited by:[§2\.3](https://arxiv.org/html/2610.00381#S2.SS3.p1.1)\.
- Zambrano Chaveset al\.\(2025\)J\. M\. Zambrano Chaves, S\. Huang, Y\. Xu, H\. Xu, N\. Usuyama, S\. Zhang, F\. Wang, Y\. Xie, M\. Khademi, Z\. Yang, H\. Awadalla, J\. Gong, H\. Hu, J\. Yang, C\. Li, J\. Gao, Y\. Gu, C\. Wong, M\. Wei, T\. Naumann, M\. Chen, M\. P\. Lungren, A\. Chaudhari, S\. Yeung\-Levy, C\. P\. Langlotz, S\. Wang, and H\. PoonA clinically accessible small multimodal radiology model and evaluation metric for chest X\-ray findings\.Nature Communications16\(1\),pp\. 3108\.External Links:[Document](https://dx.doi.org/10.1038/s41467-025-58344-x)Cited by:[§2\.1](https://arxiv.org/html/2610.00381#S2.SS1.p1.1)\.
- Zhanget al\.\(2026a\)F\. Zhang, Y\. Chen, Z\. Xie, Y\. Zhou, S\. Peng, L\. Fan, X\. Ji, C\. Zheng, H\. Shan, P\. S\. Yu, X\. Liu, Y\. Chen,et al\.Same Scores, Different Decisions: Evaluating JEV and Language Models for Legal Document Understanding\.External Links:2609\.27678,[Link](https://arxiv.org/abs/2609.27678)Cited by:[Appendix H](https://arxiv.org/html/2610.00381#A8.p1.1)\.
- Zhanget al\.\(2024\)K\. Zhang, R\. Zhou, E\. Adhikarla, Z\. Yan, Y\. Liu, J\. Yu, Z\. Liu, X\. Chen, B\. D\. Davison, H\. Ren, J\. Huang, C\. Chen, Y\. Zhou, S\. Fu, W\. Liu, T\. Liu, X\. Li, Y\. Chen, L\. He, J\. Zou, Q\. Li, H\. Liu, and L\. SunA generalist vision–language foundation model for diverse biomedical tasks\.Nature Medicine30\(11\),pp\. 3129–3141\.External Links:[Document](https://dx.doi.org/10.1038/s41591-024-03185-2)Cited by:[§1](https://arxiv.org/html/2610.00381#S1.p2.1),[§2\.1](https://arxiv.org/html/2610.00381#S2.SS1.p1.1)\.
- Zhanget al\.\(2026b\)L\. Zhang, G\. Wölflein, D\. Ferber, J\. Liang, Z\. I\. Carrero, X\. Wu, J\. Vibert, J\. Clusmann, L\. Möhrmann, E\. E\. Möhrmann, C\. Wichmann, F\. Wolf, T\. Lenz, and J\. N\. KatherOn\-premise medical AI agents for reliable clinical decision\-making\.Nature Medicine\.External Links:[Document](https://dx.doi.org/10.1038/s41591-026-04609-x)Cited by:[§H\.4](https://arxiv.org/html/2610.00381#A8.SS4.p2.1)\.
- Zhaoet al\.\(2021\)T\. Z\. Zhao, E\. Wallace, S\. Feng, D\. Klein, and S\. SinghCalibrate Before Use: Improving Few\-Shot Performance of Language Models\.InProceedings of the 38th International Conference on Machine Learning,pp\. 12697–12706\.External Links:[Link](https://proceedings.mlr.press/v139/zhao21c.html)Cited by:[§2\.2](https://arxiv.org/html/2610.00381#S2.SS2.p1.1)\.
- Zhouet al\.\(2024\)K\. Zhou, J\. D\. Hwang, X\. Ren, and M\. SapRelying on the Unreliable: The Impact of Language Models’ Reluctance to Express Uncertainty\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(ACL\),pp\. 3623–3643\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.198)Cited by:[§H\.2](https://arxiv.org/html/2610.00381#A8.SS2.p1.1),[§2\.2](https://arxiv.org/html/2610.00381#S2.SS2.p1.1)\.

## Ethics Statement

This work uses only de\-identified, publicly available medical imaging datasets and collects no new patient data\. The models studied are retrospective predictors, not clinical tools: no component of this work is intended or validated for diagnostic use, and the decision probabilities reported here have not been assessed prospectively in a clinical workflow\. The one place where a deployment claim could be read into the results is selective prediction, where a deferral policy consumes the model’s confidence; we note explicitly that a model that ranks decisions well is not thereby safe to deploy, and that the human factors of deferral remain an open question \(Appendix[H](https://arxiv.org/html/2610.00381#A8)\)\.

## Reproducibility Statement

Both models train on public datasets listed in Appendix[A](https://arxiv.org/html/2610.00381#A1), with splits and candidate grids described there\. Metric definitions and the estimator used to recover probabilities from the generative baseline are given in Sections[4\.3](https://arxiv.org/html/2610.00381#S4.SS3)and[4\.4](https://arxiv.org/html/2610.00381#S4.SS4), with the full definitions, edge cases and checks in Appendix[C](https://arxiv.org/html/2610.00381#A3)and[F](https://arxiv.org/html/2610.00381#A6)\. Training hyperparameters, hardware and checkpoint selection are in Appendix[E](https://arxiv.org/html/2610.00381#A5); the per\-checkpoint numbers underlying every figure are in Appendix[D](https://arxiv.org/html/2610.00381#A4)\. Prompt templates for both models are reproduced in Appendix[G](https://arxiv.org/html/2610.00381#A7)\.

## Appendix AEvaluation Suite Details

Table 3:The evaluation suite\. Both models train on the same 23,558 states and are evaluated on the same 726 held\-out states\. “Decisions” counts decision\-level targets in training \(Section[3\.3](https://arxiv.org/html/2610.00381#S3.SS3)\); “test” counts held\-out states\. Candidate grid sizes are given for Score decisions and per\-state candidate counts for Noul\.### A\.1\. Composition and supervision

Both models train on the same 23,558 states and are scored on the same 726 held\-out states, so the comparison holds at the level of states rather than of images\. The suite spans fifteen source datasets and eight modality labels, and the models differ in how those states become targets\. A generative baseline emits one answer per state, so a chest radiograph with fourteen annotated findings contributes one string; the decision interface factorizes the same annotation into one target per decision, so the same radiograph contributes fourteen presence decisions, and the suite as a whole yields 155,729 decision\-level targets rather than 23,558 answer\-level targets\. No additional annotation is collected: the counts differ because the annotation is read at the level of the decisions it describes\. Table[3](https://arxiv.org/html/2610.00381#A1.T3)gives the family\-level totals, and the per\-source\-dataset composition and splits are recorded in the evaluation artifacts\.

## Appendix BTyped Decision Spaces and Candidate Grids

### B\.1\. Decision types and their signatures

A decision is a tupleD=\(Q,T,C\)D=\(Q,T,C\): a questionQQ, a typeTT, and a finite candidate spaceC=\{c1,…,cK\}C=\\\{c\_\{1\},\\dots,c\_\{K\}\\\}\. The candidate space is part of the model input, not a fixed output head, and the model returns a normalised vector over it\. Four types are defined in the interface contract; only three of them are evaluated here\.

- •Choice\.CCis an unordered finite set of alternatives;pθ​\(ck∣S,D\)≥0p\_\{\\theta\}\(c\_\{k\}\\mid S,D\)\\geq 0and∑k=1Kpθ​\(ck∣S,D\)=1\\sum\_\{k=1\}^\{K\}p\_\{\\theta\}\(c\_\{k\}\\mid S,D\)=1\. The folded prediction isarg⁡maxk​pθ​\(ck∣S,D\)\\arg\\max\_\{k\}p\_\{\\theta\}\(c\_\{k\}\\mid S,D\)\. Used for classification and for the counting grid\.
- •Noul\.C=\{No,Yes\}C=\\\{\\text\{No\},\\text\{Yes\}\\\}andp⁡\(No\)\+p⁡\(Yes\)=1p\(\\text\{No\}\)\+p\(\\text\{Yes\}\)=1\. One Noul decision is emitted per finding, so a single chest radiograph state produces 14 \(chestmnist\) or 13 \(ChestX\-Det\) binary decisions in one forward pass\. TheNocandidate is the null answer for that finding: it asserts absence rather than declining to answer\. Folding keeps a label iffp⁡\(Yes\)\>0\.5p\(\\text\{Yes\}\)\>0\.5and joins the surviving labels with “;”\.
- •Score\.C=\{v1,…,vK\}C=\\\{v\_\{1\},\\dots,v\_\{K\}\\\}is an ordered grid; the model returnsp⁡\(vk\)p\(v\_\{k\}\)and the point estimate is the expectationy^=∑kp⁡\(vk\)​vk\\hat\{y\}=\\sum\_\{k\}p\(v\_\{k\}\)\\,v\_\{k\}\. Used for regression, and, with rounding, for the count decision\. Candidates are not numbered: the grid position defines the index and the candidate text is self\-contained \(\- 45 \>\>already says 45\), so moving a candidate changes only the position it occupies\.
- •Span\.The detection type, in which a box is emitted as a span decision\. Span decisions were*removed on both models*; no result reported here scores detection\. The reason is recorded in the project log: the coordinate head collapsed to a constant box rather than localizing findings, with near\-zero overlap against the annotations\.

### B\.2\. Candidate grids

Grid sizes follow the annotation space of each source dataset rather than the model\. Classification candidates are the pooled organ and tissue classes of the ten MedMNIST collections\. Noul decisions use two candidates per finding, with thirteen or fourteen decisions per chest radiograph\. Score decisions use an ordered grid: the platelet and white\-cell counts are represented on a 33\-point grid, the ejection fraction on a 101\-point grid, and the retinal score on a 5\-point grid\. Candidates carry no injected index, so a Score grid is identified by the values it contains and can be replaced at inference time without changing the model\.

### B\.3\. Open\-set behaviour of the null answer

Theadd\_noneprobe inserts an explicitnonecandidate into every decision space and measures whether its probability carries error information\. At step 2424 the meanp⁡\(none\)p\(\\texttt\{none\}\)is0\.36800\.3680, and it is higher on correct decisions than on wrong ones \(0\.40170\.4017versus0\.16970\.1697\); at step 1616 the same pattern holds \(0\.37260\.3726overall,0\.40460\.4046on correct versus0\.18580\.1858on wrong\)\. The resulting AUC ofp⁡\(none\)p\(\\texttt\{none\}\)as a predictor of error is0\.13350\.1335at step 2424 and0\.13190\.1319at step 1616, i\.e\. reversed relative to the intended direction\. The project log records this failure as unexplained; we report it rather than omit it\.

## Appendix CMetrics: Definitions, Motivation and Scope

Every quantity used in this report is produced by one pure\-function metric library, so a metric means the same thing on both sides of the comparison\. This appendix states the definitions, the reason each family is reported, and what each family can and cannot establish\.

### C\.1\. What each family can and cannot establish

ECE is an average of absolute bin gaps, so it is comparable across models but insensitive to the direction of the error, and it can hide a model that is overconfident in one region of confidence space and underconfident in another; MCE reports the worst bin and is the reason we do not read ECE alone\. Reliability and resolution are reported together because either one in isolation is uninformative: a model that always emits the class prior has reliability error near zero and resolution exactly zero, and a model that ranks well but reports unusable numbers has high resolution and high reliability error\. The pair is what separates the two failure modes, and it is the reason our central claim is about reliability rather than about accuracy\. AURC andrisk​@​k\\mathrm\{risk\}@kare the quantities a deferral policy actually consumes, but they measure the quality of an ordering, not its safety: a model can have a favourable AURC and still be miscalibrated for a subpopulation, which is the failure mode the clinical learning\-to\-defer literature reports\.DDx​@​0\.9\\mathrm\{DDx\}@0\.9and top\-kkcoverage are task\-level summaries of how the probability mass is distributed over the candidate set; they are interpretable but threshold\-dependent, and they inherit whatever error the candidate grid introduces\. Normalized entropy describes the shape of a decision distribution and is reported for descriptive purposes only\. Wilson intervals are used for proportions because several of these quantities sit near00or11where the normal approximation fails, and paired bootstrap intervals are used for every model\-to\-model difference because the two models score the same rows\.

### C\.2\. The counting comparison on matched decisions

Counting is the one family whose decision\-level count differs between models: the decision ’s counting metrics aggregate 201 decisions \(the total\-count decision of each of 126 states plus 75 cell\-type counts\), while the counting MAE and every generative\-baseline number are computed over the 126 total\-count rows\. Table[4](https://arxiv.org/html/2610.00381#A3.T4)repeats the OmniMed\-Jev numbers on the matched 126 decisions\.

Table 4:The counting family on the 126 total\-count decisions both models scored\. The upper block is the OmniMed\-Jev’s own distribution; the lower block is the baseline’s recorded values for the same rows\. Values in parentheses are the recorded 201\-decision figures that the main text reports for OmniMed\-Jev\.ArmQuantity126 matched\(201 recorded\)OmniMed\-Jevtop\-1 \(argmax\) accuracy↑\\uparrow0\.1508\(0\.3234\)OmniMed\-JevMAE of folded prediction↓\\downarrow2\.5159\(2\.5159\)OmniMed\-JevMAE of the expectation↓\\downarrow2\.5179—OmniMed\-JevRMSE of the expectation↓\\downarrow3\.3250\(3\.3747\)OmniMed\-Jevmean maximum probability0\.1811\(0\.3309\)OmniMed\-Jevmedian maximum probability0\.1697\(0\.1889\)OmniMed\-Jevtop\-3 coverage of the gold count↑\\uparrow0\.4048\(0\.5473\)OmniMed\-Jevmean expectation minus mean gold−1\.1252\-1\.1252—Gen\-SFTexact integer match of the generated string↑\\uparrow0\.1349—Gen\-SFTMAE of the generated integer↓\\downarrow2\.1429—Gen\-SFTtop\-1 of the reconstructed distribution↑\\uparrow0\.0000—Gen\-SFTtop\-3 coverage of the reconstruction↑\\uparrow0\.3254—Two things follow\. First, the recorded counting calibration figures are optimistic for the total\-count task, because the extra 75 decisions are cell\-type counts that the model finds nearly deterministic: the white\-cell count has a mean maximum probability of0\.97460\.9746, and including it lifts the family’s mean maximum probability from0\.18110\.1811to0\.33090\.3309and top\-3 coverage from0\.40480\.4048to0\.54730\.5473\. Second, the OmniMed\-Jev’s top\-1 accuracy on the total\-count task is0\.15080\.1508rather than the0\.32340\.3234the family aggregate reports, which is the same effect seen from the accuracy side\. Neither correction changes the direction of the counting comparison: the generative baseline is ahead on the task metric, and OmniMed\-Jevis far better calibrated on the same rows\.

## Appendix DFull Results Tables

This appendix reports the per\-checkpoint numbers behind Figures[2](https://arxiv.org/html/2610.00381#S5.F2)and[4](https://arxiv.org/html/2610.00381#S5.F4)and Tables[1](https://arxiv.org/html/2610.00381#S3.T1)and[2](https://arxiv.org/html/2610.00381#S5.T2)\. Both models are evaluated at the same step, and the OmniMed\-Jev’s values are recorded at every checkpoint we probed, while the baseline’s decision\-quality numbers exist only at the matched step\.

### D\.1\. Task metrics at all ten checkpoints

Table[5](https://arxiv.org/html/2610.00381#A4.T5)lists the four task metrics for both models at all ten saved checkpoints, plus the untuned backbone\. The backbone row is a single evaluation of MedGemma\-1\.5\-4B before training, so it is shared by both models; it was run on the 1,126\-row file whose four\-family subset is the 726\-row test set, which makes its four\-family numbers directly comparable with the rows below\.

Table 5:Task metrics at all ten matched checkpoints\. Classification is balanced accuracy \(↑\\uparrow\), multi\-label is F1 \(↑\\uparrow\), counting and regression are MAE \(↓\\downarrow\)\. Step 0 is the untuned backbone, evaluated once and therefore identical for both models; its counting and regression MAE count unparseable generated answers as errors \(84\.5%84\.5\\%of its regression answers parsed\), so those two cells are pessimistic\. The selected comparison checkpoint is 2424 \(in bold\)\. These are the numbers plotted in Figure[2](https://arxiv.org/html/2610.00381#S5.F2)\.Four features of Table[5](https://arxiv.org/html/2610.00381#A4.T5)are load\-bearing for the main text\. \(i\) In classification the two curves cross between steps 1616 and 2020: the generative baseline leads by\+0\.046\+0\.046at 404 and\+0\.022\+0\.022at 1616, OmniMed\-Jevleads by\+0\.048\+0\.048at 2020 and\+0\.042\+0\.042at 2424\. \(ii\) In counting the generative baseline wins every one of the ten steps and the gap widens from0\.250\.25at 404 to0\.370\.37at 2424 and0\.660\.66at the final step; the OmniMed\-Jev’s grid distribution is comparatively flat, and its folding rule takes the expectation∑kk​pk\\sum\_\{k\}k\\,p\_\{k\}, which regresses extreme counts toward the centre\. \(iii\) In regression the generative baseline has one severe mid\-training spike \(MAE11\.510011\.5100at 808\) and a smaller rebound at 3232 \(3\.60003\.6000, against2\.97502\.9750at 2828\) while the OmniMed\-Jev stays within2\.952\.95–3\.933\.93throughout\. \(iv\) In multi\-label OmniMed\-Jevpeaks at0\.59020\.5902at step 1616, above the baseline’s best value anywhere in training \(0\.56830\.5683at 2020\), and then declines to0\.56180\.5618at 2424\. The trivial empty\-prediction baseline on this family is0\.52000\.5200, and the OmniMed\-Jev’s empty\-prediction count at 2424 is 105 of 200 rows, against 190 of 200 at step 404, so the0\.56180\.5618is not produced by retreating into silence\.

The supervision\-density asymmetry of Appendix[A](https://arxiv.org/html/2610.00381#A1)is the most plausible explanation for \(i\): OmniMed\-Jevreceives 155,729 decision\-level signals from the same 23,558 images, of which 139,537 are multi\-label Noul decisions, i\.e\. a large amount of “read two candidate strings and choose one” training that transfers to classification Choice decisions, whereas the generative baseline receives one answer\-level signal per row\. We state this as a hypothesis, not a finding; the ablation that would test it \(down\-sampling the multi\-label decisions to the baseline’s count and re\-measuring the crossing point\) has not been run\.

Table 6:Dynamic decision probes on the held\-out states\. Permutation is evaluated on the subset with more than one candidate; restriction and gold removal are evaluated on decisions whose annotated candidate is present\. Chance levels are0\.500\.50for a two\-option shortlist and0\.250\.25for a four\-option shortlist;0\.50\.5is the order\-independence value for mean selected rank\. The last three rows give total variation / top\-1 agreement per family\. The two columns are two checkpoints of the same model\. These operations are not defined under the compared generative interface without an additional candidate\-scoring mechanism \(Section[5\.4](https://arxiv.org/html/2610.00381#S5.SS4)\)\.
### D\.2\. Decision\-quality metrics at the probed checkpoints

Decision\-quality extraction was run at four checkpoints\. On OmniMed\-Jevall four are available; on the generative baseline the logit extraction was run only at step 2424, so the generative column in Table[7](https://arxiv.org/html/2610.00381#A4.T7)has a single entry\. Comparing the four OmniMed\-Jev columns is therefore informative, while the generative column is a single reference point rather than a trajectory\.

Table 7:Decision\-quality metrics at the four probed checkpoints\. Columns 1–4 are OmniMed\-Jev; the last column is the baseline’s reconstructed distribution, which was extracted only at step 2424\. ECE, reliability and AURC are↓\\downarrow; resolution is↑\\uparrow; accuracy is the top\-1 accuracy of the scored distribution\.Three patterns are worth recording\. The OmniMed\-Jev’s counting resolution jumps by an order of magnitude between 404 and 1616 \(0\.0127→0\.11680\.0127\\to 0\.1168\) and then stays near0\.110\.11–0\.130\.13, while its counting ECE is worst at 3685 \(0\.17920\.1792\)\. Its multi\-label reliability falls to0\.00060\.0006at 3685, the best value anywhere in the run, even though the multi\-label F1 at that step is its worst \(0\.53450\.5345\)\. Regression calibration degrades after 2424 while its accuracy is flat, which is the reason 2424 and not 3685 is the comparison checkpoint\.

## Appendix EReproducibility, Compute and Statistical Protocol

### E\.1\. Two\-model design

The comparison holds the backbone, the training images, the optimisation budget and the option\-order augmentation fixed, and changes only the output interface\. Both models start frommodels/medgemma\-1\.5\-4b\-it, see the same 23,558 images, take the same 3,685 optimisation steps, use the same seed, and are scored on the same 726\-row test file by the same evaluator\. Table[8](https://arxiv.org/html/2610.00381#A5.T8)lists the configuration item by item; exactly one item differs\.

Table 8:Training configuration for the two models\. Every row is identical on both sides exceptmax\_length, which is a consequence of the interface: OmniMed\-Jevpacks all of a state’s decisions into one sequence, and the longest observed decision block is 2,707 tokens, which does not fit in 1,024\. This is not a tuning choice and is the only configuration difference between the models\.Augmentation is applied during training and switched off at evaluation time, because the development loss has to be comparable across steps\. On OmniMed\-Jevonly Choice and Noul candidates are permuted; Score grids are left in place because the position*is*the ordinal index, and permuting them would permute the measurement scale\. On the generative baseline only tasks carryingchoices\(classification and multi\-label\) are permuted, which is the exact counterpart\. The augmentation gives each state on average five distinct orderings over training, out of11\!11\!for the largest grid\.

### E\.2\. Two\-stage training of OmniMed\-Jev

Stage A freezes the backbone and trains the decision head only, for 200 steps, with 1,057,410 trainable parameters\. Stage B enables LoRA together with the head, giving 39,555,202 trainable parameters out of 2,529,778,162, i\.e\. 1\.564% of the model\.

### E\.3\. Wall clock, hardware and memory

Table 9:Compute for the two models\. Each model ran on one A800 on a different device\. The peak memory figure is for the OmniMed\-Jev’s Stage B\.
### E\.4\. Statistical protocol

Every reported difference is computed on the two models’ per\-row predictions, because both models score the same rows and the unpaired comparison would overstate the uncertainty\. Proportions are reported with Wilson intervals, which stay inside\[0,1\]\[0,1\]where several of these quantities sit, and differences are reported with paired bootstrap intervals\.

#### Checkpoint

The comparison is made at step 2424, the OmniMed\-Jev’s best\-validation checkpoint, and both models are evaluated at that same step\.

#### Paired tests

Because both models score the same rows, the uncertainty that matters is the paired one, and we computed it on the two models’ per\-row predictions rather than leaving the comparison at unpaired standard errors\.

- •*Classification balanced accuracy*differs by\+0\.0422\+0\.0422with a paired bootstrap 95% interval of\[−0\.0196,\+0\.1059\]\[\-0\.0196,\+0\.1059\]\. The2×22\\times 2table over the 200 rows is: both correct 109, OmniMed\-Jev only 19, generative baseline only 18, both wrong 54\. The exact McNemar test on the 37 discordant pairs givesp=1\.00p=1\.00\. The unpaired standard\-error statement above \(1\.21\.2SE\) therefore overstates how much the data support this difference: paired, the evidence is essentially absent\.
- •*Multi\-label macro F1*differs by\+0\.0170\+0\.0170with interval\[−0\.0264,\+0\.0620\]\[\-0\.0264,\+0\.0620\], and the per\-row sign test gives 19 rows to OmniMed\-Jev, 23 to the generative baseline and 158 ties \(p=0\.64p=0\.64\)\.
- •*Counting MAE*on the 126 matched rows differs by\+0\.3730\+0\.3730with interval\[\+0\.0952,\+0\.6429\]\[\+0\.0952,\+0\.6429\]\. This is the only task\-level difference that is distinguishable from zero at 95%, and it favours the generative baseline\.
- •*Regression MAE*differs by−0\.3912\-0\.3912with interval\[−0\.8756,\+0\.0173\]\[\-0\.8756,\+0\.0173\]\.

Bootstrap intervals use 20,000 paired resamples of rows with a freshly seeded generator \(seed 20260925\) and are quoted to two decimals, the precision the Monte Carlo noise supports \(±0\.005\\pm 0\.005at that budget\)\. Nothing else in this appendix is a significance claim\.

## Appendix FRecovering Probabilities from the Generative Arm

### F\.1\. Why the generative baseline has to be reconstructed

A generative model fine\-tuned to emit only an answer does not report its own confidence because a sentence is appended at inference time\. Table[10](https://arxiv.org/html/2610.00381#A6.T10)records the elicitation attempts we ran\. Prompting for a distribution did not work in either of the two direct forms, and an untuned checkpoint produced values that were all zero, which is why every generative probability in this report comes from reading the model’s own next\-token logits\. The failure is itself part of the result: obtaining confidence from a generative interface requires training it into the model, which is already a change of interface\.

Table 10:Attempts to elicit a distribution from the generative baseline\. Attempts 1 and 2 failed outright; attempt 3 produced values but all of them were zero; attempt 4 is the estimator used for every generative number in this report\.
### F\.2\. The estimator actually used

For a candidateccwith continuation tokensc1,…,cTc\_\{1\},\\dots,c\_\{T\}, the estimator scores each candidate continuation separately and then normalises across candidates:

log⁡P⁡\(c\)\\displaystyle\\log P\(c\)=∑t=1Tlog⁡p⁡\(ct∣prompt,image,c<t\),\\displaystyle=\\sum\_\{t=1\}^\{T\}\\log p\\\!\\left\(c\_\{t\}\\mid\\text\{prompt\},\\text\{image\},c\_\{<t\}\\right\),\(9\)p~​\(c\)\\displaystyle\\tilde\{p\}\(c\)=softmaxc∈C​log​P​\(c\)\.\\displaystyle=\\operatorname\*\{softmax\}\_\{c\\in C\}\\log P\(c\)\.\(10\)Each candidate’s continuation is prefixed with a space, i\.e\. in word\-initial form\. AllKKcandidates are scored in one batched forward pass\. The multi\-label family has no per\-finding likelihood in the semicolon\-joined answer, so it is rewritten as a per\-finding binary question, “Is<finding\>present in this image? Answer Yes or No\.”, asked once per finding \(14 times\), and the softmax in Equation \([10](https://arxiv.org/html/2610.00381#A6.E10)\) is taken over theYesandNocontinuations to givep⁡\(finding present\)p\(\\text\{finding present\}\)\. Classification, counting and regression use the same prompt as the generative inference path\.

### F\.3\. Two caveats on these numbers

*\(i\) The counting cell of the generative baseline is diagnostic, not decisive\.*On the counting family the top\-1 candidate of the reconstructed distribution never equals the gold count:00of126126rows\. That single fact mechanically forces resolution to exactly00and AURC to exactly1\.01\.0for that cell, because a distribution whose top\-1 is never correct cannot separate easy from hard rows under a confidence ordering, and answering the most confident fraction of decisions is then no better than answering all of them\. The counting row of the generative baseline should therefore be read as a statement about the reconstruction on a 33\-way integer grid, not as an estimate of how much confidence information a generative model carries in general\.

*\(ii\) The baseline’s post\-training is a strict superset of the OmniMed\-Jev’s\.*The generative baseline was instruction\-tuned after supervised fine\-tuning, which is a strictly larger post\-training budget than OmniMed\-Jevreceived\. If this asymmetry matters at all, it favours the generative baseline, so the decision\-quality gaps reported in Appendix[C](https://arxiv.org/html/2610.00381#A3)are conservative with respect to it\. We report the asymmetry rather than leave it implicit\.

One further distinction belongs here\. The distribution in Equations \([9](https://arxiv.org/html/2610.00381#A6.E9)\)–\([10](https://arxiv.org/html/2610.00381#A6.E10)\) is a pseudo\-distribution: nothing in training ever made these log\-likelihoods sum to one over the candidate set, and the normalisation is ours, not the model’s\. The OmniMed\-Jev’s distribution is the training objective itself\. The comparison is therefore between a native decision distribution and an inference\-time reconstruction of one, and the reconstruction is the more favourable of the two constructions we could have chosen for the baseline\.

## Appendix GPrompt Templates

Everything in this appendix is a verbatim rendering of the strings the models receive, not a paraphrase\. Two editorial conventions apply\. First, section headings and parenthetical annotations that appear*inside*the listings in the project log are rendered in English here; every model\-facing string is reproduced character for character\. Second, long lines are wrapped at spaces so that they fit the text block; no characters were added, removed or reordered\. The two decision markers are real single characters, U\+27E9 \(\>\>below; token id 241287, terminating a candidate\) and U\+2726 \(\*below; token id 244193, terminating a question\), written here as ASCII stand\-ins because the pdfLaTeX text fonts cannot typeset those code points\.

### G\.1\. Task instructions shared by both models

The same four blocks are used for training, for inference, and for the logit probe\.

```
[disease_diagnosis_classification]
You are given a medical image. Answer the classification question using only the provided
image. If options are provided, return only the correct option letter or class label.

[multi_label_classification]
You are given a medical image. Identify all applicable findings or labels. Return labels
separated by semicolons. Return an empty string if none apply.

[cell_counting]
You are given a medical image. Return only the integer count requested.

[regression]
You are given a medical image. Return only the requested numeric measurement.
```

The system prompt is identical on both models and in all three settings:You are an expert medical imaging assistant\.

### G\.2\. OmniMed\-Jev: the decision block

The markers are the real single characters named above\.

```
<bos><start_of_turn>user
<start_of_image>[256 image soft tokens]<end_of_image>
{source} | {modality}

Q1: <question text verbatim>
- <candidate text> >>
- <candidate text> >>
*
Q2: ...
<end_of_turn>
```

One structural difference from the generative baseline deserves emphasis: OmniMed\-Jevdoes*not*use a<start\_of\_turn\>modelturn\. The entire decision block sits inside the user turn and ends withadd\_generation\_prompt=False, because the model does not generate anything: it reads the hidden states at the marker positions\. Readers who assume the two sequence structures are the same will misread the interface comparison\.

The four families as actually encoded \(\.\.\.marks elision\):

```
----- classification (organamnist, 11 candidates, 1 decision) -----
state header: organamnist | CT

Q1: Which class does this image belong to?
- bladder >>
- femur-left >>
- femur-right >>
- heart >>
- kidney-left >>
- kidney-right >>
- liver >>
- lung-left >>
- lung-right >>
- pancreas >>
- spleen >>
*

----- multi-label (ChestX-Det, 13 Noul decisions) -----
state header: ChestX-Det | Radiology

Q1: Is Atelectasis present in this Radiology image?
- No >>
- Yes >>
*
Q2: Is Calcification present in this Radiology image?
- No >>
- Yes >>
*
... (13 decisions in total)

----- counting (TXL-PBC, 4 Choice decisions, 33 candidates each) -----
state header: TXL-PBC | Hematology

Q1: How many cells are present in this blood smear image?
- 0 >>
- 1 >>
... (0...32)
- 32 >>
*
Q2: How many RBCs are present in this blood smear image?
... (likewise 0...32)
*
... (4 decisions in total: total + RBC / WBC / Platelets)

----- regression (CAMUS, 1 Score decision, 101 candidates) -----
state header: CAMUS | Cardiology

Q1: What is the left ventricular ejection fraction, in percent?
- 0 >>
- 1 >>
... (0...100)
- 100 >>
*
```

The family and type names in the four headers are printed by the routine that renders these prompts, and one of them disagrees with the assignment the experiments use: the counting block is labelled4 Choice decisionsin the header, while the suite assigns counting a Score decision over an ordered grid and scores it that way \(Table[3](https://arxiv.org/html/2610.00381#A1.T3)\)\. The headers are reproduced verbatim rather than corrected\.

The key convention is that candidates carry no injected index: a Score grid is indexed by position and the candidate text is self\-contained \(\- 45 \>\>already says 45\), so moving a candidate changes only the position it occupies and never how it reads\.

## Appendix HExtended Related Work

This appendix carries the threads that the main\-text related work compresses, in the same order: how medical calibration is measured, why the deficit is structural, what the calibration\-aware objectives do, and what the deferral, human\-factors and regulatory literature assumes about the probability it consumes\. One thread is left to this appendix because it is outside medicine: the decision\-native wave now spans legal documents, video quality, scientific relation choice, game control, edge service orchestration and population experiments\([Zhang et al\., 2026a](https://arxiv.org/html/2610.00381#bib.bib106);[Robitza and others, 2026](https://arxiv.org/html/2610.00381#bib.bib107);[Deng et al\., 2026](https://arxiv.org/html/2610.00381#bib.bib105);[Ma et al\., 2026a](https://arxiv.org/html/2610.00381#bib.bib108);[Li et al\., 2026b](https://arxiv.org/html/2610.00381#bib.bib110);[Li et al\., 2026a](https://arxiv.org/html/2610.00381#bib.bib109);[Dağlı, 2026](https://arxiv.org/html/2610.00381#bib.bib111);[Li, 2026](https://arxiv.org/html/2610.00381#bib.bib104)\)\.

### H\.1\. How medical calibration is measured

Medical confidence is usually measured with the tools of probabilistic forecasting\. The Brier score and its Murphy decomposition into reliability, resolution and uncertainty\([BRIER, 1950](https://arxiv.org/html/2610.00381#bib.bib19);[Murphy, 1973](https://arxiv.org/html/2610.00381#bib.bib20)\)separate a model’s calibration from its discrimination, and clinical reporting guidance insists on the distinction: discrimination measures such as AUROC are invariant to monotone rescaling and therefore cannot substitute for a calibration statement\([Van Calster et al\., 2025](https://arxiv.org/html/2610.00381#bib.bib59);[Banerji et al\., 2023](https://arxiv.org/html/2610.00381#bib.bib60)\)\. Applied to medical multimodal models, the measurement consistently finds overconfidence: hallucination and overconfidence correlate across medical VQA benchmarks\([Kim et al\., 2025](https://arxiv.org/html/2610.00381#bib.bib61)\), calibration degrades further under distribution shift\([Guo et al\., 2026](https://arxiv.org/html/2610.00381#bib.bib67)\), per\-sample calibration error exposes cases that aggregate ECE hides\([Sadanandan and Behzadan, 2026](https://arxiv.org/html/2610.00381#bib.bib68)\), and self\-reported confidence is a poor proxy for correctness in radiology and other specialties\([Omar et al\., 2025](https://arxiv.org/html/2610.00381#bib.bib52);[Boie et al\., 2026](https://arxiv.org/html/2610.00381#bib.bib50)\)\. A parallel line in cognitive science asks whether models have any metacognitive access to their own errors at all, with results that are mixed at best\([Steyvers et al\., 2025](https://arxiv.org/html/2610.00381#bib.bib55);[Griot et al\., 2025](https://arxiv.org/html/2610.00381#bib.bib54)\)\. Our measurement protocol follows this literature \(top\-label Murphy decomposition, AURC and risk–coverage\) and adds one object it does not usually have: a native decision distribution to measure\.

### H\.2\. Why the deficit is structural

The degradation is introduced by the post\-training stack rather than by pretraining\. Instruction tuning raises confidence faster than accuracy\([Xiong et al\., 2024](https://arxiv.org/html/2610.00381#bib.bib27);[Zhou et al\., 2024](https://arxiv.org/html/2610.00381#bib.bib40)\); alignment and preference optimization can further distort the probability\([Hu et al\., 2026](https://arxiv.org/html/2610.00381#bib.bib38)\); temperature scaling recovers some of it but does not restore the ordering the model needs\([Xiao et al\., 2025](https://arxiv.org/html/2610.00381#bib.bib36)\); and models report their own confidence inconsistently across prompting and elicitation formats\([Sanz\-Guerrero et al\., 2026](https://arxiv.org/html/2610.00381#bib.bib39);[Wang et al\., 2025](https://arxiv.org/html/2610.00381#bib.bib37)\)\. Work on learning to defer has long argued that a model’s own probability is a poor basis for allocation decisions\([Mozannar and Sontag, 2020](https://arxiv.org/html/2610.00381#bib.bib30)\), which is the assumption our interface attacks at the source rather than after the fact\.

### H\.3\. Calibration\-aware objectives and their failure modes

A large literature trains the probability instead of repairing it: teaching models to express calibrated confidence in words or numbers\([Lin et al\., 2022](https://arxiv.org/html/2610.00381#bib.bib25);[Kadavath et al\., 2022](https://arxiv.org/html/2610.00381#bib.bib24);[Tian et al\., 2023](https://arxiv.org/html/2610.00381#bib.bib26)\), semantic\-entropy and distributional uncertainty over meaning\([Kuhn et al\., 2023](https://arxiv.org/html/2610.00381#bib.bib29);[Malinin and Gales, 2021](https://arxiv.org/html/2610.00381#bib.bib32)\), tokenized proper scoring rules and their extensions\([Li et al\., 2025](https://arxiv.org/html/2610.00381#bib.bib45);[Leng et al\., 2025](https://arxiv.org/html/2610.00381#bib.bib33);[Wang et al\., 2026](https://arxiv.org/html/2610.00381#bib.bib44);[Ma et al\., 2026b](https://arxiv.org/html/2610.00381#bib.bib43)\), and reward shaping that pays for honest uncertainty\([Bani\-Harouni et al\., 2026](https://arxiv.org/html/2610.00381#bib.bib42);[Verma et al\., 2023](https://arxiv.org/html/2610.00381#bib.bib31);[Strong et al\., 2025](https://arxiv.org/html/2610.00381#bib.bib75)\)\. These methods work on the reward or the readout while leaving the output interface generative, so the calibrated quantity is still a string; badly shaped confidence rewards are additionally hackable\([Tan et al\., 2026](https://arxiv.org/html/2610.00381#bib.bib46)\), and calibration gains can be bought at the cost of task performance or verifiability\([Singh et al\., 2026](https://arxiv.org/html/2610.00381#bib.bib47);[Yaldiz et al\., 2026](https://arxiv.org/html/2610.00381#bib.bib48)\)\. The acronym RLCD also denotes reinforcement learning from contrastive distillation\([Yang et al\., 2024](https://arxiv.org/html/2610.00381#bib.bib112)\), an unrelated objective, and the two are kept apart here\. Our comparison differs in that the probability is the model’s output object from the first training step\.

### H\.4\. Deferral, human factors and regulation

Selective prediction and learning\-to\-defer methods consume a probability and decide whether to act on it, including for medical tasks\([Geifman and El\-Yaniv, 2017](https://arxiv.org/html/2610.00381#bib.bib21);[Strong et al\., 2025](https://arxiv.org/html/2610.00381#bib.bib75);[Sun et al\., 2026](https://arxiv.org/html/2610.00381#bib.bib76);[Vazhentsev et al\., 2026](https://arxiv.org/html/2610.00381#bib.bib62)\)\. Three empirical findings constrain how much that machinery can recover\. First, abstention is nearly orthogonal to accuracy: models rarely abstain where it matters clinically\([Machcha et al\., 2026](https://arxiv.org/html/2610.00381#bib.bib70);[Cocchieri et al\., 2026](https://arxiv.org/html/2610.00381#bib.bib71)\)\. Second, class\-dependent miscalibration makes abstention actively harmful for under\-represented conditions even when aggregate calibration error looks small\([Lechuga Lopez et al\., 2026](https://arxiv.org/html/2610.00381#bib.bib73)\), and conformal guarantees are weaker than they appear under clinical distribution shift\([Mehrtens et al\., 2023](https://arxiv.org/html/2610.00381#bib.bib74)\)\. Third, deferral does not remove error but relocates it: clinicians told that a model abstained miss more diagnoses and treatments than with no model at all\([Jabbour et al\., 2025](https://arxiv.org/html/2610.00381#bib.bib69)\), and cascading on the same confidence signal buys no cost\-efficiency on medical questions\([Fanconi and van der Schaar, 2025](https://arxiv.org/html/2610.00381#bib.bib72)\)\. Human\-factors work quantifies the stakes: biased or overconfident output degrades clinician accuracy even after AI\-literacy training\([Qazi et al\., 2026](https://arxiv.org/html/2610.00381#bib.bib77)\), over\-reliance persists when the advice is wrong\([Shekar et al\., 2025](https://arxiv.org/html/2610.00381#bib.bib80);[Dratsch et al\., 2023](https://arxiv.org/html/2610.00381#bib.bib79);[Jabbour et al\., 2023](https://arxiv.org/html/2610.00381#bib.bib78)\), and uncertainty expressed in words does not reliably reduce it\([Kim et al\., 2024](https://arxiv.org/html/2610.00381#bib.bib83);[Nicolson et al\., 2025](https://arxiv.org/html/2610.00381#bib.bib82);[Xu et al\., 2026](https://arxiv.org/html/2610.00381#bib.bib81);[Sikora et al\., 2026](https://arxiv.org/html/2610.00381#bib.bib91)\)\.

Evaluation and governance guidance now asks for exactly the probability this literature cannot supply\. Systematic reviews of clinical LLM evaluations find uncertainty rarely reported\([Bedi et al\., 2025](https://arxiv.org/html/2610.00381#bib.bib56)\); reporting standards make calibration and uncertainty reporting explicit\([Gallifant et al\., 2025](https://arxiv.org/html/2610.00381#bib.bib89);[Collins et al\., 2024](https://arxiv.org/html/2610.00381#bib.bib90)\); and regulators and professional bodies expect calibrated uncertainty and deferral behaviour to be described per case\([World Health Organization, 2024](https://arxiv.org/html/2610.00381#bib.bib86);[U\.S\. Food and Drug Administration, Center for Devices and Radiological Health, Digital Health Center of Excellence, 2026](https://arxiv.org/html/2610.00381#bib.bib84);[National Institute of Standards and Technology, 2024](https://arxiv.org/html/2610.00381#bib.bib87);[ECRI, 2026](https://arxiv.org/html/2610.00381#bib.bib88);[European Parliament and Council of the European Union, 2024](https://arxiv.org/html/2610.00381#bib.bib85)\)\. Recent work on clinical agents treats operational safety as a property of the whole system rather than of the model\([Kim et al\., 2026](https://arxiv.org/html/2610.00381#bib.bib58);[Zhang et al\., 2026b](https://arxiv.org/html/2610.00381#bib.bib57)\), and evaluation\-methodology work warns that apparent gains can be artefacts of the protocol\([Kapoor et al\., 2024](https://arxiv.org/html/2610.00381#bib.bib35)\)\. Our study is written to that standard: the comparison is interface\-controlled, the uncertainty is reported with paired intervals, and the places where the decision interface does not help are stated in the results rather than deferred to a limitations paragraph\.

## Appendix ILimitations

#### Bounded decisions only

The interface covers decisions with enumerable candidate sets, and this paper evaluates that scope: free\-form report generation and coordinate regression are not scored here\. For detection we also have to report that our implementation did not produce usable localization output, because the coordinate head collapsed to a constant box \(Appendix[A](https://arxiv.org/html/2610.00381#A1)\), so no claim is made about that family in either direction\. A medical model that must narrate remains a generative model for that part of its output; the claim of this paper is about the bounded decisions that dominate structured clinical prediction\.

#### Factorized supervision

The two models see the same 23,558 images but the decision interface factorizes their annotations into 155,729 decision targets \(Section[3\.3](https://arxiv.org/html/2610.00381#S3.SS3)\)\. No additional annotation is collected, but the supervision is denser, and denser supervision plausibly explains part of the late classification crossover \(Section[5\.1](https://arxiv.org/html/2610.00381#S5.SS1)\)\. Separating the interface effect from the density effect would require an ablation outside this protocol, so the crossover is reported as a hypothesis\. The predictive differences themselves are small relative to the test set: the classification interval\[−0\.0196,\+0\.1059\]\[\-0\.0196,\+0\.1059\]contains zero, which is why we claim a consistent direction across families rather than significance at a point\.

#### Evaluation scope

Four families, 726 held\-out states and three decision types are enough to test whether the interface effect is general, and not enough to establish clinical utility\. The probabilities reported here are model outputs on retrospective data; no claim is made about prospective performance, workflow integration or patient outcomes, and the human\-factors evidence on deferral and over\-reliance\([Jabbour et al\., 2023](https://arxiv.org/html/2610.00381#bib.bib78);[Dratsch et al\., 2023](https://arxiv.org/html/2610.00381#bib.bib79);[Qazi et al\., 2026](https://arxiv.org/html/2610.00381#bib.bib77);[Shekar et al\., 2025](https://arxiv.org/html/2610.00381#bib.bib80);[Kim et al\., 2024](https://arxiv.org/html/2610.00381#bib.bib83);[Nicolson et al\., 2025](https://arxiv.org/html/2610.00381#bib.bib82);[Xu et al\., 2026](https://arxiv.org/html/2610.00381#bib.bib81);[Sikora et al\., 2026](https://arxiv.org/html/2610.00381#bib.bib91)\)is a reason to test those questions rather than to assume them\.