Bayesian uncertainty estimation improves clinical decision making in medical AI agents

# Bayesian uncertainty estimation improves clinical decision making in medical AI agents
Source: [https://arxiv.org/html/2607.20582](https://arxiv.org/html/2607.20582)
\[1\]\\fnmFrederik\\surHauke \[1\]\\orgdivDepartment of Diagnostic and Interventional Radiology,\\orgnameUniversity Hospital RWTH Aachen,\\orgaddress\\cityAachen,\\countryGermany 2\]\\orgdivDepartment of Medical Oncology, National Center for Tumor Diseases \(NCT\),\\orgnameHeidelberg University Hospital,\\orgaddress\\cityHeidelberg,\\countryGermany 3\]\\orgdivElse Kröner Fresenius Center for Digital Health,\\orgnameTU Dresden,\\orgaddress\\cityDresden,\\countryGermany 4\]\\orgdivDepartment of Medicine I, Faculty of Medicine and University Hospital Carl Gustav Carus,\\orgnameTUD Dresden University of Technology,\\orgaddress\\cityDresden,\\countryGermany

\\fnmChristiane\\surKuhl\\fnmDyke\\surFerber\\fnmJakob Nikolas\\surKather\\fnmSven\\surNebelung\\fnmDaniel\\surTruhn\*\[\[\[

###### Abstract

Machine learning models for medical image analysis typically lack a reliable measure of confidence, limiting their use in ambiguous or atypical cases\. Here we show that Monte Carlo dropout, applied to a multi\-task chest\-radiograph classifier \(eight thoracic findings, 137,593 training images\), provides an epistemic uncertainty signal that tracks generalisation across training\-set scales and flags confident yet error\-prone predictions\. Adding this signal to the point prediction raised error\-detection AUROC from 0\.74 to 0\.77 \(Δ\\DeltaAUROC \+0\.023, 95 % CI \[\+0\.014, \+0\.033\]\)\. In a controlled2×22\\times 2factorial experiment, a clinical\-decision\-support agent exploited this uncertainty only when it was delivered as a binary error\-risk flag rather than as raw scores, cutting confident misdiagnoses on unreliable findings from 8\.5 % to 2\.7 %\. Epistemic uncertainty estimation thus carries decision\-relevant information beyond point predictions, but its value for downstream agents depends on how it is communicated\.

###### keywords:

Bayesian deep learning, uncertainty estimation, Monte Carlo dropout, chest radiography, clinical decision support

Imaging findings in clinical radiology are frequently subtle or ambiguous, particularly in early disease stages and in patients with multiple comorbidities[s13244\_023\_01521\_7](https://arxiv.org/html/2607.20582#bib.bib1)\. Diagnostic uncertainty is especially common in chest radiography: radiologists routinely signal doubt through hedging language such as “probably”, “possible”, or “may represent”[defined2025\_clinical\_uncertainty\_radiology\_reports](https://arxiv.org/html/2607.20582#bib.bib2), and this hedging directly influences decisions about additional imaging and follow\-up\.

As machine learning models assume a growing role in medical image interpretation, an analogous capacity to represent uncertainty becomes essential\. Many current models, including large language models \(LLMs\), produce incorrect predictions with high confidence and do not provide explicit uncertainty estimates[wen2024\_llm\_overconfidence](https://arxiv.org/html/2607.20582#bib.bib3),[cash2025\_llm\_confidence](https://arxiv.org/html/2607.20582#bib.bib4)\. Bayesian deep\-learning techniques address this shortcoming by attaching an explicit uncertainty estimate to each prediction\. Their value has been demonstrated across a range of clinical tasks, including chest radiography, tuberculosis segmentation, stroke analysis and COVID\-19 detection[bargagna2023\_uncertain\_labels\_cxr](https://arxiv.org/html/2607.20582#bib.bib5),[rajaraman2022\_tb\_uncertainty](https://arxiv.org/html/2607.20582#bib.bib6),[herzog2020\_stroke\_uncertainty](https://arxiv.org/html/2607.20582#bib.bib7),[calderon2021\_covid\_uncertainty](https://arxiv.org/html/2607.20582#bib.bib8),[bayesian\_cnn\_medical\_scarcity](https://arxiv.org/html/2607.20582#bib.bib9), and they have been used to flag out\-of\-distribution inputs and rare, underrepresented classes[yang2019\_bayesian\_retinopathy](https://arxiv.org/html/2607.20582#bib.bib10),[rezaei2023\_class\_imbalance](https://arxiv.org/html/2607.20582#bib.bib11)\.

Two principal forms of uncertainty are commonly distinguished\. Aleatoric uncertainty arises from inherent noise in the data, whereas epistemic uncertainty reflects the model’s lack of knowledge and can, in principle, be reduced with more training data[wimmer2023\_aleatoric\_epistemic](https://arxiv.org/html/2607.20582#bib.bib12)\. A standard classifier trained with cross\-entropy loss produces class probabilities that, in theory, approximate aleatoric uncertainty[gustafsson2020\_scalable\_bayesian](https://arxiv.org/html/2607.20582#bib.bib16),[kurz2022\_uncertainty\_systematic\_review](https://arxiv.org/html/2607.20582#bib.bib27),[gawlikowski2023\_uncertainty\_survey](https://arxiv.org/html/2607.20582#bib.bib14)\. These probabilities cannot, however, capture epistemic uncertainty, which means they give no indication of whether a prediction falls outside the model’s training experience[gawlikowski2023\_uncertainty\_survey](https://arxiv.org/html/2607.20582#bib.bib14),[gustafsson2020\_scalable\_bayesian](https://arxiv.org/html/2607.20582#bib.bib16)\. In practice, the problem is compounded by limited and imbalanced training data, a situation common in medical imaging: the model overfits, its output probabilities become poorly calibrated, and even post\-hoc corrections such as temperature scaling provide only partial relief[gal2016\_dropout\_bayesian](https://arxiv.org/html/2607.20582#bib.bib13),[gawlikowski2023\_uncertainty\_survey](https://arxiv.org/html/2607.20582#bib.bib14)\. Separate methods are therefore needed to estimate epistemic uncertainty from the model itself\.

Among such methods[gawlikowski2023\_uncertainty\_survey](https://arxiv.org/html/2607.20582#bib.bib14), deep ensembles demand training several independent networks[lakshminarayanan2017\_deep\_ensembles](https://arxiv.org/html/2607.20582#bib.bib15), whereas Monte Carlo \(MC\) dropout obtains uncertainty from repeated stochastic forward passes at inference time alone, offering a favourable trade\-off between cost and effectiveness[gustafsson2020\_scalable\_bayesian](https://arxiv.org/html/2607.20582#bib.bib16)\. MC dropout has also been shown to yield reliable uncertainty estimates for transformer architectures[shelmanov2021\_transformer\_uncertainty](https://arxiv.org/html/2607.20582#bib.bib17)\.

Deep learning systems for chest X\-ray interpretation are already in clinical use, with U\.S\. Food and Drug Administration \(FDA\)\-cleared models achieving areas under the curve \(AUCs\) above 0\.97 for comprehensive abnormality detection across cardiac, pulmonary and pleural findings, and improving physician accuracy[anderson2024\_dl\_improves\_cxr](https://arxiv.org/html/2607.20582#bib.bib25)\. Yet an estimated 3–5% of radiological interpretations contain errors or discrepancies[brady2017\_error\_discrepancy\_radiology](https://arxiv.org/html/2607.20582#bib.bib26), and deploying models that cannot signal when their own predictions are unreliable risks propagating confident but incorrect outputs into clinical workflows\. A systematic review concluded that the benefits of uncertainty estimation in AI\-clinician collaboration remain largely uninvestigated[kurz2022\_uncertainty\_systematic\_review](https://arxiv.org/html/2607.20582#bib.bib27)\.

Recent work suggests that clinical AI will increasingly rely on coordinating agents that invoke specialised models and integrate their outputs with clinical context[s41586\_023\_05881\_4](https://arxiv.org/html/2607.20582#bib.bib18),[Towards\_Generalist\_Biomedical\_AI](https://arxiv.org/html/2607.20582#bib.bib19),[ferber2025\_ai\_agent\_oncology](https://arxiv.org/html/2607.20582#bib.bib20),[survey\_llm\_agents\_medicine\_2025](https://arxiv.org/html/2607.20582#bib.bib21)\. Here, we equip a multi\-task chest\-radiograph classifier with MC\-dropout uncertainty estimation\. We evaluate this system in three steps: a data\-scaling experiment in which models are trained on progressively smaller fractions of the data, a per\-class reliability analysis, and a controlled factorial agent experiment\. We find that predictive standard deviation tracks how well the model generalises, rising and falling with validation loss, and that it flags individual predictions that are confident yet unreliable\. Disclosing this signal measurably improves detection of the model’s own errors; a clinical\-decision\-support agent, however, realises this value only when the signal is delivered as a pre\-digested binary flag rather than as raw numbers to interpret\. An overview of the full pipeline is shown in Figure[1](https://arxiv.org/html/2607.20582#S0.F1)\.

![Refer to caption](https://arxiv.org/html/2607.20582v1/figures/UncertaintyPaperOverview.png)Figure 1:Study overview\.a, A DINOv2 vision transformer classifies chest radiographs into eight thoracic findings, producing a single sigmoid activation per class\.b, MC\-dropout uncertainty estimation: multiple stochastic forward passes with active dropout yield the same point prediction plus a predictive standard deviation, flagging unstable outputs\.c, Agent experiment: each held\-out test case is routed to a clinical\-decision\-support agent under a2×22\\times 2design that crosses the error\-risk metric \(prediction only vs prediction\+\+uncertainty\) with how it is presented \(raw model outputs vs a binary error\-risk flag\); the agent’s commit\-versus\-escalate decisions are compared against the radiologist ground\-truth label\.## 1Results

### 1\.1Standard deviation co\-evolves with validation loss

We first examined how training\-set size influences optimisation, generalisation and epistemic uncertainty\. Across the ten subset fractions, training loss decreased monotonically with epoch and with data scale, and the 100 % training\-data run \(137,593 radiographs\) reached the lowest training loss \(Figure[2](https://arxiv.org/html/2607.20582#S1.F2)a\)\. Validation loss, in contrast, was non\-monotonic in epoch for every subset: it fell rapidly during the first 20–30 epochs, reached a minimum, and then rose for the remainder of training \(Figure[2](https://arxiv.org/html/2607.20582#S1.F2)b\)\. The full\-data run reached its minimum earliest and at the lowest value\. The subsequent rise, the canonical signature of overfitting, was unambiguous and present at every data scale\.

Predictive standard deviation, averaged across the eight findings on the held\-out validation set, followed almost the same U\-shape \(Figure[2](https://arxiv.org/html/2607.20582#S1.F2)c\): a sharp drop over the first∼\{\\sim\}20–30 epochs, a minimum close to the validation\-loss minimum, and a steady rise thereafter\. More training data lowered the standard deviation at every epoch, as expected, but it did not remove the late rise\. The important point is that validation loss and predictive standard deviation rose together once the model began to overfit: MC dropout detects the loss of generalisation even when the predictions themselves remain close to 0 or 1\.

![Refer to caption](https://arxiv.org/html/2607.20582v1/figures/loss_and_variance_curves.png)Figure 2:Training fit, generalisation and epistemic uncertainty co\-evolve across training\-set size\.Curves are coloured by training\-set fraction \(1–100 %\)\.a, Training loss decreases monotonically with epoch and data scale\.b, Validation loss on the full 34,860\-image validation set; minima are reached early and rise thereafter at every subset size\.c, Mean MC\-dropout predictive standard deviation \(30 forward passes, averaged across eight findings\) follows the same U\-shape as validation loss, establishing standard deviation as a generalisation\-dependent confidence signal\.
### 1\.2The signal generalises across all eight findings

To verify that the late\-epoch standard\-deviation rise is not driven by a single anomalous class, we disaggregated predictive standard deviation by finding \(Figure[3](https://arxiv.org/html/2607.20582#S1.F3)\)\. All eight classes exhibited the same trajectory: early collapse, a minimum near epoch 30–50, and a slow upward drift through epoch 150\. The separation between subset sizes was preserved in every class\. This confirms that the standard\-deviation signal reflects the model’s epistemic state rather than annotation noise in any particular finding\.

![Refer to caption](https://arxiv.org/html/2607.20582v1/figures/uncertainty_by_class_and_subset.png)Figure 3:The standard\-deviation U\-shape and data\-scale ordering generalise across all eight findings\.Per\-class MC\-dropout predictive standard deviation on the validation set versus training epoch, coloured by training\-set fraction \(0\.1–100 %\)\. Every class reproduces the U\-shape and monotonic ordering by data scale seen in Figure[2](https://arxiv.org/html/2607.20582#S1.F2)c, confirming that standard deviation is a well\-behaved per\-class confidence signal\.Together, these results show that MC\-dropout predictive standard deviation is a reliable confidence signal across all eight findings\. It captures something the plain network output cannot: as the model starts to overfit and generalise less well, the standard deviation rises, even when the prediction itself still looks confident\.

### 1\.3High standard deviation flags confidently wrong predictions

We next assessed whether predictive standard deviation is informative about individual prediction errors\. For each of the eight findings, we plotted the deterministic absolute error\|y^−y\|\|\\hat\{y\}\-y\|against MC\-dropout standard deviation on the full validation set \(n=34,860n=34\{,\}860; Figure[4](https://arxiv.org/html/2607.20582#S1.F4)\)\. Most predictions clustered in the low\-standard\-deviation, low\-error region, but every class exhibited a tail of high\-error points at high standard deviation, including predictions the model reported with high confidence, where disclosing the uncertainty adds information that cannot be recovered from the point prediction\. We quantify this error\-detection value directly in the agent experiment below \(Figure[5](https://arxiv.org/html/2607.20582#S1.F5)\)\.

![Refer to caption](https://arxiv.org/html/2607.20582v1/figures/reliability_diagram.png)Figure 4:Predictive standard deviation is informative about prediction errors for every finding\.Per\-class scatter of absolute error\|y^−y\|\|\\hat\{y\}\-y\|versus MC\-dropout standard deviation \(n=34,860n=34\{,\}860radiographs, 30 MC samples\)\.a–h, One panel per finding; predictions with high standard deviation are enriched for large errors, motivating disclosure of the uncertainty signal alongside the point prediction\.
### 1\.4Uncertainty improves error detection

We first quantified, on the held\-out test set, whether the uncertainty improves detection of the model’s own errors beyond the point prediction alone\. As described in Methods, every ordinal grade was binarised at threshold\>0\>0across all splits \(None negative;\(\+\)\(\+\)through\+⁣\+⁣\+\+\+\+positive\), and a prediction counted as an error when it fell on the wrong side of 0\.5 relative to this binary ground truth\. Averaging across all eight findings, ranking predictions by confidence alone \(the distance of the sigmoid output from 0\.5\) detected the model’s errors with an AUROC of 0\.74; adding the MC\-dropout uncertainty raised this to 0\.77 \(Δ\\DeltaAUROC\+0\.023\+0\.023, 95 % CI \[\+0\.014\+0\.014,\+0\.033\+0\.033\],p<0\.001p<0\.001; case\-clustered bootstrap; Figure[5](https://arxiv.org/html/2607.20582#S1.F5)a, inset\)\. The uncertainty signal thus carries error\-relevant information that the point prediction does not\.

Whether the agent could act on this information, however, depended on how it was presented\. We routed each case to the agent under a2×22\\times 2design that crossed the error\-risk metric \(prediction only vs prediction\+\+uncertainty\) with its representation: either the raw per\-class model outputs, which the agent must interpret itself, or a binary error\-risk flag pre\-computed by the model \(a validation\-calibrated error detector thresholded at its Youden\-optimal point\)\. Given the raw prediction\-plus\-uncertainty numbers, the agent’s error\-catching sensitivity was 0\.63, leaving its operating point well below the achievable prediction\-plus\-uncertainty frontier and no better than an optimal use of the prediction alone \(Figure[5](https://arxiv.org/html/2607.20582#S1.F5)a\)\. Given the same information as a binary flag, the agent reached the frontier \(sensitivity 0\.79\), a gain of\+0\.16\+0\.16\(95 % CI\[\+0\.13,\+0\.18\]\[\+0\.13,\\,\+0\.18\]\) over the raw representation at the prediction\-plus\-uncertainty level, and\+0\.11\+0\.11\(\[\+0\.08,\+0\.14\]\[\+0\.08,\\,\+0\.14\]\) at the prediction\-only level \(bothp<0\.001p<0\.001\)\. The agent could not optimally interpret the raw model outputs, but followed the pre\-digested decision to the model’s own error\-detection optimum\.

This sharpened the agent’s caution where it mattered\. Splitting each finding by whether the model was reliable \(below\-median uncertainty\) or unreliable \(above\-median\), the flag concentrated escalation on the unreliable findings \(prediction\+\+uncertainty:66%→84%66\\,\\%\\to 84\\,\\%on unreliable, with little change on reliable; Figure[5](https://arxiv.org/html/2607.20582#S1.F5)b\)\. On those unreliable findings, the fraction of confident misdiagnoses \(cases where the agent committed to a call that disagreed with the radiologist ground\-truth label\) fell from8\.5%8\.5\\,\\%with the raw numbers to2\.7%2\.7\\,\\%with the flag \(−5\.8\-5\.8percentage points,p<0\.001p<0\.001\), with little change on reliable findings \(Figure[5](https://arxiv.org/html/2607.20582#S1.F5)c\)\.

![Refer to caption](https://arxiv.org/html/2607.20582v1/figures/uses_and_roc_4arm.png)Figure 5:The agent reaches the error\-detection optimum only when the uncertainty is delivered as a binary flag\.Held\-out test set; a prediction is an error when it falls on the wrong side of 0\.5 relative to the radiologist ground\-truth label\. Two factors are crossed: the X\-Ray model’s error\-risk metric \(prediction only, red; prediction\+\+uncertainty, blue\) and how it is presented to the agent \(raw numbers,×\\times; binary error\-risk flag,∙\\bullet\)\.a, Diagnosis\-error\-detection ROC: curves are the X\-Ray model’s error\-ranking frontiers; markers are the agent’s actual operating points \(sensitivity for catching the model’s errors vs specificity for passing its correct calls\)\. Both flag agents \(∙\\bullet\) sit on their model’s frontier; both raw agents \(×\\times\) fall below it\. Inset, X\-Ray\-model error\-detection AUROC \(0\.740\.74vs\.0\.770\.77;p<0\.001p<0\.001\)\.b, Escalation rate, split by whether the model is reliable \(below\-median uncertainty\) or unreliable \(above\-median\); the flag concentrates escalation on unreliable findings\.c, Confident misdiagnoses \(the agent commits to a call that disagrees with the ground\-truth label\), by the same stratification; the flag reduces them on unreliable findings \(prediction\+\+uncertainty8\.5%→2\.7%8\.5\\,\\%\\to 2\.7\\,\\%,p<0\.001p<0\.001\) with little change where the model is reliable\. Bars, mean; whiskers, case\-clustered bootstrap 95 % CI\.
### 1\.5Agents struggle to judge raw uncertainty

The agent’s shortfall was therefore one of interpretation, not of information: the identical error\-relevant signal was present in both representations, yet only the formalized flag let the agent reach the model’s error\-detection optimum, whereas the raw numbers left it well short\. This points to a design principle for agentic clinical AI: the numerical reasoning belongs in the specialist model, which should hand the agent a calibrated, pre\-digested decision rather than raw scores to interpret\.

## 2Discussion

We have shown that equipping a medical image classifier with MC\-dropout uncertainty estimation and disclosing that signal to a clinical\-decision\-support agent yields measurably better decisions, provided the signal is delivered in a form the agent can act on\. Our approach couples a vision\-transformer classifier that attaches an epistemic uncertainty estimate to every prediction with a reliability\-plane analysis showing that predictions with high standard deviation are enriched for errors; a controlled factorial experiment on a held\-out test set then demonstrated that this uncertainty improves detection of the model’s own errors, but that the agent captures the gain only when the error risk is pre\-digested into a binary flag rather than handed over as raw numbers to interpret\.

The agent experiment is, to our knowledge, the first controlled demonstration that the representation of a model’s uncertainty, not merely its availability, governs whether an AI agent can use it in a simulated clinical\-support setting\. Because the four conditions were identical except for the error\-risk metric and its representation, the shift in the agent’s operating point \(Figure[5](https://arxiv.org/html/2607.20582#S1.F5)\) is attributable to those factors alone\. Handed the raw model outputs, the agent under\-used the signal and stayed below the achievable frontier; handed a binary flag, it selectively escalated the findings the model was unsure about, ordering verification steps such as a radiologist second read where they were warranted while leaving confident, reliable calls untouched, and thereby reduced confident misdiagnoses on unreliable findings without over\-escalating reliable ones\.

More broadly, as language\-model agents are increasingly placed downstream of quantitative predictors, this argues for keeping the numerical reasoning in the specialist model and handing the agent a calibrated, formalized decision; a consideration that extends beyond uncertainty to any continuous model output\.

The data\-scaling and per\-class analyses provide the mechanistic foundation for this result\. Predictive standard deviation followed validation loss closely across training\-set sizes \(Figure[2](https://arxiv.org/html/2607.20582#S1.F2)\), rising during overfitting even as the deterministic predictions stayed pinned near 0 or 1\. This pattern held consistently across all eight findings \(Figure[3](https://arxiv.org/html/2607.20582#S1.F3)\), confirming that MC dropout tracks generalisation degradation at the per\-class level\. The reliability\-plane analysis \(Figure[4](https://arxiv.org/html/2607.20582#S1.F4)\) then translated this aggregate signal into case\-level risk stratification, flagging predictions whose instability under dropout is not apparent from the point prediction alone\. These findings connect to prior work on uncertainty\-based out\-of\-distribution detection[yang2019\_bayesian\_retinopathy](https://arxiv.org/html/2607.20582#bib.bib10),[rezaei2023\_class\_imbalance](https://arxiv.org/html/2607.20582#bib.bib11)and to the broader challenge of overconfidence in both human decision making and large language models[wen2024\_llm\_overconfidence](https://arxiv.org/html/2607.20582#bib.bib3)\.

Our study has several limitations\. Our evaluation used a single imaging modality and a single institutional cohort; extension to other modalities, pathologies and external datasets is needed to establish generalisability\. MC dropout, while practical, could be complemented or replaced by more expressive methods such as deep ensembles[lakshminarayanan2017\_deep\_ensembles](https://arxiv.org/html/2607.20582#bib.bib15)or variational inference[kingma2013\_vae](https://arxiv.org/html/2607.20582#bib.bib24)\. The agent’s commit\-versus\-escalate decisions were read from a structured field it emitted rather than adjudicated by clinician assessors: the fully automated pipeline evaluated 2,000 agent consultations \(500 cases×\\timesfour conditions;∼\{\\sim\}16,000 finding\-level decisions\), a sample size out of scope for clinician adjudication\. A formal clinical validation therefore remains future work, but because the uncertainty advantage acts on the minority of cases where the model is unreliable, a clinical deployment operating at scale would encounter many more such cases and stands to benefit correspondingly more\. Looking ahead, these results argue for integrating uncertainty\-aware models into AI\-assisted medical image analysis, particularly as clinical workflows increasingly delegate interpretation to autonomous AI agents[ferber2025\_ai\_agent\_oncology](https://arxiv.org/html/2607.20582#bib.bib20)\. We have shown that a model’s apparent confidence and the stability of its predictions can come apart, and that measuring this stability requires nothing more than repeated forward passes at inference time\. This makes uncertainty disclosure a concrete, readily deployable signal that can inform downstream decision support, though its clinical value remains to be validated with human clinicians\.

## 3Methods

### 3\.1Dataset, splits and label schema

We used the publicly available TAIX\-Ray Thorax chest\-radiograph cohort[taixray2026\_dataset](https://arxiv.org/html/2607.20582#bib.bib23)\. All experiments used a patient\-stratified split comprising 137,593 training, 34,860 validation and 42,928 test radiographs, with no patient appearing in more than one split\. The data\-scaling and reliability analyses were performed on the validation set; the agent experiment was performed on the held\-out test set\. Every radiograph carries an ordinal annotator grade for eight findings\. HeartSize is graded on a four\-level scale \(Normal, Borderline, Enlarged, Massively enlarged\), while the remaining seven findings, Pulmonary Congestion, Pleural Effusion \(Left/Right\), Pulmonary Opacities \(Left/Right\), and Atelectasis \(Left/Right\), each use five grades \(None / \(\+\) / \+ / \+\+ / \+\+\+\)\. For the training objective the grades were binarised at threshold\>0\>0; the raw 0\-4 grades were preserved on disk and re\-coupled to inference outputs post\-hoc at analysis time, including the annotator\-uncertain label \(grade 1: Borderline = “\(\+\)”\)\.

Radiographs were centre\-cropped to448×448448\\times 448px and per\-image z\-normalised\. Training augmentations included random cropping, vertical flipping and axis transposition; no augmentations were applied at validation or test time\.

### 3\.2DINOv2 backbone with MC dropout

The classifier is a DINOv2 Vision Transformer \(ViT\)\-S/14[oquab2023\_dinov2](https://arxiv.org/html/2607.20582#bib.bib22)\(∼\{\\sim\}21 M parameters\), fine\-tuned from the public self\-supervised checkpoint on the eight binarised labels via a single linear head\. The training objective was per\-class binary cross\-entropy; optimisation used AdamW \(learning rate1×10−61\\times 10^\{\-6\}, weight decay1×10−21\\times 10^\{\-2\}\) in mixed\-precision 16\-bit floating point \(FP16\)\.

For epistemic uncertainty estimation, all dropout layers inside the transformer were enabled at inference time with ratep=0\.1p=0\.1, and an additional dropout layer was inserted before the linear head\. Throughout this paper we report MC\-dropout estimates from 30 stochastic forward passes per image[gal2016\_dropout\_bayesian](https://arxiv.org/html/2607.20582#bib.bib13)\. Per\-class predictive standard deviation is the standard deviation of the sigmoid\-space MC samples; per\-class predictive entropy is the binary entropy of the MC mean\. The deterministic prediction is a single forward pass with dropout disabled, decoupling what the model predicts from how stable that prediction is under stochastic perturbation\.

### 3\.3Data\-scaling experiment

To examine the joint behaviour of fit, generalisation and epistemic uncertainty as a function of training\-set size, we trained ten independent classifiers on nested random subsets of the training split \(0\.1–100 %, spanning∼\{\\sim\}138 to 137,593 radiographs\)\. Each run lasted 150 epochs with batch size 48 and identical hyper\-parameters; the validation set was always the full 34,860\-image split\.

Every five epochs, 30\-sample MC\-dropout inference was run on the full validation set and the mean predictive standard deviation, entropy and prediction were recorded per class \(Figure[2](https://arxiv.org/html/2607.20582#S1.F2), Figure[3](https://arxiv.org/html/2607.20582#S1.F3)\)\.

### 3\.4Reliability analysis on the validation set

For the reliability analysis we used a single converged checkpoint \(epoch 37\) and ran 30\-sample MC\-dropout inference on the full validation set \(34,860 radiographs\)\. For each image\-class pair we recorded the deterministic prediction, MC\-dropout standard deviation, binary target and absolute error\|y^−y\|\|\\hat\{y\}\-y\|\(Figure[4](https://arxiv.org/html/2607.20582#S1.F4)\)\. A prediction was counted as an error when it fell on the wrong side of 0\.5 relative to the binarised ground\-truth label\.

### 3\.5Factorial agent experiment

For the agent experiment we drew a representative sample of 500 cases uniformly at random \(fixed seed\) from the held\-out test split, yielding500×8=4,000500\\times 8=4\{,\}000finding\-level predictions \(493 cases had all four conditions successfully parsed\)\. For each case GPT\-5\.1 \(OpenAI\) generated a realistic clinician request note from the image and ground\-truth grades, without reference to imaging findings\.

Each case was routed to a GPT\-5\.1\-based clinical\-decision\-support agent under a2×22\\times 2design crossing the error\-risk metric with its representation\. The metric was either the prediction alone or the prediction together with MC\-dropout uncertainty\. The representation was either raw \(the per\-class deterministic prediction, and, for the prediction\-plus\-uncertainty metric, the MC\-dropout standard deviation with a per\-class orientation block \(validation\-set minimum, 75th percentile, maximum\)\) or a binary error\-risk flag\. The flag was produced by a logistic\-regression error detector \(on standardised confidence for the prediction\-only metric, or on standardised confidence and uncertainty for the prediction\-plus\-uncertainty metric\) fitted on the validation set and thresholded at its Youden\-optimal operating point, marking each finding “elevated risk” or “normal”\. All four prompts were byte\-identical except for this data block\. The rubric asked the agent to decide, for each finding, to commit \(accept the model’s call and fold it into decisive management\) or escalate \(order a specific confirmatory or better\-suited next step, e\.g\. radiologist second read, point\-of\-care ultrasound, echocardiography, repeat radiograph or CT\)\.

For each finding in each condition, the agent emitted its decision \(commit or escalate\) in a structured output field\. A finding was counted as a confident misdiagnosis when the agent committed to a call that disagreed with the ground\-truth label\. Findings were stratified as reliable or unreliable by splitting each finding at its median MC\-dropout uncertainty across the sample\.

The X\-Ray model’s error\-detection frontiers \(Figure[5](https://arxiv.org/html/2607.20582#S1.F5)a\) are the ROC of two scores against the ground\-truth error label: prediction only \(−\|y^−0\.5\|\-\|\\hat\{y\}\-0\.5\|\) and prediction plus uncertainty \(the out\-of\-sample probability of a 5\-fold cross\-validated logistic regression on the standardised confidence and uncertainty\)\. The agent’s actual decisions define operating points \(sensitivity, specificity\) on the same axes\.

### 3\.6Statistics

Confidence intervals andppvalues were obtained by a case\-clustered bootstrap that resamples cases \(3,000 resamples\), respecting the non\-independence of the eight findings within a case\. Theppvalue for the error\-detection AUROC difference \(prediction\-plus\-uncertainty minus prediction\-only\) is the two\-sided bootstrap tail probability\. The error bars on the escalation\-rate and confident\-misdiagnosis panels \(Figure[5](https://arxiv.org/html/2607.20582#S1.F5)b,c\) are the 95 % confidence interval of this bootstrap\. No case was excluded post hoc\. P values below 0\.001 are reported asp<0\.001p<0\.001\.

The authors acknowledge the support of the European High Performance Computing Joint Undertaking \(JU\) under grant agreement No 101250682\.

## Declarations

- •FundingFunded by the European Union\. This work received funding from the European High Performance Computing Joint Undertaking \(JU\) under grant agreement No 101250682 \(JAIF\)\. DT is supported by the German Ministry of Research, Technology and Space \(TRANSFORM LIVER – 031L0312C, DECIPHER\-M – 01KD2420B\), the Deutsche Forschungsgemeinschaft \(DFG, 515639690\), and the European Union \(Horizon Europe, ODELIA – GA 101057091, ERC Starting Grant SAGMA – GA 101222556\)\.
- •Conflict of interestDT holds shares in StratifAI and Synagen\. He has received honoraria from Bayer, AstraZeneca, Philips, Roche, Pfizer, and Gilead\. JNK declares ongoing consulting services for AstraZeneca, Panakeia, and Bioptimus\. Furthermore, he holds shares in StratifAI, Synagen, and Spira Labs, has received an institutional research grant from GSK and AstraZeneca, as well as honoraria from AstraZeneca, Bayer, Daiichi Sankyo, Eisai, Janssen, Merck, MSD, BMS, Roche, Pfizer, and Fresenius\. DF holds shares and is an employee at Synagen and has received a research grant from OpenAI\. All other authors declare no conflicts of interest\.
- •Availability of data and materialsThe UKA Thorax chest\-radiograph cohort \(taIX\-Ray\) is publicly available[taixray2026\_dataset](https://arxiv.org/html/2607.20582#bib.bib23)\.
- •
- •Authors’ contributionsFrederik Hauke: Investigation, Formal Analysis, Writing – Original Draft\. Patrick Wienholt: Writing – Review & Editing\. Christiane Kuhl: Writing – Review & Editing\. Dyke Ferber: Writing – Review & Editing\. Jakob Nikolas Kather: Writing – Review & Editing\. Sven Nebelung: Writing – Review & Editing\. Daniel Truhn: Conceptualization, Writing – Review & Editing\.

## References

- \[1\]Zhang L, Wen X, Li JW, Jiang X, Yang XF, Li M\. Diagnostic error and bias in the department of radiology: a pictorial essay\.Insights into Imaging14, 163 \(2023\)\.[https://doi\.org/10\.1186/s13244\-023\-01521\-7](https://doi.org/10.1186/s13244-023-01521-7)
- \[2\]Rabaey P, Moon JH, Lee JO, Kim MG, Yoon H, Demeester T, Choi E\. Modeling clinical uncertainty in radiology reports: from explicit uncertainty markers to implicit reasoning pathways\. In:Proceedings of the Language Resources and Evaluation Conference \(LREC\), 11852–11873\. European Language Resources Association \(ELRA\), 2026\.[https://doi\.org/10\.63317/2i8nrgcnb52p](https://doi.org/10.63317/2i8nrgcnb52p)
- \[3\]Wen B, Xu C, Han B, Wolfe R, Wang LL, Howe B\. Mitigating overconfidence in large language models: a behavioral lens on confidence estimation and calibration\. In:NeurIPS 2024 Workshop on Behavioral Machine Learning, 2024\.[https://openreview\.net/forum?id=y9UdO5cmHs](https://openreview.net/forum?id=y9UdO5cmHs)
- \[4\]Cash TN, Oppenheimer DM, Christie S, Devgan M\. Quantifying uncert\-AI\-nty: testing the accuracy of LLMs’ confidence judgments\.Memory & Cognition54, 375–400 \(2025\)\.[https://doi\.org/10\.3758/s13421\-025\-01755\-4](https://doi.org/10.3758/s13421-025-01755-4)
- \[5\]Yang HY, Yang J, Pan Y, Cao K, Song Q, Gao F, Yin Y\. Learn to be uncertain: leveraging uncertain labels in chest X\-rays with Bayesian neural networks\. In:Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\) Workshops, 5–8, 2019\.[https://openaccess\.thecvf\.com/content\_CVPRW\_2019/html/Uncertainty\_and\_Robustness\_in\_Deep\_Visual\_Learning/Yang\_Learn\_To\_Be\_Uncertain\_Leveraging\_Uncertain\_Labels\_In\_Chest\_X\-rays\_CVPRW\_2019\_paper\.html](https://openaccess.thecvf.com/content_CVPRW_2019/html/Uncertainty_and_Robustness_in_Deep_Visual_Learning/Yang_Learn_To_Be_Uncertain_Leveraging_Uncertain_Labels_In_Chest_X-rays_CVPRW_2019_paper.html)
- \[6\]Rajaraman S, Zamzmi G, Yang F, Xue Z, Jaeger S, Antani SK\. Uncertainty quantification in segmenting tuberculosis\-consistent findings in frontal chest X\-rays\.Biomedicines10\(6\), 1323 \(2022\)\.[https://doi\.org/10\.3390/biomedicines10061323](https://doi.org/10.3390/biomedicines10061323)
- \[7\]Herzog L, Murina E, Dürr O, Wegener S, Sick B\. Integrating uncertainty in deep neural networks for MRI based stroke analysis\.Medical Image Analysis65, 101790 \(2020\)\.[https://doi\.org/10\.1016/j\.media\.2020\.101790](https://doi.org/10.1016/j.media.2020.101790)
- \[8\]Calderon\-Ramirez S, Yang S, Moemeni A, Colreavy\-Donnelly S, Elizondo DA, Oala L, Rodríguez\-Capitán J, Jiménez\-Navarro M, López\-Rubio E, Molina\-Cabello MA\. Improving uncertainty estimation with semi\-supervised deep learning for COVID\-19 detection using chest X\-ray images\.IEEE Access9, 85442–85454 \(2021\)\.[https://doi\.org/10\.1109/ACCESS\.2021\.3085418](https://doi.org/10.1109/ACCESS.2021.3085418)
- \[9\]Bargagna F, De Santi LA, Martini N, Genovesi D, Favilli B, Vergaro G, Emdin M, Giorgetti A, Positano V, Santarelli MF\. Bayesian convolutional neural networks in medical imaging classification: a promising solution for deep learning limits in data scarcity scenarios\.Journal of Digital Imaging36, 2567–2577 \(2023\)\.[https://doi\.org/10\.1007/s10278\-023\-00897\-8](https://doi.org/10.1007/s10278-023-00897-8)
- \[10\]Filos A, Farquhar S, Gomez AN, Rudner TGJ, Kenton Z, Smith L, Alizadeh M, de Kroon A, Gal Y\. A systematic comparison of Bayesian deep learning robustness in diabetic retinopathy tasks\. Preprint at[https://arxiv\.org/abs/1912\.10481](https://arxiv.org/abs/1912.10481)\(2019\)\.
- \[11\]Rezaei M, Näppi JJ, Bischl B, Yoshida H\. Bayesian uncertainty estimation for detection of long\-tailed and unseen conditions in medical images\.Journal of Medical Imaging10\(5\), 054501 \(2023\)\.[https://doi\.org/10\.1117/1\.JMI\.10\.5\.054501](https://doi.org/10.1117/1.JMI.10.5.054501)
- \[12\]Wimmer L, Sale Y, Hofman P, Bischl B, Hüllermeier E\. Quantifying aleatoric and epistemic uncertainty in machine learning: are conditional entropy and mutual information appropriate measures? In:Proceedings of the 39th Conference on Uncertainty in Artificial Intelligence \(UAI\), PMLR216, 2282–2292, 2023\.[https://proceedings\.mlr\.press/v216/wimmer23a\.html](https://proceedings.mlr.press/v216/wimmer23a.html)
- \[13\]Gal Y, Ghahramani Z\. Dropout as a Bayesian approximation: representing model uncertainty in deep learning\. In:Proceedings of the 33rd International Conference on Machine Learning \(ICML\), PMLR48, 1050–1059, 2016\.[https://proceedings\.mlr\.press/v48/gal16\.html](https://proceedings.mlr.press/v48/gal16.html)
- \[14\]Gawlikowski J, Tassi CRN, Ali M, Lee J, Humt M, Feng J, Kruspe A, Triebel R, Jung P, Roscher R, Shahzad M, Yang W, Bamler R, Zhu XX\. A survey of uncertainty in deep neural networks\.Artificial Intelligence Review56, 1513–1589 \(2023\)\.[https://doi\.org/10\.1007/s10462\-023\-10562\-9](https://doi.org/10.1007/s10462-023-10562-9)
- \[15\]Lakshminarayanan B, Pritzel A, Blundell C\. Simple and scalable predictive uncertainty estimation using deep ensembles\. In:Advances in Neural Information Processing Systems \(NeurIPS\)30, 6405–6416, 2017\.[https://proceedings\.neurips\.cc/paper/2017/hash/9ef2ed4b7fd2c810847ffa5fa85bce38\-Abstract\.html](https://proceedings.neurips.cc/paper/2017/hash/9ef2ed4b7fd2c810847ffa5fa85bce38-Abstract.html)
- \[16\]Gustafsson FK, Danelljan M, Schön TB\. Evaluating scalable Bayesian deep learning methods for robust computer vision\. In:Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\) Workshops, 318–319, 2020\.[https://doi\.org/10\.1109/CVPRW50498\.2020\.00167](https://doi.org/10.1109/CVPRW50498.2020.00167)
- \[17\]Shelmanov A, Tsymbalov E, Puzyrev D, Fedyanin K, Panchenko A, Panov M\. How certain is your Transformer? In:Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics \(EACL\), 1833–1840, 2021\.[https://doi\.org/10\.18653/v1/2021\.eacl\-main\.157](https://doi.org/10.18653/v1/2021.eacl-main.157)
- \[18\]Moor M, Banerjee O, Abad ZSH, Krumholz HM, Leskovec J, Topol EJ, Rajpurkar P\. Foundation models for generalist medical artificial intelligence\.Nature616, 259–265 \(2023\)\.[https://doi\.org/10\.1038/s41586\-023\-05881\-4](https://doi.org/10.1038/s41586-023-05881-4)
- \[19\]Tu T, Azizi S, Driess D, Schaekermann M, Amin M, Chang PC, Carroll A, Lau C, Tanno R, Ktena I, Palepu A, Mustafa B, Chowdhery A, Liu Y, Kornblith S, Fleet D, Mansfield P, Prakash S, Wong R, Virmani S, Semturs C, Mahdavi SS, Green B, Dominowska E, Aguera y Arcas B, Barral J, Webster D, Corrado GS, Matias Y, Singhal K, Florence P, Karthikesalingam A, Natarajan V\. Towards generalist biomedical AI\.NEJM AI1\(3\), AIoa2300138 \(2024\)\.[https://doi\.org/10\.1056/AIoa2300138](https://doi.org/10.1056/AIoa2300138)
- \[20\]Ferber D, El Nahhas OSM, Wölflein G, Wiest IC, Clusmann J, Leßmann ME, Foersch S, Lammert J, Tschochohei M, Jäger D, Salto\-Tellez M, Schultz N, Truhn D, Kather JN\. Development and validation of an autonomous artificial intelligence agent for clinical decision\-making in oncology\.Nature Cancer6, 1337–1349 \(2025\)\.[https://doi\.org/10\.1038/s43018\-025\-00991\-6](https://doi.org/10.1038/s43018-025-00991-6)
- \[21\]Wang W, Ma Z, Wang Z, Wu C, Ji J, Chen W, Li X, Yuan Y\. A survey of LLM\-based agents in medicine: how far are we from Baymax? In:Findings of the Association for Computational Linguistics: ACL 2025, 10345–10359, 2025\.[https://doi\.org/10\.18653/v1/2025\.findings\-acl\.539](https://doi.org/10.18653/v1/2025.findings-acl.539)
- \[22\]Oquab M, Darcet T, Moutakanni T, Vo H, Szafraniec M, Khalidov V, Fernandez P, Haziza D, Massa F, El\-Nouby A, Assran M, Ballas N, Galuba W, Howes R, Huang PY, Li SW, Misra I, Rabbat M, Sharma V, Synnaeve G, Xu H, Jegou H, Mairal J, Labatut P, Joulin A, Bojanowski P\. DINOv2: learning robust visual features without supervision\.Transactions on Machine Learning Research\(2024\)\.[https://openreview\.net/forum?id=a68SUt6zFt](https://openreview.net/forum?id=a68SUt6zFt)
- \[23\]Truhn, D\., Geiger, D\., Siepmann, R\., von der Stück, M\.S\., Bressem, K\.K\., Kather, J\.N\., Kuhl, C\., Müller\-Franzes, G\., & Nebelung, S\. A comprehensive bedside chest radiography dataset with structured, itemized and graded radiologic reports\.Scientific Data13, 632 \(2026\)\.[https://doi\.org/10\.1038/s41597\-026\-07271\-7](https://doi.org/10.1038/s41597-026-07271-7)
- \[24\]Kingma DP, Welling M\. Auto\-encoding variational Bayes\. In:Proceedings of the 2nd International Conference on Learning Representations \(ICLR\), 2014\. Preprint at[https://arxiv\.org/abs/1312\.6114](https://arxiv.org/abs/1312.6114)\(2013\)\.
- \[25\]Anderson PG, Tarder\-Stoll H, Alpaslan M, Keathley N, Levin DL, Venkatesh S, Bartel E, Sicular S, Howell S, Lindsey RV, Jones RM\. Deep learning improves physician accuracy in the comprehensive detection of abnormalities on chest X\-rays\.Scientific Reports14, 25151 \(2024\)\.[https://doi\.org/10\.1038/s41598\-024\-76608\-2](https://doi.org/10.1038/s41598-024-76608-2)
- \[26\]Brady AP\. Error and discrepancy in radiology: inevitable or avoidable?Insights into Imaging8, 171–182 \(2017\)\.[https://doi\.org/10\.1007/s13244\-016\-0534\-1](https://doi.org/10.1007/s13244-016-0534-1)
- \[27\]Kurz A, Hauser K, Mehrtens HA, Krieghoff\-Henning E, Hekler A, Kather JN, Fröhling S, von Kalle C, Brinker TJ\. Uncertainty estimation in medical image classification: systematic review\.JMIR Medical Informatics10\(8\), e36427 \(2022\)\.[https://doi\.org/10\.2196/36427](https://doi.org/10.2196/36427)