Lost in the Folds: When Cross-Validation Is Not a Deep Ensemble for Uncertainty Estimation
Summary
This paper compares cross-validation ensembles to deep ensembles for uncertainty estimation in medical image segmentation. Deep ensembles outperform cross-validation ensembles in calibration and failure detection, while cross-validation ensembles better approximate inter-rater variability.
View Cached Full Text
Cached at: 05/21/26, 06:12 PM
Paper page - Lost in the Folds: When Cross-Validation Is Not a Deep Ensemble for Uncertainty Estimation
Source: https://huggingface.co/papers/2605.18329
Abstract
Deep ensembles trained with fixed data and varying seeds outperform cross-validation ensembles in calibration and failure detection for medical image segmentation, while cross-validation ensembles better approximate inter-rater variability.
Ensemble disagreement is widely used as a proxy forepistemic uncertaintyin medical image segmentation. In practice, many studies form ensembles via K-foldcross-validation(CV), yet refer to them as ``deep ensembles’’ (DE). Because CV members are trained on different data subsets, their disagreement mixes seed-driven variability with data-exposure effects, which can change how uncertainty should be interpreted. We audit recent segmentation uncertainty studies and find that terminology--implementation mismatches are common. We then compare a standard 5-fold CV ensemble to a 5-member DE (fixed training set, different random seeds) under otherwise identical configurations on three multi-rater segmentation datasets spanning three modalities. We evaluate uncertainty forcalibration,failure detection,ambiguity modeling, and robustness underdistribution shift. DE match segmentation accuracy while improvingcalibrationandfailure detection, whereas CV ensembles sometimes correlate more strongly with inter-rater variability on the studied datasets. Thus, ensemble construction should be chosen to match the research question: DE for reliability-oriented use (e.g., selective referral/failure detection) and CV ensembles as a proxy for ambiguity. We provide a lightweightnnU-Netmodification enabling DE training within the default pipeline.
View arXiv pageView PDFGitHub0Add to collection
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2605.18329 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2605.18329 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2605.18329 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
Calibration, Uncertainty Communication, and Deployment Readiness in CKD Risk Prediction: A Framework Evaluation Study
This study evaluates five machine learning classifiers for chronic kidney disease risk prediction, finding that near-perfect internal performance fails under distribution shift. It emphasizes the need for calibration stability and conformal coverage transfer before clinical deployment.
A better method for identifying overconfident large language models
MIT researchers developed a new method for identifying overconfident LLMs by measuring cross-model disagreement across similar models, rather than relying solely on self-consistency metrics. This approach better captures epistemic uncertainty and more accurately identifies unreliable predictions in high-stakes applications.
TwinTrack: Post-hoc Multi-Rater Calibration for Medical Image Segmentation
TwinTrack is a post-hoc calibration framework for pancreatic cancer segmentation that aligns ensemble model probabilities with the empirical mean human response across multiple annotators, improving interpretability and calibration metrics on multi-rater benchmarks.
Cross-Modal Contrastive Learning of ECG and Angiography Representations for Severe Stenosis Classification
This paper introduces StenCE, a pretraining framework that uses cross-modal contrastive learning between ECG and X-ray angiography representations to detect severe coronary stenosis from ECGs, achieving high performance and enabling early diagnosis even in asymptomatic patients.
A Multi-Domain Red Teaming Framework for Safety, Robustness, and Fairness Evaluation of Medical Large Language Models
This paper presents a multi-domain red teaming framework for evaluating safety, robustness, and fairness of medical LLMs across 690 clinically grounded scenarios. Results show that high aggregate accuracy can mask critical failures, and hybrid evaluation with clinician oversight is necessary for credible safety assessment.