Lost in the Folds: When Cross-Validation Is Not a Deep Ensemble for Uncertainty Estimation

Hugging Face Daily Papers Papers
cross-validation deep-ensembles uncertainty-estimation medical-image-segmentation calibration failure-detection nnunet

Summary

This paper compares cross-validation ensembles to deep ensembles for uncertainty estimation in medical image segmentation. Deep ensembles outperform cross-validation ensembles in calibration and failure detection, while cross-validation ensembles better approximate inter-rater variability.

Ensemble disagreement is widely used as a proxy for epistemic uncertainty in medical image segmentation. In practice, many studies form ensembles via K-fold cross-validation (CV), yet refer to them as ``deep ensembles'' (DE). Because CV members are trained on different data subsets, their disagreement mixes seed-driven variability with data-exposure effects, which can change how uncertainty should be interpreted. We audit recent segmentation uncertainty studies and find that terminology--implementation mismatches are common. We then compare a standard 5-fold CV ensemble to a 5-member DE (fixed training set, different random seeds) under otherwise identical configurations on three multi-rater segmentation datasets spanning three modalities. We evaluate uncertainty for calibration, failure detection, ambiguity modeling, and robustness under distribution shift. DE match segmentation accuracy while improving calibration and failure detection, whereas CV ensembles sometimes correlate more strongly with inter-rater variability on the studied datasets. Thus, ensemble construction should be chosen to match the research question: DE for reliability-oriented use (e.g., selective referral/failure detection) and CV ensembles as a proxy for ambiguity. We provide a lightweight nnU-Net modification enabling DE training within the default pipeline.
Original Article
View Cached Full Text

Cached at: 05/21/26, 06:12 PM

Paper page - Lost in the Folds: When Cross-Validation Is Not a Deep Ensemble for Uncertainty Estimation

Source: https://huggingface.co/papers/2605.18329

Abstract

Deep ensembles trained with fixed data and varying seeds outperform cross-validation ensembles in calibration and failure detection for medical image segmentation, while cross-validation ensembles better approximate inter-rater variability.

Ensemble disagreement is widely used as a proxy forepistemic uncertaintyin medical image segmentation. In practice, many studies form ensembles via K-foldcross-validation(CV), yet refer to them as ``deep ensembles’’ (DE). Because CV members are trained on different data subsets, their disagreement mixes seed-driven variability with data-exposure effects, which can change how uncertainty should be interpreted. We audit recent segmentation uncertainty studies and find that terminology--implementation mismatches are common. We then compare a standard 5-fold CV ensemble to a 5-member DE (fixed training set, different random seeds) under otherwise identical configurations on three multi-rater segmentation datasets spanning three modalities. We evaluate uncertainty forcalibration,failure detection,ambiguity modeling, and robustness underdistribution shift. DE match segmentation accuracy while improvingcalibrationandfailure detection, whereas CV ensembles sometimes correlate more strongly with inter-rater variability on the studied datasets. Thus, ensemble construction should be chosen to match the research question: DE for reliability-oriented use (e.g., selective referral/failure detection) and CV ensembles as a proxy for ambiguity. We provide a lightweightnnU-Netmodification enabling DE training within the default pipeline.

View arXiv pageView PDFGitHub0Add to collection

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2605.18329 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2605.18329 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2605.18329 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Uncertainty-Aware Ensemble Deep Randomized Neural Networks for Classification

arXiv cs.LG

The paper proposes intuitionistic fuzzy deep RVFL (IF-dRVFL) and ensemble deep RVFL (IF-edRVFL) frameworks that use sample neighborhood information to improve robustness against noise and outliers in classification tasks, outperforming existing SOTA fuzzy and non-fuzzy approaches on benchmark datasets.