Confidence is Not Reliability: Rethinking MC Dropout in Brain Tumour Segmentation
arXiv:2606.19300v1 [cs.CV] 17 Jun 2026
Xin Ci Wong1,2[0000−0002−1036−8023] , Duygu Sarikaya2[0000−0002−2083−4999] , Kieran Zucker3[0000−0003−4385−3153] , Marc de Kamps2[0000−0001−7162−4425] , and Nishant Ravikumar2[0000−0003−0134−107X] 1
Centre for Doctoral Training in AI for Medical Diagnosis and Care, School of Computing, University of Leeds 2 School of Computer Science, University of Leeds 3 Leeds Cancer Centre, St James’s University Hospital, Leeds, UK [email protected]
Abstract. Glioma segmentation in multiparametric MRI is a critical component of treatment planning. A segmentation model that fails silently on treatment-critical sub-regions represents a patient safety risk that overlap-based metrics such as Dice scores cannot expose. We ask whether voxel-level uncertainty estimation via Monte Carlo (MC) Dropout can reliably identify segmentation errors in clinically critical sub-regions, and whether calibration failure modes are detectable from standard reporting metrics alone. In an empirical two-model case study on 126 BraTS21 patients, we evaluate a high-performance pretrained SegResNet and a locally trained UNet with residual units (UNet-Res). MC dropout preserved segmentation accuracy (|∆Dice| < 0.01) while achieving strong uncertainty-error alignment (AUROC for entropy (H) ≈0.97), indicating uncertainty correctly ranks erroneous voxels above correct ones. Entropy-based patient stratification identified a high-uncertainty subgroup with substantially lower segmentation performance (median whole-tumour Dice 0.835 vs. 0.925), supporting uncertainty as a practical triage signal. However, global alignment can mask important regionspecific differences. Despite similar AUROC, UNet-Res exhibited nearzero enhancing tumour entropy (0.054) and Expected Calibration Error (ECE) of 0.915, with a Dice of only 0.714, indicating severely miscalibrated confidence on the most clinically critical sub-region, a failure mode invisible to standard Dice and AUROC reporting. These findings demonstrate that strong uncertainty-error alignment is necessary but insufficient for clinical safety: sub-region-specific calibration assessment must accompany AUROC evaluation when selecting models for clinical deployment. Keywords: Brain tumour segmentation · Glioma MRI · Uncertainty quantification · Monte Carlo dropout · Epistemic uncertainty · Triage signal
2
XC Wong et al.
1
Introduction
Glioblastoma (GBM), which accounts for half of the adult glioma cases, is one of the deadliest yet common primary malignant tumours. The extent-of-resection (EOR) depends on accurate delineation of tumour sub-regions: whole tumour (WT), tumour core (TC), contrast-enhancing tumour (ET) from multiparametric MRI, and residual tumour at the margin can proliferate soon after surgery, placing high clinical stakes on the accuracy of each sub-region delineation [20]. Deep learning models now approach expert-level performance on benchmark datasets; however, a fundamental barrier to clinical deployment persists: deterministic predictions produce a single point-estimate per voxel with no associated confidence signal [9]. This is clinically consequential for two reasons. First, errors are spatially heterogeneous, and mis-labelling infiltrative tumour margins affects the EOR, while ET errors propagate into treatment response assessment. Second, without a confidence signal, models that fail on critical sub-regions do so silently, with no mechanism to trigger radiologist review before error propagate into clinical decisions. In glioma segmentation this limitation is acute: ET occupies a small fraction of total tumour voxels, and its miscalibration would be invisible to any metric computed without sub-region stratification. Uncertainty quantification (UQ) addresses this by providing voxel-wise confidence maps that flag regions requiring re-evaluation, without modifying acquisition protocols or requiring re-annotation. Among inference-time UQ methods, Monte Carlo (MC) Dropout [8] has emerged as a practical approach: by enabling dropout at inference and averaging over multiple stochastic forward passes, it approximates the posterior distribution over model weights, enabling estimation of both total predictive uncertainty and its epistemic component. Despite increasing interest in uncertainty estimation for medical image segmentation, its evaluation is often limited to global metrics or qualitative inspection [14]. These approaches may overlook clinically significant failure modes, particularly overconfident errors (low entropy, high error) in treatment-critical sub-regions such as ET [1]. Furthermore, the behaviour of uncertainty estimates across models of differing segmentation quality has not been systematically examined. This raises two important questions: can voxel-level uncertainty estimation via MC Dropout reliably identify segmentation errors in clinically critical glioma sub-regions, and how does uncertainty behaviour vary across segmentation models of different quality and calibration? We address these questions through a two-model case study empirically evaluating MC Dropout on a high-performing pretrained SegResNet [15] and a locally trained UNet-Res [5], assessed on 126 patients from the BraTS21 dataset [2]. We evaluate uncertainty quality at both voxel and patient level, and characterise model-dependent calibration behaviour across tumour sub-regions. This work makes the following contributions: 1. AUROC 6= safety. We demonstrate that strong uncertainty–error alignment (AUROC ≈ 0.97) can coexist with clinically unsafe overconfidence, showing that AUROC alone is insufficient to certify model safety for subregion-specific clinical deployment.
Confidence is Not Reliability
3
2. Sub-region calibration failure invisible to standard metrics. UNetRes ET exhibited a calibration alert: ECE = 0.915 with a flat reliability curve at observed fraction positive ≈ 0.40 across all predicted probability bins, indicating that predicted probabilities carry no discriminative information about true ET voxel status, revealing a failure mode invisible to Dice and AUROC reporting alone. 3. Entropy-based patient triage. Entropy-based patient stratification identifies a high-uncertainty subgroup and translates voxel-level uncertainty into a deployment-ready patient-level triage signal without requiring groundtruth labels at inference time.
2
Related Work
2.1
Uncertainty quantification in medical image segmentation.
Nair et al.[16] demonstrated that MC dropout can identify lesion-level segmentation failures in multiple sclerosis, providing a template for uncertainty-guided review workflows. Jungo et al.[10] provided systematic analysis of uncertainty estimation quality for brain tumour segmentation, demonstrating that while voxelwise uncertainty could be heavily miscalibrated for direct error localisation, it becomes a reliable predictor of overall segmentation failure when aggregated at the subject level, though this quality remains highly dependent on the size of the training dataset. Czolbe et al.[6] showed that uncertainty tracks inter-observer disagreement, suggesting that uncertainty-driven workflow may not outperform random selection on genuinely ambiguous cases. Our work examines the complementary failure mode: whether uncertainty signals remain informative when model errors occur in regions of different level of ambiguity, as seen in UNet’s overconfident ET predictions. 2.2
Calibration and clinical reliability.
Mehrtash et al. [13] use expected calibration error (ECE) and reliability diagrams as standard tools for assessing whether predicted probabilities reflect empirical positive rates in medical image segmentation. This is a calibrating requirement to AUROC that captures absolute probability validity rather than ranking quality alone. Zeevi et al. [21] subsequently showed that global calibration metrics are insufficient in dense prediction settings with severe class imbalance, where spatially heterogeneous errors can obscure sub-region-specific failure modes entirely. Alternative approaches to uncertainty estimation, including deep ensembles [7] and test-time augmentation [17,19], have also been explored, often demonstrating improved uncertainty quality at the cost of increased computational overhead. Prior work on sub-region-specific calibration analysis, particularly comparing ECE and reliability behaviour on ET across architectures of differing quality has not been reported [3].
4
2.3
XC Wong et al.
MC Dropout and practical consideration.
Gal and Ghahramani [8] established the theoretical grounding for MC Dropout as an approximation to Bayesian inference, showing that dropout at inference samples from an approximate posterior over model weights. Kendall and Gal [11] extended this framework to decompose predictive uncertainty into aleatoric and epistemic components by simultaneously predicting data-dependent observation noise alongside MC Dropout sampling. An important practical extension was provided by Ledda et al. [12], who demonstrated that dropout injected posthoc at inference, constitutes a competitive alternative to embedded dropout for epistemic uncertainty quantification, with appropriate rescaling of the uncertainty measure. This justifies post-hoc injection into pretrained segmentation models. Nair et al. [16] trained a 3D CNN for uncertainty measures based on MC dropout, demonstrated that small lesions and lesion borders exhibit highest uncertainty. Cross-architecture comparison of calibration behaviour, especially on sub-regions has not been systematically addressed.
3
Material and Methods
Datasets The BraTS21 dataset provides 1251 pre-operative multiparametric MRI cases (T1, T1ce, T2, FLAIR) acquired across multiple institutions4 . All labelled cases were partitioned into train/validation/test splits of 1000/125/126 with fixed seed 42. Splits were disjoint at the patient level. All volumes were skull-stripped, atlas-registered, and resampled to 1 mm isotropic resolution at source. Preprocessing comprised per-modality z-score normalisation over nonzero (brain) voxels and centre-cropping to 1283 voxels. Segmentation Models The primary segmentation model was a pretrained 3D SegResNet [15] sourced from the MONAI [5] model zoo bundle brats_mri_segmentation v0.5.45. A custom 3D UNet with residual units (UNet-Res) [5] trained from scratch, provided two things: a theoretically cleaner MC Dropout baseline (dropout present during training, satisfying the Gal & Ghahramani [8] assumptions), and a lower-performing regime to stress-test uncertainty calibration across model quality levels. Both models were evaluated on the same 126−case test split, ensuring no data leakage for either model. Details implementation refer to Appendix A. 4
The RSNA-ASNR-MICCAI Brain Tumor Segmentation (BraTS) Challenge 2021 raw data required to reproduce the findings of this study are available to download from https://www.med.upenn.edu/cbica/brats2021/#Data2 5 The reported Dice score from MONAI official site on validation set for BraTS18: Tumor core (TC): 0.8559; Whole tumor (WT): 0.9026; Enhancing tumor (ET): 0.7905; Average: 0.8518 https://github.com/Project-MONAI/model-zoo/tree/dev/models/brats_mri_segmentation
Confidence is Not Reliability
5
MC dropout We activated nn.Dropout3d (p=0.2) in each model while normalisation layers remained in evaluation mode. For the pre-trained SegResNet, MC dropout was injected post-hoc at inference [8]. Rather than sampling from a variational posterior, each stochastic forward pass applies a random channel-level mask to intermediate activations, inducing variance across passes. The resulting uncertainty is interpreted as the model’s sensitivity to feature channel ablation i.e. regions where predictions change substantially under random masking are flagged as uncertain. For the UNet-Res, dropout was trained-in, satisfying the theoretical framework [8,16]. Previous MC-dropout segmentation studies commonly employed 10 − 50 stochastic forward passes for uncertainty estimation [4,16]. In this study, we used 20 stochastic forward passes per patient (each sampling a different random subset of feature channels), with seeds derived as seedi = master_seed + patient_idx × 1000 + i for reproducibility, allow approximated inference over a distribution of model weights [11]. This ensemble asks: "do different sub-networks agree on this image?" Regions of high disagreement flag epistemic uncertainty: gaps in what the model learned from training data. No test-time augmentation was applied, isolating the MC Dropout uncertainty signal. Uncertainty Metrics All uncertainty metrics were computed from the probability stack using the Bernoulli formulation per voxel per channel: Table 1. Uncertainty type and its meaning Metric
Uncertainty type
Meaning
Predictive entropy, Hpred Expected entropy, Hexp Mutual information, MI
Total Aleatoric Epistemic
Boundary/region needs review Image ambiguity Model knowledge gap
N
p̄ =
1 X (i) p N i=1
(1)
where p(i) ∈ [0, 1] is the sigmoid output probability for a given voxel and channel at stochastic forward pass i, and N = 20 is the number of passes. Hpred = −p̄ log p̄ − (1 − p̄) log(1 − p̄)
(2)
N i 1 X h (i) −p log p(i) − (1 − p(i) ) log(1 − p(i) ) Hexp = N i=1
(3)
MI = Hpred − Hexp ≥ 0
(4)
6
XC Wong et al.
In a standard binary classification setting, such as cancer screening, the area under the receive operating characteristic curve (AUROC) evaluates whether a model’s output score can discriminate between positive (diseased) and negative (healthy) cases. In our setting, we adopt analogous formulation, where the "positive" class corresponds to erroneous voxels and the score is given by voxel-wise uncertainty, measured using entropy (H) or mutual information (MI) [16]. For a given uncertainty threshold, the true positive rate (TPR) represents the proportion of erroneous voxels assigned uncertainty above the threshold, while the false positive rate (FPR) represents the proportion of correctly segmented voxels exceeding the same threshold. AUROC summarises this relationship across all possible threshold and can be interpreted as the probability that a randomly selected erroneous voxel has higher uncertainty than a randomly selected correct voxel, i.e. does uncertainty detect errors? In dense segmentation tasks, erroneous voxels typically constitute a small fraction of the total volume. While AUROC remains a useful measure of uncertainty-error ranking, it does not capture the absolute reliability of uncertainty estimates. In particular, a model can achieve high AUROC by correctly ranking most errors above correct voxels, while still assigning low uncertainty to subset of clinically critical error. This limitation motivates the inclusion of calibration metrics, such as Expected Calibration Error (ECE), to assess whether predicted confidence is aligned with empirical correctness. ECE measures whether the predicted probabilities are trustworthy. It is the average of gaps on the reliability diagram, and was computed on foregroundrelevant voxels defined as the union of predicted positive voxels (p̄ > 0.1) and GT-positive voxels, using K = 20 equal-width bins, i.e. how well a model’s predicted probabilities match the true outcome frequencies [13]. Background exclusion was required as 98.5% of voxels have p̄ < 0.05. Population ECE was computed as a voxel-count-weighted average of per-patient bin statistics. Sensitivity to the foreground threshold was evaluated across p̄ ∈ {0.05, 0.10, 0.20, 0.30} (Appendix B).
ECE =
B X |Bb | b=1
N
· |p̄b − āb |
(5)
P where Bb is the b-th probability bin, p̄b = |B1b | i∈Bb p̂i is the mean predicted P probability in bin b, and āb = |B1b | i∈Bb 1[yi = 1] is the observed accuracy in bin b for binary GT labels, i.e. the fraction of voxels in this bin that are truly positive. The p̄ is the mean sigmoid output across 20 MC Dropout passes for a given voxel and a given channel, i.e. the average predicted probability that this voxel belongs to certain sub-regions.
Confidence is Not Reliability
4
Results
4.1
Segmentation performance
7
SegResNet MC Dropout achieved Dice of 0.853 ± 0.103 (WT), 0.918 ± 0.097 (TC), and 0.918 ± 0.064 (ET). UNet-Res MC Dropout achieved higher WT Dice (0.885±0.086) but substantially lower TC (0.756±0.108) and ET (0.714±0.117), consistent with a trade-off between global spatial context and sub-region discrimination (Table 2). The UNet-Res Dice distributions on TC and ET are bimodal, with a secondary peak below Dice 0.6 (Fig. 1), indicating a clinically relevant failure mode in a minority of patients. Paired Wilcoxon signed-rank tests (Bonferroni-corrected) confirmed that stochastic inference introduced statistically detectable but clinically negligible accuracy differences relative to deterministic baselines (|∆| < 0.01 across all sub-regions for both models), confirming MC Dropout can be deployed without segmentation quality penalty (Table 2). Table 2. Segmentation performance (Dice ↑, mean ± SD) on BraTS2021 (126 patients). Paired Wilcoxon signed-rank tests compare MC Dropout‡ with deterministic† inference (Bonferroni-corrected). While several differences are statistically significant, effect sizes are small (|∆| < 0.01), indicating negligible practical impact on segmentation performance. Best results per model are highlighted in bold. Model
WT
TC
ET
WT TC ET Wilcoxon (MC vs. Det)
SegResNet† SegResNet‡
0.860 ± 0.100 0.853 ± 0.103
0.922 ± 0.092 0.918 ± 0.097
0.920 ± 0.062 0.918 ± 0.064
— ***
— ***
— ***
UNet-Res† UNet-Res‡
0.880 ± 0.087 0.885 ± 0.086
0.762 ± 0.104 0.756 ± 0.108
0.710 ± 0.115 0.714 ± 0.117
— **
— ***
— ns
†
= Deterministic;‡ = MC Dropout; WT = whole tumour; TC = tumour core; ET = enhancing tumour; MC = MC dropout; Det = Deterministic model. ns = not significant; * p < 0.05, ** p < 0.01, *** p < 0.001. ∆ denotes MC Dropout minus deterministic Dice.
4.2
Uncertainty quality
Both models achieved strong entropy-error alignment at voxel level: AUROCH was 0.977 ± 0.028 (SegResNet) and 0.975 ± 0.037 (UNet-Res); AUROC-MI was 0.981 ± 0.020 and 0.969 ± 0.038 respectively (Table 3). The comparable AUROC values between architectures, however, may mask a critical divergence in calibration quality and spatial behaviour that has direct clinical implications. Calibration analysis (Fig. 2) confirmed this divergence. SegResNet showed moderate, consistent under-confidence across all sub-regions (ECE: WT=0.232, TC=0.138, ET=0.170), with reliability curves above the diagonal throughout. UNet-Res WT was the best-calibrated result across all measurements (ECE=
8
XC Wong et al.
Fig. 1. Distribution of Dice scores (n=126 patients per model). Kernel density estimates show per-method score distributions for whole tumour(WT), tumour core (TC) and enhancing tumour (ET). Dashed vertical lines indicate per-method medians. SegResNet methods clustered tightly near the deterministic baseline on all sub-regions; UNet-Res shows a broad, bimodal distribution on TC and ET with a tail below Dice 0.6, indicating a subset of patients with poor sub-region discrimination. MC Dropout and deterministic inference are visually indistinguishable within each architecture, confirming that stochastic inference does not shift the performance distribution. Table 3. Uncertainty quality evaluation (mean ± SD) for MC dropout method. Entropy (Ent) is voxel-wise predictive entropy. AUROC evaluates uncertainty–error alignment. Lower entropy indicates more confident predictions; higher AUROC indicates better alignment with segmentation errors. Model
Ent(WT) ↓
Ent(ET) ↓
AUROC-H ↑
AUROC-MI ↑
SegResNet
0.296 ± 0.071
0.277 ± 0.107
Ent(TC)↓
0.336 ± 0.111
0.977 ± 0.028
0.981 ± 0.020
UNet-Res
0.113 ± 0.074
0.269 ± 0.128
0.054 ± 0.060
0.975 ± 0.037
0.969 ± 0.038
Ent = voxel-level Shannon entropy; AUROC-H = entropy vs. error AUROC; AUROC-MI = mutual information vs. error AUROC.
0.091). UNet-Res ET, however, showed critical miscalibration (ECE> 0.9): the reliability diagram reveals an approximately flat curve at observed fraction positive ≈ 0.40 regardless of predicted probability across the full [0.1, 0.9] range, indicating that UNet-Res’s ET probability outputs carry no discriminative information about true ET voxel status. Sensitivity analysis across foreground thresholds p̄ ∈ {0.05, 0.10, 0.20, 0.30} confirmed the finding was threshold-independent (UNet-Res ET ECE range 0.888–0.948 versus 0.141–0.230 for SegResNet ET). 4.3
Overconfidence on Enhancing Tumour
The ET entropy–error scatterplot (Fig. 3, left) reveals the central finding of this study. UNet-Res assigns near-zero ET entropy regardless of segmentation error magnitude: patients with greater than 40% ET Dice error are indistinguishable in uncertainty from well-segmented cases (UNet-Res ET entropy = 0.054 ± 0.060). In contrast, SegResNet shows a positive entropy–error correlation across a wide entropy range, providing an actionable clinical triage signal despite achieving
Confidence is Not Reliability
9
Fig. 2. Reliability diagrams computed on foreground-relevant voxels (predicted positive p > 0.1 ∪ GT positive, K=20 equal-width bins). Each point represents one bin of foreground voxels. ECE values (∗) are computed over all foreground bins. The bar histogram on the secondary y-axis shows how many foreground voxels fall in each probability bin, revealing where the model concentrates its predictions. Curves above the diagonal indicate underconfidence across all sub-regions; UNet-Res ET (ECE = 0.915) exhibits near-constant acc ≈ 0.4 regardless of predicted probability, confirming the overconfident-but-wrong behaviour identified in entropy analysis.
substantially higher ET Dice (0.918 vs. 0.714). This dissociation between accuracy and uncertainty behaviour is the defining finding: high AUROC with low entropy magnitude produces a signal insufficient for clinical thresholding. Qualitative uncertainty maps (Figs. 4) further illustrate the spatial divergence. 4.4
Patient-Level Triage
The entropy quartile analysis (Figure 3, centre) demonstrates clinical utility directly: SegResNet patients in the highest uncertainty quartile had median WT Dice of 0.835 versus 0.925 in the lowest quartile, indicating that uncertainty stratification identifies a subset at substantially elevated error risk. The calibration comparison figures (Figs. 5) contrast representative Q1/Q4 patients per model. For SegResNet, the Q4 case presents a morphologically com-
10
XC Wong et al.
Fig. 3. Uncertainty analysis for MC Dropout (n=126 patients per model). Left: per-patient mean predictive entropy versus ET Dice error; SegResNet (blue) shows positive entropy-error correlation while UNet-Res (orange) clusters at near-zero entropy regardless of error magnitude, indicating systematic overconfidence on the most treatment-critical sub-region. Centre: WT Dice stratified by SegResNet MC Dropout entropy quartile; median Dice decreases monotonically from Q1 (0.925) to Q4 (0.835), demonstrating uncertainty as a clinically usable triage signal. Right: AUROC for entropy and mutual information versus voxel-level error for both models.
plex multifocal tumour (BraTS2021_01598, Dice WT=0.655, Ent=0.393): elevated entropy is spatially co-localised with errors at lesion margins and the inter-lesion parenchyma, correctly flagging an atypical case. For UNet-Res, the Q4 case (BraTS2021_00640, Dice WT=0.552, Ent=0.179) shows that despite catastrophic WT failure, mean entropy increases only 3-fold relative to Q1, and the entropy remains spatially misaligned with the principal error region. The model does not adequately signal the degree of failure.
5
Discussion and Conclusion
Clinical implications of overconfidence The principal finding is a dissociation between segmentation accuracy and uncertainty behaviour with direct patient safety implications. UNet-Res achieves competitive WT Dice yet is systematically onverconfident on ET, the sub-region used to assess chemotherapy response and guide stereotactic biopsy targeting for RANO criteria6 [18]. A model that assigns equivalent confidence to a 40% ET error and a good segmentation (Deterministic WT Dice: 0.880 ± 0.087) cannot support safe semi-automated workflows, regardless of its mean Dice performance. This finding demonstrates why model selection for clinical deployment must consider uncertainty behavior on sub-region level, not only overlap-based Dice scores [6]. The entropy quartile analysis translates uncertainty quantification into a clinically actionable finding: patients flagged as high-uncertainty by MC Dropout have demonstrably worse 6
The RANO (Response Assessment in Neuro-Oncology) criteria are standardised guidelines used to evaluate treatment response in glioma trials, focusing on tumor measurement via MRI.
Confidence is Not Reliability
11
Fig. 4. Qualitative comparison of segmentation and uncertainty estimation for a representative BraTS2021 case (BraTS2021_00640, axial z=82). From Left to Right: T1ce + Ground Truth (GT) contours, Entropy overlay, Boundary entropy, Segmentation error map. GT shown as contour lines not filled regions, preserves anatomy visibility underneath. Warmer colours indicate higher predictive entropy at the predicted tumour margin, identifying regions where the model’s confidence in the delineation boundary is lowest. UNet-Res shows high entropy not only at the boundary, but also concentrated at the tumour core, while SegResNet’s entropy is a clean ring at the tumour-brain interface.
12
XC Wong et al.
Fig. 5. Uncertainty heatmap of Low-Uncertainty vs. High Uncertainty Patient. For each SegResNet and UNet-Res, Q1 patient (top row): Low entropy, minimal segmentation error, clean boundary. Entropy is relatively faint, boundary entropy ring is thinner and dimmer. Q4 patient (bottom row): Entropy (0.393 vs. 0.179) both are badly failing cases. The segmentation error shows massive dark red area. SegResNet also expresses more uncertainty even on its well-segmented case.
Confidence is Not Reliability
13
segmentation, enabling targeted radiologist review without requiring full manual re-segmentation of every case[16]. This triage function i.e. directing attention to the cases most likely to contain errors, addresses a practical barrier to clinical deployment of automated segmentation[10]. Why AUROC alone is insufficient Both models achieve AUROC-H ≈ 0.975, which may appear to indicate equivalent clinical utility with good uncertaintyerror alignment. However, the voxel-level class imbalance may inflate absolute values, even a weak uncertainty signal can achieve high AUROC by correctly ranking a few erroneous voxels above the large pool of correct ones [6]. ECE analysis reveals why AUROC could be misleading. High ECE for UNet-Res ET with a flat reliability curve demonstrates that the model’s ET probability outputs are informationally invalid, i.e. the predicted probabilities do not reflect true positive rates. A model can rank well but be poorly calibrated. Therefore, ECE and entropy magnitude must be reported alongside AUROC for a complete calibration picture; AUROC alone is insufficient to certify clinical safety [13]. Limitations Evaluation is limited to a single publicly available dataset, which uses consensus labels. The 20-pass MC Dropout approximation was not evaluated for sensitivity to pass count, and post-hoc dropout injection into SegResNet weakens direct comparison with the embedded-dropout UNet-Res. Future work should incorporate multi-site data, calibration metrics with temperature scaling or Platt scaling to reduce ECE, and prospective reader studies evaluating radiologist efficiency with uncertainty overlays. Conclusion We evaluated MC Dropout uncertainty estimation across SegResNet and UNet-Res on 126 BraTS21 test set, demonstrating that high uncertaintyerror alignment (AUROC-H ≈ 0.97) can coexist with clinically unusable uncertainty signals. The central finding is a dissociation between ranking quality and calibration validity: despite comparable AUROC, the lower-performing architecture exhibited near-zero ET entropy (0.054), ECE of 0.915, and a flat reliability curve, indicating that its ET probability outputs carry no discriminative information, rendering the uncertainty signal uninformative on the sub-region most directly linked to treatment decisions. This failure mode was invisible to Dice and AUROC reporting alone. Furthermore, the UNet-Res was simultaneously better calibrated on WT yet critically miscalibrated on ET, demonstrating that sub-region performance and calibration can vary within the same model. Entropy-based patient stratification provided a practical triage signal, translating uncertainty into a patient-level review trigger. Acknowledgments. We acknowledge the support of the Centre for Doctoral Training in AI for Medical Diagnosis and Care for funding this project. We extend our gratitude to the patients and doctors who contributed to datasets. The study was facilitated using the Aire High Performance Computing resources at the University of Leeds, UK.
14
XC Wong et al.
Disclosure of Interests. KZ is a founder, director, and shareholder of Synaptome Limited. NR is co-founder and shareholder of adsilico Ltd. The remaining authors declare no competing interests relevant to the content of this article.
References 1. Aliferis, C., Simon, G.: Overfitting, underfitting and general model overconfidence and under-performance pitfalls and best practices in machine learning and AI. In: Simon, G.J., Aliferis, C. (eds.) Artificial Intelligence and Machine Learning in Health Care and Medical Sciences: Best Practices and Pitfalls, pp. 477–524. Springer International Publishing, Cham (2024). https://doi.org/10.1007/978-3-031-39355-6_10 2. Baid, U., Ghodasara, S., Mohan, S., et al., M.: The rsna-asnr-miccai brats 2021 benchmark on brain tumor segmentation and radiogenomic classification (2021) 3. Bonato, B., Nanni, L., Bertoldo, A.: Advancing precision: A comprehensive review of MRI segmentation datasets from BraTS challenges (2012-2025). Sensors (Basel) 25(6), 1838 (Mar 2025) 4. Camarasa, R., Bos, D., Hendrikse, J., Nederkoorn, P., Kooi, E., van der Lugt, A., de Bruijne, M.: Quantitative Comparison of Monte-Carlo Dropout Uncertainty Measures for Multi-class Segmentation. In: Sudre, C.H., Fehri, H., Arbel, T., Baumgartner, C.F., Dalca, A., Tanno, R., Van Leemput, K., Wells, W.M., Sotiras, A., Papiez, B., Ferrante, E., Parisot, S. (eds.) Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, and Graphs in Biomedical Image Analysis. pp. 32–41. Springer International Publishing, Cham (2020). https://doi.org/10.1007/978-3-030-60365-6_4 5. Cardoso, M.J., Li, W., Brown, R., Ma, N., , et al.: Monai: An open-source framework for deep learning in healthcare (2022), https://arxiv.org/abs/2211.02701 6. Czolbe, S., Arnavaz, K., Krause, O., Feragen, A.: Is segmentation uncertainty useful? In: Feragen, A., Sommer, S., Schnabel, J., Nielsen, M. (eds.) Information Processing in Medical Imaging. pp. 715–726. Springer International Publishing, Cham (2021) 7. Dwaracherla, V., Wen, Z., Osband, I., Lu, X., Asghari, S.M., Roy, B.V.: Ensembles for uncertainty estimation: Benefits of prior functions and bootstrapping (2022), https://arxiv.org/abs/2206.03633 8. Gal, Y., Ghahramani, Z.: Dropout as a bayesian approximation: representing model uncertainty in deep learning. In: Proceedings of the 33rd International Conference on International Conference on Machine Learning - Volume 48. p. 1050–1059. ICML’16, JMLR.org (2016) 9. Gao, Y., Jiang, Y., Peng, Y., Yuan, F., Zhang, X., Wang, J.: Medical image segmentation: A comprehensive review of deep learning-based methods. Tomography 11(5), 52 (Apr 2025) 10. Jungo, A., Balsiger, F., Reyes, M.: Analyzing the quality and challenges of uncertainty estimations for brain tumor segmentation. Front. Neurosci. 14, 282 (Apr 2020) 11. Kendall, A., Gal, Y.: What uncertainties do we need in bayesian deep learning for computer vision? In: Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R. (eds.) Advances in Neural Information Processing Systems. vol. 30. Curran Associates, Inc. (2017)
Confidence is Not Reliability
15
12. Ledda, E., Fumera, G., Roli, F.: Dropout injection at test time for post hoc uncertainty quantification in neural networks. Information Sciences 645, 119356 (2023). https://doi.org/10.1016/j.ins.2023.119356 13. Mehrtash, A., Wells, W.M., Tempany, C.M., Abolmaesumi, P., Kapur, T.: Confidence Calibration and Predictive Uncertainty Estimation for Deep Medical Image Segmentation. IEEE transactions on medical imaging 39(12), 3868–3878 (Dec 2020). https://doi.org/10.1109/TMI.2020.3006437, https://pmc.ncbi.nlm.nih.gov/articles/PMC7704933/ 14. Mehta, R., Filos, A., Baid, U., Sako, C., McKinley, R., Rebsamen, M., , et al.: QU-BraTS: MICCAI BraTS 2020 Challenge on Quantifying Uncertainty in Brain Tumor Segmentation - Analysis of Ranking Scores and Benchmarking Results. The journal of machine learning for biomedical imaging 2022, https://www.melba– journal.org/papers/2022:026.html (Aug 2022) 15. Myronenko, A.: 3D MRI brain tumor segmentation using autoencoder regularization. In: Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. pp. 311–320. Springer International Publishing, Cham (2019) 16. Nair, T., Precup, D., Arnold, D.L., Arbel, T.: Exploring uncertainty measures in deep networks for multiple sclerosis lesion detection and segmentation. Med. Image Anal. 59(101557), 101557 (Jan 2020) 17. Patel, K., Beluch, W., Zhang, D., Pfeiffer, M., Yang, B.: On-manifold adversarial data augmentation improves uncertainty calibration. In: 2020 25th International Conference on Pattern Recognition (ICPR). pp. 8029–8036 (2021). https://doi.org/10.1109/ICPR48806.2021.9413010 18. Sakata, A., Fushimi, Y., Oshima, S., Uto, M., Mineharu, Y., Nakajima, S., Okuchi, S., Yamamoto, T., Otani, S., Ikeda, S., Takada, S., Mizowaki, T., Arakawa, Y., Nakamoto, Y.: RANO 2.0: critical updates and practical considerations for radiological assessment in neuro-oncology. Jpn. J. Radiol. 43(10), 1557–1574 (Oct 2025) 19. Sherkatghanad, Z., Abdar, M., Bakhtyari, M., Pławiak, P., Makarenkov, V.: Baytta: Uncertainty-aware medical image classification with optimized test-time augmentation using bayesian model averaging. Know.-Based Syst. 327(C) (Oct 2025). https://doi.org/10.1016/j.knosys.2025.114123, https://doi.org/10.1016/j.knosys.2025.114123 20. Singh, S., Dey, D., Barik, D., Mohapatra, I., Kim, S., Sharma, M., Prasad, S., Wang, P., Singh, A., Singh, G.: Glioblastoma at the crossroads: current understanding and future therapeutic horizons. Signal Transduct. Target. Ther. 10(1), 213 (Jul 2025) 21. Zeevi, T., Lieffrig, E.V., Staib, L.H., Onofrey, J.A.: Spatially-aware evaluation of segmentation uncertainty (2025), https://arxiv.org/abs/2506.16589