Where to Drop: Tuning Monte Carlo Dropout for Uncertainty Calibration in Image Classification

Lina Benyamina, Emilien Jemelen

Keywords: Uncertainty Calibration, Reliable Machine Learning, Medical Imaging

TL;DR: Penultimate-layer MC Dropout empirically gives the most reliably calibrated uncertainty, validated on CIFAR-10 and mammography triage.

Abstract: Calibrated uncertainty is essential for deploying deep neural networks in high-stakes settings such as medical diagnosis. Monte Carlo Dropout (MC Dropout) provides a practical Bayesian approximation within a single architecture, yet key choices—dropout probability and mask placement—are typically heuristic and can produce uncertainty scores that poorly track error rates. We therefore perform a systematic grid search over MC Dropout hyperparameters and assess calibration via the monotonic relationship between accuracy and uncertainty, using accuracy-uncertainty curves with monotonicity-aware evaluation. On CIFAR-10, performance varies widely across configurations, but applying dropout in the penultimate layer consistently yields the most monotonic, actionable degradation as uncertainty increases. We validate this "penultimate-layer rule" on mammography triage for breast cancer screening, where calibrated uncertainty is crucial for safe deferral and workload allocation. Code and reproducibility artifacts are released at https://github.com/linabny/MonteCarlo_Dropout.

Submission Number: 21