Monte-Carlo Frequency Dropout for Predictive Uncertainty Estimation in Deep Learning

Tal Ze'evi, Rajesh Venkataraman, Lawrence H. Staib, John A. Onofrey

Abstract: Monte-Carlo (MC) dropout is a technique for estimating predictive uncertainty in artificial neural networks (ANNs) by leveraging an ensemble of randomly diluted instances of an optimized ANN to construct a posterior prediction distribution. However, classical MC-dropout, operating within the neuronal signal space, might overlook frequency-related uncertainties commonly observed in medical imaging data. This study explores the concept of MC-frequency dropout – a repetitive stochastic attenuation of signal frequencies as they flow through the ANN during inference, and assessing its efficacy in a selective prediction scenario, wherein decision-makers can reject uncertain model predictions. We conducted a comprehensive comparison against state-of-the-art MC-signal dropout approaches: drop-connect (dilution of ANN edges) and dropout (dilution of ANN nodes) on the public MedMNIST repository. Covering a diverse set of biomedical imaging data, our experiments demonstrated a statistical advantage of MC-frequency dropout in most benchmarks. Our source-code is accessible for the benefit of the community.