Avoiding Catastrophic Referral Failures In Medical Images Under Domain Shift

Anuj Srivastava, Pradeep Shenoy, Devarajan Sridharan

Keywords: domain generalization, selective classification, automated medical image diagnosis, uncertainty estimation, risk coverage, covariate shift, diabetic retinopathy

TL;DR: We propose post hoc solutions for state-of-the-art deep learning models which fail in selective classification on medical images under domain shift.

Abstract: Developing robust approaches for domain generalization is critical for the real world deployment of deep learning models. Here, we address a particular domain generalization challenge: selective classification for automated medical image diagnosis. In this setting, models must learn to abstain from making predictions when label confidence is low, especially when tested with samples that deviate significantly from the training set (covariate shift). Using the example of diabetic retinopathy detection we show that even state-of-the-art deep learning models, including Bayesian networks, fail during selective classification under covariate shift. Bayesian estimates of predictive uncertainty do not generalize well under covariate shift yielding catastrophic performance drops during referral. We identify the source of these failures and propose several post hoc referral solutions that enable reliable selective classification under covariate shift.

Submission Number: 32