Conformal Prediction and Monte Carlo Inference for Addressing Uncertainty in Cervical Cancer Screening

Christopher W Clark, Scott Kinder, Didem Egemen, Brian Befano, Kanan Desai, Syed Rakin Ahmed, Praveer Singh, Ana Cecilia Rodriguez, Jose Jeronimo, Silvia De Sanjose, Nicolas Wentzensen, Mark Schiffman, Jayashree Kalpathy-Cramer

Keywords: Conformal prediction, computer vision, cervical cancer

Abstract: "In the medical domain, where a misdiagnosis can have life- altering ramifications, understanding the certainty of model predictions is an important part of the model development process. However, deep learning approaches suffer from a lack of a native uncertainty metric found in other statistical learning methods. One common technique for uncertainty estimation is the use of Monte-Carlo (MC) dropout at training and inference. Another approach is Conformal Prediction for Uncertainty Quantification (CUQ). This paper will explore these two methods as applied to a cervical cancer screening algorithm currently under development for use in low-resource settings. We find that overall, CUQ and MC inference produce similar uncertainty patterns, that CUQ can aid in model development through class delineation, and that CUQ uncertainty is higher when the model is incorrect, providing further fine-grained information for clinical decisions."

Submission Number: 19