Selective Prediction for Semantic Segmentation under Distribution Shift

Bruno Laboissiere Camargos Borges, Bruno Machado Pacheco, Danilo Silva

Keywords: selective prediction, distribution shift, semantic segmentation, uncertainty estimation, medical imaging

TL;DR: Selective prediction with post-hoc confidence estimation is effective in reducing the impacts of distribution shift for semantic segmentation in low-resource settings, particularly in medical imaging tasks

Abstract: Semantic segmentation plays a crucial role in various computer vision applications, yet its efficacy is often hindered by the lack of high-quality labeled data. To address this challenge, a common strategy is to leverage models trained on data from different populations, such as publicly available datasets. This approach, however, leads to the distribution shift problem, presenting a reduced performance on the population of interest. In scenarios where model errors can have significant consequences, selective prediction methods offer a means to mitigate risks and reduce reliance on expert supervision. This paper investigates selective prediction for semantic segmentation in low-resource settings, thus focusing on post-hoc confidence estimators applied to pre-trained models operating under distribution shift. We propose a novel image-level confidence measure tailored for semantic segmentation and demonstrate its effectiveness through experiments on three medical imaging tasks. Our findings show that post-hoc confidence estimators offer a cost-effective approach to reducing the impacts of distribution shift.

Submission Number: 22