Towards a Venn-Abers Calibration Method for Object Detectors

Bruce Cyusa Mukama, Soundouss MESSOUDI, Sylvain Rousseau, Sebastien Destercke

Keywords: Object Detection, Venn-Abers Predictors, Confidence Calibration, Deferring, Trustworthy AI

TL;DR: We apply inductive Venn-Abers predictors to calibrate object detectors and this enables a principled human-in-the-loop process for the rejection of false positive predictions.

Abstract: Object detection locates and classifies a variable number of objects in image. The dedicated architectures typically output a very large number of candidate bounding boxes with confidence scores that can be used to filter-out the false positive predictions. However, these confidence scores are not well calibrated by default: they can be inappropriate for safety-critical applications. To obtain well calibrated confidence scores, we transform these scores into Venn-Abers predictors and the resulting imprecise probabilities enable a principled human-in-the-loop decision process for rejecting the false positive outputs. We evaluate this method on two autonomous driving benchmarks: the Kitti and nuScenes datasets. The results show that the empirical calibration error is reduced, but the accuracy gap is not completely closed.

Submission Number: 2