Diversifying Deep Ensembles: A Saliency Map Approach for Enhanced OOD Detection, Calibration, and Accuracy

Stanislav Dereka, Ivan Karpukhin, Maksim Zhdanov, Sergey Kolesnikov

Abstract: Deep ensembles are capable of achieving state-of-the-art results on classification and out-of-distribution (OOD) detection tasks. However, their effectiveness is limited due to the homogeneity of learned patterns within ensembles. To overcome this issue, our study introduces Saliency-Diversified Deep Ensembles (SDDE1), a novel approach that promotes diversity among ensemble members by leveraging saliency maps. Through incorporating saliency map diversification, our method outperforms conventional ensemble techniques and improves calibration on multiple classification and OOD detection tasks. In particular, the proposed method achieves state-of-the-art OOD detection quality, calibration, and accuracy on multiple benchmarks, including CIFAR10/100 and large-scale ImageNet datasets.