Semi-supervised novelty detection using ensembles with regularized disagreement

Alexandru Tifrea, Eric Petru Stavarache, Fanny Yang

Keywords: semi-supervised novelty detection, out-of-distribution detection, ensembles, ensemble diversity, early stopping, regularization

TL;DR: Leverage unlabeled data to significantly improve semi-supervised novelty detection by training ensembles with regularized disagreement.

Abstract: Deep neural networks often predict samples with high confidence even when they come from unseen classes and should instead be flagged for expert evaluation. Current novelty detection algorithms cannot reliably identify such near OOD points unless they have access to labeled data that is similar to these novel samples. In this paper, we develop a new ensemble-based procedure for semi-supervised novelty detection (SSND) that successfully leverages a mixture of unlabeled ID and novel-class samples to achieve good detection performance. In particular, we show how to achieve disagreement only on OOD data using early stopping regularization. While we prove this fact for a simple data distribution, our extensive experiments suggest that it holds true for more complex scenarios: our approach significantly outperforms state-of-the-art SSND methods on standard image data sets (SVHN/CIFAR-10/CIFAR-100) and medical image data sets with only a negligible increase in computation cost.