Deep Anomaly Detection with Outlier Exposure

Dan Hendrycks, Mantas Mazeika, Thomas Dietterich

Abstract: It is important to detect anomalous inputs when deploying machine learning systems. The use of larger and more complex inputs in deep learning magnifies the difficulty of distinguishing between anomalous and in-distribution examples. At the same time, diverse image and text data are available in enormous quantities. We propose leveraging these data to improve deep anomaly detection by training anomaly detectors against an auxiliary dataset of outliers, an approach we call Outlier Exposure (OE). This enables anomaly detectors to generalize and detect unseen anomalies. In extensive experiments on natural language processing and small- and large-scale vision tasks, we find that Outlier Exposure significantly improves detection performance. We also observe that cutting-edge generative models trained on CIFAR-10 may assign higher likelihoods to SVHN images than to CIFAR-10 images; we use OE to mitigate this issue. We also analyze the flexibility and robustness of Outlier Exposure, and identify characteristics of the auxiliary dataset that improve performance.

Keywords: confidence, uncertainty, anomaly, robustness

TL;DR: OE teaches anomaly detectors to learn heuristics for detecting unseen anomalies; experiments are in classification, density estimation, and calibration in NLP and vision settings; we do not tune on test distribution samples, unlike previous work

Code: [ hendrycks/outlier-exposure](https://github.com/hendrycks/outlier-exposure) + [ 8 community implementations](https://paperswithcode.com/paper/?openreview=HyxCxhRcY7)

Data: [CIFAR-10](https://paperswithcode.com/dataset/cifar-10), [CIFAR-100](https://paperswithcode.com/dataset/cifar-100), [Places](https://paperswithcode.com/dataset/places), [SST](https://paperswithcode.com/dataset/sst), [SVHN](https://paperswithcode.com/dataset/svhn), [Tiny ImageNet](https://paperswithcode.com/dataset/tiny-imagenet), [Tiny Images](https://paperswithcode.com/dataset/tiny-images)

Community Implementations: [ 2 code implementations](https://www.catalyzex.com/paper/arxiv:1812.04606/code)