Suppressing Outlier Reconstruction in Autoencoders for Out-of-Distribution Detection

Sangwoong Yoon, Yung-Kyun Noh, Frank C. Park

Keywords: autoencoder, outlier detection, novelty detection, energy-based model

Abstract:

While only trained to reconstruct training data, autoencoders may produce high-quality reconstructions of inputs that are well outside the training data distribution. This phenomenon, which we refer to as outlier reconstruction, has a detrimental effect on the use of autoencoders for outlier detection, as an autoencoder will misclassify a clear outlier as being in-distribution. In this paper, we introduce the Energy-Based Autoencoder (EBAE), an autoencoder that is considerably less susceptible to outlier reconstruction.
The core idea of EBAE is to treat the reconstruction error as an energy function of a normalized density and to strictly enforce the normalization constraint. We show that the reconstruction of non-training inputs can be suppressed, and the reconstruction error made highly discriminative to outliers, by enforcing this constraint. We empirically show that EBAE significantly outperforms both existing autoencoders and other generative models for several out-of-distribution detection tasks.

One-sentence Summary: We investigate the phenomenon of an autoencoder reconstructing outliers and propose Energy-Based Autoencoder, where the reconstruction of outliers are explicitly suppressed through an energy-based formulation.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Reviewed Version (pdf): https://openreview.net/references/pdf?id=AexlxEapPq