Enemy is Inside: Alleviating VAE's Overestimation in Unsupervised OOD Detection

Yewen Li, Chaojie Wang, Xiaobo Xia, Hongxin Wei, Tongliang Liu, Bo An, Xinrun Wang

Keywords: Unsupervised Out-of-Distribution Detection, VAE

TL;DR: This paper provides an analysis of the VAE's overestimation issue in unsupervised OOD detection and proposes a novel score function to alleviate the overestimation and improve unsupervisd OOD detection performance.

Abstract:

Deep generative models (DGMs) aim at characterizing the distribution of the training set by maximizing the marginal likelihood of inputs
in an unsupervised manner, making them a promising option for unsupervised out-of-distribution (OOD) detection.
However, recent works have reported that DGMs often assign higher likelihoods to OOD data than in-distribution (ID) data, $\textit{i.e.}$, $\textbf{\textit{overestimation}}$, leading to their failures in OOD detection.
Although several pioneer works have tried to analyze this phenomenon, and some VAE-based methods have also attempted to alleviate this issue by modifying their score functions for OOD detection, the root cause of the $\textit{overestimation}$ in VAE has never been revealed to our best knowledge.
To fill this gap, this paper will provide a thorough theoretical analysis on the $\textit{overestimation}$ issue of VAE, and reveal that this phenomenon arises from two Inside-Enemy aspects: 1) the improper design of prior distribution; 2) the gap of dataset entropies between ID and OOD datasets.
Based on these findings, we propose a novel score function to $\textbf{A}$lleviate $\textbf{V}$AE's $\textbf{O}$verestimation $\textbf{I}$n unsupervised OOD $\textbf{D}$etection, named $\textbf{``AVOID''}$, which contains two novel techniques, specifically post-hoc prior and dataset entropy calibration.
Experimental results verify our analysis, demonstrating that the proposed method is effective in alleviating $\textit{overestimation}$ and improving unsupervised OOD detection performance.

Submission Number: 3187