AVOID: Alleviating VAE's Overestimation in Unsupervised OOD Detection

Yewen Li, Chaojie Wang, Xiaobo Xia, Hongxin Wei, Tongliang Liu, Bo An, Lei Feng, Xinrun Wang

Primary Area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: Unsupervised Out-of-Distribution Detection, VAE

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Abstract:

Deep generative models (DGMs) aim at characterizing the distribution of the training set by maximizing the marginal likelihood of inputs
in an unsupervised manner, making them a promising option for unsupervised out-of-distribution (OOD) detection.
However, recent works have reported that DGMs often assign higher likelihoods to OOD data than in-distribution (ID) data, i.e., **overestimation**, leading to their failures in OOD detection.
Although several pioneer works have tried to analyze this phenomenon, and some VAE-based methods have also attempted to alleviate this issue by modifying their score functions for OOD detection, the root cause of the overestimation in VAE has never been revealed to our best knowledge.
To fill this gap, this paper will provide a thorough theoretical analysis on the overestimation issue of VAE, and reveal that this phenomenon arises from two aspects: 1) the improper design of prior distribution; 2) the gap of dataset entropy-mutual integration (sum of dataset entropy and mutual information terms) between ID and OOD datasets.
Based on these findings, we propose a novel score function to **A**lleviate **V**AE's **O**verestimation **I**n unsupervised OOD **D**etection, named ``**AVOID**'', which contains two novel techniques, specifically post-hoc prior and dataset entropy-mutual calibration.
Experimental results verify our theoretical analysis, demonstrating that the proposed method is effective in alleviating overestimation and improving unsupervised OOD detection performance.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 2226