Attention-Guided Energy-Based Model for Out-of-Distribution Data Detection

Zongjing Cao, Yan Li, Byeong-Seok Shin

Abstract: Detecting out-of-distribution (OOD) data is crucial for the safe and reliable deployment of deep learning models in open-world scenarios. While energy-based models (EBMs) have shown promising potential in OOD detection through the use of an energy function to capture the underlying probability distribution of data, previous approaches have primarily utilized logits or class probabilities from the fully connected layer to compute energy scores. However, logits are inherently class-specific and focus mainly on the relationship between the input and known classes, potentially ignoring the rich information embedded in raw feature representations that are essential for identifying OOD samples. This study introduces a novel approach that utilizes patterns within the feature space to calculate energy scores instead of relying on logits or class probabilities. We propose a spatial attention score to generate class-specific features for each category, which are then used to compute the energy score. Furthermore, we develop a new energy function that transforms these features into energy scores, significantly improving the OOD detection performance of EBMs. In experiments conducted on a Cifar-10 pre-trained ResNet-50, our feature-based energy score method reduced the average false positive rate at a true positive rate of 95% by 5.33% compared to the logits-based approach.

External IDs:doi:10.1007/978-3-031-78395-1_1