LINe: Out-of-distribution detection by leveraging important neurons

Yong Hyun Ahn, Gyeong-Moon Park, Seong Tae Kim

Abstract:

It is important to quantify the uncertainty of input samples, especially in mission-critical domains such as autonomous driving and healthcare, where failure predictions
on out-of-distribution (OOD) data are likely to cause big
problems. OOD detection problem fundamentally begins
in that the model cannot express what it is not aware of.
Post-hoc OOD detection approaches are widely explored
because they do not require an additional re-training process which might degrade the model’s performance and increase the training cost. In this study, from the perspective of neurons in the deep layer of the model representing
high-level features, we introduce a new aspect for analyzing
the difference in model outputs between in-distribution data
and OOD data. We propose a novel method, Leveraging
Important Neurons (LINe), for post-hoc Out of distribution
detection. Shapley value-based pruning reduces the effects
of noisy outputs by selecting only high-contribution neurons
for predicting specific classes of input data and masking the
rest. Activation clipping fixes all values above a certain
threshold into the same value, allowing LINe to treat all
the class-specific features equally and just consider the difference between the number of activated feature differences
between in-distribution and OOD data. Comprehensive experiments verify the effectiveness of the proposed method
by outperforming state-of-the-art post-hoc OOD detection
methods on CIFAR-10, CIFAR-100, and ImageNet datasets.
Code is available on https://github.com/LINe-OOD