Representation Norm Amplification for Out-of-Distribution Detection in Long-Tail Learning

Dong Geun Shin, Hye Won Chung

Keywords: Out-of-distribution detection, long-tailed recognition

TL;DR: We provide a new OOD detection method in long-tail learning, called Representation Norm Amplification (RNA), which can effectively decouple the classification and OOD detection problems.

Abstract: Detecting out-of-distribution (OOD) samples is a critical task for reliable machine learning. However, this task becomes particularly challenging when the models are trained on long-tailed datasets, as the models often struggle to distinguish tail-class in-distribution samples from OOD samples. We examine the main challenges in this problem by identifying the trade-offs between OOD detection and in-distribution (ID) classification, faced by existing methods. We then introduce our method, called Representation Norm Amplification (RNA), which solves this challenge by decoupling the two problems. The main idea is to use the norm of the representation as a new dimension for OOD detection, and to develop a training method that generates a noticeable discrepancy in the representation norm between ID and OOD data, while not perturbing the feature learning for in-distribution classification. Our experiments show that RNA achieves superior performance in both OOD detection and classification compared to the state-of-the-art methods, by 2.36\%, 1.17\%, and 7.38\% in AUROC and 2.20\%, 0.95\%, and 2.84\% in classification accuracy on CIFAR10-LT, CIFAR100-LT, and ImageNet-LT, respectively.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 4320