DOS: Dreaming Outlier Semantics for Out-of-distribution Detection

Chentao Cao, Zhun Zhong, Zhanke Zhou, Yang Liu, Tongliang Liu, Bo Han

Primary Area: societal considerations including fairness, safety, privacy

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: Out-of-distribution Detection

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Abstract: Detecting out-of-distribution (OOD) samples is essential when deploying machine learning models in open-world scenarios. Zero-shot OOD detection, requiring no training on in-distribution (ID) data, has been possible with the advent of vision-language models like CLIP. This scenario presents a more practical alternative than traditional OOD detection. By building a text-based classifier with only closed-set labels, the model can achieve impressive OOD detection performance. However, this largely restricts the inherent capability of CLIP to recognize samples from large, open label space, making it insufficient to detect hard OOD samples effectively. In this paper, we provide a new perspective to tackle the constraints posed by exclusively employing closed-set ID labels in zero-shot OOD detection. We propose leveraging the expert knowledge and reasoning capability of large language models (LLM) to Dream potential Outlier Semantics, termed DOS, without access to any actual OOD data. Owing to better consideration of open-world scenarios, DOS can be generalized to different OOD detection tasks, including far, near, and fine-grained OOD detection. Technically, we design (1) LLM prompts based on visual similarity to generate potential outlier class labels specialized for OOD detection, as well as (2) a new score function based on the proportionality between potential outlier and ID class labels to distinguish hard OOD samples effectively. Empirically, our method achieves new state-of-the-art performance across different OOD tasks and can be effectively scaled to the large-scale ImageNet-1K dataset.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 3638