A Unified and Data-Efficient Framework for Out-of-Distribution and Generalization

Zhaohui Hu, Hongli Xiao, Yonglin Li, Chuan Li, Yang Shi, Mengzhu Wang, Haotian Wang, Long Lan

Keywords: Out-of-distribution Detection, Trustworthy Machine Learning

TL;DR: This paper analyzes the optimization conflicts that arise when simultaneously addressing OOD detection and generalization, and proposes a unified method to tackle both tasks. Extensive experiments demonstrate the effectiveness of the proposed method.

Abstract: Machine learning classification models inevitably encounter two types of out-of-distribution (OOD) data in practical applications, i.e., covariate-shifted data and semantic-shifted data. Current approaches typically address them separately in OOD detection and generalization tasks, resulting in limited capability to handle both types of OOD data simultaneously. This limitation motivates us to tackle both challenges within a unified framework. However, our theoretical investigation uncovers a conflict in jointly addressing these two problems, namely Optimization Conflict (OC). Moreover, collecting OOD data remains a significant challenge, making it difficult to provide models with sufficient OOD samples for effective learning. To this end, we propose a novel method called Tackling OOD Detection and Generalization (TODG), which incorporates a regularization term to mitigate OC and employs a data generation strategy to alleviate the scarcity of OOD data. Extensive experiments demonstrate that TODG outperforms existing methods, showcasing its effectiveness in both OOD detection and generalization tasks.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Submission Number: 3455