Open-Set Domain Adaptation Under Background Distribution Shift: Challenges and A Provably Efficient Solution

Shravan S Chaudhari, Yoav Wald, Suchi Saria

Keywords: Distribution Shift, Open Set Domain Adaptation, Novel Category Detection, Anomaly Detection, Out of Distribution

TL;DR: We study the problem of Open-Set Domain Adaptation under distribution shift of non-novel classes and study the impact of various factors affecting the performance of existing methods such as novel class ratio.

Abstract:

In Open-Set Domain Adaptation (OSDA) we wish to perform classification in a target domain which contains a novel class along with $k$ non-novel classes. This work formally studies OSDA under the assumption that classes are separable, and the supports of source and target domains coincide, while other aspects of the distribution may change. We term such a distribution shift as background shift.
We develop a simple and scalable OSDA method that attains robustness to background shift and is guaranteed to solve the problem, while showing that it cannot be solved under weaker conditions for OSDA studied in the past, particularly in the presence of covariate shift. We formally define the realistic assumptions of background shift within the scope of OSDA problem that the previous literature has either overlooked or not explicitly addressed. In a thorough empirical evaluation on both image and text data, we observe that existing OSDA methods are not robust to the distribution shifts we consider.
Our proposed solution jointly learns representations via concurrently learning to classify known categories and detect novel ones using methods with formal guarantees. The results demonstrate that optimizing these two objectives in unison leads to mutual performance improvements contrary to what might be expected when objectives are considered independently. Our rigorous empirical study also examines how OSDA performance under distribution shift is affected by parameters of the problem such as the novel class size.
Taken together, our observations emphasize the importance of formalizing assumptions under which OSDA methods operate and to develop appropriate methodology that is capable of scaling with large datasets and models for different scenarios of OSDA.

Primary Area: alignment, fairness, safety, privacy, and societal considerations

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 12674