Label-Covariate Shift Chain: Unsupervised Domain Adaptation for Factorizable Joint Shift

Jinzong Dong, Zhaohui Jiang, Bo Yang

Keywords: Factorizable Joint Shift, Unsupervised Domain Adaptation, Covariate Shift, Label Shift

Abstract: The effectiveness of modern machine learning models for various tasks is fundamentally dependent on the presumption that training and test data are independent and identically distributed ($i.i.d.$). However, in some real-world scenarios, $i.i.d.$ is a luxury, i.e., distribution shifts often exist between training and test data. Factorizable joint shift is a new type of distribution shift, and unlike marginal shift (e.g., label shift or covariate shift) with strong assumptions, it imposes fewer constraints and provides broader applicability. However, unsupervised domain adaptation under factorizable joint shift is an unresolved and understudied problem. Previous methods easily collapse to trivial solutions, require the subjective selection of fixed constants, and fail to ensure the solution's existence and uniqueness when the number of categories exceeds two. To address this problem, we propose a principled method to find a non-trivial solution in a tractable manner. We first re-represent factorizable joint shift as a \textit{Label-Covariate Shift Chain}, where label shift occurs first and then covariate shift occurs, which makes factorizable joint shift more tractable. Then, \textit{Covariate Shift Minimization Principle} is introduced on the \textit{Label-Covariate Shift Chain} to obtain a non-trivial solution. Furthermore, we propose a method to generate real-world factorizable joint shift datasets using \textit{Label-Covariate Shift Chain}, and these datasets can serve as benchmarks to evaluate the effectiveness of generalization methods. Finally, the effectiveness of the proposed method is verified using real-world data for both accuracy improvement and confidence calibration tasks. We believe our exploration of factorizable joint shift will help modern machine learning models handle a wider variety of complex data scenarios, advancing the broader application of AI.

Primary Area: other topics in machine learning (i.e., none of the above)

Submission Number: 8820