Domain Generalization via Conditional Invariant Representations

Ya Li, Mingming Gong, Xinmei Tian, Tongliang Liu, Dacheng Tao

Abstract: Domain generalization aims to apply knowledge gained from multiple labeled source domains to unseen target domains. The main difficulty comes from the dataset bias: training data and test data have different distributions, and the training set contains heterogeneous samples from different distributions. Let X$ denote the features, and Y$ be the class labels. Existing domain generalization methods address the dataset bias problem by learning a domain-invariant representation h(X)$ that has the same marginal distribution \mathbb{P}(h(X))$ across multiple source domains. The functional relationship encoded in \mathbb{P}(Y|X)$ is usually assumed to be stable across domains such that \mathbb{P}(Y|h(X))$ is also invariant. However, it is unclear whether this assumption holds in practical problems. In this paper, we consider the general situation where both \mathbb{P}(X)$ and \mathbb{P}(Y|X)$ can change across all domains. We propose to learn a feature representation which has domain-invariant class conditional distributions \mathbb{P}(h(X)|Y)$. With the conditional invariant representation, the invariance of the joint distribution \mathbb{P}(h(X),Y)$ can be guaranteed if the class prior \mathbb{P}(Y)$ does not change across training and test domains. Extensive experiments on both synthetic and real data demonstrate the effectiveness of the proposed method.