Self-supervision based Semantic Alignment for Unsupervised Domain Adaptation

Ronghang Zhu, Sheng Li

Abstract: Unsupervised domain adaptation aims to learn domain-invariant features across domains to transfer knowledge from a well-labeled source domain to an unlabeled target domain. Recently, some unsupervised domain adaptation methods focus on semantically aligning data distributions with pseudo-labels of the target domain. However, semantic alignment based on pseudo-labels has potential risks, e.g., inaccurate pseudo-labeling from classifier, and error accumulation from pseudo-label bias. To alleviate these risks, we propose a novel self-supervision based semantic alignment (S3A) approach for unsupervised domain adaptation, which can jointly incorporate the source alignment and cross-domain target alignment for better semantic alignment across domains. S3A consists of a two-stage semantic alignment procedure with self-supervision. One is to capture the discriminative structure of source domain by aligning source data to source class prototypes, and the other is to match each target data to its neighbor in source domain with self-supervision. Extensive experiments on three benchmark datasets demonstrate the effectiveness of our proposed method, compared with the representative adversarial learning and self-supervised learning based unsupervised domain adaptation methods.