A Unified Domain Adaptation Framework with Distinctive Divergence Analysis
Abstract: Unsupervised domain adaptation enables knowledge transfer from a labeled source domain to an unlabeled target domain by aligning the learnt features of both domains. The idea is theoretically supported by the generalization bound analysis in Ben-David et al. (2007), which specifies the applicable task (binary classification) and designates a specific distribution divergence measure. Although most distribution-aligning domain adaptation models seek theoretical grounds from this particular bound analysis, they do not actually fit into the stringent conditions. In this paper, we bridge the long-standing theoretical gap in literature by providing a unified generalization bound. Our analysis can well accommodate the classification/regression tasks and most commonly-used divergence measures, and more importantly, it can theoretically recover a large amount of previous models. In addition, we identify the key difference in the distribution divergence measures underlying the diverse models and commit a comprehensive in-depth comparison of the commonly-used divergence measures. Based on the unified generalization bound, we propose new domain adaptation models that achieve transferability through domain-invariant representations and conduct experiments on real-world datasets that corroborate our theoretical findings. We believe these insights are helpful in guiding the future design of distribution-aligning domain adaptation algorithms.
Submission Length: Regular submission (no more than 12 pages of main content)
Changes Since Last Submission: - We better explain the UDA theory when there are conditional shifts in Section 4.
- We clarify that the framework we propose is unified in that it provides more general theoretical guarantees for the distribution-aligning DA methods in the abstract and introduction.
- We add more experimental details to the revised manuscript, including the training objectives and a graphical illustration of the distribution-aligning methodologies in Section 6.
- We design additional regression transfer tasks to verify the proposed framework's effectiveness and the performance of missing models in Section 6.
- We make minor revisions for a better explanation in Section 4.1.1.
Assigned Action Editor: ~Mingming_Gong1
License: Creative Commons Attribution 4.0 International (CC BY 4.0)
Submission Number: 442
Loading