Information-Theoretic Analysis of Unsupervised Domain Adaptation

Ziqiao Wang, Yongyi Mao

Keywords: unsupervised domain adaptation, generalization, information theory, regularization

TL;DR: We derived new information-theoretic generalization bounds for the unsupervised domain adaptation problem.

Abstract: This paper uses information-theoretic tools to analyze the generalization error in unsupervised domain adaptation (UDA). This study presents novel upper bounds for two notions of generalization errors. The first notion measures the gap between the population risk in the target domain and that in the source domain, and the second measures the gap between the population risk in the target domain and the empirical risk in the source domain. While our bounds for the first kind of error are in line with the traditional analysis and give similar insights, our bounds on the second kind of error are algorithm-dependent and also inspire insights into algorithm designs. Specifically, we present two simple techniques for improving generalization in UDA and validate them experimentally.