On the Value of Target Data in Transfer Learning

Steve Hanneke, Samory Kpotufe

Abstract: We aim to understand the value of target labels in transfer-learning, towards addressing practical questions such as how much labeled target data to sample if any, how to optimally mix or re-weight source and target data, while accounting for the usually higher costs of obtaining labeled target data. To this aim, we establish the first minimax-rates in terms of both source and target sample sizes, and show that performance limits are captured by new notions of discrepancy between source and target, which we refer to as transfer exponents. Interestingly, we find that attaining minimax performance is akin to ignoring one of the source or target samples, provided distributional parameters were known a priori. Moreover, we show that practical decisions such as the above can be made in a minimax-optimal way without prior knowledge of distributional parameters nor of the discrepancy between source and target distributions.

CMT Num: 5227