An Information-Theoretic Metric of Transferability for Task Transfer Learning

Yajie Bao, Yang Li, Shao-Lun Huang, Lin Zhang, Amir R. Zamir, Leonidas J. Guibas

Abstract: An important question in task transfer learning is to determine task transferability, i.e. given a common input domain, estimating to what extent representations learned from a source task can help in learning a target task. Typically, transferability is either measured experimentally or inferred through task relatedness, which is often defined without a clear operational meaning. In this paper, we present a novel metric, H-score, an easily-computable evaluation function that estimates the performance of transferred representations from one task to another in classification problems. Inspired by a principled information theoretic approach, H-score has a direct connection to the asymptotic error probability of the decision function based on the transferred feature. This formulation of transferability can further be used to select a suitable set of source tasks in task transfer learning problems or to devise efficient transfer learning policies. Experiments using both synthetic and real image data show that not only our formulation of transferability is meaningful in practice, but also it can generalize to inference problems beyond classification, such as recognition tasks for 3D indoor-scene understanding.

Keywords: transfer learning, task transfer learning, H-score, transferability

TL;DR: We present a provable and easily-computable evaluation function that estimates the performance of transferred representations from one learning task to another in task transfer learning.

Code: [ YaojieBao/An-Information-theoretic-Metric-of-Transferability](https://github.com/YaojieBao/An-Information-theoretic-Metric-of-Transferability)

Data: [CIFAR-100](https://paperswithcode.com/dataset/cifar-100), [Taskonomy](https://paperswithcode.com/dataset/taskonomy)