SOSELETO: A Unified Approach to Transfer Learning and Training with Noisy Labels

Or Litany, Daniel Freedman

Abstract: We present SOSELETO (SOurce SELEction for Target Optimization), a new method for exploiting a source dataset to solve a classification problem on a target dataset. SOSELETO is based on the following simple intuition: some source examples are more informative than others for the target problem. To capture this intuition, source samples are each given weights; these weights are solved for jointly with the source and target classification problems via a bilevel optimization scheme. The target therefore gets to choose the source samples which are most informative for its own classification task. Furthermore, the bilevel nature of the optimization acts as a kind of regularization on the target, mitigating overfitting. SOSELETO may be applied to both classic transfer learning, as well as the problem of training on datasets with noisy labels; we show state of the art results on both of these problems.

Keywords: transfer learning

TL;DR: Learning with limited training data by exploiting "helpful" instances from a rich data source.

Code: [ orlitany/SOSELETO](https://github.com/orlitany/SOSELETO)

Data: [CIFAR-10](https://paperswithcode.com/dataset/cifar-10), [MNIST](https://paperswithcode.com/dataset/mnist), [SVHN](https://paperswithcode.com/dataset/svhn)

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:1805.09622/code)