On the Minimal Supervision for Training Any Binary Classifier from Only Unlabeled Data

Nan Lu, Gang Niu, Aditya Krishna Menon, Masashi Sugiyama

Abstract: Empirical risk minimization (ERM), with proper loss function and regularization, is the common practice of supervised classification. In this paper, we study training arbitrary (from linear to deep) binary classifier from only unlabeled (U) data by ERM. We prove that it is impossible to estimate the risk of an arbitrary binary classifier in an unbiased manner given a single set of U data, but it becomes possible given two sets of U data with different class priors. These two facts answer a fundamental question---what the minimal supervision is for training any binary classifier from only U data. Following these findings, we propose an ERM-based learning method from two sets of U data, and then prove it is consistent. Experiments demonstrate the proposed method could train deep models and outperform state-of-the-art methods for learning from two sets of U data.

Keywords: learning from only unlabeled data, empirical risk minimization, unbiased risk estimator

TL;DR: Three class priors are all you need to train deep models from only U data, while any two should not be enough.

Code: [ lunanbit/UUlearning](https://github.com/lunanbit/UUlearning)

Data: [CIFAR-10](https://paperswithcode.com/dataset/cifar-10), [Fashion-MNIST](https://paperswithcode.com/dataset/fashion-mnist), [MNIST](https://paperswithcode.com/dataset/mnist), [SVHN](https://paperswithcode.com/dataset/svhn)

Community Implementations: [ 2 code implementations](https://www.catalyzex.com/paper/arxiv:1808.10585/code)