SIMILAR: Submodular Information Measures Based Active Learning In Realistic Scenarios

Suraj Nandkishor Kothawade, Nathan Alexander Beck, Krishnateja Killamsetty, Rishabh K Iyer

Keywords: Submodularity, Active Learning, Information Measures, Realistic, Redundancy, Rare Classes, Out of distribution, Robust

TL;DR: A unified active learning framework for active learning in realistic scenarios.

Abstract: Active learning has proven to be useful for minimizing labeling costs by selecting the most informative samples. However, existing active learning methods do not work well in realistic scenarios such as imbalance or rare classes,out-of-distribution data in the unlabeled set, and redundancy. In this work, we propose SIMILAR (Submodular Information Measures based actIve LeARning), a unified active learning framework using recently proposed submodular information measures (SIM) as acquisition functions. We argue that SIMILAR not only works in standard active learning but also easily extends to the realistic settings considered above and acts as a one-stop solution for active learning that is scalable to large real-world datasets. Empirically, we show that SIMILAR significantly outperforms existing active learning algorithms by as much as ~5%−18%in the case of rare classes and ~5%−10%in the case of out-of-distribution data on several image classification tasks like CIFAR-10, MNIST, and ImageNet.

Code Of Conduct: I certify that all co-authors of this work have read and commit to adhering to the NeurIPS Statement on Ethics, Fairness, Inclusivity, and Code of Conduct.

Code: https://github.com/decile-team/distil