Learning in Confusion: Batch Active Learning with Noisy Oracle

Gaurav Gupta, Anit Kumar Sahu, Wan-Yi Lin

TL;DR: We address the active learning in batch setting with noisy oracles and use model uncertainty to encode the decision quality of active learning algorithm during acquisition.

Abstract:

We study the problem of training machine learning models incrementally using active learning with access to imperfect or noisy oracles. We specifically consider the setting of batch active learning, in which multiple samples are selected as opposed to a single sample as in classical settings so as to reduce the training overhead. Our approach bridges between uniform randomness and score based importance sampling of clusters when selecting a batch of new samples. Experiments on
benchmark image classification datasets (MNIST, SVHN, and CIFAR10) shows improvement over existing active learning strategies. We introduce an extra denoising layer to deep networks to make active learning robust to label noises and show significant improvements.

Keywords: Active Learning, Noisy Oracle, Model Uncertainty, Image classification

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:1909.12473/code)