Deep Ensemble Bayesian Active Learning : Adressing the Mode Collapse issue in Monte Carlo dropout via Ensembles

Remus Pop, Patric Fulop

Abstract:

In image classification tasks, the ability of deep convolutional neural networks (CNNs) to deal with complex image data has proved to be unrivalled. Deep CNNs, however, require large amounts of labeled training data to reach their full potential. In specialised domains such as healthcare, labeled data can be difficult and expensive to obtain. One way to alleviate this problem is to rely on active learning, a learning technique that aims to reduce the amount of labelled data needed for a specific task while still delivering satisfactory performance.
We propose a new active learning strategy designed
for deep neural networks. This method improves upon the current state-of-the-art deep Bayesian active learning method, which suffers from the mode collapse problem. We correct for this deficiency by making use of the expressive power and statistical properties of model ensembles. Our proposed method manages to capture superior data uncertainty, which translates into improved classification performance. We demonstrate empirically that our ensemble method yields faster convergence of CNNs trained on the MNIST and CIFAR-10
datasets.

Keywords: Active Learning, Deep Learning, Bayesian Neural Networks, Bayesian Deep Learning, Ensembles

TL;DR: We present a method for Deep Bayesian Active Learning combining MC-Dropout with Ensemble Models

Data: [CIFAR-10](https://paperswithcode.com/dataset/cifar-10)