Last-layer committee machines for uncertainty quantification
Keywords: uncertainty estimation, committee machine, Bayesian neural network, Monte Carlo sampling, human-in-the-loop, ocean data
TL;DR: We propose a last-layer committee machine (LLCM) as an efficient approach to uncertainty quantification and show application to difficult ocean floor images.
Abstract: We introduce here a form of committee machines that gives good predictions of classification confidence, while being computationally efficient.
The initial development of this method was motivated by our work on benthic mapping based on a large dataset of ocean floor images.
These wild type images vary dramatically in terms of their classification difficulty and often result in low inter-rater agreement.
We show that our method is able to identify difficult to classify images using model uncertainty, consistent with Bayesian neural networks and Monte Carlo sampling.
However, our method drastically reduces the computational requirements and offers a more efficient strategy.
This enables us to provide these uncertain predictions to a human specialist and offers a form of active learning to enhance the classification accuracy of the dataset.
We provide both a benchmark study to demonstrate this approach and first results of the BenthicNet dataset.
Submission Number: 75
Loading