Evi-BALD: Bayesian Active Learning by Disagreement via Evidential Deep Learning

Minghao Li, Weishi Shi

Keywords: Active Learning, Evidential Learning, Bayesian Inference

Abstract: Bayesian Active Learning by Disagreement (BALD) is a fundamental acquisition function for active learning that measures the mutual information between model parameters and predictions. However, existing BALD computation methods rely on Monte Carlo sampling through ensembles or MC dropout, which require training and maintaining multiple independent models, leading to substantial computational overhead and time consumption. In this paper, we propose Evi-BALD, a hierarchical Bayesian approach that computes BALD efficiently using a single model instead of Monte Carlo sampling. Our method builds a second-order distribution over the predictive distribution, where the second-order parameters are directly predicted by the neural network, eliminating the need for multiple model instances. To address the computational challenge of evaluating intractable integrals in BALD calculation, we leverage Taylor expansion and demonstrate that BALD can be represented as the sum of quadratic terms and higher-order remainder terms from the entropy Taylor expansion. Through experiments on CIFAR-10, Evi-BALD achieves leading performance in active learning while significantly reducing computational time, demonstrating both efficiency and effectiveness.

Submission Number: 71