Credal Bayesian Deep Learning
Abstract: Uncertainty quantification and robustness to distribution shifts are important goals in machine learning and artificial intelligence. Although Bayesian Neural Networks (BNNs) allow for uncertainty in the predictions to be assessed, different sources of uncertainty are indistinguishable. We present Credal Bayesian Deep Learning (CBDL).
Heuristically, CBDL allows to train an (uncountably) infinite ensemble of BNNs, using only finitely many elements. This is possible thanks to prior and likelihood finitely generated credal sets (FGCSs), a concept from the imprecise probability literature. Intuitively, convex combinations of a finite collection of prior-likelihood pairs are able to represent infinitely many such pairs. After training, CBDL outputs a set of posteriors on the parameters of the neural network. At inference time, such posterior set is used to derive a set of predictive distributions that is in turn utilized to distinguish between aleatoric and epistemic uncertainties, and to quantify them. The predictive set also produces either (i) a collection of outputs enjoying desirable probabilistic guarantees, or (ii) the single output that is deemed the best, that is, the one having the highest predictive lower probability -- another imprecise-probabilistic concept. CBDL is more robust than single BNNs to prior and likelihood misspecification, and to distribution shift. We show that CBDL is better at quantifying and disentangling different types of uncertainties than single BNNs and ensemble of BNNs.
In addition, we apply CBDL to two case studies to demonstrate its downstream tasks capabilities: one, for motion prediction in autonomous driving scenarios, and two, to model blood glucose and insulin dynamics for artificial pancreas control. We show that CBDL performs better when compared to an ensemble of BNNs baseline.
Submission Length: Long submission (more than 12 pages of main content)
Previous TMLR Submission Url: https://openreview.net/forum?id=bolsjmDleF
Changes Since Last Submission: The paper underwent a major revision to meet the reviewers' suggestions. Clarity has been greatly improved, together with the description and discussion of our proposed algorithm, Credal Bayesian Deep Learning (CBDL). We improved on the existing pictures to further improve transparency. We also made uncertainty quantification the main focus point of the paper. We show empirically and discuss how CBDL improves on single and ensemble of BNNs both in uncertainty quantification, and in downstream task performances. We also discuss explicitly why we avoid comparison with non-Bayesian techniques. Furthermore, we explain why Bayesian Model Averaging is not selected as a possible baseline for our method, and we describe how we plan to derive a credal version of Bayesian Model Selection in the near future.
Code: https://github.com/PRECISE/credal-bayesian-deep-learning.git
Assigned Action Editor: ~Manuel_Haussmann1
Submission Number: 2714
Loading