Empirical Bayesian Methods and BNNs for Medical OOD Detection

Kevin Raina

Keywords: Empirical bayes · Out-of-distribution detection · Bayesian neural networks.

Abstract:

Bayesian Neural Networks (BNNs) are a principled way to
incorporate epistemic uncertainty into deep learning, and they play a significant
role in out-of-distribution (OOD) detection, especially in settings
where estimating predictive uncertainty is crucial. Empirical Bayesian
methods, which initialize priors and surrogate posteriors from the weights
of pretrained deterministic neural networks, can help in OOD detection
by providing well-informed models, thereby bridging the gap between
data-driven learning and principled uncertainty estimation — especially
when true Bayesian inference is intractable. In this work, the empirical
Bayes method MOdel Priors with Empirical Bayes using Deterministic
neural networks (MOPED) is adapted to include a Gaussian mixture
prior. Experiments on the medical datasets D7P and BreastMNIST, with
OOD images containing artefacts such as rulers and annotations, demonstrate
marked improvements in OOD detection from the proposed prior
with predictive entropy as the score. The proposed empirical Bayes methods
also performs on par with state-of-the art OOD measures.

Submission Number: 20