Cooperative variance estimation and Bayesian neural networks disentangle aleatoric and epistemic uncertainties

ICLR 2026 Conference Submission12822 Authors

Keywords: Bayesian neural networks, Variance networks, Mixture Density Networks, Aleatoric uncertainty, Epistemic uncertainty, Probabilistic and non-probabilistic machine learning

TL;DR: Method capable of separating aleatoric and epistemic uncertainties by sequential training of a variance network with a Bayesian neural network.

Abstract: Real-world data contains aleatoric uncertainty - irreducible noise arising from imperfect measurements or from incomplete knowledge about the data generation process. Mean variance estimation (MVE) networks can learn this type of uncertainty but require ad-hoc regularization strategies to avoid overfitting and are unable to predict epistemic uncertainty (model uncertainty). Conversely, Bayesian neural networks predict epistemic uncertainty but are notoriously difficult to train due to the approximate nature of Bayesian inference. We propose to cooperatively train a variance network with a Bayesian neural network and empirically demonstrate that the resulting model disentangles aleatoric and epistemic uncertainties while improving the mean estimation. We demonstrate the effectiveness and scalability of this method across a diverse range of datasets, including a time-dependent heteroscedastic regression dataset we created where the aleatoric uncertainty is known. The proposed method is straightforward to implement, robust, and adaptable to various model architectures.

Primary Area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)

Submission Number: 12822