Variance-Gated Ensembles: An Epistemic-Aware Framework for Uncertainty Estimation

Abstract: Machine learning applications require fast and reliable per-sample uncertainty estimation. A common approach is to use predictive distributions from Bayesian or approximation methods and additively decompose uncertainty into aleatoric (data-related) and epistemic (model-related) components. However, additive decomposition has recently been questioned, with evidence that it breaks down when using finite-ensemble sampling and/or mismatched predictive distributions. This paper introduces Variance-Gated Ensembles (VGE), an intuitive, differentiable framework that injects epistemic sensitivity via a signal-to-noise gate computed from ensemble statistics. VGE provides: (i) a Variance-Gated Margin Uncertainty (VGMU) score that couples decision margins with ensemble predictive variance; and (ii) a Variance-Gated Normalization (VGN) layer that generalizes the variance-gated uncertainty mechanism to training via per-class, learnable normalization of ensemble member probabilities. We derive closed-form vector-Jacobian products enabling end-to-end training through ensemble sample mean and variance. VGE matches or exceeds state-of-the-art information-theoretic baselines while remaining computationally efficient. As a result, VGE provides a practical and scalable approach to epistemic-aware uncertainty estimation in ensemble models.

Submission Type: Long submission (more than 12 pages of main content)

Previous TMLR Submission Url: https://openreview.net/forum?id=Phm0YwQBo0

Changes Since Last Submission:

This is a resubmission of Paper 7412, which received a desk reject due to blinding issues. The following changes have been made: (1) removed author names from the author contribution statement, (2) anonymized the appendix, and (3) verified that the supplementary code does not contain any identifying information. No substantive changes were made to the technical content of the paper.
We note that linking this resubmission to the previous submission number (7412) may allow reviewers to access the original submission where the blinding was compromised. We proceeded as instructed by the Action Editor but wish to flag this potential anonymity concern.

Assigned Action Editor: ~Michele_Caprio1

Submission Number: 7714