GEM-FI: Gated Evidential Mixtures with Fisher Modulation

Marco Mustafa Mohammed, Fatemeh Daneshfar, Pietro Lio

TL;DR: GEM-FI improves single-pass uncertainty estimation by combining energy-guided evidential gating, mixture-based epistemic modeling, and Fisher-informed regularization for better calibration and OOD detection.

Abstract: Evidential Deep Learning (EDL) enables single-pass uncertainty estimation by predicting Dirichlet evidence, but it can remain overconfident and poorly calibrated, and it often fails to represent multi-modal epistemic uncertainty. We introduce Gated Evidential Mixtures (GEM), a family of models that learns an in-model energy signal and uses it to gate evidential outputs end-to-end in a distance-informed manner. GEM-CORE learns a feature-level energy and maps it to a bounded gate that smoothly suppresses evidence when support is low. To capture epistemic multi-modality without multi-pass ensembling, GEM-MIX adds a lightweight mixture of evidential heads with learned routing weights while preserving single-pass inference. Finally, GEM-FI stabilizes mixture allocations via a Fisher-informed regularizer, reducing head collapse and producing smoother boundary uncertainty. Across image classification and OOD detection benchmarks, GEM improves calibration and ID/OOD separation with single-pass inference. On CIFAR-10, GEM-FI vs. DAEDL improves Acc. from 91.11 to 93.75 (+2.64 pp), reduces Brier ×100 from 14.27 to 6.81 (−7.46), and also improves misclassification-detection (AUPR) from 99.08 to 99.94 (+0.86). For epistemic OOD detection, GEM-FI achieves AUPR/AUROC of 92.59/95.09 on CIFAR-10→SVHN and 90.20/89.06 on CIFAR-10→CIFAR-100 (vs. 85.54/89.30 and 88.19/86.10 for DAEDL).

Lay Summary: Modern machine learning systems often make confident predictions even when they are wrong or when the input is very different from the data they were trained on. This can be risky in real-world applications where users need to know when a model is uncertain. This paper introduces GEM-FI, a method that helps a model estimate uncertainty more reliably in a single forward pass. GEM-FI learns when to reduce confidence, uses multiple evidential prediction heads to represent different possible explanations, and applies Fisher-informed regularization to make these predictions more stable. Experiments show that GEM-FI improves calibration and out-of-distribution detection while keeping inference efficient.

Link To Code: https://github.com/Marcorazhan/GEM-FI

Primary Area: Probabilistic Methods

Keywords: Uncertainty Estimation, Evidential Deep Learning, Energy-based Models, Fisher Information, OOD Detection

Submission Number: 2893