MEGAN: Mixture of Experts for Robust Uncertainty Estimation in Endoscopy Videos

Damola Agbelese, Krishna Chaitanya, Pushpak Pati, Chaitanya Parmar, Pooya Mobadersany, Shreyas Fadnavis, Lindsey Surace, Shadi Yarandi, Louis R. Ghanem, Molly Lucas, Tommaso Mansi, Gabriela Oana Cula, Pablo F. Damasceno, Kristopher A Standish

Keywords: Uncertainty quantification (UQ) · Ulcerative Colitis (UC) · Evidential deep learning (EDL) · Multi-Expert GAting Network (MEGAN).

Abstract:

Reliable uncertainty quantification (UQ) is essential in medical AI. Evidential
Deep Learning (EDL) offers a computationally efficient way
to quantify model uncertainty alongside predictions, unlike traditional
methods such as Monte Carlo (MC) Dropout and Deep Ensembles (DE).
However, all these methods often rely on a single expert’s annotations
as ground truth for model training, overlooking the inter-rater variability
in healthcare. To address this issue, we propose MEGAN, a Multi-
Expert Gating Network that aggregates uncertainty estimates and predictions
from multiple AI experts via EDL models trained with diverse
ground truths and modeling strategies. MEGAN’s gating network optimally
combines predictions and uncertainties from each EDL model,
enhancing overall prediction confidence and calibration. We extensively
benchmark MEGAN on endoscopy videos for Ulcerative colitis (UC) disease
severity estimation, assessed by visual labeling of Mayo Endoscopic
Subscore (MES), where inter-rater variability is prevalent. In large-scale
prospective UC clinical trial, MEGAN achieved a 3.5% improvement
in F1-score and a 30.5% reduction in Expected Calibration Error (ECE)
compared to existing methods. Furthermore,MEGAN facilitated uncertaintyguided
sample stratification, reducing the annotation burden and potentially
increasing efficiency and consistency in UC trials.

Submission Number: 1