Epistemic Uncertainty Quantification To Improve Decisions From Black-Box Models

Sébastien Melo, Gaël Varoquaux, Marine Le Morvan

Keywords: Epistemic uncertainty, Excess risk, Uncertainty quantification, Aleatoric, LLM, Deferral, Calibration

TL;DR: We introduce a novel estimator of epistemic uncertainty in confidence scores that reveals local miscalibration and supports improved decision-making and more efficient LLM cascades.

Abstract: Distinguishing epistemic uncertainty (model ignorance) from aleatoric uncertainty (task randomness) is critical for reliable AI systems, yet standard confidence evaluation metrics capture different and incomplete aspects of uncertainty. While AUC and accuracy measure predictive signal, proper scoring rules assess overall uncertainty, and calibration metrics isolate part of the epistemic uncertainty but ignore within-bin heterogeneity of errors, known as grouping loss. We bridge this evaluation gap by introducing asymptotically consistent and sample-efficient estimators of the grouping loss and excess decision risk, providing a fine-grained assessment of epistemic uncertainty that complements existing calibration metrics. Applied to LLM question-answering with inherent aleatoric noise, our estimators reveal substantial grouping loss which decreases with model scale and instruction tuning. Their local nature enables automatic identification of subgroups with systematic over- or under-confidence, supporting interpretable confidence audits. Finally, we leverage these estimates to design LLM cascades that defer high excess decision risk predictions to stronger models, achieving higher accuracy at lower cost than competing approaches.

Primary Area: other topics in machine learning (i.e., none of the above)

Submission Number: 24572