Uncovering Confident Failures: The Complementary Roles of Aleatoric and Epistemic Uncertainty in LLMs

Kimia Hamidieh, Veronika Thost, Walter Gerych, Mikhail Yurochkin, Marzyeh Ghassemi

Keywords: Uncertainty Quantification, Language Models, Epistemic Uncertainty

TL;DR: We show that epistemic uncertainty, estimated via cross-model semantic disagreement, complements aleatoric uncertainty and improves reliability in LLM predictions.

Abstract: Large language models (LLMs) often produce confident yet incorrect responses, and uncertainty quantification in LLMs is one potential solution to more robust usage. Recent works routinely rely on self-consistency to estimate aleatoric uncertainty (AU), yet this proxy collapses precisely when models are overconfident, and produce the same incorrect answer across samples. We address this failure mode by introducing an epistemic term that measures semantic disagreement across a small ensemble of scale-matched LLMs. Specifically, we operationalize epistemic uncertainty (EU) as the gap between inter-model and intra-model response similarity, and define total uncertainty (TU) as the sum of AU and EU. The estimator is training-free and uses only black-box outputs: a few responses per model suffice. Across a range of LLMs, and long-form generation tasks, we compare TU to AU and measure uncertainty calibration by AUROC with respect to correctness and selective abstention via uncertainty thresholding. We find that TU consistently achieves higher AUROC in predicting correctness and improves selective abstention compared to AU alone. EU further exposes confident errors that AU misses, especially on tasks with near-unique correct answers, and improves the reliability of LLM uncertainty estimates.

Submission Number: 67