Plug-and-Play Uncertainty estimation for NLG

ACL ARR 2026 March Submission2228 Authors

Keywords: uncertainty quantification, NLG, LLM, reliable AI, interpretability, question-answering

Abstract: Despite their remarkable capabilities in NLP, modern LLMs remain prone to ``hallucinations'', raising concerns for high-stakes applications. We propose a dual approach for uncertainty quantification (UQ) in open-source LLMs that addresses the limitations of model adaptation methods and aims to capture the nuances of language generation. While most UQ approaches rely on various sampling methods to generate the output distribution and then compute entropy, we instead use SDLG for sampling generations and develop a novel framework for decomposing uncertainty into \emph{aleatoric} and \emph{epistemic} components. We average the token-level entropy of the important tokens to estimate the aleatoric uncertainty. For epistemic uncertainty, we propose a layer-wise ensembling technique that leverages the modular knowledge representation in transformer models, contrasting early-exit distributions across top model layers while the important token is being generated. Experiments on multiple question-answering benchmarks demonstrate improvement over comparable baselines. Also, our approach requires no architectural modifications or extra training and works efficiently for off-the-shelf models.

Paper Type: Long

Research Area: Interpretability and Analysis of Models for NLP

Research Area Keywords: Uncertainty, interpretability, conversational QA, metrics

Contribution Types: Model analysis & interpretability

Languages Studied: english

Submission Number: 2228