Quantifying Uncertainty in Natural Language Explanations of Large Language Models
Keywords: Large Language Models, Explainability, Uncertainty Quantification
TL;DR: We present a way to estimate confidence of natural language explanations of LLMs. We use consistency of perturbed explanations as a way to estimate explanation confidence., and show that explanation confidence is correlated with faithfulness.
Abstract: Large Language Models (LLMs) are increasingly used as powerful tools for several high-stakes natural language processing (NLP) applications. Recent works on prompting claim to elicit intermediate reasoning steps and key tokens that serve as proxy explanations for LLM predictions. However, there is no certainty whether these explanations are reliable and reflect the LLM’s behavior. In this work, we make one of the first attempts at quantifying the uncertainty in explanations of LLMs. To this end, we propose two novel metrics --- $\textit{Verbalized Uncertainty}$ and $\textit{Probing Uncertainty}$ --- to quantify the uncertainty of generated explanations. While verbalized uncertainty involves prompting the LLM to express its confidence in its explanations, probing uncertainty leverages sample and model perturbations as a means to quantify the uncertainty. Our empirical analysis of benchmark datasets reveals that verbalized uncertainty is not a reliable estimate of explanation confidence. Further, we show that the probing uncertainty estimates are correlated with the faithfulness of an explanation, with lower uncertainty corresponding to explanations with higher faithfulness. Our study provides insights into the challenges and opportunities of quantifying uncertainty in LLM explanations, contributing to the broader discussion of the trustworthiness of foundation models.
Submission Number: 110
Loading