Geometric Uncertainty for Detecting and Correcting Hallucinations in LLMs

Edward Phillips, Sean Wu, Soheila Molaei, Danielle Belgrave, Anshul Thakur, David A. Clifton

Keywords: LLMs, hallucination detection, uncertainty quantification, black-box models, LLM reliability

TL;DR: We introduce a black-box geometric framework that detects and corrects LLM hallucinations by analyzing the convex geometry of response embeddings, offering both global and local uncertainty estimates.

Abstract: Large language models demonstrate impressive results across diverse tasks but are still known to hallucinate, generating linguistically plausible but incorrect answers to questions. Uncertainty quantification has been proposed as a strategy for hallucination detection, requiring estimates for both global uncertainty (attributed to a batch of responses) and local uncertainty (attributed to individual responses). While recent black-box approaches have shown some success, they often rely on disjoint heuristics or graph-theoretic approximations that lack a unified geometric interpretation. We introduce a geometric framework to address this, based on archetypal analysis of batches of responses sampled with only black-box model access. At the global level, we propose Geometric Volume, which measures the convex hull volume of archetypes derived from response embeddings. At the local level, we propose Geometric Suspicion, which leverages the spatial relationship between responses and these archetypes to rank reliability, enabling hallucination reduction through preferential response selection. Unlike prior methods that rely on discrete pairwise comparisons, our approach provides continuous semantic boundary points which have utility for attributing reliability to individual responses. Experiments show that our framework performs comparably to or better than prior methods on short form question-answering datasets, and achieves superior results on medical datasets where hallucinations carry particularly critical risks. We also provide theoretical justification by proving a link between convex hull volume and entropy.

Primary Area: foundation or frontier models, including LLMs

Submission Number: 4876