GAUSS: Graph-Assisted Uncertainty Quantification using Structure and Semantics for Long-Form Generation in LLMs

ICLR 2026 Conference Submission19668 Authors

Keywords: uncertainty quantification, graph alignment, paragraph uncertainty

Abstract: In high-stakes domains such as clinical reporting, legal analysis, and policy drafting, large language models (LLMs) are increasingly tasked with generating extended, fact-rich narratives rather than isolated sentences. Accurately quantifying uncertainty in these long-form outputs is essential for ensuring their reliability. Prior approaches either assign a single confidence score to an entire paragraph, often using other LLMs or assess factual consistency by comparing discrete atomic facts derived from the paragraphs across multiple generations. Some recent methods also incorporate graph-based representations, modeling fact–paragraph structures as bipartite entailment graphs and derive uncertainty from node centrality of the facts. However, these methods overlook the interdependencies among atomic facts within a paragraph, as well as the explicit organizational, structural and semantic variation across multiple paragraphs generated by an LLM for the same query, thereby missing a key source of uncertainty inherent specifically to long-form generation. In this work, we introduce GAUSS (Graph-Assisted Uncertainty Quantification using Structure and Semantics for Long-Form Generation in LLMs), a principled framework for measuring uncertainty in long-form LLM outputs through graph-based alignment. Each generated paragraph is modeled as a semantic graph, where nodes represent atomic facts about the paragraph and edges capture inter-fact relationships. We hypothesize that uncertainty arises from structural and semantic discrepancies among these graphs across different generated paragraph samples. GAUSS formalizes this intuition by computing an uncertainty score as the expected alignment cost between the semantic graph of an anchor paragraph and those of alternative reference paragraphs generated by the LLM. By jointly capturing both semantic content and structural coherence of the generated texts, GAUSS moves beyond coarse sentence-level scores to offer a more interpretable and theoretically grounded approach to uncertainty quantification.

Primary Area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)

Submission Number: 19668