Grounding the Ungrounded: A Spectral-Graph Framework for Quantifying Hallucinations in multimodal LLMs

ICLR 2026 Conference Submission21788 Authors

Keywords: Multimodal large language models, Hallucination detection, Hallucination quantification, Energy-based models, Spectral graph theory, Hypergraph Laplacian, Graph signal processing, Diffusion kernel (heat kernel), Rayleigh–Ritz bounds, KL divergence calibration, Temperature scheduling, Semantic distortion, Cross-modal alignment, RKHS / kernel methods

TL;DR: We propose an energy-based, temperature-controlled spectral hypergraph framework for multimodal LLMs that quantifies hallucinations, yields KL-calibrated Rayleigh–Ritz bounds, and proves diffusion-time decay.

Abstract: Hallucinations in LLMs—especially in multimodal settings—undermine reliability. We present a rigorous, information-geometric framework in diffusion dynamics that quantifies hallucination in MLLMs: model outputs are embedded spectrally on multimodal graph Laplacians, and gaps to a truth manifold define a semantic-distortion metric. We derive Courant–Fischer bounds on a temperature-dependent hallucination energy and use RKHS eigenmodes to obtain modality-aware, interpretable measures that track evolution over prompts and time. This reframes hallucination as measurable and bounded, providing a principled basis for evaluation and mitigation.

Primary Area: foundation or frontier models, including LLMs

Submission Number: 21788