Variance-Guided Score Regularization for Hallucination Mitigation in Diffusion Models

ICLR 2026 Conference Submission3841 Authors

Keywords: AI Safety, Diffusion Models

Abstract: Diffusion models have emerged as the backbone of modern generative AI, powering advances in vision, language, audio and other modalities. Despite their success, they suffer from \emph{hallucinations}, implausible samples that lie outside the support of true data distribution, which degrade reliability and trust. In this work, we provide a density-based perspective on hallucinations and derive an explicit lower bound linking \emph{score smoothness} to nonzero probability mass allocated to gap regions outside the data support leading to a positive hallucination rate. To mitigate hallucinations, we introduce a \emph{Variance-Guided Score Regularization} (VSR) strategy that explicitly controls the score Jacobian, thereby reducing the lower bound on gap mass. Empirical results on synthetic and real-world datasets demonstrate that our approach reduces hallucinations ($\sim25\%$) while maintaining high fidelity and diversity, providing a principled step toward more reliable diffusion-based image generation. We also propose two benchmark datasets with controlled semantic variation for systematic hallucination evaluation.

Primary Area: alignment, fairness, safety, privacy, and societal considerations

Submission Number: 3841