A Hallucination Metric and Correction Scheme for Diffusion-Based Image Restoration

Daniel Siromani, Di You, Pier Luigi Dragotti

Abstract: AI hallucination affects image inverse problem solvers using deep generative priors. Such models can produce highly realistic generations which are semantically different from the ground truth or input. The risk of hallucination limits application of these models in areas where data consistency is vital. This work considers hallucination in diffusion-based image restoration models. A metric based on deep perceptual similarity and ArcFace face ID is developed to predict the presence of hallucinated content. Using this metric a system to reduce hallucination during generation is developed. The proposed system adapts the guidance scale and diffusion sampling process based on the hallucination metric computed at each timestep. Using the metric within the DPS algorithm resulted in a significant reduction in hallucinations. Hallucinations were also reduced for INDigo to a lesser degree, with artifacts introduced where the model could not fully eliminate the hallucinated content. With a hallucination metric score and system to avoid hallucinations it is possible to reduce the frequency of hallucinations, favour input consistency over realism when a perfect reconstruction is not possible, and indicate to the user the likelihood of the generation being hallucinated.