Investigating and Mitigating the Multimodal Hallucination Snowballing in Large Vision-Language Models

Anonymous

Abstract: Though advanced in understanding visual information with human languages, Large Vision-Language Models (LVLMs) still suffer from multimodal hallucinations. A natural concern is that during multimodal interaction, the generated hallucinations could influence the LVLMs' subsequent generation. Thus, we raise a question: $\textit{When presented with a query relevant to the previously generated hallucination, will LVLMs be misled and respond incorrectly, even though the ground visual information exists?}$ To answer this, we propose a framework to evaluate LVLMs' behaviors when encountering generated hallucinations, where LVLMs are required to answer specific visual questions with a curated hallucinatory conversation. Crucially, our experiment shows that the performance of LVLMs drops by $31\%$ at least, indicating that LVLMs are prone to accept the generated hallucinations and make false claims that they would have not supported without distractions, which we term as $\textit{Multimodal Hallucination Snowballing}$. To mitigate this issue, we further propose a training-free method called $\textit{Residual Visual Decoding},$ where we revise the output distribution of LVLMs that are derived from the residual visual input, which provides models with direct access to the visual information. Experiments show that our method can mitigate more than $24%$ of the snowballed multimodal hallucination while maintaining capabilities.

Paper Type: long

Research Area: Multimodality and Language Grounding to Vision, Robotics and Beyond

Contribution Types: Model analysis & interpretability, NLP engineering experiment, Publicly available software and/or pre-trained models, Data analysis

Languages Studied: English