Visual Evidence Prompting Mitigates Hallucinations in Multimodal Large Language Models

Wei Li, Zhen Huang, Houqiang Li, Le Lu, Yang Lu, Xinmei Tian, Xu Shen, Jieping Ye

Primary Area: representation learning for computer vision, audio, language, and other modalities

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: Large Vision-Language Models, Hallucination, Visual model

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

TL;DR: We mitigate object and relation hallucinations in large vision-language models via a simple method called visual evidence prompting, where a few visual knowledge evidence are provided as contexts in prompting.

Abstract:

Despite the promising progress achieved, Large Vision-Language Models (LVLMs) still suffer from the hallucination problem, i.e., they tend to predict objects and relations which are non-existent in the target images. These unfaithful outputs degrade the model performance and greatly harm the user experiences in real-world applications. Fortunately, traditional small visual models excel at producing professional and faithful outputs, but they are not adept at interacting with humans. Therefore, this work explores how small visual models complement the LVLMs by effectively extracting contextual information from images to generate precise answers.
In particular, we show how such hallucination mitigates naturally in LVLMs via a simple method called visual evidence prompting, where a few visual knowledge evidences are provided as contexts in prompting. Experiments on three large language models show that visual evidence prompting improves performance on the evaluation of object hallucinations, as well as the new benchmark for relation hallucinations. We hope our work will not only serve as the minimal strongest baseline for the challenging hallucination benchmarks, but also highlight the importance of carefully exploring and analyzing the enormous visual evidence hidden inside small visual models before crafting finetuning LVLMs.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 3272