OViP: Online Vision-Language Preference Learning with Hallucination-Targeted Negatives

ACL ARR 2026 January Submission10241 Authors

Keywords: LVLM, hallucination, online learning

Abstract:

Large vision-language models (LVLMs) remain prone to hallucination. While recent visual Direct Preference Optimization (DPO) methods attempt to mitigate this issue by contrasting positive images with negative ones for the same correct response, they often rely on heuristic or offline constructed negative images that yield weak learning signals where the model already assigns low probability to the correct response.
We propose Online Vision-Language Preference Learning (OViP), a framework that constructs challenging negative images on the fly by explicitly targeting the model's own hallucinations. Specifically, OViP synthesizes negative images using a diffusion model guided by semantic discrepancies between the model's positive and negative responses, producing hallucination-targeted negatives that maintain non-trivial model probability while being semantically misaligned.
Moreover, we refine existing evaluation protocols to better capture the trade-off between hallucination mitigation and informativeness. Experiments on hallucination and general benchmarks show that OViP effectively reduces hallucinations while preserving core multi-modal capabilities, and also improves training efficiency.

Paper Type: Long

Research Area: NLP Applications

Research Area Keywords: multimodal applications

Languages Studied: English

Submission Number: 10241