Towards Explainable Bilingual Multimodal Misinformation Detection and Localization

ICLR 2026 Conference Submission13446 Authors

Keywords: Misinformation, Multimodal, LLM, GRPO

TL;DR: We propose BiMi,a bilingual framework for misinformation detection with localization and cross-lingual reasoning,and introduce BiMiBench,a 104K-sample benchmark built from real news with diverse multimodal manipulations.

Abstract: The increasing realism of multimodal content has made misinformation more subtle and harder to detect, especially in news media where images are frequently paired with bilingual (e.g., Chinese-English) subtitles. Such content often includes localized image edits and cross-lingual inconsistencies that jointly distort meaning while remaining superficially plausible. We introduce BiMi, a bilingual multimodal framework that jointly performs region-level localization, cross-modal and cross-lingual consistency detection, and natural language explanation for misinformation analysis. To support generalization, BiMi integrates an online retrieval module that supplements model reasoning with up-to-date external context. We further release BiMiBench, a large-scale and comprehensive benchmark constructed by systematically editing real news images and subtitles, comprising 104,000 samples with realistic manipulations across visual and linguistic modalities. To enhance interpretability, we apply Group Relative Policy Optimization (GRPO) to improve explanation quality, marking the first use of GRPO in this domain. Extensive experiments demonstrate that BiMi outperforms strong baselines by up to +8.9 in classification accuracy, +15.9 in localization accuracy, and +2.5 in explanation BERTScore, advancing state-of-the-art performance in realistic, bilingual misinformation detection. Code, models, and datasets will be released.

Primary Area: applications to computer vision, audio, language, and other modalities

Submission Number: 13446