LiveVQA: Assessing Models with Live Visual Knowledge

Mingyang Fu, Yuyang Peng, Benlin Liu, Yao Wan, Dongping Chen

Keywords: Dataset, Synthetic Data, Visual Knowledge, VQA, Multimodal LLM, News

TL;DR: LiveVQA is a automatically collected benchmark dataset that test large models' capabilities of understand and reasoning with latest visual knowledge across 14 news categories.

Abstract: We introduce LiveVQA, an automatically collected dataset of latest visual knowledge from the Internet with synthesized VQA problems. LiveVQA consists of 3,602 single- and multi-hop visual questions from 6 news websites across 14 news categories, featuring high-quality image-text coherence and authentic information. Our evaluation across 15 MLLMs (e.g., GPT-4o, Gemma-3, and Qwen-2.5-VL family) demonstrates that stronger models perform better overall, with advanced visual reasoning capabilities proving crucial for complex multi-hop questions. Despite excellent performance on textual problems, models with tools like search engines still show significant gaps when addressing visual questions requiring latest visual knowledge, highlighting important areas for future research.

Submission Number: 37