Multimodal Automated Fact-Checking: A Survey

Mubashara Akhtar, Michael Sejr Schlichtkrull, Zhijiang Guo, Oana Cocarascu, Elena Simperl, Andreas Vlachos

Submission Type: Regular Long Paper

Submission Track: Computational Social Science and Cultural Analytics

Submission Track 2: NLP Applications

Keywords: fact checking, multimodality, survey

TL;DR: This survey on multimodal automated fact-checking introduces a framework for multimodal fact-checking and studies datasets and modeling approaches for fact-checking with text, image audio, and video.

Abstract:

Misinformation is often conveyed in multiple modalities, e.g. a miscaptioned image. Multimodal misinformation is perceived as more credible by humans, and spreads faster than its text-only counterparts. While an increasing body of research investigates automated
fact-checking (AFC), previous surveys mostly focus on text. In this survey, we conceptualise a framework for AFC including subtasks unique to multimodal misinformation. Furthermore, we discuss related terms used in different communities and map them to our framework. We focus on four modalities prevalent in real-world fact-checking: text, image, audio, and video. We survey benchmarks and models, and discuss limitations and promising directions for future research

Submission Number: 1708