MemeBlip2: A Novel Light Weight Multimodal System to Detect Harmful Memes

Ran Tong, Jiaqi Liu, Aowei Shen, Shuzheng Li, Changlin Yang, Lisha Xu

Submission Type: Archive

Keywords: Large Language Models (LLMs), Multimodal System, Harmful Content, Artificial Intelligence

TL;DR: It is a multimodal model which can tell whether a Meme contains harmful content or not.

Abstract: Memes combine images with brief text to convey humour, opinions, and social commentary, but they can also spread harmful content such as hate speech. We present MemeBLIP2, a lightweight multimodal system that detects harmful memes by jointly modelling visual and textual signals. The method extends prior work by incoporating BLIP‑2 vision–language encoders. Our model is evaluated on the PrideMM dataset, where it reaches 77.5% accuracy, 81.8% AUROC, and a 79.0% macro F1 score. The results demonstrate that MemeBLIP2 captures subtle cross‑modal cues, including irony and culturally specific references, and it surpasses existing multimodal baselines on the benchmark.

Submission Number: 2