KnowMeme: A Knowledge-enriched Graph Neural Network Solution to Offensive Meme Detection

Lanyu Shang, Christina Youn, Yuheng Zha, Yang Zhang, Dong Wang

Abstract: This paper studies a critical problem of identifying offensive meme posts on online social media where images are superimposed with deliberately altered or crafted captions to deliver offensive information. Existing solutions often ignore the implicit relationship between visual and textual contents in the meme and their implied meanings, which are critical to accurately detect offensive memes. Two important challenges exist in solving the problem: i) how to effectively incorporate human commonsense knowledge to capture the symbolic or implied meaning of the meme contents that implicitly deliver offensive information? ii) How to accurately identify the cross-modal knowledge-based relations between entities in both the visual and textual content of the meme that jointly insinuate offensive messages? To address the above challenges, we develop KnowMeme, a knowledge-enriched graph neural network solution that leverages knowledge facts from human commonsense knowledge to effectively detect offensive meme posts on online social media. Evaluation results show that KnowMeme achieves significant performance gains compared to the state-of-the-art baseline methods in accurately detecting offensive memes.