The evolution of political memes: Detecting and characterizing internet memes with multi-modal deep learning

David M. Beskow, Sumeet Kumar, Kathleen M. Carley

Abstract: Highlights • Develops multimodal classification of memes using image, text, and face encoding. • Uses graph learning to create the evolutionary tree for memes. • Applies this to a large online conversation around a political event (2018 US mid-term elections). • Outlines technique for meme OCR pre-processing. Abstract Combining humor with cultural relevance, Internet memes have become an ubiquitous artifact of the digital age. As Richard Dawkins described in his book The Selfish Gene, memes behave like cultural genes as they propagate and evolve through a complex process of ‘mutation’ and ‘inheritance’. On the Internet, these memes activate inherent biases in a culture or society, sometimes replacing logical approaches to persuasive argument. Despite their fair share of success on the Internet, their detection and evolution have remained understudied. In this research, we propose and evaluate Meme-Hunter, a multi-modal deep learning model to classify images on the Internet as memes vs non-memes, and compare this to uni-modal approaches. We then use image similarity, meme specific optical character recognition, and face detection to find and study families of memes shared on Twitter in the 2018 US Mid-term elections. By mapping meme mutation in an electoral process, this study confirms Richard Dawkins’ concept of meme evolution.