Lost in Transformation: Rediscovering LLM-Generated Campaigns in Social Media

Britta Grimme, Janina Pohl, Hendrik Winkelmann, Lucas Stampe, Christian Grimme

Abstract: This paper addresses new challenges of detecting campaigns in social media, which emerged with the rise of Large Language Models (LLMs). LLMs particularly challenge algorithms focused on the temporal analysis of topical clusters. Simple similarity measures can no longer capture and map campaigns that were previously broadly similar in content. Herein, we analyze whether the classification of messages over time can be profitably used to rediscover poorly detectable campaigns at the content level. Thus, we evaluate classical classifiers and a new method based on siamese neural networks. Our results show that campaigns can be detected despite the limited reliability of the classifiers as long as they are based on a large amount of simultaneously spread artificial content.

External IDs:doi:10.1007/978-3-031-47896-3_6