Decoding synthetic news: an interpretable multimodal framework for the classification of news articles in a novel news corpus

Michael Schlee, Gillian Kant, Christoph Ehrling, Benjamin Säfken, Thomas Kneib

Abstract: Recent advancements in Artificial Intelligence (AI), notably the development of Large Language Models (LLMs) and text-to-image diffusion models, have facilitated the creation of realistic textual content and images. Specifically, platforms like ChatGPT and Midjourney have simplified the creation of high-quality text and visuals with minimal expertise and cost. The increasing sophistication of Generative AI presents challenges in ensuring the integrity of news, media, and information quality, making it increasingly difficult to distinguish between real and artificially generated textual and visual content. Our work addressed this problem in two ways. First, by means of ChatGPT and Midjourney, we created a comprehensive novel multimodal news corpus named SyN24News based on the N24News corpus, on which we evaluated our model. Second, we developed a novel explainable synthetic news detector for discriminating between real and synthetic news articles. We leveraged a Neural Additive Model (NAM)-like network structure that ensures effect separation by handling input data in separate subnetworks. Complex structures and patterns are extracted by deep features from unstructured data, i.e., images and texts, using fine-tuned VGG and DistilBERT subnetworks. We ensured further explainability by individually processing carefully chosen handcrafted text and image features in simple Multilayer Perceptrons (MLPs), allowing for graphical interpretation of corresponding structured effects. Our findings indicate that textual information are the main drivers in the decision-making finding process. Structured textual effects, particularly Flesch-Kincaid reading ease and sentiment, have a much higher influence on the classification outcome than visual features such as dissimilarity and homogeneity.