MMDFND: Multi-modal Multi-Domain Fake News Detection

Yu Tong, Weihai Lu, Zhe Zhao, Song Lai, Tong Shi

Abstract: Recently, automatic multi-domain fake news detection has attracted widespread attention. Many methods achieve domain adaptation by modeling domain category gate networks and domain-invariant features. However, existing multi-domain fake news detection faces three main challenges: (1) Inter-domain modal semantic deviation, where similar texts and images carry different meanings across various domains. (2) Inter-domain modal dependency deviation, where the dependence on different modalities varies across domains. (3) Inter-domain knowledge dependency deviation, where the reliance on cross-domain knowledge and domain-specific knowledge differs across domains. To address these issues, we propose a Multi-modal Multi-Domain Fake News Detection Model (MMDFND). MMDFND incorporates domain embeddings and attention mechanisms into a progressive hierarchical extraction network to achieve domain-adaptive domain-related knowledge extraction. Furthermore, MMDFND utilizes Stepwise Pivot Transformer networks and adaptive instance normalization to effectively utilize information from different modalities and domains. We validate the effectiveness of MMDFND through comprehensive comparative experiments on two real-world datasets and conduct ablation experiments to verify the effectiveness of each module, achieving state-of-the-art results on both datasets. The source code is available at https://github.com/yutchina/MMDFND.

Primary Subject Area: [Content] Multimodal Fusion

Secondary Subject Area: [Content] Multimodal Fusion, [Engagement] Emotional and Social Signals

Relevance To Conference: Recently, automated multi-domain fake news detection has garnered significant attention, and several multi-domain approaches have been proposed. These multi-domain fake news detection methods are all unimodal, unable to utilize image information, yet the real news data on social media is predominantly multimodal. Therefore, a multimodal, multi-domain fake news detection method is needed. Additionally, existing multi-domain fake news detection faces three main challenges: (1) inter-domain modal semantic deviation; (2) inter-domain modal dependency deviation; (3) inter-domain knowledge dependency deviation. Consequently, we propose a multimodal multi-domain fake news detection model (MMDFND), the first model to jointly model both multimodality and multidomain in the fake news detection scenario. This model can leverage multimodal data to perform multi-domain fake news detection, alleviating the aforementioned three issues. We validate the effectiveness of MMDFND through comprehensive comparative experiments on two real-world datasets and conduct ablation experiments to verify the effectiveness of each module, achieving state-of-the-art results on both datasets.

Submission Number: 3360