Multi-Task Network Guided Multimodal Fusion for Fake News Detection

Jinke Ma, Liyuan Zhang, Yong Liu, Wei Zhang

Keywords: Multi-task Networks, Cross-modal Correlation, Feature Refinement

Verify Author List: I have double-checked the author list and understand that additions and removals will not be allowed after the submission deadline.

TL;DR: Introduction of PLE and CLIP for multimodal rumour detection.

Abstract: Fake news detection has become a hot research topic in the multimodal domain. Existing multimodal fake news detection research utilizes a series of feature fusion networks to gather useful information from different modalities of news posts. However, how to form effective cross-modal features? And how cross-modal correlations impact decision-making? These remain open questions. This paper introduces MMFND, a multi-task guided multimodal fusion framework for fake news detection , which introduces multi-task modules for feature refinement and fusion. Pairwise CLIP encoders are used to extract modality-aligned deep representations, enabling accurate measurement of cross-modal correlations. Enhancing feature fusion by weighting multimodal features with normalised cross-modal correlations. Extensive experiments on typical fake news datasets demonstrate that MMFND outperforms state-of-the-art approaches.

Url Link To Your Supplementary Code: https://github.com/diga7654321/MMFND

Primary Area: Applications (bioinformatics, biomedical informatics, climate science, collaborative filtering, computer vision, healthcare, human activity recognition, information retrieval, natural language processing, social networks, etc.)

Paper Checklist Guidelines: I certify that all co-authors of this work have read and commit to adhering to the guidelines in Call for Papers.

Student Author: Yes

Submission Number: 248