Detecting Misinformation by Uncovering Commonsense Conflicts With LLM Workflows

Bing Wang, Ximing Li, Changchun Li, Bingrui Zhao, Renchu Guan, Lin Yuanbo Wu, Jungong Han

Abstract: The advancement of Internet technology has spurred a rise in the dissemination of misinformation, which has had profoundly negative impacts across a wide array of fields. To address this issue, the field of Misinformation Detection (MD), which focuses on the automated identification of online misinformation, has gained significant traction among researchers. In our study, we introduce an innovative plug-and-play augmentation technique for MD, termed DEtecting Misinformation by Uncovering Commonsense Conflict (Demuc). Our approach is grounded in previous psychological research that suggests that fake content often contains commonsense. Accordingly, we develop commonsense expressions for articles to highlight potential conflicts between the inferred commonsense triplets and the established ones derived from reliable commonsense reasoning tools. According to the used tools, we induce two variants Demuc-klm using the knowledge language model COMET and Demuc-llm using the large language models. These generated expressions are then applied as augmentations to each article, enabling any MD method to be trained on these augmented datasets. Additionally, we have manually compiled a new dataset CoMis, which consists exclusively of fake articles characterized by commonsense conflicts. By integrating Demuc with various existing MD frameworks and evaluating them on four public benchmark datasets and CoMis, our empirical findings show that both Demuc-klm and Demuc-llm consistently and significantly outperform current MD baselines, while also generating precise commonsense expressions.

External IDs:dblp:journals/tkde/WangLLZGWH26