Vaccine Misinformation Detection in X using Cooperative Multimodal Framework
Abstract: Identifying social media posts that spread vaccine misinformation can inform emerging public health risks and aid in designing effective communication interventions. Existing studies, while promising, often rely on single user posts, potentially leading to flawed conclusions. This highlights the necessity to model users' historical posts for a comprehensive understanding of their stance towards vaccines. However, users' historical posts may contain a diverse range of content that adds noise and leads to low performance. To address this gap, in this study, we present VaxMine, a cooperative multi-agent reinforcement learning method that automatically selects relevant textual and visual content from a user's posts, reducing noise. To evaluate the performance of the proposed method, we create and release a new dataset of 2,072 users with historical posts due to the unavailability of publicly available datasets. The experimental results show that our approach outperforms state-of-the-art methods with an F1-Score of 0.94 (an absolute increase of 13\%), demonstrating that extracting relevant content from users' historical posts and understanding both modalities are essential to identify anti-vaccine users on social media. We further analyze the robustness and generalizability of VaxMine, showing that extracting relevant textual and visual content from a user's posts improves performance. We conclude with a discussion on the practical implications of our study by explaining how computational methods used in surveillance can benefit from our work, with flow-on effects on the design of health communication interventions to counter vaccine misinformation on social media. We suggest that releasing a robustly annotated dataset will support further advances and benchmarking of methods
Primary Subject Area: [Experience] Multimedia Applications
Secondary Subject Area: [Content] Multimodal Fusion, [Generation] Social Aspects of Generative AI
Relevance To Conference: This study presents VaxMine, a method to identify relevant textual and visual content from users' historical social media posts to combat vaccine misinformation. Its innovative approach significantly improves performance, making it relevant to multimedia/multimodal processing. While similar studies exist, VaxMine's comprehensive approach and release of a new dataset demonstrate its relevance and contribution to the community. Its practical implications for designing effective communication interventions against vaccine misinformation highlight its significance. Given its advancements and relevance, VaxMine is highly suitable for presentation at the ACM MM conference, benefiting the multimedia research community.
Submission Number: 3905
Loading