CrowdChecked: Detecting Previously Fact-Checked Claims in Social Media

Anonymous

Abstract: While there has been substantial progress in developing systems to automate the process of fact-checking, such systems still lack credibility in the eyes of the users, and thus human fact-checkers remain the main drivers of the process. In view of that, recently, a middle-ground approach has emerged: to do automatic fact-checking by verifying whether the input claim has been previously fact-checked by professional fact-checkers, and to return back an article that explains the verdict on the claim. This is a sensible approach as people trust manual fact-checking, and as many claims are repeated multiple times online.Yet, a major issue when building such kinds of systems is the small number of known input--verified claim pairs available for training. Here, we aim to bridge this gap by making use of crowd fact-checking, i.e., mining claims in social media for which users have responded with a link to a fact-checking article. In particular, we mine a large-scale collection of 330,000 tweets paired with a corresponding fact-checking article. We further propose a new model to learn from this noisy data based on modified self-adaptive training, in a distant supervision scenario. Our experiments on a standard test set show improvements over the state of the art by two points absolute.

Paper Type: long