EviGenerate: Generative Evidence in Automated Fact-Checking

Uku Kangur, Krish Agrawal, Roshni Chakraborty, Rajesh Sharma

Keywords: Evidence generation, Knowledge extraction, Automatic fact-checking

TL;DR: Proposed approach for using LLM generated evidence for fact-checking

Abstract: In the era of widespread misinformation, the imperative tasks of fact verification and correction have become essential, especially in the realm of online social media. Traditional manual fact-checking, while crucial, is time-consuming, emphasizing the need for innovative approaches. This research introduces an automated fact-checking system leveraging sophisticated language models for evidence generation to dynamically adapt to the evolving information landscape. The proposed system \textit{EviGenerate} employs a novel evidence generation pipeline, integrating strategies such as named entity hints, question formulation, relation explanation, cross-examination, and a truthful critic. Utilizing a modified FEVER - a widely used automatic fact-checking dataset, the approach achieves a F$1$ score of $0.912$ for claim verification based on DeBERTa. Our best claim correction result based on T$5$-$3$B gives a SARI Keep score of $0.721$. The contribution of this work lies in its evidence generation approach and prompting strategies, fostering accuracy and adaptability in automated fact-checking systems.

Submission Number: 18