Developing A Novel Bidirectional Sparse Graph Attention Adaptor for Evidence-Based Fact-Checking

Anonymous

Abstract: Evidence-based Fact-checking aims to verify or debunk a claim with evidence given and has benefited from Large-Language-Model (LLM) advancements in text understanding. However, autoregressive LLMs suffer from their unidirectional nature, known as ``Reversal Curse'', causing their performance to be unsatisfactory. Therefore, in this paper, we propose to utilize bidirectional attention as an external adapter for two-way information aggregation. Further, we leverage hierarchical sparse graphs to reduce the noise impact of attention and an efficient feature-compression mechanism to reduce the number of adaptor parameters. Experimental results on both English and Chinese datasets demonstrate the significant improvements achieved by our proposed approach and its state-of-the-art performance in the Evidence-based Fact-checking task. The code will be available on GitHub.

Paper Type: long

Research Area: Information Extraction

Contribution Types: Model analysis & interpretability, NLP engineering experiment, Approaches to low-resource settings, Approaches low compute settings-efficiency, Publicly available software and/or pre-trained models, Data analysis, Theory

Languages Studied: English, Chinese