Dis2Dis: Explaining Ambiguity in Fact-Checking

ACL ARR 2024 August Submission220 Authors

Abstract:

Ambiguity is a linguistic tool for encoding information efficiently, yet it also causes misunderstandings and disagreements.
It is particularly relevant to the domain of misinformation, as fact-checking ambiguous claims is difficult even for experts.
In this paper we argue that instead of predicting a veracity label for which there is genuine disagreement, it would be more beneficial to explain the ambiguity.
Thus, this work introduces claim disambiguation, a novel constrained generation task, for explaining ambiguous claims in fact-checking. This involves editing them to spell out an interpretation that can then be unequivocally supported by the given evidence.
We collect a dataset of 1501 such claim revisions and conduct experiments with sequence-to-sequence models.
The performance is compared to a simple copy baseline and a Large Language Model baseline.
The best results are achieved by employing Minimum Bayes Decoding,
with a BertScore F1 of 92.22. According to human evaluation, the model successfully disambiguates the claims 72\% of the time.

Paper Type: Long

Research Area: NLP Applications

Research Area Keywords: fact checking

Contribution Types: Data resources

Languages Studied: English

Submission Number: 220