FIZZ: Factual Inconsistency Detection by Zoom-in Summary and Zoom-out Document
Abstract: Through the advent of pre-trained language models, there have been notable advancements in abstractive summarization systems.
Simultaneously, a considerable number of novel methods for evaluating factual consistency in abstractive summarization systems has been developed. But these evaluation approaches incorporate substantial limitations, especially on refinement and interpretability.
In this work, we propose highly effective and interpretable factual inconsistency detection method FIZZ (Factual Inconsistency Detection by Zoom-in Summary and Zoom-out Document) for abstractive summarization systems that is based on fine-grained atomic facts decomposition.
Moreover, we align atomic facts decomposed from the summary with the source document through adaptive granularity expansion.
These \atomic facts represent a more fine-grained unit of information, facilitating detailed understanding and interpretability of the summary's factual inconsistency.
Experimental results demonstrate that our proposed factual consistency checking system significantly outperforms existing systems.
We release the code at \url{https://anonymous.4open.science/status/FAIRY-B5F7}.
Paper Type: Long
Research Area: Summarization
Research Area Keywords: Summarization
Contribution Types: NLP engineering experiment, Publicly available software and/or pre-trained models
Languages Studied: English
Submission Number: 712
Loading