FRSUM: Towards Faithful Abstractive Summarization via Enhancing Factual Robustness

Anonymous

Abstract: Though current Seq2Seq summarization models are capable of generating fluent and grammatical summaries, they are still suffering from the unfaithful generation problem.In this paper, we study the faithfulness of existing systems from a new perspective of factual robustness which is the ability to correctly generate factual information over adversarial unfaithful information.We first define the measurement of a model's factual robustness as its success rate to defend against adversarial attacks when generating factual information. The factual robustness analysis on a wide range of current systems shows its good consistency with human judgments on faithfulness.Inspired by these findings, we propose to improve a model's faithfulness by enhancing its factual robustness.Specifically, we propose a novel training strategy, namely FRSUM, which teaches the model to defend against both explicit adversarial samples and implicit factual adversarial perturbations.Extensive automatic and human evaluation results show that FRSUM consistently improves the faithfulness of various Seq2Seq models, such as T5, BART and PEGASUS, and reduces up to 41\% target errors in summaries.