Improving Abstractive Summarization with Iterative Representation

Jinpeng Li, Chuang Zhang, Xiaojun Chen, Yanan Cao, Ruipeng Jia

Abstract: In the neural abstractive summarization field, comprehensive document representation and summary embellishment are two major challenges. To tackle the above problems, we propose an Iterative Abstractive Summarization (IAS) model through iterating the document and summary representation. Specifically, (1) we design a selective gated strategy to constantly update the input representation in the encoder, which is consistent with the repeated updating of human memory information in human writing. (2) We design an iterative unit to revise the comprehensive representation iteratively for polishing the summary. Moreover, we utilize reinforcement learning to optimize our model for the non-differentiable metric ROUGE, which can alleviate the exposure bias during predicting words effectively. Experiments on the CNN/Daily Mail, Gigaword and DUC-2004 datasets show that the IAS model can generate high-quality summaries with varied length, and outperforms baseline methods significantly in terms of ROUGE and Human metrics.