Multi-Document Summarization Using Selective Attention Span and Reinforcement Learning

Yash Kumar Atri, Vikram Goyal, Tanmoy Chakraborty

Abstract: ive text summarization systems using recently improved RNN-based sequence-to-sequence architecture have shown great promise for single-document summarization. However, such neural models fail to perpetuate the performance in the multi-document summarization setting owing to the long-range dependencies within the documents, overlapping/contradicting facts and extrinsic model hallucinations. These shortcomings augment the model to generate inconsistent, repetitive and non-factual summaries. In this work, we introduce REISA , a sequence-to-sequence model with a novel reinforced selective attention span that attends over the input and recalibrates the local attention weights to focus on important segments while generating output at each time step. REISA utilizes a reinforcement learning-based policy gradient algorithm to reward the model and formulate attention distributions over the encoder input. We further benchmark REISA on two widely-used multi-document summarization corpora – Multinews and CQASumm, and observe an improvement of $+2.91$ and $+6.64$ ROUGE-L scores, respectively. The qualitative analyses on semantic similarity by BERTScore, faithfulness by question-answer evaluation and human evaluation show significant improvement over the baseline-generated summaries.