Reinforcement Replaces Supervision: Query focused Summarization using Deep Reinforcement Learning
Submission Type: Regular Long Paper
Submission Track: Summarization
Submission Track 2: Natural Language Generation
Keywords: reinforcement learning for long text, query focused summarization, passage embedding, long form question answering
TL;DR: New RL algorithm for long text generation in Query-focused Summarization. Proved to be effective on benchmark datasets and new gold standard data. Also introduced a novel Passage Embedding model and released pre-training dataset.
Abstract: Query-focused Summarization (QfS) deals with systems that generate summaries from document(s) based on a query. Motivated by the insight that Reinforcement Learning (RL) provides a generalization to Supervised Learning (SL) for Natural Language Generation, and thereby performs better (empirically) than SL, we use an RL-based approach for this task of QfS. Additionally, we also resolve the conflict of employing RL in Transformers with Teacher Forcing. We develop multiple Policy Gradient networks, trained on various reward signals: ROUGE, BLEU, and Semantic Similarity, which lead to a $\mathit{10}$-point improvement over the State-of-the-Art approach on the ROUGE-L metric for a benchmark dataset (ELI5). We also show performance of our approach in zero-shot setting for another benchmark dataset (DebatePedia) -- our approach leads to results comparable to baselines, which were specifically trained on DebatePedia. To aid the RL training, we propose a better semantic similarity reward, enabled by a novel Passage Embedding scheme developed using Cluster Hypothesis. Lastly, we contribute a gold-standard test dataset to further research in QfS and Long-form Question Answering (LfQA).
Submission Number: 5329
Loading