Inverse Reinforcement Learning for Text Summarization

Yu Fu, Deyi Xiong, Yue Dong

Submission Type: Regular Long Paper

Submission Track: Summarization

Keywords: Abstractive Summarization, Inverse Reinforcement Learning, Reward Function Optimization

TL;DR: We introduce inverse reinforcement learning (IRL) for abstractive summarization, demonstrating improvements over MLE and RL baselines across diverse datasets and model sizes

Abstract: We introduce inverse reinforcement learning (IRL) as an effective paradigm for training abstractive summarization models, imitating human summarization behaviors. Our IRL model estimates the reward function using a suite of important sub-rewards for summarization and concurrently optimizes the policy network. Experimental results across datasets in different domains (CNN/DailyMail and WikiHow) and various model sizes (BART-base and BART-large) demonstrate the superiority of our proposed IRL model for summarization over MLE and RL baselines. The resulting summaries exhibit greater similarity to human-crafted gold references, outperforming MLE and RL baselines on metrics such as ROUGE, coverage, novelty, compression ratio, factuality, and human evaluations.

Submission Number: 1981