Transductive Learning for Abstractive News Summarization

Anonymous

Abstract: Pre-trained and fine-tuned news summarizers are expected to generalize to news articles unseen in the fine-tuning (training) phase. However, these articles often contain specifics, such as events and people, a summarizer could not learn about in training. This applies to scenarios such as when a news publisher trains a summarizer on dated news and wants to summarize incoming recent news. In this work, we explore the first application of transductive learning to summarization where we further fine-tune models on test set’s input. Specifically, we construct references for learning from article salient sentences and condition on the randomly masked articles. We show that this approach is also beneficial in the fine-tuning phase when extractive references are jointly predicted with abstractive ones in the training set. In general, extractive references are inexpensive to produce as they are automatically created without human effort. We show that our approach yields state-of-the-art results on CNN/DM and NYT datasets, for instance, more than 1 ROUGE-L points improvement on the former. Moreover, we show the benefits of transduction from dated to more recent CNN news. Finally, through human and automatic evaluation, we demonstrate improvements in summary abstractiveness and coherence.