Zero-Shot Aspect-Based Scientific Document Summarization using Self-Supervised Pre-training

Anonymous

Paper Link: https://openreview.net/forum?id=kY86snNIzd9

Paper Type: Long paper (up to eight pages of content + unlimited references and appendices)

Abstract: We study the zero-shot setting for the aspect-based scientific document summarization task. Summarizing scientific documents with respect to an aspect can remarkably improve document assistance systems and readers experience. However, existing large-scale datasets contain a limited variety of aspects, causing summarization models to over-fit to a small set of aspects. We establish baseline results in zero-shot performance (over unseen aspects and the presence of domain shift), paraphrasing, leave-one-out, and limited supervised samples experimental setups. We propose a self-supervised pre-training approach to enhance the zero-shot performance. Experimental results on the FacetSum and PubMed aspect-based datasets show promising performance when the model is pre-trained using unlabelled in-domain data.