SueNes: A Weakly Supervised Approach to Evaluating Single-Document Summarization via Negative Sampling![Download PDF](/images/pdf_icon_blue.svg)
Paper Link: https://openreview.net/forum?id=UiRi-PHvNlP
Paper Type: Short paper (up to four pages of content + unlimited references and appendices)
Abstract: Canonical automatic summary evaluation metrics, such as ROUGE, focus on lexical similarity which cannot well capture semantics nor linguistic quality and require a reference summary which is costly to obtain. Recently, there have been a growing number of efforts
to alleviate either or both of the two drawbacks. In this paper, we present a proof-of-concept study to a weakly supervised summary evaluation approach without the presence of reference summaries. Massive data in existing summarization datasets are transformed for training by pairing documents with corrupted reference summaries. In cross-domain tests, our strategy outperforms baselines with promising improvements, and show a great advantage in gauging linguistic qualities over all metrics.
Presentation Mode: This paper will be presented in person in Seattle
Copyright Consent Signature (type Name Or NA If Not Transferrable): Forrest Sheng Bao
Copyright Consent Name And Address: Iowa State University, Ames, IA, USA
0 Replies
Loading