Summarizing Multiple Documents with Conversational Structure for Meta-Review Generation
Submission Type: Regular Long Paper
Submission Track: Summarization
Submission Track 2: Natural Language Generation
Keywords: Multi-document Summarization, Text Generation, Multi-task Learning, Meta-review Generation, Inter-document Relationships
TL;DR: Summarizing Multiple Documents with Conversational Structure for Meta-Review Generation
Abstract: We present PeerSum, a novel dataset for generating meta-reviews of scientific papers. The meta-reviews can be interpreted as abstractive summaries of reviews, multi-turn discussions and the paper abstract. These source documents have a rich inter-document relationship with an explicit hierarchical conversational structure, cross-references and (occasionally) conflicting information. To introduce the structural inductive bias into pre-trained language models, we introduce RAMMER (Relationship-aware Multi-task Meta-review Generator), a model that uses sparse attention based on the conversational structure and a multi-task training objective that predicts metadata features (e.g., review ratings). Our experimental results show that RAMMER outperforms other strong baseline models in terms of a suite of automatic evaluation metrics. Further analyses, however, reveal that RAMMER and other models struggle to handle conflicts in source documents, suggesting meta-review generation is a challenging task and a promising avenue for further research.
Submission Number: 2469
Loading