mRedditSum: A Multimodal Abstractive Summarization Dataset of Reddit Threads with Images
Submission Type: Regular Long Paper
Submission Track: Resources and Evaluation
Submission Track 2: Summarization
Keywords: Multimodal Abstractive Summarization Dataset
TL;DR: We present mRedditSum, the first multimodal discussion summarization dataset consisting of 3,033 annotated Reddit threads with images.
Abstract: The growing number of multimodal online discussions necessitates automatic summarization to save time and reduce content overload.
However, existing summarization datasets are not suitable for this purpose, as they either do not cover discussions, multiple modalities, or both. To this end, we present mRedditSum, the first multimodal discussion summarization dataset. It consists of 3,033 discussion threads where a post solicits advice regarding an issue described with an image and text, and respective comments express diverse opinions. We annotate each thread with a human-written summary that captures both the essential information from the text, as well as the details available only in the image. Experiments show that popular summarization models---GPT-3.5, BART, and T5---consistently improve in performance when visual information is incorporated. We also introduce a novel method, cluster-based multi-stage summarization, that outperforms existing baselines and serves as a competitive baseline for future work.
Submission Number: 2931
Loading