Distilling Opinions at Scale: Incremental Opinion Summarization using XL-OPSUMM
Abstract: Opinion summarization in e-commerce encapsulates the collective views of numerous users about a product based on their reviews. Typically, a product on an e-commerce platform has thousands of reviews, each review comprising around 10-15 words. While Large Language Models (LLMs) have shown proficiency in summarization tasks, they struggle to handle such a large volume of reviews due to context limitations. To address this, we propose a scalable framework called XL-OPSUMM, that generates summaries incrementally with the help of an Aspect Dictionary (Refer to Section 3). However, the existing test set, AMASUM, has only 560 reviews per product on average. Due to the lack of a test set with thousands of reviews, we created a new test set called XL-FLIPKART by gathering data from the Flipkart website and generating summaries using GPT-4. Through various automatic evaluations and extensive analysis, we evaluated the framework's efficiency on two datasets, AMASUM and Xl-Flipkart. Experimental results show that our framework, XL-OPSUMM, powered by Llama-3-8B-8k, achieves an average ROUGE-1 F1 gain of 4.38% and a ROUGE-L F1 gain of 3.70% over the next best-performing model.
Paper Type: Long
Research Area: Summarization
Research Area Keywords: opinion summarization,text summarization,incremental summarization,LLM
Contribution Types: Model analysis & interpretability, NLP engineering experiment, Approaches low compute settings-efficiency
Languages Studied: English
Submission Number: 4838
Loading