Efficient Aspect-Based Summarization with Small Language Models: A Use-Case on Climate Change Reports
Abstract: Large Language Models (LLMs) have revolutionized many fields of Natural Language Processing (NLP), including summarization. These systems, however, consist of billions of parameters and, as such, they have the crucial shortcoming of being energy-intensive. In this work, we present a thorough evaluation of very recent, small-sized LLMs (SLMs) on the task of Aspect-Based Summarization of Climate Change Reports. In doing so, we show that modern SLMs are sufficiently good for the task and can bring value in assisting with summarization for policymakers while being more efficient than their bigger counterparts without significant performance deterioration. We also show how energy consumption among SLMs themselves does not correlate with better performance, further proving the point that smaller models can be effectively used for the task. Finally, we release the new dataset that we collected to perform our experiments, from which we hope research in NLP for climate change and research in efficient Aspect-Based Summarization with LLMs can develop further.
Paper Type: Long
Research Area: Summarization
Research Area Keywords: Summarization, NLP Applications, Efficient/Low-Resource Methods for NLP
Contribution Types: NLP engineering experiment, Approaches low compute settings-efficiency, Publicly available software and/or pre-trained models, Data resources, Data analysis
Languages Studied: English
Submission Number: 1275
Loading