From Information to Insight: Leveraging LLMs for Open Aspect-Based Educational Summarization
Abstract: This paper addresses the challenge of aspect-based summarization in education by introducing Reflective ASPect-based summarization (ReflectASP), a novel dataset that summarizes student reflections on STEM lectures. Despite the promising performance of large language models in general summarization, their application to nuanced aspect-based summaries remains under-explored. ReflectASP eases the exploration of open-aspect-based summarization (OABS), overcoming the limitations of current datasets and comes with ample human annotations. We benchmarked different types of zero-shot summarization methods and proposed two refinement methods to improve summaries, supported by both automatic and human manual evaluations. Additionally, we analyzed suggestions and revisions made during the refinement process, offering a fine-grained study of the editing strategies employed by these methods. We will make our models, dataset, and all human evaluation results available at urlannonymized_for_review.
Paper Type: Long
Research Area: NLP Applications
Research Area Keywords: educational applications, corpus creation, benchmarking, language resources, human evaluation; automatic evaluation, abstractive summarisation,
Contribution Types: Model analysis & interpretability, NLP engineering experiment, Approaches to low-resource settings, Data resources, Data analysis
Languages Studied: English
Submission Number: 1142
Loading