Generating and Evaluating Long Story Summaries with Knowledge Graphs

Anonymous

Abstract: Summarizing long stories is a challenging task due to their narrative complexity and the context length limits of language models. We propose a method that integrates knowledge graph retrieval with the summarization process to provide global context. We construct a knowledge graph containing entity descriptions and relations from the entire story, then retrieve relevant information from it to aid summary generation. Additionally, we propose a novel metric, KGScore, which evaluates summaries by comparing the similarity of knowledge graphs extracted from generated and reference summaries. Experimental results demonstrate that our knowledge graph retrieval method outperforms the baselines in terms of our KGScore metric and that KGScore is a reliable measure of factual consistency.

Paper Type: long

Research Area: Summarization

Contribution Types: NLP engineering experiment

Languages Studied: English