Enhancing Textbook Question Answering with Knowledge Graph-Augmented Large Language Models

Mengliang He, Aimin Zhou, Xiaoming Shi

Keywords: Textbook Question Answering, Large language model, knowledge graphs, Retrieve-and-Generate

Verify Author List: I have double-checked the author list and understand that additions and removals will not be allowed after the submission deadline.

TL;DR: The textbook text is converted into Knowledge Graph using the large language model, and the generated KG is then used to enhance the performance of the large language model on the TQA task.

Abstract: Previous works on Textbook Question Answering suffer from limited performance due to the small-scale neural network based backbone. To alleviate the issue, we propose to utilize LLMs as the backbone of TQA tasks. To this end, we utilize two methods, the raw-context based prompting method and the knowledge graph based prompting method. Specifically, we introduce the Textbook Question Answering-Knowledge Graph (TQA-KG) method, which first converts textbook content into structural knowledge graphs and then combining knowledge graph into LLM prompting, thereby enhancing the model's reasoning capabilities and answer accuracy. Extensive experiments conducted on the CK12-QA dataset illustrate the effectiveness of the method, achieving an improvement of 5.67% in accuracy compared to current state-of-the-art methods on average.

Primary Area: Applications (bioinformatics, biomedical informatics, climate science, collaborative filtering, computer vision, healthcare, human activity recognition, information retrieval, natural language processing, social networks, etc.)

Paper Checklist Guidelines: I certify that all co-authors of this work have read and commit to adhering to the guidelines in Call for Papers.

Student Author: Yes

Submission Number: 190