Knowledge Graph as Tokens: Knowledge Base Construction Using Language Model with Graph Neural Network and Soft Prompting

ACL ARR 2025 February Submission5732 Authors

Abstract: A knowledge graph represents real-world concepts as interconnected nodes, with widely recognized examples like WikiData, DBPedia, and YAGO. However, these graphs remain incomplete, and knowledge evolves over time. Constructing knowledge graphs involves extracting information from various sources, including text, images, and videos. Language models store knowledge in their parameters, and the ISWC has introduced a competition, LM-KBC, to extract this knowledge for enhancing knowledge graphs. Previous research has focused on hard prompting and few-shot methods, leaving an unexplored opportunity for soft prompts. This study proposes Knowledge Graph as Tokens (KGAT), inspired by Frozen and Seq2Path, using a graph neural network (GNN) to incorporate graph context as a soft prompt in language models. Evaluations on ISWC datasets (2022–2024) with Llama 3.1 8B show that KGAT outperforms the baseline, albeit with a minor improvement.

Paper Type: Short

Research Area: Information Extraction

Research Area Keywords: knowledge base construction, zero/few-shot extractio

Contribution Types: Model analysis & interpretability, NLP engineering experiment

Languages Studied: english

Submission Number: 5732