Multi-hop Knowledge Graph Reasoning Based on Hyperbolic Knowledge Graph Embedding and Reinforcement Learning

Xingchen Zhou, Peng Wang, Qiqing Luo, Zhe Pan

Abstract: Some unseen facts in knowledge graphs (KGs) can be complemented by multi-hop knowledge graph reasoning. The process of multi-hop reasoning can be presented as a serialized decision problem, and then be solved with reinforcement learning (RL). However, existing RL-based reasoning methods cannot deal with the hierarchical information in KGs effectively. To solve this issue, we propose a novel RL-based multi-hop KG reasoning model Path Additional Action-space Ranking (PAAR). Our model first proposes a hyperbolic knowledge graph embedding (KGE) model which can effectively capture hierarchical information in KGs. To introduce hierarchical information into RL, the hyperbolic KGE vector is subsequently added to the state space and helps to expand the action space. In order to alleviate the reward sparsity problem in RL, our model utilizes the score function of hyperbolic KGE model as a soft reward. Finally, the metric of hyperbolic space is added to the training of RL as a penalty strategy to constrain the sufficiency of multi-hop reasoning paths. Experimental results on two real-world datasets show that our proposed model not only provides effective answers but also offers sufficient paths.