Go for a Walk and Arrive at the Answer: Reasoning Over Paths in Knowledge Bases using Reinforcement Learning

Rajarshi Das, Shehzaad Dhuliawala, Manzil Zaheer, Luke Vilnis, Ishan Durugkar, Akshay Krishnamurthy, Alex Smola, Andrew McCallum

Abstract: Knowledge bases (KB), both automatically and manually constructed, are often incomplete --- many valid facts can be inferred from the KB by synthesizing existing information. A popular approach to KB completion is to infer new relations by combinatory reasoning over the information found along other paths connecting a pair of entities. Given the enormous size of KBs and the exponential number of paths, previous path-based models have considered only the problem of predicting a missing relation given two entities, or evaluating the truth of a proposed triple. Additionally, these methods have traditionally used random paths between fixed entity pairs or more recently learned to pick paths between them. We propose a new algorithm, MINERVA, which addresses the much more difficult and practical task of answering questions where the relation is known, but only one entity. Since random walks are impractical in a setting with unknown destination and combinatorially many paths from a start node, we present a neural reinforcement learning approach which learns how to navigate the graph conditioned on the input query to find predictive paths. On a comprehensive evaluation on seven knowledge base datasets, we found MINERVA to be competitive with many current state-of-the-art methods.

TL;DR: We present a RL agent MINERVA which learns to walk on a knowledge graph and answer queries

Keywords: Knowledge Graphs, Reinforcement Learning, Query Answering

Code: [ shehzaadzd/MINERVA](https://github.com/shehzaadzd/MINERVA) + [ 6 community implementations](https://paperswithcode.com/paper/?openreview=Syg-YfWCW)

Data: [FB15k](https://paperswithcode.com/dataset/fb15k), [NELL](https://paperswithcode.com/dataset/nell), [NELL-995](https://paperswithcode.com/dataset/nell-995), [WN18](https://paperswithcode.com/dataset/wn18), [WN18RR](https://paperswithcode.com/dataset/wn18rr), [WikiMovies](https://paperswithcode.com/dataset/wikimovies)

Community Implementations: [ 5 code implementations](https://www.catalyzex.com/paper/arxiv:1711.05851/code)