Relation Prediction as an Auxiliary Training Objective for Improving Multi-Relational Graph Representations

Yihong Chen, Pasquale Minervini, Sebastian Riedel, Pontus Stenetorp

Keywords: knowledge base completion, relation prediction, multi-relational graph

TL;DR: A simple auxiliary training objective to improve multi-relational graph representation learning

Abstract: Learning good representations of multi-relational graphs is essential to downstream applications like knowledge base completion (KBC). In this paper, we propose a new self-supervised objective for mult-relational graph representation learning, via simply incorporating relation prediction into the commonly used 1vsAll objective. We analyse how this new objective impacts multi-relational learning in KBC. Experiments on a variety of datasets and models show that relation prediction can significantly improve entity ranking, the most widely used evaluation task for KBC, yielding a $6.1\%$ increase in MRR and $9.9\%$ increase in Hits@1 on FB15k-237 as well as a $3.1 \%$ increase in MRR and $3.4\%$ in Hits@1 on Aristo-v4. Moreover, we observe that the proposed objective is especially effective on highly multi-relational datasets i.e. datasets with a large number of predicates, and generates better representations when larger embedding sizes are used.

Subject Areas: Knowledge Representation, Semantic Web and Search, Machine Learning, Relational AI

Archival Status: Archival