Aligning Path-based Link Prediction with Human Understanding of Valid Reasoning

TMLR Paper7778 Authors

Abstract:

Path-based link prediction methods reconstruct missing links between two vertices of a knowledge graph. They reconstruct a missing link by finding a path through the knowledge graph connecting both vertices. The path is the reasoning of the link prediction method. However, path-based link prediction methods are vulnerable to \emph{Clever Hans} biases. They learn invalid reasoning patterns if these patterns are dominant and generalize well to the training and validation set. As a result, performance drops when evaluated on the real-world distribution. The validity of reasoning is determined by the semantic concept underlying the missing link, which is mostly accessible through human knowledge. The paper's approach makes human understanding of valid reasoning accessible while learning to predict missing links.
This paper proposes the path-based link prediction method \emph{LiEr}. \emph{LiEr} learns valid reasoning within the knowledge graph domain from preference-based human feedback.
The paper demonstrates that \emph{LiEr}'s prediction capability is on par with other state-of-the-art link prediction methods while more aligned with human understanding of valid reasoning on various benchmark reasoning tasks. In addition, a novel benchmark knowledge graph with a \emph{Clever Hans} bias is introduced to evaluate the alignment of link prediction methods with human understanding of valid reasoning. The paper contributes by proposing the first human-in-the-loop link prediction method, capable of aligning its reasoning with the human understanding of valid reasoning.

Submission Type: Long submission (more than 12 pages of main content)

Changes Since Last Submission: N/A

Assigned Action Editor: ~Lee_Zamparo1

Submission Number: 7778