A Comparative Analysis of Neurosymbolic Methods for Link Prediction

Guillaume Delplanque, Luisa Werner, Nabil Layaïda, Pierre Geneves

Keywords: neuro-symbolic, link prediction, knowledge graph, survey, benchmarking

TL;DR: This is a comparative analysis of several neurosymbolic methods for link prediction.

Abstract:

Link prediction on knowledge graphs is relevant to various applications, such as recommendation systems, question answering, and entity search.
This task has been approached from different perspectives: symbolic methods leverage rule-based reasoning but struggle with scalability and noise, while knowledge graph embeddings (KGE) represent entities and relations in a continuous space, enabling scalability but often neglecting logical constraints from ontologies.
Recently, neurosymbolic approaches have emerged to bridge this gap by integrating embedding-based learning with symbolic reasoning.
This paper provides a structured review of state-of-the-art neurosymbolic methods for link prediction.
Beyond a qualitative analysis, a key contribution of this work is a comprehensive experimental benchmarking, where we systematically compare these methods on the same datasets using the same metrics.
This unified experimental setup allows for a fair assessment of their strengths and limitations, bringing elements of answers to following key questions: How accurate are these methods? How scalable are they?
How beneficial are they for different levels of provided knowledge and to which extent are they robust to incorrect knowledge?

Track: Knowledge Graphs, Ontologies and Neurosymbolic AI

Paper Type: Long Paper

Resubmission: Yes

Changes List:

We thank the Phase 1 reviewers for their valuable and constructive feedback.
We have made major changes and improvements to the paper. We summarize them below.

We improved our taxonomy and changed the neurosymbolic subcategories to "symbolic methods with learning", "subsymbolic methods with logical constraints" and "bidirectional neurosymbolic integration" and enhanced the completeness of the taxonomy by adding more approaches.
Reviewer 4RSJ pointed out the absence of symbolic systems like ANYBURL. We added AnyBURL (2019) to our survey in the category of symbolic systems and the more recent AnyBurl with reinforcement learning (2024) in the neurosymbolic category "Symbolic systems with learning" and included this method in our experimental evaluation.
This reviewer also wanted a discussion about faithful embeddings such as Box2EL. We added Box2EL, a reference to the rule injection possibility of the original BoxE approach and Simple+ to category "subsymbolic methods with logical constraints".
Futhermore, we added NeuralLP, pLogicNet, ReasonKGE, TransOWL, JOIE, OntoKGE, RUGE, CompGCN, SEAL and RGCN.

The presentation, motivation, soundness and results with our metric Contradicts@k have been criticized by some reviewers. We decided that this metric requires some additional work regarding evaluation and presentation. For this reason we decided to remove it from this paper and put a much stronger focus on the experimental benchmarking.

In this context, we strengthened the experimental evaluation with the following additions:
1) We conducted the experiments on more powerful hardware and more generous timeouts which leads to more insightful results on the dataset FB15k237.
2) We evaluated the methods on another additional benchmark WN18RR.
3) We also included more methods in our experimental evaluation, namely CompGCN and Anyburl+RL.
4) We conducted additional experiments regarding the robustness of the methods to incorrect rules and to the general amount of provided knowledge and enhanced the discussion of these experiments through some additional graphics and analysis.

Finally, we added useful clarifications thanks to reviewer comments. In particular, reviewer 1Wis remarked that no distinction was made between transductive and inductive methods. Since most methods only work for a transductive setting (prediction of new links between already known nodes), we clarified that we focus on the transductive setting (in section 1 and in section 3.2).

The survey paper [1] mentionned by reviewer 5mV9 is clearly related even though it does not include any experimental evaluation. We clarified this in the related work (section 2, page 2).

[1] W. Zhang, J. Chen, J. Li, Z. Xu, J. Z. Pan and H. Chen, "Knowledge Graph Reasoning with Logics and Embeddings: Survey and Perspective," 2024 IEEE International Conference on Knowledge Graph (ICKG), Abu Dhabi, United Arab Emirates, 2024

Submission Number: 6