Link Prediction or Perdition: the Seeds of Instability in Knowledge Graph Embeddings

ICLR 2026 Conference Submission17819 Authors

Keywords: Knowledge Graph Embedding, Stability, Link Prediction

TL;DR: We show that knowledge graph embeddings are instable leading to concerns for link prediction in knowledge graph completion.

Abstract:

Embedding models (KGEMs) constitute the main link prediction approach to complete knowledge graphs.
Standard evaluation protocols emphasize rank-based metrics such as MRR or Hits@$K$, but usually overlook the influence of random seeds on result stability.
Moreover, these metrics conceal potential instabilities in individual predictions and in the organization of embedding spaces.
In this work, we conduct a systematic stability analysis of multiple KGEMs across several datasets.
We find that high-performance models actually produce divergent predictions at the triple level and highly variable embedding spaces.
By isolating stochastic factors, $\textit{i.e.}$, initialization, triple ordering, negative sampling, dropout, hardware, we show that each independently induces instability of comparable magnitude.
Furthermore, our results reveal no correlation between high MRR scores and stability.
These findings highlight critical limitations of current benchmarking protocols, and raise concerns about the reliability of KGEMs for knowledge graph completion.

Primary Area: learning on graphs and other geometries & topologies

Submission Number: 17819