SMiLE: Schema-augmented Multi-level Contrastive Learning for Knowledge Graph Link Prediction

Miao Peng, Ben Liu, Qianqian Xie, Wenjie Xu, Min Peng

Abstract: Link prediction is the task of inferring miss- ing links between entities in knowledge graphs. Embedding-based methods have shown effec- tiveness in addressing this problem by mod- eling relational patterns in triples. However, the link prediction task often requires con- textual information in entity neighborhoods, while most existing embedding-based meth- ods fail to capture it. Additionally, little at- tention is paid to the diversity of entity rep- resentations in different contexts, which often leads to false prediction results. In this situ- ation, we consider that the schema of knowl- edge graph contains the specific contextual information, and it is beneficial for preserv- ing the consistency of entities across contexts. In this paper, we propose a novel Schema- augmented Multi-level contrastive LEarning framework (SMiLE) to conduct knowledge graph link prediction. Specifically, we first exploit network schema as the prior con- straint to sample negatives and pre-train our model by employing a multi-level contrastive learning method to yield both prior schema and contextual information. Then we fine- tune our model under the supervision of in- dividual triples to learn subtler representa- tions for link prediction. Extensive experimen- tal results on four knowledge graph datasets with thorough analysis of each component demonstrate the effectiveness of our proposed framework against state-of-the-art baselines. The implementation of SMiLE is available at https://github.com/GKNL/SMiLE.