Bridge Structural Knowledge and Pre-trained Language Models for Knowledge Graph Completion

Anonymous

Abstract: Knowledge graph completion (KGC) is a task of inferring missing triples based on existing Knowledge Graphs (KGs). Both structural and semantic information are vital for successful KGC. However, existing methods only use either the structural knowledge from the KG embeddings or the semantic information from pre-trained language models (PLMs), leading to suboptimal model performance. Moreover, since PLMs are not trained on KGs, directly using PLMs to encode triples is inappropriate. To overcome these limitations, we propose a novel model called Bridge, which jointly encodes structural and semantic information of KGs. Specifically, we strategically encode entities and relations separately by PLMs to better utilize the semantic knowledge of PLMs and enable structured representation learning via a structural learning principle. Furthermore, to bridge the gap between KGs and PLMs, we employ a self-supervised representation learning method called BYOL to fine-tune PLMs with two different views of a triple. Experiments demonstrate that Bridge outperforms the SOTA models on three benchmark datasets.

Paper Type: long

Research Area: NLP Applications

Contribution Types: Model analysis & interpretability, NLP engineering experiment

Languages Studied: English

Preprint Status: There is no non-anonymous preprint and we do not intend to release one.

A1: yes

A1 Elaboration For Yes Or No: 7

A2: n/a

A3: yes

A3 Elaboration For Yes Or No: abstract ans section 1

B: yes

B1: yes

B1 Elaboration For Yes Or No: 1,2,3,4,5

B2: yes

B2 Elaboration For Yes Or No: appendix

B3: yes

B3 Elaboration For Yes Or No: 1,2,3,4,5,appendix

B4: n/a

B5: yes

B5 Elaboration For Yes Or No: appendix

B6: yes

B6 Elaboration For Yes Or No: appendix

C: yes

C1: yes

C1 Elaboration For Yes Or No: appendix

C2: yes

C2 Elaboration For Yes Or No: appendix

C3: yes

C3 Elaboration For Yes Or No: appendix

C4: yes

C4 Elaboration For Yes Or No: appendix

D: no

D1: n/a

D2: n/a

D3: n/a

D4: n/a

D5: n/a

E: yes

E1: yes

E1 Elaboration For Yes Or No: 1