Link Prediction on Text Attributed Graphs: A New Benchmark and Efficient LM-nested GNN Design
Keywords: Link Prediction, Graph Convolutional Network, Pretrain Language Model
Abstract: Textual and topological information is significant for link prediction (LP) in textattributed graphs (TAGs). Recent link prediction methods have focused on improving the performance of capturing structural features by Graph Convolutional Networks (GCNs), the importance of enhancing text embeddings, powered by the powerful Pretrain Language Model (PLM), has been underestimated. We collect and introduce eight graphs with rich textual information. We further benchmarked current competitive link prediction methods and PLM-based methods in a unified experimental setting, systematically investigating the representation power of the text encoders in the link prediction task. Based on our investigation, we introduce LMGJOINT — a memory-efficient fine-tuning method. The key design features include: residual connection of textual proximity, a combination of structural and textual embeddings, and a cache embedding training strategy. Our empirical analysis shows that these design elements improve MRR by up to 19.75% over previous state-of-the-art methods and achieve competitive performance across a wide range of models and datasets.
Supplementary Material: pdf
Primary Area: learning on graphs and other geometries & topologies
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 7318
Loading