Towards Understanding Link Predictor Generalizability Under Distribution Shifts

Jay Revolinsky, Harry Shomer, Jiliang Tang

Keywords: Link Prediction, Graph-Structured Data, GNN4LP, Distribution Shifts, Structural Heuristics, Splitting Strategies

TL;DR: Novel and simple strategy to induce controlled distribution shifts on link-prediction datasets; includes benchmarking of SOTA and generalization techniques, along with further analysis.

Abstract:

State-of-the-art link prediction (LP) models demonstrate impressive benchmark
results. However, popular benchmark datasets often assume that training, validation, and testing samples are representative of the overall dataset distribution. In
real-world situations, this assumption is often incorrect; since uncontrolled factors
lead to the problem where new dataset samples come from different distributions
than training samples. The vast majority of recent work focuses on dataset shift
affecting node- and graph-level tasks, largely ignoring link-level tasks. To bridge
this gap, we introduce a novel splitting strategy, known as LPShift, which utilizes
structural properties to induce a controlled distribution shift. We verify the effect of LPShift through empirical evaluation of SOTA LP methods on 16 LPShift
generated splits of Open Graph Benchmark (OGB) datasets. When benchmarked
with LPShift datasets, GNN4LP methods frequently generalize worse than heuristics or basic GNNs. Furthermore, LP-specific generalization techniques do little
to improve performance under LPShift. Finally, further analysis provides insight
on why LP models lose much of their architectural advantages under LPShift.

Primary Area: datasets and benchmarks

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 7544