Learning Invariant Link Formation Mechanisms via Pretraining and Adaptation

Yu Song, Hui Liu

Keywords: Recommender Systems, Graph Neural Networks, Link Prediction

Abstract: Link prediction (LP) is a central task in graph-based recommendation systems, enabling the discovery of potential user-item interactions. However, existing LP models often struggle with data sparsity, leading to spurious correlations and poor generalization. In this work, we explore pretraining as a scalable approach to causal learning for LP, aiming to extract invariant link formation mechanisms from large and diverse graphs. We propose a modular framework that decomposes link prediction into node-level and edge-level reasoning, and introduce a Mixture-of-Experts (MoE) architecture to model heterogeneous causal patterns across data subsets. For deployment, we adopt a parameter-efficient adaptation strategy that aggregates expert outputs without full model retraining. Our approach, PALP, achieves state-of-the-art performance and efficiency on six real-world datasets, demonstrating the promise of pretraining and modular adaptation as a scalable path toward causal representation learning in recommendation systems.

Submission Number: 18