MARIO: Model Agnostic Recipe for Improving OOD Generalization of Graph Contrastive Learning

Yun Zhu, Haizhou Shi, Zhenshuo Zhang, Siliang Tang

Keywords: Graph Neural Networks, Domain Generalization, Self-Supervised Learning, Graph Representation Learning, Pre-Training

Abstract:

In contemporary research, large-scale graphs and graph neural networks (GNNs) serve as prevalent tools for organizing and modeling web-related data. Nevertheless, the dynamic nature of web content, characterized by continual change and evolution over time (e.g., the prevailing trends and citation patterns in online citation networks), presents a formidable challenge to the adaptability of GNNs in addressing these distributional shifts.
In this work, we investigate the problem of out-of-distribution (OOD) generalization for unsupervised learning methods on graph data.
To improve the robustness against such distributional shifts, we propose a $\underline{M}$odel-$\underline{A}$gnostic $\underline{R}$ecipe for $\underline{I}$mproving $\underline{O}$OD generalizability of unsupervised graph contrastive learning methods, which we refer to as MARIO. MARIO introduces two principles aimed at developing distributional-shift-robust graph contrastive methods to overcome the limitations of existing frameworks: (i) Invariant principle that incorporates adversarial graph augmentation to obtain invariant representations and (ii) Information Bottleneck (IB) principle for achieving generalizable representations through refining representation contrasting.
To the best of our knowledge, this is the first work that investigates the OOD generalization problem of graph contrastive learning, with a specific focus on node-level tasks. Through extensive experiments, we demonstrate that our method achieves state-of-the-art performance on the OOD test set, while maintaining comparable performance on the in-distribution test set when compared to existing approaches.

Track: Graph Algorithms and Learning for the Web

Submission Guidelines Scope: Yes

Submission Guidelines Blind: Yes

Submission Guidelines Format: Yes

Submission Guidelines Limit: Yes

Submission Guidelines Authorship: Yes

Student Author: Yes

Submission Number: 77