Revisiting Graph Contrastive Learning through the Lens of Contrastive Overfitting

Chengpei Wu, Lilin Zhang, Yang Lou

Keywords: unsupervised learning, graph learning

Abstract: Graph Contrastive Learning (GCL) has emerged as a powerful framework for unsupervised graph representation learning, typically optimized with contrastive objectives such as InfoNCE. Contrary to the common belief that lower contrastive loss implies better representations generated for downstream tasks, we observe little positive correlation between the contrastive objective and downstream performance. In fact, excessive optimization often leads to degraded performance--a clear symptom of overfitting. We attribute this phenomenon to the structure-agnostic nature of contrastive objective, which forces the encoder to discard essential structural information. Through extensive empirical and theoretical studies, we verify that the overfitted embeddings, which scarcely capture graph structural information, substantially impair generalization when applied to downstream classifiers. To address this issue, we propose a structure-preserving regularization (SPR) framework that can be seamlessly integrated as a plug-and-play module to enhance existing GCL methods. Comprehensive experiments across multiple datasets and baselines demonstrate that our approach effectively mitigates the overfitting problem.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Submission Number: 1660