Effective Graph Representation Learning via Smoothed Contrastive Learning

Maysam Behmanesh, Maks Ovsjanikov

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: Graph representation learning, Contrastive learning, Graph neural network, Laplacian smoothing, Batch-generating, Large-scaled graphs

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

TL;DR: GCL aligns node representations but suffers from misclassification due to incorporating similar negative samples. We introduce Smoothed Graph Contrastive Learning (SGCL) to enhance alignment using geometric structures and smoothing techniques

Abstract: Graph contrastive learning (GCL) aligns node representations through the utilization of positive/negative node pairs, a selection process that typically relies on the correspondences and non-correspondences among nodes within two augmented graphs. The conventional GCL approaches incorporate negative samples uniformly in the contrastive loss, resulting in the equal treatment of misclassified false negative nodes, regardless of their proximity to the true positive. In this paper, we present a Smoothed Graph Contrastive Learning model (SGCL), which leverages the geometric structure of augmented graphs to exploit proximity information associated with positive/negative pairs in contrastive loss. The proposed SGCL adjusts the significance of these pairs in contrastive loss by incorporating three distinct smoothing techniques that yield smoothed positive/negative pairs. To enhance scalability for large-scale graphs, the proposed framework incorporates a graph batch-generating strategy that partitions the given graphs into multiple subgraphs, facilitating efficient training in separate batches. Through extensive experimentation in an unsupervised setting on various benchmark datasets, particularly those of large scale, we demonstrate the superiority of our proposed framework.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 8116