Manifold Inspired Graph Contrastive Learning
Keywords: Self-supervised Learning, Graph Representation Learning
TL;DR: a novel contrastive representation learning framework that employs cross-view adjacency reconstruction paired with feature orthogonalization
Abstract: Recently, graph contrastive learning (GCL) has emerged as a promising and trending paradigm for graph representation learning, providing generalizable node embeddings for various downstream tasks. However, current GCL methods often fail to fully exploit and encode the fine-grained graph structure information, leading to less informative node representations. In this study, we argue for a holistic approach that accounts for both node attributes and fine-grained graph structures, taking inspiration from spectral-based manifold learning techniques. Accordingly, we introduce MIGCL, a cutting-edge contrastive representation learning framework that employs cross-view adjacency reconstruction and feature orthogonalization. This dual approach not only retains the fine-grained graph/manifold structure information but also minimizes feature redundancy, thus averting the risk of representation collapse. To achieve feature orthogonalization, we employ an information-theoretic objective called Total Coding Rate. Our model can also be interpreted as a practical implementation of the Maximum Entropy Principle within the GCL context. Comprehensive experiments across three pivotal tasks: node classification, node clustering, and link prediction, affirm the method's efficacy and superiority.
Primary Area: learning on graphs and other geometries & topologies
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 3371
Loading