A Teacher-Guided Framework for Graph Representation Learning
Keywords: graph neural network, representation learning
TL;DR: A genric teacher-guided contrastive learning based framework to produce better graph representations
Abstract: We consider the problem of unsupervised representation learning for Graph Neural Networks (GNNs).
Several state-of-the-art approaches to this problem are based on Contrastive Learning (CL) principles that generate transferable representations.
Their objective function can be posed as a supervised discriminative task using 'hard labels', as they consider each pair of graphs as either 'equally positive' or 'equally negative'.
However, it has been observed that using 'soft labels' in a Bayesian way can reduce the variance of the risk for discriminative tasks in supervised settings.
Motivated by this, we propose a CL framework for GNNs, called *Teacher-guided Graph Contrastive Learning (TGCL)*, that incorporates `soft labels' to facilitate a more regularized discrimination.
In particular, we propose a teacher-student framework where the student network learns the representation by distilling the representations produced by the teacher network trained using unlabelled graphs.
Our proposed approach can be adapted to any existing CL methods and empirically improves the performance across diverse downstream tasks.
Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 1714
Loading