Contrastive Graph Few-Shot Learning

Chunhui Zhang, Hongfu Liu, Jundong Li, Yanfang Ye, Chuxu Zhang

Keywords: Graph representation learning, Few-shot learning, Contrastive learning

TL;DR: We propose CGFL, a general and effective framework to mitigate the distribution shift impact for learning more generalizable representations on graph few-shot-learning tasks.

Abstract: Prevailing supervised deep graph learning models often suffer from label sparsity issue. Although many graph few-shot learning (GFL) methods have been developed to avoid performance degradation in face of limited annotated data, they excessively rely on labeled data, where the distribution shift in the test phase might result in impaired generalization ability. Additionally, they lack a general purpose as their designs are coupled with task or data-specific characteristics. To this end, we propose a general and effective Contrastive Graph Few-shot Learning framework (CGFL). CGFL leverages a self-distilled contrastive learning procedure to boost GFL. Specifically, our model firstly pre-trains a graph encoder with contrastive learning using unlabeled data. Later, the trained encoder is frozen as a teacher model to distill a student model with a contrastive loss. The distilled model is finally fed to GFL. CGFL learns data representation in a self-supervised manner, thus mitigating the distribution shift impact for better generalization and making model task and data-independent for a general graph mining purpose. Furthermore, we introduce an information-based method to quantitatively measure the capability of CGFL. Comprehensive experiments demonstrate that CGFL outperforms state- of-the-art baselines on several graph mining tasks across various datasets in the few-shot scenario. We also provide a quantitative measurement of CGFL’s success.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Submission Guidelines: Yes

Please Choose The Closest Area That Your Submission Falls Into: Deep Learning and representational learning

Community Implementations: [ 12 code implementations](https://www.catalyzex.com/paper/arxiv:2210.00084/code)