There is More to Graphs than Meets the Eye: Learning Universal Features with Self-supervision

Laya Das, Sai Munikoti, Mahantesh Halappanavar

Primary Area: learning on graphs and other geometries & topologies

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: Representation learning, Self supervised learning, Foundation models, Generalisability, Graph transformer

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

TL;DR: A self-supervision framework to learn generalizable features across multiple graphs of a family in an end-to-end-manner is presented.

Abstract: We study the problem of learning universal features from multiple graphs through self-supervision. Graph self-supervised learning has been shown to facilitate representation learning, and produce competitive models compared to supervised baselines. However, existing methods of self-supervision learn features from one graph, and thus, produce models that are specialized to a particular graph. We hypothesize that leveraging multiple graphs of a family can improve the quality of learnt representations in the model by extracting features that are universal to the family of graphs. To achieve this, we propose a framework that can learn generalisable representations from disparate node features of different graphs. We first homogenise the disparate features with graph-specific modules, which feed into a universal representation learning module for generalisable feature learning. We show that leveraging multiple graphs of the same family improves the quality of representations and results in better performance on downstream node classification task compared to self-supervision with one graph. In this paper, we present a principled way to design foundation graph models that are capable of learning from a set of graphs in a holistic manner. This approach bridges the gap between self-supervised and supervised performance, while reducing the computational time for self-supervision and parameters of the model.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 9097