Capturing Structure and Feature Signals in Graph Self-Supervised Learning

Dmitry Eremeev, Gleb Bazhenov, Oleg Platonov, Artem Babenko, Liudmila Prokhorenkova

Keywords: graph neural network, self-supervised learning, pretraining

Abstract: This paper analyzes graph self-supervised learning (SSL) methods for node-level prediction tasks. First, we thoroughly evaluate several representative SSL methods on a diverse set of graph datasets. We observe that, contrary to prior literature, two popular generative methods MaskGAE and GraphMAE often fail to outperform well-tuned supervised baselines. At the same time, the contrastive methods BGRL and GRACE on average perform better than generative methods and supervised baselines. We hypothesize that this happens because BGRL and GRACE are able to capture the information about both graph structure and node features, while MaskGAE and GraphMAE concentrate on a single source of information. We support this hypothesis by conducting an analysis on carefully designed synthetic data. Motivated by our observations, we recommend designing SSL objectives that capture both feature and structure information. To verify the effectiveness of this approach, we propose a generative method that reconstructs both graph structure and node features. While being simple, this method outperforms all other considered approaches.

Primary Area: learning on graphs and other geometries & topologies

Submission Number: 7784