Evaluating Deep Graph Neural Networks

Wentao Zhang, Zeang Sheng, Jiang Yuezihan, Yikuan Xia, Jun Gao, Zhi Yang, Bin CUI

Keywords: Deep, Graph Neural Networks

Abstract: Graph Neural Networks (GNNs) have already been widely applied in various graph mining tasks. However, most GNNs only have shallow architectures, which limits performance improvement. In this paper, we conduct a systematic experimental evaluation on the fundamental limitations of current architecture designs. Based on the experimental results, we answer the following two essential questions: (1) what actually leads to the compromised performance of deep GNNs; (2) how to build deep GNNs. The answers to the above questions provide empirical insights and guidelines for researchers to design deep GNNs. Further, we present Deep Graph Multi-Layer Perceptron (DGMLP), a powerful approach implementing our proposed guidelines. Experimental results demonstrate three advantages of DGMLP: 1) high accuracy -- it achieves state-of-the-art node classification performance on various datasets; 2) high flexibility -- it can flexibly choose different propagation and transformation depths according to certain graph properties; 3) high scalability and efficiency -- it supports fast training on large-scale graphs.

One-sentence Summary: A systematic experimental evaluation on the fundamental limitations of current GNN architecture designs.

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:2108.00955/code)