Towards More Robust GNN Training with Graph Normalization for GraphSAINT

Yuying Wang, Qinfen Hao

Abstract: Graph Neural Networks (GNNs) field has a dramatically development nowadays due to the strong representation ability for data in non-Euclidean space, such as graphs. However, with the larger graph datasets and the trend of more complex algorithms, the stability problem appears during model training. For example, GraphSAINT algorithm will not converge in training with a probability range from 0.1 to 0.4. In order to solve this problem, this paper proposes an improved GraphSAINT method. Firstly, a proper graph normalization strategy is introduced into the model as a neural network layer. Secondly, the structure of the model is modified based on the normalization strategy to normalize the original input data and the input data of the middle layer. Thirdly, the training process and the inference process of the model are adjusted to fit this normalization strategy. The improved GraphSAINT method successfully eliminates the instability and improves the robustness during training. Besides, it accelerates the training procedure convergence of the GraphSAINT algorithm and reduces the training time by about a quarter. Furthermore, it also achieves an improvement in the pre-diction accuracy. The effectiveness of the improved method is verified by using the citation dataset of Open Graph Benchmark (OGB).