Learning Graph Normalization for Graph Neural Networks
Keywords: Graph Neural Network, Normalization, Graph Normalization
Abstract: Graph Neural Networks (GNNs) have emerged as a useful paradigm to process graph-structured data. Usually, GNNs are stacked to multiple layers and the node representations in each layer are computed through propagating and aggre- gating the neighboring node features with respect to the graph. To effectively train a GNN with multiple layers, some normalization techniques are necessary. Though the existing normalization techniques have been shown to accelerate train- ing GNNs, the structure information on the graph is ignored yet. In this paper, we propose two graph-aware normalization methods to effectively train GNNs. Then, by taking into account that normalization methods for GNNs are highly task-relevant and it is hard to know in advance which normalization method is the best, we propose to learn attentive graph normalization by optimizing a weighted combination of multiple graph normalization methods at different scales. By op- timizing the combination weights, we can automatically select the best or the best combination of multiple normalization methods for a specific task. We con- duct extensive experiments on benchmark datasets for different tasks and confirm that the graph-aware normalization methods lead to promising results and that the learned weights suggest the more appropriate normalization methods for specific task
One-sentence Summary: We propose two graph-aware normalization methods for training GNNs and also propose to learn graph normalization which optimizes a weighted combination of multiple normalization methods.
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics
Community Implementations: [ 2 code implementations](https://www.catalyzex.com/paper/arxiv:2009.11746/code)
Reviewed Version (pdf): https://openreview.net/references/pdf?id=XivziPUt41
5 Replies
Loading