Graph Rhythm Network: Beyond Energy Modeling for Deep Graph Neural Networks

Yufei Jin, Xingquan Zhu

Abstract: Graph neural networks (GNN) have been commonly used for learning and classifying objects with correlated relationships. To date, many GNN architectures exist, but majority of them only work well on shallow networks due to the oversmoothing phenomenon, where node features become similar to each other, as the layer increases. In this paper, we point out that the key to create an informative deep GNN is to have an adaptive feature updating rate control for each node, where the updating rate should take each node's locality into consideration through shared trainable weight parameters. Accordingly, we advocate a new graph rhythm modeling as a generalized mechanism to the Dirichlet energy based approaches. Instead of merely modeling difference between nodes, like Dirichlet energy based approach does, graph rhythm focuses on omni-directional relationship mapping between each node and its neighbors. Such a mechanism provides a more general ways of capturing patterns between nodes (i.e. graph rhythm) for effective graph neural network learning. Experiments and comparisons, demonstrate the performance gain and show that \method\ can help create GNNs with deep layers, without suffering from performance deterioration or having better performance than shallow networks.