PDE-GCN: Novel Architectures for Graph Neural Networks Motivated by Partial Differential Equations

Moshe Eliasof, Eldad Haber, Eran Treister

Keywords: Graph Neural Networks, Deep Learning, Over-smoothing, Partial Differential Equations

TL;DR: We develop novel GCN layers based on PDE analysis to reason about their behaviour and propose a mixed construction for leanrt behaviour and enhanced performance.

Abstract:

Graph neural networks are have shown their efficacy in fields such as computer vision, computational biology and chemistry, where data are naturally explained by graphs. However, unlike convolutional neural networks, deep graph networks do not necessarily yield better performance than shallow networks.
This behaviour usually stems from the over-smoothing phenomenon. In this work, we propose a family of architectures
to control this behaviour by design. Our networks are motivated by numerical methods for solving Partial Differential Equations (PDEs) on manifolds, and as such, their behaviour can be explained by similar analysis.