Studying the Effect of GNN Spatial Convolutions On The Embedding Space's Geometry

Claire Donnat, So Won Jeong

Keywords: Graph Neural Networks, Diffusion Maps, Geometric Deep Learning

TL;DR: In this paper, we analyse the effect of the convolution operator on the embedding geometry.

Abstract:

By recursively summing node features over entire neighborhoods, spatial graph convolution operators have been heralded as key to the success of Graph Neural Networks (GNNs). Yet, despite the multiplication of GNN methods across tasks and applications, the effect of this aggregation operation has yet to be analyzed. In fact, while most recent efforts in the GNN community have focused on optimizing the architecture of the neural network, fewer works have attempted to characterize (a) the different classes of spatial convolution operators, (b) their impact on the geometry of the embedding space, and (c) how the choice of a particular convolution should relate to properties of the data. In this paper, we propose to answer all three questions by dividing existing operators into two main classes (symmetrized vs.
row-normalized spatial convolutions), and show how these correspond to different implicit biases on the data. Finally, we show that this convolution operator is in fact tunable, and explicit regimes in which certain choices of convolutions --- and therefore, embedding geometries --- might be more appropriate.