Don't stack layers in graph neural networks, wire them randomly

Diego Valsesia, Giulia Fracastoro, Enrico Magli

Keywords: graph neural networks, random architectures

TL;DR: Randomly wired architectures boost the performance of graph neural networks, providing a more effective way of increasing capacity with respect to Resnets.

Abstract: Several results suggest an inherent difficulty of graph neural networks in extracting better performance by increasing the number of layers. Recent works attribute this to a phenomenon peculiar to the extraction of node features in graph-based tasks, i.e., the need to consider multiple neighborhood sizes at the same time and adaptively tune them. In this paper, we investigate the recently proposed randomly wired architectures in the context of graph neural networks. Instead of building deeper networks by stacking many layers, we prove that employing a randomly-wired architecture can be a more effective way to increase the capacity of the network and obtain richer representations. We show that such architectures behave like an ensemble of paths, which are able to merge contributions from receptive fields of varied size. Moreover, these receptive fields can also be modulated to be wider or narrower through the trainable weights over the paths.