Don't stack layers in graph neural networks, wire them randomlyDownload PDF

08 Mar 2021, 17:52 (edited 08 Apr 2021)GTRL 2021 SpotlightReaders: Everyone
  • Keywords: graph neural networks, random architectures
  • TL;DR: Randomly wired architectures boost the performance of graph neural networks, providing a more effective way of increasing capacity with respect to Resnets.
  • Abstract: Several results suggest an inherent difficulty of graph neural networks in extracting better performance by increasing the number of layers. Recent works attribute this to a phenomenon peculiar to the extraction of node features in graph-based tasks, i.e., the need to consider multiple neighborhood sizes at the same time and adaptively tune them. In this paper, we investigate the recently proposed randomly wired architectures in the context of graph neural networks. Instead of building deeper networks by stacking many layers, we prove that employing a randomly-wired architecture can be a more effective way to increase the capacity of the network and obtain richer representations. We show that such architectures behave like an ensemble of paths, which are able to merge contributions from receptive fields of varied size. Moreover, these receptive fields can also be modulated to be wider or narrower through the trainable weights over the paths.
  • Poster: png
1 Reply