Learnable Graph Convolutional Attention NetworksDownload PDF

04 Oct 2022, 08:28 (modified: 02 Nov 2022, 09:07)HITY Workshop NeurIPS 2022Readers: Everyone
Keywords: GCN, GNN, GAT, cross-validation, interpolation
TL;DR: We propose a GNN which learns to use, in each layer, an interpolation of a GCN, GAT, and a GAT with convolved features. It outperforms existing methods, is more robust, and removes the need of cross-validating.
Abstract: Existing Graph Neural Networks (GNNs) compute the message exchange between nodes by either convolving the features of all the neighboring nodes (GCNs), or by applying attention instead (GATs). In this work, we aim at exploiting the strengths of both approaches to their full extent. To this end, we first introduce a graph convolutional attention layer (CAT), which relies on convolutions to compute the attention scores, and theoretically show that there is no clear winner between the three models, as their performance depends on the nature of the data. This brings us to our main contribution, the learnable graph convolutional attention network (L-CAT): a GNN architecture that automatically interpolates between GCN, GAT and CAT in each layer, by introducing two additional (scalar) parameters. Our results demonstrate that L-CAT is able to efficiently combine different GNN layers along the network, outperforming competing methods in a wide range of datasets, and resulting in a more robust model that reduces the need of cross-validating.
Supplementary Material: zip
3 Replies