Learnable Graph Convolutional Attention Networks

Adrián Javaloy, Pablo Sanchez Martin, Amit Levi, Isabel Valera

Keywords: GNN, GCN, GAT

TL;DR: We propose a GNN which learns to use, in each layer, an interpolation of a GCN, GAT, and a GAT with convolved features. It outperforms existing methods, is more robust, and removes the need of cross-validating.

Abstract: Existing Graph Neural Networks (GNNs) compute the message exchange between nodes by either aggregating uniformly (convolving) the features of all the neighboring nodes, or by applying a non-uniform score (attending) to the features. Recent works have shown the strengths and weaknesses of the resulting GNN architectures, respectively, GCNs and GATs. In this work, we aim at exploiting the strengths of both approaches to their full extent. To that end, we first introduce a graph convolutional attention layer (CAT), which relies on convolutions to compute the attention scores. Unfortunately, as in the case of GCNs and GATs, we then show that there exists no clear winner between the three—neither theoretically nor in practice—since their performance directly depends on the nature of the data (i.e., of the graph and features). This result brings us to the main contribution of this work, the learnable graph convolutional attention network (L-CAT): a GNN architecture that allows us to automatically interpolate between GCN, GAT and CAT in each layer, by only introducing two additional (scalar) parameters. Our results demonstrate that L-CAT is able to efficiently combine different GNN layers across the network, outperforming competing methods in a wide range of datasets, and resulting in a more robust model that needs less cross-validation.

Community Implementations: [ 2 code implementations](https://www.catalyzex.com/paper/arxiv:2211.11853/code)