Attentive Walk-Aggregating Graph Neural Networks

Mehmet F Demirel, Shengchao Liu, Siddhant Garg, Zhenmei Shi, Yingyu Liang

Abstract: Graph neural networks (GNNs) have been shown to possess strong representation power, which can be exploited for downstream prediction tasks on graph-structured data, such as molecules and social networks. They typically learn representations by aggregating information from the $K$-hop neighborhood of individual vertices or from the enumerated walks in the graph. Prior studies have demonstrated the effectiveness of incorporating weighting schemes into GNNs; however, this has been primarily limited to $K$-hop neighborhood GNNs so far. In this paper, we aim to design an algorithm incorporating weighting schemes into walk-aggregating GNNs and analyze their effect. We propose a novel GNN model, called {\AWARE}, that aggregates information about the walks in the graph using attention schemes. This leads to an end-to-end supervised learning method for graph-level prediction tasks in the standard setting where the input is the adjacency and vertex information of a graph, and the output is a predicted label for the graph. We then perform theoretical, empirical, and interpretability analyses of {\AWARE}. Our theoretical analysis in a simplified setting identifies successful conditions for provable guarantees, demonstrating how the graph information is encoded in the representation, and how the weighting schemes in {\AWARE} affect the representation and learning performance. Our experiments demonstrate the strong performance of {\AWARE} in graph-level prediction tasks in the standard setting in the domains of molecular property prediction and social networks. Lastly, our interpretation study illustrates that {\AWARE} can successfully capture the important substructures of the input graph. The code is available on \href{https://github.com/mehmetfdemirel/aware}{GitHub}.

Submission Length: Long submission (more than 12 pages of main content)

Changes Since Last Submission:

The camera-ready version has addressed the comments by the editor:
1. Added clarification about the walk-aggregation perspective.
2. Elaborated more on the representation power of the walk statistics and added an example.

The draft has also been polished in a few places, improving the presentation.

Code: https://github.com/mehmetfdemirel/aware

Assigned Action Editor: ~Yujia_Li1

License: Creative Commons Attribution 4.0 International (CC BY 4.0)

Submission Number: 55