Attentive Walk-Aggregating Graph Neural Networks

Published: 25 Aug 2022, Last Modified: 28 Feb 2023Accepted by TMLREveryoneRevisionsBibTeX
Abstract: Graph neural networks (GNNs) have been shown to possess strong representation power, which can be exploited for downstream prediction tasks on graph-structured data, such as molecules and social networks. They typically learn representations by aggregating information from the $K$-hop neighborhood of individual vertices or from the enumerated walks in the graph. Prior studies have demonstrated the effectiveness of incorporating weighting schemes into GNNs; however, this has been primarily limited to $K$-hop neighborhood GNNs so far. In this paper, we aim to design an algorithm incorporating weighting schemes into walk-aggregating GNNs and analyze their effect. We propose a novel GNN model, called {\AWARE}, that aggregates information about the walks in the graph using attention schemes. This leads to an end-to-end supervised learning method for graph-level prediction tasks in the standard setting where the input is the adjacency and vertex information of a graph, and the output is a predicted label for the graph. We then perform theoretical, empirical, and interpretability analyses of {\AWARE}. Our theoretical analysis in a simplified setting identifies successful conditions for provable guarantees, demonstrating how the graph information is encoded in the representation, and how the weighting schemes in {\AWARE} affect the representation and learning performance. Our experiments demonstrate the strong performance of {\AWARE} in graph-level prediction tasks in the standard setting in the domains of molecular property prediction and social networks. Lastly, our interpretation study illustrates that {\AWARE} can successfully capture the important substructures of the input graph. The code is available on \href{}{GitHub}.
Submission Length: Long submission (more than 12 pages of main content)
Changes Since Last Submission: The camera-ready version has addressed the comments by the editor: 1. Added clarification about the walk-aggregation perspective. 2. Elaborated more on the representation power of the walk statistics and added an example. The draft has also been polished in a few places, improving the presentation.
Assigned Action Editor: ~Yujia_Li1
License: Creative Commons Attribution 4.0 International (CC BY 4.0)
Submission Number: 55