Walking Out of the Weisfeiler Leman Hierarchy: Graph Learning Beyond Message Passing

Jan Tönshoff, Martin Ritzert, Hinrikus Wolf, Martin Grohe

Abstract: We propose CRaWl, a novel neural network architecture for graph learning. Like graph neural networks, CRaWl layers update node features on a graph and thus can freely be combined or interleaved with GNN layers. Yet CRaWl operates fundamentally different from message passing graph neural networks. CRaWl layers extract and aggregate information on subgraphs appearing along random walks through a graph using 1D Convolutions. Thereby it detects long range interactions and computes non-local features. As the theoretical basis for our approach, we prove a theorem stating that the expressiveness of CRaWl is incomparable with that of the Weisfeiler Leman algorithm and hence with graph neural networks. That is, there are functions expressible by CRaWl, but not by GNNs and vice versa. This result extends to higher levels of the Weisfeiler Leman hierarchy and thus to higher-order GNNs. Empirically, we show that CRaWl matches state-of-the-art GNN architectures across a multitude of benchmark datasets for classification and regression on graphs.

Submission Length: Long submission (more than 12 pages of main content)

Changes Since Last Submission:

We updated the paper based on the helpful feedback from the reviewers.

The key changes are as follows:
1. We expanded the related work section and discuss the relationship to random walk GNNs and subraph GNNs in greater detail.
2. The section on expressiveness has been updated with a clearer notion of distinguishability. Theorem 3 in particular has been rephrased to be more concise.
3. We added more experiments on datasets from the long-range graph benchmark [1] to the main experiments. An additional experiment on counting subgraphs in synthetic graphs has also been added to the appendix.
4. The discussion on asymptotic runtime and table of physical runtime (now Table 8, previously Table 5) have been expanded with more details.

Additionally, we also fixed typos, broken references and other minor issues that were kindly pointed out.

[1] Dwivedi et al., Long range graph benchmark, Advances in Neural Information Processing Systems, 2022

Code: https://github.com/toenshoff/CRaWl

Assigned Action Editor: ~Guido_Montufar1

License: Creative Commons Attribution 4.0 International (CC BY 4.0)

Submission Number: 1059