Efficient Learning on Large Graphs using a Densifying Regularity Lemma

Jonathan Kouchly, Ben Finkelshtein, Michael M. Bronstein, Ron Levie

Keywords: regularity lemma, graphon, graph representation learning, directed graphs, scalability

TL;DR: We introduce a low-rank graph factorization, leading to an architecture with time and space complexity linear in the number of nodes.

Abstract: Learning on large graphs presents significant challenges, with traditional Message Passing Neural Networks suffering from computational and memory costs scaling linearly with the number of edges. We introduce the Intersecting Block Graph (IBG), a low-rank factorization of large directed graphs based on combinations of intersecting bipartite components, each consisting of a pair of communities, for source and target nodes. By giving less weight to non-edges, we show how an IBG can efficiently approximate any graph, sparse or dense. Specifically, we prove a constructive version of the weak regularity lemma: for any chosen accuracy, every graph can be approximated by a dense IBG whose rank depends only on that accuracy. This improves over prior versions of the lemma, where the rank depended on the number of nodes for sparse graphs. We then introduce a graph neural network architecture operating on the IBG representation of the graph and demonstrating competitive performance on node classification, spatio-temporal graph analysis, and knowledge graph completion, while having memory and computational complexity linear in the number of nodes rather than edges.

Submission Number: 23