Understanding Graph Transformers by Generalized Propagation
Primary Area: learning on graphs and other geometries & topologies
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Keywords: geometric deep learning; graph transformer; discrete ricci curvature
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
Abstract: Graph Transformers (GTs) have recently shown stellar performance on various
graph learning benchmarks, which is typically attributed to their underlying global
self-attention mechanism. In this paper, we use generalized propagation graphs,
constructed through two abstract configurable functions and offering a unified
view across various GNN models used in the literature. We show that by con-
figuring the two abstract functions governing the generation of propagation graph,
one could recover the most popular GNN models including graph Transformers,
message-passing neural networks (MPNNs), as well as various forms of graph
rewiring. We show that the expressivity of the instances of our framework depends
on one of the governing functions (the adjacency function). Empirical results con-
firm our theory: by keeping the adjacency function while removing self-attention,
the state-of-the-art GT maintains its performance. In other words, by designing
appropriate adjacency functions, one could construct novel GNN models with di-
verse expressive power. We also study the geometric properties of the propagation
graphs across a wide range of models, using a novel extension the Ollivier-Ricci
curvature to weighted digraphs.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 3922
Loading