A Manifold Perspective on the Statistical Generalization of Graph Neural Networks

ICLR 2025 Conference Submission3844 Authors

Keywords: generalization analysis, graph neural networks, manifold neural networks

TL;DR: We leverage manifold theory to analyze the generalization gap of GNNs sampled from manifolds in the spectral domain.

Abstract:

Graph Neural Networks (GNNs) extend convolutional neural networks to operate on graphs. Despite
their impressive performances in various graph learning tasks, the theoretical understanding of
their generalization capability is still lacking. Previous GNN generalization bounds ignore the
underlying graph structures, often leading to bounds that increase with the number of nodes – a
behavior contrary to the one experienced in practice. In this paper, we take a manifold perspective
to establish the statistical generalization theory of GNNs on graphs sampled from a manifold in the
spectral domain. As demonstrated empirically, we prove that the generalization bounds of GNNs
decrease linearly with the size of the graphs in the logarithmic scale, and increase linearly with the
spectral continuity constants of the filter functions. Notably, our theory explains both node-level and
graph-level tasks. Our result has two implications: i) guaranteeing the generalization of GNNs to
unseen data over manifolds; ii) providing insights into the practical design of GNNs, i.e., restrictions
on the discriminability of GNNs are necessary to obtain a better generalization performance. We
demonstrate our generalization bounds of GNNs using synthetic and multiple real-world datasets.

Primary Area: learning on graphs and other geometries & topologies

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 3844