In-distribution Generalization and Size Generalization for Graph Neural Networks
Abstract: Graph neural networks (GNNs) are models that allow learning with structured data of varying sizes.
Despite their popularity, theoretical understanding of the generalization of GNNs is an under-explored topic.
In this work, we expand the theoretical understanding of both in-distribution and out-of-distribution generalization of GNNs.
Firstly, we improve upon the state-of-the-art PAC-Bayes (in-distribution) generalization bound primarily by reducing an exponential dependency on the node degree to a linear dependency.
Secondly, utilizing tools from spectral graph theory, we prove some rigorous guarantees about the out-of-distribution (OOD) size generalization of GNNs, where graphs in the training set have different numbers of nodes and edges from those in the test set.
To empirically verify our theoretical findings, we conduct experiments on both synthetic and real-world graph datasets.
Our computed generalization gaps for the in-distribution case significantly improve the state-of-the-art PAC-Bayes results.
For the OOD case, experiments on community classification tasks in large social networks show that GNNs achieve strong size generalization performance in cases guaranteed by our theory.
Submission Length: Regular submission (no more than 12 pages of main content)
Assigned Action Editor: ~Anastasios_Kyrillidis2
Submission Number: 1862
Loading