How do skip connections affect Graph Convolutional networks with graph sampling? A theoretical analysis on generalization

Jiawei Sun, Hongkang Li, Meng Wang

Primary Area: learning theory

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: graph neural network (GNN), skip-connection, graph samping, generalization analysis, deep learning theory

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Abstract:

Skip connections enable deep Graph Convolutional Networks (GCNs) to overcome oversmoothing, while graph sampling reduces computational demands by selecting a submatrix of the graph adjacency matrix during neighborhood aggregation. Learning deep GCNs with graph sampling has shown empirical success across various applications, but a theoretical understanding of the generalization guarantees remains limited, with existing analyses ignoring either graph sampling or skip connections. This paper presents the first generalization analysis of GCNs with skip connections using graph sampling.
Our analysis demonstrates that the generalization accuracy of the learned model closely approximates the highest achievable accuracy within a broad class of target functions dependent on the proposed sparse effective adjacency matrix, denoted by $A^*$. Thus, graph sampling maintains generalization performance when $A^*$ accurately models data correlations. Notably, our findings reveal that skip connections lead to different sampling requirements across layers. In a two-hidden-layer GCN, the generalization is more affected by the sampled matrix deviations from $A^*$ of the first layer than the second layer. To the best of our knowledge, this marks the first theoretical characterization of skip connections' role in sampling requirements. We validate our theoretical results on benchmark datasets.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 8606