Go to ICLR 2024 Conference homepageHow do skip connections affect Graph Convolutional networks with graph sampling? A theoretical analysis on generalization
Skip connections enable deep Graph Convolutional Networks (GCNs) to overcome oversmoothing, while graph sampling reduces computational demands by selecting a submatrix of the graph adjacency matrix during neighborhood aggregation. Learning deep GCNs with graph sampling has shown empirical success across various applications, but a theoretical understanding of the generalization guarantees remains limited, with existing analyses ignoring either graph sampling or skip connections. This paper presents the first generalization analysis of GCNs with skip connections using graph sampling. Our analysis demonstrates that the generalization accuracy of the learned model closely approximates the highest achievable accuracy within a broad class of target functions dependent on the proposed sparse effective adjacency matrix, denoted by $A^$. Thus, graph sampling maintains generalization performance when $A^$ accurately models data correlations. Notably, our findings reveal that skip connections lead to different sampling requirements across layers. In a two-hidden-layer GCN, the generalization is more affected by the sampled matrix deviations from $A^*$ of the first layer than the second layer. To the best of our knowledge, this marks the first theoretical characterization of skip connections' role in sampling requirements. We validate our theoretical results on benchmark datasets.