Graph Neural Diffusion with Adaptive Skip Connection

Mohammad Shirzadi, Ali Safarpoor Dehkordi, Ahad N. Zehmakan

Keywords: Graph Neural Networks, Graph Neural Diffusion, Skip Connection, Oversmoothing, Numerical Stability

TL;DR: We propose Graph Neural Diffusion with Adaptive Skip Connection (GRAND-ASC), a well-defined framework that has a stable numerical approximation and mitigates oversmoothing.

Abstract: Neural message passing on graphs can suffer from the oversmoothing problem, where repeated aggregation of neighborhood information causes node embeddings to become indistinguishable. This issue is not confined to discrete Graph Neural Networks (GNNs); it also arises in continuous-depth GNNs, such as Graph Neural Diffusion (GRAND), where a diffusion process governs feature evolution. Current solutions often involve adding auxiliary data-dependent source terms or employing nonlinear dynamics, rather than relying solely on pure diffusion. In this work, we propose a simple yet powerful linear alternative: Graph Neural Diffusion with Adaptive Skip Connection (GRAND-ASC). Our framework equips the standard GRAND model with a skip connection to the initial node features, which by itself is sufficient to prevent oversmoothing. Furthermore, to increase our model's adaptability, we introduce a learnable time-dependent parameter that dynamically balances the trade-off between integrating neighborhood information and preserving a node's initial features. We provide a theoretical foundation for GRAND-ASC, proving its analytical well-posedness and the numerical stability of its approximations. Furthermore, we formally demonstrate that our dynamics mitigate oversmoothing by ensuring the Dirichlet energy remains bounded away from zero. Through a comprehensive set of experiments, we demonstrate that our model achieves competitive state-of-the-art performance on node classification tasks, with particularly strong results on heterophilic benchmarks where preserving node-specific information is crucial. The source code is available at:~\textcolor{blue}{https://tinyurl.com/3n8r6nxn}.

Primary Area: learning on graphs and other geometries & topologies

Submission Number: 9854