DeepGRAND: Deep Graph Neural Diffusion

Khang Nguyen, Nong Minh Hieu, Tan Minh Nguyen, Nguyen Duy Khuong, Vinh Duc NGUYEN

Keywords: graph neural diffusion, graph neural networks, oversmoothing

TL;DR: We propose the Deep Graph Neural Diffusion (DeepGRAND), a class of continuous-depth graph neural networks that leverages a data-dependent scaling term and a perturbation to the graph diffusivity to overcome the oversmoothing issue.

Abstract: We propose the Deep Graph Neural Diffusion (DeepGRAND), a class of continuous-depth graph neural networks based on the diffusion process on graphs. DeepGRAND leverages a data-dependent scaling term and a perturbation to the graph diffusivity to make the real part of all eigenvalues of the diffusivity matrix become negative, which ensures two favorable theoretical properties: (i) the node representation does not exponentially converge to a constant vector as the model depth increases, thus alleviating the over-smoothing issue; (ii) the stability of the model is guaranteed by controlling the norm of the node representation. Compared to the baseline GRAND, DeepGRAND mitigates the accuracy drop-off with increasing depth and improves the overall accuracy of the model. We empirically corroborate the advantage of DeepGRAND over many existing graph neural networks on various graph deep learning benchmark tasks.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Submission Guidelines: Yes

Please Choose The Closest Area That Your Submission Falls Into: Deep Learning and representational learning