Parameter Efficient Node Classification on Homophilic Graphs

Lucas Prieto, Jeroen Den Boef, Paul Groth, Joran Cornelisse

Abstract: Deep Learning on Graphs was recently made possible with the introduction of Graph Neural Networks (GNNs). GNNs use learnable diffusion processes to propagate information through the graph and improve performance on downstream tasks. However, learning this diffusion process can be expensive in terms of memory and computation. While a lot of research has gone into making these models more expressive and able to capture more complex patterns, in practice, edges in common benchmarking datasets often encode similarity of nodes with respect to the downstream task. This property is called homophily. We argue that for these homophilic graphs, learnable diffusion processes and large receptive fields are not required to achieve competitive performance. We propose Graph Non-Parametric Diffusion (GNPD) a method that outperforms traditional GNNs using only 2 linear models and non-parameteric diffusion. Our method takes ideas from Correct & Smooth (C&S) and the Scalable Inception Graph Network (SIGN) and combines them to create a simpler model that outperforms both of them on several datasets. Our method achieves an unmatched parameter efficiency, competing with models with two orders of magnitude more parameters. Additionally GNPD can also forego spectral embeddings which are the computational bottleneck of the C&S method.

Submission Length: Regular submission (no more than 12 pages of main content)

Changes Since Last Submission:

We addressed the Action Editor's comments in the following ways:
1) Removed "sufficiency" from our third claim as we agree that statement was inaccurate.
2) Removed mentions of "scalability" in the paper to avoid the emphasis on scalability.
3) Added a limitations subsection.

Code: https://github.com/LucasPrietoAl/GNPD

Assigned Action Editor: ~Yujia_Li1

License: Creative Commons Attribution 4.0 International (CC BY 4.0)

Submission Number: 640