Speed-up Implicit Graph Neural Diffusion Model: A Simplified and Robust Strategy

Pengqing Shi, Junbin Gao

Abstract: Certain graph neural networks (GNNs) have the capability to glean insights from distant connections while preventing over-smoothing through the use of an implicit approach. Rather than directly computing node representations, these GNNs identify them as solutions to meaningful optimization problems. However, these GNNs depend on the implicit function theorem to derive solutions from the stationary conditions of optimization problems, necessitating repeated attempts with root-finding algorithms. This renders the training of such GNNs computationally expensive on graphs. This paper aims to develop fast and resilient learning algorithms that can speed up the recent implicit graph neural diffusion model based on Dirichlet energy minimization (DIGNN). Our proposed models, named F-DIGNN and L 1 -DIGNN respectively, demonstrated substantial improvements in training efficiency and robustness through extensive experiments. They consistently achieved comparable or superior performance in node classification tasks across various scenarios.