Spectral Graph Pruning Against Over-Squashing and Over-Smoothing
Keywords: graph neural networks, rewiring, spectral gap optimization, over-smoothing, over-squashing, lottery tickets
TL;DR: By deleting edges of a graph that maximize the spectral gap, we jointly address over-smoothing and over-squashing in GNNs.
Abstract: Message Passing Graph Neural Networks are known to suffer from two problems that are sometimes believed to be diametrically opposed: over-squashing and over-smoothing. The former results from topological bottlenecks that hamper the information flow from distant nodes and are mitigated by spectral gap maximization, primarily, by means of edge additions. However, such additions often promote over-smoothing that renders nodes of different classes less distinguishable. Inspired by the Braess phenomenon, we argue that deleting edges can address over-squashing and over-smoothing simultaneously. This insight explains how edge deletions can improve generalization, thus connecting spectral gap optimization to a seemingly disconnected objective of reducing computational resources by pruning graphs for lottery tickets. To this end, we propose a computationally effective spectral gap optimization framework to add or delete edges and demonstrate its effectiveness on the long range graph benchmark and on larger heterophilous datasets.
Primary Area: Graph neural networks
Submission Number: 4465
Loading