Position: Graph Learning May Have Been Misled By Over-smoothing And Over-squashing

Niklas Kormann, Benjamin Doerr, Johannes F. Lutzeyer

Keywords: Graph Representation Learning, Graph Neural Networks, Over-smoothing, Over-squashing

TL;DR: We question the practical impact of over-smoothing and over-squashing and call for statistics for measuring localisation and factorisation of label information to guide future research in graph learning.

Abstract: The prevalent focus in graph learning research on theoretical challenges like over-smoothing and over-squashing may be misguiding, as their practical relevance in real-world scenarios is questionable. While Graph Neural Networks (GNNs) have achieved significant success across various applications, theoretical work has extensively discussed issues such as over-smoothing and over-squashing for the past eight years. This paper argues that the continued emphasis on these problems might be misplaced. For node-level tasks, we suggest that performance decreases often stem from uninformative receptive fields rather than over-smoothing, as optimal model depths remain small even with mitigation techniques. For graph-level tasks, over-smoothing can even be beneficial if the smoothed state is label-informative. Similarly, we challenge the notion that over-squashing, i.e., the compression of exponentially growing information into fixed-size node embeddings, is always detrimental in practical applications. We argue that the distribution of relevant information over the graph frequently factorises and is often localised within a small $k$-hop neighbourhood, questioning the necessity of jointly observing entire receptive fields or engaging in an extensive search for long-range interactions. Our empirical findings demonstrate that while methods exist to mitigate over-smoothing and over-squashing, they often do not yield significant performance gains with increased model depth on standard benchmarks, and can lead to substantial computational costs. In this position paper, we advocate for a paradigm shift in theoretical research, urging a diligent analysis of future learning tasks and datasets to better understand the localisation and factorisation of label-relevant information. This will ensure that theoretical advancements align with the real needs and challenges of real-world graph learning problems.

Submission Number: 747