Efficient Redundancy-Free Graph Networks: Higher Expressiveness and Less Over-Squashing

Rongqin Chen, Yan Li, Leong Hou U, Ye Li

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: graph neural network, redundancy-free message passing, expressiveness, over-squashing

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

TL;DR: This study introduces an efficient and redundancy-free message passing framework for GNNs, aiming to enhance expressiveness and mitigate over-squashing.

Abstract: Message Passing Neural Networks (MPNNs) effectively learn graph structures. However, their message passing mechanism introduces redundancy, limiting expressiveness, and causing over-squashing. Prior research has addressed the problem of redundancy but often at the cost of increased complexity. Improving expressiveness and addressing over-squashing remain major concerns in MPNN research with significant room for improvement. This study explores the nature of message passing redundancy and presents efficient solutions through two surrogate structures: Directed Line Graph (DLG) and Directed Acyclic Line Graph (DALG). The surogate structures introduce two corresponding models, Directed Line Graph Network (DLGN) and Efficient Redundancy-Free Graph Network (ERFGN). DLGN, utilizing DLGs, achieves redundancy-free message passing for graphs with a minimum cycle size of \(L\) when composed of $L$ layers. ERFGN, on the other hand, leverages DALGs to achieve fully redundancy-free message passing and possesses the expressiveness to distinguish arbitrary graphs under certain conditions. Furthermore, we enhance the expressiveness of ERFGN by incorporating cycle modeling and global attention, thereby achieving higher-order expressiveness. The efficiency and efficacy of these models in improving expressiveness and mitigating over-squashing are analysed theoretically. Empirical results on realistic datasets validate the proposed methods.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 2910