Differentiable Cluster Graph Neural Network

Yanfei Dong, Mohammed Haroon Dupty, Lambert Deng, Zhuanghua Liu, Yong Liang Goh, Wee Sun Lee

Keywords: Graph Neural Networks, Graph Representation Learning, Node Classification

TL;DR: Differentiable clustering based message passing for supervised node classification

Abstract:

Graph Neural Networks often struggle with long-range information propagation and
local heterophilous neighborhood aggregation. Inspired by the observation that cluster patterns manifest at global and local levels, we propose to tackle both challenges with a unified framework that incorporates a clustering inductive bias into the message passing mechanism, using additional cluster-nodes.
Central to our approach is the formulation of an optimal transport based clustering objective.
However, optimizing this objective in a differentiable way is non-trivial.
To navigate this, we adopt an iterative process, alternating between solving for the cluster assignments and updating the node/cluster-node embeddings.
Notably, our derived optimization steps are themselves simple yet elegant message passing steps operating seamlessly on a bipartite graph of nodes and cluster-nodes.
Our clustering-based approach can effectively capture both local and global information,
demonstrated by extensive experiments on heterophilous and homophilous datasets.

Primary Area: learning on graphs and other geometries & topologies

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 5365