Differentiable Clustering for Graph Attention

Haicang Zhou, Tiantian He, Yew-Soon Ong, Gao Cong, Quan Chen

Abstract: Graph clusters (or communities) represent important graph structural information. In this paper, we present Differentiable Clustering for graph ATtention (DCAT). To the best of our knowledge, DCAT is the first solution that incorporates graph clustering into graph attention networks (GAT) to learn cluster-aware attention scores for semi-supervised learning tasks. In DCAT, we propose a novel approach to formulating graph clustering as an auxiliary differentiable objective based on modularity maximization, which can be optimized together with the learning objective of GAT for a semi-supervised task. Specifically, we propose a solution to relaxing modularity maximization from a discrete optimization problem to a differentiable objective with theoretical guarantee so that we can learn cluster-aware attention scores by jointly learning from graph clustering and a semi-supervised learning task. To address the computational challenge, we further propose to reformulate the constraint introduced by the clustering objective into a new form. Our analysis shows that DCAT allocates higher attention scores to nodes within the same cluster, allowing them to have a higher influence in node representation learning, and thus DCAT will generate better node representations for downstream applications. The experimental results on commonly used datasets show that DCAT outperforms popular and state-of-the-art graph neural networks.

External IDs:dblp:journals/tkde/ZhouHOCC24