Addressing Node Integration Skewness in Graph Neural Networks Using Hop-Wise Attention

TMLR Paper4763 Authors

Abstract:

Graph neural networks (GNNs) often suffer performance degradation as their layer count grows, typically due to the well-known problems of over-smoothing and over-squashing. In this work, we identify an additional factor contributing to this degradation, which we term the K-skewed-traversal problem: certain hop distances are disproportionately emphasized during aggregation, with this emphasis intensifying as the number of layers grows. To address this, we introduce an algorithm called Hop-wise Graph Attention Network (HGAT) that ensures uniform aggregation across hops to eliminate the K-skewed-traversal problem, and employs a hop-wise attention mechanism to adaptively prioritize specific hop distances. We theoretically prove that HGAT removes this skewness by balancing contributions from different hop distances before applying hop-wise attention. Moreover, in our extensive empirical evaluation$^*$, we observe notable improvement in terms of solution quality compared to the state-of-the-art GNN models, particularly as the number of layers increases.

* The implementation is available at https://drive.proton.me/urls/XSGJ8SJJGW#0bIyDVkZDqTi

Submission Length: Long submission (more than 12 pages of main content)

Changes Since Last Submission:

# Summary of Changes Made to the Paper Since Last Submission
1. We have added the definition of $H(k, v)$ in the revised manuscript. (2nd para. of Section 2.2.1, page 4).
2. To complement the plots in Figure 9, we provide full numerical accuracy values for each dataset and method across different layer depths ($K$) in Tables 8, 9, 10 of Appendix B.

Assigned Action Editor: ~Chuxu_Zhang2

Submission Number: 4763