Echo-GAT: Debiasing Graph Attention with Echo Nodes and Degree Diversity for Heterophilic Graphs

TMLR Paper6954 Authors

Abstract: Attention mechanisms have become a de facto standard for enhancing the expressivity of deep learning models, achieving remarkable success in graph data. Recent studies have shown that attention-based graph neural networks (GNNs) often perform poorly on heterophilic graphs and have attributed this degradation primarily to low levels of homophily. In contrast to this prevailing explanation, we find that on heterophilic graphs, under standard graph attention mechanisms, node-level homophily shows only a weak correlation with prediction accuracy, and nodes with lower homophily ratios can even achieve higher accuracy on average. These observations suggest that homophily alone is insufficient to explain the failure of graph attention. In this work, we show that standard graph attention networks exhibit a systematic performance imbalance across nodes with different degrees of diversity, favoring structurally inhomogeneous nodes (i.e., those with significantly divergent degrees compared to their neighbors). To mitigate this bias, we propose a graph attention optimization framework that integrates augmented feature attention and degree diversity-aware attention score to mitigate node-level structural bias. Experiments show that the proposed method consistently outperforms strong GAT variants and state-of-the-art heterophily-oriented GNNs. Moreover, it maintains stable performance gains across nodes with varying heterophily levels, demonstrating its effectiveness on diverse graph structures.

Submission Type: Regular submission (no more than 12 pages of main content)

Changes Since Last Submission: Since the last TMLR submission, we have added node classification experiments on three datasets: Minesweeper (Table 1), Computers, and Photo (both in Table 2). In addition, other textual modifications and clarifications have been made throughout the manuscript, and all such changes are highlighted in blue.

Assigned Action Editor: ~Jian_Kang1

Submission Number: 6954