Graph Transformer-based Heterogeneous Graph Neural Networks enhanced by multiple meta-path adjacency matrices decomposition

Shibin Li, Jun Gong, Shengnan Ke, Shengjun Tang

Abstract: Heterogeneous graph neural networks (HGNNs) are adept at processing data within multi-relational heterogeneous networks. Nonetheless, contemporary HGNNs that are relation-aware or utilize meta-path-based architectures struggle with recognizing long-range dependencies. Moreover, an over-extension in model depth frequently results in over-smoothing. To address these challenges, this paper proposes the Multiple meta-path Structural representation-aware Heterogeneous Graph Transformer (MSHGT). Firstly, the model constructs a latent fully-connected attention graph using node features, allowing all nodes to aggregate information from neighbor nodes of arbitrary hops through weighted aggregation via the attention graph. This endows the model with global perception and the ability to capture long-range dependency information, mitigating the issue of over-smoothing in node representations. Furthermore, to enhance MSHGT’s structure perception capability and further bolster its ability to capture heterogeneous connectivity patterns, a structural representation encoding method based on multiple meta-path adjacency matrices decomposition is proposed. This method processes adjacency matrices of different meta-path categories through singular value decomposition, extracting structure correlation information between nodes from various semantic perspectives and encapsulating it into structural representations that span multiple semantic dimensions. Subsequently, these structural representations and node attribute features are simultaneously integrated into the Transformer module, prompting the model to perform message-passing from both attribute and structure dimensions. Comprehensive experiments conducted on six heterogeneous graph datasets, covering node classification and link prediction tasks, demonstrate that MSHGT outperforms baseline methods significantly, confirming its efficacy in addressing the complexities of heterogeneous networks.