Type-adaptive graph Transformer for heterogeneous information networks

Yuxin Tang, Yanzhe Huang, Jingyi Hou, Zhijie Liu

Abstract: Many real-world applications use diverse types of nodes and edges to retain rich semantic information. These applications are modeled as heterogeneous graphs. Recent research on heterogeneous graph embedding has made great progress because of the powerful ability of graph neural networks (GNNs) to capture the structural information of graphs. However, the performance of existing heterogeneous graph neural networks (HGNNs) is still unsatisfactory because 1) the aggregation and update functions of GNNs do not exploit the types of nodes and edges, which provide task-relevant information in heterogeneous information networks (HINs), and 2) message-passing-based GNNs are limited by oversmoothing and oversquashing, which prevents the central node from obtaining information from its higher-order neighbors. In this paper, we propose a type-adaptive graph Transformer (Tagformer) that considers not only local structure information and higher-order neighbor information in HINs but also type information to improve performance across various downstream tasks. Specifically, Tagformer assigns each node with the corresponding type feature and uses a GNN and graph Transformer (GT) to extract local structure information and higher-order neighbor information, respectively. Furthermore, to reduce the quadratic complexity and eliminate irrelevant information, we design an intraclass pooling module to condense the large-scale nodes of a graph into a reduced set of pooling nodes. We conduct extensive experiments on four HIN benchmark datasets, demonstrating that Tagformer consistently outperforms state-of-the-art methods.