Masked Auto-Encoder with Multiple Masks for Graph Representation Learning

Boyu Chen, Cheng Xie, Haoran Duan

Abstract: Graph representation is an indispensable technique in the field of E-Business Engineering, as it plays a pivotal role in capturing the underlying structure of various data types prevalent in this domain. Notably, a multitude of data, including but not limited to social networks, online shopping networks, and recommender systems, are inherently organized in a graph structure. Consequently, the utilization of graph representation methods becomes paramount in effectively analyzing and harnessing the potential of such data sources. Nowadays, Masked Auto-Encoder (MAE) uses masking operations to randomly hide some edges or node features, which can better capture the global structure of the graph and the relationships between nodes, and therefore have achieved great success. But in reality, information in graphs is always lost, and graphs are often heterogeneous, their complex graph structures are difficult to handle. There are two challenges for applying MAE on the heterogeneous graph: (1) How to improve the model's ability to learn structured features? (2) How to mask nodes on different semantic aspects? To this end, this work is an attempt to apply MAE to heterogeneous graph representation. For the first challenge, this work uses edge masking to improve generalization ability. For the second challenge, a meta-path based node masking mechanism to mask node features on multiple meta-paths. And then, a meta-path based graph auto-encoder is proposed to learn and aggregate multiple node features into heterogeneous graph representations. After, the representations are used for two downstream learning tasks, node classification, and node clustering, on four real-world datasets. Experimental results show that our method has achieved current state-of-the-art results in self-supervised graph representation. The most brilliant highlight of the proposed method is the simple but effective model structure.