Advancing Latent Representation Ranking for Masked Graph Autoencoder

Yulan Hu, Ge Chen, Sheng Ouyang, Zhirui Yang, Junchen Wan, Fuzheng Zhang, Zhongyuan Wang, Zhao Cao, Shangquan Wu, Yong Liu

Abstract: Generative graph self-supervised learning (SSL), represented by graph autoencoders (GAEs), has increasingly garnered interest and demonstrated its potential in graph representation. However, existing generative SSL methods primarily concentrate on refining model designs, such as masking strategies and reconstruction objectives, while the latent representation generated by the encoder is scarcely explored beyond decoding by the decoder. Motivated by this observation, we identify and analyze the issues faced by previous works in utilizing latent representation and introduce GMAERank, a masked autoencoder with ranking learning. GMAERank presents a novel ranking strategy that leverages the latent representation. Specifically, GMAERank posits that the similarity between two augmented latent views should consistently outrank that between any augmented view and an unrelated random view-a condition typically achievable. Subsequently, GMAERank focus on the optimization of more difficult samples with the ranking strategy. In contrast to earlier research that involves auxiliary models for optimization, GMAERank is simple and lightweight. We conduct comprehensive experiments across node, edge, and graph-level tasks, and the results reveal that GMAERank achieves competitive performance and retains versatility when compared to state-of-the-art (SOTA) baselines.