Towards Expansive and Adaptive Hard Negative Mining: Graph Contrastive Learning via Subspace Preserving

Zhezheng Hao, Haonan Xin, Wei Long, Liaoyuan Tang, Rong Wang, Feiping Nie

Keywords: Graph neural networks, Graph contrastive learning, Hard negative mining, Subspace preserving, Web data mining

Abstract: Graph Neural Networks (GNNs) have emerged as the predominant tool for analyzing graph data on the web and beyond. Contrastive learning (CL), a self-supervised paradigm, not only mitigates the reliance on annotations but also leads to breakthroughs in performance. The hard negative sampling strategy that benefits CL in other domains proves ineffective in the context of Graph Contrastive Learning (GCL) due to the message passing mechanism. Embracing the subspace hypothesis in clustering, we propose a method towards expansive and adaptive hard negative mining, referred to as Graph contRastive leArning via subsPace prEserving (GRAPE). Beyond homophily, we argue that false negatives are prevalent over an expansive range and exploring them confers benefits upon GCL. Diverging from existing neighbor-based methods, our method seeks to mine long-range hard negatives throughout subspace, where message passing is conceived as interactions between subspaces. Additionally, our method adaptively scales the hard negatives set through subspace preservation during training. In practice, we develop two schemes to enhance GCL that are pluggable into existing GCL frameworks. The underlying mechanisms are analyzed and the connections to related methods are investigated. Comprehensive experiments demonstrate that our method achieves state-of-the-art performance on multiple graph datasets and maintains competitiveness in various application settings. Our work contributes to the improvement of representation learning on web graphs, aligning with the scope of The Web Conference. Our code is available at https://anonymous.4open.science/r/Grape-code.

Track: Graph Algorithms and Learning for the Web

Submission Guidelines Scope: Yes

Submission Guidelines Blind: Yes

Submission Guidelines Format: Yes

Submission Guidelines Limit: Yes

Submission Guidelines Authorship: Yes

Student Author: Yes

Submission Number: 99