Towards Expansive and Adaptive Hard Negative Mining: Graph Contrastive Learning via Subspace Preserving

Zhezheng Hao

Abstract:

Graph Neural Networks (GNNs) have emerged as the predominant approach for analyzing graph data on the web and beyond.
Contrastive learning (CL), a self-supervised paradigm, not only mitigates reliance on annotations but also has potential in performance.
The hard negative sampling strategy that benefits CL in other domains proves ineffective in the context of Graph Contrastive Learning (GCL) due to the message passing mechanism.
Embracing the subspace hypothesis in clustering, we propose a method towards expansive and adaptive hard negative mining, referred to as {G}raph cont{R}astive le{A}rning via subs{P}ace pr{E}serving ({GRAPE}).
Beyond homophily, we argue that false negatives are prevalent over an expansive range and exploring them confers benefits upon GCL.
Diverging from existing neighbor-based methods, our method seeks to mine long-range hard negatives throughout subspace, where message passing is conceived as interactions between subspaces.
%Empirical investigations back up this strategy.
Additionally, our method adaptively scales the hard negatives set through subspace preservation during training.
%The elastic net is imposed on the self-expression coefficients, where users can control the hardness of selecting negatives.
In practice, we develop two schemes to enhance GCL that are pluggable into existing GCL frameworks.
The underlying mechanisms are analyzed and the connections to related methods are investigated.
Comprehensive experiments demonstrate that
our method outperforms across diverse graph datasets and remains competitive across varied application scenarios\footnote{Our code is available at \url{https://github.com/zz-haooo/WWW24-GRAPE}.}.