Elaborating the Bayesian Priors in Unsupervised Graph Embedding via Graph Concepts

Xiaojun Ma, Ziyao Li, Siwei Wei, Guojie Song

Abstract: Unsupervised Graph Embedding yields specific importance because it performs well with inputs limited to the graph structure only. Proximity-preserving models, including link-preserving and Skip-Gram models, prove to be good approaches in both efficiency and accuracy on unsupervised tasks, even compared with state-of-the-art deep models. We first show that the optimization problem these models solve is equivalent to a Bayesian Inference problem, however, these models generally assume a uniform distribution for the target node representations, that is, the representations of nodes are not further constrained. In our paper, we elaborate this Bayesian prior resorting to potential concepts underlying a graph. These graph concepts can be communities in a graph, nodes with different interaction patterns et al. We further derive the optimization objective according to this elaborated prior, and proposed our learning objective. Intuitively, graph nodes of the same concept are embedded close to each other. Our paper proposes a flexible framework which is adaptable to any other proximity-based models. Experiments show that our model significantly elevates the baseline performances of proximity-preserving models, yielding state-of-the-art results on unsupervised learning tasks.