Dynamic Embedding-based Methods for Link Prediction in Machine Learning Semantic Network

Harlin Lee, Rishi Sonthalia, Jacob G. Foster

Abstract: This paper aims to accelerate scientific discovery by studying link prediction in a semantic network. The nodes are unidentified concepts in machine learning, and the time-stamped edges indicate co-occurrence in scientific papers. Taking advantage of this temporal information, we perform node embedding on the graph at every year from 1994 to 2017, and apply two methods to find features for node pairs: the first method uses a transformer, while the other uses distance metrics combined with known link prediction features. The latter feature extraction technique with a 3-layer multi-layer perceptron achieved an AUC of 0.902 on predicting edges in the 2020 graph. Inspection of the resulting features suggests that the model does indeed pay attention to the dynamic nature of the features, e.g., how node-pair distance in embedding space changes over the years.