Dynamic Graph Representation Learning via Self-Attention Networks

Aravind Sankar, Yanhong Wu, Liang Gou, Wei Zhang, Hao Yang

Abstract: Learning latent representations of nodes in graphs is an important and ubiquitous task with widespread applications such as link prediction, node classification, and graph visualization. Previous methods on graph representation learning mainly focus on static graphs, however, many real-world graphs are dynamic and evolve over time. In this paper, we present Dynamic Self-Attention Network (DySAT), a novel neural architecture that operates on dynamic graphs and learns node representations that capture both structural properties and temporal evolutionary patterns. Specifically, DySAT computes node representations by jointly employing self-attention layers along two dimensions: structural neighborhood and temporal dynamics. We conduct link prediction experiments on two classes of graphs: communication networks and bipartite rating networks. Our experimental results show that DySAT has a significant performance gain over several different state-of-the-art graph embedding baselines.

Keywords: Graph Representation Learning, Dynamic Graphs, Attention, Self-Attention, Deep Learning

TL;DR: A novel neural architecture named DySAT to learn node representations on dynamic graphs by employing self-attention along two dimensions: structural neighborhood and temporal dynamics, achieves state-of-the-art results in dynamic link prediction.

Community Implementations: [ 2 code implementations](https://www.catalyzex.com/paper/arxiv:1812.09430/code)