Memory-Augmented Graph Neural Networks: A Brain-Inspired Review

Guixiang Ma, Vy A. Vo, Theodore L. Willke, Nesreen K. Ahmed

Abstract: Graph neural networks (GNNs) have been extensively used for many domains where data are represented as graphs, including social networks, recommender systems, biology, chemistry, etc. Despite promising empirical results achieved by GNNs for many applications, there are some limitations of GNNs that hinder their performance on some tasks. For example, since GNNs update node features mainly based on local information, they have limited expressive power for capturing long-range dependencies between nodes. To address some of these limitations, several recent works have started to explore augmenting GNNs with memory to improve their performance and expressivity. We provide a comprehensive review of the existing literature on memory-augmented GNNs. We review these works through the lens of psychology and neuroscience, which has several established theories on how multiple memory systems and mechanisms operate in biological brains. We propose a taxonomy of memory-augmented GNNs and a set of criteria for comparing their memory mechanisms. We also provide critical discussions on the limitations of these works. Finally, we discuss the challenges and future directions for this area.