An Efficient Distributed Graph Engine for Deep Learning on Graphs

Gangda Deng, Ömer Faruk Akgül, Hongkuan Zhou, Hanqing Zeng, Yinglong Xia, Jianbo Li, Viktor K. Prasanna

Abstract: Traditional graph-processing algorithms have been widely used in Graph Neural Networks (GNNs). This combination has shown state-of-the-art performance in many real-world network mining tasks. Current approaches to graph processing in deep learning face two main problems. On the one hand, easy-to-use deep learning libraries lack support for widely used graph-processing algorithms and do not provide low-level APIs for building distributed graph-processing algorithms. On the other hand, existing graph-processing libraries are not user-friendly for deep learning researchers. Their graph primitives are not designed for batch processing, which is essential for deep learning use cases. In this paper, we present an efficient and easy-to-use graph engine that incorporates distributed graph processing into deep learning ecosystems. We develop a distributed graph storage system with an efficient batching technique to minimize communication overhead incurred by Remote Procedure Calls (RPC) between computing nodes. We propose an optimized method for distributed computation of Single Source Personalized PageRank (SSPPR) using the Forward Push algorithm based on lock-free parallel maps. Experimental evaluations demonstrate significant improvement, with up to three orders of magnitude in SSPPR throughput, of our graph engine compared with the tensor-based implementation. Both methods offer necessary usability with tensor operations, which are widely used for graph processing in current deep graph libraries.