Enabling Faster Convergence in Distributed Irregular Graph Processing

Abbas Mazloumi, Rajiv Gupta

Abstract: Large scale iterative graph processing has become increasingly important due to its use in analyzing large realworld networks (e.g., internet topology, social networks). Due to their scalability, distributed systems are an attractive platform for graph processing. However, the irregular power-law degree distribution of such large graphs creates many challenges for efficiently managing iterative workloads on a distributed system. We observe that in current algorithms during each iteration an active vertex is processed exactly once. In this work we demonstrate that by processing less expensive low-degree vertices more frequently we can cause more expensive high-degree vertices to be processed less frequently leading to faster algorithm convergence. Our experiments demonstrate that both the number of iterations and total execution times are reduced significantly for multiple iterative graph algorithms (PageRank, NumPaths, SSSP) on billion vertex input graphs (Friendster and Twitter).

External IDs:dblp:conf/bigdataconf/MazloumiG19