LSM-Subgraph: Log-Structured Merge-Subgraph for Temporal Graph Processing

Jingyuan Ma, Zhan Shi, Shang Liu, Wang Zhang, Yutong Wu, Fang Wang, Dan Feng

Abstract: Temporal graphs, with a time dimension, are attracting increasing interest from research communities. Existing temporal graph storage formats mainly include copy-based models, log-based models, and hybrid models that have emerged in recent years. Neither the copy-based model nor the log-based model can trade-off storage and query time well. Hybrid models try to find a compromise between the above two models, but existing models do not consider the skewness of vertex degree in temporal graphs is changing over time. Based on these considerations, we propose LSM-Subgraph, a hybrid storage format that only stores snapshots divided by the fluctuation-aware method and in-between logs. First, LSM-Subgraph uses a PMA-based snapshot creation model to store snapshots based on packed memory arrays (PMA), avoiding rebuilding the whole data structure. Second, LSM-Subgraph uses a select-timepoint method based on fluctuation-aware to divide shards during the update, which achieves a good tradeoff between storage overhead and query time cost. Extensive experimental evaluations over various real-world graphs illustrate that LSM-Subgraph outperforms state-of-the-art temporal graph systems in both memory and time consumption.