STEM: Streaming-Based FPGA Acceleration for Large-Scale Compactions in LSM KV

Dongdong Tang, Weilan Wang, Yu Mao, Jinghuan Yu, Tei-Wei Kuo, Chun Jason Xue

Abstract: Log-Structured-Merge-tree (LSM-tree) has been extensively adopted because of its exceptional write efficiency and high space utilization. Compaction is invoked periodically in LSM-tree based key-value(LSM KV) systems to maintain good system performance. As the size of LSM-KV grows, large-scale compaction is now frequently seen. Compaction throughput significantly degrades with larger inputs, leading to frequent write stalls and decrement in overall write throughput. This paper proposes STEM, a stream-based compaction framework with FPGA to address this issue. A clean-cut algorithm is introduced to enable streaming-based compaction for large-scale data. With a multi-unit pipeline and dynamic pipeline schedule, STEM can handle large-scale compaction tasks efficiently. Based on the experiment result, the compaction throughput of STEM can achieve $27\times$ on average and up to $35\times$ improvement compared with the current RocksDB compaction, $2.09\times$ to $2.27\times$ improvement compared with the state-of-the-art FPGA accelerator.