OStoreBench: Benchmarking Distributed Object Storage Systems Using Real-World Application Scenarios

Guoxin Kang, Defei Kong, Lei Wang, Jianfeng Zhan

Abstract: Their hierarchical organization and metadata management limit the traditional file storage systems’ scalability. The industry widely uses Distributed Object Storage Systems because they keep the advantages of traditional file storage systems, e.g., data sharing, and alleviate the scalability issue through benefiting from the flat namespaces and integrating the meta-data in the object. However, evaluating and comparing distributed object storage systems remains a significant challenge. The existing DOSS benchmarks provide simple read/write operations without considering the complex workload characteristics in request arrival patterns and request size distribution. This paper presents an Object Storage Benchmark suite, named OStoreBench, which characterizes critical paths of three real-world application scenarios, including online service, big data analysis, and file backup. We evaluate three state-of-the-practice object storage systems using OStoreBench, including Ceph, Openstack Swift, and Haystack. The benchmark suite is publicly available from https://github.com/EVERYGO111/OStoreBench.