Understanding Data Compression in Warehouse-Scale Datacenter Services

Geonhwa Jeong, Bikash Sharma, Nick Terrell, Abhishek Dhanotia, Zhiwei Zhao, Niket Agarwal, Arun Kejariwal, Tushar Krishna

Abstract: Data compression has emerged as a promising technique to alleviate the memory, storage, and network cost with some associated compute overheads in warehouse-scale datacenter services. Despite being one of the most important components of the overall datacenter taxes, there has not been a comprehensive characterization of compression usage in data center workloads. In this work, we first provide a holistic characterization of compression as used by various warehouse-scale datacenter services at a global social media provider (Meta). Next, we deep dive into a few representative use cases of compression in the production environment and characterize compression usage of services while running live traffic.