Optimizing encoding and repair for wide-stripe minimum bandwidth regenerating codes in in-memory key-value stores

Xuzhe Liu, Yuchong Hu, Weichun Wang, Dan Feng, Hai Zhou

Abstract: In-memory key–value (KV) stores are essential for databases and large-scale websites. While recent studies deploy wide-stripe erasure coding in such systems to ensure data reliability and achieve extreme storage savings, they also introduce a repair penalty. A class of erasure codes, Minimum Bandwidth Regenerating (MBR) codes, offers optimal single-chunk repair bandwidth. However, deploying wide-stripe MBR codes in this context results in two types of additional traffic: (i) encoding traffic incurred by transmitting large amounts of raw data between nodes; (ii) repair traffic from retrieving unnecessary data to repair failed data.This paper proposes MBRWide to optimize encoding and repair performance for wide-stripe MBR codes in in-memory KV stores. MBRWide includes an all-node cooperative encoding scheme (ACES) and a fragmented repair scheme (FRS). ACES selectively encodes raw chunks to reduce encoding traffic. FRS aims to enhance repair efficiency by dynamically fragmenting parity chunks during encoding. This study implements MBRWide in Memcached, a foundational component in real-world in-memory KV services. Experiments show that ACES improves encoding throughput by 16.02% to 72.92% compared to traditional encoding methods. FRS reduces degraded read latency to failed data and multiple failures repair latency by up to 34.19% and 44.89%, respectively.