HMT: A Hybrid Mitigating and Transferring Approach on I/O Throughput Degradation for Erasure Coded Storage Systems

Open Webpage

Piao Hu, Huangzhen Xue, Chentao Wu, Jie Li, Minyi Guo

Published: 12 Aug 2024, Last Modified: 08 Nov 2025CrossrefEveryoneRevisionsCC BY-SA 4.0
Abstract: In cloud storage systems with erasure coding (EC), increased demand for data services and EC-based data recovery lead to high volumes of concurrent I/O requests, potentially causing network congestion or server overload. Network congestion or node overload significantly reduces I/O throughput and data parallelism. Various methods have been proposed to address these issues, such as fine-grained data packet partitioning, I/O scheduling, and transfer reading. However, these methods may not be effective in different scenarios. For instance, even if most I/O paths are relieved, data may still remain inaccessible.To address this problem, we propose a novel hybrid approach called HMT, which combines mitigation and transfer techniques aimed at maintaining high I/O throughput for cloud storage systems. The key idea behind HMT is to pick an appropriate method with low overhead. Mitigating-based methods are used when light node load or congestion causes minor I/O throughput declines. Conversely, transfer-based methods are preferred when heavy load or congestion leads to significant I/O throughput reductions. HMT also provides a transformation strategy between mitigation and transfer-based techniques. To demonstrate the effectiveness of our approach, we conducted several experiments. The experimental results demonstrate that, in scenarios with dynamically changing loads, HMT can reduce the time to read <Formula format="inline"><TexMath><?TeX $95\%$?></TexMath><AltText>Math 1</AltText><File name="icpp24-14-inline1" type="svg"/></Formula> of files and the average response time by up to <Formula format="inline"><TexMath><?TeX $49.02\%$?></TexMath><AltText>Math 2</AltText><File name="icpp24-14-inline2" type="svg"/></Formula> and <Formula format="inline"><TexMath><?TeX $50.53\%$?></TexMath><AltText>Math 3</AltText><File name="icpp24-14-inline3" type="svg"/></Formula>, respectively, compared to state-of-the-art methods. Meanwhile, it also increases throughput by up to 1.70 × and optimizes load balancing by up to <Formula format="inline"><TexMath><?TeX $36.63\%$?></TexMath><AltText>Math 4</AltText><File name="icpp24-14-inline4" type="svg"/></Formula>.