Novel Bandwidth-Aware Network Coding for Fast Cloud-of-Clouds Disaster Backup

Jiajie Shen, Bochun Wu, Wang Xiang, Sai Zou, Wenyong Wang, Laizhong Cui, Wei Ni

Abstract: Cloud-of-clouds storage can enhance the data security and reliability of online applications by encrypting, encoding, and distributing user data across multiple clouds. Fast transferring large volumes of data through networks with limited bandwidths remains a practical challenge, especially in the event of disaster backup. To address this, we model a data storage process using an information flow graph and estimate inter-cloud traffic. We propose a new Network Coding-based Cloud-of-Clouds Backup (NC3B) framework, which enables collaborative encoding and data exchange among backup clouds to utilize inter-cloud bandwidth efficiently. We analytically corroborate that NC3B effectively reduces write operation latency. We also demonstrate the NC3B framework by incorporating two cutting-edge Reed-Solomon (RS) based data storage techniques, namely All-Or-Nothing Transform-RS (AONT-RS) and Converge AONT-RS (CAONT-RS), referred to as Network coding-based Backup AONT-RS (NBAONT-RS) and Network coding-based Backup CAONT-RS (NBCAONT-RS), respectively. To validate our approach, we deploy a real-world prototype storage system on Amazon EC2 using a cluster trace set, and underscore the effectiveness of NC3B, showcasing reductions in latency of up to 50% compared to state-of-the-art approaches, alongside throughput improvements of up to 98%. These findings underscore the benefits of NC3B in real-world storage scenarios.

External IDs:dblp:journals/tnsm/ShenWXZWCN25