Relay Selection and Load Allocation for LT Coded Distributed Computing in Two- Hop Heterogeneous Computation Network

Kai Qin, Shushi Gu, Zhikai Zhang, Qinyu Zhang, Wei Xiang

Abstract: Coding techniques, known as coded distributed computing (CDC), have been investigated to alleviate the impact of heterogeneous straggler effects and reduce computation latency in distributed computing systems. However, in the multi-hop complicated network topology, the execution latency of the master's task includes both the computation latency and the communication latency, in which the imbalance computation loads and inadequate path selection will lead to the greater straggler effects extremely. In this paper, we focus on the issues of CDC application in a Two-Hop Heterogeneous Computation Network (THHCN). To make full use of the completed computing results of workers, we deploy Luby transform (LT) code to derive a total execution latency expression. Then, we formulate an optimization problem to minimize the total execution latency by selecting the relays and allocating the computation loads for different workers. Furthermore, a greedy minimum penalty relay selection and load allocation (GMPRS-LA) algorithm is proposed with lower complexity compared to the exhaustive searching to solve the integer nonlinear programming problem. Simulation results demonstrate GMPRS-LA achieves a significant reduction in the total execution latency and leads to a better performance than traditional CDC load allocation algorithms.

External IDs:dblp:conf/wcnc/QinGZ0024