Resource allocation for data-parallel computing in networks with data locality

Weina Wang, Lei Ying

Abstract: This paper studies resource allocation for data-parallel applications in a networked computing system with data locality. Data-parallel computing tasks have two components: data and computation. To support efficient data-processing in the system, the resource allocation algorithm should jointly consider load-balancing, data transmissions and processing scheduling. In this paper, we consider a general model of a computing system where the computing system is a network represented by a graph, with nodes being computing devices or switches and edges being communication links. The data chunks stored at the nodes can be processed locally or be transmitted to other computing nodes in the network to be processed. The throughput of such a system for processing data-parallel applications is determined jointly by the computing capacity of the nodes and the communication capacity of the network, and the resource allocation algorithm should strike a right balance between computing and communication. In this paper, we propose a throughput-optimal resource allocation algorithm, which is also able to control the tradeoff between the expected amount of data transmitted and the expected number of backlogged tasks in steady state through a parameter q <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">th</sub> . We show that the gap between the expected data transmission rate under our algorithm and the optimal value is O(1/q <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">th</sub> ), while the expected number of total backlogged tasks is upper bounded by O(q <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">th</sub> ).