Distributed Construction of Demand-Aware Datacenter Networks

Aleksander Figiel, Darya Melnyk, Tijana Milentijević, Stefan Schmid

Abstract:

Demand-aware reconfigurable datacenter networks adapt toward the traffic they serve by providing topological shortcuts between frequently communicating racks. However, only little is known about computing optimized demand-aware networks quickly and in a distributed manner. In this paper, we investigate fast distributed algorithms to compute demand-aware networks for hybrid datacenters, where a fixed capacitated network can be enhanced with a bounded-degree demand-aware network, i.e., with a set of matchings created by optical circuit switches.

We make two main contributions. Firstly, we present a distributed algorithm, called the Coordinator algorithm for computing demand-aware networks on all underlying topologies. The algorithm is analyzed in the widely deployed Clos topology and in the Congested Clique model, where it is optimal in terms of quality and nearly optimal in distributed runtime.

Secondly, we focus on improving the round complexity at the cost of the quality of the resulting topology. We show that for tree demands, an adaptation of a distributed matching algorithm by Wattenhofer and Wattenhofer (DISC 2004) achieves a $1/6$-approximation. Based on this approach, we introduce the Propose & Reject algorithm for general demands, which we evaluate on real-world Facebook datacenter and HPC traces. Our results show that the Propose & Reject algorithm, even with limited knowledge of the demand matrix, performs nearly optimally on real traffic demands and covers over 80% of the demand. This is achieved with significantly fewer communication rounds than the optimal solution computed by the Coordinator algorithm.