DiBB: Distributing Black-Box Optimization

Giuseppe Cuccu, Luca Sven Rolshoven, Fabien Vorpe, Philippe Cudre-Mauroux, Tobias Glasmachers

Keywords: Black Box Optimization, Distributed Computing, Evolutionary Computation

Abstract: We present a novel framework for Distributing Black-Box Optimization (DiBB). DiBB can encapsulate any Black Box Optimization (BBO) method, making it of particular interest for scaling and distributing modern Evolution Strategies (ES), such as CMA-ES and its variants, which maintain a sampling covariance matrix throughout the run. Due to high algorithmic complexity however, such methods are unsuitable alone to address high-dimensional problems, e.g. for sophisticated Reinforcement Learning (RL) control. This limits the applicable methods to simpler ES, which trade off faster updates for lowered sample efficiency. DiBB overcomes this limitation by means of problem decomposition, leveraging expert knowledge in the problem structure such as a known topology for a neural network controller. This allows to distribute the workload across an arbitrary number of nodes in a cluster, while maintaining the feasibility of second order (covariance) learning on high-dimensional problems. The computational complexity per node is bounded by the (arbitrary) size of blocks of variables, which is independent of the problem size.

One-sentence Summary: DiBB creates partially-separable versions of any BBO algorithm, and provides a distributed implementation for it, allowing scaling *constantly* to arbitrarily high dimensions based on the number of available machines.