A Distributed Modular Scalable and Generic Framework for Parallelizing Population-Based Metaheuristics

Hatem Khalloof, Phil Ostheimer, Wilfried Jakob, Shadi Shahoud, Clemens Düpmeier, Veit Hagenmeyer

Abstract: In the present paper, microservices, container virtualization and the publish/subscribe messaging paradigm are exploited to develop a distributed, modular, scalable and generic framework for parallelizing population-based metaheuristics. The proposed approach paves the way for an easy deployment of existing metaheuristic algorithms such as Evolutionary Algorithms (EAs) on a scalable runtime environment with full runtime automation. Furthermore, it introduces simple mechanisms to work efficiently with other components like forecasting frameworks and simulators. In order to analyze the feasibility of the design, the EA GLEAM (General Learning Evolutionary Algorithm and Method) is integrated and deployed on a cluster with 4 nodes and 128 cores for benchmarking. The overhead of the framework is measured and the obtained results show not only low values but also a small increase with growing number of computing nodes.