MuDSE: GA-ILP-based Framework for Automated Deployment of Multiple DNNs on Heterogeneous Mixed-Criticality Systems

Alexander Hoffman, Ala Fnayou, Fedor Smirnov, Daniel Mueller-Gritschneder, Ulf Schlichtmann

Abstract: In modern edge computing, deploying multi-deep neural network (DNN) applications is essential for addressing complex tasks such as visual classification, object tracking, and navigation. The intricate nature of these Machine Learning (ML) applications, coupled with their soft and hard real-time performance requirements, underscores the necessity for automated optimisation of both scheduling and mapping configurations on computationally robust heterogeneous embedded systems. This paper introduces a novel approach for the automated mapping and scheduling of multiple DNNs onto heterogeneous hardware platforms. Our methodology leverages an integer linear program (ILP) scheduler formulation that accommodates soft and hard real-time constraints. This is complemented by a mapping generation process that employs (a) an advanced ILP formulation and (b) a Genetic Algorithm (GA) designed to identify optimised solutions for large-scale mappings. The GA is mainly utilised when the expansive design space renders the ILP formulation impractical in terms of computational solving time. We rigorously test and evaluate our framework using scaling input model configurations and a real-world mixed-model scenario. The results demonstrate that our hybrid optimisation solution, which integrates Prepositional Satisfiability Problem (SAT) decoding, the NSGA-II GA, and ILP, significantly enhances scalability. This improvement is vital for efficiently deploying complex systems, marking a substantial advancement in the embedded ML field.