Combinatorial Reinforcement Learning Based Scheduling for DNN Execution on Edge

Qiwei Yuan, Jiaqi Yin, CUNXI YU

Keywords: Reinforcement Learning, Combinatorial Optimization, Edge Computing

Abstract: The past half-decade has seen unprecedented growth in machine learning with deep neural networks (DNNs) that represent state-of-the-art in many real-world applications. However, DNNs have substantial computational and memory requirements, in which the compilation of its computational graphs has great impact in resource-constrained (e.g., computation, I/O, and memory bounded) edge computing systems. While efficient execution of its computational graph leads to high-performance and energy-efficient execution, generating an optimal computational graph schedule is known as \textit{NP-hard} problem. The complexity of scheduling the DNNs computational graphs will further increase on pipelined multi-core system considering memory communication cost, as well as the increasing size of DNNs. This work presents a reinforcement learning based scheduling framework, which imitates the behaviors of optimal optimization algorithms at the speed of inference, and compiles arbitrary DNNs computational graphs without re-training. Our framework has demonstrated up to $\sim$$2.5\times$ runtime speedups over the commercial Edge TPU compiler, using ten popular ImageNet models, on physical Google Edge TPUs system. More importantly, compared to the exact optimization methods solved by heuristics and brute-force, the proposed RL scheduling improves the scheduling runtime by several orders of magnitude.

One-sentence Summary: This work introduces a reinforcement learning based scheduling framework, which mimics optimal optimization algorithm in inference quality but in speedy way.