A Deep-Reinforcement-Learning-Based Scheduler for FPGA HLS

Hongzheng Chen, Minghua Shen

Abstract: As the most critical stage in FPGA HLS, scheduling depends heavily on heuristics due to their speed, flexibility, and scalability. However, designing heuristics easily involves human bias, which makes scheduling unpredictable in some specific cases. To solve the problem, we propose an efficient deep reinforcement learning (Deep-RL) based scheduler for FPGA HLS. It has the potential to reduce the human involvement maximumly and learn to schedule by itself. The proposed scheduler consists of three steps. First, we design a novel state and action representation for constrained scheduling problems, which is the foundation of the learning task. Then, we leverage a training pipeline to train the policy network. Specifically, supervised learning is used to initialize the weights of the network and reinforcement learning is used to improve the performance, both of which make the Deep-RL based scheduler practical for HLS. At last, we compare our scheduler with the ASAP schedule and the optimal ILP schedule. Experimental results show that the proposed scheduler can reduce up to 74% resource usage compared with the original ASAP schedule, and the gap between the optimal solution is very small. Notably, this is the first work leveraging reinforcement learning in HLS and has great potential to be integrated into different HLS systems.