An Intelligent Co-Scheduling Framework for Efficient Super-Resolution on Edge Platforms With Heterogeneous Processors

Qi Wang, Weiwei Fang, Liang Qian, Yanming Chen, Neal N. Xiong

Abstract: Deep neural networks (DNNs) have shown remarkable performance in the super-resolution (SR) task, which can upscale low-resolution images to satisfy application demands on image quality. However, the high computational intensity of DNN models poses a challenge to executing SR tasks on resource-constrained edge platforms. To leverage heterogeneous computational resources (e.g., CPU, GPU, and NPU) to speed up image reconstruction through concurrent inference, we propose a novel framework, called ESHP, for Efficient SR on edge platforms with Heterogeneous Processors. Our proposed ESHP framework boasts several advantageous characteristics: 1) it substantially speeds up SR processing over the existing approaches by leveraging all available heterogeneous hardware; 2) it uses deep reinforcement learning (DRL) to enable adaptive and optimal scheduling based on runtime states; 3) it strikes a balance between SR performance and computational cost during inference; and 4) it does not modify the original architecture of the given SR model. We have conducted extensive experiments on typical edge platforms with popular SR models and resolution data sets of different scales, which verify the effectiveness and the versatility of our ESHP against other commonly used baselines.