Energy-efficient Content-aware DNN Inference for Mobile Video via Deep Reinforcement Learning

Guangfeng Guo, Junxing Zhang

Abstract: Recent work has shown that wearable devices can assist users in cognitive decline through context-aware scene interpretation. These devices should function in real time with sufficient functions, performance, and usability. However, scene interpretation relies on the Deep Neural Network (DNN) inference of continuous video streams, which poses enormous challenges to wearable devices because the resulting computational tasks will quickly drain their batteries. Therefore, we propose an energy-efficient and content-aware DNN inference scheme to address these challenges for assistive devices. We first present the architecture of the assistive system. Then, we leverage the temporal correlation of video frames to save energy spent on the inference by designing a novel online planning method that performs Deep Reinforcement Learning (DRL) using only a subset of frames. Moreover, we come up with a lookup algorithm to select the Dynamic Voltage and Frequency Scaling (DVFS) gear scheme for further energy-saving. Finally, we implement a system prototype and evaluate its energy consumption. The experimental results show that our solution saves almost 40% of the energy on average with negligible impact on inference accuracy and latency compared with the existing approach.