EEOKD: Energy-Efficient Online Knowledge Distillation for Mobile Video Inference

Guangfeng Guo, Junxing Zhang

Abstract: Wearable devices can assist users in cognitive decline through context-aware scene interpretation. They should function in real-time with sufficient functions, performance, and usability. However, the high-accuracy and low-delay scene interpretation rely on the Deep Neural Network (DNN) inference of continuous video streams, which poses enormous challenges to wearable devices because of the tight energy budget and unpredictable delay impact. In this paper, we propose a novel framework dubbed EEOKD: Energy-Efficient Online Knowledge Distillation. The framework specializes in a high-accuracy and low-cost object detection model to automatically adapt to the target video while occupying a small amount of bandwidth and tolerating changes in network delay. First, we propose efficient asynchronous distributed algorithms based on the loss gradient to alleviate the impact of delay changes. Then, we propose a novel online knowledge distillation scheme with freshness-based importance sampling and batch training to improve the generalization ability of the student model using fewer samples at lower weight updating frequencies. The new method saves energy by accelerating the model convergence and keeps good detection performance while network delay changes considerably. Finally, we implement a system prototype and evaluate its performance and energy consumption. The experimental results show that our EEOKD saves almost 15% of the energy and about 60% of the network bandwidth, and it also improves the detection accuracy by 3% on average compared with the existing approach.