EEAI: An End-edge Architecture for Accelerating Deep Neural Network Inference

Guozhi Liu, Fei Dai, Bi Huang, Zhenping Qiang, LeCheng Li, Shuai Wang

Abstract: Deep Neural Networks (DNNs), as a key technology for Artificial Intelligence (AI) applications in the 5G era, have been widely used in the field of mobile intelligence. However, it is challenging to run computation-intensive DNN-based applications on mobile devices due to the limited resources. What's worse, traditional cloud-assisted approach requires significant amounts of data to be uploaded to the cloud through wireless network, leading to poor real-time performance and low quality of user experience. To address these challenges, in this paper, we propose EEAI, a framework that end-edge collaborative DNN inference. Firstly, we analyze the characteristics of DNN-based applications. Then, we introduce edge computing, and design EEAI, a light-weight scheduler to automatically partition DNN computation between mobile devices and edge servers at the granularity of DNN layers. Finally, we verify the effectiveness of the EEAI under different network environments (4G, 5G, and Wi-Fi). Experiments show that the proposed method is effective and has good performance.