An Efficient Distributed Deep Learning Framework for Fog-Based IoT Systems

Yijia Chang, Xi Huang, Ziyu Shao, Yang Yang

Abstract: Deep neural networks (DNNs) are the key techniques to enable edge/fog intelligence. By far, it remains challenging to conduct distributed deployment of DNN models onto resource-constrained fog nodes with low latency. Existing solutions adopt either model compression techniques to reduce the computation loads on fog nodes, or horizontal model partition techniques, which exploit particular communication and computation patterns to partition different layers of DNNs onto fog nodes. Nonetheless, sometimes even resource demands of particular layers can be unaffordable to fog nodes, which makes horizontal partition inadequate and calls for the joint design of vertical and horizontal model partition. Besides, model partition and compression may lead to degraded inference accuracy, but approaches to compensate such accuracy loss remain unexplored.In this paper, we propose an integrated efficient distributed deep learning (EDDL) framework to address the above challenges. Particularly, we adopt balanced incomplete block design (BIBD) methods to reduce computation loads on fog nodes by removing some data flows in DNNs in a systematic and structured manner. By leveraging grouped convolution techniques, we propose a practical scheme to conduct horizontal and vertical model partition jointly. Moreover, we integrate multi-task learning and ensemble learning techniques to further improve the inference accuracy. Simulation results verify the effectiveness of EDDL framework in achieving notable reduction in computation load and memory footprint with mild loss of inference accuracy.