Lightweight distributed deep learning on compressive measurements for internet of things

Guiqiang Hu, Yong Hu, Tao Wu, Yushu Zhang, Shuai Yuan

Abstract: In this work, we investigate the problem of distributed deep learning in Internet of Things (IoT). The proposed learning framework is constructed in a fog–cloud computing architecture, so as to overcome the limitation of resource constrained IoT end device. Compressive Sensing (CS) is used as a lightweight encryption in the framework to preserve the privacy of training data. Specifically, a chaotic-based CS measurement matrix construction mechanism is applied in the system to save the storage and transmission costs. With this design, the computation overhead of the learning framework in IoT can be successfully offloaded from IoT end device to the fog nodes. Theoretical analysis demonstrates that our system can guarantee security of the raw data against chosen plaintext attack (CPA). Experimental and analysis results show that our privacy-preserving proposal can significantly reduce the communication costs and computation costs with only a negligible accuracy penalty (with classification accuracy 91% testing on MNIST dataset under compression rate 0.5) compared to traditional non-private federated learning schemes. Notably, due to the chaotic-based CS measurement matrix construction mechanism, the memory requirement of end device side can be significantly reduced. This makes our framework be very suitable for the IoT applications in which end devices are equipped with low-spec chips.