A novel pruning model of deep learning for large-scale distributed data processing

Yiqiang Sheng, Chaopeng Li, Jinlin Wang, Haojiang Deng, Zhenyu Zhao

Abstract: In this paper, we propose a novel pruning model of deep learning for large-scale distributed data processing to simulate a potential application in the geographical neighbor of Internet of Things. We formulate a general model of pruning learning, and we investigate the procedure of pruning learning to satisfy hard constraint and soft constraint. The hard constraint is a class of non-flexible setting without parameter learning to match the structure of distributed data. The soft constraint is a process of adaptive parameter learning to satisfy an inequality without any degradation of accuracy if the size of training data is large enough. Based on the simulation using distributed MNIST image database with large-scale samples, the performance of the proposed pruning model is better than that of a state-of-the-art model of deep learning in case of big data processing.