Merging Deep Neural Networks for Mobile Devices

Yi-Min Chou, Yi-Ming Chan, Jia-Hong Lee, Chih-Yi Chiu, Chu-Song Chen

Abstract: In this paper, a novel method to merge convolutional neural networks for the inference stage is introduced. When two feed-forward networks already trained for handling different tasks are given, our method can align the layers of these networks and merge them into a unified model by sharing the representative weights. The performance of the merged model can be restored or improved via re-training. Without needing high-performance hardware, the proposed method effectively produces a compact model to run the original tasks simultaneously on resource-limited devices. The system development time, as well as training overhead, is substantially reduced because our method leverages the co-used weights and preserves the general architectures of the well-trained networks. The merged model is jointly compressed and can be implemented faster than the original models with a comparable accuracy. When combining VGG-Avg and ZF-Net models, our approach can achieve higher than 12 and 2.5 times of compression and speedup ratios compared to the original whole models, respectively, while the accuracy remains approximately the same.