Octave Deep Compression: In-Parallel Pruning-Quantization on Different Frequencies

Qisheng He, Ming Dong, Loren Schwiebert

Abstract: Though deep neural networks achieve great accuracy in visual recognition tasks, they contain millions of weights and thus require a large space to be stored. This presents a challenge in developing deeper neural networks as well as installing those models on mobile devices. In this paper, we propose Octave Deep Compression (ODC), a deep compression algorithm targeted toward the Octave Convolutional Networks (OCNs). ODC compresses OCNs with in-parallel pruning-quantization on different frequencies. We performed extensive experiments on Cifar10 and ImageNet, and our compression results on popular deep learning models such as VGG, ResNet50, and MobileNetV2 demonstrate that ODC can simultaneously achieve a smaller model size and a higher classification accuracy when compared to the state-of-the-art network compression methods.