Wavelet Feature Maps Compression for Low Bandwidth Convolutional Neural Networks

Yair Zohav, Shahaf E Finder, Maor Ashkenazi, Eran Treister

Keywords: Convolutional Neural Networks, Quantization, Wavelet Transform

Abstract: Quantization is one of the most effective techniques for compressing Convolutional Neural Networks (CNNs), which are known for requiring extensive computational resources. However, aggressive quantization may cause severe degradation in the prediction accuracy of such networks, especially in image-to-image tasks such as semantic segmentation and depth prediction. In this paper, we propose Wavelet Compressed Convolution (WCC)---a novel approach for activation maps compression for $1\times1$ convolutions (the workhorse of modern CNNs). WCC achieves compression ratios and computational savings that are equivalent to low bit quantization rates at a relatively minimal loss of accuracy. To this end, we use a hardware-friendly Haar-wavelet transform, known for its effectiveness in image compression, and define the convolution on the compressed activation map. WCC can be utilized with any $1\times1$ convolution in an existing network architecture. By combining WCC with light quantization, we show that we achieve compression rates equal to 2-bit and 1-bit with minimal degradation in image-to-image tasks.

One-sentence Summary: In this paper, we suggest a wavelet compressed convolution as a better alternative to aggressive activation map quantization in high-resolution CNNs.