DNN Feature Map Compression using Learned Representation over GF(2)


Nov 03, 2017 (modified: Dec 16, 2017) ICLR 2018 Conference Blind Submission readers: everyone Show Bibtex
  • Abstract: In this paper, we introduce a method to compress intermediate feature maps of deep neural networks (DNNs) to decrease memory storage and bandwidth requirements during inference. Unlike previous works, the proposed method is based on converting fixed-point activations into vectors over the smallest GF(2) finite field followed by nonlinear dimensionality reduction (NDR) layers embedded into a DNN. Such an end-to-end learned representation finds more compact feature maps by exploiting quantization redundancies within the fixed-point activations along the channel or spatial dimensions. We apply the proposed network architecture to the tasks of ImageNet classification and KITTI object detection. When combined with conventional methods, the conducted experiments show two orders of magnitude decrease in memory requirements while adding only bitwise computations.
  • TL;DR: Feature map compression method that converts quantized activations into binary vectors followed by nonlinear dimensionality reduction layers embedded into a DNN
  • Keywords: feature map, representation, compression, quantization, finite-field