Capacity Limits of Fully Binary CNN

Kamila Abdiyeva, Timur Tibeyev, Martin Lukac

Abstract: Convolutional Neural Networks (CNNs) have achieved a state-of-the-art performance on different real world information processing tasks. However, CNNs are computationally and power intensive, which makes them difficult to run on wearable and embedded systems. Using binary models instead of more complex models, was shown to significantly reduce the memory and computational resources; however, at the cost of a lower accuracy. The aim of the paper is to provide the further exploration of the binarization effect on the model capacity. We study a multiple-valued thresholding (binarization) of the input images and we combine features of several binary networks to perform a classification task. Each image is fitted to a separate binary CNN model. In this manner, we decompose the real world problem to several binary sub-problems. The separate binary models are then assembled into a final classifier and are used to predict the class label. The results show that while for MNIST the accuracy is very close to the full precision counterpart, for the more complex dataset, CIFAR-10, the binarization and the representational power of CNNs is strongly affected.