Convolutional Neural Network Compression Based on Improved Fractal Decomposition Algorithm for Large Scale Optimization

Arcadi Llanza, Fekhr Eddine Keddous, Nadiya Shvai, Amir Nakib

Abstract: Deep learning methods have shown state-of-the-art results in various application areas such as computer vision, NLP, etc. However, their practical use presents many challenges, including those caused by the large size of the models, especially in the context of model weight storage and transmission. One of the possible solutions to this problem is Neural Network (NN) compression, which is a process of obtaining a derived model serving the same task with a smaller number of parameters or with parameters of lower precision. The most common NN compression techniques include pruning, sparse representation, quantization, and knowledge transfer. In this article, the compression of Convolutional Neural Networks (CNNs) using fractional differentiation is investigated. A for-mulation of this task as a large-scale continuous optimization problem is then proposed, and its resolution is performed through a new optimization algorithm, called the Improved Fractal Decomposition Algorithm (IFDA), based on space geometric fractal decomposition. The results obtained show that MobileNetV3, for instance, is compressed by 18.5% with only a 2.5% decrease in accuracy. Additionally, the proposed IFDA algorithm outperforms all other competing metaheuristics in solving this problem.