An effective two-stage channel pruning method based on two-dimensional information entropy

Yifei Xu, Jinfu Yang, Runshi Wang, Haoqing Li

Abstract: Channel pruning can reduce the number of neural network parameters and computational cost by eliminating redundant channels, its main purpose is to adapt to resource constrained devices. Evaluation-based global pruning and network search-based pruning are two common methods of channel pruning. However, the network architecture pruned by the global mask is often not optimal, while the method that directly searches for the optimal architecture will introduce a large number of hyperparameters, which greatly increases the training cost. In this paper, we propose a novel Two-dimensional information Entropy based Channel Pruning method (TECP). The pruning process consists of two steps. First, a global mask pruning scheme is employed to obtained a pre-pruning model. Then, the two-dimensional information entropy is calculated by using feature maps of dense network to adjust the pre-pruning model adaptively to get a compact network. Moreover, the entropy values are used to determine the minimum number of reserved channels per layer based on to avoid the imbalance of network architecture and the layer collapse caused by global pruning. Extensive experiments with a variety of networks on several datasets clearly demonstrate the effectiveness of our proposed TECP method. For example, results show that on CIFAR-10, the compressed model achieves comparable accuracy to the original model, but with a significantly lower number of parameters (44.29% for ResNet-20 and 46.79% for VGG-16). This is beneficial for industrial deployment. And experimental results also show that TECP method obtain the better performance compared with state-of-the-art method.

External IDs:dblp:journals/apin/XuYWL24