Group-based network pruning via nonlinear relationship between convolution filters

Guanqun Zhang, Shuai Xu, Jing Li, Alan J. X. Guo

Abstract: Convolutional neural networks (CNNs) have many successful applications in various domains, but sometimes large computing resources are required. Therefore, pruning techniques are becoming increasingly popular for compressing and accelerating CNNs. Former commonly used group-based pruning methods address this issue using heuristic or linear grouping criteria (e.g., K-Means) without consideration of the high dimensionality and nonlinearity of convolution filters, which may lead to significant accuracy loss. In this paper, we propose a novel group-based pruning method, named kernel principal component analysis based group pruning (KPGP) method, which reduces the dimension of filters before grouping and uses nonlinear clustering criterion. Specifically, we use kernel principal component analysis (kernel-PCA) clustering to classify filters into groups, apply group pruning to each classified group, and reconstruct the pruned convolutional layers into group convolution structure. The proposed KPGP technique can maintain high performance according to its grouping criterion without the requirement of special hardware that many existing pruning methods do. Ablation study shows that the nonlinear clustering criterion in KPGP method is robust and effective. Moreover, we demonstrate the efficiency of KPGP method by applications to CIFAR-10 and ILSVRC-12, with negligible loss of accuracy, compared with state-of-the-art pruning methods.