Channel-Prioritized Convolutional Neural Networks for Sparsity and Multi-fidelity

Chun-Min Chang, Hung-Yi Ou Yang, Chia-Ching Lin, Chin-Laung Lei, Kuan-Ta Chen

Abstract: We propose a novel convolutional neural networks (CNNs) training procedure to allow dynamically trade-offs between different resource and performance requirements. Our approach prioritizes the channels to enable structured sparsity and multi-fidelity approximations at inference time. We train the VGG network with our method on various benchmark datasets. The experiment results show that, on the CIFAR-10 dataset, a 63x parameters reduction and a 11x FLOPs reduction can be achieved, with only a 2% accuracy drop.

Keywords: filter/channel prioritization, network pruning, compression, computational efficiency, multi-fidelity inference