Network slimming using Lp(p1) regularization

Fengrui Ji, Xinwang Chen, Renxin Chu, Baolin Liu

Abstract: Highlights•We apply the Lp(p<1) norm to approximate the L0 norm for network slimming.•We propose a novel optimization algorithm to solve Lp(p<1) regularization.•We propose synaptic inhibition to avoid the weights’ excessive excitation.•Experiments show that our method has SOTA on CIFAR, ImageNet, and GLUE datasets.