Pruning Neural Networks with Velocity-Constrained Optimization

Donghyun Oh, Jinseok Chung, Namhoon Lee

Keywords: pruning, constrained optimization, velocity-constrained optimization

TL;DR: We propose a novel pruning method based on velocity-constrained optimization, with competitive results.

Abstract:

Pruning has gained prominence as a way to compress over-parameterized neural networks. While
pruning can be understood as solving a sparsity-constrained optimization problem, pruning by di-
rectly solving this problem has been relatively underexplored. In this paper, we propose a method to
prune neural networks using the MJ algorithm, which interprets constrained optimization using the
framework of velocity-constrained optimization. The experimental results show that our method
can prune VGG19 and ResNet32 networks by more than 90% while preserving the high accuracy
of the dense network.

Submission Number: 95