Pruning Neural Networks with Velocity-Constrained Optimization

Published: 26 Oct 2023, Last Modified: 13 Dec 2023NeurIPS 2023 Workshop PosterEveryoneRevisionsBibTeX
Keywords: pruning, constrained optimization, velocity-constrained optimization
TL;DR: We propose a novel pruning method based on velocity-constrained optimization, with competitive results.
Abstract: Pruning has gained prominence as a way to compress over-parameterized neural networks. While pruning can be understood as solving a sparsity-constrained optimization problem, pruning by di- rectly solving this problem has been relatively underexplored. In this paper, we propose a method to prune neural networks using the MJ algorithm, which interprets constrained optimization using the framework of velocity-constrained optimization. The experimental results show that our method can prune VGG19 and ResNet32 networks by more than 90% while preserving the high accuracy of the dense network.
Submission Number: 95