Systematic Weight Pruning of DNNs using Alternating Direction Method of MultipliersDownload PDF

12 Feb 2018 (modified: 12 Feb 2018)ICLR 2018 Workshop SubmissionReaders: Everyone
Abstract: We present a systematic weight pruning framework of deep neural networks (DNNs) using the alternating direction method of multipliers (ADMM). We first formulate the weight pruning problem of DNNs as a constrained nonconvex optimization problem, and then adopt the ADMM framework for systematic weight pruning. We show that ADMM is highly suitable for weight pruning due to the computational efficiency it offers. We achieve a much higher compression ratio compared with prior work while maintaining the same test accuracy, together with a faster convergence rate.
TL;DR: We present a systematic weight pruning framework of deep neural networks (DNNs) using the alternating direction method of multipliers (ADMM).
Keywords: systematic weight pruning, deep neural networks (DNNs), alternating direction method of multipliers (ADMM)
5 Replies

Loading