Pruning has a disparate impact on model accuracyDownload PDF

Published: 31 Oct 2022, Last Modified: 13 Jan 2023NeurIPS 2022 AcceptReaders: Everyone
Keywords: Model pruning, Fairness
TL;DR: The paper observes that pruning causes disparate impacts to the accuracy of different groups of individuals and sheds light on the causes of such disparate impacts to arise.
Abstract: Network pruning is a widely-used compression technique that is able to significantly scale down overparameterized models with minimal loss of accuracy. This paper shows that pruning may create or exacerbate disparate impacts. The paper sheds light on the factors to cause such disparities, suggesting differences in gradient norms and distance to decision boundary across groups to be responsible for this critical issue. It analyzes these factors in detail, providing both theoretical and empirical support, and proposes a simple, yet effective, solution that mitigates the disparate impacts caused by pruning.
Supplementary Material: pdf
21 Replies