A Cooperation Index for Model Pruning

Do-Hoon Kim, Jay Myung, Yung-Kyun Noh

Keywords: Parameter Importance, Model Pruning, SHAP Value, Cooperation Index, Neuroplasticity

TL;DR: We propose a new criterion, cooperation Index, for effective model pruning to improve both generalizability and interpretability.

Abstract: In complex models, tools for measuring parameter importance identify its core functional element and improve both generalizability and interpretability by pruning redundant ones. Effective pruning relies on these tools, which serve as decision-making criteria. The SHAP Value (SV) has recently been considered such a criterion, interpreted as measuring the average marginal contribution across all possible paths of parameter accumulation. However, we find that this averaging process of SV systematically overweights redundant parameters. Instead, we propose that measuring the speed of decay of the marginal contribution can serve as a more effective decision-making criterion. Specifically, we quantify the number of cooperative contribution for each parameter and show that this criterion is more effective for parameter pruning in backward elimination, leading to a more optimal set of remaining parameters.

Submission Number: 112