A Cooperation Index for Model Pruning

Do-Hoon Kim, Jay Myung, Yung-Kyun Noh

Keywords: Parameter Importance, Model Pruning, SHAP value

TL;DR: We propose a new criterion, cooperation Index, for effective model pruning to improve both generalizability and interpretability of the model..

Abstract:

In complex models, tools for measuring parameter importance identify its core
functional element and improve both generalizability and interpretability by pruning
redundant ones. Effective pruning relies on these tools, which serve as decision
making criteria. The SHAP Value (SV) has recently been considered such a
criterion, interpreted as measuring the average marginal contribution across all
possible paths of parameter accumulation. However, we find that this averaging
process of SV systematically overweights redundant parameters. Instead, we
propose that measuring the speed of decay of the marginal contribution can serve
as a more effective decision-making criterion. Specifically, we quantify the number
of cooperative contribution for each parameter and show that this criterion is more
effective for parameter pruning in backward elimination, leading to a more optimal
set of remaining parameters.

Primary Area: interpretability and explainable AI

Submission Number: 18266