IPPRO: Importance-based Pruning with PRojective Offset for Magnitude-indifferent Structural Pruning

ICLR 2026 Conference Submission16959 Authors

Keywords: structured pruning, importance criteria, projective geometry, model compression

Abstract: Not only the classical methods of neural network pruning but also most importance-based pruning methods rely too much on parameter magnitudes to prune effectively. We propose a novel pruning strategy, named IPPRO, using projective space to alleviate the unfair advantage given to parameter magnitudes. We use gradient of loss in the projective space to construct PROscore, which is a magnitude-indifferent score that is in turn used by IPPRO, our novel importance-based structured pruning algorithm. Extensive experiments on Convolutional Neural Networks (CNNs), Vision Transformers (ViT), and Large Language Models (LLMs) demonstrate that IPPRO consistently outperforms, especially in high compression scenarios. Our results establish IPPRO as a task-agnostic and architecture-agnostic pruning paradigm, offering both a new theoretical foundation and a practical tool for magnitude-indifferent structured pruning.

Primary Area: other topics in machine learning (i.e., none of the above)

Submission Number: 16959