LSA-MEP: layer-wise sparsity allocation multi-metric evaluation pruning

Hong Liang, Quanyi Guo, Mingwen Shao, Qian Zhang

Abstract: As deep neural networks continue to grow in scale, their rising computational demands increasingly require high-performance computing (HPC) resources such as distributed clusters. This highlights the need for efficient, HPC-compatible models. Network pruning is a key technique for reducing model complexity, but most existing methods apply uniform sparsity and rely on single-criterion importance metrics, overlooking structural heterogeneity and the multi-dimensional nature of parameter significance. To address these limitations, we propose LSA-MEP: a pruning-at-initialization framework that integrates layer-wise sparsity allocation with multi-metric parameter evaluation. Pruning ratios are determined by each layer’s information content, while parameter importance is assessed through a multi-dimensional evaluation that incorporates static structural characteristics, dynamic optimization signals, and information propagation capacity, yielding a comprehensive and robust importance metric. The process is fully parallelizable, making it well-suited for large-scale models and distributed environments. Experiments across multiple datasets demonstrate that LSA-MEP consistently outperforms existing methods in accuracy preservation while achieving efficient execution through parallel computing.

External IDs:dblp:journals/tjs/LiangGSZ25