Verification and Training of Neural Networks for Robustness Against Neuron Pruning

TMLR Paper8828 Authors

Abstract: Structured neuron pruning removes entire hidden units to reduce model size and computation but often leads to unpredictable accuracy degradation. Existing pruning methods typically rely on heuristic importance scores and provide no formal guarantees on the behavior of pruned models. In this work, we propose a certifiable approach for structured neuron pruning in fully connected layers of feedforward neural networks that guarantees robustness against all pruning masks satisfying a given layer-wise sparsity budget. We further develop a computable upper bound on the worst-case change in pairwise class margins induced by neuron pruning. The analysis models pruning as row-zeroing (equivalently, neuron gating via binary masks) in the weight matrices and bounds the resulting deviation via operator-norm-based error propagation. These bounds are then used to develop a margin-aware robust training objective for certifiable pruning robustness. Experiments on MNIST and CIFAR-10 show that the resulting models achieve non-trivial certified accuracy under a range of pruning budgets and that our robust training substantially improves both certified and empirical robustness over standard baselines.

Submission Type: Regular submission (no more than 12 pages of main content)

Assigned Action Editor: ~Sanghyuk_Chun1

Submission Number: 8828