Automatic Complementary-Separation Pruning for Efficient CNNs

David Levin, Gonen Singer

Keywords: Neural Network Pruning, Model Compression, Layer Pruning, Computational Efficiency

TL;DR: We propose Automatic Complementary-Separation Pruning (ACSP), an automated method that prunes CNNs by selecting complementary components, cutting FLOPs by up to 2.5× while maintaining or improving accuracy.

Abstract: Reducing the complexity of neural networks without sacrificing performance is a critical challenge for deploying models in real-world, resource-constrained environments. We introduce Automatic Complementary Separation Pruning (ACSP), a novel and fully automated method for pruning convolutional neural networks that focuses on accelerating inference time. ACSP combines structured and activation-based pruning to remove redundant neurons and channels while preserving essential components. Tailored for supervised learning tasks, ACSP constructs a graph space that encodes the separation capabilities of each component across all class pairs. By leveraging complementary selection principles and clustering techniques, ACSP ensures that the selected components maintain diverse and complementary separation capabilities, reducing redundancy and maintaining high network performance. The pruning volume is determined automatically, removing the need for manual tuning. This approach significantly reduces the number of FLOPs (floating-point operations) and results in faster inference time without compromising accuracy.

Primary Area: applications to computer vision, audio, language, and other modalities

Submission Number: 3485