TropNNC: Structured Neural Network Compression Using Tropical Geometry

TMLR Paper7766 Authors

Abstract: We present TropNNC, a framework for compressing neural networks with linear and convolutional layers and ReLU-type activations using tropical geometry. By representing a network’s output as a tropical rational function, TropNNC enables structured compression via reduction of the corresponding tropical polynomials. Our method identifies redundancy via similarity and improves upon the geometric approximation of previous work by adaptively selecting the weights of retained neurons. We relate it to SVD and spectral clustering, and develop a theoretical analysis that yields useful insights into the network compression problem in general. We provide the tightest known theoretical compression bound, and the first successful application of tropical geometry to convolutional layers. TropNNC requires access only to network weights -- no training data -- and achieves competitive performance on MNIST, CIFAR, and ImageNet, matching strong baselines such as ThiNet and CUP.

Submission Type: Long submission (more than 12 pages of main content)

Assigned Action Editor: ~Grigorios_Chrysos1

Submission Number: 7766