Noise-Tolerant Hardware-Aware Pruning for Deep Neural Networks

Shun Lu, Cheng Chen, Kunlong Zhang, Yang Zheng, Zheng Hu, Wenjing Hong, Guiying Li, Xin Yao

Abstract: Existing hardware-aware pruning methods for deep neural networks do not take the uncertain execution environment of low-end hardware into consideration. That makes those methods unreliable, since the hardware environments they used for evaluating the pruned models contain uncertainty and thus the performance values contain noise. To deal with this problem, this paper proposes noise-tolerant hardware-aware pruning, i.e., NT-HP. It uses a population-based idea to iteratively generate pruned models. Each pruned model is sent to realistic low-end hardware for performance evaluations. For the noisy values of performance indicators collected from hardware, a threshold for comparison is set, where only the pruned models with significantly better performances are kept in the next generation. Our experimental results show that with the noise-tolerant technique involved, NT-HP can get better pruned models in the uncertain execution environment of low-end hardware.