Sparse Double Descent: Where Network Pruning Aggravates Overfitting

Zheng He, Zeke Xie, Quanzhi Zhu, Zengchang Qin

Abstract: People usually believe that network pruning not only reduces the computational cost of deep networks, but also prevents overfitting by decreasing model capacity. However, our work surprisingly disc...