Good Subnetworks Provably Exist: Pruning via Greedy Forward Selection

Mao Ye, Chengyue Gong, Lizhen Nie, Denny Zhou, Adam R. Klivans, Qiang Liu

Abstract: Recent empirical works show that large deep neural networks are often highly redundant and one can find much smaller subnetworks without a significant drop of accuracy. However, most existing metho...