Does the Data Induce Capacity Control in Deep Learning?

Rubing Yang, Jialin Mao, Pratik Chaudhari

Abstract: We show that the input correlation matrix of typical classification datasets has an eigenspectrum where, after a sharp initial drop, a large number of small eigenvalues are distributed uniformly ov...