The Effect of Network Width on Stochastic Gradient Descent and Generalization: an Empirical Study

Daniel S. Park, Jascha Sohl-Dickstein, Quoc V. Le, Samuel L. Smith

Abstract: We investigate how the final parameters found by stochastic gradient descent are influenced by over-parameterization. We generate families of models by increasing the number of channels in a base n...