A priori estimates of the population risk for two-layer neural networks

Weinan E, Chao Ma, Lei Wu

Abstract:

New estimates for the population risk are established for two-layer neural networks.
These estimates are nearly optimal in the sense that the error rates scale in the same way as the Monte
Carlo error rates. They are equally effective in the over-parametrized regime when the network size is
much larger than the size of the dataset. These new estimates are a priori in nature in the sense that
the bounds depend only on some norms of the underlying functions to be fitted, not the parameters in
the model, in contrast with most existing results which are a posteriori in nature. Using these a priori
estimates, we provide a perspective for understanding why two-layer neural networks perform better
than the related kernel methods.