Why do Larger Models Generalize Better? A Theoretical Perspective via the XOR Problem

Alon Brutzkus, Amir Globerson

Abstract: Empirical evidence suggests that neural networks with ReLU activations generalize better with over-parameterization. However, there is currently no theoretical analysis that explains this observati...