Tensor Programs IV: Feature Learning in Infinite-Width Neural Networks

Greg Yang, Edward J. Hu

Abstract: As its width tends to infinity, a deep neural network’s behavior under gradient descent can become simplified and predictable (e.g. given by the Neural Tangent Kernel (NTK)), if it is parametrized ...