2021 (modified: 25 Feb 2022)ICML 2021Readers: Everyone
Abstract:In the vanishing learning rate regime, stochastic gradient descent (SGD) is now relatively well understood. In this work, we propose to study the basic properties of SGD and its variants in the non...