Noise and Fluctuation of Finite Learning Rate Stochastic Gradient DescentDownload PDFOpen Website

2021 (modified: 25 Feb 2022)ICML 2021Readers: Everyone
Abstract: In the vanishing learning rate regime, stochastic gradient descent (SGD) is now relatively well understood. In this work, we propose to study the basic properties of SGD and its variants in the non...
0 Replies

Loading