Less than a Single Pass: Stochastically Controlled Stochastic Gradient

Lihua Lei, Michael I. Jordan

Abstract: We develop and analyze a procedure for gradient-based optimization that we refer to as stochastically controlled stochastic gradient (SCSG). As a member of the SVRG family of algorithms, SCSG makes...