Stochastic Nonconvex Optimization with Large Minibatches

Weiran Wang, Nathan Srebro

Abstract: We study stochastic optimization of nonconvex loss functions, which are typical objectives for training neural networks. We propose stochastic approximation algorithms which optimize a series of re...