Optimal Mini-Batch and Step Sizes for SAGA

Nidham Gazagnadou, Robert M. Gower, Joseph Salmon

Abstract: Recently it has been shown that the step sizes of a family of variance reduced gradient methods called the JacSketch methods depend on the expected smoothness constant. In particular, if this expec...