Stochastic distributed learning with gradient quantization and double-variance reduction

Samuel Horváth, Dmitry Kovalev, Konstantin Mishchenko, Peter Richtárik, Sebastian U. Stich

Abstract: We consider distributed optimization over several devices, each sending incremental model updates to a central server. This setting is considered, for instance, in federated learning. Various schem...