Decentralized Deep Learning with Arbitrary Communication Compression

Anastasia Koloskova, Tao Lin, Sebastian U. Stich, Martin Jaggi

Abstract: We propose Choco-SGD---decentralized SGD with compressed communication---for non-convex objectives and show its strong performance in various deep learning applications (on-device learning, datacenter case).