Distributed Learning over Unreliable Networks

Chen Yu, Hanlin Tang, Cédric Renggli, Simon Kassing, Ankit Singla, Dan Alistarh, Ce Zhang, Ji Liu

Abstract: Most of today’s distributed machine learning systems assume <em>reliable networks</em>: whenever two machines exchange information (e.g., gradients or models), the network should guarantee the deli...