Partial model averaging in Federated Learning: Performance guarantees and benefits

Sunwoo Lee, Anit Kumar Sahu, Chaoyang He, Salman Avestimehr

Abstract: Highlights•A novel partial model averaging scheme that accelerates the federated optimization.•Frequent partial model synchronizations strongly suppress the model discrepancy across clients.•While maintaining the total data transfer size, the partial averaging accelerate the convergence of global loss.