Recycling Model Updates in Federated Learning: Are Gradient Subspaces Low-Rank?

Sheikh Shams Azam, Seyyedali Hosseinalipour, Qiang Qiu, Christopher G. Brinton

Abstract: In this paper, we question the rationale behind propagating large numbers of parameters through a distributed system during federated learning. We start by examining the rank characteristics of the...