Differentially Private Matrix Factorization with Secure Aggregation and Random Projection

Keywords: Random projection, federated recommender system, matrix factorization

Abstract: Federated learning offers a promising approach to developing privacy-preserving matrix factorization algorithms. By combining secure aggregation (SecAgg) with differential privacy (DP), also referred to as Distributed Differential Privacy (Distributed DP), it is possible to achieve formal privacy guarantees while maintaining a satisfactory level of accuracy. Recently, the Poisson Binomial Mechanism (PBM) has emerged as a state-of-the-art Distributed DP mechanism, which provides an unbiased estimator. However, despite its effectiveness, PBM suffers from increased communication overhead caused by SecAgg. To address this issue, we propose a novel framework called Differentially Private Matrix Factorization with Random Projection (DPMF-RP). This framework integrates PBM with sparse random projection, breaking the dependency between communication costs and parameter sizes. In our approach, users apply sparse random projections to the gradient matrix, reducing its dimensionality before transmission, thereby significantly decreasing communication overhead. Our work is the first to design a differentially private matrix factorization framework that leverages the combination of PBM and random projection. We rigorously analyze how these techniques can be effectively integrated to achieve privacy and efficiency. Empirical studies in two MovieLens datasets demonstrate that our approach has little loss in accuracy with $\epsilon \geq 1$ and $m/p\geq 2$, while reducing the communication overhead by at least $50\%$.

Submission Number: 122