Cross-Silo Training of Differentially Private Models with Secure Multiparty Computation

Sikha Pentyala, Davis Railsback, Ricardo José Menezes Maia, David Melanson, Rafael Dowsley, Anderson Nascimento, Martine De Cock

Keywords: Privacy-preserving machine learning, Secure multi party computation, Differential privacy

Abstract: We address the problem of learning a machine learning model from training data that originates at multiple data holders in a cross-silo federated setup, while providing formal privacy guarantees regarding the protection of each holder's data. Existing solutions based on Differential Privacy (DP) achieve this at the cost of a drop in accuracy. Solutions based on Secure Multiparty Computation (MPC) do not incur such accuracy loss but leak information when the trained model is made publicly available. We propose an MPC solution for training differentially private models. Our solution relies on an MPC protocol for model training, and an MPC protocol for perturbing the trained model coefficients with Laplace noise in a privacy-preserving manner. The resulting MPC+DP approach achieves higher accuracy than a pure DP approach, while providing the same formal privacy guarantees.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Submission Guidelines: Yes

Please Choose The Closest Area That Your Submission Falls Into: Social Aspects of Machine Learning (eg, AI safety, fairness, privacy, interpretability, human-AI interaction, ethics)