Diminishing Noise Maintains Differential Privacy and Enhances Convergence

Alexander Shestakov, Nursultan Abdullaev, Aleksandr Beznosikov

Keywords: optimization, differential privacy, distributed optimization

Abstract: Differential Privacy (DP) is a well-established framework for training models in distributed settings while safeguarding sensitive information. Although numerous DP algorithms exist, many current solutions inject noise with constant variance to the transmitted gradients, leading to convergence only to a neighborhood of the optimal solution. To address this limitation, we propose an error compensation technique that maintains linear convergence without compromising privacy guarantees. Experimental results validate the effectiveness of our approach.

Primary Area: optimization

Submission Number: 7679