Adaptive Clipping for Differential Private Federated Learning in Interpolation Regimes

Takumi Fukami, Tomoya Murata, Kenta Niwa

Abstract: We investigate improving the utility of standard differential private optimization algorithms by adaptively determining the clipping radius in federated learning. Our adaptive clipping radius is based on the root-mean-square of the gradient norms, motivated by the interpolation property and smoothness of the objectives. In addition to Renyi Differential Privacy (RDP) analysis, we conduct theoretical utility analysis of the proposed algorithm, showing that our method enhances utility compared to DP-SGD for smooth and non-strongly convex objectives. Numerical experiments confirm the superiority of our adaptive clipping algorithm over standard DP optimization with fixed clipping radius in federated learning settings.

Submission Length: Regular submission (no more than 12 pages of main content)

Assigned Action Editor: ~Yunwen_Lei1

Submission Number: 4558