Characterizing the Training Dynamics of Private Fine-tuning with Langevin diffusion

Shuqi Ke, Charlie Hou, Sewoong Oh, Giulia Fanti

Keywords: fine-tuning theory, differential privacy, convergence, transfer learning theory, langevin diffusion, gradient flow

TL;DR: We provide a quantitative analysis of training and neuron dynamics in private fine-tuning.

Abstract: We show that differentially private full fine-tuning (DP-FFT) can distort pre-trained backbone features based on both theoretical and empirical results. We identify the cause of the distortion as the misalignment between the pre-trained backbone and the randomly initialized linear head. We prove that a sequential fine-tuning strategy can mitigate the feature distortion: first-linear-probing-then-fine-tuning (DP-LP-FFT). A new approximation scheme allows us to derive approximate upper and lower bounds on the training loss of DP-LP and DP-FFT, in a simple but canonical setting of 2-layer neural networks with ReLU activation. Experiments on real-world datasets and architectures are consistent with our theoretical insights. Moreover, our theory suggests a trade-off of privacy budget allocation in multi-phase fine-tuning methods like DP-LP-FFT.

Submission Number: 108