DualTune-GhostDP: A Unified Framework for Synergistic Differentially Private Fine-Tuning of Prompt-Based Large Language Models

TMLR Paper7553 Authors

Abstract:

With growing concerns about data privacy and confidentiality, there has been increased attention on privacy preserving integration in many applications, particularly data driven ones like Large Language Models (LLMs). LLMs are powerful in-context learners and are widely adopted in real world products. However, their dependence on sensitive private data in training and prompts exposes them to potential data leakage and privacy breaches. Differential Privacy (DP) delivers a rigorous, mathematically provable safeguard against
these vulnerabilities; however, this assurance often comes with considerable reductions in model performance and increased computational cost. While prior work has highlighted the inherent trade-off between privacy and utility, our proposed method, DualTune-GhostDP, shows that strong privacy guarantees can be maintained under a controlled budget without sacrificing high model performance. Our method adopts a two-phase fine-tuning pipeline that integrates Ghost Clipping with an EdgeWorth (EW) Advanced Privacy Accountant, replacing conventional DP accounting mechanisms. Experimental results show that the principled integration of these components in DualTune-GhostDP consistently outperforms the individual benefits of each and both the single-phase Differentially Private Stochastic Gradient Descent (DP-SGD) baseline and a two-phase fine-tuning variant using standard clipping. Specifically, it achieves higher accuracy, faster convergence, and improved computational efficiency while maintaining differential privacy guarantees. In addition, we assess robustness to Membership Inference Attacks (MIA), which aim to determine whether a particular sample was used during training. Our findings demonstrate that DualTune-GhostDP substantially mitigates membership leakage across all training stages, strengthening both the privacy assurances and the overall stability of the approach against such attack relative to existing baselines.

Submission Type: Regular submission (no more than 12 pages of main content)

Assigned Action Editor: ~Sanghyun_Hong1

Submission Number: 7553