Whisper: Efficiently Preserving Both Your Prompt and Model Privacy in LLM Fine-Tuning

ACL ARR 2026 January Submission2659 Authors

Keywords: Privacy-Preserving Fine-Tuning, Efficient Encrypted Transformer Design, Secure Protocol Optimization

Abstract: We propose Whisper, an efficient Privacy-Preserving Fine-Tuning (PPFT) framework that provides Dual-privacy for cloud-based APIs by leveraging homomorphic encryption to protect both user inputs and fine-tuned model parameters. Whisper redesigns the encrypted fine-tuning pipeline at the architectural level by introducing a backpropagation-free encrypted fine-tuning paradigm, enabling practical deployments on top of existing Privacy-Preserving Inference (PPI) schemes. To further improve efficiency, Whisper optimizes the underlying cryptographic protocols with a group-based packing strategy and comparison with bootstrapping, substantially increasing throughput and reducing execution overhead for fine-tuning workloads. Extensive experiments demonstrate strong privacy guarantees and competitive model performance, achieving a 7.43-38.39$\times$ efficiency improvement over state-of-the-art cryptographic approaches.

Paper Type: Long

Research Area: Language Models

Research Area Keywords: security and privacy, fine-tuning

Contribution Types: NLP engineering experiment, Approaches low compute settings-efficiency

Languages Studied: English

Submission Number: 2659