Go to ACL ARR 2025 February homepagePrivacyRestore: Privacy-Preserving Inference in Large Language Models via Privacy Removal and Restoration
Abstract: The widespread usage of online Large Language Models (LLMs) inference services has raised significant privacy concerns about the potential exposure of private information in user inputs.
Existing privacy protection methods for LLMs suffer from either insufficient privacy protection with performance degradation, or large inference time overhead.
To address these limitations, we propose PrivacyRestore, a plug-and-play method to protect the privacy of user inputs during LLM inference for the client-server scenario.
The server first trains restoration vectors for each privacy span type offline and then releases them to the clients.
During inference, the client aggregates restoration vectors of all privacy spans in the user query into a meta restoration vector which is later sent to the server to restore information.
Before transmission, the client removes all privacy spans in the user query and applies $d_\chi$-privacy mechanism to the meta vector for privacy protection.
We prove that our method can inherently prevent the linear growth of the privacy budget.
We conduct extensive experimental, covering the medical and legal domains, and demonstrate that PrivacyRestore effectively protects private information and maintains acceptable levels of performance and inference efficiency
Paper Type: Long
Research Area: NLP Applications
Research Area Keywords: security/privacy
Contribution Types: NLP engineering experiment, Publicly available software and/or pre-trained models, Data resources
Languages Studied: English
Submission Number: 4077
Loading