Textual Differential Privacy for Context-Aware Reasoning with Large Language Model

Junwei Yu, Jieyu Zhou, Yepeng Ding, Lingfeng Zhang, Yuheng Guo, Hiroyuki Sato

Abstract: Large language models (LLMs) have demonstrated proficiency in various language tasks but encounter difficulties in specific domain or scenario. These challenges are mitigated through prompt engineering techniques such as retrieval-augmented generation, which improves performance by integrating contextual information. However, concerns regarding the privacy implications of context-aware reasoning architectures persist, particularly regarding the transmission of sensitive data to LLMs service providers, potentially compromising personal privacy. To mitigate these challenges, this paper introduces Tex-tual Differential Privacy, a novel paradigm aimed at safeguarding user privacy in LLMs-based context-aware reasoning. The proposed Differential Embedding Hash algorithm anonymizes sensitive information while maintaining the reasoning capability of LLMs. Additionally, a quantification scheme for privacy loss is proposed to better understand the trade-off between privacy protection and loss. Through rigorous analysis and experimentation, the effectiveness and robustness of the proposed paradigm in mitigating privacy risks associated with context-aware reasoning tasks are demonstrated. This paradigm addresses privacy concerns in context-aware reasoning architectures, enhancing the trust and utility of LLMs in various applications.