Telling or Not ? An Investigation into the Privacy Flow of Large Language Models in Simulated Contextual Scenarios
Abstract: Understanding what information to share in their output, the purpose of sharing, and with whom is essential for privacy protection on agents powered by Large Language Models (LLMs). Despite existing research revealing that LLMs can inadvertently disclose privacy in contexts where humans would not, to date, no evaluations have treated these large models as genuine participants in real-life scenarios, nor have they adequately considered contextual or interactive factors. This paper introduces privacyFlow, a multi-tiered framework designed specifically to examine the privacy flow of LLMs in simulated interactive scenarios. The framework comprehensively covers 150 privacy concerns across 1,200 contextual scenarios. We conducted extensive experiments on four LLMs, evaluating the influence of type of privacy, recipient relationship, legal-moral directives, and prompting attacks on privacy-sharing behaviors. Our findings provide valuable insight into disclosure patterns and propose avenues for future alignment efforts, emphasizing the necessity for LLMs to possess the capability to regulate privacy flow in harmony with human expectations, even in extreme scenarios such as prompting attacks.
Paper Type: Long
Research Area: Resources and Evaluation
Research Area Keywords: Large Language Model, Society Simulation, Private Data Flow, Prompt Attacks
Contribution Types: Model analysis & interpretability, Data analysis, Surveys
Languages Studied: English
Submission Number: 851
Loading