No Innocence in Styling: Discovery of Privacy Protection Capabilities and Security Risks in Consumer Generative AI Writing Assistants

Mohd. Farhan Israk Soumik, SYED MHAMUDUL HASAN, Wanniarachchi Kankanamge Malithi Mithsara, Ahmed Imteaj, Abdur R. Shahid

Keywords: Platform-Integrated LLMs, Text Stylization, Generative AI Writing Assistants, Privacy–Utility Trade-off, Emotion Inference, Content Moderation, Moderation Evasion, Deployment-Scale Evaluation

TL;DR: Text stylization in consumer AI writing assistants simultaneously protects user privacy and weakens content moderation systems.

Abstract: Generative AI writing assistants are now integrated into consumer platforms such as Apple Intelligence and Microsoft Copilot, enabling millions of users to automatically rewrite and stylize their text. While positioned as productivity tools, their deployment at scale introduces important and underexplored implications for privacy and platform safety. This paper examines the dual-use nature of platform-level text stylization. Stylization can enhance privacy by suppressing stylistic signals used for profiling and personal data inference. However, the same transformations can be leveraged to evade automated safeguards, including misinformation detection systems. We conduct empirical case studies on emotion inference and misinformation detection across benchmark datasets using deployed stylization modes. We evaluate downstream impact with fine-tuned open-source models and GPT-4o in a zero-shot setting. Our results show that stylization reduces emotion inference accuracy, lowering profiling risk, while increasing error rates in misinformation detection. This discovery reveal a measurable trade-off among privacy protection, moderation robustness, and stylization, highlighting new design and governance challenges for industry deployment.

Submission Type: Discovery

Submission Number: 442