Can Generative AI Adequately Protect Queries? Analyzing the Trade-Off Between Privacy Awareness and Retrieval Effectiveness

Luca Herranz-Celotti, Blessing Guembe, Giovanni Livraga, Marco Viviani

Abstract: As users increasingly input confidential information in their queries-often through longer and more detailed prompts when interfacing with generative Information Retrieval Systems (IRSs) and Artificial Intelligence (AI) tools-the need for effective query protection deserves further investigation in current research. With respect to the literature, this paper examines whether the use of generative Large Language Models (LLMs) offers a viable solution in light of various state-of-the-art techniques aimed at safeguarding queries from the user’s privacy perspective. In particular, we investigate the effectiveness of different prompts inspired by distinct confusion-based techniques for query protection. Our study assesses how well this solution can protect user privacy while simultaneously maintaining a satisfactory trade-off with retrieval effectiveness.