Differential Privacy of Cross-Attention with Provable Guarantee
Keywords: Differential Privacy, Cross-Attention, Provable Guarantee
Abstract: Cross-attention has become a fundamental module nowadays in many important artificial intelligence applications, e.g.,
retrieval-augmented generation (RAG), system prompt, guided stable diffusion, and many more.
Ensuring cross-attention privacy is crucial and urgently needed because its key and value matrices may contain sensitive information about model providers and their users.
In this work, we design a novel differential privacy (DP) data structure to address the privacy security of cross-attention with a theoretical guarantee.
In detail, let $n$ be the input token length of system prompt/RAG data, $d$ be the feature dimension,
$R$ be the maximum value of the query and key matrices, $R_w$ be the maximum value of the value matrix, and $r,s,\epsilon_s$ be parameters of polynomial kernel methods.
Then, our data structure requires $\widetilde{O}(ndr^2)$ memory consumption with $\widetilde{O}(ndr^2)$ initialization time complexity and $\widetilde{O}(d r^2)$ query time complexity for a single token query.
In addition, our data structure can guarantee that the process of answering user query satisfies $(\epsilon, \delta)$-DP with
$\widetilde{O}((1-\epsilon_s)^{-1} n^{-1} \epsilon^{-1} R^{2s} R_w r^2)$ additive error and $2\epsilon_s/(1-\epsilon_s)$ relative errorbetween our output and the true answer.
Furthermore, our result is robust to adaptive queries in which users can intentionally attack the cross-attention system.
To our knowledge, this is the first work to provide DP for cross-attention and is promising to inspire more privacy algorithm design in large generative models (LGMs).
Primary Area: learning theory
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 1769
Loading