Convex Optimization with Local Label Differential Privacy: Tight Bounds in All Privacy Regimes

TMLR Paper7008 Authors

Abstract: We study the problem of Stochastic Convex Optimization (SCO) under the constraint of local Label Differential Privacy (L-LDP). In this setting, the features are considered public, but the corresponding labels are sensitive and must be randomized by each user locally before being sent to an untrusted analyzer. Prior work for SCO under L-LDP (Ghazi et al., 2021) established an excess population risk bound with a *linear* dependence on the size of the label space, $K$: $O\left(\frac{K}{\epsilon\sqrt{n}}\right)$ in the high-privacy regime ($\epsilon \leq 1$) and $O\left(\frac{K}{e^{\epsilon} \sqrt{n}}\right)$ in the medium-privacy regime ($1 \leq \epsilon \leq \ln K$). This left open whether this linear cost is fundamental to the L-LDP model. In this note, we resolve this question. First, we present a novel and efficient non-interactive L-LDP algorithm that achieves an excess risk of $O\left(\sqrt{\frac{K}{\epsilon n}}\right)$ in the high-privacy regime ($\epsilon \leq 1$) and $O\left(\sqrt{\frac{K}{e^{\epsilon} n}}\right)$ in the medium-privacy regime ($1 \leq \epsilon \leq \ln K$). This quadratically improves the dependency on the label space size from $O(K)$ to $O(\sqrt{K})$. Second, we prove a matching information-theoretic lower bound across all privacy regimes for any sufficiently large $n$.

Submission Type: Regular submission (no more than 12 pages of main content)

Previous TMLR Submission Url: https://openreview.net/forum?id=JL481t3BBl

Changes Since Last Submission: The previous submission (number 6989) was desk rejected due to incorrect format. We fixed the formatting issue in this submission.

Assigned Action Editor: ~Antti_Koskela1

Submission Number: 7008