Go to TMLR homepageDifferentially Private Conformal Prediction via Quantile Binary Search
Abstract: Differentially Private (DP) approaches have been widely explored and implemented for a broad variety of tasks delivering corresponding privacy guarantees in these settings. While most of these DP approaches focus on limiting privacy leakage from training data, there are fewer approaches that consider leakage when procedures involve \textit{calibration data} which is common in uncertainty quantification through Conformal Prediction (CP). Since there is a limited amount of approaches in this direction, in this work we deliver a general DP approach for CP that we call Private Conformity via Quantile Search (P-COQS). The proposed approach adapts an existing randomized binary search algorithm for computing DP quantiles in the calibration phase of CP thereby guaranteeing privacy of the consequent prediction sets. This however comes at a price of marginally under-covering with respect to the desired $(1 - \alpha)$-level when using finite-sample calibration sets (although broad empirical results show that the P-COQS generally targets the required level in the considered cases). Confirming properties of the adapted algorithm and quantifying the approximate coverage guarantees of the consequent CP, we conduct extensive experiments to examine the effects of privacy noise, sample size and significance level on the performance of P-COQS compared to existing alternatives. In addition, we empirically evaluate our approach on several benchmark datasets, including CIFAR-10, ImageNet and CoronaHack. Our results suggest that the proposed method is robust to privacy noise and performs favorably with respect to the current DP alternative in terms of \textit{empirical coverage}, \textit{efficiency}, and \textit{informativeness}. Specifically, the results indicate that P-COQS produces smaller conformal prediction sets while simultaneously targeting the desired coverage and privacy guarantees in all these experimental settings.
Submission Length: Long submission (more than 12 pages of main content)
Changes Since Last Submission: We thank all reviewers for their constructive feedback. The manuscript has been substantially improved. Below we summarize the main changes.
1. Notation and Clarity Improvements
- Replaced $\delta$ (search resolution in P-COQS) with $\Delta$ to avoid confusion with the ($\epsilon$, $\delta$) definition of differential privacy.
- Reviewed and clarified notation throughout the manuscript (bounds, privacy parameters, symbols used in proofs, etc.).
- Clarified that bounds [a,b] for the quantile search must either be known a priori, natural to the problem (e.g., classification), or estimated privately.
2. Correction and Strengthening of Theoretical Results
(a) Corrected rank-error expression (Proposition 1)
- The rank error $τ$ now correctly includes an additional term $M(\Delta)$ accounting for the number of calibration points inside the final search interval of width $\Delta$.
- Updated Proposition 1, its proof, and surrounding remarks to reflect this correction.
(b) New coverage-correction result
- Added a corollary showing how to adjust the nominal level α to guarantee exact $(1−\alpha)$ coverage with probability $1−\beta$, bringing P-COQS closer to ExponQ’s form of guarantee.
(c) Added remarks on bounds, error scaling, and choice of $\Delta$
- New remarks provide guidance for selecting [a,b], the effect of $\Delta$ on runtime and rank error, and why $\Delta$ can safely be chosen very small.
(d) Clarified that our guarantee is marginal coverage
- Added explicit discussion (as requested) that P-COQS, like standard CP, guarantees marginal coverage—not slice-wise or covariate-shift-robust coverage.
3. Expanded Comparisons and New Experimental Baselines
(a) Added a new DP baseline: Histogram + Laplace / DP-CDF (“HistLap”)
- Implemented following reviewer suggestions.
- Included in all simulation tables and discussed throughout Section 5.
- Highlighted strengths/weaknesses relative to P-COQS and ExponQ.
(b) Ablation study on the effect of $\Delta$
- Added a new ablation analysis (Appendix D) exploring performance across a wide range of $\Delta$ values ($10^{-20} to 10^{-1}$) for both Random Forest and Naive Bayes.
- Added summary discussion in the main experiments section.
(c) Clarified model used in main text
- Random Forest results are now the main focus, as they better highlight differences among the DP quantile mechanisms.
- Naive Bayes results are retained in the appendix.
(d) Added explanations for CIFAR-10 “Standard” baseline
- Clarified that “Standard” refers to non-private split conformal prediction.
(e) Added simulation results on rank error
- Varied simulation budgets to study magnitude of rank errors of the different DP quantile methods.
4. Privacy Accounting Improvements
- Added conversions between zCDP budgets ($\rho$) and their corresponding ($\epsilon$, $\delta$) values wherever relevant in the experiments section.
- Clarified overall privacy composition for model training + CP step.
5. Discussion of Membership-Inference Attacks (MIAs)
- Added a new remark explaining the role of MIAs:
- DP already upper-bounds any MIA success probability by definition.
- MIAs can nevertheless be used as empirical diagnostics.
- Explained why a full MIA evaluation was not included but is a natural extension.
6. Algorithmic Complexity
- Added a new proposition giving the computational complexity of P-COQS (binary search: O(log((b−a)/Δ)) noisy count queries).
- Added short proof sketch and explanation comparing qualitatively to ExponQ.
7. Experimental Section Improvements
- Streamlined introductory text for readability.
- Updated all tables to ensure formatting consistency and clearer comparisons.
- Added several presentation fixes (column ordering, captions, tie-handling clarification, etc.).
8. Minor Fixes
- Corrected minor inconsistencies in Theorem 1 notation.
- Clarified midpoint and return logic in Algorithm 1.
- Ensured all figures and tables reference updated notation and baselines.
Overall
We have attempted to address all of the reviewers concerns and are happy to address any further comments they may have. We thank them for the constructive feedback which allowed us to significantly improve the quality of the manuscript.
Assigned Action Editor: ~Antti_Koskela1
Submission Number: 5832
Loading