On Rate-Optimal Partitioning Classification from Observable and from Privatised Data

TMLR Paper7288 Authors

Abstract: In this paper we revisit the classical method of partitioning classification and prove novel convergence rates under relaxed conditions, both for observable (non-privatised) and for privatised data. We consider the problem of classification in a $d$ dimensional Euclidean space. Previous results on the partitioning classifier worked with the strong density assumption (SDA), which is restrictive, as we demonstrate through simple examples. Here, we study the problem under much milder assumptions. We presuppose that the distribution of the inputs is a mixture of an absolutely continuous and a discrete distribution, such that the absolutely continuous component is concentrated on a $d_a$ dimensional subspace. In addition to the standard Lipschitz and margin conditions, a novel characteristic of the absolutely continuous component is introduced, by which the convergence rate of the classification error probability is computed, both for the binary and for the multi-class cases. This bound can reach the minimax optimal convergence rate achievable using SDA, but under much milder distributional assumptions. Interestingly, this convergence rate depends only on the intrinsic dimension of the continuous inputs, $d_a$, and not on $d$. Under privacy constraints, the data cannot be directly observed, and the constructed classifiers are functions of the randomised outcome of a suitable local differential privacy mechanism. In this paper we add Laplace distributed noises to the discretisations of all possible locations of the feature vector and to its label. Again, tight upper bounds on the convergence rate of the classification error probability can be derived, without using SDA, such that this rate depends on $2d_a$.

Submission Type: Regular submission (no more than 12 pages of main content)

Previous TMLR Submission Url: https://openreview.net/forum?id=esy0F0QshW

Changes Since Last Submission:

Beside de-anonymizing the manuscript (adding author information and funding acknowledgments), we have implemented the refinements suggested by the reviewers:
- Page 3: Explicitly stated the primary goal of the paper immediately following Equation (2).
- Page 6: Added a remark after Theorem 2.2 outlining the main steps of the proof, and included a discussion regarding the bounded support of the inputs.
- Page 8: Added a note justifying the choice of the Laplace distribution for LDP, along with a brief discussion on computational complexity.
- Page 9: Inserted a discussion after Theorem 3.3 outlining the main steps of the proof, focusing specifically on the modifications required relative to the non-privatised case.

Assigned Action Editor: ~Aurélien_Bellet1

Submission Number: 7288