Differentially Private Principal Component Analysis for Vertically Partitioned Data

Ergute Bao, Fei Wei, Xiaokui Xiao, Tianyu Pang, Chao Du

Primary Area: societal considerations including fairness, safety, privacy

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: Differential Privacy, Principal Component Analysis

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

TL;DR: A differentially private mechanism for principal component analysis, in vertical federated learning.

Abstract: We study the problem of differentially private principal component analysis (DP PCA) for vertically partitioned data. In this setting, an untrusted server wants to learn the optimal rank-$k$ subspace of an underlying sensitive dataset $D$, which is partitioned among multiple clients by attributes/columns. While differential privacy has been heavily studied for horizontally partitioned data (namely, when $D$ is partitioned among clients by records/rows), its applications on vertically partitioned data are very limited. To fill this gap, we propose SPCA, which introduces minimal noise to the obtained subspace while preserving DP without assuming any trusted client or third party. The theoretical analysis shows that our solution is able to match the privacy-utility trade-off of the optimal baseline in the centralized setting. Finally, we provide experiments on real-world datasets to validate the theoretical analysis.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 5097