SoK: A Review of Differentially Private Linear Models For High Dimensional Data

Amol Khanna, Edward Raff, Nathan Inkawhich

Keywords: differential privacy, high dimensional, linear regression, logistic regression

TL;DR: This paper conducts an empirical review of differentially private optimizers for high dimensional linear models.

Abstract: Linear models are ubiquitous in data science, but are particularly prone to overfitting and data memorization in high dimensions. To combat memorization, differential privacy can be used. Many papers have proposed optimization techniques for high dimensional differentially private linear models, but a systematic comparison between these methods does not exist. We close this gap by providing a comprehensive review of optimization methods for private high dimensional linear models. Empirical tests on all methods demonstrate surprising results which can inform future research. Code for implementing all methods is released in the supplementary material.

Submission Number: 54