Differentially Private Linear Regression via Medians

Alexander Knop, Thomas Steinke

Keywords: differential privacy, linear regression, robust statistics

TL;DR: We provide an algorithm for private linear regression which, despite its simplicity, outperforms prior work.

Abstract:

Linear regression is one of the simplest machine learning tasks. Despite much work, differentially private linear regression still lacks effective algorithms.
We propose a new approach based on a multivariate extension of the Theil-Sen estimator.
The theoretical advantage of our approach is that we do not directly rely on noise addition, which requires bounding the sensitivity. Instead we compute differentially private medians as a subroutine, which are more robust.
We also show experimentally that our approach compares favourably to prior work.