\section{Differential Privacy}
\label{sec:differential_privacy}
\begin{figure}[t!]
    \centering
    \includegraphics[width=0.6\textwidth]{figures/defense/r2_vs_attack.pdf}
    \caption{Attack Accuracy vs.\ $R^2$ for models trained with differential privacy. Error bars are generated by bootstrapping the test set 5 times using 1000 samples.}
    \label{fig:r2_attack_dp}
\end{figure}


\begin{figure}[t!]
    \centering
    \includegraphics[width=0.6\textwidth]{figures/defense/MAE_vs_epsilon.pdf}
    \caption{Performance (MAE) vs.\ $\epsilon$ at $\delta=1e^{-6}$ for models trained with differential privacy. Differential privacy is a very strong notion of privacy. It destroys the performance to achieve any non-vacuous privacy guarantees.  Error bars are generated by bootstrapping the test set 5 times using 1000 samples.}
    \label{fig:epsilon_mae}
\end{figure}

Differential privacy was initially proposed as a mathematical framework to secure information about individual records while releasing group or aggregate query results on a database.  It is adapted to machine learning by considering model parameters as the output and training dataset as the database. Differential private machine learning aims to learn a parametric model so that the final parameters do not differ much if trained on another dataset differing from the original dataset by a single sample. The privacy parameter $\epsilon$ quantifies the difference, and lower $\epsilon$ is more private~\cite{dwork2014algorithmic,abadi2016deep}.  More formally, a randomized algorithm $\mathcal A: \mathcal D \rightarrow \mathcal W$ is $(\epsilon,\delta)$-differential private, if for any two adjacent inputs $d,d'\in \mathcal D$.
\[Pr[\mathcal A(d) \in w] \leq e^\epsilon Pr[\mathcal A(d')\in w] + \delta \ \ \ \ \forall \ w\in \mathcal W \]
For non-vacuous guarantees, $\epsilon$ is desired to be lower, usually less than 1. However, there is no standard agreement on how much is sufficiently small~\cite{laud2019interpreting}. $\delta$ depends on the dataset size and is desired to be less than $N^{-1}$, where N is the dataset size.
In the specific case of supervised deep learning, output is the neural network parameters, input is the labeled dataset, and algorithm is the training algorithm (usually some variant of SGD). Intuitively, to ensure strong privacy guarantees, it is desired to minimize a single training sample's influence on the neural network parameters.

We have used differential private version of SGD (DP-SGD) proposed by \citet{abadi2016deep} which achieves privacy guarantees by adding Gaussian random noise to the gradients of each sample and implemented in \texttt{pytorch-opacus}\footnote{\url{https://github.com/pytorch/opacus}}. This procedure avoids learning too much information about a single sample, thus providing privacy guarantees. In practice, we have used the \texttt{Adam} variant of DP-SGD with a learning rate of $5e^{-5}$, emulating the same training setup as \citet{gupta2021improved}.

Differential privacy assumes a powerful and worst-case adversary, which may be unrealistic. We find that to achieve non-vacuous privacy guarantees ($\epsilon<100$) with differential privacy amounted to losing the performance altogether on the brain age prediction problem (see \figureref{fig:epsilon_mae}). However, even with vacuous guarantees, we see that differential privacy could reduce the vulnerability to realistic membership inference attacks as shown in \figureref{fig:mae_dp} and \figureref{fig:r2_attack_dp}.