\section{Introduction}
\label{sec:intro}

Federated learning (FL; \citep{mcmahan2017communication}) is a popular framework for distributed model training on sensitive user data. Instead of centrally storing the training data, FL operates in a server-client setting where the server hosts the model and has no direct access to clients' data. The clients can apply the model to their private data and send gradient updates back to the server. This learning regime promises data privacy as users share only gradients but never any raw data. 
%do not need to share their raw data with the server.
%and techniques such as secure aggregation~\citep{bonawitz2016practical} can aggregate the gradient updates from all clients before revealing it to the server, further enhancing privacy.
However, recent work~\citep{zhu2019deep, zhao2020idlg, geiping2020inverting} showed that despite these efforts, the server is still able to recover training data from gradient updates, violating the promise of data privacy in FL. These so-called \emph{gradient inversion attacks} operate by optimizing over the input space to search for samples whose gradient matches that of the observed gradient, and such attacks remain effective even when clients utilize secure aggregation~\citep{bonawitz2016practical} to avoid revealing individual gradients~\citep{yin2021see, jeon2021gradient}.

As countermeasures against these gradient inversion attacks, prior work proposed both principled defenses based on differential privacy~\citep{abadi2016deep}, as well as heuristics that compress the gradient update through gradient pruning~\citep{aji2017sparse} or sign compression~\citep{bernstein2018signsgd}. In particular, gradient compression defenses have so far enjoyed great success, severely hindering the effectiveness of existing optimization-based attacks~\citep{zhu2019deep, jeon2021gradient} while maintaining a similar level of model performance.
%Despite their empirical success, optimization-based gradient inversion attacks have two major limitations. First, the attack uses gradient-based search over the input space to reconstruct the training samples, which is difficult to apply to discrete input spaces such as text. Also, gradient compression techniques such as gradient pruning~\citep{aji2017sparse} and sign compression~\citep{bernstein2018signsgd} are often used to reduce the communication cost in FL, which has been shown empirically to severely hinder the effectiveness of optimization-based attacks~\citep{zhu2019deep}.
%Theoretically principled defenses such as adding Gaussian noise to the gradient~\citep{abadi2016deep} can also make optimization-based inversion attacks challenging since the adversary only observes a single stochastic gradient.
As a result, these limitations seemingly diminish the threat of gradient inversion attacks in practical FL applications.

\begin{figure*}[t]
    \centering
    \includegraphics[width=0.88\linewidth]{sections/figs/leak_grad3.pdf}
    \caption{Illustration of federated learning (FL) and gradient inversion methods. The goal of gradient inversion is to recover training data $(\bx, y)$ from the observed gradient $\nabla_{\bw}\ell(f_{\bw}(\bx), y)$. Optimization-based methods (\emph{e.g.}, \citep{zhu2019deep, geiping2020inverting, yin2021see, jeon2021gradient}) directly optimize $(\tilde{\bx}, \tilde{y})$ in search for a sample that produces gradient similar to that of $(\bx, y)$. Our proposed learning-based approach, which we call \emph{Learning to Invert}, instead trains an inversion model $g_\theta$ to reconstruct training samples from their gradient.}
    \label{fig:intro}
\end{figure*}

In this paper, we argue that evaluating defenses on existing optimization-based attacks may provide a false sense of security. To this end, we propose a simple \emph{learning-based} attack---which we call \emph{Learning To Invert} (LTI)---that trains a model to invert gradient updates and recover client samples; see \autoref{fig:intro} for an illustration.
We assume that the adversary (\emph{i.e.}, the server) has access to an \emph{auxiliary dataset} whose distribution is similar to that of the private data. The gradient inversion model trains on samples in the auxiliary dataset, with corresponding gradients provided by the global model. Our attack is highly adaptable to different defense schemes, since applying a defense simply amounts to training data augmentation for the gradient inversion model.
%We hypothesize that learning-based gradient inversion attacks have several advantages:
%\begin{enumerate}[noitemsep, nolistsep, leftmargin=*]
%    \item Optimization-based methods often rely on handcrafted data priors to reduce the search space, such as using the smoothness prior for image data. In contrast, learning-based methods directly learn to output samples on the data manifold using auxiliary data.
%    \item Since learning-based methods directly output training samples given gradients, they circumvent difficulties in optimization such as due to the discrete nature of text tokens.
%    \item Learning-based gradient inversion adapts naturally to compression-based defense mechanisms, which existing optimization-based methods had limited success circumventing.
%\end{enumerate}

We empirically demonstrate that LTI can successfully circumvent defenses based on gradient perturbation (\emph{i.e.}, using differential privacy; ~\citep{abadi2016deep}), gradient pruning~\citep{aji2017sparse} and sign compression~\citep{bernstein2018signsgd} on both vision and language tasks.
\begin{itemize}[noitemsep, nolistsep, leftmargin=*]
    \item Vision: We evaluate on the CIFAR10~\citep{krizhevsky2009learning} classification dataset for both LeNet and ResNet20. LTI attains recovery accuracy close to that of the best optimization-based method when no defense is applied, and significantly outperforms all prior attacks under defense settings.
    \item NLP: We experiment with both text classification task on CoLA~\citep{warstadt2018neural} and causal language model training on the WikiText~\citep{merity2016pointer} dataset, where LTI attains state-of-the-art performance in all settings, with or without defense.
\end{itemize}
Given the strong empirical performance of LTI and its adaptability to different learning tasks and defense mechanisms, we advocate for its use as a simple baseline for future studies on gradient inversion attacks in FL.