\subsection{Membership Inference Attacks on Federated Training}\label{subsec:federated_result}

\begin{figure}[htbp]
  \begin{floatrow}
    \ffigbox[0.4\textwidth]{%
      \centering
      \includegraphics[width=0.4\textwidth]{figures/attack_time_series.pdf}
    }{%
      \caption{%
        Increasing %
        attack vulnerability per federation round.
      } \label{fig:federated_training_evolution}
    }
    \capbtabbox{%
      \setlength\tabcolsep{3pt}
      \centering
      \begin{tabular}{lcc}
        \toprule
        {Data distribution}  & \texttt{3D-CNN}   & \texttt{2D-slice-mean} \\
        \cmidrule(r){1-1}      \cmidrule(lr){2-2}  \cmidrule(lr){3-3}
        Uniform \& IID       & 60.06 (56)        & 58.11 (56)  \\
        Uniform \& non-IID   & 61.00 (28)        & 60.28 (29)  \\
        Skewed \& non-IID    & 64.12 (25)        & 63.81 (24)  \\
        \bottomrule
      \end{tabular}
    }%
    {%
      \caption{%
        Average attack accuracies on federation trained models. Numbers in parentheses indicate median successful attacks over 5 multiple runs.
        }%
      \label{tab:federated_training}
    }
  \end{floatrow}
\end{figure}



We consider three different federated learning environments consisting of 8 learners and investigate cases where malicious learners attack the community model. The community model is the aggregated result of learners' local models and a malicious learner may use it to extract information about other learners' training samples. In this scenario, a malicious learner can learn an attack model by leveraging its access to the community models of all federation rounds and its local training dataset; we simulate attacks using this information (see also \sectionref{subsec:attack_setup}). The model vulnerability is likely to increase with more training iterations and hence we used features derived from the community models received during the last five federation rounds, and each learner uses its private samples to learn the attack model. Each learner may try to do membership inference attacks on any of the other seven learners, resulting in 56 possible attack combinations. An attack is considered successful if accuracy is more than 50\%, which is the random prediction baseline.


\tableref{tab:federated_training} shows the average accuracy of successful attacks and the total number of successful {attack instances of learner-attacker pairs} (in parentheses) across all possible learner-attacker pairs (56 in total). For a more detailed analysis on a per-learner basis, see \appendixref{sec:appendix_federated_attack_result}. We empirically observed that the success rate of the attacks is susceptible to data distribution shifts. In particular, distribution shift agnostic features like gradient magnitudes can lead to more successful attacks (count wise) when data distribution across learners differs. For the results shown in \tableref{tab:federated_training} and \figureref{fig:federated_training_evolution}, we used all available features (i.e., gradient magnitudes, predictions, labels, and gradients of last layers).

We also observe that the overall attack accuracies are lower than the centralized counterpart discussed in \sectionref{subsec:centralized_result}. This drop can be attributed to the following: a) As we show in \sectionref{subsec:defenses}, attack accuracies are highly correlated with overfitting. Federated learning provides more regularization than  centralized training and reduces overfitting but does not eliminate the possibility of an attack. b) Federated models are slow to train, but as the model is trained for more federation rounds, the vulnerability increases (see \figureref{fig:federated_training_evolution}). Moreover, \tableref{tab:federated_training} only presents an average case view of the attacks and we observe that the attack performance depends on the data distribution of the learner-attacker pair. When the local data distribution across learners is highly diverse, i.e., \emph{Skewed \& non-IID}\, attack accuracies can be as high as 80\% for specific learner-attacker pairs (see \appendixref{sec:appendix_federated_attack_result}).


