Federated Continual Learning with Differentially Private Data Sharing

Giulio Zizzo, Ambrish Rawat, Naoise Holohan, Seshu Tirupathi

Keywords: Continual Learning, Differential Privacy, Machine Learning, Federated Learning

TL;DR: Though use of differential privacy, we share privatised data statistics between clients to prevent catastrophic forgetting in Federated Continual Learning

Abstract:

In Federated Learning (FL) many types of skews can occur, including uneven class distributions, or varying client participation. In addition, new tasks and data modalities can be encountered as time passes, which leads us to the problem domain of Federated Continual Learning (FCL).
In this work we study how we can adapt some of the simplest, but often most effective, Continual Learning approaches based on replay to FL. We focus on temporal shifts in client behaviour, and show that direct application of replay methods leads to poor results. To address these shortcomings, we explore data sharing between clients employing differential privacy. This alleviates the shortcomings in current baselines, resulting in performance gains in a wide range of cases, with our method achieving maximum gains of 49%.

Is Student: No