Tackling Spatial-Temporal Data Heterogeneity for Federated Continual Learning in Edge Networks

Junyu Shi, Kun Guo, Xijun Wang, Peng YANG, Howard Hao Yang

Keywords: Federated continual learning, Spatial-Temporal data heterogeneity, elastic weight consolidation.

TL;DR: A federated continual learning method to address spatiotemporal data heterogeneity in edge networks.

Abstract: With the rapid growth of intelligent devices such as smartphones and unmanned aerial vehicles, vast amounts of sequential data are generated at the network edge, offering rich resources for edge federated continual learning. However, the continuous influx of data introduces significant spatiotemporal heterogeneity: temporally, data distribution shifts over time lead to catastrophic forgetting; spatially, non-independent and identically distributed (non-IID) data across devices hinder global model convergence. While overcoming these challenges, it is inevitable to consider the inherent constraints of edge devices, including limited computational and storage capability. To this end, we propose Spatial-Temporal Elastic Weight Consolidation (ST-EWC) method, by which each device trains a local neural network model using only its own data within the current time period, without revisiting data from other devices and previous time periods, meanwhile local models are periodically sent to a server for global model aggregation. The key point of ST-EWC is that the local model update is guided by Fisher diagonal matrices based regularization terms applied across both spatial and temporal dimension. Experimental results demonstrate that ST-EWC significantly mitigates catastrophic forgetting, accelerates convergence, and improves average accuracy, under the settings of the temporally domain-incremental and spatially non-IID PermutedMNIST and PACS datasets.

Submission Number: 32