Structured Federated Aggregation for Personalizing On-device Intelligence

Fengwen Chen, Guodong Long, Tianyi Zhou, Zonghan Wu, Jing Jiang

Keywords: Federated Learning, Structure Aggregation, Personalisation, Graph Neural Network

Abstract: Personalizing on-device intelligence with privacy-preserving is an emerging requirement for the Mobile Internet and many other service areas. The recent development of federated learning is to embody personalization by tackling statistical heterogeneity across devices. However, these methods ignore the structural information between clients which can indicate a similar behavior pattern or decision logic among clients who are connected to each other in a graph. For example, the traffic condition is very similar to its adjacent blocks. Motivated by this assumption, we propose structured federated learning(SFL) to update each device's personalized model by leveraging its neighbors' local model. This problem has been formulated to a new optimization problem to integrate the prediction loss, federated aggregation, and structured aggregation into a unified framework. Moreover, it could be further enhanced by adding the structure learning component to learn the relation graph in the same optimization framework. The effectiveness of the proposed method has been demonstrated in experimental analysis by comparing it with other baselines in public datasets.

One-sentence Summary: The paper proposes a novel structured federated aggregation method to develop personalising on-device intelligence.