Data-Free Diversity-Based Ensemble Selection for One-Shot Federated Learning

Naibo Wang, Wenjie Feng, yuchen deng, Moming Duan, Fusheng Liu, See-Kiong Ng

Abstract: The emerging availability of various machine learning models creates a great demand to harness the collective intelligence of many independently well-trained models to improve overall performance. Considering the privacy concern and non-negligible communication costs, one-shot federated learning and ensemble learning in a data-free manner attract significant attention. However, conventional ensemble selection approaches are neither training efficient nor applicable to federated learning due to the risk of privacy leakage from local clients; meanwhile, the "many could be better than all" principle under data-free constraints makes it even more challenging. Therefore, it becomes crucial to design an effective ensemble selection strategy to find a good subset of the base models as the ensemble team for the federated learning scenario. In this paper, we propose a novel data-free diversity-based framework, DeDES, to address the ensemble selection problem with diversity consideration for models under the one-shot federated learning setting. Experimental results show that our method can achieve both better performance and higher efficiency over 5 datasets, 4 different model structures, and both homogeneous and heterogeneous model groups under four different data-partition strategies.

Submission Length: Regular submission (no more than 12 pages of main content)

Changes Since Last Submission: Camera Ready version

Code: https://github.com/NaiboWang/Data-Free-Ensemble-Selection-For-One-Shot-Federated-Learning

Assigned Action Editor: ~Sanghyun_Hong1

License: Creative Commons Attribution 4.0 International (CC BY 4.0)

Submission Number: 1408