Data-Free Diversity-Based Ensemble Selection for One-Shot Federated Learning

Published: 27 Nov 2023, Last Modified: 27 Nov 2023Accepted by TMLREveryoneRevisionsBibTeX
Abstract: The emerging availability of various machine learning models creates a great demand to harness the collective intelligence of many independently well-trained models to improve overall performance. Considering the privacy concern and non-negligible communication costs, one-shot federated learning and ensemble learning in a data-free manner attract significant attention. However, conventional ensemble selection approaches are neither training efficient nor applicable to federated learning due to the risk of privacy leakage from local clients; meanwhile, the "many could be better than all" principle under data-free constraints makes it even more challenging. Therefore, it becomes crucial to design an effective ensemble selection strategy to find a good subset of the base models as the ensemble team for the federated learning scenario. In this paper, we propose a novel data-free diversity-based framework, DeDES, to address the ensemble selection problem with diversity consideration for models under the one-shot federated learning setting. Experimental results show that our method can achieve both better performance and higher efficiency over 5 datasets, 4 different model structures, and both homogeneous and heterogeneous model groups under four different data-partition strategies.
License: Creative Commons Attribution 4.0 International (CC BY 4.0)
Submission Length: Regular submission (no more than 12 pages of main content)
Changes Since Last Submission: Camera Ready version
Supplementary Material: pdf
Assigned Action Editor: ~Sanghyun_Hong1
Submission Number: 1408