FedEntropy: Efficient Federated Learning for Non-IID Scenarios Using Maximum Entropy Judgment-based Client Selection

Zhiwei Ling, Zhihao Yue, Jun Xia, Ting Wang, Mingsong Chen, Xiang Lian

Abstract: Although various techniques have been proposed for Federated Learning (FL) to address its problem of low classification accuracy in non-IID scenarios, most of them neglect both i) distinct data distribution characteristics of heterogeneous devices (i.e., clients), and ii) benefits and hazards of local models for global model aggregation. In this paper, we present FedEntropy, an efficient FL approach with a novel two-stage dynamic client selection scheme that fully takes the above two factors into account. Unlike existing FL methods, FedEntropy firstly selects clients with high potential for benefiting global model aggregation in a coarse manner, and then further filters out inferior clients from such selected clients by using our proposed maximum entropy judgment method. Based on the pre-collected soft labels of the selected clients, FedEntropy only aggregates those local models that can maximize the overall entropy of their soft labels, thus effectively improving global model accuracy while reducing the overall communication overhead. Comprehensive experimental results on well-known benchmarks demonstrate both the superiority of FedEntropy and its compatibility with state-of-the-art FL methods.