Uncertainty Quantification in Federated Learning for Heterogeneous Health Data
Keywords: Uncertainty quantification, federated learning, healthcare, data heterogeneity, personalization
TL;DR: In this paper, we present the first evaluation of the quality of uncertainty in realistic healthcare FL settings, revealing the effectiveness of personalization.
Abstract: Safety-critical healthcare applications require deep learning to be high-performing and uncertainty-aware, yet this is challenging due to the insufficient data from individual data collectors. The recently emerged Federated learning (FL) allows collaborative training without transferring sensitive data to a central server, overcoming the barrier between data aggregation and model performance. Since FL is achieved by globally synchronizing the models learned locally, the heterogeneity in local health data, arising from variations in technologies, patient demographics, and disease prevalence, presents a significant challenge to FL and correspondingly to uncertainty quantification. It is unclear how reliable the uncertainty is to infer the confidence of the diagnoses made by the FL model.
In this paper, we present the first evaluation of the quantification of uncertainty in realistic healthcare FL settings. Our experiments on real-world applications cover tabular data-based heart disease prediction, image-driven skin pathology screening, and physiological signal-based activity detection tasks. Three uncertainty quantification methods that were previously proposed in standard centralized deep learning are adapted to a variety of FL algorithms for comparison. We found that federated deep ensembles perform consistently better than other federated uncertainty quantification methods, and personalization, i.e., training collaboratively but remaining customized models, can further enhance the performance (with an improvement by up to 19%). Our work paves the way for the future development of federated uncertainty quantification approaches.
1 Reply
Loading