NIRo: A Metric to capture non-iid robustness for Federated Learning Algorithms

Anupam Gupta, Pabitra Mitra

Keywords: federated learning, non-iid federated learning, edge intelligence, protocol comparison

TL;DR: This paper introduces NIRo, a metric to measure non-iid robustness for federated learning algorithms in order to ease comparative analysis of FL protocols.

Abstract: Federated Learning (FL) is a collaborative machine learning framework for multiple, individually trained decentralized nodes, often with non-iid distributed private data, to create a globally un-biased, high performing central model. Majority of the proposed federated learning protocols test and report their performance in varied non-iid settings. Heterogeneity in non-iid descriptions in each protocol paper makes it very hard to compare the robustness of approaches to other studied approaches in differing settings. In this paper, we define a metric, NIRo, to capture data-quantity and data label-skewness and use it to propose a cumulative area-under-the-curve metric that can be used to quantify the robustness of federated learning protocols in varied non-iid settings.

Submission Number: 236