Clustered FedStack: Intermediate Global Models with Bayesian Information Criterion

Thanveer Shaik, Xiaohui Tao, Lin Li, Niall Higgins, Raj Gururajan, Xujuan Zhou, Jianming Yong

Abstract: Highlights•A novel Clustered-FedStack framework is proposed to overcome the non-IID challenge in FL.•Improved personalized modeling in FL by building intermediate clustered models between the global and local models.•Using Bayesian information criterion to cluster the local clients and compare three different clustering mechanisms.•The proposed framework scalable for NLP tasks handling a high number of local clients with non-IID data.•Achieved superconvergence of all clustered-FedStack models using Cyclical learning rates.