All Federated or Not: Optimizing Personal Model Performance in Cross-silo Federated Learning

Juan Li, Yanmin Zhu, Jie Wu, Weifan Wu, Tianzi Zang, Liu Lu

Abstract: In cross-silo federated learning (FL), organizations cooperatively train a global model with their local data. The organizations, however, may be heterogeneous in terms of data distributions. In such cases, FL might produce a biased global model that is not optimal for each organization. Then each organization faces several fundamental questions: should I join FL or just remain alone? If joining FL, which organizations should I cooperate with? In this work, we formulate a coalition formation game in cross-silo FL to help organizations choose proper cooperators. We first build an estimation method to predict personal model performance for each organization before FL starts, and we treat performance improvement as individual utility. With estimated utilities, we design a distributed coalition formation algorithm to find stable coalition structures and optimize social welfare at the same time. Our simulations based on MNIST and FMNIST datasets show that the estimation model can predict the sign of the utility correctly with a probability of 0.9 and has an average relative error of $30 \%$. With the above errors, the obtained coalition structure performs well from both perspectives of real social welfare and individual satisfaction.