Federated Many-Task Bayesian Optimization

Hangyu Zhu, Xilu Wang, Yaochu Jin

Abstract: Bayesian optimization (BO) is a powerful surrogate-assisted algorithm for solving expensive black-box optimization problems. While BO was developed for centralized optimization, the availability of massive distributed data has attracted increased interests in exploring federated BO (FBO) that can use data on multiple clients without leaking the raw data. However, existing FBO approaches assume that either all clients jointly solve the same optimization task, or only one client solves one target optimization task by transferring knowledge from others in a federated way, making them unsuited for many real-world applications. In this article, we consider FBO for the scenario where multiple related local black-box tasks associated with different clients are jointly optimized by sharing knowledge between tasks without leaking the data privacy. An efficient federated many-task BO framework is proposed to address not independent and identically distributed (non-IID) data while protecting the data privacy in the federated setting. A novel federated knowledge transfer paradigm is developed for dynamic many-task model aggregation according to a dissimilarity matrix. The dissimilarity is measured based on the rank of the predictions and only the hyperparameters in the local Gaussian process models are shared. In addition, a federated ensemble acquisition function is constructed by integrating the predictions of two surrogates using the global and local hyperparameters, respectively, to effectively search for the optimal solution. Experimental results show that our proposed method has reliable performance on both benchmark problems and a real machine-learning problem also in the presence of non-IID data.

External IDs:dblp:journals/tec/ZhuWJ24