Federated Bayesian Optimization based on Secure Distributed Gaussian Processes

Adil Nawaz, Songlei Wang, Jianqiang Li, Jie Chen

Keywords: Bayesian Optimization, Federated Bayesian Optimization, Secure Gaussian Processes, Homomorphic Encryption

Abstract: Bayesian optimization (BO) is a powerful framework for tuning expensive black-box functions, yet existing federated BO (FBO) methods either expose private data or rely on noise-based privacy mechanisms that degrade performance. We introduce FBO-FedGP, the first secure FBO algorithm that combines multiparty homomorphic encryption with a distributed sparse Gaussian process surrogate. Each client computes and encrypts local summary statistics on a shared support set, which are then aggregated homomorphically by the server into a global surrogate without ever accessing raw data. Clients adaptively balance exploration of the global surrogate with exploitation of their local posteriors via a monotonically increasing mixing schedule, ensuring both efficiency and personalization. We provide theoretical analysis showing that FBO-FedGP achieves sublinear cumulative regret under standard kernel assumptions, with explicit bounds linking regret to the support-set approximation error. Extensive experiments on 20 synthetic benchmarks and several real-world tasks show that FBO-FedGP consistently outperforms state-of-the-art privacy-preserving FBO baselines, improvements are statistically significant under paired Wilcoxon tests with Holm correction. Our framework also includes a concrete HE instantiation with empirically verified $128$-bit security and negligible numerical error, demonstrating that strong privacy can be achieved without sacrificing accuracy. This work establishes a practical and scalable solution for secure collaborative zeroth-order optimization in federated environments.

Primary Area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)

Submission Number: 13352