Data-weighted ensemble learning for privacy-preserving distributed learning

Liyang Xie, Sergey M. Plis, Anand D. Sarwate

Abstract: In collaborative medical research settings, a moderate number of groups (sites) may wish to merge local analyses of private subject data. Differential privacy offers one way to guarantee privacy for these local analyses. We describe a novel ensemble learning method that we call the "feature method" for aggregating binary classifiers or regressors trained on local data. Our method leverages a public data set available at the aggregator to optimize a linear combination of local predictors. We provide some analysis of the method and show how it is effective when the local sites are required to learn classifiers that are differentially private. We prove that this method has near-optimal performance when local data sets are large enough under certain requirements on the parameters. Experimentally, we give a comparison of the feature method and the standard approach of averaging the local classifiers.