Boosting with Multiple Sources

Corinna Cortes, Mehryar Mohri, Dmitry Storcheus, Ananda Theertha Suresh

Keywords: boosting, multiple sources, generalization bounds, federated learning

TL;DR: Novel multi-source boosting algorithm with domain-weighted combination of weak learners.

Abstract:

We study the problem of learning accurate ensemble predictors, in
particular boosting, in the presence of multiple source domains. We
show that the standard convex combination ensembles in general
cannot succeed in this scenario and adopt instead a domain-weighted
combination. We introduce and analyze a new boosting algorithm,
MULTIBOOST, for this scenario and show that it benefits from
favorable theoretical guarantees. We also report the results of
several experiments with our algorithm demonstrating that it
outperforms natural baselines on multi-source text-based,
image-based and tabular data. We further present an extension of our
algorithm to the federated learning scenario and report favorable
experimental results for that setting as well. Additionally, we
describe in detail an extension of our algorithm to the multi-class
setting, MCMULTIBOOST, for which we also report
experimental results.

Code Of Conduct: I certify that all co-authors of this work have read and commit to adhering to the NeurIPS Statement on Ethics, Fairness, Inclusivity, and Code of Conduct.