Gradient Tree Boosting for Regression Transfer

Dag Björnberg, Jonas Nordqvist, Morgan Ericsson, Johan Lindeberg, Welf Löwe, Johan E.S. Fransson

Abstract: Many real-world modeling problems are hindered by limited data availability. In such cases, \emph{transfer learning} leverages related source domains to improve predictions in a target domain of interest. We extend the classical gradient tree boosting paradigm to a regression transfer algorithm by modeling the weak learner as a sum of two regression trees. The trees are fitted on source data and target data, respectively, and jointly optimized for the target data. We derive optimal coefficients for the model update under the least-squares, the least-absolute-deviation, and the Huber loss functions. We benchmark our approach against boosting-based regression transfer methods in twelve transfer scenarios. The results indicate that our approach constitutes a competitive alternative within the realm of boosting-based regression transfer. Moreover, we provide a theoretical justification as well as empirical validation that our approach is robust under larger domain shifts.

Submission Type: Regular submission (no more than 12 pages of main content)

Code: https://github.com/DageBjorne/transfertreeboost

Assigned Action Editor: ~Santiago_Mazuelas1

Submission Number: 7551