Gradient Tree Boosting for Regression Transfer

TMLR Paper7551 Authors

Abstract: Many real-world modeling problems are hindered by limited data availability. In such cases, *transfer learning* leverages related source domains to improve predictions in a target domain of interest. We extend the classical gradient tree boosting paradigm to a regression transfer algorithm by modeling the weak learner as a sum of two regression trees. The trees are fitted on source data and target data, respectively, and jointly optimized for the target data. We derive optimal coefficients for the model update under the least-squares, the least-absolute-deviation, and the Huber loss functions. We benchmark our approach against the widely used XGBoost algorithm in several transfer scenarios, achieving superior performance in seven out of eight cases.

Submission Type: Regular submission (no more than 12 pages of main content)

Assigned Action Editor: ~Santiago_Mazuelas1

Submission Number: 7551