A tree regression algorithm based on incremental gradient boosting

Hui Zhang, WeiWen Wu, Chunming Yang, Bo Li

Abstract: Gradient boosting is an efficient and scalable supervised machine learning technique, and most scaling models based on gradient boosting perform well on point regression tasks, but they can only be run in batch settings and cannot be learned online on the data stream. To solve this problem, this paper proposes a tree regression method based on incremental gradient boosting (IGB). The proposed method uses gradient information as a splitting metric and applies the Hoeffding inequality incrementally to construct decision trees to achieve incremental gradient improvement. After a large number of experimental evaluations, the proposed method outperforms the existing online regression trees in point regression tasks, and in some cases, has comparable performance to the batch gradient boosting tree model.