DyGB: Dynamic Gradient Boosting Decision Trees with In-Place Updates for Efficient Data Addition and Deletion

ICLR 2026 Conference Submission14741 Authors

Keywords: Dynamic Learning, Decremental Learning, Incremental Learning, Machine Unlearning, Online Learning, Gradient Boosting Decision Trees

TL;DR: We propose DyGB (Dynamic GBDT), a novel framework that enables efficient support for both incremental and decremental learning within GBDT.

Abstract: Gradient Boosting Decision Tree (GBDT) is one of the most popular machine learning algorithm in various applications. However, in the traditional settings, all data should be simultaneously accessed in the training procedure: it does not allow to add or delete any data instances after training. In this paper, we propose DyGB (Dynamic GBDT), a novel framework that enables efficient support for both incremental and decremental learning within GBDT. To reduce the learning cost, we present a collection of optimizations for DyGB, so that it can add or delete a small fraction of data on the fly. We theoretically show the relationship between the hyper-parameters of the proposed optimizations, which enables trading off accuracy and cost on incremental and decremental learning. Empirical results on backdoor and membership inference attacks demonstrate that DyGB can effectively add and remove data from a well-trained model through incremental and decremental learning. Furthermore, experiments on public datasets validate the effectiveness and efficiency of the proposed DyGB framework and optimizations.

Primary Area: other topics in machine learning (i.e., none of the above)

Submission Number: 14741