BONSAI: Depth-Constrained Decision Tree Induction via Approximation Optimization

Sungbum Jun, Cheolwoo Lee

Keywords: Decision Trees, Approximation Optimization, Mixed-Integer Programming

TL;DR: We propose BONSAI, a decision tree algorithm that improves predictive accuracy and interpretability by optimizing numerical splits under depth constraints.

Abstract: We propose BONSAI (Branch Optimization for Numerical Splits for Accurate and Interpretable models), a novel decision tree algorithm that optimizes split decisions over numerical attributes under a depth limit. In contrast to conventional greedy methods such as CART and C4.5, which construct trees via locally top-down splits, BONSAI formulates tree induction as an approximation optimization problem based on a new slot-node structure that can deal with both categorical and numerical features. To improve the performance and interpretability, BONSAI combines a compact optimization formulation with efficient search-space reduction techniques, thereby avoiding the combinatorial overhead of traditional mixed-integer programming (MIP) approaches. Empirical evaluations on benchmark datasets show that BONSAI achieves superior predictive accuracy, model compactness, and interpretability compared to both greedy and existing optimization-based methods.

Primary Area: interpretability and explainable AI

Submission Number: 1369