Feature Interactions in XGBoost

Kshitij Goyal, Sebastijan Dumancic, Hendrik Blockeel

Abstract: In this paper, we investigate how feature interactions can be identified to be used as constraints in the gradient boosting tree models using XGBoost's implementation. Our results show that accurate identification of these constraints can help improve the performance of baseline XGBoost model significantly. Further, the improvement in the model structure can also lead to better interpretability.