Identifying Interactions among Categorical Predictors with Monte-Carlo Tree Search

Tan Zhu, Fei Do, Chloe Becquey, Jinbo Bi

Keywords: interaction identification, Monte Carlo tree search, decision tree

Abstract: Identifying interpretable interactions among categorical predictors for predictive modeling is crucial in various research fields. Recent studies have examined interpretable interactions using decision tree (DT) learning methods, which construct DTs by greedy rules due to the high memory and time complexity of building and evaluating DTs, resulting in a local optimal solution. This paper formulates the selection of quadratic and higher order interactive terms into a LASSO problem and then relaxes it into multiple DT learning problems. A Monte Carlo Tree Search-based interaction selection (MCTs-IS) method is proposed to identify the optimal DT in an online learning manner. A DT pruning strategy is developed based on LASSO that can easily be applied to MCTs. We prove that MCTs-IS converges with high probability to the optimal solution of the DT learning problem. Extensive experiments have been conducted to demonstrate the effectiveness of the proposed algorithm on real-world datasets.

One-sentence Summary: We present a Monte-Carlo tree search-based method to select interactions among features, and provide theoretical and empirical results.