Online Decision Tree Construction with Deep Reinforcement Learning

Ayman Chaouki, Jesse Read, Albert Bifet

Keywords: Data Stream Learning, Reinforcement Learning, Decision Trees

Abstract: Decision Trees are widely used in machine learning and data mining tasks, mainly because they can be easily interpreted; due to their popularity, Decision Trees were adapted for settings with streams of data, commonly in the form of Hoeffding Trees. While these methods are fast and incremental, they are also greedy in the sense that they optimise multiple local criteria (generally based on Entropy or Gini impurity) which makes them prone to suboptimality with respect to a global objective metric. On the other hand, Reinforcement Learning (RL) aims at maximizing a long term objective, and as such, it is a good candidate for alleviating this suboptimality problem of the standard Decision Tree methods. In this work, we show that looking for the most accurate Decision Tree with the lowest depth is equivalent to solving an RL problem, then we implement Deep RL algorithms DQN, Double DQN and Advantage Actor-Critic to seek the optimal Decision Tree, this choice being motivated by the scalability of these methods to problems with large state spaces unlike Q-Learning. We compare these methods with Hoeffding Trees on real-world data sets and show that DQN and Double DQN perform best in general.