Decision Trees with Short Explainable RulesDownload PDF

Published: 31 Oct 2022, 18:00, Last Modified: 11 Oct 2022, 20:09NeurIPS 2022 AcceptReaders: Everyone
Keywords: decision trees, explainable models, classification, approximation algorithms
TL;DR: New algorithms with provable guarantees and good experimental performance for building more interpretable decision trees
Abstract: Decision trees are widely used in many settings where interpretable models are preferred or required. As confirmed by recent empirical studies, the interpretability/explanability of a decision tree critically depends on some of its structural parameters, like size and the average/maximum depth of its leaves. There is indeed a vast literature on the design and analysis of decision tree algorithms that aim at optimizing these parameters. This paper contributes to this important line of research: we propose as a novel criterion of measuring the interpretability of a decision tree, the sparsity of the set of attributes that are (on average) required to explain the classification of the examples. We give a tight characterization of the best possible guarantees achievable by a decision tree built to optimize both our new measure (which we call the {\em explanation size}) and the more classical measures of worst-case and average depth. In particular, we give an algorithm that guarantees $O(\ln n )$-approximation (hence optimal if $P \neq NP$) for the minimization of both the average/worst-case explanation size and the average/worst-case depth. In addition to our theoretical contributions, experiments with 20 real datasets show that our algorithm has accuracy competitive with {\tt CART} while producing trees that allow for much simpler explanations.
11 Replies