Convex Polytope Trees

Mohammadreza Armandpour, Ali Sadeghian, Mingyuan Zhou

Keywords: Bayesian Methods, Logical Disjunction, Probabilistic Models, Decision Tree, Oblique tree

TL;DR: We propose a novel generalization to the family of oblique decision trees models.

Abstract: A decision tree is commonly restricted to use a single hyperplane to split the covariate space at each of its internal nodes. It often requires a large number of nodes to achieve high accuracy. In this paper, we propose convex polytope trees (CPT) to expand the family of decision trees by an interpretable generalization of their decision boundary. The splitting function at each node of CPT is based on the logical disjunction of a community of differently weighted probabilistic linear decision-makers, which also geometrically corresponds to a convex polytope in the covariate space. We use a nonparametric Bayesian prior at each node to infer the community's size, encouraging simpler decision boundaries by shrinking the number of polytope facets. We develop a greedy method to efficiently construct CPT and scalable end-to-end training algorithms for the tree parameters when the tree structure is given. We empirically demonstrate the efficiency of CPT over existing state-of-the-art decision trees in several real-world classification and regression tasks from diverse domains.

Code Of Conduct: I certify that all co-authors of this work have read and commit to adhering to the NeurIPS Statement on Ethics, Fairness, Inclusivity, and Code of Conduct.

Code: https://github.com/rezaarmand/Convex_Polytope_Trees

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:2010.11266/code)