VaRT: Variational Regression Trees

Sebastian Salazar

Keywords: Probabilistic Machine Learning, Variational Inference, Bayesian Inference, Bayesian Nonparametrics

TL;DR: We introduce a novel Non-parametric Bayesian model over the space of decision trees and derive a variational approximation to the posterior distribution.

Abstract: Decision trees are a well-established tool in machine learning for classification and regression tasks. In this paper, we introduce a novel non-parametric Bayesian model that uses variational inference to approximate a posterior distribution over the space of stochastic decision trees. We evaluate the model's performance on 18 datasets and demonstrate its competitiveness with other state-of-the-art methods in regression tasks. We also explore its application to causal inference problems. We provide a fully vectorized implementation of our algorithm in PyTorch.

Submission Number: 3735