VaRT: Variational Regression Trees

Published: 21 Sept 2023, Last Modified: 02 Nov 2023NeurIPS 2023 spotlightEveryoneRevisionsBibTeX
Keywords: Probabilistic Machine Learning, Variational Inference, Bayesian Inference, Bayesian Nonparametrics
TL;DR: We introduce a novel Non-parametric Bayesian model over the space of decision trees and derive a variational approximation to the posterior distribution.
Abstract: Decision trees are a well-established tool in machine learning for classification and regression tasks. In this paper, we introduce a novel non-parametric Bayesian model that uses variational inference to approximate a posterior distribution over the space of stochastic decision trees. We evaluate the model's performance on 18 datasets and demonstrate its competitiveness with other state-of-the-art methods in regression tasks. We also explore its application to causal inference problems. We provide a fully vectorized implementation of our algorithm in PyTorch.
Supplementary Material: zip
Submission Number: 3735