Bayesian Additive Regression Trees for Exponential Family Distributions: A Theoretical Perspective

ICLR 2026 Conference Submission13558 Authors

Keywords: Bayesian additive regression trees, BART, Posterior concentration, Minimax rate, Exponential family, Generalized linear models

TL;DR: We derive sufficient conditions under which Bayesian Additive Regression Trees achieve minimax convergence rates for general response variables in the exponential family.

Abstract: Bayesian Additive Regression Trees (BART) are a powerful ensemble learning technique for modeling nonlinear regression functions, which along with predictions also provide posterior uncertainty estimates unlike frequentist methods designed for similar tasks, such as random forests. Although initially BART was proposed for predicting only continuous and binary response variables, over the years multiple extensions have emerged that are suitable for estimating a wider class of response variables (e.g. categorical and count data) in a multitude of application areas. In this paper we describe a Generalized framework for Bayesian trees and their additive ensembles where the response variable comes from an exponential family distribution and hence encompasses a majority of these variants of BART. We derive sufficient conditions on the response distribution, under which the posterior concentrates at a near minimax rate. Our results provide theoretical justification for the empirical success of BART and its variants. In addition, the sufficient conditions provide important insights into practical model specifications such as the choice of link functions.

Primary Area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)

Submission Number: 13558