Bayesian Rashomon Sets for Model Uncertainty: A critical comparison

Aparajithan Venkateswaran, Anirudh Sankar, Arun Chandrasekhar, Tyler McCormick

Keywords: Bayesian Model Averaging, Rashomon, Model uncertainty, Heterogeneous treatment effects

TL;DR: A comparison of a Rashomon-based approach to Bayesian model uncertainty with several existing alternatives.

Abstract: In statistical analyses, both observational and experimental, understanding how outcomes vary with covariates is crucial. Traditional methods like Bayesian and frequentist regression, regression trees, and model averaging partition data into homogeneous pools to summarize outcomes. However, these methods either focus on a single optimal partition or sample from all possible partitions, often missing high-quality ones or including low-support partitions. A recently developed Bayesian approach, Rashomon Partition Sets (RPSs), enumerates partitions with posterior densities close to the maximum a posteriori (MAP) partition, capturing uncertainty among high-evidence partitions. RPSs adhere to two principles: scientific coherence and simplicity, using a minimax optimal $\ell_0$ prior without additional dependence assumptions. In this paper, we critically compare the RPS approach with three commonly used alternatives: Bayesian Model Averaging, Bayesian/frequentist regularization, and Causal Random Forests.

Submission Number: 27