We Have It Covered: A Resampling-based Method for Uplift Model Comparison

Yang Liu, Chaoyu Yuan

Keywords: Uplift modeling, Nested Bootstrap, Model comparison

Abstract: Uplift models play a critical role in modern marketing applications to help understand the incremental benefits of interventions and identify optimal targeting strategies. A variety of techniques exist for building uplift models, and it is essential to understand the model differences in the context of intended applications. The uplift curve is a widely adopted tool for assessing uplift model performance on the selection universe when observations are available for the entire population. However, when it is uneconomical or infeasible to select the entire population or a large simple random sample, it becomes difficult or even impossible to estimate the uplift curve without appropriate sampling design. To the best of our knowledge, no prior work has addressed how to construct uplift curve estimates on the entire population along with uncertainty quantifications, using only a population subsample. We propose a two-step sampling procedure and a resampling-based approach to compare uplift models with uncertainty quantification, examine the proposed method empirically through simulations and real data applications, and conclude with a discussion.

Primary Area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)

Submission Number: 5319