Fast, Precise Thompson Sampling for Bayesian Optimization

David Sweet

Keywords: Bayesian optimization, Thompson sampling

TL;DR: High-performance Thompson sampling via novel MCMC

Abstract:

Thompson sampling (TS) has optimal regret and excellent empirical performance
in multi-armed bandit problems. Yet, in Bayesian optimization, TS underperforms
popular acquisition functions (e.g., EI, UCB). TS samples arms according to the
probability that they are optimal. A recent algorithm, P-Star Sampler (PSS), per-
forms such a sampling via Hit-and-Run. We present an improved version, Stagger
Thompson Sampler (STS). STS more precisely locates the maximizer than does
TS using less computation time. We demonstrate that STS outperforms TS, PSS,
and other acquisition methods in numerical experiments of optimizations of sev-
eral test functions across a broad range of dimension. Additionally, since PSS
was originally presented not as a standalone acquisition method but as an input to
a batching algorithm called Minimal Terminal Variance (MTV), we also demon-
strate that STS matches PSS performance when used as the input to MTV.

Submission Number: 6