Local Thompson Sampling via Prompting for Bayesian Optimization with LLM Generators

Shorya Sharma, Raul Astudillo

Abstract: Steering generative models to propose candidates that optimize expensive black-box functions is a central problem with applications in scientific discovery. Recent approaches integrate Bayesian optimization (BO) with large language models (LLMs) to guide candidate generation, but often require complex architectures or tight coupling between surrogate models and generators. We propose $\textit{Local Thompson Prompting}$ (LTP), a simple and modular alternative that leverages a Gaussian process (GP) surrogate to perform Thompson sampling over previously evaluated candidates, and uses the resulting samples to directly condition LLM prompts. This yields a lightweight mechanism for uncertainty-aware exploration without modifying the LLM or training additional components. We evaluate LTP on two domains—Semantle (word-guessing) and molecular optimization—and show that it consistently outperforms existing BO-LLM baselines while remaining architecturally minimal. These results suggest that effective BO-style exploration can be achieved through prompt-level interventions alone.

Keywords: Bayesian Optimisation, Large language models, Thompson Sampling, Adaptive experimental design

TLDR: How can we effectively steer LLMs via adaptive prompts to generate desirable candidates?

Submission Number: 22