LILO: Bayesian Optimization with Interactive Natural Language Feedback

Kasia Kobalczyk, Zhiyuan Jerry Lin, Benjamin Letham, Zhuokai Zhao, Maximilian Balandat, Eytan Bakshy

Keywords: bayesian optimization, large language models, in-context learning, experimental design

TL;DR: This work introduces a novel language-in-the-loop framework that leverages large language models to transform free-form textual feedback into scalar utilities, enabling principled bayesian optimization with Gaussian Processes.

Abstract:

For many real-world applications, feedback is essential in translating complex, nuanced, or subjective goals into quantifiable optimization objectives. We propose a language-in-the-loop framework that uses a large language model (LLM) to convert unstructured feedback in the form of natural language into scalar utilities to conduct BO over a numeric search space.
Unlike preferential BO that accepts only restricted comparison formats and requires carefully engineered customized models to handle problem-specific domain knowledge, our approach leverages LLMs to translate heterogeneous textual critiques into consistent utility signals and incorporate flexible user priors without manual kernel crafting, while still being able to retain the sample efficiency and principled uncertainty quantification of BO. We show that this hybrid method not only provides a more natural interface to the decision maker but also outperforms conventional BO baselines and LLM-only optimizers, particularly in feedback-limited regimes.

Primary Area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)

Submission Number: 14265