LILO: Bayesian Optimization with Natural Language Feedback

Kasia Kobalczyk, Zhiyuan Jerry Lin, Benjamin Letham, Zhuokai Zhao, Maximilian Balandat, Eytan Bakshy

TL;DR: This work introduces a novel language-in-the-loop framework that leverages large language models to transform free-form textual feedback into scalar utilities, enabling principled bayesian optimization with Gaussian Processes.

Abstract: Many real-world optimization problems are guided by complex, subjective preferences that are difficult to express as explicit closed-form objectives. In response, we introduce Language-in-the-Loop Optimization (LILO), a Bayesian optimization (BO) framework that employs a large language model (LLM) to translate free-form natural language feedback and prior knowledge from a decision maker into structured preference signals, going beyond the restrictive scalar or pairwise feedback formats typically assumed in preferential BO. The LLM-derived preferences are integrated by a Gaussian process proxy model, enabling principled acquisition-driven exploration with calibrated uncertainty. By placing the LLM in a supporting role rather than as the optimizer itself, LILO preserves the sample efficiency and stability of BO while providing a flexible and expressive feedback interface. Across synthetic and real-world benchmarks, LILO consistently outperforms both conventional preference-based BO methods and LLM-only optimizers, with particularly strong gains in feedback-limited regimes. The code for reproducing our experimental results is available at: https://github.com/facebookresearch/lilo.

Lay Summary:

Many real-world optimization problems involve preferences that are difficult to reduce to a single numerical objective. For example, we may care whether a design feels safe and comfortable, whether an LLM-generated response is useful, or whether an outcome matches nuanced personal or business priorities. Such preferences are often easier to describe in words than through a precise mathematical formula, numerical scores, or fixed comparison formats.

This paper introduces Language-in-the-Loop Optimization (LILO), a framework for guiding black-box optimization with free-form natural language feedback. LILO uses a language model to interpret this feedback and convert it into structured preference information. A Bayesian optimization method then uses these preferences to choose promising new candidates to evaluate, while accounting for uncertainty.

Across synthetic and real-world benchmarks, LILO identifies high-quality solutions more efficiently than methods that rely only on conventional numerical or pairwise feedback, as well as methods that use an LLM as the optimizer directly. The results demonstrate that natural language can provide a flexible and information-rich interface for optimization, particularly in settings where evaluations are costly and preferences involve complex tradeoffs.

Link To Code: https://github.com/facebookresearch/lilo

Primary Area: Optimization->Zero-order and Black-box Optimization

Keywords: large language models, preferential Bayesian optimization, natural language feedback

Submission Number: 17763