FlowGen: Bayesian Search for Pareto-Optimal Generative AI

Alexander Conway, Debadeepta Dey, Stefan Hackmann, Matthew Hausknecht, Michael Douglas Schmidt, Mark Lewis Steadman, Nick Volynets

Confirmation: our paper adheres to reproducibility best practices. In particular, we confirm that all important details required to reproduce results are described in the paper,, the authors agree to the paper being made available online through OpenReview under a CC-BY 4.0 license (https://creativecommons.org/licenses/by/4.0/), and, the authors have read and commit to adhering to the AutoML 2025 Code of Conduct (https://2025.automl.cc/code-of-conduct/).

TL;DR: FlowGen efficiently optimizes RAG pipelines for accuracy, latency, and cost using Bayesian Optimization.

Abstract:

Retrieval-Augmented Generation (RAG) pipelines are central to applying large language models (LLMs) to proprietary or dynamic data. However, building effective RAG flows is complex, requiring careful selection among vector databases, embedding models, text splitters, retrievers, and synthesizing LLMs. The challenge deepens with the rise of agentic paradigms. Modules like verifiers, rewriters, and rerankers—each with intricate hyperparameter dependencies have to be carefully tuned. Balancing tradeoffs between latency, accuracy, and cost becomes increasingly difficult in performance-sensitive applications.

We introduce FlowGen, a framework that performs efficient multi-objective search over a broad space of agentic and non-agentic RAG configurations. Using Bayesian Optimization, FlowGen discovers Pareto-optimal flows that jointly optimize task accuracy and cost. A novel early-stopping mechanism further improves efficiency by pruning clearly suboptimal candidates. Across multiple RAG benchmarks, FlowGen improves accuracy at baseline cost by 6\% or reduces cost at baseline accuracy by 37\% on average across $6$ RAG benchmarks. Without considering cost, FlowGen improves accuracy by 25\% on average. Furthermore, FlowGen's ability to design and optimize also allows integrating new modules, making it even easier and faster to realize generative AI pipelines that are high-performing and drive value.

Submission Number: 39