DUO: Diffusion Models for Universal Offline Black-Box Optimization

Yihao Zheng, Ke Xue, Rong-Xi Tan, Chao Qian

Keywords: Universal optimization; Diffusion models; Offline optimization

Abstract:

Offline black-box optimization aims to find high-performing designs from a fixed dataset without online evaluations, with applications spanning protein design, materials discovery, and robot learning.
Most existing methods are typically designed for a single task with fixed dimensionality, leaving universal offline optimization---learning one shared model across heterogeneous search spaces, mixed variable types, and scarce-data transfer settings---largely unresolved.
In this paper, we study universal offline optimization from a generative perspective for the first time and propose DUO (**D**iffusion model for **U**niversal offline black-box **O**ptimization), which bridges universal representation learning with trajectory-level diffusion modeling.
DUO uses a Transformer-based variational autoencoder to embed both continuous and discrete designs into a shared latent space, avoiding task-specific architectures for incompatible native domains.
Within this unified space, we synthesize optimization-oriented trajectories and train a conditional diffusion model, with task-level semantics injected through frozen text-metadata embeddings and classifier-free guidance. A cross-entropy consistency term further aligns continuous training with discrete evaluation objectives.
Evaluated on the Design-Bench and SOO-Bench benchmarks, DUO demonstrates strong performance across diverse continuous and discrete tasks under multitask joint training. Our experiments highlight its robust zero-shot and few-shot transfer capabilities, suggesting that metadata-aware latent trajectory diffusion provides a highly effective framework for universal offline black-box optimization.

Submission Number: 45