BAOSL: Benchmarking Active Optimization for Self-driving Laboratories

TMLR Paper5566 Authors

Abstract: Discovery of novel materials and antibiotics can be posed as an optimization problem, namely, identifying candidate formulations that maximize one or more desired properties. In practice, however, the enormous dimensionality of the design space and the high cost of each experimental evaluation make exhaustive search strategies infeasible. Active learning methods, which iteratively identify informative data points, offer a promising solution to tackle these challenges by significantly reducing the data-labeling effort and resource requirements. Integrating active learning into optimization workflows, hereafter termed active optimization, accelerates the discovery of optimal candidates while substantially cutting the number of required evaluations. Despite these advances, the absence of standardized benchmarks impedes objective comparison of methodologies, slowing progress in self-driving scientific discovery. To address this, we introduce BAOSA, a comprehensive benchmark designed to systematically evaluate active optimization in self-driving laboratories. BAOSA provides a standardized evaluation protocol and reference implementations to facilitate efficient and reproducible benchmarking. BAOSA includes both synthetic benchmarks and real-world tasks in various fields, designed to address unique challenges, particularly limited data availability, in self-driving laboratories.

Submission Length: Regular submission (no more than 12 pages of main content)

Previous TMLR Submission Url: https://openreview.net/forum?id=T1siqFh1lE&referrer=%5BAuthor%20Console%5D(%2Fgroup%3Fid%3DTMLR%2FAuthors%23your-submissions)

Changes Since Last Submission:

The major changes are listed below:
- Expanded benchmark suite:
- Increased the number of synthetic benchmark functions from 6 to 8 (added Levy and Sphere).
- Expanded baseline methods evaluated from 11 to 14, including additional advanced methods such as IPOP-CMA-ES, BIPOP-CMA-ES, and SAASBO.
- Introduced active optimization (AO) concept:
- Clearly differentiated the broader concept of Active Optimization (AO), generalizing Bayesian Optimization (BO) and Active Learning (AL), focusing on locating optimal solutions rather than just improving predictive accuracy.
- Enhanced surrogate model evaluation:
- Compared additional surrogate models (Random Forest, XGBoost, CNN) across different dimensionalities (10D–200D).
- Confirmed that CNN-based surrogate models maintained better generalization performance at higher dimensionalities.

Assigned Action Editor: ~Roman_Garnett1

Submission Number: 5566