Open Source Vizier: Distributed Infrastructure and API for Reliable and Flexible Blackbox Optimization
Abstract: Vizier is the de-facto blackbox optimization service across Google, having optimized some of Google's largest products and research efforts. To operate at the scale of tuning thousands of users' critical systems, Vizier solved key design challenges in providing multiple different features, while remaining fully fault-tolerant. In this paper, we introduce Open Source (OSS) Vizier, a Python-based interface for blackbox optimization and research, based on the Google-internal Vizier infrastructure and framework. OSS Vizier provides an API capable of defining and solving a wide variety of optimization problems, including multi-metric, early stopping, transfer learning, and conditional search. Furthermore, it is designed to be a distributed system that assures reliability, and allows multiple parallel evaluations of the user's objective function. The flexible RPC-based infrastructure allows users to access OSS Vizier from binaries written in any language. OSS Vizier also provides a back-end ("Pythia") API that gives algorithm authors a way to interface new algorithms with the core Vizier system. OSS Vizier is available at https://github.com/google/vizier.
Keywords: vizier, open source, blackbox, optimization, system, hyperparameter, tuning, rpc, protobuf, API, Google, Python
One-sentence Summary: We introduce Open Source (OSS) Vizier, a Python-based interface for blackbox optimization and research, based on the original Google Vizier infrastructure and framework.
Track: Special track for systems, benchmarks and challenges
Reproducibility Checklist: Yes
Broader Impact Statement: Yes
Paper Availability And License: Yes
Code Of Conduct: Yes
Reviewers: Xingyou Song, email@example.com
Main Paper And Supplementary Material: pdf
Steps For Environmental Footprint Reduction During Development: Not applicable. This is a systems paper and does not run expensive benchmark experiments.
CPU Hours: 0
GPU Hours: 0
TPU Hours: 0
Evaluation Metrics: No
Estimated CO2e Footprint: 0
Class Of Approaches: blackbox optimization, hyperparameter tuning, fault-tolerance, distributed systems