Adversarial Query Synthesis via Bayesian Optimization

Yimeng Zeng, Jeffrey Tao, Haydn Thomas Jones, Natalie Maus, Osbert Bastani, Jacob R. Gardner, Ryan Marcus

Keywords: Bayesian optimization, database benchmarking, query optimization, SQL, adversarial workloads

TL;DR: Automatically finding and generate hard SQL queries (with witness plans) to build better DB benchmarks that show big performance gaps.

Abstract: Benchmark workloads are extremely important to the database management research community, especially as more machine learning components are integrated into database systems. Here, we propose a Bayesian optimization technique to automatically search for difficult benchmark queries, significantly reducing the amount of manual effort usually required. In preliminary experiments, we show that our approach can generate queries with more than double the optimization headroom compared to existing benchmarks.

Submission Number: 61