Genie: An Open Box Counterfactual Policy Estimator for Optimizing Sponsored Search Marketplace

Murat Bayir

Abstract:

In this paper, we propose an offline counterfactual policy estimation
framework called Genie to optimize Sponsored Search Marketplace.
Genie employs an open box simulation engine with click calibration model to compute the KPI impact of any modification to the
system. From the experimental results on Bing traffic, we showed
that Genie performs better than existing observational approaches
that employs randomized experiments for traffic slices that have
frequent policy updates. We also show that Genie can be used to
tune completely new policies efficiently without creating risky randomized experiments due to cold start problem. As time of today,
Genie hosts more than 10000 optimization jobs yearly which runs
more than 30 Million processing node hours of big data jobs for
Bing Ads. For the last 3 years, Genie has been proven to be the one
of the major platforms to optimize Bing Ads Marketplace due to
its reliability under frequent policy changes and its efficiency to
minimize risks in real experiments.