Optimizing Mixture Importance Sampling Via Online Learning: Algorithms and Applications

Tingwei Liu, Hong Xie, John C. S. Lui

Abstract: Importance sampling (IS) is widely used in rare event simulation, but it is costly to deal with many rare events simultaneously. For example, a rare event can be the failure to provide the quality-of-service guarantee for a critical network flow. Since network providers often need to deal with many critical flows (i.e., rare events) simultaneously, if using IS, providers have to simulate each rare event with its customized importance distribution individually. To reduce such cost, we propose an efficient mixture importance distribution for multiple rare events, and formulate the mixture importance sampling optimization problem (MISOP) to select the optimal mixture. We first show that the "search direction" of mixture is computationally expensive to evaluate, making it challenging to locate the optimal mixture. We then formulate a " zero learning cost " online learning framework to estimate the "search direction", and learn the optimal mixture from simulation samples of events. We develop two multi-armed bandit online learning algorithms to: (1) Minimize the sum of estimation variances with a regret of (ln T) <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> /T; (2) Minimize the simulation cost with a regret of √ln T/T , where T denotes the number of simulation samples. We demonstrate our method on a realistic network and show that it can reduce the cost measures (i.e., sum of estimation variances and simulation cost) by as high as 61.6% compared with the uniform mixture IS.