State Space Emulation and Annealed Sequential Monte Carlo for High Dimensional Optimization

Chencheng Cai, Rong Chen

Abstract:

Many high dimensional optimization problems can be reformulated into the problems of finding the optimal path under an equivalent state space model setting. In this article, we present a general emulation
strategy for developing a state space model whose likelihood function (or posterior distribution) shares
the same general landscape as the original objective function. Then the solution of the optimization
problem is the same as the optimal state path that maximizes the likelihood function. To find such
an optimal path, we adapt a simulated annealing approach by inserting a temperature control into the
emulated dynamic system and propose a novel annealed Sequential Monte Carlo (SMC) method that
effectively generates Monte Carlo sample paths utilizing the samples obtained previously on a higher
temperature scale. Compared to the vanilla simulated annealing implementation, annealed SMC is
an iterative algorithm for state space model optimization that directly generates state paths from the
equilibrium distributions with a decreasing sequence of temperatures through sequential importance
sampling which does not require burn-in or mixing iterations to ensure quasi-equilibrium condition.
Emulation examples and the corresponding simulation results are demonstrated.