Variational autoencoder with weighted samples for high-dimensional non-parametric adaptive importance sampling

Julien Demange-Chryst, Francois Bachoc, Jérôme Morio, Timothé Krauth

Abstract: Adaptive importance sampling is a well-known family of algorithms for density approximation, generation and Monte Carlo integration including rare event estimation. The main common denominator of this family of algorithms is to perform density estimation with weighted samples at each iteration. However, the classical existing methods to do so, such as kernel smoothing or approximation by a Gaussian distribution, suffer from the curse of dimensionality and/or a lack of flexibility. Both are limitations in high dimension and when we do not have any prior knowledge on the form of the target distribution, such as its number of modes. Variational autoencoders are probabilistic tools able to represent with fidelity high-dimensional data in a lower dimensional space. They constitute a parametric family of distributions robust faced to the dimension and since they are based on deep neural networks, they are flexible enough to be considered as non-parametric models. In this paper, we propose to use a variational autoencoder as the auxiliary importance sampling distribution by extending the existing framework to weighted samples. We integrate the proposed procedure in existing adaptive importance sampling algorithms and we illustrate its practical interest on diverse examples.

Submission Length: Long submission (more than 12 pages of main content)

Changes Since Last Submission: Since the last TMLR submission, we add the numerical experiments required by the Reviewers and the Action Editor in Section 6.

Code: https://github.com/Julien6431/Importance-Sampling-VAE.git

Assigned Action Editor: ~Francisco_J._R._Ruiz1

License: Creative Commons Attribution 4.0 International (CC BY 4.0)

Submission Number: 1681