Evidence Estimation by Kullback-Leibler Integration for Flow-Based Methods

Nikolai Zaki, Théo Galy-Fajou, Manfred Opper

Keywords: evidence, integration, gradient flow

TL;DR: We show preliminary results on how for flow-based variational inference methods, it's possible to train and compute the evidence simultaneously.

Abstract:

Flow-based methods such as Stein Variational Gradient Descent caught a lot of interest due to their flexibility and the strong theory going with them.
An usual issue however is to be able to compute the KL divergence or even possibly the log evidence.
In this preliminary work we show that it's possible to obtain an approximation during inference by reusing the training computations.
We show preliminary results on Gaussian targets but face issues with more complex problems.