On the Bias of Variational Resampling

Axel Finke, Oskar Kviman, Nicola Branchini, Víctor Elvira

TL;DR: We characterise and alleviate the bias of the variational resampling scheme which was proposed for sequential Monte Carlo algorithms.

Abstract: Variational resampling (VR) is a method for deterministically resampling the $N$ particles in sequential Monte Carlo (SMC) algorithms (also known as particle filters), by minimising the Kullback--Leibler divergence from the empirical measure of the $N$ weighted original particles to the empirical measure of $M$ unweighted resampled particles. The combination of VR with a weight transformation (called smoothing weights) has shown to often yield a smaller mean-square error (MSE) than standard resampling schemes in the literature. However, its bias has never been investigated. In this paper, we first show that VR incurs a weighting bias and a truncation bias. We then propose a mechanism to alleviate the weighting bias through an uneven weighting of the resampled particles. We also show that the truncation bias implies that the particle approximation of the target distribution is restricted to a region in which the unnormalised weights are larger than some threshold with high probability. We prove that this probability approaches $1$ if $M = \mathrm{O}(N)$ as $N \to \infty$. Finally, we empirically illustrate that the smaller MSE of VR observed in the literature may be attributable to an underestimation of uncertainty caused by the use of the smoothing weights.

Code Dataset Promise: Yes

Code Dataset Url: https://github.com/AxelFinke/bias-of-variational_resampling-aistats

Format Confirmation: I agree that I have read and followed the formatting instructions for the camera ready version.

Submission Number: 2357