On free energy barriers in Gaussian priors and failure of MCMC for high-dimensional unimodal distributions

Sven Wang

Abstract:

We exhibit examples of high-dimensional unimodal posterior distributions arising in non-linear
regression models with Gaussian process priors for which MCMC methods can take an exponential
run-time to enter the regions where the bulk of the posterior measure concentrates. Our results apply
to worst-case initialised (‘cold start’) algorithms that are local in the sense that their step-sizes cannot
be too large on average. The counter-examples hold for general MCMC schemes based on gradient
or random walk steps, and the theory is illustrated for Metropolis-Hastings adjusted methods such as
pCN and MALA.