On Equivalences between Weight and Function-Space Langevin Dynamics
Abstract: Approximate inference for overparameterized Bayesian models appears challenging, due to the complex structure of the posterior. To address this issue, a recent line of work has investigated the possibility of directly conducting approximate inference in the ``function space'', the space of prediction functions. This paper provides an alternative perspective to this problem, by showing that for many models – including a simplified neural network model – Langevin dynamics in the overparameterized ``weight space'' induces equivalent function-space trajectories to certain Langevin dynamics procedures in function space. Thus, the former can already be viewed as a function-space inference algorithm, with its convergence unaffected by overparameterization. We provide simulations on Bayesian neural network models and discuss the implication of the results.
Submission Length: Regular submission (no more than 12 pages of main content)
Assigned Action Editor: ~Michael_U._Gutmann1
Submission Number: 1132
Loading