Variational State-Space Models for Localisation and Dense 3D Mapping in 6 DoFDownload PDF

28 Sept 2020, 15:48 (edited 10 Feb 2022)ICLR 2021 PosterReaders: Everyone
  • Keywords: Generative models, Bayesian inference, Variational inference, SLAM, Deep learning
  • Abstract: We solve the problem of 6-DoF localisation and 3D dense reconstruction in spatial environments as approximate Bayesian inference in a deep state-space model. Our approach leverages both learning and domain knowledge from multiple-view geometry and rigid-body dynamics. This results in an expressive predictive model of the world, often missing in current state-of-the-art visual SLAM solutions. The combination of variational inference, neural networks and a differentiable raycaster ensures that our model is amenable to end-to-end gradient-based optimisation. We evaluate our approach on realistic unmanned aerial vehicle flight data, nearing the performance of state-of-the-art visual-inertial odometry systems. We demonstrate the applicability of the model to generative prediction and planning.
  • One-sentence Summary: We propose a variational state-space model with a latent map for 6-DoF localisation, 3D dense mapping and generative modelling for planning.
  • Supplementary Material: zip
  • Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics
  • Data: [Blackbird](
10 Replies