360DIV: 360° Video Plus Depth for Fully Immersive VR Experiences

Julie Artois, Glenn Van Wallendael, Peter Lambert

Abstract: 360° cameras can easily capture events and let viewers across the world re-live them in Virtual Reality (VR). However, the viewer can only rotate their head and there is no correct sense of depth, breaking the immersion. This work describes a framework that reconstructs the captured scene in 3D from a 360° image/video. To do this, it requires depth information, either from depth estimation or depth sensors. The reconstruction is rendered in VR in real-time. Areas of the scene that were not captured, are inpainted in a fast and non-intrusive way. A demo is made available online and will be used for user experiments to assess the perceived quality of experience.