Depth estimation and camera calibration of a focused plenoptic camera for visual odometryDownload PDF

17 Feb 2020OpenReview Archive Direct UploadReaders: Everyone
Abstract: This paper presents new and improved methods of depth estimation and camera calibration for visualodometry with a focused plenoptic camera.For depth estimation we adapt an algorithm previously used in structure-from-motion approaches towork with images of a focused plenoptic camera. In the raw image of a plenoptic camera, scene patchesare recorded in several micro-images under slightly different angles. This leads to a multi-view stereo-problem. To reduce the complexity, we divide this into multiple binocular stereo problems. For each pixelwith sufficient gradient we estimate a virtual (uncalibrated) depth based on local intensity error mini-mization. The estimated depth is characterized by the variance of the estimate and is subsequentlyupdated with the estimates from other micro-images. Updating is performed in a Kalman-like fashion.The result of depth estimation in a single image of the plenoptic camera is a probabilistic depth map,where each depth pixel consists of an estimated virtual depth and a corresponding variance.Since the resulting image of the plenoptic camera contains two plains: the optical image and the depthmap, camera calibration is divided into two separate sub-problems. The optical path is calibrated basedon a traditional calibration method. For calibrating the depth map we introduce two novel model basedmethods, which define the relation of the virtual depth, which has been estimated based on the light-fieldimage, and the metric object distance. These two methods are compared to a well known curve fittingapproach. Both model based methods show significant advantages compared to the curve fitting method.For visual odometry we fuse the probabilistic depth map gained from one shot of the plenoptic camerawith the depth data gained by finding stereo correspondences between subsequent synthesized intensityimages of the plenoptic camera. These images can be synthesized totally focused and thus finding stereocorrespondences is enhanced. In contrast to monocular visual odometry approaches, due to the calibra-tion of the individual depth maps, the scale of the scene can be observed. Furthermore, due to the light-field information better tracking capabilities compared to the monocular case can be expected.As result, the depth information gained by the plenoptic camera based visual odometry algorithm pro-posed in this paper has superior accuracy and reliability compared to the depth estimated from a singlelight-field image.
0 Replies

Loading