Robust estimation of depth and motion using stochastic approximation

Rama Chellappa, Amit K. Roy-Chowdhury

Abstract: The problem of structure from motion (SfM) is to extract the three-dimensional model of a moving scene from a sequence of images. Though two images are sufficient to produce a 3D reconstruction, they usually perform poorly because of errors in the estimation of the camera motion and image correspondences, thus motivating the need for multiple frame algorithms. One common approach to this problem is to determine the estimate from pairs of images and then fuse them together. Data fusion techniques, like the Kalman filter, require estimates of the error in modeling and observations. The complexity of the SfM problem makes it difficult to reliably estimate these errors. This paper describes a new recursive algorithm to estimate the camera motion and scene structure by fusing the two-frame estimates, using stochastic approximation techniques. The method does not require estimates of the error in the two-frame case and can reconstruct the scene to arbitrary accuracy given a sufficient number of frames. Experimental results are reported to support these claims.