Mumford-Shah on the Move: Region-Based Segmentation on Deforming Manifolds with Application to 3-D Reconstruction of Shape and Appearance from Multi-View Images

Hailin Jin, Anthony J. Yezzi, Stefano Soatto

Abstract: We address the problem of estimating the shape and appearance of a scene made of smooth Lambertian surfaces with piecewise smooth albedo. We allow the scene to have self-occlusions and multiple connected components. This class of surfaces is often used as an approximation of scenes populated by man-made objects. We assume we are given a number of images taken from different vantage points. Mathematically this problem can be posed as an extension of Mumford and Shah’s approach to static image segmentation to the segmentation of a function defined on a deforming surface. We propose an iterative procedure to minimize a global cost functional that combines geometric priors on both the shape of the scene and the boundary between smooth albedo regions. We carry out the numerical implementation in the level set framework.