Continuous 3D Label Stereo Matching Using Local Expansion Moves

Tatsunori Taniai, Yasuyuki Matsushita, Yoichi Sato, Takeshi Naemura

Abstract: We present an accurate stereo matching method using <i>local expansion moves</i> based on graph cuts. This new move-making scheme is used to efficiently infer per-pixel 3D plane labels on a pairwise Markov random field (MRF) that effectively combines recently proposed slanted patch matching and curvature regularization terms. The local expansion moves are presented as many <inline-formula><tex-math notation="LaTeX">$\alpha$</tex-math></inline-formula> -expansions defined for small grid regions. The local expansion moves extend traditional expansion moves by two ways: localization and spatial propagation. By localization, we use different candidate <inline-formula><tex-math notation="LaTeX">$\alpha$</tex-math> </inline-formula> -labels according to the locations of local <inline-formula><tex-math notation="LaTeX">$\alpha$</tex-math></inline-formula> -expansions. By spatial propagation, we design our local <inline-formula><tex-math notation="LaTeX">$\alpha$</tex-math></inline-formula> -expansions to propagate currently assigned labels for nearby regions. With this localization and spatial propagation, our method can efficiently infer MRF models with a continuous label space using randomized search. Our method has several advantages over previous approaches that are based on fusion moves or belief propagation; it produces <i>submodular moves </i> deriving a <i>subproblem optimality</i> ; it helps find good, smooth, piecewise linear disparity maps; it is suitable for parallelization; it can use cost-volume filtering techniques for accelerating the matching cost computations. Even using a simple pairwise MRF, our method is shown to have best performance in the Middlebury stereo benchmark V2 and V3.