StereoPairFree: Self-Constructed Stereo Correspondence Network From Natural Images

Vinh Quang Dinh, Tae Jong Choi

Abstract: This article proposes a variant of a convolutional neural network-based stereo matching method, StereoPairFree, that requires only normal and not left or right stereo images. The corresponding image patches for training stereo matching cost networks look similar, apart from some properties, such as different illumination, occlusion, distortion, and foreshortening. We propose a method for generating synthesized pairs of corresponding image patches for a given image patch. We also propose a multipatch matching cost network that exploits various input patch sizes. The proposed matching cost network is optimized by cross-based cost aggregation and semiglobal matching, followed by consistency checks and bilateral filtering. Hence, StereoPairFree does not require a single stereo pair but standard images to build a deep learning stereo matching method. It is the first stereo matching method constructed from normal images and significantly outperforms monocular stereo matching approaches (Hur and Roth, 2020), (Hur and Roth, 2021), (Schuster et al., 2020). We evaluated StereoPairFree with the KITTI-2012, KITTI-2015, and Middlebury datasets. It significantly outperformed the baseline and several deep learning-based methods using these datasets.