Physics-Regularized Stereo Matching for Depth Estimation

Jun Cheng, Weide Liu, ZAIWANG GU, Zhengguo Li, Jiayuan Fan, Chuan-Sheng Foo

Keywords: stereo matching

Abstract: Depth estimation from stereo or multi-view images is an essential technology for a wide range of vision and robotics applications. In recent years, many deep learning based methods have been proposed for this purpose. However, training the stereo matching network is challenging and requires a large amount of data, especially for the 3D convolution networks. Existing stereo matching approaches are mostly data-driven, which often converge to a local minimum biased to the training data. In this paper, we propose a novel self-supervised physics regularization framework to improve the training of the networks using physical knowledge or constraints. More specifically, we explore the use of low-level structures as physical constraints for the regularization of the stereo-matching network via multi-task learning. Moreover, a disparity aggregation module is proposed to aggregate the disparity output with image features to consider the association in between. We also find that the canny edge can be used as a pseudo ground truth to train the network with performance comparable to the ideal ground truth edge in the Scene Flow dataset. We combine the proposed physics regularization with four existing stereo matching algorithms. The experimental results in three public datasets, including Scene Flow, KITTI 2012, and KITTI 2015, show the effectiveness and generality of the proposed framework.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Submission Guidelines: Yes

Please Choose The Closest Area That Your Submission Falls Into: Applications (eg, speech processing, computer vision, NLP)

TL;DR: This paper presents a physics regularization framework to improve the stereo matching.