Learning Confidence Measures by Multi-modal Convolutional Neural Networks

Zehua Fu, Mohsen Ardabilian Fard

Abstract: In stereo matching, the correctness of stereo pairs matches, also called confidence, is used to improve the dense disparity estimation result. In this paper, we propose a multi-modal deep learning approach for stereo matching confidence estimation. The input of our method is composed of two modalities, the initial disparity maps, and its reference color image. To effectively combine these two modalities, we explore and study multiple convolutional neural network (CNN) structures for our specific confidence prediction tasks. To the best of our knowledge, this is the first approach reported in the literature combining multiple modalities and patch based deep learning to predict the confidence. The experiments on KITTI datasets demonstrate that our multi-modal confidence network can significantly outperform the state-of-the-art methods.

External IDs:dblp:conf/wacv/FuF18