Feature Engineering and Deep Learning for Stereo Matching Under Adverse Driving Conditions

Vinh Quang Dinh, Phuc Hong Nguyen, Vinh Dinh Nguyen

Abstract: Stereo matching is a challenging research topic in driving assistance systems. Existing stereo matching methods work well under normal day-light conditions. However, they fail to operate under adverse driving conditions, such as at night and during snowfall. This paper proposes a robust stereo matching framework using both deep-learning-based features and feature engineering. The proposed method investigates the benefits of features based on feature engineering and deep learning for solving stereo matching problems. Robust feature engineering is proposed for handling specific driving under adverse weather conditions, and a robust feature based on deep learning is considered for handling unspecific driving under extreme weather conditions. The proposed study has shown significantly improved accuracy by 8.31% for the state-of-the-art census based on semi-global matching under the reflection regions using the KITTI Stereo 2012 benchmark. Moreover, the experimental results demonstrate that the proposed system obtains more stable results than existing stereo methods based on deep learning on various stereo datasets, such as the Middlebury, EISAT, HCI, and CCD datasets.