A CNN-Based Depth Estimation Approach with Multi-scale Sub-pixel Convolutions and a Smoothness Constraint

Shiyu Zhao, Lin Zhang, Ying Shen, Yongning Zhu

Abstract: Depth estimation from a single image is of paramount importance in various vision tasks, such as obstacle detection, robot navigation, 3D reconstruction, etc. However, how to get an accurate depth map with clear details and a fine resolution remains an unresolved issue. As an attempt to solve this problem, we propose a novel CNN-based approach, namely $$MSCN_{NS}$$ , which involves multi-scale sub-pixel convolutions and a neighborhood smoothness constraint. Specifically, $$MSCN_{NS}$$ makes use of sub-pixel convolutions which fuse multi-scale features from different branches of the network to retrieve a high resolution depth map with fine details of the scene. Furthermore, $$MSCN_{NS}$$ incorporates a neighborhood smoothness regularization term to make sure that spatially closer pixels with similar features would have close depth values. The effectiveness and efficiency of $$MSCN_{NS}$$ have been corroborated through extensive experiments conducted on benchmark datasets.