Relationship Spatialization for Depth Estimation

Xiaoyu Xu, Jiayan Qiu, Xinchao Wang, Zhou Wang

Abstract: Considering the role played by the inter-object relationships in monocular depth estimation (MDE), it is easy to tell that relationships, such as in front of and behind, provide explicit spatial priors. However, it is hard to answer the questions that which relationships contain useful spatial priors for depth estimation, and how much do their spatial priors contribute to the depth estimation? In this paper, we term the task of answering these two questions as ‘Relationship Spatialization’ for Depth Estimation. To this end, we strive to spatialize the relationships by devising a novel learning-based framework. Specifically, given a scene image, its image representations and relationship representations are first extracted. Then, the relationship representations are modified by spatially aligned into the visual space and redundancy elimination. Finally, the modified relationship representations are adaptively weighted to concatenate with the image ones for depth estimation, thus accomplishing the relationship spatialization. Experiments on KITTI, NYU v2, and ICL-NUIM datasets show the effectiveness of the relationship spatialization on MDE. Moreover, adopting our relationship spatialization framework to the current state-of-the-art MDE models leads to marginal improvement on most evaluation metrics.