SeasonDepth: Cross-Season Monocular Depth Prediction Dataset and Benchmark under Multiple Environments

Hanjiang Hu, Baoquan Yang, Zhijian Qiao, Ding Zhao, Hesheng Wang

Keywords: Monocular Depth Prediction, Long-term Visual Perception and Localization, Dataset and Benchmark

TL;DR: We provide a new multi-environment monocular depth prediction dataset and benchmark SOTA methods from KITTI leaderboard, showing how changing environments affects depth prediction and giving solutions to the robustness under multiple conditions.

Abstract: Different environments pose a great challenge on the outdoor robust visual perception for long-term autonomous driving and the generalization of learning-based algorithms on different environmental effects is still an open problem. Althoughmonocular depth prediction has been well studied recently, there is few work focusing on the robust learning-based depth prediction across different environments,e.g.changing illumination and seasons, owing to the lack of such a multi-environment real-world dataset and benchmark. To this end, the first cross-season monocular depth prediction dataset and benchmarkSeasonDepth (available on https://seasondepth.github.io/) is built based on CMU Visual Localizationdataset. To benchmark the depth estimation performance under different environments, we investigate representative and recent state-of-the-art open-source supervised, self-supervised and domain adaptation depth prediction methods fromKITTIbenchmark using several newly-formulated metrics. Through extensive experimental evaluation on the proposed dataset, the influence of multiple environments on performance and robustness is analyzed both qualitatively and quantitatively, showing that the long-term monocular depth prediction is far from solved even with fine-tuning. We further give promising avenues that self-supervised training and stereo geometry constraint help to enhance the robustness to changing environments.

URL: Training set v1 and the fine-tuned models are released. Updated dataset website: https://seasondepth.github.io/ ; Updated benchmark toolkit repo: https://github.com/SeasonDepth/SeasonDepth ; Updated detailed dataset information README: https://github.com/SeasonDepth/SeasonDepth/tree/master/dataset_info

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:2011.04408/code)