Uncertainty Guided Depth Fusion for Spike Camera

Jianing Li, Jiaming Liu, Qizhe Zhang, Ming Lu, Tiejun Huang, Shanghang Zhang

Keywords: Spike camera, Uncertainty, Depth estimation, Fusion Strategies

TL;DR: We propose a novel uncertainty-guided fusion framework for spike depth estimation.

Abstract: Neuromorphic spike camera captures visual streams with high frame rate in a bio-inspired way, bringing vast potential in various real-world applications such as autonomous driving.Compared with traditional cameras, spike camera data has an inherent advantage to overcome motion blur, leading to more accurate depth estimation in high-velocity circumstances. However, depth estimation with spike camera remains very challenging when using traditional monocular or stereo depth estimation algorithms, which are based on the photometric consistency. In this paper, we propose a novel and effective approach for spike depth estimation, which fuses the monocular and stereo depth estimation for spike camera based on the uncertainty of the prediction.Our approach is motivated by the fact that stereo spike depth estimation achieves better results in closer range while monocular spike depth estimation obtains better results in farther range. Therefore, we introduce an Uncertainty-Guided Depth Fusion (UGDF) framework with a joint training strategy and estimate the distributed uncertainty to fuse the monocular and stereo results. In order to demonstrate the advantage of spike depth estimation over traditional camera-based depth estimation, we contribute a spike-depth dataset named CitySpike20K, which contains 20K paired samples, for spike depth estimation. We also introduce the Spike-Kitti dataset to demonstrate the effectiveness and generalization of our method under real-world scenarios.Extensive experiments are conducted to evaluate our method on CitySpike20K and Spike-Kitti. UGDF achieves state-of-the-art results on both CitySpike20K and Spike-Kitti, surpassing all the monocular or stereo spike depth estimation baselines. To the best of our knowledge, our framework is the first end-to-end dual-task fusion framework for spike camera depth estimation. Code and dataset will be released.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Submission Guidelines: Yes

Please Choose The Closest Area That Your Submission Falls Into: Applications (eg, speech processing, computer vision, NLP)