Event-Based Depth Prediction With Deep Spiking Neural Network

Xiaoshan Wu, Weihua He, Man Yao, Ziyang Zhang, Yaoyuan Wang, Bo Xu, Guoqi Li

Abstract: Event cameras have gained popularity in depth estimation due to their superior features such as high-temporal resolution, low latency, and low-power consumption. Spiking neural network (SNN) is a promising approach for processing event camera inputs due to its spike-based event-driven nature. However, SNNs face performance degradation when the network becomes deeper, affecting their performance in depth estimation tasks. To address this issue, we propose a deep spiking U-Net model. Our spiking U-Net architecture leverages refined shortcuts and residual blocks to avoid performance degradation and boost task performance. We also propose a new event representation method designed for multistep SNNs to effectively utilize depth information in the temporal dimension. Our experiments on MVSEC dataset show that the proposed method improves accuracy by 18.50% and 25.18% compared to current state-of-the-art (SOTA) ANN and SNN models, respectively. Moreover, the energy efficiency can be improved up to 58 times by our proposed SNN model compared with the corresponding ANN with the same network structure.