Accelerating event-based deep neural networks via flexible data encoding

Yuanli Zhong, Yongqi Xu, Bosheng Liu, Yibing Tang, Jigang Wu

Abstract: Event-based deep neural networks (DNNs) have shown great promise in computer vision under difficult lighting conditions. However, existing hardware solutions cannot provide efficient event-based DNN accelerations owing to the characteristic of event streams, which are typically in low datarate and high-dynamic range. In this letter, we present a novel hardware design that can handle event-based DNNs according to the data characteristic of event streams. Furthermore, we provide a dataflow that enables flexible DNN data encodings (including both bitmask and compressed sparse row (CSR)) based on the event data characteristic for energy saving. Comprehensive evaluations based on four famous event-based benchmarks show that the proposed design can achieve higher performance and better energy efficiency compared with representative accelerator baselines.