Live Demonstration: DVS-CIS Sensor Fusion System for Real-Time DNN-Based Object Detection

Mincheol Cha, Keehyuk Lee, Bobaro Chang, Soosung Kim, Taeho Lee, Xuan Truong Nguyen, Tae Sung Kim, Hyuk-Jae Lee, Hyunsurk Ryu

Abstract: This demonstration presents a high-speed, energy-efficient sensor fusion system integrating CMOS Image Sensors (CIS) and Dynamic Vision Sensors (DVS) for advanced image recognition. Using CIS for high-res imaging and DVS for rapid event-driven capture, the FPGA-implemented architecture with an NPU running YOLOv3-Tiny achieves 18 ms inference latency with a minimal 2.78% mAP loss. Selective NPU activation based on DVS-detected regions yielded 31.5% power savings, while a custom receiver module efficiently fused DVS (13,900 fps) and CIS (60 fps) data. The system uses a power of 6.977 W on a Xilinx Zynq+ ZCU106 board.

External IDs:dblp:conf/iscas/ChaLC0LNKLR25