An Ultra-Low-Power Image Signal Processor for Hierarchical Image Recognition With Deep Neural Networks

Hyochan An, Sam Schiferl, Siddharth Venkatesan, Tim Wesley, Qirui Zhang, Jingcheng Wang, Kyojin Choo, Shiyu Liu, Bowen Liu, Ziyun Li, Luyao Gong, Hengfei Zhong, David T. Blaauw, Ronald G. Dreslinski, Hun-Seok Kim, Dennis Sylvester

Abstract: We propose an ultra-low-power (ULP) image signal processor (ISP) that performs on-the-fly in-processing frame compression/decompression and hierarchical event recognition to exploit the temporal and spatial sparsity in an image sequence. This approach reduces energy consumption spent processing and transmitting unimportant image data to achieve a 16 × imaging system energy gain in an intruder detection scenario. The ISP was fabricated in 40-nm CMOS and consumes only 170 μW at 5 frames/s for neural network-based intruder detection and 192 × compressed image recording.