Direct face detection and video reconstruction from event cameras

Souptik Barua, Yoshitaka Miyatani, Ashok Veeraraghavan

Abstract: Event cameras are emerging as a new class of cameras, to potentially rival conventional CMOS cameras, because of their high speed operation and low power consumption. Pixels in an event camera operate in parallel and fire asynchronous spikes when individual pixels encounter a change in intensity that is greater than a pre-determined threshold. Such event-based cameras have an immense potential in battery-operated or always-on application scenarios, owing to their low power consumption. These event-based cameras can be used for direct detection from event streams, and we demonstrate this potential using face detection as an example application. We first propose and develop a patch-based model for the event streams acquired from such cameras. We demonstrate the utility and robustness of the patch-based model for event-based video reconstruction and event-based direct face detection. We are able to reconstruct images and videos at over 2,000 fps from the acquired event streams. In addition, we demonstrate the first direct face detection from event streams, highlighting the potential of these event-based cameras for power-efficient vision applications.