Live Demonstration: CNN Inference on the Focal Plane with a Pixel Processor Array

Stephen J. Carey, Laurie Bose, Thomas S. Richardson, Walterio W. Mayol-Cuevas, Jianing Chen, Piotr Dudek

Abstract: We present a novel method of CNN inference on a pixel processor array device, demonstrating it using a handwritten digit (digits 0–9) classification task, with all steps of the neural network computation performed on the focal plane. The vision chip that we deploy (SCAMP-7) has a 256×256 array of processor elements (PE) integrated within the image sensor. The algorithm runs at over 3000 frames per second (FPS) and over 90% classification accuracy, with the sensor chip only outputting ten scalar values corresponding to the classification scores.