Breakthrough edge AI inference performance using NorthPole in 3U VPX form factor

Alexander Andreopoulos

Abstract:

We present preliminary results demonstrating AI
(artificial intelligence) inference using the IBM AIU NorthPole
Chip [1], [2] incorporated into a compact, rugged 3U VPX form
factor module (NP-VPX) [3]. NP-VPX allows NorthPole to be
used in edge applications with stringent cooling requirements,
high-speed switch fabrics, and rugged environments. NP-VPX
processes 965 frames per second (fps) with a Yolo-v4 network
with 640 ⇥ 640 pixel images at 73.5 W at full-precision accuracy,
achieving 13.2 frames / J (fps / W). NP-VPX processes over 40,300
fps with a ResNet-50 network with 224 ⇥ 224 pixel images at 65.9
W at full-precision accuracy, achieving 611 frames / J.