Breakthrough Edge AI Inference Performance Using Northpole in 3U Vpx Form Factor

Filipp Akopyan, William P. Risk, John V. Arthur, Andrew S. Cassidy, Michael V. DeBole, Carlos Ortega Otero, Jun Sawada, Evan Colgan, Michael E. Criscolo, Phillip V. Mann, Heinz Baier, Kai Schleupen, Arnon Amir, Alexander Andreopoulos, Rathinakumar Appuswamy, Deepika Bablani, Peter J. Carlson, Pallab Datta, Steven K. Esser, Myron D. Flickner et al. (17 additional authors not shown)

Abstract: We present preliminary results demonstrating AI (artificial intelligence) inference using the IBM AIU NorthPole Chip [1], [2] incorporated into a compact, rugged 3U VPX form factor module (NP-VPX) [3]. NP-VPX allows NorthPole to be used in edge applications with stringent cooling requirements, high-speed switch fabrics, and rugged environments. NP-VPX processes 965 frames per second (fps) with a Yolo-v4 network with $640 \times 640$ pixel images at 73.5 W at full-precision accuracy, achieving 13.2 frames / J (fps/W). NP-VPX processes over 40,300 fps with a ResNet-50 network with $224 \times 224$ pixel images at 65.9 W at full-precision accuracy, achieving 611 frames $/ \mathrm{J}$.

External IDs:dblp:conf/hpec/AkopyanRACDOSCC24