11.3 Metis AIPU: A 12nm 15TOPS/W 209.6TOPS SoC for Cost- and Energy-Efficient Inference at the Edge

Pascal Alexander Hager, Bert Moons, Stefan Cosemans, Ioannis A. Papistas, Bram Rooseleer, Jeroen Van Loon, Roel Uytterhoeven, Florian Zaruba, Spyridoula Koumousi, Milos Stanisavljevic, Stefan Mach, Sebastiaan Mutsaards, Riduan Khaddam Aljameh, Gua Hao Khov, Brecht Machiels, Cristian Olar, Anastasios Psarras, Sander Geursen, Jeroen Vermeeren, Yi Lu et al. (28 additional authors not shown)

Abstract: The Metis AI Processing Unit (AIPU) is a quad-core System-on-Chip (SoC) designed for edge inference, executing all components of an AI workload on-chip. The Metis AIPU exhibits performance of 52.4 TOPS per AI core, and a compound throughput of 209.6 TOPS. Key features of the Metis AIPU and its integration into a PCIe card-based system are shown in Fig. 11.3.1. Metis leverages the benefits from a quantized digital in-memory computing (D-IMC) architecture — with 8b weights, 8b activations, and full-precision accumulation — to decrease both the memory cost of weights and activations and the energy consumption of matrix-vector multiplications (MVM), without compromising the neural network accuracy.