Exploring Memory-Oriented Design Optimization of Edge AI Hardware for Extended Reality Applications

Vivek Parmar, Syed Shakib Sarwar, Ziyun Li, Hsien-Hsin S. Lee, Barbara De Salvo, Manan Suri

Abstract: Low-power edge AI capabilities are essential for on-device extended reality (XR) applications to support the vision of the metaverse. In this work, we investigate two representative XR workloads, 1) hand detection and 2) eye segmentation, for hardware design space exploration. For both applications, we train deep neural networks and analyze the impact of quantization and hardware-specific bottlenecks. Through simulations, we evaluate a CPU and two systolic inference accelerator implementations. Next, we compare these hardware solutions with advanced technology nodes. The impact of integrating state-of-the-art emerging nonvolatile memory (NVM) technology (STT-/SOT-/VGSOT-MRAM) into the XR–AI inference pipeline is evaluated. We found that significant energy benefits ($ \geq $≥24%) can be achieved for hand detection [inferences per second (IPS) = 10] and eye segmentation (IPS = 0.1) by introducing NVM into the memory hierarchy for designs at the 7-nm node while meeting minimum IPS values. Moreover, we can realize a substantial reduction in area ($ \geq $≥30%) owing to the small form factor of MRAM.