DEA-NIMC: Dynamic Energy-Aware Policy for Near/In-Memory Computing Hybrid Architecture

Yu-Cheng Wu, Chi-Tse Huang, An-Yeu Andy Wu

Abstract: In-memory computing (IMC) has become the current trend to accelerate the inference of deep neural networks (DNNs). Nonetheless, IMC suffers from variations that significantly degrade the inference accuracy, while near-memory computing (NMC) maintains the ideal accuracy but at the expense of energy efficiency. In this work, we leverage the NMC/IMC hybrid architecture and propose a dynamic energy-aware policy to strike a better trade-off between accuracy and energy efficiency. Our approach takes advantage of deep reinforcement learning (DRL) to dynamically allocate workloads between NMC and IMC at the data level. Furthermore, we consider the varying energy overhead of NMC usage across different DNN layers. Compared with the prior works, we enhance the accuracy by up to 8.8% on CIFAR-10 and 4.6% on CIFAR-100 while consuming the same amount of energy.