Mitigating the Impact of ReRAM I-V Nonlinearity and IR Drop via Fast Offline Network Training

Sugil Lee, Mohammed E. Fouda, Chenghao Quan, Jongeun Lee, Ahmed M. Eltawil, Fadi J. Kurdahi

Abstract: ReRAM crossbar arrays (RCAs) have the potential to provide extremely high efficiency for accelerating deep neural networks (DNNs). However, one crucial challenge for RCA-based DNN accelerators is functional inaccuracy due to nonidealities present in RCA hardware. While nonideality-aware training (NAT) could be used to mitigate the effect of nonidealities, with currently available methods it would take months to train even a medium size convolutional neural network (CNN). In this article we propose a nonideality prediction method that enables very fast training of RCA-based neural networks, and show its feasibility through NAT of DNNs. Our key ideas include 1) weight-centric nonideality modeling and 2) data-dependence elimination by tailored input randomization. Our experimental results using a multilayer perceptron and CNNs demonstrate that our method is very fast ( $100\sim 15$ $000\times $ faster training speed) while achieving much better-crossbar-level accuracy ( $2 \sim 90\times $ lower-RMS error) and post-retraining validated accuracy than previous methods.

External IDs:dblp:journals/tcad/LeeFQLEK25