Fault-Tolerance Mechanism Analysis on NVDLA-Based Design Using Open Neural Network Compiler and Quantization Calibrator

Shu-Ming Liu, Luba Tang, Ning-Chi Huang, Der-Yu Tsai, Ming-Xue Yang, Kai-Chiang Wu

Abstract: The NVIDIA Deep Learning Accelerator (NVDLA) provides free intellectual property licensing to IC chip vendors and researchers to build a chip that uses deep neural networks for inference applications. The Open Neural Network Compiler (ONNC) provides an extensible compiler, a quantization calibrator and optimization supports for running DNN models on NVDLA-based SoCs. Even with open-sourced NVDLA and ONNC, conducting the development of an AI chip still brings up many productivity issues in the mass production stage, such as SRAM MBIST (Memory Built-In Self Test) fail, scan-chain fail etc. When applying Fault-Tolerance Mechanism in error-tolerant applications such as image classification by using the AI CNN model, this paper presents a light-weight Fault-Tolerance Mechanism to effectively enhance the robustness of NVDLA-based edge AI chip when encountering internal SRAM stuck fault. Our non-accurate MAC calculation for the whole convolution computation leads to a very promising quality of results compared to the case when an exactly accurate convolution operation is used. The Fault-Tolerance Mechanism analysis and design described in this paper can also apply to the similar fixed-point deep learning accelerator design, and opens new opportunities for research as well as product development.