SRU-Q: Hardware-friendly Stochastic Rounding Unit-based Gradient Quantization for CNN Training

Sangbeom Jeong, Dahun Choi, Hyun Kim

Abstract: Quantization in convolutional neural networks (CNNs) involves using a low precision to decrease convolution operation costs, resulting in reduced power consumption and faster network performance. Notably, gradient quantization plays a crucial role in CNN training accelerators, as backpropagation typically incurs higher computational costs for calculating weight gradients than forward propagation does. Stochastic rounding (SR), which employs random number generation, is recognized as an effective method for stabilizing backpropagation quantization. However, the process of generating random numbers in hardware has significant drawbacks—notably high computational costs and substantial difficulties in implementation. This paper introduces a technique for efficient SR using a hardware-optimized random number generator, termed linear feedback shift register-bitwise-stochastic rounding unit (LBSRU). The LBSRU efficiently conducts SR with a small amount of random number generation and adapts to various network types by altering the random number generation approach for different batch sizes. Specifically, we designed and synthesized our method on the FPGA platform to create a prototype. A comparison with previous studies revealed that our method requires significantly fewer resources: 98.19% fewer lookup tables (LUTs) and 98.38% fewer flip-flops (FFs).