Utilizing Inherent Randomness in Stochastic Computing

Xiaojia Wang, Yingjie Lao

Abstract: Nowadays, low-cost and efficient integrated circuit (IC) design is of great interest due to the proliferation of computing devices, the resource-constrained nature of these systems, the desire to improve computing capability, and the need to reduce the cost. Stochastic computing has emerged as a promising alternative computing paradigm to binary deterministic computing, which enables very low-cost implementations of arithmetic operations using standard logic elements while achieving a higher degree of fault-resistance. However, it exhibits inherent randomness owing to its stochastic representation of data, which degrades the accuracy of the computation. Instead of trying to eliminate the inherent randomness to improve accuracy, this paper introduces the idea of utilizing the inherent randomness to enhance the performance of stochastic computing applications. We show that the inherent noise in stochastic computing can be utilized to perturb neural networks to escape local optima or saddle points, reduce over-fitting, and augment data. For example, the proposed methodology could improve the test accuracy of a multi-layer convolutional neural network on the CIFAR-10 dataset by 5%.