Circuit Design of a Seven-piecewise Linear Activation Function

Ren Cai, LeYang, Zhanhui Jiang, Zhixia Ding, Sai Li

Abstract: Nonlinear activation function is a type of function that operates within artificial neural networks, introducing nonlinearity into the network, which enables the network to be applied to a variety of nonlinear models. In the design of nonlinear activation function circuits for memristive neural networks, the main approach involves using three-piecewise linear activation functions to approximate the sigmoid and tanh functions. This paper proposes a seven-piecewise linear activation function to fit the sigmoid and tanh functions. The circuit is divided into three modules: Signal sending module converts the input voltage signal into line segments with different slopes; Signal processing module is used to resolve the issue of voltage signal discontinuities. The output voltage from Signal processing module is then processed by Signal output module for final output, which is subsequently transformed into sigmoid and tanh functions. PSPICE simulation was used to verify the correctness of the design. Subsequently, the proposed seven-piecewise linear activation function was incorporated into an iris classification network. The effectiveness of the design was demonstrated by the recognition rate of the iris classification task.

Submission Number: 11