Handwritten Digit Recognition Using Multi-Layer Feedforward Neural Networks with Periodic and Monotonic Activation Functions

Kwok-Wo Wong, Chi-Sing Leung, Sheng-Jiang Chang

Abstract: The problem of handwritten digit recognition is dealt with by multilayer feedforward neural networks with different types of neuronal activation functions. Three types of activation functions are adopted in the network, namely, the traditional sigmoid function, sinusoidal function and a periodic function that can be considered as a combination of the first two functions. To speed up the learning, as well as to reduce the network size, an extended Kalman filter algorithm with the pruning method is used to train the network. Simulation results show that periodic activation functions perform better than monotonic ones in solving multi-cluster classification problems such as handwritten digit recognition.