A Class Incremental Learning Algorithm for a Compact-Sized Probabilistic Neural Network and Its Empirical Comparison with Multilayered Perceptron Neural Networks

Shunpei Morita, Hiroto Iguchi, Tetsuya Hoya

Abstract:

It is well known that class incremental learning using deep
learning is difficult to achieve, since deep learning approaches inherently
suffer from the catastrophic forgetting in the training mode. In contrast,
a probabilistic neural network is capable of performing classification tasks
based upon a set of local spaces, each composed of a training pattern,
and thereby class incremental learning can be robustly performed. In this
paper, we propose a class incremental learning method by exploiting the
property of a probabilistic neural network, while reducing effectively the
number of the training patterns to be stored within the hidden layer,
and compare the performance of the class incremental learning tasks
obtained using a multilayered perceptron model with that using a prob-
abilistic neural network. Simulation results using seven publicly available
datasets show that both the classification accuracies of an original prob-
abilistic neural network and the proposed incremental learning method
are 2.59 to 26.58 times higher than that of the deep learning in class
incremental learning. Moreover, we observed that the class incremental
learning performed using a probabilistic neural network exhibited a ro-
bust performance compared to the deep neural networks with iCaRL. In
addition, it was observed that the proposed learning method was able
to reduce effectively the number of the units in the hidden layer, while
with the decrease in accuracy by only 1.77 % to 7.06 %, compared to the
original one.