Employing Convolutional Neural Networks for Continual Learning

Marcin Jasinski, Michal Wozniak

Abstract: The main motivation for the presented research was to investigate the behavior of different convolutional neural network architectures in the analysis of non-stationary data streams. Learning a model on continuously incoming data is different from learning where a complete learning set is immediately available. However, streaming data is definitely closer to reality, as nowadays, most data needs to be analyzed as soon as it arrives (e.g., in the case of anti-fraud systems, cybersecurity, and analysis of images from on-board cameras and other sensors). Besides the vital aspect related to the limitations of computational and memory resources that the proposed algorithms must consider, one of the critical difficulties is the possibility of concept drift. This phenomenon means that the probabilistic characteristics of the considered task change, and this, in consequence, may lead to a significant decrease in classification accuracy. This paper pays special attention to models of convolutional neural networks based on probabilistic methods: Monte Carlo dropout and Bayesian convolutional neural networks. Of particular interest was the aspect related to the uncertainty of predictions returned by the model. Such a situation may occur mainly during the classification of drifting data streams. Under such conditions, the prediction system should be able to return information about the high uncertainty of predictions and the need to take action to update the model used. This paper aims to study the behavior of the network of the models mentioned above in the task of classification of non-stationary data streams and to determine the impact of the occurrence of a sudden drift on the accuracy and uncertainty of the predictions.