Incremental projection learning for optimal generalization

Masashi Sugiyama, Hidemitsu Ogawa

Abstract: In many practical situations in neural network learning, it is often expected to further improve the generalization capability after the learning process has been completed. One of the common approaches is to add training data to the neural network. In view of the learning methods of human beings, it seems natural to build posterior learning results upon prior results, which is generally referred to as incremental learning. Many incremental learning methods have been devised so far. However, they provide poor generalization capability compared with batch learning methods. In this paper, a method of incremental projection learning in the presence of noise is presented, which provides exactly the same learning result as that obtained by batch projection learning. The effectiveness of the presented method is demonstrated through computer simulations.