Incremental learning paradigm with privileged information for random vector functional-link networks: IRVFL+

Wei Dai, Yanshuang Ao, Linna Zhou, Ping Zhou, Xuesong Wang

Abstract: Learning using privileged information (LUPI) paradigm, which pioneered teacher–student interaction mechanism, makes the learning models use additional information in the training stage. This paper is the first to propose an incremental learning algorithm with LUPI paradigm for random vector functional-link (RVFL) networks, named IRVFL+ . This novel algorithm can leverage privileged information into incremental RVFL (IRVFL) networks in the training stage, which provides a new constructive method to train IRVFL networks. In order to solve two scenarios that require fast speed of modeling but low-accuracy requirements and high accuracy but slow speed of modeling requirements, two algorithmic implementations of IRVFL+ , respectively, based on local update and global update strategies are presented for data classification and regression problems in this paper. Specifically, the first algorithm, named IRVFL-I+ , calculates the output weights of the newly added hidden nodes, while the input and output parameters of all the existing hidden nodes are fixed. In contrast to IRVFL-I+ , the second one named IRVFL-II + can update all the parameters of all the existing hidden nodes and newly added hidden nodes. Moreover, the convergences of two implementations have been studied in this paper. Finally, experimental results indicate that IRVFL+ indeed performs favorably.