Integrated Optimization Method of Hidden Parameters in Incremental Extreme Learning Machine

Siyuan Zhang, Linbo Xie

Abstract: Incremental Extreme Learning Machine is one of the constructive neural networks and provides a fast architecture building mechanism by adding the hidden layer neurons incrementally. There are two phases in constructing the newly added nodes. One is to assign the weights of the hidden nodes quickly with different methods, and the other is to compute the output weights by the least squares methods after obtaining the parameters in the previous phase. Nevertheless, it has a basic deficiency in the aforementioned construction scheme that there is no guarantee on the simultaneous optimization of the weights in the hidden and output layers, respectively, which may produce a lot of redundant nodes in the final models. In this paper, a new integrated optimization method is proposed to construct the simultaneously optimized weights in the first and second phases, and the corresponding integrated optimization incremental-ELM (termed as IOI-ELM) is established. Furthermore, a novel convergence analysis method is developed to provide an upper constraint bound for the selection of the parameters. The simulation results demonstrate and verify that the performance of our approach is much better than other constructive learner models.