A novel robust semi-supervised stochastic configuration network for regression tasks with noise

Shifei Ding, Zi Zhang, Li Xu, Chenglong Zhang, Lili Guo, Xuan Li

Abstract: The Stochastic Configuration Network (SCN) is an incremental neural network that adjusts hidden layer inputs using a supervised mechanism and calculates the output weights using the least squares method. However, its generalization performance and robustness significantly degrade with limited labeled data and noise interference. To enhance SCN's regression performance in these scenarios, we propose a novel Robust Semi-Supervised Stochastic Configuration Network (RS3CN) and introduce its semi-supervised variant (S3CN), along with robust versions (RS3CN-Huber and RS3CN-IQR) for comparative experiments. RS3CN employs kernel density estimation (KDE) to evaluate the distribution of labeled training samples, thereby minimizing the effects of noise and outliers. Manifold regularization (MR) is also applied to learn features from unlabeled data. Combining these techniques enhances the SCN's generalization performance in such scenarios. Additionally, we introduce an L2 regularization term to handle outliers in sparse features, reducing overfitting. Finally, we demonstrate its universal approximation property within an enhanced robust semi-supervised optimization framework. Simulation experiments on benchmark datasets show a significant improvement in both semi-supervised learning and robustness of the proposed RS3CN, compared to existing SCN-related algorithms.