Online Label Distribution Learning Using Random Vector Functional-Link Network

Jintao Huang, Chi-Man Vong, Wenbin Qian, Qin Huang, Yimin Zhou

Abstract: Compared with multi-label learning (MLL), label distribution learning (LDL) can reflect the significance of relevant labels in samples, so many LDL works have been emerging recently. Nevertheless, existing LDL algorithms cannot cope with the online label distribution data stream that may generate new data distribution regularities, resulting in reduced model accuracy, effectiveness, and stability. This paper proposes a novel LDL framework based on RVFL (random vector functional link) that can effectively and accurately deal with the online data stream, namely OLD_RVFL+, which includes three innovative modules: 1) a novel RVFL+ network for LDL is constructed, namely LD_RVFL+ network. Compared with other SLFNs (single layer feed-forward networks), RVFL+ can address the issues of model over-fitting and model stability and effectively obtain the model weights in the initial training stage with high training efficiency. 2) a novel weight update module based on LD_RVFL+ is developed for the online data stream, which can rapidly update the weight parameters without iterations; 3) a new label thresholding module is proposed to convert the model outputs into a real label distribution for improved accuracy. Experimental comparisons with seven state-of-art label distribution models are carried out on 19 real-world distribution benchmark datasets. Our OLD_RVFL+ outperforms other methods by 5-10% in evaluation performance and outperforms the offline method by 50-100 times in training time under different chunk sizes, demonstrating the effectiveness, efficiency, and stability of the proposed methods.