Decomposition-Based Evolutionary Multiobjective Optimization to Self-Paced Learning

Maoguo Gong, Hao Li, Deyu Meng, Qiguang Miao, Jia Liu

Abstract: Self-paced learning (SPL) is a recently proposed paradigm to imitate the learning process of humans/animals. SPL involves easier samples into training at first and then gradually takes more complex ones into consideration. Current SPL regimes incorporate a self-paced (SP) regularizer into the learning objective with a gradually increasing pace parameter. Therefore, it is difficult to obtain the solution path of the SPL regime and determine where to optimally stop this increasing process. In this paper, a multiobjective SPL method is proposed to optimize the loss function and the SP regularizer simultaneously. A decomposition-based multiobjective particle swarm optimization algorithm is used to simultaneously optimize the two objectives for obtaining the solutions. In the proposed method, a polynomial soft weighting regularizer is proposed to penalize the loss. Theoretical studies are conducted to show that the previous regularizers are roughly particular cases of the proposed polynomial soft weighting regularizer family. Then an implicit decomposition method is proposed to search the solutions with respect to the sample number involved into training. A set of solutions can be obtained by the proposed method and naturally constitute the solution path of the SPL regime. Then a satisfactory solution can be naturally obtained from these solutions by utilizing some effective tools in evolutionary multiobjective optimization. Experiments on matrix factorization and classification problems demonstrate the effectiveness of the proposed technique.