Rethinking learning difficulty and uncertainty of samples with a target perturbation-aware bias-variance decomposition

Rujing Yao, Ou Wu, Fang Wang

Abstract: Learning difficulty and uncertainty are two essential properties of samples in machine learning. Treating training samples unequally according to their learning difficulties or uncertainties can improve learning performance in various learning tasks. In previous literature, these two properties are usually independently applied or explored. This study revisits these two learning properties in the learning cases when perturbations exist for the ground-truth target value of each sample. First, we propose a new bias-variance decomposition for generalization errors when target perturbations exist. Second, the learning difficulty and the uncertainty of samples are reformulated in a unified view on the basis of the new decomposition. Learning difficulty is divided into data, model, and coupled difficulties. Uncertainty can be seen as a part of learning difficulty. Third, we take linear regression as an example for the inference of the target perturbation. We design regression experiments to empirically explore the influence of the target perturbation on the learning difficulty. In addition, experiments on linear regression verify the effectiveness of our two proposed perturbation-aware linear regression methods.