A unified view of forward and backward losses for learning from weak labels

Daniel Bacaicoa-Barber, Jesús Cid-Sueiro

Abstract: Training multiclass classifiers on weakly labeled datasets, where labels provide only partial or noisy information about the true class, poses a significant challenge in machine learning. To address various forms of label corruption, including noisy, complementary, supplementary, or partial labels, as well as positive-unlabeled data, forward and backward correction losses have been widely employed. Adopting a general formulation that encompasses all these types of label corruption, we introduce a new family of loss functions, termed forward-backward losses, which generalizes both forward and backward correction. We analyze the theoretical properties of this family, providing sufficient conditions under which these losses are proper, ranking-calibrated, classification-calibrated, convex, or lower-bounded. This unified view will be useful to show, through theoretical analysis and experiments, that proper forward losses consistently outperform other forward-backward losses in terms of robustness and accuracy. However, the optimal choice of loss for ranking- and classification-calibrated settings remains an open question. Our work provides a comprehensive framework for weak label learning, offering new directions to develop more robust and effective algorithms.

External IDs:dblp:journals/ml/BacaicoaBarberC25