Matrix-wise Class Imbalance Matters: On the Generalization of Micro-AUC in Multi-label Learning

Guoqiang Wu, Yan Zhang, Teng Pang, Chongxuan Li, Yilong Yin

Keywords: Micro-AUC, Statistical Learning Theory, Generalization, Multi-Label Learning

Abstract: Micro-AUC is averaging AUC on the prediction matrix in multi-label learning. While it is a commonly-used evaluation measure in practice, the theoretical understanding is far behind. To fill up this gap, this paper takes an initial step to characterize the generalization guarantees of algorithms based on three surrogate losses w.r.t. Micro-AUC. Theoretically, we identify a critical data-dependent quantity affecting the generalization bounds: \emph{the matrix-wise class imbalance}. Our results of the imbalance-aware error bounds show that the commonly-used univariate loss-based algorithm has a worse learning guarantee than the ones with the proposed pairwise and reweighting univariate loss, which probably implies its worse performance. Finally, empirical results of the linear and deep neural network-based models on various benchmarks corroborate our theory findings.

Primary Area: learning theory

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 7442