Characterizing Long-Tail Categories on Graphs via A Theory-Driven Framework
Keywords: long-tail learning
Abstract: In the context of long-tailed classification on graphs, the vast majority of existing work primarily revolves around the development of model debiasing strategies, with the aim of mitigating class imbalances and enhancing overall performance. Despite the notable success, there is very limited literature that provides a theoretical tool for characterizing the behaviors of long-tail categories in graphs and gaining insight into generalization performance in real-world scenarios. To bridge this gap, we propose the first generalization bound for long-tail classification on graphs by formulating the problem in the fashion of multi-task learning, i.e., each task corresponds to the prediction of one particular category. Our theoretical results show that the generalization performance of long-tailed classification is dominated by the overall loss range and the total number of tasks. Building upon the theoretical findings, we propose a novel generic framework Tail2Learn for long-tailed classification on graphs. In particular, we start with a hierarchical task grouping module that allows us to assign related tasks into hypertasks and thus control the complexity of task space; then, we further design a balanced contrastive learning module to adaptively balance the gradients of both head and tail classes to control the loss range across all tasks in a unified fashion. Finally, extensive experiments demonstrate the effectiveness of Tail2Learn in characterizing long-tail categories on real graphs. We publish our data and code at https://anonymous.4open.science/r/Tail2Learn-CE08/.
Primary Area: representation learning for computer vision, audio, language, and other modalities
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 6043
Loading