TailMix: Overcoming the Label Sparsity for Extreme Multi-label Classification

Sangwoo Han, Chan Lim, Jongwuk Lee

Keywords: NLP, classification, extreme multi-label learning, data augmentation

Abstract: Extreme multi-label classification (XMC) aims at finding the most relevant labels from a huge label set at the industrial scale. The XMC problem inherently poses two challenges: data scalability and label sparsity. This work introduces a new augmentation method, namely TailMix, to address the label sparsity issue, i.e., the long-tail labels in XMC have few positive instances. TailMix utilizes the context vector generated from the label attention layer in a label-wise manner instead of using the existing Mixup methods in a sample-wise manner. In this process, TailMix selectively chooses two context vectors and augments the most plausible positive instances to improve the accuracy for long-tail labels. Despite the simplicity of TailMix, extensive experimental results show that TailMix consistently improves the baseline models without TailMix and other Mixup-based methods on three benchmark datasets. Notably, TailMix is effective for improving the performance for long-tail labels on PSP@k and PSN@k, which are the common metrics that reflect the propensity of labels.

One-sentence Summary: We propose a novel Mixup-based data augmentation for extreme multi-label classification.