SeCSeq: Semantic Coding for Sequence-to-Sequence based Extreme Multi-label Classification

Wei-Cheng Chang, Hsiang-Fu Yu, Inderjit S. Dhillon, Yiming Yang

Abstract: Extreme multi-label classification (XMC) aims at assigning to an instance the most relevant subset of labels from a colossal label set. There has been some success in formulating the multi-label problem as sequence-to-sequence (Seq2Seq) learning, where the positive class labels of each input instance are used as the corresponding output sequence. Seq2Seq methods, nonetheless, have not yet been scalable to the XMC setting due to the softmax bottleneck. In this paper, we propose a semantic coding framework, namely SeCSeq, for a Seq2Seq approach to the XMC problem. To circumvent the softmax bottleneck, SeCSeq compresses labels into sequences of semantic-aware compact codes, on which Seq2Seq models are trained. For inference, the generated semantic codes are then decompressed into sequences of positive labels using ensemble techniques. Preliminary experiments on XMC benchmark datasets show that SeCSeq is competitive with the state-of-the-art while requiring significantly fewer model parameters.

Keywords: extreme multilabel classification, sequence-to-sequence, random projection quantization, softmax bottleneck