Extreme Multi-label Text Classification with Multi-layer Experts

Anonymous

Abstract: Extreme multi-label text classification (XMTC) is the task of tagging each document with the relevant labels from a very large space of predefined categories, which presents an open challenge in the recent development of neural classifiers. Popular Transformer-based XMTC methods typically use the last-layer features to represent the document and to match it against candidate labels. We argue that the last-layer features may not be sufficient for predicting labels at different levels of semantic granularity, and that multi-layer features may offer a better choice instead. Based on this insight we propose a novel multi-expert model, namely ME-XML (Multiple Experts for XMTC), which combines multi-layer embeddings in Transformer for improving the prediction power of the model.