Revisiting Hierarchical Text Classification: Inference and Metrics

Anonymous

TL;DR: We propose a new dataset and a better adapted evaluation framework for Hierarchical Text Classification, and show that simple baselines can beat SotA models when properly evaluated.

Abstract: Hierarchical text classification (HTC) is the task of assigning labels to a text within a structured space organized as a hierarchy. Recent works treat HTC as a conventional multilabel classification problem, therefore evaluating it as such. We instead propose to evaluate models based on specifically designed hierarchical metrics and we demonstrate the intricacy of metric choice and prediction inference method. We introduce a new and challenging HTC dataset and we evaluate fairly recent sophisticated models, comparing them with a range of simple but strong baselines. Finally, we show that those baselines are very often competitive with the latest HTC models. Our works shows the importance of carefully considering the evaluation methodology when proposing new methods for HTC.

Paper Type: long

Research Area: Machine Learning for NLP

Contribution Types: Reproduction study, Data resources, Theory

Languages Studied: English