SegMix: A Simple Structure-Aware Data Augmentation Method

Anonymous

Abstract: Many Natural Language Processing tasks involve predicting structures, such as Syntax Parsing and Relation Extraction (RE). One central challenge in supervised structured prediction is the lack of high-quality annotated data. The recently proposed interpolation-based data augmentation (DA) algorithms (i.e. mixup) augment the training set via making convex interpolation between training data points. However, current algorithms (e.g. SeqMix, LADA) that apply mixup to language structured prediction tasks are not aware of the syntactic or output structures of the tasks, making their performance unstable and requiring additional heuristic constraints. Furthermore, SeqMix-like algorithms expect a linear encoding scheme of the output structure, such as BIO-Scheme for Named Entity Recognition (NER), restricting its applicability.To this end, we propose SegMix, a simple framework of interpolation-based algorithms that can adapt to both the syntactic and output structures, making it robust to hyper-parameters and applicable to different tasks. We empirically show that SegMix consistently improves performance over several strong baseline models on two structured prediction tasks (NER and RE). SegMix is a flexible framework that unifies existing rule-based language DA methods, creating interesting mixtures of DA techniques. Furthermore, the method is easy to implement and adds negligible overhead to training and inference.