Modeling Multi-granularity Segmentation for Rare Words in Neural Machine Translation

Anonymous

Abstract: Segmenting rare words into subwords has become a commonly used and effective way to alleviate the open vocabulary problem in Neural Machine Translation (NMT). The existing dominant segmentation methods either give rare words a single segmentation or a fixed segmentation, which leads to a lack of morphological diversity in representing words. For rare words, we first obtain segmentation with different granularities through Byte Pair Encoding (BPE) and BPE-Dropout, and then propose \textsc{BPEatt} model to dynamically mix the BPE subwords and BPE-Dropout subwords, which enhances the encoder's ability to represent rich morphological information. Experiments on six translation benchmarks of different scales show that our proposed method significantly outperforms the baseline model and has obvious advantages over related methods.