An Embarrassingly Simple Method to Mitigate Undesirable Properties of Pretrained Language Model Tokenizers

Valentin Hofmann, Hinrich Schütze, Janet B. Pierrehumbert