Mask More and Mask Later: Efficient Pre-training of Masked Language Models by Disentangling the [MASK] Token

Baohao Liao, David Thulke, Sanjika Hewavitharana, Hermann Ney, Christof Monz