Positional Artefacts Propagate Through Masked Language Model Embeddings

Ziyang Luo, Artur Kulmizev, Xiaoxi Mao

Abstract: Ziyang Luo, Artur Kulmizev, Xiaoxi Mao. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.