Too Much in Common: Shifting of Embeddings in Transformer Language Models and its Implications

Daniel Bis, Maksim Podkorytov, Xiuwen Liu