Challenges in including extra-linguistic context in pre-trained language models

Ionut Sorodoc, Laura Aina, Gemma Boleda