Speaker Information Can Guide Models to Better Inductive Biases: A Case Study On Predicting Code-Switching

Anonymous

Abstract: Natural language processing (NLP) models trained on people-generated data can be unreliable because, without any constraints, they can learn from spurious correlations or propagate dangerous biases about personal identities. We hypothesize that enriching models with speaker information in a controlled, educated way can guide them to pick up on relevant inductive biases. For the speaker-driven task of predicting code-switching points in English--Spanish bilingual dialogues, we show that adding sociolinguistically-grounded speaker features as prepended prompts significantly helps to improve accuracy. We find that by adding influential phrases to the input, speaker-informed models learn useful and explainable linguistic information. To our knowledge, we are the first to incorporate speaker characteristics in the code-switching setup, and more generally, take a step towards developing transparent models that control for biases in person-centric tasks.