Think Twice: Measuring the Efficiency of Eliminating Model Biases in Question Answering

Anonymous

Abstract: While the Large Language Models (LLMs) dominate a majority of language understanding tasks, previous work shows that some of these results are supported by modelling biases of training datasets. Authors commonly assess model robustness by evaluating their models on out-of-distribution (OOD) datasets of the same task, but these datasets might share the biases of the training dataset. We introduce a framework for finer-grained analysis of discovered model biases and measure the significance of some previously-reported biases while uncovering several new ones.The bias-level metric allows us to assess how well different pre-trained models and state-of-the-art debiasing methods mitigate the identified biases in Question Answering (QA) and compare their results to a resampling baseline. We find cases where bias mitigation hurts OOD performance and, on the contrary, when bias enlargement corresponds to improvements in OOD, suggesting that some biases are shared among QA datasets and motivating future work to refine the analyses of LLMs' robustness.

Paper Type: long