Towards Deconfounding the Influence of Subject's Demographic Characteristics in Question AnsweringDownload PDFOpen Website

2021 (modified: 03 Nov 2022)CoRR 2021Readers: Everyone
Abstract: The goal of question answering (QA) is to answer any question. However, major QA datasets have skewed distributions over gender, profession, and nationality. Despite that skew, model accuracy analysis reveals little evidence that accuracy is lower for people based on gender or nationality; instead, there is more variation on professions (question topic). But QA's lack of representation could itself hide evidence of bias, necessitating QA datasets that better represent global diversity.
0 Replies

Loading