On the Limitations of Dataset Balancing: The Lost Battle Against Spurious Correlations

Anonymous

Paper Link: https://openreview.net/forum?id=Fh7uOUHXrLO

Paper Type: Long paper (up to eight pages of content + unlimited references and appendices)

Abstract: Recent work has shown that deep learning models in NLP are highly sensitive to low-level correlations between simple features and specific output labels, leading to over-fitting and lack of generalization. To mitigate this problem, a common practice is to balance datasets by adding new instances or by filtering out “easy” instances (Sakaguchi et al., 2020), culminating in a recent proposal to eliminate single-word correlations altogether (Gardner et al., 2021). In this opinion paper, we identify that despite these efforts, increasingly-powerful models keep exploiting ever-smaller spurious correlations, and as a result even balancing all single-word features is insufficient for mitigating all of these correlations. In parallel, a truly balanced dataset may be bound to “throw the baby out with the bathwater” and miss important signal encoding common sense and world knowledge. We highlight several alternatives to dataset balancing, focusing on enhancing datasets with richer contexts, allowing models to abstain and interact with users, and turning from large-scale fine-tuning to zero- or few-shot setups.

Copyright Consent Signature (type Name Or NA If Not Transferrable): Gabriel Stanovsky

Copyright Consent Name And Address: School of Computer Science, The Hebrew University of Jerusalem