Data Quality Issues in Multilingual Speech Datasets: The Need for Sociolinguistic Awareness and Proactive Language Planning

ACL ARR 2025 February Submission933 Authors

Abstract: Our quality audit for three widely used public multilingual speech datasets Mozilla Common Voice 17.0, FLEURS, and VoxPopuli shows that in some languages, these datasets suffer from significant quality issues. We believe addressing these issues will make these datasets more useful as evaluation sets, and improve downstream models. We divide these quality issues into two categories: micro-level and macro-level. We find that macro-level issues are more prevalent in less institutionalized, often under-resourced languages. We provide a case analysis of Taiwanese Southern Min (nan_tw}) that highlights the need for proactive language planning (e.g. orthography prescriptions, dialect boundary definition) and enhanced data quality control in the process of Automatic Speech Recognition (ASR) dataset creation. We conclude by proposing guidelines and recommendations to mitigate these issues in future dataset development, emphasizing the importance of sociolinguistic awareness in creating robust and reliable speech data resources.

Paper Type: Long

Research Area: Resources and Evaluation

Research Area Keywords: multilingual corpora, NLP datasets, datasets for low resource languages

Contribution Types: Data resources, Data analysis

Languages Studied: ab, af, am, ar, as, ast, az, ba, bas, be, bg, bn, br, bs, ca, ceb, ckb, cmn, cnh, cs, cv, cy, da, de, dv, dyu, el, en, eo, es, et, eu, fa, ff, fi, fil, fr, fy, ga, gl, gn, gu, ha, he, hi, hr, hsb, ht, hu, hy, ia, id, ig, is, it, ja, jv, ka, kab, kam, kea, kk, km, kmr, kn, ko, ky, lb, lg, lij, ln, lo, lt, ltg, luo, lv, mdf, mhr, mi, mk, ml, mn, mr, mrj, ms, mt, my, myv, nan, nb, ne, nhi, nl, nn, nso, ny, oc, om, or, os, pa, pl, ps, pt, quy, rm, ro, ru, rw, sah, sat, sc, sd, sk, skr, sl, sn, so, sq, sr, sv, sw, ta, te, tg, th, ti, tig, tk, tok, tr, tt, tw, ug, uk, umb, ur, uz, vi, vot, wo, xh, yi, yo, yue, zgh, zu, zza

Submission Number: 933