Impact of Spanish Dialect in Deep Learning Next Sentence Predictors

Pablo Duboue

Abstract:

With the second largest number of native speakers in the world, Spanish exhibits a large dialectical difference, both in grammar and word forms. We analyze the impact of these differences in a Natural Language Processing task using the proceedings of two parliamentary bodies from Costa Rica and Argentina. The task chosen is to determine whether to pieces of text are coherent, that is, whether they can follow each other in
normal discourse. We train a deep learning network to perform next sentence prediction using the BERT model. Our experiments indicate that task adaptation presents a more substantive impact than regional differences but dialect still can account for up to 8% difference in F-measure.