Improving Learning to Cross-Lingual Question Answering via Performance Enhancement with XLM-R+ model

Sudarshan Yerragunta, Paramveer Singh, Rajendra Prasath, G. N. Girish

Abstract: Recent advances in Question Answering (QA) have primarily centred on English, driven by the availability of large-scale, high-quality benchmark datasets. However, extending these advancements to other languages poses significant challenges due to the scarcity of annotated data. This motivates our research focus on enhancing Cross-Lingual QA (CLQA) by improving the performance of existing multilingual QA models. We propose XLM-R+, an extension of the XLM-R architecture that integrates sparse attention and enhanced regularization to improve CLQA performance. Evaluated on the MLQA benchmark, which includes seven languages - English, Hindi, Vietnamese, German, Arabic, Spanish, and Simplified Chinese our model demonstrates consistent improvements across language pairs. Our comprehensive experiments reveal effective strategies for extending QA capabilities to low-resource languages and highlight the broader utility of CLQA in real-world applications. In particular, XLM-R+ holds strong potential for multilingual educational platforms, such as language learning tools and intelligent tutoring systems, where content authored in one language can support question answering in another. Experimental results confirm that XLM-R+ outperforms existing SOTA CLQA approaches.

External IDs:dblp:conf/icitl/YerraguntaSPG25