Automated Question-Answer Generation for Evaluating Vietnamese RAG-Based Approaches

Dinh-Huan Nguyen, Long Nguyen

Abstract: Recent advances in natural language processing have showcased the power of Large Language Models (LLMs), but evaluating their performance on Vietnamese multi-hop reasoning tasks remains challenging. Traditional approaches to creating Vietnamese evaluation datasets have been constrained by their reliance on manual processes and proprietary models, resulting in limited scalability and substantial resource requirements. We present a novel framework that automates the generation of Vietnamese multi-hop question-answering datasets by leveraging open-source models with enhanced prompting strategies. Our experiments reveal that despite the general advantages of proprietary models, targeted prompt optimization and linguistic adjustments enable open-source alternatives to deliver strong performance. The effectiveness of our approach shows that successful multilingual systems require both robust models and careful attention to language-specific requirements. By offering an efficient and scalable methodology for Vietnamese dataset development, our work helps democratize access to sophisticated language technology while ensuring output quality.

External IDs:doi:10.1007/978-3-031-98164-7_21