VinaLLaMA: LLaMA-based Vietnamese Foundation Model

Anonymous

Abstract: In this paper, we present VinaLLaMA, an open-weight, state-of-the-art (SOTA) Large Language Model for the Vietnamese language, built upon LLaMA-2 with an additional 800 billion trained tokens. VinaLLaMA not only demonstrates fluency in Vietnamese but also exhibits a profound understanding of Vietnamese culture. VinaLLaMA-7B-chat, trained on 1 million high-quality synthetic samples, achieves SOTA results on key benchmarks, including VLSP, VMLU, and Vicuna Vietnamese Benchmark, marking a significant advancement in the Vietnamese AI landscape and offering a versatile resource for various applications.

Paper Type: short

Research Area: Generation

Contribution Types: Publicly available software and/or pre-trained models

Languages Studied: Vietnamese, English