Eagle: A Family of Advanced Arabic Large Language Models
Abstract: In this paper, we present Eagle, a suite of large language models (LLMs) designed for the Arabic language, built on Mistral-7b, LLaMA2-7b, and LLaMA3-7b. These models are pre-trained on the Oasis dataset, containing approximately 35 billion Arabic tokens, and further enhanced through instruction fine-tuning and reinforcement learning with AI feedback. We also introduce Amwaj, an Arabic embedding model for retrieval-augmented generation, and AraPO, a novel alignment method for improved Arabic culture alignment. To evaluate our models, we present OpenArabicEval, a diverse benchmark of 32 datasets covering comprehensive multiple-choice evaluation, natural language understanding, natural language generation, and long context evaluation. Extensive testing with OpenArabicEval demonstrates our models’ exceptional performance and robustness across various NLP tasks, highlighting their effectiveness in processing Arabic. OpenArabicEval is the first benchmark to feature Long Context evaluation for Arabic LLMs.
Paper Type: Long
Research Area: Multilingualism and Cross-Lingual NLP
Research Area Keywords: corpus creation, benchmarking, language resources, multilingual corpora, lexicon creation, automatic creation and evaluation of language resources
Contribution Types: Model analysis & interpretability, NLP engineering experiment, Approaches to low-resource settings, Approaches low compute settings-efficiency
Languages Studied: Arabic, English
Submission Number: 3736
Loading