DragFT: Adapting Large Language Models with Dictionary and Retrieval Augmented Fine-Tuning for Domain-specific Machine Translation
Abstract: Large language models (LLMs) have shown great potential in domain-specific machine translation (MT). However, one major issue is that LLMs trained on general corpus might not generalize well to specific domains due to the lack of domain-specific knowledge. To address this issue, this paper focuses on enhancing the domain-specific MT capability of LLMs, by providing high-quality training datasets and proposing a novel fine-tuning framework denoted by DragFT. DragFT augments LLMs via three techniques: Dictionary-enhanced prompting improves domain-specific terminology translation; RAG-based few-shot example selection provides high-quality examples that simulate both the domain and style characteristics; Fine-tuning with few-shot examples further boosts fine-tuning with in-domain examples. We deploy DragFT on three well-known LLM backbones to validate its effectiveness. The results on three domain-specific datasets show that DragFT achieves a significant performance boost and shows superior performance compared to strong baselines such as GPT-3.5 and GPT-4o. The drastic performance improvement of DragFT over existing LLMs can be attributed to the incorporation of relevant knowledge while mitigating noise.
Our three well-constructed datasets can accelerate future research in domain-specific MT: a benchmark dataset designed for MT within the IT domain, and two datasets constructed from publicly available datasets respectively in law and medicine.
Paper Type: Long
Research Area: Machine Translation
Research Area Keywords: Machine Translation, Language Modeling
Contribution Types: Model analysis & interpretability, NLP engineering experiment, Data analysis
Languages Studied: English, Chinese
Submission Number: 2465
Loading