RAG Picking Helps: Retrieval Augmented Generation for Machine Translation

ACL ARR 2024 August Submission459 Authors

Abstract: We introduce RAGMT, a retrieval augmented generation (RAG)-based multi-task framework for Machine Translation (MT) using non-parametric knowledge sources. To the best of our knowledge, we are the first to adapt the RAG framework for MT to support end-to-end training and use knowledge graphs as the non-parametric source. We also propose the use of new auxiliary training objectives that improve the performance of RAG for domain-specific MT. Our experiments demonstrate that retrieval-augmented fine-tuning of NMT models under the RAGMT framework results in an average improvement of 2.03 BLEU scores over simple fine-tuning approaches on English to German domain-specific translation. We also demonstrate the efficacy of RAGMT with using in-domain versus domain-agnostic knowledge graphs and careful ablations over the model components. Qualitatively, RAGMT is easily interpretable and demonstrates "copy-over-translation" behaviour over named entities.

Paper Type: Long

Research Area: Machine Translation

Research Area Keywords: domain adaptation, retrieval-augmented generation

Contribution Types: Model analysis & interpretability, Approaches to low-resource settings, Approaches low compute settings-efficiency, Publicly available software and/or pre-trained models, Data analysis

Languages Studied: English, German

Submission Number: 459