Combining the Best of Both Worlds: A Method for Hybrid NMT and LLM Translation

ACL ARR 2024 December Submission515 Authors

Abstract: Large language model (LLM) shows promising performances in a variety of downstream tasks, such as machine translation (MT). However, using LLMs for translation suffers from high computational costs and significant latency. Based on our evaluation, in most cases, translations using LLMs are comparable to that generated by neural machine translation (NMT) systems. Only in particular scenarios, LLM and NMT models show respective advantages. As a result, integrating NMT and LLM for translation and using LLM only when necessary seems to be a sound solution. A scheduling policy that optimizes translation result while ensuring fast speed and as less LLM usage as possible is thereby required. We compare several scheduling policies and propose a novel and straightforward decider that leverages source sentence features. We conduct extensive experiments on multilingual test sets and the result shows that we can achieve optimal translation performance with less LLM usage, demonstrating effectiveness of our decider.

Paper Type: Short

Research Area: Machine Translation

Research Area Keywords: efficient inference for MT

Contribution Types: Approaches low compute settings-efficiency

Languages Studied: English, Chinese, German, Japanese

Submission Number: 515