Direct Preference Optimization for Neural Machine Translation with Minimum Bayes Risk Decoding

Anonymous

TL;DR: A Reinforcement Learning technique, Direct Preference Optimization, can be used fine-tune MLLMs to get the gains of Minimum Bayes Risk decoding without any additional computation in inference.

Abstract: Minimum Bayes Risk (MBR) decoding can significantly improve translation performance of Multilingual Large Language Models (MLLMs). However, MBR decoding is computationally expensive. We show how the recently developed Reinforcement Learning technique, Direct Preference Optimization (DPO), can fine-tune MLLMs to get the gains of MBR without any additional computation in inference. Our method uses only a small monolingual fine-tuning set and yields significantly improved performance on multiple NMT test sets compared to MLLMs without DPO.

Paper Type: short

Research Area: Machine Translation

Contribution Types: Model analysis & interpretability, NLP engineering experiment, Approaches to low-resource settings, Approaches low compute settings-efficiency, Publicly available software and/or pre-trained models

Languages Studied: English, Chinese, French