Contrastive Decoding Reduces Hallucinations in Large Multilingual Machine Translation Models

Anonymous

Abstract: In Neural Machine Translation (NMT), models will sometimes generate repetitive or fluent output that is not grounded in the source sentence. This phenomenon is known as hallucination and is a problem even in large-scale multilingual translation models. We propose to use Contrastive Decoding, an algorithm developed to improve generation from unconditional language models, to mitigate hallucinations in NMT. Specifically, we maximise the log-likelihood difference between a model and the same model with reduced contribution from the encoder outputs. Additionally, we propose an alternative implementation of Contrastive Decoding that dynamically weights the difference based on the maximum probability in the output distribution to reduce the effect of CD when the model is confident of its prediction. We evaluate our methods using the Small (418M) and Medium (1.2B) M2M models across 21 low and medium-resource language pairs. Our results show a 14.6 ± 0.5 and 11.0 ± 0.6 maximal increase in the mean COMET scores for the Small and Medium models on those sentences for which the M2M models initially generate a hallucination., respectively.

Paper Type: long

Research Area: Machine Translation

Contribution Types: Model analysis & interpretability, Approaches to low-resource settings, Publicly available software and/or pre-trained models, Data analysis

Languages Studied: 'ar', 'ast', 'be', 'bn', 'el', 'en', 'fa', 'fr', 'hi', 'oc', 'ps', 'ru', 'sw', 'tr', 'zh'

Consent To Share Submission Details: On behalf of all authors, we agree to the terms above to share our submission details.