Extending the METEOR Machine Translation Evaluation Metric to the Phrase Level

Michael J. Denkowski, Alon Lavie

Abstract: This paper presents Meteor-next, an extended version of the Meteor metric designed to have high correlation with post-editing measures of machine translation quality. We describe changes made to the metric's sentence aligner and scoring scheme as well as a method for tuning the metric's parameters to optimize correlation with human-targeted Translation Edit Rate (HTER). We then show that Meteor-next improves correlation with HTER over baseline metrics, including earlier versions of Meteor, and approaches the correlation level of a state-of-the-art metric, TER-plus (TERp).