On A Strictly Convex IBM Model 1

Andrei Simion, Michael Collins, Cliff Stein

Abstract: IBM Model 1 is a classical alignment model. Of the first generation word-based SMT models, it was the only such model with a concave objective function. For concave optimization problems like IBM Model 1, we have guarantees on the convergence of optimization algorithms such as Expectation Maximization (EM). However, as was pointed out recently, the objective of IBM Model 1 is not strictly concave and there is quite a bit of alignment quality variance within the optimal solution set. In this work we detail a strictly concave version of IBM Model 1 whose EM algorithm is a simple modification of the original EM algorithm of Model 1 and does not require the tuning of a learning rate or the insertion of an l2 penalty. Moreover, by addressing Model 1’s shortcomings, we achieve AER and F-Measure improvements over the classical Model 1 by over 30%.