OpenKiwi: An Open Source Framework for Quality Estimation

Sony Trenous, Fabio N Kepler, Andre Martins, Marcos Vinicius Treviso, Miguel Vera

Abstract:

We introduce OpenKiwi, a PyTorch-based
open source framework for translation quality estimation. OpenKiwi supports training
and testing of word-level and sentence-level
quality estimation systems, implementing the
winning systems of the WMT 2015–18 quality estimation campaigns. We benchmark
OpenKiwi on two datasets from WMT 2018
(English-German SMT and NMT), yielding
state-of-the-art performance on the word-level
tasks and near state-of-the-art in the sentencelevel tasks.