Fixing exposure bias with imitation learning needs powerful oracles

Luca Hormann, Artem Sokolov

Abstract: We apply imitation learning (IL) to tackle the NMT exposure bias problem with error-correcting oracles, and evaluate an SMT lattice-based oracle which, despite its excel-lent performance in an unconstrained oracle translation task, turned out to be too pruned and idiosyncratic to serve as the oracle for IL.