IMPARA: Impact-based Metric for GEC using Parallel Data

Anonymous

Paper Link: https://openreview.net/forum?id=JQvJ8rvu1B_

Paper Type: Short paper (up to four pages of content + unlimited references and appendices)

Abstract:

Automatic evaluation of Grammatical Error Correction (GEC) is essential in developing efficient GEC systems.
Existing methods for automatic evaluation require multiple reference sentences or manual scores.
However, such resources are costly, which hinders automatic evaluation for various domains and correction types.
This paper proposes IMpact-based metric for GEC using PARAllel data (IMPARA) that utilizes parallel data consisting of pairs of grammatical/ungrammatical sentences and correction impacts.
Because parallel data can be obtained with less effort than manually assessing evaluation scores, IMPARA can reduce the cost of data creation.
Correlations between IMPARA and human scores show that IMPARA is comparable or better than existing methods.
Furthermore, we find that IMPARA can perform evaluations that fit different domains and correction styles by changing the parallel data.