Some Grammatical Errors are Frequent, Others are Important

Leshem Choshen, Ofir Shifman, Omri Abend

Abstract: In Grammatical Error Correction, systems are evaluated by the number of errors they correct. However, no one has assessed whether all error types are equally important. We provide and apply a method to quantify the importance of different grammatical error types to humans. We show that some rare errors are considered disturbing while other common ones are not. This affects possible directions to improve both systems and their evaluation.