GEE! Grammar Error Explanation with Large Language Models

Anonymous

Abstract:

Existing grammatical error correction tools do not provide natural language explanations of the errors that they correct in user-written text. However, such explanations are essential for helping users learn the language by gaining a deeper understanding of its grammatical rules (DeKeyser, 2003; Ellis et al., 2006).

To address this gap, we propose the task of grammar error explanation, where a system needs to provide one-sentence explanations for each grammatical error in a pair of erroneous and corrected sentences. The task is not easily solved by prompting LLMs: we find that GPT-4 only attempts to explain 60.2% of the errors using one-shot prompting, regardless of the explanation correctness.

Since LLMs struggle to identify grammar errors, we develop a two-step pipeline that leverages fine-tuned and prompted large language models to perform structured atomic token edit extraction, followed by prompting GPT-4 to explain each edit. We evaluate our pipeline on German and Chinese grammar error correction data. Our atomic edit extraction achieves F1 0.93 and 0.91 on German and Chinese. Human evaluation reveals that, among the generated explanations, 93.9% German and 96.4% Chinese errors are correctly detected and explained. To encourage further research in this area, we will open-source our data and code.

Paper Type: long

Research Area: NLP Applications

Contribution Types: Model analysis & interpretability, NLP engineering experiment

Languages Studied: German, Chinese