Uncovering the Intention Behind Equations in Mathematical Problems
Abstract: Mathematical Equation Intent Recognition(MEIR) is a novel task aimed at identifying the intentions behind mathematical equations that people produce while solving math world problems(MWPs). We observe that, in previous research, researchers have often focused on how to let large language models(LLMs) correctly solve an MWP. However, focusing solely on the reasoning behind each step of a correct inference process is insufficient. We prefer that LLMs can provide guidance on the process of solving MWPs for students in educational settings. Therefore, they need to adjust the strategy based on the student's responses. We notice that, unlike existing mathematical datasets, students typically do not provide overly detailed descriptions of their steps in the real world. As a result, it is crucial for LLMs to possess the capability to understand the intention they produce those equations. We treat MEIR as a generation task, requiring models to summarize the intent in a single sentence. We also propose a data augmentation framework and utilized this framework to generate a benchmark called Grade School Math Intention(GSMI). To evaluate MEIR task, we benchmark serveral LLMs on GSMI dataset. The results indicate that there is still significant room for improvement in the performance of general-purpose LLMs on the MEIR task. Conversely, capabilities acquired during pre-training and fine-tuning specifically in the field of mathematics significantly contribute to the model's ability to tackle those problems. Codes and datasets are available on https://github.com/ch-666-six/MEIR
Paper Type: Long
Research Area: NLP Applications
Research Area Keywords: mathematical NLP, educational applications
Contribution Types: Model analysis & interpretability
Languages Studied: English
Submission Number: 2528
Loading