Iterative Selection with Self-Review for Vocabulary Test Distractor Generation

Yu-Cheng Liu, An-Zi Yen

Abstract: Vocabulary acquisition is essential to second language learning, as it underpins all core language skills. Accurate vocabulary assessment is particularly important in standardized exams, where test items evaluate learners' comprehension and contextual use of words. Previous research has explored methods for generating distractors to aid in the design of English vocabulary tests. However, current approaches often rely on lexical databases or predefined rules, and frequently produce distractors that risk invalidating the question by introducing multiple correct options. In this study, we focus on English vocabulary questions. We analyze how teachers design test items to gain insights into distractor selection strategies. Additionally, we identify key limitations in how large language models (LLMs) support teachers in generating distractors for vocabulary test design. To address these challenges, we propose the iterative selection with self-review (ISSR) framework, which makes use of an LLM-based self-review mechanism to ensure that the distractors remain valid while offering diverse options. ISSR aims to assist educators by providing an LLM-based tool that allows them to efficiently design pedagogically sound distractors through natural language instructions. Experimental results show that ISSR achieves promising performance in generating plausible distractors, and the self-review mechanism effectively filters out distractors that could invalidate the question.

Submission Number: 20