Difficulty-Controllable Multiple-Choice Question Generation for Reading Comprehension Using Item Response Theory

Yuto Tomikawa, Masaki Uto

Abstract: In recent years, there has been increasing interest in automatically generating reading comprehension questions with controllable difficulty levels for educational purposes. A recent study proposed a method for generating reading comprehension questions with levels of difficulty suitable to a learner’s ability level involving the use of large language models and item response theory. However, the conventional method targets only the extractive question format, where the answer can be found directly in the reading passage, and does not support the multiple-choice question format that is widely used in educational settings. To address this issue, this study develops a method for automatically generating multiple-choice questions with controllable difficulty levels by extending the conventional method. We evaluate the performance of difficulty controllability of the generated questions based on the correct answer rate of responses and the nominal response model, a polytomous item response theory model. The results confirm that the proposed method can generate multiple-choice questions that accurately reflect the intended difficulty level.