Adaptive Question-Answer Generation With Difficulty Control Using Item Response Theory and Pretrained Transformer Models

Yuto Tomikawa, Ayaka Suzuki, Masaki Uto

Abstract: The automatic generation of reading comprehension questions, referred to as question generation (QG), is attracting attention in the field of education. To achieve efficient educational applications of QG methods, it is desirable to generate questions with difficulty levels that are appropriate for each learner's reading ability. Therefore, in recent years, several difficulty-controllable QG methods have been proposed. However, conventional methods generate only questions and cannot produce question–answer pairs. Furthermore, such methods ignore the relationship between question difficulty and learner ability, making it challenging to ascertain the appropriate difficulty levels for each learner. To address these issues, in this article, we propose a method for generating question–answer pairs based on difficulty, defined using a statistical model known as item response theory. The proposed difficulty-controllable generation is achieved by extending two pretrained transformer models: bidirectional encoder representations from transformers and text-to-text transfer transformer. In addition, because learners' abilities are generally not knowable in advance, we propose an adaptive QG framework that efficiently estimates the learners' abilities while generating and presenting questions with difficulty levels suitable for their abilities. Through experiments involving real data, we confirmed that the proposed method can generate question–answer pairs with difficulty levels that align with the learners' abilities while efficiently estimating their abilities.