Learning Skill-level Student Abilities with Item Response Theory

Guimei Liu, Jung-jae Kim, Yu Lu, Xiaoli Li

Keywords: knowledge tracing, item response theory, student abilities

TL;DR: We propose an item response theory based deep knowledge tracing model called SKKT-IRT for learning skill-level student abilities with SOTA accuracy on the next question correctness prediction task.

Abstract: Knowledge tracing (KT) aims to estimate knowledge states of students over a given set of skills based on their historical learning activities. The learned knowledge states of students can be used to build skill-meters to understand the weak areas of students so that proper interventions can be taken to help students. Many deep learning models have been applied to KT with encouraging performance, but they either have relatively low accuracy or do not directly generate students' knowledge states at skill level for skill-meter building. Item Response Theory (IRT) models student knowledge states (ability) and question characteristics separately. A question arising naturally is whether we can use IRT to estimate students' knowledge states at skill level while achieving high prediction accuracy at the same time. We examined existing IRT based deep KT models and found that none of them achieves this objective. Most existing IRT-based models either learn overall student abilities or question-level student abilities. Overall student abilities are too summative, and it is hard to tell the weak areas of students from a single value. Question-level abilities are too fine-grained. When there are a large number of unique questions per skill, they can cause information overload for teachers. In this paper, we propose an IRT-based deep KT model called SKKT-IRT to learn skill-level student abilities which provide just the right amount of information for teachers to understand students' knowledge states. Our model consists of an LSTM layer to learn student historical states, a student ability network for learning skill-level student abilities, a question difficulty network for learning question difficulties and a question discrimination network for learning question discrimination. It also learns question-skill relationships as an auxiliary task so that the embedding of a skill can better capture the information of its questions. We further regularize the outputs of question difficulty network and question discrimination network for better performance. Our experimental results show that our model achieves the objective of learning skill-level student abilities with SOTA accuracy. It is also very efficient and produces consistent outputs to be easily used for downstream tasks like adaptive learning and personalized recommendations.

Primary Area: applications to neuroscience & cognitive science

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 10306