Item response function variability: A strategy for model comparison research in IRT

Xing Chen, Leah Feuerstahler

DOI: 10.64028/ukmo267187

Keywords: item response theory, goodness-of-fit, model complexity, model comparison, simulation studies

TL;DR: This paper investigates influence of models on the data generating process in simulation study, and proposed a method to generate similar items from different models.

Abstract: Model complexity is defined as the ability of a model to fit various data patterns. The influence of model complexity on item response theory (IRT) models historically has been explored through simulation studies. However, simulation studies that generate items from different IRT models may confound the inherent differences in the models with artificial differences induced by the choice of data-generating model parameters. In this paper, we introduce the concept of item response function (IRF) variability which can be leveraged to make items from different IRT models as similar as possible in simulation research. Specifically, we illustrate how the distribution of IRF maximum slopes and locations can be harmonized across models. We illustrate this concept with three unidimensional models: the two-parameter model (2PL) model, the negative log-log (NLL) model, and the logistic positive exponent (LPE) model. Illustrative results are presented, followed by an overall discussion.

Submission Number: 13