An approach for combining multimodal fusion and neural architecture search applied to knowledge tracing

Xinyi Ding, Tao Han, Yili Fang, Eric C. Larson

Abstract: Knowledge Tracing is the process of tracking mastery level of different skills of students for a given learning domain. It is one of the key components for building adaptive learning systems and has been investigated for decades. The empirical success of deep neural networks in the past few years has encouraged researchers in the learning science community to take similar approaches. However, most existing deep learning based knowledge tracing models have the following limitations: (1) Only use the correct/incorrect response, ignoring useful information from other modalities. (2) For works do consider multimodality use simple concatenation, which might not be the best way for modality fusion. (3) Design their network architectures manually via trial and error. To solve these problems, we propose Multimodal Fusion and Neural Architecture Search (MFNAS) in this paper. The commonly used neural architecture search technique could be considered as a special case of our proposed approach when there is only one modality involved. We further propose to use a new metric called time-weighted Area Under the Curve (weighted AUC) to measure how a sequence model performs with time. Our proposed approach MFNAS allows more efficient design of Knowledge Tracing models. Besides, evaluated on two public real datasets, the discovered model is able to achieve around 12% improvement in coefficient of determination, 2% improvement in AUC and weighted AUC compared with state of the art models. What’s more, unlike most existing works, we conduct McNemar’s test on the model predictions and the results are statistically significant.