Predictive Modeling of Submissions and Learning Outcomes in Online Judge Systems

Md. Shahajada Mia, Yutaka Watanobe, Md. Mostafizer Rahman, Md. Faizul Ibne Amin, Muepu Mukendi Daniel

Abstract: Forecasting submissions and learning outcomes patterns in online judge (OJ) systems is important for managing workloads and supporting adaptive learning environments. However, accurate forecasting is challenging due to diverse data patterns and skewed feature distributions. This study proposes a feature shape-aware (FSA) framework guided by distributional diagnostics and integrated with forecasting models such as Naïve, seasonal autoregressive integrated moving average (SARIMA), and decision tree (DT). The framework is applied to submission data from two courses in Aizu OJ (AOJ), such as Algorithm and Data Structures I (ALDS1) and Dataset and Queries (DSL), which exhibit contrasting behaviors: ALDS1 shows higher submissions but lower acceptance rates, while DSL demonstrates fewer submissions with relatively higher acceptance. Empirical results show that the proposed approach improves forecasting accuracy compared to raw data across models, particularly for skewed features. In particular, DT achieved the lowest error metrics, such as mean absolute error (MAE), root mean square error (RMSE), and symmetric mean absolute percentage error (sMAPE) for skewed features, e.g., MAE of 153.610 vs 163.909 in ALDS1 and 17.655 vs 18.337 in DSL for submissions on FSA vs raw data. For symmetric features such as acceptance rates, both FSA and raw data show nearly identical results across models. Moreover, the proposed framework is scalable, effectively enhances forecasting of learning activity patterns, and provides practical insights for OJ-based learning platforms.

External IDs:dblp:conf/mcsoc/MiaWRAD25