Uncertainty in Predictive Process Monitoring

Pietro Portolani, Alessandro Brusaferri, Andrea Ballarino, Matteo Matteucci

Abstract: Predictive process monitoring focuses on forecasting future events in an ongoing sequence of logs coming from a process. Process owners will then use predictions to make decisions or give users information about the advancement of their case. Estimation of uncertainty is a critical uncovered topic in the task of next activity prediction. In this work, we study uncertainty estimation and model calibration in the case of the next activity prediction exploiting deep learning techniques, suggesting a framework able to take advantage of this information in a real scenario. We used an attention-based neural network as a forecasting algorithm and estimated its uncertainty through an ensemble of networks and Monte Carlo dropout. Two public datasets with different complexities and widely adopted in the Process Mining community are used. We investigate the quality of the models’ calibration on the considered datasets and how prediction accuracy is related to uncertainty. We also analyzed test cases in which the model is overconfident and showed that they could be related to uncommon data or sequences not contained in the training set.