Towards Multimodal Spatio-Temporal Transformer-based Models for Traffic Congestion Prediction

Huy Quang Ung, Yutaro Mishima, Hao Niu, Shinya Wada

Abstract: Robust predictions of traffic congestion play a crucial role in intelligent transportation systems. Recently, multimodal data have been applied to improve the performance of traffic congestion prediction models, such as rainfall, social network posts, incident reports, etc. In this paper, we attempt to use both dynamic people-flow and rainfall data, along with a transformer-based prediction model for traffic congestion prediction. We experiment an early fusion method for combining the multimodal data. Our experiments are conducted using two private datasets containing congestion and people-flow data, alongside a corresponding public dataset that provides rainfall data. The results indicate that incorporating the people-flow data into our prediction model enhances its performance.