Multimodal Temporal Fusion Transformers are Good Product Demand Forecasters

Maarten Sukel, Stevan Rudinac, Marcel Worring

Abstract: Multimodal demand forecasting aims at predicting product demand utilizing visual, textual, and contextual information. This article proposes a method for such forecasting using an integrated architecture composed of convolutional, graph-based, and transformer-based networks. Since traditional forecasting methods depend on historical demand and factors like manually generated categorical information, they face challenges such as the cold start problem and handling of category dynamics. To address these challenges, our architecture allows for incorporating multimodal information, such as geographical information, product images, and textual descriptions. Experiments with the multimodal approach are performed on a real-world dataset of more than 50 million data points of article demand. The pipeline presented in this work enhances the reliability of the predictions, demonstrating the potential of leveraging multimodal information in product demand forecasting.