Information Bottleneck-guided MLPs for Robust Spatial-temporal Forecasting

Min Chen, Guansong Pang, Wenjun Wang, Cheng Yan

TL;DR: Derive a novel and general information bottleneck based principle, along with its instantiation for robust spatial-temporal forecasting under dual noise effect in MLP networks.

Abstract: Spatial-temporal forecasting (STF) plays a pivotal role in urban planning and computing. Spatial-Temporal Graph Neural Networks (STGNNs) excel at modeling spatial-temporal dynamics, thus being robust against noise perturbations. However, they often suffer from relatively poor computational efficiency. Simplifying the architectures can improve efficiency but also weakens robustness with respect to noise interference. In this study, we investigate the problem: *can simple neural networks such as Multi-Layer Perceptrons (MLPs) achieve robust spatial-temporal forecasting while remaining efficient?* To this end, we first reveal the *dual noise effect* in spatial-temporal data and propose a theoretically grounded principle termed *Robust Spatial-Temporal Information Bottleneck* (RSTIB), which holds strong potential for improving model robustness. We then design an implementation named *RSTIB-MLP*, together with a new training regime incorporating a knowledge distillation module, to enhance the robustness of MLPs for STF while maintaining their efficiency. Comprehensive experiments demonstrate that *RSTIB-MLP* achieves an excellent trade-off between robustness and efficiency, outperforming state-of-the-art STGNNs and MLP-based models. Our code is publicly available at: [https://github.com/mchen644/RSTIB](https://github.com/mchen644/RSTIB).

Lay Summary: Forecasting how urban environments change over space and time helps city planners make better decisions, such as managing traffic or predicting weather. Currently, advanced forecasting methods known as Spatial-Temporal Graph Neural Networks (STGNNs) can handle the complexity of such tasks effectively, especially when data contains noise. However, these methods tend to be slow and computationally expensive. Simpler models like Multi-Layer Perceptrons (MLPs) are faster but typically struggle when data is noisy. In this study, we aim to investigate: can simpler forecasting models like MLPs become both efficient and robust against noisy data? We first reveal the _dual noise effect_ which characterizes how noise harms the forecasting model. Based on this understanding, we develop a new principle termed the Robust Spatial-Temporal Information Bottleneck (RSTIB). We then design an MLP-based model using RSTIB principle and special training regime. Our experiments show this new model successfully achieve a better balance between efficiency and robustness.

Application-Driven Machine Learning: This submission is on Application-Driven Machine Learning.

Link To Code: https://github.com/mchen644/RSTIB

Primary Area: Social Aspects->Robustness

Keywords: Robust spatial-temporal forecasting, Multi-Layer Perceptron, Information bottleneck

Submission Number: 8792