CGT: Clustered Graph Transformer for Urban Spatio-temporal Prediction

Xu Geng, Lingyu Zhang, Shulin Li, Yuanbo Zhang, Lulu Zhang, Leye Wang, Qiang Yang, Hongtu Zhu, Jieping Ye

Keywords: Unsmooth spatiotemporal forecasting, Clustered graph neural network, Graph-Transformer, Urban computing

TL;DR: We developed CGT (clustered graph-transformer) for handling the spatial and temporal unsmoothness, which greatly improve the model capability and lift the spatiotemporal prediction performance.

Abstract: Deep learning based approaches have been widely used in various urban spatio-temporal forecasting problems, but most of them fail to account for the unsmoothness issue of urban data in their architecture design, which significantly deteriorates their prediction performance. The aim of this paper is to develop a novel clustered graph transformer framework that integrates both graph attention network and transformer under an encoder-decoder architecture to address such unsmoothness issue. Specifically, we propose two novel structural components to refine the architectures of those existing deep learning models. In spatial domain, we propose a gradient-based clustering method to distribute different feature extractors to regions in different contexts. In temporal domain, we propose to use multi-view position encoding to address the periodicity and closeness of urban time series data. Experiments on real datasets obtained from a ride-hailing business show that our method can achieve 10\%-25\% improvement than many state-of-the-art baselines.

Code: https://github.com/CGT-ICLR2020/CGT-ICLR2020