Understanding and Simplifying Architecture Search in Spatio-Temporal Graph Neural Networks

Zhen Xu, quanming yao, Yong Li, Qiang Yang

Event Certifications: automl.cc/AutoML/2023/Journal_Track

Abstract: Compiling together spatial and temporal modules via a unified framework, Spatio-Temporal Graph Neural Networks (STGNNs) have been popularly used in the multivariate spatio-temporal forecasting task, e.g. traffic prediction. After the numerous propositions of manually designed architectures, researchers show interest in the Neural Architecture Search (NAS) of STGNNs. Existing methods suffer from two issues: (1) hyperparameters like learning rate, channel size cannot be integrated into the NAS framework, which makes the model evaluation less accurate, potentially misleading the architecture search (2) the current search space, which basically mimics Darts-like methods, is too large for the search algorithm to find a sufficiently good candidate. In this work, we deal with both issues at the same time. We first re-examine the importance and transferability of the training hyperparameters to ensure a fair and fast comparison. Next, we set up a framework that disentangles architecture design into three disjoint angles according to how spatio-temporal representations flow and transform in architectures, which allows us to understand the behavior of architectures from a distributional perspective. This way, we can obtain good guidelines to reduce the STGNN search space and find state-of-the-art architectures by simple random search. As an illustrative example, we combine these principles with random search which already significantly outperforms both state-of-the-art hand-designed models and recently automatically searched ones.

Submission Length: Regular submission (no more than 12 pages of main content)

Changes Since Last Submission:

- All text changes are highlighted in blue color.
- Fig 6, Fig 7, Table 3, Table 4 are revised.

Code: https://github.com/LARS-research/SimpleSTG

Assigned Action Editor: ~Charles_Xu1

License: Creative Commons Attribution 4.0 International (CC BY 4.0)

Submission Number: 557