STM-GAIL: Spatial-Temporal Meta-GAIL for Learning Diverse Human Driving Strategies

Yingxue Zhang, Yanhua Li, Xun Zhou, Ziming Zhang, Jun Luo

Abstract: With large amounts of human-generated spatial-temporalurban data (e.g., GPS trajectories of vehicles, passengers’trip data on buses and trains,etc.), human urban strategyanalysis has become an important problem in many urbanscenarios. This problem is hard to solve due to two ma-jor challenges: (1) data scarcity (i.e., each human agentcan only provide limited observations) and (2) data hetero-geneity (i.e., having mixed observations from many differenthuman agents). Most of the existing works on this prob-lem usually require a large amount of historical observationsaiming to correctly infer a human agent’s urban strategy andthus fail to properly address both challenges at the sametime. To solve the human urban strategy analysis prob-lem in case of data scarcity and data heterogeneity, we de-sign a novel learning paradigm — Spatial-Temporal Meta-GAIL(STM-GAIL), which can successfully learn diverse hu-man urban strategies from heterogeneous human-generatedspatial-temporal urban data. STM-GAIL models the hu-man decision processes as variable length Markov decisionprocesses (VLMDPs) and incorporates the surrounding spa-tial feature patterns (e.g., traffic volume patterns,etc.) intostates to better capture the spatial-temporal dependencies ofhuman decisions. Besides, STM-GAIL learns diverse humanurban strategies from the meta-learning perspective, andcan distinguish various human urban strategies by addingan inference network on top of the standard GAIL. STM-GAIL can be quickly adapted to a new human expert’s ur-ban strategy with a single trajectory. Extensive experimentson real-world human-generated spatial-temporal dataset areperformed.