Garden: a real-time processing framework for continuous top-k trajectory similarity search

Zhicheng Pan, Pingfu Chao, Junhua Fang, Wei Chen, Jiajie Xu, Lei Zhao

Abstract: Continuous top-k trajectory similarity Search (CkSearch) is now commonly required in real-time large-scale trajectory analysis, enabling the distributed stream processing engines to discover various dynamic patterns. As a fundamental operator, CkSearch empowers various applications, e.g., contact tracing during an outbreak and smart transportation. Although extensive efforts have been made to improve the efficiency of non-continuous top-k search, they do not consider dynamic capability of indexing (R1) and incremental capability of computing (R2). Therefore, in this paper, we propose a generic CkSearch-oriented framework for distributed real-time trajectory stream processing on Apache Flink, termed as Garden. To answer R1, we design a sophisticated distributed dynamic spatial index called Y-index, which consists of a real-time load scheduler and a two-layer indexing structure. To answer R2, we introduce a state reusing mechanism and index-based pruning methods that significantly reduce the computational cost. Empirical studies on real-world data validate the usefulness of our proposal and prove the huge advantage of our approach over state-of-the-art solutions in the literature.