Collaborative Observation Imputation and Trajectory Prediction via Consistency Evaluation

Hao Zhou, Mingyu Fan, Xu Yang

Abstract: Pedestrian trajectory prediction is a critical task in various mobile computing applications, such as video surveillance, robot navigation, autonomous driving, and human mobility analysis. Although significant progress has been made by current methods, the challenge of observation deficiency in pedestrian trajectory prediction remains largely unaddressed. Since most existing methods focus on optimizing prediction accuracy under the assumption of complete observations, while ignoring the potential for observation deficiency caused by failures in detection or tracking algorithms. To overcome this challenge, we propose a collaborative observation imputation and trajectory prediction framework, which employs consistency evaluation to jointly perform the imputation and prediction tasks. Specifically, we first build a consistency evaluation module to align features between observed and future trajectory pairs using contrastive learning. Then, we design a trajectory imputation and prediction baseline, which adopts a parallel paradigm, to mitigate the impact of coarse imputations on trajectory prediction when performing initial imputation and prediction. Next, we introduce a consistency-guided Skip-Diffusion module, which leverages consistency evaluation between initial imputations and ground truth future trajectories to refine the initial imputations. Finally, we propose a consistency-driven Cross-Mamba module, which uses consistency evaluation between ground-truth observations and initial predictions to refine the initial predictions. Extensive experiments demonstrate the effectiveness of the proposed framework in both imputation and prediction tasks.