Neural Interaction Energy for Multi-Agent Trajectory Prediction

Kaixin Shen, Ruijie Quan, Linchao Zhu, Jun Xiao, Yi Yang

Abstract: Maintaining temporal stability is crucial in multi-agent trajectory prediction. Insufficient regularization to uphold this stability often results in fluctuations in kinematic states, leading to inconsistent predictions and the amplification of errors. In this study, we introduce a framework called Multi-Agent Trajectory prediction via neural interaction Energy (MATE). This framework assesses the interactive motion of agents by employing neural interaction energy, which captures the dynamics of interactions and illustrates their influence on the future trajectories of agents. To bolster temporal stability, we introduce two constraints: inter-agent interaction constraint and intra-agent motion constraint. These constraints work together to ensure temporal stability at both the system and agent levels, effectively mitigating prediction fluctuations inherent in multi-agent systems. Comparative evaluations against previous methods on four diverse datasets highlight the superior prediction accuracy and generalization capabilities of our model. We will release our code.

Primary Subject Area: [Content] Media Interpretation

Secondary Subject Area: [Experience] Multimedia Applications

Relevance To Conference: Multi-agent trajectory prediction, on the other hand, focuses on forecasting the future paths of multiple interacting agents, such as pedestrians, vehicles, or robots, based on observed behaviors and surroundings. By integrating multimedia data, trajectory prediction systems can leverage a more holistic view of the environment, improving accuracy and reliability. For instance, audio cues can indicate off-screen activities and high-resolution videos can provide detailed spatial-temporal data. This synergy enables systems to anticipate potential interactions and collisions better, adjust in real-time to new inputs, and make more informed decisions.

Submission Number: 1932