Diverse Yet Consistent: Context-Guided Diffusion with Energy-Based Joint Refinement for Multi-Agent Motion Prediction

Lei Chu, Yuhuan Zhao

Keywords: Conext Guided Generation; Energy-Based Model; Multi-Agent Motion Prediction

Abstract: Deep generative models have become a promising approach for human motion prediction due to their ability to capture multimodal distributions and represent diverse human behaviors. However, generating predictions that are both diverse and jointly consistent among interacting agents remains challenging. In addition, most existing approaches are primarily evaluated using single-agent (marginal) metrics, which fail to fully reflect the joint dynamics of multi-agent interactions. We propose a diffusion-based framework that improves multi-agent motion prediction by leveraging rich contextual information from historical trajectories. This information is incorporated through a guidance mechanism to enhance the diversity and expressiveness of predicted motions. To further enforce interaction consistency, we introduce an energy-based formulation that refines the joint trajectory distribution while preserving the plausibility of individual trajectories. Extensive experiments on four benchmark datasets demonstrate that our approach consistently outperforms existing methods. Notably, our approach substantially improves both marginal (ADE/FDE) and joint (JADE/JFDE) metrics on ETH/UCY over strong marginal baselines. Compared with prior joint prediction methods, it delivers significant gains in marginal metrics while maintaining competitive joint performance.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 5