Evaluating Pose Forecasting for Compensating Network Latency in Full Body Movements

Jan Bohnerth, Janis Sprenger, Selvakumar Panneer, Björn Browatzki, Anindita Ghosh, Philipp Slusallek

Abstract: Network latency presents a critical challenge for interactive multi-user virtual reality (VR) applications, particularly when transmitting the users’ full-body motion data to remote collaborators. Such latency can significantly degrade the quality of interaction, leading to a less immersive and less effective user experience. Pose forecasting, which involves predicting future motion sequences based on observed past motions, offers a potential solution. However, it has not yet been widely adopted for compensating latency for full-body motion transmission. In this work, we extend and evaluate two state-of-the-art neural network architectures for predicting future animation sequences, enabling client-side compensation of network latency. To support our methods, we construct a custom motion capture dataset with VR-specific full-body movements, along with an animation-specific data encoding that integrates both joint position and rotation data. Our experimental results show that predictive latency compensation can reduce the positional error of the reconstructed motion by a factor of up to 2.5 and is robust to noisy network connections containing jitter and packet loss. Despite these promising results, we identify several open challenges and outline directions for future work necessary for successful deployment of predictive compensation in real-world VR systems.

External IDs:dblp:conf/eurovr/BohnerthSPBGS25