Immersive Telecommunication Method Based on Live-Action Video with Perspective Taking

Kai Shishido, Chun Xie, Masahiko Inoue, Itaru Kitahara

Abstract: In standard video communication, the information obtained from the video stream is limited due to the restrictions imposed by the camera’s fixed and narrow viewing angle. As a result, it can be challenging to accurately interpret the intentions, emotions, and situational context of the communication partner. To address this issue, we have developed an immersive telecommunication system that leverages multiple cameras, including a 360-degree camera, in combination with head-mounted displays (HMDs). This setup enables bidirectional image capture without being constrained by the limitations of traditional viewing angles. Users are thus able to view not only a comprehensive 360-degree representation of the partner's surrounding environment but also their perspective, offering a more holistic view of the interaction. Experiments conducted to evaluate the system’s communication effectiveness demonstrated that, compared to traditional systems, proposed approach significantly improved the visibility of body language, thereby enhancing the understanding of the partner's intentions and emotional expressions. The proposed system is anticipated to find valuable applications in areas such as remote collaborative work and remote educational activities.

External IDs:doi:10.1007/978-3-031-94150-4_17