3D Gaze Tracking for Studying Collaborative Interactions in Mixed-Reality Environments

Eduardo Davalos, Yike Zhang, Ashwin T. S., Joyce Horn Fonteles, Umesh Timalsina, Gautam Biswas

Abstract: This study presents a novel framework for 3D gaze tracking tailored for mixed-reality settings, aimed at enhancing joint attention and collaborative efforts in team-based scenarios. Conventional gaze tracking, often limited by monocular cameras and traditional eye-tracking apparatus, struggles with simultaneous data synchronization and analysis from multiple participants in group contexts. Our proposed framework leverages state-of-the-art computer vision and machine learning techniques to overcome these obstacles, enabling precise 3D gaze estimation without dependence on specialized hardware. Utilizing facial recognition and deep learning, the framework achieves real-time, tracking of gaze patterns across several individuals, addressing common depth estimation errors, and ensuring spatial and identity consistency within the dataset. This provides mechanisms for significant advances in behavior and interaction analysis in educational and professional training applications in dynamic and unstructured environments. For transparency and to promote further development, we released the code at https://github.com/edavalosanaya/3DGazeTracking_ICMIW2024.