TSG-net: a residual-based informing network for 3D Gaze estimation

Jianjun Li, Jialuo Fei, Shichao Cheng, Zheng Tang, Guobao Hui

Abstract: The appearance-based method for gaze estimation has great potential to work well under various conditions, but current learning-based methods ignore inferior eye images in datasets caused by poor eye region locating, occlusions and abnormal head poses. These images badly impact the accuracy of estimation. In the study, inspired by binocular vision characteristics, we propose two cooperative sub-networks, True Gaze Consistency Network (TG-Net) and Single Gaze Inconsistency Network(SG-Net) which composes TSG-Net. TG-Net and SG-Net cooperate through a residual paradigm and Informing module. More specially, TG-Net explicitly extracts the consistency of paired eyes and weights high-level features from two paired-eye images utilizing SE-Block and an artificial gaze direction, named True Gaze. SG-Net outputs residual momentums based on True Gaze for better estimation of paired eyes. Experimental results on three benchmark datasets demonstrate that the proposed method performs competitively against the existed representative CNN-Based methods. TSG-Net improves on the state-of-the-art by 22% on MPIIGaze and shows more advantages in additional analysis.