Dual-Branch Network with Online Knowledge Distillation for 3D Hand Pose Estimation

Yingqi He, Jinghua Li, Dehui Kong, Baocai Yin

Abstract: 3D hand pose estimation aims to infer the position information of hand joints from a single image, which is widely applied to virtual reality, natural human-computer interaction, autonomous driving and various other fields. Some methods treat all joints as a whole to estimate 3D hand pose and others divide the hand joints into different groups to regress the parts of hand. However, these methods don’t leverage the complementarity between global and local part poses. To address this issue, this paper proposes a novel dual-branch framework aiming at comprehensively capturing information about hand poses. One branch focuses on capturing the overall posture information of the hand, while the other one concentrates on detailed description of the palm joints and finger joints. This global-local dual-branch structure addresses the limitation of previous methods and can handle more complex gestures. To further enhance the association between the dual branches, this paper proposes a novel online knowledge distillation strategy. The proposed strategy integrates the prediction results of both branches to construct a teacher network and then transfers the knowledge learned by the teacher network back to two student networks. Therefore, this method can better learn knowledge that contains both global and local features. A series of experiments are conducted on publicly available STB and RHD datasets, and the experimental results validates the effectiveness of the proposed 3D hand pose estimation method.