BaSICNet: Lightweight 3-D Hand Pose Estimation Network Based on Biomechanical Structure Information for Dexterous Manipulator Teleoperation

Wenrao Pang, Qing Gao, Yinan Zhao, Zhaojie Ju, Junjie Hu

Abstract: The use of hand pose for dexterous manipulator teleoperation is an attractive method to the control of the multifingered manipulators. Furthermore, the advancement of the deep learning and depth sensors has encouraged the development of the 3-D hand pose estimation. However, developing a 3-D hand pose estimation method with an accurate and real-time performance is still a difficult task in computer vision. In this article, a lightweight depth-based network named the biomechanical structure information cascade network (BaSICNet) is proposed by considering the global and local structure of human hands to improve the performance through a cascade network and a bone-constraint loss function. In addition, the BaSICNet is applied to a five-fingered dexterous manipulator platform to achieve visual hand-based teleoperation. Extensive evaluations on two public data sets show that the BaSICNet can produce accurate and fast 3-D hand poses (9.15 and 7.59-mm mean errors on NYU and MSRA data sets with 114.7 fps), and can achieve superior 3-D hand pose estimation balance of accuracy and speed when compared with state-of-the-art (SOAT) methods. Experiment results on the dexterous manipulator platform also show that the BaSICNet can be applied well for teleoperation.