Enhanced Reconstruction of Interacting Hands for Immersive Embodiment

Yu Miao, Yu Han, Yi Xiao, Yue Liu

Abstract: Interacting hands reconstruction serves as a channel for natural user engagement in virtual reality, providing realistic embodiment that markedly elevates the immersive experiences. However, accurate prediction of the spatial relations between two hands remains challenging due to the severe occlusion and homogeneous appearance of hands. This paper presents a spatial relationship refinement method for hand reconstruction, employing a module to yield the 3D relative translation between hands and a novel loss function to limit hand mesh penetration. Our method achieves state-of-the-art performance on the InterHand2.6M dataset, offering considerable potential for interacting hands reconstruction in enhancing embodiment for virtual reality.