Capturing Implicit Spatial Cues for Monocular 3d Hand Reconstruction

Qi Wu, Joya Chen, Xu Zhou, Zhiming Yao, Xianjun Yang

Abstract: With the development of the parameterized hand model (e.g. MANO), it is possible to reconstruct the 3D hand mesh from a single 2D hand image by learning a few hand model parameters, rather than estimating hundreds of vertices on the mesh. However, it is highly non-linear to learn these parameters from the 2D hand image, as there is no explicit spatial correspondence between these parameters and image pixels. In this paper, we successfully leverage the graph convolutional network (GCN) to capture implicit spatial cues for fitting the well-known MANO hand model, thus greatly improving the performance of monocular 3D hand reconstruction. Our proposed MANO-GCN establishes the spatial hand mesh and hand joints graph for learning MANO parameters, with node features propagated along edges to utilize the spatial information. Among all monocular 3D hand reconstruction methods with MANO hand model, MANO-GCN achieves state-of-the-art accuracy on public FreiHAND and HO-3D benchmarks, without any bells and whistles. Code is available at https://github.com/ChenJoya/manogcn.