Multi-Hypothesis 3D Hand Mesh Recovering from a Single Blurry Image

Glory Rongyu CHEN

Abstract:

Recovery of 3D hand mesh from blurry hand images
is challenging due to the ambiguity. Most existing works attempt
to solve this issue by exploiting physical and temporal constraints.
However, those works ignore the fact that multiple feasible
solutions exist. In this paper, we propose a two-stage Multi-
Hypothesis Hand Mesh Recovery network, consisting of a genera-
tion and selection model. In the first stage, the generation model
explicitly extracts the temporal information with an unfolder.
Then, a multi-hypothesis Transformer generates multiple diverse
hypotheses with a lightweight hypothesis embedding set. In the
second stage, the selection model selects a subset of good-quality
hypotheses. We additionally combine the classifying and ranking
loss to better align with the target of the selection model. Exten-
sive experiments show that the proposed method produces much
more accurate results on blurry images. Source code is available
at https://github.com/RandSF/Multi Hypothesis BlurHandNet.