DeepHandMesh-lite: Learning personalized hand shape using limited data and weak supervision
Primary Area: representation learning for computer vision, audio, language, and other modalities
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Keywords: Human hand mesh, Computer Vision, Computer Graphics, Deep Learning, Human-centered computing
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
TL;DR: We present a method that can be trained on an individual’s hand shape using 24 3D-scanned poses
Abstract: Being able to control the deformation of personalized, high-fidelity hand meshes in real-time contributes strongly to the feeling of presence in virtual reality. We present a method to learn an individual's hand shape based on 3D scans of the hand in different poses. For this, we rely on the data and hand shape model from the work of Moon et al. titled "DeepHandMesh" (DHM). We propose a novel algorithm to approximate hand joint pose based on joint position, and a loss function which leverages shape information contained in the silhouette. Of the 1070 high-resolution hand scans that DHM trains on in total, we choose only 24 poses representing primarily grasping scenarios. While the scans in DHM have been obtained with highly specialized equipment, our approach makes personalization of the hand mesh more feasible using limited resources. Our model is able to create subject-specific, posed meshes in real-time using joint positions as input, though there are sometimes artefacts visible in extreme poses that detract from the realism.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 3551
Loading