NF-ICP: Neural Field ICP for Robust 3D Human Registration

Riccardo Marin, Enric Corona, Gerard Pons-Moll

Keywords: 3D Non-rigid Registration; 3D Shape matching; Neural Fields; Virtual Humans

TL;DR: We propose: a) the first self-supervised refinement for registration with Nerual Fields; b) a novel variant of the LVD approach; c) a SOTA registration pipeline for humans that obtain SOTA in many different challenges.

Abstract: Aligning a template to 3D human point clouds is a long-standing problem crucial for tasks like animation, reconstruction, and most supervised learning pipelines. Recent data-driven methods leverage predicted surface correspondences; however, they are not robust to varied poses or distributions. In contrast, industrial solutions often rely on expensive manual annotations or multi-view capture systems. In this work, we present NF-ICP, a method that, for the first time, generalizes well on a large set of challenges, including complex poses, clothed humans, and noisy scans. Leveraging the large MoCap dataset AMASS, we learn a neural field model to predict the direction towards the localized SMPL vertices on the target surface. Such neural field leads to a reasonable initialization, but the resulting template often does not overlap with the target surface. NF-ICP exploits a classical Iterative Closest Point objective adapted to our model to quickly fine-tune the model, resulting in a significantly improved template to target surface overlap. NF-ICP constitutes a simple and computationally efficient registration method that significantly improves over public benchmarks and solidly surpasses the state of the art. We will release code and checkpoints in \url{link}.

Primary Area: representation learning for computer vision, audio, language, and other modalities

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 1726