HFGaussian: Learning Generalizable Gaussian Human with Integrated Human Features

Arnab Dey, Cheng-You Lu, Andrew I. Comport, Srinath Sridhar, Chin-Teng Lin, Jean Martinet

Abstract: Recent advancements in radiance field rendering show promising results in 3D scene representation, where Gaussian splatting-based techniques emerge as state-of-the-art due to their quality and efficiency. Gaussian splatting is widely used for various applications, including 3D human representation. However, previous 3D Gaussian splatting methods can not render RGB and human features simultaneously. Furthermore, these methods either use parametric body models as additional information or fail to provide any underlying structure, like human biomechanical features, which are essential for various applications such as medical diagnosis. In this paper, we present a novel approach called HFGaussian that can estimate novel views and human features, such as the 3D skeleton, 3D key points, and dense pose, from sparse input images simultaneously in real time at 25 FPS. The proposed method leverages the generalizable Gaussian splatting technique to represent the human subject and 3D consistent human features, enabling efficient and generalizable reconstruction. By incorporating a pose regression network and the feature splatting technique with Gaussian splatting, HFGaussian demonstrates improved capabilities over existing 3D human methods, showcasing the potential of 3D human representations with integrated biomechanics. We thoroughly evaluate our HFGaussian method against the latest state-of-the-art techniques in human Gaussian splatting methods, demonstrating its real-time, state-of-the-art performance. Project page: https://github.com/arnabdeypolimi/HFGaussian_official.

External IDs:doi:10.1109/tai.2025.3644324