R5DGS: Semantic-Aware 4D Gaussian Splatting with Rigid Body Constraints for Efficient Dynamic Scene Reconstruction

Denis Gridusov, Maxim Popov, Sergey Kolyubin

Keywords: Dynamic 3D Reconstruction, 4D Gaussian Splatting, Semantic Scene Understanding, Rigid-Body Dynamics, Scene Reconstruction, Scene Representation

Abstract:

Reconstructing and predicting dynamic 3D scenes
from multi-view videos is a foundational task for robotics,
AR/VR, and digital twins. Recent physics-informed Gaussian
Splatting methods achieve impressive future frame extrapola-
tion but lack semantic awareness and suffer from large com-
putational overhead. We introduce R5DGS, a framework that
augments a physics-driven 4D Gaussian representation with
compact Identity Encoding vectors, enabling precise Gaussian-
to-object association. By constructing an offline CLIP-based
object lookup table, we support open-vocabulary text prompt-
ing to retrieve and render object-specific Gaussians across
arbitrary timestamps and viewpoints. Furthermore, we propose
a rigid-body inference constraint that predicts and integrates
physical dynamics exclusively for object centroids, propagating
motion to associated Gaussians via relative transformations.
This optimization yields a 11 FPS speedup during extrapolation
without compromising trajectories plausibility.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 47