LayeredGS: Efficient Dynamic Scene Rendering and Point Tracking with Multi-Layer Deformable Gaussian Splatting

Jipeng Lyu, Jiahua Dong, Yu-Xiong Wang

Keywords: 3D Vision, Novel View Synthesis, Dynamic Scene Reconstruction

Abstract: Dynamic novel-view synthesis and point tracking have emerged as promising tasks. However, existing methods often struggle with efficiency and accurately capturing deformations. In this paper, we propose LayeredGS, a novel Deformation-based Dynamic Gaussian Splatting method that excels in both 3D tracking of dense scene elements and real-time dynamic scene rendering. By learning Gaussian deformations between frames, LayeredGS preserves their point-like characteristics while capturing motion. Unlike previous methods, our approach optimizes efficiency by grouping Gaussians with similar deformations using a coarse-to-fine clustering structure. Experimental results show the rapid convergence within 100 iterations per time frame on fast-moving dynamic datasets, maintaining rendering quality and tracking accuracy comparable to state-of-the-art methods using only 1/20 training iterations. Additionally, we introduce applications like Articulated Objects Segmentation, highlighting the utility of deformation information for the first time.

Primary Area: applications to computer vision, audio, language, and other modalities

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 8670