3DGS IS A VERSATILE REGULATOR: MODULATING UNIVERSAL METRIC-DEPTH REPRESENTATION VIA ANCHOR-BASED GAUSSIAN-SPLATTED MULTIPLICATION

Ruikai Xu, Zhiwei Zhang, Weijian Zhang, Tianyu Jin, Qizhao, Zhizhong Zhang, Jingyu Gong, Yuan Xie, Lizhuang Ma

Keywords: Metric Depth Representation, Sparse Anchor, 3D Gaussian Splatting

Abstract: Recent advances in zero-shot affine-invariant depth estimation have achieved remarkable progress. However, extending relative depth to metric depth remains challenging due to the absence of reliable metric-scale guidance within existing depth foundation models. Building on this, we introduce a novel depth estimation paradigm—anchor–multiplier factorization—as an alternative to conventional approaches such as direct depth regression, depth completion, or feature-fusion methods. Our key insight is that sparse point anchors supply indispensable metric-scale cues, while relative-scale geometric structure can be stably regulated via Gaussian-splatted multiplication conditioned on image semantics. Accordingly, we implement GSD---an anchor-based Gaussian Splatting Depth Regulator for universal metric-depth restoration. We also propose the first theoretical analysis showing how anchor–multiplier factorization mitigates training divergence, and thereby improves metric restoration accuracy. Extensive experiments across diverse datasets demonstrate substantial accuracy gains over state-of-the-art baselines, highlighting the benefits of treating 3DGS not merely as a renderer, but as a versatile regulator for visual representation learning.

Primary Area: applications to computer vision, audio, language, and other modalities

Submission Number: 4129