Zero-shot Depth Completion via Test-time Alignment with Affine-invariant Depth Prior

Lee Hyoseok, Kyeong Seon Kim, Kwon Byung-Ki, Tae-Hyun Oh

Abstract:

Depth completion, predicting dense depth maps from sparse
depth measurements, is an ill-posed problem requiring prior
knowledge. Although recent methods adopt learning-based
approaches to implicitly learn prior, the prior primarily fits
in-domain data but does not generalize to out-of-domain scenarios. Thus, domain-generalizable depth completion remains
a challenge. To address this, we propose a zero-shot depth completion method. Our method is composed of an affine-invariant
depth diffusion model, test-time alignment, and outlier filtering. We use pre-trained depth diffusion model as depth prior
knowledge, which implicitly understands how to fill in depth
for any scene. To predict dense depth that aligns with the given
sparse depth, we propose a test-time alignment method that
incorporates optimization loops to enforce the measurements
as hard constraints. We also propose a prior-based outlier
filtering to ensure reliable guidance within the optimization
loop. Our zero-shot depth completion method demonstrates
generalization across various domain datasets, achieving an
average performance improvement of 10.8% over previous
state-of-the-art methods and improving spatial understanding
by sharpening details.