CHOSEN: Contrastive Hypothesis Selection for Multi-View Depth Refinement

Di Qiu, Yinda Zhang, Thabo Beeler, Vladimir Tankovich, Christian Häne, Sean Fanello, Christoph Rhemann, Sergio Orts-Escolano

Keywords: Multiview Stereo, Depth refinement, 3D reconstruction

TL;DR: A multiview depth refinement module based on contrastive learning, which iteratively samples and selects the best depth hypothesis

Abstract:

We propose CHOSEN, a simple yet flexible, robust and effective multi-view depth refinement framework.
It can provide significant improvement in depth and normal quality, and can be integrated in existing multi-view stereo pipelines with minimal modifications.
Given an initial depth estimation, CHOSEN iteratively re-samples and selects the best hypotheses.
The key to our approach is the application of contrastive learning in an appropriate solution space and a carefully designed hypothesis feature, based on which positive and negative hypotheses can be effectively distinguished.
We integrated CHOSEN in a basic multi-view stereo pipeline, and show that it can deliver impressive quality in terms of depth and normal accuracy compared to many other top deep learning based multi-view stereo pipelines.

Submission Number: 1