Multi-view Stereo by Fusing Monocular and a Combination of Depth Representation Methods

Fanqi Yu, Xinyang Sun

Abstract: The design of plane-sweep deep MVS primarily relies on patch-similarity based matching. However, this approach becomes impractical when dealing with low-textured, similar-textured and reflective regions in the scene, resulting in inaccurate matching results. One of the methods to avoid this kind of error is incorporating semantic information in matching process. In this paper, we propose an end-to-end method that uses monocular depth estimation to add semantic information to deep MVS. Additionally, we analyze the advantages and disadvantages of two main depth representations and propose a collaborative method to alleviate their drawbacks. Finally, we introduce a novel filtering criterion named Distribution Consistency, which can effectively filter out outliers with poor probability distribution, such as uniform distribution, to further enhance the reconstruction quality.