Visibility-Aware Pixelwise View Selection for Multi-View Stereo Matching

Zhentao Huang, Yukun Shi, Minglun Gong

Abstract: The performance of PatchMatch-based multi-view stereo algorithms is greatly influenced by the chosen source views used for matching cost computation. Existing methods usually detect occlusions in a rather ad-hoc way, which can negatively impact the computation. In contrast, our paper introduces an innovative approach that deliberately models view visibility. We present a novel visibility-guided pixelwise view selection scheme that progressively refines the set of source views for each pixel in the reference view using visibility information from validated solutions. Furthermore, the Artificial Multi-Bee Colony (AMBC) algorithm is leveraged to parallelly search optimal solutions for different pixels. To ensure smoothness of neighboring pixels and better manage textureless areas, rewards are assigned to solutions that come from validated sources. Our method, validated through experiments on two datasets, improves detail recovery in occluded and low-textured regions, demonstrating noteworthy performance on demanding scenes. Our implementation is available at https://github.com/Ricky-S/Visibility-Aware-Pixelwise-View-Selection.