GoMVS: Geometrically Consistent Cost Aggregation for Multi-View Stereo

Jiang Wu, Rui Li, Haofei Xu, Wenxun Zhao, Yu Zhu, Jinqiu Sun, Yanning Zhang

Abstract: Matching cost aggregation plays a fundamental role in learning-based multi-view stereo networks. However, di-rectly aggregating adjacent costs can lead to suboptimal results due to local geometric inconsistency. Related meth-ods either seek selective aggregation or improve aggregated depth in the 2D space, both are unable to handle geomet-ric inconsistency in the cost volume effectively. In this pa-per, we propose GoMVS to aggregate geometrically consis-tent costs, yielding better utilization of adjacent geometries. More specifically, we correspond and propagate adjacent costs to the reference pixel by leveraging the local geomet-ric smoothness in conjunction with surface normals. We achieve this by the geometric consistent propagation (GCP) module. It computes the correspondence from the adjacent depth hypothesis space to the reference depth space using surface normals, then uses the correspondence to propa-gate adjacent costs to the reference geometry, followed by a convolution for aggregation. Our method achieves new state-of-the-art performance on DTU, Tanks & Temple, and ETH3D datasets. Notably, our method ranks 1st on the Tanks & Temple Advanced benchmark. Code is available at https://github.com/Wuuu3511IGoMVS.