Screen-space Regularization on Differentiable Rasterization

Kunyao Chen, Cheolhong An, Truong Q. Nguyen

Abstract: Rasterization bridges 3D meshes of a scene and 2D visual appearance on different viewpoints. It plays a vital role in vision and graphics area. Many researches focus on designing a differentiable rasterization and make it compatible with current learning-based frameworks. Although some global-gradient methods achieve promising results, they still ignore one substantial issue existing in most of the situations that the series of 2D silhouettes may not precisely represent the underlying 3D object. To directly tackle this problem, we propose a screen-space regularization method. Unlike the common geometric regularization, our method targets the unbalanced deformation due to the limited viewpoints. By applying the regularization to both multi-view deformation and single-view reconstruction tasks, the proposed method can significantly enhance the visual appearance for the results of a local-gradient differentiable rasterizer, i.e. reducing the visual hull redundancy. Comparing to the state-of-the-art global-gradient method, the proposed method achieves better numerical results with much lower complexity.