VQA-Driven Facet-Level Texture Segmentation in 3D Surfaces

Iyyakutti Iyappan Ganapathi, Sajid Javed, Syed Sadaf Ali, Mohamad Alansari, Said Boumaraf, Naoufel Werghi

Abstract: This paper presents an approach for 3D texture segmentation by leveraging visual question answering (VQA) techniques. Our method focuses on classifying each facet of a 3D surface to distinguish between texture and non-texture regions. We propose a strategy where the geometric properties of neighboring facets are utilized to generate a 2D feature representation for each facet. These 2D images are then inputted into a VQA system, specifically designed to answer queries such as Does this facet belong to a texture?. The system provides binary responses, facilitating the segmentation of texture and non-texture regions. We evaluated our method on three different datasets, SHREC’17, SHREC’18 and KU 3DTexture, demonstrating performance compared to existing approaches. Our results highlight the effectiveness of integrating VQA techniques with 3D surface analysis, offering a robust solution for texture segmentation in 3D models.

External IDs:doi:10.1007/978-3-031-92808-6_17