Thermal-SAM: Adversarial Prompt-Based Unsupervised Building Segmentation in Thermal Aerial Imagery – A Case Study in Turin

Shuai Niu, Hongshan Guo, Maria Ferrara, Sebastiano Anselmo

Keywords: Compter Vision, Image Segmentation, Thermal Image Segmentation, AI for Archeticture

TL;DR: Thermal-SAM is an unsupervised segmentation method for thermal building images. It uses vision-language models and adversarial prompts to generate pseudo labels from color images, achieving over 10% accuracy improvement over existing methods.

Abstract: Thermal image building segmentation is essential for monitoring energy consumption and supporting environmental protection. Current segmentation methods are predominantly designed for RGB images, posing challenges for thermal images, especially when segmenting buildings of varied shapes from aerial views, due to their lower resolution, lack of detailed features, and channel differences. To address these challenges, we propose an unsupervised segmentation method Thermal-SAM, specifically for a new aerial thermal dataset from Turin, Italy. We enhance this method by incorporating color aerial images from the same region as an auxiliary modality to generate pseudo labels for unsupervised training. Our approach introduces an adversarial prompt-based pseudo-label generation method, utilizing several vision-language models, along with positive and negative prompts. Extensive experiments demonstrate that Thermal-SAM, surpasses state-of-the-art methods by more than 10%.

Submission Number: 18