Textual Supervision Enhances Geospatial Representations in Vision-Language Models

ICLR 2026 Conference Submission21204 Authors

Keywords: Mechanistic Interpretability, Vision-language models, Probing, Interpretability

TL;DR: Models pre-trained with text and images show stronger geospatial representations in their latent space.

Abstract: Geospatial understanding is a critical yet underexplored dimension in the development of machine learning systems for tasks such as image geolocation and spatial reasoning. In this work, we analyze the geospatial representations acquired by three model families: vision-only architectures (e.g., ViT), vision-language models (e.g., CLIP), and large-scale multimodal foundation models (e.g., LLaVA, Qwen, and Gemma). By evaluating across image clusters, including people, landmarks, and everyday objects, grouped based on the degree of localizability, we reveal systematic gaps in spatial accuracy and show that textual supervision enhances fine-grained geospatial representations. Our findings suggest the role of language as an effective complementary modality for encoding spatial context and multimodal learning as a key direction for advancing geospatial AI.

Primary Area: interpretability and explainable AI

Submission Number: 21204