LAGUNA: LAnguage Guided UNsupervised Adaptation with structured spaces

Anxhelo Diko, Antonino Furnari, Luigi Cinque, Giovanni Maria Farinella

Keywords: Unsupervised domain adaptation, Representation learning, language-guided adaptation

TL;DR: This work introduces LAGUNA, a novel unsupervised domain adaptation approach that aligns semantically equivalent domains using relative coordinate representation learning, guided by reference structures derived from language embeddings.

Abstract:

Unsupervised domain adaptation remains a critical challenge in enabling knowledge transfer of models across domains. Existing methods struggle to balance the need for domain-invariant representations with preserving domain-specific features. This is often caused by alignment approaches that impose the projection of different domain samples close to each other in latent spaces, despite drastic differences. We introduce LAnguage GUided domaiN Adaptation with structured spaces, a novel approach that shifts the focus from aligning representations in absolute coordinates to aligning the relative positioning of equivalent concepts in latent spaces. LAGUNA defines a domain-agnostic structure on the geometric relationships between class labels in the language space and guides the adaptation, ensuring that the organization of samples in visual space reflects reference inter-class relationships while preserving domain-specific characteristics.
Remarkably, LAGUNA surpasses previous work in 18 different adaptation scenarios across four diverse image and video datasets with average accuracy improvements of up to +3.32% on DomainNet, +5.75% on GeoPlaces, +4.77% on GeoImnet, and +1.94% mean class accuracy improvement on EgoExo4D. Finally, we also show empirically that LAGUNA competes with MLLMs 100x its size in complex adaptation scenarios.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Submission Number: 3306