Domain-Aware Fine-Tuning of Foundation Models

Uğur Ali Kaplan, Yumeng Li, Margret Keuper, Anna Khoreva, Dan Zhang

Keywords: Foundation Models, Domain Generalization, Semantic Segmentation, Finetuning

TL;DR: We investigated different foundation models for domain generalizable semantic segmentation and propose a domain-aware fine-tuning strategy to further enhance the model's generalization capability.

Abstract: Foundation models (FMs) have revolutionized computer vision, enabling effective learning across different domains. However, their performance under domain shift is yet underexplored. This paper investigates the zero-shot domain adaptation potential of FMs by comparing different backbone architectures and introducing novel domain-aware components that leverage domain related textual embeddings. We propose domain adaptive normalization, termed as Domino, which explicitly leverages domain embeddings during fine-tuning, thus making the model domain aware.Ultimately, Domino enables more robust computer vision models that can adapt effectively to various unseen domains.

Submission Number: 59