Deep Models Under Domain Shift: A Sketch-Based Study

Ritika Jain, Akshay Agarwal, Atul Kumar

Keywords: Computer Vision, Deep Learning, Model Robustness, Evaluation, Data Augmentation, Model Analysis, Optimization

Abstract: Deep learning models often struggle when applied to data that differs from what they were trained on, particularly in scenarios where visual cues like color and texture are missing. Sketch-based recognition is a prime example, where models trained on natural images perform poorly due to the lack of these features. In this study, we systematically evaluate how various image classification architectures respond to such domain shifts. We introduce and assess practical techniques, including adding synthetic color, using gray-scale data, leveraging frequency-based transformations, and analyzing image complexity. Our results reveal that model performance under domain shift depends heavily on architecture, with larger models able to adapt better to colorized inputs while simpler models depend more on shape information. Training strategies such as mixed gray-scale augmentation and low-pass filtering significantly improve robustness without sacrificing general performance. These findings highlight the limitations of current models in handling distribution changes and suggest avenues for improving their reliability across diverse visual domains. We benchmark the performance sensitivity of widely used architectures—including \textbf{AlexNet, VGG19, ResNet-50, EfficientNet-B7, and ViT-B16}—on \textbf{ImageNet-sketch dataset}, and propose cost-effective strategies to improve model robustness.

Submission Number: 227