Test-Time Training for Image Superresolution

Zeeshan Patel, Yossi Gandelsman

Abstract: We introduce a novel approach for Test-Time Training (TTT) in Single Image Super-Resolution (SR), which enhances the performance of deep learning SR models in an online setting without requiring external data for pretraining. Our TTT for SR method consists of two primary components: self-supervised fine-tuning and model patching. The baseline SwinIR transformer model is fine-tuned for every new test image encountered, using a self-supervised learning task that generates data solely from the test image. By employing an iterative updating strategy, our model adapts to new test examples as they arrive, allowing knowledge from previous test examples to influence future inferences. To obtain the best results, we merge the the pretrained and fine-tuned TTT models using a linear interpolation of their learned weights. Our approach is applicable to various vision models and has the potential to advance generalization for several vision tasks.