One-Step Residual Shifting Diffusion for Image Super-Resolution via Distillation

Daniil Selikhanovych, David Li, Aleksei Leonov, Nikita Gushchin, Sergey Kushneryuk, Alexander Filippov, Evgeny Burnaev, Iaroslav Sergeevich Koshelev, Alexander Korotin

Keywords: super-resolution, diffusion models, distillation

Abstract: Diffusion models for super-resolution (SR) produce high-quality visual results but require expensive computational costs. Despite the development of several methods to accelerate diffusion-based SR models, some (e.g., SinSR) fail to produce realistic perceptual details, while others (e.g., OSEDiff) may hallucinate non-existent structures. To overcome these issues, we present **RSD**, a new distillation method for ResShift. Our method is based on training the student network to produce images such that a new fake ResShift model trained on them will coincide with the teacher model. RSD achieves single-step restoration and outperforms the teacher by a noticeable margin in various perceptual metrics (LPIPS, CLIPIQA, MUSIQ, DISTS, NIQE, MANIQA). We show that our distillation method can surpass the other distillation-based method for ResShift - SinSR - making it on par with state-of-the-art diffusion-based SR distillation methods with low computational costs in terms of perceptual quality. Compared to SR methods based on pre-trained text-to-image models, RSD produces competitive perceptual quality and requires fewer parameters, GPU memory, and training cost. We provide experimental results on various real-world and synthetic datasets, including RealSR, RealSet65, DRealSR, ImageNet, DIV2K, RealLR200 and RealLQ250.

Primary Area: generative models

Submission Number: 5861