One-Step Residual Shifting Diffusion for Image Super-Resolution via Distillation

Daniil Selikhanovych, David Li, Aleksei Leonov, Nikita Gushchin, Sergey Kushneryuk, Alexander Filippov, Evgeny Burnaev, Iaroslav Sergeevich Koshelev, Alexander Korotin

Keywords: super-resolution, diffusion models, distillation

TL;DR: We present a new distillation method for ResShift

Abstract:

Diffusion-based super-resolution (SR) models have a high visual quality but suffer from high computational cost. Existing acceleration methods for diffusion SR either miss realistic perceptual details (e.g., SinSR) or hallucinate non-existent structures (e.g., OSEDiff).
To overcome these issues, we present **RSD**, a new distillation method for ResShift. RSD trains a student model to produce images such that a new fake ResShift model trained on them matches the teacher model. RSD achieves single-step restoration and noticeably outperforms the teacher in various perceptual metrics, such as LPIPS, CLIPIQA, and MUSIQ. RSD outperforms SinSR, another ResShift distillation method, and matches the state-of-the-art diffusion SR distillation methods with limited computational costs in terms of perceptual quality. Compared to SR methods based on pre-trained text-to-image models, RSD produces competitive perceptual results and requires fewer parameters, GPU memory, and training cost. We provide results on various real-world and synthetic datasets, including RealSR, RealSet65, DRealSR, and DIV2K.

Submission Number: 90