FlexIR: Towards Flexible and Manipulable Image Restoration

Zhengwei Yin, Guixu Lin, Mengshun Hu, Hao Zhang, Yinqiang Zheng

Abstract: The domain of image restoration encompasses a wide array of highly effective models (e.g., SwinIR, CODE, DnCNN), each exhibiting distinct advantages in either efficiency or performance. Selecting and deploying these models necessitate careful consideration of resource limitations. While some studies have explored dynamic restoration through the integration of an auxiliary network within a unified framework, these approaches often fall short in practical applications due to the complexities involved in training, retraining, and hyperparameter adjustment, as well as limitations as being totally controlled by auxiliary network and biased by training data. To address these challenges, we introduce FlexIR: a flexible and manipulable framework for image restoration. FlexIR is distinguished by three components: a meticulously designed hierarchical branch network enabling dynamic output, an innovative progressive self-distillation process, and a channel-wise evaluation method to enhance knowledge distillation efficiency. Additionally, we propose two novel inference methodologies to fully leverage FlexIR, catering to diverse user needs and deployment contexts. Through this framework, FlexIR achieves unparalleled performance across all branches, allowing users to navigate the trade-offs between quality, cost, and efficiency during the inference phase. Crucially, FlexIR employs a dynamic mechanism powered by a non-learning metric independent of training data, ensuring that FlexIR is entirely under the direct control of the user. Comprehensive experimental evaluations validate FlexIR’s flexibility, manipulability, and cost-effectiveness, showcasing its potential for straightforward adjustments and quick adaptations across a range of scenarios. Codes will be available at [URL].

Primary Subject Area: [Experience] Multimedia Applications

Secondary Subject Area: [Systems] Systems and Middleware

Relevance To Conference:

Image restoration is a cornerstone of low-level vision research and plays a crucial role in various multimedia applications. Annually, significant advancements in this field are showcased at the ACM MM conference. Traditional image restoration models, such as SwinIR, CODE, and DnCNN, have demonstrated distinct advantages in terms of efficiency or performance. However, they require careful consideration of resource limitations during deployment. Existing dynamic restoration models often fall short in practical applications due to the complexities associated with using auxiliary networks and joint training.
In this work, we introduce FlexIR: a flexible and manipulable framework designed for image restoration. FlexIR distinguishes itself by its emphasis on flexibility, manipulability, and cost-effectiveness, making it a versatile tool for restoration tasks in multimedia and multimodal processing. Unlike existing solutions, FlexIR can dynamically adapt to user needs and deployment contexts without the complexities found in previous work. This capability marks a significant contribution to the field and could pave the way for more adaptive and user-controlled image restoration technologies.

Submission Number: 505