When Prompt Meets Frequency Learning for Efficient Image Restoration
Keywords: Image restoration, prompt learning, frequency learning, all-in-one
Abstract: Image restoration, as a longstanding task, aims to recover the missing details and remove degradations from a corrupted observation. Inspired by the success of prompt learning in natural language processing, many prompt-based approaches have been developed for various image restoration tasks. However, these algorithms mostly operate in the spatial domain. As frequency learning plays an important role in image restoration by reducing the spectra discrepancy between degraded/sharp image pairs, this study explores the potential of frequency prompts for efficient image restoration by proposing a plug-and-play mechanism, which mainly comprises a prompt generation module and a prompt integration module. Specifically, the former encodes different frequency information by aggregating the pre-defined learnable parameters under the guidance of implicitly decomposed spectra of input features. Subsequently, to dynamically guide reconstruction, the learned prompts are embedded into the spectra of features via dual-dimensional attention for effective frequency learning. To demonstrate the effectiveness of our mechanism, we conduct experiments on general and all-in-one image restoration tasks. By incorporating it into a CNN-based backbone, the model achieves state-of-the-art performance on 15 benchmark datasets for five representative image restoration tasks. Furthermore, equipped with our mechanism, a pure Transformer network performs favorably against state-of-the-art algorithms under two all-in-one settings.
Primary Area: applications to computer vision, audio, language, and other modalities
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 13790
Loading