Controlling vision-language model for enhancing image restoration

Mingwen Shao, Weihan Liu, Qiwang Li, Lingzhuang Meng, Yecong Wan

Abstract: Highlights•We propose a two-stage training strategy integrating CLIP’s robust priors for better adaptability.•We introduce a Prior Adapter (DTA & DFN) to reduce CLIP prior disturbances.•We develop a prompt learning framework using CLIP’s alignment for precise restoration.•Our method shows superior adaptability across diverse degradation types in experiments.