Learning to Compose Degradations: A Codebook of Primitives for All-in-One Image Restoration

Ye Deng, Zhihao Li, Wenli Huang, YangWu, Jiang Duan

Keywords: All-in-One Image Restoration, Codebook

Abstract:

All-in-one image restoration aims to develop a single model for diverse degradations, a challenge whose success critically hinges on the precise representation of the underlying degradation process. Existing methods simplify this challenge by mapping each degradation to a coarse-grained, monolithic representation---effectively treating them as discrete categories (e.g., "haze," "noise"). This paradigm, even in prompt-learning variants, fundamentally fails to capture the continuous and fine-grained nature of real-world corruptions, such as varying intensities, leading to suboptimal performance.
To address this, we argue that degradations are better represented as a composition of a finite set of learnable, elementary degradation primitives. We introduce DACode, a novel framework built upon a global, learnable codebook embodying these primitives. The core of DACode is a two-stage, dual cross-attention mechanism. First, in the Context-Aware Code Adaptation stage, the codebook primitives act as queries to attend to the input image features, generating a contextually-adapted codebook. Subsequently, in the Code-based Feature Modulation stage, the image features query this adapted codebook, aggregating relevant primitive information to perform targeted feature restoration. This dynamic process allows DACode to construct highly specific restorative features for each input. Notably, our analysis reveals that DACode learns to activate distinct code combinations in response to both varying degradation types (e.g., haze vs. rain) and severities (e.g., light vs. heavy haze), providing direct evidence for its fine-grained modeling capability and interpretability. Extensive experiments show that DACode significantly outperforms state-of-the-art methods across all-in-one restoration benchmarks.

Primary Area: applications to computer vision, audio, language, and other modalities

Submission Number: 6179