MultiColor: Image Colorization by Learning from Multiple Color Spaces
Abstract: Deep networks have shown impressive performance in the image restoration tasks, such as image colorization. However, we find that previous approaches rely on the digital representation from single color model with a specific mapping function, a.k.a., color space, during the colorization pipeline. In this paper, we first investigate the modeling of different color spaces, and find each of them exhibiting distinctive characteristics with unique distribution of colors. The complementarity among multiple color spaces leads to benefits for the image colorization task.
We present MultiColor, a new learning-based approach to automatically colorize grayscale images that combines clues from multiple color spaces. Specifically, we employ a set of dedicated colorization modules for individual color space. Within each module, a transformer decoder is first employed to refine color query embeddings and then a color mapper produces color channel prediction using the embeddings and semantic features. With these predicted color channels representing various color spaces, a complementary network is designed to exploit the complementarity and generate pleasing and reasonable colorized images. We conduct extensive experiments on real-world datasets, and the results demonstrate superior performance over the state-of-the-arts. The code will be available.
Relevance To Conference: Within the selected area of Generative Multimedia and Multimedia Applications, this paper proposed a novel approach MultiColor, which automatically colorizes grayscale images by combining clues from multiple color spaces. The main components, such as the Transformer decoder, color mapper and Color Space Complementary Network (CSCNet) represent a notable advancement in the multimedia process. Empirically validated, our contributions bridge deep learning techniques with multimedia processing.
Primary Subject Area: [Generation] Generative Multimedia
Secondary Subject Area: [Experience] Multimedia Applications
Submission Number: 1667
Loading