Harmony in Diversity: Improving All-in-One Image Restoration via Multi-Task Collaboration

Gang Wu, Junjun Jiang, Kui Jiang, Xianming Liu

Abstract:

Deep learning-based all-in-one image restoration methods have garnered significant attention in recent years due to capable of addressing multiple degradation tasks. These methods focus on extracting task-oriented information to guide the unified model and have achieved promising results through elaborate architecture design. They commonly adopt a simple mix training paradigm, and the proper optimization strategy for all-in-one tasks has been scarcely investigated. This oversight neglects the intricate relationships and potential conflicts among various restoration tasks, consequently leading to inconsistent optimization rhythms.
In this paper, we extend and redefine the conventional all-in-one image restoration task as a multi-task learning problem and propose a straightforward yet effective active-reweighting strategy, dubbed $\textbf{Art}$, to harmonize the optimization of multiple degradation tasks. Art is a plug-and-play optimization strategy designed to mitigate hidden conflicts among multi-task optimization processes.
Through extensive experiments on a diverse range of all-in-one image restoration settings, Art has been demonstrated to substantially enhance the performance of existing methods. When incorporated into the AirNet and TransWeather models, it achieves average improvements of $\textbf{1.16}$ dB and $\textbf{1.21}$ dB on PSNR, respectively. We hope this work will provide a principled framework for collaborating multiple tasks in all-in-one image restoration and pave the way for more efficient and effective restoration models, ultimately advancing the state-of-the-art in this critical research domain. Code and pre-trained models are available at our project page https://github.com/Aitical/Art.

Primary Subject Area: [Experience] Multimedia Applications

Secondary Subject Area: [Experience] Interactions and Quality of Experience

Relevance To Conference: Image restoration is a fundamental research topic in multimedia field and this work makes significant strides in advancing the field of all-in-one image restoration by tackling the crucial problem of multi-task imbalance in deep learning-based methods. Through an in-depth analysis of state-of-the-art approaches, we uncover disparities in task convergence rates and identify a "conflict" phenomenon among tasks, shedding light on the challenges faced by existing unified models. The proposed ReIR framework offers a straightforward yet effective approach to balance multiple tasks in all-in-one image restoration. By explicitly formulating the restoration process as a multi-task learning problem and introducing task-specific rebalancing coefficients, a smooth convergence rate, and a novel rectification factor, ReIR effectively equilibrates optimization dynamics and mitigates inter-task conflicts. Extensive experiments on diverse datasets demonstrate the superiority of ReIR over existing methods, showcasing its ability to consistently improve restoration performance.

Submission Number: 181