Bilevel modeling investigated generative adversarial framework for image restoration

Zhiying Jiang, Zengxi Zhang, Yiyao Yu, Risheng Liu

Abstract: Generative adversarial network (GAN), which is developed on the alternative improvement of the generator and discriminator to obtain the optimal network, has become a problem of great interest for computer vision. In image restoration, domain knowledge and heuristic image prior are significant for low-level tasks, while the effectiveness of GAN relies on the empirical designing of networks and the massive training data only, ignoring the intrinsic principle owned by the task. Therefore, the restored results obtained by GAN usually suffer from information loss, and the structure may not be preserved well. To alleviate this issue, we develop a bilevel modeling investigated generative adversarial framework to incorporate task-specific domain knowledge with discriminative prior for adaptive image restoration. In our method, bilevel optimization is investigated to establish our basic iterative mechanism. The lower layer in the paradigm introduces domain knowledge to reveal the task essences in a physics model. At the same time, the upper layer exploits the data-driven discriminative prior to guiding the estimated one. Within the interaction between the lower and upper layers, the upper network realizes the correction of the intermediate results obtained by the lower heuristic prior guidance. We apply it to two low-level image restoration tasks, image deconvolution and single image deraining. Extensive experiments prove that the proposed method performs favorably against the state-of-the-art methods quantitatively and qualitatively.