Optimization-Inspired Deep Network for Image Restoration from Partial Random Samples

Yanchen Dong, Rui Zhao, Ruiqin Xiong, Shuyuan Zhu, Xiaopeng Fan, Tiejun Huang

Abstract: Image Restoration from Partial Random Samples (RRS) has been studied in many image restoration works. There are also some attempts to use convolutional neural networks (CNNs) to handle it. However, most existing neural network-based methods perform poorly in generalization and we need to train a specific model for each degradation situation. Besides, the sampling mask which represents the positions of the sampled pixels is not used effectively in these methods. To address the problems, we propose an optimization-inspired network called RRSNet based on our derivation of the iterative optimization formulas for RRS. In our method, we design a CNN with two encoders and one decoder for training, setting up a flexible and effective prior. To make the most of the sampling information, we concatenate the degraded image with the mask and input them into one encoder for better generalization. Then we split the pixels into two groups according to the mask and extract their features as the input of another encoder. Experiments demonstrate that our RRSNet with the mask input can handle various sampling ratios using only one trained model and achieve the best restoration performance among all comparison methods.