Detailed and Controllable Old Photo Restoration with Diffusion Priors

Xibei Liu, Han Wang, Yiwen Wang, Yuhong Zhang, Jiayi Song, Rong Xie, Li Song

Abstract: Restoring old photos that contain numerous unknown and complex defects is a challenging and ill-posed problem. Traditional methods often struggle to address both structured and unstructured defects in real old photos, frequently leading to over-smoothed and uncompleted results. In this paper, we exploit powerful diffusion priors to construct a novel solution for the restoration of old photos. Our framework begins with a coarse stage model aimed at eliminating unstructured defects, followed by a conditional diffusion model that further refines the content and enriches details. Specifically, we introduce an edge control module, designed to encode the restored edge map into the denoising network. This integration effectively guides the restoration process, allowing the edge conditional diffusion to achieve more precise and controllable results. Additionally, we incorporate a feature fusion VAE to ensure the fidelity of the final outputs. Through extensive qualitative, quantitative, and ablative experiments, we demonstrate the innovation and effectiveness of the proposed method, which offers a complete, detailed, and personalized restoration of old photos.