Semantic Masking with Curriculum Learning for Robust HDR Image Reconstruction

Zhangkai Ni, Yang Zhang, Kerui Ren, Wenhan Yang, Hanli Wang, Sam Kwong

Abstract: High Dynamic Range (HDR) image reconstruction aims to reconstruct images with a larger dynamic range from multiple Low Dynamic Range (LDR) images with different exposures. Existing methods face two challenges: visual artifacts in the restored images and insufficient model generalization capabilities. This paper addresses these issues by leveraging the inherent potential of Masked Image Modeling (MIM). We propose a Segment Anything Model (SAM)-guided masking strategy, leveraging large-model priors to direct the HDR reconstruction network via curriculum learning. This strategy gradually increases the difficulty from simple to complex tasks, guiding the model to effectively learn semantic priors that prevent the model from overfitting to the training data. Our approach starts by training the model without any masks, then progressively increasing the masking ratio of input features guided by semantic segmentation maps, which compels the model to learn semantic information during restoration. Subsequently, we make an adaption to reduce the masking ratio to minimize the input discrepancy between the training and testing stage. Besides, we manipulate the computation of the loss based on the perceptual quality of reconstructed images, where challenging areas (e.g., over-/under-exposed regions) are given more weight to improve image restoration results. Furthermore, through specialized module design, our method can be fine-tuned to any number of inputs, achieving comparable performance to models trained from scratch with only 5.5% of parameter adjustments. Extensive qualitative and quantitative experiments demonstrate that our approach surpasses state-of-the-art methods in both effectiveness and generalization. Our code is available at: https://github.com/eezkni/SMHDR

External IDs:dblp:journals/ijcv/NiZRYWK25