RefScale: Multi-temporal Assisted Image Rescaling in Repetitive Observation Scenarios

Zhen Zhang, Jing Xiao, Liang Liao, Mi Wang

Abstract:

With the continuous development of imaging technology and the gradual expansion of the amount of image data, how to achieve high compression efficiency of high-resolution images is a challenge problem for storage and transmission.
Image rescaling aims to reduce the original data amount through downscaling to facilitate data transmission and storage before encoding, and reconstruct the quality through upscaling after decoding, which is a key technology to assist in high-ratio image compression.
However, existing rescaling approaches are more focused on reconstruction quality rather than image compressibility.
In repetitive observation scenarios, multi-temporal images brought by periodic observations provide an opportunity to alleviate the conflict between reconstruction quality and compressibility, that is, the historical images as reference indicates what information can be dropped at downscaling to reduce the information content in downscaled image and provides the dropped information to improve the image restoration quality at upscaling.
Based on this consideration, we propose a novel multi-temporal assisted reference-based image rescaling framework (RefScale).
Specifically, a referencing network is proposed to calculate the similarity map to provide the referencing condition, which is then injected into the conditional invertible neural network to guide the information drop at the downscaling stage and information fusion at the upscaling stage. Additionally, a low-resolution guidance loss is proposed to further constrain the data amount of the downscaled LR image. Experiments conducted on both satellite imaging and autonomous driving show the superior performance of our approach over the state-of-the-art methods.

Primary Subject Area: [Experience] Multimedia Applications

Relevance To Conference:

This proposed work on a novel multi-temporal assisted reference-based image rescaling framework makes several important contributions to the field of multimedia/multimodal:

1) Historical reference guided downscaling for less information.
utilizing historical reference image to guide the removal of similar structural textures during the downscaling process, which allows for less informative and structure-preserving downscaling.

2) Multi-temporal fusion assisted upscaling for high fidelity.
fusing multimodal information (downscaled and reference images) enhances the upscaling performance, leading to better visual quality and content preservation.

3) Application to high-ratio image compression in repetitive observation scenarios.
introducing the proposed rescaling framework into image compression in remote sensing and autonomous driving, exploiting multi-temporal image redundancy for high-magnification rescaling-based image compression.

Overall, These advancements in this work can have significant impacts on various collaborative compression and transmission of multi-source and multi-modal data.

Submission Number: 3104