Degradation-aware Unfolding Knowledge-assist Transformer for Spectral Compressive Imaging

Zongliang Wu, Ruiying Lu, Xin Yuan

Keywords: Spectral image reconstruction, deep unfolding, vector quantization

TL;DR: A physics-driven and explicable algorithm for snapshot compressive imaging reconstruction.

Abstract:

Snapshot compressive spectral imaging offers the capability to effectively capture three-dimensional spatial-spectral images through a single-shot two-dimensional measurement, showcasing its significant potential for spectral data acquisition. However, the challenge of accurately reconstructing 3D spectral signals from 2D measurements persists, particularly when it comes to preserving fine-grained details like textures, which is caused by the lack of high-fidelity clean image information in the input compressed measurements. In this paper, we introduce a two-phase training strategy embedding high-quality knowledge prior in a deep unfolding framework, aiming at reconstructing high-fidelity spectral signals. Our experimental results on synthetic benchmarks and real-world datasets demonstrate the notably enhanced accuracy of our proposed method, both in spatial and spectral dimensions. Code and pre-trained models will be released.

Primary Area: representation learning for computer vision, audio, language, and other modalities

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 3504