S$^2$-CSNet: Scale-Aware Scalable Sampling Network for Image Compressive Sensing

Chen Hui, Haiqi Zhu, Shuya Yan, Shaohui Liu, Feng Jiang, Debin Zhao

Abstract: Deep network-based image Compressive Sensing (CS) has attracted much attention in recent years. However, there still exist the following two issues: 1) Existing methods typically use fixed-scale sampling, which leads to limited insights into the image content. 2) Most pre-trained models can only handle fixed sampling rates and fixed block scales, which restricts the scalability of the model. In this paper, we propose a novel scale-aware scalable CS network (dubbed S$^2$-CSNet), which achieves scale-aware adaptive sampling, fine granular scalability and high-quality reconstruction with one single model. Specifically, to enhance the scalability of the model, a structural sampling matrix with a predefined order is first designed, which is a universal sampling matrix that can sample multi-scale image blocks with arbitrary sampling rates. Then, based on the universal sampling matrix, a distortion-guided scale-aware scheme is presented to achieve scale-variable adaptive sampling, which predicts the reconstruction distortion at different sampling scales from the measurements and select the optimal division scale for sampling. Furthermore, a multi-scale hierarchical sub-network under a well-defined compact framework is put forward to reconstruct the image. In the multi-scale feature domain of the sub-network, a dual spatial attention is developed to explore the local and global affinities between dense feature representations for deep fusion. Extensive experiments manifest that the proposed S$^2$-CSNet outperforms existing state-of-the-art CS methods.

Primary Subject Area: [Experience] Multimedia Applications

Secondary Subject Area: [Content] Media Interpretation, [Content] Vision and Language

Relevance To Conference:

To our knowledge, this is the first deep network-based image compressive sensing (CS) that provides scale-aware adaptive sampling, fine granular scalability and high-quality reconstruction with one single model. CS is an effective signal processing technique, which can achieve fast imaging by sampling far fewer measurements than that required by Nyquist sampling. On the basis of conventional CS, this paper explores scale-aware sampling and provides efficient data compression, low-cost sampling, high-quality reconstruction, offering new insights for the acquisition, processing, sparse representation and reconstruction of multimedia such as images, audio, and video.
Block-based CS (BCS) is a classical method and has been widely adopted by most research efforts. In BCS, images are divided into non-overlapping blocks of fixed scale. Such coarse blocking leads to the mixing of information with different content features, which will degrade the performance of CS. To solve this issue, we design a scale-aware sampling method, which evaluates the reconstruction distortion at different scales and select the optimal sampling scale. Our organic combination of the classic BCS and deep networks is interesting and insightful for the signal processing community.

Submission Number: 3384