Exploring Multi-scale Deep Encoder-Decoder and PatchGAN for Perceptual Ultrasound Image Super-Resolution

Jianyong Liu, Heng Liu, Xiaoyu Zheng, Jungong Han

Abstract: Ultrasound visual imaging is currently one of three mainstream image diagnosis technologies in the medical industry, but due to the limitations of sensors, transmission media and ultrasound characteristics, the quality of ultrasound imaging may be poor, especially its low spatial resolution. We incorporate, in this paper, a new multi-scale deep encoder-decoder structure into a PatchGAN (patch generative adversarial network) based framework for fast perceptual ultrasound image super-resolution (SR). Specifically, the entire algorithm is carried out in two stages: ultrasound SR image generation and image refinement. In the first stage, a multi-scale deep encoder-decoder generator is employed to accurately super-resolve the LR ultrasound images. In the second stage, we advocate the confrontational characteristics of the discriminator to impel the generator such that more realistic high-resolution (HR) ultrasound images can be produced. The assessments in terms of PSNR/IFC/SSIM, inference efficiency and visual effects demonstrate its effectiveness and superiority, when compared to the most state-of-the-art methods.