Multiframe Super-Resolution With Dual Pyramid Multiattention Network for Droplet Measurement

Qiangqiang Liu, Hua Yang, JianKui Chen, Zhouping Yin

Abstract: Accurate visual measurement of micrometer-scale flying droplets in inkjet printing remains a challenge due to low image resolution caused by severe image conditions. Multiframe super-resolution (MFSR) has the potential to break through the measurement bottleneck. However, most existing MFSR methods are not satisfactory in multiframe information utilization, especially for fast-motion scenes, and they often suffer from detail loss. In this study, focusing on multiframe information utilization and deep feature extraction, we propose a dual pyramid multiattention network (DPMAN). First, a dual pyramid deformable alignment (DPDA) module is proposed to deal with diverse motion, which extracts explicit offsets to enhance deformable alignment and perform the coarse-to-fine alignment. Then, a gated attention fusion (GAF) module is devised to adaptively aggregate the aligned features to emphasize favorable features. Finally, a residual self-attention reconstruction (RSAR) module based on the multistage aggregation self-attention architecture is proposed to extract finer deep features for detail restoration. Experimental results on three benchmark datasets demonstrate that DPMAN achieves the state-of-the-art performance. DPMAN is applied to droplet image reconstruction and improves the measurement accuracy from 3.34% to 2.52%.