DualU-Net Mixed with Convolution and Transformers

Lu Shen, Changjian Wang, Yingwen Chen, Dandan Li, Yunbin Xiao

Abstract: As the basis of quantitative analysis of lung diseases, lung segmentation has a great impact on the accuracy of subsequent work. Most of the current lung segmentation models focus on CNN. The inherent limitations of convolution make such networks unable to utilize the global context information of the lung, which is very important for lung segmentation. To solve this problem, this paper proposes a DualU-Net with convolutions and Transformers hybrid, which complements the global features of the lung through Transformers. DualU-Net consists of Local Down-sampling, Global Down-sampling, Invertible Feature Fusion Module, Bilinear-based Up-sampling Mechanism. Local Down-sampling extracts the local detail features of the lung through the convolution-based residual structure, and Global Down-sampling extracts the global long-range dependence through Transformers. The two down-samplings extract the local and global features of the lung simultaneously and independently in a parallel manner. Then, the local and global features are fused by the Invertible Feature Fusion Module. Finally, the resolution of the image is gradually restored by the Bilinear-based Up-sampling Mechanism to obtain the final segmentation result. We conduct experiments on two public datasets. The experimental results show that global features can indeed help the network to perform better segmentation, and DualU-Net can have a relatively balanced segmentation effect on various datasets.