BUS-net: a bimodal ultrasound network for breast cancer diagnosis

Xun Gong, Xu Zhao, Lin Fan, Tianrui Li, Ying Guo, Jun Luo

Abstract: Ultrasound (US) is a fast and non-invasive imaging approach, which is recommended by current practice guidelines as early breast cancer screening. B-mode and contrast-enhanced ultrasound(CEUS) are two US modalities proven to detect abnormal tissues. However, no one has been able to make full use of these two modal ultrasound data characteristically to complete the classification problem. This paper proposed a bimodal ultrasound network (BUS-Net) capable of simultaneously dealing with the B-mode US and CEUS video. In the CEUS branch, We use seven CEUS pathological characteristics as multiple labels instead of the traditional two labels (benign and malignant) to extract the pathological semantic representative features. The model can be more general and robust by transforming the binary learning task into a multi-class learning task. In the B-mode US branch, we use a group of shape descriptors to identify hard samples with abnormal morphology. A shape constraint loss term is proposed to impose the shape constraints in the training phase and enhance its distinguish ability for hard samples. Finally, the two modal ultrasound data features are fused to realize the classification of benign and malignant tumors. Our experiments show that the classification accuracy is significantly improved using our bimodal strategy. Compared with existing breast ultrasound classification methods, our method increased by an average of 3 percentage points in each evaluation index, and the TNR and AUC index both exceeded 92%. This also demonstrates that our approach can more accurately classify ultrasound images with more complex imaging. In general, the bimodal ultrasound network proposed in this paper, which integrates bimodal data features, further improves the classification ability of the model. The multi-label learning task of the CEUS branch enhances the robustness of the model. The shape constraint loss term of the B-mode US branch improves its ability to distinguish between hard samples. The algorithm in this paper has good clinical guidance value.