Weakly-Supervised Ultrasound Video Segmentation with Minimal Annotations

Ruiheng Chang, Dong Wang, Haiyan Guo, Jia Ding, Liwei Wang

Abstract: Ultrasound segmentation models provide powerful tools for the diagnosis process of ultrasound examinations. However, developing such models for ultrasound videos requires densely annotated segmentation masks of all frames in a dataset, which is unpractical and unaffordable. Therefore, we propose a weakly-supervised learning (WSL) approach to accomplish the goal of video-based ultrasound segmentation. By only annotating the location of the start and end frames of the lesions, we obtain frame-level binary labels for WSL. We design Video Co-Attention Network to learn the correspondence between frames, where CAM and co-CAM will be obtained to perform lesion localization. Moreover, we find that the essential factor to the success of extracting video-level information is applying our proposed consistency regularization between CAM and co-CAM. Our method achieves an mIoU score of 45.43% in the breast ultrasound dataset, which significantly outperforms the baseline methods. The codes of our models will be released.