Patch-based deep learning models for breast mammographic mass classification

Wentao Xie, Qian Liu, Yongye Su, Yi Yan, Shujun Huang, Qin Kuang, Pingzhao Hu

Abstract: Background: Breast cancer is one of the greatest health threats to women worldwide. Mammography is an effective and inexpensive tool for breast cancer early detection. Mammography-based breast cancer screening requires a lot of manpower from professional experts. Thus, computer-aided diagnosis tools, especially accurate classifiers which can distinguish the breast masses from the background tissues, are needed. However, since the sample size of publicly available mammography data sets is relatively small, the performance of the published breast mass identification models was not great, and the models were not well-embraced by clinical practice due to their low interpretability. Methods: In this work, using two independent and well-known mammography data sets, the CBIS-DDSM and the INbreast, we proposed a novel patch generation method for data augmentation and negative case generation. We implemented two successful deep learning models, the ResNet and the ViT, to classify the generated mass and non-mass patches. We also proposed to apply the patch-level model to the full-view mammogram screening in a sliding window manner and visualize/interpret the prediction results using a heatmap so that the clinic practice could potentially benefit from the well-trained model. Result: For the CBIS-DDSM dataset, we compared the performance of the ResNet and the ViT with and without data augmentation. The F1 score is 0.91, 0.86, 0.85, and 0.70, respectively. We also evaluated our models using other metrices such as accuracy, precision, recall, and ROC curve. The results show that the ResNet model outperforms the ViT model. And the data augmentation improves the overall performance of the models. The similar conclusions are further supported using the independent INbreast data. Furthermore, we also explored to use probability-based heatmaps to visualize the potential mass regions in mammogram images. Conclusion: The study shows that our patch-level data augmentation is effective in improving the classification performance of the deep learning models. The comparable performance on the CBIS-DDSM data and the independent INbreast data demonstrates the generalizability of our methods. The proposed heatmap visualization tool increases the interpretability of our results and could be a potential approach for clinic utilization.