Optimized deep learning model for comprehensive medical image analysis across multiple modalities

Saif Ur Rehman Khan, Sohaib Asif, Ming Zhao, Wei Zou, Yangfan Li, Xiangmin Li

Abstract: This study presents a novel amalgamated model for the diagnosis of multiple medical conditions using various imaging modalities, including Chest X-ray, MRI, and endoscopic images. Each imaging modality has unique protocols and feature characteristics, presenting significant complexity and challenges in diagnosis. To address these issues, we propose a truncated lightweight model that effectively fuses diverse image features while reducing computational requirements. The model utilizes deep learning (DL) techniques and multi-scale feature learning to enhance diagnostic capabilities across different medical images. Specifically, it employs an efficient MobileNet architecture to simultaneously diagnose multiple diseases. Key innovations include model truncation, a modified naive inception block for multi-scale feature extraction, and metaheuristic optimization methods. By optimizing the architecture with techniques such as Chain Foraging and Cyclone Aging, the model achieves robustness and scalability, improving generalization across varying image resolutions. Additionally, an integrated ConvLSTM unit before the Softmax layer enhances feature extraction across spatial and temporal dimensions, addressing challenges related to differing feature sizes and scales in multi-disease diagnosis. We conducted comprehensive testing on publicly available multi-class medical image datasets, including brain MRI, chest X-ray, and gastro endoscopic images, which demonstrates that the proposed model outperforms existing methods, achieving an overall accuracy of 97.37 %. To further support clinical decision-making, we utilized visualization techniques such as GradCAM, and Feature Map analysis to enhance the interpretability of the model predictions. Overall, the proposed model not only showcases exceptional performance in classifying various medical images but also presents an optimal balance between accuracy and computational efficiency, establishing its potential as a practical solution for accurate multi-modal medical image analysis.