Efficient Scopeformer: Scalability and Feature Extraction Richness in Intracranial Hemorrhage Detection Challenge

Yassine Barhoumi, Ghulam Rasool

Keywords: Computed Tomography (CT) scans and slices, Intracranial hemorrhage detection, CNN, ViT

TL;DR: This paper is an extension of a recently proposed vision transformer and convolution based model called Scopeformer. We focus on optimizing the model efficiency in size and performance on the RSNA dataset.

Abstract: The feature map quality generated from the computed tomography (CT) scans constitutes a distinctive aspect for robust model performance in the medical computer vision field. Furthermore, well-engineered features are deterministic to ambiguous cases detection. A novel multi-CNN-based vision transformer model called Scopeformer was developed in our earlier work and used for the Intracranial Hemorrhage Detection challenge to classify various hemorrhage types within the RSNA2019 Brain CT Hemorrhage dataset. The model showed high scalability of the model size with an improved feature generating backbone. However, the model suffered from a large trainable parameter space resulting in a long training time. We adopt in this paper the Scopeformer model and aim to reduce the parameter size and enhance the global feature map richness. Effective feature projection methods were used to reduce the redundancy of the feature space. Furthermore, we used small vision transformer (ViT) versions with four different types of pretrained CNN architectures and introduced three ViT configurations to reduce the self-attention complexity within the transformer layers. Our best model achieved an accuracy of 96.03 % and a weighted logarithmic loss of 0.1088 with an eight times reduction of trainable parameter space. A second model with comparable performance further reduced the parameter space to 17 times our best-performing model.

Registration: I acknowledge that publication of this at MIDL and in the proceedings requires at least one of the authors to register and present the work during the conference.

Authorship: I confirm that I am the author of this work and that it has not been submitted to another publication before.

Paper Type: both

Primary Subject Area: Detection and Diagnosis

Secondary Subject Area: Transfer Learning and Domain Adaptation

Confidentiality And Author Instructions: I read the call for papers and author instructions. I acknowledge that exceeding the page limit and/or altering the latex template can result in desk rejection.