Fast and accurate view classification of echocardiograms using deep learning

Ali Madani, Ramy Arnaout, Mohammad R. K. Mofrad, Rima Arnaout

Abstract: Echocardiography is essential to cardiology. However, the need for human interpretation has limited echocardiography’s full potential for precision medicine. Deep learning is an emerging tool for analyzing images but has not yet been widely applied to echocardiograms, partly due to their complex multi-view format. The essential first step toward comprehensive computer-assisted echocardiographic interpretation is determining whether computers can learn to recognize these views. We trained a convolutional neural network to simultaneously classify 15 standard views (12 video, 3 still), based on labeled still images and videos from 267 transthoracic echocardiograms that captured a range of real-world clinical variation. Our model classified among 12 video views with 97.8% overall test accuracy without overfitting. Even on single low-resolution images, accuracy among 15 views was 91.7% vs. 70.2–84.0% for board-certified echocardiographers. Data visualization experiments showed that the model recognizes similarities among related views and classifies using clinically relevant image features. Our results provide a foundation for artificial intelligence-assisted echocardiographic interpretation. A computer model can accurately classify what heart anatomy is depicted on an ultrasound scan. Rima Arnaout from the University of California, San Francisco, and colleagues used a machine-learning technique to teach a computer to recognize different types of video and still images produced by echocardiogram tests. After training the computer with real scans gathered from patients with a range of heart conditions, the researchers showed that their model could correctly classify what heart anatomy was shown in videos with 98% accuracy. The accuracy for still images fell to 92%, but that was still above the 80% average achieved by four professional echocardiographers tested on the same dataset. Now that the researchers have achieved successful view classification, the next challenge, they say, will be comprehensive computer-assisted interpretation of echocardiogram results.