Vispi: Automatic Visual Perception and Interpretation of Chest X-rays

Xin Li, Rui Cao, Dongxiao Zhu

Keywords: Medical Image Report Generation, Disease Classification and Localization, Visual Perception, Attention, Deep Learning

Track: short paper

Paper Type: methodological development

Abstract: Medical imaging contains the essential information for rendering diagnostic and treatment decisions. Inspecting (visual perception) and interpreting image to generate a report are tedious clinical routines for a radiologist where automation is expected to greatly reduce the workload. Despite rapid development of natural image captioning, computer-aided medical image visual perception and interpretation remain a challenging task, largely due to the lack of high-quality annotated image-report pairs and tailor-made generative models for sufficient extraction and exploitation of localized semantic features, particularly those associated with abnormalities. To tackle these challenges, we present Vispi, an automatic medical image interpretation system, which first annotates an image via classifying and localizing common thoracic diseases with visual support and then followed by report generation from an attentive LSTM model. Analyzing an open IU X-ray dataset, we demonstrate a superior performance of Vispi in disease classification, localization and report generation using automatic performance evaluation metrics ROUGE and CIDEr.

TL;DR: We present an automatic medical image interpretation approach for image annotation and report generation.

Presentation Upload Agreement: I agree that my presentation material (videos and slides) will be made public.