A collaborative computer aided diagnosis (C-CAD) system with eye-tracking, sparse attentional model, and deep learning

Naji Khosravan, Haydar Celik, Baris Turkbey, Elizabeth C. Jones, Bradford J. Wood, Ulas Bagci

Abstract: Highlights•We introduce a paradigm shift CAD system, called collaborative CAD (C-CAD), that unifies both CAD and eye-tracking to improve radiographical image analysis.•We develop an eye-tracking interface that provides a real radiology reading room experience and perform an attention based clustering and sparsification of dense eye-tracking data.•We propose a new attention based data sparsification method applied to gaze patterns of radiologists which allows local and global analysis of visual search patterns based on visual attention concepts.•By utilizing gaze patterns, we build a new CAD system based on a 3D deep learning algorithm in a newly designed multi-task learning platform where both segmentation and diagnosis tasks are jointly modeled.•We show the efficacy of the system in lung cancer screening experiments with low dose CT, and then we extend the proposed eye-tracking based CAD system into a multi-modality image analysis framework where users can utilize multiple screens as in prostate screening with multi-parametric MRI.